Simpro Webhook triggers - seem to get stuck at certain times

[jonohowell]

I noticed over the past couple of days our webhook subscriptions trigger but sit in a "Not Sent" state - after doing nothing it seems to eventually go through. But today I sitting watching it sit for over 2 hours and still nothing. Even test messages say the successful to the end point but also sit in a not sent state. Really keen to understand why this occuring and if others have noticed the same. I've confirmed all our end points work fine. I've even setup new subscriptions to hit a public test probe and triggered event to send but just radio silence. Really appreciate any insight or help on this.

[donnacha]

We are also having an issue with our webhooks not being received. Have also tested with a public input and getting the same issue.

Thanks

[davetar]

Delay between webhook trigger and it being sent is currently sitting at 207 mins. The delay has been growing and growing for the last 9 hours or so. We often get delays of 5 to 15 mins and sometimes an hour but the system eventually catches up - usually before close of business. But not today - the delay just keeps growing.

[alansheppard]

This is also not working for us - the delay is really annoying!

[davetar]

Delay got down to 30 seconds 15 minutes ago - but immediately began to climb again - back to 11 mins already.

I have asked about this before - several times over the last 12 months or so - but have never had a satisfactory answer -I was led to believe it was a matter of volume - sometimes there were just too many webhooks so all went in a queue to be processed - unsure if this is the case at the moment - seems unlikely as overnight (Australian time) the delay did not seriously start to diminish until about 4am.

[jonohowell]

Okay - this is clearly a problem that has been going on for a while. I tried reaching out to support but got told this is the only forum to get assistance for anything relating to the API. I may have to looking at detecting changes via controlled polling API calls. SIMPRO - please respond! and provide a pathway to resolution.

[davetar]

I think we would all like some transparency. Also - might be nice to be told about issues as they arise, rather than us having to inform simPRO. Current delay 21.75 mins.

[jonohowell]

FYI - I've raised this with support and escalated to support management to find a path to the developer team to get some visibility and action to this. I've included this forum page as proof this impacting multiple tenancies on their platform. Ill post any updates I get back on here. Or better yet someone from the DEV team just provides some answers.

[davetar]

There are lots of other threads on this issue

here is one viewtopic.php?f=2&t=2459

This one viewtopic.php?f=2&t=2386

This one viewtopic.php?f=2&t=2334

Others you can find too

It's not a new problems. Usually put down to webhook volume.

Cheers

[Sindre]

Hi,
Thank you for reporting on the delayed queues.

We are aware of the extended queues and delays it causes and have been following them closely to verify the kind of traffic that has been processed.
Technically the system has been working as it should but the amount of traffic has been overwhelming causing long delays for all clients.

There continues to be a large amount of webhooks triggered in the system today and we are monitoring what is being sent.

Regards