Private Cloud gives unexpected responses

[ThomasPollinger]

HI,

We are in the process of migrating large amounts of data into simPRO using the API but have encountered difficulty when sending requests too fast.

I am aware that rate limiting has been implemented - viewtopic.php?f=3&p=5245#p5245
but according to the below thread private clouds like the one we are using shouldn't be affected as it is a .simprocloud.com endpoint.
viewtopic.php?f=2&t=1877

We have been able to consistently encounter this issue when uploading 300 customers in parallel (we can't post multiple as we require information contained within the customer response body even with the display=all parameter).

• throttling the requests results in successful submission of all customers

• submitting the customers in parallel results in 200 OK responses with no body

the endpoints we have been using are
/api/v1.0/companies/2/customers/individuals/?createSite=true
/api/v1.0/companies/2/customers/companies/?createSite=true

• After an average of 70 submissions the responses no longer contain data (content length is zero)

• The failure isn't linked to a specific customers data as fails occur after seemingly any number of successful responses (I have noted as low as 3 and as high as 137) and appears to be linked with how fast the requests are sent to the server.

The exact error we are encountering is that we receive a 200 OK response, but no customer data as can be seen in the below snippet from one of the affected responses.

Code: Select all

headers: {
    'strict-transport-security': 'max-age=31536000; includeSubdomains',
    'content-type': 'text/html; charset=UTF-8',
    'content-length': '0',
    connection: 'close',
    date: 'Sun, 22 Jan 2023 23:15:58 GMT',
    server: 'lighttpd'
  },
  rawHeaders: [
    'Strict-Transport-Security',
    'max-age=31536000; includeSubdomains',
    'Content-type',
    'text/html; charset=UTF-8',
    'Content-Length',
    '0',
    'Connection',
    'close',
    'Date',
    'Sun, 22 Jan 2023 23:15:58 GMT',
    'Server',
    'lighttpd'
  ],
  statusCode: 200,
  statusMessage: 'OK',

As seen above the response has a content-length of 0 which is problematic as we expect a response body. Are there any limits in place for private clouds we should be aware of? If it was rate limiting, we’d expect an 429 response code and not success 200.

Regards,
Thomas

[Sindre]

Hi Thomas,
Thanks for reporting your experiences. As you mention there are not rate limiting implemented for private cloud so this should not affect your migration. We have not registered any general problems but you seem to hit some limitations on the server.

The post multiple function is intended for these kind of instances where you need to import a large amount of records and have been optimised to run efficiently.

I understand that you need the response data, but can you try using post multiple and use the id's returned to get the data after the customers have been created?

Regards