[CRITICAL] WORKER TIMEOUT when API calls

Hi,

I have a fairly simple app that takes a photo from the user, upload it, and then send it to a couple APIs for processing (Azure for OCR recognition and ChatGPT for text summarization). I wouldn’t say that the data sent to those APIs are significant. Maybe 300-500 words.

When the data starts to be a bit bigger (but not necessarily), the app throws a [CRITICAL] WORKER TIMEOUT error and stops working. This happens 100% of the time when data is larger than usual (say >500 words), but it also happens when the data is like 200 words. It doesn’t always come from the same API, it depends. It seems that it comes up when the API takes some time to respond.

There are timeouts in place when calling those 2 APIs already, and I tried to put a timeout to the app itself but it didn’t solve the issue.

Any thoughts as to what I can do to solve this?

Thanks for the help!

Did you tried to incrase the concurrent workers to ease the load?

Also, if possible, batch multiple requests into a single API call to reduce overhead.

If anything you can try to implement a retry mechanism too.

Thanks for the suggestions. How can I increase the numbers of workers? I don’t see the option in the settings.

There are 3 API calls only per image processing, and they are subsequent to each other. I don’t believe this is too many.

Adding to my previous response. I don’t think I create that much load with those API calls. Looking at the utilization, it stays very low…

Oh, didn’t know you were actually using replit. (In this case I have no idea since Replit isn’t straightforward like other services).

But you can try to implement asynchronous programming to ensure that your app doesn’t get blocked while waiting for API responses.

Thanks. I did implement asynchronous programming but I still have this error that appears.

Can you explain me what the workers are, given that it seems that the CPU/memory usage aren’t skyrocketing as it should be, hence I don’t understand why the app is timing out whereas the load isn’t stretched.

Thanks!

Workers refer to the processes or threads that handle incoming requests. Each worker can handle one request at a time, and having multiple workers allows the server to handle multiple requests concurrently.

A worker timeout occurs when a worker takes too long to process a request. This could be due to those infinite long-running tasks or lack of resources.

Even if CPU and memory usage appear normal, timeouts can happens to various reasons, like, network latency (when you have delays in network communication with external APIs).

It can happens with blocking code too, where parts of your code that are not asynchronous may block the event loop (which cause delays).
Even the response time of APIs like, if the external API takes too long to respond, the worker waits, until having a timeout.

What I’d suggest you do to mitigate and identify those problems:

  • Log Timings: Log the start and end times of each API call, so you can identify where the delays occur.
  • Retry Mechanism: Implement retries within your code for API calls.
  • Monitor Network: Try to check the network whenever the server is running
1 Like