502 Bad Gateway status on Deployment

Question: Hi all, I currently have a Flask app Repl as a Deployment but I keep getting a 502 Bad Gateway response on one of my paths.

The app’s function is to download a file from Sharepoint, split it up into smaller files, and upload them back to Sharepoint. It is worth noting that this was working correctly with the ‘legacy hosting’ (and locally too), but since we now need to use Deployments, it has stopped working.

The flow runs for anywhere between 15-30 seconds and then returns the 502 status.

Screenshots, links, or other helpful context: Assume the following code, where process_file is imported from another file (this is where the download, splitting, and upload happens):

app = Flask(__name__)

@app.route('/', methods=['GET', 'POST'])
def index():
  return "<p>Alive</p>"

@app.route('/process', methods=['POST'])
def process():
  data = request.json
  return process_file(data)

def run():
  print("Starting app on port 8080...")
  from waitress import serve
  serve(app, host="0.0.0.0", port=8080)

def keep_alive():
  t = Thread(target=run)
  t.start()

When I send a GET request to /, I get <p>Alive</p>, as I should. However, when I now send a POST request to /process with given JSON data, the requests runs for a while before returning the 502 status, as mentioned.

At the top of the process_file function, I have a print statement as follows:

def process_file(data):
  print("Beginning processing a file...")
  # ... other code ...

However, this is not printed to the Deployment logs for some reason (the print statement in the run function does show up in the console).

The deployment is set to auto scale. I have done some reading, and a 502 status from Google Cloud could mean an out-of-memory issue which would make sense since working with files can be intensive, but what’s annoying is that there is literally no error message anywhere saying what goes wrong (not even in the http response), so I am not exactly sure what the cause is. Again, this was working before.

Thanks in advance.

Try using flush=True in the print statement.

1 Like

Thanks. This does make the print statement appear on the Deployment logs. This actually helped me debug it a little, but still no luck with the 502 error.

I tried running it on the development server’s URL and it actually worked, so it is something to do with Deployments. The code seems to be stopping here:

print("getting file segment...", flush=True)
try:
  audio = AudioSegment.from_file(file=f'./tmp/{file_name}.{file_extension}',
                                 format=file_extension)
except Exception as e:
  print(f"error getting audio segment: {e}", flush=True)
  return None

chunk_duration = (len(audio) * 25165824) / file_size

print("starting splitting...", flush=True)
#  remaining code here...

I’m using pydub for audio tasks. The text ‘getting file segment…’ is printed to the console, but then the app just crashes and restarts itself, as shown here (most recent log at the top):
image
I am unsure why the except block is not being run, if an error did indeed occur here.

Thanks.

If it got killed, then it would have had no chance to do anything further, so that makes sense.

Right, but why it’s getting killed is what I am a bit stumped about. Like I mentioned, there is no error message from the response and the whole thing works well in the development panel, so it’s something to do with Deployments but I cannot figure out what.

My best guess is that it ends up using too much CPU/RAM and gets killed by some resource manager.

Yep, this fixed it. I increased the CPU and RAM for the Deployment. When running locally, it was using a maximum of ~1.5GB of RAM while running on VSCode, so I assumed 2GB would be more than enough, clearly not. Not sure what they’re doing but the cloud systems seem to eat up many more resources than local devices.

Anyhow, thanks!

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.