Replit does not allocate enough resources when installing python packages!

So for a while, I’ve been trying to install a python package called distilbert-punctuator for a while at this point, but it keeps giving me this error whenever I try!

--> python3 -m poetry add distilbert-punctuator
Using version ^0.3.0 for distilbert-punctuator

Updating dependencies
Resolving dependencies...

Writing lock file

Package operations: 14 installs, 4 updates, 2 removals

  • Removing importlib-metadata (5.0.0)
  • Removing zipp (3.8.1)
  • Updating filelock (3.6.0 -> 3.8.0)
  • Updating packaging (20.9 -> 21.3)
  • Installing pyyaml (6.0)
  • Updating click (8.1.3 -> 7.1.2)
  • Installing huggingface-hub (0.10.1)
  • Installing nvidia-cublas-cu11 (11.10.3.66)
  • Installing nvidia-cuda-runtime-cu11 (11.7.99)
  • Installing nvidia-cuda-nvrtc-cu11 (11.7.99)
  • Installing nvidia-cudnn-cu11 (8.5.0.96)
  • Installing regex (2022.10.31)
  • Installing tokenizers (0.13.1)

  EnvCommandError

  Command ['/home/runner/VS-3/venv/bin/pip', 'install', '--no-deps', 'https://files.pythonhosted.org/packages/ce/41/fdeb62b5437996e841d83d7d2714ca75b886547ee8017ee2fe6ea409d983/nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl#sha256=d32e4d75f94ddfb93ea0a5dda08389bcc65d8916a25cb9f37ac89edaeed3bded'] errored with the following return code -9, and output: 
  Looking in indexes: https://package-proxy.replit.com/pypi/simple/
  Collecting nvidia-cublas-cu11==11.10.3.66
    Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)
  

  at venv/lib/python3.8/site-packages/poetry/utils/env.py:1195 in _run
      1191│                 output = subprocess.check_output(
      1192│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1193│                 )
      1194│         except CalledProcessError as e:
    → 1195│             raise EnvCommandError(e, input=input_)
      1196│ 
      1197│         return decode(output)
      1198│ 
      1199│     def execute(self, bin, *args, **kwargs):


  EnvCommandError

  Command ['/home/runner/VS-3/venv/bin/pip', 'install', '--no-deps', 'https://files.pythonhosted.org/packages/dc/30/66d4347d6e864334da5bb1c7571305e501dcb11b9155971421bb7bb5315f/nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl#sha256=402f40adfc6f418f9dae9ab402e773cfed9beae52333f6d86ae3107a1b9527e7'] errored with the following return code -9, and output: 
  Looking in indexes: https://package-proxy.replit.com/pypi/simple/
  Collecting nvidia-cudnn-cu11==8.5.0.96
    Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)
  

  at venv/lib/python3.8/site-packages/poetry/utils/env.py:1195 in _run
      1191│                 output = subprocess.check_output(
      1192│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1193│                 )
      1194│         except CalledProcessError as e:
    → 1195│             raise EnvCommandError(e, input=input_)
      1196│ 
      1197│         return decode(output)
      1198│ 
      1199│     def execute(self, bin, *args, **kwargs):


Failed to add packages, reverting the pyproject.toml file to its original content.
exit status 1

After this, I tried to use pip manually and I noticed that some prosses KILLED pip before it could finish.

...
Collecting typer==0.3.2
  Using cached https://package-proxy.replit.com/pypi/packages/90/34/d138832f6945432c638f32137e6c79a3b682f06a63c488dcfaca6b166c64/typer-0.3.2-py3-none-any.whl (21 kB)
Collecting pydantic==1.8.2
  Using cached https://package-proxy.replit.com/pypi/packages/34/f8/438aa7b258607ea875ca71b9f549748e75eca0f4f42a4447112c7074cca3/pydantic-1.8.2-cp38-cp38-manylinux2014_x86_64.whl (13.7 MB)
Collecting torch>=1.7.1
  Downloading https://package-proxy.replit.com/pypi/packages/06/3d/19b2090f3f01751e816d409edcc808c82c2973348713186ac153e735d49f/torch-1.13.0-cp38-cp38-manylinux1_x86_64.whl (890.2 MB)
     |████████████████████████████████| 890.2 MB 1.1 MB/s eta 0:00:01Killed

Because of this, I re-did this and monitored the resources and I noticed the RAM was being maxed out so I think that was the problem.

2 Likes

Hi @ErrorbotTHE2nd I think you should log this as a call with support who might be able to help you here.

To do this please click on the ? help icon bottom left of any Repl and select Contact Support. It might be recommended to record a video (from the support window) to share with the team so they can see the RAM maxing out.

1 Like

you could close all python files then run in shell:

kill -1 `pidof pylsp`

to stop your language server and free RAM

1 Like

I tried that, but it still crashes.

ALSO: I learned that this happens mainly because the package nvidia_cudnn_cu11 (I think) is too big.