Nix: "error: writing to file: No space left on device" when installing older package via git

I was trying to resolve my issue outlined here: Supabase CLI Error: GLIBC_2.32/GLIBC_2.34 Not Found, error: attribute 'supabase-cli' missing

In order to do that, I was trying to install an earlier version of glibc via Nix:

{ pkgs }:
let
  old_pkgs = import
    (builtins.fetchGit {
      # https://lazamar.co.uk/nix-versions/?package=glibc&version=2.34-210&fullName=glibc-2.34-210&keyName=glibc&revision=34bfa9403e42eece93d1a3740e9d8a02fceafbca&channel=nixpkgs-unstable#instructions
      name = "glibc";
      url = "https://github.com/NixOS/nixpkgs/";
      ref = "refs/heads/nixpkgs-unstable";
      rev = "34bfa9403e42eece93d1a3740e9d8a02fceafbca";
    })
    { };
  myPkg = old_pkgs.glibc;
in
{
  deps = [
    pkgs.nodejs-16_x
    pkgs.deno
    myPkg
  ];
}

However, upon reloading my shell, I get this error:

Detected change in environment, reloading shell...
nix error: building nix env: exit status 1
Output has been trimmed to the last 20 lines
evaluating file '/nix/store/l6smcclpy9f3v562q2ljb9zjdswg3ma1-nixpkgs-22.11-src/pkgs/build-support/buildenv/default.nix'
copied source '/nix/store/l6smcclpy9f3v562q2ljb9zjdswg3ma1-nixpkgs-22.11-src/pkgs/build-support/buildenv/builder.pl' -> '/nix/store/wgf2n4x6vx7fjzhdx2vbij7cblyrzm62-builder.pl'
instantiated 'builder.pl' -> '/nix/store/9yk55y5ywvm4ihnllfa2kcz158bmbfbg-builder.pl.drv'
evaluating file '/nix/store/l6smcclpy9f3v562q2ljb9zjdswg3ma1-nixpkgs-22.11-src/pkgs/build-support/setup-hooks/make-binary-wrapper/default.nix'
copied source '/nix/store/l6smcclpy9f3v562q2ljb9zjdswg3ma1-nixpkgs-22.11-src/pkgs/build-support/setup-hooks/make-binary-wrapper/make-binary-wrapper.sh' -> '/nix/store/3cj6dabzglg1jmcbxddvbdf1a1g7km4q-make-binary-wrapper.sh'
instantiated 'hook' -> '/nix/store/01hzb4f7gq2fzdpy099m3mhkqki2k5yq-hook.drv'
evaluating file '/nix/store/l6smcclpy9f3v562q2ljb9zjdswg3ma1-nixpkgs-22.11-src/pkgs/development/python-modules/pyelftools/default.nix'
instantiated 'source' -> '/nix/store/khlzh2f99sk30akvs3bf70ibvaz6w5a5-source.drv'
instantiated 'python3.10-pyelftools-0.28' -> '/nix/store/1w9kbg5hkhp1iqnmwc76kjkjfbn2mcjq-python3.10-pyelftools-0.28.drv'
instantiated 'python3-3.10.8-env' -> '/nix/store/y4r3r61y6a9f22n172wx7mxh63hcrvnf-python3-3.10.8-env.drv'
instantiated 'auto-patchelf-hook' -> '/nix/store/hfda6a94sda5c2mpb55dzbgsqcnq4vgq-auto-patchelf-hook.drv'
copied source '/nix/store/l6smcclpy9f3v562q2ljb9zjdswg3ma1-nixpkgs-22.11-src/pkgs/development/compilers/rust/setup-hook.sh' -> '/nix/store/skyp8xs2584fkc44ki1cim9ag55wbi68-setup-hook.sh'
instantiated 'rust-1.63.0-x86_64-unknown-linux-gnu.tar.gz' -> '/nix/store/xj526w4zvkagqxqlx1v32nlg38nihn4n-rust-1.63.0-x86_64-unknown-linux-gnu.tar.gz.drv'
instantiated 'rustc-bootstrap-1.63.0' -> '/nix/store/18mj4nq0wa84s0j0j5cs9k4kgnq5rchm-rustc-bootstrap-1.63.0.drv'
instantiated 'cargo-bootstrap-1.63.0' -> '/nix/store/jza7vsmrvwv27jq7xjipim6cl9z4vkp8-cargo-bootstrap-1.63.0.drv'
instantiated 'cargo-build-hook.sh' -> '/nix/store/iq8y5dq1p3ypch46p98gb6dkrq3q0shg-cargo-build-hook.sh.drv'
copied source '/nix/store/l6smcclpy9f3v562q2ljb9zjdswg3ma1-nixpkgs-22.11-src/pkgs/build-support/rust/hooks/cargo-check-hook.sh' -> '/nix/store/b7jvshpy1n3as2c1c7b45qc4fgp4nfh0-cargo-check-hook.sh'
error: writing to file: No space left on device
(use '--show-trace' to show detailed location information)

No space left on device. That’s odd.

> df -h
Filesystem                                               Size  Used Avail Use% Mounted on
overlay                                                  1.7T   42G  1.5T   3% /
tmpfs                                                     64M     0   64M   0% /dev
tmpfs                                                     30G     0   30G   0% /sys/fs/cgroup
tmpfs                                                     12G  8.0M   12G   1% /io
/dev/nbd70                                               2.0G  1.8G   92K 100% /tmp
overlay                                                  2.0G  1.8G   92K 100% /nix
/dev/mapper/docker                                       1.7T   42G  1.5T   3% /etc/hosts
/dev/root                                                 25G   14G   11G  58% /mnt/cacache
/dev/disk/by-id/google-nixmodules-1bed554-us-west1-b      44G   39G  2.3G  95% /etc/nixmodules
shm                                                       64M     0   64M   0% /dev/shm
/dev/disk/by-id/google-cacache-1687443650319-us-west1-b   16T   16T  658G  96% /mnt/cacache/nix
devtmpfs                                                  30G     0   30G   0% /dev/tty
tmpfs                                                     30G     0   30G   0% /proc/acpi
tmpfs                                                     30G     0   30G   0% /proc/scsi
tmpfs                                                     30G     0   30G   0% /sys/firmware
overlay                                                  2.0G  1.8G   92K 100% /home/runner
overlay                                                  2.0G  1.8G   92K 100% /home/runner/.cargo/registry
overlay                                                  2.0G  1.8G   92K 100% /home/runner/.m2/repository
overlay                                                  2.0G  1.8G   92K 100% /home/runner/.npm
overlay                                                  2.0G  1.8G   92K 100% /home/runner/.cache/pip
/dev/nbd113                                              256G   51M  256G   1% /home/runner/GiganticAdmiredMineral

So, apparently /nix only has 2GB of disk and is 100% full? I guess that would mean this is the “scratch disk” that Replit mentions in the resources tab?

So I guess my first questions are:

  1. How can I install an older version of a Nix package without running out of disk?
  2. Is there a way I can increase the size allocated to /nix (which I assume is the “scratch disk”)?
2 Likes

Hi @Nezteb thanks for your question.

I’m going to link @coltonatreplit into this conversation to see if he is able to assist.

1 Like

I mostly ended up fixing this.

My Replit was maxed at 2 GiB / 2 GiB, so I tried running the nix-collect-garbage command. However, even this apparently needs a little bit of disk space to run, so that would error. This is why I say I “mostly” fixed it, because it’d still be nice if Replit let you somehow increase your “scratch disk” space. Once that is full, I don’t think there’s a way to “recover” your Replit.

What I ended up doing is just creating a new Replit, and instead of using builtins.fetchGit which apparently does a full repo clone (and uses a lot of space), I switched to nixpkgsfetchFromGitHub function which intelligently pulls a tarball instead and uses much less space.

5 Likes

Okay apparently nix-collect-garbage only looks like it’s working. The command eventually hangs, even on a fresh Replit template (I’ve tried both the official Node.js and Deno ones).

> nix-collect-garbage
finding garbage collector roots...
deleting garbage...
deleting '/nix/store/vd7zphd2jpd7i638qxfqgpn6gifs8v9p-libarchive-3.5.2-lib'
deleting '/nix/store/i2ays442rwyfnvb5liiykrshpv1kwmhq-libunwind-1.6.2-dev'
deleting '/nix/store/7mblpjcysrg64zqcw7rkw50pqhpsc22c-aws-c-http-0.6.10'
deleting '/nix/store/a5yg01w4hrhxpyg5i2qgp15kj7q1p9wl-perl5.34.0-CGI-4.51'
deleting '/nix/store/5bkvi8z49bd19g4m81fdwl61ix3h3z7l-replit-git-askpass'
deleting '/nix/store/qmnr18aqd08zdkhka695ici96k6nzirv-libunistring-1.0'
deleting '/nix/store/sivifbn8k8qskd9zld1vka7cpzn2794b-llvm-13.0.0-lib'
deleting '/nix/store/hshkjnxrgidigf8x75vg6lqng2xlx9i3-prybar-sqlite-0.0.0-e0d9a52'
deleting '/nix/store/h5ss42gxq5xf4mfr9zbw331s2d53pldk-aws-c-mqtt-0.7.9'
deleting '/nix/store/22jj08nss8zvbgk52ikj09ssg6rw84rv-silver-searcher-2.2.0'
deleting '/nix/store/i30d0vrcjyyxvg3b782dg5mzv7dmsz06-hostingpid1-0.0.1'
deleting '/nix/store/j8kxvbz8qnfpbcdq3gjnbzagd3mqr4wv-libffi-3.4.2-dev'
deleting '/nix/store/xyg642b7s1hg5jh9kji925qvm169ah8b-aws-c-compression-0.2.14'
deleting '/nix/store/g7jnvxb8m5h2mz68j9zmsmcqdwfr1qa8-aws-c-auth-0.6.8'
deleting '/nix/store/f1rq5gyzf43kd10h2wzlvizyivzd2ir7-perl5.34.0-Test-Needs-0.002006'
deleting '/nix/store/60rpbrba6x98q8jzsb0lg72324llgqgk-libXxf86vm-1.1.4'
deleting '/nix/store/cjjbzdx72mnrzwcxdb6lgl7qzpvj8k68-libidn2-2.3.4'
deleting '/nix/store/wh24g3a8dmqh3hwhw93354sc19armza5-xorgproto-2021.5'
deleting '/nix/store/hc9jpmzh0hjapjlv2qdnbbr5m1wgs3l8-libid3tag-0.15.1b'
deleting '/nix/store/7kgbzirqiw4ifrcm5y34bs2d2i1ma830-imlib2-1.7.3'
deleting '/nix/store/79lpws5zg4fjfchxpd9xh89nnlxxhgln-libopus-1.3.1'
deleting '/nix/store/ypiy5bax1rsx7bq47vdkxc4si9qnl3xf-xz-5.2.5-dev'
deleting '/nix/store/xckpq3x3x4l79d8jq81d0p13cn8l3xg6-font-util-1.3.1'
deleting '/nix/store/24q63bgri76a9zy2mi79mkb1kp70h7v5-pydeppaths-1.0.0'
deleting '/nix/store/s8zb4a15cv2s922xx5bjm8jl5ym2qa8k-fluxbox-config'
deleting '/nix/store/f87abb8azh490fkjbl08sp4qa2p51gwi-replit-replbox-modules-3.0.0'
deleting '/nix/store/af5h0a4b6jz48f0vsgw2faff0jbssbnq-libepoxy-1.5.9'
deleting '/nix/store/hhpqldw0552mf4mjdm2q7zqwy9hpfchd-libpng-apng-1.6.37-dev'
deleting '/nix/store/d810g729g1c4lvp3nv1n3ah6cvpwg7by-cairo-1.16.0-dev'
deleting '/nix/store/hd9jdc86n582jz7c6n933xahfck9llpr-libXext-1.3.4-dev'
deleting '/nix/store/v0x0s9023zab77mmx67588d3ja0svvqf-perl5.34.0-HTTP-Cookies-6.09'
deleting '/nix/store/w186lhwpfdlfa506qcgpid6i3pnh9pk2-xsetroot-1.1.2'
deleting '/nix/store/3vawh2sv7p0ar3cbw0rsp7vn3w159gkj-aws-c-s3-0.1.30'
deleting '/nix/store/schwwavpkcjdj80dj00lrcmqqx7cyphm-libepoxy-1.5.9-dev'
deleting '/nix/store/2sggwyskp7iw28rsdw9r69fkhz1sxk2g-xterm-370'
deleting '/nix/store/4522jkjzj2nlx2z67325aylg6ngvvg2h-curl-7.80.0-bin'
deleting '/nix/store/b2nfjlq1qqhlkr9vx00a1157wpzfjyq4-libXpm-3.5.13'
deleting '/nix/store/k4b16aly0h618r1lakpz675m5l5nkxhl-libXinerama-1.1.4-dev'
deleting '/nix/store/dwiw7vqm8fhcpr7a8zv14cwx5nx7g7xa-zlib-1.2.11-dev'
deleting '/nix/store/gxpcbqmibijqh6y6djy970z89813fnwq-bzip2-1.0.6.0.2-dev'
deleting '/nix/store/65mzkxjcffbz4lv2gjcbd18xwjs1m1cf-libXfont-1.5.4-dev'
deleting '/nix/store/xrxggr3q0b2xabb4wxnqpvnf603bqjlg-glu-9.0.2'
deleting '/nix/store/pv40q74svqp3xs4dalhw3pq7bcdzvg5p-libelf-0.8.13'
deleting '/nix/store/xh4xz5aa4smirb66xi9xjaxjxcbsz5mg-dbus-1.12.20'
deleting '/nix/store/7yldbkifygqvm71ijk26607i5hl3cjv5-dbus-1.12.20-dev'
deleting '/nix/store/x3ccb10qpkzq9pb1bc0baqm4mhp8c0bx-libseccomp-2.5.4-lib'
deleting '/nix/store/i4kzir3m7mrisxmf5xvlnkw01x4qaqr4-prybar-elisp-0.0.0-e0d9a52'
deleting '/nix/store/b470h8znpzy0lbj5kqskzv52fg5k7gww-libpciaccess-0.16'
deleting '/nix/store/1s5l90kdzz5ii4bafi2dnyzv3smziq49-nodejs-16.13.2'
deleting '/nix/store/5nma803izq27z9xgp9y39683bljkxshk-libfontenc-1.1.4'
deleting '/nix/store/g4sjg38sly59s9yns88r07a6ynyqfrj8-perl5.34.0-HTML-Tagset-3.20'
deleting '/nix/store/68wsq8brrn56s2rdbw1xqim0n7z818ss-xgcc-12.2.0-libgcc'
deleting '/nix/store/bba5yv0b1dhcb6kdgjqn0sb5qzsqxkp4-aws-c-event-stream-0.2.7'
deleting '/nix/store/5i972w13v7635dhi3nh03gvadi7daa0m-libxshmfence-1.3'
deleting '/nix/store/kc0nrbhhxs4dhhjwz8sd43k5y7r76a0d-perl5.34.0-Encode-Locale-1.05'
deleting '/nix/store/7vhdm1d79mgm6qp1lfchp0dlxlpqfwqv-libcpuid-0.5.1'
deleting '/nix/store/vyllqmglsqwfxm5hxpimqirdkpv46g7q-libjpeg-turbo-2.1.2-dev'
deleting '/nix/store/m730z29kqnh2sdi2am007xp3g6s6ymlm-aws-sdk-cpp-1.9.121'
deleting '/nix/store/q8i4gdxq82ylczpgacakcnw6iphsc3bl-glibc-iconv-2.33'
deleting '/nix/store/z2wgsb3w13xnb81chpwrjcdh8rg3l081-moduleit'
deleting '/nix/store/p43bg9cn9f4nxa3fp2x3m3ifay0fm3mv-xauth-1.1'
deleting '/nix/store/qfxk9gzibng5ghhdq33bjx2cr20xlb8m-replit-git-editor'
deleting '/nix/store/8nw2i2pqc52i5m79jd1g6clmspfn02bm-sbc-1.4'
deleting '/nix/store/7rlm1l417gg6sc5mydf5ywsarm7rf268-soxr-0.1.3'
deleting '/nix/store/0ckz20b6824grwy2g5fgdysz4hcfwwp0-libXft-2.3.4-dev'
deleting '/nix/store/gvzyha79wkr1xazax4fd9nnbj8yp9rv6-node-dap'
deleting '/nix/store/492vd7iyr1npsailrb1c7v97l5h50qr9-iana-etc-20230316'
deleting '/nix/store/xi7y0ywzy0qhl98yf7y8fw1wh7spbxrp-xkeyboard-config-2.33'
deleting '/nix/store/m4zflhr10wz4frhgxqfi43rwvapki1pi-fontconfig-2.13.94-bin'
deleting '/nix/store/6l4wq77g8mj64111f66lwgizd931dg2s-perl5.34.0-IO-HTML-1.004'
deleting '/nix/store/533sqcildib4q6f3h216xrydpxzf321q-libogg-1.3.5'
deleting '/nix/store/sfhnqsiyb1qsmvd3k1b1pjr09lqryq9b-aws-crt-cpp-0.17.8'
deleting '/nix/store/d088w3wp3pvymd1ypbk4b8krhy5k8hnx-editline-1.17.1'
deleting '/nix/store/zff626j0lj5zn3qfi9iw9a5nl78qf6pp-libunwind-1.6.2'
deleting '/nix/store/4f5x1000fryla2pgzsf355yqdw28d3v3-perl5.34.0-Try-Tiny-0.30'
deleting '/nix/store/3w2h5hx2vrsjwa7r09gbxwz8hm537l3f-fzy-1.0'
deleting '/nix/store/j3ddxd078zc6fydzpjr7gqqrwl2q1fcm-mesa-21.3.3'
deleting '/nix/store/hz0xpslmqzawjcnxbmab0xpgw8cgy1w9-aws-checksums-0.1.11'
deleting '/nix/store/wlhmpa2qk786ddp05ahv3jjlds6d44a5-python3-3.8.12'
deleting '/nix/store/a1587xy70kz0kv8gg9kmmjwma3wlxwhk-libGL-1.4.0-dev'
deleting '/nix/store/jwciykjsflgpx34np8cg67z1sqfy7v8a-openssl-1.1.1m-dev'
deleting '/nix/store/zaamc32wijqka1mg8w2gr1f6bgvs7yyz-vscode-cpptools-dap-1.3.1'
deleting '/nix/store/hydbwa0ri39334340v0imcvk07srwrvv-perl5.34.0-TermReadKey-2.38'
deleting '/nix/store/kni8mhh8z77crjwznjcni8lxfkdvchzl-perl5.34.0-WWW-RobotRules-6.02'
deleting '/nix/store/afm3mdc4aby2r1rwzzn84ad5ki14xsy6-libXcursor-1.2.0-dev'
deleting '/nix/store/ij5rm5y6lmqzrwqd1zxckhbii3dg2nq5-glib-2.70.2-dev'
deleting '/nix/store/zk1fx6rqr8k9sv2l9m04rqlq30d4529j-perl5.34.0-FCGI-ProcManager-0.28'
deleting '/nix/store/nlmkq6qiri3fbpysiqz7jk2as50lsh01-libpulseaudio-14.2'
deleting '/nix/store/lvaaigk4j4941jhld2jpfi99s2b5yagx-libglvnd-1.4.0-dev'
deleting '/nix/store/r9hgikwb6jg2k8rwiv43xh8m9sjbfyv0-xcb-util-0.4.0-dev'
deleting '/nix/store/s62fv3slg63c3y14kcp2jrrjv5vwdla8-libunistring-1.1'
deleting '/nix/store/2r98if9wj5lhfangxyq1jgdqw2kb57rm-perl5.34.0-CGI-Fast-2.15'
deleting '/nix/store/jvn1nv8p8i9dgpzcy36nqxyc5mvjb4v2-perl5.34.0-HTTP-Negotiate-6.01'
deleting '/nix/store/k60lki4qx28nplgygcq992ygp761940j-libjpeg-turbo-2.1.2-bin'
deleting '/nix/store/nm3h3wrl4l620lj5yzbq2spmrj34164c-icu4c-70.1-dev'
deleting '/nix/store/mav4251if1ahjq5vkpwycyyvnmy459c4-pulseaudio-14.2'
deleting '/nix/store/zagi7gm1201dplqqynsv11jxdnawdywf-perl5.34.0-LWP-MediaTypes-6.04'
deleting '/nix/store/b63dcjck5pajj344cq8pc5wspj1fylsp-libXfixes-6.0.0-dev'
deleting '/nix/store/xnk2z26fqy86xahiz3q797dzqx96sidk-glibc-2.37-8'
deleting '/nix/store/c89fi7v456mr2c5qgair8yjcwl07pjw3-lowdown-0.10.0-lib'
deleting '/nix/store/146hw02zpnhkn5cabjf3pmdl32clz0jn-rfbproxy-0.0.0-9c8ca29'
deleting '/nix/store/84jawkvkysa35bisiqakxwb4rabx0634-aws-c-io-0.10.13'
deleting '/nix/store/9dv146bb7684r69431y8da9cxv0gimnx-aws-c-common-0.6.18'
deleting '/nix/store/0b2bvviijkvq2nvm04dbq3lwlq7h742m-tigervnc-1.12.0'
deleting '/nix/store/cp50fjdhcp9lx9y4fw78bxd2lkr8xpha-tzdata-2023c'
deleting '/nix/store/v2ypdwhzxhm5kb27hm2ajxfj5fgmv862-replit-glsetup'
deleting '/nix/store/gxj28g1cg4hq3qanmjv07156h800qr58-prybar-python3-0.0.0-e0d9a52'
deleting '/nix/store/44papibkmjkl0854bd3jq5cdsxkf7qry-sd-0.7.6'
deleting '/nix/store/mc468j5q40sl05h5craa5493n5dgba57-libxkbfile-1.1.0'
deleting '/nix/store/4frkcb5bdvqnn2nv16lx1qcfpzqgn5pi-nix-2.5.1-man'
deleting '/nix/store/k11ldf56nrw2dss4siil1yn2srfyz4gc-fltk-1.3.8'
deleting '/nix/store/7v07bmyfjq0jdnx6xwxlib05na8kksdf-xwininfo-1.1.4'
deleting '/nix/store/pb6sp9j9bv7s3bk1biqq05w73f40gpg8-ripgrep-13.0.0'
deleting '/nix/store/prkgrc9rywg7f1pmwddic2xiv1a6qp5d-libXrender-0.9.10-dev'
deleting '/nix/store/3ljjd465fia2ra2i7j4qly7kyg1nci2l-bash'
deleting '/nix/store/lzv9blgah1w3226q10mwxrqay4z0zl0y-replit-bashrc'
deleting '/nix/store/crzv5jw2vb3r74dvv5w558c27hv7phjn-iana-etc-20211124'
deleting '/nix/store/qjrgvafkk818hnvwpirwsp9y39c07s4z-libXfont2-2.0.5'
deleting '/nix/store/hw95g92w6lgvwdb01g88pxq48ijszh8l-libsndfile-1.0.31'
deleting '/nix/store/f472rqx45rdhq2dl8xhjlwwj56wjpjsj-openssl-1.1.1m-bin'
deleting '/nix/store/76l4v99sk83ylfwkz8wmwrm4s8h73rhd-glibc-2.35-224'
deleting '/nix/store/608nyba3g4jyn2h00m5cpwb4nzqijrvq-xdg-open'
deleting '/nix/store/ann2w0blx5vrc8mw1hwnvb37gry6ci9a-replit-python-dap-wrapper-1.0.0'
deleting '/nix/store/v0ys53df1fraqqdrsd7fjqpz58mdvsch-dconf-0.40.0-lib'
deleting '/nix/store/xwl0wqr3g68mx2b60qja3kn865gf45x5-toml-editor-0.0.0-5aba775'
deleting '/nix/store/5r12pcbz6la382zmv1iz8km0j0wdcw75-libXdamage-1.1.5'
deleting '/nix/store/37rlm9ls5irs165b5w9a161mrl45n747-libuv-1.42.0'
deleting '/nix/store/awhfrgcv6c5kmmw4gfrpvmbm6pvbx0yc-python3.8-replit-pip-21.1.3.b53d7fb'
deleting '/nix/store/pv89r9i1d8ajin2vp7qfs63nakg7s17k-replbox'
deleting '/nix/store/816nc03zz1fq9vjdrx0ryx9ikmndifbg-perl5.34.0-HTTP-Date-6.05'
deleting '/nix/store/ljz696b36v9mgynzl7qixzcfsk2vy250-libxcb-1.14-dev'
deleting '/nix/store/lyyq4apj3nn6a9z6008dmfiasvnfq5gm-openssh-8.8p1'
deleting '/nix/store/sla1aaa9sga3amg5kr9ipkf7f9xi404n-libvorbis-1.3.7'
deleting '/nix/store/q69g6ij5csvvfciwpmjfw3b68xbkz6dv-xkbcomp-1.4.5'
deleting '/nix/store/y52xqxp4r311d6941a1whv5ym2nagmy5-libasyncns-0.8'
deleting '/nix/store/3i7a458p31v8ni7cvp0sfn36q0y2s86v-pid1'
deleting '/nix/store/2025j2r6qri1qy4b2lzf9af749cakbz1-perl5.34.0-HTTP-Message-6.26'
deleting '/nix/store/0y8930yzrf4cglmjbyfqq237bvkldj2x-perl5.34.0-libwww-perl-6.49'
deleting '/nix/store/p11z3ldwjlmj8w2kvfy5vp9x1kj99j8b-perl5.34.0-URI-5.05'
deleting '/nix/store/k16np53a43gvzix7q5i5gnd298k4wbjb-perl5.34.0-HTML-TagCloud-0.38'
deleting '/nix/store/lyzn4qly8k878lgfy1b399w58q4khys2-fluxbox'
deleting '/nix/store/994snqhc53hi8pi5ig82fhyi3ayyfg4l-upm-0.0.0-228b4c8'
deleting '/nix/store/y6xh1i42jlfbd3k085q5vb8819gvxvz8-libX11-1.7.2-dev'
deleting '/nix/store/qpnf0dafrfsbcwb4z7cphjm0dff0vl7l-prybar-nodejs-0.0.0-e0d9a52'
deleting '/nix/store/j5raylzz6fsafbgayyfaydadjl0x22s0-freetype-2.11.1-dev'
deleting '/nix/store/i3d23in6871kdv2q1bcml05q072vbrfl-perl5.34.0-TimeDate-2.33'
deleting '/nix/store/afm9529jnpi8589y112v8i7hh927bjh0-systemd-249.7-dev'
deleting '/nix/store/hpbvrlwd5viqmvcaisb3x0118i913a9n-stderred-0.1.0'
deleting '/nix/store/1c41n7dri6007spy041r8qhxwf9sn77j-lame-3.100-lib'
deleting '/nix/store/39ba5cbdy20ndrh7awbschhbdz9n8cka-perl5.34.0-HTTP-Daemon-6.01'
deleting '/nix/store/7ccycincvjj9rlk7x3mlv1sdqi3x2qzl-perl5.34.0-File-Listing-6.14'
deleting '/nix/store/csidl7dwbl9qzchlzczrrxlxzsaazxfj-perl5.34.0-Net-HTTP-6.19'
deleting '/nix/store/128xaj1flh9shi5x8pdbpiw0k2n21j6y-nix-index-0.1.2'
deleting '/nix/store/1b4366xcg7x7fw6809mvck4nrlx519la-boehm-gc-8.0.6'
deleting '/nix/store/yv9z0pi9vwzqj0pqyl3yh3qazwhi6dd7-libXaw-1.0.14'
deleting '/nix/store/45636agiysjfajk142llsv10b1grix9q-webrtc-audio-processing-0.3.1'
deleting '/nix/store/jha53c5kh54c46fgywh37v3m3h4hcl0k-git-2.34.1'
deleting '/nix/store/lwwivxf6h7x3cvxp9zv8sj0mv7dak99r-prybar-lua-0.0.0-e0d9a52'
deleting '/nix/store/128k97ig6hhvjw88mikmnncgm05jz8ss-speexdsp-1.2.0'
deleting '/nix/store/n58mx9iy8kal6amy15n5hwfihrgx78vw-stderred-unstable-2021-04-28'
deleting '/nix/store/c4w01bpjgyy5ixmwdk8m1pw87zq4sl26-prybar-tcl-0.0.0-e0d9a52'
deleting '/nix/store/af6dc7crblrc1m3qr9x7s4mwm0wb4pp1-libedit-20210910-3.1'
deleting '/nix/store/2k1khra5qf3qhjmv8ainaxjnvgvxpz0r-mailcap-2.1.53'
deleting '/nix/store/abvijrl56pyn7bvfggj6nlzrwz7l912a-nix-index-0.1.2'
deleting '/nix/store/2y755vwnypibgvhi3xkd80h9m4yiiq74-libXfont-1.5.4'
deleting '/nix/store/l461670jfs63f0aclmyclfad7i5cqlv9-busybox-static-x86_64-unknown-linux-musl-1.34.1'
deleting '/nix/store/r3i5yzzr7v5j0dh4nnqpab9kzsmjhja4-s2n-tls-1.3.0'
deleting '/nix/store/fd4a776gbp7qpdqw7s9k4ls1mwhngfwb-fftw-single-3.3.10'
deleting '/nix/store/9b3q2b4h5cbpfzx7w2rp8mjhlj6js82f-fontconfig-2.13.94-dev'
deleting '/nix/store/b61qqqjgzmf9wk0m23yp7l6zxm6pc4wr-twm-1.0.10'
deleting '/nix/store/lz9isarnhrbwmh5v087n3sqvy604a7x8-libXau-1.0.9-dev'
deleting '/nix/store/1mr0b05mimyp0m85mpw6zmkvygcl7vjh-perl5.34.0-FCGI-0.79'
deleting '/nix/store/bfmyvk668bl0n294qancg8sg5x42p9vx-nix-editor-0.0.0-8e65b7f'
deleting '/nix/store/28l3nwnl6nplzypx93gghgj1sykkksvf-libsodium-1.0.18'
deleting '/nix/store/p36pgcv991mq5srvg765b78yqpxvk3qs-mesa-21.3.3-drivers'
deleting '/nix/store/c5mljiq20nin31v8whan6nh73457mqlc-aws-c-cal-0.5.12'
deleting '/nix/store/xzh48v7xb1fyp5vaw46ja8if76fd86rw-glu-9.0.2-dev'
deleting '/nix/store/dkxy0xhc47aph9l05adc4z5xnp7df6aa-aws-c-sdkutils-0.1.1'
deleting '/nix/store/6ifc61p92wcjfc63cfjbkbqwzax751xn-flac-1.3.3'
deleting '/nix/store/farb2h21v4fynpzdzsww6pmp6lqfwbdw-fluxbox-1.3.7'
deleting '/nix/store/1jzl8nawbli2g2x94p6255wcs7m0w9al-libdrm-2.4.109'
deleting '/nix/store/n4pxf9y2w9qy6kaam1azn8fa36nvwydw-libxshmfence-1.3-dev'
deleting '/nix/store/g2s02jghkv4ylzjpnrlfj1v7b7g1wfbr-wayland-1.20.0'
deleting '/nix/store/s89yn85ih2i09h7sjfjgwjwcywn4pml4-ztoc-0.1.0'
deleting '/nix/store/05kg55dyr06zfz4q28g603sz7q21xfc5-perl5.34.0-Test-RequiresInternet-0.05'
deleting '/nix/store/rnaxv877m5snk2mh7y8w96g3v2wazncv-perl5.34.0-Test-Fatal-0.016'
deleting '/nix/store/5v83ni9crm70xkgd4vws6y4fayzv7xc3-libsamplerate-0.1.9'
deleting '/nix/store/gxyhqkpahahn4h8wbanzfhr1zkxbysid-expat-2.4.2-dev'
deleting '/nix/store/5p96sifyavb407mnharhyzlw6pn6km1b-glib-2.70.2-bin'
deleting '/nix/store/af5np7r9yw2lv678kaavs297bpspqai7-git-2.34.1-doc'
deleting '/nix/store/ijsi6sjhmx9vvcyblklw6kagq7g20vk8-perl5.34.0-HTML-Parser-3.75'
deleting '/nix/store/krqdfixxbjqxsmx70wlwr69lah6mr2r2-nix-2.5.1'
deleting '/nix/store/vv6rlzln7vhxk519rdsrzmhhlpyb5q2m-libidn2-2.3.2'
(hangs here)

If I refresh my browser tab, Replit loads fine but the scratch disk doesn’t change, and running nix-collect-garbage again gives the same output and still hangs. :thinking:

If I try to manually rm -rf /nix/store/vv6rlzln7vhxk519rdsrzmhhlpyb5q2m-libidn2-2.3.2, I get a bunch of permission denied errors and Replit doesn’t support sudo usage. Is there no way for a Replit user to manually trigger a Nix garbage collect?

2 Likes

I’m still running into this. I tried cloning GitHub - Budibase/budibase: Low code platform for creating internal tools, workflows, and admin panels in minutes. Supports PostgreSQL, MySQL, MSSQL, MongoDB, Rest API, Docker, K8s, and more 🚀. Budibase, the low code platform you'll enjoy using ⚡ into a new Replit so I could build/hack on it. As part of that, I added lerna to my Nix packages. While my Nix env was reloading though, I started getting errors about no space. Apparently trying to install regular Node packages with yarn also uses “scratch disk”.

My storage looks like:

And once again, because my “scratch disk” is so close to full, I essentially can’t do anything on the container. nix-collect-garbage still does nothing even after waiting for it to finish for 20 minutes.

The docs say “[scratch disk] does not count towards account storage, and resets regularly”, but how regularly?

Any ideas?

1 Like

It’d have to delete a giant list of derivations (even just the list of directories in /nix/store is over 20 MiB), so that won’t work.

That happens when the repl restarts afaik. (Run kill 1 in the shell to restart it)

1 Like

I ran nix path-info --json --all | jq 'map(.narSize) | add' to “Show the size of the entire Nix store” and got 9426116176512 bytes, which is 9.42 terabytes, so that doesn’t seem right. My only guess is that Replits are not unique containers/VMs but instead Replits are partitioned to a set of powerful/large machines and users share resources while being sandboxed from each other somehow?

I then tried running the nix-store --optimise command, which ran for a bit before running out of space (on both a Replit with full scratch disk and a fresh/clean Replit).

I then tried running the nix-collect-garbage -d command (the -d only clears user profiles, not system ones), which looked like it was working, but then died because of no free space. However, if I create a fresh new Replit, that command runs for a bit and slowly starts eating more and more RAM. It’s been 10 minutes so far but I’ll edit this when it finishes.

After more searching, I’ve realized that Replit stores node_modules and other random temp files in “scratch disk”, not just Nix stuff. So all of my attempts to forcing Nix to garbage collect wouldn’t really matter anyway. This makes building even somewhat “large” projects (like Budibase) impossible with only 2gb to use.

tl;dr I don’t know how to increase “scratch disk” nor manually clear it effectively; it’s probably not possible.

Stuck with same error for last few hours, were you able to fix it?

Hi @Nezteb! Apologies for the delay in response!

I see you’ve been trying nix-collect-garbage - this won’t work.

which is 9.42 terabytes, so that doesn’t seem right. My only guess is that Replits are not unique containers/VMs but instead Replits are partitioned to a set of powerful/large machines and users share resources while being sandboxed from each other somehow?

not quite - Repls are indeed unique containers, but we have a giant cache disk of the nix store pre-built and hosted in the cloud that contains every derivation built by hydra for every version of the channels we’ve given Replit users access to. More succinctly: /nix/store is a read-only disk that we attach to every Repl. We do the same for a few other directories (why ~/.cargois always pre-populated, even in non-Rust Repls, for example).

The scratch disk is the ephemeral part of the filesystem (read: what the container operates on by default). Every time your container is destroyed (disconnect, kill 1, etc), the contents of the scratch disk are lost. This affects $HOME, /tmp, /etc, and more. The size of this disk cannot be changed by users, and we don’t offer ways to increase this size. As such, many Nix features (like custom derivations and flakes) aren’t officially supported by Replit at the moment.

If you’d like to take advantage of your account-level storage, it must go in your $REPL_HOME - which is an env var pointing to /home/runner/<repl slug>. Any modifications to other directories will be lost and are limited by the ~2 GB scratch disk storage quota.

If you’d like to try it, any nix derivations you build are in /mnt/scratch/nix/store/upper. If you need to clear out intermediate dependencies, this may give some of your scratch disk storage back temporarily. Note that anything in this directory will be lost upon container destruction anyways.

PS As you noticed, nix-collect-garbage was actually successfully removing directories. While I said that /nix/store is read-only, you can actually make it seem like you’re editing the disk. However, these directories are restored upon container construction, so you’re only temporarily removing those paths during your container session but otherwise this isn’t productive to what you’re trying to achieve.

8 Likes

If you’d like to use glibc version 2.33, you can use our legacy nix channel by commenting-out the channel = "<something here>" line in your .replit file. Other ideas I have that may or may not work but you’re free to try:

  1. make a wrapper for supabase-cli that forces the legacy channel’s version of glibc to be used. I think the 2 env vars you want are LD_PRELOAD and LD_LIBRARY_PATH.
  2. Use a version of supabase that was compiled with musl
  3. Ask the supabase devs to enable linking up to glibc 2.37 (the latest release available in Repls)
4 Likes

Thank you for the detailed response Colton!

Is there a way to do this with other things like node_modules? Building large(ish) Node/JS/TS projects fills up the 2 GB of scratch disk quite quickly. :sweat_smile:

1 Like

Thank you for the detailed response Colton!

of course! that’s what I’m here for :slight_smile:

Is there a way to do this with other things like node_modules ? Building large(ish) Node/JS/TS projects fills up the 2 GB of scratch disk quite quickly. :sweat_smile:

Your project directory (aka what you see when you open the Repl Workspace) is $REPL_HOME. Unless you’re installing node modules globally on a non-recent fork, this is already done :slight_smile: (we added support for globally-installed node modules late last week in https://replit.com/@replit/Nodejs?v=1 and https://replit.com/@replit/TypeScript?v=1, but project-level node modules have never used the scratch disk)

2 Likes

Ahh, that makes sense. I was trying to clone and build GitHub - Budibase/budibase: Low code platform for creating internal tools, workflows, and admin panels in minutes. Supports PostgreSQL, MySQL, MSSQL, MongoDB, Rest API, Docker, K8s, and more 🚀. Budibase, the low code platform you'll enjoy using ⚡ but during yarn install the Replit runs out of scratch space each time. :thinking:

1 Like

That’s… strange. I suggest opening a new topic here on Ask and a Supporter will be able to help out!

2 Likes

Done!

2 Likes