ChatGPT解决这个技术问题 Extra ChatGPT

How to free disk space taken up by (ana)conda?

I am using the conda package manager - a lot. By now I have quite a few environments and a lot of downloaded packages taking a lot of space on my SSD. An obvious path to free some of that space is to use the command

conda env export > environment.yml

from https://conda.io/docs/user-guide/tasks/manage-environments.html#exporting-the-environment-file to export which packages my old, inactive projects use(d) and then delete these environments. As far as I understand, this should free some of the space in anaconda2/envs/, but not in anaconda2/pkgs/. How do I get rid of these packages? Also, I suspect that there might be quite a few packages still sitting around, to which no environment is linking to - could that happen?

Questions:

In general: What is the best way to reduce the space taken up by conda? How do I get rid of packages that no environment is using anymore? How do I prune my packages? I am searching for something like sudo apt-get autoremove from Ubuntu/Debian.


M
Mike Müller

You can free some space with:

conda clean --all

clean Remove unused packages and caches.

Conda already use symlinks when possible for packages. So, not much to improve here, I guess.

Ok, thanks, but I would like to know "not for a specific environment, but in general" - for all environments.

You can list all packages in all envs with a few lines of Python:

import os
import subprocess
for env in os.listdir('/Users/me/miniconda3/envs'):
    subprocess.call(['conda', 'list', '-n', env])

I tried it out with conda clean --dry-run --all. More than 3GB would be deleted - not too bad. However, it lists packages I recently installed and actively use. That confuses me: What does it mean when .tar.bz2 are about to be deleted? Are those only the downloads and not the installs? Or maybe it is something old and I have a never version installed (and I forgot)?
.tar.bz2 is just the download that is cached for a potential second install. They are called tarballs. The installs stay as long as there are in use.
Ok, thanks, but I would like to know "not for a specific environment, but in general" - for all environments.
Add a solution to my answer. BTW, you can accept an answer if it solves your problem. ;)
Was some change done to the command conda clean --all? These days I tried using it again and it reliably cleaned up all the packages that were not used by any packages.
M
Make42

Finally I got around dealing with this issue. In the end it was a couple of days work:

For all my Python projects I use PyCharm and with it I checked which project uses which environment. For all environments I used the conda env export > environment.yml to save the settings of the environment from https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#exporting-the-environment-file Check whether my projects still work with new environments created from the environment.yml. Use anaconda-clean from option B in https://docs.anaconda.com/anaconda/install/uninstall and put the created backup in a save place. Rename the old anaconda2 directory to anaconda2_backup. Install a new conda environment - miniconda3 in my case. Build new environments which are need for current projects from the environment.ymls and check whether these work. Delete the old anaconda backups.

Finally I also reduced my logical volume with https://blog.shadypixel.com/how-to-shrink-an-lvm-volume-safely/ but this is only for Linux users using LVMs.

This way I was able to free 20 to 30 GB of space.


I don't quite understand the point here. The conda clean --all suggested above does not work with individual environments, it works with all environments. Can you explain what you achieve here that is not achieved by conda clean --all. Not that it takes 30s instead of a few days! It does not automatically remove envs you don't need anymore but reading your question it was about removing packages not used by any env not about envs you are not using anymore.
@ARoebel: stackoverflow.com/questions/48706548/… hints that at the time the clean --all did not work for me: Too little was cleaned then, while for current versions it cleans much more. That is why back then, I accepted my own answer as the solution and later changed that to Mike Müller's answer. Also, I had the issue that I had a lot of environments of dormant projects which configs I had to back up and remove. (Mybe that was the real problem, but I cannot reproduce this.)