ChatGPT解决这个技术问题 Extra ChatGPT

How to upgrade all Python packages with pip

Want to improve this post? Provide detailed answers to this question, including citations and an explanation of why your answer is correct. Answers without enough detail may be edited or deleted.

Is it possible to upgrade all Python packages at one time with pip?

Note: that there is a feature request for this on the official issue tracker.

Beware software rot—upgrading dependencies might break your app. You can list the exact version of all installed packages with pip freeze (like bundle install or npm shrinkwrap). Best to save a copy of that before tinkering.
If you want to update a single package and all of its dependencies (arguably a more sensible approach), do this: pip install -U --upgrade-strategy eager your-package
I use PowerShell 7 and currently I use this one-liner: pip list --format freeze | %{pip install --upgrade $_.split('==')[0]} (I am unable to post an answer here yet)

M
Mateen Ulhaq

There isn't a built-in flag yet, but you can use:

pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 pip install -U

For older versions of pip:

pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 pip install -U

The grep is to skip editable ("-e") package definitions, as suggested by @jawache. (Yes, you could replace grep+cut with sed or awk or perl or...).

The -n1 flag for xargs prevents stopping everything if updating one package fails (thanks @andsens).

Note: there are infinite potential variations for this. I'm trying to keep this answer short and simple, but please do suggest variations in the comments!


Right :( The issue now lives at github.com/pypa/pip/issues/59 . But every suggestion seems to be answered with "Yeah, but I'm too sure if X is the right way to do Y"... Now is better than never? Practicality beats purity? :(
It also prints those packages that were installed with a normal package manager (like apt-get or Synaptic). If I execute this pip install -U, it will update all packages. I'm afraid it can cause some conflict with apt-get.
How about changing grep to: egrep -v '^(\-e|#)' (i get this line when running it on ubuntu 12.10: "## FIXME: could not find svn URL in dependency_links for this package:".
I'd throw in a tee before doing the actual upgrade so that you can get a list of the original verisons. E.g. pip freeze --local | tee before_upgrade.txt | ... That way it would be easier to revert if there's any problems.
I added -H to sudo to avoid an annoying error message: $ pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 sudo -H pip install -U
M
MSS

You can use the following Python code. Unlike pip freeze, this will not print warnings and FIXME errors. For pip < 10.0.1

import pip
from subprocess import call

packages = [dist.project_name for dist in pip.get_installed_distributions()]
call("pip install --upgrade " + ' '.join(packages), shell=True)

For pip >= 10.0.1

import pkg_resources
from subprocess import call

packages = [dist.project_name for dist in pkg_resources.working_set]
call("pip install --upgrade " + ' '.join(packages), shell=True)

This works amazingly well… It's always so satisfying when a task takes a REALLY long time… and gives you a bunch of new stuff! PS: Run it as root if you're on OS X!
Is there no way to install using pip without calling a subprocess? Something like import pip pip.install('packagename')?
I wrapped this up in a fabfile.py. Thanks!
@BenMezger: You really shouldn't be using system packages in your virtualenv. You also really shouldn't run more than a handful of trusted, well-known programs as root. Run your virtualenvs with --no-site-packages (default in recent versions).
Thumbs up for this one, the chosen answer (above) fails if a package can't be found any more. This script simply continues to the next packages, wonderful.
P
Pikamander2

To upgrade all local packages, you can install pip-review:

$ pip install pip-review

After that, you can either upgrade the packages interactively:

$ pip-review --local --interactive

Or automatically:

$ pip-review --local --auto

pip-review is a fork of pip-tools. See pip-tools issue mentioned by @knedlsepp. pip-review package works but pip-tools package no longer works.

pip-review works on Windows since version 0.5.


@hauzer: It doesn't support Python 3. Though it might be a bug
@mkoistinen It's a good tool but until it's merged in PIP it means installing something additional which not everyone may desire to do.
@Daniel: pip-tools no longer works, pip-review (fork of pip-tools) works.
pip-review works just fine (at least for Python version 3.5.0)
To update all without interactive mode: pip-review --local --auto
a
azazelspeaks

The following works on Windows and should be good for others too ($ is whatever directory you're in, in the command prompt. For example, C:/Users/Username).

Do

$ pip freeze > requirements.txt

Open the text file, replace the == with >=, or have sed do it for you:

$ sed -i 's/==/>=/g' requirements.txt

and execute:

$ pip install -r requirements.txt --upgrade

If you have a problem with a certain package stalling the upgrade (NumPy sometimes), just go to the directory ($), comment out the name (add a # before it) and run the upgrade again. You can later uncomment that section back. This is also great for copying Python global environments.

Another way:

I also like the pip-review method:

py2
$ pip install pip-review

$ pip-review --local --interactive

py3
$ pip3 install pip-review

$ py -3 -m pip-review --local --interactive

You can select 'a' to upgrade all packages; if one upgrade fails, run it again and it continues at the next one.


You should remove requirements.txt's =={version}. For example: python-dateutil==2.4.2 to python-dateutil for all lines.
I found that this didn't actually upgrade the packages on macOS.
@youngminz I would recommand a quick 'Replace all "==" > ">=" ' in your editor/ide before running 'pip install...' to fix this
for linux: $ pip freeze | cut -d '=' -f1> requirements.txt in order to remove the version
If the shell you use is bash, you can shorten it into one command via pip3 install -r <(pip3 freeze) --upgrade Effectively, <(pip3 freeze) is an anonymous pipe, but it will act as a file object
P
Peter Mortensen

Use pipupgrade!

$ pip install pipupgrade
$ pipupgrade --verbose --latest --yes

pipupgrade helps you upgrade your system, local or packages from a requirements.txt file! It also selectively upgrades packages that don't break change.

pipupgrade also ensures to upgrade packages present within multiple Python environments. It is compatible with Python 2.7+, Python 3.4+ and pip 9+, pip 10+, pip 18+, pip 19+.

https://i.stack.imgur.com/37F74.gif

Note: I'm the author of the tool.


This method has much cleaner output than @rbp's answer
Nice idea, but it's stuck at Checking... forever when I tried it.
Got an error on Windows 10 and Python 3.7.5: ModuleNotFoundError: No module named 'ctypes.windll'
Seems to have some problems: Checking... 2020-03-16 11:37:03,587 | WARNING | Unable to save package name. UNIQUE constraint failed: tabPackage.name 2020-03-16 11:37:13,604 | WARNING | Unable to save package name. database is locked 2020-03-16 11:37:13,609 | WARNING | Unable to save package name. database is locked
It seems that this will upgrade all packages to the latest version and that may break some dependencies.
w
wilson

Windows version after consulting the excellent documentation for FOR by Rob van der Woude:

for /F "delims===" %i in ('pip freeze') do pip install --upgrade %i

for /F "delims= " %i in ('pip list --outdated') do pip install -U %i Quicker since it'll only try and update "outdated" packages
@RefaelAckermann I suspect this will be slower than the original :) To know which packages are outdated pip has to first check what's the latest version of each package. It does exactly the same as the first step when updating and does not proceed if there's no newer version available. However in your version pip will check versions two times, the first time to establish the list of outdated packages and the second time when updating packages on this list.
@RefaelAckermann Spinning up pip is order of magnitude faster than checking version of a package over network so that's number of checks which should be optimized not number of spin ups. Mine makes n checks, yours makes n+m checks.
+1 - It's 6.20.2019, I'm using Python 3.7.3 on WIndows 10, and this was the best way for me to update all my local packages.
Need to skip the first two lines of the output: for /F "skip=2 delims= " %i in ('pip list --outdated') do pip install --upgrade %i. If this is run from a batch file, make sure to use %%i instead of %i. Also note that it's cleaner to update pip prior to running this command using python -m pip install --upgrade pip.
M
Marc

This option seems to me more straightforward and readable:

pip install -U `pip list --outdated | awk 'NR>2 {print $1}'`

The explanation is that pip list --outdated outputs a list of all the outdated packages in this format:

Package   Version Latest Type 
--------- ------- ------ -----
fonttools 3.31.0  3.32.0 wheel
urllib3   1.24    1.24.1 wheel
requests  2.20.0  2.20.1 wheel

In the awk command, NR>2 skips the first two records (lines) and {print $1} selects the first word of each line (as suggested by SergioAraujo, I removed tail -n +3 since awk can indeed handle skipping records).


If one upgrade fails, none of the upgrades happen.
r
raratiru

The following one-liner might prove of help:

(pip > 20.0)

pip list --format freeze --outdated | sed 's/=.*//g' | xargs -n1 pip install -U

Older Versions: pip list --format freeze --outdated | sed 's/(.*//g' | xargs -n1 pip install -U

xargs -n1 keeps going if an error occurs.

If you need more "fine grained" control over what is omitted and what raises an error you should not add the -n1 flag and explicitly define the errors to ignore, by "piping" the following line for each separate error:

| sed 's/^<First characters of the error>.*//'

Here is a working example:

pip list --format freeze --outdated | sed 's/=.*//g' | sed 's/^<First characters of the first error>.*//' | sed 's/^<First characters of the second error>.*//' | xargs pip install -U

Had to add filters for lines beginning with 'Could' and 'Some' because apparently pip sends warnings to stdout :(
OK, this is fair: You can add as many | sed 's/^<First characters of the error>.*//' as needed. Thank you!
Or: pip list --outdated | cut -d ' ' -f 1 | xargs -n 1 pip install --upgrade
P
Peter Mortensen

You can just print the packages that are outdated:

pip freeze | cut -d = -f 1 | xargs -n 1 pip search | grep -B2 'LATEST:'

Inside a virtualenv, I do it like this: pip freeze --local | cut -d = -f 1 | xargs -n 1 pip search | grep -B2 'LATEST:'
Nowadays you can also do that with python -m pip list outdated (though it's not in requirements format).
@Jacktose I think you meant python -m pip list --outdated.
M
Michel

More Robust Solution

For pip3, use this:

pip3 freeze --local |sed -rn 's/^([^=# \t\\][^ \t=]*)=.*/echo; echo Processing \1 ...; pip3 install -U \1/p' |sh

For pip, just remove the 3s as such:

pip freeze --local |sed -rn 's/^([^=# \t\\][^ \t=]*)=.*/echo; echo Processing \1 ...; pip install -U \1/p' |sh

OS X Oddity

OS X, as of July 2017, ships with a very old version of sed (a dozen years old). To get extended regular expressions, use -E instead of -r in the solution above.

Solving Issues with Popular Solutions

This solution is well designed and tested1, whereas there are problems with even the most popular solutions.

Portability issues due to changing pip command line features

Crashing of xargs because of common pip or pip3 child process failures

Crowded logging from the raw xargs output

Relying on a Python-to-OS bridge while potentially upgrading it3

The above command uses the simplest and most portable pip syntax in combination with sed and sh to overcome these issues completely. Details of the sed operation can be scrutinized with the commented version2.

Details

[1] Tested and regularly used in a Linux 4.8.16-200.fc24.x86_64 cluster and tested on five other Linux/Unix flavors. It also runs on Cygwin64 installed on Windows 10. Testing on iOS is needed.

[2] To see the anatomy of the command more clearly, this is the exact equivalent of the above pip3 command with comments:

# Match lines from pip's local package list output
# that meet the following three criteria and pass the
# package name to the replacement string in group 1.
# (a) Do not start with invalid characters
# (b) Follow the rule of no white space in the package names
# (c) Immediately follow the package name with an equal sign
sed="s/^([^=# \t\\][^ \t=]*)=.*"

# Separate the output of package upgrades with a blank line
sed="$sed/echo"

# Indicate what package is being processed
sed="$sed; echo Processing \1 ..."

# Perform the upgrade using just the valid package name
sed="$sed; pip3 install -U \1"

# Output the commands
sed="$sed/p"

# Stream edit the list as above
# and pass the commands to a shell
pip3 freeze --local | sed -rn "$sed" | sh

[3] Upgrading a Python or PIP component that is also used in the upgrading of a Python or PIP component can be a potential cause of a deadlock or package database corruption.


another way to overcome the jurassic BSD sed of OS X is to use gsed (GNU sed) instead. To get it, brew install gnu-sed
@WalterTross ... Jurassic ... good adjective use. So we now have two ways to group update pip packages with a nice audit trail on the terminal. (1) Use the -E option as in the answer and (2) install gsed to leave the Jurassic period.
P
Peter Mortensen

I had the same problem with upgrading. Thing is, I never upgrade all packages. I upgrade only what I need, because project may break.

Because there was no easy way for upgrading package by package, and updating the requirements.txt file, I wrote this pip-upgrader which also updates the versions in your requirements.txt file for the packages chosen (or all packages).

Installation

pip install pip-upgrader

Usage

Activate your virtualenv (important, because it will also install the new versions of upgraded packages in current virtualenv).

cd into your project directory, then run:

pip-upgrade

Advanced usage

If the requirements are placed in a non-standard location, send them as arguments:

pip-upgrade path/to/requirements.txt

If you already know what package you want to upgrade, simply send them as arguments:

pip-upgrade -p django -p celery -p dateutil

If you need to upgrade to pre-release / post-release version, add --prerelease argument to your command.

Full disclosure: I wrote this package.


This is what pip should do by default.
heads up with your tool some character escapes don't seem to work correctly on my windows machine but other than that it's fine
haven't really tested it on windows, but i'll install a virtual machine. Thanks
If virtualenv is not enabled pip-upgrade --skip-virtualenv-check
This works also with a requirements folder having common, dev and prod requirements. Simply great!
S
Shihao Xu

This seems more concise.

pip list --outdated | cut -d ' ' -f1 | xargs -n1 pip install -U

Explanation:

pip list --outdated gets lines like these

urllib3 (1.7.1) - Latest: 1.15.1 [wheel]
wheel (0.24.0) - Latest: 0.29.0 [wheel]

In cut -d ' ' -f1, -d ' ' sets "space" as the delimiter, -f1 means to get the first column.

So the above lines becomes:

urllib3
wheel

then pass them to xargs to run the command, pip install -U, with each line as appending arguments

-n1 limits the number of arguments passed to each command pip install -U to be 1


I received this warning DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
@Reman: that is because you are using Pip v9.0.1. This is just a deprecation message meaning that some functionalities will not survive in a future Pip release. Nothing to be concerned about ;)
However, this has to be marked as the final solution. Indeed the accepted answer will run all over your pip packages, which is a waste of time if you have to update only 1 or 2 packages. This solution, as instead, will run just all over the outdated packages
P
Peter Mortensen

One-liner version of Ramana's answer.

python -c 'import pip, subprocess; [subprocess.call("pip install -U " + d.project_name, shell=1) for d in pip.get_installed_distributions()]'

subprocess.call("sudo pip install... in case you need permissions
@MaximilianoRios Please do not sudo pip install, use a virtual env, instead.
P
Peter Mortensen

From https://github.com/cakebread/yolk:

$ pip install -U `yolk -U | awk '{print $1}' | uniq`

However, you need to get yolk first:

$ sudo pip install -U yolk

Last commit 7 years ago
P
Peter Mortensen

When using a virtualenv and if you just want to upgrade packages added to your virtualenv, you may want to do:

pip install `pip freeze -l | cut --fields=1 -d = -` --upgrade

A
Andrew Jaffe

The pip_upgrade_outdated (based on this older script) does the job. According to its documentation:

usage: pip_upgrade_outdated [-h] [-3 | -2 | --pip_cmd PIP_CMD]
                            [--serial | --parallel] [--dry_run] [--verbose]
                            [--version]

Upgrade outdated python packages with pip.

optional arguments:
  -h, --help         show this help message and exit
  -3                 use pip3
  -2                 use pip2
  --pip_cmd PIP_CMD  use PIP_CMD (default pip)
  --serial, -s       upgrade in serial (default)
  --parallel, -p     upgrade in parallel
  --dry_run, -n      get list, but don't upgrade
  --verbose, -v      may be specified multiple times
  --version          show program's version number and exit

Step 1:

pip install pip-upgrade-outdated

Step 2:

pip_upgrade_outdated

Step 1: pip install pip-upgrade-outdated Step 2: pip-upgrade-outdated ...done
This is indeed a really good package. Needs more publicity, I have been working in python for a long while and this is the first time I hear about it. Nice!
@MarioChapa Thanks -- I wrote it (based on a gist).
In windows, %USERPROFILE%\anaconda3\envs\bridge\scripts\pip_upgrade_outdated.exe
R
RedEyed

The simplest and fastest solution that I found in the pip issue discussion is:

pip install pipdate
pipdate

Source: https://github.com/pypa/pip/issues/3819


Whereas other solutions stalled upon encountering the slightest anomaly, this solution warned and then skipped the problem to continue with the other packages. Great!
up voting this, Works perfectly in windows
I used pipdate and now can't find pip or python. Use at your own risk.
P
Peter Mortensen

Windows PowerShell solution

pip freeze | %{$_.split('==')[0]} | %{pip install --upgrade $_}

pip list --outdated | %{$_.split('==')[0]} | %{pip install --upgrade $_}?
Perhaps pip list --outdated --format freeze | %{$_.split('==')[0]} | %{pip install --upgrade $_} would be more appropriate.
Why is pip list --outdated --format freeze.. preferred over the suggested answer in Powershell, @brainplot
@Timo When I wrote that comment the suggested answer only used pip list instead of pip freeze. I figured --format freeze would be more robust against possible changes in future updates than letting pip list decide the format. pip freeze also works!
its even better to have it as function in your profile! This is perfect for anyone using powershell
P
Peter Mortensen

Use AWK update packages:

pip install -U $(pip freeze | awk -F'[=]' '{print $1}')

Windows PowerShell update

foreach($p in $(pip freeze)){ pip install -U $p.Split("=")[0]}

And for python 3... pip3 install -U $(pip3 freeze | awk -F'[=]' '{print $1}')
I
Isaque Elcio

Updating Python Packages On Windows Or Linux

1-Output a list of installed packages into a requirements file (requirements.txt):

pip freeze > requirements.txt

2- Edit requirements.txt, and replace all ‘==’ with ‘>=’. Use the ‘Replace All’ command in the editor.

3 - Upgrade all outdated packages

pip install -r requirements.txt --upgrade

Source:https://www.activestate.com/resources/quick-reads/how-to-update-all-python-packages/


This just works. Just do an pip freeze > requirements.txt afterwards to see the acutal diff.
pip freeze | sed 's/==/>=/' > requirements.txt to swap the == with >= automatically.
P
Peter Mortensen

One line in PowerShell 5.1 with administrator rights, Python 3.6.5, and pip version 10.0.1:

pip list -o --format json | ConvertFrom-Json | foreach {pip install $_.name -U --no-warn-script-location}

It works smoothly if there are no broken packages or special wheels in the list...


For purely aesthetic reasons, I like this approach the most. The output-producing executable provides our shell the object schema and there is no need for un-labelled index values [0] in the script.
G
German Lashevich

A pure Bash/Z shell one-liner for achieving that:

for p in $(pip list -o --format freeze); do pip install -U ${p%%=*}; done

Or, in a nicely-formatted way:

for p in $(pip list -o --format freeze)
do
    pip install -U ${p%%=*}
done

What does stand for?
@ᐅdevrimbaris this removes version spec and leaves only package name. You can see it by running for p in $(pip list -o --format freeze); do echo "${p} -> ${p%%=*}"; done. In more general way, ${haystack%%needle} means delete longest match of needle from back of haystack.
P
Peter Mortensen

You can try this:

for i in ` pip list | awk -F ' ' '{print $1}'`; do pip install --upgrade $i; done

this is the cleanest, highest readable way to update pip packages in the most amount of brevity. great.
P
Peter Mortensen

The rather amazing yolk makes this easy.

pip install yolk3k # Don't install `yolk`, see https://github.com/cakebread/yolk/issues/35
yolk --upgrade

For more information on yolk: https://pypi.python.org/pypi/yolk/0.4.3

It can do lots of things you'll probably find useful.


P
Peter Mortensen

Use:

pip install -r <(pip freeze) --upgrade

P
Peter Mortensen

Ramana's answer worked the best for me, of those here, but I had to add a few catches:

import pip
for dist in pip.get_installed_distributions():
    if 'site-packages' in dist.location:
        try:
            pip.call_subprocess(['pip', 'install', '-U', dist.key])
        except Exception, exc:
            print exc

The site-packages check excludes my development packages, because they are not located in the system site-packages directory. The try-except simply skips packages that have been removed from PyPI.

To endolith: I was hoping for an easy pip.install(dist.key, upgrade=True), too, but it doesn't look like pip was meant to be used by anything but the command line (the docs don't mention the internal API, and the pip developers didn't use docstrings).


On Ubuntu (and other Debian derivatives), pip apparently puts packages in /usr/local/lib/python2.7/dist-packages or similar. You could use '/usr/local/lib/' instead of 'site-packages' in the if statement in this case.
P
Pang

The shortest and easiest on Windows.

pip freeze > requirements.txt && pip install --upgrade -r requirements.txt && rm requirements.txt

@Enkouyami on windows 7 this command does not work without the -r. -r must preclude the path to the requirements file.
In what context? CMD? PowerShell? Cygwin? Anaconda? Something else?
P
Peter Mortensen

Sent through a pull-request to the pip folks; in the meantime use this pip library solution I wrote:

from pip import get_installed_distributions
from pip.commands import install

install_cmd = install.InstallCommand()

options, args = install_cmd.parse_args([package.project_name
                                        for package in
                                        get_installed_distributions()])

options.upgrade = True
install_cmd.run(options, args)  # Chuck this in a try/except and print as wanted

Version with error handling (as per comment): gist.github.com/SamuelMarks/7885f2e8e5f0562b1063
P
Peter Mortensen

This seemed to work for me...

pip install -U $(pip list --outdated | awk '{printf $1" "}')

I used printf with a space afterwards to properly separate the package names.


C
Cheung

There is not necessary to be so troublesome or install some package.

Update pip packages on Linux shell:

pip list --outdated --format=freeze | awk -F"==" '{print $1}' | xargs -i pip install -U {}

Update pip packages on Windows powershell:

pip list --outdated --format=freeze | ForEach { pip install -U $_.split("==")[0] }

Some points:

Replace pip as your python version to pip3 or pip2.

pip list --outdated to check outdated pip packages.

--format on my pip version 22.0.3 only has 3 types: columns (default), freeze, or json. freeze is better option in command pipes.

Keep command simple and usable as many systems as possible.


Thank you for the PowerShell snippet, this was the most useful answer for me!