Python包管理

大家都知道每个项目都是由很多个不同功能的依赖库组合而成的,那么对项目而言,对这些依赖的包的管理显得尤为重要了。

什么是包依赖管理

包管理器主要是管理的集合,这样能够一次下载某一个包,而不是单独下载所有的库。

依赖关系管理用来帮助一个应用管理所有的库依赖。当你在多个环境中工作,这能够提供很大的好处。并且依赖管理能够让你注重库版本,更快更简单的更新库,当然也会处理类如一个包依赖另一个包的情况。

目前团队项目主要是Java和Python语言开发,对比发现Java有很成熟的项目构建工具,例如:有使用xml的Maven,有使用groovy的Gradle,这样的构建工具能够快速的处理项目的依赖和构建问题,那对于Python而言,在依赖管理这方面,有哪些做法呢?

当下比较流行的包依赖管理工具

  1. 在包管理方面,现在在役的包管理工具有 distutils、 setuptools、 distribute、 distlib、 pip,不过最流行的,应该还是pip了,它能够帮助开发者升级、安装、查看和卸载库。
  2. 在项目依赖隔离方面,Python主要使用虚拟环境,而关于虚拟环境的模块主要有:pyenv、 venv、 virtualenv、 virtualenvwrapper、pipenv

pip日常使用

pip常用的命令,可以是用pip help 查看:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
➜  ~ pip help

Usage:
pip <command> [options]

Commands:
install Install packages.
download Download packages.
uninstall Uninstall packages.
freeze Output installed packages in requirements format.
list List installed packages.
show Show information about installed packages.
check Verify installed packages have compatible dependencies.
config Manage local and global configuration.
search Search PyPI for packages.
wheel Build wheels from your requirements.
hash Compute hashes of package archives.
completion A helper command used for command completion.
debug Show information useful for debugging.
help Show help for commands.

主要介绍下几个常用的命令:

install : 安装包;pip install例如:pip install Flask==0.10.1(不指定版本号默认安装最新版本);升级包:pip install -U或:pip install–upgrade

download:下载包,与 install的区别就是,download只下载不安装。例如:pip download Flask,将下载Flask相关依赖到当前目录,主要用于非在线机器的包安装,后面进行介绍。

uninstall:卸载包;例如:pip uninstall Flask

show:显示包所在目录及信息;例如:pip show Flask

search:搜索包;例如:pip search Flask

requirement.txt文件使用

requirement.txt文件主要是用来记录程序中需要的第三方依赖,以方便快速重构项目环境。格式如下:

1
2
requests==1.2.0 
Flask==0.10.1

requirement.txt文件是通过pip命令生成的,具体命令为:pip freeze > requirement.txt

需要注意的是,pip freeze这个命令生成的requirement.txt文件是当前Python环境下所有类库,包括那些你没有在当前项目中使用的库。 (如果没有使用虚拟环境,后面专题讲述)。

如果只想将当前项目使用的类库导出生成为 requirements.txt,可以使用pipreqs。使用方法如下:

1
2
pip install pipreqs
pipreqs .

基于requirement.txt文件快速安装项目所需要的依赖,命令为:pip install -r requirements.txt

离线安装包

在生产环境中,往往线上主机与外网是隔离的,那直接使用pip install是无法完成的。这就需要使用前面提到的download来离线下载相关依赖包了。主要流程如下:

1、在有网络的机器,通过download命令下载离线安装包。假如需要离线安装Flask包,新建flask_offline目录,通过pip download Flask命令将Flask相关的依赖下载到该目录。可以看到Flask相关的依赖包都已经下载到flask_offline目录。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
➜  flask_offline pip download Flask
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting Flask
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/9b/93/628509b8d5dc749656a9641f4caf13540e2cdec85276964ff8f43bbb1d3b/Flask-1.1.1-py2.py3-none-any.whl (94 kB)
|████████████████████████████████| 94 kB 805 kB/s
Saved ./Flask-1.1.1-py2.py3-none-any.whl
Collecting itsdangerous>=0.24
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Saved ./itsdangerous-1.1.0-py2.py3-none-any.whl
Collecting Jinja2>=2.10.1
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/27/24/4f35961e5c669e96f6559760042a55b9bcfcdb82b9bdb3c8753dbe042e35/Jinja2-2.11.1-py2.py3-none-any.whl (126 kB)
|████████████████████████████████| 126 kB 3.0 MB/s
Saved ./Jinja2-2.11.1-py2.py3-none-any.whl
Collecting Werkzeug>=0.15
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/ba/a5/d6f8a6e71f15364d35678a4ec8a0186f980b3bd2545f40ad51dd26a87fb1/Werkzeug-1.0.0-py2.py3-none-any.whl (298 kB)
|████████████████████████████████| 298 kB 4.3 MB/s
Saved ./Werkzeug-1.0.0-py2.py3-none-any.whl
Collecting click>=5.1
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/dd/c0/4d8f43a9b16e289f36478422031b8a63b54b6ac3b1ba605d602f10dd54d6/click-7.1.1-py2.py3-none-any.whl (82 kB)
|████████████████████████████████| 82 kB 1.1 MB/s
Saved ./click-7.1.1-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/f0/00/a6aea33f5598b080b86d6b6d1214b51afe3ffa6100b902d5aa465080083f/MarkupSafe-1.1.1-cp36-cp36m-macosx_10_6_intel.whl (18 kB)
Saved ./MarkupSafe-1.1.1-cp36-cp36m-macosx_10_6_intel.whl
Successfully downloaded Flask itsdangerous Jinja2 Werkzeug click MarkupSafe
➜ flask_offline ls
Flask-1.1.1-py2.py3-none-any.whl Werkzeug-1.0.0-py2.py3-none-any.whl
Jinja2-2.11.1-py2.py3-none-any.whl click-7.1.1-py2.py3-none-any.whl
MarkupSafe-1.1.1-cp36-cp36m-macosx_10_6_intel.whl itsdangerous-1.1.0-py2.py3-none-any.whl
  1. flask_offline目录拷贝到无外网的线上机器上,执行命令:

    1
    pip install --no-index --find-links=file:/flask_offline flask

需要注意的是,线上机器和线下机器的环境需要完全一致,一样的OS架构、python版本等。那么问题来了,如果线上线下机器环境不完全一致,就没有办法了吗?当然肯定不是这样的。其中第一种做法是,修改download的参数,让他能够适配不同的环境。方法二就是使用虚拟环境。

download参数如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
Download Options:
-c, --constraint <file> Constrain versions using the given constraints file. This option can be used
multiple times.
-r, --requirement <file> Install from the given requirements file. This option can be used multiple times.
-b, --build <dir> Directory to unpack packages into and build in. Note that an initial build still
takes place in a temporary directory. The location of temporary directories can
be controlled by setting the TMPDIR environment variable (TEMP on Windows)
appropriately. When passed, build directories are not cleaned in case of
failures.
--no-deps Don't install package dependencies.
--global-option <options> Extra global options to be supplied to the setup.py call before the install
command.
--no-binary <format_control>
Do not use binary packages. Can be supplied multiple times, and each time adds to
the existing value. Accepts either :all: to disable all binary packages, :none:
to empty the set, or one or more package names with commas between them (no
colons). Note that some packages are tricky to compile and may fail to install
when this option is used on them.
--only-binary <format_control>
Do not use source packages. Can be supplied multiple times, and each time adds to
the existing value. Accepts either :all: to disable all source packages, :none:
to empty the set, or one or more package names with commas between them. Packages
without binary distributions will fail to install when this option is used on
them.
--prefer-binary Prefer older binary packages over newer source packages.
--src <dir> Directory to check out editable projects into. The default in a virtualenv is
"<venv path>/src". The default for global installs is "<current dir>/src".
--pre Include pre-release and development versions. By default, pip only finds stable
versions.
--no-clean Don't clean up build directories.
--require-hashes Require a hash to check each requirement against, for repeatable installs. This
option is implied when any package in a requirements file has a --hash option.
--progress-bar <progress_bar>
Specify type of progress to be displayed [off|on|ascii|pretty|emoji] (default:
on)
--no-build-isolation Disable isolation when building a modern source distribution. Build dependencies
specified by PEP 518 must be already installed if this option is used.
--use-pep517 Use PEP 517 for building source distributions (use --no-use-pep517 to force
legacy behaviour).
-d, --dest <dir> Download packages into <dir>.
--platform <platform> Only use wheels compatible with <platform>. Defaults to the platform of the
running system.
--python-version <python_version>
The Python interpreter version to use for wheel and "Requires-Python"
compatibility checks. Defaults to a version derived from the running interpreter.
The version can be specified using up to three dot-separated integers (e.g. "3"
for 3.0.0, "3.7" for 3.7.0, or "3.7.3"). A major-minor version can also be given
as a string without dots (e.g. "37" for 3.7.0).
--implementation <implementation>
Only use wheels compatible with Python implementation <implementation>, e.g.
'pp', 'jy', 'cp', or 'ip'. If not specified, then the current interpreter
implementation is used. Use 'py' to force implementation-agnostic wheels.
--abi <abi> Only use wheels compatible with Python abi <abi>, e.g. 'pypy_41'. If not
specified, then the current interpreter abi tag is used. Generally you will need
to specify --implementation, --platform, and --python-version when using this
option.

常用的有如下参数:

1
2
3
4
--only-binary=:all: \ # 只下载二进制package(即wheel或egg)
--platform linux_x86_64 \ # 指定linux架构为linux 64位架构
--python-version 27 \ # 指定python版本为Python 2.7
--abi cp27mu
感谢您的阅读,本文由 vic的博客 版权所有。如若转载,请注明出处:vic的博客(http://fuyong.net.cn/2020/03/01/Python包管理/
Intellij IDEA快捷键介绍-常用篇
代码审查流程