Python PyPi mirror in virtualenv (bandersnatch)
First of all PEP381 is no longer supported and won't work. Use bandersnatch instead.
For customized mirrors with whitelists, etc use devpi.
Example:
Python 2.7
Virtualenv path: /data/pypi_mirror/
Mirror path: /data/pypi_mirror/data/
Url: local-pypi.local.com
For customized mirrors with whitelists, etc use devpi.
Example:
Python 2.7
Virtualenv path: /data/pypi_mirror/
Mirror path: /data/pypi_mirror/data/
Url: local-pypi.local.com
Create and activate virtualenv
cd /data/
mkdir /pypi_mirror
mkdir /pypi_mirror/data
virtualenv pypi_mirror
source ./pypi_mirror/bin/activate
Install packages
sudo pip install bandersnatch
sudo pip install configparser
sudo pip install --upgrade future
sudo pip install --upgrade websocket
sudo pip install --upgrade urllib3
Allow Apache to access files:
cd /pypi_mirror
sudo chown -R apache:apache data
Configure Bandersnatch:
sudo vi /etc/bandersnatch.conf
[mirror]
directory = /data/pypi_mirror/data/
master = https://pypi.python.org
timeout = 10
workers = 3
stop-on-error = false
timeout = 10
workers = 3
stop-on-error = false
delete-packages = true
Enable Virtual Host in Apache:
Create virtual host:
cd /etc/httpd/sites-available/
vi webapps_pypi.conf :
NameVirtualHost <SERVER IP>:80
<VirtualHost <SERVER IP>:443>
SSLEngine on
SSLCertificateFile *****
SSLCertificateKeyFile *****
SSLProtocol all -SSLv3
ServerName local-pypi.local.com
ServerAlias local-pypi
DocumentRoot /data/pypi_mirror/data/web/
SetEnv /data/pypi_mirror/data/
ErrorLog /data/pypi_mirror/pypi.error.log
CustomLog /data/pypi_mirror/pypi.custom.log combined
</VirtualHost>
<VirtualHost <SERVER IP>:80>
ServerName local-pypi.local.com
DocumentRoot /data/pypi_mirror/data/web/
Redirect permanent / https://local-pypi.local.com/
</VirtualHost>
Enable the Virtual Host and restart Apache:
cd /etc/httpd/sites-enabled/
sudo ln -s ../sites-available/webapps_pypi.conf webapps_pypi.conf
sudo /usr/sbin/apachectl restart
Run mirror synchronization
sudo bandersnatch mirror
Execution Errors:
Cannot find parse module in urllib.parse
sudo vi /lib/python2.7/site-packages/bandersnatch/package.py
Replace line:
from urllib.parse import urlparse, unquote
With:
try:
from urllib.parse import urlparse, unquote
except ImportError:
from urlparse import urlparse, unquote
Errors encoding parameter not supported:
sudo vi /lib/python2.7/site-packages/bandersnatch/mirror.py
Search for:
with open(
Remove parameter:
*, encoding="..." )