PythonModuleInstallation

From NJIT-ARCS HPC Wiki
Jump to: navigation, search

ARCS supports Anaconda Python for both Python 2.x and Python 3.x. The default Anaconda installation comes with most of the modules researches and students need to accomplish their tasks. If additional modules are needed send an email to arcs@njit.edu . If Anacaconda supports the module ARCS will install the module for you. In the event that Anaconda does not have the module available or you prefer to maintain your own python anvironment there are two methods you can use. You do NOT need root permissions to do this.

Install the module in your home directory

The following is an example of installing the "stemming" module in the user home directory without root privileges using "pip":

kong-43 ~ >: pip install --user stemming
Collecting stemming
Installing collected packages: stemming
Successfully installed stemming
kong-44 ~ >: python
Python 2.7.11 |Anaconda 4.0.0 (64-bit)| (default, Dec  6 2015, 18:08:32)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import stemming
>>>

The --user switch tells the pip command to install the module in ~/.local/lib/pythonX.Y/site-packages. The directory will be created if it doesn't exist.

Virtualenv

Virtualenv allows a user to maintain a virtual python environment where the user can install packages as needed.

This example uses the installed Intel Python3

band ~ > mkdir pythonapps
band ~ >cd pythonapps
band pythonapps >: virtualenv testenv
Using base prefix '/afs/cad.njit.edu/linux/intel/intelpython3'
New python executable in /afs/cad.njit.edu/u/g/w/gwolosh/projects/pythonapps/testenv/bin/python
copying /afs/cad.njit.edu/linux/intel/intelpython3/bin/python => /afs/cad.njit.edu/u/g/w/gwolosh/projects/pythonapps/testenv/bin/python
copying /afs/cad.njit.edu/linux/intel/intelpython3/bin/../lib/libpython3.6m.so.1.0 => /afs/cad.njit.edu/u/g/w/gwolosh/projects/pythonapps/testenv/lib/libpython3.6m.so.1.0
Installing setuptools, pip, wheel...done.
band pythonapps >: source testenv/bin/activate
(testenv) band pythonapps >: python
Python 3.6.3 |Intel Corporation| (default, Feb 12 2018, 06:37:09)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
(testenv) band-73 pythonapps >: pip install numpy
Collecting numpy
  Downloading https://files.pythonhosted.org/packages/71/90/ca61e203e0080a8cef7ac21eca199829fa8d997f7c4da3e985b49d0a107d/numpy-1.14.3-cp36-cp36m-manylinux1_x86_64.whl (12.2MB)
    100% |████████████████████████████████| 12.2MB 137kB/s
Installing collected packages: numpy
Successfully installed numpy-1.14.3
(testenv) band pythonapps >: python
Python 3.6.3 |Intel Corporation| (default, Feb 12 2018, 06:37:09)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import numpy
>>> exit()

To leave the virtualenv

(testenv) band pythonapps >: deactivate
band pythonapps >: