Virtual Environments Top Tips

1160562_digital_dreams

Virtual Environments are a tool for isolating and tracking the software packages that you use for your work. Why is this useful to you?  Let Research Software Engineers Douglas Lowe and Ann Gledson explain!

Even if you’re new to Python (or other programming languages), there’s a good chance you’ll be aware of the importance of using libraries and packages. You might have also come across the ‘laboratory equipment analogy’, which compares using packages in your application or script with a scientist installing only the equipment in their lab that is required for each particular experiment, to avoid things getting cluttered and over complicated. The Software Carpentry have an excellent tutorial about this.

If your code relies on particular versions of libraries or packages, then it is useful to be able to record your environment setup, and enable others to create the same setup, when re-using your code.

Back to the laboratory analogy: what if other researchers wish to replicate the scientist’s experiments in other labs, despite recent equipment upgrades and improvements? For successful replication, our scientist would need to provide all equipment/versions used, along with any other important experimental conditions. That is, for each experiment (or Python application/script in our case), the full environment must be documented. Thankfully, with Python the ability to create and share such environments is baked in, using virtual environments (venv) and pip.

Virtual environments are also important for isolating your code from your system version of Python. The operating system that you work on will, by necessity, usually be using an older (and more stable) version of Python – which tends to only get updated for security reasons, not feature updates. If you want to use the latest features of a language such as Python, or other libraries, then it is useful to be able to use virtual environments to isolate these from the system libraries.

So how do we know what packages are available and how do we install, upgrade or uninstall them?

pip

One way to add, upgrade or remove packages into your virtual environments is by using pip, a built-in and easy to use Python package management program. Full instructions on how to install pip (if not already included as part of your Python install) are in  https://pip.pypa.io/en/stable/installing/. Then to find out how to to use pip, see: https://docs.python.org/3/tutorial/venv.html#managing-packages-with-pip

conda

The above functionality has proved so useful that it has been extended for packages which are not pure Python (or, indeed, are not Python based at all), using the conda environment manager, and conda-forge software channel. For example, there are packages available for other languages (e.g. R), as well as libraries (e.g. HDF5 and BLAS), and complete software programs (e.g. prismatic, ncview).

Instructions for installing conda are available as are instructions for working with the conda environment manager.

To get the most out of conda it is recommended to use the conda-forge channel, which contains a lot of community-contributed packages.

Further reading

venv + pip

Conda

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s