Setting up Python development environment

Posted on Thu, 31 July 2014 in development

I use Python for all kinds of data munging and analytical (e.g. Machine learning) tasks, and generating this site with Pelican.

OS X comes with preinstalled Python, which in latest version 10.9 Mavericks is Python 2.7.5. We are all good to go then? Well, not quite on my opinion. I like to have a development environment, which is easy to maintain ja easily configurable, and preferably up-to-date.

Prerequisites

Your operating system should be OS X 10.9 Mavericks, with Xcode installed (we need the command line tools). You can download and install Xcode from Mac App Store if needed. Preinstalled Ruby is also needed for the next step.

Bash profile setup

We need to edit the PATH definition in bash profile in order to make sure, that stuff installed via Homebrew will take precedence over preinstalled stuff. Open Terminal and let’s start (Note: I have created a symlink sublime which enables me to open my favorite text editor Sublime Text from Terminal).

# Navigate to your home directory
cd

# List files to see, if the .bash_profile already exists
ls -a

# If not, you can create it by
touch ~/.bash_profile

# Open .bash_profile in editor (some might even use Vim!)
sublime ~/.bash_profile

Add the following to your .bash_profile and save it.

# Ensure user-installed binaries take precedence
export PATH=/usr/local/bin:$PATH

# Load .bashrc if it exists
test -f ~/.bashrc && source ~/.bashrc

Since these directives take effect only on the next login, let’s source the file to make them count in the current session. In Terminal type

. ~/.bash_profile

There we go, all set!

Installing Homebrew

Homebrew is extremely handy package manager for OS X, which enables you to install, update and uninstall vast selection of mostly command-line tools by using its formulae. Definitely a must have!

In Terminal run the command

ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

Follow the instructions the script is giving and enter your administror’s password when asked. After the installation is complete, let’s see that everything is in working order. Run the following command

brew doctor

If all is good, let’s make sure we are running the latest versions. This command will update Homebrew itself and all the formulae.

brew update

Now you can check if tehere are any outdated installations

brew outdated

If there are, and you want to update them, run

brew upgrade

# If you want to update just a specific install, define formula, e.g.
brew upgrade python

Excellent, we are now ready to move to our next phase!

Installing Python

Python has two active versions Python 2 and Python 3, latest versions being 2.7.8 and 3.4.1 respectively at the time of writing. You can read more on the versions from official Python.org wiki. To install run the following Homebrew formula in Terminal

brew install python

# If you want to install Python 3, run
brew install python3

You can actually install both, if you need to.

These Homebrew formulae contain necessary Python package management tools Pip and Setuptools, out of which I like to use Pip. We can now start to install Python packages. Let’s start with the most important.

Installing and setting up virtualenv

Virtualenv is a tool to create isolated Python environments. With virtualenv you can have numerous differently configured development environments. Need for this may arise for example from a need to have different version of certain package, or completely different package configuraion. Virtualenv makes this very easy.

The following command installs virtualenv globally

pip install virtualenv

Let’s create some work directories

# I have directory called Devs in my home directory for all the dev stuff, lets go in there
cd Devs

# Directory for our virtualenvs
mkdir -p ~/Virtualenvs

# Diretory for our Python projects
mkdir -p ~/PythonProjects

Let’s create our first virtual environment, named demo

# Lets go to correct directory
cd ~/Devs/Virtualenvs

virtualenv demo

# To create Python 3 based environment
virtualenv -p python3 demo

Making life easier with Pip restrictions

Managing Python packages with many virtual environments can become a pain. When you accidentally install a package globally instead of in virtual environment, you can find yourself in interesting debug situations. Luckily this can be easily fixed, restrictions to the rescue!

We can easily add a requirement that we are in a active virtualenv in order to use Pip. To install packages globally, can be enabled just as easily. Let’s create (if necessary) and edit .bashrc in our home directory.

# Move to home directory
cd

# If file does not exist, create with
touch ~/.bashrc

# Open .bashrc in editor
sublime ~/.bashrc

Add the following to .bashrc and save it.

# pip should only run if there is a virtualenv currently activated
export PIP_REQUIRE_VIRTUALENV=true
# cache pip-installed packages to avoid re-downloading
export PIP_DOWNLOAD_CACHE=$HOME/.pip/cache

syspip(){
   PIP_REQUIRE_VIRTUALENV="" pip "$@"
}

And to make things count in current session

. ~/.bash_profile

Now, when we want to install package to specific virtualenv, we need to be in that active virtualenv and run Pip. To activate virtualenv and install some stuff

# Go to directory containing the desired virtualenv
cd ~/Devs/Virtualenvs/demo

# Activate the environment
. bin/activate

# Install some cool packages (pelican and markdown)
pip install pelican markdown

When we want to install package globally, we use command syspip instead, outside any active virtualenv.

# If you are in active virtualenv, deactivate it before installation.
deactivate

# Install some global package
syspip install markdown

Managing virtualenvs

As already described abowe, you can create new virtual environments with

# Lets go to correct directory
cd ~/Devs/Virtualenvs

virtualenv major-script-project

And to activate it

# Go to directory containing the desired virtualenv
cd ~/Devs/Virtualenvs/major-script-project

# Activate the environment
. bin/activate

To deactivate an environment

deactivate

And to delete a virtual environment, simply deactivate it first, and then you can simply delete the appropriate directory (e.g. ~/Devs/Virtualenvs/major-script-project).

Final thoughts

Setting up Python with Homebrew and using virtualenv and Pip takes a few steps to complete and some time to get familiar with. Once you get your head around it all, your life as a Python developer gets a lot of easier. By the way, with Homebrew you can install a lot more, than just Python, see Homebrew GitHub list.

By no means is this a complete or perfect development environment. As a first addition, I recommend using version control system, for example Git, which you can install via Homebrew, of course!