November 15, 2017

Creating python package using pbr

WORDS BY   Tadej Borovšak

POSTED IN   python | pbr | tutorial


In this post, we will learn how to create and maintain a python package with a little help from OpenStack's pbr package.

Why on earth would I want to write a python package?

Most of the time, when we decide that this world needs more python code, we want to make that code easily accessible to us and other people. And currently, creating python package and hosting it on PyPI seems to fit the bill nicely, since instructing people to run

$ virtualenv venv && pip install thebestpackageever

is way nicer than adding detailed instructions on how to obtain the code and where to put it so the interpreter will find it. Knowing all this stuff is still useful and somewhat expected from the users that decided to use our library in their product, but lowering the barrier for newcomers is equally important or no one will give our library a chance.

With this being out of the way, we can have a look at the pbr helper package that is the main focus of this post.

What is pbr?

The name of the package stands for Python Build Reasonabless and provides some reasonable defaults to the python package creation and maintenance process, like auto-generating version numbers from git commit history, creating manifest file and finding python packages. Because of this opinionated approach to package maintenance, experts might find it a bit limiting, but then they should be writing blog posts on advanced topics and not reading this entry-level text.

While it would be possible to just talk about how pbr makes maintaining a package easy and smooth, we will move on and actually create a sample package that uses pbr.

Writing new python library

First, we will create new, empty folder that will hold all of our code and initialize empty git repository within it. Why new folder? Because when things go haywire and we start feeling depressed, running rm -r folder feels unnervingly soothing. But back to our library.

Next, we must name our library. We will call it fana, because no package with this name exists on PyPI (and we have no imagination whatsoever).

Commands that we should run are

$ mkdir -p ~/fana
$ cd ~/fana
$ git init

Next, we need to create a README.rst file that describes the purpose of our library and installation instructions. Adding a LICENSE file is also a must or no one will dare to use our library in production.

$ cat <<EOF > README.rst
Farm animal noise assistant
===========================

Ever wondered what noise does a certain farm animal make and how would
this look transcribed to ASCII? We neither, but that did not stop us from
creating this library that features over two farm animals.
EOF
$ wget https://www.apache.org/licenses/LICENSE-2.0.txt -O LICENSE

And with all this being done, we are ready to create our first commit.

$ git add README.rst LICENSE
$ git commit -m "Initial commit"

The hardest part of the project is now behind us and we can focus on writing some actual source code. The following content should be placed inside the fana/fana.py file:

ANIMALS = {
    "cat": "MEOW",
    "cow": "MOO",
    "dog": "WOOF",
}


class NoSuchAnimal(Exception):
    pass


def list_animals():
    return ANIMALS.keys()


def make_a_noise(animal):
    try:
        return ANIMALS[animal]
    except KeyError:
        raise NoSuchAnimal(animal)

In order to make our fana folder a real python package, we also need to add __init__.py file by running

$ touch fana/__init__.py

After we commit our changes, we can move on and convert our simple library to python package.

$ git add fana
$ git commit -m "Add initial fana library implementation"

Creating python package

Now that we have our library all set, we need to add setup.py and setup.cfg files that describe our package. setup.py is simpler of the two, since it only contains one function call:

from setuptools import setup

setup(setup_requires=["pbr"], pbr=True)

Awesome. The setup.cfg file contains all of the metadata about our package that will be injected into setup call by the pbr. In our simple case, this file looks like this:

[metadata]
name = fana
author = XLAB d.o.o.
author-email = pypi@xlab.si
summary = Farm animal noise assistant
description-file = README.rst
home-page = https://github.com/xlab-si/fana
license = Apache-2
classifier =
    Development Status :: 4 - Beta
    Intended Audience :: Developers
    Intended Audience :: Information Technology
    License :: OSI Approved :: Apache Software License
    Operating System :: OS Independent
    Programming Language :: Python
keywords =
    tutorial

[files]
packages =
    fana

Most of the entries in the metadata section should be self-explanatory, with possible exception of classifiers. This key should contain a list of Trove classifiers that are applicable to the project we are creating.

Files section should at minimum contain packages key, since this is used by the pbr to find python modules that should be packaged. Names of the packages should match the names of the folders that packages reside in. In our simple case, there is only one package that resides in folder fana.

We commit those two files and prepare ourselves for the big moment we have all been waiting for: the package creation.

$ git add setup.{py,cfg}
$ git commit -m "Add setup files"
$ python setup.py sdist
running sdist
[pbr] Writing ChangeLog
[pbr] Generating ChangeLog
...
Writing fana-0.0.1.dev2/setup.cfg
Creating tar archive
removing 'fana-0.0.1.dev2' (and everything under it)

We can see that we just created package version 0.0.1.dev2. How did pbr came up with this version? Well, we have three commits in the repository. Initial one set the version to 0.0.1, and the other two bumped it to dev2. In order to create a release with version number that we selected, we need to tag the commit.

$ git tag -am "Version 0.1.0" 0.1.0

If we now package our library again, we get

$ python setup.py sdist
running sdist
[pbr] Writing ChangeLog
[pbr] Generating ChangeLog
...
Writing fana-0.1.0/setup.cfg
Creating tar archive
removing 'fana-0.1.0' (and everything under it)

Now we can actually install out package into virtual environment.

Testing prepared package

In order to safely test the package, we will use virtualenv to create isolated python environment and install our package inside it.

$ cd /tmp
$ virtualenv venv
$ . venv/bin/activate
(venv) $ pip install ~/fana/dist/fana-0.1.0.tar.gz

To actually test the functionality of the package, we can execute commands directly from terminal:

$ python -c 'from fana import fana; print(fana.list_animals())'

or we can write a wrapper.py application

import sys

from fana import fana

print(fana.make_a_noise(sys.argv[1]))

that can be then executed like

$ python wrapper.py cow
MOO
$ python wrapper.py sheep
Traceback (most recent call last):
  File "wrapper.py", line 5, in <module>
    print(fana.make_a_noise(sys.argv[1]))
  File "/tmp/venv/lib/python2.7/site-packages/fana/fana.py", line 20, in make_a_noise
    raise NoSuchAnimal(animal)
fana.fana.NoSuchAnimal: sheep

And this exception concludes today's post. All of the code is available from accompanying Github repository.