Pip, buildout and TDD
pip just works, but you need buildout
Many pip supporters argue that "pip just works" so they "don't need buildout and its complexity". From my own experience, buildout's workflow is safer than pip's one, so I actually need buildout. And I don't recommend pip when you come to share work, which is the case in a team or in an open-source project.
pip: script afterwards
Let's consider a common workflow with pip:
# Suppose we are using virtualenv. virtualenv --distribute --no-site-packages demo source demo/bin/activate # Install some dependencies. pip install django pip install gunicorn supervisord pip install sphinx # If your "myproject" Django project is an egg: python myproject/setup.py develop # Deployment is not only a matter of dependencies. mkdir -p docs/_static mkdir -p var/log mkdir -p var/run mkdir -p var/upload mkdir -p etc/ touch etc/supervisord.conf # Check deployment success. python myproject/manage.py test # Run some tests. # If it doesn't work, go back and perform some manual tasks to fix the # deployment. # Else, it works. Let's share our work! pip freeze > requirements.txt # Did I performed other actions? history > docs/INSTALL.txt # Ok, let's actually share our work now! hg commit requirements.txt docs/INSTALL.txt # or git ;)
- Remember what we did
- Commit and share what we remembered (not what we did actually)
And say "it just works". Fine!
In fact, we'd rather say "it just worked for me"...
To be sure, let's run it again in another place... or wait a colleague tries it (hope you will be online if he needs support).
Buildout: start with configuration
Let's consider a common workflow with buildout:
# Configure deployment. vim buildout.cfg # Deploy. bin/buildout # Test. bin/django test # If it doesn't work, edit buildout.cfg and retry. # Else, it worked for me. It will work for others too. Let's share our work. echo "python bootstrap.py -d && bin/buildout" > docs/INSTALL.txt hg commit buildout.cfg docs/INSTALL.txt
Don't believe that the sequence is shorter: work performed in buildout.cfg is not shown here.
Your work is de facto repeatable: your work is the configuration file, just share it.
In fact, we didn't write a script, we reused existing (maintained) scripts and configured them!
Remember what is test driven development?
- Write tests
- Run tests and see them fail
- Tests pass.
- Be happy!
When you write tests after implementation, you may forget some checks. In fact, in most cases, you don't write tests after implementation...
TDD is a best practice. Let's make a parallel between tests (in a development scope) and procedures (in a deployment scope). In short, it's a matter of repeatability: tests assert that product functionalities or behaviour are reproduced, and procedures are used to reproduce the product...
With pip, we can write procedures first: start with the edition of requirements.txt file and then run "pip install -r requirements.txt". Who does it? Most pip tutorials start with pip install and then talk about pip freeze.
With buildout, we have to write procedures first. Everybody does it.
Pip only covers Python dependencies, whereas buildout can cover more... but this is another story.
Yes, you'd better use "fabric + pip" than only pip.
But don't reinvent the wheel! I mean buildout is here, it works, it is extensible. It already has many useful recipes and extensions.
So I would recommend using "fabric + buildout" instead of "fabric + pip":
- buildout is really good at deploying Python stuff. Use it!
- buildout can deploy most non-Python stuff as normal user via recipes. Use it or not depending on your team's preferences. Here is where interactions between sysadmins and developers are required.
- fabric or any remote control utility can run buildout to manage, at least, python stuff.
- fabric or any remote control utility are better than buildout to run tasks as power user (sudoer).
- fabric or any remote control utility are not limited to deployment. They are handy to trigger many other tasks, such as managing services (via tools like supervisord), performing upgrades (via tools like south)...
Tests matter. Deployment procedures too.
Yes, pip "just works". But it's like writing tests after implementation. You usually don't write deployment procedures or forget some steps. So repeatability is not guaranteed.
Buildout's design makes you produce repeatable deployment from the start. Every team should appreciate it.
Fabric and other "remote control utilities" are great, they can run many tasks, including buildout.