CAFAI, le 29/08/2013 à 12:43
Over the past few months a number of people have come to me for guidance regarding various aspects of Python development, so I decided I’d post something about that.
Toolset

I like my tools simple, flexible and, above all, easy to understand and extend. Being able to poke under the hood is essential, and that assumes you don’t have to spend days poring over source code — which is a big reason why I do mostly Python, without any IDEs to hamper my judgement — all I need is vim and a terminal window, even if I do tweak it or use Sublime Text.

Since I mostly do back-end stuff and REST APIs (and then, on occasion, JavaScript front-ends atop those APIs), my current Python toolset revolves around three libraries these days:

   Bottle for request routing and templating
   Peewee as my lightweight ORM of choice
   Celery for task queueing1

The first two have small, compact codebases you can read through in one sitting and understand pretty thoroughly (or at least well enough to debug if the need arises).

Celery is in a league of its own here (it’s definitely not small), but the sheer power of it and the simplicity with which you can get a reliable, scalable and distributed task queue off the ground more than makes up for its size. It’s not something most people realize they need, but believe me, you do.

I’ve been meaning to clean up and publish my own micro-task-queue for a while (as part of my own little Python utilities library), but have yet to get around to it. Celery is overkill for doing simple batch processing, but excels at running massive workloads (hundreds of thousands of tasks an hour across dozens of CPUs), so I’m sticking with it for the time being.
(Permalink)