The importance of running tests quickly

Here, MAAS, and everywhere

MAAS’s inception date was 16th January 2012 and it has been continually developed ever since, including the development of many, many unit tests. Since almost the beginning we’ve had a landing robot that runs those unit tests before merging a new branch into trunk.

At the time of writing it runs 14337 tests, and that number grows daily. Until recently the landing robot would take over an hour to test and merge each branch.

This is too slow — and I’ll explain why I think this — and this is how my journey to fix it began.

[Read More]

Bazaar repositories for fun/profit/shenanigans

Save time and disk with Bazaar's shared repositories

Bazaar can support you whether you like the Git model of having a single working tree for each clone of your repository, or if you prefer to have multiple working trees.

When dealing with large projects the latter can get slow and disk hungry. This is because, by default, each new working tree created by bzr branch a-branch new-branch also holds a complete copy of the repository history.

I use a mix of both development models when I’m using Bazaar. Fortunately there’s an easy and out-of-the-box way to prevent those slow downs and get back your disk space: shared repositories. Read on to find out how.

[Read More]
MAAS  Python 

Porting MAAS to Python 3

MAAS, up to and including version 1.9, is a Python 2.7 application. We have wanted to move to Python 3 for a long time but it hasn’t been feasible. Recently the wind changed and we decided to port MAAS to Python 3.5 for MAAS’s 2.0 release.

[Read More]
MAAS  Python 

Transactions in MAAS

... and how Django gets it so wrong

In MAAS, to ensure that a function is run within its own database transaction, decorate it with @transactional:

from maasserver.utils.orm import transactional

@transactional
def do_something_databasey():
    ...

If a transaction is already in progress when do_something_databasey is called, it will instead be called within a savepoint.

That’s it.

Now for the why.

[Read More]
MAAS  Python 

Introduction to blocking and non-blocking code in MAAS

MAAS has a couple of function decorators that are designed to help blocking code work with non-blocking code: synchronous and asynchronous.

The blocking, or synchronous, code in MAAS is primarily though not solely the realm of Django, which handles all web API calls and some web views. The biggest responsibility of Django these days is the ORM and database migrations.

Django doesn’t do non-blocking, or asynchronous, at all. For that MAAS uses Twisted. Twisted has many useful pieces that can be used in other projects, but most of the time the Twisted reactor is what you want.

[Read More]
MAAS  Python 

South, South 2, and Django Migrations

A couple of months ago we on the MAAS team found ourselves in a bit of a pickle: we needed to be able to support a product targeted at both Django <1.7 and Django ≥1.7 with database migrations. This is a problem because South is replaced by Django’s own migration support in 1.7, and there are differences.

[Read More]
Django  MAAS 

Preparing for Python 3 in MAAS

Something we’ve done in MAAS — which is Python 2 only so far — is to put:

from __future__ import (
    absolute_import,
    print_function,
    unicode_literals,
)

__metaclass__ = type

str = None

at the top of every source file. We knew that we would port MAAS to Python 3 at some point, and we hoped that doing this would help that effort. We’ll find out if that’s the case soon enough: one of our goals for this cycle is to port MAAS to Python 3.

[Read More]