Traits in Python using multiple inheritance

As you probably know, if a class needs to handle two mostly unrelated concerns, the class should probably be split into two. This way, your will achieve higher cohesion in your code, which is generally considered a good thing. Though, there are times where you need to address two mostly unrelated concerns in a single class.

The Scala programming language has a concept they call traits. Scala traits can be compared to Java interfaces, except that Scala traits may be partially or fully implemented, like abstract classes. As with Java interfaces, you can mixin multiple traits in a class. If any of the traits has colliding method signatures, the last trait mixed in overrides the earlier ones. Just as with regular inheritance and interfaces, the class must implement any methods that has not been implemented by the traits, and may override any methods that already has been implemented.

trait Similarity {
  def isSimilar(x: Any): Boolean
  def isNotSimilar(x: Any): Boolean = !isSimilar(x)
}

class Point(xc: Int, yc: Int) extends Similarity {
  var x: Int = xc
  var y: Int = yc
  def isSimilar(obj: Any) =
    obj.isInstanceOf[Point] &&
    obj.asInstanceOf[Point].x == x
}

Traits as a concept for separating reusable parts of your code is not new. Like most good ideas, someone has been thinking about them previously. According to Wikipedia traits originated in the Self programming language from 1987. Today, multiple programming languages has variants of traits as native constructs in the language. One example being Scala traits, another being module mixins in Ruby.

Given almost 25 years of history, one should think traits should be well known by now and more widely used for achieving good and reusable code. My first meeting with traits wasn’t until during an otherwise unrelated course in late 2009 where Jim Coplien introduced me to the DCI architecture during lunch, and then again a bit later when I picked up the Scala language. I ask myself: Why didn’t I learn stuff like this in university?

Example: Mixing in the actor life cycle in Mopidy

Back to Python. I’m currently introducing Pykka’s actor model implementation in the Mopidy music server as a replacement for Mopidy’s existing thread management and inter-thread communication code. I’m replacing untested application wiring–some reused throughout the application, some custom for specific problems–with a fully tested library. Using the actor model I apply the same simple semantics to all concurrent components in the application, making it simpler to reason about, at least compared to having custom solutions all over the place. As an added bonus, the current git diff --stat is at about -700/+300, leaving 400 fewer lines of code to maintain.

Mopidy has several extension points where one can add new components such as music library backends, audio outputs, audio mixers, and frontend servers. When you implement an extension, you must subclass the corresponding base class, e.g. the BaseOutput class, which defines and documents the API that the extension must implement for it to be usable by the rest of the system.

Many of these components interface with external libraries or systems, but we can’t block the entire system while waiting for these interactions to complete. We want to be able to respond to requests from MPD clients while we adjust the volume on the NAD amplifier, or queue the next block of audio data for playback. Thus, the extensions need to either run in its own thread, or to dispatch work to a worker thread.

Clearly, the components got two separate sets of concerns. First of all, they have some application logic which implements the API. E.g. methods like play_uri() and get_volume(). Secondly, the components got a life cycle, consiting of methods like start() and stop().

An example from Mopidy is the Last.fm scrobbler, which previously was split in two; a main class which implements the frontend API, and a worker class, which is a thread which contains most of the application logic.

import multiprocessing

from mopidy.frontends.base import BaseFrontend
from mopidy.utils.process import BaseThread

class LastfmFrontend(BaseFrontend):
    def __init__(self, *args, **kwargs):
        super(LastfmFrontend, self).__init__(*args, **kwargs)
        (self.connection, other_end) = multiprocessing.Pipe()
        self.thread = LastfmFrontendThread(self.core_queue, other_end)

    def start(self):
        self.thread.start()

    def destroy(self):
        self.thread.destroy()

    def process_message(self, message):
        if self.thread.is_alive():
            self.connection.send(message)

class LastfmFrontendThread(BaseThread):
    def __init__(self, core_queue, connection):
        # ...

    def run_inside_try(self):
        # ...

    def setup(self):
        # ...

    def process_message(self, message):
        # ...

    def started_playing(self, track):
        # ...

    def stopped_playing(self, track, stop_position):
        # ...

After refactoring the code is down to a single class just implementing core application logic, which no library can replace. The trick? Using multiple inheritance to mix in the actor life cycle.

from pykka.actor import ThreadingActor

from mopidy.frontends.base import BaseFrontend

class LastfmFrontend(ThreadingActor, BaseFrontend):
    def __init__(self):
        # ...

    def post_start(self):
        # ...

    def react(self, message):
        # ...

    def started_playing(self, track):
        # ...

    def stopped_playing(self, track, stop_position):
        # ...

I think this is a beautiful way of separating concerns that needs to be part of the same class. I also think the code reads well:

The above example is an frontend extension, which has the trait of also being an actor. It has some application logic, but also happens to have an actor life cycle.

Disclaimer: Before refactoring all your code to use multiple inheritance in Python, get familiar with Python’s Method Resolution Order (MRO) and the semantics of super() when using multiple inheritance.

Recent posts

Archive of all posts