"top" in an animated GIF

I noticed a lot of animated GIFs showing up in my Google+ feed recently.  Mainly it was cats.

I thought, why not render "top" output as an animated GIF?  It's not the most practical thing, but it seemed like a fun challenge to see if I could do this with minimal python coding.

The Result!

A 770kb GIF showing about 12 seconds of top output.  You can see pretty easily that I have a misbehaving chrome tab, chewing up an entire CPU core.  I added the green pie-chart thingy to give a visual indication of position within the animation, since it loops 10 times and would be confusing.

The Code

#!/usr/bin/python

from PIL import Image, ImageDraw, ImageFont, ImageSequence
from images2gif import writeGif
import os, subprocess, sys, time

FRAMES = 12
FRAME_DELAY = 0.75
WIDTH, HEIGHT = 650, 300
PIE_POS = (WIDTH-50,10, WIDTH-10,50)
FONT = ImageFont.truetype('/usr/share/fonts/liberation/LiberationMono-Regular.ttf', 12)

def make_frame(txt, count, font=FONT):
  image = Image.new("RGBA", (WIDTH, HEIGHT), (255,255,255))
  draw = ImageDraw.Draw(image)
  fontsize = font.getsize('')[1]
  for row, line in enumerate(txt.split('\n')):
    draw.text((5, fontsize * row), line, (0,0,0), font=font)
  draw.pieslice(PIE_POS, 0, 360, (255,255,204))
  draw.pieslice(PIE_POS, 0, int(360.0/FRAMES*(1+count)), (0,128,0))
  return image


frames = []
for count in range(FRAMES):
  txt = subprocess.Popen('top -c -n 1 -b'.split(), stdout=subprocess.PIPE).stdout.read()
  frames.append(make_frame(txt, count))
  time.sleep(FRAME_DELAY)

writeGif("topmovie.gif", frames, duration=FRAME_DELAY, loops=10, dither=0)

The Libraries: PIL and images2gif

I run Fedora (18) which provides the PIL library as an RPM named python-imaging, so you can install that easily:

  sudo yum install python-imaging

The PIL modules provide the basic drawing and canvas processing features, and the 'images2gif.py' module works on top of PIL to implement the animated GIF standard.  

The images2gif.py library is available for download here: http://bit.ly/XMMn5h

The images2gif.py code looked like it could use some cleanup, and I didn't need its support for numpy arrays, so a slightly smaller cleaned up version is also here: http://pastebin.com/1xDHnWFK

How It Works

The code uses subprocess to run 'top' once per frame interval, and builds a new frame by calling the draw.text() method for each line in the output.  I was a little surprised that PIL's text() method doesn't handle newlines, but it was easy to implement using split() and enumerate() on the text, and multiplying the line number by the font height to get the next offset.  All the frames are collected into a list and passed to the writeGif() method from images2gif which outputs the special header and per-frame extension blocks needed to animate the GIF.

The pie slice rendering is done by first drawing a full circle and then a partial pie with the filled-in degrees calculated from the current frame number.

This code can be improved a lot, but as an example to work from, it's small and should be easy to hack on.

Multiple event loops with asyncore and asynchat.async_chat and threads

I recently needed to support multiple concurrent event loops with asyncore, and found a version specific obstacle in asynchat and came up with a simple and clean workaround.


I was developing a protocol client library that needed to be asynchronous.  My code used asyncore and asynchat, and worked in a single-threaded environment.  But then my needs evolved and it now needed to support execution from multiple threads.  Unfortunately, the asyncore and asynchat modules in the Python Standard Library are not thread-safe.  When two threads enter the main event loop in asyncore.loop() they collide spectacularly, causing spurious IO exceptions and corrupting asyncore's internal data structures.

I read the asyncore.py code and saw why it wasn't thread-safe.  The way the main event loop in asyncore.loop() works is that a module-scoped dict() named socket_map is used to store the sockets that are then used with select() or poll().  Without getting into the deep details of how asyncore works, it's enough to note that asyncore's use of it's own private socket_map dict() is the source of the thread-safety problems.  When two threads both operate on this shared object, they quickly render it corrupt.


In python 2.4, the asyncore.dispatcher class constructor gained a new optional map parameter.  It is used to override the internal socket_map variable, letting you supply your own dict() object.  This lets you provide a different dict() object to each instance of asyncore.dispatcher, making it perfectly thread-safe.  The module-level asyncore.loop() method also takes an optional map parameter, so you can have multiple threads safely running in their own individual event loops.  You still must ensure your threads are safe for any other data they share, but that is easily solved using threading.Lock and other tools from the standard threading module.

This works great, even for derived classes of asyncore.dispatcher, like asynchat.async_chat, a class that provides a simplified interface and somewhat flexible input buffering features with its set_terminator() method and found_terminator() callback.


But there is one small problem that only affects python environments earlier than 2.6.  The asynchat.async_chat class constructor doesn't implement the map parameter.  This causes the asyncore.dispatch class to receive a default value of map=None which tells it to use the shared module-level socket_map object.  Newer versions of async_chat in python 2.6+ provide the map parameter, but unfortunately any code intended to run on stock RHEL 5 is stuck with python 2.4.

The following code adds support for the map parameter to the asyncore.async_chat class by creating a new class named AsyncChat26:

import asyncore, asynchat, sys

class AsyncChat26(asynchat.async_chat):
    '''helper to fix for python2.4 asynchat missing a 'map' parameter'''
    def __init__ (self, conn=None, map=None):
        # if python version < 2.6:
        if sys.version_info[0:2] < (2,6):
            # python 2.4 and 2.5 need to do this:
            self.ac_in_buffer = ''
            self.ac_out_buffer = ''
            self.producer_fifo = asynchat.fifo()
            # the fix passes 'map' to the superclass constructor
            asyncore.dispatcher.__init__ (self, conn, map)
        else:
            # otherwise, we defer 100% to the parent class, since it works fine
            asynchat.async_chat.__init__(self, conn, map) 

It works by simple subclassing.  It works by overriding the asynchat.async_chat constructor adding the missing map parameter, and re-implements 4 lines of setup code from the newer async_chat constructor.  The code then invokes the parent class constructor directly but includes the map parameter.  If the code executes in python 2.6+, it simply delegates all work up to the original constructor.

For code that subclasses asynchat.async_chat, you can simply use AsyncChat26 as the new parent class, and your class will support map whether it runs on python 2.4 or python2.6+.  Multiple threads in python 2.4 may now run separate asyncore.loop() main event loops without colliding.