[Stackless] verify my performance numbers?

Andrew Dalke dalke at dalkescientific.com
Fri Jan 26 23:42:06 CET 2007

Hi all,

   I'm looking for someone to double check my timing findings on
other OSes.  In short, stackless has 1/10th of the overhead for
this task than using threads but I don't know how OS-specific
that is.

   I'm working on my presentation for PyCon.  I'm giving an
intro to Stackless talk.  I'm about 1/2 way through my rough
draft version -- text only, not in slide format, etc.  I compared
using Stackless to using threads for the same task, which is
to invert a callback solution (parsing XML using SAX) into an
iterator solution.

   In doing the testing now I found (using best of 3):

threaded version: 93 seconds (94.9s, 94.5s, 93.1s)
stackless version: 20 seconds (20.8s, 19.9s, 20.4s)
low-level SAX interface: 8.4 seconds  (8.36s, 8.57s, 8.49s)

About 1 second of the stackless version is spent in
constructing the message to send through the channel
and the function call overhead.

About 3/4 seconds is spent in the termination test
for endDocument.  (I got about 0.7 seconds by switching
to use an 'is' test instead of a data[0] == "endDocument")

My test data set was my iTunes library file listing all
of my songs.  It is 6,050,866 bytes with 223,075 calls
each of startElement, characters and endElement, plus one
call each of startDocument and endDocument.  That makes
for 669,227*2 context switches.

I'm happy, and a bit surprised, that the stackless vs.
thread solutions are that big.  Would some here care
to re-test my code and verify that for me?

I tested this on a Mac PowerBook G4 running 10.3 and would
like to see number for other OSes/thread implementations.

The programs are:

1) itersax_stackless.py -- the inverted solution using stackless

-------------- next part --------------
A non-text attachment was scrubbed...
Name: itersax_stackless.py
Type: application/octet-stream
Size: 1511 bytes
Desc: not available
URL: <http://www.stackless.com/pipermail/stackless/attachments/20070126/f33066a8/attachment.obj>
-------------- next part --------------

2) itersax_threads.py -- the inverted solution using threading.py

-------------- next part --------------
A non-text attachment was scrubbed...
Name: itersax_threads.py
Type: application/octet-stream
Size: 1515 bytes
Desc: not available
URL: <http://www.stackless.com/pipermail/stackless/attachments/20070126/f33066a8/attachment-0001.obj>
-------------- next part --------------

3) callback_sax.py -- the standard callback solution
     (uncomment a line towards the end to get the version which
      adds the overhead for constructing the message tuple and
      doing the function call overhead.)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: callback_sax.py
Type: application/octet-stream
Size: 759 bytes
Desc: not available
URL: <http://www.stackless.com/pipermail/stackless/attachments/20070126/f33066a8/attachment-0002.obj>
-------------- next part --------------

I would like to see the program output (an elapsed time) for
each program run 3 times on your machine.  You'll have to point
to a local XML file.  Just pick a comparable large one and let
me know the rough size and number of elements in it.  Or I can
send you my iTunes index and you can find out what I listen to.  :)

					dalke at dalkescientific.com
-------------- next part --------------
Stackless mailing list
Stackless at stackless.com

More information about the Stackless mailing list