[Stackless] Advice on Writing a Stackless Reactor for Twisted

Andrew Francis andrewfr_ice at yahoo.com
Tue Jan 16 20:31:35 CET 2007

> Message: 2
> Date: Tue, 16 Jan 2007 02:23:00 -0500
> From: "David Wyand" <TheHeadGnome at gnometech.com>
> Subject: [Stackless]  Advice on Writing a Stackless
> Reactor for
> 	Twisted
> To: <stackless at stackless.com>
> Message-ID:
> <000501c7393f$27cdcc70$0601a8c0 at INTELXP6>
> Content-Type: text/plain; charset="iso-8859-1"

Hi Dave:

> My goal is to is to use these three technologies
>along with the Torque Game Engine to build a
>multiplayer game.  I'm currently in the technology
>testing stage.

Sounds cools! I am using Stackless Python and Twisted
to write WS-BPEL processor prototypes. I really think
that there is much to be gained by making Stackless
Python and Twisted cooperate.

>After looking at what has been proposed on the
Twisted >mailing list, I took the direction of
Stackless >driving Twisted rather than the other way
around.  >After combining some techniques from Josh
Ritter (over >at GarageGames), Richard Tew (uthread is
cool) and
> other sources, here's what I've come up with.

I don't know if you have been following my questions
on the Twisted list? 

I find that it is fairly straight forward to use
Stackless with Twisted, provided you are writing a
client or a server (or serialising client and server
calls). My problems are a result of trying to do these
calls in parallel and getting around the fact that the
Twisted reactor blocks, hence creating a bottleneck. 

I am in the process of coding a solution. So far, it
looks like it is working. I need to run more tests
before I declare victory :-)

I am taking the approach that Stackless and Twisted
really like being executed in their own threads. So, I
have the Twisted and Stackless thread share a request
and response queue of type Queue.queue. 

For instance, the structure of my Stackless response
loops is :

while (self.flag):
    if responseQueue.empty()
       #give other tasklets a chance to run
        response = queue.get()
        # do something useful with response
In turn, I wrap Twisted in a thread that has an event
loop that reads messages from request queue and
executes the appropriate Twisted code. When a callback
 or request Handler is activated, it places messages
on the response queue.

This way Stackless and Twisted are isolated. The
asynchrony provided by the queues adds an additional
level of decoupling. I avoid stuff like custom
reactors and callFromThread(). 

> So far my own testing indicates everything is
> working as expected.  But it would be great to know
> if anyone finds any issues with my implementation. 
> And hopefully this'll help Andrew out. 

Dave thanks for your input. When I have time I will
look at your solution in more detail and try it out. I
think your solution is more general than mine. My
solution (if it is one) caters more to the
idiosyncrasies of my WS-BPEL processor architecture. 

>I've included my stackless reactor as well as a
>snippet on how to use it.  You'll have to provide
your >own Twisted-based HTTP server if you want to run
>example as is.


Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.

Stackless mailing list
Stackless at stackless.com

More information about the Stackless mailing list