[Stackless] Stackless and Twisted

[Andrew Francis]

Hello Christian et al:

>Trying to get up-to-date, reading all stuff about
>Stackless that I almost forgot, I found the ideas
>about twisted integration, again.

>It was mentioned from time to time, also different
>approaches were tried.

>Now I'm a bit confused if there is any consensus what
>a stackless/twisted integration should be, and if
>there are any problems left.

I am about to give a talk to the Ottawa Python User's
Group (OPAG) about my experiences with Stackless
Python and Stackless. I would like to believe my
WS-BPEL prototypes are non-trivial applications. So
far I am really pleased with Stackless and Twisted.
Still I have much to learn.

Based on my experiences, here are some of my thoughts.

1. For the vast bulk of projects, one can readily
integrate Twisted with Stackless in a straightforward
fashion. Look at the examples in the repository.

2. To address a concern I believe you raised in the
past (and advice from Glyph about the reactor), it
seems that nothing have to be done to the Stackless or
Twisted code bases, to get them to inter-operate. I
feel this is a big win.

Note, it seems that much of the time, there is not a
need to use native threads...

3. If your application is not purely reactive (i.e.,
it just sits there and waits for input) AND/OR you
wish to make Twisted calls from within a request
handler, you need a slightly more complicated
architecture 

A) Typically the reactor is a tasklet. Therefore it is
desirable to have the reactor tasklet yield, so that
other tasklets that wish to run, can run. I find the
easiest approach is to have a Twisted Task
periodically call schedule(). This works because the
Task is a part of the reactor tasklet.

B) Given A,  It is possible to "deadlock" the reactor.
I have done this by having a request handler  wait on
channels, while other tasklets made Twisted calls. In
turn, the tasklet that would issue the HTTP response
was waiting for aformentioned tasklet to complete.
Since the request handler is a part of the reactor
tasklet, the reactor main event loop cannot iterate.
Hence tasklets that depend on Twisted will grind to a
halt.

I avoided this by having the request handler execute
in its own tasklet.
 
Christian assuming I am right, the two downsides are
speed and I use an extra tasklet per request. Then
again, that last solution is probably specific to my
project and overkill for most folks. When I have time,
I would like to learn how to use something like
slprofile to analyse what is going on. Without data
and tests, it is difficult to improve the system .

On a slightly different note, I see stuff mentioned in
the Twisted mailing list like inline callbacks that
use generators. It would be interesting to compare
their use to tasklets.


Cheers,
Andrew



      ____________________________________________________________________________________
Shape Yahoo! in your own image.  Join our Network Research Panel today!   http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 



_______________________________________________
Stackless mailing list
Stackless at stackless.com
http://stackless.com/cgi-bin/mailman/listinfo/stackless