[Stackless] A Solution Re: Advice on Writing a Stackless Reactor for Twisted

Andrew Francis andrewfr_ice at yahoo.com
Mon Jan 29 18:33:51 CET 2007


Hello David and Colleagues:

--- stackless-request at stackless.com wrote:

>I'm certainly new at this so any guidance is greatly
>appreciated, although I  don't want to hijack
Andrew's >original query.

I posted a second time to the Twisted mailing list,
describing my problem in more detail. Folks were
helpful, but Phil Mayers really provided the
information I was looking for. I am still testing but
things are looking good. Once I am comfortable with
the solution, I will post to the Wiki complete with
examples. I am interested in two things :

1. Finding problems with following solution.
2. Making the approach more general and efficient.
Right now, I am trying to solve problems specific to
my project.

The Problem :

How to run Twisted and Stackless so that when Twisted
blocks, Stackless can continue. Typically this is not
an issue. For me it is a big issue. 

A Solution :

A part of my solution is to run Twisted and Stackless
in separate threads. This is the approach taken to
make Twisted work with frameworks that have event
loops. The good news is that subclassing the Twisted
reactor is not necessary. Rather one can run the
"stackless" code in its own thread. For instance, the
body of the stackless code could look like:

def stacklessThread(arguments):
    
    """
    launch stackless tasklets here
    """
    
    while (stackless.getruncount() > 1):
        stackless.schedule()

~~~

from the Twisted thread (the main thread), start the
stackless thread with the following Twisted call :

reactor.callInThread(stacklessThread, arguments)

For now, I am being conservative by using a
threading.deque to communicate between the Stackless
and Twisted thread. I don't see a reason why
stackless.channels could not be used.

To ensure that Stackless tasklets can run, I have a
tasklet that reads responses from the deque. I do the
following :

while (some_flag):
    if (len(deque) > 0):
       response = deque.pop()

       """
       do something
       """
    else:
       stackless.schedule()


>From within the Stackless thread, when I wish to make
a Twisted call, I do the following :

reactor.callFromThread(twisted_call)
i.e.,
reactor.callFromThread(GetWebPage(...).execute)

for instance, a call to get a web page could like like

class GetWebPage(object):
    def __init__(self, responseQueue, url):
        self.responseQueue = responseQueue
        self.url = url

    def execute(self):
        # make the deferred call     
          client.getPage(url).\
              addCallback(self.handlerResponse). \
              addErrback(errorHandler)
         
    def handlerResponse(self, data):
       self.responseQueue.append(data)


Hopefully this will help others that wish to use
Stackless with Twisted.

Cheers,
Andrew




 
____________________________________________________________________________________
Any questions? Get answers on any topic at www.Answers.yahoo.com.  Try it now.

_______________________________________________
Stackless mailing list
Stackless at stackless.com
http://www.stackless.com/mailman/listinfo/stackless



More information about the Stackless mailing list