[Stackless] Practical uses of stackless technology

Richard Tew richard.m.tew at gmail.com
Tue Oct 24 13:18:07 CEST 2006


On 10/24/06, Michael Hudson <mwh at python.net> wrote:
> "Richard Tew" <richard.m.tew at gmail.com> writes:
> > And readability is what I think of, when I think of the primary
> > benefit of using Stackless.  Through the use of tasklets and channels
> > almost anything can be hidden behind a synchronous call, naturally
> > extending the readability of Python by allowing a large disregard
> > for techniques like callbacks and the need to otherwise use
> > lots of boilerplate.  That you can just schedule functions as
> > microthreads through tasklets, and have them predictably
> > scheduled through cooperative multitasking fleshes this out.
>
> This sort of thing is definitely the impression I'm getting.  Games or
> simulations where you have arbitrarily many 'agents' acting in
> parallel.
>
> It also seems like stackless ought to be a fit for concurrent
> networking apps, but people don't seem to be seriously using stackless
> for this (yet?).  I guess it's partly the desire to rely on someone
> else's, e.g., HTTP code.  Though I guess EVE *is* a concurrent
> networking app :)

Yes, EVE's networking framework is pretty much written in Python
and based on tasklets and channels.  The only part that isn't is the
raw IO which is done in some asynchronous manner and wrapped
with channels, so that calling tasklets on the sockets can block. I
think we do the same with our file IO.

This is something that Andrew Dalke recently pointed out to me,
that this is something which should have been in Stackless from
the get-go, to just make it work.  That all the interpreter blocking
IO in the Python runtime should be transparently done with
asynchronous IO instead and wrapped with channels to block
the calling tasklets.

The replacement socket module I wrote does this just for the
socket layer, and can be found on the stackless.com
examples page.  And I am working (at a slow pace) on a
general module which will do this for all IO as well,
slpmonkeypatch which can be found in the stackless
svn sandbox libraries directory.

And since Andrew pointed this out, it makes a lot of sense
that it might be a large reason why people just aren't using
it for concurrent networking apps.  Along with the fact
Stackless is a fork of Python of course, there also seems
to be some reticence because of that and I know if I hadn't
already bought into Stackless, I would also be reticent
because of it.

Does PyPy do this legwork and make IO stackless
compatible, doing it somehow asynchronously in the
background?  Or does it leave it to the user to discover
the problem and address it on their own?

While I am not that familiar with the generator coroutines
apart from what I have read about their limitations, (and
having a personal opinion about the ugliness of generators
when used for this purpose at least compared to Stackless),
I have to wonder whether mainline Python could do most of
this asynchronous IO legwork to the benefit of both
Stackless and generator coroutines.  And saving me a lot
of monkeypatching work.

Richard.

_______________________________________________
Stackless mailing list
Stackless at stackless.com
http://www.stackless.com/mailman/listinfo/stackless



More information about the Stackless mailing list