[Stackless] Questions on the Evolution of Stackless

[Andreas Kostyrka]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Andrew Francis wrote:
| 3) I see greenlets, generators, tasklets and now
| eventlets - all different types of coroutines. In
| Twisted, I see inlineCallbacks (based on generators).
|
| Silly question, besides the programming interface,
| what really is the difference between a
| tasklet/channel and generator? Tasklets and channels
| seem more general purpose and powerful.

Well, basically:

generators: cannot yield from a subfunction call.
greenlets: can yield basically everywhere.
stackless: adds cooperative (and uncooperative) scheduling.

IMHO, generator based coroutines are quite limited by the fact that it's
impossible to develop functions to do work. You can yield back some form
~ of say IO request to the outside, but you cannot encapsulate that
functionality in a library for easy usage.

greenlets/stackless are both in some way enough to implement frameworks
that hide much of the implementation details.

Now, where is this going, some observations:

- -) Twisted is way more efficient for handling many connections than
other models. Notice what happens when a newbie writes a http mirror
script with Twisted. Instant denial-of-service, because while basically
any computer running Twisted can handle 2000 concurrent http
connections, most Apaches (single host) go through the roof when they
see 2000 requests concurrently.

- -) Twisted is twisty. I personally have no problems with Twisted, but I
do know that some relative good developers consider Twisted apps to
painful to maintain. Especially, multiple experiences for me suggest
that typical average programmers are not capable of doing something
"simple" as web development. So Twisted is not and will probably never
become a mainstream solution.

Now the question is how to marry the performance benefits of async
programming with a more traditional programming model.

The answer seems to be advanced control structures, namely coroutines as
a minimum.

As for numbers, a quick and dirty test suggest that on my laptop, the
wsgiref demoapp gives the following results:

wsgiref.WSGIServer (single threaded)            ~300 req/s
wsigref.WSGIServer (ThreadingMixIn)             ~375 req/s
stackless WSGIServer                            ~700 req/s

Notice that it's the same app, just different ways to publish it.
(Compare this to CGI scripts on Apache2 that give less then 10 req/s on
my laptop, or static files served by Apache2 with ~2500 req/s)


| Bigger question - Where is all this going? The one
| thing I see is that the desire to do efficient network
| I/O is propelling the evolution of coroutines in
| Python. Would this be an accurate statement?

Well, perhaps not for the big mainstream, but there is clearly a group
of people who care about performance ;)

Andreas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHxEckHJdudm4KnO0RAoEOAKCSeKfKOvR2C6++XbUuaFGO3MrEPwCfaRZ6
xIBcD00oGJPq7DNr+PmsZZ8=
=xgZC
-----END PGP SIGNATURE-----