W.: W.: Re: [Stackless] problem with Plone (in Zope) and stackless python

Christian Tismer tismer at stackless.com
Thu Oct 28 21:44:52 CEST 2004

taskless at arcor.de wrote:

> You are a bit mysterious about what you did in your version of asyncore. Below is 
> my guessing of what you propose:
> - You want to have Zope create tasklets instead of threads, so that it appears for 
>     Zope that it is running in several threads, but in fact it will run in several 
>     tasklets of the same thread
> - Each Zope tasklet uses your stackless version of asyncore, receiving all incomming traffic 
>     from a single in-channel and sending all outgoing traffic to a single out-channel
> - You have a special tasklet (presumably the main tasklet) dedicated to feeding the 
>     in-channel with events on the sockets and writing to them the messages received 
>     on the out-channel 

No, much simpler.
The original asyncore was great for IO bound performance.
Its way to poll stuff is just fine.
When it comes to Zope, the problem is computation.
While some IO operations are to be processed, there is
no way to continue with the computation of new pages,
unless you have multiple threads.

My asyncore is similar to the original, with the difference that
it works for computation bound performance as well.
This is transparent to the user. There is no need to create
extra tasklets. async_slp does this alone for every connection
and schedules the next tasklet that can do I/O or computation.

> But how about disk I/O? Is ZODB also based on asyncore? How about Zope's usage 
> of relational databases? Are the database libraries based on asyncore?
> I doubt.

I didn't look into that. This might need some deeper analysis.
It can be done with a non-blocking FS interface as well,
but I'm not that deep into Zope.

> On one hand, the strenght of stackless is that it offers a way to avoid blocking 
> your threads. On the other hand, the original asyncore implementation already provides 
> non-thread-blocking I/O on sockets. 

Yes, but as said, it cannot help with pending computation.
You would drop threads from Zope completely, move it
back where it was some years ago.
Maybe one thread for I/O is needed, maybe not.

But we have a single CPU, usually. If we can switch
between execution contexts without threads, why use them?

> I think providing an asyncore-compliant event-driven programing interface
> to the stackless.channel-based communication facility among tasklets is still a good idea:
> it would complement the exclusively synchronous "send" and "receive" channel operations
> (ideal for non-event-driven, sequential programs) with asynchronous event detection, allowing
> existing event-driven programs (perhaps written originally to handle sockets) to
> handle stackless.channels. 

The opposite is right.
After implementing a stackless asyncore, I realized
that asyncore is not needed at all. All the explicit
polling stuff, like
   you call a receive, but it can be that you didn't
   receive, because nothing was available.
This is old style, not necessary with stackless.
Instead, I implement the old blocking socket layer
with Stackless now, which makes programming much
simpler than with asyncore, because you don't need
to check whether you sent or got something.
You just read and write.
Under the hood, the stackless socket does of course
everything with non-blocking calls. It does not
block your (single) thread. But it assigns a tasklet
to every logical I/O channel and blocks that until
it can continue.

In a sense, threads are replaced by tasklets, and
that part of the OS that schedules time when threads
block is moved into Python, scheduled by stackless
inside the single process.

> Even more valuable would be to rewrite the traditionally thread-blocking operations of I/O objects
> (e.g. read/write/flush etc. operations of the built-in file object, socket objects, etc.)
> so that they block the calling tasklet, but not the thread! A tasklet-blocking version 
> (as opposed to its original thread-blocking version) of "sleep" would be welcome as well.

Yes, I think this is exactly what I wrote above.

If all I/O is mapped to an internally non-blocking, externally
pseudo-blocking stackless layer, then threads are gone.
This is of course a problem with legacy database drivers.
Wrap those into one extra real thread. Stackless 3.1 handles this
too, meanwhile.

ciao - chris
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Carmerstr. 2                 :    *Starship* http://starship.python.net/
10623 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 31 86 04 18  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

Stackless mailing list
Stackless at stackless.com

More information about the Stackless mailing list