[Stackless] More on a Stackless Select

Christian Tismer tismer at stackless.com
Fri Apr 16 21:49:07 CEST 2010


Hi Andrew,

On 4/16/10 5:08 PM, Andrew Francis wrote:
> Hi Colleagues:
>
> In previous posts, I talked about implementing a select in Stackless Python. Conceptually it is not difficult. And I still think most of a select like mechanism can and should be implemented as a module. However the more I work with the stackless.py prototype and look at C Stackless Python, I believe I was optimistic in the amount of changes required to C Stackless Python. The changes are not huge, but they are not trivial either as I originally planned. I plan to give a talk on this at the end of this month to my local Python group :-) I still need to run benchmarks that will determine if all of this is really worth it.
>
>    
That is good to hear. Stackless, as it is, puts a big impact on
the developer. The changes to the core are way too big.
Developing modules against it is not recommended now.
> To date some of the difficulties:
>
> 1) blocked is overloaded to both indicate state and the operation - this has to be separated.
>
Not sure what you mean. Blocked is meant to be an indicator.
Maybe I overloaded it.
> 2) What does _channel meanwhen a tasklet is blocked on a selector? Should it point to a selector implementation. Should _channel become some list of channels. What would this break?
>
_channel is an implementation detail, to find the channel if
necessary. In case of multiple channels, it should become
a set of channels. The channel structure might get extended,
or we create a channel set. Not sure which structure would
be best. Again this is better to model in Python.
> 3) Select needs the most cooperation from the channel class. The particularly complicating case - selector is waiting for one or more receivers to rendezvous.
>
I think for prototyping you should try to implement multi-channels
on top of channels. You need to use a tasklet per channel that does this
communication. The point is to wake up when any of the
channels becomes active. Then all other waiting helper tasklets
get an exception to quit.
To make this efficient is not trivial. Select suffers from the call 
overhead.
If you want to wait on 10000 channels, then when one fires, the select is
done, and for the next operation, you will have to select the 10000
channels again. You need a sophisticated data structure, a channel
set, that stays intact after a call.
The channel action would then just remove the tasklet and the
channel from the set.
I never got my thought about this clear enough to implement it.
Again, model this in Python.
> 4) To avoid race conditions or really really complicated data structures, the tasklet with the selector has to be scheduled immediately. This implies that anything using a select like mechanism will run with high priority. Then again, it is acting like a dispatcher in an event loop.
>
>    
Not really. The action becomes two-fold. The reaction on the channel
activity is one thing. The tasklet gets freed from the channel set.
But it does not have to run immediately. I would implement this
with a helper tasklet.

Once again:
I highly dis-recommend writing any C extensions for Stackless.
Not only because stackless is going to change internally a lot,
but also because of upcoming Psyco support.
See my other post on that, to come right now.
Whatever you write in Python is subject to compilation.
It will most probably accelerated by Psyco. And if it's not,
then I will put effort in to support it, because that brings
much more benefits than optimizing C code, which is
very limited and not flexible.

cheers - chris

-- 
Christian Tismer             :^)<mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key ->  http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/




More information about the Stackless mailing list