[Stackless] Stackless sprint at PyCon (Richard Tew)

Andrew Francis andrewfr_ice at yahoo.com
Sat Mar 19 19:27:41 CET 2011


Hi Folks:

> Message: 1
> Date: Fri, 18 Mar 2011 07:36:50 +0800
> From: Richard Tew <richard.m.tew at gmail.com>
> To: stackless at stackless.com
> Subject: [Stackless] Stackless sprint at PyCon
> Message-ID:
>     <AANLkTimU+FHS-jaZY9XOmHhsokWSfoePDcqtgKYH66mY at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> Hi,
> 
> Unfortunately, due to ill health I could not attend PyCon,
> so I have little
> idea how the sprint worked.  However, there were
> around 10 people listed on
> the sprint project page, which would make it a record
> Stackless sprint
> attendance!  (last time, it was just Christian,
> Kristjan Valur and I).
> 
> So, anyone who attended want to post about how it went?

I thought the sprints went well.

It was a *great* joy to finally meet Christian. 

1.0 What makes Stackless stackless

Christian was gracious in giving me a description of what makes Stackless stackless. In particular, I was interested in when a hard switch occurs. I will follow up by looking at stackless python under gdb.

2.0 Select

In turn, I gave Christian a rundown on how select worked and how the version of stackless.py my friend Kevin and I developed could be extended to support new constructs. Kevin had the good instincts and skills to work with the Plan9 channel model and make it happen. I originally opted for a clean room version that worked but was not as elegant. I now believe the Plan9 libthread/channel.c model can be generalised to

1) support new constructs - I am very keen on join patterns.
2) Be optimized for certain conditions mainly:

while True:
    chanop = stackless.select(...)
    if chanop in [  ]
       ....
    elif chanop in [ ]
       ....
    else:
       .....

if the select tears down internal structures only to build them again,
then we may be able to get massive speed gains on big select sets. As a tangential remark, 

The question that I have not answered yet is how much of a performance hit does one take re-organizing the way channels work? I could answer this question by executing simple programmes with the old stackless.py against the new stackless.py.  However I will probably take a stab at a new C Stackless Python version that is more like with how stackless.py works.

3.0 Scheduler:

The only real thing I did on the sprint list was to look at the scheduler code to see if it could be modularised. Right now the scheduler is scattered in about eleven files. For the most part, I believe an initial scheduler API would be rather simple. Now if I understand Christian, he is interested also in how to extend this new module to Python.

Maybe my own other thought on structuring a scheduler module is make it more "object-oriented" in the way say, vnodes are structured in UNIX: a bunch of pointers to functions that manipulate the data structures. In that way it is easier to plug in and play with new scheduler algorithms.

4.0 Stacklesslib:

I also looked at stacklesslib. I need a version that works to really give insights. 

5.0 Experimental Branch

About an experimental stackless branch. That was nixed. 'C' based stackless will be done in probably in bitbucket or something to that effect. It will take a back seat to work in stackless.py.

6.0 Stackless.py

Fix bugs (listed in stackless.py file). Addition of a new class: abstract guard. Add join conditions. Add iteration over a channel. Experiment with the loop optimization and faster ways of checking if a chanop is in a set (in maybe slow for large sets)

7.0 StacklessReactor:

Finally on Wednesday I sat with the Twisted team. We started the beginning of a Stackless Reactor. Eventlets had integration with Twisted. I think I can do better. Again, this will be prototyped in stackless.py. A scheduler module would help.

8.0 Extra comments

I was surprised by how much attention greenlet based solutions got at PyCon 2011. I really think Stackless has to do a better job at selling itself.


Cheers,
Andrew









      



More information about the Stackless mailing list