[Stackless] rough PyCon'07 draft; feedback wanted

Andrew Dalke dalke at dalkescientific.com
Tue Feb 13 17:29:53 CET 2007


On Feb 13, 2007, at 4:31 AM, Richard Tew wrote:
>>    - what should I stress / deemphasize?

> The most interesting one to me is the IO one as that one really shows
> the benefits of using Stackless.  I don't see it for the other  
> examples
> but that would be because of my own lack of interest in those
> activities.

The I/O one is very cool but it's also the the most complicated.
I don't even go into a full description of it.  Hmm, and I like
the XML parsing and file sniffer ones because parsing is my
bread and butter and because neither are easy to do w/o Stackless.

I do need to cut things down, I think, because of the length.
For the presentation I think I'll drop:
   - the code for the threaded content handler in the XML section
   - the factorial example
   - the code for having a tasklet call a function in another thread

If I do write something up I can say to people "for details
see blah.blah.blah."


>>    - if people build a stackless binary do they usually call
>>       that binary "python", "stackless", "spython" or something else?
>
> I use it on Windows so can't help with this.  But I would expect
> that most people isolate it out of fear because it is some arcane
> fork and may clash with their distributions installation.

Those are my reasons.  And because I wasn't sure if there was
some worrisome performance hits.

>> The code still busy waits, and I've added a new problem.  The
>> sleep_scheduler never finishes so "run" never finishes.  My code no
>> longer exits.
>>
>> In talking things over with people on the Stackless mailing list, I
>> think the problem is that "run" isn't quite smart enough.

> I replied to your post about this, but didn't have the energy or time
> to go into further detail then.
>
> I said something along the lines that it was better to discourage
> users from expecting stackless.run to work as a magic "exit when
> things are done" method by encouraging them to use another approach
> from the start.  Well, I think I said it and if I didn't it was
> what I was thinking anyway :-)

That's what you said and that's what I understood.  I thought
I was saying roughly the same thing.


> The way I see running the scheduler working is a standard form.
>
> yieldChannel = stackless.channel()
> sleepQueue = []
>
> while CheckExitCondition():
>    RunNiceTasklets()			# Sends to all on yieldChannel
>    CheckSleepingTasklets()		# Processes sleepQueue.
>
>    runawayTasklet = stackless.run(2000000)
>    if runawayTasklet:
>        print_a_stacktrace_for(runawayTasklet)
>        runawayTasklet.kill()
>
>    DoPolling()				# Check events on async resources

That's roughly the code I wrote.  I did find that having
the scheduler code in the main tasklet caused problems because
the DoPolling() through asyncore ended up working through
channels and Stackless refused to block the main tasklet.

You suggest that nice tasklets yield control through

   yieldChannel.receive()

where RunNiceTasklets() is like

   yieldChannel.send_sequence([None] * -yieldChannel.balance)

instead of using stackless.schedule()?  (I've looked at
uthread so I know the answer is "yes" and the advantage
is "to keep the scheduler empty."  Why is that important?

I was not going to mention preemptive scheduling through
"run(200000)" because I thought it would complicate the
user model to say there are two ways to do scheduling.
I also think that preemptive scheduling is scary for
stackless tasks because so much code expects atomic
operations and does not mark critical sections.


> http://svn.python.org/view/stackless/sandbox/libraries/uthread-ccp/ 
> uthread.py

The reason for my slightly different approach was to prevent
busy waits.  The uthread's CheckSleepingTasklets always runs.

> And this is why I
> have come to think that whatever sits in the place of  
> CheckExitCondition
> should explicitly check whether the application has a reason to exit
> rather than whether any arbitrary Stackless resources are still
> around.

Agreed.

> I wouldn't let this affect your presentation unless you see some value
> in it.

I think I cover most of those points so I'm already affected.  :)

> How about another name instead of main tasklet?  On one hand it is  
> what
> Stackless calls the tasklet that you are creating these in and on the
> other when you launch several, as in the example at the end, it looks
> a little confusing.

D'oh!  I didn't even notice that.  Yes, that needs to be fixed!
Thanks for spotting that.

>> stacklesssocket.socket = slib_socket
>
> I do this if I want to bypass the 'automatic poller':
>
> import stacklesssocket
> stacklesssocket.managerRunning = True
>
> But maybe you want this approach to illustrate monkeypatching.

I didn't think about doing it that way, which I think is
cleaner.  I was thinking too much about monkeypatching, it's
true, but there's no need to have that much explanation.
I can instead say "because I've implemented my own manager there's
no need to use the default one in stacklesssocket."


> I've been thinking that the uthread-ccp module should really be
> cleaned up and possibly included as part of Stackless.  Perhaps
> if you think it suits you can integrate the things you think help
> into it, perhaps extending the uthread.run method?  Improving
> the sleep method or whatever.  That's something to think about
> anyway.

I can pull some things from it but there are things I
would change.  Like function names, many of which aren't PEP 8.

Hmm, I wonder if
       sleepingTasklets.sort()
is better than my using heapq.  There just aren't going to
be that many sleeping tasklets so sort may be faster despite
the theoretical n*n*log(n) performance.


				Andrew
				dalke at dalkescientific.com



_______________________________________________
Stackless mailing list
Stackless at stackless.com
http://www.stackless.com/mailman/listinfo/stackless



More information about the Stackless mailing list