[Stackless] switching improvements

Christian Tismer tismer at stackless.com
Thu Dec 6 00:35:33 CET 2012


I'm also waiting for more input.

I have no time for testing before the weekend.
and I'd appreciate to get opinions.
Also I would like to split the patch into the pieces
that were depicted below.

Besides the enhancements to the API and implementation,
what is your opinion about switch_trap?

I'm like +0.5 or better, not a real use-case but also no objection.

What do people think?

cheers - Chris


On 05.12.12 13:55, Kristján Valur Jónsson wrote:
> Any comments yet?
> The impetus for this change is to be able to add the stackless.switch_trap() feature.  This is something our engineers are asking for.  I obviously don't want to bring the CCP stackless branch too far away from the official one, which is why I was proposing to add this to the official stackless too.
>
> K
>
>> -----Original Message-----
>> From: stackless-bounces at stackless.com [mailto:stackless-
>> bounces at stackless.com] On Behalf Of Kristján Valur Jónsson
>> Sent: 30. nóvember 2012 08:45
>> To: Christian Tismer
>> Cc: The Stackless Python Mailing List
>> Subject: Re: [Stackless] switching improvements
>>
>> Thanks for looking this over, Christian.
>> Yes, there are several things in there, really:
>> 1) Macro fixups,
>> 2) The adding of new methods to remove/reinsert tasks from channels and
>> the runnable queue.
>> 3) A change to slp_schedule_task, explicitily signalling a failure to switch,
>> rather than just an exploded bomb on return.
>> 4) Ability to "undo" changes to the state if switching fails
>> 5) Various apis can now fail and return with an unchanged state: Channel
>> actions, stackless.schedule, stackless.run.  I probably  need to add tasklet.run
>> to this list.  Oh, and tasklet.kill, tasklet.raise_exception.
>> 6) Unit tests for the failure modes.
>>
>>
>> I thought a bit about the fact that these apis have no semantic difference
>> between an error that occurs as the result of a switch, and an error that is
>> "sent" to the tasklet.
>> E.g. channel.recv() can fail with a RuntimeError (perhaps it is time Stackless
>> got its own errors, no?) or MemoryError if it cannot switch, but it can also fail
>> with any error if another tasklet doesn't handle its error, or someone sends it
>> an exception (tasklet.raise_exception).  The caller cannot know which case it
>> is.  But maybe it is unimportant.
>>
>> K
>>
>>> -----Original Message-----
>>> From: Christian Tismer [mailto:tismer at stackless.com]
>>> Sent: 29. nóvember 2012 23:01
>>> To: Kristján Valur Jónsson
>>> Cc: The Stackless Python Mailing List
>>> Subject: Re: [Stackless] switching improvements
>>>
>>> Yeah,
>>>
>>> I looked quite a lot into it, but did not have time to try it, yet.
>>>
>>> The patch is rather large and needs more people to review and try it out.
>>>
>>> There are quite some changes which make pretty much sense in any case,
>>> btw.:
>>> Things like the improved macro syntax should go in directly with no doubt.
>>> It would be good if the patch would not mix style improvements and
>>> functional changes.
>>> Also, some changes of the call parameters make sense and should
>>> probably go in without hesitating.
>>>
>>> So I'd split it in two or maybe three patches instead of one.
>>>
>>> If you don't mind, I can try that in a few days.
>>>
>>> cheers - chris
>>>
>>> On 11/29/12 12:27 PM, Kristján Valur Jónsson wrote:
>>>> Ok, take a look at https://bitbucket.org/krisvale/stacklessswitching
>>>>
>>>>
>>>> From: Christian Tismer [mailto:tismer at stackless.com]
>>>> Sent: 29. nóvember 2012 01:25
>>>> To: The Stackless Python Mailing List
>>>> Cc: Kristján Valur Jónsson
>>>> Subject: Re: [Stackless] switching improvements
>>>>
>>>> Hi Kristjan,
>>>>
>>>> how about checking it into a branch firt, let us try and check that
>>>> intensively, and after enough confidence merge it in?
>>>> That's the way that works best for me with Mercurial/Git.
>>>>
>>>> If you don't like to put a branch into python.org, you also can do a
>>>> clone on bitbucket easily and let us use that for review.
>>>>
>>>> cheers - chris
>>>>
>>>> On 28.11.12 16:11, Kristján Valur Jónsson wrote:
>>>> Hello All.
>>>> I was recently prompted to add a flag to stackless, a way to block
>>>> all tasklet
>>> switching.
>>>> This springs from the way that we are embedding stackless python in
>>>> an
>>> game engine (UnReal) which sometimes makes callbacks into python.
>>> Sometimes, this code will do nasty stuff that results in tasks
>>> switching, causing havoc with the control flow of the game engine.
>>>> To simplify this, I added a per-thread flag, switch_trap, which can
>>>> be
>>> controlled in a similar way to block_trap.  If the logic causes a
>>> switch to be attempted, this should be trappable and the code should
>>> be easily fixable, or we can otherwise deal with it.
>>>> Anyway, doing this, adding it to slp_schedule_task(), and so on,
>>>> uncovered
>>> a subtle flaw in stackless:
>>>> It turns out that slp_schedule_task() had no way of differentiating
>>>> whether
>>> an exception result from this call came as a result of a failure to
>>> switch, or an exception being sent to the tasklet when it wakes up again.
>>>> So, I have changed the interface to be able to do this properly.
>>>> There are
>>> other reasons why switching can fail, including memory allocation
>>> failures and so on, so this seems like a necessary change.  I also
>>> fixed code both in stacklesseval.c and taskletmodule and channelobject
>>> to be able to cope with switch failure like this.
>>>> Now:
>>>> How does this sound to you?  The change is somewhat large and I
>>>> would
>>> hesitate to simply check it in without some sort of review or
>>> otherwise approval.  Any suggestions?
>>>

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/




More information about the Stackless mailing list