[Stackless] switching improvements

Kristján Valur Jónsson kristjan at ccpgames.com
Fri Nov 30 09:44:59 CET 2012


Thanks for looking this over, Christian.
Yes, there are several things in there, really:
1) Macro fixups,
2) The adding of new methods to remove/reinsert tasks from channels and the runnable queue.
3) A change to slp_schedule_task, explicitily signalling a failure to switch, rather than just an exploded bomb on return.
4) Ability to "undo" changes to the state if switching fails
5) Various apis can now fail and return with an unchanged state: Channel actions, stackless.schedule, stackless.run.  I probably  need to add tasklet.run to this list.  Oh, and tasklet.kill, tasklet.raise_exception.
6) Unit tests for the failure modes.


I thought a bit about the fact that these apis have no semantic difference between an error that occurs as the result of a switch, and an error that is "sent" to the tasklet.
E.g. channel.recv() can fail with a RuntimeError (perhaps it is time Stackless got its own errors, no?) or MemoryError if it cannot switch, but it can also fail with any error if another tasklet doesn't handle its error, or someone sends it an exception (tasklet.raise_exception).  The caller cannot know which case it is.  But maybe it is unimportant.

K

> -----Original Message-----
> From: Christian Tismer [mailto:tismer at stackless.com]
> Sent: 29. nóvember 2012 23:01
> To: Kristján Valur Jónsson
> Cc: The Stackless Python Mailing List
> Subject: Re: [Stackless] switching improvements
> 
> Yeah,
> 
> I looked quite a lot into it, but did not have time to try it, yet.
> 
> The patch is rather large and needs more people to review and try it out.
> 
> There are quite some changes which make pretty much sense in any case,
> btw.:
> Things like the improved macro syntax should go in directly with no doubt.
> It would be good if the patch would not mix style improvements and
> functional changes.
> Also, some changes of the call parameters make sense and should probably
> go in without hesitating.
> 
> So I'd split it in two or maybe three patches instead of one.
> 
> If you don't mind, I can try that in a few days.
> 
> cheers - chris
> 
> On 11/29/12 12:27 PM, Kristján Valur Jónsson wrote:
> > Ok, take a look at https://bitbucket.org/krisvale/stacklessswitching
> >
> >
> > From: Christian Tismer [mailto:tismer at stackless.com]
> > Sent: 29. nóvember 2012 01:25
> > To: The Stackless Python Mailing List
> > Cc: Kristján Valur Jónsson
> > Subject: Re: [Stackless] switching improvements
> >
> > Hi Kristjan,
> >
> > how about checking it into a branch firt, let us try and check that
> > intensively, and after enough confidence merge it in?
> > That's the way that works best for me with Mercurial/Git.
> >
> > If you don't like to put a branch into python.org, you also can do a
> > clone on bitbucket easily and let us use that for review.
> >
> > cheers - chris
> >
> > On 28.11.12 16:11, Kristján Valur Jónsson wrote:
> > Hello All.
> > I was recently prompted to add a flag to stackless, a way to block all tasklet
> switching.
> > This springs from the way that we are embedding stackless python in an
> game engine (UnReal) which sometimes makes callbacks into python.
> Sometimes, this code will do nasty stuff that results in tasks switching,
> causing havoc with the control flow of the game engine.
> > To simplify this, I added a per-thread flag, switch_trap, which can be
> controlled in a similar way to block_trap.  If the logic causes a switch to be
> attempted, this should be trappable and the code should be easily fixable, or
> we can otherwise deal with it.
> >
> > Anyway, doing this, adding it to slp_schedule_task(), and so on, uncovered
> a subtle flaw in stackless:
> > It turns out that slp_schedule_task() had no way of differentiating whether
> an exception result from this call came as a result of a failure to switch, or an
> exception being sent to the tasklet when it wakes up again.
> >
> > So, I have changed the interface to be able to do this properly.  There are
> other reasons why switching can fail, including memory allocation failures and
> so on, so this seems like a necessary change.  I also fixed code both in
> stacklesseval.c and taskletmodule and channelobject to be able to cope with
> switch failure like this.
> >
> > Now:
> > How does this sound to you?  The change is somewhat large and I would
> hesitate to simply check it in without some sort of review or otherwise
> approval.  Any suggestions?
> >
> > K
> >
> >
> >
> >
> > _______________________________________________
> >
> > Stackless mailing list
> >
> > Stackless at stackless.com<mailto:Stackless at stackless.com>
> >
> > http://www.stackless.com/mailman/listinfo/stackless
> >
> >
> >
> >
> > --
> >
> > Christian Tismer             :^)
> <mailto:tismer at stackless.com><mailto:tismer at stackless.com>
> >
> > Software Consulting          :     Have a break! Take a ride on Python's
> >
> > Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
> >
> > 14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
> >
> > phone +49 173 24 18 776  fax +49 (30) 700143-0023
> >
> > PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
> >
> >        whom do you want to sponsor today?   http://www.stackless.com/
> 
> 
> --
> Christian Tismer             :^)   <mailto:tismer at stackless.com>
> Software Consulting          :     Have a break! Take a ride on Python's
> Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
> 14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
> PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
>        whom do you want to sponsor today?   http://www.stackless.com/
> 



More information about the Stackless mailing list