[Stackless] switching improvements

Kristján Valur Jónsson kristjan at ccpgames.com
Wed Dec 5 13:55:32 CET 2012


Any comments yet?
The impetus for this change is to be able to add the stackless.switch_trap() feature.  This is something our engineers are asking for.  I obviously don't want to bring the CCP stackless branch too far away from the official one, which is why I was proposing to add this to the official stackless too.

K

> -----Original Message-----
> From: stackless-bounces at stackless.com [mailto:stackless-
> bounces at stackless.com] On Behalf Of Kristján Valur Jónsson
> Sent: 30. nóvember 2012 08:45
> To: Christian Tismer
> Cc: The Stackless Python Mailing List
> Subject: Re: [Stackless] switching improvements
> 
> Thanks for looking this over, Christian.
> Yes, there are several things in there, really:
> 1) Macro fixups,
> 2) The adding of new methods to remove/reinsert tasks from channels and
> the runnable queue.
> 3) A change to slp_schedule_task, explicitily signalling a failure to switch,
> rather than just an exploded bomb on return.
> 4) Ability to "undo" changes to the state if switching fails
> 5) Various apis can now fail and return with an unchanged state: Channel
> actions, stackless.schedule, stackless.run.  I probably  need to add tasklet.run
> to this list.  Oh, and tasklet.kill, tasklet.raise_exception.
> 6) Unit tests for the failure modes.
> 
> 
> I thought a bit about the fact that these apis have no semantic difference
> between an error that occurs as the result of a switch, and an error that is
> "sent" to the tasklet.
> E.g. channel.recv() can fail with a RuntimeError (perhaps it is time Stackless
> got its own errors, no?) or MemoryError if it cannot switch, but it can also fail
> with any error if another tasklet doesn't handle its error, or someone sends it
> an exception (tasklet.raise_exception).  The caller cannot know which case it
> is.  But maybe it is unimportant.
> 
> K
> 
> > -----Original Message-----
> > From: Christian Tismer [mailto:tismer at stackless.com]
> > Sent: 29. nóvember 2012 23:01
> > To: Kristján Valur Jónsson
> > Cc: The Stackless Python Mailing List
> > Subject: Re: [Stackless] switching improvements
> >
> > Yeah,
> >
> > I looked quite a lot into it, but did not have time to try it, yet.
> >
> > The patch is rather large and needs more people to review and try it out.
> >
> > There are quite some changes which make pretty much sense in any case,
> > btw.:
> > Things like the improved macro syntax should go in directly with no doubt.
> > It would be good if the patch would not mix style improvements and
> > functional changes.
> > Also, some changes of the call parameters make sense and should
> > probably go in without hesitating.
> >
> > So I'd split it in two or maybe three patches instead of one.
> >
> > If you don't mind, I can try that in a few days.
> >
> > cheers - chris
> >
> > On 11/29/12 12:27 PM, Kristján Valur Jónsson wrote:
> > > Ok, take a look at https://bitbucket.org/krisvale/stacklessswitching
> > >
> > >
> > > From: Christian Tismer [mailto:tismer at stackless.com]
> > > Sent: 29. nóvember 2012 01:25
> > > To: The Stackless Python Mailing List
> > > Cc: Kristján Valur Jónsson
> > > Subject: Re: [Stackless] switching improvements
> > >
> > > Hi Kristjan,
> > >
> > > how about checking it into a branch firt, let us try and check that
> > > intensively, and after enough confidence merge it in?
> > > That's the way that works best for me with Mercurial/Git.
> > >
> > > If you don't like to put a branch into python.org, you also can do a
> > > clone on bitbucket easily and let us use that for review.
> > >
> > > cheers - chris
> > >
> > > On 28.11.12 16:11, Kristján Valur Jónsson wrote:
> > > Hello All.
> > > I was recently prompted to add a flag to stackless, a way to block
> > > all tasklet
> > switching.
> > > This springs from the way that we are embedding stackless python in
> > > an
> > game engine (UnReal) which sometimes makes callbacks into python.
> > Sometimes, this code will do nasty stuff that results in tasks
> > switching, causing havoc with the control flow of the game engine.
> > > To simplify this, I added a per-thread flag, switch_trap, which can
> > > be
> > controlled in a similar way to block_trap.  If the logic causes a
> > switch to be attempted, this should be trappable and the code should
> > be easily fixable, or we can otherwise deal with it.
> > >
> > > Anyway, doing this, adding it to slp_schedule_task(), and so on,
> > > uncovered
> > a subtle flaw in stackless:
> > > It turns out that slp_schedule_task() had no way of differentiating
> > > whether
> > an exception result from this call came as a result of a failure to
> > switch, or an exception being sent to the tasklet when it wakes up again.
> > >
> > > So, I have changed the interface to be able to do this properly.
> > > There are
> > other reasons why switching can fail, including memory allocation
> > failures and so on, so this seems like a necessary change.  I also
> > fixed code both in stacklesseval.c and taskletmodule and channelobject
> > to be able to cope with switch failure like this.
> > >
> > > Now:
> > > How does this sound to you?  The change is somewhat large and I
> > > would
> > hesitate to simply check it in without some sort of review or
> > otherwise approval.  Any suggestions?
> > >
> > > K
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > >
> > > Stackless mailing list
> > >
> > > Stackless at stackless.com<mailto:Stackless at stackless.com>
> > >
> > > http://www.stackless.com/mailman/listinfo/stackless
> > >
> > >
> > >
> > >
> > > --
> > >
> > > Christian Tismer             :^)
> > <mailto:tismer at stackless.com><mailto:tismer at stackless.com>
> > >
> > > Software Consulting          :     Have a break! Take a ride on Python's
> > >
> > > Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
> > >
> > > 14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
> > >
> > > phone +49 173 24 18 776  fax +49 (30) 700143-0023
> > >
> > > PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
> > >
> > >        whom do you want to sponsor today?   http://www.stackless.com/
> >
> >
> > --
> > Christian Tismer             :^)   <mailto:tismer at stackless.com>
> > Software Consulting          :     Have a break! Take a ride on Python's
> > Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
> > 14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
> > phone +49 173 24 18 776  fax +49 (30) 700143-0023
> > PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
> >        whom do you want to sponsor today?   http://www.stackless.com/
> >
> 
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless


More information about the Stackless mailing list