[Stackless] switching improvements

Sat Jan 5 17:04:04 CET 2013

There is a new repo, 
https://bitbucket.org/krisvale/switching2
I've split the changes into a bunch of smaller incremental ones.  

I also found a problem with tasklet.raise_exception which I fixed.  I'm working on checking if it works at all for 3.2 and 2.7, and am installing 2008 on my workstation from remote to be able to test those platforms.

I created a number of tickets on www.stackless.com as well, #6, #7 and #8 considering that.

K

________________________________________
Frá: stackless-bounces at stackless.com [stackless-bounces at stackless.com] fyrir hönd Kristján Valur Jónsson [kristjan at ccpgames.com]
Sent: 7. desember 2012 10:02
To: Christian Tismer
Cc: The Stackless Python Mailing List
Efni: Re: [Stackless] switching improvements

Ok, I'll split this up, and put the macro changes and the new utility apis in as separate commitments to the main branch.  That should make this patch smaller.
K

> -----Original Message-----
> From: Christian Tismer [mailto:tismer at stackless.com]
> Sent: 5. desember 2012 23:36
> To: Kristján Valur Jónsson
> Cc: The Stackless Python Mailing List
> Subject: Re: [Stackless] switching improvements
>
> I'm also waiting for more input.
>
> I have no time for testing before the weekend.
> and I'd appreciate to get opinions.
> Also I would like to split the patch into the pieces that were depicted below.
>
> Besides the enhancements to the API and implementation, what is your
> opinion about switch_trap?
>
> I'm like +0.5 or better, not a real use-case but also no objection.
>
> What do people think?
>
> cheers - Chris
>
>
> On 05.12.12 13:55, Kristján Valur Jónsson wrote:
> > Any comments yet?
> > The impetus for this change is to be able to add the stackless.switch_trap()
> feature.  This is something our engineers are asking for.  I obviously don't
> want to bring the CCP stackless branch too far away from the official one,
> which is why I was proposing to add this to the official stackless too.
> >
> > K
> >
> >> -----Original Message-----
> >> From: stackless-bounces at stackless.com [mailto:stackless-
> >> bounces at stackless.com] On Behalf Of Kristján Valur Jónsson
> >> Sent: 30. nóvember 2012 08:45
> >> To: Christian Tismer
> >> Cc: The Stackless Python Mailing List
> >> Subject: Re: [Stackless] switching improvements
> >>
> >> Thanks for looking this over, Christian.
> >> Yes, there are several things in there, really:
> >> 1) Macro fixups,
> >> 2) The adding of new methods to remove/reinsert tasks from channels
> >> and the runnable queue.
> >> 3) A change to slp_schedule_task, explicitily signalling a failure to
> >> switch, rather than just an exploded bomb on return.
> >> 4) Ability to "undo" changes to the state if switching fails
> >> 5) Various apis can now fail and return with an unchanged state:
> >> Channel actions, stackless.schedule, stackless.run.  I probably  need
> >> to add tasklet.run to this list.  Oh, and tasklet.kill, tasklet.raise_exception.
> >> 6) Unit tests for the failure modes.
> >>
> >>
> >> I thought a bit about the fact that these apis have no semantic
> >> difference between an error that occurs as the result of a switch,
> >> and an error that is "sent" to the tasklet.
> >> E.g. channel.recv() can fail with a RuntimeError (perhaps it is time
> >> Stackless got its own errors, no?) or MemoryError if it cannot
> >> switch, but it can also fail with any error if another tasklet
> >> doesn't handle its error, or someone sends it an exception
> >> (tasklet.raise_exception).  The caller cannot know which case it is.  But
> maybe it is unimportant.
> >>
> >> K
> >>
> >>> -----Original Message-----
> >>> From: Christian Tismer [mailto:tismer at stackless.com]
> >>> Sent: 29. nóvember 2012 23:01
> >>> To: Kristján Valur Jónsson
> >>> Cc: The Stackless Python Mailing List
> >>> Subject: Re: [Stackless] switching improvements
> >>>
> >>> Yeah,
> >>>
> >>> I looked quite a lot into it, but did not have time to try it, yet.
> >>>
> >>> The patch is rather large and needs more people to review and try it out.
> >>>
> >>> There are quite some changes which make pretty much sense in any
> >>> case,
> >>> btw.:
> >>> Things like the improved macro syntax should go in directly with no
> doubt.
> >>> It would be good if the patch would not mix style improvements and
> >>> functional changes.
> >>> Also, some changes of the call parameters make sense and should
> >>> probably go in without hesitating.
> >>>
> >>> So I'd split it in two or maybe three patches instead of one.
> >>>
> >>> If you don't mind, I can try that in a few days.
> >>>
> >>> cheers - chris
> >>>
> >>> On 11/29/12 12:27 PM, Kristján Valur Jónsson wrote:
> >>>> Ok, take a look at
> >>>> https://bitbucket.org/krisvale/stacklessswitching
> >>>>
> >>>>
> >>>> From: Christian Tismer [mailto:tismer at stackless.com]
> >>>> Sent: 29. nóvember 2012 01:25
> >>>> To: The Stackless Python Mailing List
> >>>> Cc: Kristján Valur Jónsson
> >>>> Subject: Re: [Stackless] switching improvements
> >>>>
> >>>> Hi Kristjan,
> >>>>
> >>>> how about checking it into a branch firt, let us try and check that
> >>>> intensively, and after enough confidence merge it in?
> >>>> That's the way that works best for me with Mercurial/Git.
> >>>>
> >>>> If you don't like to put a branch into python.org, you also can do
> >>>> a clone on bitbucket easily and let us use that for review.
> >>>>
> >>>> cheers - chris
> >>>>
> >>>> On 28.11.12 16:11, Kristján Valur Jónsson wrote:
> >>>> Hello All.
> >>>> I was recently prompted to add a flag to stackless, a way to block
> >>>> all tasklet
> >>> switching.
> >>>> This springs from the way that we are embedding stackless python in
> >>>> an
> >>> game engine (UnReal) which sometimes makes callbacks into python.
> >>> Sometimes, this code will do nasty stuff that results in tasks
> >>> switching, causing havoc with the control flow of the game engine.
> >>>> To simplify this, I added a per-thread flag, switch_trap, which can
> >>>> be
> >>> controlled in a similar way to block_trap.  If the logic causes a
> >>> switch to be attempted, this should be trappable and the code should
> >>> be easily fixable, or we can otherwise deal with it.
> >>>> Anyway, doing this, adding it to slp_schedule_task(), and so on,
> >>>> uncovered
> >>> a subtle flaw in stackless:
> >>>> It turns out that slp_schedule_task() had no way of differentiating
> >>>> whether
> >>> an exception result from this call came as a result of a failure to
> >>> switch, or an exception being sent to the tasklet when it wakes up again.
> >>>> So, I have changed the interface to be able to do this properly.
> >>>> There are
> >>> other reasons why switching can fail, including memory allocation
> >>> failures and so on, so this seems like a necessary change.  I also
> >>> fixed code both in stacklesseval.c and taskletmodule and
> >>> channelobject to be able to cope with switch failure like this.
> >>>> Now:
> >>>> How does this sound to you?  The change is somewhat large and I
> >>>> would
> >>> hesitate to simply check it in without some sort of review or
> >>> otherwise approval.  Any suggestions?
> >>>
>
> --
> Christian Tismer             :^)   <mailto:tismer at stackless.com>
> Software Consulting          :     Have a break! Take a ride on Python's
> Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
> 14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
> PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
>        whom do you want to sponsor today?   http://www.stackless.com/
>

_______________________________________________
Stackless mailing list
Stackless at stackless.com
http://www.stackless.com/mailman/listinfo/stackless