[Stackless] switching improvements

Christian Tismer tismer at stackless.com
Sat Jan 5 18:43:00 CET 2013


Hi ristjan,

very nice, I'm just reading it.

One funny effect of the new repos:
There are a lot of commits listed as 2 hours old. It is funny
because some of them are from Guido (seldom today).

Actually, I found this as being from 1994 :-)

https://bitbucket.org/krisvale/switching2/commits/65cc727f1436c079727ffdd70916cb45

This did not happen the last time when you created a temp repos.
Do you remember what was different:

cheers - chris


On 05.01.13 17:04, Kristján Valur Jónsson wrote:
> There is a new repo,
> https://bitbucket.org/krisvale/switching2
> I've split the changes into a bunch of smaller incremental ones.
>
> I also found a problem with tasklet.raise_exception which I fixed.  I'm working on checking if it works at all for 3.2 and 2.7, and am installing 2008 on my workstation from remote to be able to test those platforms.
>
> I created a number of tickets on www.stackless.com as well, #6, #7 and #8 considering that.
>
> K
>
> ________________________________________
> Frá: stackless-bounces at stackless.com [stackless-bounces at stackless.com] fyrir hönd Kristján Valur Jónsson [kristjan at ccpgames.com]
> Sent: 7. desember 2012 10:02
> To: Christian Tismer
> Cc: The Stackless Python Mailing List
> Efni: Re: [Stackless] switching improvements
>
> Ok, I'll split this up, and put the macro changes and the new utility apis in as separate commitments to the main branch.  That should make this patch smaller.
> K
>
>> -----Original Message-----
>> From: Christian Tismer [mailto:tismer at stackless.com]
>> Sent: 5. desember 2012 23:36
>> To: Kristján Valur Jónsson
>> Cc: The Stackless Python Mailing List
>> Subject: Re: [Stackless] switching improvements
>>
>> I'm also waiting for more input.
>>
>> I have no time for testing before the weekend.
>> and I'd appreciate to get opinions.
>> Also I would like to split the patch into the pieces that were depicted below.
>>
>> Besides the enhancements to the API and implementation, what is your
>> opinion about switch_trap?
>>
>> I'm like +0.5 or better, not a real use-case but also no objection.
>>
>> What do people think?
>>
>> cheers - Chris
>>
>>
>> On 05.12.12 13:55, Kristján Valur Jónsson wrote:
>>> Any comments yet?
>>> The impetus for this change is to be able to add the stackless.switch_trap()
>> feature.  This is something our engineers are asking for.  I obviously don't
>> want to bring the CCP stackless branch too far away from the official one,
>> which is why I was proposing to add this to the official stackless too.
>>> K
>>>
>>>> -----Original Message-----
>>>> From: stackless-bounces at stackless.com [mailto:stackless-
>>>> bounces at stackless.com] On Behalf Of Kristján Valur Jónsson
>>>> Sent: 30. nóvember 2012 08:45
>>>> To: Christian Tismer
>>>> Cc: The Stackless Python Mailing List
>>>> Subject: Re: [Stackless] switching improvements
>>>>
>>>> Thanks for looking this over, Christian.
>>>> Yes, there are several things in there, really:
>>>> 1) Macro fixups,
>>>> 2) The adding of new methods to remove/reinsert tasks from channels
>>>> and the runnable queue.
>>>> 3) A change to slp_schedule_task, explicitily signalling a failure to
>>>> switch, rather than just an exploded bomb on return.
>>>> 4) Ability to "undo" changes to the state if switching fails
>>>> 5) Various apis can now fail and return with an unchanged state:
>>>> Channel actions, stackless.schedule, stackless.run.  I probably  need
>>>> to add tasklet.run to this list.  Oh, and tasklet.kill, tasklet.raise_exception.
>>>> 6) Unit tests for the failure modes.
>>>>
>>>>
>>>> I thought a bit about the fact that these apis have no semantic
>>>> difference between an error that occurs as the result of a switch,
>>>> and an error that is "sent" to the tasklet.
>>>> E.g. channel.recv() can fail with a RuntimeError (perhaps it is time
>>>> Stackless got its own errors, no?) or MemoryError if it cannot
>>>> switch, but it can also fail with any error if another tasklet
>>>> doesn't handle its error, or someone sends it an exception
>>>> (tasklet.raise_exception).  The caller cannot know which case it is.  But
>> maybe it is unimportant.
>>>> K
>>>>
>>>>> -----Original Message-----
>>>>> From: Christian Tismer [mailto:tismer at stackless.com]
>>>>> Sent: 29. nóvember 2012 23:01
>>>>> To: Kristján Valur Jónsson
>>>>> Cc: The Stackless Python Mailing List
>>>>> Subject: Re: [Stackless] switching improvements
>>>>>
>>>>> Yeah,
>>>>>
>>>>> I looked quite a lot into it, but did not have time to try it, yet.
>>>>>
>>>>> The patch is rather large and needs more people to review and try it out.
>>>>>
>>>>> There are quite some changes which make pretty much sense in any
>>>>> case,
>>>>> btw.:
>>>>> Things like the improved macro syntax should go in directly with no
>> doubt.
>>>>> It would be good if the patch would not mix style improvements and
>>>>> functional changes.
>>>>> Also, some changes of the call parameters make sense and should
>>>>> probably go in without hesitating.
>>>>>
>>>>> So I'd split it in two or maybe three patches instead of one.
>>>>>
>>>>> If you don't mind, I can try that in a few days.
>>>>>
>>>>> cheers - chris
>>>>>
>>>>> On 11/29/12 12:27 PM, Kristján Valur Jónsson wrote:
>>>>>> Ok, take a look at
>>>>>> https://bitbucket.org/krisvale/stacklessswitching
>>>>>>
>>>>>>
>>>>>> From: Christian Tismer [mailto:tismer at stackless.com]
>>>>>> Sent: 29. nóvember 2012 01:25
>>>>>> To: The Stackless Python Mailing List
>>>>>> Cc: Kristján Valur Jónsson
>>>>>> Subject: Re: [Stackless] switching improvements
>>>>>>
>>>>>> Hi Kristjan,
>>>>>>
>>>>>> how about checking it into a branch firt, let us try and check that
>>>>>> intensively, and after enough confidence merge it in?
>>>>>> That's the way that works best for me with Mercurial/Git.
>>>>>>
>>>>>> If you don't like to put a branch into python.org, you also can do
>>>>>> a clone on bitbucket easily and let us use that for review.
>>>>>>
>>>>>> cheers - chris
>>>>>>
>>>>>> On 28.11.12 16:11, Kristján Valur Jónsson wrote:
>>>>>> Hello All.
>>>>>> I was recently prompted to add a flag to stackless, a way to block
>>>>>> all tasklet
>>>>> switching.
>>>>>> This springs from the way that we are embedding stackless python in
>>>>>> an
>>>>> game engine (UnReal) which sometimes makes callbacks into python.
>>>>> Sometimes, this code will do nasty stuff that results in tasks
>>>>> switching, causing havoc with the control flow of the game engine.
>>>>>> To simplify this, I added a per-thread flag, switch_trap, which can
>>>>>> be
>>>>> controlled in a similar way to block_trap.  If the logic causes a
>>>>> switch to be attempted, this should be trappable and the code should
>>>>> be easily fixable, or we can otherwise deal with it.
>>>>>> Anyway, doing this, adding it to slp_schedule_task(), and so on,
>>>>>> uncovered
>>>>> a subtle flaw in stackless:
>>>>>> It turns out that slp_schedule_task() had no way of differentiating
>>>>>> whether
>>>>> an exception result from this call came as a result of a failure to
>>>>> switch, or an exception being sent to the tasklet when it wakes up again.
>>>>>> So, I have changed the interface to be able to do this properly.
>>>>>> There are
>>>>> other reasons why switching can fail, including memory allocation
>>>>> failures and so on, so this seems like a necessary change.  I also
>>>>> fixed code both in stacklesseval.c and taskletmodule and
>>>>> channelobject to be able to cope with switch failure like this.
>>>>>> Now:
>>>>>> How does this sound to you?  The change is somewhat large and I
>>>>>> would
>>>>> hesitate to simply check it in without some sort of review or
>>>>> otherwise approval.  Any suggestions?
>>>>>
>> --
>> Christian Tismer             :^)   <mailto:tismer at stackless.com>
>> Software Consulting          :     Have a break! Take a ride on Python's
>> Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
>> 14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
>> phone +49 173 24 18 776  fax +49 (30) 700143-0023
>> PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
>>         whom do you want to sponsor today?   http://www.stackless.com/
>>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless


-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/




More information about the Stackless mailing list