[Stackless] stackless context managers

Christian Tismer tismer at stackless.com
Thu Oct 10 17:25:17 CEST 2013


Hey Anselm,

this is really fascinating to read.

cheers - chris

On 05.09.13 12:00, Anselm Kruis wrote:
> Hi,
>
> all in all it is quite simple. As you already noted, the context manager is just syntactical sugar. I really only need id to hide complexity from our customers.
>
> So, how does the migration work:
>
> The originating side runs tasklet t1.
>
> def f():
>     do_local_work()
>     with get_context_manager():
>        do_remote_work()
>     more_local_work()
>
> t1=tasklet(f)()
>
> The context manager delegates the work to the main tasklet.
> It looks roughly looks like
>
> def __enter__(self):
>     stackless.schedule_remove(self)  # (1)
>     return None
>
> def __exit__(self, exc_type, exc_value, traceback):
>     tmp = stackless.schedule_remove((self, exc_type, exc_value, traceback)) # (2)
>     return self.end_exit(tmp)
>
> After __enter__ transferred control back to the main tasklet, the main tasklet
> - pickles (t1, additional data), result is string p1
> - unbinds t1: t1.bind(None)
> - uses RPyC to run p on the remote site
>
> The remote side:
> - unpickles p1, result is tasklet t2
> - t2.insert()
> - stackless.run()
> - t2 runs till (2)
> - the remote side now pickles t2, result is string p2
> - t2.bind(None)
> - t2 = None
> - return p2 to the originating side
>
> The originating side
> - unpickles p2, result is tasklet t3
> - replaces the original t1 by t3: t1 = t3
> - deletes t3: t3 = None
> - continues t1: t1.insert(); stackless.run()
>
> I use the extended pickler form the sPickle library. This pickler can replace
> certain resource-objects (i.e. open files) by RPyC-Proxy objects on the fly.
>
> Cheers
>    Anselm
>
>
>
> Am 05.09.2013 08:48, schrieb Kristján Valur Jónsson:
>> Hi,
>> I think this is a better spot to discuss this topic than the checkin comments in Anselm's repo:
>> https://bitbucket.org/akruis/fg2python/commits/852868afc498e800e4091e83aa818bf5d7b35939#comment-406276
>>
>> So, Anselm said:
>>
>> Sorry for the delayed answer, here is the use case.
>>
>> I tried to write a context manager that changes the execution host of the body. In __enter__ I pickle the tasklet and transfer the pickle to another process and unpickle it there. Then the tasklet executes until it calls __exit__. In exit I pickle it again and transfer it back to the original process.
>>
>> This is really interesting.  How did it pan out?  Also, How does it work on the originating site?  A context manager does not (currently) allow the body code to be skipped, which I presume is what must happen on the "source" side.  (although, this here patch<http://bugs.python.org/issue18677> would allow that)
>>
>> I'm very much a fan of context managers these days.  I just added stackless.atomic(), a built-in to provide performance for this ubiquitous thing.
>>
>>
>>
>> Hi,
>>
>> I think this is a better spot to discuss this topic than the checkin
>> comments in Anselm’s repo:
>>
>> https://bitbucket.org/akruis/fg2python/commits/852868afc498e800e4091e83aa818bf5d7b35939#comment-406276
>>
>> So, Anselm said:
>>
>> Sorry for the delayed answer, here is the use case.
>>
>> I tried to write a context manager that changes the execution host of
>> the body. In|__enter__|I pickle the tasklet and transfer the pickle to
>> another process and unpickle it there. Then the tasklet executes until
>> it calls|__exit__|. In exit I pickle it again and transfer it back to
>> the original process.
>>
>> This is really interesting.  How did it pan out?  Also, How does it work
>> on the originating site?  A context manager does not (currently) allow
>> the body code to be skipped, which I presume is what must happen on the
>> “source” side.  (although, this here patch
>> <http://bugs.python.org/issue18677> would allow that)
>>
>> I’m very much a fan of context managers these days.  I just added
>> stackless.atomic(), a built-in to provide performance for this
>> ubiquitous thing.
>>
>>
>>
>> _______________________________________________
>> Stackless mailing list
>> Stackless at stackless.com
>> http://www.stackless.com/mailman/listinfo/stackless
>>


-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/




More information about the Stackless mailing list