[Stackless] stackless context managers

Anselm Kruis a.kruis at science-computing.de
Thu Sep 5 12:00:34 CEST 2013


Hi,

all in all it is quite simple. As you already noted, the context manager is just syntactical sugar. I really only need id to hide complexity from our customers.

So, how does the migration work:

The originating side runs tasklet t1.

def f():
   do_local_work()
   with get_context_manager():
      do_remote_work()
   more_local_work()

t1=tasklet(f)()

The context manager delegates the work to the main tasklet.
It looks roughly looks like

def __enter__(self):
   stackless.schedule_remove(self)  # (1)
   return None

def __exit__(self, exc_type, exc_value, traceback):
   tmp = stackless.schedule_remove((self, exc_type, exc_value, traceback)) # (2)
   return self.end_exit(tmp)

After __enter__ transferred control back to the main tasklet, the main tasklet 
- pickles (t1, additional data), result is string p1
- unbinds t1: t1.bind(None)
- uses RPyC to run p on the remote site

The remote side:
- unpickles p1, result is tasklet t2
- t2.insert()
- stackless.run()
- t2 runs till (2)
- the remote side now pickles t2, result is string p2 
- t2.bind(None)
- t2 = None
- return p2 to the originating side

The originating side
- unpickles p2, result is tasklet t3
- replaces the original t1 by t3: t1 = t3
- deletes t3: t3 = None
- continues t1: t1.insert(); stackless.run()

I use the extended pickler form the sPickle library. This pickler can replace 
certain resource-objects (i.e. open files) by RPyC-Proxy objects on the fly.

Cheers
  Anselm



Am 05.09.2013 08:48, schrieb Kristján Valur Jónsson:
> Hi,
> I think this is a better spot to discuss this topic than the checkin comments in Anselm's repo:
> https://bitbucket.org/akruis/fg2python/commits/852868afc498e800e4091e83aa818bf5d7b35939#comment-406276
> 
> So, Anselm said:
> 
> Sorry for the delayed answer, here is the use case.
> 
> I tried to write a context manager that changes the execution host of the body. In __enter__ I pickle the tasklet and transfer the pickle to another process and unpickle it there. Then the tasklet executes until it calls __exit__. In exit I pickle it again and transfer it back to the original process.
> 
> This is really interesting.  How did it pan out?  Also, How does it work on the originating site?  A context manager does not (currently) allow the body code to be skipped, which I presume is what must happen on the "source" side.  (although, this here patch<http://bugs.python.org/issue18677> would allow that)
> 
> I'm very much a fan of context managers these days.  I just added stackless.atomic(), a built-in to provide performance for this ubiquitous thing.
> 
> 
> 
> Hi,
> 
> I think this is a better spot to discuss this topic than the checkin 
> comments in Anselm’s repo:
> 
> https://bitbucket.org/akruis/fg2python/commits/852868afc498e800e4091e83aa818bf5d7b35939#comment-406276
> 
> So, Anselm said:
> 
> Sorry for the delayed answer, here is the use case.
> 
> I tried to write a context manager that changes the execution host of 
> the body. In|__enter__|I pickle the tasklet and transfer the pickle to 
> another process and unpickle it there. Then the tasklet executes until 
> it calls|__exit__|. In exit I pickle it again and transfer it back to 
> the original process.
> 
> This is really interesting.  How did it pan out?  Also, How does it work 
> on the originating site?  A context manager does not (currently) allow 
> the body code to be skipped, which I presume is what must happen on the 
> “source” side.  (although, this here patch 
> <http://bugs.python.org/issue18677> would allow that)
> 
> I’m very much a fan of context managers these days.  I just added 
> stackless.atomic(), a built-in to provide performance for this 
> ubiquitous thing.
> 
> 
> 
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless
> 

-- 
 Dipl. Phys. Anselm Kruis                       science + computing ag
 Senior Solution Architect                      Ingolstädter Str. 22
 email A.Kruis at science-computing.de             80807 München, Germany
 phone +49 89 356386 874  fax 737               www.science-computing.de
-- 
Vorstandsvorsitzender/Chairman of the board of management:
Gerd-Lothar Leonhart
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Michael Heinrichs, 
Dr. Arno Steitz, Dr. Ingrid Zech
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196




More information about the Stackless mailing list