[Stackless] stackless context managers
Kristján Valur Jónsson
kristjan at ccpgames.com
Fri Oct 11 13:46:47 CEST 2013
Yes.
I wish that Guido were more positive towards adding features to context managers.
For example, this is one thing I'd like to be able to do:
with run_this_in_new_tasklet():
print 'hello from', stackless.getcurrent()
I have this patch here, http://bugs.python.org/issue18677
wich adds the capability to context managers to not run the context code, but I'm told that Guido doesn't want that.
Here is another example:
class OnceLock(threading.RWLock):
def __init__(self):
self.active = False
class OnceError(RuntimeError): pass
@contextmanager
def once2(lock):
if lock.active:
raise OnceError
lock.active = True
try:
yield
finally:
lock.active = False
@contextmanager
def once1(lock):
try:
with lock:
yield
except OnceError:
pass
lock = OnceLock()
def myfunc():
with once1(lock), once2(lock):
#this code is run only once. recursion into here will be ignored
foo()
bar()
Here we have a context manager that locks a section, and then runs it, but only if no one else is running it. But because a single context manager cannot skip code without raising an exception, we have to have an outer one which only silences exceptions.
I would like to be able to do:
with lock: #redefine __enter__ and __exit__ of OnceLock to do this right
foo()
bar()
I think this is a fairly non-radical extension of the context manager concept.
We would write the new one like this:
@contextmanager
def onceLocked(lock):
with lock:
if lock.active:
raise ContextManagerExit # special exception that leaves the context manager but is silenced
lock.active = True
try:
yield
except:
lock.active = False
But imagine if context managers could be written differently. What if a context manager were given a _callable_, representing the code?
like this:
class NewContextManager(object):
def __init__(self, lock):
self.lock = lock
def __contextcall__(self, code):
with lock:
if lock.active:
return
lock.active = True
try:
return code(None) # optionally pass value to the code as in "with foo() as X"
finally:
lock.active = False
The cool thing here though, is that "code" could, for example, be run on a different tasklet. Or a different thread. Or a different universe.
This sort of thing would need compiler and syntax support, of course. The compiler would need to create an anonymous function object. The return value out of "code" would be some token that could be special if the code returned....
A pipedream, of course. But this would allow enormous, indeed, complete, flexibility for context managers.....
K
> -----Original Message-----
> From: stackless-bounces at stackless.com [mailto:stackless-
> bounces at stackless.com] On Behalf Of Christian Tismer
> Sent: 10. október 2013 15:25
> To: The Stackless Python Mailing List
> Cc: Anselm Kruis
> Subject: Re: [Stackless] stackless context managers
>
> Hey Anselm,
>
> this is really fascinating to read.
>
> cheers - chris
>
> On 05.09.13 12:00, Anselm Kruis wrote:
> > Hi,
> >
> > all in all it is quite simple. As you already noted, the context manager is just
> syntactical sugar. I really only need id to hide complexity from our customers.
> >
> > So, how does the migration work:
> >
> > The originating side runs tasklet t1.
> >
> > def f():
> > do_local_work()
> > with get_context_manager():
> > do_remote_work()
> > more_local_work()
> >
> > t1=tasklet(f)()
> >
> > The context manager delegates the work to the main tasklet.
> > It looks roughly looks like
> >
> > def __enter__(self):
> > stackless.schedule_remove(self) # (1)
> > return None
> >
> > def __exit__(self, exc_type, exc_value, traceback):
> > tmp = stackless.schedule_remove((self, exc_type, exc_value,
> traceback)) # (2)
> > return self.end_exit(tmp)
> >
> > After __enter__ transferred control back to the main tasklet, the main
> > tasklet
> > - pickles (t1, additional data), result is string p1
> > - unbinds t1: t1.bind(None)
> > - uses RPyC to run p on the remote site
> >
> > The remote side:
> > - unpickles p1, result is tasklet t2
> > - t2.insert()
> > - stackless.run()
> > - t2 runs till (2)
> > - the remote side now pickles t2, result is string p2
> > - t2.bind(None)
> > - t2 = None
> > - return p2 to the originating side
> >
> > The originating side
> > - unpickles p2, result is tasklet t3
> > - replaces the original t1 by t3: t1 = t3
> > - deletes t3: t3 = None
> > - continues t1: t1.insert(); stackless.run()
> >
> > I use the extended pickler form the sPickle library. This pickler can
> > replace certain resource-objects (i.e. open files) by RPyC-Proxy objects on
> the fly.
> >
> > Cheers
> > Anselm
> >
> >
> >
> > Am 05.09.2013 08:48, schrieb Kristján Valur Jónsson:
> >> Hi,
> >> I think this is a better spot to discuss this topic than the checkin comments
> in Anselm's repo:
> >>
> https://bitbucket.org/akruis/fg2python/commits/852868afc498e800e4091e
> >> 83aa818bf5d7b35939#comment-406276
> >>
> >> So, Anselm said:
> >>
> >> Sorry for the delayed answer, here is the use case.
> >>
> >> I tried to write a context manager that changes the execution host of the
> body. In __enter__ I pickle the tasklet and transfer the pickle to another
> process and unpickle it there. Then the tasklet executes until it calls
> __exit__. In exit I pickle it again and transfer it back to the original process.
> >>
> >> This is really interesting. How did it pan out? Also, How does it
> >> work on the originating site? A context manager does not (currently)
> >> allow the body code to be skipped, which I presume is what must
> >> happen on the "source" side. (although, this here
> >> patch<http://bugs.python.org/issue18677> would allow that)
> >>
> >> I'm very much a fan of context managers these days. I just added
> stackless.atomic(), a built-in to provide performance for this ubiquitous thing.
> >>
> >>
> >>
> >> Hi,
> >>
> >> I think this is a better spot to discuss this topic than the checkin
> >> comments in Anselm’s repo:
> >>
> >>
> https://bitbucket.org/akruis/fg2python/commits/852868afc498e800e4091e
> >> 83aa818bf5d7b35939#comment-406276
> >>
> >> So, Anselm said:
> >>
> >> Sorry for the delayed answer, here is the use case.
> >>
> >> I tried to write a context manager that changes the execution host of
> >> the body. In|__enter__|I pickle the tasklet and transfer the pickle
> >> to another process and unpickle it there. Then the tasklet executes
> >> until it calls|__exit__|. In exit I pickle it again and transfer it
> >> back to the original process.
> >>
> >> This is really interesting. How did it pan out? Also, How does it
> >> work on the originating site? A context manager does not (currently)
> >> allow the body code to be skipped, which I presume is what must
> >> happen on the “source” side. (although, this here patch
> >> <http://bugs.python.org/issue18677> would allow that)
> >>
> >> I’m very much a fan of context managers these days. I just added
> >> stackless.atomic(), a built-in to provide performance for this
> >> ubiquitous thing.
> >>
> >>
> >>
> >> _______________________________________________
> >> Stackless mailing list
> >> Stackless at stackless.com
> >> http://www.stackless.com/mailman/listinfo/stackless
> >>
>
>
> --
> Christian Tismer :^) <mailto:tismer at stackless.com>
> Software Consulting : Have a break! Take a ride on Python's
> Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/
> 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de
> phone +49 173 24 18 776 fax +49 (30) 700143-0023
> PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
> whom do you want to sponsor today? http://www.stackless.com/
>
>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless
More information about the Stackless
mailing list