[Stackless] Tasklet cleanup?
Kristján Valur Jónsson
kristjan at ccpgames.com
Mon May 21 13:21:26 CEST 2012
It is due to this code here, in taskletojbect:
tasklet_traverse(PyTaskletObject *t, visitproc visit, void *arg)
PyThreadState *ts = PyThreadState_GET();
/* tasklets that need to be switched to for the kill, can't be collected.
* Only trivial decrefs are allowed during GC collect
PyObject_GC_Collectable((PyObject *)t, visit, arg, 0);
What can be done during garbage collection has always been a bit restricted. Tranditionally, gcmodule disallows all objects with __del__ methods and puts them in gc.garbage. I have recently discovered, by asking on python-dev, that this is not due to a fundamental fragility of the garbage collection process, but rather the fact that it is likely that __del__ methods invoked during garbage collection will cause exceptions, since the members of the cycle would refer to each other. See this discussion:
Anyway, for tasklets, the problem is that when killing tasklets, we have to "switch" to them. If other tasklets are runnable, this may kick off a sequence of running tasklets, there is no _guarantee_ that we will switch immediately back to the tasklet currently doing the garbage collection.
In the past, we have had a number of crashes related to this. I don't have the details anymore.
So, anyway, at one point I decided that it were safer to put such tasklets into gc.garbage.
It is possible that my fears are unfounded. We could possibly engineer tasklet.kill() in such a way that we are very sure that no other tasklets will run and we will switch immediately back to the originating tasklet.
> -----Original Message-----
> From: Richard Tew [mailto:richard.m.tew at gmail.com]
> Sent: 20. maí 2012 00:06
> To: The Stackless Python Mailing List
> Cc: Kristján Valur Jónsson
> Subject: Re: [Stackless] Tasklet cleanup?
> On Sun, May 20, 2012 at 10:29 AM, Sylvain Prat <sylvain.prat at gmail.com>
> > I'm wondering how tasklets can clean themselves up when they are
> > destroyed due to garbage collection (i.e. when they are not in the
> > runnables and not referenced by any object anymore). Greenlet solves
> > this problem by raising a GreenletExit exception in the greenlet's run
> > function when the greenlet is about to die due to garbage collection.
> > However, in stackless, it seems that no TaskletExit exception is
> > raised when the tasklet is about to die, so we can't simply use a
> > try/finally in the tasklet's callable to clean up resources.
> Greenlet is derived from Stackless, so similarly TaskletExit should be raised
> on a tasklet that is being garbage collected.
> > I tried to wrap my tasklet in a parent object which has the same
> > lifespan as my tasklet and has a __del__ function for cleaning up, but
> > I keep having problems with circular references (the wrapper/parent
> > object also provides the callable of the tasklet, i.e. a bound method)
> > that make the tasklet/parent object uncollectable (circular references
> > : wrapper --> tasklet --> stackless machinery? --> callable stack
> > frame (bound method of wrapper) --> wrapper). Same problem when
> > to inherit from tasklet.
> > So, how can I clean up resources in tasklets? (I'm pretty sure I've
> > missed something obvious)
> I've attached two example scripts where a tasklet dies, but does not get the
> TaskletExit raised on it. This is something Kristjan Valur has been working on,
> but there have been upsides and downsides to the different approaches and
> as I understand it the ideal solution to tasklet destruction is yet to be found.
> Kristjan, why is TaskletExit not being raised? Any ideas?
More information about the Stackless