[Stackless] Tasklet cleanup?

Kristján Valur Jónsson kristjan at ccpgames.com
Mon May 21 13:21:26 CEST 2012


Well.
It is due to this code here, in taskletojbect:

static int
tasklet_traverse(PyTaskletObject *t, visitproc visit, void *arg)
{
    PyFrameObject *f;
    PyThreadState *ts = PyThreadState_GET();

    /* tasklets that need to be switched to for the kill, can't be collected.
     * Only trivial decrefs are allowed during GC collect
     */
    if (tasklet_has_c_stack(t))
        PyObject_GC_Collectable((PyObject *)t, visit, arg, 0);

What can be done during garbage collection has always been a bit restricted.  Tranditionally, gcmodule disallows all objects with __del__ methods and puts them in gc.garbage.  I have recently discovered, by asking on python-dev, that this is not due to a fundamental fragility of the garbage collection process, but rather the fact that it is likely that __del__ methods invoked during garbage collection will cause exceptions, since the members of the cycle would refer to each other.  See this discussion:  
http://grokbase.com/p/python/python-dev/124hc36dtk/issue-9141-finalizers-and-gc-module

Anyway, for tasklets, the problem is that when killing tasklets, we have to "switch" to them.  If other tasklets are runnable, this may kick off a sequence of running tasklets, there is no _guarantee_ that we will switch immediately back to the tasklet currently doing the garbage collection.
In the past, we have had a number of crashes related to this.  I don't have the details anymore.

So, anyway, at one point I decided that it were safer to put such tasklets into gc.garbage.

It is possible that my fears are unfounded.  We could possibly engineer tasklet.kill() in such a way that we are very sure that no other tasklets will run and we will switch immediately back to the originating tasklet.

K



> -----Original Message-----
> From: Richard Tew [mailto:richard.m.tew at gmail.com]
> Sent: 20. maí 2012 00:06
> To: The Stackless Python Mailing List
> Cc: Kristján Valur Jónsson
> Subject: Re: [Stackless] Tasklet cleanup?
> 
> On Sun, May 20, 2012 at 10:29 AM, Sylvain Prat <sylvain.prat at gmail.com>
> wrote:
> > I'm wondering how tasklets can clean themselves up when they are
> > destroyed due to garbage collection (i.e. when they are not in the
> > runnables and not referenced by any object anymore). Greenlet solves
> > this problem by raising a GreenletExit exception in the greenlet's run
> > function when the greenlet is about to die due to garbage collection.
> > However, in stackless, it seems that no TaskletExit exception is
> > raised when the tasklet is about to die, so we can't simply use a
> > try/finally in the tasklet's callable to clean up resources.
> 
> Greenlet is derived from Stackless, so similarly TaskletExit should be raised
> on a tasklet that is being garbage collected.
> 
> > I tried to wrap my tasklet in a parent object which has the same
> > lifespan as my tasklet and has a __del__ function for cleaning up, but
> > I keep having problems with circular references (the wrapper/parent
> > object also provides the callable of the tasklet, i.e. a bound method)
> > that make the tasklet/parent object uncollectable (circular references
> > : wrapper --> tasklet --> stackless machinery? --> callable stack
> > frame (bound method of wrapper) --> wrapper). Same problem when
> trying
> > to inherit from tasklet.
> >
> > So, how can I clean up resources in tasklets? (I'm pretty sure I've
> > missed something obvious)
> 
> I've attached two example scripts where a tasklet dies, but does not get the
> TaskletExit raised on it.  This is something Kristjan Valur has been working on,
> but there have been upsides and downsides to the different approaches and
> as I understand it the ideal solution to tasklet destruction is yet to be found.
> 
> Kristjan, why is TaskletExit not being raised?  Any ideas?
> 
> Cheers,
> Richard.




More information about the Stackless mailing list