[Stackless] Stackless breaks generators that do not yield

[Sune Kirkeby]

[ Christian Tismer ]
> Sune Kirkeby wrote:
> >[sune at melpomene:/usr/src/stackless-python/stackless/src]$./python bloop
> >Adding parser accelerators ...
> >Done.
> >Python/ceval.c:165 negative ref count -1
> >Python/ceval.c:129 negative ref count -1
> >Traceback (most recent call last):
> >  File "bloop", line 5, in ?
> >    g().next()
> >StopIteration
> >[5812 refs]
> >python: Objects/frameobject.c:510: PyFrame_Fini: Assertion `numfree == 0' 
> >failed.
> >Aborted
> >[sune at melpomene:/usr/src/stackless-python/stackless/src]$
> 
> I believe you are referring to the current CVS version?

Yup, that I am.

> Thank you very much, this fits nicely in the set of
> problems which I'm just working on.
> I'm currently struggling with refcounts. The policy of
> decreffing frames has changed quite much since generators
> were introduced. Before, a frame derefenced itself.
> Now it is done by the caller, and the callers of
> generator frames know when to drop the frame and when not.

I think I have found the problem wrt generators. What happens is
that slp_dispath_frame Py_DECREFs frames when they are done, but
in some instances (when the generator does not yield, I suspect)
the frame has already been garbage-collected. Needless to say this
makes things go spectacularly bad from time to time.

I solved the problem by Py_INCREFing the frame in gen_new and
Py_DECREFfing it in gen_dealloc, but only if the frame is not done,
when it is done slp_dispatch_frame will have Py_DECREFfed it
already.

This seems to work in my code (which I take as a good sign, since
it has a history of stumbling upon pathological bugs).

Attached is the patch.  Comments?

-- 
Sune Kirkeby | ``Det er helt retfærdigt, at teleselskaberne kommer til at
             | betale en afgift for den trafik, som vi generer på deres
             | telelinier''        -- Peter Budtz, direktør Krak Internet
-------------- next part --------------
Index: Python/ceval.c
===================================================================
RCS file: /home/cvs/stackless/src/Python/ceval.c,v
retrieving revision 1.23
diff -u -r1.23 ceval.c
--- Python/ceval.c	2002/12/02 07:17:34	1.23
+++ Python/ceval.c	2003/03/17 10:33:02
@@ -112,6 +112,9 @@
 		Py_DECREF(f);
 		return NULL;
 	}
+#ifdef STACKLESS
+        Py_INCREF(f);
+#endif
 	gen->gi_frame = f;
 	gen->gi_running = 0;
 	return (PyObject *)gen;
@@ -126,6 +129,11 @@
 static void
 gen_dealloc(genobject *gen)
 {
+#ifdef STACKLESS
+        if(! gen->gi_frame->f_flags.is_done) {
+            Py_DECREF(gen->gi_frame);
+        }
+#endif
 	Py_DECREF(gen->gi_frame);
 	PyObject_Del(gen);
 }
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 2377 bytes
Desc: not available
URL: <http://www.stackless.com/pipermail/stackless/attachments/20030317/57468c0b/attachment.pgp>