[Stackless] I need Input (was: Stackless breaks generators that do not yield)

Christian Tismer tismer at tismer.com
Wed Mar 19 00:08:21 CET 2003

Sune Kirkeby wrote:
> [ Christian Tismer ]

>>>b remove the decreffing code from frame_dispatch
>>> and revert the policy to "a frame has to take
>>> care for itself", while keeping generators untouched

> I would have given my .2e earlier, but I was busy hacking away on
> a patch, that implements what I think you meant by option (b). The
> patch removes the is_done flag, and add normal "you own it, you
> DECREF it rules" to frames.


> This means there are three places frames need to be DECREFed, (1) in
> gen_dealloc, (2) in PyEval_EvalFrame when a frame is reentered after
> a function-call (the Py_UnwindToken'ing kind) and (3) in
> PyEval_EvalCodeEx.
> The patched stackless passes all tests but minidom, but I can't
> figure out why it fails (I suspect the test or the expected
> output-file to be b0rked, since neither stock 2.2.2 nor the patched
> stackless produce any output for the test...) And I can't seem to
> trigger any memory-leaks with my code -- which is a first :)

Sounds really promising.
I will check out the prior version tomorrow and
apply your patch, to see how it feels.

> I'll have a look at your code rsn(tm).  Comments?

Well, I'm about to get to Stackless 3.0, which means
that almost *all* functions will get a continuation-
passing style companion. Especially things like
slp_run_tasklet or slp_tasklet_end will have to be turned
into f_execute functions of tiny frames, see my larger
check-in from this evening.
The reason is that I want t get rid of most of this
brute-force stack switching and get back to the elegance
of Stackless 1.0, where everythign was done by *returning*
to proper frames.

In 3,0 that is, the tasklets will no longer be created
by some magic, but a special frame will be created and pushed
that is able to run a tasklet. Frame chains will no longer
be disjoint, ended by NULL, but they will all end up in
one special frame which a special f-execute that redirects
all returns to the one main tasklet. And so on.
So the concept is to do the complete callback game, and
only to switch C stacks if it is unavoidable.

The decision about "who onws the frame" is harder than
I thought, sine it has to be complete:
Either, the frame is always self-owned, or it is never.
Reason? If a frame may destruct itself, a calling function
can never inspect it and expect a proper frame, unless
it puts extra references, which I hate.
Or, a frame may never destruct itself, unless it is done
in a simple recursive function call context; compare the
special frame handling in pyexpat.c.
In the second case, a returning, finished frame would
always set the flags.done flag in order to get destroyed.

Do you think things are simpler with self-destructive
frames, and do you think this can be easily implemented,
in the context of what I'm going to do concerning
the new frame-pushing concept?
I'd really like to know your opinion, since it's hard
to wind my mind around it. Finally, the "better"
solution should be choosen, for every possible definition
of better.

thanks & cheers - chris

Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

Stackless mailing list
Stackless at www.tismer.com

More information about the Stackless mailing list