[Stackless] Old Error! API is broken since 2.4.3 (was: new error)

[Christian Tismer]

This error exists since 2006. I think we introduced it on the
Stackless sprint. I won't blame anybody, maybe it was me, no idea.

In any case, the Stackless API is broken. PyEval_EvalFrame cannot
be used.

The code is working in the path

svn+ssh://pythondev@svn.python.org/stackless/Python-2.4.2/src/Python/ceval.c

and broken in

svn+ssh://pythondev@svn.python.org/stackless/Python-2.4.2/dev/Python/ceval.c
svn+ssh://pythondev@svn.python.org/stackless/Python-2.4.3/dev/Python/ceval.c

and all higher versions in branches, tags, and trunk of course.

This means: nobody ever used the API explicitly since then, but me. :-)

The reason why this is broken:
In 2.4.2/src we had no retval parameter. It was a local, initialized to 
NULL,
but it was then initialized here:

PyObject *
PyEval_EvalFrame_value(PyFrameObject *f)
{
     /* unfortunately we repeat all the variables here... */
#ifdef DXPAIRS
     int lastopcode = 0;
#endif
     register PyObject **stack_pointer;   /* Next free slot in value 
stack */
     register unsigned char *next_instr;
     register int opcode;    /* Current opcode */
     register int oparg;    /* Current opcode argument, if any */
     register enum why_code why; /* Reason for block stack unwind */
     register int err;    /* Error status -- nonzero if error */
     register PyObject *x;    /* Result object -- NULL if error */
     register PyObject *v;    /* Temporary objects popped off stack */
     register PyObject *w;
     register PyObject *u;
     register PyObject *t;
     register PyObject *stream = NULL;    /* for PRINT opcodes */
     register PyObject **fastlocals, **freevars;
     PyObject *retval = NULL;    /* Return value */
     PyThreadState *tstate = PyThreadState_GET();
     PyCodeObject *co;

     /* when tracing we set things up so that

                not (instr_lb <= current_bytecode_offset < instr_ub)

        is true when the line being executed has changed.  The
            initial values are such as to make this false the first
            time it is tested. */
     int instr_ub = -1, instr_lb = 0, instr_prev = -1;

     unsigned char *first_instr;
     PyObject *names;
     PyObject *consts;
#ifdef LLTRACE
     int lltrace;
#endif
#if defined(Py_DEBUG) || defined(LLTRACE)
     /* Make it easier to find out where we are with a debugger */
     char *filename;
#endif

fast_reentry:
     retval = tstate->st.tempval;
     tstate->st.tempval = NULL;
#endif /* STACKLESS */

     co = f->f_code;

The fast_reentry case vanished without any trace, therefore pushing
the burden to initialize retval to the call-sites of
PyEval_EvalFrame_Value. But this never happened. And it was not
recognized because the function was only called indirectly
via frame->f_execute, where a value is passed in.

The correction would be to either make the NULL check more
careful, or to pass in a dummy object which needs no reference.
In any case, this is a necessary change to un-break the
Stackless API.

Nevertheless, and even worse, the API is broken even more.
----------------------------------------------------------

An API function that has no "slp" in it is never allowed to
return the Py_UnwindToken.
Exactly that is possible in the end of PyEval_EvalFrame_value:

stackless_interrupt_call:

     f->f_execute = PyEval_EvalFrame_noval;
     f->f_stacktop = stack_pointer;

     /* the -1 is to adjust for the f_lasti change.
        (look for the word 'Promise' above) */
     f->f_lasti = INSTR_OFFSET() - 1;
     f = tstate->frame;
        return (PyObject *) Py_UnwindToken;
#endif
}

So the API needs a bit more protection, to really be compliant
to the Python API.

I came across this because I got into trouble when implementing
Stackless support for Psyco. I think I can get along this by
adding the necessary support functions to Psyco. This is easier
right now than to consistently fix all Stackless versions.
But the latter needs to be done, ASAP.

cheers - chris

On 4/12/10 8:36 PM, Christian Tismer wrote:
> There appears to be a stackless error in release25-maint:
>
> When I try to use the API call PyEval_EvalFrameEx, I always get
> a crash, because PyEval_EvalFrameEx calls
> PyEval_EvalFrameEx_slp, which does
>
>     return PyEval_EvalFrameEx_slp(f, throwflag, NULL);
>
> This NULL value gets passed to
>
>     return PyEval_EvalFrame_value(f, throwflag, retval);
>
> and then retval is checked: stacklesseval.c#692
>
>     /* always check for an error flag */
>     if (retval == NULL) {
>         why = WHY_EXCEPTION;
>         goto on_error;
>     }
>
> This cannot be correct, it must be possible to enter the evaluator
> with a NULL retval. Semantically it originates from a local variable
> which is initialized as NULL, so somehow the above test must be
> misplaced.
>
> This blocks me at the moment because I'm confused how this error
> crept in; suffering too many versions in the moment.
>
> help - chris
>


-- 
Christian Tismer             :^)<mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key ->  http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20100413/a7d5a7be/attachment-0001.htm>