[Stackless] pickling bug (was: Stackless Digest, Vol 90, Issue 3)

Christian Tismer tismer at stackless.com
Tue Aug 9 16:06:05 CEST 2011


Hi,

I sent this almost-patch a while ago, but it was probably invisible,
due to a bad subject line. Here it is again:

file stackless_util.c:

static PyObject *
tasklet_get_frame(PyTaskletObject *task)
{
     PyFrameObject *f = (PyFrameObject *) slp_get_frame(task);
     PyObject *ret;

     while (f != NULL && !PyFrame_Check(f)) {
             f = f->f_back;
     }
     ret = (PyObject *) f;

     if (ret == NULL) ret = Py_None;
     Py_INCREF(ret);
     return ret;
}

This should resolve the problem.

ciao - chris

On 7/8/11 2:26 PM, Christian Tismer wrote:
> On 7/8/11 1:34 PM, Christian Tismer wrote:
>> On 7/8/11 1:53 AM, Richard Tew wrote:
>>> On Thu, Jul 7, 2011 at 10:10 PM, Hussain Bohra
>>> <hussainbohra_30 at yahoo.com>  wrote:
>>>> Hi Christian,
>>>>
>>>> Thanks for your feedback.
>>>>
>>>> Could you kindly let us know when you are done with the changes, so 
>>>> that we
>>>> can complete the existing tasks that are dependent on this fix.
>>> Can't you work around it?  Like check if a frame is a cframe and skip
>>> it, if it is the case?  I haven't looked at the traceback generation
>>> in detail, but I imagine that something along the following lines
>>> would work..
>>>
>>> lastTasklet = stackless.current
>>> thisTasklet = lastTasklet.next
>>> while True:
>>>      thisFrame = thisTasklet.frame
>>>      while thisFrame is not None:
>>>          if not isinstance(thisFrame, stackless.cframe):
>>>              # Do traceback related stuff.
>>>          thisFrame = thisFrame.f_back
>>>      thisTasklet = thisTasklet.next
>>>      if lastTasklet is thisTasklet:
>>>          break
>>
>> Hi Richard, Hussain,
>>
>> yes, I had a look now, and you are right. CFrames get into the way,
>> but are a small problem. Your workaround should be a simple quick
>> solution in the first place, and I encourage Hussain to try that first.
>> It is a really simple patch to traceback.py: insert a check into 
>> extract_stack
>> before "if limit is None:":
>>
>>     while f and not isinstance(f, types.FrameType):
>>         f = f.f_back
>>
>> That should do.
>>
>> ---------------------------------------------
>> Now to the omission in stackless:
>> The special treatment of cframes is quite limited in stackless:
>>
>> - sys._getframe has a patch to skip over anything that is no normal 
>> frame.
>> - frame objects have a getter function frame_getback() instead of the 
>> f_back
>>    field.
>>
>> The only way how the treatment of cframes might be circumvented
>> in extract_stack is when a cframe is given as an argument, and that
>> was the omission in stackless:
>> tasklet.frame exposes whatever it has to the user. Only for the running
>> tasklet, this gets redirected to the threadstate's frame, which in the
>> from python visible context never is a cframe.
>>
>> The function slp_get_frame is the culprit (stackless_util.c):
>>
>> /* CAUTION: This function returns a borrowed reference */
>> PyFrameObject *
>> slp_get_frame(PyTaskletObject *task)
>> {
>>     PyThreadState *ts = PyThreadState_GET();
>>
>>     return ts->st.current == task ? ts->frame : task->f.frame;
>> }
>>
>> This is correct for builtin API functions, but not for the python
>> interface. The getter methon tasklet_get_frame (taskletobject.c)
>> should skip over cframes.
>>
>> static PyObject *
>> tasklet_get_frame(PyTaskletObject *task)
>> {
>>     PyObject *ret = (PyObject*) slp_get_frame(task);
>>
>>     if (ret == NULL) ret = Py_None;
>>     Py_INCREF(ret);
>>     return ret;
>> }
>>
>> I think something like the following should do it:
>>
>> static PyObject *
>> tasklet_get_frame(PyTaskletObject *task)
>> {
>>     PyFrameObject *f = (PyFrameObject *) slp_get_frame(task);
>>     PyObject *ret;
>>
>>     while (f != NULL && !PyFrame_Check(f)) {
>>             f = f_fback;
>>     }
>>     ret = (PyObject *) f;
>>
>>     if (ret == NULL) ret = Py_None;
>>     Py_INCREF(ret);
>>     return ret;
>> }
>>
>>
>> Not yet tested, but I guess that should do it.
>> Do you want to try when removing tasklet_become, or should I?
>>
>> cheers -- chris
>>
>
> Correction:
>             f = f_fback;
> should be
>             f = f->f_back;
>
> And as a remark:
> The long-term solution to such problems is to get rid of these old 
> tricks,
> minimizing the patches to cpython as much as possible.
> We should remove all additions/changes to frames and replace
> cframes by helper structures that are associated with a frame
> invisibly and indirectly through a dictionary.
> Of course a bit tedious, because frames do not support weak references
> (http://docs.python.org/library/weakref.html).
> I'm thinking of a shadow stack of thunks as an execution context
> which references the frames, but is used instead of the frame stack.
>
> Maybe it's time for another stackless sprint, to start a major 
> clean-up? :-)
>
> cheers - chris
>


-- 
Christian Tismer             :^)<mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key ->  http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/




More information about the Stackless mailing list