[Stackless] 3.2.5 unittest failure

Anselm Kruis a.kruis at science-computing.de
Tue Mar 11 13:31:52 CET 2014


Hi Chris,

Am 11.03.2014 12:08, schrieb Christian Tismer:
> Hi Anselm,
>
> right, but is that not just a matter of proper reconstruction of the
> linkage?
> I think we introduced traceback pickling as well,

That's right.

  so we are free to choose
> its representation, too. And that could be a list or something, whatever
> gives a simple and obvious structure with no recursion that is hard to
> introspect.

Yea, in theory. But I found the current implementation to be slightly 
broken. I don't remember the details right now.

Cheers
   Anselm


>
> cheers - Chris
>
>
> On 11.03.14 11:29, Anselm Kruis wrote:
>> Hi,
>>
>> just a comment about pickling frames. I noticed the odd behaviour of
>> f_back being None a while ago during my work on pyheapdump. This
>> design decision breaks pickling of trace-back objects. That could be a
>> reason to reconsider frame pickling.
>>
>> Cheers
>>    Anselm
>>
>>
>>
>> Am 07.03.2014 17:50, schrieb Christian Tismer:
>>> Frame stacks:
>>>
>>> Yes, for some reason I did the pickling of frame stacks as a list
>>> that is not linked in the pickle.
>>>
>>> One reason was just for easier debugging, because the frame stack
>>> is just a list that has a length, there is no chance of doing something
>>> wrong, no recursive stuff going on, etc.
>>>
>>> There is also the consideration that the way of linking does not
>>> necessarily have to be by using f_back, while sitting in a pickle.
>>> Remember, the reference counting on f_back is a little different on
>>> CPython and Stackless, actually an implementation detail that does
>>> not belong into the pickle.
>>>
>>> Keeping the frame stack as a list was the most natural way for me to
>>> handle this.
>>>
>>> I believe the fact that f_back == None is also used as a marker of
>>> """this frame came fron unpickling""".
>>>
>>> Without a need to improve this, I would like to keep it this way.
>>>
>>> cheers - Chris
>>>
>>>
>>> On 06/03/14 10:01, Kristján Valur Jónsson wrote:
>>>> I'm not sure I understand.
>>>> What exactly is blocking us?
>>>> I fixed the unittest crash. The rest of my question was musings on
>>>> how frame pickling works,
>>>> because it seems to only pickle one frame at a time, marking its
>>>> parent with the special valid None
>>>> to indicate this....
>>>> Ah, tasklets are pickled with a list of frames, that they then
>>>> re-assemble when unpickled.
>>>>
>>>> This is what confused me.  I would have pickled frames as a linked
>>>> list, recursively.
>>>> I'm sure there is a good reason why they are pickled the way they
>>>> are :)
>>>>
>>>> K
>>>>
>>>>> -----Original Message-----
>>>>> From: stackless-bounces at stackless.com [mailto:stackless-
>>>>> bounces at stackless.com] On Behalf Of Richard Tew
>>>>> Sent: 6. mars 2014 07:34
>>>>> To: The Stackless Python Mailing List
>>>>> Subject: Re: [Stackless] 3.2.5 unittest failure
>>>>>
>>>>> This is the blocker for the releases of 2.7.x, 3.2.x, 3.3.x and 3.4.
>>>>>
>>>>> Is this something that is a problem with the way we are pickling?
>>>>> Or is it
>>>>> related to how unpickling happens?
>>>>>
>>>>> Cheers,
>>>>> Richard.
>>>>>
>>>>> On 3/5/14, Kristján Valur Jónsson <kristjan at ccpgames.com> wrote:
>>>>>> ok, I have fixed it and pushed a new version.
>>>>>> I don't quite understand, why are unpickled frames marked with
>>>>>> None in
>>>>>> their f_back?  Does that mean that we never pickle frame stacks, but
>>>>>> only single frames?
>>>>>> I looked for other cases where we test for this None but didn't
>>>>>> find it...
>>>>>> K
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: stackless-bounces at stackless.com [mailto:stackless-
>>>>>>> bounces at stackless.com] On Behalf Of Richard Tew
>>>>>>> Sent: 3. mars 2014 07:32
>>>>>>> To: stackless at stackless.com
>>>>>>> Subject: [Stackless] 3.2.5 unittest failure
>>>>>>>
>>>>>>> Kristjan, is it possible you missed a graft?
>>>>>>>
>>>>>>> This is the last line before a release build of 3.2.5 crashes for
>>>>>>> me:
>>>>>>>
>>>>>>> testCrasher (test_defects.TestCrashUponFrameUnpickling) ...
>>>>>>>
>>>>>>>>      python32.dll!frame_getback(_frame * f=0x02836700, void *
>>>>>>> nope=0x00000000)  Line 375 + 0x9 bytes    C
>>>>>>>        python32.dll!getset_get(PyGetSetDescrObject *
>>>>>>> descr=0x02069440,
>>>>>>> _object * obj=0x02836700, _object * type=0x1e2b03f8)  Line 149 +
>>>>>>> 0x19
>>>>>>> bytes    C
>>>>>>>        python32.dll!_PyObject_GenericGetAttrWithDict(_object *
>>>>>>> obj=0x02836700, _object * name=0x0205fe80, _object *
>>>>>>> dict=0x00000000)
>>>>>>> Line 988 + 0x12 bytes    C
>>>>>>>        python32.dll!PyObject_GenericGetAttr(_object * obj=0x02836700,
>>>>>>> _object * name=0x0205fe80)  Line 1050 + 0xf bytes    C
>>>>>>>        python32.dll!PyObject_GetAttr(_object * v=0x02836700,
>>>>>>> _object *
>>>>>>> name=0x0205fe80)  Line 835 + 0x10 bytes    C
>>>>>>>        python32.dll!PyEval_EvalFrame_value(_frame * f=0x027f56b8, int
>>>>>>> throwflag=506147444, _object * retval=0x1e2b3274)  Line 2963 + 0x7
>>>>>>> bytes    C
>>>>>>>
>>>>>>> It crashes in:
>>>>>>>
>>>>>>> static PyObject *
>>>>>>> frame_getback(PyFrameObject *f, void *nope) {
>>>>>>>      PyFrameObject *fb = f->f_back;
>>>>>>>      PyObject *ret;
>>>>>>>      while (fb != NULL && ! PyFrame_Check(fb))
>>>>>>>          fb = fb->f_back;
>>>>>>>
>>>>>>> Where fb = 0x03.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Richard.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Stackless mailing list
>>>>>>> Stackless at stackless.com
>>>>>>> http://www.stackless.com/mailman/listinfo/stackless
>>>>>>
>>>>>> _______________________________________________
>>>>>> Stackless mailing list
>>>>>> Stackless at stackless.com
>>>>>> http://www.stackless.com/mailman/listinfo/stackless
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Stackless mailing list
>>>>> Stackless at stackless.com
>>>>> http://www.stackless.com/mailman/listinfo/stackless
>>>>
>>>> _______________________________________________
>>>> Stackless mailing list
>>>> Stackless at stackless.com
>>>> http://www.stackless.com/mailman/listinfo/stackless
>>>>
>>>
>>>
>>
>
>

-- 
  Dipl. Phys. Anselm Kruis                       science + computing ag
  Senior Solution Architect                      Ingolstädter Str. 22
  email A.Kruis at science-computing.de             80807 München, Germany
  phone +49 89 356386 874  fax 737               www.science-computing.de
-- 
Vorstandsvorsitzender/Chairman of the board of management:
Gerd-Lothar Leonhart
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Michael Heinrichs, 
Dr. Arno Steitz, Dr. Ingrid Zech
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196


More information about the Stackless mailing list