[Stackless] Windows 2.7.6 stability

Kristján Valur Jónsson kristjan at ccpgames.com
Wed Feb 5 22:22:58 CET 2014


I think that the problem may be a bit more insidious than just the size of the PyTypeObject.
Imagine this scenario:
An extension creates a C extension of a builtin type.  This extension is compiled against vanilla Python.  So, its own default flags don't have the STACKLESS flag.  But it does inherit
a bunch of slots from the builtin stackless object, and those are defined to be STACKLESS aware.

so, stackless will call those inherited slots thinking they don't adhere to the STACKLESS protocol.

Is it safe to call "stackless" type calls using the old method?  Or is this purely an "added extra", so that the new, stackless aware, slot methods _can_ work with the old calling mechanism?  I don't know.  Hm, perhaps they do....  In which case, the stackless flag is an opt-in thing.  If it is missing, even for a stackless method, then it would be ok.....


I think that with the method here, of having the STACKLESS flag indicate that there is extra data in
the PyMappingMethods structure, we can in fact apply the STACKLESS flag to any inherited type,
which calls PyType_Ready().  If the parent type has the STACKLESS flag, we can modify its tp_as_mapping member to point to a dynamically allocated, larger, structure, and then proceed to do inheritance correctly.

So, we can inherit the stackless flag for subtypes that were compiled unaware of stackless.  If PySide were for example to create a subtype of PyFunctionType, then we could decorate it with the STACKLESS flag and have it inherit the stackless slot flags, by giving the subtype a new tp_as_mapping member.

K


________________________________________
From: stackless-bounces at stackless.com [stackless-bounces at stackless.com] on behalf of Richard Tew [richard.m.tew at gmail.com]
Sent: Wednesday, February 05, 2014 19:58
To: The Stackless Python Mailing List
Subject: Re: [Stackless] Windows 2.7.6 stability

On one hand I wonder why we just didn't do this long ago.

On the other, it crashes :-)  I'll look into it later as I am
celebrating Waitangi Day in the traditional way of sitting around
doing nothing.

When this is working, I'll rerelease 2.7.6 and try and get people to
rebuild all the binaries.

>       python27.dll!getset_get(PyGetSetDescrObject * descr=0x0056a1e8, _object * obj=0x03848b98, _object * type=0x00000000)  Line 147 + 0xa bytes      C
        python27.dll!PyObject_Call(_object * func=0x0056a1e8, _object *
arg=0x03848b98, _object * kw=0x00000000)  Line 2539 + 0x1c bytes        C
        python27.dll!PyObject_CallFunctionObjArgs(_object *
callable=0x0056a1e8, ...)  Line 2786 + 0x13 bytes       C
        python27.dll!build_class(_object * methods=0x0384c390, _object *
bases=0x1e0efddf, _object * name=0x037ba270)  Line 5117 + 0xf bytes     C
        python27.dll!PyEval_EvalFrame_value(_frame * f=0x00502428, int
throwflag=58434160, _object * retval=0x0384c390)  Line 3394     C
        python27.dll!PyEval_EvalFrameEx_slp(_frame * f=0x00502428, int
throwflag=0, _object * retval=0x1e234ff4)  Line 964 + 0x17 bytes        C
        python27.dll!slp_eval_frame_newstack(_frame * f=0x00502428, int
exc=0, _object * retval=0x1e234ff4)  Line 521 + 0x1b bytes      C
        python27.dll!PyEval_EvalFrameEx_slp(_frame * f=0x00502428, int
throwflag=0, _object * retval=0x1e234ff4)  Line 964     C
        python27.dll!slp_frame_dispatch_top(_object * retval=0x1e234ff4)
Line 810 + 0x9 bytes    C
        python27.dll!slp_run_tasklet(_frame * f=0x00502428)  Line 1504  C
        python27.dll!slp_eval_frame(_frame * f=0x00502428)  Line 317 + 0xa bytes        C
        python27.dll!climb_stack_and_eval_frame(_frame * f=0x00502428)  Line
274 + 0x9 bytes C
        python27.dll!slp_eval_frame(_frame * f=0x00502428)  Line 303 + 0x6 bytes        C
        python27.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x022faa40, _object
* globals=0x00502428, _object * locals=0x004bc930, _object * *
args=0x00000000, int argcount=0, _object * * kws=0x00000000, int
kwcount=0, _object * * defs=0x00000000, int defcount=0, _object *
closure=0x00000000)  Line 3697 + 0x6 bytes      C
        python27.dll!PyEval_EvalCode(PyCodeObject * co=0x022faa40, _object *
globals=0x004bc930, _object * locals=0x004bc930)  Line 674 + 0x22
bytes   C
        python27.dll!run_mod(_mod * mod=0x023613e8, const char *
filename=0x00000000, _object * globals=0x004bc930, _object *
locals=0x004bc930, PyCompilerFlags * flags=0x00000000, _arena *
arena=0x00000000)  Line 1408 + 0x16 bytes       C
        python27.dll!PyRun_FileExFlags(_iobuf * fp=0x73247408, const char *
filename=0x00354be4, int start=257, _object * globals=0x004bc930,
_object * locals=0x004bc930, int closeit=1, PyCompilerFlags *
flags=0x0027ff00)  Line 1393    C
        python27.dll!PyRun_SimpleFileExFlags(_iobuf * fp=0x73247408, const
char * filename=0x00354be4, int closeit=1, PyCompilerFlags *
flags=0x0027ff00)  Line 967 + 0x18 bytes        C
        python27.dll!PyRun_AnyFileExFlags(_iobuf * fp=0x73247408, const char
* filename=0x00354be4, int closeit=1, PyCompilerFlags *
flags=0x0027ff00)  Line 770 + 0x11 bytes        C
        python27.dll!Py_Main(int argc=2, char * * argv=0x00354ba0)  Line 643
+ 0x27 bytes    C
        python.exe!__tmainCRTStartup()  Line 582 + 0x17 bytes   C

>From the output window...

HEAP[python.exe]: HEAP: Free Heap block 3812418 modified at 3812438
after it was freed
Windows has triggered a breakpoint in python.exe.

This may be due to a corruption of the heap, which indicates a bug in
python.exe or any of the DLLs it has loaded.

This may also be due to the user pressing F12 while python.exe has focus.

The output window may have more diagnostic information.

Cheers,
Richard.

On 2/6/14, Kristján Valur Jónsson <kristjan at ccpgames.com> wrote:
> Ok, I have a proposed patch in a branch on
> https://bitbucket.org/krisvale/stackless-scratch
>
> The idea is to move the flags from the PyTypeObject into the
> PyMappingMethods structure, which is unlikely to be extended by third party
> apps.
> This runs the stackless testsuite.
>
> K
>
> From: stackless-bounces at stackless.com
> [mailto:stackless-bounces at stackless.com] On Behalf Of lars van Gemerden
> Sent: 5. febrúar 2014 12:17
> To: The Stackless Python Mailing List
> Subject: Re: [Stackless] Windows 2.7.6 stability
>
> I would like to see better PySide compatibility, since my code relies on
> stackless and PySide.
>
> The crashes have become less though since maybe a year ago; maybe some
> improvements on the PySide?
>
> I am using stackless python 2.7.5.
>
> CL
>
> On Wed, Feb 5, 2014 at 9:55 AM, Kristján Valur Jónsson
> <kristjan at ccpgames.com<mailto:kristjan at ccpgames.com>> wrote:
> At one point I had a patch going in stackless, which changed the way we
> extended PyHeapType, IIRC.
> I think this is the way to go, make sure we stay compatible.
>
>> -----Original Message-----
>> From:
>> stackless-bounces at stackless.com<mailto:stackless-bounces at stackless.com>
>> [mailto:stackless-<mailto:stackless->
>> bounces at stackless.com<mailto:bounces at stackless.com>] On Behalf Of Richard
>> Tew
>> Sent: 4. febrúar 2014 20:11
>> To: The Stackless Python Mailing List
>> Subject: Re: [Stackless] Windows 2.7.6 stability
>>
>> Well, I think Christian was deep into it, and trying to get the PySide
>> people to
>> do a compatibility patch or something.  Last I recall, from several months
>> ago.
>>
>> It is indeed a pity we can't say we're compatible with all Python
>> extensions,
>> and it would be good to get it fixed.
>>
>> Cheers,
>> Richard.
>>
>> On 2/4/14, Kristján Valur Jónsson
>> <kristjan at ccpgames.com<mailto:kristjan at ccpgames.com>> wrote:
>> > This is so annoying.  Time for another stab at this problem, perhaps?
>> >
>> >> -----Original Message-----
>> >> From:
>> >> stackless-bounces at stackless.com<mailto:stackless-bounces at stackless.com>
>> >> [mailto:stackless-<mailto:stackless->
>> >> bounces at stackless.com<mailto:bounces at stackless.com>] On Behalf Of
>> >> Richard Tew
>> >> Sent: 3. febrúar 2014 18:53
>> >> To: The Stackless Python Mailing List
>> >> Subject: Re: [Stackless] Windows 2.7.6 stability
>> >>
>> >> No, thanks for the suggestion.  But this is definitely PySide and
>> >> it's modification of the base object type that stackless also
>> >> modifies.
>> >>
>> >> Cheers,
>> >> Richard.
>> >>
>> >> On 2/4/14, Anselm Kruis
>> >> <a.kruis at science-computing.de<mailto:a.kruis at science-computing.de>>
>> >> wrote:
>> >> > Could it be related to the PGO optimised build?
>> >> > As far as I know, the mainline python installer is build without
>> >> > PGO.
>> >> > Our installer is build with PGO optimisation.
>> >> >
>> >> > Another question: can you try our 2.7.5 build? It is PGO optimized
>> >> > too.
>> >> >
>> >> > Cheers
>> >> >    Anselm
>> >> >
>> >> >
>> >> > Am 03.02.2014 12<tel:03.02.2014%2012>:13, schrieb Kristján Valur
>> >> > Jónsson:
>> >> >> Hi.
>> >> >> Do you get this only with the installed version?
>> >> >> Could you try replacing it with your own build?  If so, could you
>> >> >> go into the source code and disable stack spilling ?
>> >> >> You have to nerf the macro CSTACK_SAVE_NOW.
>> >> >>
>> >> >> I saw some mysterious crashes recently in a live build in Shanghai
>> >> >> that went away when I disabled this.
>> >> >>
>> >> >> K
>> >> >>
>> >> >>> -----Original Message-----
>> >> >>> From:
>> >> >>> stackless-bounces at stackless.com<mailto:stackless-bounces at stackless.com>
>> >> >>> [mailto:stackless-<mailto:stackless->
>> >> >>> bounces at stackless.com<mailto:bounces at stackless.com>] On Behalf Of
>> >> >>> Richard Tew
>> >> >>> Sent: 3. febrúar 2014 03:04
>> >> >>> To: stackless at stackless.com<mailto:stackless at stackless.com>
>> >> >>> Subject: [Stackless] Windows 2.7.6 stability
>> >> >>>
>> >> >>> Hi,
>> >> >>>
>> >> >>> I've got the installer we provide for 2.7.6, on Windows.  And
>> >> >>> I've been getting lots of non-crashing premature exits:
>> >> >>>
>> >> >>> SystemError: unknown opcode
>> >> >>> XXX lineno: 314, opcode: 0
>> >> >>>
>> >> >>> It's not consistently reproducible using running the same code,
>> >> >>> but can be sometimes, and I'm not using any Stackless features.
>> >> >>>
>> >> >>> If I take the mainline python repo and sync to v2.7.6 and
>> >> >>> generate a dll, and put it in c:\python27, all the problems go
>> >> >>> away.
>> >> >>>
>> >> >>> I thought it might be pyside 1.1.2 which I was using, but
>> >> >>> upgraded that and the problem remained with pyside 1.2.1.  That's
>> >> >>> the only external dependency my code uses.
>> >> >>>
>> >> >>> Anyone else using this installer?
>> >> >>>
>> >> >>> Cheers,
>> >> >>> Richard.
>> >> >>>
>> >> >>> _______________________________________________
>> >> >>> Stackless mailing list
>> >> >>> Stackless at stackless.com<mailto:Stackless at stackless.com>
>> >> >>> http://www.stackless.com/mailman/listinfo/stackless
>> >> >>
>> >> >>
>> >> >>
>> >> >> _______________________________________________
>> >> >> Stackless mailing list
>> >> >> Stackless at stackless.com<mailto:Stackless at stackless.com>
>> >> >> http://www.stackless.com/mailman/listinfo/stackless
>> >> >>
>> >> >
>> >> > --
>> >> >   Dipl. Phys. Anselm Kruis                       science + computing
>> >> > ag
>> >> >   Senior Solution Architect                      Ingolstädter Str.
>> >> > 22
>> >> >   email
>> >> > A.Kruis at science-computing.de<mailto:A.Kruis at science-computing.de>
>> >> >         80807 München, Germany
>> >> >   phone +49 89 356386 874<tel:%2B49%2089%20356386%20874>  fax 737
>> >> > www.science-computing.de<http://www.science-computing.de>
>> >> > --
>> >> > Vorstandsvorsitzender/Chairman of the board of management:
>> >> > Gerd-Lothar Leonhart
>> >> > Vorstand/Board of Management:
>> >> > Dr. Bernd Finkbeiner, Michael Heinrichs, Dr. Arno Steitz, Dr.
>> >> > Ingrid Zech Vorsitzender des Aufsichtsrats/ Chairman of the
>> >> > Supervisory
>> >> > Board:
>> >> > Philippe Miltin
>> >> > Sitz/Registered Office: Tuebingen
>> >> > Registergericht/Registration Court: Stuttgart
>> >> > Registernummer/Commercial Register No.: HRB 382196
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > Stackless mailing list
>> >> > Stackless at stackless.com<mailto:Stackless at stackless.com>
>> >> > http://www.stackless.com/mailman/listinfo/stackless
>> >> >
>> >>
>> >> _______________________________________________
>> >> Stackless mailing list
>> >> Stackless at stackless.com<mailto:Stackless at stackless.com>
>> >> http://www.stackless.com/mailman/listinfo/stackless
>> >
>> >
>> >
>> > _______________________________________________
>> > Stackless mailing list
>> > Stackless at stackless.com<mailto:Stackless at stackless.com>
>> > http://www.stackless.com/mailman/listinfo/stackless
>> >
>>
>> _______________________________________________
>> Stackless mailing list
>> Stackless at stackless.com<mailto:Stackless at stackless.com>
>> http://www.stackless.com/mailman/listinfo/stackless
>
>
>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com<mailto:Stackless at stackless.com>
> http://www.stackless.com/mailman/listinfo/stackless
>
>
>
> --
> ====================================
> Lars van Gemerden
> lars at rational-it.com<mailto:lars at rational-it.com>
> +31 6 26 88 55 39
> ====================================
>

_______________________________________________
Stackless mailing list
Stackless at stackless.com
http://www.stackless.com/mailman/listinfo/stackless



More information about the Stackless mailing list