ABI incompatibility in Objects/structseq.c PyStructSequence_InitType

[akruis]

In Objects/structseq.c

void
PyStructSequence_InitType(PyTypeObject? *type, PyStructSequence_Desc *desc)
{

....

memcpy(type, &_struct_sequence_template, sizeof(PyTypeObject?));

Now if type comes from an extension module compiled against vanilla python, stackless writes its extensions into memory behind the end of *type.

I'm not sure about my fix.

[akruis]

An experimental fix for this bug

[ctismer]

It looks right, but I'm not sure in the first place as well.
How about adding tests which stress this that would fail without the patch?

[akruis]

I used the module "grp" as test case and the patch indeed fixed that case. At least importing grp no longer causes a segmentation fault. (I discovered this issue, because the module "grp" failed to load). But just because something now works it is not necessarily correct. And I don't understand the extensions made by Stackless well enough to judge whether my fix is sufficient or not.

[rmtew]

Replying to akruis:

Now if type comes from an extension module compiled against vanilla python, stackless writes its extensions into memory behind the end of *type.

I'm not sure about my fix.

Looks good to me and makes sense. I'll give the code base a going over for sizeof(PyTypeObject?) references, and look for other cases, before committing a version based on the suggested macro.

[ctismer]

Hi Richard,

I was about to do the changes today.
But if you are already working on it, then I better let you continue.

those captchas are really annoying ;-)

[rmtew]

Fixed in: ​http://hg.python.org/stackless/rev/70a46c5443a1

I've only tested that it compiles on Windows, with Visual Studio (mingw won't compile Python for me). If I over vigorously trimmed the patch, let me know. I'll test on a Linux box tomorrow, and then if it passes all tests appropriately, I'll propagate it round suitable branches and then close this.

I am not seeing any captchas :-)

[admin]

Looks very good.
Thanks to you both for your work.

as admin there is no captcha, indeed.
but do I need to make this less annoying, to make people comment at all?

[rmtew]

Actually I added a bug by removing the flag clearing, but didn't understand why it was needed until my run this morning. I'll amend the patch, test and propagate it between branches.

[admin]

Sorry, I did not check your patch enough.
Not removing that flag code was essential.
I almost fell into the same trap, but
discussed that with Anselm offline.

Instead of removing that line, I think it
is crucial to add a comment why it is so
urgently needed, because it looks redundant,
but is very important for the _copy_ to work.

[admin]

Hi Richard,
Please add a comment that keeps every
maintainer from the same wrong reasoning.

And as a hint to Anselm:
The patch is highly appreciated.
The lack of a comment on the other hand
has cost two people maybe two extra hours
at least.

I think it is a benefit for everybody
if you can avoid such traps. You have
had lots of thought for the few lines.
Please share them in the future with
such a patch.

Ciao - Chris

[rmtew]

Well for me the confusion came from forgetting about how Stackless & Python for that matter works. I included comments in my second changeset.

​http://hg.python.org/stackless/rev/aff6741558ad