[Stackless] Stackless runtime smacking into GIL while a callback is running

[Adam Preble]

I put in double effort to get this out in the morning because of the volume
of response I got on the topic yesterday.  I am wondering now if this is
truly a Stackless Python issue.  I need to try linking this to legacy
Python too, but I rushed on the code to at least have something to show to
those whose interest was piqued yesterday.  Normally I post anything with
interoperability in it to the cplusplus-sig where the Boost.Python people
hang out, but the stack trace for the PyThreadState_get call is in the
Python stack, not through my code.  So I figure it's a runtime issue of
some kind--whether Stackless or conventional Python.

I managed to reproduce it without having to use the whole hulk of code.
 Here I have a C++ class running its own thread that makes a callback to a
Python object.

TimeBomb.hpp  http://pastebin.com/tjiSrRaN
TimeBomb.cpp  http://pastebin.com/CfrmvGEB
StacklessPythonCppInterop.cpp  http://pastebin.com/Fc6HyTKQ  (has main in
it)
test_script.py http://pastebin.com/L5r7sGFc

Here's the output:

Inside the inner body of the time bomb.
Five seconds are up.  Time to deliver payload.
Got explosion
GFatal Python error: PyThreadState_Get: no current thread
ot explosion
Got explosion
Got explosion
Got explosion
Payload delivered.
[Crash here]

On Mon, Mar 12, 2012 at 4:39 PM, Richard Tew <richard.m.tew at gmail.com>wrote:

> I will try and answer this on my iPad, but the usability of this for text
> entry is horrendous.
>
>
> On Tuesday, March 13, 2012, Adam Preble <adam.preble at gmail.com> wrote:
> > Anyways I had a thought about what might be wrong, but I'm not sure what
> I can do about it.  Let's say when one of my C++ threads does the callback,
> I'm toggling a boolean value in a Python object.  That Python object was
> created prior in the original Python thread.  In that original thread, it
> is basically spinning in a loop on that bool.  When the foreign thread
> touches the bool, do I invite myself to trouble?  I can get away with
> horrors like that in C++ since bools are just value types and cross-core
> cache coherency protocols will get the memo across eventually; but I wonder
> in PythonLand--where everything is a reference--if that is a smart thing to
> try to do.  I was assuming that by grabbing the GIL I was in control but
> maybe that's not safe.  Or perhaps worse still in the Stackless context
> switch stuff, the bool would get mashed up.
>
> In this case, with respect to Stackless specifically, you know what Python
> code is being executed, and you know that unless you're doing some
> horrendous custom logic within overriding of attribute setting or getting,
> none of the task lets created on other threads are being invoked.  So
> setting a Boolean on a python object would be completely safe.  With spect
> to python and multithreading in general, the Gil makes access to the
> Boolean safe in the same way as. If you were doing interthread programming
> with non Stackless pythons.
>
> Note that each Pyhton thread has a scheduler, and task let's created
> within a thread will be associated with that thread for the purposes of
> scheduler usage.  It's best to just only create task.ets on the main
> thread, and use channels or some other form of interfered communication to
> either communicate with other threads or pass values across.
>
>
> > So I wonder, if I used a channel instead would that be a safe resource
> for communicating across two literal threads, versus just two green threads?
> > Anyways I think I owe the distribution some code so I'll start pounding
> away on that in the evening (CST).
>
> Channels are implemented with support for inter thread communication, in
> addition to their primary purpose of green thread communication.
>
> Cheers,
> Richard.
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20120313/46f3f6e3/attachment.html>