[Stackless] Killing a tasklet waiting on PyChannel_Receive in C module fails

Chris Jacobson mainecoon at gmail.com
Tue Feb 24 07:53:49 CET 2009


Ok, more spam (sorry folks, I promise not to make a habit of it)!  I
threw together a quick test that doesn't use boost.

Not using boost, I must return NULL (rather than None) from my
PyCFunction to propagate the task death upwards, otherwise it won't
kill the task.  I suppose boosts' py::throw_error_already_set handles
this behavior, as it fully wraps the functions.

With boost, I only need to throw error_already_set to kill the task.
I don't need to set the Err, either, just throw.

Looks like this can be chalked up to new user and lack of
understanding of the two SDKs.

That does bring me to a question that came up as I looked through it:
when are the appropriate situations for using the _nr functions?

- Chris Jacobson


On Mon, Feb 23, 2009 at 10:15 PM, Chris Jacobson <mainecoon at gmail.com> wrote:
> Sorry for the spam... so I had no luck with the Raising the exception,
> but did find PyErr_SetString and throwing an exception in boost did
> work.  So it is possible boost is partly to blame.
>
> First an update to WaiterWontDie():
>
> void WaiterWontDie()
> {
>       PyImport_AppendInittab("killtest", &initkilltest);
>       Py_Initialize();
>
>       //      Waiter Won't Die
>       PyRun_SimpleString(
>               "import killtest, stackless\n"
>               "\n"
>               "def wontDieFunc(chan):\n"
>               "    while (1):\n"
>               "        print \"Dying soon...\"\n"
>               "        killtest.wait(chan)\n"
>               "\n"
>               "chan = stackless.channel()\n"
>               "task = stackless.tasklet(wontDieFunc)(chan)\n"
>               "\n"
>               "while (1):\n"
>               "    stackless.schedule()\n"
>               "    if task:\n"
>               "        task.kill()\n"
>               "    if task and not task.alive:\n"
>               "        print \"Task died\"\n"
>               "        task = None\n"
>               "    if chan.balance:\n"
>               "        print \"Sending on channel\"\n"
>               "        chan.send(None)\n"
>               );
> }
>
> This will detect if the task actually dies.
>
> Next I updated Wait() to raise an exception, but that still does not
> cause the tasklet to exit:
>
> void Wait( PyObject *chan )
> {
>       PyObject *result =
> PyChannel_Receive(reinterpret_cast<PyChannelObject *>(chan) );
>       Py_XDECREF(result);     // result == NULL
>
>       if (!result)
>       {
>              PyObject *currentTask = PyStackless_GetCurrent();
>              Py_INCREF(PyExc_TaskletExit);
>              Py_INCREF(Py_None);
>              PyTasklet_RaiseException(
> reinterpret_cast<PyTaskletObject *>(currentTask), PyExc_TaskletExit,
> Py_None );
>              Py_DECREF(currentTask);
>       }
> }
>
> This actually causes the bomb to explode immediately:
>
>        python26.dll!slp_bomb_explode(_object * _bomb=0x00e6f5d0)  Line 163     C
>>       python26.dll!slp_schedule_task(_tasklet * prev=0x00e6cbb0, _tasklet * next=0x00e6cbb0, int stackless=0)  Line 953 + 0x6 bytes   C
>        python26.dll!impl_tasklet_raise_exception(_tasklet *
> self=0x00e6cbb0, _object * klass=0x1e20a098, _object *
> args=0x1e215750)  Line 903 + 0xb bytes  C
>        python26.dll!PyTasklet_RaiseException(_tasklet * self=0x00e6cbb0,
> _object * klass=0x1e20a098, _object * args=0x1e215750)  Line 883 +
> 0x1a bytes      C
>        TestApp_d.exe!Wait(_object * chan=0x00dd7e70)  Line 308 + 0x1b bytes    C++
>
>
> However, setting the error string and throwing an exception does appear to work:
>
>
> void Wait( PyObject *chan )
> {
>       PyObject *result =
> PyChannel_Receive(reinterpret_cast<PyChannelObject *>(chan) );
>       Py_XDECREF(result);     // result == NULL
>
>       if (!result)
>       {
>              PyErr_SetString(PyExc_TaskletExit, "Sleeper killed");
>              py::throw_error_already_set();
>       }
> }
>
> The task does not resume executing the Python code in the tasklet
> (thus it only prints once as expected), the task is no longer alive,
> and the channel balance is now 0.  After assigning None to the task in
> Python, the refcount will reach 0.  I do have to throw the exception;
> there doesn't appear to be any state changes or clearing of the
> exception if I don't, but failure to throw the exception acts like
> nothing happened at all.  I'll write up a quick non-boost test case.
>
> Today, after a fresh nights sleep (I was trying to diagnose this issue
> for a couple hours last night, finally gave up and sent the email at
> 1:30 AM), I realized that with a C stack, it could be fairly dangerous
> to kill a tasklet without properly unwinding it.  Fortunately I won't
> have Python->C->Python cases; the C is lower level control for
> blocking tasklets on asynchronous events (network, DB, file, etc).
>
> - Chris Jacobson
>
>
> On Mon, Feb 23, 2009 at 9:23 PM, Chris Jacobson <mainecoon at gmail.com> wrote:
>> These changes had no effect on either of the test cases under 2.6.1 -
>> they both continue to loop, the exception is ignored back up in the
>> Python code.
>>
>> The diffs you provided only alter PyErr_PrintEx() ?  Was there more to it?
>>
>> - Chris
>>
>> On Mon, Feb 23, 2009 at 3:33 PM, Richard Tew <richard.m.tew at gmail.com> wrote:
>>> On Mon, Feb 23, 2009 at 6:17 PM, Richard Tew <richard.m.tew at gmail.com> wrote:
>>>> On Mon, Feb 23, 2009 at 4:24 AM, Chris Jacobson <mainecoon at gmail.com> wrote:
>>>>> If a tasklet that is currently in a C function waiting on a
>>>>> PyChannel_Receive is killed from another tasklet, the receiving
>>>>> tasklet resumes with NULL returned on the Receive, and doesn't die.
>>>>
>>>> What version of Python are you using?  2.6, 3.0 or something earlier?
>>>
>>> Fixed.  In any case, I compiled against 3.0.1, as it was what I had
>>> locally.  I'll try 2.6 as well, but if you encounter any more bugs,
>>> please mention your platform and Python version.  I'll have to do
>>> rebuilds and rereleases of 3.0 and 3.0.1, so let's hope there's not
>>> any more :-)
>>>
>>> Cheers,
>>> Richard.
>>>
>>> Author: richard.tew
>>> Date: Tue Feb 24 00:27:50 2009
>>> New Revision: 69919
>>>
>>> Log:
>>> Fix for strange application restarting bug, where in some
>>> circumstances an exception was compared to SystemExit, and if it
>>> matched, the application was asked to exit.  TaskletExit is a subclass
>>> of SystemExit, which means that when it is raised, if it hits this
>>> execution path, the restarting occurs.
>>>
>>> Reported by Chris Jacobson.
>>>
>>> Modified:
>>>  stackless/branches/release30-maint/Python/pythonrun.c
>>>
>>> Modified: stackless/branches/release30-maint/Python/pythonrun.c
>>> ==============================================================================
>>> --- stackless/branches/release30-maint/Python/pythonrun.c       (original)
>>> +++ stackless/branches/release30-maint/Python/pythonrun.c       Tue
>>> Feb 24 00:27:50 2009
>>> @@ -1382,7 +1382,7 @@
>>>  {
>>>       PyObject *exception, *v, *tb, *hook;
>>>
>>> -       if (PyErr_ExceptionMatches(PyExc_SystemExit)) {
>>> +       if (PyErr_ExceptionMatches(PyExc_SystemExit) &&
>>> !PyErr_ExceptionMatches(PyExc_TaskletExit)) {
>>>               handle_system_exit();
>>>       }
>>>       PyErr_Fetch(&exception, &v, &tb);
>>> @@ -1408,7 +1408,7 @@
>>>               PyObject *result = PyEval_CallObject(hook, args);
>>>               if (result == NULL) {
>>>                       PyObject *exception2, *v2, *tb2;
>>> -                       if (PyErr_ExceptionMatches(PyExc_SystemExit)) {
>>> +                       if (PyErr_ExceptionMatches(PyExc_SystemExit)
>>> && !PyErr_ExceptionMatches(PyExc_TaskletExit)) {
>>>                               handle_system_exit();
>>>                       }
>>>                       PyErr_Fetch(&exception2, &v2, &tb2);
>>>
>>
>




More information about the Stackless mailing list