[Stackless] Killing a tasklet waiting on PyChannel_Receive in C module fails

Chris Jacobson mainecoon at gmail.com
Tue Feb 24 07:15:31 CET 2009


Sorry for the spam... so I had no luck with the Raising the exception,
but did find PyErr_SetString and throwing an exception in boost did
work.  So it is possible boost is partly to blame.

First an update to WaiterWontDie():

void WaiterWontDie()
{
       PyImport_AppendInittab("killtest", &initkilltest);
       Py_Initialize();

       //      Waiter Won't Die
       PyRun_SimpleString(
               "import killtest, stackless\n"
               "\n"
               "def wontDieFunc(chan):\n"
               "    while (1):\n"
               "        print \"Dying soon...\"\n"
               "        killtest.wait(chan)\n"
               "\n"
               "chan = stackless.channel()\n"
               "task = stackless.tasklet(wontDieFunc)(chan)\n"
               "\n"
               "while (1):\n"
               "    stackless.schedule()\n"
               "    if task:\n"
               "        task.kill()\n"
               "    if task and not task.alive:\n"
               "        print \"Task died\"\n"
               "        task = None\n"
               "    if chan.balance:\n"
               "        print \"Sending on channel\"\n"
               "        chan.send(None)\n"
               );
}

This will detect if the task actually dies.

Next I updated Wait() to raise an exception, but that still does not
cause the tasklet to exit:

void Wait( PyObject *chan )
{
       PyObject *result =
PyChannel_Receive(reinterpret_cast<PyChannelObject *>(chan) );
       Py_XDECREF(result);     // result == NULL

       if (!result)
       {
              PyObject *currentTask = PyStackless_GetCurrent();
              Py_INCREF(PyExc_TaskletExit);
              Py_INCREF(Py_None);
              PyTasklet_RaiseException(
reinterpret_cast<PyTaskletObject *>(currentTask), PyExc_TaskletExit,
Py_None );
              Py_DECREF(currentTask);
       }
}

This actually causes the bomb to explode immediately:

 	python26.dll!slp_bomb_explode(_object * _bomb=0x00e6f5d0)  Line 163	C
>	python26.dll!slp_schedule_task(_tasklet * prev=0x00e6cbb0, _tasklet * next=0x00e6cbb0, int stackless=0)  Line 953 + 0x6 bytes	C
 	python26.dll!impl_tasklet_raise_exception(_tasklet *
self=0x00e6cbb0, _object * klass=0x1e20a098, _object *
args=0x1e215750)  Line 903 + 0xb bytes	C
 	python26.dll!PyTasklet_RaiseException(_tasklet * self=0x00e6cbb0,
_object * klass=0x1e20a098, _object * args=0x1e215750)  Line 883 +
0x1a bytes	C
 	TestApp_d.exe!Wait(_object * chan=0x00dd7e70)  Line 308 + 0x1b bytes	C++


However, setting the error string and throwing an exception does appear to work:


void Wait( PyObject *chan )
{
       PyObject *result =
PyChannel_Receive(reinterpret_cast<PyChannelObject *>(chan) );
       Py_XDECREF(result);     // result == NULL

       if (!result)
       {
              PyErr_SetString(PyExc_TaskletExit, "Sleeper killed");
              py::throw_error_already_set();
       }
}

The task does not resume executing the Python code in the tasklet
(thus it only prints once as expected), the task is no longer alive,
and the channel balance is now 0.  After assigning None to the task in
Python, the refcount will reach 0.  I do have to throw the exception;
there doesn't appear to be any state changes or clearing of the
exception if I don't, but failure to throw the exception acts like
nothing happened at all.  I'll write up a quick non-boost test case.

Today, after a fresh nights sleep (I was trying to diagnose this issue
for a couple hours last night, finally gave up and sent the email at
1:30 AM), I realized that with a C stack, it could be fairly dangerous
to kill a tasklet without properly unwinding it.  Fortunately I won't
have Python->C->Python cases; the C is lower level control for
blocking tasklets on asynchronous events (network, DB, file, etc).

- Chris Jacobson


On Mon, Feb 23, 2009 at 9:23 PM, Chris Jacobson <mainecoon at gmail.com> wrote:
> These changes had no effect on either of the test cases under 2.6.1 -
> they both continue to loop, the exception is ignored back up in the
> Python code.
>
> The diffs you provided only alter PyErr_PrintEx() ?  Was there more to it?
>
> - Chris
>
> On Mon, Feb 23, 2009 at 3:33 PM, Richard Tew <richard.m.tew at gmail.com> wrote:
>> On Mon, Feb 23, 2009 at 6:17 PM, Richard Tew <richard.m.tew at gmail.com> wrote:
>>> On Mon, Feb 23, 2009 at 4:24 AM, Chris Jacobson <mainecoon at gmail.com> wrote:
>>>> If a tasklet that is currently in a C function waiting on a
>>>> PyChannel_Receive is killed from another tasklet, the receiving
>>>> tasklet resumes with NULL returned on the Receive, and doesn't die.
>>>
>>> What version of Python are you using?  2.6, 3.0 or something earlier?
>>
>> Fixed.  In any case, I compiled against 3.0.1, as it was what I had
>> locally.  I'll try 2.6 as well, but if you encounter any more bugs,
>> please mention your platform and Python version.  I'll have to do
>> rebuilds and rereleases of 3.0 and 3.0.1, so let's hope there's not
>> any more :-)
>>
>> Cheers,
>> Richard.
>>
>> Author: richard.tew
>> Date: Tue Feb 24 00:27:50 2009
>> New Revision: 69919
>>
>> Log:
>> Fix for strange application restarting bug, where in some
>> circumstances an exception was compared to SystemExit, and if it
>> matched, the application was asked to exit.  TaskletExit is a subclass
>> of SystemExit, which means that when it is raised, if it hits this
>> execution path, the restarting occurs.
>>
>> Reported by Chris Jacobson.
>>
>> Modified:
>>  stackless/branches/release30-maint/Python/pythonrun.c
>>
>> Modified: stackless/branches/release30-maint/Python/pythonrun.c
>> ==============================================================================
>> --- stackless/branches/release30-maint/Python/pythonrun.c       (original)
>> +++ stackless/branches/release30-maint/Python/pythonrun.c       Tue
>> Feb 24 00:27:50 2009
>> @@ -1382,7 +1382,7 @@
>>  {
>>       PyObject *exception, *v, *tb, *hook;
>>
>> -       if (PyErr_ExceptionMatches(PyExc_SystemExit)) {
>> +       if (PyErr_ExceptionMatches(PyExc_SystemExit) &&
>> !PyErr_ExceptionMatches(PyExc_TaskletExit)) {
>>               handle_system_exit();
>>       }
>>       PyErr_Fetch(&exception, &v, &tb);
>> @@ -1408,7 +1408,7 @@
>>               PyObject *result = PyEval_CallObject(hook, args);
>>               if (result == NULL) {
>>                       PyObject *exception2, *v2, *tb2;
>> -                       if (PyErr_ExceptionMatches(PyExc_SystemExit)) {
>> +                       if (PyErr_ExceptionMatches(PyExc_SystemExit)
>> && !PyErr_ExceptionMatches(PyExc_TaskletExit)) {
>>                               handle_system_exit();
>>                       }
>>                       PyErr_Fetch(&exception2, &v2, &tb2);
>>
>




More information about the Stackless mailing list