[Stackless] st.serial --> st.serial_last_jump patch

Jeff Senn senn at maya.com
Wed Nov 25 16:06:29 CET 2009


On Nov 24, 2009, at 3:43 PM, Kristján Valur Jónsson wrote:

> Yes, what you describe sounds like a fair normal scenario and is precisely what the cstack's "serial" member is supposed to make work.
> 
> Now, in the first "main", that main tasklet returns.  You then enter a second "main" and when this is returning, it does an incorrect slp_transfer_return, somehow.
> 
> It is this last bit that is hard to understand, because it is difficult to see how it can be wrong.  The code is simple:
> 
> 	PyTaskletObject *task = ts->st.current;
> 	int ismain = task == ts->st.main;
> 
> ...
> 
> if (ismain) {
> 		if (ts->st.serial_last_jump != ts->st.serial) {
> 			slp_transfer_return(task->cstate);
> 
> So, the slp_transfer_return is only triggered if indeed it is _the_ main tasklet (the last "main" tasklet) that is returning.  And an slp_transfer_return() should be safe, because it will switch to the place where this tasklet created its initial stub, even if we are already on the correct stack.  If you follow the code through the debugger right past the actual stack switch, you should find yourself in make_initial_stub.

No... I find myself several levels down in slp_schedule_task inside PyStackless_RunWatchdogEx...
so the slp_transfer_return is bad, since at this point ts->frame has been cleared (==0) and there
are no frames to pop to get back up (this is the cause of the crash).  However why am I there?  
I don't know... coincidentally it is the place of the previous transfer into (and out of) the main tasklet...

> 
> Are you working on windows?

No. OS-X at the moment... but I have to get it to work "everywhere"...

I have a small failing test, see below...

> Now, I'm quite curious as to why your bug is happening, and we should fix it, but if all else fails, there is a workaround.  This is the workaround that we have employed in EVE for years, since before the time when multiple "main" tasklets became possible:  At the start of your program, enter a "main" context and stay there, by using PyStackless_Call_Main, having exposed your program's stackless_main function to python as a c function.  Then you stay within a main tasklet.  Something like

Hm... that appears to have the same problem (if I understand it, it maybe just avoids some overhead on
the main tasklet)

I have, just now, boiled it down to a test program that crashes:

test1.c
--------------------------------------------------

#include <Python.h>
int main(int argc, char *argv[]) {
  PyObject* globals = 0, *m, *f;

  Py_InitializeEx(0);
  m = PyImport_AddModule("__main__");
  if(m) globals = PyModule_GetDict(m);

  if((f = PyRun_String("execfile('./test1.py')",Py_file_input,globals,globals))) {
      Py_DECREF(f);
      f = PyDict_GetItemString(globals, "my_entry");
      if(f && PyCallable_Check(f)) {
	int i;
	PyObject* args = PyTuple_New(0);
	printf("Got callable to test...calling 10 times...");
	for(i=0; i<10; i++) {
	  printf("Calling %d\n",i);
	  PyObject *r
/*either way crashes*/
#if 0 
	    = PyStackless_Call_Main(f, args, NULL);
#else
	    = PyObject_CallObject(f, args);
#endif
	}
      }
  }
}


test1.py
--------------------------------------------------
import stackless 

def forever():
    while 1: pass

tlist = []
i = 0
def my_entry():
    global i
    if i == 0:
        tlist.append(stackless.tasklet(forever)())
    else:
        t = stackless.run(1000)
        print "Run returned",t
    i += 1








More information about the Stackless mailing list