[Stackless] How to "transfer" a tasklet to the current thread

Anselm Kruis a.kruis at science-computing.de
Mon Nov 11 10:07:00 CET 2013

Hi Kristján,

Am 08.11.2013 10:31, schrieb Kristján Valur Jónsson:
>> -----Original Message-----
> data and decides, that the tasklet shall be run on a worker thread.
>> Problem: as far as I know, a tasklet bound to a thread (via cstate) and this
>> association can't be changed. The best we can do is to create a new tasklet
>> on the worker thread, that belongs to the worker thread.
> This is right.  Although actually, for picklable tasklets (tasklets without a c state) this
> restriction Is artificial.

Adding a method to switch the thread could be a useful extension.

>>       reducedTask = task.__reduce__()
>>       # raise RuntimeError, if task is alive but not paused
>>       task.bind(None)
>>       if True:  # python will crash if set to False
>>           frameList = reducedTask[2][3]
>>           for i in range(len(frameList)):
>>               frame = frameList[i]
>>               if isinstance(frame, stackless.cframe):
>>                   reducedFrame = frame.__reduce__()
>>                   newFrame = reducedFrame[0](*reducedFrame[1])
>>                   newFrame.__setstate__(reducedFrame[2])
>>                   frameList[i] = newFrame
>>       # rebind the task
>>       task = reducedTask[0](*reducedTask[1])
>>       task.__setstate__(reducedTask[2])
>>       return task
> Looks like you are doing recursive pickling of the tasklet by hand, but only for the cframes, not regular frames.
> Why can't you just use
> return pickle.loads(pickle.dumps(task))?

A simple pickle.loads(pickle.dumps(task)) creates a deep-copy of the 
tasklet. There are two reasons to avoid the deep-copy:
1. it is fairly expensive
2. In this particular case the set of objects to be pickled could 
contain objects which register themselves upon unpickling with other 
components of the system and I don't want that to happen a second time.

But there is really a good point in your question. In a very quick first 
test, I observed only cframes in the frameList. If I understand you 
correctly, frameList could contain regular frames too. I'll adapt my 
method. But not today, a  customer is waiting.

> We can discuss the possibility of transferring picklable tasklets between threads.  As long as a tasklet is soft-switchable
> it really doesn't care which thread it runs on.  A "bind_to_thread() method could be provided...

Well, a method tasklet.bind_to_thread(thread) would be a sensible extension.

> But we can also think more of thread agnostic tasklets.
> We could even have a global run-queue of thread-agnostic tasklets that threads could access....
> I'd like to understand your use case, however.  It looks as though you are unpickling tasklets on one thread, then a pool of worker
> threads are accessing these tasklets and attempting to run them, is that right?  For that purpose, a worker thread needs to "claim"
> the tasklet as its own.--

It is really simple. A tasklet is part of the data, that defines an 
instance of a particular workflow. A user continues a serialised 
(pickled) flow by starting an executable with appropriate parameters in 
a shell (Unix-shell Windows cmd). The executable unpickles the flow data 
during command line procession (main-thread) and decides to display a 
GTK-GUI for further interaction with the user. In this case the 
executable uses the main thread for the GUI and one secondary thread for 
each flow.


  Dipl. Phys. Anselm Kruis                       science + computing ag
  Senior Solution Architect                      Ingolstädter Str. 22
  email A.Kruis at science-computing.de             80807 München, Germany
  phone +49 89 356386 874  fax 737               www.science-computing.de
Vorstandsvorsitzender/Chairman of the board of management:
Gerd-Lothar Leonhart
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Michael Heinrichs, 
Dr. Arno Steitz, Dr. Ingrid Zech
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196

More information about the Stackless mailing list