[Stackless] How to "transfer" a tasklet to the current thread

Anselm Kruis a.kruis at science-computing.de
Mon Nov 11 10:07:00 CET 2013


Hi Kristján,


Am 08.11.2013 10:31, schrieb Kristján Valur Jónsson:
>
>
>> -----Original Message-----
> data and decides, that the tasklet shall be run on a worker thread.
>>
>> Problem: as far as I know, a tasklet bound to a thread (via cstate) and this
>> association can't be changed. The best we can do is to create a new tasklet
>> on the worker thread, that belongs to the worker thread.
>
> This is right.  Although actually, for picklable tasklets (tasklets without a c state) this
> restriction Is artificial.

Adding a method to switch the thread could be a useful extension.


>
>>       reducedTask = task.__reduce__()
>>       # raise RuntimeError, if task is alive but not paused
>>       task.bind(None)
>>
>>       if True:  # python will crash if set to False
>>           frameList = reducedTask[2][3]
>>           for i in range(len(frameList)):
>>               frame = frameList[i]
>>               if isinstance(frame, stackless.cframe):
>>                   reducedFrame = frame.__reduce__()
>>                   newFrame = reducedFrame[0](*reducedFrame[1])
>>                   newFrame.__setstate__(reducedFrame[2])
>>                   frameList[i] = newFrame
>>       # rebind the task
>>       task = reducedTask[0](*reducedTask[1])
>>       task.__setstate__(reducedTask[2])
>>       return task
>>
>
> Looks like you are doing recursive pickling of the tasklet by hand, but only for the cframes, not regular frames.
> Why can't you just use
> return pickle.loads(pickle.dumps(task))?

A simple pickle.loads(pickle.dumps(task)) creates a deep-copy of the 
tasklet. There are two reasons to avoid the deep-copy:
1. it is fairly expensive
2. In this particular case the set of objects to be pickled could 
contain objects which register themselves upon unpickling with other 
components of the system and I don't want that to happen a second time.

But there is really a good point in your question. In a very quick first 
test, I observed only cframes in the frameList. If I understand you 
correctly, frameList could contain regular frames too. I'll adapt my 
method. But not today, a  customer is waiting.

>
> We can discuss the possibility of transferring picklable tasklets between threads.  As long as a tasklet is soft-switchable
> it really doesn't care which thread it runs on.  A "bind_to_thread() method could be provided...

Well, a method tasklet.bind_to_thread(thread) would be a sensible extension.

> But we can also think more of thread agnostic tasklets.
> We could even have a global run-queue of thread-agnostic tasklets that threads could access....
>
> I'd like to understand your use case, however.  It looks as though you are unpickling tasklets on one thread, then a pool of worker
> threads are accessing these tasklets and attempting to run them, is that right?  For that purpose, a worker thread needs to "claim"
> the tasklet as its own.--

It is really simple. A tasklet is part of the data, that defines an 
instance of a particular workflow. A user continues a serialised 
(pickled) flow by starting an executable with appropriate parameters in 
a shell (Unix-shell Windows cmd). The executable unpickles the flow data 
during command line procession (main-thread) and decides to display a 
GTK-GUI for further interaction with the user. In this case the 
executable uses the main thread for the GUI and one secondary thread for 
each flow.

Cheers
   Anselm

-- 
  Dipl. Phys. Anselm Kruis                       science + computing ag
  Senior Solution Architect                      Ingolstädter Str. 22
  email A.Kruis at science-computing.de             80807 München, Germany
  phone +49 89 356386 874  fax 737               www.science-computing.de
-- 
Vorstandsvorsitzender/Chairman of the board of management:
Gerd-Lothar Leonhart
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Michael Heinrichs, 
Dr. Arno Steitz, Dr. Ingrid Zech
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196




More information about the Stackless mailing list