[Stackless] How to "transfer" a tasklet to the current thread
a.kruis at science-computing.de
Mon Nov 11 10:07:00 CET 2013
Am 08.11.2013 10:31, schrieb Kristján Valur Jónsson:
>> -----Original Message-----
> data and decides, that the tasklet shall be run on a worker thread.
>> Problem: as far as I know, a tasklet bound to a thread (via cstate) and this
>> association can't be changed. The best we can do is to create a new tasklet
>> on the worker thread, that belongs to the worker thread.
> This is right. Although actually, for picklable tasklets (tasklets without a c state) this
> restriction Is artificial.
Adding a method to switch the thread could be a useful extension.
>> reducedTask = task.__reduce__()
>> # raise RuntimeError, if task is alive but not paused
>> if True: # python will crash if set to False
>> frameList = reducedTask
>> for i in range(len(frameList)):
>> frame = frameList[i]
>> if isinstance(frame, stackless.cframe):
>> reducedFrame = frame.__reduce__()
>> newFrame = reducedFrame(*reducedFrame)
>> frameList[i] = newFrame
>> # rebind the task
>> task = reducedTask(*reducedTask)
>> return task
> Looks like you are doing recursive pickling of the tasklet by hand, but only for the cframes, not regular frames.
> Why can't you just use
> return pickle.loads(pickle.dumps(task))?
A simple pickle.loads(pickle.dumps(task)) creates a deep-copy of the
tasklet. There are two reasons to avoid the deep-copy:
1. it is fairly expensive
2. In this particular case the set of objects to be pickled could
contain objects which register themselves upon unpickling with other
components of the system and I don't want that to happen a second time.
But there is really a good point in your question. In a very quick first
test, I observed only cframes in the frameList. If I understand you
correctly, frameList could contain regular frames too. I'll adapt my
method. But not today, a customer is waiting.
> We can discuss the possibility of transferring picklable tasklets between threads. As long as a tasklet is soft-switchable
> it really doesn't care which thread it runs on. A "bind_to_thread() method could be provided...
Well, a method tasklet.bind_to_thread(thread) would be a sensible extension.
> But we can also think more of thread agnostic tasklets.
> We could even have a global run-queue of thread-agnostic tasklets that threads could access....
> I'd like to understand your use case, however. It looks as though you are unpickling tasklets on one thread, then a pool of worker
> threads are accessing these tasklets and attempting to run them, is that right? For that purpose, a worker thread needs to "claim"
> the tasklet as its own.--
It is really simple. A tasklet is part of the data, that defines an
instance of a particular workflow. A user continues a serialised
(pickled) flow by starting an executable with appropriate parameters in
a shell (Unix-shell Windows cmd). The executable unpickles the flow data
during command line procession (main-thread) and decides to display a
GTK-GUI for further interaction with the user. In this case the
executable uses the main thread for the GUI and one secondary thread for
Dipl. Phys. Anselm Kruis science + computing ag
Senior Solution Architect Ingolstädter Str. 22
email A.Kruis at science-computing.de 80807 München, Germany
phone +49 89 356386 874 fax 737 www.science-computing.de
Vorstandsvorsitzender/Chairman of the board of management:
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Michael Heinrichs,
Dr. Arno Steitz, Dr. Ingrid Zech
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196
More information about the Stackless