[Stackless] How to "transfer" a tasklet to the current thread
Kristján Valur Jónsson
kristjan at ccpgames.com
Mon Nov 11 16:34:46 CET 2013
Ok, I have made some commits to the 2.7 branch.
I found some bugs which are probably the reason you were seeing a crash when not manually cloning the frames.
I wasn't able to reproduce your crash, but at least I found something very broken which I fixed.
I have also added the experimental bind_thread() method, and some unittests in test_thread.
Can you see If it works as expected?
K
> -----Original Message-----
> From: stackless-bounces at stackless.com [mailto:stackless-
> bounces at stackless.com] On Behalf Of Anselm Kruis
> Sent: 11. nóvember 2013 09:07
> To: stackless at stackless.com
> Subject: Re: [Stackless] How to "transfer" a tasklet to the current thread
>
> Hi Kristján,
>
>
> Am 08.11.2013 10:31, schrieb Kristján Valur Jónsson:
> >
> >
> >> -----Original Message-----
> > data and decides, that the tasklet shall be run on a worker thread.
> >>
> >> Problem: as far as I know, a tasklet bound to a thread (via cstate)
> >> and this association can't be changed. The best we can do is to
> >> create a new tasklet on the worker thread, that belongs to the worker
> thread.
> >
> > This is right. Although actually, for picklable tasklets (tasklets
> > without a c state) this restriction Is artificial.
>
> Adding a method to switch the thread could be a useful extension.
>
>
> >
> >> reducedTask = task.__reduce__()
> >> # raise RuntimeError, if task is alive but not paused
> >> task.bind(None)
> >>
> >> if True: # python will crash if set to False
> >> frameList = reducedTask[2][3]
> >> for i in range(len(frameList)):
> >> frame = frameList[i]
> >> if isinstance(frame, stackless.cframe):
> >> reducedFrame = frame.__reduce__()
> >> newFrame = reducedFrame[0](*reducedFrame[1])
> >> newFrame.__setstate__(reducedFrame[2])
> >> frameList[i] = newFrame
> >> # rebind the task
> >> task = reducedTask[0](*reducedTask[1])
> >> task.__setstate__(reducedTask[2])
> >> return task
> >>
> >
> > Looks like you are doing recursive pickling of the tasklet by hand, but only
> for the cframes, not regular frames.
> > Why can't you just use
> > return pickle.loads(pickle.dumps(task))?
>
> A simple pickle.loads(pickle.dumps(task)) creates a deep-copy of the tasklet.
> There are two reasons to avoid the deep-copy:
> 1. it is fairly expensive
> 2. In this particular case the set of objects to be pickled could contain objects
> which register themselves upon unpickling with other components of the
> system and I don't want that to happen a second time.
>
> But there is really a good point in your question. In a very quick first test, I
> observed only cframes in the frameList. If I understand you correctly,
> frameList could contain regular frames too. I'll adapt my method. But not
> today, a customer is waiting.
>
> >
> > We can discuss the possibility of transferring picklable tasklets
> > between threads. As long as a tasklet is soft-switchable it really doesn't
> care which thread it runs on. A "bind_to_thread() method could be
> provided...
>
> Well, a method tasklet.bind_to_thread(thread) would be a sensible
> extension.
>
> > But we can also think more of thread agnostic tasklets.
> > We could even have a global run-queue of thread-agnostic tasklets that
> threads could access....
> >
> > I'd like to understand your use case, however. It looks as though you
> > are unpickling tasklets on one thread, then a pool of worker threads are
> accessing these tasklets and attempting to run them, is that right? For that
> purpose, a worker thread needs to "claim"
> > the tasklet as its own.--
>
> It is really simple. A tasklet is part of the data, that defines an instance of a
> particular workflow. A user continues a serialised
> (pickled) flow by starting an executable with appropriate parameters in a
> shell (Unix-shell Windows cmd). The executable unpickles the flow data
> during command line procession (main-thread) and decides to display a GTK-
> GUI for further interaction with the user. In this case the executable uses the
> main thread for the GUI and one secondary thread for each flow.
>
> Cheers
> Anselm
>
> --
> Dipl. Phys. Anselm Kruis science + computing ag
> Senior Solution Architect Ingolstädter Str. 22
> email A.Kruis at science-computing.de 80807 München, Germany
> phone +49 89 356386 874 fax 737 www.science-computing.de
> --
> Vorstandsvorsitzender/Chairman of the board of management:
> Gerd-Lothar Leonhart
> Vorstand/Board of Management:
> Dr. Bernd Finkbeiner, Michael Heinrichs,
> Dr. Arno Steitz, Dr. Ingrid Zech
> Vorsitzender des Aufsichtsrats/
> Chairman of the Supervisory Board:
> Philippe Miltin
> Sitz/Registered Office: Tuebingen
> Registergericht/Registration Court: Stuttgart
> Registernummer/Commercial Register No.: HRB 382196
>
>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless
More information about the Stackless
mailing list