[Stackless] How to "transfer" a tasklet to the current thread

Kristján Valur Jónsson kristjan at ccpgames.com
Mon Nov 11 16:34:46 CET 2013


Ok, I have made some commits to the 2.7 branch.
I found some bugs which are probably the reason you were seeing a crash when not manually cloning the frames.
I wasn't able to reproduce your crash, but at least I found something very broken which I fixed.
I have also added the experimental bind_thread() method, and some unittests in test_thread.
Can you see If it works as expected?

K

> -----Original Message-----
> From: stackless-bounces at stackless.com [mailto:stackless-
> bounces at stackless.com] On Behalf Of Anselm Kruis
> Sent: 11. nóvember 2013 09:07
> To: stackless at stackless.com
> Subject: Re: [Stackless] How to "transfer" a tasklet to the current thread
> 
> Hi Kristján,
> 
> 
> Am 08.11.2013 10:31, schrieb Kristján Valur Jónsson:
> >
> >
> >> -----Original Message-----
> > data and decides, that the tasklet shall be run on a worker thread.
> >>
> >> Problem: as far as I know, a tasklet bound to a thread (via cstate)
> >> and this association can't be changed. The best we can do is to
> >> create a new tasklet on the worker thread, that belongs to the worker
> thread.
> >
> > This is right.  Although actually, for picklable tasklets (tasklets
> > without a c state) this restriction Is artificial.
> 
> Adding a method to switch the thread could be a useful extension.
> 
> 
> >
> >>       reducedTask = task.__reduce__()
> >>       # raise RuntimeError, if task is alive but not paused
> >>       task.bind(None)
> >>
> >>       if True:  # python will crash if set to False
> >>           frameList = reducedTask[2][3]
> >>           for i in range(len(frameList)):
> >>               frame = frameList[i]
> >>               if isinstance(frame, stackless.cframe):
> >>                   reducedFrame = frame.__reduce__()
> >>                   newFrame = reducedFrame[0](*reducedFrame[1])
> >>                   newFrame.__setstate__(reducedFrame[2])
> >>                   frameList[i] = newFrame
> >>       # rebind the task
> >>       task = reducedTask[0](*reducedTask[1])
> >>       task.__setstate__(reducedTask[2])
> >>       return task
> >>
> >
> > Looks like you are doing recursive pickling of the tasklet by hand, but only
> for the cframes, not regular frames.
> > Why can't you just use
> > return pickle.loads(pickle.dumps(task))?
> 
> A simple pickle.loads(pickle.dumps(task)) creates a deep-copy of the tasklet.
> There are two reasons to avoid the deep-copy:
> 1. it is fairly expensive
> 2. In this particular case the set of objects to be pickled could contain objects
> which register themselves upon unpickling with other components of the
> system and I don't want that to happen a second time.
> 
> But there is really a good point in your question. In a very quick first test, I
> observed only cframes in the frameList. If I understand you correctly,
> frameList could contain regular frames too. I'll adapt my method. But not
> today, a  customer is waiting.
> 
> >
> > We can discuss the possibility of transferring picklable tasklets
> > between threads.  As long as a tasklet is soft-switchable it really doesn't
> care which thread it runs on.  A "bind_to_thread() method could be
> provided...
> 
> Well, a method tasklet.bind_to_thread(thread) would be a sensible
> extension.
> 
> > But we can also think more of thread agnostic tasklets.
> > We could even have a global run-queue of thread-agnostic tasklets that
> threads could access....
> >
> > I'd like to understand your use case, however.  It looks as though you
> > are unpickling tasklets on one thread, then a pool of worker threads are
> accessing these tasklets and attempting to run them, is that right?  For that
> purpose, a worker thread needs to "claim"
> > the tasklet as its own.--
> 
> It is really simple. A tasklet is part of the data, that defines an instance of a
> particular workflow. A user continues a serialised
> (pickled) flow by starting an executable with appropriate parameters in a
> shell (Unix-shell Windows cmd). The executable unpickles the flow data
> during command line procession (main-thread) and decides to display a GTK-
> GUI for further interaction with the user. In this case the executable uses the
> main thread for the GUI and one secondary thread for each flow.
> 
> Cheers
>    Anselm
> 
> --
>   Dipl. Phys. Anselm Kruis                       science + computing ag
>   Senior Solution Architect                      Ingolstädter Str. 22
>   email A.Kruis at science-computing.de             80807 München, Germany
>   phone +49 89 356386 874  fax 737               www.science-computing.de
> --
> Vorstandsvorsitzender/Chairman of the board of management:
> Gerd-Lothar Leonhart
> Vorstand/Board of Management:
> Dr. Bernd Finkbeiner, Michael Heinrichs,
> Dr. Arno Steitz, Dr. Ingrid Zech
> Vorsitzender des Aufsichtsrats/
> Chairman of the Supervisory Board:
> Philippe Miltin
> Sitz/Registered Office: Tuebingen
> Registergericht/Registration Court: Stuttgart
> Registernummer/Commercial Register No.: HRB 382196
> 
> 
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless





More information about the Stackless mailing list