[Stackless] stackless and threads
Richard Emslie
rxe at ukshells.co.uk
Mon Jul 14 00:25:35 CEST 2003
On Sun, 13 Jul 2003, Vladimir Vukicevic wrote:
> On Sat, 2003-07-12 at 14:44, Christian Tismer wrote:
>
> > At the moment, under circumstances, it will crash.
> > I had that complaint before. What I have to do at least
> > is to avoid switches between tasklets from different threads.
> > I don't know whether it makes sense to build real thread
> > switching into tasklets, or just let them raise an exception.
>
> Mm. I was hoping to have the ability to migrate tasklets between
> threads.. i.e. create a tasklet in a thread that does, say, network
> processing, and then add it to the tasklet runnable queue of another OS
> thread. Thus the network thread could do natural poll() or whatnot
> based waiting for input, and the tasklets could be cooperatively
> scheduled on another thread.
In a similar vien, I was wanting to migrate tasklets between different
interpreters running in same process space. I would create the threads in
C using os primitives and create an independent python interpreter in each
one. Attempting to solve via running in completely seperate process
spaces, you just move the problem else where having to poll some io for
the communicating processes.
However, I am making the gross assumption that (a) the python interpreters
running in seperate threads are thread safe and (b) don't suffer from GIL.
Christian : can we get away with tasklet switches in seperate interpreters
or do we have the same problem?
An alternative solution would be to use nonblocking IO and create our own
poll like functions via a tasklet and a scheduler object. It wouldn't be
as efficient - but it would mean we could run the likes of networking code
in one thread. A high level library such as Pyro (stackless-ified ;-))
could be used to farm out work to other processes if the box has more than
processor and migrate pickled tasklets to other processes.
The second solution also allows us have very large number of connections
open without grinding the OS to halt with excessive amounts of threads &
context switching.
Cheers,
Richard
_______________________________________________
Stackless mailing list
Stackless at www.tismer.com
http://www.tismer.com/mailman/listinfo/stackless
More information about the Stackless
mailing list