[Stackless] Real Threading for multiprocessors

Christian Tismer tismer at tismer.com
Tue Jun 10 19:28:42 CEST 2003

Breton M. Saunders wrote:

> Guys,
>   I've been designing a large-scale server at work that could benefit
> from running multiple threads in python simultaneously - aka real
> threading, not the sequential-execution threading provided by the
> standard python.  Currently I implement threading at the C-code level
> only.
>   I have two questions:
>     1) Does stackless provide a real threading model providing N
> simultaneously executing threads on N processors?  (I suspect not AFAIK)

Right, it doesn't, yet.

>     2) How complex would it be to make the interpreter _really_ thread
> safe?

 From former experiements of Greg Stein, I have the memory of about
30 percent extra overhead, for making every mutable thread-safe.

>   I've been looking at the stackless code; but have really only just
> started.

I have been told of applications which would benefit from free
threading with general shared objects, regardless the rather high costs.

My personal taste is a bit different: I think it is more efficient
to have free-running, disjoint interpreters, and to make object
access between threads explicit, involving locks. This would allow
to let the available CPUs run at full speed as long as their
computation is local, and only to involve some special communication
wrappers run when information is interchanged.

I was pointed to an implementation by Stephan Diehl, which does this,
it is named "POSH", available at http://poshmodule.sourceforge.net

In combination with Stackless, this might become an interesting
combination. (Didn't investigate further, yet)

My vision is about tasklets, which are run by a number of
Python processes, one for each processor. These tasklets
should be able to migrate, like "nomad tasks", to the
processor with the smallest work-load.

Something like this might become my next challenge after
Stackless 3.0 ist presented on EuroPy.

cheers - chris
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

Stackless mailing list
Stackless at www.tismer.com

More information about the Stackless mailing list