[Stackless] Real Threading for multiprocessors

Christian Tismer tismer at tismer.com
Wed Jun 11 03:36:06 CEST 2003

Hi Breton,

>   I should have been more clear with my mail in the first place.  I
> would prefer to have manual access controls on critical data for maximum
> efficiency (like one would write using a C++/pthreads app).

So far this matches my expectations, exactly.

> However,
> there is a major tradeoff against interpreter safety.  I suspect that
> modifying a python list in two separate threads at the same time without
> access restriction could crash the interpreter - hence the 30%
> performance loss because locking is instated on each python object.

Clearly, here comes the cost. If you want the objects to be visible
and accessible all the time, you pay this overhead, whether you
are instrumenting cross-thread object access or not.

>   I like your idea of tasklets - but am wondering how they could be
> implemented.  One id4ea I'm tossing around is a TIL (replacing the
> GIL).  Each Thread Interpreter Lock would be acquired and held by an
> executing thread.  Objects created in threads would automatically be
> assigned ownership to the specific creating thread requiring no TIL
> locking to access.  However, for other threads requiring access to
> objects owned in other threads the TIL would need to be acquired.

Sure. In the first place, your application would not even know
that there is such a thing in another thread, that it would like
to access. After gathering knowledge of existence, both parties
would agree upon it, and give up their direct access.
Instead, they would always access these objects via special
proxy objects.

> Hence, multi-threading but automatic locking on object sharing.


> However, I'm concerned about the access costs to check permission on
> objects - it could degrade performance to the lock-per-object model.

How that! In a lock-per-object model, you are providing general
locking methods to any of all shared mutable objects. This is
only efficient, when a remarkable number of these objects really
gets involved between threads. I really assume that this is very
unlikely in practice. The only reason why this necessity appears to
get raised is the fact that Python had been designed so from the beginning
-- but for sure not with efficient object sharing in mind. ?

I think this is just a special case which was not interesting
enough to be designed carefully enough in the first place.
Without having explicit sharing of every and all objects at the same
time, I would not expect that anybody would support Python's model
of object sharing any longer, today, unless he really thinks to need
everything be accessible at the same time. I guess the number of
applications is rather limited.

>   I'm probably going to have to chase the CS literature for help on this
> one.  Anyone know any good papers on threading in interpreted languages?

Using your own good estimate about what makes sense is probably
fair enough. I also do believe that an algorithm that *tries*
to keep information as local as possible, is also superior than
others, which are requiring access to the whole "world" all the time.
There will of course exist counter-examples, of whom to hear
I'd be very glad and thankful.

sincerely -- chris

Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

Stackless mailing list
Stackless at www.tismer.com

More information about the Stackless mailing list