[Stackless] pi-ton 2.9 / Gillesspie
Christian Tismer
tismer at stackless.com
Fri Jan 3 01:44:33 CET 2014
As I told Andrew just right now,
I am not sure if we should instrument STM ourselves.
The matter is quite involved, and already pretty hard for PyPy.
Stackless on top of CPython is much more limited and should investigate
other ways which are orthogonal to STM and can be used in addition.
I'm thinking of something that is already available more or less, but
was never used:
In my post below, I talked about non-shared.
Tonight I had the even simpler idea of using different memory pools
and interpreter instances, together with disjoint globals, but still in
the same process space.
Threads, as they are today in Python, together with the globals and
how everything can reach everything, are the reason why we even _have_
the GIL problem.
In GIL-less Python we don't have this problem if we avoid normal threads.
There are no super-globals across mtasklets which can be freely accessed,
but everything must go through
some enforced protocol between the isolated threadpools.
So if you just use mtasklets and no normal threads at all, you can run
each of these super-threads in its own core, until you need to communicate
between these for some data exchange.
Still very vague, and still no idea how this works with resources like files
and GUI frameworks. That needs some exploration.
I need to play a bit more with the implications...
comments/ideas welcome! - Chris
On 02.01.14 14:31, Kristján Valur Jónsson wrote:
>
> Sounds exciting.
>
> I wonder how much of the STM work can be salvaged and made to work for us?
>
> Are you thinking about “automatic parallelism” like Armin Rigo has
> been thinking of, or just explicit tasklets that can, by nature of
> just using non-shared objects, can operate gil-less?
>
> non-shared is useful, but remember how everything in python Is
> interconnected. So, we might have to put practical limits on
> reachability.
>
> For example, a tasklet’s execution frame and hence its locals can be
> reached from anyone that has a reference to it. And I’m sure that you
> don’t want to mark locals and function arguments as non-shared.
>
> This incredible generalism of Python is what makes it also so hard to
> do any optimizations J. Time to remove some of that, perhaps?
>
> K
>
> *From:*stackless-bounces at stackless.com
> [mailto:stackless-bounces at stackless.com] *On Behalf Of *Christian Tismer
> *Sent:* 31. desember 2013 00:56
> *To:* The Stackless Python Mailing List
> *Subject:* Re: [Stackless] pi-ton 2.9 / Gillesspie
>
>
> -----------
>
> I want to go for a GIL-Less Python.
> That Python uses thread- (or better tasklet-) local storage instead of
> the globals.
> To make that possible, a completely new storage class needs to be
> installed,
> probably with a new keyword. This storage class will be automatically used
> by tasklets. They will use copy-on-write semantics to make all
> modifications
> local.
>
> The namespace is named "non-shared".
>
> The idea is to let things by default happen only locally, in a
> sub-interpreter.
> Objects that are not in the local namespace are installed copy-on-write,
> creating a non-shared object after being touched.
>
> The info about being sharable or non-shared is a new attribute for every
> object. The implementation is undecided, but may be implicit by using
> the memory pool information.
>
> A tasklet that uses "non-shared" data can run without the GIL, until it
> writes other data that it does not own privately.
>
> When a tasklet touches data that it does not own, then the regular
> actions dig in - either locking, or actions of an STM implementation,
> see the PyPy blogs.
>
> But the idea is to de-couple threads of python execution as much as
> possible,
> without making Python an intractable beast.
>
> For "normal" applications, Stackless 2.9 should just work as expected.
> But whenever the effect of an action should be globally visible, an extra
> function call is needed that opens the hidden state to others.
>
> This is the rendevous case that we try to avoid, most of the time.
>
> The needed functionality can be folded into Python at some cost for the
> general case. But I think it can become very effective on modern hardware
> and current software paradigms. Simultaneous access to certain things
> can be avoided in over 95 % of typical observations.
>
> Code Name
> ----------
>
> As always, I am better at assigning a code name than creating a really
> one.
> My suggestion is:
>
> - Gillespy
>
> This is not only a contribution to Dizzy Gillespie, but also a
> word-playing
> game for those who wonder what a GIL-less Py should be.
>
> I want to regard this post as an undirected starter.
>
> Opinions?
>
>
> --
> Christian Tismer :^)<mailto:tismer at stackless.com> <mailto:tismer at stackless.com>
> Software Consulting : Have a break! Take a ride on Python's
> Karl-Liebknecht-Str. 121 : *Starship*http://starship.python.net/
> 14482 Potsdam : PGP key ->http://pgp.uni-mainz.de
> phone +49 173 24 18 776 fax +49 (30) 700143-0023
> PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
> whom do you want to sponsor today?http://www.stackless.com/
>
>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless
--
Christian Tismer :^) <mailto:tismer at stackless.com>
Software Consulting : Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/
14482 Potsdam : PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776 fax +49 (30) 700143-0023
PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
whom do you want to sponsor today? http://www.stackless.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20140103/ed211c4b/attachment.html>
More information about the Stackless
mailing list