[Stackless] Stackless Multicore Module
Carlos Eduardo de Paula
carlosedp at gmail.com
Wed Mar 26 23:21:59 CET 2008
One nice thing about this implementation is testing it using the
Processing and the Threading modules. If you switch only one line in
the stacklessmulticore.py module and run the test_smc2.py that
performs like a very crude benchmark, we can see a big improvement in
performance showing that the GIL is taking its toll on the Threading
implementation.
Just switch the line:
p = processing.Process(target=processSpawner, args=[n, processesChannels[n]])
To:
p = threading.Thread(target=processSpawner, args=[n, processesChannels[n]])
And run the test_smc2.py... in my computer I got almost double
performance increase... (below follows the print for the test for each
module).
Processing:
...
Woke after 1 seconds. ( 1.0 )
Woke after 2 seconds. ( 2.0 )
100000 operations took: 2.40700006485 seconds.
Woke after 3 seconds. ( 3.0 )
200000 operations took: 4.78200006485 seconds.
Woke after 5 seconds. ( 5.0 )
300000 operations took: 6.70300006866 seconds.
400000 operations took: 8.78200006485 seconds.
500000 operations took: 10.6560001373 seconds.
Threading:
...
Woke after 0.2 seconds. ( 0.202999830246 )
Woke after 1 seconds. ( 1.0 )
Woke after 2 seconds. ( 2.0 )
Woke after 3 seconds. ( 3.0 )
100000 operations took: 4.5 seconds.
Woke after 5 seconds. ( 5.0 )
200000 operations took: 7.73399996758 seconds.
300000 operations took: 12.0309998989 seconds.
400000 operations took: 15.8120000362 seconds.
500000 operations took: 19.9680001736 seconds.
Carlos
On Wed, Mar 26, 2008 at 11:40 AM, Andrew Francis <andrewfr_ice at yahoo.com> wrote:
> Hi Carlos:
>
>
> >I have created a small sandbox module to test running
> >stackless tasklets over multiple processes instead of
> >multiple threads. With whis we can overcome the
> >problem with GIL and use full power from multicore
> >processors.
>
> This sounds neat! I just came back from Pycon 2008 and
> I was asked about how I would go about having
> Stackless support multiple CPUs. I also attended a
> open session on multi-processor support in Python.
>
>
> >Its far from complete but it does the tasklet
> >distribution on a round-robin fashion and does the
> job >of starting a scheduler for each process.
>
> >It would be interesting to distribute the scheduler
> >and have a CPU for each scheduler. If anyone is up to
>
> >help me out in this idea or have suggestions
>
> I have to find a computer with multiple CPUs, but I
> would be interested in trying this out.
>
> Sounds like really cool work Carlos!
>
> Cheers,
> Andrew
>
>
>
>
> ____________________________________________________________________________________
> Looking for last minute shopping deals?
> Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
>
>
>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless
>
--
-------------------------------------------------------------------
Visit Stackless Examples Project
http://code.google.com/p/stacklessexamples/
Stackless Python - www.stackless.com
-------------------------------------------------------------------
More information about the Stackless
mailing list