[Stackless] Stackless Multicore Module

Carlos Eduardo de Paula carlosedp at gmail.com
Wed Mar 26 23:21:59 CET 2008


One nice thing about this implementation is testing it using the
Processing and the Threading modules. If you switch only one line in
the stacklessmulticore.py module and run the test_smc2.py that
performs like a very crude benchmark, we can see a big improvement in
performance showing that the GIL is taking its toll on the Threading
implementation.

Just switch the line:
p = processing.Process(target=processSpawner, args=[n, processesChannels[n]])

To:
p = threading.Thread(target=processSpawner, args=[n, processesChannels[n]])

And run the test_smc2.py... in my computer I got almost double
performance increase... (below follows the print for the test for each
module).

Processing:
...
Woke after  1  seconds. ( 1.0 )
Woke after  2  seconds. ( 2.0 )
100000  operations took:  2.40700006485  seconds.
Woke after  3  seconds. ( 3.0 )
200000  operations took:  4.78200006485  seconds.
Woke after  5  seconds. ( 5.0 )
300000  operations took:  6.70300006866  seconds.
400000  operations took:  8.78200006485  seconds.
500000  operations took:  10.6560001373  seconds.


Threading:
...
Woke after  0.2  seconds. ( 0.202999830246 )
Woke after  1  seconds. ( 1.0 )
Woke after  2  seconds. ( 2.0 )
Woke after  3  seconds. ( 3.0 )
100000  operations took:  4.5  seconds.
Woke after  5  seconds. ( 5.0 )
200000  operations took:  7.73399996758  seconds.
300000  operations took:  12.0309998989  seconds.
400000  operations took:  15.8120000362  seconds.
500000  operations took:  19.9680001736  seconds.

Carlos


On Wed, Mar 26, 2008 at 11:40 AM, Andrew Francis <andrewfr_ice at yahoo.com> wrote:
> Hi Carlos:
>
>
>  >I have created a small sandbox module to test running
>  >stackless tasklets over multiple processes instead of
>  >multiple threads. With whis we can overcome the
>  >problem with GIL and use full power from multicore
>  >processors.
>
>  This sounds neat! I just came back from Pycon 2008 and
>  I was asked about how I would go about having
>  Stackless support multiple CPUs. I also attended a
>  open session on multi-processor support in Python.
>
>
>  >Its far from complete but it does the tasklet
>  >distribution on a round-robin fashion and does the
>  job >of starting a scheduler for each process.
>
>  >It would be interesting to distribute the scheduler
>  >and have a CPU for each scheduler. If anyone is up to
>
> >help me out in this idea or have suggestions
>
>  I have to find a computer with multiple CPUs, but I
>  would be interested in trying this out.
>
>  Sounds like really cool work Carlos!
>
>  Cheers,
>  Andrew
>
>
>
>
>       ____________________________________________________________________________________
>  Looking for last minute shopping deals?
>  Find them fast with Yahoo! Search.  http://tools.search.yahoo.com/newsearch/category.php?category=shopping
>
>
>
>  _______________________________________________
>  Stackless mailing list
>  Stackless at stackless.com
>  http://www.stackless.com/mailman/listinfo/stackless
>



-- 
-------------------------------------------------------------------
Visit Stackless Examples Project
http://code.google.com/p/stacklessexamples/
Stackless Python - www.stackless.com
-------------------------------------------------------------------




More information about the Stackless mailing list