[Stackless] Multi core/cpu?
allen.fowler at yahoo.com
Fri Oct 19 08:17:26 CEST 2007
Richard Tew <richard.m.tew at gmail.com> wrote: On 10/18/07, Allen Fowler wrote:
> Is there any (near-term) hope to fix the common case of trying to easily get
> more than one CPU core in a single physical machine to process tasklets?
No. Not by anyone I can name on this list. But if you want it, I
encourage you to give it a shot. Most of the things people post about
wanting to this mailing list, they could probably write themselves if
they set themselves to it rather than to posting.
I suppose you are correct. However, as a python newbie, if I knew how to do this, I probably would not have posted here the first place.
Frankly, I am not sure I know what a "thread" really is.
I get the idea that a "process" has it's own slab of memory space it to which program code (and variables) are loaded.
As I understand it, the CPU then runs through the slab of memory executing instructions. (jmp-ing around inside as the code dictates. )
Now, threads supposedly "share a memory space". In my simplistic view of things, I don't see how that could work.
I mean variables would be changing under-foot at all times as other threads do their thing. (?!)
How could you even make something simple as a c-style for loop? The index variable would be clobbered by the other thread running the same code.
But, obviously, it is do-able. I just don't see how.
Put a Stackless Python interpreter on each core. Make aware of each
other. Then pickle the tasklets you want to move to other cores, and
send them there. Do the same across sockets to other cores on other
computers on the network. There are some technical issues involved in
which tasklets can be pickled in this way, but finding out what they
are is half the fun.
IPC is complex. IPC is slow. IPC is hard.
IPC (and concurrency) has already been figured out by many smart people. (MPI,Spread,etc.)
IPC seems like an overkill for my laptop's dual-core CPU.
I really don't want to fork/spawn/whatever a 2nd/3rd/4th 100MB+ Python process.
> It would be a huge boon to me, and i'm sure many others.
Then I hope you will find value in giving it a shot.
I'd like to, however I'd first need to understand how plain-old Threads work.
Then I'd need to understand how CPython works.
Then, I'd need to comprehend what exactly stackless is doing inside CPython.
All this being hampered by a near-zero knowledge of C.
And at the end, I have no guarantee that any of this is actually possible to do in a performance-sane way.
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Stackless