[Stackless] Newbie questions

[Peter Kropf]

Out of curiousity, why did you switch from MPI to BSP?

- Peter

-----Original Message-----
From: stackless-admin at tismer.com [mailto:stackless-admin at tismer.com] On
Behalf Of Konrad Hinsen
Sent: Sunday, February 16, 2003 1:24 AM
To: Christian Tismer
Cc: stackless at tismer.com
Subject: Re: [Stackless] Newbie questions


On Sunday 16 February 2003 01:39, Christian Tismer wrote:

> Fascinating to have *you* in this. I'm pretty sure
> you will force me to implement MPI and other stuff, soon.

No! I have switched from MPI to BSP (see www.bsp-worldwide.org, or a recent 
copy of Scientific Python), and I intend to implement it in Stackless myself

:-)

> Yes. With 2.0, there is an issue, which didn't exist in 1.0, because 
> it wasn't able to switch extensions at all.

I could even live with that, as long as I can use my extensions at all.

> problem in callbacks. This is also the reason why 2.0 refuses to use 
> pre-emptive multitasking, since a switch could happen inside of an 
> extension that cannot stand it.

So when do task switches occur? Only when channel operations are called?

> In other words: If your extension does not call back into Python, or 
> if it does, but there are no switches intended, you have no problem.

Fine. So if I make sure that all callbacks set the "atomic" flag, then 
everything should be fine.

> The sent objects of channels are identical to the
> sent ones. This might change, since I want to use

But the object ids are not the same as the ones of the sent objects - as I 
just discovered experimentally.

> the channel protocol over different machines as well,
> but for now, this is implemented in the cheapest possible way.

Fine, then I make copies myself if needed.

> Very much interested!

OK, so here is a brief description. I work on high-level parallelization
using 
the BSP model. Python/BSP is explained in 
http://starship.python.net/~hinsen/ScientificPython/BSP_Tutorial.pdf, which 
for the moment is all the available documentation.  BSP divides a
computation 
in consecutive "compute" and "communicate" steps. One of the advantages is 
that communication is simple to implement, and easy to optimize.

My next project is automatic load balancing. The idea is to have many more 
"virtual" processors than real ones. Each real CPU measures the time spent
on 
dealing with each "virtual" CPU, and once in a while the real CPUs exchange 
virtual CPUs among each other to equalize CPU load.

The obvious solution for each "virtual" CPU would be a thread. But there are

two problems: Many low-level communications libraries don't coexist well
with 
threads, and there is a significant overhead, which is completely
unnecessary 
because pseudo-simultaneous execution is not even required, only the 
possiblity of task switches. Which is where Stackless comes in. I plan to 
make each virtual CPU a tasklet, which runs atomically most of the time, 
until it reaches the communication step and waits.

BTW, I saw hints about thread pickling, is that a reality already? It could
be 
useful in moving threads between CPUs.

Konrad.

_______________________________________________
Stackless mailing list
Stackless at www.tismer.com http://www.tismer.com/mailman/listinfo/stackless

_______________________________________________
Stackless mailing list
Stackless at www.tismer.com
http://www.tismer.com/mailman/listinfo/stackless