[Stackless] Multi-CPU Actor Based Python

OvermindDL1 overminddl1 at gmail.com
Fri Nov 21 23:02:12 CET 2008


I do not know if you all are still dealing with this, but I made a
library on top of Stackless Python that I named Pylang.  It is
basically a more 'Erlang'ish interface for Stackless, basically
instead of the channel model that Stackless uses, it exposed an Actor
model, worked near identically to Erlang's model, it could even
communicate to other processes through shared memory or the network,
or to other computers over a network.

However, it ran bog slower then Stackless, because of pickling
(working in a single process it ran admirably, but I needed
multi-computer usage).

I might still have it somewhere if anyone wanted it, mostly just test
code, very simple, was just getting it to work (I even had timeouts on
the receive working in the Erlang-style), and I gave it to someone on
a python list sometime a long while ago, but as stated, nothing overly
special due to restrictions in Python.  As long as you pretended like
it was an Actor model (not modifying messages after you send them and
other such things), it was perfect.

As stated though, Stackless Python still had far too many
restrictions, especially on speed (and what I *really* needed was
speed), lack of safety (practically everything is mutable, cannot
really fix that), etc., so I went back to my home language (C++).

Back in C++ I already had some code from multi-threaded work before
that used assembly for atomic CAS (like some of you mentioned earlier
in this thread) that worked so freakishly well for multi-threaded
programming without locking, and I knew the different way in which I
had to write the code quite well, but no one else seemed to be able to
wrap their head around it (it is *very* different).  So I started work
on a little C++ style Actor library like how I was doing in Stackless,
just to see if I could, damn the syntax at this point (I *needed*
speed for what I was doing, and since I would be the only real one
working on the project that needed it, screw safety as well, I know
how to work around all the safety issues, even if others could not).
It worked well enough, and some GCC specific language extensions made
much of it so easy, but it also needed to run on Windows XP, and the
work-arounds I came up with 'worked' well enough, but made the syntax
even more long and ugly.

I should note at this point that yes, I do know Erlang well enough to
use it, but I really *HATE* its syntax.  I do not mind functional
syntax, but I just really do not like that prologish syntax, I prefer
things more like C, Python, Lisp, etc...

I finished that basic project I needed it for and now had a lot more
experience with it and came up and discovered other methods for doing
things.  I wanted 'real' Actors at this point, with their own little
stacks and mutable vars and all.  My C++ hack 'worked', but I knew it
could be so much better without the compiler in the way.  So I thought
of making a language that compiled to C++ (as actors can be
represented as structs and so forth easily enough, the 'compiler to
C++' could easily see what all could possibly exist during a lifetime
of any given Actor).  Instead of compiling down to C++ I figured why
not just compile down to something even lower, so I delved into LLVM.

At this point is now about a half-year ago.  I started working on my
own language, just the back-end for now, testing the limits of the
LLVM compiler, making sure it worked on Windows (I poked the LLVM guys
until it did), etc...  Through my testing it would be a simple to make
my compiler see how large an Actor will ever get, and upon the
'spawn'ing (Erlang term) of that Actor, could allocate all the memory
it would need.  I also made it so there could be splits in the
allocated 'stack' as well, so if an Actor mostly resided using lower
amount of stack, then extra stack could be allocated and destroyed as
necessary at the split points.  It all worked quite well and it would
allow me to build an Actor model on top of it perfectly.  I then
started on the language itself about four months ago, basing it on a C
syntax initially (minus the Actor-harmful constructs like global vars
and such), but it started mutating a bit.  However, as of about three
months ago I got another job and classes started back up, so with
classes and two jobs (one in daytime, one in night), I have had just
about no time to work on it besides a little hour here and there every
week or so (as well as having almost no time to sleep), thus it has
stagnated for about 3 months now.  When the holiday break starts back
up in a few weeks, I will delve back into, donating all the time I
have.

At this point the language has only been shaped by comments from other
C++ users, and my experiences in Python/Erlang/etc...  As such it is
heavily C in style.  What I am doing sounds a lot like what you all
are wanting done.  This language compiles down to machine code, it
outperforms C in speed in most cases (thanks to LLVM's optimization
passes, it is about the same speed in all other cases), I have no
garbage collector as of yet (and no one has suggested I add one yet
either), but for Actor's it is mostly unnecessary, however I have been
thinking about adding one to manage a few behind-the-scenes memory
structures for speed reasons at the very least.

I love the Python syntax, but due to the fact it has only been other
C++ writers that have been suggesting things, there is very little
Python influence, and having some Python writers suggesting language
choices would probably be a great boon.  I am curious if any of you
might want some say in how the language is designed?  If anyone even
wanted to work on some code, that would be great as well as thus far I
have been the only coder.  As stated though, I may not be able to get
back into it for a few more weeks, and the language may not be usable
for a good year, at least, but over this break I do hope to get it to
a status where I can start writing code in it.




More information about the Stackless mailing list