[Stackless] Stackless based replacement

Larry Dickson ldickson at cuttedge.com
Fri Oct 10 16:55:49 CEST 2008

Hi Arnar,

You said, "There have been discussions here occasionally about something
generic, like wrapping libevent or similar in an interface that "looks"
synchronous but in the background does async I/O and uses channels to make
it look synchronous." That is of course what this does. Points:
 - "There have been discussions" implies a blocking point ;-) This is a
design that is complete. simple, and proven, and could be put in the virtual
machine tomorrow.
 - The notion of "asynchronous IO" is vague and, in the typical manner,
therefore threatens to complicate matters with a huge palette (your
"myriad") of explicit tools that work at cross-purposes. By contrast, the
first phase of this design (Channel Communication plus Hardware Select)
would result in NO VISIBLE CHANGE, no libraries - just a clearing of the
"blocking" logjam and freedom for other tasklets to run efficiently while
the hardware wait is going on, just as David wanted.
 - There is also no visible change if you add the second phase, "Timer": the
sleeping tasklet (or simultaneously sleeping tasklets) would take no
blocking time on the round robin. (Your time.sleep code indicates this is
not currently the case - am I right, or does Py_BEGIN_ALLOW_THREADS set up a
separate queue?)
 - The complication added by these phases is in the virtual machine only: a
couple of new queues (channel and timer) in addition to Stackless's round
robin scheduler. The channel queues have maximum length 1. I am assuming
that "tasklet" = "process" in the sense I defined in the note (which
certainly appears to be the case, with the examples you gave). I am also
assuming there is some memory dedicated to the tasklet for its whole life
(but I can't imagine anything being "stackless" without that!).
 - The only thing that would create a new visible interpreted code
option would be the last phase, the user-level ALT or select. And that could
be done in any way that people like: for instance, as a standard
Unix-like select call, or a transputer-like ALT branching on readiness of IO
channels. In my experience, that is the central key to any serious parallel
coding (changing disorder into order, as it were).
 - And all without a myriad: the only other such structure that I've ever
found useful is a simple binary semaphore (which works just like the timer
queue, and avoids the ALT/select overhead in a common case). That could be
added later.
 - The only question is whether something about the object structure forbids
input from several channels in one tasklet.


On 10/9/08, Arnar Birgisson <arnarbi at gmail.com> wrote:
> Hi Larry,
> On Fri, Oct 10, 2008 at 00:54, Larry Dickson <ldickson at cuttedge.com>
> wrote:
> > On Fri, Sep 26 17:09:54 CEST 2008, Arnar Birgisson <arnarbi at gmail.com
> >
> >> > Surely there is a way around this?  Some kind of pooling select?  If
> >> > there is
> >> > no work around then I cannot see too much practical use for my thread
> >> > library
> >> > [except having to avoid learning tasklets for someone who is familiar
> >> > with
> >> > threads].  As I understand it, due to the GIL the only real practical
> >> > use for
> >> > threads is if one has blocking function calls (IO-type, etc)
> >>
> >> The solution would be asynchronous I/O. There have been discussions
> >> here occasionally about something generic, like wrapping libevent or
> >> similar in an interface that "looks" synchronous but in the background
> >> does async I/O and uses channels to make it look synchronous. I figure
> >> such a thing would be an excellent component of your thread library.
> >>
> >> > [Has the GIL restriction been fixed in 3k?  As far as I know Jython
> does
> >> > not
> >> > have this limitation...]
> >>
> >> The GIL has not been removed in Py 3.0, nor will it be removed any
> >> time soon. Jython does not have such a thing.
> >
> > This design solves all these problems, using only C/Unix select (which
> you
> > pointed out is already used to do time.sleep) in the virtual machine; and
> > it runs in only one thread. There is no need to remove the GIL.
> What problems? If you read my message you can clearly see that I
> pointed to a _solution_, namely asynchronous I/O. :) What I meant in
> my last message is that I don't see how your suggestion improves on
> the myriad of async solutions out there already (which are based on
> select, poll, epoll, Windows ASIO, etc).
> cheers,
> Arnar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20081010/61228bf2/attachment.htm>

More information about the Stackless mailing list