[Stackless] CSPish Re: stacklesslib.async

Andrew Francis andrewfr_ice at yahoo.com
Tue Sep 3 21:05:51 CEST 2013

Hi Richard and Folks:

Message: 4
Date: Tue, 3 Sep 2013 11:05:46 +1200
From: Richard Tew <richard.m.tew at gmail.com>
To: The Stackless Python Mailing List <stackless at stackless.com>
Subject: Re: [Stackless] stacklesslib.async
    <CAN=X-TEZZ8P2vOgU2FDEiQ7WcWodHVY5-bkiJrjn0bBDzbzq+g at mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

I've created a different title but I do want to look at Kristjan's module and keep those comments independent.

>And having written all this, the elephant in the room is that I
>believe CSP, which Andrew Francis has long been advocating, is closer
>to what I think we need.  


It is my belief that synchronous first order channels (that is the channels themselves can be sent in messages) with buffering provides a simple yet powerful concurrency model. I strongly believe that we can hide asynchrony from the application programmer.

>Then again, why return indexes.  That just means you have to waste
>time holding onto the original list, if you do not have a permanent

Looking at Go's API version of Select in the reflect package, it also returns an index. I think returning an index is a more flexible way of identifying a completed channel from a big list. From my experience with my own implementations, you tend to want to create the list once before entering a loop. Hence you are holding on to it, even manipulating it along the way. Or for that matter, you are taking the result and doing some set inclusion operation. So again having the original list helps.

>However, I've long had problems with using it ad hoc, as I think it is not that user friendly and is >more theoretically balanced than functionally balanced.

I took my initial cues for an API from Polyphonic C# and the Concurrent Visual Basic join library. This was a mistake. Also the initial thrust of my work was to figure out what the internal model would look like, so API design was an afterthought. 

Until now, I have felt that folks on the Stackless mailing list have thrown the baby out with the bathwater when I introduced select and join. It doesn't help that I was defensive at the time, haven't released a copy of the code (it is messy with few tests) or for that matter given a good presentation. 

I have been redesigning the API and I am about to write a new module (I'll call it joinStackless.py so I don't create confusion with stackless.py). 

I decided that to keep the API simple by not introducing new objects (i.e., chanops, joinPatterns, methods on channels). Consequently you have to deal with lists and tuples and some constants. Then again, Python has comprehensions so this is a more Pythonic approach. Also from Pycon Canada 2012, I found I could simplify the user's experience by of the API by implementing a crude parametric polymorphism and type check arguments.

>But I think that this should not just handle tasklet completion, it
>should also handle tasklets producing results via channels.  In that
>case, it is useful to poll whether any tasklets have results, rather
>than waitX with a minimum timeout.

>I think that this should be in, but there surely can be a more natural
>fit.  And this might mean a requirement for lower level support.

To me, tasklets signalling completion through a channel is a perfectly fine way to do things. I think with C# style Tasks, you have to look at the result property (which may block, but I have to check this). As for the low level support. That is a select method with some join pattern support.

Right now I am toying with a new stackless.select (or stackless.alt, the name 'select' seems to confuse folks)

def select(operations, default = False)
      block on a set of operations until an operation is COMPLETED.
      in case of a tie, an operation is chosen at random
      if default is True, select returns if no operation has completed.

      operations List:  list of operations in the form the tuple (channel, channel operation, value)
                                ([], count)
                                where [] is a list of channels. Only RECV is allowed here
                                count the number of operations to wait on.

                                for bevity
                                if all operations must complete, the tuple can be omitted

      Return: tuple (index, value)  
      value is (None, None) if default invoked
      value is (index, None) if a SEND operation
      value is (index, [(index, value)]) if a join pattern has completed

To me, waitAny would look like

operations = [(chan1, RECV),  (chan2, RECV), (cha3, SEND, 10), (ch_timeout, RECV)]
(index, value) = stackless.select(operations)

as for waitAll

using the Santa Claus problem as an example:

elves = [ a list of elf channels ]
reindeer = [ a list of reindeer channels ]

(index, value) = stackless.select([(elves, 3), reindeer])

so if any 3 of the 10 elves have completed, say elf 0, elf 5 and elf 9

(0, [(0, value), (5, value), (9, value)])

if all nine reindeer completed,

(1,[0, value),(1, value) .... (8, value)])

Hopefully I'll start working on this during the weekend and get something out in a week that folks here can say Yeah or Nay or why don't you do it this way.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20130903/7fdea50a/attachment-0001.html>

More information about the Stackless mailing list