[Stackless] Stackless based replacement

Andrew Francis andrewfr_ice at yahoo.com
Wed Oct 22 18:33:49 CEST 2008


Hello Larry:


>What is needed is the PAR construct: I do not believe any combination of
>channels can do the same exactly. If C starts after (A|B) finish, etc, >then the occam pseudocode (which I hope is self-explanatory) is:

I solved my problem in 2005. I wrote a synchronizer construct similar to what I previously posted. I believe the uthread module had counting semaphores. I make the assumption that tasklets terminate soon after they join. This keeps the code simple. Under the hood, only one, not number of children + 1  channels, are used.

That said, I think we are going around in circles....  

I think Jeff Senn is right in that most Stackless users have a similar philosophical approach - that is if you want a new construct - roll your own in a module, rather than add to Stackless. Again, I believe channels are a very very powerful building block. However channels by themselves can be a very blunt instrument for complex tasks.

In regard to networking, I think the anarchical approach has worked. Some Stackless users like stacklessSocket module with LibEvent. I like using Twisted (and Twisted could be used with LibEvent). 

Since I believe this thread originally began with asynchronous I/O, regardless of technique, there are basic gotchas. Perhaps if we could all agree on the basic gotchas, this knowledge could be modelled with PyPy and if it works, be placed in Stackless Python.

Cheers,
Andrew




--- On Wed, 10/22/08, Larry Dickson <ldickson at cuttedge.com> wrote:

> From: Larry Dickson <ldickson at cuttedge.com>
> Subject: Re: [Stackless] Stackless based replacement
> To: "Andrew Francis" <andrewfr_ice at yahoo.com>
> Cc: stackless at stackless.com
> Date: Wednesday, October 22, 2008, 7:21 AM
> Hi Andrew,
> 
> What is needed is the PAR construct: I do not believe any
> combination of
> channels can do the same exactly. If C starts after (A|B)
> finish, etc, then
> the occam pseudocode (which I hope is self-explanatory) is:
> 
> SEQ
>   PAR
>     A
>     B
>   --END PAR
>   C
>   PAR
>     D
>     E
>   --END PAR
>   F
> 
> If you want to start them all at the same time it would
> require some
> channels to make them end in order:
> 
> PAR
>   SEQ
>     PAR
>       SEQ
>         PAR
>           A
>           B
>         --END PAR
>         communicate to C
>       C (communicate from AB)
>     --END PAR
>     communicate to D
>     communicate to E
>   SEQ
>     PAR
>       D (communicate from ABC)
>       E (communicate from ABC)
>     --END PAR
>     communicate to F
>   F (communicate from DE)
> --END PAR
> 
> A PAR requires the parent store two words (its --END PAR
> resumption
> instruction address and n-1 where n is the number of
> children) and start
> (i.e. schedule) n-1 child tasklets (and branch to the
> n-th). Each child must
> terminate with an 'endp' which checks the second
> word. If it is >0 it
> decrements it and abandons its instruction stream. If it is
> 0 it branches to
> the parent's resumption address.
> 
> Larry
> 
> On 10/22/08, Andrew Francis <andrewfr_ice at yahoo.com>
> wrote:
> >
> > Larry:
> >
> > >Do you mean C starts after (A|B) finish, or they
> all start together, or
> > >..?
> >
> > Why did I write that in 2005?
> >
> > Yes C should start after A AND B  finish. However the
> way the code is
> > written,  tasklets actually start before they block on
>  channels.
> >
> > The point of that example was it was *very* easy to
> create a deadlock
> > scenario with a relatively simple precedent graph
> implemented with just
> > channels.
> >
> > Cheers,
> > Andrew
> >
> >
> >
> >
> >
> >
> > --- On Tue, 10/21/08, Larry Dickson
> <ldickson at cuttedge.com> wrote:
> >
> > > From: Larry Dickson <ldickson at cuttedge.com>
> > > Subject: Re: [Stackless] Stackless based
> replacement
> > > To: andrewfr_ice at yahoo.com
> > > Cc: stackless at stackless.com
> > > Date: Tuesday, October 21, 2008, 9:46 AM
> > > Hi Andrew,
> > >
> > > I read your 2005 note, and I do not understand
> > >
> > > >I expect the following execution trace
> > > >
> > > >(A | B) C (D | E) F
> > > >
> > > >|* - means processes can execute in parallel.
> > > *>
> > > >It is okay to see B finish before A. However
> it is
> > > >wrong to see C finish before B and A finish.
> > >
> > > Do you mean C starts after (A|B) finish, or they
> all start
> > > together, or ..?
> > >
> > > Larry
> > >
> > > On 10/21/08, Andrew Francis
> <andrewfr_ice at yahoo.com>
> > > wrote:
> > > >
> > > > Hi Larry:
> > > >
> > > > > We seem to have different ideas of what
> is
> > > simple. You
> > > > > propose exceptions, waits, signals,
> barriers,
> > > re-initialization,
> > > > > and presumably global signal values.
> This in my
> > > opinion is a whole
> > > > >Italian restaurant of spaghetti, and it
> sounds
> > > intrinsically global, >which
> > > > is poison to maintainable multiprocessing in
> my
> > > experience.
> > > >
> > > > This is what I had in mind. A sketch.
> > > >
> > > > def producer(synchronizer):
> > > >    try:
> > > >        # do some computation
> > > >        synchronizer.signal(result)
> > > >    except Signalled, data
> > > >        # oops the consumer has moved on....
> maybe
> > > >        # I should gracefully terminate.....
> > > >
> > > >
> > > > def consumer(synchronizer):
> > > >    output = synchronizer.wait()
> > > >    # okay, let us do some stuff with the
> output
> > > >
> > > > synchronizer =
> Synchronizer(numberOfProducers)
> > > > stackless.tasklet(consumer)(synchronizer)
> > > > for .... :
> > > >    stackless.tasklet(producer)(synchronizer)
> > > >
> > > > Most of the effort is in defining the right
> > > behaviour.....
> > > >
> > > > Once again, my experiences are when you
> naively use
> > > channels, it is easy to
> > > > get in trouble.  A construct like
> receive_first()
> > > looks difficult to
> > > > implement, in comparison to a synchronizer.
> > > >
> > > > Here is a thread from December 2005 (2005!)
> > > >
> > > >
> > >
> http://www.stackless.com/pipermail/stackless/2005-December/000290.html
> > > >
> > > > Cheers,
> > > > Andrew
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> >
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam
> protection around
> > http://mail.yahoo.com
> >


      




More information about the Stackless mailing list