[Stackless] Google Application Engine Thread on Stackless

Alain Poirier alain.poirier at net-ng.com
Thu Mar 26 22:44:27 CET 2009


On Thursday 26 March 2009 18:27:26 Kristján Valur Jónsson wrote:
> Is pickling really necessary?  Can't you just have the tasklet sleeping,
> until a HTTP request comes back? You can put a tasklet to sleep by calling
> stackless.schedule_remove() or by calling receive() on a channel. I can't
> see pickling being required unless you need to pass it to a different
> process. K

You're right, pickling is not required, but is convenient. In Nagare we pickle 
all the components graph for 3 reasons :

  1. To make a copy of the components graph on each request, to let the
     framework sanely and automatically handle the back and fork actions
     of the users. For example, imagine a scenario where a user clicks 10
     times on the "next" link of a grid component. Then, from the 10th page,
     he uses the back button to return to the 2nd one and finally click "next"
     again. With Nagare, the 3rd page is displayed, not the 11th. Or when a
     user uses the "Open in a new window" forking action, the two windows
     become really independent.

  2. To be able to migrate the graph, with these sleeping tasklets, between
     different processes. Which is the case in a multi-processes based HTTP
     server.

  3. To be able to deploy Nagare on a physical cluster of machines, without
     setting a sessions affinity aware dispatcher. In this case the pickles
     are sent to and retrieve from a memcached server.

Of course if :

  - For 1., you don't care about this capability (after all the users don't
    complain anymore about completely broken back button behaviour with the
    vast majority of ajaxified "web 2.0" applications),
  - 2., you have a tasklet based HTTP server like StacklessIO,
  - 3., you don't need a cluster or you can play the sessions affinity game

then you don't need to pickle. Note that, in this case, you could also use one 
of the stackless API emulation for greenlets to have a working Nagare on 
CPython.

> -----Original Message-----
> From: stackless-bounces at stackless.com
> [mailto:stackless-bounces at stackless.com] On Behalf Of Andrew Francis Sent:
> 26. mars 2009 11:43
> To: Christian Tismer; Alain Poirier
> Cc: stackless at stackless.com
> Subject: Re: [Stackless] Google Application Engine Thread on Stackless
>
>
> Hi Alain, Christian and Company:
>
> --- On Mon, 3/23/09, Alain Poirier <alain.poirier at net-ng.com> wrote:
> > From: Alain Poirier <alain.poirier at net-ng.com>
> > Subject: Re: [Stackless] Google Application Engine Thread on Stackless
> > To: "Christian Tismer" <tismer at stackless.com>
> > Cc: stackless at stackless.com, "Andrew Francis" <andrewfr_ice at yahoo.com>
> > Date: Monday, March 23, 2009, 5:10 PM
> >
> > Thanks Christian. But without the pickle enhancement you
> > created nothing could be possible. I personally think this golden
> >nugget is not known enough by the community. BTW, do you think it could be
> > >possible ? easy ? to extrac it into an extension module to the classic
> > >CPython ? A GSoC perhaps.
>
> It was pickling that originally attracted me to Stackless Python. Alain, I
> need to look at Nagare (and Seaside) more to understand how you use
> pickling. However what you are doing sounds cool.
>
> That said, at PyCon 2008, a few people after my talk, asked me about
> pickling. So I demoed it. I also had a really interesting conversation with
> some folks from Second Life.
>
> The main thing I use pickling for is to serialize the execution state of
> the WS-BPEL processor and individual WS-BPEL processes. However I think
> pickling may also simplify something called "compensation." Compensations
> are based on Molina's concept of a 'saga.' A part of compensation is to
> take snapshots of the system and in case of an error, roll the system back
> to that state in order to take corrective action.
>
> Implementing compensation will be loads of fun :-|
>
> About why more people don't know about pickling. My answer: need. I think
> we still live in a HTTP-centric web where the web server in handling a
> request, only needs to spawn a single process, talk to at most, a single
> machine at the backend and a transaction that takes a few seconds is
> considered an eternity. This is probably good for most web based
> activities. So stuff like Google Application Engine is okay for most
> developers. I think things break down quickly once you move out of this
> simple paradigm.
>
> Cheers,
> Andrew





More information about the Stackless mailing list