[Stackless] Google Application Engine Thread on Stackless

[Andrew Francis]

Jeff:

>I'm not saying "pickling is bad" -- it has it's uses -- perhaps I am
>saying there is a dangerous "siren's call" there.... You have to be very
>careful in also assessing the costs of using it in a design.   Using 
>it as discussed certainly seems to violate one of the first principles of 
>information systems design -- put simply (and slightly inaccurately):
>"keep your data and your program separate" -- so one might assume that it >has some high (architectural) costs.

>This has hopped sideways into a philosophical design discussion and 
>perhaps doesn't belong on this list (mea cupla...)

I view pickling as akin to an operating system swapping out and swapping a process. The three differences are 1) Swapping is under programmer control. 2) potential time span. 3) The image has a different structure (but that is cosmetic to the discussion). 

I have no problem getting into philosophy.  "Keep your data and programme separate" is usually good. However one has to know when to break a rule. I have outlined a problem where we are not dealing with conventional data - we are dealing with execution state. 

In regard to design philosophy I subscribe to what Richard Gabriel calls "The Worse is Better" or New Jersey school. An excerpt from the "Worse is Better" paper (http://www.jwz.org/doc/worse-is-better.html):

<blockquote>

    * Simplicity -- the design must be simple, both in implementation and interface. It is more important for the implementation to be simple than the interface. Simplicity is the most important consideration in a design.
    * Correctness -- the design must be correct in all observable aspects. It is slightly better to be simple than correct.
    * Consistency -- the design must not be overly inconsistent. Consistency can be sacrificed for simplicity in some cases, but it is better to drop those parts of the design that deal with less common circumstances than to introduce either implementational complexity or inconsistency.
    * Completeness -- the design must cover as many important situations as is practical. All reasonably expected cases should be covered. Completeness can be sacrificed in favor of any other quality. In fact, completeness must sacrificed whenever implementation simplicity is jeopardized. Consistency can be sacrificed to achieve completeness if simplicity is retained; especially worthless is consistency of interface. 

</blockquote>

I would argue the above is pretty Pythonic too.

Back to the issue of restoring a pickled programme to an environment that has changed (i.e., the encompassing Python programme, the classes that constitute the pickled programme, the underlying interpreter). My policy would be simple:

DON'T DO IT.

Either depickle the programme in its original environment and let nature take its course. Or revive the programme in its original environment and try to gracefully terminate the programme. This is what I mean by sometimes you can't avoid bad things from happening and trying to mitigate the situation only makes thing worse. I had stuff like resumption semantics in mind.

It may turn out that pickling with simple termination rules covers 80% of the cases and results in an application that is 80% less complex. 

Jeff, I think it would be best to do experimentation. Come up with thought experiments and scenarios. Research how systems like Erlang handle these problems. Take a look at other WS-BPEL processors (as I mentioned before, Apache ODE has an approach to serialising execution state). Set up tests and do simulations to see what the issues are. Perhaps pickling will turn out to be dangerous and impractical. Or we learn how to make pickling safer and more effective - and in the process stake out new areas where Stackless Python can shine. 

Cheers,
Andrew