[Stackless] Planning Stackless Lite

[Andrew Francis]

Hi Christian:

--- On Tue, 2/2/10, Christian Tismer <tismer at stackless.com> wrote:

CT> This sounds strange without some commentary:
CT> PyPy supports soft switching only. This is not a
CT> deficiency, but a virtue.

I wasn't implying that soft switching was a deficiency. Rather I 
thought it would be a benefit. From reading documentation, there is
an order of magnitude difference in speed between hard and soft
switching. I thought that even though  PyPy is slower than CPython/Stackless, the soft switching should allow a PyPy programme 
that uses I/O to get comparable performance to its CPython/Stackless counterpart.

I am assuming the two major bottlenecks are the context switching and the
system calls. 

CT> This soft switching is possible since PyPy has full control
CT> over itself. And in this context, things like Greenlets do not
CT> exist.  

Okay. I am trying to get a handle on this as a end-user. If I run say
a simple network test with pypy-c compiled with the --stackless option,
all other things being equal, I should get comparable performance
to a Stackless Python equivalent? I guess I can test this.

>Yes, as an application on top, but that is unrelated.

CT> For this soft switching to work, the underlying system must
CT> be able to do stack unwinding. When PyPy is translated to C,
CT> this is implemented by all the transformations of flow graphs
CT> into C routines.

So greenlets can be taken out of the stackless.py module if one
is compiling to pypy-c? Again, pardon my lack of knowledge. In this
context, what are greenlets providing? Does this imply that stackless.py
needs a minor rewriting? 

CT> What you are referring to, and that is a unification that I
CT> cannot leave leave as is, is the case when PyPy runs on top of a
CT> Python interpreter. The Python interpreter must then emulate soft
CT> switching, for testing purposes. Exactly for that case
CT> using the greenlets for emulation came in handy, just as a
CT> surrogate.
 
CT> But this use case of Greenlets has nothing at all to do
CT> with PyPy and does not belong to the PyPy world. It is an CT>implementation detail for a feature that seemed impossible to 
CT>emulate for PyPy on top of Python, but it works great.

I understand. However this capacity, this setup (translate.py or CPython
with greenlets and stackless.py) works great for prototyping
features. I think  prototyping features directly in Stackless
Python (and 'C') would be a rough ride. 

CT> These advantages will now end up in Stackless, something
CT> that I should have done ten years earlier.
CT> Worst thing that can happen is the same solution as with
CT> Greenlets. With a bit of luck, most hard switching can be turned into
CT> soft, and that will make a difference, concerning performance and
CT> pickling.

CT> sorry about my tone -- sincerely - chris

Actually Christian, thank you for your detailed explanation. 

Cheers,
Andrew