[Stackless] Umbrella Project? (was: pip install stackless-python

Christian Tismer tismer at stackless.com
Thu Jul 5 16:36:19 CEST 2012

Hi all, and those explicitly addressed,

first of all, I'm happy that my talk worked quite ok, and that Guido
reacted very positively on the idea of making it pip-installable,
including virtualenv.

Of course we are cheating quite a bit, because stackless is not
really a package, but suffers that little wart of "being an executable 
after all".

And today, I heard the talk of Mark Shannon which was just great.
I'm sitting in the shadow at the conference, let the impressions
flow, and try to get something sorted out.

On Stackless PyPy:

Yes, special care must be taken to at least pretend that we have simple
packages, although we don't.

For instance, as you said, having PyPy installed and then with a configured
dependency of stackless-python, something like this will probable happen
right now (but I'm not sure how much checking stackless does right now?)

I guess, pypy running "pip install stackless-python" should right now reject
installation, since there is no stackless binary built for PyPy.

I would imagine that, if done right, pypy running "pip install 
should figure out that on PyPy, this only means to at most ensure that 
the PyPy
installation is able to "import stackless". If that works, then PyPy was 
built with
stackless support. Right now this is in some broken/unready state, but 
will be
working, pretty soon.

On HotPy and Stackless

As far as I understood Mark Shannon today, in some private chat during 
lunch, HotPy
does some changes to the obvious modules, like ceval.c (heavily), 
and probably other kernel-related stuff (traceback, pystate etc.).
I will need a closer look how these changes are made, and how feasible it is
to add a stackless version. I would like to implant the idea right now 
and early
enough. Right now I'm even wondering if HotPy is maybe better suited to
implement stackless features than CPython itself, because HotPy does in the
first place, what Stackless achieves painfully via crazy tricks: Make 
CPython tail-recursive.
So I have the gut feeling that Stackless-Hotpy could become a new and 
possibly simpler
branch, because all the hairy changes that are needed are already in HotPy.

But this still is Vodoo.

In any case, I would like to encourage everybody to look into HotPy and what
we can do for it to thrive! It is a really nice improvement to CPython 
and can become
a valid alternative for  people who cannot use PyPy right now.
HotPy needs support, committers, repository, community, all of that. And 
that Guido is interested in it.
I'm interested to contribute. Of course, I want it to have a stackless 
option in every
Python flavor.

General thoughts on PIP and the future:

I think the PIP approach to simplify stackless is very valid, although 
it has
its subleties, like "not scaling well". This can be improved and better 
later. I think it is important, and to do it right now, to state a fact. 
We then get into
a better position, not asking for something to be included somewhere, but to
show that we have a pip installer, which puts us into a better position.

This kind of installation does not scale well, because if somebody does a
"pip install HotPy" after "pip install stackless-python", (s)he will 
loose stackless.
A better installer would need to know about these combinations and to draw
the appropriate decision when detecting an existing stackless. It would need
to download/unpack/build a stackless-enabled HotPy version (and the 
right one),
instead of failing.
I think that could also have some impact on how easy_install and distribute
are developed further, since today this would not work at all.

In any case, there _are_ certain projects that need a common way to 
and give the user a hassle-free experience, as if they just were simple
packages. We need to support the useful and valid combination of that and
pretend composability to some extent.

I think by doing so, we can avoid what happened to Stackless for so many 
A niche-existence, where people were reluctant to try it.

So in some sense, my idea is to start some "umbrella" project, that 
gives smaller
projects that need to change the interpreter binary a better position. 
There is
no longer every other single project in some concurrence with CPython.
I guess nobody ever wanted that. By doing a combined effort, we instead
would gain a positive, collaborative attitude:
We all love CPython, and you get your choices to select what you are
specially interested in. That is where I want to go.

Right now the choices seem to be
PyPy   (maybe, to some extent)

but I'm open do discuss what else could go into this list.



Should we come up with an umbrella project on BitBucket, for instance

    "alternative-python"  or  "special-python"

which then has

   - an installer project
   - HotPy
   - stackless
   - pypy (concerning setup issues)
   - (more proposals?)

as repositories?

Then this project would have its own mailing list and issue tracker, and 
a couple
of submitters.

I would certainly like to be in this, as I'm interested in every aspect 
of Python
and how to support improvements in any direction.

Q: would Jython fit into this picture as well?

Please let me know your thoughts.
I'm sitting at EuroPython, Florence including Friday and maybe Sat. evening.
Contact me by email or find me at the conference.
Let's talk!

Cheers -- Chris

On 7/5/12 12:29 AM, Hervé Coatanhay wrote:
> Hi all,
> I was wondering. Now that `stackless-python` is like any other pypi 
> package on a pip perspective, what will/should happen if I want to use 
> a package that has `stackless-python` as a dependency in a pypy 
> virtualenv ?
> cheers,
> Hervé
> On Thu, Jun 28, 2012 at 8:42 PM, Christian Tismer 
> <tismer at stackless.com <mailto:tismer at stackless.com>> wrote:
>     Hi Stefan,
>     On 6/28/12 5:37 PM, Stefan Drees wrote:
>         Hi Chris,
>         Am 28.06.12 16:51, Christian Tismer wrote (slightly edited):
>             ...
>             Stackless becomes a PyPI package that contains only the
>             binary and the
>             few changed library modules.
>             After
>                      pip install stackless-python
>             you have stackless inside site-packages, and you get a
>             message that
>             tells you
>                      "Along-side with pythonX.Y, you now have a command
>                       slpythonX.Y which you can run as an alternative"
>             and you can also
>                      pip uninstall stackless-python
>         > ...
>         to me this sounds good enough for easing the people into
>         wanting to try stackless out.
>         Maybe a different name for the modified python eg. matching
>         the package name from the install command stackless-python,
>         stackless-python2.7 etc?
>     Possible. The name "stackless-python" was proposed in a parallel
>     discussion on the virtualenv list. Maybe it is better to use a shorter
>     name since I assume some laziness if people need this package
>     at all ;-)
>     But there are many packages with a name different from what gets
>     installed - for instance distribute, which is cheating, it just
>     replaces
>     the easy-install package with a different version.
>         I share the view, that a virtualenv is and should allways be
>         an additional option, transparent for the package.
>     Yes. I thought things would be easier abusing virtualenv, but that
>     seems to be not true.
>     So I ended up with a "piggy-back stackless".
>     Btw., I am still playing with wild ideas, like on "import stackless"
>     replacing and restarting the interpreter completely...or some
>     other tricks
>     that emulate an extension.
>     But it is not trivial without a mechanism to intercept the normal
>     startup-code of cpython.
>     Mayme I'll keep it simple and get somewhere at all ;-)
>     cheers -- chris
>     -- 
>     Christian Tismer             :^)   <mailto:tismer at stackless.com
>     <mailto:tismer at stackless.com>>
>     tismerysoft GmbH             :     Have a break! Take a ride on
>     Python's
>     Karl-Liebknecht-Str. 121     :    *Starship*
>     http://starship.python.net/
>     14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
>     work +49 173 24 18 776 <tel:%2B49%20173%2024%2018%20776>  mobile
>     +49 173 24 18 776 <tel:%2B49%20173%2024%2018%20776>  fax n.a.
>     PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3
>     BF04
>           whom do you want to sponsor today? http://www.stackless.com/
>     _______________________________________________
>     Stackless mailing list
>     Stackless at stackless.com <mailto:Stackless at stackless.com>
>     http://www.stackless.com/mailman/listinfo/stackless
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless

Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
work +49 173 24 18 776  mobile +49 173 24 18 776  fax n.a.
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20120705/046b8090/attachment.html>

More information about the Stackless mailing list