[Stackless] stacklesslib & tealet

Kristján Valur Jónsson kristjan at ccpgames.com
Thu May 2 11:05:40 CEST 2013


Hi.
The tealet thing:
This has been a hobby project for me.  I am fascinated by stack slicing and always thought it could be done better.
Armin's improvement to the original one is great (just in time saving of the stack, meaning no initial "starting point").
Tealet is already independent of Python, so that it can be used in other projects, but I wanted to add a few features so that it were more flexible:
a) Removed the dependency on malloc(), it now uses a user-supplied allocator
b) Add error handling in case of malloc() failure
c) Add tealet "duplication".  This allows us to create an initial stub for spawning new tealets, rather than having to always spawn from the current position.  This allows us to do tricks such as stack spilling, something stackless python does.  It also allows tealet recursion without exhausting the stac. (i.e. a loop where each new tealet is created from the previous one)

So anyway,
I don't have any specific plans other than:
1) Putting it out there for your reviewing pleasure.  
2) Writing Greenlet emulation module for fun. (already done)
3) Writing Stackless emulation module for fun (not done)
4) Thinking more about the idea to use this as the switching engine in stackless python.

4) would be a large project and I wouldn't want to embark upon it unless people had actually toyed with the tealet module itself.  It would have some benefits, though.  Architecturally, it would be wise to leave stack switching to a special library.  This would also simplify a lot of stackless code.  I think it would help us restructure it a bit.  In addition, the tealet/greenlet stack saving approach would save us from some of the initial-stub machinery, in particular, the very dodgy code that "climbs stacks" just to enter the right zone.

So, that's the plan.
I should also probably get in touch with Armin Rigo and ask him about licensing :)  

K

> -----Original Message-----
> From: Christian Tismer [mailto:tismer at stackless.com]
> Sent: 30. apríl 2013 14:47
> To: The Stackless Python Mailing List
> Cc: Kristján Valur Jónsson
> Subject: Re: [Stackless] stacklesslib & tealet
> 
> Hi Kristjan,
> 
> what is your current plan for the tealet?
> 
> To all:
> 
> I think we should try to discuss a road map for Stackless, where the journey
> should go in the future.
> 
> What is the best way to set up a discussion? Does the stackless list suffice for
> that, or is it better to use some Google groups stuff?
> 
> cheers - chris
> 
> 
> On 4/18/13 8:00 PM, Kristján Valur Jónsson wrote:
> > Ok, fixed the issue.  It was a gcc compiler problem.  The optimizer would
> assume that a memory value wouldn't change across the switch call.
> > Fixed with a  "volatile" attribute.
> > I also recreated the repo, this time with the latest stuff and also the
> pytealet project, which brings python into play.
> >
> > K
> >
> > -----Original Message-----
> > From: stackless-bounces at stackless.com
> > [mailto:stackless-bounces at stackless.com] On Behalf Of Kristján Valur
> > Jónsson
> > Sent: 18. apríl 2013 15:38
> > To: The Stackless Python Mailing List
> > Subject: Re: [Stackless] stacklesslib & tealet
> >
> > There is some weirdness going on.  I'm drilling down with print statements,
> but the problem is that those do affect optimization.
> > Also, I realized that I uploaded an old version of the lib to the
> > repo.  I will put a newer version in place, one that actually includes
> > the pytealet project :) K
> >
> >> -----Original Message-----
> >> From: stackless-bounces at stackless.com [mailto:stackless-
> >> bounces at stackless.com] On Behalf Of Kristján Valur Jónsson
> >> Sent: 18. apríl 2013 11:27
> >> To: The Stackless Python Mailing List
> >> Subject: Re: [Stackless] stacklesslib & tealet
> >>
> >> installed mingw (no simple feat) and reproed your problem.
> >> However, it does not seem to include any debug symbols.  I didn´t
> >> find any way to make sure those were gerenrated.  So debugging with
> >> visual studio is off.
> >> I'll try some print statements.
> >> How do you make sure your .exe is linked with debug symbols?
> >> K
> >>
> >>> -----Original Message-----
> >>> From: stackless-bounces at stackless.com [mailto:stackless-
> >>> bounces at stackless.com] On Behalf Of Kristján Valur Jónsson
> >>> Sent: 18. apríl 2013 09:34
> >>> To: The Stackless Python Mailing List
> >>> Subject: Re: [Stackless] stacklesslib & tealet
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>>
> >>>> What is the license of tealets?  Has it been derived from any parts
> >>>> of Stckless, which might force an derivative usage Python license on it?
> >>>>
> >>> I don't understand, is this some lawyer talk?  My education is in
> >>> the natural sciences.
> >>> the 'platf_slp' folder is copied verbatim from stackless.  I added
> >>> that as a fallback since stackless supports more platforms than the
> >>> original tealet version.
> >>> However, a better way would be to take some of the greenlet stuff
> >>> since that is receiving more active support these days.
> >>>
> >>>> Do you have any future plans for it?
> >>> One of the plans was to use this as the hard-switching engine in
> >>> Stackless Python.
> >>> It would provide a clear api and separation of responsibility, and
> >>> possibly simplify Stackless too to boot.
> >>>
> >>>> It compiles okay on mingw / Windows 7 (64 bit), but tests do not do
> >>>> so
> >> well:
> >>>> $ make tests
> >>>> gcc -g -o run_tests_static_g tealet.c tests.c run_tests_static_g
> >>>> +++ Running tests with newmode = 0 Running test 0... +++ Running
> >>>> +++ test 1... +++ Running test 2... +++ Running test 3... +++
> >>>> +++ Running test 4... +++ Running test 5... +++ Running test 6...
> >>>> +++ +++ All ok. +++ Running tests with newmode = -1 Running test
> >>>> +++ 0... +++ Running test 1... +++ Running test 2... +++ Running
> >>>> +++ test 3... +++ Running test 4... +++ Running test 5... +++
> >>>> +++ Running test 6... +++ All ok. +++
> >>>> gcc -g -O2 -o run_tests_static_o tealet.c tests.c
> >>>> run_tests_static_o
> >>>> +++ Running tests with newmode = 0 Running test 0... +++ Running
> >>>> +++ test 1... +++
> >>>> Assertion failed: g_main->g_current == (tealet_sub_t *)g_main, file
> >>>> tealet.c, li ne 625
> >>>>
> >>>> Thoughts? :-)
> >>> Well, I confess to developing this with windows and visual c only.
> >>> Optimizations, always pesky.  You can see code such as this:
> >>>
> >>> /* set up the following field with an indirection, which is needed
> >>>       to prevent any inlining */
> >>>      _tealet_initialstub = tealet_initialstub;
> >>>      _tealet_switchstack = tealet_switchstack;
> >>>
> >>> Perhaps this is failing?  Inlining and modification of functions
> >>> will do damage.....
> >>>
> >>> Test 1 is simple:
> >>>
> >>> tealet_t *test_simple_run(tealet_t *t1, void *arg) {
> >>>    assert(t1 != g_main);
> >>>    status = 1;
> >>>    return g_main;
> >>> }
> >>>
> >>> void test_simple(void)
> >>> {
> >>>    init_test();
> >>>    tealet_new(g_main, test_simple_run, NULL);
> >>>    assert(status == 1);
> >>>    fini_test();
> >>> }
> >>>
> >>> The assertion that is failing (why are the assertions being
> >>> triggered in a O2 build, btw?) is this:
> >>>
> >>> void tealet_finalize(tealet_t *tealet) {
> >>>      tealet_main_t *g_main = TEALET_GET_MAIN(tealet);
> >>>      assert(TEALET_IS_MAIN_STACK(g_main));
> >>>      assert(g_main->g_current == (tealet_sub_t *)g_main);
> >>>      tealet_free(g_main, g_main);
> >>> }
> >>>
> >>> It is asserting that the current tealet is the main tealet.  If it
> >>> isn't, then something failed in switching back.
> >>> Since this is the simplest of all tests, newmode is 0, it should be
> >>> very easy to analyse.
> >>>
> >>> I'll see if I can give this mingw of which you speak  a spin.
> >>>
> >>> K
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Stackless mailing list
> >>> Stackless at stackless.com
> >>> http://www.stackless.com/mailman/listinfo/stackless
> >>
> >>
> >> _______________________________________________
> >> Stackless mailing list
> >> Stackless at stackless.com
> >> http://www.stackless.com/mailman/listinfo/stackless
> >
> >
> > _______________________________________________
> > Stackless mailing list
> > Stackless at stackless.com
> > http://www.stackless.com/mailman/listinfo/stackless
> >
> >
> >
> > _______________________________________________
> > Stackless mailing list
> > Stackless at stackless.com
> > http://www.stackless.com/mailman/listinfo/stackless
> 
> 
> --
> Christian Tismer             :^)   <mailto:tismer at stackless.com>
> Software Consulting          :     Have a break! Take a ride on Python's
> Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
> 14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
> PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
>        whom do you want to sponsor today?   http://www.stackless.com/
> 





More information about the Stackless mailing list