[Stackless] An idea for making Stackless more naturally usable

[Richard Tew]

On 9/9/06, Andrew Dalke <dalke at dalkescientific.com> wrote:
> I'm confused.  slpmonkeypatch/socketmodule.py imports stdsocket
> but I don't see where that is created.  I did a svn update on
> the main stackless code base but didn't see it there.

In "__init__.py" for now.

import slpmonkeypatch
slpmonkeypatch.monkeypatch()

Was what I was using to test it, where 'monkeypatch' is defined
in the aforementioned "__init__.py" file.

> I propose a "scheduler" module which manages three things:
>    1. the asyncore loop
>    2. 'sleep' requests
>    3. emulation polling checks
> and possibly
>    4. other "alive" requests
>
> 1. is obvious.  You have it already in stacklesssocket.py
>
> 2. needs to be merged with #1.  I don't want 100% busy wait so
> if there are only sleeping objects and socket listens then
> select should use the timeout until the next object wakes.
>
> 3. there are two ways to handle blocking request.  getaddrinfo
> must be done in a single posix thread or as a spawned off program.
> There is no asynchronous API.
>
> OTOH, waitpid can be emulated.  If it is not called WNOHANG then
> it is supposed to block.  It could be in a thread, or the
> emulation layer can do a occasional polling.
>
> def emulated_waitpid(pid, options):
>    if options & WNOHANG:
>      return real_waitpit(pid, options)
>    while 1:
>       new_pid, status = real_waitpid(pid, options | WNOHANG)
>       if new_pid != 0:
>           return (new_pid, status)
>       stackless_sleep(EMULATION_TIMEOUT)
>
> however, it might be better to have a special "emulation_timeout"
> sleep queue in addition to the normal sleep queue.  If there's
> anything in that queue then every ... 1/100th second? wake up
> those tasklets.  That's to prevent time.time() calls for every
> emulation.
>
> 4. is for things like requests sent to other posix threads.
> I think the scheduler should exit it main loop when there are
> no sockets, no sleeping requests, and no emulation requests.
> But the scheduler doesn't know about everything.  For example,
> a tasklet could be waiting for a response from a posix thread,
> which the scheduler doesn't know about.
>
> I'm calling those "active" objects.  Though it might be a simple
> counter.  Perhaps with a new with-style context manager.
>
> The scheduler doesn't exit if there are actives.
>
> It should have some way to register new non-asyncore sockets,
> eg, for GUI events not managed by stackless.  This is more
> common under X.
>
> > I think that a tasklet sleep function would also be good to
> > monkeypatch in as part of the library, perhaps to
> > stackless.sleep.
>
> It needs to work together with asyncore.  Otherwise the CPU
> is always at 100%.

I agree with all this.  But before getting it all into shape in the
way you describe, I would like to concentrate on getting what
will for me be the hardest part out of the way.  The Windows
socket and file replacements based on IO completion ports.

If you want to do any changes/cleanup or whatever, please
feel free to go for it in the meantime.  Otherwise, I will
have a go when the Windows work is done, or when I need
a break from it.

Using IO completion ports to manage asynchronous IO on
Windows became a lot harder when I realised that because
there was no 'seek' method, and offsets were passed into
the 'ReadFile' and 'WriteFile' API calls directly, I need to do
a lot more work than I expected to create a replacement
fileobject for Windows.

_______________________________________________
Stackless mailing list
Stackless at stackless.com
http://www.stackless.com/mailman/listinfo/stackless