[Stackless] integrate stackless with other event driven framework: merge two main loop

[Richard Tew]

On 10/14/06, Jimsfba at aol.com <Jimsfba at aol.com> wrote:
> I have a C++ event driven main loop framework that embedded the stackless
> framework.
> The only way I can make these two framework work together is: switching
> between two frameworks in a fix time range.
>
> 1. timeout the select() call in my C++ main loop every few hundreds
> milliseconds,
> 2. give a few hundress millisecond for stackless code to run:
>    watchdogHandle =
> handle<>(allow_null(PyStackless_RunWatchdog(1000)));

This runs for around 1000 instructions.  Which means that it will not exit
the currently running tasklet where you explicitly cooperatively schedule
or yield using some other manner, but in a preemptive manner.  As long as
you know that.

At CCP Games we take a different approach, which you can read about in
the slideshow Kristján Jónsson gave at PyCon 2006. Instead of interrupting
the permanently scheduled tasklets, we use custom yielding methods (BeNice,
Sleep or on a channel waiting for an event) and never schedule to divert
all the scheduled tasklets so that it is natural for all tasklets that have
been scheduled to run once and then the scheduler will be empty.  At which
point the watchdog exits cleanly, cooperative scheduling is maintained and
the embedding C++ code can do whatever it needs.  From what Kristján tells
me (I don't work on this section of code myself) it is common that the
scheduler is empty so we use a very large timeout for the watchdog in order
to detect badly programmed tasklets which are infinite looping or something
similar.

Links of interest:

The slideshow:
http://www.stackless.com/Members/rmtew/code/PyCon2006-StacklessEvePresentation.zip/download

Basic C++ framework with alternate yielding method BeNice:
http://svn.python.org/view/stackless/sandbox/examples/embedding/watchdog-cpp/

Basic C framework with alternate yielding method BeNice:
http://svn.python.org/view/stackless/sandbox/examples/embedding/watchdog-c/

> 3. go back to the C++ main loop and block on select() again
>
> This way works but not so elegant. It might be idle and waste a few hundred
> millisecond per loop in one framework while the other framework has work
> waiting for it to do.
>
> I am thinking a "perfect" model will be ...
>
> If stackless internally is also blocking on a select( fdset2 ) call, we can
> add a stackless API to export that file descriptor set (fdset2).
> Then we can merge two select() calls of two main loops into a single select(
> fdset1+ fdset2 ) call. We can then listen to all events in one blocking
> select call.

Stackless does not do any calls to select.  All it does is add tasklets,
channels and scheduling to the Python runtime.  It has no need to wait
on any file descriptors itself.

Hope this helps,
Richard.

_______________________________________________
Stackless mailing list
Stackless at stackless.com
http://www.stackless.com/mailman/listinfo/stackless