[Stackless] Stackless based replacement

Arnar Birgisson arnarbi at gmail.com
Fri Oct 3 20:35:26 CEST 2008


Hi Larry,

On Fri, Oct 3, 2008 at 19:52, Larry Dickson <ldickson at cuttedge.com> wrote:
> Two questions:
>
> (1) Why does it wait for the first round the dispatcher tasklet runs? Why
> doesn't the callback just get chained off the interrupt that responds to the
> readiness of the IO? The main tasklet got PAST the call to
> event.notify_when_ready_for_op and is blocked on the receive, so is there
> another tasklet that just keeps trying for readiness every time the round
> robin comes back?

You cheated, that's three questions right there :D

Interrupts are just as hard to manage as preemptive scheduling. You
need all kinds of locking primitives etc. in your code if you rely on
interrupts, in which case you shouldn't be using libevent at all (it
serves an entirely different purpose). That is why libevent must rely
on a loop that is either continuously running (and you do everything
else in event handlers) or your application must take care of calling
the dispatcher regularly. In my code, there is a dedicated tasklet for
this, I guess that qualifies as the tasklet you refer to that checks
for readiness.

I don't really know how to explain this more clearly than in my last
message. I'll give it one more go, otherwise someone more elaborate
than me needs to step in.

Say tasklet A wants to read from a FD, and tasklet L is running the
libevent distpatcher regularly.

* Tasklet A asks libevent to call a specific function when the FD is ready.

* Tasklet A defines this function and has it perform the read and
write the output on a channel. The function is not executed by tasklet
A.

* Tasklet A does a .receive() on said channel, and thus blocks.

* Tasklet L now runs, but there are no IO events available, if there
are other tasklets it simply puts itself at the back of the queue.

* Other tasklets run, not A, because it is still blocked

* Tasklet L runs again, now there is an event ready with an attached callback

* Tasklet L (specifically the libeven dispatcher) calls the callback,
so it is really tasklet L that performs the actual read and sends the
result on the channel, marking tasklet A as runnable

* Tasklet A runs and picks the value off the channel, returning it to
the calling code.

I think you need to study conventional asynchronous I/O, such as
asyncore, Twisted, epoll & friends etc. Then my code should be pretty
easy to understand.

> (2) If libevent has nothing to do with Stackless, how can its callback
> perform a Stackless function (the channel send)?

I don't follow. The callback is just a regular python function and can
do whatever python functions can do, including sending stuff on
channels.

> A non-blocking select is just a blocking select that always happens to be
> ready (because of the 0 timeout). The blocking select does not have a
> timeout (unless that is explicitly coded), but it selects on ALL the
> currently outstanding IOs. Note that a select does not actually do the IO,
> but it returns when at least one of them is ready. In your case, the
> blocking select on both FD1 and FD2 would be won by FD1, and Tasklet A would
> wake up able to read the memory-mapped file immediately.

Ah, in that case you need an elaborate handler when the select comes
out of the block, to resume the correct tasklet depending on which
operation should be performed etc. This is one of the things libevent
handles completely transparently.

> No, I mean milliseconds. It would seem that either your time.sleep is using
> some other mechanism than a timer interrupt, or you have faster timer ticks
> (10 usec?) than my motherboard. You might want to try C/Linux nanosleep or
> usleep and see if you get the same result. And/or do a longer Stackless test
> and watch CPU usage, just in case Stackless is using a busy loop.

I millisecond is pretty long for a computer. Unfortunately I don't
have time to verify this, but if you do I'd be interested to see the
results. Here is the implementation of time.sleep for linux (in fact,
any platform that has select so this probably includes OS-X also):

	struct timeval t;
	double frac;
	frac = fmod(secs, 1.0);
	secs = floor(secs);
	t.tv_sec = (long)secs;
	t.tv_usec = (long)(frac*1000000.0);
	Py_BEGIN_ALLOW_THREADS
	if (select(0, (fd_set *)0, (fd_set *)0, (fd_set *)0, &t) != 0) {
#ifdef EINTR
		if (errno != EINTR) {
#else
		if (1) {
#endif
			Py_BLOCK_THREADS
			PyErr_SetFromErrno(PyExc_IOError);
			return -1;
		}
	}
	Py_END_ALLOW_THREADS

You can see other platforms in [1], the function is called "floatsleep".

[1] http://svn.python.org/view/python/trunk/Modules/timemodule.c?rev=64745&view=auto

cheers,
Arnar




More information about the Stackless mailing list