[Stackless] Stackless API

Tom Locke tom at livelogix.com
Thu Jan 29 08:06:15 CET 2004


[this was going to be "RE: [Stackless] Re: return values from tasklets",
 but it's turned into more than that]

Thought I'd jump in with some thoughts.

The idea of a return value from a tasklet seems like a nonsense to me,
as there is no caller waiting for said value. If you want this kind of
behavior, pass the tasklet a channel, and have the last statement the
tasklet executes be:

  resultChan.send(result)

Then the part of the program that needs this value can just receive on
this channel.

I think maybe there is some confusion about what the various methods do,
that could (maybe??) be cleared up quite easily with some renaming

> > * It seems wrong to me that tasklets call "schedule".  Scheduling
> >   should be up to the scheduler.  My suggestion is that tasklets
call
> >   "yield_control" which yields control back to whoever called
> >   tasklet.run.

When I call schedule(), I think of it as calling yield() (shame that
keyword is taken - I guess yield_control is better). But which process
I'm yielding *to* is not my concern - that's up to the scheduler. This
is VERY IMPORTANT! The whole point of using a good concurrency model
such as CSP, is that it allows you to understand the semantics of your
program, without having to think about the scheduler. There might even
be real concurrency on a multi-processor machine. If programmers start
thinking about the order in which tasklets are scheduled, they are
thinking there way into a black hole from which they will never return!

I think renaming schedule() to something similar to yield() would be a
good idea.

> Calling of schedule() is at the moment a thing that any code may
> do at any time. There isn't any stack-like structure imposed
> right now.

And nor should there be!
 
> The notation of tasklet.run() is arbitrary.
> Currently, it only says that the spelled tasklet is put
> into a position which lets it be run immediately, and
> after it is done, *my* current tasklet is the next one in the
> chain. But there is no guarantee that this will happen in this
> order. That's the reason why I was asking.
> Aaron Watters also asked me to make run() more restricted, like
> guaranteeing that it will get back to "me".
> This is possible, but it would enforce some structure like a "caller"
> of tasklets. I need more input like this, to go that way.

Again I think this is a very bad idea.

It's important to make a clear distinction between the 'advertised'
semantics of run() - i.e. its contract, and the details of what
currently happens in the implementation. 

I feel we need something more like Thread.start() in Java. A method that
simply means 'make this tasklet runnable', and if we're doing
cooperative scheduling, perhaps it should explicitly be a scheduling
point.

Note that this says *nothing* about, "it's going to run right now, and
then the tasklet that called run() will get the next turn..." etc. If
your code needs such guarantees, you are in big trouble.

I think tasklet.start() is the way to go, and drop tasklet.run()

> > * If the tasklet has a single frame and it performs a python
"return",
> >   the caller of tasklet.run gets the return value.

The caller or run() is somewhere else by now, because run() does not
block. That's the whole point!

> We might have to re-consider run(), if it should carry semantics
> like being a real caller.
> Again, I need this discussion on the mailing list!

A big 'no' vote from me :) actually I can't even see how this would make
sense.

OK... what about various ideas about custom scheduling? Here's my
proposal. (I'm thinking as I go here).

A module property stackless.scheduler, to which one can assign an
object.

This object should provide a certain interface. Here, to describe the
interface, is a simple round-robin implementation:

------
class RoundRobin(object):

    def __init__(self):
        self.runQ = [] # maintain a queue of processes to run

    # Called when a tasklet is created (needed?)
    def init(self, tasklet):
        pass # round-robin doesn't have anything to do here

    # Called when a tasklet tries to communicate on a
    # channel, and is blocked
    # sending=True: tasklet wants to send
    # sending=False: tasklet wants to recv
    def block(self, tasklet, channel, sending):
        # remove the tasklet from the run queue
        self.runQ.remove(tasklet)
        if len(self.runQ) == 0:
            raise ScheduleError("Progam deadlocked")

    # Called when the tasklet is unblocked - i.e. the communication is
    # ready to take place
    def unblock(self, tasklet, channel, sending):
        # Re-join the run queue at the back
        self.runQ.append(tasklet)

    # Called at every scheduling point (i.e. channel send/recv,
    # stackless.schedule() )
    # The returned tasklet is the next to run
    # It is an error to return a tasklet that is blocked on a channel
    # (or maybe the kernel just calls schedule again immediately?)
    def schedule(self):
        # next tasklet is at the front of the queue
        next = self.runQ[0]

        # move it to the back of the queue before returning
        del self.runQ[0]
        self.runQ.append(next)
        
        return next
------
You can implement almost any strategy with this approach. E.g. you could
do Bob's "sub-watchdogs".

Note my round-robin is not efficient, with all its list mangling. A
better approach would add a 'next' field to the tasklets, and maintain a
linked list.

The scheduler API could be extended to support non-busy timeouts. We
could add a method 'blockUntil(tasklet, wakeTime)'

Some other thoughts:

I think we need a some kind of syncing primitive that is lower-level
than a channel. Maybe a good old semaphore?

Once this, and the scheduler are in place, I feel methods like capture()
and become() should be removed. These methods scare me :) They look like
they exist to expose cool things you can do as a result of the current
implementation strategy, but are semantically hair-raising.

Tom.


_______________________________________________
Stackless mailing list
Stackless at stackless.com
http://www.stackless.com/mailman/listinfo/stackless



More information about the Stackless mailing list