[Stackless] stacklesslib.async

Kristján Valur Jónsson kristjan at ccpgames.com
Mon Sep 2 12:23:51 CEST 2013


Hi there.
I just checked in an addition to the stacklesslib, the async module.
See https://bitbucket.org/krisvale/stacklesslib
This was inspired by the C#async feature, http://msdn.microsoft.com/en-us/library/vstudio/hh191443.aspx

The module provides two things:

1)      A Task class, which represents an execution unit, and functions to create tasks and decorators to help create "task" functions.

2)      "Await" support, through a decorator that automatically passes in an AwaitManager object as the first argument to a function.


It is 2) that I was really gunning for, because it is a cool feature in C# (and VBScript).
Basically, in C# you can create a function roughly like this:


int async fooAsync(string bar) {
    Task  task = loadFormWebAsync(bar);
    do_other_stuff();
    data = await task;
    return do_stuff_with_data();


and call it using:

void caller() {
    Task t = fooAsync("stuff");
    do_other_stuff();
    int result = t.get_result();
}

The magic with the await keyword (only in an async method) is this:

1)      When the async method is invoked, (as in the first line in "caller" above) execution immediately continues in the async method in the same thread.  It is just like a function call, until the async  method calls "await" for a Task object.

2)      at this point, execution resumes in the caller, and the rest of the async method will execute as a continuation when task it is waiting on completes.

What this means is that execution is depth-first with no context switches until they are actually required.  the C# compiler will generate helper code to to this and post continuation callbacks to the Task objects and so on.

In stacklesslib, I'm experimenting with doing similar things with explicit task scheduling

from stacklesslib import async
@async.async
def fooAsync(awaiter, arg1, arg2):
    t1  = getSomeOtherTask(arg1)
    t2  = getSomeOtherTask(arg2)
    do_some_other_work()
    return awaiter.await(t1) + awaiter(t2)

def caller():
    t = fooAsync("foo", "bar")
    # fooAsync has now run up to its first "await"
    r = t.get_result()
    # fooAsync has not completed.


Stackless has no continuations anymore, so fooAsync is actually run entirely as a new tasklet, but we use explicit switching to run it, and to jump back to the caller.  Stackless actually lacks a tasklet.switch() argument, or perhaps, tasklet.run(remove=True) which runs a target tasklet, and removes the previous one from the scheduler in an atomic operation.  I may add that later, since it is a useful building block.

The task based structure is also cool, and there are primitives to wait for all tasks, or any task, and to create Task on worker tasklets or worker thread.

@async.task(async.threadTaskFactory)
def httpJob(url):
    return urllib.download_url(url)

@async.task(async.taskletTaskFactory):
def executeWithDelay(delay, callable):
    stacklesslib.main.sleep(delay)
    return callable()

def caller():
    t1 = httpJob("myurl")
    t2 = executeWithDelay(0.1, (lambda:print"hi"))
    #wait for all of them:
    async.Task.waitAll([t1, t2])
    assert t1.ready and t2.ready
    # or any:
   which  =    async.Task.waitAny([t1, t2])
    assert [t1,t2][which].ready

    #or create helper jobs that do that:
    t3 = async.Task.whenAll([t1, t2])
    t4 = async.Task.whenAny([t1, t2])




Thoughts:

1)      is the "await" behavioru useful, i.e. depth first execution until blocking?  It makes sense in c# because it does not involve threads.

2)      What about the names of classes, modules, functions?

3)      whenAny, whenAll, waitAny, waitAll are currently Task static methods.  This is copied from the C# "Task" class, but it may be un-pythonic.  Perhaps just global functions in the module?

Anyway, see if you think this is interesting.

Cheers,
K

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20130902/d96a73c6/attachment-0001.html>


More information about the Stackless mailing list