[Stackless] Stackless API

Kristján V. Jónsson kristjan at ccpgames.com
Fri Jan 30 10:01:49 CET 2004

First off, let me repeat that I don't see what Arman is trying to achieve which cannot be done with a function call.  To launch a tasklet, and then
to wait for it to finish, without doing anything in between, is just a regular function call.

Second, just to avoid confusion, when talking about pthreads, I was
discussing pthread_join().  Just to branch off and give a little perspective (this is a very mature threading model, sort of the result of years of threading experiments on various unix platforms):  pthreads are created either "detached" or not.  If not, they can be "joined". Joining has two purposes
a) making sure the thread has finished
b) receiving a Word of information as the thread's result value.

Now, both things can be achieved using "condition variables" and "mutexes" but the wait mechanism is a simple shortcut for the above purposes and in the non-refcounted world of C simplifies things a bit.

A "job scheduler" that I once wrote in used this concept.  A single job would schedule dependency jobs to run, then use pthread_join() to make sure they had finished running.  I would then pick up their results from a previously agreed place.

Back to Python.  With it's ubiquitous return values and reference counting, a lot of stuff is simpler.  The distinction detached or not goes away, that's handled via reference counting of the tasklet object.  But sometimes you want side-effects of a tasklet run, and you don't necessarily want to create a Channel for it to tell you when it's done.  Or you want a simple function type return value.  As Christian said, there is already a place in the tasklet where the starting frame's result is stored.  The only thing needed to give join() functionality is that the tasklet that calls "join" blocks until this result is written to.

notifier = tasklet.new(notify)
#the notify tasklet will go and notify someone about what we are about to do and get permission
#okay, notification is complete, continu

another example.  Here's a simple job class
class job:
	def set_prereqs(self, list):
		self.prereqs = list

	def go(self): #we are ready, run our dostuff routine in a tasklet
		self.tasklet = tasklet.new(dostuff)

	def join(self): #return out result.
		return self.tasklet.join()

	#optional.  make sure the prereqs are run before joining this job, if one likes.
	def run_prereqs(self):
		for p in self.prereqs:
			p.join()	#they have now run!

	#basic dostuff() routine.  just adds up prerequisite result, can be overridden in inherited classes
	def dostuff(self):
		return [prereq.join() for prereq in self.prereqs]

You then set up a network of jobs (subclassed according to what they should do)
a = job()
b = job()
c = job()
print c.join()

This is quite similar to the way one would code this in pthreads.  Note the tasklet communication using join(), and no channel in sight.  Note also that in python, unlike phtreads, we can join a tasklet multiple times since the result value is referenced by the (finished) tasklet object.  For the stackless implementation, this would mean simply that tasklet.join() would cause the caller to block until "tasklet" has finished runnin, at which point it simply returns a new reference to its result value.


p.s. the type of job classes mentioned above were written to utilize CPUS on multiprocessor machines for numberchrunching.  Obviously stackless isn't mp (yet?) so the point is somewhat moot.  For CPU bound stuff, tasklets aren't needed, really.

-----Original Message-----
From: stackless-bounces at stackless.com [mailto:stackless-bounces at stackless.com] On Behalf Of Christian Tismer
Sent: 29. janúar 2004 19:43
To: Arman Bostani
Cc: stackless at stackless.com
Subject: Re: [Stackless] Stackless API

> Assuming we go with the model I'm suggesting, the caller of
> tasklet.run() is "waiting" for a return value.  If its a scheduler
> calling tasklet.run, then it may have a policy to collect the return
> values from the tasklets it runs.

Right now, I'm a bit tending to be in favor of
the channel idea. Channels are cheap and clean.
But please don't stop, keep the discussion rolling.
My POV is of minor importance, since I have been
fiddling with this stuff for too long now.
My sight is distorted.

[still digesting the meat of this interesting discussion]

Stackless mailing list
Stackless at stackless.com

More information about the Stackless mailing list