[Stackless] Connecting To A Database

Christopher Armstrong radix at twistedmatrix.com
Mon Jun 11 17:58:43 CEST 2007

On 6/11/07, Jeff Senn <senn at maya.com> wrote:

> Twisted helps in that it makes many higher level protocols act
> similarly (in terms
> of asynchrony) - so "use Twisted and stackless" is good advice if you
> are trying
> to overlap I/O & CPU in a situation where there is potential overlap
> (and all of
> your asynchrony is available in the Twisted impl).  You have
> to be willing to write your code a certain way... but one might argue
> that Stackless
> augments the Twisted core in some way.  If I understand correctly
> this still
> requires one extra thread - (but I suppose that is not theoretically
> necessary?)

It's not theoretically necessary and it's not practically necessary.
I'm pretty sure the only thing that caused that idea in the Stackless
community was that Andrew Francis was confused for a while, but now I
believe he is not using threads in his integration. I have integrated
stackless with Twisted nicely without threads in the past.

And it's quite possible to write your stackless code in the totally
natural way while integrating with Twisted; you don't need to change
the way you write your code (perhaps some more utility wrappers will
be necessary to e.g. write Twisted protocol implementations in an
implicit cooperative-threading style, though).

> In the database situation, I suppose the thing to think about is
> whether there
> is any overlap to be exploited -- and, more importantly, where it
> is.  (e.g. say
> in some join operation you are trying to intersect two sets...it
> doesn't matter
> much whether you interleave the fetching or not.  You still have to
> wait for the
> whole thing to complete before you can move on).

In many applications, making database access asynchronous will help a
lot: not necessarily to make concurrent database requests, but to do
database requests concurrently with other things that your application
does. Like, processing web requests, or allowing a game's AI to run. I
have written applications that don't bother to make database access
concurrent with anything else, but these are generally trivial little
things. Most of my applications, especially any server application,
will find much advantage to doing database access asynchronously.

> Maybe I just should have said: "Just because you make it asynchronous
> does not
> mean it will be faster". :-)

I agree. It's an argument I use against people who want to use
threading for everything. "Hey look, if I do these two web requests in
two different threads, my app is a billion times faster!". Then I go
on to tell them about how it's quite possible to do concurrent I/O
without threads.

However, I think you're understating the frequency of I/O-using
applications that can take advantage of this kind of concurrency.
Maybe you just write applications that don't often need it, or maybe I
write more applications that do. However, I generally find only the
trivial things don't need it.

Christopher Armstrong
International Man of Twistery

Stackless mailing list
Stackless at stackless.com

More information about the Stackless mailing list