[Stackless] StacklessWSGI example, HTTP daemons in general

Arnar Birgisson arnarbi at gmail.com
Sat May 17 15:33:07 CEST 2008


Hey Japhy,

On Sat, May 17, 2008 at 5:58 AM, Japhy Bartlett <japhy at nolimyn.com> wrote:
> Unfortunately, it seems to quietly die after a random (as far as I can
> tell) period of inactivity.  I added a few debug prints in the main
> loop, without any luck.  Anyone have any thoughts on what's triggering
> it?

I haven't looked at it for a while. There have been some changes to
stackless since last time I did, I know at least one person was having
trouble running it. Unfortunately I'm stretched for time these days,
but if you figure something out I'd be happy to commit it.

> More interestingly, the server seems to handle loads pretty well (not
> surprising, considering the nature of Stackless), and had me thinking
> about server models:
>
> ( Briefly, is there any info on what happens when Stackless starts
> hitting a max number of tasklets?  What determines that upper limit? )
>
> If you added a counter to the pool of client tasklets added, as you
> started to approach the upper limits of the individual process, you
> could spawn a new server process, (perhaps listening on 8001) and
> start forwarding clients to that process.  Which, in turn, could spawn
> a new process as it began to peak, ( a process per CPU on the host
> machine? ) perhaps on another server altogether.

I doubt the max number of stackless tasklets is a limiting factor
here. I'm not sure what it is (if it is limited by anything other than
memory) but I've heard of people running up to hundreds of thousands
of tasklets in production. Serving that many simultaneous http clients
would put a strain on any machine.

As for load balancing between processes and even machines, you might
have better luck with tools specifically built for that, such as
perlbal [1].

I have a feeling, and have been wanting to test this out, that a
solution for a high-performance Stackless HTTP server might be built
on the asynchronous HTTP server code found in recent versions of
libevent [2]. Phoenix Sol has been doing some experiments with this.

[1] http://www.danga.com/perlbal/
[2] http://www.monkey.org/~provos/libevent/

cheers,
Arnar




More information about the Stackless mailing list