[Stackless] stacklessocket with threads, and stacklessfile
tim at metaweb.com
Thu Aug 9 01:26:45 CEST 2007
The problem with today's stacklesssocket is the I/O
pump tasklet that repeatedly calls select().
Remember that select() is itself a blocking system call,
so the I/O pump invokes it with a short timeout
so that other tasklets get serviced.
The application that Alec is working on has a mix of
computation tasklets and I/O tasklets and the current
stacklesssocket design causes the computation tasklets
to get stalled everytime select() gets called by the
Our proposal: run the select() loop in a separate
thread so that the select() doesn't block all other
Does this make sense?
P.S. Using a select() alternative doesn't help.
poll(), epoll(), kqueue(), etc, all have exactly
the same problem.
Christian Tismer wrote:
> On Aug 8, 2007, at 7:50 PM, Alec Flett wrote:
>> Hey folks -
>> So we're using stacklessocket with an app that is somewhat CPU
>> intense, but also does I/O. We're trying to use stackless socket, but
>> what we're actually finding is that overall performance is suffering
>> because of the somewhat long wait time in the asynccore runner
>> tasklet. ( 0.05) - if we block for more than 0.05 seconds, this ends
>> up wasting about 5% of of our CPU.
> Could you please supply exact reference to what implementation
> you are using? I'm not exactly up to date.
>> We can certainly play with that number to find out what's optimal for
>> our particular application, but I'm curious if anyone has thought
>> about taking the same approach that stacklessfile does of having a
>> pool of worker threads so that real hardware threads can go into a
>> more effective blocking state?
> Is this superior to select/poll solutions, or kqueue?
>> And in that case, I wonder if some generalized socket/file replacement
>> would be even more helpful.
>> I'm especially curious how well the stacklessfile approach has worked
>> for people? Does it perform?
> Having been inactive for quite a while, I'm wondering about asyncore.
> I remember that Sam Rushing, it's author, has said long time ago,
> that with Stackless he could have written it in half the size, and
> very much reactive to computation bound applications.
> While I think the file issue is quite platform dependent.
> What I would like to initiate is trying to find a set of best possible
> solutions, adjusting to what the OS supports.
> I don't believe that it is necessary to loose measurable time.
> The only problem I understand right now without using real
> threads seems to be loosing packets with UDP if I remember.
> cheers - chris
> Christian Tismer :^) <mailto:tismer at stackless.com>
> tismerysoft GmbH : Have a break! Take a ride on Python's
> Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/
> 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/
> work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05
> PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04
> whom do you want to sponsor today? http://www.stackless.com/
> Stackless mailing list
> Stackless at stackless.com
Stackless mailing list
Stackless at stackless.com
More information about the Stackless