[Stackless] stacklessocket with threads, and stacklessfile

[Tim Kientzle]

Chris,

The problem with today's stacklesssocket is the I/O
pump tasklet that repeatedly calls select().
Remember that select() is itself a blocking system call,
so the I/O pump invokes it with a short timeout
so that other tasklets get serviced.

The application that Alec is working on has a mix of
computation tasklets and I/O tasklets and the current
stacklesssocket design causes the computation tasklets
to get stalled everytime select() gets called by the
I/O pump.

Our proposal: run the select() loop in a separate
thread so that the select() doesn't block all other
tasklets.

Does this make sense?

Tim Kientzle

P.S.  Using a select() alternative doesn't help.
poll(), epoll(), kqueue(), etc, all have exactly
the same problem.

Christian Tismer wrote:
> Alec,
> 
> On Aug 8, 2007, at 7:50 PM, Alec Flett wrote:
> 
>> Hey folks -
>> So we're using stacklessocket with an app that is somewhat CPU 
>> intense, but also does I/O. We're trying to use stackless socket, but 
>> what we're actually finding is that overall performance is suffering 
>> because of the somewhat long wait time in the asynccore runner 
>> tasklet. ( 0.05) - if we block for more than 0.05 seconds, this ends 
>> up wasting about 5% of of our CPU.
> 
> Could you please supply exact reference to what implementation
> you are using? I'm not exactly up to date.
> 
>> We can certainly play with that number to find out what's optimal for 
>> our particular application, but I'm curious if anyone has thought 
>> about taking the same approach that stacklessfile does of having a 
>> pool of worker threads so that real hardware threads can go into a 
>> more effective blocking state? 
> 
> Is this superior to select/poll solutions, or kqueue?
> 
>> And in that case, I wonder if some generalized socket/file replacement 
>> would be even more helpful.
>>
>> I'm especially curious how well the stacklessfile approach has worked 
>> for people? Does it perform?
> 
> Having been inactive for quite a while, I'm wondering about asyncore.
> I remember that Sam Rushing, it's author, has said long time ago,
> that with Stackless he could have written it in half the size, and
> very much reactive to computation bound applications.
> 
> While I think the file issue is quite platform dependent.
> 
> What I would like to initiate is trying to find a set of best possible
> solutions, adjusting to what the OS supports.
> 
> I don't believe that it is necessary to loose measurable time.
> The only problem I understand right now without using real
> threads seems to be loosing packets with UDP if I remember.
> 
> cheers - chris
> 
> --
> Christian Tismer             :^)   <mailto:tismer at stackless.com>
> tismerysoft GmbH             :     Have a break! Take a ride on Python's
> Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
> 14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
> work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
> PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
>       whom do you want to sponsor today?   http://www.stackless.com/
> 
> 
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://stackless.com/cgi-bin/mailman/listinfo/stackless

_______________________________________________
Stackless mailing list
Stackless at stackless.com
http://stackless.com/cgi-bin/mailman/listinfo/stackless