[Stackless] irc threads

Larry Dickson ldickson at cuttedge.com
Thu Jul 23 17:08:36 CEST 2009


This all sounds like good stuff, and I had a colleague in another context
mention some of these select-like Windows capabalities to me. All that
matters is the effect - and, of course, avoidance of hidden complexity that
causes hangs and crashes. One thing, though, you say you do non-blocking IO
because the server must run a simulation loop 20 times a second. In my other
context, I block on both IO and timeout (coincidentally, 20 times a second!)
and get the timeout very precise with the help of gettimeofday. This lets
you force exactly 20 timeouts a second and also allows
better-than-50-millisecond response to the IO event. I can see though how
your application might be well-served with 50-millisecond response and then
rush through all the IOs that have accumulated during that time.

Larry Dickson
Cutting Edge Networked Storage

On 7/21/09, Kristján Valur Jónsson <kristjan at ccpgames.com> wrote:
>
>  Well, select() is discouraged for non-blocking IO, because there are
> other ways of doing it.
>
> The best way on windows is to use IO Completion ports, where you associate
> a socket with such a port and then wait for completion.  There are many
> different ways to use that model.  For example, have all the sockets share a
> single port and then check it  regularly (by blocking or polling it).  This
> is effectively the same as using select(), except that is scales better.
> There is no expensive setting up of file descriptors for the call, and no
> maximum number of sockets. (512 for select on windows, I think)
>
> Another way is to have a thread from the windows threadpool call a callback
> once an IO event on the port completes, and this is what we use in
> StacklessIO.  This enables us to set certain flags within python immediately
> and wake up the blocking tasklets as soon as possible.
>
>
>
> As for why we want to do non-blocking IO, well, a server may have to do
> other stuff than _*only*_ respond to network events.  In the case of EVE,
> it is a game and it runs a game loop.  We try to make it run the simulation
> loop at least 20 times a second.  It does this, even if there is no socket
> IO.  Sometimes, we _*do*_ actually block.  We maintain a queue of
> scheduled internal events and if not much is happening in the simulation, we
> will use the windows WaitForMultipleObjects() sleep function with a delay.
> Any IO will break that sleep loop.  It is another feature of StacklessIO
> that it maintains a windows Event object, that the user can use to break out
> of such wait calls in the event of IO Completion.
>
>
>
> StacklessIO is currently deployed and is handling some 3000 client socket
> connections on a single thread for each proxy, while running a program loop.
>
>
>
> Cheers,
>
>
>
> Kristján
>
>
>
>
>
>
>
> *From:* Larry Dickson [mailto:ldickson at cuttedge.com]
> *Sent:* 16. júlí 2009 14:23
> *To:* Kristján Valur Jónsson
> *Cc:* Henning Diedrich; stackless at stackless.com
> *Subject:* Re: [Stackless] irc threads
>
>
>
> This is interesting - your reference notes that select is discouraged for
> NON-BLOCKING IO, which it seems to me forces you into polling mode, which is
> always bad - not only because of busy resource use, but also because of
> slowness of response since one presumably chains everything off the timer in
> this case. As for scaling badly, timer ticks that have to check everything
> every time have the same problem - and there are ways of making an n-fold
> select loop have n log n overhead instead of n squared.
>
>
>
> What I'm seeing in these Windows notes is "not seeing the forest for the
> trees" in questions of responsive event programming with multiple inputs. I
> suspect that, hiding behind it all, the idea that you can have a separate
> thread for each event is still lurking, but that is only OK if the threads
> hardly ever share data or timing. The prevalence of non-blocking IO, a.k.a.
> busy loop, just makes me scratch my head. That just means someone has not
> finished designing the program.
>
>
> Larry
>
>
> On 7/15/09, *Kristján Valur Jónsson* <kristjan at ccpgames.com> wrote:
>
> Also, I‘d like to add that we don‘t generally want to block, even for
> select().
>
> In our framework, the application may have numerous tasks to do when it is
> not servicing IO.  This necessitates using select to poll all the sockets,
> with the associated setup overhead.
>
> select() as such is recognized as scaling badly, even on unix() which is
> why (I‘ve been told) there are now alternatives for asynchronous IO commonly
> available on linux.
>
> K
>
>
>
> *From:* stackless-bounces at stackless.com [mailto:
> stackless-bounces at stackless.com] *On Behalf Of *Kristján Valur Jónsson
> *Sent:* 14. júlí 2009 23:08
> *To:* Larry Dickson
>
>
> *Cc:* Henning Diedrich; stackless at stackless.com
> *Subject:* Re: [Stackless] irc threads
>
>
>
> Well, on Windows, one of the rules of efficient network programming is to
> not use select().
>
> See:  http://tangentsoft.net/wskfaq/articles/io-strategies.html
>
> and http://msdn.microsoft.com/en-us/magazine/dvdarchive/cc302334.aspx
>
> select() is discouraged on windows except for compatibility reasons, since
> it works contrary to the internal IO scheduling mechanisms.
>
>
>
> K
>
>
>
> *From:* Larry Dickson [mailto:ldickson at cuttedge.com]
> *Sent:* 14. júlí 2009 15:34
> *To:* Kristján Valur Jónsson
> *Cc:* Henning Diedrich; stackless at stackless.com
> *Subject:* Re: [Stackless] irc threads
>
>
>
> There seems to be a false assumption here: select does not "poll the
> sockets" (or anything else) and it is not inefficient. In fact, it blocks
> until one of  the events in question takes place, then reawakens a single
> process (e.g. main tasklet). This is as efficient as you can get, and
> "native facilities" are almost certainly just doing the same thing in a
> hidden place.
>
>
>
> Larry Dickson
> Cutting Edge Networked Storage
>
>
> On 7/9/09, *Kristján Valur Jónsson* <kristjan at ccpgames.com> wrote:
>
> StacklessIO is written in C++.  It uses native facilities to notify an
> internal event queue when an IO request has completed, instead of requiring
> the "main tasklet" to regularly poll the sockets using select() which is
> inefficient.  This aims to minimize latency and reduce overhead.  It can
> also use the async. notification facility of the python C api to wake up
> sleeping tasklets without requiring the main tasklet to poll the event
> queue.
> It is also more 'complete' in its emulation of the native socket module, I
> think.
> But performance tests by Richard on his module have still shown it to be
> very capable, and to scale better than threaded solutions like your irc
> server, so it may well be quite adequate for the task.
>
> K
>
> > -----Original Message-----
> > From: Henning Diedrich [mailto:hd at authentic-internet.de]
> > Sent: 9. júlí 2009 14:02
> > To: Kristján Valur Jónsson; stackless at stackless.com
> > Subject: Re: [Stackless] irc threads
> >
> >
> >
> > Kristján Valur Jónsson wrote:
> > > Yes, stacklessIO is designed to do that for you, to provide a
> > transparently tasklet-blocking replacement module for socket and
> > others.
> > > However, I hven't still managed to release it (although I do intend
> > to as soon as I can) and it is at them moment Windows only, and likely
> > to remain so for a bit.
> > >
> > > Meanwhile, there is Richard Tew's async socket implementation.
> > >
> > > K
> > >
> >
> > Thanks for the clarification. What is the difference between
> > stacklessIO
> > and Richard's implementation? This question is probably naive but maybe
> > some pointers are possible?
> >
> > Thanks,
> > Henning
>
>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20090723/e9afc818/attachment.htm>


More information about the Stackless mailing list