something to munch on (Was: Re: [Stackless] cool stuff)

Paul Sheer psheer at icon.co.za
Thu Aug 15 13:04:21 CEST 2002


> 
> Multiple channel wait is hard for me to understand completely.

> I want to do it right, once and forever. 

I can't talk about kqueue, but as far as select()
goes (and select() really should be supported for
portability) this is my idea (i'm probably
oversimplifying things):

(also see
  http://threading.2038bug.com/select.html
for definative select() law)


 fd_set global_read_set
 fd_set global_write_set
 fd_set global_oob_set

 _stackless_recieve(fd)
 {
    FD_SET(fd, global_read_set)
    this_thread.waiting_fd_read = fd
    wait_on_continuation()
    this_thread.waiting_fd_read = -1
    FD_CLR(fd, global_read_set)
    return recv_as_much_as_is_available(fd)
 }

 _stackless_send(fd)
 {
    FD_SET(fd, global_write_set)
    this_thread.waiting_fd_write = fd
    wait_on_continuation()
    this_thread.waiting_fd_write = -1
    FD_CLR(fd, global_write_set)
    return send_as_much_as_will_be_accepted(fd)
 }
 
 _stackless_yeild(fd)
 {
    i.just_yielding_to_other_threads = 1
    wait_on_continuation()
    i.just_yielding_to_other_threads = 0
 }
 
 most_global_loop_of_all_loops()
 {
    for_each_thread(i) {
        run_thread(i)
    }
    for(;;) {
       do { /* i think a loop is needed something like this */
          keep_with_this_theme = 0
          for_each_thread(i) {
             if (i.just_yielding_to_other_threads) {
                run_thread(i)
                keep_with_this_theme = 1
             }
          }
       } while (keep_with_this_theme)

       select(global_read_set, global_write_set, global_oob_set)

       for_each_thread(i) {
          if (i.waiting_fd_read >= 0 &&
             FD_ISSET(i.waiting_fd_read, global_read_set))
                run_thread(i)
          else if (i.waiting_fd_write >= 0 &&
             FD_ISSET(i.waiting_fd_write, global_write_set))
                run_thread(i)
          else if (i.waiting_fd_oob >= 0 &&
             FD_ISSET(i.waiting_fd_oob, global_oob_set))
                run_thread(i)
       }
    }
 }


So each thread will automatically pause on send() and
recv(), and can volantarily yield to other threads using
_stackless_yield().

looking at the glib code (VERY important) they build
everything around select(). They have an interesting
feature: You can add callbacks to be run before the
select() (to set fd's into their sets), after the
select() (to do any reading and writing, but then ALSO
you can add a third callback outside of both of these.

 loop_over_all_modules()
   run_callbacks_out_of_race()   /* third callback */

 loop_over_all_modules()
   run_callbacks_setup_fdsets()

 select()

 loop_over_all_modules()
   run_callbacks_do_IO()

WHY this third callback???? Now i discovered (completely
independently, and before reading the glib sources) that
you need this third callback to prevent race conditions
between your modules. A module may need to do something
with the data that another module has recv()d before
deciding whether to do an FD_SET().

This callback sorts out certain kinds of dependencies
between modules.

I BELIEVE THAT THE REASON TRADITIONAL MULTITHREADING
FAILS IS BECAUSE IT IMPLICITLY OMITS THIS THIRD
OUT-OF-RACE HANDLER (??????)

The explicit yield is equiavlent to this third
callback.

The point here is that we are moving from an event
based model toward a thread model. It used to be an
"either thread based or event based". But now we are
somewhere in between. So my question is: at what point
do we cross the line and end up with the same problems
that traditional multithreading has?

-paul

Paul Sheer Consulting IT Services . . Tel . . . +27 (0)21 6869634
Email . . . psheer at icon.co.za . . . . . . Pager . . . 088 0057245
Linux development, cryptography, recruitment,  support,  training
http://www.icon.co.za/~psheer . . . . http://rute.sourceforge.net
L I N U X . . . . . . . . . . . .  The Choice of a GNU Generation

_______________________________________________
Stackless mailing list
Stackless at www.tismer.com
http://www.tismer.com/mailman/listinfo/stackless



More information about the Stackless mailing list