[Stackless] monkeypatching
Kristján Valur Jónsson
kristjan at ccpgames.com
Mon Dec 2 00:48:36 CET 2013
Apart from the bug you found (and crafted by me), your example illustrates some of the shortcomings of monkeypatching.
The framework you are working with is clearly using socket.select() to do timeouts when listening to connections. This is fairly common in Python (socketserver.py does this) but IMHO bad practice.
First, "accept" itself can timeout and python oughty to simply better support timeout in that api.
Second, there is no "asynchronous" select. We monkeypatch select by farming it off to a worker thread.
In a stackless application this introduces latency.
To accept a connection, your application has to wake up from select(), then enter an accept() state and then wake up from that.
Select() is not a nice API and the reason we do IO in stackless (using multiple tasklets) is generally in order to avoid using select().
K
________________________________
From: stackless-bounces at stackless.com [stackless-bounces at stackless.com] on behalf of lars van Gemerden [lars at rational-it.com]
Sent: Sunday, December 01, 2013 1:58 PM
To: The Stackless Python Mailing List
Subject: Re: [Stackless] monkeypatching
OK, thanks for the input,
i am trying to get this to work, but i might have found a bug (which i find hard to debug):
i have reimplemented the code above as:
--------------------------------------------------------------
import asyncore, traceback, sys, logging
import stacklesslib.main
import stacklesslib.app
# the unittests use time.time() for various time tests.
# Therefore, we must make sure that main uses this
from time import time as elapsed_time
stacklesslib.main.elapsed_time = elapsed_time
class MonkeyTasklets(object):
_stop = False
stacklesslib.app.install_stackless()
@classmethod
def run(cls, world, webserver = None, maxcount = None): #param killer is ignored
world.flowmodel.start()
if webserver:
cls.server = webserver.start_server(world_model = world) #starts a tasklet for running the webserver
counter = 0
while not cls._stop or counter > maxcount:
counter += 1
tick_time = elapsed_time()
try:
stacklesslib.main.mainloop.loop()
except Exception as e:
import asyncore
if isinstance(e, ReferenceError):
print("run:EXCEPTION", str(e), asyncore.socket_map)
else:
print("run:EXCEPTION", asyncore.socket_map)
traceback.print_exc()
sys.exc_clear()
world.flowmodel.stop()
@classmethod
def schedule(cls):
stackless.schedule()
@classmethod
def tasklet(cls, func):
return stackless.tasklet(func)
@classmethod
def stop(cls):
if hasattr(cls, "server"):
cls.server.stop()
cls._stop = True
tasklets = MonkeyTasklets
--------------------------------------------------------------
and i get the exception:
File "d:\Documents\Code\python\floware\server\bottle.py", line 641, in run
run(self, **kwargs)
File "d:\Documents\Code\python\floware\server\bottle.py", line 2720, in run
server.run(app)
File "d:\Documents\Code\python\floware\server\webserver.py", line 329, in run
self.server.serve_forever()
File "C:\Python27\lib\SocketServer.py", line 236, in serve_forever
poll_interval)
File "C:\Python27\lib\SocketServer.py", line 155, in _eintr_retry
return func(*args)
File "d:\Documents\Code\python\floware\stacklesslib\replacements\select.py", line 27, in select
return stacklesslib.threadpool.call_on_thread(real_select.select, args, kwargs)
File "d:\Documents\Code\python\floware\stacklesslib\threadpool.py", line 123, in call_on_thread
return tasklet_call(wrapped, dispatcher, timeout=timeout, onOrphaned=onOrphaned)
File "d:\Documents\Code\python\floware\stacklesslib\util.py", line 209, in tasklet_call
return channel_wait(chan, timeout)
File "d:\Documents\Code\python\floware\stacklesslib\util.py", line 53, in channel_wait
return chan.receive()
File "d:\Documents\Code\python\floware\stacklesslib\util.py", line 192, in helper
result = function(*args, **kwargs)
TypeError: wrapped() argument after * must be a sequence, not function
--------------------------------------------------------------
is this something i am doing wrong or indeed a buggy in stacklesslib and if so, how can it be fixed?
Cheers, Lars
On Sun, Dec 1, 2013 at 4:05 AM, Richard Tew <richard.m.tew at gmail.com<mailto:richard.m.tew at gmail.com>> wrote:
latest stacklesslib, i mean.
On 12/1/13, Richard Tew <richard.m.tew at gmail.com<mailto:richard.m.tew at gmail.com>> wrote:
> Hi Lars,
>
> stacklessio is internal ccp stuff.
>
> main.py in stacklesslib is pretty much all you need to understand,
> beyond calling patch_all(). It is a scheduler, and makes your Open
> scheduler redundant.
>
> I suggest you get the latest stacklessio from:
>
> https://bitbucket.org/krisvale/stacklesslib
>
> And then you check out test / teststdlibunittests.py, which is a short
> example of monkey patching and running the stacklesslib scheduler.
>
> Cheers,
> Richard.
>
> On 12/1/13, lars van Gemerden <lars at rational-it.com<mailto:lars at rational-it.com>> wrote:
>> sorry,
>>
>> patch_all seemed to simple, question was a bit rethoric ;-) tried it 2
>> mins
>> after pressing send just in case and indeed no cigar.
>>
>> I am trying to get this demo running and all kinds of last minute #### is
>> popping up.
>>
>> I tried reading the source but it doesn't click; i am not familiar with
>> using sockets etc. at all.
>>
>> e.g.
>>
>> - do i need 'stacklessio'?
>> - can i (essentially) just put a stackless.schedule() (or
>> OpenTasklets.schedule() in the code below) in the serve_forever loop?
>> - will this work at all with an adapted scheduler like this:
>>
>> class OpenTasklets(object):
>>
>> _stop = False
>>
>> @classmethod
>> def run(cls, maxcount = None):
>> cls.schedule_channel = stackless.channel()
>> cls.schedule_channel.preference = 1
>> counter = 0
>> OpenTasklets._stop = False
>> while stackless.getruncount() != 1:
>> stackless.run()
>> cls.reschedule(cls._stop or (maxcount and counter >
>> maxcount))
>> counter += 1
>>
>> @classmethod
>> def schedule(cls):
>> if cls.schedule_channel.receive():
>> raise TaskletExit
>>
>> @classmethod
>> def reschedule(cls, stop = False):
>> while cls.schedule_channel.balance < 0:
>> cls.schedule_channel.send(stop)
>>
>> Any help is appreciated ..
>>
>> Lars
>>
>>
>>
>>
>> On Sat, Nov 30, 2013 at 8:53 PM, Richard Tew
>> <richard.m.tew at gmail.com<mailto:richard.m.tew at gmail.com>>wrote:
>>
>>> Lars :-)
>>>
>>> Yes, there is no documentation for stacklesslib yet. To use it, you
>>> really need to be willing to read the source code.
>>>
>>> You should already know whether calling patch_all would be enough. It
>>> would have been something you could have tried immediately, rather
>>> than asking the list and waiting.
>>>
>>> Cheers,
>>> Richard.
>>>
>>> On 11/30/13, lars van Gemerden <lars at rational-it.com<mailto:lars at rational-it.com>> wrote:
>>> > Hi all,
>>> >
>>> > I though i could avoid it to run my demo but it seems i need to let
>>> > the
>>> > webserver (simple server?, single thread, 'bottle' micro framework)
>>> > yield
>>> > to the scheduler i am using. I have downloaded stacklesslib 1.0.3, but
>>> > can't find any documentation to help with monkeypatching.
>>> >
>>> > - Do i just run patch_all at the start of the program?
>>> >
>>> > Any help is very welcome ..
>>> >
>>> > Cheers, Lars
>>> >
>>> > --
>>> > ====================================
>>> > Lars van Gemerden
>>> > lars at rational-it.com<mailto:lars at rational-it.com>
>>> > +31 6 26 88 55 39<tel:%2B31%206%2026%2088%2055%2039>
>>> > ====================================
>>> >
>>>
>>> _______________________________________________
>>> Stackless mailing list
>>> Stackless at stackless.com<mailto:Stackless at stackless.com>
>>> http://www.stackless.com/mailman/listinfo/stackless
>>>
>>
>>
>>
>> --
>> ====================================
>> Lars van Gemerden
>> lars at rational-it.com<mailto:lars at rational-it.com>
>> +31 6 26 88 55 39<tel:%2B31%206%2026%2088%2055%2039>
>> ====================================
>>
>
_______________________________________________
Stackless mailing list
Stackless at stackless.com<mailto:Stackless at stackless.com>
http://www.stackless.com/mailman/listinfo/stackless
--
====================================
Lars van Gemerden
lars at rational-it.com<mailto:lars at rational-it.com>
+31 6 26 88 55 39[X]
====================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20131201/ff5a2bc5/attachment-0001.html>
More information about the Stackless
mailing list