OK. I boiled down the code to a small snippet that triggers the bug. It is a bug that is exposed through my use of Twisted. If i create a factory, and then two callbacks that are closures, each callback setting thread level variables. Then I add the callbacks to the factory's deferred, and then I sleep the threadlet forever.<br>
<br>Initiate the tasklet, pickle it, then unpickle it, and BAM, you have your error:<br><br>$ /usr/local/stackless/bin/python break-stackless.py<br>Traceback (most recent call last):<br> File "break-stackless.py", line 47, in <module><br>
test()<br> File "break-stackless.py", line 37, in test<br> obj = pickle.loads(data)<br> File "/usr/local/stackless/lib/python2.6/pickle.py", line 1409, in loads<br> return Unpickler(file).load()<br>
File "/usr/local/stackless/lib/python2.6/pickle.py", line 893, in load<br> dispatch[key](self)<br> File "/usr/local/stackless/lib/python2.6/pickle.py", line 1252, in load_build<br> setstate(state)<br>
TypeError: arg 5 (closure) expected cell, found stackless._wrap.cell<br><br>The code is....<br><br>-----------------------------<br><br>import stackless<br>import pickle<br><br>from twisted.web import client<br><br>def tasklet():<br>
# create an example state<br> factory = client.HTTPClientFactory("<a href="http://www.google.com/">http://www.google.com/</a>", agent = "stackless/1.0" )<br> <br> get_complete = [False]<br>
get_failed = [False]<br> <br> def _doFailure(data):<br> get_failed[0] = factory.status<br> <br> def _doSuccess(data):<br> get_complete[0] = factory.status<br> <br> factory.deferred.addCallback(_doSuccess).addErrback(_doFailure)<br>
<br> while True:<br> stackless.schedule()<br><br>def test():<br> # run the task<br> task = stackless.tasklet(tasklet)<br> task.setup()<br> task.run()<br> <br> # pump once<br> stackless.schedule()<br>
<br> # now serialise<br> data = pickle.dumps(task)<br> <br> # now deserialise<br> obj = pickle.loads(data)<br> <br> # resume<br> task.kill()<br> obj.insert()<br> <br> # pump again<br>
stackless.schedule()<br> <br>if __name__=="__main__":<br> test()<br><br>-----------------------<br><br>If code formatting is an issue, I've attached the code to the email as a file...<br><br>Can anyone shed any light on this bug?<br>
<br>Kind Regards<br><br>Crispin<br><br><br><div class="gmail_quote">On Sat, Oct 10, 2009 at 2:21 AM, Richard Tew <span dir="ltr"><<a href="mailto:richard.m.tew@gmail.com">richard.m.tew@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><div class="h5">On Fri, Oct 9, 2009 at 10:38 PM, Crispin Wellington<br>
<<a href="mailto:retrogradeorbit@gmail.com">retrogradeorbit@gmail.com</a>> wrote:<br>
> I have written a server that contains stackless threads. When the server<br>
> shuts down these threads are serialised to disk. This all happens without<br>
> incident and I end up with a reasonably sized file per pickled object.<br>
><br>
> When I go to deserialise them I hit a problem. I get this...<br>
> -------------------<br>
> File<br>
> "/home/cwellington/Development/yabi/yabi-be-twisted/trunk/TaskManager/Tasklets.py",<br>
> line 54, in load<br>
> task = pickle.loads(data)<br>
> File "/usr/local/stackless/lib/python2.6/pickle.py", line 1415, in<br>
> loads<br>
> return Unpickler(file).load()<br>
> File "/usr/local/stackless/lib/python2.6/pickle.py", line 898, in load<br>
> dispatch[key](self)<br>
> File "/usr/local/stackless/lib/python2.6/pickle.py", line 1258, in<br>
> load_build<br>
> setstate(state)<br>
> exceptions.TypeError: arg 5 (closure) expected cell, found<br>
> stackless._wrap.cell<br>
> --------------------<br>
><br>
> I think this might be a bug. The arguments are being passed back into the<br>
> constructor of the stackless._wrap.function and the constructor is throwing<br>
> that exception. So in essence, the __reduce__ or __reduce_ex__ functions are<br>
> creating instatiation values that do not work on reinstantiation.<br>
><br>
> I changed pickle.py to print out some debug. setstate is... <built-in method<br>
> __setstate__ of stackless._wrap.function object at 0x21ca0c8> and (state)[4]<br>
> is... (<cell at 0x2605f30: empty>, <cell at 0x2940718: list object at<br>
> 0x259f518>, <cell at 0x2940670: str object at 0x2940768>, <cell at<br>
> 0x2940750: str object at 0x29407d8>, <cell at 0x29407c0: int object at<br>
> 0x25cd338>)<br>
><br>
> These all seem to be "cell" objects. Why aren't they (stackless._wrap.cell)<br>
> recognised as "cells"?<br>
><br>
> I can provide a pickletools.dis() dump of the pickle stream if wanted.<br>
><br>
> Using Stackless Python 2.6.2 on 64 bit linux (x86_64).<br>
><br>
> Can anybody help me decode my objects? or even find a way to encode them so<br>
> the are decodable? Is this a bug, or am I doing it wrong?<br>
<br>
</div></div>It definitely looks like a bug. If you can provide the smallest<br>
possible reproduction case, it would help us look into it and we could<br>
add it to the test suite as a regression test.<br>
<br>
If you want your objects back in the meantime, you might try compiling<br>
2.6 with the relevant code raising these errors commented out.<br>
<font color="#888888"><br>
Richard.<br>
</font></blockquote></div><br>