<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
        {font-family:Tahoma;
        panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0cm;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:purple;
        text-decoration:underline;}
span.EmailStyle17
        {mso-style-type:personal-reply;
        font-family:"Calibri","sans-serif";
        color:#1F497D;}
.MsoChpDefault
        {mso-style-type:export-only;}
@page Section1
        {size:612.0pt 792.0pt;
        margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.Section1
        {page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=IS link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Ok, I understand the issue.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Cell objects potentially can be part in a cycle. Therefore for
pickling, the three argument __reduce__ protocol is used, with an initial ‚new‘,
followed by a ‚setstate‘ later.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Only in the ‚setstate‘ call will the type of the
object be changed from the special „_wrap“ type. But precisely
because a reference cycle is being constructed, a fresh cell is being inserted
into a function, before having had its state set, and the function constructor
balks at this.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>I see no way around this using the current wrapped pickle
technique except to fix the test in the function object. Once I have created a
simple regression test, I will submit a fix.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>K<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'>
<p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:
"Tahoma","sans-serif"'>From:</span></b><span lang=EN-US style='font-size:10.0pt;
font-family:"Tahoma","sans-serif"'> stackless-bounces@stackless.com
[mailto:stackless-bounces@stackless.com] <b>On Behalf Of </b>Crispin Wellington<br>
<b>Sent:</b> 12. október 2009 05:27<br>
<b>To:</b> Richard Tew<br>
<b>Cc:</b> stackless@stackless.com<br>
<b>Subject:</b> Re: [Stackless] exceptions.TypeError: arg 5 (closure) expected
cell, found stackless._wrap.cell when unpickling stackless thread<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal style='margin-bottom:12.0pt'>OK. I boiled down the code to a
small snippet that triggers the bug. It is a bug that is exposed through my use
of Twisted. If i create a factory, and then two callbacks that are closures,
each callback setting thread level variables. Then I add the callbacks to the
factory's deferred, and then I sleep the threadlet forever.<br>
<br>
Initiate the tasklet, pickle it, then unpickle it, and BAM, you have your
error:<br>
<br>
$ /usr/local/stackless/bin/python break-stackless.py<br>
Traceback (most recent call last):<br>
File "break-stackless.py", line 47, in <module><br>
test()<br>
File "break-stackless.py", line 37, in test<br>
obj = pickle.loads(data)<br>
File "/usr/local/stackless/lib/python2.6/pickle.py", line
1409, in loads<br>
return Unpickler(file).load()<br>
File "/usr/local/stackless/lib/python2.6/pickle.py", line 893,
in load<br>
dispatch[key](self)<br>
File "/usr/local/stackless/lib/python2.6/pickle.py", line
1252, in load_build<br>
setstate(state)<br>
TypeError: arg 5 (closure) expected cell, found stackless._wrap.cell<br>
<br>
The code is....<br>
<br>
-----------------------------<br>
<br>
import stackless<br>
import pickle<br>
<br>
from twisted.web import client<br>
<br>
def tasklet():<br>
# create an example state<br>
factory = client.HTTPClientFactory("<a
href="http://www.google.com/">http://www.google.com/</a>", agent =
"stackless/1.0" )<br>
<br>
get_complete = [False]<br>
get_failed = [False]<br>
<br>
def _doFailure(data):<br>
get_failed[0] = factory.status<br>
<br>
def _doSuccess(data):<br>
get_complete[0] = factory.status<br>
<br>
factory.deferred.addCallback(_doSuccess).addErrback(_doFailure)<br>
<br>
while True:<br>
stackless.schedule()<br>
<br>
def test():<br>
# run the task<br>
task = stackless.tasklet(tasklet)<br>
task.setup()<br>
task.run()<br>
<br>
# pump once<br>
stackless.schedule()<br>
<br>
# now serialise<br>
data = pickle.dumps(task)<br>
<br>
# now deserialise<br>
obj = pickle.loads(data)<br>
<br>
# resume<br>
task.kill()<br>
obj.insert()<br>
<br>
# pump again<br>
stackless.schedule()<br>
<br>
if __name__=="__main__":<br>
test()<br>
<br>
-----------------------<br>
<br>
If code formatting is an issue, I've attached the code to the email as a
file...<br>
<br>
Can anyone shed any light on this bug?<br>
<br>
Kind Regards<br>
<br>
Crispin<br>
<br>
<o:p></o:p></p>
<div>
<p class=MsoNormal>On Sat, Oct 10, 2009 at 2:21 AM, Richard Tew <<a
href="mailto:richard.m.tew@gmail.com">richard.m.tew@gmail.com</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p class=MsoNormal style='margin-bottom:12.0pt'>On Fri, Oct 9, 2009 at 10:38
PM, Crispin Wellington<br>
<<a href="mailto:retrogradeorbit@gmail.com">retrogradeorbit@gmail.com</a>>
wrote:<br>
> I have written a server that contains stackless threads. When the server<br>
> shuts down these threads are serialised to disk. This all happens without<br>
> incident and I end up with a reasonably sized file per pickled object.<br>
><br>
> When I go to deserialise them I hit a problem. I get this...<br>
> -------------------<br>
> File<br>
>
"/home/cwellington/Development/yabi/yabi-be-twisted/trunk/TaskManager/Tasklets.py",<br>
> line 54, in load<br>
> task = pickle.loads(data)<br>
> File "/usr/local/stackless/lib/python2.6/pickle.py",
line 1415, in<br>
> loads<br>
> return Unpickler(file).load()<br>
> File
"/usr/local/stackless/lib/python2.6/pickle.py", line 898, in load<br>
> dispatch[key](self)<br>
> File
"/usr/local/stackless/lib/python2.6/pickle.py", line 1258, in<br>
> load_build<br>
> setstate(state)<br>
> exceptions.TypeError: arg 5 (closure) expected cell,
found<br>
> stackless._wrap.cell<br>
> --------------------<br>
><br>
> I think this might be a bug. The arguments are being passed back into the<br>
> constructor of the stackless._wrap.function and the constructor is
throwing<br>
> that exception. So in essence, the __reduce__ or __reduce_ex__ functions
are<br>
> creating instatiation values that do not work on reinstantiation.<br>
><br>
> I changed pickle.py to print out some debug. setstate is... <built-in
method<br>
> __setstate__ of stackless._wrap.function object at 0x21ca0c8> and
(state)[4]<br>
> is... (<cell at 0x2605f30: empty>, <cell at 0x2940718: list
object at<br>
> 0x259f518>, <cell at 0x2940670: str object at 0x2940768>,
<cell at<br>
> 0x2940750: str object at 0x29407d8>, <cell at 0x29407c0: int object
at<br>
> 0x25cd338>)<br>
><br>
> These all seem to be "cell" objects. Why aren't they
(stackless._wrap.cell)<br>
> recognised as "cells"?<br>
><br>
> I can provide a pickletools.dis() dump of the pickle stream if wanted.<br>
><br>
> Using Stackless Python 2.6.2 on 64 bit linux (x86_64).<br>
><br>
> Can anybody help me decode my objects? or even find a way to encode them
so<br>
> the are decodable? Is this a bug, or am I doing it wrong?<o:p></o:p></p>
</div>
</div>
<p class=MsoNormal>It definitely looks like a bug. If you can provide the
smallest<br>
possible reproduction case, it would help us look into it and we could<br>
add it to the test suite as a regression test.<br>
<br>
If you want your objects back in the meantime, you might try compiling<br>
2.6 with the relevant code raising these errors commented out.<br>
<span style='color:#888888'><br>
Richard.</span><o:p></o:p></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</div>
</body>
</html>