<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="&#1;" xmlns="http://www.w3.org/TR/REC-html40">

<head>
<meta http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
 /* Font Definitions */
 @font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
        {font-family:Tahoma;
        panose-1:2 11 6 4 3 5 4 4 2 4;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0cm;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:purple;
        text-decoration:underline;}
span.EmailStyle17
        {mso-style-type:personal-reply;
        font-family:"Calibri","sans-serif";
        color:#1F497D;}
.MsoChpDefault
        {mso-style-type:export-only;}
@page Section1
        {size:612.0pt 792.0pt;
        margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.Section1
        {page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
 <o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
 <o:shapelayout v:ext="edit">
  <o:idmap v:ext="edit" data="1" />
 </o:shapelayout></xml><![endif]-->
</head>

<body lang=IS link=blue vlink=purple>

<div class=Section1>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Ok, I understand the issue.<o:p></o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Cell objects potentially can be part in a cycle.  Therefore for
pickling, the three argument __reduce__ protocol is used, with an initial &#8218;new&#8216;,
followed by a &#8218;setstate&#8216; later.<o:p></o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Only in the &#8218;setstate&#8216; call will the type of the
object be changed from the special &#8222;_wrap&#8220; type.  But precisely
because a reference cycle is being constructed, a fresh cell is being inserted
into a function, before having had its state set, and the function constructor
balks at this.<o:p></o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>I see no way around this using the current wrapped pickle
technique except to fix the test in the function object.  Once I have created a
simple regression test, I will submit a fix.<o:p></o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p>&nbsp;</o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>K<o:p></o:p></span></p>

<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p>&nbsp;</o:p></span></p>

<div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'>

<div>

<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'>

<p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:
"Tahoma","sans-serif"'>From:</span></b><span lang=EN-US style='font-size:10.0pt;
font-family:"Tahoma","sans-serif"'> stackless-bounces@stackless.com
[mailto:stackless-bounces@stackless.com] <b>On Behalf Of </b>Crispin Wellington<br>
<b>Sent:</b> 12. október 2009 05:27<br>
<b>To:</b> Richard Tew<br>
<b>Cc:</b> stackless@stackless.com<br>
<b>Subject:</b> Re: [Stackless] exceptions.TypeError: arg 5 (closure) expected
cell, found stackless._wrap.cell when unpickling stackless thread<o:p></o:p></span></p>

</div>

</div>

<p class=MsoNormal><o:p>&nbsp;</o:p></p>

<p class=MsoNormal style='margin-bottom:12.0pt'>OK. I boiled down the code to a
small snippet that triggers the bug. It is a bug that is exposed through my use
of Twisted. If i create a factory, and then two callbacks that are closures,
each callback setting thread level variables. Then I add the callbacks to the
factory's deferred, and then I sleep the threadlet forever.<br>
<br>
Initiate the tasklet, pickle it, then unpickle it, and BAM, you have your
error:<br>
<br>
$ /usr/local/stackless/bin/python break-stackless.py<br>
Traceback (most recent call last):<br>
&nbsp; File &quot;break-stackless.py&quot;, line 47, in &lt;module&gt;<br>
&nbsp;&nbsp;&nbsp; test()<br>
&nbsp; File &quot;break-stackless.py&quot;, line 37, in test<br>
&nbsp;&nbsp;&nbsp; obj =&nbsp; pickle.loads(data)<br>
&nbsp; File &quot;/usr/local/stackless/lib/python2.6/pickle.py&quot;, line
1409, in loads<br>
&nbsp;&nbsp;&nbsp; return Unpickler(file).load()<br>
&nbsp; File &quot;/usr/local/stackless/lib/python2.6/pickle.py&quot;, line 893,
in load<br>
&nbsp;&nbsp;&nbsp; dispatch[key](self)<br>
&nbsp; File &quot;/usr/local/stackless/lib/python2.6/pickle.py&quot;, line
1252, in load_build<br>
&nbsp;&nbsp;&nbsp; setstate(state)<br>
TypeError: arg 5 (closure) expected cell, found stackless._wrap.cell<br>
<br>
The code is....<br>
<br>
-----------------------------<br>
<br>
import stackless<br>
import pickle<br>
<br>
from twisted.web import client<br>
<br>
def tasklet():<br>
&nbsp;&nbsp;&nbsp; # create an example state<br>
&nbsp;&nbsp;&nbsp; factory = client.HTTPClientFactory(&quot;<a
href="http://www.google.com/">http://www.google.com/</a>&quot;, agent =
&quot;stackless/1.0&quot; )<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; get_complete = [False]<br>
&nbsp;&nbsp;&nbsp; get_failed = [False]<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; def _doFailure(data):<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; get_failed[0] = factory.status<br>
&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; def _doSuccess(data):<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; get_complete[0] = factory.status<br>
&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp;
factory.deferred.addCallback(_doSuccess).addErrback(_doFailure)<br>
&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; while True:<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; stackless.schedule()<br>
<br>
def test():<br>
&nbsp;&nbsp;&nbsp; # run the task<br>
&nbsp;&nbsp;&nbsp; task = stackless.tasklet(tasklet)<br>
&nbsp;&nbsp;&nbsp; task.setup()<br>
&nbsp;&nbsp;&nbsp; task.run()<br>
&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; # pump once<br>
&nbsp;&nbsp;&nbsp; stackless.schedule()<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; # now serialise<br>
&nbsp;&nbsp;&nbsp; data = pickle.dumps(task)<br>
&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; # now deserialise<br>
&nbsp;&nbsp;&nbsp; obj =&nbsp; pickle.loads(data)<br>
&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; # resume<br>
&nbsp;&nbsp;&nbsp; task.kill()<br>
&nbsp;&nbsp;&nbsp; obj.insert()<br>
&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; # pump again<br>
&nbsp;&nbsp;&nbsp; stackless.schedule()<br>
&nbsp;&nbsp;&nbsp;&nbsp; <br>
if __name__==&quot;__main__&quot;:<br>
&nbsp;&nbsp;&nbsp; test()<br>
<br>
-----------------------<br>
<br>
If code formatting is an issue, I've attached the code to the email as a
file...<br>
<br>
Can anyone shed any light on this bug?<br>
<br>
Kind Regards<br>
<br>
Crispin<br>
<br>
<o:p></o:p></p>

<div>

<p class=MsoNormal>On Sat, Oct 10, 2009 at 2:21 AM, Richard Tew &lt;<a
href="mailto:richard.m.tew@gmail.com">richard.m.tew@gmail.com</a>&gt; wrote:<o:p></o:p></p>

<div>

<div>

<p class=MsoNormal style='margin-bottom:12.0pt'>On Fri, Oct 9, 2009 at 10:38
PM, Crispin Wellington<br>
&lt;<a href="mailto:retrogradeorbit@gmail.com">retrogradeorbit@gmail.com</a>&gt;
wrote:<br>
&gt; I have written a server that contains stackless threads. When the server<br>
&gt; shuts down these threads are serialised to disk. This all happens without<br>
&gt; incident and I end up with a reasonably sized file per pickled object.<br>
&gt;<br>
&gt; When I go to deserialise them I hit a problem. I get this...<br>
&gt; -------------------<br>
&gt; &nbsp;&nbsp;&nbsp; &nbsp; File<br>
&gt;
&quot;/home/cwellington/Development/yabi/yabi-be-twisted/trunk/TaskManager/Tasklets.py&quot;,<br>
&gt; line 54, in load<br>
&gt; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; task = pickle.loads(data)<br>
&gt; &nbsp;&nbsp;&nbsp; &nbsp; File &quot;/usr/local/stackless/lib/python2.6/pickle.py&quot;,
line 1415, in<br>
&gt; loads<br>
&gt; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; return Unpickler(file).load()<br>
&gt; &nbsp;&nbsp;&nbsp; &nbsp; File
&quot;/usr/local/stackless/lib/python2.6/pickle.py&quot;, line 898, in load<br>
&gt; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; dispatch[key](self)<br>
&gt; &nbsp;&nbsp;&nbsp; &nbsp; File
&quot;/usr/local/stackless/lib/python2.6/pickle.py&quot;, line 1258, in<br>
&gt; load_build<br>
&gt; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; setstate(state)<br>
&gt; &nbsp;&nbsp;&nbsp; exceptions.TypeError: arg 5 (closure) expected cell,
found<br>
&gt; stackless._wrap.cell<br>
&gt; --------------------<br>
&gt;<br>
&gt; I think this might be a bug. The arguments are being passed back into the<br>
&gt; constructor of the stackless._wrap.function and the constructor is
throwing<br>
&gt; that exception. So in essence, the __reduce__ or __reduce_ex__ functions
are<br>
&gt; creating instatiation values that do not work on reinstantiation.<br>
&gt;<br>
&gt; I changed pickle.py to print out some debug. setstate is... &lt;built-in
method<br>
&gt; __setstate__ of stackless._wrap.function object at 0x21ca0c8&gt; and
(state)[4]<br>
&gt; is... (&lt;cell at 0x2605f30: empty&gt;, &lt;cell at 0x2940718: list
object at<br>
&gt; 0x259f518&gt;, &lt;cell at 0x2940670: str object at 0x2940768&gt;,
&lt;cell at<br>
&gt; 0x2940750: str object at 0x29407d8&gt;, &lt;cell at 0x29407c0: int object
at<br>
&gt; 0x25cd338&gt;)<br>
&gt;<br>
&gt; These all seem to be &quot;cell&quot; objects. Why aren't they
(stackless._wrap.cell)<br>
&gt; recognised as &quot;cells&quot;?<br>
&gt;<br>
&gt; I can provide a pickletools.dis() dump of the pickle stream if wanted.<br>
&gt;<br>
&gt; Using Stackless Python 2.6.2 on 64 bit linux (x86_64).<br>
&gt;<br>
&gt; Can anybody help me decode my objects? or even find a way to encode them
so<br>
&gt; the are decodable? Is this a bug, or am I doing it wrong?<o:p></o:p></p>

</div>

</div>

<p class=MsoNormal>It definitely looks like a bug. &nbsp;If you can provide the
smallest<br>
possible reproduction case, it would help us look into it and we could<br>
add it to the test suite as a regression test.<br>
<br>
If you want your objects back in the meantime, you might try compiling<br>
2.6 with the relevant code raising these errors commented out.<br>
<span style='color:#888888'><br>
Richard.</span><o:p></o:p></p>

</div>

<p class=MsoNormal><o:p>&nbsp;</o:p></p>

</div>

</div>

</body>

</html>