[Stackless] Fwd: deepcopying (pickling) channels
lars van Gemerden
lars at rational-it.com
Thu Oct 10 17:23:42 CEST 2013
the callstack is:
pickle.PicklingError: Can't pickle <type 'function'>: it's not found as
__builtin__.function
File "d:\Documents\Code\python\floware\toolshed\tests\stackless_test.py",
line 34, in <module>
print pickle.dumps(t) #ERROR
File "C:\Python27\lib\pickle.py", line 1411, in dumps
Pickler(file, protocol).dump(obj)
File "C:\Python27\lib\pickle.py", line 231, in dump
self.save(obj)
File "C:\Python27\lib\pickle.py", line 338, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 426, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 661, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 675, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 338, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 426, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 569, in save_tuple
save(element)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 607, in save_list
self._batch_appends(iter(obj))
File "C:\Python27\lib\pickle.py", line 622, in _batch_appends
save(x)
File "C:\Python27\lib\pickle.py", line 338, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 426, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 569, in save_tuple
save(element)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 607, in save_list
self._batch_appends(iter(obj))
File "C:\Python27\lib\pickle.py", line 622, in _batch_appends
save(x)
File "C:\Python27\lib\pickle.py", line 338, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 426, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 569, in save_tuple
save(element)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 661, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 675, in _batch_setitems
save(v)
<== THIS LINE
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 802, in save_function
return self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 426, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 569, in save_tuple
save(element)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 661, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 675, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 338, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 426, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 661, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 675, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 293, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 760, in save_global
(obj, module, name))
Probably not that usefull without values ... in THIS LINE above, v is
module C:\python27\lib\new.pyc.
I don't think there is a framework in place that would influence this
script, the imports (pickle, stackless) are unaltered.
I run this with the default settings in wing IDE. If i run it in IDLE,
there is no exception.
On Thu, Oct 10, 2013 at 3:22 PM, Kristján Valur Jónsson <
kristjan at ccpgames.com> wrote:
> The error is weird. It seems to be trying to pickle the “function”
> type. It shouldn’t try doing that.****
>
> Now, I know that pickling channels with tasklets on them works. This is
> what stackless does.****
>
> I tried saving your code into a module and just running it from the
> command line.****
>
> I wonder if there is some copy_reg magic that is interfering with the
> code. How are you running it? Is this when running the code as a module,
> or when typing it in from the console?****
>
> Are you trying this in a stand-alone environment or in some application
> framework?****
>
> ** **
>
> Finally, can I see the full traceback for the pickle exception. If you
> are using pickle (and not cPickle) a traceback can help us find out why it
> is failing.****
>
> ** **
>
> K****
>
> ** **
>
> *From:* stackless-bounces at stackless.com [mailto:
> stackless-bounces at stackless.com] *On Behalf Of *lars van Gemerden
> *Sent:* 9. október 2013 12:48
>
> *To:* The Stackless Python Mailing List
> *Subject:* Re: [Stackless] Fwd: deepcopying (pickling) channels****
>
> ** **
>
> Hi Kristjan,****
>
> ** **
>
> the version i am using is: python 2.7.5 Stackless 3.1b3 (from binary).****
>
> ** **
>
> I think the stackless mailsystem was a little off, i tried with
> pickle.dumps (see mail history below), with the error: pickle.PicklingError:
> Can't pickle <type 'function'>: it's not found as __builtin__.function.***
> *
>
> ** **
>
> "stackless.run() returns when there are no *runnable* tasklets ...", ok,
> clear, thanks.****
>
> ** **
>
> Cheers, Lars****
>
> ** **
>
> On Wed, Oct 9, 2013 at 2:36 PM, lars van Gemerden <lars at rational-it.com>
> wrote:****
>
> I am reasonably familiar with pickle (e.g. i know you cant pickle most
> functions), but this is pretty stackless specific: why can i pickle an
> empty channel, but not a sending/receiving channel?****
>
> ** **
>
> Cheers, Lars****
>
> ** **
>
> On Mon, Oct 7, 2013 at 7:50 PM, Anselm Kruis <a.kruis at science-computing.de>
> wrote:****
>
> Hi Lars,
>
> you are observing well known deficiencies of the pickle/cPickle
> implementation of Python. Perhaps sPickle (
> http://pypi.python.org/pypi/sPickle) can pickle the channels.
>
> Regards
> Anselm
>
>
> Am 06.10.2013 08:58, schrieb lars van Gemerden:****
>
> maybe double but i got a mail bounce ..
>
> ---------- Forwarded message ----------
> From: lars van Gemerden <lars at rational-it.com>
> Date: Thu, Oct 3, 2013 at 6:15 PM
> Subject: Re: [Stackless] deepcopying (pickling) channels
> To: The Stackless Python Mailing List <stackless at stackless.com>
>
>
> actually both give an error. The only thing different is the error message.
> with pickle the code would be:
>
> ---------------------------------------------------------------------------
> import stackless, pickle
>
> class Test(object):
> def __init__(self):
> self.channel = stackless.channel()
>
> def run(self):
> stackless.tasklet(self.sender)()
> stackless.tasklet(self.receiver)()
> stackless.run()
>
> def sender(self):
> counter = 0
> while counter < 10:
> self.channel.send(counter)
> counter += 1
>
> def receiver(self):
> counter = 0
> while counter < 12:
> counter = self.channel.receive()
> print counter,
> print "done"
>
> if __name__ == "__main__":
>
> t = Test()
> t.run()
> t.channel = None
> s = pickle.dumps(t) #OK
>
> t = Test()
> t.run()
> s = pickle.dumps(t) #ERROR
>
> -----------------------------------------------------------------------------
> and the error:
>
> pickle.PicklingError: Can't pickle <type 'function'>: it's not found as
> __builtin__.function
>
> otherwise the error behavior is the same as decribed above.
>
> Cheers, Lars
>
> (as far as i have read and tested a bit, deepcopy allows more to be copied
> then pickle, but cPickle is faster, so i have my own deepcopy:
>
> def deepcopy(obj):
> try:
> return cPickle.loads(cPickle.dumps(obj, -1))
> except PicklingError:
> return copy.deepcopy(obj)
> )
>
>
> On Thu, Oct 3, 2013 at 5:19 AM, Richard Tew <richard.m.tew at gmail.com>
> wrote:****
>
> I'm a little confused. Are you pickling the tasklets, or copying
> them? These are different things to Python.
>
> As far as I know using copy or deepcopy has never been supported for
> tasklets and therefore channels.
>
> Pickling however, is another matter.
>
> Cheers,
> Richard.
>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless****
>
>
>
>
>
>
> maybe double but i got a mail bounce ..
>
> ---------- Forwarded message ----------****
>
> From: *lars van Gemerden* <lars at rational-it.com
> <mailto:lars at rational-it.com>>
> Date: Thu, Oct 3, 2013 at 6:15 PM
> Subject: Re: [Stackless] deepcopying (pickling) channels
> To: The Stackless Python Mailing List <stackless at stackless.com****
>
> <mailto:stackless at stackless.com>>
>
>
> actually both give an error. The only thing different is the error
> message. with pickle the code would be:
>
> ---------------------------------------------------------------------------
> import stackless, pickle
>
> class Test(object):
> def __init__(self):
> self.channel = stackless.channel()
> def run(self):
> stackless.tasklet(self.sender)()
> stackless.tasklet(self.receiver)()
> stackless.run()
> def sender(self):
> counter = 0
> while counter < 10:
> self.channel.send(counter)
> counter += 1
> def receiver(self):
> counter = 0
> while counter < 12:
> counter = self.channel.receive()
> print counter,
> print "done"
> if __name__ == "__main__":
> t = Test()
> t.run()
> t.channel = None
> s = pickle.dumps(t) #OK
> t = Test()
> t.run()
> s = pickle.dumps(t) #ERROR
>
> -----------------------------------------------------------------------------
> and the error:
>
> pickle.PicklingError: Can't pickle <type 'function'>: it's not found as
> __builtin__.function
>
> otherwise the error behavior is the same as decribed above.
>
> Cheers, Lars
>
> (as far as i have read and tested a bit, deepcopy allows more to be
> copied then pickle, but cPickle is faster, so i have my own deepcopy:
>
> def deepcopy(obj):
> try:
> return cPickle.loads(cPickle.dumps(obj, -1))
> except PicklingError:
> return copy.deepcopy(obj)
> )
>
>
> On Thu, Oct 3, 2013 at 5:19 AM, Richard Tew <richard.m.tew at gmail.com****
>
> <mailto:richard.m.tew at gmail.com>> wrote:
>
> I'm a little confused. Are you pickling the tasklets, or copying
> them? These are different things to Python.
>
> As far as I know using copy or deepcopy has never been supported for
> tasklets and therefore channels.
>
> Pickling however, is another matter.
>
> Cheers,
> Richard.
>
> _______________________________________________
> Stackless mailing list****
>
> Stackless at stackless.com <mailto:Stackless at stackless.com>****
>
>
> http://www.stackless.com/mailman/listinfo/stackless
>
>
>
>
> --
> ====================================
> Lars van Gemerden****
>
> lars at rational-it.com <mailto:lars at rational-it.com>
> +31 6 26 88 55 39 <tel:%2B31%206%2026%2088%2055%2039<%2B31%206%2026%2088%2055%2039>
> >****
>
>
> ====================================
>
>
>
> --
> ====================================
> Lars van Gemerden****
>
> lars at rational-it.com <mailto:lars at rational-it.com>****
>
>
> +31 6 26 88 55 39
> ====================================
>
> ****
>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless****
>
>
> --
> Dipl. Phys. Anselm Kruis science + computing ag
> Senior Solution Architect Ingolstädter Str. 22
> email A.Kruis at science-computing.de 80807 München, Germany
> phone +49 89 356386 874 fax 737 www.science-computing.de
> --
> Vorstandsvorsitzender/Chairman of the board of management:
> Gerd-Lothar Leonhart
> Vorstand/Board of Management:
> Dr. Bernd Finkbeiner, Michael Heinrichs, Dr. Arno Steitz, Dr. Ingrid Zech
> Vorsitzender des Aufsichtsrats/
> Chairman of the Supervisory Board:
> Philippe Miltin
> Sitz/Registered Office: Tuebingen
> Registergericht/Registration Court: Stuttgart
> Registernummer/Commercial Register No.: HRB 382196****
>
>
>
>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless****
>
>
>
> ****
>
> ** **
>
> --
> ====================================
> Lars van Gemerden
> lars at rational-it.com
> +31 6 26 88 55 39
> ==================================== ****
>
>
>
> ****
>
> ** **
>
> --
> ====================================
> Lars van Gemerden
> lars at rational-it.com
> +31 6 26 88 55 39
> ==================================== ****
>
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless
>
--
====================================
Lars van Gemerden
lars at rational-it.com
+31 6 26 88 55 39
====================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20131010/78cbb140/attachment-0001.html>
More information about the Stackless
mailing list