[Stackless] Fwd: deepcopying (pickling) channels

Kristján Valur Jónsson kristjan at ccpgames.com
Thu Oct 10 15:22:20 CEST 2013


The error is weird.  It seems to be trying to pickle the "function" type.  It shouldn't try doing that.
Now, I know that pickling channels with tasklets on them works.  This is what stackless does.
I tried saving your code into a module and just running it from the command line.
I wonder if there is some copy_reg magic that is interfering with the code.  How are you running it?  Is this when running the code as a module, or when typing it in from the console?
Are you trying this in a stand-alone environment or in some application framework?

Finally, can I see the full traceback for the pickle exception.  If you are using pickle (and not cPickle) a traceback can help us find out why it is failing.

K

From: stackless-bounces at stackless.com [mailto:stackless-bounces at stackless.com] On Behalf Of lars van Gemerden
Sent: 9. október 2013 12:48
To: The Stackless Python Mailing List
Subject: Re: [Stackless] Fwd: deepcopying (pickling) channels

Hi Kristjan,

the version i am using is: python 2.7.5 Stackless 3.1b3 (from binary).

I think the stackless mailsystem was a little off, i tried with pickle.dumps (see mail history below), with the error: pickle.PicklingError: Can't pickle <type 'function'>: it's not found as __builtin__.function.

"stackless.run() returns when there are no runnable tasklets ...", ok, clear, thanks.

Cheers, Lars

On Wed, Oct 9, 2013 at 2:36 PM, lars van Gemerden <lars at rational-it.com<mailto:lars at rational-it.com>> wrote:
I am reasonably familiar with pickle (e.g. i know you cant pickle most functions), but this is pretty stackless specific: why can i pickle an empty channel, but not a sending/receiving channel?

Cheers, Lars

On Mon, Oct 7, 2013 at 7:50 PM, Anselm Kruis <a.kruis at science-computing.de<mailto:a.kruis at science-computing.de>> wrote:
Hi Lars,

you are observing well known deficiencies of the pickle/cPickle implementation of Python. Perhaps sPickle (http://pypi.python.org/pypi/sPickle) can pickle the channels.

Regards
  Anselm


Am 06.10.2013 08<tel:06.10.2013%2008>:58, schrieb lars van Gemerden:
maybe double but i got a mail bounce ..

---------- Forwarded message ----------
From: lars van Gemerden <lars at rational-it.com<mailto:lars at rational-it.com>>
Date: Thu, Oct 3, 2013 at 6:15 PM
Subject: Re: [Stackless] deepcopying (pickling) channels
To: The Stackless Python Mailing List <stackless at stackless.com<mailto:stackless at stackless.com>>


actually both give an error. The only thing different is the error message.
with pickle the code would be:

---------------------------------------------------------------------------
import stackless, pickle

class Test(object):
     def __init__(self):
         self.channel = stackless.channel()

     def run(self):
         stackless.tasklet(self.sender)()
         stackless.tasklet(self.receiver)()
         stackless.run()

     def sender(self):
         counter = 0
         while counter < 10:
             self.channel.send(counter)
             counter += 1

     def receiver(self):
         counter = 0
         while counter < 12:
             counter = self.channel.receive()
             print counter,
         print "done"

if __name__ == "__main__":

     t = Test()
     t.run()
     t.channel = None
     s = pickle.dumps(t) #OK

     t = Test()
     t.run()
     s = pickle.dumps(t) #ERROR
-----------------------------------------------------------------------------
and the error:

pickle.PicklingError: Can't pickle <type 'function'>: it's not found as
__builtin__.function

otherwise the error behavior is the same as decribed above.

Cheers, Lars

(as far as i have read and tested a bit, deepcopy allows more to be copied
then pickle, but cPickle is faster, so i have my own deepcopy:

def deepcopy(obj):
     try:
         return cPickle.loads(cPickle.dumps(obj, -1))
     except PicklingError:
         return copy.deepcopy(obj)
)


On Thu, Oct 3, 2013 at 5:19 AM, Richard Tew <richard.m.tew at gmail.com<mailto:richard.m.tew at gmail.com>> wrote:
I'm a little confused.  Are you pickling the tasklets, or copying
them?  These are different things to Python.

As far as I know using copy or deepcopy has never been supported for
tasklets and therefore channels.

Pickling however, is another matter.

Cheers,
Richard.

_______________________________________________
Stackless mailing list
Stackless at stackless.com<mailto:Stackless at stackless.com>
http://www.stackless.com/mailman/listinfo/stackless





maybe double but i got a mail bounce ..

---------- Forwarded message ----------
From: *lars van Gemerden* <lars at rational-it.com<mailto:lars at rational-it.com>
<mailto:lars at rational-it.com<mailto:lars at rational-it.com>>>
Date: Thu, Oct 3, 2013 at 6:15 PM
Subject: Re: [Stackless] deepcopying (pickling) channels
To: The Stackless Python Mailing List <stackless at stackless.com<mailto:stackless at stackless.com>
<mailto:stackless at stackless.com<mailto:stackless at stackless.com>>>


actually both give an error. The only thing different is the error
message. with pickle the code would be:

---------------------------------------------------------------------------
import stackless, pickle

class Test(object):
     def __init__(self):
         self.channel = stackless.channel()
     def run(self):
         stackless.tasklet(self.sender)()
         stackless.tasklet(self.receiver)()
         stackless.run()
     def sender(self):
         counter = 0
         while counter < 10:
             self.channel.send(counter)
             counter += 1
     def receiver(self):
         counter = 0
         while counter < 12:
             counter = self.channel.receive()
             print counter,
         print "done"
if __name__ == "__main__":
     t = Test()
     t.run()
     t.channel = None
     s = pickle.dumps(t) #OK
     t = Test()
     t.run()
     s = pickle.dumps(t) #ERROR
-----------------------------------------------------------------------------
and the error:

pickle.PicklingError: Can't pickle <type 'function'>: it's not found as
__builtin__.function

otherwise the error behavior is the same as decribed above.

Cheers, Lars

(as far as i have read and tested a bit, deepcopy allows more to be
copied then pickle, but cPickle is faster, so i have my own deepcopy:

def deepcopy(obj):
     try:
         return cPickle.loads(cPickle.dumps(obj, -1))
     except PicklingError:
         return copy.deepcopy(obj)
)


On Thu, Oct 3, 2013 at 5:19 AM, Richard Tew <richard.m.tew at gmail.com<mailto:richard.m.tew at gmail.com>
<mailto:richard.m.tew at gmail.com<mailto:richard.m.tew at gmail.com>>> wrote:

    I'm a little confused.  Are you pickling the tasklets, or copying
    them?  These are different things to Python.

    As far as I know using copy or deepcopy has never been supported for
    tasklets and therefore channels.

    Pickling however, is another matter.

    Cheers,
    Richard.

    _______________________________________________
    Stackless mailing list
    Stackless at stackless.com<mailto:Stackless at stackless.com> <mailto:Stackless at stackless.com<mailto:Stackless at stackless.com>>

    http://www.stackless.com/mailman/listinfo/stackless




--
====================================
Lars van Gemerden
lars at rational-it.com<mailto:lars at rational-it.com> <mailto:lars at rational-it.com<mailto:lars at rational-it.com>>
+31 6 26 88 55 39<tel:%2B31%206%2026%2088%2055%2039> <tel:%2B31%206%2026%2088%2055%2039>

====================================



--
====================================
Lars van Gemerden
lars at rational-it.com<mailto:lars at rational-it.com> <mailto:lars at rational-it.com<mailto:lars at rational-it.com>>

+31 6 26 88 55 39<tel:%2B31%206%2026%2088%2055%2039>
====================================

_______________________________________________
Stackless mailing list
Stackless at stackless.com<mailto:Stackless at stackless.com>
http://www.stackless.com/mailman/listinfo/stackless

--
 Dipl. Phys. Anselm Kruis                       science + computing ag
 Senior Solution Architect                      Ingolstädter Str. 22
 email A.Kruis at science-computing.de<mailto:A.Kruis at science-computing.de>             80807 München, Germany
 phone +49 89 356386 874<tel:%2B49%2089%20356386%20874>  fax 737               www.science-computing.de<http://www.science-computing.de>
--
Vorstandsvorsitzender/Chairman of the board of management:
Gerd-Lothar Leonhart
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Michael Heinrichs, Dr. Arno Steitz, Dr. Ingrid Zech
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196



_______________________________________________
Stackless mailing list
Stackless at stackless.com<mailto:Stackless at stackless.com>
http://www.stackless.com/mailman/listinfo/stackless



--
====================================
Lars van Gemerden
lars at rational-it.com<mailto:lars at rational-it.com>
+31 6 26 88 55 39<tel:%2B31%206%2026%2088%2055%2039>
====================================



--
====================================
Lars van Gemerden
lars at rational-it.com<mailto:lars at rational-it.com>
+31 6 26 88 55 39
====================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.stackless.com/pipermail/stackless/attachments/20131010/791d31d6/attachment-0001.html>


More information about the Stackless mailing list