Cannot Pickle DisplayString & other objects that use TextualConvention

[MGough]

It's entirely possible that what I'm trying to achieve is completely insane, or the wrong approach, if so then I do apologise.

I have tried Pickle but it fails on DisplayString objects.

Is there any way I can serialise and deserialize these objects, to store them in a file, without needing to know which type I have (i.e. DisplayString, TruthValue, etc)?

[etingof]

Can you please try pickling with the this pyasn1 branch and let me know if this helps?

The relevant discussion is here.

[MGough]

I still seem to be having the same issue after changing to the devel-0.4.1 branch of pyasn1.
Can't pickle <class 'DisplayString'>: attribute lookup DisplayString on builtins failed
as well as
Can't pickle <class 'SnmpAdminString'>: attribute lookup SnmpAdminString on builtins failed

For some context, I'm pickling those objects when they are returned by MibInstrumController.readNextVars() or MibInstrumController.readVars()

[MGough]

I was able to recreate this just via this snippet:

toBePickled = DisplayString('Linux i386')
pickle.dumps(toBePickled)

Which produces the same error:
_pickle.PicklingError: Can't pickle <class 'DisplayString'>: attribute lookup DisplayString on builtins failed

I was able to pickle both BitString and NamedValues from pyasn1.

[etingof]

I'd attribute this to pickle weirdness. Here is a quick hack that would hopefully help:

import pickle
from pysnmp.smi import builder

mibBuilder = builder.MibBuilder()

DisplayString, = mibBuilder.importSymbols("SNMPv2-TC", "DisplayString")

# Pretend this `DisplayString` object belongs to some module
DisplayString.__module__ = '__main__'

toBePickled = DisplayString('Linux i386')

pickle.dumps(toBePickled)

May be we should have importSymbols() injecting this __module__ magic...? Probably not.

[MGough]

Thank you for your response. I've since realised that it would probably make more sense to use your ASN.1 library to encode and decode the data as described here as data is transmitted in those formats for SNMP and the only objects I'm interested in (de)serializing are values from PySNMP to be transmitted via SNMP.

I can't seem to work out how this is being done in PySNMP. The problem I am currently having is when trying to deserialize a 'SysUpTime' object. It seems to be encoded fine, however when I try to decode it an exception is thrown. I think I have misunderstood how this should work, but cannot seem to find any encoding/decoding of these objects within PySNMP itself.

from pyasn1.codec.ber import encoder, decoder
toBeSerialized = <Some SysUpTime object previously created containing value>
serialized = encoder.encode(toBeSerialized)
theObject = decoder.decode(serialized, asn1Spec=??)

I have tried not providing the asn1Spec argument but receive an error. Based on your example and the comments in the PyASN.1 code my intuition was that I should provide an instance of a SysUpTime object. For testing purposes I provided toBeSerialized but received an error stating that clone() takes 1 positional argument but 2 were given.

What I'm looking for is a way to take an object that is some derivative of ASN.1, serialize it for storage, and then later deserialize it (without needing to know the original class of the object if possible).

Can you help me find a good approach to serialize and deserialize instances of these objects?

[etingof]

I think you are on the right track here! Yes, you need to pass asn1Spec=YourSysUptimeClassInstance() to the decoder to make sure it decodes it precisely.

Can you please either paste a minimal reproducer or reveal the traceback? Your pseudocode looks good to me (except this minor nitpick theObject, _ = decoder.decode(...)).

[MGough]

Thanks for the reassurance! I've found that I'm not having this issue for other objects, it seems to work for DisplayString.
Here's an unconventional reproducer:

from pyasn1.codec.ber import encoder, decoder
from pysnmp.entity import engine, config
from pysnmp.entity.rfc3413 import cmdrsp, context
from pysnmp.carrier.asyncore.dgram import udp
from pysnmp.smi import instrum
from pysnmp.proto.api import v2c

snmpEngine = engine.SnmpEngine()


config.addTransport(
    snmpEngine,
    udp.domainName,
    udp.UdpTransport().openServerMode(('127.0.0.1', 1061))
)

config.addV3User(
    snmpEngine, 'usr-none-none'
)

config.addVacmUser(snmpEngine, 3, 'usr-none-none', 'noAuthNoPriv', (1, 3, 6, 1, 2, 1), (1, 3, 6, 1, 2, 1))

snmpContext = context.SnmpContext(snmpEngine)


class SomeInstrum(instrum.AbstractMibInstrumController):

    def __init__(self, mibinstrum_controller):
        self.default_controller = mibinstrum_controller

    def readVars(self, varBinds, acInfo=(None, None)):
        standard_results = self.default_controller.readVars(varBinds)

        result = standard_results[0]
        encoded = encoder.encode(result[1])
        decoded_object, _ = decoder.decode(encoded, asn1Spec=result[1])

        return standard_results


instrumController = snmpContext.getMibInstrum()

snmpContext.registerContextName(
    v2c.OctetString('my-context'),
    SomeInstrum(instrumController)
)


cmdrsp.GetCommandResponder(snmpEngine, snmpContext)
cmdrsp.SetCommandResponder(snmpEngine, snmpContext)

snmpEngine.transportDispatcher.jobStarted(1)

try:
    snmpEngine.transportDispatcher.runDispatcher()
except:
    snmpEngine.transportDispatcher.closeDispatcher()
    raise

I have a suspicion that the strange passing and usage of the standard MibInstrum in a different, custom, MibInstrum is causing it. Although the different behaviour for different objects makes me unsure.

[MGough]

The problematic request:
snmpget -v 3 -l noAuthNoPriv -u usr-none-none -n my-context 127.0.0.1:1061 1.3.6.1.2.1.1.3.0

However this works fine:
snmpget -v 3 -l noAuthNoPriv -u usr-none-none -n my-context 127.0.0.1:1061 1.3.6.1.2.1.1.1.0