invalid proof error when executing tx raw chan-open-try

[andynog]

Crate

ibc-relayer-cli, ibc-relayer

Summary of Bug

Cannot execute a tx raw chan-open-try between chains irishub-1 and cosmoshub-4. Getting invalid proof error.

Version

Branch: master
Commit: 1adb18d3d4c70239115bc5d98abeb8d114cf9c15 (May 7th, 2021)

Steps to Reproduce

Running a tx raw chan-open-try

hermes tx raw chan-open-try irishub-1 cosmoshub-4 connection-3 transfer transfer -s channel-103

May 10 18:06:03.141  INFO ibc_relayer_cli::commands: Using default configuration from: '.hermes/config.toml'
May 10 18:06:03.152  INFO ibc_relayer::event::monitor: starting event monitor chain.id=cosmoshub-4
May 10 18:06:03.162  INFO ibc_relayer::event::monitor: starting event monitor chain.id=irishub-1
May 10 18:06:03.164  INFO ibc_relayer_cli::commands::tx::channel: Message ChanOpenTry: Channel { ordering: Unordered, a_side: ChannelSide { chain: ProdChainHandle { chain_id: ChainId { id: "cosmoshub-4", version: 4 }, runtime_sender: Sender { .. } }, client_id: ClientId("07-tendermint-0"), connection_id: ConnectionId("connection-0"), port_id: PortId("transfer"), channel_id: ChannelId("channel-103") }, b_side: ChannelSide { chain: ProdChainHandle { chain_id: ChainId { id: "irishub-1", version: 1 }, runtime_sender: Sender { .. } }, client_id: ClientId("07-tendermint-4"), connection_id: ConnectionId("connection-3"), port_id: PortId("transfer"), channel_id: ChannelId("channel-0") }, connection_delay: 0ns, version: None }
May 10 18:06:04.730  WARN ibc_relayer::event::rpc: error while building event send_packet.packet_data
Error: tx error: failed with underlying cause: tx response error: deliver_tx reports error: log=Log("failed to execute message; message index: 1: channel handshake open try failed: failed channel state verification for client (07-tendermint-4): chained membership proof failed to verify membership of value: 080110011A0A0A087472616E73666572220E636F6E6E656374696F6E2D3230312A0769637332302D31 in subroot C61F007B389FC61D553C5D88B5454A1FB548A3A379868654A7E18BE7230CD356 at index 0. Please ensure the path and value are both correct.: invalid proof")

One thing I've noticed on the error above is that the information for client id and connection id for the destination chain doesn't seem right:

client_id: ClientId("07-tendermint-0"), connection_id: ConnectionId("connection-0"), port_id: PortId("transfer"), channel_id: ChannelId("channel-103") }

I should be client 07-tendermint-157 and connection connection-201 I think

If I query the channel, I get the right connection id:

hermes query channel end cosmoshub-4 transfer channel-103
May 10 18:18:39.822  INFO ibc_relayer_cli::commands: Using default configuration from: '.hermes/config.toml'
Success: ChannelEnd {
    state: Init,
    ordering: Unordered,
    remote: Counterparty {
        port_id: PortId(
            "tranfer",
        ),
        channel_id: None,
    },
    connection_hops: [
        ConnectionId(
            "connection-201",
        ),
    ],
    version: "ics20-1",
}

and the client id associated with the connection id:

hermes query connection end cosmoshub-4 connection-201
May 10 18:19:29.415  INFO ibc_relayer_cli::commands: Using default configuration from: '.hermes/config.toml'
Success: ConnectionEnd {
    state: Open,
    client_id: ClientId(
        "07-tendermint-157",
    ),
    counterparty: Counterparty {
        client_id: ClientId(
            "07-tendermint-4",
        ),
        connection_id: Some(
            ConnectionId(
                "connection-3",
            ),
        ),
        prefix: ibc,
    },
    versions: [
        Version {
            identifier: "1",
            features: [
                "ORDER_ORDERED",
                "ORDER_UNORDERED",
            ],
        },
    ],
    delay_period: 0ns,
}

Also, querying the connection-3 on irishub-1 I get:

hermes query connection end irishub-1 connection-3
May 10 18:23:33.976  INFO ibc_relayer_cli::commands: Using default configuration from: '.hermes/config.toml'
Success: ConnectionEnd {
    state: Open,
    client_id: ClientId(
        "07-tendermint-4",
    ),
    counterparty: Counterparty {
        client_id: ClientId(
            "07-tendermint-157",
        ),
        connection_id: Some(
            ConnectionId(
                "connection-201",
            ),
        ),
        prefix: ibc,
    },
    versions: [
        Version {
            identifier: "1",
            features: [
                "ORDER_ORDERED",
                "ORDER_UNORDERED",
            ],
        },
    ],
    delay_period: 0ns,
}

I've also tried to do an update-client to both chains and then execute the chan-open-try transaction but didn't help.

Acceptance Criteria

Been able to execute the chan-open-try transaction without errors.

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate milestone (priority) applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[ancazamfir]

One thing I've noticed on the error above is that the information for client id and connection id for the destination chain doesn't seem right:

are you referring to this INFO?:
May 10 18:06:03.164 INFO ibc_relayer_cli::commands::tx::channel: Message ChanOpenTry: Channel { ordering: Unordered, a_side: ChannelSide { chain: ProdChainHandle { chain_id: ChainId { id: "cosmoshub-4", version: 4 }, runtime_sender: Sender { .. } }, client_id: ClientId("07-tendermint-0"), connection_id: ConnectionId("connection-0"), port_id: PortId("transfer"), channel_id: ChannelId("channel-103") }, b_side: ChannelSide { chain: ProdChainHandle { chain_id: ChainId { id: "irishub-1", version: 1 }, runtime_sender: Sender { .. } }, client_id: ClientId("07-tendermint-4"), connection_id: ConnectionId("connection-3"), port_id: PortId("transfer"), channel_id: ChannelId("channel-0") }, connection_delay: 0ns, version: None }

That output will be fixed in next release. Only the entered parameters have valid values in that INFO message, the rest are currently defaulted to the 0-IDs (e.g. 07-tendermint-0, connection-0, etc)

Otherwise not sure what the cause of the error is. It says that the proof included in openTry doesn't verify, e.g. this would happen if channel-103 was not in Init state at the time the proof was taken. There could be also incompatibilities between the SDK versions (changes in datastructures, proofs, etc).

What is the version of irishub-1? Are there any channels established with the hub there?

[andynog]

Here's some answers:

irishub-1

name: iris
server_name: iris
version: 1.1.0
commit: 8604b5ebb2d6e3e34e4543ea7f4e1b8d1e7a4b58
build_tags: netgo,ledger
go: go version go1.16.3 linux/amd64
build_deps:
- github.com/99designs/keyring@v1.1.6
- github.com/ChainSafe/go-schnorrkel@v0.0.0-20200405005733-88cbf1b4c40d
- github.com/Workiva/go-datastructures@v1.0.52
- github.com/armon/go-metrics@v0.3.6
- github.com/beorn7/perks@v1.0.1
- github.com/bgentry/speakeasy@v0.1.0
- github.com/btcsuite/btcd@v0.21.0-beta
- github.com/cespare/xxhash/v2@v2.1.1
- github.com/confio/ics23/go@v0.6.3
- github.com/cosmos/cosmos-sdk@v0.42.3
-...

Channels Query

hermes query channels irishub-1
Success: [
    PortChannelId {
        channel_id: ChannelId(
            "channel-0",
        ),
        port_id: PortId(
            "transfer",
        ),
    },
]

Channel End Query

hermes query channel end irishub-1 transfer channel-0
Success: ChannelEnd {
    state: Open,
    ordering: Unordered,
    remote: Counterparty {
        port_id: PortId(
            "transfer",
        ),
        channel_id: Some(
            ChannelId(
                "channel-91",
            ),
        ),
    },
    connection_hops: [
        ConnectionId(
            "connection-1",
        ),
    ],
    version: "ics20-1",
}

Connection End Query

ConnectionEnd {
    state: Open,
    client_id: ClientId(
        "07-tendermint-1",
    ),
    counterparty: Counterparty {
        client_id: ClientId(
            "07-tendermint-141",
        ),
        connection_id: Some(
            ConnectionId(
                "connection-187",
            ),
        ),
        prefix: ibc,
    },
    versions: [
        Version {
            identifier: "1",
            features: [
                "ORDER_ORDERED",
                "ORDER_UNORDERED",
            ],
        },
    ],
    delay_period: 0ns,
}

Client State Query

hermes query client state irishub-1 07-tendermint-1
Success: ClientState {
    chain_id: ChainId {
        id: "cosmoshub-4",
        version: 4,
    },
    trust_level: TrustThresholdFraction {
        numerator: 1,
        denominator: 3,
    },
    trusting_period: 1209600s,
    unbonding_period: 1814400s,
    max_clock_drift: 600s,
    frozen_height: Height {
        revision: 0,
        height: 0,
    },
    latest_height: Height {
        revision: 4,
        height: 6159401,
    },
    upgrade_path: [
        "upgrade",
        "upgradedIBCState",
    ],
    allow_update: AllowUpdate {
        after_expiry: true,
        after_misbehaviour: true,
    },
}

[ancazamfir]

What is the status of the fullnode you are connecting to on irishub?

[andynog]

What is the status of the fullnode you are connecting to on irishub?

I'm assuming you're asking for this info (/status endpoint)

{
  "jsonrpc": "2.0",
  "id": -1,
  "result": {
    "node_info": {
      "protocol_version": {
        "p2p": "8",
        "block": "11",
        "app": "0"
      },
      "id": "bcba51f327b52f75ae58ab9c5d7b011357345faa",
      "listen_addr": "tcp://0.0.0.0:26656",
      "network": "irishub-1",
      "version": "",
      "channels": "40202122233038606100",
      "moniker": "node-lnx-01-irishub-1",
      "other": {
        "tx_index": "on",
        "rpc_address": "tcp://0.0.0.0:26657"
      }
    },
    "sync_info": {
      "latest_block_hash": "B7E4CDED0B209D0F1F6E7F233F47FD020086710EBF9C14AFBD522F0BA95DCDE6",
      "latest_app_hash": "7AA6E8F0E249649DECF79A9115A83F5665B16677ACF5E4E99F7AC3BB0CC648EB",
      "latest_block_height": "10134833",
      "latest_block_time": "2021-05-11T19:42:40.780147414Z",
      "earliest_block_hash": "D71C602D0BBB9C080AC01C244B328183DF185842EB5798913F1020B786319B44",
      "earliest_app_hash": "E3B0C44298FC1C149AFBF4C8996FB92427AE41E4649B934CA495991B7852B855",
      "earliest_block_height": "9146456",
      "earliest_block_time": "2019-03-01T05:30:00Z",
      "catching_up": false
    },
    "validator_info": {
      "address": "1641EC62C52AB07E6DDCF93FBDD301CA5D329284",
      "pub_key": {
        "type": "tendermint/PubKeyEd25519",
        "value": "brezgJYNzMDA5aOntbXutfsuKT0QXWim+6KJswAErBo="
      },
      "voting_power": "0"
    }
  }
}

[ancazamfir]

The channel on cosmoshub-4 is created with counterparty port tranfer instead of transfer

[andynog]

Yes, that was the problem, thanks @ancazamfir

Here's the result's of a channel opened on both sides:

hermes query channel end cosmoshub-4 transfer channel-112
May 12 14:12:59.884  INFO ibc_relayer_cli::commands: Using default configuration from: '.hermes/config.toml'
Success: ChannelEnd {
    state: Open,
    ordering: Unordered,
    remote: Counterparty {
        port_id: PortId(
            "transfer",
        ),
        channel_id: Some(
            ChannelId(
                "channel-2",
            ),
        ),
    },
    connection_hops: [
        ConnectionId(
            "connection-201",
        ),
    ],
    version: "ics20-1",
}

hermes query channel end irishub-1 transfer channel-2
May 12 14:13:53.795  INFO ibc_relayer_cli::commands: Using default configuration from: '.hermes/config.toml'
Success: ChannelEnd {
    state: Open,
    ordering: Unordered,
    remote: Counterparty {
        port_id: PortId(
            "transfer",
        ),
        channel_id: Some(
            ChannelId(
                "channel-112",
            ),
        ),
    },
    connection_hops: [
        ConnectionId(
            "connection-3",
        ),
    ],
    version: "ics20-1",
}

[andynog]

I think this ticket might be closed. Not sure if there's anything that can be done to address the issue of specifying an invalid port on the destination chain.