Failed to restore the mongo dump

[zqdou]

Hi Team, I failed to restore the dump created by replibyte to my test mongodb server. The dump was created and stored in the GCP bucket.
replibyte version: 0.10.0
Mongodb version: 4.4.24

Step1, creating the dump from the mongdb prod env

# replibyte -c mongo_conf.yaml dump create
⠄ 
⠠ 
⠁ [00:00:18] [############################################################################################################################################################] 3.39MiB/3.39MiB (0s)
⠒ [00:00:18] [############################################################################################################################################################] 3.39MiB/3.39MiB (0s)

# replibyte -c mongo_conf.yaml dump list
⠐ 
⠁ 
--------------------+--------+----------------+------------+-----------
 dump-1703686622454 | 572 kB | 16 minutes ago | true       | false 
 dump-1703684341553 | 572 kB | 54 minutes ago | true       | false 

step 2, check the dump in dev env

# replibyte -c mongo.yaml dump list
⠁ 
⠓ 
--------------------+--------+----------------+------------+-----------
 dump-1703686622454 | 572 kB | 24 minutes ago | true       | false 
 dump-1703684341553 | 572 kB | 1 hour ago     | true       | false 

step3, restore the dump

[root@dev-replibyte-0 opt]# export RUST_BACKTRACE=1   
[root@dev-replibyte-0 opt]# replibyte -c mongo.yaml dump restore remote -v dump-1703684341553
PaymentTransaction> 'db.runCommand("ping").ok'
db.runCommand("ping").ok
PaymentTransaction> [2023-12-27T14:31:42Z ERROR replibyte::datastore::s3] Error { code: "NoSuchKey", message: "The specified key does not exist." }
thread '<unnamed>' panicked at 'Custom { kind: Other, error: "failed to list objects from bucket 'test-bucket-data/mongo'" }', replibyte/src/tasks/full_restore.rs:69:29
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:143:14
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'main' panicked at 'RecvError', replibyte/src/tasks/full_restore.rs:79:29
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:143:14
   2: <replibyte::tasks::full_restore::FullRestoreTask<D> as replibyte::tasks::Task>::run
   3: replibyte::commands::dump::restore_remote
   4: replibyte::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

The configuration file mongo.yaml

datastore:
  gcp:
    bucket: test-bucket-data/mongo
    region: asia-southeast1
    access_key: key
    secret: secret
destination:
  connection_uri: mongodb://dump_user:<password>@<dev-mongo>:27017/testdb?authSource=admin

[Tchekroun]

Hello,

I have the same issue using a Scaleway bucket with postgreSQL.
replibyte version: 0.10.0

Here there is my replibyte configuration :

  connection_uri: $STAGING_DATABASE_URL
datastore:
  aws:
    bucket: $REPLIBYTE_PRODUCT_BUCKET_NAME
    region: $REPLIBYTE_S3_REGION
    credentials:
      access_key_id: $REPLIBYTE_ACCESS_KEY_ID
      secret_access_key: $REPLIBYTE_SECRET_ACCESS_KEY
    endpoint:
      custom: $SCALEWAY_BUCKET_URL
destination:
  connection_uri: $NEW_DATABASE_URL
  wipe_database: false

and when i run replibyte -c replibyte.yaml dump restore remote -v latest i got the same error :

ERROR replibyte::datastore::s3] Error parsing XML: invalid root, expected ListBucketResult got StartEl { name: Name { prefix: "", local: "CreateBucketConfiguration" }, attributes: [Attr { name: Name { prefix: "", local: "xmlns" }, value: "http://s3.amazonaws.com/doc/2006-03-01/" }], closed: false, depth: 0 }
thread '<unnamed>' panicked at 'Custom { kind: Other, error: "failed to list objects from bucket 'db-test'" }', replibyte/src/tasks/full_restore.rs:69:29

but when i try to list the dump with replibyte -c replibyte.yaml dump list i can display the list of my dump present in my bucket.

Any idea ?