Data Connect emulator with "--import" doesn't work

[hatoya]

[REQUIRED] Environment info

firebase-tools: 13.28.0

Platform: macOS

[REQUIRED] Test case

[REQUIRED] Steps to reproduce

1. Run firebase emulators:start --only dataconnect
2. Run firebase emulators:export ./saved_data
3. Stop emulator
4. Run firebase emulators:start --import ./saved_data --only dataconnect

[REQUIRED] Expected behavior

The emulator starts without any problems.

[REQUIRED] Actual behavior

firebase-debug.log

[debug] [2024-12-09T09:06:16.394Z] ----------------------------------------------------------------------
[debug] [2024-12-09T09:06:16.396Z] Command:       /Users/hatoya/.volta/tools/image/node/20.18.0/bin/node /Users/hatoya/Desktop/momiji/node_modules/.bin/firebase emulators:start --import ./saved_data --only auth,dataconnect,storage,functions --project default
[debug] [2024-12-09T09:06:16.396Z] CLI Version:   13.28.0
[debug] [2024-12-09T09:06:16.396Z] Platform:      darwin
[debug] [2024-12-09T09:06:16.396Z] Node Version:  v20.18.0
[debug] [2024-12-09T09:06:16.396Z] Time:          Mon Dec 09 2024 18:06:16 GMT+0900 (日本標準時)
[debug] [2024-12-09T09:06:16.396Z] ----------------------------------------------------------------------
[debug] 
[debug] [2024-12-09T09:06:16.467Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[debug] [2024-12-09T09:06:16.467Z] > authorizing via signed-in user (hayato.wada@lpconsulting.jp)
[debug] [2024-12-09T09:06:16.680Z] java version "11.0.15.1" 2022-04-22 LTS
[debug] [2024-12-09T09:06:16.680Z] 
Java(TM) SE Runtime Environment 18.9 (build 11.0.15.1+2-LTS-10)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.15.1+2-LTS-10, mixed mode)

[debug] [2024-12-09T09:06:16.686Z] Parsed Java major version: 11
[info] i  emulators: Starting emulators: auth, functions, storage, dataconnect {"metadata":{"emulator":{"name":"hub"},"message":"Starting emulators: auth, functions, storage, dataconnect"}}
[debug] [2024-12-09T09:06:17.641Z] [logging] Logging Emulator only supports listening on one address (127.0.0.1). Not listening on ::1
[debug] [2024-12-09T09:06:17.642Z] [auth] Authentication Emulator only supports listening on one address (127.0.0.1). Not listening on ::1
[debug] [2024-12-09T09:06:17.642Z] [storage] Storage Emulator only supports listening on one address (127.0.0.1). Not listening on ::1
[debug] [2024-12-09T09:06:17.642Z] [dataconnect.postgres] postgres server for dataconnect only supports listening on one address (127.0.0.1). Not listening on ::1
[debug] [2024-12-09T09:06:17.642Z] assigned listening specs for emulators {"user":{"hub":[{"address":"127.0.0.1","family":"IPv4","port":4400},{"address":"::1","family":"IPv6","port":4400}],"ui":[{"address":"127.0.0.1","family":"IPv4","port":4000},{"address":"::1","family":"IPv6","port":4000}],"logging":[{"address":"127.0.0.1","family":"IPv4","port":4500}],"auth":[{"address":"127.0.0.1","family":"IPv4","port":9099}],"storage":[{"address":"127.0.0.1","family":"IPv4","port":9199}],"dataconnect":[{"address":"127.0.0.1","family":"IPv4","port":9399},{"address":"::1","family":"IPv6","port":9399}],"dataconnect.postgres":[{"address":"127.0.0.1","family":"IPv4","port":5432}]},"metadata":{"message":"assigned listening specs for emulators"}}
[warn] ⚠  emulators: It seems that you are running multiple instances of the emulator suite for project momiji-b5434. This may result in unexpected behavior. 
[debug] [2024-12-09T09:06:17.644Z] [hub] writing locator at /var/folders/ld/x6lzhgm52v52k_lwbq1k5j040000gn/T/hub-momiji-b5434.json
[debug] [2024-12-09T09:06:18.093Z] [functions] Functions Emulator only supports listening on one address (127.0.0.1). Not listening on ::1
[debug] [2024-12-09T09:06:18.093Z] [eventarc] Eventarc Emulator only supports listening on one address (127.0.0.1). Not listening on ::1
[debug] [2024-12-09T09:06:18.093Z] [tasks] Cloud Tasks Emulator only supports listening on one address (127.0.0.1). Not listening on ::1
[debug] [2024-12-09T09:06:18.093Z] late-assigned ports for functions and eventarc emulators {"user":{"hub":[{"address":"127.0.0.1","family":"IPv4","port":4400},{"address":"::1","family":"IPv6","port":4400}],"ui":[{"address":"127.0.0.1","family":"IPv4","port":4000},{"address":"::1","family":"IPv6","port":4000}],"logging":[{"address":"127.0.0.1","family":"IPv4","port":4500}],"auth":[{"address":"127.0.0.1","family":"IPv4","port":9099}],"storage":[{"address":"127.0.0.1","family":"IPv4","port":9199}],"dataconnect":[{"address":"127.0.0.1","family":"IPv4","port":9399},{"address":"::1","family":"IPv6","port":9399}],"dataconnect.postgres":[{"address":"127.0.0.1","family":"IPv4","port":5432}],"functions":[{"address":"127.0.0.1","family":"IPv4","port":5001}],"eventarc":[{"address":"127.0.0.1","family":"IPv4","port":9299}],"tasks":[{"address":"127.0.0.1","family":"IPv4","port":9499}]},"metadata":{"message":"late-assigned ports for functions and eventarc emulators"}}
[warn] ⚠  functions: The following emulators are not running, calls to these services from the Functions emulator will affect production: apphosting, firestore, database, hosting, pubsub {"metadata":{"emulator":{"name":"functions"},"message":"The following emulators are not running, calls to these services from the Functions emulator will affect production: \u001b[1mapphosting, firestore, database, hosting, pubsub\u001b[22m"}}
[debug] [2024-12-09T09:06:18.095Z] defaultcredentials: writing to file /Users/hatoya/.config/firebase/hayato_wada_lpconsulting.jp_application_default_credentials.json
[debug] [2024-12-09T09:06:18.096Z] Setting GAC to /Users/hatoya/.config/firebase/hayato_wada_lpconsulting.jp_application_default_credentials.json {"metadata":{"emulator":{"name":"functions"},"message":"Setting GAC to /Users/hatoya/.config/firebase/hayato_wada_lpconsulting.jp_application_default_credentials.json"}}
[debug] [2024-12-09T09:06:18.096Z] Checked if tokens are valid: true, expires at: 1733737900424
[debug] [2024-12-09T09:06:18.096Z] Checked if tokens are valid: true, expires at: 1733737900424
[debug] [2024-12-09T09:06:18.097Z] >>> [apiv2][query] GET https://firebase.googleapis.com/v1beta1/projects/momiji-b5434/adminSdkConfig [none]
[debug] [2024-12-09T09:06:18.432Z] <<< [apiv2][status] GET https://firebase.googleapis.com/v1beta1/projects/momiji-b5434/adminSdkConfig 403
[debug] [2024-12-09T09:06:18.432Z] <<< [apiv2][body] GET https://firebase.googleapis.com/v1beta1/projects/momiji-b5434/adminSdkConfig {"error":{"code":403,"message":"The caller does not have permission","status":"PERMISSION_DENIED"}}
[debug] [2024-12-09T09:06:18.433Z] Failed to get Admin SDK config for momiji-b5434, falling back to cache Failed to get Admin SDK for Firebase project momiji-b5434. Please make sure the project exists and your account has permission to access it. {"name":"FirebaseError","children":[],"exit":2,"message":"Failed to get Admin SDK for Firebase project momiji-b5434. Please make sure the project exists and your account has permission to access it.","original":{"name":"FirebaseError","children":[],"context":{"body":{"error":{"code":403,"message":"The caller does not have permission","status":"PERMISSION_DENIED"}},"response":{"statusCode":403}},"exit":1,"message":"Request to https://firebase.googleapis.com/v1beta1/projects/momiji-b5434/adminSdkConfig had HTTP Error: 403, The caller does not have permission","status":403},"status":500}
[info] i  auth: Importing config from /Users/hatoya/Desktop/momiji/saved_data/auth_export/config.json {"metadata":{"emulator":{"name":"auth"},"message":"Importing config from /Users/hatoya/Desktop/momiji/saved_data/auth_export/config.json"}}
[info] i  auth: Importing accounts from /Users/hatoya/Desktop/momiji/saved_data/auth_export/accounts.json {"metadata":{"emulator":{"name":"auth"},"message":"Importing accounts from /Users/hatoya/Desktop/momiji/saved_data/auth_export/accounts.json"}}
[info] i  dataconnect: Importing data from /Users/hatoya/Desktop/momiji/saved_data/dataconnect_export {"metadata":{"emulator":{"name":"dataconnect"},"message":"Importing data from /Users/hatoya/Desktop/momiji/saved_data/dataconnect_export"}}
[debug] [2024-12-09T09:06:21.487Z] I1209 18:06:21.442187   90025 control.go:45] [engine.Service /emulator/data-connect 359a] UpdateSchema(): done sources: schema/schema.gql [2941B] 
I1209 18:06:21.484514   90025 control.go:73] [engine.Service /emulator/data-connect 359a] UpdateConnector(default): done sources: connector/mutations.gql [16185B] connector/queries.gql [3655B] 
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.442187   90025 control.go:45] [engine.Service /emulator/data-connect 359a] UpdateSchema(): done sources: schema/schema.gql [2941B] \nI1209 18:06:21.484514   90025 control.go:73] [engine.Service /emulator/data-connect 359a] UpdateConnector(default): done sources: connector/mutations.gql [16185B] connector/queries.gql [3655B] \n"}}
[debug] [2024-12-09T09:06:21.488Z] Ignoring unsupported arg: auto_download {"metadata":{"emulator":{"name":"dataconnect"},"message":"Ignoring unsupported arg: auto_download"}}
[debug] [2024-12-09T09:06:21.488Z] Ignoring unsupported arg: port {"metadata":{"emulator":{"name":"dataconnect"},"message":"Ignoring unsupported arg: port"}}
[debug] [2024-12-09T09:06:21.488Z] Starting Data Connect Emulator with command {"binary":"/Users/hatoya/.cache/firebase/emulators/dataconnect-emulator-1.7.4","args":["--logtostderr","-v=2","dev","--listen=127.0.0.1:9399,[::1]:9399","--config_dir=/Users/hatoya/Desktop/momiji/dataconnect","--enable_output_schema_extensions=true","--enable_output_generated_sdk=true"],"optionalArgs":["listen","config_dir","enable_output_schema_extensions","enable_output_generated_sdk"],"joinArgs":true,"shell":false,"port":9399} {"metadata":{"emulator":{"name":"dataconnect"},"message":"Starting Data Connect Emulator with command {\"binary\":\"/Users/hatoya/.cache/firebase/emulators/dataconnect-emulator-1.7.4\",\"args\":[\"--logtostderr\",\"-v=2\",\"dev\",\"--listen=127.0.0.1:9399,[::1]:9399\",\"--config_dir=/Users/hatoya/Desktop/momiji/dataconnect\",\"--enable_output_schema_extensions=true\",\"--enable_output_generated_sdk=true\"],\"optionalArgs\":[\"listen\",\"config_dir\",\"enable_output_schema_extensions\",\"enable_output_generated_sdk\"],\"joinArgs\":true,\"shell\":false,\"port\":9399}"}}
[info] i  dataconnect: Data Connect Emulator logging to dataconnect-debug.log {"metadata":{"emulator":{"name":"dataconnect"},"message":"Data Connect Emulator logging to \u001b[1mdataconnect-debug.log\u001b[22m"}}
[info] i  dataconnect: Started up Postgres server, listening on {"address":"127.0.0.1","family":"IPv4","port":5432} {"metadata":{"emulator":{"name":"dataconnect"},"message":"Started up Postgres server, listening on {\"address\":\"127.0.0.1\",\"family\":\"IPv4\",\"port\":5432}"}}
[debug] [2024-12-09T09:06:21.498Z] [Data Connect] Connecting to postgres://127.0.0.1:5432/database?sslmode=disable}...
[debug] [2024-12-09T09:06:21.498Z] >>> [apiv2][query] POST http://127.0.0.1:9399/emulator/configure [none]
[debug] [2024-12-09T09:06:21.498Z] >>> [apiv2][body] POST http://127.0.0.1:9399/emulator/configure {"connectionString":"postgres://127.0.0.1:5432/database?sslmode=disable","database":"database","serviceId":"data-connect","maxOpenConnections":1}
[debug] [2024-12-09T09:06:21.499Z] *** [apiv2] error from fetch(http://127.0.0.1:9399/emulator/configure, {"headers":{},"method":"POST","body":"{\"connectionString\":\"postgres://127.0.0.1:5432/database?sslmode=disable\",\"database\":\"database\",\"serviceId\":\"data-connect\",\"maxOpenConnections\":1}"}): FetchError: request to http://127.0.0.1:9399/emulator/configure failed, reason: connect ECONNREFUSED 127.0.0.1:9399
[debug] [2024-12-09T09:06:21.499Z] [Data Connect] Retrying connectToPostgress call (1 of 3 attempts): FirebaseError: Data Connect emulator: undefined
[debug] [2024-12-09T09:06:21.641Z] I1209 18:06:21.639620   90030 load.go:37] Reloading schema and connectors...
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.639620   90030 load.go:37] Reloading schema and connectors...\n"}}
[debug] [2024-12-09T09:06:21.677Z] I1209 18:06:21.677771   90030 control.go:45] [engine.Service /emulator/data-connect bd10] UpdateSchema(): done sources: schema/schema.gql [2941B] 
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.677771   90030 control.go:45] [engine.Service /emulator/data-connect bd10] UpdateSchema(): done sources: schema/schema.gql [2941B] \n"}}
[debug] [2024-12-09T09:06:21.717Z] I1209 18:06:21.717490   90030 control.go:73] [engine.Service /emulator/data-connect bd10] UpdateConnector(default): done sources: connector/mutations.gql [16185B] connector/queries.gql [3655B] 
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.717490   90030 control.go:73] [engine.Service /emulator/data-connect bd10] UpdateConnector(default): done sources: connector/mutations.gql [16185B] connector/queries.gql [3655B] \n"}}
[debug] [2024-12-09T09:06:21.756Z] I1209 18:06:21.756596   90030 codegen.go:82] [connector "default" javascriptSdk] Generating sources into "/Users/hatoya/Desktop/momiji/generated"
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.756596   90030 codegen.go:82] [connector \"default\" javascriptSdk] Generating sources into \"/Users/hatoya/Desktop/momiji/generated\"\n"}}
[debug] [2024-12-09T09:06:21.768Z] I1209 18:06:21.768114   90030 collector.go:107] schema extensions wrote into "/Users/hatoya/Desktop/momiji/dataconnect/.dataconnect/schema"
Generated sources: main/implicit.gql [3492B] main/input.gql [56882B] main/relation.gql [8879B] main/query.gql [8379B] main/mutation.gql [26453B] prelude.gql [65151B] 
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.768114   90030 collector.go:107] schema extensions wrote into \"/Users/hatoya/Desktop/momiji/dataconnect/.dataconnect/schema\"\nGenerated sources: main/implicit.gql [3492B] main/input.gql [56882B] main/relation.gql [8879B] main/query.gql [8379B] main/mutation.gql [26453B] prelude.gql [65151B] \n"}}
[debug] [2024-12-09T09:06:21.768Z] I1209 18:06:21.768606   90030 collector.go:107] connector "default" javascriptSdk wrote into "/Users/hatoya/Desktop/momiji/generated"
Generated sources: esm/package.json [17B] index.cjs.js [9231B] package.json [626B] index.d.ts [16738B] esm/index.esm.js [8275B] 
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.768606   90030 collector.go:107] connector \"default\" javascriptSdk wrote into \"/Users/hatoya/Desktop/momiji/generated\"\nGenerated sources: esm/package.json [17B] index.cjs.js [9231B] package.json [626B] index.d.ts [16738B] esm/index.esm.js [8275B] \n"}}
[debug] [2024-12-09T09:06:21.771Z] I1209 18:06:21.771002   90030 installsdk.go:48] Package @firebasegen/connector is already installed. Skipping installation
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.771002   90030 installsdk.go:48] Package @firebasegen/connector is already installed. Skipping installation\n"}}
[debug] [2024-12-09T09:06:21.771Z] I1209 18:06:21.771134   90030 load.go:115] Finished reloading
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.771134   90030 load.go:115] Finished reloading\n"}}
[debug] [2024-12-09T09:06:21.774Z] I1209 18:06:21.774363   90030 dev.go:95] Listening on address (HTTP + gRPC): 127.0.0.1:9399
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.774363   90030 dev.go:95] Listening on address (HTTP + gRPC): 127.0.0.1:9399\n"}}
[debug] [2024-12-09T09:06:21.775Z] I1209 18:06:21.775308   90030 dev.go:95] Listening on address (HTTP + gRPC): [::1]:9399
 {"metadata":{"emulator":{"name":"dataconnect"},"message":"I1209 18:06:21.775308   90030 dev.go:95] Listening on address (HTTP + gRPC): [::1]:9399\n"}}
[debug] [2024-12-09T09:06:23.500Z] [Data Connect] Connecting to postgres://127.0.0.1:5432/database?sslmode=disable}...
[debug] [2024-12-09T09:06:23.502Z] >>> [apiv2][query] POST http://127.0.0.1:9399/emulator/configure [none]
[debug] [2024-12-09T09:06:23.502Z] >>> [apiv2][body] POST http://127.0.0.1:9399/emulator/configure {"connectionString":"postgres://127.0.0.1:5432/database?sslmode=disable","database":"database","serviceId":"data-connect","maxOpenConnections":1}
[debug] [2024-12-09T09:06:23.539Z] Frontend message 
[debug] [2024-12-09T09:06:23.540Z] Backend message AuthenticationResponse
[debug] [2024-12-09T09:06:23.541Z] Backend message ParameterStatus
[debug] [2024-12-09T09:06:23.541Z] Backend message ReadyForQuery
[debug] [2024-12-09T09:06:23.543Z] Frontend message Query
[debug] [2024-12-09T09:06:23.549Z] Importing from /Users/hatoya/Desktop/momiji/saved_data/dataconnect_export/postgres.tar.gz
[debug] [2024-12-09T09:06:23.934Z] RuntimeError: unreachable
    at postgres.wasm.InitPostgres (wasm://wasm/postgres.wasm-02840556:wasm-function[8769]:0x48ff35)
    at postgres.wasm.AsyncPostgresSingleUserMain (wasm://wasm/postgres.wasm-02840556:wasm-function[7291]:0x3f864e)
    at postgres.wasm.pg_initdb (wasm://wasm/postgres.wasm-02840556:wasm-function[4958]:0x255f71)
    at Module._pg_initdb (/Users/hatoya/Desktop/momiji/node_modules/@electric-sql/pglite/dist/postgres.cjs:3:114245)
    at Hr.cs (/Users/hatoya/Desktop/momiji/node_modules/@electric-sql/pglite/dist/index.cjs:10:13471)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Hr.create (/Users/hatoya/Desktop/momiji/node_modules/@electric-sql/pglite/dist/index.cjs:10:8308)
    at async PostgresServer.getDb (/Users/hatoya/Desktop/momiji/node_modules/firebase-tools/lib/emulator/dataconnect/pgliteServer.js:93:23)
    at async Object.onMessage (/Users/hatoya/Desktop/momiji/node_modules/firebase-tools/lib/emulator/dataconnect/pgliteServer.js:52:32)
[error] 
[error] Error: An unexpected error has occurred.

dataconnect-debug.log

I1209 18:06:21.639620   90030 load.go:37] Reloading schema and connectors...
I1209 18:06:21.677771   90030 control.go:45] [engine.Service /emulator/data-connect bd10] UpdateSchema(): done sources: schema/schema.gql [2941B] 
I1209 18:06:21.717490   90030 control.go:73] [engine.Service /emulator/data-connect bd10] UpdateConnector(default): done sources: connector/mutations.gql [16185B] connector/queries.gql [3655B] 
I1209 18:06:21.756596   90030 codegen.go:82] [connector "default" javascriptSdk] Generating sources into "/Users/hatoya/Desktop/momiji/generated"
I1209 18:06:21.768114   90030 collector.go:107] schema extensions wrote into "/Users/hatoya/Desktop/momiji/dataconnect/.dataconnect/schema"
Generated sources: main/implicit.gql [3492B] main/input.gql [56882B] main/relation.gql [8879B] main/query.gql [8379B] main/mutation.gql [26453B] prelude.gql [65151B] 
I1209 18:06:21.768606   90030 collector.go:107] connector "default" javascriptSdk wrote into "/Users/hatoya/Desktop/momiji/generated"
Generated sources: esm/package.json [17B] index.cjs.js [9231B] package.json [626B] index.d.ts [16738B] esm/index.esm.js [8275B] 
I1209 18:06:21.771002   90030 installsdk.go:48] Package @firebasegen/connector is already installed. Skipping installation
I1209 18:06:21.771134   90030 load.go:115] Finished reloading
I1209 18:06:21.774363   90030 dev.go:95] Listening on address (HTTP + gRPC): 127.0.0.1:9399
I1209 18:06:21.775308   90030 dev.go:95] Listening on address (HTTP + gRPC): [::1]:9399

[joehan]

Hey @hatoya - thanks for reporting this. @aalej and I have repro'ed the issue and I'm working on a fix. I also found another related issue while investigating this, which is fixed in #8048

[joehan]

I believe the root cause here was data corruption in the dump due to improperly shutting down Pglite. We've fixed that in #8056, so this should be fixed in 13.29.0 .

If you still run into this or similar issues, please rerun the command with --debug and share the logs here or in a new issue. We added some better logging of postgres internal errors, so hopefully we'll be able to more efficiently debug this next time.

[hatoya]

@joehan
It is now functioning properly. Thank you for your contributions.