geth import: Fatal: Could not open database: resource temporarily unavailable

[meowsbits]

System information

Geth version:

> ./build/bin/geth version
Geth
Version: 1.9.19-unstable
Git Commit: 90dedea40fc174c914ef038b8b480c2c0ff031b9
Git Commit Date: 20200803
Architecture: amd64
Protocol Versions: [65 64 63]
Go Version: go1.14.3
Operating System: linux
GOPATH=/home/ia/go
GOROOT=/home/ia/go1.14.3.linux-amd64

90dedea

The issue is not present on tag v1.9.18.

Expected behaviour

geth import imports an exported .rlp.gz file successfully.

Actual behaviour

geth import throws Fatal: Could not open database: resource temporarily unavailable.

Steps to reproduce the behaviour

These steps use an exported .rlp.gz file ./tests/regression/simulated/foundation-condense-state/export.rlp.gz and a corresponding config file ./tests/regression/simulated/foundation-condense-state/foundation.conf.json, which are available here if you want to reproduce exactly. Otherwise, any correct config/export pair should be sufficient.

> rm -rf dd

> ./build/bin/geth --debug --datadir ./dd init ./tests/regression/simulated/foundation-condense-state/foundation.conf.json
INFO [08-04|10:31:37.468|cmd/utils/flags.go:1164] Maximum peer count                       ETH=50 LES=0 total=50
INFO [08-04|10:31:37.468|cmd/utils/flags.go:1238] Smartcard socket not found, disabling    err="stat /run/pcscd/pcscd.comm: no such file or directory"
INFO [08-04|10:31:37.469|cmd/utils/flags.go:1566] Set global gas cap                       cap=25000000
INFO [08-04|10:31:37.469|ethdb/leveldb/leveldb.go:95] Allocated cache and file handles         database=/home/ia/go/src/github.com/ethereum/go-ethereum/dd/geth/chaindata cache=16.00MiB handles=16
INFO [08-04|10:31:37.488|core/genesis.go:165]         Writing custom genesis block
INFO [08-04|10:31:37.488|trie/database.go:768]        Persisted trie from memory database      nodes=1 size=150.00B time="27.352µs" gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=0.00B
INFO [08-04|10:31:37.488|geth/chaincmd.go:257]        Successfully wrote genesis state         database=chaindata hash="529b11…c00a48"
INFO [08-04|10:31:37.488|ethdb/leveldb/leveldb.go:95] Allocated cache and file handles         database=/home/ia/go/src/github.com/ethereum/go-ethereum/dd/geth/lightchaindata cache=16.00MiB handles=16
INFO [08-04|10:31:37.498|core/genesis.go:165]         Writing custom genesis block
INFO [08-04|10:31:37.498|trie/database.go:768]        Persisted trie from memory database      nodes=1 size=150.00B time="24.732µs" gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=0.00B
INFO [08-04|10:31:37.499|geth/chaincmd.go:257]        Successfully wrote genesis state         database=lightchaindata hash="529b11…c00a48"

> ./build/bin/geth --debug --datadir ./dd import ./tests/regression/simulated/foundation-condense-state/export.rlp.gz
INFO [08-04|10:31:45.123|cmd/utils/flags.go:1164] Maximum peer count                       ETH=50 LES=0 total=50
INFO [08-04|10:31:45.123|cmd/utils/flags.go:1238] Smartcard socket not found, disabling    err="stat /run/pcscd/pcscd.comm: no such file or directory"
INFO [08-04|10:31:45.125|cmd/utils/flags.go:1566] Set global gas cap                       cap=25000000
INFO [08-04|10:31:45.125|eth/backend.go:114]      Allocated trie memory caches             clean=256.00MiB dirty=256.00MiB
INFO [08-04|10:31:45.125|ethdb/leveldb/leveldb.go:95] Allocated cache and file handles         database=/home/ia/go/src/github.com/ethereum/go-ethereum/dd/geth/chaindata cache=512.00MiB handles=524288
INFO [08-04|10:31:45.170|core/rawdb/freezer.go:127]   Opened ancient database                  database=/home/ia/go/src/github.com/ethereum/go-ethereum/dd/geth/chaindata/ancient
INFO [08-04|10:31:45.171|eth/backend.go:125]          Initialised chain configuration          config="{ChainID: 1 Homestead: 50 DAO: 69 DAOSupport: true EIP150: 100 EIP155: 180 EIP158: 180 Byzantium: 437 Constantinople: 728 Petersburg: 728 Istanbul: 906, Muir Glacier: 920, YOLO v1: <nil>, Engine: ethash}"
INFO [08-04|10:31:45.171|consensus/ethash/ethash.go:453] Disk storage enabled for ethash caches   dir=/home/ia/go/src/github.com/ethereum/go-ethereum/dd/geth/ethash count=3
INFO [08-04|10:31:45.171|consensus/ethash/ethash.go:456] Disk storage enabled for ethash DAGs     dir=/home/ia/.ethash count=2
INFO [08-04|10:31:45.171|eth/backend.go:147]             Initialising Ethereum protocol           versions="[65 64 63]" network=1 dbversion=<nil>
WARN [08-04|10:31:45.171|eth/backend.go:153]             Upgrade blockchain database version      from=<nil> to=7
INFO [08-04|10:31:45.172|core/blockchain.go:424]         Loaded most recent local header          number=0 hash="529b11…c00a48" td=16384 age=51y3mo4w
INFO [08-04|10:31:45.172|core/blockchain.go:425]         Loaded most recent local full block      number=0 hash="529b11…c00a48" td=16384 age=51y3mo4w
INFO [08-04|10:31:45.172|core/blockchain.go:426]         Loaded most recent local fast block      number=0 hash="529b11…c00a48" td=16384 age=51y3mo4w
INFO [08-04|10:31:45.173|core/tx_journal.go:166]         Regenerated local transaction journal    transactions=0 accounts=0
INFO [08-04|10:31:45.191|trie/sync_bloom.go:75]          Allocated fast sync bloom                size=512.00MiB
INFO [08-04|10:31:45.193|trie/sync_bloom.go:128]         Initialized fast sync bloom              items=1 errorrate=0.000 elapsed=1.268ms
INFO [08-04|10:31:45.194|ethdb/leveldb/leveldb.go:95]    Allocated cache and file handles         database=/home/ia/go/src/github.com/ethereum/go-ethereum/dd/geth/chaindata cache=512.00MiB handles=524288
Fatal: Could not open database: resource temporarily unavailable

[holiman]

Oh, nice catch -- and thanks for the detailed repro!

Oddly enough, this is what git bisect tells me:

[user@work go-ethereum]$ git bisect bad
b2b14e6ce34734542cff21b504a4579de2ff79ad is the first bad commit
commit b2b14e6ce34734542cff21b504a4579de2ff79ad

[holiman]

Ran it again, I think I failed to rebuild during one of the bisections. Here's the probable culprit:

c0c01612e95dd20f125154646b38283ab780f357 is the first bad commit
commit c0c01612e95dd20f125154646b38283ab780f357
Author: rene <41963722+renaynay@users.noreply.github.com>
Date:   Mon Aug 3 19:40:46 2020 +0200

    node: refactor package node (#21105)
    

cc @renaynay @fjl

[meowsbits]

The error seems to apply to geth export as well.