etcd crash when start member added into cluster #8608
Comments
|
How did you configure etcd cluster? e.g. etcd flags, or env vars? |
|
@gyuho looks like something went wrong when restoring auth store. |
|
So you bootstrap 3-node cluster with SRV with TLS. |
|
@gyuho My bad English misleaded you. I mean that etcd member has been
|
|
@jpbetz can you help on this issue? |
|
@xiang90 Sure, I'll have a look. |
|
@zyf0330 I cannot reproduce this... We've recently added containers to simulate local SRV-based cluster. Below is what I tried to reproduce this, with master branch: diff --git a/e2e/docker-dns-srv/run.sh b/e2e/docker-dns-srv/run.sh
index 7c7415f8d..710e3da3a 100755
--- a/e2e/docker-dns-srv/run.sh
+++ b/e2e/docker-dns-srv/run.sh
@@ -14,3 +14,12 @@ ETCDCTL_API=3 ./etcdctl \
--key=/certs//server-wildcard.key.insecure \
--discovery-srv etcd.local \
put foo bar
+
+ETCDCTL_API=3 ./etcdctl \
+ --cacert=/certs/ca.crt \
+ --cert=/certs/server-wildcard.crt \
+ --key=/certs//server-wildcard.key.insecure \
+ --discovery-srv etcd.local \
+ member list
+
+sleep 1hAnd make docker-test-build
make docker-test-compile
make docker-dns-srv-test-build
make docker-dns-srv-test-runAnd # SSH into SRV docker container
docker exec -it 076cd3c03d40 /bin/bash
cd /etcd
ETCDCTL_API=3 ./etcdctl \
--cacert=/certs/ca.crt \
--cert=/certs/server-wildcard.crt \
--key=/certs//server-wildcard.key.insecure \
--discovery-srv etcd.local \
member list
# remove m3
ETCDCTL_API=3 ./etcdctl \
--cacert=/certs/ca.crt \
--cert=/certs/server-wildcard.crt \
--key=/certs//server-wildcard.key.insecure \
--discovery-srv etcd.local \
member remove 1757793ebd94f915
# remove data dir
rm -rf m3.etcd/
# add member
ETCDCTL_API=3 ./etcdctl \
--cacert=/certs/ca.crt \
--cert=/certs/server-wildcard.crt \
--key=/certs//server-wildcard.key.insecure \
--discovery-srv etcd.local \
member add m3 --peer-urls=https://m3.etcd.local:32380To start with '--initial-cluster-state existing'
And everything works as expected in my side. @zyf0330 Did you run with etcd master? |
|
No, I ran with commit 4afb99f |
|
@gyuho You're one step ahead of me! I just finished attempting to reproduce it, using steps nearly identical to yours. I was also unable to reproduce. Everything worked as expected. I'll have look at the stack trace in more detail, see if I back track what might have gone wrong from it. |
|
Looks like the disabled state for |
|
i believe this is fixed by #8695 |
When I startup a etcd member which is added into a cluster at the first time startup, I got these logs. And this problem always appears and when I startup etcd at second times, it works normally.
The text was updated successfully, but these errors were encountered: