Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.2.3 启动spark集群失败,web-ui界面中缺少worker #3

Closed
trevery opened this issue Sep 17, 2020 · 3 comments
Closed

3.2.3 启动spark集群失败,web-ui界面中缺少worker #3

trevery opened this issue Sep 17, 2020 · 3 comments

Comments

@trevery
Copy link
Collaborator

trevery commented Sep 17, 2020

环境:

两台阿里服务器:master 和slave01

问题描述:

跟着教程走到启动Spark集群的时候。运行如下命令

sbin/start-slaves.sh

并没有在web ui界面中看到worker,说明slave启动失败。
启动成功应该如下图所示,注意worker:

image

而运行该命令后并没有启动成功slave。如下图所示:

image

解决方案:

运行如下start-slave命令并输入相关参数,而不是运行start-slaves.sh命令。

sbin/start-slave.sh spark://localhost:7077
或者将localhost换成内网ip(我第一次使用上面的可以,第二次上面不可以,需要使用内网ip)
sbin/start-slave.sh spark://master内网ip:7077

原因应该是这个命令应该是缺少了主机信息的配置,可以使用start-slave命令指定主机的地址和端口。

@Wanghui-Huang
Copy link
Owner

已更新到最新文档,但是这个issue持续关注下~

@lympassion
Copy link

sbin/start-slave.sh spark://master内网ip:7077 这条命令中开始一直不行,就把master内网ip替换成master外网ip他就成功了

@hehaorui
Copy link

hehaorui commented Nov 23, 2020

也可以在spark的配置文件spark-env.sh最后加一个 export MASTER_PORT=7077 来手动指定MASTER的端口,master和slave的配置文件都要改

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants