Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

state breaks highstate on master #278

Closed
EvaSDK opened this issue Jan 9, 2017 · 5 comments
Closed

state breaks highstate on master #278

EvaSDK opened this issue Jan 9, 2017 · 5 comments

Comments

@EvaSDK
Copy link

EvaSDK commented Jan 9, 2017

If master is being installed from packages and an configured by this formula, salt '*' state.highstate goes into a timeout when applying changes to master configuration as it tries to restart the master and results are never returned.

# salt-master --version
salt-master 2016.3.4 (Boron)

My question is, is this intentional ? Otherwise I'll just remove the following line https://github.com/saltstack-formulas/salt-formula/blob/master/salt/master.sls#L21 and call it a day.

I think salt-master and salt-minion, at the very least, should not be automatically restarted by anything in this formulas unless explicitly requested.

@ek9
Copy link
Contributor

ek9 commented Jan 13, 2017

Which OS is this happening on?
And can you describe the scenarion in more detail? It seems you already have salt installed, but the package updates it? If it does that, it's possible an update procedure can trigger it. More details would be helpful.

Just outright removing this line would cause problems when bootstrapping salt using this formula. Restarts are tied in to configuration file change, so you can push changes to master. It would be very BC breaking change for a formula like this and I believe some sort of option to trigger restarts might be necessary, but I need more info to understand the sitation better.

@ek9
Copy link
Contributor

ek9 commented Jan 13, 2017

I also did some tests locally and I find that salt-master is able to restart itself mid-process without breaking. I suspect the problem you are having is due to resulting bad configuration file after state is applied. It hangs your salt-master, which is another problem on itself. Should salt-master be reachable after restart, the process would go on, without interrupting.

@EvaSDK
Copy link
Author

EvaSDK commented Jan 13, 2017

I'm using it on Debian so it is using the packages from repo.saltstack.com.
Salt is already installed and configured but I wanted to cleanup the configuration and make sure all masters are running the same formulas etc.

This triggered a restart and highstate would not continue at least with 2016.3. The configuration was clean though as simply running ctrl+c and relaunching the highstate would complete correctly. Maybe I should retry. For now, I just removed the lines and ran state.apply for salt master configuration updates.

@ek9
Copy link
Contributor

ek9 commented Jan 13, 2017

Yeah but if you remove the line, then you won't be able to push any further updates to the master. For exmaple if you add new formula, it will update the config, but you have to restart server manually. It might be good temporary solution, but not something we'd want in the formula itself.

Additionally, all of the files for running state are fetched in beforehand, so even if you restart master, there is no need to further communicate with it.

I have tried a few more times, and updating the configs + restarting works, without needing restarts. It's possible you have some unrelated problem which caused this behaviour, but I doubt it's an issue with the formula itself.

There are no reproduction steps, and what you suggestion isn't an option either (to remove the dependency).

@EvaSDK
Copy link
Author

EvaSDK commented Jun 8, 2017

Switched back to this formula to check if everything is good now. I have been running it for a while now and did not have this issue again, at least with 2016.11.5 and 2016.3.6. Thanks for your help.

@EvaSDK EvaSDK closed this as completed Jun 8, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants