[Elastic Agent] Add ability to communicate and control the running agent daemon.

[blakerouse]

To provide a rich UX for the Elastic Agent we need to provide a way to communicate and control the running Elastic Agent.

Information that needs to be able to be retrieved from the local agent:

• Running version (possible different then the executing binary, but rare)
• Current status
  • Overall agent status
  • Status of each application
• Ability to restart Elastic Agent (self-restarts self with re-exec)
• Ability to trigger self-upgrade Elastic Agent

Below documents the commands and expected outputs:

version

Same version

$ elastic-agent version
Running Daemon: 7.9.0-SNAPSHOT (build: unknown at 0001-01-01 00:00:00 +0000 UTC)
Executable: 7.9.0-SNAPSHOT (build: unknown at 0001-01-01 00:00:00 +0000 UTC)

Not running

$ elastic-agent version
Running Daemon: <not running>
Executable: 7.9.0-SNAPSHOT (build: unknown at 0001-01-01 00:00:00 +0000 UTC)

Different versions running

$ elastic-agent version
WARN: The running version of Elastic Agent does not match this version.
Running Daemon: 7.9.0-SNAPSHOT (build: unknown at 0001-01-01 00:00:00 +0000 UTC)
Executable: 8.0.0-SNAPSHOT (build: unknown at 0001-01-01 00:00:00 +0000 UTC)

status

$ elastic-agent status
Status: Degraded
Message: 1 application is degraded

Application           Status             Message
Filebeat              Healthy            Running
Metricbeat            Degraded           system/load is an invalid metricset
Endpoint Security     Healthy            Protected with policy {000-....}        

restart

The different [/] below are showing interactive progress of what the command will report as its performing the restart action. It will then just print Successfully restarted over the progress.

$ elastic-agent restart
[/] Triggering restart
[-] Waiting to reconnect
[\] Reconnecting
[|] Connected
Successfully restarted

upgrade

The different [/] below are showing interactive progress of what the command will report as its performing the restart action. It will then just print Successfully restarted over the progress.

Next-version in same major

$ elastic-agent upgrade
[/] Triggering upgrade
[-] Upgrading to 7.9.1
[\] Reconnecting
[|] Connected
Successfully upgraded to 7.9.1

Next major

$ elastic-agent upgrade --version 8.0.0
[/] Triggering upgrade
[-] Upgrading to 8.0.0
[\] Reconnecting
[|] Connected
Successfully upgraded to 8.0.0

Specific version

$ elastic-agent upgrade --version 7.9.2-SNAPSHOT
[/] Triggering upgrade
[-] Upgrading to 8.9.2-SNAPSHOT
[\] Reconnecting
[|] Connected
Successfully upgraded to 7.9.2-SNAPSHOT

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[blakerouse]

@ph @ruflin @michalpristas Implementing this will greatly help with UX, re-exec testing, and upgrade testing. Let me know what you think.

[blakerouse]

Another improvement this brings is also the installation process. Currently we install Agent without starting the service and the configuration is basic and trying to write system metrics to localhost:4200.

I believe with the above changes we should do the following:

• Change the installation to use an empty elastic-agent.yml configuration, with nothing configured.
• Change the installation to start elastic-agent run on install and startup (but its doing nothing, because of empty config).
• Change the ending result of successful enroll to internally call elastic-agent restart, which will cause the daemon to re-exec switch into the fleet mode and start working. (removing the need for the user to worry about enabling and starting services)

[ph]

@blakerouse I really like this plan, the description is top!

I like the proposal for the empty configuration and the reload, we had an issue to allow have a enroll and run from a same subcommand. But I think what you are suggesting for service is even better than that and will simplify the workflow.

@EricDavisX The above will impact testings for the best, you might have suggestion too?

[EricDavisX]

sounds good to me.

[ph]

@blakerouse Can this be closed?

[blakerouse]

@ph I was leaving it open until upgrade was implemented.

[EricDavisX]

how much, if any, coordination do we need to do now to ensure design is compatible with Endpoint, @blakerouse ? @ferullo fyi.

• I'm guessing it all 'just works' right? Just trying to envision at what point the new Endpoint binary is downloaded and started, is it after the new Agent version is re-started or (before) as part of the setup before restart?

[blakerouse]

@EricDavisX Endpoint should just work, once Agent is upgraded it will then upgrade Endpoint to match the same version of Agent.