[Ingest Manager] Support for UPGRADE_ACTION

[michalpristas]

we discussed the form of the action to be

{
  Version string
  SourceURI Uri
}

version is the target version agent should upgrade into,
sourceURI is the URI compatible with artifacts.elastic.co containing binaries, hashes and asc files.

Upgrade action needs to work with snapshots as well, in this case action will look as following

{
  "Version": "8.0.0-SNAPSHOT"
   "SourceURI": "https://snapshots.elastic.co/8.0.0-65ab0964/downloads/beats"
}

Action will get ACKnowledged once upgrade operation is finished.
Agent will report state SUCCESS/FAILURE together with ACK

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[blakerouse]

This fits perfectly with the protocol I proposed so we can test it out locally - https://github.com/elastic/beats/pull/20146/files#diff-8c2bfe38adf296a51ff5fbe4b2ed2f0aR55

Perfect!

[neptunian]

Needing some clarification about how the agent upgrade works.

• Nicolas told me the agent can’t be greater than the user’s kibana version so we’d just be checking to see if the agent is less than the kibana version they are on. So it seems we just need to send the agent version to update as their kibana version if its not aligned?
• I plan to create an /agent/agentid/upgrade endpoint. If the user uses the api directly, do we want to allow the user to update to any version or are we going to block them updating the agent to a version higher than their kibana version? Or higher than their current version but lower than their kibana version. Will we strictly not allow downgrading? Or should we not have them pass any parameters and choose the right version ourselves?

[michalpristas]

i think we should prevent getting into a situation of having disconnected agent by enabling agent.ver > fleet.ver
downgrade should still work in case new version of agent is not ok, so it can go back to working version and talk to fleet as usual. even though its version is < fleet version.
@ruflin is this behavior ok?

[blakerouse]

We should not allow upgrade passed to current Kibana version, that way we do not need to worry about agent.version > kibana.version. Just like it's not possible to enroll an Agent that is greater than the running kibana.version.

Downgrading needs to be possible, because there could be a situation where after an upgrade the results are not correct (even though Agent thinks everything is fine), and the user wants to revert the Agent back to it's previous version. I don't know if they need to ability to downgrade to any version, but back to the previous needs to be supported.

I think if you upgrade from Kibana 7.10 to 7.12 (skipping 7.11) then you can only upgrade your Elastic Agents to 7.12 not to 7.11. After upgrade of Agent to 7.12 and it's not working as expected then the user should be able to downgrade back to 7.10. I do not see the use-case to allow them to downgrade that agent to 7.11.

[neptunian]

@blakerouse It is possible to enroll an agent version > kibana version if the patch version is higher:
#75176 (comment)
#76238

[blakerouse]

@neptunian based on comments from @ruflin I think that patched versions are allowed to be higher.

[neptunian]

@ph @ruflin Are we wanting to allow the user to specify as part of the upgrade api the version they want to up/down grade to? If so, it sounds like we'll need to add these restrictions. The one @blakerouse mentioned about not being able to upgrade to a version that kibana never upgraded to, sounds like we'd have to know the versions kibana was ever on and compare against that. Is it possible to know that information?

Other than that, for simplicity's sake, I think they should be able to install any version so long as it passes validateAgentVersion described in #75176 (comment) . If we really want to restrict what they are allowed to downgrade (previous version only), we'll need to save that.

As far as the verson kibana will send from the UI, it would be the same version as the kibana version they are on.

[ph]

Other than that, for simplicity's sake, I think they should be able to install any version so long as it passes validateAgentVersion described in #75176 (comment) . If we really want to restrict what they are allowed to downgrade (previous version only), we'll need to save that.

Could we do that in a followup PRs?

I would even be ok to default to the current Kibana version for first release.

[ruflin]

++ on fixing it the exact Kibana version for now. Also ++ on what Blake described that downgrades are only possible to the exact version the user had before as we know it was working and likely still have the binary around.