Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EDR Workflows][Fleet] Accurate endpoint count across multiple agent policies #193705

Conversation

szwarckonrad
Copy link
Contributor

@szwarckonrad szwarckonrad commented Sep 23, 2024

This PR updates the method for counting endpoint statuses. Previously, we fetched agent status using a single agent policy ID. With this change, we now pass an array of policy IDs, allowing us to include the returned stats for endpoints that share the same integration policy assigned to multiple agent policies.

Screenshot 2024-09-23 at 13 53 57
Screenshot 2024-09-23 at 13 53 24
Screenshot 2024-09-23 at 13 53 45

@szwarckonrad szwarckonrad self-assigned this Sep 23, 2024
@szwarckonrad szwarckonrad added release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team v9.0.0 Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v8.16.0 backport:version Backport to applied version labels labels Sep 23, 2024
@szwarckonrad szwarckonrad changed the title Proper endpoint count across multiple agent policies [EDR Workflows][Fleet] Accurate endpoint count across multiple agent policies Sep 23, 2024
@szwarckonrad szwarckonrad marked this pull request as ready for review September 23, 2024 12:04
@szwarckonrad szwarckonrad requested review from a team as code owners September 23, 2024 12:04
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 20.4MB 20.4MB +8.0B
Unknown metric groups

References to deprecated APIs

id before after diff
securitySolution 470 468 -2

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @szwarckonrad

Copy link
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fleet change LGTM

@szwarckonrad szwarckonrad requested review from tomsonpl and removed request for gergoabraham September 23, 2024 14:24
Copy link
Contributor

@tomsonpl tomsonpl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm 🚀 , left a few questions :)

@@ -328,7 +328,8 @@ export const getAgentStatusForAgentPolicyHandler: FleetRequestHandler<
soClient,
request.query.policyId,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we get rid of this one since we have policyIds ?

@@ -71,8 +82,14 @@ export async function getAgentStatusForAgentPolicy(
);
clauses.push(kueryAsElasticsearchQuery);
}

if (agentPolicyId) {
// If agentPolicyIds is provided, we filter by those, otherwise we filter by depreciated agentPolicyId
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a chance that policy_ids is not provided?

@szwarckonrad szwarckonrad merged commit 9cd2cfa into elastic:main Sep 24, 2024
53 checks passed
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 24, 2024
…policies (elastic#193705)

This PR updates the method for counting endpoint statuses. Previously,
we fetched agent status using a single agent policy ID. With this
change, we now pass an array of policy IDs, allowing us to include the
returned stats for endpoints that share the same integration policy
assigned to multiple agent policies.

![Screenshot 2024-09-23 at 13 53
57](https://github.com/user-attachments/assets/570027b7-79d7-4c9a-aa64-c0ecfe76cb7f)
![Screenshot 2024-09-23 at 13 53
24](https://github.com/user-attachments/assets/17d62c24-9d46-4133-a817-ea5849930435)
![Screenshot 2024-09-23 at 13 53
45](https://github.com/user-attachments/assets/c9fb5ed7-e4a0-4faa-a24d-253def10f163)

(cherry picked from commit 9cd2cfa)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Sep 24, 2024
…agent policies (#193705) (#193844)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[EDR Workflows][Fleet] Accurate endpoint count across multiple agent
policies (#193705)](#193705)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Konrad
Szwarc","email":"konrad.szwarc@elastic.co"},"sourceCommit":{"committedDate":"2024-09-24T10:08:54Z","message":"[EDR
Workflows][Fleet] Accurate endpoint count across multiple agent policies
(#193705)\n\nThis PR updates the method for counting endpoint statuses.
Previously,\r\nwe fetched agent status using a single agent policy ID.
With this\r\nchange, we now pass an array of policy IDs, allowing us to
include the\r\nreturned stats for endpoints that share the same
integration policy\r\nassigned to multiple agent
policies.\r\n\r\n![Screenshot 2024-09-23 at 13
53\r\n57](https://github.com/user-attachments/assets/570027b7-79d7-4c9a-aa64-c0ecfe76cb7f)\r\n![Screenshot
2024-09-23 at 13
53\r\n24](https://github.com/user-attachments/assets/17d62c24-9d46-4133-a817-ea5849930435)\r\n![Screenshot
2024-09-23 at 13
53\r\n45](https://github.com/user-attachments/assets/c9fb5ed7-e4a0-4faa-a24d-253def10f163)","sha":"9cd2cfa861713ab4dd8351794e3abfdbc7807ca1","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","v9.0.0","Team:Defend
Workflows","v8.16.0","backport:version"],"title":"[EDR Workflows][Fleet]
Accurate endpoint count across multiple agent
policies","number":193705,"url":"https://github.com/elastic/kibana/pull/193705","mergeCommit":{"message":"[EDR
Workflows][Fleet] Accurate endpoint count across multiple agent policies
(#193705)\n\nThis PR updates the method for counting endpoint statuses.
Previously,\r\nwe fetched agent status using a single agent policy ID.
With this\r\nchange, we now pass an array of policy IDs, allowing us to
include the\r\nreturned stats for endpoints that share the same
integration policy\r\nassigned to multiple agent
policies.\r\n\r\n![Screenshot 2024-09-23 at 13
53\r\n57](https://github.com/user-attachments/assets/570027b7-79d7-4c9a-aa64-c0ecfe76cb7f)\r\n![Screenshot
2024-09-23 at 13
53\r\n24](https://github.com/user-attachments/assets/17d62c24-9d46-4133-a817-ea5849930435)\r\n![Screenshot
2024-09-23 at 13
53\r\n45](https://github.com/user-attachments/assets/c9fb5ed7-e4a0-4faa-a24d-253def10f163)","sha":"9cd2cfa861713ab4dd8351794e3abfdbc7807ca1"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193705","number":193705,"mergeCommit":{"message":"[EDR
Workflows][Fleet] Accurate endpoint count across multiple agent policies
(#193705)\n\nThis PR updates the method for counting endpoint statuses.
Previously,\r\nwe fetched agent status using a single agent policy ID.
With this\r\nchange, we now pass an array of policy IDs, allowing us to
include the\r\nreturned stats for endpoints that share the same
integration policy\r\nassigned to multiple agent
policies.\r\n\r\n![Screenshot 2024-09-23 at 13
53\r\n57](https://github.com/user-attachments/assets/570027b7-79d7-4c9a-aa64-c0ecfe76cb7f)\r\n![Screenshot
2024-09-23 at 13
53\r\n24](https://github.com/user-attachments/assets/17d62c24-9d46-4133-a817-ea5849930435)\r\n![Screenshot
2024-09-23 at 13
53\r\n45](https://github.com/user-attachments/assets/c9fb5ed7-e4a0-4faa-a24d-253def10f163)","sha":"9cd2cfa861713ab4dd8351794e3abfdbc7807ca1"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Konrad Szwarc <konrad.szwarc@elastic.co>
szwarckonrad added a commit that referenced this pull request Sep 27, 2024
https://github.com/user-attachments/assets/2b64c1e0-0e6d-4ef5-952d-e4364b4403c4



The PR #193705 introduced an issue when counting active agents for
integration policies with only one agent policy assigned. In such cases,
`query.policyIds` was treated as a single string instead of an array of
strings (as expected with multiple agent policy ids like
`/?policyIds=x&policyIds=y`). This PR resolves the issue by ensuring
consistent handling of policyIds, regardless of the number of associated
agent policies.
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 27, 2024
…4294)

https://github.com/user-attachments/assets/2b64c1e0-0e6d-4ef5-952d-e4364b4403c4

The PR elastic#193705 introduced an issue when counting active agents for
integration policies with only one agent policy assigned. In such cases,
`query.policyIds` was treated as a single string instead of an array of
strings (as expected with multiple agent policy ids like
`/?policyIds=x&policyIds=y`). This PR resolves the issue by ensuring
consistent handling of policyIds, regardless of the number of associated
agent policies.

(cherry picked from commit 847285b)
kibanamachine added a commit that referenced this pull request Sep 27, 2024
) (#194351)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[EDR Workflows] Fix agent count for single agent policies
(#194294)](#194294)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Konrad
Szwarc","email":"konrad.szwarc@elastic.co"},"sourceCommit":{"committedDate":"2024-09-27T21:22:28Z","message":"[EDR
Workflows] Fix agent count for single agent policies
(#194294)\n\nhttps://github.com/user-attachments/assets/2b64c1e0-0e6d-4ef5-952d-e4364b4403c4\r\n\r\n\r\n\r\nThe
PR #193705 introduced an issue when counting active agents
for\r\nintegration policies with only one agent policy assigned. In such
cases,\r\n`query.policyIds` was treated as a single string instead of an
array of\r\nstrings (as expected with multiple agent policy ids
like\r\n`/?policyIds=x&policyIds=y`). This PR resolves the issue by
ensuring\r\nconsistent handling of policyIds, regardless of the number
of associated\r\nagent
policies.","sha":"847285ba7191aa6d26fb3dccc06748e1c4a202b1","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","v9.0.0","Team:Defend
Workflows","v8.16.0","backport:version"],"title":"[EDR Workflows] Fix
agent count for single agent
policies","number":194294,"url":"https://github.com/elastic/kibana/pull/194294","mergeCommit":{"message":"[EDR
Workflows] Fix agent count for single agent policies
(#194294)\n\nhttps://github.com/user-attachments/assets/2b64c1e0-0e6d-4ef5-952d-e4364b4403c4\r\n\r\n\r\n\r\nThe
PR #193705 introduced an issue when counting active agents
for\r\nintegration policies with only one agent policy assigned. In such
cases,\r\n`query.policyIds` was treated as a single string instead of an
array of\r\nstrings (as expected with multiple agent policy ids
like\r\n`/?policyIds=x&policyIds=y`). This PR resolves the issue by
ensuring\r\nconsistent handling of policyIds, regardless of the number
of associated\r\nagent
policies.","sha":"847285ba7191aa6d26fb3dccc06748e1c4a202b1"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/194294","number":194294,"mergeCommit":{"message":"[EDR
Workflows] Fix agent count for single agent policies
(#194294)\n\nhttps://github.com/user-attachments/assets/2b64c1e0-0e6d-4ef5-952d-e4364b4403c4\r\n\r\n\r\n\r\nThe
PR #193705 introduced an issue when counting active agents
for\r\nintegration policies with only one agent policy assigned. In such
cases,\r\n`query.policyIds` was treated as a single string instead of an
array of\r\nstrings (as expected with multiple agent policy ids
like\r\n`/?policyIds=x&policyIds=y`). This PR resolves the issue by
ensuring\r\nconsistent handling of policyIds, regardless of the number
of associated\r\nagent
policies.","sha":"847285ba7191aa6d26fb3dccc06748e1c4a202b1"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Konrad Szwarc <konrad.szwarc@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:version Backport to applied version labels release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution Team:Fleet Team label for Observability Data Collection Fleet team v8.16.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants