Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Admin By Request EPM Connector #12402

Draft
wants to merge 21 commits into
base: main
Choose a base branch
from

Conversation

ravikumar5555
Copy link
Contributor

@ravikumar5555 ravikumar5555 commented Jan 20, 2025

Overview

This Pull Request introduces the initial release of a new integration for Admin By Request EPM. The integration captures events, specifically

Datasets API Description
Auditlog /auditlog Provides a list of audit events
Events /events Provides a list of events

It also allows to use real-time data using CEL agent by starting a listener on elastic side.

Key features include:

  • Data streams for auditlog and events from Admin By Request EPM.
  • Data collection logic for the auditlog and events data streams.
  • Ingest pipeline for the auditlog and events data streams.
  • Mapped fields according to the actual ECS
  • schema and added fields metadata in the appropriate yaml files.
  • Dashboard and visualizations of auditlog and events.
  • Tests for the pipeline for the auditlog and events data streams.
  • System test cases for the auditlog and events data streams.
  • User documentation on configuring Admin By Request EPM for this integration.

Proposed commit message

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • New integration in packages/admin_by_request_epm folder.
  • Data Stream & Mappings
  • Ingest Pipelines
  • Pipeline Tests
  • Testing
    • Asset
    • Pipeline
    • Static
    • System
    • Documentation
  • Visualizations (Dashboard UI)
    • Total Number of auditlogs and events
    • Auditlog distribution by type
    • Event distribution by event code
    • Auditlog distribution by computer platform
    • Average response time for user requests
    • User requests over time with status
    • Requests distribution by approval count
    • Installed application count by vendor
    • Request type distribution by request status
    • Application scan result over time
    • Event distribution by status
    • Auditlog table
    • Events log table

How to test this PR locally

Manual testing

Running elastic stack is required.

  1. Clone the integration repository.
  2. Install the admin_by_request_epm package locally.
cd packages/admin_by_request_epm && elastic-package install
  1. Navigate to your kibana dashboard and install integration as usual
  2. Credentials are required (as described in README.md)
  3. Use discover/dashboard to review the results received. filter by event.dataset :"admin_by_request_epm.auditlog" for auditlog data stream and filter by event.dataset :"admin_by_request_epm.events" for events data stream
  4. data_stream/auditlog/_dev/test/pipeline/ and data_stream/events/_dev/test/pipeline/ gives a few examples of requests can be used to test the endpoint.

Automated testing

  1. Clone the integrations repository.
  2. Start the Elastic stack using the elastic-package.
  3. Install the Elastic package locally.
  4. Navigate to the integrations/packages/admin_by_request_epm directory.
cd packages/admin_by_request_epm
  1. Run the following command to execute tests:
$ elastic-package test
Run asset tests for the package
--- Test results for package: admin_by_request_epm - START ---
╭──────────────────────┬─────────────┬───────────┬───────────────────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE              │ DATA STREAM │ TEST TYPE │ TEST NAME                                                                     │ RESULT │ TIME ELAPSED │
├──────────────────────┼─────────────┼───────────┼───────────────────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ admin_by_request_epm │             │ asset     │ dashboard admin_by_request_epm-4192141d-1c22-44dd-bd2c-87c704964755 is loaded │ PASS   │       3.48µs │
│ admin_by_request_epm │             │ asset     │ dashboard admin_by_request_epm-aa221d9f-4324-474f-bbc7-3f827c41266b is loaded │ PASS   │        321ns │
│ admin_by_request_epm │ auditlog    │ asset     │ index_template logs-admin_by_request_epm.auditlog is loaded                   │ PASS   │        435ns │
│ admin_by_request_epm │ auditlog    │ asset     │ ingest_pipeline logs-admin_by_request_epm.auditlog-0.0.1 is loaded            │ PASS   │        357ns │
│ admin_by_request_epm │ events      │ asset     │ index_template logs-admin_by_request_epm.events is loaded                     │ PASS   │        403ns │
│ admin_by_request_epm │ events      │ asset     │ ingest_pipeline logs-admin_by_request_epm.events-0.0.1 is loaded              │ PASS   │        145ns │
╰──────────────────────┴─────────────┴───────────┴───────────────────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: admin_by_request_epm - END   ---
Done
Run pipeline tests for the package
--- Test results for package: admin_by_request_epm - START ---
╭──────────────────────┬─────────────┬───────────┬───────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE              │ DATA STREAM │ TEST TYPE │ TEST NAME                                     │ RESULT │ TIME ELAPSED │
├──────────────────────┼─────────────┼───────────┼───────────────────────────────────────────────┼────────┼──────────────┤
│ admin_by_request_epm │ auditlog    │ pipeline  │ (ingest pipeline warnings test-auditlog.json) │ PASS   │ 766.228676ms │
│ admin_by_request_epm │ auditlog    │ pipeline  │ test-auditlog.json                            │ PASS   │ 219.133248ms │
│ admin_by_request_epm │ events      │ pipeline  │ (ingest pipeline warnings test-events.json)   │ PASS   │ 1.098503043s │
│ admin_by_request_epm │ events      │ pipeline  │ test-events.json                              │ PASS   │ 187.495364ms │
╰──────────────────────┴─────────────┴───────────┴───────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: admin_by_request_epm - END   ---
Done
Run policy tests for the package
--- Test results for package: admin_by_request_epm - START ---
No test results
--- Test results for package: admin_by_request_epm - END   ---
Done
Run static tests for the package
--- Test results for package: admin_by_request_epm - START ---
╭──────────────────────┬─────────────┬───────────┬──────────────────────────┬────────┬──────────────╮
│ PACKAGE              │ DATA STREAM │ TEST TYPE │ TEST NAME                │ RESULT │ TIME ELAPSED │
├──────────────────────┼─────────────┼───────────┼──────────────────────────┼────────┼──────────────┤
│ admin_by_request_epm │ auditlog    │ static    │ Verify sample_event.json │ PASS   │ 119.997634ms │
│ admin_by_request_epm │ events      │ static    │ Verify sample_event.json │ PASS   │ 138.155317ms │
╰──────────────────────┴─────────────┴───────────┴──────────────────────────┴────────┴──────────────╯
--- Test results for package: admin_by_request_epm - END   ---
Done

Related issues

Screenshots

@andrewkroh andrewkroh added the New Integration Issue or pull request for creating a new integration package. label Jan 20, 2025
@qcorporation qcorporation force-pushed the main branch 2 times, most recently from eda4138 to f728ca7 Compare February 5, 2025 22:00
@ravikumar5555 ravikumar5555 changed the title Admin By Request Connector Admin By Request EPM Connector Feb 11, 2025
Copy link
Contributor

@HeroicHorizon HeroicHorizon Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove this file.

resource.timeout: {{http_client_timeout}}
{{/if}}
resource.ssl.renegotiation: "freely"
resource.ssl.verification_mode: "certificate"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resource.ssl.renegotiation: "freely"
resource.ssl.verification_mode: "certificate"

remove this

{{#if http_client_timeout}}
resource.timeout: {{http_client_timeout}}
{{/if}}
resource.ssl.renegotiation: freely
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this: resource.ssl.renegotiation: freely

description: "Admin By Request EPM is a solution for managing and monitoring privileged access to Windows and Mac computers. It enables real-time monitoring of privileged account access, session recordings, and password checkout patterns to help security teams maintain compliance and quickly identify potential privilege abuse."
type: integration
categories:
- custom
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- custom
- security

@@ -0,0 +1,5 @@
- version: "0.0.1"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- version: "0.0.1"
- version: "0.1.0"

format_version: 3.3.0
name: admin_by_request_epm
title: Admin By Request EPM
version: 0.0.1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
version: 0.0.1
version: 0.1.0

@@ -10,6 +10,7 @@
/packages/1password @elastic/security-service-integrations
/packages/abnormal_security @elastic/security-service-integrations
/packages/activemq @elastic/obs-infraobs-integrations
/packages/adminbyrequest @elastic/security-service-integrations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/packages/adminbyrequest @elastic/security-service-integrations
/packages/admin_by_request_epm @elastic/security-service-integrations

- fingerprint:
fields:
- admin_by_request_epm.auditlog.id
target_field: "_id"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so for fingerprint we can have either ignore_missing: true or have a more robust on_failure condition something like:

- fingerprint:
     fields:
       - admin_by_request_epm.auditlog.id
     target_field: "_id"
     on_failure:
        - append:
            field: error.message
            value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'```

- fingerprint:
fields:
- admin_by_request_epm.events.id
target_field: "_id"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so for fingerprint we can have either ignore_missing: true or have a more robust on_failure condition something like:

  - fingerprint:
      fields:
        - admin_by_request_epm.events.id
      tag: fingerprinting
      target_field: "_id"
      on_failure:
        - append:
            field: error.message
            value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'


- **`auditlog`**: Provides audit data that includes elevation requests, approvals, application installations, and scan results.
- [Auditlog](https://www.adminbyrequest.com/en/docs/auditlog-api) are records generated when user takes action such as installing a software, running an application with admin privileges, requesting for admin session, approval or denial of requests and scan results.
- This data stream leverages the Admin By Request EPM API `/auditlog/delta` endpoint to retrieve data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add to Admin By Request EPM API /auditlog/delta
link: https://www.adminbyrequest.com/en/docs/auditlog-api#:~:text=throttle%20your%20account-,Delta%20Data,-To%20avoid%20having


- **`events`**: Provides system security events and administrative changes, including group modifications, policy changes and security violations. This allows tracking of administrative activities and security-critical events. Some events have corresponding audit log entries.
- [Events](https://www.adminbyrequest.com/en/docs/events-api) are records that are generated on various actions done by users and administrators. These include group modifications, policy changes, security violations, and other administrative activities.
- This data stream leverages the Admin By Request EPM API `/events` endpoint to retrieve data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add link to Admin By Request EPM API /events endpoint
link: https://www.adminbyrequest.com/en/docs/events-api

@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link

elasticmachine commented Feb 14, 2025

💔 Build Failed

Failed CI Steps

History

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Integration:abnormal_security Abnormal Security Integration:1password 1Password New Integration Issue or pull request for creating a new integration package.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants