Skip to content

Latest commit

 

History

History
260 lines (221 loc) · 11.4 KB

README.md

File metadata and controls

260 lines (221 loc) · 11.4 KB

elastic_labs

Project Overview

The elastic_labs project is designed to simulate a Elastic Stack environment using Ansible for automated deployment and management. This setup focuses on configuring a comprehensive SIEM system that includes Elasticsearch, Kibana, Zeek integration, and Elastic Agents on a controlled RHEL environment.

Objectives

  • Automate majority of the setup using Ansible, from system configuration to application deployment
  • Integrate Elasticsearch and Kibana for data indexing and visualization
  • Deploy Zeek on selected nodes to monitor network traffic and log activities
  • Manage Elastic Agents on RHEL and Windows VMs for endpoint security and log collection
  • Create and enforce policies through Fleet management in Kibana, enhancing the real-time security posture
  • Create dashboards and rules for testing
  • Generate log telemetry and trigger alerts for analysis

Architecture

  • Control Node: Hosts Elasticsearch and Kibana orchestrating the SIEM framework
  • Node1: Setup as Fleet Server to manage agent policies in a centralized environment
  • Node1 and Node2: Run Zeek for network monitoring and Elastic Agents for endpoint security
  • Node3-Windows: Extends the monitoring to include Windows-specific threats using Sysmon and Windows Defender integrations

Prerequisites

Before we begin, ensure the following are prepared:

  • VirtualBox 7.0
  • 3 RHEL 8 VMs
  • 1 Windows 10 VM
Server Role CPU RAM
Control(rhel 8) Management 4 8 GB
Node1(rhel 8) Fleet/Zeek 2 4 GB
Node2(rhel 8) Elastic Agent/zeek 2 2 GB
Node3(windows 10) Elastic Agent 4 8 GB
  • Network Configuration: Set IP addresses and hostnames for each VM. Networking mode is set to NAT Network with port forwarding configured to allow access from host.

Note: Throughout this project, root password set to 'password'

Setup Environment

  • Insert the RHEL ISO on control node

  • Run the command to mount the ISO

    sudo mount /dev/sr0 /mnt

  • Add and configure the repository from the ISO

    dnf config-manager --add-repo=file:///mnt/AppStream
dnf config-manager --add-repo=file:///mnt/BaseOS
echo "gpgcheck=0" >> /etc/yum.repos.d/mnt_AppStream.repo
echo "gpgcheck=0" >> /etc/yum.repos.d/mnt_BaseOS.repo

  • Install git and ansible-core

    dnf install -y git ansible-core

  • Clone the repository

    git clone https://github.com/Thuynh808/elastic_labs
cd elastic_labs

  • Configure inventory hosts

    vim inventory

  • Run initial setup

    ./initial-setup.sh

    Script Breakdown
    • Install collections from requirements file
    • Generate root SSH keypair
    • Copy root public key to nodes
    • Configure /etc/hosts file for nodes
    • Setup ftp server on control node as repository
    • Add repo to nodes
    • Ensure python is installed on nodes
    • Use rhel-system-roles-timesync to synchronize all nodes

Installation

  • Install and configure elasticsearch and kibana

    ./install.sh

    Script Breakdown
    • Setup repositories for Elasticsearch and Kibana
    • Install Elasticsearch and Kibana
    • Open firewall ports for services
    • Set SELinux ports for services
    • Generate Elasticsearch token for Kibana
    • Enroll Kibana
    • Reset password for elastic user
    • Generate encryption keys for Kibana
    • Create Fleet server policy
    • Add Zeek integration policy
    • Add System logs/metrics integration policy
    • Adjust Kibana service file
    • Install Zeek on node1 and node2
    • Confirm services are up and running on necessary ports

  • Retrieve elastic password

    cat password_result

  • Access Kibana with browser

    http://localhost:5601

    Note: Make sure to setup port forwarding for NAT Network to allow host machine to access VMs

  • Log in with user elastic with password from password_result

  • Add Fleet Server through Kibana UI

    Expand guide
    • Navigate to Fleet and add Fleet Server

    • Set Fleet Server host URL and generate service token

      elastic_labs

      elastic_labs

    • Copy provided command to install Fleet Server

      elastic_labs

    • SSH into node1 and execute the copied command

      elastic_labs

      elastic_labs

  • Enroll RHEL Agents defined in inventory file 

    ansible-playbook enroll_agents.yaml -vv

  • Add Sysmon and Windows Defender integrations to Agent Policy 

    ansible-playbook windows_integration.yaml -vv

  • Enroll Windows agent through Kibana UI

    Expand Guide
    • Navigate to Fleet and add Agent

    • Choose the Agent Policy and copy provided windows command to install and enroll Elastic Agent

    Note: Add --insecure to the command to trust self signed certificate

    elastic_labs

    • Confirm Agent enrollment and incoming data

      elastic_labs

Operational Verification

In this section, let's make sure our installation process was a success and all components are up and running.

  • Confirm /etc/hosts on nodes

    elastic_labs

  • Synchronized time across all nodes

    elastic_labs

  • Elasticsearch and Kibana are running with no errors

    elastic_labs

  • Confirm zeek is running on node1 and node2

    elastic_labs

  • Through Kibana, verify agents are present

    elastic_labs

  • Verify Agent Policies

    elastic_labs

  • Verify Integrations

    elastic_labs

    elastic_labs

  • Check health and integrations on individual nodes

    elastic_labs

    elastic_labs

    elastic_labs

  • Verify logs are coming in from our integrations

    elastic_labs

    elastic_labs

    elastic_labs

Rules and Alerts

In this section, we'll create simple brute force rules to test our deployment.

  • Navigate to Rules section and create new rule

    elastic_labs

  • Specify custom query and threshold for our brute force rules

    elastic_labs

  • Set MITRE ATT@CK tactics and techniques

    elastic_labs

  • Confirm our created brute force rules

    elastic_labs

    elastic_labs

    elastic_labs

Testing and Analysis

⚠️ Trigger Alerts

Now let's trigger some alerts!

  • Using ncrack to generate SSH brute force attack on RHEL vm

    elastic_labs

  • For the Windows vm, manually fail login attempts to trigger the RDP alert

  • Dashboard created to get a hollistic view and track our alerts

    elastic_labs

  • Navigating to our alerts page, we can see a few have been triggered

    elastic_labs

🔍 SSH Brute Force
  • Upon clicking on SSH Brute Force Alert, we can review the alert description and alert reason

    elastic_labs

  • Diving into our logs, we can see multiple entries where the root account failed ssh_login on node2.streetrack.org

    elastic_labs

    elastic_labs

  • Here we can confirm details of the host thats been targeted, username, source IP, and process name

    elastic_labs

    elastic_labs

  • Diving into zeek logs, we can verify the network connection between the related IPs, ports used, and SSH client version

    elastic_labs

    elastic_labs

    elastic_labs

🔍 RDP Brute force
  • Now let's take a look at the RDP Brute Force alert

    elastic_labs

  • Diving into our logs, we can see that node3-windows that's been targeted, as well as useful details in the message section

    elastic_labs

    elastic_labs

  • Here we can confirm more details like failure reason, username and source IP

    elastic_labs

    elastic_labs

  • Digging into zeek logs, we can verify the network connection between the related IPs, ports used, and the protocol used

    elastic_labs

    elastic_labs

Conclusion

What an incredibly rewarding journey! From deep diving into documentations to troubleshooting agent policies, I learned a lot along the way. Automating the Elastic Stack deployment with Ansible made the setup smoother and reduced errors. Compiling Zeek for RHEL was challenging but proved how powerful open-source tools can be. The testing phase allowed me to simulate security incidents and fine-tune our system's response. Overall, this project sharpened my automation and security analysis skills, getting me ready for real-world challenges.