Fix agents purge #82

Lifka · 2018-05-22T11:13:37Z

Hello team,

this PR solves the issue #79, and it's part of https://github.com/wazuh/wazuh/tree/fix-agents-purge.

Remove

POST/agents/purge
GET/agents/purgeable

Add

Filter status in GET/agents can filter by several status separated by commas. Sample:

$ curl -u foo:bar -k -X GET "http://127.0.0.1:55000/agents?status=Disconnected,neverconnected&pretty"

Added new filter to GET/agents resquest: timeframe. Timeframe filters out agents that have not connected within the specified time. In case of having never connected status, filter using the date they were registered. Sample:

$ date
Tue May 22 09:49:01 UTC 2018

$ curl -u foo:bar -k -X GET "http://127.0.0.1:55000/agents?pretty&timeframe=1s&status=neverconnected"
{
   "error": 0,
   "data": {
      "totalItems": 3,
      "items": [
         {
            "status": "Never connected",
            "dateAdd": "2018-05-21 09:27:49",
            "name": "agent001",
            "ip": "any",
            "id": "001",
            "node_name": "unknown"
         },
         {
            "status": "Never connected",
            "dateAdd": "2018-05-21 09:28:19",
            "name": "agent004",
            "ip": "any",
            "id": "004",
            "node_name": "unknown"
         },
         {
            "status": "Never connected",
            "dateAdd": "2018-05-22 09:36:31",
            "name": "agent_test1",
            "ip": "any",
            "id": "007",
            "node_name": "unknown"
         }
      ]
   }
}

$ curl -u foo:bar -k -X GET "http://127.0.0.1:55000/agents?pretty&timeframe=1h&status=neverconnected"
{
   "error": 0,
   "data": {
      "totalItems": 2,
      "items": [
         {
            "status": "Never connected",
            "dateAdd": "2018-05-21 09:27:49",
            "name": "agent001",
            "ip": "any",
            "id": "001",
            "node_name": "unknown"
         },
         {
            "status": "Never connected",
            "dateAdd": "2018-05-21 09:28:19",
            "name": "agent004",
            "ip": "any",
            "id": "004",
            "node_name": "unknown"
         }
      ]
   }
}

$ date
Tue May 22 09:55:53 UTC 2018

$ curl -u foo:bar -k -X GET "http://127.0.0.1:55000/agents?pretty&timeframe=2d&status=neverconnected,disconnected"
{
   "error": 0,
   "data": {
      "totalItems": 1,
      "items": [
         {
            "status": "Disconnected",
            "configSum": "ab73af41699f13fdd81903b5f23d8d00",
            "group": "default",
            "name": "agent005",
            "mergedSum": "d9835ca466a5f6ede52e0684537f76bd",
            "ip": "any",
            "node_name": "node01",
            "dateAdd": "2018-05-20 09:55:29",
            "version": "Wazuh v3.2.3",
            "manager_host": "manager",
            "lastKeepAlive": "2018-05-20 09:55:29",
            "os": {
               "major": "7",
               "name": "CentOS Linux",
               "uname": "Linux |manager |3.10.0-693.17.1.el7.x86_64 |#1 SMP Thu Jan 25 20:13:58 UTC 2018 |x86_64",
               "platform": "centos",
               "version": "7",
               "codename": "Core",
               "arch": "x86_64"
            },
            "id": "005"
         }
      ]
   }
}

Added timeframe and status filter to DELETE /agents. It's possible to delete agents not connected at a specific time with a specific status. By default, timeframe is 7d (7 days). Samples:

Remove all agents that have been disconnected for the last week, or added more than a week ago and are never connected:

$ date
Tue May 22 10:09:57 UTC 2018

$ curl -u foo:bar -k -X GET "http://127.0.0.1:55000/agents/003?pretty"
{
   "error": 0,
   "data": {
      "status": "Disconnected",
      "configSum": "ab73af41699f13fdd81903b5f23d8d00",
      "name": "agent003",
      "mergedSum": "d9835ca466a5f6ede52e0684537f76bd",
      "ip": "any",
      "dateAdd": "2018-05-15 10:09:09",
      "version": "Wazuh v3.2.3",
      "manager_host": "manager",
      "lastKeepAlive": "2018-05-15 10:09:09",
      "os": {
         "major": "7",
         "name": "CentOS Linux",
         "uname": "Linux |manager |3.10.0-693.17.1.el7.x86_64 |#1 SMP Thu Jan 24 10:13:58 UTC 2018 |x86_64",
         "platform": "centos",
         "version": "7",
         "codename": "Core",
         "arch": "x86_64"
      },
      "id": "003"
   }
}

$ curl -u foo:bar -k -X DELETE "http://127.0.0.1:55000/agents?pretty&status=disconnected,neverconnected"
{
   "error": 0,
   "data": {
      "msg": "All selected agents were removed",
      "affected_agents": [
         "003"
      ]
   }
}

Remove all agents from a list that have not been active from more than 1 day:

$ date
Tue May 22 10:14:41 UTC 2018

$ curl -u foo:bar -k -X GET "http://127.0.0.1:55000/agents?pretty&timeframe=1d"       
{
   "error": 0,
   "data": {
      "totalItems": 3,
      "items": [
         {
            "status": "Never connected",
            "dateAdd": "2018-05-21 09:27:49",
            "name": "agent001",
            "ip": "any",
            "id": "001",
            "node_name": "unknown"
         },
         {
            "status": "Disconnected",
            "configSum": "ab73af41699f13fdd81903b5f23d8d00",
            "name": "agent003",
            "mergedSum": "d9835ca466a5f6ede52e0684537f76bd",
            "ip": "any",
            "node_name": "node01",
            "dateAdd": "2018-05-21 09:28:08",
            "version": "Wazuh v3.2.3",
            "manager_host": "manager",
            "lastKeepAlive": "2018-05-15 10:09:09",
            "os": {
               "major": "7",
               "name": "CentOS Linux",
               "uname": "Linux |manager |3.10.0-693.17.1.el7.x86_64 |#1 SMP Thu Jan 24 10:13:58 UTC 2018 |x86_64",
               "platform": "centos",
               "version": "7",
               "codename": "Core",
               "arch": "x86_64"
            },
            "id": "003"
         },
         {
            "status": "Never connected",
            "dateAdd": "2018-05-21 09:28:19",
            "name": "agent004",
            "ip": "any",
            "id": "004",
            "node_name": "unknown"
         }
      ]
   }
}

$ curl -u foo:bar -k -X DELETE -H "Content-Type:application/json" -d '{"ids":["003","004","006"]}' "http://127.0.0.1:55000/agents?pretty&timeframe=1d"
{
   "error": 0,
   "data": {
      "msg": "Some agents were not removed",
      "failed_ids": [
         {
            "id": "006",
            "error": {
               "message": "Agent is not eligible for removal: The agent has a status different to 'all' or the specified timeframe '1d' does not apply.",
               "code": 1731
            }
         }
      ],
      "affected_agents": [
         "003",
         "004"
      ]
   }
}

Delete all agents never connected:

$ /var/ossec/bin/agent_control -l

Wazuh agent_control. List of available agents:
   ID: 000, Name: manager (server), IP: 127.0.0.1, Active/Local
   ID: 002, Name: agent002, IP: any, Disconnected
   ID: 005, Name: agent005, IP: any, Disconnected
   ID: 007, Name: agent_test1, IP: any, Never connected
   ID: 008, Name: agent_nc1, IP: any, Never connected
   ID: 009, Name: agent_nc2, IP: any, Never connected

List of agentless devices:

$ curl -u foo:bar -k -X DELETE "http://127.0.0.1:55000/agents?pretty&status=neverconnected&timeframe=1s"
{
   "error": 0,
   "data": {
      "msg": "All selected agents were removed",
      "affected_agents": [
         "007",
         "008",
         "009"
      ]
   }
}
$ /var/ossec/bin/agent_control -l

Wazuh agent_control. List of available agents:
   ID: 000, Name: manager (server), IP: 127.0.0.1, Active/Local
   ID: 002, Name: agent002, IP: any, Disconnected
   ID: 005, Name: agent005, IP: any, Disconnected

List of agentless devices:

Regards.

#79

mgmacias95 · 2018-05-22T16:13:27Z

Hello @Lifka,

Please, update the changelog and the mocha tests with these changes.

Also, filtering by active status, also returns pending agents. Filtering by pending status only returns pending agents:

# curl -u foo:bar "localhost:55000/agents?pretty&status=Active"
{
   "error": 0,
   "data": {
      "totalItems": 3,
      "items": [
         {
            "status": "Active",
            "name": "localhost.localdomain",
            "ip": "127.0.0.1",
            "node_name": "node01",
            "dateAdd": "2018-05-23 09:17:40",
            "version": "Wazuh v3.2.3",
            "manager_host": "localhost.localdomain",
            "lastKeepAlive": "9999-12-31 23:59:59",
            "os": {
               "major": "7",
               "name": "CentOS Linux",
               "uname": "Linux |localhost.localdomain |3.10.0-693.21.1.el7.x86_64 |#1 SMP Wed Mar 7 19:03:37 UTC 2018 |x86_64",
               "platform": "centos",
               "version": "7",
               "codename": "Core",
               "arch": "x86_64"
            },
            "id": "000"
         },
         {
            "status": "Active",
            "configSum": "ab73af41699f13fdd81903b5f23d8d00",
            "group": "default",
            "name": "perico",
            "mergedSum": "d9835ca466a5f6ede52e0684537f76bd",
            "ip": "any",
            "node_name": "node01",
            "dateAdd": "2018-05-23 09:17:40",
            "version": "Wazuh v3.2.3",
            "manager_host": "localhost.localdomain",
            "lastKeepAlive": "2018-05-23 09:29:47",
            "os": {
               "major": "7",
               "name": "CentOS Linux",
               "uname": "Linux |localhost.localdomain |3.10.0-693.el7.x86_64 |#1 SMP Tue Aug 22 21:09:27 UTC 2017 |x86_64",
               "platform": "centos",
               "version": "7",
               "codename": "Core",
               "arch": "x86_64"
            },
            "id": "001"
         },
         {
            "status": "Pending",
            "dateAdd": "2018-05-23 09:27:32",
            "name": "josefa",
            "ip": "any",
            "lastKeepAlive": "2018-05-23 09:25:47",
            "id": "004",
            "node_name": "unknown"
         }
      ]
   }
}
# curl -u foo:bar "localhost:55000/agents?pretty&status=pending"
{
   "error": 0,
   "data": {
      "totalItems": 1,
      "items": [
         {
            "status": "Pending",
            "dateAdd": "2018-05-23 09:27:32",
            "name": "josefa",
            "ip": "any",
            "lastKeepAlive": "2018-05-23 09:25:47",
            "id": "004",
            "node_name": "unknown"
         }
      ]
   }
}

I think the problem is the database query used to filter active agents:

query += '(last_keepalive >= :time_active or id = 0) OR '

Note that calculate_status function considers "pending" agents all those whose keep alive would be "active" but have not sent their "version" yet:

    @staticmethod
    def calculate_status(last_keep_alive, pending, today=datetime.today()):
        """
        Calculates state based on last keep alive
        """
        if not last_keep_alive:
            return "Never connected"
        else:
            limit_seconds = 1830 # 600*3 + 30
            # divide date in format YY:mm:dd HH:MM:SS to create a datetime object.
            last_date = datetime(year=int(last_keep_alive[:4]), month=int(last_keep_alive[5:7]), day=int(last_keep_alive[8:10]),
                                hour=int(last_keep_alive[11:13]), minute=int(last_keep_alive[14:16]), second=int(last_keep_alive[17:19]))
            difference = (today - last_date).total_seconds()

            return "Disconnected" if difference > limit_seconds else ("Pending" if pending else "Active")

One more thing, we're moving input validation to the framework level (more info: #80 (comment)). In the API is ok to use the "alphanumeric" filter. It's in the framework where the semantic validation takes place. In fact, it's already implemented.

The default timeframe when removing agents is 7 days. It would be nice to add the used timeframe to the response to avoid situations like the following:

# curl -u foo:bar "localhost:55000/agents?pretty&status=pending"
{
   "error": 0,
   "data": {
      "totalItems": 1,
      "items": [
         {
            "status": "Pending",
            "dateAdd": "2018-05-23 09:27:32",
            "name": "josefa",
            "ip": "any",
            "lastKeepAlive": "2018-05-23 10:45:47",
            "id": "004",
            "node_name": "unknown"
         }
      ]
   }
}
# curl -u foo:bar -XDELETE "localhost:55000/agents?pretty&status=pending"
{
   "error": 0,
   "data": {
      "msg": "All selected agents were removed",
      "affected_agents": []
   }
}
# curl -u foo:bar -XDELETE "localhost:55000/agents?pretty&status=pending&timeframe=10m"
{
   "error": 0,
   "data": {
      "msg": "All selected agents were removed",
      "affected_agents": [
         "004"
      ]
   }
}

Best regards,
Marta

Lifka · 2018-05-24T08:33:04Z

Done.
Thanks for the review @mgmacias95!

jesuslinares · 2018-05-24T14:35:06Z

Great job!

Some changes:

timeframe -> older_than
older_than can't be used alone. It must be used with status (We don't want to remove Active agents easily)

Thanks.

Lifka · 2018-05-24T16:44:56Z

Done!
timeframe is now older_than.
older_than can never be used alone:

$ curl -u foo:bar -k -X DELETE "http://127.0.0.1:55000/agents?pretty"
{
   "error": 604,
   "message": "Filter error. Missing field: You have to specified 'ids' or status."
}

$ curl -u foo:bar -k -X DELETE "http://127.0.0.1:55000/agents?pretty&older_than=1s"
{
   "error": 604,
   "message": "Filter error. Missing field: You have to specified 'ids' or status."
}

Thank you @jesuslinares!

The last commit also fixes a bug handling python exceptions in agents functions:

Before:

$ curl -u foo:bar -k -X DELETE  "http://127.0.0.1:55000/agents?pretty&older_than=1s&status=neverconnected"
{
   "error": 1000,
   "message": "Wazuh-Python Internal Error: 'exceptions.Exception' object has no attribute 'code'"
}

Then:

$ curl -u foo:bar -k -X DELETE  "http://127.0.0.1:55000/agents?pretty&older_than=1s&status=neverconnected"
{
   "error": 0,
   "data": {
      "msg": "Some agents were not removed",
      "failed_ids": [
         {
            "id": "001",
            "error": {
               "message": "Sample exception",
               "code": 1000
            }
         },
         {
            "id": "002",
            "error": {
               "message": "Sample exception",
               "code": 1000
            }
         },
         {
            "id": "005",
            "error": {
               "message": "Sample exception",
               "code": 1000
            }
         }
      ],
      "older_than": "1s",
      "affected_agents": []
   }
}

Lifka · 2018-05-24T16:45:01Z

Summary:

Requests for remove agents:

DELETE/agents: Removes agents, using a list of them or a criterion based on the status or time of the last connection.
- ids: List of agents.
- status: Filters by agent status. It's possible filter by several status using commas as a separator. Status can be active, disconnected, pending or neverconnected.
- older_than: Filters out disconnected agents for longer than specified. Time in seconds or [n_days]d[n_hours]h[n_minutes]m[n_seconds]s. For never connected agents, uses the register date. By default is 7 days.
DELETE/agents/:agent_id: Remove an agent.

Also, the filters status and older_than are availables in GET/agents.

Samples:

Remove all agents that have been disconnected for the last week, or added more than a week ago and are never connected:

$ date
Thu May 24 16:01:53 UTC 2018

$ curl -u foo:bar -k -X GET "http://127.0.0.1:55000/agents/004?pretty"
{
   "error": 0,
   "data": {
      "status": "Disconnected",
      "name": "agent004",
      "ip": "any",
      "dateAdd": "2018-05-17 16:01:44",
      "lastKeepAlive": "2018-05-17 16:01:44",
      "id": "004"
   }
}

$ curl -u foo:bar -k -X DELETE "http://127.0.0.1:55000/agents?pretty&status=disconnected,neverconnected"
{
   "error": 0,
   "data": {
      "msg": "All selected agents were removed",
      "older_than": "7d",
      "affected_agents": [
         "004"
      ]
   }
}

Remove all agents from a list that have not been active from more than 1 day:

$ date
Thu May 24 16:10:33 UTC 2018

$ curl -u foo:bar -k -X GET "http://127.0.0.1:55000/agents?pretty&status=Disconnected&older_than=1d&select=id,name,last_keepalive"
{
   "error": 0,
   "data": {
      "totalItems": 2,
      "items": [
         {
            "lastKeepAlive": "2018-05-23 16:08:11",
            "name": "ag-ubuntu",
            "id": "007"
         },
         {
            "lastKeepAlive": "2018-05-23 16:08:17",
            "name": "ag-redhat",
            "id": "008"
         }
      ]
   }
}

$ curl -u foo:bar -k -X DELETE -H "Content-Type:application/json" -d '{"ids":["004", "007"]}' "http://127.0.0.1:55000/agents?pretty&older_than=1d&status=disconnected"
{
   "error": 0,
   "data": {
      "msg": "Some agents were not removed",
      "failed_ids": [
         {
            "id": "004",
            "error": {
               "message": "Agent is not eligible for removal: The agent has a status different to 'disconnected' or the specified time frame 'older_than 1d' does not apply.",
               "code": 1731
            }
         }
      ],
      "older_than": "1d",
      "affected_agents": [
         "007"
      ]
   }
}

Remove all agents never connected:

$ /var/ossec/bin/agent_control -l

Wazuh agent_control. List of available agents:
   ID: 000, Name: manager (server), IP: 127.0.0.1, Active/Local
   ID: 001, Name: agent001, IP: any, Never connected
   ID: 002, Name: agent002, IP: any, Pending
   ID: 003, Name: agent003, IP: any, Never connected
   ID: 004, Name: agent004, IP: any, Pending
   ID: 005, Name: agent005, IP: any, Pending
   ID: 006, Name: agent106, IP: any, Active
   ID: 007, Name: ag-ubuntu, IP: any, Disconnected
   ID: 008, Name: ag-redhat, IP: any, Disconnected

List of agentless devices:

$ curl -u foo:bar -k -X DELETE "http://127.0.0.1:55000/agents?pretty&older_than=1s&status=neverconnected"
{
   "error": 0,
   "data": {
      "msg": "All selected agents were removed",
      "older_than": "1s",
      "affected_agents": [
         "001",
         "003"
      ]
   }
}
$ /var/ossec/bin/agent_control -l

Wazuh agent_control. List of available agents:
   ID: 000, Name: manager (server), IP: 127.0.0.1, Active/Local
   ID: 002, Name: agent002, IP: any, Pending
   ID: 004, Name: agent004, IP: any, Pending
   ID: 005, Name: agent005, IP: any, Pending
   ID: 006, Name: agent106, IP: any, Active
   ID: 007, Name: ag-ubuntu, IP: any, Disconnected
   ID: 008, Name: ag-redhat, IP: any, Disconnected

List of agentless devices:

Regards!

Lifka added 2 commits May 21, 2018 19:11

Fix agents purge

56c8c4a

#79

Add wrong status error

5d8d819

Lifka requested review from snaow and jesuslinares May 22, 2018 11:13

Lifka mentioned this pull request May 22, 2018

Fix agents purge wazuh/wazuh#650

Merged

Update changelog and mocha test, remove status validation

5ab3689

Lifka added 2 commits May 24, 2018 17:30

Replace "timeframe" by "older_than" and improve documentation

1f02e95

older_than can never be used alone in DELETE/agent request

97ae6fc

Lifka force-pushed the fix-agents-purge branch from 1be504a to b39c380 Compare May 25, 2018 13:00

Update changelog and remove input validation error

907806e

Lifka force-pushed the fix-agents-purge branch from b39c380 to 907806e Compare May 25, 2018 13:29

jesuslinares merged commit 3af250c into 3.2 May 25, 2018

jesuslinares deleted the fix-agents-purge branch May 25, 2018 13:56

mgmacias95 mentioned this pull request Jun 19, 2018

Change default "older_than" parameter to 1s in DELETE/agents. #113

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix agents purge #82

Fix agents purge #82

Lifka commented May 22, 2018 •

edited

Loading

mgmacias95 commented May 22, 2018 •

edited

Loading

Lifka commented May 24, 2018

jesuslinares commented May 24, 2018

Lifka commented May 24, 2018

Lifka commented May 24, 2018

Fix agents purge #82

Fix agents purge #82

Conversation

Lifka commented May 22, 2018 • edited Loading

Remove

Add

mgmacias95 commented May 22, 2018 • edited Loading

Lifka commented May 24, 2018

jesuslinares commented May 24, 2018

Lifka commented May 24, 2018

Lifka commented May 24, 2018

Summary:

Samples:

Lifka commented May 22, 2018 •

edited

Loading

mgmacias95 commented May 22, 2018 •

edited

Loading