Cortex Analysers problem

[imad-testing]

Hello, I'm new to cortex. I am encountering this problem whenever use my analysers. Please can anyone help me?

[garanews]

Hello, welcome on board!
Please describe your environment and your config file

[imad-testing]

I am running my docker-compose file on ubuntu 18.04:

docker-compose.yml:

version: "2"
services:

kibana:
image: kibana:5.6.0
container_name: kibana
ports:
- "5601:5601"

elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.6.0
container_name: elasticsearch
environment:
- http.host=0.0.0.0
- transport.host=0.0.0.0
- xpack.security.enabled=false
- cluster.name=hive
- script.inline=true
- thread_pool.index.queue_size=100000
- thread_pool.search.queue_size=100000
- thread_pool.bulk.queue_size=100000
ulimits:
nofile:
soft: 65536
hard: 65536
volumes:
- 1esdata:/usr/share/elasticsearch/data
ports:
- "9200:9200"

thehive:
image: thehiveproject/thehive:latest
container_name: thehive
depends_on:
- elasticsearch
- cortex
ports:
- "0.0.0.0:9000:9000"
volumes:
- './thehive/:/etc/thehive/'
command: --no-config-secret
command: --cortex-port 9001 --cortex-key tQqgFug13614Zf+6yvfoqC1K92SG1B8E --no-config

cortex:
container_name: cortex
image: thehiveproject/cortex:latest
ports:
- "0.0.0.0:9001:9001"
volumes:
- './cortex/:/etc/cortex/'
- './cortex/Cortex-Analyzers/analyzers:/opt/Cortex-Analyzers/analyzers:ro'
- './cortex/Cortex-Analyzers/responders:/opt/Cortex-Analyzers/responders:ro'
command: --no-config
command: --no-config-secret
depends_on:
- elasticsearch

volumes:
1esdata:

The geo.py for maxmind is:

#!/usr/bin/env python3
#encoding: utf-8
import os
import geoip2.database
from geoip2.errors import AddressNotFoundError
from cortexutils.analyzer import Analyzer

class MaxMindAnalyzer(Analyzer):

def dump_city(self, city):
    return {
        'confidence': city.confidence,
        'geoname_id': city.geoname_id,
        'name': city.name,
        'names': city.names
    }

def dump_continent(self, continent):
    return {
        'code': continent.code,
        'geoname_id': continent.geoname_id,
        'name': continent.name,
        'names': continent.names,
    }

def dump_country(self, country):
    return {
        'confidence': country.confidence,
        'geoname_id': country.geoname_id,
        'iso_code': country.iso_code,
        'name': country.name,
        'names': country.names
    }

def dump_location(self, location):
    return {
        'accuracy_radius': location.accuracy_radius,
        'latitude': location.latitude,
        'longitude': location.longitude,
        'metro_code': location.metro_code,
        'time_zone': location.time_zone
    }

def dump_traits(self, traits):
    return {
        'autonomous_system_number': traits.autonomous_system_number,
        'autonomous_system_organization': traits.autonomous_system_organization,
        'domain': traits.domain,
        'ip_address': traits.ip_address,
        'is_anonymous_proxy': traits.is_anonymous_proxy,
        'is_satellite_provider': traits.is_satellite_provider,
        'isp': traits.isp,
        'organization': traits.organization,
        'user_type': traits.user_type
    }

def summary(self, raw):
    taxonomies = []
    level = "info"
    namespace = "MaxMind"
    predicate = "Location"

    if "continent" in raw:
        value = "{}/{}".format(raw["country"]["name"], raw["continent"]["name"])
        taxonomies.append(self.build_taxonomy(level, namespace, predicate, value))

    return {"taxonomies": taxonomies}

def run(self):
    Analyzer.run(self)

    if self.data_type == 'ip':
        try:
            data = self.get_data()

            city = geoip2.database.Reader(os.path.dirname(__file__) + '/GeoLite2-City.mmdb').city(data)

            self.report({
                'city': self.dump_city(city.city),
                'continent': self.dump_continent(city.continent),
                'country': self.dump_country(city.country),
                'location': self.dump_location(city.location),
                'registered_country': self.dump_country(city.registered_country),
                'represented_country': self.dump_country(city.represented_country),
                'subdivisions': self.dump_country(city.subdivisions.most_specific),
                'traits': self.dump_traits(city.traits)
            })
        except ValueError as e:
            self.error('Invalid IP address')
        except AddressNotFoundError as e:
            self.error('Unknown IP address')
        except Exception as e:
            self.unexpectedError(type(e))
    else:
        self.notSupported()

if name == 'main':
MaxMindAnalyzer().run()

the json file for maxmind:

{
"name": "MaxMind_GeoIP",
"version": "4.0",
"author": "CERT-BDF",
"url": "https://github.com/TheHive-Project/Cortex-Analyzers",
"license": "AGPL-V3",
"description": "Use MaxMind to geolocate an IP address.",
"dataTypeList": ["ip"],
"baseConfig": "MaxMind",
"command": "MaxMind/geo.py"
}

It does not seem the issue is from maxmind specifically, because the analyser also failed when i tried it on abuseIPDB

[imad-testing]

In addition to that the docker-compose.yml file and the cortex file are here:

My application.conf:

application.zip

But it do not seem like i'm using my application.conf file to run cortex. I'm sorry the environment is so new to me

[garanews]

Thanks.
passing command: --no-config means that you don't want cortex uses default application.conf file.
This is fine, but you should map an external application.conf file for example:

   volumes:
      - ./cortex/application.conf:/etc/cortex/application.conf```

[imad-testing]

Thank you. Do you mean that I should map the external application in the cortex section in docker-compose.yml? And if I did that, what will change? I'm sorry for my questions I'm trying to understand.

I was able to solve the issue anw, it was a permission problem on the .py that could no be able to execute

[garanews]

Good.
With external application.conf you have more control, in case you need to configure different elastic server instead localhost, authentication methofds, etc.
To map it, as i suggested, map directly the application.conf file: - ./cortex/application.conf:/etc/cortex/application.conf

For your info, if you want to start to play with new versions of tools like thehive4 (that uses cassandra as db) and cortex 3.1 (that uses elastic 7.x instead 6,x) you can have a look here: https://github.com/TheHive-Project/TheHive/tree/develop-th4/docker
In this docker compose there is also MISP and an automation tool, that you can remove if not needed.

[imad-testing]

Oh great! Thank you so much! Will keep you posted about any news or problems that I will face in the future.

[dadokkio]

I'm going to close the issue, if you have any other issue reopen it.