Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoiding issues with analyst data objects #1307

Merged
merged 12 commits into from
Dec 20, 2024
Merged

Conversation

chrisr3d
Copy link
Member

@chrisr3d chrisr3d commented Oct 30, 2024

Using the different PyMISP helpers to add for instance an Attribute or an Object was causing issues when the given Attribute or Object was containing at least a Note, Opinion or Relationship. This was caused by the presence of the 'Note', 'Opinion' and 'Relationship' fields in every data layer inheriting from AnalysDataBehaviorMixin (including Notes, Opinions or Relationships themselves)

On the other hand, a Note, Opinion or Relationship information extracted from MISP is always contained in a 'Note', 'Opinion' or 'Relationship' fields, respectively.
A note extracted from MISP looks for instance like:

{
    "Note": {
        "uuid": "848337f8-781c-4998-8575-4fa4661b2a90",
        "object_uuid": "dbb85655-ec62-42b6-b05e-7085ee7af504",
        "object_type": "Attribute",
        "authors": "foo@bar.org",
        "created": "2024-10-03 14:13:56",
        "modified": "2024-10-03 14:13:56",
        "note": "Some random note",
        "language": "en"
    }
}

This makes the test for the presence of such fields mandatory when we want to use our Analyst Data information from MISP to created the equivalent objects in PyMISP.

With this PR, we reconcile both use cases with an additional contained parameter that is only used with the from_dict method of each Analyst Data object, set to True when we use add_note, add_opinion or add_relationship, in which case the analyst data information is obviously attached to another data layer.

This should fix #1306

UPDATE:

PyMISP is now supporting flat lists. Previously nested Opinions and Notes are now attached to the parent container (Attribute, Event, Object, etc.) and the reference to the object uuid is kept to allow a nested display on MISP UI

Users still can keep the logic of adding notes/opinions to a note or an opinion, but they are added to the list of analyst data objects to the parent container.

For instance:

from pymisp import MISPAttribute, MISPOpinion

attribute = MISPAttribute()
attribute.from_dict(type='domain', value='circl.lu')

note1 = attribute.add_note(note='note1')
note2 = note1.add_note(note='note2')
attribute.notes[1].add_note(note='note3')

my_opinion = MISPOpinion()
my_opinion.from_dict(opinion=50, comment='my opinion on note2', object_type='Note', object_uuid=note2.uuid)
note2.add_opinion(my_opinion)

The Attribute has flat lists of Analyst Data objects:

{
  "Note": [
    {
      "uuid": "094a0d61-d01a-4338-b431-99aed49f8b01",
      "note": "note1",
      "object_uuid": "a1daff2f-62a1-4541-832c-098c30d8082b",
      "object_type": "Attribute"
    },
    {
      "uuid": "ee049478-b5d7-438a-85f0-13290da49b19",
      "note": "note2",
      "object_uuid": "094a0d61-d01a-4338-b431-99aed49f8b01",
      "object_type": "Note"
    },
    {
      "uuid": "d5fac1eb-ae1f-46bc-863b-44cd2a1019fe",
      "note": "note3",
      "object_uuid": "ee049478-b5d7-438a-85f0-13290da49b19",
      "object_type": "Note"
    }
  ],
  "Opinion": [
    {
      "uuid": "5c63b1f0-d19b-4ff5-ab9a-2ea94b2979df",
      "opinion": "50",
      "comment": "my opinion on note2",
      "object_uuid": "ee049478-b5d7-438a-85f0-13290da49b19",
      "object_type": "Note"
    },
  ],
  "uuid": "a1daff2f-62a1-4541-832c-098c30d8082b",
  "type": "domain",
  "value": "circl.lu",
  "category": "Network activity",
  "to_ids": true,
  "disable_correlation": false
}

…s on MISP standard format

- Adding a note or an opinion will always add the
  new analyst data object to the list of notes or
  opinions at the parent data layer level
- `from_dict` on a JSON blob is also able to parse
  properly analyst data and generate flat lists
  regardless of whether the given data described
  in the new flat or previously nested format
- Additional checks for parent to support both
  the standalone and attached analyst data objects
- Standalone Analyst data objects with nested
  notes or opinions are defined with the nesting
  as they have no parent. When they are added to
  a parent data layer, the nested objects are then
  flattened
- Testing different ways to attach analyst data
- Testing that no matter what object type the
  analyst data is attached to, the `object_type`
  & `object_uuid` are correct, and the parent
  container does contain every analyst data object
  in flat lists with no nesting
@Rafiot Rafiot merged commit 23b5d3a into MISP:main Dec 20, 2024
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error when adding a Note, Opinion or Relationship
2 participants