Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ADD] Add the ability to batch upload IOC by providing a CSV file #26

Merged
merged 3 commits into from
Jan 2, 2022
Merged

[ADD] Add the ability to batch upload IOC by providing a CSV file #26

merged 3 commits into from
Jan 2, 2022

Conversation

wagga40
Copy link
Contributor

@wagga40 wagga40 commented Jan 2, 2022

This PR :

  • allows to upload a lot of IOC at once by providing a CSV import
  • add an upload dialog that shows expected format, provide an example and a sample CSV (check screenshots in Discord)

Please note that :

  • during the process no file is uploaded to the backend, the CSV in sent to the backend as a string
  • TLP are checked and converted into their respective database id
  • provided data is compared to the db schema
  • types are not verified (cf. below)
  • recent modifications in branch "i25_ioc_management" will have impact on this code

@whikernel whikernel self-assigned this Jan 2, 2022
@whikernel whikernel added the enhancement New feature or request label Jan 2, 2022
@whikernel
Copy link
Contributor

Nice !
There's a small issue when we upload a CSV with duplicates entries. It generates a BG alert and stops importing.
It's coming from the returns in the loop. We'll just change that to log.error I guess and continue to the next item.
We'll fix it with the merge with i25.
Otherwise super useful feature, thanks a lot !

whikernel
whikernel reviewed Jan 2, 2022
View reviewed changes
source/app/blueprints/case/case_ioc_routes.py
ioc = add_ioc_schema.load(row)

if ioc.ioc_type not in choices_ioc_types:
return response_error("Not a valid IOC type")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To change to log.error to prevent break of the loop

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Set a list with the failed IOCs and return to UI for awarness

whikernel
whikernel reviewed Jan 2, 2022
View reviewed changes
source/app/blueprints/case/case_ioc_routes.py
link_existed = add_ioc_link(ioc.ioc_id, caseid)

if link_existed:
return response_error("IOC already exists and linked to this case")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idem as above https://github.com/dfir-iris/iris-web/pull/26/files#r777227267

@whikernel whikernel merged commit d06bea6 into dfir-iris:develop Jan 2, 2022
whikernel added a commit that referenced this pull request Jan 2, 2022
@whikernel
[MRG] Merge branch 'i25_ioc_management' into develop
ffeaed0 
[FIX] Conflict between #25 and #26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@whikernel whikernel whikernel left review comments

Assignees

@whikernel whikernel

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@wagga40 @whikernel