Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(fetch): delete --deep option #51

Merged
merged 11 commits into from
Aug 9, 2021

Conversation

MaineK00n
Copy link
Collaborator

@MaineK00n MaineK00n commented Jun 20, 2021

What did you implement:

The --deep option was to look for CVE information in offensive-security/exploitdb. However, it is an implementation that executes a lot of REST APIs, and it took 22m53s for the whole fetch with --deep option. (Without the --deep option, it was 38s, so the deep option was too slow.

Basically, I believe that the information I want can be obtained from the following two CSVs. These information can be inserted without using the deep option.

Refs #7

Fixed a bug that prevented the opening of links shown in the inserted data.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

delete --deep option

// PR
go-exploitdb fetch exploitdb --deep
unknown flag: --deep

fix link

// PR
sqlite> SELECT * FROM documents WHERE exploit_unique_id = 3219;
offensive_security_id|exploit_unique_id|document_url|description|author|type|platform|port
1|3219|https://github.com/offensive-security/exploitdb/blob/master/exploits/osx/local/3219.rb|Apple Mac OSX 10.4.8 (8L2127) - 'crashdump' Local Privilege Escalation|MoAB|local|osx|

$ curl -D - -s -o /dev/null https://github.com/offensive-security/exploitdb/blob/master/exploits/osx/local/3219.rb
HTTP/2 200

// upstream/master
sqlite> SELECT * FROM documents WHERE exploit_unique_id = 3219;
offensive_security_id|exploit_unique_id|document_url|description|author|type|platform|port
2911|3219|https://github.com/offensive-security/exploitdb/exploits/osx/local/3219.rb|Apple Mac OSX 10.4.8 (8L2127) - 'crashdump' Local Privilege Escalation|MoAB|local|osx|

$ curl -D - -s -o /dev/null https://github.com/offensive-security/exploitdb/exploits/osx/local/3219.rb
HTTP/2 404

Checklist:

You don't have to satisfy all of the following.

  • Write tests
  • Write documentation
  • Check that there aren't other open pull requests for the same issue/feature
  • Format your source code by make fmt
  • Pass the test by make test
  • Provide verification config / commands
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: NO

Reference

@MaineK00n MaineK00n changed the title [WIP] feat(fetch): delete --deep option feat(fetch): delete --deep option Aug 8, 2021
@kotakanbe kotakanbe merged commit eb8625d into vulsio:master Aug 9, 2021
@kotakanbe kotakanbe self-requested a review August 9, 2021 00:15
@kotakanbe
Copy link
Collaborator

LGTM

@MaineK00n MaineK00n deleted the delete-deep-option branch August 9, 2021 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants