Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract ip addresses #34

Merged
merged 12 commits into from
May 25, 2024
Merged

Extract ip addresses #34

merged 12 commits into from
May 25, 2024

Conversation

thomasyu888
Copy link
Member

@thomasyu888 thomasyu888 commented Apr 15, 2024

Problem
It would be nice if we gathered the individual IP address information so we can make interesting analysis around cost and usage.

Solution
Leveraging a free API to extract the IP information and pushing it to a snowflake table.

Admittedly, the code is a bit hacky right now...

Copy link

Quality Gate Passed Quality Gate passed

Issues
8 New issues
0 Accepted issues

Measures
1 Security Hotspot
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@thomasyu888 thomasyu888 marked this pull request as ready for review April 17, 2024 03:14
ip_info_response = requests.post('http://ip-api.com/batch', json=ip_list)
except Exception as err:
print(ip_info_response.status_code)
if ip_info_response.status_code == 429:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Future change would be to use the backoff package.

https://pypi.org/project/backoff/

Copy link
Contributor

@BryanFauble BryanFauble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving script in its current state since I know it's a hack to get the first round of data in the door. If these becomes an airflow DAG the items noted should be addressed.

@thomasyu888 thomasyu888 requested a review from a team as a code owner May 25, 2024 19:57
Copy link

Quality Gate Passed Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
1 Security Hotspot
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@thomasyu888 thomasyu888 merged commit 714dddc into dev May 25, 2024
3 checks passed
@thomasyu888 thomasyu888 deleted the extract-ip-addresses branch May 25, 2024 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants