Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document and guard against cost pitfalls #174

Closed
5 tasks done
dylanbannon opened this issue Sep 11, 2019 · 3 comments
Closed
5 tasks done

Document and guard against cost pitfalls #174

dylanbannon opened this issue Sep 11, 2019 · 3 comments
Assignees
Labels
documentation A change in documentation is requested enhancement New feature or request

Comments

@dylanbannon
Copy link
Contributor

dylanbannon commented Sep 11, 2019

Functionality:

  • When starting a cluster, ensure that the chosen bucket is in the same region as the cluster, and warn the user if not.
  • Ensure that the bucket is not multi-regional (?)
  • Upon cluster deletion, confirm whether user is ok with cleaning out files from chosen bucket. If so, delete all uploads and outputs, but not the bucket itself.

Documentation:

  • Add notes about the potential cost premiums for not having the bucket in the same zone as the cluster (or having bucket be "interzone"?) and not deleting files after run ends.
  • Include instructions on manually checking the regions of buckets and clusters, and deleting files manually from a bucket.
@dylanbannon dylanbannon self-assigned this Sep 11, 2019
@dylanbannon dylanbannon added the documentation A change in documentation is requested label Sep 11, 2019
@dylanbannon dylanbannon changed the title Document cost pitfalls Document and guard against cost pitfalls Sep 11, 2019
@dylanbannon dylanbannon added the enhancement New feature or request label Sep 11, 2019
@willgraf
Copy link
Contributor

Upon cluster deletion, confirm whether user is ok with cleaning out files from chosen bucket. If so, delete all uploads and outputs, but not the bucket itself.

This was resolved by the bucket-monitor update. Currently, the bucket monitor will crawl the bucket every INTERVAL (which is configurable), and remove all files from a set of prefixes ("/uploads" and "/output" by default) that are older than 3 days (also configurable).

This should prevent files from building up and costing extra 💰💰💰.

@willgraf
Copy link
Contributor

willgraf commented Apr 12, 2020

We can also build a warning in the menu to help notify users that they might end up having interzon egress charges with a multi-regional bucket.

We cannot "ensure" that they do not have a mutlizone bucket, there is a use for that. I think the best we can do is inform them that a multizone bucket will cost extra money. This warning is addressed in #329

@willgraf
Copy link
Contributor

Closed by #329

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation A change in documentation is requested enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants