Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GAIA: include table size in the class TapTableMeta #2970

Conversation

cosmoJFH
Copy link
Contributor

User tables do not display information on their physical size on disk. This makes account of the user quota difficult. That attribute should be visible to Astroquery users.

A new test was made.

cc @esdc-esac-esa-int

Jira: GAIAPCR-1036

Copy link

codecov bot commented Mar 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 66.81%. Comparing base (5f54ec6) to head (04fb9cc).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2970   +/-   ##
=======================================
  Coverage   66.81%   66.81%           
=======================================
  Files         237      237           
  Lines       18319    18324    +5     
=======================================
+ Hits        12239    12244    +5     
  Misses       6080     6080           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@bsipocz bsipocz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test data file needs to be trimmed, otherwise this looks all OK.

Thanks!

@@ -74,11 +104,13 @@ def test_job_results_parser():
file.close()


def __check_table(table, baseName, numColumns, columnsData):
qualifiedName = f"public.{baseName}"
def __check_table(table, qualifiedName, numColumns, columnsData, size_bytes=-1):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's inconsequential here as this is private and in the tests, but generally, we do go with None defaults rather than something custom invalid filler, e.g. -999, or -1 as here, etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you. I've updated the default value to None.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to add such a big file for the purpose of one reasonably straightforward test? (it's 10+% of the size of the whole package). Could you trim it back to contain hardly much more than what the tests actually use from it, or come up with a similar dummy file? (and then we need to overwrite the history, so it's not entering in the git history either)

Copy link
Contributor Author

@cosmoJFH cosmoJFH Mar 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I trimmed the file, since the entire file was not required for the test.

CHANGES.rst Outdated
@@ -211,6 +211,9 @@ gaia

- Default Gaia catalog updated to DR3. [#2596]

- Include table size in the class TapTableMeta returned by the functions load_tables and load_table, in the class Tap.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changelog needs to be moved to the new release section

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand where should I include this information.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an almost empty 0.4.8 section at the very top of the file, but I can move this one (and those from your other PRs right before merging them)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for you answer. I included the information for the PR in the section you told me.

@bsipocz bsipocz added this to the v0.4.8 milestone Mar 19, 2024
@cosmoJFH
Copy link
Contributor Author

cosmoJFH commented Mar 23, 2024

We have reviewed the comments in #2863 and we would like to use this PR to update the documentation for the functions load_table and load_tables.

@bsipocz bsipocz force-pushed the ESA_gaia_include_table_size_in_TapTableMeta branch from 5915fbe to 04fb9cc Compare April 6, 2024 03:52
Copy link
Member

@bsipocz bsipocz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went ahead and rebased the PR to squash the commits about the new file together, so the big one doesn't end up in the git history.

Merging it now if CI is all green.

Thanks!

@bsipocz bsipocz merged commit cf7d2a7 into astropy:main Apr 6, 2024
10 checks passed
bsipocz added a commit that referenced this pull request Apr 19, 2024
…ead_http_response_GAIAPCR-1308

GAIA fix the method read_http_response to retrieve json files recently included in  #2970
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants