-
-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a UAT registry constraint #649
base: main
Are you sure you want to change the base?
Conversation
This is for somewhat formal searches against semantic concepts.
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #649 +/- ##
==========================================
- Coverage 82.47% 82.30% -0.17%
==========================================
Files 72 72
Lines 7480 7506 +26
==========================================
+ Hits 6169 6178 +9
- Misses 1311 1328 +17 ☔ View full report in Codecov by Sentry. |
Most of them are unrelated to the UAT constraint, but they seem so minor that I'd say an extra PR won't buy us anything.
aa36f45
to
8869ab7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some comments for the failing tests
docs/registry/index.rst
Outdated
@@ -154,7 +170,7 @@ interactive data discovery, however, it is usually preferable to use the | |||
|
|||
And to look for tap resources *in* a specific cone, you would do | |||
|
|||
.. doctest-remote-data:: | |||
.. doctest-remote-data:: # doctest: +IGNORE_OUTPUT |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This won't work, the #doctest commands should be added at the end of the lines (each one of them) that are affected. But this example already has it.
class TestUATConstraint: | ||
def test_basic(self): | ||
cons = rtcons.UAT("solar-flares") | ||
assert (cons.get_search_condition(FAKE_GAVO) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we have some tests that query the remote service rather than the mock version? (OTOH, if you say the doctests should be enough for that, I would be signing off on it)
.. doctest-remote-data:: | ||
|
||
>>> resources = registry.search( | ||
... registry.UAT("cepheid-variable-stars", expand_down=3)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Somehow this throws an error, even though cepheid-variable-stars
are in the list on the referred website
_____________________________ [doctest] index.rst ______________________________
102 constraints is generally preferable with more complex queries.
103 An advantage of using explicit constraints is that you can pass
104 additional parameters to the constraints. For instance, the UAT
105 constraint can optionally expand your keyword to narrower or wider
106 concepts. When looking for resources talking about Cepheids of all
107 kinds, you can thus say:
108
109 .. doctest-remote-data::
110
111 >>> resources = registry.search(
UNEXPECTED EXCEPTION: DALQueryError: cepheid-variable-stars does not identify an IVOA uat concept (see http://www.ivoa.net/rdf/uat).
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/doctest.py", line 1355, in __run
exec(compile(example.source, filename, "single",
File "<doctest index.rst[4]>", line 2, in <module>
File "/home/runner/work/pyvo/pyvo/.tox/py311-test-alldeps-online/lib/python3.11/site-packages/pyvo/registry/rtcons.py", line 713, in __init__
raise dalq.DALQueryError(
pyvo.dal.exceptions.DALQueryError: cepheid-variable-stars does not identify an IVOA uat concept (see http://www.ivoa.net/rdf/uat).
/home/runner/work/pyvo/pyvo/docs/registry/index.rst:111: UnexpectedException
=========================== short test summary info ============================
137b765
to
80e53e8
Compare
It was a bad idea to have a compact UAT as a local resource; it poisons astropy's download cache and is then hard to get rid of again, in particular because pyvo caches the vocabularies in RAM. In consequence, the unit tests for the UAT constraint are now marked as using the network. It does, too, when you run the tests in a freshly made environment. I'd still like to keep the commit with the fixture in the history. Perhaps we want something like that elsewhere some day.
80e53e8
to
86d99bd
Compare
On Thu, Feb 13, 2025 at 09:20:00AM -0800, Brigitta Sipőcz wrote:
> @@ -154,7 +170,7 @@ interactive data discovery, however, it is usually preferable to use the
And to look for tap resources *in* a specific cone, you would do
-.. doctest-remote-data::
+.. doctest-remote-data:: # doctest: +IGNORE_OUTPUT
This won't work, the #doctest commands should be added at the end of the lines (each one of them) that are affected. But this example already has it.
Ah... hu... Sorry about this. I wonder why I felt the need to shut
this one up, then. I've reverted that change.
Can we have some tests that query the remote service rather than
the mock version? (OTOH, if you say the doctests should be enough
for that, I would be signing off on it)
Well... I would have preferred using a mock vocabulary because we
shouldn't be testing the UAT as such. But as the comment to the
following commit says, having artificial mock vocabularies is tricky
due to the heavy caching involved. Hence, the tests now to use the
remote vocabulary, and the expense of always pulling the UAT (700k in
desise) in new installations. That's moderately painful for CIs (but
free for local tests). But then the doctest would have pulled the
full UAT anyway, so that's what it is now.
+ >>> resources = registry.search(
+ ... registry.UAT("cepheid-variable-stars", expand_down=3))
Somehow this throws an error, even though `cepheid-variable-stars`
are in the list on the referred website
Yeah, that was a stale cache with the mock vocabulary.
Thanks for the early review. It think this PR is now good to go.
Could you have another look? Thanks!
|
The test failures look relevant, lots of VocabularyErrors |
This PR adds a constraint for doing keyword searches over the controlled vocabulary of the terms supposed to be in rr.res_subject. The rationale and a lot more is discussed in https://blog.g-vo.org/a-new-constraint-class-in-pyvo-s-registry-api-uat.html