Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tweak sensor value setter to validate dtype #98

Merged
merged 14 commits into from
Sep 3, 2024

Conversation

amishatishpatel
Copy link
Contributor

@amishatishpatel amishatishpatel commented Jul 25, 2024

Apologies for the unannounced feedback request @ludwigschwardt @KimMcAlpine.

We spotted something a while back where aiokatcp's Sensor.set_value didn't validate
the dtype of the value coming in to update. Bruce has made me quite aware of the ripple
effect a change like this could make (and likely deconstructive interference). Tagging you
both here mainly to

  • Draw your attention to this proposed update, and
  • Let me know if there's a way I could test this for you.

There is still potential to consolidate the sensor-type attributes (Sensor.stype).

Whenever you can, please and thank you.

Contributes to: NGC-1062.

Still potential to consolidate the sensor-type attributes.

Contributes to: NGC-1062.
@coveralls
Copy link

coveralls commented Jul 25, 2024

Coverage Status

coverage: 92.721% (+0.06%) from 92.665%
when pulling 8ac62db on NGC-1062-sensor-value-setter
into a519d63 on master.

Copy link
Contributor

@bmerry bmerry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still need to refresh my memory of how the encoding/decoding works in aiokatcp, but here are some minor suggestions so long.

src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
Copy link
Contributor

@ludwigschwardt ludwigschwardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except for the small suggestion, I'm happy. I found aiokatcp.Sensor.set_value in these SDP bits:

  • katsdpcontroller
  • katsdpingest
  • katsdpcal
  • kattelmod
  • various *writers

Nothing in there suggests problems but I guess we'll see :-)

src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
Copy link
Contributor

@bmerry bmerry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few corner cases that aren't working:

  • The default parameter to the Sensor constructor isn't going through the check.
  • Setting a discrete sensor to an enum of a different Enum class to the intended one doesn't complain.
  • Code that sets the value of a Timestamp sensor using a float used to work fine, but now complains. I think we probably want to support that, since in Python land, Timestamp is really just an alias for float. It's only a different type so that (a) ?sensor-list reports the sensor type correctly and (b) the encoding rules for timestamps are stricter (although that's currently not implemented). I think that can probably be handled as a special case: if the sensor type is Timestamp and the value is an instance of numbers.Real, cast it to Timestamp.

Additionally, you should write unit tests to verify that the new code is raising the exception.

As per comments on PR #98, to check the default value and rework logic
to account for cases missed in the naive check previously.

Contributes to: NGC-1062.
I'd missed it in the previous update.

Contributes to: NGC-1062.
@amishatishpatel
Copy link
Contributor Author

My updates since the initial review certainly aren't meant to look like anything polished. I was just fed up with trying to neatly wrap the logical checks.

Similarly, the added unit test is very naive/plain.

  • I had initially used numpy.intXX and numpy.floatXX to set the int and float sensor values, but
  • The pre-commit config reminded me numpy isn't actually used/a dependency for aiokatcp
  • So I removed my one-off usages.

src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
src/aiokatcp/sensor.py Show resolved Hide resolved
src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Show resolved Hide resolved
As per @bmerry's comments on #98. Also add numpy to the list of
(optional) dependencies, used in the unit test.

Contributes to: NGC-1062.
To ensure it conforms to the ipaddress.{IPv4, IPv6}Address type.

Contributes to: NGC-1062.
tests/test_sensor.py Outdated Show resolved Hide resolved
@amishatishpatel
Copy link
Contributor Author

amishatishpatel commented Aug 21, 2024

Re: Unit tests failure on (ubuntu-20.04, 3.12), I was under the impression I need to ensure compatibility with 3.8(.10)

  • Which is why I pinned the numpy version to 1.24.4 (in pre-commit-config)
  • I see the run failed because of some import-related issues, do I need to debug this before requesting another round of reviewing @bmerry ?
    • EDIT: Looking back at it, I just assumed it was numpy's fault, didn't quite parse the issue correctly. Either way, please do let me know.

@bmerry
Copy link
Contributor

bmerry commented Aug 21, 2024

I see the run failed because of some import-related issues, do I need to debug this before requesting another round of reviewing @bmerry ?

Urgh, I've had similar issues with spead2: there is no version of numpy that installs on both 3.8 and 3.12. Request a review so long (to remind me) and I'll think about how best to handle this when I review. It's tempting to say we should only bother testing the numpy stuff on one version, but numpy has also made subtle tweaks to its type system over time (and less subtle in the 1.x -> 2.0 transition) so we probably should test more than one version.

@amishatishpatel
Copy link
Contributor Author

amishatishpatel commented Aug 22, 2024

I see the run failed because of some import-related issues, do I need to debug this before requesting another round of reviewing @bmerry ?

Urgh, I've had similar issues with spead2: there is no version of numpy that installs on both 3.8 and 3.12. Request a review so long (to remind me) and I'll think about how best to handle this when I review. It's tempting to say we should only bother testing the numpy stuff on one version, but numpy has also made subtle tweaks to its type system over time (and less subtle in the 1.x -> 2.0 transition) so we probably should test more than one version.

Noted all around. I figured it had something to do with the version conundrum. Let me know if I should create a technical-debt ticket for that.

src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
As per @bmerry's comments on #98, rework the unit test to be a bit
neater and check a few more cases. Also remove the dependency on numpy,
and use the builtin fractions module instead.

Contributes to: NGC-1062.
tests/test_sensor.py Outdated Show resolved Hide resolved
Copy link
Contributor

@bmerry bmerry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good - minor comments and ideas for more tests at this point.

I think we'll need to do an update to the documentation to explain what can and can't be used to set sensor values, but that can be in a follow-up PR.

pyproject.toml Outdated Show resolved Hide resolved
src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
src/aiokatcp/core.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
Missed in a previous commit, apologies.

Contributes to: NGC-1062.
As per @bmerry's suggestion on #98.

Contributes to: NGC-1062
To more accurately reflect/describe what it does, as per @bmerry's
suggestion on #98.

Contributes to: NGC-1062.
As per @bmerry's suggestion on #98.

Contributes to: NGC-1062.
Copy link
Contributor

@bmerry bmerry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of very minor tweaks left.

tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Outdated Show resolved Hide resolved
tests/test_sensor.py Show resolved Hide resolved
src/aiokatcp/sensor.py Outdated Show resolved Hide resolved
As per @bmerry's suggestions on #98.

Contributes to: NGC-1062.
Copy link
Contributor

@bmerry bmerry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy, but coveralls isn't - see comment on a unit test to add to fix up the coverage.

@@ -79,6 +79,8 @@ class Address:
_IPV6_RE = re.compile(r"^\[(?P<host>[^]]+)\](:(?P<port>\d+))?$")

def __init__(self, host: _IPAddress, port: Optional[int] = None) -> None:
if not isinstance(host, typing.get_args(_IPAddress)):
raise TypeError(f"{host} is not of either {typing.get_args(_IPAddress)}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coveralls is failing and it looks like it's because there is no test coverage for this. Add a unit test that tries to construct an Address from the wrong type.

Specifically to make sure it is of the correct type.

Contributes to: NGC-1062.
@pytest.mark.parametrize("value", ["1.2.3.4"])
def test_bad_host(self, value) -> None:
with pytest.raises(TypeError):
_ = Address(value)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need to assign it to anything. See the test above for example.

Suggested change
_ = Address(value)
Address(value)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Man, and I was wondering about doing it the same way as parse_bad.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I think I'm going to change the 'bad value' passed in here (to 127.0.0.1) to follow suit with the rest of the tests. No need to be unnecessarily different.

As per @bmerry's comments on #98.

Contributes to: NGC-1062.
@amishatishpatel amishatishpatel merged commit b6d34df into master Sep 3, 2024
21 of 22 checks passed
@amishatishpatel amishatishpatel deleted the NGC-1062-sensor-value-setter branch September 3, 2024 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants