Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding best practices and assertion checks to workflow_lint #1213

Merged
merged 4 commits into from
Feb 4, 2022

Conversation

simonbray
Copy link
Member

Fixes #1178, #1181, #1211

This was linked to issues Jan 12, 2022
if not found_valid_expectation:
lint_context.warn("Found no valid test expectations for workflow test")
test_valid = False

return test_valid


def _check_test_assertions(lint_context, assertion_definitions):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this can be improved, if someone knows more about importing Python functions than me?

@simonbray simonbray changed the title Adding best practices and assertion checks to workflow_lint [WIP] Adding best practices and assertion checks to workflow_lint Jan 12, 2022
@simonbray simonbray force-pushed the best-practices branch 2 times, most recently from 49bda50 to 2fe2f6b Compare January 13, 2022 13:32
@simonbray simonbray changed the title [WIP] Adding best practices and assertion checks to workflow_lint Adding best practices and assertion checks to workflow_lint Jan 14, 2022
for module in asserts.assertion_modules:
for function_name in dir(module):
if function_name.split('assert_')[-1] in assertion_definitions:
f = module.__dict__[function_name]
Copy link
Member

@mvdbeek mvdbeek Jan 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could check that this is a function with https://docs.python.org/3/library/inspect.html#inspect.isfunction
and inspect the signature with https://docs.python.org/3/library/inspect.html#inspect.signature instead of relying on TypeError. OTOH that's some manual parsing that is maybe not necessary. A healthy middleground could be to run https://docs.python.org/3/library/inspect.html#inspect.Signature.bind, so we don't have to execute the test function.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

run https://docs.python.org/3/library/inspect.html#inspect.Signature.bind, so we don't have to execute the test function.

That is cool, thanks!

Copy link
Member

@mvdbeek mvdbeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really cool, thanks for all the tests!

Comment on lines +221 to +231
for function_name in dir(module):
if function_name.split('assert_')[-1] in assertion_definitions:
signature = inspect.signature(module.__dict__[function_name])
try:
# try mapping the function with the attributes supplied and check for TypeError
signature.bind('', **assertion_definitions[function_name.split('assert_')[-1]])
except AssertionError:
pass
except TypeError as e:
lint_context.error(f'Invalid assertion in tests: {function_name} {str(e)}')
assertions_valid = False
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for function_name in dir(module):
if function_name.split('assert_')[-1] in assertion_definitions:
signature = inspect.signature(module.__dict__[function_name])
try:
# try mapping the function with the attributes supplied and check for TypeError
signature.bind('', **assertion_definitions[function_name.split('assert_')[-1]])
except AssertionError:
pass
except TypeError as e:
lint_context.error(f'Invalid assertion in tests: {function_name} {str(e)}')
assertions_valid = False
for function_name, function in asserts.assertion_functions.items():
if function_name.split('assert_')[-1] in assertion_definitions:
signature = inspect.signature(function)
try:
# try mapping the function with the attributes supplied and check for TypeError
signature.bind('', **assertion_definitions[function_name.split('assert_')[-1]])
except AssertionError:
pass
except TypeError as e:
lint_context.error(f'Invalid assertion in tests: {function_name} {str(e)}')
assertions_valid = False

When galaxyproject/galaxy#12528 is incorporated into a published version of galaxy-tool-util this change will be needed.

@mvdbeek mvdbeek merged commit 4158b37 into galaxyproject:master Feb 4, 2022
@simonbray simonbray deleted the best-practices branch February 4, 2022 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Workflow Linting Documentation Lint workflow test file Linting of workflow best practices
2 participants