Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate actions during their creation, store only configVersion in the Job schema #1341

Open
despadam opened this issue Jul 29, 2024 · 4 comments
Assignees
Labels
Release Jobs Jobs migration

Comments

@despadam
Copy link

despadam commented Jul 29, 2024

Based on the comment under PR #1286 and discussing afterwards with @sofyalaski:

  • There is no need for method validate() in the action class. Validation should happen inside the constructor of each action.
  • Since we are now using a global job configuration - instead of job specific - we should only store the configuration version inside each job in the db.
  • Before create: No need for additional logic. If there is any error while constructing the actions, the application fails. When all actions have been initialized successfully, then each job that gets created will do so by automatically using the current global configuration.
  • Before statusUpdate: We need to check if the configVersion is the same. If no changes were made to the configuration, then we can proceed directly to performing the update. But if the configuration has changed, would we need to update the value that is stored in the db? Note that when updating the configuration, the application needs to get restarted, so the actions have already been reconstructed automatically with the new configuration values.
@despadam despadam self-assigned this Jul 29, 2024
@despadam despadam added the Release Jobs Jobs migration label Jul 29, 2024
@despadam despadam changed the title Validate actions during their creation and check whether the configuration version matches on Job creation and statusUpdate Validate actions during their creation Aug 5, 2024
@despadam despadam changed the title Validate actions during their creation Validate actions during their creation, store only configVersion in the Job schema Aug 7, 2024
@sbliven
Copy link
Contributor

sbliven commented Aug 12, 2024

I'm back from my vacation and just seeing this.

Validation of the job config file needs to happen in the constructor, as you say.

The reason for the validate() method on actions was to allow validation of CreateJobDto.jobParams. For instance, one could imagine a job where users have to supply an authentication token when creating a job, which then gets validated by URLAction before making the request.

None of the MVP actions used jobParams (other than the datasetIds list) so including validate() in the abstract interface was more about future-proofing the API than from an immediate need. But if we're removing validate() then we should probably remove jobParams itself (making datasetIds a top-level field as proposed elsewhere).

@sofyalaski
Copy link
Contributor

sofyalaski commented Aug 19, 2024

@sbliven I was thinking about that validate() function when I stumbled upon this enum JobsConfigSchema. According to this enum, job wouldn't have the jobParams, but only datasetIds, so essentially the case you propose. However, in the dto the jobParams is the required parameter. We should make it consistent. We could introduce another enum for job parameters, that would include datasetIds but I don't know what else this enum should include for now

@despadam
Copy link
Author

despadam commented Sep 3, 2024

configVersion in the job schema: has been resolved by PRs #1364 and #1395

TODO: validation action

@sbliven
Copy link
Contributor

sbliven commented Oct 7, 2024

validate() restored in #1421

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Release Jobs Jobs migration
Projects
None yet
Development

No branches or pull requests

3 participants