Add validate_model_tbl() convenience function to validate data frame before writing a file.

[zkamvar]

The impetus for this are the issues encountered in reichlab/variant-nowcast-hub#116

In this, there were problems with the submission, but it's not immediately clear if this was due to the model generating character NAs or if it was due to a problem with file I/O.

At the moment, to validate a submission, we have to write the submission to a hub model-outputs folder and run validate_submission(). This is a good and working setup and I am suggesting a convenience function that validate_submission() can wrap and separate the concerns of validating submission data as opposed to metadata.

By giving the option of validate_model_tbl(), then people can generate the data and validate before writing to disk and they can iterate more rapidly.

It would also help something like reichlab/variant-nowcast-hub/src/sample_submission_data.R demonstrate that the structure of the data is valid. At the moment, whoever runs the script must change the output directory to the model-outputs folder and then clean up the extra file they created in the meantime. If the validate_model_tbl() function were available, then the data could be generated and validated and there would be no mess to clean up or scripts to modify.

It would also allow people to run validations on subsets of their model output (e.g. subsets of optional task ID variables).

[zkamvar]

I've gotten feedback from the modelers in the Variant Nowcast Hub that this would be a desired feature.

[zkamvar]

I'm moving this to the wishlist because, after reviewing the code involved in the validations, this cannot be easily implemented for two big reasons:

1. without a hub, custom functions cannot be run
2. many of the validation functions need to validate against the config file and thus are designed to run with hub_path as an input so that the config file can be fetched via read_config().

One potential solution would be to implement a hub class of object that can be created with either hub path or configuration files that could store and carry around the contents of the configuration files, but that would involve a lot more work than this function is worth.

Part of the reason this issue came up is the fact that modelers often do not clone the hubs locally because the hubs can grow quite large due to all the data in the model outputs. One way to alleviate this burden is for modelers to use git sparse-checkout:

git init hub-name && cd hub-name
git remote add origin git@github.com:org-name/hub-name.git
git sparse-checkout set --no-cone '/*' '!/model-output/'
git pull origin main

This can cut the size of the repo on a local machine by a significant amount (though the size might still be quite large because sparse-checkout still downloads the git database).

[annakrystalli]

I'm wondering if our new functionality for reading config files from GitHub or the cloud might help here? Although this would not address the inability to run custom checks which would need a lot more work to enable.

Having said that one thing I want to avoid is code duplication but parcelling this up into a sub-function of validate_model_data() is not sth I'm really excited about tbh.

I definitely think the best way forward would be the sparse git checkout and ideally using the sparse repo to submit too. If the file is written out, the modellers can perform the full range of checks via validate_submission(), including file naming, file location and custom validations which I think we should try and encourage as much as possible.

A couple things that could potentially help is:

1. a wrapper function for easily creating sparse local repos
2. a detailed walkthrough of how to write out, commit, push and open a PR from within a hub sparse repo. (would a wrapper for function for this also potentially be useful? eg submit_model_out()?