Direct tabulation from Dominion input

[HEdingfield]

Per discussion in #408, it'd be nice to be able to tabulate Dominion inputs directly. This would likely do the conversion to generic .csv behind-the-scenes, and include those intermediary files as additional output.

Relevant discussion from Slack:

@HEdingfield:
Isn't our end goal to make the Dominion conversion process kind of behind-the-scenes? So instead of creating a separate button / area in the GUI for it, wouldn't it be better to just make the normal load function handle Dominion files? Or is this too advanced / too much work?

@moldover
so you’re right that, we do want to support Dominion as a first-class data source (just as we do for ES&S)
however, we expose the conversion process as a way to verify the conversion process itself.
we’ll need to do the same for any data source which has non-trivial input logic.
so to your question, ‘wouldn’t it be better to just support it as a data source’ …
yes, that would be better.
we could read from data sources AND export the converted csv files for verification
best of both worlds

@HEdingfield
Couldn't that just include providing the converted file as another output along with the results?

@moldover
right
but there’s a few more wrinkles to consider -
Dominion data sources specify multiple contest Ids
Dominion data sources specify multiple contests (by Id)
and tabulation process is done against for a single contest at a time
(yes, one could tabulate all the contests for an election “at once”, but in actual practice, that is not what election admins do. at least according to Caleb.)
so actually getting all that figured out in a reasonable way is I think, a much bigger chunk of work, than just… plugging in a new fileReader to a new data source
but that said, I do think we should do it
so to recap: the “convert cvr” functionality is needed on it’s own right.

@tarheel
so if we wanted to support Dominion as a regular input, we would need the user to specify a contest ID param in the config also

@moldover
something like that, yes
the dem primaires only have one contest, but take SF - it has like 20 or 30 contests
Maybe it’s not that much work?

[gngilbert]

Can't thank you guys enough for all the work you have done. Truly amazing to those of us looking in from the "outside!"

I wholly agree with the need for direct tabulation from Dominion input while retaining the conversion to .csv format for auditing and transparency.

One issue I am not sure about is the question of tabulating multiple contests vs. single contests. While the current practice is to tabulate contests one at a time, I can foresee that changing as administrators become more accustomed to RCV. No need to worry about it now, but let's keep the question on the table for future consideration.

Thanks @HEdingfield , @tarheel , @moldover , @CalebKleppner , @freedomcounts

[HEdingfield]

Per our Slack conversation today, this should be prioritized for the next release so we can avoid providing awkward instructions about using the "convert Dominion to generic format" functionality, opening that .csv in Excel, saving it as a .xlsx, then calling it an ES&S input in the config.

[gngilbert]

I agree 100%!

…

On Mon, Jul 13, 2020, 11:34 PM HEdingfield ***@***.***> wrote: Per our Slack conversation today, this should be prioritized for the next release so we can avoid providing awkward instructions about using the "convert Dominion to generic format" functionality, opening that .csv in Excel, saving it as a .xlsx, then calling it an ES&S input in the config. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#438 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJODE7SWUFEC7F4GCN4IFNDR3PG55ANCNFSM4LTVDSNQ> .

[HEdingfield]

According to #456, we need the contestId field for Dominion as well as Hart. The PR that added this (#463) includes language in the .txt file and validation checks that only allow contestId to work with Hart. We'll need to make sure to expand this when implementing Dominion.