enable tokenizer customization in HFDetector

[jmartin-tech]

Commonize parameters utilized in HFDetector, existing detectors currently use common arguments.

• promote tokenizer_kwargs as DEFAULT_PARAM of parent class
• remove __init()__ overrides not longer needed

Expands on earlier exposure of model selection for HFDetectors in #810

These values seem independent from hf_args used to load the model.

Some additional code review suggests that misleading.MustContradictNLI does not actually consume these values since it overrides detect() with a custom method. Should this PR be expanded to consume values from self.tokenizer_kwargs for max_length and truncation? The values are currently hardcoded in detect() as local values. It does not look like padding is ever consumed in misleading.MustContradictNLI and could be suppressed with instance defaults for that class.

[leondz]

I suspect max_length might want to come from model ctx_len - but this is out of scope for this PR

[erickgalinkin]

I'd actually exclude max_length since it's very specifically model/config dependent and this might override defaults. If the model's max_length is, for some reason <512, this default could throw errors.

[leondz]

On the other hand, running without a length cap also risks breaking models (this happens regularly on e.g. some NVCF instances). I think it's OK to offer a default control over this with a mild value that balances both being too high and too low. 512 seems reasonable (bearing in mind that detectors tend to be smaller models - this is a side effect of garak needing to run them at scale, but sometimes having primary compute allocated to the target model, reducing detector resources).

[erickgalinkin]

LGTM

[erickgalinkin]

I'd actually exclude max_length since it's very specifically model/config dependent and this might override defaults. If the model's max_length is, for some reason <512, this default could throw errors.

Did not intend to approve

[leondz]

On the other hand, running without a length cap also risks breaking models (this happens regularly on e.g. some NVCF instances). I think it's OK to offer a default control over this with a mild value that balances both being too high and too low. 512 seems reasonable (bearing in mind that detectors tend to be smaller models - this is a side effect of garak needing to run them at scale, but sometimes having primary compute allocated to the target model, reducing detector resources).