fix output error when parallel_requests > 1 and supports_multiple_generations is False

[arjun-krishna1]

Resolve: #861 (comment)

• FIX Probes fail with nim generator when parallel_requests > 1
• Error: TypeError: expected string or bytes-like object, got 'list'
• When a generator's 'supports_multiple_generations' is false
• And the 'parallel_requests' > 1,
• Ensure the generator output is added as a string, not a list.

[arjun-krishna1]

This branch fixes error: TypeError: expected string or bytes-like object, got 'list'
On main when nim generator is run with parallel_requests > 1

(.venv) arjun@arjun-ThinkPad-L14-Gen-3:~/garak$ python -m garak -m nim -n nvidia/nemotron-mini-4b-instruct -p xss --parallel_requests 2
garak LLM vulnerability scanner v0.9.0.15.post1 ( https://github.com/leondz/garak ) at 2024-08-29T19:19:55.972963
📜 logging to /home/arjun/.local/share/garak/garak.log
🦜 loading generator: NIM: nvidia/nemotron-mini-4b-instruct
📜 reporting to /home/arjun/.local/share/garak/garak_runs/garak.8a08484d-3ff3-4714-ac76-e79d9058d497.report.jsonl
🕵️  queue of probes: xss.MarkdownImageExfil
Traceback (most recent call last):                                                                                                                                                              
  File "<frozen runpy>", line 198, in _run_module_as_main                                                                                                                                       
  File "<frozen runpy>", line 88, in _run_code
  File "/home/arjun/garak/garak/__main__.py", line 14, in <module>
    main()
  File "/home/arjun/garak/garak/__main__.py", line 9, in main
    cli.main(sys.argv[1:])
  File "/home/arjun/garak/garak/cli.py", line 509, in main
    command.probewise_run(
  File "/home/arjun/garak/garak/command.py", line 222, in probewise_run
    probewise_h.run(generator, probe_names, evaluator, buffs)
  File "/home/arjun/garak/garak/harnesses/probewise.py", line 107, in run
    h.run(model, [probe], detectors, evaluator, announce_probe=False)
  File "/home/arjun/garak/garak/harnesses/base.py", line 122, in run
    d.detect(attempt)
  File "/home/arjun/garak/garak/detectors/xss.py", line 28, in detect
    results.append(1.0 if re.search(_MARKDOWN_REGEX, output) else 0.0)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/re/__init__.py", line 177, in search
    return _compile(pattern, flags).search(string)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected string or bytes-like object, got 'list'
(.venv) arjun@arjun-ThinkPad-L14-Gen-3:~/garak$ git checkout bugfix/nim-generator-fix-expected-string-got-list 
Switched to branch 'bugfix/nim-generator-fix-expected-string-got-list'
(.venv) arjun@arjun-ThinkPad-L14-Gen-3:~/garak$ python -m garak -m nim -n nvidia/nemotron-mini-4b-instruct -p xss --parallel_requests 2
garak LLM vulnerability scanner v0.9.0.15.post1 ( https://github.com/leondz/garak ) at 2024-08-29T19:21:42.243693
📜 logging to /home/arjun/.local/share/garak/garak.log
🦜 loading generator: NIM: nvidia/nemotron-mini-4b-instruct
📜 reporting to /home/arjun/.local/share/garak/garak_runs/garak.4b3ce712-1647-49e1-a998-8ed1e8b68718.report.jsonl
🕵️  queue of probes: xss.MarkdownImageExfil
xss.MarkdownImageExfil                                                        xss.MarkdownExfilBasic: PASS  ok on  120/ 120                                                                     
xss.MarkdownImageExfil                                                      xss.MarkdownExfilContent: PASS  ok on  120/ 120                                                                     
📜 report closed :) /home/arjun/.local/share/garak/garak_runs/garak.4b3ce712-1647-49e1-a998-8ed1e8b68718.report.jsonl
📜 report html summary being written to /home/arjun/.local/share/garak/garak_runs/garak.4b3ce712-1647-49e1-a998-8ed1e8b68718.report.html
✔️  garak run complete in 94.20s
(.venv) arjun@arjun-ThinkPad-L14-Gen-3:~/garak$ 

[leondz]

Thanks for this. It looks like there is an issue around outputs being built when parallel_requests > 1, because outputs should be a list of strings and Nones, but the output of _call_model (a list of strings/nones) is appended to it.

Recommendations:

• Rename this PR to be about parallel_requests
• Check that _call_model is returning just one result at a time in the Pool content
• Move the tests out of test_nim

Changes addressed in 8187035-3551a34