How do I set the variable "user" or "system" for AI inference

[Silverdutchman]

Hello,

I use Perplexity to connect Hoarder. But I get the following error message.

2024-06-30T08:42:33.745Z error: [inference][31] inference job failed: Error: 400 Last message must have role user.

it seems that Perplexity, that's how I interpretate it, need the variabele "role" to be set. My question is: how and where can I do that in the config?

[kamtschatka]

Perplexity is not on the supported list, what gives you reason to believe, that this should work?

[MohamedBassem]

@kamtschatka we support openAI compatible APIs and perplexity is one of them (https://docs.perplexity.ai/docs/getting-started)

[Silverdutchman]

@kamtschatka I never stated that I believed that Perplexity is supported, although it is an openAI compatible API that is supported by the Hoarder Team, that doesn't mean I should/could work out of the box. Hence the question, nothing more and nothing less 🙏

@MohamedBassem thank you for the clarification, it was the foundation on why I asked this.

[kamtschatka]

@MohamedBassem we should probably update that then and state that we only support OpenAI. It doesn't work for #137 and it does not work here.

[MohamedBassem]

@kamtschatka If it doesn't work for azure and doesn't work for perplexity, it doesn't mean that it doesn't work for other openAI compatible APIs.
I'd argue that the problem here is that perplexity is out of spec (or more restrictive) than openAI.

@Silverdutchman I don't think the problem here is a variable that you have to tune or a role that you have to set. It's probably because we're requesting the inference with the message having the role system here (https://github.com/hoarder-app/hoarder/blob/main/apps/workers/inference.ts#L46) while perplexity wants it to be user instead of system. I think user should work, I'm just a bit concerned if this might degrade the inference performance in any ways.

[MohamedBassem]

I'm happy to try to changing the role to user, I don't expect this to cause problems.