Remove max_tokens from openai_chat.py

[boeckers]

In vanna, max_tokens is used to control the size of the prompt that is generated from the user question. This should be sufficiently large (defaults to 14k)
In openai, max_tokens defines the maximum length of the generated output (max. ca. 5k). This should be controlled by a different parameter (if at all)

[boeckers]

@zainhoda Can you please review?

[zainhoda]

Ah yeah I see the issue. max_tokens is being used for both input and output tokens. While I like the idea, I think this might be a breaking change for some people so we need to collect some more feedback.

@andreped do you have any thoughts on this?

I'll go ask around in the meantime

[andreped]

@andreped do you have any thoughts on this?

Sorry for the late reply. We currently override the relevant methods in our current solution, so we won't be affected by this.

This should be controlled by a different parameter (if at all)

I think the last detail is quite important. I am not so sure we should set max_tokens at all. If I recall correctly, this was necessary way back, as the max_tokens used was hardcoded to 500 throughout this class, hence there was added a feature to alter it.

Perhaps the default should be None in the constructor and then people can override this parameter if they would like? Thoughts, @zainhoda, @boeckers?