SerenityGPT

[fatihozdemir23]

Hello, as it is known, ChatGpt can be customized for special purposes. It would be great if an artificial intelligence tool called SerenityGPT was developed within Serenity. I believe that with this tool we can solve many of our technical problems and find answers to our how-to questions.

[VictorTomaili]

Managing such as gpt is has different difficulties.

We can use OpenAI GPTs functionality. Then users can use this GPT from the ChatGPT. Will be easiest solution but not efficient because telling and describing how to use serenity to ChatGPT with system prompt and adding documentations to there is not enough. Because serenity uses rolling release and this data should be updated on nearly every commit. And there are huge number of documents and data exist including discussions and issues in serenity repository.

We can fine tune a code model like Code Llama etc. But most of the opensource code models are trained heavily in python code. So, we need to put too much effort into creating a nice code model in csharp and typescript. Maybe LoRA like adapters can be the easiest solution, I haven't tried yet. Also, we need to access at least NVIDIA GPU like A100 and more. We didn't have such a GPU and experience with large language models. This means we need to fail many times to achieve nice results.

Even If we can create this model, we need to serve this model as a service. This means we need some hardware requirements to serve this model for many customers. Server means cost, and cost is increasing while user count rising. This means at least we need to announce a monthly subscription plan for this model. And maintain this model continuously to achieve a good result.

We are not an AI software company, (at least for now). So, we need more experience to achieve this type of chat bot or copilot like model and good user experience. We don't want to release a model or use a service which responds with low quality responses. Many services including OpenAI are hallucinate about Serenity code. And this means there are too many wrong responses about serenity related questions. We tried this already.

I have some plans, but they are not guaranteeing this type of chat bot will be released in the future. For the short term, we can release an OpenAI integration sample in StartSharp. Maybe the sample can include a RAG and function call functionality. I need to work on these. They are ideas for now.