How difficult would it be to use another open source project as a chatbot front end?

[masonjames]

This project is picking up some steam and it's easy to see why. It replicates the ChatGPT interface nicely, has a couple other cool features and has a low barrier to entry:
Chatbot UI

What kinda obstacles would there be to leveraging this as a front-end experience for users with langchain opening up opportunities to add basically everything else?

I'm knew to langchain, but could you run it on a headless server and connect it to a Chatbot UI instance for the front end?

edit: fixed repo link for chabot ui

[sergerdn]

I wasn't aware of the project you mentioned earlier, but if it involves handling API requests to a server and maintaining certain middlewares, then integrating or replacing any functionality should not be a problem.

I'm not sure what you need, but in any case, you will require three things:

• A server to manage your database with custom vectors (if needed). Additionally, you should maintain your own custom scripts for populating data.
• A server to handle API requests from a some chat UI.
• A chat UI.

Your link has been broken. Here is the correct one: https://github.com/mckaywrigley/chatbot-ui

[sergerdn]

I install chatbot-ui using Docker and asked some questions.

https://github.com/mckaywrigley/chatbot-ui/blob/main/pages/api/chat.ts

The answers provided by ChatGPT may not cover everything, but I think they serve as a good starting point. If you use ChatGPT-4 and ask the right questions, you may find everything you need in as little as 30 minutes.

================== GPT answers below this line ==================

If your own server is responsible for handling all the functions previously performed by the tiktoken library, then you wouldn't need to use tiktoken in this case.

Instead, you would need to implement an appropriate tokenization method on your server, and then send the tokenized input to your AI model. You would receive a tokenized response from your AI model, which you could then decode and format to match the expected output format.

The specifics of how you implement tokenization on your server will depend on the technology stack you're using, but one common approach is to use a natural language processing (NLP) library that supports tokenization, such as the Natural Language Toolkit (NLTK) in Python.

In this case, you would send the user input text to your server, where it would be tokenized using whatever method you have chosen. The resulting tokenized text would then be sent to your AI model, which would generate a tokenized response. This tokenized response would be sent back to your server, where it would be decoded and formatted as needed, and then sent back to the client.

By implementing your own tokenization method and using your own server to handle input and output for the ChatGPT model, you can have full control over the entire process and ensure that it meets your specific needs and requirements.

To use your own server instead of OpenAI's, you will need to make some changes to the existing code. Here's how you can do it:

1. Remove the import statement for OpenAIStream and the OpenAIError:

   import { ChatBody, Message } from '@/types/chat';
   import { DEFAULT_SYSTEM_PROMPT } from '@/utils/app/const';
   import tiktokenModel from '@dqbd/tiktoken/encoders/cl100k_base.json';
   import { Tiktoken, init } from '@dqbd/tiktoken/lite/init';
   // @ts-expect-error
   import wasm from '../../node_modules/@dqbd/tiktoken/lite/tiktoken_bg.wasm?module';
   
   export const config = {
     runtime: 'edge',
   };
   
   const handler = async (req: Request): Promise<Response> => {
     try {
       const { model, messages, prompt } = (await req.json()) as ChatBody;
   
       await init((imports) => WebAssembly.instantiate(wasm, imports));
       const encoding = new Tiktoken(
         tiktokenModel.bpe_ranks,
         tiktokenModel.special_tokens,
         tiktokenModel.pat_str,
       );
   
       let promptToSend = prompt;
       if (!promptToSend) {
         promptToSend = DEFAULT_SYSTEM_PROMPT;
       }
   
       const prompt_tokens = encoding.encode(promptToSend);
   
       let tokenCount = prompt_tokens.length;
       let messagesToSend: Message[] = [];
   
       for (let i = messages.length - 1; i >= 0; i--) {
         const message = messages[i];
         const tokens = encoding.encode(message.content);
   
         if (tokenCount + tokens.length + 1000 > model.tokenLimit) {
           break;
         }
         tokenCount += tokens.length;
         messagesToSend = [message, ...messagesToSend];
       }
   
       encoding.free();
   
       // TODO: send messagesToSend to your own backend
       // and receive the response from your backend
       // const responseFromBackend = await fetch("https://your-backend-url.com", {
       //   method: "POST",
       //   body: JSON.stringify(messagesToSend)
       // });
   
       // const responseFromBackendJson = await responseFromBackend.json();
       // let responses: string[] = responseFromBackendJson.responses;
       const responses: string[] = ['Response from my server'];
   
       // Format the response to match the expected format
       let resultToSend = '';
       for (let i = responses.length - 1; i >= 0 ; i--) {
         resultToSend += `You: ${messagesToSend[i].content}\n`;
         resultToSend += `AI: ${responses[i]}\n\n`;
       }
       // remove the last newline
       resultToSend = resultToSend.slice(0, -1);
       resultToSend += '\n';
   
       return new Response(resultToSend);
     } catch (error) {
       console.error(error);
       return new Response('Error', { status: 500 });
     }
   };
   
   export default handler;

2. In the handler function, send the messagesToSend array to your own backend instead of the OpenAI API. You can use the fetch API to do this. The response you get from your backend should be an array of strings, containing the responses from your AI model. This array of strings should be stored in the responses variable.

3. Update the final response to be sent to the user to match the expected format. The format needs to have alternating lines of "You: [user message]" and "AI: [AI response]". To do this, iterate over the messagesToSend array and the responses array in reverse order, and concatenate the messages and responses into a single string.

4. Finally, return a Response object containing the formatted response.

That's it! With these changes, your ChatGPT code will now use your own server instead of OpenAI's API to generate responses. Remember to test your updated code thoroughly to make sure everything is working as expected.