This repo houses a collection of my random project ideas. I will be running experiments on my own AI crowdfunding platform DreamcatcherAI
Dynamic Document Retrieval in Retrieval Augmented Generation (RAG) Models (ReRag.md): The dynamic retrieval system in RAG models refreshes document queries based on their continued relevance, addressing issues of obsolescence in retrieved document matches during response generation. This approach optimizes response accuracy by utilizing the Atlas fine-tuning method and allowing query-document pairings to be pre-calculated, resulting in enhanced adaptability for multi-faceted queries. ReRag.md
Mixture of Adapters (MoA.md): The Mixture of Adapters system seeks to store and selectively merge model adapters based on their relevance to tasks, enhancing next token generation efficiency. This method uses the 'Contriever' mechanism for retrieval, taps into adapter error contributions as a training signal (similar to RAG systems), and further enhances performance by segmenting the training of specialized "expert adapters". MoA.md