experiment details

[hopelin99]

Thank you for providing such excellent work! I have a few questions I would like to consult with you.

1. would like to know if you will release the training scripts.
2. Have you conducted experiments involving the segmentation of different layers of the LLM?
3. Have you conducted a comparison between fine-tuning with and without the inclusion of the CAGIT dataset?
4. I noticed that you used different batch sizes for CAGIT and image-text data. Did you perform a two-stage training process, first using image-text data and then fine-tuning with CAGIT?

I'm looking forward to your answers to these questions. Thank you.

[YYJMJC]

Thank you for your interest and very insightful questions. We are currently in the process of organizing the training code and haven't completed some systematic exploration experiments (e.g., different injection layers, different proportions of CAGIT data and image-text data...) . Once this phase is finished, we will update the paper and release the code. To train the CLORI module, we now perform one-stage training with mixed data.