Multi-GPU support for fine-tuning

[eckartal]

Problem

• Fine-tuning large models is resource-intensive, and users need multi-GPU support to distribute the workload efficiently
• Jan currently lacks this capability, limiting performance for those with multiple GPUs.

Success Criteria

• Implement multi-GPU support for fine-tuning.
• Ensure efficient workload distribution across GPUs.
• Optimize memory usage to prevent bottlenecks.

Additional Context

• Helps users fine-tune QLoRAs and other models faster.
• Reduces reliance on a single GPU, improving performance.

Source