[BUG] Data shuffling is off by default outside of mosaic shuffling for training in the dataloader

[kimbayashanologatiniburgettoilalalourvr]

Hi there,

We'd like to report a bug, it appears that data shuffling is off in the dataloader outside of the mosaicing function (which unfortunately only has partial shuffling at the moment, it appears). This caused us a lot of issues when we were training with mosaicing off as it seems like the recommendation has been to turn on mosaicing in order to have pseudo-shuffled data during training, otherwise it does not happen.

Having fully shuffled data in this case should benefit even in the case of mosaicing since 1 out of the 4 images in the 4 mosaic is not shuffled, this actually can cause some very subtle problems (one can compensate with momentum, for example, but at a loss of final network accuracy).

To fix this on our end, we've added a shuffle = (mode == Train) line to our code and it has vastly improved performance in areas where we do not have (or cannot have) mosaicing on. We've found in practice that shuffling every epoch (even with preshuffled data) almost never loses anything (better coverage over the distribution) while gaining a lot.

Attached is the original line in reference.

yolov5/utils/datasets.py

Line 118 in 2993c3f

sampler=sampler,

[glenn-jocher]

@kimbayashanologatiniburgettoilalalourvr hi, thank you for your feature suggestion on how to improve YOLOv5 🚀!

The fastest and easiest way to incorporate your ideas into the official codebase is to submit a Pull Request (PR) implementing your idea, and if applicable providing before and after profiling/inference/training results to help us understand the improvement your feature provides. This allows us to directly see the changes in the code and to understand how they affect workflows and performance.

Please see our ✅ Contributing Guide to get started.

[kimbayashanologatiniburgettoilalalourvr]

Hi there Glenn,

I believe there's been some other suggestions within other threads about this particular instance, this was just a more detailed and solo thread about this particular issue to highlight it.

If you're seeing better performance, for example, with mosaicing-9, this may be a part of the reason why.

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[glenn-jocher]

@kimbayashanologatiniburgettoilalalourvr good news 😃! Your original issue may now be fixed ✅ in PR #5623 by @werner-duvaud. This PR turns on shuffling in the YOLOv5 training DataLoader by default, which was missing until now. This works for all training formats: CPU, Single-GPU, Multi-GPU DDP.

train_loader, dataset = create_dataloader(train_path, imgsz, batch_size // WORLD_SIZE, gs, single_cls,
                                          hyp=hyp, augment=True, cache=opt.cache, rect=opt.rect, rank=LOCAL_RANK,
                                          workers=workers, image_weights=opt.image_weights, quad=opt.quad,
                                          prefix=colorstr('train: '), shuffle=True)  # <--- NEW

I evaluated this PR against master on VOC finetuning for 50 epochs, and the results show a slight improvement in most metrics and losses, particularly in objectness loss and mAP@0.5, perhaps indicating that the shuffle addition may help delay overtraining.

https://wandb.ai/glenn-jocher/VOC

To receive this update:

• Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
• PyTorch Hub – Force-reload model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
• Notebooks – View updated notebooks
• Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!