Add thread args to ThreadBuffer

[Nic-Ma]

Description

This PR added the threadbuffer args to the ThreadDataLoader.

Status

Ready

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[Nic-Ma]

/black

[wyli]

Thanks, I think the default value of buffer size should be the dataloader's batch size, that is to prefetch a batch when loading

[Nic-Ma]

Hi @ericspod ,

What do you think about @wyli ‘s suggestion？

Thanks in advance.

[Nic-Ma]

Hi @wyli ,

I think maybe the source data of thread_buffer in ThreadDataLoader is already batched data:
https://github.com/pytorch/pytorch/blob/v1.9.0/torch/utils/data/dataloader.py#L373
So we don't need to increase the buffer size according to batch size.
Of course, it may have some relationship to batch_size, depends on the processing speed of transforms and network sides.

Thanks.

[ericspod]

Hi @ericspod ,

What do you think about @wyli ‘s suggestion？

Thanks in advance.

Hi @Nic-Ma I don't think a buffer size beyond 1 is going to make a difference, except in the possible situation where the time taken to generate batches varies wildly one to the other. If it's set to a larger number it'll function but just take longer to initialize at the start.

[Nic-Ma]

Hi @ericspod ,
What do you think about @wyli ‘s suggestion？
Thanks in advance.

Hi @Nic-Ma I don't think a buffer size beyond 1 is going to make a difference, except in the possible situation where the time taken to generate batches varies wildly one to the other. If it's set to a larger number it'll function but just take longer to initialize at the start.

Hi @ericspod ,

Thanks for your comments, yes, we didn't change the default buffer size in this PR.