Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix the conv_2d batch mode #737

Merged
merged 1 commit into from
Feb 20, 2024
Merged

Conversation

bssrdf
Copy link
Contributor

@bssrdf bssrdf commented Feb 19, 2024

I think there is a bug in ggml_conv_2d which mixed up convolution results from different images when run in batch mode.
Specifically, this line

result = ggml_reshape_4d(ctx, result, im2col->ne[1], im2col->ne[2], a->ne[3], im2col->ne[3]); // [N, OC, OH, OW]

is supposed to reshape result to [N, OC, OH, OW]. However, values for a particular i of dimension N (batch size) are mixed from
j's.

This can be checked by making a stack of 2 input images each with identical but different values and observe the convolution mixed from both images.

The proper way to do this step is

result = ggml_reshape_4d(ctx, result, im2col->ne[1], im2col->ne[2], im2col->ne[3], a->ne[3]); // [ OC, N, OH, OW]    
result = ggml_cont(ctx, ggml_permute(ctx, result, 0, 1, 3, 2)); // [N, OC, OH, OW]

@ggerganov ggerganov merged commit 25cc1d5 into ggml-org:master Feb 20, 2024
4 checks passed
@bssrdf bssrdf deleted the fix-conv2d-batch-mode branch February 21, 2024 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants