Skip to content

Commit

Permalink
test: trigger enzyme tests
Browse files Browse the repository at this point in the history
  • Loading branch information
@avik-pal
avik-pal committed Jul 24, 2024
1 parent c903992 commit 8d2e585
Showing 1 changed file with 9 additions and 7 deletions.
16 changes: 9 additions & 7 deletions src/layers/conv.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,15 @@
Standard convolutional layer.
Image data should be stored in WHCN order (width, height, channels, batch). In other words,
a `100 x 100` RGB image would be a `100 x 100 x 3 x 1` array, and a batch of 50 would be a
`100 x 100 x 3 x 50` array. This has `N = 2` spatial dimensions, and needs a kernel size
like `(5, 5)`, a 2-tuple of integers. To take convolutions along `N` feature dimensions,
this layer expects as input an array with `ndims(x) == N + 2`, where
`size(x, N + 1) == in_chs` is the number of input channels, and `size(x, ndims(x))` is the
number of observations in a batch.
!!! tip "Conv2D"
Image data should be stored in WHCN order (width, height, channels, batch). In other
words, a `100 x 100` RGB image would be a `100 x 100 x 3 x 1` array, and a batch of 50
would be a `100 x 100 x 3 x 50` array. This has `N = 2` spatial dimensions, and needs
a kernel size like `(5, 5)`, a 2-tuple of integers. To take convolutions along `N`
feature dimensions, this layer expects as input an array with `ndims(x) == N + 2`, where
`size(x, N + 1) == in_chs` is the number of input channels, and `size(x, ndims(x))` is
the number of observations in a batch.
!!! warning
Expand Down

1 comment on commit 8d2e585

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmark Results

Benchmark suite Current: 8d2e585 Previous: c903992 Ratio
Dense(2 => 2)/cpu/reverse/ReverseDiff (compiled)/(2, 128) 3898.5 ns 3694.5 ns 1.06
Dense(2 => 2)/cpu/reverse/Zygote/(2, 128) 7307.6 ns 7386.714285714285 ns 0.99
Dense(2 => 2)/cpu/reverse/Tracker/(2, 128) 20728 ns 20659 ns 1.00
Dense(2 => 2)/cpu/reverse/ReverseDiff/(2, 128) 9790.2 ns 9768.4 ns 1.00
Dense(2 => 2)/cpu/reverse/Flux/(2, 128) 9184.5 ns 8888.7 ns 1.03
Dense(2 => 2)/cpu/reverse/SimpleChains/(2, 128) 4464.625 ns 4470.875 ns 1.00
Dense(2 => 2)/cpu/reverse/Enzyme/(2, 128) 4689.875 ns 4682.625 ns 1.00
Dense(2 => 2)/cpu/forward/NamedTuple/(2, 128) 1119.3288590604027 ns 1103.671875 ns 1.01
Dense(2 => 2)/cpu/forward/ComponentArray/(2, 128) 1186.4705882352941 ns 1186.351145038168 ns 1.00
Dense(2 => 2)/cpu/forward/Flux/(2, 128) 1781.9122807017543 ns 1776.8823529411766 ns 1.00
Dense(2 => 2)/cpu/forward/SimpleChains/(2, 128) 179.221140472879 ns 179.70380818053596 ns 1.00
Dense(20 => 20)/cpu/reverse/ReverseDiff (compiled)/(20, 128) 17212 ns 17422 ns 0.99
Dense(20 => 20)/cpu/reverse/Zygote/(20, 128) 16962 ns 17012 ns 1.00
Dense(20 => 20)/cpu/reverse/Tracker/(20, 128) 37089 ns 37251 ns 1.00
Dense(20 => 20)/cpu/reverse/ReverseDiff/(20, 128) 29054 ns 29145 ns 1.00
Dense(20 => 20)/cpu/reverse/Flux/(20, 128) 21500 ns 20058 ns 1.07
Dense(20 => 20)/cpu/reverse/SimpleChains/(20, 128) 17192 ns 17483 ns 0.98
Dense(20 => 20)/cpu/reverse/Enzyme/(20, 128) 25507 ns 25438 ns 1.00
Dense(20 => 20)/cpu/forward/NamedTuple/(20, 128) 3858.5 ns 3887.25 ns 0.99
Dense(20 => 20)/cpu/forward/ComponentArray/(20, 128) 3944.875 ns 3965 ns 0.99
Dense(20 => 20)/cpu/forward/Flux/(20, 128) 4901.857142857143 ns 4967.857142857143 ns 0.99
Dense(20 => 20)/cpu/forward/SimpleChains/(20, 128) 1653.1 ns 1655.1 ns 1.00
Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 3, 128) 39417430 ns 38860153 ns 1.01
Conv((3, 3), 3 => 3)/cpu/reverse/Zygote/(64, 64, 3, 128) 59026753 ns 58591525.5 ns 1.01
Conv((3, 3), 3 => 3)/cpu/reverse/Tracker/(64, 64, 3, 128) 78753982.5 ns 77243718 ns 1.02
Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff/(64, 64, 3, 128) 91577217 ns 89813513 ns 1.02
Conv((3, 3), 3 => 3)/cpu/reverse/Flux/(64, 64, 3, 128) 75064628 ns 69964192.5 ns 1.07
Conv((3, 3), 3 => 3)/cpu/reverse/SimpleChains/(64, 64, 3, 128) 12478072 ns 12192813 ns 1.02
Conv((3, 3), 3 => 3)/cpu/reverse/Enzyme/(64, 64, 3, 128) 88187659 ns 92566221 ns 0.95
Conv((3, 3), 3 => 3)/cpu/forward/NamedTuple/(64, 64, 3, 128) 7734313 ns 7747332.5 ns 1.00
Conv((3, 3), 3 => 3)/cpu/forward/ComponentArray/(64, 64, 3, 128) 7616145 ns 7636139 ns 1.00
Conv((3, 3), 3 => 3)/cpu/forward/Flux/(64, 64, 3, 128) 10169419 ns 10090932 ns 1.01
Conv((3, 3), 3 => 3)/cpu/forward/SimpleChains/(64, 64, 3, 128) 6475298.5 ns 6409069.5 ns 1.01
vgg16/cpu/reverse/Zygote/(32, 32, 3, 16) 714601212 ns 707833076 ns 1.01
vgg16/cpu/reverse/Zygote/(32, 32, 3, 64) 2575011212 ns 2590098238 ns 0.99
vgg16/cpu/reverse/Zygote/(32, 32, 3, 2) 148434028 ns 141928214 ns 1.05
vgg16/cpu/reverse/Tracker/(32, 32, 3, 16) 835656592 ns 831399760 ns 1.01
vgg16/cpu/reverse/Tracker/(32, 32, 3, 64) 2880099801 ns 3031770120 ns 0.95
vgg16/cpu/reverse/Tracker/(32, 32, 3, 2) 230296223 ns 230519426 ns 1.00
vgg16/cpu/reverse/Flux/(32, 32, 3, 16) 754723456 ns 685293775 ns 1.10
vgg16/cpu/reverse/Flux/(32, 32, 3, 64) 2620964530 ns 2436741285 ns 1.08
vgg16/cpu/reverse/Flux/(32, 32, 3, 2) 137992622 ns 128440598 ns 1.07
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 16) 177381390 ns 176803514.5 ns 1.00
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 64) 664563096 ns 664156553.5 ns 1.00
vgg16/cpu/forward/NamedTuple/(32, 32, 3, 2) 35498073.5 ns 45698849 ns 0.78
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 16) 167948188 ns 167708912 ns 1.00
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 64) 655194974 ns 655353797 ns 1.00
vgg16/cpu/forward/ComponentArray/(32, 32, 3, 2) 30691371.5 ns 30695339 ns 1.00
vgg16/cpu/forward/Flux/(32, 32, 3, 16) 190361878 ns 187370588 ns 1.02
vgg16/cpu/forward/Flux/(32, 32, 3, 64) 742561584.5 ns 730398133 ns 1.02
vgg16/cpu/forward/Flux/(32, 32, 3, 2) 37831252 ns 38162204.5 ns 0.99
Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 64, 128) 1299516480 ns 1272794204 ns 1.02
Conv((3, 3), 64 => 64)/cpu/reverse/Zygote/(64, 64, 64, 128) 1906477766 ns 1891207049 ns 1.01
Conv((3, 3), 64 => 64)/cpu/reverse/Tracker/(64, 64, 64, 128) 2299658099 ns 2377396310 ns 0.97
Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff/(64, 64, 64, 128) 2458436737 ns 2412275494 ns 1.02
Conv((3, 3), 64 => 64)/cpu/reverse/Flux/(64, 64, 64, 128) 1945065634.5 ns 1854467971 ns 1.05
Conv((3, 3), 64 => 64)/cpu/reverse/Enzyme/(64, 64, 64, 128) 2069602165 ns 2076822407 ns 1.00
Conv((3, 3), 64 => 64)/cpu/forward/NamedTuple/(64, 64, 64, 128) 344998452 ns 340573567.5 ns 1.01
Conv((3, 3), 64 => 64)/cpu/forward/ComponentArray/(64, 64, 64, 128) 346311953.5 ns 333252236 ns 1.04
Conv((3, 3), 64 => 64)/cpu/forward/Flux/(64, 64, 64, 128) 379997474 ns 459555023 ns 0.83
Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 1, 128) 12056841 ns 11956140 ns 1.01
Conv((3, 3), 1 => 1)/cpu/reverse/Zygote/(64, 64, 1, 128) 18191891 ns 18159068 ns 1.00
Conv((3, 3), 1 => 1)/cpu/reverse/Tracker/(64, 64, 1, 128) 19288745 ns 19333664 ns 1.00
Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff/(64, 64, 1, 128) 23960447 ns 24033998 ns 1.00
Conv((3, 3), 1 => 1)/cpu/reverse/Flux/(64, 64, 1, 128) 18001949 ns 17981126 ns 1.00
Conv((3, 3), 1 => 1)/cpu/reverse/SimpleChains/(64, 64, 1, 128) 1175012 ns 1161262 ns 1.01
Conv((3, 3), 1 => 1)/cpu/reverse/Enzyme/(64, 64, 1, 128) 23334989 ns 23276535.5 ns 1.00
Conv((3, 3), 1 => 1)/cpu/forward/NamedTuple/(64, 64, 1, 128) 2293583.5 ns 2359332 ns 0.97
Conv((3, 3), 1 => 1)/cpu/forward/ComponentArray/(64, 64, 1, 128) 2214902 ns 2307143 ns 0.96
Conv((3, 3), 1 => 1)/cpu/forward/Flux/(64, 64, 1, 128) 2070653 ns 2093660.5 ns 0.99
Conv((3, 3), 1 => 1)/cpu/forward/SimpleChains/(64, 64, 1, 128) 207386 ns 203713 ns 1.02
Dense(200 => 200)/cpu/reverse/ReverseDiff (compiled)/(200, 128) 295220 ns 296287 ns 1.00
Dense(200 => 200)/cpu/reverse/Zygote/(200, 128) 269611 ns 267995 ns 1.01
Dense(200 => 200)/cpu/reverse/Tracker/(200, 128) 368927 ns 371109 ns 0.99
Dense(200 => 200)/cpu/reverse/ReverseDiff/(200, 128) 411035 ns 412547 ns 1.00
Dense(200 => 200)/cpu/reverse/Flux/(200, 128) 278338 ns 276370.5 ns 1.01
Dense(200 => 200)/cpu/reverse/SimpleChains/(200, 128) 408440 ns 410583 ns 0.99
Dense(200 => 200)/cpu/reverse/Enzyme/(200, 128) 397870 ns 398600 ns 1.00
Dense(200 => 200)/cpu/forward/NamedTuple/(200, 128) 81491 ns 84133.5 ns 0.97
Dense(200 => 200)/cpu/forward/ComponentArray/(200, 128) 81742 ns 87244 ns 0.94
Dense(200 => 200)/cpu/forward/Flux/(200, 128) 87142.5 ns 87415 ns 1.00
Dense(200 => 200)/cpu/forward/SimpleChains/(200, 128) 104725 ns 104687 ns 1.00
Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 16, 128) 189895146 ns 191531913 ns 0.99
Conv((3, 3), 16 => 16)/cpu/reverse/Zygote/(64, 64, 16, 128) 330378973 ns 327882868 ns 1.01
Conv((3, 3), 16 => 16)/cpu/reverse/Tracker/(64, 64, 16, 128) 425406175 ns 422399750.5 ns 1.01
Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff/(64, 64, 16, 128) 484809766 ns 483565853 ns 1.00
Conv((3, 3), 16 => 16)/cpu/reverse/Flux/(64, 64, 16, 128) 357948899 ns 387480865 ns 0.92
Conv((3, 3), 16 => 16)/cpu/reverse/SimpleChains/(64, 64, 16, 128) 344315432 ns 337771969 ns 1.02
Conv((3, 3), 16 => 16)/cpu/reverse/Enzyme/(64, 64, 16, 128) 449554706 ns 480545110.5 ns 0.94
Conv((3, 3), 16 => 16)/cpu/forward/NamedTuple/(64, 64, 16, 128) 47365748 ns 47647427 ns 0.99
Conv((3, 3), 16 => 16)/cpu/forward/ComponentArray/(64, 64, 16, 128) 46685949.5 ns 47230335 ns 0.99
Conv((3, 3), 16 => 16)/cpu/forward/Flux/(64, 64, 16, 128) 59744200 ns 50071719.5 ns 1.19
Conv((3, 3), 16 => 16)/cpu/forward/SimpleChains/(64, 64, 16, 128) 28213801 ns 28845102 ns 0.98
Dense(2000 => 2000)/cpu/reverse/ReverseDiff (compiled)/(2000, 128) 19193456.5 ns 19018583 ns 1.01
Dense(2000 => 2000)/cpu/reverse/Zygote/(2000, 128) 19696227 ns 19702711 ns 1.00
Dense(2000 => 2000)/cpu/reverse/Tracker/(2000, 128) 23742308 ns 23539011.5 ns 1.01
Dense(2000 => 2000)/cpu/reverse/ReverseDiff/(2000, 128) 24224045 ns 24139841.5 ns 1.00
Dense(2000 => 2000)/cpu/reverse/Flux/(2000, 128) 19666246.5 ns 19766095 ns 0.99
Dense(2000 => 2000)/cpu/reverse/Enzyme/(2000, 128) 20955027 ns 21127350.5 ns 0.99
Dense(2000 => 2000)/cpu/forward/NamedTuple/(2000, 128) 6545190 ns 6590543 ns 0.99
Dense(2000 => 2000)/cpu/forward/ComponentArray/(2000, 128) 6556441 ns 6541563 ns 1.00
Dense(2000 => 2000)/cpu/forward/Flux/(2000, 128) 6537709 ns 6538585 ns 1.00

This comment was automatically generated by workflow using github-action-benchmark.

Please sign in to comment.