-
Notifications
You must be signed in to change notification settings - Fork 3
/
Copy pathcaffe_commands_train_and_test~
190 lines (137 loc) · 11.5 KB
/
caffe_commands_train_and_test~
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
googlenet_categories from Food-101 ----------------------
../code/caffe/build/tools/caffe train -solver models/googlenet_categories/solver_just_FC.prototxt -weights models/googlenet_categories/snapshots/foodRecognition_googlenet_finetunning_v2_1_iter_448000.caffemodel
##################################################################################
##################################################################################
##################################################################################
##################################################################################
googlenet_categories from imageNet ----------------------
../code/caffe/build/tools/caffe train -solver models/googlenet_categories/solver.prototxt -weights models/googlenet_categories/snapshots/bvlc_googlenet.caffemodel
googlenet_categories from Food-101 ----------------------
../code/caffe/build/tools/caffe train -solver models/googlenet_categories/solver.prototxt -weights models/googlenet_categories/snapshots/foodRecognition_googlenet_finetunning_v2_1_iter_448000.caffemodel
##################################################################################
##################################################################################
##################################################################################
##################################################################################
DONE
############################### [*****] fine-tunning SR BALANCED from Imagenet ################## (From folder TFG)
# Resized 101 dataset from imagenet snapshot
GOOGLENET ----------------------
../code/caffe/build/tools/caffe train -solver models/googlenet_SR_balanced/solver.prototxt -weights models/googlenet_SR_balanced/snapshots/bvlc_googlenet.caffemodel
DONE
############################### [*****] fine-tunning SR from Imagenet ################## (From folder TFG)
# Resized 101 dataset from imagenet snapshot
GOOGLENET ----------------------
../code/caffe/build/tools/caffe train -solver models/googlenet_SR/solver.prototxt -weights models/googlenet_SR/snapshots/bvlc_googlenet.caffemodel
##################################################################################
##################################################################################
##################################################################################
##################################################################################
############################### fine-tunning Resized101 500 BALANCED ################## (From folder TFG)
# Resized 101 dataset with balanced classes
GOOGLENET (from fine-tuning food101)----------------------
../code/caffe/build/tools/caffe train -solver models/googlenet_resized_balanced/solver.prototxt -weights models/googlenet_resized_balanced/snapshots/foodRecognition_googlenet_finetunning_v2_1_iter_448000.caffemodel
GOOGLENET (from fine-tuning IMAGENET) ---------------------- FALTAAAAAAAAA
../code/caffe/build/tools/caffe train -solver models/googlenet_resized_balanced/solver_from_imagenet.prototxt -weights models/googlenet_resized_balanced/snapshots_from_imagenet/bvlc_googlenet.caffemodel
############################### fine-tunning Resized101 ################## (From folder TFG)
# Resized 101 dataset with unbalanced classes
GOOGLENET (from fine-tuning food101)----------------------
../code/caffe/build/tools/caffe train -solver models/googlenet_resized/solver.prototxt -weights models/googlenet_resized/snapshots/foodRecognition_googlenet_finetunning_v2_1_iter_448000.caffemodel
GOOGLENET (from fine-tuning IMAGENET) ----------------------
../code/caffe/build/tools/caffe train -solver models/googlenet_resized/solver_from_imagenet.prototxt -weights models/googlenet_resized/snapshots_from_imagenet/bvlc_googlenet.caffemodel
##################################################################################
##################################################################################
##################################################################################
##################################################################################
############################### fine-tunning 500 ################## (From folder TFG)
GOOGLENET ----------------------
../code/caffe/build/tools/caffe train -solver models/foodCAT_googlenet_food101_500/solver.prototxt -weights models/foodCAT_googlenet_food101_500/snapshots/ss_foodCAT_googlenet_food101_train_iter_490000.caffemodel
VGG ----------------------
../code/caffe/build/tools/caffe train -solver models/foodCAT_VGG_ILSVRC_19_layers_500/solver.prototxt -weights models/foodCAT_VGG_ILSVRC_19_layers_500/snapshots/ss_foodCAT_VGG_ILSVRC_19_layers_train_iter_80000.caffemodel
############################### fine-tunning ################## (From folder TFG)
LENET ----------------------
../code/caffe/build/tools/caffe train -solver models/foodCAT_googlenet_food101/solver.prototxt -weights models/foodCAT_googlenet_food101/snapshots/
ss_foodCAT_googlenet_food101_train_iter_490000.caffemodel
VGG ----------------------
../code/caffe/build/tools/caffe train -solver models/foodCAT_VGG_ILSVRC_19_layers/solver.prototxt -weights models/foodCAT_VGG_ILSVRC_19_layers/snapshots/VGG_ILSVRC_19_layers.caffemodel > models/foodCAT_VGG_ILSVRC_19_layers/CLUSTER/LOGS/VGG_log_29-04-2016.txt
VGG - CLUSTER ----------------------
../code/caffe/build/tools/caffe train -solver models/foodCAT_VGG_ILSVRC_19_layers/CLUSTER/solver.prototxt -weights models/foodCAT_VGG_ILSVRC_19_layers/snapshots/VGG_ILSVRC_19_layers.caffemodel > models/foodCAT_VGG_ILSVRC_19_layers/CLUSTER/LOGS/VGG_log_29-04-2016.txt
ALEXNET - CLUSTER ----------------------
caffe train -solver models/foodCAT_alexnet/solver.prototxt -weights models/foodCAT_alexnet/snapshots/bvlc_alexnet.caffemodel
############################### RESUME fine-tunning ################## (From folder TFG)
LENET ----------------------
../code/caffe/build/tools/caffe train --solver=models/foodCAT_googlenet_food101/solver.prototxt --snapshot=models/foodCAT_googlenet_food101/snapshots/
ss_foodCAT_googlenet_food101_train_iter_490000.solverstate
VGG ----------------------
caffe train -solver models/foodCAT_VGG_ILSVRC_19_layers/CLUSTER/solver.prototxt --snapshot=models/foodCAT_VGG_ILSVRC_19_layers/snapshots/ss_foodCAT_VGG_ILSVRC_19_layers_train_iter_50000.solverstate
############################### TEST ################## (From folder TFG) 914,375 = 14630(TOTAL_IMAGES_TEST)÷16(BATCH_SIZE) --> 915
## Better to use my own script
LENET ----------------------
../code/caffe/build/tools/caffe test -model models/foodCAT_googlenet_food101/test.prototxt -weights models/foodCAT_googlenet_food101/snapshots/first_TRAIN_73_91/ss_foodCAT_googlenet_food101_train_iter_490000.caffemodel -gpu 0 -iterations 915 2> models/foodCAT_googlenet_food101/LOGS/accuracy_test.log
LENET - TEST_JUST_foodCAT ----------------------
../code/caffe/build/tools/caffe test -model models/foodCAT_googlenet_food101/test_just_foodCAT.prototxt -weights models/foodCAT_googlenet_food101/snapshots/first_TRAIN_73_91/ss_foodCAT_googlenet_food101_train_iter_490000.caffemodel -gpu 0 -iterations 915 2> models/foodCAT_googlenet_food101/LOGS/accuracy_test_just_foodCAT.log
VGG ----------------------
../code/caffe/build/tools/caffe test -model models/foodCAT_VGG_ILSVRC_19_layers/test.prototxt -weights models/foodCAT_VGG_ILSVRC_19_layers/snapshots/ss_foodCAT_VGG_ILSVRC_19_layers_train_iter_80000.caffemodel -gpu 0 -iterations 915 2> models/foodCAT_VGG_ILSVRC_19_layers/LOGS/accuracy_test.log
VGG - TEST_JUST_foodCAT ----------------------
../code/caffe/build/tools/caffe test -model models/foodCAT_VGG_ILSVRC_19_layers/test_just_foodCAT.prototxt -weights models/foodCAT_VGG_ILSVRC_19_layers/snapshots/ss_foodCAT_VGG_ILSVRC_19_layers_train_iter_80000.caffemodel -gpu 0 -iterations 915 2> models/foodCAT_VGG_ILSVRC_19_layers/LOGS/accuracy_test_just_foodCAT.log
############################### Generate Accuracy Picture ################## (From folder TFG)
OBS: The tool that we are using is located in caffe/tools/extra
GOOGLENET ---------------------- I don't have it (I didn't save it)
VGG ----------------------
$CAFFE_ROOT/tools/extra/plot_training_log.py.example 0 models/foodCAT_VGG_ILSVRC_19_layers/LOGS/plot_f0_to50000.png models/foodCAT_VGG_ILSVRC_19_layers/LOGS/log.log
GOOGLENET_500 ----------------------
$CAFFE_ROOT/tools/extra/plot_training_log.py.example 0 models/foodCAT_googlenet_food101_500/LOGS/accuracy.png models/foodCAT_googlenet_food101_500/LOGS/log.log
VGG_500 ----------------------
$CAFFE_ROOT/tools/extra/plot_training_log.py.example 0 models/foodCAT_VGG_ILSVRC_19_layers_500/LOGS/accuracy.png models/foodCAT_VGG_ILSVRC_19_layers_500/LOGS/log.log
############################### Parse and Plot the LOG ################## (From folder TFG)
# rename your caffe log to log.log (in order to use the next instruccions just with copy-paste)
cp yourCaffeLOG log.log
# 1. parse the log
python $CAFFE_ROOT/tools/extra/parse_log.py log.log $(pwd)
# 2. plot the log test file generated by step 1
gnuplot
reset
set terminal png
set output "googlenetACCURACY.png"
set style data lines
set key right
set xlabel "iteratations"
set ylabel "accuracy"
set datafile separator ","
plot "log.log.test" using 1:12 title "top-5", '' using 1:11 title "top-1"
# (for VGG model)
plot "log.log.test" using 1:5 title "top-5", '' using 1:4 title "top-1"
# Well, a nice chart like the generated with the above instruction contains too much information to let you choose your best iteration.
# Also, as in file 'solver.prototxt' you set a parameter called 'snapshot', thus, just the iteration multiples to this variable has been saved by caffe.
# And so, you want to choose your best model based on the best iteration saved by caffe (could be not the best one in all iterations)
# 3. generate a new csv file to plot just the iterations saved by caffe. You need to specify:
# the file created by #1 'log.log.test'
# the fields you are interested to plot in that file (first give the x-axes and then a sorted y-axes, where the first that you specified will be the most important)
# the value of the parameter 'snapshot' specified in the file 'solver.prototxt'
# go the folder that contains the python script and paste there the file 'log.log.test'. Then run with 1 as the iterations, 12 as top-1 and 12 as top-5 accurazy
##### VGG> Does not work how we expect, because the parameter snapshot was not well defined in the solver
##### googlenet categories
python ../../../tools/scripts/get_best_accuracy_iter.py 'log.log.test' 1 11 12 5448
python ../../../tools/scripts/get_best_accuracy_iter.py 'log.log.test' 1 4 5 10000
##### googlenet SR BALANCED
python ../../../tools/scripts/get_best_accuracy_iter.py 'log.log.test' 1 11 12 2280
##### googlenet SR
python ../../../tools/scripts/get_best_accuracy_iter.py 'log.log.test' 1 11 12 5448
##### googlenet resized from imagenet BALANCED
python ../../../tools/scripts/get_best_accuracy_iter.py 'log.log.test' 1 11 12 2280
##### googlenet resized from imagenet
python ../../../tools/scripts/get_best_accuracy_iter.py 'log.log.test' 1 11 12 5448
##### googlenet resized
python get_best_accuracy_iter.py 'log.log.test' 1 11 12 5448
##### googlenet resized and balanced
python ../../../tools/scripts/get_best_accuracy_iter.py 'log.log.test' 1 11 12 2280
# 4. plot the saved iterations generated by step #4
gnuplot
reset
set terminal png
set output "googlenet_saved_ACCURACY.png"
set style data lines
set key right
set xlabel "iteratations"
set ylabel "accuracy"
set datafile separator ","
plot "acc_in_saved_iters" using 1:3 title "top-5" with lp, '' using 1:2 title "top-1" with lp