-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathsearch.xml
340 lines (163 loc) · 359 KB
/
search.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>考研数学题集</title>
<link href="2021/11/07/%E8%80%83%E7%A0%94%E6%95%B0%E5%AD%A6/"/>
<url>2021/11/07/%E8%80%83%E7%A0%94%E6%95%B0%E5%AD%A6/</url>
<content type="html"><![CDATA[<h2 id="数学模拟卷"><a href="#数学模拟卷" class="headerlink" title="数学模拟卷"></a>数学模拟卷</h2><h3 id="张宇8套"><a href="#张宇8套" class="headerlink" title="张宇8套"></a>张宇8套</h3><h4 id="卷二"><a href="#卷二" class="headerlink" title="卷二"></a>卷二</h4><p>2 12 20 21（3）</p><p><img src="https://z3.ax1x.com/2021/11/06/IQ24UA.jpg" alt=""></p><p>不可导点集中在$x=1,x=2,x=3$需要用$x^2+ax+b$把它们消掉</p><p> <img src="https://z3.ax1x.com/2021/11/06/IQ2hEd.jpg" /> </p><p> 令$x=t+ {\frac{\pi}{2}}$</p><p> <img src="https://z3.ax1x.com/2021/11/06/IQ254I.jpg" alt=""></p><p> 根据$|f’(x)|\leq{M}$推测需要用到泰勒展开,直接展开$f(x)$不太够，所以对$\int_a^xf(t)dt$和$\int_x^bf(t)dt$泰勒展开</p><p><img src="https://z3.ax1x.com/2021/11/06/IQ2T8P.jpg" alt=""></p><p>$r(A^TA)=r(A)且Ax=0的解也是A^TAx=0的解，所以A^TAx=0与Ax=0同解$</p><h4 id="卷三"><a href="#卷三" class="headerlink" title="卷三"></a>卷三</h4><p> 2 18 </p><p><img src="https://z3.ax1x.com/2021/11/06/IQ27gf.jpg" alt=""></p><p>间断点处分母为0</p><p><img src="https://z3.ax1x.com/2021/11/06/IQ2Hv8.jpg" alt=""></p><p>尝试用对曲面的面积积分解题，但积不出来，所以实际考察的是多元函数极值，截面是一个椭圆，点到原点距离的最大值和最小值分别是长半轴长和短半轴长</p><h4 id="卷四"><a href="#卷四" class="headerlink" title="卷四"></a>卷四</h4><p>4 7 18 19 20</p><p><img src="https://z3.ax1x.com/2021/11/06/IQ2qKS.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ2LDg.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ2ObQ.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ2jEj.jpg" alt=""></p><p> $三重积分的先二后一法，重点消化$</p><p><img src="https://z3.ax1x.com/2021/11/06/IQ2x5n.jpg" alt=""></p><h4 id="卷五"><a href="#卷五" class="headerlink" title="卷五"></a>卷五</h4><p>1</p><p><img src="https://z3.ax1x.com/2021/11/06/IQRSCq.jpg" alt=""></p><h4 id="卷六"><a href="#卷六" class="headerlink" title="卷六"></a>卷六</h4><p>18 22</p><p><img src="https://z3.ax1x.com/2021/11/06/IQRp80.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQR92V.jpg" alt=""></p><h4 id="卷七"><a href="#卷七" class="headerlink" title="卷七"></a>卷七</h4><p>4</p><p><img src="https://z3.ax1x.com/2021/11/06/IQRCvT.jpg" alt=""></p><h4 id="卷八"><a href="#卷八" class="headerlink" title="卷八"></a>卷八</h4><p>7</p><p><img src="https://z3.ax1x.com/2021/11/06/IQRiKU.jpg" alt=""></p><h3 id="张宇4套"><a href="#张宇4套" class="headerlink" title="张宇4套"></a>张宇4套</h3><h4 id="卷一"><a href="#卷一" class="headerlink" title="卷一"></a>卷一</h4><p>3 13 15 17</p><p><img src="https://z3.ax1x.com/2021/11/06/IQ78Z6.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7QMR.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ71qx.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7ls1.jpg" alt=""></p><h4 id="卷二-1"><a href="#卷二-1" class="headerlink" title="卷二"></a>卷二</h4><p>1 2 20</p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7GdK.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7JIO.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7tiD.jpg" alt=""></p><h4 id="卷三-1"><a href="#卷三-1" class="headerlink" title="卷三"></a>卷三</h4><p>2 3 9 18 2 0</p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7NJe.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7URH.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7azd.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7wQA.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ70sI.jpg" alt=""></p><h4 id="卷四-1"><a href="#卷四-1" class="headerlink" title="卷四"></a>卷四</h4><p>2 4 7</p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7BLt.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7reP.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IQ7sdf.jpg" alt=""></p><h3 id="李林6套"><a href="#李林6套" class="headerlink" title="李林6套"></a>李林6套</h3><h4 id="一卷"><a href="#一卷" class="headerlink" title="一卷"></a>一卷</h4><p>4 8 15 18 22</p><p><img src="https://z3.ax1x.com/2021/11/06/IlFVPJ.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlFkaF.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlFA54.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlFZG9.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlFFVU.jpg" alt=""></p><h4 id="二卷"><a href="#二卷" class="headerlink" title="二卷"></a>二卷</h4><p>3 4 8 22（3）</p><p><img src="https://z3.ax1x.com/2021/11/06/IlFmx1.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlFuKx.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlFKr6.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlFMqK.jpg" alt=""></p><h4 id="三卷"><a href="#三卷" class="headerlink" title="三卷"></a>三卷</h4><p>2 3 9 10</p><p><img src="https://z3.ax1x.com/2021/11/06/IlFlVO.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlF1aD.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlF3Ie.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlFGPH.jpg" alt=""></p><h2 id="数学真题"><a href="#数学真题" class="headerlink" title="数学真题"></a>数学真题</h2><h4 id="2018数学一"><a href="#2018数学一" class="headerlink" title="2018数学一"></a>2018数学一</h4><p><img src="https://z3.ax1x.com/2021/11/09/INls8e.jpg" alt=""></p><p> 对于特征值，迹，模完全相同的两个矩阵，判断是否相似还需根据$r(\lambda{E-A})$是否一致</p><p><img src="https://z3.ax1x.com/2021/11/09/INl6vd.jpg" alt=""></p><p> 这里分块矩阵的秩更主要取决于行秩，部分矩阵的列变换并不影响整体的行秩</p><p><img src="https://z3.ax1x.com/2021/11/09/INlygH.jpg" alt=""></p><p> 有些直接转化成统一的积分变量好算，有些使用高斯公式好算，转化时不要只想到转到$dxdy$,根据实际积分区域选择</p><p><img src="https://z3.ax1x.com/2021/11/09/INl0UK.jpg" alt=""></p><p> 细心</p><p><img src="https://z3.ax1x.com/2021/11/09/INlrCD.jpg" alt=""></p><p> 第二问依然不太清楚</p><p><img src="https://z3.ax1x.com/2021/11/09/INlB4O.jpg" alt=""></p><p> 不可逆矩阵间的转化，貌似似曾相识？</p><h4 id="2020数学一"><a href="#2020数学一" class="headerlink" title="2020数学一"></a>2020数学一</h4><p>2，3，4，19 </p><p><img src="https://z3.ax1x.com/2021/11/06/IlkuWj.jpg" alt=""> </p><p><img src="https://z3.ax1x.com/2021/11/06/IlknYQ.jpg" alt=""></p><p> 设$f(x,y)=x$?</p><p><img src="https://z3.ax1x.com/2021/11/06/IlkmFg.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlkZTS.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/11/06/IlkVw8.jpg" alt=""></p><p> </p><h4 id="2021数学一"><a href="#2021数学一" class="headerlink" title="2021数学一"></a>2021数学一</h4><p> 7，10，15</p><p><img src="https://z3.ax1x.com/2021/11/06/IlkMSs.jpg" style="zoom:120%;" /></p><p><img src="https://z3.ax1x.com/2021/11/06/IlkQln.jpg" style="zoom:110%;" /></p><p><img src="https://z3.ax1x.com/2021/11/06/Ilklyq.jpg" style="zoom:110%;" /></p>]]></content>
<categories>
<category> 数学 </category>
</categories>
</entry>
<entry>
<title>电路934疑难点</title>
<link href="2021/10/06/2021-10-6-%E7%94%B5%E8%B7%AF934%E7%96%91%E9%9A%BE%E7%82%B9/"/>
<url>2021/10/06/2021-10-6-%E7%94%B5%E8%B7%AF934%E7%96%91%E9%9A%BE%E7%82%B9/</url>
<content type="html"><![CDATA[<h3 id="1-Y与Δ的相互转换"><a href="#1-Y与Δ的相互转换" class="headerlink" title="1.Y与Δ的相互转换"></a>1.Y与Δ的相互转换</h3><p><img src="https://z3.ax1x.com/2021/10/06/4xOd4U.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/10/06/4xO7DI.jpg" alt=""></p><h3 id="2-实际电源两种模型的相互转换"><a href="#2-实际电源两种模型的相互转换" class="headerlink" title="2.实际电源两种模型的相互转换"></a>2.实际电源两种模型的相互转换</h3><p> 等效变换仅保证端子外部电路的电压，电流和功率相同，对内部并无等效可言</p><h3 id="3-KCL和KVL-的独立方程数"><a href="#3-KCL和KVL-的独立方程数" class="headerlink" title="3.KCL和KVL 的独立方程数"></a>3.KCL和KVL 的独立方程数</h3><p>对于具有$b$条支路和$n$个结点的电路，根据KCL可以列出$(n-1)$个独立方程，根据KVL可以列出$(b-n+1)$个独立方程</p><h3 id="4-结点电压法的特殊情况"><a href="#4-结点电压法的特殊情况" class="headerlink" title="4.结点电压法的特殊情况"></a>4.结点电压法的特殊情况</h3><p>1.无伴独立电压源：</p><p> 以电压源的一端连接点作为参考点，则关于另一端的节点电压已知 </p><p>2.受控电流源：</p><p> 作用可以看成是独立电流源但是需将控制量用节点电压表示</p><p>3.有伴受控电压源：</p><p> 直接转换为等效受控电流源</p><p>4.无伴受控电压源：</p><p> 以电压源的一端连接点作为参考点，则关于另一端的节点电压已知，控制量需用节点电压表示</p><h3 id="5-叠加定理"><a href="#5-叠加定理" class="headerlink" title="5.叠加定理"></a>5.叠加定理</h3><p>独立源置零时受控源仍应保留在各分电路中 </p><h3 id="6-戴维南定理和诺顿定理的两种使用方法"><a href="#6-戴维南定理和诺顿定理的两种使用方法" class="headerlink" title="6.戴维南定理和诺顿定理的两种使用方法"></a>6.戴维南定理和诺顿定理的两种使用方法</h3><p>使用对象是==<strong>一个含独立电源、线性电阻和受控源的一端口</strong>==</p><p>1.根据开路电压$u_{oc}$、短路电流的$i_{sc}$以及等效电阻$R_{eq}$中的任意两个求剩余的参数</p><p>2.在端口外施加电压源${u}$,流过电压源的电流为${i}$，根据$u=u_{oc}-R_{eq}i$直接求得${u_{oc}}$或${R_{eq}}$或根据</p><p>$i=i_{sc}-u/R_{eq}$来直接求得$i_{sc}$ 和 $R_{eq}$</p><h3 id="7-特勒根定理"><a href="#7-特勒根定理" class="headerlink" title="7.特勒根定理"></a>7.特勒根定理</h3><p>使用对象是一个具有n个结点和b条支路的电路，对支路内容无详细要求</p><p>==<strong>电压电流必须取参考方向，即电压与电流乘积的正负只取决于两者方向是否一致！！！</strong>==</p><p>定理1：</p><script type="math/tex; mode=display">\prod\limits_{k=1}^{b}{u_k}{i_k}=0</script><p>定理2：</p><p>对于两个图相同但是支路内容不同的电路</p><script type="math/tex; mode=display">\prod\limits_{k=1}^{b}{u_k}{\hat{i_k}}=0</script><script type="math/tex; mode=display">\prod\limits_{k=1}^{b}{\hat{u_k}}{i_k}=0</script><h3 id="8-等电位法"><a href="#8-等电位法" class="headerlink" title="8.等电位法"></a>8.等电位法</h3><p>确定等电位点间==<strong>无电流通过</strong>==，认为它们间可以直接用导线相连（或断开），==<strong>可能存在等电位但是有电流的情况</strong>==</p><h3 id="9-互易定理"><a href="#9-互易定理" class="headerlink" title="9.互易定理"></a>9.互易定理</h3><p>推导自特勒根定理2</p><p>基本公式：${u_1}{\hat{i_1}}+{u_2}{\hat{i_2}}={\hat{u_1}}{i_1}+{\hat{u_2}}{i_2}$</p><p>推导过程：$\prod\limits_{k=1}^{b}{u_k}{\hat{i_k}}={u_1}{\hat{i_1}}+{u_2}{\hat{i_2}}+\prod\limits_{k=3}^{b}{u_k}{\hat{i_k}}={u_1}{\hat{i_1}}+{u_2}{\hat{i_2}}+\prod\limits_{k=3}^{b}{R_k}{i_k}{\hat{i_k}}=0$</p><p> $\prod\limits_{k=1}^{b}{\hat{u_k}}{i_k}={\hat{u_1}}{i_1}+{\hat{u_2}}{i_2}+\prod\limits_{k=3}^{b}{\hat{u_k}}{i_k}={\hat{u_1}}{i_1}+{\hat{u_2}}{i_2}+\prod\limits_{k=3}^{b}{R_k}{i_k}{\hat{i_k}}=0$</p><p> $\therefore{u_1}{\hat{i_1}}+{u_2}{\hat{i_2}}={\hat{u_1}}{i_1}+{\hat{u_2}}{i_2}$</p><p><strong>==使用条件：==</strong></p><p>1.该电路是纯电阻构成的网络</p><p>2.网络外部只有一个独立源</p><p>3.独立源置零后电路的拓扑结构不变</p><p>形式1（只有电压源）：</p><p><img src="https://z3.ax1x.com/2021/10/06/4zRFF1.jpg" alt=""></p><p> <img src="https://z3.ax1x.com/2021/10/06/4zWLvT.jpg" alt=""></p><p>形式2（只有电流源）：</p><p><img src="https://z3.ax1x.com/2021/10/06/4zRPoR.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/10/06/4zWq2V.jpg" alt=""></p><p>形式3（两者都有）：</p><p><img src="https://z3.ax1x.com/2021/10/06/4zRCw9.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/10/06/4zWLvT.jpg" alt=""></p><h3 id="10-运算放大器的两种处理"><a href="#10-运算放大器的两种处理" class="headerlink" title="10.运算放大器的两种处理"></a>10.运算放大器的两种处理</h3><p>$\enclose{circle}{1}$.节点电压法，一般情况正反相输入端选一个，地线选一个，输出端不选，其他点视具体电路而定</p><p>$\enclose{circle}{2}$.利用虚短虚断的关系</p><h3 id="11-动态电路中储能元件电压电流的关系"><a href="#11-动态电路中储能元件电压电流的关系" class="headerlink" title="11.动态电路中储能元件电压电流的关系"></a>11.动态电路中储能元件电压电流的关系</h3><p>1.电感放电，相当于电源，电压与电流取非关联参考方向，$u_L=-L\frac{di_l}{dt}$</p><p>2.电感充电，相当于用电器，电压与电流取关联参考方向，$u_L=L\frac{di_l}{dt}$</p><p>3.电容放电，相当于电源，电压与电流取非关联参考方向，$i_c=-C\frac{du_c}{dt}$。</p><p>4.电容充电，相当于用电器，电压与电流取关联参考方向，$i_c=C\frac{du_c}{dt}$</p><p>==<strong>所以在都取关联方向的情况下，$u_L=L\frac{di_l}{dt}$，$i_c=C\frac{du_c}{dt}$</strong>==</p><p><img src="https://z3.ax1x.com/2021/10/07/5pwnXR.jpg" style="zoom:15%;" /></p><p>（a）$u=\frac{1}{C}\int_{-\infty}^{t}id\xi=u(0)+\frac{1}{C}\int_{0}^{t}id\xi$</p><p> (b) $i=-\frac{1}{L}\int_{-\infty}^{t}ud\xi=i(0)-\frac{1}{L}\int_{0}^{t}ud\xi$</p><h3 id="12-二阶动态电路R、L，C对放电情况的影响"><a href="#12-二阶动态电路R、L，C对放电情况的影响" class="headerlink" title="12.二阶动态电路R、L，C对放电情况的影响"></a>12.二阶动态电路R、L，C对放电情况的影响</h3><p>1.RLC串联电路</p><p><img src="https://z3.ax1x.com/2021/10/07/5pNdFe.jpg" style="zoom:20%;" /></p><p> $LC\frac{d^2u_c}{dt^2}+RC\frac{du_c}{dt}+u_c=0$</p><p> $LCp^2+RCp+1=0$</p><p> $p=-\frac{R}{2L}\pm\sqrt{(\frac{R}{2L})^2-\frac{1}{LC}}$</p><p> {1} $R>2\sqrt{\frac{L}{C}}$ 非振荡放电</p><p> {2} $R=2\sqrt{\frac{L}{C}}$ 临界非振荡过程</p><p> {3} $R<2\sqrt{\frac{L}{C}}$ 振荡放电</p><h3 id="13-一二阶电路的冲激响应和阶跃响应"><a href="#13-一二阶电路的冲激响应和阶跃响应" class="headerlink" title="13.一二阶电路的冲激响应和阶跃响应"></a>13.一二阶电路的冲激响应和阶跃响应</h3><p>首选运算法，其次是先列微分方程求出阶跃响应然后直接利用阶跃响应求导得到冲激响应</p><h3 id="14-动态电路中的有效值"><a href="#14-动态电路中的有效值" class="headerlink" title="14.动态电路中的有效值"></a>14.动态电路中的有效值</h3><p>正弦电路中利用相量图求得有效值</p><p><img src="https://z3.ax1x.com/2021/10/07/5ptAHO.jpg" alt="30" style="zoom: 35%;" /></p><p> $Us=\sqrt{U_L^2+U_R^2}$ $Us=\sqrt{(U_C-U_L)^2+U_R^2}$ </p><h3 id="15-复阻抗Z-R-jX中R为负值的情况"><a href="#15-复阻抗Z-R-jX中R为负值的情况" class="headerlink" title="15.复阻抗Z=R+jX中R为负值的情况"></a>15.复阻抗Z=R+jX中R为负值的情况</h3><p><img src="https://z3.ax1x.com/2021/10/07/5pBFWF.jpg" style="zoom:15%;" /></p><h3 id="16-结点电压两结点间只有导线"><a href="#16-结点电压两结点间只有导线" class="headerlink" title="16.结点电压两结点间只有导线"></a>16.结点电压两结点间只有导线</h3><p><img src="https://z3.ax1x.com/2021/10/08/5CgrCR.jpg" style="zoom: 20%;" /></p><p>结点$\enclose{circle}{1}$，$\enclose{circle}{3}$直接可视为电流源</p><h3 id="17-去耦等效电路"><a href="#17-去耦等效电路" class="headerlink" title="17.去耦等效电路"></a>17.去耦等效电路</h3><p>——串联</p><p><img src="https://z3.ax1x.com/2021/10/11/5Zz5OH.jpg" alt=""></p><p>——并联</p><p><img src="https://z3.ax1x.com/2021/10/11/5ZzhlD.jpg" alt=""> <img src="https://z3.ax1x.com/2021/10/11/5Zz46e.jpg" alt=""></p><h3 id="18-三个电感相耦合"><a href="#18-三个电感相耦合" class="headerlink" title="18.三个电感相耦合"></a>18.三个电感相耦合</h3><p><img src="https://z3.ax1x.com/2021/10/08/5Pngl8.jpg" style="zoom:15%;" /></p><p><img src="https://z3.ax1x.com/2021/10/08/5PnROg.jpg" style="zoom:15%;" /></p><p><img src="https://z3.ax1x.com/2021/10/11/5ZzCRO.jpg" style="zoom: 80%;" /></p><p>==<strong>plus:三角形连接快速去耦</strong>==</p><p><img src="https://z3.ax1x.com/2021/10/11/5Zzeot.jpg" alt=""></p><h3 id="19-电路相量图示范"><a href="#19-电路相量图示范" class="headerlink" title="19.电路相量图示范"></a>19.电路相量图示范</h3><p>$R_1=R_2=100\Omega$ , $L_1=3H$, $L_2=10H$ ,$M=5H$ ,$U=220V$, $w=100rad/s$</p><p><img src="https://z3.ax1x.com/2021/10/08/5PVlCT.jpg" style="zoom:15%;" /></p><p><img src="https://z3.ax1x.com/2021/10/08/5PefgS.jpg" style="zoom:15%;" /></p><p> <img src="https://z3.ax1x.com/2021/10/08/5PeW38.jpg" style="zoom:10%;" /></p><p><img src="https://z3.ax1x.com/2021/10/08/5PV18U.jpg" style="zoom:20%;" /></p><h3 id="20-最大功率"><a href="#20-最大功率" class="headerlink" title="20.最大功率"></a>20.最大功率</h3><h3 id="21-串并联谐振"><a href="#21-串并联谐振" class="headerlink" title="21.串并联谐振"></a>21.串并联谐振</h3><p>1.$RLC$串联电路</p><p><img src="https://z3.ax1x.com/2021/10/08/5P8CaF.jpg" alt=""></p><p> $w_0=\frac{1}{\sqrt{LC}}$</p><p> $U_L(jw_0)=U_C(jw_0)=QU_s(jw_0)$</p><p> $Q=\frac{w_0L}{R}=\frac{1}{W_0CR}=\frac{1}{R}\sqrt{\frac{L}{C}}$</p><p> $BW=\frac{w_0}{Q}$</p><p> </p><p> 2.$RLC$并联电路</p><p><img src="https://z3.ax1x.com/2021/10/08/5P89VU.jpg" alt=""></p><p> $w_0=\frac{1}{\sqrt{LC}}$</p><p> $Y(jw_0)=G$</p><p> $Q=\frac{I_L(w_0)}{I_s}=\frac{I_C(w_0)}{I_s}=\frac{1}{W_0LG}=\frac{w_0C}{G}=\frac{1}{G}\sqrt{\frac{C}{L}}$</p><p>3.另一种$RLC$并联电路</p><p><img src="https://z3.ax1x.com/2021/10/08/5PGg1A.jpg" alt=""></p><p> $w_0=\frac{1}{\sqrt{LC}}\sqrt{1-\frac{CR^2}{L}}$</p><p> $Y(jw_0)=\frac{CR}{L}$</p><h3 id="22-波特图"><a href="#22-波特图" class="headerlink" title="22.波特图"></a>22.波特图</h3><p>$\enclose{circle}{1}$幅频波特图</p><p> $H_{dB}=20lg(|H(jw)|)$</p><p> $20lg(|jw|)$为$20dB/10$倍频的直线</p><p> $-20lg(|1+j\frac{w}{a}|)$由两段直线构成,$w<a$时，斜率约等于0，$w>a$时，斜率约等于$-20dB/10$倍频</p><p>$\enclose{circle}{2}$相频波特图</p><p>$-arctan(\frac{w}{a})$由三段直线构成，$0<w<\frac{a}{10}$间为$0\degree$,$\frac{a}{10}<w<a$间为$-45\degree/10$倍频的直线，$w>a$时为$-90\degree$</p><h3 id="23-三相电路负载有功功率"><a href="#23-三相电路负载有功功率" class="headerlink" title="23.三相电路负载有功功率"></a>23.三相电路负载有功功率</h3><p><img src="https://z3.ax1x.com/2021/10/09/5FUoy4.jpg" style="zoom:20%;" /></p><p> $P_{motor}=\sqrt{3}U_{A’B^”}I_A\lambda$</p><h3 id="24-三相电路中的戴维南"><a href="#24-三相电路中的戴维南" class="headerlink" title="24.三相电路中的戴维南"></a>24.三相电路中的戴维南</h3><p><img src="https://z3.ax1x.com/2021/10/09/5FdjMD.jpg" style="zoom:25%;" /></p><p> $S$打开后，可以等效为</p><p> <img src="https://z3.ax1x.com/2021/10/09/5FBNP1.jpg" style="zoom:20%;" /></p><h3 id="25-非正弦周期函数的傅里叶级数"><a href="#25-非正弦周期函数的傅里叶级数" class="headerlink" title="25.非正弦周期函数的傅里叶级数"></a>25.非正弦周期函数的傅里叶级数</h3><p>满足收敛条件的周期函数f(t)可以展开为</p><p> $f(t)=\frac{a_0}{2}+\sum\limits_{k=1}^{\infty}[{a_k}cos(kw_1t)+{b_k}sin(kw_1t)]$</p><p> $=\frac{A_0}{2}+\sum\limits_{k=1}^{\infty}A_{km}cos(kw_1t+\Phi_k)$</p><p> $cos\Phi_k=\frac{a_k}{A_{km}}$ ， $sin\Phi_k=\frac{-b_k}{A_{km}}$</p><p> $A_{km}e^{j\Phi_k}=a_k-jb_k=\frac{2}{T}\int_0^Tf(t)e^{-jkw_1}dt$</p><p> $a_k=\frac{2}{T}\int_0^Tf(t)cos(kw_1t)dt=\frac{2}{T}\int_{-\frac{T}{2}}^{\frac{T}{2}}f(t)cos(kw_1t)dt$</p><p> $b_k=\frac{2}{T}\int_0^Tf(t)sin(kw_1t)dt=\frac{2}{T}\int_{-\frac{T}{2}}^{\frac{T}{2}}f(t)sin(kw_1t)dt$</p><p>指数形式的傅里叶级数</p><p> $c_k=\frac{1}{T}\int_0^Tf(t)e^{-jkw_!}dt$</p><p> $f(t)=\sum\limits_{k=-{\infty}}^{\infty}c_ke^{jkw_1t}$</p><p>特例：</p><p> $f(t)=-f(t+\frac{T}{2})$时，级数展开式不含偶次谐波</p><h3 id="26-不同倍频下的三相电路"><a href="#26-不同倍频下的三相电路" class="headerlink" title="26.不同倍频下的三相电路"></a>26.不同倍频下的三相电路</h3><p><img src="https://z3.ax1x.com/2021/10/09/5FqNFA.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/10/09/5FhkrR.jpg" style="zoom:20%;" /></p><p>$\enclose{circle}{1}$单倍频下三相电路为正序对称电路 ，$\dot{I_B}=a^2\dot{I_A}$ ,$\dot{I_C}=a\dot{I_A}$ </p><p>$\enclose{circle}{2}$三倍频下三相电路为零序对称电路，$\dot{I_A}=\dot{I_B}=\dot{I_C}$</p><p>$\enclose{circle}{3}$五倍频下三相电路为逆序对称电路，$\dot{I_B}=a\dot{I_A}$ ，$\dot{I_C}=a^2\dot{I_A}$ </p><h3 id="27-拉普拉斯变换"><a href="#27-拉普拉斯变换" class="headerlink" title="27.拉普拉斯变换"></a>27.拉普拉斯变换</h3><p><img src="https://z3.ax1x.com/2021/10/09/5Fxfp9.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/10/09/5FxhlR.jpg" style="zoom: 50%;" /></p><h3 id="28-拉普拉斯逆变换"><a href="#28-拉普拉斯逆变换" class="headerlink" title="28.拉普拉斯逆变换"></a>28.拉普拉斯逆变换</h3><p>$\enclose{circle}{1}$.重根</p><p>对于$F(s)=\frac{K_{11}}{(s-p_1)^q}+\frac{K_{12}}{(s-p_1)^{q-1}}+…+\frac{K_{1q}}{(s-p_1)}$</p><p>$K_{1q}=\frac{1}{(q-1)!}\frac{d^{q-1}}{ds^{q-1}}[(s-p_1)^qF(s)]|_{s=p_1}$</p><p>$\enclose{circle}{2}$.共轭复根</p><p>对于$F(s)=\frac{1}{(s-\alpha-jw)(s-\alpha+jw)}=\frac{K_1}{s-\alpha-jw}+\frac{K_2}{s-\alpha+jw}$</p><p>$K_1=[(s-\alpha-jw)F(s)]_{s=\alpha+jw}$</p><p>$K_2=[(s-\alpha+jw)F(s)]_{s=\alpha-jw}$</p><p>$f(t)=2|K_1|e^{\alpha t}cos(w_1t+\theta_1)$</p><h3 id="29-运算法中电容电感的初值等效"><a href="#29-运算法中电容电感的初值等效" class="headerlink" title="29.运算法中电容电感的初值等效"></a>29.运算法中电容电感的初值等效</h3><p>$\enclose{circle}{1}$电感</p><p><img src="https://z3.ax1x.com/2021/10/11/5ZTlfs.jpg" alt=""></p><p>$\enclose{circle}{2}$电容</p><p><img src="https://z3.ax1x.com/2021/10/11/5ZTQYj.jpg" alt=""></p><p>$\enclose{circle}{3}$耦合电感</p><p><img src="https://z3.ax1x.com/2021/10/11/5ZT3pn.jpg" alt=""></p><p>==<strong>同向耦合时，自感电压与互感电压均与电流反向；反向耦合时，自感电压与电流反向，互感电压则与电流同向！！！</strong>==</p><h3 id="30-二端口"><a href="#30-二端口" class="headerlink" title="30.二端口"></a>30.二端口</h3><p>——————参数矩阵</p><p>$\enclose{circle}{1}$Y参数矩阵</p><script type="math/tex; mode=display">\left\{ \begin{aligned} \dot{I_1}=Y_{11}\dot{U_1}+Y_{12}\dot{U_2} \\ \dot{I_2}=Y_{21}\dot{U_1}+Y_{22}\dot{U_2} \end{aligned} \right.</script><p>$Y=Z^{-1}$</p><p>$\enclose{circle}{2}$Z参数矩阵</p><script type="math/tex; mode=display">\left\{ \begin{aligned} \dot{U_1}=Z_{11}\dot{I_1}+Z_{12}\dot{I_2} \\ \dot{U_2}=Z_{21}\dot{I_1}+Z_{22}\dot{I_2} \end{aligned} \right.</script><p>$Z=Y^{-1}$</p><p>没有受控源情况下，$Y_{12}=Y_{21}$ , $Z_{12}=Z_{21}$ , 若端口对称，还有$Y_{11}=Y_{22}$ , $Z_{11}=Z_{22}$</p><p>$\enclose{circle}{3}$T参数矩阵</p><script type="math/tex; mode=display">\left\{ \begin{aligned} \dot{U_1}=A\dot{U_2}-B\dot{I_2} \\ \dot{I_1}=C\dot{U_2}-D\dot{I_2} \end{aligned} \right.</script><p>对于无源线性二端口，有$AD-BC=1$,若端口对称，还将有$A=D$</p><p>$\enclose{circle}{4}$H参数矩阵</p><script type="math/tex; mode=display">\left\{ \begin{aligned} \dot{U_1}=H_{11}\dot{I_1}+H_{12}\dot{U_2} \\ \dot{I_2}=H_{21}\dot{I_1}+H_{22}\dot{U_2} \end{aligned} \right.</script><p>对于无源线性二端口，有$H_{21}=-H_{12}$,若端口对称，还将有$H_{11}H_{22}-H_{12}H_{21}=1$</p><p>—————二端口的等效电路</p><p><img src="https://z3.ax1x.com/2021/10/11/5ZLhl9.jpg" alt=""></p><p> $Z_1=Z_{11}-Z_{12} , Z_2=Z_{12} , Z_3=Z_{22}-Z_{12}$</p><p><img src="https://z3.ax1x.com/2021/10/11/5ZLfSJ.jpg" alt=""></p><p> $Y_1=Y_{11}+Y_{12} , Y_2=-Y_{12}=-Y_{21} , Y_3=Y_{22}+Y_{21}$</p><p> 如果给出的是二端口的T或H参数，需转换为Y或Z之后再求其$T$形等效电路或$\Pi$形电路</p><p>如果二端口内部含有受控源，那么</p><p><img src="https://z3.ax1x.com/2021/10/11/5ZO59g.jpg" alt=""></p><p>—————————二端口的连接（级联，并联，串联）</p><p>$\enclose{circle}{1}$级联</p><p><img src="https://z3.ax1x.com/2021/10/11/5ZjPzQ.jpg" alt=""></p><p> $T=T’T^”$</p><p>$\enclose{circle}{2}$串联</p><p><img src="https://z3.ax1x.com/2021/10/11/5Zj9JS.jpg" alt=""></p><p> $Z=Z’+Z^”$</p><p>$\enclose{circle}{3}$并联</p><p><img src="https://z3.ax1x.com/2021/10/11/5ZjCRg.jpg" alt=""></p><h3 id="Y-Y’-Y-”"><a href="#Y-Y’-Y-”" class="headerlink" title="$Y=Y’+Y^”$"></a>$Y=Y’+Y^”$</h3><p>—————————回转器</p><p><img src="https://z3.ax1x.com/2021/10/11/5ZvV6H.jpg" alt=""></p><p>回转器是一种线性非互易的多端元件，可以将电压或电流反向或是实现负的R、L、C</p><p>它的端口电压、电流关系可以表示为</p><script type="math/tex; mode=display">\left\{ \begin{aligned} u_1=-ri_2 \\ u_2=ri_1 \end{aligned} \right.</script><p>或</p><script type="math/tex; mode=display">\left\{ \begin{aligned} i_1=gu_2 \\ i_2=-gu_1 \end{aligned} \right.</script><h3 id="31-立方体等效电阻计算"><a href="#31-立方体等效电阻计算" class="headerlink" title="31.立方体等效电阻计算"></a>31.立方体等效电阻计算</h3><h3 id="32-动态电路黑盒"><a href="#32-动态电路黑盒" class="headerlink" title="32.动态电路黑盒"></a>32.动态电路黑盒</h3><p><img src="https://z3.ax1x.com/2021/11/08/IJJcrR.jpg" style="zoom: 19%;" /></p><p><img src="https://z3.ax1x.com/2021/11/08/IJJgq1.jpg" style="zoom:12%;" /></p><h3 id="33-非对称三相电路的处理"><a href="#33-非对称三相电路的处理" class="headerlink" title="33.非对称三相电路的处理"></a>33.非对称三相电路的处理</h3><p>1.节点电压法 </p><p><img src="https://z3.ax1x.com/2021/11/08/IJJRVx.jpg" style="zoom: 15%;" /> </p><p>2.戴维南或诺顿等效</p><h3 id="34-三相电路中的去耦等效"><a href="#34-三相电路中的去耦等效" class="headerlink" title="34.三相电路中的去耦等效"></a>34.三相电路中的去耦等效</h3><h3 id=""><a href="#" class="headerlink" title=" "></a> </h3>]]></content>
<categories>
<category> 电路 </category>
</categories>
</entry>
<entry>
<title>Gazebo下的双臂协作机器人的工件装配（三）基于点云和yolo</title>
<link href="2021/06/26/Gazebo%E4%B8%8B%E7%9A%84%E5%8F%8C%E8%87%82%E5%8D%8F%E4%BD%9C%E6%9C%BA%E5%99%A8%E4%BA%BA%E7%9A%84%E5%B7%A5%E4%BB%B6%E8%A3%85%E9%85%8D%EF%BC%88%E5%9B%9B%EF%BC%89%E5%9F%BA%E4%BA%8E%E7%82%B9%E4%BA%91%E5%92%8Cyolo/"/>
<url>2021/06/26/Gazebo%E4%B8%8B%E7%9A%84%E5%8F%8C%E8%87%82%E5%8D%8F%E4%BD%9C%E6%9C%BA%E5%99%A8%E4%BA%BA%E7%9A%84%E5%B7%A5%E4%BB%B6%E8%A3%85%E9%85%8D%EF%BC%88%E5%9B%9B%EF%BC%89%E5%9F%BA%E4%BA%8E%E7%82%B9%E4%BA%91%E5%92%8Cyolo/</url>
<content type="html"><![CDATA[<h2 id="安装PCL"><a href="#安装PCL" class="headerlink" title="安装PCL"></a>安装PCL</h2><p>PCL（Point Cloud Library）是在吸收了前人点云相关研究基础上建立起来的大型跨平台开源C++编程库，它实现了大量点云相关的通用算法和高效数据结构，涉及到点云获取、滤波、分割、配准、检索、特征提取、识别、追踪、曲面重建、可视化等。</p><p>具体安装过程不在赘述，大致安装过程如下：</p><p>1.安装各种依赖库</p><p>2.Qt4切换到Qt5</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">qmake -v</span><br><span class="line">cd /usr/lib/x86_64-linux-gnu/qt-default/qtchooser</span><br><span class="line">sudo gedit default.conf </span><br></pre></td></tr></table></figure><p>将里面的Qt4目录改为QT5的安装目录</p><p>3.通过编译源码安装VTK7.1.1，需要cmake gui</p><p>4.推荐直接通过命令行安装，通过源码安装依赖太多编译不容易通过</p><p>（通过命令行安装的PCL版本是1.7.2）</p><p>安装好后可以找个例程测试一下pcl visualizer能否正常显示</p><p>相关例程及cmakelist百度云:<a href="https://pan.baidu.com/s/1hZU0FzA3iyXTCS-AHXWS1Q">https://pan.baidu.com/s/1hZU0FzA3iyXTCS-AHXWS1Q</a></p><h2 id="搭建环境"><a href="#搭建环境" class="headerlink" title="搭建环境"></a>搭建环境</h2><p>该world在原来食品分类的基础上修改得到</p><p>工件STL模型百度云：<a href="https://pan.baidu.com/s/13mkX0bAiHVC86QqyWYthbQ">https://pan.baidu.com/s/13mkX0bAiHVC86QqyWYthbQ</a></p><p><img src="https://z3.ax1x.com/2021/06/26/RGwviq.png" alt=""></p><p><strong><em>由于相机畸变等因素，对于较长的柱状工件，其侧面在点云中或是RGB图像中对位姿估计的干扰都太大，因此为了 简化任务量，索性将任务改为将正方体工件装配到固定位置的柱状工件上面</em></strong></p><h2 id="位姿估计"><a href="#位姿估计" class="headerlink" title="位姿估计"></a>位姿估计</h2><ol><li>位姿估计采用点云进行欧几里得聚类和RANSAC拟合直线的方法；</li><li>至于正方体的分类并没有采用RGB图像进行边缘检测和多边形逼近，而是采用了深度神经网络进行分类再将分类的预测框中心和各个聚类的中心进行匹配从而得出各个聚类对应的工件名称</li></ol><h3 id="点云处理"><a href="#点云处理" class="headerlink" title="点云处理"></a>点云处理</h3><p>创建功能包</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">catkin_create_pkg pclproc pcl_conversions pcl_ros pcl_msgs roscpp rospy std_msgs darknet_ros_msgs sensor_msgs</span><br><span class="line">catkin_make -DCATKIN_WHITELIST_PACKAGES=<span class="string">"pclproc"</span></span><br></pre></td></tr></table></figure><h4 id="1-格式转换"><a href="#1-格式转换" class="headerlink" title="1.格式转换"></a>1.格式转换</h4><p>由 <code>sensor_msgs::PCLPointCloud2</code>转换为 <code>pcl::PointCloud<pcl::PointXYZ></code></p><p>具体方法方法可参考（<a href="https://www.cnblogs.com/li-yao7758258/p/6659451.html">PCL中点云数据格式之间的转化 - Being_young - 博客园 (cnblogs.com)</a>）</p><p> <code>void pcl::toROSMsg(const pcl::PointCloud<T> &, sensor_msgs::PointCloud2 &);</code></p><h4 id="2-直通滤波"><a href="#2-直通滤波" class="headerlink" title="2.直通滤波"></a>2.直通滤波</h4><p>对于在空间分布有一定空间特征的点云数据，比如使用线结构光扫描的方式采集点云，沿z向分布较广，但x,y向的分布处于有限范围内。此时可使用直通滤波器，确定点云在x或y方向上的范围，可较快剪除离群点，达到第一步粗处理的目的。</p><figure class="highlight c++"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">cout << <span class="string">" Cloud before filtering: "</span> << cloud->points.<span class="built_in">size</span>() << endl;</span><br><span class="line">pcl::PassThrough<pcl::PointXYZ> pass;</span><br><span class="line">pass.<span class="built_in">setInputCloud</span>(cloud);</span><br><span class="line">pass.<span class="built_in">setFilterFieldName</span>(<span class="string">"z"</span>);<span class="comment">//设置过滤时所需要的点云Z字段</span></span><br><span class="line">pass.<span class="built_in">setFilterLimits</span>(<span class="number">0.49</span>, <span class="number">0.52</span>);<span class="comment">//设置在过滤字段上的范围</span></span><br><span class="line">pass.<span class="built_in">setNegative</span>(<span class="literal">false</span>);<span class="comment">//设置保留范围内的还是过滤掉范围内的</span></span><br><span class="line">pass.<span class="built_in">filter</span>(*cloudf);</span><br><span class="line">cout << <span class="string">"Cloud after filtering: "</span> << cloudf->points.<span class="built_in">size</span>() << endl;</span><br></pre></td></tr></table></figure><h4 id="3-体素栅格下采样"><a href="#3-体素栅格下采样" class="headerlink" title="3.体素栅格下采样"></a>3.体素栅格下采样</h4><p>由于点云的海量和无序性，直接处理的方式在对邻域进行搜索时需要较高的计算成本。一个常用的解决方式就是对点云进行下采样，将对全部点云的操作转换到下采样所得到的点上，降低计算量；所谓体素栅格是将点云空间划分为一个个极小的格子，格子里包含几个点。对体素栅格里面的点取平均或者加权平均，得到一个点，代替原来的几个点。显然，栅格选的越大，滤波后点云点数越少，速度快，但是会对原来的点云过度模糊;栅格选的越小，作用反之。</p><figure class="highlight c++"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">std::cout << <span class="string">"PointCloud before sampling has: "</span> << cloudf->points.<span class="built_in">size</span> () << <span class="string">" data points."</span> << std::endl; <span class="comment">//*</span></span><br><span class="line"><span class="comment">/*从输入的.PCD文件载入数据后，我们创建了一个VoxelGrid滤波器对数据进行下采样，我们在这里进行下采样的原因是来加速处理过程，越少的点意味着分割循环中处理起来越快。*/</span></span><br><span class="line"><span class="comment">// Create the filtering object: downsample the dataset using a leaf size of 1cm</span></span><br><span class="line">pcl::VoxelGrid<pcl::PointXYZ> vg; <span class="comment">//体素栅格下采样对象</span></span><br><span class="line">pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">cloud_filtered</span> <span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line">vg.<span class="built_in">setInputCloud</span> (cloudf);</span><br><span class="line">vg.<span class="built_in">setLeafSize</span> (<span class="number">0.005f</span>, <span class="number">0.005f</span>, <span class="number">0.005f</span>); <span class="comment">//设置采样的体素大小</span></span><br><span class="line">vg.<span class="built_in">filter</span> (*cloud_filtered); <span class="comment">//执行采样保存数据</span></span><br><span class="line">std::cout << <span class="string">"PointCloud after sampling has: "</span> << cloud_filtered->points.<span class="built_in">size</span> () << <span class="string">" data points."</span> << std::endl; <span class="comment">//*</span></span><br></pre></td></tr></table></figure><p><img src="https://z3.ax1x.com/2021/06/26/RGW5YF.jpg" alt=""></p><p><strong><em>左图为直通滤波，右图为下采样</em></strong></p><h4 id="4-欧几里得聚类"><a href="#4-欧几里得聚类" class="headerlink" title="4.欧几里得聚类"></a>4.欧几里得聚类</h4><p><img src="https://z3.ax1x.com/2021/06/26/RGocy4.jpg" alt=""></p><figure class="highlight c++"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//欧几里德聚类</span></span><br><span class="line"> pcl::search::KdTree<pcl::PointXYZ>::<span class="function">Ptr <span class="title">tree</span> <span class="params">(<span class="keyword">new</span> pcl::search::KdTree<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> tree-><span class="built_in">setInputCloud</span> (cloud_filtered); <span class="comment">//创建点云索引向量，用于存储实际的点云信息</span></span><br><span class="line"> pcl::PCDWriter writer;</span><br><span class="line"> std::vector<pcl::PointIndices> cluster_indices;</span><br><span class="line"> pcl::EuclideanClusterExtraction<pcl::PointXYZ> ec;</span><br><span class="line"> ec.<span class="built_in">setClusterTolerance</span> (<span class="number">0.015</span>); <span class="comment">//设置近邻搜索的搜索半径为2cm</span></span><br><span class="line"> ec.<span class="built_in">setMinClusterSize</span> (<span class="number">100</span>);<span class="comment">//设置一个聚类需要的最少点数目为100</span></span><br><span class="line"> ec.<span class="built_in">setMaxClusterSize</span> (<span class="number">25000</span>);<span class="comment">//设置一个聚类需要的最大点数目为25000</span></span><br><span class="line"> ec.<span class="built_in">setSearchMethod</span> (tree);<span class="comment">//设置点云的搜索机制</span></span><br><span class="line"> ec.<span class="built_in">setInputCloud</span> (cloud_filtered);</span><br><span class="line"> ec.<span class="built_in">extract</span> (cluster_indices);<span class="comment">//从点云中提取聚类，并将点云索引保存在cluster_indices中</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">/*为了从点云索引向量中分割出每个聚类，必须迭代访问点云索引，每次创建一个新的点云数据集，并且将所有当前聚类的点写入到点云数据集中。*/</span></span><br><span class="line"> <span class="comment">//迭代访问点云索引cluster_indices，直到分割出所有聚类</span></span><br><span class="line"> <span class="keyword">int</span> j = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span> (std::vector<pcl::PointIndices>::const_iterator it = cluster_indices.<span class="built_in">begin</span> (); it != cluster_indices.<span class="built_in">end</span> (); ++it)</span><br><span class="line"> {</span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">cloud_cluster</span> <span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> <span class="comment">//创建新的点云数据集cloud_cluster，将所有当前聚类写入到点云数据集中</span></span><br><span class="line"> <span class="keyword">for</span> (std::vector<<span class="keyword">int</span>>::const_iterator pit = it->indices.<span class="built_in">begin</span> (); pit != it->indices.<span class="built_in">end</span> (); ++pit)</span><br><span class="line"> cloud_cluster->points.<span class="built_in">push_back</span> (cloud_filtered->points[*pit]); <span class="comment">//*</span></span><br><span class="line"> cloud_cluster->width = cloud_cluster->points.<span class="built_in">size</span> ();</span><br><span class="line"> cloud_cluster->height = <span class="number">1</span>;</span><br><span class="line"> cloud_cluster->is_dense = <span class="literal">true</span>;</span><br><span class="line"></span><br><span class="line"> std::cout << <span class="string">"PointCloud representing the Cluster: "</span> << cloud_cluster->points.<span class="built_in">size</span> () << <span class="string">" data points."</span> << std::endl;</span><br><span class="line"> std::stringstream ss;</span><br><span class="line"> ss << <span class="string">"cloud_cluster_"</span> << j << <span class="string">".pcd"</span>;</span><br><span class="line"> writer.write<pcl::PointXYZ> (ss.<span class="built_in">str</span> (), *cloud_cluster, <span class="literal">false</span>); <span class="comment">//*</span></span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p><img src="https://z3.ax1x.com/2021/06/26/RGwbLQ.png" alt=""></p><h4 id="5-边缘提取与质心估计"><a href="#5-边缘提取与质心估计" class="headerlink" title="5.边缘提取与质心估计"></a>5.边缘提取与质心估计</h4><p>对点云数据集的每个点的法线估计，可以看作是对表面法线的近似推断，从原始点云上计算出法线，再由法线结合数据估计出边界。</p><figure class="highlight c++"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//对各个聚类提取边缘</span></span><br><span class="line"> pcl::PointCloud<pcl::Normal>::<span class="function">Ptr <span class="title">normals</span><span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::Normal>)</span></span>;</span><br><span class="line"> pcl::PointCloud<pcl::Boundary> boundaries;</span><br><span class="line"> pcl::BoundaryEstimation<pcl::PointXYZ, pcl::Normal, pcl::Boundary> est;</span><br><span class="line"> pcl::search::KdTree<pcl::PointXYZ>::<span class="function">Ptr <span class="title">tree2</span><span class="params">(<span class="keyword">new</span> pcl::search::KdTree<pcl::PointXYZ>())</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">//pcl::NormalEstimation<pcl::PointXYZ, pcl::Normal> normEst; //其中pcl::PointXYZ表示输入类型数据，pcl::Normal表示输出类型,且pcl::Normal前三项是法向，最后一项是曲率</span></span><br><span class="line"> pcl::NormalEstimationOMP<pcl::PointXYZ, pcl::Normal> normEst;</span><br><span class="line"> normEst.<span class="built_in">setNumberOfThreads</span>(<span class="number">12</span>); <span class="comment">// 手动设置线程数，否则提示错误</span></span><br><span class="line"> normEst.<span class="built_in">setInputCloud</span>(cloud_cluster);</span><br><span class="line"> normEst.<span class="built_in">setSearchMethod</span>(tree2);</span><br><span class="line"> normEst.<span class="built_in">setRadiusSearch</span>(<span class="number">0.02</span>); <span class="comment">//法向估计的半径</span></span><br><span class="line"> <span class="comment">//normEst.setKSearch(9); //法向估计的点数</span></span><br><span class="line"> normEst.<span class="built_in">compute</span>(*normals);</span><br><span class="line"> <span class="comment">//cout << "normal size is " << normals->size() << endl;</span></span><br><span class="line"> est.<span class="built_in">setInputCloud</span>(cloud_cluster);</span><br><span class="line"> est.<span class="built_in">setInputNormals</span>(normals);</span><br><span class="line"> est.<span class="built_in">setSearchMethod</span>(tree2);</span><br><span class="line"> est.<span class="built_in">setKSearch</span>(<span class="number">500</span>); <span class="comment">//一般这里的数值越高，最终边界识别的精度越好</span></span><br><span class="line"> est.<span class="built_in">compute</span>(boundaries);</span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">boundPoints</span><span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ> noBoundPoints;</span><br><span class="line"> <span class="keyword">int</span> countBoundaries = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i<cloud_cluster-><span class="built_in">size</span>(); i++) </span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">uint8_t</span> x = (boundaries.points[i].boundary_point);</span><br><span class="line"> <span class="keyword">int</span> a = <span class="keyword">static_cast</span><<span class="keyword">int</span>>(x); <span class="comment">//该函数的功能是强制类型转换</span></span><br><span class="line"> <span class="keyword">if</span> (a == <span class="number">1</span>)</span><br><span class="line"> {</span><br><span class="line"> (*boundPoints).<span class="built_in">push_back</span>(cloud_cluster->points[i]);</span><br><span class="line"> countBoundaries++;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> noBoundPoints.<span class="built_in">push_back</span>(cloud_cluster->points[i]);</span><br><span class="line"> }</span><br><span class="line"> std::cout << <span class="string">"boundary "</span><<j<<<span class="string">" size is："</span> << countBoundaries << std::endl;</span><br></pre></td></tr></table></figure><p><img src="https://z3.ax1x.com/2021/06/26/RGwIRf.png" alt=""></p><p>质心估计</p><figure class="highlight c++"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">Eigen::Vector4f centroid;</span><br><span class="line">pcl::<span class="built_in">compute3DCentroid</span>(*boundPoints, centroid); </span><br></pre></td></tr></table></figure><p>得到的centroid[]是一个长度为3的数组</p><h4 id="6-RANSAC拟合边缘直线"><a href="#6-RANSAC拟合边缘直线" class="headerlink" title="6.RANSAC拟合边缘直线"></a>6.RANSAC拟合边缘直线</h4><p>RANSAC(RAndom SAmple Consensus,随机采样一致)算法是从一组含有“外点”(outliers)的数据中正确估计数学模型参数的迭代算法，一直迭代到估计出认为比较好的模型。<br>具体的实现步骤可以分为以下几步：</p><ol><li>选择出可以估计出模型的最小数据集；(对于直线拟合来说就是两个点，对于计算Homography矩阵就是4个点)</li><li>使用这个数据集来计算出数据模型；</li><li>将所有数据带入这个模型，计算出“内点”的数目；(累加在一定误差范围内的适合当前迭代推出模型的数据)</li><li>比较当前模型和之前推出的最好的模型的“内点“的数量，记录最大“内点”数的模型参数和“内点”数；</li><li>重复1-4步，直到迭代结束或者当前模型已经足够好了(“内点数目大于一定数量”)。</li></ol><figure class="highlight c++"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//对边缘的四条边分别通过RANSAC进行直线拟合</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">double</span> list_dir[<span class="number">2</span>];</span><br><span class="line"> <span class="keyword">double</span> list_pre_dir[<span class="number">4</span>];</span><br><span class="line"> Eigen::Vector4f centroid;</span><br><span class="line"> pcl::<span class="built_in">compute3DCentroid</span>(*boundPoints, centroid); </span><br><span class="line"> <span class="comment">// std::cout << "点云 "<<j<<" 质心是（"</span></span><br><span class="line"> <span class="comment">// << 0.4-centroid[1] << ","</span></span><br><span class="line"> <span class="comment">// << -centroid[0] << ","</span></span><br><span class="line"> <span class="comment">// << centroid[2] << ")." << std::endl; </span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">//创建一个模型参数对象，用于记录结果</span></span><br><span class="line"> pcl::<span class="function">ModelCoefficients::Ptr <span class="title">coefficients</span><span class="params">(<span class="keyword">new</span> pcl::ModelCoefficients)</span></span>;</span><br><span class="line"> pcl::<span class="function">PointIndices::Ptr <span class="title">inliers</span><span class="params">(<span class="keyword">new</span> pcl::PointIndices)</span></span>; <span class="comment">//inliers表示误差能容忍的点 记录的是点云的序号</span></span><br><span class="line"> pcl::SACSegmentation<pcl::PointXYZ> seg2; <span class="comment">// 创建一个分割器</span></span><br><span class="line"> seg2.<span class="built_in">setOptimizeCoefficients</span>(<span class="literal">true</span>); <span class="comment">// Optional，这个设置可以选定结果平面展示的点是分割掉的点还是分割剩下的点。</span></span><br><span class="line"> seg2.<span class="built_in">setModelType</span>(pcl::SACMODEL_LINE); <span class="comment">// Mandatory-设置目标几何形状</span></span><br><span class="line"> seg2.<span class="built_in">setMethodType</span>(pcl::SAC_RANSAC); <span class="comment">//分割方法：随机采样法</span></span><br><span class="line"> seg2.<span class="built_in">setDistanceThreshold</span>(<span class="number">0.001</span>); <span class="comment">//设置误差容忍范围，也就是阈值</span></span><br><span class="line"></span><br><span class="line"> </span><br><span class="line"> </span><br><span class="line"> <span class="keyword">for</span>(<span class="keyword">int</span> i=<span class="number">0</span>;i<<span class="number">4</span>;i++)</span><br><span class="line"> { </span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">cloud_line</span><span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">cloud_out_line</span><span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> seg2.<span class="built_in">setInputCloud</span>(boundPoints); <span class="comment">//输入点云</span></span><br><span class="line"> seg2.<span class="built_in">segment</span>(*inliers, *coefficients); <span class="comment">//分割点云，获得平面和法向量</span></span><br><span class="line"> <span class="comment">//打印直线方程</span></span><br><span class="line"> <span class="comment">//std::cout << "d：" << coefficients->values[3] << endl;</span></span><br><span class="line"> <span class="comment">//std::cout << "e：" << coefficients->values[4] << endl;</span></span><br><span class="line"> list_pre_dir[i]=coefficients->values[<span class="number">4</span>]/coefficients->values[<span class="number">3</span>];</span><br><span class="line"> <span class="comment">//std::cout << list_pre_dir[i]<<endl;</span></span><br><span class="line"> pcl::ExtractIndices<pcl::PointXYZ> extract; <span class="comment">//创建点云提取对象</span></span><br><span class="line"> extract.<span class="built_in">setInputCloud</span> (boundPoints); <span class="comment">//设置输入点云</span></span><br><span class="line"> extract.<span class="built_in">setIndices</span> (inliers); <span class="comment">//设置分割后的内点为需要提取的点集</span></span><br><span class="line"> extract.<span class="built_in">setNegative</span> (<span class="literal">false</span>); <span class="comment">//设置提取内点而非外点</span></span><br><span class="line"> extract.<span class="built_in">filter</span> (*cloud_line); <span class="comment">//提取输出存储到cloud_line</span></span><br><span class="line"> std::cout << <span class="string">"PointCloud"</span><<i<< <span class="string">"representing the line component: "</span> << cloud_line->points.<span class="built_in">size</span> () << <span class="string">" data points."</span> << std::endl;</span><br><span class="line"> <span class="comment">// Remove the planar inliers, extract the rest</span></span><br><span class="line"> extract.<span class="built_in">setNegative</span> (<span class="literal">true</span>);</span><br><span class="line"> extract.<span class="built_in">filter</span> (*cloud_out_line);</span><br><span class="line"> *boundPoints = *cloud_out_line;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p>由于正方形边缘具有四条边，每拟合出一条边余下的点便继续用于直线拟合，最后会拟合出四条直线，根据直线的前两个方向向量之比估算出直线斜率，四条直线斜率符号相同的两两之和取平均，便有了工件坐标系的x,y轴分量</p><p><img src="https://z3.ax1x.com/2021/06/26/RGwoz8.png" alt=""></p><h4 id="7-工件分类"><a href="#7-工件分类" class="headerlink" title="7.工件分类"></a>7.工件分类</h4><p>对于分类问题，尝试过1.用聚类中的点云数目比较大小，2.用上表面的高度比较大小—-但是效果都不太好<br>3.决定采用yolo进行分类，然后将预测框与之前点云估计出的质心相匹配，为什么这一步是跟pointcloud结合而不是rgb图像？1.正方体侧面干扰太大2.正方体是规则几何体，主方向容易取到对角线</p><p>采用yolo-v3的ros版本—darknet_ros,实现视频流的检测与分类</p><p>数据集百度云：链接：<a href="https://pan.baidu.com/s/11uRc4wOMqCsnLiakXJDeqg">https://pan.baidu.com/s/11uRc4wOMqCsnLiakXJDeqg</a><br>复制这段内容后打开百度网盘手机App，操作更方便哦—来自百度网盘超级会员V1的分享</p><p>权重文件百度云：链接：<a href="https://pan.baidu.com/s/1mz-b-k-9gyn-ViGrn6yI6g">https://pan.baidu.com/s/1mz-b-k-9gyn-ViGrn6yI6g</a><br>提取码：1234<br>复制这段内容后打开百度网盘手机App，操作更方便哦—来自百度网盘超级会员V1的分享</p><p><img src="https://z3.ax1x.com/2021/06/26/RGqVrF.png" alt=""></p><p>网络结构解析：</p><ol><li>Yolov3中，只有卷积层，通过<strong>调节卷积步长控制输出特征图的尺寸</strong>。所以对于输入图片尺寸没有特别限制。流程图中，输入图片以256*256作为样例。</li><li>Yolov3借鉴了<strong>金字塔特征图</strong>思想，<strong>小尺寸特征图用于检测大尺寸物体</strong>，而<strong>大尺寸特征图检测小尺寸物体</strong>。特征图的输出维度为 <img src="https://www.zhihu.com/equation?tex=N%5Ctimes+N+%5Ctimes+%5B3+%5Ctimes+%284+%2B+1+%2B+80%29%5D" alt="[公式]"> ， <img src="https://www.zhihu.com/equation?tex=N%5Ctimes+N" alt="[公式]"> 为输出特征图格点数，一共3个Anchor框，每个框有4维预测框数值 <img src="https://www.zhihu.com/equation?tex=t_x+%2Ct_y+%2Ct_w%2C+t_h" alt="[公式]"> ，1维预测框置信度，80维物体类别数。所以第一层特征图的输出维度为 <img src="https://www.zhihu.com/equation?tex=8+%5Ctimes+8+%5Ctimes+255" alt="[公式]"> 。</li><li>Yolov3总共输出3个特征图，第一个特征图下采样32倍，第二个特征图下采样16倍，第三个下采样8倍。输入图像经过Darknet-53（无全连接层），再经过Yoloblock生成的特征图被当作两用，第一用为经过3<em>3卷积层、1</em>1卷积之后生成特征图一，第二用为经过1*1卷积层加上采样层，与Darnet-53网络的中间层输出结果进行拼接，产生特征图二。同样的循环之后产生特征图三。</li><li>concat操作与加和操作的区别：加和操作来源于ResNet思想，将输入的特征图，与输出特征图对应维度进行相加，即 <img src="https://www.zhihu.com/equation?tex=y+%3D+f%28x%29%2Bx" alt="[公式]"> ；而concat操作源于DenseNet网络的设计思路，将特征图按照通道维度直接进行拼接，例如8<em>8</em>16的特征图与8<em>8</em>16的特征图拼接后生成8<em>8</em>32的特征图。</li><li>上采样层(upsample)：作用是将小尺寸特征图通过插值等方法，生成大尺寸图像。例如使用最近邻插值算法，将8<em>8的图像变换为16</em>16。上采样层不改变特征图的通道数。</li></ol><p>Yolo的整个网络，吸取了Resnet、Densenet、FPN的精髓，可以说是融合了目标检测当前业界最有效的全部技巧。</p><h4 id="8-yolo预测框与各个点云聚类匹配"><a href="#8-yolo预测框与各个点云聚类匹配" class="headerlink" title="8.yolo预测框与各个点云聚类匹配"></a>8.yolo预测框与各个点云聚类匹配</h4><p>思路就是计算两者质心的距离，根据最小距离匹配出对应的预测框和点云</p><h2 id="完整源码"><a href="#完整源码" class="headerlink" title="完整源码"></a>完整源码</h2><figure class="highlight c++"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br><span class="line">224</span><br><span class="line">225</span><br><span class="line">226</span><br><span class="line">227</span><br><span class="line">228</span><br><span class="line">229</span><br><span class="line">230</span><br><span class="line">231</span><br><span class="line">232</span><br><span class="line">233</span><br><span class="line">234</span><br><span class="line">235</span><br><span class="line">236</span><br><span class="line">237</span><br><span class="line">238</span><br><span class="line">239</span><br><span class="line">240</span><br><span class="line">241</span><br><span class="line">242</span><br><span class="line">243</span><br><span class="line">244</span><br><span class="line">245</span><br><span class="line">246</span><br><span class="line">247</span><br><span class="line">248</span><br><span class="line">249</span><br><span class="line">250</span><br><span class="line">251</span><br><span class="line">252</span><br><span class="line">253</span><br><span class="line">254</span><br><span class="line">255</span><br><span class="line">256</span><br><span class="line">257</span><br><span class="line">258</span><br><span class="line">259</span><br><span class="line">260</span><br><span class="line">261</span><br><span class="line">262</span><br><span class="line">263</span><br><span class="line">264</span><br><span class="line">265</span><br><span class="line">266</span><br><span class="line">267</span><br><span class="line">268</span><br><span class="line">269</span><br><span class="line">270</span><br><span class="line">271</span><br><span class="line">272</span><br><span class="line">273</span><br><span class="line">274</span><br><span class="line">275</span><br><span class="line">276</span><br><span class="line">277</span><br><span class="line">278</span><br><span class="line">279</span><br><span class="line">280</span><br><span class="line">281</span><br><span class="line">282</span><br><span class="line">283</span><br><span class="line">284</span><br><span class="line">285</span><br><span class="line">286</span><br><span class="line">287</span><br><span class="line">288</span><br><span class="line">289</span><br><span class="line">290</span><br><span class="line">291</span><br><span class="line">292</span><br><span class="line">293</span><br><span class="line">294</span><br><span class="line">295</span><br><span class="line">296</span><br><span class="line">297</span><br><span class="line">298</span><br><span class="line">299</span><br><span class="line">300</span><br><span class="line">301</span><br><span class="line">302</span><br><span class="line">303</span><br><span class="line">304</span><br><span class="line">305</span><br><span class="line">306</span><br><span class="line">307</span><br><span class="line">308</span><br><span class="line">309</span><br><span class="line">310</span><br><span class="line">311</span><br><span class="line">312</span><br><span class="line">313</span><br><span class="line">314</span><br><span class="line">315</span><br><span class="line">316</span><br><span class="line">317</span><br><span class="line">318</span><br><span class="line">319</span><br><span class="line">320</span><br><span class="line">321</span><br><span class="line">322</span><br><span class="line">323</span><br><span class="line">324</span><br><span class="line">325</span><br><span class="line">326</span><br><span class="line">327</span><br><span class="line">328</span><br><span class="line">329</span><br><span class="line">330</span><br><span class="line">331</span><br><span class="line">332</span><br><span class="line">333</span><br><span class="line">334</span><br><span class="line">335</span><br><span class="line">336</span><br><span class="line">337</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><vector></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><ctime></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><iostream></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><string></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span><span class="meta-string"><math.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><ros/ros.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><Eigen/Core></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/ModelCoefficients.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/point_types.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/io/pcd_io.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/filters/extract_indices.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/filters/voxel_grid.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/features/normal_3d.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/kdtree/kdtree.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/sample_consensus/method_types.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/sample_consensus/model_types.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/segmentation/sac_segmentation.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/segmentation/extract_clusters.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/filters/passthrough.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/point_cloud.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/visualization/pcl_visualizer.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><boost/thread/thread.hpp></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/console/parse.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/features/eigen.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/features/feature.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/features/normal_3d_omp.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/impl/point_types.hpp></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/features/boundary.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/visualization/cloud_viewer.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/filters/extract_indices.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/segmentation/progressive_morphological_filter.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/segmentation/sac_segmentation.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl/common/centroid.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><pcl_conversions/pcl_conversions.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sensor_msgs/PointCloud2.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><message_filters/sync_policies/approximate_time.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><message_filters/synchronizer.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><message_filters/subscriber.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><message_filters/time_synchronizer.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><darknet_ros_msgs/BoundingBoxes.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><darknet_ros_msgs/ObjectCount.h></span></span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> std;</span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> sensor_msgs;</span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> darknet_ros_msgs;</span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> message_filters;</span><br><span class="line"><span class="keyword">int</span> objectnum;</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">Bounding</span></span></span><br><span class="line"><span class="class">{</span></span><br><span class="line"> string obj;</span><br><span class="line"> <span class="keyword">int</span> center_x;</span><br><span class="line"> <span class="keyword">int</span> center_y;</span><br><span class="line">};</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">Grasp</span></span></span><br><span class="line"><span class="class">{</span></span><br><span class="line"> string name;</span><br><span class="line"> <span class="keyword">double</span> x;</span><br><span class="line"> <span class="keyword">double</span> y;</span><br><span class="line"> <span class="keyword">double</span> angle;</span><br><span class="line">};</span><br><span class="line">vector<Bounding>objects;</span><br><span class="line">vector<Grasp>grasps;</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">BubbleSort</span><span class="params">(<span class="keyword">double</span> arr[], <span class="keyword">int</span> n)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i < n - <span class="number">1</span>; i++) {</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> j = <span class="number">0</span>; j < n - i - <span class="number">1</span>; j++) {</span><br><span class="line"> <span class="keyword">if</span> (arr[j] > arr[j + <span class="number">1</span>]) {</span><br><span class="line"> <span class="keyword">double</span> temp = arr[j];</span><br><span class="line"> arr[j] = arr[j + <span class="number">1</span>];</span><br><span class="line"> arr[j + <span class="number">1</span>] = temp;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">msgtrans</span><span class="params">(<span class="keyword">const</span> BoundingBoxes::ConstPtr &data)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="comment">//由于这个vector是全局变量，每次使用前要清空</span></span><br><span class="line"> objects.<span class="built_in">clear</span>();</span><br><span class="line"> Bounding b;</span><br><span class="line"> <span class="keyword">for</span>(<span class="keyword">int</span> i=<span class="number">0</span>;i<objectnum;i++)</span><br><span class="line"> {</span><br><span class="line"> <span class="comment">//此处作用：将订阅消息中的类别名和预测框中心的像素坐标存入vector</span></span><br><span class="line"> b.obj=data->bounding_boxes[i].Class;</span><br><span class="line"> b.center_x=<span class="built_in">round</span>(((data->bounding_boxes[i].xmin)+(data->bounding_boxes[i].xmax))/<span class="number">2</span>);</span><br><span class="line"> b.center_y=<span class="built_in">round</span>(((data->bounding_boxes[i].ymin)+(data->bounding_boxes[i].ymax))/<span class="number">2</span>);</span><br><span class="line"> objects.<span class="built_in">push_back</span>(b);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">yolomatchpcl</span><span class="params">(vector<Bounding>&a,vector<vector<<span class="keyword">double</span>> >&b)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> grasps.<span class="built_in">clear</span>();</span><br><span class="line"> <span class="comment">// for(int i=0;i<a.size();i++)</span></span><br><span class="line"> <span class="keyword">int</span> i,distance,minid=<span class="number">0</span>;</span><br><span class="line"> <span class="keyword">int</span> mindistance=<span class="number">10000</span>;</span><br><span class="line"> Grasp g;</span><br><span class="line"> <span class="keyword">while</span>(grasps.<span class="built_in">size</span>()<a.<span class="built_in">size</span>())</span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">for</span>(<span class="keyword">int</span> j=<span class="number">0</span>;j<a.<span class="built_in">size</span>();j++)</span><br><span class="line"> {</span><br><span class="line"> distance=(a[i].center_x-b[j][<span class="number">0</span>])*(a[i].center_x-b[j][<span class="number">0</span>])+(a[i].center_y-b[j][<span class="number">1</span>])*(a[i].center_y-b[j][<span class="number">1</span>]);</span><br><span class="line"> <span class="keyword">if</span>(distance<=mindistance)</span><br><span class="line"> {</span><br><span class="line"> mindistance=distance;</span><br><span class="line"> minid=j;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> cout<<mindistance<<endl;</span><br><span class="line"> g.name=a[i].obj;</span><br><span class="line"> g.x=b[minid][<span class="number">0</span>];</span><br><span class="line"> g.y=b[minid][<span class="number">1</span>];</span><br><span class="line"> g.angle=b[minid][<span class="number">2</span>];</span><br><span class="line"> grasps.<span class="built_in">push_back</span>(g);</span><br><span class="line"> minid=<span class="number">0</span>;</span><br><span class="line"> mindistance=<span class="number">10000</span>;</span><br><span class="line"> ++i;</span><br><span class="line"> }</span><br><span class="line"> cout<<<span class="string">"匹配完成"</span><<endl;</span><br><span class="line">}</span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">xtopixel</span><span class="params">(<span class="keyword">double</span> x)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="comment">//画面的y轴对应世界坐标系的x轴，所以需要颠倒过来，此处作用相当于手眼标定</span></span><br><span class="line"> <span class="keyword">int</span> ypixel=<span class="built_in">round</span>((<span class="number">0.4</span>-x)*<span class="number">480</span>/<span class="number">0.475</span>+<span class="number">240</span>);</span><br><span class="line"> <span class="keyword">return</span> ypixel;</span><br><span class="line">}</span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">ytopixel</span><span class="params">(<span class="keyword">double</span> y)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="comment">//画面的x轴对应世界坐标系的y轴，所以需要颠倒过来，此处作用相当于手眼标定</span></span><br><span class="line"> <span class="keyword">int</span> xpixel=<span class="built_in">round</span>(-y*<span class="number">640</span>/<span class="number">0.633</span>+<span class="number">320</span>);</span><br><span class="line"> <span class="keyword">return</span> xpixel;</span><br><span class="line">}</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">callback</span><span class="params">(<span class="keyword">const</span> PointCloud2::ConstPtr &msg1,</span></span></span><br><span class="line"><span class="params"><span class="function"> <span class="keyword">const</span> BoundingBoxes::ConstPtr &msg2,<span class="keyword">const</span> ObjectCount::ConstPtr &msg3)</span> </span>{</span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">cloud</span> <span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">cloudf</span> <span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> </span><br><span class="line"> objectnum=msg3->count;</span><br><span class="line"> <span class="comment">// pcl::PointCloud<pcl::PointXYZ> origincloud;</span></span><br><span class="line"> <span class="comment">//cloud=origincloud.makeShared();</span></span><br><span class="line"> <span class="comment">// pcl::fromROSMsg(*msg1, origincloud);</span></span><br><span class="line"></span><br><span class="line"> pcl::<span class="built_in">fromROSMsg</span>(*msg1, *cloud);</span><br><span class="line"> <span class="built_in">msgtrans</span>(msg2);</span><br><span class="line"> cout << <span class="string">" Cloud before filtering: "</span> << cloud->points.<span class="built_in">size</span>() << endl;</span><br><span class="line"> pcl::PassThrough<pcl::PointXYZ> pass;</span><br><span class="line"> pass.<span class="built_in">setInputCloud</span>(cloud);</span><br><span class="line"> pass.<span class="built_in">setFilterFieldName</span>(<span class="string">"z"</span>);<span class="comment">//设置过滤时所需要的点云Z字段</span></span><br><span class="line"> pass.<span class="built_in">setFilterLimits</span>(<span class="number">0.49</span>, <span class="number">0.52</span>);<span class="comment">//设置在过滤字段上的范围</span></span><br><span class="line"> pass.<span class="built_in">setNegative</span>(<span class="literal">false</span>);<span class="comment">//设置保留范围内的还是过滤掉范围内的</span></span><br><span class="line"> pass.<span class="built_in">filter</span>(*cloudf);</span><br><span class="line"> std::cout << <span class="string">"Cloud after filtering: "</span> << cloudf->points.<span class="built_in">size</span>() << endl;</span><br><span class="line"> std::cout << <span class="string">"PointCloud before sampling has: "</span> << cloudf->points.<span class="built_in">size</span> () << <span class="string">" data points."</span> << std::endl; <span class="comment">//*</span></span><br><span class="line"> <span class="comment">/*从输入的.PCD文件载入数据后，我们创建了一个VoxelGrid滤波器对数据进行下采样，我们在这里进行下采样的原因是来加速处理过程，越少的点意味着分割循环中处理起来越快。*/</span></span><br><span class="line"> <span class="comment">// Create the filtering object: downsample the dataset using a leaf size of 1cm</span></span><br><span class="line"> pcl::VoxelGrid<pcl::PointXYZ> vg; <span class="comment">//体素栅格下采样对象</span></span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">cloud_filtered</span> <span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> vg.<span class="built_in">setInputCloud</span> (cloudf);</span><br><span class="line"> vg.<span class="built_in">setLeafSize</span> (<span class="number">0.005f</span>, <span class="number">0.005f</span>, <span class="number">0.005f</span>); <span class="comment">//设置采样的体素大小</span></span><br><span class="line"> vg.<span class="built_in">filter</span> (*cloud_filtered); <span class="comment">//执行采样保存数据</span></span><br><span class="line"> std::cout << <span class="string">"PointCloud after sampling has: "</span> << cloud_filtered->points.<span class="built_in">size</span> () << <span class="string">" data points."</span> << std::endl; <span class="comment">//*</span></span><br><span class="line"> <span class="comment">//欧几里德聚类</span></span><br><span class="line"> pcl::search::KdTree<pcl::PointXYZ>::<span class="function">Ptr <span class="title">tree</span> <span class="params">(<span class="keyword">new</span> pcl::search::KdTree<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> tree-><span class="built_in">setInputCloud</span> (cloud_filtered); <span class="comment">//创建点云索引向量，用于存储实际的点云信息</span></span><br><span class="line"> <span class="comment">// pcl::PCDWriter writer;</span></span><br><span class="line"> std::vector<pcl::PointIndices> cluster_indices;</span><br><span class="line"> pcl::EuclideanClusterExtraction<pcl::PointXYZ> ec;</span><br><span class="line"> ec.<span class="built_in">setClusterTolerance</span> (<span class="number">0.015</span>); <span class="comment">//设置近邻搜索的搜索半径为2cm</span></span><br><span class="line"> ec.<span class="built_in">setMinClusterSize</span> (<span class="number">100</span>);<span class="comment">//设置一个聚类需要的最少点数目为100</span></span><br><span class="line"> ec.<span class="built_in">setMaxClusterSize</span> (<span class="number">25000</span>);<span class="comment">//设置一个聚类需要的最大点数目为25000</span></span><br><span class="line"> ec.<span class="built_in">setSearchMethod</span> (tree);<span class="comment">//设置点云的搜索机制</span></span><br><span class="line"> ec.<span class="built_in">setInputCloud</span> (cloud_filtered);</span><br><span class="line"> ec.<span class="built_in">extract</span> (cluster_indices);<span class="comment">//从点云中提取聚类，并将点云索引保存在cluster_indices中</span></span><br><span class="line"> <span class="comment">/*为了从点云索引向量中分割出每个聚类，必须迭代访问点云索引，每次创建一个新的点云数据集，并且将所有当前聚类的点写入到点云数据集中。*/</span></span><br><span class="line"> <span class="comment">//迭代访问点云索引cluster_indices，直到分割出所有聚类</span></span><br><span class="line"> <span class="keyword">int</span> j = <span class="number">0</span>;</span><br><span class="line"> vector<vector<<span class="keyword">double</span>> > out;</span><br><span class="line"> <span class="keyword">for</span> (std::vector<pcl::PointIndices>::const_iterator it = cluster_indices.<span class="built_in">begin</span> (); it != cluster_indices.<span class="built_in">end</span> (); ++it)</span><br><span class="line"> {</span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">cloud_cluster</span> <span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> <span class="comment">//创建新的点云数据集cloud_cluster，将所有当前聚类写入到点云数据集中</span></span><br><span class="line"> <span class="keyword">for</span> (std::vector<<span class="keyword">int</span>>::const_iterator pit = it->indices.<span class="built_in">begin</span> (); pit != it->indices.<span class="built_in">end</span> (); ++pit)</span><br><span class="line"> cloud_cluster->points.<span class="built_in">push_back</span> (cloud_filtered->points[*pit]); <span class="comment">//*</span></span><br><span class="line"> cloud_cluster->width = cloud_cluster->points.<span class="built_in">size</span> ();</span><br><span class="line"> cloud_cluster->height = <span class="number">1</span>;</span><br><span class="line"> cloud_cluster->is_dense = <span class="literal">true</span>;</span><br><span class="line"></span><br><span class="line"> std::cout << <span class="string">"PointCloud representing the Cluster: "</span> << cloud_cluster->points.<span class="built_in">size</span> () << <span class="string">" data points."</span> << std::endl;</span><br><span class="line"> <span class="comment">// std::stringstream ss;</span></span><br><span class="line"> <span class="comment">// ss << "cloud_cluster_" << j << ".pcd";</span></span><br><span class="line"> <span class="comment">// writer.write<pcl::PointXYZ> (ss.str (), *cloud_cluster, false); //*</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">//对各个聚类提取边缘</span></span><br><span class="line"> pcl::PointCloud<pcl::Normal>::<span class="function">Ptr <span class="title">normals</span><span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::Normal>)</span></span>;</span><br><span class="line"> pcl::PointCloud<pcl::Boundary> boundaries;</span><br><span class="line"> pcl::BoundaryEstimation<pcl::PointXYZ, pcl::Normal, pcl::Boundary> est;</span><br><span class="line"> pcl::search::KdTree<pcl::PointXYZ>::<span class="function">Ptr <span class="title">tree2</span><span class="params">(<span class="keyword">new</span> pcl::search::KdTree<pcl::PointXYZ>())</span></span>;</span><br><span class="line"></span><br><span class="line"> <span class="comment">//pcl::NormalEstimation<pcl::PointXYZ, pcl::Normal> normEst; //其中pcl::PointXYZ表示输入类型数据，pcl::Normal表示输出类型,且pcl::Normal前三项是法向，最后一项是曲率</span></span><br><span class="line"> pcl::NormalEstimationOMP<pcl::PointXYZ, pcl::Normal> normEst;</span><br><span class="line"> normEst.<span class="built_in">setNumberOfThreads</span>(<span class="number">12</span>); <span class="comment">// 手动设置线程数，否则提示错误</span></span><br><span class="line"> normEst.<span class="built_in">setInputCloud</span>(cloud_cluster);</span><br><span class="line"> normEst.<span class="built_in">setSearchMethod</span>(tree2);</span><br><span class="line"> normEst.<span class="built_in">setRadiusSearch</span>(<span class="number">0.02</span>); <span class="comment">//法向估计的半径</span></span><br><span class="line"> <span class="comment">//normEst.setKSearch(9); //法向估计的点数</span></span><br><span class="line"> normEst.<span class="built_in">compute</span>(*normals);</span><br><span class="line"> <span class="comment">//cout << "normal size is " << normals->size() << endl;</span></span><br><span class="line"> est.<span class="built_in">setInputCloud</span>(cloud_cluster);</span><br><span class="line"> est.<span class="built_in">setInputNormals</span>(normals);</span><br><span class="line"> est.<span class="built_in">setSearchMethod</span>(tree2);</span><br><span class="line"> est.<span class="built_in">setKSearch</span>(<span class="number">500</span>); <span class="comment">//一般这里的数值越高，最终边界识别的精度越好</span></span><br><span class="line"> est.<span class="built_in">compute</span>(boundaries);</span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">boundPoints</span><span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ> noBoundPoints;</span><br><span class="line"> <span class="keyword">int</span> countBoundaries = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i<cloud_cluster-><span class="built_in">size</span>(); i++) </span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">uint8_t</span> x = (boundaries.points[i].boundary_point);</span><br><span class="line"> <span class="keyword">int</span> a = <span class="keyword">static_cast</span><<span class="keyword">int</span>>(x); <span class="comment">//该函数的功能是强制类型转换</span></span><br><span class="line"> <span class="keyword">if</span> (a == <span class="number">1</span>)</span><br><span class="line"> {</span><br><span class="line"> (*boundPoints).<span class="built_in">push_back</span>(cloud_cluster->points[i]);</span><br><span class="line"> countBoundaries++;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> noBoundPoints.<span class="built_in">push_back</span>(cloud_cluster->points[i]);</span><br><span class="line"> }</span><br><span class="line"> std::cout << <span class="string">"boundary "</span><<j<<<span class="string">" size is："</span> << countBoundaries << std::endl;</span><br><span class="line"> <span class="comment">// std::stringstream ss2;</span></span><br><span class="line"> <span class="comment">// pcl::PCDWriter writer;</span></span><br><span class="line"> <span class="comment">// ss2 << "cloud_boundry"<<j<< ".pcd";</span></span><br><span class="line"> <span class="comment">// writer.write<pcl::PointXYZ> (ss2.str (), *boundPoints, false); //*</span></span><br><span class="line"> <span class="comment">//对边缘的四条边分别通过RANSAC进行直线拟合</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">double</span> list_dir[<span class="number">2</span>];</span><br><span class="line"> <span class="keyword">double</span> list_pre_dir[<span class="number">4</span>];</span><br><span class="line"> Eigen::Vector4f centroid;</span><br><span class="line"> pcl::<span class="built_in">compute3DCentroid</span>(*boundPoints, centroid); </span><br><span class="line"> <span class="comment">// std::cout << "点云 "<<j<<" 质心是（"</span></span><br><span class="line"> <span class="comment">// << 0.4-centroid[1] << ","</span></span><br><span class="line"> <span class="comment">// << -centroid[0] << ","</span></span><br><span class="line"> <span class="comment">// << centroid[2] << ")." << std::endl; </span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">//创建一个模型参数对象，用于记录结果</span></span><br><span class="line"> pcl::<span class="function">ModelCoefficients::Ptr <span class="title">coefficients</span><span class="params">(<span class="keyword">new</span> pcl::ModelCoefficients)</span></span>;</span><br><span class="line"> pcl::<span class="function">PointIndices::Ptr <span class="title">inliers</span><span class="params">(<span class="keyword">new</span> pcl::PointIndices)</span></span>; <span class="comment">//inliers表示误差能容忍的点 记录的是点云的序号</span></span><br><span class="line"> pcl::SACSegmentation<pcl::PointXYZ> seg2; <span class="comment">// 创建一个分割器</span></span><br><span class="line"> seg2.<span class="built_in">setOptimizeCoefficients</span>(<span class="literal">true</span>); <span class="comment">// Optional，这个设置可以选定结果平面展示的点是分割掉的点还是分割剩下的点。</span></span><br><span class="line"> seg2.<span class="built_in">setModelType</span>(pcl::SACMODEL_LINE); <span class="comment">// Mandatory-设置目标几何形状</span></span><br><span class="line"> seg2.<span class="built_in">setMethodType</span>(pcl::SAC_RANSAC); <span class="comment">//分割方法：随机采样法</span></span><br><span class="line"> seg2.<span class="built_in">setDistanceThreshold</span>(<span class="number">0.001</span>); <span class="comment">//设置误差容忍范围，也就是阈值</span></span><br><span class="line"></span><br><span class="line"> </span><br><span class="line"> </span><br><span class="line"> <span class="keyword">for</span>(<span class="keyword">int</span> i=<span class="number">0</span>;i<<span class="number">4</span>;i++)</span><br><span class="line"> { </span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">cloud_line</span><span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> pcl::PointCloud<pcl::PointXYZ>::<span class="function">Ptr <span class="title">cloud_out_line</span><span class="params">(<span class="keyword">new</span> pcl::PointCloud<pcl::PointXYZ>)</span></span>;</span><br><span class="line"> seg2.<span class="built_in">setInputCloud</span>(boundPoints); <span class="comment">//输入点云</span></span><br><span class="line"> seg2.<span class="built_in">segment</span>(*inliers, *coefficients); <span class="comment">//分割点云，获得平面和法向量</span></span><br><span class="line"> <span class="comment">//打印直线方程</span></span><br><span class="line"> <span class="comment">//std::cout << "d：" << coefficients->values[3] << endl;</span></span><br><span class="line"> <span class="comment">//std::cout << "e：" << coefficients->values[4] << endl;</span></span><br><span class="line"> list_pre_dir[i]=coefficients->values[<span class="number">4</span>]/coefficients->values[<span class="number">3</span>];</span><br><span class="line"> <span class="comment">//std::cout << list_pre_dir[i]<<endl;</span></span><br><span class="line"> pcl::ExtractIndices<pcl::PointXYZ> extract; <span class="comment">//创建点云提取对象</span></span><br><span class="line"> extract.<span class="built_in">setInputCloud</span> (boundPoints); <span class="comment">//设置输入点云</span></span><br><span class="line"> extract.<span class="built_in">setIndices</span> (inliers); <span class="comment">//设置分割后的内点为需要提取的点集</span></span><br><span class="line"> extract.<span class="built_in">setNegative</span> (<span class="literal">false</span>); <span class="comment">//设置提取内点而非外点</span></span><br><span class="line"> extract.<span class="built_in">filter</span> (*cloud_line); <span class="comment">//提取输出存储到cloud_line</span></span><br><span class="line"> std::cout << <span class="string">"PointCloud"</span><<i<< <span class="string">"representing the line component: "</span> << cloud_line->points.<span class="built_in">size</span> () << <span class="string">" data points."</span> << std::endl;</span><br><span class="line"> <span class="comment">// Remove the planar inliers, extract the rest</span></span><br><span class="line"> extract.<span class="built_in">setNegative</span> (<span class="literal">true</span>);</span><br><span class="line"> extract.<span class="built_in">filter</span> (*cloud_out_line);</span><br><span class="line"> *boundPoints = *cloud_out_line;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">BubbleSort</span>(list_pre_dir,<span class="number">4</span>);</span><br><span class="line"> <span class="comment">// std::cout << list_pre_dir[0]<<list_pre_dir[1]<<list_pre_dir[2]<<list_pre_dir[3]<<endl;</span></span><br><span class="line"> list_dir[<span class="number">0</span>]=(list_pre_dir[<span class="number">0</span>]+list_pre_dir[<span class="number">1</span>])/<span class="number">2</span>;</span><br><span class="line"> list_dir[<span class="number">1</span>]=(list_pre_dir[<span class="number">2</span>]+list_pre_dir[<span class="number">3</span>])/<span class="number">2</span>;</span><br><span class="line"> <span class="keyword">double</span> angle_y=<span class="number">1.57</span>-<span class="built_in">atan</span>(list_dir[<span class="number">1</span>]);</span><br><span class="line"> <span class="comment">//std::cout << "x,y方向的斜率：" << list_dir[0]<<","<<list_dir[1]<< endl;</span></span><br><span class="line"> std::cout << <span class="string">"Z轴旋转角为："</span> << angle_y<< endl;</span><br><span class="line"> vector<<span class="keyword">double</span>>out_element;</span><br><span class="line"> out_element.<span class="built_in">push_back</span>(<span class="built_in">ytopixel</span>(-centroid[<span class="number">0</span>]));</span><br><span class="line"> out_element.<span class="built_in">push_back</span>(<span class="built_in">xtopixel</span>(<span class="number">0.4</span>-centroid[<span class="number">1</span>]));</span><br><span class="line"> out_element.<span class="built_in">push_back</span>(angle_y);</span><br><span class="line"> out.<span class="built_in">push_back</span>(out_element);</span><br><span class="line"> cout <<out.<span class="built_in">size</span>()<<endl;</span><br><span class="line"> <span class="comment">// cout << "点云 "<<j<<" 质心是（"</span></span><br><span class="line"> <span class="comment">// << out[j][0] << ","</span></span><br><span class="line"> <span class="comment">// << out[j][1] <<")" << endl; </span></span><br><span class="line"> <span class="comment">// cout << "yolo "<<j<<" 质心是（"</span></span><br><span class="line"> <span class="comment">// << objects[j].center_x << ","</span></span><br><span class="line"> <span class="comment">// << objects[j].center_y <<")" << endl; </span></span><br><span class="line"> j++;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">yolomatchpcl</span>(objects,out);</span><br><span class="line"> <span class="keyword">for</span>(<span class="keyword">int</span> k=<span class="number">0</span>;k<out.<span class="built_in">size</span>();k++)</span><br><span class="line"> {</span><br><span class="line"> cout <<grasps[k].name<<<span class="string">" 质心是（"</span></span><br><span class="line"> << grasps[k].x << <span class="string">","</span></span><br><span class="line"> << grasps[k].y <<<span class="string">")"</span></span><br><span class="line"> <<<span class="string">"旋转角为"</span><<grasps[k].angle<<<span class="string">"."</span> << endl; </span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> out.<span class="built_in">clear</span>();</span><br><span class="line"></span><br><span class="line"> <span class="comment">//cout<<msg2->bounding_boxes[0].Class<<endl;</span></span><br><span class="line"> <span class="comment">//cout<<objects[0].obj<<objects[0].center_x<<objects[0].center_y<<endl;</span></span><br><span class="line"></span><br><span class="line"> cout<<<span class="string">"copy that"</span><<endl;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">(<span class="keyword">int</span> argc, <span class="keyword">char</span>** argv)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> ros::<span class="built_in">init</span>(argc, argv, <span class="string">"pclproc"</span>);</span><br><span class="line"></span><br><span class="line"> ros::NodeHandle nh;</span><br><span class="line"></span><br><span class="line"> <span class="function">message_filters::Subscriber<PointCloud2> <span class="title">pcl_sub</span><span class="params">(nh, <span class="string">"/camera/depth/points"</span>, <span class="number">1</span>)</span></span>;</span><br><span class="line"> <span class="function">message_filters::Subscriber<BoundingBoxes> <span class="title">yolo_sub</span><span class="params">(nh, <span class="string">"/darknet_ros/bounding_boxes"</span>, <span class="number">1</span>)</span></span>;</span><br><span class="line"> <span class="function">message_filters::Subscriber<ObjectCount> <span class="title">count_sub</span><span class="params">(nh, <span class="string">"/darknet_ros/found_object"</span>, <span class="number">1</span>)</span></span>;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">typedef</span> sync_policies::ApproximateTime<PointCloud2,</span><br><span class="line"> BoundingBoxes,ObjectCount></span><br><span class="line"> syncPolicy;</span><br><span class="line"> <span class="function">Synchronizer<syncPolicy> <span class="title">sync</span><span class="params">(syncPolicy(<span class="number">10</span>), pcl_sub, yolo_sub,count_sub)</span></span>;</span><br><span class="line"> sync.<span class="built_in">registerCallback</span>(boost::<span class="built_in">bind</span>(&callback, _1, _2,_3));</span><br><span class="line"></span><br><span class="line"> ros::<span class="built_in">spin</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"></span><br></pre></td></tr></table></figure><h2 id="效果演示"><a href="#效果演示" class="headerlink" title="效果演示"></a>效果演示</h2><p><img src="https://z3.ax1x.com/2021/06/26/RGwxJ0.png" alt=""></p><p>在构思好位置信息的发布形式后，食品分类和工件装配将同步进行，预期通过某种时序控制完成笛卡尔规划与双臂运动</p>]]></content>
<categories>
<category> 工业机器人 </category>
</categories>
<tags>
<tag> ROS </tag>
<tag> Gazebo </tag>
<tag> Moveit </tag>
<tag> Yolov3 </tag>
<tag> PCL </tag>
</tags>
</entry>
<entry>
<title>双臂协作机器人的食品分类抓取仿真（三）二维抓姿检测</title>
<link href="2021/06/14/%E5%8F%8C%E8%87%82%E5%8D%8F%E4%BD%9C%E6%9C%BA%E5%99%A8%E4%BA%BA%E7%9A%84%E9%A3%9F%E5%93%81%E5%88%86%E7%B1%BB%E6%8A%93%E5%8F%96%E4%BB%BF%E7%9C%9F%EF%BC%88%E4%B8%89%EF%BC%89%E4%BA%8C%E7%BB%B4%E6%8A%93%E5%A7%BF%E6%A3%80%E6%B5%8B/"/>
<url>2021/06/14/%E5%8F%8C%E8%87%82%E5%8D%8F%E4%BD%9C%E6%9C%BA%E5%99%A8%E4%BA%BA%E7%9A%84%E9%A3%9F%E5%93%81%E5%88%86%E7%B1%BB%E6%8A%93%E5%8F%96%E4%BB%BF%E7%9C%9F%EF%BC%88%E4%B8%89%EF%BC%89%E4%BA%8C%E7%BB%B4%E6%8A%93%E5%A7%BF%E6%A3%80%E6%B5%8B/</url>
<content type="html"><![CDATA[<h1 id="效果演示"><a href="#效果演示" class="headerlink" title="效果演示"></a>效果演示</h1><iframe height=498 width=510 src='https://player.youku.com/embed/XNTE2ODk3NTMyMA==' frameborder=0 'allowfullscreen'></iframe><h3 id="将yumi的功能模块和darknet-ros的检测模块整合到了一个工作空间，在原有食品检测分类的基础上实现了bounding-boxes的获取以及抓姿的检测和发布"><a href="#将yumi的功能模块和darknet-ros的检测模块整合到了一个工作空间，在原有食品检测分类的基础上实现了bounding-boxes的获取以及抓姿的检测和发布" class="headerlink" title="将yumi的功能模块和darknet_ros的检测模块整合到了一个工作空间，在原有食品检测分类的基础上实现了bounding boxes的获取以及抓姿的检测和发布"></a>将yumi的功能模块和darknet_ros的检测模块整合到了一个工作空间，在原有食品检测分类的基础上实现了bounding boxes的获取以及抓姿的检测和发布</h3><h2 id="历史遗留问题"><a href="#历史遗留问题" class="headerlink" title="历史遗留问题"></a>历史遗留问题</h2><p>darknet_ros_msgs包中自定义了三种消息类型，如果和yumi的相关功能包处在不同的工作空间下yumi将无法法订阅这些自定义消息，因此必须想办法将工作空间重新整合在一起</p><h3 id="1"><a href="#1" class="headerlink" title="1."></a>1.</h3><p>为确保配置不受编译后的文件影响，在~/catkin_ws/src下新建darknet_ros文件夹<br>先不要编译，重新将daknet_ros下的相关文件按上一篇博客配置好，原来的yolov3-tiny_5000.weights,ros.yaml,yolov3-tiny.yaml,yolov3-tiny.cfg,darknet_ros.launch，这些文件都要搬到当前的darknet_ros里面来</p><h3 id="2"><a href="#2" class="headerlink" title="2."></a>2.</h3><p>准备好这些后就可以开始编译了</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">cd ~/catkin_ws</span><br><span class="line">catkin_make -DCATKIN_WHITELIST_PACKAGES=“darknet;darknet_ros;darknet_ros_msgs” -DCMAKE_BUILD_TYPE=Release</span><br></pre></td></tr></table></figure><h3 id="3"><a href="#3" class="headerlink" title="3."></a>3.</h3><p>编译好后在其它功能包创建的时候就可以用到darknet_ros_msgs里面自定义消息啦</p><p>这里我们新建一个功能包用于抓姿检测</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">cd ~/catkin_ws/src</span><br><span class="line">catkin_create_pkg yoloproc darknet_ros_msgs roscpp rospy std_msgs </span><br><span class="line">cd ..</span><br><span class="line">catkin_make -DCATKIN_WHITELIST_PACKAGES='yoloproc'</span><br></pre></td></tr></table></figure><p>编译完成之后，python代码就可以直接 <code>from darknet_ros_msgs.msg import BoundingBoxes</code></p><h3 id="检查-catkin-ws工作空间的文件结构"><a href="#检查-catkin-ws工作空间的文件结构" class="headerlink" title="检查~/catkin_ws工作空间的文件结构"></a>检查~/catkin_ws工作空间的文件结构</h3><p>生成文件结构用到tree</p><p><img src="https://z3.ax1x.com/2021/06/14/2TNCZQ.png" alt=""></p><p><code>tree -L 2 > /home/li/tree.txt</code></p><p>生成结果保存到了tree.txt,内容如下</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br></pre></td><td class="code"><pre><span class="line">.</span><br><span class="line">├── CMakeLists.txt -> /opt/ros/kinetic/share/catkin/cmake/toplevel.cmake</span><br><span class="line">├── darknet_ros</span><br><span class="line">│ ├── darknet</span><br><span class="line">│ ├── darknet_ros</span><br><span class="line">│ ├── darknet_ros_msgs</span><br><span class="line">│ ├── jenkins-pipeline</span><br><span class="line">│ ├── LICENSE</span><br><span class="line">│ └── README.md</span><br><span class="line">├── gazebo_mimic</span><br><span class="line">│ ├── CHANGELOG.rst</span><br><span class="line">│ ├── CMakeLists.txt</span><br><span class="line">│ ├── include</span><br><span class="line">│ ├── package.xml</span><br><span class="line">│ ├── README.md</span><br><span class="line">│ └── src</span><br><span class="line">├── README</span><br><span class="line">├── yoloproc</span><br><span class="line">│ ├── CMakeLists.txt</span><br><span class="line">│ ├── include</span><br><span class="line">│ ├── package.xml</span><br><span class="line">│ └── src</span><br><span class="line">├── yumi_control</span><br><span class="line">│ ├── CHANGELOG.rst</span><br><span class="line">│ ├── CMakeLists.txt</span><br><span class="line">│ ├── config</span><br><span class="line">│ └── package.xml</span><br><span class="line">├── yumi_description</span><br><span class="line">│ ├── CHANGELOG.rst</span><br><span class="line">│ ├── CMakeLists.txt</span><br><span class="line">│ ├── meshes</span><br><span class="line">│ ├── package.xml</span><br><span class="line">│ ├── README.md</span><br><span class="line">│ └── urdf</span><br><span class="line">├── yumi_hw</span><br><span class="line">│ ├── CHANGELOG.rst</span><br><span class="line">│ ├── CMakeLists.txt</span><br><span class="line">│ ├── include</span><br><span class="line">│ ├── launch</span><br><span class="line">│ ├── package.xml</span><br><span class="line">│ ├── rapid</span><br><span class="line">│ ├── scripts</span><br><span class="line">│ ├── src</span><br><span class="line">│ └── srv</span><br><span class="line">├── yumi_launch</span><br><span class="line">│ ├── CHANGELOG.rst</span><br><span class="line">│ ├── CMakeLists.txt</span><br><span class="line">│ ├── launch</span><br><span class="line">│ ├── package.xml</span><br><span class="line">│ └── worlds</span><br><span class="line">├── yumi_moveit_config</span><br><span class="line">│ ├── CHANGELOG.rst</span><br><span class="line">│ ├── CMakeLists.txt</span><br><span class="line">│ ├── config</span><br><span class="line">│ ├── launch</span><br><span class="line">│ └── package.xml</span><br><span class="line">├── yumi_support</span><br><span class="line">│ ├── CHANGELOG.rst</span><br><span class="line">│ ├── CMakeLists.txt</span><br><span class="line">│ ├── config</span><br><span class="line">│ ├── launch</span><br><span class="line">│ ├── package.xml</span><br><span class="line">│ └── rapid</span><br><span class="line">└── yumi_test_controllers</span><br><span class="line"> ├── CMakeLists.txt</span><br><span class="line"> ├── include</span><br><span class="line"> ├── package.xml</span><br><span class="line"> ├── scripts</span><br><span class="line"> └── src</span><br><span class="line"></span><br><span class="line">36 directories, 32 files</span><br></pre></td></tr></table></figure><h2 id="基于RGB图像的平面抓姿检测"><a href="#基于RGB图像的平面抓姿检测" class="headerlink" title="基于RGB图像的平面抓姿检测"></a>基于RGB图像的平面抓姿检测</h2><p><img src="https://z3.ax1x.com/2021/06/14/2TNVzV.png" alt=""></p><p>主要思路根据yolov3-tiny的预测框对检测出的物体依次进行图像阈值分割，轮廓提取，利用图像矩求重心和主方向（这里也可以尝试PCA主成分分析），最后由重心和主方向计算抓姿</p><p>在windows PyCharm上编写好抓姿检测的代码后移植到了linux下的ROS中</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#!/usr/bin/env python</span></span><br><span class="line"><span class="comment"># -*- coding: utf-8 -*-</span></span><br><span class="line"><span class="keyword">import</span> rospy</span><br><span class="line"><span class="keyword">import</span> cv2</span><br><span class="line"><span class="keyword">import</span> math</span><br><span class="line"><span class="keyword">import</span> cv_bridge</span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> message_filters</span><br><span class="line"><span class="keyword">from</span> sensor_msgs.msg <span class="keyword">import</span> Image</span><br><span class="line"><span class="keyword">from</span> darknet_ros_msgs.msg <span class="keyword">import</span> BoundingBoxes</span><br><span class="line"><span class="keyword">from</span> darknet_ros_msgs.msg <span class="keyword">import</span> ObjectCount</span><br><span class="line"><span class="keyword">import</span> message_filters</span><br><span class="line"><span class="keyword">from</span> cv_bridge <span class="keyword">import</span> CvBridge, CvBridgeError</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">find_pose</span>(<span class="params">contour, flag</span>):</span></span><br><span class="line"> mu = cv2.moments(contour, flag)</span><br><span class="line"> x = mu[<span class="string">'m10'</span>] / mu[<span class="string">'m00'</span>]</span><br><span class="line"> y = mu[<span class="string">'m01'</span>] / mu[<span class="string">'m00'</span>]</span><br><span class="line"> a = mu[<span class="string">'m20'</span>] / mu[<span class="string">'m00'</span>] - x * x</span><br><span class="line"> b = mu[<span class="string">'m11'</span>] / mu[<span class="string">'m00'</span>] - x * y</span><br><span class="line"> c = mu[<span class="string">'m02'</span>] / mu[<span class="string">'m00'</span>] - y * y</span><br><span class="line"> Angle = math.atan2(<span class="number">2</span> * b, (a - c)) / <span class="number">2</span></span><br><span class="line"> <span class="keyword">return</span> x, y, Angle</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">draw_grasp_line</span>(<span class="params">image2, w, h, x, y, ang</span>):</span></span><br><span class="line"> <span class="comment"># 画线宽度为最小外接矩形宽度的1.3倍</span></span><br><span class="line"> h = <span class="number">1.3</span> * h</span><br><span class="line"> ver_ang = (<span class="number">90</span> - <span class="built_in">abs</span>(ang)) * math.pi / <span class="number">180</span></span><br><span class="line"> <span class="keyword">if</span> w < h:</span><br><span class="line"> w, h = h, w</span><br><span class="line"> sin_ver_ang = math.sin(ver_ang)</span><br><span class="line"> cos_ver_ang = math.cos(ver_ang)</span><br><span class="line"> <span class="comment"># print(sin_ver_ang*h / 2,sin_ver_ang*h / 2)</span></span><br><span class="line"> <span class="comment"># 确定垂直主轴的一条线段</span></span><br><span class="line"> <span class="keyword">if</span> ang < <span class="number">0</span>:</span><br><span class="line"> A_x = <span class="built_in">int</span>(x - cos_ver_ang * h / <span class="number">2</span>)</span><br><span class="line"> B_x = <span class="built_in">int</span>(x + cos_ver_ang * h / <span class="number">2</span>)</span><br><span class="line"> A_y = <span class="built_in">int</span>(y + sin_ver_ang * h / <span class="number">2</span>)</span><br><span class="line"> B_y = <span class="built_in">int</span>(y - sin_ver_ang * h / <span class="number">2</span>)</span><br><span class="line"> <span class="keyword">else</span>:</span><br><span class="line"> A_x = <span class="built_in">int</span>(x - cos_ver_ang * h / <span class="number">2</span>)</span><br><span class="line"> B_x = <span class="built_in">int</span>(x + cos_ver_ang * h / <span class="number">2</span>)</span><br><span class="line"> A_y = <span class="built_in">int</span>(y - sin_ver_ang * h / <span class="number">2</span>)</span><br><span class="line"> B_y = <span class="built_in">int</span>(y + sin_ver_ang * h / <span class="number">2</span>)</span><br><span class="line"> A = (A_x, A_y)</span><br><span class="line"> B = (B_x, B_y)</span><br><span class="line"> <span class="comment"># print(x,y)</span></span><br><span class="line"> <span class="comment"># print(A,B)</span></span><br><span class="line"> cv2.line(image2, A, B, (<span class="number">0</span>, <span class="number">0</span>, <span class="number">255</span>), <span class="number">5</span>, <span class="number">8</span>)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">img_proc</span>(<span class="params">image, position_list</span>):</span></span><br><span class="line"> img_proc_list = []</span><br><span class="line"> <span class="comment"># print(img_proc_list)</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(position_list)):</span><br><span class="line"> img = image[position_list[i][<span class="number">2</span>]:position_list[i][<span class="number">4</span>],position_list[i][<span class="number">1</span>]: position_list[i][<span class="number">3</span>]]</span><br><span class="line"> <span class="comment"># cv2.imshow('cropped',img)剪裁分块处理</span></span><br><span class="line"> img2 = cv2.GaussianBlur(img, (<span class="number">3</span>, <span class="number">3</span>), <span class="number">0</span>)</span><br><span class="line"> <span class="comment"># cv2.imshow('blurred', img2)高斯模糊滤波</span></span><br><span class="line"> img3 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)</span><br><span class="line"> retval, img4 = cv2.threshold(img3, <span class="number">232</span>, <span class="number">255</span>, cv2.THRESH_BINARY_INV)</span><br><span class="line"> <span class="comment"># cv2.imshow('gray and binary', img4)灰度化二值化</span></span><br><span class="line"> <span class="comment"># img5 = cv2.Canny(img4, 50, 150)Canny边缘检测</span></span><br><span class="line"> <span class="comment">#检测轮廓</span></span><br><span class="line"> binary,contours, hierarchy = cv2.findContours(img4, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)</span><br><span class="line"> n = <span class="built_in">len</span>(contours) <span class="comment"># 轮廓的个数</span></span><br><span class="line"> contours_area = []</span><br><span class="line"> <span class="comment"># 找到最大的轮廓</span></span><br><span class="line"> <span class="keyword">for</span> j <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(contours)):</span><br><span class="line"> contours_area.append(cv2.contourArea(contours[j]))</span><br><span class="line"> max_id = np.argmax(np.array(contours_area))</span><br><span class="line"> <span class="comment"># print(max_id)</span></span><br><span class="line"> <span class="comment"># 画轮廓</span></span><br><span class="line"> cv2.drawContours(img, contours, max_id, (<span class="number">0</span>, <span class="number">255</span>, <span class="number">0</span>), <span class="number">3</span>)</span><br><span class="line"> <span class="comment"># 图像矩找重心和主方向</span></span><br><span class="line"> center_x, center_y, angle = find_pose(contours[max_id], <span class="literal">False</span>)</span><br><span class="line"> <span class="comment"># 数据类型转换</span></span><br><span class="line"> inverse_angle2 = -<span class="built_in">round</span>(angle * <span class="number">180</span> / math.pi, <span class="number">1</span>)</span><br><span class="line"> int_center_x = <span class="built_in">int</span>(center_x)</span><br><span class="line"> int_center_y = <span class="built_in">int</span>(center_y)</span><br><span class="line"> <span class="comment"># 画重心</span></span><br><span class="line"> cv2.circle(img, (int_center_x, int_center_y), <span class="number">5</span>, (<span class="number">0</span>, <span class="number">0</span>, <span class="number">255</span>), -<span class="number">1</span>)</span><br><span class="line"> <span class="comment"># print(int_center_x, int_center_y, inverse_angle2)</span></span><br><span class="line"> <span class="comment"># cv2.imshow('contours', img)</span></span><br><span class="line"> <span class="comment"># 4.获取最小外接矩形并打印</span></span><br><span class="line"> min_rect = cv2.minAreaRect(contours[max_id])</span><br><span class="line"> min_rect_height = <span class="built_in">int</span>(min_rect[<span class="number">1</span>][<span class="number">1</span>])</span><br><span class="line"> min_rect_width = <span class="built_in">int</span>(min_rect[<span class="number">1</span>][<span class="number">0</span>])</span><br><span class="line"> <span class="comment"># rect_points = cv2.boxPoints(min_rect)</span></span><br><span class="line"> <span class="comment"># rect_points = np.int0(rect_points)</span></span><br><span class="line"> <span class="comment"># # print(min_rect_width)</span></span><br><span class="line"> <span class="comment"># cv2.drawContours(img, [rect_points], 0, (0, 255, 0), 3)</span></span><br><span class="line"> <span class="comment"># 画出抓取线</span></span><br><span class="line"> draw_grasp_line(img, min_rect_width, min_rect_height, int_center_x, int_center_y, inverse_angle2)</span><br><span class="line"> <span class="comment"># 图像处理好后把抓取中心坐标加上左上角的x,y坐标得出实际的坐标</span></span><br><span class="line"> int_center_x = int_center_x + <span class="built_in">int</span>(position_list[i][<span class="number">1</span>])</span><br><span class="line"> int_center_y = int_center_y + <span class="built_in">int</span>(position_list[i][<span class="number">2</span>])</span><br><span class="line"> <span class="comment"># 存放类别和抓取位姿信息</span></span><br><span class="line"> img_proc_list.append([position_list[i][<span class="number">0</span>], int_center_x, int_center_y, inverse_angle2])</span><br><span class="line"> <span class="comment">#print(img_proc_list[i][0])</span></span><br><span class="line"> <span class="comment">#print(str(int_center_x))</span></span><br><span class="line"> <span class="comment">#print(img_proc_list)</span></span><br><span class="line"> <span class="comment">#cv2.imshow('gasp line', image)</span></span><br><span class="line"> <span class="comment">#cv2.waitKey(3000)</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> img_proc_list,image</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">multi_callback</span>(<span class="params">subscriber_bounding, subscriber_image</span>):</span></span><br><span class="line"> detected_list=[]</span><br><span class="line"> bridge= cv_bridge.CvBridge()</span><br><span class="line"> cv_image = bridge.imgmsg_to_cv2(subscriber_image, <span class="string">"bgr8"</span>)</span><br><span class="line"> frame = np.array(cv_image, dtype=np.uint8)</span><br><span class="line"> <span class="comment"># cv2.imshow('Food', frame)</span></span><br><span class="line"> <span class="comment"># cv2.waitKey(2000)</span></span><br><span class="line"> <span class="comment"># cv2.destroyAllWindows()</span></span><br><span class="line"> <span class="comment">#rospy.loginfo("数目是："+str(len(subscriber_bounding.bounding_boxes)))</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(subscriber_bounding.bounding_boxes)):</span><br><span class="line"> detected_list.append([subscriber_bounding.bounding_boxes[i].Class,subscriber_bounding.bounding_boxes[i].xmin,</span><br><span class="line">subscriber_bounding.bounding_boxes[i].ymin,subscriber_bounding.bounding_boxes[i].xmax,subscriber_bounding.bounding_boxes[i].ymax])</span><br><span class="line"> <span class="comment">#rospy.loginfo(subscriber_bounding.bounding_boxes[1].Class)</span></span><br><span class="line"> detected_result,detected_image=img_proc(cv_image, detected_list)</span><br><span class="line"> ros_image=bridge.cv2_to_imgmsg(detected_image, <span class="string">"bgr8"</span>)</span><br><span class="line"> image_pub.publish(ros_image)</span><br><span class="line"> rospy.loginfo(<span class="string">"抓姿已生成！！!"</span>)</span><br><span class="line"> rospy.loginfo(detected_result)</span><br><span class="line"> <span class="comment">#position_list.clear()</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> </span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line"> <span class="keyword">try</span>:</span><br><span class="line"> </span><br><span class="line"> rospy.init_node(<span class="string">"darknet_msgs_proc"</span>)</span><br><span class="line"> rospy.loginfo(<span class="string">"开始测试消息订阅...."</span>)</span><br><span class="line"> <span class="comment"># GraspDetect()</span></span><br><span class="line"> image_pub=rospy.Publisher(<span class="string">'/processed_image'</span>,Image,queue_size = <span class="number">1</span>) <span class="comment">#定义话题</span></span><br><span class="line"> subscriber_bounding = message_filters.Subscriber(<span class="string">'/darknet_ros/bounding_boxes'</span>, BoundingBoxes, queue_size=<span class="number">1</span>)</span><br><span class="line"> subscriber_image = message_filters.Subscriber(<span class="string">'/camera/color/image_raw'</span>, Image,queue_size=<span class="number">1</span>)</span><br><span class="line"></span><br><span class="line"> </span><br><span class="line"> sync = message_filters.ApproximateTimeSynchronizer([subscriber_bounding, subscriber_image],<span class="number">10</span>,<span class="number">1</span>)</span><br><span class="line"> </span><br><span class="line"> sync.registerCallback(multi_callback)</span><br><span class="line"> rospy.spin()</span><br><span class="line"> <span class="keyword">except</span> KeyboardInterrupt:</span><br><span class="line"> <span class="built_in">print</span>(<span class="string">"KeyboardInterrupt"</span>)</span><br><span class="line"> cv2.destroyAllWindows()</span><br><span class="line"></span><br></pre></td></tr></table></figure><p>代码注释很详细，这里不做过多解释，需要注意的就是订阅的<code>'/darknet_ros/bounding_boxes'</code>和<code>/camera/color/image_raw'</code>两个话题进行了时间戳同步，防止不同时间的相机图像与yolov3-tiny的检测结果混叠在一起，最后的检测结果包括图像和抓姿也都发布了出来</p><h3 id="关于-darknet-ros-bounding-boxes"><a href="#关于-darknet-ros-bounding-boxes" class="headerlink" title="关于/darknet_ros/bounding_boxes"></a>关于/darknet_ros/bounding_boxes</h3><p>/darknet_ros/bounding_boxes话题的消息类型是BoundingBoxes</p><p><img src="https://z3.ax1x.com/2021/06/14/2TNPaj.png" alt=""></p><p>可以看出BoundingBoxes里面有一个数组，数组的元素是BoundingBox,格式如下</p><p><img src="https://z3.ax1x.com/2021/06/14/2TNpqg.png" alt=""></p><p>于是就可以知道如果使用里面的数据，示例：data.bounding_boxes[i].Class就代表第i个预测框里面的物体种类</p><h3 id="遇到的问题"><a href="#遇到的问题" class="headerlink" title="遇到的问题"></a>遇到的问题</h3><p>callback里面如果用cv2.imshow()，由于callback执行速度快且一直在循环执行，因此会造成图像显示窗口卡死，即使加上cv2.waitKey()也不太好使，于是想到一种办法将抓姿检测的结果以图片形式发布出去然后就可以在rviz里面或者通过rqt_image_view 显示发布的图片，这里我使用到了 <code>rqt_image_view /processed_image</code></p><h3 id="结果"><a href="#结果" class="headerlink" title="结果"></a>结果</h3><p><img src="https://z3.ax1x.com/2021/06/14/2TNAGq.png" alt=""></p><p>左上角为抓姿的示意图，左下角为检测与分类结果，右下角为实时输出的抓取角度与像素坐标</p><h3 id="相机坐标系抓姿到机器人坐标系的转换"><a href="#相机坐标系抓姿到机器人坐标系的转换" class="headerlink" title="相机坐标系抓姿到机器人坐标系的转换"></a>相机坐标系抓姿到机器人坐标系的转换</h3><p>相机标定与手眼标定，敬请期待</p>]]></content>
<categories>
<category> 工业机器人 </category>
</categories>
<tags>
<tag> ROS </tag>
<tag> Gazebo </tag>
<tag> Moveit </tag>
<tag> Yolov3 </tag>
<tag> opencv-python </tag>
</tags>
</entry>
<entry>
<title>Ubuntu 16.04安装ROS Kinetic</title>
<link href="2021/06/11/Ubuntu-16.04%E5%AE%89%E8%A3%85ROS-Kinetic/"/>
<url>2021/06/11/Ubuntu-16.04%E5%AE%89%E8%A3%85ROS-Kinetic/</url>
<content type="html"><![CDATA[<h2 id="起因"><a href="#起因" class="headerlink" title="起因"></a>起因</h2><p> remove qt5时误删ROS，心态很崩，忍痛卸载重装ROS</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">sudo apt-get purge ros-*</span><br><span class="line">sudo rm -rf /etc/ros</span><br></pre></td></tr></table></figure><p>这时已成功卸载ROS</p><h2 id="安装ROS-Kinetic"><a href="#安装ROS-Kinetic" class="headerlink" title="安装ROS Kinetic"></a>安装ROS Kinetic</h2><p>1.设置sources.list</p><p><code>sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'</code></p><p>2.设置key（公钥已更新）</p><p><code>sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654</code></p><p>3.更新package</p><p><code>sudo apt-get update</code></p><p>4.安装ROS kinetic完整版</p><p><code>sudo apt-get install ros-kinetic-desktop-full</code></p><p>5.初始化rosdep</p><p><code>sudo rosdep init</code></p><p><code>rosdep update</code></p><p>6.配置ROS环境</p><p><code>echo "source /opt/ros/kinetic/setup.bash" >> ~/.bashrc</code></p><p><code>source ~/.bashrc</code></p><p>7.安装依赖项</p><p><code>sudo apt-get install python-rosinstall</code></p><p><code>python-rosinstall-generator python-wstool build-essential</code></p><p>8.测试ROS是否安装成功</p><p><code>roscore</code></p><h2 id="关于rosdep-update超时"><a href="#关于rosdep-update超时" class="headerlink" title="关于rosdep update超时"></a>关于rosdep update超时</h2><p> 由于近期国内Github Raw的可用IP越来越少，通过修改hosts文件解决rosdep update超时问题的方法已经不太好用，本文通过修改rosdep源码中下载资源的函数来解决这一问题。<br>网站<a href="https://ghproxy.com/支持github的资源代理，非常好用，我们将用此代理加速rosdep对Github">https://ghproxy.com/支持github的资源代理，非常好用，我们将用此代理加速rosdep对Github</a> Raw的访问，进而解决rosdep update超时问题。</p><p>1./usr/lib/python2.7/dist-packages/rosdep2/sources_list.py里面的download_rosdep_data函数第一行添加url=”<a href="https://ghproxy.com/"+url">https://ghproxy.com/"+url</a><br>2./usr/lib/python2.7/dist-packages/rosdistro/<strong>init</strong>.py里面的DEFAULT_INDEX_URL…..用DEFAULT_INDEX_URL = ‘<a href="https://ghproxy.com/https://raw.githubusercontent.com/ros/rosdistro/master/index-v4.yaml'代替">https://ghproxy.com/https://raw.githubusercontent.com/ros/rosdistro/master/index-v4.yaml'代替</a><br>3.<br>/usr/lib/python2.7/dist-packages/rosdep2/gbpdistro_support.py 36行(这一个有一点小问题，因为里面的URL貌似用\拼接起来了，不太清楚作用，但是影响不大，待验证)<br>/usr/lib/python2.7/dist-packages/rosdep2/sources_list.py 72行<br>/usr/lib/python2.7/dist-packages/rosdep2/rep3.py 39行<br>/usr/lib/python2.7/dist-packages/rosdistro/manifest_provider/github.py 68行 119行<br>这些都可以通过在地址前添加<a href="https://ghproxy.com/前缀来应用代理服务。">https://ghproxy.com/前缀来应用代理服务。</a><br>重新rosdep就好了</p><h2 id="安装Gazebo-ROS-Control-Moveit等组件"><a href="#安装Gazebo-ROS-Control-Moveit等组件" class="headerlink" title="安装Gazebo,ROS Control,Moveit等组件"></a>安装Gazebo,ROS Control,Moveit等组件</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">sudo apt-get install ros-kinetic-moveit-full</span><br><span class="line">sudo apt-get install ros-kinetic-control*</span><br><span class="line">sudo apt-get install ros-kinetic-gazebo-ros*</span><br><span class="line">sudo apt-get install ros-kinetic-joint-trajectory*</span><br><span class="line">sudo apt-get install ros-kinetic-gazebo7-ros-pkgs ros-kinetic-gazebo7-ros-control</span><br></pre></td></tr></table></figure><p>完成到这里，Moveit和Gazebo联合仿真所需的依赖基本都满足啦！！！</p>]]></content>
<categories>
<category> 环境 </category>
</categories>
<tags>
<tag> ROS Kinetic </tag>
</tags>
</entry>
<entry>
<title>Ubuntu16.04安装cuda9.2,cudnn 7.6.3</title>
<link href="2021/06/11/Ubuntu16.04%E5%AE%89%E8%A3%85cuda9.2,cudnn-7.6.3/"/>
<url>2021/06/11/Ubuntu16.04%E5%AE%89%E8%A3%85cuda9.2,cudnn-7.6.3/</url>
<content type="html"><![CDATA[<h2 id="安装Cuda-9-2"><a href="#安装Cuda-9-2" class="headerlink" title="安装Cuda 9.2"></a>安装Cuda 9.2</h2><p>由于驱动版本430，根据cuda官网要求可知下载9.2比较合适</p><p><a href="https://developer.nvidia.com/cuda-toolkit-archive">https://developer.nvidia.com/cuda-toolkit-archive</a></p><p>下载cuda_9.2.148_396.37_linux.run</p><p>放到Home目录下</p><p><code>sudo bash cuda_9.2.148_396.37_linux.run</code></p><p><img src="https://z3.ax1x.com/2021/06/11/2W9s8s.png" alt=""></p><p><code>sudo source ~/.bashrc</code></p><p><strong>添加到环境变量</strong></p><p>export PATH=/usr/local/cuda-9.2/bin/:$PATH;<br>export LD_LIBRARY_PATH=/usr/local/cuda-9.2/lib64/:$LD_LIBRARY_PATH;</p><p><strong>除了不安装驱动，其它均选择y</strong></p><p>之后通过 <code>nvcc -V</code>查看cuda是否成功安装</p><p><img src="https://z3.ax1x.com/2021/06/11/2W90Ug.png" alt=""></p><p><strong>这里cuda 9.2已经成功安装！！！！！！</strong></p><h2 id="安装Cudnn7-6-3"><a href="#安装Cudnn7-6-3" class="headerlink" title="安装Cudnn7.6.3"></a>安装Cudnn7.6.3</h2><p>与Cuda9.2匹配的Cudnn库有7.6.3，所以这里到<a href="https://developer.nvidia.com/rdp/cudnn-archive">https://developer.nvidia.com/rdp/cudnn-archive</a></p><p>下载cudnn-9.2-linux-x64-v7.6.3.30.tgz</p><p>在Home目录解压缩之后</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">sudo cp cuda/include/cudnn.h /usr/<span class="built_in">local</span>/cuda/include/</span><br><span class="line">sudo cp cuda/lib64/libcudnn* /usr/<span class="built_in">local</span>/cuda/lib64/</span><br></pre></td></tr></table></figure><p>检查Cudnn库是否成功安装</p><p><img src="https://z3.ax1x.com/2021/06/11/2W9B5Q.png" alt=""></p><p><strong>这里已经成功安装Cudnn7.6.3!!!!!!</strong></p>]]></content>
<categories>
<category> 环境 </category>
</categories>
<tags>
<tag> Cuda </tag>
</tags>
</entry>
<entry>
<title>各种工具的快捷键（持续更新）</title>
<link href="2021/06/11/%E5%90%84%E7%A7%8D%E5%B7%A5%E5%85%B7%E7%9A%84%E5%BF%AB%E6%8D%B7%E9%94%AE%EF%BC%88%E6%8C%81%E7%BB%AD%E6%9B%B4%E6%96%B0%EF%BC%89/"/>
<url>2021/06/11/%E5%90%84%E7%A7%8D%E5%B7%A5%E5%85%B7%E7%9A%84%E5%BF%AB%E6%8D%B7%E9%94%AE%EF%BC%88%E6%8C%81%E7%BB%AD%E6%9B%B4%E6%96%B0%EF%BC%89/</url>
<content type="html"><![CDATA[<h2 id="Pycharm"><a href="#Pycharm" class="headerlink" title="Pycharm"></a>Pycharm</h2><p>打开工程后Add Configuuration为灰色，Ctrl+Shift+F10</p><p>自动调整格式：Ctrl+Alt+L</p><h2 id="labelImage"><a href="#labelImage" class="headerlink" title="labelImage"></a>labelImage</h2><p>Ctrl + s 保存<br>Ctrl + d Copy the current label and rect box<br>Space 标记当前图片已标记 |<br>w 创建一个矩形<br>d 下一张图片<br>a 上一张图片 </p><h2 id="SublimeText3"><a href="#SublimeText3" class="headerlink" title="SublimeText3"></a>SublimeText3</h2><p>注释：Ctrl+/</p><p>取消注释： Ctrl+Shift+L</p><p>缩进：Ctrl+]</p><p>前进：Ctrl+[</p>]]></content>
<categories>
<category> 环境 </category>
</categories>
<tags>
<tag> Pycharm </tag>
</tags>
</entry>
<entry>
<title>双臂协作机器人的食品分类抓取仿真（二）yolov3-tiny食品检测与分类</title>
<link href="2021/06/11/%E5%8F%8C%E8%87%82%E5%8D%8F%E4%BD%9C%E6%9C%BA%E5%99%A8%E4%BA%BA%E7%9A%84%E9%A3%9F%E5%93%81%E5%88%86%E7%B1%BB%E6%8A%93%E5%8F%96%E4%BB%BF%E7%9C%9F%EF%BC%88%E4%BA%8C%EF%BC%89yolov3-tiny%E9%A3%9F%E5%93%81%E6%A3%80%E6%B5%8B%E4%B8%8E%E5%88%86%E7%B1%BB/"/>
<url>2021/06/11/%E5%8F%8C%E8%87%82%E5%8D%8F%E4%BD%9C%E6%9C%BA%E5%99%A8%E4%BA%BA%E7%9A%84%E9%A3%9F%E5%93%81%E5%88%86%E7%B1%BB%E6%8A%93%E5%8F%96%E4%BB%BF%E7%9C%9F%EF%BC%88%E4%BA%8C%EF%BC%89yolov3-tiny%E9%A3%9F%E5%93%81%E6%A3%80%E6%B5%8B%E4%B8%8E%E5%88%86%E7%B1%BB/</url>
<content type="html"><![CDATA[<h2 id="Darknet-ROS下的食品分类"><a href="#Darknet-ROS下的食品分类" class="headerlink" title="Darknet ROS下的食品分类"></a>Darknet ROS下的食品分类</h2><h3 id="初期成果："><a href="#初期成果：" class="headerlink" title="初期成果："></a>初期成果：</h3><iframe height=498 width=510 src='https://player.youku.com/embed/XNTE2NzczMDc2NA==' frameborder=0 'allowfullscreen'></iframe><h3 id="配置Darknet-ROS"><a href="#配置Darknet-ROS" class="headerlink" title="配置Darknet_ROS"></a>配置Darknet_ROS</h3><h5 id="1-创建工作空间并下载darknet-ros"><a href="#1-创建工作空间并下载darknet-ros" class="headerlink" title="1.创建工作空间并下载darknet_ros"></a>1.创建工作空间并下载darknet_ros</h5><p>根据Darknet ROS项目文档的介绍：</p><p>This is a ROS package developed for <strong>object detection in camera images</strong>. You only look once (YOLO) is a state-of-the-art, real-time object detection system. In the following ROS package you are able to use <strong>YOLO (V3) on GPU and CPU</strong>. The pre-trained model of the convolutional neural network is able to detect pre-trained classes including the data set from VOC and COCO, or you can also create a network with your own detection objects. For more information about YOLO, Darknet, available training data and training YOLO see the following link: <a href="http://pjreddie.com/darknet/yolo/">YOLO: Real-Time Object Detection</a>.</p><p>The YOLO packages have been tested under <strong>ROS Noetic</strong> and <strong>Ubuntu 20.04</strong>. Note: We also provide branches that work under <strong>ROS Melodic</strong>, <strong>ROS Foxy</strong> and <strong>ROS2</strong>.This is research code, expect that it changes often and any fitness for a particular purpose is disclaimed.</p><p><img src="https://z3.ax1x.com/2021/06/11/2WCPMt.png" alt=""></p><p> 了解到这是一个为ROS下的利用darknet进行目标检测的功能包，特点是可以自定义用于目标检测的订阅话题，同时通过三个话题发布classes和bounding box等信息，可以实现实时的视频流的目标检测和分类；而这个项目一直在更新，为了保证下述操作可以复现，可以从百度云盘：<a href="https://pan.baidu.com/s/1_ddm3U_KKWDvGCtpL2ZGrg（提取码：1234）下载，需要注意的是直接下载的darknet_ros下是darknet文件夹是空的，可以从百度网盘：https://pan.baidu.com/s/1J-xWuHGqBUNoPvL2qpLfjQ（提取码：1234）下载darknet，***将darknet_ros下的darknet替换***。">https://pan.baidu.com/s/1_ddm3U_KKWDvGCtpL2ZGrg（提取码：1234）下载，需要注意的是直接下载的darknet_ros下是darknet文件夹是空的，可以从百度网盘：https://pan.baidu.com/s/1J-xWuHGqBUNoPvL2qpLfjQ（提取码：1234）下载darknet，***将darknet_ros下的darknet替换***。</a></p><p><strong>将完整的darknet_ros下放到yolo_detect工作空间的src目录下</strong></p><p><strong><em>在前面的操作完成后</em></strong></p><h5 id="2-下载预训练权重"><a href="#2-下载预训练权重" class="headerlink" title="2.下载预训练权重"></a>2.下载预训练权重</h5><p>访问<br>pjreddie.com/media/files/yolov2.weights<br>pjreddie.com/media/files/yolov2-tiny.weights<br>pjreddie.com/media/files/yolov3.weights</p><p>下载预训练权重放到yolo_detect/src/darknet_ros/darknet_ros/yolo_network_config/weights文件夹下</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">cd ~/yolo_detect</span><br><span class="line">catkin_make -DCMAKE_BUILD_TYPE=Release</span><br></pre></td></tr></table></figure><h5 id="这里除了有关具体训练配置和检测配置文件还没修改，已经完成了darknet-ros的配置！！！"><a href="#这里除了有关具体训练配置和检测配置文件还没修改，已经完成了darknet-ros的配置！！！" class="headerlink" title="这里除了有关具体训练配置和检测配置文件还没修改，已经完成了darknet_ros的配置！！！"></a><em>这里除了有关具体训练配置和检测配置文件还没修改，已经完成了darknet_ros的配置！！！</em></h5><h3 id="制作数据集"><a href="#制作数据集" class="headerlink" title="制作数据集"></a>制作数据集</h3><h5 id="1-从-camera-color-image-raw采集数据集"><a href="#1-从-camera-color-image-raw采集数据集" class="headerlink" title="1.从/camera/color/image_raw采集数据集"></a>1.从/camera/color/image_raw采集数据集</h5><p>Kinetc相机节点启动后，</p><p><code>rqt_image_view /camera/color/image_raw</code></p><p><img src="https://z3.ax1x.com/2021/06/11/2W9fVU.png" alt=""></p><p>这样就可以一张张存，缺点就是命名麻烦，这样最后有100张png</p><p><strong>这里曾经尝试写一个节点订阅/camera/color/image_raw，键盘输入enter就可以存，后面发现每张图片都一样，仔细分析后和订阅器发布器对消息的处理机制有关系，要实现订阅的为实时的消息，publisher和subscriber的队列大小应该都为1。</strong></p><h5 id="2-数据标注和增广"><a href="#2-数据标注和增广" class="headerlink" title="2.数据标注和增广"></a>2.数据标注和增广</h5><p> 在windows上面进行，标注使用labelImg,增广使用的是Github上面的项目（需要修改，不然会出现xmin>xmaxh或者ymin>ymx的问题）</p><p><strong><em>标注比较简单，香蕉、芬达、可乐和雪碧标签分别为banan,fenda, cola, spirit</em></strong></p><p><strong><em>具体过程不在这里叙述，最后得到的应该是与那100张图片相对应的xml文件</em></strong></p><p>具体介绍数据的增广：</p><p><strong><em>将原始数据集图片进行剪裁、平移、改变亮度、加噪声、旋转、镜像等变换扩充数据集，同时将其对应的xml标签进行转换</em></strong></p><p><strong>第一步：新建DataAugmentforLabelImg.py</strong></p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br><span class="line">224</span><br><span class="line">225</span><br><span class="line">226</span><br><span class="line">227</span><br><span class="line">228</span><br><span class="line">229</span><br><span class="line">230</span><br><span class="line">231</span><br><span class="line">232</span><br><span class="line">233</span><br><span class="line">234</span><br><span class="line">235</span><br><span class="line">236</span><br><span class="line">237</span><br><span class="line">238</span><br><span class="line">239</span><br><span class="line">240</span><br><span class="line">241</span><br><span class="line">242</span><br><span class="line">243</span><br><span class="line">244</span><br><span class="line">245</span><br><span class="line">246</span><br><span class="line">247</span><br><span class="line">248</span><br><span class="line">249</span><br><span class="line">250</span><br><span class="line">251</span><br><span class="line">252</span><br><span class="line">253</span><br><span class="line">254</span><br><span class="line">255</span><br><span class="line">256</span><br><span class="line">257</span><br><span class="line">258</span><br><span class="line">259</span><br><span class="line">260</span><br><span class="line">261</span><br><span class="line">262</span><br><span class="line">263</span><br><span class="line">264</span><br><span class="line">265</span><br><span class="line">266</span><br><span class="line">267</span><br><span class="line">268</span><br><span class="line">269</span><br><span class="line">270</span><br><span class="line">271</span><br><span class="line">272</span><br><span class="line">273</span><br><span class="line">274</span><br><span class="line">275</span><br><span class="line">276</span><br><span class="line">277</span><br><span class="line">278</span><br><span class="line">279</span><br><span class="line">280</span><br><span class="line">281</span><br><span class="line">282</span><br><span class="line">283</span><br><span class="line">284</span><br><span class="line">285</span><br><span class="line">286</span><br><span class="line">287</span><br><span class="line">288</span><br><span class="line">289</span><br><span class="line">290</span><br><span class="line">291</span><br><span class="line">292</span><br><span class="line">293</span><br><span class="line">294</span><br><span class="line">295</span><br><span class="line">296</span><br><span class="line">297</span><br><span class="line">298</span><br><span class="line">299</span><br><span class="line">300</span><br><span class="line">301</span><br><span class="line">302</span><br><span class="line">303</span><br><span class="line">304</span><br><span class="line">305</span><br><span class="line">306</span><br><span class="line">307</span><br><span class="line">308</span><br><span class="line">309</span><br><span class="line">310</span><br><span class="line">311</span><br><span class="line">312</span><br><span class="line">313</span><br><span class="line">314</span><br><span class="line">315</span><br><span class="line">316</span><br><span class="line">317</span><br><span class="line">318</span><br><span class="line">319</span><br><span class="line">320</span><br><span class="line">321</span><br><span class="line">322</span><br><span class="line">323</span><br><span class="line">324</span><br><span class="line">325</span><br><span class="line">326</span><br><span class="line">327</span><br><span class="line">328</span><br><span class="line">329</span><br><span class="line">330</span><br><span class="line">331</span><br><span class="line">332</span><br><span class="line">333</span><br><span class="line">334</span><br><span class="line">335</span><br><span class="line">336</span><br><span class="line">337</span><br><span class="line">338</span><br><span class="line">339</span><br><span class="line">340</span><br><span class="line">341</span><br><span class="line">342</span><br><span class="line">343</span><br><span class="line">344</span><br><span class="line">345</span><br><span class="line">346</span><br><span class="line">347</span><br><span class="line">348</span><br><span class="line">349</span><br><span class="line">350</span><br><span class="line">351</span><br><span class="line">352</span><br><span class="line">353</span><br><span class="line">354</span><br><span class="line">355</span><br><span class="line">356</span><br><span class="line">357</span><br><span class="line">358</span><br><span class="line">359</span><br><span class="line">360</span><br><span class="line">361</span><br><span class="line">362</span><br><span class="line">363</span><br><span class="line">364</span><br><span class="line">365</span><br><span class="line">366</span><br><span class="line">367</span><br><span class="line">368</span><br><span class="line">369</span><br><span class="line">370</span><br><span class="line">371</span><br><span class="line">372</span><br><span class="line">373</span><br><span class="line">374</span><br><span class="line">375</span><br><span class="line">376</span><br><span class="line">377</span><br><span class="line">378</span><br><span class="line">379</span><br><span class="line">380</span><br><span class="line">381</span><br><span class="line">382</span><br><span class="line">383</span><br><span class="line">384</span><br><span class="line">385</span><br><span class="line">386</span><br><span class="line">387</span><br><span class="line">388</span><br><span class="line">389</span><br><span class="line">390</span><br><span class="line">391</span><br><span class="line">392</span><br><span class="line">393</span><br><span class="line">394</span><br><span class="line">395</span><br><span class="line">396</span><br><span class="line">397</span><br><span class="line">398</span><br><span class="line">399</span><br><span class="line">400</span><br><span class="line">401</span><br><span class="line">402</span><br><span class="line">403</span><br><span class="line">404</span><br><span class="line">405</span><br><span class="line">406</span><br><span class="line">407</span><br><span class="line">408</span><br><span class="line">409</span><br><span class="line">410</span><br><span class="line">411</span><br><span class="line">412</span><br><span class="line">413</span><br><span class="line">414</span><br><span class="line">415</span><br><span class="line">416</span><br><span class="line">417</span><br><span class="line">418</span><br><span class="line">419</span><br><span class="line">420</span><br><span class="line">421</span><br><span class="line">422</span><br><span class="line">423</span><br><span class="line">424</span><br><span class="line">425</span><br><span class="line">426</span><br><span class="line">427</span><br><span class="line">428</span><br><span class="line">429</span><br><span class="line">430</span><br><span class="line">431</span><br><span class="line">432</span><br><span class="line">433</span><br><span class="line">434</span><br><span class="line">435</span><br><span class="line">436</span><br><span class="line">437</span><br><span class="line">438</span><br><span class="line">439</span><br><span class="line">440</span><br><span class="line">441</span><br><span class="line">442</span><br><span class="line">443</span><br><span class="line">444</span><br><span class="line">445</span><br><span class="line">446</span><br><span class="line">447</span><br><span class="line">448</span><br><span class="line">449</span><br><span class="line">450</span><br><span class="line">451</span><br><span class="line">452</span><br><span class="line">453</span><br><span class="line">454</span><br><span class="line">455</span><br><span class="line">456</span><br><span class="line">457</span><br><span class="line">458</span><br><span class="line">459</span><br><span class="line">460</span><br><span class="line">461</span><br><span class="line">462</span><br><span class="line">463</span><br><span class="line">464</span><br><span class="line">465</span><br><span class="line">466</span><br><span class="line">467</span><br><span class="line">468</span><br><span class="line">469</span><br><span class="line">470</span><br><span class="line">471</span><br><span class="line">472</span><br><span class="line">473</span><br><span class="line">474</span><br><span class="line">475</span><br><span class="line">476</span><br><span class="line">477</span><br><span class="line">478</span><br><span class="line">479</span><br><span class="line">480</span><br><span class="line">481</span><br><span class="line">482</span><br><span class="line">483</span><br><span class="line">484</span><br><span class="line">485</span><br><span class="line">486</span><br><span class="line">487</span><br><span class="line">488</span><br><span class="line">489</span><br><span class="line">490</span><br><span class="line">491</span><br><span class="line">492</span><br><span class="line">493</span><br><span class="line">494</span><br><span class="line">495</span><br><span class="line">496</span><br><span class="line">497</span><br><span class="line">498</span><br><span class="line">499</span><br><span class="line">500</span><br><span class="line">501</span><br><span class="line">502</span><br><span class="line">503</span><br><span class="line">504</span><br><span class="line">505</span><br><span class="line">506</span><br><span class="line">507</span><br><span class="line">508</span><br><span class="line">509</span><br><span class="line">510</span><br><span class="line">511</span><br><span class="line">512</span><br><span class="line">513</span><br><span class="line">514</span><br><span class="line">515</span><br><span class="line">516</span><br><span class="line">517</span><br><span class="line">518</span><br><span class="line">519</span><br><span class="line">520</span><br><span class="line">521</span><br><span class="line">522</span><br><span class="line">523</span><br><span class="line">524</span><br><span class="line">525</span><br><span class="line">526</span><br><span class="line">527</span><br><span class="line">528</span><br><span class="line">529</span><br><span class="line">530</span><br><span class="line">531</span><br><span class="line">532</span><br><span class="line">533</span><br><span class="line">534</span><br><span class="line">535</span><br><span class="line">536</span><br><span class="line">537</span><br><span class="line">538</span><br><span class="line">539</span><br><span class="line">540</span><br><span class="line">541</span><br><span class="line">542</span><br><span class="line">543</span><br><span class="line">544</span><br><span class="line">545</span><br><span class="line">546</span><br><span class="line">547</span><br><span class="line">548</span><br><span class="line">549</span><br><span class="line">550</span><br><span class="line">551</span><br><span class="line">552</span><br><span class="line">553</span><br><span class="line">554</span><br><span class="line">555</span><br><span class="line">556</span><br><span class="line">557</span><br><span class="line">558</span><br><span class="line">559</span><br><span class="line">560</span><br><span class="line">561</span><br><span class="line">562</span><br><span class="line">563</span><br><span class="line">564</span><br><span class="line">565</span><br><span class="line">566</span><br><span class="line">567</span><br><span class="line">568</span><br><span class="line">569</span><br><span class="line">570</span><br><span class="line">571</span><br><span class="line">572</span><br><span class="line">573</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># -*- coding=utf-8 -*-</span></span><br><span class="line"><span class="comment">##############################################################</span></span><br><span class="line"><span class="comment"># description:</span></span><br><span class="line"><span class="comment"># data augmentation for obeject detection</span></span><br><span class="line"><span class="comment"># author:</span></span><br><span class="line"><span class="comment"># pureyang 2019-08-26</span></span><br><span class="line"><span class="comment"># 参考：https://github.com/maozezhong/CV_ToolBox/blob/master/DataAugForObjectDetection</span></span><br><span class="line"></span><br><span class="line"><span class="comment">##############################################################</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 包括:</span></span><br><span class="line"><span class="comment"># 1. 裁剪(需改变bbox)</span></span><br><span class="line"><span class="comment"># 2. 平移(需改变bbox)</span></span><br><span class="line"><span class="comment"># 3. 改变亮度</span></span><br><span class="line"><span class="comment"># 4. 加噪声</span></span><br><span class="line"><span class="comment"># 5. 旋转角度(需要改变bbox)</span></span><br><span class="line"><span class="comment"># 6. 镜像(需要改变bbox)</span></span><br><span class="line"><span class="comment"># 7. cutout</span></span><br><span class="line"><span class="comment"># 注意:</span></span><br><span class="line"><span class="comment"># random.seed(),相同的seed,产生的随机数是一样的!!</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> time</span><br><span class="line"><span class="keyword">import</span> random</span><br><span class="line"><span class="keyword">import</span> copy</span><br><span class="line"><span class="keyword">import</span> cv2</span><br><span class="line"><span class="keyword">import</span> os</span><br><span class="line"><span class="keyword">import</span> math</span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">from</span> skimage.util <span class="keyword">import</span> random_noise</span><br><span class="line"><span class="keyword">from</span> lxml <span class="keyword">import</span> etree, objectify</span><br><span class="line"><span class="keyword">import</span> xml.etree.ElementTree <span class="keyword">as</span> ETl</span><br><span class="line"><span class="keyword">import</span> argparse</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment"># 显示图片</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">show_pic</span>(<span class="params">img, bboxes=<span class="literal">None</span></span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 输入:</span></span><br><span class="line"><span class="string"> img:图像array</span></span><br><span class="line"><span class="string"> bboxes:图像的所有boudning box list, 格式为[[x_min, y_min, x_max, y_max]....]</span></span><br><span class="line"><span class="string"> names:每个box对应的名称</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(bboxes)):</span><br><span class="line"> bbox = bboxes[i]</span><br><span class="line"> x_min = bbox[<span class="number">0</span>]</span><br><span class="line"> y_min = bbox[<span class="number">1</span>]</span><br><span class="line"> x_max = bbox[<span class="number">2</span>]</span><br><span class="line"> y_max = bbox[<span class="number">3</span>]</span><br><span class="line"> cv2.rectangle(img, (<span class="built_in">int</span>(x_min), <span class="built_in">int</span>(y_min)), (<span class="built_in">int</span>(x_max), <span class="built_in">int</span>(y_max)), (<span class="number">0</span>, <span class="number">255</span>, <span class="number">0</span>), <span class="number">3</span>)</span><br><span class="line"> cv2.namedWindow(<span class="string">'pic'</span>, <span class="number">0</span>) <span class="comment"># 1表示原图</span></span><br><span class="line"> cv2.moveWindow(<span class="string">'pic'</span>, <span class="number">0</span>, <span class="number">0</span>)</span><br><span class="line"> cv2.resizeWindow(<span class="string">'pic'</span>, <span class="number">1200</span>, <span class="number">800</span>) <span class="comment"># 可视化的图片大小</span></span><br><span class="line"> cv2.imshow(<span class="string">'pic'</span>, img)</span><br><span class="line"> cv2.waitKey(<span class="number">0</span>)</span><br><span class="line"> cv2.destroyAllWindows()</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment"># 图像均为cv2读取</span></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">DataAugmentForObjectDetection</span>():</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">__init__</span>(<span class="params">self, rotation_rate=<span class="number">0.5</span>, max_rotation_angle=<span class="number">5</span>,</span></span></span><br><span class="line"><span class="params"><span class="function"> crop_rate=<span class="number">0.5</span>, shift_rate=<span class="number">0.5</span>, change_light_rate=<span class="number">0.5</span>,</span></span></span><br><span class="line"><span class="params"><span class="function"> add_noise_rate=<span class="number">0.5</span>, flip_rate=<span class="number">0.5</span>,</span></span></span><br><span class="line"><span class="params"><span class="function"> cutout_rate=<span class="number">0.5</span>, cut_out_length=<span class="number">50</span>, cut_out_holes=<span class="number">1</span>, cut_out_threshold=<span class="number">0.5</span>,</span></span></span><br><span class="line"><span class="params"><span class="function"> is_addNoise=<span class="literal">True</span>, is_changeLight=<span class="literal">True</span>, is_cutout=<span class="literal">True</span>, is_rotate_img_bbox=<span class="literal">True</span>,</span></span></span><br><span class="line"><span class="params"><span class="function"> is_crop_img_bboxes=<span class="literal">True</span>, is_shift_pic_bboxes=<span class="literal">True</span>, is_filp_pic_bboxes=<span class="literal">True</span></span>):</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 配置各个操作的属性</span></span><br><span class="line"> self.rotation_rate = rotation_rate</span><br><span class="line"> self.max_rotation_angle = max_rotation_angle</span><br><span class="line"> self.crop_rate = crop_rate</span><br><span class="line"> self.shift_rate = shift_rate</span><br><span class="line"> self.change_light_rate = change_light_rate</span><br><span class="line"> self.add_noise_rate = add_noise_rate</span><br><span class="line"> self.flip_rate = flip_rate</span><br><span class="line"> self.cutout_rate = cutout_rate</span><br><span class="line"></span><br><span class="line"> self.cut_out_length = cut_out_length</span><br><span class="line"> self.cut_out_holes = cut_out_holes</span><br><span class="line"> self.cut_out_threshold = cut_out_threshold</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 是否使用某种增强方式</span></span><br><span class="line"> self.is_addNoise = is_addNoise</span><br><span class="line"> self.is_changeLight = is_changeLight</span><br><span class="line"> self.is_cutout = is_cutout</span><br><span class="line"> self.is_rotate_img_bbox = is_rotate_img_bbox</span><br><span class="line"> self.is_crop_img_bboxes = is_crop_img_bboxes</span><br><span class="line"> self.is_shift_pic_bboxes = is_shift_pic_bboxes</span><br><span class="line"> self.is_filp_pic_bboxes = is_filp_pic_bboxes</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 加噪声</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">_addNoise</span>(<span class="params">self, img</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 输入:</span></span><br><span class="line"><span class="string"> img:图像array</span></span><br><span class="line"><span class="string"> 输出:</span></span><br><span class="line"><span class="string"> 加噪声后的图像array,由于输出的像素是在[0,1]之间,所以得乘以255</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> <span class="comment"># return cv2.GaussianBlur(img, (11, 11), 0)</span></span><br><span class="line"> <span class="keyword">return</span> random_noise(img, mode=<span class="string">'gaussian'</span>, seed=<span class="built_in">int</span>(time.time()), clip=<span class="literal">True</span>) * <span class="number">255</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 调整亮度</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">_changeLight</span>(<span class="params">self, img</span>):</span></span><br><span class="line"> alpha = random.uniform(<span class="number">0.35</span>, <span class="number">1</span>)</span><br><span class="line"> blank = np.zeros(img.shape, img.dtype)</span><br><span class="line"> <span class="keyword">return</span> cv2.addWeighted(img, alpha, blank, <span class="number">1</span> - alpha, <span class="number">0</span>)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># cutout</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">_cutout</span>(<span class="params">self, img, bboxes, length=<span class="number">100</span>, n_holes=<span class="number">1</span>, threshold=<span class="number">0.5</span></span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 原版本：https://github.com/uoguelph-mlrg/Cutout/blob/master/util/cutout.py</span></span><br><span class="line"><span class="string"> Randomly mask out one or more patches from an image.</span></span><br><span class="line"><span class="string"> Args:</span></span><br><span class="line"><span class="string"> img : a 3D numpy array,(h,w,c)</span></span><br><span class="line"><span class="string"> bboxes : 框的坐标</span></span><br><span class="line"><span class="string"> n_holes (int): Number of patches to cut out of each image.</span></span><br><span class="line"><span class="string"> length (int): The length (in pixels) of each square patch.</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">cal_iou</span>(<span class="params">boxA, boxB</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> boxA, boxB为两个框，返回iou</span></span><br><span class="line"><span class="string"> boxB为bouding box</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> <span class="comment"># determine the (x, y)-coordinates of the intersection rectangle</span></span><br><span class="line"> xA = <span class="built_in">max</span>(boxA[<span class="number">0</span>], boxB[<span class="number">0</span>])</span><br><span class="line"> yA = <span class="built_in">max</span>(boxA[<span class="number">1</span>], boxB[<span class="number">1</span>])</span><br><span class="line"> xB = <span class="built_in">min</span>(boxA[<span class="number">2</span>], boxB[<span class="number">2</span>])</span><br><span class="line"> yB = <span class="built_in">min</span>(boxA[<span class="number">3</span>], boxB[<span class="number">3</span>])</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> xB <= xA <span class="keyword">or</span> yB <= yA:</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0.0</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># compute the area of intersection rectangle</span></span><br><span class="line"> interArea = (xB - xA + <span class="number">1</span>) * (yB - yA + <span class="number">1</span>)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># compute the area of both the prediction and ground-truth</span></span><br><span class="line"> <span class="comment"># rectangles</span></span><br><span class="line"> boxAArea = (boxA[<span class="number">2</span>] - boxA[<span class="number">0</span>] + <span class="number">1</span>) * (boxA[<span class="number">3</span>] - boxA[<span class="number">1</span>] + <span class="number">1</span>)</span><br><span class="line"> boxBArea = (boxB[<span class="number">2</span>] - boxB[<span class="number">0</span>] + <span class="number">1</span>) * (boxB[<span class="number">3</span>] - boxB[<span class="number">1</span>] + <span class="number">1</span>)</span><br><span class="line"> iou = interArea / <span class="built_in">float</span>(boxBArea)</span><br><span class="line"> <span class="keyword">return</span> iou</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 得到h和w</span></span><br><span class="line"> <span class="keyword">if</span> img.ndim == <span class="number">3</span>:</span><br><span class="line"> h, w, c = img.shape</span><br><span class="line"> <span class="keyword">else</span>:</span><br><span class="line"> _, h, w, c = img.shape</span><br><span class="line"> mask = np.ones((h, w, c), np.float32)</span><br><span class="line"> <span class="keyword">for</span> n <span class="keyword">in</span> <span class="built_in">range</span>(n_holes):</span><br><span class="line"> chongdie = <span class="literal">True</span> <span class="comment"># 看切割的区域是否与box重叠太多</span></span><br><span class="line"> <span class="keyword">while</span> chongdie:</span><br><span class="line"> y = np.random.randint(h)</span><br><span class="line"> x = np.random.randint(w)</span><br><span class="line"></span><br><span class="line"> y1 = np.clip(y - length // <span class="number">2</span>, <span class="number">0</span>,</span><br><span class="line"> h) <span class="comment"># numpy.clip(a, a_min, a_max, out=None), clip这个函数将将数组中的元素限制在a_min, a_max之间，大于a_max的就使得它等于 a_max，小于a_min,的就使得它等于a_min</span></span><br><span class="line"> y2 = np.clip(y + length // <span class="number">2</span>, <span class="number">0</span>, h)</span><br><span class="line"> x1 = np.clip(x - length // <span class="number">2</span>, <span class="number">0</span>, w)</span><br><span class="line"> x2 = np.clip(x + length // <span class="number">2</span>, <span class="number">0</span>, w)</span><br><span class="line"></span><br><span class="line"> chongdie = <span class="literal">False</span></span><br><span class="line"> <span class="keyword">for</span> box <span class="keyword">in</span> bboxes:</span><br><span class="line"> <span class="keyword">if</span> cal_iou([x1, y1, x2, y2], box) > threshold:</span><br><span class="line"> chongdie = <span class="literal">True</span></span><br><span class="line"> <span class="keyword">break</span></span><br><span class="line"> mask[y1: y2, x1: x2, :] = <span class="number">0.</span></span><br><span class="line"> img = img * mask</span><br><span class="line"> <span class="keyword">return</span> img</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 旋转</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">_rotate_img_bbox</span>(<span class="params">self, img, bboxes, angle=<span class="number">5</span>, scale=<span class="number">1.</span></span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 参考:https://blog.csdn.net/u014540717/article/details/53301195crop_rate</span></span><br><span class="line"><span class="string"> 输入:</span></span><br><span class="line"><span class="string"> img:图像array,(h,w,c)</span></span><br><span class="line"><span class="string"> bboxes:该图像包含的所有boundingboxs,一个list,每个元素为[x_min, y_min, x_max, y_max],要确保是数值</span></span><br><span class="line"><span class="string"> angle:旋转角度</span></span><br><span class="line"><span class="string"> scale:默认1</span></span><br><span class="line"><span class="string"> 输出:</span></span><br><span class="line"><span class="string"> rot_img:旋转后的图像array</span></span><br><span class="line"><span class="string"> rot_bboxes:旋转后的boundingbox坐标list</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> <span class="comment"># ---------------------- 旋转图像 ----------------------</span></span><br><span class="line"> w = img.shape[<span class="number">1</span>]</span><br><span class="line"> h = img.shape[<span class="number">0</span>]</span><br><span class="line"> <span class="comment"># 角度变弧度</span></span><br><span class="line"> rangle = np.deg2rad(angle) <span class="comment"># angle in radians</span></span><br><span class="line"> <span class="comment"># now calculate new image width and height</span></span><br><span class="line"> nw = (<span class="built_in">abs</span>(np.sin(rangle) * h) + <span class="built_in">abs</span>(np.cos(rangle) * w)) * scale</span><br><span class="line"> nh = (<span class="built_in">abs</span>(np.cos(rangle) * h) + <span class="built_in">abs</span>(np.sin(rangle) * w)) * scale</span><br><span class="line"> <span class="comment"># ask OpenCV for the rotation matrix</span></span><br><span class="line"> rot_mat = cv2.getRotationMatrix2D((nw * <span class="number">0.5</span>, nh * <span class="number">0.5</span>), angle, scale)</span><br><span class="line"> <span class="comment"># calculate the move from the old center to the new center combined</span></span><br><span class="line"> <span class="comment"># with the rotation</span></span><br><span class="line"> rot_move = np.dot(rot_mat, np.array([(nw - w) * <span class="number">0.5</span>, (nh - h) * <span class="number">0.5</span>, <span class="number">0</span>]))</span><br><span class="line"> <span class="comment"># the move only affects the translation, so update the translation</span></span><br><span class="line"> rot_mat[<span class="number">0</span>, <span class="number">2</span>] += rot_move[<span class="number">0</span>]</span><br><span class="line"> rot_mat[<span class="number">1</span>, <span class="number">2</span>] += rot_move[<span class="number">1</span>]</span><br><span class="line"> <span class="comment"># 仿射变换</span></span><br><span class="line"> rot_img = cv2.warpAffine(img, rot_mat, (<span class="built_in">int</span>(math.ceil(nw)), <span class="built_in">int</span>(math.ceil(nh))), flags=cv2.INTER_LANCZOS4)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># ---------------------- 矫正bbox坐标 ----------------------</span></span><br><span class="line"> <span class="comment"># rot_mat是最终的旋转矩阵</span></span><br><span class="line"> <span class="comment"># 获取原始bbox的四个中点，然后将这四个点转换到旋转后的坐标系下</span></span><br><span class="line"> rot_bboxes = <span class="built_in">list</span>()</span><br><span class="line"> <span class="keyword">for</span> bbox <span class="keyword">in</span> bboxes:</span><br><span class="line"> xmin = bbox[<span class="number">0</span>]</span><br><span class="line"> ymin = bbox[<span class="number">1</span>]</span><br><span class="line"> xmax = bbox[<span class="number">2</span>]</span><br><span class="line"> ymax = bbox[<span class="number">3</span>]</span><br><span class="line"> point1 = np.dot(rot_mat, np.array([(xmin + xmax) / <span class="number">2</span>, ymin, <span class="number">1</span>]))</span><br><span class="line"> point2 = np.dot(rot_mat, np.array([xmax, (ymin + ymax) / <span class="number">2</span>, <span class="number">1</span>]))</span><br><span class="line"> point3 = np.dot(rot_mat, np.array([(xmin + xmax) / <span class="number">2</span>, ymax, <span class="number">1</span>]))</span><br><span class="line"> point4 = np.dot(rot_mat, np.array([xmin, (ymin + ymax) / <span class="number">2</span>, <span class="number">1</span>]))</span><br><span class="line"> <span class="comment"># 合并np.array</span></span><br><span class="line"> concat = np.vstack((point1, point2, point3, point4))</span><br><span class="line"> <span class="comment"># 改变array类型</span></span><br><span class="line"> concat = concat.astype(np.int32)</span><br><span class="line"> <span class="comment"># 得到旋转后的坐标</span></span><br><span class="line"> rx, ry, rw, rh = cv2.boundingRect(concat)</span><br><span class="line"> rx_min = rx</span><br><span class="line"> ry_min = ry</span><br><span class="line"> rx_max = rx + rw</span><br><span class="line"> ry_max = ry + rh</span><br><span class="line"> <span class="comment"># 加入list中</span></span><br><span class="line"> rot_bboxes.append([rx_min, ry_min, rx_max, ry_max])</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> rot_img, rot_bboxes</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 裁剪</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">_crop_img_bboxes</span>(<span class="params">self, img, bboxes</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 裁剪后的图片要包含所有的框</span></span><br><span class="line"><span class="string"> 输入:</span></span><br><span class="line"><span class="string"> img:图像array</span></span><br><span class="line"><span class="string"> bboxes:该图像包含的所有boundingboxs,一个list,每个元素为[x_min, y_min, x_max, y_max],要确保是数值</span></span><br><span class="line"><span class="string"> 输出:</span></span><br><span class="line"><span class="string"> crop_img:裁剪后的图像array</span></span><br><span class="line"><span class="string"> crop_bboxes:裁剪后的bounding box的坐标list</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> <span class="comment"># ---------------------- 裁剪图像 ----------------------</span></span><br><span class="line"> w = img.shape[<span class="number">1</span>]</span><br><span class="line"> h = img.shape[<span class="number">0</span>]</span><br><span class="line"> x_min = w <span class="comment"># 裁剪后的包含所有目标框的最小的框</span></span><br><span class="line"> x_max = <span class="number">0</span></span><br><span class="line"> y_min = h</span><br><span class="line"> y_max = <span class="number">0</span></span><br><span class="line"> <span class="keyword">for</span> bbox <span class="keyword">in</span> bboxes:</span><br><span class="line"> x_min = <span class="built_in">min</span>(x_min, bbox[<span class="number">0</span>])</span><br><span class="line"> y_min = <span class="built_in">min</span>(y_min, bbox[<span class="number">1</span>])</span><br><span class="line"> x_max = <span class="built_in">max</span>(x_max, bbox[<span class="number">2</span>])</span><br><span class="line"> y_max = <span class="built_in">max</span>(y_max, bbox[<span class="number">3</span>])</span><br><span class="line"></span><br><span class="line"> d_to_left = x_min <span class="comment"># 包含所有目标框的最小框到左边的距离</span></span><br><span class="line"> d_to_right = w - x_max <span class="comment"># 包含所有目标框的最小框到右边的距离</span></span><br><span class="line"> d_to_top = y_min <span class="comment"># 包含所有目标框的最小框到顶端的距离</span></span><br><span class="line"> d_to_bottom = h - y_max <span class="comment"># 包含所有目标框的最小框到底部的距离</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 随机扩展这个最小框</span></span><br><span class="line"> crop_x_min = <span class="built_in">int</span>(x_min - random.uniform(<span class="number">0</span>, d_to_left))</span><br><span class="line"> crop_y_min = <span class="built_in">int</span>(y_min - random.uniform(<span class="number">0</span>, d_to_top))</span><br><span class="line"> crop_x_max = <span class="built_in">int</span>(x_max + random.uniform(<span class="number">0</span>, d_to_right))</span><br><span class="line"> crop_y_max = <span class="built_in">int</span>(y_max + random.uniform(<span class="number">0</span>, d_to_bottom))</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 随机扩展这个最小框 , 防止别裁的太小</span></span><br><span class="line"> <span class="comment"># crop_x_min = int(x_min - random.uniform(d_to_left//2, d_to_left))</span></span><br><span class="line"> <span class="comment"># crop_y_min = int(y_min - random.uniform(d_to_top//2, d_to_top))</span></span><br><span class="line"> <span class="comment"># crop_x_max = int(x_max + random.uniform(d_to_right//2, d_to_right))</span></span><br><span class="line"> <span class="comment"># crop_y_max = int(y_max + random.uniform(d_to_bottom//2, d_to_bottom))</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 确保不要越界</span></span><br><span class="line"> crop_x_min = <span class="built_in">max</span>(<span class="number">0</span>, crop_x_min)</span><br><span class="line"> crop_y_min = <span class="built_in">max</span>(<span class="number">0</span>, crop_y_min)</span><br><span class="line"> crop_x_max = <span class="built_in">min</span>(w, crop_x_max)</span><br><span class="line"> crop_y_max = <span class="built_in">min</span>(h, crop_y_max)</span><br><span class="line"></span><br><span class="line"> crop_img = img[crop_y_min:crop_y_max, crop_x_min:crop_x_max]</span><br><span class="line"></span><br><span class="line"> <span class="comment"># ---------------------- 裁剪boundingbox ----------------------</span></span><br><span class="line"> <span class="comment"># 裁剪后的boundingbox坐标计算</span></span><br><span class="line"> crop_bboxes = <span class="built_in">list</span>()</span><br><span class="line"> <span class="keyword">for</span> bbox <span class="keyword">in</span> bboxes:</span><br><span class="line"> crop_bboxes.append([bbox[<span class="number">0</span>] - crop_x_min, bbox[<span class="number">1</span>] - crop_y_min, bbox[<span class="number">2</span>] - crop_x_min, bbox[<span class="number">3</span>] - crop_y_min])</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> crop_img, crop_bboxes</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 平移</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">_shift_pic_bboxes</span>(<span class="params">self, img, bboxes</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 参考:https://blog.csdn.net/sty945/article/details/79387054</span></span><br><span class="line"><span class="string"> 平移后的图片要包含所有的框</span></span><br><span class="line"><span class="string"> 输入:</span></span><br><span class="line"><span class="string"> img:图像array</span></span><br><span class="line"><span class="string"> bboxes:该图像包含的所有boundingboxs,一个list,每个元素为[x_min, y_min, x_max, y_max],要确保是数值</span></span><br><span class="line"><span class="string"> 输出:</span></span><br><span class="line"><span class="string"> shift_img:平移后的图像array</span></span><br><span class="line"><span class="string"> shift_bboxes:平移后的bounding box的坐标list</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> <span class="comment"># ---------------------- 平移图像 ----------------------</span></span><br><span class="line"> w = img.shape[<span class="number">1</span>]</span><br><span class="line"> h = img.shape[<span class="number">0</span>]</span><br><span class="line"> x_min = w <span class="comment"># 裁剪后的包含所有目标框的最小的框</span></span><br><span class="line"> x_max = <span class="number">0</span></span><br><span class="line"> y_min = h</span><br><span class="line"> y_max = <span class="number">0</span></span><br><span class="line"> <span class="keyword">for</span> bbox <span class="keyword">in</span> bboxes:</span><br><span class="line"> x_min = <span class="built_in">min</span>(x_min, bbox[<span class="number">0</span>])</span><br><span class="line"> y_min = <span class="built_in">min</span>(y_min, bbox[<span class="number">1</span>])</span><br><span class="line"> x_max = <span class="built_in">max</span>(x_max, bbox[<span class="number">2</span>])</span><br><span class="line"> y_max = <span class="built_in">max</span>(y_max, bbox[<span class="number">3</span>])</span><br><span class="line"></span><br><span class="line"> d_to_left = x_min <span class="comment"># 包含所有目标框的最大左移动距离</span></span><br><span class="line"> d_to_right = w - x_max <span class="comment"># 包含所有目标框的最大右移动距离</span></span><br><span class="line"> d_to_top = y_min <span class="comment"># 包含所有目标框的最大上移动距离</span></span><br><span class="line"> d_to_bottom = h - y_max <span class="comment"># 包含所有目标框的最大下移动距离</span></span><br><span class="line"></span><br><span class="line"> x = random.uniform(-(d_to_left - <span class="number">1</span>) / <span class="number">3</span>, (d_to_right - <span class="number">1</span>) / <span class="number">3</span>)</span><br><span class="line"> y = random.uniform(-(d_to_top - <span class="number">1</span>) / <span class="number">3</span>, (d_to_bottom - <span class="number">1</span>) / <span class="number">3</span>)</span><br><span class="line"></span><br><span class="line"> M = np.float32([[<span class="number">1</span>, <span class="number">0</span>, x], [<span class="number">0</span>, <span class="number">1</span>, y]]) <span class="comment"># x为向左或右移动的像素值,正为向右负为向左; y为向上或者向下移动的像素值,正为向下负为向上</span></span><br><span class="line"> shift_img = cv2.warpAffine(img, M, (img.shape[<span class="number">1</span>], img.shape[<span class="number">0</span>]))</span><br><span class="line"></span><br><span class="line"> <span class="comment"># ---------------------- 平移boundingbox ----------------------</span></span><br><span class="line"> shift_bboxes = <span class="built_in">list</span>()</span><br><span class="line"> <span class="keyword">for</span> bbox <span class="keyword">in</span> bboxes:</span><br><span class="line"> shift_bboxes.append([bbox[<span class="number">0</span>] + x, bbox[<span class="number">1</span>] + y, bbox[<span class="number">2</span>] + x, bbox[<span class="number">3</span>] + y])</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> shift_img, shift_bboxes</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 镜像</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">_filp_pic_bboxes</span>(<span class="params">self, img, bboxes</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 参考:https://blog.csdn.net/jningwei/article/details/78753607</span></span><br><span class="line"><span class="string"> 平移后的图片要包含所有的框</span></span><br><span class="line"><span class="string"> 输入:</span></span><br><span class="line"><span class="string"> img:图像array</span></span><br><span class="line"><span class="string"> bboxes:该图像包含的所有boundingboxs,一个list,每个元素为[x_min, y_min, x_max, y_max],要确保是数值</span></span><br><span class="line"><span class="string"> 输出:</span></span><br><span class="line"><span class="string"> flip_img:平移后的图像array</span></span><br><span class="line"><span class="string"> flip_bboxes:平移后的bounding box的坐标list</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> <span class="comment"># ---------------------- 翻转图像 ----------------------</span></span><br><span class="line"></span><br><span class="line"> flip_img = copy.deepcopy(img)</span><br><span class="line"> h, w, _ = img.shape</span><br><span class="line"></span><br><span class="line"> sed = random.random()</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> <span class="number">0</span> < sed < <span class="number">0.33</span>: <span class="comment"># 0.33的概率水平翻转，0.33的概率垂直翻转,0.33是对角反转</span></span><br><span class="line"> flip_img = cv2.flip(flip_img, <span class="number">0</span>) <span class="comment"># _flip_x</span></span><br><span class="line"> inver = <span class="number">0</span></span><br><span class="line"> <span class="keyword">elif</span> <span class="number">0.33</span> < sed < <span class="number">0.66</span>:</span><br><span class="line"> flip_img = cv2.flip(flip_img, <span class="number">1</span>) <span class="comment"># _flip_y</span></span><br><span class="line"> inver = <span class="number">1</span></span><br><span class="line"> <span class="keyword">else</span>:</span><br><span class="line"> flip_img = cv2.flip(flip_img, -<span class="number">1</span>) <span class="comment"># flip_x_y</span></span><br><span class="line"> inver = -<span class="number">1</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># ---------------------- 调整boundingbox ----------------------</span></span><br><span class="line"> flip_bboxes = <span class="built_in">list</span>()</span><br><span class="line"> <span class="keyword">for</span> box <span class="keyword">in</span> bboxes:</span><br><span class="line"> x_min = box[<span class="number">0</span>]</span><br><span class="line"> y_min = box[<span class="number">1</span>]</span><br><span class="line"> x_max = box[<span class="number">2</span>]</span><br><span class="line"> y_max = box[<span class="number">3</span>]</span><br><span class="line"> <span class="keyword">if</span> inver == <span class="number">0</span>:</span><br><span class="line"> flip_bboxes.append([x_max, h - y_min, x_min, h - y_max])</span><br><span class="line"> <span class="keyword">elif</span> inver == <span class="number">1</span>:</span><br><span class="line"> flip_bboxes.append([w - x_max, y_min, w - x_min, y_max])</span><br><span class="line"> <span class="keyword">elif</span> inver == -<span class="number">1</span>:</span><br><span class="line"> flip_bboxes.append([w - x_min, h - y_max, w - x_max, h - y_min])</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> flip_img, flip_bboxes</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 图像增强方法</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">dataAugment</span>(<span class="params">self, img, bboxes</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 图像增强</span></span><br><span class="line"><span class="string"> 输入:</span></span><br><span class="line"><span class="string"> img:图像array</span></span><br><span class="line"><span class="string"> bboxes:该图像的所有框坐标</span></span><br><span class="line"><span class="string"> 输出:</span></span><br><span class="line"><span class="string"> img:增强后的图像</span></span><br><span class="line"><span class="string"> bboxes:增强后图片对应的box</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> change_num = <span class="number">0</span> <span class="comment"># 改变的次数</span></span><br><span class="line"> <span class="comment"># print('------')</span></span><br><span class="line"> <span class="keyword">while</span> change_num < <span class="number">1</span>: <span class="comment"># 默认至少有一种数据增强生效</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> self.is_rotate_img_bbox:</span><br><span class="line"> <span class="keyword">if</span> random.random() > self.rotation_rate: <span class="comment"># 旋转</span></span><br><span class="line"> change_num += <span class="number">1</span></span><br><span class="line"> angle = random.uniform(-self.max_rotation_angle, self.max_rotation_angle)</span><br><span class="line"> scale = random.uniform(<span class="number">0.7</span>, <span class="number">0.8</span>)</span><br><span class="line"> img, bboxes = self._rotate_img_bbox(img, bboxes, angle, scale)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> self.is_shift_pic_bboxes:</span><br><span class="line"> <span class="keyword">if</span> random.random() < self.shift_rate: <span class="comment"># 平移</span></span><br><span class="line"> change_num += <span class="number">1</span></span><br><span class="line"> img, bboxes = self._shift_pic_bboxes(img, bboxes)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> self.is_changeLight:</span><br><span class="line"> <span class="keyword">if</span> random.random() > self.change_light_rate: <span class="comment"># 改变亮度</span></span><br><span class="line"> change_num += <span class="number">1</span></span><br><span class="line"> img = self._changeLight(img)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> self.is_addNoise:</span><br><span class="line"> <span class="keyword">if</span> random.random() < self.add_noise_rate: <span class="comment"># 加噪声</span></span><br><span class="line"> change_num += <span class="number">1</span></span><br><span class="line"> img = self._addNoise(img)</span><br><span class="line"> <span class="keyword">if</span> self.is_cutout:</span><br><span class="line"> <span class="keyword">if</span> random.random() < self.cutout_rate: <span class="comment"># cutout</span></span><br><span class="line"> change_num += <span class="number">1</span></span><br><span class="line"> img = self._cutout(img, bboxes, length=self.cut_out_length, n_holes=self.cut_out_holes,</span><br><span class="line"> threshold=self.cut_out_threshold)</span><br><span class="line"> <span class="keyword">if</span> self.is_filp_pic_bboxes:</span><br><span class="line"> <span class="keyword">if</span> random.random() < self.flip_rate: <span class="comment"># 翻转</span></span><br><span class="line"> change_num += <span class="number">1</span></span><br><span class="line"> img, bboxes = self._filp_pic_bboxes(img, bboxes)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> img, bboxes</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment"># xml解析工具</span></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ToolHelper</span>():</span></span><br><span class="line"> <span class="comment"># 从xml文件中提取bounding box信息, 格式为[[x_min, y_min, x_max, y_max, name]]</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">parse_xml</span>(<span class="params">self, path</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 输入：</span></span><br><span class="line"><span class="string"> xml_path: xml的文件路径</span></span><br><span class="line"><span class="string"> 输出：</span></span><br><span class="line"><span class="string"> 从xml文件中提取bounding box信息, 格式为[[x_min, y_min, x_max, y_max, name]]</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> tree = ETl.parse(path)</span><br><span class="line"> root = tree.getroot()</span><br><span class="line"> objs = root.findall(<span class="string">'object'</span>)</span><br><span class="line"> coords = <span class="built_in">list</span>()</span><br><span class="line"> <span class="keyword">for</span> ix, obj <span class="keyword">in</span> <span class="built_in">enumerate</span>(objs):</span><br><span class="line"> name = obj.find(<span class="string">'name'</span>).text</span><br><span class="line"> box = obj.find(<span class="string">'bndbox'</span>)</span><br><span class="line"> x_min = <span class="built_in">int</span>(box[<span class="number">0</span>].text)</span><br><span class="line"> y_min = <span class="built_in">int</span>(box[<span class="number">1</span>].text)</span><br><span class="line"> x_max = <span class="built_in">int</span>(box[<span class="number">2</span>].text)</span><br><span class="line"> y_max = <span class="built_in">int</span>(box[<span class="number">3</span>].text)</span><br><span class="line"> coords.append([x_min, y_min, x_max, y_max, name])</span><br><span class="line"> <span class="keyword">return</span> coords</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 保存图片结果</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">save_img</span>(<span class="params">self, file_name, save_folder, img</span>):</span></span><br><span class="line"> cv2.imwrite(os.path.join(save_folder, file_name), img)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 保持xml结果</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">save_xml</span>(<span class="params">self, file_name, save_folder, img_info, height, width, channel, bboxs_info,image_name,image_folder,image</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> :param file_name:文件名</span></span><br><span class="line"><span class="string"> :param save_folder:#保存的xml文件的结果</span></span><br><span class="line"><span class="string"> :param height:图片的信息</span></span><br><span class="line"><span class="string"> :param width:图片的宽度</span></span><br><span class="line"><span class="string"> :param channel:通道</span></span><br><span class="line"><span class="string"> :return:</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> folder_name, img_name = img_info <span class="comment"># 得到图片的信息</span></span><br><span class="line"></span><br><span class="line"> E = objectify.ElementMaker(annotate=<span class="literal">False</span>)</span><br><span class="line"></span><br><span class="line"> anno_tree = E.annotation(</span><br><span class="line"> E.folder(folder_name),</span><br><span class="line"> E.filename(img_name),</span><br><span class="line"> E.path(os.path.join(folder_name, img_name)),</span><br><span class="line"> E.source(</span><br><span class="line"> E.database(<span class="string">'Unknown'</span>),</span><br><span class="line"> ),</span><br><span class="line"> E.size(</span><br><span class="line"> E.width(width),</span><br><span class="line"> E.height(height),</span><br><span class="line"> E.depth(channel)</span><br><span class="line"> ),</span><br><span class="line"> E.segmented(<span class="number">0</span>),</span><br><span class="line"> )</span><br><span class="line"></span><br><span class="line"> labels, bboxs = bboxs_info <span class="comment"># 得到边框和标签信息</span></span><br><span class="line"> <span class="keyword">for</span> label, box <span class="keyword">in</span> <span class="built_in">zip</span>(labels, bboxs):</span><br><span class="line"> <span class="keyword">if</span> box[<span class="number">0</span>] > box[<span class="number">2</span>]:</span><br><span class="line"> box[<span class="number">0</span>], box[<span class="number">2</span>] = box[<span class="number">2</span>], box[<span class="number">0</span>]</span><br><span class="line"> <span class="keyword">if</span> box[<span class="number">1</span>] > box[<span class="number">3</span>]:</span><br><span class="line"> box[<span class="number">1</span>], box[<span class="number">3</span>] = box[<span class="number">3</span>], box[<span class="number">1</span>]</span><br><span class="line"> cv2.rectangle(image, (box[<span class="number">0</span>],box[<span class="number">1</span>]), (box[<span class="number">2</span>], box[<span class="number">3</span>]), (<span class="number">0</span>, <span class="number">255</span>, <span class="number">0</span>), <span class="number">4</span>)</span><br><span class="line"> cv2.imwrite(os.path.join(image_folder, image_name), image)</span><br><span class="line"> anno_tree.append(</span><br><span class="line"> E.<span class="built_in">object</span>(</span><br><span class="line"> E.name(label),</span><br><span class="line"> E.pose(<span class="string">'Unspecified'</span>),</span><br><span class="line"> E.truncated(<span class="string">'0'</span>),</span><br><span class="line"> E.difficult(<span class="string">'0'</span>),</span><br><span class="line"> E.bndbox(</span><br><span class="line"> E.xmin(box[<span class="number">0</span>]),</span><br><span class="line"> E.ymin(box[<span class="number">1</span>]),</span><br><span class="line"> E.xmax(box[<span class="number">2</span>]),</span><br><span class="line"> E.ymax(box[<span class="number">3</span>])</span><br><span class="line"> )</span><br><span class="line"> ))</span><br><span class="line"></span><br><span class="line"> etree.ElementTree(anno_tree).write(os.path.join(save_folder, file_name), pretty_print=<span class="literal">True</span>)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line"></span><br><span class="line"> need_aug_num = <span class="number">30</span> <span class="comment"># 每张图片需要增强的次数</span></span><br><span class="line"></span><br><span class="line"> is_endwidth_dot = <span class="literal">True</span> <span class="comment"># 文件是否以.jpg或者png结尾</span></span><br><span class="line"></span><br><span class="line"> dataAug = DataAugmentForObjectDetection() <span class="comment"># 数据增强工具类</span></span><br><span class="line"></span><br><span class="line"> toolhelper = ToolHelper() <span class="comment"># 工具</span></span><br><span class="line"> <span class="comment"># 获取相关参数</span></span><br><span class="line"> parser = argparse.ArgumentParser()</span><br><span class="line"> parser.add_argument(<span class="string">'--source_img_path'</span>, <span class="built_in">type</span>=<span class="built_in">str</span>, default=<span class="string">'data/Images'</span>)</span><br><span class="line"> parser.add_argument(<span class="string">'--source_xml_path'</span>, <span class="built_in">type</span>=<span class="built_in">str</span>, default=<span class="string">'data/Annotations'</span>)</span><br><span class="line"> parser.add_argument(<span class="string">'--save_img_path'</span>, <span class="built_in">type</span>=<span class="built_in">str</span>, default=<span class="string">'data/Images2'</span>)</span><br><span class="line"> parser.add_argument(<span class="string">'--save_img_path2'</span>, <span class="built_in">type</span>=<span class="built_in">str</span>, default=<span class="string">'data/Images3'</span>)</span><br><span class="line"> parser.add_argument(<span class="string">'--save_xml_path'</span>, <span class="built_in">type</span>=<span class="built_in">str</span>, default=<span class="string">'data/Annotations2'</span>)</span><br><span class="line"> args = parser.parse_args()</span><br><span class="line"> source_img_path = args.source_img_path <span class="comment"># 图片原始位置</span></span><br><span class="line"></span><br><span class="line"> source_xml_path = args.source_xml_path <span class="comment"># xml的原始位置</span></span><br><span class="line"></span><br><span class="line"> save_img_path = args.save_img_path <span class="comment"># 图片增强结果保存文件</span></span><br><span class="line"> save_img_path2 = args.save_img_path2 <span class="comment"># 图片增强结果验证文件</span></span><br><span class="line"> save_xml_path = args.save_xml_path <span class="comment"># xml增强结果保存文件</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 如果保存文件夹不存在就创建</span></span><br><span class="line"> <span class="keyword">if</span> <span class="keyword">not</span> os.path.exists(save_img_path):</span><br><span class="line"> os.mkdir(save_img_path)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> <span class="keyword">not</span> os.path.exists(save_xml_path):</span><br><span class="line"> os.mkdir(save_xml_path)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> parent, _, files <span class="keyword">in</span> os.walk(source_img_path):</span><br><span class="line"> files.sort()</span><br><span class="line"> <span class="keyword">for</span> file <span class="keyword">in</span> files:</span><br><span class="line"> cnt = <span class="number">0</span></span><br><span class="line"> pic_path = os.path.join(parent, file)</span><br><span class="line"> xml_path = os.path.join(source_xml_path, file[:-<span class="number">4</span>] + <span class="string">'.xml'</span>)</span><br><span class="line"> values = toolhelper.parse_xml(xml_path) <span class="comment"># 解析得到box信息，格式为[[x_min,y_min,x_max,y_max,name]]</span></span><br><span class="line"> coords = [v[:<span class="number">4</span>] <span class="keyword">for</span> v <span class="keyword">in</span> values] <span class="comment"># 得到框</span></span><br><span class="line"> labels = [v[-<span class="number">1</span>] <span class="keyword">for</span> v <span class="keyword">in</span> values] <span class="comment"># 对象的标签</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 如果图片是有后缀的</span></span><br><span class="line"> <span class="keyword">if</span> is_endwidth_dot:</span><br><span class="line"> <span class="comment"># 找到文件的最后名字</span></span><br><span class="line"> dot_index = file.rfind(<span class="string">'.'</span>)</span><br><span class="line"> _file_prefix = file[:dot_index] <span class="comment"># 文件名的前缀</span></span><br><span class="line"> _file_suffix = file[dot_index:] <span class="comment"># 文件名的后缀</span></span><br><span class="line"> img = cv2.imread(pic_path)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># show_pic(img, coords) # 显示原图</span></span><br><span class="line"> <span class="keyword">while</span> cnt < need_aug_num: <span class="comment"># 继续增强</span></span><br><span class="line"> auged_img, auged_bboxes = dataAug.dataAugment(img, coords)</span><br><span class="line"></span><br><span class="line"> auged_bboxes_int = np.array(auged_bboxes).astype(np.int32)</span><br><span class="line"> height, width, channel = auged_img.shape <span class="comment"># 得到图片的属性</span></span><br><span class="line"> img_name = <span class="string">'{}_{}{}'</span>.<span class="built_in">format</span>(_file_prefix, cnt + <span class="number">1</span>, _file_suffix) <span class="comment"># 图片保存的信息</span></span><br><span class="line"> toolhelper.save_img(img_name, save_img_path,</span><br><span class="line"> auged_img) <span class="comment"># 保存增强图片</span></span><br><span class="line"></span><br><span class="line"> toolhelper.save_xml(<span class="string">'{}_{}.xml'</span>.<span class="built_in">format</span>(_file_prefix, cnt + <span class="number">1</span>),</span><br><span class="line"> save_xml_path, (save_img_path, img_name), height, width, channel,</span><br><span class="line"> (labels, auged_bboxes_int),img_name,save_img_path2,auged_img) <span class="comment"># 保存xml文件</span></span><br><span class="line"> <span class="comment"># show_pic(auged_img, auged_bboxes) # 强化后的图</span></span><br><span class="line"> <span class="built_in">print</span>(img_name)</span><br><span class="line"> cnt += <span class="number">1</span> <span class="comment"># 继续增强下一张</span></span><br><span class="line"></span><br></pre></td></tr></table></figure><p><strong><em>注意 need_aug_num = 30 # 每张图片需要增强的次数，这里将得到3000张变换后的图片</em></strong></p><p>代码中有五处路径，分别为</p><p>data/Images：100张图片</p><p>data/Images2：3000张变换后的图片</p><p>data/Images3：变换后的标注（用于检验数据增广是否正确）</p><p>data/Annotations：100张图片对应的xml标签</p><p>data/Annotations2：3000张变换后的图片对应的xml标签</p><p>注意data和DataAugmentforLabelImg.py在同一个目录！！！</p><p>增广后的图片：</p><p><img src="https://z3.ax1x.com/2021/06/11/2WCisP.png" alt=""></p><p>增广后的xml标签：</p><p><img src="https://z3.ax1x.com/2021/06/11/2W9XVO.png" alt=""></p><p>对增广是否正确的验证：</p><p><img src="https://z3.ax1x.com/2021/06/11/2WC9xI.png" alt=""></p><p><strong><em>可以看出每个画框都很准确！！！！</em></strong></p><h5 id="第二步：新建transform-py用于将标签转换为txt标签"><a href="#第二步：新建transform-py用于将标签转换为txt标签" class="headerlink" title="第二步：新建transform.py用于将标签转换为txt标签"></a>第二步：新建transform.py用于将标签转换为txt标签</h5><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br></pre></td><td class="code"><pre><span class="line">import os</span><br><span class="line">import xml.etree.ElementTree as ET</span><br><span class="line"></span><br><span class="line">classes = ["banana", "fenda", "cola", "spirit"]</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"># 将x1, y1, x2, y2转换成yolov5所需要的x, y, w, h格式</span><br><span class="line">def xyxy2xywh(size, box):</span><br><span class="line"> dw = 1. / size[0]</span><br><span class="line"> dh = 1. / size[1]</span><br><span class="line"> x = (box[0] + box[2]) / 2 * dw</span><br><span class="line"> y = (box[1] + box[3]) / 2 * dh</span><br><span class="line"> w = (box[2] - box[0]) * dw</span><br><span class="line"> h = (box[3] - box[1]) * dh</span><br><span class="line"> return (x, y, w, h) # 返回的都是标准化后的值</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">def voc2yolo(path):</span><br><span class="line"> # 可以打印看看该路径是否正确</span><br><span class="line"> print(len(os.listdir(path)))</span><br><span class="line"> # 遍历每一个xml文件</span><br><span class="line"> for file in os.listdir(path):</span><br><span class="line"> # xml文件的完整路径, 注意：因为是路径所以要确保准确，我是直接使用了字符串拼接, 为了保险可以用os.path.join(path, file)</span><br><span class="line"> label_file = path + file</span><br><span class="line"> print(label_file)</span><br><span class="line"> # 最终要改成的txt格式文件,这里我是放在voc2007/labels/下面</span><br><span class="line"> # 注意: labels文件夹必须存在，没有就先创建，不然会报错</span><br><span class="line"> out_file = open(path.replace('Annotations2', 'labels2') + file.replace('xml', 'txt'), 'w')</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> # 开始解析xml文件</span><br><span class="line"> tree = ET.parse(label_file)</span><br><span class="line"> root = tree.getroot()</span><br><span class="line"> size = root.find('size') # 图片的shape值</span><br><span class="line"> w = int(size.find('width').text)</span><br><span class="line"> h = int(size.find('height').text)</span><br><span class="line"> for obj in root.iter('object'):</span><br><span class="line"> cls = obj.find('name').text</span><br><span class="line"> # print(cls)</span><br><span class="line"> if cls not in classes:</span><br><span class="line"> continue</span><br><span class="line"> cls_id = classes.index(cls)</span><br><span class="line"> # 获取整个bounding box框</span><br><span class="line"> bndbox = obj.find('bndbox')</span><br><span class="line"> # xml给出的是x1, y1, x2, y2</span><br><span class="line"> box = [float(bndbox.find('xmin').text), float(bndbox.find('ymin').text), float(bndbox.find('xmax').text),</span><br><span class="line"> float(bndbox.find('ymax').text)]</span><br><span class="line"> # 将x1, y1, x2, y2转换成yolov5所需要的x_center, y_center, w, h格式</span><br><span class="line"> bbox = xyxy2xywh((w, h), box)</span><br><span class="line"> # 写入目标文件中，格式为 id x y w h</span><br><span class="line"></span><br><span class="line"> out_file.write(str(cls_id) + " " + " ".join(str(x) for x in bbox) + '\n')</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">if __name__ == '__main__':</span><br><span class="line"> # 这里要改成自己数据集路径的格式</span><br><span class="line"> path = 'Annotations2/'</span><br><span class="line"> voc2yolo(path)</span><br><span class="line"></span><br></pre></td></tr></table></figure><p><strong><em>这里需要十分注意的是：classes = [“banana”, “fenda”, “cola”, “spirit”] 分别对应序号0，1，2，3！！！！</em></strong></p><p>要在.py文件同目录下新建Annotations2文件夹和labels2文件夹，把之前增广后的xml标签文件全部放进Annotations2文件夹中，运行后labels2下即有3000个txt格式的标签</p><p><img src="https://z3.ax1x.com/2021/06/11/2WCpRA.png" alt=""></p><p><strong><em>到这里所需数据集已经准备好，下一步就是用于训练啦！！！</em></strong></p><p>这里也提供我的数据集的下载，百度网盘链接：<a href="https://pan.baidu.com/s/1mYqwDH46OD3p9Jrfbvj6xQ（提取码1234）大小一个多G">https://pan.baidu.com/s/1mYqwDH46OD3p9Jrfbvj6xQ（提取码1234）大小一个多G</a></p><h2 id="训练前的准备"><a href="#训练前的准备" class="headerlink" title="训练前的准备"></a>训练前的准备</h2><p><strong><em>darknet训练前还是有很多地方需要配置和修改的，下面是是最简洁快速的配置方法了</em></strong></p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">cd</span> ~/yolo_detect/src/darknet_ros/darknet</span><br><span class="line">make</span><br></pre></td></tr></table></figure><p>编译好后在darknet下新建文件夹classify</p><h5 id="第一步：模仿以下的文件结构创建文件"><a href="#第一步：模仿以下的文件结构创建文件" class="headerlink" title="第一步：模仿以下的文件结构创建文件"></a>第一步：模仿以下的文件结构创建文件</h5><p>文件夹中的文件形式如下所示：</p><p><img src="https://z3.ax1x.com/2021/06/11/2W92rV.png" alt=""></p><p><strong><em>其中：</em></strong></p><p><strong><em>backup存放了权重文件</em></strong><br><strong><em>dataset里面存放了image和txt格式的标签</em></strong><br><strong><em>classify.data指定了训练用到的数据集和权重的路径</em></strong><br><strong><em>classify.names指定了类别名称，顺序应与label中的对应</em></strong><br><strong><em>detapre.sh用于生成text.txt,train.txt这两个文本</em></strong><br><strong><em>test.txt,train.txt中分别存放了数据集和验证集每张图片和标签的绝对路径</em></strong></p><p><img src="https://z3.ax1x.com/2021/06/11/2W9RbT.png" alt=""></p><p><img src="https://z3.ax1x.com/2021/06/11/2W9oG9.png" alt=""></p><p><img src="https://z3.ax1x.com/2021/06/11/2W9454.png" alt=""></p><p><img src="https://z3.ax1x.com/2021/06/11/2W9IPJ.png" alt=""></p><p><strong><em>必须准备好这些文件后再进行下一步！！！！！！！</em></strong></p><h5 id="第二步：修改yolov3-tiny网络的配置"><a href="#第二步：修改yolov3-tiny网络的配置" class="headerlink" title="第二步：修改yolov3-tiny网络的配置"></a>第二步：修改yolov3-tiny网络的配置</h5><p>需要修改的地方：</p><p>需要修改的地方为cfg下的yolov3-tiny.cfg<br>将#Testing注释,将#Traning取消注释</p><p><img src="https://z3.ax1x.com/2021/06/11/2W9T2R.png" alt=""></p><p>同时有两个yolo层及与它们相邻的convolutional层需要修改<br>classes的值改为4（香蕉，可乐，雪碧，芬达）<br>filters的值为3*(classes+5)=27</p><p><img src="https://z3.ax1x.com/2021/06/11/2W9T2R.png" alt=""></p><h5 id="第三步：修改保存权重的频率"><a href="#第三步：修改保存权重的频率" class="headerlink" title="第三步：修改保存权重的频率"></a>第三步：修改保存权重的频率</h5><p>其次在examples下的detector.c中可以将138行改为<code>if(i%1000==0 || (i < 1000 && i%100 == 0)){</code>即1000代以前没一百代保存一次权重，1000代以后每隔1000代保存一次</p><h5 id="第四步：开始训练"><a href="#第四步：开始训练" class="headerlink" title="第四步：开始训练"></a>第四步：开始训练</h5><p><code>sudo ./darknet detector train classify/classify.data cfg/yolov3-tiny.cfg</code></p><p>出现报错：</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">./darknet: error while loading shared libraries: libcudart.so.9.2: cannot open shared object file: No such file or directory</span><br></pre></td></tr></table></figure><p>解决办法：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">sudo cp /usr/<span class="built_in">local</span>/cuda-9.2/lib64/libcudart.so.9.2 /usr/<span class="built_in">local</span>/lib/libcudart.so.9.2 && sudo ldconfig </span><br><span class="line">sudo cp /usr/<span class="built_in">local</span>/cuda-9.2/lib64/libcublas.so.9.2 /usr/<span class="built_in">local</span>/lib/libcublas.so.9.2 && sudo ldconfig </span><br><span class="line">sudo cp /usr/<span class="built_in">local</span>/cuda-9.2/lib64/libcurand.so.9.2 /usr/<span class="built_in">local</span>/lib/libcurand.so.9.2 && sudo ldconfig</span><br><span class="line">sudo cp /usr/<span class="built_in">local</span>/cuda-9.2/lib64/libcudnn.so.7 /usr/<span class="built_in">local</span>/lib/libcudnn.so.7 && sudo ldconfig</span><br></pre></td></tr></table></figure><p>重新执行：</p><p><code>sudo ./darknet detector train classify/classify.data cfg/yolov3-tiny.cfg</code></p><p>成功开始训练：</p><p><img src="https://z3.ax1x.com/2021/06/11/2W9gK0.png" alt=""></p><p><strong><em>大概训练了一个小时训练到5000代，得到权重文件yolov3-tiny_5000.weights</em></strong></p><h2 id="检验训练结果"><a href="#检验训练结果" class="headerlink" title="检验训练结果"></a>检验训练结果</h2><h5 id="第一步："><a href="#第一步：" class="headerlink" title="第一步："></a>第一步：</h5><p>进入darknet ros<br>将所的权重文件放入yolo_network_config/weights下<br>由于darknet并没有yolov3-tiny的配置文件<br>所以需要在config下新建yolov3-tiny.yaml，内容如图所示</p><p><img src="https://z3.ax1x.com/2021/06/11/2W9bKx.png" alt=""></p><h5 id="第二步："><a href="#第二步：" class="headerlink" title="第二步："></a>第二步：</h5><p>复制刚刚改过的yolov3-tiny.cfg到yolo_network_config/cfg下</p><h5 id="第三步："><a href="#第三步：" class="headerlink" title="第三步："></a>第三步：</h5><p>将launch下的darknet_ros.launch<br>内容修改为如图所示</p><p><img src="https://z3.ax1x.com/2021/06/11/2W9qr6.png" alt=""></p><h5 id="第四步："><a href="#第四步：" class="headerlink" title="第四步："></a>第四步：</h5><p>打开config下的yaml</p><p>修改相机话题为 topic: /camera/color/image_raw</p><p><img src="https://z3.ax1x.com/2021/06/11/2W9rCj.png" alt=""></p><p>从中可以看到该节点运行时会发布三个话题<br>/darknet_ros/found_object<br>/darknet_ros/bounding_boxes<br>/darknet_ros/detection_image</p><p>/darknet_ros/bounding_boxes的消息示例如图所示</p><p><img src="https://z3.ax1x.com/2021/06/11/2W96vq.png" alt=""></p><h5 id="第五步："><a href="#第五步：" class="headerlink" title="第五步："></a>第五步：</h5><p>上述步骤都完成后</p><p>通过gazebo加载仿真环境<br>运行roslaunch darknet_ros darknet_ros.launch<br>可以发现检测效果很好且处理速度快，检测结果见视频</p><h2 id="分类之后的抓取姿态检测"><a href="#分类之后的抓取姿态检测" class="headerlink" title="分类之后的抓取姿态检测"></a>分类之后的抓取姿态检测</h2><p><strong>ORK或者几何方法？？？敬请期待</strong></p>]]></content>
<categories>
<category> 工业机器人 </category>
</categories>
<tags>
<tag> ROS </tag>
<tag> Gazebo </tag>
<tag> Moveit </tag>
<tag> Yolov3 </tag>
</tags>
</entry>
<entry>
<title>ROS Kinetic & Gazebo7下的ABB-Yumi双臂协作机器人的食品分类抓取仿真（一）联合仿真与环境搭建</title>
<link href="2021/06/05/ROS%20Kinetic%20&%20Gazebo7%E4%B8%8B%E7%9A%84ABB-Yumi%E5%8F%8C%E8%87%82%E5%8D%8F%E4%BD%9C%E6%9C%BA%E5%99%A8%E4%BA%BA%E7%9A%84%E9%A3%9F%E5%93%81%E5%88%86%E7%B1%BB%E6%8A%93%E5%8F%96%E4%BB%BF%E7%9C%9F%EF%BC%88%E4%B8%80%EF%BC%89%E8%81%94%E5%90%88%E4%BB%BF%E7%9C%9F%E4%B8%8E%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/"/>
<url>2021/06/05/ROS%20Kinetic%20&%20Gazebo7%E4%B8%8B%E7%9A%84ABB-Yumi%E5%8F%8C%E8%87%82%E5%8D%8F%E4%BD%9C%E6%9C%BA%E5%99%A8%E4%BA%BA%E7%9A%84%E9%A3%9F%E5%93%81%E5%88%86%E7%B1%BB%E6%8A%93%E5%8F%96%E4%BB%BF%E7%9C%9F%EF%BC%88%E4%B8%80%EF%BC%89%E8%81%94%E5%90%88%E4%BB%BF%E7%9C%9F%E4%B8%8E%E7%8E%AF%E5%A2%83%E6%90%AD%E5%BB%BA/</url>
<content type="html"><![CDATA[<h2 id="项目简介"><a href="#项目简介" class="headerlink" title="项目简介"></a>项目简介</h2><p> <img src="https://z3.ax1x.com/2021/06/08/2rYacD.jpg" alt=""></p><p> <strong>本项目原计划研究基于双臂机器人的物体插孔装配操作，这个问题是双臂机器人协作规划与双臂协调运动控制的关键问题。研究基于项目现有的物理仿真环境，在双臂机器人抓取到物体之后，规划双臂的运动轨迹，实现两个可相互装配物体的插孔操作。</strong></p><p> <strong>但是经过仿真测试发现Gazebo对物理效果的仿真有缺陷，两个工件是否能成功装配很大程度上收工件尺寸，质量，惯量，材质，碰撞等参数以及逆运动学求解精度的影响，因此任务调整为结合Yolov3目标检测算法完成双臂的协作控制以及水果以及各种饮料的分类抓取</strong></p><h2 id="环境"><a href="#环境" class="headerlink" title="环境"></a>环境</h2><p>Linux: ubuntu 16.04+Ros Kinetic+Gazebo7+Rviz+MoveitSetupAssistants+Blender2.9.3+Yolov3</p><h2 id="Gazebo和Moveit的联合仿真"><a href="#Gazebo和Moveit的联合仿真" class="headerlink" title="Gazebo和Moveit的联合仿真"></a>Gazebo和Moveit的联合仿真</h2><p> ABB-Yumi是一款比较新型的双臂协助机器人，开源资源相当有限且很不完善，因此从零搭建仿真环境十分困难，同时联合仿真的相关资料基本上都是基于自己搭建的模型以及UR系列的单臂机器人，可以借鉴但是实现起来并不简单。</p><p> 此项目的前期工作就是通过一个简单的urdf模型一步步实现Moveit到Gazebo的联合仿真，中间涉及到各种控制器的配置，篇幅有限，所以暂时不在这里介绍，中间遇到了各种各样的bug,到最后虽然实现了联合仿真但是发现机器人总是和预定的位姿有一个小的偏差，推测可能和模型文件本身或者Moveit里面对position的tolerence有关系，经过一番寻找，在Github上面找到了一个有用的功能配置包，包括它的urdf model以及moveit config package，可以通过<a href="https://pan.baidu.com/s/1mD1M7extrkIwS10XVGYKJA">百度网盘</a>下载，提取码为1234,使用方法具体可以看包里面的md说明文档。</p><p> </p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">- After pulling and building the project, use the following <span class="built_in">command</span> to start YuMi control and simulation: </span><br><span class="line"></span><br><span class="line">$ roslaunch yumi_launch yumi_gazebo_traj_pos_control.launch</span><br><span class="line"></span><br><span class="line">- To start planning excution with moveit, use:</span><br><span class="line"></span><br><span class="line">$ roslaunch yumi_moveit_config demo_planning_excution.launch</span><br><span class="line"></span><br><span class="line">- An example of controlling the robot to reach a desired position is included <span class="keyword">in</span> python script <span class="keyword">in</span> yumi_test_controllers/scripts/:</span><br><span class="line"></span><br><span class="line">$ rosrun yumi_test_controllers test_moveit.py</span><br></pre></td></tr></table></figure><h3 id="1"><a href="#1" class="headerlink" title="1."></a>1.</h3><p> <strong>第一步启动Gazebo加载worlds里面的my_world.world模型文件，打开可以看见里面对kinect摄像头的模型渲染采用了dae网格文件，但是网格文件的路径是有问题的，可以在下载Gazebo的官方模型库后把<mesh>….</mesh>内容用</strong></p><figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="name">mesh</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">uri</span>></span>model://kinect/meshes/kinect.dae<span class="tag"></<span class="name">uri</span>></span></span><br><span class="line"><span class="tag"></<span class="name">mesh</span>></span></span><br></pre></td></tr></table></figure><p><strong>代替即可加载出kinect相机的渲染</strong></p><h3 id="2"><a href="#2" class="headerlink" title="2."></a>2.</h3><p> <strong>第二步启动了Moveit的相关节点和服务，其中这里如果需要Rviz可以自己手动加载</strong></p><h3 id="3"><a href="#3" class="headerlink" title="3."></a>3.</h3><p> <strong>第三步是抓取的例程，手动设定目标位姿可以前往目标点以及实现夹爪的开闭，但是里面的抓取函数没有用，表现为即使夹爪闭合也无法夹住物体，推测与夹爪材料的阻尼系数以及木块的阻尼系数有关系，因此改用夹爪闭合后直接连接两个link的方式来夹取物体。</strong></p><p> <strong>于是这里又要用到Github上面的一个项目，叫做gazebo_ros_link_attacher-master，<a href="https://pan.baidu.com/s/1KzVdCBzdfhOJov_HXRiUHQ">百度网盘链接</a>，提取码：1234，系统是Kinetic。Melodic版本 <a href="https://pan.baidu.com/s/1hePLw_qtlOhB631TJXQ4tg">百度网盘链接</a>，提取码同上。</strong></p><p> <strong>使用方法官方的说明文档很详细，要注意的就是需要编译然后把插件</strong><code><plugin name="ros_link_attacher_plugin" filename="libgazebo_ros_link_attacher.so"/></code></p><p><strong>加到自己的那个world里面而前面那个ABB-Yumi的配置包不需要编译，并且在后面ABB-Yumi用到这里面的服务时需要</strong></p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">cd</span> gazebo_link_attacher_ws</span><br><span class="line"><span class="built_in">source</span> devel/setup.bash</span><br></pre></td></tr></table></figure><p><strong><em>attach.py</em></strong></p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#!/usr/bin/env python</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> rospy</span><br><span class="line"><span class="keyword">from</span> gazebo_ros_link_attacher.srv <span class="keyword">import</span> Attach, AttachRequest, AttachResponse</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line"> rospy.init_node(<span class="string">'demo_attach_links'</span>)</span><br><span class="line"> rospy.loginfo(<span class="string">"Creating ServiceProxy to /link_attacher_node/attach"</span>)</span><br><span class="line"> attach_srv = rospy.ServiceProxy(<span class="string">'/link_attacher_node/attach'</span>,</span><br><span class="line"> Attach)</span><br><span class="line"> attach_srv.wait_for_service()</span><br><span class="line"> rospy.loginfo(<span class="string">"Created ServiceProxy to /link_attacher_node/attach"</span>)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Link them</span></span><br><span class="line"> rospy.loginfo(<span class="string">"Attaching cube1 and cube2"</span>)</span><br><span class="line"> req = AttachRequest()</span><br><span class="line"> req.model_name_1 = <span class="string">"cube1"</span></span><br><span class="line"> req.link_name_1 = <span class="string">"link"</span></span><br><span class="line"> req.model_name_2 = <span class="string">"cube2"</span></span><br><span class="line"> req.link_name_2 = <span class="string">"link"</span></span><br><span class="line"></span><br><span class="line"> attach_srv.call(req)</span><br><span class="line"> <span class="comment"># From the shell:</span></span><br><span class="line"> <span class="string">"""</span></span><br><span class="line"><span class="string">rosservice call /link_attacher_node/attach "model_name_1: 'cube1'</span></span><br><span class="line"><span class="string">link_name_1: 'link'</span></span><br><span class="line"><span class="string">model_name_2: 'cube2'</span></span><br><span class="line"><span class="string">link_name_2: 'link'"</span></span><br><span class="line"><span class="string"> """</span></span><br><span class="line"></span><br><span class="line"> rospy.loginfo(<span class="string">"Attaching cube2 and cube3"</span>)</span><br><span class="line"> req = AttachRequest()</span><br><span class="line"> req.model_name_1 = <span class="string">"cube2"</span></span><br><span class="line"> req.link_name_1 = <span class="string">"link"</span></span><br><span class="line"> req.model_name_2 = <span class="string">"cube3"</span></span><br><span class="line"> req.link_name_2 = <span class="string">"link"</span></span><br><span class="line"></span><br><span class="line"> attach_srv.call(req)</span><br><span class="line"></span><br><span class="line"> rospy.loginfo(<span class="string">"Attaching cube3 and cube1"</span>)</span><br><span class="line"> req = AttachRequest()</span><br><span class="line"> req.model_name_1 = <span class="string">"cube3"</span></span><br><span class="line"> req.link_name_1 = <span class="string">"link"</span></span><br><span class="line"> req.model_name_2 = <span class="string">"cube1"</span></span><br><span class="line"> req.link_name_2 = <span class="string">"link"</span></span><br><span class="line"></span><br><span class="line"> attach_srv.call(req)</span><br></pre></td></tr></table></figure><p> <strong>请求了服务把两个杆件连在一块</strong></p><p><strong>经过修改test_moveit.py调整目标点位姿，完成了对抓取的测试</strong></p><p><img src="https://z3.ax1x.com/2021/06/08/2rYGA1.png" alt=""></p><p> <strong>查看Rviz,可以看到已经显示深度相机采集的图像</strong></p><p><img src="https://z3.ax1x.com/2021/06/08/2rY1B9.png" alt=""></p><p><img src="https://z3.ax1x.com/2021/06/08/2rY37R.png" alt=""></p><p> <strong>发现点云数据飘在天上，经过分析应该是kinect模型参数里面的cameraFrame不对</strong></p><p><strong>应该在yumi.urdf.xacro中添加</strong></p><figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="name">link</span> <span class="attr">name</span>=<span class="string">"camera_link"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">visual</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">origin</span> <span class="attr">xyz</span>=<span class="string">" 0 0 0 "</span> <span class="attr">rpy</span>=<span class="string">"0 0 0"</span> /></span></span><br><span class="line"> <span class="tag"><<span class="name">geometry</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">sphere</span> <span class="attr">radius</span>=<span class="string">"0.01"</span> /></span></span><br><span class="line"> <span class="tag"></<span class="name">geometry</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">material</span> <span class="attr">name</span>=<span class="string">"black"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">color</span> <span class="attr">rgba</span>=<span class="string">"0 0 0 0.95"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">material</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">visual</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">link</span>></span></span><br><span class="line"><span class="tag"><<span class="name">joint</span> <span class="attr">name</span>=<span class="string">"camera_joint"</span> <span class="attr">type</span>=<span class="string">"fixed"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">origin</span> <span class="attr">xyz</span>=<span class="string">"0.7 0 1.0"</span> <span class="attr">rpy</span>=<span class="string">"0 ${PI} ${PI / 2}"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">parent</span> <span class="attr">link</span>=<span class="string">"world"</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">child</span> <span class="attr">link</span>=<span class="string">"camera_link"</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">joint</span>></span></span><br></pre></td></tr></table></figure><p> <strong>同时修改相机模型文件中的参考坐标系为camera link</strong></p><p><strong>到这里Kinect就已经成功加入到Gazebo里面了</strong></p><h2 id="搭建机器人的工作环境"><a href="#搭建机器人的工作环境" class="headerlink" title="搭建机器人的工作环境"></a>搭建机器人的工作环境</h2><p> <strong>由于Gazebo模型库的模型并不很符合此次任务的需求并且渲染效果不好不适合后面的目标检测，因此这里待抓取的香蕉、雪碧、芬达，可口可乐以及收纳盒的模型均来自网上下载之后再通过Blender2.93.0编辑</strong></p><p><strong>如果有需要，这里有百度网盘的下载链接：</strong></p><p><strong>香蕉：<a href="https://pan.baidu.com/s/1VjebDv5ycb6tPdLPDVbMnw">https://pan.baidu.com/s/1VjebDv5ycb6tPdLPDVbMnw</a></strong></p><p><strong>饮料：<a href="https://pan.baidu.com/s/1UUygB3csYz5b9SJJv6S9UQ">https://pan.baidu.com/s/1UUygB3csYz5b9SJJv6S9UQ</a></strong></p><p><strong>收纳盒：<a href="https://pan.baidu.com/s/1N93MCk2b2KTLfYSIjovaQw">https://pan.baidu.com/s/1N93MCk2b2KTLfYSIjovaQw</a></strong></p><p> <strong>后面两个均为blend工程文件修改尺寸可以直接导出贴图以及dae网格文件，前面的香蕉暂时还无法给它增加贴图。</strong></p><p><img src="https://z3.ax1x.com/2021/06/08/2rYJtx.png" alt=""></p><p> <strong>导出的文件：</strong></p><p><img src="https://z3.ax1x.com/2021/06/08/2su2Hf.jpg" alt=""></p><p><strong>使用时只加载dae就可以</strong></p><p><strong>使用方法</strong>：</p><p><strong>加入香蕉</strong></p><p><strong>在my_world.world文件中加入以下内容：</strong></p><figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="name">model</span> <span class="attr">name</span>=<span class="string">"banana"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">pose</span>></span>0.5 0 0 0 0 0<span class="tag"></<span class="name">pose</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">link</span> <span class="attr">name</span>=<span class="string">"banana_link"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">inertial</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">mass</span>></span>0.001<span class="tag"></<span class="name">mass</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">inertia</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">ixx</span>></span>0.166667<span class="tag"></<span class="name">ixx</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">ixy</span>></span>0<span class="tag"></<span class="name">ixy</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">ixz</span>></span>0<span class="tag"></<span class="name">ixz</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">iyy</span>></span>0.166667<span class="tag"></<span class="name">iyy</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">iyz</span>></span>0<span class="tag"></<span class="name">iyz</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">izz</span>></span>0.166667<span class="tag"></<span class="name">izz</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">inertia</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">inertial</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">collision</span> <span class="attr">name</span>=<span class="string">'collision2'</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">geometry</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">mesh</span>></span><span class="tag"><<span class="name">uri</span>></span>/home/li/catkin_ws/src/yumi_launch/worlds/Banana/file.dae<span class="tag"></<span class="name">uri</span>></span><span class="tag"></<span class="name">mesh</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">geometry</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">max_contacts</span>></span>10<span class="tag"></<span class="name">max_contacts</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">surface</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">contact</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">ode</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">contact</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">bounce</span>/></span></span><br><span class="line"> <span class="tag"><<span class="name">friction</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">torsional</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">ode</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">torsional</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">ode</span>/></span></span><br><span class="line"> <span class="tag"></<span class="name">friction</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">surface</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">collision</span>></span></span><br><span class="line"> </span><br><span class="line"> <span class="tag"><<span class="name">visual</span> <span class="attr">name</span>=<span class="string">"visual2"</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">geometry</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">mesh</span>></span><span class="tag"><<span class="name">uri</span>></span>/home/li/catkin_ws/src/yumi_launch/worlds/Banana/file.dae<span class="tag"></<span class="name">uri</span>></span><span class="tag"></<span class="name">mesh</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">geometry</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">visual</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">self_collide</span>></span>0<span class="tag"></<span class="name">self_collide</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">kinematic</span>></span>0<span class="tag"></<span class="name">kinematic</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">link</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">model</span>></span></span><br></pre></td></tr></table></figure><p><strong>poseframe调整位置；</strong></p><p><strong>如果物体轻飘飘的可以减小mass；</strong></p><p><strong>collision是碰撞区域，visual是可视化区域，两者都可以用dae渲染；</strong></p><p><strong>这里因为相机太大影响抓取索性把kinect模型的collision调整为一个边长0.001的小正方体；</strong></p><p><strong>惯性,质量等参数也可以考虑用meshlab来配置。</strong></p><p><strong>经过不断调试：</strong></p><p><img src="https://z3.ax1x.com/2021/06/08/2rYN9K.png" alt=""></p><p> <strong>发现物品视野很暗影响检测，因此在Gazebo中加了一个点光源，保存为新的my_lighted_world.world,并在将 yumi_gazebo_traj_pos_control.launch中加载的world文件改为my_lighted_world.world</strong></p><h3 id="最终工作环境如下："><a href="#最终工作环境如下：" class="headerlink" title="最终工作环境如下："></a>最终工作环境如下：</h3><p><img src="https://z3.ax1x.com/2021/06/08/2rYYh6.png" alt=""></p><p><img src="https://z3.ax1x.com/2021/06/08/2rYlnJ.png" alt=""></p><p> <strong>再次运行roslaunch yumi_launch yumi_gazebo_traj_pos_control.launch时发现gazebo_link_attacher的服务没有启动，原因在于my_lighted_world.world和原来的my_world.world sdf版本不相同，前者为sdf1.4,后者为sdf1.6,需要修改的地方就是把 <code><plugin name='ros_link_attacher_plugin' filename='libgazebo_ros_link_attacher.so'/></code>改为 <code><plugin name="ros_link_attacher_plugin" filename="libgazebo_ros_link_attacher.so"/></code>即可加载出link-attacher的相关服务。</strong></p><h2 id="Darknet-ROS下的食品分类"><a href="#Darknet-ROS下的食品分类" class="headerlink" title="Darknet ROS下的食品分类"></a>Darknet ROS下的食品分类</h2><p><strong>见下文：ROS-Kinetic-&-Gazebo7下的ABB-Yumi双臂协作机器人的食品分类抓取仿真（二）</strong></p>]]></content>
<categories>
<category> 工业机器人 </category>
</categories>
<tags>
<tag> ROS </tag>
<tag> Gazebo </tag>
<tag> Moveit </tag>
<tag> Yolov3 </tag>
</tags>
</entry>
<entry>
<title>YOLOv5实现中医药软包装检测</title>
<link href="2021/06/05/YOLOv5%E5%AE%9E%E7%8E%B0%E4%B8%AD%E5%8C%BB%E8%8D%AF%E8%BD%AF%E5%8C%85%E8%A3%85%E6%A3%80%E6%B5%8B/"/>
<url>2021/06/05/YOLOv5%E5%AE%9E%E7%8E%B0%E4%B8%AD%E5%8C%BB%E8%8D%AF%E8%BD%AF%E5%8C%85%E8%A3%85%E6%A3%80%E6%B5%8B/</url>
<content type="html"><![CDATA[<h2 id="运行环境"><a href="#运行环境" class="headerlink" title="运行环境"></a>运行环境</h2><p>windows10+Anaconda3+pytorch1.8+python3.8+pycharm</p><h2 id="安装Anaconda"><a href="#安装Anaconda" class="headerlink" title="安装Anaconda"></a>安装Anaconda</h2><p>1.访问<a href="https://www.anaconda.com/products/individual">Anaconda官网</a>下载Anaconda3并按照安装程序引导完成安装</p><p><img src="https://z3.ax1x.com/2021/06/05/2ta7Y4.jpg" style="zoom: 80%;" /></p><p>2.找到.condarc文件</p><p><img src="https://z3.ax1x.com/2021/06/05/2taHfJ.jpg" style="zoom: 50%;" /></p><p>将channels下的default项去掉</p><p>3.启动Anaconda Prompt</p><p>添加镜像源</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/</span><br><span class="line">conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge </span><br><span class="line">conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/</span><br><span class="line">conda config --add channels https://mirrors.ustc.edu.cn/anaconda/pkgs/main/</span><br><span class="line">conda config --add channels https://mirrors.ustc.edu.cn/anaconda/pkgs/free/</span><br><span class="line">conda config --add channels https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/</span><br><span class="line">conda config --add channels https://mirrors.ustc.edu.cn/anaconda/cloud/msys2/</span><br><span class="line">conda config --add channels https://mirrors.ustc.edu.cn/anaconda/cloud/bioconda/</span><br><span class="line">conda config --add channels https://mirrors.ustc.edu.cn/anaconda/cloud/menpo/</span><br><span class="line"></span><br><span class="line">conda config --<span class="built_in">set</span> show_channel_urls yes</span><br></pre></td></tr></table></figure><h2 id="安装Pytorch-amp-Cuda-amp-Cudnn"><a href="#安装Pytorch-amp-Cuda-amp-Cudnn" class="headerlink" title="安装Pytorch&Cuda&Cudnn"></a>安装Pytorch&Cuda&Cudnn</h2><p>1.创建虚拟环境</p><p><code>conda create -n yolov5 python=3.8</code></p><p>2.安装pytorch,torchvision,torchaudio,cudatoolkit</p><p><code>conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch</code></p><p>这里安装的pytorch默认为官网的最新版，cudatoolkit指定为10.2，如有其它版本需要</p><p>则访问<a href="https://pytorch.org/get-started/previous-versions/">以前版本</a>找到所需版本</p><p><img src="https://z3.ax1x.com/2021/06/05/2tdbE8.jpg" style="zoom: 80%;" /></p><p>至于cudatoolkit的版本，则需要根据nvidia驱动程序的版本来选择</p><p><img src="https://z3.ax1x.com/2021/06/05/2twmK1.jpg" style="zoom:80%;" /> </p><h2 id="安装YOLOv5"><a href="#安装YOLOv5" class="headerlink" title="安装YOLOv5"></a>安装YOLOv5</h2><p><strong>由于YOLOv5版本在不断更新，依赖也一直在变化</strong></p><p>本文的YOLOv5下载链接为：<a href="https://pan.baidu.com/s/1ZltpFOtg12Q-824O5WRn8A">https://pan.baidu.com/s/1ZltpFOtg12Q-824O5WRn8A</a></p><p>提取码：1234</p><p>1.安装YOLOv5</p><p><code>conda activate yolov5</code></p><p>切换到yolov5-develop目录</p><p><code>pip install -r requirements.txt</code></p><p><strong>到这里依赖全部安装完成</strong></p><p>2.使用pycharm打开yolov5-develop，编译环境选择刚刚所创建的虚拟环境</p><h2 id="数据集准备"><a href="#数据集准备" class="headerlink" title="数据集准备"></a>数据集准备</h2><p><strong>原始图片35张</strong></p><p><img src="https://z3.ax1x.com/2021/06/05/2trZFK.jpg" style="zoom:80%;" /></p><p><img src="https://z3.ax1x.com/2021/06/05/2trmWD.png" alt=""></p><p><strong>使用labelimg进行数据集标注</strong></p><p><strong>标注好后由于样本数量太小，需要进行数据增广</strong></p><p>具体标注和增广过程这里不在赘述</p><p>数据增广后的部分样本：</p><p><img src="https://z3.ax1x.com/2021/06/05/2tr2pF.jpg" alt=""></p><p>相应的每一张图片都有对应的标签，标注时要将标签保存为txt格式的，如果是xml格式的还需进行格式转换</p><p><strong>XML转txt</strong></p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> os</span><br><span class="line"><span class="keyword">import</span> xml.etree.ElementTree <span class="keyword">as</span> ET</span><br><span class="line"></span><br><span class="line">classes = [<span class="string">"aeroplane"</span>, <span class="string">"bicycle"</span>, <span class="string">"bird"</span>, <span class="string">"boat"</span>, <span class="string">"bottle"</span>, <span class="string">"bus"</span>, <span class="string">"car"</span>, <span class="string">"cat"</span>, <span class="string">"chair"</span>, <span class="string">"cow"</span>, <span class="string">"diningtable"</span>, <span class="string">"dog"</span>,</span><br><span class="line"> <span class="string">"horse"</span>, <span class="string">"motorbike"</span>, <span class="string">"person"</span>, <span class="string">"pottedplant"</span>, <span class="string">"sheep"</span>, <span class="string">"sofa"</span>, <span class="string">"train"</span>, <span class="string">"tvmonitor"</span>]</span><br><span class="line"></span><br><span class="line"><span class="comment"># 将x1, y1, x2, y2转换成yolov5所需要的x, y, w, h格式</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">xyxy2xywh</span>(<span class="params">size, box</span>):</span></span><br><span class="line"> dw = <span class="number">1.</span> / size[<span class="number">0</span>]</span><br><span class="line"> dh = <span class="number">1.</span> / size[<span class="number">1</span>]</span><br><span class="line"> x = (box[<span class="number">0</span>] + box[<span class="number">2</span>]) / <span class="number">2</span> * dw</span><br><span class="line"> y = (box[<span class="number">1</span>] + box[<span class="number">3</span>]) / <span class="number">2</span> * dh</span><br><span class="line"> w = (box[<span class="number">2</span>] - box[<span class="number">0</span>]) * dw</span><br><span class="line"> h = (box[<span class="number">3</span>] - box[<span class="number">1</span>]) * dh</span><br><span class="line"> <span class="keyword">return</span> (x, y, w, h) <span class="comment"># 返回的都是标准化后的值</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">voc2yolo</span>(<span class="params">path</span>):</span></span><br><span class="line"> <span class="comment"># 可以打印看看该路径是否正确</span></span><br><span class="line"> <span class="built_in">print</span>(<span class="built_in">len</span>(os.listdir(path)))</span><br><span class="line"> <span class="comment"># 遍历每一个xml文件</span></span><br><span class="line"> <span class="keyword">for</span> file <span class="keyword">in</span> os.listdir(path):</span><br><span class="line"> <span class="comment"># xml文件的完整路径, 注意：因为是路径所以要确保准确，我是直接使用了字符串拼接, 为了保险可以用os.path.join(path, file)</span></span><br><span class="line"> label_file = path + file</span><br><span class="line"> <span class="comment"># 最终要改成的txt格式文件,这里我是放在voc2007/labels/下面</span></span><br><span class="line"> <span class="comment"># 注意: labels文件夹必须存在，没有就先创建，不然会报错</span></span><br><span class="line"> out_file = <span class="built_in">open</span>(path.replace(<span class="string">'Annotations'</span>, <span class="string">'labels'</span>) + file.replace(<span class="string">'xml'</span>, <span class="string">'txt'</span>), <span class="string">'w'</span>)</span><br><span class="line"> <span class="comment"># print(label_file)</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 开始解析xml文件</span></span><br><span class="line"> tree = ET.parse(label_file)</span><br><span class="line"> root = tree.getroot()</span><br><span class="line"> size = root.find(<span class="string">'size'</span>) <span class="comment"># 图片的shape值</span></span><br><span class="line"> w = <span class="built_in">int</span>(size.find(<span class="string">'width'</span>).text)</span><br><span class="line"> h = <span class="built_in">int</span>(size.find(<span class="string">'height'</span>).text)</span><br><span class="line"> <span class="keyword">for</span> obj <span class="keyword">in</span> root.<span class="built_in">iter</span>(<span class="string">'object'</span>):</span><br><span class="line"> cls_id = <span class="number">0</span></span><br><span class="line"> <span class="comment"># 获取整个bounding box框</span></span><br><span class="line"> bndbox = obj.find(<span class="string">'bndbox'</span>)</span><br><span class="line"> <span class="comment"># xml给出的是x1, y1, x2, y2</span></span><br><span class="line"> box = [<span class="built_in">float</span>(bndbox.find(<span class="string">'xmin'</span>).text), <span class="built_in">float</span>(bndbox.find(<span class="string">'ymin'</span>).text), <span class="built_in">float</span>(bndbox.find(<span class="string">'xmax'</span>).text),</span><br><span class="line"> <span class="built_in">float</span>(bndbox.find(<span class="string">'ymax'</span>).text)]</span><br><span class="line"> <span class="comment"># 将x1, y1, x2, y2转换成yolov5所需要的x_center, y_center, w, h格式</span></span><br><span class="line"> bbox = xyxy2xywh((w, h), box)</span><br><span class="line"> <span class="comment"># 写入目标文件中，格式为 id x y w h</span></span><br><span class="line"> out_file.write(<span class="built_in">str</span>(cls_id) + <span class="string">" "</span> + <span class="string">" "</span>.join(<span class="built_in">str</span>(x) <span class="keyword">for</span> x <span class="keyword">in</span> bbox) + <span class="string">'\n'</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line"><span class="comment"># 这里要改成自己数据集路径的格式</span></span><br><span class="line"> path = <span class="string">'Annotations/'</span></span><br><span class="line"> voc2yolo(path)</span><br><span class="line"></span><br></pre></td></tr></table></figure><p>完成转换后每一张图片都有一个对应的txt文件</p><h2 id="训练前的准备"><a href="#训练前的准备" class="headerlink" title="训练前的准备"></a>训练前的准备</h2><p>新建medicine_data文件夹,<strong>文件结构：</strong></p><p>—-medicine_data</p><p> —-images</p><p> —-labels</p><p> —-medicine.yaml</p><p><strong>images即为而训练集，labels文件夹下为训练集的标签</strong></p><p><strong>其中medicine.yaml:</strong></p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">train:</span> <span class="string">D:/medicine_data/images/</span></span><br><span class="line"><span class="attr">val:</span> <span class="string">D:/medicine_data/images/</span></span><br><span class="line"></span><br><span class="line"><span class="attr">nc:</span> <span class="number">1</span></span><br><span class="line"></span><br><span class="line"><span class="attr">names:</span> [<span class="string">'m'</span>]</span><br></pre></td></tr></table></figure><p>打开yolov5-develop/models下的的yolov5s.yaml</p><p><strong>将nc一项的值改为1</strong></p><h2 id="开始训练"><a href="#开始训练" class="headerlink" title="开始训练"></a>开始训练</h2><p>在pycharm的Terminator中运行</p><p><code>python train.py --data D:/medicine_data/medicine.yaml --cfg ./models/yolov5s.yaml --weights ./weights/yolov5s.pt --batch-size 8 --epochs 450</code></p><p><strong>这里的batch-size根据显存大小适当调整，总共训练450个epoch</strong></p><p><strong>训练开始后可能出现“页面太小”的报错，这时候要进入高级系统设置下 的性能调整分给D盘的虚拟内存大小</strong></p><p>正在训练</p><p><img src="https://z3.ax1x.com/2021/06/05/2tgzJx.jpg" alt=""></p><p><strong>如果想要训练过程可视化，需要先安装wandb</strong></p><p><code>pip install wandb</code></p><p>大概训练了十三的小时后，得到了训练好的最后一次模型和最佳模型</p><h2 id="训练结果"><a href="#训练结果" class="headerlink" title="训练结果"></a>训练结果</h2><p>训练后的模型保存在runs/train目录下面</p><p><img src="https://z3.ax1x.com/2021/06/05/2tWULF.jpg" alt=""></p><h3 id="检查训练效果"><a href="#检查训练效果" class="headerlink" title="检查训练效果"></a>检查训练效果</h3><p><code>python detect.py --source data/images/medicine5.jpg --weights ./runs/train/exp38/weights/best.pt --conf 0.4</code></p><p><img src="https://z3.ax1x.com/2021/06/05/2tRNEd.jpg" alt=""></p><p><img src="https://z3.ax1x.com/2021/06/05/2tRYHH.jpg" alt=""></p><p> <strong>可以看到大部分在表面的中药软包装都检测出来了并且置信度高，少部分被覆盖的没有检测出来，检测效果良好</strong></p><h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p><strong>使用yolov5进行中药软包装的最终检测效果良好，但是本项目也存在明显的缺陷：原始样本数太少，虽然可以数据增广但是许多数据相比原始数据仍没有很大的变换，如果一开始准备大量数据然后进行几倍的增广可能到最后的训练和检测结果会和现在有较大出入</strong></p>]]></content>
<categories>
<category> 机器学习 </category>
</categories>
<tags>
<tag> 目标检测 </tag>
<tag> YOLOv5 </tag>
<tag> pytorch </tag>
</tags>
</entry>
<entry>
<title>各种BUG</title>
<link href="2021/06/05/%E5%90%84%E7%A7%8DBUG/"/>
<url>2021/06/05/%E5%90%84%E7%A7%8DBUG/</url>
<content type="html"><![CDATA[<h2 id="1-hexo-d部署出现SSL-connect-SSL-ERROR-SYSCALL-in-connection-to-github-com-443"><a href="#1-hexo-d部署出现SSL-connect-SSL-ERROR-SYSCALL-in-connection-to-github-com-443" class="headerlink" title="1.hexo d部署出现SSL_connect: SSL_ERROR_SYSCALL in connection to github.com:443"></a>1.hexo d部署出现SSL_connect: SSL_ERROR_SYSCALL in connection to github.com:443</h2><p>解决：</p><p><img src="https://z3.ax1x.com/2021/06/05/2tT5B6.jpg" alt=""></p><p><strong><em>重点方法二！！！！</em></strong></p><h2 id="2-ubuntu16-04已经连接wifi仍无法上网"><a href="#2-ubuntu16-04已经连接wifi仍无法上网" class="headerlink" title="2.ubuntu16.04已经连接wifi仍无法上网"></a>2.ubuntu16.04已经连接wifi仍无法上网</h2><p><strong>问题：</strong></p><p><img src="https://z3.ax1x.com/2021/06/05/2NQxsg.png" alt=""></p><p><strong>解决</strong>：</p><p><img src="https://z3.ax1x.com/2021/06/05/2NQvQS.png" alt=""></p><p><strong>修改网络代理为自动即可</strong></p><h2 id="3-ROS运行python节点报错"><a href="#3-ROS运行python节点报错" class="headerlink" title="3.ROS运行python节点报错"></a>3.ROS运行python节点报错</h2><p><img src="https://z3.ax1x.com/2021/06/14/2TNiIs.png" alt=""></p>]]></content>
<categories>
<category> BUG </category>
</categories>
</entry>
<entry>
<title>SVM+HOG实现行人检测</title>
<link href="2021/06/01/SVM%20HOG%E8%A1%8C%E4%BA%BA%E6%A3%80%E6%B5%8B/"/>
<url>2021/06/01/SVM%20HOG%E8%A1%8C%E4%BA%BA%E6%A3%80%E6%B5%8B/</url>
<content type="html"><![CDATA[<h2 id="运行环境："><a href="#运行环境：" class="headerlink" title="运行环境："></a>运行环境：</h2><p>windows10+anaconda+python3.6+opencv3.4.1+pycharm</p><h2 id="数据集准备"><a href="#数据集准备" class="headerlink" title="数据集准备"></a>数据集准备</h2><h3 id="1-数据集一：正样本924张，负样本2120张，验证集179张，大小均为64-128像素"><a href="#1-数据集一：正样本924张，负样本2120张，验证集179张，大小均为64-128像素" class="headerlink" title="1.数据集一：正样本924张，负样本2120张，验证集179张，大小均为64*128像素"></a>1.数据集一：正样本924张，负样本2120张，验证集179张，大小均为64*128像素</h3><p>网盘链接：<a href="">https://pan.baidu.com/s/1EEfVVQKUAkEFWD2Si-v-4A</a></p><p>提取码：1234</p><p><img src="https://z3.ax1x.com/2021/06/01/2uiBi4.jpg" style="zoom:80%;" /></p><h3 id="2-数据集二：提取了INRIA-Person-Dataset中的正样本和负样本，因为尺寸不合适，需要进行缩放和剪裁："><a href="#2-数据集二：提取了INRIA-Person-Dataset中的正样本和负样本，因为尺寸不合适，需要进行缩放和剪裁：" class="headerlink" title="2.数据集二：提取了INRIA Person Dataset中的正样本和负样本，因为尺寸不合适，需要进行缩放和剪裁："></a>2.数据集二：提取了INRIA Person Dataset中的正样本和负样本，因为尺寸不合适，需要进行缩放和剪裁：</h3><p>网盘链接：<a href="">https://pan.baidu.com/s/1UplbM5N2_KgNYxX63vL6fQ</a></p><p>提取码：1234</p><p><img src="https://z3.ax1x.com/2021/06/01/2uiDJJ.jpg" style="zoom:80%;" /></p><p>图像的缩放和剪裁</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> PIL <span class="keyword">import</span> Image</span><br><span class="line"><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> plt</span><br><span class="line"><span class="keyword">import</span> os</span><br><span class="line"></span><br><span class="line"><span class="comment"># 定义待批量裁剪图像的路径地址</span></span><br><span class="line">IMAGE_INPUT_PATH = <span class="string">'Positiv'</span></span><br><span class="line"><span class="comment"># 定义裁剪后的图像存放地址</span></span><br><span class="line">IMAGE_OUTPUT_PATH = <span class="string">'Positive'</span></span><br><span class="line"><span class="comment"># 定义裁剪图片左、上、右、下的像素坐标</span></span><br><span class="line">BOX_LEFT, BOX_UP, BOX_RIGHT, BOX_DOWN = <span class="number">0</span>, <span class="number">0</span>,<span class="number">64</span> , <span class="number">128</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> each_image <span class="keyword">in</span> os.listdir(IMAGE_INPUT_PATH):</span><br><span class="line"> <span class="comment"># 每个图像全路径</span></span><br><span class="line"> image_input_fullname = IMAGE_INPUT_PATH + <span class="string">'/'</span> + each_image</span><br><span class="line"> <span class="comment"># PIL库打开每一张图像</span></span><br><span class="line"> img = Image.<span class="built_in">open</span>(image_input_fullname)</span><br><span class="line"> plt.figure(<span class="string">"image_input_fullname"</span>)</span><br><span class="line"> plt.subplot(<span class="number">1</span>, <span class="number">2</span>, <span class="number">1</span>)</span><br><span class="line"> plt.imshow(img)</span><br><span class="line"> plt.axis(<span class="string">'off'</span>)</span><br><span class="line"> <span class="built_in">print</span>(img.<span class="built_in">format</span>, img.size, img.mode)</span><br><span class="line"> <span class="comment"># 从原始图像返回一个矩形区域，区域是一个4元组定义左上右下像素坐标</span></span><br><span class="line"> box = (BOX_LEFT, BOX_UP, BOX_RIGHT + BOX_LEFT, BOX_DOWN + BOX_UP)</span><br><span class="line"> <span class="comment"># 进行roi裁剪</span></span><br><span class="line"> roi_area = img.crop(box)</span><br><span class="line"></span><br><span class="line"> <span class="built_in">print</span>(roi_area.<span class="built_in">format</span>, roi_area.size, roi_area.mode)</span><br><span class="line"> <span class="comment"># 裁剪后每个图像的路径+名称</span></span><br><span class="line"> image_output_fullname = IMAGE_OUTPUT_PATH + <span class="string">"/"</span> + each_image</span><br><span class="line"> <span class="comment"># 存储裁剪得到的图像</span></span><br><span class="line"> roi_area.save(image_output_fullname)</span><br><span class="line"> <span class="built_in">print</span>(<span class="string">'{0} crop done.'</span>.<span class="built_in">format</span>(each_image))</span><br></pre></td></tr></table></figure><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> PIL</span><br><span class="line"><span class="keyword">from</span> PIL <span class="keyword">import</span> Image</span><br><span class="line"><span class="keyword">import</span> os.path</span><br><span class="line"><span class="keyword">import</span> glob</span><br><span class="line"><span class="comment">#将原图像裁剪到64*128像素</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">convertjpg</span>(<span class="params">jpgfile,outdir,width=<span class="number">64</span>,height=<span class="number">128</span></span>):</span></span><br><span class="line"> img=Image.<span class="built_in">open</span>(jpgfile)</span><br><span class="line"> <span class="keyword">try</span>:</span><br><span class="line"> new_img=img.resize((width,height),Image.BILINEAR)</span><br><span class="line"> new_img.save(os.path.join(outdir,os.path.basename(jpgfile)))</span><br><span class="line"> <span class="keyword">except</span> Exception <span class="keyword">as</span> e:</span><br><span class="line"> <span class="built_in">print</span>(e)</span><br><span class="line"><span class="comment">#将该文件夹下的所有png格式统一进行裁剪</span></span><br><span class="line"><span class="keyword">for</span> jpgfile <span class="keyword">in</span> glob.glob(<span class="string">'Negativ/*.png'</span>):</span><br><span class="line"> <span class="comment">#输出文件夹</span></span><br><span class="line"> convertjpg(jpgfile,<span class="string">'Negative'</span>)</span><br></pre></td></tr></table></figure><h2 id="HOG特征提取"><a href="#HOG特征提取" class="headerlink" title="HOG特征提取"></a>HOG特征提取</h2><p> Dalal提出的Hog特征提取的过程：把样本图像分割为若干个像素的单元（cell），把梯度方向平均划分为9个区间（bin），在每个单元里面对所有像素的梯度方向在各个方向区间进行直方图统计，得到一个9维的特征向量，每相邻的4个单元构成一个块（block），把一个块内的特征向量联起来得到36维的特征向量，用块对样本图像进行扫描，扫描步长为一个单元。最后将所有块的特征串联起来，就得到了人体的特征。例如，对于64<em>128的图像而言，每8</em>8的像素组成一个cell，每2<em>2个cell组成一个块，因为每个cell有9个特征，所以每个块内有4</em>9=36个特征，以8个像素为步长，那么，水平方向将有7个扫描窗口，垂直方向将有15个扫描窗口。也就是说，64<em>128的图片，总共有36</em>7*15=3780个特征。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">extract_hog</span>(<span class="params">samples, logger</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 从训练数据集中提取HOG特征，并返回</span></span><br><span class="line"><span class="string"> :param samples: 训练数据集</span></span><br><span class="line"><span class="string"> :param logger: 日志信息打印模块</span></span><br><span class="line"><span class="string"> :return train: 从训练数据集中提取的HOG特征</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> train = []</span><br><span class="line"> logger.info(<span class="string">'Extracting HOG Descriptors...'</span>)</span><br><span class="line"> num = <span class="number">0.</span></span><br><span class="line"> total = <span class="built_in">len</span>(samples)</span><br><span class="line"> <span class="keyword">for</span> f <span class="keyword">in</span> samples:</span><br><span class="line"> num += <span class="number">1.</span></span><br><span class="line"> logger.info(<span class="string">'Processing {} {:2.1f}%'</span>.<span class="built_in">format</span>(f, num/total*<span class="number">100</span>))</span><br><span class="line"> hog = cv2.HOGDescriptor((<span class="number">64</span>,<span class="number">128</span>), (<span class="number">16</span>,<span class="number">16</span>), (<span class="number">8</span>,<span class="number">8</span>), (<span class="number">8</span>,<span class="number">8</span>), <span class="number">9</span>)</span><br><span class="line"> <span class="comment"># hog = cv2.HOGDescriptor()</span></span><br><span class="line"> img = cv2.imread(f, <span class="number">1</span>)</span><br><span class="line"> img = cv2.resize(img, (<span class="number">64</span>,<span class="number">128</span>))</span><br><span class="line"> descriptors = hog.compute(img)</span><br><span class="line"> logger.info(<span class="string">'hog feature descriptor size: {}'</span>.<span class="built_in">format</span>(descriptors.shape)) <span class="comment"># (3780, 1)</span></span><br><span class="line"> train.append(descriptors)</span><br><span class="line"></span><br><span class="line"> train = np.float32(train)</span><br><span class="line"> train = np.resize(train, (total, <span class="number">3780</span>))</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> train</span><br></pre></td></tr></table></figure><h2 id="构建SVM分类器并开始训练"><a href="#构建SVM分类器并开始训练" class="headerlink" title="构建SVM分类器并开始训练"></a>构建SVM分类器并开始训练</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">train_svm</span>(<span class="params">train, labels, logger</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 训练SVM分类器</span></span><br><span class="line"><span class="string"> :param train: 训练数据集</span></span><br><span class="line"><span class="string"> :param labels: 对应训练集的标签</span></span><br><span class="line"><span class="string"> :param logger: 日志信息打印模块</span></span><br><span class="line"><span class="string"> :return: SVM检测器（注意：opencv的hogdescriptor中的svm不能直接用opencv的svm模型，而是要导出对应格式的数组）</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> logger.info(<span class="string">'Configuring SVM classifier.'</span>)</span><br><span class="line"> svm = cv2.ml.SVM_create()</span><br><span class="line"> svm.setCoef0(<span class="number">0.0</span>)</span><br><span class="line"> svm.setDegree(<span class="number">3</span>)</span><br><span class="line"> criteria = (cv2.TERM_CRITERIA_MAX_ITER + cv2.TERM_CRITERIA_EPS, <span class="number">1000</span>, <span class="number">1e-3</span>)</span><br><span class="line"> svm.setTermCriteria(criteria)</span><br><span class="line"> svm.setGamma(<span class="number">0</span>)</span><br><span class="line"> svm.setKernel(cv2.ml.SVM_LINEAR)</span><br><span class="line"> svm.setNu(<span class="number">0.5</span>)</span><br><span class="line"> svm.setP(<span class="number">0.1</span>) <span class="comment"># for EPSILON_SVR, epsilon in loss function?</span></span><br><span class="line"> svm.setC(<span class="number">0.01</span>) <span class="comment"># From paper, soft classifier</span></span><br><span class="line"> svm.setType(cv2.ml.SVM_EPS_SVR)</span><br><span class="line"></span><br><span class="line"> logger.info(<span class="string">'Starting training svm.'</span>)</span><br><span class="line"> svm.train(train, cv2.ml.ROW_SAMPLE, labels)</span><br><span class="line"> logger.info(<span class="string">'Training done.'</span>)</span><br><span class="line"></span><br><span class="line"> pwd = os.getcwd()</span><br><span class="line"> model_path = os.path.join(pwd, <span class="string">'svm.xml'</span>)</span><br><span class="line"> svm.save(model_path)</span><br><span class="line"> logger.info(<span class="string">'Trained SVM classifier is saved as: {}'</span>.<span class="built_in">format</span>(model_path))</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> get_svm_detector(svm)</span><br><span class="line"></span><br></pre></td></tr></table></figure><p>训练过程：训练过程很短，五秒左右便能结束</p><p><img src="https://z3.ax1x.com/2021/06/01/2uZrWD.jpg" alt=""></p><h2 id="利用训练好的SVM进行检测"><a href="#利用训练好的SVM进行检测" class="headerlink" title="利用训练好的SVM进行检测"></a>利用训练好的SVM进行检测</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">test_hog_detect</span>(<span class="params">test, svm_detector, logger</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 导入测试集，测试结果</span></span><br><span class="line"><span class="string"> :param test: 测试数据集</span></span><br><span class="line"><span class="string"> :param svm_detector: 用于HOGDescriptor的SVM检测器</span></span><br><span class="line"><span class="string"> :param logger: 日志信息打印模块</span></span><br><span class="line"><span class="string"> :return: 无</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> hog = cv2.HOGDescriptor()</span><br><span class="line"> hog.setSVMDetector(svm_detector)</span><br><span class="line"> <span class="comment"># opencv自带的训练好了的分类器</span></span><br><span class="line"> <span class="comment">#hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())</span></span><br><span class="line"> pwd = os.getcwd()</span><br><span class="line"> test_dir = os.path.join(pwd, <span class="string">'TestData'</span>)</span><br><span class="line"> cv2.namedWindow(<span class="string">'Detect'</span>)</span><br><span class="line"> <span class="keyword">for</span> f <span class="keyword">in</span> test:</span><br><span class="line"> file_path = os.path.join(test_dir, f)</span><br><span class="line"> logger.info(<span class="string">'Processing {}'</span>.<span class="built_in">format</span>(file_path))</span><br><span class="line"> img = cv2.imread(file_path)</span><br><span class="line"> rects, _ = hog.detectMultiScale(img, winStride=(<span class="number">4</span>,<span class="number">4</span>), padding=(<span class="number">8</span>,<span class="number">8</span>), scale=<span class="number">1.05</span>)</span><br><span class="line"> <span class="keyword">for</span> (x,y,w,h) <span class="keyword">in</span> rects:</span><br><span class="line"> cv2.rectangle(img, (x,y), (x+w,y+h), (<span class="number">0</span>,<span class="number">0</span>,<span class="number">255</span>), <span class="number">2</span>)</span><br><span class="line"> cv2.imshow(<span class="string">'Detect'</span>, img)</span><br><span class="line"> c = cv2.waitKey(<span class="number">0</span>) & <span class="number">0xff</span></span><br><span class="line"> <span class="keyword">if</span> c == <span class="number">27</span>:</span><br><span class="line"> <span class="keyword">break</span></span><br><span class="line"> cv2.destroyAllWindows()</span><br></pre></td></tr></table></figure><h2 id="检测结果"><a href="#检测结果" class="headerlink" title="检测结果"></a>检测结果</h2><h3 id="数据集一："><a href="#数据集一：" class="headerlink" title="数据集一："></a>数据集一：</h3><p><img src="https://z3.ax1x.com/2021/06/01/2uek01.jpg" alt=""></p><p>在测试集上效果相对较好，人之间距离较近时可能会有重复框</p><h3 id="数据集二："><a href="#数据集二：" class="headerlink" title="数据集二："></a>数据集二：</h3><p><img src="https://z3.ax1x.com/2021/06/01/2ueFmR.jpg" alt=""></p><p>效果很差，经分析可能与正样本数过大以及人在正样本图片中所占范围不够大有关系</p><h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p>可是尝试通过增多负样本来提高样本多样性，同时在特征提取之前可以通过图像增强和灰度化有效地降低图像局部的阴影和光照变化,相比之下Opencv已经训练好的SVM分类器效果更佳，后面也可以尝试采用深度神经网络来实现更高精度的检测</p><h2 id="源代码："><a href="#源代码：" class="headerlink" title="源代码："></a>源代码：</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> os</span><br><span class="line"><span class="keyword">import</span> sys</span><br><span class="line"><span class="keyword">import</span> cv2</span><br><span class="line"><span class="keyword">import</span> logging</span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">logger_init</span>():</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 自定义python的日志信息打印配置</span></span><br><span class="line"><span class="string"> :return logger: 日志信息打印模块</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 获取logger实例，如果参数为空则返回root logger</span></span><br><span class="line"> logger = logging.getLogger(<span class="string">"PedestranDetect"</span>)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 指定logger输出格式</span></span><br><span class="line"> formatter = logging.Formatter(<span class="string">'%(asctime)s %(levelname)-8s: %(message)s'</span>)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 文件日志</span></span><br><span class="line"> <span class="comment"># file_handler = logging.FileHandler("test.log")</span></span><br><span class="line"> <span class="comment"># file_handler.setFormatter(formatter) # 可以通过setFormatter指定输出格式</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 控制台日志</span></span><br><span class="line"> console_handler = logging.StreamHandler(sys.stdout)</span><br><span class="line"> console_handler.formatter = formatter <span class="comment"># 也可以直接给formatter赋值</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 为logger添加的日志处理器</span></span><br><span class="line"> <span class="comment"># logger.addHandler(file_handler)</span></span><br><span class="line"> logger.addHandler(console_handler)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 指定日志的最低输出级别，默认为WARN级别</span></span><br><span class="line"> logger.setLevel(logging.INFO)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> logger</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">load_data_set</span>(<span class="params">logger</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 导入数据集</span></span><br><span class="line"><span class="string"> :param logger: 日志信息打印模块</span></span><br><span class="line"><span class="string"> :return pos: 正样本文件名的列表</span></span><br><span class="line"><span class="string"> :return neg: 负样本文件名的列表</span></span><br><span class="line"><span class="string"> :return test: 测试数据集文件名的列表。</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> logger.info(<span class="string">'Checking data path!'</span>)</span><br><span class="line"> pwd = os.getcwd()</span><br><span class="line"> logger.info(<span class="string">'Current path is:{}'</span>.<span class="built_in">format</span>(pwd))</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 提取正样本</span></span><br><span class="line"> pos_dir = os.path.join(pwd, <span class="string">'Positive'</span>)</span><br><span class="line"> <span class="keyword">if</span> os.path.exists(pos_dir):</span><br><span class="line"> logger.info(<span class="string">'Positive data path is:{}'</span>.<span class="built_in">format</span>(pos_dir))</span><br><span class="line"> pos = os.listdir(pos_dir)</span><br><span class="line"> logger.info(<span class="string">'Positive samples number:{}'</span>.<span class="built_in">format</span>(<span class="built_in">len</span>(pos)))</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 提取负样本</span></span><br><span class="line"> neg_dir = os.path.join(pwd, <span class="string">'Negative'</span>)</span><br><span class="line"> <span class="keyword">if</span> os.path.exists(neg_dir):</span><br><span class="line"> logger.info(<span class="string">'Negative data path is:{}'</span>.<span class="built_in">format</span>(neg_dir))</span><br><span class="line"> neg = os.listdir(neg_dir)</span><br><span class="line"> logger.info(<span class="string">'Negative samples number:{}'</span>.<span class="built_in">format</span>(<span class="built_in">len</span>(neg)))</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 提取测试集</span></span><br><span class="line"> test_dir = os.path.join(pwd, <span class="string">'TestData'</span>)</span><br><span class="line"> <span class="keyword">if</span> os.path.exists(test_dir):</span><br><span class="line"> logger.info(<span class="string">'Test data path is:{}'</span>.<span class="built_in">format</span>(test_dir))</span><br><span class="line"> test = os.listdir(test_dir)</span><br><span class="line"> logger.info(<span class="string">'Test samples number:{}'</span>.<span class="built_in">format</span>(<span class="built_in">len</span>(test)))</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> pos, neg, test</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">load_train_samples</span>(<span class="params">pos, neg</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 合并正样本pos和负样本pos，创建训练数据集和对应的标签集</span></span><br><span class="line"><span class="string"> :param pos: 正样本文件名列表</span></span><br><span class="line"><span class="string"> :param neg: 负样本文件名列表</span></span><br><span class="line"><span class="string"> :return samples: 合并后的训练样本文件名列表</span></span><br><span class="line"><span class="string"> :return labels: 对应训练样本的标签列表</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> pwd = os.getcwd()</span><br><span class="line"> pos_dir = os.path.join(pwd, <span class="string">'Positive'</span>)</span><br><span class="line"> neg_dir = os.path.join(pwd, <span class="string">'Negative'</span>)</span><br><span class="line"></span><br><span class="line"> samples = []</span><br><span class="line"> labels = []</span><br><span class="line"> <span class="keyword">for</span> f <span class="keyword">in</span> pos:</span><br><span class="line"> file_path = os.path.join(pos_dir, f)</span><br><span class="line"> <span class="keyword">if</span> os.path.exists(file_path):</span><br><span class="line"> samples.append(file_path)</span><br><span class="line"> labels.append(<span class="number">1.</span>)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> f <span class="keyword">in</span> neg:</span><br><span class="line"> file_path = os.path.join(neg_dir, f)</span><br><span class="line"> <span class="keyword">if</span> os.path.exists(file_path):</span><br><span class="line"> samples.append(file_path)</span><br><span class="line"> labels.append(-<span class="number">1.</span>)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># labels 要转换成numpy数组，类型为np.int32</span></span><br><span class="line"> labels = np.int32(labels)</span><br><span class="line"> labels_len = <span class="built_in">len</span>(pos) + <span class="built_in">len</span>(neg)</span><br><span class="line"> labels = np.resize(labels, (labels_len, <span class="number">1</span>))</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> samples, labels</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">extract_hog</span>(<span class="params">samples, logger</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 从训练数据集中提取HOG特征，并返回</span></span><br><span class="line"><span class="string"> :param samples: 训练数据集</span></span><br><span class="line"><span class="string"> :param logger: 日志信息打印模块</span></span><br><span class="line"><span class="string"> :return train: 从训练数据集中提取的HOG特征</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> train = []</span><br><span class="line"> logger.info(<span class="string">'Extracting HOG Descriptors...'</span>)</span><br><span class="line"> num = <span class="number">0.</span></span><br><span class="line"> total = <span class="built_in">len</span>(samples)</span><br><span class="line"> <span class="keyword">for</span> f <span class="keyword">in</span> samples:</span><br><span class="line"> num += <span class="number">1.</span></span><br><span class="line"> logger.info(<span class="string">'Processing {} {:2.1f}%'</span>.<span class="built_in">format</span>(f, num/total*<span class="number">100</span>))</span><br><span class="line"> hog = cv2.HOGDescriptor((<span class="number">64</span>,<span class="number">128</span>), (<span class="number">16</span>,<span class="number">16</span>), (<span class="number">8</span>,<span class="number">8</span>), (<span class="number">8</span>,<span class="number">8</span>), <span class="number">9</span>)</span><br><span class="line"> <span class="comment"># hog = cv2.HOGDescriptor()</span></span><br><span class="line"> img = cv2.imread(f, <span class="number">1</span>)</span><br><span class="line"> img = cv2.resize(img, (<span class="number">64</span>,<span class="number">128</span>))</span><br><span class="line"> descriptors = hog.compute(img)</span><br><span class="line"> logger.info(<span class="string">'hog feature descriptor size: {}'</span>.<span class="built_in">format</span>(descriptors.shape)) <span class="comment"># (3780, 1)</span></span><br><span class="line"> train.append(descriptors)</span><br><span class="line"></span><br><span class="line"> train = np.float32(train)</span><br><span class="line"> train = np.resize(train, (total, <span class="number">3780</span>))</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> train</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">get_svm_detector</span>(<span class="params">svm</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 导出可以用于cv2.HOGDescriptor()的SVM检测器，实质上是训练好的SVM的支持向量和rho参数组成的列表</span></span><br><span class="line"><span class="string"> :param svm: 训练好的SVM分类器</span></span><br><span class="line"><span class="string"> :return: SVM的支持向量和rho参数组成的列表，可用作cv2.HOGDescriptor()的SVM检测器</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> sv = svm.getSupportVectors()</span><br><span class="line"> rho, _, _ = svm.getDecisionFunction(<span class="number">0</span>)</span><br><span class="line"> sv = np.transpose(sv)</span><br><span class="line"> <span class="keyword">return</span> np.append(sv, [[-rho]], <span class="number">0</span>)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">train_svm</span>(<span class="params">train, labels, logger</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 训练SVM分类器</span></span><br><span class="line"><span class="string"> :param train: 训练数据集</span></span><br><span class="line"><span class="string"> :param labels: 对应训练集的标签</span></span><br><span class="line"><span class="string"> :param logger: 日志信息打印模块</span></span><br><span class="line"><span class="string"> :return: SVM检测器（注意：opencv的hogdescriptor中的svm不能直接用opencv的svm模型，而是要导出对应格式的数组）</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> logger.info(<span class="string">'Configuring SVM classifier.'</span>)</span><br><span class="line"> svm = cv2.ml.SVM_create()</span><br><span class="line"> svm.setCoef0(<span class="number">0.0</span>)</span><br><span class="line"> svm.setDegree(<span class="number">3</span>)</span><br><span class="line"> criteria = (cv2.TERM_CRITERIA_MAX_ITER + cv2.TERM_CRITERIA_EPS, <span class="number">1000</span>, <span class="number">1e-3</span>)</span><br><span class="line"> svm.setTermCriteria(criteria)</span><br><span class="line"> svm.setGamma(<span class="number">0</span>)</span><br><span class="line"> svm.setKernel(cv2.ml.SVM_LINEAR)</span><br><span class="line"> svm.setNu(<span class="number">0.5</span>)</span><br><span class="line"> svm.setP(<span class="number">0.1</span>) <span class="comment"># for EPSILON_SVR, epsilon in loss function?</span></span><br><span class="line"> svm.setC(<span class="number">0.01</span>) <span class="comment"># From paper, soft classifier</span></span><br><span class="line"> svm.setType(cv2.ml.SVM_EPS_SVR)</span><br><span class="line"></span><br><span class="line"> logger.info(<span class="string">'Starting training svm.'</span>)</span><br><span class="line"> svm.train(train, cv2.ml.ROW_SAMPLE, labels)</span><br><span class="line"> logger.info(<span class="string">'Training done.'</span>)</span><br><span class="line"></span><br><span class="line"> pwd = os.getcwd()</span><br><span class="line"> model_path = os.path.join(pwd, <span class="string">'svm.xml'</span>)</span><br><span class="line"> svm.save(model_path)</span><br><span class="line"> logger.info(<span class="string">'Trained SVM classifier is saved as: {}'</span>.<span class="built_in">format</span>(model_path))</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> get_svm_detector(svm)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">test_hog_detect</span>(<span class="params">test, svm_detector, logger</span>):</span></span><br><span class="line"> <span class="string">'''</span></span><br><span class="line"><span class="string"> 导入测试集，测试结果</span></span><br><span class="line"><span class="string"> :param test: 测试数据集</span></span><br><span class="line"><span class="string"> :param svm_detector: 用于HOGDescriptor的SVM检测器</span></span><br><span class="line"><span class="string"> :param logger: 日志信息打印模块</span></span><br><span class="line"><span class="string"> :return: 无</span></span><br><span class="line"><span class="string"> '''</span></span><br><span class="line"> hog = cv2.HOGDescriptor()</span><br><span class="line"> hog.setSVMDetector(svm_detector)</span><br><span class="line"> <span class="comment"># opencv自带的训练好了的分类器</span></span><br><span class="line"> <span class="comment">#hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())</span></span><br><span class="line"> pwd = os.getcwd()</span><br><span class="line"> test_dir = os.path.join(pwd, <span class="string">'TestData'</span>)</span><br><span class="line"> cv2.namedWindow(<span class="string">'Detect'</span>)</span><br><span class="line"> <span class="keyword">for</span> f <span class="keyword">in</span> test:</span><br><span class="line"> file_path = os.path.join(test_dir, f)</span><br><span class="line"> logger.info(<span class="string">'Processing {}'</span>.<span class="built_in">format</span>(file_path))</span><br><span class="line"> img = cv2.imread(file_path)</span><br><span class="line"> rects, _ = hog.detectMultiScale(img, winStride=(<span class="number">4</span>,<span class="number">4</span>), padding=(<span class="number">8</span>,<span class="number">8</span>), scale=<span class="number">1.05</span>)</span><br><span class="line"> <span class="keyword">for</span> (x,y,w,h) <span class="keyword">in</span> rects:</span><br><span class="line"> cv2.rectangle(img, (x,y), (x+w,y+h), (<span class="number">0</span>,<span class="number">0</span>,<span class="number">255</span>), <span class="number">2</span>)</span><br><span class="line"> cv2.imshow(<span class="string">'Detect'</span>, img)</span><br><span class="line"> c = cv2.waitKey(<span class="number">0</span>) & <span class="number">0xff</span></span><br><span class="line"> <span class="keyword">if</span> c == <span class="number">27</span>:</span><br><span class="line"> <span class="keyword">break</span></span><br><span class="line"> cv2.destroyAllWindows()</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</span><br><span class="line"> logger = logger_init()</span><br><span class="line"> pos, neg, test = load_data_set(logger=logger)</span><br><span class="line"> samples, labels = load_train_samples(pos, neg)</span><br><span class="line"> train = extract_hog(samples, logger=logger)</span><br><span class="line"> logger.info(<span class="string">'Size of feature vectors of samples: {}'</span>.<span class="built_in">format</span>(train.shape))</span><br><span class="line"> logger.info(<span class="string">'Size of labels of samples: {}'</span>.<span class="built_in">format</span>(labels.shape))</span><br><span class="line"> svm_detector = train_svm(train, labels, logger=logger)</span><br><span class="line"> test_hog_detect(test, svm_detector, logger)</span><br><span class="line"></span><br></pre></td></tr></table></figure>]]></content>
<categories>
<category> 机器学习 </category>
</categories>
<tags>
<tag> SVM </tag>
<tag> HOG </tag>
<tag> 特征提取 </tag>
<tag> 目标检测 </tag>
</tags>
</entry>
<entry>
<title>Hello World</title>
<link href="2021/05/27/hello-world/"/>
<url>2021/05/27/hello-world/</url>
<content type="html"><![CDATA[<p>Welcome to <a href="https://hexo.io/">Hexo</a>! This is your very first post. Check <a href="https://hexo.io/docs/">documentation</a> for more info. If you get any problems when using Hexo, you can find the answer in <a href="https://hexo.io/docs/troubleshooting.html">troubleshooting</a> or you can ask me on <a href="https://github.com/hexojs/hexo/issues">GitHub</a>.</p><h2 id="Quick-Start"><a href="#Quick-Start" class="headerlink" title="Quick Start"></a>Quick Start</h2><h3 id="Create-a-new-post"><a href="#Create-a-new-post" class="headerlink" title="Create a new post"></a>Create a new post</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo new <span class="string">"My New Post"</span></span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/writing.html">Writing</a></p><h3 id="Run-server"><a href="#Run-server" class="headerlink" title="Run server"></a>Run server</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo server</span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/server.html">Server</a></p><h3 id="Generate-static-files"><a href="#Generate-static-files" class="headerlink" title="Generate static files"></a>Generate static files</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo generate</span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/generating.html">Generating</a></p><h3 id="Deploy-to-remote-sites"><a href="#Deploy-to-remote-sites" class="headerlink" title="Deploy to remote sites"></a>Deploy to remote sites</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo deploy</span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/one-command-deployment.html">Deployment</a></p>]]></content>
</entry>
</search>