-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
323 lines (291 loc) · 13.1 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="description"
content="HSTforU: Anomaly Detection in Aerial and Ground-based Videos with Hierarchical Spatio-Temporal Transformer for U-net">
<meta name="keywords" content="PySTformer, pystformer">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>HSTforU: Anomaly Detection in Aerial and Ground-based Videos with Hierarchical Spatio-Temporal Transformer for U-net</title>
<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-ECQJZZNYSB"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-ECQJZZNYSB');
</script>
<link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro"
rel="stylesheet">
<link rel="stylesheet" href="./static/css/bulma.min.css">
<link rel="stylesheet" href="./static/css/bulma-carousel.min.css">
<link rel="stylesheet" href="./static/css/bulma-slider.min.css">
<link rel="stylesheet" href="./static/css/fontawesome.all.min.css">
<link rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css">
<link rel="stylesheet" href="./static/css/index.css">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script>
<script defer src="./static/js/fontawesome.all.min.js"></script>
<script src="./static/js/bulma-carousel.min.js"></script>
<script src="./static/js/bulma-slider.min.js"></script>
<script src="./static/js/index.js"></script>
</head>
<body>
<section class="hero">
<div class="hero-body">
<div class="container is-max-desktop">
<div class="columns is-centered">
<div class="column has-text-centered">
<h1 class="title is-1 publication-title">HSTforU: Anomaly Detection in Aerial and Ground-based Videos with Hierarchical Spatio-Temporal Transformer for U-net</h1>
<div class="is-size-5 publication-authors">
<span class="author-block">
<a href="https://vt-le.github.io/">Viet-Tuan Le</a><sup>1</sup>,
</span>
<span class="author-block">
<a href=#>Hulin Jin</a><sup>2</sup>,
</span>
<span class="author-block">
<a href="http://home.sejong.ac.kr/~ykim/">Yong-Guk Kim</a><sup>1</sup>
</span>
</div>
<div class="is-size-5 publication-authors">
<span class="author-block"><sup>1</sup>Department of Computer Science and Engineering, Sejong University, Seoul, Korea,</span>
</div>
<div class="is-size-5 publication-authors">
<span class="author-block"><sup>2</sup>School of Big Data and Statistics, Anhui University, Hefei, China</span>
</div>
<div class="column has-text-centered">
<div class="publication-links">
<!-- PDF Link. -->
<span class="link-block">
<a href="https://arxiv.org/"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fas fa-file-pdf"></i>
</span>
<span>Paper (coming soon ...)</span>
</a>
</span>
<!-- Video Link. -->
<span class="link-block">
<a href="https://youtu.be/s5wmFbDnBCI"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-youtube"></i>
</span>
<span>Video</span>
</a>
</span>
<!-- Code Link. -->
<span class="link-block">
<a href="https://github.com/vt-le/HSTforU"
class="external-link button is-normal is-rounded is-dark">
<span class="icon">
<i class="fab fa-github"></i>
</span>
<span>Code (with password)</span>
</a>
</span>
</a>
</div>
</div>
</div>
</div>
</div>
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<!-- Abstract. -->
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Abstract</h2>
<div class="content has-text-justified">
<p>
Anomaly detection is to identify abnormal events against normal ones within surveillance videos mainly collected in ground-based settings. Recently, the need for drone data processing is rapidly growing as drones find new applications. However, as most aerial videos collected by flying drones contain moving backgrounds and others, it is necessary to deal with their spatio-temporal features in detecting anomalies. This study presents a transformer-based video anomaly detection method whereby we investigate a challenging aerial dataset and three benchmark ground-based datasets. The encoder of our U-net has a four-stage pyramid transformer structure, and each stage has a link to a corresponding spatio-temporal transformer stage. Then, this transformer produces hierarchical feature maps that are conveyed to the decoder as skip connections. Extensive evaluations including several ablation studies suggest that this network outperforms the state-of-the-art on Drone-anomaly dataset and three benchmark datasets. We expect the proposed transformer for U-net can find diverse applications in the video processing area. Code and model are available at <a href="https://vt-le.github.io/HSTforU/">https://vt-le.github.io/HSTforU/</a>.
</p>
</div>
</div>
</div>
<!--/ Abstract. -->
<!-- Paper video. -->
<div class="columns is-centered has-text-centered">
<div class="column is-four-fifths">
<h2 class="title is-3">Video</h2>
<div class="publication-video">
<iframe width="560" height="315" src="https://www.youtube.com/embed/s5wmFbDnBCI" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
</div>
</div>
</div>
<!--/ Paper video. -->
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<!-- Animation. -->
<div class="columns is-centered">
<div class="column is-full-width">
<h3 class="title is-3">Ground-based Videos</h3>
<h4 class="title is-4">CUHK Avenue Dataset</h4>
<p>
The future frame (middle column) and prediction error (right column) are generated corresponding with the input frame.
</p>
<br>
<!-- Interpolating. -->
<div class="columns is-vcentered interpolation-panel">
<div class="column is-3 has-text-centered">
<img src="static/images/560.png"
class="interpolation-image"
alt="Interpolate start reference image."/>
<p>Start Frame</p>
</div>
<div class="column interpolation-video-column">
<div id="interpolation-image-wrapper">
Loading...
</div>
<input class="slider is-fullwidth is-large is-info"
id="interpolation-slider"
step="1" min="0" max="241" value="0" type="range">
</div>
<div class="column is-3 has-text-centered">
<img src="static/images/800.png"
class="interpolation-image"
alt="Interpolation end reference image."/>
<p class="is-bold">End Frame</p>
</div>
</div>
</div>
</div>
<!--/ Animation. -->
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<h3 class="title is-3">Aerial Videos</h3>
<div class="columns is-centered">
<!-- Visual Effects. -->
<div class="column">
<div class="content">
<h4 class="title is-4">Highway</h4>
<p>
In this scene, a cow herd walking on the highway is anomalous.
</p>
<video id="dollyzoom" autoplay controls muted loop playsinline height="100%">
<source src="./static/videos/highway.mp4"
type="video/mp4">
</video>
</div>
</div>
<!--/ Visual Effects. -->
<!-- Matting. -->
<div class="column">
<h2 class="title is-4">Bike</h2>
<div class="columns is-centered">
<div class="column content">
<p>
Vehicles moving on the roundabout used for bikes is anomalous in this scene.
</p>
<video id="matting-video" autoplay controls muted loop playsinline height="100%">
<source src="./static/videos/bike.mp4"
type="video/mp4">
</video>
</div>
</div>
</div>
</div>
<!--/ Matting. -->
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<!-- Concurrent Work. -->
<div class="columns is-centered">
<div class="column is-full-width">
<h2 class="title is-3">Anomaly score</h2>
<div class="content has-text-justified">
<p>
For each video, we compare the anomaly score of the proposed method with and without Hierarchical Spatio-temporal Transfomrer.
</p>
<h4 class="title is-4">Ground-based Datasets</h4>
<div class="content has-text-centered">
<img src="static/images/fig_exp_abnormal_score_ped2.png"
class="interpolation-image"
alt="Interpolation end reference image."/>
<p class="is-bold"> UCSD Ped2 Dataset</p>
</div>
<div class="content has-text-centered">
<img src="static/images/fig_exp_abnormal_score_avenue.png"
class="interpolation-image"
alt="Interpolation end reference image."/>
<p class="is-bold">CUHK Avenue Dataset</p>
</div>
<div class="content has-text-centered">
<img src="static/images/fig_exp_abnormal_score_shanghai.png"
class="interpolation-image"
alt="Interpolation end reference image."/>
<p class="is-bold">ShanghaiTech Dataset</p>
</div>
<h4 class="title is-4">Drone-anomaly Dataset</h4>
<div class="content has-text-centered">
<img src="static/images/fig_exp_abnormal_score_bike.png"
class="interpolation-image"
alt="Interpolation end reference image."/>
<p class="is-bold"> Bike roundabout</p>
</div>
<div class="content has-text-centered">
<img src="static/images/fig_exp_abnormal_score_highway.png"
class="interpolation-image"
alt="Interpolation end reference image."/>
<p class="is-bold">Highway</p>
</div>
</div>
</div>
</div>
<!--/ Concurrent Work. -->
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<!-- Animation. -->
<div class="columns is-centered">
<div class="column is-full-width">
<h2 class="title is-3">Related Works</h2>
<ul>
<li>
<a href="https://moguprediction.github.io/">MoGuP:Motion-guided Prediction for Video Anomaly Detection</a>
</li>
<li>
<a href="https://vt-le.github.io/astnet/">ASTNet: Attention-based Residual Autoencoder for Video Anomaly Detection</a>
</li>
</ul>
</div>
</div>
</div>
</section>
<section class="section">
<div class="container is-max-desktop">
<!-- Animation. -->
<div class="columns is-centered">
<div class="column is-full-width">
<h2 class="title is-3">Acknowledgements</h2>
<p>
This work was supported by the Institute of Information \& communications Technology Planning \& Evaluation (IITP), a grant funded by the Korean government (MSIT) (No.2019-0-00231), and by the Information Technology Research Center (ITRC) support program (IITP-2022-RS-2022-00156354) as well as by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2020R1A6A1A03038540).
</p>
</div>
</div>
</div>
</section>
<section class="section" id="BibTeX">
<div class="container is-max-desktop content">
<h2 class="title">BibTeX</h2>
<pre><code>@article{le2023hstforu,
author = {Le, Viet-Tuan and Jin, Hulin and Kim, Yong-Guk},
title = {HSTforU: Anomaly Detection in Aerial and Ground-based Videos with Hierarchical Spatio-Temporal Transformer for U-net},
}</code></pre>
</div>
</section>
<div class="content has-text-centered">
<script type='text/javascript' id='clustrmaps' src='//cdn.clustrmaps.com/map_v2.js?cl=ffffff&w=100&t=tt&d=q49wzaeszt6saUdZ2-PmswLp2WGKt0cVRzic2a8avqc'></script>
</div>
</body>
</html>