-
Notifications
You must be signed in to change notification settings - Fork 0
/
rr-prize-mlsp-2014.html
266 lines (187 loc) · 20.4 KB
/
rr-prize-mlsp-2014.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" dir="ltr">
<!-- Mirrored from soundsoftware.ac.uk/rr-prize-mlsp-2014 by HTTrack Website Copier/3.x [XR&CO'2014], Tue, 01 Aug 2017 18:33:32 GMT -->
<!-- Added by HTTrack --><meta http-equiv="content-type" content="text/html;charset=utf-8" /><!-- /Added by HTTrack -->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>MLSP 2014 SoundSoftware.ac.uk Prizes for Reproducibility in Signal Processing | Sound Software .ac.uk</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link rel="shortcut icon" href="sites/soundsoftware.ac.uk/files/favicon.png" type="image/x-icon" />
<link type="text/css" rel="stylesheet" media="all" href="sites/soundsoftware.ac.uk/files/css/css_96672be50d2b2bf1d283b8d9324e7be5.css" />
<link type="text/css" rel="stylesheet" media="print" href="sites/soundsoftware.ac.uk/files/css/css_9cb387b73479bd590fedb6ec230dc035.css" />
<!--[if IE]>
<link type="text/css" rel="stylesheet" media="all" href="/sites/all/themes/zen/zen/ie.css?j" />
<![endif]-->
<script type="text/javascript" src="sites/soundsoftware.ac.uk/files/js/js_5be44a4bd1f33d1988c9f73fe4b7f463.js"></script>
<script type="text/javascript">
<!--//--><![CDATA[//><!--
jQuery.extend(Drupal.settings, { "basePath": "/", "googleanalytics": { "trackOutbound": 1, "trackMailto": 1, "trackDownload": 1, "trackDownloadExtensions": "7z|aac|arc|arj|asf|asx|avi|bin|csv|doc(x|m)?|dot(x|m)?|exe|flv|gif|gz|gzip|hqx|jar|jpe?g|js|mp(2|3|4|e?g)|mov(ie)?|msi|msp|pdf|phps|png|ppt(x|m)?|pot(x|m)?|pps(x|m)?|ppam|sld(x|m)?|thmx|qtm?|ra(m|r)?|sea|sit|tar|tgz|torrent|txt|wav|wma|wmv|wpd|xls(x|m|b)?|xlt(x|m)|xlam|xml|z|zip" } });
//--><!]]>
</script>
<script type="text/javascript">
<!--//--><![CDATA[//><!--
(function(i,s,o,g,r,a,m){i["GoogleAnalyticsObject"]=r;i[r]=i[r]||function(){(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)})(window,document,"script","http://www.google-analytics.com/analytics.js","ga");ga("create", "UA-18698611-1", { "cookieDomain": "auto" });ga("set", "anonymizeIp", true);ga("send", "pageview");
//--><!]]>
</script>
<link rel="alternate" type="application/rss+xml" title="Subscribe" href="rss.xml" />
<meta name="google-site-verification" content="gKmyKIK_OFaFd5f07E7wIMUM59VMmUIZc7xa3qKgFEo" />
</head>
<body class="not-front not-logged-in node-type-story two-sidebars page-rr-prize-mlsp-2014 section-rr-prize-mlsp-2014">
<div id="page"><div id="page-inner">
<a name="navigation-top" id="navigation-top"></a>
<div id="skip-to-nav"><a href="#navigation">Skip to Navigation</a></div>
<div id="header"><div id="header-inner" class="clear-block">
<div id="logo-title">
<a href="index.html" title="Home" rel="home">
<div id="site-name"><strong>
Sound Software .ac.uk </strong></div>
</a>
</div> <!-- /#logo-title -->
</div></div> <!-- /#header-inner, /#header -->
<div id="main"><div id="main-inner" class="clear-block with-navbar">
<div id="content"><div id="content-inner">
<div id="content-header">
<h1 class="title">MLSP 2014 SoundSoftware.ac.uk Prizes for Reproducibility in Signal Processing</h1>
</div> <!-- /#content-header -->
<div id="content-area">
<div id="node-144" class="node node-type-story"><div class="node-inner">
<div class="content">
<p>The SoundSoftware Project, in collaboration with the IEEE Signal Processing Society, will be sponsoring a Prize for Reproducibility in Signal Processing for work published at the <a href="http://mlsp2014.conwiz.dk/" target="_blank">IEEE International Workshop on Machine Learning for Signal Processing</a> (MLSP 2014).</p>
<p>This edition of the Reproducible Research prizes follows our earlier prize series in conjuction with the <a href="rr-prize-semantic-audio.html">Audio Engineering Society's 53rd Conference</a> and our <a href="rr-prize-apr2013.html">SoundSoftware.ac.uk Prizes for Reproducibility in Audio and Music Research</a>. </p>
<h2>Eligibility</h2>
<p>Authors of papers submitted to the <a href="http://mlsp2014.conwiz.dk/" target="blank">IEEE International Workshop on Machine Learning for Signal Processing</a> (MLSP 2014) may present their submitted papers for this edition of the SoundSoftware.ac.uk Prizes for Reproducibility. Please read below for details on how to <a href="#submission">submit your work</a>.</p>
<p>All researchers will be considered for this award, both from UK institutions eligible for EPSRC support and from other UK and international institutions.</p>
<h2>Important dates</h2>
<p><em>New</em> — Entry has been re-opened with a new deadline of Thursday, 31 July!</p>
<ul>
<li><em>Fri 16 May - Paper submission deadline for MLSP; Announcement of Prizes for Reproducibility</em></li>
<li><em><del>Sun 8 June</del></em> - <a href="#submission">Submission deadline</a> — Now re-opened! New deadline: <em>Thurs 31 July</em></li>
</ul>
<h2>Categories</h2>
<dl>
<dt><strong>Category A - Fully reproducible work</strong></dt>
<dd>Awarded for a paper whose results can be reproduced using the datasets, software and methodologies described in the paper;</dd>
<dt><strong>Category B - Reproducibility-enabling work</strong></dt>
<dd>Awarded for a paper presenting infrastructure, datasets, or standards intended to enable future reproducible work from other authors.</dd>
<p>Within both categories we will separately consider regular and student-led submissions. Participants will be expected to choose a category on submission.</p>
<h2>Prizes</h2>
<p>While the value of this prize is in its formal recognition of the quality and potential impact of your work, winning entrants will also receive a £100 Amazon voucher.</p>
<h2><a name="submission">How to enter</a></h2>
<p>To submit your application to this edition of the SoundSoftware.ac.uk Prizes for Reproducibility in Signal Processing, please complete the following form: <strong><a href="https://docs.google.com/forms/d/15eD6iEGfALfJvMkxzEA4FuHam44kURtqxm0cWoCLv0A/viewform" target="_blank">application form</a></strong>.</p>
<p>Please take note that the deadline for submissions is now <b>July 31, 2014</b>.</p>
<h2>Evaluation</h2>
<p>The entries will be evaluated by the SoundSoftware project. The evaluation panel will consider the following factors:</p>
<ul>
<li><strong>Ease of reproducibility of the results</strong>: a straightforward baseline replicability test in which we will evaluate whether each submission's published results can be regenerated using the software and data associated with the paper (where applicable).</li>
<li><strong>Quality of sustainability planning</strong>: each submission's sustainability (of associated software and data) will be evaluated based on the Software Sustainability Institute's sustainability evaluation criteria (see <a href="http://www.software.ac.uk/online-sustainability-evaluation" title="http://www.software.ac.uk/online-sustainability-evaluation">http://www.software.ac.uk/online-sustainability-evaluation</a>).</li>
<li><strong>Potential to enable high quality research in the UK signal processing research community</strong>: all submissions will be sent to (at least) two external reviewers in order to assess their potential to enable further high quality research in their field</li>
</ul>
<h3>Some advice on how to prepare your submission</h3>
<p>To give you the best chance of winning the prize, we suggest that you test both the sustainability and the reproducibility of your entry before submitting. </p>
<p>To test for sustainability, you can refer to the Software Sustainability Institute's Online Sustainability evaluation tool: <a href="http://www.software.ac.uk/online-sustainability-evaluation" title="http://www.software.ac.uk/online-sustainability-evaluation">http://www.software.ac.uk/online-sustainability-evaluation</a>. Of particular note are:</p>
<ul>
<li>whether the code and data are available in a suitable public repository;</li>
<li>whether version control is used;</li>
<li>which tools for community involvement are available;</li>
<li>whether the work includes a useful README file, copyright and licence information.</li>
</ul>
<p>To test for reproducibility, we suggest that you simply test your software on a different machine. You might, for instance, ask a colleague to test it for you, preferably without standing over them explaining what to do as they do it! Usually this is enough to detect many significant oversights, such as missing files, references to nonexistent paths (eg <tt>/home/myusername/data/</tt>) or even incomplete or (completely missing) instructions.</p>
<p>If for any reason you can't ask a colleague to test your software for you, we'd advise you to test it on a virtual machine. For example (if you are not using Matlab) try running your code within <a href="http://c4dm.eecs.qmul.ac.uk/rdr/handle/123456789/32">this Linux image</a>, which was originally provided for use by entrants to an IEEE AASP acoustic scene challenge.</p>
<p>Ideally, a submission would contain individual scripts that regenerate all results presented in your paper. We are aware that many results (either in the form of tables or plots) are generated a few hours before the submission deadline ends; writing scripts that generate the results on your paper at the same time you are writing the paper itself is usually the best option! Don't forget that the person most likely to want to reproduce your results is you, at a later date - so why not prepare for that from the start?</p>
<p>If you wish to know more about the previous edition's evaluation process, please see the following page: <a href="rr-prize-how.html" title="http://soundsoftware.ac.uk/rr-prize-how">http://soundsoftware.ac.uk/rr-prize-how</a>.</p>
<h3>Notes</h3>
<p>While we encourage fully open access and open source submissions, research that can reasonably be reproduced by other researchers using common non-open tools or data is also acceptable (e.g. using Matlab). However, do not assume that the testers will have access to special datasets in your particular field, e.g. image processing, bioinformatics, speech processing, etc. </p>
<p>If no submissions of suitable quality are received for a particular category, that prize may not be awarded.</p>
<p>For further advice or assistance contact the SoundSoftware project at <a href="mailto:info@soundsoftware.ac.uk">info@soundsoftware.ac.uk</a>.</p>
</dl> </div>
<ul class="links inline"><li class="comment_forbidden first last"></li>
</ul>
</div></div> <!-- /node-inner, /node -->
</div>
</div></div> <!-- /#content-inner, /#content -->
<div id="navbar"><div id="navbar-inner" class="clear-block region region-navbar">
<a name="navigation" id="navigation"></a>
<div id="primary" class="clear-block">
<!-- <ul class="links"><li class="menu-245 first"><a href="/resources" title="">Resources</a></li>
<li class="menu-117"><a href="/activities" title="Activities">Activities</a></li>
<li class="menu-116 last"><a href="/aboutus" title="">About</a></li>
</ul> -->
<ul class="menu"><li class="expanded first"><a href="resources.html" title="">Resources</a><ul class="menu"><li class="leaf first"><a href="tools.html" title="Tools and Facilities">Tools</a></li>
<li class="leaf"><a href="handouts-guides.html" title="Printable Handouts and Guides">Handouts/Guides</a></li>
<li class="leaf"><a href="videos.html" title="Videos and slide presentations">Videos/Slides</a></li>
<li class="leaf"><a href="programming-examples.html" title="">Code examples</a></li>
<li class="leaf last"><a href="archive.html" title="">Blog archive</a></li>
</ul></li>
<li class="expanded"><a href="activities.html" title="Activities">Activities</a><ul class="menu"><li class="leaf first"><a href="rr-prize.html" title="Reproducible Research Prizes">Reproducible Research Prizes</a></li>
<li class="leaf"><a href="soundsoftware2014.html" title="SoundSoftware 2014: Third Workshop on Software and Data for Audio and Music Research">SoundSoftware 2014</a></li>
<li class="leaf"><a href="soundsoftware2013.html" title="SoundSoftware 2013: Workshop on Software and Data for Audio and Music Research">SoundSoftware 2013</a></li>
<li class="leaf last"><a href="soundsoftware2012.html" title="SoundSoftware 2012: Workshop on Software and Data for Audio and Music Research">SoundSoftware 2012</a></li>
</ul></li>
<li class="expanded last"><a href="aboutus.html" title="">About</a><ul class="menu"><li class="leaf first"><a href="overview.html" title="Our aim">Our aim</a></li>
<li class="leaf"><a href="aboutus.html" title="">Who we are</a></li>
<li class="leaf last"><a href="contact.html" title="Contact us">Contact us</a></li>
</ul></li>
</ul> </div> <!-- /#primary -->
</div></div> <!-- /#navbar-inner, /#navbar -->
<div id="sidebar-left"><div id="sidebar-left-inner" class="region region-left">
<div id="block-block-9" class="block block-block region-odd odd region-count-1 count-1"><div class="block-inner">
<div class="content">
<p><a class="twitter-timeline" href="https://twitter.com/soundsoftwareuk" data-widget-id="368361225786126337" height="500" width="auto" data-chrome="nofooter noborders transparent" data-border-color="#3e442c" data-link-color="#be5600">Tweets by @soundsoftwareuk</a></p>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+"://platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script> </div>
</div></div> <!-- /block-inner, /block -->
<div id="block-views-recent_stories-block_1" class="block block-views region-even even region-count-2 count-2"><div class="block-inner">
<h2 class="title">Recent notes</h2>
<div class="content">
<div class="view view-recent-stories view-id-recent_stories view-display-id-block_1 recentnotes view-dom-id-efdd1910d77f168630244309daca950b">
<div class="view-content">
<div class="views-row views-row-1 views-row-odd views-row-first">
<div class="views-field views-field-title"> <span class="field-content"><a href="rr-prize-mlsp-2014-winners.html">MLSP Prizes for Reproducibility: Winners announced!</a></span> </div>
<div class="views-field views-field-teaser"> <div class="field-content"><p>Announcing the winners of the MLSP 2014 and SoundSoftware.ac.uk Prizes for Reproducibility in Signal Processing, organised by SoundSoftware.ac.uk in conjuction with the IEEE Signal Processing Society for the 2014 IEEE International Workshop on Machine Learning for Signal Processing.</p>
</div> </div> </div>
<div class="views-row views-row-2 views-row-even">
<div class="views-field views-field-title"> <span class="field-content"><a href="soundsoftware2014-videos-available.html">SoundSoftware 2014: Videos now available!</a></span> </div>
<div class="views-field views-field-teaser"> <div class="field-content"><p>The SoundSoftware 2014 workshop, our third annual workshop on software and data in audio and music research, was just as enjoyable as the previous two. Because so much research in this field ends up being expressed through software, a software workshop turns out to be all about the means by which research becomes useful and relevant to people other than the original researchers—fertile ground for interesting and thought-provoking talks.</p>
<p>The workshop videos are now available online at <a href="soundsoftware2014.html" title="http://soundsoftware.ac.uk/soundsoftware2014">http://soundsoftware.ac.uk/soundsoftware2014</a>, so if you weren't able to make it in person, catch up here!</p>
</div> </div> </div>
<div class="views-row views-row-3 views-row-odd views-row-last">
<div class="views-field views-field-title"> <span class="field-content"><a href="soundsoftware2014-registernow.html">Register now for the SoundSoftware Third Workshop!</a></span> </div>
<div class="views-field views-field-teaser"> <div class="field-content"><p>Our third annual one-day workshop on Software and Data for Audio and Music Research takes place on the 8th of July 2014 at Queen Mary, University of London. The workshop includes talks on issues such as robust software development for audio and music research, reproducible research in general, management of research data, and open access. <a href="soundsoftware2014.html">Read more here</a>, clear your calendar, and register now!</p>
</div> </div> </div>
</div>
</div> </div>
</div></div> <!-- /block-inner, /block -->
</div></div> <!-- /#sidebar-left-inner, /#sidebar-left -->
<div id="sidebar-right"><div id="sidebar-right-inner" class="region region-right">
<div id="block-search-0" class="block block-search region-odd odd region-count-1 count-3"><div class="block-inner">
<div class="content">
<form action="http://soundsoftware.ac.uk/rr-prize-mlsp-2014" accept-charset="UTF-8" method="post" id="search-block-form">
<div><div class="container-inline">
<div class="form-item" id="edit-search-block-form-1-wrapper">
<label for="edit-search-block-form-1">Search this site: </label>
<input type="text" maxlength="128" name="search_block_form" id="edit-search-block-form-1" size="15" value="" title="Enter the terms you wish to search for." class="form-text" />
</div>
<input type="submit" name="op" id="edit-submit" value="Search" class="form-submit" />
<input type="hidden" name="form_build_id" id="form-nsUc4UpRp72iqoNiC7A60ns4WwC5m2janwgLH6fnQo8" value="form-nsUc4UpRp72iqoNiC7A60ns4WwC5m2janwgLH6fnQo8" />
<input type="hidden" name="form_id" id="edit-search-block-form" value="search_block_form" />
</div>
</div></form>
</div>
</div></div> <!-- /block-inner, /block -->
<div id="block-block-8" class="block block-block region-even even region-count-2 count-4"><div class="block-inner">
<div class="content">
<p><a href="archive.html">Archive</a></p>
</div>
</div></div> <!-- /block-inner, /block -->
</div></div> <!-- /#sidebar-right-inner, /#sidebar-right -->
</div></div> <!-- /#main-inner, /#main -->
<div id="footer"><div id="footer-inner" class="region region-footer">
<div id="block-block-1" class="block block-block region-odd odd region-count-1 count-5"><div class="block-inner">
<div class="content">
<p><span class="f-left"><a rel="license" href="http://creativecommons.org/licenses/by-nc/3.0/" target="_blank"><img alt="Creative Commons Licence" style="margin-top: 2px; margin-right: 4px; border-width:0; float:left; " src="http://i.creativecommons.org/l/by-nc/3.0/88x31.png" /></a><span>This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-nc/3.0/">Creative Commons Attribution-NonCommercial 3.0 License</a>.<br /> © 2011 Queen Mary University of London </span></span><span class="f-right"><a rel="rss" href="rss.xml" target="_blank"><img alt="Subscribe to this site's RSS feed" style="margin-top: -2px; margin-right: 4px; border-width:0; float:left; " src="sites/all/themes/soundsoftware/socialnet_icons/rss_16.png" /></a><a rel="twitter" href="http://twitter.com/soundsoftwareuk" target="_blank"><img alt="Follow us on Twitter!" style="margin-top: -2px; margin-right: 4px; border-width:0; float:left; " src="sites/all/themes/soundsoftware/socialnet_icons/twitter_16.png" /></a><a rel="linkedin group" href="http://www.linkedin.com/groups?mostPopular=&gid=3472350" target="_blank"><img alt="Join or follow our LinkedIn group" style="margin-top: -2px; margin-right: 4px; border-width:0; float:left; " src="sites/all/themes/soundsoftware/socialnet_icons/linkedin_16.png" /></a></span></p>
</div>
</div></div> <!-- /block-inner, /block -->
</div></div> <!-- /#footer-inner, /#footer -->
</div></div> <!-- /#page-inner, /#page -->
</body>
<!-- Mirrored from soundsoftware.ac.uk/rr-prize-mlsp-2014 by HTTrack Website Copier/3.x [XR&CO'2014], Tue, 01 Aug 2017 18:33:36 GMT -->
</html>