Make some urls with whitespace acceptable to JavaScript extractor.

[vonrosen]

No description provided.

[vonrosen]

Make some urls with whitespace acceptable to JavaScript extractor. Also, allow UTF-16 forward-slash to be used as path separator for use-case at https://agency.wisconsin.gov/sites/prb/SitePages/Home.aspx where many urls have the format: "\u002fsites\u002fprb\u002fPublic Comment Emails PDF\u002fOpen records of electronic records_1r71vt0k.pdf"

[nlevitt]

Thanks Hunter.

• ExtractorJS.considerString() calls StringEscapeUtils.unescapeJavaScript() -- it would be best if handling of "\u002f" happened somewhere around here -- it looks like StringEscapeUtils.unescapeJavaScript() already unescapes "\u002f" et al, so you can probably just get rid of your code for that
• looks like you have a stray close-parenthesis here: if (TextUtils.matches(".*[\\s)]+.*", candidate)) { (also, the square brackets are superfluous)
• could you add some tests to ExtractorJSTest?