6.12.1

6.12.1 - 2022-08-11

Bugs

• "Show Only Rated" feature in Quepid busted for ES. Highlighting on _id after a terms match in ES causes a index out of bounds exception. Also, the API method in explainOther needed to have the right case, might make a constants file at some point. o19s/splainer-search#109 by @worleydl fixed the issue in Splainer. Bumped in Quepid via #547 by @epugh.

6.12.0 - 2022-08-09

including 6.12.0 in this release notes as Eric forgot to tag it.

Features

• Search box (to find queries). #506 and #512 by @epugh fixes #493 by @theaaronposc. Thanks Aaron!

• You can now generate a new Case through the thor command line tools, useful when setting up lots of cases or as part of a script. #533 by @epugh.

• Surface how many unrated documents you have for a query as an alert bubble. #532 by @epugh.

• The Frog Pond Report tells you about the state of your ratings. Understand the distribution of queries that need ratings and decide if you need to "hop to it" ;-). #526 by @epugh.

• Create missing queries when importing ratings. #509 by @epugh.

Improvements

• Gray out Queries with no ratings. Previously they showed up as Red with a 0.0 score, the same as if everything had bad ratings! #516 by @worleydl and suggested by @renekrie.

• Keycloak Version 17 changed the default url to not have the /auth in the namespaces. #536 by @epugh fixes #528. We now test Quepid with Keycloak 18.

• Average Precision Scorer AP@10 wasn't implemented following the most common definition of AP. #535 by @david-fisher fixes this! Thanks David!

• Taking a Snapshot is intensive process that makes a unique query to the search engine. #513 by @epugh provides nicer error message.

• Remove some more unused AngularJS code. #529 by @epugh.

• 'Information need' import doesn't works for an empty case. #508 by @epugh fixes #507 by @atarora.

• Bump to latest Rails to pick up CVE fixes, Ruby 2.7.6 recommended by Heroku, Bundler 2.3.18, plus general updates to gem dependencies.

Bugs

• The detailed export of the case doesn't exports the 0 documents matching queries. #504 by @atarora fixes #501.

• Move the logic for the default ES and Solr server definitions from server AND client side to just client side. #505 by @epugh fixes #503.

• Limited encoding renders snapshots unusable. #510 by @epugh fixes #499 by @KennyLindahl. Thanks Kenny for the analysis!

• Fixed missing query fields from cloning a case. #520 by @epugh

6.11.0

6.11.0 - 2022-04-23

Features

• Documenting the explicit Information Need of the user is a core part of running human judgements, and historically people used the Notes field for this. Now we have a Information Need attribute for every Query, and you can import/export that data via CSV, as well as edit it in the UI. Mouse over the query to see exactly what the user is trying to do! #480 by @epugh.

• The details of a specific query exposed by the Explain Query modal is in a Human readable format, but isn't in a JSON compliant format. Now we sort the keys alphabetically, which helps the user dig through the details, and have a Copy as JSON button so you can use other tools to look at the JSON information. #497 by
  @epugh fixes #492.

• Delete All Queries for a Case was requested by @negyesi in #475. At the case level added a Delete button that brings up a modal that lets you delete the case, archive the case, or delete all the queries. #486 by @epugh.

• Frustrated that your custom API that looks like Solr to Quepid has to deal with JSONP callbacks? Now you don't, as we have introduced the ability to swap from the default JSONP to GET api methods when you define your Solr end point. This is going to simplify the work that folks have to do. #495 by @epugh. For more information refer to the docs at https://github.com/o19s/quepid/blob/master/docs/endpoints_solr.md#solr-endpoints-structure.

• Do you have a relative image url in your search engine that you want to show via thumb: or image:? You can now provide some JSON in the field specification to provide that data: id,title:title,{"name": "image_url", "type":"image", "prefix": "https://www.example.org/images"}, description. This feature is experimental as we learn more about how to have richer field specifications, and may change! #487 by @epugh.

Improvements

• The link to view a single Solr document specifies XML, which I realize is a very OK boomer approach these days! o19s/splainer-search#107 by @begomeister fixes #494 by swapping to JSON format instead.

• github.com/teleborder/FileSaver.js repo disappeared, so migrate to an actual published node module file-saver instead. #498 by @epugh.

• We had a mish-mash of query and query_text as headers in our CSV files, so we've standardized on query to make export/import simpler. #489 by @epugh.

• We discovered that Redis was ONLY used in Quepid to support sending events to Google Analytics! If your QUEPID_GA isn't specified, then Redis is no longer required. Thanks to @rbednarzCBI for updating the production docker settings to remove Redis in #488.

Bugs

• The various CSV upload and export features didn't really handle , in the text properly, so we gave that a good going over. #482 by @atarora fixed by #483 and #485 by @epugh

• Tracking the history of your tries, we had a limit in the database that prevented you from having lots of tries. #478 by @KennyLindahl and #470 by @korlowski. Fixed by #479 by @epugh. Thanks @KennyLindahl for the additional testing work you did to help me fix this!

• Fixed the ability to peek at the next page of result when you have less then 10 results left. Thanks @anegyesi for submitting #473 and fixed in #496.

• Noticed the case level graph isn't useful? PR #491 by @worleydl, Save scores less often, fixes stretching of graph, fixes this.

• @thePanz opened a ticket a year ago (#330) about ratings getting progressively slower in the UI, and a huge amount of updates to the backend. Thanks to @worleydl and #490 this should be fixed.

6.10.1

6.10.1 - 2022-02-28

Bugs

• Updated Splainer-Search to https://github.com/o19s/splainer-search/releases/tag/v2.13.0 to enhance snapshots with ES, and make our parsing of ES.

• When Quepid switches between HTTP and HTTPS protocols, the flash message "Reloaded on HTTP protocol for Quepid app to match search engine URL" causes some CSS layout issues which led to the "Run my Queries" button to slide off the page! Darn you CSS. First spotted by @nathancday, while giving a demo of course! Fixed by #463 by @epugh.

• When reloading from HTTP to HTTPS Elasticsearch during the case wizard, we lose the default query params for ES, we end up
  with the default Solr query params! #457 by @epugh and @wrigleyDan. Fixed by #464 by @epugh.

• Build was failing for Apple Silicon M1 users! #430 by @llermaly and fixed by @wrigleyDan via #460.

v6.10.0

6.10.0 - 2022-01-03

We've taken a rag and polished up the Find and Rate Missing Documents modal box in this release. We've also completely
redone how we attempted to handle http and https connections from Quepid to http and https search engines that was first
introduced into 6.9.0 based on what we've learned in the real world.

Improvements

• Reworked the handing of http and https connections to Quepid when connecting to http and https search engines. Added in Development mode a nginx based proxy to make http://localhost/ and https://localhost/ work, which is also how you would deploy Quepid in a Production set up. Still works fine on http://localhost:3000 if you only use http based search engine connection. Big thanks to @jzonthemtn for #451 which fixes #444 and #438.

• The Find and Rate Missing Documents query interface assumes you use the Lucene query syntax. Turns out we have a highlighter built in, so enable that for Lucene syntax. #453 by @epugh.

• Writing your own scorer? The modal popup window is rather cramped, so let's give the editor room to breathe by making them larger! #452 by @epugh.

• In the early days of Quepid, looking up a single Solr doc would trigger a facet.field on every field you display in Quepid. For some Solr's, this can turn a quick lookup for a single document to a 30 second or more ordeal because of unexpected faceting! Thanks to @jeffryedvm for opening #442. Fixed by #456 by @epugh.

Bugs

• On the Find and Rate Missing Documents screen the ability to show just the rated documents had an issue that you had to click the button twice, making you think it was broken. #454 and #423 by @epugh are fixed in #455 by @epugh.

v6.9.1

6.9.1 - 2021-10-27

We forgot one improvement to splainer-search, so we are rolling another release!

Improvements

• Splainer-search 2.11.0 handles nulls the same as empty arrays ({ }) in Solr output. Nulls might happen when a Proxy is used. #440 by @epugh fixes issue discovered by @mkr in o19s/splainer-search#97.

• Node 15 is not a LTS version, which leads to a warning message! #437 by @epugh bumps this.

v6.9.0

6.9.0 - 2021-10-27 - Spooky HTTPS Support!

Features

• Chrome 94 forces you to be on https URL to access HTTPS secured search engines. Quepid now checks to see whether it is running on the same protocol, HTTPS or HTTP that the search engine is running on, and if not, prompts you to reload the app on the same protocol as the search engine. This required updating the messenging in the Case Wizard and the Tune Relevance screens. We also simplified the logic in the home_controller.rb around bootstrapping a Case and Try. Thanks to @tboeghk for pairing with @epugh on how to solve this problem. #431 by @epugh fixes #426 by @DmitryKey and #432 by @bbirrell.

Improvements

• Remove remnants of "SecureApp" angular1 app that was refactored a while ago into regular Rails app for login screens. #434 by @epugh fixes #433.

• Logic to enable Google Analytics tracking was crufty and caused classloader issues in development mode. #419 by @epugh fixes #418.

v6.8.0

6.8.0 - 2021-09-28

Features

• Understand the evolution of your various Tries by visualizing the history as a tree data structure. Understand where you went back to an earlier Try and branched off in new directions. #412 by @epugh fixes #412 by @DmitryKey.

• Deploying Quepid in a cloud native way requires having a healthcheck end point to confirm Quepid is available and happy. #404 by @epugh fixes #404 by @mkr. Thanks to @xaviersanchez for the inspiration to do this.

• Now sort queries by most recently modified. #334 by @DmitryKey and @epugh fixes #307.

• Excited to share the improvements you made to your Case? You can now drop your link to the Case into Slack or other messaging tools, and it will unfurl, i.e show the Case Name and the most recent Try number. You will still need to log into Quepid and have access to that Case. #417 by @epugh fixes #416.

Improvements

• The messaging around how many documents are being scored, i.e @10 or @40 is now driven by the variable k defined in various Scorers. Commit 73a378f by @epugh fixes #413 by @DmitryKey.

• We audited our Docker builds, and _shrank the images from 2.42GB to 1.44GB! Bumping Node version to 15, be smarter about what folders we copy into our production image, and some general fixups. #409 by @dworley and @epugh.

• The logic to check if Google Analytics is enabled via ENV variable is different then the normal Quepid ENV handling. #419 by @epugh cleaned this up, and solves a class loading issue in the DEV mode Quepid, increasing developer happiness!

• Trying to find a doc that SHOULD match and be rated, but can't find it? Using a global scorer like NDCG and curious what all the ratings are? We have reworked the Explain Missing Documents UI into a Find and Rate Missing Documents that lets you see all your ratings associated with a query, and then search and find other documents that are missing. Preview video here https://share.descript.com/view/IkxWkjSzzKF. #407 by @epugh fixes #406 by @mkr.

v6.7.0

6.7.0 - 2021-08-26

Features

• Elasticsearch fields computed using script query now work! Back in February @CGamesPlay contributed a PR (o19s/splainer-search#90) to splainer-search to support this, and now it automagically works in Quepid! Thanks @DmitryKey for opening #327 and supporting this feature.

• Elasticsearch templates can now be used in Quepid! #400 by @epugh exposes the underlying splainer-search functionality in Quepid. Thanks @danielibars for opening #274 and supporting this feature.

• Curious what parameters are being used by Solr to construct the query? We're now adding a echoParams=all to queries and surfacing them in the Explain Query modal at the query level in Quepid. Currently a Solr only feature. #397 by @epugh with inspiration from @mkr.

Improvements

• Continue cleanup of JavaScript, eliminated last remnants of "unit test" style scorers, and renamed customScorer to just scorer in front end app.

v6.6.1

6.6.1 - 2021-08-11

• In testing the Keycloak oAuth integration via the Chorus project, identified that the name of the Client that is passed to Keycloak was tied to the KEYCLOAK_REALM env setting. Changed this to be hardcoded to quepid so it can be different then the Realm name (i.e the Realm is Chorus in the Chorus project, and the Client is quepid). Commmit de9e0e6 by @epugh.

v6.6.0

6.6.0 - 2021-08-09

Features

• OpenID Support! Historically you had to create your own account on Quepid, but that is another barrier to entry, and something else to keep track of. Inspired by the issue Add OAuth Providers by @gregoryduckworth #188, #280 and #389 by @epugh implements the first two providers, Google and Keycloak.

• Explain Query Parsing! We often want to know what the search engine is doing to our query. Now you can click the "Explain Query" popup and see what is happening to your query and the various filter queries. This works with both Solr and Elasticsearch. https://twitter.com/dep4b/status/1420480501710827520. #391 by @epugh.

• Frequently type deftype instead of defType in Solr? I found an old, but broken feature that provided a popup warning on common Solr query syntax errors, which has now been restored. #394 by @epugh fixes #390.

Improvements

• Quepid is running on Rails 6! Rails 6 was released ~26 months ago, and represents the future of Rails. I'm excited that this push initiated by @DmitryKey is going to bring us some great new features like: better developer experience with Webpack for JavaScript, ActionText to handle better text formatting of notes and messages about Cases and Queries, ActionCable which will let us notify all the users of the same case that data has changed when it changes. #381 by @DmitryKey with assist from @epugh.

• Make our ActiveRecord modeling for ownership the same. Teams have an owner, Scorers have an owner, but Cases have a user. Now we have case.owner relationship. #359 by @epugh.

• The nav bar link for Scorers is labeled "Custom Scorers", and takes you to an page called /advanced. Both of the names are rather offputting. Creating Scorers is a normal common thing to do in Quepid, so lets label it Scorers. #392 by @epugh fixes #351.

• We had, as someone put it "dinosaur" age CSS tags for mozilla and webkit browsers that triggered lots of browser warnings in the console. #380 by @epugh cleaned up many of them. It has also prompted some thinking about migrating to Bootstrap 5 for the non core pages. If anyone has some CSS skills, please contact @epugh, to help refine the header menu in Bootstrap 5 CSS.