Improve full-text searching by expanding reference primary

[prowns]

Problem Description:
All records (YPM, AAT, Wikidata, GBIF) have the same primary name: Equus Caballus. Most users would search "horse". The issue is raised by art museums, who use Horse as subject heading, and include this in their data under "represents". Discussion surfaced an inconsistency in how simple search is working.

Question: how is simple search working? It is not consistent for this use case.

• boxley horse retrieves the right records
• boxley sheep does NOT retrieve the right record (this one) but it should as Ovis is listed in Depicts.

Expected Behavior/Solution:
Simple search should work such that boxley sheep would retrieve the correct record, even though the primary name is Ovis.

Change simple search to use referenceName instead of referencePrimaryName

Make the same change for the Name field in Advanced Search.

More detailed description of changes:

1. The related documents portion of the keyword search pattern switched from the referencePrimaryName field to the referenceName field. By search scope, here are the record types that included or excluded from reference fields:
   • Included:
     • Agent: Group, Person
     • Concept: Language, MeasurementUnit, Type
     • Event: Activity, Period
     • Place: Place
   • Excluded:
     • Concept: Currency, Material
     • Item: DigitalObject, HumanMadeObject
     • Work: LinguisticObject, Set, VisualItem
2. All name search terms less the set search scope switched from their search scope-specific primary name field to their broader name field; for instance, the name search term in the agent scope was changed from agentPrimaryName to agentName.

Requirements:

• New Simple Search index name will be referencePrimaryName
• Advanced Search name field will use the extended index in Simple Search (referencePrimaryName)

Needed for promotion:
If an item on the list is not needed, it should be crossed off but not removed.

• Wireframe/Mockup - Mike
• Committee discussions - Sarah
• Feasibility/Team discussion - Sarah
• Performance test
• Backend requirements - TBD
• Frontend requirements- TBD
• Questions
• List of questions for discussions. Answers should be documented within the issue.

UAT/LUX Examples:

The searches below should show this record in the results: https://lux.collections.yale.edu/view/visual/b50fb93f-dc1d-4e49-a3b8-acc1cfd4a174

• boxley horse
• boxley sheep

Dependencies/Blocks:

• Blocked By: Issues that are blocking the completion of the current issue.
• Blocking: Issues being blocked by the completion of the current issue.

Related Github Issues:

• Issues that contain similar work but are not blocking or being blocked by the current issue.

Related links:

• Additional context in 2/29/24 data wg notes
• Additional context in 4/25 Interactions notes
• https://www.bugherd.com/projects/284041/tasks/2188

Wireframe/Mockup:
Place wireframe/mockup for the proposed solution at end of ticket.

[prowns]

Horse is a primary name in Equus Caballus, but not labelled as english. This is why it appears, but Ovis does not have this. @clarkepeterf will try using the referencename index to in DEV so it can be reviewed. This include anything in identifedBy with type of name.

[clarkepeterf]

Preliminary deployment in DEV looks good. Propose to also do the following:

• change LUX JSON name searches to use the [scope]Name fields instead of [scope]PrimaryName.
• add a new LUX JSON primaryName field in each scope which uses [scope]PrimaryName

[prowns]

Per Interactions 4/25

• New Simple Search name index is great; will continue testing
• Advanced Search
• • Name search should used this extended index as well
• • No need for a separate “primary” name option in Adv. Search

[prowns]

Performance test needed before this is deployed.

[brent-hartwig]

@clarkepeterf, #11 (comment) captures related comparisons.

[prowns]

Performance tests on 5/9 failed; still determining if issue is with this issue or ML.

[roamye]

IT meeting 6/21:
Performance test 6/7 results are successful. This ticket is no longer blocked and will be deployed in 6/24 cycle.

[roamye]

Approved by UAT

[roamye]

Looks good in prod! Closing.