Another try at fixing path prompting #844
Conversation
The example was just wrong (matching *.pdf) will never work unless there is a file with the name "/path/to/data/*.pdf". In my testing it caused openai to try matching properties.path.keyword = "*22*" when asked about item 22. Be more explicit in the prompt about using keyword for exact matches and including full paths.
| @@ -33,7 +33,7 @@ class QueryDatabase(LogicalOperator): | |||
| }, | |||
| { | |||
| "match": { | |||
| "properties.path.keyword": "/path/to/data/*.pdf", | |||
| "properties.path": "/path/to/data/*.pdf", | |||
There was a problem hiding this comment.
I'm confused by this. We want is to use the .keyword version of the field so the wildcard match will work. I realize more few-shots may be needed to get this dialed in but I don't think you want to take .keyword out here.
There was a problem hiding this comment.
Near as I can tell wildcard match doesn't work.
https://opensearch.org/docs/latest/field-types/supported-field-types/keyword/
says "A keyword field type contains a string that is not analyzed. It allows only exact, case-sensitive matches."
When LUNA tried *22* in a keyword match it got nothing back despite there being a path that matches that.
This change made path matching work properly on all the test cases I have for the customer. The two main patterns were match 1 file or match 2 files. Depending on the variant of the prompt it would work for one of those or the other. With this prompt it has worked for both in all cases I've tried.
There was a problem hiding this comment.
Oh, sorry. The correct syntax is:
{"query": {"wildcard": {"properties.path.keyword": "/path/to/data/*.pdf"}}}
That is, it needs the wildcard, not match operator, for this to work.
$ curl --insecure -u admin:admin -X GET "https://localhost:9200/const_ntsb/_search" -d '{"query": {"wildcard": {"properties.path.keyword": "s3://aryn-public/ntsb/*3*.pdf"}}}' -H 'Content-Type: application/json' | python -m json.tool | grep 'path'
"path": "s3://aryn-public/ntsb/73.pdf",
"path": "s3://aryn-public/ntsb/3.pdf",
"path": "s3://aryn-public/ntsb/31.pdf",
"path": "s3://aryn-public/ntsb/31.pdf",
"path": "s3://aryn-public/ntsb/34.pdf",
"path": "s3://aryn-public/ntsb/35.pdf",
"path": "s3://aryn-public/ntsb/36.pdf",
"path": "s3://aryn-public/ntsb/37.pdf",
"path": "s3://aryn-public/ntsb/37.pdf",
"path": "s3://aryn-public/ntsb/43.pdf",
The example was just wrong (matching .pdf) will never work unless there is a file with the name "/path/to/data/.pdf". In my testing it caused openai to try matching properties.path.keyword = "22" when asked about item 22.
Be more explicit in the prompt about using keyword for exact matches and including full paths.