Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor query shape field data maps #111

Merged
merged 1 commit into from
Sep 19, 2024

Conversation

dzane17
Copy link
Collaborator

@dzane17 dzane17 commented Sep 9, 2024

Description

Refactor query shape field data maps

With the introduction of WithFieldName interface in OpenSearch core (see opensearch-project/OpenSearch#15705), we can now simplify the field data logic in Query-Insights.

Issues Resolved

Resolves #109
Related RFC #69

Testing

Large Search

curl -XGET 'http://localhost:9200/_search?pretty' -H 'Content-Type: application/json' -d '{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "order_date": {
              "gte": "2023-01-01",
              "lte": "2023-12-31"
            }
          }
        },
        {
          "term": {
            "status": "completed"
          }
        }
      ],
      "filter": [
        {
          "terms": {
            "category": ["electronics", "clothing"]
          }
        }
      ]
    }
  },
  "aggs": {
    "sales_per_month": {
      "date_histogram": {
        "field": "order_date",
        "calendar_interval": "month",
        "format": "yyyy-MM",
        "order": {
          "_key": "desc"
        }
      },
      "aggs": {
        "total_sales": {
          "sum": {
            "field": "total_amount"
          }
        },
        "avg_order_quantity": {
          "avg": {
            "field": "order_quantity"
          }
        },
        "max_order_amount": {
          "max": {
            "field": "total_amount"
          }
        }
      }
    },
    "avg_monthly_sales": {
      "avg_bucket": {
        "buckets_path": "sales_per_month>total_sales"
      }
    },
    "avg_max_order_amount": {
      "avg_bucket": {
        "buckets_path": "sales_per_month>max_order_amount"
      }
    }
  },
  "sort": [
    { "sales_per_month>key": { "order": "desc" } },
    { "total_sales": { "order": "desc" } },
    { "avg_order_quantity": { "order": "asc" } }
  ]
}'
[2024-09-09T13:15:16,352][INFO ][stdout                   ] [integTest-0] bool []
[2024-09-09T13:15:16,354][INFO ][stdout                   ] [integTest-0]   must:
[2024-09-09T13:15:16,354][INFO ][stdout                   ] [integTest-0]     range [order_date]
[2024-09-09T13:15:16,355][INFO ][stdout                   ] [integTest-0]     term [status]
[2024-09-09T13:15:16,355][INFO ][stdout                   ] [integTest-0]   filter:
[2024-09-09T13:15:16,355][INFO ][stdout                   ] [integTest-0]     terms [category]
[2024-09-09T13:15:16,355][INFO ][stdout                   ] [integTest-0] aggregation:
[2024-09-09T13:15:16,356][INFO ][stdout                   ] [integTest-0]   date_histogram [order_date]
[2024-09-09T13:15:16,356][INFO ][stdout                   ] [integTest-0]     aggregation:
[2024-09-09T13:15:16,356][INFO ][stdout                   ] [integTest-0]       avg [order_quantity]
[2024-09-09T13:15:16,357][INFO ][stdout                   ] [integTest-0]       max [total_amount]
[2024-09-09T13:15:16,357][INFO ][stdout                   ] [integTest-0]       sum [total_amount]
[2024-09-09T13:15:16,357][INFO ][stdout                   ] [integTest-0]   pipeline aggregation:
[2024-09-09T13:15:16,357][INFO ][stdout                   ] [integTest-0]     avg_bucket
[2024-09-09T13:15:16,358][INFO ][stdout                   ] [integTest-0]     avg_bucket
[2024-09-09T13:15:16,358][INFO ][stdout                   ] [integTest-0] sort:
[2024-09-09T13:15:16,358][INFO ][stdout                   ] [integTest-0]   desc [sales_per_month>key]
[2024-09-09T13:15:16,358][INFO ][stdout                   ] [integTest-0]   desc [total_sales]
[2024-09-09T13:15:16,358][INFO ][stdout                   ] [integTest-0]   asc [avg_order_quantity]

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dzane17 dzane17 force-pushed the field-name-simplify branch 2 times, most recently from 715e373 to dc1e87a Compare September 9, 2024 21:12
@dzane17 dzane17 force-pushed the field-name-simplify branch 3 times, most recently from 77e08c9 to 5c55429 Compare September 17, 2024 00:33
@dzane17
Copy link
Collaborator Author

dzane17 commented Sep 17, 2024

Unrelated build failure fix: #117

@dzane17 dzane17 force-pushed the field-name-simplify branch 2 times, most recently from 030b5ef to 00c8f18 Compare September 18, 2024 17:49
@dzane17
Copy link
Collaborator Author

dzane17 commented Sep 18, 2024

@jainankitk @deshsidd @ansjcy Please take a look again. Since all relevant builders now implement the WithFieldName interface, I combined all three query, aggregation, sort maps into one map.

Signed-off-by: David Zane <davizane@amazon.com>
@jainankitk jainankitk merged commit f1827d8 into opensearch-project:main Sep 19, 2024
16 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 19, 2024
…Name interface (#111)

Signed-off-by: David Zane <davizane@amazon.com>
(cherry picked from commit f1827d8)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@dzane17 dzane17 deleted the field-name-simplify branch September 20, 2024 00:12
dzane17 pushed a commit that referenced this pull request Sep 20, 2024
…Name interface (#111) (#125)

(cherry picked from commit f1827d8)

Signed-off-by: David Zane <davizane@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove unnecessary map of QueryBuilders to fieldName function
4 participants