[ESQL] Add TO_DATE_NANOS conversion function (elastic#112150) (elasti…

…c#113641) Resolves elastic#111842 This adds a conversion function that yields DATE_NANOS. Mostly this is straight forward. It is worth noting that when converting a millisecond date into a nanosecond date, the conversion function truncates it to 0 nanoseconds (i.e. first nanosecond of that millisecond). This is, of course, a bit of an assumption, but I don't have a better assumption we can make. I'd thought about adding a second, optional, parameter to control this behavior, but it's important that TO_DATE_NANOS extend AbstractConvertFunction, which itself extends UnaryScalarFunction, so that it will work correctly with union types. Also, it's unlikely the user will have any better guess than we do for filling in the nanoseconds. Making that assumption does, however, create some weirdness. Consider two comparisons: TO_DATETIME("2023-03-23T12:15:03.360103847") == TO_DATETIME("2023-03-23T12:15:03.360") will return true while TO_DATE_NANOS("2023-03-23T12:15:03.360103847") == TO_DATE_NANOS("2023-03-23T12:15:03.360") will return false. This is akin to casting between longs and doubles, where things may compare equal in one type that are not equal in the other. This seems fine, and I can't think of a better way to do it, but it's worth being aware of. --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
thecoop · Sep 27, 2024 · d2a8996 · d2a8996
1 parent 4ea42af
commit d2a8996
Show file tree

Hide file tree

Showing 34 changed files with 1,104 additions and 128 deletions.
diff --git a/docs/reference/esql/functions/description/qstr.asciidoc b/docs/reference/esql/functions/description/qstr.asciidoc
diff --git a/docs/reference/esql/functions/description/to_date_nanos.asciidoc b/docs/reference/esql/functions/description/to_date_nanos.asciidoc
diff --git a/docs/reference/esql/functions/examples/qstr.asciidoc b/docs/reference/esql/functions/examples/qstr.asciidoc
diff --git a/docs/reference/esql/functions/kibana/definition/date_diff.json b/docs/reference/esql/functions/kibana/definition/date_diff.json
diff --git a/docs/reference/esql/functions/kibana/definition/date_format.json b/docs/reference/esql/functions/kibana/definition/date_format.json
diff --git a/docs/reference/esql/functions/kibana/definition/qstr.json b/docs/reference/esql/functions/kibana/definition/qstr.json
diff --git a/docs/reference/esql/functions/kibana/definition/to_date_nanos.json b/docs/reference/esql/functions/kibana/definition/to_date_nanos.json
diff --git a/docs/reference/esql/functions/kibana/docs/date_format.md b/docs/reference/esql/functions/kibana/docs/date_format.md
diff --git a/docs/reference/esql/functions/kibana/docs/mv_avg.md b/docs/reference/esql/functions/kibana/docs/mv_avg.md
diff --git a/docs/reference/esql/functions/kibana/docs/mv_sum.md b/docs/reference/esql/functions/kibana/docs/mv_sum.md
diff --git a/docs/reference/esql/functions/kibana/docs/qstr.md b/docs/reference/esql/functions/kibana/docs/qstr.md
diff --git a/docs/reference/esql/functions/kibana/docs/to_date_nanos.md b/docs/reference/esql/functions/kibana/docs/to_date_nanos.md
diff --git a/docs/reference/esql/functions/kibana/inline_cast.json b/docs/reference/esql/functions/kibana/inline_cast.json
diff --git a/docs/reference/esql/functions/layout/qstr.asciidoc b/docs/reference/esql/functions/layout/qstr.asciidoc
diff --git a/docs/reference/esql/functions/layout/to_date_nanos.asciidoc b/docs/reference/esql/functions/layout/to_date_nanos.asciidoc
diff --git a/...e/esql/functions/parameters/qstr.asciidoc → ...nctions/parameters/to_date_nanos.asciidoc b/...e/esql/functions/parameters/qstr.asciidoc → ...nctions/parameters/to_date_nanos.asciidoc
diff --git a/docs/reference/esql/functions/signature/qstr.svg b/docs/reference/esql/functions/signature/qstr.svg
diff --git a/docs/reference/esql/functions/signature/to_date_nanos.svg b/docs/reference/esql/functions/signature/to_date_nanos.svg
diff --git a/...erence/esql/functions/types/qstr.asciidoc → ...ql/functions/types/to_date_nanos.asciidoc b/...erence/esql/functions/types/qstr.asciidoc → ...ql/functions/types/to_date_nanos.asciidoc
diff --git a/x-pack/plugin/esql/qa/testFixtures/src/main/resources/date_nanos.csv b/x-pack/plugin/esql/qa/testFixtures/src/main/resources/date_nanos.csv
@@ -1,9 +1,9 @@
-millis:date,nanos:date_nanos
-2023-10-23T13:55:01.543Z,2023-10-23T13:55:01.543123456Z
-2023-10-23T13:53:55.832Z,2023-10-23T13:53:55.832987654Z
-2023-10-23T13:52:55.015Z,2023-10-23T13:52:55.015787878Z
-2023-10-23T13:51:54.732Z,2023-10-23T13:51:54.732102837Z
-2023-10-23T13:33:34.937Z,2023-10-23T13:33:34.937193000Z
-2023-10-23T12:27:28.948Z,2023-10-23T12:27:28.948000000Z
-2023-10-23T12:15:03.360Z,2023-10-23T12:15:03.360103847Z
-1999-10-23T12:15:03.360Z,[2023-03-23T12:15:03.360103847Z, 2023-02-23T13:33:34.937193000Z, 2023-01-23T13:55:01.543123456Z]
+millis:date,nanos:date_nanos,num:long
+2023-10-23T13:55:01.543Z,2023-10-23T13:55:01.543123456Z,1698069301543123456
+2023-10-23T13:53:55.832Z,2023-10-23T13:53:55.832987654Z,1698069235832987654
+2023-10-23T13:52:55.015Z,2023-10-23T13:52:55.015787878Z,1698069175015787878
+2023-10-23T13:51:54.732Z,2023-10-23T13:51:54.732102837Z,1698069114732102837
+2023-10-23T13:33:34.937Z,2023-10-23T13:33:34.937193000Z,1698068014937193000
+2023-10-23T12:27:28.948Z,2023-10-23T12:27:28.948000000Z,1698064048948000000
+2023-10-23T12:15:03.360Z,2023-10-23T12:15:03.360103847Z,1698063303360103847
+1999-10-23T12:15:03.360Z,[2023-03-23T12:15:03.360103847Z, 2023-02-23T13:33:34.937193000Z, 2023-01-23T13:55:01.543123456Z], 0