Fractional seconds from strptime using %OS?

[tinkerware]

It would be nice if there was a way to get fractional seconds from a formatted timestamp:

strptime("2015-08-28T13:33:21.345678Z", "%Y-%m-%dT%H:%M:%OSZ") = 1440768801.345678

The other tool I know of that supports this convention is tplot/splot at http://jkff.info/software/timeplotters/index.html.

This would help in uses where the log file is a fine-grained trace of sub-second resolution events.

Thanks for the nice software!

[johnkerl]

Awesome! I definitely like fractional seconds whenever I can get them.

In this case I'm just directly invoking the C library's strptime on whatever the platform is. I'll look into but my guess is I've have to re-implement strptime, if your platform isn't giving it to you already. :(

man 3 strptime will show you what's in your platform's C library.

[tinkerware]

tm structure does not seem to include a field for sub-second quantities, so stock strptime will not work I think. Quick search on stack overflow reveals http://stackoverflow.com/questions/7114690/how-to-parse-syslog-timestamp as a way of using strptime to parse timestamp and get the unparsed remainder, then parse the fractional seconds in second step (I guess you could just interpret %S to parse fractional seconds if they exist).

[johnkerl]

nice! :)

[johnkerl]

Working on the Windows port for a while ...

[johnkerl]

OK the last 7 commits ending in 547f164 introduce this feature.

• For strptime you can just use %S. It will read integer or floating-point seconds.
• For strftime you can use %S to format as integer, or %1S through %9S to format with 1..9 decimal places respectively.
• The gmt2sec keystroke-saver works out of the box. For sec2gmt I'll create a two-argument variant which allows you to specify the number of decimal places.

[johnkerl]

Docs:

• http://johnkerl.org/miller-releases/miller-head/doc/reference-dsl.html#strftime
• http://johnkerl.org/miller-releases/miller-head/doc/reference-dsl.html#strptime

[johnkerl]

f74b323 is the last bit for sec2gmt. This will go out in 5.1.0 (real soon now).

Thank you for pushing for this!!!!!

[tinkerware]

Thanks for implementing this, has been useful for me!

[derekmahar]

Why might strptime in Miller 5.2.0 be unable to parse timestamp "2016-02-16T02:13:22.238733" having fractional seconds?

1. Format string %S doesn't work, but should according to your earlier comment:

   $ printf "unixtime\n\n" | mlr --csv put '$unixtime = strptime("2016-02-16T02:13:22.238733", "%Y-%m-%dT%H:%M:%S");'
   mlr: could not strptime("2016-02-16T02:13:22.238733", "%Y-%m-%dT%H:%M:%S"). See "mlr --help-function strptime".
   

2. Format string %6S also doesn't work, but I hoped that it would since it is supposed to work for strftime:

   $ printf "unixtime\n\n" | mlr --csv put '$unixtime = strptime("2016-02-16T02:13:22.238733", "%Y-%m-%dT%H:%M:%6S");'
   mlr: could not strptime("2016-02-16T02:13:22.238733", "%Y-%m-%dT%H:%M:%6S"). See "mlr --help-function strptime".
   

3. Note that strptime("2016-02-16T02:13:22", "%Y-%m-%dT%H:%M:%S") behaves as I expected:

   $ printf "unixtime\n\n" | mlr --csv put '$unixtime = strptime("2016-02-16T02:13:22", "%Y-%m-%dT%H:%M:%S");'
   unixtime
   1455588802.000000
   

[johnkerl]

will look, thanks!

[johnkerl]

@derekmahar

$ echo stamp=2016-02-16T02:13:22.238733 | mlr put '$unixtime = strptime($stamp, "%Y-%m-%dT%H:%M:%S.%f")'
stamp=2016-02-16T02:13:22.238733,unixtime=1455588802.238733

following on #882

See also https://miller.readthedocs.io/en/latest/reference-dsl-time/#fractional-seconds

Please let me know if my closing this is in error, and we can re-open -- thank you!