Allow timestamp RFC names in json parser's json_time_format

[maathor]

Relevant telegraf.conf:

[[outputs.influxdb]]
  urls = ["http://localhost:8086"]
  database = "AAAAAAAAAAAAA"
  retention_policy = ""
  write_consistency = "any"
  timeout = "5s"
  username = "DDDDDDDDDDD"
  password = "FFFFFFFFFFFFF"
  user_agent = "telegraf"

[[inputs.kafka_consumer]]
  brokers = ["XXXXX:9092","XXXXX:9092","XXXXX:9092"]
  topics = ["elb_logs"]
  consumer_group = "telegraf_consumer_elb_0"
  offset = "oldest"
  data_format ="json"
  name_override = "elb"
  max_message_len = 0
  json_time_format = "RFC3339" // or  json_time_format = "2018-10-26T13:30:33Z"
  tag_keys = ["Elb"]
  json_string_fields = [
    "Elb",
    "ClientPort",
    "BackendPort",
    "Request_processing_time",
    "Backend_processing_time",
    "Response_processing_time",
    "Elb_status_code",
    "Backend_status_code",
    "Received_bytes",
    "Sent_bytes",
    "Request",
    "User_agent",
    "Ssl_cipher",
    "Ssl_protocol"
    ]
  json_time_key = "Timestamp"

System info:

Telegraf 1.8.1
Linux ip-172-40-16-175 4.15.0-1021-aws #21-Ubuntu SMP Tue Aug 28 10:23:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Ubuntu 16.04

Expected behavior:

I add to kafka json message , with golang
These timsteamp are formated like that:

timestring.Format(time.RFC3339) I use time library in golang

Actual behavior:

for json_time_format = "2018-10-26T13:30:33Z"
I got :

message: {"Timestamp":"2018-10-26T14:00:52Z","Elb":"blabla-prod","ClientPort":"213.152.17.19:54314","BackendPort":"-","Request_processing_time":"-1","Backend_processing_time":"-1","Response_processing_time":"-1","Elb_status_code":"503","Backend_status_code":"0","Received_bytes":"0","Sent_bytes":"0","Request":"GET https://YYYYY:443/ HTTP/1.1","User_agent":"curl/7.47.0","Ssl_cipher":"ECDHE-RSA-AES128-GCM-SHA256","Ssl_protocol":"TLSv1.2"}
error: parsing time "2018-10-26T14:00:52Z": month out of range

for json_time_format = "RFC3339"

error: parsing time "2018-10-26T13:30:33Z" as "RFC3339": cannot parse "2018-10-26T13:30:33Z" as "RFC"

Additional info:

telegraf parser json drove me nuts

[maathor]

working with:
json_time_format = "2006-01-02T15:04:05Z"
you should explain somewhere we should use precisely the datetime 2006-01-02T15:04:05 but I can't explain why timestamp formated with timestring.Format(time.RFC3339) cannot be converted with json_time_format = "RFC3339"

[glinton]

Thanks for the suggestion, I do like the idea of allowing the RFCXXXX codes as defined in the stdlib. We have already documented that the json_time_format needs to be in go time using the date you mentioned in the json parser readme

[paale]

I've struggled with this for quite some time myself.
My json config:

  json_time_key="timestamp"
  json_time_format="2021-02-17T21:21:23.890Z"

I also tried:

  json_time_key="timestamp"
  json_time_format="rfc3339"

...but telegraf seems unable to parse the timestamps.

Are timestamps in the format "2021-02-17T21:21:23.890Z" not supported? I cannot find any examples of them in the stdlib, linked in the previous post, but I wanted to check if this really is the case.

[ssoroka]

It's supported. For that format, use json_time_format = "2006-01-02T15:04:05.999999999Z" or json_time_format = "2006-01-02T15:04:05.999999999Z07:00"

[paale]

Thanks! It worked!