Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Time converter to CSV::Converters #313

Merged
merged 1 commit into from
Aug 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion doc/csv/recipes/parsing.rdoc
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ All code snippets on this page assume that the following has been executed:
- {Recipe: Convert Fields to Numerics}[#label-Recipe-3A+Convert+Fields+to+Numerics]
- {Recipe: Convert Fields to Dates}[#label-Recipe-3A+Convert+Fields+to+Dates]
- {Recipe: Convert Fields to DateTimes}[#label-Recipe-3A+Convert+Fields+to+DateTimes]
- {Recipe: Convert Fields to Times}[#label-Recipe-3A+Convert+Fields+to+Times]
- {Recipe: Convert Assorted Fields to Objects}[#label-Recipe-3A+Convert+Assorted+Fields+to+Objects]
- {Recipe: Convert Fields to Other Objects}[#label-Recipe-3A+Convert+Fields+to+Other+Objects]
- {Recipe: Filter Field Strings}[#label-Recipe-3A+Filter+Field+Strings]
Expand Down Expand Up @@ -339,6 +340,7 @@ There are built-in field converters for converting to objects of certain classes
- \Integer
- \Date
- \DateTime
- \Time

Other built-in field converters include:
- +:numeric+: converts to \Integer and \Float.
Expand Down Expand Up @@ -381,6 +383,13 @@ Convert fields to \DateTime objects using built-in converter +:date_time+:
parsed = CSV.parse(source, headers: true, converters: :date_time)
parsed.map {|row| row['DateTime'].class} # => [DateTime, DateTime, DateTime]

===== Recipe: Convert Fields to Times

Convert fields to \Time objects using built-in converter +:time+:
source = "Name,Time\nfoo,2001-02-03\nbar,2001-02-04\nbaz,2020-05-07T14:59:00-05:00\n"
parsed = CSV.parse(source, headers: true, converters: :time)
parsed.map {|row| row['Time'].class} # => [Time, Time, Time]

===== Recipe: Convert Assorted Fields to Objects

Convert assorted fields to objects using built-in converter +:all+:
Expand Down Expand Up @@ -542,4 +551,4 @@ Output:
#<struct CSV::FieldInfo index=0, line=2, header=nil>
#<struct CSV::FieldInfo index=1, line=2, header=nil>
#<struct CSV::FieldInfo index=0, line=3, header=nil>
#<struct CSV::FieldInfo index=1, line=3, header=nil>
#<struct CSV::FieldInfo index=1, line=3, header=nil>
24 changes: 22 additions & 2 deletions lib/csv.rb
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@

require "forwardable"
require "date"
require "time"
require "stringio"

require_relative "csv/fields_converter"
Expand Down Expand Up @@ -521,6 +522,7 @@
# - <tt>:float</tt>: converts each \String-embedded float into a true \Float.
# - <tt>:date</tt>: converts each \String-embedded date into a true \Date.
# - <tt>:date_time</tt>: converts each \String-embedded date-time into a true \DateTime
# - <tt>:time</tt>: converts each \String-embedded time into a true \Time
# .
# This example creates a converter proc, then stores it:
# strip_converter = proc {|field| field.strip }
Expand Down Expand Up @@ -631,6 +633,7 @@
# [:numeric, [:integer, :float]]
# [:date, Proc]
# [:date_time, Proc]
# [:time, Proc]
# [:all, [:date_time, :numeric]]
#
# Each of these converters transcodes values to UTF-8 before attempting conversion.
Expand Down Expand Up @@ -675,6 +678,15 @@
# csv = CSV.parse_line(data, converters: :date_time)
# csv # => [#<DateTime: 2020-05-07T14:59:00-05:00 ((2458977j,71940s,0n),-18000s,2299161j)>, "x"]
#
# Converter +time+ converts each field that Time::parse accepts:
# data = '2020-05-07T14:59:00-05:00,x'
# # Without the converter
# csv = CSV.parse_line(data)
# csv # => ["2020-05-07T14:59:00-05:00", "x"]
# # With the converter
# csv = CSV.parse_line(data, converters: :time)
# csv # => [2020-05-07 14:59:00 -0500, "x"]
#
# Converter +:numeric+ converts with both +:date_time+ and +:numeric+..
#
# As seen above, method #convert adds \converters to a \CSV instance,
Expand Down Expand Up @@ -871,10 +883,10 @@ def initialize(encoding, line_number)
# A Regexp used to find and convert some common Date formats.
DateMatcher = / \A(?: (\w+,?\s+)?\w+\s+\d{1,2},?\s+\d{2,4} |
\d{4}-\d{2}-\d{2} )\z /x
# A Regexp used to find and convert some common DateTime formats.
# A Regexp used to find and convert some common (Date)Time formats.
DateTimeMatcher =
/ \A(?: (\w+,?\s+)?\w+\s+\d{1,2}\s+\d{1,2}:\d{1,2}:\d{1,2},?\s+\d{2,4} |
# ISO-8601 and RFC-3339 (space instead of T) recognized by DateTime.parse
# ISO-8601 and RFC-3339 (space instead of T) recognized by (Date)Time.parse
\d{4}-\d{2}-\d{2}
(?:[T\s]\d{2}:\d{2}(?::\d{2}(?:\.\d+)?(?:[+-]\d{2}(?::\d{2})|Z)?)?)?
)\z /x
Expand Down Expand Up @@ -912,6 +924,14 @@ def initialize(encoding, line_number)
f
end
},
time: lambda { |f|
begin
e = f.encode(ConverterEncoding)
e.match?(DateTimeMatcher) ? Time.parse(e) : f
rescue # encoding conversion or parse errors
f
end
},
all: [:date_time, :numeric],
}

Expand Down
142 changes: 142 additions & 0 deletions test/csv/test_data_converters.rb
Original file line number Diff line number Diff line change
Expand Up @@ -187,4 +187,146 @@ def test_builtin_date_time_converter_rfc3339_tab_utc
assert_equal(datetime,
CSV::Converters[:date_time][rfc3339_string])
end

def test_builtin_time_converter
# does convert
assert_instance_of(Time,
CSV::Converters[:time][@win_safe_time_str])

# does not convert
assert_instance_of(String, CSV::Converters[:time]["junk"])
end

def test_builtin_time_converter_iso8601_date
iso8601_string = "2018-01-14"
time = Time.new(2018, 1, 14)
assert_equal(time,
CSV::Converters[:time][iso8601_string])
end

def test_builtin_time_converter_iso8601_minute
iso8601_string = "2018-01-14T22:25"
time = Time.new(2018, 1, 14, 22, 25)
assert_equal(time,
CSV::Converters[:time][iso8601_string])
end

def test_builtin_time_converter_iso8601_second
iso8601_string = "2018-01-14T22:25:19"
time = Time.new(2018, 1, 14, 22, 25, 19)
assert_equal(time,
CSV::Converters[:time][iso8601_string])
end

def test_builtin_time_converter_iso8601_under_second
iso8601_string = "2018-01-14T22:25:19.1"
time = Time.new(2018, 1, 14, 22, 25, 19.1r)
assert_equal(time,
CSV::Converters[:time][iso8601_string])
end

def test_builtin_time_converter_iso8601_under_second_offset
iso8601_string = "2018-01-14T22:25:19.1+09:00"
time = Time.new(2018, 1, 14, 22, 25, 19.1r, "+09:00")
assert_equal(time,
CSV::Converters[:time][iso8601_string])
end

def test_builtin_time_converter_iso8601_offset
iso8601_string = "2018-01-14T22:25:19+09:00"
time = Time.new(2018, 1, 14, 22, 25, 19, "+09:00")
assert_equal(time,
CSV::Converters[:time][iso8601_string])
end

def test_builtin_time_converter_iso8601_utc
iso8601_string = "2018-01-14T22:25:19Z"
time = Time.utc(2018, 1, 14, 22, 25, 19)
assert_equal(time,
CSV::Converters[:time][iso8601_string])
end

def test_builtin_time_converter_rfc3339_minute
rfc3339_string = "2018-01-14 22:25"
time = Time.new(2018, 1, 14, 22, 25)
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_second
rfc3339_string = "2018-01-14 22:25:19"
time = Time.new(2018, 1, 14, 22, 25, 19)
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_under_second
rfc3339_string = "2018-01-14 22:25:19.1"
time = Time.new(2018, 1, 14, 22, 25, 19.1r)
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_under_second_offset
rfc3339_string = "2018-01-14 22:25:19.1+09:00"
time = Time.new(2018, 1, 14, 22, 25, 19.1r, "+09:00")
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_offset
rfc3339_string = "2018-01-14 22:25:19+09:00"
time = Time.new(2018, 1, 14, 22, 25, 19, "+09:00")
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_utc
rfc3339_string = "2018-01-14 22:25:19Z"
time = Time.utc(2018, 1, 14, 22, 25, 19)
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_tab_minute
rfc3339_string = "2018-01-14\t22:25"
time = Time.new(2018, 1, 14, 22, 25)
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_tab_second
rfc3339_string = "2018-01-14\t22:25:19"
time = Time.new(2018, 1, 14, 22, 25, 19)
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_tab_under_second
rfc3339_string = "2018-01-14\t22:25:19.1"
time = Time.new(2018, 1, 14, 22, 25, 19.1r)
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_tab_under_second_offset
rfc3339_string = "2018-01-14\t22:25:19.1+09:00"
time = Time.new(2018, 1, 14, 22, 25, 19.1r, "+09:00")
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_tab_offset
rfc3339_string = "2018-01-14\t22:25:19+09:00"
time = Time.new(2018, 1, 14, 22, 25, 19, "+09:00")
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end

def test_builtin_time_converter_rfc3339_tab_utc
rfc3339_string = "2018-01-14\t22:25:19Z"
time = Time.utc(2018, 1, 14, 22, 25, 19)
assert_equal(time,
CSV::Converters[:time][rfc3339_string])
end
end