-
Notifications
You must be signed in to change notification settings - Fork 115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Console application for filtering CSV #321
base: master
Are you sure you want to change the base?
Changes from 1 commit
02a733d
c23e371
f6a7b29
e929d19
c6f9dd6
6a01bbf
8fee505
c354176
97444e7
7102e61
cdc2b47
119e98c
ebb8f48
1776b21
c3e960e
0658a05
b5cdca6
ae72bb6
a63628d
b176c63
60ff670
31c2f2f
2d67ff1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
#!/usr/bin/env ruby | ||
|
||
require 'optparse' | ||
require 'csv' | ||
|
||
options = {} | ||
|
||
parser = OptionParser.new | ||
|
||
parser.version = CSV::VERSION | ||
parser.banner = <<-BANNER | ||
Usage: #{parser.program_name} [options] | ||
|
||
Reads and parses the CSV text content of the standard input per the given input options. | ||
From that content, generates CSV text per the given output options | ||
and writes that text to the standard output. | ||
|
||
BANNER | ||
|
||
parser.separator('Generic Options') | ||
parser.separator(nil) | ||
|
||
parser.on('-h', '--help', 'Prints this help.') do | ||
puts parser | ||
exit | ||
end | ||
|
||
parser.on('-v', '--version', 'Prints version.') do | ||
puts CSV::VERSION | ||
exit | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need this? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed. |
||
|
||
parser.parse! | ||
|
||
CSV.filter(**options) do |row| | ||
end |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,179 @@ | ||
# -*- coding: utf-8 -*- | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't need this for newly created file. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed. |
||
# frozen_string_literal: false | ||
|
||
require_relative '../helper' | ||
|
||
require 'csv' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you remove this because There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed. |
||
|
||
class TestFilter < Test::Unit::TestCase | ||
|
||
BurdetteLamar marked this conversation as resolved.
Show resolved
Hide resolved
|
||
# Some rows data (useful as default). | ||
Rows = [ | ||
%w[aaa bbb ccc], | ||
%w[ddd eee fff], | ||
] | ||
BurdetteLamar marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
def setup | ||
# In case the previous test left this as true. | ||
$TEST_DEBUG = false | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we really need this? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed. |
||
end | ||
|
||
# Print debugging information if indicated. | ||
def debug(label, value, newline: false) | ||
return unless $TEST_DEBUG | ||
print("\n") if newline | ||
printf("%15s: %s\n", label, value.inspect) | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we really need this? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed. |
||
|
||
# Return the test name (retrieved from the call stack). | ||
def get_test_name | ||
caller.each do |x| | ||
method_name = x.split(' ').last.gsub(/\W/, '') | ||
return method_name if method_name.start_with?('test') | ||
end | ||
raise RuntimeError.new('No test method name found.') | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed. |
||
|
||
# Perform the testing defined in the caller's block. | ||
def do_test(debugging: false) | ||
# Just the caller's block, unless debugging. | ||
unless debugging | ||
yield | ||
return | ||
end | ||
# Wrap the caller's block with debugging information. | ||
$TEST_DEBUG = true | ||
test_name = get_test_name | ||
debug('BEGIN', test_name, newline: true) | ||
yield | ||
debug('END', test_name) | ||
$TEST_DEBUG = false | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we really need this? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed. |
||
|
||
# Return CSV string generated from rows array and options. | ||
def make_csv_s(rows: Rows, **options) | ||
csv_s = CSV.generate(**options) do|csv| | ||
rows.each do |row| | ||
csv << row | ||
end | ||
end | ||
csv_s | ||
BurdetteLamar marked this conversation as resolved.
Show resolved
Hide resolved
|
||
end | ||
|
||
# Return filepath of file containing CSV data. | ||
def csv_filepath(csv_in_s, dirpath, option_sym) | ||
filename = "#{option_sym}.csv" | ||
filepath = File.join(dirpath, filename) | ||
File.write(filepath, csv_in_s) | ||
filepath | ||
end | ||
|
||
# Return stdout and stderr from CLI execution. | ||
def execute_in_cli(filepath, cli_options_s = '') | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Renamed; modified to use array. |
||
debug('cli_options_s', cli_options_s) | ||
command = "cat #{filepath} | ruby bin/csv-filter #{cli_options_s}" | ||
capture_subprocess_io do | ||
system(command) | ||
end | ||
BurdetteLamar marked this conversation as resolved.
Show resolved
Hide resolved
|
||
end | ||
|
||
# Return results for CLI-only option (or invalid option). | ||
def results_for_cli_option(option_name) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we use better name? This executes |
||
cli_out_s = '' | ||
cli_err_s = '' | ||
Dir.mktmpdir do |dirpath| | ||
sym = option_name.to_sym | ||
filepath = csv_filepath('', dirpath, sym) | ||
cli_out_s, cli_err_s = execute_in_cli(filepath, option_name) | ||
end | ||
[cli_out_s, cli_err_s] | ||
end | ||
|
||
# Get and return the actual output from the API. | ||
def get_via_api(csv_in_s, **api_options) | ||
cli_out_s = '' | ||
CSV.filter(csv_in_s, cli_out_s, **api_options) {|row| } | ||
cli_out_s | ||
end | ||
|
||
# Test for invalid option. | ||
|
||
def test_invalid_option | ||
do_test(debugging: false) do | ||
%w[-Z --ZZZ].each do |option_name| | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do we need to test both of short option and long option? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Modified. |
||
cli_out_s, cli_err_s = results_for_cli_option(option_name) | ||
assert_empty(cli_out_s) | ||
BurdetteLamar marked this conversation as resolved.
Show resolved
Hide resolved
|
||
assert_match(/OptionParser::InvalidOption/, cli_err_s) | ||
end | ||
end | ||
end | ||
|
||
# Test for no options. | ||
|
||
def test_no_options | ||
do_test(debugging: false) do | ||
csv_in_s = make_csv_s | ||
cli_out_s = get_via_api(csv_in_s) | ||
assert_equal(csv_in_s, cli_out_s) | ||
end | ||
end | ||
|
||
# Tests for general options. | ||
|
||
def test_option_h | ||
do_test(debugging: false) do | ||
%w[-h --help].each do |option_name| | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you avoid using Could you define separated tests instead of checking multiple items in one test? It's for easy to debug on error. def test_option_h
end
def test_option_help
end There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Modified. |
||
cli_out_s, cli_err_s = results_for_cli_option(option_name) | ||
assert_match(/Usage/, cli_out_s) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
assert_equal("Usage: csv-filter [options]\n", cli_out_s.lines.first) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Modified. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Modified. |
||
assert_empty(cli_err_s) | ||
end | ||
end | ||
end | ||
|
||
def test_option_v | ||
do_test(debugging: false) do | ||
%w[-v --version].each do |option_name| | ||
cli_out_s, cli_err_s = results_for_cli_option(option_name) | ||
assert_match(/\d+\.\d+\.\d+/, cli_out_s) | ||
assert_empty(cli_err_s) | ||
end | ||
end | ||
end | ||
|
||
# Two methods copied from module Minitest::Assertions. | ||
# because we need access to the subprocess io. | ||
|
||
def _synchronize # :nodoc: | ||
yield | ||
end | ||
|
||
def capture_subprocess_io | ||
_synchronize do | ||
begin | ||
require "tempfile" | ||
|
||
captured_stdout, captured_stderr = Tempfile.new("out"), Tempfile.new("err") | ||
|
||
orig_stdout, orig_stderr = $stdout.dup, $stderr.dup | ||
$stdout.reopen captured_stdout | ||
$stderr.reopen captured_stderr | ||
|
||
yield | ||
|
||
$stdout.rewind | ||
$stderr.rewind | ||
|
||
return captured_stdout.read, captured_stderr.read | ||
ensure | ||
$stdout.reopen orig_stdout | ||
$stderr.reopen orig_stderr | ||
|
||
orig_stdout.close | ||
orig_stderr.close | ||
captured_stdout.close! | ||
captured_stderr.close! | ||
end | ||
end | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We don't need them. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removed. |
||
|
||
BurdetteLamar marked this conversation as resolved.
Show resolved
Hide resolved
|
||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this?
--help
will work by default.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed.