Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

solid mode with placeholder (-I) #54

Merged
merged 5 commits into from
Mar 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

29 changes: 28 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,8 @@ FLAGS:
-s Execute new command for each bypassed chunk
--chomp Command spawned by -s receives standard input without trailing
newlines
-I <replace-str> Replace the <replace-str> with bypassed chunk in the <command>
then -s is forcefully enabled.
-v Invert the range of bypassing
-z Line delimiter is NUL instead of a newline

Expand Down Expand Up @@ -525,6 +527,31 @@ $ echo $?

However, this option is not suitable for processing large files because of its high processing overhead, which can significantly degrade performance.

#### Solid mode with placeholder (`-I <replace-str>`)

If you want to use the contents of the hole as an argument of the targeted command, use the `-I` option.

```bash
$ echo AAA BBB CCC | teip -f 2 -I @ -- echo '[@]'
AAA [BBB] CCC
```

`<replace-str>` can be any strings and multiple characters are allowed.

```bash
$ seq 5 | teip -f 1 -I NUMBER -- awk 'BEGIN{print NUMBER * 3}'
3
6
9
12
15
```

Please note that `-s` is automatically enabled with `-I`.
Therefore, it is not suitable for processing huge files.
In addition, the targeted command does not get any input from stdin.
The targeted command is expected to work without stdin.

#### Solid mode with `--chomp`

If `-s` option does not work as expected, `--chomp` may be helpful.
Expand All @@ -543,7 +570,7 @@ $ echo AAABBBCCC | teip -og BBB -s -- tr '\n' '@'
AAABBB@CCC
```

The above is an example where the targeted command is a "tr command that converts line field (\x0A) to @".
The above is an example where the targeted command is a "tr command that converts line field (`\x0A`) to @".
"BBB" does not contain a newline, but the result is "BBB@", because implicitly added line breaks have been processed.
To prevent this behavior, use the `--chomp` option.
This option gives the targeted command pure input with no newlines added.
Expand Down
6 changes: 5 additions & 1 deletion completion/bash/teip
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ _teip() {

case "${cmd}" in
teip)
opts=" -o -G -s -v -z -h -V -g -f -d -D -c -l -e -A -B -C --csv --unko --chomp --help --version --sed --awk --completion <command>... "
opts=" -o -G -s -v -z -h -V -g -f -d -D -c -l -I -e -A -B -C --csv --unko --chomp --help --version --sed --awk --completion <command>... "
if [[ ${cur} == -* || ${COMP_CWORD} -eq 1 ]] ; then
COMPREPLY=( $(compgen -W "${opts}" -- "${cur}") )
return 0
Expand Down Expand Up @@ -51,6 +51,10 @@ _teip() {
COMPREPLY=($(compgen -f "${cur}"))
return 0
;;
-I)
COMPREPLY=($(compgen -f "${cur}"))
return 0
;;
-e)
COMPREPLY=($(compgen -f "${cur}"))
return 0
Expand Down
1 change: 1 addition & 0 deletions completion/fish/teip.fish
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ complete -c teip -n "__fish_use_subcommand" -s d -d 'Use <delimiter> for field d
complete -c teip -n "__fish_use_subcommand" -s D -d 'Use regular expression <pattern> for field delimiter of -f'
complete -c teip -n "__fish_use_subcommand" -s c -d 'Bypassing these characters'
complete -c teip -n "__fish_use_subcommand" -s l -d 'Bypassing those lines'
complete -c teip -n "__fish_use_subcommand" -s I -d 'Replace the <replace-str> with bypassed chunk in the <command> then -s is forcefully enabled.'
complete -c teip -n "__fish_use_subcommand" -s e -d 'Execute <string> on another process that will receive identical standard input as the teip, and numbers given by the result are used as line numbers for bypassing'
complete -c teip -n "__fish_use_subcommand" -s A -d 'Alias of -e \'grep -n -A <number> <pattern>\''
complete -c teip -n "__fish_use_subcommand" -s B -d 'Alias of -e \'grep -n -B <number> <pattern>\''
Expand Down
7 changes: 7 additions & 0 deletions completion/generate.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
set -eu
THIS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cargo run -- --completion bash > "$THIS_DIR"/bash/teip
cargo run -- --completion fish > "$THIS_DIR"/fish/teip.fish
cargo run -- --completion zsh > "$THIS_DIR"/zsh/_teip
cargo run -- --completion powershell > "$THIS_DIR"/powershell/teip.ps1
1 change: 1 addition & 0 deletions completion/powershell/teip.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Register-ArgumentCompleter -Native -CommandName 'teip' -ScriptBlock {
[CompletionResult]::new('-D', 'D', [CompletionResultType]::ParameterName, 'Use regular expression <pattern> for field delimiter of -f')
[CompletionResult]::new('-c', 'c', [CompletionResultType]::ParameterName, 'Bypassing these characters')
[CompletionResult]::new('-l', 'l', [CompletionResultType]::ParameterName, 'Bypassing those lines')
[CompletionResult]::new('-I', 'I', [CompletionResultType]::ParameterName, 'Replace the <replace-str> with bypassed chunk in the <command> then -s is forcefully enabled.')
[CompletionResult]::new('-e', 'e', [CompletionResultType]::ParameterName, 'Execute <string> on another process that will receive identical standard input as the teip, and numbers given by the result are used as line numbers for bypassing')
[CompletionResult]::new('-A', 'A', [CompletionResultType]::ParameterName, 'Alias of -e ''grep -n -A <number> <pattern>''')
[CompletionResult]::new('-B', 'B', [CompletionResultType]::ParameterName, 'Alias of -e ''grep -n -B <number> <pattern>''')
Expand Down
1 change: 1 addition & 0 deletions completion/zsh/_teip
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ _teip() {
'-D+[Use regular expression <pattern> for field delimiter of -f]' \
'-c+[Bypassing these characters]' \
'-l+[Bypassing those lines]' \
'-I+[Replace the <replace-str> with bypassed chunk in the <command> then -s is forcefully enabled.]' \
'-e+[Execute <string> on another process that will receive identical standard input as the teip, and numbers given by the result are used as line numbers for bypassing]' \
'-A+[Alias of -e '\''grep -n -A <number> <pattern>'\'']' \
'-B+[Alias of -e '\''grep -n -B <number> <pattern>'\'']' \
Expand Down
3 changes: 3 additions & 0 deletions man/man.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,9 @@ OPTIONS
`-s`
Execute new command for each bypassed chunk

`-I`
Replace the <replace-str> with bypassed chunk in the <command> then -s is forcefully enabled.

`--chomp`
Command spawned by -s receives standard input without trailing newlines

Expand Down
13 changes: 11 additions & 2 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,8 @@ FLAGS:
-s Execute new command for each bypassed chunk
--chomp Command spawned by -s receives standard input without trailing
newlines
-I <replace-str> Replace the <replace-str> with bypassed chunk in the <command>
then -s is forcefully enabled.
-v Invert the range of bypassing
-z Line delimiter is NUL instead of a newline

Expand Down Expand Up @@ -138,6 +140,8 @@ struct Args {
line: Option<String>,
#[structopt(short = "s", help = "Execute new command for each bypassed chunk")]
solid: bool,
#[structopt(short = "I", help = "Replace the <replace-str> with bypassed chunk in the <command> then -s is forcefully enabled.")]
replace: Option<String>,
#[structopt(long = "chomp", help = "Command spawned by -s receives standard input without trailing newlines")]
solid_chomp: bool,
#[structopt(short = "v", help = "Invert the range of bypassing")]
Expand Down Expand Up @@ -179,8 +183,9 @@ fn main() {
let flag_only = args.only_matched;
let mut flag_regex = args.regex.is_some();
let flag_onig = args.onig_enabled;
let flag_solid = args.solid;
let mut flag_solid = args.solid;
let flag_solid_chomp = args.solid_chomp;
let flag_replace = args.replace.is_some();
let flag_invert = args.invert;
let flag_char = args.char.is_some();
let flag_lines = args.line.is_some();
Expand Down Expand Up @@ -353,9 +358,13 @@ fn main() {
process_each_line = false;
}

if flag_replace {
// If -I option is specified, enable -s option
flag_solid = true;
}
if flag_solid {
ch =
PipeIntercepter::start_solid_output(cmds, line_end, flag_dryrun, flag_solid_chomp)
PipeIntercepter::start_solid_output(cmds, line_end, flag_dryrun, flag_solid_chomp, args.replace)
.unwrap_or_else(|e| error_exit(&e.to_string()));
} else {
ch = PipeIntercepter::start_output(cmds, line_end, flag_dryrun)
Expand Down
19 changes: 15 additions & 4 deletions src/pipeintercepter.rs
Original file line number Diff line number Diff line change
Expand Up @@ -154,8 +154,11 @@ impl PipeIntercepter {
line_end: u8,
dryrun: bool,
chomp: bool,
replace_str: Option<String>,
) -> Result<PipeIntercepter, errors::SpawnError> {
let (tx, rx) = mpsc::channel();
let is_replace = replace_str.is_some();
let replace_str = replace_str.unwrap_or_else(|| "".to_string());
let handler = thread::spawn(move || {
debug!("thread: spawn");
let mut writer = BufWriter::new(io::stdout());
Expand All @@ -176,10 +179,18 @@ impl PipeIntercepter {
}
Chunk::SHole(msg) => {
debug!("thread: rx.recv <= SHole:[{:?}]", msg);
let result = spawnutils::exec_cmd_sync(msg, &cmds, line_end, chomp);
writer
.write(result.as_bytes())
.unwrap_or_else(|e| exit_silently(&e.to_string()));
// -I option
if is_replace {
let result = spawnutils::exec_cmd_sync_replace(msg, &cmds, line_end, chomp, replace_str.as_ref());
writer
.write(result.as_bytes())
.unwrap_or_else(|e| exit_silently(&e.to_string()));
} else {
let result = spawnutils::exec_cmd_sync(msg, &cmds, line_end, chomp);
writer
.write(result.as_bytes())
.unwrap_or_else(|e| exit_silently(&e.to_string()));
}
}
Chunk::EOF => {
debug!("thread: rx.recv <= EOF");
Expand Down
34 changes: 34 additions & 0 deletions src/spawnutils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,40 @@ pub fn exec_cmd(
))
}

/// Execute single command and return the stdout of the command as String synchronously
pub fn exec_cmd_sync_replace(input: String, cmds: &Vec<String>, line_end: u8, chomp: bool, replace_str: &str) -> String {
debug!("thread: exec_cmd_sync: {:?}", &cmds);
// check each element of cmds. If it contains replace_str, replace it with input
let mut cmds_new = Vec::new();
for cmd in cmds {
if cmd.contains(replace_str) {
cmds_new.push(cmd.replace(replace_str, &input));
} else {
cmds_new.push(cmd.to_string());
}
}
let child = Command::new(&cmds_new[0])
.args(&cmds_new[1..])
.stdout(Stdio::piped())
.spawn()
.expect("Failed to spawn child process");
let mut output = child
.wait_with_output()
.expect("Failed to read stdout")
.stdout;
if !chomp {
// Remove training new line.
// In the vast majority of cases,
// this new line is likely added by this function (see ADD NEW LINE)
if output.ends_with(&[line_end]) {
output.pop();
}
}
String::from_utf8_lossy(&output).to_string()
}



/// Execute single command and return the stdout of the command as String synchronously
pub fn exec_cmd_sync(input: String, cmds: &Vec<String>, line_end: u8, chomp: bool) -> String {
debug!("thread: exec_cmd_sync: {:?}", &cmds);
Expand Down
40 changes: 40 additions & 0 deletions tests/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -961,6 +961,46 @@ mod cmdtest {
.stdout("名前,作者@,ノート@\n1レコード目,\"あいう@えお\"@,かきく@\n2レコード目,\"さしす@せそ\"@,\"たちつ@てと,@@");
}

#[test]
fn test_solid_replace() {
let mut cmd = assert_cmd::Command::cargo_bin(env!("CARGO_PKG_NAME")).unwrap();
// separaate arguments -I @
cmd.args(&["-I", "@", "-s", "-f", "2", "--", _ECHO_CMD, ">>@<<"])
.write_stdin("AAA BBB CCC\nDDD EEE FFF\n")
.assert()
.stdout("AAA >>BBB<< CCC\nDDD >>EEE<< FFF\n");
}

#[test]
fn test_solid_replace_join() {
let mut cmd = assert_cmd::Command::cargo_bin(env!("CARGO_PKG_NAME")).unwrap();
// merged argument -I@
cmd.args(&["-I@", "-s", "-f", "2", "--", _ECHO_CMD, ">>@<<"])
.write_stdin("AAA BBB CCC\nDDD EEE FFF\n")
.assert()
.stdout("AAA >>BBB<< CCC\nDDD >>EEE<< FFF\n");

}

#[test]
fn test_solid_replace_multi() {
let mut cmd = assert_cmd::Command::cargo_bin(env!("CARGO_PKG_NAME")).unwrap();
// separaate arguments -I@{}
cmd.args(&["-I","**", "-c", "2-4", "--", _ECHO_CMD, "[**]"])
.write_stdin("AAA BBB CCC\nDDD EEE FFF\n")
.assert()
.stdout("A[AA ]BBB CCC\nD[DD ]EEE FFF\n");
}

#[test]
fn test_solid_replace_multi_join() {
let mut cmd = assert_cmd::Command::cargo_bin(env!("CARGO_PKG_NAME")).unwrap();
// merged argument -I{}
cmd.args(&["-I**", "-c", "2-4", "--", _ECHO_CMD, "[**]"])
.write_stdin("AAA BBB CCC\nDDD EEE FFF\n")
.assert()
.stdout("A[AA ]BBB CCC\nD[DD ]EEE FFF\n");
}
}
}

Expand Down