diff --git a/README.md b/README.md index 235bf1ee..d4943e95 100644 --- a/README.md +++ b/README.md @@ -117,6 +117,10 @@ the first line: } ``` +Bats, as part of its [Standard Library](#standard-library), provides +[assertion](#assertions) type helper functions that perform a test and +output useful details when the test fails. + ### `load`: Share common code You may want to share common code across multiple test files. Bats @@ -201,6 +205,419 @@ case. in the test file. * `$BATS_TMPDIR` is the location to a directory that may be used to store temporary files. +* `$BATS_LIB` is the directory in which the standard libraries are +located. + + +## Standard library + +The Standard Library is a collection of test helpers intended to +simplify testing. It helps building test suits that provide relevant +information when a test fails to speed up debugging. + +It contains the following types of test helpers. + +* [**Assertions**](#assertions) are functions that perform a test and + output relevant information on failure to help debugging. + +Test helpers send all output to the standard error, making them usable +outside of `@test` functions too. Output is [formatted](#output) for +readability. + +The Standard Library is automatically loaded and available in all test +files. + +### Output + +On failure, in addition to the usual output generated by Bats, +[**Assertions**](#assertions) display information relevant to the failed +test to help debugging. The output is formatted for readability and +displayed as key-value pairs on the standard error. + +When the value is one line long, the pair is displayed in a columnar +fashion called ***two-column*** format. + +``` +-- output differs -- +expected : want +actual : have +-- +``` + +When the value is longer than one line, the number of lines in the value +is displayed after the key, and the value starts on the next line and is +indented by two spaces for added readability. This is called +***multi-line*** format. + +For convenience, sometimes related values are also displayed in this +format even if they are only one line long. + +``` +-- output differs -- +expected (1 lines): + want +actual (3 lines): + have 1 + have 2 + have 3 +-- +``` + +### Assertions + +Assertions are functions that perform a test and output relevant +information on failure to help debugging. They return 1 on failure and 0 +otherwise. + +Assertions about exit code and output operate on the results of the most +recent invocation of `run`. + +#### `flunk` + +Display an error message and fail. This function provides a convenient +way to report failure in arbitrary situations. You can use it to +implement your own helpers when the ones available do not meet your +needs. Other functions use it internally as well. + +```bash +@test 'flunk()' { + flunk 'this test always fails' +} +``` + +The message can also be specified on the standard input. + +```bash +@test 'flunk() with pipe' { + echo 'this test always fails' | flunk +} +``` + +This function always fails and simply outputs the given message. + +``` +this test always fails +``` + +#### `assert` + +Fail if the given expression evaluates to false. + +```bash +@test 'assert()' { + local value=1 + assert [ "$value" -eq 0 ] +} +``` + +On failure the failed expression is displayed. + +``` +-- assertion failed -- +condition : [ 1 -eq 0 ] +-- +``` + +***Note:*** *The expression can be a simple command only. [Compund +commands] (https://www.gnu.org/software/bash/manual/bash.html#Compound-Commands), +such as `[[,` are not supported.* + +#### `assert_equal` + +Fail if the two parameters, expected and actual value respectively, do +not equal. + +```bash +@test 'assert_equal()' { + local expected='want' + local actual='have' + assert_equal "$expected" "$actual" +} +``` + +On failure the expected and actual values are displayed. + +``` +-- values do not equal -- +expected : want +actual : have +-- +``` + +If either value is longer than one line both are displayed in +*multi-line* format. + +#### `assert_output` + +Fail if `$output` does not equal the expected output. + +```bash +@test 'assert_output()' { + run echo 'have' + local expected='want' + assert_output "$expected" +} +``` + +The expected output can also be specified on the standard input. + +```bash +@test 'assert_output() with pipe' { + run echo 'have' + local expected='want' + echo "$expected" | assert_output +} +``` + +On failure the expected and actual outputs are displayed. + +``` +-- output differs -- +expected : want +actual : have +-- +``` + +If either value is longer than one line both are displayed in +*multi-line* format. + +#### `assert_success` + +This function tests `$status` and additionally `$output` depending on +the number of parameters. + +##### `$status` test + +When no parameters are specified, fail if `$status` is not zero. + +```bash +@test 'assert_success() status only' { + run bash -c "echo 'have'; exit 1" + assert_success +} +``` + +On failure `$status` and `$output` are displayed. + +``` +-- command failed -- +status : 1 +output : have +-- +``` + +If `$output` is longer than one line, it is displayed in *multi-line* +format. + +##### Optional `$output` test + +Additionally, if one parameter is specified and the status test passed, +`$output` is compared against the parameter as expected output. + +```bash +@test 'assert_success() status and output' { + run echo 'have' + local expected='want' + assert_success "$expected" +} +``` + +If the output test fails, the expected and actual outputs are displayed. + +``` +-- command succeeded, but output differs -- +expected : want +actual : have +-- +``` + +If either value is longer than one line both are displayed in +*multi-line* format. + +If the status test fails, the output test is not performed and the +output is identical to the status-only form's. + +#### `assert_failure` + +This function tests `$status` and additionally `$output` depending on +the number of parameters. + +##### `$status` test + +When no parameters are specified, fail if `$status` is zero. + +```bash +@test 'assert_failure() status only' { + run echo 'have' + assert_failure +} +``` + +On failure `$output` is displayed. + +``` +-- command succeeded, but it was expected to fail -- +output : have +-- +``` + +If `$output` is longer than one line, it is displayed in *multi-line* +format. + +##### Optional `$output` test + +Additionally, if one parameter is specified and the status test passed, +`$output` is compared against the parameter as expected output. + +```bash +@test 'assert_failure() status and output' { + run bash -c "echo 'have'; exit 1" + local expected='want' + assert_failure "$expected" +} +``` + +If the output test fails, the expected and actual outputs are displayed. + +``` +-- command failed as expected, but output differs -- +expected : want +actual : have +-- +``` + +If either value is longer than one line both are displayed in +*multi-line* format. + +If the status test fails, the output test is not performed and the +output is identical to the status-only form's. + +#### `assert_line` + +Depending on the number of parameters, this function tests either that +the output contains the expected line or that the expected line appears +in a specific line of the output identified by its index. + +This function is the opposite of `refute_line()`. + +***Note:*** *Due to a [bug in Bats][bats-93], empty lines are discarded, +causing line indices to change and preventing testing for empty lines.* + +##### Entire output + +When one parameter is specified, fail if `${lines[@]}` does not contain +the line specified by the parameter. + +```bash +@test 'assert_line() in entire output' { + run echo $'have 1\nhave 2\nhave 3' + assert_line 'want' +} +``` + +On failure `$output` and the expected line are displayed. + +``` +-- line is not in output -- +line : want +output (3 lines): + have 1 + have 2 + have 3 +-- +``` + +If `$output` is not longer than one line, it is displayed in +*two-column* format. + +##### Specific line + +When two parameters are specified, zero-based line index and expected +line respectively, fail if the output line identified by its index in +`${lines[@]}` does not equal the expected line. + +```bash +@test 'assert_line() in specific line' { + run echo $'have 1\nhave 2\nhave 3' + assert_line 1 'want' +} +``` + +On failure the index, and the expected and actual lines are displayed. + +``` +-- line differs -- +index : 1 +expected : want +actual : have 2 +-- +``` + +#### `refute_line` + +Depending on the number of parameters, this function tests either that +the output does not contain the unexpected line or that the unexpected +line does not appear in a specific line of the output identified by its +index. + +This function is the opposite of `assert_line()`. + +***Note:*** *Due to a [bug in Bats][bats-93], empty lines are discarded, +causing line indices to change and preventing testing for empty lines.* + + +##### Entire output + +When one parameter is specified, fail if `${lines[@]}` contains the line +specified by the parameter. + +```bash +@test 'refute_line() in entire output' { + run echo $'have 1\nwant\nhave 2' + refute_line 'want' +} +``` + +On failure the unexpected line, its zero-based index, and `$output` with +the unexpected line highlighted are displayed. + +``` +-- line should not be in output -- +line : want +index : 1 +output (3 lines): + have 1 +> want + have 2 +-- +``` + +If `$output` is not longer than one line, it is displayed in +*two-column* format. + +##### Specific line + +When two parameters are specified, zero-based line index and unexpected +line respectively, fail if the output line identified by its index in +`${lines[@]}` equals the unexpected line. + +```bash +@test 'refute_line() in specific line' { + run echo $'have 1\nwant\nhave 2' + refute_line 1 'want' +} +``` + +On failure the unexpected line and its index are displayed. + +``` +-- line should differ from expected -- +index : 1 +line : want +-- +``` + +[bats-93]: https://github.com/sstephenson/bats/pull/93 ## Installing Bats from source diff --git a/install.sh b/install.sh index 8bbdd16b..c10dc76d 100755 --- a/install.sh +++ b/install.sh @@ -31,7 +31,9 @@ BATS_ROOT="$(abs_dirname "$0")" mkdir -p "$PREFIX"/{bin,libexec,share/man/man{1,7}} cp -R "$BATS_ROOT"/bin/* "$PREFIX"/bin cp -R "$BATS_ROOT"/libexec/* "$PREFIX"/libexec +cp -R "$BATS_ROOT"/lib/* "$PREFIX"/lib cp "$BATS_ROOT"/man/bats.1 "$PREFIX"/share/man/man1 cp "$BATS_ROOT"/man/bats.7 "$PREFIX"/share/man/man7 +cp "$BATS_ROOT"/man/batslib.7 "$PREFIX"/share/man/man7 echo "Installed Bats to $PREFIX/bin/bats" diff --git a/lib/bats/batslib.bash b/lib/bats/batslib.bash new file mode 100644 index 00000000..fee8074d --- /dev/null +++ b/lib/bats/batslib.bash @@ -0,0 +1,300 @@ +# +# batslib.bash +# ------------ +# +# The Standard Library is a collection of test helpers intended to +# simplify testing. It contains the following types of test helpers. +# +# - Assertions are functions that perform a test and output relevant +# information on failure to help debugging. They return 1 on failure +# and 0 otherwise. +# +# All output is formatted for readability using the functions of +# `output.bash' and sent to the standard error. +# + +source "${BATS_LIB}/batslib/output.bash" + + +######################################################################## +# ASSERTIONS +######################################################################## + +# Fail and display an error message. The message is specified either by +# positional parameters or on the standard input (by piping or +# redirection). +# +# Globals: +# none +# Arguments: +# $@ - [opt = STDIN] message to display +# Returns: +# 1 - always +# Inputs: +# STDIN - [opt = $@] message to display +# Outputs: +# STDERR - error message +flunk() { + (( $# == 0 )) && batslib_err || batslib_err "$@" + return 1 +} + +# Fail and display the given condition if it evaluates to false. Only +# simple commands can be used in the expression given to `assert'. +# Compound commands, such as `[[', are not supported. +# +# Globals: +# none +# Arguments: +# $@ - condition to evaluate +# Returns: +# 0 - condition evaluated to TRUE +# 1 - condition evaluated to FALSE +# Outputs: +# STDERR - failed condition, on failure +assert() { + if ! "$@"; then + echo "condition : $@" | batslib_decorate 'assertion failed' | flunk + fi +} + +# Fail and display an error message if the two parameters, expected and +# actual value respectively, do not equal. The error message contains +# both parameters. +# +# Globals: +# none +# Arguments: +# $1 - expected value +# $2 - actual value +# Returns: +# 0 - expected equals actual value +# 1 - otherwise +# Outputs: +# STDERR - expected and actual value, on failure +assert_equal() { + if [[ $1 != "$2" ]]; then + batslib_print_kv_single_or_multi 8 \ + 'expected' "$1" \ + 'actual' "$2" \ + | batslib_decorate 'values do not equal' \ + | flunk + fi +} + +# Fail and display an error message if `$output' does not equal the +# expected output as specified either by the first positional parameter +# or on the standard input (by piping or redirection). The error message +# contains the expected and the actual output. +# +# Globals: +# output +# Arguments: +# $1 - [opt = STDIN] expected output +# Returns: +# 0 - expected equals actual output +# 1 - otherwise +# Inputs: +# STDIN - [opt = $1] expected output +# Outputs: +# STDERR - expected and actual output, on failure +assert_output() { + local expected + (( $# == 0 )) && expected="$(cat -)" || expected="$1" + if [[ $expected != "$output" ]]; then + batslib_print_kv_single_or_multi 8 \ + 'expected' "$expected" \ + 'actual' "$output" \ + | batslib_decorate 'output differs' \ + | flunk + fi +} + +# Fail and display an error message if `$status' is not zero. The error +# message contains `$status' and `$output'. +# +# Optionally, if the status check passed, `$output' can be tested +# against the first positional parameter as expected output. If the +# output check fails the error message contains the expected and the +# actual output. +# +# Globals: +# status +# output +# Arguments: +# $1 - [opt] expected output +# Returns: +# 0 - status and, optionally, output check passed +# 1 - otherwise +# Outputs: +# STDERR - `$status' and `$output', if status check fails +# expected and actual output, if output check fails +assert_success() { + (( $# > 0 )) && local -r expected="$1" + if (( status != 0 )); then + { local -ir width=6 + batslib_print_kv_single "$width" 'status' "$status" + batslib_print_kv_single_or_multi "$width" 'output' "$output" + } | batslib_decorate 'command failed' \ + | flunk + elif (( $# > 0 )) && [[ $output != "$1" ]]; then + batslib_print_kv_single_or_multi 8 \ + 'expected' "$expected" \ + 'actual' "$output" \ + | batslib_decorate 'command succeeded, but output differs' \ + | flunk + fi +} + +# Fail and display `$output' if `$status' is zero. +# +# Optionally, if the status check passed, `$output' can be tested +# against the first positional parameter as expected output. If the +# output check fails the error message contains the expected and the +# actual output. +# +# Globals: +# status +# output +# Arguments: +# $1 - [opt] expected output +# Returns: +# 0 - status and, optionally, output check passed +# 1 - otherwise +# Outputs: +# STDERR - `$output', if status check fails +# expected and actual output, if output check fails +assert_failure() { + if (( status == 0 )); then + batslib_print_kv_single_or_multi 6 'output' "$output" \ + | batslib_decorate 'command succeeded, but it was expected to fail' \ + | flunk + elif (( $# > 0 )) && [[ $output != "$1" ]]; then + batslib_print_kv_single_or_multi 8 \ + 'expected' "$1" \ + 'actual' "$output" \ + | batslib_decorate 'command failed as expected, but output differs' \ + | flunk + fi +} + +# Fail and display an error message if `${lines[@]}' does not contain +# the expected line. The error message contains the expected line and +# `$output'. +# +# Optionally, if two positional parameters are specified, the expected +# line is only sought in the line whose index is given in the first +# parameter. In this case, the error message contains the line index, +# and the expected and actual line at the given index. +# +# Globals: +# lines +# output +# Arguments: +# $1 - [opt] zero-based index of line to match against +# $2 - line to look for +# Returns: +# 0 - line found +# 1 - otherwise +# Outputs: +# STDERR - expected line and `$output', on failure +# index, expected and actual line at index, on failure +assert_line() { + if (( $# > 1 )); then + local -ir idx="$1" + local -r line="$2" + + if [[ ${lines[$idx]} != "$line" ]]; then + batslib_print_kv_single 8 \ + 'index' "$idx" \ + 'expected' "$line" \ + 'actual' "${lines[$idx]}" \ + | batslib_decorate 'line differs' \ + | flunk + fi + else + local -r line="$1" + local temp_line + + for temp_line in "${lines[@]}"; do + [[ $temp_line == "$line" ]] && return 0 + done + { local -ar single=( + 'line' "$line" + ) + local -ar may_be_multi=( + 'output' "$output" + ) + local -ir width="$( batslib_get_max_single_line_key_width \ + "${single[@]}" "${may_be_multi[@]}" )" + batslib_print_kv_single "$width" "${single[@]}" + batslib_print_kv_single_or_multi "$width" "${may_be_multi[@]}" + } | batslib_decorate 'line is not in output' \ + | flunk + fi +} + +# Fail and display an error message if `${lines[@]}' contains the given +# line. The error message contains the unexpected line, its index in +# `$output', and `$output'. +# +# Optionally, if two positional parameters are specified, the unexpected +# line is only sought in the line whose index is given in the first +# parameter. In this case, the error message contains the line index, +# and the unexpected line. +# +# Globals: +# lines +# output +# Arguments: +# $1 - [opt] zero-based index of line to match against +# $2 - line to look for +# Returns: +# 0 - line not found +# 1 - otherwise +# Outputs: +# STDERR - unexpected line, its index and `$output', on failure +# index and unexpected line, on failure +refute_line() { + if (( $# > 1 )); then + local -ir idx="$1" + local -r line="$2" + + if [[ ${lines[$idx]} == "$line" ]]; then + batslib_print_kv_single 5 \ + 'index' "$idx" \ + 'line' "$line" \ + | batslib_decorate 'line should differ from expected' \ + | flunk + fi + else + local -r line="$1" + + local idx + for (( idx = 0; idx < ${#lines[@]}; ++idx )); do + if [[ ${lines[$idx]} == "$line" ]]; then + { local -ar single=( + 'line' "$line" + 'index' "$idx" + ) + local -a may_be_multi=( + 'output' "$output" + ) + local -ir width="$( batslib_get_max_single_line_key_width \ + "${single[@]}" "${may_be_multi[@]}" )" + batslib_print_kv_single "$width" "${single[@]}" + if batslib_is_single_line "${may_be_multi[1]}"; then + batslib_print_kv_single "$width" "${may_be_multi[@]}" + else + may_be_multi[1]="$( printf '%s' "${may_be_multi[1]}" \ + | batslib_prefix \ + | batslib_mark '>' "$idx" )" + batslib_print_kv_multi "${may_be_multi[@]}" + fi + } | batslib_decorate 'line should not be in output' \ + | flunk + return 1 + fi + done + fi +} diff --git a/lib/bats/batslib/output.bash b/lib/bats/batslib/output.bash new file mode 100644 index 00000000..95cef47d --- /dev/null +++ b/lib/bats/batslib/output.bash @@ -0,0 +1,256 @@ +# +# output.bash +# ----------- +# +# Private functions implementing output formatting. Used by public +# helper functions. +# + +# Print an error message to the standard error. The message is specified +# either by positional parameters or on the standard input (by piping or +# redirection). +# +# Globals: +# none +# Arguments: +# $@ - [opt = STDIN] message to print +# Returns: +# none +# Inputs: +# STDIN - [opt = $@] message to print +# Outputs: +# STDERR - error message +batslib_err() { + { if (( $# > 0 )); then + echo "$@" + else + cat - + fi + } >&2 +} + +# Counts the number of lines in the given text. +# +# TODO(ztombol): Remove this notice and fix tests after Bats merged #93! +# NOTE: Due to a bug in Bats, `batslib_count_lines "$output"' does not +# give the same result as `${#lines[@]}' when the output contains +# empty lines. +# See PR #93 (https://github.com/sstephenson/bats/pull/93). +# +# Globals: +# none +# Arguments: +# $1 - text +# Returns: +# none +# Outputs: +# STDOUT - number of lines in given text +batslib_count_lines() { + local -i n_lines=0 + local line + while IFS='' read -r line || [[ -n $line ]]; do + (( ++n_lines )) + done < <(printf '%s' "$1") + echo "$n_lines" +} + +# Determine whether all parameters are single-line strings. +# +# Globals: +# none +# Arguments: +# $@ - strings to test +# Returns: +# 0 - all parameters are single-line strings +# 1 - otherwise +batslib_is_single_line() { + for string in "$@"; do + (( $(batslib_count_lines "$string") > 1 )) && return 1 + done + return 0 +} + +# Determine the length of the longest key that has a single-line value. +# Useful in determining the column width for printing key-value pairs in +# a two-column format when some keys may have multi-line values and thus +# should be excluded. +# +# Globals: +# none +# Arguments: +# $@ - strings to test +# Returns: +# none +# Outputs: +# STDOUT - length of longest key +batslib_get_max_single_line_key_width() { + local -i max_len=-1 + while (( $# != 0 )); do + local -i key_len="${#1}" + batslib_is_single_line "$2" && (( key_len > max_len )) && max_len="$key_len" + shift 2 + done + echo "$max_len" +} + +# Print key-value pairs in two-column format. The first column contains +# the keys. Its width is specified with the first positional parameter, +# usually acquired using `batslib_get_max_single_line_key_width()', to +# nicely line up the values in the second column. The rest of the +# parameters are used to supply the key-value pairs. Every even-numbered +# parameter is a key and the following parameter is its value. +# +# Globals: +# none +# Arguments: +# $1 - column width +# $even - key +# $odd - value of the previous key +# Returns: +# none +# Outputs: +# STDOUT - key-value pairs in two-column format +batslib_print_kv_single() { + local -ir col_width="$1"; shift + while (( $# != 0 )); do + printf '%-*s : %s\n' "$col_width" "$1" "$2" + shift 2 + done +} + +# Print key-value pairs in multi-line format. First, the key and the +# number of lines in the value is printed. Next, the value on a separate +# line. Every odd-numbered parameter is a key and the following +# parameters is its value. +# +# Globals: +# none +# Arguments: +# $odd - key +# $even - value of the previous key +# Returns: +# none +# Outputs: +# STDOUT - key-value pairs in multi-line format +batslib_print_kv_multi() { + while (( $# != 0 )); do + printf '%s (%d lines):\n' "$1" "$( batslib_count_lines "$2" )" + printf '%s\n' "$2" + shift 2 + done +} + +# Print all key-value pairs in either two-column or multi-line format. +# If all values are one line long, all pairs are printed in two-column +# format using `batslib_print_kv_single()'. Otherwise, they are printed +# in multi-line format using `batslib_print_kv_multi()' after each line +# of all values being prefixed with two spaces. +# +# Globals: +# none +# Arguments: +# $1 - column width for two-column format +# $even - key +# $odd - value of the previous key +# Returns: +# none +# Outputs: +# STDOUT - key-value pairs in two-column format, if all values are +# single-line +# key-value pairs in multi-line format, otherwise +batslib_print_kv_single_or_multi() { + local -ir width="$1"; shift + local -a pairs=( "$@" ) + + local -a values=() + local -i i + for (( i=1; i < ${#pairs[@]}; i+=2 )); do + values+=( "${pairs[$i]}" ) + done + + if batslib_is_single_line "${values[@]}"; then + batslib_print_kv_single "$width" "${pairs[@]}" + else + local -i i + for (( i=1; i < ${#pairs[@]}; i+=2 )); do + pairs[$i]="$( batslib_prefix < <(printf '%s' "${pairs[$i]}") )" + done + batslib_print_kv_multi "${pairs[@]}" + fi +} + +# Prefix each line of the input with an arbitrary string. +# +# Globals: +# none +# Arguments: +# $1 - [opt = ' '] prefix string +# Returns: +# none +# Inputs: +# STDIN - lines to prefix +# Outputs: +# STDOUT - prefixed lines +batslib_prefix() { + local -r prefix="${1:- }" + local line + while IFS='' read -r line || [[ -n $line ]]; do + printf '%s%s\n' "$prefix" "$line" + done +} + +# Mark select lines of the input by overwriting their first few +# characters with an arbitrary string. Usually, the input is indented by +# spaces first using `batslib_prefix()'. +# +# Globals: +# none +# Arguments: +# $1 - marking string +# $@ - zero-based indices of lines to mark +# Returns: +# none +# Inputs: +# STDIN - lines to work on +# Outputs: +# STDOUT - lines after marking +batslib_mark() { + local -r symbol="$1"; shift + # Sort line numbers. + set -- $( sort -nu <<< "$( printf '%d\n' "$@" )" ) + + local line + local -i idx=0 + while IFS='' read -r line || [[ -n $line ]]; do + if (( ${1:--1} == idx )); then + printf '%s\n' "${symbol}${line:${#symbol}}" + shift + else + printf '%s\n' "$line" + fi + (( ++idx )) + done +} + +# Enclose the input in header and footer lines. The header contains an +# arbitrary title specified with the first positional parameter. The +# output is preceded and followed by an additional newline to make it +# stand out more. +# +# Globals: +# none +# Arguments: +# $1 - title +# Returns: +# none +# Inputs: +# STDIN - text to enclose +# Outputs: +# STDOUT - enclosed text +batslib_decorate() { + echo + echo "-- $1 --" + cat - + echo '--' + echo +} diff --git a/libexec/bats b/libexec/bats index 71f392f7..f8a40cef 100755 --- a/libexec/bats +++ b/libexec/bats @@ -54,6 +54,7 @@ expand_path() { BATS_LIBEXEC="$(abs_dirname "$0")" export BATS_PREFIX="$(abs_dirname "$BATS_LIBEXEC")" +export BATS_LIB="${BATS_PREFIX}/lib/bats" export BATS_CWD="$(abs_dirname .)" export PATH="$BATS_LIBEXEC:$PATH" diff --git a/libexec/bats-exec-test b/libexec/bats-exec-test index 8f3bd510..452dfbe5 100755 --- a/libexec/bats-exec-test +++ b/libexec/bats-exec-test @@ -331,6 +331,9 @@ bats_evaluate_preprocessed_source() { exec 3<&1 +# Load Standard Library. +load "${BATS_LIB}/batslib.bash" + if [ "$#" -eq 0 ]; then bats_preprocess_source bats_evaluate_preprocessed_source diff --git a/libexec/std/core.bash b/libexec/std/core.bash new file mode 100644 index 00000000..c04b904a --- /dev/null +++ b/libexec/std/core.bash @@ -0,0 +1,273 @@ +# +# core.bash +# ----------- +# +# A collection of helper functions implementing common assertions about exit +# codes and outputs. When an assertion fails, relevant information is printed +# on STDERR to ease debugging. +# +# Assertions about exit code and output operate on the results of the most +# recent invocation of `run()' by accessing the global variables `$status', +# `$output' and `${lines[@]}' set by `run()'. +# +# Assertions return 1 when they fail, and 0 when they succeed. +# + +source "${BATS_LIBSTD}/internal/output.bash" + + +# Fail and display an error message. The message is specified either by +# positional parameters or on the standard input (by piping or redirection). +# +# Globals: +# none +# Arguments: +# $@ - [opt = STDIN] message to display +# Returns: +# 1 - always +# Inputs: +# STDIN - [opt = $@] message to display +# Outputs: +# STDERR - error message +flunk () { + (( $# == 0 )) && _err || _err "$@" + return 1 +} + +# Fail and display the given condition if it evaluates to false. +# +# Globals: +# none +# Arguments: +# $@ - condition to evaluate +# Returns: +# 0 - condition evaluated to TRUE +# 1 - condition evaluated to FALSE +# Outputs: +# STDERR - failed condition, on failure +assert () { + if ! "$@"; then + echo "condition : $@" | _decorate 'assertion failed' | flunk + fi +} + +# Fail and display an error message if the two parameters, expected and actual +# value respectively, do not equal. The error message contains both parameters. +# +# Globals: +# none +# Arguments: +# $1 - expected value +# $2 - actual value +# Returns: +# 0 - expected equals actual value +# 1 - otherwise +# Outputs: +# STDERR - expected and actual value, on failure +assert_equal () { + if [[ $1 != "$2" ]]; then + _print_kv_single_or_multi 8 'expected' "$1" \ + 'actual' "$2" \ + | _decorate 'values do not equal' | flunk + fi +} + +# Fail and display an error message if `$output' does not equal the expected +# output as specified either by the first positional parameter or on the +# standard input (by piping or redirection). The error message contains the +# expected and the actual output. +# +# Globals: +# output +# Arguments: +# $1 - [opt = STDIN] expected output +# Returns: +# 0 - expected equals actual output +# 1 - otherwise +# Inputs: +# STDIN - [opt = $1] expected output +# Outputs: +# STDERR - expected and actual output, on failure +assert_output () { + local expected + (( $# == 0 )) && expected="$(cat -)" || expected="$1" + if [[ $expected != "$output" ]]; then + _print_kv_single_or_multi 8 'expected' "$expected" \ + 'actual' "$output" \ + | _decorate 'output differs' | flunk + fi +} + +# Fail and display an error message if `$status' is not zero. The error message +# contains `$status' and `$output'. +# +# Optionally, if the status check passed, `$output' can be tested against the +# first positional parameter as expected output. If the output check fails the +# error message contains the expected and the actual output. +# +# Globals: +# status +# output +# Arguments: +# $1 - [opt] expected output +# Returns: +# 0 - status and, optionally, output check passed +# 1 - otherwise +# Outputs: +# STDERR - `$status' and `$output', if status check fails +# expected and actual output, if output check fails +assert_success () { + (( $# > 0 )) && local -r expected="$1" + if (( status != 0 )); then + { local -ir width=6 + _print_kv_single "$width" 'status' "$status" + _print_kv_single_or_multi "$width" 'output' "$output" + } | _decorate 'command failed' | flunk + elif (( $# > 0 )) && [[ $output != "$1" ]]; then + _print_kv_single_or_multi 8 'expected' "$expected" \ + 'actual' "$output" \ + | _decorate 'command succeeded, but output differs' | flunk + fi +} + +# Fail and display `$output' if `$status' is zero. +# +# Optionally, if the status check passed, `$output' can be tested against the +# first positional parameter as expected output. If the output check fails the +# error message contains the expected and the actual output. +# +# Globals: +# status +# output +# Arguments: +# $1 - [opt] expected output +# Returns: +# 0 - status and, optionally, output check passed +# 1 - otherwise +# Outputs: +# STDERR - `$output', if status check fails +# expected and actual output, if output check fails +assert_failure () { + if (( status == 0 )); then + _print_kv_single_or_multi 6 'output' "$output" \ + | _decorate 'command succeeded, but it was expected to fail' | flunk + elif (( $# > 0 )) && [[ $output != "$1" ]]; then + _print_kv_single_or_multi 8 'expected' "$1" \ + 'actual' "$output" \ + | _decorate 'command failed as expected, but output differs' | flunk + fi +} + +# Fail and display an error message if `${lines[@]}' does not contain the +# expected line. The error message contains the expected line and `$output'. +# +# Optionally, if two positional parameters are specified, the expected line is +# only sought in the line whose index is given in the first parameter. In this +# case, the error message contains the line index, and the expected and actual +# line at the given index. +# +# Globals: +# lines +# output +# Arguments: +# $1 - [opt] zero-based index of line to match against +# $2 - line to look for +# Returns: +# 0 - line found +# 1 - otherwise +# Outputs: +# STDERR - expected line and `$output', on failure +# index, expected and actual line at index, on failure +assert_line () { + if (( $# > 1 )); then + local -ir idx="$1" + local -r line="$2" + + if [[ ${lines[$idx]} != "$line" ]]; then + _print_kv_single 8 'index' "$idx" \ + 'expected' "$line" \ + 'actual' "${lines[$idx]}" \ + | _decorate 'line differs' | flunk + fi + else + local -r line="$1" + local temp_line + + for temp_line in "${lines[@]}"; do + [[ $temp_line == "$line" ]] && return 0 + done + { local -ar single=( + 'line' "$line" + ) + local -ar may_be_multi=( + 'output' "$output" + ) + local -ir width="$(_get_max_single_line_key_width \ + "${single[@]}" "${may_be_multi[@]}")" + _print_kv_single "$width" "${single[@]}" + _print_kv_single_or_multi "$width" "${may_be_multi[@]}" + } | _decorate 'line is not in output' | flunk + fi +} + +# Fail and display an error message if `${lines[@]}' contains the given line. +# The error message contains the unexpected line, its index in `$output', and +# `$output'. +# +# Optionally, if two positional parameters are specified, the unexpected line +# is only sought in the line whose index is given in the first parameter. In +# this case, the error message contains the line index, and the unexpected +# line. +# +# Globals: +# lines +# output +# Arguments: +# $1 - [opt] zero-based index of line to match against +# $2 - line to look for +# Returns: +# 0 - line not found +# 1 - otherwise +# Outputs: +# STDERR - unexpected line, its index and `$output', on failure +# index and unexpected line, on failure +refute_line () { + if (( $# > 1 )); then + local -ir idx="$1" + local -r line="$2" + + if [[ ${lines[$idx]} == "$line" ]]; then + _print_kv_single 5 'index' "$idx" \ + 'line' "$line" \ + | _decorate 'line should differ from expected' | flunk + fi + else + local -r line="$1" + + local idx + for (( idx = 0; idx < ${#lines[@]}; ++idx )); do + if [[ ${lines[$idx]} == "$line" ]]; then + { local -ar single=( + 'line' "$line" + 'index' "$idx" + ) + local -a may_be_multi=( + 'output' "$output" + ) + local -ir width="$( _get_max_single_line_key_width \ + "${single[@]}" "${may_be_multi[@]}" )" + _print_kv_single "$width" "${single[@]}" + if _is_single_line "${may_be_multi[1]}"; then + _print_kv_single "$width" "${may_be_multi[@]}" + else + may_be_multi[1]="$( printf '%s' "${may_be_multi[1]}" \ + | _prefix \ + | _mark '>' "$idx" )" + _print_kv_multi "${may_be_multi[@]}" + fi + } | _decorate 'line should not be in output' | flunk + return 1 + fi + done + fi +} diff --git a/libexec/std/internal/output.bash b/libexec/std/internal/output.bash new file mode 100644 index 00000000..c1b265c8 --- /dev/null +++ b/libexec/std/internal/output.bash @@ -0,0 +1,251 @@ +# +# output.bash +# ----------- +# +# Private functions implementing output formatting. Used by public helper +# functions. +# + + +# Print an error message to the standard error. The message is specified either +# by positional parameters or on the standard input (by piping or redirection). +# +# Globals: +# none +# Arguments: +# $@ - [opt = STDIN] message to print +# Returns: +# none +# Inputs: +# STDIN - [opt = $@] message to print +# Outputs: +# STDERR - error message +_err () { + { if (( $# > 0 )); then + echo "$@" + else + cat - + fi + } >&2 +} + +# Counts the number of lines in the given text. +# +# FIXME: Remove this notice and fix tests after Bats merged #93! +# NOTE: Due to a bug in Bats, `_count_lines "$output"' does not give the same +# result as `${#lines[@]}' when the output contains empty lines. +# See PR #93 (https://github.com/sstephenson/bats/pull/93). +# +# Globals: +# none +# Arguments: +# $1 - text +# Returns: +# none +# Outputs: +# STDOUT - number of lines in given text +_count_lines () { + local -i n_lines=0 + local line + while IFS='' read -r line || [[ -n $line ]]; do + (( ++n_lines )) + done < <(printf '%s' "$1") + echo "$n_lines" +} + +# Determine whether all parameters are single-line strings. +# +# Globals: +# none +# Arguments: +# $@ - strings to test +# Returns: +# 0 - all parameters are single-line strings +# 1 - otherwise +_is_single_line () { + for string in "$@"; do + (( $(_count_lines "$string") > 1 )) && return 1 + done + return 0 +} + +# Determine the length of the longest key that has a single-line value. Useful +# in determining the column width for printing key-value pairs in a two-column +# format when some keys may have multi-line values and thus should be excluded. +# +# Globals: +# none +# Arguments: +# $@ - strings to test +# Returns: +# none +# Outputs: +# STDOUT - length of longest key +_get_max_single_line_key_width () { + local -i max_len=-1 + while (( $# != 0 )); do + local -i key_len="${#1}" + _is_single_line "$2" && (( key_len > max_len )) && max_len="$key_len" + shift 2 + done + echo "$max_len" +} + +# Print key-value pairs in two-column format. The first column contains the +# keys. Its width is specified with the first positional parameter, usually +# acquired using `_get_max_single_line_key_width()', to nicely line up the +# values in the second column. The rest of the parameters are used to supply +# the key-value pairs. Every even-numbered parameter is a key and the following +# parameter is its value. +# +# Globals: +# none +# Arguments: +# $1 - column width +# $even - key +# $odd - value of the previous key +# Returns: +# none +# Outputs: +# STDOUT - key-value pairs in two-column format +_print_kv_single () { + local -ir col_width="$1"; shift + while (( $# != 0 )); do + printf '%-*s : %s\n' "$col_width" "$1" "$2" + shift 2 + done +} + +# Print key-value pairs in multi-line format. First, the key and the number of +# lines in the value is printed. Next, the value on a separate line. Every +# odd-numbered parameter is a key and the following parameters is its value. +# +# Globals: +# none +# Arguments: +# $odd - key +# $even - value of the previous key +# Returns: +# none +# Outputs: +# STDOUT - key-value pairs in multi-line format +_print_kv_multi () { + while (( $# != 0 )); do + printf '%s (%d lines):\n' "$1" "$( _count_lines "$2" )" + printf '%s\n' "$2" + shift 2 + done +} + +# Print all key-value pairs in either two-column or multi-line format. If all +# values are one line long, all pairs are printed in two-column format using +# `_print_kv_single()'. Otherwise, they are printed in multi-line format using +# `_print_kv_multi()' after each line of all values being prefixed with two +# spaces. +# +# Globals: +# none +# Arguments: +# $1 - column width for two-column format +# $even - key +# $odd - value of the previous key +# Returns: +# none +# Outputs: +# STDOUT - key-value pairs in two-column format, if values are single-line +# key-value pairs in multi-line format, otherwise +_print_kv_single_or_multi () { + local -ir width="$1"; shift + local -a pairs=( "$@" ) + + local -a values=() + local -i i + for (( i=1; i < ${#pairs[@]}; i+=2 )); do + values+=( "${pairs[$i]}" ) + done + + if _is_single_line "${values[@]}"; then + _print_kv_single "$width" "${pairs[@]}" + else + local -i i + for (( i=1; i < ${#pairs[@]}; i+=2 )); do + pairs[$i]="$( _prefix < <(printf '%s' "${pairs[$i]}") )" + done + _print_kv_multi "${pairs[@]}" + fi +} + +# Prefix each line of the input with an arbitrary string. +# +# Globals: +# none +# Arguments: +# $1 - [opt = ' '] prefix string +# Returns: +# none +# Inputs: +# STDIN - lines to prefix +# Outputs: +# STDOUT - prefixed lines +_prefix () { + local -r prefix="${1:- }" + local line + while IFS='' read -r line || [[ -n $line ]]; do + printf '%s%s\n' "$prefix" "$line" + done +} + +# Mark select lines of the input by overwriting their first few characters with +# an arbitrary string. Usually, the input is indented by spaces first using +# `_prefix()'. +# +# Globals: +# none +# Arguments: +# $1 - marking string +# $@ - zero-based indices of lines to mark +# Returns: +# none +# Inputs: +# STDIN - lines to work on +# Outputs: +# STDOUT - lines after marking +_mark () { + local -r symbol="$1"; shift + # Sort line numbers. + set -- $( sort -gu <<< "$( printf '%d\n' "$@" )" ) + + local line + local -i idx=0 + while IFS='' read -r line || [[ -n $line ]]; do + if (( ${1:--1} == idx )); then + printf '%s\n' "${symbol}${line:${#symbol}}" + shift + else + printf '%s\n' "$line" + fi + (( ++idx )) + done +} + +# Enclose the input in header and footer lines. The header contains an +# arbitrary title specified with the first positional parameter. The output is +# preceded and followed by an additional newline to make it stand out more. +# +# Globals: +# none +# Arguments: +# $1 - title +# Returns: +# none +# Inputs: +# STDIN - text to enclose +# Outputs: +# STDOUT - enclosed text +_decorate () { + echo + echo "-- $1 --" + cat - + echo '--' + echo +} diff --git a/man/Makefile b/man/Makefile index b3a44bdb..8d60eb69 100644 --- a/man/Makefile +++ b/man/Makefile @@ -1,5 +1,5 @@ RONN := ronn -PAGES := bats.1 bats.7 +PAGES := bats.1 bats.7 batslib.7 all: $(PAGES) @@ -8,3 +8,6 @@ bats.1: bats.1.ronn bats.7: bats.7.ronn $(RONN) -r $< + +batslib.7: batslib.7.ronn + $(RONN) -r $< diff --git a/man/bats.1 b/man/bats.1 index 82f73016..83c08009 100644 --- a/man/bats.1 +++ b/man/bats.1 @@ -1,7 +1,7 @@ .\" generated with Ronn/v0.7.3 .\" http://github.com/rtomayko/ronn/tree/0.7.3 . -.TH "BATS" "1" "August 2014" "" "" +.TH "BATS" "1" "August 2015" "" "" . .SH "NAME" \fBbats\fR \- Bash Automated Testing System @@ -92,7 +92,7 @@ The \fBbats\fR interpreter exits with a value of \fB0\fR if all test cases pass, Bats wiki: \fIhttps://github\.com/sstephenson/bats/wiki/\fR . .P -\fBbash\fR(1), \fBbats\fR(7) +\fBbash\fR(1), \fBbats\fR(7), \fBbatslib\fR(7) . .SH "COPYRIGHT" (c) 2014 Sam Stephenson diff --git a/man/bats.1.ronn b/man/bats.1.ronn index bd8f45b5..beb773ae 100644 --- a/man/bats.1.ronn +++ b/man/bats.1.ronn @@ -95,7 +95,7 @@ SEE ALSO Bats wiki: _https://github.com/sstephenson/bats/wiki/_ -`bash`(1), `bats`(7) +`bash`(1), `bats`(7), `batslib`(7) COPYRIGHT diff --git a/man/bats.7 b/man/bats.7 index d0836e52..43970c06 100644 --- a/man/bats.7 +++ b/man/bats.7 @@ -1,7 +1,7 @@ .\" generated with Ronn/v0.7.3 .\" http://github.com/rtomayko/ronn/tree/0.7.3 . -.TH "BATS" "7" "November 2013" "" "" +.TH "BATS" "7" "August 2015" "" "" . .SH "NAME" \fBbats\fR \- Bats test file format @@ -88,6 +88,25 @@ load test_helper .P will source the script \fBtest/test_helper\.bash\fR in your test file\. This can be useful for sharing functions to set up your environment or load fixtures\. . +.P +THE LOAD_STD COMMAND +. +.P +Common test helpers are compiled into libraries and included in the \fIStandard Library\fR for convenience\. To load a library use the \fBload_std\fR command\. For example, the command below loads the \fICore\fR library in your test file\. +. +.IP "" 4 +. +.nf + +load_std \'core\' +. +.fi +. +.IP "" 0 +. +.P +See the documentation of the Standard Library for the available libraries and the test helpers they provide\. +. .SH "THE SKIP COMMAND" Tests can be skipped by using the \fBskip\fR command at the point in a test you wish to skip\. . @@ -175,4 +194,4 @@ There are several global variables you can use to introspect on Bats tests: .IP "" 0 . .SH "SEE ALSO" -\fBbash\fR(1), \fBbats\fR(1) +\fBbash\fR(1), \fBbats\fR(1), \fBbatslib\fR(7) diff --git a/man/bats.7.ronn b/man/bats.7.ronn index 7f6dd184..aa2fcbda 100644 --- a/man/bats.7.ronn +++ b/man/bats.7.ronn @@ -78,6 +78,19 @@ can be useful for sharing functions to set up your environment or load fixtures. +THE LOAD_STD COMMAND + +Common test helpers are compiled into libraries and included in the +*Standard Library* for convenience. To load a library use the `load_std` +command. For example, the command below loads the *Core* library in your +test file. + + load_std 'core' + +See the documentation of the Standard Library for the available +libraries and the test helpers they provide. + + THE SKIP COMMAND ---------------- @@ -153,4 +166,4 @@ store temporary files. SEE ALSO -------- -`bash`(1), `bats`(1) +`bash`(1), `bats`(1), `batslib`(7) diff --git a/man/batslib.7 b/man/batslib.7 new file mode 100644 index 00000000..4572bc44 --- /dev/null +++ b/man/batslib.7 @@ -0,0 +1,139 @@ +.\" generated with Ronn/v0.7.3 +.\" http://github.com/rtomayko/ronn/tree/0.7.3 +. +.TH "BATSLIB" "7" "August 2015" "" "" +. +.SH "NAME" +\fBbatslib\fR \- Bats Standard Library of Test Helpers +. +.SH "DESCRIPTION" +The Standard Library is a collection of test helpers intended to simplify testing\. It helps building test suits that provide relevant information when a test fails to speed up debugging\. +. +.P +It contains the following types of test helpers\. +. +.IP "\(bu" 4 +\fIASSERTIONS\fR are functions that perform a test and output relevant information on failure to help debugging\. +. +.IP "" 0 +. +.P +Test helpers send all output to the standard error, making them usable outside of \fB@test\fR functions too\. Output is formatted for readability as described in \fIOUTPUT\fR\. +. +.P +The Standard Library is automatically loaded and available in all test files\. +. +.SH "OUTPUT" +On failure, in addition to the usual output generated by Bats, \fIASSERTIONS\fR display information relevant to the failed test to help debugging\. The output is formatted for readability and displayed as key\-value pairs on the standard error\. +. +.P +When the value is one line long, the pair is displayed in a columnar fashion called \fBtwo\-column\fR format\. +. +.IP "" 4 +. +.nf + +\-\- output differs \-\- +expected : want +actual : have +\-\- +. +.fi +. +.IP "" 0 +. +.P +When the value is longer than one line, the number of lines in the value is displayed after the key, and the value starts on the next line and is indented by two spaces for added readability\. This is called \fBmulti\-line\fR format\. +. +.P +For convenience, sometimes related values are also displayed in this format even if they are only one line long\. +. +.IP "" 4 +. +.nf + +\-\- output differs \-\- +expected (1 lines): + want +actual (3 lines): + have 1 + have 2 + have 3 +\-\- +. +.fi +. +.IP "" 0 +. +.SH "ASSERTIONS" +Assertions are functions that perform a test and output relevant information on failure to help debugging\. They return 1 on failure and 0 otherwise\. +. +.P +Assertions about exit code and output operate on the results of the most recent invocation of \fBrun\fR\. +. +.TP +\fBflunk\fR [\fIMESSAGE\fR] +Display \fIMESSAGE\fR and fail\. This function provides a convenient way to report various failures\. Other test helpers also use it to display output on failure\. When no parameters are specified, \fIMESSAGE\fR is read from the standard input\. +. +.TP +\fBassert\fR \fIEXPRESSION\fR +Fail if \fIEXPRESSION\fR evaluates to false\. On failure, \fIEXPRESSION\fR is displayed\. +. +.IP +\fBNote:\fR \fIEXPRESSION\fR can only be a simple command\. Compound commands, such as \fB[[\fR, are not supported\. +. +.TP +\fBassert_equal\fR \fIEXPECTED\fR \fIACTUAL\fR +Fail if \fIEXPECTED\fR and \fIACTUAL\fR do not equal\. On failure both values are displayed\. If either one is longer than one line, both are displayed in \fImulit\-line\fR format\. +. +.TP +\fBassert_output\fR [\fIEXPECTED\fR] +Fail if \fIEXPECTED\fR does not equal \fB$output\fR\. On failure both values are displayed\. If either one is longer than one line, both are displayed in \fImulit\-line\fR format\. When no parameters are specified, \fIEXPECTED\fR is read from the standard input\. +. +.TP +\fBassert_success\fR [\fIEXPECTED\fR] +When no parameters are specified, fail if \fB$status\fR is not \fB0\fR\. On failure \fB$status\fR and \fB$output\fR are displayed\. If \fB$output\fR is longer than one line, it is displayed in \fImulit\-line\fR format\. +. +.IP +When \fIEXPECTED\fR is specified and \fB$status\fR is \fB0\fR, fail if \fIEXPECTED\fR does not equal \fB$output\fR\. On failure both values are displayed\. If either one is longer than one line, both are displayed in \fImulti\-line\fR format\. +. +.TP +\fBassert_failure\fR [\fIEXPECTED\fR] +When no parameters are specified, fail if \fB$status\fR is \fB0\fR\. On failure \fB$output\fR is displayed\. If \fB$output\fR is longer than one line, it is displayed in \fImulit\-line\fR format\. +. +.IP +When \fIEXPECTED\fR is specified and \fB$status\fR is not \fB0\fR, fail if \fIEXPECTED\fR does not equal \fB$output\fR\. On failure both values are displayed\. If either one is longer than one line, both are displayed in \fImulti\-line\fR format\. +. +.TP +\fBassert_line\fR [\fIINDEX\fR] \fILINE\fR +When only \fILINE\fR is specified, fail if \fB${lines[@]}\fR does not contain \fILINE\fR\. On failure \fILINE\fR and \fB$output\fR are displayed\. If \fB$output\fR is longer than one line, it is displayed in \fImulit\-line\fR format\. +. +.IP +When \fIINDEX\fR is specified, fail if \fILINE\fR does not equal \fB${lines[INDEX]}\fR\. On failure \fIINDEX\fR, \fILINE\fR and \fB${lines[INDEX]}\fR are dispalyed\. +. +.IP +\fBNOTE:\fR Due to a bug in Bats, empty lines are discarded, causing line indices to change and preventing testing for empty lines\. See \fIBUGS\fR for more\. +. +.TP +\fBrefute_line\fR [\fIINDEX\fR] \fILINE\fR +When only \fILINE\fR is specified, fail if \fB${lines[@]}\fR contains \fILINE\fR\. On failure \fILINE\fR, its zero\-based index in \fB${lines[@]}\fR, and \fB$output\fR are dispalyed\. If \fB$output\fR is longer than one line, it is displayed in \fImulit\-line\fR format with \fILINE\fR highlighted\. +. +.IP +When \fIINDEX\fR is specified, fail if \fILINE\fR equals \fB${lines[INDEX]}\fR\. On failure \fILINE\fR and \fIINDEX\fR are dispalyed\. +. +.IP +\fBNOTE:\fR Due to a bug in Bats, empty lines are discarded, causing line indices to change and preventing testing for empty lines\. See \fIBUGS\fR for more\. +. +.SH "BUGS" +. +.SS "Report bugs" +Report bugs on Bats\' GitHub issue tracker at \fIhttps://github\.com/sstephenson/bats/issues\fR\. +. +.SS "Known bugs" +Due to a bug in Bats, empty lines are missing from \fB${lines[@]}\fR, causing line indices to change and preventing testing for empty lines when using \fBassert_line\fR and \fBrefute_line\fR\. See PR #93 on Github at \fIhttps://github\.com/sstephenson/bats/pull/93\fR\. +. +.SH "COPYRIGHT" +TODO(ztombol): Find a suitable license\. +. +.SH "SEE ALSO" +\fBbash\fR(1), \fBbats\fR(1), \fBbats\fR(7) diff --git a/man/batslib.7.ronn b/man/batslib.7.ronn new file mode 100644 index 00000000..7f532d45 --- /dev/null +++ b/man/batslib.7.ronn @@ -0,0 +1,160 @@ +batslib(7) -- Bats Standard Library of Test Helpers +=================================================== + + +## DESCRIPTION + +The Standard Library is a collection of test helpers intended to +simplify testing. It helps building test suits that provide relevant +information when a test fails to speed up debugging. + +It contains the following types of test helpers. + + * [ASSERTIONS][] are functions that perform a test and output relevant + information on failure to help debugging. + +Test helpers send all output to the standard error, making them usable +outside of `@test` functions too. Output is formatted for readability as +described in [OUTPUT][]. + +The Standard Library is automatically loaded and available in all test +files. + + +## OUTPUT + +On failure, in addition to the usual output generated by Bats, +[ASSERTIONS][] display information relevant to the failed test to help +debugging. The output is formatted for readability and displayed as +key-value pairs on the standard error. + +When the value is one line long, the pair is displayed in a columnar +fashion called **two-column** format. + + -- output differs -- + expected : want + actual : have + -- + +When the value is longer than one line, the number of lines in the value +is displayed after the key, and the value starts on the next line and is +indented by two spaces for added readability. This is called +**multi-line** format. + +For convenience, sometimes related values are also displayed in this +format even if they are only one line long. + + -- output differs -- + expected (1 lines): + want + actual (3 lines): + have 1 + have 2 + have 3 + -- + + +## ASSERTIONS + +Assertions are functions that perform a test and output relevant +information on failure to help debugging. They return 1 on failure and 0 +otherwise. + +Assertions about exit code and output operate on the results of the most +recent invocation of `run`. + +* `flunk` []: + Display and fail. This function provides a convenient way to + report various failures. Other test helpers also use it to display + output on failure. When no parameters are specified, is read + from the standard input. + +* `assert` : + Fail if evaluates to false. On failure, is + displayed. + + **Note:** can only be a simple command. Compound + commands, such as `[[`, are not supported. + +* `assert_equal` : + Fail if and do not equal. On failure both values + are displayed. If either one is longer than one line, both are + displayed in _mulit-line_ format. + +* `assert_output` []: + Fail if does not equal `$output`. On failure both values + are displayed. If either one is longer than one line, both are + displayed in _mulit-line_ format. When no parameters are specified, + is read from the standard input. + +* `assert_success` []: + When no parameters are specified, fail if `$status` is not `0`. On + failure `$status` and `$output` are displayed. If `$output` is longer + than one line, it is displayed in _mulit-line_ format. + + When is specified and `$status` is `0`, fail if + does not equal `$output`. On failure both values are displayed. If + either one is longer than one line, both are displayed in _multi-line_ + format. + +* `assert_failure` []: + When no parameters are specified, fail if `$status` is `0`. On failure + `$output` is displayed. If `$output` is longer than one line, it is + displayed in _mulit-line_ format. + + When is specified and `$status` is not `0`, fail if + does not equal `$output`. On failure both values are + displayed. If either one is longer than one line, both are displayed + in _multi-line_ format. + +* `assert_line` [] : + When only is specified, fail if `${lines[@]}` does not contain + . On failure and `$output` are displayed. If `$output` is + longer than one line, it is displayed in _mulit-line_ format. + + When is specified, fail if does not equal + `${lines[INDEX]}`. On failure , and `${lines[INDEX]}` + are dispalyed. + + **NOTE:** Due to a bug in Bats, empty lines are discarded, causing + line indices to change and preventing testing for empty lines. See + [BUGS][] for more. + +* `refute_line` [] : + When only is specified, fail if `${lines[@]}` contains . + On failure , its zero-based index in `${lines[@]}`, and + `$output` are dispalyed. If `$output` is longer than one line, it is + displayed in _mulit-line_ format with highlighted. + + When is specified, fail if equals `${lines[INDEX]}`. On + failure and are dispalyed. + + **NOTE:** Due to a bug in Bats, empty lines are discarded, causing + line indices to change and preventing testing for empty lines. See + [BUGS][] for more. + + +## BUGS + +### Report bugs + +Report bugs on Bats' GitHub issue tracker at +. + + +### Known bugs + +Due to a bug in Bats, empty lines are missing from `${lines[@]}`, +causing line indices to change and preventing testing for empty lines +when using `assert_line` and `refute_line`. See PR \#93 on Github at +. + + +## COPYRIGHT + +TODO(ztombol): Find a suitable license. + + +## SEE ALSO + +`bash`(1), `bats`(1), `bats`(7) diff --git a/test/20-bats.bats b/test/20-bats.bats index f1aff293..db8bbef9 100755 --- a/test/20-bats.bats +++ b/test/20-bats.bats @@ -262,3 +262,7 @@ fixtures bats [ $status -eq 0 ] [ "${lines[1]}" = "ok 1 loop_func" ] } + +@test "standard library is loaded automatically" { + assert true +} diff --git a/test/50-lib-internal-output-10-batslib_err.bats b/test/50-lib-internal-output-10-batslib_err.bats new file mode 100755 index 00000000..21a1a876 --- /dev/null +++ b/test/50-lib-internal-output-10-batslib_err.bats @@ -0,0 +1,15 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'batslib_err() prints positional parameters' { + run batslib_err 'message' + [ "$status" -eq 0 ] + [ "$output" == 'message' ] +} + +@test 'batslib_err() prints STDIN when no positional parameters are specified' { + run bash -c "source '${BATS_LIB}/batslib.bash'; echo 'message' | batslib_err" + [ "$status" -eq 0 ] + [ "$output" == 'message' ] +} diff --git a/test/50-lib-internal-output-11-batslib_count_lines.bats b/test/50-lib-internal-output-11-batslib_count_lines.bats new file mode 100755 index 00000000..dff37bbc --- /dev/null +++ b/test/50-lib-internal-output-11-batslib_count_lines.bats @@ -0,0 +1,21 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'batslib_count_lines() prints the number of lines in the input' { + run batslib_count_lines $'a\nb\nc\n' + [ "$status" -eq 0 ] + [ "$output" == '3' ] +} + +@test 'batslib_count_lines() counts last line when it is not terminated by a newline' { + run batslib_count_lines $'a\nb\nc' + [ "$status" -eq 0 ] + [ "$output" == '3' ] +} + +@test 'batslib_count_lines() counts empty lines' { + run batslib_count_lines $'\n\n\n' + [ "$status" -eq 0 ] + [ "$output" == '3' ] +} diff --git a/test/50-lib-internal-output-12-batslib_is_single_line.bats b/test/50-lib-internal-output-12-batslib_is_single_line.bats new file mode 100755 index 00000000..9af68bab --- /dev/null +++ b/test/50-lib-internal-output-12-batslib_is_single_line.bats @@ -0,0 +1,13 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'batslib_is_single_line() returns 0 if all parameters are one-line strings' { + run batslib_is_single_line 'a' $'b\n' 'c' + [ "$status" -eq 0 ] +} + +@test 'batslib_is_single_line() returns 1 if at least one parameter is longer than one line' { + run batslib_is_single_line 'a' $'b\nb' 'c' + [ "$status" -eq 1 ] +} diff --git a/test/50-lib-internal-output-13-batslib_get_max_single_line_key_width.bats b/test/50-lib-internal-output-13-batslib_get_max_single_line_key_width.bats new file mode 100755 index 00000000..9b405a5e --- /dev/null +++ b/test/50-lib-internal-output-13-batslib_get_max_single_line_key_width.bats @@ -0,0 +1,17 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'batslib_get_max_single_line_key_width() prints the length of the longest key' { + local -ar pairs=( 'k _1' 'v 1' 'k 2' 'v 2' 'k __3' 'v 3' ) + run batslib_get_max_single_line_key_width "${pairs[@]}" + [ "$status" -eq 0 ] + [ "$output" == '5' ] +} + +@test 'batslib_get_max_single_line_key_width() only considers keys with one-line values' { + local -ar pairs=( 'k _1' 'v 1' 'k 2' 'v 2' 'k __3' $'v\n3' ) + run batslib_get_max_single_line_key_width "${pairs[@]}" + [ "$status" -eq 0 ] + [ "$output" == '4' ] +} diff --git a/test/50-lib-internal-output-14-batslib_print_kv_single.bats b/test/50-lib-internal-output-14-batslib_print_kv_single.bats new file mode 100755 index 00000000..70942de0 --- /dev/null +++ b/test/50-lib-internal-output-14-batslib_print_kv_single.bats @@ -0,0 +1,31 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'batslib_print_kv_single() prints key-value pairs with keys in fixed width columns' { + local -ar pairs=( + 'k _1' 'v 1' + 'k 2 ' 'v 2' + 'k __3' 'v 3' + ) + run batslib_print_kv_single 5 "${pairs[@]}" + [ "$status" -eq 0 ] + [ "${#lines[@]}" == '3' ] + [ "${lines[0]}" == 'k _1 : v 1' ] + [ "${lines[1]}" == 'k 2 : v 2' ] + [ "${lines[2]}" == 'k __3 : v 3' ] +} + +@test 'batslib_print_kv_single() does not truncate keys when the column is too narrow' { + local -ar pairs=( + 'k _1' 'v 1' + 'k 2' 'v 2' + 'k __3' 'v 3' + ) + run batslib_print_kv_single 0 "${pairs[@]}" + [ "$status" -eq 0 ] + [ "${#lines[@]}" == '3' ] + [ "${lines[0]}" == 'k _1 : v 1' ] + [ "${lines[1]}" == 'k 2 : v 2' ] + [ "${lines[2]}" == 'k __3 : v 3' ] +} diff --git a/test/50-lib-internal-output-15-batslib_print_kv_multi.bats b/test/50-lib-internal-output-15-batslib_print_kv_multi.bats new file mode 100755 index 00000000..885a6b1a --- /dev/null +++ b/test/50-lib-internal-output-15-batslib_print_kv_multi.bats @@ -0,0 +1,21 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'batslib_print_kv_multi() prints keys and values on separate lines' { + local -ar pairs=( + 'k _1' 'v 1' + 'k 2' $'v 2-1\nv 2-2' + 'k __3' 'v 3' + ) + run batslib_print_kv_multi "${pairs[@]}" + [ "$status" -eq 0 ] + [ "${#lines[@]}" == '7' ] + [ "${lines[0]}" == 'k _1 (1 lines):' ] + [ "${lines[1]}" == 'v 1' ] + [ "${lines[2]}" == 'k 2 (2 lines):' ] + [ "${lines[3]}" == 'v 2-1' ] + [ "${lines[4]}" == 'v 2-2' ] + [ "${lines[5]}" == 'k __3 (1 lines):' ] + [ "${lines[6]}" == 'v 3' ] +} diff --git a/test/50-lib-internal-output-16-batslib_print_kv_single_or_multi.bats b/test/50-lib-internal-output-16-batslib_print_kv_single_or_multi.bats new file mode 100755 index 00000000..5e7cb1b2 --- /dev/null +++ b/test/50-lib-internal-output-16-batslib_print_kv_single_or_multi.bats @@ -0,0 +1,35 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'batslib_print_kv_single_or_multi() prints in two-column format if all values are one line long' { + local -ar pairs=( + 'k _1' 'v 1' + 'k 2 ' 'v 2' + 'k __3' 'v 3' + ) + run batslib_print_kv_single_or_multi 5 "${pairs[@]}" + [ "$status" -eq 0 ] + [ "${#lines[@]}" == '3' ] + [ "${lines[0]}" == 'k _1 : v 1' ] + [ "${lines[1]}" == 'k 2 : v 2' ] + [ "${lines[2]}" == 'k __3 : v 3' ] +} + +@test 'batslib_print_kv_single_or_multi() prints in multi-line format if a at least one value is longer than one line' { + local -ar pairs=( + 'k _1' 'v 1' + 'k 2' $'v 2-1\nv 2-2' + 'k __3' 'v 3' + ) + run batslib_print_kv_single_or_multi 5 "${pairs[@]}" + [ "$status" -eq 0 ] + [ "${#lines[@]}" == '7' ] + [ "${lines[0]}" == 'k _1 (1 lines):' ] + [ "${lines[1]}" == ' v 1' ] + [ "${lines[2]}" == 'k 2 (2 lines):' ] + [ "${lines[3]}" == ' v 2-1' ] + [ "${lines[4]}" == ' v 2-2' ] + [ "${lines[5]}" == 'k __3 (1 lines):' ] + [ "${lines[6]}" == ' v 3' ] +} diff --git a/test/50-lib-internal-output-17-batslib_prefix.bats b/test/50-lib-internal-output-17-batslib_prefix.bats new file mode 100755 index 00000000..23a0e3d3 --- /dev/null +++ b/test/50-lib-internal-output-17-batslib_prefix.bats @@ -0,0 +1,39 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'batslib_prefix() prefixes each line with the given string' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf 'a\nb\nc\n' | batslib_prefix '_'" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == '_a' ] + [ "${lines[1]}" == '_b' ] + [ "${lines[2]}" == '_c' ] +} + +@test 'batslib_prefix() uses two spaces as default prefix' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf 'a\nb\nc\n' | batslib_prefix" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == ' a' ] + [ "${lines[1]}" == ' b' ] + [ "${lines[2]}" == ' c' ] +} + +@test 'batslib_prefix() prefixes the last line when it is not terminated by a newline' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf 'a\nb\nc' | batslib_prefix" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == ' a' ] + [ "${lines[1]}" == ' b' ] + [ "${lines[2]}" == ' c' ] +} + +@test 'batslib_prefix() prefixes empty lines' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf '\n\n\n' | batslib_prefix" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == ' ' ] + [ "${lines[1]}" == ' ' ] + [ "${lines[2]}" == ' ' ] +} diff --git a/test/50-lib-internal-output-18-batslib_mark.bats b/test/50-lib-internal-output-18-batslib_mark.bats new file mode 100755 index 00000000..4502e975 --- /dev/null +++ b/test/50-lib-internal-output-18-batslib_mark.bats @@ -0,0 +1,64 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'batslib_mark() highlights a single line' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf ' a\n b\n c\n' | batslib_mark '>' 0" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == '>a' ] + [ "${lines[1]}" == ' b' ] + [ "${lines[2]}" == ' c' ] +} + +@test 'batslib_mark() highlights lines when indices are in ascending order' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf ' a\n b\n c\n' | batslib_mark '>' 1 2" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == ' a' ] + [ "${lines[1]}" == '>b' ] + [ "${lines[2]}" == '>c' ] +} + +@test 'batslib_mark() highlights lines when indices are in random order' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf ' a\n b\n c\n' | batslib_mark '>' 2 1" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == ' a' ] + [ "${lines[1]}" == '>b' ] + [ "${lines[2]}" == '>c' ] +} + +@test 'batslib_mark() ignores duplicate line indices' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf ' a\n b\n c\n' | batslib_mark '>' 1 2 1" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == ' a' ] + [ "${lines[1]}" == '>b' ] + [ "${lines[2]}" == '>c' ] +} + +@test 'batslib_mark() outputs the input untouched if the marking string is the empty string' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf ' a\n b\n c\n' | batslib_mark '' 1" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == ' a' ] + [ "${lines[1]}" == ' b' ] + [ "${lines[2]}" == ' c' ] +} + +@test 'batslib_mark() highlights the last line when it is not terminated by a newline' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf ' a\n b\n c' | batslib_mark '>' 2" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == ' a' ] + [ "${lines[1]}" == ' b' ] + [ "${lines[2]}" == '>c' ] +} + +@test 'batslib_mark() replaces the line with the marking string if the line is shorter or equally long' { + run bash -c "source '${BATS_LIB}/batslib.bash'; printf '\n' | batslib_mark '>' 0" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 1 ] + [ "${lines[0]}" == '>' ] +} diff --git a/test/50-lib-internal-output-19-batslib_decorate.bats b/test/50-lib-internal-output-19-batslib_decorate.bats new file mode 100755 index 00000000..d7bcbe8a --- /dev/null +++ b/test/50-lib-internal-output-19-batslib_decorate.bats @@ -0,0 +1,12 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'batslib_decorate() encloses body in header and footer lines' { + run bash -c "source '${BATS_LIB}/batslib.bash'; echo 'body' | batslib_decorate 'title'" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == '-- title --' ] + [ "${lines[1]}" == 'body' ] + [ "${lines[2]}" == '--' ] +} diff --git a/test/60-lib-assertion-10-flunk.bats b/test/60-lib-assertion-10-flunk.bats new file mode 100755 index 00000000..d424181b --- /dev/null +++ b/test/60-lib-assertion-10-flunk.bats @@ -0,0 +1,20 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'flunk() returns 1' { + run flunk '' + [ "$status" -eq 1 ] +} + +@test 'flunk() prints positional parameters' { + run flunk 'message' + [ "$status" -eq 1 ] + [ "$output" == 'message' ] +} + +@test 'flunk() prints STDIN if no positional parameters are specified' { + run bash -c "source '${BATS_LIB}/batslib.bash'; echo 'message' | flunk" + [ "$status" -eq 1 ] + [ "$output" == 'message' ] +} diff --git a/test/60-lib-assertion-11-assert.bats b/test/60-lib-assertion-11-assert.bats new file mode 100755 index 00000000..003ab0e2 --- /dev/null +++ b/test/60-lib-assertion-11-assert.bats @@ -0,0 +1,18 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'assert() returns 0 if the condition evaluates to TRUE' { + run assert true + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'assert() returns 1 and displays the condition if it evaluates to FALSE' { + run assert false + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == '-- assertion failed --' ] + [ "${lines[1]}" == 'condition : false' ] + [ "${lines[2]}" == '--' ] +} diff --git a/test/60-lib-assertion-12-assert_equal.bats b/test/60-lib-assertion-12-assert_equal.bats new file mode 100755 index 00000000..0404dfc7 --- /dev/null +++ b/test/60-lib-assertion-12-assert_equal.bats @@ -0,0 +1,37 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'assert_equal() returns 0 if the actual value equals the expected' { + run assert_equal 'a' 'a' + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'assert_equal() returns 1 and displays the actual and expected value if they do not equal' { + run assert_equal 'a' 'b' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 4 ] + [ "${lines[0]}" == '-- values do not equal --' ] + [ "${lines[1]}" == 'expected : a' ] + [ "${lines[2]}" == 'actual : b' ] + [ "${lines[3]}" == '--' ] +} + +@test 'assert_equal() displays the expected and actual value in multi-line format if necessary' { + run assert_equal 'a' $'b 1\nb 2' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 7 ] + [ "${lines[0]}" == '-- values do not equal --' ] + [ "${lines[1]}" == 'expected (1 lines):' ] + [ "${lines[2]}" == ' a' ] + [ "${lines[3]}" == 'actual (2 lines):' ] + [ "${lines[4]}" == ' b 1' ] + [ "${lines[5]}" == ' b 2' ] + [ "${lines[6]}" == '--' ] +} + +@test 'assert_equal() performs literal matching' { + run assert_equal 'a' '*' + [ "$status" -eq 1 ] +} diff --git a/test/60-lib-assertion-13-assert_output.bats b/test/60-lib-assertion-13-assert_output.bats new file mode 100755 index 00000000..7514323c --- /dev/null +++ b/test/60-lib-assertion-13-assert_output.bats @@ -0,0 +1,49 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'assert_output() returns 0 if $output equals the expected output' { + run echo 'a' + run assert_output 'a' + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'assert_output() returns 1 and displays the expected and actual output if they do not equal' { + run echo 'b' + run assert_output 'a' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 4 ] + [ "${lines[0]}" == '-- output differs --' ] + [ "${lines[1]}" == 'expected : a' ] + [ "${lines[2]}" == 'actual : b' ] + [ "${lines[3]}" == '--' ] +} + +@test 'assert_output() displays the expected and actual output in multi-line format if necessary' { + run echo $'b 1\nb 2' + run assert_output 'a' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 7 ] + [ "${lines[0]}" == '-- output differs --' ] + [ "${lines[1]}" == 'expected (1 lines):' ] + [ "${lines[2]}" == ' a' ] + [ "${lines[3]}" == 'actual (2 lines):' ] + [ "${lines[4]}" == ' b 1' ] + [ "${lines[5]}" == ' b 2' ] + [ "${lines[6]}" == '--' ] +} + +@test 'assert_output() reads the expected output from STDIN when no positional parameters are specified' { + run echo 'a' + export output + run bash -c "source '${BATS_LIB}/batslib.bash'; echo 'a' | assert_output" + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'assert_output() performs literal matching' { + run echo '*' + run assert_output 'a' + [ "$status" -eq 1 ] +} diff --git a/test/60-lib-assertion-14-assert_success.bats b/test/60-lib-assertion-14-assert_success.bats new file mode 100755 index 00000000..8caa5af2 --- /dev/null +++ b/test/60-lib-assertion-14-assert_success.bats @@ -0,0 +1,72 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'assert_success() returns 0 if $status is 0' { + run true + run assert_success + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'assert_success() returns 1 and displays $status and $output if $status is not 0' { + run bash -c 'echo error; exit 1' + run assert_success + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 4 ] + [ "${lines[0]}" == '-- command failed --' ] + [ "${lines[1]}" == 'status : 1' ] + [ "${lines[2]}" == 'output : error' ] + [ "${lines[3]}" == '--' ] +} + +@test 'assert_success() displays $output in multi-line format if necessary' { + run bash -c "echo $'error 1\nerror 2'; exit 1" + run assert_success + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 6 ] + [ "${lines[0]}" == '-- command failed --' ] + [ "${lines[1]}" == 'status : 1' ] + [ "${lines[2]}" == 'output (2 lines):' ] + [ "${lines[3]}" == ' error 1' ] + [ "${lines[4]}" == ' error 2' ] + [ "${lines[5]}" == '--' ] +} + +@test 'assert_success() tests $output against the first positional parameter if specified' { + run echo 'a' + run assert_success 'a' + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'assert_success() displays the expected and actual output if they differ' { + run echo 'b' + run assert_success 'a' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 4 ] + [ "${lines[0]}" == '-- command succeeded, but output differs --' ] + [ "${lines[1]}" == 'expected : a' ] + [ "${lines[2]}" == 'actual : b' ] + [ "${lines[3]}" == '--' ] +} + +@test 'assert_success() displays the expected and actual output in multi-line format if necessary' { + run echo $'b 1\nb 2' + run assert_success 'a' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 7 ] + [ "${lines[0]}" == '-- command succeeded, but output differs --' ] + [ "${lines[1]}" == 'expected (1 lines):' ] + [ "${lines[2]}" == ' a' ] + [ "${lines[3]}" == 'actual (2 lines):' ] + [ "${lines[4]}" == ' b 1' ] + [ "${lines[5]}" == ' b 2' ] + [ "${lines[6]}" == '--' ] +} + +@test 'assert_success() performs literal matching on $output' { + run echo 'a' + run assert_success '*' + [ "$status" -eq 1 ] +} diff --git a/test/60-lib-assertion-15-assert_failure.bats b/test/60-lib-assertion-15-assert_failure.bats new file mode 100755 index 00000000..c01df855 --- /dev/null +++ b/test/60-lib-assertion-15-assert_failure.bats @@ -0,0 +1,70 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'assert_failure() returns 0 if $status is not 0' { + run false + run assert_failure + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'assert_failure() returns 1 and displays $output if $status is 0' { + run bash -c 'echo ok; exit 0' + run assert_failure + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 3 ] + [ "${lines[0]}" == '-- command succeeded, but it was expected to fail --' ] + [ "${lines[1]}" == 'output : ok' ] + [ "${lines[2]}" == '--' ] +} + +@test 'assert_failure() displays $output in multi-line format if necessary' { + run bash -c "echo $'ok 1\nok 2'; exit 0" + run assert_failure + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 5 ] + [ "${lines[0]}" == '-- command succeeded, but it was expected to fail --' ] + [ "${lines[1]}" == 'output (2 lines):' ] + [ "${lines[2]}" == ' ok 1' ] + [ "${lines[3]}" == ' ok 2' ] + [ "${lines[4]}" == '--' ] +} + +@test 'assert_failure() tests $output against the first positional parameter if specified' { + run bash -c 'echo a; exit 1' + run assert_failure 'a' + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'assert_failure() displays the expected and actual output if they differ' { + run bash -c 'echo b; exit 1' + run assert_failure 'a' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 4 ] + [ "${lines[0]}" == '-- command failed as expected, but output differs --' ] + [ "${lines[1]}" == 'expected : a' ] + [ "${lines[2]}" == 'actual : b' ] + [ "${lines[3]}" == '--' ] +} + +@test 'assert_failure() displays the expected and actual output in multi-line format if necessary' { + run bash -c "echo $'b 1\nb 2'; exit 1" + run assert_failure 'a' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 7 ] + [ "${lines[0]}" == '-- command failed as expected, but output differs --' ] + [ "${lines[1]}" == 'expected (1 lines):' ] + [ "${lines[2]}" == ' a' ] + [ "${lines[3]}" == 'actual (2 lines):' ] + [ "${lines[4]}" == ' b 1' ] + [ "${lines[5]}" == ' b 2' ] + [ "${lines[6]}" == '--' ] +} + +@test 'assert_failure() performs literal matching on $output' { + run bash -c 'echo a; exit 1' + run assert_failure '*' + [ "$status" -eq 1 ] +} diff --git a/test/60-lib-assertion-16-assert_line.bats b/test/60-lib-assertion-16-assert_line.bats new file mode 100755 index 00000000..76d8d494 --- /dev/null +++ b/test/60-lib-assertion-16-assert_line.bats @@ -0,0 +1,66 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'assert_line() returns 0 if the expected line is found' { + run echo $'a\nb\nc' + run assert_line 'b' + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'assert_line() returns 1 and displays $output and the expected line if it was not found' { + run echo 'a' + run assert_line 'd' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 4 ] + [ "${lines[0]}" == '-- line is not in output --' ] + [ "${lines[1]}" == 'line : d' ] + [ "${lines[2]}" == 'output : a' ] + [ "${lines[3]}" == '--' ] +} + +@test 'assert_line() displays $output in multi-line format if necessary' { + run echo $'a\nb\nc' + run assert_line 'd' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 7 ] + [ "${lines[0]}" == '-- line is not in output --' ] + [ "${lines[1]}" == 'line : d' ] + [ "${lines[2]}" == 'output (3 lines):' ] + [ "${lines[3]}" == ' a' ] + [ "${lines[4]}" == ' b' ] + [ "${lines[5]}" == ' c' ] + [ "${lines[6]}" == '--' ] +} + +@test 'assert_line() returns 0 if the expected line is found at the given index' { + run echo $'a\nb\nc' + run assert_line 1 'b' + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'assert_line() returns 1 and displays the expected and the actual line at the given index if they do not equal' { + run echo $'a\nb\nc' + run assert_line 1 'd' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 5 ] + [ "${lines[0]}" == '-- line differs --' ] + [ "${lines[1]}" == 'index : 1' ] + [ "${lines[2]}" == 'expected : d' ] + [ "${lines[3]}" == 'actual : b' ] + [ "${lines[4]}" == '--' ] +} + +@test 'assert_line() performs literal matching when the expected line is sought in the entire output' { + run echo $'a\nb\nc' + run assert_line '*' + [ "$status" -eq 1 ] +} + +@test 'assert_line() performs literal matching when the expected line is sought at a given index' { + run echo $'a\nb\nc' + run assert_line 1 '*' + [ "$status" -eq 1 ] +} diff --git a/test/60-lib-assertion-17-refute_line.bats b/test/60-lib-assertion-17-refute_line.bats new file mode 100755 index 00000000..4303dab6 --- /dev/null +++ b/test/60-lib-assertion-17-refute_line.bats @@ -0,0 +1,67 @@ +#!/usr/bin/env bats + +load test_helper + +@test 'refute_line() returns 0 if the unexpected line is not found' { + run echo $'a\nb\nc' + run refute_line 'd' + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'refute_line() returns 1 and displays $output, the unexpected line and its index if it was found' { + run echo $'b' + run refute_line 'b' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 5 ] + [ "${lines[0]}" == '-- line should not be in output --' ] + [ "${lines[1]}" == 'line : b' ] + [ "${lines[2]}" == 'index : 0' ] + [ "${lines[3]}" == 'output : b' ] + [ "${lines[4]}" == '--' ] +} + +@test 'refute_line() displays $output in multi-line format with the unexpected line highlighted if necessary' { + run echo $'a\nb\nc' + run refute_line 'b' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 8 ] + [ "${lines[0]}" == '-- line should not be in output --' ] + [ "${lines[1]}" == 'line : b' ] + [ "${lines[2]}" == 'index : 1' ] + [ "${lines[3]}" == 'output (3 lines):' ] + [ "${lines[4]}" == ' a' ] + [ "${lines[5]}" == '> b' ] + [ "${lines[6]}" == ' c' ] + [ "${lines[7]}" == '--' ] +} + +@test 'refute_line() returns 0 if the unexpected line is not found at the given index' { + run echo $'a\nb\nc' + run refute_line 1 'd' + [ "$status" -eq 0 ] + [ "${#lines[@]}" -eq 0 ] +} + +@test 'refute_line() returns 1 and displays the unexpected line and the index if it was found at the given index' { + run echo $'a\nb\nc' + run refute_line 1 'b' + [ "$status" -eq 1 ] + [ "${#lines[@]}" -eq 4 ] + [ "${lines[0]}" == '-- line should differ from expected --' ] + [ "${lines[1]}" == 'index : 1' ] + [ "${lines[2]}" == 'line : b' ] + [ "${lines[3]}" == '--' ] +} + +@test 'refute_line() performs literal matching when the unexpected line is sought in the entire output' { + run echo $'a\nb\nc' + run refute_line '*' + [ "$status" -eq 0 ] +} + +@test 'refute_line() performs literal matching when the unexpected line is sought at a given index' { + run echo $'a\nb\nc' + run refute_line 1 '*' + [ "$status" -eq 0 ] +}