BASHJAZZ/utest

unit testing for bash-scripts & functions

All sub-projects are under the BSD licence and are free for anyone to use and modify. DONATIONS would be much appreciated.

INTRODUCTION

With utest you can test any program's output, not just Bash scripts. What makes this library special is that you can also test Bash functions and nested functions. It is especially appealing when writing Bash scripts and there's just too much...
DEPENDENCIES:

Internal: none. No need for any other BashJazz sub-project to be present in your system.

External: bash >= v4.2, awk, sed, grep.

USAGE

Currently, the following usage pattern is considered to be the most intuitive one:

  1. Write the script you're about to test - let's call it my_script.sh
  2. Create a test script named my_script.test.sh and make it executable:
    • touch my_script.test.sh
    • chmod +x my_script.test.sh
  3. Inside the test-script file, source utest.sh from this directory and then source `my_script.sh` file that you'll be testing.

    You can put your test files files in a separate directory, but beware of relative paths and their correctness. To source utest.sh you'd probably want to add BASHJAZZ_PATH=/path/to/bashjazz into your shell's environment files: .bashrc, .zshrc, etc.

    Then, your bare minimum my_script.test.sh file would look like this:

    • #!/usr/bin/env bash
    • source $BASHJAZZ_PATH/utest/utest.sh
    • source ../path/to/my_script.sh

    Following these lines, you may start adding your unit tests.

  4. To run your tests, go to the directory where you placed your test-script file and use this command
    • ./my_script.test.sh

WRITING TESTS

There's just one function command - utest - which you'll use to write all of your unit tests. However, this function has nested functions, which serve different kinds of purposes. Here's a full list of possible subcommands (nested functions) which you can make use of (arguments in [] indicate they're optional):

  • utest begin NAME [DESCRIPTION]
  • utest cmd [COMMAND] [ARGUMENTS]
  • utest add_cmd [COMMAND] [ARGUMENTS]
  • utest assert VALUE1 == VALUE2
  • utest assert VALUE1 eq VALUE2 (identical previous line)
  • utest assert VALUE1 == ''
  • utest assert VALUE1 is blank (identical previous line)
  • utest assert VALUE1 not == ''
  • utest assert VALUE1 not eq '' (identical previous line)
  • utest assert VALUE1 is_not blank (identical previous line)
  • utest end [NAME]
NOTE:

The characters == are not representing an operator here, but are, in fact, just another positional argument that's internally translated to the eq, which is a function nested inside the assert() function.

DESCRIPTION is optional, but it's highly recommended you write it. The "NAME" is required only for the `begin` subcommand, however in not to get lost in the code it's recommended you use it for the `end` subcommand too.

The cmd subcommand runs the specified COMMAND with the "ARGUMENTS" and writes the output into the $UTOUT variable accessible inside the test-script file. You can, therefore, use the assert subcommand to verify the output in this way:

my_script.test.sh
  • utest begin MyScript
  • utest cmd bash my_script.sh
  • utest assert "$UTOUT" == "hello world"
  • utest end MyScript

You may want to run multiple commands before running the one which output you'd like to test, because different commands may set various variables or modify files or do a number of different kinds of things. In that case, use the utest add_cmd subcommand. When you're done, run utest cmd without arguments and the output of the last command will be written into the $UTOUT variable.

my_script.test.sh
  • utest begin MyScript
  • utest add_cmd bash my_script.sh something1
  • utest add_cmd bash my_script.sh
  • utest cmd
  • utest assert "$UTOUT" == "hello world"
  • utest end MyScript

You can have multiple levels of nesting for tests:

my_script.test.sh
  • utest begin MyScript 'A script that does something I need'
  • utest begin function_one 'it had one job...'
  • ...
  • utest end function_one
  • utest end MyScript

The names don't affect anything, so you can pick any name for the begin/end blocks, so long as it doesn't have spaces in it. However, the suggested convention is to use function names you're testing.

Adding a second argument to utest end call would print a "pending message" next to to the script, but it won't actually do anything to prevent commands or assertions inside the test from running - it will simply not print their output. So remember to remove that argument when you're ready to run your test and there's some code in it. Or, it may be used to just temporarily ignore the failing tests and you may write something like "(WIP) some assertions fail".

  • utest begin MyScript
  • utest begin function_two 'it has two jobs...'
  • ...
  • utest end function_two 'WIP: assertion 2 fails'
  • utest end MyScript

To print descriptions along with the test names in the test-script output - prepend the call to your test script file with PRINT_DESCRIPTIONS=1, for example:

PRINT_DESCRIPTIONS=1 ./my_script.test.sh

TESTING NESTED FUNCTIONS

Strictly speaking, there isn't any particular hardcoded demand that would force you into one way of testing nested functions or the other. The simplest way to test nested functions is to make your top-level function recognize the -c argument along with the name of the nested function - which would be the value in the argument after the -c flag. So if you decide to go for the nested functions paradigm, try this inside your script:

my_script.sh
  • MyScript() {
  • if test $1 = '-c'; then
  • local CALL_NESTED="$2"
  • shift 2
  • fi
  • function_one() {
  • echo "I'm function one"
  • }
  • test -n $CALL_NESTED && $CALL_NESTED $@ && exit $?
  • }
In order for this to work, you're going to need to add the code presented above into your original script: lines 2 through 5 and line 11 - the latter has to be added after all required functions are defined, but not necessarily before the closing bracket (in fact, you probably wouldn't want anything else, but the specific function to be invoked when you're calling it with the -c flag, therefore the rule if thumb is: add line 11 after all the nested functions are defined, but before any other code inside the top-level function). Then, in your test-script, you can write things such as:
my_script.test.sh
  • ...
  • utest cmd MyScript -c function_one
  • utest assert "$UTOUT" == "I'm function one"
  • ...

TESTING ERRORS

Normally, upon calling utest assert, if $UTERR global variable isn't empty (which would be the case when one of the utest cmd calls prints anything into stderr), the actual error message produced by a process or a function that was called through utest cmd would be printed into the terminal and the particular unit test case would be marked as "failed". However, sometimes, you'd actually want to verify the contents of the error message itself. For these purposes, you may use the special --error flag to the utest assert command:

my_script.test.sh
  • ...
  • utest cmd MyScript parse --input
  • utest assert --error "$UTERR" contains "argument --input requires a value"
  • ...

In this example we assume that MyScript parse expects cli-argument --inputto have value. If it doesn't, it must print the message we're testing against. The implementation of this error message print out is the responsibility of the author of the script, of course, and its exact wording has nothing to do with utest script itself. $UTERR would always contain anything printed into the stderr and, once again, unless the --ignore flag is provided when calling utest assert, any assertion command would be halted and $UTERR would simply be printed (once) into the terminal.

Note the use of the matcher `contains`, which, of course, can also be used in non-error assertions as well.

EXAMPLES

Take a look at ./example.sh and ./example.test.sh files, then try running the example test:

./example.test.sh

SUB-PROJECTS

PAGES

SATELLITE PROJECTS

  • DOCK (pain free containers)