WIP: testing #96
|
|
@ -99,9 +99,10 @@ their editor or terminal.
|
|||
|
||||
For usage text and help messages, do not implement a -h option. Instead, print
|
||||
usage information when any erroneous option is specified. Follow the NetBSD
|
||||
style guide for the usage text’s output format [1].
|
||||
style guide for the usage text’s output format [0].
|
||||
|
||||
[1] <http://cvsweb.netbsd.org/bsdweb.cgi/~checkout~/src/share/misc/style>
|
||||
If committing a new utility, please include tests and documentation (see
|
||||
tests/ and docs/) for the new tool.
|
||||
|
||||
If committing a new source file, format the commit message following these
|
||||
guidelines:
|
||||
|
|
@ -130,6 +131,7 @@ $ git commit -m 'tool(1): fix #42 & add feature x'
|
|||
|
||||
Commit messages should be written in the present tense.
|
||||
|
||||
[0] <http://cvsweb.netbsd.org/bsdweb.cgi/~checkout~/src/share/misc/style>
|
||||
--
|
||||
This work © 2023–2024 by Emma Tebibyte is licensed under CC BY-SA 4.0. To view a
|
||||
copy of this license, visit <http://creativecommons.org/licenses/by-sa/4.0/>
|
||||
|
|
|
|||
4
Makefile
4
Makefile
|
|
@ -53,9 +53,9 @@ install: dist
|
|||
cp -r $(DESTDIR)/* /
|
||||
|
||||
.PHONY: test
|
||||
test: build /tmp/getopt
|
||||
test: all build /tmp/getopt
|
||||
tests/test.sh
|
||||
/tmp/getopt
|
||||
tests/posix-compat.sh
|
||||
|
||||
/tmp/getopt: src/libgetopt.rs
|
||||
$(RUSTC) --test -o /tmp/getopt src/libgetopt.rs
|
||||
|
|
|
|||
60
tests/README
Normal file
60
tests/README
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
The testing suite contains two main trees (plus translations for strings used in
|
||||
the shell scripts): the Bonsai tree and the POSIX tree:
|
||||
|
||||
.
|
||||
├── bonsai
|
||||
│ ├── test_env
|
||||
│ ├── dj.sh
|
||||
│ ├── false.sh
|
||||
│ ├── fop.sh
|
||||
│ ├── hru.sh
|
||||
│ ├── intcmp.sh
|
||||
│ ├── mm.sh
|
||||
│ ├── strcmp.sh
|
||||
│ └── true.sh
|
||||
├── locales
|
||||
│ ├── en_US.UTF-8
|
||||
│ └── tok
|
||||
├── posix
|
||||
│ ├── bin
|
||||
│ │ ├── cat
|
||||
│ │ ├── false
|
||||
│ │ └── true
|
||||
│ └── posix_env
|
||||
├── README
|
||||
└── test.sh
|
||||
|
||||
The Bonsai tree tests the functionality of Harakit utilities for regressions and
|
||||
other issues relating to compliance to our standards of practice.
|
||||
|
||||
The POSIX tree tests the use of Harakit utilities in place of the standard usage
|
||||
of POSIX utilities. These scripts test the ability of Harakit to comply to POSIX
|
||||
standards using its native utilities in shell scripts as a compatibility shim.
|
||||
Each shell script in the top directory should contain a set of tests for each
|
||||
POSIX utility and be named for that utility. The bin directory should contain
|
||||
a set of shim scripts which will be imported into the path as POSIX utilities.
|
||||
Each test will compare the behavior of the shim script to the real utility on
|
||||
the system.
|
||||
|
||||
Currently, due to the limitations of POSIX shell quoting, a subset of argument
|
||||
parsing is supported: arguments containing characters from POSIX’s Portable
|
||||
Filename Character Set [0].
|
||||
|
||||
The bonsai/test_env and posix/posix_env files contain prerequisite shared
|
||||
environments for each of the tests. These scripts both contain lines which set
|
||||
the shell to write all commands run in them (-x) and to fail if any command
|
||||
fails (-e). See set(1p) for more information.
|
||||
|
||||
Both sets of tests also inherit the environment set by the test.sh script, which
|
||||
sets the $BIN environment variable to the bin directory at the root of the
|
||||
project for easy and idiomatic access to the built Harakit binaries. When
|
||||
calling the POSIX test scripts, test.sh also sets the variable $realutil to be
|
||||
the absolute path to the currently tested utility’s counterpart on the system.
|
||||
|
||||
[0] <https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282>
|
||||
|
||||
--
|
||||
Copyright © 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
|
||||
This work is licensed under CC BY-SA 4.0. To view a copy of this license, visit
|
||||
<http://creativecommons.org/licenses/by-sa/4.0/>.
|
||||
29
tests/bonsai/dj.sh
Executable file
29
tests/bonsai/dj.sh
Executable file
|
|
@ -0,0 +1,29 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 DTB <trinity@trinity.moe>
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
. tests/bonsai/test_env
|
||||
|
||||
! dj -h
|
||||
|
||||
# Linux has a /dev/full pseudodevice useful for testing errors.
|
||||
case "$(uname)" in
|
||||
Linux)
|
||||
dj -Hi /dev/zero -o /dev/full 2>&1 \
|
||||
| xargs -I out "$BIN"/strcmp '1+0 > 0+0; 1024 > 0' out
|
||||
;;
|
||||
esac
|
||||
|
||||
# Read nothing from /dev/null, write nothing to /dev/null.
|
||||
dj -Hi /dev/null -o /dev/null 2>&1 \
|
||||
| xargs -I out "$BIN"/strcmp '0+0 > 0+0; 0 > 0' out
|
||||
|
||||
# Test skipping stdin.
|
||||
#dd count=1 bs=1024 </dev/zero 2>/dev/null \
|
||||
# | dj -H -s 24 -o /dev/null 2>&1 \
|
||||
# | xargs -I out "$BIN"/strcmp '1+0 > 1+0; 1024 > 1000' out
|
||||
13
tests/bonsai/false.sh
Executable file
13
tests/bonsai/false.sh
Executable file
|
|
@ -0,0 +1,13 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 DTB <trinity@trinity.moe>
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
. tests/bonsai/test_env
|
||||
|
||||
! false
|
||||
! false -h
|
||||
|
emma marked this conversation as resolved
|
||||
21
tests/bonsai/fop.sh
Executable file
21
tests/bonsai/fop.sh
Executable file
|
|
@ -0,0 +1,21 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
. tests/bonsai/test_env
|
||||
|
||||
! fop -h
|
||||
|
||||
"$BIN/strcmp" "$(printf 'test0␞test1␞test2\n' | fop 1 sed 's/1/4/g')" \
|
||||
'test0␞test4␞test2'
|
||||
|
||||
"$BIN/strcmp" "$(printf 'test0 test1 test2\n' | fop -d' ' 2 sed 's/2/4/g')" \
|
||||
'test0 test1 test4'
|
||||
|
||||
! printf 'test\n' | fop 1 cat
|
||||
! printf 'test\n' | fop 'test' cat
|
||||
! printf 'test\n' | fop -d'test' cat
|
||||
26
tests/bonsai/hru.sh
Executable file
26
tests/bonsai/hru.sh
Executable file
|
|
@ -0,0 +1,26 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
. tests/bonsai/test_env
|
||||
|
||||
alias strcmp="$BIN/strcmp"
|
||||
alias rpn="$BIN/rpn"
|
||||
|
||||
strcmp "$(printf '1234\n' | hru)" '1.2 kB'
|
||||
strcmp "$(printf '0\n' | hru)" '0 B'
|
||||
|
||||
# doesn’t currently work but would be useful for testing for regressions
|
||||
#n=1
|
||||
#while "$BIN/true"; do
|
||||
# n="$(rpn "$n" 10 ×)"
|
||||
#
|
||||
# printf '%s\n' "$n" | hru || break
|
||||
#done
|
||||
#printf 'integer limit: ~%s\n' "$(rpn "$n" 10 ÷)"
|
||||
|
||||
! printf '%s\n' '-1' | hru
|
||||
25
tests/bonsai/intcmp.sh
Executable file
25
tests/bonsai/intcmp.sh
Executable file
|
|
@ -0,0 +1,25 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 DTB <trinity@trinity.moe>
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
. tests/bonsai/test_env
|
||||
|
||||
intcmp -e 3 3 3
|
||||
intcmp -g 3 2 1
|
||||
intcmp -l 1 2 3
|
||||
intcmp -ge 3 3 1
|
||||
intcmp -le 1 3 3
|
||||
intcmp -gl 1 2 3
|
||||
intcmp -egl 3 1 1 2
|
||||
! intcmp -e 1 2 3
|
||||
! intcmp -g 1 3 3
|
||||
! intcmp -l 3 3 1
|
||||
! intcmp -ge 1 2 3
|
||||
! intcmp -le 3 2 1
|
||||
! intcmp -gl 3 3 3
|
||||
! intcmp -egl foo
|
||||
19
tests/bonsai/mm.sh
Executable file
19
tests/bonsai/mm.sh
Executable file
|
|
@ -0,0 +1,19 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
. tests/bonsai/test_env
|
||||
|
||||
exec 3>&1
|
||||
|
||||
! mm -h
|
||||
|
||||
# mm(1) will error if positional arguments are given without -i or -o
|
||||
! mm argument
|
||||
|
||||
# check if stderr is empty upon specifying -e
|
||||
! "$BIN/strcmp" "$(printf 'test\n' | mm -i - -e 2>&1 1>&3)" ''
|
||||
34
tests/bonsai/npc.sh
Executable file
34
tests/bonsai/npc.sh
Executable file
|
|
@ -0,0 +1,34 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 DTB <trinity@trinity.moe>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
. tests/bonsai/test_env
|
||||
|
||||
! npc -h
|
||||
|
||||
# arg parsing
|
||||
npc -e </dev/null
|
||||
npc -t </dev/null
|
||||
npc -et </dev/null
|
||||
! npc -et 5 </dev/null
|
||||
|
||||
#i=0; while "$BIN"/intcmp -l $i 178; do
|
||||
# printf '\\%s\\n' "$i" \
|
||||
# | xargs -I fmt printf fmt
|
||||
#
|
||||
# # simulate octal
|
||||
# i="$(printf '1 + %s\n' "$i" | bc)"
|
||||
# printf '%s\n' "$i" \
|
||||
# | tail -c 1 \
|
||||
# | xargs "$BIN"/intcmp -e 8 \
|
||||
# && i="$(printf '2 + %s\n' "$i" | bc)"
|
||||
# printf '%s\n' "$i" \
|
||||
# | tail -c 2 \
|
||||
# | dd count=1 bs=1 2>/dev/null \
|
||||
# | xargs "$BIN"/intcmp -e 8 \
|
||||
# && i="$(printf '20 + %s\n' "$i" | bc)"
|
||||
#done
|
||||
16
tests/bonsai/strcmp.sh
Executable file
16
tests/bonsai/strcmp.sh
Executable file
|
|
@ -0,0 +1,16 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 DTB <trinity@trinity.moe>
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
. tests/bonsai/test_env
|
||||
|
||||
strcmp equals equals
|
||||
! strcmp inequals equals
|
||||
strcmp - -
|
||||
! strcmp -h
|
||||
! strcmp nocmp
|
||||
5
tests/bonsai/test_env
Normal file
5
tests/bonsai/test_env
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
#!/bin/sh
|
||||
|
||||
set -ex
|
||||
|
||||
alias "$UTIL=$BIN/$UTIL"
|
||||
13
tests/bonsai/true.sh
Executable file
13
tests/bonsai/true.sh
Executable file
|
|
@ -0,0 +1,13 @@
|
|||
#!/bin/sh
|
||||
trinity
commented
Why is this separate from Why is this separate from `tests/posix-compat/true.sh`?
emma
commented
This is This is `true(1)` from Bonsai and the one in `posix-compat/true.sh` is `true(1p)`
trinity
commented
Our true(1) and false(1) implementations are POSIX without extensions and shouldn't be extended. Maybe it would be better to only have them in the POSIX compatibility section. Our true(1) and false(1) implementations are POSIX without extensions and shouldn't be extended. Maybe it would be better to only have them in the POSIX compatibility section.
emma
commented
It makes sense to have them in both because the true(1p) test script leverages our script; however, I have just realized that I have implemented the POSIX test suite in a different way than I originally imagined and will make more changes before we can continue this conversation. It makes sense to have them in both because the true(1p) test script *leverages* our script; however, I have just realized that I have implemented the POSIX test suite in a different way than I originally imagined and will make more changes before we can continue this conversation.
|
||||
# Copyright (c) 2024 DTB <trinity@trinity.moe>
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
. tests/bonsai/test_env
|
||||
|
||||
true
|
||||
true -h
|
||||
4
tests/locales/en_US.UTF-8
Normal file
4
tests/locales/en_US.UTF-8
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
#!/bin/sh
|
||||
|
||||
export RUN_ERR='Run this script in the root of the project'
|
||||
export TEST_STR='Testing utility'
|
||||
5
tests/locales/tok
Normal file
5
tests/locales/tok
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
#!/bin/sh
|
||||
|
||||
export RUN_ERR="ilo ni li ken ala pali lon ma ni. sina ken kepeken ona lon ma \
|
||||
sewi"
|
||||
export TEST_STR='ilo pali'
|
||||
|
|
@ -1,24 +0,0 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Copyright (c) 2023–2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
set -e
|
||||
|
||||
if ! ls Makefile >/dev/null 2>&1
|
||||
then
|
||||
printf '%s: Run this script in the root of the project.\n' "$0" 1>&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
printf "Starting POSIX compatibility testing.\n"
|
||||
|
||||
for utility in tests/posix/*; do
|
||||
printf '%s: %s: Testing utility.\n' "$0" "$utility"
|
||||
"$utility"
|
||||
printf '\n'
|
||||
done
|
||||
22
tests/posix/bin/cat
Executable file
22
tests/posix/bin/cat
Executable file
|
|
@ -0,0 +1,22 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
# Strictly POSIX-compliant cat(1) implementation. See cat(1p)
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
-u) args="$(printf '%s %s\n' "$args" "$arg")" ;;
|
||||
*) args="$(printf -- '%s -i %s\n' "$args" "$arg")" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# See IEEE Std 1003.1-2017 3.282
|
||||
# https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282
|
||||
IFS=' '
|
||||
|
||||
mm $args
|
||||
12
tests/posix/bin/false
Executable file
12
tests/posix/bin/false
Executable file
|
|
@ -0,0 +1,12 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 DTB <trinity@trinity.moe>
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
# Strictly POSIX-compliant false(1) implementation. See false(1p)
|
||||
|
||||
false "$@"
|
||||
11
tests/posix/bin/true
Executable file
11
tests/posix/bin/true
Executable file
|
|
@ -0,0 +1,11 @@
|
|||
#!/bin/sh
|
||||
# Copyright (c) 2024 DTB <trinity@trinity.moe>
|
||||
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
# Strictly POSIX-compliant true(1) implementation. See true(1p)
|
||||
true "$@"
|
||||
5
tests/posix/posix_env
Normal file
5
tests/posix/posix_env
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
#!/bin/sh
|
||||
|
||||
set -ex
|
||||
|
||||
PATH="$PWD/bin:$PATH"
|
||||
38
tests/test.sh
Executable file
38
tests/test.sh
Executable file
|
|
@ -0,0 +1,38 @@
|
|||
#!/bin/sh
|
||||
|
||||
# Copyright (c) 2023–2024 Emma Tebibyte <emma@tebibyte.media>
|
||||
# SPDX-License-Identifier: FSFAP
|
||||
#
|
||||
# Copying and distribution of this file, with or without modification, are
|
||||
# permitted in any medium without royalty provided the copyright notice and this
|
||||
# notice are preserved. This file is offered as-is, without any warranty.
|
||||
|
||||
set -e
|
||||
|
||||
. "tests/locales/$LANG"
|
||||
|
||||
export BIN=build/bin
|
||||
|
||||
if ! ls Makefile >/dev/null 2>&1
|
||||
then
|
||||
printf '%s: %s\n' "$0" "$RUN_ERR" 1>&2
|
||||
exit 1
|
||||
fi
|
||||
|
emma marked this conversation as resolved
trinity
commented
This seems ineligant. Programmatically print the test suite name at This seems ineligant. Programmatically print the test suite name at `tests/$suite/Name`, falling back to the folder name itself.
```sh
for suite in tests/*; do
if ! scrut -d "$suite"
then continue
fi
s="$suite" sh -c '
scrut -e "$s"/Name \
&& mm <"$s/Name" \
|| printf "%s\n" "$s"'
| xargs printf 'Testing <%s>...\n'
# testing stuff
done
|
||||
|
||||
for script in tests/bonsai/*.sh; do
|
||||
export UTIL="$(printf '%s\n' "$script" \
|
||||
| sed -e 's/\.sh//g' -e 's;tests\/bonsai\/;;g')"
|
||||
|
||||
printf '%s: %s: %s\n' "$0" "$UTIL" "$TEST_STR"
|
||||
"$script"
|
||||
printf '\n'
|
||||
done
|
||||
|
||||
for test_util in tests/posix/*.sh; do
|
||||
|
emma marked this conversation as resolved
Outdated
trinity
commented
This should go before the previous This should go before the previous `for` loop as it will crash faster on incorrect invocation.
|
||||
realutil="$(command -v "$(printf '%s\n' "$test" | sed 's/\.sh$//g')")"
|
||||
export realutil
|
||||
export PATH="$BIN:$PATH"
|
||||
printf '%s: %s: %s\n' "$0" "$test" "$TEST_STR"
|
||||
"$test_util"
|
||||
printf '\n'
|
||||
done
|
||||
Loading…
Reference in New Issue
Block a user
GNU false(1), a notoriously POSIX non-compliant implementation, would pass here, as (if I recall) it returns
EXIT_FAILUREfor improper usage. It shouldn't pass, though, because it prints a usage text to the standard output when usage help or version information are queried.I think another good test would be
false --help | wc -c | xargs test 0 =, though you should double check the exit statuses here before adding it.Why would we test for GNU extensions in our implementation of
true(1)?It's testing for bloat. It drives home the fact that we will not compromise on making good, simple tools. If we did our tests would fail us.
But if we changed our minds, we could just change our test, and if someone made a PR with changes like that, they would just change the test too.
That's true. I suppose such a test isn't necessary.