WIP: testing #96

Draft
emma wants to merge 15 commits from testing into main
15 changed files with 234 additions and 7 deletions

View File

@ -97,9 +97,10 @@ their editor or terminal.
For usage text and help messages, do not implement a -h option. Instead, print
usage information when any erroneous option is specified. Follow the NetBSD
style guide for the usage texts output format [1].
style guide for the usage texts output format [0].
[1] <http://cvsweb.netbsd.org/bsdweb.cgi/~checkout~/src/share/misc/style>
If committing a new utility, please include tests and documentation (see
tests/ and docs/) for the new tool.
If committing a new source file, format the commit message following these
guidelines:
@ -128,6 +129,7 @@ $ git commit -m 'tool(1): fix #42 & add feature x'
Commit messages should be written in the present tense.
[0] <http://cvsweb.netbsd.org/bsdweb.cgi/~checkout~/src/share/misc/style>
--
This work © 20232024 by Emma Tebibyte is licensed under CC BY-SA 4.0. To view a
copy of this license, visit <http://creativecommons.org/licenses/by-sa/4.0/>

View File

@ -14,6 +14,7 @@
PREFIX=/usr/local
CC=cc
MAKE=make
RUSTC=rustc
.PHONY: all
@ -39,8 +40,8 @@ install: dist
cp -r dist/* $(PREFIX)/
.PHONY: test
test: build
tests/posix-compat.sh
test: all
tests/test.sh
$(RUSTC) --test src/getopt-rs/lib.rs -o build/test/getopt
build/o/libsysexits.rlib: build

27
tests/bonsai/dj.sh Executable file
View File

@ -0,0 +1,27 @@
#!/bin/sh
# Copyright (c) 2024 DTB <trinity@trinity.moe>
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
. tests/bonsai/test_env
! dj -h
# This test is theoretically Linux-dependent; write(2) should return -1 on
# error.
# Right now dj(1) interprets the return value of write(2) as the amount of
# bytes written. This can decrement the stored quantity of bytes written,
# which is an int, so doesn't underflow but goes negative. dj(1) tries to
# again to write(2) if an error occurs in which no bytes are written, so in
# total two write(2)s are attempted and so the written byte quantity is -2.
# This is a bug and will change, but for now is at least documented.
dj -Hi /dev/zero -o /dev/full \
| xargs -I out "$BIN/strcmp" '1+0 > 0+0; 1024 > -2' out
# Read nothing from /dev/null, write nothing to /dev/null.
dj -Hi /dev/null -o /dev/null \
| xargs -I out "$BIN/strcmp" '0+0 > 0+0; 0 > 0' out

13
tests/bonsai/false.sh Executable file
View File

@ -0,0 +1,13 @@
#!/bin/sh
# Copyright (c) 2024 DTB <trinity@trinity.moe>
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
. tests/bonsai/test_env
! false
! false -h
Review

GNU false(1), a notoriously POSIX non-compliant implementation, would pass here, as (if I recall) it returns EXIT_FAILURE for improper usage. It shouldn't pass, though, because it prints a usage text to the standard output when usage help or version information are queried.

I think another good test would be false --help | wc -c | xargs test 0 =, though you should double check the exit statuses here before adding it.

GNU false(1), a notoriously POSIX non-compliant implementation, would pass here, as (if I recall) it returns `EXIT_FAILURE` for improper usage. It shouldn't pass, though, because [it prints a usage text to the standard output when usage help or version information are queried](https://git.savannah.gnu.org/cgit/coreutils.git/tree/src/true.c#n36). I think another good test would be `false --help | wc -c | xargs test 0 =`, though you should double check the exit statuses here before adding it.
Review

Why would we test for GNU extensions in our implementation of true(1)?

Why would we test for GNU extensions in our implementation of `true(1)`?
Review

It's testing for bloat. It drives home the fact that we will not compromise on making good, simple tools. If we did our tests would fail us.

It's testing for bloat. It drives home the fact that we *will not* compromise on making good, simple tools. If we did our tests would fail us.
Review

But if we changed our minds, we could just change our test, and if someone made a PR with changes like that, they would just change the test too.

But if we changed our minds, we could just change our test, and if someone made a PR with changes like that, they would just change the test too.
Review

That's true. I suppose such a test isn't necessary.

That's true. I suppose such a test isn't necessary.

21
tests/bonsai/fop.sh Executable file
View File

@ -0,0 +1,21 @@
#!/bin/sh
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
. tests/bonsai/test_env
! fop -h
"$BIN/strcmp" "$(printf 'test0␞test1␞test2\n' | fop 1 sed 's/1/4/g')" \
'test0␞test4␞test2'
"$BIN/strcmp" "$(printf 'test0 test1 test2\n' | fop -d' ' 2 sed 's/2/4/g')" \
'test0 test1 test4'
! printf 'test\n' | fop 1 cat
! printf 'test\n' | fop 'test' cat
! printf 'test\n' | fop -d'test' cat

26
tests/bonsai/hru.sh Executable file
View File

@ -0,0 +1,26 @@
#!/bin/sh
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
. tests/bonsai/test_env
alias strcmp="$BIN/strcmp"
alias rpn="$BIN/rpn"
strcmp "$(printf '1234\n' | hru)" '1.2 kB'
strcmp "$(printf '0\n' | hru)" '0 B'
# doesnt currently work but would be useful for testing for regressions
#n=1
#while "$BIN/true"; do
# n="$(rpn "$n" 10 ×)"
#
# printf '%s\n' "$n" | hru || break
#done
#printf 'integer limit: ~%s\n' "$(rpn "$n" 10 ÷)"
! printf '%s\n' '-1' | hru

25
tests/bonsai/intcmp.sh Executable file
View File

@ -0,0 +1,25 @@
#!/bin/sh
# Copyright (c) 2024 DTB <trinity@trinity.moe>
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
. tests/bonsai/test_env
intcmp -e 3 3 3
intcmp -g 3 2 1
intcmp -l 1 2 3
intcmp -ge 3 3 1
intcmp -le 1 3 3
intcmp -gl 1 2 3
intcmp -egl 3 1 1 2
! intcmp -e 1 2 3
! intcmp -g 1 3 3
! intcmp -l 3 3 1
! intcmp -ge 1 2 3
! intcmp -le 3 2 1
! intcmp -gl 3 3 3
! intcmp -egl foo

19
tests/bonsai/mm.sh Executable file
View File

@ -0,0 +1,19 @@
#!/bin/sh
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
. tests/bonsai/test_env
exec 3>&1
! mm -h
# mm(1) will error if positional arguments are given without -i or -o
! mm argument
# check if stderr is empty upon specifying -e
! "$BIN/strcmp" "$(printf 'test\n' | mm -i - -e 2>&1 1>&3)" ''

16
tests/bonsai/strcmp.sh Executable file
View File

@ -0,0 +1,16 @@
#!/bin/sh
# Copyright (c) 2024 DTB <trinity@trinity.moe>
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
. tests/bonsai/test_env
strcmp equals equals
! strcmp inequals equals
strcmp - -
strcmp -h
! strcmp nocmp

5
tests/bonsai/test_env Normal file
View File

@ -0,0 +1,5 @@
#!/bin/sh
set -x
alias "$UTIL=$BIN/$UTIL"

13
tests/bonsai/true.sh Executable file
View File

@ -0,0 +1,13 @@
#!/bin/sh
Review

Why is this separate from tests/posix-compat/true.sh?

Why is this separate from `tests/posix-compat/true.sh`?
Review

This is true(1) from Bonsai and the one in posix-compat/true.sh is true(1p)

This is `true(1)` from Bonsai and the one in `posix-compat/true.sh` is `true(1p)`
Review

Our true(1) and false(1) implementations are POSIX without extensions and shouldn't be extended. Maybe it would be better to only have them in the POSIX compatibility section.

Our true(1) and false(1) implementations are POSIX without extensions and shouldn't be extended. Maybe it would be better to only have them in the POSIX compatibility section.
Review

It makes sense to have them in both because the true(1p) test script leverages our script; however, I have just realized that I have implemented the POSIX test suite in a different way than I originally imagined and will make more changes before we can continue this conversation.

It makes sense to have them in both because the true(1p) test script *leverages* our script; however, I have just realized that I have implemented the POSIX test suite in a different way than I originally imagined and will make more changes before we can continue this conversation.
# Copyright (c) 2024 DTB <trinity@trinity.moe>
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
. tests/bonsai/test_env
true
true -h

22
tests/posix/bin/cat Executable file
View File

@ -0,0 +1,22 @@
#!/bin/sh
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
# Strictly POSIX-compliant cat(1) implementation. See cat(1p)
for arg in "$@"; do
case "$arg" in
-u) args="$(printf '%s %s\n' "$args" "$arg")" ;;
*) args="$(printf -- '%s -i %s\n' "$args" "$arg")" ;;
esac
done
# See IEEE Std 1003.1-2017 3.282
# https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282
IFS=' '
mm $args

12
tests/posix/bin/false Executable file
View File

@ -0,0 +1,12 @@
#!/bin/sh
# Copyright (c) 2024 DTB <trinity@trinity.moe>
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
# Strictly POSIX-compliant false(1) implementation. See false(1p)
false "$@"

11
tests/posix/bin/true Executable file
View File

@ -0,0 +1,11 @@
#!/bin/sh
# Copyright (c) 2024 DTB <trinity@trinity.moe>
# Copyright (c) 2024 Emma Tebibyte <emma@tebibyte.media>
# SPDX-License-Identifier: FSFAP
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice and this
# notice are preserved. This file is offered as-is, without any warranty.
# Strictly POSIX-compliant true(1) implementation. See true(1p)
true "$@"

View File

@ -9,16 +9,30 @@
set -e
export BIN=build/bin
if ! ls Makefile >/dev/null 2>&1
then
printf '%s: Run this script in the root of the project.\n' "$0" 1>&2
exit 1
fi
printf "Starting POSIX compatibility testing.\n"
printf "Starting Bonsai testing.\n\n"
Review

This seems ineligant. Programmatically print the test suite name at tests/$suite/Name, falling back to the folder name itself.

for suite in tests/*; do
    if ! scrut -d "$suite"
        then continue
    fi
    s="$suite" sh -c '
            scrut -e "$s"/Name \
                && mm <"$s/Name" \
                || printf "%s\n" "$s"'
        | xargs printf 'Testing <%s>...\n'
    # testing stuff
done
This seems ineligant. Programmatically print the test suite name at `tests/$suite/Name`, falling back to the folder name itself. ```sh for suite in tests/*; do if ! scrut -d "$suite" then continue fi s="$suite" sh -c ' scrut -e "$s"/Name \ && mm <"$s/Name" \ || printf "%s\n" "$s"' | xargs printf 'Testing <%s>...\n' # testing stuff done
for utility in tests/posix/*; do
printf '%s: %s: Testing utility.\n' "$0" "$utility"
for script in tests/bonsai/*.sh; do
export UTIL="$(printf '%s\n' "$script" \
| sed -e 's/\.sh//g' -e 's;tests\/bonsai\/;;g')"
printf '%s: %s: Testing utility.\n' "$0" "$UTIL"
"$script"
printf '\n'
done
printf "Starting POSIX compatibility testing.\n\n"
for test in tests/posix/*.sh; do
export PATH="$BIN:$PATH"
printf '%s: %s: Testing utility.\n' "$0" "$test"
"$utility"
printf '\n'
done