From 08631865e944731cf0e7cf78ab1dea3e48163400 Mon Sep 17 00:00:00 2001 From: Dylan Araps Date: Mon, 19 Jul 2021 14:04:01 +0300 Subject: [PATCH] docs: initial rewrite --- contrib/kiss-graph | 18 -- doc/package-manager.txt | 619 +++++----------------------------------- 2 files changed, 71 insertions(+), 566 deletions(-) delete mode 100755 contrib/kiss-graph diff --git a/contrib/kiss-graph b/contrib/kiss-graph deleted file mode 100755 index 88fa904..0000000 --- a/contrib/kiss-graph +++ /dev/null @@ -1,18 +0,0 @@ -#!/bin/sh -e -# list all packages by size - -cd "$KISS_ROOT/var/db/kiss/installed" - -[ "$1" ] || set -- * - -for pkg do - sum=$(kiss size "$pkg" 2>&1 >/dev/null) - num=${sum%%[A-Z][A-Z] *} - - case ${sum%% *} in - *MB) num=$((num * 1024)) ;; - *GB) num=$((num * 1024 * 1024)) ;; - esac - - printf '%10s %s\n' "$num" "$pkg" -done diff --git a/doc/package-manager.txt b/doc/package-manager.txt index 53971fa..537ab80 100644 --- a/doc/package-manager.txt +++ b/doc/package-manager.txt @@ -2,41 +2,20 @@ KISS PACKAGE MANAGER ________________________________________________________________________________ The KISS package manager is a self-contained POSIX shell script which is written -in a highly portable way. It should run in anywhere with a POSIX shell and -POSIX compliant coreutils. +in a highly portable way. It should run anywhere with a POSIX shell and POSIX +compliant core utilities. Source: $/kisslinux/kiss -[0.0] Index +Index ________________________________________________________________________________ - Usage [1.0] - Dependencies [2.0] - -- Interesting Features [3.0] - - Runtime dependency detector built around 'ldd' [3.1] - - Fully dynamic (and automatic) alternatives system [3.2] - - 3-way handshake for files in /etc/ [3.3] - -- Configuration [4.0] - -- Repositories [5.0] - - What is a repository? [5.1] - - Enabling a remote repository [5.2] - - Preventing a package from receiving updates [5.3] - - Package fallbacks [5.4] - - Bypassing KISS_PATH [5.5] - -- Package Manager Hooks [6.0] - - Usage [6.1] - - Removing unneeded files from packages [6.2] - - Drop into a subshell on build fail [6.3] - -- Package Manager Extensions [7.0] - -- Tips and Tricks [8.0] - - Swap grep implementations for a major speed up [8.1] +- Configuration [3.0] +- Package Manager Hooks [4.0] +- Package Manager Extensions [5.0] [1.0] Usage @@ -64,13 +43,6 @@ ________________________________________________________________________________ [2.0] Dependencies ________________________________________________________________________________ -POSIX utilities are used where appropriate _and_ where they exist to solve a -particular problem. Utilities of which there is only a single and cross-platform -implementation are considered "portable" (git, curl, etc) - -If a dependency can be made optional, it will be made so. Dependencies are also -kept to a minimum (though we must also remain realistic). - +----------------------+--------------------------------------------+----------+ | Dependency | Reason for use | Required | +----------------------+--------------------------------------------+----------| @@ -107,16 +79,14 @@ kept to a minimum (though we must also remain realistic). sources will simply become unusable. [2] There is no standard utility for the generation of sha256 checksums. While - sha256sum is listed above, the package manager also supports sha256, shasum - and openssl as fallbacks. + sha256sum is listed above, the package manager also supports sha256, shasum, + digest and openssl as fallbacks. -[3] The tar command has no standard! This came as a shock. The POSIX equivalent - is "pax" though this isn't in wide use (at least on Linux). +[3] The tar command has no standard! The POSIX equivalent is "pax" though this + has its own issues (-s is unclear about links). - Our usage of tar is merely, cf, xf and tf. A patch is applied to sbase's - tar so that it supports "dashless" arguments (as all others do). Our usage - of tar cannot become any more basic than it is now. Portability should no - longer be a concern. + Our usage of tar is merely, cf, xf and tf. Our usage of tar cannot become + any more basic than it is now. Portability should not be a concern. Tested tar implementations include: busybox, toybox, sbase, GNU and libarchive (though all tar implementations should work in theory). @@ -127,10 +97,6 @@ kept to a minimum (though we must also remain realistic). The other compression methods are optional as no package sources (in the official repositories) make use of them. - If a compression method has 1-3 uses (hasn't yet happened), the compression - method will simply become a 'make' dependency of the package until usage - increases to a "normality". - [5] A privilege escalation utility is only needed when using the package manager as a normal user for system-wide package installation. @@ -146,132 +112,11 @@ kept to a minimum (though we must also remain realistic). contain any arguments whatsoever. This makes our usage of 'strip' non-POSIX. That being said, our usage is - compatible with these 'strip' implementations. - - strip binutils, elfutils, elftoolchain, llvm, etc. - readelf Just needs -d flag. + compatible with most 'strip' implementations (binutils, elfutils, + elftoolchain, llvm, etc). - -[3.0] Interesting Features -________________________________________________________________________________ - - - [3.1] Runtime dependency detector built around 'ldd'/'readelf' - ____________________________________________________________________________ - - Dynamic dependencies brought in by build systems (which are missing from the - package's dependency list) are fixed on-the-fly by checking which libraries - link to the package's files. - - This prevents an incomplete dependency list from causing system breakage as - the package manager is able to complete the list. - - A lot of packages make use of this "implicit" to "explicit" dependency list - "conversion" to provide optional dependencies. - - Example output: - - +--------------------------------------------------------------------------+ - | | - | -> libXmu Checking for missing dependencies | - | --- /home/dylan/conf/cache/kiss/build-4477/d | - | depends | - | @@ -1,3 +1,8 @@ | - | +libX11 | - | +libXau | - | libXext | - | libXt | - | +libxcb | - | xorg-util-macros make | - | | - +--------------------------------------------------------------------------+ - - - [3.2] Fully dynamic (and automatic) alternatives system - ____________________________________________________________________________ - - Any file conflicts between two packages automatically become choices in the - alternatives system. - - This allows one to swap providers of files without needing to explicitly - tell the package manager that two packages conflict, provide the same - utilities, etc. - - In other words, no changes need to be made to packages. In fact, nothing - needs to be done at all. It's entirely automatic. - - +--------------------------------------------------------------------------+ - | List available alternatives ('a' is an alias to 'alternatives'). | - +--------------------------------------------------------------------------+ - | | - | $ kiss a | - | gnugrep /usr/bin/grep | - | ncurses /usr/bin/clear | - | | - +--------------------------------------------------------------------------+ - | Swap to GNU grep. | - +--------------------------------------------------------------------------+ - | | - | $ kiss a gnugrep /usr/bin/grep | - | -> Swapping '/usr/bin/grep' from busybox to gnugrep | - | | - +--------------------------------------------------------------------------+ - | Swap back to busybox grep. | - +--------------------------------------------------------------------------+ - | | - | $ kiss a busybox /usr/bin/grep | - | -> Swapping '/usr/bin/grep' from gnugrep to busybox | - | | - +--------------------------------------------------------------------------+ - | Swap to all alternatives for a given package (sbase for example). | - +--------------------------------------------------------------------------+ - | | - | $ kiss a | grep ^sbase | kiss a - | - | -> Swapping '/usr/bin/cat' from busybox to sbase | - | -> Swapping '/usr/bin/cut' from busybox to sbase | - | -> Swapping '/usr/bin/yes' from busybox to sbase | - | ...Many more lines of output... | - | | - +--------------------------------------------------------------------------+ - - The above command works as the output of the alternatives listing is - directly usable as input to 'kiss a'. - - - [3.3] 3-way handshake for files in /etc/ - ____________________________________________________________________________ - - Files in /etc/ are handled differently to those elsewhere on the system. A - reinstallation or update to a package will not always overwrite these files. - - Instead, a 3-way handshake happens during installation to determine how the - new /etc/ file should be handled. - - If the user has made modifications to the file and those modifications - differ to the to-be-installed package's file, the file is installed with the - suffix '.new' - - If the user hasn't touched the file, it will be automatically overwritten by - the package manager as it will contain updated/new contents.. - - If the user has touched the file but the file has not changed between - package versions, it will simply be skipped over. - - +--------------------------------------------------------------------------+ - | Example (with sha256 checksums truncated to fit). | - +--------------------------------------------------------------------------+ - | | - | -> opendoas Doing 3-way handshake for etc/doas.conf | - | Previous: 1656eee66c235cb717f9f8f35aa9c3587bb768d7fe etc/doas.conf | - | System: 4a51871b3190fa74726ea2b12ffafb96f40c172b68 etc/doas.conf | - | New: 1656eee66c235cb717f9f8f35aa9c3587bb768d7fe etc/doas.conf | - | -> Skipping etc/doas.conf | - | | - +--------------------------------------------------------------------------+ - - -[4.0] Configuration +[3.0] Configuration ________________________________________________________________________________ The package manager has no configuration files and no changes need to be made to @@ -279,11 +124,9 @@ the system prior to its use. While there is no configuration file, this does not mean that there is no possibility for configuration. The package manager can be configured via the use of environment variables. I -believe this to be the best configuration method (where realistic). Environment -variables can be set system-wide, per-user, conditionally, for a single -invocation, etc, etc. - -They require little to no extra code in the package manager to support them. +believe this to be the best configuration method for the problem at hand. +Environment variables can be set system-wide, per-user, conditionally, for a +single invocation and so on. +-----------------+------------------------------------------------------------+ | Variable | Description | @@ -325,415 +168,95 @@ They require little to no extra code in the package manager to support them. | | Set to '0' to disable. | | | | | KISS_SU | Force usage of a different sudo tool. | -| | Valid: su, sudo, doas, ssu | +| | NOTE: Tool must support sudo-like arguments. | | | | | KISS_TMPDIR | Temporary directory for builds. Can be set to a tmpfs so | | | builds happen in memory. | | | | +-----------------+------------------------------------------------------------+ -There are also a myriad of "3rd-party" environment variables which control GCC, -Make, CMake, etc. These aren't used by the package manager. They're used by the -tools called by the package's build script. -+-----------------+------------------------------------------------------------+ -| Variable | Description | -+-----------------+------------------------------------------------------------+ -| XDG_CACHE_HOME | Cache directory location. | -| | | -| CC | C compiler. | -| CXX | C++ compiler. | -| AR | Archive tool. | -| NM | Symbol tool. | -| RANLIB | Index tool. | -| | | -| CFLAGS | C compiler flags. | -| CXXFLAGS | C++ compiler flags. | -| LDFLAGS | Linker flags. | -| MAKEFLAGS | Make flags. | -| SAMUFLAGS | Samurai flags. | -| RUSTFLAGS | Rust compiler flags. | -| | | -| CMAKE_GENERATOR | 'Unix Makefiles' or 'Ninja'. | -| | | -+-----------------+------------------------------------------------------------+ - - -[5.0] Repositories -________________________________________________________________________________ - -Repository management in KISS is very simple. Repositories are configurable via -an environment variable. This environment variable can be set system-wide, -per-user, conditionally (via a script or program), for a single invocation, etc. - -The environment variable is called '$KISS_PATH' and is functionally identical to -the '$PATH' variable. A colon separated list of paths in other words. - -+------------------------------------------------------------------------------+ -| Example KISS_PATH | -+------------------------------------------------------------------------------+ -| | -| $ KISS_PATH=/var/db/kiss/repo/core:/var/db/kiss/repo/extra | -| | -+------------------------------------------------------------------------------+ - -In the above example, two repositories are enabled (Core and Extra). The package -manager will search this list for packages in the order it is written. - -Repositories can live anywhere on the system. In your '$HOME' directory, -somewhere system-wide in '/', etc. The only requirement is that a full path be -used. - - - [5.1] What is a repository? - ____________________________________________________________________________ - - A KISS repository is simply a directory of directories. The quickest way to - get started is as follows. - - +--------------------------------------------------------------------------+ - | 1. Create the repository | - +--------------------------------------------------------------------------+ - | | - | $ mkdir -p repo | - | $ cd repo | - | | - +--------------------------------------------------------------------------+ - | 2. Let's 'fork' a few packages into our new repository. | - +--------------------------------------------------------------------------+ - | | - | $ kiss fork curl | - | $ kiss fork xz | - | $ kiss fork zlib | - | | - +--------------------------------------------------------------------------+ - - This is now a fully usable repository and it can be added to your KISS_PATH. - - - [5.2] Enabling a remote repository - ____________________________________________________________________________ - - Let's assume that our KISS_PATH matches the above example (Core and Extra). - - +--------------------------------------------------------------------------+ - | 1. Clone the repository | - +--------------------------------------------------------------------------+ - | | - | # This can live anywhere on the system. | - | $ git clone https://example.com/repo | - | | - +--------------------------------------------------------------------------+ - | 2. Enable the repository | - +--------------------------------------------------------------------------+ - | | - | $ export KISS_PATH=$KISS_PATH:/path/to/repo | - | | - +--------------------------------------------------------------------------+ - - - [5.3] Preventing a package from receiving updates - ____________________________________________________________________________ - - Preventing a package from receiving updates can be accomplished in a myriad - of different ways. The easiest method is to leverage a user repository. - - +--------------------------------------------------------------------------+ - | 1. Create a new repository | - +--------------------------------------------------------------------------+ - | | - | $ mkdir -p no_updates | - | $ cd no_updates | - | | - +--------------------------------------------------------------------------+ - | 2. Copy the package to the new repository | - +--------------------------------------------------------------------------+ - | | - | $ cp -r /var/db/kiss/installed/PKG_NAME /path/to/no_updates | - | | - +--------------------------------------------------------------------------+ - | 3. Add the new repository to the /START/ of your KISS_PATH | - +--------------------------------------------------------------------------+ - | | - | $ export KISS_PATH=/path/to/no_updates:$KISS_PATH | - | | - +--------------------------------------------------------------------------+ - - The package manager will search KISS_PATH in order. It will see that the - no_updates repository provides PKG_NAME and the version matches that which - is installed. No updates will again happen for the package. - - - [5.4] Package fallbacks - ____________________________________________________________________________ - - If you would like to package something in your own repository but would - like the package manager to prefer other repositories before yours, simply - add your repository to the end of KISS_PATH. - - The moment that your package is available elsewhere, the package manager - will prefer the new location to yours. The list is searched (in order) and - the first match is picked. - - - [5.5] Bypassing KISS_PATH - ____________________________________________________________________________ - - There is a special case where one can bypass the regular KISS_PATH for a - single invocation of the package manager. This has been called "CRUX-like - usage" by users. - - +--------------------------------------------------------------------------+ - | | - | $ cd /path/to/myrepo/firefox | - | $ kiss b | - | | - +--------------------------------------------------------------------------+ - - As seen above, various package manager commands will work without arguments, - so long as you are in a package's repository directory. This will prepend - the current directory to '$KISS_PATH' _only_ for this invocation. - - -[6.0] Package Manager Hooks +[4.0] Package Manager Hooks ________________________________________________________________________________ KISS' package manager is extensible via hooks which fire at various different places inside the utility. Hooks allow the user to modify behavior, add new features or conditionally do things on a per-package basis. -This setting is controlled by the '$KISS_HOOK' environment variable which takes -a colon separated list of absolute file paths to executables. - -Example: - -+------------------------------------------------------------------------------+ -| | -| export KISS_HOOK=$HOME/.local/bin/kiss-hook:/path/to/other/hook | -| | -+------------------------------------------------------------------------------+ - [6.1] Usage ____________________________________________________________________________ + This setting is controlled by the '$KISS_HOOK' environment variable which + takes a colon separated list of absolute file paths to executables. + + +--------------------------------------------------------------------------+ + | | + | export KISS_HOOK=$HOME/.local/bin/kiss-hook:/path/to/other/hook | + | | + +--------------------------------------------------------------------------+ + + + [6.2] List of hooks + ____________________________________________________________________________ + Each hook is executed in the order it appears in KISS_HOOK and is given its own environment/arguments accordingly. The hooks are documented as follows. - +--------------+-----------------------------------------------------------+ - | hook | arguments | - +--------------+-----------------------------------------------------------+ - | build-fail | hook_type pkg_name build_directory | - | post-build | hook_type pkg_name DESTDIR | - | post-install | hook_type pkg_name installed_database | - | post-package | hook_type pkg_name tarball | - | post-update | hook_type | - | pre-build | hook_type pkg_name build_directory | - | pre-extract | hook_type pkg_name DESTDIR | - | pre-install | hook_type pkg_name extract_directory | - | pre-remove | hook_type pkg_name installed_database | - | pre-update | hook_type | - | | | - +--------------------------------------------------------------------------+ + +---------------+--------+----------+--------------------+-----------------+ + | hook | arg1 | arg2 | arg3 | arg4 | + +---------------+--------+----------+--------------------+-----------------+ + | post-build | Type | Package | DESTDIR | | + | post-install | Type | Package | Installed database | | + | post-package | Type | Package | Tarball | | + | post-update | Type | [7] | | | + | pre-build | Type | Package | Build directory | | + | pre-extract | Type | Package | DESTDIR | | + | pre-install | Type | Package | Extracted package | | + | pre-remove | Type | Package | Installed database | | + | pre-update | Type | [7] | | | + | queue-status | Type | Package | Number in queue | Total in queue | + | | | | | | + +---------------+--------+----------+--------------------+-----------------+ - build-fail - ________________________________________________________________________ - - arg1: build-fail - arg2: package name - arg3: build directory - - post-build - ________________________________________________________________________ - - arg1: post-build - arg2: package name - arg3: DESTDIR - - post-install - ________________________________________________________________________ - - arg1: post-install - arg2: package name - arg3: installed database (/var/db/kiss/installed/pkg_name) - - post-package - ________________________________________________________________________ - - arg1: post-package - arg2: package name - arg3: path to tarball - - post-update - ________________________________________________________________________ - - arg1: post-update - env: PWD is set to the current repository. - - pre-build - ________________________________________________________________________ - - arg1: pre-build - arg2: build directory - - pre-extract - ________________________________________________________________________ - - arg1: pre-extract - arg2: DESTDIR - - pre-install - ________________________________________________________________________ - - arg1: pre-install - arg2: extraction directory - - pre-remove - ________________________________________________________________________ - - arg1: pre-remove - arg2: installed database (/var/db/kiss/installed/pkg_name) - - pre-update - ________________________________________________________________________ - - arg1: pre-update - env: PWD is set to the current repository. + [7] The -update hooks start in the current repository. In other words, you + can operate on the repository directly or grab the value from '$PWD'. - [6.2] Removing unneeded files from packages - ____________________________________________________________________________ - - Packages can contain files which you will have no use for. A simple hook can - be defined to remove them from packages. - - NOTE: This is the default 'KISS_HOOK' script. If defining your own, be sure - to include this if you would like to continue to remove these files. - - +--------------------------------------------------------------------------+ - | | - | case $1 in | - | post-build) | - | # Ensure that '$3' is set. | - | : "${3:?DESTDIR is unset}" | - | | - | rm -rf "$3/usr/share/gettext" \ | - | "$3/usr/share/polkit-1" \ | - | "$3/usr/share/locale" \ | - | "$3/usr/share/info" \ | - | "$3/usr/lib/charset.alias" | - | ;; | - | esac | - | | - +--------------------------------------------------------------------------+ - - - [6.3] Drop into a subshell on build fail - ____________________________________________________________________________ - - Handy for debugging. - - +--------------------------------------------------------------------------+ - | | - | case $1 in | - | build-fail) | - | log "$2" "Dropped into subshell" | - | sh >/dev/tty | - | ;; | - | esac | - | | - +--------------------------------------------------------------------------+ - - -[7.0] Package Manager Extensions +[5.0] Package Manager Extensions ________________________________________________________________________________ Anything in the user's '$PATH' which matches the glob 'kiss-*' will be directly usable via the package manager. For example, 'kiss-size' is also usable as 'kiss size' (and even 'kiss si') (the shortest available alias). -The detected 'kiss-*' utilities will appear in the package manager's help output -with the second line in the script acting as a doc-string. +The detected 'kiss-*' utilities will appear in the package manager's help-ext +output with the second line in the script acting as a doc-string. Example help output: -+--------------------------------------------------------------------------+ -| kiss extensions | -+--------------------------------------------------------------------------+ -| | -| -> Installed extensions (kiss-* in $PATH) | -| -> chbuild Create/destroy temporary chroots | -| -> chroot Enter a kiss chroot | -| -> depends Display a package's dependencies | -| -> export Installed package to tarball | -| -> fork Fork a package into the current dir | -| -> help Read KISS documentation | -| -> link Link a repo file to another repo | -| -> maintainer Find the maintainer of a package | -| -> manifest Display all files owned by a package | -| -> new Create a boilerplate package | -| -> orphans List orphaned packages | -| -> outdated Check repository packages for updates | -| -> owns Check which package owns a file | -| -> revdepends Packages which depend on package | -| -> size Show the size on disk for a package | -| | -+--------------------------------------------------------------------------+ - -These are in effect, optional utilities which interact with the package system -in one way or another. My hope behind them is to act as an example as to how -easy it is to interface with the plain-text and "static" package system. - -Example utility: - +------------------------------------------------------------------------------+ -| kiss-depends (kiss depends, kiss de) | +| kiss help-ext | +------------------------------------------------------------------------------+ | | -| #!/bin/sh -ef | -| # Display a package's dependencies | -| | -| # Ignore shellcheck as we want the warning's behavior. | -| # shellcheck disable=2015 | -| [ "$1" ] && kiss l "${1:-null}" >/dev/null || { | -| printf 'usage: kiss-depends [pkg]\n' | -| exit 1 | -| } | -| | -| cat "$KISS_ROOT/var/db/kiss/installed/$1/depends" 2>/dev/null | +| -> Installed extensions (kiss-* in $PATH) | +| -> chroot Enter a kiss chroot | +| -> depends Display a package's dependencies | +| -> export Installed package to tarball | +| -> fork Fork a package into the current directory | +| -> help Read KISS documentation | +| -> link Link a repo file to another repo | +| -> maintainer Find the maintainer of a package | +| -> manifest Display all files owned by a package | +| -> new Create a boilerplate package | +| -> orphans List orphaned packages | +| -> outdated Check repository packages for updates | +| -> owns Check which package owns a file | +| -> preferred Lists the owners of all files with conflicts | +| -> revdepends Packages which depend on package | +| -> repo-orphans List packages which aren't present in any repository | +| -> size Show the size on disk for a package | | | +------------------------------------------------------------------------------+ -[8.0] Tips and Tricks -________________________________________________________________________________ - -A lot of the package manager's features are hard to discover or otherwise -non-obvious to its users. This section will document these features, how to use -them and the benefits they bring. - - - [8.1] Swap grep implementations for a major speed up. - ____________________________________________________________________________ - - The default grep implementation in KISS is busybox grep. This version of - grep works very well and supports a large number of features. The one issue - is that it is painfully slow when compared to other popular implementations. - - A fairly major speedup can be attained by swapping to a different grep via - the alternatives system. The fastest grep implementation around is GNU grep - which is available in the official repositories as 'gnugrep'. - - +--------------------------------------------------------------------------+ - | 1. Install GNU grep | - +--------------------------------------------------------------------------+ - | | - | $ kiss b gnugrep | - | | - +--------------------------------------------------------------------------+ - | 2. Swap to GNU grep | - +--------------------------------------------------------------------------+ - | | - | $ kiss a gnugrep /usr/bin/grep | - | | - +--------------------------------------------------------------------------+