Directory structure #89

New Issue

trinity · 2024-03-30T13:06:19-06:00

trinity commented

2024-03-30 13:06:19 -06:00

The current directory structure of the project looks like this:

- .gitconfig
- CONTRIBUTING
- COPYING
- Makefile
- README
- docs/
  - utility.1
  - another_utility.8
- include/
  - sysexits.h
- src/
  - utility.c
  - another_utility.rs
- tests/

There have been some ideas for other structures based on other existing works. This is a dedicated place for that discussion.

The current directory structure of the project looks like this: ``` - .gitconfig - CONTRIBUTING - COPYING - Makefile - README - docs/ - utility.1 - another_utility.8 - include/ - sysexits.h - src/ - utility.c - another_utility.rs - tests/ ``` There have been some ideas for other structures based on other existing works. This is a dedicated place for that discussion.

trinity added the

question

label 2024-03-30 13:06:19 -06:00

trinity commented

2024-03-30 13:25:03 -06:00

I'm gonna paste a discussion from the IRC/XMPP channel feeling.murderu.us#subgeneral, which I run.

All the users have the same username as on git.tebibyte.media; <aryal>, <emma>, <silt>, and myself <trinity>.

<aryal> Emma: ive been thinking about changing the bonsai coreutils project strcture
<aryal> i dont like how there are binaries and libraries all in one place inside src/
<aryal> so i came up with something like this https://i.imgur.com/EhUVys6.png

- .gitconfig
- CONTRIBUTING
- COPYING
- Makefile
- README
- bin/
  - Makefile
  - utility/
    - Makefile
    - utility.1
    - utility.c
- build/
- config.mk
- dist/
- include/
  - sysexits.h
- lib/
  - strerror.rs
- tests/

<aryal> the man pages are kept like this bin/{name}/{name}.1
<aryal> the bin sources are kept like this bin/{name}/{name}.{c/rs}
<aryal> inspried by openbsd and freebsd source tree
<aryal> tests can be done in a similar manner
<aryal> https://cgit.freebsd.org/src/tree/bin/cat/tests

FreeBSD src/bin/cat/tests

<silt> i am in support of not having binaries and libraries laying flat in src/
<emma> why would we place single source files in directories
<emma> we dont need more directories
<trinity> good morning
<emma> all the tools are single files and the libraries will be soon too. the getopt implementation will soon be replaced with a single-file C binding im writing
<trinity> this is a rare place where emma and i saw eye to eye pretty quickly. everything's a single file
<emma> src/ is for cleanliness, but theres nothing more clean about src/mm/mm.c
<trinity> though i wish getopt-rs was. it should be. i thought about changing it a little before emma proposed using the C getopt
<aryal> is everything going to be single file?
<aryal> the shell too?
<trinity> imo it should be
<aryal> >but theres nothing more clean about src/mm/mm.c
<aryal> im propsing for bin/mm/mm.c
<aryal> also the man page will be in the same dir
<emma> manpages are not binaries
<trinity> so like how my source tree is, https://git.tebibyte.media/trinity/src/src/branch/main

trinity src/

<emma> src/mm/mm.{1,c} would make sense but i dont really see manpages as source either
<trinity> https://git.tebibyte.media/trinity/src/src/branch/main/psroute
<aryal> yes trinity https://git.tebibyte.media/trinity/src/src/branch/main/peek something like this
<emma> i like docs/ because it gives a simple overview of all the things in the repo from a documentation perspective
<emma> a user looking to learn about it before installing can just `cd docs/` and then `man ./dj.1` or whatever
<emma> and they dont have to cd more to see other docs
<trinity> i like my way but i like to see all the source files flat in src
<trinity> imo ideally the tree would be 1 dir with tool.rs|tool.c, tool.1|tool.8, took.mk
<trinity> tool.mk
<trinity> nobody else wants that though
<aryal> usually /docs has stuff like coding style guides, how to hack into it etc
<aryal> its supposed to be the documentation for the source
<emma> do you have a reference for that? i like to follow conventions
<aryal> GNU does that
<aryal> but GNU also has everything flat inside src/
<aryal> they have doc/ src/ and man/
<aryal> man/ for man pages
<aryal> BSD does it a bit different
<aryal> i think doing it the way GNU coreutils would be more resonable
<aryal> src/ has each bin source laid out flat
<aryal> things need to be consistent if you are going to have like 30 single-source binaries with 3 or 5 single file library in the same directory that would be annoying
<aryal> if you are going to lay out binaries all in one dir, each of those files should be seperate binaries
<emma> we’re replacing man(1) with doc(1)
<emma> >things need to be consistent if you are going to have like 30 single-source binaries with 3 or 5 single file library in the same directory that would be annoying
<emma> i find it to be better than like 30 directories, each of which contain 2 files @_@
<aryal> ive push a commit to aryal/coreutils, check it out, i think its pretty neat

considering a different project directory structure

<aryal> im going to sleep
<trinity> i've always found it difficult to navigate gnu coreutils' source
<emma> gn
<aryal> nn
<aryal> ive modified makefiles too so you can locally play with that project structure
<aryal> if you guys approve of this, im thinking about including the another Makefile inside each bin dir
<aryal> this makes all makefiles across the project very small and might be easier to modify and update stuff in the future
<trinity> https://git.tebibyte.media/bonsai/coreutils/issues/89

I'm gonna paste a discussion from the IRC/XMPP channel feeling.[murderu.us](https://murderu.us)#subgeneral, which I run. All the users have the same username as on git.tebibyte.media; <[aryal](https://git.tebibyte.media/aryal/)>, <[emma](https://git.tebibyte.media/emma)>, <[silt](https://git.tebibyte.media/silt)>, and myself <[trinity](https://git.tebibyte.media/trinity)>. ``` <aryal> Emma: ive been thinking about changing the bonsai coreutils project strcture <aryal> i dont like how there are binaries and libraries all in one place inside src/ <aryal> so i came up with something like this https://i.imgur.com/EhUVys6.png ``` ![This figure is transcribed in the next code block.](https://i.imgur.com/EhUVys6.png) ``` - .gitconfig - CONTRIBUTING - COPYING - Makefile - README - bin/ - Makefile - utility/ - Makefile - utility.1 - utility.c - build/ - config.mk - dist/ - include/ - sysexits.h - lib/ - strerror.rs - tests/ ``` ``` <aryal> the man pages are kept like this bin/{name}/{name}.1 <aryal> the bin sources are kept like this bin/{name}/{name}.{c/rs} <aryal> inspried by openbsd and freebsd source tree <aryal> tests can be done in a similar manner <aryal> https://cgit.freebsd.org/src/tree/bin/cat/tests ``` [FreeBSD src/bin/cat/tests](https://cgit.freebsd.org/src/tree/bin/cat/tests) ``` <silt> i am in support of not having binaries and libraries laying flat in src/ <emma> why would we place single source files in directories <emma> we dont need more directories <trinity> good morning <emma> all the tools are single files and the libraries will be soon too. the getopt implementation will soon be replaced with a single-file C binding im writing <trinity> this is a rare place where emma and i saw eye to eye pretty quickly. everything's a single file <emma> src/ is for cleanliness, but theres nothing more clean about src/mm/mm.c <trinity> though i wish getopt-rs was. it should be. i thought about changing it a little before emma proposed using the C getopt <aryal> is everything going to be single file? <aryal> the shell too? <trinity> imo it should be <aryal> >but theres nothing more clean about src/mm/mm.c <aryal> im propsing for bin/mm/mm.c <aryal> also the man page will be in the same dir <emma> manpages are not binaries <trinity> so like how my source tree is, https://git.tebibyte.media/trinity/src/src/branch/main ``` [trinity src/](https://git.tebibyte.media/trinity/src/) ``` <emma> src/mm/mm.{1,c} would make sense but i dont really see manpages as source either <trinity> https://git.tebibyte.media/trinity/src/src/branch/main/psroute <aryal> yes trinity https://git.tebibyte.media/trinity/src/src/branch/main/peek something like this <emma> i like docs/ because it gives a simple overview of all the things in the repo from a documentation perspective <emma> a user looking to learn about it before installing can just `cd docs/` and then `man ./dj.1` or whatever <emma> and they dont have to cd more to see other docs <trinity> i like my way but i like to see all the source files flat in src <trinity> imo ideally the tree would be 1 dir with tool.rs|tool.c, tool.1|tool.8, took.mk <trinity> tool.mk <trinity> nobody else wants that though <aryal> usually /docs has stuff like coding style guides, how to hack into it etc <aryal> its supposed to be the documentation for the source <emma> do you have a reference for that? i like to follow conventions <aryal> GNU does that <aryal> but GNU also has everything flat inside src/ <aryal> they have doc/ src/ and man/ <aryal> man/ for man pages <aryal> BSD does it a bit different <aryal> i think doing it the way GNU coreutils would be more resonable <aryal> src/ has each bin source laid out flat <aryal> things need to be consistent if you are going to have like 30 single-source binaries with 3 or 5 single file library in the same directory that would be annoying <aryal> if you are going to lay out binaries all in one dir, each of those files should be seperate binaries <emma> we’re replacing man(1) with doc(1) <emma> >things need to be consistent if you are going to have like 30 single-source binaries with 3 or 5 single file library in the same directory that would be annoying <emma> i find it to be better than like 30 directories, each of which contain 2 files @_@ <aryal> ive push a commit to aryal/coreutils, check it out, i think its pretty neat ``` [considering a different project directory structure](https://git.tebibyte.media/aryal/coreutils/commit/7ae3b4690828f14eec1380a989fd63924fde54e2) ``` <aryal> im going to sleep <trinity> i've always found it difficult to navigate gnu coreutils' source <emma> gn <aryal> nn <aryal> ive modified makefiles too so you can locally play with that project structure <aryal> if you guys approve of this, im thinking about including the another Makefile inside each bin dir <aryal> this makes all makefiles across the project very small and might be easier to modify and update stuff in the future <trinity> https://git.tebibyte.media/bonsai/coreutils/issues/89 ```

aryal commented

2024-03-31 00:41:10 -06:00

I forgot to take into consideration that this repo is only for the bonsai coreutils, the freebsd-src structure would be more suitable for the entire bonsai computer system source tree.

lib/ might be pointless if its only going to have wrappers and bindings to existing libraries.

<emma> ... the getopt implementation will soon be replaced with a single-file C binding im writing ...

and if there are going to be any libraries written in the future as a part of bonsai computer system, it will probably not be a part of the coreutils source tree.

There are many ways to approach this, the GNU operating system does not have a "unified" source tree like freebsd does, but GNU was a thing years before git.

With git there could be a single source tree which will contains the submodule links to bonsai projects like the coreutils (and maybe other projects like buildutils, init systems and package management) and a build system to build the entire operating system in one go.

The dir sys/ in the bonsai source tree could be have submodule which links to the linux kernel source tree, any other kernel with our build system adjustments, and also drivers and related things.

The dir lib/ in the bonsai source tree could have submodules for musl libc and other core libraries which would be part of the operating system.

The dir bin/init could be a submodule for a init system we maintain for the bonsai operating system and so on.

All this considering the bonsai projects wants to be a A simple operating system to replace POSIX-compliant ones. as emma states in faer web page. The things im suggesting might sound grand but this is a good approach to maintaining operating systems.

After the recent xz incident I more and more believe in having my operating system as simple as possible, and I am willing to maintain a small set of packages like the coreutils, init systems and basic libraries which should just be enough to set up a usable operating system and install nix or guix and set up containers to set up rest of my development and work environment.

I forgot to take into consideration that this repo is only for the `bonsai coreutils`, the `freebsd-src` structure would be more suitable for the entire `bonsai computer system` source tree. `lib/` might be pointless if its only going to have wrappers and bindings to existing libraries. > ` <emma> ... the getopt implementation will soon be replaced with a single-file C binding im writing ...` and if there are going to be any libraries written in the future as a part of `bonsai computer system`, it will probably not be a part of the coreutils source tree. There are many ways to approach this, the GNU operating system does not have a "unified" source tree like freebsd does, but GNU was a thing years before `git`. With `git` there could be a single source tree which will contains the submodule links to `bonsai` projects like the coreutils (and maybe other projects like `buildutils`, init systems and package management) and a build system to build the entire operating system in one go. The dir `sys/` in the `bonsai` source tree could be have submodule which links to the linux kernel source tree, any other kernel with our build system adjustments, and also drivers and related things. The dir `lib/` in the `bonsai` source tree could have submodules for musl libc and other core libraries which would be part of the operating system. The dir `bin/init` could be a submodule for a init system we maintain for the `bonsai` operating system and so on. All this considering the `bonsai` projects wants to be a `A simple operating system to replace POSIX-compliant ones.` as emma states in faer [web page](https://tebibyte.media/~emma/). The things im suggesting might sound grand but this is a good approach to maintaining operating systems. After the recent `xz` incident I more and more believe in having my operating system as simple as possible, and I am willing to maintain a small set of packages like the coreutils, init systems and basic libraries which should just be enough to set up a usable operating system and install `nix` or `guix` and set up containers to set up rest of my development and work environment.

sashakoshka commented

2024-04-01 08:30:37 -06:00

My thoughts:

Neither source files and manpages are binaries, why put them in a directory called bin?
I've been told this project is accepting tools written in any compiled systems language. This poses a problem if each tool has a single source file, all sharing the same directory, because not all languages support this.

I usually do something like this:

src
- cmd
  - tool1
    - sourcefile
  - tool2
    - sourcefile
  - tool3...
    - sourcefile
- lib1
  - sourcefile
- lib2
  - sourcefile
- lib3...
  - sourcefile

My thoughts: - Neither source files and manpages are binaries, why put them in a directory called `bin`? - I've been told this project is accepting tools written in any compiled systems language. This poses a problem if each tool has a single source file, all sharing the same directory, because not all languages support this. I usually do something like this: - src - cmd - tool1 - sourcefile - tool2 - sourcefile - tool3... - sourcefile - lib1 - sourcefile - lib2 - sourcefile - lib3... - sourcefile

emma commented

2024-04-01 08:57:55 -06:00

I guess this hasn’t been put anywhere specifically but Trinity and I have decided that for now Rust and C are our focus and that if someone contributes with another language we will review on a case-by-case basis. I should add this information to CONTRIBUTING.

👍 1

trinity commented

2024-04-01 16:22:06 -06:00

I like C and Rust because they interoperate well enough, only requiring a handful of utilities keeps the toolchain easy to explain/install/use, and the two languages cover a very wide variety of use cases (Rust for its useability and C for its bit-twidling ease).

I would be fine with other languages only if they are as easy as the currently-used ones and they prove very useful for the needed task. It wouldn't and shouldn't be hard to fit more languages into Bonsai. However, I don't believe accommodating languages that only work with directories and not single files is necessary - I can't name a language that requires this, and any that does is probably too complicated to coexist with these tools anyway.

I like C and Rust because they interoperate well enough, only requiring a handful of utilities keeps the toolchain easy to explain/install/use, and the two languages cover a very wide variety of use cases (Rust for its useability and C for its bit-twidling ease). I would be fine with other languages only if they are as easy as the currently-used ones and they prove very useful for the needed task. It wouldn't and shouldn't be hard to fit more languages into Bonsai. However, I don't believe accommodating languages that *only* work with directories and not single files is necessary - I can't name a language that requires this, and any that does is probably too complicated to coexist with these tools anyway.

emma commented

2024-04-01 17:38:33 -06:00

I don't believe accommodating languages that only work with directories and not single files is necessary

I agree with this sentiment. The only other language that has seriously been brought up with regard to Bonsai is Go and single-file programs are simple in it.

> I don't believe accommodating languages that only work with directories and not single files is necessary I agree with this sentiment. The only other language that has seriously been brought up with regard to Bonsai is Go and single-file programs are simple in it.

emma commented

2024-04-01 20:43:33 -06:00

lib/ might be pointless if its only going to have wrappers and bindings to existing libraries.
<emma> ... the getopt implementation will soon be replaced with a single-file C binding im writing ...

#91

> lib/ might be pointless if its only going to have wrappers and bindings to existing libraries. > > > <emma> ... the getopt implementation will soon be replaced with a single-file C binding im writing ... #91

sashakoshka commented

2024-04-02 13:04:11 -06:00

I can't name a language that requires this, and any that does is probably too complicated to coexist with these tools anyway.

Mine does, although it's not production ready yet and this may change.

> I can't name a language that requires this, and any that does is probably too complicated to coexist with these tools anyway. Mine does, although it's not production ready yet and this may change.

emma pinned this 2024-06-26 13:50:28 -06:00

emma commented

2024-06-26 13:51:35 -06:00

Issues that have been brought up to me verbally with this layout by @silt is detailed in this paper: https://www.accu.org/journals/overload/14/71/miller_2004/

emma unpinned this 2024-06-26 15:59:24 -06:00

emma pinned this 2024-06-29 17:26:56 -06:00

emma commented

2024-06-29 17:39:02 -06:00

@aryal thoughts?

emma unpinned this 2024-07-15 14:18:28 -06:00

emma commented

2024-12-16 15:54:16 -07:00

I’m closing this for now to clear up our issues. If anyone else has any concerns regarding the directory structure, please feel free to comment here.

emma closed this issue

2024-12-16 15:54:16 -07:00

emma reopened this issue

2025-04-01 12:17:25 -06:00

emma commented

2025-04-01 12:20:59 -06:00

After exposure to BSD operating systems and given the direction of Bonsai as an operating system of its own, it makes sense to make this tree the home of the whole of the Bonsai Computer System. Any ideas toward how this could be set up should be placed in this issue.