diff --git a/Tokens.md b/Tokens.md new file mode 100644 index 0000000..35d6234 --- /dev/null +++ b/Tokens.md @@ -0,0 +1,117 @@ +Arf's syntax is composed of basic elements called tokens. They can be arranged to build higher level syntactical features. + +Important: some characters in this documented may be affected by font ligatures, and do not reflect what the language actually accepts as valid. If it is not immediately apparent what a given character is, paste it into a text field that is not ligaturized. +# List of Tokens +## Newline +A simple line break. + +## Indent +A series of tabs characters. It's value should describe its indentation level. + +## Separator +A series of three dashes (`---`). It is used so separate a file's [metadata](Metadata) from its [body](Body), and in [functions](Functions-and-Methods). + +## Permission +A [permission value](Permissions). + +## Return Direction +A [return direction](Return-Direction) arrow (`->`). + +## Int +A negative integer number. This says nothing about the actual intended type of the value, just the representation of it found in the source code. + +## UInt +An positive integer number. This says nothing about the actual intended type of the value, just the representation of it found in the source code. + +## Float +A floating point number. + +## String +A string literal delimited by double quotes. It supports these escape codes: + +- `\a` +- `\b` +- `\f` +- `\n` +- `\r` +- `\t` +- `\v` +- `\'` +- `\"` +- `\\` + +Hexidecimal and unicode escape codes are also permitted: + +- `\xNN` +- `\uNNNN` +- `\UNNNNNNNN` + +... where N is a single digit from 0-F. + +String and rune literals support both `\'` and `\"`. + +## Rune +A unicode character literal delimited by single quotes. Only one character value is supported. All escape codes that work in strings work in rune literals. + +## Name +Names are composed of characters a-z, A-Z, and 0-9. However, they must start with an alphabetical character. They do not support underscores. + +## Colon +A colon symbol. It is used in type notation. + +## Dot +A dot symbol. It is used in [member selection](Identifiers) and [object initialization](Initialization). + +## Elipsis +Two consecutive dots. It is used in type notation to describe an array with a variable length, and will in the future be used in functions with variable input counts. + +## Comma +This token is not used for anything as of now. + +## LBracket and RBracket +Left and right square brackets (`[` and `]`). These are used to delimit [prases](Phrases). + +## LBrace and RBrace +Left and right curly braces (`{` and `}`). These are used for pointer dereferencing and array subscript notation, as well as in type notation to describe pointers and arrays. + +## Plus and Minus +Plus and minus symbols (`+` and `-`). These are used to add and subtract values. + +## Increment and Decrement +The plus and minus symbols respectively, but each repeated once (`++` and `--`). These are used to increment and decrement values. + +## Asterisk and Slash +Multiplication and division symbols (`*` and `/`). These are used to multiply and divide values. + +## At +An `@` symbol. This is used to define method recievers in [function/method sections](Functions-and-Methods). + +## Exclamation +An exclamation point. This is used as a logical not. In the future may be used to notate exception handling in functions. + +## Percent +A percent sign. This is used for modulo operations. + +## Tilde +A tilde (`~`) symbol. This is used as a binary not. + +## Equal To +An equals symbol (`=`). This is used to check equality, and *not* to set values. `set` phrases are used for that purpose. + +## Not Equal To +A not equals symbol (`!=`). This is used to check inequality. + +## Less Than and Greater Than +Less than and greater than symbols (`<` and `>`). These are used to compare numbers, and denote inputs and outputs in [function/method sections](Functions-and-Methods). + +## Less Than Equal To and Greater Than Equal To +Like the less than and greater than tokens, except each followed by an equals symbol (`<=` and `>=`). These are used to compare numbers. + +## Left Shift and Right Shift +The less than and greater than sumbols respectively, but each repeated once (`<<` and `>>`). These are used to perform left and right bit shift options on values. + +## Binary Or and Logical Or +A binary or is a single pipe symbol (`|`). A logical or is a double pipe symbol (`||`). + +## Binary And and Logical And +A binary and is a single amperseand symbol (`&&`). A logical or is a double amperseand symbol (`&&`). \ No newline at end of file