Update 'Tokens'

Sasha Koshka 2022-08-30 06:00:01 +00:00
parent 69be3e7ff3
commit a1e6cf9fa9

117
Tokens.md Normal file

@ -0,0 +1,117 @@
Arf's syntax is composed of basic elements called tokens. They can be arranged to build higher level syntactical features.
Important: some characters in this documented may be affected by font ligatures, and do not reflect what the language actually accepts as valid. If it is not immediately apparent what a given character is, paste it into a text field that is not ligaturized.
# List of Tokens
## Newline
A simple line break.
## Indent
A series of tabs characters. It's value should describe its indentation level.
## Separator
A series of three dashes (`---`). It is used so separate a file's [metadata](Metadata) from its [body](Body), and in [functions](Functions-and-Methods).
## Permission
A [permission value](Permissions).
## Return Direction
A [return direction](Return-Direction) arrow (`->`).
## Int
A negative integer number. This says nothing about the actual intended type of the value, just the representation of it found in the source code.
## UInt
An positive integer number. This says nothing about the actual intended type of the value, just the representation of it found in the source code.
## Float
A floating point number.
## String
A string literal delimited by double quotes. It supports these escape codes:
- `\a`
- `\b`
- `\f`
- `\n`
- `\r`
- `\t`
- `\v`
- `\'`
- `\"`
- `\\`
Hexidecimal and unicode escape codes are also permitted:
- `\xNN`
- `\uNNNN`
- `\UNNNNNNNN`
... where N is a single digit from 0-F.
String and rune literals support both `\'` and `\"`.
## Rune
A unicode character literal delimited by single quotes. Only one character value is supported. All escape codes that work in strings work in rune literals.
## Name
Names are composed of characters a-z, A-Z, and 0-9. However, they must start with an alphabetical character. They do not support underscores.
## Colon
A colon symbol. It is used in type notation.
## Dot
A dot symbol. It is used in [member selection](Identifiers) and [object initialization](Initialization).
## Elipsis
Two consecutive dots. It is used in type notation to describe an array with a variable length, and will in the future be used in functions with variable input counts.
## Comma
This token is not used for anything as of now.
## LBracket and RBracket
Left and right square brackets (`[` and `]`). These are used to delimit [prases](Phrases).
## LBrace and RBrace
Left and right curly braces (`{` and `}`). These are used for pointer dereferencing and array subscript notation, as well as in type notation to describe pointers and arrays.
## Plus and Minus
Plus and minus symbols (`+` and `-`). These are used to add and subtract values.
## Increment and Decrement
The plus and minus symbols respectively, but each repeated once (`++` and `--`). These are used to increment and decrement values.
## Asterisk and Slash
Multiplication and division symbols (`*` and `/`). These are used to multiply and divide values.
## At
An `@` symbol. This is used to define method recievers in [function/method sections](Functions-and-Methods).
## Exclamation
An exclamation point. This is used as a logical not. In the future may be used to notate exception handling in functions.
## Percent
A percent sign. This is used for modulo operations.
## Tilde
A tilde (`~`) symbol. This is used as a binary not.
## Equal To
An equals symbol (`=`). This is used to check equality, and *not* to set values. `set` phrases are used for that purpose.
## Not Equal To
A not equals symbol (`!=`). This is used to check inequality.
## Less Than and Greater Than
Less than and greater than symbols (`<` and `>`). These are used to compare numbers, and denote inputs and outputs in [function/method sections](Functions-and-Methods).
## Less Than Equal To and Greater Than Equal To
Like the less than and greater than tokens, except each followed by an equals symbol (`<=` and `>=`). These are used to compare numbers.
## Left Shift and Right Shift
The less than and greater than sumbols respectively, but each repeated once (`<<` and `>>`). These are used to perform left and right bit shift options on values.
## Binary Or and Logical Or
A binary or is a single pipe symbol (`|`). A logical or is a double pipe symbol (`||`).
## Binary And and Logical And
A binary and is a single amperseand symbol (`&&`). A logical or is a double amperseand symbol (`&&`).