Commit Graph

56 Commits

Author SHA1 Message Date
Sasha Koshka
dd8bbf8b13 Lexer now uses go's strconv number parsing functions
This allows for better accuracy when parsing floats, at the cost of only
supporting decimal floating point literals. The lexer now passes all tests.
2022-08-18 00:31:27 -04:00
Sasha Koshka
3407aa7c59 Fixed lexing digraph tokens
Lexer gave wrong token locations and would skip an extra rune when digraph was
not found.
2022-08-16 20:29:00 -04:00
Sasha Koshka
9e01eef45b Added elipsis token 2022-08-16 20:24:27 -04:00
Sasha Koshka
8b28fe5a4c Added a comma token 2022-08-15 14:50:09 -04:00
Sasha Koshka
608162fa92 Fixed bug in number lexing function that prevented reading zero 2022-08-15 14:42:32 -04:00
Sasha Koshka
d27c0ff07c Added single digit zero and eight to lexer number test 2022-08-15 14:32:59 -04:00
b3071d4ac9 Token kind values can now be described 2022-08-12 16:30:32 -05:00
856d5763d3 Lexer tokens are now created by the lexer
This is so positional information can be accurately embedded into them.
2022-08-12 14:34:07 -05:00
accf528869 Locations and tokens are now capable of creating errors on their own 2022-08-12 13:51:38 -05:00
7914f0df45 Location now stores width instead of Error 2022-08-12 13:43:09 -05:00
f4f19a809a Lexer now eats :arf symbol at file beginning 2022-08-12 10:38:23 -05:00
c09c9860b8 Parser tests are now arf files 2022-08-12 10:26:16 -05:00
81b47f7734 Replaced all occurences of github.com with git.tebibyte.media 2022-08-12 10:21:36 -05:00
31a2d84483 Removed erroneous indentation from indent test file
Need to find a more controlled and accurate way to test erroneous indentation.
Possibly by analyzing the returned error object.
2022-08-11 18:34:02 -05:00
ee9fa3a766 Indentation tokens now carry their indentation level
They are no longer stored in sequence
2022-08-11 12:53:20 -05:00
471eb863f4 Created indent test 2022-08-11 12:47:50 -05:00
71157122d6 Added support for hex, octal, and unicode escape sequences 2022-08-11 12:39:49 -05:00
f7a823687e Added new test for string and rune literals (which it passes) 2022-08-11 12:12:41 -05:00
4f37715a8c Hexidecimal numbers are now properly read 2022-08-11 04:25:56 -05:00
7a0ed247e2 Added number lexing unit test 2022-08-11 04:23:06 -05:00
6acc11d046 Removed debug rune printing th 2022-08-11 04:19:39 -05:00
a0d5b3a1a2 Added basic escape sequences to string parser 2022-08-11 04:05:55 -05:00
4d73fa4b83 Fixed equality issue with int 2022-08-11 03:58:45 -05:00
bef5b0328e Fixed PermissionFrom() 2022-08-11 03:52:16 -05:00
af6f170833 Normalized whitespace tokens 2022-08-11 03:47:42 -05:00
0a31ea7bf8 Support for digraphs such as &&, <<, ++ 2022-08-11 02:54:20 -05:00
758b85e735 Added naieve string and char literal parsing 2022-08-11 02:47:51 -05:00
3768e3454f Added float support 2022-08-11 02:17:35 -05:00
40ad569870 Collapsed all number tokenization functions into just one 2022-08-11 02:03:54 -05:00
9e4684dbed Number tokenizing functions use one digit conversion functions 2022-08-11 01:57:04 -05:00
8813928b68 Negative numbers are now supported 2022-08-11 01:10:34 -05:00
0f603e3bf7 I am an idiot 2022-08-11 01:00:57 -05:00
7a45224449 Reduced code du0plicaion with number lexing functions 2022-08-11 00:57:04 -05:00
3d610df44e Added support for decimal and octal numbers 2022-08-11 00:21:04 -05:00
8f38203684 Added support for binary literals 2022-08-11 00:17:22 -05:00
77f3c590c6 Added lexing hexidecimal numbers 2022-08-11 00:14:41 -05:00
040e14d27c Test now prints out list of tokens 2022-08-10 15:02:08 -04:00
2220b95cd2 Name and permission tokens are now supported 2022-08-10 14:55:26 -04:00
9c1baf8216 Lexer can now tokenize
minus
decrement
return direction
separator
2022-08-10 14:18:28 -04:00
314c045e65 Lexer no longer hangs when encountering letters 2022-08-10 13:59:09 -04:00
9f8b48161a Lexer unit test now prints out resulting error 2022-08-10 13:03:48 -04:00
cf04dfd600 Lexer unit test now works properly 2022-08-10 12:58:47 -04:00
2e965d6f68 Added Permission type 2022-08-10 12:52:20 -04:00
488220f3a7 Added all tokens to unit test 2022-08-10 11:43:21 -04:00
3f16d34aa1 Added unit test for lexer 2022-08-10 11:28:29 -04:00
245798f33d Print warning when a tab is discovered where it shouldn't be 2022-08-10 01:22:53 -04:00
0d2d10fe04 All basic symbol tokens are now parsed 2022-08-10 01:03:59 -04:00
a0ce033c27 Added a ton more token kinds 2022-08-10 00:48:18 -04:00
64de897de5 Added lexing various symbol tokens 2022-08-09 22:18:12 -04:00
861400b4a1 LexingOperation now stores the list of tokens 2022-08-09 22:12:14 -04:00