pass-test-tokenize-all #1

Merged
sashakoshka merged 20 commits from pass-test-tokenize-all into main 2022-08-11 09:07:36 +00:00

20 Commits

Author SHA1 Message Date
Sasha Koshka a0d5b3a1a2 Added basic escape sequences to string parser 2022-08-11 04:05:55 -05:00
Sasha Koshka 4d73fa4b83 Fixed equality issue with int 2022-08-11 03:58:45 -05:00
Sasha Koshka bef5b0328e Fixed PermissionFrom() 2022-08-11 03:52:16 -05:00
Sasha Koshka af6f170833 Normalized whitespace tokens 2022-08-11 03:47:42 -05:00
Sasha Koshka 0a31ea7bf8 Support for digraphs such as &&, <<, ++ 2022-08-11 02:54:20 -05:00
Sasha Koshka 758b85e735 Added naieve string and char literal parsing 2022-08-11 02:47:51 -05:00
Sasha Koshka 3768e3454f Added float support 2022-08-11 02:17:35 -05:00
Sasha Koshka 40ad569870 Collapsed all number tokenization functions into just one 2022-08-11 02:03:54 -05:00
Sasha Koshka 9e4684dbed Number tokenizing functions use one digit conversion functions 2022-08-11 01:57:04 -05:00
Sasha Koshka 8813928b68 Negative numbers are now supported 2022-08-11 01:10:34 -05:00
Sasha Koshka 0f603e3bf7 I am an idiot 2022-08-11 01:00:57 -05:00
Sasha Koshka 7a45224449 Reduced code du0plicaion with number lexing functions 2022-08-11 00:57:04 -05:00
Sasha Koshka 3d610df44e Added support for decimal and octal numbers 2022-08-11 00:21:04 -05:00
Sasha Koshka 8f38203684 Added support for binary literals 2022-08-11 00:17:22 -05:00
Sasha Koshka 77f3c590c6 Added lexing hexidecimal numbers 2022-08-11 00:14:41 -05:00
Sasha Koshka 040e14d27c Test now prints out list of tokens 2022-08-10 15:02:08 -04:00
Sasha Koshka 2220b95cd2 Name and permission tokens are now supported 2022-08-10 14:55:26 -04:00
Sasha Koshka 9c1baf8216 Lexer can now tokenize
minus
decrement
return direction
separator
2022-08-10 14:18:28 -04:00
Sasha Koshka d43a2540cf Fixed error formatting column indicator 2022-08-10 14:08:04 -04:00
Sasha Koshka 314c045e65 Lexer no longer hangs when encountering letters 2022-08-10 13:59:09 -04:00