Formal language specification #6

Open
opened 2022-03-17 04:36:42 +00:00 by Ghost · 1 comment

I am interested in specifying a formal language, grammar, and parser for Sprite's syntax, mostly for the sake of learning about parsing. I'm hoping the language is currently context-free and can be described using a context-free grammar (e.g. in EBNF), which could be used to generate a parser as well as railroad diagrams for documentation (e.g. using gll for parsing and railroad for diagrams).

Due to features like macros and typedefs, many commonly used programming languages are not fully context-free, limiting the use of context-free parsing to the initial stage or eliminating it entirely. That said, there are common parsing techniques capable of dealing with context sensitivity, such as parsing expression grammars.

References

I am interested in specifying a formal language, grammar, and parser for Sprite's syntax, mostly for the sake of learning about parsing. I'm hoping the language is currently context-free and can be described using a context-free grammar (e.g. in EBNF), which could be used to generate a parser as well as railroad diagrams for documentation (e.g. using `gll` for parsing and `railroad` for diagrams). Due to features like macros and typedefs, many commonly used programming languages are not fully context-free, limiting the use of context-free parsing to the initial stage or eliminating it entirely. That said, there are common parsing techniques capable of dealing with context sensitivity, such as parsing expression grammars. ### References - [StackOverflow post on how most real-world languages are context-sensitive](https://stackoverflow.com/a/17686190) - [Wikipedia page on syntax, which breaks down syntactic parsing into levels and discusses languages with complex grammars](https://en.wikipedia.org/wiki/Syntax_(programming_languages)) - [Wikipedia's list of parser generators, useful for surveying common parser classes and approaches](https://en.wikipedia.org/wiki/Comparison_of_parser_generators) - [an article comparing recursive descent, generalized parsing, and LL and LR parsers](https://tratt.net/laurie/essays/entries/which_parsing_approach.html) - [architecture documentation for `rust-analyzer`](https://github.com/rust-analyzer/rust-analyzer/blob/master/docs/dev/architecture.md) - [Rust grammar working group's resources page](https://github.com/rust-lang/wg-grammar/blob/master/misc/resources.md) - [the research paper that introduces GLL parsing](https://dotat.at/tmp/gll.pdf) - [`gll`, the Rust grammar working group's implementation of GLL parsing](https://github.com/rust-lang/gll) - [`gll-pg`, an unrelated GLL parser generator in Rust that uses Logos as a lexer](https://github.com/jiegec/gll-pg) - [`railroad`, a package library for making railroad diagrams (a visual representation of context-free grammars)](https://github.com/lukaslueg/railroad) - [Python language reference](https://docs.python.org/3/reference/index.html) - [Rust language reference](https://doc.rust-lang.org/stable/reference)
Owner

I'm currently re-implementing Sprite parsing using Pest, which consumes a parsing expression grammar (PEG) file and autogenerates parsing code.

Here's what the grammar looks like so far: b47137d27d/src/sprite.pest

No, PEGs are generally not context-free, and I'm not educated enough on CFGs to determine if Spritelang as it currently exists could be represented in a CFG, but right now this is working well as a parsing framework for the current examples.

I'm currently re-implementing Sprite parsing using [Pest](https://pest.rs), which consumes a parsing expression grammar (PEG) file and autogenerates parsing code. Here's what the grammar looks like so far: https://git.tebibyte.media/mars/sprite-rs/src/commit/b47137d27d6ecb910ae3c2eccbd00a0a6d53d8de/src/sprite.pest No, PEGs are generally not context-free, and I'm not educated enough on CFGs to determine if Spritelang as it currently exists could be represented in a CFG, but right now this is working well as a parsing framework for the current examples.
Sign in to join this conversation.
No Label
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: mars/sprite-rs#6
No description provided.