Blender and ASTDef: Language Definition ToolsAndrew Begel)
Blender is a lexer and parser generator that reads in lexical descriptions (written in a variant of Flex format) and grammars (written in a variant of Bison format) and combines them to produce lexer and parser tables. In addition, it also writes graph-based data structures in ASTDef format for the parser to refer to the grammar at runtime.
The first task of the ASTDef translator is to process all of the AST definition code. Since the ASTDef language is a derivative of C++, the specifications need to be parsed much like any other program. However, C++ is notoriously difficult to parse; thus, when we designed the syntax for ASTDef, we included some syntactic sugar that made parsing easier. Some modifications were to precede each method declaration with the method keyword, and each field declaration with the slot keyword. Additionally, method bodies are not parsed at all. Instead, lexical tricks are used to treat them as strings which are then stored within ASTDef's internal representation.
After processing AST definitions, ASTDef performs some simple validations such as checking that no method or field names clash (more rigorous error checking is left to the C++ compiler). It then carries out a number of transformations on the internal representation, and translates AST definitions to C++. ASTDef also generates all of the runtime support code.