v 0.3

Instructions

Fully operational but main method & test case are missing.

Manual testing required.

Global process

1. Write your own syntax grammar corresponding to the Grammar data type
2. You call makeGrammarParser giving your grammar as input and you will obtain a parse function
3. Apply this parse function to a stream and you will obtain the corresponding parse tree
   

Comments

This is a complete change from version 0.2. It was rewritten from scratch and is from ground up completely different. They have absolutely nothing in common. For more information on why it happened, read this. 

Grammar definition excerpt

grammar = Grammar {
       mainRule = "statement",

       skipped = [
               " ",
               "\t",
               "#.*"],

       delimiters = [
               ".",
               ",",
               ";",
               "(",
               ")"],

       rules = [
               ("statement",      Syntax statement),

               ("symbol-def",     Syntax symbolDef),
               ("function-def",   Syntax functionDef),
               ("lambda-def",     Syntax lambdaDef),

               ("parameters",     Syntax parameters),
               ("arguments",      Syntax arguments),

               ("expression",     Syntax expression),
...

functionCall
       = Sequence [
               Rule "symbol",
               DirectToken "(",
               Rule "arguments",
               DirectToken ")"]


parameters
       = Maybe ( Sequence [
               Rule "symbol",
               ZeroOrMore ( Sequence [
                       DirectToken ",",
                       Rule "symbol"
                       ])
               ])
...

Remarks

• declarative syntax definition, very easy to change it
• no more tokenizer, scannerless
• spaces are important, they delimit the tokens
• things like "@%*!" are valid symbols (try it!)
• operator priority not taken into account because we expect that operators can be defined in the code, so it must be determined in a later step