Custom Expression Item Parsers

Custom expression parsers allow to plugin into parsing process and provide special parsing of some portion of the parsed expression.

The expression parser (i.e., UniversalExpressionParser.IExpressionParser) iteratively parses keywords (see the section above on keywords), before parsing any other symbols.

Then the expression parser loops through all the custom expression parsers of type UniversalExpressionParser.ExpressionItems.Custom.ICustomExpressionItemParser in property CustomExpressionItemParsers in interface UniversalExpressionParser.IExpressionLanguageProvider passed to the parser, and for each custom expression parser executes the method

ICustomExpressionItem ICustomExpressionItemParser.TryParseCustomExpressionItem(IParseExpressionItemContext context, IReadOnlyList<IExpressionItemBase> parsedPrefixExpressionItems, IReadOnlyList<IKeywordExpressionItem> keywordExpressionItems).

If method call TryParseCustomExpressionItem(…) returns non-null value of type UniversalExpressionParser.ExpressionItems.Custom.ICustomExpressionItem, the parser uses the parsed custom expression item.

Otherwise, if TryParseCustomExpressionItem(…) returns null, the parser tries to parse a non custom expression item at current position (i.e., operators, a literal, function, code block, etc.).

Interface ICustomExpressionItem has a property UniversalExpressionParser.ExpressionItems.Custom.CustomExpressionItemCategory which is used by the parser to determine if the parsed custom expression item should be used as

  • a prefix for subsequently parsed regular expression (i.e., literal, function, braces, etc.).

  • should be treated as regular expression (which can be part of operators, function parameter, etc.).

  • or should be used as a postfix for the previously parsed expression item.

In the example below the parser parses “::pragma x” to a regular custom expression item of type UniversalExpressionParser.DemoExpressionLanguageProviders.CustomExpressions.PragmaCustomExpressionItem (i.e., the value of CustomExpressionItemCategory property in UniversalExpressionParser.ExpressionItems.Custom.ICustomExpressionItem is equal to UniversalExpressionParser.ExpressionItems.Custom.CustomExpressionItemCategory.Regular). As a result, the expression “::pragma x+y;” below is logically similar to “(::pragma x)+y;”

In a similar manner the expression “::types[T1, T2]” is parsed to a prefix custom expression item of type UniversalExpressionParser.DemoExpressionLanguageProviders.CustomExpressions.GenericTypesCustomExpressionItem by custom expression item parser UniversalExpressionParser.DemoExpressionLanguageProviders.CustomExpressions.GenericTypesExpressionItemParser.

The custom expression item parsed from “::types[T1, T2]” is added as a prefix to an expression item parsed from F1(x:T1, y:T2).

Also, the expression “where T1:int where T2:double whereend” is parsed to postfix custom expression item of type UniversalExpressionParser.DemoExpressionLanguageProviders.CustomExpressions.WhereCustomExpressionItem by custom expression parser UniversalExpressionParser.DemoExpressionLanguageProviders.CustomExpressions.WhereCustomExpressionItemParserForTests.

The parser adds the parsed custom expression as a postfix to the preceding regular expression item parsed from text “F1(x:T1, y:T2)”.

In this example, the code block after “whereend” (the expression “{…}”) is parsed as a postfix expression item of type UniversalExpressionParser.ExpressionItems.ICodeBlockExpressionItem and is added as a postfix to regular expression item parsed from “F1(x:T1, y:T2)” as well, since the parser adds all the prefixes/postfixes to regular expression item it finds after/before the prefixes/postfixes.

1::pragma x+y;
2::types[T1,T2] F1(x:T1, y:T2) where T1:int where T2:double whereend
3{
4   // This code block will be added as a postfix to expression item parsed from "F1(x:T1, y:T2)".
5}

Click here to see the visualized instance of UniversalExpressionParser.IParseExpressionResult

  • This is another example demonstrating that the parsed expression can have multiple prefix and postfix custom expressions items applied to the same regular expression item parsed from “F1(x:T1, y:T2, z:T3)”.

 1// The expression below ("::metadata {...}") is parsed to a prefix custom expression item and added to list of prefixes of regular
 2// expression item parsed from F1(x:T1, y:T2, z:T3)
 3::metadata {description: "F1 demoes regular function expression item to which multiple prefix and postfix custom expression items are added."}
 4
 5// ::types[T1,T2] is also parsed to a prefix custom expression item and added to list of prefixes of regular
 6// expression item parsed from F1(x:T1, y:T2, z:T3)
 7::types[T1,T2]
 8F1(x:T1, y:T2, z:T3)
 9
10// The postfix custom expression item parsed from "where T1:int where T2:double whereend" is added to list of postfixes of regular expression
11// parsed from "F1(x:T1, y:T2, z:T3)".
12where T1:int,class where T2:double whereend
13
14// The postfix custom expression item parsed from "where T3 : T1 whereend " is also added to list of postfixes of regular expression
15// parsed from "F1(x:T1, y:T2, z:T3)".
16where T3 : T1 whereend
17{
18   // This code block will be added as a postfix to expression item parsed from "F1(x:T1, y:T2, z:T3)".
19}

Click here to see the visualized instance of UniversalExpressionParser.IParseExpressionResult

Implementing Custom Expression Parsers

For examples of custom expression item parsers look at some examples in demo project UniversalExpressionParser.DemoExpressionLanguageProviders.

The following demo implementations of UniversalExpressionParser.ExpressionItems.Custom.ICustomExpressionItemParserByKeywordId might be useful when implementing custom expression parses:

  • UniversalExpressionParser.DemoExpressionLanguageProviders.CustomExpressions.WhereCustomExpressionItemParserBase

  • UniversalExpressionParser.DemoExpressionLanguageProviders.CustomExpressions.PragmaCustomExpressionItemParser

  • UniversalExpressionParser.DemoExpressionLanguageProviders.CustomExpressions.MetadataCustomExpressionItemParser

Also, these custom expression parser implementations demonstrate how to use the helper class UniversalExpressionParser.IParseExpressionItemContext that is passed as a parameter to method DoParseCustomExpressionItem(IParseExpressionItemContext context,…) in UniversalExpressionParser.ExpressionItems.Custom.CustomExpressionItemParserByKeywordId to parse the text at current position, as well as how to report errors, if any.

  • To add a new custom expression parser, one needs to implement an interface UniversalExpressionParser.ExpressionItems.Custom.ICustomExpressionItemParser and make sure the property CustomExpressionItemParsers in interface UniversalExpressionParser.IExpressionLanguageProvider includes an instance of the implemented parser class.

  • In most cases the default implementation UniversalExpressionParser.ExpressionItems.Custom.AggregateCustomExpressionItemParser of UniversalExpressionParser.ExpressionItems.Custom.ICustomExpressionItemParser can be used to initialize the list of all custom expression parers that will be used by Universal Expression Parser.

UniversalExpressionParser.ExpressionItems.Custom.AggregateCustomExpressionItemParser has a dependency on IEnumerable<ICustomExpressionItemParserByKeywordId> (injected into constructor).

  • Using a single instance of AggregateCustomExpressionItemParser in property CustomExpressionItemParsers in interface UniversalExpressionParser.IExpressionLanguageProvider instead of multiple custom expression parsers in this property improves the performance.

AggregateCustomExpressionItemParser keeps internally a mapping from keyword Id to all the instances of UniversalExpressionParser.ExpressionItems.Custom.ICustomExpressionItemParserByKeywordId injected in constructor. When the parser executes the method TryParseCustomExpressionItem(…,IReadOnlyList<IKeywordExpressionItem> parsedKeywordExpressionItems,…) in interface UniversalExpressionParser.ExpressionItems.Custom, the custom expression item parser of type AggregateCustomExpressionItemParser evaluates the last keyword in list in parameter parsedKeywordExpressionItems to retrieve all the parsers mapped to this keyword Id, to try to parse a custom expression item using only those custom expression item parsers.

  • Below is some of the code from classes AggregateCustomExpressionItemParser and ICustomExpressionItemParserByKeywordId.

 1namespace UniversalExpressionParser.ExpressionItems.Custom;
 2
 3public class AggregateCustomExpressionItemParser : ICustomExpressionItemParser
 4{
 5    public AggregateCustomExpressionItemParser(
 6        IEnumerable<ICustomExpressionItemParserByKeywordId> customExpressionItemParsers)
 7    {
 8        ...
 9    }
10
11    public ICustomExpressionItem TryParseCustomExpressionItem(IParseExpressionItemContext context,
12            IReadOnlyList<IExpressionItemBase> parsedPrefixExpressionItems,
13            IReadOnlyList<IKeywordExpressionItem> parsedKeywordExpressionItems)
14    {
15        ...
16    }
17}
18
19public interface ICustomExpressionItemParserByKeywordId
20{
21    long ParsedKeywordId { get; }
22
23    ICustomExpressionItem TryParseCustomExpressionItem(IParseExpressionItemContext context,
24            IReadOnlyList<IExpressionItemBase> parsedPrefixExpressionItems,
25            IReadOnlyList<IKeywordExpressionItem> parsedKeywordExpressionItemsWithoutLastKeyword,
26            IKeywordExpressionItem lastKeywordExpressionItem);
27}

  • Here is the code from demo custom expression item parser PragmaCustomExpressionItemParser

 1using System.Collections.Generic;
 2using UniversalExpressionParser.ExpressionItems;
 3using UniversalExpressionParser.ExpressionItems.Custom;
 4
 5namespace UniversalExpressionParser.DemoExpressionLanguageProviders.CustomExpressions
 6{
 7    /// <summary>
 8    ///  Example: ::pragma x
 9    /// </summary>
10    public class PragmaCustomExpressionItemParser : CustomExpressionItemParserByKeywordId
11    {
12        public PragmaCustomExpressionItemParser() : base(KeywordIds.Pragma)
13        {
14        }
15
16        /// <inheritdoc />
17        protected override ICustomExpressionItem DoParseCustomExpressionItem(IParseExpressionItemContext context, IReadOnlyList<IExpressionItemBase> parsedPrefixExpressionItems,
18                                                                           IReadOnlyList<IKeywordExpressionItem> parsedKeywordExpressionItemsWithoutLastKeyword,
19                                                                           IKeywordExpressionItem pragmaKeywordExpressionItem)
20        {
21            var pragmaKeywordInfo = pragmaKeywordExpressionItem.LanguageKeywordInfo;
22
23            var textSymbolsParser = context.TextSymbolsParser;
24
25            if (!context.SkipSpacesAndComments() || !context.TryParseSymbol(out var literalExpressionItem))
26            {
27                if (!context.ParseErrorData.HasCriticalErrors)
28                {
29                    // Example: print("Is in debug mode=" + ::pragma IsDebugMode)
30                    context.AddParseErrorItem(new ParseErrorItem(textSymbolsParser.PositionInText,
31                        () => $"Pragma keyword '{pragmaKeywordInfo.Keyword}' should be followed with pragma symbol. Example: println(\"Is in debug mode = \" + {pragmaKeywordInfo.Keyword} IsDebug);",
32                        CustomExpressionParseErrorCodes.PragmaKeywordShouldBeFollowedByValidSymbol));
33                }
34
35                return null;
36            }
37
38            return new PragmaCustomExpressionItem(parsedPrefixExpressionItems, parsedKeywordExpressionItemsWithoutLastKeyword,
39                pragmaKeywordExpressionItem,
40                new NameExpressionItem(literalExpressionItem, textSymbolsParser.PositionInText - literalExpressionItem.Length));
41        }
42    }
43}

Click here to see the definition of interface UniversalExpressionParser.IParseExpressionItemContext