Question 301132 in PostgreSQL Question posted in
The official documentation can be found here.

Is the ANTLR4 PostgreSQL Grammar broken?

I have downloaded Lexer and Parser .g4 files from github grammars-v4

In Visual Studio I started a new Class Library project with the packages Antlr4 4.6.6 and Antlr4.Runtime.Standard 4.13.1

When I build the class library I see the following Errors

mismatched input 'options {' expecting COLON while matching a lexer rule    PostgresSqlClasses  ...PostgreSQLLexer.g4  1421

mismatched input '=' expecting COLON while matching a rule  PostgresSqlClasses  ...PostgreSQLLexer.g4  1422    

mismatched input ';' expecting COLON while matching a rule  PostgresSqlClasses  ...PostgreSQLLexer.g4  1422

mismatched input 'options {' expecting COLON while matching a lexer rule    PostgresSqlClasses  ...PostgreSQLLexer.g4  1616

and

mismatched character '<EOF>' expecting ']'  PostgresSqlClasses  ...antlr4.codegenerator4.6.6buildAntlr4.CodeGenerator.targets   132

I can see that the grammar files have been updated very recently. Am I missing something or could those .g4 files be in error?

— Bart Kiers’ answer below would suggest it is not the Grammar files in error, but something to do with my Visual Studio 2022 or, perhaps, Java setup.

— Confirmed that there is nothing wrong with the grammars following Bart’s generous assistance below. Perhaps the alternative antlr c# target is the answer

Tags:

2

Answers


  1. 0

    Without any code generator/build tooling, I have no issues.

    I downloaded the following:

    • https://raw.githubusercontent.com/antlr/grammars-v4/master/sql/postgresql/PostgreSQLLexer.g4
    • https://raw.githubusercontent.com/antlr/grammars-v4/master/sql/postgresql/PostgreSQLParser.g4
    • https://raw.githubusercontent.com/antlr/grammars-v4/master/sql/postgresql/CSharp/LexerDispatchingErrorListener.cs
    • https://raw.githubusercontent.com/antlr/grammars-v4/master/sql/postgresql/CSharp/ParserDispatchingErrorListener.cs
    • https://raw.githubusercontent.com/antlr/grammars-v4/master/sql/postgresql/CSharp/PostgreSQLLexerBase.cs
    • https://raw.githubusercontent.com/antlr/grammars-v4/master/sql/postgresql/CSharp/PostgreSQLParserBase.cs

    and put the classes in the namesapce TestAntlr. I then did:

    java -jar antlr-4.13.1-complete.jar -Dlanguage=CSharp -package TestAntlr *.g4

    Then I ran:

    // <PackageReference Include="Antlr4.Runtime.Standard" Version="4.13.1" />
var lexer = new PostgreSQLLexer(CharStreams.fromString("SELECT something FROM somewhere"));
var parser = new PostgreSQLParser(new CommonTokenStream(lexer));

Console.WriteLine(parser.root().ToStringTree(recog: parser));

    without issues.

    Login or Signup to reply.
  2. 0

    I normally don’t "answer" SO questions because people down vote my answers. But, this time seems as good as any to correct misconceptions in using the grammars-v4 grammars, the Antlr4 tool and runtime for C#.

    The postgresql grammar is not broken. But, you must use a version of Antlr greater than or equal to 4.10. The postgresql grammar requires version 4.10 or newer because it is declared in the desc.xml file. The grammar uses the caseInsensitive option, which was added in Antlr 4.10. Other grammars use more recent features. It is also a good idea to peruse the desc.xml file for the <targets> to see if the grammar has been validated for the target. We see it has. You may see that this grammar has Go language files but the grammar has not been validated for Go–the target is missing in the <targets> element.

    You should not use Sam’s Antlr4cs fork, not only because it can’t really work with this grammar, but because it’s pretty old. You can tell if you are using this fork if your .csproj contains any <PackageReference> to any of these packages:

    Development of the Antlr4cs fork stop years ago. Instead of <PackageReference Include="Antlr4.Runtime"/>, use <PackageReference Include="antlr4.Runtime.Standard" />.Antlr4.Runtime.Standard is the package build from the official Antlr4 source runtime. You really should avoid having a .csproj that contains a reference to Antlr4.Runtime.Standard and any of the old Antlr4cs packages.

    You can write a driver, download Java, download the Antlr4 .jar, etc., etc., etc., or you can test using the procedure outlined in the Wiki using Trash.

    Since you already have dotnet installed, chances are you only need to install a few dotnet tools to generate a C# driver.

    dotnet tool install -g trgen
dotnet tool install -g trglob
dotnet tool install -g triconv
dotnet tool install -g trparse
dotnet tool install -g trxml

    Then, you can generate and test a parser in C#.

    git clone https://github.com/antlr/grammars-v4.git
cd grammars-v4/sql/postgresql
trgen -t CSharp
cd Generated-CSharp/
dotnet build Test.csproj
./bin/Debug/net7.0/Test.exe ../examples/aggregates.sql
./bin/Debug/net7.0/Test.exe ../examples/aggregates.sql -tree

    You can use the command line to build, compile, and run, or you can open the generated Test.csproj file in VSCode or VS2022, and just "F5" to build the parser from scratch, compile, link, and run the parser from the input window.

    The generated parser driver uses Antlr4BuildTasks (read the .csproj file generated), which is a drop-in-replacement for the Antlr4 Nuget package. It has all the logic to download a Java JRE, generate the parser source code from the Antlr tool, compile and link. You can build directly in VSCode, VS2022, or from the command line Cmd/Cygwin/Powershell/etc using dotnet or msbuild. Antlr4BuildTasks was forked from the Antlr4cs code, and updated. It contains many bug fixes and improvements on the old Antlr4cs package.

    There is currently no good extension for VS2022 for Antlr4 grammars. https://marketplace.visualstudio.com/search?term=antlr&target=VS&category=All%20categories&vsVersion=&sortBy=Relevance . Your best bet is to use VSCode and Mike’s excellent extension for Antlr. Or, you could use Rider with the Antlr4 extension. Normally, I use VS2022 and use Mads’ "Open in VSCode" if I need to have an extension that is Antlr4 aware.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search