How to use Fuzzer
===

The fuzzer consists of auto-generated files:

    ClickHouseUnlexer.py
    ClickHouseUnparser.py

They are generated from grammar files (.g4) using Grammarinator:

    pip3 install grammarinator
    grammarinator-process ClickHouseLexer.g4 ClickHouseParser.g4 -o fuzzer/

Then you can generate test input for ClickHouse client:

    cd fuzzer
    grammarinator-generate \
      -r query_list                                               \ # top-level rule
      -o /tmp/sql_test_%d.sql                                     \ # template for output test names
      -n 10                                                       \ # number of tests
      -c 0.3                                                      \
      -d 20                                                       \ # depth of recursion
      -p ClickHouseUnparser.py -l ClickHouseUnlexer.py            \ # auto-generated unparser and unlexer
      --test-transformers SpaceTransformer.single_line_whitespace \ # transform function to insert whitespace

For more details see `grammarinator-generate --help`. As a test-transformer function also can be used `SpaceTransformer.multi_line_transformer` - both functions reside in `fuzzer/SpaceTransformer.py` file.


Parsing steps
===

1. Replace all operators with corresponding functions.
2. Replace all asterisks with columns - if it's inside function call, then expand it as multiple arguments. Warn about undeterministic invocations when functions have positional arguments.

Old vs. new parser
===

- `a as b [c]` - accessing aliased array expression is not possible.
- `a as b . 1` - accessing aliased tuple expression is not possible.
- `between a is not null and b` - `between` operator should have lower priority than `is null`.
- `*.1` - accessing asterisk tuple expression is not possible.