Commit Graph

67 Commits

Author SHA1 Message Date
Val Doroshchuk
6cfbaf9f86 MaterializedMySQL: Introduce Character Set Introducers
If some string literals are used in DDL, they might have `Character Set Introducers`
https://dev.mysql.com/doc/refman/8.0/en/charset-introducer.html
f.e. _utf8mb4'1' which is not parsable by current ParserStringLiteral.

Since we use utf8 by default, suggesting to automatically convert the string literals to utf8
before executing the query and avoid any charset introducers there.
Conversion from utf8 to utf8 is not needed and skipped.

Also it might convert double quotes to single quotes if any
which might solve issues with COMMENT and empty string literals "" in DEFAULT expressions.

SELECT _latin1"abc"; -- might be also valid for MySQL
... DEFAULT "",
... COMMENT "abc"

Currently no DEFAULT expressions are supported, not parsable at all,
but using this MR it allows to parse the expressions together with double quotes as string literals.
2023-09-14 08:24:25 +02:00
Filipp Ozinov
ce87451b66
Merge branch 'ClickHouse:master' into bakwc-patch-2 2023-08-17 18:37:00 +04:00
Alexander Tokmakov
76702a03bb
Update materialized_with_ddl.py (#53494) 2023-08-17 13:08:27 +03:00
Filipp Ozinov
5003406ef7
Style 2023-08-17 13:52:21 +04:00
Filipp Ozinov
f402d80bce
Test for system.data_skipping_indices with MaterializedMySQL 2023-08-17 12:51:41 +04:00
Filipp Ozinov
e609a9f3d1
Test for system.data_skipping_indices with MaterializedMySQL 2023-08-17 12:50:44 +04:00
Val Doroshchuk
3d7bf0626c MaterializedMySQL: Add tests to alter named collections 2023-08-04 11:50:53 +02:00
Val Doroshchuk
7da85cc0de MaterializedMySQL: Add tests for unquoted utf8 column names in DML 2023-08-03 11:53:56 +02:00
alesapin
efba3a2113 Add more logging and touch test for materialize mysql 2023-07-31 14:32:09 +02:00
Val Doroshchuk
bd09ad6736 MaterializedMySQL: Fix typos in tests 2023-07-25 16:20:24 +02:00
Kseniia Sumarokova
e05e0ec557
Merge pull request #52318 from valbok/utf8-names
MaterializedMySQL: Support unquoted utf-8 strings in DDL
2023-07-24 21:49:33 +02:00
Alexander Tokmakov
c3e2077449
Merge branch 'master' into comment-db 2023-07-24 20:53:46 +03:00
Val Doroshchuk
efa638ef3c MaterializedMySQL: Support unquoted utf-8 strings in DDL
Since ClickHouse does not support unquoted utf-8 strings but MySQL does.

Instead of fixing Lexer to recognize utf-8 chars as TokenType::BareWord,
suggesting to quote all unrecognized tokens before applying any DDL.

Actual parsing and validating the syntax will be done by particular Parser.

If there is any TokenType::Error, the query is unable to be parsed anyway.
Quoting such tokens can provide the support of utf-8 names.

See `tryQuoteUnrecognizedTokens` and `QuoteUnrecognizedTokensTest`.

mysql> CREATE TABLE 道.渠(...

is converted to

CREATE TABLE `道`.`渠`(...

Also fixed the bug with missing * while doing SELECT in full sync because db or table name are back quoted when not needed.
2023-07-24 11:12:10 +02:00
robot-ch-test-poll4
1d6e85da91
Merge pull request #52084 from valbok/conv-charsets
MaterializedMySQL: Introduce charset conversion
2023-07-23 18:36:30 +02:00
Alexey Milovidov
7af9af1a3f
Merge pull request #52103 from azat/tests/test_materialized_mysql_database
tests: drop existing view in test_materialized_mysql_database
2023-07-22 03:41:52 +03:00
Val Doroshchuk
d16d444943 MaterializedMySQL: Add support of double quoted comments 2023-07-20 13:59:01 +02:00
Song Liyong
7837559dbf MaterializedMySQL: Support CREATE TABLE AS SELECT 2023-07-19 13:33:02 +02:00
Song Liyong
6ae5207819 MaterializedMySQL: Introduce charset conversion 2023-07-18 15:55:55 +02:00
Alexey Milovidov
0d48dca171
Update materialized_with_ddl.py 2023-07-15 20:36:16 +03:00
Azat Khuzhin
4f6d59ce7f tests: drop existing view in test_materialized_mysql_database
This should avoid failures of other tests in case of some previous test
failed

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-14 16:50:41 +02:00
Val Doroshchuk
24e2bae7bd MaterializedMySQL: Rename materialize_with_ddl.py -> materialized_with_ddl.py
Make it more consistent to current names.
Keep some tests with `ENGINE = MaterializeMySQL` to test BC.
2023-06-16 10:16:11 +02:00
Kruglov Pavel
953af94ee0
Merge pull request #50874 from valbok/materialized-mysql-named-collections-test
MaterializedMySQL: Add test_named_collections
2023-06-15 16:46:49 +02:00
Kseniia Sumarokova
b7fbc4dd8e
Merge pull request #50977 from valbok/empty-table-overrides
MaterializedMySQL: Keep parentheses for empty table overrides
2023-06-15 12:48:04 +02:00
Alexander Tokmakov
f73a3cf255
Merge pull request #50924 from valbok/missing-drops
MaterializedMySQL: Add missing DROP DATABASE for tests
2023-06-15 13:33:28 +03:00
Val Doroshchuk
e7c5991b39 MaterializedMySQL: Keep parenthesises for empty table overrides
Empty table overrides are formatted without any parenthesises,
but they are required by a parser,
and it is not possible to parse empty table overrides without it.

:) CREATE DATABASE db ... TABLE OVERRIDE t1()

CREATE DATABASE db
...
TABLE OVERRIDE `t1`

This query will be saved to metadata and ClickHouse will not be able to start up, since
table overrides require ().
2023-06-14 13:37:49 +02:00
Val Doroshchuk
8ea7560d89
MaterializedMySQL: Add additional test case to insert_with_modify_binlog_checksum (#50884) 2023-06-13 17:28:53 +03:00
Val Doroshchuk
1a4b7e8ebe MaterializedMySQL: Add missing DROP DATABASE for tests 2023-06-13 14:39:29 +02:00
Val Doroshchuk
879db5098a MaterializedMySQL: Add test_named_collections 2023-06-13 10:09:12 +02:00
Val Doroshchuk
c37b80593c
MaterializedMySQL: Add support of TRUNCATE db.table (#50624)
Additional to `TRUNCATE TABLE db.table`.

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2023-06-08 14:22:55 +03:00
Alexander Tokmakov
ba6c6255a3 hide too noisy log messages, fix some tests 2023-03-14 00:23:36 +01:00
zzsmdfj
a0f391745d to 15357_MaterializeMySQL_support_drop_mulit_table-fix style-black 2022-11-21 11:48:57 +08:00
zzsmdfj
e177927efb to 15357_MaterializeMySQL_support_drop_mulit_table-fix code-style 2022-11-19 12:05:27 +08:00
zzsmdfj
6a998917b0 to MaterializeMySQL_support_drop_mulit_table 2022-11-18 21:29:13 +08:00
Kruglov Pavel
4ecbcbf5e1
Merge branch 'master' into MaterializedMySQL_improvement_bug_fix 2022-11-11 12:54:02 +01:00
zzsmdfj
a9a208c3cd to MaterializedMySQL_improvement_bug_fix-fix code style 2022-11-11 11:46:22 +08:00
zzsmdfj
0e614aec12 to MaterializedMySQL_improvement_bug_fix-fix code sytle 2022-11-11 11:24:29 +08:00
zzsmdfj
d4e270b522 to MaterializedMySQL_improvement_bug_fix-fix codestyle 2022-11-11 10:44:16 +08:00
Stig Bakken
9020631826 Ignore SAVEPOINT queries in MaterializedMySQL 2022-11-10 23:27:21 +01:00
zzsmdfj
eeb9c57c59 to fix issue 42856 and MaterializedMySQL improvement 2022-11-03 20:21:02 +08:00
Alexey Milovidov
b42d26acfe Remove one line from XML, because we do not care 2022-08-28 02:44:02 +02:00
Roman Vasin
ac2a42cfb0 Fix test 00189_time_zones_long and integration test test_materialized_mysql_database 2022-07-26 10:44:58 +00:00
Alexander Tokmakov
2c5a88faba deprecate Ordinary database 2022-06-23 10:20:14 +02:00
Nikita Taranov
41ba0118b5
Bring back #36396 (#38110)
* Revert "Revert "More parallel execution for queries with `FINAL` (#36396)""

This reverts commit 5bfb15262c.

* fix tests

* fix review suggestions

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-06-22 15:05:07 +02:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
alesapin
1eefdc8df6 Fix more test 2022-03-15 17:31:33 +01:00
zzsmdfj
e7f567ddfd to #31469_MaterializedMysql_mysqlDate2CkDate32-adjust code-style 2022-03-08 18:25:21 +08:00
zzsmdfj
67b9f81104 to #31469_MaterializedMysql_mysqlDate2CkDate32 2022-03-08 18:17:22 +08:00
zzsmdfj
4d9bdfa58c to 32977_MaterializedMySQL_add_table_list_settings 2022-03-01 10:21:58 +08:00
zzsmdfj
2fa5289637 MaterializedMySQL add table_list_settings-update test 2022-02-10 17:29:51 +08:00
zzsmdfj
ac264e33d5 MaterializedMySQL_add_table_list_settings 2022-02-10 13:52:13 +08:00