thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-13 09:52:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
126,244 Commits 9,324 Branches 1,783 Tags 2.1 GiB
a98ffd4801
Commit Graph

126244 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Robert Schulze
66d4706338
Try to fix flakyness 2023-10-08 07:56:42 +00:00
Robert Schulze
f4bc58ea92
Merge remote-tracking branch 'rschu1ze/master' into more-mysql-compat 2023-10-08 07:49:37 +00:00
Azat Khuzhin
099665478d Fix incorrect merging of Nested for SELECT FINAL FROM SummingMergeTree 
The problem was the order of the columns, in case of SELECT FINAL it got
"counters_Map.count", "counters_Map.id"

But in case of OPTIMIZE FINAL it got "counters_Map.id",
"counters_Map.count" correctly.

Note, that this bugs exists there from the very recent versions, I've
checked 19.x and it was there.

P.S. there is a workaround for this problem, if you will use one of the
following patterns for key columns:
- *ID
- *Key
- *Type

That way it will be explicitly matched as key and everything will work.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 07:32:47 +02:00
Azat Khuzhin
2a27a8b0bf tests/test_throttling: check the maximum execution time as well 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 07:31:33 +02:00
Azat Khuzhin
c883da2b26 tests/test_throttling: adjust delays due to HTTPS usage 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 07:31:33 +02:00
Azat Khuzhin
9c8096455e tests: switch test_throttling to S3 over https to make it more production like 
One interesting thing about S3 C++ SDK is that it can read file multiple
times for calculating checksums and signature, but the last is not done
for the https protocol, though the checksum "could".

And indeed it does, since default checksum algorithm (MD5) does not
support streaming, and so it always calculated, regardless of the
protocol, however everything else (CRC*/SHA*) supports streamming and
actually will not be calculated for https at all!

This will be fixed in the follow up patch.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 07:31:33 +02:00
Azat Khuzhin
4179ff7111 Adjust the test_storage_delta::test_multiple_log_files after MinIO upgrade 
After upgrade MinIO stopped handling of objects with leading slash:

    (Pdb) p minio_client.fput_object(bucket_name='root', object_name='/test_multiple_log_files/_delta_log/00000000000000000001.json', file_path='/test_multiple_log_files/_delta_log/00000000000000000001.json').http_headers
    HTTPHeaderDict({'Accept-Ranges': 'bytes', 'Content-Length': '0', 'ETag': '"f1f3276c787b3f6724b9a8edb313c4fa"', 'Server': 'MinIO', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'Vary': 'Origin, Accept-Encoding', 'X-Amz-Id-2': '7987905dee74cdeb212432486a178e511309594cee7cb75f892cd53e35f09ea4', 'X-Amz-Request-Id': '178B9B0904A9640D', 'X-Content-Type-Options': 'nosniff', 'X-Xss-Protection': '1; mode=block', 'Date': 'Fri, 06 Oct 2023 19:22:27 GMT'})
    (Pdb) p list(minio_client.list_objects('root', '/test_multiple_log_files/_delta_log/'))
    []
    (Pdb) p minio_client.get_object(bucket_name='root', object_name='/test_multiple_log_files/_delta_log/00000000000000000001.json').read()
    b'{"commitInfo":{"timestamp":1696618985577,"operation":"WRITE","operationParameters":{"mode":"Overwrite","partitionBy":"[]"},"readVersion":0,"isolationLevel":"Serializable","isBlindAppend":false,"operationMetrics":{"numFiles":"1","numOutputRows":"100","numOutputBytes":"2459"},"engineInfo":"Apache-Spark/3.3.2 Delta-Lake/2.3.0","txnId":"2a69a4c9-f72e-4ac9-9374-bdff96707edf"}}\n{"add":{"path":"part-00000-9ff4728b-d5fe-49ac-a773-e2d6e01f4139-c000.parquet","partitionValues":{},"size":2459,"modificationTime":1696618985310,"dataChange":true,"stats":"{\\"numRecords\\":100,\\"minValues\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":0,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":\\"1\\"},\\"maxValues\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":99,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":\\"99\\"},\\"nullCount\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":0,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":0}}"}}\n{"remove":{"path":"part-00000-7c29f5d9-e14d-419a-a273-58d07775e1e9-c000.parquet","deletionTimestamp":1696618985576,"dataChange":true,"extendedFileMetadata":true,"partitionValues":{},"size":2459}}\n'
    (Pdb) p list(minio_client.list_objects(bucket_name='root', prefix='test_multiple_log_files/', recursive=True))
    [<minio.datatypes.Object object at 0x7ffff1ec2fe0>, <minio.datatypes.Object object at 0x7ffff1ec39d0>, <minio.datatypes.Object object at 0x7ffff1ec2590>, <minio.datatypes.Object object at 0x7ffff1ec3dc0>, <minio.datatypes.Object object at 0x7ffff1ec3a00>, <minio.datatypes.Object object at 0x7ffff1ec3fa0>, <minio.datatypes.Object object at 0x7ffff1ec3f40>, <minio.datatypes.Object object at 0x7ffff1ec2770>]

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 07:30:13 +02:00
Azat Khuzhin
ce9cdc408c Upgrade MinIO to support accepting non signed requests 
During fixing one issue with multiple reads of input files while sending
them to S3 this had been pops up.

The problem is in this place [1].

  [1]: 200caab82b/cmd/object-handlers.go (L1573-L1591)

And this had been added only in [2].

  [2]: https://github.com/minio/minio/pull/16484

But even this implementation is not complete, since it does not
correctly support STREAMING-UNSIGNED-PAYLOAD-TRAILER, but this will be
the first step, to see if there are still problems with newer MinIO.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-08 07:30:13 +02:00
Alexey Milovidov
b38d4b5b0f
Merge pull request #55119 from jmaicher/fix/54318/deadlock-ldap-role-update 
Fix deadlock in LDAP assigned role update
 2023-10-08 00:38:10 +02:00
robot-clickhouse-ci-1
48ce595e24
Merge pull request #55220 from den-crane/Docs/use_local_cache_for_remote_storage 
Doc. Update use_local_cache_for_remote_storage settings name
 2023-10-08 00:23:54 +02:00
Alexey Milovidov
f5b0fb27a4
Update DiskObjectStorageMetadata.cpp 2023-10-08 00:53:33 +03:00
Alexey Milovidov
25a7f86da7
Update hive.md 2023-10-08 00:51:19 +03:00
Alexey Milovidov
857ac2705d
Merge pull request #55226 from ClickHouse/server-provides-keeper 
Make clickhouse-server provides keeper
 2023-10-07 23:49:14 +02:00
Alexey Milovidov
186bd9c859
Merge pull request #55243 from den-crane/patch-62 
Doc. fix a mistake about refcount
 2023-10-07 23:43:16 +02:00
Alexey Milovidov
6b9696fd97
Update 02891_functions_over_sparse_columns.sql 2023-10-08 00:38:04 +03:00
Robert Schulze
8d58c4d389
Suppress some shellcheck errors 2023-10-07 20:19:47 +00:00
robot-clickhouse-ci-1
2cd23c293a
Merge pull request #55307 from ClickHouse/make_code_more_stupid 
Make `HandlingRuleHTTPHandlerFactory` more stupid, but less error prone
 2023-10-07 22:12:43 +02:00
Robert Schulze
0450da9bac
Naming fixes 2023-10-07 20:12:10 +00:00
Robert Schulze
312fe8e31d
Merge pull request #55298 from rschu1ze/use_mysql_types_in_show_columns 
Make `use_mysql_types_in_show_columns` independent from the connection
 2023-10-07 22:03:24 +02:00
Robert Schulze
727f831c0c
Update croaring 
Fixes: #55270
 2023-10-07 19:44:19 +00:00
alesapin
ec69b28386 Black 2023-10-07 21:18:54 +02:00
alesapin
77d96e67d1 Merge branch 'fix_redundant_check' of github.com:ClickHouse/ClickHouse into fix_redundant_check 2023-10-07 21:18:39 +02:00
alesapin
aa130b74e5 Fix offsets bug 2023-10-07 21:18:23 +02:00
Robert Schulze
73ff017fb5
Merge remote-tracking branch 'rschu1ze/master' into israelov/generate-random-int-array 2023-10-07 18:10:09 +00:00
Robert Schulze
3580c4d730
Fixups 2023-10-07 18:08:54 +00:00
robot-clickhouse
041a9c8180 Automatic style fix 2023-10-07 17:27:42 +00:00
Robert Schulze
4c4f81d2a8
Remove x bit from sql test file 2023-10-07 17:12:12 +00:00
alesapin
eec64ba1d9 Fix bug with inability to drop detached partition in replicated merge tree on top of S3 without zero copy 2023-10-07 19:09:47 +02:00
Robert Schulze
4b7ffa52c5
Refactorings 2023-10-07 16:45:52 +00:00
alesapin
4f7b81b6a5 Make HandlingRuleHTTPHandlerFactory more stupid, but less error prone 2023-10-07 18:12:58 +02:00
Kseniia Sumarokova
ab9e6f5f6a
Merge pull request #55282 from Priyansh121096/patch-1 
Fix typo in TableFunctionS3.cpp
 2023-10-07 18:05:52 +02:00
robot-ch-test-poll1
59a3b3356f
Merge pull request #55296 from rschu1ze/qc-better-logging 
Improve logging in query cache
 2023-10-07 17:59:19 +02:00
kssenii
1a839adec7 Add one more try-catch 2023-10-07 15:31:15 +02:00
Robert Schulze
cfa2e99b6e
Merge remote-tracking branch 'rschu1ze/master' into more-mysql-compat 2023-10-07 13:08:24 +00:00
Robert Schulze
2e8cac6e9c
Make use_mysql_types_in_show_columns independent from the connection 
Fixes: #55273
 2023-10-07 13:06:41 +00:00
Kseniia Sumarokova
f0747b7a6e
Update MaterializedPostgreSQLConsumer.cpp 2023-10-07 14:28:52 +02:00
Robert Schulze
a72ae92328
Make the logging in the query cache easier to understand 2023-10-07 11:33:11 +00:00
robot-ch-test-poll4
2406542931
Merge pull request #55266 from AleksaC/patch-1 
Fix query_log cache usage column name in docs
 2023-10-07 13:00:24 +02:00
alesapin
e55a615f06
Merge pull request #55253 from vitlibar/fix-async-insert-access-check-for-defaults 
Evaluate defaults during async insert safer
 2023-10-07 12:26:35 +02:00
kssenii
289020f454 Merge remote-tracking branch 'upstream/master' into fix-background-download-in-cache 2023-10-07 12:19:33 +02:00
Kseniia Sumarokova
97976bc978
Merge pull request #55260 from ClickHouse/kssenii-patch-6 
Fix data race in context
 2023-10-07 12:17:54 +02:00
Ilya Yatsishin
262373c9ef
Merge pull request #51435 from qoega/sources-with-subs 
Build artifact with sources with submodules in Darwin special build
 2023-10-07 11:07:27 +02:00
Daniel Byta
158e1ed703
Update client-libraries.md 2023-10-07 03:50:08 +03:00
Priyansh Agrawal
2562f66efd
Fix typo in TableFunctionS3.cpp 2023-10-07 00:55:08 +01:00
Azat Khuzhin
ce7f0e5ae7 Fix functions execution over sparse columns 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-06 16:16:50 +02:00
kssenii
9df0b8f67c Fix 2023-10-06 12:41:37 +02:00
Duc Canh Le
bea4a4eb85 Merge branch 'master' into clickhouse_fix_too_many_marks 
rebase and re-trigger CI
 2023-10-06 09:45:48 +00:00
Aleksa Cukovic
db0cf729f5
Fix query_log cache usage column name in docs 
It is not called `query_log_usage` but `query_cache_usage`
 2023-10-05 22:31:48 +02:00
Vitaly Baranov
cf7eea569b Fix quota consumption 2023-10-05 15:46:36 +02:00
Mikhail f. Shiryaev
351aeb0189
Beautify packages/build a bit 2023-10-05 15:43:47 +02:00