Commit Graph

125805 Commits

Author SHA1 Message Date
Alexey Milovidov
d6aa3cdb4c
Merge pull request #55357 from ClickHouse/revert-54477-view-sampling
Revert "Support SAMPLE BY for VIEW"
2023-10-09 02:41:55 +02:00
Alexey Milovidov
831969abf9
Merge pull request #55354 from ClickHouse/fix-bad-test-5
Fix bad test `01605_dictinct_two_level`
2023-10-09 01:21:43 +02:00
Alexey Milovidov
75a4d55afd
Merge pull request #55328 from ClickHouse/better-docs-55326
Better docs for #55326
2023-10-09 01:10:53 +02:00
Alexey Milovidov
2f8ca43e33
Merge pull request #55359 from ClickHouse/alexey-milovidov-patch-7
Update CHANGELOG.md
2023-10-09 01:05:38 +02:00
Alexey Milovidov
b3c72baede
Update CHANGELOG.md 2023-10-09 02:05:21 +03:00
Alexey Milovidov
eac4e3e915 Add a test 2023-10-08 23:17:40 +02:00
Alexey Milovidov
e1dc80b5ab
Revert "Support SAMPLE BY for VIEW" 2023-10-09 00:08:46 +03:00
Alexey Milovidov
61caa9a660
Merge pull request #55245 from azat/bump-minio
Upgrade MinIO to support accepting non signed requests
2023-10-08 22:46:01 +02:00
Robert Schulze
0e1aed2a38
Merge pull request #55327 from rschu1ze/to_start_of_interval_hours
Refactorings and better documentation for `toStartOfInterval()`
2023-10-08 22:33:12 +02:00
Alexey Milovidov
12714d54b2 Fix bad test 01605_dictinct_two_level 2023-10-08 22:06:30 +02:00
Alexey Milovidov
0b939fae5d Fix bad test 2023-10-08 22:01:15 +02:00
Alexey Milovidov
fa54b21424
Merge pull request #55103 from ClickHouse/fix_shell_command_abort
Proper cleanup in case of exception in ctor of ShellCommandSource
2023-10-08 21:36:28 +02:00
Alexey Milovidov
8ae8371260
Merge pull request #55336 from ClickHouse/revert-51946
Review #51946 and partially revert it
2023-10-08 21:27:02 +02:00
Robert Schulze
d4e7fa2542
Merge pull request #55123 from garcher22/untuple
Use the tuple element names as column names in `untuple()`
2023-10-08 20:14:29 +02:00
Robert Schulze
0f7ca6149b
(Hopefully) fix clang-tidy 2023-10-08 16:30:02 +00:00
Robert Schulze
32a77ca1eb
Merge pull request #54391 from itayisraelov/israelov/generate-random-int-array
Add function `arrayRandomSample()`
2023-10-08 18:28:58 +02:00
Alexey Milovidov
972f3678fa Review #51946 and partially revert it 2023-10-08 18:00:59 +02:00
Alexey Milovidov
9ac7cfc026
Merge pull request #55339 from ClickHouse/tylerhannan-patch-1
Update README.md
2023-10-08 17:43:14 +02:00
Tyler Hannan
76345a5f60
Update README.md
Updating meetups
2023-10-08 17:42:15 +02:00
Sema Checherinda
bfabeaf474
Merge pull request #55218 from CheSema/metadata-version-is-fixed
remove the blocker to grow the metadata file version
2023-10-08 17:35:34 +02:00
Robert Schulze
38c64b3cd1
Fix tests 2023-10-08 14:26:31 +00:00
Alexey Milovidov
ba12695525
Merge pull request #55276 from azat/SummingMergeTree-Nested-fix
Fix incorrect merging of Nested for SELECT FINAL FROM SummingMergeTree
2023-10-08 16:08:25 +02:00
Robert Schulze
558b2ff7b9
Fix spelling 2023-10-08 13:50:33 +00:00
Alexey Milovidov
caf3c85b38
Merge pull request #55275 from azat/sparse-columns-function-fix
Fix functions execution over sparse columns
2023-10-08 15:27:46 +02:00
Alexey Milovidov
a7716a56ac
Update IFunction.cpp 2023-10-08 16:27:33 +03:00
Alexey Milovidov
781da580ac
Merge pull request #55283 from bytadaniel/patch-2
Update client-libraries.md
2023-10-08 15:25:51 +02:00
Alexey Milovidov
c7ae1afb8f Review #51946 and partially revert it 2023-10-08 15:17:41 +02:00
Alexey Milovidov
6e502a899d
Merge pull request #55311 from rschu1ze/croaring-tsan
Fix tsan issue in croaring
2023-10-08 13:31:06 +02:00
Alexey Milovidov
75e8e190f6 Better docs for #55326 2023-10-08 13:28:51 +02:00
Robert Schulze
3f33c32684
Fix style 2023-10-08 10:46:28 +00:00
Robert Schulze
70eb2ab4fc
Try to fix flakyness, pt. II 2023-10-08 10:43:46 +00:00
alesapin
55fc3fe8ab
Merge pull request #55247 from azat/tests/test_throttling-https
tests: switch test_throttling to S3 over https to make it more production like
2023-10-08 12:39:54 +02:00
Robert Schulze
8c35ccf9e8
Some fixups 2023-10-08 10:29:46 +00:00
Robert Schulze
282200ef50
Merge pull request #55215 from rschu1ze/more-mysql-compat
Add `information_schema.columns.extra` and `information_schema.tables.table_rows`
2023-10-08 12:21:34 +02:00
Robert Schulze
8a8f769508
Update docs 2023-10-08 10:16:05 +00:00
Kseniia Sumarokova
20868f3b65
Merge pull request #55252 from kssenii/fix-background-download-in-cache
Fix for background download in fs cache
2023-10-08 10:38:23 +02:00
Kseniia Sumarokova
0af252ae31
Merge pull request #55297 from ClickHouse/kssenii-patch-6
MaterializedPostgreSQL: remove back check
2023-10-08 10:31:23 +02:00
Robert Schulze
66d4706338
Try to fix flakyness 2023-10-08 07:56:42 +00:00
Robert Schulze
f4bc58ea92
Merge remote-tracking branch 'rschu1ze/master' into more-mysql-compat 2023-10-08 07:49:37 +00:00
Azat Khuzhin
099665478d Fix incorrect merging of Nested for SELECT FINAL FROM SummingMergeTree
The problem was the order of the columns, in case of SELECT FINAL it got
"counters_Map.count", "counters_Map.id"

But in case of OPTIMIZE FINAL it got "counters_Map.id",
"counters_Map.count" correctly.

Note, that this bugs exists there from the very recent versions, I've
checked 19.x and it was there.

P.S. there is a workaround for this problem, if you will use one of the
following patterns for key columns:
- *ID
- *Key
- *Type

That way it will be explicitly matched as key and everything will work.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-08 07:32:47 +02:00
Azat Khuzhin
2a27a8b0bf tests/test_throttling: check the maximum execution time as well
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-08 07:31:33 +02:00
Azat Khuzhin
c883da2b26 tests/test_throttling: adjust delays due to HTTPS usage
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-08 07:31:33 +02:00
Azat Khuzhin
9c8096455e tests: switch test_throttling to S3 over https to make it more production like
One interesting thing about S3 C++ SDK is that it can read file multiple
times for calculating checksums and signature, but the last is not done
for the https protocol, though the checksum "could".

And indeed it does, since default checksum algorithm (MD5) does not
support streaming, and so it always calculated, regardless of the
protocol, however everything else (CRC*/SHA*) supports streamming and
actually will not be calculated for https at all!

This will be fixed in the follow up patch.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-08 07:31:33 +02:00
Azat Khuzhin
4179ff7111 Adjust the test_storage_delta::test_multiple_log_files after MinIO upgrade
After upgrade MinIO stopped handling of objects with leading slash:

    (Pdb) p minio_client.fput_object(bucket_name='root', object_name='/test_multiple_log_files/_delta_log/00000000000000000001.json', file_path='/test_multiple_log_files/_delta_log/00000000000000000001.json').http_headers
    HTTPHeaderDict({'Accept-Ranges': 'bytes', 'Content-Length': '0', 'ETag': '"f1f3276c787b3f6724b9a8edb313c4fa"', 'Server': 'MinIO', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'Vary': 'Origin, Accept-Encoding', 'X-Amz-Id-2': '7987905dee74cdeb212432486a178e511309594cee7cb75f892cd53e35f09ea4', 'X-Amz-Request-Id': '178B9B0904A9640D', 'X-Content-Type-Options': 'nosniff', 'X-Xss-Protection': '1; mode=block', 'Date': 'Fri, 06 Oct 2023 19:22:27 GMT'})
    (Pdb) p list(minio_client.list_objects('root', '/test_multiple_log_files/_delta_log/'))
    []
    (Pdb) p minio_client.get_object(bucket_name='root', object_name='/test_multiple_log_files/_delta_log/00000000000000000001.json').read()
    b'{"commitInfo":{"timestamp":1696618985577,"operation":"WRITE","operationParameters":{"mode":"Overwrite","partitionBy":"[]"},"readVersion":0,"isolationLevel":"Serializable","isBlindAppend":false,"operationMetrics":{"numFiles":"1","numOutputRows":"100","numOutputBytes":"2459"},"engineInfo":"Apache-Spark/3.3.2 Delta-Lake/2.3.0","txnId":"2a69a4c9-f72e-4ac9-9374-bdff96707edf"}}\n{"add":{"path":"part-00000-9ff4728b-d5fe-49ac-a773-e2d6e01f4139-c000.parquet","partitionValues":{},"size":2459,"modificationTime":1696618985310,"dataChange":true,"stats":"{\\"numRecords\\":100,\\"minValues\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":0,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":\\"1\\"},\\"maxValues\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":99,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":\\"99\\"},\\"nullCount\\":{\\"col-ab7641d2-f0a9-4ffb-ae37-27087d13dfc8\\":0,\\"col-6cbfced4-5921-4e3b-9226-558171c11949\\":0}}"}}\n{"remove":{"path":"part-00000-7c29f5d9-e14d-419a-a273-58d07775e1e9-c000.parquet","deletionTimestamp":1696618985576,"dataChange":true,"extendedFileMetadata":true,"partitionValues":{},"size":2459}}\n'
    (Pdb) p list(minio_client.list_objects(bucket_name='root', prefix='test_multiple_log_files/', recursive=True))
    [<minio.datatypes.Object object at 0x7ffff1ec2fe0>, <minio.datatypes.Object object at 0x7ffff1ec39d0>, <minio.datatypes.Object object at 0x7ffff1ec2590>, <minio.datatypes.Object object at 0x7ffff1ec3dc0>, <minio.datatypes.Object object at 0x7ffff1ec3a00>, <minio.datatypes.Object object at 0x7ffff1ec3fa0>, <minio.datatypes.Object object at 0x7ffff1ec3f40>, <minio.datatypes.Object object at 0x7ffff1ec2770>]

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-08 07:30:13 +02:00
Azat Khuzhin
ce9cdc408c Upgrade MinIO to support accepting non signed requests
During fixing one issue with multiple reads of input files while sending
them to S3 this had been pops up.

The problem is in this place [1].

  [1]: 200caab82b/cmd/object-handlers.go (L1573-L1591)

And this had been added only in [2].

  [2]: https://github.com/minio/minio/pull/16484

But even this implementation is not complete, since it does not
correctly support STREAMING-UNSIGNED-PAYLOAD-TRAILER, but this will be
the first step, to see if there are still problems with newer MinIO.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-10-08 07:30:13 +02:00
Alexey Milovidov
b38d4b5b0f
Merge pull request #55119 from jmaicher/fix/54318/deadlock-ldap-role-update
Fix deadlock in LDAP assigned role update
2023-10-08 00:38:10 +02:00
robot-clickhouse-ci-1
48ce595e24
Merge pull request #55220 from den-crane/Docs/use_local_cache_for_remote_storage
Doc. Update use_local_cache_for_remote_storage settings name
2023-10-08 00:23:54 +02:00
Alexey Milovidov
f5b0fb27a4
Update DiskObjectStorageMetadata.cpp 2023-10-08 00:53:33 +03:00
Alexey Milovidov
25a7f86da7
Update hive.md 2023-10-08 00:51:19 +03:00
Alexey Milovidov
857ac2705d
Merge pull request #55226 from ClickHouse/server-provides-keeper
Make clickhouse-server provides keeper
2023-10-07 23:49:14 +02:00