thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-18 13:42:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
126,587 Commits 9,138 Branches 1,774 Tags 2 GiB
abe6df6fbc
Commit Graph

126587 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexey Milovidov
5ad055df98
Merge pull request #55372 from azat/web-disk-fix-race 
Fix data-race in web disk
 2023-10-11 01:03:48 +02:00
Robert Schulze
b7936cb953
Update src/Functions/byteSwap.cpp 
Co-authored-by: Priyansh Agrawal <agrawal.priyansh@yahoo.in>
 2023-10-10 23:45:57 +02:00
Robert Schulze
96e3a4f9b4
Update src/Functions/byteSwap.cpp 
Co-authored-by: Priyansh Agrawal <agrawal.priyansh@yahoo.in>
 2023-10-10 23:45:23 +02:00
Mike Kot
06f3b82cc9 remove moving/ dir if allow_remove_stale_ is off 2023-10-10 19:56:22 +00:00
Serge Klochkov
da3a9ac707
Add #pragma once 2023-10-10 21:44:49 +02:00
Azat Khuzhin
59569b62b9 Add a comment about thread-safety of getFileOffsetOfBufferEnd() 2023-10-10 21:12:31 +02:00
Azat Khuzhin
8021d8d747 Fix data-race in web disk 
TSan report:

    WARNING: ThreadSanitizer: data race (pid=8)
     Write of size 8 at 0x7b54002b0400 by thread T310:
       0 DB::ReadBufferFromWebServer::seek(long, int) build_docker/./src/Disks/IO/ReadBufferFromWebServer.cpp:159:12 (clickhouse+0x18a6a45b) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       1 DB::CachedOnDiskReadBufferFromFile::getImplementationBuffer(DB::FileSegment&) build_docker/./src/Disks/IO/CachedOnDiskReadBufferFromFile.cpp:471:47 (clickhouse+0x17f1aace) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       2 DB::CachedOnDiskReadBufferFromFile::nextImplStep() build_docker/./src/Disks/IO/CachedOnDiskReadBufferFromFile.cpp:844:33 (clickhouse+0x17f1f5af) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       3 DB::CachedOnDiskReadBufferFromFile::nextImpl() build_docker/./src/Disks/IO/CachedOnDiskReadBufferFromFile.cpp:774:16 (clickhouse+0x17f1f33a) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       4 DB::ReadBuffer::next() build_docker/./src/IO/ReadBuffer.h:70:20 (clickhouse+0x17f1307f) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       5 DB::ReadBufferFromRemoteFSGather::readImpl() build_docker/./src/Disks/IO/ReadBufferFromRemoteFSGather.cpp:188:32 (clickhouse+0x17f1307f)
       6 DB::ReadBufferFromRemoteFSGather::nextImpl() build_docker/./src/Disks/IO/ReadBufferFromRemoteFSGather.cpp:160:9 (clickhouse+0x17f12f89) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       7 DB::ReadBuffer::next() build_docker/./src/IO/ReadBuffer.h:70:20 (clickhouse+0x17d695ca) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       8 DB::ThreadPoolRemoteFSReader::execute(DB::IAsynchronousReader::Request) build_docker/./src/Disks/IO/ThreadPoolRemoteFSReader.cpp:95:25 (clickhouse+0x17d695ca)
       9 DB::ThreadPoolRemoteFSReader::submit(DB::IAsynchronousReader::Request)::$_0::operator()() const build_docker/./src/Disks/IO/ThreadPoolRemoteFSReader.cpp:70:80 (clickhouse+0x17d6ab48) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)

     Previous read of size 8 at 0x7b54002b0400 by thread T297:
       0 DB::ReadBufferFromWebServer::getFileOffsetOfBufferEnd() const build_docker/./src/Disks/IO/ReadBufferFromWebServer.h:36:63 (clickhouse+0x18a6ac9d) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       1 DB::CachedOnDiskReadBufferFromFile::getInfoForLog() build_docker/./src/Disks/IO/CachedOnDiskReadBufferFromFile.cpp:1223:71 (clickhouse+0x17f24835) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       2 DB::CachedOnDiskReadBufferFromFile::nextImplStep()::$_0::operator()() const build_docker/./src/Disks/IO/CachedOnDiskReadBufferFromFile.cpp:799:5 (clickhouse+0x17f2347c) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       3 BasicScopeGuard<DB::CachedOnDiskReadBufferFromFile::nextImplStep()::$_0>::invoke() build_docker/./base/base/../base/scope_guard.h:99:9 (clickhouse+0x17f2347c)
       4 BasicScopeGuard<DB::CachedOnDiskReadBufferFromFile::nextImplStep()::$_0>::~BasicScopeGuard() build_docker/./base/base/../base/scope_guard.h:48:26 (clickhouse+0x17f2347c)
       5 DB::CachedOnDiskReadBufferFromFile::nextImplStep() build_docker/./src/Disks/IO/CachedOnDiskReadBufferFromFile.cpp:1071:1 (clickhouse+0x17f22480) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       6 DB::CachedOnDiskReadBufferFromFile::nextImpl() build_docker/./src/Disks/IO/CachedOnDiskReadBufferFromFile.cpp:774:16 (clickhouse+0x17f1f33a) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       7 DB::ReadBuffer::next() build_docker/./src/IO/ReadBuffer.h:70:20 (clickhouse+0x17f1307f) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       8 DB::ReadBufferFromRemoteFSGather::readImpl() build_docker/./src/Disks/IO/ReadBufferFromRemoteFSGather.cpp:188:32 (clickhouse+0x17f1307f)
       9 DB::ReadBufferFromRemoteFSGather::nextImpl() build_docker/./src/Disks/IO/ReadBufferFromRemoteFSGather.cpp:160:9 (clickhouse+0x17f12f89) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       10 DB::ReadBuffer::next() build_docker/./src/IO/ReadBuffer.h:70:20 (clickhouse+0x17d695ca) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)
       11 DB::ThreadPoolRemoteFSReader::execute(DB::IAsynchronousReader::Request) build_docker/./src/Disks/IO/ThreadPoolRemoteFSReader.cpp:95:25 (clickhouse+0x17d695ca)
       12 DB::ThreadPoolRemoteFSReader::submit(DB::IAsynchronousReader::Request)::$_0::operator()() const build_docker/./src/Disks/IO/ThreadPoolRemoteFSReader.cpp:70:80 (clickhouse+0x17d6ab48) (BuildId: cb41845e0edd51bbbf1abf7c1716e74d12491aae)

CI: https://s3.amazonaws.com/clickhouse-test-reports/55261/de503f75dcbc5a4d0e7fbb2e6b08c2106d62848a/integration_tests__tsan__[5_6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-10 21:11:57 +02:00
Azat Khuzhin
b107712e0c Fix filtering by virtual columns with OR filter in query 
The problem with the initial implementation #52653 was:
- OR can have multiple arguments
- It simply not correct to assume that if there are two arguments this is OK.
  Consider the following example:

    "WHERE (column_not_from_partition_by = 1) OR false OR false"

  Will be converted to:

    "WHERE false OR false"

And it will simply read nothing.

Yes, we could apply some optimization for bool, but this will not always
work, since to optimize things like "0 = 1" we need to execute it.

And the only way to make handle this correctly (with ability to ignore
some commands during filtering) is to make is_constant() function return
has it use something from the input block, so that we can be sure, that
we have some sensible, and not just "false".

Plus we cannot simply ignore the difference of the input and output
arguments of handling OR, we need to add always-true (1/true) if the
size is different, since otherwise it could break invariants (see
comment in the code).

This includes (but not limited to):
- _part* filtering for MergeTree
- _path/_file for various File/HDFS/... engines
- _table for Merge
- ...

P.S. analyzer does not have this bug, since it execute expression as
whole, and this is what filterBlockWithQuery() should do actually
instead, but this will be a more complex patch.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-10 20:47:52 +02:00
slvrtrn
76da9cf780 Remove old Java client tests 
Revert Java client docker image JDK bump (no AArch64 available)
 2023-10-10 20:35:51 +02:00
slvrtrn
e06d3ca1a5 Fix MySQL text protocol DateTime 
Introduce `removeLowCardinalityAndNullable` function
Fix incorrect removeLowCar/removeNullable usages
Add more MySQL text protocol tests
Deprecate old Java client tests
Use JDK 17 for test MySQL Java container
 2023-10-10 19:51:09 +02:00
Vitaly Baranov
2af66dd9e9
Merge pull request #55089 from vitlibar/improve-hiding-secrets-in-logs 
Improve hiding secrets in logs
 2023-10-10 19:02:40 +02:00
Nikita Mikhaylov
4456fe40f9
Remove the old code for projection analysis (#55112) 2023-10-10 17:13:32 +02:00
alesapin
36b4d028ef Merge branch 'fix_out_of_bound_disk_web' of github.com:ClickHouse/ClickHouse into fix_out_of_bound_disk_web 2023-10-10 17:10:41 +02:00
alesapin
3e2c9c662c Fix tabulation 2023-10-10 17:09:26 +02:00
robot-clickhouse-ci-2
a9cb206044
Merge pull request #55453 from ClickHouse/update-rabbitq 
Fix test that never worked test_rabbitmq_random_detach
 2023-10-10 17:05:44 +02:00
robot-clickhouse
d500ebcb73 Automatic style fix 2023-10-10 14:21:35 +00:00
alesapin
b677318415 Add oneliner to test 2023-10-10 15:59:33 +02:00
Han Fei
bb204ed0bd refine exception messages 2023-10-10 15:46:52 +02:00
alesapin
37f5c445b4 Fix out of bound error in disk web 2023-10-10 15:44:27 +02:00
Han Fei
3757bcf2cf
Revert "Revert "refine error code of duplicated index in create query"" 2023-10-10 15:44:15 +02:00
Robert Schulze
8b7d1021cd
Docs: Update field descriptions of system.part_log 2023-10-10 13:32:20 +00:00
Kruglov Pavel
5ded0005a3
Merge pull request #55064 from AVMusorin/system-drop-format-cache 
Allow drop cache for Protobuf format
 2023-10-10 14:23:11 +02:00
Mikhail f. Shiryaev
3ed7cffbb4
Merge branch 'master' into docker-initialization-once 2023-10-10 13:40:05 +02:00
Robert Schulze
24fecaeb7d
Merge pull request #55427 from slvrtrn/mysql-boolean-format-fix 
Fix MySQL interface boolean representation
 2023-10-10 13:11:33 +02:00
Robert Schulze
31ae84604f
Some fixups 2023-10-10 09:54:06 +00:00
alesapin
666c690b4f
Merge pull request #55309 from ClickHouse/fix_redundant_check 
Fix bug with inability to drop detached partition in replicated merge tree on top of S3 without zero copy
 2023-10-10 11:48:00 +02:00
robot-clickhouse
9640e09f38 Automatic style fix 2023-10-10 08:38:11 +00:00
Yatsishin Ilya
1f72bd85e9 fix test that never worked 2023-10-10 08:25:04 +00:00
Yatsishin Ilya
47512f7b40 try another fix 2023-10-10 07:42:30 +00:00
helifu
ca0c4ba147
Merge branch 'ClickHouse:master' into master13 2023-10-10 13:49:49 +08:00
凌涛
e29a828d6a rebase master 2023-10-10 10:58:13 +08:00
凌涛
51e2e59a97 Merge remote-tracking branch 'origin/master' into feature/function-getClientHttpHeader 2023-10-10 10:56:24 +08:00
Alexey Milovidov
68ce6b9b00
Merge pull request #55424 from ClickHouse/update-rabbitq 
Updated RabbitMQ image and fixed log retrieval in failed tests.
 2023-10-10 03:32:21 +02:00
robot-ch-test-poll4
8609cd25fc
Merge pull request #55392 from azat/dist-insert-threads-fix 
Fix missing thread accounting for insert_distributed_sync=1
 2023-10-10 03:18:16 +02:00
Alexey Milovidov
8dad8ea8c3
Merge pull request #55206 from den-crane/patch-61 
Update CHANGELOG.md  / issue with with a downgrade because of ratio_of_defaults_for_sparse_serialization
 2023-10-10 01:39:02 +02:00
Alexey Milovidov
e204b1d616
Merge pull request #55419 from evillique/mongo-fix 
Fix MongoDB connection issues
 2023-10-10 01:30:04 +02:00
Alexey Milovidov
bc7bcef71e
Merge pull request #50944 from ClickHouse/anti-join-invalid-number-of-rows-in-chunk 
ANTI JOIN: Invalid number of rows in Chunk
 2023-10-10 01:22:45 +02:00
Alexey Milovidov
814f38050c
Update 02785_left_anti_join_bug.sql 2023-10-10 02:22:27 +03:00
Alexey Milovidov
9ef6959b7a
Merge pull request #55425 from rschu1ze/better-untuple-tests 
Improve tests for untuple()
 2023-10-10 01:19:34 +02:00
Alexey Milovidov
25d4e41b5e
Merge pull request #55377 from k-morozov/feature/add_internal_replication_to_system_clusters 
add internal_replication to system.clusters
 2023-10-10 01:18:01 +02:00
Alexey Milovidov
76977a6a72
Merge pull request #55423 from hanfei1991/hanfei/refine-ttl-doc 
change a docs example about creating table columns with ttl
 2023-10-10 01:14:38 +02:00
slvrtrn
fa28e01535 Trigger CI 2023-10-10 00:57:15 +02:00
Anton Popov
e9ca893387
Merge pull request #55417 from CurtizJ/fix-cast-operator 
Fix parsing of arrays in cast operator
 2023-10-09 23:54:25 +02:00
Alexey Milovidov
b50889a8ff
Merge pull request #55186 from ClickHouse/remove-in-memory-data-parts 
Remove the support for in memory data parts (part 2)
 2023-10-09 23:53:25 +02:00
Alexey Milovidov
33bc6010ea
Merge pull request #55362 from ClickHouse/merging-43085 
Merging #43085
 2023-10-09 23:52:50 +02:00
Priyansh Agrawal
9677dedd87 Add negative tests. 
Also:
- Add comments in tests.
- Add an example in docs where an IPv4 is casted to an int, byteswapped
  and then casted back to an IPv4.
 2023-10-09 21:52:06 +00:00
Priyansh Agrawal
2e4709fd10 Remove "unsigned" from docs now that we support all ints. 2023-10-09 21:16:46 +00:00
Han Fei
8d5eb26185 address comments 2023-10-09 23:06:23 +02:00
Alexey Milovidov
04ed4ece7a
Merge pull request #55208 from arthurpassos/add_option_to_set_env_variables_for_a_single_node_integration_tests 
Add option to set env variables for a single node in integration tests
 2023-10-09 22:37:04 +02:00
Alexey Milovidov
0094be5d43
Merge pull request #55352 from kitaisreal/context-locks-small-fixes 
Context locks small fixes
 2023-10-09 21:50:59 +02:00