thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-30 05:30:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
76,398 Commits 8,961 Branches 1,767 Tags 1.8 GiB
bdf640326b
Commit Graph

19514 Commits

Author SHA1 Message Date
Maksim Kita
772c326f2d
Merge pull request #30500 from kitaisreal/dictionaries-read-multiple-threads 
Dictionaries read support multiple threads
 2021-10-25 15:31:50 +03:00
Alexander Tokmakov
075c3a37f4 Merge branch 'master' into fix_some_fetches_may_stuck 2021-10-25 15:28:54 +03:00
Nikolai Kochetov
a0a5c1a21d Fix argument types for now and now64 2021-10-25 12:06:53 +03:00
alesapin
28bde879ed
Merge pull request #30455 from kssenii/deprecate-setting 
Deprecate write_final_mark setting
 2021-10-25 10:51:49 +03:00
alesapin
490ca93162
Merge pull request #30372 from ClickHouse/update_keeper_config 
Updatable keeper configuration.
 2021-10-25 10:37:03 +03:00
alesapin
aca3166a7c
Merge pull request #30550 from ClickHouse/fix_typo_update_nuraft 
Fix typo and update NuRaft
 2021-10-25 10:36:06 +03:00
kssenii
52296e8b04 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-25 00:22:32 +03:00
alexey-milovidov
7200cb8bba
Merge pull request #30611 from azat/readbuffer-real-size 
Replace estimated_size with optional<size> for readFile/createReadBufferFromFileBase
 2021-10-24 23:37:30 +03:00
kssenii
13a2a3d307 Merge branch 'master' of github.com:ClickHouse/ClickHouse into deprecate-setting 2021-10-24 23:05:21 +03:00
Anton Popov
37de54937c fix use-after-free 2021-10-24 22:32:28 +03:00
Anton Popov
76cb191865
Merge pull request #30433 from kssenii/fix-positional 
More full support of positional arguments
 2021-10-24 21:48:17 +03:00
kssenii
658b33717f A little nicer 2021-10-24 20:43:34 +03:00
kssenii
5b15e2474c Checkout reading from local fs back to master version 2021-10-24 20:41:18 +03:00
Kruglov Pavel
855b10261c
Merge pull request #30285 from kssenii/fix-local-exceptions 
Fix printing stacktraces for clickhouse-local
 2021-10-24 19:54:59 +03:00
Kseniia Sumarokova
4667ea7883
Merge pull request #29774 from kssenii/remote-connection 
Predefined configuration for table function remote
 2021-10-24 19:38:01 +03:00
Azat Khuzhin
f9b27608f5 Fix reading of columns if offset_in_compressed_file is the same for mark range 
Fixes: 00078_group_by_arrays
 2021-10-24 18:58:22 +03:00
Azat Khuzhin
258b62d375 Remove one unused CompressedReadBufferFromFile ctor 2021-10-24 18:58:22 +03:00
Azat Khuzhin
34d8f78c33 Explictly pass size in DiskCacheWrapper::writeFile() 
v0: Add explicit default value for DiskLocal::readFile()
v2: Explictly pass size in DiskCacheWrapper::writeFile()
    (since v0 not google-default-arguments compatible)
 2021-10-24 18:58:22 +03:00
Azat Khuzhin
93c34824c0 Fix explicit 0 size for createReadBufferFromFileBase() 2021-10-24 18:58:22 +03:00
Azat Khuzhin
4599e9191b Fix explicit 0 size for readFile() 2021-10-24 18:58:22 +03:00
Azat Khuzhin
2b96433daf Replace estimated_size with optional<size> for IDisk::readFile() 
v2: rebase to fix conflicts in DiskEncrypted.cpp
 2021-10-24 18:58:11 +03:00
Vitaly Baranov
c0b01e7b0b
Merge pull request #30494 from vitlibar/fix-reading-from-empty-file-on-encrypted-disk 
Fix reading from empty file on encrypted disk
 2021-10-24 16:24:47 +03:00
Nickita Taranov
877e8b579b fix style 2021-10-24 14:45:12 +03:00
Kseniia Sumarokova
7383bdd5fa
Merge pull request #25969 from ucasfl/filelog-engine 
Implementation of a table engine to consume application log files in ClickHouse
 2021-10-24 10:14:05 +03:00
alexey-milovidov
f1e96ee9ac
Merge pull request #30594 from azat/fix-logical-error-on-connection-drain 
Fix LOGICAL_ERROR on connection draining in case of ECONNRESET
 2021-10-24 05:34:27 +03:00
alexey-milovidov
830b83218e
Merge pull request #30579 from azat/tsan-recursion 
Fix tests that relies on checking stack size under TSan
 2021-10-24 03:14:07 +03:00
alexey-milovidov
8c2413f6fe
Merge pull request #30562 from ClickHouse/fix-header-after-limit-push-down 
Fix LimitStep header after limit push down optimization.
 2021-10-24 00:19:02 +03:00
Azat Khuzhin
83d68246c3 Fix LOGICAL_ERROR on connection draining in case of ECONNRESET 
In case of ECONNRESET (Connection reset by peer) the "cancelled" will
not be set, and so drain() will fail with LOGICAL_ERROR:

<details>

    2021.10.16 04:10:55.054771 [ 7635 ] {882164e0-ca26-4955-8759-708ba1f6e85c} <Error> PullingAsyncPipelineExecutor: Code: 210. DB::NetException: Connection reset by peer, while writing to socket (127.0.0.2:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

    ...
    4. ./obj-x86_64-linux-gnu/../src/Common/NetException.h:12: DB::NetException::NetException(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x20db5749 in /usr/bin/clickhouse
    5. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromPocoSocket.cpp:56: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x211d56bb in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:47: DB::WriteBuffer::next() @ 0x14e4c874 in /usr/bin/clickhouse
    7. ./obj-x86_64-linux-gnu/../src/Client/Connection.cpp:560: DB::Connection::sendCancel() @ 0x227c0c22 in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Client/HedgedConnections.cpp:242: DB::HedgedConnections::sendCancel() @ 0x227f1b53 in /usr/bin/clickhouse
    ...
    2021.10.16 04:12:35.555530 [ 5575 ] {} <Fatal> : Logical error: 'Cannot drain connections: cancel first.'.

</details>

Found by stress tests on CI [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/0/d3cb99701d4aaec384fd183168d10e0f650ecc1d/stress_test_(debug).html#fail1
 2021-10-23 22:44:32 +03:00
Vitaly Baranov
26cb62de32 Add more tests for the "encrypted" disk type. 2021-10-23 20:58:37 +03:00
Kseniia Sumarokova
00d0665bf2
Merge pull request #30477 from kssenii/postgres-fix-query 
More reliable fetch query in postgres database
 2021-10-23 19:07:34 +03:00
kssenii
e04e37916e Review fixes 2021-10-23 18:20:31 +03:00
Anton Popov
cdfa36a181
Merge pull request #30266 from CurtizJ/aggregation-in-order-4 
Improve performance of aggregation in order of primary key
 2021-10-23 18:09:17 +03:00
kssenii
36d0f67b38 Merge branch 'master' of github.com:ClickHouse/ClickHouse into remote-connection 2021-10-23 17:12:46 +03:00
Maksim Kita
4ccc619bd5 Added performance tests 2021-10-23 15:01:30 +03:00
Azat Khuzhin
b73092c169 Fix tests that relies on checking stack size under TSan 
Under TSan using too much stack requires too much RSS for shadow memory,
and neither of this TSAN_OPTIONS helps:
- history_size=2
- flush_memory_ms=2000
- memory_limit_mb=50000

So instead, decrease allowed limit of the stack size in checkStackSize()
under TSan, to address exessive memory usage for the server compiled
with TSan.

Note, that before this patch 01763_max_distributed_depth test can
increase RSS of the server to 70GiB.
 2021-10-23 14:09:19 +03:00
feng lv
235339d530 fix 2021-10-23 09:35:33 +00:00
Kseniia Sumarokova
05150f151c
Update ExpressionAnalyzer.cpp 2021-10-23 12:19:40 +03:00
Vladimir C
410624749e
Merge pull request #30492 from vdimir/fix-alter-scalar-deadlock 2021-10-23 12:01:00 +03:00
Kseniia Sumarokova
f362d420b8
Update fetchPostgreSQLTableStructure.cpp 2021-10-23 11:38:18 +03:00
kssenii
1fa123ee5c Properly done 2021-10-23 01:33:17 +03:00
Anton Popov
74cf57b4b0 fix for empty chunks 2021-10-22 21:48:03 +03:00
Nikolai Kochetov
ee6bc3883f Try to fix 30397 2021-10-22 19:25:48 +03:00
Nickita Taranov
2d23e3b17d support for ColumnString 2021-10-22 18:53:02 +03:00
Nickita Taranov
a211c9ecbc support nullable for ColumnConst 2021-10-22 18:52:59 +03:00
Anton Popov
7c458fbc4c fix style check 2021-10-22 18:04:11 +03:00
Nikolai Kochetov
e00d55c08d Try to fix 30397 2021-10-22 17:42:20 +03:00
Kseniia Sumarokova
153dd9a895
Update MergeTreeSettings.h 2021-10-22 16:27:01 +03:00
Nikolai Kochetov
54cc8e1d27 Fix LimitStep header after limit push down optimization. 2021-10-22 16:19:49 +03:00
kssenii
39b9e9c258 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-22 15:40:41 +03:00
kssenii
a1a2e276ae Review fixes 2021-10-22 12:26:28 +00:00
Filatenkov Artur
1c72421e53
Add columns in columns.sql for INFORMATION_SCHEMA (#29637) 
* add columns in columns.sql for INFORMATION_SCHEMA
 2021-10-22 15:18:16 +03:00
Kseniia Sumarokova
5a60d9a3a3
Merge pull request #30521 from azat/local-fix 
clickhouse-local: fix block lost in interactive mode
 2021-10-22 14:49:39 +03:00
Maksim Kita
3a6f55607d Fixed IPAddressDictionary tests 2021-10-22 14:18:45 +03:00
Maksim Kita
243be5028a
Merge pull request #30014 from jinjunzh/master 
add x86 feature avx2/avx512 support for accelerating filter operation
 2021-10-22 14:16:47 +03:00
Anton Popov
151daf61c1
Merge pull request #30522 from azat/remove-check_columns 
Remove check_columns argument from MergeTree code (false was never passed)
 2021-10-22 14:06:49 +03:00
Maksim Kita
fc8a515b42
Merge pull request #30431 from vesslanjin/remove-branchy-code-in-ColumnDecimal 
remove branch in ColumnDecimal, same as pull req #29881
 2021-10-22 13:24:44 +03:00
Kseniia Sumarokova
057fe51ec8
Update TableFunctionPostgreSQL.cpp 2021-10-22 13:08:38 +03:00
kssenii
c95a29edac Fix quoting 2021-10-22 10:07:23 +00:00
vdimir
cfb98d381c
Do not use default scalars in MutationsInterpreter::prepareInterpreterSelectQuery 2021-10-22 12:47:28 +03:00
Nikolai Kochetov
1533f4af57
Merge pull request #30335 from ClickHouse/single-sorting-step 
Single sorting step
 2021-10-22 11:34:29 +03:00
Vladimir C
d60f00bab8
Merge pull request #30520 from azat/fix-unary-postfix-rewrite 2021-10-22 11:31:36 +03:00
alesapin
d849f56cd4 Fix typo and update NuRaft 2021-10-22 11:24:18 +03:00
Maksim Kita
6561f9bed2
Merge pull request #30483 from kitaisreal/sql-user-defined-functions-composition-fix 
SQLUserDefinedFunctions composition fix
 2021-10-22 10:16:31 +03:00
mergify[bot]
b043a04767
Merge branch 'master' into update_keeper_config 2021-10-22 07:12:42 +00:00
feng lv
54dc9693fb fix 2021-10-22 07:08:08 +00:00
feng lv
5cb783aed0 fix 2021-10-22 02:36:24 +00:00
kssenii
7f2c5dcaae Fix clang-tidy 2021-10-22 01:08:58 +03:00
Azat Khuzhin
b0984a3337 clickhouse-local: fix block lost in interactive mode 
In case only one block in the query execution, LocalConnection can
return Progress packet instead of the block itself, after it will call
poll() again and the block will got lost.

Fix this by processing block before Progress packet.

And actually AFAICS Progress can be removed after pollImpl(), since
there is Progress handling before.

Fixes: #30282 (cc @kssenii)
 2021-10-22 00:46:01 +03:00
Maksim Kita
117cb6c614 Fixed RangeHashedDictionary tests 2021-10-22 00:22:32 +03:00
Maksim Kita
adaa14f8aa
Merge pull request #30502 from kitaisreal/storage-dictionary-fix-potential-configuration-race 
StorageDictionary fix potential configuration race
 2021-10-22 00:14:11 +03:00
Azat Khuzhin
496aa43a9f Remove check_columns argument from MergeTree code (false was never passed) 2021-10-21 21:24:59 +03:00
Azat Khuzhin
e1b6acfa52 Add missing parenthesis for isNotNull/isNull rewrites to IS [NOT] NULL 
After #29446 the following query was "broken":

    select isNotNull(1)+isNotNull(2) from remote('127.2', system.one)

This was also a problem for queries to external storages (i.e. mysql),
since it also uses query rewrite, like non-local distributed queries.

Fixes: #29446
 2021-10-21 21:18:00 +03:00
kssenii
56f0560c46 Asserts and read till end option 2021-10-21 20:54:03 +03:00
Vitaly Baranov
5695ef55b3
Merge pull request #28125 from vitlibar/fix-writing-storage-log-marks 
Fix writing marks in StorageLog
 2021-10-21 20:10:01 +03:00
Vitaly Baranov
172a6f2e0d Fix reading from empty file on encrypted disk. 2021-10-21 19:31:29 +03:00
Maksim Kita
4ea969350b Fixed build 2021-10-21 18:43:33 +03:00
kssenii
120b01e089 Better 2021-10-21 18:22:17 +03:00
vdimir
039af96b61
Set execute_scalar_subqueries in MutationsInterpreter only for MergeTree and prepareInterpreterSelectQuery 2021-10-21 18:05:46 +03:00
avogar
8aad00818a Fix build 2021-10-21 18:01:23 +03:00
Dmitry Novik
58ee783be4
Merge pull request #30293 from amosbird/projection-fix19 
Fix projections with JOIN alias columns
 2021-10-21 17:49:21 +03:00
avogar
b50fff4e2b Fix style 2021-10-21 17:33:26 +03:00
Maksim Kita
8885748a6c StorageDictionary fix potential configuration race 2021-10-21 17:25:53 +03:00
Maksim Kita
a3a780bbf5 Dictionaries read support multiple threads 2021-10-21 17:17:53 +03:00
Anton Popov
b1a74af44d
Merge pull request #30180 from CurtizJ/remove-sample-by 
Allow to remove `SAMPLE BY` expression
 2021-10-21 17:09:53 +03:00
tavplubix
4c5104b30d
Merge pull request #30432 from ClickHouse/tavplubix-patch-3 
Fix segfault on REPLACE PARTITION if session expired
 2021-10-21 17:08:57 +03:00
avogar
7a37e24b52 Small fix in deserializeWholeText 2021-10-21 16:58:25 +03:00
avogar
d5c5a3213b Add custom null representation support for TSV/CSV input formats, fix bugs in deserializing NULLs in some cases 2021-10-21 16:52:27 +03:00
mergify[bot]
86e1bb6b55
Merge branch 'master' into fix_like_function 2021-10-21 12:24:28 +00:00
Ilya Golshtein
551a1065c1 output_format_avro_rows_in_file default is 1000000 2021-10-21 14:19:25 +03:00
vdimir
ed0bb2cec2
Fix deadlock on ALTER with scalar subquery to the same table 2021-10-21 14:02:51 +03:00
Kseniia Sumarokova
da0021c82f
Merge pull request #30412 from ClickHouse/kssenii-fix-test 
Follow-up for #30282
 2021-10-21 12:48:10 +03:00
Kruglov Pavel
27e8a83269
Merge pull request #30426 from amosbird/jsonextractstring3 
JSONExtractString raw string support.
 2021-10-21 12:29:11 +03:00
Vladimir C
388c3123f0
Merge pull request #30370 from Enmk/fix_session_log 2021-10-21 11:07:48 +03:00
mergify[bot]
aee446352a
Merge branch 'master' into update_keeper_config 2021-10-21 07:48:23 +00:00
Maksim Kita
650a79a907 SQLUserDefinedFunctions composition fix 2021-10-21 10:08:08 +03:00
Kseniia Sumarokova
feeb572219
Merge pull request #30474 from kssenii/sql-json-fix 
sql json change order of arguments
 2021-10-21 09:47:12 +03:00
Kseniia Sumarokova
5544e866f8
Merge pull request #30219 from lhuang09287750/master 
make date32 type comparable with date/datetime/string
 2021-10-21 09:38:55 +03:00
Ilya Golshtein
82f33151e7 output_format_avro_rows_in_file fixes per code review 2021-10-21 02:53:39 +03:00
kssenii
ec7c93728b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into fix-local-exceptions 2021-10-20 23:42:27 +00:00
kssenii
5a51d1c29b Merge branch 'fix-positional' of github.com:kssenii/ClickHouse into fix-positional 2021-10-20 23:40:52 +00:00
kssenii
bc4c2f2366 Fix clang-tidy 2021-10-20 23:40:21 +00:00
kssenii
a223526f45 Fix 2021-10-20 23:29:35 +00:00
kssenii
e4e157688d Review fixes 2021-10-21 01:57:43 +03:00
kssenii
c96b884951 Fix 2021-10-20 20:47:34 +00:00
alexey-milovidov
d3df1c02bc
Merge pull request #29672 from ClickHouse/clickhouse-obfuscator-2 
More enhancements for query obfuscator
 2021-10-20 23:06:29 +03:00
Maksim Kita
e550e0c22d
Merge pull request #30435 from kitaisreal/sql-user-defined-functions-support-lambdas 
SQLUserDefinedFunctions support lambdas
 2021-10-20 21:36:52 +03:00
Maksim Kita
1b4829d071
Merge pull request #30454 from kitaisreal/sql-user-defined-functions-create-or-replace-no-exists-support 
SQLUserDefinedFunctions support CREATE OR REPLACE, CREATE IF NOT EXISTS
 2021-10-20 21:36:42 +03:00
Kseniia Sumarokova
cae31437e8
Merge pull request #30336 from kssenii/fix-local-defaults 
Send table columns in clickhouse-local
 2021-10-20 21:18:02 +03:00
kssenii
220791b21c Fix 2021-10-20 17:15:07 +00:00
Nikita Mikhaylov
56869c0ec4
Merge pull request #30360 from nikitamikhaylov/allow-graphite-final 
Allow to use FINAL clause in SELECT from GraphiteMergeTree
 2021-10-20 19:57:44 +03:00
feng lv
6565e1d4b2 fix conflict 2021-10-20 15:22:16 +00:00
Maksim Kita
ff48017f4a SQLUserDefinedFunctions support CREATE OR REPLACE, CREATE IF NOT EXISTS 2021-10-20 17:56:46 +03:00
Maksim Kita
73e6110b2a
Merge pull request #30437 from kitaisreal/sql-user-defined-functions-drop-if-exists 
SQLUserDefinedFunctions added DROP IF EXISTS support
 2021-10-20 17:24:57 +03:00
Anton Popov
6b34398e9c support VALUES format in async inserts 2021-10-20 17:20:58 +03:00
kssenii
0a90be8317 Remove write_final_mark setting 2021-10-20 14:13:04 +00:00
Amos Bird
2f615e9176
Use original_query for projection analysis 2021-10-20 21:30:15 +08:00
Amos Bird
240895fba7
Fix projections with JOIN alias columns 2021-10-20 21:30:15 +08:00
Nikolai Kochetov
8986de7e50
Merge pull request #30384 from amosbird/projection-fix21 
Fix crash when minmax_count projection is used with primary key in partition expr
 2021-10-20 16:17:13 +03:00
Maksim Kita
54d3720426 ASTDropFunctionQuery formatting fix 2021-10-20 16:08:13 +03:00
avogar
872cca550a Make better 2021-10-20 15:47:20 +03:00
Anton Popov
78c925ddef
Merge pull request #30177 from CurtizJ/initializeAggregation-nullable 
Support nullable arguments in function `initializeAggregation`
 2021-10-20 15:39:07 +03:00
Maksim Kita
a9ac7fb394
Merge pull request #30451 from kitaisreal/executable-udf-clickhouse-example 
ExecutableUDF example fix style check
 2021-10-20 15:33:32 +03:00
Maksim Kita
ac358d08a6 Fix style check 2021-10-20 15:33:20 +03:00
mergify[bot]
0a4360c43e
Merge branch 'master' into tsv-csv 2021-10-20 11:57:06 +00:00
Kruglov Pavel
906d6da5d3
Update TabSeparatedRowInputFormat.cpp 2021-10-20 14:55:01 +03:00
avogar
7007286088 Fix WithNamesAndTypes parallel parsing, add new tests, small refactoring 2021-10-20 14:48:54 +03:00
vesslanjin
f607a5d0ab remove branch in ColumnDecimal, same as pull req #29881 
Co-authored-by: Zhu Jasper jasper.zhu@intel.com

Signed-off-by: vesslanjin <jun.i.jin@intel.com>
 2021-10-20 07:40:06 -04:00
kssenii
5f24eb10ec Test changes in MergeTreeReader for local fs 2021-10-20 13:52:30 +03:00
mergify[bot]
d1f7e74e6f
Merge branch 'master' into fix-positional 2021-10-20 10:40:39 +00:00
Maksim Kita
dc96408012 Fixed build 2021-10-20 13:09:48 +03:00
mergify[bot]
df4c75b1cb
Merge branch 'master' into update_keeper_config 2021-10-20 10:08:10 +00:00
Maksim Kita
ad409d9b47 SQLUserDefinedFunctions added DROP IF EXISTS support 2021-10-20 13:05:57 +03:00
Maksim Kita
b8407e53ed
Merge pull request #30436 from kitaisreal/executable-udf-clickhouse-example 
ExecutableUDF example
 2021-10-20 12:58:50 +03:00
Amos Bird
ddcf12c83b
JSONExtractString raw string support. 2021-10-20 17:57:14 +08:00
mergify[bot]
35b5cf23cb
Merge branch 'master' into update_keeper_config 2021-10-20 09:57:09 +00:00
Maksim Kita
3496cd1bfe ExecutableUDF example 2021-10-20 12:53:03 +03:00
Vasily Nemkov
0ad7f9bba2 Fixed PVS warning 2021-10-20 12:45:32 +03:00
Maksim Kita
83787e26f2 SQLUserDefinedFunctions support lambdas 2021-10-20 12:43:03 +03:00
Kseniia Sumarokova
c692155c7e
Merge pull request #30309 from kssenii/fix-file-symlinks 
Fix symlinks in file table function
 2021-10-20 11:09:51 +03:00
kssenii
6c990400d1 Better positional args 2021-10-20 07:52:39 +00:00
kssenii
31cd71c849 Clean up 2021-10-20 09:37:18 +03:00
tavplubix
5e24f337a0
Update StorageReplicatedMergeTree.cpp 2021-10-20 09:29:41 +03:00
kssenii
8f97765cbd Fix 2021-10-20 05:51:35 +00:00
lhuang0928
008a13df8b fix date32 comparison with datetime/datetime64 2021-10-20 03:36:07 +00:00
alexey-milovidov
23312c4808
Merge pull request #30361 from nikitamikhaylov/rename-geo-functions 
Rename readWkt* functions to readWKT*, added an alias SVG for svg function
 2021-10-20 03:46:42 +03:00
Ilya Golshtein
9f9f4a561c output_format_avro_rows_in_file - fixes 2021-10-19 23:26:55 +03:00
Kseniia Sumarokova
2266856c47
Follow-up for #30282 2021-10-19 23:03:45 +03:00
Kseniia Sumarokova
5324cc8359
Merge pull request #30282 from kssenii/fix-local-less-threads 
Less threads in clickhouse-local, fix Ok. printing
 2021-10-19 22:54:28 +03:00
alesapin
e530682529 Build fixes 2021-10-19 22:47:04 +03:00
kssenii
143e912e4e Fix split build 2021-10-19 22:45:46 +03:00
Vasily Nemkov
e72ec27d5b Fixed builds and using magic_enum 2021-10-19 19:45:48 +03:00
alesapin
28b96ff20f
Update gtest_coordination.cpp 2021-10-19 18:38:20 +03:00
Amos Bird
05d93796dc
Fix minmax_count projection with primary key in partition expr 2021-10-19 23:36:07 +08:00
Amos Bird
aa8bc93be8
Fix build 2021-10-19 23:35:59 +08:00
Vasily Nemkov
cef993233f Attempt to fix #30162 
Added some logging to the Session
 2021-10-19 18:11:46 +03:00
Maksim Kita
0b3926950d
Merge pull request #30143 from amosbird/useupstreamreplxx 
Use upstream replxx
 2021-10-19 17:50:43 +03:00
Maksim Kita
382c82805f
Merge pull request #30352 from kitaisreal/storage-executable-fix-small-issues 
StorageExecutable fix small issues
 2021-10-19 17:43:11 +03:00
Maksim Kita
0f2e23d775
Update StorageExecutable.cpp 2021-10-19 17:41:59 +03:00
Maksim Kita
0246381e8f
Merge pull request #30345 from kitaisreal/executable-pool-dictionary-source-fix-borrow-timeout-milliseconds 
ExecutablePool dictionary source fix borrow timeout milliseconds
 2021-10-19 17:41:36 +03:00
alesapin
18cceedc00 Fix build and update for clickhouse-keeper 2021-10-19 17:29:49 +03:00
alesapin
9f939364e3 Add some comments 2021-10-19 17:10:09 +03:00
Anton Popov
8d5ed8a342 remove useless line 2021-10-19 17:05:54 +03:00
alesapin
da14e5e784 Fix typos 2021-10-19 16:49:36 +03:00
Anton Popov
82bff416f6 Merge remote-tracking branch 'upstream/master' into HEAD 2021-10-19 16:48:47 +03:00
alesapin
042eebd981 Add test for leader remove 2021-10-19 16:37:28 +03:00
Anton Popov
c547ffa313 improve performance of aggregation in order primary key with function with heavy states 2021-10-19 16:36:31 +03:00
alesapin
647856658f Add synchronization and active wait 2021-10-19 16:11:29 +03:00
feng lv
05fd6f7caf fix comflict 2021-10-19 12:50:22 +00:00
Nikita Mikhaylov
75487be899 White list of storages that supports final 2021-10-19 12:16:33 +00:00
Nikita Mikhaylov
1114d06bc0 Done 2021-10-19 12:11:18 +00:00
Nikolai Kochetov
9195e4e887
Merge pull request #30359 from ClickHouse/remove-streams-from-comments 
Update obsolete comments.
 2021-10-19 15:04:04 +03:00
alesapin
2d4b601d38 Better 2021-10-19 15:00:26 +03:00
Alexander Tokmakov
caf972db48 fix 2021-10-19 14:56:04 +03:00
Nikita Mikhaylov
4fbd332bf1 Done 2021-10-19 11:53:55 +00:00
Vladimir C
49685f0b9a
Merge pull request #30292 from Joeywzr/trim_dev 2021-10-19 14:49:43 +03:00
Nikita Mikhaylov
ea2a647724
Merge pull request #30192 from azat/granula-adjust-LOGICAL_ERROR-fix 
Fix reading from MergeTree with max_read_buffer_size=0
 2021-10-19 13:25:55 +03:00
avogar
74fd33b62e Fix style 2021-10-19 13:25:22 +03:00
mergify[bot]
69667aab1a
Merge branch 'master' into fix-backward-compatibility-after-21196 2021-10-19 10:11:16 +00:00
Nikolai Kochetov
a92dc0a826 Update obsolete comments. 2021-10-19 12:58:10 +03:00
Ilya Yatsishin
4396daeb1f
Merge pull request #30064 from azat/client-print-profile-events 2021-10-19 12:42:09 +03:00
avogar
a16eda68dc Fix tests and style 2021-10-19 12:39:14 +03:00
Nikolai Kochetov
40677bffa5 Fix PVS warning. 2021-10-19 12:28:49 +03:00
Kruglov Pavel
5052ec3ab0
Merge branch 'master' into tsv-csv 2021-10-19 12:03:52 +03:00
Kruglov Pavel
1e2ceeb2e7
Merge pull request #29291 from Avogar/capnproto 
Add CapnProto output format, refactor CapnProto input format
 2021-10-19 11:54:55 +03:00
Kruglov Pavel
de2b75abf0
Merge pull request #30053 from Avogar/clickhouse-local 
Remove unused headers and handle exception 'unrecognised option' in clickhouse-local
 2021-10-19 11:52:15 +03:00
kssenii
4bf1f2ca12 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into fix-local-exceptions 2021-10-19 08:28:47 +00:00
kssenii
e53335bc6f Better way 2021-10-19 08:21:38 +00:00
Nikolai Kochetov
cc7a3ea9ff
Merge pull request #30322 from ClickHouse/update-mergine-sorted-memory-optimization 
Update memory optimisation for MergingSorted.
 2021-10-19 11:12:50 +03:00
kssenii
df5dc4e991 Adjust MergeTreeIndexReader 2021-10-19 10:51:45 +03:00
alesapin
f2d266acce Merge branch 'master' into update_keeper_config 2021-10-19 10:16:57 +03:00
alesapin
a992895b09 Debug 2021-10-19 10:14:53 +03:00
alesapin
53ad965f8e
Merge pull request #30170 from ClickHouse/debug_keeper 
Start keeper asynchronously in some cases
 2021-10-19 10:14:36 +03:00
kssenii
49106f407f Fix some checks 2021-10-19 09:34:27 +03:00
Maksim Kita
fb91b1d92b StorageExecutable fix small issues 2021-10-19 01:04:07 +03:00
Azat Khuzhin
424bf6fcf4 client: add ability to print raw profile events 
This can be useful for debugging and for testing (since you will not
need to obtain query_id and look at query_log).

v2:
- mark test as long
- add option to docs
- add type of profile event into logs too
v3:
- resolve conflicts
- and fix onProfileEvents callback
v4:
- add --print-profile-events separate switch
 2021-10-19 00:54:38 +03:00
Azat Khuzhin
f7b76373ce Move ProfileEvents packet type from TCPHandler into ProfileEventsExt 2021-10-19 00:54:38 +03:00
Alexander Tokmakov
89fe606d15 try fix 'some fetches may stuck' 2021-10-18 23:16:02 +03:00
Nikita Mikhaylov
d004fd96ed
Merge pull request #30318 from azat/fix-shared-build 
Move SquashingTransform to Interpreters (to fix split build)
 2021-10-18 22:41:01 +03:00
alexey-milovidov
885c4daf94
Merge pull request #30191 from azat/aio-O_DIRECT-fix 
Fix pread_fake_async/pread_threadpool with min_bytes_to_use_direct_io
 2021-10-18 22:29:31 +03:00
Azat Khuzhin
e576fd17bd Do not allow reading to empty buffer in MergeTreeReaderStream 
Changelog:
- Verify that all rows was read in MergeTreeReaderWide
- Ignore some exceptions for Wide parts
- Take max_rows_to_read into account
- Verify that requested rows is not zero (otherwise it is too tricky)
- Simply verify that buffer is not empty
 2021-10-18 21:19:34 +03:00
Azat Khuzhin
cd4b33c8c9 Verify that all rows was read in MergeTreeReaderCompact 
v0: Use fmt-like style exception in MergeTreeReaderCompact
v2: Update the check
 2021-10-18 21:19:34 +03:00
Azat Khuzhin
41e6df0f4b Remove unused offset_columns from MergeTreeReaderWide::readRows() 2021-10-18 21:19:34 +03:00
Maksim Kita
3ffca6e138 ExecutablePool dictionary source fix borrow timeout milliseconds 2021-10-18 21:04:40 +03:00
Nikolai Kochetov
34d46245c3 Try fix some tests. 2021-10-18 20:53:30 +03:00
Anton Popov
3e742524b4 update comments 2021-10-18 19:25:25 +03:00
Nikolai Kochetov
ef3597f054 Fix build. 2021-10-18 19:21:49 +03:00
Nikolai Kochetov
7962eddf60 Review fixes. 2021-10-18 19:18:09 +03:00
Ilya Golshtein
d90302aa3b output_format_avro_rows_in_file 2021-10-18 19:01:06 +03:00
feng lv
ab0d2504e5 fix 2021-10-18 15:44:54 +00:00
kssenii
d5d4817350 Add disk proxies back, add setReadUntilPosition to ReadBuffer 2021-10-18 18:35:11 +03:00
Anton Popov
1d9cfc04ef improve performance of aggregation in order of primary key 2021-10-18 18:32:09 +03:00
alesapin
0a838d5926 Remove fuzzers 2021-10-18 18:28:17 +03:00
alesapin
bfe2a937eb At least able to start 2021-10-18 18:27:51 +03:00
kssenii
b7a53df9de Send columns description in clickhouse-local 2021-10-18 14:53:42 +00:00
Dmitry Novik
e9f3cf6651 Fix used memory calculation 2021-10-18 17:49:26 +03:00
Nikolai Kochetov
5a4a752cfb Unite sorting steps. 2021-10-18 17:37:46 +03:00
Nikolay Degterinsky
f861da2dd1 Fix [I]LIKE function 2021-10-18 17:19:02 +03:00
Kruglov Pavel
8cae60c0f6
Fix resetParser 2021-10-18 15:51:40 +03:00
Kruglov Pavel
2a780bf968
Fix build 2021-10-18 15:48:43 +03:00
Vladimir C
4a4700424e
Merge pull request #30230 from vdimir/CLOUDSUPPORT-92764 2021-10-18 14:46:02 +03:00
Kruglov Pavel
dbc2f3408e
Merge branch 'master' into tsv-csv 2021-10-18 14:38:22 +03:00
Kruglov Pavel
6350957709
Fix special build 2021-10-18 14:30:02 +03:00
Kruglov Pavel
003955b59f
Merge branch 'master' into clickhouse-local 2021-10-18 14:24:08 +03:00
vdimir
042e61febf
Allow identifiers staring with numbers in multiple joins 2021-10-18 13:41:40 +03:00
Nikolai Kochetov
6e479b301a Update memory optimisation for MergingSorted. 2021-10-18 12:54:12 +03:00
Anton Popov
e543f720c6 fix clang-tidy 2021-10-18 12:29:34 +03:00
alesapin
6d24ca4c3a Review fixes 2021-10-18 12:13:24 +03:00
Anton Popov
71b474793c
Merge pull request #30174 from CurtizJ/better-interfaces 
Better interfaces for `IDataType` and `ISerialization`
 2021-10-18 12:07:04 +03:00
lhuang0928
37441fe44e Merge remote-tracking branch 'origin/master' 2021-10-18 08:57:25 +00:00
lhuang0928
109e664188 fix date32 comparison with datetime/datetime64 2021-10-18 08:53:25 +00:00
kssenii
1cc511789d Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into fix-local-less-threads 2021-10-18 08:30:59 +00:00
feng lv
81fb4bcf76 fix style 2021-10-18 08:14:54 +00:00
kssenii
3d39ea8e37 Fix WHERE 2021-10-18 11:06:30 +03:00
feng lv
0c628726af fix build 2021-10-18 07:48:58 +00:00
feng lv
1e3df5780c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine 2021-10-18 07:33:11 +00:00
Azat Khuzhin
6d4af3bac1 Move SquashingTransform to Interpreters (to fix split build) 
clickhouse_common_io requires clickhouse_core:

    ld.lld: error: undefined symbol: DB::blocksHaveEqualStructure(DB::Block const&, DB::Block const&)
    >>> referenced by SquashingTransform.cpp:92 (/src/ch/clickhouse/src/Common/SquashingTransform.cpp:92)
    >>>               src/CMakeFiles/clickhouse_common_io.dir/Common/SquashingTransform.cpp.o:(void DB::SquashingTransform::append<DB::Block&&>(DB::Block&&))
    >>> referenced by SquashingTransform.cpp:92 (/src/ch/clickhouse/src/Common/SquashingTransform.cpp:92)
    >>>               src/CMakeFiles/clickhouse_common_io.dir/Common/SquashingTransform.cpp.o:(void DB::SquashingTransform::append<DB::Block const&>(DB::Block const&))

while clickhouse_core requires clickhouse_common_io:

    "clickhouse_core" of type SHARED_LIBRARY
      depends on "roaring" (weak)
      depends on "clickhouse_common_io" (weak)
      depends on "clickhouse_common_config" (weak)
      depends on "clickhouse_common_zookeeper" (weak)
      depends on "clickhouse_dictionaries_embedded" (weak)
      depends on "clickhouse_parsers" (weak)

Follow-up for: #30247 (cc @KochetovNicolai)
 2021-10-18 10:28:36 +03:00
feng lv
6cf82f859c fix 2021-10-18 07:27:37 +00:00
alesapin
f1fe96e194 Merge branch 'master' into debug_keeper 2021-10-18 10:16:07 +03:00
feng lv
a32abda322 fix 2021-10-18 07:08:37 +00:00
feng lv
074e02eb14 fix 2021-10-18 06:38:47 +00:00
kssenii
ab9d5d8cc7 Better 2021-10-18 06:06:38 +00:00
Azat Khuzhin
0e34a9d550 Add MSan instrumentation for preadv2 
MSan report [1]:

    ==10672==WARNING: MemorySanitizer: use-of-uninitialized-value
        3 0x30dd6005 in DB::CompressionCodecFactory::get(unsigned char) const obj-x86_64-linux-gnu/../src/Compression/CompressionFactory.cpp:105:65
        4 0x30dab4d5 in DB::CompressedReadBufferBase::readCompressedData(unsigned long&, unsigned long&, bool) obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferBase.cpp:127:53
        5 0x30db54af in DB::CompressedReadBufferFromFile::readBig(char*, unsigned long) obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferFromFile.cpp:119:38

      Uninitialized value was stored to memory at
        0 0x9ef4e99 in __msan_memcpy (/src/ch/tmp/30191/clickhouse-msan+0x9ef4e99)
        1 0xa0c1999 in DB::ReadBuffer::read() obj-x86_64-linux-gnu/../src/IO/ReadBuffer.h:173:13
        2 0xa0c1999 in DB::ReadBuffer::readStrict() obj-x86_64-linux-gnu/../src/IO/ReadBuffer.h:184:27
        3 0x30daafd1 in DB::CompressedReadBufferBase::readCompressedData() obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferBase.cpp:120:20
        4 0x30db54af in DB::CompressedReadBufferFromFile::readBig() obj-x86_64-linux-gnu/../src/Compression/CompressedReadBufferFromFile.cpp:119:38

      Uninitialized value was created by a heap allocation
        0 0x9ef5285 in posix_memalign (/src/ch/tmp/30191/clickhouse-msan+0x9ef5285)
        1 0xa0cb98d in Allocator<false, false>::allocNoTrack(unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Common/Allocator.h:235:27
        2 0xa0caf0d in Allocator<false, false>::alloc(unsigned long, unsigned long) obj-x86_64-linux-gnu/../src/Common/Allocator.h:96:16
        3 0xa0caf0d in DB::Memory<Allocator<false, false> >::alloc() obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:126:49
        4 0xa0ca4a1 in DB::Memory<Allocator<false, false> >::Memory() obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:43:9
        5 0xa0ca4a1 in DB::BufferWithOwnMemory<DB::SeekableReadBuffer>::BufferWithOwnMemory(unsigned long, char*, unsigned long) obj-x86_64-linux-gnu/../src/IO/BufferWithOwnMemory.h:153:29
        6 0xa0ca4a1 in DB::ReadBufferFromFileBase::ReadBufferFromFileBase() obj-x86_64-linux-gnu/../src/IO/ReadBufferFromFileBase.cpp:11:7
        7 0xa59d980 in DB::AsynchronousReadBufferFromFileDescriptor::AsynchronousReadBufferFromFileDescriptor() obj-x86_64-linux-gnu/../src/IO/AsynchronousReadBufferFromFileDescriptor.h:40:11

    SUMMARY: MemorySanitizer: use-of-uninitialized-value obj-x86_64-linux-gnu/../contrib/libcxx/include/__hash_table:116:10 in std::__1::__constrain_hash(unsigned long, unsigned long)

  [1]: https://clickhouse-test-reports.s3.yandex.net/30191/18bb0e7d8d47dd694390f3f7b2ecc921a167afac/fuzzer_msan/report.html#fail1
 2021-10-17 23:54:58 +03:00
Azat Khuzhin
fb54d6b6e0 BufferWithOwnMemory: take reallocs into account 2021-10-17 23:54:58 +03:00
Azat Khuzhin
348033d8bd BufferWithOwnMemory: make size aligned not capacity 
This will fix pread_fake_async/pread_threadpool with min_bytes_to_use_direct_io
 2021-10-17 23:54:58 +03:00
Azat Khuzhin
9cc1178ebc BufferWithOwnMemory: do not try to align if buffer already aligned 2021-10-17 23:54:57 +03:00
Azat Khuzhin
34810456e1 Fix alignment for prefetch in AsynchronousReadBufferFromFileDescriptor 2021-10-17 23:54:57 +03:00
alexey-milovidov
7beffb04ad
Merge pull request #29495 from ClickHouse/improvement_for_strange_mutations 
Minor improvement for strange mutations
 2021-10-17 20:57:06 +03:00
feng lv
9378b93e41 fix 2021-10-17 14:49:27 +00:00
Kseniia Sumarokova
5633865df1
Update src/Storages/StorageFile.cpp 
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
 2021-10-17 14:16:40 +03:00
Nikolai Kochetov
2a95d7fc87
Merge pull request #29918 from amosbird/projection-improve5 
Add primary key to minmax_count_projection
 2021-10-17 14:04:18 +03:00