thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-18 04:12:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
86,728 Commits 9,324 Branches 1,783 Tags 2.2 GiB
99ad2f5fa6
Commit Graph

24905 Commits

Author SHA1 Message Date
Yakov Olkhovskiy
ff4d295265 style fix 2022-04-03 22:19:35 -04:00
Yakov Olkhovskiy
95ad1bf6e1 use aliases if exist for original_select_set 2022-04-03 22:10:36 -04:00
alesapin
a8c1ccdb20 Forbit to reset non existing settings 2022-04-03 18:29:24 +02:00
Kseniia Sumarokova
30c1afe29d
Merge pull request #35646 from bigo-sg/reduce-mutex-scope 
narrow mutex scope when setenv LIBHDFS3_CONF
 2022-04-03 17:59:24 +02:00
alesapin
7b35920d4c Make more alters of nested types metadata-only 2022-04-03 14:03:34 +02:00
alesapin
91453fe4d6
Merge pull request #35759 from chen9t/fix_dirty_metacache 
Clean dirty meta cache when part is broken during part loading phase
 2022-04-03 13:33:46 +02:00
Azat Khuzhin
3ce4844d4b Properly cancel the query after client format error 
In case of format error (i.e. select 2 format Template settings
format_template_row='/dev/null') the client will reset the connection,
since it will not expect Data blocks.

To fix this, catch this client error and properly cancel the query. This
should also fix query hang checks (the one that executed after each test).

v2: use getCurrentExceptionMessage()/getCurrentExceptionCode()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-03 14:01:16 +03:00
Alexey Milovidov
5a47958744
Merge pull request #35736 from CurtizJ/quota-written-bytes 
Add quota for written bytes
 2022-04-03 05:26:49 +03:00
Azat Khuzhin
195b4c47ea Avoid processing per-column TTL multiple times 
Before this patch ttl.txt will not be written for per-column TTLs, and
hence it will be calculated every time after server restart of
DETACH/ATTACH cycle (note, that it will work w/o restart since in-memory
representation will avoid this).

v2: convert test to .sh to get correct current database over default for MV
v3: extract UUID to avoid error like in [1]:
    [ 490 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) []: Code: 349. DB::Exception: Cannot convert NULL value to non-Nullable type: While processing query_id LIKE concat('%', CAST(_CAST(NULL, 'Nullable(UUID)') AS uuid, 'String'), '%'): while pushing to view test_0hc2ro.this_text_log (c64e5af4-059e-4330-a728-354ecf83c031). (CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN)
    [1]: https://s3.amazonaws.com/clickhouse-test-reports/35820/a512d322b024d37d2f1082c4833f59f86057555f/stateless_tests_flaky_check__address__actions_.html
v4: add no-parallel to avoid issues with disappeared underlying table while pushing to text_log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-02 15:19:39 +03:00
mergify[bot]
1e43e26fa1
Merge branch 'master' into fix-order 2022-04-02 12:00:29 +00:00
Kruglov Pavel
687e9e7aeb
Merge pull request #35760 from Avogar/insert-from-input 
Improve schema inference for insert select from table functions
 2022-04-02 13:50:48 +02:00
Yakov Olkhovskiy
ec0ad8804a style fix 2022-04-01 21:45:58 -04:00
shuchaome
a565a93740 reduce mutex scope when setenv LIBHDFS3_CONF 2022-04-02 06:52:39 +08:00
Yakov Olkhovskiy
0116233d36 allow INTERPOLATE to reference optimized out columns 2022-04-01 16:18:19 -04:00
Anton Popov
a10bf550c4
Update AsynchronousInsertQueue.cpp 2022-04-01 20:57:41 +02:00
tavplubix
97f29ac0fe
Merge pull request #35817 from tcoyvwac/fix/exception-in-destructor-needs-noexcept-false 
Fix missing `noexcept(false)` flag on destructor which causes builds to fail
 2022-04-01 20:35:03 +03:00
Anton Popov
687942ce70 more strict quota for written bytes 2022-04-01 15:02:49 +00:00
Kseniia Sumarokova
fe8c91541d
Merge pull request #35833 from ucasfl/context 
Try remove unneed variable
 2022-04-01 16:58:42 +02:00
Yakov Olkhovskiy
38993f215f
Merge pull request #35706 from yakov-olkhovskiy/ephemeral-35641 
Allow EPHEMERAL without explicit default expression
 2022-04-01 10:49:29 -04:00
kssenii
6c8e073a61 Merge master 2022-04-01 16:48:29 +02:00
tcoyvwac
cb698c4cc1
Replace throw in debug mode in destructor 
FileSegmentsHolder:
- Replaced throw in destructor with assert for all #ifdef modes.
 2022-04-01 15:42:34 +02:00
Alexander Tokmakov
6591d1ceb7 Merge branch 'master' into mvcc_prototype 2022-04-01 15:38:46 +02:00
Antonio Andelic
5bcf772d7c
Merge pull request #35789 from amosbird/better-pipeline1 
Refactoring QueryPipeline
 2022-04-01 14:38:19 +02:00
zzsmdfj
3cae0c74d9 to #34966_fix_dateTime_deserialize 2022-04-01 20:16:07 +08:00
zzsmdfj
ececee3817 to #34966_fix_dateTime_deserialize 2022-04-01 20:13:34 +08:00
Anton Popov
075bcf3f5f
Merge pull request #35803 from Avogar/fix-formats-with-names 
Fix bug in indexes of not presented columns in -WithNames formats
 2022-04-01 13:33:35 +02:00
fenglv
97523a3329 Try remove unneed variable 2022-04-01 09:58:23 +00:00
Kseniia Sumarokova
cafff71d2f
Merge pull request #35799 from evillique/extract-parser-fix 
Fix extract function parser
 2022-04-01 10:59:39 +02:00
alesapin
f2c6387a8d
Merge pull request #35381 from kssenii/remote-filesystem-cache-fixes 
Cache fixes for high concurrency on corner cases
 2022-04-01 10:48:31 +02:00
Kseniia Sumarokova
d3b3294872
Merge pull request #35365 from bigo-sg/improve_access_type 
Improve check access in table functions
 2022-04-01 10:47:02 +02:00
Kseniia Sumarokova
9c39320e75
Merge pull request #35800 from ucasfl/mutex 
remove unused variable
 2022-04-01 10:46:23 +02:00
Kseniia Sumarokova
04729ecc70
Merge pull request #35798 from xiedeyantu/patch-1 
Delete duplicate code
 2022-04-01 10:43:31 +02:00
mergify[bot]
33115a55c3
Merge branch 'master' into better-pipeline1 2022-04-01 06:44:43 +00:00
Antonio Andelic
8cc18c851d
Merge pull request #35705 from ClickHouse/is-secure-client-info 
Added `is_secure` column to `query_log`
 2022-04-01 08:27:20 +02:00
jewisliu
9a4cc78dfb support ALTER TABLE t DETACH PARTITION ALL syntax 2022-04-01 13:13:07 +08:00
Alexey Milovidov
5c6b84c398
Merge pull request #35586 from kssenii/fix-virtual-columns-s3 
Fix s3 engine getting virtual columns
 2022-04-01 04:43:32 +03:00
Alexey Milovidov
475dec3809
Merge pull request #35814 from ClickHouse/merge_tree_arguments_error 
Better error message for incorrect *MergeTree arguments
 2022-04-01 03:04:43 +03:00
kssenii
603dcbb1b7 Extend test 2022-03-31 18:23:15 +02:00
Yakov Olkhovskiy
538373a79b style fix 2022-03-31 12:13:49 -04:00
Yakov Olkhovskiy
a15996315e bugfix - columns order tracking 2022-03-31 11:51:13 -04:00
filimonov
24d72a18d0
Update getNumberOfPhysicalCPUCores.cpp 
In Kubernetes shares are calculated from resources.requests.cpu, quota & period - from limits.

When you configure only requests without limits - the container still can use all the cores. So you see all the cores the system has.
(because of requests the pod just can't be scheduled on the node which have less cpus)
So the effective CPU count is all the available cores if you don't have limits, or calculated from quota & period if you have the limits.
 2022-03-31 17:21:54 +02:00
Alexander Tokmakov
63e8d73ba4 better message for incorrect *MergeTree arguments 2022-03-31 17:08:30 +02:00
Antonio Andelic
1d480178c4
Merge pull request #35785 from justnoise/remove-atomic-flag-init 
remove ATOMIC_FLAG_INIT: depreciated in C++20 and warns in clang-14
 2022-03-31 16:44:56 +02:00
avogar
ab2a963287 Merge branch 'master' of github.com:ClickHouse/ClickHouse into allow-read-bools-as-numbers 2022-03-31 14:09:43 +00:00
Kruglov Pavel
252d66e80d
Update src/Processors/Formats/ISchemaReader.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-03-31 16:08:37 +02:00
mergify[bot]
24ade25d61
Merge branch 'master' into improve-schema-inference 2022-03-31 13:42:47 +00:00
Yakov Olkhovskiy
0367cbad3c messed merge fixed 2022-03-31 09:35:19 -04:00
Anton Popov
42acb1dc29 fix inserts to columns of type Object in partitioned tables 2022-03-31 13:30:01 +00:00
Yakov Olkhovskiy
b5682c1f02 minor refactoring 2022-03-31 08:33:50 -04:00
Kruglov Pavel
a217e9e9d0
Remove unused header 2022-03-31 14:27:43 +02:00
mergify[bot]
3183b61c74
Merge branch 'master' into mvcc_prototype 2022-03-31 12:27:09 +00:00
avogar
836e7dae67 Fix bug in indexes of not presented columns in -WithNames formats 2022-03-31 12:24:40 +00:00
Nikita Mikhaylov
5f27f690f4 Revert "Added support for schema inference for hdfsCluster (#35602)" 
This reverts commit f6bfdcc0c9.
 2022-03-31 14:19:56 +02:00
fenglv
e2e4b02d13 remove unused variable 2022-03-31 12:01:10 +00:00
alesapin
bd2ab32e3f
Merge pull request #35772 from azat/keeper-eof 
clickhouse-keeper: correctly handle closed client connection
 2022-03-31 13:57:28 +02:00
Alexander Tokmakov
461ecd56a4 fix race when database is Ordinary 2022-03-31 13:47:38 +02:00
Nikolay Degterinsky
e022f8527b Fix extract parser 2022-03-31 11:42:35 +00:00
xiedeyantu
d141dbc388
Delete duplicate code 
Delete duplicate code
 2022-03-31 19:34:30 +08:00
Amos Bird
12449caf89
Refactoring QueryPipeline 2022-03-31 19:26:35 +08:00
avogar
d272356324 Minor code improvement 2022-03-31 10:55:09 +00:00
avogar
74275da7ee Make better 2022-03-31 10:52:34 +00:00
Kruglov Pavel
564a77c462
Fix build 2022-03-31 12:49:23 +02:00
Nikita Mikhaylov
f6bfdcc0c9
Added support for schema inference for hdfsCluster (#35602) 2022-03-31 12:47:36 +02:00
alesapin
e12860f646
Update src/Disks/IO/CachedReadBufferFromRemoteFS.cpp 2022-03-31 11:42:42 +02:00
alesapin
4c66c2e10f
Update src/Disks/IO/CachedReadBufferFromRemoteFS.cpp 2022-03-31 11:42:38 +02:00
Alexander Tokmakov
5a50ad9de3 Merge branch 'master' into mvcc_prototype 2022-03-31 11:35:04 +02:00
alesapin
2365bbf0d3
Merge pull request #34566 from ClickHouse/startup-sanity-checks 
add simple sanity checks for server startup
 2022-03-31 11:33:59 +02:00
Kseniia Sumarokova
a711fc0402
Merge pull request #35572 from bigo-sg/hdfs_config_prefix 
Deduce absolute hdfs config path
 2022-03-31 10:08:32 +02:00
Kseniia Sumarokova
0b20329156
Merge pull request #35727 from kssenii/fix-any-all 
Fix any/all(subquery)
 2022-03-31 10:05:37 +02:00
Antonio Andelic
e336302650 Merge branch 'master' into is-secure-client-info 2022-03-31 06:58:16 +00:00
Antonio Andelic
049559d510 Remove extra newline 2022-03-31 06:57:59 +00:00
Antonio Andelic
13b4fabee4
Merge pull request #35743 from bigo-sg/remote_filter_for_hive 
Respect remote_url_allow_hosts for hive
 2022-03-31 08:54:37 +02:00
Antonio Andelic
cdedcee0ce
Merge pull request #35746 from ClickHouse/use-common-io-thread-pool 
Use common IOThreadPool for S3 and URL
 2022-03-31 08:41:54 +02:00
Azat Khuzhin
ff0e46c394 clickhouse-keeper: finish session on client EOF (fixes heap-use-after-free) 
CI founds [1]:

    WARNING: ThreadSanitizer: heap-use-after-free (pid=1)
      Read of size 8 at 0x7b48001e0088 by thread T7 (mutexes: write M2588):
        ...
        1 DB::KeeperTCPHandler::runImpl()::$_0::operator()() const build_docker/../src/Server/KeeperTCPHandler.cpp:384:14 (clickhouse+0x194f974f)
        ...
        8 DB::KeeperDispatcher::setResponse() build_docker/../src/Coordination/KeeperDispatcher.cpp:222:9 (clickhouse+0x1999de82)
        9 DB::KeeperDispatcher::responseThread() build_docker/../src/Coordination/KeeperDispatcher.cpp:161:18 (clickhouse+0x1999d7a7)
        10 DB::KeeperDispatcher::initialize()::$_1::operator()() const build_docker/../src/Coordination/KeeperDispatcher.cpp:273:54 (clickhouse+0x199a55d1)

      Previous write of size 8 at 0x7b48001e0088 by thread T3:
        0 operator delete(void*, unsigned long) <null> (clickhouse+0xa0dad50)
        1 DB::KeeperTCPHandler::~KeeperTCPHandler() build_docker/../src/Server/KeeperTCPHandler.cpp:648:1 (clickhouse+0x194f8ee6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/35772/38873d726d22b15d4c5a284410956927b2e2f95f/integration_tests__thread__actions__[4/4].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-31 09:31:34 +03:00
Brendan Cox
87fff8b2d3 remove ATOMIC_FLAG_INIT: depreciated in C++20 and warns in clang-14 2022-03-30 21:45:48 -07:00
taiyang-li
539ce5f1b8 Merge branch 'master' into use_minmax_index 2022-03-31 12:27:32 +08:00
taiyang-li
a5765dccb1 Merge branch 'master' into function_enumerate_streams 2022-03-31 12:21:00 +08:00
taiyang-li
70087bc959 Merge branch 'master' into improve_access_type 2022-03-31 12:19:17 +08:00
taiyang-li
f6ccda8097 Merge branch 'master' into hdfs_config_prefix 2022-03-31 12:16:05 +08:00
Yakov Olkhovskiy
3b40b56e56 refactored to remove parsers rependency on datatypes 2022-03-30 23:47:16 -04:00
taiyang-li
c11f563d47 Merge branch 'master' into remote_filter_for_hive 2022-03-31 10:54:55 +08:00
chen9t
b40a69f59a Code style 2022-03-31 10:10:05 +08:00
Nikolay Degterinsky
f055d7b692
Merge pull request #34055 from palegre-tiny/groupSortedArray 
Add groupSortedArray() function
 2022-03-31 01:20:15 +03:00
alesapin
81ce991494 Merge branch 'master' into startup-sanity-checks 2022-03-31 00:08:43 +02:00
Kruglov Pavel
4ec3c35e14
Merge pull request #35755 from Avogar/fix-custom-to-string 
Fix bug in conversion from custom types to string
 2022-03-31 00:06:48 +02:00
Nikita Taranov
30f2a942c5
Predict size of hash table for GROUP BY (#33439) 
* use AggregationMethod ctor with reserve

* add new settings

* add HashTablesStatistics

* support queries with limit

* support distributed and with external aggregation

* add new profile events

* add some tests

* add perf test

* export cache stats through AsynchronousMetrics

* rm redundant trace

* fix style

* fix 02122_parallel_formatting test

* review fixes

* fix 02122_parallel_formatting test

* apply also to two-level HTs

* try simpler strategy

* increase max_size_to_preallocate_for_aggregation for experiment

* fixes

* Revert "increase max_size_to_preallocate_for_aggregation for experiment"

This reverts commit 6cf6f75704.

* fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-03-30 22:47:51 +02:00
Yakov Olkhovskiy
6a1e116c46 refactoring 2022-03-30 16:34:19 -04:00
Azat Khuzhin
38873d726d clickhouse-keeper: fix implementation of server with poll() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-30 21:30:12 +03:00
kssenii
98e85dd68e Fix build 2022-03-30 20:27:04 +02:00
kssenii
ac4897f4f9 Fix 2022-03-30 20:16:41 +02:00
Azat Khuzhin
9820230e95 clickhouse-keeper: correctly handle closed client connection 
This will avoid noisy message like:

    DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-30 20:29:04 +03:00
Maksim Kita
e43fdcd7eb Function hasAll added dynamic dispatch for SSE4.2, AVX2 2022-03-30 18:41:34 +02:00
Maksim Kita
8d0a9689e4 Update gatherutils CMakeLists to use X86_INTRINSICS_FLAGS from cpu_features 2022-03-30 18:40:18 +02:00
Maksim Kita
91eec8962f Rename test 2022-03-30 18:39:28 +02:00
shuchaome
738966b6b6 fix filebuffer pos in RemoteReadBuffer 2022-03-31 00:03:45 +08:00
kssenii
c1ae5814a7 Merge master 2022-03-30 17:59:36 +02:00
Alexander Tokmakov
996d57b918 try fix build with gcc 2022-03-30 17:42:47 +02:00
kssenii
099fa0e668 Merge master 2022-03-30 17:33:32 +02:00
kssenii
0fc92fe2aa Get rid of all "remote" mentions 2022-03-30 17:15:29 +02:00
mergify[bot]
4813f0b305
Merge branch 'master' into is-secure-client-info 2022-03-30 15:12:56 +00:00
kssenii
b30948dd53 Relax test 2022-03-30 17:12:23 +02:00
Kruglov Pavel
771f34d793
Merge pull request #35691 from Avogar/fix-matview 
Fix creating materialized view with subquery after server restart
 2022-03-30 14:57:14 +02:00
avogar
849596c7a2 Improve schema inference for insert select queries 2022-03-30 12:44:53 +00:00
chen9t
5cb2301e39 Clean dirty meta cache when part is broken during part loading phase 2022-03-30 20:40:20 +08:00
Antonio Andelic
7bc5731025 Merge branch 'master' into is-secure-client-info 2022-03-30 11:26:05 +00:00
Antonio Andelic
a3ad99be03 Create CallbackRunner in Storage 2022-03-30 11:24:50 +00:00
avogar
af4bfec051 Fix bug in conversion from custom types to string 2022-03-30 11:19:03 +00:00
Maksim Kita
371cdc956a Added input format settings for parsing invalid IPv4, IPv6 addresses as default values 2022-03-30 12:54:19 +02:00
Vladimir C
31c367d3cd
Merge pull request #35651 from amosbird/columntransformerrename 2022-03-30 12:37:30 +02:00
kssenii
98ad3f4a91 Fix tests, rename some remote mentions 2022-03-30 12:21:59 +02:00
lgbo-ustc
08e1fdc6b8 Fixed :a elimate AMBIGUOUS_COLUMN_NAME exception 
elimate AMBIGUOUS_COLUMN_NAME exception in hdfs/s3 cluster when run a query with join
 2022-03-30 17:06:38 +08:00
Antonio Andelic
de2c6684a3 Remove setup and cleanup 2022-03-30 09:00:06 +00:00
Antonio Andelic
1173eab15e Use same IOThreadPool for URL and S3 2022-03-30 08:15:20 +00:00
taiyang-li
16bb4c4ad0 respect remote_url_allow_hosts for hive 2022-03-30 15:33:59 +08:00
Antonio Andelic
e15fccd31b Add is_secure info to HTTP 2022-03-30 06:39:40 +00:00
Antonio Andelic
d14ad227b1 Polish TCP is_secure flag 2022-03-30 06:39:40 +00:00
Antonio Andelic
b982959c4c Fix writting is_secure 2022-03-30 06:39:40 +00:00
Antonio Andelic
893b8a9513 Initial implementation of is_secure 2022-03-30 06:39:40 +00:00
Antonio Andelic
03a09dc914
Merge pull request #35571 from ClickHouse/parallel-s3-downloading 
Parallel S3 downloading
 2022-03-30 08:14:49 +02:00
Antonio Andelic
d85ed8f2a9
Merge pull request #35655 from ClickHouse/exception-compile-time-message-check 
Use compile-time check for `Exception` messages
 2022-03-30 08:11:32 +02:00
taiyang-li
0af6fdb576 fix building 2022-03-30 11:28:21 +08:00
taiyang-li
fcfabf4a0c Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-30 11:23:24 +08:00
taiyang-li
47f3e9330e merge master and fix conflict 2022-03-30 11:06:51 +08:00
taiyang-li
e76ed3acde Merge branch 'master' into improve_access_type 2022-03-30 11:04:57 +08:00
taiyang-li
9ffb42d507 Merge branch 'master' into hdfs_config_prefix 2022-03-30 11:02:31 +08:00
Anton Popov
a842a81aba
Merge pull request #35690 from CurtizJ/flatten-tuple 
Add function `flattenTuple`
 2022-03-30 00:24:36 +02:00
Anton Popov
1cba31c305
Merge pull request #35687 from CurtizJ/dynamic-columns-7 
Fix reading from type object
 2022-03-30 00:21:32 +02:00
Anton Popov
caacc7d385 add quota for written bytes 2022-03-29 18:21:29 +00:00
kssenii
ce9131f905 Fix unit test 2022-03-29 19:57:55 +02:00
kssenii
f0e0d97752 Fix lask of query id in merges 2022-03-29 19:49:42 +02:00
avogar
000f3043e7 Make better 2022-03-29 17:40:07 +00:00
avogar
3fc36627b3 Allow to infer and parse bools as numbers in JSON input formats 2022-03-29 17:37:31 +00:00
kssenii
a2e950bd3b Fix 2022-03-29 17:34:50 +02:00
kssenii
421b1e5a81 Update 2022-03-29 17:33:02 +02:00
avogar
ce97ccbfb9 Improve schema inference for JSONEachRow and TSKV formats 2022-03-29 14:47:51 +00:00
Maksim Kita
2742b88e6c
Merge pull request #27653 from ContentSquare/hasAllAny_SIMD 
Implement HasAll specialization for SSE and AVX2
 2022-03-29 16:35:59 +02:00
Kruglov Pavel
a2fd09e031
Fix style 2022-03-29 16:34:07 +02:00
Alexander Tokmakov
287d858fda Merge branch 'master' into mvcc_prototype 2022-03-29 16:24:12 +02:00
Dmitry Novik
8f935a72d6
Merge pull request #33230 from CurtizJ/read_in_order_max_rows_to_read 
Proper handle of 'max_rows_to_read' in case of reading in order of sorting key
 2022-03-29 15:16:34 +02:00
Antonio Andelic
ef484547bf Fix GCC build 2022-03-29 13:16:11 +00:00
Antonio Andelic
9990abb76a Use compile-time check for Exception messages, fix wrong messages 2022-03-29 13:16:11 +00:00
Yakov Olkhovskiy
46fd69d2bf
Update ParserCreateQuery.h 
style fix
 2022-03-29 09:15:27 -04:00
avogar
97f5033ea9 Fix tests 2022-03-29 13:07:37 +00:00
mergify[bot]
343588de2c
Merge branch 'master' into improve-schema-inference 2022-03-29 13:06:00 +00:00
taiyang-li
67c3c0be3d Merge branch 'master' into improve_access_type 2022-03-29 20:42:57 +08:00
taiyang-li
631352ba4b Merge branch 'master' into hdfs_config_prefix 2022-03-29 20:40:32 +08:00
alesapin
842c0b8ff0 Merge remote-tracking branch 'origin/startup-sanity-checks' into startup-sanity-checks 2022-03-29 14:29:33 +02:00
kssenii
4374415bd7 Fix clang-tidy, style check 2022-03-29 14:20:21 +02:00
Anton Popov
9610139477
Merge pull request #35629 from CurtizJ/dynamic-columns-5 
Support schema inference for type `Object` in format `JSONEachRow`
 2022-03-29 14:17:09 +02:00
alesapin
ed2c461fe1 Merge branch 'master' into startup-sanity-checks 2022-03-29 14:09:20 +02:00
kssenii
7a70e3a6da Fix 2022-03-29 14:08:24 +02:00
Kruglov Pavel
fe60560c03
Merge pull request #35619 from Avogar/check-remote-url-allow-hosts 
Check remote_url_allow_hosts before schema inference in URL engine
 2022-03-29 14:08:02 +02:00
Anton Popov
d42632dc13
Merge pull request #32928 from bigo-sg/rocksdb_metacache 
Speed up parts loading process of MergeTree by metadata cache
 2022-03-29 14:05:22 +02:00
taiyang-li
59727a423e Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-29 19:39:09 +08:00
Alexander Gololobov
bf376ee2f5
Merge pull request #35628 from ClickHouse/make_date 
Implementation of makeDate and makeDate32
 2022-03-29 13:35:26 +02:00
Maksim Kita
a1a4552740
Merge pull request #35184 from DevTeamBK/clang-tidy-issues 
clang-tidy report issues with Medium priority
 2022-03-29 13:19:54 +02:00
Antonio Andelic
f0b4a9fd15 Fix construction of ReadBufferFromS3 2022-03-29 10:56:49 +00:00
kssenii
eab04674fa Fix review comments 2022-03-29 12:38:17 +02:00
taiyang-li
77314246f3 Merge branch 'master' into hdfs_config_prefix 2022-03-29 18:32:59 +08:00
taiyang-li
834ac3a1b9 Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-29 18:32:06 +08:00
Anton Popov
5d8988ad4e
Merge pull request #35645 from CurtizJ/fix-quotas-async-inserts 
Fix usage of quota with async inserts
 2022-03-29 11:54:24 +02:00
Nikolai Kochetov
86690f18ee
Merge pull request #35667 from amosbird/projection-fix2 
Another fix of projection part check
 2022-03-29 10:22:01 +02:00
Yakov Olkhovskiy
ef34f7dbd7 allow EPHEMERAL without expression 2022-03-29 03:54:13 -04:00
Antonio Andelic
362e877529 Merge branch 'master' into parallel-s3-downloading 2022-03-29 07:05:26 +00:00
taiyang-li
db436ad621 Merge branch 'master' into function_enumerate_streams 2022-03-29 11:35:21 +08:00
Amos Bird
3ab8fc1e7b
Fix projection part check 
Fix projection part check when projection part type is different from
parent part type.
 2022-03-29 10:39:07 +08:00
Alexey Milovidov
5e262fba85
Merge pull request #35204 from azat/build-gcc 
Add build with GCC
 2022-03-29 04:55:15 +03:00
kssenii
44157e5c80 Merge master 2022-03-29 02:19:32 +02:00
kssenii
c6b1f86306 Merge with master 2022-03-29 02:03:28 +02:00
Yakov Olkhovskiy
615efa1381 aliases processing fixed 2022-03-28 19:15:53 -04:00
Nikita Mikhaylov
85282f1b4d
Made schema inference work with s3Cluster table function (#35544) 2022-03-29 00:46:35 +02:00
mergify[bot]
bf5a9dcb7a
Merge branch 'master' into groupSortedArray 2022-03-28 20:57:07 +00:00
mergify[bot]
9efc8bda78
Merge branch 'master' into remote-filesystem-cache-fixes 2022-03-28 20:53:29 +00:00
kssenii
1d49a85963 Merge master 2022-03-28 22:51:56 +02:00
Alexander Gololobov
b49993f993 Fixes according to the code review 2022-03-28 22:47:39 +02:00
kssenii
a2013ebe2b Better 2022-03-28 22:40:27 +02:00
Kseniia Sumarokova
9a8e9198e5
Merge pull request #35593 from kssenii/named-collections-fix 
Fix headers with named collections, add compression_method
 2022-03-28 22:10:32 +02:00
avogar
36ea9ef11f Fix creating materialized view with subquery after server restart 2022-03-28 19:52:10 +00:00
Anton Popov
24c0cf86d4 add function 'flattenTuple' 2022-03-28 19:32:12 +00:00
alesapin
7ab8d7178c
Merge pull request #35308 from ClickHouse/clickhouse-keeper 
New package clickhouse-keeper
 2022-03-28 21:23:17 +02:00
Azat Khuzhin
bf4df5c6bb Fix SIGSEGV for build under gcc-11 (due to auto deduction) 
During building with gcc-11 you will got SIGSEGV for building
InterpretersMySQLDDLQuery (and some others), and it is due to endless
recursion:

    (gdb) bt 5
    0  0x00000000010978f2 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=0) at ../../src/gcc/cp/typeck.c:1274
    1  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    2  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    3  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    4  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    (gdb) bt -X
    1397454 0x0000000001097bd9 in structural_comptypes (t1=0x7fde028d1540, t2=0x7fde028d27e0, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    1397455 0x0000000000f2d8b5 in cp_tree_equal (t1=<optimized out>, t2=<optimized out>) at ../../src/gcc/cp/tree.c:4144
    1397456 0x00000000010909cb in template_args_equal (ot=0x7fde028cf578, nt=0x7fde028cfc58, partial_order=<optimized out>) at ../../src/gcc/cp/pt.c:9256
    1397457 0x0000000001090422 in template_args_equal (partial_order=false, nt=0x7fde028cfc58, ot=0x7fde028cf578) at ../../src/gcc/cp/pt.c:9295
    1397458 comp_template_args (oldargs=0x7fde028cf550, newargs=0x7fde028cfc30, oldarg_ptr=0x0, newarg_ptr=0x0, partial_order=false) at ../../src/gcc/cp/pt.c:9285
    1397459 0x00000000010a08f4 in spec_hasher::equal (e1=0x7fde028c95d0, e2=0x7ffd1194e8c0) at ../../src/gcc/cp/pt.c:1726
    1397460 0x0000000001085965 in hash_table<spec_hasher, false, xcallocator>::find_with_hash (this=0x7fde36b7f450, comparable=@0x7ffd1194e8b8: 0x7ffd1194e8c0, hash=<optimized out>) at ../../src/gcc/hash-table.h:936
    1397461 0x0000000001079698 in lookup_template_class_1 (d1=<optimized out>, arglist=0x7fde028cfc30, in_decl=0x0, context=<optimized out>, entering_scope=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:9896
    1397462 0x000000000109f8ef in lookup_template_class (complain=3, entering_scope=1, context=0x7fde27558e40, in_decl=0x0, arglist=0x7fde028cfc08, d1=0x7fde269bcd20) at ../../src/gcc/cp/pt.c:10251
    1397463 tsubst_aggr_type (t=0x7fde269bcd20, args=<optimized out>, complain=3, in_decl=0x0, entering_scope=1) at ../../src/gcc/cp/pt.c:13646
    1397464 0x000000000108f797 in tsubst (t=0x7fde269bcdc8, args=0x7fde028cf7a8, complain=3, in_decl=<optimized out>) at ../../src/gcc/cp/pt.c:16108
    1397465 0x0000000000e61bf2 in rewrite_template_parm (level=1, complain=3, tsubst_args=0x7fde028cf7a8, index=5, olddecl=0x7fde269b5600) at ../../src/gcc/cp/pt.c:28556
    1397466 rewrite_tparm_list(tree_node*, unsigned int, unsigned int, tree_node*, unsigned int, int) [clone .constprop.0] (oldelt=0x7fde269bd190, index=5, targs=0x7fde028cf7a8, targs_index=4, complain=3, level=1) at ../../src/gcc/cp/pt.c:28640
    1397467 0x00000000009f3748 in build_deduction_guide (type=type@entry=0x7fde26e13dc8, ctor=0x7fde269ac300, outer_args=outer_args@entry=0x0, complain=complain@entry=3) at ../../src/gcc/cp/pt.c:28769
    1397468 0x00000000009f444f in ctor_deduction_guides_for (complain=3, tmpl=<optimized out>) at ../../src/gcc/cp/cp-tree.h:842
    1397469 deduction_guides_for (tmpl=<optimized out>, any_dguides_p=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:29282
    1397470 0x00000000008507a8 in do_class_deduction (complain=3, flags=1, init=<optimized out>, tmpl=0x7fde26e0f980, ptype=0x7fde028c7b28) at ../../src/gcc/cp/pt.c:29402
    1397471 do_auto_deduction (type=0x7fde028c7b28, init=<optimized out>, auto_node=<optimized out>, complain=3, context=<optimized out>, outer_targs=<optimized out>, flags=1) at ../../src/gcc/cp/pt.c:29572
    1397472 0x00000000007c9569 in finish_compound_literal (type=<optimized out>, compound_literal=0x7fde028c95b8, complain=3, fcl_context=fcl_functional) at ../../src/gcc/cp/semantics.c:3060
    1397473 0x0000000001123a79 in cp_parser_functional_cast (parser=0x7fde27558da8, type=0x7fde028c7b28) at ../../src/gcc/cp/parser.c:30670
    1397474 0x0000000000fd7873 in cp_parser_postfix_expression (parser=0x7fde27558da8, address_p=<optimized out>, cast_p=<optimized out>, member_access_only_p=<optimized out>, decltype_p=false, pidk_return=0x0) at ../../src/gcc/cp/parser.c:7437
    1397475 0x0000000000fd4ddf in cp_parser_binary_expression (parser=0x7fde27558da8, cast_p=<optimized out>, no_toplevel_fold_p=false, decltype_p=<optimized out>, prec=PREC_NOT_OPERATOR, pidk=<optimized out>) at ../../src/gcc/cp/parser.c:9842
    1397476 0x0000000000fd4595 in cp_parser_assignment_expression (parser=0x7fde27558da8, pidk=<optimized out>, cast_p=<optimized out>, decltype_p=<optimized out>) at ../../src/gcc/cp/parser.c:10146
    1397477 0x0000000000fd3b90 in cp_parser_constant_expression (parser=0x7fde27558da8, allow_non_constant_p=2, non_constant_p=0x7ffd1194f1d7, strict_p=<optimized out>) at ../../src/gcc/cp/parser.c:10449
    1397478 0x0000000000fcfdd5 in cp_parser_initializer_clause (non_constant_p=<optimized out>, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:24253
    1397479 cp_parser_initializer (parser=0x7fde27558da8, is_direct_init=<optimized out>, non_constant_p=<optimized out>, subexpression_p=<optimized out>) at ../../src/gcc/cp/parser.c:24193
    1397480 0x000000000062e5d8 in cp_parser_decomposition_declaration (init_loc=0x7ffd1194f1d8, maybe_range_for_decl=0x7ffd1194f498, decl_specifiers=0x7ffd1194f1f0, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:14734
    1397481 cp_parser_simple_declaration (parser=0x7fde27558da8, function_definition_allowed_p=<optimized out>, maybe_range_for_decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:14393
    1397482 0x000000000109b870 in cp_parser_init_statement (parser=0x7fde27558da8, decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:13420
    1397483 0x00000000010996f0 in cp_parser_for (unroll=0, ivdep=false, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:12708
    1397484 cp_parser_iteration_statement (parser=0x7fde27558da8, if_p=0x0, ivdep=<optimized out>, unroll=<optimized out>) at ../../src/gcc/cp/parser.c:13343
    1397485 0x0000000000fe5c46 in cp_parser_statement (parser=0x7fde27558da8, in_statement_expr=0x0, in_compound=<optimized out>, if_p=0x0, chain=0x0, loc_after_labels=0x0) at ../../src/gcc/cp/parser.c:11718
    1397486 0x0000000000fe15ac in cp_parser_statement_seq_opt (in_statement_expr=<optimized out>, parser=<optimized out>) at ../../src/gcc/cp/parser.c:12201
    1397487 cp_parser_compound_statement (parser=0x7fde27558da8, in_statement_expr=0x0, bcs_flags=<optimized out>, function_body=<optimized out>) at ../../src/gcc/cp/parser.c:12150

Interesting frame is 1397471, from which we can extract location:

    (gdb) p line_table[0].info_ordinary.maps[line_table[0].info_ordinary.cache]
    $54 = {
      <line_map> = {
        start_location = 1396581280
      },
      members of line_map_ordinary:
      reason = LC_RENAME,
      sysp = 0 '\000',
      m_column_and_range_bits = 8,
      m_range_bits = 0,
      to_file = 0x3eb4bb0 "/ch/src/Interpreters/MySQL/InterpretersMySQLDDLQuery.cpp",
      to_line = 46,
      included_from = 0
    }

By replicating SOURCE_LINE() macro from gcc-11 (libcpp/include/line-map.h):

    /* Converts a map and a location_t to source line.  */
    inline linenum_type
    SOURCE_LINE (const line_map_ordinary *ord_map, location_t loc)
    {
      return ((loc - ord_map->start_location)
          >> ord_map->m_column_and_range_bits) + ord_map->to_line;
    }

We got line 154:

    (gdb) p ((input_location-1396581280) >> 8) + 46
    $61 = 154

And this is:

    auto [column_name_and_type, declare_column_ast] = std::tuple{columns_name_and_type.begin(), columns_definition->children.begin()};

After rewriting it, everything works correctly.

Also here is a reproducer for gcc-11 (and no failures for gcc-12, but
gcc-12 has other issues, like [1] and one more for hash table):

    # cat /tmp/test.cpp
    #include <tuple>

    auto multi()
    {
            return std::tuple{1, 1};
    }
    double foo()
    {
            auto [a, b] = multi();
            return a - b;
    }

    # g++-11 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp
    g++-11: internal compiler error: Segmentation fault signal terminated program cc1plus
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-11/README.Bugs> for instructions.
    # g++-12 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp

  [1]: https://reviews.llvm.org/D122598

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-28 22:21:45 +03:00
Kseniia Sumarokova
859af97fc7
Merge pull request #35614 from kssenii/fix-materialized-postgresql 
Fix possible segfault in materialised postgresql
 2022-03-28 21:11:44 +02:00
Anton Popov
6cbdc6af00 remove obsolete parameter 2022-03-28 18:44:53 +00:00
Vladimir C
ecdd89e811
Merge pull request #35666 from vdimir/fix-fuse 2022-03-28 20:40:43 +02:00
Alexander Tokmakov
208b242188 Merge branch 'master' into mvcc_prototype 2022-03-28 19:58:06 +02:00
Anton Popov
d677635cd8
Merge pull request #35592 from CurtizJ/dynamic-columns-4 
Add parallel parsing and schema inference for format `JSONAsObject`
 2022-03-28 19:29:55 +02:00
Anton Popov
0e4af89f69 fix reading from type object 2022-03-28 17:23:34 +00:00
Alexander Gololobov
93021e387e Implementation of makeDate and makeDate32 2022-03-28 18:57:24 +02:00
Anton Popov
bbfe8a2ca7 fix possible loss of subcolumns in type Object 2022-03-28 15:36:05 +00:00
mergify[bot]
f09ebea2d8
Merge branch 'master' into read_in_order_max_rows_to_read 2022-03-28 13:30:34 +00:00
vdimir
a216bc26c1
Correct check asof join key nullability 2022-03-28 13:29:34 +00:00
taiyang-li
38702be40c Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-28 21:03:22 +08:00
mergify[bot]
70b339ea4e
Merge branch 'master' into check-remote-url-allow-hosts 2022-03-28 11:55:48 +00:00
mergify[bot]
3924609ab2
Merge branch 'master' into metric-async-insert-bytes 2022-03-28 11:37:49 +00:00
vdimir
1122db89db
Use float devision for avg after optimize_fuse_sum_count_avg 2022-03-28 11:26:42 +00:00
mergify[bot]
a022de4a6e
Merge branch 'master' into fix-materialized-postgresql 2022-03-28 10:55:41 +00:00
tavplubix
8de655bf1f
Merge pull request #35621 from ClickHouse/update_comment_mismatching_checksums 
Update comment about mismatching checksums
 2022-03-28 13:42:12 +03:00
Amos Bird
6003a49a87
Fix style 2022-03-28 17:48:47 +08:00
alesapin
77e700b1cf
Update src/Compression/CompressionFactory.cpp 2022-03-28 11:25:08 +02:00
Antonio Andelic
d9d826c813 Address PR review 2022-03-28 08:19:23 +00:00
Amos Bird
58a78fba12
formatString bug fix and refactoring 2022-03-28 15:47:48 +08:00
kssenii
c4832f7633 Fix 2022-03-28 09:28:02 +02:00
Kseniia Sumarokova
482a5e362b
Merge pull request #35620 from kssenii/fix-positional-arguments-with-aliases 
Fix positional arguments
 2022-03-28 09:18:49 +02:00
Antonio Andelic
cbe8e747b4 Merge branch 'master' into parallel-s3-downloading 2022-03-28 07:04:45 +00:00
Anton Popov
4ddfff3084 fix usage of quota with async inserts 2022-03-28 00:46:35 +00:00
Alexey Milovidov
6fbf5bd9ad More correct 2022-03-28 02:11:27 +02:00
Alexey Milovidov
bb35184da1 Add metric about size of async INSERTs 2022-03-28 02:04:19 +02:00
Yakov Olkhovskiy
5a4694f340 major refactoring, simplified, optimized, bugs fixed 2022-03-27 14:32:09 -04:00
Azat Khuzhin
13f2b2c0a3 Use proper type for RadixSortTraits in RowRefs.cpp 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-27 11:32:51 +03:00
mergify[bot]
0db8ae0e42
Merge branch 'master' into update_comment_mismatching_checksums 2022-03-27 00:23:39 +00:00
Azat Khuzhin
6a3986f41b Fix redundant-move in ReadWriteBufferFromHTTP.h 2022-03-26 23:42:46 +03:00
Azat Khuzhin
d518614524 Fix multi-line comment in threadPoolCallbackRunner.cpp 2022-03-26 23:42:46 +03:00
Azat Khuzhin
55e9f60448 Fix building of SerializationObject 
gcc11 reports:

    ld.lld: error: undefined symbol: void DB::readEscapedStringInto<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, DB::ReadBuffer&)
    >>> referenced by SerializationObject.cpp:201 (./src/DataTypes/Serializations/SerializationObject.cpp:201)
    >>>               SerializationObject.cpp.o:(void DB::SerializationObject<DB::JSONDataParser<DB::SimdJSONParser> >::deserializeTextImpl<DB::SerializationObject<DB::JSONDataParser<DB::SimdJSONParser> >::deserializeTextEscaped(DB::IColumn&, DB::ReadBuffer&, DB::FormatSettings const&) const::'lambda'(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&)>(DB::IColumn&, DB::SerializationObject<DB::JSONDataParser<DB::SimdJSONParser> >::deserializeTextEscaped(DB::IColumn&, DB::ReadBuffer&, DB::FormatSettings const&) const::'lambda'(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&)&&) const) in archive src/libdbms.a
    collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-26 23:42:46 +03:00
Azat Khuzhin
be4fea335c Fix non-existing include dir (rapidjson) 
gcc reports:

    Mar 21 16:10:33 cc1plus: error: ../src/ch_contrib::rapidjson: No such file or directory [-Werror=missing-include-dirs]

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-26 23:42:46 +03:00
Azat Khuzhin
a847b71611 Fix SubcolumnsTree declaration 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-26 23:42:46 +03:00
Azat Khuzhin
6da2c48b7a Add missing __builtin_unreachable for switch/case for gcc 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-26 23:42:21 +03:00
Anton Popov
a6450be8b6 fix schema inference 2022-03-26 01:33:10 +00:00
mergify[bot]
93b3b46caf
Merge branch 'master' into fix-lowcardinality-hash-join 2022-03-25 22:56:59 +00:00
Anton Popov
67195bfdd5 support schema inference for type Object in format JSONEachRow 2022-03-25 21:51:53 +00:00
alesapin
14e84ff6ef
Merge pull request #34802 from ClickHouse/overcommit-doc 
Add documentation for memory overcommit
 2022-03-25 20:50:44 +01:00
kssenii
d8960cde62 Fix positional order/etc by literal 2022-03-25 20:03:12 +01:00
kssenii
c6706b8bfa Better 2022-03-25 19:30:13 +01:00
kssenii
141c290344 Fix build 2022-03-25 18:31:15 +01:00
mergify[bot]
1dcbfe8739
Merge branch 'master' into clickhouse-keeper 2022-03-25 17:25:50 +00:00
kssenii
e59dbe72c7 Revert accidental commit 2022-03-25 17:53:02 +01:00
taiyang-li
b79cec6806 Merge branch 'use_minmax_index' of https://github.com/bigo-sg/ClickHouse into use_minmax_index 2022-03-25 23:33:49 +08:00
taiyang-li
eee8949150 fix code 2022-03-25 23:33:46 +08:00
kssenii
9aaba43a0a Remove incorrect assertion 2022-03-25 16:29:30 +01:00
Antonio Andelic
e27a68ef8c
Merge pull request #35576 from ClickHouse/fix-casewithexpression-type-deduction 
Fix `caseWithExpression` return type deduction
 2022-03-25 16:01:31 +01:00
Vladimir C
cfb12aff6f
Merge pull request #35460 from helifu/master 2022-03-25 15:55:17 +01:00
Alexander Tokmakov
9ec0951de5 update comment for mismatching checksums 2022-03-25 15:54:05 +01:00
taiyang-li
4aaa361f2e Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-25 22:48:03 +08:00
taiyang-li
7889059f7d fix building 2022-03-25 22:39:57 +08:00
taiyang-li
4788006030 Merge branch 'master' into function_enumerate_streams 2022-03-25 22:31:10 +08:00
taiyang-li
fbadd8fc0e only deduce absolute path of hdfs config when it's relative path do not exists 2022-03-25 22:28:47 +08:00
taiyang-li
c013cd0b52 Merge branch 'master' into hdfs_config_prefix 2022-03-25 22:13:45 +08:00
taiyang-li
3ca487030d Merge branch 'master' into improve_access_type 2022-03-25 22:11:08 +08:00
kssenii
5103aefd34 Fix 2022-03-25 14:33:54 +01:00
avogar
ecc57b436c Check remote_url_allow_hosts before schema inference in URL engine 2022-03-25 13:30:45 +00:00
Antonio Andelic
394682b983 Merge branch 'master' into parallel-s3-downloading 2022-03-25 13:00:44 +00:00
avogar
6fb3c3be04 Fix comments and build 2022-03-25 12:02:21 +00:00
tavplubix
ce99b95230
Merge pull request #35594 from ClickHouse/fix_stupid_bug 
Fix stupid bug in ATTACH TABLE
 2022-03-25 14:38:27 +03:00
Antonio Andelic
e76074ef7f Merge branch 'master' into fix-casewithexpression-type-deduction 2022-03-25 11:12:38 +00:00
Antonio Andelic
d7fb51269a Merge branch 'master' into fix-lowcardinality-hash-join 2022-03-25 11:12:31 +00:00
Antonio Andelic
9e11e611c7 Match cardinality when adding columns 2022-03-25 11:12:02 +00:00
Kruglov Pavel
d45143ffe0
Merge branch 'master' into improve-schema-inference 2022-03-25 12:05:40 +01:00
Vladimir C
ae92963b15
Fix build error in Formats/ISchemaReader.cpp 2022-03-25 11:30:25 +01:00
alesapin
c95a9971bc Merge branch 'master' into clickhouse-keeper 2022-03-25 11:22:57 +01:00
Antonio Andelic
f6439efcad
Merge pull request #35459 from ClickHouse/case-insensitive-column-matching 
Support for case insensitive column matching for ORC/Arrow/Parquet files
 2022-03-25 08:09:29 +01:00
taiyang-li
dd0873189d rename all files 2022-03-25 14:54:01 +08:00
taiyang-li
c9acc55008 rename function name 2022-03-25 14:51:59 +08:00
mergify[bot]
6267ba4445
Merge branch 'master' into fix_stupid_bug 2022-03-25 01:30:48 +00:00
Kruglov Pavel
2f2d3cdbca
Merge pull request #35578 from ClickHouse/Avogar-patch-3 
Increase fiber stack size a bit in attempt to fix stack overflow in tests with address sanitizer
 2022-03-25 00:33:05 +01:00
Nikita Mikhaylov
64f79f0c69
Added an ability to specify cluster secret in replicated database (#35333) 2022-03-25 00:14:26 +01:00
Maksim Kita
7b03617341
Merge pull request #35539 from kitaisreal/parse-ipv6-long-fix 
Fix parsing of IPv6 addresses longer than 39 characters
 2022-03-24 22:28:51 +01:00
kssenii
0c6577eee0 Merge master 2022-03-24 20:49:02 +01:00
kssenii
6769b84638 Merge master 2022-03-24 20:47:43 +01:00
Yakov Olkhovskiy
adefcfd299
Merge branch 'master' into interpolate-feature 2022-03-24 15:33:09 -04:00
Yakov Olkhovskiy
83f406b722 optimization, INTERPOLATE without expr. list, any column is allowed except WITH FILL 2022-03-24 15:29:29 -04:00
Alexander Tokmakov
7c0bdbfa93 fix stupid bug 2022-03-24 20:24:05 +01:00
Kruglov Pavel
1823cac89d
Update src/Formats/EscapingRuleUtils.h 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-03-24 19:19:32 +01:00
Kruglov Pavel
287e1a6efc
Update src/Processors/Formats/ISchemaReader.cpp 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-03-24 19:16:52 +01:00
Kruglov Pavel
6a9df9d471
Update src/Processors/Formats/ISchemaReader.cpp 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-03-24 19:16:47 +01:00
Kruglov Pavel
3b801a4093
Update src/Processors/Formats/ISchemaReader.cpp 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-03-24 19:16:41 +01:00
kssenii
d9d9f3bc6d Fix 2022-03-24 19:10:53 +01:00
Anton Popov
78100abc5f add parallel parsing and schema inference for type Object 2022-03-24 17:51:35 +00:00
Dmitry Novik
56d20e00cc
Update src/Common/OvercommitTracker.cpp 
Co-authored-by: alesapin <alesapin@clickhouse.com>
 2022-03-24 17:47:48 +01:00
kssenii
e91e30aaa4 Fix 2022-03-24 17:10:27 +01:00
kssenii
dad4e5b0c5 Fix 2022-03-24 17:10:04 +01:00
avogar
151067a3f9 Merge branch 'master' of github.com:ClickHouse/ClickHouse into Avogar-patch-3 2022-03-24 15:32:07 +00:00
mergify[bot]
52099b23a1
Merge branch 'master' into case-insensitive-column-matching 2022-03-24 15:24:46 +00:00
Sergei Trifonov
a90e83665d
Merge pull request #35343 from ClickHouse/parallel-multipart-upload-for-s3storage 
allow parallelization of multipart S3 storage upload; fixes #34350
 2022-03-24 15:58:29 +01:00
Alexey Milovidov
66a6352378
Merge pull request #35556 from ClickHouse/revert-35524-fix-stack-overflow 
Revert "Decrease data_type_max_parse_depth a little to avoid stack overflow in coroutines"
 2022-03-24 17:57:24 +03:00
Alexey Milovidov
31420615c0
Merge pull request #35553 from ClickHouse/kostik-consistent-hash 
Rename `yandexConsistentHash` to `kostikConsistentHash`
 2022-03-24 17:56:59 +03:00
Kruglov Pavel
f07918c659
Increase stack size 2022-03-24 15:50:32 +01:00
kssenii
f9b538fb0c Fix 2022-03-24 15:38:29 +01:00
kssenii
d6ab6de30a Minor changes 2022-03-24 15:32:08 +01:00
Sergei Trifonov
2e2a1c06a2 Merge branch 'startup-sanity-checks' of github.com:ClickHouse/ClickHouse into startup-sanity-checks 2022-03-24 15:30:59 +01:00
Sergei Trifonov
041e348076 cosmetic bugfix 2022-03-24 15:30:10 +01:00
mergify[bot]
394fdb4fb8
Merge branch 'master' into startup-sanity-checks 2022-03-24 14:23:38 +00:00
Antonio Andelic
34b2307259 Convert columns to full when multiple disjunct used 2022-03-24 13:56:00 +00:00
taiyang-li
c0eb9f73a9 remote std::cout 2022-03-24 21:17:43 +08:00
avogar
abc020a502 Clean up 2022-03-24 13:08:58 +00:00
avogar
557edbd172 Add some improvements and fixes in schema inference 2022-03-24 12:54:12 +00:00
Kruglov Pavel
98be162d20
Update comment 2022-03-24 13:49:26 +01:00
Kruglov Pavel
b16c5916a0
Merge pull request #35507 from ClickHouse/Avogar-patch-2 
Add more validations in mask operations
 2022-03-24 13:06:15 +01:00
Kruglov Pavel
37286c6141
Increase fiber stack size a bit in attempt to fix stack overflow in tests with address sanitizer 2022-03-24 12:55:56 +01:00
vdimir
f106e2dd49
fix style in QueryPlan.cpp 2022-03-24 11:53:58 +00:00
tavplubix
7b96d61057
Merge pull request #35478 from ClickHouse/merging_30325 
Merging #30325
 2022-03-24 14:53:20 +03:00
vdimir
d16ae46589
remove description for ReadFromMergeTree from pipeline, adjust tests for plan 2022-03-24 11:31:52 +00:00
Antonio Andelic
fbba450cff Take ELSE branch into account for result type deduction 2022-03-24 11:23:18 +00:00
kssenii
142a5938a1 Merge master 2022-03-24 12:10:26 +01:00
tavplubix
ee611a9af4
Merge pull request #35503 from ClickHouse/fix_assertion_ddl_worker 
Fix too strict assertion in DDLWorker
 2022-03-24 13:57:07 +03:00
helifu
8a5bd2defa
Add explicit table info to the scan node of query plan and pipeline 
:) explain plan select * from table1 t1 left join table2 t2 on t1.name = t2.name;
┌─explain──────────────────────────────────────────────────────────────────────────────────────┐
│ Expression ((Projection + Before ORDER BY))                                                  │
│   Join (JOIN)                                                                                │
│     Expression (Before JOIN)                                                                 │
│       SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│         ReadFromMergeTree (default.table1)                                                   │
│     Expression ((Joined actions + (Rename joined columns + (Projection + Before ORDER BY)))) │
│       SettingQuotaAndLimits (Set limits and quota after reading from storage)                │
│         ReadFromMergeTree (default.table2)                                                   │
└──────────────────────────────────────────────────────────────────────────────────────────────┘

:) explain pipeline select * from table1 t1 left join table2 t2 on t1.name = t2.name;
┌─explain──────────────────────────────────────────┐
│ (Expression)                                     │
│ ExpressionTransform × 24                         │
│   (Join)                                         │
│   JoiningTransform × 24 2 → 1                    │
│     Resize 1 → 24                                │
│       FillingRightJoinSide                       │
│         Resize 24 → 1                            │
│           (Expression)                           │
│           ExpressionTransform × 24               │
│             (SettingQuotaAndLimits)              │
│               (ReadFromMergeTree default.table1) │
│               MergeTreeThread × 24 0 → 1         │
│           (Expression)                           │
│           ExpressionTransform × 24               │
│             (SettingQuotaAndLimits)              │
│               (ReadFromMergeTree default.table2) │
│               MergeTreeThread × 24 0 → 1         │
└──────────────────────────────────────────────────┘
 2022-03-24 10:49:12 +00:00
alesapin
d3fbd9b2ad Fix style 2022-03-24 11:25:28 +01:00
alesapin
93d1b1d198 Merge branch 'master' into startup-sanity-checks 2022-03-24 11:17:06 +01:00
taiyang-li
6eccd09e2b deduce hdfs config path 2022-03-24 18:04:29 +08:00
Antonio Andelic
62a495a5fc Small refactor 2022-03-24 09:45:30 +00:00
Nikolai Kochetov
283e20a9a5
Merge pull request #35395 from amosbird/distributedmultiplejoin 
Validate some thoughts over making sets
 2022-03-24 10:30:26 +01:00
Antonio Andelic
c52cee2219 Merge branch 'master' into parallel-s3-downloading 2022-03-24 08:11:52 +00:00
mergify[bot]
bf90edc362
Merge branch 'master' into case-insensitive-column-matching 2022-03-24 08:00:42 +00:00
Alexey Milovidov
8bf8632cd8
Revert "Decrease data_type_max_parse_depth a little to avoid stack overflow in coroutines" 2022-03-24 07:01:20 +03:00
taiyang-li
95dd4a5c14 push again 2022-03-24 11:32:35 +08:00
Alexey Milovidov
31eeeeb428 Add compatibility alias 2022-03-24 02:21:57 +01:00
Alexey Milovidov
d54138425f Rename yandexConsistentHash to kostikConsistentHash 2022-03-24 02:18:25 +01:00
Maksim Kita
4b88c6f934
Merge pull request #35534 from kitaisreal/set-in-ipv4-ipv6-address-cast-fix 
Fix cast into IPv4, IPv6 address in IN section
 2022-03-24 01:26:25 +01:00
Maksim Kita
1df1721648
Merge pull request #35546 from kitaisreal/refactor-external-models-loader 
ExternalModelsLoader refactoring
 2022-03-24 01:24:45 +01:00
Kruglov Pavel
9ae52910f0
Fix style 2022-03-23 23:15:07 +01:00
Kruglov Pavel
44b6ea5564
Merge pull request #35524 from Avogar/fix-stack-overflow 
Decrease data_type_max_parse_depth a little to avoid stack overflow in coroutines
 2022-03-23 23:13:08 +01:00
kssenii
d4161b5925 Add optin read_from_cache_if_exists_otherwise_bypass_cache (for merges) 2022-03-23 20:24:00 +01:00
Maksim Kita
3df6a86387 ExternalModelsLoader refactoring 2022-03-23 20:05:17 +01:00
Maksim Kita
31b3d2e840 Fixed tests 2022-03-23 19:59:26 +01:00
mergify[bot]
b745b67026
Merge branch 'master' into parse-ipv6-long-fix 2022-03-23 18:56:40 +00:00
mergify[bot]
abbed251c1
Merge branch 'master' into asof-join-improve-performance 2022-03-23 18:53:17 +00:00
Sergei Trifonov
c7cf14e743 fix style and darwin build 2022-03-23 19:15:01 +01:00
kssenii
d2a3cfe5dc Cache on all write operations 2022-03-23 19:14:33 +01:00
mergify[bot]
881d5721a0
Merge branch 'master' into Avogar-patch-2 2022-03-23 17:38:43 +00:00
mergify[bot]
853bea62db
Merge branch 'master' into fix-stack-overflow 2022-03-23 18:37:18 +01:00
Kruglov Pavel
826b933b08
Merge pull request #35332 from Avogar/fix-tskv-schema-inference 
Fix schema inference for TSKV format while using small max_read_buffer_size
 2022-03-23 18:37:07 +01:00
Yakov Olkhovskiy
1631656803
Merge pull request #34353 from zvonand/issue_33147 
Extended usage of Milliseconds, Microseconds, Nanoseconds
 2022-03-23 13:33:07 -04:00
kssenii
e39aba37a2 Minor changes 2022-03-23 18:11:52 +01:00
Alexander Tokmakov
9aed0507b7 Merge branch 'master' into mvcc_prototype 2022-03-23 18:07:22 +01:00
Alexander Tokmakov
3c6a10237c Merge branch 'master' into merging_30325 2022-03-23 18:07:04 +01:00
mergify[bot]
13bc93c171
Merge branch 'master' into fix_assertion_ddl_worker 2022-03-23 17:06:33 +00:00
Sergei Trifonov
bf9c6be6c6 move threadPoolCallbackRunner from Common into Interpreters folder to fix split build issue 2022-03-23 17:57:19 +01:00
Antonio Andelic
cbd20af706 Merge branch 'master' into parallel-s3-downloading 2022-03-23 16:35:05 +00:00
Antonio Andelic
e9520335df Merge branch 'master' into case-insensitive-column-matching 2022-03-23 16:20:12 +00:00
Antonio Andelic
8561c366be Move nested table name 2022-03-23 16:19:59 +00:00
kssenii
c2c7f365ce Merge master 2022-03-23 16:58:35 +01:00
kssenii
8c1be8950e Improve test 2022-03-23 16:55:18 +01:00
Antonio Andelic
052057f2ef Address PR comments 2022-03-23 15:42:46 +00:00
Amos Bird
ab7923a26c
Remove comments 2022-03-23 23:21:02 +08:00
Maksim Kita
df0d3c9304 Fix parsing of IPv6 addresses longer than 39 characters 2022-03-23 16:11:46 +01:00
kssenii
34c0690a69 Better 2022-03-23 15:43:14 +01:00
Antonio Andelic
08cb71a067 Merge branch 'master' into case-insensitive-column-matching 2022-03-23 14:37:05 +00:00
Antonio Andelic
1a5f5c32bb Remove commented out code 2022-03-23 14:36:25 +00:00
Maksim Kita
ff2d5dae83 Fix cast into IPv4, IPv6 address in IN section 2022-03-23 15:22:08 +01:00
Maksim Kita
da235f9cda
Merge pull request #34536 from kitaisreal/system-log-tables-comment-fix 
System log tables allow to specify COMMENT in ENGINE declaration
 2022-03-23 15:06:05 +01:00
李扬
9cc528b01f
Update HiveFile.h 2022-03-23 21:57:58 +08:00
taiyang-li
07bf891c85 rm useless file 2022-03-23 21:56:01 +08:00
taiyang-li
f84a88e48f push again 2022-03-23 21:41:42 +08:00
Maksim Kita
561ce88ff3 Fixed tests 2022-03-23 14:25:21 +01:00
xiedeyantu
9441111833
fix INSERT INTO table FROM INFILE does not display progress bar (#35429) 
* fix INSERT INTO table FROM INFILE does not display progress bar

* Update StorageFile.cpp

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-03-23 14:24:59 +01:00
kssenii
f3e1ca44a9 Add system.remote_data_paths table 2022-03-23 13:01:18 +01:00
Antonio Andelic
f4b07fd809 Merge branch 'master' into parallel-s3-downloading 2022-03-23 11:53:01 +00:00
Antonio Andelic
93ad209910 Attach memory tracker 2022-03-23 11:52:31 +00:00
Maksim Kita
3c5c267c17 Improve performance of ASOF join 2022-03-23 12:19:38 +01:00
Alexander Tokmakov
94d32c6b80 Merge branch 'master' into merging_30325 2022-03-23 11:58:57 +01:00
Alexander Tokmakov
e8b7f963b7 fix too strict assertion 2022-03-23 11:55:28 +01:00
Maksim Kita
21735db126 Fixed style check 2022-03-23 11:52:37 +01:00
avogar
99cf1560da Decrease data_type_max_parse_depth a little to avoid stack overflow in coroutines 2022-03-23 10:50:55 +00:00
Kruglov Pavel
c4713a9159
Update MaskOperations.cpp 2022-03-23 11:39:38 +01:00
Maksim Kita
1f940f9e3b
Merge pull request #35512 from CurtizJ/fix-fuzzer 
Fix crash with enabled `optimize_functions_to_subcolumns`
 2022-03-23 11:31:32 +01:00
Alexander Tokmakov
bcec53aadc Merge branch 'master' into mvcc_prototype 2022-03-23 11:28:43 +01:00
Maksim Kita
a03bfa0fae Fixed style check 2022-03-23 10:46:27 +01:00
Antonio Andelic
131b3a091c Refactor StorageS3 2022-03-23 08:57:01 +00:00
Antonio Andelic
f32ef2a556 Small polishing for S3 reader 2022-03-23 08:15:18 +00:00
Antonio Andelic
bb6b9f3799 Merge branch 'parallel-downloading-url-engine' into parallel-s3-downloading 2022-03-23 07:46:14 +00:00
Antonio Andelic
8dc3c6901d Merge branch 'master' into case-insensitive-column-matching 2022-03-23 07:42:21 +00:00
Antonio Andelic
9829f2d051 Merge branch 'master' into parallel-downloading-url-engine 2022-03-23 07:41:14 +00:00
taiyang-li
ae3d55c6a2 merge master and fix conflict 2022-03-23 14:31:12 +08:00
taiyang-li
6fe4c3d17e some minor changes 2022-03-23 12:13:42 +08:00
taiyang-li
8dbf1c60e7 merge master and fix conflict 2022-03-23 11:36:50 +08:00
taiyang-li
68d5b538aa fix build error 2022-03-23 11:15:42 +08:00
Anton Popov
4ff9627f60 fix crash with enabled optimize_functions_to_subcolumns 2022-03-23 01:27:52 +00:00
Mikhail f. Shiryaev
71fb04ea4a
Merge pull request #35466 from ClickHouse/black 
Check python black formatting
 2022-03-22 23:53:02 +01:00
Maksim Kita
1a096a87da Added support for SystemLog to specify COMMENT in ENGINE section 2022-03-22 22:44:46 +01:00
Kruglov Pavel
9bf1cb6636
Fix 2022-03-22 20:15:05 +01:00
Alexander Tokmakov
7e904ac789 more logs 2022-03-22 19:05:37 +01:00
Kruglov Pavel
763a80e405
Add more validations in mask operations 2022-03-22 18:46:59 +01:00
Kruglov Pavel
ea62f83842
Merge pull request #35502 from kitaisreal/short-circuit-short-column-crash-fix 
Fix crash during short circuit function evaluation
 2022-03-22 18:43:00 +01:00
Mikhail f. Shiryaev
e6f5a3f98b
Apply black formatter to all *.py files in the repo 2022-03-22 17:39:58 +01:00
Anton Popov
93dc5c0b81
Merge pull request #35488 from CurtizJ/add-subcolumns-sizes 
Add sizes of subcolumns to `system.parts_columns` table
 2022-03-22 16:53:23 +01:00
Alexander Tokmakov
bc5a689899 add debug logging 2022-03-22 15:34:20 +01:00
tavplubix
dbf3e0e767
Merge pull request #35498 from ClickHouse/fix_keeper_timeout_bug 
Fix keeper client timeout bug
 2022-03-22 17:33:29 +03:00
Anton Popov
f693eba568 fix tests with approx rows 2022-03-22 14:30:40 +00:00
Maksim Kita
d24795405c Fixed tests 2022-03-22 15:12:43 +01:00
Antonio Andelic
6785ad165a Fix issue for mutliple download threads 2022-03-22 14:06:30 +00:00
alesapin
a4ab73619f
Fix UBSan build 2022-03-22 15:05:31 +01:00
alesapin
0ab9a9c0f6
Merge pull request #33664 from ClickHouse/release-steps 
Refactor releasing process
 2022-03-22 14:43:18 +01:00
Maksim Kita
fcbf97ba68 Fix crash during short circuit function evaluation 2022-03-22 14:10:47 +01:00
Antonio Andelic
df99a9cc3c Merge branch 'parallel-downloading-url-engine' into parallel-s3-downloading 2022-03-22 12:49:35 +00:00
Maksim Kita
0576c1b0f3
Merge pull request #35500 from kitaisreal/throw-if-constant-argument-crash-fix 
Fix crash for function throwIf with constant arguments
 2022-03-22 13:44:37 +01:00
Antonio Andelic
5da3058368 Merge branch 'master' into parallel-downloading-url-engine 2022-03-22 12:20:36 +00:00
Maksim Kita
22112feef3 Fix crash for function throwIf with constant arguments 2022-03-22 12:30:25 +01:00
Antonio Andelic
6b6190554b Fix conversion of arrow to CH column with hint header 2022-03-22 11:15:48 +00:00
Kruglov Pavel
594fa8f449
Merge pull request #35476 from Avogar/decimal-logical-error 
Fix bug in function if that led to logical error
 2022-03-22 12:12:24 +01:00
Antonio Andelic
0c23cd7b94 Add support for case insensitive column matching in arrow 2022-03-22 10:55:10 +00:00
alesapin
70d410dc4d Fix keeper client timeout bug 2022-03-22 11:41:50 +01:00
kssenii
d8739318ea Fix bug 2022-03-22 11:22:57 +01:00
alesapin
bad78f0daa
Don't unwind inlines if we unable to do it 2022-03-22 11:11:36 +01:00
Antonio Andelic
ca7844e338 Fix tests 2022-03-22 09:27:20 +00:00
Antonio Andelic
246c66bf3e
Merge pull request #35393 from bigo-sg/hive_column_pruning_bug 
Fix erasing nonexistent column
 2022-03-22 08:59:33 +01:00
Antonio Andelic
6cebb6bc88 Merge branch 'master' into case-insensitive-column-matching 2022-03-22 07:36:35 +00:00
Anton Popov
0c210a831c add sizes of subcolumns to system.parts_columns table 2022-03-21 21:12:26 +00:00
kssenii
a9ba14e386 Fix checks 2022-03-21 21:53:46 +01:00
kssenii
ccdf7d0231 Fix typos check 2022-03-21 21:51:17 +01:00
mergify[bot]
9a505476a5
Merge branch 'master' into mvcc_prototype 2022-03-21 20:33:07 +00:00
Maksim Kita
ce2883a13d
Merge pull request #35462 from kitaisreal/mysqlxx-pool-with-failover-style-fix 
mysqlxx PoolWithFailover style fix
 2022-03-21 20:30:55 +01:00
tavplubix
2fdc937ae1
Merge pull request #35480 from ClickHouse/fix_34586 
Fix show create for information_schema
 2022-03-21 22:20:34 +03:00