thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 20:02:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
86,728 Commits 9,330 Branches 1,783 Tags 2.1 GiB
99ad2f5fa6
Commit Graph

24905 Commits

Author SHA1 Message Date
Raúl Marín
060f5118f0 Improvements based on PR review 2022-04-06 10:48:12 +02:00
taiyang-li
acb9f1632e suppoort skip splits in orc and parquet 2022-04-06 16:40:22 +08:00
Kseniia Sumarokova
0e1637dfaf
Merge pull request #35354 from kssenii/enable-testmode-by-default 
Remove testmode option, enable it unconditionally
 2022-04-06 10:06:08 +02:00
Antonio Andelic
bf1f34ddb3 Fix unit tests 2022-04-06 07:43:48 +00:00
Antonio Andelic
84eef61d17 Pull under reader mutex 2022-04-06 06:39:56 +00:00
fenglv
a2ce366c34 parallel reading files for FileLog Engine 2022-04-06 04:49:43 +00:00
Yakov Olkhovskiy
ac441b9dcf compiler suggestions 2022-04-06 00:05:06 -04:00
taiyang-li
43e8af697a fix code style 2022-04-06 11:41:16 +08:00
kssenii
d59d4eda4f Fix tests 2022-04-06 01:58:16 +02:00
Yakov Olkhovskiy
ea9ce3ea18 'T' is added as delimiter, tests added 2022-04-05 19:50:16 -04:00
Mikhail Filimonov
f9a4cd12d4
Make parallel reading of kafka consumers work again (issue #35153) 2022-04-05 21:46:56 +02:00
Yakov Olkhovskiy
6b9a349083
Update SortDescription.h 
wrong merge fix
 2022-04-05 15:02:34 -04:00
Yakov Olkhovskiy
90c4cd3de7
Merge branch 'master' into interpolate-feature 2022-04-05 14:39:07 -04:00
Yakov Olkhovskiy
e0d6033c39 all columns can participate in interpolate expression despite if they are selected or not, some optimization on expressionless INTERPOLATE 2022-04-05 14:26:49 -04:00
Mikhail Filimonov
3412be9d4d
Change KafkaDirectReads to KafkaConsumersInUse 2022-04-05 20:25:05 +02:00
Nikolai Kochetov
def33323a0
Merge pull request #35788 from amosbird/better-projection1 
Improve minmax_count_projection
 2022-04-05 19:59:45 +02:00
Alexander Tokmakov
37a06eec1a fixes 2022-04-05 17:36:53 +02:00
Raúl Marín
39cc3baf5f Keep the previous exception code 2022-04-05 16:37:53 +02:00
Antonio Andelic
d296eeee2d Small changes for Keeper 2022-04-05 13:56:28 +00:00
kssenii
83488b2d13 Better 2022-04-05 14:48:26 +02:00
Alexander Tokmakov
d9e3e9b69f Merge branch 'master' into faster_replicated_ddl 2022-04-05 14:38:38 +02:00
Alexander Tokmakov
1fe50ad201 Merge branch 'master' into mvcc_prototype 2022-04-05 14:38:02 +02:00
Maksim Kita
c3c284e6e6
Merge pull request #35917 from kitaisreal/executable-user-defined-function-fix 
Executable user defined functions prevent executing during analysis
 2022-04-05 14:05:52 +02:00
Raúl Marín
8a05cf3927 DNS cache: Add option to drop elements after several consecutive failures 2022-04-05 13:00:27 +02:00
Azat Khuzhin
8ebaf84989 Fix clang-tidy readability-simplify-boolean-expr warning in AlterCommands.cpp 
I want to make the code more readable before, but clang-tidy is too
smart...

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-05 13:39:39 +03:00
Nikolai Kochetov
4479b68980
Merge pull request #35623 from nickitat/function_calculation_after_sorting_and_limit 
Functions calculation after sorting
 2022-04-05 12:09:56 +02:00
alesapin
eaa72ef002
Merge pull request #35904 from ClickHouse/more_logs_on_unsuccessful_part_removal 
More logs on unsuccessful part removal
 2022-04-05 11:29:08 +02:00
Maksim Kita
6e1d837439 Fixed style check 2022-04-05 11:24:33 +02:00
Alexander Tokmakov
da00beaf7f Merge branch 'master' into mvcc_prototype 2022-04-05 11:14:42 +02:00
Maksim Kita
b160ffd726
Merge pull request #35723 from ClickHouse/array-has-all-sse-avx2-optimizations 
Merging #27653
 2022-04-05 11:09:14 +02:00
Kseniia Sumarokova
77c7545395
Merge pull request #35922 from kssenii/fix-race-in-cached-buffer 
Fix race in cached buffer
 2022-04-05 10:55:58 +02:00
Azat Khuzhin
c6f182163a Require mutations for per-table TTL only when it had been changed 
Before this patch only per-column TTL did not requires mutation if it
had not been changed, after per-table TTL will also check whether it had
been changed or not.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-05 11:08:35 +03:00
Amos Bird
5bc09550d8
Fix tests 2022-04-05 15:56:39 +08:00
Amos Bird
35a8bb2a9b
add comment 2022-04-05 15:56:38 +08:00
Amos Bird
163664fad7
Improve minmax_count_projection 2022-04-05 15:56:37 +08:00
Vladimir C
54b4049ea5
Merge pull request #35794 from awakeljw/fork_chmaster 2022-04-05 09:44:45 +02:00
Alexander Tokmakov
4e9ec5dc2f make some replicated DDL faster 2022-04-05 00:51:48 +02:00
kssenii
3fd462bc05 Merge master 2022-04-05 00:06:18 +02:00
Alexey Milovidov
9d5c13c71d
Merge pull request #35886 from azat/format-insert-infile-fix 
Fix formatting of INSERT INFILE queries (missing quotes)
 2022-04-05 00:47:54 +03:00
larryluogit
808d9afd0f
Fix optin.cplusplus.UninitializedObject issue (#35626) 
* Fix optin.cplusplus.UninitializedObject issue

* Enable optin.cplusplus.UninitializedObject
 2022-04-04 23:47:14 +03:00
Alexey Milovidov
1b562923f9
Merge pull request #35941 from ClickHouse/revert-35184-clang-tidy-issues 
Revert "clang-tidy report issues with Medium priority"
 2022-04-04 23:42:21 +03:00
alesapin
69fceb7e6d
Merge pull request #35926 from ClickHouse/better_consumers 
Better check for kafka_num_consumers
 2022-04-04 22:42:05 +02:00
Alexey Milovidov
4d6c030d23
Revert "clang-tidy report issues with Medium priority" 2022-04-04 23:41:42 +03:00
kssenii
79627798c4 Fix race 2022-04-04 20:44:39 +02:00
Mikhail Filimonov
53c7376e37
Add some metrics to engine Kafka 2022-04-04 20:25:53 +02:00
alesapin
d04c48e67a Fix build 2022-04-04 20:14:09 +02:00
alesapin
1e4fe5e3b5
Merge pull request #35882 from ClickHouse/more_metadata_alters 
Make more alters of nested types metadata-only
 2022-04-04 20:11:31 +02:00
Alexander Gololobov
f0de8eb625 Extracted argument handling into a separate class to reuse it for makeDateTime() and makeDateTime64() 2022-04-04 19:57:04 +02:00
alesapin
0477e74f42 Get rid of caps 2022-04-04 19:41:54 +02:00
alesapin
72331856eb fix message 2022-04-04 18:28:26 +02:00
alesapin
86f42e7a3a Better check for kafka_num_consumers 2022-04-04 18:07:36 +02:00
Kseniia Sumarokova
d697576967
Update ReadBufferFromS3.h 2022-04-04 17:53:01 +02:00
Vladimir C
b80f24957e
Merge pull request #35674 from vdimir/asof-key-null-issue-35565 2022-04-04 16:56:04 +02:00
Anton Popov
bd93be65aa
Merge pull request #35820 from azat/columns-ttl-fix 
Avoid processing per-column TTL multiple times
 2022-04-04 16:42:50 +02:00
kssenii
6c6fb5c3e8 Fix race in cached buffer 2022-04-04 16:32:37 +02:00
Anton Popov
d08d4a2437
Merge pull request #35806 from CurtizJ/dynamic-columns-8 
Fix inserts to columns of type `Object` in partitioned tables
 2022-04-04 16:16:54 +02:00
Maksim Kita
2a8e479277 ExecutableUserDefinedFunction prevent function execution during query analysis 2022-04-04 15:56:01 +02:00
Antonio Andelic
db75bf6f5d
Merge pull request #35852 from ClickHouse/bigo-sg-add_hints 
Refactoring of hints for column descriptor
 2022-04-04 15:37:08 +02:00
alesapin
3ccf99c3d7
Merge pull request #35884 from ClickHouse/don_t_allow_to_reset_settings 
Forbid to reset non existing settings
 2022-04-04 15:27:34 +02:00
Nickita Taranov
0f94a58f3a use getName() 2022-04-04 14:59:38 +02:00
Nickita Taranov
698a984c07 throw if sorting column not found 2022-04-04 14:33:58 +02:00
Nickita Taranov
440e57769a more fizes 2022-04-04 14:33:58 +02:00
Nickita Taranov
ce40d84eef more fixes 2022-04-04 14:33:58 +02:00
Nickita Taranov
a39427f00b clean up 2022-04-04 14:33:57 +02:00
Nickita Taranov
eedcd61479 fix 2022-04-04 14:33:57 +02:00
Nickita Taranov
a08c035443 stash 2022-04-04 14:33:57 +02:00
Nickita Taranov
b095838444 stash 2022-04-04 14:33:57 +02:00
Nickita Taranov
4c51329ad6 stash 2022-04-04 14:33:57 +02:00
Alexander Tokmakov
a2167f12b8 Merge branch 'master' into mvcc_prototype 2022-04-04 14:24:23 +02:00
Nikita Taranov
bd89fcafdb
Make SortDescription::column_name always non-empty (#35805) 2022-04-04 14:17:15 +02:00
Maksim Kita
47528de78b Fix build 2022-04-04 14:07:05 +02:00
alesapin
9b75ef6ce9 Fix build 2022-04-04 14:03:16 +02:00
何李夫
09c04e4993
Improve the pipeline description for JOIN (#35612) 
Improve the pipeline description for JOIN
 2022-04-04 13:56:41 +02:00
Maksim Kita
af405d3ba6 Fixed style check 2022-04-04 13:34:27 +02:00
alesapin
885447342c More logs on unsuccessful part removal 2022-04-04 13:17:33 +02:00
Maksim Kita
e6c9a36ac7
Merge pull request #35733 from kitaisreal/ipv6-invalid-insert-test 
Added test for insert of invalid IPv6 value
 2022-04-04 12:28:16 +02:00
Azat Khuzhin
e5a837e574 Add id/parent_ids into system.processors_profile_log 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
58ee917e94 Mesure processors profiles only if it was enabled 
Since it may use little extra CPU.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
99528e296c Rename need_data_elapsed_us/port_full_elapsed_us to input_wait_us/output_wait_us 
$ gg -e need_data_ -e port_full_  | cut -d: -f1 | sort -u | xargs sed -i -e s/port_full_/output_wait_/g -e s/need_data_/input_wait_/g -e s/getPortFull/getOutputWait/g -e s/getNeedData/getInputWait/g

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
765b4bc45a Implement log_processors_profiles (write to processors_profile_log, OFF by default) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
5fd402eaba Measure time that Process spent in work/NeedData/PortFull 
Note, that right now it is done not in IProcessor, but in
ExecutingGraph/ExecutionThreadContext, to avoid lots of changes in the
IProcessor interface, to make review easier, but I'm not against of
change the IProcessor interface to incapsulate it there.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
14538f6456 Add system.processors_profile_log 
This is the system table that will contain Processors level profiling.

v2: one entry per Processor, not 3 (PortFull/NeedData/work())
v3: us over ms
v4: Enable processors_profile_log table by default

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:41 +03:00
Nikolai Kochetov
19819c72f8
Merge pull request #35290 from bigo-sg/function_enumerate_streams 
Add function getTypeSerializationStreams
 2022-04-04 12:09:53 +02:00
alesapin
803a1a2a9c Fix tests and check 2022-04-04 11:19:02 +02:00
Anton Popov
11e18a16f3
Merge pull request #35724 from Avogar/fix-order 
Improve schema inference for JSONEachRow and TSKV formats
 2022-04-04 11:00:21 +02:00
kssenii
36c583d0de Better version of cache on insert 2022-04-04 10:58:59 +02:00
Anton Popov
ae53aae106 fix clang-tidy 2022-04-04 08:48:31 +00:00
Azat Khuzhin
93bbe9641a Fix formatting of INSERT ... COMPRESSION 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 10:33:12 +03:00
Azat Khuzhin
9a76efb850 Fix formatting of INSERT INFILE queries (missing quotes) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 10:33:12 +03:00
taiyang-li
38f149b533 optimize trivial count hive query 2022-04-04 15:28:26 +08:00
Antonio Andelic
a926bc19ea Address PR comments 2022-04-04 07:24:42 +00:00
Antonio Andelic
d96b682a55 Refactor 2022-04-04 07:24:42 +00:00
taiyang-li
6bc1786047 fix style 2022-04-04 07:24:42 +00:00
taiyang-li
d6247338de fix failed stateless tests 2022-04-04 07:24:42 +00:00
taiyang-li
eda299b48b fix building 2022-04-04 07:24:42 +00:00
taiyang-li
fd9a10ef53 add hints for projections 2022-04-04 07:24:42 +00:00
taiyang-li
4547ed370a add hints for column description 2022-04-04 07:24:42 +00:00
Alexey Milovidov
d9e5ca2119
Merge pull request #34394 from holadepo/last_day 
Add toLastDayOfMonth function
 2022-04-04 07:02:08 +03:00
taiyang-li
4e2d5f1841 Merge remote-tracking branch 'bigo-sg/use_minmax_index' into optimize_trivial_hive_query 2022-04-04 10:42:28 +08:00
taiyang-li
cbfc0f6bac fix typo 2022-04-04 10:42:22 +08:00
Yakov Olkhovskiy
ff4d295265 style fix 2022-04-03 22:19:35 -04:00
Yakov Olkhovskiy
95ad1bf6e1 use aliases if exist for original_select_set 2022-04-03 22:10:36 -04:00
alesapin
a8c1ccdb20 Forbit to reset non existing settings 2022-04-03 18:29:24 +02:00
Kseniia Sumarokova
30c1afe29d
Merge pull request #35646 from bigo-sg/reduce-mutex-scope 
narrow mutex scope when setenv LIBHDFS3_CONF
 2022-04-03 17:59:24 +02:00
alesapin
7b35920d4c Make more alters of nested types metadata-only 2022-04-03 14:03:34 +02:00
alesapin
91453fe4d6
Merge pull request #35759 from chen9t/fix_dirty_metacache 
Clean dirty meta cache when part is broken during part loading phase
 2022-04-03 13:33:46 +02:00
Azat Khuzhin
3ce4844d4b Properly cancel the query after client format error 
In case of format error (i.e. select 2 format Template settings
format_template_row='/dev/null') the client will reset the connection,
since it will not expect Data blocks.

To fix this, catch this client error and properly cancel the query. This
should also fix query hang checks (the one that executed after each test).

v2: use getCurrentExceptionMessage()/getCurrentExceptionCode()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-03 14:01:16 +03:00
Alexey Milovidov
5a47958744
Merge pull request #35736 from CurtizJ/quota-written-bytes 
Add quota for written bytes
 2022-04-03 05:26:49 +03:00
Azat Khuzhin
195b4c47ea Avoid processing per-column TTL multiple times 
Before this patch ttl.txt will not be written for per-column TTLs, and
hence it will be calculated every time after server restart of
DETACH/ATTACH cycle (note, that it will work w/o restart since in-memory
representation will avoid this).

v2: convert test to .sh to get correct current database over default for MV
v3: extract UUID to avoid error like in [1]:
    [ 490 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) []: Code: 349. DB::Exception: Cannot convert NULL value to non-Nullable type: While processing query_id LIKE concat('%', CAST(_CAST(NULL, 'Nullable(UUID)') AS uuid, 'String'), '%'): while pushing to view test_0hc2ro.this_text_log (c64e5af4-059e-4330-a728-354ecf83c031). (CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN)
    [1]: https://s3.amazonaws.com/clickhouse-test-reports/35820/a512d322b024d37d2f1082c4833f59f86057555f/stateless_tests_flaky_check__address__actions_.html
v4: add no-parallel to avoid issues with disappeared underlying table while pushing to text_log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-02 15:19:39 +03:00
mergify[bot]
1e43e26fa1
Merge branch 'master' into fix-order 2022-04-02 12:00:29 +00:00
Kruglov Pavel
687e9e7aeb
Merge pull request #35760 from Avogar/insert-from-input 
Improve schema inference for insert select from table functions
 2022-04-02 13:50:48 +02:00
Yakov Olkhovskiy
ec0ad8804a style fix 2022-04-01 21:45:58 -04:00
shuchaome
a565a93740 reduce mutex scope when setenv LIBHDFS3_CONF 2022-04-02 06:52:39 +08:00
Yakov Olkhovskiy
0116233d36 allow INTERPOLATE to reference optimized out columns 2022-04-01 16:18:19 -04:00
Anton Popov
a10bf550c4
Update AsynchronousInsertQueue.cpp 2022-04-01 20:57:41 +02:00
tavplubix
97f29ac0fe
Merge pull request #35817 from tcoyvwac/fix/exception-in-destructor-needs-noexcept-false 
Fix missing `noexcept(false)` flag on destructor which causes builds to fail
 2022-04-01 20:35:03 +03:00
Anton Popov
687942ce70 more strict quota for written bytes 2022-04-01 15:02:49 +00:00
Kseniia Sumarokova
fe8c91541d
Merge pull request #35833 from ucasfl/context 
Try remove unneed variable
 2022-04-01 16:58:42 +02:00
Yakov Olkhovskiy
38993f215f
Merge pull request #35706 from yakov-olkhovskiy/ephemeral-35641 
Allow EPHEMERAL without explicit default expression
 2022-04-01 10:49:29 -04:00
kssenii
6c8e073a61 Merge master 2022-04-01 16:48:29 +02:00
tcoyvwac
cb698c4cc1
Replace throw in debug mode in destructor 
FileSegmentsHolder:
- Replaced throw in destructor with assert for all #ifdef modes.
 2022-04-01 15:42:34 +02:00
Alexander Tokmakov
6591d1ceb7 Merge branch 'master' into mvcc_prototype 2022-04-01 15:38:46 +02:00
Antonio Andelic
5bcf772d7c
Merge pull request #35789 from amosbird/better-pipeline1 
Refactoring QueryPipeline
 2022-04-01 14:38:19 +02:00
zzsmdfj
3cae0c74d9 to #34966_fix_dateTime_deserialize 2022-04-01 20:16:07 +08:00
zzsmdfj
ececee3817 to #34966_fix_dateTime_deserialize 2022-04-01 20:13:34 +08:00
Anton Popov
075bcf3f5f
Merge pull request #35803 from Avogar/fix-formats-with-names 
Fix bug in indexes of not presented columns in -WithNames formats
 2022-04-01 13:33:35 +02:00
fenglv
97523a3329 Try remove unneed variable 2022-04-01 09:58:23 +00:00
Kseniia Sumarokova
cafff71d2f
Merge pull request #35799 from evillique/extract-parser-fix 
Fix extract function parser
 2022-04-01 10:59:39 +02:00
alesapin
f2c6387a8d
Merge pull request #35381 from kssenii/remote-filesystem-cache-fixes 
Cache fixes for high concurrency on corner cases
 2022-04-01 10:48:31 +02:00
Kseniia Sumarokova
d3b3294872
Merge pull request #35365 from bigo-sg/improve_access_type 
Improve check access in table functions
 2022-04-01 10:47:02 +02:00
Kseniia Sumarokova
9c39320e75
Merge pull request #35800 from ucasfl/mutex 
remove unused variable
 2022-04-01 10:46:23 +02:00
Kseniia Sumarokova
04729ecc70
Merge pull request #35798 from xiedeyantu/patch-1 
Delete duplicate code
 2022-04-01 10:43:31 +02:00
mergify[bot]
33115a55c3
Merge branch 'master' into better-pipeline1 2022-04-01 06:44:43 +00:00
Antonio Andelic
8cc18c851d
Merge pull request #35705 from ClickHouse/is-secure-client-info 
Added `is_secure` column to `query_log`
 2022-04-01 08:27:20 +02:00
jewisliu
9a4cc78dfb support ALTER TABLE t DETACH PARTITION ALL syntax 2022-04-01 13:13:07 +08:00
Alexey Milovidov
5c6b84c398
Merge pull request #35586 from kssenii/fix-virtual-columns-s3 
Fix s3 engine getting virtual columns
 2022-04-01 04:43:32 +03:00
Alexey Milovidov
475dec3809
Merge pull request #35814 from ClickHouse/merge_tree_arguments_error 
Better error message for incorrect *MergeTree arguments
 2022-04-01 03:04:43 +03:00
kssenii
603dcbb1b7 Extend test 2022-03-31 18:23:15 +02:00
Yakov Olkhovskiy
538373a79b style fix 2022-03-31 12:13:49 -04:00
Yakov Olkhovskiy
a15996315e bugfix - columns order tracking 2022-03-31 11:51:13 -04:00
filimonov
24d72a18d0
Update getNumberOfPhysicalCPUCores.cpp 
In Kubernetes shares are calculated from resources.requests.cpu, quota & period - from limits.

When you configure only requests without limits - the container still can use all the cores. So you see all the cores the system has.
(because of requests the pod just can't be scheduled on the node which have less cpus)
So the effective CPU count is all the available cores if you don't have limits, or calculated from quota & period if you have the limits.
 2022-03-31 17:21:54 +02:00
Alexander Tokmakov
63e8d73ba4 better message for incorrect *MergeTree arguments 2022-03-31 17:08:30 +02:00
Antonio Andelic
1d480178c4
Merge pull request #35785 from justnoise/remove-atomic-flag-init 
remove ATOMIC_FLAG_INIT: depreciated in C++20 and warns in clang-14
 2022-03-31 16:44:56 +02:00
avogar
ab2a963287 Merge branch 'master' of github.com:ClickHouse/ClickHouse into allow-read-bools-as-numbers 2022-03-31 14:09:43 +00:00
Kruglov Pavel
252d66e80d
Update src/Processors/Formats/ISchemaReader.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-03-31 16:08:37 +02:00
mergify[bot]
24ade25d61
Merge branch 'master' into improve-schema-inference 2022-03-31 13:42:47 +00:00
Yakov Olkhovskiy
0367cbad3c messed merge fixed 2022-03-31 09:35:19 -04:00
Anton Popov
42acb1dc29 fix inserts to columns of type Object in partitioned tables 2022-03-31 13:30:01 +00:00
Yakov Olkhovskiy
b5682c1f02 minor refactoring 2022-03-31 08:33:50 -04:00
Kruglov Pavel
a217e9e9d0
Remove unused header 2022-03-31 14:27:43 +02:00
mergify[bot]
3183b61c74
Merge branch 'master' into mvcc_prototype 2022-03-31 12:27:09 +00:00
avogar
836e7dae67 Fix bug in indexes of not presented columns in -WithNames formats 2022-03-31 12:24:40 +00:00
Nikita Mikhaylov
5f27f690f4 Revert "Added support for schema inference for hdfsCluster (#35602)" 
This reverts commit f6bfdcc0c9.
 2022-03-31 14:19:56 +02:00
fenglv
e2e4b02d13 remove unused variable 2022-03-31 12:01:10 +00:00
alesapin
bd2ab32e3f
Merge pull request #35772 from azat/keeper-eof 
clickhouse-keeper: correctly handle closed client connection
 2022-03-31 13:57:28 +02:00
Alexander Tokmakov
461ecd56a4 fix race when database is Ordinary 2022-03-31 13:47:38 +02:00
Nikolay Degterinsky
e022f8527b Fix extract parser 2022-03-31 11:42:35 +00:00
xiedeyantu
d141dbc388
Delete duplicate code 
Delete duplicate code
 2022-03-31 19:34:30 +08:00
Amos Bird
12449caf89
Refactoring QueryPipeline 2022-03-31 19:26:35 +08:00
avogar
d272356324 Minor code improvement 2022-03-31 10:55:09 +00:00
avogar
74275da7ee Make better 2022-03-31 10:52:34 +00:00
Kruglov Pavel
564a77c462
Fix build 2022-03-31 12:49:23 +02:00
Nikita Mikhaylov
f6bfdcc0c9
Added support for schema inference for hdfsCluster (#35602) 2022-03-31 12:47:36 +02:00
alesapin
e12860f646
Update src/Disks/IO/CachedReadBufferFromRemoteFS.cpp 2022-03-31 11:42:42 +02:00
alesapin
4c66c2e10f
Update src/Disks/IO/CachedReadBufferFromRemoteFS.cpp 2022-03-31 11:42:38 +02:00
Alexander Tokmakov
5a50ad9de3 Merge branch 'master' into mvcc_prototype 2022-03-31 11:35:04 +02:00
alesapin
2365bbf0d3
Merge pull request #34566 from ClickHouse/startup-sanity-checks 
add simple sanity checks for server startup
 2022-03-31 11:33:59 +02:00
Kseniia Sumarokova
a711fc0402
Merge pull request #35572 from bigo-sg/hdfs_config_prefix 
Deduce absolute hdfs config path
 2022-03-31 10:08:32 +02:00
Kseniia Sumarokova
0b20329156
Merge pull request #35727 from kssenii/fix-any-all 
Fix any/all(subquery)
 2022-03-31 10:05:37 +02:00
Antonio Andelic
e336302650 Merge branch 'master' into is-secure-client-info 2022-03-31 06:58:16 +00:00
Antonio Andelic
049559d510 Remove extra newline 2022-03-31 06:57:59 +00:00
Antonio Andelic
13b4fabee4
Merge pull request #35743 from bigo-sg/remote_filter_for_hive 
Respect remote_url_allow_hosts for hive
 2022-03-31 08:54:37 +02:00
Antonio Andelic
cdedcee0ce
Merge pull request #35746 from ClickHouse/use-common-io-thread-pool 
Use common IOThreadPool for S3 and URL
 2022-03-31 08:41:54 +02:00
Azat Khuzhin
ff0e46c394 clickhouse-keeper: finish session on client EOF (fixes heap-use-after-free) 
CI founds [1]:

    WARNING: ThreadSanitizer: heap-use-after-free (pid=1)
      Read of size 8 at 0x7b48001e0088 by thread T7 (mutexes: write M2588):
        ...
        1 DB::KeeperTCPHandler::runImpl()::$_0::operator()() const build_docker/../src/Server/KeeperTCPHandler.cpp:384:14 (clickhouse+0x194f974f)
        ...
        8 DB::KeeperDispatcher::setResponse() build_docker/../src/Coordination/KeeperDispatcher.cpp:222:9 (clickhouse+0x1999de82)
        9 DB::KeeperDispatcher::responseThread() build_docker/../src/Coordination/KeeperDispatcher.cpp:161:18 (clickhouse+0x1999d7a7)
        10 DB::KeeperDispatcher::initialize()::$_1::operator()() const build_docker/../src/Coordination/KeeperDispatcher.cpp:273:54 (clickhouse+0x199a55d1)

      Previous write of size 8 at 0x7b48001e0088 by thread T3:
        0 operator delete(void*, unsigned long) <null> (clickhouse+0xa0dad50)
        1 DB::KeeperTCPHandler::~KeeperTCPHandler() build_docker/../src/Server/KeeperTCPHandler.cpp:648:1 (clickhouse+0x194f8ee6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/35772/38873d726d22b15d4c5a284410956927b2e2f95f/integration_tests__thread__actions__[4/4].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-31 09:31:34 +03:00
Brendan Cox
87fff8b2d3 remove ATOMIC_FLAG_INIT: depreciated in C++20 and warns in clang-14 2022-03-30 21:45:48 -07:00
taiyang-li
539ce5f1b8 Merge branch 'master' into use_minmax_index 2022-03-31 12:27:32 +08:00
taiyang-li
a5765dccb1 Merge branch 'master' into function_enumerate_streams 2022-03-31 12:21:00 +08:00
taiyang-li
70087bc959 Merge branch 'master' into improve_access_type 2022-03-31 12:19:17 +08:00
taiyang-li
f6ccda8097 Merge branch 'master' into hdfs_config_prefix 2022-03-31 12:16:05 +08:00
Yakov Olkhovskiy
3b40b56e56 refactored to remove parsers rependency on datatypes 2022-03-30 23:47:16 -04:00
taiyang-li
c11f563d47 Merge branch 'master' into remote_filter_for_hive 2022-03-31 10:54:55 +08:00
chen9t
b40a69f59a Code style 2022-03-31 10:10:05 +08:00
Nikolay Degterinsky
f055d7b692
Merge pull request #34055 from palegre-tiny/groupSortedArray 
Add groupSortedArray() function
 2022-03-31 01:20:15 +03:00
alesapin
81ce991494 Merge branch 'master' into startup-sanity-checks 2022-03-31 00:08:43 +02:00
Kruglov Pavel
4ec3c35e14
Merge pull request #35755 from Avogar/fix-custom-to-string 
Fix bug in conversion from custom types to string
 2022-03-31 00:06:48 +02:00
Nikita Taranov
30f2a942c5
Predict size of hash table for GROUP BY (#33439) 
* use AggregationMethod ctor with reserve

* add new settings

* add HashTablesStatistics

* support queries with limit

* support distributed and with external aggregation

* add new profile events

* add some tests

* add perf test

* export cache stats through AsynchronousMetrics

* rm redundant trace

* fix style

* fix 02122_parallel_formatting test

* review fixes

* fix 02122_parallel_formatting test

* apply also to two-level HTs

* try simpler strategy

* increase max_size_to_preallocate_for_aggregation for experiment

* fixes

* Revert "increase max_size_to_preallocate_for_aggregation for experiment"

This reverts commit 6cf6f75704.

* fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-03-30 22:47:51 +02:00
Yakov Olkhovskiy
6a1e116c46 refactoring 2022-03-30 16:34:19 -04:00
Azat Khuzhin
38873d726d clickhouse-keeper: fix implementation of server with poll() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-30 21:30:12 +03:00
kssenii
98e85dd68e Fix build 2022-03-30 20:27:04 +02:00
kssenii
ac4897f4f9 Fix 2022-03-30 20:16:41 +02:00
Azat Khuzhin
9820230e95 clickhouse-keeper: correctly handle closed client connection 
This will avoid noisy message like:

    DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-30 20:29:04 +03:00
Maksim Kita
e43fdcd7eb Function hasAll added dynamic dispatch for SSE4.2, AVX2 2022-03-30 18:41:34 +02:00
Maksim Kita
8d0a9689e4 Update gatherutils CMakeLists to use X86_INTRINSICS_FLAGS from cpu_features 2022-03-30 18:40:18 +02:00
Maksim Kita
91eec8962f Rename test 2022-03-30 18:39:28 +02:00
shuchaome
738966b6b6 fix filebuffer pos in RemoteReadBuffer 2022-03-31 00:03:45 +08:00
kssenii
c1ae5814a7 Merge master 2022-03-30 17:59:36 +02:00
Alexander Tokmakov
996d57b918 try fix build with gcc 2022-03-30 17:42:47 +02:00
kssenii
099fa0e668 Merge master 2022-03-30 17:33:32 +02:00
kssenii
0fc92fe2aa Get rid of all "remote" mentions 2022-03-30 17:15:29 +02:00
mergify[bot]
4813f0b305
Merge branch 'master' into is-secure-client-info 2022-03-30 15:12:56 +00:00
kssenii
b30948dd53 Relax test 2022-03-30 17:12:23 +02:00
Kruglov Pavel
771f34d793
Merge pull request #35691 from Avogar/fix-matview 
Fix creating materialized view with subquery after server restart
 2022-03-30 14:57:14 +02:00
avogar
849596c7a2 Improve schema inference for insert select queries 2022-03-30 12:44:53 +00:00
chen9t
5cb2301e39 Clean dirty meta cache when part is broken during part loading phase 2022-03-30 20:40:20 +08:00
Antonio Andelic
7bc5731025 Merge branch 'master' into is-secure-client-info 2022-03-30 11:26:05 +00:00
Antonio Andelic
a3ad99be03 Create CallbackRunner in Storage 2022-03-30 11:24:50 +00:00
avogar
af4bfec051 Fix bug in conversion from custom types to string 2022-03-30 11:19:03 +00:00
Maksim Kita
371cdc956a Added input format settings for parsing invalid IPv4, IPv6 addresses as default values 2022-03-30 12:54:19 +02:00
Vladimir C
31c367d3cd
Merge pull request #35651 from amosbird/columntransformerrename 2022-03-30 12:37:30 +02:00
kssenii
98ad3f4a91 Fix tests, rename some remote mentions 2022-03-30 12:21:59 +02:00
lgbo-ustc
08e1fdc6b8 Fixed :a elimate AMBIGUOUS_COLUMN_NAME exception 
elimate AMBIGUOUS_COLUMN_NAME exception in hdfs/s3 cluster when run a query with join
 2022-03-30 17:06:38 +08:00
Antonio Andelic
de2c6684a3 Remove setup and cleanup 2022-03-30 09:00:06 +00:00
Antonio Andelic
1173eab15e Use same IOThreadPool for URL and S3 2022-03-30 08:15:20 +00:00
taiyang-li
16bb4c4ad0 respect remote_url_allow_hosts for hive 2022-03-30 15:33:59 +08:00
Antonio Andelic
e15fccd31b Add is_secure info to HTTP 2022-03-30 06:39:40 +00:00
Antonio Andelic
d14ad227b1 Polish TCP is_secure flag 2022-03-30 06:39:40 +00:00
Antonio Andelic
b982959c4c Fix writting is_secure 2022-03-30 06:39:40 +00:00
Antonio Andelic
893b8a9513 Initial implementation of is_secure 2022-03-30 06:39:40 +00:00
Antonio Andelic
03a09dc914
Merge pull request #35571 from ClickHouse/parallel-s3-downloading 
Parallel S3 downloading
 2022-03-30 08:14:49 +02:00
Antonio Andelic
d85ed8f2a9
Merge pull request #35655 from ClickHouse/exception-compile-time-message-check 
Use compile-time check for `Exception` messages
 2022-03-30 08:11:32 +02:00
taiyang-li
0af6fdb576 fix building 2022-03-30 11:28:21 +08:00
taiyang-li
fcfabf4a0c Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-30 11:23:24 +08:00
taiyang-li
47f3e9330e merge master and fix conflict 2022-03-30 11:06:51 +08:00
taiyang-li
e76ed3acde Merge branch 'master' into improve_access_type 2022-03-30 11:04:57 +08:00
taiyang-li
9ffb42d507 Merge branch 'master' into hdfs_config_prefix 2022-03-30 11:02:31 +08:00
Anton Popov
a842a81aba
Merge pull request #35690 from CurtizJ/flatten-tuple 
Add function `flattenTuple`
 2022-03-30 00:24:36 +02:00
Anton Popov
1cba31c305
Merge pull request #35687 from CurtizJ/dynamic-columns-7 
Fix reading from type object
 2022-03-30 00:21:32 +02:00
Anton Popov
caacc7d385 add quota for written bytes 2022-03-29 18:21:29 +00:00
kssenii
ce9131f905 Fix unit test 2022-03-29 19:57:55 +02:00
kssenii
f0e0d97752 Fix lask of query id in merges 2022-03-29 19:49:42 +02:00
avogar
000f3043e7 Make better 2022-03-29 17:40:07 +00:00
avogar
3fc36627b3 Allow to infer and parse bools as numbers in JSON input formats 2022-03-29 17:37:31 +00:00
kssenii
a2e950bd3b Fix 2022-03-29 17:34:50 +02:00
kssenii
421b1e5a81 Update 2022-03-29 17:33:02 +02:00
avogar
ce97ccbfb9 Improve schema inference for JSONEachRow and TSKV formats 2022-03-29 14:47:51 +00:00
Maksim Kita
2742b88e6c
Merge pull request #27653 from ContentSquare/hasAllAny_SIMD 
Implement HasAll specialization for SSE and AVX2
 2022-03-29 16:35:59 +02:00
Kruglov Pavel
a2fd09e031
Fix style 2022-03-29 16:34:07 +02:00
Alexander Tokmakov
287d858fda Merge branch 'master' into mvcc_prototype 2022-03-29 16:24:12 +02:00
Dmitry Novik
8f935a72d6
Merge pull request #33230 from CurtizJ/read_in_order_max_rows_to_read 
Proper handle of 'max_rows_to_read' in case of reading in order of sorting key
 2022-03-29 15:16:34 +02:00
Antonio Andelic
ef484547bf Fix GCC build 2022-03-29 13:16:11 +00:00
Antonio Andelic
9990abb76a Use compile-time check for Exception messages, fix wrong messages 2022-03-29 13:16:11 +00:00
Yakov Olkhovskiy
46fd69d2bf
Update ParserCreateQuery.h 
style fix
 2022-03-29 09:15:27 -04:00
avogar
97f5033ea9 Fix tests 2022-03-29 13:07:37 +00:00
mergify[bot]
343588de2c
Merge branch 'master' into improve-schema-inference 2022-03-29 13:06:00 +00:00
taiyang-li
67c3c0be3d Merge branch 'master' into improve_access_type 2022-03-29 20:42:57 +08:00
taiyang-li
631352ba4b Merge branch 'master' into hdfs_config_prefix 2022-03-29 20:40:32 +08:00
alesapin
842c0b8ff0 Merge remote-tracking branch 'origin/startup-sanity-checks' into startup-sanity-checks 2022-03-29 14:29:33 +02:00
kssenii
4374415bd7 Fix clang-tidy, style check 2022-03-29 14:20:21 +02:00
Anton Popov
9610139477
Merge pull request #35629 from CurtizJ/dynamic-columns-5 
Support schema inference for type `Object` in format `JSONEachRow`
 2022-03-29 14:17:09 +02:00
alesapin
ed2c461fe1 Merge branch 'master' into startup-sanity-checks 2022-03-29 14:09:20 +02:00
kssenii
7a70e3a6da Fix 2022-03-29 14:08:24 +02:00
Kruglov Pavel
fe60560c03
Merge pull request #35619 from Avogar/check-remote-url-allow-hosts 
Check remote_url_allow_hosts before schema inference in URL engine
 2022-03-29 14:08:02 +02:00
Anton Popov
d42632dc13
Merge pull request #32928 from bigo-sg/rocksdb_metacache 
Speed up parts loading process of MergeTree by metadata cache
 2022-03-29 14:05:22 +02:00
taiyang-li
59727a423e Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-29 19:39:09 +08:00
Alexander Gololobov
bf376ee2f5
Merge pull request #35628 from ClickHouse/make_date 
Implementation of makeDate and makeDate32
 2022-03-29 13:35:26 +02:00
Maksim Kita
a1a4552740
Merge pull request #35184 from DevTeamBK/clang-tidy-issues 
clang-tidy report issues with Medium priority
 2022-03-29 13:19:54 +02:00
Antonio Andelic
f0b4a9fd15 Fix construction of ReadBufferFromS3 2022-03-29 10:56:49 +00:00
kssenii
eab04674fa Fix review comments 2022-03-29 12:38:17 +02:00
taiyang-li
77314246f3 Merge branch 'master' into hdfs_config_prefix 2022-03-29 18:32:59 +08:00
taiyang-li
834ac3a1b9 Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-29 18:32:06 +08:00
Anton Popov
5d8988ad4e
Merge pull request #35645 from CurtizJ/fix-quotas-async-inserts 
Fix usage of quota with async inserts
 2022-03-29 11:54:24 +02:00
Nikolai Kochetov
86690f18ee
Merge pull request #35667 from amosbird/projection-fix2 
Another fix of projection part check
 2022-03-29 10:22:01 +02:00
Yakov Olkhovskiy
ef34f7dbd7 allow EPHEMERAL without expression 2022-03-29 03:54:13 -04:00
Antonio Andelic
362e877529 Merge branch 'master' into parallel-s3-downloading 2022-03-29 07:05:26 +00:00
taiyang-li
db436ad621 Merge branch 'master' into function_enumerate_streams 2022-03-29 11:35:21 +08:00
Amos Bird
3ab8fc1e7b
Fix projection part check 
Fix projection part check when projection part type is different from
parent part type.
 2022-03-29 10:39:07 +08:00
Alexey Milovidov
5e262fba85
Merge pull request #35204 from azat/build-gcc 
Add build with GCC
 2022-03-29 04:55:15 +03:00
kssenii
44157e5c80 Merge master 2022-03-29 02:19:32 +02:00
kssenii
c6b1f86306 Merge with master 2022-03-29 02:03:28 +02:00
Yakov Olkhovskiy
615efa1381 aliases processing fixed 2022-03-28 19:15:53 -04:00
Nikita Mikhaylov
85282f1b4d
Made schema inference work with s3Cluster table function (#35544) 2022-03-29 00:46:35 +02:00
mergify[bot]
bf5a9dcb7a
Merge branch 'master' into groupSortedArray 2022-03-28 20:57:07 +00:00
mergify[bot]
9efc8bda78
Merge branch 'master' into remote-filesystem-cache-fixes 2022-03-28 20:53:29 +00:00
kssenii
1d49a85963 Merge master 2022-03-28 22:51:56 +02:00
Alexander Gololobov
b49993f993 Fixes according to the code review 2022-03-28 22:47:39 +02:00
kssenii
a2013ebe2b Better 2022-03-28 22:40:27 +02:00
Kseniia Sumarokova
9a8e9198e5
Merge pull request #35593 from kssenii/named-collections-fix 
Fix headers with named collections, add compression_method
 2022-03-28 22:10:32 +02:00
avogar
36ea9ef11f Fix creating materialized view with subquery after server restart 2022-03-28 19:52:10 +00:00
Anton Popov
24c0cf86d4 add function 'flattenTuple' 2022-03-28 19:32:12 +00:00
alesapin
7ab8d7178c
Merge pull request #35308 from ClickHouse/clickhouse-keeper 
New package clickhouse-keeper
 2022-03-28 21:23:17 +02:00
Azat Khuzhin
bf4df5c6bb Fix SIGSEGV for build under gcc-11 (due to auto deduction) 
During building with gcc-11 you will got SIGSEGV for building
InterpretersMySQLDDLQuery (and some others), and it is due to endless
recursion:

    (gdb) bt 5
    0  0x00000000010978f2 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=0) at ../../src/gcc/cp/typeck.c:1274
    1  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    2  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    3  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    4  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    (gdb) bt -X
    1397454 0x0000000001097bd9 in structural_comptypes (t1=0x7fde028d1540, t2=0x7fde028d27e0, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    1397455 0x0000000000f2d8b5 in cp_tree_equal (t1=<optimized out>, t2=<optimized out>) at ../../src/gcc/cp/tree.c:4144
    1397456 0x00000000010909cb in template_args_equal (ot=0x7fde028cf578, nt=0x7fde028cfc58, partial_order=<optimized out>) at ../../src/gcc/cp/pt.c:9256
    1397457 0x0000000001090422 in template_args_equal (partial_order=false, nt=0x7fde028cfc58, ot=0x7fde028cf578) at ../../src/gcc/cp/pt.c:9295
    1397458 comp_template_args (oldargs=0x7fde028cf550, newargs=0x7fde028cfc30, oldarg_ptr=0x0, newarg_ptr=0x0, partial_order=false) at ../../src/gcc/cp/pt.c:9285
    1397459 0x00000000010a08f4 in spec_hasher::equal (e1=0x7fde028c95d0, e2=0x7ffd1194e8c0) at ../../src/gcc/cp/pt.c:1726
    1397460 0x0000000001085965 in hash_table<spec_hasher, false, xcallocator>::find_with_hash (this=0x7fde36b7f450, comparable=@0x7ffd1194e8b8: 0x7ffd1194e8c0, hash=<optimized out>) at ../../src/gcc/hash-table.h:936
    1397461 0x0000000001079698 in lookup_template_class_1 (d1=<optimized out>, arglist=0x7fde028cfc30, in_decl=0x0, context=<optimized out>, entering_scope=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:9896
    1397462 0x000000000109f8ef in lookup_template_class (complain=3, entering_scope=1, context=0x7fde27558e40, in_decl=0x0, arglist=0x7fde028cfc08, d1=0x7fde269bcd20) at ../../src/gcc/cp/pt.c:10251
    1397463 tsubst_aggr_type (t=0x7fde269bcd20, args=<optimized out>, complain=3, in_decl=0x0, entering_scope=1) at ../../src/gcc/cp/pt.c:13646
    1397464 0x000000000108f797 in tsubst (t=0x7fde269bcdc8, args=0x7fde028cf7a8, complain=3, in_decl=<optimized out>) at ../../src/gcc/cp/pt.c:16108
    1397465 0x0000000000e61bf2 in rewrite_template_parm (level=1, complain=3, tsubst_args=0x7fde028cf7a8, index=5, olddecl=0x7fde269b5600) at ../../src/gcc/cp/pt.c:28556
    1397466 rewrite_tparm_list(tree_node*, unsigned int, unsigned int, tree_node*, unsigned int, int) [clone .constprop.0] (oldelt=0x7fde269bd190, index=5, targs=0x7fde028cf7a8, targs_index=4, complain=3, level=1) at ../../src/gcc/cp/pt.c:28640
    1397467 0x00000000009f3748 in build_deduction_guide (type=type@entry=0x7fde26e13dc8, ctor=0x7fde269ac300, outer_args=outer_args@entry=0x0, complain=complain@entry=3) at ../../src/gcc/cp/pt.c:28769
    1397468 0x00000000009f444f in ctor_deduction_guides_for (complain=3, tmpl=<optimized out>) at ../../src/gcc/cp/cp-tree.h:842
    1397469 deduction_guides_for (tmpl=<optimized out>, any_dguides_p=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:29282
    1397470 0x00000000008507a8 in do_class_deduction (complain=3, flags=1, init=<optimized out>, tmpl=0x7fde26e0f980, ptype=0x7fde028c7b28) at ../../src/gcc/cp/pt.c:29402
    1397471 do_auto_deduction (type=0x7fde028c7b28, init=<optimized out>, auto_node=<optimized out>, complain=3, context=<optimized out>, outer_targs=<optimized out>, flags=1) at ../../src/gcc/cp/pt.c:29572
    1397472 0x00000000007c9569 in finish_compound_literal (type=<optimized out>, compound_literal=0x7fde028c95b8, complain=3, fcl_context=fcl_functional) at ../../src/gcc/cp/semantics.c:3060
    1397473 0x0000000001123a79 in cp_parser_functional_cast (parser=0x7fde27558da8, type=0x7fde028c7b28) at ../../src/gcc/cp/parser.c:30670
    1397474 0x0000000000fd7873 in cp_parser_postfix_expression (parser=0x7fde27558da8, address_p=<optimized out>, cast_p=<optimized out>, member_access_only_p=<optimized out>, decltype_p=false, pidk_return=0x0) at ../../src/gcc/cp/parser.c:7437
    1397475 0x0000000000fd4ddf in cp_parser_binary_expression (parser=0x7fde27558da8, cast_p=<optimized out>, no_toplevel_fold_p=false, decltype_p=<optimized out>, prec=PREC_NOT_OPERATOR, pidk=<optimized out>) at ../../src/gcc/cp/parser.c:9842
    1397476 0x0000000000fd4595 in cp_parser_assignment_expression (parser=0x7fde27558da8, pidk=<optimized out>, cast_p=<optimized out>, decltype_p=<optimized out>) at ../../src/gcc/cp/parser.c:10146
    1397477 0x0000000000fd3b90 in cp_parser_constant_expression (parser=0x7fde27558da8, allow_non_constant_p=2, non_constant_p=0x7ffd1194f1d7, strict_p=<optimized out>) at ../../src/gcc/cp/parser.c:10449
    1397478 0x0000000000fcfdd5 in cp_parser_initializer_clause (non_constant_p=<optimized out>, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:24253
    1397479 cp_parser_initializer (parser=0x7fde27558da8, is_direct_init=<optimized out>, non_constant_p=<optimized out>, subexpression_p=<optimized out>) at ../../src/gcc/cp/parser.c:24193
    1397480 0x000000000062e5d8 in cp_parser_decomposition_declaration (init_loc=0x7ffd1194f1d8, maybe_range_for_decl=0x7ffd1194f498, decl_specifiers=0x7ffd1194f1f0, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:14734
    1397481 cp_parser_simple_declaration (parser=0x7fde27558da8, function_definition_allowed_p=<optimized out>, maybe_range_for_decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:14393
    1397482 0x000000000109b870 in cp_parser_init_statement (parser=0x7fde27558da8, decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:13420
    1397483 0x00000000010996f0 in cp_parser_for (unroll=0, ivdep=false, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:12708
    1397484 cp_parser_iteration_statement (parser=0x7fde27558da8, if_p=0x0, ivdep=<optimized out>, unroll=<optimized out>) at ../../src/gcc/cp/parser.c:13343
    1397485 0x0000000000fe5c46 in cp_parser_statement (parser=0x7fde27558da8, in_statement_expr=0x0, in_compound=<optimized out>, if_p=0x0, chain=0x0, loc_after_labels=0x0) at ../../src/gcc/cp/parser.c:11718
    1397486 0x0000000000fe15ac in cp_parser_statement_seq_opt (in_statement_expr=<optimized out>, parser=<optimized out>) at ../../src/gcc/cp/parser.c:12201
    1397487 cp_parser_compound_statement (parser=0x7fde27558da8, in_statement_expr=0x0, bcs_flags=<optimized out>, function_body=<optimized out>) at ../../src/gcc/cp/parser.c:12150

Interesting frame is 1397471, from which we can extract location:

    (gdb) p line_table[0].info_ordinary.maps[line_table[0].info_ordinary.cache]
    $54 = {
      <line_map> = {
        start_location = 1396581280
      },
      members of line_map_ordinary:
      reason = LC_RENAME,
      sysp = 0 '\000',
      m_column_and_range_bits = 8,
      m_range_bits = 0,
      to_file = 0x3eb4bb0 "/ch/src/Interpreters/MySQL/InterpretersMySQLDDLQuery.cpp",
      to_line = 46,
      included_from = 0
    }

By replicating SOURCE_LINE() macro from gcc-11 (libcpp/include/line-map.h):

    /* Converts a map and a location_t to source line.  */
    inline linenum_type
    SOURCE_LINE (const line_map_ordinary *ord_map, location_t loc)
    {
      return ((loc - ord_map->start_location)
          >> ord_map->m_column_and_range_bits) + ord_map->to_line;
    }

We got line 154:

    (gdb) p ((input_location-1396581280) >> 8) + 46
    $61 = 154

And this is:

    auto [column_name_and_type, declare_column_ast] = std::tuple{columns_name_and_type.begin(), columns_definition->children.begin()};

After rewriting it, everything works correctly.

Also here is a reproducer for gcc-11 (and no failures for gcc-12, but
gcc-12 has other issues, like [1] and one more for hash table):

    # cat /tmp/test.cpp
    #include <tuple>

    auto multi()
    {
            return std::tuple{1, 1};
    }
    double foo()
    {
            auto [a, b] = multi();
            return a - b;
    }

    # g++-11 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp
    g++-11: internal compiler error: Segmentation fault signal terminated program cc1plus
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-11/README.Bugs> for instructions.
    # g++-12 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp

  [1]: https://reviews.llvm.org/D122598

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-28 22:21:45 +03:00
Kseniia Sumarokova
859af97fc7
Merge pull request #35614 from kssenii/fix-materialized-postgresql 
Fix possible segfault in materialised postgresql
 2022-03-28 21:11:44 +02:00
Anton Popov
6cbdc6af00 remove obsolete parameter 2022-03-28 18:44:53 +00:00
Vladimir C
ecdd89e811
Merge pull request #35666 from vdimir/fix-fuse 2022-03-28 20:40:43 +02:00
Alexander Tokmakov
208b242188 Merge branch 'master' into mvcc_prototype 2022-03-28 19:58:06 +02:00
Anton Popov
d677635cd8
Merge pull request #35592 from CurtizJ/dynamic-columns-4 
Add parallel parsing and schema inference for format `JSONAsObject`
 2022-03-28 19:29:55 +02:00
Anton Popov
0e4af89f69 fix reading from type object 2022-03-28 17:23:34 +00:00
Alexander Gololobov
93021e387e Implementation of makeDate and makeDate32 2022-03-28 18:57:24 +02:00
Anton Popov
bbfe8a2ca7 fix possible loss of subcolumns in type Object 2022-03-28 15:36:05 +00:00
mergify[bot]
f09ebea2d8
Merge branch 'master' into read_in_order_max_rows_to_read 2022-03-28 13:30:34 +00:00
vdimir
a216bc26c1
Correct check asof join key nullability 2022-03-28 13:29:34 +00:00
taiyang-li
38702be40c Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-28 21:03:22 +08:00
mergify[bot]
70b339ea4e
Merge branch 'master' into check-remote-url-allow-hosts 2022-03-28 11:55:48 +00:00
mergify[bot]
3924609ab2
Merge branch 'master' into metric-async-insert-bytes 2022-03-28 11:37:49 +00:00
vdimir
1122db89db
Use float devision for avg after optimize_fuse_sum_count_avg 2022-03-28 11:26:42 +00:00
mergify[bot]
a022de4a6e
Merge branch 'master' into fix-materialized-postgresql 2022-03-28 10:55:41 +00:00
tavplubix
8de655bf1f
Merge pull request #35621 from ClickHouse/update_comment_mismatching_checksums 
Update comment about mismatching checksums
 2022-03-28 13:42:12 +03:00
Amos Bird
6003a49a87
Fix style 2022-03-28 17:48:47 +08:00
alesapin
77e700b1cf
Update src/Compression/CompressionFactory.cpp 2022-03-28 11:25:08 +02:00
Antonio Andelic
d9d826c813 Address PR review 2022-03-28 08:19:23 +00:00