Commit Graph

5964 Commits

Author SHA1 Message Date
Alexandra Latysheva
e89a56969f first part 2020-09-30 23:16:27 +00:00
tavplubix
2f95609f98
Merge pull request #15448 from ClickHouse/allow_synchronous_drop_for_atomic_db
Allow synchronous DROP/DETACH TABLE for Atomic
2020-10-01 01:14:31 +03:00
Artem Zuikov
0f6a6453fe
Remove multiple_joins_rewriter v1 (#15472) 2020-09-30 23:11:49 +03:00
Artem Zuikov
0ac18a382f
Fix JOIN with GROUP BY over StorageMerge (#15242) 2020-09-30 23:11:27 +03:00
alesapin
97c2579fb9 Use separate function 2020-09-30 22:50:23 +03:00
alesapin
ac9f0ff4da More strict check for version column 2020-09-30 22:44:35 +03:00
Alexander Kuzmenkov
e45edd57f2 Add reinterpretAsUUID function 2020-09-30 20:19:32 +03:00
Vladimir Chebotarev
0497ce4f9c Possibly better fix. 2020-09-30 18:29:10 +03:00
alesapin
cfe9a0c4a1 Fix event subscription in DDLWorker 2020-09-30 17:58:27 +03:00
Nikolai Kochetov
45618c9d4c
Merge pull request #15437 from ClickHouse/azat-ThreadPool-UAF-fixes
Destroy resources captured by lambda after `ThreadFromGlobalPool::join()`
2020-09-30 16:50:30 +03:00
Nikolai Kochetov
0b1fe69ad7 Fix build. 2020-09-30 16:31:06 +03:00
Vladimir Chebotarev
b460024356 Fixed bug in GZIP compression in S3 storage. 2020-09-30 16:09:55 +03:00
Nikolai Kochetov
a1aca04711 Fix build. 2020-09-30 15:32:37 +03:00
Vladimir Chebotarev
a38c7ff2a8 Somehow uncompressed PUT works and gzipped PUT doesn't, in S3 storage. 2020-09-30 15:04:42 +03:00
Nikolai Kochetov
782823de4d Fix build. 2020-09-30 14:45:47 +03:00
Vladimir Chebotarev
698768add0 Fixed format argument in S3 storage when compression method is selected. 2020-09-30 13:55:44 +03:00
alesapin
01d8a36cd6
Merge pull request #15456 from ClickHouse/more-exception-safety-in-shellcommand
Fix exception safety in ShellCommand
2020-09-30 12:56:55 +03:00
tavplubix
c9eeb149fd
Merge pull request #15384 from zhang2014/fix/ISSUES-15365
ISSUES-15365 fix attach mysql database database
2020-09-30 11:27:15 +03:00
Nikolai Kochetov
0e412686bc Better comments. 2020-09-30 11:25:22 +03:00
alexey-milovidov
0c07c2d3a3
Merge pull request #15089 from ClickHouse/codecs_for_nested_types
Codecs for nested types
2020-09-30 07:18:19 +03:00
feng lv
c857b5dd5d fix 2020-09-30 09:50:23 +08:00
alexey-milovidov
412a54e356
Merge pull request #14693 from hustnn/zk_exist_fix2
Fix potential memory leak caused by zookeeper exist calling
2020-09-30 04:48:11 +03:00
alexey-milovidov
b6bccfc4f4
Merge pull request #14776 from Vxider/disable_set_and_join_persistency
Support memory only mode for StorageJoin and StorageSet.
2020-09-30 04:35:43 +03:00
alexey-milovidov
ac989f3a8f
Merge pull request #15285 from ucasFL/now-with-timezone
function `now` allow an argument with timezone
2020-09-30 04:32:36 +03:00
Alexey Milovidov
78dbf10c7b Fix exception safety in ShellCommand 2020-09-30 03:00:45 +03:00
Alexander Tokmakov
e156e6115e add setting 2020-09-30 01:30:34 +03:00
alexey-milovidov
b658f348a7
Update MergeTreeDataPartWriterWide.cpp 2020-09-30 01:23:42 +03:00
alexey-milovidov
9f944424cf
Merge pull request #15364 from ClickHouse/format-raw-blob
Add format RawBLOB
2020-09-30 01:14:40 +03:00
alexey-milovidov
3e99ca797b
Merge pull request #15426 from ClickHouse/format-regexp-escaping-rule-change-default-to-raw
Format Regexp: change default escaping rule to Raw
2020-09-30 01:14:08 +03:00
Nikolai Kochetov
727289cb09 Destroy resurces captured by lambda after ThreadFromGlobalPool::join(). 2020-09-29 23:43:02 +03:00
Nikolai Kochetov
a2c70c4611 Destroy resurces captured by lambda after ThreadFromGlobalPool::join(). 2020-09-29 22:11:40 +03:00
Artem Zuikov
86cfc6f914
change error code (#15452) 2020-09-29 20:36:14 +03:00
alesapin
4ab39d2bee
Fix style 2020-09-29 18:19:30 +03:00
Alexander Tokmakov
cf86e19f38 fix build 2020-09-29 17:15:05 +03:00
Alexander Tokmakov
8692c10495 add alias 2020-09-29 17:12:40 +03:00
Alexander Tokmakov
5d19ed04f6 allow synchronous drop/detach table for Atomic 2020-09-29 16:47:08 +03:00
Nikolai Kochetov
33487c0e4a
Merge pull request #15422 from ClickHouse/query-plan-header-remove-columns
Remove column from header descriotion for EXPLAIN PLAN
2020-09-29 15:51:09 +03:00
feng lv
4f000388a7 add setting output_format_pretty_row_numbers 2020-09-29 20:30:36 +08:00
alesapin
65a05a970d Better message on error 2020-09-29 15:07:01 +03:00
Alexander Kuzmenkov
2d456e2cd6
Merge pull request #15407 from detailyang/master
bugfix:aggregate:fix boundingratio args check
2020-09-29 14:12:54 +03:00
Nikolai Kochetov
a7e7fd084f Destroy resurces captured by lambda after ThreadFromGlobalPool::join(). 2020-09-29 13:48:11 +03:00
Nikolai Kochetov
de56c02525 Merge branch 'master' into expression-dag 2020-09-29 12:18:29 +03:00
Nikolai Kochetov
53d4e5d195 Merge branch 'master' into expression-dag 2020-09-29 12:15:50 +03:00
Azat Khuzhin
1fcc36ef80 Use ThreadPool to ensure that everything has been finished in PipelineExecutor
This will fix possible UAF, since ThreadFromGlobalPool cannot wait until
the job will be reseted, for this you need to call ThreadPool::wait()
(that waits until scheduled_jobs will be zero), and this guarantee that
the job was reseted, otherwise some variables that job is referencing
may be already destroyed, while job hasn't been destoyed yet (example
that I found PipelineExecutor -> ThreadGroupStatusPtr -> MemoryTracker
-> ~MemoryTracker -> log, while log had been destroyed already, here is
TSAN report --
https://gist.github.com/azat/d480dc3af5a0a44de4b038e20807c4b9, copied
from
https://clickhouse-test-reports.s3.yandex.net/15035/79133a426fdf042e383ea0cdccc4dc8273baa3a7/functional_stateless_tests_(thread)/test_run.txt.out.log)
2020-09-29 11:00:03 +03:00
Azat Khuzhin
789434994a Reset job on failure too in ThreadPool::worker()
job should be reseted before decrementing scheduled_jobs to
ensure that the Job destroyed before wait() returns.
2020-09-29 10:56:36 +03:00
zhang2014
9e94c077e7 IESSUES-15365 try fix integration test 2020-09-29 10:39:30 +08:00
Alexey Milovidov
dee57d7976 Format Regexp: change default escaping rule to Raw 2020-09-29 00:19:13 +03:00
Alexey Milovidov
e4800e7b35 Fix comment 2020-09-29 00:10:36 +03:00
Nikolai Kochetov
347080ff9a Remove column from header descriotion for EXPLAIN PLAN and EXPLAIN PIPELINE 2020-09-28 20:37:11 +03:00
Alexander Kuzmenkov
51b9aaf4d8
Merge pull request #15363 from ClickHouse/format-regexp-raw
Add support for "Raw" column format for Regexp format
2020-09-28 20:03:32 +03:00
tavplubix
4578d89819
Merge pull request #15348 from ClickHouse/replicate_default_args
Continuation of #14791
2020-09-28 18:53:02 +03:00
tavplubix
421d3b0b86
Merge pull request #14760 from bharatnc/ncb/event_time_microsec_p2
add event_time_microseconds field for system.{trace,text,query,query_thread}_log tables
2020-09-28 18:34:21 +03:00
zhang2014
e5a6d06820 ISSUES-15365 try fix integration test failure 2020-09-28 21:15:02 +08:00
detailyang
6e70f2ce7a bugfix:aggregate:fix boundingratio args check 2020-09-28 19:56:42 +08:00
zhang2014
e40d23cfe5 ISSUES-15365 add ya make 2020-09-28 19:11:40 +08:00
Nikita Mikhaylov
72e4b10c00
Merge pull request #14842 from hexiaoting/hxt_dev
Fix bug for LineAsString Format
2020-09-28 14:02:23 +03:00
Nikita Mikhaylov
a4832ed5ce
Merge branch 'master' into HyaZz-master 2020-09-28 13:55:45 +03:00
zhang2014
d3047d0783 ISSUES-15365 fix attach mysql database failure 2020-09-28 18:02:30 +08:00
Amos Bird
8eb8175258
Fix multiple column transformers. 2020-09-28 17:53:39 +08:00
Nikita Mikhaylov
73573472e9
Merge pull request #15299 from CurtizJ/fix-mysql-hung
Fix hang of queries with a lot of subqueries to same mysql table
2020-09-28 12:51:35 +03:00
Anton Ivashkin
c2b692f2c7 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into multi_delete_in_s3 2020-09-28 12:00:34 +03:00
Anton Ivashkin
c18cdec77c Use S3 multi object delete
Some strange with s3_with_proxy test
2020-09-28 11:47:04 +03:00
tavplubix
ffc0499952
Merge pull request #15243 from ClickHouse/tavplubix-patch-1
Try fix race on Context::settings
2020-09-28 11:44:27 +03:00
alesapin
bc662ac4e0
Merge pull request #15293 from amosbird/bc3
Replace useless multiset with unordered_set
2020-09-28 10:30:04 +03:00
hustnn
a7024134dd Merge remote-tracking branch 'upstream/master' into zk_exist_fix2 2020-09-28 14:44:55 +08:00
Alexey Milovidov
57c6af479c Fix "Arcadia" 2020-09-28 03:26:15 +03:00
Alexey Milovidov
79f510d63a Fix error in LowCardinality; add test 2020-09-28 03:22:50 +03:00
Alexey Milovidov
a7ddd8489a Added format RawBLOB 2020-09-28 03:11:19 +03:00
Alexey Milovidov
b800a7fdf1 Add support for "Raw" column format for Regexp format 2020-09-28 01:09:32 +03:00
Alexander Tokmakov
b0d99217fb Merge branch 'master' into write_structure_of_table_functions 2020-09-27 14:26:47 +03:00
Alexander Tokmakov
3bc8d9daa9 fix sync 2020-09-27 14:10:45 +03:00
Alexander Tokmakov
34addcf61f add review suggestions 2020-09-26 22:18:28 +03:00
alexey-milovidov
bbbe51033d
Merge pull request #15321 from ClickHouse/query-obfuscator
Query obfuscator
2020-09-26 19:43:36 +03:00
tavplubix
df6221f7bf
Merge pull request #15302 from ClickHouse/tavplubix-patch-2
Fix ASTColumnsReplaceTransformer::Replacement::clone()
2020-09-26 19:10:00 +03:00
alesapin
59f2bbe2a3
Merge pull request #15304 from ClickHouse/fix_race_condition_on_rename
Fix race condition between write ahead log cleanup and rename
2020-09-26 10:43:26 +03:00
alesapin
adf8c94d70
Merge pull request #15300 from ClickHouse/fix_race_on_default_database
Fix for default database race on server startup
2020-09-26 10:43:17 +03:00
Alexey Milovidov
da8a938069 Replace unit test to functional test 2020-09-26 09:50:34 +03:00
feng lv
6d5cd95d3f fix 2020-09-26 14:48:59 +08:00
alexey-milovidov
fc1cf426dd
Merge pull request #15048 from markpapadakis/patch-5
Update ForkProcessor.cpp
2020-09-26 07:35:31 +03:00
alexey-milovidov
d1fce06292
Merge pull request #15297 from hexiaoting/build_bugfix
fix build bug for merge_selector2.cpp
2020-09-26 07:32:33 +03:00
Alexey Milovidov
57b480e65d Fix PVS warning 2020-09-26 07:03:55 +03:00
Alexey Milovidov
5e146eda04 Fix error 2020-09-26 06:02:05 +03:00
Alexey Milovidov
f3349c8d13 Keywords are case-insensitive 2020-09-26 05:13:20 +03:00
Alexey Milovidov
c51502bce4 Remove words with dashes 2020-09-26 05:06:14 +03:00
Alexey Milovidov
f97d40584d Fix "Arcadia" 2020-09-26 04:10:26 +03:00
Alexey Milovidov
bda2c2f58f Remove sensitive words 2020-09-26 03:53:26 +03:00
Alexey Milovidov
fc10803def Fix comment 2020-09-26 03:50:56 +03:00
Alexey Milovidov
d78fc63deb Remove duplicate word 2020-09-26 03:50:00 +03:00
Alexey Milovidov
22861b0c3d Remove some profanity 2020-09-26 03:49:30 +03:00
Alexey Milovidov
673f72cf8c Remove some profanity 2020-09-26 03:48:09 +03:00
Alexey Milovidov
ed4ff51ed8 Integrate into clickhouse-format; add test and comments 2020-09-26 03:38:59 +03:00
Alexey Milovidov
62c2c2000c Query obfuscator: development 2020-09-26 03:00:54 +03:00
bharatnc
5c8c636cf1 Fix - allow empty tz in exp like toStartOf*(today()) 2020-09-25 11:03:16 -07:00
bharatnc
4f8266c7a1 Make trace_log test to pass 2020-09-25 10:39:37 -07:00
bharatnc
b27579e3f7 Use std::chrono instead of clock_gettime 2020-09-25 10:39:37 -07:00
bharatnc
24ea6b2380 TraceCollector - time s&us using same timespec 2020-09-25 10:39:37 -07:00
bharatnc
324a06145c EventTime - construct time in s and us from same timespec 2020-09-25 10:39:37 -07:00
bharatnc
f252523442 Query,QueryThread Logs - add event_time_microseconds field 2020-09-25 10:39:37 -07:00
bharatnc
b0d4fe5114 TraceLog - add field event_time_microseconds 2020-09-25 10:39:37 -07:00
bharatnc
a351980717 TextLog - add event_time_microseconds field 2020-09-25 10:39:37 -07:00
alexey-milovidov
6ec5bcc88f
Merge pull request #15286 from zlobober/public_create_block_for_set
Put createBlockForSet declaration to header file.
2020-09-25 20:12:44 +03:00
Maxim Akhmedov
6914a86673 Put createBlockForSet declaration to header file. 2020-09-25 17:49:54 +03:00
Alexander Tokmakov
68669b8a92 Merge branch 'master' into replicate_default_args 2020-09-25 15:52:17 +03:00
alesapin
f5c38b34ae Fix race condition during write ahead log cleanup and rename 2020-09-25 15:07:38 +03:00
tavplubix
d4c8ba8410
Update ASTColumnsTransformers.h 2020-09-25 14:58:04 +03:00
Alexander Tokmakov
8f21c73b76 Revert "Update ASTColumnsTransformers.h"
This reverts commit b30e860166.
2020-09-25 14:52:55 +03:00
tavplubix
b30e860166
Update ASTColumnsTransformers.h 2020-09-25 14:51:00 +03:00
alesapin
97671a1771 Ugly fix for default database race 2020-09-25 14:27:00 +03:00
Anton Popov
085f63a9bf fix hang of queries with a lot of subqueries to same mysql table 2020-09-25 14:12:48 +03:00
tavplubix
b96157e32b
Merge branch 'master' into enable_atomic_database_by_default 2020-09-25 13:42:39 +03:00
hexiaoting
4666ae3211 fix build bug for merge_selector2.cpp 2020-09-25 17:59:19 +08:00
Amos Bird
81d08b59e5
Replace useless multiset with unordered_set 2020-09-25 16:38:09 +08:00
feng lv
87726a7af2 function allow an argument with timezone 2020-09-25 15:20:42 +08:00
alesapin
2bacb6f6c8
Merge pull request #15220 from amosbird/fixjgc
Fix joinGet crash with LowCardinality types
2020-09-25 10:09:05 +03:00
alexey-milovidov
9446e43146
Merge pull request #15258 from ClickHouse/fix-msan-failure-2
Fix MSan report in QueryLog
2020-09-25 08:06:45 +03:00
alexey-milovidov
13529265c4
Revert "Avoid deadlocks in Log/TinyLog" 2020-09-25 02:28:57 +03:00
Alexey Milovidov
e51d9dd2e7 Fix MSan report in QueryLog 2020-09-25 02:24:58 +03:00
Alexander Tokmakov
fb826557b7 Merge branch 'master' into enable_atomic_database_by_default 2020-09-25 02:04:01 +03:00
tavplubix
2566478349
Merge pull request #15240 from ClickHouse/fix-random-seed
Fix strange code
2020-09-25 01:26:22 +03:00
Nikita Mikhaylov
788b62b931
Merge pull request #14879 from nikitamikhaylov/yet_another_dictionary_bugfix
Segfault in CacheDictionary
2020-09-24 20:23:42 +03:00
tavplubix
5ff5ca1fa6
Update Context.cpp 2020-09-24 20:08:15 +03:00
tavplubix
ab01fb36da
Update ProcessList.cpp 2020-09-24 20:05:13 +03:00
Alexey Milovidov
dce6a436f3 Fix strange code 2020-09-24 19:35:17 +03:00
Vitaly Baranov
0605808ed1
Merge pull request #14492 from vitlibar/use-previous-users-xml-if-failed-to-reload
Use previous users.xml if failed to reload
2020-09-24 16:25:19 +03:00
Alexander Kuzmenkov
813a876e93
Merge pull request #14962 from ClickHouse/log-avoid-deadlock
Avoid deadlocks in Log/TinyLog
2020-09-24 13:54:11 +03:00
Amos Bird
a2845acf37
Fix joinGet crash with LowCardinality types. 2020-09-24 17:29:52 +08:00
Nikolai Kochetov
e0c7cf842b
Merge pull request #15201 from ClickHouse/explain-optimize-pipeline-setting
Add optimize setting to explain plan.
2020-09-24 10:35:50 +03:00
alesapin
96c419974c Merge branch 'master' into codecs_for_nested_types 2020-09-24 10:28:20 +03:00
alesapin
7e6536097f
Merge pull request #15192 from ClickHouse/fix_storage_buffer_alter
Fix insert into storage Buffer after alter.
2020-09-24 10:22:58 +03:00
tavplubix
aff7ec2df5
Merge pull request #15152 from maqroll/master
decimal field in mysql column definition packet shouldn't have size 1?
2020-09-23 22:22:40 +03:00
Alexander Tokmakov
f0f91f84af Merge branch 'master' into enable_atomic_database_by_default 2020-09-23 21:50:15 +03:00
Alexander Tokmakov
a8266a3d3c fix DDDL with cross-replication and Atomic 2020-09-23 21:28:59 +03:00
Nikolai Kochetov
74e785a2fe Add optimize setting to explain plan. 2020-09-23 20:42:46 +03:00
alexey-milovidov
3bc09bb867
Merge pull request #15178 from amosbird/dtcf
Fix performance issue when doing datetime comparison
2020-09-23 20:33:23 +03:00
alesapin
39f23fbb8f
Merge pull request #14711 from myrrc/feature/cmake-flags-doc-generator
CMake flags reference generator, the guide for adding new options, and the attempt to correct the existing options
2020-09-23 18:56:32 +03:00
alesapin
87b64baea5 Fix again 2020-09-23 16:21:06 +03:00
alesapin
f404925397 More optimal 2020-09-23 15:19:45 +03:00
alesapin
1eb8f0a495 Fix bug in storage buffer during alter 2020-09-23 15:06:54 +03:00
alesapin
74671c909c Merge branch 'master' into codecs_for_nested_types 2020-09-23 13:12:23 +03:00
alesapin
bb4b739f43 Fix PVS-Studio warning 2020-09-23 13:11:48 +03:00
alesapin
50eba27d1a
Update CompressionFactory.h 2020-09-23 10:24:25 +03:00
Alexander Kuzmenkov
1321832673
Merge pull request #15135 from amosbird/ctefix
Fix ASTWithElement clone.
2020-09-23 09:51:55 +03:00
Alexander Kuzmenkov
27036f5079
Merge pull request #15158 from ClickHouse/aku/rethrow-copy
Rethrow copy of exception in SSD dictionaries
2020-09-23 09:50:26 +03:00
Amos Bird
ee45395942
Fix datetime comparison 2020-09-23 10:29:55 +08:00
alexey-milovidov
31ece3bda5
Merge pull request #15145 from 4ertus2/debt
Better min/max for Int128
2020-09-23 02:03:51 +03:00
filimonov
cc24ef9f83
Better debug message from MergeTreeDataSelectExecutor
See #15168
2020-09-22 21:35:29 +02:00
Alexander Kuzmenkov
74f3c592c7
update comment after #14940 2020-09-22 19:20:37 +03:00
Alexander Kuzmenkov
0752fa0137 Rethrow copy of exception in SSD dictionaries
A follow-up to https://github.com/ClickHouse/ClickHouse/pull/14689 ,
that should fix asan reports in stress test in master.
2020-09-22 18:54:11 +03:00
Alexander Tokmakov
7fcf20e48c enable more tests 2020-09-22 17:29:09 +03:00
alesapin
269d225637 Merge branch 'master' into myrrc-feature-cmake-flags-doc-generator 2020-09-22 17:00:02 +03:00
alesapin
7015f272e7 Better comment 2020-09-22 16:48:38 +03:00
Alexander Kuzmenkov
679c0988a5 Merge remote-tracking branch 'origin/master' into HEAD 2020-09-22 16:25:52 +03:00
Nikolai Kochetov
62ef40394a
Merge pull request #14856 from ClickHouse/query-plan-for-creating-sets
Query plan for creating sets
2020-09-22 16:19:20 +03:00
maqroll
85cf3e798a decimal field in column definition packet shouldn't have size 1? 2020-09-22 15:05:43 +02:00
alesapin
fd394f699c Better comments 2020-09-22 15:49:55 +03:00
alesapin
96d06c6ae6 Less changes 2020-09-22 15:16:15 +03:00
Artem Zuikov
2e6edc250c better min/max for Int128 2020-09-22 14:35:46 +03:00
Amos Bird
def110f835
Fix ASTWithElement clone. 2020-09-22 19:05:23 +08:00
Alexander Kuzmenkov
dde4cf70e1 Merge remote-tracking branch 'origin/master' into HEAD 2020-09-22 14:03:59 +03:00
Alexander Kuzmenkov
478c7309d4
Merge pull request #14843 from ClickHouse/aku/global-pool
Exception on double init of global thread pool
2020-09-22 13:44:39 +03:00
Artem Zuikov
2280482530
Keep LowCardinality in MergeJoin left key result type (#15088) 2020-09-22 13:16:22 +03:00
Alexander Kuzmenkov
c0241b617a Merge remote-tracking branch 'origin/master' into HEAD 2020-09-22 13:14:31 +03:00
Alexander Tokmakov
dc1cc11764 Merge branch 'master' into enable_atomic_database_by_default 2020-09-22 12:40:32 +03:00
alesapin
b57708c3dd Merge branch 'master' into codecs_for_nested_types 2020-09-22 09:52:31 +03:00
hustnn
3a59a7d5b6 Remove root path before adding to watches 2020-09-22 09:28:13 +08:00
Niu Zhaojie
76e0147bf4 Use opt num to identify request type 2020-09-22 09:28:12 +08:00
hustnn
35f1db0043 Fix potential memory leak caused by zookeeper exist calling 2020-09-22 09:28:12 +08:00
Alexander Tokmakov
91cbd49e85 fix tests 2020-09-22 00:09:50 +03:00
Nikita Mikhaylov
123f389f1a
Merge pull request #14689 from ClickHouse/aku/broken-perf
Add context to errors in ExpressionAction
2020-09-21 23:33:42 +03:00
Alexey Milovidov
1e65105b06 Merge branch 'master' into log-avoid-deadlock 2020-09-21 21:09:06 +03:00
alexey-milovidov
e29c4c3cc4
Merge pull request #15024 from ClickHouse/fix_renameat2_einval_on_macos
Fix EINVAL in renameat2 on MacOS
2020-09-21 21:08:17 +03:00
alexey-milovidov
c240f464d2
Merge pull request #15061 from azat/UAF-from-ThreadPool-jobs
Fix possible UAF from jobs in the ThreadPool on shutdown
2020-09-21 20:49:46 +03:00
alexey-milovidov
10d3a1d730
Update ThreadPool.cpp 2020-09-21 20:49:15 +03:00
alesapin
263d570f7e
Merge pull request #15076 from ClickHouse/fix_modifying_ast_create_in_database_memory
Fix data race when using DatabaseMemory::getCreateTableQueryImpl(...)
2020-09-21 20:49:06 +03:00
alesapin
e87b961095 Trying to add same logic for compact parts 2020-09-21 20:35:09 +03:00
Nikolai Kochetov
c77a6967a3 Remove commented code. 2020-09-21 19:22:23 +03:00
alesapin
fbd6f10ff7 Use default codec instead of throwing exception 2020-09-21 19:07:14 +03:00
alesapin
bcea1a4207 Remove redundant change 2020-09-21 18:55:21 +03:00
alesapin
8517dda807 First working test 2020-09-21 17:47:10 +03:00
alesapin
4debccb7fe Almost working version 2020-09-21 17:22:13 +03:00
Nikolai Kochetov
334289fd57
Merge pull request #15082 from ClickHouse/try-improve-array-has-built-time
Update SourceVisitor for GatherUtils.
2020-09-21 17:08:14 +03:00
Nikolai Kochetov
0f3eab1e48
Merge pull request #15084 from amosbird/lef
Don't hide interface symbols
2020-09-21 17:07:52 +03:00
Amos Bird
741912cafd
Don't hide interface symbols 2020-09-21 21:03:17 +08:00
tavplubix
fd2cc343e6
Update renameat2.cpp 2020-09-21 15:37:34 +03:00
alesapin
a3e999784b Buildable code 2020-09-21 14:24:10 +03:00
tavplubix
7f629c8185
Update DatabaseMemory.cpp 2020-09-21 14:12:45 +03:00
Alexander Tokmakov
13e41cdd20 try enable atomic database by default 2020-09-21 13:55:34 +03:00
Nikolai Kochetov
efa493fbcb Update SourceVisitor for GatherUtils. 2020-09-21 13:29:31 +03:00
tavplubix
450c1a5709
Update renameat2.cpp 2020-09-21 13:19:02 +03:00
Alexander Tokmakov
ef85ce90c3 fix build 2020-09-21 13:17:33 +03:00
Nikolai Kochetov
54c1f04b38 Review fixes. 2020-09-21 11:36:12 +03:00
alesapin
387b97aba5 Merge branch 'master' into codecs_for_nested_types 2020-09-21 11:06:28 +03:00
alesapin
7fc5a2dae1 Some intermediate stage 2 2020-09-21 10:18:23 +03:00
alesapin
09f6a454ae Some intermediate stage 2020-09-21 10:17:58 +03:00
alexey-milovidov
3e0509af69
Merge pull request #13573 from amosbird/rms
specific ReplicatedMergeTree settings
2020-09-21 01:36:33 +03:00
Azat Khuzhin
30f96dc525 Fix possible UAF from jobs in the ThreadPool on shutdown
ThreadPoolImpl<>::worker signaling that job is done while still storing
std::function<> object, and this can lead to problems on shutdown, since
in in this cast std::function<> can refers to some global/static object
that had been already destroyed (typical example is Logger).

I believe that this is exactly what TSAN reports about (decoded
manually, since llvm-symbolizer does not work in the test env):

<details>

```

- 2020-09-20 17:44:43   Write of size 8 at 0x7b1000008f78 by main thread (mutexes: write M1432):
    operator delete(void*, unsigned long)
    ??:0:0

    Poco::Logger::~Logger()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:39:1
    non-virtual thunk to Poco::Logger::~Logger()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:0:0

    Poco::RefCountedObject::release() const
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/RefCountedObject.h:82:24
    Poco::Logger::shutdown()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:346:16

    Poco::AutoLoggerShutdown::~AutoLoggerShutdown()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:459:4

    cxa_at_exit_wrapper(void*)
    ??:0:0

```

```
- 2020-09-20 17:44:43   Previous atomic read of size 4 at 0x7b1000008f78 by thread T116:

    __tsan_atomic32_load
    ??:0:0

    int std::__1::__cxx_atomic_load<int>(std::__1::__cxx_atomic_base_impl<int> const*, std::__1::memory_order)
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:970:12
    std::__1::__atomic_base<int, false>::load(std::__1::memory_order) const
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1487:17
    std::__1::__atomic_base<int, false>::operator int() const
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1491:53
    Poco::Logger::is(int) const
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Logger.h:1264:9
    MemoryTracker::logPeakMemoryUsage() const
    /build/obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:59:5

    MemoryTracker::~MemoryTracker()
    /build/obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:46:13

    DB::ThreadGroupStatus::~ThreadGroupStatus()
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadStatus.h:51:7
    std::__1::__shared_ptr_emplace<DB::ThreadGroupStatus, std::__1::allocator<DB::ThreadGroupStatus> >::__on_zero_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3602:23

    std::__1::__shared_count::__release_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3440:9
    std::__1::__shared_weak_count::__release_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3482:27
    std::__1::shared_ptr<DB::ThreadGroupStatus>::~shared_ptr()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:4207:19
    DB::PipelineExecutor::executeImpl(unsigned long)::$_4::~$_4()
    /build/obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:720:34
    ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'()::~()
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:161:54
    std::__1::__compressed_pair_elem<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), 0, false>::~__compressed_pair_elem()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2188:8
    std::__1::__function::__alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::Pip>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1559:38
    std::__1::__function::__func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineE>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1706:10

    std::__1::__function::__value_func<void ()>::~__value_func()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1828:19
    std::__1::function<void ()>::~function()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2460:43
    ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>)
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:268:5

    void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:116:73
    decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleI>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3519:1
    void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'(>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:273:5
    void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned lon>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:284:5

    __tsan_thread_start_func
    ??:0:0
```

```
- 2020-09-20 17:44:43   Mutex M1432 (0x0000181213a8) created at:
    pthread_mutex_init
    ??:0:0

    Poco::MutexImpl::MutexImpl()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Mutex_POSIX.cpp:64:6

    Poco::Mutex::Mutex()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Mutex.cpp:34:8

    __cxx_global_var_init
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:28:15
    _GLOBAL__sub_I_Logger.cpp
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:0:0

    __libc_csu_init
    ??:0:0
```

</details>
2020-09-20 23:46:28 +03:00
alexey-milovidov
ffe4f4b366
Update ForkProcessor.cpp 2020-09-20 22:59:26 +03:00
Mark Papadakis
25a1f82631
Update ForkProcessor.cpp
Got rid of an unnecessary local
2020-09-20 20:26:44 +03:00
alexey-milovidov
8630d1baf4
Merge pull request #15033 from amosbird/fixstringkey
Fix incorrect key condition of fixed strings.
2020-09-20 19:51:24 +03:00
alexey-milovidov
85483f8532
Merge pull request #14853 from ClickHouse/akz/optimized_index_binary_search
Optimized marks selection algorithm for continuous marks ranges
2020-09-20 19:48:45 +03:00
alexey-milovidov
f7cae4a57d
Merge pull request #14975 from ClickHouse/fix_merge_tree_settings_check
Fix check for merge tree settings sanity on server startup
2020-09-20 19:43:40 +03:00
alexey-milovidov
55eb533fda
Merge pull request #15034 from amosbird/fff
Improve performance of grouping by single LowCardinality(FixedString)
2020-09-20 19:42:38 +03:00
alexey-milovidov
7b35a6a59a
Merge pull request #15028 from ClickHouse/fix-bar
Fix buffer overflow in "bar" function
2020-09-20 19:04:15 +03:00
alexey-milovidov
3f5d7843f6
Merge pull request #15029 from ClickHouse/cast-single-argument
Proper exception message for wrong number of arguments of CAST
2020-09-20 19:03:47 +03:00
alexey-milovidov
068e9f372e
Merge pull request #15030 from ClickHouse/base64-unpoison
Maybe fix MSan report in base64
2020-09-20 19:03:28 +03:00
Vitaly Baranov
46ab2f76cd
Merge pull request #14919 from vitlibar/add-login-function-to-access-storage
Add login() function to IAccessStorage
2020-09-20 16:58:22 +03:00
alesapin
5ffee8808a
Alter remove column properties and TTLs (#14742) 2020-09-20 16:27:33 +03:00
Amos Bird
8ca4d6634e
better fixed string group by support 2020-09-20 15:20:09 +08:00
Amos Bird
fcee786320
Fix incorrect key condition of fixed strings. 2020-09-20 11:53:29 +08:00
alexey-milovidov
1474b80ecc
Merge pull request #15008 from ClickHouse/clang-tidy-11-without-enabling
Fix terribly wrong code
2020-09-20 02:00:54 +03:00
Alexey Milovidov
0029c5512d Maybe fix MSan report in base64 2020-09-19 22:43:26 +03:00
Alexey Milovidov
5a47e64685 Proper exception message for wrong number of arguments of CAST 2020-09-19 22:34:24 +03:00
Alexey Milovidov
dae3b40a55 Fix buffer overflow in "bar" function 2020-09-19 22:15:16 +03:00
alexey-milovidov
c185db46f9
Merge pull request #15023 from ClickHouse/remove-opencl
Remove OpenCL
2020-09-19 21:31:40 +03:00
myrrc
146dd0f3f0 fixed comments parser 2020-09-19 20:14:37 +03:00
myrrc
c78d9f9f38 moved the generators to docs/ folder 2020-09-19 19:42:36 +03:00
alexey-milovidov
cd7354117c
Update StorageJoin.cpp 2020-09-19 19:38:36 +03:00
alexey-milovidov
7ae4d01744
Update convertMySQLDataType.cpp 2020-09-19 19:31:08 +03:00
Alexey Milovidov
9fa49a6843 Fix clang-tidy 2020-09-19 19:29:15 +03:00
Alexey Milovidov
c7a98132ac Slightly better code 2020-09-19 18:13:51 +03:00
Alexey Milovidov
b87ecc2ec1 Even more 2020-09-19 18:09:12 +03:00
Alexey Milovidov
c304032707 Even more 2020-09-19 18:02:11 +03:00
Alexey Milovidov
772eccf03d Move code to .cpp 2020-09-19 17:55:08 +03:00
Alexey Milovidov
059646743b Remove OpenCL 2020-09-19 17:20:00 +03:00
Alexander Tokmakov
46aa307d0a fix EINVAL in renameat2 on MacOS 2020-09-19 17:19:06 +03:00
alexey-milovidov
2cab30ba8d
Merge pull request #14521 from hexiaoting/dev_like
Add new feature: SHOW DATABASES like 'xxx'
2020-09-19 16:34:37 +03:00
alexey-milovidov
acbdc51460
Merge pull request #14771 from amosbird/withsub
First attempt of supporting CTE
2020-09-19 16:25:34 +03:00
Vitaly Baranov
33efb36714 Use previous users.xml if failed to reload. 2020-09-19 16:21:12 +03:00
Alexander Tokmakov
de540c6955 minor fixes 2020-09-19 15:59:37 +03:00
Alexey Milovidov
76147cc0c1 Generate ya.make 2020-09-19 15:52:17 +03:00
Alexey Milovidov
3003ebabc2 Merge branch 'master' into Vxider-disable_set_and_join_persistency 2020-09-19 15:51:40 +03:00
alexey-milovidov
eb9ee723c3
Merge pull request #11913 from ClickHouse/compact-parts-by-default
Enable compact parts by default for small parts
2020-09-19 15:31:15 +03:00
Alexey Milovidov
2264622f69 Fix clang-tidy 2020-09-19 15:24:43 +03:00
tavplubix
6e0bdaf46d
Merge pull request #14535 from zhang2014/fix/datetime
ISSUES-4006 support decimal data type for MaterializedMySQL
2020-09-19 14:05:32 +03:00
Alexey Milovidov
4d06bc62b1 Allow concurrent reads while writing to TinyLog 2020-09-19 14:02:11 +03:00
Alexey Milovidov
111acdc63b Continuation 2020-09-19 14:02:11 +03:00
Alexey Milovidov
bc8fc3e280 Simplification 2020-09-19 14:02:11 +03:00
Alexey Milovidov
66ea3c0c43 Remove useless method 2020-09-19 14:02:11 +03:00
Alexey Milovidov
8193da847b Simplification 2020-09-19 14:02:11 +03:00
Alexey Milovidov
3bb0088c86 Whitespace 2020-09-19 14:02:11 +03:00
Alexey Milovidov
dc7e3b75d3 Whitespace 2020-09-19 14:02:11 +03:00
Alexey Milovidov
8a8284ce31 Avoid deadlocks in Log/TinyLog 2020-09-19 14:02:11 +03:00
alexey-milovidov
65517da62b
Merge pull request #14495 from nikitamikhaylov/update-permutation-bugfix-3
updatePermumation with Nullable
2020-09-19 13:53:55 +03:00
alexey-milovidov
c461d782e4
Merge pull request #14929 from ClickHouse/mlock-code-segment
Only mlock code segment
2020-09-19 13:46:49 +03:00
alexey-milovidov
0d26228599
Merge pull request #15002 from 4ertus2/bugs
Fix crash in RIGHT or FULL JOIN switch
2020-09-19 13:45:10 +03:00
alexey-milovidov
1fcebce926
Merge pull request #15000 from Jokser/disable-ttl-move-on-insert
Option to disable TTL move on data part insert
2020-09-19 13:45:02 +03:00
alexey-milovidov
3463d97f8c
Merge pull request #14973 from amosbird/bm2
Ignore key constraints when doing mutations.
2020-09-19 13:44:30 +03:00
alexey-milovidov
20a294e392
Merge pull request #14959 from ClickHouse/storages-small-improvements
Small improvements for IStorage::rename
2020-09-19 13:44:08 +03:00
Alexey Milovidov
eb260f6a89 Fix bad code 2020-09-19 13:43:26 +03:00
Alexey Milovidov
1238822785 Fix clang-tidy 2020-09-19 13:37:40 +03:00