thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
39,599 Commits 9,319 Branches 1,783 Tags 2.1 GiB
f8d3999fa2
Commit Graph

729 Commits

Author SHA1 Message Date
alexey-milovidov
436e4e3f5e
Merge pull request #10502 from oandrew/avro-nested2 
Avro nested fields - support complex types
 2020-04-27 13:44:58 +03:00
Andrew Onyshchuk
570be94a48 Use correct type for DateTime64 (ubsan) 2020-04-26 21:10:30 -05:00
Alexey Milovidov
cac130ad3b Use time zone when comparing DateTime with string literal 2020-04-27 01:13:51 +03:00
Alexey Milovidov
a7d7dc5034 Fix some bad code 2020-04-26 20:34:36 +03:00
alexey-milovidov
c6ae6a7ba8
Merge pull request #10418 from ClickHouse/fix_logical_error_in_diagnostic_info 
Better diagnostic info in input formats
 2020-04-26 20:26:59 +03:00
alexey-milovidov
4c17542096
Merge pull request #10465 from kekekekule/ddl_settings 
Add custom settings support in DDL-queries for CREATE DICTIONARY
 2020-04-26 20:25:29 +03:00
alexey-milovidov
7d58dc154c
Update ASTDictionary.h 2020-04-26 20:24:18 +03:00
alexey-milovidov
0a9bcae0ed
Merge pull request #10511 from ClickHouse/fix-ubsan-datetime64 
Fixed UBSan for DateTime64; added a test
 2020-04-26 20:17:01 +03:00
alexey-milovidov
378d73d477
Merge pull request #10449 from azat/metrics-for-new-bg-schedule-pools 
Add tasks/memory metrics for distributed/buffer schedule pools
 2020-04-26 20:16:02 +03:00
Alexey Milovidov
a70637ab4b Fixed UBSan; added a test 2020-04-26 17:57:45 +03:00
alexey-milovidov
d1464a441f
Merge pull request #10503 from oandrew/read-columns-perf-fix 
IRowInputFormat performance fix
 2020-04-26 17:36:54 +03:00
alexey-milovidov
62e2f10959
Merge pull request #10474 from hczhcz/patch-0422 
Fix overflow in simpleLinearRegression
 2020-04-26 17:34:57 +03:00
Vitaliy Zakaznikov
626468d255 Adding support for streaming mode to JSONEachRowRowOutputFormat 2020-04-26 15:56:45 +02:00
Vitaliy Zakaznikov
369b4d53ef Adding support for output_format_enable_streaming format setting. 2020-04-26 15:44:11 +02:00
Andrew Onyshchuk
4a5497498e IRowInputFormat performance fix 2020-04-25 19:33:36 -05:00
Alexey Milovidov
d01bd44f14 Merge branch 'master' into execute-query-flush-format-2 2020-04-26 03:31:46 +03:00
alexey-milovidov
8820b43606
Merge pull request #10479 from Avogar/markdown_format 
Markdown format
 2020-04-26 03:25:48 +03:00
Andrew Onyshchuk
e7ba75ce98 Avro nested fields - support complex types 2020-04-25 17:42:49 -05:00
alexey-milovidov
8b1e504b19
Merge pull request #10497 from azat/kafka 
Tiny kafka fixes
 2020-04-25 23:50:22 +03:00
alexey-milovidov
092efc53ba
Merge pull request #10496 from ClickHouse/fix-visible-drift-memory-tracking 
Fix visible drift of MemoryTracking metric
 2020-04-25 23:50:08 +03:00
alexey-milovidov
1e25991c62
Merge pull request #10494 from kyprizel/master 
Split fuzzers and sanitizers
 2020-04-25 23:49:45 +03:00
alexey-milovidov
c9334d3fde
Merge pull request #10491 from azat/dist-shutdown 
Proper Distributed shutdown (fixes UAF, avoid waiting for sending all batches)
 2020-04-25 23:47:59 +03:00
Vitaliy Zakaznikov
71a40d6ec9 Revert "Working on adding AUTO REFRESH to LIVE VIEW tables." 
This reverts commit 855e06b031.
 2020-04-25 09:56:57 -04:00
Vitaliy Zakaznikov
1fc345f5a0 Revert "Reverting all the changes in LIVE VIEW table." 
This reverts commit 684c7f943a.
 2020-04-25 09:56:41 -04:00
Vitaliy Zakaznikov
e73166c78d Removing SquashingBlockOutputStream from the stream to 
allow blocks from WATCH query to propagate to target table
as is.
 2020-04-25 09:46:46 -04:00
Alexey Milovidov
c75c2fac29 Fix visible drift of MemoryTracking metric 2020-04-25 15:36:01 +03:00
Azat Khuzhin
40c65f7bee IRowInputFormat: omit "Skipped 0 rows with errors while reading the input stream" 
This will just pollute the output log.

And this is significant for kafka streaming, since it will print this
message for each message (due to how
ReadBufferFromKafkaConsumer/KafkaBlockInputStream works).

You can see this in the server log of the test_storage_kafka
(integration test), for the table with kafka_skip_broken_messages > 0,
it will have 52 lines of such message, 2 with "Skipped 1 rows" and 50
with "Skipped 0 rows" (since there are 52 messages in summary, 2
malformed and 50 correct).
 2020-04-25 15:16:29 +03:00
Vitaliy Zakaznikov
684c7f943a Reverting all the changes in LIVE VIEW table. 2020-04-25 07:39:15 -04:00
Vitaliy Zakaznikov
8621e93460 Starting to remove changes to LIVE VIEW table. 2020-04-25 07:37:33 -04:00
Vitaliy Zakaznikov
34a03939ea First changes to try to support INSERT INTO table WATCH query. 2020-04-25 07:33:47 -04:00
Eldar Zaitov
d7a037c26b split fuzzers and sanitizers 2020-04-25 10:12:38 +03:00
Azat Khuzhin
747a74215f Avoid processing all batches before Distributed shutdown 2020-04-25 02:03:27 +03:00
Azat Khuzhin
8ad6b37913 Proper StorageDistributed shutdown to avoid UAF in DistributedMonitor 
StorageDistributed::shutdown() does not acquire the lock, that controls
access to the cluster_nodes_data, thus it does not synced with the
requireDirectoryMonitor(), hence some monitors can be untracked that
will trigger UAF (use-after-free) after DROP TABLE dist:

This is for the SIGSEGV from the DirectoryMonitor (with already destroyed storage):
    0  0x0000000008e9f760 in std::__1::__cxx_atomic_load<int> (__order=std::__1::memory_order::seq_cst, __a=0x0)
    1  std::__1::__atomic_base<int, false>::load (__m=std::__1::memory_order::seq_cst, this=0x0) <-- this is nullptr
    2  std::__1::__atomic_base<int, false>::operator int (this=0x0)
    3  DB::ActionBlocker::isCancelled (this=0x7f85e31c9bb8) at ../src/Common/ActionBlocker.h:18
    4  DB::StorageDistributedDirectoryMonitor::run (this=0x7f85f93b2a00) at ../src/Storages/Distributed/DirectoryMonitor.cpp:140
 2020-04-25 02:03:26 +03:00
Vitaliy Zakaznikov
855e06b031 Working on adding AUTO REFRESH to LIVE VIEW tables. 2020-04-24 17:15:17 -04:00
tavplubix
7928f2eaa1
Merge pull request #7512 from ClickHouse/database_atomic 
DatabaseAtomic
 2020-04-24 22:28:18 +03:00
Alexander Tokmakov
04d4130b0d update yamake 2020-04-24 20:14:10 +03:00
Артем Стрельцов
5423506393 sure this will exactly fix the fail with build check 2020-04-24 19:04:54 +03:00
Alexander Tokmakov
53dcce55e9 better diagnostic info in input formats 2020-04-24 15:03:48 +03:00
Avogar
778f1aa1e7 Merge branch 'markdown_format' of github.com:avogar/ClickHouse into markdown_format 2020-04-24 14:07:04 +03:00
Avogar
2458da802c Update MarkdownRowOutputFormat 2020-04-24 14:06:41 +03:00
alesapin
793f4b734a Remove obsolete comment 2020-04-24 13:31:03 +03:00
alesapin
b4425c2dc4 Remove getColumn and hasColumn methods from IStorage 2020-04-24 13:20:03 +03:00
Артем Стрельцов
26a6353284 tried to fix build fail & fixed codestyle 2020-04-24 12:59:49 +03:00
alesapin
0c622370c5 Better 2020-04-24 12:21:38 +03:00
alesapin
dc2dd77d2e Remove redundant overrides from IStorage 2020-04-24 12:20:09 +03:00
alesapin
a2cb67c542
Merge pull request #10459 from ClickHouse/refactor_istorage 
Remove virtuals from IStorage as separate column
 2020-04-24 11:42:04 +03:00
hcz
541cd638ba Fix overflow in simpleLinearRegression 2020-04-24 16:33:09 +08:00
alexey-milovidov
79d531af87
Merge pull request #10466 from ClickHouse/aku/arena-infinite-loop 
Work around a bug leading to an infinite loop in addressToLine
 2020-04-24 10:38:07 +03:00
Alexander Kuzmenkov
2bedc82c18 Work around a bug leading to an infinite loop in addressToLine. 
Fixing this is hopeless and would entail rewriting the entire
WriteBuffer thing, and also this hack is something we can backport.
 2020-04-24 02:42:04 +03:00
Артем Стрельцов
aab2e8f9c1 unsignificant change 2020-04-24 02:21:51 +03:00
tavplubix
7d6ab32eb6
Update DatabaseCatalog.cpp 2020-04-24 01:08:26 +03:00
Артем Стрельцов
d7e27d4326 added SETTINGS section to DDL-query for CREATE DICTIONARY 2020-04-24 00:57:40 +03:00
Артем Стрельцов
f39fdf7182 some progress in parsing settings (i did this commit for having something to backup to) 2020-04-23 23:50:01 +03:00
Alexander Tokmakov
8be4d51111 fix build 2020-04-23 22:50:05 +03:00
Alexey Milovidov
2b569cf260 Added clarification in exception message #10405 2020-04-23 21:31:37 +03:00
Alexander Tokmakov
15e7ce0053 fixes 2020-04-23 21:00:43 +03:00
Alexander Tokmakov
f413061e0d remove Context from some IDatabase methods 2020-04-23 19:51:48 +03:00
alesapin
5ae25b68e4 Remove redundant virtual from function 2020-04-23 18:18:33 +03:00
alesapin
b0a73589b0 Remove virtual columns as separate field from IStorage 2020-04-23 18:09:00 +03:00
Alexander Tokmakov
04d6b59ac0 Merge branch 'master' into database_atomic 2020-04-23 17:31:37 +03:00
Артем Стрельцов
5da371b9c0 added settings parsing from DDL-query (they aren't applied now) 2020-04-23 15:39:27 +03:00
alexey-milovidov
1ecfa27cd3
Merge pull request #10441 from ClickHouse/remove_casts_from_defaults 
Remove CAST from DEFAULT expressions during table creation.
 2020-04-23 12:36:12 +03:00
Nikita Mikhaylov
3520755ab5 fix c++20 usage 2020-04-23 12:00:43 +03:00
Azat Khuzhin
4f5ec14f27 Configure thread names for buffer/distributed schedule pools 2020-04-23 03:16:14 +03:00
Azat Khuzhin
d854049d21 Add tasks/memory metrics for distributed/buffer schedule pools 
Follow-up-for: #10315
Follow-up-for: #10263
 2020-04-23 02:18:37 +03:00
Azat Khuzhin
c04c73b4c4 Drop unused MemoryTrackingForMerges from Context 
It set in src/Storages/MergeTree/MergeList.cpp anyway.
 2020-04-23 01:57:33 +03:00
alexey-milovidov
77969d11a5
Merge pull request #10442 from zhang2014/fix/create_database_failure 
Not remove metadata directory when attach database fails
 2020-04-23 01:07:13 +03:00
alexey-milovidov
b8e3655f0d
Merge pull request #10445 from azat/logger-source-coloring-client 
Make logger name colored in client too
 2020-04-23 00:35:40 +03:00
alexey-milovidov
fd27ce619b
Merge pull request #10444 from ClickHouse/server-memory-profiler 
Simple server wide memory profiler
 2020-04-23 00:34:23 +03:00
alexey-milovidov
4f6bb00424
Merge pull request #10420 from ClickHouse/clang-tidy-8 
Fixes for clang-tidy on clang-10
 2020-04-23 00:33:41 +03:00
Alexander Tokmakov
e02b5790af fix 2020-04-22 23:49:13 +03:00
Alexander Tokmakov
1833ac7f16 fixes 2020-04-22 23:43:10 +03:00
Nikita Mikhaylov
3cf2016867
Merge pull request #10188 from nikitamikhaylov/quorum_bug 
insert_quorum vs drop partition or ttl
 2020-04-23 00:11:57 +04:00
alexey-milovidov
9e14603ae5
Update DatabaseFactory.cpp 2020-04-22 22:37:30 +03:00
Azat Khuzhin
8715cef94c Make logger name colored in client too 
Follow-up-for: #10343
 2020-04-22 22:34:35 +03:00
Alexey Milovidov
282b5785c1 Fixed error 2020-04-22 21:41:19 +03:00
Alexander Kuzmenkov
3d5809a4ac
Merge pull request #10340 from ClickHouse/aku/unstable-codec-test 
Copy less memory in squashing transform.
 2020-04-22 21:18:42 +03:00
Alexey Milovidov
73dd78bac1 Merge branch 'master' into clang-tidy-8 2020-04-22 21:04:47 +03:00
Alexey Milovidov
07dcf40a6a Simple server wide memory profiler 2020-04-22 20:52:21 +03:00
zhang2014
a622208284 Not remove metadata directory when attach database fails 2020-04-23 00:03:58 +08:00
Alexey Milovidov
d189375051 Fix Mac OS build 2020-04-22 17:56:31 +03:00
alesapin
45b8cb5b37 Remove casts from table creation by default 2020-04-22 17:29:38 +03:00
Alexander Tokmakov
fefbbd37df Merge branch 'master' into database_atomic 2020-04-22 17:02:30 +03:00
zhang2014
0a0555d4d7 ISSUES-9623 fix failure test case 2020-04-22 18:34:42 +08:00
Alexey Milovidov
d3418bbb02 Fix clang-tidy 2020-04-22 11:45:14 +03:00
Alexey Milovidov
8c9408f927 Checkpoint 2020-04-22 11:31:10 +03:00
zhang2014
bfb87f0eab ISSUES-9623 check the number and type of arguments when creating BloomFilter index 2020-04-22 16:17:09 +08:00
Alexey Milovidov
03e17405cd Checkpoint 2020-04-22 10:03:43 +03:00
Alexey Milovidov
603d132d45 Checkpoint 2020-04-22 09:34:20 +03:00
Alexey Milovidov
1e325a9fd9 Checkpoint 2020-04-22 09:22:14 +03:00
alexey-milovidov
7065a8ba9c
Merge pull request #10422 from azat/fallback_to_stale_replicas-over-skip_unavailable_shards 
Prefer fallback_to_stale_replicas over skip_unavailable_shards
 2020-04-22 09:20:25 +03:00
Alexey Milovidov
0a7edce036 Checkpoint 2020-04-22 09:01:33 +03:00
Alexey Milovidov
be22a4b94e Checkpoint 2020-04-22 08:39:31 +03:00
Alexey Milovidov
b9b5036749 Merge branch 'master' into clang-tidy-8 2020-04-22 07:54:27 +03:00
Alexey Milovidov
e6acb35d95 Added a fix from Vadim Plakhtinskiy @VadimPlh 2020-04-22 07:32:05 +03:00
alexey-milovidov
eb0a1ec755
Merge pull request #10415 from azat/dict-load-after-detach-attach 
Fix dictionary name (RELOAD and system.dictionaries) after DETACH/ATTACH
 2020-04-22 06:57:08 +03:00
alexey-milovidov
7690c34e5b
Merge pull request #10424 from ClickHouse/tsv-bom-extra-case 
Allow to parse BOM in TSV if the first column cannot contain BOM in its value
 2020-04-22 06:56:43 +03:00
Alexander Kuzmenkov
d1c5e38a21
Merge pull request #10385 from CurtizJ/tuple-literal 
Parse tuples of tuples as literals
 2020-04-22 05:49:14 +03:00
alexey-milovidov
98294dc9fa
Merge pull request #10421 from ClickHouse/max_server_memory_usage_as_server_level_setting 
Make max_server_memory_usage server level setting.
 2020-04-22 03:34:17 +03:00
Alexey Milovidov
c805ed4dea Fixes for clang-tidy on clang-10 2020-04-22 03:29:38 +03:00
Alexey Milovidov
e8a9c7d518 Allow to parse BOM in TSV if the first column cannot contain BOM in its value 2020-04-22 02:46:42 +03:00
Alexander Kuzmenkov
c232f641b4
Merge branch 'master' into aku/unstable-codec-test 2020-04-22 02:40:53 +03:00
alexey-milovidov
0754eb7e78
Merge pull request #10416 from 4ertus2/perf 
Improve HashJoin memory usage
 2020-04-22 01:44:46 +03:00
Azat Khuzhin
0cc9dbd35b Prefer fallback_to_stale_replicas over skip_unavailable_shards 
Patch from: @alex-zaitsev
 2020-04-22 01:42:23 +03:00
Alexey Milovidov
0482e2e3ea Implement suggestion from Sergey Veletsky 2020-04-22 01:40:18 +03:00
alexey-milovidov
fcf5c3c6e1
Merge pull request #10399 from ClickHouse/fix-distributed_group_by_no_merge-segfault 
Fix segfault with distributed_group_by_no_merge
 2020-04-22 01:28:02 +03:00
Ivan
555488a458
Fix build in Arcadia (#10413) 2020-04-21 23:23:44 +03:00
Artem Zuikov
96b9edae29 fix comment 2020-04-21 22:09:18 +03:00
Artem Zuikov
17b0641b02 improve HashJoin memory usage 2020-04-21 22:01:34 +03:00
Alexey Milovidov
33338b722a Fix Mac OS #10246 2020-04-21 21:49:28 +03:00
Azat Khuzhin
9173895f85 Fix dictionary name (RELOAD and system.dictionaries) after DETACH/ATTACH 
After DETACH/ATTACH (or at server startup) the following will be broken:
- SYSTEM RELOAD DICTIONARY db_name.dict_name
- system.dictionaries will not have database for those dictionaries

The problem is that after DETACH/ATTACH the dictionary information is
obtained from metadata/$db/$dict.sql, but it does not have database
name.

Fix this by passing database name to the getDictionaryConfigurationFromAST().

Introduced-in: #10234
 2020-04-21 21:03:12 +03:00
Alexander Kuzmenkov
fcfbe4bfea
Update SquashingTransform.h 2020-04-21 19:27:56 +03:00
Alexander Kuzmenkov
a32ef00bf1 less copying 2020-04-21 17:59:57 +03:00
Anton Popov
3ccff833c1 fix ConstantExpressionTemplate with arrays 2020-04-21 15:54:36 +03:00
Nikita Mikhaylov
2173bcf28e Merge branch 'master' of github.com:ClickHouse/ClickHouse into quorum_bug 2020-04-21 14:35:06 +03:00
Nikolai Kochetov
87a0b40ace Do not create extra totals and extremes ports. 2020-04-21 12:31:40 +03:00
Nikolai Kochetov
1d2388de39 Drop currently calculated totals and extremes before aggregation. 2020-04-21 11:45:28 +03:00
Alexey Milovidov
4cc0269de0 Fix flattening of Array(Tuple(...)) #10259 2020-04-21 02:44:51 +03:00
alexey-milovidov
4ecc86beca
Merge pull request #9950 from ClickHouse/aku/literal-columns 
Create unique columns for literals
 2020-04-21 02:30:17 +03:00
alexey-milovidov
2645dede2c
Merge pull request #10386 from palasonic1/palasonic-fix-read-hash-table 
Fix reading HashTable from buffer
 2020-04-21 02:26:47 +03:00
tavplubix
1a067484c8
Merge pull request #10137 from kekekekule/direct_queries 
Add custom settings support for certain dictionary sources
 2020-04-21 01:21:32 +03:00
Alexander Tokmakov
17c0317b80 minor fixes 2020-04-21 01:17:53 +03:00
alexey-milovidov
8c2839d3c9
Merge pull request #10308 from ClickHouse/shared-context-lifetime 
Shared context lifetime
 2020-04-21 00:12:45 +03:00
alexey-milovidov
fe1309e575
Merge pull request #10381 from ClickHouse/fix-concat-prepare 
Fix concat prepare
 2020-04-21 00:07:32 +03:00
alexey-milovidov
ab922d44d5
Merge pull request #10379 from ClickHouse/only_fetch_repliac 
Add setting always_fetch_merged_part
 2020-04-21 00:07:00 +03:00
alexey-milovidov
2c8576798b
Merge pull request #10354 from oandrew/avro-nested 
Avro nested fields insert support
 2020-04-20 23:36:02 +03:00
alexey-milovidov
3d4696d4ef
Update AvroRowInputFormat.cpp 2020-04-20 23:34:48 +03:00
alexey-milovidov
c4758b69a2
Merge pull request #10378 from hczhcz/patch-0420 
Add JSONExtractKeysAndValuesRaw
 2020-04-20 23:30:40 +03:00
alexey-milovidov
4912940009
Update FunctionsJSON.h 2020-04-20 23:28:54 +03:00
alexey-milovidov
2d799cf096
Merge pull request #10375 from azat/aggregator-drop-disk-reservation 
Drop disks reservation in Aggregator
 2020-04-20 23:22:14 +03:00
Maxim Serebryakov
a99a340b0b fix reading HashTable and HashSet 2020-04-20 23:19:03 +03:00
Anton Popov
e1b7e2de2f parse tuples of tuples as literals 2020-04-20 23:14:56 +03:00
alexey-milovidov
69b6f135e7
Merge pull request #10382 from excitoon-favorites/altercodec 
Allowed to alter column in non-modifying data mode when the same type is specified
 2020-04-20 22:54:53 +03:00
alexey-milovidov
f6de9c6ffd
Update AlterCommands.cpp 2020-04-20 22:54:08 +03:00
alexey-milovidov
1eca5088ec
Update AlterCommands.cpp 2020-04-20 22:53:19 +03:00
alexey-milovidov
faeab5f649
Merge pull request #10370 from amosbird/clang-10 
clang-10 fix
 2020-04-20 22:47:12 +03:00
Alexander Kuzmenkov
3ee89344af Statically dispatch on whether the argument is a Tuple 2020-04-20 18:37:28 +03:00
Ivan
d5b0a7e691
Fix Arcadia build (#10380) 2020-04-20 15:21:25 +03:00
Vladimir Chebotarev
82985ff594 Allowed to alter column in non-modifying data mode when the same type is specified. 2020-04-20 14:15:06 +03:00
Alexander Tokmakov
b29bddac12 Merge branch 'master' into database_atomic 2020-04-20 14:09:09 +03:00
Nikolai Kochetov
3dbc9b5562 Fix prepare for ConcatProcessor. 2020-04-20 14:03:29 +03:00
Nikita Mikhaylov
05206df88c better 2020-04-20 13:56:59 +03:00
alesapin
1bc84ec8b4 Fix comments 2020-04-20 13:56:23 +03:00
alesapin
8767ea7efa Add setting which restrict replica to merge parts by itself and always prefer dowloading from other replicas. 2020-04-20 13:53:29 +03:00
Nikolai Kochetov
c2c99b02bc Fix prepare for ConcatProcessor. 2020-04-20 13:36:38 +03:00
hcz
32573c0590 Add JSONExtractKeysAndValuesRaw 2020-04-20 18:08:22 +08:00
Nikolai Kochetov
052598534a Try fix unit tests. 2020-04-20 12:42:56 +03:00
Azat Khuzhin
9ca86dbf34 Drop disks reservation in Aggregator 
True reservation (IVolume::reserve()) will not takes compression into
account, and you aggregate a lot of data then it will likely fail even
though it can be completed, but just because you have big
max_bytes_before_external_group_by and huge max_threads it will fail.
 2020-04-20 11:33:46 +03:00
alexey-milovidov
9d0004cba0
Merge pull request #10365 from ClickHouse/avoid-infinite-loop-in-hierarchical-dictionaries 
Avoid infinite loop in hierarchical dictionaries
 2020-04-20 10:54:16 +03:00
Amos Bird
2d08c8e4ab
clang-10 fix 2020-04-20 15:45:37 +08:00