thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-22 15:42:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
56,889 Commits 9,152 Branches 1,781 Tags 2.1 GiB
6aa86fb533
Commit Graph

6249 Commits

Author SHA1 Message Date
Alexey Milovidov
47a56b9d1d Merge branch 'master' into point-in-polygon-ubsan 2021-02-04 08:26:41 +03:00
Azat Khuzhin
984445b44b Fix UBsan reports in quantileTiming 
UBsan reports [1]:

    ../src/AggregateFunctions/QuantileTiming.h:442:27: runtime error: 1.84467e+19 is outside the range of representable values of type 'unsigned long' Received signal -3 Received signal Unknown signal (-3)

  [1]: https://clickhouse-test-reports.s3.yandex.net/19971/e15f5d9cb5b36482d1ae9ca069074fb200f2ab37/fuzzer_ubsan/report.html#fail1

Follow-up for: #19394
 2021-02-04 02:01:48 +03:00
Azat Khuzhin
cc43f92719 Fix tests *.reference files 
Using:

    sed -i 's/MergeTree  with order/MergeTree with order/g' tests/queries/0_stateless/*.reference

Fixes: dc2afc4795 ("Fix double whitespace #18383")
 2021-02-03 22:45:54 +03:00
alexey-milovidov
ca06d3b317
Merge branch 'master' into fix-rocksdb 2021-02-03 21:28:50 +03:00
alexey-milovidov
402c018d1e
Merge pull request #19947 from azat/dist-INSERT-block-structure-mismatch 
Fix "Block structure mismatch" for INSERT into Distributed
 2021-02-03 21:27:45 +03:00
alexey-milovidov
0aca4a740c
Update 01700_point_in_polygon_ubsan.sql 2021-02-03 21:26:03 +03:00
Alexey Milovidov
eb8b2e883a Add a test 2021-02-03 21:25:04 +03:00
Nikolai Kochetov
c39c965612
Merge pull request #19962 from keen-wolf/timezoneOffset 
Implement the timezoneOffset function
 2021-02-03 20:37:34 +03:00
Alexey Milovidov
82ab793731 Fix missing type check in StorageEmbeddedRocksDB 2021-02-03 20:35:23 +03:00
Alexander Kuzmenkov
ec382d8812 BETWEEN CURRENT ROW AND CURRENT ROW 2021-02-03 17:55:40 +03:00
Alexander Kuzmenkov
35754abb4a CURRENT ROW frame start for RANGE frame 2021-02-03 17:22:37 +03:00
Alexander Kuzmenkov
7c55ecf67d cleanup 2021-02-03 16:41:59 +03:00
Alexander Kuzmenkov
d8a9d347fe
Merge pull request #20000 from ClickHouse/aku/perf 
update perf tests
 2021-02-03 16:06:58 +03:00
Alexander Kuzmenkov
f31d2206a7 more fuzzing and less bugs 2021-02-03 15:50:25 +03:00
Pavel Kovalenko
b7151aa754 Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata 
# Conflicts:
#	src/Disks/DiskDecorator.h
#	src/Disks/IDisk.h
#	src/Disks/S3/DiskS3.cpp
 2021-02-03 14:22:18 +03:00
Alexander Kuzmenkov
53052f7040
Update group_by_sundy_li.xml 2021-02-03 12:22:02 +03:00
Alexander Kuzmenkov
bb45c5a8ff Merge remote-tracking branch 'origin/master' into HEAD 2021-02-03 12:13:23 +03:00
Maksim Kita
ba45cf32ab
Merge pull request #19858 from kitaisreal/reinterpret-as-updated-to-big-integers 
Function reinterpretAs updated to support big integers
 2021-02-03 12:12:13 +03:00
Alexander Kuzmenkov
1b47cf08a9
Merge pull request #19921 from ClickHouse/aku/window-prototype-2 
UNBOUNDED FOLLOWING frame end
 2021-02-03 12:10:37 +03:00
alexey-milovidov
830b0eb36e
Merge pull request #19974 from qoega/kerberized-hdfs-port-clash 
Fixed port clash from test_storage_kerberized_hdfs test.
 2021-02-03 10:16:14 +03:00
Alexander Kuzmenkov
a164abf23e add frame fuzzing and one query that fails under msan 2021-02-03 09:42:54 +03:00
Alexander Kuzmenkov
7e945bab03 fix the calculation for moving frame start 2021-02-03 08:53:21 +03:00
alexey-milovidov
a4d86c43e9
Merge pull request #19946 from ClickHouse/improve-performance-of-aggregate-functions 
Improve performance of aggregate functions
 2021-02-03 08:46:51 +03:00
Alexander Kuzmenkov
d0bac8bfd9 Merge remote-tracking branch 'origin/master' into HEAD 2021-02-03 08:45:16 +03:00
Alexander Kuzmenkov
85f8918753 fix the row order to be like in the original test 2021-02-03 07:47:53 +03:00
Azat Khuzhin
ac510f9770 test_read_temporary_tables_on_failure: increase timeout to avoid false-positive 
In [1]:

    start_cluster = <helpers.cluster.ClickHouseCluster object at 0x7f2d46895dd8>

        def test_different_versions(start_cluster):
            with pytest.raises(QueryTimeoutExceedException):
                node.query("SELECT sleep(3)", timeout=1)
            with pytest.raises(QueryRuntimeException):
    >           node.query("SELECT 1", settings={'max_concurrent_queries_for_user': 1})
    E           Failed: DID NOT RAISE <class 'helpers.client.QueryRuntimeException'>

  [1]: https://clickhouse-test-reports.s3.yandex.net/19451/b68508002d134ead05bf2ca0e22a13a34a5c55c6/integration_tests_(thread).html#fail1
 2021-02-02 22:04:45 +03:00
Azat Khuzhin
33628e1bd9 test_materialize_mysql_database: add retries for test_select_without_columns_8_0 
Here is an example:

    2021.01.31 14:04:43.615618 [ 54 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute dump data*/ INSERT INTO t(a, b, _sign, _version) VALUES
    2021.01.31 14:04:43.616453 [ 54 ] {} <Trace> db.t (fc55a47e-1cae-4f41-9a13-91ccde5a73bb): Renaming temporary part tmp_insert_0_1_1_0 to 0_1_1_0.
    ...
    2021.01.31 14:04:43.786823 [ 12 ] {4a0d1f80-4a96-4809-a84e-d55d7cd25cdd} <Debug> executeQuery: (from 172.18.0.1:45162, using production parser) SELECT count(_version) FROM db.t FORMAT TSV
    ...
    2021.01.31 14:04:44.566037 [ 54 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute dump data*/ INSERT INTO t(a, b, _sign, _version) VALUES
    2021.01.31 14:04:44.566943 [ 54 ] {} <Trace> db.t (fc55a47e-1cae-4f41-9a13-91ccde5a73bb): Renaming temporary part tmp_insert_0_2_2_0 to 0_2_2_0.
    ...
    2021.01.31 14:04:52.343911 [ 54 ] {} <Error> MaterializeMySQLSyncThread: Code: 1002, e.displayText() = DB::Exception: Master maybe lost, Stack trace (when copying this message, always include the lines below):

  [1]: https://clickhouse-test-reports.s3.yandex.net/19584/978f23e3437d0d0eb64727e87ffe1b80289c9fc2/integration_tests_(release).html#fail1
 2021-02-02 22:04:45 +03:00
Azat Khuzhin
3d3d6777d3 test_dictionaries_dependency_xml: decrease dictionaries lifetime 
Since ExternalLoader::PeriodicUpdater::check_period_sec = 5, and so if
it will be scheduled too early the reload will be skipped, and indeed
this is what you can see in logs [1], the reload is done each 10
seconds, not 5:

    2021.01.31 14:20:22.590999 [ 48 ] {} <Trace> ExternalDictionariesLoader: Supposed update time for 'dep_x' is 2021-01-31 14:20:27 (loaded, lifetime [5, 5], no errors)
    2021.01.31 14:20:22.591016 [ 48 ] {} <Trace> ExternalDictionariesLoader: Next update time for 'dep_x' was set to 2021-01-31 14:20:27
    ...
    2021.01.31 14:20:32.164882 [ 50 ] {} <Trace> ExternalDictionariesLoader: Start loading object 'dep_x'

  [1]: https://clickhouse-test-reports.s3.yandex.net/19584/37797fdf5b30dc97147e73b3ac8ca9025b80aaed/integration_tests_(release).html#fail1
 2021-02-02 22:04:32 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
keenwolf
b7e4f7bd1e modify file attribute 2021-02-03 00:41:11 +08:00
Alexander Kuzmenkov
264aea20be Merge remote-tracking branch 'origin/master' into HEAD 2021-02-02 19:12:46 +03:00
keenwolf
9beb7689cc add comments and update test cases. 2021-02-02 23:35:45 +08:00
Vitaly Baranov
9427d5d94b
Merge pull request #19805 from vitlibar/allow-getting-return-type-of-dictget-without-loading-dictionary 
Fix starting the server with tables having default expressions containing dictGet()
 2021-02-02 18:19:55 +03:00
Alexander Kuzmenkov
d2a0e44a3a
Merge pull request #19299 from ClickHouse/aku/window-prototype 
[wip] support RANGE frame for window functions
 2021-02-02 17:59:16 +03:00
Alexander Kuzmenkov
6317208add
Merge branch 'master' into aku/perf 2021-02-02 17:24:17 +03:00
Alexey Milovidov
b9647e5326 Fix double whitespace 2021-02-02 17:22:03 +03:00
Alexander Kuzmenkov
b4afc49d3c update perf tests 
sed -i s'/^<test.*$/<test>/g' tests/performance/*.xml

WITH ceil(max(q[3]), 1) AS h
SELECT concat('sed -i s\'/^<test.*$/<test max_ignored_relative_change="', toString(h), '">/g\' tests/performance/', test, '.xml') AS s
FROM
(
    SELECT
        test,
        query_index,
        count(*),
        min(event_time),
        max(event_time) AS t,
        arrayMap(x -> floor(x, 3), quantiles(0, 0.5, 0.95, 1)(stat_threshold)) AS q,
        median(stat_threshold) AS m
    FROM perftest.query_metrics_v2
    WHERE ((pr_number != 0) AND (event_date > '2021-01-01')) AND (metric = 'client_time') AND (abs(diff) < 0.05) AND (old_value > 0.1)
    GROUP BY
        test,
        query_index,
        query_display_name
    HAVING (t > '2021-01-01 00:00:00') AND ((q[3]) > 0.1)
    ORDER BY test DESC
)
GROUP BY test
ORDER BY h DESC
FORMAT PrettySpace
 2021-02-02 17:21:43 +03:00
alexey-milovidov
695e28079d
Merge pull request #19960 from ClickHouse/json-extract-inaccurate-conversion-from-double-to-float 
Allow inaccurate conversion from double to float in function JSONExtract beacuse the users want that
 2021-02-02 15:15:37 +03:00
alexey-milovidov
4d0973cf25
Merge pull request #19959 from depressed-pho/fix-segv-fromModifiedJulianDay 
Fix a segfault in fromModifiedJulianDay()
 2021-02-02 13:50:14 +03:00
alexey-milovidov
05b8164133
Merge pull request #19956 from ClickHouse/geohash-ubsan 
Fix UBSan report in geoHashesInBox
 2021-02-02 12:52:01 +03:00
Maksim Kita
c889f51e9e Updated performance benchmark 2021-02-02 12:50:58 +03:00
feng lv
0edf65c094 fix test 
fix

update

fix spell
 2021-02-02 09:22:30 +00:00
Alexey Milovidov
0b166e1cbd Add performance test 2021-02-02 12:11:20 +03:00
Yatsishin Ilya
19d6e867c6 HDFS test had a clash with 9000 ClickHouse port. 2021-02-02 12:08:47 +03:00
Kruglov Pavel
fd5adf1ff6
Merge pull request #19925 from ClickHouse/fix_insert_values_with_semicolon 
Fix logical error in INSERT VALUES
 2021-02-02 12:05:19 +03:00
tavplubix
befee42ab0
Merge pull request #19684 from yiguolei/master 
Should fail ddl query as soon as possible if table is shutdown
 2021-02-02 11:46:58 +03:00
tavplubix
dbcddbcebd
Merge pull request #19770 from stigsb/materializemysql_system_parts 
Show MaterializeMySQL tables in system.parts
 2021-02-02 11:43:42 +03:00
alexey-milovidov
f5dc38a33c
Merge pull request #19584 from azat/client-complete-error-codes 
client: more suggestions
 2021-02-02 11:05:17 +03:00
alesapin
a9c50caa9a
Merge pull request #19942 from ClickHouse/fix_test_keeper_b2b_test 
Fix test keeper integration tests
 2021-02-02 10:17:27 +03:00
feng lv
4279c7da41 add setting insert_shard_id 
add test

fix style

fix
 2021-02-02 04:26:59 +00:00
keenwolf
f21e22ae2c add test for function timezoneOffset 2021-02-02 12:22:45 +08:00
Alexey Milovidov
078011ef2d Allow conversion from double to float in function JSONExtract beacuse the users want that 2021-02-02 07:16:47 +03:00
PHO
5f7a852b92 Fix a segfault in fromModifiedJulianDay() 
It was crashing when the argument type was Nullable(T) where T was any integral type other than Int32.
 2021-02-02 12:43:36 +09:00
Alexey Milovidov
9930bb0bf6 Fix UBSan report in geoHashesInBox 2021-02-02 06:37:24 +03:00
Alexey Milovidov
b9fff3676b Add exception for ANTLR tests 2021-02-02 06:03:10 +03:00
Alexander Kuzmenkov
d084625436 cleanup 2021-02-02 03:51:35 +03:00
Alexander Kuzmenkov
78a9624fc5 CURRENT ROW frame start for ROWS frame 2021-02-02 02:27:23 +03:00
Alexander Kuzmenkov
ef46c36317 start offset for ROWS frame 2021-02-02 02:27:17 +03:00
Azat Khuzhin
37797fdf5b Merge remote-tracking branch 'upstream/master' into client-complete-error-codes 2021-02-02 00:14:53 +03:00
alexey-milovidov
6b2f0435c8
Merge pull request #19375 from Avogar/select-final 
Improve do_not_merge_across_partitions_select_final optimization.
 2021-02-01 20:31:08 +03:00
alexey-milovidov
1f0a91e0ab
Merge pull request #19677 from kitaisreal/executable-dictionary-implicit-key-option 
ExecutableDictionarySource added implicit_key option
 2021-02-01 20:13:25 +03:00
alexey-milovidov
2901591f81
Merge pull request #19876 from ClickHouse/intdiv-ubsan 
Fix UBSan report in intDiv
 2021-02-01 20:12:30 +03:00
alesapin
e6fd6abe6f Fix test keeper integration tests 2021-02-01 18:07:34 +03:00
alesapin
2aa8a6304b
Merge pull request #15450 from CurtizJ/fix-ttl-group-by 
Fix some cases of TTL expressions
 2021-02-01 16:48:07 +03:00
Nikolai Kochetov
19e4a33f9d
Merge pull request #19544 from amosbird/limitconcurrency 
Per MergeTree table query limit
 2021-02-01 16:09:12 +03:00
alexey-milovidov
f09158de0c
Merge pull request #19884 from kitaisreal/bloom-filter-index-crash-fix 
BloomFilter index crash fix
 2021-02-01 15:42:56 +03:00
alexey-milovidov
9161a5f570
Merge pull request #19875 from ClickHouse/text-log-fix-deadlock 
Fix deadlock in system.text_log
 2021-02-01 15:42:16 +03:00
Alexander Tokmakov
accf4d262e fix logical error 2021-02-01 15:40:57 +03:00
tavplubix
2a20f20f21
Merge pull request #19790 from taiyang-li/fix_client_select 
fix client bug while executing select
 2021-02-01 13:17:14 +03:00
tavplubix
1e44e3f341
Update 01671_ddl_hang_timeout.sh 2021-02-01 12:53:13 +03:00
Maksim Kita
beee7a4f02 Added test to arcadia_skip_list 2021-02-01 12:35:47 +03:00
Maksim Kita
d3f299269f Fixed code review issues 2021-02-01 12:35:47 +03:00
Maksim Kita
d194294d0b Fixed tests 2021-02-01 12:35:47 +03:00
Maksim Kita
dafb0ef4e9 ExecutableDictionarySource added implicit_key option 2021-02-01 12:35:47 +03:00
Maksim Kita
9a1e52a80d Added performance test 2021-02-01 12:25:52 +03:00
Stig Bakken
bedc472bb0 Show MaterializeMySQL tables in system.parts 2021-02-01 16:59:10 +08:00
alexey-milovidov
ef72ba7349
Merge pull request #19861 from amosbird/fixcte 
Fix crash when pushing down predicates to union distinct subquery
 2021-02-01 10:28:41 +03:00
alesapin
d9598c47b4
Merge pull request #19702 from ClickHouse/fix_rare_bug_after_part_corruption 
Fix incorrect virtual_parts after part corruption
 2021-02-01 10:09:48 +03:00
yiguolei
fa03fbdc5b fix functional test 2021-02-01 13:16:54 +08:00
alexey-milovidov
2459777f57
Update 01681_bloom_filter_nullable_column.sql 2021-02-01 07:55:58 +03:00
Alexey Milovidov
1c0c3b3aab Merge branch 'master' into intdiv-ubsan 2021-02-01 07:46:40 +03:00
yiguolei
bef5af3f57 fix functional test 2021-02-01 11:35:02 +08:00
alexey-milovidov
08f713f177
Merge pull request #14822 from vzakaznikov/live_view_periodic_refresh 
Adding support for periodically refreshed LIVE VIEW tables
 2021-02-01 06:11:30 +03:00
yiguolei
768e461d28 add functional test 2021-02-01 10:40:48 +08:00
Alexey Milovidov
efb9293587 Adjust test scale 2021-02-01 05:28:18 +03:00
alexey-milovidov
2193baafde
Merge pull request #19852 from vzakaznikov/update_testflows_to_support_retries_during_container_start 
Adding retries for docker-compose start, stop and restart in TestFlows tests
 2021-02-01 03:52:38 +03:00
alexey-milovidov
7567bb2e31
Merge pull request #19860 from ClickHouse/hyperscan-assert 
Fix debug assertion in Hyperscan
 2021-02-01 03:45:08 +03:00
alexey-milovidov
a6b0fdd3eb
Merge pull request #19862 from ClickHouse/ubsan-gather-utils 
Fix UBSan report in GatherUtils
 2021-02-01 03:43:43 +03:00
alexey-milovidov
9407c5baf4
Merge pull request #19868 from amosbird/fixif 
Fix argMinMaxIf crash
 2021-02-01 03:42:48 +03:00
Alexander Tokmakov
84d3d6e728 fix 'Unmatched parentheses', add test 2021-02-01 02:10:41 +03:00
Maksim Kita
65183f1fea Fixed tests 2021-01-31 23:54:52 +03:00
Maksim Kita
1aa8d7fdfb BloomFilter index crash fix 2021-01-31 21:49:27 +03:00
Maksim Kita
a4812aa97b Updated tests 2021-01-31 18:32:47 +03:00
alexey-milovidov
342baa4223
Merge pull request #19720 from ClickHouse/unicode-asan 
Fix unsufficient args check (trash code) in StringSearcher
 2021-01-31 15:43:05 +03:00
alexey-milovidov
b80ad9f9bc
Merge pull request #19859 from ClickHouse/date-time-add-ubsan 
Fix UBSan report in DateTimeAddInterval
 2021-01-31 14:09:04 +03:00
vdimir
000b51f95a
Merge pull request #19518 from bharatnc/ncb/toIPv6-parses-IPv4 
toIPv6 function parses IPv4 addresses
 2021-01-31 13:56:45 +03:00
Alexey Milovidov
4eb21ca396 Add a test 2021-01-31 13:25:31 +03:00
Alexey Milovidov
051e4832b9 Add a test 2021-01-31 13:05:00 +03:00
Amos Bird
18f3df31d5
Fix crash when pushing down predicates to union distinct subquery 2021-01-31 16:29:29 +08:00
Amos Bird
c39ee3f58c
Fix argMinMaxIf crash 2021-01-31 16:27:46 +08:00