ClickHouse/CHANGELOG.md
myrrc 2c8582a2a8 Squashed commit of the following:
commit 17331d2732
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:51:05 2020 +0300

    another fix

commit 61e189d84b
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:50:05 2020 +0300

    another fix

commit cb2ddc7d01
Merge: 0e235d35f8 985876b737
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:49:18 2020 +0300

    aMerge branch 'improvement/multiply-decimals-and-floats' of github:myrrc/clickhouse into improvement/multiply-decimals-and-floats

commit 0e235d35f8
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:48:37 2020 +0300

    aa

commit 985876b737
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:37:00 2020 +0300

    another fix

commit e2556659e0
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:28:21 2020 +0300

    fix3

commit a199348860
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:26:58 2020 +0300

    fix2

commit fbaa8d3951
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:24:31 2020 +0300

    fix

commit ed6d74b218
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:21:10 2020 +0300

    another bad solution

commit 6f1c0b118a
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:17:03 2020 +0300

    typo

commit 0dca856237
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:16:38 2020 +0300

    solution fix

commit d22d045a59
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:12:31 2020 +0300

    fixed the impl

commit 9d24cafccd
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 18:06:24 2020 +0300

    added vector mult branch

commit 7611f8d15b
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:54:48 2020 +0300

    compile fix

commit 6b956a2a0d
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:50:16 2020 +0300

    possible fix for non-const mult

commit 58ef4c34c0
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:44:24 2020 +0300

    prep for debug version

commit 3c01cfe630
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:34:51 2020 +0300

    no scale added

commit fa1f7f40f5
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:32:44 2020 +0300

    another try

commit 6488846c92
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:23:51 2020 +0300

    signature fix

commit bc651b3047
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:21:32 2020 +0300

    added the suboptimal solution

commit 764d462288
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:16:54 2020 +0300

    updated test with possible right values

commit 611d400106
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:13:15 2020 +0300

    fixed compile error

commit 6c91b3dba0
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:10:37 2020 +0300

    added explicit scale calc

commit 1f10514599
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 17:02:02 2020 +0300

    another attempt

commit 07bc3a2a4d
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 16:49:00 2020 +0300

    reverted the overengineering

commit c4ea410a4d
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 16:45:44 2020 +0300

    updated the comments and branches

commit 15f0915ad3
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 16:35:37 2020 +0300

    replaced ternary if

commit 84013d253d
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 16:33:00 2020 +0300

    fix: ternary if

commit f34c0500db
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 16:29:53 2020 +0300

    enlarged the test, updated scales calc

commit d4f8c46210
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 16:04:19 2020 +0300

    fixed the test

commit 283268c50f
Merge: 5b6143bf2f 6a9397a64b
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 15:42:39 2020 +0300

    Merge branch 'improvement/multiply-decimals-and-floats' of github:myrrc/clickhouse into improvement/multiply-decimals-and-floats

commit 5b6143bf2f
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 15:42:12 2020 +0300

    fix style error

commit 05abfb04bd
Merge: a50e9548fd 26637bd7ee
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 15:42:01 2020 +0300

    Merge remote-tracking branch 'upstream/master' into improvement/multiply-decimals-and-floats

commit 26637bd7ee
Merge: 0d9519a0a2 911a1cd48f
Author: alesapin <alesapin@gmail.com>
Date:   Fri Dec 18 12:41:16 2020 +0300

    Merge pull request #17943 from ClickHouse/try_rewrite_writer

    Small simplification of MergeTreeDataWriter

commit 0d9519a0a2
Merge: 6ee2ada592 e4433157e7
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 08:08:26 2020 +0300

    Merge pull request #17567 from ClickHouse/insert_quorum_parallel-default

    Insert quorum parallel by default

commit 6ee2ada592
Merge: 55604bde47 fb3d235e84
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 08:07:46 2020 +0300

    Merge pull request #18204 from ClickHouse/remove-strerror

    Do not use non thread-safe function `strerror`

commit 55604bde47
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 08:07:09 2020 +0300

    Whitespaces

commit c343fa1ae4
Merge: 6a5ce2eea7 4a1618d0b8
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 08:03:11 2020 +0300

    Merge pull request #18202 from ClickHouse/update-libc-headers-2

    Update libc headers

commit 6a5ce2eea7
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 07:58:56 2020 +0300

    Update config.xml

commit 44734a829e
Merge: aa7793e38f b4a59aecfe
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 06:39:35 2020 +0300

    Merge pull request #18207 from ClickHouse/better-config

    Better config: example ports, security advices

commit b4a59aecfe
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 06:39:21 2020 +0300

    Update config.xml

commit aa7793e38f
Merge: 8bc9d8ed7e d198f03301
Author: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
Date:   Fri Dec 18 01:54:42 2020 +0300

    Merge pull request #18084 from nikitamikhaylov/fix-arcadia-third

    Try to fix Arcadia

commit 8bc9d8ed7e
Merge: 6fc3ca8b7b 52ef4289fe
Author: tavplubix <avtokmakov@yandex-team.ru>
Date:   Fri Dec 18 01:50:32 2020 +0300

    Merge pull request #18198 from ClickHouse/materialize_mysql_optimize_on_insert

    Enable optimize_on_insert for MaterializeMySQL

commit 6a9397a64b
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 01:35:37 2020 +0300

    Update FunctionBinaryArithmetic.h

commit a50e9548fd
Merge: 43348bd708 0462340d8a
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 01:26:23 2020 +0300

    Merge branch 'improvement/multiply-decimals-and-floats' of github:myrrc/clickhouse into improvement/multiply-decimals-and-floats

commit 43348bd708
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 01:25:30 2020 +0300

    fixed the possible values case

commit 0462340d8a
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 01:03:25 2020 +0300

    Update FunctionBinaryArithmetic.h

commit 5a78eafb05
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 00:59:39 2020 +0300

    Update FunctionBinaryArithmetic.h

commit 40c702c54a
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 00:56:42 2020 +0300

    Update multiply.cpp

commit 6fc3ca8b7b
Merge: 8140fbd0fb 0c04a263e5
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 00:45:27 2020 +0300

    Merge pull request #16724 from kitaisreal/added-function-accurate-cast-or-null

    Added function accurateCastOrNull, allow different types inside IN subquery

commit 92d8840ae5
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Fri Dec 18 00:31:09 2020 +0300

    Better config: example ports, security advices

commit 8140fbd0fb
Merge: 799997db35 aecdd02be4
Author: alesapin <alesapin@gmail.com>
Date:   Fri Dec 18 00:22:19 2020 +0300

    Merge pull request #18200 from ClickHouse/more_logs_during_quorum

    More logs during quorum insert

commit fe1b396256
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 00:14:06 2020 +0300

    added test reference

commit bf5e836414
Author: myrrc <me@myrrec.space>
Date:   Fri Dec 18 00:03:45 2020 +0300

    updated the test

commit 7db7a9ef5b
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 23:55:59 2020 +0300

    added another constexpr branch

commit 205db4fa92
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 23:50:59 2020 +0300

    fixed the scale bug, added the test (wo reference)

commit b5e6b233ee
Merge: 4f7d019a9e 61d9d213e9
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 23:20:14 2020 +0300

    Merge branch 'improvement/multiply-decimals-and-floats' of github:myrrc/clickhouse into improvement/multiply-decimals-and-floats

commit 4f7d019a9e
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 23:19:57 2020 +0300

    Squashed commit of the following:

    commit 61d9d213e9
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 23:14:02 2020 +0300

        fixed another variable init

    commit 73f3195fe1
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 23:12:41 2020 +0300

        fixed variable init

    commit 488f4ef42e
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 23:09:33 2020 +0300

        fixed compile condition

    commit e50af17545
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 23:05:10 2020 +0300

        fixed template args1

    commit f93385dcf0
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 23:01:29 2020 +0300

        replaced lambda with helper function

    commit 5974e784c2
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 22:52:52 2020 +0300

        added the conversion func

    commit 6eecef35a6
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 22:35:46 2020 +0300

        fixed result type trait

    commit 31e3ca1670
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 22:26:15 2020 +0300

        restored type signature

    commit 613f4e0f04
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 22:24:09 2020 +0300

        removed explicit types

    commit 1c52842e83
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 22:21:23 2020 +0300

        simplified the condition

    commit c61af32360
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 22:12:12 2020 +0300

        fixed const column branch

    commit fd1d24c928
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 22:10:33 2020 +0300

        fixed overflow check

    commit b428f8fcd8
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 22:01:04 2020 +0300

        fixed scales

    commit 39c82fe50c
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 21:52:19 2020 +0300

        fixed result type

    commit 54887dacbf
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 21:38:36 2020 +0300

        fix scale init

    commit 3e9428515c
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 21:36:15 2020 +0300

        fix deleted constuctor

    commit ea2da764f6
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 21:33:51 2020 +0300

        fixed result data type init

    commit 1f67b0ce8d
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 21:28:51 2020 +0300

        reverting if constexpr

    commit 494e9066c3
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 21:17:38 2020 +0300

        fixing first compile bug

    commit d24fc58c94
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 21:12:24 2020 +0300

        case fix 2

    commit 9f217c5df5
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 21:11:39 2020 +0300

        case fix

    commit fa6a1d3b4a
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 20:55:35 2020 +0300

        fix

    commit af8a72a45f
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 20:16:37 2020 +0300

        fix

    commit 737d4fc22f
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 19:43:29 2020 +0300

        debug fix

    commit fcacf5c510
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 19:31:54 2020 +0300

        another try

    commit 49f3b71b5f
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 18:48:22 2020 +0300

        another fix

    commit 0d319c8c61
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 18:47:14 2020 +0300

        fix

    commit 9d5998fd10
    Author: myrrc <me@myrrec.space>
    Date:   Thu Dec 17 18:46:01 2020 +0300

        added debug info

    commit 17d996054e
    Author: myrrc <me@myrrec.space>
    Date:   Wed Dec 16 18:47:24 2020 +0300

        style fix

    commit 02baaddc13
    Author: myrrc <me@myrrec.space>
    Date:   Wed Dec 16 18:20:41 2020 +0300

        removed some casts

commit 799997db35
Merge: 30c98adde4 832d99515b
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 23:16:58 2020 +0300

    Merge pull request #16621 from arenadata/ADQM-148

    HDFS configuration and Kerberos Authenticaiton

commit 61d9d213e9
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 23:14:02 2020 +0300

    fixed another variable init

commit 30c98adde4
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 23:13:49 2020 +0300

    Fix local variable in clickhouse-test

commit 73f3195fe1
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 23:12:41 2020 +0300

    fixed variable init

commit 488f4ef42e
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 23:09:33 2020 +0300

    fixed compile condition

commit e50af17545
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 23:05:10 2020 +0300

    fixed template args1

commit f93385dcf0
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 23:01:29 2020 +0300

    replaced lambda with helper function

commit b9c537ac37
Merge: d9125a4822 da18af96a3
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Thu Dec 17 22:59:33 2020 +0300

    Merge pull request #18104 from vitlibar/fix-access-rights-for-merge-table-function-2

    Fix access rights required for the merge() table function #2.

commit d9125a4822
Merge: dbfe50c0c1 72df75fc50
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Thu Dec 17 22:59:09 2020 +0300

    Merge pull request #18014 from vitlibar/fix-showing-temporary-tables-in-system-tables

    Fix showing temporary tables in system tables

commit 5974e784c2
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 22:52:52 2020 +0300

    added the conversion func

commit dbfe50c0c1
Merge: 82ed528d98 a207e3db75
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 22:52:15 2020 +0300

    Merge pull request #18199 from ClickHouse/add_one_more_test_in_util

    Add one more case in zk test util

commit 82ed528d98
Merge: f82274a722 fa266790b4
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 22:51:52 2020 +0300

    Merge pull request #18191 from ClickHouse/hard_watches_test

    Test watches for in-memory zookeeper

commit e4433157e7
Merge: c2b0f9d530 f82274a722
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 22:42:32 2020 +0300

    Merge branch 'master' into insert_quorum_parallel-default

commit c2b0f9d530
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 22:42:15 2020 +0300

    Update integration test

commit 6eecef35a6
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 22:35:46 2020 +0300

    fixed result type trait

commit f82274a722
Merge: e9d97160f4 fd7fe0f84c
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 22:34:05 2020 +0300

    Merge pull request #16527 from kitaisreal/mmap-read-buffer-from-file-descriptor-use-getpagesize

    MMapReadBufferFromFileDescriptor use getpagesize

commit e9d97160f4
Merge: dd78c9dc75 f7f8ff8758
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 22:28:22 2020 +0300

    Merge pull request #16346 from ClickHouse/sigaltstack

    Provide diagnostics on stack overflow

commit 31e3ca1670
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 22:26:15 2020 +0300

    restored type signature

commit 613f4e0f04
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 22:24:09 2020 +0300

    removed explicit types

commit 1c52842e83
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 22:21:23 2020 +0300

    simplified the condition

commit f7f8ff8758
Merge: 01d3aef890 dd78c9dc75
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 22:16:30 2020 +0300

    Merge branch 'master' into sigaltstack

commit c61af32360
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 22:12:12 2020 +0300

    fixed const column branch

commit fd1d24c928
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 22:10:33 2020 +0300

    fixed overflow check

commit 0c04a263e5
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Dec 17 22:04:26 2020 +0300

    Fixed ya.make

commit b428f8fcd8
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 22:01:04 2020 +0300

    fixed scales

commit 39c82fe50c
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 21:52:19 2020 +0300

    fixed result type

commit 54887dacbf
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 21:38:36 2020 +0300

    fix scale init

commit 3e9428515c
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 21:36:15 2020 +0300

    fix deleted constuctor

commit ea2da764f6
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 21:33:51 2020 +0300

    fixed result data type init

commit 9aaa2f9345
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Dec 17 21:32:25 2020 +0300

    Fixed compilation timeout

commit 1f67b0ce8d
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 21:28:51 2020 +0300

    reverting if constexpr

commit fb3d235e84
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 21:08:42 2020 +0300

    Do not use strerror

commit 494e9066c3
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 21:17:38 2020 +0300

    fixing first compile bug

commit d24fc58c94
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 21:12:24 2020 +0300

    case fix 2

commit 9f217c5df5
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 21:11:39 2020 +0300

    case fix

commit fa6a1d3b4a
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 20:55:35 2020 +0300

    fix

commit dd78c9dc75
Merge: a77acd5aa7 b25f126569
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 20:44:44 2020 +0300

    Merge pull request #17566 from ClickHouse/relax-too-many-parts-1

    Relax "Too many parts" threshold, part 1

commit 4a1618d0b8
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 20:25:51 2020 +0300

    Update libc headers

commit af8a72a45f
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 20:16:37 2020 +0300

    fix

commit 737d4fc22f
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 19:43:29 2020 +0300

    debug fix

commit fcacf5c510
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 19:31:54 2020 +0300

    another try

commit a207e3db75
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 19:21:46 2020 +0300

    Fix bad sleep

commit aecdd02be4
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 19:14:42 2020 +0300

    Better message

commit f45993cd1e
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 19:13:01 2020 +0300

    More logs during quorum insert

commit 49f3b71b5f
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 18:48:22 2020 +0300

    another fix

commit 0d319c8c61
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 18:47:14 2020 +0300

    fix

commit d198f03301
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Thu Dec 17 18:46:26 2020 +0300

    suppress warnings

commit 9d5998fd10
Author: myrrc <me@myrrec.space>
Date:   Thu Dec 17 18:46:01 2020 +0300

    added debug info

commit a77acd5aa7
Author: Anton Popov <pad11rus@gmail.com>
Date:   Thu Dec 17 18:12:24 2020 +0300

    Fix flaky test

commit e80b93939f
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 18:09:09 2020 +0300

    Add one more case in zk test uti

commit cb91d64fe7
Author: hexiaoting <hewenting_ict@163.com>
Date:   Thu Dec 17 22:16:14 2020 +0800

    Add new feature: limit length and offset setting (#17633)

    Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>

commit 52ef4289fe
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Thu Dec 17 16:05:38 2020 +0300

    enable optimize_on_insert for MaterializeMySQL

commit 0f3582883c
Merge: d302685e72 8f49946a72
Author: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
Date:   Thu Dec 17 15:44:55 2020 +0300

    Merge pull request #18147 from kitaisreal/array-aggregation-big-integer-types-support

    Added array aggregation big integer types support

commit d302685e72
Merge: 7e73b6acd8 3e07094364
Author: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
Date:   Thu Dec 17 15:12:05 2020 +0300

    Merge pull request #18172 from nikitamikhaylov/julian-day-msan

    toModifiedJulianDayOrNull msan fix

commit 3e07094364
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Thu Dec 17 15:11:32 2020 +0300

    style

commit fa266790b4
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 14:29:29 2020 +0300

    Better test

commit 7e73b6acd8
Merge: d82c23d5cf 294e8bbb4e
Author: tavplubix <avtokmakov@yandex-team.ru>
Date:   Thu Dec 17 14:26:51 2020 +0300

    Merge pull request #17642 from filimonov/detach-permanently

    DETACH TABLE PERMANENTLY

commit b6bd84df84
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 14:25:59 2020 +0300

    One more test for watches

commit d82c23d5cf
Merge: 10449695bc c6ac831d4e
Author: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
Date:   Thu Dec 17 14:00:05 2020 +0300

    Merge pull request #16853 from amosbird/ss

    Add -SimpleState combinator

commit 911a1cd48f
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 13:25:31 2020 +0300

    Suppress gdb warning

commit da18af96a3
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Thu Dec 10 23:16:53 2020 +0300

    Fix access rights required for the merge() table function.

commit fd7fe0f84c
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Dec 17 12:10:46 2020 +0300

    Updated ya.make

commit 832d99515b
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Thu Dec 17 12:05:49 2020 +0300

    tag for yandex/clickhouse-kerberized-hadoop

commit 72df75fc50
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Fri Dec 11 23:48:27 2020 +0300

    Add tests.

commit af3ccb49f1
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Fri Dec 11 22:44:18 2020 +0300

    Show create query for temporary tables in system.tables

commit ede755d3bc
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Fri Dec 11 23:32:57 2020 +0300

    Show temporary tables in system.columns as tables with empty database.

commit d7a3cc8c90
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Fri Dec 11 22:42:14 2020 +0300

    Don't show the "_temporary_and_external_tables" database in system tables
    system.databases, system.tables, system.columns.

commit b74e7accf1
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 11:17:39 2020 +0300

    Review fixes

commit f2957d6f05
Merge: 1bd2f49fb5 10449695bc
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 10:40:25 2020 +0300

    Merge branch 'master' into try_rewrite_writer

commit 10449695bc
Author: Nikolai Kochetov <nik-kochetov@yandex-team.ru>
Date:   Thu Dec 17 10:32:11 2020 +0300

    Update boost.

commit c91c5d0a68
Author: Nikolai Kochetov <nik-kochetov@yandex-team.ru>
Date:   Thu Dec 17 10:29:47 2020 +0300

    Update boost.

commit 66228f4054
Merge: bc0186eede 1f3702ff9e
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 17 10:15:30 2020 +0300

    Merge pull request #18065 from ClickHouse/fix_watch_race_testkeeper

    Remove dead list watches from TestKeeperStorage

commit bc0186eede
Merge: 81c00a4e94 6c8f844442
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 07:19:04 2020 +0300

    Merge pull request #18168 from ClickHouse/unknown_function_better_exception

    Better exception message for unknown function

commit 81c00a4e94
Merge: b159e15b1b 76e73e4e8b
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 07:18:53 2020 +0300

    Merge pull request #18167 from ClickHouse/fix_checking_profile_settings_constraints

    Fix `Unknown setting profile` error on attempt to set settings profile

commit 01d3aef890
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 05:53:43 2020 +0300

    Disable trap back

commit b159e15b1b
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 05:50:04 2020 +0300

    Update easy_tasks_sorted_ru.md

commit 22363f6086
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Thu Dec 17 05:29:27 2020 +0300

    better

commit db362e4585
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Thu Dec 17 05:20:56 2020 +0300

    done

commit 4e8978e022
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 05:21:27 2020 +0300

    Merge with master

commit 7f280bef37
Merge: 2e232bad74 16e524e0de
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 05:09:10 2020 +0300

    Merge branch 'master' into sigaltstack

commit 7abc58526a
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 04:55:07 2020 +0300

    Update test

commit 6a7c73a7bd
Merge: f46eb6f2be 16e524e0de
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 04:54:17 2020 +0300

    Merge branch 'master' into insert_quorum_parallel-default

commit b25f126569
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 04:52:27 2020 +0300

    Added a test

commit 9a017b77df
Merge: 2e6bedce91 16e524e0de
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 04:40:39 2020 +0300

    Merge branch 'master' into relax-too-many-parts-1

commit d2653f91e1
Merge: 792d8981ab 4edbb42ba1
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 03:40:13 2020 +0300

    Merge pull request #18112 from GrigoryPervakov/join_set_s3

    Use IDisk in Set and Join storages

commit 792d8981ab
Merge: 37da513c36 60b916fe0b
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 03:36:07 2020 +0300

    Merge pull request #18115 from AnaUvarova/ana-uvarova-DOCSUP-2030-rus-faq-order

    DOCSUP-2030 Rus FAQ order

commit 37da513c36
Merge: 484f20f77d d96ca3fb8d
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 03:35:19 2020 +0300

    Merge pull request #18136 from taiyang-li/explain_ast_nonselect_query

    Explain ast support nonselect querys

commit 484f20f77d
Merge: 908784a0d9 849db47f8a
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 03:34:39 2020 +0300

    Merge pull request #18140 from ClickHouse/fix_flaky_test

    Trying to fix 00620_optimize_on_nonleader_test

commit 908784a0d9
Merge: 2e2cc6d344 8139bab240
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 03:32:10 2020 +0300

    Merge pull request #18148 from ClickHouse/suppress_error_in_test

    Suppress error in 00993_system_parts_race_condition_drop_zookeeper

commit 71bd7e55d3
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Thu Dec 17 03:15:13 2020 +0300

    Bump

commit 2e2cc6d344
Merge: a999850378 2413e82e96
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Thu Dec 17 01:19:08 2020 +0300

    Merge pull request #18141 from staticor/patch-5

    Update misc.md

commit d70f58a98f
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Thu Dec 17 00:52:04 2020 +0300

    remove ryu

commit 6c8f844442
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Thu Dec 17 00:42:14 2020 +0300

    better exception message for unknown function

commit 685099af7f
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Dec 17 00:23:41 2020 +0300

    Move getPageSize in common

commit 8f49946a72
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Dec 17 00:21:44 2020 +0300

    Reused Result type

commit 4b32bb323a
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Dec 17 00:04:03 2020 +0300

    Fixed style issues

commit 76e73e4e8b
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Wed Dec 16 23:49:56 2020 +0300

    fix 'Unknown setting profile' error

commit 5a096a75b8
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Wed Dec 16 23:10:40 2020 +0300

    Fixed issues with average

commit 1f3702ff9e
Merge: 422467628a a999850378
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 23:03:07 2020 +0300

    Merge branch 'master' into fix_watch_race_testkeeper

commit a999850378
Merge: 521e3fc4ae dbbe761fa1
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 22:05:16 2020 +0300

    Merge pull request #18130 from ClickHouse/fix_remove_column_ttl

    Fix remove TTL for column

commit 521e3fc4ae
Merge: 170cd132da 153610b2c3
Author: Ilya Yatsishin <2159081+qoega@users.noreply.github.com>
Date:   Wed Dec 16 21:43:32 2020 +0300

    Merge pull request #17910 from sevirov/sevirov-DOCSUP-4919-document_replication_queue_system_table

    DOCSUP-4919: Document the system.replication_queue system table

commit 60b916fe0b
Author: ana-uvarova <ana-uvarova@yandex-team.ru>
Date:   Wed Dec 16 21:24:06 2020 +0300

    polishing links

commit 23270915d3
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Wed Dec 16 21:03:39 2020 +0300

    better

commit 170cd132da
Merge: b8250882ac 95c35b318e
Author: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
Date:   Wed Dec 16 20:49:43 2020 +0300

    Merge pull request #18113 from azat/tld-list-symlink-fix

    Avoid using symlinks for top_level_domains

commit b8250882ac
Merge: 16e524e0de a7933ddf31
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 20:35:18 2020 +0300

    Merge pull request #18056 from zhangjmruc/master

    Fix issue #17160 by adding show settings like '...'

commit 294e8bbb4e
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Wed Dec 16 18:16:47 2020 +0100

    Fix2

commit d96ca3fb8d
Author: Nikolai Kochetov <nik-kochetov@yandex-team.ru>
Date:   Wed Dec 16 20:11:32 2020 +0300

    Fixing build.

commit 16e524e0de
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 19:58:00 2020 +0300

    Remove outdated info from the docs

commit 422467628a
Merge: b347fa322c d873439846
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 19:45:57 2020 +0300

    Merge branch 'master' into fix_watch_race_testkeeper

commit d873439846
Merge: d30099fd78 9ad2a76d51
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 19:29:32 2020 +0300

    Merge pull request #15236 from ClickHouse/run_func_tests_in_parallel

    Collect list of non-parallel test

commit 8139bab240
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 19:12:57 2020 +0300

    More relaxed check

commit 9cc6fec36c
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 19:10:46 2020 +0300

    Supress error in 00993_system_parts_race_condition_drop_zookeeper

commit 80dbbc7e58
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Wed Dec 16 18:59:53 2020 +0300

    Style check fix

commit b82770c2ad
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Wed Dec 16 18:47:30 2020 +0300

    Added array aggregation big integer types support

commit 17d996054e
Author: myrrc <me@myrrec.space>
Date:   Wed Dec 16 18:47:24 2020 +0300

    style fix

commit 50c29d9ed0
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Wed Dec 16 18:34:29 2020 +0300

    better

commit d30099fd78
Merge: d046164049 44d2194326
Author: Nikolai Kochetov <KochetovNicolai@users.noreply.github.com>
Date:   Wed Dec 16 18:23:45 2020 +0300

    Merge pull request #18132 from ClickHouse/try-fix-mysql-integration-test

    Try fix integration tests.

commit d046164049
Merge: b1dc8074ef 69ed3d1cb5
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 18:22:38 2020 +0300

    Merge pull request #18128 from ClickHouse/boringssl-krb5-2

    Port Kerberos to BoringSSL

commit 02baaddc13
Author: myrrc <me@myrrec.space>
Date:   Wed Dec 16 18:20:41 2020 +0300

    removed some casts

commit f143a37693
Author: myrrc <me@myrrec.space>
Date:   Wed Dec 16 18:11:26 2020 +0300

    simplified case to multiply only

commit f14cafee51
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Wed Dec 16 16:06:43 2020 +0100

    Fix tests

commit d7145a5ee1
Author: myrrc <me@myrrec.space>
Date:   Wed Dec 16 18:01:18 2020 +0300

    working

commit 484e6e7de4
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Wed Dec 16 17:38:17 2020 +0300

    Removed getPageSize from small files

commit 849db47f8a
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 17:31:17 2020 +0300

    Better exception messages

commit b1dc8074ef
Merge: d99792e614 45361f752c
Author: Anton Popov <pad11rus@gmail.com>
Date:   Wed Dec 16 16:58:02 2020 +0300

    Merge pull request #17829 from CurtizJ/merging-map

    Merging data type Map #15806

commit 74ee36bdb8
Author: ana-uvarova <ana-uvarova@yandex-team.ru>
Date:   Wed Dec 16 16:39:14 2020 +0300

    done

commit 2413e82e96
Author: Steve-金勇 <staticor@me.com>
Date:   Wed Dec 16 21:35:55 2020 +0800

    Update misc.md

    Update translation by machine, and improve document more human-readable.

commit c15f5fba3b
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 16:16:24 2020 +0300

    Fix 00620_optimize_on_nonleader_test

commit b347fa322c
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 15:58:00 2020 +0300

    Trying to fix races one more time

commit 5521abba1e
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Wed Dec 16 15:57:42 2020 +0300

    fix boost

commit c906b1ca76
Author: myrrc <me@myrrec.space>
Date:   Wed Dec 16 15:21:23 2020 +0300

    added the initial solution

commit 4edbb42ba1
Author: Pervakov Grigorii <pervakovg@yandex-team.ru>
Date:   Tue Dec 15 19:45:13 2020 +0300

    Use IDisk in Set and Join storages

commit 0a2bda002d
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Wed Dec 16 14:46:58 2020 +0300

    remove last commit

commit 61c1db3823
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Wed Dec 16 14:45:40 2020 +0300

    skip test

commit bcd6fc1c88
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Wed Dec 16 02:53:25 2020 +0300

    better

commit 18d52dbc63
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Wed Dec 16 02:30:35 2020 +0300

    better

commit 20aaf48e38
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Tue Dec 15 22:56:47 2020 +0300

    pragmas

commit 5c4e1d1db5
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Tue Dec 15 20:53:10 2020 +0300

    better

commit 3288f1fd0c
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Tue Dec 15 16:00:40 2020 +0300

    ya make

commit c3a6499f6d
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Tue Dec 15 14:48:24 2020 +0300

    fix ya.make

commit 4ccdb3ca20
Author: nikitamikhaylov <mikhaylovnikitka@gmail.com>
Date:   Tue Dec 15 06:07:43 2020 +0300

    done

commit a5a33f1963
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Tue Dec 15 14:52:10 2020 +0300

    rpm rebuilddb and yum update in kerberized_hadoop/Dockerfile

commit 39877b1a6f
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Tue Dec 15 12:46:02 2020 +0300

    yum clean all in kerberized_hadoop/Dockerfile

commit a4dad0acb7
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Mon Dec 14 21:35:13 2020 +0300

    CentOS 6 became too old

commit 09d1e89674
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Mon Dec 14 15:07:05 2020 +0300

    yandex/clickhouse-kerberized-hadoop + missed header

commit 407565b47f
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Fri Dec 11 12:16:58 2020 +0300

    fix - wrong error code

commit 9701d6a2d5
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Fri Dec 11 00:52:05 2020 +0300

    style fix per code review, doc improvement, params consistency check

commit 815856bc3d
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Sat Nov 28 01:17:05 2020 +0300

    pass config, not context to createHDFSBuilder

commit fc0a1af214
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Thu Nov 19 00:08:17 2020 +0300

    doc minor changes, cleanup, krb5-user as a recommended package

commit 7e97814859
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Fri Oct 30 22:40:16 2020 +0300

    cleanup, fixes, new submodules, ShellCommand, WriteBufferFromString

commit d1d657335b
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Thu Oct 29 23:40:47 2020 +0300

    USE_INTERNAL_HDFS3_LIBRARY + bash lint

commit d9166a0645
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Mon Sep 28 20:20:04 2020 +0300

    HADOOP_SECURE_DN_USER way, kinit thread, docker capabilities

commit eb10948ca2
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Thu Sep 10 13:02:46 2020 +0300

    kerberized_hdfs cluster started

commit 185ac744b1
Author: Ilya Golshtein <gie@arenadata.io>
Date:   Wed Sep 9 15:13:20 2020 +0300

    kerberized hdfs compiled

commit 7d23451e81
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sat Oct 31 00:24:16 2020 +0300

    Replaced usages of 4096 constant with getPageSize

commit dbb2fbcdd5
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Oct 29 22:52:12 2020 +0300

    Unified usages of getPageSize

    1. Introduced getPageSize function.
    2. Replaced usages of getpagesize with getPageSize function.
    3. Replaced usages of sysconf(_SG_PAGESIZE) with getPageSize function.

commit 058fbd5222
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Oct 29 21:49:44 2020 +0300

    MMapReadBufferFromFileDescriptor use getpagesize

commit dbbe761fa1
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 13:23:11 2020 +0300

    Fix test

commit cd7827f70c
Author: taiyang-li <taiyang_li@163.com>
Date:   Wed Dec 16 18:00:21 2020 +0800

    explain ast support nonselect queries

commit ea4d11cb73
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 13:04:46 2020 +0300

    Wait for connections to special servers

commit 44d2194326
Author: Nikolai Kochetov <nik-kochetov@yandex-team.ru>
Date:   Wed Dec 16 11:52:04 2020 +0300

    Try fix integration tests.

commit 54aeb8fcff
Merge: 2a4a6b22b1 d99792e614
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 11:25:19 2020 +0300

    Merge branch 'master' into fix_remove_column_ttl

commit d99792e614
Merge: 37363ae89d 86fc9ea4df
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 10:54:58 2020 +0300

    Merge pull request #18105 from ClickHouse/fix_docker_images

    Fix dependencies for docker stateful_with_coverage

commit 2a4a6b22b1
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 16 10:26:18 2020 +0300

    Fix remove TTL for column

commit 37363ae89d
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 09:43:05 2020 +0300

    Comment update

commit 69ed3d1cb5
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 09:40:30 2020 +0300

    Update submodule URL

commit e3b490e9b3
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 09:36:26 2020 +0300

    Port Kerberos to BoringSSL

commit 42b401c893
Merge: 4bd5d3b662 cf35e144ff
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 09:00:38 2020 +0300

    Merge pull request #18127 from chienlungcheung/patch-1

    Correct the unit from Kb to KB in the last section.

commit cf35e144ff
Author: Chienlung Cheung <chienlungcheung@gmail.com>
Date:   Wed Dec 16 13:55:17 2020 +0800

    correct the unit from Kb to KB in the last section.

commit 4bd5d3b662
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 06:36:43 2020 +0300

    Comment update

commit 62ce33153a
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 06:34:25 2020 +0300

    Added valuable comment

commit 88c5031dfa
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 06:27:28 2020 +0300

    Add a commit #16595

commit ca9ccf4850
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 06:03:43 2020 +0300

    Comment update

commit a7933ddf31
Merge: b56523fd94 eed767bbab
Author: jianmei zhang <zhangjmruc@sina.com>
Date:   Wed Dec 16 09:38:38 2020 +0800

    Merge and rebase with master

commit 45361f752c
Author: Anton Popov <pad11rus@gmail.com>
Date:   Wed Dec 16 02:36:03 2020 +0300

    fix test

commit b485b5bd1c
Author: Anton Popov <pad11rus@gmail.com>
Date:   Wed Dec 16 01:22:21 2020 +0300

    minor fixes and comments added

commit aec820362a
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 00:28:53 2020 +0300

    Update DecimalFunctions.h

commit eed767bbab
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Wed Dec 16 00:02:44 2020 +0300

    Update tips.md

commit b842cb0455
Author: ana-uvarova <ana-uvarova@yandex-team.ru>
Date:   Tue Dec 15 23:38:36 2020 +0300

    one more try

commit 4e3cc606e6
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Tue Dec 15 23:36:45 2020 +0300

    Fixed tests

commit 97c93fa984
Author: ana-uvarova <ana-uvarova@yandex-team.ru>
Date:   Tue Dec 15 23:23:58 2020 +0300

    one more try

commit 86fc9ea4df
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 23:18:53 2020 +0300

    Update base image to 20.04

commit 0887cf476d
Merge: b0d27d6747 2e4aa6b9e1
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 23:09:37 2020 +0300

    Merge pull request #18098 from ClickHouse/fix_clickhouse_test

    Fix database drop timeout in clickhouse-test

commit 93b58321d0
Author: ana-uvarova <ana-uvarova@yandex-team.ru>
Date:   Tue Dec 15 23:00:30 2020 +0300

    fixes

commit acfc3d4281
Author: ana-uvarova <ana-uvarova@yandex-team.ru>
Date:   Tue Dec 15 22:47:24 2020 +0300

    host fixes

commit 8aaef7f634
Author: ana-uvarova <ana-uvarova@yandex-team.ru>
Date:   Tue Dec 15 22:34:11 2020 +0300

    host fixes

commit 99ecb8ae68
Author: ana-uvarova <ana-uvarova@yandex-team.ru>
Date:   Tue Dec 15 22:02:37 2020 +0300

    host fixes

commit b0d27d6747
Merge: f1e1672b54 3ec48cec53
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Tue Dec 15 22:01:57 2020 +0300

    Merge pull request #18082 from den-crane/patch-3

    Doc. toYYYYMM  translation to Russian

commit f1e1672b54
Merge: 01286a6a09 96f22a5bcc
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Tue Dec 15 22:01:08 2020 +0300

    Merge pull request #18094 from staticor/patch-4

    Update limit.md

commit 95c35b318e
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Tue Dec 15 21:48:24 2020 +0300

    Avoid using symlinks for top_level_domains

    Move the main copy into the tests/config and ship it to the perf tests
    via the perf package.

commit 51f53ce8e9
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Tue Dec 15 21:54:16 2020 +0300

    Added tryParseDecimal functions

commit e51cf03f21
Merge: ba63635251 01286a6a09
Author: Anton Popov <pad11rus@gmail.com>
Date:   Tue Dec 15 21:08:01 2020 +0300

    Merge remote-tracking branch 'upstream/master' into HEAD

commit ba63635251
Author: Anton Popov <pad11rus@gmail.com>
Date:   Tue Dec 15 21:07:13 2020 +0300

    add tests for formats with type map

commit 9df7ecb8e6
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Tue Dec 15 18:06:23 2020 +0100

    fixes

commit 2e4aa6b9e1
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 19:20:09 2020 +0300

    Kill not only database proc

commit 01286a6a09
Author: Nikolai Kochetov <nik-kochetov@yandex-team.ru>
Date:   Tue Dec 15 19:18:28 2020 +0300

    Update libunwind.

commit e10ee944be
Author: ana-uvarova <ana-uvarova@yandex-team.ru>
Date:   Tue Dec 15 19:17:42 2020 +0300

    links fixes

commit a325c0db74
Author: ana-uvarova <ana-uvarova@yandex-team.ru>
Date:   Tue Dec 15 19:07:25 2020 +0300

    base

commit 1bd2f49fb5
Merge: 8670836573 063488d943
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 19:05:04 2020 +0300

    Merge branch 'master' into try_rewrite_writer

commit dff71850a8
Merge: 9d17f01dc9 063488d943
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 19:04:24 2020 +0300

    Merge branch 'master' into fix_watch_race_testkeeper

commit 9ad2a76d51
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 18:27:47 2020 +0300

    Update run.sh

commit 554fd738e9
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 18:22:21 2020 +0300

    Remove strange line

commit ccabb4680d
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 18:19:56 2020 +0300

    Fix dependencies

commit 063488d943
Merge: 31bef163ba 22aba554c0
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Tue Dec 15 18:07:42 2020 +0300

    Merge pull request #18103 from ClickHouse/revert-17983-fix-access-rights-for-merge-table-function

    Revert "Fix access rights required for the merge() table function."

commit 22aba554c0
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Tue Dec 15 18:07:20 2020 +0300

    Revert "Fix access rights required for the merge() table function."

commit 27ff3a5214
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Tue Dec 15 15:46:13 2020 +0100

    Rework using a separate flag file instead of renames

commit 31bef163ba
Merge: b252321ef7 c3a4e5e7b7
Author: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
Date:   Tue Dec 15 17:34:38 2020 +0300

    Merge pull request #17848 from azat/max_distributed_connections-fix

    Fix max_distributed_connections

commit dd09e1d783
Author: Anton Popov <pad11rus@gmail.com>
Date:   Tue Dec 15 17:34:37 2020 +0300

    fix type map ser/de to json

commit b252321ef7
Merge: def080c8ba 6defcbb662
Author: Nikolai Kochetov <KochetovNicolai@users.noreply.github.com>
Date:   Tue Dec 15 16:51:54 2020 +0300

    Merge pull request #16954 from Avogar/optimize-data-on-insert

    Optimize data on INSERT

commit 6defcbb662
Merge: c8c543ca94 def080c8ba
Author: Nikolai Kochetov <KochetovNicolai@users.noreply.github.com>
Date:   Tue Dec 15 16:50:42 2020 +0300

    Merge branch 'master' into optimize-data-on-insert

commit 873d9d1e1e
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 16:34:53 2020 +0300

    Better

commit 9bb0326fc6
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 16:33:14 2020 +0300

    Fix database drop timeout in clickhouse-test

commit 96f22a5bcc
Author: Steve-金勇 <staticor@me.com>
Date:   Tue Dec 15 18:55:52 2020 +0800

    Update limit.md

    Update translation of LIMIT , and providing more details for with ties modifier.

    Make document more human-readable for Chinese.

commit 9d17f01dc9
Merge: 73780838a1 def080c8ba
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 13:41:04 2020 +0300

    Merge branch 'master' into fix_watch_race_testkeeper

commit 8670836573
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 13:34:28 2020 +0300

    Add some comments

commit b56523fd94
Author: jianmei zhang <zhangjmruc@sina.com>
Date:   Tue Dec 15 17:56:54 2020 +0800

    Update test case to avoid the latest added parameter

commit 74c2211510
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 12:54:48 2020 +0300

    More clear

commit def080c8ba
Author: Alexander Kazakov <Akazz@users.noreply.github.com>
Date:   Tue Dec 15 11:16:25 2020 +0300

    Fixed unsafe ast rewrite (#18064)

    Fixed unsafe assignment to a reference in InterpreterSelectWithUnionQuery::InterpreterSelectWithUnionQuery()

commit a50187c995
Author: alesapin <alesapin@gmail.com>
Date:   Tue Dec 15 11:04:37 2020 +0300

    Slightly better

commit 508b50d121
Author: jianmei zhang <zhangjmruc@sina.com>
Date:   Tue Dec 15 14:44:39 2020 +0800

    support SHOW CHANGED SETTINGS query and add test cases, also do code changes to compile SHOW SETTINGS PROFILES.

commit 5a8c750b0b
Merge: 45b9d79f81 9310437920
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Tue Dec 15 09:17:52 2020 +0300

    Merge pull request #17983 from vitlibar/fix-access-rights-for-merge-table-function

    Fix access rights required for the merge() table function.

commit 45b9d79f81
Merge: a0405182ed 8de0c0e8fc
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Tue Dec 15 07:51:14 2020 +0300

    Merge pull request #18081 from azat/gcc10-sanitizers

    Do not pass -fsanitize-blacklist for gcc (it does not support it) for UBSAN

commit 3ec48cec53
Author: Denny Crane <deniszhuravlov@gmail.com>
Date:   Mon Dec 14 22:19:06 2020 -0400

    Update date-time-functions.md

    toYYYYMM  translation to Russian

commit a0405182ed
Merge: 49f65ecf9d fe6f477881
Author: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
Date:   Tue Dec 15 03:12:23 2020 +0300

    Merge pull request #18066 from ClickHouse/aku/utils-split-build

    Build utils in CI, at least in split build

commit 8de0c0e8fc
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Tue Dec 15 02:24:50 2020 +0300

    Fix warning message for unusupported -fsanitize-blacklist

commit 519d89dc9f
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Tue Dec 15 02:19:37 2020 +0300

    Do not pass -fsanitize-blacklist for gcc (it does not support it) for UBSAN

commit c78861e86e
Merge: afc2d7099b 49f65ecf9d
Author: Anton Popov <pad11rus@gmail.com>
Date:   Tue Dec 15 01:45:01 2020 +0300

    Merge remote-tracking branch 'upstream/master' into HEAD

commit afc2d7099b
Author: Anton Popov <pad11rus@gmail.com>
Date:   Tue Dec 15 01:16:04 2020 +0300

    add conversion from array to map

commit 49f65ecf9d
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 23:07:28 2020 +0300

    Update CHANGELOG.md

commit 91be7a6f27
Merge: 363ff79cca 5da9315ceb
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 23:03:21 2020 +0300

    Merge pull request #16809 from kitaisreal/read-float-8digits-unroll-performance-improvement

    ReadFloat 8digits unroll performance improvement

commit 363ff79cca
Merge: 491e086435 346c241c20
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 23:00:43 2020 +0300

    Merge pull request #17934 from Jokser/s3-adaptive-single-part-upload

    Adaptive choose of single/multi part upload in WriteBufferFromS3

commit 491e086435
Merge: 906415540a bf23bc6f0c
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 22:57:27 2020 +0300

    Merge pull request #18057 from azat/fix-01600_remerge_sort_lowered_memory_bytes_ratio-flap

    Fix 01600_remerge_sort_lowered_memory_bytes_ratio flap

commit 906415540a
Merge: 7184746f45 14dedae9f1
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 22:57:02 2020 +0300

    Merge pull request #18060 from kitaisreal/create-query-create-as-select-with-columns-specification

    CreateQuery create as select with columns specification

commit 7184746f45
Merge: 5f424f7c5c 60efab623d
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 22:56:39 2020 +0300

    Merge pull request #18059 from excitoon/patch-2

    Fixed `std::out_of_range: basic_string` in S3 URL parsing

commit 17b43ca721
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Tue Dec 8 13:54:33 2020 +0300

    Fixed tests

commit 16a30eb4aa
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Tue Dec 8 10:43:18 2020 +0300

    Updated non-finite floating conversion into int

commit e945821e1f
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 7 20:34:16 2020 +0300

    Update castColumn.h

commit 26e582272f
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 7 20:34:01 2020 +0300

    Update castColumn.h

commit 6873b13587
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 6 14:58:54 2020 +0300

    Fixed clang build issue

commit 0b565b679e
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sat Dec 5 17:55:37 2020 +0300

    Fixed Set execute for columns that cannot be inside Nullable

commit 18dc118298
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sat Dec 5 16:46:14 2020 +0300

    Fixed compile issues

commit 0464859cfe
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sat Dec 5 16:18:56 2020 +0300

    Updated usage of different types during IN query

    1. Added accurateCast function.
    2. Use accurateCast in Set during execute.
    3. Added accurateCast tests.
    4. Updated select_in_different_types tests.

commit 6f4bc77dbc
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Wed Dec 2 15:28:29 2020 +0300

    Added equality comparsion

commit f4b8e8ef99
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Wed Dec 2 13:09:13 2020 +0300

    Allow different types inside IN subquery

commit dff31e8de7
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Mon Nov 30 11:26:00 2020 +0300

    Fixed compile issues

commit 0dac432942
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sat Nov 28 15:55:29 2020 +0300

    Added const to arguments

commit dd072a20c9
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Nov 15 22:24:15 2020 +0300

    Fixed compile issues

commit 7b66dc374c
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Nov 15 13:03:53 2020 +0300

    Fixed clang compile issues

commit 4435642e39
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Nov 12 18:56:17 2020 +0300

    Fixed style issues

commit db5d0944ed
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Nov 12 14:27:02 2020 +0300

    Added Decimal support

commit 476fb3b677
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Nov 12 13:32:37 2020 +0300

    Updated documentation

commit baad78d2f2
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Nov 12 12:28:49 2020 +0300

    Fixed setting null value in vector. Updated tests.

commit 4cd5025e83
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Tue Nov 10 16:18:58 2020 +0300

    Added numeric limits for UInt128

commit d636e1618b
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Thu Nov 5 22:09:17 2020 +0300

    Added function accurate cast or null

commit fe6f477881
Author: Alexander Kuzmenkov <akuzm@yandex-team.ru>
Date:   Mon Dec 14 20:39:52 2020 +0300

    Build utils in CI, at least in split build

    Currently we don't build them at all, so they were broken by https://github.com/ClickHouse/ClickHouse/pull/17988

commit 5f424f7c5c
Author: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
Date:   Mon Dec 14 20:08:32 2020 +0300

    Update zzz-perf-comparison-tweaks-config.xml

commit 19af691aa6
Merge: e7f93b5563 b08842d44e
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 19:44:13 2020 +0300

    Merge pull request #17738 from vklimontovich/docker-cli-example

    Docker cli example

commit 73780838a1
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 19:01:29 2020 +0300

    Remove dead list watches

commit b08842d44e
Author: filimonov <1549571+filimonov@users.noreply.github.com>
Date:   Mon Dec 14 16:54:37 2020 +0100

    Update README.md

commit 1436986a86
Author: filimonov <1549571+filimonov@users.noreply.github.com>
Date:   Mon Dec 14 16:53:58 2020 +0100

    Update README.md

commit bca41bb7fd
Author: filimonov <1549571+filimonov@users.noreply.github.com>
Date:   Mon Dec 14 16:50:48 2020 +0100

    Update README.md

commit e7f93b5563
Merge: 2edabd5519 653ddab63e
Author: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
Date:   Mon Dec 14 17:54:47 2020 +0300

    Merge pull request #18058 from azat/perf-test-zk-port-fix

    Remove test_keeper_server in perf tests

commit 2edabd5519
Author: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
Date:   Mon Dec 14 17:45:58 2020 +0300

    Update zzz-perf-comparison-tweaks-config.xml

commit 24abcc970b
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Mon Dec 14 15:21:31 2020 +0100

    Tests and fixes

commit 87ea1528cb
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Tue Dec 8 16:54:49 2020 +0100

    support for Lazy database

commit 4ba80996b0
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Tue Dec 8 16:48:30 2020 +0100

    Detach permanently for DatabaseConnectionMySQL

commit 799141362d
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Tue Dec 8 11:00:32 2020 +0100

    Prevent attempts to recreate the table which was detached permanently

commit ad9cb580a9
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Mon Dec 7 16:19:05 2020 +0100

    Fix

commit 2aeb72c09a
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Mon Dec 7 15:01:09 2020 +0100

    Fix for tiny issue #13850

commit 3f5e5eb15a
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Mon Dec 7 14:55:24 2020 +0100

    Handle not implemented cases properly

commit f6097cf818
Author: Mikhail Filimonov <mfilimonov@altinity.com>
Date:   Mon Nov 30 18:52:32 2020 +0100

    Adding DETACH TABLE PERMANENTLY

commit 34d1ad1aca
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 17:27:39 2020 +0300

    Fix writer compact

commit dd2ae6926d
Merge: 7a8510d1c4 7f7eed4031
Author: tavplubix <avtokmakov@yandex-team.ru>
Date:   Mon Dec 14 16:27:13 2020 +0300

    Merge pull request #14849 from ClickHouse/allow_atomic_database_inside_materialize_mysql

    Allow Atomic database inside MaterializeMySQL

commit 35d2a689dd
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 16:06:11 2020 +0300

    One more time

commit 2a2b02d395
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 16:05:48 2020 +0300

    One more time

commit 9e98cdad5a
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 16:05:07 2020 +0300

    Accident changes

commit 47e3f619e5
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 16:01:01 2020 +0300

    Remove redundant code

commit a87fb3fae0
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 15:52:15 2020 +0300

    Fix style

commit 303c12d2e4
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 15:51:14 2020 +0300

    Slightly better

commit 14dedae9f1
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Mon Dec 14 15:50:11 2020 +0300

    Updated sql test file

commit 7a8510d1c4
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 15:45:13 2020 +0300

    Update CHANGELOG.md

commit ad70b0bd97
Merge: 0bc2603b1d 9f4929bdce
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 15:43:51 2020 +0300

    Merge pull request #18062 from ClickHouse/changelog-20.12

    Add changelog for 20.12

commit 9f4929bdce
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 15:30:37 2020 +0300

    Add changelog for 20.12

commit 55f991adde
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Mon Dec 14 15:23:53 2020 +0300

    Updated test

commit 6fb23da851
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 15:03:49 2020 +0300

    Refactoring

commit 70be03aa7d
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 14:36:15 2020 +0300

    Working compact parts

commit 0bc2603b1d
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 14:14:59 2020 +0300

    Update 01081_PartialSortingTransform_full_column.sql

commit ab17927c85
Merge: d55c5bec94 8a20d449d9
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 14:11:09 2020 +0300

    Merge pull request #18032 from kitaisreal/array-aggregation-functions

    Array aggregation functions

commit fbcf85cb42
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 14:06:02 2020 +0300

    Something working for wide parts

commit 8a20d449d9
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 14:01:31 2020 +0300

    Update arrayAggregation.cpp

commit d55c5bec94
Merge: 493ee6732b 2bf226decc
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 13:59:11 2020 +0300

    Merge pull request #18053 from azat/librdkafka-snappy-UB-fix

    Fix possible corruption in librdkafka snappy decompression

commit 493ee6732b
Merge: aac8b85b34 1bccd6dff6
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 13:58:45 2020 +0300

    Merge pull request #18052 from kitaisreal/memory-storage-read-optimization

    MemoryStorage read optimization

commit aac8b85b34
Merge: 86cef85186 6341521b10
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 13:58:15 2020 +0300

    Merge pull request #18054 from ClickHouse/remove-obsolete-settings

    Remove obsolete settings

commit 60efab623d
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 13:53:40 2020 +0300

    Update S3Common.cpp

commit 86cef85186
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 13:46:39 2020 +0300

    Remove old changelog generator to avoid confusion

commit 5f7c867da3
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Mon Dec 14 12:33:16 2020 +0300

    Added parse error if no select after as keyword

commit d8534ae1af
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Mon Dec 14 12:30:08 2020 +0300

    CreateQuery create as select with columns specification

commit a98e2311f8
Merge: e7ffae89e9 1b3893bcab
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Mon Dec 14 12:10:17 2020 +0300

    Merge pull request #17908 from vitlibar/fix-checking-introspection-grants

    Fix checking introspection grants

commit 418c974eaf
Author: Vladimir Chebotarev <vladimir.chebotarev@gmail.com>
Date:   Mon Dec 14 11:17:29 2020 +0300

    Fixed `std::out_of_range: basic_string` in S3 URL parsing.

commit 346c241c20
Merge: 59a5e63417 e7ffae89e9
Author: Pavel Kovalenko <jokserfn@yandex-team.ru>
Date:   Mon Dec 14 11:00:30 2020 +0300

    Merge remote-tracking branch 'origin/master' into s3-adaptive-single-part-upload

    # Conflicts:
    #	src/Disks/S3/DiskS3.cpp

commit 653ddab63e
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Mon Dec 14 10:59:01 2020 +0300

    Remove test_keeper_server in perf tests

    To avoid port overlaps for right and left server (anyway it is not
    used).

commit bf23bc6f0c
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Mon Dec 14 10:46:30 2020 +0300

    Fix 01600_remerge_sort_lowered_memory_bytes_ratio flap

commit 12b6a54a24
Merge: fd695c9ff4 e7ffae89e9
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 10:30:01 2020 +0300

    Merge branch 'master' into try_rewrite_writer

commit fd695c9ff4
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 14 10:28:42 2020 +0300

    Dive into debug

commit f829fbb928
Merge: 2169e8f7f4 e7ffae89e9
Author: jianmei zhang <zhangjmruc@sina.com>
Date:   Mon Dec 14 11:21:42 2020 +0800

    Merge remote-tracking branch 'upstream/master'

commit 2169e8f7f4
Author: jianmei zhang <zhangjmruc@sina.com>
Date:   Mon Dec 14 10:33:51 2020 +0800

    Fix issue #17160 by adding show settings like '...'

commit 5da9315ceb
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 05:22:17 2020 +0300

    Update readFloatText.h

commit 66b2424e33
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 05:20:00 2020 +0300

    Update readFloatText.h

commit 6341521b10
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 02:40:11 2020 +0300

    Remove obsolete settings

commit e7ffae89e9
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 02:31:35 2020 +0300

    Update build.sh

commit 52f03d942e
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Dec 14 02:31:18 2020 +0300

    Update Dockerfile

commit 1bccd6dff6
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Mon Dec 14 00:50:55 2020 +0300

    Fixed style issues

commit 2bf226decc
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Mon Dec 14 00:38:10 2020 +0300

    Fix typo in KafkaSettings comments

commit 3b431b054e
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Mon Dec 14 00:34:38 2020 +0300

    Bump librdkafka to fix UB in snappy decompression.

    In incremental_copy_fast_path there is undefined behavior (and in some
    other places too).

    And under this circumstances gcc10 with -O1 -ftree-loop-vectorize (or
    simply -O3), due to loop unroll, generates code that do copy by 16 bytes
    at a time for the second loop (MOVDQU+MOVUPS), while this is not correct
    since the memory may be overlapped and may be changed in the previous
    iteration.

    Updated version includes [1], that fixes those UB, by using memcpy over
    direct store/load since these days direct store/loads looks redundant.
    Even on ARM.

      [1]: https://github.com/ClickHouse-Extras/librdkafka/pull/3

    NOTE: clang is fine, and other older versions of gcc too

commit 2e569899d1
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Thu Dec 10 10:18:49 2020 +0300

    Add a test for kafka with snappy compression method (regression for UB in snappy)

    I've checked it manually and it works with clang build and does not with
    gcc (there are NULL bytes in the middle of value).

commit 69127ca0ed
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Mon Dec 14 00:21:25 2020 +0300

    Changed BlocksList to Blocks

commit 153610b2c3
Author: Dmitriy <sevirov@yandex-team.ru>
Date:   Sun Dec 13 23:44:09 2020 +0300

    Update distribution_queue.md

    Поменял also на Also.

commit 698737681a
Merge: 629f5f0797 8702809c9f
Author: Dmitriy <sevirov@yandex-team.ru>
Date:   Sun Dec 13 23:41:22 2020 +0300

    Merge remote-tracking branch 'remotes/clickhouse/master' into sevirov-DOCSUP-4919-document_replication_queue_system_table

commit 9310437920
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Thu Dec 10 23:16:53 2020 +0300

    Fix access rights required for the merge() table function.

commit 3c6df61e5c
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 13 22:24:15 2020 +0300

    MemoryStorage read optimization

commit 8702809c9f
Merge: 246e1fbb2f 0954cc2b5c
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 21:28:03 2020 +0300

    Merge pull request #17464 from amosbird/ddlworkermetric

    ddlworker max entry metric

commit 0be51cbabd
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 13 21:27:48 2020 +0300

    Updated tests

commit 246e1fbb2f
Merge: 4a2aaa68a8 149431ffc3
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 21:09:09 2020 +0300

    Merge pull request #18048 from ClickHouse/play-auto-dark-theme

    Automatic support for dark theme in Play interface

commit 4a2aaa68a8
Merge: 9116eb7aca 6d87743b0e
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 21:08:50 2020 +0300

    Merge pull request #18049 from ClickHouse/datasets-cdn

    CDN for datasets

commit 6d87743b0e
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 21:06:27 2020 +0300

    CDN for datasets

commit 149431ffc3
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 20:54:57 2020 +0300

    Automatic support for dark theme in Play interface

commit 9116eb7aca
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 19:35:56 2020 +0300

    Add checksums

commit d75c7ec30c
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 19:28:45 2020 +0300

    Redundancy.

commit 629f5f0797
Author: Dmitriy <sevirov@yandex-team.ru>
Date:   Sun Dec 13 18:42:34 2020 +0300

    Update replication_queue.md

    Выполнил перевод на русский язык, внес изменения в английскую версию.

commit d8484d9c31
Merge: 73890fa3a7 ef064696e7
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 18:28:52 2020 +0300

    Merge pull request #18033 from ClickHouse/encode-xml-rename-files

    encodeXMLComponent: rename files after #17659

commit 73890fa3a7
Merge: cd5f827265 b394487d4f
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 18:27:52 2020 +0300

    Merge pull request #18046 from baibaichen/feature/update_doc

    update Mac-OS build documentation

commit cd5f827265
Merge: 443d1724b3 06b047235a
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 18:27:01 2020 +0300

    Merge pull request #18047 from kitaisreal/space-saving-remove-last-element-from-map-fix

    HashTable erase added tests

commit 7f7eed4031
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Sun Dec 13 17:57:15 2020 +0300

    fix test

commit 0c80fce7f0
Merge: ed355f8663 443d1724b3
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Sun Dec 13 17:56:50 2020 +0300

    Merge branch 'master' into allow_atomic_database_inside_materialize_mysql

commit 5d6036e9c8
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 13 14:38:56 2020 +0300

    Fixed style issues

commit 06b047235a
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 13 14:35:52 2020 +0300

    Fixed style issues

commit 547f452ba9
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 13 13:30:25 2020 +0300

    HashTable erase added tests

commit b394487d4f
Author: chang.chen <baibaichen@gmail.com>
Date:   Sun Dec 13 16:10:07 2020 +0800

    update Mac-OS build documentation

commit 443d1724b3
Merge: a714b82f9b 065a64cc37
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 04:54:14 2020 +0300

    Merge pull request #18031 from sevirov/sevirov-DOCSUP-4709-document_distribution_queue_system_table

    DOCSUP-4709: Document system.distribution_queue table

commit a714b82f9b
Merge: 785317e542 784a484d80
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 04:52:39 2020 +0300

    Merge pull request #18036 from ClickHouse/benchmark-macbook2020

    Added results from MacBook Pro

commit 784a484d80
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 04:51:56 2020 +0300

    Added results from MacBook Pro

commit 785317e542
Merge: bce02de691 10beda9d39
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 04:44:47 2020 +0300

    Merge pull request #18035 from ClickHouse/benchmark-results-seodo

    Add benchmark results Yiğit Konur and Metehan Çetinkaya of seo.do

commit 10beda9d39
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 04:44:07 2020 +0300

    Add benchmark results Yiğit Konur and Metehan Çetinkaya of seo.do

commit bce02de691
Merge: 436adf34df ccd794a676
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 04:38:50 2020 +0300

    Merge pull request #17539 from azat/remerge_sort_lowered_memory_bytes_ratio

    Add remerge_sort_lowered_memory_bytes_ratio setting

commit 436adf34df
Merge: 4ce148d2ea 8d79ae56f0
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 04:37:29 2020 +0300

    Merge pull request #17881 from gyuton/gyuton-DOCSUP-4263-Update_the_sql_statement_ORDER_BY

    DOCSUP-4263: Updated the sql statement order by

commit 4ce148d2ea
Merge: 8df4789113 17a8a4e966
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 04:34:46 2020 +0300

    Merge pull request #18034 from edani/master

    Expand max_parallel_replicas documentation

commit 8df4789113
Merge: 04e222f6f3 6f50a86025
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 04:09:51 2020 +0300

    Merge pull request #17845 from kitaisreal/space-saving-remove-last-element-from-map-fix

    SpaceSaving remove last element from map fix

commit 04e222f6f3
Merge: 2ad5c8eed3 41bc43325b
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 01:12:05 2020 +0300

    Merge pull request #17986 from azat/fwd-decl

    More forward declaration for generic headers

commit 17a8a4e966
Author: Eric Daniel <6210036+edani@users.noreply.github.com>
Date:   Sat Dec 12 13:57:01 2020 -0800

    Expand max_parallel_replicas documentation

    Based on discussion in #17807

commit 5ae1652d51
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 13 00:47:17 2020 +0300

    Fixed aggregate function avg

commit 2ad5c8eed3
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 00:45:23 2020 +0300

    Allow server to start with TestKeeper port in development environment

commit b810b28b08
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 13 00:40:34 2020 +0300

    Added comment

commit 4887d59b06
Merge: 2eabd41749 c2c497e290
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 00:39:41 2020 +0300

    Merge pull request #18027 from Jokser/fix-flappy-log-family-s3

    Fix flappy test_log_family_s3

commit 0dcbcec19a
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 13 00:29:09 2020 +0300

    Added tests

commit ef064696e7
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sun Dec 13 00:17:37 2020 +0300

    Add perf test

commit 532557ecfd
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sat Dec 12 23:47:37 2020 +0300

    encodeXMLComponent: rename files after #17659

commit 18bde6ef02
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sat Dec 12 23:47:23 2020 +0300

    Fixed compile issues

commit 8d79ae56f0
Author: George <gyuton@yandex-team.ru>
Date:   Sat Dec 12 23:07:24 2020 +0300

    created lowcardinality files for ja and tr

commit 3d261c82e9
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sat Dec 12 22:50:28 2020 +0300

    Array aggregation functions

commit 065a64cc37
Author: Dmitriy <sevirov@yandex-team.ru>
Date:   Sat Dec 12 22:43:52 2020 +0300

    Update distribution_queue.md

    Добавил ссылку на движок таблиц Distributed.

commit a244a4f0e3
Merge: c03e91eec0 6e417088dc
Author: Dmitriy <sevirov@yandex-team.ru>
Date:   Sat Dec 12 22:28:09 2020 +0300

    Merge branch 'sevirov-DOCSUP-4709-document_distribution_queue_system_table' of https://github.com/sevirov/ClickHouse into sevirov-DOCSUP-4709-document_distribution_queue_system_table

commit c03e91eec0
Author: Dmitriy <sevirov@yandex-team.ru>
Date:   Sat Dec 12 22:27:01 2020 +0300

    Create distribution_queue.md

    Выполнил перевод на русский язык.

commit 6f50a86025
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sat Dec 12 18:57:07 2020 +0300

    Update gtest_hash_table.cpp

commit 41bc43325b
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Fri Dec 11 21:10:37 2020 +0300

    Make 01290_max_execution_speed_distributed less flaky

commit 77a1d00dee
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Fri Dec 11 21:10:36 2020 +0300

    Add new remote context as interpreter context to the Pipe

commit 5b3ab48861
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Fri Dec 11 01:05:02 2020 +0300

    More forward declaration for generic headers

    The following headers are pretty generic, so use forward declaration as
    much as possible:
    - Context.h
    - Settings.h
    - ConnectionTimeouts.h
    (Also this shows that some missing some includes -- this has been fixed)

    And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
    module part cannot be added for it, due to recursive build dependencies
    that will be introduced)

    Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
    and just pass the context, since settings was passed only in speicifc
    places, that can allow making a copy of Context (i.e. Copier).

    Approx results (How much units will be recompiled after changing file X?):

    - ConnectionTimeouts.h
      - mainline: 100

    - Context.h:
      - mainline: ~800
      - patched:  415

    - Settings.h:
      - mainline: 900-1K
      - patched:  440 (most of them because of the Context.h)

commit c2c497e290
Author: Pavel Kovalenko <jokserfn@yandex-team.ru>
Date:   Sat Dec 12 16:44:19 2020 +0300

    Use s3 disk explicitly for log tables in test_log_family_s3.

commit 07c8293043
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sat Dec 12 15:08:46 2020 +0300

    Added tests

commit c7d48c48ce
Author: George <gyuton@yandex-team.ru>
Date:   Sat Dec 12 11:58:07 2020 +0300

    more troubleshooting

commit fa9257cd82
Author: George <gyuton@yandex-team.ru>
Date:   Sat Dec 12 11:14:46 2020 +0300

    fix

commit 1567444628
Author: George <gyuton@yandex-team.ru>
Date:   Sat Dec 12 10:36:39 2020 +0300

    troubleshooting - fixed links to original articles

commit 0954cc2b5c
Author: Amos Bird <amosbird@gmail.com>
Date:   Sun Nov 29 21:28:11 2020 +0800

    test is not working

commit 8d0e949462
Author: Amos Bird <amosbird@gmail.com>
Date:   Fri Nov 27 12:27:43 2020 +0800

    ddlworker max entry metric

commit 9b1d919110
Author: George <gyuton@yandex-team.ru>
Date:   Sat Dec 12 06:39:22 2020 +0300

    more troubleshooting

commit 3794cb2d3f
Author: George <gyuton@yandex-team.ru>
Date:   Sat Dec 12 05:48:23 2020 +0300

    troubleshooting the error

commit c74011851a
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sat Dec 12 04:58:07 2020 +0300

    Update HashTable.h

commit d94d9aa8d7
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sat Dec 12 04:17:32 2020 +0300

    Update HashTable.h

commit d5f87efba5
Author: George <gyuton@yandex-team.ru>
Date:   Sat Dec 12 04:16:58 2020 +0300

    fixed a typo in translation

commit 99212dcb73
Author: George <gyuton@yandex-team.ru>
Date:   Sat Dec 12 04:06:37 2020 +0300

    added translation

commit 9e48b9b656
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sat Dec 12 03:47:43 2020 +0300

    Update HashTable.h

commit cabe1463d1
Author: George <gyuton@yandex-team.ru>
Date:   Sat Dec 12 03:39:45 2020 +0300

    fixed an error

commit eba1299845
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sat Dec 12 03:38:55 2020 +0300

    Update HashTable.h

commit 245dea6f2e
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sat Dec 12 03:33:57 2020 +0300

    Update HashTable.h

commit 7e2bc1e7a6
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sat Dec 12 03:30:12 2020 +0300

    Update HashTable.h

commit 6e7366f921
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Sat Dec 12 03:28:41 2020 +0300

    Update HashTable.h

commit 59a5e63417
Merge: 5802633f95 f1a145a2b2
Author: Pavel Kovalenko <jokserfn@yandex-team.ru>
Date:   Fri Dec 11 18:28:41 2020 +0300

    Merge remote-tracking branch 'origin/master' into s3-adaptive-single-part-upload

    # Conflicts:
    #	src/Disks/S3/DiskS3.cpp
    #	src/Disks/S3/DiskS3.h
    #	src/Disks/S3/registerDiskS3.cpp
    #	src/IO/WriteBufferFromS3.cpp
    #	src/IO/WriteBufferFromS3.h

commit a24a76828e
Author: alesapin <alesapin@gmail.com>
Date:   Fri Dec 11 18:18:51 2020 +0300

    Fix

commit 4b83707c63
Author: alesapin <alesapin@gmail.com>
Date:   Fri Dec 11 18:00:58 2020 +0300

    Compileable code for some reason

commit c6ac831d4e
Author: Amos Bird <amosbird@gmail.com>
Date:   Fri Dec 11 22:44:22 2020 +0800

    Fix type

commit b70b98c415
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Wed Dec 9 23:56:15 2020 +0300

    Fix style issues

commit 4ae5619714
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Wed Dec 9 22:44:43 2020 +0300

    Optimized erase in HashTable

commit bf5d75853c
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Wed Dec 9 15:47:22 2020 +0300

    Added erase into HashTable

commit bfceb06659
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 6 22:16:21 2020 +0300

    Fixed test

commit d8b78682e1
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Sun Dec 6 22:14:40 2020 +0300

    SpaceSaving remove last element from map fix

commit ed355f8663
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Fri Dec 11 16:50:45 2020 +0300

    fix

commit 1b3893bcab
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Tue Dec 8 21:09:13 2020 +0300

    User with allow_ddl=0 cannot do DDL but can grant DDL.

commit 648be453a4
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Tue Dec 8 21:07:50 2020 +0300

    User with allow_introspection_functions=0 cannot call introspection functions but can grant INTROSPECTION.

commit 710ba6f617
Author: Vitaly Baranov <vitbar@yandex-team.ru>
Date:   Tue Dec 8 17:24:32 2020 +0300

    Simplify class ContextAccess.

commit 57c30c4164
Author: alesapin <alesapin@gmail.com>
Date:   Fri Dec 11 16:20:19 2020 +0300

    Garbage

commit 58a31e4882
Merge: 0ba712a2d3 fb1221148d
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Fri Dec 11 16:14:03 2020 +0300

    Merge branch 'master' into allow_atomic_database_inside_materialize_mysql

commit 7ffd76a605
Author: alesapin <alesapin@gmail.com>
Date:   Fri Dec 11 11:41:02 2020 +0300

    Fix bug

commit cbba73672e
Merge: 1cb06bd975 e11790f61a
Author: alesapin <alesapin@gmail.com>
Date:   Fri Dec 11 11:09:47 2020 +0300

    Merge branch 'master' into try_rewrite_writer

commit 5b1e5679b4
Author: Amos Bird <amosbird@gmail.com>
Date:   Wed Nov 11 12:05:54 2020 +0800

    Add -SimpleState combinator

commit 1cb06bd975
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 10 19:29:10 2020 +0300

    Split one method

commit 90fa9a2073
Merge: f3d1dc360c b1c00d4045
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 10 18:36:10 2020 +0300

    Merge branch 'master' into try_rewrite_writer

commit f3d1dc360c
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 10 12:24:52 2020 +0300

    Fix style

commit 0704b478ee
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 10 12:22:43 2020 +0300

    Less constuctors

commit 2787b615b0
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 10 11:57:52 2020 +0300

    Make interface as simple as possible

commit 7979b3655c
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 9 21:22:07 2020 +0300

    Better formating

commit 3b88c077ed
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 9 21:19:49 2020 +0300

    Mechanical refactoring

commit af73db93e6
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 9 21:10:09 2020 +0300

    Mechanical refactoring

commit 7b88f9a115
Merge: fa64d314df 89a3a86b71
Author: alesapin <alesapin@gmail.com>
Date:   Wed Dec 9 19:23:53 2020 +0300

    Merge branch 'master' into try_rewrite_writer

commit 5802633f95
Author: Pavel Kovalenko <jokserfn@yandex-team.ru>
Date:   Wed Dec 9 18:38:02 2020 +0300

    Fix parameter name in constructor of DiskS3

commit c8c543ca94
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Wed Dec 9 18:07:58 2020 +0300

    Add throwing exceptions

commit db7a87089b
Author: Pavel Kovalenko <jokserfn@yandex-team.ru>
Date:   Wed Dec 9 17:09:04 2020 +0300

    Adaptive choose of single/multi part upload in WriteBufferFromS3.

commit 215c5675dd
Author: Dmitriy <sevirov@yandex-team.ru>
Date:   Tue Dec 8 23:25:45 2020 +0300

    Update replication_queue.md

    Изменил описание столбца node_name.

commit ce721f95ee
Author: Dmitriy <sevirov@yandex-team.ru>
Date:   Tue Dec 8 21:46:25 2020 +0300

    Create replication_queue.md

    Задокументировал системную таблицу replication_queue.

commit 291959c1b3
Author: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
Date:   Tue Dec 8 19:19:51 2020 +0300

    Update comment

commit 0ba712a2d3
Merge: a0ef3a113a d9b52f94f7
Author: Ivan <5627721+abyss7@users.noreply.github.com>
Date:   Tue Dec 8 16:45:51 2020 +0300

    Merge branch 'master' into allow_atomic_database_inside_materialize_mysql

commit e19eb6f17a
Merge: 5ae6c6dab9 d9b52f94f7
Author: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
Date:   Tue Dec 8 15:57:46 2020 +0300

    Merge branch 'master' into optimize-data-on-insert

commit 4e49c59ed0
Author: George <gyuton@yandex-team.ru>
Date:   Tue Dec 8 04:55:57 2020 +0300

    minor fix

commit 6be3933cea
Author: George <gyuton@yandex-team.ru>
Date:   Tue Dec 8 04:27:26 2020 +0300

    fixed links v2

commit 8760fc1227
Author: George <gyuton@yandex-team.ru>
Date:   Tue Dec 8 04:07:34 2020 +0300

    Fixed links

commit 1f0cfbaca8
Author: George <gyuton@yandex-team.ru>
Date:   Tue Dec 8 03:51:41 2020 +0300

    Added Collation description

commit 371cb23946
Merge: b68aa1c76d dd8362a5eb
Author: alesapin <alesapin@gmail.com>
Date:   Mon Dec 7 13:30:41 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit c3a4e5e7b7
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Mon Dec 7 01:12:54 2020 +0300

    Fix max_distributed_connections (one more time)

    With prefer_localhost_replica=1 max_distributed_connections uses
    max_threads before this patch.

    Fixes: #14936
    Previous fix: #9673

commit ce03a896a1
Author: Anton Popov <pad11rus@gmail.com>
Date:   Sat Dec 5 22:51:16 2020 +0300

    remove unused files

commit 7aaa2c052f
Merge: 6b09669d66 67c82eb61b
Author: Anton Popov <pad11rus@gmail.com>
Date:   Sat Dec 5 22:41:46 2020 +0300

    Merge branch 'master' into merging-map

commit 6b09669d66
Author: Anton Popov <pad11rus@gmail.com>
Date:   Thu Dec 3 06:52:41 2020 +0300

    data type map: several fixes

commit 5ae6c6dab9
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Fri Dec 4 20:40:28 2020 +0300

    Fix build error

commit 4b58528b9e
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Fri Dec 4 19:25:30 2020 +0300

    Rename getPos to getRow, change mergeBlock, pass setting instead of context

commit 905ba78adc
Merge: e5588b94aa 7c2a04d77d
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Fri Dec 4 18:56:46 2020 +0300

    Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert

commit b68aa1c76d
Merge: 9df414fa0c 2b3281888a
Author: alesapin <alesapin@gmail.com>
Date:   Thu Dec 3 17:22:09 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit 9bf5ab9f4c
Author: Vladimir Klimontovich <v@klimontovich.com>
Date:   Wed Dec 2 19:01:31 2020 +0300

    Removed unnecessary mapping

commit 274c87d4c7
Author: Vladimir Klimontovich <v@klimontovich.com>
Date:   Wed Dec 2 19:00:23 2020 +0300

    Remove extra space

commit a0ef3a113a
Merge: e82c63e05c 9b72323594
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Wed Dec 2 18:29:02 2020 +0300

    Merge branch 'master' into allow_atomic_database_inside_materialize_mysql

commit d402d68071
Author: Vladimir Klimontovich <v@klimontovich.com>
Date:   Wed Dec 2 18:26:16 2020 +0300

    Added HTTP CLI example and expose 8123 port

    Exposing the 8123 (HTTP) interface in basic example and giving CLI connection instruction makes it easier to start. For a lot of developers, curl is a native and very familiar tool

commit 12604ce0e6
Merge: 21bbc7bc19 5b3a7f5c98
Author: Anton Popov <pad11rus@gmail.com>
Date:   Tue Dec 1 14:52:30 2020 +0300

    Merge branch 'dev_map2' of git://github.com/hexiaoting/ClickHouse into merging-map

commit ccd794a676
Author: Azat Khuzhin <a3at.mail@gmail.com>
Date:   Sat Nov 28 19:32:28 2020 +0300

    Add remerge_sort_lowered_memory_bytes_ratio setting

    Sometimes default ratio is not good enough (2), since it depends on lots
    of factors:
    - size of LIMIT
    - max_memory_usage
    - ...

    So add a separate setting for it.

    But note that it make sense to set
    remerge_sort_lowered_memory_bytes_ratio only in range (1, 2]

    Since <= 1 will just use more CPU, and > 2 will use more RAM and will
    unlikely be faster.

commit f46eb6f2be
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sun Nov 29 18:16:47 2020 +0300

    Update tests

commit 7d79a3cc35
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sun Nov 29 18:13:57 2020 +0300

    Make insert_quorum_parallel by default

commit 2e6bedce91
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sun Nov 29 18:08:02 2020 +0300

    Relax "Too many parts" threshold, part 1

commit 5b3a7f5c98
Merge: 74d875c837 747453b008
Author: hexiaoting <hewenting_ict@163.com>
Date:   Thu Nov 26 16:43:27 2020 +0800

    Merge remote-tracking branch 'origin/master' into dev_map2

commit 74d875c837
Merge: c7c14cc6ad b494b18748
Author: hexiaoting <hewenting_ict@163.com>
Date:   Thu Nov 26 15:13:52 2020 +0800

    Merge remote-tracking branch 'origin/master' into dev_map2

commit e82c63e05c
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Tue Nov 24 15:28:54 2020 +0300

    fixes

commit dce0bf8efc
Merge: 5f6bbe8967 45f09fe293
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Tue Nov 24 15:28:31 2020 +0300

    Merge branch 'master' into allow_atomic_database_inside_materialize_mysql

commit e5588b94aa
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Fri Nov 20 13:38:53 2020 +0300

    Fix style and tests

commit 625962fb59
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Fri Nov 20 03:19:36 2020 +0300

    Remove debug logging

commit c3c32ff9f8
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Fri Nov 20 03:16:57 2020 +0300

    Minor change

commit 608722b6ab
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Fri Nov 20 02:59:58 2020 +0300

    Fix integration tests

commit 3ee4972586
Merge: cc1b16ea81 4b7a7ca12a
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Fri Nov 20 01:23:51 2020 +0300

    Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert

commit cc1b16ea81
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Wed Nov 18 13:40:58 2020 +0300

    Fix test

commit 547ec19fb3
Merge: 4ed7657008 0bc60e2d53
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Wed Nov 18 12:01:59 2020 +0300

    Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert

commit 4ed7657008
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Tue Nov 17 22:07:19 2020 +0300

    fix typo

commit 53b8ff42c2
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Tue Nov 17 19:56:36 2020 +0300

    Add test and set setting to 0 in some other tests

commit 437bbef5b9
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Mon Nov 9 21:56:49 2020 +0300

    Fix marker check

commit 1b9c20a5df
Author: Maksim Kita <kitaetoya@gmail.com>
Date:   Mon Nov 9 14:36:44 2020 +0300

    ReadFloat 8digits unroll performance improvement

commit 264e9daf6f
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Fri Nov 13 11:16:30 2020 +0300

    Fix style

commit 6a57c0a8cf
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Fri Nov 13 10:54:05 2020 +0300

    Move merge in MergeTreeDataWriter

commit 8d5e0784d3
Author: Pavel Kruglov <avogar@sandbox-633380738>
Date:   Thu Nov 12 23:29:36 2020 +0300

    Add setting optimize_on_insert

commit 9df414fa0c
Author: alesapin <alesapin@gmail.com>
Date:   Tue Nov 10 13:54:56 2020 +0300

    One more test to skip

commit fe4bd6a7d2
Merge: e2ece9ef0a 059357d51e
Author: alesapin <alesapin@gmail.com>
Date:   Tue Nov 10 13:54:43 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit c7c14cc6ad
Author: hexiaoting <hewenting_ict@163.com>
Date:   Tue Nov 10 11:24:39 2020 +0800

    fix bug

commit e2ece9ef0a
Merge: 8fa6831185 24e57c659a
Author: alesapin <alesapin@gmail.com>
Date:   Mon Nov 9 13:24:47 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit ef0b78cf12
Author: hexiaoting <hewenting_ict@163.com>
Date:   Fri Nov 6 17:21:26 2020 +0800

    style error fix

commit 28e87da976
Merge: 66bca89001 ccf57300eb
Author: hexiaoting <hewenting_ict@163.com>
Date:   Fri Nov 6 15:07:41 2020 +0800

    remove map literal and Merge remote-tracking branch 'origin/master' into dev_map2

commit 8fa6831185
Merge: 5622a3f57f 01898be459
Author: alesapin <alesapin@gmail.com>
Date:   Tue Nov 3 10:16:28 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit 66bca89001
Author: hexiaoting <hewenting_ict@163.com>
Date:   Mon Nov 2 17:23:02 2020 +0800

    Style error fix

commit 5622a3f57f
Author: alesapin <alesapin@gmail.com>
Date:   Mon Nov 2 10:46:43 2020 +0300

    Update skip list json

commit 3fe6f1e3fd
Author: hexiaoting <hewenting_ict@163.com>
Date:   Mon Nov 2 15:10:29 2020 +0800

    Add test case for tuple convert to map

commit 3a70cf3773
Merge: e1ff6c3870 dc5fc9ddcb
Author: hexiaoting <hewenting_ict@163.com>
Date:   Mon Nov 2 14:46:56 2020 +0800

    Merge remote-tracking branch 'origin/master' into dev_map2

commit e1ff6c3870
Author: hexiaoting <hewenting_ict@163.com>
Date:   Mon Nov 2 14:46:32 2020 +0800

    Update codes

commit 619fe6b10b
Merge: 2ba68d7494 ceab1add26
Author: hexiaoting <hewenting_ict@163.com>
Date:   Mon Nov 2 14:07:00 2020 +0800

    Merge remote-tracking branch 'hexiaoting/dev_map2' into dev_map2

commit 2ba68d7494
Author: hexiaoting <hewenting_ict@163.com>
Date:   Mon Nov 2 14:05:53 2020 +0800

    Add map function

commit 4e45253c02
Author: alesapin <alesapin@gmail.com>
Date:   Thu Oct 29 19:05:56 2020 +0300

    Update skip_list.json

commit b5553c0c23
Author: alesapin <alesapin@gmail.com>
Date:   Thu Oct 29 15:48:02 2020 +0300

    Update skip_list.json

commit 8a6574767a
Merge: 6345769b01 2e8594159c
Author: alesapin <alesapin@gmail.com>
Date:   Thu Oct 29 12:57:56 2020 +0300

    Merge branch 'run_func_tests_in_parallel' of github.com:ClickHouse/ClickHouse into run_func_tests_in_parallel

commit 6345769b01
Merge: 362b2c1cb2 745cb4ab2f
Author: alesapin <alesapin@gmail.com>
Date:   Thu Oct 29 12:56:54 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit 2e8594159c
Author: alesapin <alesapin@gmail.com>
Date:   Wed Oct 28 22:25:57 2020 +0300

    Update skip_list.json

commit 964149586e
Author: alesapin <alesapin@gmail.com>
Date:   Wed Oct 28 18:20:21 2020 +0300

    Update tests/queries/skip_list.json

    Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>

commit ceab1add26
Merge: 483be134b2 e0d85ffd1d
Author: hexiaoting <hewenting_ict@163.com>
Date:   Wed Oct 28 19:13:40 2020 +0800

    Merge pull request #1 from Enmk/ColumnMap_fixes

    Allow casting Tuple as Map.

commit 362b2c1cb2
Merge: b9f63645b6 4a63f1271b
Author: alesapin <alesapin@gmail.com>
Date:   Wed Oct 28 14:12:45 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit e0d85ffd1d
Author: Vasily Nemkov <V.Nemkov@gmail.com>
Date:   Mon Oct 26 21:20:45 2020 +0300

    Allow casting Tuple as Map.

    SELECT CAST(([1, 2, 3], ['1', '2', 'foo']), 'Map(UInt8, String)') AS map

commit 2e232bad74
Merge: aa3a43c964 cc5f15da29
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Oct 26 22:51:58 2020 +0300

    Merge branch 'master' into sigaltstack

commit aa3a43c964
Merge: 0ba2fa8105 a591d1f32a
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Oct 26 22:51:55 2020 +0300

    Merge branch 'sigaltstack' of github.com:yandex/ClickHouse into sigaltstack

commit a591d1f32a
Author: alexey-milovidov <milovidov@yandex-team.ru>
Date:   Mon Oct 26 22:46:25 2020 +0300

    Update ThreadStatus.cpp

commit b9f63645b6
Author: tavplubix <avtokmakov@yandex-team.ru>
Date:   Mon Oct 26 12:45:30 2020 +0300

    Update 01107_atomic_db_detach_attach.sh

commit eab3371d5f
Author: tavplubix <avtokmakov@yandex-team.ru>
Date:   Mon Oct 26 12:42:54 2020 +0300

    Update 01114_database_atomic.sh

commit 0ba2fa8105
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Oct 26 11:56:30 2020 +0300

    Fix "Arcadia"

commit c1cce28470
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 26 09:25:20 2020 +0300

    Update skip_list.json

commit 46fc6733e2
Merge: b472bd35b7 18458f36e7
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Oct 26 08:23:47 2020 +0300

    Merge branch 'master' into sigaltstack

commit b472bd35b7
Merge: 88e846b36d bfe17d7dca
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Oct 26 07:42:33 2020 +0300

    Merge branch 'master' into sigaltstack

commit 88e846b36d
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Oct 26 07:42:22 2020 +0300

    Fix build on Mac OS

commit 068235c2d7
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Oct 26 05:18:36 2020 +0300

    Disable alt stack for sanitizers

commit 1228396eff
Merge: 4e2c0ba226 772faeb74f
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Mon Oct 26 05:09:33 2020 +0300

    Merge branch 'master' into sigaltstack

commit 4e2c0ba226
Author: Alexey Milovidov <milovidov@yandex-team.ru>
Date:   Sun Oct 25 03:10:05 2020 +0300

    Provide diagnostics on stack overflow

commit 96a10f3664
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 23 15:19:03 2020 +0300

    Update skip_list.json

commit bbc421acb7
Merge: b158a22a4f 6285fd1378
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 23 13:22:34 2020 +0300

    Merge branch 'run_func_tests_in_parallel' of github.com:ClickHouse/ClickHouse into run_func_tests_in_parallel

commit b158a22a4f
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 23 13:21:40 2020 +0300

    Bump CI

commit 483be134b2
Merge: 521b07d37a 0082c56b67
Author: hexiaoting <hewenting_ict@163.com>
Date:   Fri Oct 23 16:36:17 2020 +0800

    Support MergeTree Engine

commit 6285fd1378
Author: alesapin <alesapin@gmail.com>
Date:   Thu Oct 22 17:41:34 2020 +0300

    Update skip_list.json

commit d04edef048
Author: alesapin <alesapin@gmail.com>
Date:   Thu Oct 22 10:20:45 2020 +0300

    More tests to skip

commit fedf169bb6
Merge: 96daa31436 0082c56b67
Author: alesapin <alesapin@gmail.com>
Date:   Thu Oct 22 10:19:48 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit 96daa31436
Merge: 620c8e3d38 4261455621
Author: alesapin <alesapin@gmail.com>
Date:   Wed Oct 21 22:53:18 2020 +0300

    Merge branch 'run_func_tests_in_parallel' of github.com:ClickHouse/ClickHouse into run_func_tests_in_parallel

commit 620c8e3d38
Merge: cc43b002cc 6a57bc8c0b
Author: alesapin <alesapin@gmail.com>
Date:   Wed Oct 21 22:52:30 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit 4261455621
Author: alesapin <alesapin@gmail.com>
Date:   Wed Oct 21 17:18:48 2020 +0300

    Update skip_list.json

commit cc43b002cc
Merge: df4620b7bd c53f59dece
Author: alesapin <alesapin@gmail.com>
Date:   Wed Oct 21 12:51:43 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit df4620b7bd
Merge: 85ca6f7e6a b4f0e08369
Author: alesapin <alesapin@gmail.com>
Date:   Tue Oct 20 17:15:56 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit 521b07d37a
Author: hexiaoting <hewenting_ict@163.com>
Date:   Sun Oct 18 20:40:51 2020 +0800

    Fix build error

commit fcd35b0619
Merge: db8ba39c9c 6dc5cb166f
Author: hexiaoting <hewenting_ict@163.com>
Date:   Sun Oct 18 16:18:13 2020 +0800

    Merge branch 'master' into dev_map2

commit db8ba39c9c
Author: hexiaoting <hewenting_ict@163.com>
Date:   Sat Oct 17 23:45:35 2020 +0800

    Implement Integer And Array for value type

commit 85ca6f7e6a
Merge: 8f3d6dae2d e89a3b5d09
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 16 13:43:30 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit 8f3d6dae2d
Merge: 09bb6e9944 0fa35cdc80
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 16 10:26:57 2020 +0300

    Merge branch 'run_func_tests_in_parallel' of github.com:ClickHouse/ClickHouse into run_func_tests_in_parallel

commit 09bb6e9944
Merge: 9a235cfb8b 0bb4480fee
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 16 10:26:33 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit cb9e88efa3
Author: hexiaoting <hewenting_ict@163.com>
Date:   Wed Oct 14 15:35:22 2020 +0800

    Update with fresh master to build sucessfully.

commit 4f3f221987
Author: hexiaoting <hewenting_ict@163.com>
Date:   Tue Oct 13 15:02:03 2020 +0800

    Fix build error

commit 03102fead3
Author: hexiaoting <hewenting_ict@163.com>
Date:   Sun Oct 11 22:22:54 2020 +0800

    Fix style check error.

commit 974352a8e1
Author: hexiaoting <hewenting_ict@163.com>
Date:   Sat Oct 10 18:16:05 2020 +0800

    Fix style check errors.

commit cdb4dd57a6
Author: hexiaoting <hewenting_ict@163.com>
Date:   Sat Oct 10 14:49:03 2020 +0800

    Add new DataType Map(key,value)

commit 0fa35cdc80
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Tue Oct 13 19:03:25 2020 +0300

    use async drop in some tests

commit 9a235cfb8b
Author: alesapin <alesapin@gmail.com>
Date:   Tue Oct 13 17:29:24 2020 +0300

    Database atomic sync drop detach

commit 7d735b4725
Author: alesapin <alesapin@gmail.com>
Date:   Tue Oct 13 13:40:00 2020 +0300

    Update max_concurrent_queries.xml

commit 2909378bbc
Author: alesapin <alesapin@gmail.com>
Date:   Tue Oct 13 09:52:37 2020 +0300

    Bump CI

commit 97e89afea7
Merge: d75d486029 38c7132c0f
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 12 22:40:57 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit d75d486029
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 12 13:20:52 2020 +0300

    Update skip_list.json

commit 2a63a619cc
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 9 22:42:51 2020 +0300

    Update skip_list.json

commit e112fbe2a9
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 9 14:51:34 2020 +0300

    Update skip_list.json

commit cd28dab666
Merge: 7b3d430149 e6c6930397
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 9 10:26:40 2020 +0300

    Merge branch 'run_func_tests_in_parallel' of github.com:ClickHouse/ClickHouse into run_func_tests_in_parallel

commit 7b3d430149
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 9 10:25:50 2020 +0300

    Add more tests

commit 2b66180c70
Merge: 9982c17814 66b7748b63
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 9 10:25:35 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit e6c6930397
Author: alesapin <alesapin@gmail.com>
Date:   Wed Oct 7 23:01:03 2020 +0300

    Update skip_list.json

commit e7d1d34a60
Author: alesapin <alesapin@gmail.com>
Date:   Wed Oct 7 12:43:49 2020 +0300

    Update skip_list.json

commit d2762624f5
Author: alesapin <alesapin@gmail.com>
Date:   Tue Oct 6 22:45:17 2020 +0300

    Update skip_list.json

commit 9982c17814
Author: alesapin <alesapin@gmail.com>
Date:   Tue Oct 6 19:02:33 2020 +0300

    Skip more

commit 4f6d794816
Author: alesapin <alesapin@gmail.com>
Date:   Tue Oct 6 14:41:35 2020 +0300

    Trying 8

commit a5f8439761
Author: alesapin <alesapin@gmail.com>
Date:   Tue Oct 6 12:18:49 2020 +0300

    Trying 12 threads

commit 4330fc0f39
Merge: a257751924 a49591b250
Author: alesapin <alesapin@gmail.com>
Date:   Tue Oct 6 10:26:55 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit a257751924
Author: alesapin <alesapin@gmail.com>
Date:   Tue Oct 6 10:13:55 2020 +0300

    Ignore max concurrent queries in tests

commit 99ea5cca10
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 5 23:30:14 2020 +0300

    Bump

commit 766802ae4d
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 5 22:34:50 2020 +0300

    Bump

commit 2f9a6fe479
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 5 18:49:40 2020 +0300

    More tests to skip

commit b2f2a274c0
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 5 16:08:39 2020 +0300

    One more time

commit 8561a11620
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 5 12:56:48 2020 +0300

    Revert "Better fix"

    This reverts commit 0496987c8f.

commit 0496987c8f
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 5 12:56:20 2020 +0300

    Better fix

commit 916cef0487
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 5 12:45:11 2020 +0300

    Bump CI

commit edd5a4b936
Author: alesapin <alesapin@gmail.com>
Date:   Mon Oct 5 10:15:08 2020 +0300

    Less tests

commit 933999b4ae
Author: alesapin <alesapin@gmail.com>
Date:   Sun Oct 4 11:37:11 2020 +0300

    less tests

commit 7ccf1cc316
Author: alesapin <alesapin@gmail.com>
Date:   Sun Oct 4 11:36:34 2020 +0300

    Less tests

commit 744eb8a2e4
Author: alesapin <alesapin@gmail.com>
Date:   Sat Oct 3 16:10:10 2020 +0300

    Trying 6

commit 7cef000b72
Author: alesapin <alesapin@gmail.com>
Date:   Sat Oct 3 10:53:02 2020 +0300

    More tests to skip

commit 182f510565
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 2 21:58:11 2020 +0300

    Try 8 threads

commit d8f7dce286
Merge: 626e9a82cf 77c9f7af79
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 2 21:45:43 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit 5f6bbe8967
Merge: 32776da69a 9cb3c743bd
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Fri Oct 2 20:14:08 2020 +0300

    Merge branch 'master' into allow_atomic_database_inside_materialize_mysql

commit 32776da69a
Merge: aa5e5013f2 404c199448
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Fri Oct 2 20:09:37 2020 +0300

    Merge branch 'master' into allow_atomic_database_inside_materialize_mysql

commit 626e9a82cf
Merge: 91ba9c2e5f 7d75866722
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 2 17:34:20 2020 +0300

    Merge branch 'fix_ilike' into run_func_tests_in_parallel

commit 91ba9c2e5f
Merge: 76e37a6406 c0d1416bbd
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 2 10:44:34 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit 76e37a6406
Author: alesapin <alesapin@gmail.com>
Date:   Fri Oct 2 10:43:21 2020 +0300

    More tests to skip

commit aa5e5013f2
Merge: 1c37ac7c9d bbbe51033d
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Sun Sep 27 14:11:53 2020 +0300

    Merge branch 'master' into allow_atomic_database_inside_materialize_mysql

commit 2452b40093
Author: alesapin <alesapin@gmail.com>
Date:   Fri Sep 25 22:44:13 2020 +0300

    Skip more tests

commit 7c1c548b43
Author: alesapin <alesapin@gmail.com>
Date:   Fri Sep 25 22:43:35 2020 +0300

    Disable more tests

commit 4ee1621ebf
Author: alesapin <alesapin@gmail.com>
Date:   Fri Sep 25 19:55:05 2020 +0300

    Disable all tests with attach

commit 6669b7da29
Author: alesapin <alesapin@gmail.com>
Date:   Fri Sep 25 19:41:09 2020 +0300

    More parallel tests

commit bedab199af
Author: alesapin <alesapin@gmail.com>
Date:   Fri Sep 25 16:20:16 2020 +0300

    Fix category name

commit 66be1580fd
Author: alesapin <alesapin@gmail.com>
Date:   Fri Sep 25 16:18:08 2020 +0300

    Correctly merge with master

commit 987a46faee
Merge: 8e9013561d 8f21c73b76
Author: alesapin <alesapin@gmail.com>
Date:   Fri Sep 25 16:16:42 2020 +0300

    Merge branch 'master' into run_func_tests_in_parallel

commit 8e9013561d
Author: alesapin <alesapin@gmail.com>
Date:   Thu Sep 24 17:54:10 2020 +0300

    Trying to split tests in sequential and parallel

commit 1c37ac7c9d
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Thu Sep 17 21:19:02 2020 +0300

    fix

commit 41e99cf261
Merge: 18e48838bb 9b6c62e82b
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Thu Sep 17 20:28:50 2020 +0300

    Merge branch 'master' into allow_atomic_database_inside_materialize_mysql

commit 18e48838bb
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Tue Sep 15 17:22:20 2020 +0300

    fix build

commit 5455068226
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Tue Sep 15 17:17:30 2020 +0300

    try fix flacky test

commit 0ecf8e595f
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Tue Sep 15 16:30:30 2020 +0300

    cleanup code, add test

commit 3671c91c37
Merge: a812f7eb75 1d8ee50769
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Tue Sep 15 12:43:01 2020 +0300

    Merge branch 'master' into allow_atomic_database_inside_materialize_mysql

commit a812f7eb75
Author: Alexander Tokmakov <avtokmakov@yandex-team.ru>
Date:   Mon Sep 14 22:25:02 2020 +0300

    allow using Atomic database inside MaterializeMySQL
2020-12-18 18:52:33 +03:00

643 KiB

ClickHouse release 20.12

ClickHouse release v20.12.3.3-stable, 2020-12-13

Backward Incompatible Change

  • Enable use_compact_format_in_distributed_parts_names by default (see the documentation for the reference). #16728 (Azat Khuzhin).
  • Accept user settings related to file formats (e.g. format_csv_delimiter) in the SETTINGS clause when creating a table that uses File engine, and use these settings in all INSERTs and SELECTs. The file format settings changed in the current user session, or in the SETTINGS clause of a DML query itself, no longer affect the query. #16591 (Alexander Kuzmenkov).

New Feature

  • add *.xz compression/decompression support.It enables using *.xz in file() function. This closes #8828. #16578 (Abi Palagashvili).
  • Introduce the query ALTER TABLE ... DROP|DETACH PART 'part_name'. #15511 (nvartolomei).
  • Added new ALTER UPDATE/DELETE IN PARTITION syntax. #13403 (Vladimir Chebotarev).
  • Allow formatting named tuples as JSON objects when using JSON input/output formats, controlled by the output_format_json_named_tuples_as_objects setting, disabled by default. #17175 (Alexander Kuzmenkov).
  • Add a possibility to input enum value as it's id in TSV and CSV formats by default. #16834 (Kruglov Pavel).
  • Add COLLATE support for Nullable, LowCardinality, Array and Tuple, where nested type is String. Also refactor the code associated with collations in ColumnString.cpp. #16273 (Kruglov Pavel).
  • New tcpPort function returns TCP port listened by this server. #17134 (Ivan).
  • Add new math functions: acosh, asinh, atan2, atanh, cosh, hypot, log1p, sinh. #16636 (Konstantin Malanchev).
  • Possibility to distribute the merges between different replicas. Introduces the execute_merges_on_single_replica_time_threshold mergetree setting. #16424 (filimonov).
  • Add setting aggregate_functions_null_for_empty for SQL standard compatibility. This option will rewrite all aggregate functions in a query, adding -OrNull suffix to them. Implements 10273. #16123 (flynn).
  • Updated DateTime, DateTime64 parsing to accept string Date literal format. #16040 (Maksim Kita).
  • Make it possible to change the path to history file in clickhouse-client using the --history_file parameter. #15960 (Maksim Kita).

Bug Fix

  • Fix the issue when server can stop accepting connections in very rare cases. #17542 (Amos Bird).
  • Fixed Function not implemented error when executing RENAME query in Atomic database with ClickHouse running on Windows Subsystem for Linux. Fixes #17661. #17664 (tavplubix).
  • Do not restore parts from WAL if in_memory_parts_enable_wal is disabled. #17802 (detailyang).
  • fix incorrect initialization of max_compress_block_size of MergeTreeWriterSettings with min_compress_block_size. #17833 (flynn).
  • Exception message about max table size to drop was displayed incorrectly. #17764 (alexey-milovidov).
  • Fixed possible segfault when there is not enough space when inserting into Distributed table. #17737 (tavplubix).
  • Fixed problem when ClickHouse fails to resume connection to MySQL servers. #17681 (Alexander Kazakov).
  • In might be determined incorrectly if cluster is circular- (cross-) replicated or not when executing ON CLUSTER query due to race condition when pool_size > 1. It's fixed. #17640 (tavplubix).
  • Exception fmt::v7::format_error can be logged in background for MergeTree tables. This fixes #17613. #17615 (alexey-milovidov).
  • When clickhouse-client is used in interactive mode with multiline queries, single line comment was erronously extended till the end of query. This fixes #13654. #17565 (alexey-milovidov).
  • Fix alter query hang when the corresponding mutation was killed on the different replica. Fixes #16953. #17499 (alesapin).
  • Fix issue when mark cache size was underestimated by clickhouse. It may happen when there are a lot of tiny files with marks. #17496 (alesapin).
  • Fix ORDER BY with enabled setting optimize_redundant_functions_in_order_by. #17471 (Anton Popov).
  • Fix duplicates after DISTINCT which were possible because of incorrect optimization. Fixes #17294. #17296 (li chengxiang). #17439 (Nikolai Kochetov).
  • Fix crash while reading from JOIN table with LowCardinality types. Fixes #17228. #17397 (Nikolai Kochetov).
  • fix toInt256(inf) stack overflow. Int256 is an experimental feature. Closed #17235. #17257 (flynn).
  • Fix possible Unexpected packet Data received from client error logged for Distributed queries with LIMIT. #17254 (Azat Khuzhin).
  • Fix set index invalidation when there are const columns in the subquery. This fixes #17246. #17249 (Amos Bird).
  • Fix possible wrong index analysis when the types of the index comparison are different. This fixes #17122. #17145 (Amos Bird).
  • Fix ColumnConst comparison which leads to crash. This fixed #17088 . #17135 (Amos Bird).
  • Multiple fixed for MaterializeMySQL (experimental feature). Fixes #16923 Fixes #15883 Fix MaterializeMySQL SYNC failure when the modify MySQL binlog_checksum. #17091 (Winter Zhang).
  • Fix bug when ON CLUSTER queries may hang forever for non-leader ReplicatedMergeTreeTables. #17089 (alesapin).
  • Fixed crash on CREATE TABLE ... AS some_table query when some_table was created AS table_function() Fixes #16944. #17072 (tavplubix).
  • Bug unfinished implementation for funciton fuzzBits, related issue: #16980. #17051 (hexiaoting).
  • Fix LLVM's libunwind in the case when CFA register is RAX. This is the bug in LLVM's libunwind. We already have workarounds for this bug. #17046 (alexey-milovidov).
  • Avoid unnecessary network errors for remote queries which may be cancelled while execution, like queries with LIMIT. #17006 (Azat Khuzhin).
  • Fix optimize_distributed_group_by_sharding_key setting (that is disabled by default) for query with OFFSET only. #16996 (Azat Khuzhin).
  • Fix for Merge tables over Distributed tables with JOIN. #16993 (Azat Khuzhin).
  • Fixed wrong result in big integers (128, 256 bit) when casting from double. Big integers support is experimental. #16986 (Mike).
  • Fix possible server crash after ALTER TABLE ... MODIFY COLUMN ... NewType when SELECT have WHERE expression on altering column and alter doesn't finished yet. #16968 (Amos Bird).
  • Blame info was not calculated correctly in clickhouse-git-import. #16959 (alexey-milovidov).
  • Fix order by optimization with monotonous functions. Fixes #16107. #16956 (Anton Popov).
  • Fix optimization of group by with enabled setting optimize_aggregators_of_group_by_keys and joins. Fixes #12604. #16951 (Anton Popov).
  • Fix possible error Illegal type of argument for queries with ORDER BY. Fixes #16580. #16928 (Nikolai Kochetov).
  • Fix strange code in InterpreterShowAccessQuery. #16866 (tavplubix).
  • Prevent clickhouse server crashes when using the function timeSeriesGroupSum. The function is removed from newer ClickHouse releases. #16865 (filimonov).
  • Fix rare silent crashes when query profiler is on and ClickHouse is installed on OS with glibc version that has (supposedly) broken asynchronous unwind tables for some functions. This fixes #15301. This fixes #13098. #16846 (alexey-milovidov).
  • Fix crash when using any without any arguments. This is for #16803 . cc @azat. #16826 (Amos Bird).
  • If no memory can be allocated while writing table metadata on disk, broken metadata file can be written. #16772 (alexey-milovidov).
  • Fix trivial query optimization with partition predicate. #16767 (Azat Khuzhin).
  • Fix IN operator over several columns and tuples with enabled transform_null_in setting. Fixes #15310. #16722 (Anton Popov).
  • Return number of affected rows for INSERT queries via MySQL protocol. Previously ClickHouse used to always return 0, it's fixed. Fixes #16605. #16715 (Winter Zhang).
  • Fix remote query failure when using 'if' suffix aggregate function. Fixes #16574 Fixes #16231 #16610 (Winter Zhang).
  • Fix inconsistent behavior caused by select_sequential_consistency for optimized trivial count query and system.tables. #16309 (Hao Chen).

Improvement

  • Remove empty parts after they were pruned by TTL, mutation, or collapsing merge algorithm. #16895 (Anton Popov).
  • Enable compact format of directories for asynchronous sends in Distributed tables: use_compact_format_in_distributed_parts_names is set to 1 by default. #16788 (Azat Khuzhin).
  • Abort multipart upload if no data was written to S3. #16840 (Pavel Kovalenko).
  • Reresolve the IP of the format_avro_schema_registry_url in case of errors. #16985 (filimonov).
  • Mask password in data_path in the system.distribution_queue. #16727 (Azat Khuzhin).
  • Throw error when use column transformer replaces non existing column. #16183 (hexiaoting).
  • Turn off parallel parsing when there is no enough memory for all threads to work simultaneously. Also there could be exceptions like "Memory limit exceeded" when somebody will try to insert extremely huge rows (> min_chunk_bytes_for_parallel_parsing), because each piece to parse has to be independent set of strings (one or more). #16721 (Nikita Mikhaylov).
  • Install script should always create subdirs in config folders. This is only relevant for Docker build with custom config. #16936 (filimonov).
  • Correct grammar in error message in JSONEachRow, JSONCompactEachRow, and RegexpRow input formats. #17205 (nico piderman).
  • Set default host and port parameters for SOURCE(CLICKHOUSE(...)) to current instance and set default user value to 'default'. #16997 (vdimir).
  • Throw an informative error message when doing ATTACH/DETACH TABLE <DICTIONARY>. Before this PR, detach table <dict> works but leads to an ill-formed in-memory metadata. #16885 (Amos Bird).
  • Add cutToFirstSignificantSubdomainWithWWW(). #16845 (Azat Khuzhin).
  • Server refused to startup with exception message if wrong config is given (metric_log.collect_interval_milliseconds is missing). #16815 (Ivan).
  • Better exception message when configuration for distributed DDL is absent. This fixes #5075. #16769 (Nikita Mikhaylov).
  • Usability improvement: better suggestions in syntax error message when CODEC expression is misplaced in CREATE TABLE query. This fixes #12493. #16768 (alexey-milovidov).
  • Remove empty directories for async INSERT at start of Distributed engine. #16729 (Azat Khuzhin).
  • Workaround for use S3 with nginx server as proxy. Nginx currenty does not accept urls with empty path like http://domain.com?delete, but vanilla aws-sdk-cpp produces this kind of urls. This commit uses patched aws-sdk-cpp version, which makes urls with "/" as path in this cases, like http://domain.com/?delete. #16709 (ianton-ru).
  • Allow reinterpretAs* functions to work for integers and floats of the same size. Implements 16640. #16657 (flynn).
  • Now, <auxiliary_zookeepers> configuration can be changed in config.xml and reloaded without server startup. #16627 (Amos Bird).
  • Support SNI in https connections to remote resources. This will allow to connect to Cloudflare servers that require SNI. This fixes #10055. #16252 (alexey-milovidov).
  • Make it possible to connect to clickhouse-server secure endpoint which requires SNI. This is possible when clickhouse-server is hosted behind TLS proxy. #16938 (filimonov).
  • Fix possible stack overflow if a loop of materialized views is created. This closes #15732. #16048 (alexey-milovidov).
  • Simplify the implementation of background tasks processing for the MergeTree table engines family. There should be no visible changes for user. #15983 (alesapin).
  • Improvement for MaterializeMySQL (experimental feature). Throw exception about right sync privileges when MySQL sync user has error privileges. #15977 (TCeason).
  • Made indexOf() use BloomFilter. #14977 (achimbab).

Performance Improvement

  • Use Floyd-Rivest algorithm, it is the best for the ClickHouse use case of partial sorting. Bechmarks are in https://github.com/danlark1/miniselect and here. #16825 (Danila Kutenin).
  • Now ReplicatedMergeTree tree engines family uses a separate thread pool for replicated fetches. Size of the pool limited by setting background_fetches_pool_size which can be tuned with a server restart. The default value of the setting is 3 and it means that the maximum amount of parallel fetches is equal to 3 (and it allows to utilize 10G network). Fixes #520. #16390 (alesapin).
  • Fixed uncontrolled growth of the state of quantileTDigest. #16680 (hrissan).
  • Add VIEW subquery description to EXPLAIN. Limit push down optimisation for VIEW. Add local replicas of Distributed to query plan. #14936 (Nikolai Kochetov).
  • Fix optimize_read_in_order/optimize_aggregation_in_order with max_threads > 0 and expression in ORDER BY. #16637 (Azat Khuzhin).
  • Fix performance of reading from Merge tables over huge number of MergeTree tables. Fixes #7748. #16988 (Anton Popov).
  • Now we can safely prune partitions with exact match. Useful case: Suppose table is partitioned by intHash64(x) % 100 and the query has condition on intHash64(x) % 100 verbatim, not on x. #16253 (Amos Bird).

Experimental Feature

  • Add EmbeddedRocksDB table engine (can be used for dictionaries). #15073 (sundyli).

Build/Testing/Packaging Improvement

ClickHouse release 20.11

ClickHouse release v20.11.3.3-stable, 2020-11-13

Bug Fix

  • Fix rare silent crashes when query profiler is on and ClickHouse is installed on OS with glibc version that has (supposedly) broken asynchronous unwind tables for some functions. This fixes #15301. This fixes #13098. #16846 (alexey-milovidov).

ClickHouse release v20.11.2.1, 2020-11-11

Backward Incompatible Change

  • If some profile was specified in distributed_ddl config section, then this profile could overwrite settings of default profile on server startup. It's fixed, now settings of distributed DDL queries should not affect global server settings. #16635 (tavplubix).
  • Restrict to use of non-comparable data types (like AggregateFunction) in keys (Sorting key, Primary key, Partition key, and so on). #16601 (alesapin).
  • Remove ANALYZE and AST queries, and make the setting enable_debug_queries obsolete since now it is the part of full featured EXPLAIN query. #16536 (Ivan).
  • Aggregate functions boundingRatio, rankCorr, retention, timeSeriesGroupSum, timeSeriesGroupRateSum, windowFunnel were erroneously made case-insensitive. Now their names are made case sensitive as designed. Only functions that are specified in SQL standard or made for compatibility with other DBMS or functions similar to those should be case-insensitive. #16407 (alexey-milovidov).
  • Make rankCorr function return nan on insufficient data #16124. #16135 (hexiaoting).
  • When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

New Feature

  • Added support of LDAP as a user directory for locally non-existent users. #12736 (Denis Glazachev).
  • Add system.replicated_fetches table which shows currently running background fetches. #16428 (alesapin).
  • Added setting date_time_output_format. #15845 (Maksim Kita).
  • Added minimal web UI to ClickHouse. #16158 (alexey-milovidov).
  • Allows to read/write Single protobuf message at once (w/o length-delimiters). #15199 (filimonov).
  • Added initial OpenTelemetry support. ClickHouse now accepts OpenTelemetry traceparent headers over Native and HTTP protocols, and passes them downstream in some cases. The trace spans for executed queries are saved into the system.opentelemetry_span_log table. #14195 (Alexander Kuzmenkov).
  • Allow specify primary key in column list of CREATE TABLE query. This is needed for compatibility with other SQL dialects. #15823 (Maksim Kita).
  • Implement OFFSET offset_row_count {ROW | ROWS} FETCH {FIRST | NEXT} fetch_row_count {ROW | ROWS} {ONLY | WITH TIES} in SELECT query with ORDER BY. This is the SQL-standard way to specify LIMIT. #15855 (hexiaoting).
  • errorCodeToName function - return variable name of the error (useful for analyzing query_log and similar). system.errors table - shows how many times errors has been happened (respects system_events_show_zero_values). #16438 (Azat Khuzhin).
  • Added function untuple which is a special function which can introduce new columns to the SELECT list by expanding a named tuple. #16242 (Nikolai Kochetov, Amos Bird).
  • Now we can provide identifiers via query parameters. And these parameters can be used as table objects or columns. #16594 (Amos Bird).
  • Added big integers (UInt256, Int128, Int256) and UUID data types support for MergeTree BloomFilter index. Big integers is an experimental feature. #16642 (Maksim Kita).
  • Add farmFingerprint64 function (non-cryptographic string hashing). #16570 (Jacob Hayes).
  • Add log_queries_min_query_duration_ms, only queries slower than the value of this setting will go to query_log/query_thread_log (i.e. something like slow_query_log in mysql). #16529 (Azat Khuzhin).
  • Ability to create a docker image on the top of Alpine. Uses precompiled binary and glibc components from ubuntu 20.04. #16479 (filimonov).
  • Added toUUIDOrNull, toUUIDOrZero cast functions. #16337 (Maksim Kita).
  • Add max_concurrent_queries_for_all_users setting, see #6636 for use cases. #16154 (nvartolomei).
  • Add a new option print_query_id to clickhouse-client. It helps generate arbitrary strings with the current query id generated by the client. Also print query id in clickhouse-client by default. #15809 (Amos Bird).
  • Add tid and logTrace functions. This closes #9434. #15803 (flynn).
  • Add function formatReadableTimeDelta that format time delta to human readable string ... #15497 (Filipe Caixeta).
  • Added disable_merges option for volumes in multi-disk configuration. #13956 (Vladimir Chebotarev).

Experimental Feature

  • New functions encrypt, aes_encrypt_mysql, decrypt, aes_decrypt_mysql. These functions are working slowly, so we consider it as an experimental feature. #11844 (Vasily Nemkov).

Bug Fix

  • Mask password in data_path in the system.distribution_queue. #16727 (Azat Khuzhin).
  • Fix IN operator over several columns and tuples with enabled transform_null_in setting. Fixes #15310. #16722 (Anton Popov).
  • The setting max_parallel_replicas worked incorrectly if the queried table has no sampling. This fixes #5733. #16675 (alexey-milovidov).
  • Fix optimize_read_in_order/optimize_aggregation_in_order with max_threads > 0 and expression in ORDER BY. #16637 (Azat Khuzhin).
  • Calculation of DEFAULT expressions was involving possible name collisions (that was very unlikely to encounter). This fixes #9359. #16612 (alexey-milovidov).
  • Fix query_thread_log.query_duration_ms unit. #16563 (Azat Khuzhin).
  • Fix a bug when using MySQL Master -> MySQL Slave -> ClickHouse MaterializeMySQL Engine. MaterializeMySQL is an experimental feature. #16504 (TCeason).
  • Specifically crafted argument of round function with Decimal was leading to integer division by zero. This fixes #13338. #16451 (alexey-milovidov).
  • Fix DROP TABLE for Distributed (racy with INSERT). #16409 (Azat Khuzhin).
  • Fix processing of very large entries in replication queue. Very large entries may appear in ALTER queries if table structure is extremely large (near 1 MB). This fixes #16307. #16332 (alexey-milovidov).
  • Fixed the inconsistent behaviour when a part of return data could be dropped because the set for its filtration wasn't created. #16308 (Nikita Mikhaylov).
  • Fix dictGet in sharding_key (and similar places, i.e. when the function context is stored permanently). #16205 (Azat Khuzhin).
  • Fix the exception thrown in clickhouse-local when trying to execute OPTIMIZE command. Fixes #16076. #16192 (filimonov).
  • Fixes #15780 regression, e.g. indexOf([1, 2, 3], toLowCardinality(1)) now is prohibited but it should not be. #16038 (Mike).
  • Fix bug with MySQL database. When MySQL server used as database engine is down some queries raise Exception, because they try to get tables from disabled server, while it's unnecessary. For example, query SELECT ... FROM system.parts should work only with MergeTree tables and don't touch MySQL database at all. #16032 (Kruglov Pavel).
  • Now exception will be thrown when ALTER MODIFY COLUMN ... DEFAULT ... has incompatible default with column type. Fixes #15854. #15858 (alesapin).
  • Fixed IPv4CIDRToRange/IPv6CIDRToRange functions to accept const IP-column values. #15856 (vladimir-golovchenko).

Improvement

  • Treat INTERVAL '1 hour' as equivalent to INTERVAL 1 HOUR, to be compatible with Postgres and similar. This fixes #15637. #15978 (flynn).
  • Enable parsing enum values by their numeric ids for CSV, TSV and JSON input formats. #15685 (vivarum).
  • Better read task scheduling for JBOD architecture and MergeTree storage. New setting read_backoff_min_concurrency which serves as the lower limit to the number of reading threads. #16423 (Amos Bird).
  • Add missing support for LowCardinality in Avro format. #16521 (Mike).
  • Workaround for use S3 with nginx server as proxy. Nginx currenty does not accept urls with empty path like http://domain.com?delete, but vanilla aws-sdk-cpp produces this kind of urls. This commit uses patched aws-sdk-cpp version, which makes urls with "/" as path in this cases, like http://domain.com/?delete. #16814 (ianton-ru).
  • Better diagnostics on parse errors in input data. Provide row number on Cannot read all data errors. #16644 (alexey-milovidov).
  • Make the behaviour of minMap and maxMap more desireable. It will not skip zero values in the result. Fixes #16087. #16631 (Ildus Kurbangaliev).
  • Better update of ZooKeeper configuration in runtime. #16630 (sundyli).
  • Apply SETTINGS clause as early as possible. It allows to modify more settings in the query. This closes #3178. #16619 (alexey-milovidov).
  • Now event_time_microseconds field stores in Decimal64, not UInt64. #16617 (Nikita Mikhaylov).
  • Now paratmeterized functions can be used in APPLY column transformer. #16589 (Amos Bird).
  • Improve scheduling of background task which removes data of dropped tables in Atomic databases. Atomic databases do not create broken symlink to table data directory if table actually has no data directory. #16584 (tavplubix).
  • Subqueries in WITH section (CTE) can reference previous subqueries in WITH section by their name. #16575 (Amos Bird).
  • Add current_database into system.query_thread_log. #16558 (Azat Khuzhin).
  • Allow to fetch parts that are already committed or outdated in the current instance into the detached directory. It's useful when migrating tables from another cluster and having N to 1 shards mapping. It's also consistent with the current fetchPartition implementation. #16538 (Amos Bird).
  • Multiple improvements for RabbitMQ: Fixed bug for #16263. Also minimized event loop lifetime. Added more efficient queues setup. #16426 (Kseniia Sumarokova).
  • Fix debug assertion in quantileDeterministic function. In previous version it may also transfer up to two times more data over the network. Although no bug existed. This fixes #15683. #16410 (alexey-milovidov).
  • Add TablesToDropQueueSize metric. It's equal to number of dropped tables, that are waiting for background data removal. #16364 (tavplubix).
  • Better diagnostics when client has dropped connection. In previous versions, Attempt to read after EOF and Broken pipe exceptions were logged in server. In new version, it's information message Client has dropped the connection, cancel the query.. #16329 (alexey-milovidov).
  • Add total_rows/total_bytes (from system.tables) support for Set/Join table engines. #16306 (Azat Khuzhin).
  • Now it's possible to specify PRIMARY KEY without ORDER BY for MergeTree table engines family. Closes #15591. #16284 (alesapin).
  • If there is no tmp folder in the system (chroot, misconfigutation etc) clickhouse-local will create temporary subfolder in the current directory. #16280 (filimonov).
  • Add support for nested data types (like named tuple) as sub-types. Fixes #15587. #16262 (Ivan).
  • Support for database_atomic_wait_for_drop_and_detach_synchronously/NO DELAY/SYNC for DROP DATABASE. #16127 (Azat Khuzhin).
  • Add allow_nondeterministic_optimize_skip_unused_shards (to allow non deterministic like rand() or dictGet() in sharding key). #16105 (Azat Khuzhin).
  • Fix memory_profiler_step/max_untracked_memory for queries via HTTP (test included). Fix the issue that adjusting this value globally in xml config does not help either, since those settings are not applied anyway, only default (4MB) value is used. Fix query_id for the most root ThreadStatus of the http query (by initializing QueryScope after reading query_id). #16101 (Azat Khuzhin).
  • Now it's allowed to execute ALTER ... ON CLUSTER queries regardless of the <internal_replication> setting in cluster config. #16075 (alesapin).
  • Fix rare issue when clickhouse-client may abort on exit due to loading of suggestions. This fixes #16035. #16047 (alexey-milovidov).
  • Add support of cache layout for Redis dictionaries with complex key. #15985 (Anton Popov).
  • Fix query hang (endless loop) in case of misconfiguration (connections_with_failover_max_tries set to 0). #15876 (Azat Khuzhin).
  • Change level of some log messages from information to debug, so information messages will not appear for every query. This closes #5293. #15816 (alexey-milovidov).
  • Remove MemoryTrackingInBackground* metrics to avoid potentially misleading results. This fixes #15684. #15813 (alexey-milovidov).
  • Add reconnects to zookeeper-dump-tree tool. #15711 (alexey-milovidov).
  • Allow explicitly specify columns list in CREATE TABLE table AS table_function(...) query. Fixes #9249 Fixes #14214. #14295 (tavplubix).

Performance Improvement

  • Do not merge parts across partitions in SELECT FINAL. #15938 (Kruglov Pavel).
  • Improve performance of -OrNull and -OrDefault aggregate functions. #16661 (alexey-milovidov).
  • Improve performance of quantileMerge. In previous versions it was obnoxiously slow. This closes #1463. #16643 (alexey-milovidov).
  • Improve performance of logical functions a little. #16347 (alexey-milovidov).
  • Improved performance of merges assignment in MergeTree table engines. Shouldn't be visible for the user. #16191 (alesapin).
  • Speedup hashed/sparse_hashed dictionary loading by preallocating the hash table. #15454 (Azat Khuzhin).
  • Now trivial count optimization becomes slightly non-trivial. Predicates that contain exact partition expr can be optimized too. This also fixes #11092 which returns wrong count when max_parallel_replicas > 1. #15074 (Amos Bird).

Build/Testing/Packaging Improvement

ClickHouse release 20.10

ClickHouse release v20.10.4.1-stable, 2020-11-13

Bug Fix

  • Fix rare silent crashes when query profiler is on and ClickHouse is installed on OS with glibc version that has (supposedly) broken asynchronous unwind tables for some functions. This fixes #15301. This fixes #13098. #16846 (alexey-milovidov).
  • Fix IN operator over several columns and tuples with enabled transform_null_in setting. Fixes #15310. #16722 (Anton Popov).
  • This will fix optimize_read_in_order/optimize_aggregation_in_order with max_threads>0 and expression in ORDER BY. #16637 (Azat Khuzhin).
  • Now when parsing AVRO from input the LowCardinality is removed from type. Fixes #16188. #16521 (Mike).
  • Fix rapid growth of metadata when using MySQL Master -> MySQL Slave -> ClickHouse MaterializeMySQL Engine, and slave_parallel_worker enabled on MySQL Slave, by properly shrinking GTID sets. This fixes #15951. #16504 (TCeason).
  • Fix DROP TABLE for Distributed (racy with INSERT). #16409 (Azat Khuzhin).
  • Fix processing of very large entries in replication queue. Very large entries may appear in ALTER queries if table structure is extremely large (near 1 MB). This fixes #16307. #16332 (alexey-milovidov).
  • Fix bug with MySQL database. When MySQL server used as database engine is down some queries raise Exception, because they try to get tables from disabled server, while it's unnecessary. For example, query SELECT ... FROM system.parts should work only with MergeTree tables and don't touch MySQL database at all. #16032 (Kruglov Pavel).

Improvement

  • Workaround for use S3 with nginx server as proxy. Nginx currenty does not accept urls with empty path like http://domain.com?delete, but vanilla aws-sdk-cpp produces this kind of urls. This commit uses patched aws-sdk-cpp version, which makes urls with "/" as path in this cases, like http://domain.com/?delete. #16813 (ianton-ru).

ClickHouse release v20.10.3.30, 2020-10-28

Backward Incompatible Change

  • Make multiple_joins_rewriter_version obsolete. Remove first version of joins rewriter. #15472 (Artem Zuikov).
  • Change default value of format_regexp_escaping_rule setting (it's related to Regexp format) to Raw (it means - read whole subpattern as a value) to make the behaviour more like to what users expect. #15426 (alexey-milovidov).
  • Add support for nested multiline comments /* comment /* comment */ */ in SQL. This conforms to the SQL standard. #14655 (alexey-milovidov).
  • Added MergeTree settings (max_replicated_merges_with_ttl_in_queue and max_number_of_merges_with_ttl_in_pool) to control the number of merges with TTL in the background pool and replicated queue. This change breaks compatibility with older versions only if you use delete TTL. Otherwise, replication will stay compatible. You can avoid incompatibility issues if you update all shard replicas at once or execute SYSTEM STOP TTL MERGES until you finish the update of all replicas. If you'll get an incompatible entry in the replication queue, first of all, execute SYSTEM STOP TTL MERGES and after ALTER TABLE ... DETACH PARTITION ... the partition where incompatible TTL merge was assigned. Attach it back on a single replica. #14490 (alesapin).
  • When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

New Feature

  • Background data recompression. Add the ability to specify TTL ... RECOMPRESS codec_name for MergeTree table engines family. #14494 (alesapin).
  • Add parallel quorum inserts. This closes #15601. #15601 (Latysheva Alexandra).
  • Settings for additional enforcement of data durability. Useful for non-replicated setups. #11948 (Anton Popov).
  • When duplicate block is written to replica where it does not exist locally (has not been fetched from replicas), don't ignore it and write locally to achieve the same effect as if it was successfully replicated. #11684 (alexey-milovidov).
  • Now we support WITH <identifier> AS (subquery) ... to introduce named subqueries in the query context. This closes #2416. This closes #4967. #14771 (Amos Bird).
  • Introduce enable_global_with_statement setting which propagates the first select's WITH statements to other select queries at the same level, and makes aliases in WITH statements visible to subqueries. #15451 (Amos Bird).
  • Secure inter-cluster query execution (with initial_user as current query user). #13156 (Azat Khuzhin). #15551 (Azat Khuzhin).
  • Add the ability to remove column properties and table TTLs. Introduced queries ALTER TABLE MODIFY COLUMN col_name REMOVE what_to_remove and ALTER TABLE REMOVE TTL. Both operations are lightweight and executed at the metadata level. #14742 (alesapin).
  • Added format RawBLOB. It is intended for input or output a single value without any escaping and delimiters. This closes #15349. #15364 (alexey-milovidov).
  • Add the reinterpretAsUUID function that allows to convert a big-endian byte string to UUID. #15480 (Alexander Kuzmenkov).
  • Implement force_data_skipping_indices setting. #15642 (Azat Khuzhin).
  • Add a setting output_format_pretty_row_numbers to numerate the result in Pretty formats. This closes #15350. #15443 (flynn).
  • Added query obfuscation tool. It allows to share more queries for better testing. This closes #15268. #15321 (alexey-milovidov).
  • Add table function null('structure'). #14797 (vxider).
  • Added formatReadableQuantity function. It is useful for reading big numbers by human. #14725 (Artem Hnilov).
  • Add format LineAsString that accepts a sequence of lines separated by newlines, every line is parsed as a whole as a single String field. #14703 (Nikita Mikhaylov), #13846 (hexiaoting).
  • Add JSONStrings format which output data in arrays of strings. #14333 (hcz).
  • Add support for "Raw" column format for Regexp format. It allows to simply extract subpatterns as a whole without any escaping rules. #15363 (alexey-milovidov).
  • Allow configurable NULL representation for TSV output format. It is controlled by the setting output_format_tsv_null_representation which is \N by default. This closes #9375. Note that the setting only controls output format and \N is the only supported NULL representation for TSV input format. #14586 (Kruglov Pavel).
  • Support Decimal data type for MaterializeMySQL. MaterializeMySQL is an experimental feature. #14535 (Winter Zhang).
  • Add new feature: SHOW DATABASES LIKE 'xxx'. #14521 (hexiaoting).
  • Added a script to import (arbitrary) git repository to ClickHouse as a sample dataset. #14471 (alexey-milovidov).
  • Now insert statements can have asterisk (or variants) with column transformers in the column list. #14453 (Amos Bird).
  • New query complexity limit settings max_rows_to_read_leaf, max_bytes_to_read_leaf for distributed queries to limit max rows/bytes read on the leaf nodes. Limit is applied for local reads only, excluding the final merge stage on the root node. #14221 (Roman Khavronenko).
  • Allow user to specify settings for ReplicatedMergeTree* storage in <replicated_merge_tree> section of config file. It works similarly to <merge_tree> section. For ReplicatedMergeTree* storages settings from <merge_tree> and <replicated_merge_tree> are applied together, but settings from <replicated_merge_tree> has higher priority. Added system.replicated_merge_tree_settings table. #13573 (Amos Bird).
  • Add mapPopulateSeries function. #13166 (Ildus Kurbangaliev).
  • Supporting MySQL types: decimal (as ClickHouse Decimal) and datetime with sub-second precision (as DateTime64). #11512 (Vasily Nemkov).
  • Introduce event_time_microseconds field to system.text_log, system.trace_log, system.query_log and system.query_thread_log tables. #14760 (Bharat Nallan).
  • Add event_time_microseconds to system.asynchronous_metric_log & system.metric_log tables. #14514 (Bharat Nallan).
  • Add query_start_time_microseconds field to system.query_log & system.query_thread_log tables. #14252 (Bharat Nallan).

Bug Fix

  • Fix the case when memory can be overallocated regardless to the limit. This closes #14560. #16206 (alexey-milovidov).
  • Fix executable dictionary source hang. In previous versions, when using some formats (e.g. JSONEachRow) data was not feed to a child process before it outputs at least something. This closes #1697. This closes #2455. #14525 (alexey-milovidov).
  • Fix double free in case of exception in function dictGet. It could have happened if dictionary was loaded with error. #16429 (Nikolai Kochetov).
  • Fix group by with totals/rollup/cube modifers and min/max functions over group by keys. Fixes #16393. #16397 (Anton Popov).
  • Fix async Distributed INSERT with prefer_localhost_replica=0 and internal_replication. #16358 (Azat Khuzhin).
  • Fix a very wrong code in TwoLevelStringHashTable implementation, which might lead to memory leak. #16264 (Amos Bird).
  • Fix segfault in some cases of wrong aggregation in lambdas. #16082 (Anton Popov).
  • Fix ALTER MODIFY ... ORDER BY query hang for ReplicatedVersionedCollapsingMergeTree. This fixes #15980. #16011 (alesapin).
  • MaterializeMySQL (experimental feature): Fix collate name & charset name parser and support length = 0 for string type. #16008 (Winter Zhang).
  • Allow to use direct layout for dictionaries with complex keys. #16007 (Anton Popov).
  • Prevent replica hang for 5-10 mins when replication error happens after a period of inactivity. #15987 (filimonov).
  • Fix rare segfaults when inserting into or selecting from MaterializedView and concurrently dropping target table (for Atomic database engine). #15984 (tavplubix).
  • Fix ambiguity in parsing of settings profiles: CREATE USER ... SETTINGS profile readonly is now considered as using a profile named readonly, not a setting named profile with the readonly constraint. This fixes #15628. #15982 (Vitaly Baranov).
  • MaterializeMySQL (experimental feature): Fix crash on create database failure. #15954 (Winter Zhang).
  • Fixed DROP TABLE IF EXISTS failure with Table ... doesn't exist error when table is concurrently renamed (for Atomic database engine). Fixed rare deadlock when concurrently executing some DDL queries with multiple tables (like DROP DATABASE and RENAME TABLE) - Fixed DROP/DETACH DATABASE failure with Table ... doesn't exist when concurrently executing DROP/DETACH TABLE. #15934 (tavplubix).
  • Fix incorrect empty result for query from Distributed table if query has WHERE, PREWHERE and GLOBAL IN. Fixes #15792. #15933 (Nikolai Kochetov).
  • Fixes #12513: difference expressions with same alias when query is reanalyzed. #15886 (Winter Zhang).
  • Fix possible very rare deadlocks in RBAC implementation. #15875 (Vitaly Baranov).
  • Fix exception Block structure mismatch in SELECT ... ORDER BY DESC queries which were executed after ALTER MODIFY COLUMN query. Fixes #15800. #15852 (alesapin).
  • MaterializeMySQL (experimental feature): Fix select count() inaccuracy. #15767 (tavplubix).
  • Fix some cases of queries, in which only virtual columns are selected. Previously Not found column _nothing in block exception may be thrown. Fixes #12298. #15756 (Anton Popov).
  • Fix drop of materialized view with inner table in Atomic database (hangs all subsequent DROP TABLE due to hang of the worker thread, due to recursive DROP TABLE for inner table of MV). #15743 (Azat Khuzhin).
  • Possibility to move part to another disk/volume if the first attempt was failed. #15723 (Pavel Kovalenko).
  • Fix error Cannot find column which may happen at insertion into MATERIALIZED VIEW in case if query for MV containes ARRAY JOIN. #15717 (Nikolai Kochetov).
  • Fixed too low default value of max_replicated_logs_to_keep setting, which might cause replicas to become lost too often. Improve lost replica recovery process by choosing the most up-to-date replica to clone. Also do not remove old parts from lost replica, detach them instead. #15701 (tavplubix).
  • Fix rare race condition in dictionaries and tables from MySQL. #15686 (alesapin).
  • Fix (benign) race condition in AMQP-CPP. #15667 (alesapin).
  • Fix error Cannot add simple transform to empty Pipe which happened while reading from Buffer table which has different structure than destination table. It was possible if destination table returned empty result for query. Fixes #15529. #15662 (Nikolai Kochetov).
  • Proper error handling during insert into MergeTree with S3. MergeTree over S3 is an experimental feature. #15657 (Pavel Kovalenko).
  • Fixed bug with S3 table function: region from URL was not applied to S3 client configuration. #15646 (Vladimir Chebotarev).
  • Fix the order of destruction for resources in ReadFromStorage step of query plan. It might cause crashes in rare cases. Possibly connected with #15610. #15645 (Nikolai Kochetov).
  • Subtract ReadonlyReplica metric when detach readonly tables. #15592 (sundyli).
  • Fixed Element ... is not a constant expression error when using JSON* function result in VALUES, LIMIT or right side of IN operator. #15589 (tavplubix).
  • Query will finish faster in case of exception. Cancel execution on remote replicas if exception happens. #15578 (Azat Khuzhin).
  • Prevent the possibility of error message Could not calculate available disk space (statvfs), errno: 4, strerror: Interrupted system call. This fixes #15541. #15557 (alexey-milovidov).
  • Fix Database <db> doesn't exist. in queries with IN and Distributed table when there's no database on initiator. #15538 (Artem Zuikov).
  • Mutation might hang waiting for some non-existent part after MOVE or REPLACE PARTITION or, in rare cases, after DETACH or DROP PARTITION. It's fixed. #15537 (tavplubix).
  • Fix bug when ILIKE operator stops being case insensitive if LIKE with the same pattern was executed. #15536 (alesapin).
  • Fix Missing columns errors when selecting columns which absent in data, but depend on other columns which also absent in data. Fixes #15530. #15532 (alesapin).
  • Throw an error when a single parameter is passed to ReplicatedMergeTree instead of ignoring it. #15516 (nvartolomei).
  • Fix bug with event subscription in DDLWorker which rarely may lead to query hangs in ON CLUSTER. Introduced in #13450. #15477 (alesapin).
  • Report proper error when the second argument of boundingRatio aggregate function has a wrong type. #15407 (detailyang).
  • Fixes #15365: attach a database with MySQL engine throws exception (no query context). #15384 (Winter Zhang).
  • Fix the case of multiple occurrences of column transformers in a select query. #15378 (Amos Bird).
  • Fixed compression in S3 storage. #15376 (Vladimir Chebotarev).
  • Fix bug where queries like SELECT toStartOfDay(today()) fail complaining about empty time_zone argument. #15319 (Bharat Nallan).
  • Fix race condition during MergeTree table rename and background cleanup. #15304 (alesapin).
  • Fix rare race condition on server startup when system logs are enabled. #15300 (alesapin).
  • Fix hang of queries with a lot of subqueries to same table of MySQL engine. Previously, if there were more than 16 subqueries to same MySQL table in query, it hang forever. #15299 (Anton Popov).
  • Fix MSan report in QueryLog. Uninitialized memory can be used for the field memory_usage. #15258 (alexey-milovidov).
  • Fix 'Unknown identifier' in GROUP BY when query has JOIN over Merge table. #15242 (Artem Zuikov).
  • Fix instance crash when using joinGet with LowCardinality types. This fixes #15214. #15220 (Amos Bird).
  • Fix bug in table engine Buffer which doesn't allow to insert data of new structure into Buffer after ALTER query. Fixes #15117. #15192 (alesapin).
  • Adjust Decimal field size in MySQL column definition packet. #15152 (maqroll).
  • Fixes Data compressed with different methods in join_algorithm='auto'. Keep LowCardinality as type for left table join key in join_algorithm='partial_merge'. #15088 (Artem Zuikov).
  • Update jemalloc to fix percpu_arena with affinity mask. #15035 (Azat Khuzhin). #14957 (Azat Khuzhin).
  • We already use padded comparison between String and FixedString (https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/FunctionsComparison.h#L333). This PR applies the same logic to field comparison which corrects the usage of FixedString as primary keys. This fixes #14908. #15033 (Amos Bird).
  • If function bar was called with specifically crafted arguments, buffer overflow was possible. This closes #13926. #15028 (alexey-milovidov).
  • Fixed Cannot rename ... errno: 22, strerror: Invalid argument error on DDL query execution in Atomic database when running clickhouse-server in Docker on Mac OS. #15024 (tavplubix).
  • Fix crash in RIGHT or FULL JOIN with join_algorith='auto' when memory limit exceeded and we should change HashJoin with MergeJoin. #15002 (Artem Zuikov).
  • Now settings number_of_free_entries_in_pool_to_execute_mutation and number_of_free_entries_in_pool_to_lower_max_size_of_merge can be equal to background_pool_size. #14975 (alesapin).
  • Fix to make predicate push down work when subquery contains finalizeAggregation function. Fixes #14847. #14937 (filimonov).
  • Publish CPU frequencies per logical core in system.asynchronous_metrics. This fixes #14923. #14924 (Alexander Kuzmenkov).
  • MaterializeMySQL (experimental feature): Fixed .metadata.tmp File exists error. #14898 (Winter Zhang).
  • Fix the issue when some invocations of extractAllGroups function may trigger "Memory limit exceeded" error. This fixes #13383. #14889 (alexey-milovidov).
  • Fix SIGSEGV for an attempt to INSERT into StorageFile with file descriptor. #14887 (Azat Khuzhin).
  • Fixed segfault in cache dictionary #14837. #14879 (Nikita Mikhaylov).
  • MaterializeMySQL (experimental feature): Fixed bug in parsing MySQL binlog events, which causes Attempt to read after eof and Packet payload is not fully read in MaterializeMySQL database engine. #14852 (Winter Zhang).
  • Fix rare error in SELECT queries when the queried column has DEFAULT expression which depends on the other column which also has DEFAULT and not present in select query and not exists on disk. Partially fixes #14531. #14845 (alesapin).
  • Fix a problem where the server may get stuck on startup while talking to ZooKeeper, if the configuration files have to be fetched from ZK (using the from_zk include option). This fixes #14814. #14843 (Alexander Kuzmenkov).
  • Fix wrong monotonicity detection for shrunk Int -> Int cast of signed types. It might lead to incorrect query result. This bug is unveiled in #14513. #14783 (Amos Bird).
  • Replace column transformer should replace identifiers with cloned ASTs. This fixes #14695 . #14734 (Amos Bird).
  • Fixed missed default database name in metadata of materialized view when executing ALTER ... MODIFY QUERY. #14664 (tavplubix).
  • Fix bug when ALTER UPDATE mutation with Nullable column in assignment expression and constant value (like UPDATE x = 42) leads to incorrect value in column or segfault. Fixes #13634, #14045. #14646 (alesapin).
  • Fix wrong Decimal multiplication result caused wrong decimal scale of result column. #14603 (Artem Zuikov).
  • Fix function has with LowCardinality of Nullable. #14591 (Mike).
  • Cleanup data directory after Zookeeper exceptions during CreateQuery for StorageReplicatedMergeTree Engine. #14563 (Bharat Nallan).
  • Fix rare segfaults in functions with combinator -Resample, which could appear in result of overflow with very large parameters. #14562 (Anton Popov).
  • Fix a bug when converting Nullable(String) to Enum. Introduced by #12745. This fixes #14435. #14530 (Amos Bird).
  • Fixed the incorrect sorting order of Nullable column. This fixes #14344. #14495 (Nikita Mikhaylov).
  • Fix currentDatabase() function cannot be used in ON CLUSTER ddl query. #14211 (Winter Zhang).
  • MaterializeMySQL (experimental feature): Fixed Packet payload is not fully read error in MaterializeMySQL database engine. #14696 (BohuTANG).

Improvement

  • Enable Atomic database engine by default for newly created databases. #15003 (tavplubix).
  • Add the ability to specify specialized codecs like Delta, T64, etc. for columns with subtypes. Implements #12551, fixes #11397, fixes #4609. #15089 (alesapin).
  • Dynamic reload of zookeeper config. #14678 (sundyli).
  • Now it's allowed to execute ALTER ... ON CLUSTER queries regardless of the <internal_replication> setting in cluster config. #16075 (alesapin).
  • Now joinGet supports multi-key lookup. Continuation of #12418. #13015 (Amos Bird).
  • Wait for DROP/DETACH TABLE to actually finish if NO DELAY or SYNC is specified for Atomic database. #15448 (tavplubix).
  • Now it's possible to change the type of version column for VersionedCollapsingMergeTree with ALTER query. #15442 (alesapin).
  • Unfold {database}, {table} and {uuid} macros in zookeeper_path on replicated table creation. Do not allow RENAME TABLE if it may break zookeeper_path after server restart. Fixes #6917. #15348 (tavplubix).
  • The function now allows an argument with timezone. This closes 15264. #15285 (flynn).
  • Do not allow connections to ClickHouse server until all scripts in /docker-entrypoint-initdb.d/ are executed. #15244 (Aleksei Kozharin).
  • Added optimize setting to EXPLAIN PLAN query. If enabled, query plan level optimisations are applied. Enabled by default. #15201 (Nikolai Kochetov).
  • Proper exception message for wrong number of arguments of CAST. This closes #13992. #15029 (alexey-milovidov).
  • Add option to disable TTL move on data part insert. #15000 (Pavel Kovalenko).
  • Ignore key constraints when doing mutations. Without this pull request, it's not possible to do mutations when force_index_by_date = 1 or force_primary_key = 1. #14973 (Amos Bird).
  • Allow to drop Replicated table if previous drop attempt was failed due to ZooKeeper session expiration. This fixes #11891. #14926 (alexey-milovidov).
  • Fixed excessive settings constraint violation when running SELECT with SETTINGS from a distributed table. #14876 (Amos Bird).
  • Provide a load_balancing_first_offset query setting to explicitly state what the first replica is. It's used together with FIRST_OR_RANDOM load balancing strategy, which allows to control replicas workload. #14867 (Amos Bird).
  • Show subqueries for SET and JOIN in EXPLAIN result. #14856 (Nikolai Kochetov).
  • Allow using multi-volume storage configuration in storage Distributed. #14839 (Pavel Kovalenko).
  • Construct query_start_time and query_start_time_microseconds from the same timespec. #14831 (Bharat Nallan).
  • Support for disabling persistency for StorageJoin and StorageSet, this feature is controlled by setting disable_set_and_join_persistency. And this PR solved issue #6318. #14776 (vxider).
  • Now COLUMNS can be used to wrap over a list of columns and apply column transformers afterwards. #14775 (Amos Bird).
  • Add merge_algorithm to system.merges table to improve merging inspections. #14705 (Amos Bird).
  • Fix potential memory leak caused by zookeeper exists watch. #14693 (hustnn).
  • Allow parallel execution of distributed DDL. #14684 (Azat Khuzhin).
  • Add QueryMemoryLimitExceeded event counter. This closes #14589. #14647 (fastio).
  • Fix some trailing whitespaces in query formatting. #14595 (Azat Khuzhin).
  • ClickHouse treats partition expr and key expr differently. Partition expr is used to construct an minmax index containing related columns, while primary key expr is stored as an expr. Sometimes user might partition a table at coarser levels, such as partition by i / 1000. However, binary operators are not monotonic and this PR tries to fix that. It might also benifit other use cases. #14513 (Amos Bird).
  • Add an option to skip access checks for DiskS3. s3 disk is an experimental feature. #14497 (Pavel Kovalenko).
  • Speed up server shutdown process if there are ongoing S3 requests. #14496 (Pavel Kovalenko).
  • SYSTEM RELOAD CONFIG now throws an exception if failed to reload and continues using the previous users.xml. The background periodic reloading also continues using the previous users.xml if failed to reload. #14492 (Vitaly Baranov).
  • For INSERTs with inline data in VALUES format in the script mode of clickhouse-client, support semicolon as the data terminator, in addition to the new line. Closes #12288. #13192 (Alexander Kuzmenkov).
  • Support custom codecs in compact parts. #12183 (Anton Popov).

Performance Improvement

  • Enable compact parts by default for small parts. This will allow to process frequent inserts slightly more efficiently (4..100 times). #11913 (alexey-milovidov).
  • Improve quantileTDigest performance. This fixes #2668. #15542 (Kruglov Pavel).
  • Significantly reduce memory usage in AggregatingInOrderTransform/optimize_aggregation_in_order. #15543 (Azat Khuzhin).
  • Faster 256-bit multiplication. #15418 (Artem Zuikov).
  • Improve performance of 256-bit types using (u)int64_t as base type for wide integers. Original wide integers use 8-bit types as base. #14859 (Artem Zuikov).
  • Explicitly use a temporary disk to store vertical merge temporary data. #15639 (Grigory Pervakov).
  • Use one S3 DeleteObjects request instead of multiple DeleteObject in a loop. No any functionality changes, so covered by existing tests like integration/test_log_family_s3. #15238 (ianton-ru).
  • Fix DateTime <op> DateTime mistakenly choosing the slow generic implementation. This fixes #15153. #15178 (Amos Bird).
  • Improve performance of GROUP BY key of type FixedString. #15034 (Amos Bird).
  • Only mlock code segment when starting clickhouse-server. In previous versions, all mapped regions were locked in memory, including debug info. Debug info is usually splitted to a separate file but if it isn't, it led to +2..3 GiB memory usage. #14929 (alexey-milovidov).
  • ClickHouse binary become smaller due to link time optimization.

Build/Testing/Packaging Improvement

  • Now we use clang-11 for production ClickHouse build. #15239 (alesapin).
  • Now we use clang-11 to build ClickHouse in CI. #14846 (alesapin).
  • Switch binary builds (Linux, Darwin, AArch64, FreeDSD) to clang-11. #15622 (Ilya Yatsishin).
  • Now all test images use llvm-symbolizer-11. #15069 (alesapin).
  • Allow to build with llvm-11. #15366 (alexey-milovidov).
  • Switch from clang-tidy-10 to clang-tidy-11. #14922 (alexey-milovidov).
  • Use LLVM's experimental pass manager by default. #15608 (Danila Kutenin).
  • Don't allow any C++ translation unit to build more than 10 minutes or to use more than 10 GB or memory. This fixes #14925. #15060 (alexey-milovidov).
  • Make performance test more stable and representative by splitting test runs and profile runs. #15027 (alexey-milovidov).
  • Attempt to make performance test more reliable. It is done by remapping the executable memory of the process on the fly with madvise to use transparent huge pages - it can lower the number of iTLB misses which is the main source of instabilities in performance tests. #14685 (alexey-milovidov).
  • Convert to python3. This closes #14886. #15007 (Azat Khuzhin).
  • Fail early in functional tests if server failed to respond. This closes #15262. #15267 (alexey-milovidov).
  • Allow to run AArch64 version of clickhouse-server without configs. This facilitates #15174. #15266 (alexey-milovidov).
  • Improvements in CI docker images: get rid of ZooKeeper and single script for test configs installation. #15215 (alesapin).
  • Fix CMake options forwarding in fast test script. Fixes error in #14711. #15155 (alesapin).
  • Added a script to perform hardware benchmark in a single command. #15115 (alexey-milovidov).
  • Splitted huge test test_dictionaries_all_layouts_and_sources into smaller ones. #15110 (Nikita Mikhaylov).
  • Maybe fix MSan report in base64 (on servers with AVX-512). This fixes #14006. #15030 (alexey-milovidov).
  • Reformat and cleanup code in all integration test *.py files. #14864 (Bharat Nallan).
  • Fix MaterializeMySQL empty transaction unstable test case found in CI. #14854 (Winter Zhang).
  • Attempt to speed up build a little. #14808 (alexey-milovidov).
  • Speed up build a little by removing unused headers. #14714 (alexey-milovidov).
  • Fix build failure in OSX. #14761 (Winter Zhang).
  • Enable ccache by default in cmake if it's found in OS. #14575 (alesapin).
  • Control CI builds configuration from the ClickHouse repository. #14547 (alesapin).
  • In CMake files: - Moved some options' descriptions' parts to comments above. - Replace 0 -> OFF, 1 -> ON in options default values. - Added some descriptions and links to docs to the options. - Replaced FUZZER option (there is another option ENABLE_FUZZING which also enables same functionality). - Removed ENABLE_GTEST_LIBRARY option as there is ENABLE_TESTS. See the full description in PR: #14711 (Mike).
  • Make binary a bit smaller (~50 Mb for debug version). #14555 (Artem Zuikov).
  • Use std::filesystem::path in ConfigProcessor for concatenating file paths. #14558 (Bharat Nallan).
  • Fix debug assertion in bitShiftLeft() when called with negative big integer. #14697 (Artem Zuikov).

ClickHouse release 20.9

ClickHouse release v20.9.5.5-stable, 2020-11-13

Bug Fix

  • Fix rare silent crashes when query profiler is on and ClickHouse is installed on OS with glibc version that has (supposedly) broken asynchronous unwind tables for some functions. This fixes #15301. This fixes #13098. #16846 (alexey-milovidov).
  • Now when parsing AVRO from input the LowCardinality is removed from type. Fixes #16188. #16521 (Mike).
  • Fix rapid growth of metadata when using MySQL Master -> MySQL Slave -> ClickHouse MaterializeMySQL Engine, and slave_parallel_worker enabled on MySQL Slave, by properly shrinking GTID sets. This fixes #15951. #16504 (TCeason).
  • Fix DROP TABLE for Distributed (racy with INSERT). #16409 (Azat Khuzhin).
  • Fix processing of very large entries in replication queue. Very large entries may appear in ALTER queries if table structure is extremely large (near 1 MB). This fixes #16307. #16332 (alexey-milovidov).
  • Fixed the inconsistent behaviour when a part of return data could be dropped because the set for its filtration wasn't created. #16308 (Nikita Mikhaylov).
  • Fix bug with MySQL database. When MySQL server used as database engine is down some queries raise Exception, because they try to get tables from disabled server, while it's unnecessary. For example, query SELECT ... FROM system.parts should work only with MergeTree tables and don't touch MySQL database at all. #16032 (Kruglov Pavel).

ClickHouse release v20.9.4.76-stable (2020-10-29)

Bug Fix

  • Fix double free in case of exception in function dictGet. It could have happened if dictionary was loaded with error. #16429 (Nikolai Kochetov).
  • Fix group by with totals/rollup/cube modifers and min/max functions over group by keys. Fixes #16393. #16397 (Anton Popov).
  • Fix async Distributed INSERT w/ prefer_localhost_replica=0 and internal_replication. #16358 (Azat Khuzhin).
  • Fix a very wrong code in TwoLevelStringHashTable implementation, which might lead to memory leak. I'm suprised how this bug can lurk for so long.... #16264 (Amos Bird).
  • Fix the case when memory can be overallocated regardless to the limit. This closes #14560. #16206 (alexey-milovidov).
  • Fix ALTER MODIFY ... ORDER BY query hang for ReplicatedVersionedCollapsingMergeTree. This fixes #15980. #16011 (alesapin).
  • Fix collate name & charset name parser and support length = 0 for string type. #16008 (Winter Zhang).
  • Allow to use direct layout for dictionaries with complex keys. #16007 (Anton Popov).
  • Prevent replica hang for 5-10 mins when replication error happens after a period of inactivity. #15987 (filimonov).
  • Fix rare segfaults when inserting into or selecting from MaterializedView and concurrently dropping target table (for Atomic database engine). #15984 (tavplubix).
  • Fix ambiguity in parsing of settings profiles: CREATE USER ... SETTINGS profile readonly is now considered as using a profile named readonly, not a setting named profile with the readonly constraint. This fixes #15628. #15982 (Vitaly Baranov).
  • Fix a crash when database creation fails. #15954 (Winter Zhang).
  • Fixed DROP TABLE IF EXISTS failure with Table ... doesn't exist error when table is concurrently renamed (for Atomic database engine). Fixed rare deadlock when concurrently executing some DDL queries with multiple tables (like DROP DATABASE and RENAME TABLE) Fixed DROP/DETACH DATABASE failure with Table ... doesn't exist when concurrently executing DROP/DETACH TABLE. #15934 (tavplubix).
  • Fix incorrect empty result for query from Distributed table if query has WHERE, PREWHERE and GLOBAL IN. Fixes #15792. #15933 (Nikolai Kochetov).
  • Fix possible deadlocks in RBAC. #15875 (Vitaly Baranov).
  • Fix exception Block structure mismatch in SELECT ... ORDER BY DESC queries which were executed after ALTER MODIFY COLUMN query. Fixes #15800. #15852 (alesapin).
  • Fix select count() inaccuracy for MaterializeMySQL. #15767 (tavplubix).
  • Fix some cases of queries, in which only virtual columns are selected. Previously Not found column _nothing in block exception may be thrown. Fixes #12298. #15756 (Anton Popov).
  • Fixed too low default value of max_replicated_logs_to_keep setting, which might cause replicas to become lost too often. Improve lost replica recovery process by choosing the most up-to-date replica to clone. Also do not remove old parts from lost replica, detach them instead. #15701 (tavplubix).
  • Fix error Cannot add simple transform to empty Pipe which happened while reading from Buffer table which has different structure than destination table. It was possible if destination table returned empty result for query. Fixes #15529. #15662 (Nikolai Kochetov).
  • Fixed bug with globs in S3 table function, region from URL was not applied to S3 client configuration. #15646 (Vladimir Chebotarev).
  • Decrement the ReadonlyReplica metric when detaching read-only tables. This fixes #15598. #15592 (sundyli).
  • Throw an error when a single parameter is passed to ReplicatedMergeTree instead of ignoring it. #15516 (nvartolomei).

Improvement

  • Now it's allowed to execute ALTER ... ON CLUSTER queries regardless of the <internal_replication> setting in cluster config. #16075 (alesapin).
  • Unfold {database}, {table} and {uuid} macros in ReplicatedMergeTree arguments on table creation. #16160 (tavplubix).

ClickHouse release v20.9.3.45-stable (2020-10-09)

Bug Fix

  • Fix error Cannot find column which may happen at insertion into MATERIALIZED VIEW in case if query for MV containes ARRAY JOIN. #15717 (Nikolai Kochetov).
  • Fix race condition in AMQP-CPP. #15667 (alesapin).
  • Fix the order of destruction for resources in ReadFromStorage step of query plan. It might cause crashes in rare cases. Possibly connected with #15610. #15645 (Nikolai Kochetov).
  • Fixed Element ... is not a constant expression error when using JSON* function result in VALUES, LIMIT or right side of IN operator. #15589 (tavplubix).
  • Prevent the possibility of error message Could not calculate available disk space (statvfs), errno: 4, strerror: Interrupted system call. This fixes #15541. #15557 (alexey-milovidov).
  • Significantly reduce memory usage in AggregatingInOrderTransform/optimize_aggregation_in_order. #15543 (Azat Khuzhin).
  • Mutation might hang waiting for some non-existent part after MOVE or REPLACE PARTITION or, in rare cases, after DETACH or DROP PARTITION. It's fixed. #15537 (tavplubix).
  • Fix bug when ILIKE operator stops being case insensitive if LIKE with the same pattern was executed. #15536 (alesapin).
  • Fix Missing columns errors when selecting columns which absent in data, but depend on other columns which also absent in data. Fixes #15530. #15532 (alesapin).
  • Fix bug with event subscription in DDLWorker which rarely may lead to query hangs in ON CLUSTER. Introduced in #13450. #15477 (alesapin).
  • Report proper error when the second argument of boundingRatio aggregate function has a wrong type. #15407 (detailyang).
  • Fix bug where queries like SELECT toStartOfDay(today()) fail complaining about empty time_zone argument. #15319 (Bharat Nallan).
  • Fix race condition during MergeTree table rename and background cleanup. #15304 (alesapin).
  • Fix rare race condition on server startup when system.logs are enabled. #15300 (alesapin).
  • Fix MSan report in QueryLog. Uninitialized memory can be used for the field memory_usage. #15258 (alexey-milovidov).
  • Fix instance crash when using joinGet with LowCardinality types. This fixes #15214. #15220 (Amos Bird).
  • Fix bug in table engine Buffer which doesn't allow to insert data of new structure into Buffer after ALTER query. Fixes #15117. #15192 (alesapin).
  • Adjust decimals field size in mysql column definition packet. #15152 (maqroll).
  • Fixed Cannot rename ... errno: 22, strerror: Invalid argument error on DDL query execution in Atomic database when running clickhouse-server in docker on Mac OS. #15024 (tavplubix).
  • Fix to make predicate push down work when subquery contains finalizeAggregation function. Fixes #14847. #14937 (filimonov).
  • Fix a problem where the server may get stuck on startup while talking to ZooKeeper, if the configuration files have to be fetched from ZK (using the from_zk include option). This fixes #14814. #14843 (Alexander Kuzmenkov).

Improvement

  • Now it's possible to change the type of version column for VersionedCollapsingMergeTree with ALTER query. #15442 (alesapin).

ClickHouse release v20.9.2.20, 2020-09-22

Backward Incompatible Change

  • When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

New Feature

  • Added column transformers EXCEPT, REPLACE, APPLY, which can be applied to the list of selected columns (after * or COLUMNS(...)). For example, you can write SELECT * EXCEPT(URL) REPLACE(number + 1 AS number). Another example: select * apply(length) apply(max) from wide_string_table to find out the maxium length of all string columns. #14233 (Amos Bird).
  • Added an aggregate function rankCorr which computes a rank correlation coefficient. #11769 (antikvist) #14411 (Nikita Mikhaylov).
  • Added table function view which turns a subquery into a table object. This helps passing queries around. For instance, it can be used in remote/cluster table functions. #12567 (Amos Bird).

Bug Fix

  • Fix bug when ALTER UPDATE mutation with Nullable column in assignment expression and constant value (like UPDATE x = 42) leads to incorrect value in column or segfault. Fixes #13634, #14045. #14646 (alesapin).
  • Fix wrong Decimal multiplication result caused wrong decimal scale of result column. #14603 (Artem Zuikov).
  • Fixed the incorrect sorting order of Nullable column. This fixes #14344. #14495 (Nikita Mikhaylov).
  • Fixed inconsistent comparison with primary key of type FixedString on index analysis if they're compered with a string of less size. This fixes #14908. #15033 (Amos Bird).
  • Fix bug which leads to wrong merges assignment if table has partitions with a single part. #14444 (alesapin).
  • If function bar was called with specifically crafted arguments, buffer overflow was possible. This closes #13926. #15028 (alexey-milovidov).
  • Publish CPU frequencies per logical core in system.asynchronous_metrics. This fixes #14923. #14924 (Alexander Kuzmenkov).
  • Fixed .metadata.tmp File exists error when using MaterializeMySQL database engine. #14898 (Winter Zhang).
  • Fix the issue when some invocations of extractAllGroups function may trigger "Memory limit exceeded" error. This fixes #13383. #14889 (alexey-milovidov).
  • Fix SIGSEGV for an attempt to INSERT into StorageFile(fd). #14887 (Azat Khuzhin).
  • Fix rare error in SELECT queries when the queried column has DEFAULT expression which depends on the other column which also has DEFAULT and not present in select query and not exists on disk. Partially fixes #14531. #14845 (alesapin).
  • Fix wrong monotonicity detection for shrunk Int -> Int cast of signed types. It might lead to incorrect query result. This bug is unveiled in #14513. #14783 (Amos Bird).
  • Fixed missed default database name in metadata of materialized view when executing ALTER ... MODIFY QUERY. #14664 (tavplubix).
  • Fix possibly incorrect result of function has when LowCardinality and Nullable types are involved. #14591 (Mike).
  • Cleanup data directory after Zookeeper exceptions during CREATE query for tables with ReplicatedMergeTree Engine. #14563 (Bharat Nallan).
  • Fix rare segfaults in functions with combinator -Resample, which could appear in result of overflow with very large parameters. #14562 (Anton Popov).
  • Check for array size overflow in topK aggregate function. Without this check the user may send a query with carefully crafted parameters that will lead to server crash. This closes #14452. #14467 (alexey-milovidov).
  • Proxy restart/start/stop/reload of SysVinit to systemd (if it is used). #14460 (Azat Khuzhin).
  • Stop query execution if exception happened in PipelineExecutor itself. This could prevent rare possible query hung. #14334 #14402 (Nikolai Kochetov).
  • Fix crash during ALTER query for table which was created AS table_function. Fixes #14212. #14326 (alesapin).
  • Fix exception during ALTER LIVE VIEW query with REFRESH command. LIVE VIEW is an experimental feature. #14320 (Bharat Nallan).
  • Fix QueryPlan lifetime (for EXPLAIN PIPELINE graph=1) for queries with nested interpreter. #14315 (Azat Khuzhin).
  • Better check for tuple size in SSD cache complex key external dictionaries. This fixes #13981. #14313 (alexey-milovidov).
  • Disallows CODEC on ALIAS column type. Fixes #13911. #14263 (Bharat Nallan).
  • Fix GRANT ALL statement when executed on a non-global level. #13987 (Vitaly Baranov).
  • Fix arrayJoin() capturing in lambda (exception with logical error message was thrown). #13792 (Azat Khuzhin).

Experimental Feature

  • Added db-generator tool for random database generation by given SELECT queries. It may faciliate reproducing issues when there is only incomplete bug report from the user. #14442 (Nikita Mikhaylov) #10973 (ZeDRoman).

Improvement

Performance Improvement

  • Optimize queries with LIMIT/LIMIT BY/ORDER BY for distributed with GROUP BY sharding_key (under optimize_skip_unused_shards and optimize_distributed_group_by_sharding_key). #10373 (Azat Khuzhin).
  • Creating sets for multiple JOIN and IN in parallel. It may slightly improve performance for queries with several different IN subquery expressions. #14412 (Nikolai Kochetov).
  • Improve Kafka engine performance by providing independent thread for each consumer. Separate thread pool for streaming engines (like Kafka). #13939 (fastio).

Build/Testing/Packaging Improvement

  • Lower binary size in debug build by removing debug info from Functions. This is needed only for one internal project in Yandex who is using very old linker. #14549 (alexey-milovidov).
  • Prepare for build with clang 11. #14455 (alexey-milovidov).
  • Fix the logic in backport script. In previous versions it was triggered for any labels of 100% red color. It was strange. #14433 (alexey-milovidov).
  • Integration tests use default base config. All config changes are explicit with main_configs, user_configs and dictionaries parameters for instance. #13647 (Ilya Yatsishin).

ClickHouse release 20.8

ClickHouse release v20.8.6.6-lts, 2020-11-13

Bug Fix

  • Fix rare silent crashes when query profiler is on and ClickHouse is installed on OS with glibc version that has (supposedly) broken asynchronous unwind tables for some functions. This fixes #15301. This fixes #13098. #16846 (alexey-milovidov).
  • Now when parsing AVRO from input the LowCardinality is removed from type. Fixes #16188. #16521 (Mike).
  • Fix rapid growth of metadata when using MySQL Master -> MySQL Slave -> ClickHouse MaterializeMySQL Engine, and slave_parallel_worker enabled on MySQL Slave, by properly shrinking GTID sets. This fixes #15951. #16504 (TCeason).
  • Fix DROP TABLE for Distributed (racy with INSERT). #16409 (Azat Khuzhin).
  • Fix processing of very large entries in replication queue. Very large entries may appear in ALTER queries if table structure is extremely large (near 1 MB). This fixes #16307. #16332 (alexey-milovidov).
  • Fixed the inconsistent behaviour when a part of return data could be dropped because the set for its filtration wasn't created. #16308 (Nikita Mikhaylov).
  • Fix bug with MySQL database. When MySQL server used as database engine is down some queries raise Exception, because they try to get tables from disabled server, while it's unnecessary. For example, query SELECT ... FROM system.parts should work only with MergeTree tables and don't touch MySQL database at all. #16032 (Kruglov Pavel).

ClickHouse release v20.8.5.45-lts, 2020-10-29

Bug Fix

  • Fix double free in case of exception in function dictGet. It could have happened if dictionary was loaded with error. #16429 (Nikolai Kochetov).
  • Fix group by with totals/rollup/cube modifers and min/max functions over group by keys. Fixes #16393. #16397 (Anton Popov).
  • Fix async Distributed INSERT w/ prefer_localhost_replica=0 and internal_replication. #16358 (Azat Khuzhin).
  • Fix a possible memory leak during GROUP BY with string keys, caused by an error in TwoLevelStringHashTable implementation. #16264 (Amos Bird).
  • Fix the case when memory can be overallocated regardless to the limit. This closes #14560. #16206 (alexey-milovidov).
  • Fix ALTER MODIFY ... ORDER BY query hang for ReplicatedVersionedCollapsingMergeTree. This fixes #15980. #16011 (alesapin).
  • Fix collate name & charset name parser and support length = 0 for string type. #16008 (Winter Zhang).
  • Allow to use direct layout for dictionaries with complex keys. #16007 (Anton Popov).
  • Prevent replica hang for 5-10 mins when replication error happens after a period of inactivity. #15987 (filimonov).
  • Fix rare segfaults when inserting into or selecting from MaterializedView and concurrently dropping target table (for Atomic database engine). #15984 (tavplubix).
  • Fix ambiguity in parsing of settings profiles: CREATE USER ... SETTINGS profile readonly is now considered as using a profile named readonly, not a setting named profile with the readonly constraint. This fixes #15628. #15982 (Vitaly Baranov).
  • Fix a crash when database creation fails. #15954 (Winter Zhang).
  • Fixed DROP TABLE IF EXISTS failure with Table ... doesn't exist error when table is concurrently renamed (for Atomic database engine). Fixed rare deadlock when concurrently executing some DDL queries with multiple tables (like DROP DATABASE and RENAME TABLE) Fixed DROP/DETACH DATABASE failure with Table ... doesn't exist when concurrently executing DROP/DETACH TABLE. #15934 (tavplubix).
  • Fix incorrect empty result for query from Distributed table if query has WHERE, PREWHERE and GLOBAL IN. Fixes #15792. #15933 (Nikolai Kochetov).
  • Fix possible deadlocks in RBAC. #15875 (Vitaly Baranov).
  • Fix exception Block structure mismatch in SELECT ... ORDER BY DESC queries which were executed after ALTER MODIFY COLUMN query. Fixes #15800. #15852 (alesapin).
  • Fix some cases of queries, in which only virtual columns are selected. Previously Not found column _nothing in block exception may be thrown. Fixes #12298. #15756 (Anton Popov).
  • Fix error Cannot find column which may happen at insertion into MATERIALIZED VIEW in case if query for MV containes ARRAY JOIN. #15717 (Nikolai Kochetov).
  • Fixed too low default value of max_replicated_logs_to_keep setting, which might cause replicas to become lost too often. Improve lost replica recovery process by choosing the most up-to-date replica to clone. Also do not remove old parts from lost replica, detach them instead. #15701 (tavplubix).
  • Fix error Cannot add simple transform to empty Pipe which happened while reading from Buffer table which has different structure than destination table. It was possible if destination table returned empty result for query. Fixes #15529. #15662 (Nikolai Kochetov).
  • Fixed bug with globs in S3 table function, region from URL was not applied to S3 client configuration. #15646 (Vladimir Chebotarev).
  • Decrement the ReadonlyReplica metric when detaching read-only tables. This fixes #15598. #15592 (sundyli).
  • Throw an error when a single parameter is passed to ReplicatedMergeTree instead of ignoring it. #15516 (nvartolomei).

Improvement

  • Now it's allowed to execute ALTER ... ON CLUSTER queries regardless of the <internal_replication> setting in cluster config. #16075 (alesapin).
  • Unfold {database}, {table} and {uuid} macros in ReplicatedMergeTree arguments on table creation. #16159 (tavplubix).

ClickHouse release v20.8.4.11-lts, 2020-10-09

Bug Fix

  • Fix the order of destruction for resources in ReadFromStorage step of query plan. It might cause crashes in rare cases. Possibly connected with #15610. #15645 (Nikolai Kochetov).
  • Fixed Element ... is not a constant expression error when using JSON* function result in VALUES, LIMIT or right side of IN operator. #15589 (tavplubix).
  • Prevent the possibility of error message Could not calculate available disk space (statvfs), errno: 4, strerror: Interrupted system call. This fixes #15541. #15557 (alexey-milovidov).
  • Significantly reduce memory usage in AggregatingInOrderTransform/optimize_aggregation_in_order. #15543 (Azat Khuzhin).
  • Mutation might hang waiting for some non-existent part after MOVE or REPLACE PARTITION or, in rare cases, after DETACH or DROP PARTITION. It's fixed. #15537 (tavplubix).
  • Fix bug when ILIKE operator stops being case insensitive if LIKE with the same pattern was executed. #15536 (alesapin).
  • Fix Missing columns errors when selecting columns which absent in data, but depend on other columns which also absent in data. Fixes #15530. #15532 (alesapin).
  • Fix bug with event subscription in DDLWorker which rarely may lead to query hangs in ON CLUSTER. Introduced in #13450. #15477 (alesapin).
  • Report proper error when the second argument of boundingRatio aggregate function has a wrong type. #15407 (detailyang).
  • Fix race condition during MergeTree table rename and background cleanup. #15304 (alesapin).
  • Fix rare race condition on server startup when system.logs are enabled. #15300 (alesapin).
  • Fix MSan report in QueryLog. Uninitialized memory can be used for the field memory_usage. #15258 (alexey-milovidov).
  • Fix instance crash when using joinGet with LowCardinality types. This fixes #15214. #15220 (Amos Bird).
  • Fix bug in table engine Buffer which doesn't allow to insert data of new structure into Buffer after ALTER query. Fixes #15117. #15192 (alesapin).
  • Adjust decimals field size in mysql column definition packet. #15152 (maqroll).
  • We already use padded comparison between String and FixedString (https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/FunctionsComparison.h#L333). This PR applies the same logic to field comparison which corrects the usage of FixedString as primary keys. This fixes #14908. #15033 (Amos Bird).
  • If function bar was called with specifically crafted arguments, buffer overflow was possible. This closes #13926. #15028 (alexey-milovidov).
  • Fixed Cannot rename ... errno: 22, strerror: Invalid argument error on DDL query execution in Atomic database when running clickhouse-server in docker on Mac OS. #15024 (tavplubix).
  • Now settings number_of_free_entries_in_pool_to_execute_mutation and number_of_free_entries_in_pool_to_lower_max_size_of_merge can be equal to background_pool_size. #14975 (alesapin).
  • Fix to make predicate push down work when subquery contains finalizeAggregation function. Fixes #14847. #14937 (filimonov).
  • Publish CPU frequencies per logical core in system.asynchronous_metrics. This fixes #14923. #14924 (Alexander Kuzmenkov).
  • Fixed .metadata.tmp File exists error when using MaterializeMySQL database engine. #14898 (Winter Zhang).
  • Fix a problem where the server may get stuck on startup while talking to ZooKeeper, if the configuration files have to be fetched from ZK (using the from_zk include option). This fixes #14814. #14843 (Alexander Kuzmenkov).
  • Fix wrong monotonicity detection for shrunk Int -> Int cast of signed types. It might lead to incorrect query result. This bug is unveiled in #14513. #14783 (Amos Bird).
  • Fixed the incorrect sorting order of Nullable column. This fixes #14344. #14495 (Nikita Mikhaylov).

Improvement

  • Now it's possible to change the type of version column for VersionedCollapsingMergeTree with ALTER query. #15442 (alesapin).

ClickHouse release v20.8.3.18-stable, 2020-09-18

Bug Fix

  • Fix the issue when some invocations of extractAllGroups function may trigger "Memory limit exceeded" error. This fixes #13383. #14889 (alexey-milovidov).
  • Fix SIGSEGV for an attempt to INSERT into StorageFile(fd). #14887 (Azat Khuzhin).
  • Fix rare error in SELECT queries when the queried column has DEFAULT expression which depends on the other column which also has DEFAULT and not present in select query and not exists on disk. Partially fixes #14531. #14845 (alesapin).
  • Fixed missed default database name in metadata of materialized view when executing ALTER ... MODIFY QUERY. #14664 (tavplubix).
  • Fix bug when ALTER UPDATE mutation with Nullable column in assignment expression and constant value (like UPDATE x = 42) leads to incorrect value in column or segfault. Fixes #13634, #14045. #14646 (alesapin).
  • Fix wrong Decimal multiplication result caused wrong decimal scale of result column. #14603 (Artem Zuikov).
  • Added the checker as neither calling lc->isNullable() nor calling ls->getDictionaryPtr()->isNullable() would return the correct result. #14591 (myrrc).
  • Cleanup data directory after Zookeeper exceptions during CreateQuery for StorageReplicatedMergeTree Engine. #14563 (Bharat Nallan).
  • Fix rare segfaults in functions with combinator -Resample, which could appear in result of overflow with very large parameters. #14562 (Anton Popov).

Improvement

ClickHouse release v20.8.2.3-stable, 2020-09-08

Backward Incompatible Change

  • Now OPTIMIZE FINAL query doesn't recalculate TTL for parts that were added before TTL was created. Use ALTER TABLE ... MATERIALIZE TTL once to calculate them, after that OPTIMIZE FINAL will evaluate TTL's properly. This behavior never worked for replicated tables. #14220 (alesapin).
  • Extend parallel_distributed_insert_select setting, adding an option to run INSERT into local table. The setting changes type from Bool to UInt64, so the values false and true are no longer supported. If you have these values in server configuration, the server will not start. Please replace them with 0 and 1, respectively. #14060 (Azat Khuzhin).
  • Remove support for the ODBCDriver input/output format. This was a deprecated format once used for communication with the ClickHouse ODBC driver, now long superseded by the ODBCDriver2 format. Resolves #13629. #13847 (hexiaoting).
  • When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

New Feature

  • Add the ability to specify Default compression codec for columns that correspond to settings specified in config.xml. Implements: #9074. #14049 (alesapin).
  • Support Kerberos authentication in Kafka, using krb5 and cyrus-sasl libraries. #12771 (Ilya Golshtein).
  • Add function normalizeQuery that replaces literals, sequences of literals and complex aliases with placeholders. Add function normalizedQueryHash that returns identical 64bit hash values for similar queries. It helps to analyze query log. This closes #11271. #13816 (alexey-milovidov).
  • Add time_zones table. #13880 (Bharat Nallan).
  • Add function defaultValueOfTypeName that returns the default value for a given type. #13877 (hcz).
  • Add countDigits(x) function that count number of decimal digits in integer or decimal column. Add isDecimalOverflow(d, [p]) function that checks if the value in Decimal column is out of its (or specified) precision. #14151 (Artem Zuikov).
  • Add quantileExactLow and quantileExactHigh implementations with respective aliases for medianExactLow and medianExactHigh. #13818 (Bharat Nallan).
  • Added date_trunc function that truncates a date/time value to a specified date/time part. #13888 (Vladimir Golovchenko).
  • Add new optional section <user_directories> to the main config. #13425 (Vitaly Baranov).
  • Add ALTER SAMPLE BY statement that allows to change table sample clause. #13280 (Amos Bird).
  • Function position now supports optional start_pos argument. #13237 (vdimir).

Bug Fix

  • Fix visible data clobbering by progress bar in client in interactive mode. This fixes #12562 and #13369 and #13584 and fixes #12964. #13691 (alexey-milovidov).
  • Fixed incorrect sorting order if LowCardinality column when sorting by multiple columns. This fixes #13958. #14223 (Nikita Mikhaylov).
  • Check for array size overflow in topK aggregate function. Without this check the user may send a query with carefully crafted parameters that will lead to server crash. This closes #14452. #14467 (alexey-milovidov).
  • Fix bug which can lead to wrong merges assignment if table has partitions with a single part. #14444 (alesapin).
  • Stop query execution if exception happened in PipelineExecutor itself. This could prevent rare possible query hung. Continuation of #14334. #14402 #14334 (Nikolai Kochetov).
  • Fix crash during ALTER query for table which was created AS table_function. Fixes #14212. #14326 (alesapin).
  • Fix exception during ALTER LIVE VIEW query with REFRESH command. Live view is an experimental feature. #14320 (Bharat Nallan).
  • Fix QueryPlan lifetime (for EXPLAIN PIPELINE graph=1) for queries with nested interpreter. #14315 (Azat Khuzhin).
  • Fix segfault in clickhouse-odbc-bridge during schema fetch from some external sources. This PR fixes #13861. #14267 (Vitaly Baranov).
  • Fix crash in mark inclusion search introduced in #12277. #14225 (Amos Bird).
  • Fix creation of tables with named tuples. This fixes #13027. #14143 (alexey-milovidov).
  • Fix formatting of minimal negative decimal numbers. This fixes #14111. #14119 (Alexander Kuzmenkov).
  • Fix DistributedFilesToInsert metric (zeroed when it should not). #14095 (Azat Khuzhin).
  • Fix pointInPolygon with const 2d array as polygon. #14079 (Alexey Ilyukhov).
  • Fixed wrong mount point in extra info for Poco::Exception: no space left on device. #14050 (tavplubix).
  • Fix GRANT ALL statement when executed on a non-global level. #13987 (Vitaly Baranov).
  • Fix parser to reject create table as table function with engine. #13940 (hcz).
  • Fix wrong results in select queries with DISTINCT keyword and subqueries with UNION ALL in case optimize_duplicate_order_by_and_distinct setting is enabled. #13925 (Artem Zuikov).
  • Fixed potential deadlock when renaming Distributed table. #13922 (tavplubix).
  • Fix incorrect sorting for FixedString columns when sorting by multiple columns. Fixes #13182. #13887 (Nikolai Kochetov).
  • Fix potentially imprecise result of topK/topKWeighted merge (with non-default parameters). #13817 (Azat Khuzhin).
  • Fix reading from MergeTree table with INDEX of type SET fails when comparing against NULL. This fixes #13686. #13793 (Amos Bird).
  • Fix arrayJoin capturing in lambda (LOGICAL_ERROR). #13792 (Azat Khuzhin).
  • Add step overflow check in function range. #13790 (Azat Khuzhin).
  • Fixed Directory not empty error when concurrently executing DROP DATABASE and CREATE TABLE. #13756 (alexey-milovidov).
  • Add range check for h3KRing function. This fixes #13633. #13752 (alexey-milovidov).
  • Fix race condition between DETACH and background merges. Parts may revive after detach. This is continuation of #8602 that did not fix the issue but introduced a test that started to fail in very rare cases, demonstrating the issue. #13746 (alexey-milovidov).
  • Fix logging Settings.Names/Values when log_queries_min_type > QUERY_START. #13737 (Azat Khuzhin).
  • Fixes /replicas_status endpoint response status code when verbose=1. #13722 (javi santana).
  • Fix incorrect message in clickhouse-server.init while checking user and group. #13711 (ylchou).
  • Do not optimize any(arrayJoin()) -> arrayJoin() under optimize_move_functions_out_of_any setting. #13681 (Azat Khuzhin).
  • Fix crash in JOIN with StorageMerge and set enable_optimize_predicate_expression=1. #13679 (Artem Zuikov).
  • Fix typo in error message about The value of 'number_of_free_entries_in_pool_to_lower_max_size_of_merge' setting. #13678 (alexey-milovidov).
  • Concurrent ALTER ... REPLACE/MOVE PARTITION ... queries might cause deadlock. It's fixed. #13626 (tavplubix).
  • Fixed the behaviour when sometimes cache-dictionary returned default value instead of present value from source. #13624 (Nikita Mikhaylov).
  • Fix secondary indices corruption in compact parts. Compact parts are experimental feature. #13538 (Anton Popov).
  • Fix premature ON CLUSTER timeouts for queries that must be executed on a single replica. Fixes #6704, #7228, #13361, #11884. #13450 (alesapin).
  • Fix wrong code in function netloc. This fixes #13335. #13446 (alexey-milovidov).
  • Fix possible race in StorageMemory. #13416 (Nikolai Kochetov).
  • Fix missing or excessive headers in TSV/CSVWithNames formats in HTTP protocol. This fixes #12504. #13343 (Azat Khuzhin).
  • Fix parsing row policies from users.xml when names of databases or tables contain dots. This fixes #5779, #12527. #13199 (Vitaly Baranov).
  • Fix access to redis dictionary after connection was dropped once. It may happen with cache and direct dictionary layouts. #13082 (Anton Popov).
  • Removed wrong auth access check when using ClickHouseDictionarySource to query remote tables. #12756 (sundyli).
  • Properly distinguish subqueries in some cases for common subexpression elimination. #8333. #8367 (Amos Bird).

Improvement

  • Disallows CODEC on ALIAS column type. Fixes #13911. #14263 (Bharat Nallan).
  • When waiting for a dictionary update to complete, use the timeout specified by query_wait_timeout_milliseconds setting instead of a hard-coded value. #14105 (Nikita Mikhaylov).
  • Add setting min_index_granularity_bytes that protects against accidentally creating a table with very low index_granularity_bytes setting. #14139 (Bharat Nallan).
  • Now it's possible to fetch partitions from clusters that use different ZooKeeper: ALTER TABLE table_name FETCH PARTITION partition_expr FROM 'zk-name:/path-in-zookeeper'. It's useful for shipping data to new clusters. #14155 (Amos Bird).
  • Slightly better performance of Memory table if it was constructed from a huge number of very small blocks (that's unlikely). Author of the idea: Mark Papadakis. Closes #14043. #14056 (alexey-milovidov).
  • Conditional aggregate functions (for example: avgIf, sumIf, maxIf) should return NULL when miss rows and use nullable arguments. #13964 (Winter Zhang).
  • Increase limit in -Resample combinator to 1M. #13947 (Mikhail f. Shiryaev).
  • Corrected an error in AvroConfluent format that caused the Kafka table engine to stop processing messages when an abnormally small, malformed, message was received. #13941 (Gervasio Varela).
  • Fix wrong error for long queries. It was possible to get syntax error other than Max query size exceeded for correct query. #13928 (Nikolai Kochetov).
  • Better error message for null value of TabSeparated format. #13906 (jiang tao).
  • Function arrayCompact will compare NaNs bitwise if the type of array elements is Float32/Float64. In previous versions NaNs were always not equal if the type of array elements is Float32/Float64 and were always equal if the type is more complex, like Nullable(Float64). This closes #13857. #13868 (alexey-milovidov).
  • Fix data race in lgamma function. This race was caught only in tsan, no side effects a really happened. #13842 (Nikolai Kochetov).
  • Avoid too slow queries when arrays are manipulated as fields. Throw exception instead. #13753 (alexey-milovidov).
  • Added Redis requirepass authorization (for redis dictionary source). #13688 (Ivan Torgashov).
  • Add MergeTree Write-Ahead-Log (WAL) dump tool. WAL is an experimental feature. #13640 (BohuTANG).
  • In previous versions lcm function may produce assertion violation in debug build if called with specifically crafted arguments. This fixes #13368. #13510 (alexey-milovidov).
  • Provide monotonicity for toDate/toDateTime functions in more cases. Monotonicity information is used for index analysis (more complex queries will be able to use index). Now the input arguments are saturated more naturally and provides better monotonicity. #13497 (Amos Bird).
  • Support compound identifiers for custom settings. Custom settings is an integration point of ClickHouse codebase with other codebases (no benefits for ClickHouse itself) #13496 (Vitaly Baranov).
  • Move parts from DiskLocal to DiskS3 in parallel. DiskS3 is an experimental feature. #13459 (Pavel Kovalenko).
  • Enable mixed granularity parts by default. #13449 (alesapin).
  • Proper remote host checking in S3 redirects (security-related thing). #13404 (Vladimir Chebotarev).
  • Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds to system.events. #13336 (ianton-ru).
  • Fix debug assertion when Decimal has too large negative exponent. Fixes #13188. #13228 (alexey-milovidov).
  • Added cache layer for DiskS3 (cache to local disk mark and index files). DiskS3 is an experimental feature. #13076 (Pavel Kovalenko).
  • Fix readline so it dumps history to file now. #13600 (Amos Bird).
  • Create system database with Atomic engine by default (a preparation to enable Atomic database engine by default everywhere). #13680 (tavplubix).

Performance Improvement

  • Slightly optimize very short queries with LowCardinality. #14129 (Anton Popov).
  • Enable parallel INSERTs for table engines Null, Memory, Distributed and Buffer when the setting max_insert_threads is set. #14120 (alexey-milovidov).
  • Fail fast if max_rows_to_read limit is exceeded on parts scan. The motivation behind this change is to skip ranges scan for all selected parts if it is clear that max_rows_to_read is already exceeded. The change is quite noticeable for queries over big number of parts. #13677 (Roman Khavronenko).
  • Slightly improve performance of aggregation by UInt8/UInt16 keys. #13099 (alexey-milovidov).
  • Optimize has(), indexOf() and countEqual() functions for Array(LowCardinality(T)) and constant right arguments. #12550 (myrrc).
  • When performing trivial INSERT SELECT queries, automatically set max_threads to 1 or max_insert_threads, and set max_block_size to min_insert_block_size_rows. Related to #5907. #12195 (flynn).

Experimental Feature

  • ClickHouse can work as MySQL replica - it is implemented by MaterializeMySQL database engine. Implements #4006. #10851 (Winter Zhang).
  • Add types Int128, Int256, UInt256 and related functions for them. Extend Decimals with Decimal256 (precision up to 76 digits). New types are under the setting allow_experimental_bigint_types. It is working extremely slow and bad. The implementation is incomplete. Please don't use this feature. #13097 (Artem Zuikov).

Build/Testing/Packaging Improvement

  • Added clickhouse install script, that is useful if you only have a single binary. #13528 (alexey-milovidov).
  • Allow to run clickhouse binary without configuration. #13515 (alexey-milovidov).
  • Enable check for typos in code with codespell. #13513 #13511 (alexey-milovidov).
  • Enable Shellcheck in CI as a linter of .sh tests. This closes #13168. #13530 #13529 (alexey-milovidov).
  • Add a CMake option to fail configuration instead of auto-reconfiguration, enabled by default. #13687 (Konstantin).
  • Expose version of embedded tzdata via TZDATA_VERSION in system.build_options. #13648 (filimonov).
  • Improve generation of system.time_zones table during build. Closes #14209. #14215 (filimonov).
  • Build ClickHouse with the most fresh tzdata from package repository. #13623 (alexey-milovidov).
  • Add the ability to write js-style comments in skip_list.json. #14159 (alesapin).
  • Ensure that there is no copy-pasted GPL code. #13514 (alexey-milovidov).
  • Switch tests docker images to use test-base parent. #14167 (Ilya Yatsishin).
  • Adding retry logic when bringing up docker-compose cluster; Increasing COMPOSE_HTTP_TIMEOUT. #14112 (vzakaznikov).
  • Enabled system.text_log in stress test to find more bugs. #13855 (Nikita Mikhaylov).
  • Testflows LDAP module: adding missing certificates and dhparam.pem for openldap4. #13780 (vzakaznikov).
  • ZooKeeper cannot work reliably in unit tests in CI infrastructure. Using unit tests for ZooKeeper interaction with real ZooKeeper is bad idea from the start (unit tests are not supposed to verify complex distributed systems). We already using integration tests for this purpose and they are better suited. #13745 (alexey-milovidov).
  • Added docker image for style check. Added style check that all docker and docker compose files are located in docker directory. #13724 (Ilya Yatsishin).
  • Fix cassandra build on Mac OS. #13708 (Ilya Yatsishin).
  • Fix link error in shared build. #13700 (Amos Bird).
  • Updating LDAP user authentication suite to check that it works with RBAC. #13656 (vzakaznikov).
  • Removed -DENABLE_CURL_CLIENT for contrib/aws. #13628 (Vladimir Chebotarev).
  • Increasing health-check timeouts for ClickHouse nodes and adding support to dump docker-compose logs if unhealthy containers found. #13612 (vzakaznikov).
  • Make sure #10977 is invalid. #13539 (Amos Bird).
  • Skip PR's from robot-clickhouse. #13489 (Nikita Mikhaylov).
  • Move Dockerfiles from integration tests to docker/test directory. docker_compose files are available in runner docker container. Docker images are built in CI and not in integration tests. #13448 (Ilya Yatsishin).

ClickHouse release 20.7

ClickHouse release v20.7.2.30-stable, 2020-08-31

Backward Incompatible Change

  • Function modulo (operator %) with at least one floating point number as argument will calculate remainder of division directly on floating point numbers without converting both arguments to integers. It makes behaviour compatible with most of DBMS. This also applicable for Date and DateTime data types. Added alias mod. This closes #7323. #12585 (alexey-milovidov).
  • Deprecate special printing of zero Date/DateTime values as 0000-00-00 and 0000-00-00 00:00:00. #12442 (alexey-milovidov).
  • The function groupArrayMoving* was not working for distributed queries. It's result was calculated within incorrect data type (without promotion to the largest type). The function groupArrayMovingAvg was returning integer number that was inconsistent with the avg function. This fixes #12568. #12622 (alexey-milovidov).
  • Add sanity check for MergeTree settings. If the settings are incorrect, the server will refuse to start or to create a table, printing detailed explanation to the user. #13153 (alexey-milovidov).
  • Protect from the cases when user may set background_pool_size to value lower than number_of_free_entries_in_pool_to_execute_mutation or number_of_free_entries_in_pool_to_lower_max_size_of_merge. In these cases ALTERs won't work or the maximum size of merge will be too limited. It will throw exception explaining what to do. This closes #10897. #12728 (alexey-milovidov).
  • When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

New Feature

  • Polygon dictionary type that provides efficient "reverse geocoding" lookups - to find the region by coordinates in a dictionary of many polygons (world map). It is using carefully optimized algorithm with recursive grids to maintain low CPU and memory usage. #9278 (achulkov2).
  • Added support of LDAP authentication for preconfigured users ("Simple Bind" method). #11234 (Denis Glazachev).
  • Introduce setting alter_partition_verbose_result which outputs information about touched parts for some types of ALTER TABLE ... PARTITION ... queries (currently ATTACH and FREEZE). Closes #8076. #13017 (alesapin).
  • Add bayesAB function for bayesian-ab-testing. #12327 (achimbab).
  • Added system.crash_log table into which stack traces for fatal errors are collected. This table should be empty. #12316 (alexey-milovidov).
  • Added http headers X-ClickHouse-Database and X-ClickHouse-Format which may be used to set default database and output format. #12981 (hcz).
  • Add minMap and maxMap functions support to SimpleAggregateFunction. #12662 (Ildus Kurbangaliev).
  • Add setting allow_non_metadata_alters which restricts to execute ALTER queries which modify data on disk. Disabled be default. Closes #11547. #12635 (alesapin).
  • A function formatRow is added to support turning arbitrary expressions into a string via given format. It's useful for manipulating SQL outputs and is quite versatile combined with the columns function. #12574 (Amos Bird).
  • Add FROM_UNIXTIME function for compatibility with MySQL, related to 12149. #12484 (flynn).
  • Allow Nullable types as keys in MergeTree tables if allow_nullable_key table setting is enabled. Closes #5319. #12433 (Amos Bird).
  • Integration with COS. #12386 (fastio).
  • Add mapAdd and mapSubtract functions for adding/subtracting key-mapped values. #11735 (Ildus Kurbangaliev).

Bug Fix

  • Fix premature ON CLUSTER timeouts for queries that must be executed on a single replica. Fixes #6704, #7228, #13361, #11884. #13450 (alesapin).
  • Fix crash in mark inclusion search introduced in #12277. #14225 (Amos Bird).
  • Fix race condition in external dictionaries with cache layout which can lead server crash. #12566 (alesapin).
  • Fix visible data clobbering by progress bar in client in interactive mode. This fixes #12562 and #13369 and #13584 and fixes #12964. #13691 (alexey-milovidov).
  • Fixed incorrect sorting order for LowCardinality columns when ORDER BY multiple columns is used. This fixes #13958. #14223 (Nikita Mikhaylov).
  • Removed hardcoded timeout, which wrongly overruled query_wait_timeout_milliseconds setting for cache-dictionary. #14105 (Nikita Mikhaylov).
  • Fixed wrong mount point in extra info for Poco::Exception: no space left on device. #14050 (tavplubix).
  • Fix wrong query optimization of select queries with DISTINCT keyword when subqueries also have DISTINCT in case optimize_duplicate_order_by_and_distinct setting is enabled. #13925 (Artem Zuikov).
  • Fixed potential deadlock when renaming Distributed table. #13922 (tavplubix).
  • Fix incorrect sorting for FixedString columns when ORDER BY multiple columns is used. Fixes #13182. #13887 (Nikolai Kochetov).
  • Fix potentially lower precision of topK/topKWeighted aggregations (with non-default parameters). #13817 (Azat Khuzhin).
  • Fix reading from MergeTree table with INDEX of type SET fails when compared against NULL. This fixes #13686. #13793 (Amos Bird).
  • Fix step overflow in function range(). #13790 (Azat Khuzhin).
  • Fixed Directory not empty error when concurrently executing DROP DATABASE and CREATE TABLE. #13756 (alexey-milovidov).
  • Add range check for h3KRing function. This fixes #13633. #13752 (alexey-milovidov).
  • Fix race condition between DETACH and background merges. Parts may revive after detach. This is continuation of #8602 that did not fix the issue but introduced a test that started to fail in very rare cases, demonstrating the issue. #13746 (alexey-milovidov).
  • Fix logging Settings.Names/Values when log_queries_min_type greater than QUERY_START. #13737 (Azat Khuzhin).
  • Fix incorrect message in clickhouse-server.init while checking user and group. #13711 (ylchou).
  • Do not optimize any(arrayJoin()) to arrayJoin() under optimize_move_functions_out_of_any. #13681 (Azat Khuzhin).
  • Fixed possible deadlock in concurrent ALTER ... REPLACE/MOVE PARTITION ... queries. #13626 (tavplubix).
  • Fixed the behaviour when sometimes cache-dictionary returned default value instead of present value from source. #13624 (Nikita Mikhaylov).
  • Fix secondary indices corruption in compact parts (compact parts is an experimental feature). #13538 (Anton Popov).
  • Fix wrong code in function netloc. This fixes #13335. #13446 (alexey-milovidov).
  • Fix error in parseDateTimeBestEffort function when unix timestamp was passed as an argument. This fixes #13362. #13441 (alexey-milovidov).
  • Fix invalid return type for comparison of tuples with NULL elements. Fixes #12461. #13420 (Nikolai Kochetov).
  • Fix wrong optimization caused aggregate function any(x) is found inside another aggregate function in query error with SET optimize_move_functions_out_of_any = 1 and aliases inside any(). #13419 (Artem Zuikov).
  • Fix possible race in StorageMemory. #13416 (Nikolai Kochetov).
  • Fix empty output for Arrow and Parquet formats in case if query return zero rows. It was done because empty output is not valid for this formats. #13399 (hcz).
  • Fix select queries with constant columns and prefix of primary key in ORDER BY clause. #13396 (Anton Popov).
  • Fix PrettyCompactMonoBlock for clickhouse-local. Fix extremes/totals with PrettyCompactMonoBlock. Fixes #7746. #13394 (Azat Khuzhin).
  • Fixed deadlock in system.text_log. #12452 (alexey-milovidov). It is a part of #12339. This fixes #12325. #13386 (Nikita Mikhaylov).
  • Fixed File(TSVWithNames*) (header was written multiple times), fixed clickhouse-local --format CSVWithNames* (lacks header, broken after #12197), fixed clickhouse-local --format CSVWithNames* with zero rows (lacks header). #13343 (Azat Khuzhin).
  • Fix segfault when function groupArrayMovingSum deserializes empty state. Fixes #13339. #13341 (alesapin).
  • Throw error on arrayJoin() function in JOIN ON section. #13330 (Artem Zuikov).
  • Fix crash in LEFT ASOF JOIN with join_use_nulls=1. #13291 (Artem Zuikov).
  • Fix possible error Totals having transform was already added to pipeline in case of a query from delayed replica. #13290 (Nikolai Kochetov).
  • The server may crash if user passed specifically crafted arguments to the function h3ToChildren. This fixes #13275. #13277 (alexey-milovidov).
  • Fix potentially low performance and slightly incorrect result for uniqExact, topK, sumDistinct and similar aggregate functions called on Float types with NaN values. It also triggered assert in debug build. This fixes #12491. #13254 (alexey-milovidov).
  • Fix assertion in KeyCondition when primary key contains expression with monotonic function and query contains comparison with constant whose type is different. This fixes #12465. #13251 (alexey-milovidov).
  • Return passed number for numbers with MSB set in function roundUpToPowerOfTwoOrZero(). It prevents potential errors in case of overflow of array sizes. #13234 (Azat Khuzhin).
  • Fix function if with nullable constexpr as cond that is not literal NULL. Fixes #12463. #13226 (alexey-milovidov).
  • Fix assert in arrayElement function in case of array elements are Nullable and array subscript is also Nullable. This fixes #12172. #13224 (alexey-milovidov).
  • Fix DateTime64 conversion functions with constant argument. #13205 (Azat Khuzhin).
  • Fix parsing row policies from users.xml when names of databases or tables contain dots. This fixes #5779, #12527. #13199 (Vitaly Baranov).
  • Fix access to redis dictionary after connection was dropped once. It may happen with cache and direct dictionary layouts. #13082 (Anton Popov).
  • Fix wrong index analysis with functions. It could lead to some data parts being skipped when reading from MergeTree tables. Fixes #13060. Fixes #12406. #13081 (Anton Popov).
  • Fix error Cannot convert column because it is constant but values of constants are different in source and result for remote queries which use deterministic functions in scope of query, but not deterministic between queries, like now(), now64(), randConstant(). Fixes #11327. #13075 (Nikolai Kochetov).
  • Fix crash which was possible for queries with ORDER BY tuple and small LIMIT. Fixes #12623. #13009 (Nikolai Kochetov).
  • Fix Block structure mismatch error for queries with UNION and JOIN. Fixes #12602. #12989 (Nikolai Kochetov).
  • Corrected merge_with_ttl_timeout logic which did not work well when expiration affected more than one partition over one time interval. (Authored by @excitoon). #12982 (Alexander Kazakov).
  • Fix columns duplication for range hashed dictionary created from DDL query. This fixes #10605. #12857 (alesapin).
  • Fix unnecessary limiting for the number of threads for selects from local replica. #12840 (Nikolai Kochetov).
  • Fix rare bug when ALTER DELETE and ALTER MODIFY COLUMN queries executed simultaneously as a single mutation. Bug leads to an incorrect amount of rows in count.txt and as a consequence incorrect data in part. Also, fix a small bug with simultaneous ALTER RENAME COLUMN and ALTER ADD COLUMN. #12760 (alesapin).
  • Wrong credentials being used when using clickhouse dictionary source to query remote tables. #12756 (sundyli).
  • Fix CAST(Nullable(String), Enum()). #12745 (Azat Khuzhin).
  • Fix performance with large tuples, which are interpreted as functions in IN section. The case when user writes WHERE x IN tuple(1, 2, ...) instead of WHERE x IN (1, 2, ...) for some obscure reason. #12700 (Anton Popov).
  • Fix memory tracking for input_format_parallel_parsing (by attaching thread to group). #12672 (Azat Khuzhin).
  • Fix wrong optimization optimize_move_functions_out_of_any=1 in case of any(func(<lambda>)). #12664 (Artem Zuikov).
  • Fixed #10572 fix bloom filter index with const expression. #12659 (Winter Zhang).
  • Fix SIGSEGV in StorageKafka when broker is unavailable (and not only). #12658 (Azat Khuzhin).
  • Add support for function if with Array(UUID) arguments. This fixes #11066. #12648 (alexey-milovidov).
  • CREATE USER IF NOT EXISTS now doesn't throw exception if the user exists. This fixes #12507. #12646 (Vitaly Baranov).
  • Exception There is no supertype... can be thrown during ALTER ... UPDATE in unexpected cases (e.g. when subtracting from UInt64 column). This fixes #7306. This fixes #4165. #12633 (alexey-milovidov).
  • Fix possible Pipeline stuck error for queries with external sorting. Fixes #12617. #12618 (Nikolai Kochetov).
  • Fix error Output of TreeExecutor is not sorted for OPTIMIZE DEDUPLICATE. Fixes #11572. #12613 (Nikolai Kochetov).
  • Fix the issue when alias on result of function any can be lost during query optimization. #12593 (Anton Popov).
  • Remove data for Distributed tables (blocks from async INSERTs) on DROP TABLE. #12556 (Azat Khuzhin).
  • Now ClickHouse will recalculate checksums for parts when file checksums.txt is absent. Broken since #9827. #12545 (alesapin).
  • Fix bug which lead to broken old parts after ALTER DELETE query when enable_mixed_granularity_parts=1. Fixes #12536. #12543 (alesapin).
  • Fixing race condition in live view tables which could cause data duplication. LIVE VIEW is an experimental feature. #12519 (vzakaznikov).
  • Fix backwards compatibility in binary format of AggregateFunction(avg, ...) values. This fixes #12342. #12486 (alexey-milovidov).
  • Fix crash in JOIN with dictionary when we are joining over expression of dictionary key: t JOIN dict ON expr(dict.id) = t.id. Disable dictionary join optimisation for this case. #12458 (Artem Zuikov).
  • Fix overflow when very large LIMIT or OFFSET is specified. This fixes #10470. This fixes #11372. #12427 (alexey-milovidov).
  • kafka: fix SIGSEGV if there is a message with error in the middle of the batch. #12302 (Azat Khuzhin).

Improvement

  • Keep smaller amount of logs in ZooKeeper. Avoid excessive growing of ZooKeeper nodes in case of offline replicas when having many servers/tables/inserts. #13100 (alexey-milovidov).
  • Now exceptions forwarded to the client if an error happened during ALTER or mutation. Closes #11329. #12666 (alesapin).
  • Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds to system.events, along with system.metrics, processes, query_log, etc. #13028 (ianton-ru).
  • Added SelectedRows and SelectedBytes to system.events, along with system.metrics, processes, query_log, etc. #12638 (ianton-ru).
  • Added current_database information to system.query_log. #12652 (Amos Bird).
  • Allow TabSeparatedRaw as input format. #12009 (hcz).
  • Now joinGet supports multi-key lookup. #12418 (Amos Bird).
  • Allow *Map aggregate functions to work on Arrays with NULLs. Fixes #13157. #13225 (alexey-milovidov).
  • Avoid overflow in parsing of DateTime values that will lead to negative unix timestamp in their timezone (for example, 1970-01-01 00:00:00 in Moscow). Saturate to zero instead. This fixes #3470. This fixes #4172. #12443 (alexey-milovidov).
  • AvroConfluent: Skip Kafka tombstone records - Support skipping broken records #13203 (Andrew Onyshchuk).
  • Fix wrong error for long queries. It was possible to get syntax error other than Max query size exceeded for correct query. #13928 (Nikolai Kochetov).
  • Fix data race in lgamma function. This race was caught only in tsan, no side effects really happened. #13842 (Nikolai Kochetov).
  • Fix a 'Week'-interval formatting for ATTACH/ALTER/CREATE QUOTA-statements. #13417 (vladimir-golovchenko).
  • Now broken parts are also reported when encountered in compact part processing. Compact parts is an experimental feature. #13282 (Amos Bird).
  • Fix assert in geohashesInBox. This fixes #12554. #13229 (alexey-milovidov).
  • Fix assert in parseDateTimeBestEffort. This fixes #12649. #13227 (alexey-milovidov).
  • Minor optimization in Processors/PipelineExecutor: breaking out of a loop because it makes sense to do so. #13058 (Mark Papadakis).
  • Support TRUNCATE table without TABLE keyword. #12653 (Winter Zhang).
  • Fix explain query format overwrite by default. This fixes #12541. #12541 (BohuTANG).
  • Allow to set JOIN kind and type in more standad way: LEFT SEMI JOIN instead of SEMI LEFT JOIN. For now both are correct. #12520 (Artem Zuikov).
  • Changes default value for multiple_joins_rewriter_version to 2. It enables new multiple joins rewriter that knows about column names. #12469 (Artem Zuikov).
  • Add several metrics for requests to S3 storages. #12464 (ianton-ru).
  • Use correct default secure port for clickhouse-benchmark with --secure argument. This fixes #11044. #12440 (alexey-milovidov).
  • Rollback insertion errors in Log, TinyLog, StripeLog engines. In previous versions insertion error lead to inconsisent table state (this works as documented and it is normal for these table engines). This fixes #12402. #12426 (alexey-milovidov).
  • Implement RENAME DATABASE and RENAME DICTIONARY for Atomic database engine - Add implicit {uuid} macro, which can be used in ZooKeeper path for ReplicatedMergeTree. It works with CREATE ... ON CLUSTER ... queries. Set show_table_uuid_in_table_create_query_if_not_nil to true to use it. - Make ReplicatedMergeTree engine arguments optional, /clickhouse/tables/{uuid}/{shard}/ and {replica} are used by default. Closes #12135. - Minor fixes. - These changes break backward compatibility of Atomic database engine. Previously created Atomic databases must be manually converted to new format. Atomic database is an experimental feature. #12343 (tavplubix).
  • Separated AWSAuthV4Signer into different logger, removed excessive AWSClient: AWSClient from log messages. #12320 (Vladimir Chebotarev).
  • Better exception message in disk access storage. #12625 (alesapin).
  • Better exception for function in with invalid number of arguments. #12529 (Anton Popov).
  • Fix error message about adaptive granularity. #12624 (alesapin).
  • Fix SETTINGS parse after FORMAT. #12480 (Azat Khuzhin).
  • If MergeTree table does not contain ORDER BY or PARTITION BY, it was possible to request ALTER to CLEAR all the columns and ALTER will stuck. Fixed #7941. #12382 (alexey-milovidov).
  • Avoid re-loading completion from the history file after each query (to avoid history overlaps with other client sessions). #13086 (Azat Khuzhin).

Performance Improvement

  • Lower memory usage for some operations up to 2 times. #12424 (alexey-milovidov).
  • Optimize PK lookup for queries that match exact PK range. #12277 (Ivan Babrou).
  • Slightly optimize very short queries with LowCardinality. #14129 (Anton Popov).
  • Slightly improve performance of aggregation by UInt8/UInt16 keys. #13091 and #13055 (alexey-milovidov).
  • Push down LIMIT step for query plan (inside subqueries). #13016 (Nikolai Kochetov).
  • Parallel primary key lookup and skipping index stages on parts, as described in #11564. #12589 (Ivan Babrou).
  • Converting String-type arguments of function "if" and "transform" into enum if set optimize_if_transform_strings_to_enum = 1. #12515 (Artem Zuikov).
  • Replaces monotonic functions with its argument in ORDER BY if set optimize_monotonous_functions_in_order_by=1. #12467 (Artem Zuikov).
  • Add order by optimization that rewrites ORDER BY x, f(x) with ORDER by x if set optimize_redundant_functions_in_order_by = 1. #12404 (Artem Zuikov).
  • Allow pushdown predicate when subquery contains WITH clause. This fixes #12293 #12663 (Winter Zhang).
  • Improve performance of reading from compact parts. Compact parts is an experimental feature. #12492 (Anton Popov).
  • Attempt to implement streaming optimization in DiskS3. DiskS3 is an experimental feature. #12434 (Vladimir Chebotarev).

Build/Testing/Packaging Improvement

ClickHouse release 20.6

ClickHouse release v20.6.3.28-stable

Backward Incompatible Change

  • When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

New Feature

Bug Fix

  • Fixed aggregate function any(x) is found inside another aggregate function in query error with SET optimize_move_functions_out_of_any = 1 and aliases inside any(). #13419 (Artem Zuikov).
  • Fixed PrettyCompactMonoBlock for clickhouse-local. Fixed extremes/totals with PrettyCompactMonoBlock. This fixes #7746. #13394 (Azat Khuzhin).
  • Fixed possible error Totals having transform was already added to pipeline in case of a query from delayed replica. #13290 (Nikolai Kochetov).
  • The server may crash if user passed specifically crafted arguments to the function h3ToChildren. This fixes #13275. #13277 (alexey-milovidov).
  • Fixed potentially low performance and slightly incorrect result for uniqExact, topK, sumDistinct and similar aggregate functions called on Float types with NaN values. It also triggered assert in debug build. This fixes #12491. #13254 (alexey-milovidov).
  • Fixed function if with nullable constexpr as cond that is not literal NULL. Fixes #12463. #13226 (alexey-milovidov).
  • Fixed assert in arrayElement function in case of array elements are Nullable and array subscript is also Nullable. This fixes #12172. #13224 (alexey-milovidov).
  • Fixed DateTime64 conversion functions with constant argument. #13205 (Azat Khuzhin).
  • Fixed wrong index analysis with functions. It could lead to pruning wrong parts, while reading from MergeTree tables. Fixes #13060. Fixes #12406. #13081 (Anton Popov).
  • Fixed error Cannot convert column because it is constant but values of constants are different in source and result for remote queries which use deterministic functions in scope of query, but not deterministic between queries, like now(), now64(), randConstant(). Fixes #11327. #13075 (Nikolai Kochetov).
  • Fixed unnecessary limiting for the number of threads for selects from local replica. #12840 (Nikolai Kochetov).
  • Fixed rare bug when ALTER DELETE and ALTER MODIFY COLUMN queries executed simultaneously as a single mutation. Bug leads to an incorrect amount of rows in count.txt and as a consequence incorrect data in part. Also, fix a small bug with simultaneous ALTER RENAME COLUMN and ALTER ADD COLUMN. #12760 (alesapin).
  • Fixed CAST(Nullable(String), Enum()). #12745 (Azat Khuzhin).
  • Fixed a performance with large tuples, which are interpreted as functions in IN section. The case when user write WHERE x IN tuple(1, 2, ...) instead of WHERE x IN (1, 2, ...) for some obscure reason. #12700 (Anton Popov).
  • Fixed memory tracking for input_format_parallel_parsing (by attaching thread to group). #12672 (Azat Khuzhin).
  • Fixed bloom filter index with const expression. This fixes #10572. #12659 (Winter Zhang).
  • Fixed SIGSEGV in StorageKafka when broker is unavailable (and not only). #12658 (Azat Khuzhin).
  • Added support for function if with Array(UUID) arguments. This fixes #11066. #12648 (alexey-milovidov).
  • CREATE USER IF NOT EXISTS now doesn't throw exception if the user exists. This fixes #12507. #12646 (Vitaly Baranov).
  • Better exception message in disk access storage. #12625 (alesapin).
  • The function groupArrayMoving* was not working for distributed queries. It's result was calculated within incorrect data type (without promotion to the largest type). The function groupArrayMovingAvg was returning integer number that was inconsistent with the avg function. This fixes #12568. #12622 (alexey-milovidov).
  • Fixed lack of aliases with function any. #12593 (Anton Popov).
  • Fixed race condition in external dictionaries with cache layout which can lead server crash. #12566 (alesapin).
  • Remove data for Distributed tables (blocks from async INSERTs) on DROP TABLE. #12556 (Azat Khuzhin).
  • Fixed bug which lead to broken old parts after ALTER DELETE query when enable_mixed_granularity_parts=1. Fixes #12536. #12543 (alesapin).
  • Better exception for function in with invalid number of arguments. #12529 (Anton Popov).
  • Fixing race condition in live view tables which could cause data duplication. #12519 (vzakaznikov).
  • Fixed performance issue, while reading from compact parts. #12492 (Anton Popov).
  • Fixed backwards compatibility in binary format of AggregateFunction(avg, ...) values. This fixes #12342. #12486 (alexey-milovidov).
  • Fixed SETTINGS parse after FORMAT. #12480 (Azat Khuzhin).
  • Fixed the deadlock if text_log is enabled. #12452 (alexey-milovidov).
  • Fixed overflow when very large LIMIT or OFFSET is specified. This fixes #10470. This fixes #11372. #12427 (alexey-milovidov).
  • Fixed possible segfault if StorageMerge. This fixes #12054. #12401 (tavplubix).
  • Reverted change introduced in #11079 to resolve #12098. #12397 (Mike).
  • Additional check for arguments of bloom filter index. This fixes #11408. #12388 (alexey-milovidov).
  • Avoid exception when negative or floating point constant is used in WHERE condition for indexed tables. This fixes #11905. #12384 (alexey-milovidov).
  • Allowed to CLEAR column even if there are depending DEFAULT expressions. This fixes #12333. #12378 (alexey-milovidov).
  • Fix TOTALS/ROLLUP/CUBE for aggregate functions with -State and Nullable arguments. This fixes #12163. #12376 (alexey-milovidov).
  • Fixed error message and exit codes for ALTER RENAME COLUMN queries, when RENAME is not allowed. Fixes #12301 and #12303. #12335 (alesapin).
  • Fixed very rare race condition in ReplicatedMergeTreeQueue. #12315 (alexey-milovidov).
  • When using codec Delta or DoubleDelta with non fixed width types, exception with code LOGICAL_ERROR was returned instead of exception with code BAD_ARGUMENTS (we ensure that exceptions with code logical error never happen). This fixes #12110. #12308 (alexey-milovidov).
  • Fixed order of columns in WITH FILL modifier. Previously order of columns of ORDER BY statement wasn't respected. #12306 (Anton Popov).
  • Avoid "bad cast" exception when there is an expression that filters data by virtual columns (like _table in Merge tables) or by "index" columns in system tables such as filtering by database name when querying from system.tables, and this expression returns Nullable type. This fixes #12166. #12305 (alexey-milovidov).
  • Fixed TTL after renaming column, on which depends TTL expression. #12304 (Anton Popov).
  • Fixed SIGSEGV if there is an message with error in the middle of the batch in Kafka Engine. #12302 (Azat Khuzhin).
  • Fixed the situation when some threads might randomly hang for a few seconds during DNS cache updating. #12296 (tavplubix).
  • Fixed typo in setting name. #12292 (alexey-milovidov).
  • Show error after TrieDictionary failed to load. #12290 (Vitaly Baranov).
  • The function arrayFill worked incorrectly for empty arrays that may lead to crash. This fixes #12263. #12279 (alexey-milovidov).
  • Implement conversions to the common type for LowCardinality types. This allows to execute UNION ALL of tables with columns of LowCardinality and other columns. This fixes #8212. This fixes #4342. #12275 (alexey-milovidov).
  • Fixed the behaviour on reaching redirect limit in request to S3 storage. #12256 (ianton-ru).
  • Fixed the behaviour when during multiple sequential inserts in StorageFile header for some special types was written more than once. This fixed #6155. #12197 (Nikita Mikhaylov).
  • Fixed logical functions for UInt8 values when they are not equal to 0 or 1. #12196 (Alexander Kazakov).
  • Cap max_memory_usage* limits to the process resident memory. #12182 (Azat Khuzhin).
  • Fix dictGet arguments check during GROUP BY injective functions elimination. #12179 (Azat Khuzhin).
  • Fixed the behaviour when SummingMergeTree engine sums up columns from partition key. Added an exception in case of explicit definition of columns to sum which intersects with partition key columns. This fixes #7867. #12173 (Nikita Mikhaylov).
  • Don't split the dictionary source's table name into schema and table name itself if ODBC connection doesn't support schema. #12165 (Vitaly Baranov).
  • Fixed wrong logic in ALTER DELETE that leads to deleting of records when condition evaluates to NULL. This fixes #9088. This closes #12106. #12153 (alexey-milovidov).
  • Fixed transform of query to send to external DBMS (e.g. MySQL, ODBC) in presense of aliases. This fixes #12032. #12151 (alexey-milovidov).
  • Fixed bad code in redundant ORDER BY optimization. The bug was introduced in #10067. #12148 (alexey-milovidov).
  • Fixed potential overflow in integer division. This fixes #12119. #12140 (alexey-milovidov).
  • Fixed potential infinite loop in greatCircleDistance, geoDistance. This fixes #12117. #12137 (alexey-milovidov).
  • Normalize "pid" file handling. In previous versions the server may refuse to start if it was killed without proper shutdown and if there is another process that has the same pid as previously runned server. Also pid file may be removed in unsuccessful server startup even if there is another server running. This fixes #3501. #12133 (alexey-milovidov).
  • Fixed bug which leads to incorrect table metadata in ZooKeepeer for ReplicatedVersionedCollapsingMergeTree tables. Fixes #12093. #12121 (alesapin).
  • Avoid "There is no query" exception for materialized views with joins or with subqueries attached to system logs (system.query_log, metric_log, etc) or to engine=Buffer underlying table. #12120 (filimonov).
  • Fixed handling dependency of table with ENGINE=Dictionary on dictionary. This fixes #10994. This fixes #10397. #12116 (Vitaly Baranov).
  • Format Parquet now properly works with LowCardinality and LowCardinality(Nullable) types. Fixes #12086, #8406. #12108 (Nikolai Kochetov).
  • Fixed performance for selects with UNION caused by wrong limit for the total number of threads. Fixes #12030. #12103 (Nikolai Kochetov).
  • Fixed segfault with -StateResample combinators. #12092 (Anton Popov).
  • Fixed empty result_rows and result_bytes metrics in system.quey_log for selects. Fixes #11595. #12089 (Nikolai Kochetov).
  • Fixed unnecessary limiting the number of threads for selects from VIEW. Fixes #11937. #12085 (Nikolai Kochetov).
  • Fixed SIGSEGV in StorageKafka on DROP TABLE. #12075 (Azat Khuzhin).
  • Fixed possible crash while using wrong type for PREWHERE. Fixes #12053, #12060. #12060 (Nikolai Kochetov).
  • Fixed error Cannot capture column for higher-order functions with Tuple(LowCardinality) argument. Fixes #9766. #12055 (Nikolai Kochetov).
  • Fixed constraints check if constraint is a constant expression. This fixes #11360. #12042 (alexey-milovidov).
  • Fixed wrong result and potential crash when invoking function if with arguments of type FixedString with different sizes. This fixes #11362. #12021 (alexey-milovidov).

Improvement

  • Allowed to set JOIN kind and type in more standard way: LEFT SEMI JOIN instead of SEMI LEFT JOIN. For now both are correct. #12520 (Artem Zuikov).
  • lifetime_rows/lifetime_bytes for Buffer engine. #12421 (Azat Khuzhin).
  • Write the detail exception message to the client instead of 'MySQL server has gone away'. #12383 (BohuTANG).
  • Allows to change a charset which is used for printing grids borders. Available charsets are following: UTF-8, ASCII. Setting output_format_pretty_grid_charset enables this feature. #12372 (Sabyanin Maxim).
  • Supported MySQL 'SELECT DATABASE()' #9336 2. Add MySQL replacement query integration test. #12314 (BohuTANG).
  • Added KILL QUERY [connection_id] for the MySQL client/driver to cancel the long query, issue #12038. #12152 (BohuTANG).
  • Added support for %g (two digit ISO year) and %G (four digit ISO year) substitutions in formatDateTime function. #12136 (vivarum).
  • Added 'type' column in system.disks. #12115 (ianton-ru).
  • Improved REVOKE command: now it requires grant/admin option for only access which will be revoked. For example, to execute REVOKE ALL ON *.* FROM user1 now it doesn't require to have full access rights granted with grant option. Added command REVOKE ALL FROM user1 - it revokes all granted roles from user1. #12083 (Vitaly Baranov).
  • Added replica priority for load_balancing (for manual prioritization of the load balancing). #11995 (Azat Khuzhin).
  • Switched paths in S3 metadata to relative which allows to handle S3 blobs more easily. #11892 (Vladimir Chebotarev).

Performance Improvement

  • Improved performace of 'ORDER BY' and 'GROUP BY' by prefix of sorting key (enabled with optimize_aggregation_in_order setting, disabled by default). #11696 (Anton Popov).
  • Removed injective functions inside uniq*() if set optimize_injective_functions_inside_uniq=1. #12337 (Ruslan Kamalov).
  • Index not used for IN operator with literals, performance regression introduced around v19.3. This fixes #10574. #12062 (nvartolomei).
  • Implemented single part uploads for DiskS3 (experimental feature). #12026 (Vladimir Chebotarev).

Experimental Feature

  • Added new in-memory format of parts in MergeTree-family tables, which stores data in memory. Parts are written on disk at first merge. Part will be created in in-memory format if its size in rows or bytes is below thresholds min_rows_for_compact_part and min_bytes_for_compact_part. Also optional support of Write-Ahead-Log is available, which is enabled by default and is controlled by setting in_memory_parts_enable_wal. #10697 (Anton Popov).

Build/Testing/Packaging Improvement

  • Implement AST-based query fuzzing mode for clickhouse-client. See this label for the list of issues we recently found by fuzzing. Most of them were found by this tool, and a couple by SQLancer and 00746_sql_fuzzy.pl. #12111 (Alexander Kuzmenkov).
  • Add new type of tests based on Testflows framework. #12090 (vzakaznikov).
  • Added S3 HTTPS integration test. #12412 (Pavel Kovalenko).
  • Log sanitizer trap messages from separate thread. This will prevent possible deadlock under thread sanitizer. #12313 (alexey-milovidov).
  • Now functional and stress tests will be able to run with old version of clickhouse-test script. #12287 (alesapin).
  • Remove strange file creation during build in orc. #12258 (Nikita Mikhaylov).
  • Place common docker compose files to integration docker container. #12168 (Ilya Yatsishin).
  • Fix warnings from CodeQL. CodeQL is another static analyzer that we will use along with clang-tidy and PVS-Studio that we use already. #12138 (alexey-milovidov).
  • Minor CMake fixes for UNBUNDLED build. #12131 (Matwey V. Kornilov).
  • Added a showcase of the minimal Docker image without using any Linux distribution. #12126 (alexey-milovidov).
  • Perform an upgrade of system packages in the clickhouse-server docker image. #12124 (Ivan Blinkov).
  • Add UNBUNDLED flag to system.build_options table. Move skip lists for clickhouse-test to clickhouse repo. #12107 (alesapin).
  • Regular check by Anchore Container Analysis security analysis tool that looks for CVE in clickhouse-server Docker image. Also confirms that Dockerfile is buildable. Runs daily on master and on pull-requests to Dockerfile. #12102 (Ivan Blinkov).
  • Daily check by GitHub CodeQL security analysis tool that looks for CWE. #12101 (Ivan Blinkov).
  • Install ca-certificates before the first apt-get update in Dockerfile. #12095 (Ivan Blinkov).

ClickHouse release 20.5

ClickHouse release v20.5.4.40-stable 2020-08-10

Bug Fix

  • Fixed wrong index analysis with functions. It could lead to pruning wrong parts, while reading from MergeTree tables. Fixes #13060. Fixes #12406. #13081 (Anton Popov).
  • Fixed unnecessary limiting for the number of threads for selects from local replica. #12840 (Nikolai Kochetov).
  • Fixed performance with large tuples, which are interpreted as functions in IN section. The case when user write WHERE x IN tuple(1, 2, ...) instead of WHERE x IN (1, 2, ...) for some obscure reason. #12700 (Anton Popov).
  • Fixed memory tracking for input_format_parallel_parsing (by attaching thread to group). #12672 (Azat Khuzhin).
  • Fixed bloom filter index with const expression. This fixes #10572. #12659 (Winter Zhang).
  • Fixed SIGSEGV in StorageKafka when broker is unavailable (and not only). #12658 (Azat Khuzhin).
  • Added support for function if with Array(UUID) arguments. This fixes #11066. #12648 (alexey-milovidov).
  • Fixed lack of aliases with function any. #12593 (Anton Popov).
  • Fixed race condition in external dictionaries with cache layout which can lead server crash. #12566 (alesapin).
  • Remove data for Distributed tables (blocks from async INSERTs) on DROP TABLE. #12556 (Azat Khuzhin).
  • Fixed bug which lead to broken old parts after ALTER DELETE query when enable_mixed_granularity_parts=1. Fixes #12536. #12543 (alesapin).
  • Better exception for function in with invalid number of arguments. #12529 (Anton Popov).
  • Fixed race condition in live view tables which could cause data duplication. #12519 (vzakaznikov).
  • Fixed performance issue, while reading from compact parts. #12492 (Anton Popov).
  • Fixed backwards compatibility in binary format of AggregateFunction(avg, ...) values. This fixes #12342. #12486 (alexey-milovidov).
  • Fixed the deadlock if text_log is enabled. #12452 (alexey-milovidov).
  • Fixed overflow when very large LIMIT or OFFSET is specified. This fixes #10470. This fixes #11372. #12427 (alexey-milovidov).
  • Fixed possible segfault if StorageMerge. Closes #12054. #12401 (tavplubix).
  • Reverts change introduced in #11079 to resolve #12098. #12397 (Mike).
  • Avoid exception when negative or floating point constant is used in WHERE condition for indexed tables. This fixes #11905. #12384 (alexey-milovidov).
  • Allow to CLEAR column even if there are depending DEFAULT expressions. This fixes #12333. #12378 (alexey-milovidov).
  • Fixed TOTALS/ROLLUP/CUBE for aggregate functions with -State and Nullable arguments. This fixes #12163. #12376 (alexey-milovidov).
  • Fixed SIGSEGV if there is an message with error in the middle of the batch in Kafka Engine. #12302 (Azat Khuzhin).
  • Fixed the behaviour when SummingMergeTree engine sums up columns from partition key. Added an exception in case of explicit definition of columns to sum which intersects with partition key columns. This fixes #7867. #12173 (Nikita Mikhaylov).
  • Fixed transform of query to send to external DBMS (e.g. MySQL, ODBC) in presense of aliases. This fixes #12032. #12151 (alexey-milovidov).
  • Fixed bug which leads to incorrect table metadata in ZooKeepeer for ReplicatedVersionedCollapsingMergeTree tables. Fixes #12093. #12121 (alesapin).
  • Fixed unnecessary limiting the number of threads for selects from VIEW. Fixes #11937. #12085 (Nikolai Kochetov).
  • Fixed crash in JOIN with LowCardinality type with join_algorithm=partial_merge. #12035 (Artem Zuikov).
  • Fixed wrong result for if() with NULLs in condition. #11807 (Artem Zuikov).

Performance Improvement

  • Index not used for IN operator with literals, performance regression introduced around v19.3. This fixes #10574. #12062 (nvartolomei).

Build/Testing/Packaging Improvement

  • Install ca-certificates before the first apt-get update in Dockerfile. #12095 (Ivan Blinkov).

ClickHouse release v20.5.2.7-stable 2020-07-02

Backward Incompatible Change

  • Return non-Nullable result from COUNT(DISTINCT), and uniq aggregate functions family. If all passed values are NULL, return zero instead. This improves SQL compatibility. #11661 (alexey-milovidov).
  • Added a check for the case when user-level setting is specified in a wrong place. User-level settings should be specified in users.xml inside <profile> section for specific user profile (or in <default> for default settings). The server won't start with exception message in log. This fixes #9051. If you want to skip the check, you can either move settings to the appropriate place or add <skip_check_for_incorrect_settings>1</skip_check_for_incorrect_settings> to config.xml. #11449 (alexey-milovidov).
  • The setting input_format_with_names_use_header is enabled by default. It will affect parsing of input formats -WithNames and -WithNamesAndTypes. #10937 (alexey-milovidov).
  • Remove experimental_use_processors setting. It is enabled by default. #10924 (Nikolai Kochetov).
  • Update zstd to 1.4.4. It has some minor improvements in performance and compression ratio. If you run replicas with different versions of ClickHouse you may see reasonable error messages Data after merge is not byte-identical to data on another replicas. with explanation. These messages are Ok and you should not worry. This change is backward compatible but we list it here in changelog in case you will wonder about these messages. #10663 (alexey-milovidov).
  • Added a check for meaningless codecs and a setting allow_suspicious_codecs to control this check. This closes #4966. #10645 (alexey-milovidov).
  • Several Kafka setting changes their defaults. See #11388.
  • When upgrading from versions older than 20.5, if rolling update is performed and cluster contains both versions 20.5 or greater and less than 20.5, if ClickHouse nodes with old versions are restarted and old version has been started up in presence of newer versions, it may lead to Part ... intersects previous part errors. To prevent this error, first install newer clickhouse-server packages on all cluster nodes and then do restarts (so, when clickhouse-server is restarted, it will start up with the new version).

New Feature

  • TTL DELETE WHERE and TTL GROUP BY for automatic data coarsening and rollup in tables. #10537 (expl0si0nn).
  • Implementation of PostgreSQL wire protocol. #10242 (Movses).
  • Added system tables for users, roles, grants, settings profiles, quotas, row policies; added commands SHOW USER, SHOW [CURRENT|ENABLED] ROLES, SHOW SETTINGS PROFILES. #10387 (Vitaly Baranov).
  • Support writes in ODBC Table function #10554 (ageraab). #10901 (tavplubix).
  • Add query performance metrics based on Linux perf_events (these metrics are calculated with hardware CPU counters and OS counters). It is optional and requires CAP_SYS_ADMIN to be set on clickhouse binary. #9545 Andrey Skobtsov. #11226 (Alexander Kuzmenkov).
  • Now support NULL and NOT NULL modifiers for data types in CREATE query. #11057 (Павел Потемкин).
  • Add ArrowStream input and output format. #11088 (hcz).
  • Support Cassandra as external dictionary source. #4978 (favstovol).
  • Added a new layout direct which loads all the data directly from the source for each query, without storing or caching data. #10622 (Artem Streltsov).
  • Added new complex_key_direct layout to dictionaries, that does not store anything locally during query execution. #10850 (Artem Streltsov).
  • Added support for MySQL style global variables syntax (stub). This is needed for compatibility of MySQL protocol. #11832 (alexey-milovidov).
  • Added syntax highligting to clickhouse-client using replxx. #11422 (Tagir Kuskarov).
  • minMap and maxMap functions were added. #11603 (Ildus Kurbangaliev).
  • Add the system.asynchronous_metric_log table that logs historical metrics from system.asynchronous_metrics. #11588 (Alexander Kuzmenkov).
  • Add functions extractAllGroupsHorizontal(haystack, re) and extractAllGroupsVertical(haystack, re). #11554 (Vasily Nemkov).
  • Add SHOW CLUSTER(S) queries. #11467 (hexiaoting).
  • Add netloc function for extracting network location, similar to urlparse(url), netloc in python. #11356 (Guillaume Tassery).
  • Add 2 more virtual columns for engine=Kafka to access message headers. #11283 (filimonov).
  • Add _timestamp_ms virtual column for Kafka engine (type is Nullable(DateTime64(3))). #11260 (filimonov).
  • Add function randomFixedString. #10866 (Andrei Nekrashevich).
  • Add function fuzzBits that randomly flips bits in a string with given probability. #11237 (Andrei Nekrashevich).
  • Allow comparison of numbers with constant string in comparison operators, IN and VALUES sections. #11647 (alexey-milovidov).
  • Add round_robin load_balancing mode. #11645 (Azat Khuzhin).
  • Add cast_keep_nullable setting. If set CAST(something_nullable AS Type) return Nullable(Type). #11733 (Artem Zuikov).
  • Added column position to system.columns table and column_position to system.parts_columns table. It contains ordinal position of a column in a table starting with 1. This closes #7744. #11655 (alexey-milovidov).
  • ON CLUSTER support for SYSTEM {FLUSH DISTRIBUTED,STOP/START DISTRIBUTED SEND}. #11415 (Azat Khuzhin).
  • Add system.distribution_queue table. #11394 (Azat Khuzhin).
  • Support for all format settings in Kafka, expose some setting on table level, adjust the defaults for better performance. #11388 (filimonov).
  • Add port function (to extract port from URL). #11120 (Azat Khuzhin).
  • Now dictGet* functions accept table names. #11050 (Vitaly Baranov).
  • The clickhouse-format tool is now able to format multiple queries when the -n argument is used. #10852 (Darío).
  • Possibility to configure proxy-resolver for DiskS3. #10744 (Pavel Kovalenko).
  • Make pointInPolygon work with non-constant polygon. PointInPolygon now can take Array(Array(Tuple(..., ...))) as second argument, array of polygon and holes. #10623 (Alexey Ilyukhov) #11421 (Alexey Ilyukhov).
  • Added move_ttl_info to system.parts in order to provide introspection of move TTL functionality. #10591 (Vladimir Chebotarev).
  • Possibility to work with S3 through proxies. #10576 (Pavel Kovalenko).
  • Add NCHAR and NVARCHAR synonims for data types. #11025 (alexey-milovidov).
  • Resolved #7224: added FailedQuery, FailedSelectQuery and FailedInsertQuery metrics to system.events table. #11151 (Nikita Orlov).
  • Add more jemalloc statistics to system.asynchronous_metrics, and ensure that we see up-to-date values for them. #11748 (Alexander Kuzmenkov).
  • Allow to specify default S3 credentials and custom auth headers. #11134 (Grigory Pervakov).
  • Added new functions to import/export DateTime64 as Int64 with various precision: to-/fromUnixTimestamp64Milli/-Micro/-Nano. #10923 (Vasily Nemkov).
  • Allow specifying mongodb:// URI for MongoDB dictionaries. #10915 (Alexander Kuzmenkov).
  • OFFSET keyword can now be used without an affiliated LIMIT clause. #10802 (Guillaume Tassery).
  • Added system.licenses table. This table contains licenses of third-party libraries that are located in contrib directory. This closes #2890. #10795 (alexey-milovidov).
  • New function function toStartOfSecond(DateTime64) -> DateTime64 that nullifies sub-second part of DateTime64 value. #10722 (Vasily Nemkov).
  • Add new input format JSONAsString that accepts a sequence of JSON objects separated by newlines, spaces and/or commas. #10607 (Kruglov Pavel).
  • Allowed to profile memory with finer granularity steps than 4 MiB. Added sampling memory profiler to capture random allocations/deallocations. #10598 (alexey-milovidov).
  • SimpleAggregateFunction now also supports sumMap. #10000 (Ildus Kurbangaliev).
  • Support ALTER RENAME COLUMN for the distributed table engine. Continuation of #10727. Fixes #10747. #10887 (alesapin).

Bug Fix

  • Fix UBSan report in Decimal parse. This fixes #7540. #10512 (alexey-milovidov).
  • Fix potential floating point exception when parsing DateTime64. This fixes #11374. #11875 (alexey-milovidov).
  • Fix rare crash caused by using Nullable column in prewhere condition. #11895 #11608 #11869 (Nikolai Kochetov).
  • Don't allow arrayJoin inside higher order functions. It was leading to broken protocol synchronization. This closes #3933. #11846 (alexey-milovidov).
  • Fix wrong result of comparison of FixedString with constant String. This fixes #11393. This bug appeared in version 20.4. #11828 (alexey-milovidov).
  • Fix wrong result for if with NULLs in condition. #11807 (Artem Zuikov).
  • Fix using too many threads for queries. #11788 (Nikolai Kochetov).
  • Fixed Scalar doesn't exist exception when using WITH <scalar subquery> ... in SELECT ... FROM merge_tree_table ... #11621. #11767 (Amos Bird).
  • Fix unexpected behaviour of queries like SELECT *, xyz.* which were success while an error expected. #11753 (hexiaoting).
  • Now replicated fetches will be cancelled during metadata alter. #11744 (alesapin).
  • Parse metadata stored in zookeeper before checking for equality. #11739 (Azat Khuzhin).
  • Fixed LOGICAL_ERROR caused by wrong type deduction of complex literals in Values input format. #11732 (tavplubix).
  • Fix ORDER BY ... WITH FILL over const columns. #11697 (Anton Popov).
  • Fix very rare race condition in SYSTEM SYNC REPLICA. If the replicated table is created and at the same time from the separate connection another client is issuing SYSTEM SYNC REPLICA command on that table (this is unlikely, because another client should be aware that the table is created), it's possible to get nullptr dereference. #11691 (alexey-milovidov).
  • Pass proper timeouts when communicating with XDBC bridge. Recently timeouts were not respected when checking bridge liveness and receiving meta info. #11690 (alexey-milovidov).
  • Fix LIMIT n WITH TIES usage together with ORDER BY statement, which contains aliases. #11689 (Anton Popov).
  • Fix possible Pipeline stuck for selects with parallel FINAL. Fixes #11636. #11682 (Nikolai Kochetov).
  • Fix error which leads to an incorrect state of system.mutations. It may show that whole mutation is already done but the server still has MUTATE_PART tasks in the replication queue and tries to execute them. This fixes #11611. #11681 (alesapin).
  • Fix syntax hilite in CREATE USER query. #11664 (alexey-milovidov).
  • Add support for regular expressions with case-insensitive flags. This fixes #11101 and fixes #11506. #11649 (alexey-milovidov).
  • Remove trivial count query optimization if row-level security is set. In previous versions the user get total count of records in a table instead filtered. This fixes #11352. #11644 (alexey-milovidov).
  • Fix bloom filters for String (data skipping indices). #11638 (Azat Khuzhin).
  • Without -q option the database does not get created at startup. #11604 (giordyb).
  • Fix error Block structure mismatch for queries with sampling reading from Buffer table. #11602 (Nikolai Kochetov).
  • Fix wrong exit code of the clickhouse-client, when exception.code() % 256 == 0. #11601 (filimonov).
  • Fix race conditions in CREATE/DROP of different replicas of ReplicatedMergeTree. Continue to work if the table was not removed completely from ZooKeeper or not created successfully. This fixes #11432. #11592 (alexey-milovidov).
  • Fix trivial error in log message about "Mark cache size was lowered" at server startup. This closes #11399. #11589 (alexey-milovidov).
  • Fix error Size of offsets doesn't match size of column for queries with PREWHERE column in (subquery) and ARRAY JOIN. #11580 (Nikolai Kochetov).
  • Fixed rare segfault in SHOW CREATE TABLE Fixes #11490. #11579 (tavplubix).
  • All queries in HTTP session have had the same query_id. It is fixed. #11578 (tavplubix).
  • Now clickhouse-server docker container will prefer IPv6 checking server aliveness. #11550 (Ivan Starkov).
  • Fix the error Data compressed with different methods that can happen if min_bytes_to_use_direct_io is enabled and PREWHERE is active and using SAMPLE or high number of threads. This fixes #11539. #11540 (alexey-milovidov).
  • Fix shard_num/replica_num for <node> (breaks use_compact_format_in_distributed_parts_names). #11528 (Azat Khuzhin).
  • Fix async INSERT into Distributed for prefer_localhost_replica=0 and w/o internal_replication. #11527 (Azat Khuzhin).
  • Fix memory leak when exception is thrown in the middle of aggregation with -State functions. This fixes #8995. #11496 (alexey-milovidov).
  • Fix Pipeline stuck exception for INSERT SELECT FINAL where SELECT (max_threads>1) has multiple streams but INSERT has only one (max_insert_threads==0). #11455 (Azat Khuzhin).
  • Fix wrong result in queries like select count() from t, u. #11454 (Artem Zuikov).
  • Fix return compressed size for codecs. #11448 (Nikolai Kochetov).
  • Fix server crash when a column has compression codec with non-literal arguments. Fixes #11365. #11431 (alesapin).
  • Fix potential uninitialized memory read in MergeTree shutdown if table was not created successfully. #11420 (alexey-milovidov).
  • Fix crash in JOIN over LowCarinality(T) and Nullable(T). #11380. #11414 (Artem Zuikov).
  • Fix error code for wrong USING key. #11373. #11404 (Artem Zuikov).
  • Fixed geohashesInBox with arguments outside of latitude/longitude range. #11403 (Vasily Nemkov).
  • Better errors for joinGet() functions. #11389 (Artem Zuikov).
  • Fix possible Pipeline stuck error for queries with external sort and limit. Fixes #11359. #11366 (Nikolai Kochetov).
  • Remove redundant lock during parts send in ReplicatedMergeTree. #11354 (alesapin).
  • Fix support for \G (vertical output) in clickhouse-client in multiline mode. This closes #9933. #11350 (alexey-milovidov).
  • Fix potential segfault when using Lazy database. #11348 (alexey-milovidov).
  • Fix crash in direct selects from Join table engine (without JOIN) and wrong nullability. #11340 (Artem Zuikov).
  • Fix crash in quantilesExactWeightedArray. #11337 (Nikolai Kochetov).
  • Now merges stopped before change metadata in ALTER queries. #11335 (alesapin).
  • Make writing to MATERIALIZED VIEW with setting parallel_view_processing = 1 parallel again. Fixes #10241. #11330 (Nikolai Kochetov).
  • Fix visitParamExtractRaw when extracted JSON has strings with unbalanced { or [. #11318 (Ewout).
  • Fix very rare race condition in ThreadPool. #11314 (alexey-milovidov).
  • Fix insignificant data race in clickhouse-copier. Found by integration tests. #11313 (alexey-milovidov).
  • Fix potential uninitialized memory in conversion. Example: SELECT toIntervalSecond(now64()). #11311 (alexey-milovidov).
  • Fix the issue when index analysis cannot work if a table has Array column in primary key and if a query is filtering by this column with empty or notEmpty functions. This fixes #11286. #11303 (alexey-milovidov).
  • Fix bug when query speed estimation can be incorrect and the limit of min_execution_speed may not work or work incorrectly if the query is throttled by max_network_bandwidth, max_execution_speed or priority settings. Change the default value of timeout_before_checking_execution_speed to non-zero, because otherwise the settings min_execution_speed and max_execution_speed have no effect. This fixes #11297. This fixes #5732. This fixes #6228. Usability improvement: avoid concatenation of exception message with progress bar in clickhouse-client. #11296 (alexey-milovidov).
  • Fix crash when SET DEFAULT ROLE is called with wrong arguments. This fixes #10586. #11278 (Vitaly Baranov).
  • Fix crash while reading malformed data in Protobuf format. This fixes #5957, fixes #11203. #11258 (Vitaly Baranov).
  • Fixed a bug when cache dictionary could return default value instead of normal (when there are only expired keys). This affects only string fields. #11233 (Nikita Mikhaylov).
  • Fix error Block structure mismatch in QueryPipeline while reading from VIEW with constants in inner query. Fixes #11181. #11205 (Nikolai Kochetov).
  • Fix possible exception Invalid status for associated output. #11200 (Nikolai Kochetov).
  • Now primary.idx will be checked if it's defined in CREATE query. #11199 (alesapin).
  • Fix possible error Cannot capture column for higher-order functions with Array(Array(LowCardinality)) captured argument. #11185 (Nikolai Kochetov).
  • Fixed S3 globbing which could fail in case of more than 1000 keys and some backends. #11179 (Vladimir Chebotarev).
  • If data skipping index is dependent on columns that are going to be modified during background merge (for SummingMergeTree, AggregatingMergeTree as well as for TTL GROUP BY), it was calculated incorrectly. This issue is fixed by moving index calculation after merge so the index is calculated on merged data. #11162 (Azat Khuzhin).
  • Fix for the hang which was happening sometimes during DROP of table engine=Kafka (or during server restarts). #11145 (filimonov).
  • Fix excessive reserving of threads for simple queries (optimization for reducing the number of threads, which was partly broken after changes in pipeline). #11114 (Azat Khuzhin).
  • Remove logging from mutation finalization task if nothing was finalized. #11109 (alesapin).
  • Fixed deadlock during server startup after update with changes in structure of system log tables. #11106 (alesapin).
  • Fixed memory leak in registerDiskS3. #11074 (Pavel Kovalenko).
  • Fix error No such name in Block::erase() when JOIN appears with PREWHERE or optimize_move_to_prewhere makes PREWHERE from WHERE. #11051 (Artem Zuikov).
  • Fixes the potential missed data during termination of Kafka engine table. #11048 (filimonov).
  • Fixed parseDateTime64BestEffort argument resolution bugs. #10925. #11038 (Vasily Nemkov).
  • Now it's possible to ADD/DROP and RENAME the same one column in a single ALTER query. Exception message for simultaneous MODIFY and RENAME became more clear. Partially fixes #10669. #11037 (alesapin).
  • Fixed parsing of S3 URLs. #11036 (Vladimir Chebotarev).
  • Fix memory tracking for two-level GROUP BY when there is a LIMIT. #11022 (Azat Khuzhin).
  • Fix very rare potential use-after-free error in MergeTree if table was not created successfully. #10986 (alexey-milovidov).
  • Fix metadata (relative path for rename) and data (relative path for symlink) handling for Atomic database. #10980 (Azat Khuzhin).
  • Fix server crash on concurrent ALTER and DROP DATABASE queries with Atomic database engine. #10968 (tavplubix).
  • Fix incorrect raw data size in method getRawData(). #10964 (Igr).
  • Fix incompatibility of two-level aggregation between versions 20.1 and earlier. This incompatibility happens when different versions of ClickHouse are used on initiator node and remote nodes and the size of GROUP BY result is large and aggregation is performed by a single String field. It leads to several unmerged rows for a single key in result. #10952 (alexey-milovidov).
  • Avoid sending partially written files by the DistributedBlockOutputStream. #10940 (Azat Khuzhin).
  • Fix crash in SELECT count(notNullIn(NULL, [])). #10920 (Nikolai Kochetov).
  • Fix for the hang which was happening sometimes during DROP of table engine=Kafka (or during server restarts). #10910 (filimonov).
  • Now it's possible to execute multiple ALTER RENAME like a TO b, c TO a. #10895 (alesapin).
  • Fix possible race which could happen when you get result from aggregate function state from multiple thread for the same column. The only way (which I found) it can happen is when you use finalizeAggregation function while reading from table with Memory engine which stores AggregateFunction state for quanite* function. #10890 (Nikolai Kochetov).
  • Fix backward compatibility with tuples in Distributed tables. #10889 (Anton Popov).
  • Fix SIGSEGV in StringHashTable (if such key does not exist). #10870 (Azat Khuzhin).
  • Fixed WATCH hangs after LiveView table was dropped from database with Atomic engine. #10859 (tavplubix).
  • Fixed bug in ReplicatedMergeTree which might cause some ALTER on OPTIMIZE query to hang waiting for some replica after it become inactive. #10849 (tavplubix).
  • Now constraints are updated if the column participating in CONSTRAINT expression was renamed. Fixes #10844. #10847 (alesapin).
  • Fix potential read of uninitialized memory in cache dictionary. #10834 (alexey-milovidov).
  • Fix columns order after Block::sortColumns() (also add a test that shows that it affects some real use case - Buffer engine). #10826 (Azat Khuzhin).
  • Fix the issue with ODBC bridge when no quoting of identifiers is requested. This fixes #7984. #10821 (alexey-milovidov).
  • Fix UBSan and MSan report in DateLUT. #10798 (alexey-milovidov).
  • Make use of src_type for correct type conversion in key conditions. Fixes #6287. #10791 (Andrew Onyshchuk).
  • Get rid of old libunwind patches. 500aa22791 (r39048012) This allows to disable -fno-omit-frame-pointer in clang builds that improves performance at least by 1% in average. #10761 (Amos Bird).
  • Fix avgWeighted when using floating-point weight over multiple shards. #10758 (Baudouin Giard).
  • Fix parallel_view_processing behavior. Now all insertions into MATERIALIZED VIEW without exception should be finished if exception happened. Fixes #10241. #10757 (Nikolai Kochetov).
  • Fix combinator -OrNull and -OrDefault when combined with -State. #10741 (hcz).
  • Fix crash in generateRandom with nested types. Fixes #10583. #10734 (Nikolai Kochetov).
  • Fix data corruption for LowCardinality(FixedString) key column in SummingMergeTree which could have happened after merge. Fixes #10489. #10721 (Nikolai Kochetov).
  • Fix usage of primary key wrapped into a function with 'FINAL' modifier and 'ORDER BY' optimization. #10715 (Anton Popov).
  • Fix possible buffer overflow in function h3EdgeAngle. #10711 (alexey-milovidov).
  • Fix disappearing totals. Totals could have being filtered if query had had join or subquery with external where condition. Fixes #10674. #10698 (Nikolai Kochetov).
  • Fix atomicity of HTTP insert. This fixes #9666. #10687 (Andrew Onyshchuk).
  • Fix multiple usages of IN operator with the identical set in one query. #10686 (Anton Popov).
  • Fixed bug, which causes http requests stuck on client close when readonly=2 and cancel_http_readonly_queries_on_client_close=1. Fixes #7939, #7019, #7736, #7091. #10684 (tavplubix).
  • Fix order of parameters in AggregateTransform constructor. #10667 (palasonic1).
  • Fix the lack of parallel execution of remote queries with distributed_aggregation_memory_efficient enabled. Fixes #10655. #10664 (Nikolai Kochetov).
  • Fix possible incorrect number of rows for queries with LIMIT. Fixes #10566, #10709. #10660 (Nikolai Kochetov).
  • Fix bug which locks concurrent alters when table has a lot of parts. #10659 (alesapin).
  • Fix nullptr dereference in StorageBuffer if server was shutdown before table startup. #10641 (alexey-milovidov).
  • Fix predicates optimization for distributed queries (enable_optimize_predicate_expression=1) for queries with HAVING section (i.e. when filtering on the server initiator is required), by preserving the order of expressions (and this is enough to fix), and also force aggregator use column names over indexes. Fixes: #10613, #11413. #10621 (Azat Khuzhin).
  • Fix optimize_skip_unused_shards with LowCardinality. #10611 (Azat Khuzhin).
  • Fix segfault in StorageBuffer when exception on server startup. Fixes #10550. #10609 (tavplubix).
  • On SYSTEM DROP DNS CACHE query also drop caches, which are used to check if user is allowed to connect from some IP addresses. #10608 (tavplubix).
  • Fixed incorrect scalar results inside inner query of MATERIALIZED VIEW in case if this query contained dependent table. #10603 (Nikolai Kochetov).
  • Fixed handling condition variable for synchronous mutations. In some cases signals to that condition variable could be lost. #10588 (Vladimir Chebotarev).
  • Fixes possible crash createDictionary() is called before loadStoredObject() has finished. #10587 (Vitaly Baranov).
  • Fix error the BloomFilter false positive must be a double number between 0 and 1 #10551. #10569 (Winter Zhang).
  • Fix SELECT of column ALIAS which default expression type different from column type. #10563 (Azat Khuzhin).
  • Implemented comparison between DateTime64 and String values (just like for DateTime). #10560 (Vasily Nemkov).
  • Fix index corruption, which may occur in some cases after merge compact parts into another compact part. #10531 (Anton Popov).
  • Disable GROUP BY sharding_key optimization by default (optimize_distributed_group_by_sharding_key had been introduced and turned of by default, due to trickery of sharding_key analyzing, simple example is if in sharding key) and fix it for WITH ROLLUP/CUBE/TOTALS. #10516 (Azat Khuzhin).
  • Fixes: #10263 (after that PR dist send via INSERT had been postponing on each INSERT) Fixes: #8756 (that PR breaks distributed sends with all of the following conditions met (unlikely setup for now I guess): internal_replication == false, multiple local shards (activates the hardlinking code) and distributed_storage_policy (makes link(2) fails on EXDEV)). #10486 (Azat Khuzhin).
  • Fixed error with "max_rows_to_sort" limit. #10268 (alexey-milovidov).
  • Get dictionary and check access rights only once per each call of any function reading external dictionaries. #10928 (Vitaly Baranov).

Improvement

  • Apply TTL for old data, after ALTER MODIFY TTL query. This behaviour is controlled by setting materialize_ttl_after_modify, which is enabled by default. #11042 (Anton Popov).
  • When parsing C-style backslash escapes in string literals, VALUES and various text formats (this is an extension to SQL standard that is endemic for ClickHouse and MySQL), keep backslash if unknown escape sequence is found (e.g. \% or \w) that will make usage of LIKE and match regular expressions more convenient (it's enough to write name LIKE 'used\_cars' instead of name LIKE 'used\\_cars') and more compatible at the same time. This fixes #10922. #11208 (alexey-milovidov).
  • When reading Decimal value, cut extra digits after point. This behaviour is more compatible with MySQL and PostgreSQL. This fixes #10202. #11831 (alexey-milovidov).
  • Allow to DROP replicated table if the metadata in ZooKeeper was already removed and does not exist (this is also the case when using TestKeeper for testing and the server was restarted). Allow to RENAME replicated table even if there is an error communicating with ZooKeeper. This fixes #10720. #11652 (alexey-milovidov).
  • Slightly improve diagnostic of reading decimal from string. This closes #10202. #11829 (alexey-milovidov).
  • Fix sleep invocation in signal handler. It was sleeping for less amount of time than expected. #11825 (alexey-milovidov).
  • (Only Linux) OS related performance metrics (for CPU and I/O) will work even without CAP_NET_ADMIN capability. #10544 (Alexander Kazakov).
  • Added hostname as an alias to function hostName. This feature was suggested by Victor Tarnavskiy from Yandex.Metrica. #11821 (alexey-milovidov).
  • Added support for distributed DDL (update/delete/drop partition) on cross replication clusters. #11703 (Nikita Mikhaylov).
  • Emit warning instead of error in server log at startup if we cannot listen one of the listen addresses (e.g. IPv6 is unavailable inside Docker). Note that if server fails to listen all listed addresses, it will refuse to startup as before. This fixes #4406. #11687 (alexey-milovidov).
  • Default user and database creation on docker image starting. #10637 (Paramtamtam).
  • When multiline query is printed to server log, the lines are joined. Make it to work correct in case of multiline string literals, identifiers and single-line comments. This fixes #3853. #11686 (alexey-milovidov).
  • Multiple names are now allowed in commands: CREATE USER, CREATE ROLE, ALTER USER, SHOW CREATE USER, SHOW GRANTS and so on. #11670 (Vitaly Baranov).
  • Add support for distributed DDL (UPDATE/DELETE/DROP PARTITION) on cross replication clusters. #11508 (frank lee).
  • Clear password from command line in clickhouse-client and clickhouse-benchmark if the user has specified it with explicit value. This prevents password exposure by ps and similar tools. #11665 (alexey-milovidov).
  • Don't use debug info from ELF file if it doesn't correspond to the running binary. It is needed to avoid printing wrong function names and source locations in stack traces. This fixes #7514. #11657 (alexey-milovidov).
  • Return NULL/zero when value is not parsed completely in parseDateTimeBestEffortOrNull/Zero functions. This fixes #7876. #11653 (alexey-milovidov).
  • Skip empty parameters in requested URL. They may appear when you write http://localhost:8123/?&a=b or http://localhost:8123/?a=b&&c=d. This closes #10749. #11651 (alexey-milovidov).
  • Allow using groupArrayArray and groupUniqArrayArray as SimpleAggregateFunction. #11650 (Volodymyr Kuznetsov).
  • Allow comparison with constant strings by implicit conversions when analysing index conditions on other types. This may close #11630. #11648 (alexey-milovidov).
  • https://github.com/ClickHouse/ClickHouse/pull/7572#issuecomment-642815377 Support config default HTTPHandlers. #11628 (Winter Zhang).
  • Make more input formats to work with Kafka engine. Fix the issue with premature flushes. Fix the performance issue when kafka_num_consumers is greater than number of partitions in topic. #11599 (filimonov).
  • Improve multiple_joins_rewriter_version=2 logic. Fix unknown columns error for lambda aliases. #11587 (Artem Zuikov).
  • Better exception message when cannot parse columns declaration list. This closes #10403. #11537 (alexey-milovidov).
  • Improve enable_optimize_predicate_expression=1 logic for VIEW. #11513 (Artem Zuikov).
  • Adding support for PREWHERE in live view tables. #11495 (vzakaznikov).
  • Automatically update DNS cache, which is used to check if user is allowed to connect from an address. #11487 (tavplubix).
  • OPTIMIZE FINAL will force merge even if concurrent merges are performed. This closes #11309 and closes #11322. #11346 (alexey-milovidov).
  • Suppress output of cancelled queries in clickhouse-client. In previous versions result may continue to print in terminal even after you press Ctrl+C to cancel query. This closes #9473. #11342 (alexey-milovidov).
  • Now history file is updated after each query and there is no race condition if multiple clients use one history file. This fixes #9897. #11453 (Tagir Kuskarov).
  • Better log messages in while reloading configuration. #11341 (alexey-milovidov).
  • Remove trailing whitespaces from formatted queries in clickhouse-client or clickhouse-format in some cases. #11325 (alexey-milovidov).
  • Add setting "output_format_pretty_max_value_width". If value is longer, it will be cut to avoid output of too large values in terminal. This closes #11140. #11324 (alexey-milovidov).
  • Better exception message in case when there is shortage of memory mappings. This closes #11027. #11316 (alexey-milovidov).
  • Support (U)Int8, (U)Int16, Date in ASOF JOIN. #11301 (Artem Zuikov).
  • Support kafka_client_id parameter for Kafka tables. It also changes the default client.id used by ClickHouse when communicating with Kafka to be more verbose and usable. #11252 (filimonov).
  • Keep the value of DistributedFilesToInsert metric on exceptions. In previous versions, the value was set when we are going to send some files, but it is zero, if there was an exception and some files are still pending. Now it corresponds to the number of pending files in filesystem. #11220 (alexey-milovidov).
  • Add support for multi-word data type names (such as DOUBLE PRECISION and CHAR VARYING) for better SQL compatibility. #11214 (Павел Потемкин).
  • Provide synonyms for some data types. #10856 (Павел Потемкин).
  • The query log is now enabled by default. #11184 (Ivan Blinkov).
  • Show authentication type in table system.users and while executing SHOW CREATE USER query. #11080 (Vitaly Baranov).
  • Remove data on explicit DROP DATABASE for Memory database engine. Fixes #10557. #11021 (tavplubix).
  • Set thread names for internal threads of rdkafka library. Make logs from rdkafka available in server logs. #10983 (Azat Khuzhin).
  • Support for unicode whitespaces in queries. This helps when queries are copy-pasted from Word or from web page. This fixes #10896. #10903 (alexey-milovidov).
  • Allow large UInt types as the index in function tupleElement. #10874 (hcz).
  • Respect prefer_localhost_replica/load_balancing on INSERT into Distributed. #10867 (Azat Khuzhin).
  • Introduce min_insert_block_size_rows_for_materialized_views, min_insert_block_size_bytes_for_materialized_views settings. This settings are similar to min_insert_block_size_rows and min_insert_block_size_bytes, but applied only for blocks inserted into MATERIALIZED VIEW. It helps to control blocks squashing while pushing to MVs and avoid excessive memory usage. #10858 (Azat Khuzhin).
  • Get rid of exception from replicated queue during server shutdown. Fixes #10819. #10841 (alesapin).
  • Ensure that varSamp, varPop cannot return negative results due to numerical errors and that stddevSamp, stddevPop cannot be calculated from negative variance. This fixes #10532. #10829 (alexey-milovidov).
  • Better DNS exception message. This fixes #10813. #10828 (alexey-milovidov).
  • Change HTTP response code in case of some parse errors to 400 Bad Request. This fix #10636. #10640 (alexey-milovidov).
  • Print a message if clickhouse-client is newer than clickhouse-server. #10627 (alexey-milovidov).
  • Adding support for INSERT INTO [db.]table WATCH query. #10498 (vzakaznikov).
  • Allow to pass quota_key in clickhouse-client. This closes #10227. #10270 (alexey-milovidov).

Performance Improvement

  • Allow multiple replicas to assign merges, mutations, partition drop, move and replace concurrently. This closes #10367. #11639 (alexey-milovidov) #11795 (alexey-milovidov).
  • Optimization of GROUP BY with respect to table sorting key, enabled with optimize_aggregation_in_order setting. #9113 (dimarub2000).
  • Selects with final are executed in parallel. Added setting max_final_threads to limit the number of threads used. #10463 (Nikolai Kochetov).
  • Improve performance for INSERT queries via INSERT SELECT or INSERT with clickhouse-client when small blocks are generated (typical case with parallel parsing). This fixes #11275. Fix the issue that CONSTRAINTs were not working for DEFAULT fields. This fixes #11273. Fix the issue that CONSTRAINTS were ignored for TEMPORARY tables. This fixes #11274. #11276 (alexey-milovidov).
  • Optimization that eliminates min/max/any aggregators of GROUP BY keys in SELECT section, enabled with optimize_aggregators_of_group_by_keys setting. #11667 (xPoSx). #11806 (Azat Khuzhin).
  • New optimization that takes all operations out of any function, enabled with optimize_move_functions_out_of_any #11529 (Ruslan).
  • Improve performance of clickhouse-client in interactive mode when Pretty formats are used. In previous versions, significant amount of time can be spent calculating visible width of UTF-8 string. This closes #11323. #11323 (alexey-milovidov).
  • Improved performance for queries with ORDER BY and small LIMIT (less, then max_block_size). #11171 (Albert Kidrachev).
  • Add runtime CPU detection to select and dispatch the best function implementation. Add support for codegeneration for multiple targets. This closes #1017. #10058 (DimasKovas).
  • Enable mlock of clickhouse binary by default. It will prevent clickhouse executable from being paged out under high IO load. #11139 (alexey-milovidov).
  • Make queries with sum aggregate function and without GROUP BY keys to run multiple times faster. #10992 (alexey-milovidov).
  • Improving radix sort (used in ORDER BY with simple keys) by removing some redundant data moves. #10981 (Arslan Gumerov).
  • Sort bigger parts of the left table in MergeJoin. Buffer left blocks in memory. Add partial_merge_join_left_table_buffer_bytes setting to manage the left blocks buffers sizes. #10601 (Artem Zuikov).
  • Remove duplicate ORDER BY and DISTINCT from subqueries, this optimization is enabled with optimize_duplicate_order_by_and_distinct #10067 (Mikhail Malafeev).
  • This feature eliminates functions of other keys in GROUP BY section, enabled with optimize_group_by_function_keys #10051 (xPoSx).
  • New optimization that takes arithmetic operations out of aggregate functions, enabled with optimize_arithmetic_operations_in_aggregate_functions #10047 (Ruslan).
  • Use HTTP client for S3 based on Poco instead of curl. This will improve performance and lower memory usage of s3 storage and table functions. #11230 (Pavel Kovalenko).
  • Fix Kafka performance issue related to reschedules based on limits, which were always applied. #11149 (filimonov).
  • Enable percpu_arena:percpu for jemalloc (This will reduce memory fragmentation due to thread pool). #11084 (Azat Khuzhin).
  • Optimize memory usage when reading a response from an S3 HTTP client. #11561 (Pavel Kovalenko).
  • Adjust the default Kafka settings for better performance. #11388 (filimonov).

Experimental Feature

  • Add data type Point (Tuple(Float64, Float64)) and Polygon (Array(Array(Tuple(Float64, Float64))). #10678 (Alexey Ilyukhov).
  • Add's a hasSubstr function that allows for look for subsequences in arrays. Note: this function is likely to be renamed without further notice. #11071 (Ryad Zenine).
  • Added OpenCL support and bitonic sort algorithm, which can be used for sorting integer types of data in single column. Needs to be build with flag -DENABLE_OPENCL=1. For using bitonic sort algorithm instead of others you need to set bitonic_sort for Setting's option special_sort and make sure that OpenCL is available. This feature does not improve performance or anything else, it is only provided as an example and for demonstration purposes. It is likely to be removed in near future if there will be no further development in this direction. #10232 (Ri).

Build/Testing/Packaging Improvement

ClickHouse release v20.4

ClickHouse release v20.4.8.99-stable 2020-08-10

Bug Fix

  • Fixed error in parseDateTimeBestEffort function when unix timestamp was passed as an argument. This fixes #13362. #13441 (alexey-milovidov).
  • Fixed potentially low performance and slightly incorrect result for uniqExact, topK, sumDistinct and similar aggregate functions called on Float types with NaN values. It also triggered assert in debug build. This fixes #12491. #13254 (alexey-milovidov).
  • Fixed function if with nullable constexpr as cond that is not literal NULL. Fixes #12463. #13226 (alexey-milovidov).
  • Fixed assert in arrayElement function in case of array elements are Nullable and array subscript is also Nullable. This fixes #12172. #13224 (alexey-milovidov).
  • Fixed wrong index analysis with functions. It could lead to pruning wrong parts, while reading from MergeTree tables. Fixes #13060. Fixes #12406. #13081 (Anton Popov).
  • Fixed unnecessary limiting for the number of threads for selects from local replica. #12840 (Nikolai Kochetov).
  • Fixed possible extra overflow row in data which could appear for queries WITH TOTALS. #12747 (Nikolai Kochetov).
  • Fixed performance with large tuples, which are interpreted as functions in IN section. The case when user write WHERE x IN tuple(1, 2, ...) instead of WHERE x IN (1, 2, ...) for some obscure reason. #12700 (Anton Popov).
  • Fixed memory tracking for input_format_parallel_parsing (by attaching thread to group). #12672 (Azat Khuzhin).
  • Fixed #12293 allow push predicate when subquery contains with clause. #12663 (Winter Zhang).
  • Fixed #10572 fix bloom filter index with const expression. #12659 (Winter Zhang).
  • Fixed SIGSEGV in StorageKafka when broker is unavailable (and not only). #12658 (Azat Khuzhin).
  • Added support for function if with Array(UUID) arguments. This fixes #11066. #12648 (alexey-milovidov).
  • Fixed race condition in external dictionaries with cache layout which can lead server crash. #12566 (alesapin).
  • Removed data for Distributed tables (blocks from async INSERTs) on DROP TABLE. #12556 (Azat Khuzhin).
  • Fixed bug which lead to broken old parts after ALTER DELETE query when enable_mixed_granularity_parts=1. Fixes #12536. #12543 (alesapin).
  • Better exception for function in with invalid number of arguments. #12529 (Anton Popov).
  • Fixed performance issue, while reading from compact parts. #12492 (Anton Popov).
  • Fixed crash in JOIN with dictionary when we are joining over expression of dictionary key: t JOIN dict ON expr(dict.id) = t.id. Disable dictionary join optimisation for this case. #12458 (Artem Zuikov).
  • Fixed possible segfault if StorageMerge. Closes #12054. #12401 (tavplubix).
  • Fixed order of columns in WITH FILL modifier. Previously order of columns of ORDER BY statement wasn't respected. #12306 (Anton Popov).
  • Avoid "bad cast" exception when there is an expression that filters data by virtual columns (like _table in Merge tables) or by "index" columns in system tables such as filtering by database name when querying from system.tables, and this expression returns Nullable type. This fixes #12166. #12305 (alexey-milovidov).
  • Show error after TrieDictionary failed to load. #12290 (Vitaly Baranov).
  • The function arrayFill worked incorrectly for empty arrays that may lead to crash. This fixes #12263. #12279 (alexey-milovidov).
  • Implemented conversions to the common type for LowCardinality types. This allows to execute UNION ALL of tables with columns of LowCardinality and other columns. This fixes #8212. This fixes #4342. #12275 (alexey-milovidov).
  • Fixed the behaviour when during multiple sequential inserts in StorageFile header for some special types was written more than once. This fixed #6155. #12197 (Nikita Mikhaylov).
  • Fixed logical functions for UInt8 values when they are not equal to 0 or 1. #12196 (Alexander Kazakov).
  • Cap max_memory_usage* limits to the process resident memory. #12182 (Azat Khuzhin).
  • Fixed dictGet arguments check during GROUP BY injective functions elimination. #12179 (Azat Khuzhin).
  • Don't split the dictionary source's table name into schema and table name itself if ODBC connection doesn't support schema. #12165 (Vitaly Baranov).
  • Fixed wrong logic in ALTER DELETE that leads to deleting of records when condition evaluates to NULL. This fixes #9088. This closes #12106. #12153 (alexey-milovidov).
  • Fixed transform of query to send to external DBMS (e.g. MySQL, ODBC) in presense of aliases. This fixes #12032. #12151 (alexey-milovidov).
  • Fixed potential overflow in integer division. This fixes #12119. #12140 (alexey-milovidov).
  • Fixed potential infinite loop in greatCircleDistance, geoDistance. This fixes #12117. #12137 (alexey-milovidov).
  • Normalize "pid" file handling. In previous versions the server may refuse to start if it was killed without proper shutdown and if there is another process that has the same pid as previously runned server. Also pid file may be removed in unsuccessful server startup even if there is another server running. This fixes #3501. #12133 (alexey-milovidov).
  • Fixed handling dependency of table with ENGINE=Dictionary on dictionary. This fixes #10994. This fixes #10397. #12116 (Vitaly Baranov).
  • Fixed performance for selects with UNION caused by wrong limit for the total number of threads. Fixes #12030. #12103 (Nikolai Kochetov).
  • Fixed segfault with -StateResample combinators. #12092 (Anton Popov).
  • Fixed empty result_rows and result_bytes metrics in system.quey_log for selects. Fixes #11595. #12089 (Nikolai Kochetov).
  • Fixed unnecessary limiting the number of threads for selects from VIEW. Fixes #11937. #12085 (Nikolai Kochetov).
  • Fixed possible crash while using wrong type for PREWHERE. Fixes #12053, #12060. #12060 (Nikolai Kochetov).
  • Fixed error Expected single dictionary argument for function for function defaultValueOfArgumentType with LowCardinality type. Fixes #11808. #12056 (Nikolai Kochetov).
  • Fixed error Cannot capture column for higher-order functions with Tuple(LowCardinality) argument. Fixes #9766. #12055 (Nikolai Kochetov).
  • Parse tables metadata in parallel when loading database. This fixes slow server startup when there are large number of tables. #12045 (tavplubix).
  • Make topK aggregate function return Enum for Enum types. This fixes #3740. #12043 (alexey-milovidov).
  • Fixed constraints check if constraint is a constant expression. This fixes #11360. #12042 (alexey-milovidov).
  • Fixed incorrect comparison of tuples with Nullable columns. Fixes #11985. #12039 (Nikolai Kochetov).
  • Fixed calculation of access rights when allow_introspection_functions=0. #12031 (Vitaly Baranov).
  • Fixed wrong result and potential crash when invoking function if with arguments of type FixedString with different sizes. This fixes #11362. #12021 (alexey-milovidov).
  • A query with function neighbor as the only returned expression may return empty result if the function is called with offset -9223372036854775808. This fixes #11367. #12019 (alexey-milovidov).
  • Fixed calculation of access rights when allow_ddl=0. #12015 (Vitaly Baranov).
  • Fixed potential array size overflow in generateRandom that may lead to crash. This fixes #11371. #12013 (alexey-milovidov).
  • Fixed potential floating point exception. This closes #11378. #12005 (alexey-milovidov).
  • Fixed wrong setting name in log message at server startup. #11997 (alexey-milovidov).
  • Fixed Query parameter was not set in Values format. Fixes #11918. #11936 (tavplubix).
  • Keep aliases for substitutions in query (parametrized queries). This fixes #11914. #11916 (alexey-milovidov).
  • Fixed bug with no moves when changing storage policy from default one. #11893 (Vladimir Chebotarev).
  • Fixed potential floating point exception when parsing DateTime64. This fixes #11374. #11875 (alexey-milovidov).
  • Fixed memory accounting via HTTP interface (can be significant with wait_end_of_query=1). #11840 (Azat Khuzhin).
  • Parse metadata stored in zookeeper before checking for equality. #11739 (Azat Khuzhin).

Performance Improvement

  • Index not used for IN operator with literals, performance regression introduced around v19.3. This fixes #10574. #12062 (nvartolomei).

Build/Testing/Packaging Improvement

  • Install ca-certificates before the first apt-get update in Dockerfile. #12095 (Ivan Blinkov).

ClickHouse release v20.4.6.53-stable 2020-06-25

Bug Fix

  • Fix rare crash caused by using Nullable column in prewhere condition. Continuation of #11608. #11869 (Nikolai Kochetov).
  • Don't allow arrayJoin inside higher order functions. It was leading to broken protocol synchronization. This closes #3933. #11846 (alexey-milovidov).
  • Fix wrong result of comparison of FixedString with constant String. This fixes #11393. This bug appeared in version 20.4. #11828 (alexey-milovidov).
  • Fix wrong result for if() with NULLs in condition. #11807 (Artem Zuikov).
  • Fix using too many threads for queries. #11788 (Nikolai Kochetov).
  • Fix unexpected behaviour of queries like SELECT *, xyz.* which were success while an error expected. #11753 (hexiaoting).
  • Now replicated fetches will be cancelled during metadata alter. #11744 (alesapin).
  • Fixed LOGICAL_ERROR caused by wrong type deduction of complex literals in Values input format. #11732 (tavplubix).
  • Fix ORDER BY ... WITH FILL over const columns. #11697 (Anton Popov).
  • Pass proper timeouts when communicating with XDBC bridge. Recently timeouts were not respected when checking bridge liveness and receiving meta info. #11690 (alexey-milovidov).
  • Fix LIMIT n WITH TIES usage together with ORDER BY statement, which contains aliases. #11689 (Anton Popov).
  • Fix error which leads to an incorrect state of system.mutations. It may show that whole mutation is already done but the server still has MUTATE_PART tasks in the replication queue and tries to execute them. This fixes #11611. #11681 (alesapin).
  • Add support for regular expressions with case-insensitive flags. This fixes #11101 and fixes #11506. #11649 (alexey-milovidov).
  • Remove trivial count query optimization if row-level security is set. In previous versions the user get total count of records in a table instead filtered. This fixes #11352. #11644 (alexey-milovidov).
  • Fix bloom filters for String (data skipping indices). #11638 (Azat Khuzhin).
  • Fix rare crash caused by using Nullable column in prewhere condition. (Probably it is connected with #11572 somehow). #11608 (Nikolai Kochetov).
  • Fix error Block structure mismatch for queries with sampling reading from Buffer table. #11602 (Nikolai Kochetov).
  • Fix wrong exit code of the clickhouse-client, when exception.code() % 256 = 0. #11601 (filimonov).
  • Fix trivial error in log message about "Mark cache size was lowered" at server startup. This closes #11399. #11589 (alexey-milovidov).
  • Fix error Size of offsets doesn't match size of column for queries with PREWHERE column in (subquery) and ARRAY JOIN. #11580 (Nikolai Kochetov).
  • Fixed rare segfault in SHOW CREATE TABLE Fixes #11490. #11579 (tavplubix).
  • All queries in HTTP session have had the same query_id. It is fixed. #11578 (tavplubix).
  • Now clickhouse-server docker container will prefer IPv6 checking server aliveness. #11550 (Ivan Starkov).
  • Fix shard_num/replica_num for <node> (breaks use_compact_format_in_distributed_parts_names). #11528 (Azat Khuzhin).
  • Fix race condition which may lead to an exception during table drop. It's a bit tricky and not dangerous at all. If you want an explanation, just notice me in telegram. #11523 (alesapin).
  • Fix memory leak when exception is thrown in the middle of aggregation with -State functions. This fixes #8995. #11496 (alexey-milovidov).
  • If data skipping index is dependent on columns that are going to be modified during background merge (for SummingMergeTree, AggregatingMergeTree as well as for TTL GROUP BY), it was calculated incorrectly. This issue is fixed by moving index calculation after merge so the index is calculated on merged data. #11162 (Azat Khuzhin).
  • Get rid of old libunwind patches. 500aa22791 (r39048012) This allows to disable -fno-omit-frame-pointer in clang builds that improves performance at least by 1% in average. #10761 (Amos Bird).
  • Fix usage of primary key wrapped into a function with 'FINAL' modifier and 'ORDER BY' optimization. #10715 (Anton Popov).

Build/Testing/Packaging Improvement

ClickHouse release v20.4.5.36-stable 2020-06-10

Bug Fix

  • Fix the error Data compressed with different methods that can happen if min_bytes_to_use_direct_io is enabled and PREWHERE is active and using SAMPLE or high number of threads. This fixes #11539. #11540 (alexey-milovidov).
  • Fix return compressed size for codecs. #11448 (Nikolai Kochetov).
  • Fix server crash when a column has compression codec with non-literal arguments. Fixes #11365. #11431 (alesapin).
  • Fix pointInPolygon with nan as point. Fixes #11375. #11421 (Alexey Ilyukhov).
  • Fix potential uninitialized memory read in MergeTree shutdown if table was not created successfully. #11420 (alexey-milovidov).
  • Fixed geohashesInBox with arguments outside of latitude/longitude range. #11403 (Vasily Nemkov).
  • Fix possible Pipeline stuck error for queries with external sort and limit. Fixes #11359. #11366 (Nikolai Kochetov).
  • Remove redundant lock during parts send in ReplicatedMergeTree. #11354 (alesapin).
  • Fix support for \G (vertical output) in clickhouse-client in multiline mode. This closes #9933. #11350 (alexey-milovidov).
  • Fix potential segfault when using Lazy database. #11348 (alexey-milovidov).
  • Fix crash in quantilesExactWeightedArray. #11337 (Nikolai Kochetov).
  • Now merges stopped before change metadata in ALTER queries. #11335 (alesapin).
  • Make writing to MATERIALIZED VIEW with setting parallel_view_processing = 1 parallel again. Fixes #10241. #11330 (Nikolai Kochetov).
  • Fix visitParamExtractRaw when extracted JSON has strings with unbalanced { or [. #11318 (Ewout).
  • Fix very rare race condition in ThreadPool. #11314 (alexey-milovidov).
  • Fix insignificant data race in clickhouse-copier. Found by integration tests. #11313 (alexey-milovidov).
  • Fix potential uninitialized memory in conversion. Example: SELECT toIntervalSecond(now64()). #11311 (alexey-milovidov).
  • Fix the issue when index analysis cannot work if a table has Array column in primary key and if a query is filtering by this column with empty or notEmpty functions. This fixes #11286. #11303 (alexey-milovidov).
  • Fix bug when query speed estimation can be incorrect and the limit of min_execution_speed may not work or work incorrectly if the query is throttled by max_network_bandwidth, max_execution_speed or priority settings. Change the default value of timeout_before_checking_execution_speed to non-zero, because otherwise the settings min_execution_speed and max_execution_speed have no effect. This fixes #11297. This fixes #5732. This fixes #6228. Usability improvement: avoid concatenation of exception message with progress bar in clickhouse-client. #11296 (alexey-milovidov).
  • Fix crash when SET DEFAULT ROLE is called with wrong arguments. This fixes #10586. #11278 (Vitaly Baranov).
  • Fix crash while reading malformed data in Protobuf format. This fixes #5957, fixes #11203. #11258 (Vitaly Baranov).
  • Fixed a bug when cache-dictionary could return default value instead of normal (when there are only expired keys). This affects only string fields. #11233 (Nikita Mikhaylov).
  • Fix error Block structure mismatch in QueryPipeline while reading from VIEW with constants in inner query. Fixes #11181. #11205 (Nikolai Kochetov).
  • Fix possible exception Invalid status for associated output. #11200 (Nikolai Kochetov).
  • Fix possible error Cannot capture column for higher-order functions with Array(Array(LowCardinality)) captured argument. #11185 (Nikolai Kochetov).
  • Fixed S3 globbing which could fail in case of more than 1000 keys and some backends. #11179 (Vladimir Chebotarev).
  • If data skipping index is dependent on columns that are going to be modified during background merge (for SummingMergeTree, AggregatingMergeTree as well as for TTL GROUP BY), it was calculated incorrectly. This issue is fixed by moving index calculation after merge so the index is calculated on merged data. #11162 (Azat Khuzhin).
  • Fix Kafka performance issue related to reschedules based on limits, which were always applied. #11149 (filimonov).
  • Fix for the hang which was happening sometimes during DROP of table engine=Kafka (or during server restarts). #11145 (filimonov).
  • Fix excessive reserving of threads for simple queries (optimization for reducing the number of threads, which was partly broken after changes in pipeline). #11114 (Azat Khuzhin).
  • Fix predicates optimization for distributed queries (enable_optimize_predicate_expression=1) for queries with HAVING section (i.e. when filtering on the server initiator is required), by preserving the order of expressions (and this is enough to fix), and also force aggregator use column names over indexes. Fixes: #10613, #11413. #10621 (Azat Khuzhin).

Build/Testing/Packaging Improvement

ClickHouse release v20.4.4.18-stable 2020-05-26

No changes compared to v20.4.3.16-stable.

ClickHouse release v20.4.3.16-stable 2020-05-23

Bug Fix

  • Removed logging from mutation finalization task if nothing was finalized. #11109 (alesapin).
  • Fixed memory leak in registerDiskS3. #11074 (Pavel Kovalenko).
  • Fixed the potential missed data during termination of Kafka engine table. #11048 (filimonov).
  • Fixed parseDateTime64BestEffort argument resolution bugs. #11038 (Vasily Nemkov).
  • Fixed very rare potential use-after-free error in MergeTree if table was not created successfully. #10986, #10970 (alexey-milovidov).
  • Fixed metadata (relative path for rename) and data (relative path for symlink) handling for Atomic database. #10980 (Azat Khuzhin).
  • Fixed server crash on concurrent ALTER and DROP DATABASE queries with Atomic database engine. #10968 (tavplubix).
  • Fixed incorrect raw data size in getRawData() method. #10964 (Igr).
  • Fixed incompatibility of two-level aggregation between versions 20.1 and earlier. This incompatibility happens when different versions of ClickHouse are used on initiator node and remote nodes and the size of GROUP BY result is large and aggregation is performed by a single String field. It leads to several unmerged rows for a single key in result. #10952 (alexey-milovidov).
  • Fixed sending partially written files by the DistributedBlockOutputStream. #10940 (Azat Khuzhin).
  • Fixed crash in SELECT count(notNullIn(NULL, [])). #10920 (Nikolai Kochetov).
  • Fixed the hang which was happening sometimes during DROP of Kafka table engine. (or during server restarts). #10910 (filimonov).
  • Fixed the impossibility of executing multiple ALTER RENAME like a TO b, c TO a. #10895 (alesapin).
  • Fixed possible race which could happen when you get result from aggregate function state from multiple thread for the same column. The only way it can happen is when you use finalizeAggregation function while reading from table with Memory engine which stores AggregateFunction state for quantile* function. #10890 (Nikolai Kochetov).
  • Fixed backward compatibility with tuples in Distributed tables. #10889 (Anton Popov).
  • Fixed SIGSEGV in StringHashTable if such a key does not exist. #10870 (Azat Khuzhin).
  • Fixed WATCH hangs after LiveView table was dropped from database with Atomic engine. #10859 (tavplubix).
  • Fixed bug in ReplicatedMergeTree which might cause some ALTER on OPTIMIZE query to hang waiting for some replica after it become inactive. #10849 (tavplubix).
  • Now constraints are updated if the column participating in CONSTRAINT expression was renamed. Fixes #10844. #10847 (alesapin).
  • Fixed potential read of uninitialized memory in cache-dictionary. #10834 (alexey-milovidov).
  • Fixed columns order after Block::sortColumns(). #10826 (Azat Khuzhin).
  • Fixed the issue with ODBC bridge when no quoting of identifiers is requested. Fixes #7984. #10821 (alexey-milovidov).
  • Fixed UBSan and MSan report in DateLUT. #10798 (alexey-milovidov).
  • Fixed incorrect type conversion in key conditions. Fixes #6287. #10791 (Andrew Onyshchuk).
  • Fixed parallel_view_processing behavior. Now all insertions into MATERIALIZED VIEW without exception should be finished if exception happened. Fixes #10241. #10757 (Nikolai Kochetov).
  • Fixed combinator -OrNull and -OrDefault when combined with -State. #10741 (hcz).
  • Fixed possible buffer overflow in function h3EdgeAngle. #10711 (alexey-milovidov).
  • Fixed bug which locks concurrent alters when table has a lot of parts. #10659 (alesapin).
  • Fixed nullptr dereference in StorageBuffer if server was shutdown before table startup. #10641 (alexey-milovidov).
  • Fixed optimize_skip_unused_shards with LowCardinality. #10611 (Azat Khuzhin).
  • Fixed handling condition variable for synchronous mutations. In some cases signals to that condition variable could be lost. #10588 (Vladimir Chebotarev).
  • Fixed possible crash when createDictionary() is called before loadStoredObject() has finished. #10587 (Vitaly Baranov).
  • Fixed SELECT of column ALIAS which default expression type different from column type. #10563 (Azat Khuzhin).
  • Implemented comparison between DateTime64 and String values. #10560 (Vasily Nemkov).
  • Disable GROUP BY sharding_key optimization by default (optimize_distributed_group_by_sharding_key had been introduced and turned of by default, due to trickery of sharding_key analyzing, simple example is if in sharding key) and fix it for WITH ROLLUP/CUBE/TOTALS. #10516 (Azat Khuzhin).
  • Fixed #10263. #10486 (Azat Khuzhin).
  • Added tests about max_rows_to_sort setting. #10268 (alexey-milovidov).
  • Added backward compatibility for create bloom filter index. #10551. #10569 (Winter Zhang).

ClickHouse release v20.4.2.9, 2020-05-12

Backward Incompatible Change

  • System tables (e.g. system.query_log, system.trace_log, system.metric_log) are using compact data part format for parts smaller than 10 MiB in size. Compact data part format is supported since version 20.3. If you are going to downgrade to version less than 20.3, you should manually delete table data for system logs in /var/lib/clickhouse/data/system/.
  • When string comparison involves FixedString and compared arguments are of different sizes, do comparison as if smaller string is padded to the length of the larger. This is intented for SQL compatibility if we imagine that FixedString data type corresponds to SQL CHAR. This closes #9272. #10363 (alexey-milovidov)
  • Make SHOW CREATE TABLE multiline. Now it is more readable and more like MySQL. #10049 (Azat Khuzhin)
  • Added a setting validate_polygons that is used in pointInPolygon function and enabled by default. #9857 (alexey-milovidov)

New Feature

  • Add support for secured connection from ClickHouse to Zookeeper #10184 (Konstantin Lebedev)
  • Support custom HTTP handlers. See #5436 for description. #7572 (Winter Zhang)
  • Add MessagePack Input/Output format. #9889 (Kruglov Pavel)
  • Add Regexp input format. #9196 (Kruglov Pavel)
  • Added output format Markdown for embedding tables in markdown documents. #10317 (Kruglov Pavel)
  • Added support for custom settings section in dictionaries. Also fixes issue #2829. #10137 (Artem Streltsov)
  • Added custom settings support in DDL-queries for CREATE DICTIONARY #10465 (Artem Streltsov)
  • Add simple server-wide memory profiler that will collect allocation contexts when server memory usage becomes higher than the next allocation threshold. #10444 (alexey-milovidov)
  • Add setting always_fetch_merged_part which restrict replica to merge parts by itself and always prefer dowloading from other replicas. #10379 (alesapin)
  • Add function JSONExtractKeysAndValuesRaw which extracts raw data from JSON objects #10378 (hcz)
  • Add memory usage from OS to system.asynchronous_metrics. #10361 (alexey-milovidov)
  • Added generic variants for functions least and greatest. Now they work with arbitrary number of arguments of arbitrary types. This fixes #4767 #10318 (alexey-milovidov)
  • Now ClickHouse controls timeouts of dictionary sources on its side. Two new settings added to cache dictionary configuration: strict_max_lifetime_seconds, which is max_lifetime by default, and query_wait_timeout_milliseconds, which is one minute by default. The first settings is also useful with allow_read_expired_keys settings (to forbid reading very expired keys). #10337 (Nikita Mikhaylov)
  • Add log_queries_min_type to filter which entries will be written to query_log #10053 (Azat Khuzhin)
  • Added function isConstant. This function checks whether its argument is constant expression and returns 1 or 0. It is intended for development, debugging and demonstration purposes. #10198 (alexey-milovidov)
  • add joinGetOrNull to return NULL when key is missing instead of returning the default value. #10094 (Amos Bird)
  • Consider NULL to be equal to NULL in IN operator, if the option transform_null_in is set. #10085 (achimbab)
  • Add ALTER TABLE ... RENAME COLUMN for MergeTree table engines family. #9948 (alesapin)
  • Support parallel distributed INSERT SELECT. #9759 (vxider)
  • Add ability to query Distributed over Distributed (w/o distributed_group_by_no_merge) ... #9923 (Azat Khuzhin)
  • Add function arrayReduceInRanges which aggregates array elements in given ranges. #9598 (hcz)
  • Add Dictionary Status on prometheus exporter. #9622 (Guillaume Tassery)
  • Add function arrayAUC #8698 (taiyang-li)
  • Support DROP VIEW statement for better TPC-H compatibility. #9831 (Amos Bird)
  • Add 'strict_order' option to windowFunnel() #9773 (achimbab)
  • Support DATE and TIMESTAMP SQL operators, e.g. SELECT date '2001-01-01' #9691 (Artem Zuikov)

Experimental Feature

  • Added experimental database engine Atomic. It supports non-blocking DROP and RENAME TABLE queries and atomic EXCHANGE TABLES t1 AND t2 query #7512 (tavplubix)
  • Initial support for ReplicatedMergeTree over S3 (it works in suboptimal way) #10126 (Pavel Kovalenko)

Bug Fix

  • Fixed incorrect scalar results inside inner query of MATERIALIZED VIEW in case if this query contained dependent table #10603 (Nikolai Kochetov)
  • Fixed bug, which caused HTTP requests to get stuck on client closing connection when readonly=2 and cancel_http_readonly_queries_on_client_close=1. #10684 (tavplubix)
  • Fix segfault in StorageBuffer when exception is thrown on server startup. Fixes #10550 #10609 (tavplubix)
  • The querySYSTEM DROP DNS CACHE now also drops caches used to check if user is allowed to connect from some IP addresses #10608 (tavplubix)
  • Fix usage of multiple IN operators with an identical set in one query. Fixes #10539 #10686 (Anton Popov)
  • Fix crash in generateRandom with nested types. Fixes #10583. #10734 (Nikolai Kochetov)
  • Fix data corruption for LowCardinality(FixedString) key column in SummingMergeTree which could have happened after merge. Fixes #10489. #10721 (Nikolai Kochetov)
  • Fix logic for aggregation_memory_efficient_merge_threads setting. #10667 (palasonic1)
  • Fix disappearing totals. Totals could have being filtered if query had JOIN or subquery with external WHERE condition. Fixes #10674 #10698 (Nikolai Kochetov)
  • Fix the lack of parallel execution of remote queries with distributed_aggregation_memory_efficient enabled. Fixes #10655 #10664 (Nikolai Kochetov)
  • Fix possible incorrect number of rows for queries with LIMIT. Fixes #10566, #10709 #10660 (Nikolai Kochetov)
  • Fix index corruption, which may occur in some cases after merging compact parts into another compact part. #10531 (Anton Popov)
  • Fix the situation, when mutation finished all parts, but hung up in is_done=0. #10526 (alesapin)
  • Fix overflow at beginning of unix epoch for timezones with fractional offset from UTC. Fixes #9335. #10513 (alexey-milovidov)
  • Better diagnostics for input formats. Fixes #10204 #10418 (tavplubix)
  • Fix numeric overflow in simpleLinearRegression() over large integers #10474 (hcz)
  • Fix use-after-free in Distributed shutdown, avoid waiting for sending all batches #10491 (Azat Khuzhin)
  • Add CA certificates to clickhouse-server docker image #10476 (filimonov)
  • Fix a rare endless loop that might have occurred when using the addressToLine function or AggregateFunctionState columns. #10466 (Alexander Kuzmenkov)
  • Handle zookeeper "no node error" during distributed query #10050 (Daniel Chen)
  • Fix bug when server cannot attach table after column's default was altered. #10441 (alesapin)
  • Implicitly cast the default expression type to the column type for the ALIAS columns #10563 (Azat Khuzhin)
  • Don't remove metadata directory if ATTACH DATABASE fails #10442 (Winter Zhang)
  • Avoid dependency on system tzdata. Fixes loading of Africa/Casablanca timezone on CentOS 8. Fixes #10211 #10425 (alexey-milovidov)
  • Fix some issues if data is inserted with quorum and then gets deleted (DROP PARTITION, TTL, etc.). It led to stuck of INSERTs or false-positive exceptions in SELECTs. Fixes #9946 #10188 (Nikita Mikhaylov)
  • Check the number and type of arguments when creating BloomFilter index #9623 #10431 (Winter Zhang)
  • Prefer fallback_to_stale_replicas over skip_unavailable_shards, otherwise when both settings specified and there are no up-to-date replicas the query will fail (patch from @alex-zaitsev ) #10422 (Azat Khuzhin)
  • Fix the issue when a query with ARRAY JOIN, ORDER BY and LIMIT may return incomplete result. Fixes #10226. #10427 (Vadim Plakhtinskiy)
  • Add database name to dictionary name after DETACH/ATTACH. Fixes system.dictionaries table and SYSTEM RELOAD query #10415 (Azat Khuzhin)
  • Fix possible incorrect result for extremes in processors pipeline. #10131 (Nikolai Kochetov)
  • Fix possible segfault when the setting distributed_group_by_no_merge is enabled (introduced in 20.3.7.46 by #10131). #10399 (Nikolai Kochetov)
  • Fix wrong flattening of Array(Tuple(...)) data types. Fixes #10259 #10390 (alexey-milovidov)
  • Fix column names of constants inside JOIN that may clash with names of constants outside of JOIN #9950 (Alexander Kuzmenkov)
  • Fix order of columns after Block::sortColumns() #10826 (Azat Khuzhin)
  • Fix possible Pipeline stuck error in ConcatProcessor which may happen in remote query. #10381 (Nikolai Kochetov)
  • Don't make disk reservations for aggregations. Fixes #9241 #10375 (Azat Khuzhin)
  • Fix wrong behaviour of datetime functions for timezones that has altered between positive and negative offsets from UTC (e.g. Pacific/Kiritimati). Fixes #7202 #10369 (alexey-milovidov)
  • Avoid infinite loop in dictIsIn function. Fixes #515 #10365 (alexey-milovidov)
  • Disable GROUP BY sharding_key optimization by default and fix it for WITH ROLLUP/CUBE/TOTALS #10516 (Azat Khuzhin)
  • Check for error code when checking parts and don't mark part as broken if the error is like "not enough memory". Fixes #6269 #10364 (alexey-milovidov)
  • Show information about not loaded dictionaries in system tables. #10234 (Vitaly Baranov)
  • Fix nullptr dereference in StorageBuffer if server was shutdown before table startup. #10641 (alexey-milovidov)
  • Fixed DROP vs OPTIMIZE race in ReplicatedMergeTree. DROP could left some garbage in replica path in ZooKeeper if there was concurrent OPTIMIZE query. #10312 (tavplubix)
  • Fix 'Logical error: CROSS JOIN has expressions' error for queries with comma and names joins mix. Fixes #9910 #10311 (Artem Zuikov)
  • Fix queries with max_bytes_before_external_group_by. #10302 (Artem Zuikov)
  • Fix the issue with limiting maximum recursion depth in parser in certain cases. This fixes #10283 This fix may introduce minor incompatibility: long and deep queries via clickhouse-client may refuse to work, and you should adjust settings max_query_size and max_parser_depth accordingly. #10295 (alexey-milovidov)
  • Allow to use count(*) with multiple JOINs. Fixes #9853 #10291 (Artem Zuikov)
  • Fix error Pipeline stuck with max_rows_to_group_by and group_by_overflow_mode = 'break'. #10279 (Nikolai Kochetov)
  • Fix 'Cannot add column' error while creating range_hashed dictionary using DDL query. Fixes #10093. #10235 (alesapin)
  • Fix rare possible exception Cannot drain connections: cancel first. #10239 (Nikolai Kochetov)
  • Fixed bug where ClickHouse would throw "Unknown function lambda." error message when user tries to run ALTER UPDATE/DELETE on tables with ENGINE = Replicated*. Check for nondeterministic functions now handles lambda expressions correctly. #10237 (Alexander Kazakov)
  • Fixed reasonably rare segfault in StorageSystemTables that happens when SELECT ... FROM system.tables is run on a database with Lazy engine. #10209 (Alexander Kazakov)
  • Fix possible infinite query execution when the query actually should stop on LIMIT, while reading from infinite source like system.numbers or system.zeros. #10206 (Nikolai Kochetov)
  • Fixed "generateRandom" function for Date type. This fixes #9973. Fix an edge case when dates with year 2106 are inserted to MergeTree tables with old-style partitioning but partitions are named with year 1970. #10218 (alexey-milovidov)
  • Convert types if the table definition of a View does not correspond to the SELECT query. This fixes #10180 and #10022 #10217 (alexey-milovidov)
  • Fix parseDateTimeBestEffort for strings in RFC-2822 when day of week is Tuesday or Thursday. This fixes #10082 #10214 (alexey-milovidov)
  • Fix column names of constants inside JOIN that may clash with names of constants outside of JOIN. #10207 (alexey-milovidov)
  • Fix move-to-prewhere optimization in presense of arrayJoin functions (in certain cases). This fixes #10092 #10195 (alexey-milovidov)
  • Fix issue with separator appearing in SCRAMBLE for native mysql-connector-java (JDBC) #10140 (BohuTANG)
  • Fix using the current database for an access checking when the database isn't specified. #10192 (Vitaly Baranov)
  • Fix ALTER of tables with compact parts. #10130 (Anton Popov)
  • Add the ability to relax the restriction on non-deterministic functions usage in mutations with allow_nondeterministic_mutations setting. #10186 (filimonov)
  • Fix DROP TABLE invoked for dictionary #10165 (Azat Khuzhin)
  • Convert blocks if structure does not match when doing INSERT into Distributed table #10135 (Azat Khuzhin)
  • The number of rows was logged incorrectly (as sum across all parts) when inserted block is split by parts with partition key. #10138 (alexey-milovidov)
  • Add some arguments check and support identifier arguments for MySQL Database Engine #10077 (Winter Zhang)
  • Fix incorrect index_granularity_bytes check while creating new replica. Fixes #10098. #10121 (alesapin)
  • Fix bug in CHECK TABLE query when table contain skip indices. #10068 (alesapin)
  • Fix Distributed-over-Distributed with the only one shard in a nested table #9997 (Azat Khuzhin)
  • Fix possible rows loss for queries with JOIN and UNION ALL. Fixes #9826, #10113. ... #10099 (Nikolai Kochetov)
  • Fix bug in dictionary when local clickhouse server is used as source. It may caused memory corruption if types in dictionary and source are not compatible. #10071 (alesapin)
  • Fixed replicated tables startup when updating from an old ClickHouse version where /table/replicas/replica_name/metadata node doesn't exist. Fixes #10037. #10095 (alesapin)
  • Fix error Cannot clone block with columns because block has 0 columns ... While executing GroupingAggregatedTransform. It happened when setting distributed_aggregation_memory_efficient was enabled, and distributed query read aggregating data with mixed single and two-level aggregation from different shards. #10063 (Nikolai Kochetov)
  • Fix deadlock when database with materialized view failed attach at start #10054 (Azat Khuzhin)
  • Fix a segmentation fault that could occur in GROUP BY over string keys containing trailing zero bytes (#8636, #8925). ... #10025 (Alexander Kuzmenkov)
  • Fix wrong results of distributed queries when alias could override qualified column name. Fixes #9672 #9714 #9972 (Artem Zuikov)
  • Fix possible deadlock in SYSTEM RESTART REPLICAS #9955 (tavplubix)
  • Fix the number of threads used for remote query execution (performance regression, since 20.3). This happened when query from Distributed table was executed simultaneously on local and remote shards. Fixes #9965 #9971 (Nikolai Kochetov)
  • Fixed DeleteOnDestroy logic in ATTACH PART which could lead to automatic removal of attached part and added few tests #9410 (Vladimir Chebotarev)
  • Fix a bug with ON CLUSTER DDL queries freezing on server startup. #9927 (Gagan Arneja)
  • Fix bug in which the necessary tables weren't retrieved at one of the processing stages of queries to some databases. Fixes #9699. #9949 (achulkov2)
  • Fix 'Not found column in block' error when JOIN appears with TOTALS. Fixes #9839 #9939 (Artem Zuikov)
  • Fix parsing multiple hosts set in the CREATE USER command #9924 (Vitaly Baranov)
  • Fix TRUNCATE for Join table engine (#9917). #9920 (Amos Bird)
  • Fix race condition between drop and optimize in ReplicatedMergeTree. #9901 (alesapin)
  • Fix DISTINCT for Distributed when optimize_skip_unused_shards is set. #9808 (Azat Khuzhin)
  • Fix "scalar doesn't exist" error in ALTERs (#9878). ... #9904 (Amos Bird)
  • Fix error with qualified names in distributed_product_mode=\'local\'. Fixes #4756 #9891 (Artem Zuikov)
  • For INSERT queries shards now do clamp the settings from the initiator to their constraints instead of throwing an exception. This fix allows to send INSERT queries to a shard with another constraints. This change improves fix #9447. #9852 (Vitaly Baranov)
  • Add some retries when commiting offsets to Kafka broker, since it can reject commit if during offsets.commit.timeout.ms there were no enough replicas available for the __consumer_offsets topic #9884 (filimonov)
  • Fix Distributed engine behavior when virtual columns of the underlying table used in WHERE #9847 (Azat Khuzhin)
  • Fixed some cases when timezone of the function argument wasn't used properly. #9574 (Vasily Nemkov)
  • Fix 'Different expressions with the same alias' error when query has PREWHERE and WHERE on distributed table and SET distributed_product_mode = 'local'. #9871 (Artem Zuikov)
  • Fix mutations excessive memory consumption for tables with a composite primary key. This fixes #9850. #9860 (alesapin)
  • Fix calculating grants for introspection functions from the setting allow_introspection_functions. #9840 (Vitaly Baranov)
  • Fix max_distributed_connections (w/ and w/o Processors) #9673 (Azat Khuzhin)
  • Fix possible exception Got 0 in totals chunk, expected 1 on client. It happened for queries with JOIN in case if right joined table had zero rows. Example: select * from system.one t1 join system.one t2 on t1.dummy = t2.dummy limit 0 FORMAT TabSeparated;. Fixes #9777. ... #9823 (Nikolai Kochetov)
  • Fix 'COMMA to CROSS JOIN rewriter is not enabled or cannot rewrite query' error in case of subqueries with COMMA JOIN out of tables lists (i.e. in WHERE). Fixes #9782 #9830 (Artem Zuikov)
  • Fix server crashing when optimize_skip_unused_shards is set and expression for key can't be converted to its field type #9804 (Azat Khuzhin)
  • Fix empty string handling in splitByString. #9767 (hcz)
  • Fix broken ALTER TABLE DELETE COLUMN query for compact parts. #9779 (alesapin)
  • Fixed missing rows_before_limit_at_least for queries over http (with processors pipeline). Fixes #9730 #9757 (Nikolai Kochetov)
  • Fix excessive memory consumption in ALTER queries (mutations). This fixes #9533 and #9670. #9754 (alesapin)
  • Fix possible permanent "Cannot schedule a task" error. #9154 (Azat Khuzhin)
  • Fix bug in backquoting in external dictionaries DDL. Fixes #9619. #9734 (alesapin)
  • Fixed data race in text_log. It does not correspond to any real bug. #9726 (alexey-milovidov)
  • Fix bug in a replication that doesn't allow replication to work if the user has executed mutations on the previous version. This fixes #9645. #9652 (alesapin)
  • Fixed incorrect internal function names for sumKahan and sumWithOverflow. It led to exception while using this functions in remote queries. #9636 (Azat Khuzhin)
  • Add setting use_compact_format_in_distributed_parts_names which allows to write files for INSERT queries into Distributed table with more compact format. This fixes #9647. #9653 (alesapin)
  • Fix RIGHT and FULL JOIN with LowCardinality in JOIN keys. #9610 (Artem Zuikov)
  • Fix possible exceptions Size of filter doesn't match size of column and Invalid number of rows in Chunk in MergeTreeRangeReader. They could appear while executing PREWHERE in some cases. #9612 (Anton Popov)
  • Allow ALTER ON CLUSTER of Distributed tables with internal replication. This fixes #3268 #9617 (shinoi2)
  • Fix issue when timezone was not preserved if you write a simple arithmetic expression like time + 1 (in contrast to an expression like time + INTERVAL 1 SECOND). This fixes #5743 #9323 (alexey-milovidov)

Improvement

  • Use time zone when comparing DateTime with string literal. This fixes #5206. #10515 (alexey-milovidov)
  • Print verbose diagnostic info if Decimal value cannot be parsed from text input format. #10205 (alexey-milovidov)
  • Add tasks/memory metrics for distributed/buffer schedule pools #10449 (Azat Khuzhin)
  • Display result as soon as it's ready for SELECT DISTINCT queries in clickhouse-local and HTTP interface. This fixes #8951 #9559 (alexey-milovidov)
  • Allow to use SAMPLE OFFSET query instead of cityHash64(PRIMARY KEY) % N == n for splitting in clickhouse-copier. To use this feature, pass --experimental-use-sample-offset 1 as a command line argument. #10414 (Nikita Mikhaylov)
  • Allow to parse BOM in TSV if the first column cannot contain BOM in its value. This fixes #10301 #10424 (alexey-milovidov)
  • Add Avro nested fields insert support #10354 (Andrew Onyshchuk)
  • Allowed to alter column in non-modifying data mode when the same type is specified. #10382 (Vladimir Chebotarev)
  • Auto distributed_group_by_no_merge on GROUP BY sharding key (if optimize_skip_unused_shards is set) #10341 (Azat Khuzhin)
  • Optimize queries with LIMIT/LIMIT BY/ORDER BY for distributed with GROUP BY sharding_key #10373 (Azat Khuzhin)
  • Added a setting max_server_memory_usage to limit total memory usage of the server. The metric MemoryTracking is now calculated without a drift. The setting max_memory_usage_for_all_queries is now obsolete and does nothing. This closes #10293. #10362 (alexey-milovidov)
  • Add config option system_tables_lazy_load. If it's set to false, then system tables with logs are loaded at the server startup. Alexander Burmak, Svyatoslav Tkhon Il Pak, #9642 #10359 (alexey-milovidov)
  • Use background thread pool (background_schedule_pool_size) for distributed sends #10263 (Azat Khuzhin)
  • Use background thread pool for background buffer flushes. #10315 (Azat Khuzhin)
  • Support for one special case of removing incompletely written parts. This fixes #9940. #10221 (alexey-milovidov)
  • Use isInjective() over manual list of such functions for GROUP BY optimization. #10342 (Azat Khuzhin)
  • Avoid printing error message in log if client sends RST packet immediately on connect. It is typical behaviour of IPVS balancer with keepalived and VRRP. This fixes #1851 #10274 (alexey-milovidov)
  • Allow to parse +inf for floating point types. This closes #1839 #10272 (alexey-milovidov)
  • Implemented generateRandom table function for Nested types. This closes #9903 #10219 (alexey-milovidov)
  • Provide max_allowed_packed in MySQL compatibility interface that will help some clients to communicate with ClickHouse via MySQL protocol. #10199 (BohuTANG)
  • Allow literals for GLOBAL IN (i.e. SELECT * FROM remote('localhost', system.one) WHERE dummy global in (0)) #10196 (Azat Khuzhin)
  • Fix various small issues in interactive mode of clickhouse-client #10194 (alexey-milovidov)
  • Avoid superfluous dictionaries load (system.tables, DROP/SHOW CREATE TABLE) #10164 (Azat Khuzhin)
  • Update to RWLock: timeout parameter for getLock() + implementation reworked to be phase fair #10073 (Alexander Kazakov)
  • Enhanced compatibility with native mysql-connector-java(JDBC) #10021 (BohuTANG)
  • The function toString is considered monotonic and can be used for index analysis even when applied in tautological cases with String or LowCardinality(String) argument. #10110 (Amos Bird)
  • Add ON CLUSTER clause support to commands {CREATE|DROP} USER/ROLE/ROW POLICY/SETTINGS PROFILE/QUOTA, GRANT. #9811 (Vitaly Baranov)
  • Virtual hosted-style support for S3 URI #9998 (Pavel Kovalenko)
  • Now layout type for dictionaries with no arguments can be specified without round brackets in dictionaries DDL-queries. Fixes #10057. #10064 (alesapin)
  • Add ability to use number ranges with leading zeros in filepath #9989 (Olga Khvostikova)
  • Better memory usage in CROSS JOIN. #10029 (Artem Zuikov)
  • Try to connect to all shards in cluster when getting structure of remote table and skip_unavailable_shards is set. #7278 (nvartolomei)
  • Add total_rows/total_bytes into the system.tables table. #9919 (Azat Khuzhin)
  • System log tables now use polymorpic parts by default. #9905 (Anton Popov)
  • Add type column into system.settings/merge_tree_settings #9909 (Azat Khuzhin)
  • Check for available CPU instructions at server startup as early as possible. #9888 (alexey-milovidov)
  • Remove ORDER BY stage from mutations because we read from a single ordered part in a single thread. Also add check that the rows in mutation are ordered by sorting key and this order is not violated. #9886 (alesapin)
  • Implement operator LIKE for FixedString at left hand side. This is needed to better support TPC-DS queries. #9890 (alexey-milovidov)
  • Add force_optimize_skip_unused_shards_no_nested that will disable force_optimize_skip_unused_shards for nested Distributed table #9812 (Azat Khuzhin)
  • Now columns size is calculated only once for MergeTree data parts. #9827 (alesapin)
  • Evaluate constant expressions for optimize_skip_unused_shards (i.e. SELECT * FROM foo_dist WHERE key=xxHash32(0)) #8846 (Azat Khuzhin)
  • Check for using Date or DateTime column from TTL expressions was removed. #9967 (Vladimir Chebotarev)
  • DiskS3 hard links optimal implementation. #9760 (Pavel Kovalenko)
  • If set multiple_joins_rewriter_version = 2 enables second version of multiple JOIN rewrites that keeps not clashed column names as is. It supports multiple JOINs with USING and allow select * for JOINs with subqueries. #9739 (Artem Zuikov)
  • Implementation of "non-blocking" alter for StorageMergeTree #9606 (alesapin)
  • Add MergeTree full support for DiskS3 #9646 (Pavel Kovalenko)
  • Extend splitByString to support empty strings as separators. #9742 (hcz)
  • Add a timestamp_ns column to system.trace_log. It contains a high-definition timestamp of the trace event, and allows to build timelines of thread profiles ("flame charts"). #9696 (Alexander Kuzmenkov)
  • When the setting send_logs_level is enabled, avoid intermixing of log messages and query progress. #9634 (Azat Khuzhin)
  • Added support of MATERIALIZE TTL IN PARTITION. #9581 (Vladimir Chebotarev)
  • Support complex types inside Avro nested fields #10502 (Andrew Onyshchuk)

Performance Improvement

  • Better insert logic for right table for Partial MergeJoin. #10467 (Artem Zuikov)
  • Improved performance of row-oriented formats (more than 10% for CSV and more than 35% for Avro in case of narrow tables). #10503 (Andrew Onyshchuk)
  • Improved performance of queries with explicitly defined sets at right side of IN operator and tuples on the left side. #10385 (Anton Popov)
  • Use less memory for hash table in HashJoin. #10416 (Artem Zuikov)
  • Special HashJoin over StorageDictionary. Allow rewrite dictGet() functions with JOINs. It's not backward incompatible itself but could uncover #8400 on some installations. #10133 (Artem Zuikov)
  • Enable parallel insert of materialized view when its target table supports. #10052 (vxider)
  • Improved performance of index analysis with monotonic functions. #9607#10026 (Anton Popov)
  • Using SSE2 or SSE4.2 SIMD intrinsics to speed up tokenization in bloom filters. #9968 (Vasily Nemkov)
  • Improved performance of queries with explicitly defined sets at right side of IN operator. This fixes performance regression in version 20.3. #9740 (Anton Popov)
  • Now clickhouse-copier splits each partition in number of pieces and copies them independently. #9075 (Nikita Mikhaylov)
  • Adding more aggregation methods. For example TPC-H query 1 will now pick FixedHashMap<UInt16, AggregateDataPtr> and gets 25% performance gain #9829 (Amos Bird)
  • Use single row counter for multiple streams in pre-limit transform. This helps to avoid uniting pipeline streams in queries with limit but without order by (like select f(x) from (select x from t limit 1000000000)) and use multiple threads for further processing. #9602 (Nikolai Kochetov)

Build/Testing/Packaging Improvement

ClickHouse release v20.3

ClickHouse release v20.3.21.2-lts, 2020-11-02

Bug Fix

  • Fix dictGet in sharding_key (and similar places, i.e. when the function context is stored permanently). #16205 (Azat Khuzhin).
  • Fix incorrect empty result for query from Distributed table if query has WHERE, PREWHERE and GLOBAL IN. Fixes #15792. #15933 (Nikolai Kochetov).
  • Fix missing or excessive headers in TSV/CSVWithNames formats. This fixes #12504. #13343 (Azat Khuzhin).

ClickHouse release v20.3.20.6-lts, 2020-10-09

Bug Fix

  • Mutation might hang waiting for some non-existent part after MOVE or REPLACE PARTITION or, in rare cases, after DETACH or DROP PARTITION. It's fixed. #15724, #15537 (tavplubix).
  • Fix hang of queries with a lot of subqueries to same table of MySQL engine. Previously, if there were more than 16 subqueries to same MySQL table in query, it hang forever. #15299 (Anton Popov).
  • Fix 'Unknown identifier' in GROUP BY when query has JOIN over Merge table. #15242 (Artem Zuikov).
  • Fix to make predicate push down work when subquery contains finalizeAggregation function. Fixes #14847. #14937 (filimonov).
  • Concurrent ALTER ... REPLACE/MOVE PARTITION ... queries might cause deadlock. It's fixed. #13626 (tavplubix).

ClickHouse release v20.3.19.4-lts, 2020-09-18

Bug Fix

  • Fix rare error in SELECT queries when the queried column has DEFAULT expression which depends on the other column which also has DEFAULT and not present in select query and not exists on disk. Partially fixes #14531. #14845 (alesapin).
  • Fix bug when ALTER UPDATE mutation with Nullable column in assignment expression and constant value (like UPDATE x = 42) leads to incorrect value in column or segfault. Fixes #13634, #14045. #14646 (alesapin).
  • Fix wrong Decimal multiplication result caused wrong decimal scale of result column. #14603 (Artem Zuikov).

Improvement

ClickHouse release v20.3.18.10-lts, 2020-09-08

Bug Fix

  • Stop query execution if exception happened in PipelineExecutor itself. This could prevent rare possible query hung. Continuation of #14334. #14402 (Nikolai Kochetov).
  • Fixed the behaviour when sometimes cache-dictionary returned default value instead of present value from source. #13624 (Nikita Mikhaylov).
  • Fix parsing row policies from users.xml when names of databases or tables contain dots. This fixes #5779, #12527. #13199 (Vitaly Baranov).
  • Fix CAST(Nullable(String), Enum()). #12745 (Azat Khuzhin).
  • Fixed data race in text_log. It does not correspond to any real bug. #9726 (alexey-milovidov).

Improvement

  • Fix wrong error for long queries. It was possible to get syntax error other than Max query size exceeded for correct query. #13928 (Nikolai Kochetov).
  • Return NULL/zero when value is not parsed completely in parseDateTimeBestEffortOrNull/Zero functions. This fixes #7876. #11653 (alexey-milovidov).

Performance Improvement

Build/Testing/Packaging Improvement

  • Fix UBSan report (adding zero to nullptr) in HashTable that appeared after migration to clang-10. #10638 (alexey-milovidov).

ClickHouse release v20.3.17.173-lts, 2020-08-15

Bug Fix

  • Fix crash in JOIN with StorageMerge and set enable_optimize_predicate_expression=1. #13679 (Artem Zuikov).
  • Fix invalid return type for comparison of tuples with NULL elements. Fixes #12461. #13420 (Nikolai Kochetov).
  • Fix queries with constant columns and ORDER BY prefix of primary key. #13396 (Anton Popov).
  • Return passed number for numbers with MSB set in roundUpToPowerOfTwoOrZero(). #13234 (Azat Khuzhin).

ClickHouse release v20.3.16.165-lts 2020-08-10

Bug Fix

  • Fixed error in parseDateTimeBestEffort function when unix timestamp was passed as an argument. This fixes #13362. #13441 (alexey-milovidov).
  • Fixed potentially low performance and slightly incorrect result for uniqExact, topK, sumDistinct and similar aggregate functions called on Float types with NaN values. It also triggered assert in debug build. This fixes #12491. #13254 (alexey-milovidov).
  • Fixed function if with nullable constexpr as cond that is not literal NULL. Fixes #12463. #13226 (alexey-milovidov).
  • Fixed assert in arrayElement function in case of array elements are Nullable and array subscript is also Nullable. This fixes #12172. #13224 (alexey-milovidov).
  • Fixed unnecessary limiting for the number of threads for selects from local replica. #12840 (Nikolai Kochetov).
  • Fixed possible extra overflow row in data which could appear for queries WITH TOTALS. #12747 (Nikolai Kochetov).
  • Fixed performance with large tuples, which are interpreted as functions in IN section. The case when user write WHERE x IN tuple(1, 2, ...) instead of WHERE x IN (1, 2, ...) for some obscure reason. #12700 (Anton Popov).
  • Fixed memory tracking for input_format_parallel_parsing (by attaching thread to group). #12672 (Azat Khuzhin).
  • Fixed #12293 allow push predicate when subquery contains with clause. #12663 (Winter Zhang).
  • Fixed #10572 fix bloom filter index with const expression. #12659 (Winter Zhang).
  • Fixed SIGSEGV in StorageKafka when broker is unavailable (and not only). #12658 (Azat Khuzhin).
  • Fixed race condition in external dictionaries with cache layout which can lead server crash. #12566 (alesapin).
  • Fixed bug which lead to broken old parts after ALTER DELETE query when enable_mixed_granularity_parts=1. Fixes #12536. #12543 (alesapin).
  • Better exception for function in with invalid number of arguments. #12529 (Anton Popov).
  • Fixed performance issue, while reading from compact parts. #12492 (Anton Popov).
  • Fixed the deadlock if text_log is enabled. #12452 (alexey-milovidov).
  • Fixed possible segfault if StorageMerge. Closes #12054. #12401 (tavplubix).
  • Fixed TOTALS/ROLLUP/CUBE for aggregate functions with -State and Nullable arguments. This fixes #12163. #12376 (alexey-milovidov).
  • Fixed order of columns in WITH FILL modifier. Previously order of columns of ORDER BY statement wasn't respected. #12306 (Anton Popov).
  • Avoid "bad cast" exception when there is an expression that filters data by virtual columns (like _table in Merge tables) or by "index" columns in system tables such as filtering by database name when querying from system.tables, and this expression returns Nullable type. This fixes #12166. #12305 (alexey-milovidov).
  • Show error after TrieDictionary failed to load. #12290 (Vitaly Baranov).
  • The function arrayFill worked incorrectly for empty arrays that may lead to crash. This fixes #12263. #12279 (alexey-milovidov).
  • Implement conversions to the common type for LowCardinality types. This allows to execute UNION ALL of tables with columns of LowCardinality and other columns. This fixes #8212. This fixes #4342. #12275 (alexey-milovidov).
  • Fixed the behaviour when during multiple sequential inserts in StorageFile header for some special types was written more than once. This fixed #6155. #12197 (Nikita Mikhaylov).
  • Fixed logical functions for UInt8 values when they are not equal to 0 or 1. #12196 (Alexander Kazakov).
  • Fixed dictGet arguments check during GROUP BY injective functions elimination. #12179 (Azat Khuzhin).
  • Fixed wrong logic in ALTER DELETE that leads to deleting of records when condition evaluates to NULL. This fixes #9088. This closes #12106. #12153 (alexey-milovidov).
  • Fixed transform of query to send to external DBMS (e.g. MySQL, ODBC) in presense of aliases. This fixes #12032. #12151 (alexey-milovidov).
  • Fixed potential overflow in integer division. This fixes #12119. #12140 (alexey-milovidov).
  • Fixed potential infinite loop in greatCircleDistance, geoDistance. This fixes #12117. #12137 (alexey-milovidov).
  • Avoid There is no query exception for materialized views with joins or with subqueries attached to system logs (system.query_log, metric_log, etc) or to engine=Buffer underlying table. #12120 (filimonov).
  • Fixed performance for selects with UNION caused by wrong limit for the total number of threads. Fixes #12030. #12103 (Nikolai Kochetov).
  • Fixed segfault with -StateResample combinators. #12092 (Anton Popov).
  • Fixed unnecessary limiting the number of threads for selects from VIEW. Fixes #11937. #12085 (Nikolai Kochetov).
  • Fixed possible crash while using wrong type for PREWHERE. Fixes #12053, #12060. #12060 (Nikolai Kochetov).
  • Fixed error Expected single dictionary argument for function for function defaultValueOfArgumentType with LowCardinality type. Fixes #11808. #12056 (Nikolai Kochetov).
  • Fixed error Cannot capture column for higher-order functions with Tuple(LowCardinality) argument. Fixes #9766. #12055 (Nikolai Kochetov).
  • Parse tables metadata in parallel when loading database. This fixes slow server startup when there are large number of tables. #12045 (tavplubix).
  • Make topK aggregate function return Enum for Enum types. This fixes #3740. #12043 (alexey-milovidov).
  • Fixed constraints check if constraint is a constant expression. This fixes #11360. #12042 (alexey-milovidov).
  • Fixed incorrect comparison of tuples with Nullable columns. Fixes #11985. #12039 (Nikolai Kochetov).
  • Fixed wrong result and potential crash when invoking function if with arguments of type FixedString with different sizes. This fixes #11362. #12021 (alexey-milovidov).
  • A query with function neighbor as the only returned expression may return empty result if the function is called with offset -9223372036854775808. This fixes #11367. #12019 (alexey-milovidov).
  • Fixed potential array size overflow in generateRandom that may lead to crash. This fixes #11371. #12013 (alexey-milovidov).
  • Fixed potential floating point exception. This closes #11378. #12005 (alexey-milovidov).
  • Fixed wrong setting name in log message at server startup. #11997 (alexey-milovidov).
  • Fixed Query parameter was not set in Values format. Fixes #11918. #11936 (tavplubix).
  • Keep aliases for substitutions in query (parametrized queries). This fixes #11914. #11916 (alexey-milovidov).
  • Fixed potential floating point exception when parsing DateTime64. This fixes #11374. #11875 (alexey-milovidov).
  • Fixed memory accounting via HTTP interface (can be significant with wait_end_of_query=1). #11840 (Azat Khuzhin).
  • Fixed wrong result for if() with NULLs in condition. #11807 (Artem Zuikov).
  • Parse metadata stored in zookeeper before checking for equality. #11739 (Azat Khuzhin).
  • Fixed LIMIT n WITH TIES usage together with ORDER BY statement, which contains aliases. #11689 (Anton Popov).
  • Fix potential read of uninitialized memory in cache dictionary. #10834 (alexey-milovidov).

Performance Improvement

  • Index not used for IN operator with literals, performance regression introduced around v19.3. This fixes #10574. #12062 (nvartolomei).

ClickHouse release v20.3.12.112-lts 2020-06-25

Bug Fix

  • Fix rare crash caused by using Nullable column in prewhere condition. Continuation of #11608. #11869 (Nikolai Kochetov).
  • Don't allow arrayJoin inside higher order functions. It was leading to broken protocol synchronization. This closes #3933. #11846 (alexey-milovidov).
  • Fix using too many threads for queries. #11788 (Nikolai Kochetov).
  • Fix unexpected behaviour of queries like SELECT *, xyz.* which were success while an error expected. #11753 (hexiaoting).
  • Now replicated fetches will be cancelled during metadata alter. #11744 (alesapin).
  • Fixed LOGICAL_ERROR caused by wrong type deduction of complex literals in Values input format. #11732 (tavplubix).
  • Fix ORDER BY ... WITH FILL over const columns. #11697 (Anton Popov).
  • Pass proper timeouts when communicating with XDBC bridge. Recently timeouts were not respected when checking bridge liveness and receiving meta info. #11690 (alexey-milovidov).
  • Fix error which leads to an incorrect state of system.mutations. It may show that whole mutation is already done but the server still has MUTATE_PART tasks in the replication queue and tries to execute them. This fixes #11611. #11681 (alesapin).
  • Add support for regular expressions with case-insensitive flags. This fixes #11101 and fixes #11506. #11649 (alexey-milovidov).
  • Remove trivial count query optimization if row-level security is set. In previous versions the user get total count of records in a table instead filtered. This fixes #11352. #11644 (alexey-milovidov).
  • Fix bloom filters for String (data skipping indices). #11638 (Azat Khuzhin).
  • Fix rare crash caused by using Nullable column in prewhere condition. (Probably it is connected with #11572 somehow). #11608 (Nikolai Kochetov).
  • Fix error Block structure mismatch for queries with sampling reading from Buffer table. #11602 (Nikolai Kochetov).
  • Fix wrong exit code of the clickhouse-client, when exception.code() % 256 = 0. #11601 (filimonov).
  • Fix trivial error in log message about "Mark cache size was lowered" at server startup. This closes #11399. #11589 (alexey-milovidov).
  • Fix error Size of offsets doesn't match size of column for queries with PREWHERE column in (subquery) and ARRAY JOIN. #11580 (Nikolai Kochetov).
  • All queries in HTTP session have had the same query_id. It is fixed. #11578 (tavplubix).
  • Now clickhouse-server docker container will prefer IPv6 checking server aliveness. #11550 (Ivan Starkov).
  • Fix shard_num/replica_num for <node> (breaks use_compact_format_in_distributed_parts_names). #11528 (Azat Khuzhin).
  • Fix memory leak when exception is thrown in the middle of aggregation with -State functions. This fixes #8995. #11496 (alexey-milovidov).
  • Fix wrong results of distributed queries when alias could override qualified column name. Fixes #9672 #9714. #9972 (Artem Zuikov).

ClickHouse release v20.3.11.97-lts 2020-06-10

New Feature

  • Now ClickHouse controls timeouts of dictionary sources on its side. Two new settings added to cache dictionary configuration: strict_max_lifetime_seconds, which is max_lifetime by default and query_wait_timeout_milliseconds, which is one minute by default. The first settings is also useful with allow_read_expired_keys settings (to forbid reading very expired keys). #10337 (Nikita Mikhaylov).

Bug Fix

  • Fix the error Data compressed with different methods that can happen if min_bytes_to_use_direct_io is enabled and PREWHERE is active and using SAMPLE or high number of threads. This fixes #11539. #11540 (alexey-milovidov).
  • Fix return compressed size for codecs. #11448 (Nikolai Kochetov).
  • Fix server crash when a column has compression codec with non-literal arguments. Fixes #11365. #11431 (alesapin).
  • Fix pointInPolygon with nan as point. Fixes #11375. #11421 (Alexey Ilyukhov).
  • Fix crash in JOIN over LowCarinality(T) and Nullable(T). #11380. #11414 (Artem Zuikov).
  • Fix error code for wrong USING key. #11373. #11404 (Artem Zuikov).
  • Fixed geohashesInBox with arguments outside of latitude/longitude range. #11403 (Vasily Nemkov).
  • Better errors for joinGet() functions. #11389 (Artem Zuikov).
  • Fix possible Pipeline stuck error for queries with external sort and limit. Fixes #11359. #11366 (Nikolai Kochetov).
  • Remove redundant lock during parts send in ReplicatedMergeTree. #11354 (alesapin).
  • Fix support for \G (vertical output) in clickhouse-client in multiline mode. This closes #9933. #11350 (alexey-milovidov).
  • Fix crash in direct selects from StorageJoin (without JOIN) and wrong nullability. #11340 (Artem Zuikov).
  • Fix crash in quantilesExactWeightedArray. #11337 (Nikolai Kochetov).
  • Now merges stopped before change metadata in ALTER queries. #11335 (alesapin).
  • Make writing to MATERIALIZED VIEW with setting parallel_view_processing = 1 parallel again. Fixes #10241. #11330 (Nikolai Kochetov).
  • Fix visitParamExtractRaw when extracted JSON has strings with unbalanced { or [. #11318 (Ewout).
  • Fix very rare race condition in ThreadPool. #11314 (alexey-milovidov).
  • Fix potential uninitialized memory in conversion. Example: SELECT toIntervalSecond(now64()). #11311 (alexey-milovidov).
  • Fix the issue when index analysis cannot work if a table has Array column in primary key and if a query is filtering by this column with empty or notEmpty functions. This fixes #11286. #11303 (alexey-milovidov).
  • Fix bug when query speed estimation can be incorrect and the limit of min_execution_speed may not work or work incorrectly if the query is throttled by max_network_bandwidth, max_execution_speed or priority settings. Change the default value of timeout_before_checking_execution_speed to non-zero, because otherwise the settings min_execution_speed and max_execution_speed have no effect. This fixes #11297. This fixes #5732. This fixes #6228. Usability improvement: avoid concatenation of exception message with progress bar in clickhouse-client. #11296 (alexey-milovidov).
  • Fix crash while reading malformed data in Protobuf format. This fixes #5957, fixes #11203. #11258 (Vitaly Baranov).
  • Fixed a bug when cache-dictionary could return default value instead of normal (when there are only expired keys). This affects only string fields. #11233 (Nikita Mikhaylov).
  • Fix error Block structure mismatch in QueryPipeline while reading from VIEW with constants in inner query. Fixes #11181. #11205 (Nikolai Kochetov).
  • Fix possible exception Invalid status for associated output. #11200 (Nikolai Kochetov).
  • Fix possible error Cannot capture column for higher-order functions with Array(Array(LowCardinality)) captured argument. #11185 (Nikolai Kochetov).
  • Fixed S3 globbing which could fail in case of more than 1000 keys and some backends. #11179 (Vladimir Chebotarev).
  • If data skipping index is dependent on columns that are going to be modified during background merge (for SummingMergeTree, AggregatingMergeTree as well as for TTL GROUP BY), it was calculated incorrectly. This issue is fixed by moving index calculation after merge so the index is calculated on merged data. #11162 (Azat Khuzhin).
  • Fix excessive reserving of threads for simple queries (optimization for reducing the number of threads, which was partly broken after changes in pipeline). #11114 (Azat Khuzhin).
  • Fix predicates optimization for distributed queries (enable_optimize_predicate_expression=1) for queries with HAVING section (i.e. when filtering on the server initiator is required), by preserving the order of expressions (and this is enough to fix), and also force aggregator use column names over indexes. Fixes: #10613, #11413. #10621 (Azat Khuzhin).
  • Introduce commit retry logic to decrease the possibility of getting duplicates from Kafka in rare cases when offset commit was failed. #9884 (filimonov).

Performance Improvement

  • Get dictionary and check access rights only once per each call of any function reading external dictionaries. #10928 (Vitaly Baranov).

Build/Testing/Packaging Improvement

ClickHouse release v20.3.10.75-lts 2020-05-23

Bug Fix

  • Removed logging from mutation finalization task if nothing was finalized. #11109 (alesapin).
  • Fixed parseDateTime64BestEffort argument resolution bugs. #11038 (Vasily Nemkov).
  • Fixed incorrect raw data size in method getRawData(). #10964 (Igr).
  • Fixed incompatibility of two-level aggregation between versions 20.1 and earlier. This incompatibility happens when different versions of ClickHouse are used on initiator node and remote nodes and the size of GROUP BY result is large and aggregation is performed by a single String field. It leads to several unmerged rows for a single key in result. #10952 (alexey-milovidov).
  • Fixed backward compatibility with tuples in Distributed tables. #10889 (Anton Popov).
  • Fixed SIGSEGV in StringHashTable if such a key does not exist. #10870 (Azat Khuzhin).
  • Fixed bug in ReplicatedMergeTree which might cause some ALTER on OPTIMIZE query to hang waiting for some replica after it become inactive. #10849 (tavplubix).
  • Fixed columns order after Block::sortColumns(). #10826 (Azat Khuzhin).
  • Fixed the issue with ODBC bridge when no quoting of identifiers is requested. Fixes #7984. #10821 (alexey-milovidov).
  • Fixed UBSan and MSan report in DateLUT. #10798 (alexey-milovidov).
  • Fixed incorrect type conversion in key conditions. Fixes #6287. #10791 (Andrew Onyshchuk)
  • Fixed parallel_view_processing behavior. Now all insertions into MATERIALIZED VIEW without exception should be finished if exception happened. Fixes #10241. #10757 (Nikolai Kochetov).
  • Fixed combinator -OrNull and -OrDefault when combined with -State. #10741 (hcz).
  • Fixed crash in generateRandom with nested types. Fixes #10583. #10734 (Nikolai Kochetov).
  • Fixed data corruption for LowCardinality(FixedString) key column in SummingMergeTree which could have happened after merge. Fixes #10489. #10721 (Nikolai Kochetov).
  • Fixed possible buffer overflow in function h3EdgeAngle. #10711 (alexey-milovidov).
  • Fixed disappearing totals. Totals could have being filtered if query had had join or subquery with external where condition. Fixes #10674. #10698 (Nikolai Kochetov).
  • Fixed multiple usages of IN operator with the identical set in one query. #10686 (Anton Popov).
  • Fixed bug, which causes http requests stuck on client close when readonly=2 and cancel_http_readonly_queries_on_client_close=1. Fixes #7939, #7019, #7736, #7091. #10684 (tavplubix).
  • Fixed order of parameters in AggregateTransform constructor. #10667 (palasonic1).
  • Fixed the lack of parallel execution of remote queries with distributed_aggregation_memory_efficient enabled. Fixes #10655. #10664 (Nikolai Kochetov).
  • Fixed possible incorrect number of rows for queries with LIMIT. Fixes #10566, #10709. #10660 (Nikolai Kochetov).
  • Fixed a bug which locks concurrent alters when table has a lot of parts. #10659 (alesapin).
  • Fixed a bug when on SYSTEM DROP DNS CACHE query also drop caches, which are used to check if user is allowed to connect from some IP addresses. #10608 (tavplubix).
  • Fixed incorrect scalar results inside inner query of MATERIALIZED VIEW in case if this query contained dependent table. #10603 (Nikolai Kochetov).
  • Fixed SELECT of column ALIAS which default expression type different from column type. #10563 (Azat Khuzhin).
  • Implemented comparison between DateTime64 and String values. #10560 (Vasily Nemkov).
  • Fixed index corruption, which may occur in some cases after merge compact parts into another compact part. #10531 (Anton Popov).
  • Fixed the situation, when mutation finished all parts, but hung up in is_done=0. #10526 (alesapin).
  • Fixed overflow at beginning of unix epoch for timezones with fractional offset from UTC. This fixes #9335. #10513 (alexey-milovidov).
  • Fixed improper shutdown of Distributed storage. #10491 (Azat Khuzhin).
  • Fixed numeric overflow in simpleLinearRegression over large integers. #10474 (hcz).

Build/Testing/Packaging Improvement

Bug fix

ClickHouse release v20.3.8.53, 2020-04-23

Bug Fix

  • Fixed wrong behaviour of datetime functions for timezones that has altered between positive and negative offsets from UTC (e.g. Pacific/Kiritimati). This fixes #7202 #10369 (alexey-milovidov)
  • Fix possible segfault with distributed_group_by_no_merge enabled (introduced in 20.3.7.46 by #10131). #10399 (Nikolai Kochetov)
  • Fix wrong flattening of Array(Tuple(...)) data types. This fixes #10259 #10390 (alexey-milovidov)
  • Drop disks reservation in Aggregator. This fixes bug in disk space reservation, which may cause big external aggregation to fail even if it could be completed successfully #10375 (Azat Khuzhin)
  • Fixed DROP vs OPTIMIZE race in ReplicatedMergeTree. DROP could left some garbage in replica path in ZooKeeper if there was concurrent OPTIMIZE query. #10312 (tavplubix)
  • Fix bug when server cannot attach table after column default was altered. #10441 (alesapin)
  • Do not remove metadata directory when attach database fails before loading tables. #10442 (Winter Zhang)
  • Fixed several bugs when some data was inserted with quorum, then deleted somehow (DROP PARTITION, TTL) and this leaded to the stuck of INSERTs or false-positive exceptions in SELECTs. This fixes #9946 #10188 (Nikita Mikhaylov)
  • Fix possible Pipeline stuck error in ConcatProcessor which could have happened in remote query. #10381 (Nikolai Kochetov)
  • Fixed wrong behavior in HashTable that caused compilation error when trying to read HashMap from buffer. #10386 (palasonic1)
  • Allow to use count(*) with multiple JOINs. Fixes #9853 #10291 (Artem Zuikov)
  • Prefer fallback_to_stale_replicas over skip_unavailable_shards, otherwise when both settings specified and there are no up-to-date replicas the query will fail (patch from @alex-zaitsev). Fixes: #2564. #10422 (Azat Khuzhin)
  • Fix the issue when a query with ARRAY JOIN, ORDER BY and LIMIT may return incomplete result. This fixes #10226. Author: Vadim Plakhtinskiy. #10427 (alexey-milovidov)
  • Check the number and type of arguments when creating BloomFilter index #9623 #10431 (Winter Zhang)

Performance Improvement

  • Improved performance of queries with explicitly defined sets at right side of IN operator and tuples in the left side. This fixes performance regression in version 20.3. #9740, #10385 (Anton Popov)

ClickHouse release v20.3.7.46, 2020-04-17

Bug Fix

  • Fix Logical error: CROSS JOIN has expressions error for queries with comma and names joins mix. #10311 (Artem Zuikov).
  • Fix queries with max_bytes_before_external_group_by. #10302 (Artem Zuikov).
  • Fix move-to-prewhere optimization in presense of arrayJoin functions (in certain cases). This fixes #10092. #10195 (alexey-milovidov).
  • Add the ability to relax the restriction on non-deterministic functions usage in mutations with allow_nondeterministic_mutations setting. #10186 (filimonov).

ClickHouse release v20.3.6.40, 2020-04-16

New Feature

  • Added function isConstant. This function checks whether its argument is constant expression and returns 1 or 0. It is intended for development, debugging and demonstration purposes. #10198 (alexey-milovidov).

Bug Fix

  • Fix error Pipeline stuck with max_rows_to_group_by and group_by_overflow_mode = 'break'. #10279 (Nikolai Kochetov).
  • Fix rare possible exception Cannot drain connections: cancel first. #10239 (Nikolai Kochetov).
  • Fixed bug where ClickHouse would throw "Unknown function lambda." error message when user tries to run ALTER UPDATE/DELETE on tables with ENGINE = Replicated*. Check for nondeterministic functions now handles lambda expressions correctly. #10237 (Alexander Kazakov).
  • Fixed "generateRandom" function for Date type. This fixes #9973. Fix an edge case when dates with year 2106 are inserted to MergeTree tables with old-style partitioning but partitions are named with year 1970. #10218 (alexey-milovidov).
  • Convert types if the table definition of a View does not correspond to the SELECT query. This fixes #10180 and #10022. #10217 (alexey-milovidov).
  • Fix parseDateTimeBestEffort for strings in RFC-2822 when day of week is Tuesday or Thursday. This fixes #10082. #10214 (alexey-milovidov).
  • Fix column names of constants inside JOIN that may clash with names of constants outside of JOIN. #10207 (alexey-milovidov).
  • Fix possible inifinite query execution when the query actually should stop on LIMIT, while reading from infinite source like system.numbers or system.zeros. #10206 (Nikolai Kochetov).
  • Fix using the current database for access checking when the database isn't specified. #10192 (Vitaly Baranov).
  • Convert blocks if structure does not match on INSERT into Distributed(). #10135 (Azat Khuzhin).
  • Fix possible incorrect result for extremes in processors pipeline. #10131 (Nikolai Kochetov).
  • Fix some kinds of alters with compact parts. #10130 (Anton Popov).
  • Fix incorrect index_granularity_bytes check while creating new replica. Fixes #10098. #10121 (alesapin).
  • Fix SIGSEGV on INSERT into Distributed table when its structure differs from the underlying tables. #10105 (Azat Khuzhin).
  • Fix possible rows loss for queries with JOIN and UNION ALL. Fixes #9826, #10113. #10099 (Nikolai Kochetov).
  • Fixed replicated tables startup when updating from an old ClickHouse version where /table/replicas/replica_name/metadata node doesn't exist. Fixes #10037. #10095 (alesapin).
  • Add some arguments check and support identifier arguments for MySQL Database Engine. #10077 (Winter Zhang).
  • Fix bug in clickhouse dictionary source from localhost clickhouse server. The bug may lead to memory corruption if types in dictionary and source are not compatible. #10071 (alesapin).
  • Fix bug in CHECK TABLE query when table contain skip indices. #10068 (alesapin).
  • Fix error Cannot clone block with columns because block has 0 columns ... While executing GroupingAggregatedTransform. It happened when setting distributed_aggregation_memory_efficient was enabled, and distributed query read aggregating data with different level from different shards (mixed single and two level aggregation). #10063 (Nikolai Kochetov).
  • Fix a segmentation fault that could occur in GROUP BY over string keys containing trailing zero bytes (#8636, #8925). #10025 (Alexander Kuzmenkov).
  • Fix the number of threads used for remote query execution (performance regression, since 20.3). This happened when query from Distributed table was executed simultaneously on local and remote shards. Fixes #9965. #9971 (Nikolai Kochetov).
  • Fix bug in which the necessary tables weren't retrieved at one of the processing stages of queries to some databases. Fixes #9699. #9949 (achulkov2).
  • Fix 'Not found column in block' error when JOIN appears with TOTALS. Fixes #9839. #9939 (Artem Zuikov).
  • Fix a bug with ON CLUSTER DDL queries freezing on server startup. #9927 (Gagan Arneja).
  • Fix parsing multiple hosts set in the CREATE USER command, e.g. CREATE USER user6 HOST NAME REGEXP 'lo.?*host', NAME REGEXP 'lo*host'. #9924 (Vitaly Baranov).
  • Fix TRUNCATE for Join table engine (#9917). #9920 (Amos Bird).
  • Fix "scalar doesn't exist" error in ALTERs (#9878). #9904 (Amos Bird).
  • Fix race condition between drop and optimize in ReplicatedMergeTree. #9901 (alesapin).
  • Fix error with qualified names in distributed_product_mode='local'. Fixes #4756. #9891 (Artem Zuikov).
  • Fix calculating grants for introspection functions from the setting 'allow_introspection_functions'. #9840 (Vitaly Baranov).

Build/Testing/Packaging Improvement

ClickHouse release v20.3.5.21, 2020-03-27

Bug Fix

  • Fix 'Different expressions with the same alias' error when query has PREWHERE and WHERE on distributed table and SET distributed_product_mode = 'local'. #9871 (Artem Zuikov).
  • Fix mutations excessive memory consumption for tables with a composite primary key. This fixes #9850. #9860 (alesapin).
  • For INSERT queries shard now clamps the settings got from the initiator to the shard's constaints instead of throwing an exception. This fix allows to send INSERT queries to a shard with another constraints. This change improves fix #9447. #9852 (Vitaly Baranov).
  • Fix 'COMMA to CROSS JOIN rewriter is not enabled or cannot rewrite query' error in case of subqueries with COMMA JOIN out of tables lists (i.e. in WHERE). Fixes #9782. #9830 (Artem Zuikov).
  • Fix possible exception Got 0 in totals chunk, expected 1 on client. It happened for queries with JOIN in case if right joined table had zero rows. Example: select * from system.one t1 join system.one t2 on t1.dummy = t2.dummy limit 0 FORMAT TabSeparated;. Fixes #9777. #9823 (Nikolai Kochetov).
  • Fix SIGSEGV with optimize_skip_unused_shards when type cannot be converted. #9804 (Azat Khuzhin).
  • Fix broken ALTER TABLE DELETE COLUMN query for compact parts. #9779 (alesapin).
  • Fix max_distributed_connections (w/ and w/o Processors). #9673 (Azat Khuzhin).
  • Fixed a few cases when timezone of the function argument wasn't used properly. #9574 (Vasily Nemkov).

Improvement

  • Remove order by stage from mutations because we read from a single ordered part in a single thread. Also add check that the order of rows in mutation is ordered in sorting key order and this order is not violated. #9886 (alesapin).

ClickHouse release v20.3.4.10, 2020-03-20

Bug Fix

  • This release also contains all bug fixes from 20.1.8.41
  • Fix missing rows_before_limit_at_least for queries over http (with processors pipeline). This fixes #9730. #9757 (Nikolai Kochetov)

ClickHouse release v20.3.3.6, 2020-03-17

Bug Fix

  • This release also contains all bug fixes from 20.1.7.38
  • Fix bug in a replication that doesn't allow replication to work if the user has executed mutations on the previous version. This fixes #9645. #9652 (alesapin). It makes version 20.3 backward compatible again.
  • Add setting use_compact_format_in_distributed_parts_names which allows to write files for INSERT queries into Distributed table with more compact format. This fixes #9647. #9653 (alesapin). It makes version 20.3 backward compatible again.

ClickHouse release v20.3.2.1, 2020-03-12

Backward Incompatible Change

  • Fixed the issue file name too long when sending data for Distributed tables for a large number of replicas. Fixed the issue that replica credentials were exposed in the server log. The format of directory name on disk was changed to [shard{shard_index}[_replica{replica_index}]]. #8911 (Mikhail Korotov) After you upgrade to the new version, you will not be able to downgrade without manual intervention, because old server version does not recognize the new directory format. If you want to downgrade, you have to manually rename the corresponding directories to the old format. This change is relevant only if you have used asynchronous INSERTs to Distributed tables. In the version 20.3.3 we will introduce a setting that will allow you to enable the new format gradually.
  • Changed the format of replication log entries for mutation commands. You have to wait for old mutations to process before installing the new version.
  • Implement simple memory profiler that dumps stacktraces to system.trace_log every N bytes over soft allocation limit #8765 (Ivan) #9472 (alexey-milovidov) The column of system.trace_log was renamed from timer_type to trace_type. This will require changes in third-party performance analysis and flamegraph processing tools.
  • Use OS thread id everywhere instead of internal thread number. This fixes #7477 Old clickhouse-client cannot receive logs that are send from the server when the setting send_logs_level is enabled, because the names and types of the structured log messages were changed. On the other hand, different server versions can send logs with different types to each other. When you don't use the send_logs_level setting, you should not care. #8954 (alexey-milovidov)
  • Remove indexHint function #9542 (alexey-milovidov)
  • Remove findClusterIndex, findClusterValue functions. This fixes #8641. If you were using these functions, send an email to clickhouse-feedback@yandex-team.com #9543 (alexey-milovidov)
  • Now it's not allowed to create columns or add columns with SELECT subquery as default expression. #9481 (alesapin)
  • Require aliases for subqueries in JOIN. #9274 (Artem Zuikov)
  • Improved ALTER MODIFY/ADD queries logic. Now you cannot ADD column without type, MODIFY default expression doesn't change type of column and MODIFY type doesn't loose default expression value. Fixes #8669. #9227 (alesapin)
  • Require server to be restarted to apply the changes in logging configuration. This is a temporary workaround to avoid the bug where the server logs to a deleted log file (see #8696). #8707 (Alexander Kuzmenkov)
  • The setting experimental_use_processors is enabled by default. This setting enables usage of the new query pipeline. This is internal refactoring and we expect no visible changes. If you will see any issues, set it to back zero. #8768 (alexey-milovidov)

New Feature

  • Add Avro and AvroConfluent input/output formats #8571 (Andrew Onyshchuk) #8957 (Andrew Onyshchuk) #8717 (alexey-milovidov)
  • Multi-threaded and non-blocking updates of expired keys in cache dictionaries (with optional permission to read old ones). #8303 (Nikita Mikhaylov)
  • Add query ALTER ... MATERIALIZE TTL. It runs mutation that forces to remove expired data by TTL and recalculates meta-information about TTL in all parts. #8775 (Anton Popov)
  • Switch from HashJoin to MergeJoin (on disk) if needed #9082 (Artem Zuikov)
  • Added MOVE PARTITION command for ALTER TABLE #4729 #6168 (Guillaume Tassery)
  • Reloading storage configuration from configuration file on the fly. #8594 (Vladimir Chebotarev)
  • Allowed to change storage_policy to not less rich one. #8107 (Vladimir Chebotarev)
  • Added support for globs/wildcards for S3 storage and table function. #8851 (Vladimir Chebotarev)
  • Implement bitAnd, bitOr, bitXor, bitNot for FixedString(N) datatype. #9091 (Guillaume Tassery)
  • Added function bitCount. This fixes #8702. #8708 (alexey-milovidov) #8749 (ikopylov)
  • Add generateRandom table function to generate random rows with given schema. Allows to populate arbitrary test table with data. #8994 (Ilya Yatsishin)
  • JSONEachRowFormat: support special case when objects enclosed in top-level array. #8860 (Kruglov Pavel)
  • Now it's possible to create a column with DEFAULT expression which depends on a column with default ALIAS expression. #9489 (alesapin)
  • Allow to specify --limit more than the source data size in clickhouse-obfuscator. The data will repeat itself with different random seed. #9155 (alexey-milovidov)
  • Added groupArraySample function (similar to groupArray) with reservior sampling algorithm. #8286 (Amos Bird)
  • Now you can monitor the size of update queue in cache/complex_key_cache dictionaries via system metrics. #9413 (Nikita Mikhaylov)
  • Allow to use CRLF as a line separator in CSV output format with setting output_format_csv_crlf_end_of_line is set to 1 #8934 #8935 #8963 (Mikhail Korotov)
  • Implement more functions of the H3 API: h3GetBaseCell, h3HexAreaM2, h3IndexesAreNeighbors, h3ToChildren, h3ToString and stringToH3 #8938 (Nico Mandery)
  • New setting introduced: max_parser_depth to control maximum stack size and allow large complex queries. This fixes #6681 and #7668. #8647 (Maxim Smirnov)
  • Add a setting force_optimize_skip_unused_shards setting to throw if skipping of unused shards is not possible #8805 (Azat Khuzhin)
  • Allow to configure multiple disks/volumes for storing data for send in Distributed engine #8756 (Azat Khuzhin)
  • Support storage policy (<tmp_policy>) for storing temporary data. #8750 (Azat Khuzhin)
  • Added X-ClickHouse-Exception-Code HTTP header that is set if exception was thrown before sending data. This implements #4971. #8786 (Mikhail Korotov)
  • Added function ifNotFinite. It is just a syntactic sugar: ifNotFinite(x, y) = isFinite(x) ? x : y. #8710 (alexey-milovidov)
  • Added last_successful_update_time column in system.dictionaries table #9394 (Nikita Mikhaylov)
  • Add blockSerializedSize function (size on disk without compression) #8952 (Azat Khuzhin)
  • Add function moduloOrZero #9358 (hcz)
  • Added system tables system.zeros and system.zeros_mt as well as tale functions zeros() and zeros_mt(). Tables (and table functions) contain single column with name zero and type UInt8. This column contains zeros. It is needed for test purposes as the fastest method to generate many rows. This fixes #6604 #9593 (Nikolai Kochetov)

Experimental Feature

  • Add new compact format of parts in MergeTree-family tables in which all columns are stored in one file. It helps to increase performance of small and frequent inserts. The old format (one file per column) is now called wide. Data storing format is controlled by settings min_bytes_for_wide_part and min_rows_for_wide_part. #8290 (Anton Popov)
  • Support for S3 storage for Log, TinyLog and StripeLog tables. #8862 (Pavel Kovalenko)

Bug Fix

  • Fixed inconsistent whitespaces in log messages. #9322 (alexey-milovidov)
  • Fix bug in which arrays of unnamed tuples were flattened as Nested structures on table creation. #8866 (achulkov2)
  • Fixed the issue when "Too many open files" error may happen if there are too many files matching glob pattern in File table or file table function. Now files are opened lazily. This fixes #8857 #8861 (alexey-milovidov)
  • DROP TEMPORARY TABLE now drops only temporary table. #8907 (Vitaly Baranov)
  • Remove outdated partition when we shutdown the server or DETACH/ATTACH a table. #8602 (Guillaume Tassery)
  • For how the default disk calculates the free space from data subdirectory. Fixed the issue when the amount of free space is not calculated correctly if the data directory is mounted to a separate device (rare case). This fixes #7441 #9257 (Mikhail Korotov)
  • Allow comma (cross) join with IN () inside. #9251 (Artem Zuikov)
  • Allow to rewrite CROSS to INNER JOIN if there's [NOT] LIKE operator in WHERE section. #9229 (Artem Zuikov)
  • Fix possible incorrect result after GROUP BY with enabled setting distributed_aggregation_memory_efficient. Fixes #9134. #9289 (Nikolai Kochetov)
  • Found keys were counted as missed in metrics of cache dictionaries. #9411 (Nikita Mikhaylov)
  • Fix replication protocol incompatibility introduced in #8598. #9412 (alesapin)
  • Fixed race condition on queue_task_handle at the startup of ReplicatedMergeTree tables. #9552 (alexey-milovidov)
  • The token NOT didn't work in SHOW TABLES NOT LIKE query #8727 #8940 (alexey-milovidov)
  • Added range check to function h3EdgeLengthM. Without this check, buffer overflow is possible. #8945 (alexey-milovidov)
  • Fixed up a bug in batched calculations of ternary logical OPs on multiple arguments (more than 10). #8718 (Alexander Kazakov)
  • Fix error of PREWHERE optimization, which could lead to segfaults or Inconsistent number of columns got from MergeTreeRangeReader exception. #9024 (Anton Popov)
  • Fix unexpected Timeout exceeded while reading from socket exception, which randomly happens on secure connection before timeout actually exceeded and when query profiler is enabled. Also add connect_timeout_with_failover_secure_ms settings (default 100ms), which is similar to connect_timeout_with_failover_ms, but is used for secure connections (because SSL handshake is slower, than ordinary TCP connection) #9026 (tavplubix)
  • Fix bug with mutations finalization, when mutation may hang in state with parts_to_do=0 and is_done=0. #9022 (alesapin)
  • Use new ANY JOIN logic with partial_merge_join setting. It's possible to make ANY|ALL|SEMI LEFT and ALL INNER joins with partial_merge_join=1 now. #8932 (Artem Zuikov)
  • Shard now clamps the settings got from the initiator to the shard's constaints instead of throwing an exception. This fix allows to send queries to a shard with another constraints. #9447 (Vitaly Baranov)
  • Fixed memory management problem in MergeTreeReadPool. #8791 (Vladimir Chebotarev)
  • Fix toDecimal*OrNull() functions family when called with string e. Fixes #8312 #8764 (Artem Zuikov)
  • Make sure that FORMAT Null sends no data to the client. #8767 (Alexander Kuzmenkov)
  • Fix bug that timestamp in LiveViewBlockInputStream will not updated. LIVE VIEW is an experimental feature. #8644 (vxider) #8625 (vxider)
  • Fixed ALTER MODIFY TTL wrong behavior which did not allow to delete old TTL expressions. #8422 (Vladimir Chebotarev)
  • Fixed UBSan report in MergeTreeIndexSet. This fixes #9250 #9365 (alexey-milovidov)
  • Fixed the behaviour of match and extract functions when haystack has zero bytes. The behaviour was wrong when haystack was constant. This fixes #9160 #9163 (alexey-milovidov) #9345 (alexey-milovidov)
  • Avoid throwing from destructor in Apache Avro 3rd-party library. #9066 (Andrew Onyshchuk)
  • Don't commit a batch polled from Kafka partially as it can lead to holes in data. #8876 (filimonov)
  • Fix joinGet with nullable return types. #8919 #9014 (Amos Bird)
  • Fix data incompatibility when compressed with T64 codec. #9016 (Artem Zuikov) Fix data type ids in T64 compression codec that leads to wrong (de)compression in affected versions. #9033 (Artem Zuikov)
  • Add setting enable_early_constant_folding and disable it in some cases that leads to errors. #9010 (Artem Zuikov)
  • Fix pushdown predicate optimizer with VIEW and enable the test #9011 (Winter Zhang)
  • Fix segfault in Merge tables, that can happen when reading from File storages #9387 (tavplubix)
  • Added a check for storage policy in ATTACH PARTITION FROM, REPLACE PARTITION, MOVE TO TABLE. Otherwise it could make data of part inaccessible after restart and prevent ClickHouse to start. #9383 (Vladimir Chebotarev)
  • Fix alters if there is TTL set for table. #8800 (Anton Popov)
  • Fix race condition that can happen when SYSTEM RELOAD ALL DICTIONARIES is executed while some dictionary is being modified/added/removed. #8801 (Vitaly Baranov)
  • In previous versions Memory database engine use empty data path, so tables are created in path directory (e.g. /var/lib/clickhouse/), not in data directory of database (e.g. /var/lib/clickhouse/db_name). #8753 (tavplubix)
  • Fixed wrong log messages about missing default disk or policy. #9530 (Vladimir Chebotarev)
  • Fix not(has()) for the bloom_filter index of array types. #9407 (achimbab)
  • Allow first column(s) in a table with Log engine be an alias #9231 (Ivan)
  • Fix order of ranges while reading from MergeTree table in one thread. It could lead to exceptions from MergeTreeRangeReader or wrong query results. #9050 (Anton Popov)
  • Make reinterpretAsFixedString to return FixedString instead of String. #9052 (Andrew Onyshchuk)
  • Avoid extremely rare cases when the user can get wrong error message (Success instead of detailed error description). #9457 (alexey-milovidov)
  • Do not crash when using Template format with empty row template. #8785 (Alexander Kuzmenkov)
  • Metadata files for system tables could be created in wrong place #8653 (tavplubix) Fixes #8581.
  • Fix data race on exception_ptr in cache dictionary #8303. #9379 (Nikita Mikhaylov)
  • Do not throw an exception for query ATTACH TABLE IF NOT EXISTS. Previously it was thrown if table already exists, despite the IF NOT EXISTS clause. #8967 (Anton Popov)
  • Fixed missing closing paren in exception message. #8811 (alexey-milovidov)
  • Avoid message Possible deadlock avoided at the startup of clickhouse-client in interactive mode. #9455 (alexey-milovidov)
  • Fixed the issue when padding at the end of base64 encoded value can be malformed. Update base64 library. This fixes #9491, closes #9492 #9500 (alexey-milovidov)
  • Prevent losing data in Kafka in rare cases when exception happens after reading suffix but before commit. Fixes #9378 #9507 (filimonov)
  • Fixed exception in DROP TABLE IF EXISTS #8663 (Nikita Vasilev)
  • Fix crash when a user tries to ALTER MODIFY SETTING for old-formated MergeTree table engines family. #9435 (alesapin)
  • Support for UInt64 numbers that don't fit in Int64 in JSON-related functions. Update SIMDJSON to master. This fixes #9209 #9344 (alexey-milovidov)
  • Fixed execution of inversed predicates when non-strictly monotinic functional index is used. #9223 (Alexander Kazakov)
  • Don't try to fold IN constant in GROUP BY #8868 (Amos Bird)
  • Fix bug in ALTER DELETE mutations which leads to index corruption. This fixes #9019 and #8982. Additionally fix extremely rare race conditions in ReplicatedMergeTree ALTER queries. #9048 (alesapin)
  • When the setting compile_expressions is enabled, you can get unexpected column in LLVMExecutableFunction when we use Nullable type #8910 (Guillaume Tassery)
  • Multiple fixes for Kafka engine: 1) fix duplicates that were appearing during consumer group rebalance. 2) Fix rare 'holes' appeared when data were polled from several partitions with one poll and committed partially (now we always process / commit the whole polled block of messages). 3) Fix flushes by block size (before that only flushing by timeout was working properly). 4) better subscription procedure (with assignment feedback). 5) Make tests work faster (with default intervals and timeouts). Due to the fact that data was not flushed by block size before (as it should according to documentation), that PR may lead to some performance degradation with default settings (due to more often & tinier flushes which are less optimal). If you encounter the performance issue after that change - please increase kafka_max_block_size in the table to the bigger value ( for example CREATE TABLE ...Engine=Kafka ... SETTINGS ... kafka_max_block_size=524288). Fixes #7259 #8917 (filimonov)
  • Fix Parameter out of bound exception in some queries after PREWHERE optimizations. #8914 (Baudouin Giard)
  • Fixed the case of mixed-constness of arguments of function arrayZip. #8705 (alexey-milovidov)
  • When executing CREATE query, fold constant expressions in storage engine arguments. Replace empty database name with current database. Fixes #6508, #3492 #9262 (tavplubix)
  • Now it's not possible to create or add columns with simple cyclic aliases like a DEFAULT b, b DEFAULT a. #9603 (alesapin)
  • Fixed a bug with double move which may corrupt original part. This is relevant if you use ALTER TABLE MOVE #8680 (Vladimir Chebotarev)
  • Allow interval identifier to correctly parse without backticks. Fixed issue when a query cannot be executed even if the interval identifier is enclosed in backticks or double quotes. This fixes #9124. #9142 (alexey-milovidov)
  • Fixed fuzz test and incorrect behaviour of bitTestAll/bitTestAny functions. #9143 (alexey-milovidov)
  • Fix possible crash/wrong number of rows in LIMIT n WITH TIES when there are a lot of rows equal to n'th row. #9464 (tavplubix)
  • Fix mutations with parts written with enabled insert_quorum. #9463 (alesapin)
  • Fix data race at destruction of Poco::HTTPServer. It could happen when server is started and immediately shut down. #9468 (Anton Popov)
  • Fix bug in which a misleading error message was shown when running SHOW CREATE TABLE a_table_that_does_not_exist. #8899 (achulkov2)
  • Fixed Parameters are out of bound exception in some rare cases when we have a constant in the SELECT clause when we have an ORDER BY and a LIMIT clause. #8892 (Guillaume Tassery)
  • Fix mutations finalization, when already done mutation can have status is_done=0. #9217 (alesapin)
  • Prevent from executing ALTER ADD INDEX for MergeTree tables with old syntax, because it doesn't work. #8822 (Mikhail Korotov)
  • During server startup do not access table, which LIVE VIEW depends on, so server will be able to start. Also remove LIVE VIEW dependencies when detaching LIVE VIEW. LIVE VIEW is an experimental feature. #8824 (tavplubix)
  • Fix possible segfault in MergeTreeRangeReader, while executing PREWHERE. #9106 (Anton Popov)
  • Fix possible mismatched checksums with column TTLs. #9451 (Anton Popov)
  • Fixed a bug when parts were not being moved in background by TTL rules in case when there is only one volume. #8672 (Vladimir Chebotarev)
  • Fixed the issue Method createColumn() is not implemented for data type Set. This fixes #7799. #8674 (alexey-milovidov)
  • Now we will try finalize mutations more frequently. #9427 (alesapin)
  • Fix intDiv by minus one constant #9351 (hcz)
  • Fix possible race condition in BlockIO. #9356 (Nikolai Kochetov)
  • Fix bug leading to server termination when trying to use / drop Kafka table created with wrong parameters. #9513 (filimonov)
  • Added workaround if OS returns wrong result for timer_create function. #8837 (alexey-milovidov)
  • Fixed error in usage of min_marks_for_seek parameter. Fixed the error message when there is no sharding key in Distributed table and we try to skip unused shards. #8908 (Azat Khuzhin)

Improvement

  • Implement ALTER MODIFY/DROP queries on top of mutations for ReplicatedMergeTree* engines family. Now ALTERS blocks only at the metadata update stage, and don't block after that. #8701 (alesapin)
  • Add ability to rewrite CROSS to INNER JOINs with WHERE section containing unqialified names. #9512 (Artem Zuikov)
  • Make SHOW TABLES and SHOW DATABASES queries support the WHERE expressions and FROM/IN #9076 (sundyli)
  • Added a setting deduplicate_blocks_in_dependent_materialized_views. #9070 (urykhy)
  • After recent changes MySQL client started to print binary strings in hex thereby making them not readable (#9032). The workaround in ClickHouse is to mark string columns as UTF-8, which is not always, but usually the case. #9079 (Yuriy Baranov)
  • Add support of String and FixedString keys for sumMap #8903 (Baudouin Giard)
  • Support string keys in SummingMergeTree maps #8933 (Baudouin Giard)
  • Signal termination of thread to the thread pool even if the thread has thrown exception #8736 (Ding Xiang Fei)
  • Allow to set query_id in clickhouse-benchmark #9416 (Anton Popov)
  • Don't allow strange expressions in ALTER TABLE ... PARTITION partition query. This addresses #7192 #8835 (alexey-milovidov)
  • The table system.table_engines now provides information about feature support (like supports_ttl or supports_sort_order). #8830 (Max Akhmedov)
  • Enable system.metric_log by default. It will contain rows with values of ProfileEvents, CurrentMetrics collected with "collect_interval_milliseconds" interval (one second by default). The table is very small (usually in order of megabytes) and collecting this data by default is reasonable. #9225 (alexey-milovidov)
  • Initialize query profiler for all threads in a group, e.g. it allows to fully profile insert-queries. Fixes #6964 #8874 (Ivan)
  • Now temporary LIVE VIEW is created by CREATE LIVE VIEW name WITH TIMEOUT [42] ... instead of CREATE TEMPORARY LIVE VIEW ..., because the previous syntax was not consistent with CREATE TEMPORARY TABLE ... #9131 (tavplubix)
  • Add text_log.level configuration parameter to limit entries that goes to system.text_log table #8809 (Azat Khuzhin)
  • Allow to put downloaded part to a disks/volumes according to TTL rules #8598 (Vladimir Chebotarev)
  • For external MySQL dictionaries, allow to mutualize MySQL connection pool to "share" them among dictionaries. This option significantly reduces the number of connections to MySQL servers. #9409 (Clément Rodriguez)
  • Show nearest query execution time for quantiles in clickhouse-benchmark output instead of interpolated values. It's better to show values that correspond to the execution time of some queries. #8712 (alexey-milovidov)
  • Possibility to add key & timestamp for the message when inserting data to Kafka. Fixes #7198 #8969 (filimonov)
  • If server is run from terminal, highlight thread number, query id and log priority by colors. This is for improved readability of correlated log messages for developers. #8961 (alexey-milovidov)
  • Better exception message while loading tables for Ordinary database. #9527 (alexey-milovidov)
  • Implement arraySlice for arrays with aggregate function states. This fixes #9388 #9391 (alexey-milovidov)
  • Allow constant functions and constant arrays to be used on the right side of IN operator. #8813 (Anton Popov)
  • If zookeeper exception has happened while fetching data for system.replicas, display it in a separate column. This implements #9137 #9138 (alexey-milovidov)
  • Atomically remove MergeTree data parts on destroy. #8402 (Vladimir Chebotarev)
  • Support row-level security for Distributed tables. #8926 (Ivan)
  • Now we recognize suffix (like KB, KiB...) in settings values. #8072 (Mikhail Korotov)
  • Prevent out of memory while constructing result of a large JOIN. #8637 (Artem Zuikov)
  • Added names of clusters to suggestions in interactive mode in clickhouse-client. #8709 (alexey-milovidov)
  • Initialize query profiler for all threads in a group, e.g. it allows to fully profile insert-queries #8820 (Ivan)
  • Added column exception_code in system.query_log table. #8770 (Mikhail Korotov)
  • Enabled MySQL compatibility server on port 9004 in the default server configuration file. Fixed password generation command in the example in configuration. #8771 (Yuriy Baranov)
  • Prevent abort on shutdown if the filesystem is readonly. This fixes #9094 #9100 (alexey-milovidov)
  • Better exception message when length is required in HTTP POST query. #9453 (alexey-milovidov)
  • Add _path and _file virtual columns to HDFS and File engines and hdfs and file table functions #8489 (Olga Khvostikova)
  • Fix error Cannot find column while inserting into MATERIALIZED VIEW in case if new column was added to view's internal table. #8766 #8788 (vzakaznikov) #8788 #8806 (Nikolai Kochetov) #8803 (Nikolai Kochetov)
  • Fix progress over native client-server protocol, by send progress after final update (like logs). This may be relevant only to some third-party tools that are using native protocol. #9495 (Azat Khuzhin)
  • Add a system metric tracking the number of client connections using MySQL protocol (#9013). #9015 (Eugene Klimov)
  • From now on, HTTP responses will have X-ClickHouse-Timezone header set to the same timezone value that SELECT timezone() would report. #9493 (Denis Glazachev)

Performance Improvement

  • Improve performance of analysing index with IN #9261 (Anton Popov)
  • Simpler and more efficient code in Logical Functions + code cleanups. A followup to #8718 #8728 (Alexander Kazakov)
  • Overall performance improvement (in range of 5%..200% for affected queries) by ensuring even more strict aliasing with C++20 features. #9304 (Amos Bird)
  • More strict aliasing for inner loops of comparison functions. #9327 (alexey-milovidov)
  • More strict aliasing for inner loops of arithmetic functions. #9325 (alexey-milovidov)
  • A ~3 times faster implementation for ColumnVector::replicate(), via which ColumnConst::convertToFullColumn() is implemented. Also will be useful in tests when materializing constants. #9293 (Alexander Kazakov)
  • Another minor performance improvement to ColumnVector::replicate() (this speeds up the materialize function and higher order functions) an even further improvement to #9293 #9442 (Alexander Kazakov)
  • Improved performance of stochasticLinearRegression aggregate function. This patch is contributed by Intel. #8652 (alexey-milovidov)
  • Improve performance of reinterpretAsFixedString function. #9342 (alexey-milovidov)
  • Do not send blocks to client for Null format in processors pipeline. #8797 (Nikolai Kochetov) #8767 (Alexander Kuzmenkov)

Build/Testing/Packaging Improvement

ClickHouse release v20.1

ClickHouse release v20.1.16.120-stable 2020-60-26

Bug Fix

  • Fix rare crash caused by using Nullable column in prewhere condition. Continuation of #11608. #11869 (Nikolai Kochetov).
  • Don't allow arrayJoin inside higher order functions. It was leading to broken protocol synchronization. This closes #3933. #11846 (alexey-milovidov).
  • Fix unexpected behaviour of queries like SELECT *, xyz.* which were success while an error expected. #11753 (hexiaoting).
  • Fixed LOGICAL_ERROR caused by wrong type deduction of complex literals in Values input format. #11732 (tavplubix).
  • Fix ORDER BY ... WITH FILL over const columns. #11697 (Anton Popov).
  • Pass proper timeouts when communicating with XDBC bridge. Recently timeouts were not respected when checking bridge liveness and receiving meta info. #11690 (alexey-milovidov).
  • Add support for regular expressions with case-insensitive flags. This fixes #11101 and fixes #11506. #11649 (alexey-milovidov).
  • Fix bloom filters for String (data skipping indices). #11638 (Azat Khuzhin).
  • Fix rare crash caused by using Nullable column in prewhere condition. (Probably it is connected with #11572 somehow). #11608 (Nikolai Kochetov).
  • Fix wrong exit code of the clickhouse-client, when exception.code() % 256 = 0. #11601 (filimonov).
  • Fix trivial error in log message about "Mark cache size was lowered" at server startup. This closes #11399. #11589 (alexey-milovidov).
  • Now clickhouse-server docker container will prefer IPv6 checking server aliveness. #11550 (Ivan Starkov).
  • Fix memory leak when exception is thrown in the middle of aggregation with -State functions. This fixes #8995. #11496 (alexey-milovidov).
  • Fix usage of primary key wrapped into a function with 'FINAL' modifier and 'ORDER BY' optimization. #10715 (Anton Popov).

ClickHouse release v20.1.15.109-stable 2020-06-19

Bug Fix

ClickHouse release v20.1.14.107-stable 2020-06-11

Bug Fix

  • Fix error Size of offsets doesn't match size of column for queries with PREWHERE column in (subquery) and ARRAY JOIN. #11580 (Nikolai Kochetov).

ClickHouse release v20.1.13.105-stable 2020-06-10

Bug Fix

  • Fix the error Data compressed with different methods that can happen if min_bytes_to_use_direct_io is enabled and PREWHERE is active and using SAMPLE or high number of threads. This fixes #11539. #11540 (alexey-milovidov).
  • Fix return compressed size for codecs. #11448 (Nikolai Kochetov).
  • Fix server crash when a column has compression codec with non-literal arguments. Fixes #11365. #11431 (alesapin).
  • Fix pointInPolygon with nan as point. Fixes #11375. #11421 (Alexey Ilyukhov).
  • Fixed geohashesInBox with arguments outside of latitude/longitude range. #11403 (Vasily Nemkov).
  • Fix possible Pipeline stuck error for queries with external sort and limit. Fixes #11359. #11366 (Nikolai Kochetov).
  • Fix crash in quantilesExactWeightedArray. #11337 (Nikolai Kochetov).
  • Make writing to MATERIALIZED VIEW with setting parallel_view_processing = 1 parallel again. Fixes #10241. #11330 (Nikolai Kochetov).
  • Fix visitParamExtractRaw when extracted JSON has strings with unbalanced { or [. #11318 (Ewout).
  • Fix very rare race condition in ThreadPool. #11314 (alexey-milovidov).
  • Fix potential uninitialized memory in conversion. Example: SELECT toIntervalSecond(now64()). #11311 (alexey-milovidov).
  • Fix the issue when index analysis cannot work if a table has Array column in primary key and if a query is filtering by this column with empty or notEmpty functions. This fixes #11286. #11303 (alexey-milovidov).
  • Fix bug when query speed estimation can be incorrect and the limit of min_execution_speed may not work or work incorrectly if the query is throttled by max_network_bandwidth, max_execution_speed or priority settings. Change the default value of timeout_before_checking_execution_speed to non-zero, because otherwise the settings min_execution_speed and max_execution_speed have no effect. This fixes #11297. This fixes #5732. This fixes #6228. Usability improvement: avoid concatenation of exception message with progress bar in clickhouse-client. #11296 (alexey-milovidov).
  • Fix crash while reading malformed data in Protobuf format. This fixes #5957, fixes #11203. #11258 (Vitaly Baranov).
  • Fix possible error Cannot capture column for higher-order functions with Array(Array(LowCardinality)) captured argument. #11185 (Nikolai Kochetov).
  • If data skipping index is dependent on columns that are going to be modified during background merge (for SummingMergeTree, AggregatingMergeTree as well as for TTL GROUP BY), it was calculated incorrectly. This issue is fixed by moving index calculation after merge so the index is calculated on merged data. #11162 (Azat Khuzhin).
  • Remove logging from mutation finalization task if nothing was finalized. #11109 (alesapin).
  • Fixed parseDateTime64BestEffort argument resolution bugs. #10925. #11038 (Vasily Nemkov).
  • Fix incorrect raw data size in method getRawData(). #10964 (Igr).
  • Fix backward compatibility with tuples in Distributed tables. #10889 (Anton Popov).
  • Fix SIGSEGV in StringHashTable (if such key does not exist). #10870 (Azat Khuzhin).
  • Fixed bug in ReplicatedMergeTree which might cause some ALTER on OPTIMIZE query to hang waiting for some replica after it become inactive. #10849 (tavplubix).
  • Fix columns order after Block::sortColumns() (also add a test that shows that it affects some real use case - Buffer engine). #10826 (Azat Khuzhin).
  • Fix the issue with ODBC bridge when no quoting of identifiers is requested. This fixes #7984. #10821 (alexey-milovidov).
  • Fix UBSan and MSan report in DateLUT. #10798 (alexey-milovidov).
  • Fix parallel_view_processing behavior. Now all insertions into MATERIALIZED VIEW without exception should be finished if exception happened. Fixes #10241. #10757 (Nikolai Kochetov).
  • Fix combinator -OrNull and -OrDefault when combined with -State. #10741 (hcz).
  • Fix disappearing totals. Totals could have being filtered if query had had join or subquery with external where condition. Fixes #10674. #10698 (Nikolai Kochetov).
  • Fix multiple usages of IN operator with the identical set in one query. #10686 (Anton Popov).
  • Fix order of parameters in AggregateTransform constructor. #10667 (palasonic1).
  • Fix the lack of parallel execution of remote queries with distributed_aggregation_memory_efficient enabled. Fixes #10655. #10664 (Nikolai Kochetov).
  • Fix predicates optimization for distributed queries (enable_optimize_predicate_expression=1) for queries with HAVING section (i.e. when filtering on the server initiator is required), by preserving the order of expressions (and this is enough to fix), and also force aggregator use column names over indexes. Fixes: #10613, #11413. #10621 (Azat Khuzhin).
  • Fix error the BloomFilter false positive must be a double number between 0 and 1 #10551. #10569 (Winter Zhang).
  • Fix SELECT of column ALIAS which default expression type different from column type. #10563 (Azat Khuzhin).
    • Implemented comparison between DateTime64 and String values (just like for DateTime). #10560 (Vasily Nemkov).

ClickHouse release v20.1.12.86, 2020-05-26

Bug Fix

  • Fixed incompatibility of two-level aggregation between versions 20.1 and earlier. This incompatibility happens when different versions of ClickHouse are used on initiator node and remote nodes and the size of GROUP BY result is large and aggregation is performed by a single String field. It leads to several unmerged rows for a single key in result. #10952 (alexey-milovidov).
  • Fixed data corruption for LowCardinality(FixedString) key column in SummingMergeTree which could have happened after merge. Fixes #10489. #10721 (Nikolai Kochetov).
  • Fixed bug, which causes http requests stuck on client close when readonly=2 and cancel_http_readonly_queries_on_client_close=1. Fixes #7939, #7019, #7736, #7091. #10684 (tavplubix).
  • Fixed a bug when on SYSTEM DROP DNS CACHE query also drop caches, which are used to check if user is allowed to connect from some IP addresses. #10608 (tavplubix).
  • Fixed incorrect scalar results inside inner query of MATERIALIZED VIEW in case if this query contained dependent table. #10603 (Nikolai Kochetov).
  • Fixed the situation when mutation finished all parts, but hung up in is_done=0. #10526 (alesapin).
  • Fixed overflow at beginning of unix epoch for timezones with fractional offset from UTC. This fixes #9335. #10513 (alexey-milovidov).
  • Fixed improper shutdown of Distributed storage. #10491 (Azat Khuzhin).
  • Fixed numeric overflow in simpleLinearRegression over large integers. #10474 (hcz).
  • Fixed removing metadata directory when attach database fails. #10442 (Winter Zhang).
  • Added a check of number and type of arguments when creating BloomFilter index #9623. #10431 (Winter Zhang).
  • Fixed the issue when a query with ARRAY JOIN, ORDER BY and LIMIT may return incomplete result. This fixes #10226. #10427 (alexey-milovidov).
  • Prefer fallback_to_stale_replicas over skip_unavailable_shards. #10422 (Azat Khuzhin).
  • Fixed wrong flattening of Array(Tuple(...)) data types. This fixes #10259. #10390 (alexey-milovidov).
  • Fixed wrong behavior in HashTable that caused compilation error when trying to read HashMap from buffer. #10386 (palasonic1).
  • Fixed possible Pipeline stuck error in ConcatProcessor which could have happened in remote query. #10381 (Nikolai Kochetov).
  • Fixed error Pipeline stuck with max_rows_to_group_by and group_by_overflow_mode = 'break'. #10279 (Nikolai Kochetov).
  • Fixed several bugs when some data was inserted with quorum, then deleted somehow (DROP PARTITION, TTL) and this leaded to the stuck of INSERTs or false-positive exceptions in SELECTs. This fixes #9946. #10188 (Nikita Mikhaylov).
  • Fixed incompatibility when versions prior to 18.12.17 are used on remote servers and newer is used on initiating server, and GROUP BY both fixed and non-fixed keys, and when two-level group by method is activated. #3254 (alexey-milovidov).

Build/Testing/Packaging Improvement

  • Added CA certificates to clickhouse-server docker image. #10476 (filimonov).

ClickHouse release v20.1.10.70, 2020-04-17

Bug Fix

  • Fix rare possible exception Cannot drain connections: cancel first. #10239 (Nikolai Kochetov).
  • Fixed bug where ClickHouse would throw 'Unknown function lambda.' error message when user tries to run ALTER UPDATE/DELETE on tables with ENGINE = Replicated*. Check for nondeterministic functions now handles lambda expressions correctly. #10237 (Alexander Kazakov).
  • Fix parseDateTimeBestEffort for strings in RFC-2822 when day of week is Tuesday or Thursday. This fixes #10082. #10214 (alexey-milovidov).
  • Fix column names of constants inside JOIN that may clash with names of constants outside of JOIN. #10207 (alexey-milovidov).
  • Fix possible inifinite query execution when the query actually should stop on LIMIT, while reading from infinite source like system.numbers or system.zeros. #10206 (Nikolai Kochetov).
  • Fix move-to-prewhere optimization in presense of arrayJoin functions (in certain cases). This fixes #10092. #10195 (alexey-milovidov).
  • Add the ability to relax the restriction on non-deterministic functions usage in mutations with allow_nondeterministic_mutations setting. #10186 (filimonov).
  • Convert blocks if structure does not match on INSERT into table with Distributed engine. #10135 (Azat Khuzhin).
  • Fix SIGSEGV on INSERT into Distributed table when its structure differs from the underlying tables. #10105 (Azat Khuzhin).
  • Fix possible rows loss for queries with JOIN and UNION ALL. Fixes #9826, #10113. #10099 (Nikolai Kochetov).
  • Add arguments check and support identifier arguments for MySQL Database Engine. #10077 (Winter Zhang).
  • Fix bug in clickhouse dictionary source from localhost clickhouse server. The bug may lead to memory corruption if types in dictionary and source are not compatible. #10071 (alesapin).
  • Fix error Cannot clone block with columns because block has 0 columns ... While executing GroupingAggregatedTransform. It happened when setting distributed_aggregation_memory_efficient was enabled, and distributed query read aggregating data with different level from different shards (mixed single and two level aggregation). #10063 (Nikolai Kochetov).
  • Fix a segmentation fault that could occur in GROUP BY over string keys containing trailing zero bytes (#8636, #8925). #10025 (Alexander Kuzmenkov).
  • Fix bug in which the necessary tables weren't retrieved at one of the processing stages of queries to some databases. Fixes #9699. #9949 (achulkov2).
  • Fix 'Not found column in block' error when JOIN appears with TOTALS. Fixes #9839. #9939 (Artem Zuikov).
  • Fix a bug with ON CLUSTER DDL queries freezing on server startup. #9927 (Gagan Arneja).
  • Fix TRUNCATE for Join table engine (#9917). #9920 (Amos Bird).
  • Fix 'scalar doesn't exist' error in ALTER queries (#9878). #9904 (Amos Bird).
  • Fix race condition between drop and optimize in ReplicatedMergeTree. #9901 (alesapin).
  • Fixed DeleteOnDestroy logic in ATTACH PART which could lead to automatic removal of attached part and added few tests. #9410 (Vladimir Chebotarev).

Build/Testing/Packaging Improvement

ClickHouse release v20.1.9.54, 2020-03-28

Bug Fix

  • Fix 'Different expressions with the same alias' error when query has PREWHERE and WHERE on distributed table and SET distributed_product_mode = 'local'. #9871 (Artem Zuikov).
  • Fix mutations excessive memory consumption for tables with a composite primary key. This fixes #9850. #9860 (alesapin).
  • For INSERT queries shard now clamps the settings got from the initiator to the shard's constaints instead of throwing an exception. This fix allows to send INSERT queries to a shard with another constraints. This change improves fix #9447. #9852 (Vitaly Baranov).
  • Fix possible exception Got 0 in totals chunk, expected 1 on client. It happened for queries with JOIN in case if right joined table had zero rows. Example: select * from system.one t1 join system.one t2 on t1.dummy = t2.dummy limit 0 FORMAT TabSeparated;. Fixes #9777. #9823 (Nikolai Kochetov).
  • Fix SIGSEGV with optimize_skip_unused_shards when type cannot be converted. #9804 (Azat Khuzhin).
  • Fixed a few cases when timezone of the function argument wasn't used properly. #9574 (Vasily Nemkov).

Improvement

  • Remove ORDER BY stage from mutations because we read from a single ordered part in a single thread. Also add check that the order of rows in mutation is ordered in sorting key order and this order is not violated. #9886 (alesapin).

Build/Testing/Packaging Improvement

  • Clean up duplicated linker flags. Make sure the linker won't look up an unexpected symbol. #9433 (Amos Bird).

ClickHouse release v20.1.8.41, 2020-03-20

Bug Fix

  • Fix possible permanent Cannot schedule a task error (due to unhandled exception in ParallelAggregatingBlockInputStream::Handler::onFinish/onFinishThread). This fixes #6833. #9154 (Azat Khuzhin)
  • Fix excessive memory consumption in ALTER queries (mutations). This fixes #9533 and #9670. #9754 (alesapin)
  • Fix bug in backquoting in external dictionaries DDL. This fixes #9619. #9734 (alesapin)

ClickHouse release v20.1.7.38, 2020-03-18

Bug Fix

  • Fixed incorrect internal function names for sumKahan and sumWithOverflow. I lead to exception while using this functions in remote queries. #9636 (Azat Khuzhin). This issue was in all ClickHouse releases.
  • Allow ALTER ON CLUSTER of Distributed tables with internal replication. This fixes #3268. #9617 (shinoi2). This issue was in all ClickHouse releases.
  • Fix possible exceptions Size of filter doesn't match size of column and Invalid number of rows in Chunk in MergeTreeRangeReader. They could appear while executing PREWHERE in some cases. Fixes #9132. #9612 (Anton Popov)
  • Fixed the issue: timezone was not preserved if you write a simple arithmetic expression like time + 1 (in contrast to an expression like time + INTERVAL 1 SECOND). This fixes #5743. #9323 (alexey-milovidov). This issue was in all ClickHouse releases.
  • Now it's not possible to create or add columns with simple cyclic aliases like a DEFAULT b, b DEFAULT a. #9603 (alesapin)
  • Fixed the issue when padding at the end of base64 encoded value can be malformed. Update base64 library. This fixes #9491, closes #9492 #9500 (alexey-milovidov)
  • Fix data race at destruction of Poco::HTTPServer. It could happen when server is started and immediately shut down. #9468 (Anton Popov)
  • Fix possible crash/wrong number of rows in LIMIT n WITH TIES when there are a lot of rows equal to n'th row. #9464 (tavplubix)
  • Fix possible mismatched checksums with column TTLs. #9451 (Anton Popov)
  • Fix crash when a user tries to ALTER MODIFY SETTING for old-formated MergeTree table engines family. #9435 (alesapin)
  • Now we will try finalize mutations more frequently. #9427 (alesapin)
  • Fix replication protocol incompatibility introduced in #8598. #9412 (alesapin)
  • Fix not(has()) for the bloom_filter index of array types. #9407 (achimbab)
  • Fixed the behaviour of match and extract functions when haystack has zero bytes. The behaviour was wrong when haystack was constant. This fixes #9160 #9163 (alexey-milovidov) #9345 (alexey-milovidov)

Build/Testing/Packaging Improvement

ClickHouse release v20.1.6.30, 2020-03-05

Bug Fix

  • Fix data incompatibility when compressed with T64 codec. #9039 (abyss7)
  • Fix order of ranges while reading from MergeTree table in one thread. Fixes #8964. #9050 (CurtizJ)
  • Fix possible segfault in MergeTreeRangeReader, while executing PREWHERE. Fixes #9064. #9106 (CurtizJ)
  • Fix reinterpretAsFixedString to return FixedString instead of String. #9052 (oandrew)
  • Fix joinGet with nullable return types. Fixes #8919 #9014 (amosbird)
  • Fix fuzz test and incorrect behaviour of bitTestAll/bitTestAny functions. #9143 (alexey-milovidov)
  • Fix the behaviour of match and extract functions when haystack has zero bytes. The behaviour was wrong when haystack was constant. Fixes #9160 #9163 (alexey-milovidov)
  • Fixed execution of inversed predicates when non-strictly monotinic functional index is used. Fixes #9034 #9223 (Akazz)
  • Allow to rewrite CROSS to INNER JOIN if there's [NOT] LIKE operator in WHERE section. Fixes #9191 #9229 (4ertus2)
  • Allow first column(s) in a table with Log engine be an alias. #9231 (abyss7)
  • Allow comma join with IN() inside. Fixes #7314. #9251 (4ertus2)
  • Improve ALTER MODIFY/ADD queries logic. Now you cannot ADD column without type, MODIFY default expression doesn't change type of column and MODIFY type doesn't loose default expression value. Fixes #8669. #9227 (alesapin)
  • Fix mutations finalization, when already done mutation can have status is_done=0. #9217 (alesapin)
  • Support "Processors" pipeline for system.numbers and system.numbers_mt. This also fixes the bug when max_execution_time is not respected. #7796 (KochetovNicolai)
  • Fix wrong counting of DictCacheKeysRequestedFound metric. #9411 (nikitamikhaylov)
  • Added a check for storage policy in ATTACH PARTITION FROM, REPLACE PARTITION, MOVE TO TABLE which otherwise could make data of part inaccessible after restart and prevent ClickHouse to start. #9383 (excitoon)
  • Fixed UBSan report in MergeTreeIndexSet. This fixes #9250 #9365 (alexey-milovidov)
  • Fix possible datarace in BlockIO. #9356 (KochetovNicolai)
  • Support for UInt64 numbers that don't fit in Int64 in JSON-related functions. Update SIMDJSON to master. This fixes #9209 #9344 (alexey-milovidov)
  • Fix the issue when the amount of free space is not calculated correctly if the data directory is mounted to a separate device. For default disk calculate the free space from data subdirectory. This fixes #7441 #9257 (millb)
  • Fix the issue when TLS connections may fail with the message OpenSSL SSL_read: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error and SSL Exception: error:2400006E:random number generator::error retrieving entropy. Update OpenSSL to upstream master. #8956 (alexey-milovidov)
  • When executing CREATE query, fold constant expressions in storage engine arguments. Replace empty database name with current database. Fixes #6508, #3492. Also fix check for local address in ClickHouseDictionarySource. #9262 (tabplubix)
  • Fix segfault in StorageMerge, which can happen when reading from StorageFile. #9387 (tabplubix)
  • Prevent losing data in Kafka in rare cases when exception happens after reading suffix but before commit. Fixes #9378. Related: #7175 #9507 (filimonov)
  • Fix bug leading to server termination when trying to use / drop Kafka table created with wrong parameters. Fixes #9494. Incorporates #9507. #9513 (filimonov)

New Feature

  • Add deduplicate_blocks_in_dependent_materialized_views option to control the behaviour of idempotent inserts into tables with materialized views. This new feature was added to the bugfix release by a special request from Altinity. #9070 (urykhy)

ClickHouse release v20.1.2.4, 2020-01-22

Backward Incompatible Change

  • Make the setting merge_tree_uniform_read_distribution obsolete. The server still recognizes this setting but it has no effect. #8308 (alexey-milovidov)
  • Changed return type of the function greatCircleDistance to Float32 because now the result of calculation is Float32. #7993 (alexey-milovidov)
  • Now it's expected that query parameters are represented in "escaped" format. For example, to pass string a<tab>b you have to write a\tb or a\<tab>b and respectively, a%5Ctb or a%5C%09b in URL. This is needed to add the possibility to pass NULL as \N. This fixes #7488. #8517 (alexey-milovidov)
  • Enable use_minimalistic_part_header_in_zookeeper setting for ReplicatedMergeTree by default. This will significantly reduce amount of data stored in ZooKeeper. This setting is supported since version 19.1 and we already use it in production in multiple services without any issues for more than half a year. Disable this setting if you have a chance to downgrade to versions older than 19.1. #6850 (alexey-milovidov)
  • Data skipping indices are production ready and enabled by default. The settings allow_experimental_data_skipping_indices, allow_experimental_cross_to_join_conversion and allow_experimental_multiple_joins_emulation are now obsolete and do nothing. #7974 (alexey-milovidov)
  • Add new ANY JOIN logic for StorageJoin consistent with JOIN operation. To upgrade without changes in behaviour you need add SETTINGS any_join_distinct_right_table_keys = 1 to Engine Join tables metadata or recreate these tables after upgrade. #8400 (Artem Zuikov)
  • Require server to be restarted to apply the changes in logging configuration. This is a temporary workaround to avoid the bug where the server logs to a deleted log file (see #8696). #8707 (Alexander Kuzmenkov)

New Feature

  • Added information about part paths to system.merges. #8043 (Vladimir Chebotarev)
  • Add ability to execute SYSTEM RELOAD DICTIONARY query in ON CLUSTER mode. #8288 (Guillaume Tassery)
  • Add ability to execute CREATE DICTIONARY queries in ON CLUSTER mode. #8163 (alesapin)
  • Now user's profile in users.xml can inherit multiple profiles. #8343 (Mikhail f. Shiryaev)
  • Added system.stack_trace table that allows to look at stack traces of all server threads. This is useful for developers to introspect server state. This fixes #7576. #8344 (alexey-milovidov)
  • Add DateTime64 datatype with configurable sub-second precision. #7170 (Vasily Nemkov)
  • Add table function clusterAllReplicas which allows to query all the nodes in the cluster. #8493 (kiran sunkari)
  • Add aggregate function categoricalInformationValue which calculates the information value of a discrete feature. #8117 (hcz)
  • Speed up parsing of data files in CSV, TSV and JSONEachRow format by doing it in parallel. #7780 (Alexander Kuzmenkov)
  • Add function bankerRound which performs banker's rounding. #8112 (hcz)
  • Support more languages in embedded dictionary for region names: 'ru', 'en', 'ua', 'uk', 'by', 'kz', 'tr', 'de', 'uz', 'lv', 'lt', 'et', 'pt', 'he', 'vi'. #8189 (alexey-milovidov)
  • Improvements in consistency of ANY JOIN logic. Now t1 ANY LEFT JOIN t2 equals t2 ANY RIGHT JOIN t1. #7665 (Artem Zuikov)
  • Add setting any_join_distinct_right_table_keys which enables old behaviour for ANY INNER JOIN. #7665 (Artem Zuikov)
  • Add new SEMI and ANTI JOIN. Old ANY INNER JOIN behaviour now available as SEMI LEFT JOIN. #7665 (Artem Zuikov)
  • Added Distributed format for File engine and file table function which allows to read from .bin files generated by asynchronous inserts into Distributed table. #8535 (Nikolai Kochetov)
  • Add optional reset column argument for runningAccumulate which allows to reset aggregation results for each new key value. #8326 (Sergey Kononenko)
  • Add ability to use ClickHouse as Prometheus endpoint. #7900 (vdimir)
  • Add section <remote_url_allow_hosts> in config.xml which restricts allowed hosts for remote table engines and table functions URL, S3, HDFS. #7154 (Mikhail Korotov)
  • Added function greatCircleAngle which calculates the distance on a sphere in degrees. #8105 (alexey-milovidov)
  • Changed Earth radius to be consistent with H3 library. #8105 (alexey-milovidov)
  • Added JSONCompactEachRow and JSONCompactEachRowWithNamesAndTypes formats for input and output. #7841 (Mikhail Korotov)
  • Added feature for file-related table engines and table functions (File, S3, URL, HDFS) which allows to read and write gzip files based on additional engine parameter or file extension. #7840 (Andrey Bodrov)
  • Added the randomASCII(length) function, generating a string with a random set of ASCII printable characters. #8401 (BayoNet)
  • Added function JSONExtractArrayRaw which returns an array on unparsed json array elements from JSON string. #8081 (Oleg Matrokhin)
  • Add arrayZip function which allows to combine multiple arrays of equal lengths into one array of tuples. #8149 (Winter Zhang)
  • Add ability to move data between disks according to configured TTL-expressions for *MergeTree table engines family. #8140 (Vladimir Chebotarev)
  • Added new aggregate function avgWeighted which allows to calculate weighted average. #7898 (Andrey Bodrov)
  • Now parallel parsing is enabled by default for TSV, TSKV, CSV and JSONEachRow formats. #7894 (Nikita Mikhaylov)
  • Add several geo functions from H3 library: h3GetResolution, h3EdgeAngle, h3EdgeLength, h3IsValid and h3kRing. #8034 (Konstantin Malanchev)
  • Added support for brotli (br) compression in file-related storages and table functions. This fixes #8156. #8526 (alexey-milovidov)
  • Add groupBit* functions for the SimpleAggregationFunction type. #8485 (Guillaume Tassery)

Bug Fix

  • Fix rename of tables with Distributed engine. Fixes issue #7868. #8306 (tavplubix)
  • Now dictionaries support EXPRESSION for attributes in arbitrary string in non-ClickHouse SQL dialect. #8098 (alesapin)
  • Fix broken INSERT SELECT FROM mysql(...) query. This fixes #8070 and #7960. #8234 (tavplubix)
  • Fix error "Mismatch column sizes" when inserting default Tuple from JSONEachRow. This fixes #5653. #8606 (tavplubix)
  • Now an exception will be thrown in case of using WITH TIES alongside LIMIT BY. Also add ability to use TOP with LIMIT BY. This fixes #7472. #7637 (Nikita Mikhaylov)
  • Fix unintendent dependency from fresh glibc version in clickhouse-odbc-bridge binary. #8046 (Amos Bird)
  • Fix bug in check function of *MergeTree engines family. Now it doesn't fail in case when we have equal amount of rows in last granule and last mark (non-final). #8047 (alesapin)
  • Fix insert into Enum* columns after ALTER query, when underlying numeric type is equal to table specified type. This fixes #7836. #7908 (Anton Popov)
  • Allowed non-constant negative "size" argument for function substring. It was not allowed by mistake. This fixes #4832. #7703 (alexey-milovidov)
  • Fix parsing bug when wrong number of arguments passed to (O|J)DBC table engine. #7709 (alesapin)
  • Using command name of the running clickhouse process when sending logs to syslog. In previous versions, empty string was used instead of command name. #8460 (Michael Nacharov)
  • Fix check of allowed hosts for localhost. This PR fixes the solution provided in #8241. #8342 (Vitaly Baranov)
  • Fix rare crash in argMin and argMax functions for long string arguments, when result is used in runningAccumulate function. This fixes #8325 #8341 (dinosaur)
  • Fix memory overcommit for tables with Buffer engine. #8345 (Azat Khuzhin)
  • Fixed potential bug in functions that can take NULL as one of the arguments and return non-NULL. #8196 (alexey-milovidov)
  • Better metrics calculations in thread pool for background processes for MergeTree table engines. #8194 (Vladimir Chebotarev)
  • Fix function IN inside WHERE statement when row-level table filter is present. Fixes #6687 #8357 (Ivan)
  • Now an exception is thrown if the integral value is not parsed completely for settings values. #7678 (Mikhail Korotov)
  • Fix exception when aggregate function is used in query to distributed table with more than two local shards. #8164 (小路)
  • Now bloom filter can handle zero length arrays and doesn't perform redundant calculations. #8242 (achimbab)
  • Fixed checking if a client host is allowed by matching the client host to host_regexp specified in users.xml. #8241 (Vitaly Baranov)
  • Relax ambiguous column check that leads to false positives in multiple JOIN ON section. #8385 (Artem Zuikov)
  • Fixed possible server crash (std::terminate) when the server cannot send or write data in JSON or XML format with values of String data type (that require UTF-8 validation) or when compressing result data with Brotli algorithm or in some other rare cases. This fixes #7603 #8384 (alexey-milovidov)
  • Fix race condition in StorageDistributedDirectoryMonitor found by CI. This fixes #8364. #8383 (Nikolai Kochetov)
  • Now background merges in *MergeTree table engines family preserve storage policy volume order more accurately. #8549 (Vladimir Chebotarev)
  • Now table engine Kafka works properly with Native format. This fixes #6731 #7337 #8003. #8016 (filimonov)
  • Fixed formats with headers (like CSVWithNames) which were throwing exception about EOF for table engine Kafka. #8016 (filimonov)
  • Fixed a bug with making set from subquery in right part of IN section. This fixes #5767 and #2542. #7755 (Nikita Mikhaylov)
  • Fix possible crash while reading from storage File. #7756 (Nikolai Kochetov)
  • Fixed reading of the files in Parquet format containing columns of type list. #8334 (maxulan)
  • Fix error Not found column for distributed queries with PREWHERE condition dependent on sampling key if max_parallel_replicas > 1. #7913 (Nikolai Kochetov)
  • Fix error Not found column if query used PREWHERE dependent on table's alias and the result set was empty because of primary key condition. #7911 (Nikolai Kochetov)
  • Fixed return type for functions rand and randConstant in case of Nullable argument. Now functions always return UInt32 and never Nullable(UInt32). #8204 (Nikolai Kochetov)
  • Disabled predicate push-down for WITH FILL expression. This fixes #7784. #7789 (Winter Zhang)
  • Fixed incorrect count() result for SummingMergeTree when FINAL section is used. #3280 #7786 (Nikita Mikhaylov)
  • Fix possible incorrect result for constant functions from remote servers. It happened for queries with functions like version(), uptime(), etc. which returns different constant values for different servers. This fixes #7666. #7689 (Nikolai Kochetov)
  • Fix complicated bug in push-down predicate optimization which leads to wrong results. This fixes a lot of issues on push-down predicate optimization. #8503 (Winter Zhang)
  • Fix crash in CREATE TABLE .. AS dictionary query. #8508 (Azat Khuzhin)
  • Several improvements ClickHouse grammar in .g4 file. #8294 (taiyang-li)
  • Fix bug that leads to crashes in JOINs with tables with engine Join. This fixes #7556 #8254 #7915 #8100. #8298 (Artem Zuikov)
  • Fix redundant dictionaries reload on CREATE DATABASE. #7916 (Azat Khuzhin)
  • Limit maximum number of streams for read from StorageFile and StorageHDFS. Fixes #7650. #7981 (alesapin)
  • Fix bug in ALTER ... MODIFY ... CODEC query, when user specify both default expression and codec. Fixes 8593. #8614 (alesapin)
  • Fix error in background merge of columns with SimpleAggregateFunction(LowCardinality) type. #8613 (Nikolai Kochetov)
  • Fixed type check in function toDateTime64. #8375 (Vasily Nemkov)
  • Now server do not crash on LEFT or FULL JOIN with and Join engine and unsupported join_use_nulls settings. #8479 (Artem Zuikov)
  • Now DROP DICTIONARY IF EXISTS db.dict query doesn't throw exception if db doesn't exist. #8185 (Vitaly Baranov)
  • Fix possible crashes in table functions (file, mysql, remote) caused by usage of reference to removed IStorage object. Fix incorrect parsing of columns specified at insertion into table function. #7762 (tavplubix)
  • Ensure network be up before starting clickhouse-server. This fixes #7507. #8570 (Zhichang Yu)
  • Fix timeouts handling for secure connections, so queries doesn't hang indefenitely. This fixes #8126. #8128 (alexey-milovidov)
  • Fix clickhouse-copier's redundant contention between concurrent workers. #7816 (Ding Xiang Fei)
  • Now mutations doesn't skip attached parts, even if their mutation version were larger than current mutation version. #7812 (Zhichang Yu) #8250 (alesapin)
  • Ignore redundant copies of *MergeTree data parts after move to another disk and server restart. #7810 (Vladimir Chebotarev)
  • Fix crash in FULL JOIN with LowCardinality in JOIN key. #8252 (Artem Zuikov)
  • Forbidden to use column name more than once in insert query like INSERT INTO tbl (x, y, x). This fixes #5465, #7681. #7685 (alesapin)
  • Added fallback for detection the number of physical CPU cores for unknown CPUs (using the number of logical CPU cores). This fixes #5239. #7726 (alexey-milovidov)
  • Fix There's no column error for materialized and alias columns. #8210 (Artem Zuikov)
  • Fixed sever crash when EXISTS query was used without TABLE or DICTIONARY qualifier. Just like EXISTS t. This fixes #8172. This bug was introduced in version 19.17. #8213 (alexey-milovidov)
  • Fix rare bug with error "Sizes of columns doesn't match" that might appear when using SimpleAggregateFunction column. #7790 (Boris Granveaud)
  • Fix bug where user with empty allow_databases got access to all databases (and same for allow_dictionaries). #7793 (DeifyTheGod)
  • Fix client crash when server already disconnected from client. #8071 (Azat Khuzhin)
  • Fix ORDER BY behaviour in case of sorting by primary key prefix and non primary key suffix. #7759 (Anton Popov)
  • Check if qualified column present in the table. This fixes #6836. #7758 (Artem Zuikov)
  • Fixed behavior with ALTER MOVE ran immediately after merge finish moves superpart of specified. Fixes #8103. #8104 (Vladimir Chebotarev)
  • Fix possible server crash while using UNION with different number of columns. Fixes #7279. #7929 (Nikolai Kochetov)
  • Fix size of result substring for function substr with negative size. #8589 (Nikolai Kochetov)
  • Now server does not execute part mutation in MergeTree if there are not enough free threads in background pool. #8588 (tavplubix)
  • Fix a minor typo on formatting UNION ALL AST. #7999 (litao91)
  • Fixed incorrect bloom filter results for negative numbers. This fixes #8317. #8566 (Winter Zhang)
  • Fixed potential buffer overflow in decompress. Malicious user can pass fabricated compressed data that will cause read after buffer. This issue was found by Eldar Zaitov from Yandex information security team. #8404 (alexey-milovidov)
  • Fix incorrect result because of integers overflow in arrayIntersect. #7777 (Nikolai Kochetov)
  • Now OPTIMIZE TABLE query will not wait for offline replicas to perform the operation. #8314 (javi santana)
  • Fixed ALTER TTL parser for Replicated*MergeTree tables. #8318 (Vladimir Chebotarev)
  • Fix communication between server and client, so server read temporary tables info after query failure. #8084 (Azat Khuzhin)
  • Fix bitmapAnd function error when intersecting an aggregated bitmap and a scalar bitmap. #8082 (Yue Huang)
  • Refine the definition of ZXid according to the ZooKeeper Programmer's Guide which fixes bug in clickhouse-cluster-copier. #8088 (Ding Xiang Fei)
  • odbc table function now respects external_table_functions_use_nulls setting. #7506 (Vasily Nemkov)
  • Fixed bug that lead to a rare data race. #8143 (Alexander Kazakov)
  • Now SYSTEM RELOAD DICTIONARY reloads a dictionary completely, ignoring update_field. This fixes #7440. #8037 (Vitaly Baranov)
  • Add ability to check if dictionary exists in create query. #8032 (alesapin)
  • Fix Float* parsing in Values format. This fixes #7817. #7870 (tavplubix)
  • Fix crash when we cannot reserve space in some background operations of *MergeTree table engines family. #7873 (Vladimir Chebotarev)
  • Fix crash of merge operation when table contains SimpleAggregateFunction(LowCardinality) column. This fixes #8515. #8522 (Azat Khuzhin)
  • Restore support of all ICU locales and add the ability to apply collations for constant expressions. Also add language name to system.collations table. #8051 (alesapin)
  • Fix bug when external dictionaries with zero minimal lifetime (LIFETIME(MIN 0 MAX N), LIFETIME(N)) don't update in background. #7983 (alesapin)
  • Fix crash when external dictionary with ClickHouse source has subquery in query. #8351 (Nikolai Kochetov)
  • Fix incorrect parsing of file extension in table with engine URL. This fixes #8157. #8419 (Andrey Bodrov)
  • Fix CHECK TABLE query for *MergeTree tables without key. Fixes #7543. #7979 (alesapin)
  • Fixed conversion of Float64 to MySQL type. #8079 (Yuriy Baranov)
  • Now if table was not completely dropped because of server crash, server will try to restore and load it. #8176 (tavplubix)
  • Fixed crash in table function file while inserting into file that doesn't exist. Now in this case file would be created and then insert would be processed. #8177 (Olga Khvostikova)
  • Fix rare deadlock which can happen when trace_log is in enabled. #7838 (filimonov)
  • Add ability to work with different types besides Date in RangeHashed external dictionary created from DDL query. Fixes 7899. #8275 (alesapin)
  • Fixes crash when now64() is called with result of another function. #8270 (Vasily Nemkov)
  • Fixed bug with detecting client IP for connections through mysql wire protocol. #7743 (Dmitry Muzyka)
  • Fix empty array handling in arraySplit function. This fixes #7708. #7747 (hcz)
  • Fixed the issue when pid-file of another running clickhouse-server may be deleted. #8487 (Weiqing Xu)
  • Fix dictionary reload if it has invalidate_query, which stopped updates and some exception on previous update tries. #8029 (alesapin)
  • Fixed error in function arrayReduce that may lead to "double free" and error in aggregate function combinator Resample that may lead to memory leak. Added aggregate function aggThrow. This function can be used for testing purposes. #8446 (alexey-milovidov)

Improvement

  • Improved logging when working with S3 table engine. #8251 (Grigory Pervakov)
  • Printed help message when no arguments are passed when calling clickhouse-local. This fixes #5335. #8230 (Andrey Nagorny)
  • Add setting mutations_sync which allows to wait ALTER UPDATE/DELETE queries synchronously. #8237 (alesapin)
  • Allow to set up relative user_files_path in config.xml (in the way similar to format_schema_path). #7632 (hcz)
  • Add exception for illegal types for conversion functions with -OrZero postfix. #7880 (Andrey Konyaev)
  • Simplify format of the header of data sending to a shard in a distributed query. #8044 (Vitaly Baranov)
  • Live View table engine refactoring. #8519 (vzakaznikov)
  • Add additional checks for external dictionaries created from DDL-queries. #8127 (alesapin)
  • Fix error Column ... already exists while using FINAL and SAMPLE together, e.g. select count() from table final sample 1/2. Fixes #5186. #7907 (Nikolai Kochetov)
  • Now table the first argument of joinGet function can be table identifier. #7707 (Amos Bird)
  • Allow using MaterializedView with subqueries above Kafka tables. #8197 (filimonov)
  • Now background moves between disks run it the seprate thread pool. #7670 (Vladimir Chebotarev)
  • SYSTEM RELOAD DICTIONARY now executes synchronously. #8240 (Vitaly Baranov)
  • Stack traces now display physical addresses (offsets in object file) instead of virtual memory addresses (where the object file was loaded). That allows the use of addr2line when binary is position independent and ASLR is active. This fixes #8360. #8387 (alexey-milovidov)
  • Support new syntax for row-level security filters: <table name='table_name'>…</table>. Fixes #5779. #8381 (Ivan)
  • Now cityHash function can work with Decimal and UUID types. Fixes #5184. #7693 (Mikhail Korotov)
  • Removed fixed index granularity (it was 1024) from system logs because it's obsolete after implementation of adaptive granularity. #7698 (alexey-milovidov)
  • Enabled MySQL compatibility server when ClickHouse is compiled without SSL. #7852 (Yuriy Baranov)
  • Now server checksums distributed batches, which gives more verbose errors in case of corrupted data in batch. #7914 (Azat Khuzhin)
  • Support DROP DATABASE, DETACH TABLE, DROP TABLE and ATTACH TABLE for MySQL database engine. #8202 (Winter Zhang)
  • Add authentication in S3 table function and table engine. #7623 (Vladimir Chebotarev)
  • Added check for extra parts of MergeTree at different disks, in order to not allow to miss data parts at undefined disks. #8118 (Vladimir Chebotarev)
  • Enable SSL support for Mac client and server. #8297 (Ivan)
  • Now ClickHouse can work as MySQL federated server (see https://dev.mysql.com/doc/refman/5.7/en/federated-create-server.html). #7717 (Maxim Fedotov)
  • clickhouse-client now only enable bracketed-paste when multiquery is on and multiline is off. This fixes #7757. #7761 (Amos Bird)
  • Support Array(Decimal) in if function. #7721 (Artem Zuikov)
  • Support Decimals in arrayDifference, arrayCumSum and arrayCumSumNegative functions. #7724 (Artem Zuikov)
  • Added lifetime column to system.dictionaries table. #6820 #7727 (kekekekule)
  • Improved check for existing parts on different disks for *MergeTree table engines. Addresses #7660. #8440 (Vladimir Chebotarev)
  • Integration with AWS SDK for S3 interactions which allows to use all S3 features out of the box. #8011 (Pavel Kovalenko)
  • Added support for subqueries in Live View tables. #7792 (vzakaznikov)
  • Check for using Date or DateTime column from TTL expressions was removed. #7920 (Vladimir Chebotarev)
  • Information about disk was added to system.detached_parts table. #7833 (Vladimir Chebotarev)
  • Now settings max_(table|partition)_size_to_drop can be changed without a restart. #7779 (Grigory Pervakov)
  • Slightly better usability of error messages. Ask user not to remove the lines below Stack trace:. #7897 (alexey-milovidov)
  • Better reading messages from Kafka engine in various formats after #7935. #8035 (Ivan)
  • Better compatibility with MySQL clients which don't support sha2_password auth plugin. #8036 (Yuriy Baranov)
  • Support more column types in MySQL compatibility server. #7975 (Yuriy Baranov)
  • Implement ORDER BY optimization for Merge, Buffer and Materilized View storages with underlying MergeTree tables. #8130 (Anton Popov)
  • Now we always use POSIX implementation of getrandom to have better compatibility with old kernels (< 3.17). #7940 (Amos Bird)
  • Better check for valid destination in a move TTL rule. #8410 (Vladimir Chebotarev)
  • Better checks for broken insert batches for Distributed table engine. #7933 (Azat Khuzhin)
  • Add column with array of parts name which mutations must process in future to system.mutations table. #8179 (alesapin)
  • Parallel merge sort optimization for processors. #8552 (Nikolai Kochetov)
  • The settings mark_cache_min_lifetime is now obsolete and does nothing. In previous versions, mark cache can grow in memory larger than mark_cache_size to accomodate data within mark_cache_min_lifetime seconds. That was leading to confusion and higher memory usage than expected, that is especially bad on memory constrained systems. If you will see performance degradation after installing this release, you should increase the mark_cache_size. #8484 (alexey-milovidov)
  • Preparation to use tid everywhere. This is needed for #7477. #8276 (alexey-milovidov)

Performance Improvement

  • Performance optimizations in processors pipeline. #7988 (Nikolai Kochetov)
  • Non-blocking updates of expired keys in cache dictionaries (with permission to read old ones). #8303 (Nikita Mikhaylov)
  • Compile ClickHouse without -fno-omit-frame-pointer globally to spare one more register. #8097 (Amos Bird)
  • Speedup greatCircleDistance function and add performance tests for it. #7307 (Olga Khvostikova)
  • Improved performance of function roundDown. #8465 (alexey-milovidov)
  • Improved performance of max, min, argMin, argMax for DateTime64 data type. #8199 (Vasily Nemkov)
  • Improved performance of sorting without a limit or with big limit and external sorting. #8545 (alexey-milovidov)
  • Improved performance of formatting floating point numbers up to 6 times. #8542 (alexey-milovidov)
  • Improved performance of modulo function. #7750 (Amos Bird)
  • Optimized ORDER BY and merging with single column key. #8335 (alexey-milovidov)
  • Better implementation for arrayReduce, -Array and -State combinators. #7710 (Amos Bird)
  • Now PREWHERE should be optimized to be at least as efficient as WHERE. #7769 (Amos Bird)
  • Improve the way round and roundBankers handling negative numbers. #8229 (hcz)
  • Improved decoding performance of DoubleDelta and Gorilla codecs by roughly 30-40%. This fixes #7082. #8019 (Vasily Nemkov)
  • Improved performance of base64 related functions. #8444 (alexey-milovidov)
  • Added a function geoDistance. It is similar to greatCircleDistance but uses approximation to WGS-84 ellipsoid model. The performance of both functions are near the same. #8086 (alexey-milovidov)
  • Faster min and max aggregation functions for Decimal data type. #8144 (Artem Zuikov)
  • Vectorize processing arrayReduce. #7608 (Amos Bird)
  • if chains are now optimized as multiIf. #8355 (kamalov-ruslan)
  • Fix performance regression of Kafka table engine introduced in 19.15. This fixes #7261. #7935 (filimonov)
  • Removed "pie" code generation that gcc from Debian packages occasionally brings by default. #8483 (alexey-milovidov)
  • Parallel parsing data formats #6553 (Nikita Mikhaylov)
  • Enable optimized parser of Values with expressions by default (input_format_values_deduce_templates_of_expressions=1). #8231 (tavplubix)

Build/Testing/Packaging Improvement

Experimental Feature

  • Added experimental setting min_bytes_to_use_mmap_io. It allows to read big files without copying data from kernel to userspace. The setting is disabled by default. Recommended threshold is about 64 MB, because mmap/munmap is slow. #8520 (alexey-milovidov)
  • Reworked quotas as a part of access control system. Added new table system.quotas, new functions currentQuota, currentQuotaKey, new SQL syntax CREATE QUOTA, ALTER QUOTA, DROP QUOTA, SHOW QUOTA. #7257 (Vitaly Baranov)
  • Allow skipping unknown settings with warnings instead of throwing exceptions. #7653 (Vitaly Baranov)
  • Reworked row policies as a part of access control system. Added new table system.row_policies, new function currentRowPolicies(), new SQL syntax CREATE POLICY, ALTER POLICY, DROP POLICY, SHOW CREATE POLICY, SHOW POLICIES. #7808 (Vitaly Baranov)

Security Fix

  • Fixed the possibility of reading directories structure in tables with File table engine. This fixes #8536. #8537 (alexey-milovidov)

Changelog for 2019