Commit Graph

30700 Commits

Author SHA1 Message Date
Amos Bird
2c75a51d4f
Instant count() for MergeTree
Use (Replicated)MergeTree's metadata to do trivial count()
2019-10-29 17:45:47 +08:00
alexey-milovidov
a17cefb9c6
Merge pull request from Felixoid/mark_cache_doc
Add user setting mark_cache_min_lifetime documentation
2019-10-28 16:17:28 +03:00
Alexander Kuzmenkov
6e2af3db41
Merge pull request from azat/crc-v3
Add CRC32IEEE()/CRC64() support
2019-10-28 16:16:37 +03:00
alexey-milovidov
b1a21ce0be
Merge pull request from SQkaer/master
update trainslation of fixedstring and fix the given examples of nullable in zh doc
2019-10-28 16:15:56 +03:00
alexey-milovidov
3b4c5cc324
Merge pull request from ClickHouse/filimonov-mutationsdocs-patch
remove some obsolete notes about mutations
2019-10-28 16:15:14 +03:00
Alexander Kuzmenkov
c3ea94209b
Merge pull request from traceon/odbc-bridge-type-mapping-fix
Add handling of SQL_TINYINT and SQL_BIGINT, fix SQL_FLOAT in ODBC Bridge
2019-10-28 13:30:18 +03:00
Nikolai Kochetov
43c8d13de6
Merge pull request from amosbird/fix15
Fix HAVING without GROUP BY
2019-10-28 02:44:24 +03:00
Amos Bird
77bdfb3350
Fix HAVING without GROUP BY 2019-10-27 22:10:54 +08:00
Mikhail f. Shiryaev
0cdd7ca6b9 Add user setting mark_cache_min_lifetime documentation 2019-10-25 23:10:04 +02:00
Azat Khuzhin
fe98e90d0d Avoid conflicts of static zlib and mariadb-connector-c zlib
After crc32() had been replaced with crc32_z() the following error will
happen with two different zlib:
  2019-10-25 09:48:42 /usr/bin/ld.gold: error: contrib/zlib-ng/libz.a(crc32.c.o): multiple definition of 'get_crc_table'
  2019-10-25 09:48:42 /usr/bin/ld.gold: contrib/mariadb-connector-c/libmariadb/libmariadbclient.a(crc32.c.o): previous definition here

Fix this by using zlibstatic compiled for and by CH in
mariadb-connector-c, and wrap into function reduce variable scopes.
2019-10-25 23:32:33 +03:00
Azat Khuzhin
04f1e6b2cc Use crc32_z() over crc32(), since it size_t for length 2019-10-25 23:32:29 +03:00
Denis Glazachev
8e48430159 Add handling of SQL_TINYINT and SQL_BIGINT
Fix handling of SQL_FLOAT
2019-10-25 21:49:49 +04:00
alesapin
c7cd911305
Merge pull request from ClickHouse/dictionaries_ddl_loader
Dictionaries ddl loader
2019-10-25 18:34:11 +03:00
Azat Khuzhin
2d2e738085 Add CRC32IEEE()/CRC64() support
zlib's implementation uses CRC-32-IEEE 802.3 polynomial (0xedb88320) but
with starting value 0xffffffff, so introduce another crc32
implementation - CRC32IEEE that has starting value - 0

Also add CRC64 with ECMA polynomial.

v2: s/crc*_data./crc*_data./ to avoid conflicts with other crc32.h in contrib
v3: join with existing CRC32()
2019-10-25 15:52:41 +03:00
filimonov
cbecfd2432
Fixing 'note' markdown 2019-10-25 13:04:07 +02:00
filimonov
3e9e258693
remove some obsolete notes about mutations 2019-10-25 11:37:22 +02:00
alesapin
04ca163518 Merge branch 'master' into dictionaries_ddl_loader 2019-10-25 10:58:04 +03:00
alesapin
b808172e91
Merge pull request from ClickHouse/revert-7460-patch-2
Revert "Remove hardcoded paths in unwind target"
2019-10-24 23:55:58 +03:00
alesapin
7e5b05bbe8
Revert "Remove hardcoded paths in unwind target" 2019-10-24 23:55:41 +03:00
alesapin
8755f5ffa9 Add missed config 2019-10-24 23:08:36 +03:00
Artem Zuikov
df32d92ee5
Merge pull request from 4ertus2/ast
Better ambiguous column detection
2019-10-24 20:56:23 +03:00
alesapin
5d8d1df802
Merge pull request from excitoon-favorites/fix-7424
Allowed to have some parts on destination disk or volume in MOVE PARTITION
2019-10-24 19:45:22 +03:00
alexey-milovidov
e865136b2c
Merge pull request from excitoon-favorites/test-7414
Added integration test for  (validation of `max_data_part_size_bytes`).
2019-10-24 19:23:21 +03:00
alexey-milovidov
8c01dcca25
Merge pull request from podshumok/patch-2
Remove hardcoded paths in unwind target
2019-10-24 19:20:52 +03:00
alexey-milovidov
0d1d6e590b
Merge pull request from Felixoid/partition_id_doc
Add PARTITION ID to OPTIMIZE documentation
2019-10-24 19:15:52 +03:00
rainbowsysu
8e5a6cd206 update trainslation of fixedstring in zh 2019-10-24 21:22:50 +08:00
chertus
f2028e901d review related changes 2019-10-24 16:04:50 +03:00
Ivan
c250db4922
Update Docker Image for Binary Packager () 2019-10-24 15:56:30 +03:00
chertus
12cd21f3c3 Merge branch 'master' into ast 2019-10-24 15:20:09 +03:00
rainbowsysu
08788e3443 fixed the given examples of nullable in zh doc 2019-10-24 20:07:10 +08:00
Artem Zuikov
39b64dff87
Merge pull request from 4ertus2/refactoring
Refactoring: extract non aliases logic out of QueryNormalizer
2019-10-24 14:34:53 +03:00
Vladimir Chebotarev
64f158ff28 Fixed message for ALTER MOVE PART. 2019-10-24 13:56:32 +03:00
Alexander Kuzmenkov
29052b6a37
Merge pull request from azat/INSERT-Distributed-MATERIALIZED-cols
* Fix INSERT into Distributed non local node with MATERIALIZED columns

Previous patch e527def18a ("Fix INSERT
into Distributed() table with MATERIALIZED column") fixes it only for
cases when the node is local, i.e. direct insert.

This patch address the problem when the node is not local
(`is_local == false`), by erasing materialized columns on INSERT into
Distributed.

And this patch fixes two cases, depends on `insert_distributed_sync`
setting:

- `insert_distributed_sync=0`

    ```
    Not found column value in block. There are only columns: date. Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5d6cf6 DB::Block::getByName(...) dbms/src/Core/Block.cpp:187
    4. 0x7fffec2fe067 DB::NativeBlockInputStream::readImpl() dbms/src/DataStreams/NativeBlockInputStream.cpp:159
    5. 0x7fffec2d223f DB::IBlockInputStream::read() dbms/src/DataStreams/IBlockInputStream.cpp:61
    6. 0x7ffff7c6d40d DB::TCPHandler::receiveData() dbms/programs/server/TCPHandler.cpp:971
    7. 0x7ffff7c6cc1d DB::TCPHandler::receivePacket() dbms/programs/server/TCPHandler.cpp:855
    8. 0x7ffff7c6a1ef DB::TCPHandler::readDataNext(unsigned long const&, int const&) dbms/programs/server/TCPHandler.cpp:406
    9. 0x7ffff7c6a41b DB::TCPHandler::readData(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:437
    10. 0x7ffff7c6a5d9 DB::TCPHandler::processInsertQuery(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:464
    11. 0x7ffff7c687b5 DB::TCPHandler::runImpl() dbms/programs/server/TCPHandler.cpp:257
    ```

- `insert_distributed_sync=1`

    ```
    2019.10.18 13:23:22.114578 [ 44 ] {a78f669f-0b08-4337-abf8-d31e958f6d12} <Error> executeQuery: Code: 171, e.displayText() = DB::Exception: Block structure mismatch in RemoteBlockOutputStream stream: different number of columns:
    date Date UInt16(size = 1), value Date UInt16(size = 1)
    date Date UInt16(size = 0): Insertion status:
    Wrote 1 blocks and 0 rows on shard 0 replica 0, 127.0.0.1:59000 (average 0 ms per block)
    Wrote 0 blocks and 0 rows on shard 1 replica 0, 127.0.0.2:59000 (average 2 ms per block)
     (version 19.16.1.1) (from [::1]:3624) (in query: INSERT INTO distributed_00952 VALUES ), Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5da4e9 DB::checkBlockStructure<void>(...)::{...}::operator()(...) const dbms/src/Core/Block.cpp:460
    4. 0x7fffec5da671 void DB::checkBlockStructure<void>(...) dbms/src/Core/Block.cpp:467
    5. 0x7fffec5d8d58 DB::assertBlocksHaveEqualStructure(...) dbms/src/Core/Block.cpp:515
    6. 0x7fffec326630 DB::RemoteBlockOutputStream::write(DB::Block const&) dbms/src/DataStreams/RemoteBlockOutputStream.cpp:68
    7. 0x7fffe98bd154 DB::DistributedBlockOutputStream::runWritingJob(DB::DistributedBlockOutputStream::JobReplica&, DB::Block const&)::{lambda()#1}::operator()() const dbms/src/Storages/Distributed/DistributedBlockOutputStream.cpp:280
    <snip>
    ````

Fixes: 
Fixes: 
Refs: 

* Cover INSERT into Distributed with MATERIALIZED columns and !is_local node

I guess that adding new cluster into server-test.xml is not required,
but it won't harm.

* Update DistributedBlockOutputStream.cpp
2019-10-24 12:35:09 +03:00
Alexander Kuzmenkov
050de71ef4
Update DistributedBlockOutputStream.cpp 2019-10-24 12:33:45 +03:00
alesapin
a6199b7e69 Merge with master 2019-10-24 12:33:40 +03:00
alesapin
7edd80c9b7 Add test for existing dictionary 2019-10-24 12:25:28 +03:00
Vladimir Chebotarev
255da8f5e0 Fixed style. 2019-10-24 12:11:06 +03:00
Vladimir Chebotarev
3debdc2119 Added integration tests for ALTER MOVE PARTITION and fixed minor things. 2019-10-24 11:52:33 +03:00
Mikhail f. Shiryaev
3d2eab7535 Add PARTITION ID to OPTIMIZE documentation 2019-10-24 09:49:58 +02:00
Vladimir Chebotarev
e8e5cefc35 Fixed integration test for . 2019-10-24 08:59:33 +03:00
Ivan Blinkov
f8a401bbf7
Labeler seems to require additional permissions from PR authors ()
* Delete labeler.yml

* Delete labeler.keywords.yml

* Delete labeler.yml
2019-10-24 08:56:53 +08:00
Artem Zuikov
18a72fa91a
Merge pull request from amosbird/scalar
better scalar query
2019-10-23 22:54:30 +03:00
Azat Khuzhin
80cf86f100 Cover INSERT into Distributed with MATERIALIZED columns and !is_local node
I guess that adding new cluster into server-test.xml is not required,
but it won't harm.
2019-10-23 21:55:08 +03:00
Azat Khuzhin
ab9d9f8997 Fix INSERT into Distributed non local node with MATERIALIZED columns
Previous patch e527def18a ("Fix INSERT
into Distributed() table with MATERIALIZED column") fixes it only for
cases when the node is local, i.e. direct insert.

This patch address the problem when the node is not local
(`is_local == false`), by erasing materialized columns on INSERT into
Distributed.

And this patch fixes two cases, depends on `insert_distributed_sync`
setting:

- `insert_distributed_sync=0`

    ```
    Not found column value in block. There are only columns: date. Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5d6cf6 DB::Block::getByName(...) dbms/src/Core/Block.cpp:187
    4. 0x7fffec2fe067 DB::NativeBlockInputStream::readImpl() dbms/src/DataStreams/NativeBlockInputStream.cpp:159
    5. 0x7fffec2d223f DB::IBlockInputStream::read() dbms/src/DataStreams/IBlockInputStream.cpp:61
    6. 0x7ffff7c6d40d DB::TCPHandler::receiveData() dbms/programs/server/TCPHandler.cpp:971
    7. 0x7ffff7c6cc1d DB::TCPHandler::receivePacket() dbms/programs/server/TCPHandler.cpp:855
    8. 0x7ffff7c6a1ef DB::TCPHandler::readDataNext(unsigned long const&, int const&) dbms/programs/server/TCPHandler.cpp:406
    9. 0x7ffff7c6a41b DB::TCPHandler::readData(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:437
    10. 0x7ffff7c6a5d9 DB::TCPHandler::processInsertQuery(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:464
    11. 0x7ffff7c687b5 DB::TCPHandler::runImpl() dbms/programs/server/TCPHandler.cpp:257
    ```

- `insert_distributed_sync=1`

    ```
    2019.10.18 13:23:22.114578 [ 44 ] {a78f669f-0b08-4337-abf8-d31e958f6d12} <Error> executeQuery: Code: 171, e.displayText() = DB::Exception: Block structure mismatch in RemoteBlockOutputStream stream: different number of columns:
    date Date UInt16(size = 1), value Date UInt16(size = 1)
    date Date UInt16(size = 0): Insertion status:
    Wrote 1 blocks and 0 rows on shard 0 replica 0, 127.0.0.1:59000 (average 0 ms per block)
    Wrote 0 blocks and 0 rows on shard 1 replica 0, 127.0.0.2:59000 (average 2 ms per block)
     (version 19.16.1.1) (from [::1]:3624) (in query: INSERT INTO distributed_00952 VALUES ), Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5da4e9 DB::checkBlockStructure<void>(...)::{...}::operator()(...) const dbms/src/Core/Block.cpp:460
    4. 0x7fffec5da671 void DB::checkBlockStructure<void>(...) dbms/src/Core/Block.cpp:467
    5. 0x7fffec5d8d58 DB::assertBlocksHaveEqualStructure(...) dbms/src/Core/Block.cpp:515
    6. 0x7fffec326630 DB::RemoteBlockOutputStream::write(DB::Block const&) dbms/src/DataStreams/RemoteBlockOutputStream.cpp:68
    7. 0x7fffe98bd154 DB::DistributedBlockOutputStream::runWritingJob(DB::DistributedBlockOutputStream::JobReplica&, DB::Block const&)::{lambda()#1}::operator()() const dbms/src/Storages/Distributed/DistributedBlockOutputStream.cpp:280
    <snip>
    ````

Fixes: 
Fixes: 
Refs: 
2019-10-23 21:54:27 +03:00
Konstantin Podshumok
d1a19d26e8
Remove hardcoded paths in unwind target
In most cases they match defaults now, but it is too hard to override when one needs to (alternative builds)
2019-10-23 20:33:40 +03:00
Artem Zuikov
bb1c1d0ed9
Merge pull request from arenadata/master
fix aggregation (avg and quantiles) over empty decimal columns.
2019-10-23 20:28:35 +03:00
Ivan
32ca372b9d
Revert "Update Dockerfile for binary packager ()" ()
This reverts commit fa05a5860f.
2019-10-23 18:54:18 +03:00
Koblikov Mihail
6dc497f7bc fix typo in ontime.md ()
* fix type in ontime.md

* Update docs/en/getting_started/example_datasets/ontime.md

Co-Authored-By: Ivan Blinkov <github@blinkov.ru>

* Update docs/en/getting_started/example_datasets/ontime.md

Co-Authored-By: Ivan Blinkov <github@blinkov.ru>

* Update docs/en/getting_started/example_datasets/ontime.md

Co-Authored-By: Ivan Blinkov <github@blinkov.ru>

* Update ontime.md
2019-10-23 18:32:42 +03:00
Ivan
fa05a5860f
Update Dockerfile for binary packager () 2019-10-23 17:41:17 +03:00
chertus
20093fa065 extract more logic out of QueryNormalizer 2019-10-23 16:59:03 +03:00