Commit Graph

471 Commits

Author SHA1 Message Date
Andrei Bodrov
8cbc52e711 add gzip read/write to file/s3/url/hdfs 2019-11-19 15:46:07 +03:00
Vitaly Baranov
f3d72b9ec1 Skip unknown settings with warnings. 2019-11-17 03:51:58 +03:00
alexey-milovidov
5017a090bf
Update MySQLHandler.cpp 2019-11-14 20:32:56 +03:00
Dmitry Muzyka
38f34bd437 fix detection of remote ip 2019-11-12 22:57:39 +03:00
Nikolai Kochetov
3bde663681 Wailt for AsynchronousBlockInputStream inner thread before getting totals to avoid possible race. 2019-11-06 15:34:51 +03:00
Amos Bird
8f902dc2a9
Better null format for tcp handler. 2019-11-04 11:53:26 +08:00
proller
6a573b4092 Allow to use mysql format without ssl - try2 with mysql interface split (#7524)
* Allow use mysql format without ssl

* fix

* fix

*  fix

* Also disable poco's optional libraries

* clean

* fix

* fix

* Requested changes

* clean

* Requested fixes

* Update MySQLHandler.cpp
2019-11-02 13:20:46 +03:00
alesapin
04ca163518 Merge branch 'master' into dictionaries_ddl_loader 2019-10-25 10:58:04 +03:00
Alexander Kuzmenkov
29052b6a37
Merge pull request #7377 from azat/INSERT-Distributed-MATERIALIZED-cols
* Fix INSERT into Distributed non local node with MATERIALIZED columns

Previous patch e527def18a ("Fix INSERT
into Distributed() table with MATERIALIZED column") fixes it only for
cases when the node is local, i.e. direct insert.

This patch address the problem when the node is not local
(`is_local == false`), by erasing materialized columns on INSERT into
Distributed.

And this patch fixes two cases, depends on `insert_distributed_sync`
setting:

- `insert_distributed_sync=0`

    ```
    Not found column value in block. There are only columns: date. Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5d6cf6 DB::Block::getByName(...) dbms/src/Core/Block.cpp:187
    4. 0x7fffec2fe067 DB::NativeBlockInputStream::readImpl() dbms/src/DataStreams/NativeBlockInputStream.cpp:159
    5. 0x7fffec2d223f DB::IBlockInputStream::read() dbms/src/DataStreams/IBlockInputStream.cpp:61
    6. 0x7ffff7c6d40d DB::TCPHandler::receiveData() dbms/programs/server/TCPHandler.cpp:971
    7. 0x7ffff7c6cc1d DB::TCPHandler::receivePacket() dbms/programs/server/TCPHandler.cpp:855
    8. 0x7ffff7c6a1ef DB::TCPHandler::readDataNext(unsigned long const&, int const&) dbms/programs/server/TCPHandler.cpp:406
    9. 0x7ffff7c6a41b DB::TCPHandler::readData(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:437
    10. 0x7ffff7c6a5d9 DB::TCPHandler::processInsertQuery(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:464
    11. 0x7ffff7c687b5 DB::TCPHandler::runImpl() dbms/programs/server/TCPHandler.cpp:257
    ```

- `insert_distributed_sync=1`

    ```
    2019.10.18 13:23:22.114578 [ 44 ] {a78f669f-0b08-4337-abf8-d31e958f6d12} <Error> executeQuery: Code: 171, e.displayText() = DB::Exception: Block structure mismatch in RemoteBlockOutputStream stream: different number of columns:
    date Date UInt16(size = 1), value Date UInt16(size = 1)
    date Date UInt16(size = 0): Insertion status:
    Wrote 1 blocks and 0 rows on shard 0 replica 0, 127.0.0.1:59000 (average 0 ms per block)
    Wrote 0 blocks and 0 rows on shard 1 replica 0, 127.0.0.2:59000 (average 2 ms per block)
     (version 19.16.1.1) (from [::1]:3624) (in query: INSERT INTO distributed_00952 VALUES ), Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5da4e9 DB::checkBlockStructure<void>(...)::{...}::operator()(...) const dbms/src/Core/Block.cpp:460
    4. 0x7fffec5da671 void DB::checkBlockStructure<void>(...) dbms/src/Core/Block.cpp:467
    5. 0x7fffec5d8d58 DB::assertBlocksHaveEqualStructure(...) dbms/src/Core/Block.cpp:515
    6. 0x7fffec326630 DB::RemoteBlockOutputStream::write(DB::Block const&) dbms/src/DataStreams/RemoteBlockOutputStream.cpp:68
    7. 0x7fffe98bd154 DB::DistributedBlockOutputStream::runWritingJob(DB::DistributedBlockOutputStream::JobReplica&, DB::Block const&)::{lambda()#1}::operator()() const dbms/src/Storages/Distributed/DistributedBlockOutputStream.cpp:280
    <snip>
    ````

Fixes: #7365
Fixes: #5429
Refs: #6891

* Cover INSERT into Distributed with MATERIALIZED columns and !is_local node

I guess that adding new cluster into server-test.xml is not required,
but it won't harm.

* Update DistributedBlockOutputStream.cpp
2019-10-24 12:35:09 +03:00
alesapin
a6199b7e69 Merge with master 2019-10-24 12:33:40 +03:00
Azat Khuzhin
80cf86f100 Cover INSERT into Distributed with MATERIALIZED columns and !is_local node
I guess that adding new cluster into server-test.xml is not required,
but it won't harm.
2019-10-23 21:55:08 +03:00
Amos Bird
295864e6e0
better scalar query 2019-10-23 21:37:54 +08:00
alesapin
ff7850cfbd Remove strange code in favor of less strange code 2019-10-21 17:59:35 +03:00
alesapin
35e66d6696 Merge with master 2019-10-21 11:46:11 +03:00
alesapin
cdc195727e Fix some races from lazy load and add ugly interface to loader (need to be refactored) 2019-10-18 19:04:09 +03:00
tavplubix
0046b9f137 Wait for jobs to finish on exception (fixes rare segfaults) (#7350) 2019-10-17 17:41:27 +03:00
alesapin
155731a676 Merge branch 'master' into dictionaries_ddl_interpreter 2019-10-14 11:49:18 +03:00
Vitaly Baranov
12d1d87d64 Move authentication code to a separate class. 2019-10-11 00:18:52 +03:00
alesapin
2a5713e6e4 Compilable code 2019-10-10 23:47:47 +03:00
Alexey Milovidov
a19f6513ce Added example config with macros 2019-10-09 04:15:23 +03:00
alexey-milovidov
5c2d478543
Merge pull request #7212 from ClickHouse/aku/field-includes
Include Field.h and FieldVisitor.h into fewer files.
2019-10-08 22:40:30 +03:00
alexey-milovidov
b9aa5d812d
Merge pull request #7171 from nikvas0/nikvas0/lazy_db
Lazy Database
2019-10-08 22:15:55 +03:00
Alexander Kuzmenkov
367c537106 Do not include Field.h everywhere through SettingsCommon.h.
Move the methods of SettingsCollection<> that are dependent on Field to
a separate file, and include it once for each instantiation. This allows
to work on Field without always recompiling the entire project.
2019-10-07 20:23:05 +03:00
proller
594c535bd4 Build fixes (#7063)
* Build fixes

*            fix

* Fix

* fix

* Fix install(export..

* Freebsd fixes

* Freebsd fixes

* Fix warning

* fix

* More PRIVATE linking

* Fix review

* Timmy

* fix

* fix
2019-10-05 22:25:31 +03:00
Vasilyev Nikita
5793a02ef9 impr Lazy database 2019-10-01 15:44:17 +03:00
Azat Khuzhin
740e4c5851
Avoid use of uninitialized values in MetricsTransmitter
Defer thread creation after values had been initialized correctly.
2019-10-01 10:16:39 +03:00
alesapin
4fdb5a61e5 Rename ExternalDictionaries to ExternalDictionariesLoader 2019-09-26 13:41:33 +03:00
Ivan Lezhankin
4b143c3e0f Merge remote-tracking branch 'upstream/master' into issue-6459 2019-09-23 16:07:19 +03:00
Alexey Milovidov
70bc89557f Updates for #6969 2019-09-23 01:57:06 +03:00
Azat Khuzhin
59763cbb3a
Add ability to send events with absolute values to graphite
<events> sends only deltas for events, while sometimes it is more
convenient to get absolute values (for example to calculate rate).

This patch introduces another directive -- <events_absolute> that will handle
this.

v0: use <events> as bool/string
v2: use <events_absolute>true</events_absolute> over <events>absolute</events>
2019-09-21 02:46:47 +03:00
Ivan Lezhankin
9e46118edd Merge remote-tracking branch 'upstream/master' into issue-6459 2019-09-20 21:25:55 +03:00
Ivan Lezhankin
01fdb802d4 Useful changes 2019-09-15 13:35:12 +03:00
alesapin
02cff38fef Merge with master 2019-09-13 13:58:23 +03:00
Nikolai Kochetov
86bee0b8b5
Merge pull request #6684 from yandex/processors-2
Processors 2
2019-09-13 11:58:04 +03:00
alexey-milovidov
12c26e6013
Merge pull request #6891 from yandex/low_cardinality_in_native_http
Low cardinality in native http
2019-09-13 09:01:08 +03:00
Nikolai Kochetov
1319168a3a Merge branch 'master' into processors-2 2019-09-12 16:28:51 +03:00
Nikolai Kochetov
d7596e51b7 Removed convert_types_to_low_cardinality from NativeBlockInputStream. 2019-09-12 14:33:46 +03:00
Amos Bird
93c0880ca7 Fix non-ssl build. 2019-09-12 04:20:27 +08:00
Nikolai Kochetov
038aa82305 Update TCPHandler 2019-09-09 17:57:12 +03:00
Nikolai Kochetov
c9cca3ba8b Merged with master 2019-09-09 16:20:30 +03:00
alesapin
27419c965e Merge with master 2019-09-09 14:13:36 +03:00
Yuriy
6700314ce5 destroying context after executeQuery 2019-09-09 04:04:37 +03:00
alexey-milovidov
8389200229
Merge pull request #6832 from CurtizJ/merging_input_function
Merging #5450 (table function input).
2019-09-07 20:16:45 +03:00
filimonov
526afd44ed Make SensitiveDataMasker a singleton. (#6810)
Among other things, it is used to filter logs, which are being written even after the global server context is deinitialized, so we can't keep masker there.
2019-09-06 20:48:27 +03:00
Anton Popov
69da4f30b3
Merge branch 'master' into merging_input_function 2019-09-06 16:20:00 +03:00
Nikolai Kochetov
52305cefbc Merge branch 'merging-aggregated-processors' into processors-2 2019-09-06 15:21:17 +03:00
alesapin
b71faee8cd Merge branch 'master' into ObjatieGroba-feature_multiple_disks 2019-09-05 13:19:05 +03:00
Alexey Milovidov
a4c1d64aba Allowed to run test with the server run from source tree 2019-09-05 07:50:08 +03:00
alesapin
3ffa9fa972 Resolve conflict 2019-09-04 15:44:12 +03:00
CurtizJ
7accc09066 Merge branch 'palasonic-add-input-function' of https://github.com/palasonic1/ClickHouse into merging_input_function 2019-09-04 15:25:20 +03:00