Commit Graph

1073 Commits

Author SHA1 Message Date
Alexander Burmak
3e5ef56644 Filesystem abstraction layer 2019-11-27 14:33:27 +03:00
VDimir
3c4220017c Add async metrics to Prometheus output 2019-11-26 23:27:24 +03:00
VDimir
eadc58bbd6 Style fixes in HTTPHandlerFactory 2019-11-26 22:03:29 +03:00
Maksim Fedotov
5e40311266 support clickhouse as mysql federated server. apply code review notes 2019-11-26 18:39:36 +03:00
Pervakov Grigory
9fbc55c61a Change max_table_size_to_drop and max_partition_size_to_drop to atmoic. Update hint in default config. 2019-11-25 14:42:09 +03:00
Pervakov Grigory
651e0f3a1c Update max_table_size_to_drop and max_partition_size_to_drop with config reload 2019-11-25 14:42:09 +03:00
VDimir
2ddb801d0b Metric names in prometheus-friendly snake_case format 2019-11-24 23:59:22 +03:00
VDimir
0e65716378 Style fixes 2019-11-24 23:03:09 +03:00
VDimir
ffcd9022a4 Prometheus run on custom port 2019-11-24 19:27:00 +03:00
VDimir
b19e33e819 Make HTTPRequestHandlerFactory composable, add PrometeusRequestHandlerFactory 2019-11-24 18:43:53 +03:00
VDimir
947fc8d134 Create http handler responding metrics in prometheus format 2019-11-24 15:06:10 +03:00
alexey-milovidov
eb7f48aaf6
Merge pull request #7816 from dingxiangfei2009/fix-copier-contention-master
Fix clickhouse-copier cleaning-tainting contention between concurrent workers
2019-11-23 06:49:03 +03:00
Nikita Mikhaylov
728ede25f9
Merge pull request #7840 from apbodrov/read_write_gzip
Gzip read/write to file/s3/url/hdfs
2019-11-22 11:48:58 +03:00
Ding Xiang Fei
8330a09575
apply style guide and fix CI warnings 2019-11-22 09:48:58 +08:00
Ding Xiang Fei
608aa276ab
add safeguard against contention to check target partition row counts 2019-11-22 09:48:57 +08:00
Nikita Mikhaylov
762d295ab0
Merge pull request #7780 from ClickHouse/aku/parallel-parsing
Merging parallel format parsing
2019-11-22 01:54:47 +03:00
alexey-milovidov
420aeb25a4
Update MySQLHandler.cpp 2019-11-20 07:30:00 +03:00
Yuriy
5be62948bc merged master 2019-11-20 02:34:05 +03:00
Yuriy
1bf4d21c67 more precise error messages in MySQL protocol 2019-11-20 02:27:30 +03:00
Yuriy
21cc7ae25e enabled MySQL compatibility server when ClickHouse is compiled without SSL 2019-11-20 00:10:21 +03:00
Alexander Kuzmenkov
9a2b864940 Cleanup 2019-11-19 21:42:51 +03:00
Andrei Bodrov
a2b5989177 BrotliReadBuffer refactoring 2019-11-19 16:57:54 +03:00
Alexander Kuzmenkov
7e5731d64e Merge remote-tracking branch 'origin/master' into HEAD 2019-11-19 16:15:47 +03:00
Andrei Bodrov
8cbc52e711 add gzip read/write to file/s3/url/hdfs 2019-11-19 15:46:07 +03:00
Vitaly Baranov
681f03c625
Merge pull request #7653 from vitlibar/skip-unknown-settings-with-warnings
Skip unknown settings with warnings.
2019-11-18 13:50:52 +03:00
Vitaly Baranov
f3d72b9ec1 Skip unknown settings with warnings. 2019-11-17 03:51:58 +03:00
Amos Bird
755904f51f
better INCLUDE_DEBUG_HELPERS 2019-11-16 00:34:43 +08:00
alexey-milovidov
698cb6c3ee
Merge pull request #7743 from dmitriy-myz/fix-mysql-wire-auth
MySQL wire protocol: wrong source ip
2019-11-14 20:33:17 +03:00
alexey-milovidov
5017a090bf
Update MySQLHandler.cpp 2019-11-14 20:32:56 +03:00
Alexander Kuzmenkov
7f34c0089c Merge remote-tracking branch 'origin/master' into HEAD 2019-11-14 18:53:36 +03:00
alexey-milovidov
ed5e4cd4b5
Merge pull request #7760 from amosbird/clientfixexit
Fix multi query exit for clickhouse-client
2019-11-14 12:50:31 +03:00
Amos Bird
646c5f124c
Enable bracketed-paste in certain case only.
Only enable bracketed-paste when multiquery is on and multiline is off.
2019-11-14 09:02:51 +08:00
Amos Bird
f5688e6046
Fix multi query exit for clickhouse-client 2019-11-13 23:45:23 +08:00
Dmitry Muzyka
38f34bd437 fix detection of remote ip 2019-11-12 22:57:39 +03:00
Amos Bird
5a28f1dee9
fix utf-8 paste 2019-11-12 20:17:57 +08:00
Maksim Fedotov
0403444f14 support clickhouse as mysql federated server 2019-11-11 17:31:12 +03:00
Vasily Nemkov
4ac21ca6ba PR-review fixes: handling bad value of 'external_table_functions_use_nulls' param, fixed typo 2019-11-08 14:30:11 +03:00
Vasily Nemkov
5624bb3abb odbc table function now respects external_table_functions_use_nulls setting
* Passing setting value to ODBC-bridge on each request
 * Handling that setting value correctly in ODBC-bridge
 * Fixed issue with executing table-functions on context with no settings
   applied in `SELECT ... SETTINGS x=foo` query.
 * Added tests to verify fix.
2019-11-08 14:30:11 +03:00
millb
7a2dcf451a Merge branch 'master' into url_engine_problem 2019-11-08 02:19:09 +03:00
Nikolai Kochetov
3bde663681 Wailt for AsynchronousBlockInputStream inner thread before getting totals to avoid possible race. 2019-11-06 15:34:51 +03:00
Alexander Kuzmenkov
233212ac68
Merge pull request #7531 from amosbird/getkeymapped
Unify hash tables interface (separated for review)
2019-11-06 13:49:39 +03:00
Amos Bird
fbfbe161ec
Unify hash tables interface. 2019-11-06 00:08:04 +08:00
Mikhail Korotov
3687ccfa1e Merge branch 'url_engine_problem' of https://github.com/millb/ClickHouse into url_engine_problem 2019-11-05 15:46:03 +03:00
Mikhail Korotov
82d09a8df0 working commit 2019-11-05 15:45:43 +03:00
millb
d62cd6efa8 Merge branch 'master' into url_engine_problem 2019-11-05 14:42:53 +03:00
Nikita Mikhaylov
817b52314e Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel_parsing 2019-11-05 12:32:26 +03:00
Amos Bird
8f902dc2a9
Better null format for tcp handler. 2019-11-04 11:53:26 +08:00
alexey-milovidov
f5531fb436
Merge pull request #7590 from amosbird/typetrait
ubsan fix
2019-11-02 13:22:39 +03:00
proller
6a573b4092 Allow to use mysql format without ssl - try2 with mysql interface split (#7524)
* Allow use mysql format without ssl

* fix

* fix

*  fix

* Also disable poco's optional libraries

* clean

* fix

* fix

* Requested changes

* clean

* Requested fixes

* Update MySQLHandler.cpp
2019-11-02 13:20:46 +03:00
Amos Bird
ad12099642
ubsan fix 2019-11-02 17:00:03 +08:00
Amos Bird
4609ca3189
Enable bracketed paste if possible 2019-10-31 02:33:45 +08:00
Nikita Mikhaylov
756a10707d Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel_parsing 2019-10-28 13:51:52 +03:00
Alexander Kuzmenkov
c3ea94209b
Merge pull request #7491 from traceon/odbc-bridge-type-mapping-fix
Add handling of SQL_TINYINT and SQL_BIGINT, fix SQL_FLOAT in ODBC Bridge
2019-10-28 13:30:18 +03:00
Denis Glazachev
8e48430159 Add handling of SQL_TINYINT and SQL_BIGINT
Fix handling of SQL_FLOAT
2019-10-25 21:49:49 +04:00
alesapin
04ca163518 Merge branch 'master' into dictionaries_ddl_loader 2019-10-25 10:58:04 +03:00
Alexander Kuzmenkov
29052b6a37
Merge pull request #7377 from azat/INSERT-Distributed-MATERIALIZED-cols
* Fix INSERT into Distributed non local node with MATERIALIZED columns

Previous patch e527def18a ("Fix INSERT
into Distributed() table with MATERIALIZED column") fixes it only for
cases when the node is local, i.e. direct insert.

This patch address the problem when the node is not local
(`is_local == false`), by erasing materialized columns on INSERT into
Distributed.

And this patch fixes two cases, depends on `insert_distributed_sync`
setting:

- `insert_distributed_sync=0`

    ```
    Not found column value in block. There are only columns: date. Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5d6cf6 DB::Block::getByName(...) dbms/src/Core/Block.cpp:187
    4. 0x7fffec2fe067 DB::NativeBlockInputStream::readImpl() dbms/src/DataStreams/NativeBlockInputStream.cpp:159
    5. 0x7fffec2d223f DB::IBlockInputStream::read() dbms/src/DataStreams/IBlockInputStream.cpp:61
    6. 0x7ffff7c6d40d DB::TCPHandler::receiveData() dbms/programs/server/TCPHandler.cpp:971
    7. 0x7ffff7c6cc1d DB::TCPHandler::receivePacket() dbms/programs/server/TCPHandler.cpp:855
    8. 0x7ffff7c6a1ef DB::TCPHandler::readDataNext(unsigned long const&, int const&) dbms/programs/server/TCPHandler.cpp:406
    9. 0x7ffff7c6a41b DB::TCPHandler::readData(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:437
    10. 0x7ffff7c6a5d9 DB::TCPHandler::processInsertQuery(DB::Settings const&) dbms/programs/server/TCPHandler.cpp:464
    11. 0x7ffff7c687b5 DB::TCPHandler::runImpl() dbms/programs/server/TCPHandler.cpp:257
    ```

- `insert_distributed_sync=1`

    ```
    2019.10.18 13:23:22.114578 [ 44 ] {a78f669f-0b08-4337-abf8-d31e958f6d12} <Error> executeQuery: Code: 171, e.displayText() = DB::Exception: Block structure mismatch in RemoteBlockOutputStream stream: different number of columns:
    date Date UInt16(size = 1), value Date UInt16(size = 1)
    date Date UInt16(size = 0): Insertion status:
    Wrote 1 blocks and 0 rows on shard 0 replica 0, 127.0.0.1:59000 (average 0 ms per block)
    Wrote 0 blocks and 0 rows on shard 1 replica 0, 127.0.0.2:59000 (average 2 ms per block)
     (version 19.16.1.1) (from [::1]:3624) (in query: INSERT INTO distributed_00952 VALUES ), Stack trace:

    2. 0x7ffff7be92e0 DB::Exception::Exception() dbms/src/Common/Exception.h:27
    3. 0x7fffec5da4e9 DB::checkBlockStructure<void>(...)::{...}::operator()(...) const dbms/src/Core/Block.cpp:460
    4. 0x7fffec5da671 void DB::checkBlockStructure<void>(...) dbms/src/Core/Block.cpp:467
    5. 0x7fffec5d8d58 DB::assertBlocksHaveEqualStructure(...) dbms/src/Core/Block.cpp:515
    6. 0x7fffec326630 DB::RemoteBlockOutputStream::write(DB::Block const&) dbms/src/DataStreams/RemoteBlockOutputStream.cpp:68
    7. 0x7fffe98bd154 DB::DistributedBlockOutputStream::runWritingJob(DB::DistributedBlockOutputStream::JobReplica&, DB::Block const&)::{lambda()#1}::operator()() const dbms/src/Storages/Distributed/DistributedBlockOutputStream.cpp:280
    <snip>
    ````

Fixes: #7365
Fixes: #5429
Refs: #6891

* Cover INSERT into Distributed with MATERIALIZED columns and !is_local node

I guess that adding new cluster into server-test.xml is not required,
but it won't harm.

* Update DistributedBlockOutputStream.cpp
2019-10-24 12:35:09 +03:00
alesapin
a6199b7e69 Merge with master 2019-10-24 12:33:40 +03:00
Azat Khuzhin
80cf86f100 Cover INSERT into Distributed with MATERIALIZED columns and !is_local node
I guess that adding new cluster into server-test.xml is not required,
but it won't harm.
2019-10-23 21:55:08 +03:00
Amos Bird
295864e6e0
better scalar query 2019-10-23 21:37:54 +08:00
Nikita Mikhaylov
34d8ade4a0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel_parsing 2019-10-22 15:56:51 +03:00
Alexander Kuzmenkov
9c1f8109c7
Merge pull request #7402 from Enmk/odbc_bridge_nullable_fix
Fixed NULL-values in nullable columns through ODBC-bridge
2019-10-21 19:09:27 +03:00
alesapin
ff7850cfbd Remove strange code in favor of less strange code 2019-10-21 17:59:35 +03:00
Vasily Nemkov
e080324fd0 Fixed NULL-values in nullable columns through ODBC-bridge 2019-10-21 12:13:33 +03:00
alesapin
35e66d6696 Merge with master 2019-10-21 11:46:11 +03:00
millb
f64f6658ce Merge branch 'master' into url_engine_problem 2019-10-21 11:44:31 +03:00
alesapin
cdc195727e Fix some races from lazy load and add ugly interface to loader (need to be refactored) 2019-10-18 19:04:09 +03:00
tavplubix
0046b9f137 Wait for jobs to finish on exception (fixes rare segfaults) (#7350) 2019-10-17 17:41:27 +03:00
alesapin
155731a676 Merge branch 'master' into dictionaries_ddl_interpreter 2019-10-14 11:49:18 +03:00
Vitaly Baranov
18ccb4d64d Move backQuote() and quoteString() to a separate file, use StringRefs. 2019-10-11 00:18:52 +03:00
Vitaly Baranov
12d1d87d64 Move authentication code to a separate class. 2019-10-11 00:18:52 +03:00
alesapin
2a5713e6e4 Compilable code 2019-10-10 23:47:47 +03:00
millb
350f0b8a1c Merge remote-tracking branch 'origin/url_engine_problem' into url_engine_problem 2019-10-10 16:47:17 +03:00
millb
a00089f0b4 Renamed StorageOfAllowedURL -> RemoteHostFilter 2019-10-10 15:58:06 +03:00
Mikhail Korotov
769ec9ea6c
Update config.xml 2019-10-10 01:22:05 +03:00
millb
a97739f67f Created class StorageOfAllowedURL
Created test for table function remote and remoteSecure
Created check allowed url in remote and remoteSecure
Fixed tests
2019-10-09 23:29:41 +03:00
Alexey Milovidov
a19f6513ce Added example config with macros 2019-10-09 04:15:23 +03:00
alexey-milovidov
5c2d478543
Merge pull request #7212 from ClickHouse/aku/field-includes
Include Field.h and FieldVisitor.h into fewer files.
2019-10-08 22:40:30 +03:00
alexey-milovidov
b9aa5d812d
Merge pull request #7171 from nikvas0/nikvas0/lazy_db
Lazy Database
2019-10-08 22:15:55 +03:00
Alexander Kuzmenkov
367c537106 Do not include Field.h everywhere through SettingsCommon.h.
Move the methods of SettingsCollection<> that are dependent on Field to
a separate file, and include it once for each instantiation. This allows
to work on Field without always recompiling the entire project.
2019-10-07 20:23:05 +03:00
proller
594c535bd4 Build fixes (#7063)
* Build fixes

*            fix

* Fix

* fix

* Fix install(export..

* Freebsd fixes

* Freebsd fixes

* Fix warning

* fix

* More PRIVATE linking

* Fix review

* Timmy

* fix

* fix
2019-10-05 22:25:31 +03:00
Alexander Tokmakov
a58996817f allow query parameters in Values 2019-10-02 22:54:40 +03:00
Vasilyev Nikita
5793a02ef9 impr Lazy database 2019-10-01 15:44:17 +03:00
Nikita Mikhaylov
d47d4cd6c1 parallel parsing 2019-10-01 13:48:46 +03:00
Azat Khuzhin
740e4c5851
Avoid use of uninitialized values in MetricsTransmitter
Defer thread creation after values had been initialized correctly.
2019-10-01 10:16:39 +03:00
millb
9246f258e8 Created ability to configure allowed URL in config.xml without tests. 2019-09-30 18:29:05 +03:00
Amos Bird
dadc613072 Get rid of malloc symbols in libcommon (#7134) 2019-09-30 14:58:32 +03:00
alesapin
4fdb5a61e5 Rename ExternalDictionaries to ExternalDictionariesLoader 2019-09-26 13:41:33 +03:00
Alexander Kuzmenkov
f3bde19b74 Do not use iterators in find() and emplace() methods of hash tables.
Instead, these methods return a pointer to the required data as they are
stored inside the hash table. The caller uses overloaded functions to
get the key and "mapped" values from this pointer. Such an interface
avoids the need for constructing iterator-like wrapper objects, which is
especially important for compound hash tables such as the future
StringHashMap.
2019-09-24 17:44:35 +03:00
Ivan Lezhankin
4b143c3e0f Merge remote-tracking branch 'upstream/master' into issue-6459 2019-09-23 16:07:19 +03:00
Alexey Milovidov
70bc89557f Updates for #6969 2019-09-23 01:57:06 +03:00
Azat Khuzhin
59763cbb3a
Add ability to send events with absolute values to graphite
<events> sends only deltas for events, while sometimes it is more
convenient to get absolute values (for example to calculate rate).

This patch introduces another directive -- <events_absolute> that will handle
this.

v0: use <events> as bool/string
v2: use <events_absolute>true</events_absolute> over <events>absolute</events>
2019-09-21 02:46:47 +03:00
Ivan Lezhankin
9e46118edd Merge remote-tracking branch 'upstream/master' into issue-6459 2019-09-20 21:25:55 +03:00
Ivan Lezhankin
8cb0e58bc7 Fix build for all targets 2019-09-20 21:13:51 +03:00
alexey-milovidov
d8cd375c8b
Merge pull request #6958 from proller/fix25
Build fixes
2019-09-18 01:48:46 +03:00
alesapin
110568f980
Merge pull request #6947 from yandex/add_gdb_index
Add gdb index to builds with debug info
2019-09-17 17:59:27 +03:00
proller
e7f069c78c fixes 2019-09-17 12:33:46 +00:00
alesapin
33c3f7a6a2 Add gdb-index for gold linker with gdb-add-index tool 2019-09-17 12:49:27 +03:00
alesapin
719854b49b
Merge pull request #6939 from dimarub2000/perfomance_test_all_of
all_of setting in performance-test now works properly
2019-09-16 13:04:23 +03:00
Ivan Lezhankin
8c356a3830 WIP 2019-09-15 18:20:31 +03:00
Ivan Lezhankin
01fdb802d4 Useful changes 2019-09-15 13:35:12 +03:00