thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-17 13:13:36 +00:00
Code Issues Packages Projects Releases Wiki Activity
78,318 Commits 9,138 Branches 1,774 Tags 2.1 GiB
5a273d4dda
Commit Graph

726 Commits

Author SHA1 Message Date
Alexey Milovidov
02ce9cc725 Untangle UUID 2021-05-04 01:46:51 +03:00
Alexey Milovidov
1fda866881 Merge branch 'master' into normalize-bigint 2021-05-02 21:24:45 +03:00
alexey-milovidov
7ca42d2ce1
Merge pull request #23805 from ClickHouse/fix-hedged-requests-ext-tables 
Fix sending external table for hedged connections.
 2021-05-01 08:13:13 +03:00
kssenii
7643d84095 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-30 22:02:51 +00:00
Nikolai Kochetov
0c063755d9 Fix style 2021-04-30 18:21:55 +03:00
Nikolai Kochetov
352ec049ad Fix sending external table for hedged connections. 2021-04-30 17:19:48 +03:00
Maksim Kita
318c4bb80d Add examples folder filter to ya.make.in 2021-04-30 11:25:52 +03:00
Maksim Kita
0d0a14a925 Poco::Timespan reference fix 2021-04-29 19:11:20 +03:00
Nikita Mikhaylov
9f55424250 move to examples everywhere 2021-04-27 01:51:42 +03:00
Nikolai Kochetov
092ff0faeb
Merge pull request #23524 from kssenii/insert-union-select 
Allow to insert default instead of Null in insert-select / insert-select-union-all
 2021-04-26 12:25:49 +03:00
Alexey Milovidov
649550c5ab Attempt to normalize big integers 2021-04-25 12:30:43 +03:00
kssenii
8f34199538 Fixes 2021-04-23 12:36:40 +00:00
kssenii
54a7a6ef0d Allow null to default convertion only if column has default 2021-04-23 09:55:57 +00:00
Ivan Lezhankin
d9626e73c3 Revert "CC" 
This reverts commit ee194928d2.
 2021-04-22 16:55:18 +03:00
Anton Popov
3548e31afd fix corner cases in vertical merges with ReplacingMergeTree 2021-04-22 01:27:18 +03:00
Ivan Lezhankin
ee194928d2 CC 2021-04-21 16:19:28 +03:00
Ivan Lezhankin
7a88ca31cc Merge branch 'master' into async-insert 2021-04-19 18:33:43 +03:00
Ivan Lezhankin
6ed9e34750 [WIP] 2021-04-19 17:51:26 +03:00
kssenii
88b50621d4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-18 14:40:37 +00:00
Ivan Lezhankin
7f43dddf0b CC 2021-04-16 20:49:38 +03:00
ruct
303813c1d8 Maybe 2021-04-16 16:07:56 +03:00
ruct
adbd2d65c2 Try 2021-04-16 16:07:56 +03:00
Azat Khuzhin
6b9534cad8 Add copyData() with std::atomic and progress callback 2021-04-14 23:15:31 +03:00
Azat Khuzhin
3a2e8fea68 Change logging levels for PushingToViewsBlockOutputStream 
Use different logging levels for overall push and push per view.
 2021-04-14 23:15:31 +03:00
Ivan Lezhankin
9e5d274467 Merge remote-tracking branch 'upstream/master' into async-insert 2021-04-14 20:16:05 +03:00
Maksim Kita
470b0db8a2
Merge pull request #23049 from ClickHouse/remove-useless-code-6 
Remove useless files
 2021-04-14 13:27:54 +03:00
Alexey Milovidov
07b610cf70 Remove useless files 2021-04-13 23:04:13 +03:00
Nikita Mikhaylov
024374a2ec review fixes 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
7a68820342 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
c333c3dedb review fixes 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
efef179b89 remove prints 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
36a8419f60 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
8a4b5a586e fixed uuid 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
7276b40556 better 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4843f86329 use only one connection between initiator and worker 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
a5d55781b1 remote debug prints 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
4e4b383214 added hash of itiator address 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
2549468c14 better 2021-04-13 22:39:41 +03:00
kssenii
01075677cf Better 2021-04-11 20:22:33 +00:00
kssenii
c1adece94f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-11 06:00:18 +00:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
kssenii
f7361250b2 Fixes 2021-04-09 11:02:12 +00:00
kssenii
4482a35a3a Check table in a suggested way 2021-04-08 20:39:16 +00:00
kssenii
b544c9fe9a Add default template 2021-04-08 13:08:51 +00:00
kssenii
8586565ce9 Move connection-related filed to Core/PostgreSQL 2021-04-08 12:43:02 +00:00
kssenii
9fa21a6272 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-08 12:18:54 +00:00
Kseniia Sumarokova
ad85467128
Merge pull request #22668 from kssenii/postgres-add-bool 
Add bool type in postgres engine
 2021-04-06 13:29:33 +03:00
kssenii
cc2fdc6c62 Add bool type 2021-04-05 14:49:52 +00:00
kssenii
99f0b08ac3 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-01 13:48:33 +00:00
Alexey Milovidov
b0539eb28f Move method 2021-04-01 06:28:25 +03:00
kssenii
ce05087b1b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-03-30 17:27:10 +00:00
Ivan Lezhankin
de05803177 Merge branch 'master' into async-insert 2021-03-29 20:59:02 +03:00
Anton Popov
ea82e7725f
Merge pull request #21562 from CurtizJ/serialization-refactoring-4 
Refactoring of data types serialization
 2021-03-29 16:36:44 +03:00
kssenii
22b515fbc9 Add namespace, simplify names 2021-03-27 20:14:02 +00:00
Azat Khuzhin
82c79fe4ce Fix query cancellation with use_hedged_requests=0 and async_socket_for_remote=1 
In #21643 async_socket_for_remote=1 was fixed to avoid leaving the
connection in the unsynchronised state.

But one should not try to wait for the current packet in case of timeout
because this will exceed the timeout.

Anyway if the timeout is exceeded, then the connection will be shutdown
(disconnected), so it will not left in an unsynchronised state.
 2021-03-26 21:24:42 +03:00
Anton Popov
6a15431be7 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-25 15:57:35 +03:00
Alexander Kuzmenkov
2f5dbf57b6
Merge pull request #21839 from kssenii/add-postgres-connection-pool 
Add connection pool for postgres engine
 2021-03-22 19:49:51 +03:00
kssenii
caffccd73e Fixes 2021-03-20 11:53:41 +00:00
kssenii
619f0f51fd Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-20 09:47:05 +00:00
Ivan Lezhankin
4f1fad97ab Merge remote-tracking branch 'upstream/master' into async-insert 2021-03-19 23:41:01 +03:00
kssenii
9057aad798 Better version 2021-03-19 08:11:36 +00:00
kssenii
f1ef87d966 Fix 2021-03-18 20:04:54 +00:00
Nikolai Kochetov
c3c393a7aa Merge branch 'master' into refactor-actions-dag 2021-03-18 14:33:07 +03:00
kssenii
3903d59d30 Better 2021-03-17 14:34:04 +00:00
kssenii
ae64a24844 Add connection pool 2021-03-17 13:55:47 +00:00
kssenii
e722cee375 Better templates 2021-03-17 10:52:56 +00:00
alexey-milovidov
d02726bcac
Merge pull request #9404 from Enmk/DateTime64_extended_range 
Date time64 extended range
 2021-03-17 11:06:03 +03:00
kssenii
a03e849a8f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-16 10:06:10 +00:00
Alexey Milovidov
3f67f4f47b Saturation for DateTime 2021-03-15 23:40:33 +03:00
Alexey Milovidov
671395e8c8 Most likely improve performance 2021-03-15 22:23:27 +03:00
Anton Popov
81ac6382a3 slightly better performance 2021-03-13 21:05:18 +03:00
Azat Khuzhin
65f90f2ce9 Fix distributed requests cancellation with async_socket_for_remote=1 
Before this patch for distributed queries, that requires cancellation
(simple select from multiple shards with limit, i.e. `select * from
remote('127.{2,3}', system.numbers) limit 100`) it is very easy to
trigger the situation when remote shard is in the middle of sending Data
block while the initiator already send Cancel and expecting some new
packet, but it will receive not new packet, but part of the Data block
that was in the middle of sending before cancellation, and this will
lead to some various errors, like:
- Unknown packet X from server Y
- Unexpected packet from server Y
- and a lot more...

Fix this, by correctly waiting for the pending packet before
cancellation.

It is not very easy to write a test, since localhost is too fast.

Also note, that it is not possible to get these errors with hedged
requests (use_hedged_requests=1) since handle fibers correctly.

But it had been disabled by default for 21.3 in #21534, while
async_socket_for_remote is enabled by default.
 2021-03-11 21:55:21 +03:00
Anton Popov
bc417cf54a refactoring of serializations 2021-03-09 17:46:52 +03:00
Nikolai Kochetov
a669f7d641 Merge branch 'master' into refactor-actions-dag 2021-03-05 18:21:14 +03:00
Nikolai Kochetov
9a39459888 Refactor ActionsDAG 2021-03-04 20:38:12 +03:00
Ivan Lezhankin
74214865eb Merge remote-tracking branch 'upstream/master' into async-insert 2021-03-04 14:14:28 +03:00
Ivan Lezhankin
b726845be2 [WIP] 2021-03-04 14:10:21 +03:00
Alexey Milovidov
4e8239e098 Merge branch 'master' into DateTime64_extended_range 2021-03-03 23:43:20 +03:00
Nikolai Kochetov
4775ea305e Remove index by name from ActionsDAG 2021-03-02 20:51:54 +03:00
Pavel Kruglov
153bfbfc28 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-03-02 11:59:32 +03:00
kssenii
a3c7402200 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-02 07:07:56 +00:00
Maksim Kita
315824978d CheckConstraintsBlockOutputStream optimize nullable column case 2021-02-27 19:19:21 +03:00
Ivan Lezhankin
c9dc07fddd Another fix 2021-02-26 20:37:34 +03:00
Ivan Lezhankin
dcf907649b Fix test 2021-02-26 18:47:57 +03:00
Maksim Kita
da321c2bfe Fixed check for null value in null map 2021-02-25 16:08:04 +03:00
Maksim Kita
23af53067d Updated support for Nullable column 2021-02-25 14:27:46 +03:00
Maksim Kita
2eec1d021b Fixed unused code 2021-02-25 14:27:46 +03:00
Maksim Kita
8fec34af12 Constraints complex types support 2021-02-25 14:27:46 +03:00
Vasily Nemkov
2d03d330bc Extended range of DateTime64 to years 1925 - 2238 
The Year 1925 is a starting point because most of the timezones
switched to saner (mostly 15-minutes based) offsets somewhere
during 1924 or before. And that significantly simplifies implementation.

2238 is to simplify arithmetics for sanitizing LUT index access;
there are less than 0x1ffff days from 1925.

* Extended DateLUTImpl internal LUT to 0x1ffff items, some of which
  represent negative (pre-1970) time values.
  As a collateral benefit, Date now correctly supports dates up to 2149
  (instead of 2106).
* Added a new strong typedef ExtendedDayNum, which represents dates
  pre-1970 and post 2149.
* Functions that used to return DayNum now return ExtendedDayNum.
* Refactored DateLUTImpl to untie DayNum from the dual role of being
  a value and an index (due to negative time). Index is now a different
  type LUTIndex with explicit conversion functions from DatNum, time_t,
  and ExtendedDayNum.
* Updated DateLUTImpl to properly support values close to epoch start
  (1970-01-01 00:00), including negative ones.
* Reduced resolution of DateLUTImpl::Values::time_at_offset_change
  to multiple of 15-minutes to allow storing 64-bits of time_t in
  DateLUTImpl::Value while keeping same size.
* Minor performance updates to DateLUTImpl when building month LUT
  by skipping non-start-of-month days.
* Fixed extractTimeZoneFromFunctionArguments to work correctly
  with DateTime64.
* New unit-tests and stateless integration tests for both DateTime
  and DateTime64.
 2021-02-24 17:08:35 +02:00
kssenii
d590c32708 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-02-20 18:31:26 +00:00
Nikolai Kochetov
ed4697cffc Fix timeout in epoll_wait for RemoteQueryExecutorReadContext 2021-02-19 12:20:24 +03:00
Nikolai Kochetov
b9d6df9618 Check for eintr in epoll_wait 2021-02-19 11:49:41 +03:00
Ivan Lezhankin
dee051dc8e Merge remote-tracking branch 'upstream/master' into async-insert 2021-02-17 21:48:48 +03:00
Ivan Lezhankin
f2b1708164 Refactor inserts and its tails 2021-02-15 21:57:35 +03:00
Kruglov Pavel
598576ce70
Merge branch 'master' into hedged-requests 2021-02-15 16:51:45 +03:00
kssenii
95c93aca41 Better table sync 2021-02-13 21:03:25 +00:00
kssenii
5bc0010f94 Better 2021-02-12 21:14:36 +00:00
kssenii
0d19992128 Replicate delete queries 2021-02-12 21:12:27 +00:00
kssenii
4aadd0c3f2 Replicate insert queries 2021-02-12 21:11:55 +00:00
kssenii
b842ce114b Initial sync into ReplacingMergeTree table, select via nested table 2021-02-12 21:11:09 +00:00
alexey-milovidov
dc3ffd3fe2
Merge pull request #19451 from azat/safe-writes 
Do not silently ignore write errors
 2021-02-11 21:19:11 +03:00
alesapin
c2bb2c2902
Merge pull request #20097 from ClickHouse/remove-adding-missed-step 
Build actions dag to evaluate missing defaults.
 2021-02-11 10:51:21 +03:00
Nikolai Kochetov
af214e794f Review fixes. 2021-02-10 15:45:39 +03:00
Nikolai Kochetov
27d607a955 Respect header in addMissingDefaults 2021-02-09 22:48:34 +03:00
Pavel Kruglov
9048dc43d4 Fix style and build 2021-02-06 22:13:50 +03:00
Pavel Kruglov
8ff3dde290 Add sendIgnoredPartUUIDs to HedgedRequests 2021-02-06 18:26:36 +03:00
Pavel Kruglov
f946aab759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-06 17:38:56 +03:00
Pavel Kruglov
0704d3cf27 Refactor 2021-02-06 03:54:27 +03:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID 
CLICKHOUSE-606: query deduplication based on parts' UUID
 2021-02-05 22:47:34 +03:00
Nikolai Kochetov
85c175883e Rename functions. 2021-02-05 18:11:26 +03:00
Nikolai Kochetov
9869f70a0d Remove AddMissed step and transform. 2021-02-05 14:41:44 +03:00
alesapin
7cbc135e72 More isolated code 2021-02-05 12:54:34 +03:00
Azat Khuzhin
98e3a99a88 Do not catch exceptions during final flush in writers destructors 
Since this hides real problems, since destructor does final flush and if
it fails, then data will be lost.

One of such examples if MEMORY_LIMIT_EXCEEDED exception, so lock
exceptions from destructors, by using
MemoryTracker::LockExceptionInThread to block these exception, and allow
others (so std::terminate will be called, since this is c++11 with
noexcept for destructors by default).

Here is an example, that leads to empty block in the distributed batch:

    2021.01.21 12:43:18.619739 [ 46468 ] {7bd60d75-ebcb-45d2-874d-260df9a4ddac} <Error> virtual DB::CompressedWriteBuffer::~CompressedWriteBuffer(): Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 332.07 GiB (attempt to allocate chunk of 4355342 bytes), maximum: 256.00 GiB, Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception<>() @ 0x86f7b88 in /usr/bin/clickhouse
    ...
    4. void DB::PODArrayBase<>::resize<>(unsigned long) @ 0xe9e878d in /usr/bin/clickhouse
    5. DB::CompressedWriteBuffer::nextImpl() @ 0xe9f0296 in /usr/bin/clickhouse
    6. DB::CompressedWriteBuffer::~CompressedWriteBuffer() @ 0xe9f0415 in /usr/bin/clickhouse
    7. DB::DistributedBlockOutputStream::writeToShard() @ 0xf6bed4a in /usr/bin/clickhouse
 2021-02-05 01:31:45 +03:00
Nikolai Kochetov
d9aa1096cf Build actions dag to evaluate missing defaults. 2021-02-04 23:36:50 +03:00
Azat Khuzhin
c59f22d7b4 Change PushingToViewsBlockOutputStream::process, to accept ref to ViewInfo 
It is safe, no need to use quirks with passing view_num.
 2021-02-04 00:44:24 +03:00
Azat Khuzhin
50d8a87c27 Fix logging of elapsed time while pushing to views 
Logging from PushingToViewsBlockOutputStream::write() is incorrect,
since it does not takes into account squashing (default to 1<<20 rows
and 256<<20), whle write will be done from
PushingToViewsBlockOutputStream::writeSuffix().

Fixes: #19378
 2021-02-04 00:44:24 +03:00
Aleksei Semiglazov
d05c6446b9
Send cancel packet and cancel read_context before retrying the query 2021-02-03 00:07:00 +00:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
Pavel Kruglov
60a92e9a99 Fix build, add comments, update tests 2021-02-02 15:14:31 +03:00
Pavel Kruglov
5b16a54233 Fix synchronization 2021-02-01 20:23:46 +03:00
Pavel Kruglov
25e85d71ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-01-29 21:08:47 +03:00
Pavel Kruglov
01a0cb649a Fix build, style, tests 2021-01-29 18:46:28 +03:00
Anton Popov
e5125b8c73 add comments and test for compatibility 2021-01-28 02:39:15 +03:00
Anton Popov
a8f3078ce9 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-27 19:48:55 +03:00
Pavel Kruglov
b3b832cde7 Work with any number of replicas simultaneously, support max_parallel_replicas 2021-01-27 12:33:11 +03:00
Azat Khuzhin
2c42600cf9 Add log message with elapsed time while pushing to view 
Since right now theses queries are not logged and there is no way to
determine the time that it takes.

Proper fix will be to account them in system.processes (and all other
places), but this is not that easy.
 2021-01-21 21:09:23 +03:00
Pavel Kruglov
97b5179e55 Implement HedgedRequests 2021-01-19 22:41:05 +03:00
alexey-milovidov
15f4ae26c2
Merge pull request #17310 from CurtizJ/multiple-nested 
Allow nested with multiple nesting and subcolumns of complex types
 2021-01-17 15:00:26 +03:00
alexey-milovidov
7de745ce77
Merge pull request #18554 from kssenii/pg2ch 
Add PostgreSQL table function, dictionary source, database engine
 2021-01-16 23:55:05 +03:00
Alexey Milovidov
24c8e53440 Merge branch 'master' into multiple-nested 2021-01-16 16:28:40 +03:00
Azat Khuzhin
ee45c122ea Fix leaking of pipe fd for async_socket_for_remote 2021-01-16 01:57:36 +03:00
Azat Khuzhin
cf085b0687 Split RemoteQueryExecutorReadContext into module part 2021-01-16 01:57:36 +03:00
Alexey Milovidov
a19e7edd14 Merge branch 'master' into kssenii-pg2ch 2021-01-15 17:33:19 +03:00
alexey-milovidov
78fff6bc39
Merge branch 'master' into multiple-nested 2021-01-15 14:54:27 +03:00
Anton Popov
0e903552a0 fix TTLs with WHERE 2021-01-13 17:04:27 +03:00
Anton Popov
15ead18673 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-12 19:46:10 +03:00
Anton Popov
60b88986bf minor changes near TTL computation 2021-01-12 19:42:49 +03:00
Anton Popov
5822ee1f01 allow multiple rows TTL with WHERE expression 2021-01-12 02:07:21 +03:00
alesapin
c5df8f324c More checks in writer wide 2021-01-11 15:03:00 +03:00
kssenii
38a9cba850 Fix 2021-01-11 10:55:38 +00:00
Anton Popov
36ae0e4d35 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-11 13:51:12 +03:00
kssenii
6ec59f1304 Update libpq, tiny fix 2021-01-10 15:38:46 +00:00
kssenii
d952b0897e Minor adjustments 2021-01-10 12:06:18 +00:00
Alexey Milovidov
8af19c3251 Fix Arcadia 2021-01-07 15:29:02 +03:00
Alexey Milovidov
60d4db421c Fix Arcadia 2021-01-07 06:45:12 +03:00
alexey-milovidov
72b142a00a
Merge branch 'master' into pg2ch 2021-01-06 23:18:59 +03:00
Nikolai Kochetov
a20c4cce76
Merge pull request #18715 from ClickHouse/try-fix-tsan-forremote-query-executor 
Use relaxed for flag in RemoteQueryExecutorReadContext.
 2021-01-06 18:04:30 +03:00
alexey-milovidov
70d899340e
Update PushingToViewsBlockOutputStream.h 2021-01-05 20:10:12 +03:00
Alexey Milovidov
dab4719aac Remove some headers 2021-01-05 06:22:06 +03:00
Nikolai Kochetov
e48ea5c5b3 Use relaxed for flag in RemoteQueryExecutorReadContext. 2021-01-04 10:59:01 +03:00
Alexey Milovidov
4b3ae495d6 Merge branch 'master' into CurtizJ-multiple-nested 2021-01-02 00:25:16 +03:00
kssenii
8efd85bef2 Fix build 2020-12-29 16:56:50 +00:00
Anton Popov
a8f1786d95 fix TTL with GROUP BY 2020-12-29 18:19:11 +03:00
kssenii
9b25890674 Generate ya.make, fix fast test 2020-12-28 12:54:52 +00:00
Nikita Mikhailov
8667842390 ya make update 2020-12-28 15:54:46 +03:00
Nikita Mikhailov
964e12d8b7 Fix 2020-12-28 15:53:58 +03:00
kssenii
8f8920a7ee Add table cache, better drop table 2020-12-27 15:52:15 +00:00
kssenii
375e8e9736 Add postgres dictionary source 2020-12-27 12:18:09 +00:00
kssenii
aa3484515d Better 2020-12-27 12:17:23 +00:00
kssenii
69f6714461 Add table function 2020-12-27 12:16:46 +00:00
kssenii
42023f4b95 Support insert into table 2020-12-27 12:16:27 +00:00
Anton Popov
11283e3d81 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-25 21:25:59 +03:00
Anton Popov
b60c00ba74 refactoring of TTL stream 2020-12-25 18:46:13 +03:00
Anton Popov
42725b27c0 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-24 15:18:15 +03:00
Nikolai Kochetov
97ffc40321
Merge pull request #18409 from ClickHouse/try-to-fix-ya-check 
Try fix ya.make
 2020-12-23 20:02:53 +03:00
Nikita Mikhaylov
c005dcdd26
Merge pull request #17641 from nikitamikhaylov/parallel_row_numbers 
Added an offset to exception message for parallel parsing
 2020-12-23 17:24:35 +03:00
Nikolai Kochetov
7a9282ee7b Try fix ya.make 2020-12-23 15:31:16 +03:00
Anton Popov
57857dda63 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-23 15:16:43 +03:00
Nikolai Kochetov
af7f5c9518
Merge pull request #17868 from ClickHouse/async-read-from-socket 
Async read from socket
 2020-12-23 12:20:42 +03:00
Nikita Mikhaylov
e1fc9122cc
Update ParallelParsingBlockInputStream.cpp 2020-12-23 05:28:53 +03:00
Nikita Mikhaylov
54d2fe847f
Update src/DataStreams/ParallelParsingBlockInputStream.cpp 
Co-authored-by: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
 2020-12-23 05:02:54 +03:00
Nikita Mikhailov
138b74aa3b better 2020-12-23 01:04:02 +03:00
nikitamikhaylov
8d4465ddeb better 2020-12-23 01:02:01 +03:00
nikitamikhaylov
a285cb83df style 2020-12-23 01:02:01 +03:00
nikitamikhaylov
c60c161168 add ParsingException 2020-12-23 01:02:01 +03:00
nikitamikhaylov
27f647f93d done 2020-12-23 01:01:05 +03:00
Nikolai Kochetov
dc450e9a83 Update comment. 2020-12-22 11:55:21 +03:00
Nikolai Kochetov
110e76e71d Fix special build. 2020-12-21 15:42:57 +03:00
Nikolai Kochetov
855c4bc7d7 Added comments. 2020-12-18 18:12:31 +03:00
Nikolai Kochetov
01c8d5dccc Use replace fiber to callback in ReadBufferFromPocoSocket 2020-12-18 17:24:44 +03:00
Nikolai Kochetov
c7ef57c6fd Add setting async_socket_for_remote 2020-12-18 16:15:03 +03:00
Anton Popov
009efeff40 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-18 16:14:09 +03:00
Anton Popov
a42b00c9aa Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-17 20:43:23 +03:00
Nikolai Kochetov
a860f128f5 Try fix race in RemoteQueryExecutor. 2020-12-17 13:07:28 +03:00
Nikolai Kochetov
263aa88cd9 Fixing tests. 2020-12-16 23:27:31 +03:00
nikitamikhaylov
bcd6fc1c88 better 2020-12-16 14:55:33 +03:00
nikitamikhaylov
18d52dbc63 better 2020-12-16 14:55:33 +03:00
Nikolai Kochetov
645e0d2c0c Fixing tests. 2020-12-15 21:23:41 +03:00
Nikolai Kochetov
4905201985 Merge branch 'master' into async-read-from-socket 2020-12-15 21:04:47 +03:00
Nikolai Kochetov
6defcbb662
Merge branch 'master' into optimize-data-on-insert 2020-12-15 16:50:42 +03:00
Nikolai Kochetov
716e0b472b Update FiberStack 2020-12-15 13:08:13 +03:00
Nikolai Kochetov
c4e80ccfc5 Try fix special builds. 2020-12-15 11:27:42 +03:00
Nikolai Kochetov
35dda5851c Try remove connection lock. 2020-12-15 10:02:15 +03:00
Nikolai Kochetov
00de8abb0a Fixing tests. 2020-12-15 00:07:10 +03:00
Nikolai Kochetov
fc89078f81 Fixing tests. 2020-12-15 00:03:11 +03:00
Nikolai Kochetov
f26c0aafc2 Fixing tests. 2020-12-14 20:46:27 +03:00
Nikolai Kochetov
3af47d2bda Fixing tests. 2020-12-14 19:18:12 +03:00
Nikolai Kochetov
db9ad80168 Fixing tests. 2020-12-14 19:16:08 +03:00
Nikolai Kochetov
8de5cd5bc7 Merge branch 'master' into async-read-from-socket 2020-12-14 17:45:38 +03:00
Nikolai Kochetov
75ac87c241 Fixing build. 2020-12-14 17:42:12 +03:00
Nikolai Kochetov
e295dfe6e3 Use ucontext for asan 2020-12-14 17:42:08 +03:00
Azat Khuzhin
77a1d00dee Add new remote context as interpreter context to the Pipe 2020-12-12 17:43:10 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers 
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
 2020-12-12 17:43:10 +03:00
Nikolai Kochetov
32b38f389e Merge branch 'master' into async-read-from-socket 2020-12-09 17:15:36 +03:00
Nikolai Kochetov
e8667bad45 Fix build and tests. 2020-12-09 17:12:27 +03:00
Nikolai Kochetov
156f44808f Fixing crash. 2020-12-09 17:11:45 +03:00
Nikolai Kochetov
088c128832 Fixing build and tests. 2020-12-09 17:11:20 +03:00
Anton Popov
d7aad3bf79 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-07 19:13:01 +03:00
Pavel Kruglov
4b58528b9e Rename getPos to getRow, change mergeBlock, pass setting instead of context 2020-12-04 19:25:30 +03:00
Pavel Kruglov
905ba78adc Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-12-04 18:56:46 +03:00
Nikolai Kochetov
9ca837d535 Add async status to RemoteSource. 2020-12-04 13:52:57 +03:00
Nikolai Kochetov
92d60a7f38 Add AsyncTaskQueue. 2020-12-03 20:03:13 +03:00
Nikolai Kochetov
082a496364 Add async read to RemoteQueryExecutor 2020-12-03 15:21:10 +03:00
Nikolai Kochetov
e3946bc2b5 Add async read to RemoteQueryExecutor. 2020-12-02 20:02:14 +03:00
Alexey Milovidov
7e06037e59 Fix typo 2020-11-28 08:38:05 +03:00
alexey-milovidov
fb3a69b298
Merge pull request #17254 from azat/fix-dist-query-cancelation 
Fix "Unexpected packet Data received from client"  for Distributed queries
 2020-11-26 03:46:27 +03:00
Anton Popov
b384beb564 Merge remote-tracking branch 'upstream/master' into HEAD 2020-11-23 17:46:51 +03:00
Azat Khuzhin
9689b293ca Fix "Unexpected packet Data received from client" error 
Fix query cancelation in case of Distributed queries with LIMIT (when
the initator does not required to read all the data), since this cannot
be done until the query was sent (from the Query packet up to the empty
data Block), otherwise you will get:

    2020.11.21 21:47:23.297161 [ 184 ] {} <Error> TCPHandler: Code: 101, e.displayText() = DB::Exception: Unexpected packet Data received from client, Stack trace:

    0. /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:129: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x244f5bc9 in /usr/bin/clickhouse
    1. /build/obj-x86_64-linux-gnu/../src/Common/Exception.cpp:40: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0xa14a421 in /usr/bin/clickhouse
    2. /build/obj-x86_64-linux-gnu/../src/Common/NetException.h:0: DB::TCPHandler::receiveUnexpectedData() @ 0x1e032a74 in /usr/bin/clickhouse
    3. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:824: DB::TCPHandler::receivePacket() @ 0x1e024685 in /usr/bin/clickhouse
    4. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:178: DB::TCPHandler::runImpl() @ 0x1e01736b in /usr/bin/clickhouse
    5. /build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:0: DB::TCPHandler::run() @ 0x1e035c1b in /usr/bin/clickhouse
    6. /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x243559cf in /usr/bin/clickhouse
    7. /build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:0: Poco::Net::TCPServerDispatcher::run() @ 0x24356521 in /usr/bin/clickhouse
    8. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:0: Poco::PooledThread::run() @ 0x24609175 in /usr/bin/clickhouse
    9. /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:0: Poco::ThreadImpl::runnableEntry(void*) @ 0x24603cb7 in /usr/bin/clickhouse
    10. start_thread @ 0x9669 in /usr/lib/x86_64-linux-gnu/libpthread-2.30.so
    11. __clone @ 0x1222b3 in /usr/lib/x86_64-linux-gnu/libc-2.30.so
 2020-11-22 12:42:15 +03:00
Anton Popov
1f65478b49 implement nested with multiple nesting 2020-11-21 20:27:42 +03:00
Alexander Kuzmenkov
7958b739e8 compile 2020-11-18 20:44:33 +03:00
Alexander Kuzmenkov
1570320e20 fixes for context hierarchy 2020-11-18 20:43:18 +03:00
Pavel Kruglov
547ec19fb3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-11-18 12:01:59 +03:00
Nikita Mikhaylov
68b6f08e36
Merge pull request #16721 from nikitamikhaylov/parsing-constraints 
Turn off parallel parsing when memory limit is small.
 2020-11-17 17:37:14 +03:00
alexey-milovidov
b494b18748
Merge pull request #17108 from ClickHouse/ya-make-owners 
Add "OWNER" to ya.make files.
 2020-11-17 05:19:46 +03:00
Alexey Milovidov
24f4fa6edf Follow Arcadia ya.make rules 2020-11-17 00:16:50 +03:00
Alexey Milovidov
3df04ce0c2 Follow Arcadia ya.make rules 2020-11-16 21:24:58 +03:00
tavplubix
88de1b052c
Merge pull request #16048 from ClickHouse/materialized-view-check-stack-size 
Fix possible stack overflow in materialized views.
 2020-11-16 12:02:27 +03:00
Pavel Kruglov
8d5e0784d3 Add setting optimize_on_insert 2020-11-12 23:37:23 +03:00
Nikita Mikhaylov
33bada767c
Merge branch 'master' into parsing-constraints 2020-11-12 23:25:39 +03:00
Nikolai Kochetov
46f70dd0de Merge branch 'master' into actions-dag-f14 2020-11-12 11:54:44 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Nikolai Kochetov
6717c7a0af Merge branch 'master' into actions-dag-f14 2020-11-09 14:57:48 +03:00
alexey-milovidov
f4ba5f1f9a
Merge pull request #16772 from ClickHouse/fix-stringstream 
Fix "server failed to start" error
 2020-11-08 14:27:08 +03:00
Alexey Milovidov
5314185e25 Merge branch 'master' into azat-optimize_skip_unused_shards-optimization 2020-11-08 00:17:59 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
nikitamikhaylov
dabb23b668 done 2020-11-06 21:55:13 +03:00
Nikolai Kochetov
9aeb757da4 Merge branch 'master' into actions-dag-f14 2020-11-06 15:04:20 +03:00
Alexey Milovidov
b56486510f Improve Arcadia 2020-11-05 12:55:01 +03:00
Nikolai Kochetov
9f7de6a1a4 Try fix header for TTLBlockInputStream. 2020-11-03 17:26:47 +03:00
Nikolai Kochetov
6767a226fc Merge branch 'master' into actions-dag-f14 2020-11-03 15:21:06 +03:00
Nikolai Kochetov
07a7c46b89 Refactor ExpressionActions [Part 3] 2020-11-03 14:28:28 +03:00
Maxim Akhmedov
3627fabfb9 Remove -g0 form Arcadia build settings. 2020-10-29 17:37:23 +03:00
Alexey Milovidov
32ed8c9681 Fix trailing whitespace 2020-10-26 22:12:40 +03:00
Alexey Milovidov
cc3feb36a6 Avoid stack overflow in materialized views, part 3: more checks 2020-10-16 03:21:50 +03:00
Amos Bird
47fcd8bffb
Fix some problems introduced by refactoring. 2020-10-12 17:58:09 +08:00
Amos Bird
d2dcfc3f0d
Refactor processors. 2020-10-12 17:30:05 +08:00
alexey-milovidov
10e9d14466
Merge pull request #15818 from ClickHouse/style-pragma-once 
Check for #pragma once in headers
 2020-10-11 13:14:09 +03:00
alexey-milovidov
e16c1c781e
Merge pull request #15816 from ClickHouse/change-level-of-some-log-messages 
Change level of some log messages
 2020-10-11 13:13:13 +03:00
Alexey Milovidov
269b6383f5 Check for #pragma once in headers 2020-10-10 21:37:02 +03:00
Alexey Milovidov
7502fad77c Change level of some log messages 2020-10-10 20:47:34 +03:00
Alexey Milovidov
5b482f4191 Cleanups 2020-10-10 19:31:10 +03:00
Azat Khuzhin
b838214a35 Pass non-const SelectQueryInfo (and drop mutable qualifiers) 2020-10-02 22:42:35 +03:00
Azat Khuzhin
587cde853e Avoid skipping unused shards twice (for query processing stage and read itself) 2020-10-02 22:42:09 +03:00
alesapin
17b86dec96 Recursive defaults 2020-10-02 15:38:50 +03:00
Anton Popov
0d79474acc Fix TTL with GROUP BY and fix test according to new logic 2020-09-30 19:10:15 +03:00
Anton Popov
8ea4c2e26f Fix TTL in cases, when its expression is a function and is the same as ORDER BY key 2020-09-29 19:31:19 +03:00
Alexander Kuzmenkov
fb64cf210a straighten the protocol version 2020-09-17 17:37:29 +03:00
Nikolai Kochetov
e29f27e333
Merge pull request #14815 from ClickHouse/storage-read-query-plan 
Return query plan from storage.
 2020-09-15 18:34:11 +03:00
Nikolai Kochetov
6eac433c55 Review fixes. 2020-09-15 13:40:39 +03:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
Nikolai Kochetov
1b618fd130 Fix build 2020-09-14 23:56:56 +03:00
Nikolai Kochetov
d09422e60f Fix build. 2020-09-14 22:49:46 +03:00
Nikolai Kochetov
bcfc83022a Return query plan from storage. 2020-09-14 17:13:58 +03:00
alesapin
bd14c79b0b
Merge pull request #14494 from ClickHouse/recompression_in_background 
Recompression TTL
 2020-09-09 17:46:01 +03:00
Vasily Nemkov
3973a17530
MySql datatypes dateTime64 and decimal (#11512) 2020-09-09 15:18:02 +03:00
alesapin
a696cf12f6 Merge branch 'master' into recompression_in_background 2020-09-09 10:12:03 +03:00
alexey-milovidov
f2e9a09b11
Merge pull request #14585 from ClickHouse/join-processor 
Remove Join from ExpressionActions
 2020-09-08 23:56:09 +03:00
alesapin
f0f7f2b5d2 Merge branch 'master' into recompression_in_background 2020-09-08 15:43:37 +03:00
Nikolai Kochetov
39aad9979a Remove Join from ExpressionActions. 2020-09-08 13:40:53 +03:00
Ilya Yatsishin
11a247d2f4
Merge pull request #14556 from ClickHouse/fix-arcadia-2 
Lower binary size in "Arcadia" build
 2020-09-08 12:08:35 +03:00
Alexey Milovidov
e3924b8057 Fix "Arcadia" 2020-09-08 01:14:13 +03:00
alesapin
f274ffc9d1 Add comments 2020-09-07 10:59:14 +03:00
alesapin
fecb2f1311 Less copypaste 2020-09-04 17:08:43 +03:00
Nikolai Kochetov
92c937db8b Remove CreatingSetsBlockInputStream 2020-09-02 16:13:13 +03:00
alesapin
f0dc5a3085 First working test 2020-09-01 13:49:53 +03:00
alesapin
10c7a6c45e
Add ability to specify Default codec for columns (#14049) 
* Add ability to specify DefaultCompression codec which correspond to settings specified in config.xml

* Fix style

* Rename DefaultCompression to simple Default

* Fix compression codec

* Better codec description representation

* Less strange code and one method

* Fix delta
 2020-08-28 20:40:45 +03:00
alesapin
7428c46737 Merge branch 'master' into recompressing_in_background 2020-08-26 10:28:03 +03:00
alesapin
1180bad3a7 Add ability to specify DefaultCompression codec which correspond to settings specified in config.xml 2020-08-25 18:02:32 +03:00
Alexey Milovidov
612382b74e Minor modification 2020-08-24 17:45:00 +03:00
Alexey Milovidov
e1ccebe8f3 Minor modification 2020-08-24 17:42:12 +03:00
Azat Khuzhin
2e933a94a6 Handle Protocol::Server::Log in the RemoteQueryExecutor::finish() 
Even when finish() is called, there can be no EndOfStream sent, so it is
100% correct to handle Log packages there.

W/o fix in RemoteQueryExecutor:

    Code: 100, e.displayText() = DB::Exception: Unknown packet 10 from one of the following replicas: : While executing Remote (version 20.8.1.1) (from [::1]:56960) (in query: select * from remote('127.{2,3}', system.numbers) where number = 10 limit 1; ), Stack trace (when copying this message, always include the lines below):

    0. Common/StackTrace.cpp:291: StackTrace::tryCapture() @ 0x28b80a
    1. Common/StackTrace.cpp:256: StackTrace::StackTrace() @ 0x28b6ab
    2. Common/Exception.cpp:42: DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) @ 0x2481ba
    3. Common/Exception.h:28: DB::Exception::Exception<>() @ 0x334d48
    4. DataStreams/RemoteQueryExecutor.cpp:288: DB::RemoteQueryExecutor::finish() @ 0x330314
    5. Processors/Sources/RemoteSource.cpp:46: DB::RemoteSource::generate() @ 0x6fce5
 2020-08-17 21:55:09 +03:00
Azat Khuzhin
3c6a8ed7be Improve error message on unknown packet for distributed queries 
Although replica information is likely already empty (since it was
reseted due to error or disconnect), but let's keep it for now.
 2020-08-17 21:38:24 +03:00
Nikolai Kochetov
9b67cd9faf Merge branch 'master' into refactor-pipes-3 2020-08-10 10:50:17 +03:00
Alexey Milovidov
edd89a8610 Fix half of typos 2020-08-08 03:47:03 +03:00
Nikolai Kochetov
20e63d2271 Refactor Pipe [part 6] 2020-08-06 15:24:05 +03:00
Nikolai Kochetov
9385f3de0e Refactor Pipe [part 4]. 2020-08-04 16:06:59 +03:00
Nikolai Kochetov
09fbce1b1e Merge branch 'master' into refactor-pipes-3 2020-08-04 11:32:34 +03:00
Nikolai Kochetov
e411916bde Refactor Pipe [part 1]. 2020-08-03 14:33:11 +03:00
Alexey Milovidov
6f690b7c0d Normalize ya.make files, fix "Arcadia" build 2020-08-02 16:57:38 +03:00
Vxider
8868213104 Merged with master 2020-08-02 21:25:11 +08:00
Vitaly Baranov
18b21511a9
Merge pull request #13013 from vitlibar/implement-custom-settings 
Implement custom settings
 2020-08-02 05:01:14 +03:00
Nikolai Kochetov
2ae94f4570 Refactor Pipes part 1. 2020-07-31 19:54:54 +03:00
Vitaly Baranov
cd372de417 Use SettingUInt64 only in Settings, use UInt64 in other places. 2020-07-31 19:11:27 +03:00
Nikolai Kochetov
39530f837e Remove TreeExecutorBlockInputStream. 2020-07-31 16:23:19 +03:00
alexey-milovidov
065db05665
Merge pull request #12638 from ianton-ru/selected_rows_event 
Add SelectedRows and SelectedBytes events
 2020-07-31 12:48:46 +03:00
Alexander Kuzmenkov
f6f2812f73 Block structure mismatch is a program error 2020-07-28 21:35:18 +03:00
Anton Ivashkin
89a59d43f2 Merge 2020-07-27 17:59:12 +03:00
Vxider
c78dd2cd79 Merged with master 2020-07-27 18:11:26 +08:00
Azat Khuzhin
417dc43353 Fix thread group for ParallelParsingBlockInputStream after removing std::bind 2020-07-23 19:52:09 +03:00
Azat Khuzhin
a5885b476f Fix modernize-avoid-bind in ParallelParsingBlockInputStream 2020-07-23 11:31:13 +03:00