Commit Graph

44699 Commits

Author SHA1 Message Date
Alexey Milovidov
eb2ec74709 Fix comment 2020-07-13 01:32:24 +03:00
Alexey Milovidov
ab683860c9 Remove debug output 2020-07-13 01:31:08 +03:00
Alexey Milovidov
f6f8421c2f Better test 2020-07-13 01:30:27 +03:00
Alexey Milovidov
8f0ec09372 Fix error; add a test 2020-07-13 01:23:21 +03:00
Alexey Milovidov
8991192ddb Improved comment 2020-07-13 00:45:23 +03:00
Alexey Milovidov
7407a08750 Fix error 2020-07-13 00:44:27 +03:00
Alexey Milovidov
36de452fdd Merge branch 'master' into fix-limit-overflow 2020-07-13 00:24:24 +03:00
Alexey Milovidov
5ef8d49ebe Fix typo 2020-07-12 23:02:51 +03:00
Alexey Milovidov
3858164438 Fix UBSan report in base64 2020-07-12 22:38:37 +03:00
Alexey Milovidov
e8a3f0abb3 Correct default secure port for clickhouse-benchmark #11044 2020-07-12 21:54:13 +03:00
Alexey Milovidov
72ac7b008e Remove test with bug #10697 2020-07-12 21:02:06 +03:00
tavplubix
d64e51d1a7
Merge pull request #12421 from azat/lifetime-table-metrics
lifetime_rows/lifetime_bytes for Buffer engine
2020-07-12 19:00:02 +03:00
Ivan Blinkov
eef67fd9f6
Update arithmetic-functions.md (#12431)
Update its Chinese translation.
2020-07-12 18:52:48 +03:00
Tom Bombadil
60a151350b
Update in-functions.md (#12430)
optimize its Chinese doc translation.
2020-07-12 18:52:23 +03:00
Amos Bird
cac5a89169
Allow nullable key in MergeTree 2020-07-12 22:21:51 +08:00
Tom Bombadil
246c86d14a
Update arithmetic-functions.md
Update its Chinese translation.
2020-07-12 16:28:00 +08:00
Kseniia Sumarokova
2721f34fa6
[docs] add rabbitmq docs (#12326)
* [docs] add rabbitmq docs

* Fix

* Fix
2020-07-12 09:50:20 +03:00
Alexey Milovidov
49f60ef3a4 Fix build 2020-07-12 08:26:33 +03:00
Alexey Milovidov
ef930df484 Avoid overflow in LIMIT #10470 #11372 2020-07-12 08:18:01 +03:00
Alexey Milovidov
204a4af394 Rollback insertion error in Log engines #12402 2020-07-12 05:32:18 +03:00
Ivan Babrou
8784994d65 Allow conditions outside of PK with exact range
Conditions that are outside of PK are marked as `unknown` in `KeyCondition`,
so it's safe to allow them, as long as they are always combined by `AND`.
2020-07-11 18:59:26 -07:00
Alexey Milovidov
bc8e7e04df Lower block sizes and look what will happen #9248 2020-07-12 04:05:09 +03:00
alexey-milovidov
26d32f2ce5
Merge pull request #12412 from Jokser/s3-https
S3 HTTPS integration tests
2020-07-12 03:36:37 +03:00
Azat Khuzhin
3bee98c6f0 Fix lifetime_bytes/lifetime_rows for Buffer direct block write 2020-07-12 01:16:05 +03:00
Denis Glazachev
a6a6f43422 Fix the test: auth_params is a JSON object 2020-07-12 01:40:24 +04:00
alexey-milovidov
160dcf715d
Merge pull request #12383 from BohuTANG/mysql_write_through_poco_exception
Write through Poco::Exception exception to MySQL protocol client
2020-07-11 23:56:22 +03:00
alexey-milovidov
b1b4f8832d
Merge pull request #12397 from myrrc/bug/part-dict-update
Resolve #12098
2020-07-11 23:07:00 +03:00
Ivan Babrou
d9d8d0242e Optimize PK lookup for queries that match exact PK range
Existing code that looks up marks that match the query has a pathological
case, when most of the part does in fact match the query.

The code works by recursively splitting a part into ranges and then discarding
the ranges that definitely do not match the query, based on primary key.

The problem is that it requires visiting every mark that matches the query,
making the complexity of this sort of look up O(n).

For queries that match exact range on the primary key, we can find
both left and right parts of the range with O(log 2) complexity.

This change implements exactly that.

To engage this optimization, the query must:

* Have a prefix list of the primary key.
* Have only range or single set element constraints for columns.
* Have only AND as a boolean operator.

Consider a table with `(service, timestamp)` as the primary key.

The following conditions will be optimized:

* `service = 'foo'`
* `service = 'foo' and timestamp >= now() - 3600`
* `service in ('foo')`
* `service in ('foo') and timestamp >= now() - 3600 and timestamp <= now`

The following will fall back to previous lookup algorithm:

* `timestamp >= now() - 3600`
* `service in ('foo', 'bar') and timestamp >= now() - 3600`
* `service = 'foo'`

Note that the optimization won't engage when PK has a range expression
followed by a point expression, since in that case the range is not continuous.

Trace query logging provides the following messages types of messages,
each representing a different kind of PK usage for a part:

```
Used optimized inclusion search over index for part 20200711_5710108_5710108_0 with 9 steps
Used generic exclusion search over index for part 20200711_5710118_5710228_5 with 1495 steps
Not using index on part 20200710_5710473_5710473_0
```

Number of steps translates to computational complexity.

Here's a comparison for before and after for a query over 24h of data:

```
Read 4562944 rows, 148.05 MiB in 45.19249672 sec.,   100966 rows/sec.,   3.28 MiB/sec.
Read 4183040 rows, 135.78 MiB in 0.196279627 sec., 21311636 rows/sec., 691.75 MiB/sec.
```

This is especially useful for queries that read data in order
and terminate early to return "last X things" matching a query.

See #11564 for more thoughts on this.
2020-07-11 12:26:54 -07:00
Denis Glazachev
b68d7b6c24
Merge branch 'master' into ldap-per-user-authentication 2020-07-11 21:37:52 +04:00
Denis Glazachev
edb6ef8c09 Merge commit 'ceac649c01b0158090cd271776f3219f5e7ff57c' into ldap-per-user-authentication
* commit 'ceac649c01b0158090cd271776f3219f5e7ff57c': (75 commits)
  [docs] split misc statements (#12403)
  Update 00405_pretty_formats.reference
  Update PrettyCompactBlockOutputFormat.cpp
  Update PrettyBlockOutputFormat.cpp
  Update DataTypeNullable.cpp
  Update 01383_remote_ambiguous_column_shard.sql
  add output_format_pretty_grid_charset setting in docs
  add setting output_format_pretty_grid_charset
  Added a test for #11135
  Update index.md
  RIGHT and FULL JOIN for MergeJoin (#12118)
  Update MergeTreeIndexFullText.cpp
  restart the tests
  [docs] add syntax highlight (#12398)
  query fuzzer
  Fix std::bad_typeid when JSON functions called with argument of wrong type.
  Allow typeid_cast() to cast nullptr to nullptr.
  fix another context-related segfault
  [security docs] actually, only admins can create advisories
  query fuzzer
  ...
2020-07-11 21:32:36 +04:00
Denis Glazachev
2a3a0d47b3 Style fix 2020-07-11 21:31:00 +04:00
Denis Glazachev
3e68368b59 Refactor ExternalAuthenticators configuration process 2020-07-11 21:06:01 +04:00
alexey-milovidov
3c2af9653a
Merge pull request #12416 from vzakaznikov/fixing_flopping_live_view_tests
Increasing default timeout for live view tests from 20 sec to 120 sec
2020-07-11 19:38:39 +03:00
Vitaly Baranov
fe6122a1dd
Merge pull request #12394 from vitlibar/fix-calculating-implicit-access-rights
Fix calculating implicit access rights
2020-07-11 19:31:56 +03:00
Pavel Kovalenko
d3fa78392e Retrigger CI 2020-07-11 18:35:51 +03:00
BohuTANG
d6b3509664 Fix up test_mysql_protocol failed 2020-07-11 21:30:20 +08:00
Azat Khuzhin
32a45d0dee Implement lifetime_rows/lifetime_bytes for Buffer engine
Buffer engine is usually used on INSERTs, but right now there is no way
to track number of INSERTed rows per-table, since only summary metrics
exists:
- StorageBufferRows
- StorageBufferBytes

But it can be pretty useful to track INSERTed rows rate (and it can be
exposed via http_handlers for i.e. prometheus)
2020-07-11 16:06:11 +03:00
Pavel Kovalenko
8b5f8c592f Add comment regarding proxy tunnel usage in PocoHTTPClient.cpp 2020-07-11 15:43:30 +03:00
Azat Khuzhin
433fdffc19 Add lifetime_rows/lifetime_bytes interface (exported via system.tables) 2020-07-11 15:33:11 +03:00
Azat Khuzhin
84c93a6b02 Tiny IStorage refactoring 2020-07-11 15:17:06 +03:00
Ivan Blinkov
ceac649c01
[docs] split misc statements (#12403) 2020-07-11 14:05:49 +03:00
Alexander Kuzmenkov
10b0287fa4
Merge pull request #12341 from ClickHouse/aku/perfect-visitor
More perfect forwarding in field visitors
2020-07-11 13:36:37 +03:00
Alexander Kuzmenkov
48862a1edf fixup 2020-07-11 13:35:44 +03:00
Alexander Kuzmenkov
920c62b629 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-11 13:32:31 +03:00
Pavel Kovalenko
5a42dca252 Trigger integration-test-runner image rebuild. 2020-07-11 12:16:10 +03:00
emironyuk
9ad56867cd Merge branch 'master' of https://github.com/ClickHouse/ClickHouse 2020-07-11 07:32:41 +00:00
alexey-milovidov
c615ea658b
Merge pull request #12400 from vitlibar/fix-bad_typeid
Fix std::bad_typeid when JSON functions called with argument of wrong type
2020-07-11 05:40:44 +03:00
alexey-milovidov
60bc8c8a8e
Delete log.txt 2020-07-11 05:33:41 +03:00
alexey-milovidov
7bc2c83b80
Merge pull request #12372 from s-mx/issue-10429-add_setting_for_ascii_grid_symbols
add setting for ascii grid symbols
2020-07-11 05:31:41 +03:00
Vitaliy Zakaznikov
1ed7d176fc Increasing default timeout for live view tests from 20 sec to 120 sec
to account for slowdowns.
2020-07-11 04:30:53 +02:00