kssenii
3b4843a1a9
Apply changed from pr 27135
2021-09-16 18:44:42 +00:00
kssenii
55619dcc1b
Correct merge
2021-09-16 16:14:17 +00:00
kssenii
c88dc460d4
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local
2021-09-16 16:13:04 +00:00
Nikolai Kochetov
b997214620
Rename QueryPipeline to QueryPipelineBuilder.
2021-09-14 20:48:18 +03:00
kssenii
4605299262
Correct merge
2021-09-13 06:45:27 +00:00
Sergei Semin
f74350c148
fix special build
2021-09-12 02:49:31 +03:00
kssenii
33ff6ec060
Fix tests
2021-09-11 17:17:48 +00:00
kssenii
7f1770cc3c
Fix contexts
2021-09-11 12:03:28 +00:00
kssenii
ca81c13775
Move readArguments to base
2021-09-11 11:49:42 +00:00
kssenii
ddfe1e4d64
Some fixes
2021-09-11 11:34:22 +00:00
kssenii
fa04b13175
Break some tests
2021-09-09 13:25:25 +00:00
kssenii
8f77855981
Some review fixes
2021-09-05 00:52:35 +03:00
kssenii
1c20b223cb
Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local
2021-09-04 20:01:31 +03:00
Alexey Milovidov
19445ac096
Drop unused headers
2021-09-01 02:47:52 +03:00
Dmitrii Kovalkov
9871ad70ff
Exclude fuzzers
2021-08-30 11:12:25 +03:00
kssenii
578a750b8b
Small clean up
2021-08-27 11:47:47 +03:00
kssenii
7d6f3d96c1
Better
2021-08-23 13:01:42 +03:00
kssenii
95645e8f26
Fix progress
2021-08-23 10:13:27 +03:00
kssenii
deeb942ccb
Fix
2021-08-21 21:42:59 +03:00
kssenii
1f0bb0d81b
Reorganize a bit more, fix tests
2021-08-21 18:17:02 +03:00
kssenii
3a39e3f039
Reorganize options
2021-08-21 00:39:28 +03:00
kssenii
21f9622cad
Reorganize multiquery processing
2021-08-20 23:39:13 +03:00
kssenii
41d735fd56
Fix context
2021-08-20 15:16:13 +03:00
kssenii
39e80a47a3
Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local
2021-08-20 10:13:58 +03:00
kssenii
ceca7a7e2c
Fix some tests
2021-08-19 23:27:40 +03:00
kssenii
b5f6a7cb97
Fix local
2021-08-19 14:07:47 +03:00
kssenii
d1484a6f2e
Use more client code than local, hope tests survive
2021-08-19 00:01:17 +03:00
kssenii
e187bb781c
Move everything to base
2021-08-18 17:39:04 +03:00
kssenii
fb0b445ee0
Lets add LocalConnection (a start)
2021-08-17 22:59:51 +03:00
Amos Bird
27ff081403
Fix some leftover TODOs
2021-08-17 17:22:12 +08:00
kssenii
cefd9b473e
Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local
2021-08-16 16:41:24 +03:00
kssenii
30cc705464
Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local
2021-08-07 22:53:50 +03:00
kssenii
9f9be65ac7
Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local
2021-08-05 10:42:32 +03:00
kssenii
b5b9624cca
Fixes, add test
2021-08-03 23:49:16 +03:00
kssenii
f6e45b3193
Fix tests, style. Better cancel query
2021-08-02 17:46:21 +03:00
Nikolai Kochetov
6951e8147d
Merge pull request #27020 from amosbird/shardlevelconstness
...
Shard-level const column
2021-08-02 13:45:20 +03:00
kssenii
6d4440836b
Process SIGINT in interactive mode
2021-08-02 01:22:07 +03:00
kssenii
3982031215
Completion for clickhouse-local
2021-08-01 23:28:39 +03:00
Amos Bird
b76a854f5a
backward compatible
2021-08-01 09:06:16 +08:00
kssenii
6e8eb598bf
Some review fixes
2021-07-31 15:34:29 +03:00
kssenii
c9e607f1e8
Progress for interactive clickhouse local
2021-07-31 15:08:30 +03:00
kssenii
48f4f2a0c2
Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local
2021-07-29 18:41:54 +03:00
kssenii
3f259e195b
Better
2021-07-29 15:58:50 +03:00
Nikolai Kochetov
cfa3fd4b87
Hold context in HedgedConnections to prevent use-after-free on settings.
2021-07-29 13:29:36 +03:00
kssenii
22e3bde548
Refactoring, fix test, fix local interactive
2021-07-29 12:45:58 +03:00
kssenii
62d1e18b7f
Refactoring
2021-07-28 18:54:32 +03:00
kssenii
fb18f7fc72
Fix tests
2021-07-27 17:15:31 +03:00
Nikolai Kochetov
a9effdad18
Merge branch 'master' into output-streams-to-processors
2021-07-27 11:31:53 +03:00
kssenii
d30f54b335
Refactoring, fix tests
2021-07-27 01:28:27 +03:00
Nikolai Kochetov
7da5bb3871
Do not start new hedged connection if query was already canceled.
2021-07-26 20:48:38 +03:00
Nikolai Kochetov
61d8f880cd
Rename some files.
2021-07-26 19:48:25 +03:00
kssenii
4886d931a1
Fix some tests
2021-07-26 11:24:04 +03:00
alexey-milovidov
7b76bfc719
Merge pull request #26318 from amosbird/connectionpoolfactory
...
Connection pool cache
2021-07-26 06:45:03 +03:00
Amos Bird
2ba6ef1487
Better implementation
2021-07-24 18:02:51 +08:00
kssenii
4dbb2e5bd8
Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local
2021-07-24 00:36:25 +03:00
kssenii
accd6d5c0f
Refactoring
2021-07-24 00:32:38 +03:00
kssenii
8c05e4f038
Refactoring
2021-07-23 23:54:49 +03:00
vdimir
dccc379d39
Fix use after free in AsyncDrain connection from S3Cluster
2021-07-23 10:40:03 +03:00
kssenii
59b0ce21e4
Tiny refactoring
2021-07-23 00:27:26 +03:00
kssenii
b337f481dc
Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local
2021-07-22 22:24:19 +03:00
Nikolai Kochetov
20b2837b27
Fix destructor for PocoSocketWrapper.
2021-07-20 16:50:31 +03:00
Amos Bird
dbfb699690
Asynchronously drain connections.
2021-07-19 21:53:29 +08:00
kssenii
022a0e00f2
Refactor client, merging non-interactive
2021-07-18 23:18:11 +03:00
kssenii
bb0f0c18b2
Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local
2021-07-18 18:12:53 +03:00
Nikolai Kochetov
0d549c197f
Fix 01822_async_read_from_socket_crash for hedged connection.
2021-07-16 13:42:22 +03:00
kssenii
ae4644d05a
Fix some tests
2021-07-15 10:02:14 +03:00
Amos Bird
3f05211b09
Connection pool factory.
2021-07-15 00:01:45 +08:00
kssenii
7a57f2a983
Turn on fuzzing, fix tests
2021-07-12 20:26:21 +03:00
kssenii
bc3c038ac7
Better
2021-07-12 17:01:46 +03:00
kssenii
9ff54216ea
More merges in query execution code
2021-07-12 15:25:17 +03:00
kssenii
e31eccfaf2
Support multiquery option
2021-07-12 12:34:37 +03:00
kssenii
a10b9ed63a
Start merging query processing code
2021-07-12 02:17:14 +03:00
kssenii
d58a5537c2
Fix details
2021-07-12 00:54:41 +03:00
kssenii
026fa0ca64
Merge main, mainImpl code, allow interactive mode for clickhouse-local
2021-07-11 23:35:29 +03:00
kssenii
0b5dab1326
Merge init code for local and client
2021-07-11 14:36:27 +03:00
alexey-milovidov
aca575cc85
Merge branch 'master' into refactor-pipeline-executor
2021-07-04 02:08:31 +03:00
Alexey Milovidov
e3653e70da
Update prompt in client when reconnecting
2021-06-15 04:43:35 +03:00
Alexey Milovidov
273226de32
Remove string parameter for Density
2021-05-24 06:43:25 +03:00
Alexey Milovidov
40d4f0678f
Remove overload (harmful)
2021-05-23 04:25:06 +03:00
fibersel
cb53bbb7b0
add experimental codecs flag, add integration test for experimental codecs
2021-05-06 14:57:22 +03:00
alexey-milovidov
7ca42d2ce1
Merge pull request #23805 from ClickHouse/fix-hedged-requests-ext-tables
...
Fix sending external table for hedged connections.
2021-05-01 08:13:13 +03:00
Nikolai Kochetov
352ec049ad
Fix sending external table for hedged connections.
2021-04-30 17:19:48 +03:00
Maksim Kita
318c4bb80d
Add examples folder filter to ya.make.in
2021-04-30 11:25:52 +03:00
Maksim Kita
0d0a14a925
Poco::Timespan reference fix
2021-04-29 19:11:20 +03:00
Nikita Mikhaylov
9f55424250
move to examples everywhere
2021-04-27 01:51:42 +03:00
Nikolai Kochetov
548fb52d24
Fix build.
2021-04-20 08:51:51 +03:00
Nikolai Kochetov
08d1896950
Fix build.
2021-04-19 22:06:05 +03:00
Nikolai Kochetov
0420b7bb33
Reset replica state in MultiplexedConnections in case of exception.
2021-04-19 17:25:43 +03:00
Alexey Milovidov
17d04cbca9
Merge branch 'master' into protocol-compression-auto
2021-04-15 00:24:07 +03:00
Alexey Milovidov
6f56c3280f
Uncompress data in Distributed sends if needed
2021-04-14 00:53:39 +03:00
Nikita Mikhaylov
024374a2ec
review fixes
2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
7a68820342
style
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
c333c3dedb
review fixes
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
36a8419f60
style
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
2555ae5d3f
better processing stage
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4843f86329
use only one connection between initiator and worker
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
e601a432e5
better[2]
2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
cef9e19eb2
better
2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
2549468c14
better
2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
64b4cd0e63
save
2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
31b4f9b17f
save
2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
44ca65a9a4
save
2021-04-13 22:39:41 +03:00
Alexey Milovidov
fafe8c00fe
Fix warning #22987
2021-04-12 20:57:03 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr ( #22297 )
...
* Replace all Context references with std::weak_ptr
* Fix shared context captured by value
* Fix build
* Fix Context with named sessions
* Fix copy context
* Fix gcc build
* Merge with master and fix build
* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Pavel Kruglov
6f65a72112
Remove socket from epoll before cancelling packet receiver to prevent race
2021-04-01 17:35:00 +03:00
alexey-milovidov
25f02f13b0
Merge pull request #22238 from ClickHouse/remove-harmful-code
...
Remove harmful default parameters
2021-03-29 23:46:18 +03:00
Alexey Milovidov
169369e90f
Remove harmful default parameters
2021-03-29 04:08:28 +03:00
Alexey Milovidov
b336de9ebd
Remove harmful default arguments
2021-03-29 04:07:42 +03:00
Pavel Kruglov
90afc9ce2f
Fix build
2021-03-26 21:25:30 +03:00
Pavel Kruglov
2979feb23f
Prevent busy waiting in hedged requests when async_socket_for_remote=0
2021-03-26 18:11:06 +03:00
Kruglov Pavel
66b8431e71
Merge pull request #21429 from Avogar/secure-socket
...
Fix receive and send timeouts and non-blocking read in secure socket
2021-03-25 17:30:17 +03:00
Azat Khuzhin
17eaddc3ab
Prevent hedged connections overlaps
...
Some replicas may be in the establishing state, this means that hedged
connection is waiting for TablesStatusResponse, and if the connection
will not be canceled, then next user of the connection will get
TablesStatusResponse, while this is not the expected package, and you
will see:
DB::Exception: Received from localhost:9000. DB::Exception: Received from ch-1:9000. DB::Exception: Unknown packet 9 from server ch-2:9000: While executing Remote. Stack trace:
Fix this by disconnecting connection to such replicas, note that this
should be fine in general, since they are "slow" anyway (you may wish
configure hedged requests settings to increase timeouts and similar).
2021-03-20 19:05:29 +03:00
Kruglov Pavel
055242fe95
Style
2021-03-19 01:38:15 +03:00
Pavel Kruglov
d380454044
Change read_data_timeout from sec to ms, add profile event for hedged requests and better process receiving packets
2021-03-18 20:16:17 +03:00
Pavel Kruglov
dbae58f30b
Add slowdown_count and show actual information in system.clusters
2021-03-06 02:45:17 +03:00
Pavel Kruglov
c113af39f4
Merge branch 'master' of github.com:ClickHouse/ClickHouse into secure-socket
2021-03-05 16:18:45 +03:00
Pavel Kruglov
1c7f16e0ff
Add test and move TimeoutSetter in IO/
2021-03-03 20:47:27 +03:00
Pavel Kruglov
dc9dd8ccdc
Fix typo
2021-03-03 16:16:17 +03:00
Pavel Kruglov
82a6be44cf
Disable changing replica when we have made progress
2021-03-03 01:03:03 +03:00
Pavel Kruglov
4166ae1f45
Restart timeout when we make progress
2021-03-02 15:40:24 +03:00
Pavel Kruglov
1b1f50e324
Update comment
2021-02-27 11:45:32 +03:00
Pavel Kruglov
4d7ba0443f
Add checking pending data
2021-02-27 11:40:03 +03:00
Kruglov Pavel
cb638e0f87
Fix style
2021-02-26 21:06:17 +03:00
Pavel Kruglov
91f6f9227b
Small fixes and tests updates
2021-02-26 18:55:58 +03:00
Pavel Kruglov
24e9abddb8
Fix
2021-02-24 15:11:07 +03:00
Pavel Kruglov
5628fd6da5
Small update
2021-02-24 02:22:07 +03:00
Pavel Kruglov
edee592cb0
Fix
2021-02-22 20:57:44 +03:00
Pavel Kruglov
1dbab60235
Fix
2021-02-22 10:52:19 +03:00
Pavel Kruglov
6d8eb44c01
Fix PSV and remove debug logging
2021-02-22 00:59:07 +03:00
Pavel Kruglov
4320782032
Style
2021-02-21 20:43:36 +03:00
Pavel Kruglov
db591309a3
fix style
2021-02-21 17:51:36 +03:00
Pavel Kruglov
93162716f4
remove logging
2021-02-21 17:50:47 +03:00
Pavel Kruglov
dd5185c779
Refactor 4.0
2021-02-21 17:03:24 +03:00
Pavel Kruglov
9bd9ea9fbc
Try to fix sigsev
2021-02-20 00:45:58 +03:00
Pavel Kruglov
2f5b4c20ae
Fix
2021-02-18 14:21:48 +03:00
Pavel Kruglov
80b6db7f72
Style
2021-02-17 22:07:14 +03:00
Pavel Kruglov
bc6fe4f28c
Minor fixes
2021-02-17 20:44:11 +03:00
Pavel Kruglov
1b78de2142
Use fibers in HedgedRequests
2021-02-17 20:34:52 +03:00
Pavel Kruglov
c57daf2140
Merge branch 'hedged-requests' of github.com:Avogar/ClickHouse into hedged-requests
2021-02-17 20:34:11 +03:00
Pavel Kruglov
0e1d67ad9a
Add LOG_DEBUG to debug test_distributed_load_balancing
2021-02-16 10:56:45 +03:00
Kruglov Pavel
6dcb306060
Style
2021-02-16 00:46:51 +03:00
Kruglov Pavel
28dec516ac
Style
2021-02-15 19:12:03 +03:00
Pavel Kruglov
eb0847e2b6
Style
2021-02-15 17:44:05 +03:00
Pavel Kruglov
df09a5cac9
Refactor 3.0 and fix tests
2021-02-15 16:21:36 +03:00
Pavel Kruglov
4cc7e2c5c6
Update comment
2021-02-09 05:13:47 +03:00
Pavel Kruglov
22b8bc6c99
Refactor 2.0
2021-02-09 05:01:09 +03:00
Pavel Kruglov
df5dc102c4
Don't use 'ReplicaStatePtr &' to prevent use-after-free
2021-02-08 23:58:02 +03:00
Pavel Kruglov
a594c738c2
Remove code duplication
2021-02-08 17:14:36 +03:00
Pavel Kruglov
7b45860b06
Style fix
2021-02-08 17:02:11 +03:00
Pavel Kruglov
134a686fa5
Fix 2
2021-02-08 16:16:09 +03:00
Pavel Kruglov
d3ebf59376
Fix
2021-02-08 16:08:15 +03:00
Pavel Kruglov
12b925dd7a
Small update
2021-02-08 14:06:45 +03:00
Kruglov Pavel
636ef5394b
Remove redundant field
2021-02-07 01:45:11 +03:00
Pavel Kruglov
3d8e05dc94
Fix style
2021-02-07 00:54:29 +03:00
Pavel Kruglov
8ff3dde290
Add sendIgnoredPartUUIDs to HedgedRequests
2021-02-06 18:26:36 +03:00
Pavel Kruglov
f946aab759
Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests
2021-02-06 17:38:56 +03:00
Pavel Kruglov
794f185442
Fix
2021-02-06 17:23:48 +03:00
Pavel Kruglov
0704d3cf27
Refactor
2021-02-06 03:54:27 +03:00
Pavel Kruglov
3fc8b294e8
Reset changes in tryGetEntry
2021-02-03 15:56:42 +03:00
Pavel Kruglov
b8ae9caa61
Fix style
2021-02-03 14:27:26 +03:00
Aleksei Semiglazov
921518db0a
CLICKHOUSE-606: query deduplication based on parts' UUID
...
* add the query data deduplication excluding duplicated parts in MergeTree family engines.
query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1
allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.
data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.
NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.
* add _part_uuid virtual column, allowing to use UUIDs in predicates.
Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>
address comments
2021-02-02 16:53:39 +00:00
Pavel Kruglov
2c928f11e1
Remove code duplication
2021-02-02 19:39:30 +03:00
Pavel Kruglov
02cc43502e
Remove LOG_DEBUG
2021-02-02 15:17:06 +03:00
Pavel Kruglov
60a92e9a99
Fix build, add comments, update tests
2021-02-02 15:14:31 +03:00
Kruglov Pavel
3e3ee19818
Restart tests
2021-02-02 00:10:55 +03:00
Pavel Kruglov
f5ad1281f7
Fix style
2021-02-01 20:14:53 +03:00
Pavel Kruglov
7d9eb966f0
Fix
2021-02-01 20:09:55 +03:00
Alexey Milovidov
01a703ae50
Update test
2021-01-31 10:48:18 +03:00
Pavel Kruglov
d27f5114c5
Add LOG_DEBUG for tests debug
2021-01-29 23:21:11 +03:00
Pavel Kruglov
25e85d71ee
Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests
2021-01-29 21:08:47 +03:00
Pavel Kruglov
6029597877
Fix style error
2021-01-29 19:56:47 +03:00
Pavel Kruglov
01a0cb649a
Fix build, style, tests
2021-01-29 18:46:28 +03:00
Alexey Milovidov
bfcb12c2e9
Add test-connect tool
2021-01-29 09:13:43 +03:00
Alexey Milovidov
9477f8a8b1
Revert "Remove old non-automated test"
...
This reverts commit 217d05443a
.
2021-01-29 08:27:58 +03:00
Pavel Kruglov
b3b832cde7
Work with any number of replicas simultaneously, support max_parallel_replicas
2021-01-27 12:33:11 +03:00
Azat Khuzhin
e97c01c3ea
Fix UAF of the CompressedWriteBuffer after Connection::disconnect ( #19599 )
...
ASan report [1]:
<details>
Stacktrace with stripped shared_ptr and vector stuff:
```
==86==ERROR: AddressSanitizer: heap-use-after-free on address 0x60d0002b4888 at pc 0x00000a997056 bp 0x7f9e2ad55c00 sp 0x7f9e2ad55bf8
READ of size 8 at 0x60d0002b4888 thread T3 (TCPHandler)
0 0xa997055 in DB::BufferBase::Buffer::end() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:40:46
1 0xa997055 in DB::BufferBase::available() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:81:68
2 0xa997055 in DB::BufferBase::hasPendingData() const obj-x86_64-linux-gnu/../src/IO/BufferBase.h:94:56
3 0xa997055 in DB::WriteBuffer::nextIfAtEnd() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:67:14
4 0xa997055 in DB::WriteBuffer::write(char const*, unsigned long) obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:78:13
5 0x1dcff45e in DB::CompressedWriteBuffer::nextImpl() obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:37:9
6 0x1dcffb8a in DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:46:13
7 0x1dcffb8a in DB::CompressedWriteBuffer::~CompressedWriteBuffer() obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:54:9
11 0xab600cf in DB::Connection::~Connection() obj-x86_64-linux-gnu/../src/Client/Connection.h:114:28
15 0xac4adb9 in PoolBase<DB::Connection>::PooledObject::~PooledObject() obj-x86_64-linux-gnu/../src/Common/PoolBase.h:35:12
30 0xac485e4 in PoolBase<DB::Connection>::~PoolBase() obj-x86_64-linux-gnu/../src/Common/PoolBase.h:105:26
44 0xad2722f in DB::Cluster::ShardInfo::~ShardInfo() obj-x86_64-linux-gnu/../src/Interpreters/Cluster.h:167:12
52 0xad393b0 in DB::Cluster::~Cluster() obj-x86_64-linux-gnu/../src/Interpreters/Cluster.h:30:7
56 0x1f99f269 in DB::StorageDistributed::~StorageDistributed() obj-x86_64-linux-gnu/../src/Storages/StorageDistributed.cpp:338:41
69 0x1e231846 in DB::Context::~Context() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:501:19
71 0x2073ccd3 in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:416:23
72 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9
0x60d0002b4888 is located 56 bytes inside of 136-byte region [0x60d0002b4850,0x60d0002b48d8)
freed by thread T3 (TCPHandler) here:
0 0xa93d682 in operator delete(void*, unsigned long) (/workspace/clickhouse+0xa93d682)
1 0x2059d592 in std::__1::__shared_weak_count::__release_shared() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2518:9
2 0x2059d592 in std::__1::shared_ptr<DB::WriteBuffer>::~shared_ptr() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3212:19
3 0x2059d592 in std::__1::shared_ptr<DB::WriteBuffer>::operator=(std::__1::shared_ptr<DB::WriteBuffer>&&) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3243:5
4 0x2059d592 in DB::Connection::disconnect() obj-x86_64-linux-gnu/../src/Client/Connection.cpp:143:9
5 0x205d6e6d in DB::MultiplexedConnections::disconnect() obj-x86_64-linux-gnu/../src/Client/MultiplexedConnections.cpp:159:25
6 0x1de6ec19 in DB::RemoteQueryExecutor::~RemoteQueryExecutor() obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutor.cpp:86:34
10 0x20bf0e2c in DB::RemoteSource::~RemoteSource() obj-x86_64-linux-gnu/../src/Processors/Sources/RemoteSource.cpp:22:29
21 0x20869680 in DB::Pipe::~Pipe() obj-x86_64-linux-gnu/../src/Processors/Pipe.h:25:7
22 0x20869680 in DB::QueryPipeline::reset() obj-x86_64-linux-gnu/../src/Processors/QueryPipeline.cpp:79:1
23 0x1de2d89d in DB::BlockIO::reset() obj-x86_64-linux-gnu/../src/DataStreams/BlockIO.cpp:45:14
24 0x1de2d9f7 in DB::BlockIO::operator=(DB::BlockIO&&) obj-x86_64-linux-gnu/../src/DataStreams/BlockIO.cpp:57:5
25 0x20762731 in DB::QueryState::operator=(DB::QueryState&&) obj-x86_64-linux-gnu/../src/Server/TCPHandler.h:31:8
26 0x2073c70c in DB::QueryState::reset() obj-x86_64-linux-gnu/../src/Server/TCPHandler.h:85:15
27 0x2073c70c in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:399:19
28 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9
29 0x266eeebe in Poco::Net::TCPServerConnection::start() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:43:3
30 0x266ef9db in Poco::Net::TCPServerDispatcher::run() obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:112:19
31 0x269b1204 in Poco::PooledThread::run() obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/ThreadPool.cpp:199:14
32 0x269ab756 in Poco::ThreadImpl::runnableEntry(void*) obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread_POSIX.cpp:345:27
33 0x7f9f06ea8608 in start_thread /build/glibc-ZN95T4/glibc-2.31/nptl/pthread_create.c:477:8
previously allocated by thread T3 (TCPHandler) here:
0 0xa93ca1d in operator new(unsigned long) (/workspace/clickhouse+0xa93ca1d)
1 0x2059b8cd in void* std::__1::__libcpp_operator_new<unsigned long>(unsigned long) obj-x86_64-linux-gnu/../contrib/libcxx/include/new:235:10
8 0x2059b8cd in DB::Connection::connect(DB::ConnectionTimeouts const&) obj-x86_64-linux-gnu/../src/Client/Connection.cpp:112:15
9 0x205a0a1d in DB::Connection::getServerRevision(DB::ConnectionTimeouts const&) obj-x86_64-linux-gnu/../src/Client/Connection.cpp:289:9
10 0x205bdafa in DB::ConnectionPoolWithFailover::tryGetEntry(DB::IConnectionPool&, DB::ConnectionTimeouts const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, DB::Settings const*, DB::QualifiedTableName const*) obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:251:45
11 0x205c06cf in DB::ConnectionPoolWithFailover::getManyChecked()::$_8::operator()() const obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:169:16
20 0x205bd61f in DB::ConnectionPoolWithFailover::getManyChecked(DB::ConnectionTimeouts const&, DB::Settings const*, DB::PoolMode, DB::QualifiedTableName const&) obj-x86_64-linux-gnu/../src/Client/ConnectionPoolWithFailover.cpp:172:12
28 0x1de6f425 in DB::RemoteQueryExecutor::sendQuery() obj-x86_64-linux-gnu/../src/DataStreams/RemoteQueryExecutor.cpp:143:31
29 0x1fdd1410 in DB::getStructureOfRemoteTableInShard(DB::Cluster const&, DB::Cluster::ShardInfo const&, DB::StorageID const&, DB::Context const&, std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Storages/getStructureOfRemoteTable.cpp:78:12
30 0x1fdd74a8 in DB::getStructureOfRemoteTable(DB::Cluster const&, DB::StorageID const&, DB::Context const&, std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Storages/getStructureOfRemoteTable.cpp:131:32
31 0x1d42cd79 in DB::TableFunctionRemote::getActualTableStructure(DB::Context const&) const obj-x86_64-linux-gnu/../src/TableFunctions/TableFunctionRemote.cpp:261:12
32 0x1d42b4e2 in DB::TableFunctionRemote::executeImpl(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ColumnsDescription) const obj-x86_64-linux-gnu/../src/TableFunctions/TableFunctionRemote.cpp:222:26
33 0x1e2c8b15 in DB::ITableFunction::execute(std::__1::shared_ptr<DB::IAST> const&, DB::Context const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::ColumnsDescription) const obj-x86_64-linux-gnu/../src/TableFunctions/ITableFunction.cpp:24:16
34 0x1e2417ff in DB::Context::executeTableFunction(std::__1::shared_ptr<DB::IAST> const&) obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:1007:35
35 0x1f15bd67 in DB::JoinedTables::getLeftTableStorage() obj-x86_64-linux-gnu/../src/Interpreters/JoinedTables.cpp:162:42
36 0x1ebf393f in DB::InterpreterSelectQuery::InterpreterSelectQuery() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:306:33
42 0x1eb3bf90 in DB::InterpreterFactory::get(std::__1::shared_ptr<DB::IAST>&, DB::Context&, DB::SelectQueryOptions const&) obj-x86_64-linux-gnu/../src/Interpreters/InterpreterFactory.cpp:110:16
43 0x1f4d9dee in DB::executeQueryImpl(char const*, char const*, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:520:28
44 0x1f4d7067 in DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Context&, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:900:30
45 0x2073b0bc in DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:260:24
46 0x2075db1c in DB::TCPHandler::run() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1417:9
```
</details>
[1]: https://clickhouse-test-reports.s3.yandex.net/19583/9f8ab99dd12a6f60a20f5f84ab2f5d53874c6ae7/fuzzer_asan/report.htmlfail1
2021-01-26 14:27:58 +03:00
Nikolai Kochetov
a3512f50d4
Merge branch 'master' into refactor-pipeline-executor
2021-01-25 23:16:06 +03:00
Azat Khuzhin
109dbe5df4
Check the stream before sending while hanlding async INSERTs into Distributed
...
It is possible to get corruption (even though it is very unlikely, and
initially it wasn't corruption) just before the data block goes in the
file on disk, and in case of batching, it will break the packets, since
it will write the packet type but will not write any data after.
2021-01-22 21:29:58 +03:00
Nikolai Kochetov
f0d2dd943b
Fix build.
2021-01-21 18:35:14 +03:00
Pavel Kruglov
2aa29e1bc7
Fix build 2
2021-01-20 13:29:31 +03:00
Pavel Kruglov
507695cbcb
Fix build
2021-01-20 02:15:13 +03:00
Azat Khuzhin
e4350e078c
Add ability to distinguish remote exceptions from local
2021-01-20 01:10:17 +03:00
Pavel Kruglov
97b5179e55
Implement HedgedRequests
2021-01-19 22:41:05 +03:00
Azat Khuzhin
cf085b0687
Split RemoteQueryExecutorReadContext into module part
2021-01-16 01:57:36 +03:00
Nikolai Kochetov
af7f5c9518
Merge pull request #17868 from ClickHouse/async-read-from-socket
...
Async read from socket
2020-12-23 12:20:42 +03:00
Nikolai Kochetov
01c8d5dccc
Use replace fiber to callback in ReadBufferFromPocoSocket
2020-12-18 17:24:44 +03:00
nikitamikhaylov
18d52dbc63
better
2020-12-16 14:55:33 +03:00
nikitamikhaylov
4ccdb3ca20
done
2020-12-16 14:55:33 +03:00
Nikolai Kochetov
4c290996dc
Fixing tests.
2020-12-15 21:22:20 +03:00
Nikolai Kochetov
3d6ace5890
Fixing tests.
2020-12-15 21:20:14 +03:00
Nikolai Kochetov
00de8abb0a
Fixing tests.
2020-12-15 00:07:10 +03:00
Nikolai Kochetov
8de5cd5bc7
Merge branch 'master' into async-read-from-socket
2020-12-14 17:45:38 +03:00
Nikolai Kochetov
75ac87c241
Fixing build.
2020-12-14 17:42:12 +03:00
Nikolai Kochetov
e295dfe6e3
Use ucontext for asan
2020-12-14 17:42:08 +03:00
Azat Khuzhin
5b3ab48861
More forward declaration for generic headers
...
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)
And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)
Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).
Approx results (How much units will be recompiled after changing file X?):
- ConnectionTimeouts.h
- mainline: 100
- Context.h:
- mainline: ~800
- patched: 415
- Settings.h:
- mainline: 900-1K
- patched: 440 (most of them because of the Context.h)
2020-12-12 17:43:10 +03:00
Nikolai Kochetov
156f44808f
Fixing crash.
2020-12-09 17:11:45 +03:00
Nikolai Kochetov
082a496364
Add async read to RemoteQueryExecutor
2020-12-03 15:21:10 +03:00
Nikolai Kochetov
e3946bc2b5
Add async read to RemoteQueryExecutor.
2020-12-02 20:02:14 +03:00
Pavel Kruglov
5b94dd2f74
Add eof check in receiveHello
2020-11-24 18:15:13 +03:00
Alexey Milovidov
24f4fa6edf
Follow Arcadia ya.make rules
2020-11-17 00:16:50 +03:00
Alexey Milovidov
3df04ce0c2
Follow Arcadia ya.make rules
2020-11-16 21:24:58 +03:00
filimonov
7f7f66a1f1
add comment & restart CI...
2020-11-13 17:45:58 +01:00
filimonov
be16b4ef77
Update Connection.cpp
2020-11-13 12:04:56 +01:00
filimonov
178d8e9b75
Update Connection.cpp
2020-11-12 22:07:19 +01:00
filimonov
8eff47420b
Update Connection.cpp
2020-11-12 18:18:34 +01:00
Alexander Tokmakov
a06be511df
pcg serialization
2020-11-09 16:07:38 +03:00
Alexey Milovidov
fd84d16387
Fix "server failed to start" error
2020-11-07 03:14:53 +03:00
Maxim Akhmedov
3627fabfb9
Remove -g0 form Arcadia build settings.
2020-10-29 17:37:23 +03:00
Amos Bird
d2dcfc3f0d
Refactor processors.
2020-10-12 17:30:05 +08:00
Alexander Kuzmenkov
dde4cf70e1
Merge remote-tracking branch 'origin/master' into HEAD
2020-09-22 14:03:59 +03:00
alexey-milovidov
9b6c62e82b
Merge pull request #14867 from amosbird/lbo
...
Explicit define what first replica is.
2020-09-17 19:37:15 +03:00
Alexander Kuzmenkov
fb64cf210a
straighten the protocol version
2020-09-17 17:37:29 +03:00
Amos Bird
38d53c38f6
Explicit define what first replica is.
2020-09-16 17:54:41 +08:00
Azat Khuzhin
785d1b2a75
OpenSSLHelpers cleanup
...
Add few more specializations for encodeSHA256():
- std::string encodeSHA256(const std::string_view &);
- std::string encodeSHA256(const void *, size_t);
- void encodeSHA256(const void *, size_t, unsigned char *);
2020-09-15 01:36:28 +03:00
Azat Khuzhin
0159c74f21
Secure inter-cluster query execution (with initial_user as current query user) [v3]
...
Add inter-server cluster secret, it is used for Distributed queries
inside cluster, you can configure in the configuration file:
<remote_servers>
<logs>
<shard>
<secret>foobar</secret> <!-- empty -- works as before -->
...
</shard>
</logs>
</remote_servers>
And this will allow clickhouse to make sure that the query was not
faked, and was issued from the node that knows the secret. And since
trust appeared it can use initial_user for query execution, this will
apply correct *_for_user (since with inter-server secret enabled, the
query will be executed from the same user on the shards as on initator,
unlike "default" user w/o it).
v2: Change user to the initial_user for Distributed queries if secret match
v3: Add Protocol::Cluster package
v4: Drop Protocol::Cluster and use plain Protocol::Hello + user marker
v5: Do not use user from Hello for cluster-secure (superfluous)
2020-09-15 01:36:28 +03:00
Alexey Milovidov
e3924b8057
Fix "Arcadia"
2020-09-08 01:14:13 +03:00
alesapin
10c7a6c45e
Add ability to specify Default codec for columns ( #14049 )
...
* Add ability to specify DefaultCompression codec which correspond to settings specified in config.xml
* Fix style
* Rename DefaultCompression to simple Default
* Fix compression codec
* Better codec description representation
* Less strange code and one method
* Fix delta
2020-08-28 20:40:45 +03:00
Vasily Nemkov
f94f786cc3
First attempt to fix data race in ConnectionPoolWithFailover::getStatus()
2020-08-20 23:25:38 +03:00
Nikolai Kochetov
9b67cd9faf
Merge branch 'master' into refactor-pipes-3
2020-08-10 10:50:17 +03:00
Alexey Milovidov
edd89a8610
Fix half of typos
2020-08-08 03:47:03 +03:00
Nikolai Kochetov
20e63d2271
Refactor Pipe [part 6]
2020-08-06 15:24:05 +03:00
Alexey Milovidov
6f690b7c0d
Normalize ya.make files, fix "Arcadia" build
2020-08-02 16:57:38 +03:00
Vitaly Baranov
56665a15f7
Rework and rename the template class SettingsCollection => BaseSettings.
2020-07-31 20:54:18 +03:00
Nikita Mikhaylov
4d49d2c671
another removes
2020-07-30 13:31:14 +03:00
Alexander Kuzmenkov
ac436c79eb
Merge remote-tracking branch 'origin/master' into HEAD
2020-07-07 15:42:11 +03:00
Alexander Kuzmenkov
5c417f45b8
streaming wip
2020-06-30 12:25:23 +03:00
Azat Khuzhin
ebff4eae7d
Add replica priority for load_balancing
...
Make default 1, to match with <weight>
2020-06-29 23:03:28 +03:00
Azat Khuzhin
b8ee2ea7d9
Pass max_ignored_errors/fallback_to_stale_replicas to PoolWithFailoverBase::get() too
2020-06-20 11:46:13 +03:00
Azat Khuzhin
dd054380bf
Drop default values for some arguments of PoolWithFailoverBase::getMany
2020-06-20 11:21:18 +03:00
Azat Khuzhin
caa195c034
Add number of errors to ignore while choosing replicas (distributed_replica_max_ignored_errors)
...
This will allow avoid switching to another replica in case of error
(since error can be temporary).
2020-06-20 11:20:16 +03:00
Nikita Mikhaylov
ff0262626a
Merge pull request #11645 from azat/load-balancing-round-robin
...
Add round_robin load_balancing
2020-06-17 14:34:59 +04:00
Nikita Mikhaylov
9b88065e77
add small comment
2020-06-17 13:33:35 +03:00
Alexey Milovidov
217d05443a
Remove old non-automated test
2020-06-14 20:42:11 +03:00
Azat Khuzhin
0e378590fe
Add load_balancing=round_robin
2020-06-14 01:09:21 +03:00
Alexey Milovidov
25f941020b
Remove namespace pollution
2020-05-31 00:57:37 +03:00
Alexey Milovidov
5aff138956
Preparation for structured logging
2020-05-31 00:35:52 +03:00
Alexey Milovidov
9082a70ee6
Fix clang-tidy
2020-05-24 21:25:58 +03:00
Alexey Milovidov
7e1813825b
Return old names of macros
2020-05-24 01:24:01 +03:00
Alexey Milovidov
bab24879e9
Progress on task
2020-05-24 00:16:05 +03:00
Alexey Milovidov
7c0c328a35
Progress on task
2020-05-24 00:05:47 +03:00
Alexey Milovidov
d2621fee30
Progress on task
2020-05-23 23:57:48 +03:00
Alexey Milovidov
f75587e562
Progress on task
2020-05-23 23:13:34 +03:00
Alexey Milovidov
cd6d3c7509
Progress on task
2020-05-23 22:55:06 +03:00
Alexey Milovidov
eacff92d0e
Progress on task
2020-05-23 22:35:08 +03:00
Alexey Milovidov
86b4d5a86f
Remove duplicate whitespaces (preparation)
2020-05-23 22:09:06 +03:00
Alexey Milovidov
ab0562a574
Make all LOG in single line (preparation)
2020-05-23 22:05:41 +03:00
Alexey Milovidov
45b6abdfee
Progress on task
2020-05-23 21:22:40 +03:00
Alexey Milovidov
9d24908e53
Progress on task
2020-05-23 20:52:11 +03:00
Alexey Milovidov
8d2e80a5e2
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/'
2020-05-23 19:42:39 +03:00
Alexey Milovidov
e884800662
Remove commented code
2020-05-23 19:38:59 +03:00
Azat Khuzhin
d93b9a57f6
Forward declaration for Context as much as possible.
...
Now after changing Context.h 488 modules will be recompiled instead of 582.
2020-05-21 01:53:18 +03:00
Alexey Milovidov
397859ccb8
Fix error
2020-05-17 08:45:20 +03:00
Alexey Milovidov
f4ac1eec04
Merge branch 'master' into quota-key-in-client
2020-05-17 08:15:13 +03:00
Ivan
85d783c247
Poco contrib refactoring ( #10396 )
...
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
2020-05-08 17:11:19 +03:00
Alexey Milovidov
99c18c5a09
Check for suspicious codecs #4966
2020-05-04 03:11:49 +03:00
Alexey Milovidov
1d885b3ff9
Merge branch 'master' into quota-key-in-client
2020-04-20 02:15:37 +03:00
Ivan Lezhankin
e230632645
Changes required for auto-sync with Arcadia
2020-04-16 15:31:57 +03:00
Alexey Milovidov
fe59524481
Implemented in a different way
2020-04-15 04:59:14 +03:00
Alexey Milovidov
c22f91db01
Allow to pass quota_key in clickhouse-client
2020-04-15 04:15:05 +03:00
Ivan Lezhankin
06446b4f08
dbms/ → src/
2020-04-03 18:14:31 +03:00