thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 02:12:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
97,487 Commits 9,330 Branches 1,783 Tags 2.1 GiB
3623807527
Commit Graph

668 Commits

Author SHA1 Message Date
Azat Khuzhin
1d25ec3e82 Merge ProfileEvents in case they were not printed 
That way with --profile-events-delay-ms=-1 you will always get totals.

Plus, this will fix periodic failures, that can be reproduced by
limitting CPU (5% is enough in my setup), i.e.:

    $ systemd-run --collect --unit ch -p CPUQuota=5% --user clickhouse-server
    $ while clickhouse-client --print-profile-events --profile-events-delay-ms=-1 -q 'select * from numbers (1e5) format Null' |& tee /dev/stderr | fgrep 'SelectedRows: 131010 (increment)'; do :; done

And as a bonus it will make 02050_client_profile_events deterministic.
 2021-12-19 23:43:05 +03:00
Anton Popov
99ebabd822 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-17 19:02:29 +03:00
凌涛
bf1098484e modify code style 2021-12-16 20:31:06 +08:00
凌涛
ad34217873 Add hints for clickhouse client options 2021-12-16 20:19:00 +08:00
N. Kolotov
adf5fcdd0a Fixed --echo option is not used by clickhouse-client in batch mode with single query. 2021-12-16 13:10:49 +03:00
Azat Khuzhin
510eab25a1 Add timeout into "Timeout exceeded while reading from" message 2021-12-14 10:18:57 +03:00
Azat Khuzhin
eb9968babf Add timeout into "Read timeout while draining from" message 2021-12-14 10:18:57 +03:00
Azat Khuzhin
4e4837758a Remove readline support 
- it was not nested for a long time
- replxx is an upstream way for completion
 2021-12-11 11:29:59 +03:00
Azat Khuzhin
fa6090f588 Fix processing initial table (--table/stdin) in clickhouse-local 
This patch will:
- fix the issue when table had been tried to create multiple times for
  --queries-files
- create these table for --interactive mode (before it works only if you
  had some queries already, i.e. when it run interactive after
  non-interactive)

This will also make ClientBase interface a little bit cleaner, by
removing one abstract method getQueryTextPrefix()
 2021-12-10 22:08:03 +03:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
alexey-milovidov
50c23a9fd6
Merge pull request #31123 from amosbird/repl-improvement 
Better clickhouse-client multiline input
 2021-12-06 12:22:13 +03:00
Anton Popov
6f4d9a53b2 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-12-01 15:54:33 +03:00
mergify[bot]
b4f43c59a2
Merge branch 'master' into stress-test 2021-12-01 10:12:22 +00:00
alexey-milovidov
96ec92c7cd
Merge pull request #31155 from ClickHouse/incremental-profile-events 
Send incremental profile events to client
 2021-12-01 04:04:41 +03:00
mergify[bot]
e6f5792ba3
Merge branch 'master' into stress-test 2021-11-29 09:01:33 +00:00
Raúl Marín
f0ee0724ac Reduce dependencies on ASTSelectQuery.h 
243 -> 152
 2021-11-26 18:35:24 +01:00
mergify[bot]
ce0c112586
Merge branch 'master' into stress-test 2021-11-25 11:09:26 +00:00
Anton Popov
ccd78e3838 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-22 17:19:35 +03:00
Vitaly Baranov
6634fcbac7 Rename Quota::ResourceType -> QuotaType and move it to Access/Common. 2021-11-19 00:14:23 +03:00
Kruglov Pavel
f27fcf8372
Merge pull request #31004 from Avogar/output-formats 
Small refactoring in formats
 2021-11-17 20:44:46 +03:00
Amos Bird
93294734d3
Better clickhouse-client multiline input 2021-11-17 22:11:30 +08:00
Kseniia Sumarokova
d1374481df
Merge pull request #31457 from azat/local-pager 
Add --pager support for clickhouse-local
 2021-11-17 09:29:56 +03:00
Azat Khuzhin
bbbbd261b8 Add --pager support for clickhouse-local 2021-11-16 22:25:32 +03:00
mergify[bot]
923874060a
Merge branch 'master' into stress-test 2021-11-16 16:42:50 +00:00
Nikolay Degterinsky
ba50e8387d Merge branch 'master' into query_parameters 2021-11-15 19:55:54 +00:00
Nikolay Degterinsky
5d07387987 Fix INFILE tests 2021-11-15 19:55:27 +00:00
Kseniia Sumarokova
846fbd059b
Merge pull request #31403 from kssenii/fix-client-output 
Fix client
 2021-11-14 15:01:30 +03:00
kssenii
ad1b7c83cd Fix 2021-11-14 07:19:59 +00:00
alexey-milovidov
e323b83f6a
Merge pull request #30876 from pmed/backslash_letter_interactive_input 
add aliases for `\<letter>` interactive client commands
 2021-11-14 01:43:13 +03:00
Nikolai Kochetov
6623c609cf
Merge pull request #19587 from ClickHouse/refactor-pipeline-executor 
Refactor pipeline executor
 2021-11-12 11:49:05 +03:00
Kseniia Sumarokova
2313981fd7
Merge pull request #31260 from azat/external-cleanup 
Cleanup extern ProfileEvents/CurrentMetrics and add a style check
 2021-11-12 00:02:57 +03:00
avogar
e5661b9b62 Minor change 2021-11-11 21:09:21 +03:00
mergify[bot]
58c5981cab
Merge branch 'master' into refactor-pipeline-executor 2021-11-11 16:16:52 +00:00
Nikolay Degterinsky
7b5954b736 Merge branch 'master' into query_parameters 2021-11-11 11:43:00 +00:00
Azat Khuzhin
baf14444e6 Cleanup ProfileEvents and CurrentMetrics 2021-11-10 21:15:27 +03:00
kssenii
6bc7fb1edf Fixes 2021-11-10 09:33:48 +00:00
Anton Popov
a20922b2d3 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-11-09 15:36:25 +03:00
kssenii
bfde7fd36c Better 2021-11-09 12:06:43 +00:00
mergify[bot]
f55e1439cb
Merge branch 'master' into output-formats 2021-11-08 15:33:45 +00:00
Dmitry Novik
a093395b9b Process incremental profile events on client 2021-11-08 16:38:31 +03:00
kssenii
dead99011b Merge branch 'master' of github.com:ClickHouse/ClickHouse into intersect-except-fix 2021-11-08 16:15:33 +03:00
Kseniia Sumarokova
908d78febe
Merge pull request #30851 from kssenii/clickhouse-local-improve 
Allow delayed interactive mode
 2021-11-08 10:07:29 +03:00
tavplubix
94a66f724e
Merge pull request #30135 from FArthur-cmd/add_parallel_reading_from_infile 
Add parallel file reading `FROM INFILE` in client
 2021-11-06 15:54:27 +03:00
kssenii
26c4affc18 Fixes 2021-11-04 11:00:35 +00:00
avogar
2dd4393ca1 Small refactoring in formats 2021-11-03 20:07:05 +03:00
mergify[bot]
8ef9e61e36
Merge branch 'master' into stress-test 2021-11-03 08:28:43 +00:00
kssenii
feb3d8bc75 Merge branch 'master' of github.com:ClickHouse/ClickHouse into clickhouse-local-improve 2021-11-02 20:01:15 +00:00
Kseniia Sumarokova
351c7bc8d6
Merge pull request #30881 from kssenii/fix-local-verbose 
clickhouse-local interactive fix --verbose and allow logging into file
 2021-11-01 23:08:49 +03:00
kssenii
35349056b8 Fix 2021-11-01 16:33:19 +03:00
zhangxiao871
45c12f9b2a merge from master 2021-11-01 18:32:56 +08:00
alexey-milovidov
04f5c4bd9a
Merge pull request #30938 from kssenii/fix-local-file-progress 
Fix file progress for clickhouse-local
 2021-11-01 12:39:12 +03:00
kssenii
d7dab834ec Fix file progress for local 2021-11-01 00:33:03 +03:00
Alexey Milovidov
993f3faefd Miscellaneous 2021-10-31 18:11:46 +03:00
Pavel Medvedev
7b50ab7136 skip spaces before and after \<letter> alias 
... for proper single letter like `\l`, `\d` handling.

Using `std::search()` for alias substring search instead of `.starts_with()`
to get the alias start position, to check the leading range contains only space
characters, and to get the rest of input after the space.

See issue #9339
 2021-10-31 15:46:30 +01:00
alexey-milovidov
c504e0c08d
Update ClientBase.cpp 2021-10-30 21:17:38 +03:00
Alexey Milovidov
05c2cd098c Fix parallel formatting and progress flicker in clickhouse-client 2021-10-30 21:02:33 +03:00
kssenii
e97233f33d Fix --verbose in local and logging 2021-10-30 17:23:24 +03:00
Pavel Medvedev
f4e3d053c2 add aliases for \<letter> interactive client commands 
initial implementation for:
  * \d aka SHOW TABLES
  * \l aka SHOW DATABASES
  * \c aka USE

Replace prefix of interactive `input` string with an aliased command
inside of the loop of `ClientBase::runInteractive()` in order to allow
command parameters, like `\c db_name`.

See issue #9339
 2021-10-30 09:00:45 +02:00
kssenii
07bab5193f Delayed interactive 2021-10-29 16:23:53 +03:00
Artur
89d8fb7622 correct test 2021-10-28 21:23:58 +00:00
Artur
72685d0c78 refactoring and test improvement 2021-10-28 18:46:51 +00:00
mergify[bot]
c1bff1b17d
Merge branch 'master' into stress-test 2021-10-28 14:53:30 +00:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
mergify[bot]
d9d9d38e4f
Merge branch 'master' into stress-test 2021-10-27 19:00:31 +00:00
Artur
20c5e9d321 refactoring, adding tests for local 2021-10-26 15:16:58 +00:00
zhangxiao871
bf9aebac90 Fix test and build 2021-10-26 12:45:09 +08:00
Nikolai Kochetov
05f42e2d07 Merge branch 'master' into refactor-pipeline-executor 2021-10-25 12:09:38 +03:00
Kruglov Pavel
855b10261c
Merge pull request #30285 from kssenii/fix-local-exceptions 
Fix printing stacktraces for clickhouse-local
 2021-10-24 19:54:59 +03:00
Azat Khuzhin
83d68246c3 Fix LOGICAL_ERROR on connection draining in case of ECONNRESET 
In case of ECONNRESET (Connection reset by peer) the "cancelled" will
not be set, and so drain() will fail with LOGICAL_ERROR:

<details>

    2021.10.16 04:10:55.054771 [ 7635 ] {882164e0-ca26-4955-8759-708ba1f6e85c} <Error> PullingAsyncPipelineExecutor: Code: 210. DB::NetException: Connection reset by peer, while writing to socket (127.0.0.2:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):

    ...
    4. ./obj-x86_64-linux-gnu/../src/Common/NetException.h:12: DB::NetException::NetException(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x20db5749 in /usr/bin/clickhouse
    5. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromPocoSocket.cpp:56: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x211d56bb in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:47: DB::WriteBuffer::next() @ 0x14e4c874 in /usr/bin/clickhouse
    7. ./obj-x86_64-linux-gnu/../src/Client/Connection.cpp:560: DB::Connection::sendCancel() @ 0x227c0c22 in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Client/HedgedConnections.cpp:242: DB::HedgedConnections::sendCancel() @ 0x227f1b53 in /usr/bin/clickhouse
    ...
    2021.10.16 04:12:35.555530 [ 5575 ] {} <Fatal> : Logical error: 'Cannot drain connections: cancel first.'.

</details>

Found by stress tests on CI [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/0/d3cb99701d4aaec384fd183168d10e0f650ecc1d/stress_test_(debug).html#fail1
 2021-10-23 22:44:32 +03:00
Filatenkov Artur
2384108ea5
remove trailing whitespaces 2021-10-22 15:32:24 +03:00
zhangxiao871
4e49eba087 Fix data race 2021-10-22 20:23:25 +08:00
Filatenkov Artur
328a605f7e
Merge branch 'ClickHouse:master' into add_parallel_reading_from_infile 2021-10-22 15:20:00 +03:00
Azat Khuzhin
b0984a3337 clickhouse-local: fix block lost in interactive mode 
In case only one block in the query execution, LocalConnection can
return Progress packet instead of the block itself, after it will call
poll() again and the block will got lost.

Fix this by processing block before Progress packet.

And actually AFAICS Progress can be removed after pollImpl(), since
there is Progress handling before.

Fixes: #30282 (cc @kssenii)
 2021-10-22 00:46:01 +03:00
zhangxiao871
6cd0f18bfd Fix PVS check 2021-10-21 17:59:24 +08:00
zhangxiao871
ac421b7faf Fix conflicts 2021-10-21 15:50:38 +08:00
zhangxiao871
e6cf9605a5 Refactor and add test. 2021-10-21 15:46:34 +08:00
mergify[bot]
943a1cbba0
Merge branch 'master' into stress-test 2021-10-21 04:41:23 +00:00
kssenii
ec7c93728b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into fix-local-exceptions 2021-10-20 23:42:27 +00:00
Kseniia Sumarokova
cae31437e8
Merge pull request #30336 from kssenii/fix-local-defaults 
Send table columns in clickhouse-local
 2021-10-20 21:18:02 +03:00
zhangxiao871
8480ae631a Refactor and add test. 2021-10-20 16:35:37 +08:00
Kseniia Sumarokova
5324cc8359
Merge pull request #30282 from kssenii/fix-local-less-threads 
Less threads in clickhouse-local, fix Ok. printing
 2021-10-19 22:54:28 +03:00
Nikolai Kochetov
171a6e35ce Fix build. 2021-10-19 21:39:34 +03:00
Nikolai Kochetov
1e1bd568ac Merge branch 'master' into refactor-pipeline-executor 2021-10-19 17:57:58 +03:00
Maksim Kita
0b3926950d
Merge pull request #30143 from amosbird/useupstreamreplxx 
Use upstream replxx
 2021-10-19 17:50:43 +03:00
Ilya Yatsishin
4396daeb1f
Merge pull request #30064 from azat/client-print-profile-events 2021-10-19 12:42:09 +03:00
kssenii
4bf1f2ca12 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into fix-local-exceptions 2021-10-19 08:28:47 +00:00
Azat Khuzhin
424bf6fcf4 client: add ability to print raw profile events 
This can be useful for debugging and for testing (since you will not
need to obtain query_id and look at query_log).

v2:
- mark test as long
- add option to docs
- add type of profile event into logs too
v3:
- resolve conflicts
- and fix onProfileEvents callback
v4:
- add --print-profile-events separate switch
 2021-10-19 00:54:38 +03:00
kssenii
b7a53df9de Send columns description in clickhouse-local 2021-10-18 14:53:42 +00:00
Nikolay Degterinsky
d76976e272 Merge branch 'master' into query_parameters 2021-10-18 17:34:20 +03:00
Kruglov Pavel
003955b59f
Merge branch 'master' into clickhouse-local 2021-10-18 14:24:08 +03:00
kssenii
1cc511789d Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into fix-local-less-threads 2021-10-18 08:30:59 +00:00
Nikolai Kochetov
bfcbf5abe0 Merge branch 'master' into removing-data-streams-folder 2021-10-17 10:42:37 +03:00
kssenii
4390dde76c Fix local break on timeout 2021-10-16 19:50:05 +00:00
kssenii
09e3aec96e Fix local break on timeout 2021-10-16 19:48:51 +00:00
alexey-milovidov
3d2fb2f67b
Merge pull request #30275 from kssenii/fix-local-query-stage 
--stage for clickhouse-local
 2021-10-16 18:42:28 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
kssenii
5db75f9e77 Better exception 2021-10-16 11:30:24 +00:00
kssenii
9525437499 Less threads in local, fix Ok. printing 2021-10-16 10:25:39 +00:00
kssenii
6c0eaf76da Query stage for local 2021-10-16 08:43:18 +00:00
Nikolai Kochetov
067eaadadd Merge branch 'master' into removing-data-streams-folder 2021-10-16 09:46:05 +03:00
Nikolai Kochetov
c668696047
Merge pull request #30171 from ClickHouse/remove-stream-interfaces 
Remove stream interfaces
 2021-10-16 09:34:01 +03:00
Nikolai Kochetov
fd14faeae2 Remove DataStreams folder. 2021-10-15 23:18:20 +03:00
alexey-milovidov
50b54b37ca
Merge pull request #30072 from ClickHouse/client-profile-msg 
Fix hardware utilization info printing in client
 2021-10-15 23:05:43 +03:00
avogar
be4fc79d32 Better handling exceptions, update tests 2021-10-15 18:30:32 +03:00
Filatenkov Artur
77bbd949dc
Merge branch 'ClickHouse:master' into add_parallel_reading_from_infile 2021-10-15 15:26:45 +03:00
avogar
a1a4df2501 Fix handling exception 'unrecognised option' in clickhouse-local and client 2021-10-14 16:34:05 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Nikolai Kochetov
3d3e143c29
Merge pull request #30001 from ClickHouse/remove-streams-from-formats 
Remove streams from formats.
 2021-10-14 12:39:07 +03:00
Nikolay Degterinsky
63faf0212d Merge branch 'master' into query_parameters 2021-10-13 23:55:09 +03:00
Amos Bird
8851cb8459
Use upstream replxx 2021-10-14 00:52:53 +08:00
Filatenkov Artur
bf1d45362d
Merge branch 'master' into add_parallel_reading_from_infile 2021-10-13 18:47:13 +03:00
Artur
18c2ac2e1e add parallel reading from infile in client 2021-10-13 15:43:52 +00:00
zhangxiao871
923c76fc99 Modify comments 2021-10-13 18:38:24 +08:00
Nikolai Kochetov
a5fa5c7ea3 Move formats to Impl 2021-10-13 13:01:08 +03:00
Nikolay Degterinsky
49c1a52f34 Merge branch 'master' into query_parameters 2021-10-13 03:04:40 +03:00
Nikolay Degterinsky
5fa0f9e7fd Add parameters to INSERT queries 2021-10-13 03:01:41 +03:00
Dmitry Novik
f1bbc7f9b6 Fix hardware utilization info printing in client 2021-10-12 23:17:15 +03:00
Nikolai Kochetov
2c92caac17 Fix clickhouse local. 2021-10-11 21:20:35 +03:00
Nikolai Kochetov
ec18340351 Remove streams from formats. 2021-10-11 19:11:50 +03:00
Dmitry Novik
bfdd34c13d code cleanup 2021-10-11 17:39:24 +03:00
Dmitry Novik
7c3192735a Reset profile events stream in Connection::sendQuery 2021-10-11 17:39:24 +03:00
Dmitry Novik
9f9af28b5e Output memory usage with progress 2021-10-11 17:39:24 +03:00
Dmitry Novik
73df6190df Cleanup code 2021-10-11 17:39:24 +03:00
Dmitry Novik
7e3caf96be Fix cores approximation 2021-10-11 17:39:24 +03:00
Dmitry Novik
4c6b3c40f2 Calculate approximate cores number used 2021-10-11 17:39:24 +03:00
Dmitry Novik
15ac65aa33 Add thread usage info on client side 2021-10-11 17:39:24 +03:00
Dmitry Novik
356723427d WIP on ProfileEvents forwarding 2021-10-11 17:39:23 +03:00
Dmitry Novik
362bcb2f66 Introduce ProfileEvents packet 2021-10-11 17:29:50 +03:00
Nikolai Kochetov
a95c28ec4b
Merge pull request #29898 from ClickHouse/remove-native-stream 
Remove some more streams.
 2021-10-10 21:01:16 +03:00
Alexey Milovidov
103b3c91ba Remove 'printf' function usage. 2021-10-10 04:21:07 +03:00
Nikolai Kochetov
c6bce1a4cf Update Native. 2021-10-08 20:21:19 +03:00
Nikolai Kochetov
340b53ef85 Remove some more streams. 2021-10-08 17:03:54 +03:00
kssenii
c734ada95b Fix 2021-10-07 20:28:38 +00:00
mergify[bot]
0910a8ec74
Merge branch 'master' into stress-test 2021-10-05 09:46:18 +00:00
Azat Khuzhin
bbee102e58 Fix --stage for clickhouse-local 
This also fixes UBsan error, since query_processing_stage was not
initialized before for clickhouse-local.
 2021-10-04 21:00:48 +03:00
Pavel Kruglov
2db11bc6c5 Add backward compatibility check in stress test 2021-10-04 16:34:14 +03:00
Kseniia Sumarokova
ddc775b1c8
Merge pull request #29626 from kssenii/fix-signals 
Follow-up for #26231
 2021-10-03 22:13:54 +03:00
Kseniia Sumarokova
ad0fcda713
Update ClientBase.h 2021-10-03 12:11:59 +03:00
kssenii
aa539937fb Fix build check 2021-10-03 06:23:05 +00:00
kssenii
0ed92d85e2 Fix 2021-10-02 13:05:19 +00:00
kssenii
e10255bba3 Fix 2021-10-02 08:10:34 +00:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
kssenii
dabab85c60 Fix 2021-10-01 17:31:00 +00:00
kssenii
05181c6900 Fix 2021-10-01 14:13:02 +00:00
kssenii
59cbe8579d Interrupt listener 2021-09-29 22:01:52 +00:00
kssenii
07b979bda3 Rewrite to processors 2021-09-29 19:17:02 +00:00
kssenii
2c005db9ad Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-09-29 17:45:15 +00:00
Nikolai Kochetov
78a7665f43 Merge branch 'master' into rewrite-pushing-to-views 2021-09-27 10:56:50 +03:00
kssenii
8e200b68a4 Small fix 2021-09-26 21:22:04 +00:00
kssenii
2129230b1e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-09-25 09:16:05 +00:00
kssenii
f3784780fd Fixes 2021-09-25 08:46:57 +00:00
Alexey Milovidov
cd7f9d981c Remove ya.make 2021-09-25 04:22:54 +03:00
Nikolai Kochetov
998d29ebc7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-23 13:10:27 +03:00
Azat Khuzhin
015695b3bf Fix connection timeouts (send_timeout/receive_timeout) 
Query is not executed within sendQuery() function.

INSERT query consist from multiple parts:
- sendQuery()
- write()/writePrepared() -- send Data blocks
- writeSuffix() -- send empty Data block and receive EndOfStream
(see RemoteBlockOutputStream for more info)
So as you can see it is not executed completely from the sendQuery() function.

SELECT query (and others) also send Data blocks to the client after.

And what this means that temporary timeout (via TimeoutSetter) is not
enough, since next query can use timeout from the previous query.

Usually you do not see it, because:
- You don't use custom send_timeout/receive_timeout
- SELECT is fast enough

However it is not the case with INSERT, especially with
insert_distributed_sync=1, since
DistributedBlockOutputStream::writeSuffix() may stack because remote
server still INSERT'ing the data (i.e. creating znodes in zookeeper for
ReplicatedMergeTree, syncing blocks, and similar) that was issued from
DistributedBlockOutputStream::write(), and it will lead to the following
exception:

    Code: 209, e.displayText() = DB::NetException: Timeout exceeded while reading from socket (127.1:9000): while receiving packet from ch-47-drt.dpa.semrush.net:9000: Insertion status:
    Wrote X blocks and Y rows on shard Z replica 0, 127.1:9000 (average 0 ms per block, the slowest block N ms)
    0. DB::Exception::Exception()
    1. DB::ReadBufferFromPocoSocket::nextImpl()
    2. void DB::readVarUIntImpl<false>(unsigned long&, DB::ReadBuffer&)
    3. DB::Connection::receivePacket()
    4. DB::RemoteBlockOutputStream::writeSuffix()
    ...
 2021-09-23 09:03:13 +03:00
kssenii
04575eb12b Review fixes, fix style check and clang-tidy 2021-09-22 21:35:29 +00:00
Azat Khuzhin
e5f66fd103 Add socket timeout values into SOCKET_TIMEOUT exception 2021-09-20 09:51:42 +03:00
kssenii
8d19ed5ee7 Fix remaining tests 2021-09-20 05:39:13 +00:00
kssenii
db45363c19 Fix tests 2021-09-19 18:42:45 +00:00
kssenii
3b4843a1a9 Apply changed from pr 27135 2021-09-16 18:44:42 +00:00
kssenii
55619dcc1b Correct merge 2021-09-16 16:14:17 +00:00
kssenii
c88dc460d4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-09-16 16:13:04 +00:00
Nikolai Kochetov
b997214620 Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 20:48:18 +03:00
kssenii
4605299262 Correct merge 2021-09-13 06:45:27 +00:00
Sergei Semin
f74350c148 fix special build 2021-09-12 02:49:31 +03:00
kssenii
33ff6ec060 Fix tests 2021-09-11 17:17:48 +00:00
kssenii
7f1770cc3c Fix contexts 2021-09-11 12:03:28 +00:00
kssenii
ca81c13775 Move readArguments to base 2021-09-11 11:49:42 +00:00
kssenii
ddfe1e4d64 Some fixes 2021-09-11 11:34:22 +00:00
kssenii
fa04b13175 Break some tests 2021-09-09 13:25:25 +00:00
kssenii
8f77855981 Some review fixes 2021-09-05 00:52:35 +03:00
kssenii
1c20b223cb Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-09-04 20:01:31 +03:00
Alexey Milovidov
19445ac096 Drop unused headers 2021-09-01 02:47:52 +03:00
Dmitrii Kovalkov
9871ad70ff Exclude fuzzers 2021-08-30 11:12:25 +03:00
kssenii
578a750b8b Small clean up 2021-08-27 11:47:47 +03:00
kssenii
7d6f3d96c1 Better 2021-08-23 13:01:42 +03:00
kssenii
95645e8f26 Fix progress 2021-08-23 10:13:27 +03:00
kssenii
deeb942ccb Fix 2021-08-21 21:42:59 +03:00
kssenii
1f0bb0d81b Reorganize a bit more, fix tests 2021-08-21 18:17:02 +03:00
kssenii
3a39e3f039 Reorganize options 2021-08-21 00:39:28 +03:00
kssenii
21f9622cad Reorganize multiquery processing 2021-08-20 23:39:13 +03:00
kssenii
41d735fd56 Fix context 2021-08-20 15:16:13 +03:00
kssenii
39e80a47a3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-08-20 10:13:58 +03:00
kssenii
ceca7a7e2c Fix some tests 2021-08-19 23:27:40 +03:00
kssenii
b5f6a7cb97 Fix local 2021-08-19 14:07:47 +03:00
kssenii
d1484a6f2e Use more client code than local, hope tests survive 2021-08-19 00:01:17 +03:00
kssenii
e187bb781c Move everything to base 2021-08-18 17:39:04 +03:00
kssenii
fb0b445ee0 Lets add LocalConnection (a start) 2021-08-17 22:59:51 +03:00
Amos Bird
27ff081403
Fix some leftover TODOs 2021-08-17 17:22:12 +08:00
kssenii
cefd9b473e Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-08-16 16:41:24 +03:00
kssenii
30cc705464 Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-08-07 22:53:50 +03:00
kssenii
9f9be65ac7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-08-05 10:42:32 +03:00
kssenii
b5b9624cca Fixes, add test 2021-08-03 23:49:16 +03:00
kssenii
f6e45b3193 Fix tests, style. Better cancel query 2021-08-02 17:46:21 +03:00
Nikolai Kochetov
6951e8147d
Merge pull request #27020 from amosbird/shardlevelconstness 
Shard-level const column
 2021-08-02 13:45:20 +03:00
kssenii
6d4440836b Process SIGINT in interactive mode 2021-08-02 01:22:07 +03:00
kssenii
3982031215 Completion for clickhouse-local 2021-08-01 23:28:39 +03:00
Amos Bird
b76a854f5a
backward compatible 2021-08-01 09:06:16 +08:00
kssenii
6e8eb598bf Some review fixes 2021-07-31 15:34:29 +03:00
kssenii
c9e607f1e8 Progress for interactive clickhouse local 2021-07-31 15:08:30 +03:00
kssenii
48f4f2a0c2 Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-07-29 18:41:54 +03:00
kssenii
3f259e195b Better 2021-07-29 15:58:50 +03:00
Nikolai Kochetov
cfa3fd4b87 Hold context in HedgedConnections to prevent use-after-free on settings. 2021-07-29 13:29:36 +03:00
kssenii
22e3bde548 Refactoring, fix test, fix local interactive 2021-07-29 12:45:58 +03:00
kssenii
62d1e18b7f Refactoring 2021-07-28 18:54:32 +03:00
kssenii
fb18f7fc72 Fix tests 2021-07-27 17:15:31 +03:00
Nikolai Kochetov
a9effdad18 Merge branch 'master' into output-streams-to-processors 2021-07-27 11:31:53 +03:00
kssenii
d30f54b335 Refactoring, fix tests 2021-07-27 01:28:27 +03:00
Nikolai Kochetov
7da5bb3871 Do not start new hedged connection if query was already canceled. 2021-07-26 20:48:38 +03:00
Nikolai Kochetov
61d8f880cd Rename some files. 2021-07-26 19:48:25 +03:00
kssenii
4886d931a1 Fix some tests 2021-07-26 11:24:04 +03:00
alexey-milovidov
7b76bfc719
Merge pull request #26318 from amosbird/connectionpoolfactory 
Connection pool cache
 2021-07-26 06:45:03 +03:00
Amos Bird
2ba6ef1487
Better implementation 2021-07-24 18:02:51 +08:00
kssenii
4dbb2e5bd8 Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-07-24 00:36:25 +03:00
kssenii
accd6d5c0f Refactoring 2021-07-24 00:32:38 +03:00
kssenii
8c05e4f038 Refactoring 2021-07-23 23:54:49 +03:00
vdimir
dccc379d39
Fix use after free in AsyncDrain connection from S3Cluster 2021-07-23 10:40:03 +03:00
kssenii
59b0ce21e4 Tiny refactoring 2021-07-23 00:27:26 +03:00
kssenii
b337f481dc Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-07-22 22:24:19 +03:00
Nikolai Kochetov
20b2837b27 Fix destructor for PocoSocketWrapper. 2021-07-20 16:50:31 +03:00
Amos Bird
dbfb699690
Asynchronously drain connections. 2021-07-19 21:53:29 +08:00
kssenii
022a0e00f2 Refactor client, merging non-interactive 2021-07-18 23:18:11 +03:00
kssenii
bb0f0c18b2 Merge branch 'master' of github.com:ClickHouse/ClickHouse into interactive-mode-for-clickhouse-local 2021-07-18 18:12:53 +03:00
Nikolai Kochetov
0d549c197f Fix 01822_async_read_from_socket_crash for hedged connection. 2021-07-16 13:42:22 +03:00
kssenii
ae4644d05a Fix some tests 2021-07-15 10:02:14 +03:00
Amos Bird
3f05211b09
Connection pool factory. 2021-07-15 00:01:45 +08:00
kssenii
7a57f2a983 Turn on fuzzing, fix tests 2021-07-12 20:26:21 +03:00
kssenii
bc3c038ac7 Better 2021-07-12 17:01:46 +03:00
kssenii
9ff54216ea More merges in query execution code 2021-07-12 15:25:17 +03:00
kssenii
e31eccfaf2 Support multiquery option 2021-07-12 12:34:37 +03:00
kssenii
a10b9ed63a Start merging query processing code 2021-07-12 02:17:14 +03:00
kssenii
d58a5537c2 Fix details 2021-07-12 00:54:41 +03:00
kssenii
026fa0ca64 Merge main, mainImpl code, allow interactive mode for clickhouse-local 2021-07-11 23:35:29 +03:00
kssenii
0b5dab1326 Merge init code for local and client 2021-07-11 14:36:27 +03:00
alexey-milovidov
aca575cc85
Merge branch 'master' into refactor-pipeline-executor 2021-07-04 02:08:31 +03:00
Alexey Milovidov
e3653e70da Update prompt in client when reconnecting 2021-06-15 04:43:35 +03:00
Alexey Milovidov
273226de32 Remove string parameter for Density 2021-05-24 06:43:25 +03:00
Alexey Milovidov
40d4f0678f Remove overload (harmful) 2021-05-23 04:25:06 +03:00
fibersel
cb53bbb7b0 add experimental codecs flag, add integration test for experimental codecs 2021-05-06 14:57:22 +03:00
alexey-milovidov
7ca42d2ce1
Merge pull request #23805 from ClickHouse/fix-hedged-requests-ext-tables 
Fix sending external table for hedged connections.
 2021-05-01 08:13:13 +03:00
Nikolai Kochetov
352ec049ad Fix sending external table for hedged connections. 2021-04-30 17:19:48 +03:00
Maksim Kita
318c4bb80d Add examples folder filter to ya.make.in 2021-04-30 11:25:52 +03:00
Maksim Kita
0d0a14a925 Poco::Timespan reference fix 2021-04-29 19:11:20 +03:00
Nikita Mikhaylov
9f55424250 move to examples everywhere 2021-04-27 01:51:42 +03:00
Nikolai Kochetov
548fb52d24 Fix build. 2021-04-20 08:51:51 +03:00
Nikolai Kochetov
08d1896950 Fix build. 2021-04-19 22:06:05 +03:00
Nikolai Kochetov
0420b7bb33 Reset replica state in MultiplexedConnections in case of exception. 2021-04-19 17:25:43 +03:00
Alexey Milovidov
17d04cbca9 Merge branch 'master' into protocol-compression-auto 2021-04-15 00:24:07 +03:00
Alexey Milovidov
6f56c3280f Uncompress data in Distributed sends if needed 2021-04-14 00:53:39 +03:00
Nikita Mikhaylov
024374a2ec review fixes 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
7a68820342 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
c333c3dedb review fixes 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
36a8419f60 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
2555ae5d3f better processing stage 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4843f86329 use only one connection between initiator and worker 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
e601a432e5 better[2] 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
cef9e19eb2 better 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
2549468c14 better 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
64b4cd0e63 save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
31b4f9b17f save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
44ca65a9a4 save 2021-04-13 22:39:41 +03:00
Alexey Milovidov
fafe8c00fe Fix warning #22987 2021-04-12 20:57:03 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
Pavel Kruglov
6f65a72112 Remove socket from epoll before cancelling packet receiver to prevent race 2021-04-01 17:35:00 +03:00
alexey-milovidov
25f02f13b0
Merge pull request #22238 from ClickHouse/remove-harmful-code 
Remove harmful default parameters
 2021-03-29 23:46:18 +03:00
Alexey Milovidov
169369e90f Remove harmful default parameters 2021-03-29 04:08:28 +03:00
Alexey Milovidov
b336de9ebd Remove harmful default arguments 2021-03-29 04:07:42 +03:00
Pavel Kruglov
90afc9ce2f Fix build 2021-03-26 21:25:30 +03:00
Pavel Kruglov
2979feb23f Prevent busy waiting in hedged requests when async_socket_for_remote=0 2021-03-26 18:11:06 +03:00
Kruglov Pavel
66b8431e71
Merge pull request #21429 from Avogar/secure-socket 
Fix receive and send timeouts and non-blocking read in secure socket
 2021-03-25 17:30:17 +03:00
Azat Khuzhin
17eaddc3ab Prevent hedged connections overlaps 
Some replicas may be in the establishing state, this means that hedged
connection is waiting for TablesStatusResponse, and if the connection
will not be canceled, then next user of the connection will get
TablesStatusResponse, while this is not the expected package, and you
will see:

    DB::Exception: Received from localhost:9000. DB::Exception: Received from ch-1:9000. DB::Exception: Unknown packet 9 from server ch-2:9000: While executing Remote. Stack trace:

Fix this by disconnecting connection to such replicas, note that this
should be fine in general, since they are "slow" anyway (you may wish
configure hedged requests settings to increase timeouts and similar).
 2021-03-20 19:05:29 +03:00
Kruglov Pavel
055242fe95
Style 2021-03-19 01:38:15 +03:00
Pavel Kruglov
d380454044 Change read_data_timeout from sec to ms, add profile event for hedged requests and better process receiving packets 2021-03-18 20:16:17 +03:00
Pavel Kruglov
dbae58f30b Add slowdown_count and show actual information in system.clusters 2021-03-06 02:45:17 +03:00
Pavel Kruglov
c113af39f4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into secure-socket 2021-03-05 16:18:45 +03:00
Pavel Kruglov
1c7f16e0ff Add test and move TimeoutSetter in IO/ 2021-03-03 20:47:27 +03:00
Pavel Kruglov
dc9dd8ccdc Fix typo 2021-03-03 16:16:17 +03:00
Pavel Kruglov
82a6be44cf Disable changing replica when we have made progress 2021-03-03 01:03:03 +03:00
Pavel Kruglov
4166ae1f45 Restart timeout when we make progress 2021-03-02 15:40:24 +03:00
Pavel Kruglov
1b1f50e324 Update comment 2021-02-27 11:45:32 +03:00
Pavel Kruglov
4d7ba0443f Add checking pending data 2021-02-27 11:40:03 +03:00
Kruglov Pavel
cb638e0f87
Fix style 2021-02-26 21:06:17 +03:00
Pavel Kruglov
91f6f9227b Small fixes and tests updates 2021-02-26 18:55:58 +03:00
Pavel Kruglov
24e9abddb8 Fix 2021-02-24 15:11:07 +03:00
Pavel Kruglov
5628fd6da5 Small update 2021-02-24 02:22:07 +03:00
Pavel Kruglov
edee592cb0 Fix 2021-02-22 20:57:44 +03:00
Pavel Kruglov
1dbab60235 Fix 2021-02-22 10:52:19 +03:00
Pavel Kruglov
6d8eb44c01 Fix PSV and remove debug logging 2021-02-22 00:59:07 +03:00
Pavel Kruglov
4320782032 Style 2021-02-21 20:43:36 +03:00
Pavel Kruglov
db591309a3 fix style 2021-02-21 17:51:36 +03:00
Pavel Kruglov
93162716f4 remove logging 2021-02-21 17:50:47 +03:00
Pavel Kruglov
dd5185c779 Refactor 4.0 2021-02-21 17:03:24 +03:00
Pavel Kruglov
9bd9ea9fbc Try to fix sigsev 2021-02-20 00:45:58 +03:00
Pavel Kruglov
2f5b4c20ae Fix 2021-02-18 14:21:48 +03:00
Pavel Kruglov
80b6db7f72 Style 2021-02-17 22:07:14 +03:00
Pavel Kruglov
bc6fe4f28c Minor fixes 2021-02-17 20:44:11 +03:00
Pavel Kruglov
1b78de2142 Use fibers in HedgedRequests 2021-02-17 20:34:52 +03:00
Pavel Kruglov
c57daf2140 Merge branch 'hedged-requests' of github.com:Avogar/ClickHouse into hedged-requests 2021-02-17 20:34:11 +03:00
Pavel Kruglov
0e1d67ad9a Add LOG_DEBUG to debug test_distributed_load_balancing 2021-02-16 10:56:45 +03:00
Kruglov Pavel
6dcb306060
Style 2021-02-16 00:46:51 +03:00
Kruglov Pavel
28dec516ac
Style 2021-02-15 19:12:03 +03:00