Commit Graph

150 Commits

Author SHA1 Message Date
Dmitry Novik
362bcb2f66 Introduce ProfileEvents packet 2021-10-11 17:29:50 +03:00
Nikolai Kochetov
c6bce1a4cf Update Native. 2021-10-08 20:21:19 +03:00
Nikolai Kochetov
78e1db209f
Remove more data streams (#29491)
* Remove more streams.

* Fixing build.

* Fixing build.

* Rename files.

* Fix fast test.

* Fix StorageKafka.

* Try fix kafka test.

* Move createBuffer to KafkaSource ctor.

* Revert "Move createBuffer to KafkaSource ctor."

This reverts commit 81fa94d27e.

* Revert "Try fix kafka test."

This reverts commit 2107e54969.

* Comment some rows in test.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-10-07 11:26:08 +03:00
Azat Khuzhin
4f5118b85f Fix data-race between fatal error handler and progress packets
This is not that important, but because of packets overlaps the client
may not render the fatal error.

You will find TSan report in `details`.

<details>

```
WARNING: ThreadSanitizer: data race (pid=13384)
  Write of size 8 at 0x7b24000ad818 by thread T47:
    0 DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:43:15 (clickhouse-tsan+0x16124d61)
    1 DB::TCPHandler::sendProgress() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1602:10 (clickhouse-tsan+0x16124d61)
    2 DB::TCPHandler::runImpl()::$_6::operator()() const obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:318:25 (clickhouse-tsan+0x1612cf54)

  Previous write of size 8 at 0x7b24000ad818 by thread T229:
    0 DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:43:15 (clickhouse-tsan+0x1612b793)
    1 DB::TCPHandler::sendLogData(DB::Block const&) obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1561:10 (clickhouse-tsan+0x1612b793)
    2 DB::TCPHandler::sendLogs() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1632:9 (clickhouse-tsan+0x161228ab)
    3 DB::TCPHandler::runImpl()::$_1::operator()() const obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:229:62 (clickhouse-tsan+0x1612c4e1)
```

</details>
2021-10-05 21:21:32 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Azat Khuzhin
e8a90b8ff2 Do not allow to reuse previous credentials in case of inter-server secret
Before this patch INSERT via Buffer/Kafka may re-use previously set user
for that connection, while this is not correct, it should reset the
user, and use global context.

Note, before [1] there was a fallback to default user, but that code had
been removed, and now it got back.

  [1]: 0159c74f21 ("Secure inter-cluster query execution (with initial_user as current query user) [v3]")

Also note, that context for Buffer table (and others) cannot be changed,
since they don't have any user only profile.

I've tested this patch manually using the following:

    create table dist (key Int) engine=Distributed(test_cluster_two_shards_secure, default, data, key);
    create table buffer (key Int) engine=Buffer(default, dist, 1, 0, 0, 0, 0, 0, 0);
    create table data (key Int) engine=Memory();

    # to start the connection with readonly user
    $ clickhouse-client --user readonly -q 'select * from dist'
    $ clickhouse-client -q 'insert into buffer values (1)'
    # before this patch this produces errors like:
    # 2021.09.27 23:46:48.384920 [ 19474 ] {} <Error> default.dist.DirectoryMonitor: Code: 164. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: readonly: Cannot execute query in readonly mode. Stack trace:

v2: reset the authentication instead of using default user (as suggested by @vitlibar)
v3: reset Session::user and introduce ClientInfo::resetAuthentication (as suggested by @vitlibar)
v4: reset the session every time in interserver mode (suggested by @vitlibar)
2021-10-01 01:13:15 +03:00
Nikolai Kochetov
78a7665f43 Merge branch 'master' into rewrite-pushing-to-views 2021-09-27 10:56:50 +03:00
Azat Khuzhin
b3d1bfc67a Send UNKNOWN_DATABASE to the client (via TCP)
Before this patch:

    $ clickhouse-client --database foo -q 'select 1'
    Code: 32. DB::Exception: Attempt to read after eof: while receiving packet from localhost:9000. (ATTEMPT_TO_READ_AFTER_EOF)

After:

    $ clickhouse-client --database foo -q 'select 1'
    Received exception from server (version 21.11.1):
    Code: 81. DB::Exception: Received from localhost:9000. DB::Exception: Database foo doesn't exist. (UNKNOWN_DATABASE)
    (query: select 1)

Fixes: #26864 (cc @vitlibar)
2021-09-26 23:45:29 +03:00
Nikolai Kochetov
eed4e8c754 Fix progress for insert select. 2021-09-22 16:29:58 +03:00
Nikolai Kochetov
81bf13a247 Refactor pushing to views. 2021-09-22 13:57:00 +03:00
Nikolai Kochetov
db6f1e198c Add cancell callback to completed executor. 2021-09-21 19:37:32 +03:00
Nikolai Kochetov
a8c3b02598 Merge branch 'master' into rewrite-pushing-to-views 2021-09-17 15:38:11 +03:00
Nikolai Kochetov
58bb5fe462 Fix some tests. 2021-09-17 14:40:03 +03:00
Nikolai Kochetov
341553febd Fix build. 2021-09-16 20:40:42 +03:00
Nikolai Kochetov
e616732743 Small refactoring. 2021-09-15 22:35:48 +03:00
Nikolai Kochetov
3a9d88fbc9 Fix more tests. 2021-09-10 17:52:24 +03:00
Nikolai Kochetov
f569a3e3f7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-09 20:30:23 +03:00
Nikolai Kochetov
66a76ab70f Rewrite PushingToViewsBlockOutputStream part 6 2021-09-03 20:29:36 +03:00
Nikolai Kochetov
358babc31b Rewrite PushingToViewsBlockOutputStream part 5. 2021-09-02 14:29:20 +03:00
Anton Popov
6d3274c22c Merge remote-tracking branch 'upstream/master' into HEAD 2021-08-26 03:31:46 +03:00
Vitaly Baranov
64f69462c6
Merge pull request #27921 from vitlibar/add-tests-for-field-init
Allow implicit cast bool to Field
2021-08-24 17:15:44 +03:00
Nikita Mikhaylov
fb6462d70a
Merge pull request #27886 from vitlibar/add-separate-constants-for-interfaces
Add separate constants for interfaces LOCAL and TCP_INTERSERVER.
2021-08-23 16:25:01 +03:00
Vitaly Baranov
ff1fad8521 Use setSetting() with implicit conversion true/false -> Field. 2021-08-23 08:57:00 +03:00
Alexey Milovidov
c3c31e2895 Move function to appropriate place to make code more readable 2021-08-20 12:56:39 +03:00
Vitaly Baranov
3ab1177d7e Add separate constants for interface LOCAL and TCP_INTERSERVER. 2021-08-19 23:04:36 +03:00
Vitaly Baranov
fabd7193bd Code cleanups and improvements. 2021-08-18 14:24:52 +03:00
Vasily Nemkov
51ffc33457 Introduce sessions.
This is required to add system.session_log table.
2021-08-18 14:24:52 +03:00
Braulio Valdivielso
a833d1cf9e fixup! change http error response 2021-08-07 19:57:19 +01:00
Braulio Valdivielso
6189757fed change http error response
As reported in #27171, the logic that generates the HTTP error
response didn't work in case the configuration hadn't specified a
`http_port`. The new logic checks whether the `http_port` was set in
the first place.
2021-08-06 23:32:37 +01:00
Anton Popov
3a0d4807a5 Merge branch 'async-insert' of git://github.com/abyss7/ClickHouse into merging-20557 2021-08-06 16:20:02 +03:00
Nikolai Kochetov
9c92f43359 Update storages. 2021-07-23 22:33:59 +03:00
Amos Bird
dbfb699690
Asynchronously drain connections. 2021-07-19 21:53:29 +08:00
Vladimir
f120bd3d09
Directly pass getExceptionMessage() to LOG_ERROR in TCPHandler.cpp 2021-07-10 13:30:38 +03:00
Alexey Milovidov
1f91c65aca Add error id to exceptions 2021-07-10 05:43:53 +03:00
Ivan Lezhankin
37439a8af6 Fix typos 2021-06-29 01:51:01 +03:00
Ivan Lezhankin
09b3db38f0 Don't create input stream for native data 2021-06-28 23:43:37 +03:00
Ivan Lezhankin
37365589ed Merge branch 'master' into async-insert 2021-06-17 16:57:45 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Amos Bird
7d9a1106b8
Add initial_query_start_time to query log 2021-06-11 22:24:53 +08:00
Ivan Lezhankin
973cea73b7 Merge branch 'master' into async-insert 2021-06-08 15:10:30 +03:00
Ivan Lezhankin
84c23dc060 Merge branch 'master' into async-insert 2021-06-02 18:06:21 +03:00
Alexey Milovidov
273226de32 Remove string parameter for Density 2021-05-24 06:43:25 +03:00
Alexey Milovidov
40d4f0678f Remove overload (harmful) 2021-05-23 04:25:06 +03:00
alexey-milovidov
1ed30f7246
Merge branch 'master' into issue-16775 2021-05-22 17:14:23 +03:00
Alexey Milovidov
ad88819ee4 Fix a bunch of warnings from PVS-Studio 2021-05-08 19:13:10 +03:00
fibersel
cb53bbb7b0 add experimental codecs flag, add integration test for experimental codecs 2021-05-06 14:57:22 +03:00
Maksim Kita
0d0a14a925 Poco::Timespan reference fix 2021-04-29 19:11:20 +03:00
Azat Khuzhin
18575c4433 Avoid flushing of BlockOutputStream from destructor
That may lead to uncaught exception [1].

  [1]: https://github.com/ClickHouse/ClickHouse/pull/21624#issuecomment-808066747

v0: New method BlockOutputStream::finalize() -- bad (forgot some places
    and superfluous)
v2: Flush BlockOutputStream in QueryStatus::releaseQueryStreams()
    Safer query termination (report some uncaugh errors to user)
v3: Flush BlockOutputStream from TCPHandler only
    Other places uses different executeQuery() implementation
v4: Do not call writeSuffix() twice
2021-04-25 12:49:44 +03:00
Nikolai Kochetov
44112587d4
Merge pull request #23309 from ClickHouse/try-fix-crash-with-unknown-packet
Fix crash in case of unknown packet
2021-04-21 08:47:17 +03:00
Ivan
10c5ba3022
Merge branch 'master' into async-insert 2021-04-20 17:58:15 +03:00