thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 11:52:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
86,382 Commits 9,326 Branches 1,784 Tags 2.1 GiB
daceac5c0b
Commit Graph

774 Commits

Author SHA1 Message Date
alesapin
f1fe96e194 Merge branch 'master' into debug_keeper 2021-10-18 10:16:07 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
Nikolai Kochetov
067eaadadd Merge branch 'master' into removing-data-streams-folder 2021-10-16 09:46:05 +03:00
Nikolai Kochetov
c668696047
Merge pull request #30171 from ClickHouse/remove-stream-interfaces 
Remove stream interfaces
 2021-10-16 09:34:01 +03:00
Nikolai Kochetov
fd14faeae2 Remove DataStreams folder. 2021-10-15 23:18:20 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Maksim Kita
8ed4e73705
Merge pull request #30000 from ClickHouse/profile-snapshot 
Make read of Counters snapshot non-atomic
 2021-10-14 13:22:44 +03:00
alesapin
90ff7f05fd Start keeper asynchronously if has connection to other nodes 2021-10-14 13:21:41 +03:00
Nikolai Kochetov
3d3e143c29
Merge pull request #30001 from ClickHouse/remove-streams-from-formats 
Remove streams from formats.
 2021-10-14 12:39:07 +03:00
Maksim Kita
3d2e1a24d9 Fixed tests 2021-10-14 00:33:18 +03:00
Maksim Kita
04047f76c7 Fixed tests 2021-10-14 00:33:18 +03:00
Nikolai Kochetov
49ac602ee6 Fix GRPC 2021-10-13 15:06:56 +03:00
Nikolai Kochetov
06d9fc602e Merge branch 'master' into fix-backward-compatibility-after-21196 2021-10-12 14:41:23 +03:00
Dmitry Novik
8be70bc417 Update profile event forwarding 2021-10-12 14:15:07 +03:00
Vitaly Baranov
4a4d913cfd
Merge pull request #29954 from vitlibar/fix-flaky-test_grpc_protocol 
gRPC: Fix releasing query ID and session ID at the end of query processing
 2021-10-12 13:32:27 +03:00
Nikolai Kochetov
ec18340351 Remove streams from formats. 2021-10-11 19:11:50 +03:00
Dmitry Novik
bfdd34c13d code cleanup 2021-10-11 17:39:24 +03:00
Dmitry Novik
9590c97a19 Create ProfileEventsQueue in GRPCServer 2021-10-11 17:39:24 +03:00
Dmitry Novik
3a0764634e Update InternalProfileEventsQueue usage 2021-10-11 17:39:24 +03:00
Dmitry Novik
0bdabf46f2 Send ProfileEvents only to supported clients 2021-10-11 17:39:24 +03:00
Dmitry Novik
9f9af28b5e Output memory usage with progress 2021-10-11 17:39:24 +03:00
Dmitry Novik
73df6190df Cleanup code 2021-10-11 17:39:24 +03:00
Dmitry Novik
1d2e2d7305 cleanup 2021-10-11 17:39:24 +03:00
Dmitry Novik
7e3caf96be Fix cores approximation 2021-10-11 17:39:24 +03:00
Dmitry Novik
9071a7151e Fix communication & race conditions 2021-10-11 17:39:23 +03:00
Dmitry Novik
356723427d WIP on ProfileEvents forwarding 2021-10-11 17:39:23 +03:00
Dmitry Novik
74cdaba7fa WIP on profile events forwarding 2021-10-11 17:29:50 +03:00
Dmitry Novik
803b8623c1 Fix TCPHandler::sendProfileEvents 2021-10-11 17:29:50 +03:00
Dmitry Novik
e9b1e05461 Send profile events from all threads of current group 2021-10-11 17:29:50 +03:00
Dmitry Novik
362bcb2f66 Introduce ProfileEvents packet 2021-10-11 17:29:50 +03:00
Nikolai Kochetov
a95c28ec4b
Merge pull request #29898 from ClickHouse/remove-native-stream 
Remove some more streams.
 2021-10-10 21:01:16 +03:00
Vitaly Baranov
1dda596689 Fix releasing query ID and session ID at the end of query processing. 2021-10-10 17:34:45 +03:00
alexey-milovidov
ee577e1ab4
Merge pull request #29155 from FArthur-cmd/add_CORS 
Add Cors support
 2021-10-09 20:18:17 +03:00
Artur
a853cd57ca refactoring 2021-10-09 12:56:00 +00:00
Artur
12f59f2dca refactor 2021-10-09 12:52:20 +00:00
Artur
473f7bee3a refactor 2021-10-09 12:16:37 +00:00
Nikolai Kochetov
c6bce1a4cf Update Native. 2021-10-08 20:21:19 +03:00
Nikolai Kochetov
78e1db209f
Remove more data streams (#29491) 
* Remove more streams.

* Fixing build.

* Fixing build.

* Rename files.

* Fix fast test.

* Fix StorageKafka.

* Try fix kafka test.

* Move createBuffer to KafkaSource ctor.

* Revert "Move createBuffer to KafkaSource ctor."

This reverts commit 81fa94d27e.

* Revert "Try fix kafka test."

This reverts commit 2107e54969.

* Comment some rows in test.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-10-07 11:26:08 +03:00
alexey-milovidov
16ad5953d6
Merge pull request #29783 from azat/tcphandler-progress-race 
Fix data-race between fatal error handler and progress packets
 2021-10-07 01:47:29 +03:00
Filatenkov Artur
545673248a
Merge branch 'master' into add_CORS 2021-10-06 16:09:34 +03:00
Filatenkov Artur
a9e8ba0c00
Correct behaviour with unknown methods (#29057) 
* correct behaviour with unknown methods

* move check on earlier stage

* add test

* correct filter for handler

* Update HTTPServerConnection.cpp

* remove trying to get params

* correct filter for head requests

* Update HTTPHandlerFactory.h

* Update HTTPHandlerFactory.h

* Trigger Build
 2021-10-06 14:43:05 +03:00
Azat Khuzhin
4f5118b85f Fix data-race between fatal error handler and progress packets 
This is not that important, but because of packets overlaps the client
may not render the fatal error.

You will find TSan report in `details`.

<details>

```
WARNING: ThreadSanitizer: data race (pid=13384)
  Write of size 8 at 0x7b24000ad818 by thread T47:
    0 DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:43:15 (clickhouse-tsan+0x16124d61)
    1 DB::TCPHandler::sendProgress() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1602:10 (clickhouse-tsan+0x16124d61)
    2 DB::TCPHandler::runImpl()::$_6::operator()() const obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:318:25 (clickhouse-tsan+0x1612cf54)

  Previous write of size 8 at 0x7b24000ad818 by thread T229:
    0 DB::WriteBuffer::next() obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:43:15 (clickhouse-tsan+0x1612b793)
    1 DB::TCPHandler::sendLogData(DB::Block const&) obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1561:10 (clickhouse-tsan+0x1612b793)
    2 DB::TCPHandler::sendLogs() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1632:9 (clickhouse-tsan+0x161228ab)
    3 DB::TCPHandler::runImpl()::$_1::operator()() const obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:229:62 (clickhouse-tsan+0x1612c4e1)
```

</details>
 2021-10-05 21:21:32 +03:00
Artur Filatenkov
9b1a39fdb9 refactor after move common to base 2021-10-05 17:43:33 +03:00
Filatenkov Artur
60d84df56a
Merge branch 'ClickHouse:master' into add_CORS 2021-10-05 17:27:13 +03:00
Filatenkov Artur
82e6ac8fa2
Update HTTPHandler.cpp 2021-10-05 13:39:18 +03:00
Nikolai Kochetov
e2d07fbdf5
Merge branch 'master' into fix-backward-compatibility-after-21196 2021-10-04 17:35:32 +03:00
Nikolai Kochetov
6169b180cb Fix backward compatibility after #21196. 2021-10-04 17:27:44 +03:00
Maksim Kita
8eda58acfb
Merge pull request #29516 from lingtaolf/feature/support-HSTS 
Support HSTS in Clickhouse HTTP server
 2021-10-04 12:49:51 +03:00
Filatenkov Artur
8df228e6b8
Merge branch 'master' into add_CORS 2021-10-04 10:55:09 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Filatenkov Artur
8959b348a1
Merge branch 'ClickHouse:master' into add_CORS 2021-10-01 14:42:46 +03:00
Azat Khuzhin
e8a90b8ff2 Do not allow to reuse previous credentials in case of inter-server secret 
Before this patch INSERT via Buffer/Kafka may re-use previously set user
for that connection, while this is not correct, it should reset the
user, and use global context.

Note, before [1] there was a fallback to default user, but that code had
been removed, and now it got back.

  [1]: 0159c74f21 ("Secure inter-cluster query execution (with initial_user as current query user) [v3]")

Also note, that context for Buffer table (and others) cannot be changed,
since they don't have any user only profile.

I've tested this patch manually using the following:

    create table dist (key Int) engine=Distributed(test_cluster_two_shards_secure, default, data, key);
    create table buffer (key Int) engine=Buffer(default, dist, 1, 0, 0, 0, 0, 0, 0);
    create table data (key Int) engine=Memory();

    # to start the connection with readonly user
    $ clickhouse-client --user readonly -q 'select * from dist'
    $ clickhouse-client -q 'insert into buffer values (1)'
    # before this patch this produces errors like:
    # 2021.09.27 23:46:48.384920 [ 19474 ] {} <Error> default.dist.DirectoryMonitor: Code: 164. DB::Exception: Received from 127.0.0.2:9000. DB::Exception: readonly: Cannot execute query in readonly mode. Stack trace:

v2: reset the authentication instead of using default user (as suggested by @vitlibar)
v3: reset Session::user and introduce ClientInfo::resetAuthentication (as suggested by @vitlibar)
v4: reset the session every time in interserver mode (suggested by @vitlibar)
 2021-10-01 01:13:15 +03:00
Maksim Kita
7bc6b8fd70
Merge pull request #28604 from azat/safer-http-handler 
Proper buffer flush while handling HTTP queries (avoids std::terminate)
 2021-09-30 16:44:09 +03:00
凌涛
a13d6b0aea Check all requests 2021-09-30 13:40:48 +08:00
凌涛
bd78af435d update code and add document 2021-09-30 04:51:02 +08:00
Artur
fcebf7b985 correct tests 2021-09-29 11:29:24 +00:00
凌涛
c298fba774 Support HSTS in Clickhouse HTTP server 2021-09-29 16:37:48 +08:00
Nikolai Kochetov
78a7665f43 Merge branch 'master' into rewrite-pushing-to-views 2021-09-27 10:56:50 +03:00
Nikolai Kochetov
09232788c0 Fix GRPC tests. 2021-09-27 10:55:33 +03:00
Azat Khuzhin
b3d1bfc67a Send UNKNOWN_DATABASE to the client (via TCP) 
Before this patch:

    $ clickhouse-client --database foo -q 'select 1'
    Code: 32. DB::Exception: Attempt to read after eof: while receiving packet from localhost:9000. (ATTEMPT_TO_READ_AFTER_EOF)

After:

    $ clickhouse-client --database foo -q 'select 1'
    Received exception from server (version 21.11.1):
    Code: 81. DB::Exception: Received from localhost:9000. DB::Exception: Database foo doesn't exist. (UNKNOWN_DATABASE)
    (query: select 1)

Fixes: #26864 (cc @vitlibar)
 2021-09-26 23:45:29 +03:00
Nikolai Kochetov
efbd8e4911 Remove ExceptionHandlingSink. 2021-09-26 17:54:59 +03:00
Alexey Milovidov
cd7f9d981c Remove ya.make 2021-09-25 04:22:54 +03:00
Filatenkov Artur
7bbd08cb5d
Update HTTPHandler.cpp 2021-09-24 15:40:27 +03:00
Artur
e5c49c2e86 Merge branch 'add_CORS' of https://github.com/FArthur-cmd/ClickHouse into add_CORS 2021-09-22 17:15:54 +00:00
Artur
ce4193fe95 small refactoring 2021-09-22 17:10:15 +00:00
Filatenkov Artur
1fe4555dec
Merge branch 'ClickHouse:master' into add_CORS 2021-09-22 19:55:15 +03:00
Artur
2ad7641e7f activate add_http_cors_header by default. 2021-09-22 16:54:36 +00:00
Nikolai Kochetov
eed4e8c754 Fix progress for insert select. 2021-09-22 16:29:58 +03:00
Artur
36223b5f23 update comments for filter 2021-09-22 12:30:55 +00:00
Artur
2cffa98a60 add test and comments in config 2021-09-22 12:22:21 +00:00
Nikolai Kochetov
81bf13a247 Refactor pushing to views. 2021-09-22 13:57:00 +03:00
Artur
c8892ec7a7 add options support 2021-09-22 10:34:48 +00:00
Nikolai Kochetov
db6f1e198c Add cancell callback to completed executor. 2021-09-21 19:37:32 +03:00
Artur
82a849ba8e add options method 2021-09-18 09:36:02 +00:00
Nikolai Kochetov
a8c3b02598 Merge branch 'master' into rewrite-pushing-to-views 2021-09-17 15:38:11 +03:00
Nikolai Kochetov
58bb5fe462 Fix some tests. 2021-09-17 14:40:03 +03:00
Azat Khuzhin
60013cff1a Do not call finalize() for HTTP buffer multiple times 
Before only non-compressed was flushed, and so after compressed will
write to non-compressed and this will trigger multiple finalize() for
raw buffer, fix this, but adding Output::finalize() that take care of
all buffers.

This should fix CI failure [1] and also not only CI failures but also an
assert() that had been added in the previous commit.

    2021-09-05 15:08:43 00429_long_http_bufferization:                                          [ FAIL ] 6.68 sec. - return code 1
    2021-09-05 15:08:43 cmp: EOF on /tmp/clickhouse-test/0_stateless/test_e4xs61/res3 after byte 1048576, in line 4127
    2021-09-05 15:08:43 , result:

    The problem was with the following query:

        curl -q -sS 'http://localhost:8123/?database=default&log_comment=/src/ch/clickhouse/tests/queries/0_stateless/00429_long_http_bufferization.sh&max_block_size=500000&compress=1&buffer_size=1048576&wait_end_of_query=1' -d 'SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 1500000 FORMAT RowBinary' | clickhouse-compressor --decompress | less

    It returns less rows.

  [1]: https://clickhouse-test-reports.s3.yandex.net/28604/1f34fd9a9eaff19fde0199f78b45656ab74f0590/functional_stateless_tests_(release).html#fail1
 2021-09-16 22:36:09 +03:00
Azat Khuzhin
9bb9d5952b Catch write-after-finalize bugs in WriteBufferFromHTTPServerResponse 2021-09-16 22:35:51 +03:00
Azat Khuzhin
7b741c28e4 Do not try to write HTTP response after exception 
CI reports [1]:

    [ 25285 ] {} <Trace> DynamicQueryHandler: Request URI: /?database=test_27&log_comment=/usr/share/clickhouse-test/queries/0_stateless/00429_long_http_bufferization.sh&max_block_size=500000&compress=1&buffer_size=1500000&wait_end_of_query=1
    ...
    [ 25285 ] {ee96eb9c-44cc-4b76-9f30-3cc545a5506f} <Debug> executeQuery: (from [::1]:41578) (comment: /usr/share/clickhouse-test/queries/0_stateless/00429_long_http_bufferization.sh) SELECT greatest(toUInt8(1), toUInt8(intHash64(number))) FROM system.numbers LIMIT 1048576 FORMAT RowBinary
    [ 25285 ] {ee96eb9c-44cc-4b76-9f30-3cc545a5506f} <Information> executeQuery: Read 1500000 rows, 11.44 MiB in 63.924507821 sec., 23465 rows/sec., 183.32 KiB/sec.
    [ 25285 ] {ee96eb9c-44cc-4b76-9f30-3cc545a5506f} <Debug> DynamicQueryHandler: Done processing query
    ...
    [ 455 ] {} <Fatal> BaseDaemon: Code: 24. DB::Exception: Cannot write to ostream at offset 2097152. (CANNOT_WRITE_TO_OSTREAM), Stack trace (when copying this message, always include the lines below):
    [ 455 ] {} <Fatal> BaseDaemon:
    [ 455 ] {} <Fatal> BaseDaemon: 0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:0: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x18fa949b in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:59: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x99f38dc in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 2. ./obj-x86_64-linux-gnu/../src/IO/WriteBufferFromOStream.cpp:0: DB::WriteBufferFromOStream::nextImpl() @ 0x9adf185 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 3. ./obj-x86_64-linux-gnu/../src/IO/BufferBase.h:39: DB::WriteBufferFromHTTPServerResponse::nextImpl() @ 0x15b71ec0 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 4. ./obj-x86_64-linux-gnu/../src/IO/WriteBuffer.h:0: DB::WriteBuffer::write(char const*, unsigned long) @ 0x99be96a in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 5. ./obj-x86_64-linux-gnu/../src/Compression/CompressedWriteBuffer.cpp:36: DB::CompressedWriteBuffer::nextImpl() @ 0x1436ca4d in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 6. ./obj-x86_64-linux-gnu/../src/IO/BufferBase.h:39: DB::CompressedWriteBuffer::~CompressedWriteBuffer() @ 0x1436cd8f in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 7. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2615: std::__1::__shared_ptr_emplace<DB::CompressedWriteBuffer, std::__1::allocator<DB::CompressedWriteBuffer> >::__on_zero_shared() @ 0x15a88ee8 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2518: DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x15b0566b in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Server/HTTP/HTTPServerConnection.cpp:0: DB::HTTPServerConnection::run() @ 0x15b6ba61 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x18ec71a3 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerDispatcher.cpp:114: Poco::Net::TCPServerDispatcher::run() @ 0x18ec7930 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/ScopedLock.h:36: Poco::PooledThread::run() @ 0x1903e042 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Thread.cpp:56: Poco::(anonymous namespace)::RunnableHolder::run() @ 0x1903c4d0 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/SharedPtr.h:277: Poco::ThreadImpl::runnableEntry(void*) @ 0x1903ac48 in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 15. __tsan_thread_start_func @ 0x990a76d in /usr/bin/clickhouse
    [ 455 ] {} <Fatal> BaseDaemon: 16. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    [ 455 ] {} <Fatal> BaseDaemon: 17. clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    [ 455 ] {} <Fatal> BaseDaemon:  (version 21.10.1.7997)
    [ 25878 ] {} <Fatal> BaseDaemon: ########################################
    [ 25878 ] {} <Fatal> BaseDaemon: (version 21.10.1.7997, build id: DB9812D7EA89EB06D2C7243F5C8BD293AA9BA37D) (from thread 25285) (query_id: ee96eb9c-44cc-4b76-9f30-3cc545a5506f) Received signal Aborted (6)
    [ 25878 ] {} <Fatal> BaseDaemon:
    [ 25878 ] {} <Fatal> BaseDaemon: Stack trace: 0x7efe2a35818b 0x7efe2a337859 0x990ff6e 0x13fa707d 0x1bb17614 0x1bb17507 0x999c97b 0x1436cf11 0x15a88ee8 0x15b0566b 0x15b6ba61 0x18ec71a3 0x18ec7930 0x1903e042 0x1903c4d0 0x1903ac48 0x990a76d 0x7efe2a50d609 0x7efe2a434293
    [ 25878 ] {} <Fatal> BaseDaemon: 5. raise @ 0x4618b in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    [ 25878 ] {} <Fatal> BaseDaemon: 6. abort @ 0x25859 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
    [ 450 ] {} <Fatal> Application: Child process was terminated by signal 6.

  [1]: https://clickhouse-test-reports.s3.yandex.net/28604/d6e86dab2754dbea3ed19d9ae72276e7d161aef1/stress_test_(thread).html#fail1
 2021-09-16 22:35:51 +03:00
Azat Khuzhin
230f29ac4e Always flush buffer while handling HTTP queries 
This will avoid possible std::terminate, like in [1]:

  [1]: https://github.com/ClickHouse/ClickHouse/pull/24023#issuecomment-842979875
 2021-09-16 22:35:50 +03:00
Azat Khuzhin
47ad10061b Reduce scope for HTTPHandler::trySendExceptionToClient() 2021-09-16 22:35:50 +03:00
Nikolai Kochetov
341553febd Fix build. 2021-09-16 20:40:42 +03:00
Anton Popov
dfbb1ffec0
Merge pull request #27537 from CurtizJ/merging-20557 
Merging #20557 (Asynchronous inserts)
 2021-09-16 03:08:34 +03:00
Nikita Mikhaylov
6bf5b4cf2d
Merge pull request #28953 from vitlibar/allow-change-server-side-compression-from-client 
gRPC: Allow change server-side compression from client.
 2021-09-16 00:55:06 +03:00
Nikolai Kochetov
e616732743 Small refactoring. 2021-09-15 22:35:48 +03:00
Nikolai Kochetov
b997214620 Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 20:48:18 +03:00
Vitaly Baranov
113ddceabb gRPC: Allow change server-side compression from client. 2021-09-13 01:07:28 +03:00
Nikolai Kochetov
3a9d88fbc9 Fix more tests. 2021-09-10 17:52:24 +03:00
Nikolai Kochetov
f569a3e3f7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-09 20:30:23 +03:00
Anton Popov
8e5b3b2f6c Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-09 15:12:34 +03:00
Vitaly Baranov
70c6623036
Merge branch 'master' into governance/session_log 2021-09-07 10:12:54 +03:00
Vitaly Baranov
bcc31f1f3e Remove unnecessary changes. 2021-09-07 01:37:28 +03:00
Anton Popov
59128412ea Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-03 22:17:11 +03:00
Nikolai Kochetov
66a76ab70f Rewrite PushingToViewsBlockOutputStream part 6 2021-09-03 20:29:36 +03:00
Nikolai Kochetov
358babc31b Rewrite PushingToViewsBlockOutputStream part 5. 2021-09-02 14:29:20 +03:00
alesapin
a8003e444b Review fixes 2021-09-02 12:20:46 +03:00
Anton Popov
5e421ab272 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-02 02:42:50 +03:00
Vasily Nemkov
c902afddde Added system.session_log table 
Which logs all the info about LogIn, LogOut and LogIn Failure events.
Additional info that is logged:
- User name
- event type (LogIn, LogOut, LoginFailure)
- Event date\time\time with microseconds
- authentication type (same as for IDENTIFIED BY of CREATE USER statement)
- array of active settings profiles upon login
- array of active roles upon login
- array of changed settings with corresponding values
- client address and port
- interface (TCP\HTTP\MySQL\PostgreSQL, etc.)
- client info (name, version info)
- optional LoginFailure reason text message.

Added some tests to verify that events are properly saved with all necessary info via following interfaces:
- TCP
- HTTP
- MySQL

Known limitations
- Not tested against named HTTP sessions, PostgreSQL and gRPC, hence those are not guaranteed to work 100% properly.
 2021-08-30 18:28:28 +03:00
alesapin
47b995467e
Merge pull request #28080 from ClickHouse/some_renames_in_keeper 
Clickhouse-keeper: renames and comments
 2021-08-26 10:16:21 +03:00
Anton Popov
6d3274c22c Merge remote-tracking branch 'upstream/master' into HEAD 2021-08-26 03:31:46 +03:00
Vitaly Baranov
64f69462c6
Merge pull request #27921 from vitlibar/add-tests-for-field-init 
Allow implicit cast bool to Field
 2021-08-24 17:15:44 +03:00
alesapin
95cf0634dc Clickhouse-keeper: renames and comments 2021-08-24 15:30:31 +03:00
Nikita Mikhaylov
fb6462d70a
Merge pull request #27886 from vitlibar/add-separate-constants-for-interfaces 
Add separate constants for interfaces LOCAL and TCP_INTERSERVER.
 2021-08-23 16:25:01 +03:00
Vitaly Baranov
ff1fad8521 Use setSetting() with implicit conversion true/false -> Field. 2021-08-23 08:57:00 +03:00
Alexey Milovidov
c3c31e2895 Move function to appropriate place to make code more readable 2021-08-20 12:56:39 +03:00
Vitaly Baranov
3ab1177d7e Add separate constants for interface LOCAL and TCP_INTERSERVER. 2021-08-19 23:04:36 +03:00
Vitaly Baranov
fabd7193bd Code cleanups and improvements. 2021-08-18 14:24:52 +03:00
Vasily Nemkov
51ffc33457 Introduce sessions. 
This is required to add system.session_log table.
 2021-08-18 14:24:52 +03:00
hermano
14e251f158 Fixes style 2021-08-16 18:54:40 -03:00
hermano
5b520fbe0f Preloads query parameters 2021-08-16 07:58:52 -03:00
hermano
3366fc7c7e Enables query parameters in request body 2021-08-15 15:46:31 -03:00
Raúl Marín
a451bf6eac Remove unused code 2021-08-12 11:30:01 +02:00
Raúl Marín
f6788fc660 Mysql handler: Move format check to the handler 2021-08-12 11:29:50 +02:00
tavplubix
e7290fb134
Merge pull request #27385 from BraulioVM/fix/27171 
Do not fail if HTTP port is not set and user tries to send request to TCP port
 2021-08-09 15:30:58 +03:00
Vitaly Baranov
5a7fe51532
Merge pull request #27431 from vitlibar/grpc-protocol-bytes-instead-of-strings 
Use bytes instead of strings in the GRPC protocol.
 2021-08-09 09:51:24 +03:00
alexey-milovidov
d2efe3869f
Merge pull request #26984 from azat/keeper-server-fix 
Remove test_keeper_server usage (for {operation/session}_timeout_ms)
 2021-08-08 04:04:42 +03:00
Braulio Valdivielso
a833d1cf9e fixup! change http error response 2021-08-07 19:57:19 +01:00
Braulio Valdivielso
6189757fed change http error response 
As reported in #27171, the logic that generates the HTTP error
response didn't work in case the configuration hadn't specified a
`http_port`. The new logic checks whether the `http_port` was set in
the first place.
 2021-08-06 23:32:37 +01:00
Anton Popov
3a0d4807a5 Merge branch 'async-insert' of git://github.com/abyss7/ClickHouse into merging-20557 2021-08-06 16:20:02 +03:00
Vitaly Baranov
9a40ce87e9 Use bytes instead of strings in the GRPC protocol. 2021-08-06 13:55:49 +03:00
mergify[bot]
91931feed7
Merge branch 'master' into dynamic-tls-v20 2021-08-05 19:24:26 +00:00
Vitaly Baranov
40a3c8281a Fix synchronization in GRPCServer 2021-08-03 09:44:47 +03:00
Azat Khuzhin
1feee18455 Remove test_keeper_server usage (for {operation/session}_timeout_ms) 2021-07-29 21:24:32 +03:00
Nikolai Kochetov
9b5a816b43 Merge branch 'master' into output-streams-to-processors 2021-07-26 18:03:11 +03:00
Nikolai Kochetov
9c92f43359 Update storages. 2021-07-23 22:33:59 +03:00
Nikolai Kochetov
3c17a62686
Merge pull request #26590 from ClickHouse/remove-some-more-streams 
Remove some streams.
 2021-07-22 21:28:50 +03:00
Nikolai Kochetov
3ed3f7a9f7 Fix integration tests. 2021-07-22 13:38:22 +03:00
Nikolai Kochetov
5ffd99dfd4
Merge pull request #25674 from amosbird/distributedreturnconnection 
Drain connection asynchronously
 2021-07-22 11:04:49 +03:00
Nikolai Kochetov
f38de35b14 Rename some constants. 2021-07-21 19:13:17 +03:00
Nikolai Kochetov
179ec05a72 Remove some streams. 2021-07-20 21:18:43 +03:00
anneji-dev
d8f45fbe13
Set client query kind for mysql and postgresql handler (#26498) 
* client query kind is not set for mysql and postgresql

* Update MySQLHandler.cpp

* Update PostgreSQLHandler.cpp

Co-authored-by: anneji <anneji@tencent.com>
 2021-07-20 09:23:58 +03:00
Amos Bird
dbfb699690
Asynchronously drain connections. 2021-07-19 21:53:29 +08:00
Vitaly Baranov
4f1926550b
Merge pull request #26429 from vitlibar/remove-mysql-wire-context 
Remove MySQLWireContext
 2021-07-19 12:21:24 +03:00
Alexey Milovidov
261a220227 Remove some code 2021-07-17 21:06:46 +03:00
alexey-milovidov
38dfe1fc0a
Merge pull request #26172 from ClickHouse/exception_id 
Add error id to exceptions
 2021-07-17 02:07:17 +03:00
Vitaly Baranov
0f8b196682 Remove MySQLWireContext. 2021-07-16 22:21:20 +03:00
Ivan Lezhankin
69e7f082fe Merge remote-tracking branch 'upstream/master' into async-insert 2021-07-13 17:15:32 +03:00
Nikita Mikhaylov
c255f152aa
Merge pull request #26051 from ClickHouse/fix_21184 
Fix sequence_id in MySQL protocol
 2021-07-13 16:03:25 +03:00
mergify[bot]
b7ae4e4ced
Merge branch 'master' into async-insert 2021-07-12 09:21:37 +00:00
Alexey Milovidov
3887e82dac Merge branch 'exception_id' of github.com:yandex/ClickHouse into exception_id 2021-07-12 03:18:39 +03:00
Alexey Milovidov
c4f34689ec Merge branch 'master' into exception_id 2021-07-11 22:12:06 +03:00
Vladimir
f120bd3d09
Directly pass getExceptionMessage() to LOG_ERROR in TCPHandler.cpp 2021-07-10 13:30:38 +03:00
Alexey Milovidov
588c31d9f5 Better messages 2021-07-10 11:18:06 +03:00
Alexey Milovidov
21774d6700 Correct misleading comments 2021-07-10 06:01:24 +03:00
Alexey Milovidov
9b299fbb56 Simplification 2021-07-10 05:57:44 +03:00
Alexey Milovidov
1f91c65aca Add error id to exceptions 2021-07-10 05:43:53 +03:00
vdimir
c01f4588b4
Validate prometheus metric name with regex 2021-07-09 19:12:26 +03:00
Alexey Milovidov
12b2a77ba3 More normal 2021-07-09 05:27:01 +03:00
Alexey Milovidov
f73605365c Fix build 2021-07-09 03:13:04 +03:00
Alexander Tokmakov
1a470fb777 fix sequence_id in MySQL protocol 2021-07-07 20:03:28 +03:00
Alexey Milovidov
7c218b51e6 Fix build 2021-07-07 03:13:37 +03:00
Alexey Milovidov
0b8729370c More twiddling 2021-07-07 02:52:24 +03:00
Alexey Milovidov
3351b135e7 More twiddling 2021-07-07 02:34:43 +03:00
Alexey Milovidov
e52680479e Twiddling 2021-07-07 02:15:30 +03:00
alexey-milovidov
6007ea9423
Update CertReloader.cpp 2021-07-07 01:20:14 +03:00
alexey-milovidov
f3c8821919
Update CertReloader.cpp 2021-07-07 01:19:26 +03:00
alexey-milovidov
019edee013
Update CertReloader.h 2021-07-07 01:19:05 +03:00
alexey-milovidov
e0b98cfa76
Update CertReloader.h 2021-07-07 01:18:41 +03:00
alexey-milovidov
303c38cb6f
Merge branch 'master' into dynamic-tls-v20 2021-07-07 01:13:03 +03:00
kssenii
0dda5b67c4 Fix set SQL_SELECT_LIMIT 2021-07-04 16:17:59 +03:00
Ivan Lezhankin
37439a8af6 Fix typos 2021-06-29 01:51:01 +03:00
Ivan Lezhankin
09b3db38f0 Don't create input stream for native data 2021-06-28 23:43:37 +03:00
Ivan Lezhankin
1bb41593d1 Merge remote-tracking branch 'upstream/master' into async-insert 2021-06-28 22:55:56 +03:00
sundy-li
d423d07a87 [mysql] fix mysql select user() return empty 2021-06-25 10:20:22 +08:00
Ivan Lezhankin
37365589ed Merge branch 'master' into async-insert 2021-06-17 16:57:45 +03:00
alexey-milovidov
04be5437d9
Merge pull request #25296 from abyss7/http-issues 
Add settings for HTTP header limitations
 2021-06-17 01:50:48 +03:00
alexey-milovidov
16931a661c
Update HTTPServerRequest.h 2021-06-17 01:50:06 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Ivan Lezhankin
ba08a580f8 Add test 2021-06-16 17:33:14 +03:00
Ivan Lezhankin
b182d87d9c Add settings for HTTP header limitations 2021-06-15 17:33:46 +03:00
Amos Bird
7d9a1106b8
Add initial_query_start_time to query log 2021-06-11 22:24:53 +08:00
Ivan Lezhankin
973cea73b7 Merge branch 'master' into async-insert 2021-06-08 15:10:30 +03:00
Alexey Milovidov
17962459f5 Merge branch 'master' into issue-16775 2021-06-06 02:18:28 +03:00
Ivan Lezhankin
791cbcf0b1 Merge remote-tracking branch 'upstream/master' into async-insert 2021-06-02 18:50:37 +03:00
Ivan Lezhankin
84c23dc060 Merge branch 'master' into async-insert 2021-06-02 18:06:21 +03:00
Yatsishin Ilya
d51a6a2c1f Merge remote-tracking branch 'origin' into integration-2 2021-06-01 17:23:01 +03:00
Nikolai Kochetov
c673eb2040 Use mutable context for TCPHandler::customizeContext 2021-06-01 12:27:05 +03:00
Yatsishin Ilya
9b3cc2eefb Merge remote-tracking branch 'origin' into integration-2 2021-06-01 09:08:42 +03:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs 
Poco::File to std::filesystem
 2021-05-31 23:17:02 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Yatsishin Ilya
b683f71ae6 Merge remote-tracking branch 'origin' into integration-2 2021-05-31 07:03:27 +03:00
kssenii
69816e6eff Fix checks 2021-05-30 15:44:58 +03:00
kssenii
db9f4762eb Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-29 01:15:43 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
Ivan
ce2a809773
Consider last CRLF as part of boundary (#24399) 
* Consider last CRLF as part of boundary
* Update HTMLForm.cpp
* Don't use xxd
 2021-05-28 14:59:11 +03:00
kssenii
31107816bb fs::absolute to fs::canonical 2021-05-24 20:37:54 +03:00
Yatsishin Ilya
c32cea4dd2 Merge remote-tracking branch 'origin' into integration-2 2021-05-24 09:59:16 +03:00
Alexey Milovidov
273226de32 Remove string parameter for Density 2021-05-24 06:43:25 +03:00
Alexey Milovidov
6f70feed2f Merge branch 'master' into issue-16775 2021-05-24 05:56:57 +03:00
kssenii
bbf56e35ac Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-23 10:54:48 +03:00
Alexey Milovidov
40d4f0678f Remove overload (harmful) 2021-05-23 04:25:06 +03:00
kssenii
1c43b333ea Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-22 21:26:48 +03:00
alexey-milovidov
1ed30f7246
Merge branch 'master' into issue-16775 2021-05-22 17:14:23 +03:00
alesapin
e4ce6faab6 Fix build 2021-05-22 15:32:53 +03:00
alesapin
8c63f0f8e2 Fix race condition on keeper shutdown 2021-05-22 10:46:12 +03:00
Yatsishin Ilya
fe630a0139 Merge remote-tracking branch 'origin/master' into integration-2 2021-05-21 17:57:44 +03:00
Ivan Lezhankin
33f816d28d Allow empty HTTP headers 2021-05-19 16:20:23 +03:00
kssenii
5f083b779d A little better, fix checks 2021-05-18 14:25:19 +03:00
kssenii
393b026f2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-11 16:49:23 +03:00
Yatsishin Ilya
9d159072ef mERGe remote-tracking branch 'origin' into integration-2 2021-05-11 13:32:44 +03:00