Commit Graph

483 Commits

Author SHA1 Message Date
pingyu
08126030d6 wip #14893 2021-02-26 00:44:01 +08:00
Ivan
414f470c79
Make Poco HTTP Server zero-copy again (#19516)
* Refactoring: part 1

* Refactoring: part 2

* Handle request using ReadBuffer interface

* Struggles with ReadBuffer's

* Fix URI parsing

* Implement parsing of multipart/form-data

* Check HTTP_LENGTH_REQUIRED before eof() or will hang

* Fix HTTPChunkedReadBuffer

* Fix build and style

* Fix test

* Resist double-eof

* Fix arcadian build
2021-02-19 15:51:26 +03:00
Nikolai Kochetov
d582160af4 Merge branch 'master' into filter-push-down 2021-02-18 10:58:27 +03:00
Nikolai Kochetov
ec4dafaa5f Fix build. 2021-02-17 22:19:39 +03:00
alesapin
5f32f65211 Merge branch 'master' into in_memory_raft 2021-02-11 23:09:29 +03:00
Nikolai Kochetov
77cb68c3e7 Fixing build. 2021-02-08 12:03:04 +03:00
Nicolae Vartolomei
479b45d772 Fix linker flags for shared linking on macOS
This combination now works: `-DUSE_STATIC_LIBRARIES=0 -DSPLIT_SHARED_LIBRARIES=1`.

Without `SPLIT_SHARED_LIBRARIES` it is still failing.
2021-02-07 14:58:52 +00:00
alesapin
57c9b6c864 Fix build without nuraft 2021-02-01 16:18:17 +03:00
alesapin
0aca40d8cb Merge branch 'master' into in_memory_raft 2021-02-01 14:30:11 +03:00
Nikolai Kochetov
e9f893d9b0 Move query plan optimixations into separate files. 2021-01-27 16:57:26 +03:00
alesapin
4886c79625 Merge branch 'master' into in_memory_raft 2021-01-18 16:18:05 +03:00
Alexey Milovidov
7705420220 Better 2021-01-16 00:23:58 +03:00
Alexey Milovidov
67f207d8ca More safe connection string 2021-01-16 00:18:51 +03:00
alesapin
d7e6c8393f Some useless code 2021-01-13 13:32:20 +03:00
kssenii
00a37404ca Add postgresql database engine 2020-12-27 12:26:29 +00:00
kssenii
2d8e35b4bd Add libpq and libpqxx 2020-12-27 12:02:41 +00:00
alesapin
3c104035b1 Add NuRaft library 2020-12-24 23:10:47 +03:00
Nikolai Kochetov
af7f5c9518
Merge pull request #17868 from ClickHouse/async-read-from-socket
Async read from socket
2020-12-23 12:20:42 +03:00
Alexey Milovidov
bf2df558d4 Merge branch 'master' into FawnD2-switch-upstream-for-arrow-submodule 2020-12-20 09:38:46 +03:00
alexey-milovidov
41b2503d1a
Merge pull request #18114 from azat/20.13-gcc10-sanitizers
gcc10 sanitizers support
2020-12-19 12:54:36 +03:00
Nikolai Kochetov
01c8d5dccc Use replace fiber to callback in ReadBufferFromPocoSocket 2020-12-18 17:24:44 +03:00
Azat Khuzhin
995c3da1c3 Disable memory tracking for gcc TSAN/ASAN
Since libtsan.a provides new/delete overrides too:

    ld.lld: error: duplicate symbol: operator new[](unsigned long, std::nothrow_t const&)
    >>> defined at new_delete.cpp:64 (../src/Common/new_delete.cpp:64)
    >>>            new_delete.cpp.o:(operator new[](unsigned long, std::nothrow_t const&)) in archive src/libclickhouse_new_delete.a
    >>> defined at tsan_new_delete.o:(.text+0xE0) in archive /usr/lib/gcc/x86_64-linux-gnu/10/libtsan.a

v2: Fix order of src vs programs for clickhouse_new_delete target detection
2020-12-16 21:31:03 +03:00
Ilya Golshtein
7e97814859 cleanup, fixes, new submodules, ShellCommand, WriteBufferFromString 2020-12-16 14:48:33 +03:00
Nikolai Kochetov
c4e80ccfc5 Try fix special builds. 2020-12-15 11:27:42 +03:00
Nikolai Kochetov
8de5cd5bc7 Merge branch 'master' into async-read-from-socket 2020-12-14 17:45:38 +03:00
Azat Khuzhin
2a814afd85 build: re-enable jobs limits 2020-12-11 01:32:38 +03:00
Nikolai Kochetov
32b38f389e Merge branch 'master' into async-read-from-socket 2020-12-09 17:15:36 +03:00
Nikolai Kochetov
f9b45b83a9 Try fix CMakeLists.txt 2020-12-09 17:11:20 +03:00
Maksim Kita
5b57c59a3b Updated fast_float integration 2020-12-06 23:37:55 +03:00
Maksim Kita
b6bfb1cf20 Use fast_float by default 2020-12-06 23:37:36 +03:00
Maksim Kita
87c50602bc Fast float updated implementation 2020-12-06 23:37:35 +03:00
Maksim Kita
80d66b8868 FastFloat library integration 2020-12-06 23:37:35 +03:00
Maksim Kita
977da3d052 Removed ryu 2020-12-06 15:39:39 +03:00
Maksim Kita
64549702a2 DragonBox integration 2020-12-06 00:32:03 +03:00
Ivan
315ff4f0d9
ANTLR4 Grammar for ClickHouse and new parser (#11298) 2020-12-04 05:15:44 +03:00
FawnD2
26488467f8 Merge branch 'master' into switch-upstream-for-arrow-submodule 2020-11-29 15:53:28 +03:00
a.palagashvili
502355922d added errors checking, removed redundant variables 2020-11-29 15:14:42 +03:00
Nikolai Kochetov
729272391f
Merge branch 'master' into ip-dict-no-trie 2020-11-25 23:07:19 +03:00
Vitaly Baranov
44717797c9 Fix code style and compilation. 2020-11-24 17:53:34 +03:00
vdimir
a277a5bb16
Rename TrieDictionary -> IPAddressDictionary 2020-11-21 21:38:10 +03:00
vdimir
e2fac19686
Remove libbtrie 2020-11-21 21:31:15 +03:00
a.palagashvili
6f5390cc70 introduce zstd compression 2020-11-17 19:02:10 +03:00
sundyli
5ef809931e
Merge branch 'master' into storage-rocksdb 2020-11-12 08:26:27 +08:00
a.palagashvili
124ef2f5fa added and successfully passed tests for content encoding and file() function, formatted code 2020-11-11 04:50:56 +03:00
Alexey Milovidov
630aa9b0c3 Rename Rocksdb to RocksDB 2020-11-11 09:08:53 +08:00
sundy-li
c7617173c4 add sync_file_range for glibc-compatibility 2020-11-11 09:08:52 +08:00
root
e9de5b6ad4 Add StorageEmbeddedRocksdb Engine 2020-11-11 09:08:52 +08:00
a.palagashvili
f999ea2a6f renamed files, added new library, changed error codes, added tests for file() function 2020-11-10 01:52:22 +03:00
Danila Kutenin
3c6794bf37 Add Floyd-Rivest selection algorithm instead of std::partial_sort 2020-11-09 20:22:05 +03:00
FawnD2
65e4fa555b Remove build branch with internal parquet native cmake 2020-11-07 19:49:36 +03:00
a.palagashvili
805bfd286c resolved git modules 2020-11-02 15:16:49 +03:00
a.palagashvili
be2b002ff4 fixed cmake for building, added test for compression method, added basic code for encoding 2020-11-01 21:40:05 +03:00
Azat Khuzhin
60aae56266 Link dbms w/ atomic to fix undefined reference in unbundled build
This will fix undefined reference for __atomic_load under clang10 and
unbundled build (libstdc++ shared linkage):

    clickhouse-server: symbol lookup error: /src/ch/clickhouse/.cmake/src/libclickhouse_disksd.so: undefined symbol: __atomic_load

From [1]:

  "libatomic (GNU)

   libgcc_s does not provide an implementation of an atomics library.
   Instead, GCC’s libatomic library can be used to supply these when using
   libgcc_s.

   Note

   Clang does not currently automatically link against libatomic when using
   libgcc_s. You may need to manually add -latomic to support this
   configuration when using non-native atomic operations (if you see link
   errors referring to __atomic_* functions)."

  [1]: https://clang.llvm.org/docs/Toolchain.html
2020-10-21 22:33:53 +03:00
alexey-milovidov
1474b80ecc
Merge pull request #15008 from ClickHouse/clang-tidy-11-without-enabling
Fix terribly wrong code
2020-09-20 02:00:54 +03:00
Alexey Milovidov
059646743b Remove OpenCL 2020-09-19 17:20:00 +03:00
Alexey Milovidov
ef5363b87f Fix terribly wrong code 2020-09-19 02:05:13 +03:00
Alexey Milovidov
f9361acb88 Disable in shared libraries mode 2020-09-14 21:00:45 +03:00
Konstantin Podshumok
17c46faaa7
cmake: fix parquet/arrow variables
squashed:

- sync FindArrow.cmake and FindParquet.cmake with arrow repo
- unbundled: add arrow to dbms link libraries
- cmake: fix-up unbundled Arrow support

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:41 +03:00
alexey-milovidov
fb0e68f808
Merge pull request #12771 from arenadata/ADQM-109
krb5 + cyrus-sasl + kerberized kafka
2020-08-20 20:13:01 +03:00
zhang2014
0162c39838 ISSUES-4006 split mysql protocol 2020-08-13 13:02:16 +08:00
zhang2014
5d569f56a7 ISSUES-4006 support convert mysql create to clickhouse create 2020-08-04 10:26:23 +08:00
zhang2014
c53a48fe01 ISSUES-4006 support parse MySQL create query 2020-08-04 10:26:23 +08:00
Ilya Golshtein
d93f0c7548 initial - kerberized kafka works 2020-07-31 13:49:07 +03:00
Vxider
c78dd2cd79 Merged with master 2020-07-27 18:11:26 +08:00
Vitaly Baranov
000b197ad1
Merge pull request #11234 from traceon/ldap-per-user-authentication
Add LDAP authentication support
2020-07-16 13:17:21 +03:00
Vxider
61b15e6f6d merged with master 2020-07-15 22:38:50 +08:00
alesapin
a9824fd650 Better cmake and less libraries 2020-07-10 22:08:18 +03:00
alesapin
df176c633e Disable msgpack 2020-07-10 21:27:10 +03:00
Denis Glazachev
5a4762bb8a
Merge branch 'master' into ldap-per-user-authentication 2020-07-09 14:50:02 +04:00
Nikita Mikhaylov
1918d1d417
Fix ORC build (#12258)
* first try

* change submodule

* Update .gitmodules

* include build directory

* Update .gitmodules

Co-authored-by: Nikita Mikhailov <jakalletti@jakalletti-build.sas.yp-c.yandex.net>
2020-07-09 03:49:21 +04:00
Denis Glazachev
9effacfbc1 Merge branch 'master' into ldap-per-user-authentication
* master: (1102 commits)
  Update README.md
  Update README.md
  Update README.md
  Update index.md
  [docs] add intrdocution for statements page (#12189)
  Revert "Run perf tests with memory sampling (for allocations >1M)"
  Sanitize LINK_LIBRARIES property for the directories (#12160)
  [docs] refactor Domains overview (#12186)
  DOCS-647: toStartOfSecond (#12190)
  [docs] add intrdocution for commercial page (#12187)
  DOCSUP-1348 Russian translation for new functions (#133) (#12194)
  changelog fixes
  Update index.md (#12191)
  Update zh kafka.md title (#12192)
  Added test for #3767
  style fix for #12152
  Tests for fixed issues #10846 and #7347
  changelog fixes
  [docs] introduction for special table engines (#12170)
  [docs] introduction for third-party interfaces (#12175)
  ...

# Conflicts:
#	src/Access/ya.make
#	src/Common/ErrorCodes.cpp
2020-07-08 00:42:09 +04:00
Nikita Mikhaylov
53522c728b
Merge pull request #11662 from Avogar/orc_output_format
Add ORCBlockOutputFormat
2020-07-06 18:51:35 +04:00
alesapin
64583ceb22 Merge branch 'master' into add-storage-rabbitmq-read-only 2020-07-02 16:08:29 +03:00
Nikita Mikhaylov
33e627777b Merge branch 'master' of github.com:ClickHouse/ClickHouse into Avogar_orc_output_format 2020-06-29 17:25:33 +03:00
Avogar
81d459a5a8 Small update ORCBlockOutputFormat and add ORC output format to performance test. 2020-06-26 23:17:28 +03:00
alesapin
11f88340a5 Merge branch 'mongo' of https://github.com/ageraab/ClickHouse into storage_mongodb 2020-06-26 16:03:06 +03:00
alesapin
3fc65b3269 Merge branch 'master' into kssenii-rabbit-mq 2020-06-24 20:14:28 +03:00
Nikita Mikhaylov
717f63923a
Merge branch 'master' into orc_output_format 2020-06-23 15:07:05 +04:00
Denis Glazachev
6c1b2a4831 Merge branch 'master' into ldap-per-user-authentication
* master: (114 commits)
  Update alter.md (#11823)
  Update PODArray.h
  Added a test for #6549
  Update create.md
  Update create.md
  replacingmergetree translate into "替换合并树“ better (#11814)
  Whitespace
  Fix style
  Added a test
  Fix another 10% of bad code #11529
  Update join.md (#11798)
  Update memory.md (#11800)
  Whitespace
  Check for tests with "fail" in their names
  Fix 10% of bad code #11756
  Update PODArray.h
  avoid duplicate short system name
  adapt recent result as well
  [website] tune benchmark pages styling + add extra checks
  Added MSan suppression for Hyperscan
  ...

# Conflicts:
#	src/Common/ErrorCodes.cpp
2020-06-20 22:01:12 +04:00
Nikolai Kochetov
4d70cc34c1 Merge branch 'master' into query-plan 2020-06-20 10:53:48 +03:00
Vxider
add49518f3 Merged with master 2020-06-19 23:12:25 +08:00
Nikolai Kochetov
cda4de40d7 Merge branch 'master' into query-plan 2020-06-19 11:29:15 +03:00
Denis Glazachev
5db60202b6 Merge branch 'master' into ldap-per-user-authentication 2020-06-19 00:11:08 +04:00
Azat Khuzhin
2bd05baa7a Move AggregateFunctionCount out from clickhouse_aggregate_functions to dbms
As suggested by @alexey-milovidov
2020-06-18 11:22:40 +03:00
Azat Khuzhin
65c17f0c2c Fix usage of static dbms library (add clickhouse_aggregate_functions)
The following error reported:

    FAILED: src/Interpreters/tests/hash_map
    ...
    ld.lld: error: undefined symbol: typeinfo for DB::AggregateFunctionCount
    >>> referenced by typeinfo:308 (../contrib/libcxx/include/typeinfo:308)
    >>>               InterpreterSelectQuery.cpp.o:(std::__1::enable_if<is_pointer_v<DB::AggregateFunctionCount*>, DB::AggregateFunctionCount*>::type typeid_cast<DB::AggregateFunctionCount*, DB::IAggregateFunction>(DB::IAggregateFunction*)) in archive src/libdbms.a
    collect2: error: ld returned 1 exit status

And also this one with gold linker:

  FAILED: programs/clickhouse-odbc-bridge
  ...  src/AggregateFunctions/libclickhouse_aggregate_functions.a ... src/libdbms.a ...
  ../contrib/libcxx/include/typeinfo:308: error: undefined reference to 'typeinfo for DB::AggregateFunctionCount'

Due to order?

Introduced by 394fb64a9cda376ca8dfa8bac08a4fbcfa9bf3bf/#11661 I guess.
2020-06-18 01:53:14 +03:00
Avogar
7780a74bd8 Update ORCBlockOutputFormat and add orc include directory in include path 2020-06-17 22:32:11 +03:00
Nikolai Kochetov
03e0a5ddbc Merge branch 'master' into query-plan 2020-06-15 23:38:35 +03:00
Alexey Milovidov
394fb64a9c Better way of implementation 2020-06-14 20:42:11 +03:00
kssenii
18820814f5 Merge 2020-06-10 23:25:33 +00:00
Denis Glazachev
9e3a28a6b8 Merge branch 'master' into ldap-per-user-authentication
* master: (414 commits)
  Update file.md
  Update merge.md
  Update dictionary.md
  Update external-data.md
  Update distributed.md
  Update null.md
  Update set.md
  Update join.md
  Update url.md
  Update view.md
  Update materializedview.md
  Update memory.md
  Update buffer.md
  Update generate.md
  removed a sentence about global lock during rename (#11577)
  greatCircleAngle en translation (#11584)
  Update configuration-files.md
  try fix flacky test
  Update why.html
  Update rich.html
  ...

# Conflicts:
#	src/Common/ErrorCodes.cpp
#	utils/ci/jobs/quick-build/run.sh
2020-06-11 03:06:17 +04:00
Alexander Tokmakov
6bb68329f8 Merge branch 'master' into merging_external_source_cassandra 2020-06-09 19:51:14 +03:00
Ivan
47ad338cb2
Refactor CMake build files (#11390)
* Get rid of lib_name.cmake
* Refactor Boost and HyperScan libraries
* Refactor lz4
* Fix build with xxHash
2020-06-09 13:54:49 +03:00
Vxider
ab160a3d92 Merged with master 2020-06-09 16:48:04 +08:00
Nikolai Kochetov
108575c8ad Added IQueryPlanStep. 2020-06-05 18:38:03 +03:00
Denis Glazachev
c61cbe8e98 Merge branch 'master' into ldap-per-user-authentication
* master: (114 commits)
  Update PushingToViewsBlockOutputStream.cpp
  Update PushingToViewsBlockOutputStream.cpp
  make clang-10 happy
  Fix sync_async test (remove timeout)
  CLICKHOUSEDOCS-631: temporary_files_codec, join_on_disk_max_files_to_merge settings. (#11242)
  Suppress output of cancelled queries in clickhouse-client #9473
  Better log messages in ConfigReloader
  fix select from StorageJoin
  Fix unit tests under MSan
  Added test.
  Fix build.
  Fix arguments for AggregateFunctionQuantile/
  Update style.md
  Add a guide on error messages.
  Report dictionary name on dictionary load errors.
  more types in ASOF JOIN (#11301)
  Fix part_log test
  Update test.
  Add perftest.
  Parallel processing for PushingToViewsBlockOutputStream::writeSuffix
  ...
2020-06-02 12:42:29 +04:00
kssenii
d3b069e37b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into add-storage-rabbitmq-read-only 2020-06-01 18:25:36 +00:00
Alexander Tokmakov
62761eddcc Merge branch 'master' into merging_external_source_cassandra 2020-06-01 14:44:46 +03:00
alexey-milovidov
256f84ddc2
Merge pull request #11230 from Jokser/s3-poco-http-client
S3 Poco Http Client
2020-06-01 01:20:21 +03:00
kssenii
e80b405359 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into add-storage-rabbitmq-read-only 2020-05-31 10:30:53 +00:00
Alexander Tokmakov
8921041706 Merge branch 'master' into merging_external_source_cassandra 2020-05-29 18:20:39 +03:00
Denis Glazachev
d9ca9cd9b2 Compilation fix
Typo fix
2020-05-29 14:00:12 +04:00
Ivan
5b858de374
Split programs/server into actual program and library (#11186)
* Split programs/server into actual program and library
2020-05-27 20:52:52 +03:00
Pavel Kovalenko
2b07f69e57 S3 Poco HTTP Client WIP. 2020-05-27 18:02:25 +03:00
bobrovskij artemij
dbf21183d2 Merge branch 'mongo' of https://github.com/ageraab/ClickHouse into mongo 2020-05-26 14:18:56 +03:00
alesapin
58711c924b
Fix cmake 2020-05-26 13:19:59 +03:00
Alexander Tokmakov
e6ca09e134 fix build 2020-05-22 17:19:33 +03:00
Alexander Tokmakov
4b30b3168c try fix build 2020-05-20 23:26:39 +03:00
kssenii
a055e33087 Add libevent & AMQP-CPP libraries 2020-05-20 09:08:27 +03:00
alexey-milovidov
2e2ef08619
Merge pull request #10967 from myrrc/gtest-fix
gtest macro find fix
2020-05-17 03:49:01 +03:00
myrrc
a56aba0211 gtest macro find fix 2020-05-17 02:38:30 +03:00
alexey-milovidov
5898f611da Merge pull request #10861 from traceon/contrib-openldap
Add OpenLDAP third-party library integration (system or contrib/build)
2020-05-15 23:13:23 +03:00
Ri
fc7afaa639
Bitonic sort on GPU (OpenCL) (#10232) 2020-05-15 03:01:14 +03:00
Denis Glazachev
36cb24377b Merge branch 'master' into contrib-openldap
* master: (204 commits)
  DOCS-289: randConstant (#10838)
  Update performance_comparison.md
  fix docs build
  Adjust ld+json images, remove feather icons (#10843)
  [docs] faster local/debug build (#10840)
  CLICKHOUSE-4862: some markdown fixes @ ru mergetree.md (#10835)
  CLICKHOUSE-4862: fix link (#10836)
  Some fixes at ru merge-tree-settings.md (#10837)
  Fix MSan failure in cache dictionary
  Fix double whitespace
  Additional assert in ColumnVector
  Fix paths in compiler error messages #10434
  Fix "Arcadia" build
  Fix style check in Block::sortColumns() (sigh)
  Sort iterators to avoid extra std::string creation in Block::sortColumns()
  Optimize Block::sortColumns()
  Better DNS exception message
  Fix logical error in convertFieldToType
  Make order of columns strict in Block::sortColumns()
  Add a test for INSERT into Buffer() with different order of columns (via MV)
  ...
2020-05-13 00:29:09 +04:00
Denis Glazachev
acd8cfc5d2 Added contrib/openldap submodule
Added OpenLDAP find/detection cmake scripts
Added integration for OpenLDAP Linux and Darwin x86_64 platforms (following OpenSSL integration approach)
2020-05-12 23:19:15 +04:00
Alexey Milovidov
a576a4fbcd Enable extra warnings for base, utils, programs 2020-05-10 01:59:34 +03:00
Alexey Milovidov
fb32dbe6ee Enable more warnings 2020-05-10 00:44:33 +03:00
Ivan
85d783c247
Poco contrib refactoring (#10396)
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
2020-05-08 17:11:19 +03:00
alexey-milovidov
e908d1cd01
Revert "Partial fix of clang build stack unwinding (omit-frame-pointer)" 2020-05-05 14:36:32 +03:00
alexey-milovidov
a235533d5f
Merge pull request #10576 from Jokser/s3-with-dynamic-proxies
S3 with dynamic proxy configuration
2020-05-03 01:25:13 +03:00
Amos Bird
014a1233ff
Partial fix of clang build stack unwinding 2020-05-02 18:52:29 +08:00
Pavel Kovalenko
2ecf4b5a23 Dynamic proxy configuration for DiskS3. 2020-04-29 11:40:29 +03:00
Alexey Milovidov
b16ee538ab Merge branch 'master' into sorting-processors 2020-04-17 05:31:55 +03:00
Nikolai Kochetov
434a13a8bd Move algorithm to separate files. 2020-04-14 12:21:24 +03:00
Ivan Lezhankin
910ab8a265 Move contrib/libpcg-random → base/pcg-random 2020-04-13 18:42:58 +03:00
Nikolai Kochetov
82ef20d6dc Fix build. 2020-04-12 17:17:34 +03:00
Nikolai Kochetov
1b6d80bfae Merge branch 'master' into sorting-processors 2020-04-10 12:07:21 +03:00
alexey-milovidov
4a73fe7477
Merge pull request #9889 from Avogar/msgpack_format
Msgpack format
2020-04-10 00:07:10 +03:00
Vxider
a8cd174fc4 merged with master 2020-04-09 22:34:02 +08:00
Ivan
f6b31f344d
Add cross-compile build for FreeBSD (#9643)
* Add toolchain to Docker image
2020-04-07 11:33:49 +03:00
Vxider
2fb9138505 merged with master 2020-04-06 17:36:44 +08:00
Nikolai Kochetov
e23fda80aa Merge with master 2020-04-04 19:02:17 +03:00
Avogar
fbc11d3bf9 Move files 2020-04-04 17:04:20 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00