mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-11-22 07:31:57 +00:00
Merge remote-tracking branch 'main/master' into h3-integration-fix
This commit is contained in:
Ivan Remen
commit
b439769aa7
9
.gitmodules
vendored
9
.gitmodules
vendored
@ -79,3 +79,12 @@
|
|||||||
[submodule "contrib/h3"]
|
[submodule "contrib/h3"]
|
||||||
path = contrib/h3
|
path = contrib/h3
|
||||||
url = https://github.com/uber/h3
|
url = https://github.com/uber/h3
|
||||||
|
[submodule "contrib/hyperscan"]
|
||||||
|
path = contrib/hyperscan
|
||||||
|
url = https://github.com/ClickHouse-Extras/hyperscan.git
|
||||||
|
[submodule "contrib/simdjson"]
|
||||||
|
path = contrib/simdjson
|
||||||
|
url = https://github.com/lemire/simdjson.git
|
||||||
|
[submodule "contrib/rapidjson"]
|
||||||
|
path = contrib/rapidjson
|
||||||
|
url = https://github.com/Tencent/rapidjson
|
||||||
|
|||||||
392
CHANGELOG.md
392
CHANGELOG.md
@ -1,3 +1,377 @@
|
|||||||
|
## ClickHouse release 19.7.3.9, 2019-05-30
|
||||||
|
|
||||||
|
### New Features
|
||||||
|
* Allow to limit the range of a setting that can be specified by user.
|
||||||
|
These constraints can be set up in user settings profile.
|
||||||
|
[#4931](https://github.com/yandex/ClickHouse/pull/4931) ([Vitaly
|
||||||
|
Baranov](https://github.com/vitlibar))
|
||||||
|
* Add a second version of the function `groupUniqArray` with an optional
|
||||||
|
`max_size` parameter that limits the size of the resulting array. This
|
||||||
|
behavior is similar to `groupArray(max_size)(x)` function.
|
||||||
|
[#5026](https://github.com/yandex/ClickHouse/pull/5026) ([Guillaume
|
||||||
|
Tassery](https://github.com/YiuRULE))
|
||||||
|
* For TSVWithNames/CSVWithNames input file formats, column order can now be
|
||||||
|
determined from file header. This is controlled by
|
||||||
|
`input_format_with_names_use_header` parameter.
|
||||||
|
[#5081](https://github.com/yandex/ClickHouse/pull/5081)
|
||||||
|
([Alexander](https://github.com/Akazz))
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
* Crash with uncompressed_cache + JOIN during merge (#5197)
|
||||||
|
[#5133](https://github.com/yandex/ClickHouse/pull/5133) ([Danila
|
||||||
|
Kutenin](https://github.com/danlark1))
|
||||||
|
* Segmentation fault on a clickhouse-client query to system tables. #5066
|
||||||
|
[#5127](https://github.com/yandex/ClickHouse/pull/5127)
|
||||||
|
([Ivan](https://github.com/abyss7))
|
||||||
|
* Data loss on heavy load via KafkaEngine (#4736)
|
||||||
|
[#5080](https://github.com/yandex/ClickHouse/pull/5080)
|
||||||
|
([Ivan](https://github.com/abyss7))
|
||||||
|
* Fixed very rare data race condition that could happen when executing a query with UNION ALL involving at least two SELECTs from system.columns, system.tables, system.parts, system.parts_tables or tables of Merge family and performing ALTER of columns of the related tables concurrently. [#5189](https://github.com/yandex/ClickHouse/pull/5189) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
### Performance Improvements
|
||||||
|
* Use radix sort for sorting by single numeric column in `ORDER BY` without
|
||||||
|
`LIMIT`. [#5106](https://github.com/yandex/ClickHouse/pull/5106),
|
||||||
|
[#4439](https://github.com/yandex/ClickHouse/pull/4439)
|
||||||
|
([Evgenii Pravda](https://github.com/kvinty),
|
||||||
|
[alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
### Documentation
|
||||||
|
* Translate documentation for some table engines to Chinese.
|
||||||
|
[#5107](https://github.com/yandex/ClickHouse/pull/5107),
|
||||||
|
[#5094](https://github.com/yandex/ClickHouse/pull/5094),
|
||||||
|
[#5087](https://github.com/yandex/ClickHouse/pull/5087)
|
||||||
|
([张风啸](https://github.com/AlexZFX)),
|
||||||
|
[#5068](https://github.com/yandex/ClickHouse/pull/5068) ([never
|
||||||
|
lee](https://github.com/neverlee))
|
||||||
|
|
||||||
|
### Build/Testing/Packaging Improvements
|
||||||
|
* Print UTF-8 characters properly in `clickhouse-test`.
|
||||||
|
[#5084](https://github.com/yandex/ClickHouse/pull/5084)
|
||||||
|
([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Add command line parameter for clickhouse-client to always load suggestion
|
||||||
|
data. [#5102](https://github.com/yandex/ClickHouse/pull/5102)
|
||||||
|
([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Resolve some of PVS-Studio warnings.
|
||||||
|
[#5082](https://github.com/yandex/ClickHouse/pull/5082)
|
||||||
|
([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Update LZ4 [#5040](https://github.com/yandex/ClickHouse/pull/5040) ([Danila
|
||||||
|
Kutenin](https://github.com/danlark1))
|
||||||
|
* Add gperf to build requirements for upcoming pull request #5030.
|
||||||
|
[#5110](https://github.com/yandex/ClickHouse/pull/5110)
|
||||||
|
([proller](https://github.com/proller))
|
||||||
|
|
||||||
|
## ClickHouse release 19.6.2.11, 2019-05-13
|
||||||
|
|
||||||
|
### New Features
|
||||||
|
* TTL expressions for columns and tables. [#4212](https://github.com/yandex/ClickHouse/pull/4212) ([Anton Popov](https://github.com/CurtizJ))
|
||||||
|
* Added support for `brotli` compression for HTTP responses (Accept-Encoding: br) [#4388](https://github.com/yandex/ClickHouse/pull/4388) ([Mikhail](https://github.com/fandyushin))
|
||||||
|
* Added new function `isValidUTF8` for checking whether a set of bytes is correctly utf-8 encoded. [#4934](https://github.com/yandex/ClickHouse/pull/4934) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Add new load balancing policy `first_or_random` which sends queries to the first specified host and if it's inaccessible send queries to random hosts of shard. Useful for cross-replication topology setups. [#5012](https://github.com/yandex/ClickHouse/pull/5012) ([nvartolomei](https://github.com/nvartolomei))
|
||||||
|
|
||||||
|
### Experimental Features
|
||||||
|
* Add setting `index_granularity_bytes` (adaptive index granularity) for MergeTree* tables family. [#4826](https://github.com/yandex/ClickHouse/pull/4826) ([alesapin](https://github.com/alesapin))
|
||||||
|
|
||||||
|
### Improvements
|
||||||
|
* Added support for non-constant and negative size and length arguments for function `substringUTF8`. [#4989](https://github.com/yandex/ClickHouse/pull/4989) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Disable push-down to right table in left join, left table in right join, and both tables in full join. This fixes wrong JOIN results in some cases. [#4846](https://github.com/yandex/ClickHouse/pull/4846) ([Ivan](https://github.com/abyss7))
|
||||||
|
* `clickhouse-copier`: auto upload task configuration from `--task-file` option [#4876](https://github.com/yandex/ClickHouse/pull/4876) ([proller](https://github.com/proller))
|
||||||
|
* Added typos handler for storage factory and table functions factory. [#4891](https://github.com/yandex/ClickHouse/pull/4891) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Support asterisks and qualified asterisks for multiple joins without subqueries [#4898](https://github.com/yandex/ClickHouse/pull/4898) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Make missing column error message more user friendly. [#4915](https://github.com/yandex/ClickHouse/pull/4915) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
|
||||||
|
### Performance Improvements
|
||||||
|
* Significant speedup of ASOF JOIN [#4924](https://github.com/yandex/ClickHouse/pull/4924) ([Martijn Bakker](https://github.com/Gladdy))
|
||||||
|
|
||||||
|
### Backward Incompatible Changes
|
||||||
|
* HTTP header `Query-Id` was renamed to `X-ClickHouse-Query-Id` for consistency. [#4972](https://github.com/yandex/ClickHouse/pull/4972) ([Mikhail](https://github.com/fandyushin))
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
* Fixed potential null pointer dereference in `clickhouse-copier`. [#4900](https://github.com/yandex/ClickHouse/pull/4900) ([proller](https://github.com/proller))
|
||||||
|
* Fixed error on query with JOIN + ARRAY JOIN [#4938](https://github.com/yandex/ClickHouse/pull/4938) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fixed hanging on start of the server when a dictionary depends on another dictionary via a database with engine=Dictionary. [#4962](https://github.com/yandex/ClickHouse/pull/4962) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Partially fix distributed_product_mode = local. It's possible to allow columns of local tables in where/having/order by/... via table aliases. Throw exception if table does not have alias. There's not possible to access to the columns without table aliases yet. [#4986](https://github.com/yandex/ClickHouse/pull/4986) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fix potentially wrong result for `SELECT DISTINCT` with `JOIN` [#5001](https://github.com/yandex/ClickHouse/pull/5001) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fixed very rare data race condition that could happen when executing a query with UNION ALL involving at least two SELECTs from system.columns, system.tables, system.parts, system.parts_tables or tables of Merge family and performing ALTER of columns of the related tables concurrently. [#5189](https://github.com/yandex/ClickHouse/pull/5189) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
### Build/Testing/Packaging Improvements
|
||||||
|
* Fixed test failures when running clickhouse-server on different host [#4713](https://github.com/yandex/ClickHouse/pull/4713) ([Vasily Nemkov](https://github.com/Enmk))
|
||||||
|
* clickhouse-test: Disable color control sequences in non tty environment. [#4937](https://github.com/yandex/ClickHouse/pull/4937) ([alesapin](https://github.com/alesapin))
|
||||||
|
* clickhouse-test: Allow use any test database (remove `test.` qualification where it possible) [#5008](https://github.com/yandex/ClickHouse/pull/5008) ([proller](https://github.com/proller))
|
||||||
|
* Fix ubsan errors [#5037](https://github.com/yandex/ClickHouse/pull/5037) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Yandex LFAlloc was added to ClickHouse to allocate MarkCache and UncompressedCache data in different ways to catch segfaults more reliable [#4995](https://github.com/yandex/ClickHouse/pull/4995) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Python util to help with backports and changelogs. [#4949](https://github.com/yandex/ClickHouse/pull/4949) ([Ivan](https://github.com/abyss7))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.5.4.22, 2019-05-13
|
||||||
|
|
||||||
|
### Bug fixes
|
||||||
|
* Fixed possible crash in bitmap* functions [#5220](https://github.com/yandex/ClickHouse/pull/5220) [#5228](https://github.com/yandex/ClickHouse/pull/5228) ([Andy Yang](https://github.com/andyyzh))
|
||||||
|
* Fixed very rare data race condition that could happen when executing a query with UNION ALL involving at least two SELECTs from system.columns, system.tables, system.parts, system.parts_tables or tables of Merge family and performing ALTER of columns of the related tables concurrently. [#5189](https://github.com/yandex/ClickHouse/pull/5189) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed error `Set for IN is not created yet in case of using single LowCardinality column in the left part of IN`. This error happened if LowCardinality column was the part of primary key. #5031 [#5154](https://github.com/yandex/ClickHouse/pull/5154) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Modification of retention function: If a row satisfies both the first and NTH condition, only the first satisfied condition is added to the data state. Now all conditions that satisfy in a row of data are added to the data state. [#5119](https://github.com/yandex/ClickHouse/pull/5119) ([小路](https://github.com/nicelulu))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.5.3.8, 2019-04-18
|
||||||
|
|
||||||
|
### Bug fixes
|
||||||
|
* Fixed type of setting `max_partitions_per_insert_block` from boolean to UInt64. [#5028](https://github.com/yandex/ClickHouse/pull/5028) ([Mohammad Hossein Sekhavat](https://github.com/mhsekhavat))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.5.2.6, 2019-04-15
|
||||||
|
|
||||||
|
### New Features
|
||||||
|
|
||||||
|
* [Hyperscan](https://github.com/intel/hyperscan) multiple regular expression matching was added (functions `multiMatchAny`, `multiMatchAnyIndex`, `multiFuzzyMatchAny`, `multiFuzzyMatchAnyIndex`). [#4780](https://github.com/yandex/ClickHouse/pull/4780), [#4841](https://github.com/yandex/ClickHouse/pull/4841) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* `multiSearchFirstPosition` function was added. [#4780](https://github.com/yandex/ClickHouse/pull/4780) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Implement the predefined expression filter per row for tables. [#4792](https://github.com/yandex/ClickHouse/pull/4792) ([Ivan](https://github.com/abyss7))
|
||||||
|
* A new type of data skipping indices based on bloom filters (can be used for `equal`, `in` and `like` functions). [#4499](https://github.com/yandex/ClickHouse/pull/4499) ([Nikita Vasilev](https://github.com/nikvas0))
|
||||||
|
* Added `ASOF JOIN` which allows to run queries that join to the most recent value known. [#4774](https://github.com/yandex/ClickHouse/pull/4774) [#4867](https://github.com/yandex/ClickHouse/pull/4867) [#4863](https://github.com/yandex/ClickHouse/pull/4863) [#4875](https://github.com/yandex/ClickHouse/pull/4875) ([Martijn Bakker](https://github.com/Gladdy), [Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Rewrite multiple `COMMA JOIN` to `CROSS JOIN`. Then rewrite them to `INNER JOIN` if possible. [#4661](https://github.com/yandex/ClickHouse/pull/4661) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
|
||||||
|
### Improvement
|
||||||
|
|
||||||
|
* `topK` and `topKWeighted` now supports custom `loadFactor` (fixes issue [#4252](https://github.com/yandex/ClickHouse/issues/4252)). [#4634](https://github.com/yandex/ClickHouse/pull/4634) ([Kirill Danshin](https://github.com/kirillDanshin))
|
||||||
|
* Allow to use `parallel_replicas_count > 1` even for tables without sampling (the setting is simply ignored for them). In previous versions it was lead to exception. [#4637](https://github.com/yandex/ClickHouse/pull/4637) ([Alexey Elymanov](https://github.com/digitalist))
|
||||||
|
* Support for `CREATE OR REPLACE VIEW`. Allow to create a view or set a new definition in a single statement. [#4654](https://github.com/yandex/ClickHouse/pull/4654) ([Boris Granveaud](https://github.com/bgranvea))
|
||||||
|
* `Buffer` table engine now supports `PREWHERE`. [#4671](https://github.com/yandex/ClickHouse/pull/4671) ([Yangkuan Liu](https://github.com/LiuYangkuan))
|
||||||
|
* Add ability to start replicated table without metadata in zookeeper in `readonly` mode. [#4691](https://github.com/yandex/ClickHouse/pull/4691) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Fixed flicker of progress bar in clickhouse-client. The issue was most noticeable when using `FORMAT Null` with streaming queries. [#4811](https://github.com/yandex/ClickHouse/pull/4811) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Allow to disable functions with `hyperscan` library on per user basis to limit potentially excessive and uncontrolled resource usage. [#4816](https://github.com/yandex/ClickHouse/pull/4816) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Add version number logging in all errors. [#4824](https://github.com/yandex/ClickHouse/pull/4824) ([proller](https://github.com/proller))
|
||||||
|
* Added restriction to the `multiMatch` functions which requires string size to fit into `unsigned int`. Also added the number of arguments limit to the `multiSearch` functions. [#4834](https://github.com/yandex/ClickHouse/pull/4834) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Improved usage of scratch space and error handling in Hyperscan. [#4866](https://github.com/yandex/ClickHouse/pull/4866) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Fill `system.graphite_detentions` from a table config of `*GraphiteMergeTree` engine tables. [#4584](https://github.com/yandex/ClickHouse/pull/4584) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
|
||||||
|
* Rename `trigramDistance` function to `ngramDistance` and add more functions with `CaseInsensitive` and `UTF`. [#4602](https://github.com/yandex/ClickHouse/pull/4602) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Improved data skipping indices calculation. [#4640](https://github.com/yandex/ClickHouse/pull/4640) ([Nikita Vasilev](https://github.com/nikvas0))
|
||||||
|
* Keep ordinary, `DEFAULT`, `MATERIALIZED` and `ALIAS` columns in a single list (fixes issue [#2867](https://github.com/yandex/ClickHouse/issues/2867)). [#4707](https://github.com/yandex/ClickHouse/pull/4707) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
|
||||||
|
### Bug Fix
|
||||||
|
|
||||||
|
* Avoid `std::terminate` in case of memory allocation failure. Now `std::bad_alloc` exception is thrown as expected. [#4665](https://github.com/yandex/ClickHouse/pull/4665) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixes capnproto reading from buffer. Sometimes files wasn't loaded successfully by HTTP. [#4674](https://github.com/yandex/ClickHouse/pull/4674) ([Vladislav](https://github.com/smirnov-vs))
|
||||||
|
* Fix error `Unknown log entry type: 0` after `OPTIMIZE TABLE FINAL` query. [#4683](https://github.com/yandex/ClickHouse/pull/4683) ([Amos Bird](https://github.com/amosbird))
|
||||||
|
* Wrong arguments to `hasAny` or `hasAll` functions may lead to segfault. [#4698](https://github.com/yandex/ClickHouse/pull/4698) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Deadlock may happen while executing `DROP DATABASE dictionary` query. [#4701](https://github.com/yandex/ClickHouse/pull/4701) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix undefinied behavior in `median` and `quantile` functions. [#4702](https://github.com/yandex/ClickHouse/pull/4702) ([hcz](https://github.com/hczhcz))
|
||||||
|
* Fix compression level detection when `network_compression_method` in lowercase. Broken in v19.1. [#4706](https://github.com/yandex/ClickHouse/pull/4706) ([proller](https://github.com/proller))
|
||||||
|
* Fixed ignorance of `<timezone>UTC</timezone>` setting (fixes issue [#4658](https://github.com/yandex/ClickHouse/issues/4658)). [#4718](https://github.com/yandex/ClickHouse/pull/4718) ([proller](https://github.com/proller))
|
||||||
|
* Fix `histogram` function behaviour with `Distributed` tables. [#4741](https://github.com/yandex/ClickHouse/pull/4741) ([olegkv](https://github.com/olegkv))
|
||||||
|
* Fixed tsan report `destroy of a locked mutex`. [#4742](https://github.com/yandex/ClickHouse/pull/4742) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed TSan report on shutdown due to race condition in system logs usage. Fixed potential use-after-free on shutdown when part_log is enabled. [#4758](https://github.com/yandex/ClickHouse/pull/4758) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix recheck parts in `ReplicatedMergeTreeAlterThread` in case of error. [#4772](https://github.com/yandex/ClickHouse/pull/4772) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Arithmetic operations on intermediate aggregate function states were not working for constant arguments (such as subquery results). [#4776](https://github.com/yandex/ClickHouse/pull/4776) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Always backquote column names in metadata. Otherwise it's impossible to create a table with column named `index` (server won't restart due to malformed `ATTACH` query in metadata). [#4782](https://github.com/yandex/ClickHouse/pull/4782) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix crash in `ALTER ... MODIFY ORDER BY` on `Distributed` table. [#4790](https://github.com/yandex/ClickHouse/pull/4790) ([TCeason](https://github.com/TCeason))
|
||||||
|
* Fix segfault in `JOIN ON` with enabled `enable_optimize_predicate_expression`. [#4794](https://github.com/yandex/ClickHouse/pull/4794) ([Winter Zhang](https://github.com/zhang2014))
|
||||||
|
* Fix bug with adding an extraneous row after consuming a protobuf message from Kafka. [#4808](https://github.com/yandex/ClickHouse/pull/4808) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Fix crash of `JOIN` on not-nullable vs nullable column. Fix `NULLs` in right keys in `ANY JOIN` + `join_use_nulls`. [#4815](https://github.com/yandex/ClickHouse/pull/4815) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fix segmentation fault in `clickhouse-copier`. [#4835](https://github.com/yandex/ClickHouse/pull/4835) ([proller](https://github.com/proller))
|
||||||
|
* Fixed race condition in `SELECT` from `system.tables` if the table is renamed or altered concurrently. [#4836](https://github.com/yandex/ClickHouse/pull/4836) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed data race when fetching data part that is already obsolete. [#4839](https://github.com/yandex/ClickHouse/pull/4839) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed rare data race that can happen during `RENAME` table of MergeTree family. [#4844](https://github.com/yandex/ClickHouse/pull/4844) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed segmentation fault in function `arrayIntersect`. Segmentation fault could happen if function was called with mixed constant and ordinary arguments. [#4847](https://github.com/yandex/ClickHouse/pull/4847) ([Lixiang Qian](https://github.com/fancyqlx))
|
||||||
|
* Fixed reading from `Array(LowCardinality)` column in rare case when column contained a long sequence of empty arrays. [#4850](https://github.com/yandex/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Fix crash in `FULL/RIGHT JOIN` when we joining on nullable vs not nullable. [#4855](https://github.com/yandex/ClickHouse/pull/4855) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fix `No message received` exception while fetching parts between replicas. [#4856](https://github.com/yandex/ClickHouse/pull/4856) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Fixed `arrayIntersect` function wrong result in case of several repeated values in single array. [#4871](https://github.com/yandex/ClickHouse/pull/4871) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Fix a race condition during concurrent `ALTER COLUMN` queries that could lead to a server crash (fixes issue [#3421](https://github.com/yandex/ClickHouse/issues/3421)). [#4592](https://github.com/yandex/ClickHouse/pull/4592) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* Fix incorrect result in `FULL/RIGHT JOIN` with const column. [#4723](https://github.com/yandex/ClickHouse/pull/4723) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fix duplicates in `GLOBAL JOIN` with asterisk. [#4705](https://github.com/yandex/ClickHouse/pull/4705) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fix parameter deduction in `ALTER MODIFY` of column `CODEC` when column type is not specified. [#4883](https://github.com/yandex/ClickHouse/pull/4883) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Functions `cutQueryStringAndFragment()` and `queryStringAndFragment()` now works correctly when `URL` contains a fragment and no query. [#4894](https://github.com/yandex/ClickHouse/pull/4894) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Fix rare bug when setting `min_bytes_to_use_direct_io` is greater than zero, which occures when thread have to seek backward in column file. [#4897](https://github.com/yandex/ClickHouse/pull/4897) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Fix wrong argument types for aggregate functions with `LowCardinality` arguments (fixes issue [#4919](https://github.com/yandex/ClickHouse/issues/4919)). [#4922](https://github.com/yandex/ClickHouse/pull/4922) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Fix wrong name qualification in `GLOBAL JOIN`. [#4969](https://github.com/yandex/ClickHouse/pull/4969) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fix function `toISOWeek` result for year 1970. [#4988](https://github.com/yandex/ClickHouse/pull/4988) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix `DROP`, `TRUNCATE` and `OPTIMIZE` queries duplication, when executed on `ON CLUSTER` for `ReplicatedMergeTree*` tables family. [#4991](https://github.com/yandex/ClickHouse/pull/4991) ([alesapin](https://github.com/alesapin))
|
||||||
|
|
||||||
|
### Backward Incompatible Change
|
||||||
|
|
||||||
|
* Rename setting `insert_sample_with_metadata` to setting `input_format_defaults_for_omitted_fields`. [#4771](https://github.com/yandex/ClickHouse/pull/4771) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Added setting `max_partitions_per_insert_block` (with value 100 by default). If inserted block contains larger number of partitions, an exception is thrown. Set it to 0 if you want to remove the limit (not recommended). [#4845](https://github.com/yandex/ClickHouse/pull/4845) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Multi-search functions were renamed (`multiPosition` to `multiSearchAllPositions`, `multiSearch` to `multiSearchAny`, `firstMatch` to `multiSearchFirstIndex`). [#4780](https://github.com/yandex/ClickHouse/pull/4780) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
|
||||||
|
### Performance Improvement
|
||||||
|
|
||||||
|
* Optimize Volnitsky searcher by inlining, giving about 5-10% search improvement for queries with many needles or many similar bigrams. [#4862](https://github.com/yandex/ClickHouse/pull/4862) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Fix performance issue when setting `use_uncompressed_cache` is greater than zero, which appeared when all read data contained in cache. [#4913](https://github.com/yandex/ClickHouse/pull/4913) ([alesapin](https://github.com/alesapin))
|
||||||
|
|
||||||
|
|
||||||
|
### Build/Testing/Packaging Improvement
|
||||||
|
|
||||||
|
* Hardening debug build: more granular memory mappings and ASLR; add memory protection for mark cache and index. This allows to find more memory stomping bugs in case when ASan and MSan cannot do it. [#4632](https://github.com/yandex/ClickHouse/pull/4632) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Add support for cmake variables `ENABLE_PROTOBUF`, `ENABLE_PARQUET` and `ENABLE_BROTLI` which allows to enable/disable the above features (same as we can do for librdkafka, mysql, etc). [#4669](https://github.com/yandex/ClickHouse/pull/4669) ([Silviu Caragea](https://github.com/silviucpp))
|
||||||
|
* Add ability to print process list and stacktraces of all threads if some queries are hung after test run. [#4675](https://github.com/yandex/ClickHouse/pull/4675) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Add retries on `Connection loss` error in `clickhouse-test`. [#4682](https://github.com/yandex/ClickHouse/pull/4682) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Add freebsd build with vagrant and build with thread sanitizer to packager script. [#4712](https://github.com/yandex/ClickHouse/pull/4712) [#4748](https://github.com/yandex/ClickHouse/pull/4748) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Now user asked for password for user `'default'` during installation. [#4725](https://github.com/yandex/ClickHouse/pull/4725) ([proller](https://github.com/proller))
|
||||||
|
* Suppress warning in `rdkafka` library. [#4740](https://github.com/yandex/ClickHouse/pull/4740) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Allow ability to build without ssl. [#4750](https://github.com/yandex/ClickHouse/pull/4750) ([proller](https://github.com/proller))
|
||||||
|
* Add a way to launch clickhouse-server image from a custom user. [#4753](https://github.com/yandex/ClickHouse/pull/4753) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
|
||||||
|
* Upgrade contrib boost to 1.69. [#4793](https://github.com/yandex/ClickHouse/pull/4793) ([proller](https://github.com/proller))
|
||||||
|
* Disable usage of `mremap` when compiled with Thread Sanitizer. Surprisingly enough, TSan does not intercept `mremap` (though it does intercept `mmap`, `munmap`) that leads to false positives. Fixed TSan report in stateful tests. [#4859](https://github.com/yandex/ClickHouse/pull/4859) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Add test checking using format schema via HTTP interface. [#4864](https://github.com/yandex/ClickHouse/pull/4864) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
|
||||||
|
## ClickHouse release 19.4.4.33, 2019-04-17
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
* Avoid `std::terminate` in case of memory allocation failure. Now `std::bad_alloc` exception is thrown as expected. [#4665](https://github.com/yandex/ClickHouse/pull/4665) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixes capnproto reading from buffer. Sometimes files wasn't loaded successfully by HTTP. [#4674](https://github.com/yandex/ClickHouse/pull/4674) ([Vladislav](https://github.com/smirnov-vs))
|
||||||
|
* Fix error `Unknown log entry type: 0` after `OPTIMIZE TABLE FINAL` query. [#4683](https://github.com/yandex/ClickHouse/pull/4683) ([Amos Bird](https://github.com/amosbird))
|
||||||
|
* Wrong arguments to `hasAny` or `hasAll` functions may lead to segfault. [#4698](https://github.com/yandex/ClickHouse/pull/4698) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Deadlock may happen while executing `DROP DATABASE dictionary` query. [#4701](https://github.com/yandex/ClickHouse/pull/4701) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix undefinied behavior in `median` and `quantile` functions. [#4702](https://github.com/yandex/ClickHouse/pull/4702) ([hcz](https://github.com/hczhcz))
|
||||||
|
* Fix compression level detection when `network_compression_method` in lowercase. Broken in v19.1. [#4706](https://github.com/yandex/ClickHouse/pull/4706) ([proller](https://github.com/proller))
|
||||||
|
* Fixed ignorance of `<timezone>UTC</timezone>` setting (fixes issue [#4658](https://github.com/yandex/ClickHouse/issues/4658)). [#4718](https://github.com/yandex/ClickHouse/pull/4718) ([proller](https://github.com/proller))
|
||||||
|
* Fix `histogram` function behaviour with `Distributed` tables. [#4741](https://github.com/yandex/ClickHouse/pull/4741) ([olegkv](https://github.com/olegkv))
|
||||||
|
* Fixed tsan report `destroy of a locked mutex`. [#4742](https://github.com/yandex/ClickHouse/pull/4742) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed TSan report on shutdown due to race condition in system logs usage. Fixed potential use-after-free on shutdown when part_log is enabled. [#4758](https://github.com/yandex/ClickHouse/pull/4758) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix recheck parts in `ReplicatedMergeTreeAlterThread` in case of error. [#4772](https://github.com/yandex/ClickHouse/pull/4772) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Arithmetic operations on intermediate aggregate function states were not working for constant arguments (such as subquery results). [#4776](https://github.com/yandex/ClickHouse/pull/4776) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Always backquote column names in metadata. Otherwise it's impossible to create a table with column named `index` (server won't restart due to malformed `ATTACH` query in metadata). [#4782](https://github.com/yandex/ClickHouse/pull/4782) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix crash in `ALTER ... MODIFY ORDER BY` on `Distributed` table. [#4790](https://github.com/yandex/ClickHouse/pull/4790) ([TCeason](https://github.com/TCeason))
|
||||||
|
* Fix segfault in `JOIN ON` with enabled `enable_optimize_predicate_expression`. [#4794](https://github.com/yandex/ClickHouse/pull/4794) ([Winter Zhang](https://github.com/zhang2014))
|
||||||
|
* Fix bug with adding an extraneous row after consuming a protobuf message from Kafka. [#4808](https://github.com/yandex/ClickHouse/pull/4808) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Fix segmentation fault in `clickhouse-copier`. [#4835](https://github.com/yandex/ClickHouse/pull/4835) ([proller](https://github.com/proller))
|
||||||
|
* Fixed race condition in `SELECT` from `system.tables` if the table is renamed or altered concurrently. [#4836](https://github.com/yandex/ClickHouse/pull/4836) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed data race when fetching data part that is already obsolete. [#4839](https://github.com/yandex/ClickHouse/pull/4839) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed rare data race that can happen during `RENAME` table of MergeTree family. [#4844](https://github.com/yandex/ClickHouse/pull/4844) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed segmentation fault in function `arrayIntersect`. Segmentation fault could happen if function was called with mixed constant and ordinary arguments. [#4847](https://github.com/yandex/ClickHouse/pull/4847) ([Lixiang Qian](https://github.com/fancyqlx))
|
||||||
|
* Fixed reading from `Array(LowCardinality)` column in rare case when column contained a long sequence of empty arrays. [#4850](https://github.com/yandex/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Fix `No message received` exception while fetching parts between replicas. [#4856](https://github.com/yandex/ClickHouse/pull/4856) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Fixed `arrayIntersect` function wrong result in case of several repeated values in single array. [#4871](https://github.com/yandex/ClickHouse/pull/4871) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Fix a race condition during concurrent `ALTER COLUMN` queries that could lead to a server crash (fixes issue [#3421](https://github.com/yandex/ClickHouse/issues/3421)). [#4592](https://github.com/yandex/ClickHouse/pull/4592) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* Fix parameter deduction in `ALTER MODIFY` of column `CODEC` when column type is not specified. [#4883](https://github.com/yandex/ClickHouse/pull/4883) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Functions `cutQueryStringAndFragment()` and `queryStringAndFragment()` now works correctly when `URL` contains a fragment and no query. [#4894](https://github.com/yandex/ClickHouse/pull/4894) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Fix rare bug when setting `min_bytes_to_use_direct_io` is greater than zero, which occures when thread have to seek backward in column file. [#4897](https://github.com/yandex/ClickHouse/pull/4897) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Fix wrong argument types for aggregate functions with `LowCardinality` arguments (fixes issue [#4919](https://github.com/yandex/ClickHouse/issues/4919)). [#4922](https://github.com/yandex/ClickHouse/pull/4922) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Fix function `toISOWeek` result for year 1970. [#4988](https://github.com/yandex/ClickHouse/pull/4988) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix `DROP`, `TRUNCATE` and `OPTIMIZE` queries duplication, when executed on `ON CLUSTER` for `ReplicatedMergeTree*` tables family. [#4991](https://github.com/yandex/ClickHouse/pull/4991) ([alesapin](https://github.com/alesapin))
|
||||||
|
|
||||||
|
### Improvements
|
||||||
|
|
||||||
|
* Keep ordinary, `DEFAULT`, `MATERIALIZED` and `ALIAS` columns in a single list (fixes issue [#2867](https://github.com/yandex/ClickHouse/issues/2867)). [#4707](https://github.com/yandex/ClickHouse/pull/4707) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
|
||||||
|
## ClickHouse release 19.4.3.11, 2019-04-02
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
* Fix crash in `FULL/RIGHT JOIN` when we joining on nullable vs not nullable. [#4855](https://github.com/yandex/ClickHouse/pull/4855) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fix segmentation fault in `clickhouse-copier`. [#4835](https://github.com/yandex/ClickHouse/pull/4835) ([proller](https://github.com/proller))
|
||||||
|
|
||||||
|
### Build/Testing/Packaging Improvement
|
||||||
|
|
||||||
|
* Add a way to launch clickhouse-server image from a custom user. [#4753](https://github.com/yandex/ClickHouse/pull/4753) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
|
||||||
|
|
||||||
|
## ClickHouse release 19.4.2.7, 2019-03-30
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
* Fixed reading from `Array(LowCardinality)` column in rare case when column contained a long sequence of empty arrays. [#4850](https://github.com/yandex/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
|
||||||
|
## ClickHouse release 19.4.1.3, 2019-03-19
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
* Fixed remote queries which contain both `LIMIT BY` and `LIMIT`. Previously, if `LIMIT BY` and `LIMIT` were used for remote query, `LIMIT` could happen before `LIMIT BY`, which led to too filtered result. [#4708](https://github.com/yandex/ClickHouse/pull/4708) ([Constantin S. Pan](https://github.com/kvap))
|
||||||
|
|
||||||
|
## ClickHouse release 19.4.0.49, 2019-03-09
|
||||||
|
|
||||||
|
### New Features
|
||||||
|
* Added full support for `Protobuf` format (input and output, nested data structures). [#4174](https://github.com/yandex/ClickHouse/pull/4174) [#4493](https://github.com/yandex/ClickHouse/pull/4493) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Added bitmap functions with Roaring Bitmaps. [#4207](https://github.com/yandex/ClickHouse/pull/4207) ([Andy Yang](https://github.com/andyyzh)) [#4568](https://github.com/yandex/ClickHouse/pull/4568) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Parquet format support. [#4448](https://github.com/yandex/ClickHouse/pull/4448) ([proller](https://github.com/proller))
|
||||||
|
* N-gram distance was added for fuzzy string comparison. It is similar to q-gram metrics in R language. [#4466](https://github.com/yandex/ClickHouse/pull/4466) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Combine rules for graphite rollup from dedicated aggregation and retention patterns. [#4426](https://github.com/yandex/ClickHouse/pull/4426) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
|
||||||
|
* Added `max_execution_speed` and `max_execution_speed_bytes` to limit resource usage. Added `min_execution_speed_bytes` setting to complement the `min_execution_speed`. [#4430](https://github.com/yandex/ClickHouse/pull/4430) ([Winter Zhang](https://github.com/zhang2014))
|
||||||
|
* Implemented function `flatten`. [#4555](https://github.com/yandex/ClickHouse/pull/4555) [#4409](https://github.com/yandex/ClickHouse/pull/4409) ([alexey-milovidov](https://github.com/alexey-milovidov), [kzon](https://github.com/kzon))
|
||||||
|
* Added functions `arrayEnumerateDenseRanked` and `arrayEnumerateUniqRanked` (it's like `arrayEnumerateUniq` but allows to fine tune array depth to look inside multidimensional arrays). [#4475](https://github.com/yandex/ClickHouse/pull/4475) ([proller](https://github.com/proller)) [#4601](https://github.com/yandex/ClickHouse/pull/4601) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Multiple JOINS with some restrictions: no asterisks, no complex aliases in ON/WHERE/GROUP BY/... [#4462](https://github.com/yandex/ClickHouse/pull/4462) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
* This release also contains all bug fixes from 19.3 and 19.1.
|
||||||
|
* Fixed bug in data skipping indices: order of granules after INSERT was incorrect. [#4407](https://github.com/yandex/ClickHouse/pull/4407) ([Nikita Vasilev](https://github.com/nikvas0))
|
||||||
|
* Fixed `set` index for `Nullable` and `LowCardinality` columns. Before it, `set` index with `Nullable` or `LowCardinality` column led to error `Data type must be deserialized with multiple streams` while selecting. [#4594](https://github.com/yandex/ClickHouse/pull/4594) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Correctly set update_time on full `executable` dictionary update. [#4551](https://github.com/yandex/ClickHouse/pull/4551) ([Tema Novikov](https://github.com/temoon))
|
||||||
|
* Fix broken progress bar in 19.3. [#4627](https://github.com/yandex/ClickHouse/pull/4627) ([filimonov](https://github.com/filimonov))
|
||||||
|
* Fixed inconsistent values of MemoryTracker when memory region was shrinked, in certain cases. [#4619](https://github.com/yandex/ClickHouse/pull/4619) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed undefined behaviour in ThreadPool. [#4612](https://github.com/yandex/ClickHouse/pull/4612) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixed a very rare crash with the message `mutex lock failed: Invalid argument` that could happen when a MergeTree table was dropped concurrently with a SELECT. [#4608](https://github.com/yandex/ClickHouse/pull/4608) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* ODBC driver compatibility with `LowCardinality` data type. [#4381](https://github.com/yandex/ClickHouse/pull/4381) ([proller](https://github.com/proller))
|
||||||
|
* FreeBSD: Fixup for `AIOcontextPool: Found io_event with unknown id 0` error. [#4438](https://github.com/yandex/ClickHouse/pull/4438) ([urgordeadbeef](https://github.com/urgordeadbeef))
|
||||||
|
* `system.part_log` table was created regardless to configuration. [#4483](https://github.com/yandex/ClickHouse/pull/4483) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix undefined behaviour in `dictIsIn` function for cache dictionaries. [#4515](https://github.com/yandex/ClickHouse/pull/4515) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Fixed a deadlock when a SELECT query locks the same table multiple times (e.g. from different threads or when executing multiple subqueries) and there is a concurrent DDL query. [#4535](https://github.com/yandex/ClickHouse/pull/4535) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* Disable compile_expressions by default until we get own `llvm` contrib and can test it with `clang` and `asan`. [#4579](https://github.com/yandex/ClickHouse/pull/4579) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Prevent `std::terminate` when `invalidate_query` for `clickhouse` external dictionary source has returned wrong resultset (empty or more than one row or more than one column). Fixed issue when the `invalidate_query` was performed every five seconds regardless to the `lifetime`. [#4583](https://github.com/yandex/ClickHouse/pull/4583) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Avoid deadlock when the `invalidate_query` for a dictionary with `clickhouse` source was involving `system.dictionaries` table or `Dictionaries` database (rare case). [#4599](https://github.com/yandex/ClickHouse/pull/4599) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fixes for CROSS JOIN with empty WHERE. [#4598](https://github.com/yandex/ClickHouse/pull/4598) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fixed segfault in function "replicate" when constant argument is passed. [#4603](https://github.com/yandex/ClickHouse/pull/4603) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix lambda function with predicate optimizer. [#4408](https://github.com/yandex/ClickHouse/pull/4408) ([Winter Zhang](https://github.com/zhang2014))
|
||||||
|
* Multiple JOINs multiple fixes. [#4595](https://github.com/yandex/ClickHouse/pull/4595) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
|
||||||
|
### Improvements
|
||||||
|
* Support aliases in JOIN ON section for right table columns. [#4412](https://github.com/yandex/ClickHouse/pull/4412) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Result of multiple JOINs need correct result names to be used in subselects. Replace flat aliases with source names in result. [#4474](https://github.com/yandex/ClickHouse/pull/4474) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Improve push-down logic for joined statements. [#4387](https://github.com/yandex/ClickHouse/pull/4387) ([Ivan](https://github.com/abyss7))
|
||||||
|
|
||||||
|
### Performance Improvements
|
||||||
|
* Improved heuristics of "move to PREWHERE" optimization. [#4405](https://github.com/yandex/ClickHouse/pull/4405) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Use proper lookup tables that uses HashTable's API for 8-bit and 16-bit keys. [#4536](https://github.com/yandex/ClickHouse/pull/4536) ([Amos Bird](https://github.com/amosbird))
|
||||||
|
* Improved performance of string comparison. [#4564](https://github.com/yandex/ClickHouse/pull/4564) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Cleanup distributed DDL queue in a separate thread so that it doesn't slow down the main loop that processes distributed DDL tasks. [#4502](https://github.com/yandex/ClickHouse/pull/4502) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* When `min_bytes_to_use_direct_io` is set to 1, not every file was opened with O_DIRECT mode because the data size to read was sometimes underestimated by the size of one compressed block. [#4526](https://github.com/yandex/ClickHouse/pull/4526) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
### Build/Testing/Packaging Improvement
|
||||||
|
* Added support for clang-9 [#4604](https://github.com/yandex/ClickHouse/pull/4604) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Fix wrong `__asm__` instructions (again) [#4621](https://github.com/yandex/ClickHouse/pull/4621) ([Konstantin Podshumok](https://github.com/podshumok))
|
||||||
|
* Add ability to specify settings for `clickhouse-performance-test` from command line. [#4437](https://github.com/yandex/ClickHouse/pull/4437) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Add dictionaries tests to integration tests. [#4477](https://github.com/yandex/ClickHouse/pull/4477) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Added queries from the benchmark on the website to automated performance tests. [#4496](https://github.com/yandex/ClickHouse/pull/4496) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* `xxhash.h` does not exist in external lz4 because it is an implementation detail and its symbols are namespaced with `XXH_NAMESPACE` macro. When lz4 is external, xxHash has to be external too, and the dependents have to link to it. [#4495](https://github.com/yandex/ClickHouse/pull/4495) ([Orivej Desh](https://github.com/orivej))
|
||||||
|
* Fixed a case when `quantileTiming` aggregate function can be called with negative or floating point argument (this fixes fuzz test with undefined behaviour sanitizer). [#4506](https://github.com/yandex/ClickHouse/pull/4506) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Spelling error correction. [#4531](https://github.com/yandex/ClickHouse/pull/4531) ([sdk2](https://github.com/sdk2))
|
||||||
|
* Fix compilation on Mac. [#4371](https://github.com/yandex/ClickHouse/pull/4371) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Build fixes for FreeBSD and various unusual build configurations. [#4444](https://github.com/yandex/ClickHouse/pull/4444) ([proller](https://github.com/proller))
|
||||||
|
|
||||||
|
## ClickHouse release 19.3.9.1, 2019-04-02
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
* Fix crash in `FULL/RIGHT JOIN` when we joining on nullable vs not nullable. [#4855](https://github.com/yandex/ClickHouse/pull/4855) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fix segmentation fault in `clickhouse-copier`. [#4835](https://github.com/yandex/ClickHouse/pull/4835) ([proller](https://github.com/proller))
|
||||||
|
* Fixed reading from `Array(LowCardinality)` column in rare case when column contained a long sequence of empty arrays. [#4850](https://github.com/yandex/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
|
||||||
|
### Build/Testing/Packaging Improvement
|
||||||
|
|
||||||
|
* Add a way to launch clickhouse-server image from a custom user [#4753](https://github.com/yandex/ClickHouse/pull/4753) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.3.7, 2019-03-12
|
||||||
|
|
||||||
|
### Bug fixes
|
||||||
|
|
||||||
|
* Fixed error in #3920. This error manifestate itself as random cache corruption (messages `Unknown codec family code`, `Cannot seek through file`) and segfaults. This bug first appeared in version 19.1 and is present in versions up to 19.1.10 and 19.3.6. [#4623](https://github.com/yandex/ClickHouse/pull/4623) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.3.6, 2019-03-02
|
||||||
|
|
||||||
|
### Bug fixes
|
||||||
|
|
||||||
|
* When there are more than 1000 threads in a thread pool, `std::terminate` may happen on thread exit. [Azat Khuzhin](https://github.com/azat) [#4485](https://github.com/yandex/ClickHouse/pull/4485) [#4505](https://github.com/yandex/ClickHouse/pull/4505) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Now it's possible to create `ReplicatedMergeTree*` tables with comments on columns without defaults and tables with columns codecs without comments and defaults. Also fix comparison of codecs. [#4523](https://github.com/yandex/ClickHouse/pull/4523) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Fixed crash on JOIN with array or tuple. [#4552](https://github.com/yandex/ClickHouse/pull/4552) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fixed crash in clickhouse-copier with the message `ThreadStatus not created`. [#4540](https://github.com/yandex/ClickHouse/pull/4540) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Fixed hangup on server shutdown if distributed DDLs were used. [#4472](https://github.com/yandex/ClickHouse/pull/4472) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* Incorrect column numbers were printed in error message about text format parsing for columns with number greater than 10. [#4484](https://github.com/yandex/ClickHouse/pull/4484) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
### Build/Testing/Packaging Improvements
|
||||||
|
|
||||||
|
* Fixed build with AVX enabled. [#4527](https://github.com/yandex/ClickHouse/pull/4527) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Enable extended accounting and IO accounting based on good known version instead of kernel under which it is compiled. [#4541](https://github.com/yandex/ClickHouse/pull/4541) ([nvartolomei](https://github.com/nvartolomei))
|
||||||
|
* Allow to skip setting of core_dump.size_limit, warning instead of throw if limit set fail. [#4473](https://github.com/yandex/ClickHouse/pull/4473) ([proller](https://github.com/proller))
|
||||||
|
* Removed the `inline` tags of `void readBinary(...)` in `Field.cpp`. Also merged redundant `namespace DB` blocks. [#4530](https://github.com/yandex/ClickHouse/pull/4530) ([hcz](https://github.com/hczhcz))
|
||||||
|
|
||||||
|
|
||||||
## ClickHouse release 19.3.5, 2019-02-21
|
## ClickHouse release 19.3.5, 2019-02-21
|
||||||
|
|
||||||
### Bug fixes
|
### Bug fixes
|
||||||
@ -34,7 +408,7 @@
|
|||||||
* Added support of `Nullable` types in `mysql` table function. [#4198](https://github.com/yandex/ClickHouse/pull/4198) ([Emmanuel Donin de Rosière](https://github.com/edonin))
|
* Added support of `Nullable` types in `mysql` table function. [#4198](https://github.com/yandex/ClickHouse/pull/4198) ([Emmanuel Donin de Rosière](https://github.com/edonin))
|
||||||
* Support for arbitrary constant expressions in `LIMIT` clause. [#4246](https://github.com/yandex/ClickHouse/pull/4246) ([k3box](https://github.com/k3box))
|
* Support for arbitrary constant expressions in `LIMIT` clause. [#4246](https://github.com/yandex/ClickHouse/pull/4246) ([k3box](https://github.com/k3box))
|
||||||
* Added `topKWeighted` aggregate function that takes additional argument with (unsigned integer) weight. [#4245](https://github.com/yandex/ClickHouse/pull/4245) ([Andrew Golman](https://github.com/andrewgolman))
|
* Added `topKWeighted` aggregate function that takes additional argument with (unsigned integer) weight. [#4245](https://github.com/yandex/ClickHouse/pull/4245) ([Andrew Golman](https://github.com/andrewgolman))
|
||||||
* `StorageJoin` now supports `join_overwrite` setting that allows overwriting existing values of the same key. [#3973](https://github.com/yandex/ClickHouse/pull/3973) ([Amos Bird](https://github.com/amosbird)
|
* `StorageJoin` now supports `join_any_take_last_row` setting that allows overwriting existing values of the same key. [#3973](https://github.com/yandex/ClickHouse/pull/3973) ([Amos Bird](https://github.com/amosbird)
|
||||||
* Added function `toStartOfInterval`. [#4304](https://github.com/yandex/ClickHouse/pull/4304) ([Vitaly Baranov](https://github.com/vitlibar))
|
* Added function `toStartOfInterval`. [#4304](https://github.com/yandex/ClickHouse/pull/4304) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
* Added `RowBinaryWithNamesAndTypes` format. [#4200](https://github.com/yandex/ClickHouse/pull/4200) ([Oleg V. Kozlyuk](https://github.com/DarkWanderer))
|
* Added `RowBinaryWithNamesAndTypes` format. [#4200](https://github.com/yandex/ClickHouse/pull/4200) ([Oleg V. Kozlyuk](https://github.com/DarkWanderer))
|
||||||
* Added `IPv4` and `IPv6` data types. More effective implementations of `IPv*` functions. [#3669](https://github.com/yandex/ClickHouse/pull/3669) ([Vasily Nemkov](https://github.com/Enmk))
|
* Added `IPv4` and `IPv6` data types. More effective implementations of `IPv*` functions. [#3669](https://github.com/yandex/ClickHouse/pull/3669) ([Vasily Nemkov](https://github.com/Enmk))
|
||||||
@ -123,6 +497,20 @@
|
|||||||
* Improved server shutdown time and ALTERs waiting time. [#4372](https://github.com/yandex/ClickHouse/pull/4372) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
* Improved server shutdown time and ALTERs waiting time. [#4372](https://github.com/yandex/ClickHouse/pull/4372) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
* Added info about the replicated_can_become_leader setting to system.replicas and add logging if the replica won't try to become leader. [#4379](https://github.com/yandex/ClickHouse/pull/4379) ([Alex Zatelepin](https://github.com/ztlpn))
|
* Added info about the replicated_can_become_leader setting to system.replicas and add logging if the replica won't try to become leader. [#4379](https://github.com/yandex/ClickHouse/pull/4379) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.1.14, 2019-03-14
|
||||||
|
|
||||||
|
* Fixed error `Column ... queried more than once` that may happen if the setting `asterisk_left_columns_only` is set to 1 in case of using `GLOBAL JOIN` with `SELECT *` (rare case). The issue does not exist in 19.3 and newer. [6bac7d8d](https://github.com/yandex/ClickHouse/pull/4692/commits/6bac7d8d11a9b0d6de0b32b53c47eb2f6f8e7062) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
|
||||||
|
## ClickHouse release 19.1.13, 2019-03-12
|
||||||
|
|
||||||
|
This release contains exactly the same set of patches as 19.3.7.
|
||||||
|
|
||||||
|
## ClickHouse release 19.1.10, 2019-03-03
|
||||||
|
|
||||||
|
This release contains exactly the same set of patches as 19.3.6.
|
||||||
|
|
||||||
|
|
||||||
## ClickHouse release 19.1.9, 2019-02-21
|
## ClickHouse release 19.1.9, 2019-02-21
|
||||||
|
|
||||||
### Bug fixes
|
### Bug fixes
|
||||||
@ -293,7 +681,7 @@
|
|||||||
|
|
||||||
### New features:
|
### New features:
|
||||||
|
|
||||||
* `DEFAULT` expressions are evaluated for missing fields when loading data in semi-structured input formats (`JSONEachRow`, `TSKV`). [#3555](https://github.com/yandex/ClickHouse/pull/3555)
|
* `DEFAULT` expressions are evaluated for missing fields when loading data in semi-structured input formats (`JSONEachRow`, `TSKV`). The feature is enabled with the `insert_sample_with_metadata` setting. [#3555](https://github.com/yandex/ClickHouse/pull/3555)
|
||||||
* The `ALTER TABLE` query now has the `MODIFY ORDER BY` action for changing the sorting key when adding or removing a table column. This is useful for tables in the `MergeTree` family that perform additional tasks when merging based on this sorting key, such as `SummingMergeTree`, `AggregatingMergeTree`, and so on. [#3581](https://github.com/yandex/ClickHouse/pull/3581) [#3755](https://github.com/yandex/ClickHouse/pull/3755)
|
* The `ALTER TABLE` query now has the `MODIFY ORDER BY` action for changing the sorting key when adding or removing a table column. This is useful for tables in the `MergeTree` family that perform additional tasks when merging based on this sorting key, such as `SummingMergeTree`, `AggregatingMergeTree`, and so on. [#3581](https://github.com/yandex/ClickHouse/pull/3581) [#3755](https://github.com/yandex/ClickHouse/pull/3755)
|
||||||
* For tables in the `MergeTree` family, now you can specify a different sorting key (`ORDER BY`) and index (`PRIMARY KEY`). The sorting key can be longer than the index. [#3581](https://github.com/yandex/ClickHouse/pull/3581)
|
* For tables in the `MergeTree` family, now you can specify a different sorting key (`ORDER BY`) and index (`PRIMARY KEY`). The sorting key can be longer than the index. [#3581](https://github.com/yandex/ClickHouse/pull/3581)
|
||||||
* Added the `hdfs` table function and the `HDFS` table engine for importing and exporting data to HDFS. [chenxing-xc](https://github.com/yandex/ClickHouse/pull/3617)
|
* Added the `hdfs` table function and the `HDFS` table engine for importing and exporting data to HDFS. [chenxing-xc](https://github.com/yandex/ClickHouse/pull/3617)
|
||||||
|
|||||||
379
CHANGELOG_RU.md
379
CHANGELOG_RU.md
@ -1,3 +1,366 @@
|
|||||||
|
## ClickHouse release 19.7.3.9, 2019-05-30
|
||||||
|
|
||||||
|
### Новые возможности
|
||||||
|
* Добавлена возможность ограничить значения конфигурационных параметров,
|
||||||
|
которые может задать пользователь. Эти ограничения устанавливаются в профиле
|
||||||
|
настроек пользователя. [#4931](https://github.com/yandex/ClickHouse/pull/4931)
|
||||||
|
([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Добавлен вариант функции `groupUniqArray` с дополнительным параметром
|
||||||
|
`max_size`, который ограничивает размер результирующего массива, аналогично
|
||||||
|
функции `groupArray(max_size)(x)`.
|
||||||
|
[#5026](https://github.com/yandex/ClickHouse/pull/5026) ([Guillaume
|
||||||
|
Tassery](https://github.com/YiuRULE))
|
||||||
|
* Для входных файлов формата TSVWithNames и CSVWithNames появилась возможность
|
||||||
|
определить порядок колонок в файле исходя из его заголовка. Это поведение
|
||||||
|
управляется конфигурационным параметром `input_format_with_names_use_header`.
|
||||||
|
[#5081](https://github.com/yandex/ClickHouse/pull/5081)
|
||||||
|
([Alexander](https://github.com/Akazz))
|
||||||
|
|
||||||
|
### Исправления ошибок
|
||||||
|
* Падение в процессе слияния при использовании uncompressed_cache и JOIN
|
||||||
|
(#5197). [#5133](https://github.com/yandex/ClickHouse/pull/5133) ([Danila
|
||||||
|
Kutenin](https://github.com/danlark1))
|
||||||
|
* Segmentation fault на запросе к системным таблицам (#5066).
|
||||||
|
[#5127](https://github.com/yandex/ClickHouse/pull/5127)
|
||||||
|
([Ivan](https://github.com/abyss7))
|
||||||
|
* Потеря загружаемых данных при больших потоках загрузки через KafkaEngine
|
||||||
|
(#4736). [#5080](https://github.com/yandex/ClickHouse/pull/5080)
|
||||||
|
([Ivan](https://github.com/abyss7))
|
||||||
|
* Исправлен очень редкий data race condition который мог произойти при выполнении запроса с UNION ALL включающего минимум два SELECT из таблиц system.columns, system.tables, system.parts, system.parts_tables или таблиц семейства Merge и одновременно выполняющихся запросов ALTER столбцов соответствующих таблиц. [#5189](https://github.com/yandex/ClickHouse/pull/5189) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
### Улучшения производительности
|
||||||
|
* Используется поразрядная сортировка числовых колонок для `ORDER BY` без
|
||||||
|
`LIMIT`. [#5106](https://github.com/yandex/ClickHouse/pull/5106),
|
||||||
|
[#4439](https://github.com/yandex/ClickHouse/pull/4439) ([Evgenii
|
||||||
|
Pravda](https://github.com/kvinty),
|
||||||
|
[alexey-milovidov](https://github.com/alexey-milovidov)
|
||||||
|
|
||||||
|
### Документация
|
||||||
|
* Документация для некоторых табличных движков переведена на китайский.
|
||||||
|
[#5107](https://github.com/yandex/ClickHouse/pull/5107),
|
||||||
|
[#5094](https://github.com/yandex/ClickHouse/pull/5094),
|
||||||
|
[#5087](https://github.com/yandex/ClickHouse/pull/5087)
|
||||||
|
([张风啸](https://github.com/AlexZFX),
|
||||||
|
[#5068](https://github.com/yandex/ClickHouse/pull/5068) ([never
|
||||||
|
lee](https://github.com/neverlee))
|
||||||
|
|
||||||
|
### Улучшения сборки, тестирования и пакетирования
|
||||||
|
* Правильно отображаются символы в кодировке UTF-8 в `clickhouse-test`.
|
||||||
|
[#5084](https://github.com/yandex/ClickHouse/pull/5084)
|
||||||
|
([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Добавлен параметр командной строки для `clickhouse-client`, позволяющий
|
||||||
|
всегда загружать данные подсказок.
|
||||||
|
[#5102](https://github.com/yandex/ClickHouse/pull/5102)
|
||||||
|
([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлены некоторые предупреждения PVS-Studio.
|
||||||
|
[#5082](https://github.com/yandex/ClickHouse/pull/5082)
|
||||||
|
([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Обновлена библиотека LZ4.
|
||||||
|
[#5040](https://github.com/yandex/ClickHouse/pull/5040) ([Danila
|
||||||
|
Kutenin](https://github.com/danlark1))
|
||||||
|
* В зависимости сборки добавлен gperf для поддержки готовящегося PR #5030.
|
||||||
|
[#5110](https://github.com/yandex/ClickHouse/pull/5110)
|
||||||
|
([proller](https://github.com/proller))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.6.2.11, 2019-05-13
|
||||||
|
|
||||||
|
### Новые возможности
|
||||||
|
* TTL выражения, позволяющие настроить время жизни и автоматическую очистку данных в таблице или в отдельных её столбцах. [#4212](https://github.com/yandex/ClickHouse/pull/4212) ([Anton Popov](https://github.com/CurtizJ))
|
||||||
|
* Добавлена поддержка алгоритма сжатия `brotli` в HTTP ответах (`Accept-Encoding: br`). Для тела POST запросов, эта возможность уже существовала. [#4388](https://github.com/yandex/ClickHouse/pull/4388) ([Mikhail](https://github.com/fandyushin))
|
||||||
|
* Добавлена функция `isValidUTF8` для проверки, содержит ли строка валидные данные в кодировке UTF-8. [#4934](https://github.com/yandex/ClickHouse/pull/4934) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Добавлены новое правило балансировки (`load_balancing`) `first_or_random` по которому запросы посылаются на первый заданый хост и если он недоступен - на случайные хосты шарда. Полезно для топологий с кросс-репликацией. [#5012](https://github.com/yandex/ClickHouse/pull/5012) ([nvartolomei](https://github.com/nvartolomei))
|
||||||
|
|
||||||
|
### Экспериментальные возможности
|
||||||
|
* Добавлена настройка `index_granularity_bytes` (адаптивная гранулярность индекса) для таблиц семейства MergeTree* . [#4826](https://github.com/yandex/ClickHouse/pull/4826) ([alesapin](https://github.com/alesapin))
|
||||||
|
|
||||||
|
### Улучшения
|
||||||
|
* Добавлена поддержка для не константных и отрицательных значений аргументов смещения и длины для функции `substringUTF8`. [#4989](https://github.com/yandex/ClickHouse/pull/4989) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Отключение push-down в правую таблицы в left join, левую таблицу в right join, и в обе таблицы в full join. Это исправляет неправильные JOIN результаты в некоторых случаях. [#4846](https://github.com/yandex/ClickHouse/pull/4846) ([Ivan](https://github.com/abyss7))
|
||||||
|
* `clickhouse-copier`: Автоматическая загрузка конфигурации задачи в zookeeper из `--task-file` опции [#4876](https://github.com/yandex/ClickHouse/pull/4876) ([proller](https://github.com/proller))
|
||||||
|
* Добавлены подсказки с учётом опечаток для имён движков таблиц и табличных функций. [#4891](https://github.com/yandex/ClickHouse/pull/4891) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Поддержка выражений `select *` и `select tablename.*` для множественных join без подзапросов [#4898](https://github.com/yandex/ClickHouse/pull/4898) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Сообщения об ошибках об отсутствующих столбцах стали более понятными. [#4915](https://github.com/yandex/ClickHouse/pull/4915) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
|
||||||
|
### Улучшение производительности
|
||||||
|
* Существенное ускорение ASOF JOIN [#4924](https://github.com/yandex/ClickHouse/pull/4924) ([Martijn Bakker](https://github.com/Gladdy))
|
||||||
|
|
||||||
|
### Обратно несовместимые изменения
|
||||||
|
* HTTP заголовок `Query-Id` переименован в `X-ClickHouse-Query-Id` для соответствия. [#4972](https://github.com/yandex/ClickHouse/pull/4972) ([Mikhail](https://github.com/fandyushin))
|
||||||
|
|
||||||
|
### Исправления ошибок
|
||||||
|
* Исправлены возможные разыменования нулевого указателя в `clickhouse-copier`. [#4900](https://github.com/yandex/ClickHouse/pull/4900) ([proller](https://github.com/proller))
|
||||||
|
* Исправлены ошибки в запросах с JOIN + ARRAY JOIN [#4938](https://github.com/yandex/ClickHouse/pull/4938) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлено зависание на старте сервера если внешний словарь зависит от другого словаря через использование таблицы из БД с движком `Dictionary`. [#4962](https://github.com/yandex/ClickHouse/pull/4962) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* При использовании `distributed_product_mode = 'local'` корректно работает использование столбцов локальных таблиц в where/having/order by/... через табличные алиасы. Выкидывает исключение если таблица не имеет алиас. Доступ к столбцам без алиасов пока не возможен. [#4986](https://github.com/yandex/ClickHouse/pull/4986) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлен потенциально некорректный результат для `SELECT DISTINCT` с `JOIN` [#5001](https://github.com/yandex/ClickHouse/pull/5001) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлен очень редкий data race condition который мог произойти при выполнении запроса с UNION ALL включающего минимум два SELECT из таблиц system.columns, system.tables, system.parts, system.parts_tables или таблиц семейства Merge и одновременно выполняющихся запросов ALTER столбцов соответствующих таблиц. [#5189](https://github.com/yandex/ClickHouse/pull/5189) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
### Улучшения сборки/тестирования/пакетирования
|
||||||
|
* Исправлена неработоспособность тестов, если `clickhouse-server` запущен на удалённом хосте [#4713](https://github.com/yandex/ClickHouse/pull/4713) ([Vasily Nemkov](https://github.com/Enmk))
|
||||||
|
* `clickhouse-test`: Отключена раскраска результата, если команда запускается не в терминале. [#4937](https://github.com/yandex/ClickHouse/pull/4937) ([alesapin](https://github.com/alesapin))
|
||||||
|
* `clickhouse-test`: Возможность использования не только базы данных test [#5008](https://github.com/yandex/ClickHouse/pull/5008) ([proller](https://github.com/proller))
|
||||||
|
* Исправлены ошибки при запуске тестов под UBSan [#5037](https://github.com/yandex/ClickHouse/pull/5037) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Добавлен аллокатор Yandex LFAlloc для аллоцирования MarkCache и UncompressedCache данных разными способами для более надежного отлавливания проездов по памяти [#4995](https://github.com/yandex/ClickHouse/pull/4995) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Утилита для упрощения бэкпортирования изменений в старые релизы и составления changelogs. [#4949](https://github.com/yandex/ClickHouse/pull/4949) ([Ivan](https://github.com/abyss7))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.5.4.22, 2019-05-13
|
||||||
|
|
||||||
|
### Исправления ошибок
|
||||||
|
* Исправлены возможные падения в bitmap* функциях [#5220](https://github.com/yandex/ClickHouse/pull/5220) [#5228](https://github.com/yandex/ClickHouse/pull/5228) ([Andy Yang](https://github.com/andyyzh))
|
||||||
|
* Исправлен очень редкий data race condition который мог произойти при выполнении запроса с UNION ALL включающего минимум два SELECT из таблиц system.columns, system.tables, system.parts, system.parts_tables или таблиц семейства Merge и одновременно выполняющихся запросов ALTER столбцов соответствующих таблиц. [#5189](https://github.com/yandex/ClickHouse/pull/5189) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена ошибка `Set for IN is not created yet in case of using single LowCardinality column in the left part of IN`. Эта ошибка возникала когда LowCardinality столбец была частью primary key. #5031 [#5154](https://github.com/yandex/ClickHouse/pull/5154) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Исправление функции retention: только первое соответствующее условие добавлялось в состояние данных. Сейчас все условия которые удовлетворяют в строке данных добавляются в состояние. [#5119](https://github.com/yandex/ClickHouse/pull/5119) ([小路](https://github.com/nicelulu))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.5.3.8, 2019-04-18
|
||||||
|
|
||||||
|
### Исправления ошибок
|
||||||
|
* Исправлен тип настройки `max_partitions_per_insert_block` с булевого на UInt64. [#5028](https://github.com/yandex/ClickHouse/pull/5028) ([Mohammad Hossein Sekhavat](https://github.com/mhsekhavat))
|
||||||
|
|
||||||
|
## ClickHouse release 19.5.2.6, 2019-04-15
|
||||||
|
|
||||||
|
### Новые возможности
|
||||||
|
|
||||||
|
* Добавлены функции для работы с несколькими регулярными выражениями с помощью библиотеки [Hyperscan](https://github.com/intel/hyperscan). (`multiMatchAny`, `multiMatchAnyIndex`, `multiFuzzyMatchAny`, `multiFuzzyMatchAnyIndex`). [#4780](https://github.com/yandex/ClickHouse/pull/4780), [#4841](https://github.com/yandex/ClickHouse/pull/4841) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Добавлена функция `multiSearchFirstPosition`. [#4780](https://github.com/yandex/ClickHouse/pull/4780) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Реализована возможность указания построчного ограничения доступа к таблицам. [#4792](https://github.com/yandex/ClickHouse/pull/4792) ([Ivan](https://github.com/abyss7))
|
||||||
|
* Добавлен новый тип вторичного индекса на базе фильтра Блума (используется в функциях `equal`, `in` и `like`). [#4499](https://github.com/yandex/ClickHouse/pull/4499) ([Nikita Vasilev](https://github.com/nikvas0))
|
||||||
|
* Добавлен `ASOF JOIN` которые позволяет джойнить строки по наиболее близкому известному значению. [#4774](https://github.com/yandex/ClickHouse/pull/4774) [#4867](https://github.com/yandex/ClickHouse/pull/4867) [#4863](https://github.com/yandex/ClickHouse/pull/4863) [#4875](https://github.com/yandex/ClickHouse/pull/4875) ([Martijn Bakker](https://github.com/Gladdy), [Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Теперь запрос `COMMA JOIN` переписывается `CROSS JOIN`. И затем оба переписываются в `INNER JOIN`, если это возможно. [#4661](https://github.com/yandex/ClickHouse/pull/4661) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
|
||||||
|
### Улучшения
|
||||||
|
|
||||||
|
* Функции `topK` и `topKWeighted` теперь поддерживают произвольный `loadFactor` (исправляет issue [#4252](https://github.com/yandex/ClickHouse/issues/4252)). [#4634](https://github.com/yandex/ClickHouse/pull/4634) ([Kirill Danshin](https://github.com/kirillDanshin))
|
||||||
|
* Добавлена возможность использования настройки `parallel_replicas_count > 1` для таблиц без семплирования (ранее настройка просто игнорировалась). [#4637](https://github.com/yandex/ClickHouse/pull/4637) ([Alexey Elymanov](https://github.com/digitalist))
|
||||||
|
* Поддержан запрос `CREATE OR REPLACE VIEW`. Позволяет создать `VIEW` или изменить запрос в одном выражении. [#4654](https://github.com/yandex/ClickHouse/pull/4654) ([Boris Granveaud](https://github.com/bgranvea))
|
||||||
|
* Движок таблиц `Buffer` теперь поддерживает `PREWHERE`. [#4671](https://github.com/yandex/ClickHouse/pull/4671) ([Yangkuan Liu](https://github.com/LiuYangkuan))
|
||||||
|
* Теперь реплицируемые таблицы могу стартовать в `readonly` режиме даже при отсутствии zookeeper. [#4691](https://github.com/yandex/ClickHouse/pull/4691) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Исправлено мигание прогресс-бара в clickhouse-client. Проблема была наиболее заметна при использовании `FORMAT Null` в потоковых запросах. [#4811](https://github.com/yandex/ClickHouse/pull/4811) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Добавлена возможность отключения функций, использующих библиотеку `hyperscan`, для пользователей, чтобы ограничить возможное неконтролируемое потребление ресурсов. [#4816](https://github.com/yandex/ClickHouse/pull/4816) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Добавлено логирование номера версии во все исключения. [#4824](https://github.com/yandex/ClickHouse/pull/4824) ([proller](https://github.com/proller))
|
||||||
|
* Добавлено ограничение на размер строк и количество параметров в функции `multiMatch`. Теперь они принимают строки умещающиеся в `unsigned int`. [#4834](https://github.com/yandex/ClickHouse/pull/4834) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Улучшено использование памяти и обработка ошибок в Hyperscan. [#4866](https://github.com/yandex/ClickHouse/pull/4866) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Теперь системная таблица `system.graphite_detentions` заполняется из конфигурационного файла для таблиц семейства `*GraphiteMergeTree`. [#4584](https://github.com/yandex/ClickHouse/pull/4584) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
|
||||||
|
* Функция `trigramDistance` переименована в функцию `ngramDistance`. Добавлено несколько функций с `CaseInsensitive` и `UTF`. [#4602](https://github.com/yandex/ClickHouse/pull/4602) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Улучшено вычисление вторичных индексов. [#4640](https://github.com/yandex/ClickHouse/pull/4640) ([Nikita Vasilev](https://github.com/nikvas0))
|
||||||
|
* Теперь обычные колонки, а также колонки `DEFAULT`, `MATERIALIZED` и `ALIAS` хранятся в одном списке (исправляет issue [#2867](https://github.com/yandex/ClickHouse/issues/2867)). [#4707](https://github.com/yandex/ClickHouse/pull/4707) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
|
||||||
|
### Исправления ошибок
|
||||||
|
|
||||||
|
* В случае невозможности выделить память вместо вызова `std::terminate` бросается исключение `std::bad_alloc`. [#4665](https://github.com/yandex/ClickHouse/pull/4665) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлены ошибки чтения capnproto из буфера. Иногда файлы не загружались по HTTP. [#4674](https://github.com/yandex/ClickHouse/pull/4674) ([Vladislav](https://github.com/smirnov-vs))
|
||||||
|
* Исправлена ошибка `Unknown log entry type: 0` после запроса `OPTIMIZE TABLE FINAL`. [#4683](https://github.com/yandex/ClickHouse/pull/4683) ([Amos Bird](https://github.com/amosbird))
|
||||||
|
* При передаче неправильных аргументов в `hasAny` и `hasAll` могла происходить ошибка сегментирования. [#4698](https://github.com/yandex/ClickHouse/pull/4698) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлен дедлок, который мог происходить при запросе `DROP DATABASE dictionary`. [#4701](https://github.com/yandex/ClickHouse/pull/4701) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено неопределенное поведение в функциях `median` и `quantile`. [#4702](https://github.com/yandex/ClickHouse/pull/4702) ([hcz](https://github.com/hczhcz))
|
||||||
|
* Исправлено определение уровня сжатия при указании настройки `network_compression_method` в нижнем регистре. Было сломано в v19.1. [#4706](https://github.com/yandex/ClickHouse/pull/4706) ([proller](https://github.com/proller))
|
||||||
|
* Настройка `<timezone>UTC</timezone>` больше не игнорируется (исправляет issue [#4658](https://github.com/yandex/ClickHouse/issues/4658)). [#4718](https://github.com/yandex/ClickHouse/pull/4718) ([proller](https://github.com/proller))
|
||||||
|
* Исправлено поведение функции `histogram` с `Distributed` таблицами. [#4741](https://github.com/yandex/ClickHouse/pull/4741) ([olegkv](https://github.com/olegkv))
|
||||||
|
* Исправлено срабатывание thread-санитайзера с ошибкой `destroy of a locked mutex`. [#4742](https://github.com/yandex/ClickHouse/pull/4742) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено срабатывание thread-санитайзера при завершении сервера, вызванное гонкой при использовании системных логов. Также исправлена потенциальная ошибка use-after-free при завершении сервера в котором был включен `part_log`. [#4758](https://github.com/yandex/ClickHouse/pull/4758) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена перепроверка кусков в `ReplicatedMergeTreeAlterThread` при появлении ошибок. [#4772](https://github.com/yandex/ClickHouse/pull/4772) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Исправлена работа арифметических операций с промежуточными состояниями агрегатных функций для константных аргументов (таких как результаты подзапросов). [#4776](https://github.com/yandex/ClickHouse/pull/4776) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Теперь имена колонок всегда экранируются в файлах с метаинформацией. В противном случае было невозможно создать таблицу с колонкой с именем `index`. [#4782](https://github.com/yandex/ClickHouse/pull/4782) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено падение в запросе `ALTER ... MODIFY ORDER BY` к `Distributed` таблице. [#4790](https://github.com/yandex/ClickHouse/pull/4790) ([TCeason](https://github.com/TCeason))
|
||||||
|
* Исправлена ошибка сегментирования при запросах с `JOIN ON` и включенной настройкой `enable_optimize_predicate_expression`. [#4794](https://github.com/yandex/ClickHouse/pull/4794) ([Winter Zhang](https://github.com/zhang2014))
|
||||||
|
* Исправлено добавление лишней строки после чтения protobuf-сообщения из таблицы с движком `Kafka`. [#4808](https://github.com/yandex/ClickHouse/pull/4808) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Исправлено падение при запросе с `JOIN ON` с не `nullable` и nullable колонкой. Также исправлено поведение при появлении `NULLs` среди ключей справа в`ANY JOIN` + `join_use_nulls`. [#4815](https://github.com/yandex/ClickHouse/pull/4815) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлена ошибка сегментирования в `clickhouse-copier`. [#4835](https://github.com/yandex/ClickHouse/pull/4835) ([proller](https://github.com/proller))
|
||||||
|
* Исправлена гонка при `SELECT` запросе из `system.tables` если таблица была конкурентно переименована или к ней был применен `ALTER` запрос. [#4836](https://github.com/yandex/ClickHouse/pull/4836) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена гонка при скачивании куска, который уже является устаревшим. [#4839](https://github.com/yandex/ClickHouse/pull/4839) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена редкая гонка при `RENAME` запросах к таблицам семейства MergeTree. [#4844](https://github.com/yandex/ClickHouse/pull/4844) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена ошибка сегментирования в функции `arrayIntersect`. Ошибка возникала при вызове функции с константными и не константными аргументами. [#4847](https://github.com/yandex/ClickHouse/pull/4847) ([Lixiang Qian](https://github.com/fancyqlx))
|
||||||
|
* Исправлена редкая ошибка при чтении из колонки типа `Array(LowCardinality)`, которая возникала, если в колонке содержалось большее количество подряд идущих пустых массивов. [#4850](https://github.com/yandex/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Исправлено паление в запроса с `FULL/RIGHT JOIN` когда объединение происходило по nullable и не nullable колонке. [#4855](https://github.com/yandex/ClickHouse/pull/4855) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлена ошибка `No message received`, возникавшая при скачивании кусков между репликами. [#4856](https://github.com/yandex/ClickHouse/pull/4856) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Исправлена ошибка в функции `arrayIntersect` приводившая к неправильным результатам в случае нескольких повторяющихся значений в массиве. [#4871](https://github.com/yandex/ClickHouse/pull/4871) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Исправлена гонка при конкурентных `ALTER COLUMN` запросах, которая могла приводить к падению сервера (исправляет issue [#3421](https://github.com/yandex/ClickHouse/issues/3421)). [#4592](https://github.com/yandex/ClickHouse/pull/4592) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* Исправлен некорректный результат в `FULL/RIGHT JOIN` запросах с константной колонкой. [#4723](https://github.com/yandex/ClickHouse/pull/4723) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлено появление дубликатов в `GLOBAL JOIN` со звездочкой. [#4705](https://github.com/yandex/ClickHouse/pull/4705) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлено определение параметров кодеков в запросах `ALTER MODIFY`, если тип колонки не был указан. [#4883](https://github.com/yandex/ClickHouse/pull/4883) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Функции `cutQueryStringAndFragment()` и `queryStringAndFragment()` теперь работают корректно, когда `URL` содержит фрагмент, но не содержит запроса. [#4894](https://github.com/yandex/ClickHouse/pull/4894) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Исправлена редкая ошибка, возникавшая при установке настройки `min_bytes_to_use_direct_io` больше нуля. Она возникла при необходимости сдвинутся в файле, который уже прочитан до конца. [#4897](https://github.com/yandex/ClickHouse/pull/4897) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Исправлено неправильное определение типов аргументов для агрегатных функций с `LowCardinality` аргументами (исправляет [#4919](https://github.com/yandex/ClickHouse/issues/4919)). [#4922](https://github.com/yandex/ClickHouse/pull/4922) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Исправлена неверная квалификация имён в `GLOBAL JOIN`. [#4969](https://github.com/yandex/ClickHouse/pull/4969) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлен результат функции `toISOWeek` для 1970 года. [#4988](https://github.com/yandex/ClickHouse/pull/4988) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено дублирование `DROP`, `TRUNCATE` и `OPTIMIZE` запросов, когда они выполнялись `ON CLUSTER` для семейства таблиц `ReplicatedMergeTree*`. [#4991](https://github.com/yandex/ClickHouse/pull/4991) ([alesapin](https://github.com/alesapin))
|
||||||
|
|
||||||
|
### Обратно несовместимые изменения
|
||||||
|
|
||||||
|
* Настройка `insert_sample_with_metadata` переименована в `input_format_defaults_for_omitted_fields`. [#4771](https://github.com/yandex/ClickHouse/pull/4771) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Добавлена настройка `max_partitions_per_insert_block` (со значением по умолчанию 100). Если вставляемый блок содержит большое количество партиций, то бросается исключение. Лимит можно убрать выставив настройку в 0 (не рекомендуется). [#4845](https://github.com/yandex/ClickHouse/pull/4845) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Функции мультипоиска были переименованы (`multiPosition` в `multiSearchAllPositions`, `multiSearch` в `multiSearchAny`, `firstMatch` в `multiSearchFirstIndex`). [#4780](https://github.com/yandex/ClickHouse/pull/4780) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
|
||||||
|
### Улучшение производительности
|
||||||
|
|
||||||
|
* Оптимизирован поиска с помощью алгоритма Volnitsky с помощью инлайнинга. Это дает около 5-10% улучшения производительности поиска для запросов ищущих множество слов или много одинаковых биграмм. [#4862](https://github.com/yandex/ClickHouse/pull/4862) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Исправлено снижение производительности при выставлении настройки `use_uncompressed_cache` больше нуля для запросов, данные которых целиком лежат в кеше. [#4913](https://github.com/yandex/ClickHouse/pull/4913) ([alesapin](https://github.com/alesapin))
|
||||||
|
|
||||||
|
|
||||||
|
### Улучшения сборки/тестирования/пакетирования
|
||||||
|
|
||||||
|
* Более строгие настройки для debug-сборок: более гранулярные маппинги памяти и использование ASLR; добавлена защита памяти для кеша засечек и индекса. Это позволяет найти больше ошибок порчи памяти, которые не обнаруживают address-санитайзер и thread-санитайзер. [#4632](https://github.com/yandex/ClickHouse/pull/4632) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Добавлены настройки `ENABLE_PROTOBUF`, `ENABLE_PARQUET` и `ENABLE_BROTLI` которые позволяют отключить соответствующие компоненты. [#4669](https://github.com/yandex/ClickHouse/pull/4669) ([Silviu Caragea](https://github.com/silviucpp))
|
||||||
|
* Теперь при зависании запросов во время работы тестов будет показан список запросов и стек-трейсы всех потоков. [#4675](https://github.com/yandex/ClickHouse/pull/4675) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Добавлены ретраи при ошибке `Connection loss` в `clickhouse-test`. [#4682](https://github.com/yandex/ClickHouse/pull/4682) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Добавлена возможность сборки под FreeBSD в `packager`-скрипт. [#4712](https://github.com/yandex/ClickHouse/pull/4712) [#4748](https://github.com/yandex/ClickHouse/pull/4748) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Теперь при установке предлагается установить пароль для пользователя `'default'`. [#4725](https://github.com/yandex/ClickHouse/pull/4725) ([proller](https://github.com/proller))
|
||||||
|
* Убраны предупреждения из библиотеки `rdkafka` при сборке. [#4740](https://github.com/yandex/ClickHouse/pull/4740) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Добавлена возможность сборки без поддержки ssl. [#4750](https://github.com/yandex/ClickHouse/pull/4750) ([proller](https://github.com/proller))
|
||||||
|
* Добавлена возможность запускать докер-образ с clickhouse-server из под любого пользователя. [#4753](https://github.com/yandex/ClickHouse/pull/4753) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
|
||||||
|
* Boost обновлен до 1.69. [#4793](https://github.com/yandex/ClickHouse/pull/4793) ([proller](https://github.com/proller))
|
||||||
|
* Отключено использование `mremap` при сборке с thread-санитайзером, что приводило к ложным срабатываниям. Исправлены ошибки thread-санитайзера в stateful-тестах. [#4859](https://github.com/yandex/ClickHouse/pull/4859) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Добавлен тест проверяющий использование схемы форматов для HTTP-интерфейса. [#4864](https://github.com/yandex/ClickHouse/pull/4864) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
|
||||||
|
## ClickHouse release 19.4.4.33, 2019-04-17
|
||||||
|
|
||||||
|
### Исправление ошибок
|
||||||
|
* В случае невозможности выделить память вместо вызова `std::terminate` бросается исключение `std::bad_alloc`. [#4665](https://github.com/yandex/ClickHouse/pull/4665) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлены ошибки чтения capnproto из буфера. Иногда файлы не загружались по HTTP. [#4674](https://github.com/yandex/ClickHouse/pull/4674) ([Vladislav](https://github.com/smirnov-vs))
|
||||||
|
* Исправлена ошибка `Unknown log entry type: 0` после запроса `OPTIMIZE TABLE FINAL`. [#4683](https://github.com/yandex/ClickHouse/pull/4683) ([Amos Bird](https://github.com/amosbird))
|
||||||
|
* При передаче неправильных аргументов в `hasAny` и `hasAll` могла происходить ошибка сегментирования. [#4698](https://github.com/yandex/ClickHouse/pull/4698) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлен дедлок, который мог происходить при запросе `DROP DATABASE dictionary`. [#4701](https://github.com/yandex/ClickHouse/pull/4701) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено неопределенное поведение в функциях `median` и `quantile`. [#4702](https://github.com/yandex/ClickHouse/pull/4702) ([hcz](https://github.com/hczhcz))
|
||||||
|
* Исправлено определение уровня сжатия при указании настройки `network_compression_method` в нижнем регистре. Было сломано в v19.1. [#4706](https://github.com/yandex/ClickHouse/pull/4706) ([proller](https://github.com/proller))
|
||||||
|
* Настройка `<timezone>UTC</timezone>` больше не игнорируется (исправляет issue [#4658](https://github.com/yandex/ClickHouse/issues/4658)). [#4718](https://github.com/yandex/ClickHouse/pull/4718) ([proller](https://github.com/proller))
|
||||||
|
* Исправлено поведение функции `histogram` с `Distributed` таблицами. [#4741](https://github.com/yandex/ClickHouse/pull/4741) ([olegkv](https://github.com/olegkv))
|
||||||
|
* Исправлено срабатывание thread-санитайзера с ошибкой `destroy of a locked mutex`. [#4742](https://github.com/yandex/ClickHouse/pull/4742) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено срабатывание thread-санитайзера при завершении сервера, вызванное гонкой при использовании системных логов. Также исправлена потенциальная ошибка use-after-free при завершении сервера в котором был включен `part_log`. [#4758](https://github.com/yandex/ClickHouse/pull/4758) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена перепроверка кусков в `ReplicatedMergeTreeAlterThread` при появлении ошибок. [#4772](https://github.com/yandex/ClickHouse/pull/4772) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Исправлена работа арифметических операций с промежуточными состояниями агрегатных функций для константных аргументов (таких как результаты подзапросов). [#4776](https://github.com/yandex/ClickHouse/pull/4776) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Теперь имена колонок всегда экранируются в файлах с метаинформацией. В противном случае было невозможно создать таблицу с колонкой с именем `index`. [#4782](https://github.com/yandex/ClickHouse/pull/4782) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено падение в запросе `ALTER ... MODIFY ORDER BY` к `Distributed` таблице. [#4790](https://github.com/yandex/ClickHouse/pull/4790) ([TCeason](https://github.com/TCeason))
|
||||||
|
* Исправлена ошибка сегментирования при запросах с `JOIN ON` и включенной настройкой `enable_optimize_predicate_expression`. [#4794](https://github.com/yandex/ClickHouse/pull/4794) ([Winter Zhang](https://github.com/zhang2014))
|
||||||
|
* Исправлено добавление лишней строки после чтения protobuf-сообщения из таблицы с движком `Kafka`. [#4808](https://github.com/yandex/ClickHouse/pull/4808) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Исправлена ошибка сегментирования в `clickhouse-copier`. [#4835](https://github.com/yandex/ClickHouse/pull/4835) ([proller](https://github.com/proller))
|
||||||
|
* Исправлена гонка при `SELECT` запросе из `system.tables` если таблица была конкурентно переименована или к ней был применен `ALTER` запрос. [#4836](https://github.com/yandex/ClickHouse/pull/4836) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена гонка при скачивании куска, который уже является устаревшим. [#4839](https://github.com/yandex/ClickHouse/pull/4839) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена редкая гонка при `RENAME` запросах к таблицам семейства MergeTree. [#4844](https://github.com/yandex/ClickHouse/pull/4844) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена ошибка сегментирования в функции `arrayIntersect`. Ошибка возникала при вызове функции с константными и не константными аргументами. [#4847](https://github.com/yandex/ClickHouse/pull/4847) ([Lixiang Qian](https://github.com/fancyqlx))
|
||||||
|
* Исправлена редкая ошибка при чтении из колонки типа `Array(LowCardinality)`, которая возникала, если в колонке содержалось большее количество подряд идущих пустых массивов. [#4850](https://github.com/yandex/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Исправлена ошибка `No message received`, возникавшая при скачивании кусков между репликами. [#4856](https://github.com/yandex/ClickHouse/pull/4856) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Исправлена ошибка в функции `arrayIntersect` приводившая к неправильным результатам в случае нескольких повторяющихся значений в массиве. [#4871](https://github.com/yandex/ClickHouse/pull/4871) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Исправлена гонка при конкурентных `ALTER COLUMN` запросах, которая могла приводить к падению сервера (исправляет issue [#3421](https://github.com/yandex/ClickHouse/issues/3421)). [#4592](https://github.com/yandex/ClickHouse/pull/4592) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* Исправлено определение параметров кодеков в запросах `ALTER MODIFY`, если тип колонки не был указан. [#4883](https://github.com/yandex/ClickHouse/pull/4883) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Функции `cutQueryStringAndFragment()` и `queryStringAndFragment()` теперь работают корректно, когда `URL` содержит фрагмент, но не содержит запроса. [#4894](https://github.com/yandex/ClickHouse/pull/4894) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Исправлена редкая ошибка, возникавшая при установке настройки `min_bytes_to_use_direct_io` больше нуля. Она возникла при необходимости сдвинутся в файле, который уже прочитан до конца. [#4897](https://github.com/yandex/ClickHouse/pull/4897) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Исправлено неправильное определение типов аргументов для агрегатных функций с `LowCardinality` аргументами (исправляет [#4919](https://github.com/yandex/ClickHouse/issues/4919)). [#4922](https://github.com/yandex/ClickHouse/pull/4922) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Исправлен результат функции `toISOWeek` для 1970 года. [#4988](https://github.com/yandex/ClickHouse/pull/4988) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено дублирование `DROP`, `TRUNCATE` и `OPTIMIZE` запросов, когда они выполнялись `ON CLUSTER` для семейства таблиц `ReplicatedMergeTree*`. [#4991](https://github.com/yandex/ClickHouse/pull/4991) ([alesapin](https://github.com/alesapin))
|
||||||
|
|
||||||
|
### Улучшения
|
||||||
|
|
||||||
|
* Теперь обычные колонки, а также колонки `DEFAULT`, `MATERIALIZED` и `ALIAS` хранятся в одном списке (исправляет issue [#2867](https://github.com/yandex/ClickHouse/issues/2867)). [#4707](https://github.com/yandex/ClickHouse/pull/4707) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
|
||||||
|
## ClickHouse release 19.4.3.11, 2019-04-02
|
||||||
|
|
||||||
|
### Исправление ошибок
|
||||||
|
|
||||||
|
* Исправлено паление в запроса с `FULL/RIGHT JOIN` когда объединение происходило по nullable и не nullable колонке. [#4855](https://github.com/yandex/ClickHouse/pull/4855) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлена ошибка сегментирования в `clickhouse-copier`. [#4835](https://github.com/yandex/ClickHouse/pull/4835) ([proller](https://github.com/proller))
|
||||||
|
|
||||||
|
### Улучшения сборки/тестирования/пакетирования
|
||||||
|
|
||||||
|
* Добавлена возможность запускать докер-образ с clickhouse-server из под любого пользователя. [#4753](https://github.com/yandex/ClickHouse/pull/4753) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
|
||||||
|
|
||||||
|
## ClickHouse release 19.4.2.7, 2019-03-30
|
||||||
|
|
||||||
|
### Исправление ошибок
|
||||||
|
* Исправлена редкая ошибка при чтении из колонки типа `Array(LowCardinality)`, которая возникала, если в колонке содержалось большее количество подряд идущих пустых массивов. [#4850](https://github.com/yandex/ClickHouse/pull/4850) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
|
||||||
|
## ClickHouse release 19.4.1.3, 2019-03-19
|
||||||
|
|
||||||
|
### Исправление ошибок
|
||||||
|
* Исправлено поведение удаленных запросов, которые одновременно содержали `LIMIT BY` и `LIMIT`. Раньше для таких запросов `LIMIT` мог быть выполнен до `LIMIT BY`, что приводило к перефильтрации. [#4708](https://github.com/yandex/ClickHouse/pull/4708) ([Constantin S. Pan](https://github.com/kvap))
|
||||||
|
|
||||||
|
## ClickHouse release 19.4.0.49, 2019-03-09
|
||||||
|
|
||||||
|
### Новые возможности
|
||||||
|
* Добавлена полная поддержка формата `Protobuf` (чтение и запись, вложенные структуры данных). [#4174](https://github.com/yandex/ClickHouse/pull/4174) [#4493](https://github.com/yandex/ClickHouse/pull/4493) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Добавлены функции для работы с битовыми масками с использованием библиотеки Roaring Bitmaps. [#4207](https://github.com/yandex/ClickHouse/pull/4207) ([Andy Yang](https://github.com/andyyzh)) [#4568](https://github.com/yandex/ClickHouse/pull/4568) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Поддержка формата `Parquet` [#4448](https://github.com/yandex/ClickHouse/pull/4448) ([proller](https://github.com/proller))
|
||||||
|
* Вычисление расстояния между строками с помощью подсчёта N-грам - для приближённого сравнения строк. Алгоритм похож на q-gram metrics в языке R. [#4466](https://github.com/yandex/ClickHouse/pull/4466) ([Danila Kutenin](https://github.com/danlark1))
|
||||||
|
* Движок таблиц GraphiteMergeTree поддерживает отдельные шаблоны для правил агрегации и для правил времени хранения. [#4426](https://github.com/yandex/ClickHouse/pull/4426) ([Mikhail f. Shiryaev](https://github.com/Felixoid))
|
||||||
|
* Добавлены настройки `max_execution_speed` и `max_execution_speed_bytes` для того, чтобы ограничить потребление ресурсов запросами. Добавлена настройка `min_execution_speed_bytes` в дополнение к `min_execution_speed`. [#4430](https://github.com/yandex/ClickHouse/pull/4430) ([Winter Zhang](https://github.com/zhang2014))
|
||||||
|
* Добавлена функция `flatten` - конвертация многомерных массивов в плоский массив. [#4555](https://github.com/yandex/ClickHouse/pull/4555) [#4409](https://github.com/yandex/ClickHouse/pull/4409) ([alexey-milovidov](https://github.com/alexey-milovidov), [kzon](https://github.com/kzon))
|
||||||
|
* Добавлены функции `arrayEnumerateDenseRanked` и `arrayEnumerateUniqRanked` (похожа на `arrayEnumerateUniq` но позволяет указать глубину, на которую следует смотреть в многомерные массивы). [#4475](https://github.com/yandex/ClickHouse/pull/4475) ([proller](https://github.com/proller)) [#4601](https://github.com/yandex/ClickHouse/pull/4601) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Добавлена поддержка множества JOIN в одном запросе без подзапросов, с некоторыми ограничениями: без звёздочки и без алиасов сложных выражений в ON/WHERE/GROUP BY/... [#4462](https://github.com/yandex/ClickHouse/pull/4462) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
|
||||||
|
### Исправления ошибок
|
||||||
|
* Этот релиз также содержит все исправления из 19.3 и 19.1.
|
||||||
|
* Исправлена ошибка во вторичных индексах (экспериментальная возможность): порядок гранул при INSERT был неверным. [#4407](https://github.com/yandex/ClickHouse/pull/4407) ([Nikita Vasilev](https://github.com/nikvas0))
|
||||||
|
* Исправлена работа вторичного индекса (экспериментальная возможность) типа `set` для столбцов типа `Nullable` и `LowCardinality`. Ранее их использование вызывало ошибку `Data type must be deserialized with multiple streams` при запросе SELECT. [#4594](https://github.com/yandex/ClickHouse/pull/4594) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
|
||||||
|
* Правильное запоминание времени последнего обновления при полной перезагрузке словарей типа `executable`. [#4551](https://github.com/yandex/ClickHouse/pull/4551) ([Tema Novikov](https://github.com/temoon))
|
||||||
|
* Исправлена неработоспособность прогресс-бара, возникшая в версии 19.3 [#4627](https://github.com/yandex/ClickHouse/pull/4627) ([filimonov](https://github.com/filimonov))
|
||||||
|
* Исправлены неправильные значения MemoryTracker, если кусок памяти был уменьшен в размере, в очень редких случаях. [#4619](https://github.com/yandex/ClickHouse/pull/4619) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено undefined behaviour в ThreadPool [#4612](https://github.com/yandex/ClickHouse/pull/4612) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено очень редкое падение с сообщением `mutex lock failed: Invalid argument`, которое могло произойти, если таблица типа MergeTree удалялась одновременно с SELECT. [#4608](https://github.com/yandex/ClickHouse/pull/4608) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* Совместимость ODBC драйвера с типом данных `LowCardinality` [#4381](https://github.com/yandex/ClickHouse/pull/4381) ([proller](https://github.com/proller))
|
||||||
|
* Исправление ошибки `AIOcontextPool: Found io_event with unknown id 0` под ОС FreeBSD [#4438](https://github.com/yandex/ClickHouse/pull/4438) ([urgordeadbeef](https://github.com/urgordeadbeef))
|
||||||
|
* Таблица `system.part_log` создавалась независимо от того, была ли она объявлена в конфигурации. [#4483](https://github.com/yandex/ClickHouse/pull/4483) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлено undefined behaviour в функции `dictIsIn` для словарей типа `cache`. [#4515](https://github.com/yandex/ClickHouse/pull/4515) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Исправлен deadlock в случае, если запрос SELECT блокирует одну и ту же таблицу несколько раз (например - из разных потоков, либо при выполнении разных подзапросов) и одновременно с этим производится DDL запрос. [#4535](https://github.com/yandex/ClickHouse/pull/4535) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* Настройка `compile_expressions` выключена по-умолчанию до тех пор, пока мы не зафиксируем исходники используемой библиотеки `LLVM` и не будем проверять её под `ASan` (сейчас библиотека LLVM берётся из системы). [#4579](https://github.com/yandex/ClickHouse/pull/4579) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Исправлено падение по `std::terminate`, если `invalidate_query` для внешних словарей с источником `clickhouse` вернул неправильный результат (пустой; более чем одну строку; более чем один столбец). Исправлена ошибка, из-за которой запрос `invalidate_query` производился каждые пять секунд, независимо от указанного `lifetime`. [#4583](https://github.com/yandex/ClickHouse/pull/4583) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлен deadlock в случае, если запрос `invalidate_query` для внешнего словаря с источником `clickhouse` использовал таблицу `system.dictionaries` или базу данных типа `Dictionary` (редкий случай). [#4599](https://github.com/yandex/ClickHouse/pull/4599) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена работа CROSS JOIN с пустым WHERE [#4598](https://github.com/yandex/ClickHouse/pull/4598) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлен segfault в функции `replicate` с константным аргументом. [#4603](https://github.com/yandex/ClickHouse/pull/4603) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена работа predicate pushdown (настройка `enable_optimize_predicate_expression`) с лямбда-функциями. [#4408](https://github.com/yandex/ClickHouse/pull/4408) ([Winter Zhang](https://github.com/zhang2014))
|
||||||
|
* Множественные исправления для множества JOIN в одном запросе. [#4595](https://github.com/yandex/ClickHouse/pull/4595) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
|
||||||
|
### Улучшения
|
||||||
|
* Поддержка алиасов в секции JOIN ON для правой таблицы [#4412](https://github.com/yandex/ClickHouse/pull/4412) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Используются правильные алиасы в случае множественных JOIN с подзапросами. [#4474](https://github.com/yandex/ClickHouse/pull/4474) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлена логика работы predicate pushdown (настройка `enable_optimize_predicate_expression`) для JOIN. [#4387](https://github.com/yandex/ClickHouse/pull/4387) ([Ivan](https://github.com/abyss7))
|
||||||
|
|
||||||
|
### Улучшения производительности
|
||||||
|
* Улучшена эвристика оптимизации "перенос в PREWHERE". [#4405](https://github.com/yandex/ClickHouse/pull/4405) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Используются настоящие lookup таблицы вместо хэш-таблиц в случае 8 и 16 битных ключей. Интерфейс хэш-таблиц обобщён, чтобы поддерживать этот случай. [#4536](https://github.com/yandex/ClickHouse/pull/4536) ([Amos Bird](https://github.com/amosbird))
|
||||||
|
* Улучшена производительность сравнения строк. [#4564](https://github.com/yandex/ClickHouse/pull/4564) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Очередь DDL операций (для запросов ON CLUSTER) очищается в отдельном потоке, чтобы не замедлять основную работу. [#4502](https://github.com/yandex/ClickHouse/pull/4502) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* Даже если настройка `min_bytes_to_use_direct_io` выставлена в 1, не каждый файл открывался в режиме O_DIRECT, потому что размер файлов иногда недооценивался на размер одного сжатого блока. [#4526](https://github.com/yandex/ClickHouse/pull/4526) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
### Улучшения сборки/тестирования/пакетирования
|
||||||
|
* Добавлена поддержка компилятора clang-9 [#4604](https://github.com/yandex/ClickHouse/pull/4604) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлены неправильные `__asm__` инструкции [#4621](https://github.com/yandex/ClickHouse/pull/4621) ([Konstantin Podshumok](https://github.com/podshumok))
|
||||||
|
* Добавлена поддержка задания настроек выполнения запросов для `clickhouse-performance-test` из командной строки. [#4437](https://github.com/yandex/ClickHouse/pull/4437) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Тесты словарей перенесены в интеграционные тесты. [#4477](https://github.com/yandex/ClickHouse/pull/4477) ([alesapin](https://github.com/alesapin))
|
||||||
|
* В набор автоматизированных тестов производительности добавлены запросы, находящиеся в разделе "benchmark" на официальном сайте. [#4496](https://github.com/yandex/ClickHouse/pull/4496) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправления сборки в случае использования внешних библиотек lz4 и xxhash. [#4495](https://github.com/yandex/ClickHouse/pull/4495) ([Orivej Desh](https://github.com/orivej))
|
||||||
|
* Исправлен undefined behaviour, если функция `quantileTiming` была вызвана с отрицательным или нецелым аргументом (обнаружено с помощью fuzz test под undefined behaviour sanitizer). [#4506](https://github.com/yandex/ClickHouse/pull/4506) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлены опечатки в коде. [#4531](https://github.com/yandex/ClickHouse/pull/4531) ([sdk2](https://github.com/sdk2))
|
||||||
|
* Исправлена сборка под Mac. [#4371](https://github.com/yandex/ClickHouse/pull/4371) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
|
* Исправлена сборка под FreeBSD и для некоторых необычных конфигурациях сборки. [#4444](https://github.com/yandex/ClickHouse/pull/4444) ([proller](https://github.com/proller))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.3.7, 2019-03-12
|
||||||
|
|
||||||
|
### Исправления ошибок
|
||||||
|
|
||||||
|
* Исправлена ошибка в #3920. Ошибка проявлялась в виде случайных повреждений кэша (сообщения `Unknown codec family code`, `Cannot seek through file`) и segfault. Ошибка впервые возникла в 19.1 и присутствует во всех версиях до 19.1.10 и 19.3.6. [#4623](https://github.com/yandex/ClickHouse/pull/4623) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
|
||||||
|
## ClickHouse release 19.3.6, 2019-03-02
|
||||||
|
|
||||||
|
### Исправления ошибок
|
||||||
|
|
||||||
|
* Если в пуле потоков было более 1000 потоков, то при выходе из потока, вызывается `std::terminate`. [Azat Khuzhin](https://github.com/azat) [#4485](https://github.com/yandex/ClickHouse/pull/4485) [#4505](https://github.com/yandex/ClickHouse/pull/4505) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Теперь возможно создавать таблицы `ReplicatedMergeTree*` с комментариями столбцов без указания DEFAULT, а также с CODEC но без COMMENT и DEFAULT. Исправлено сравнение CODEC друг с другом. [#4523](https://github.com/yandex/ClickHouse/pull/4523) ([alesapin](https://github.com/alesapin))
|
||||||
|
* Исправлено падение при JOIN по массивам и кортежам. [#4552](https://github.com/yandex/ClickHouse/pull/4552) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлено падение `clickhouse-copier` с сообщением `ThreadStatus not created`. [#4540](https://github.com/yandex/ClickHouse/pull/4540) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
* Исправлено зависание сервера при завершении работы в случае использования распределённых DDL. [#4472](https://github.com/yandex/ClickHouse/pull/4472) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
* В сообщениях об ошибке при парсинге текстовых форматов, выдавались неправильные номера столбцов, в случае, если номер больше 10. [#4484](https://github.com/yandex/ClickHouse/pull/4484) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
|
||||||
|
### Улучшения сборки/тестирования/пакетирования
|
||||||
|
|
||||||
|
* Исправлена сборка с включенным AVX. [#4527](https://github.com/yandex/ClickHouse/pull/4527) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
|
* Исправлена поддержка расширенных метрик выполнения запроса в случае, если ClickHouse был собран на системе с новым ядром Linux, а запускается на системе с существенно более старым ядром. [#4541](https://github.com/yandex/ClickHouse/pull/4541) ([nvartolomei](https://github.com/nvartolomei))
|
||||||
|
* Продолжение работы в случае невозможности применить настройку `core_dump.size_limit` с выводом предупреждения. [#4473](https://github.com/yandex/ClickHouse/pull/4473) ([proller](https://github.com/proller))
|
||||||
|
* Удалено `inline` для `void readBinary(...)` в `Field.cpp`. [#4530](https://github.com/yandex/ClickHouse/pull/4530) ([hcz](https://github.com/hczhcz))
|
||||||
|
|
||||||
|
|
||||||
## ClickHouse release 19.3.5, 2019-02-21
|
## ClickHouse release 19.3.5, 2019-02-21
|
||||||
|
|
||||||
### Исправления ошибок:
|
### Исправления ошибок:
|
||||||
@ -39,7 +402,7 @@
|
|||||||
* Добавлена поддержка `Nullable` типов в табличной функции `mysql`. [#4198](https://github.com/yandex/ClickHouse/pull/4198) ([Emmanuel Donin de Rosière](https://github.com/edonin))
|
* Добавлена поддержка `Nullable` типов в табличной функции `mysql`. [#4198](https://github.com/yandex/ClickHouse/pull/4198) ([Emmanuel Donin de Rosière](https://github.com/edonin))
|
||||||
* Добавлена поддержка произвольных константных выражений в секции `LIMIT`. [#4246](https://github.com/yandex/ClickHouse/pull/4246) ([k3box](https://github.com/k3box))
|
* Добавлена поддержка произвольных константных выражений в секции `LIMIT`. [#4246](https://github.com/yandex/ClickHouse/pull/4246) ([k3box](https://github.com/k3box))
|
||||||
* Добавлена агрегатная функция `topKWeighted` - вариант `topK`, позволяющий задавать (целый неотрицательный) вес добавляемого значения. [#4245](https://github.com/yandex/ClickHouse/pull/4245) ([Andrew Golman](https://github.com/andrewgolman))
|
* Добавлена агрегатная функция `topKWeighted` - вариант `topK`, позволяющий задавать (целый неотрицательный) вес добавляемого значения. [#4245](https://github.com/yandex/ClickHouse/pull/4245) ([Andrew Golman](https://github.com/andrewgolman))
|
||||||
* Движок `Join` теперь поддерживает настройку `join_overwrite`, которая позволяет перезаписывать значения для существующих ключей. [#3973](https://github.com/yandex/ClickHouse/pull/3973) ([Amos Bird](https://github.com/amosbird))
|
* Движок `Join` теперь поддерживает настройку `join_any_take_last_row`, которая позволяет перезаписывать значения для существующих ключей. [#3973](https://github.com/yandex/ClickHouse/pull/3973) ([Amos Bird](https://github.com/amosbird))
|
||||||
* Добавлена функция `toStartOfInterval`. [#4304](https://github.com/yandex/ClickHouse/pull/4304) ([Vitaly Baranov](https://github.com/vitlibar))
|
* Добавлена функция `toStartOfInterval`. [#4304](https://github.com/yandex/ClickHouse/pull/4304) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
* Добавлена функция `toStartOfTenMinutes`. [#4298](https://github.com/yandex/ClickHouse/pull/4298) ([Vitaly Baranov](https://github.com/vitlibar))
|
* Добавлена функция `toStartOfTenMinutes`. [#4298](https://github.com/yandex/ClickHouse/pull/4298) ([Vitaly Baranov](https://github.com/vitlibar))
|
||||||
* Добавлен формат `RowBinaryWithNamesAndTypes`. [#4200](https://github.com/yandex/ClickHouse/pull/4200) ([Oleg V. Kozlyuk](https://github.com/DarkWanderer))
|
* Добавлен формат `RowBinaryWithNamesAndTypes`. [#4200](https://github.com/yandex/ClickHouse/pull/4200) ([Oleg V. Kozlyuk](https://github.com/DarkWanderer))
|
||||||
@ -133,6 +496,18 @@
|
|||||||
* Уменьшено время ожидания завершения сервера и завершения запросов `ALTER`. [#4372](https://github.com/yandex/ClickHouse/pull/4372) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
* Уменьшено время ожидания завершения сервера и завершения запросов `ALTER`. [#4372](https://github.com/yandex/ClickHouse/pull/4372) ([alexey-milovidov](https://github.com/alexey-milovidov))
|
||||||
* Добавлена информация о значении настройки `replicated_can_become_leader` в таблицу `system.replicas`. Добавлено логирование в случае, если реплика не собирается стать лидером. [#4379](https://github.com/yandex/ClickHouse/pull/4379) ([Alex Zatelepin](https://github.com/ztlpn))
|
* Добавлена информация о значении настройки `replicated_can_become_leader` в таблицу `system.replicas`. Добавлено логирование в случае, если реплика не собирается стать лидером. [#4379](https://github.com/yandex/ClickHouse/pull/4379) ([Alex Zatelepin](https://github.com/ztlpn))
|
||||||
|
|
||||||
|
## ClickHouse release 19.1.14, 2019-03-14
|
||||||
|
|
||||||
|
* Исправлена ошибка `Column ... queried more than once`, которая могла произойти в случае включенной настройки `asterisk_left_columns_only` в случае использования `GLOBAL JOIN` а также `SELECT *` (редкий случай). Эта ошибка изначально отсутствует в версиях 19.3 и более новых. [6bac7d8d](https://github.com/yandex/ClickHouse/pull/4692/commits/6bac7d8d11a9b0d6de0b32b53c47eb2f6f8e7062) ([Artem Zuikov](https://github.com/4ertus2))
|
||||||
|
|
||||||
|
## ClickHouse release 19.1.13, 2019-03-12
|
||||||
|
|
||||||
|
Этот релиз содержит такие же исправления ошибок, как и 19.3.7.
|
||||||
|
|
||||||
|
## ClickHouse release 19.1.10, 2019-03-03
|
||||||
|
|
||||||
|
Этот релиз содержит такие же исправления ошибок, как и 19.3.6.
|
||||||
|
|
||||||
## ClickHouse release 19.1.9, 2019-02-21
|
## ClickHouse release 19.1.9, 2019-02-21
|
||||||
|
|
||||||
### Исправления ошибок:
|
### Исправления ошибок:
|
||||||
@ -302,7 +677,7 @@
|
|||||||
|
|
||||||
### Новые возможности:
|
### Новые возможности:
|
||||||
|
|
||||||
* Вычисление `DEFAULT` выражений для отсутствующих полей при загрузке данных в полуструктурированных форматах (`JSONEachRow`, `TSKV`). [#3555](https://github.com/yandex/ClickHouse/pull/3555)
|
* Вычисление `DEFAULT` выражений для отсутствующих полей при загрузке данных в полуструктурированных форматах (`JSONEachRow`, `TSKV`) (требуется включить настройку запроса `insert_sample_with_metadata`). [#3555](https://github.com/yandex/ClickHouse/pull/3555)
|
||||||
* Для запроса `ALTER TABLE` добавлено действие `MODIFY ORDER BY` для изменения ключа сортировки при одновременном добавлении или удалении столбца таблицы. Это полезно для таблиц семейства `MergeTree`, выполняющих дополнительную работу при слияниях, согласно этому ключу сортировки, как например, `SummingMergeTree`, `AggregatingMergeTree` и т. п. [#3581](https://github.com/yandex/ClickHouse/pull/3581) [#3755](https://github.com/yandex/ClickHouse/pull/3755)
|
* Для запроса `ALTER TABLE` добавлено действие `MODIFY ORDER BY` для изменения ключа сортировки при одновременном добавлении или удалении столбца таблицы. Это полезно для таблиц семейства `MergeTree`, выполняющих дополнительную работу при слияниях, согласно этому ключу сортировки, как например, `SummingMergeTree`, `AggregatingMergeTree` и т. п. [#3581](https://github.com/yandex/ClickHouse/pull/3581) [#3755](https://github.com/yandex/ClickHouse/pull/3755)
|
||||||
* Для таблиц семейства `MergeTree` появилась возможность указать различный ключ сортировки (`ORDER BY`) и индекс (`PRIMARY KEY`). Ключ сортировки может быть длиннее, чем индекс. [#3581](https://github.com/yandex/ClickHouse/pull/3581)
|
* Для таблиц семейства `MergeTree` появилась возможность указать различный ключ сортировки (`ORDER BY`) и индекс (`PRIMARY KEY`). Ключ сортировки может быть длиннее, чем индекс. [#3581](https://github.com/yandex/ClickHouse/pull/3581)
|
||||||
* Добавлена табличная функция `hdfs` и движок таблиц `HDFS` для импорта и экспорта данных в HDFS. [chenxing-xc](https://github.com/yandex/ClickHouse/pull/3617)
|
* Добавлена табличная функция `hdfs` и движок таблиц `HDFS` для импорта и экспорта данных в HDFS. [chenxing-xc](https://github.com/yandex/ClickHouse/pull/3617)
|
||||||
|
|||||||
152
CMakeLists.txt
152
CMakeLists.txt
@ -1,7 +1,20 @@
|
|||||||
project(ClickHouse)
|
project(ClickHouse)
|
||||||
cmake_minimum_required(VERSION 3.3)
|
cmake_minimum_required(VERSION 3.3)
|
||||||
|
|
||||||
|
foreach(policy
|
||||||
|
CMP0023
|
||||||
|
CMP0074 # CMake 3.12
|
||||||
|
)
|
||||||
|
if(POLICY ${policy})
|
||||||
|
cmake_policy(SET ${policy} NEW)
|
||||||
|
endif()
|
||||||
|
endforeach()
|
||||||
|
|
||||||
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/cmake/Modules/")
|
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/cmake/Modules/")
|
||||||
|
set(CMAKE_EXPORT_COMPILE_COMMANDS 1) # Write compile_commands.json
|
||||||
|
set(CMAKE_LINK_DEPENDS_NO_SHARED 1) # Do not relink all depended targets on .so
|
||||||
|
set(CMAKE_CONFIGURATION_TYPES "RelWithDebInfo;Debug;Release;MinSizeRel" CACHE STRING "" FORCE)
|
||||||
|
set(CMAKE_DEBUG_POSTFIX "d" CACHE STRING "Generate debug library name with a postfix.") # To be consistent with CMakeLists from contrib libs.
|
||||||
|
|
||||||
option(ENABLE_IPO "Enable inter-procedural optimization (aka LTO)" OFF) # need cmake 3.9+
|
option(ENABLE_IPO "Enable inter-procedural optimization (aka LTO)" OFF) # need cmake 3.9+
|
||||||
if(ENABLE_IPO)
|
if(ENABLE_IPO)
|
||||||
@ -37,9 +50,6 @@ if (EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/.git" AND NOT EXISTS "${ClickHouse_SOURC
|
|||||||
message (FATAL_ERROR "Submodules are not initialized. Run\n\tgit submodule update --init --recursive")
|
message (FATAL_ERROR "Submodules are not initialized. Run\n\tgit submodule update --init --recursive")
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
# Write compile_commands.json
|
|
||||||
set(CMAKE_EXPORT_COMPILE_COMMANDS 1)
|
|
||||||
|
|
||||||
include (cmake/find_ccache.cmake)
|
include (cmake/find_ccache.cmake)
|
||||||
|
|
||||||
if (NOT CMAKE_BUILD_TYPE OR CMAKE_BUILD_TYPE STREQUAL "None")
|
if (NOT CMAKE_BUILD_TYPE OR CMAKE_BUILD_TYPE STREQUAL "None")
|
||||||
@ -49,7 +59,6 @@ endif ()
|
|||||||
string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_UC)
|
string(TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_UC)
|
||||||
message (STATUS "CMAKE_BUILD_TYPE: ${CMAKE_BUILD_TYPE}")
|
message (STATUS "CMAKE_BUILD_TYPE: ${CMAKE_BUILD_TYPE}")
|
||||||
|
|
||||||
set (CMAKE_CONFIGURATION_TYPES "RelWithDebInfo;Debug;Release;MinSizeRel" CACHE STRING "" FORCE)
|
|
||||||
|
|
||||||
option (USE_STATIC_LIBRARIES "Set to FALSE to use shared libraries" ON)
|
option (USE_STATIC_LIBRARIES "Set to FALSE to use shared libraries" ON)
|
||||||
option (MAKE_STATIC_LIBRARIES "Set to FALSE to make shared libraries" ${USE_STATIC_LIBRARIES})
|
option (MAKE_STATIC_LIBRARIES "Set to FALSE to make shared libraries" ${USE_STATIC_LIBRARIES})
|
||||||
@ -58,8 +67,12 @@ if (NOT MAKE_STATIC_LIBRARIES)
|
|||||||
option (CLICKHOUSE_SPLIT_BINARY "Make several binaries instead one bundled (clickhouse-server, clickhouse-client, ... )" OFF)
|
option (CLICKHOUSE_SPLIT_BINARY "Make several binaries instead one bundled (clickhouse-server, clickhouse-client, ... )" OFF)
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (SPLIT_SHARED_LIBRARIES)
|
if (MAKE_STATIC_LIBRARIES AND SPLIT_SHARED_LIBRARIES)
|
||||||
set (LINK_MODE SHARED)
|
message(FATAL_ERROR "Defining SPLIT_SHARED_LIBRARIES=1 without MAKE_STATIC_LIBRARIES=0 has no effect.")
|
||||||
|
endif()
|
||||||
|
|
||||||
|
if (NOT MAKE_STATIC_LIBRARIES AND SPLIT_SHARED_LIBRARIES)
|
||||||
|
set(BUILD_SHARED_LIBS 1 CACHE INTERNAL "")
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (USE_STATIC_LIBRARIES)
|
if (USE_STATIC_LIBRARIES)
|
||||||
@ -84,6 +97,11 @@ if (COMPILER_GCC OR COMPILER_CLANG)
|
|||||||
set (CXX_WARNING_FLAGS "${CXX_WARNING_FLAGS} -Wnon-virtual-dtor")
|
set (CXX_WARNING_FLAGS "${CXX_WARNING_FLAGS} -Wnon-virtual-dtor")
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
|
if (COMPILER_GCC AND CMAKE_CXX_COMPILER_VERSION VERSION_GREATER "8.3.0")
|
||||||
|
# Warnings in protobuf generating
|
||||||
|
set (CXX_WARNING_FLAGS "${CXX_WARNING_FLAGS} -Wno-array-bounds")
|
||||||
|
endif ()
|
||||||
|
|
||||||
if (CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
|
if (CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
|
||||||
# clang: warning: argument unused during compilation: '-stdlib=libc++'
|
# clang: warning: argument unused during compilation: '-stdlib=libc++'
|
||||||
# clang: warning: argument unused during compilation: '-specs=/usr/share/dpkg/no-pie-compile.specs' [-Wunused-command-line-argument]
|
# clang: warning: argument unused during compilation: '-specs=/usr/share/dpkg/no-pie-compile.specs' [-Wunused-command-line-argument]
|
||||||
@ -96,12 +114,8 @@ option (ENABLE_TESTS "Enables tests" ON)
|
|||||||
if (CMAKE_SYSTEM_PROCESSOR MATCHES "amd64|x86_64")
|
if (CMAKE_SYSTEM_PROCESSOR MATCHES "amd64|x86_64")
|
||||||
option (USE_INTERNAL_MEMCPY "Use internal implementation of 'memcpy' function instead of provided by libc. Only for x86_64." ON)
|
option (USE_INTERNAL_MEMCPY "Use internal implementation of 'memcpy' function instead of provided by libc. Only for x86_64." ON)
|
||||||
|
|
||||||
if (OS_LINUX AND NOT UNBUNDLED)
|
if (OS_LINUX AND NOT UNBUNDLED AND MAKE_STATIC_LIBRARIES AND NOT SPLIT_SHARED_LIBRARIES AND CMAKE_VERSION VERSION_GREATER "3.9.0")
|
||||||
option (GLIBC_COMPATIBILITY "Set to TRUE to enable compatibility with older glibc libraries. Only for x86_64, Linux. Implies USE_INTERNAL_MEMCPY." ON)
|
option (GLIBC_COMPATIBILITY "Set to TRUE to enable compatibility with older glibc libraries. Only for x86_64, Linux. Implies USE_INTERNAL_MEMCPY." ON)
|
||||||
if (GLIBC_COMPATIBILITY)
|
|
||||||
message (STATUS "Some symbols from glibc will be replaced for compatibility")
|
|
||||||
link_libraries(glibc-compatibility)
|
|
||||||
endif ()
|
|
||||||
endif ()
|
endif ()
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
@ -177,6 +191,67 @@ set (CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} -O0 -g3 -ggdb3
|
|||||||
|
|
||||||
include (cmake/use_libcxx.cmake)
|
include (cmake/use_libcxx.cmake)
|
||||||
|
|
||||||
|
|
||||||
|
# Set standard, system and compiler libraries explicitly.
|
||||||
|
# This is intended for more control of what we are linking.
|
||||||
|
|
||||||
|
set (DEFAULT_LIBS "")
|
||||||
|
if (OS_LINUX AND NOT UNBUNDLED AND (GLIBC_COMPATIBILITY OR USE_LIBCXX))
|
||||||
|
# Note: this probably has no effect, but I'm not an expert in CMake.
|
||||||
|
set (CMAKE_C_IMPLICIT_LINK_LIBRARIES "")
|
||||||
|
set (CMAKE_CXX_IMPLICIT_LINK_LIBRARIES "")
|
||||||
|
|
||||||
|
# Disable default linked libraries.
|
||||||
|
set (DEFAULT_LIBS "-nodefaultlibs")
|
||||||
|
|
||||||
|
# We need builtins from Clang's RT even without libcxx - for ubsan+int128. See https://bugs.llvm.org/show_bug.cgi?id=16404
|
||||||
|
set (BUILTINS_LIB_PATH "")
|
||||||
|
if (COMPILER_CLANG)
|
||||||
|
execute_process (COMMAND ${CMAKE_CXX_COMPILER} --print-file-name=libclang_rt.builtins-${CMAKE_SYSTEM_PROCESSOR}.a OUTPUT_VARIABLE BUILTINS_LIB_PATH OUTPUT_STRIP_TRAILING_WHITESPACE)
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
# Add C++ libraries.
|
||||||
|
#
|
||||||
|
# This consist of:
|
||||||
|
# - C++ standard library (like implementation of std::string);
|
||||||
|
# - C++ ABI implementation (functions for exceptions like __cxa_throw, RTTI, etc);
|
||||||
|
# - functions for internal implementation of exception handling (stack unwinding based on DWARF info; TODO replace with bundled libunwind);
|
||||||
|
# - compiler builtins (example: functions for implementation of __int128 operations);
|
||||||
|
#
|
||||||
|
# There are two variants of C++ library: libc++ (from LLVM compiler infrastructure) and libstdc++ (from GCC).
|
||||||
|
if (USE_LIBCXX)
|
||||||
|
set (DEFAULT_LIBS "${DEFAULT_LIBS} -Wl,-Bstatic -lc++ -lc++abi -lgcc_eh ${BUILTINS_LIB_PATH} -Wl,-Bdynamic")
|
||||||
|
else ()
|
||||||
|
set (DEFAULT_LIBS "${DEFAULT_LIBS} -Wl,-Bstatic -lstdc++ -lgcc_eh -lgcc ${BUILTINS_LIB_PATH} -Wl,-Bdynamic")
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
# Linking with GLIBC prevents portability of binaries to older systems.
|
||||||
|
# We overcome this behaviour by statically linking with our own implementation of all new symbols (that don't exist in older Libc or have infamous "symbol versioning").
|
||||||
|
# The order of linking is important: 'glibc-compatibility' must be before libc but after all other libraries.
|
||||||
|
if (GLIBC_COMPATIBILITY)
|
||||||
|
message (STATUS "Some symbols from glibc will be replaced for compatibility")
|
||||||
|
|
||||||
|
string (TOUPPER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_UC)
|
||||||
|
set (CMAKE_POSTFIX_VARIABLE "CMAKE_${CMAKE_BUILD_TYPE_UC}_POSTFIX")
|
||||||
|
|
||||||
|
# FIXME: glibc-compatibility may be non-static in some builds!
|
||||||
|
set (DEFAULT_LIBS "${DEFAULT_LIBS} ${ClickHouse_BINARY_DIR}/libs/libglibc-compatibility/libglibc-compatibility${${CMAKE_POSTFIX_VARIABLE}}.a")
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
# Add Libc. GLIBC is actually a collection of interdependent libraries.
|
||||||
|
set (DEFAULT_LIBS "${DEFAULT_LIBS} -lrt -ldl -lpthread -lm -lc")
|
||||||
|
|
||||||
|
# Note: we'd rather use Musl libc library, but it's little bit more difficult to use.
|
||||||
|
|
||||||
|
message(STATUS "Default libraries: ${DEFAULT_LIBS}")
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
if (DEFAULT_LIBS)
|
||||||
|
# Add default libs to all targets as the last dependency.
|
||||||
|
set(CMAKE_CXX_STANDARD_LIBRARIES ${DEFAULT_LIBS})
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
|
||||||
if (NOT MAKE_STATIC_LIBRARIES)
|
if (NOT MAKE_STATIC_LIBRARIES)
|
||||||
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
|
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
|
||||||
endif ()
|
endif ()
|
||||||
@ -244,6 +319,7 @@ include (cmake/find_rt.cmake)
|
|||||||
include (cmake/find_execinfo.cmake)
|
include (cmake/find_execinfo.cmake)
|
||||||
include (cmake/find_readline_edit.cmake)
|
include (cmake/find_readline_edit.cmake)
|
||||||
include (cmake/find_re2.cmake)
|
include (cmake/find_re2.cmake)
|
||||||
|
include (cmake/find_libgsasl.cmake)
|
||||||
include (cmake/find_rdkafka.cmake)
|
include (cmake/find_rdkafka.cmake)
|
||||||
include (cmake/find_capnp.cmake)
|
include (cmake/find_capnp.cmake)
|
||||||
include (cmake/find_llvm.cmake)
|
include (cmake/find_llvm.cmake)
|
||||||
@ -252,7 +328,6 @@ include (cmake/find_cpuid.cmake) # Freebsd, bundled
|
|||||||
if (NOT USE_CPUID)
|
if (NOT USE_CPUID)
|
||||||
include (cmake/find_cpuinfo.cmake) # Debian
|
include (cmake/find_cpuinfo.cmake) # Debian
|
||||||
endif()
|
endif()
|
||||||
include (cmake/find_libgsasl.cmake)
|
|
||||||
include (cmake/find_libxml2.cmake)
|
include (cmake/find_libxml2.cmake)
|
||||||
include (cmake/find_brotli.cmake)
|
include (cmake/find_brotli.cmake)
|
||||||
include (cmake/find_protobuf.cmake)
|
include (cmake/find_protobuf.cmake)
|
||||||
@ -260,12 +335,17 @@ include (cmake/find_pdqsort.cmake)
|
|||||||
include (cmake/find_hdfs3.cmake) # uses protobuf
|
include (cmake/find_hdfs3.cmake) # uses protobuf
|
||||||
include (cmake/find_consistent-hashing.cmake)
|
include (cmake/find_consistent-hashing.cmake)
|
||||||
include (cmake/find_base64.cmake)
|
include (cmake/find_base64.cmake)
|
||||||
|
include (cmake/find_hyperscan.cmake)
|
||||||
|
include (cmake/find_lfalloc.cmake)
|
||||||
|
include (cmake/find_simdjson.cmake)
|
||||||
|
include (cmake/find_rapidjson.cmake)
|
||||||
find_contrib_lib(cityhash)
|
find_contrib_lib(cityhash)
|
||||||
find_contrib_lib(farmhash)
|
find_contrib_lib(farmhash)
|
||||||
find_contrib_lib(metrohash)
|
find_contrib_lib(metrohash)
|
||||||
find_contrib_lib(btrie)
|
find_contrib_lib(btrie)
|
||||||
find_contrib_lib(double-conversion)
|
find_contrib_lib(double-conversion)
|
||||||
include (cmake/find_parquet.cmake)
|
include (cmake/find_parquet.cmake)
|
||||||
|
|
||||||
if (ENABLE_TESTS)
|
if (ENABLE_TESTS)
|
||||||
include (cmake/find_gtest.cmake)
|
include (cmake/find_gtest.cmake)
|
||||||
endif ()
|
endif ()
|
||||||
@ -285,3 +365,51 @@ add_subdirectory (utils)
|
|||||||
add_subdirectory (dbms)
|
add_subdirectory (dbms)
|
||||||
|
|
||||||
include (cmake/print_include_directories.cmake)
|
include (cmake/print_include_directories.cmake)
|
||||||
|
|
||||||
|
if (GLIBC_COMPATIBILITY)
|
||||||
|
# FIXME: actually glibc-compatibility should always be built first,
|
||||||
|
# because it's unconditionally linked via $DEFAULT_LIBS,
|
||||||
|
# and these looks like the first places that get linked.
|
||||||
|
function (add_glibc_compat target_name)
|
||||||
|
if (TARGET ${target_name})
|
||||||
|
add_dependencies(${target_name} glibc-compatibility)
|
||||||
|
endif ()
|
||||||
|
endfunction ()
|
||||||
|
|
||||||
|
add_glibc_compat(ltdl)
|
||||||
|
add_glibc_compat(zlibstatic)
|
||||||
|
add_glibc_compat(jemalloc)
|
||||||
|
add_glibc_compat(unwind)
|
||||||
|
add_glibc_compat(memcpy)
|
||||||
|
add_glibc_compat(Foundation)
|
||||||
|
add_glibc_compat(common)
|
||||||
|
add_glibc_compat(gtest)
|
||||||
|
add_glibc_compat(lz4)
|
||||||
|
add_glibc_compat(zstd)
|
||||||
|
add_glibc_compat(snappy)
|
||||||
|
add_glibc_compat(arrow)
|
||||||
|
add_glibc_compat(protoc)
|
||||||
|
add_glibc_compat(thrift_static)
|
||||||
|
add_glibc_compat(cityhash)
|
||||||
|
add_glibc_compat(farmhash)
|
||||||
|
add_glibc_compat(murmurhash)
|
||||||
|
add_glibc_compat(metrohash)
|
||||||
|
add_glibc_compat(metrohash128)
|
||||||
|
add_glibc_compat(consistent-hashing)
|
||||||
|
add_glibc_compat(double-conversion)
|
||||||
|
add_glibc_compat(cctz)
|
||||||
|
add_glibc_compat(kj)
|
||||||
|
add_glibc_compat(simdjson)
|
||||||
|
add_glibc_compat(apple_rt)
|
||||||
|
add_glibc_compat(re2)
|
||||||
|
add_glibc_compat(re2_st)
|
||||||
|
add_glibc_compat(hs_compile_shared)
|
||||||
|
add_glibc_compat(hs_exec_shared)
|
||||||
|
add_glibc_compat(hs_shared)
|
||||||
|
add_glibc_compat(widechar_width)
|
||||||
|
add_glibc_compat(string_utils)
|
||||||
|
add_glibc_compat(consistent-hashing-sumbur)
|
||||||
|
add_glibc_compat(boost_program_options_internal)
|
||||||
|
add_glibc_compat(boost_system_internal)
|
||||||
|
add_glibc_compat(boost_regex_internal)
|
||||||
|
endif ()
|
||||||
|
|||||||
@ -12,6 +12,7 @@ ClickHouse is an open-source column-oriented database management system that all
|
|||||||
* You can also [fill this form](https://forms.yandex.com/surveys/meet-yandex-clickhouse-team/) to meet Yandex ClickHouse team in person.
|
* You can also [fill this form](https://forms.yandex.com/surveys/meet-yandex-clickhouse-team/) to meet Yandex ClickHouse team in person.
|
||||||
|
|
||||||
## Upcoming Events
|
## Upcoming Events
|
||||||
|
* [ClickHouse on HighLoad++ Siberia](https://www.highload.ru/siberia/2019/abstracts/5348) on June 24-25.
|
||||||
* [ClickHouse Community Meetup](https://www.eventbrite.com/e/meetup-clickhouse-in-the-wild-deployment-success-stories-registration-55305051899) in San Francisco on February 19.
|
* [ClickHouse Meetup in Novosibirsk](https://events.yandex.ru/events/ClickHouse/26-June-2019/) on June 26.
|
||||||
* [ClickHouse Community Meetup](https://www.eventbrite.com/e/clickhouse-meetup-in-madrid-registration-55376746339) in Madrid on April 2.
|
* [ClickHouse Meetup in Shenzhen](https://www.huodongxing.com/event/3483759917300) on October 20.
|
||||||
|
* [ClickHouse Meetup in Shanghai](https://www.huodongxing.com/event/4483760336000) on October 27.
|
||||||
|
|||||||
17
SECURITY.md
Normal file
17
SECURITY.md
Normal file
@ -0,0 +1,17 @@
|
|||||||
|
# Security Policy
|
||||||
|
|
||||||
|
## Supported Versions
|
||||||
|
|
||||||
|
The following versions of ClickHouse server are
|
||||||
|
currently being supported with security updates:
|
||||||
|
|
||||||
|
| Version | Supported |
|
||||||
|
| ------- | ------------------ |
|
||||||
|
| 1.x | :x: |
|
||||||
|
| 18.x | :x: |
|
||||||
|
| 19.x | :white_check_mark: |
|
||||||
|
|
||||||
|
## Reporting a Vulnerability
|
||||||
|
|
||||||
|
To report a potential vulnerability in ClickHouse please use the security advisory feature of GitHub:
|
||||||
|
https://github.com/yandex/ClickHouse/security/advisories
|
||||||
@ -21,7 +21,7 @@ BUILD_TARGETS=clickhouse
|
|||||||
BUILD_TYPE=Debug
|
BUILD_TYPE=Debug
|
||||||
ENABLE_EMBEDDED_COMPILER=0
|
ENABLE_EMBEDDED_COMPILER=0
|
||||||
|
|
||||||
CMAKE_FLAGS="-D CMAKE_C_FLAGS_ADD=-g0 -D CMAKE_CXX_FLAGS_ADD=-g0 -D ENABLE_JEMALLOC=0 -D ENABLE_CAPNP=0 -D ENABLE_RDKAFKA=0 -D ENABLE_UNWIND=0 -D ENABLE_ICU=0 -D ENABLE_POCO_MONGODB=0 -D ENABLE_POCO_NETSSL=0 -D ENABLE_POCO_ODBC=0 -D ENABLE_ODBC=0 -D ENABLE_MYSQL=0"
|
CMAKE_FLAGS="-D CMAKE_C_FLAGS_ADD=-g0 -D CMAKE_CXX_FLAGS_ADD=-g0 -D ENABLE_JEMALLOC=0 -D ENABLE_CAPNP=0 -D ENABLE_RDKAFKA=0 -D ENABLE_UNWIND=0 -D ENABLE_ICU=0 -D ENABLE_POCO_MONGODB=0 -D ENABLE_POCO_NETSSL=0 -D ENABLE_POCO_ODBC=0 -D ENABLE_ODBC=0 -D ENABLE_MYSQL=0 -D ENABLE_SSL=0 -D ENABLE_POCO_NETSSL=0"
|
||||||
|
|
||||||
[[ $(uname) == "FreeBSD" ]] && COMPILER_PACKAGE_VERSION=devel && export COMPILER_PATH=/usr/local/bin
|
[[ $(uname) == "FreeBSD" ]] && COMPILER_PACKAGE_VERSION=devel && export COMPILER_PATH=/usr/local/bin
|
||||||
|
|
||||||
|
|||||||
@ -1,88 +1,147 @@
|
|||||||
# This file copied from contrib/poco/cmake/FindODBC.cmake to allow build without submodules
|
# Distributed under the OSI-approved BSD 3-Clause License. See accompanying
|
||||||
|
# file Copyright.txt or https://cmake.org/licensing for details.
|
||||||
|
|
||||||
|
#.rst:
|
||||||
|
# FindMySQL
|
||||||
|
# -------
|
||||||
#
|
#
|
||||||
# Find the ODBC driver manager includes and library.
|
# Find ODBC Runtime
|
||||||
#
|
#
|
||||||
# ODBC is an open standard for connecting to different databases in a
|
# This will define the following variables::
|
||||||
# semi-vendor-independent fashion. First you install the ODBC driver
|
|
||||||
# manager. Then you need a driver for each separate database you want
|
|
||||||
# to connect to (unless a generic one works). VTK includes neither
|
|
||||||
# the driver manager nor the vendor-specific drivers: you have to find
|
|
||||||
# those yourself.
|
|
||||||
#
|
#
|
||||||
# This module defines
|
# ODBC_FOUND - True if the system has the libraries
|
||||||
# ODBC_INCLUDE_DIRECTORIES, where to find sql.h
|
# ODBC_INCLUDE_DIRS - where to find the headers
|
||||||
# ODBC_LIBRARIES, the libraries to link against to use ODBC
|
# ODBC_LIBRARIES - where to find the libraries
|
||||||
# ODBC_FOUND. If false, you cannot build anything that requires ODBC.
|
# ODBC_DEFINITIONS - compile definitons
|
||||||
|
#
|
||||||
|
# Hints:
|
||||||
|
# Set ``ODBC_ROOT_DIR`` to the root directory of an installation.
|
||||||
|
#
|
||||||
|
include(FindPackageHandleStandardArgs)
|
||||||
|
|
||||||
option (ENABLE_ODBC "Enable ODBC" ${OS_LINUX})
|
find_package(PkgConfig QUIET)
|
||||||
if (OS_LINUX)
|
pkg_check_modules(PC_ODBC QUIET odbc)
|
||||||
option (USE_INTERNAL_ODBC_LIBRARY "Set to FALSE to use system odbc library instead of bundled" ${NOT_UNBUNDLED})
|
|
||||||
else ()
|
if(WIN32)
|
||||||
option (USE_INTERNAL_ODBC_LIBRARY "Set to FALSE to use system odbc library instead of bundled" OFF)
|
get_filename_component(kit_dir "[HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows Kits\\Installed Roots;KitsRoot]" REALPATH)
|
||||||
|
get_filename_component(kit81_dir "[HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows Kits\\Installed Roots;KitsRoot81]" REALPATH)
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
if (USE_INTERNAL_ODBC_LIBRARY AND NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/unixodbc/README")
|
find_path(ODBC_INCLUDE_DIR
|
||||||
message (WARNING "submodule contrib/unixodbc is missing. to fix try run: \n git submodule update --init --recursive")
|
|
||||||
set (USE_INTERNAL_ODBC_LIBRARY 0)
|
|
||||||
endif ()
|
|
||||||
|
|
||||||
if (ENABLE_ODBC)
|
|
||||||
if (USE_INTERNAL_ODBC_LIBRARY)
|
|
||||||
set (ODBC_LIBRARIES unixodbc)
|
|
||||||
set (ODBC_INCLUDE_DIRECTORIES ${CMAKE_SOURCE_DIR}/contrib/unixodbc/include)
|
|
||||||
set (ODBC_FOUND 1)
|
|
||||||
set (USE_ODBC 1)
|
|
||||||
else ()
|
|
||||||
find_path(ODBC_INCLUDE_DIRECTORIES
|
|
||||||
NAMES sql.h
|
NAMES sql.h
|
||||||
HINTS
|
HINTS
|
||||||
|
${ODBC_ROOT_DIR}/include
|
||||||
|
${ODBC_ROOT_INCLUDE_DIRS}
|
||||||
|
PATHS
|
||||||
|
${PC_ODBC_INCLUDE_DIRS}
|
||||||
/usr/include
|
/usr/include
|
||||||
/usr/include/iodbc
|
|
||||||
/usr/include/odbc
|
|
||||||
/usr/local/include
|
/usr/local/include
|
||||||
/usr/local/include/iodbc
|
|
||||||
/usr/local/include/odbc
|
|
||||||
/usr/local/iodbc/include
|
|
||||||
/usr/local/odbc/include
|
/usr/local/odbc/include
|
||||||
|
/usr/local/iodbc/include
|
||||||
"C:/Program Files/ODBC/include"
|
"C:/Program Files/ODBC/include"
|
||||||
"C:/Program Files/Microsoft SDKs/Windows/v7.0/include"
|
"C:/Program Files/Microsoft SDKs/Windows/v7.0/include"
|
||||||
"C:/Program Files/Microsoft SDKs/Windows/v6.0a/include"
|
"C:/Program Files/Microsoft SDKs/Windows/v6.0a/include"
|
||||||
"C:/ODBC/include"
|
"C:/ODBC/include"
|
||||||
|
"${kit_dir}/Include/um"
|
||||||
|
"${kit81_dir}/Include/um"
|
||||||
|
PATH_SUFFIXES
|
||||||
|
odbc
|
||||||
|
iodbc
|
||||||
DOC "Specify the directory containing sql.h."
|
DOC "Specify the directory containing sql.h."
|
||||||
)
|
)
|
||||||
|
|
||||||
find_library(ODBC_LIBRARIES
|
if(NOT ODBC_INCLUDE_DIR AND WIN32)
|
||||||
NAMES iodbc odbc iodbcinst odbcinst odbc32
|
set(ODBC_INCLUDE_DIR "")
|
||||||
|
else()
|
||||||
|
set(REQUIRED_INCLUDE_DIR ODBC_INCLUDE_DIR)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
if(WIN32 AND CMAKE_SIZEOF_VOID_P EQUAL 8)
|
||||||
|
set(WIN_ARCH x64)
|
||||||
|
elseif(WIN32 AND CMAKE_SIZEOF_VOID_P EQUAL 4)
|
||||||
|
set(WIN_ARCH x86)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
find_library(ODBC_LIBRARY
|
||||||
|
NAMES unixodbc iodbc odbc odbc32
|
||||||
HINTS
|
HINTS
|
||||||
|
${ODBC_ROOT_DIR}/lib
|
||||||
|
${ODBC_ROOT_LIBRARY_DIRS}
|
||||||
|
PATHS
|
||||||
|
${PC_ODBC_LIBRARY_DIRS}
|
||||||
/usr/lib
|
/usr/lib
|
||||||
/usr/lib/iodbc
|
|
||||||
/usr/lib/odbc
|
|
||||||
/usr/local/lib
|
/usr/local/lib
|
||||||
/usr/local/lib/iodbc
|
|
||||||
/usr/local/lib/odbc
|
|
||||||
/usr/local/iodbc/lib
|
|
||||||
/usr/local/odbc/lib
|
/usr/local/odbc/lib
|
||||||
|
/usr/local/iodbc/lib
|
||||||
"C:/Program Files/ODBC/lib"
|
"C:/Program Files/ODBC/lib"
|
||||||
"C:/ODBC/lib/debug"
|
"C:/ODBC/lib/debug"
|
||||||
"C:/Program Files (x86)/Microsoft SDKs/Windows/v7.0A/Lib"
|
"C:/Program Files (x86)/Microsoft SDKs/Windows/v7.0A/Lib"
|
||||||
|
"${kit81_dir}/Lib/winv6.3/um"
|
||||||
|
"${kit_dir}/Lib/win8/um"
|
||||||
|
PATH_SUFIXES
|
||||||
|
odbc
|
||||||
|
${WIN_ARCH}
|
||||||
DOC "Specify the ODBC driver manager library here."
|
DOC "Specify the ODBC driver manager library here."
|
||||||
)
|
)
|
||||||
|
|
||||||
# MinGW find usually fails
|
if(NOT ODBC_LIBRARY AND WIN32)
|
||||||
if(MINGW)
|
# List names of ODBC libraries on Windows
|
||||||
set(ODBC_INCLUDE_DIRECTORIES ".")
|
set(ODBC_LIBRARY odbc32.lib)
|
||||||
set(ODBC_LIBRARIES odbc32)
|
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
include(FindPackageHandleStandardArgs)
|
# List additional libraries required to use ODBC library
|
||||||
|
if(WIN32 AND MSVC OR CMAKE_CXX_COMPILER_ID MATCHES "Intel")
|
||||||
|
set(_odbc_required_libs_names odbccp32;ws2_32)
|
||||||
|
endif()
|
||||||
|
foreach(_lib_name IN LISTS _odbc_required_libs_names)
|
||||||
|
find_library(_lib_path
|
||||||
|
NAMES ${_lib_name}
|
||||||
|
HINTS
|
||||||
|
${ODBC_ROOT_DIR}/lib
|
||||||
|
${ODBC_ROOT_LIBRARY_DIRS}
|
||||||
|
PATHS
|
||||||
|
${PC_ODBC_LIBRARY_DIRS}
|
||||||
|
/usr/lib
|
||||||
|
/usr/local/lib
|
||||||
|
/usr/local/odbc/lib
|
||||||
|
/usr/local/iodbc/lib
|
||||||
|
"C:/Program Files/ODBC/lib"
|
||||||
|
"C:/ODBC/lib/debug"
|
||||||
|
"C:/Program Files (x86)/Microsoft SDKs/Windows/v7.0A/Lib"
|
||||||
|
PATH_SUFFIXES
|
||||||
|
odbc
|
||||||
|
)
|
||||||
|
if (_lib_path)
|
||||||
|
list(APPEND _odbc_required_libs_paths ${_lib_path})
|
||||||
|
endif()
|
||||||
|
unset(_lib_path CACHE)
|
||||||
|
endforeach()
|
||||||
|
unset(_odbc_lib_paths)
|
||||||
|
unset(_odbc_required_libs_names)
|
||||||
|
|
||||||
|
|
||||||
find_package_handle_standard_args(ODBC
|
find_package_handle_standard_args(ODBC
|
||||||
DEFAULT_MSG
|
FOUND_VAR ODBC_FOUND
|
||||||
ODBC_INCLUDE_DIRECTORIES
|
REQUIRED_VARS
|
||||||
ODBC_LIBRARIES)
|
ODBC_LIBRARY
|
||||||
|
${REQUIRED_INCLUDE_DIR}
|
||||||
|
VERSION_VAR ODBC_VERSION
|
||||||
|
)
|
||||||
|
|
||||||
mark_as_advanced(ODBC_FOUND ODBC_LIBRARIES ODBC_INCLUDE_DIRECTORIES)
|
if(ODBC_FOUND)
|
||||||
endif ()
|
set(ODBC_LIBRARIES ${ODBC_LIBRARY} ${_odbc_required_libs_paths})
|
||||||
|
set(ODBC_INCLUDE_DIRS ${ODBC_INCLUDE_DIR})
|
||||||
|
set(ODBC_DEFINITIONS ${PC_ODBC_CFLAGS_OTHER})
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
message (STATUS "Using odbc: ${ODBC_INCLUDE_DIRECTORIES} : ${ODBC_LIBRARIES}")
|
if(ODBC_FOUND AND NOT TARGET ODBC::ODBC)
|
||||||
|
add_library(ODBC::ODBC UNKNOWN IMPORTED)
|
||||||
|
set_target_properties(ODBC::ODBC PROPERTIES
|
||||||
|
IMPORTED_LOCATION "${ODBC_LIBRARY}"
|
||||||
|
INTERFACE_LINK_LIBRARIES "${_odbc_required_libs_paths}"
|
||||||
|
INTERFACE_COMPILE_OPTIONS "${PC_ODBC_CFLAGS_OTHER}"
|
||||||
|
INTERFACE_INCLUDE_DIRECTORIES "${ODBC_INCLUDE_DIR}"
|
||||||
|
)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
mark_as_advanced(ODBC_LIBRARY ODBC_INCLUDE_DIR)
|
||||||
|
|||||||
@ -203,12 +203,12 @@ endforeach()
|
|||||||
|
|
||||||
if(Poco_DataODBC_LIBRARY)
|
if(Poco_DataODBC_LIBRARY)
|
||||||
list(APPEND Poco_DataODBC_LIBRARY ${ODBC_LIBRARIES} ${LTDL_LIBRARY})
|
list(APPEND Poco_DataODBC_LIBRARY ${ODBC_LIBRARIES} ${LTDL_LIBRARY})
|
||||||
list(APPEND Poco_INCLUDE_DIRS ${ODBC_INCLUDE_DIRECTORIES})
|
list(APPEND Poco_INCLUDE_DIRS ${ODBC_INCLUDE_DIRS})
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
if(Poco_SQLODBC_LIBRARY)
|
if(Poco_SQLODBC_LIBRARY)
|
||||||
list(APPEND Poco_SQLODBC_LIBRARY ${ODBC_LIBRARIES} ${LTDL_LIBRARY})
|
list(APPEND Poco_SQLODBC_LIBRARY ${ODBC_LIBRARIES} ${LTDL_LIBRARY})
|
||||||
list(APPEND Poco_INCLUDE_DIRS ${ODBC_INCLUDE_DIRECTORIES})
|
list(APPEND Poco_INCLUDE_DIRS ${ODBC_INCLUDE_DIRS})
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
if(Poco_NetSSL_LIBRARY)
|
if(Poco_NetSSL_LIBRARY)
|
||||||
|
|||||||
@ -5,7 +5,7 @@ endmacro()
|
|||||||
|
|
||||||
macro(add_headers_and_sources prefix common_path)
|
macro(add_headers_and_sources prefix common_path)
|
||||||
add_glob(${prefix}_headers RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} ${common_path}/*.h)
|
add_glob(${prefix}_headers RELATIVE ${CMAKE_CURRENT_SOURCE_DIR} ${common_path}/*.h)
|
||||||
add_glob(${prefix}_sources ${common_path}/*.cpp ${common_path}/*.h)
|
add_glob(${prefix}_sources ${common_path}/*.cpp ${common_path}/*.c ${common_path}/*.h)
|
||||||
endmacro()
|
endmacro()
|
||||||
|
|
||||||
macro(add_headers_only prefix common_path)
|
macro(add_headers_only prefix common_path)
|
||||||
|
|||||||
@ -1,9 +1,12 @@
|
|||||||
option (USE_INTERNAL_BOOST_LIBRARY "Set to FALSE to use system boost library instead of bundled" ${NOT_UNBUNDLED})
|
option (USE_INTERNAL_BOOST_LIBRARY "Set to FALSE to use system boost library instead of bundled" ${NOT_UNBUNDLED})
|
||||||
|
|
||||||
# Test random file existing in all package variants
|
# Test random file existing in all package variants
|
||||||
if (USE_INTERNAL_BOOST_LIBRARY AND NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/boost/libs/system/src/error_code.cpp")
|
if (NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/boost/libs/system/src/error_code.cpp")
|
||||||
|
if(USE_INTERNAL_BOOST_LIBRARY)
|
||||||
message(WARNING "submodules in contrib/boost is missing. to fix try run: \n git submodule update --init --recursive")
|
message(WARNING "submodules in contrib/boost is missing. to fix try run: \n git submodule update --init --recursive")
|
||||||
|
endif()
|
||||||
set (USE_INTERNAL_BOOST_LIBRARY 0)
|
set (USE_INTERNAL_BOOST_LIBRARY 0)
|
||||||
|
set (MISSING_INTERNAL_BOOST_LIBRARY 1)
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (NOT USE_INTERNAL_BOOST_LIBRARY)
|
if (NOT USE_INTERNAL_BOOST_LIBRARY)
|
||||||
@ -21,10 +24,9 @@ if (NOT USE_INTERNAL_BOOST_LIBRARY)
|
|||||||
set (Boost_INCLUDE_DIRS "")
|
set (Boost_INCLUDE_DIRS "")
|
||||||
set (Boost_SYSTEM_LIBRARY "")
|
set (Boost_SYSTEM_LIBRARY "")
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (NOT Boost_SYSTEM_LIBRARY)
|
if (NOT Boost_SYSTEM_LIBRARY AND NOT MISSING_INTERNAL_BOOST_LIBRARY)
|
||||||
set (USE_INTERNAL_BOOST_LIBRARY 1)
|
set (USE_INTERNAL_BOOST_LIBRARY 1)
|
||||||
set (Boost_SYSTEM_LIBRARY boost_system_internal)
|
set (Boost_SYSTEM_LIBRARY boost_system_internal)
|
||||||
set (Boost_PROGRAM_OPTIONS_LIBRARY boost_program_options_internal)
|
set (Boost_PROGRAM_OPTIONS_LIBRARY boost_program_options_internal)
|
||||||
@ -44,7 +46,6 @@ if (NOT Boost_SYSTEM_LIBRARY)
|
|||||||
|
|
||||||
# For packaged version:
|
# For packaged version:
|
||||||
list (APPEND Boost_INCLUDE_DIRS "${ClickHouse_SOURCE_DIR}/contrib/boost")
|
list (APPEND Boost_INCLUDE_DIRS "${ClickHouse_SOURCE_DIR}/contrib/boost")
|
||||||
|
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
message (STATUS "Using Boost: ${Boost_INCLUDE_DIRS} : ${Boost_PROGRAM_OPTIONS_LIBRARY},${Boost_SYSTEM_LIBRARY},${Boost_FILESYSTEM_LIBRARY},${Boost_REGEX_LIBRARY}")
|
message (STATUS "Using Boost: ${Boost_INCLUDE_DIRS} : ${Boost_PROGRAM_OPTIONS_LIBRARY},${Boost_SYSTEM_LIBRARY},${Boost_FILESYSTEM_LIBRARY},${Boost_REGEX_LIBRARY}")
|
||||||
|
|||||||
@ -1,3 +1,7 @@
|
|||||||
|
option (ENABLE_BROTLI "Enable brotli" ON)
|
||||||
|
|
||||||
|
if (ENABLE_BROTLI)
|
||||||
|
|
||||||
option (USE_INTERNAL_BROTLI_LIBRARY "Set to FALSE to use system libbrotli library instead of bundled" ${NOT_UNBUNDLED})
|
option (USE_INTERNAL_BROTLI_LIBRARY "Set to FALSE to use system libbrotli library instead of bundled" ${NOT_UNBUNDLED})
|
||||||
|
|
||||||
if (NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/brotli/c/include/brotli/decode.h")
|
if (NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/brotli/c/include/brotli/decode.h")
|
||||||
@ -27,4 +31,6 @@ elseif (NOT MISSING_INTERNAL_BROTLI_LIBRARY)
|
|||||||
set (USE_BROTLI 1)
|
set (USE_BROTLI 1)
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
|
endif()
|
||||||
|
|
||||||
message (STATUS "Using brotli=${USE_BROTLI}: ${BROTLI_INCLUDE_DIR} : ${BROTLI_LIBRARY}")
|
message (STATUS "Using brotli=${USE_BROTLI}: ${BROTLI_INCLUDE_DIR} : ${BROTLI_LIBRARY}")
|
||||||
|
|||||||
@ -1,4 +1,3 @@
|
|||||||
|
|
||||||
find_program (CCACHE_FOUND ccache)
|
find_program (CCACHE_FOUND ccache)
|
||||||
if (CCACHE_FOUND AND NOT CMAKE_CXX_COMPILER_LAUNCHER MATCHES "ccache" AND NOT CMAKE_CXX_COMPILER MATCHES "ccache")
|
if (CCACHE_FOUND AND NOT CMAKE_CXX_COMPILER_LAUNCHER MATCHES "ccache" AND NOT CMAKE_CXX_COMPILER MATCHES "ccache")
|
||||||
execute_process(COMMAND ${CCACHE_FOUND} "-V" OUTPUT_VARIABLE CCACHE_VERSION)
|
execute_process(COMMAND ${CCACHE_FOUND} "-V" OUTPUT_VARIABLE CCACHE_VERSION)
|
||||||
|
|||||||
@ -1,9 +1,8 @@
|
|||||||
if (OS_FREEBSD)
|
if (OS_FREEBSD)
|
||||||
find_library (EXECINFO_LIBRARY execinfo)
|
find_library (EXECINFO_LIBRARY execinfo)
|
||||||
find_library (ELF_LIBRARY elf)
|
find_library (ELF_LIBRARY elf)
|
||||||
message (STATUS "Using execinfo: ${EXECINFO_LIBRARY}")
|
set (EXECINFO_LIBRARIES ${EXECINFO_LIBRARY} ${ELF_LIBRARY})
|
||||||
message (STATUS "Using elf: ${ELF_LIBRARY}")
|
message (STATUS "Using execinfo: ${EXECINFO_LIBRARIES}")
|
||||||
else ()
|
else ()
|
||||||
set (EXECINFO_LIBRARY "")
|
set (EXECINFO_LIBRARIES "")
|
||||||
set (ELF_LIBRARY "")
|
|
||||||
endif ()
|
endif ()
|
||||||
|
|||||||
13
cmake/find_gperf.cmake
Normal file
13
cmake/find_gperf.cmake
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
# Check if gperf was installed
|
||||||
|
find_program(GPERF gperf)
|
||||||
|
if(GPERF)
|
||||||
|
option(ENABLE_GPERF "Use gperf function hash generator tool" ON)
|
||||||
|
endif()
|
||||||
|
if (ENABLE_GPERF)
|
||||||
|
if(NOT GPERF)
|
||||||
|
message(FATAL_ERROR "Could not find the program gperf")
|
||||||
|
endif()
|
||||||
|
set(USE_GPERF 1)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
message(STATUS "Using gperf=${USE_GPERF}: ${GPERF}")
|
||||||
@ -20,11 +20,12 @@ if (NOT GTEST_SRC_DIR AND NOT GTEST_INCLUDE_DIRS AND NOT MISSING_INTERNAL_GTEST_
|
|||||||
set (USE_INTERNAL_GTEST_LIBRARY 1)
|
set (USE_INTERNAL_GTEST_LIBRARY 1)
|
||||||
set (GTEST_MAIN_LIBRARIES gtest_main)
|
set (GTEST_MAIN_LIBRARIES gtest_main)
|
||||||
set (GTEST_LIBRARIES gtest)
|
set (GTEST_LIBRARIES gtest)
|
||||||
|
set (GTEST_BOTH_LIBRARIES ${GTEST_MAIN_LIBRARIES} ${GTEST_LIBRARIES})
|
||||||
set (GTEST_INCLUDE_DIRS ${ClickHouse_SOURCE_DIR}/contrib/googletest/googletest)
|
set (GTEST_INCLUDE_DIRS ${ClickHouse_SOURCE_DIR}/contrib/googletest/googletest)
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if((GTEST_INCLUDE_DIRS AND GTEST_MAIN_LIBRARIES) OR GTEST_SRC_DIR)
|
if((GTEST_INCLUDE_DIRS AND GTEST_BOTH_LIBRARIES) OR GTEST_SRC_DIR)
|
||||||
set(USE_GTEST 1)
|
set(USE_GTEST 1)
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
message (STATUS "Using gtest=${USE_GTEST}: ${GTEST_INCLUDE_DIRS} : ${GTEST_LIBRARIES}, ${GTEST_MAIN_LIBRARIES} : ${GTEST_SRC_DIR}")
|
message (STATUS "Using gtest=${USE_GTEST}: ${GTEST_INCLUDE_DIRS} : ${GTEST_BOTH_LIBRARIES} : ${GTEST_SRC_DIR}")
|
||||||
|
|||||||
33
cmake/find_hyperscan.cmake
Normal file
33
cmake/find_hyperscan.cmake
Normal file
@ -0,0 +1,33 @@
|
|||||||
|
if (HAVE_SSSE3)
|
||||||
|
option (ENABLE_HYPERSCAN "Enable hyperscan" ON)
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
if (ENABLE_HYPERSCAN)
|
||||||
|
|
||||||
|
option (USE_INTERNAL_HYPERSCAN_LIBRARY "Set to FALSE to use system hyperscan instead of the bundled" ${NOT_UNBUNDLED})
|
||||||
|
|
||||||
|
if (NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/hyperscan/CMakeLists.txt")
|
||||||
|
if (USE_INTERNAL_HYPERSCAN_LIBRARY)
|
||||||
|
message (WARNING "submodule contrib/hyperscan is missing. to fix try run: \n git submodule update --init --recursive")
|
||||||
|
endif ()
|
||||||
|
set (MISSING_INTERNAL_HYPERSCAN_LIBRARY 1)
|
||||||
|
set (USE_INTERNAL_HYPERSCAN_LIBRARY 0)
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
if (NOT USE_INTERNAL_HYPERSCAN_LIBRARY)
|
||||||
|
find_library (HYPERSCAN_LIBRARY hs)
|
||||||
|
find_path (HYPERSCAN_INCLUDE_DIR NAMES hs/hs.h hs.h PATHS ${HYPERSCAN_INCLUDE_PATHS})
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
if (HYPERSCAN_LIBRARY AND HYPERSCAN_INCLUDE_DIR)
|
||||||
|
set (USE_HYPERSCAN 1)
|
||||||
|
elseif (NOT MISSING_INTERNAL_HYPERSCAN_LIBRARY)
|
||||||
|
set (HYPERSCAN_INCLUDE_DIR ${ClickHouse_SOURCE_DIR}/contrib/hyperscan/src)
|
||||||
|
set (HYPERSCAN_LIBRARY hs)
|
||||||
|
set (USE_HYPERSCAN 1)
|
||||||
|
set (USE_INTERNAL_HYPERSCAN_LIBRARY 1)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
message (STATUS "Using hyperscan=${USE_HYPERSCAN}: ${HYPERSCAN_INCLUDE_DIR} : ${HYPERSCAN_LIBRARY}")
|
||||||
|
|
||||||
|
endif ()
|
||||||
@ -1,6 +1,9 @@
|
|||||||
option(ENABLE_ICU "Enable ICU" ON)
|
option(ENABLE_ICU "Enable ICU" ON)
|
||||||
|
|
||||||
if(ENABLE_ICU)
|
if(ENABLE_ICU)
|
||||||
|
if (APPLE)
|
||||||
|
set(ICU_ROOT "/usr/local/opt/icu4c" CACHE STRING "")
|
||||||
|
endif()
|
||||||
find_package(ICU COMPONENTS i18n uc data) # TODO: remove Modules/FindICU.cmake after cmake 3.7
|
find_package(ICU COMPONENTS i18n uc data) # TODO: remove Modules/FindICU.cmake after cmake 3.7
|
||||||
#set (ICU_LIBRARIES ${ICU_I18N_LIBRARY} ${ICU_UC_LIBRARY} ${ICU_DATA_LIBRARY} CACHE STRING "")
|
#set (ICU_LIBRARIES ${ICU_I18N_LIBRARY} ${ICU_UC_LIBRARY} ${ICU_DATA_LIBRARY} CACHE STRING "")
|
||||||
if(ICU_FOUND)
|
if(ICU_FOUND)
|
||||||
|
|||||||
11
cmake/find_lfalloc.cmake
Normal file
11
cmake/find_lfalloc.cmake
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
# TODO(danlark1). Disable LFAlloc for a while to fix mmap count problem
|
||||||
|
if (NOT OS_LINUX AND NOT SANITIZE AND NOT ARCH_ARM AND NOT ARCH_32 AND NOT ARCH_PPC64LE AND NOT OS_FREEBSD AND NOT APPLE)
|
||||||
|
option (ENABLE_LFALLOC "Set to FALSE to use system libgsasl library instead of bundled" ${NOT_UNBUNDLED})
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
if (ENABLE_LFALLOC)
|
||||||
|
set (USE_LFALLOC 1)
|
||||||
|
set (USE_LFALLOC_RANDOM_HINT 1)
|
||||||
|
set (LFALLOC_INCLUDE_DIR ${ClickHouse_SOURCE_DIR}/contrib/lfalloc/src)
|
||||||
|
message (STATUS "Using lfalloc=${USE_LFALLOC}: ${LFALLOC_INCLUDE_DIR}")
|
||||||
|
endif ()
|
||||||
@ -22,4 +22,8 @@ elseif (NOT MISSING_INTERNAL_LIBGSASL_LIBRARY AND NOT APPLE AND NOT ARCH_32)
|
|||||||
set (LIBGSASL_LIBRARY libgsasl)
|
set (LIBGSASL_LIBRARY libgsasl)
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
message (STATUS "Using libgsasl: ${LIBGSASL_INCLUDE_DIR} : ${LIBGSASL_LIBRARY}")
|
if(LIBGSASL_LIBRARY AND LIBGSASL_INCLUDE_DIR)
|
||||||
|
set (USE_LIBGSASL 1)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
message (STATUS "Using libgsasl=${USE_LIBGSASL}: ${LIBGSASL_INCLUDE_DIR} : ${LIBGSASL_LIBRARY}")
|
||||||
|
|||||||
@ -1,21 +1,7 @@
|
|||||||
# This file copied from contrib/poco/cmake/FindODBC.cmake to allow build without submodules
|
# This file copied from contrib/poco/cmake/FindODBC.cmake to allow build without submodules
|
||||||
|
|
||||||
#
|
|
||||||
# Find the ODBC driver manager includes and library.
|
|
||||||
#
|
|
||||||
# ODBC is an open standard for connecting to different databases in a
|
|
||||||
# semi-vendor-independent fashion. First you install the ODBC driver
|
|
||||||
# manager. Then you need a driver for each separate database you want
|
|
||||||
# to connect to (unless a generic one works). VTK includes neither
|
|
||||||
# the driver manager nor the vendor-specific drivers: you have to find
|
|
||||||
# those yourself.
|
|
||||||
#
|
|
||||||
# This module defines
|
|
||||||
# ODBC_INCLUDE_DIRECTORIES, where to find sql.h
|
|
||||||
# ODBC_LIBRARIES, the libraries to link against to use ODBC
|
|
||||||
# ODBC_FOUND. If false, you cannot build anything that requires ODBC.
|
|
||||||
|
|
||||||
option (ENABLE_ODBC "Enable ODBC" ${OS_LINUX})
|
option (ENABLE_ODBC "Enable ODBC" ${OS_LINUX})
|
||||||
|
if(ENABLE_ODBC)
|
||||||
if (OS_LINUX)
|
if (OS_LINUX)
|
||||||
option(USE_INTERNAL_ODBC_LIBRARY "Set to FALSE to use system odbc library instead of bundled" ${NOT_UNBUNDLED})
|
option(USE_INTERNAL_ODBC_LIBRARY "Set to FALSE to use system odbc library instead of bundled" ${NOT_UNBUNDLED})
|
||||||
else ()
|
else ()
|
||||||
@ -25,69 +11,24 @@ endif ()
|
|||||||
if(USE_INTERNAL_ODBC_LIBRARY AND NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/unixodbc/README")
|
if(USE_INTERNAL_ODBC_LIBRARY AND NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/unixodbc/README")
|
||||||
message(WARNING "submodule contrib/unixodbc is missing. to fix try run: \n git submodule update --init --recursive")
|
message(WARNING "submodule contrib/unixodbc is missing. to fix try run: \n git submodule update --init --recursive")
|
||||||
set(USE_INTERNAL_ODBC_LIBRARY 0)
|
set(USE_INTERNAL_ODBC_LIBRARY 0)
|
||||||
|
set(MISSING_INTERNAL_ODBC_LIBRARY 1)
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
set (ODBC_INCLUDE_DIRECTORIES ) # Include directories will be either used automatically by target_include_directories or set later.
|
set(ODBC_INCLUDE_DIRS ) # Include directories will be either used automatically by target_include_directories or set later.
|
||||||
|
if(USE_INTERNAL_ODBC_LIBRARY AND NOT MISSING_INTERNAL_ODBC_LIBRARY)
|
||||||
if (ENABLE_ODBC)
|
set(ODBC_LIBRARY unixodbc)
|
||||||
if (USE_INTERNAL_ODBC_LIBRARY)
|
set(ODBC_LIBRARIES ${ODBC_LIBRARY})
|
||||||
set (ODBC_LIBRARIES unixodbc)
|
set(ODBC_INCLUDE_DIRS "${ClickHouse_SOURCE_DIR}/contrib/unixodbc/include")
|
||||||
set(ODBC_FOUND 1)
|
set(ODBC_FOUND 1)
|
||||||
set (USE_ODBC 1)
|
|
||||||
else()
|
else()
|
||||||
find_path(ODBC_INCLUDE_DIRECTORIES
|
find_package(ODBC)
|
||||||
NAMES sql.h
|
|
||||||
HINTS
|
|
||||||
/usr/include
|
|
||||||
/usr/include/iodbc
|
|
||||||
/usr/include/odbc
|
|
||||||
/usr/local/include
|
|
||||||
/usr/local/include/iodbc
|
|
||||||
/usr/local/include/odbc
|
|
||||||
/usr/local/iodbc/include
|
|
||||||
/usr/local/odbc/include
|
|
||||||
"C:/Program Files/ODBC/include"
|
|
||||||
"C:/Program Files/Microsoft SDKs/Windows/v7.0/include"
|
|
||||||
"C:/Program Files/Microsoft SDKs/Windows/v6.0a/include"
|
|
||||||
"C:/ODBC/include"
|
|
||||||
DOC "Specify the directory containing sql.h."
|
|
||||||
)
|
|
||||||
|
|
||||||
find_library(ODBC_LIBRARIES
|
|
||||||
NAMES iodbc odbc iodbcinst odbcinst odbc32
|
|
||||||
HINTS
|
|
||||||
/usr/lib
|
|
||||||
/usr/lib/iodbc
|
|
||||||
/usr/lib/odbc
|
|
||||||
/usr/local/lib
|
|
||||||
/usr/local/lib/iodbc
|
|
||||||
/usr/local/lib/odbc
|
|
||||||
/usr/local/iodbc/lib
|
|
||||||
/usr/local/odbc/lib
|
|
||||||
"C:/Program Files/ODBC/lib"
|
|
||||||
"C:/ODBC/lib/debug"
|
|
||||||
"C:/Program Files (x86)/Microsoft SDKs/Windows/v7.0A/Lib"
|
|
||||||
DOC "Specify the ODBC driver manager library here."
|
|
||||||
)
|
|
||||||
|
|
||||||
# MinGW find usually fails
|
|
||||||
if (MINGW)
|
|
||||||
set(ODBC_INCLUDE_DIRECTORIES ".")
|
|
||||||
set(ODBC_LIBRARIES odbc32)
|
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
include(FindPackageHandleStandardArgs)
|
if(ODBC_FOUND)
|
||||||
find_package_handle_standard_args(ODBC
|
set(USE_ODBC 1)
|
||||||
DEFAULT_MSG
|
set(ODBC_INCLUDE_DIRECTORIES ${ODBC_INCLUDE_DIRS}) # for old poco
|
||||||
ODBC_INCLUDE_DIRECTORIES
|
set(ODBC_INCLUDE_DIR ${ODBC_INCLUDE_DIRS}) # for old poco
|
||||||
ODBC_LIBRARIES)
|
|
||||||
|
|
||||||
if (USE_STATIC_LIBRARIES)
|
|
||||||
list(APPEND ODBC_LIBRARIES ${LTDL_LIBRARY})
|
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
mark_as_advanced(ODBC_FOUND ODBC_LIBRARIES ODBC_INCLUDE_DIRECTORIES)
|
message(STATUS "Using odbc=${USE_ODBC}: ${ODBC_INCLUDE_DIRS} : ${ODBC_LIBRARIES}")
|
||||||
endif()
|
endif()
|
||||||
endif ()
|
|
||||||
|
|
||||||
message (STATUS "Using odbc=${ODBC_FOUND}: ${ODBC_INCLUDE_DIRECTORIES} : ${ODBC_LIBRARIES}")
|
|
||||||
|
|||||||
@ -1,3 +1,7 @@
|
|||||||
|
option (ENABLE_PARQUET "Enable parquet" ON)
|
||||||
|
|
||||||
|
if (ENABLE_PARQUET)
|
||||||
|
|
||||||
if (NOT OS_FREEBSD) # Freebsd: ../contrib/arrow/cpp/src/arrow/util/bit-util.h:27:10: fatal error: endian.h: No such file or directory
|
if (NOT OS_FREEBSD) # Freebsd: ../contrib/arrow/cpp/src/arrow/util/bit-util.h:27:10: fatal error: endian.h: No such file or directory
|
||||||
option(USE_INTERNAL_PARQUET_LIBRARY "Set to FALSE to use system parquet library instead of bundled" ${NOT_UNBUNDLED})
|
option(USE_INTERNAL_PARQUET_LIBRARY "Set to FALSE to use system parquet library instead of bundled" ${NOT_UNBUNDLED})
|
||||||
endif()
|
endif()
|
||||||
@ -61,6 +65,8 @@ elseif(NOT MISSING_INTERNAL_PARQUET_LIBRARY AND NOT OS_FREEBSD)
|
|||||||
endif()
|
endif()
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
|
endif()
|
||||||
|
|
||||||
if(USE_PARQUET)
|
if(USE_PARQUET)
|
||||||
message(STATUS "Using Parquet: ${ARROW_LIBRARY}:${ARROW_INCLUDE_DIR} ; ${PARQUET_LIBRARY}:${PARQUET_INCLUDE_DIR} ; ${THRIFT_LIBRARY}")
|
message(STATUS "Using Parquet: ${ARROW_LIBRARY}:${ARROW_INCLUDE_DIR} ; ${PARQUET_LIBRARY}:${PARQUET_INCLUDE_DIR} ; ${THRIFT_LIBRARY}")
|
||||||
else()
|
else()
|
||||||
|
|||||||
@ -36,6 +36,8 @@ elseif (NOT MISSING_INTERNAL_POCO_LIBRARY)
|
|||||||
set (ENABLE_DATA_SQLITE 0 CACHE BOOL "")
|
set (ENABLE_DATA_SQLITE 0 CACHE BOOL "")
|
||||||
set (ENABLE_DATA_MYSQL 0 CACHE BOOL "")
|
set (ENABLE_DATA_MYSQL 0 CACHE BOOL "")
|
||||||
set (ENABLE_DATA_POSTGRESQL 0 CACHE BOOL "")
|
set (ENABLE_DATA_POSTGRESQL 0 CACHE BOOL "")
|
||||||
|
set (ENABLE_ENCODINGS 0 CACHE BOOL "")
|
||||||
|
|
||||||
# new after 2.0.0:
|
# new after 2.0.0:
|
||||||
set (POCO_ENABLE_ZIP 0 CACHE BOOL "")
|
set (POCO_ENABLE_ZIP 0 CACHE BOOL "")
|
||||||
set (POCO_ENABLE_PAGECOMPILER 0 CACHE BOOL "")
|
set (POCO_ENABLE_PAGECOMPILER 0 CACHE BOOL "")
|
||||||
@ -74,7 +76,7 @@ elseif (NOT MISSING_INTERNAL_POCO_LIBRARY)
|
|||||||
set (Poco_SQLODBC_INCLUDE_DIR
|
set (Poco_SQLODBC_INCLUDE_DIR
|
||||||
"${ClickHouse_SOURCE_DIR}/contrib/poco/SQL/ODBC/include/"
|
"${ClickHouse_SOURCE_DIR}/contrib/poco/SQL/ODBC/include/"
|
||||||
"${ClickHouse_SOURCE_DIR}/contrib/poco/Data/ODBC/include/"
|
"${ClickHouse_SOURCE_DIR}/contrib/poco/Data/ODBC/include/"
|
||||||
${ODBC_INCLUDE_DIRECTORIES}
|
${ODBC_INCLUDE_DIRS}
|
||||||
)
|
)
|
||||||
set (Poco_SQLODBC_LIBRARY PocoSQLODBC ${ODBC_LIBRARIES} ${LTDL_LIBRARY})
|
set (Poco_SQLODBC_LIBRARY PocoSQLODBC ${ODBC_LIBRARIES} ${LTDL_LIBRARY})
|
||||||
endif ()
|
endif ()
|
||||||
@ -86,7 +88,7 @@ elseif (NOT MISSING_INTERNAL_POCO_LIBRARY)
|
|||||||
set (USE_POCO_DATAODBC 1)
|
set (USE_POCO_DATAODBC 1)
|
||||||
set (Poco_DataODBC_INCLUDE_DIR
|
set (Poco_DataODBC_INCLUDE_DIR
|
||||||
"${ClickHouse_SOURCE_DIR}/contrib/poco/Data/ODBC/include/"
|
"${ClickHouse_SOURCE_DIR}/contrib/poco/Data/ODBC/include/"
|
||||||
${ODBC_INCLUDE_DIRECTORIES}
|
${ODBC_INCLUDE_DIRS}
|
||||||
)
|
)
|
||||||
set (Poco_DataODBC_LIBRARY PocoDataODBC ${ODBC_LIBRARIES} ${LTDL_LIBRARY})
|
set (Poco_DataODBC_LIBRARY PocoDataODBC ${ODBC_LIBRARIES} ${LTDL_LIBRARY})
|
||||||
endif ()
|
endif ()
|
||||||
|
|||||||
@ -1,10 +1,8 @@
|
|||||||
option(USE_INTERNAL_PROTOBUF_LIBRARY "Set to FALSE to use system protobuf instead of bundled" ${NOT_UNBUNDLED})
|
option (ENABLE_PROTOBUF "Enable protobuf" ON)
|
||||||
|
|
||||||
if(OS_FREEBSD AND SANITIZE STREQUAL "address")
|
if (ENABLE_PROTOBUF)
|
||||||
# ../contrib/protobuf/src/google/protobuf/arena_impl.h:45:10: fatal error: 'sanitizer/asan_interface.h' file not found
|
|
||||||
set(MISSING_INTERNAL_PROTOBUF_LIBRARY 1)
|
option(USE_INTERNAL_PROTOBUF_LIBRARY "Set to FALSE to use system protobuf instead of bundled" ${NOT_UNBUNDLED})
|
||||||
set(USE_INTERNAL_PROTOBUF_LIBRARY 0)
|
|
||||||
endif()
|
|
||||||
|
|
||||||
if(NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/protobuf/cmake/CMakeLists.txt")
|
if(NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/protobuf/cmake/CMakeLists.txt")
|
||||||
if(USE_INTERNAL_PROTOBUF_LIBRARY)
|
if(USE_INTERNAL_PROTOBUF_LIBRARY)
|
||||||
@ -94,4 +92,16 @@ elseif(NOT MISSING_INTERNAL_PROTOBUF_LIBRARY)
|
|||||||
endfunction()
|
endfunction()
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
|
if(OS_FREEBSD AND SANITIZE STREQUAL "address")
|
||||||
|
# ../contrib/protobuf/src/google/protobuf/arena_impl.h:45:10: fatal error: 'sanitizer/asan_interface.h' file not found
|
||||||
|
# #include <sanitizer/asan_interface.h>
|
||||||
|
if(LLVM_INCLUDE_DIRS)
|
||||||
|
set(Protobuf_INCLUDE_DIR ${Protobuf_INCLUDE_DIR} ${LLVM_INCLUDE_DIRS})
|
||||||
|
else()
|
||||||
|
set(USE_PROTOBUF 0)
|
||||||
|
endif()
|
||||||
|
endif()
|
||||||
|
|
||||||
|
endif()
|
||||||
|
|
||||||
message(STATUS "Using protobuf=${USE_PROTOBUF}: ${Protobuf_INCLUDE_DIR} : ${Protobuf_LIBRARY}")
|
message(STATUS "Using protobuf=${USE_PROTOBUF}: ${Protobuf_INCLUDE_DIR} : ${Protobuf_LIBRARY}")
|
||||||
|
|||||||
28
cmake/find_rapidjson.cmake
Normal file
28
cmake/find_rapidjson.cmake
Normal file
@ -0,0 +1,28 @@
|
|||||||
|
option(ENABLE_RAPIDJSON "Use rapidjson" ON)
|
||||||
|
if(NOT ENABLE_RAPIDJSON)
|
||||||
|
return()
|
||||||
|
endif()
|
||||||
|
|
||||||
|
option(USE_INTERNAL_RAPIDJSON_LIBRARY "Set to FALSE to use system rapidjson library instead of bundled" ${NOT_UNBUNDLED})
|
||||||
|
|
||||||
|
if(NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/rapidjson/include/rapidjson/rapidjson.h")
|
||||||
|
if(USE_INTERNAL_RAPIDJSON_LIBRARY)
|
||||||
|
message(WARNING "submodule contrib/rapidjson is missing. to fix try run: \n git submodule update --init --recursive")
|
||||||
|
set(USE_INTERNAL_RAPIDJSON_LIBRARY 0)
|
||||||
|
endif()
|
||||||
|
set(MISSING_INTERNAL_RAPIDJSON_LIBRARY 1)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
if(NOT USE_INTERNAL_RAPIDJSON_LIBRARY)
|
||||||
|
find_path(RAPIDJSON_INCLUDE_DIR NAMES rapidjson/rapidjson.h PATHS ${RAPIDJSON_INCLUDE_PATHS})
|
||||||
|
endif()
|
||||||
|
|
||||||
|
if(RAPIDJSON_INCLUDE_DIR)
|
||||||
|
set(USE_RAPIDJSON 1)
|
||||||
|
elseif(NOT MISSING_INTERNAL_RAPIDJSON_LIBRARY)
|
||||||
|
set(RAPIDJSON_INCLUDE_DIR "${ClickHouse_SOURCE_DIR}/contrib/rapidjson/include")
|
||||||
|
set(USE_INTERNAL_RAPIDJSON_LIBRARY 1)
|
||||||
|
set(USE_RAPIDJSON 1)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
message(STATUS "Using rapidjson=${USE_RAPIDJSON}: ${RAPIDJSON_INCLUDE_DIR}")
|
||||||
@ -1,5 +1,5 @@
|
|||||||
# Freebsd: contrib/cppkafka/include/cppkafka/detail/endianness.h:53:23: error: 'betoh16' was not declared in this scope
|
# Freebsd: contrib/cppkafka/include/cppkafka/detail/endianness.h:53:23: error: 'betoh16' was not declared in this scope
|
||||||
if (NOT ARCH_ARM AND NOT ARCH_32 AND NOT APPLE AND NOT OS_FREEBSD)
|
if (NOT ARCH_ARM AND NOT ARCH_32 AND NOT APPLE AND NOT OS_FREEBSD AND OPENSSL_FOUND)
|
||||||
option (ENABLE_RDKAFKA "Enable kafka" ON)
|
option (ENABLE_RDKAFKA "Enable kafka" ON)
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
@ -10,7 +10,7 @@ endif ()
|
|||||||
|
|
||||||
if (ENABLE_RDKAFKA)
|
if (ENABLE_RDKAFKA)
|
||||||
|
|
||||||
if (OS_LINUX AND NOT ARCH_ARM)
|
if (OS_LINUX AND NOT ARCH_ARM AND USE_LIBGSASL)
|
||||||
option (USE_INTERNAL_RDKAFKA_LIBRARY "Set to FALSE to use system librdkafka instead of the bundled" ${NOT_UNBUNDLED})
|
option (USE_INTERNAL_RDKAFKA_LIBRARY "Set to FALSE to use system librdkafka instead of the bundled" ${NOT_UNBUNDLED})
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
|
|||||||
@ -1,5 +1,13 @@
|
|||||||
option (USE_INTERNAL_RE2_LIBRARY "Set to FALSE to use system re2 library instead of bundled [slower]" ${NOT_UNBUNDLED})
|
option (USE_INTERNAL_RE2_LIBRARY "Set to FALSE to use system re2 library instead of bundled [slower]" ${NOT_UNBUNDLED})
|
||||||
|
|
||||||
|
if(NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/re2/CMakeLists.txt")
|
||||||
|
if(USE_INTERNAL_RE2_LIBRARY)
|
||||||
|
message(WARNING "submodule contrib/re2 is missing. to fix try run: \n git submodule update --init --recursive")
|
||||||
|
endif()
|
||||||
|
set(USE_INTERNAL_RE2_LIBRARY 0)
|
||||||
|
set(MISSING_INTERNAL_RE2_LIBRARY 1)
|
||||||
|
endif()
|
||||||
|
|
||||||
if (NOT USE_INTERNAL_RE2_LIBRARY)
|
if (NOT USE_INTERNAL_RE2_LIBRARY)
|
||||||
find_library (RE2_LIBRARY re2)
|
find_library (RE2_LIBRARY re2)
|
||||||
find_path (RE2_INCLUDE_DIR NAMES re2/re2.h PATHS ${RE2_INCLUDE_PATHS})
|
find_path (RE2_INCLUDE_DIR NAMES re2/re2.h PATHS ${RE2_INCLUDE_PATHS})
|
||||||
|
|||||||
14
cmake/find_simdjson.cmake
Normal file
14
cmake/find_simdjson.cmake
Normal file
@ -0,0 +1,14 @@
|
|||||||
|
if (NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/simdjson/include/simdjson/jsonparser.h")
|
||||||
|
message (WARNING "submodule contrib/simdjson is missing. to fix try run: \n git submodule update --init --recursive")
|
||||||
|
return()
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
if (NOT HAVE_AVX2)
|
||||||
|
message (WARNING "submodule contrib/simdjson requires AVX2 support")
|
||||||
|
return()
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
option (USE_SIMDJSON "Use simdjson" ON)
|
||||||
|
set (SIMDJSON_LIBRARY "simdjson")
|
||||||
|
|
||||||
|
message(STATUS "Using simdjson=${USE_SIMDJSON}: ${SIMDJSON_LIBRARY}")
|
||||||
@ -1,7 +1,19 @@
|
|||||||
|
option (ENABLE_SSL "Enable ssl" ON)
|
||||||
|
|
||||||
|
if (ENABLE_SSL)
|
||||||
|
|
||||||
if(NOT ARCH_32)
|
if(NOT ARCH_32)
|
||||||
option(USE_INTERNAL_SSL_LIBRARY "Set to FALSE to use system *ssl library instead of bundled" ${NOT_UNBUNDLED})
|
option(USE_INTERNAL_SSL_LIBRARY "Set to FALSE to use system *ssl library instead of bundled" ${NOT_UNBUNDLED})
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
|
if(NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/ssl/CMakeLists.txt")
|
||||||
|
if(USE_INTERNAL_SSL_LIBRARY)
|
||||||
|
message(WARNING "submodule contrib/ssl is missing. to fix try run: \n git submodule update --init --recursive")
|
||||||
|
endif()
|
||||||
|
set(USE_INTERNAL_SSL_LIBRARY 0)
|
||||||
|
set(MISSING_INTERNAL_SSL_LIBRARY 1)
|
||||||
|
endif()
|
||||||
|
|
||||||
set (OPENSSL_USE_STATIC_LIBS ${USE_STATIC_LIBRARIES})
|
set (OPENSSL_USE_STATIC_LIBS ${USE_STATIC_LIBRARIES})
|
||||||
|
|
||||||
if (NOT USE_INTERNAL_SSL_LIBRARY)
|
if (NOT USE_INTERNAL_SSL_LIBRARY)
|
||||||
@ -28,7 +40,7 @@ if (NOT USE_INTERNAL_SSL_LIBRARY)
|
|||||||
endif ()
|
endif ()
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (NOT OPENSSL_FOUND)
|
if (NOT OPENSSL_FOUND AND NOT MISSING_INTERNAL_SSL_LIBRARY)
|
||||||
set (USE_INTERNAL_SSL_LIBRARY 1)
|
set (USE_INTERNAL_SSL_LIBRARY 1)
|
||||||
set (OPENSSL_ROOT_DIR "${ClickHouse_SOURCE_DIR}/contrib/ssl")
|
set (OPENSSL_ROOT_DIR "${ClickHouse_SOURCE_DIR}/contrib/ssl")
|
||||||
set (OPENSSL_INCLUDE_DIR "${OPENSSL_ROOT_DIR}/include")
|
set (OPENSSL_INCLUDE_DIR "${OPENSSL_ROOT_DIR}/include")
|
||||||
@ -43,4 +55,77 @@ if (NOT OPENSSL_FOUND)
|
|||||||
set (OPENSSL_FOUND 1)
|
set (OPENSSL_FOUND 1)
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
message (STATUS "Using ssl=${OPENSSL_FOUND}: ${OPENSSL_INCLUDE_DIR} : ${OPENSSL_LIBRARIES}")
|
if(OPENSSL_FOUND)
|
||||||
|
# we need keep OPENSSL_FOUND for many libs in contrib
|
||||||
|
set(USE_SSL 1)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
# used by new poco
|
||||||
|
# part from /usr/share/cmake-*/Modules/FindOpenSSL.cmake, with removed all "EXISTS "
|
||||||
|
if(OPENSSL_FOUND AND NOT USE_INTERNAL_SSL_LIBRARY)
|
||||||
|
if(NOT TARGET OpenSSL::Crypto AND
|
||||||
|
(OPENSSL_CRYPTO_LIBRARY OR
|
||||||
|
LIB_EAY_LIBRARY_DEBUG OR
|
||||||
|
LIB_EAY_LIBRARY_RELEASE)
|
||||||
|
)
|
||||||
|
add_library(OpenSSL::Crypto UNKNOWN IMPORTED)
|
||||||
|
set_target_properties(OpenSSL::Crypto PROPERTIES
|
||||||
|
INTERFACE_INCLUDE_DIRECTORIES "${OPENSSL_INCLUDE_DIR}")
|
||||||
|
if(OPENSSL_CRYPTO_LIBRARY)
|
||||||
|
set_target_properties(OpenSSL::Crypto PROPERTIES
|
||||||
|
IMPORTED_LINK_INTERFACE_LANGUAGES "C"
|
||||||
|
IMPORTED_LOCATION "${OPENSSL_CRYPTO_LIBRARY}")
|
||||||
|
endif()
|
||||||
|
if(LIB_EAY_LIBRARY_RELEASE)
|
||||||
|
set_property(TARGET OpenSSL::Crypto APPEND PROPERTY
|
||||||
|
IMPORTED_CONFIGURATIONS RELEASE)
|
||||||
|
set_target_properties(OpenSSL::Crypto PROPERTIES
|
||||||
|
IMPORTED_LINK_INTERFACE_LANGUAGES_RELEASE "C"
|
||||||
|
IMPORTED_LOCATION_RELEASE "${LIB_EAY_LIBRARY_RELEASE}")
|
||||||
|
endif()
|
||||||
|
if(LIB_EAY_LIBRARY_DEBUG)
|
||||||
|
set_property(TARGET OpenSSL::Crypto APPEND PROPERTY
|
||||||
|
IMPORTED_CONFIGURATIONS DEBUG)
|
||||||
|
set_target_properties(OpenSSL::Crypto PROPERTIES
|
||||||
|
IMPORTED_LINK_INTERFACE_LANGUAGES_DEBUG "C"
|
||||||
|
IMPORTED_LOCATION_DEBUG "${LIB_EAY_LIBRARY_DEBUG}")
|
||||||
|
endif()
|
||||||
|
endif()
|
||||||
|
if(NOT TARGET OpenSSL::SSL AND
|
||||||
|
(OPENSSL_SSL_LIBRARY OR
|
||||||
|
SSL_EAY_LIBRARY_DEBUG OR
|
||||||
|
SSL_EAY_LIBRARY_RELEASE)
|
||||||
|
)
|
||||||
|
add_library(OpenSSL::SSL UNKNOWN IMPORTED)
|
||||||
|
set_target_properties(OpenSSL::SSL PROPERTIES
|
||||||
|
INTERFACE_INCLUDE_DIRECTORIES "${OPENSSL_INCLUDE_DIR}")
|
||||||
|
if(OPENSSL_SSL_LIBRARY)
|
||||||
|
set_target_properties(OpenSSL::SSL PROPERTIES
|
||||||
|
IMPORTED_LINK_INTERFACE_LANGUAGES "C"
|
||||||
|
IMPORTED_LOCATION "${OPENSSL_SSL_LIBRARY}")
|
||||||
|
endif()
|
||||||
|
if(SSL_EAY_LIBRARY_RELEASE)
|
||||||
|
set_property(TARGET OpenSSL::SSL APPEND PROPERTY
|
||||||
|
IMPORTED_CONFIGURATIONS RELEASE)
|
||||||
|
set_target_properties(OpenSSL::SSL PROPERTIES
|
||||||
|
IMPORTED_LINK_INTERFACE_LANGUAGES_RELEASE "C"
|
||||||
|
IMPORTED_LOCATION_RELEASE "${SSL_EAY_LIBRARY_RELEASE}")
|
||||||
|
endif()
|
||||||
|
if(SSL_EAY_LIBRARY_DEBUG)
|
||||||
|
set_property(TARGET OpenSSL::SSL APPEND PROPERTY
|
||||||
|
IMPORTED_CONFIGURATIONS DEBUG)
|
||||||
|
set_target_properties(OpenSSL::SSL PROPERTIES
|
||||||
|
IMPORTED_LINK_INTERFACE_LANGUAGES_DEBUG "C"
|
||||||
|
IMPORTED_LOCATION_DEBUG "${SSL_EAY_LIBRARY_DEBUG}")
|
||||||
|
endif()
|
||||||
|
if(TARGET OpenSSL::Crypto)
|
||||||
|
set_target_properties(OpenSSL::SSL PROPERTIES
|
||||||
|
INTERFACE_LINK_LIBRARIES OpenSSL::Crypto)
|
||||||
|
endif()
|
||||||
|
endif()
|
||||||
|
endif()
|
||||||
|
|
||||||
|
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
message (STATUS "Using ssl=${USE_SSL}: ${OPENSSL_INCLUDE_DIR} : ${OPENSSL_LIBRARIES}")
|
||||||
|
|||||||
@ -2,11 +2,6 @@ if (NOT OS_FREEBSD AND NOT ARCH_32)
|
|||||||
option (USE_INTERNAL_ZLIB_LIBRARY "Set to FALSE to use system zlib library instead of bundled" ${NOT_UNBUNDLED})
|
option (USE_INTERNAL_ZLIB_LIBRARY "Set to FALSE to use system zlib library instead of bundled" ${NOT_UNBUNDLED})
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (NOT USE_INTERNAL_ZLIB_LIBRARY)
|
|
||||||
find_package (ZLIB)
|
|
||||||
endif ()
|
|
||||||
|
|
||||||
if (NOT ZLIB_FOUND)
|
|
||||||
if (NOT MSVC)
|
if (NOT MSVC)
|
||||||
set (INTERNAL_ZLIB_NAME "zlib-ng" CACHE INTERNAL "")
|
set (INTERNAL_ZLIB_NAME "zlib-ng" CACHE INTERNAL "")
|
||||||
else ()
|
else ()
|
||||||
@ -16,6 +11,19 @@ if (NOT ZLIB_FOUND)
|
|||||||
endif ()
|
endif ()
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
|
if(NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/${INTERNAL_ZLIB_NAME}/zlib.h")
|
||||||
|
if(USE_INTERNAL_ZLIB_LIBRARY)
|
||||||
|
message(WARNING "submodule contrib/${INTERNAL_ZLIB_NAME} is missing. to fix try run: \n git submodule update --init --recursive")
|
||||||
|
endif()
|
||||||
|
set(USE_INTERNAL_ZLIB_LIBRARY 0)
|
||||||
|
set(MISSING_INTERNAL_ZLIB_LIBRARY 1)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
if (NOT USE_INTERNAL_ZLIB_LIBRARY)
|
||||||
|
find_package (ZLIB)
|
||||||
|
endif ()
|
||||||
|
|
||||||
|
if (NOT ZLIB_FOUND AND NOT MISSING_INTERNAL_ZLIB_LIBRARY)
|
||||||
set (USE_INTERNAL_ZLIB_LIBRARY 1)
|
set (USE_INTERNAL_ZLIB_LIBRARY 1)
|
||||||
set (ZLIB_INCLUDE_DIR "${ClickHouse_SOURCE_DIR}/contrib/${INTERNAL_ZLIB_NAME}" "${ClickHouse_BINARY_DIR}/contrib/${INTERNAL_ZLIB_NAME}" CACHE INTERNAL "") # generated zconf.h
|
set (ZLIB_INCLUDE_DIR "${ClickHouse_SOURCE_DIR}/contrib/${INTERNAL_ZLIB_NAME}" "${ClickHouse_BINARY_DIR}/contrib/${INTERNAL_ZLIB_NAME}" CACHE INTERNAL "") # generated zconf.h
|
||||||
set (ZLIB_INCLUDE_DIRS ${ZLIB_INCLUDE_DIR}) # for poco
|
set (ZLIB_INCLUDE_DIRS ${ZLIB_INCLUDE_DIR}) # for poco
|
||||||
|
|||||||
@ -1,8 +1,11 @@
|
|||||||
option (USE_INTERNAL_ZSTD_LIBRARY "Set to FALSE to use system zstd library instead of bundled" ${NOT_UNBUNDLED})
|
option (USE_INTERNAL_ZSTD_LIBRARY "Set to FALSE to use system zstd library instead of bundled" ${NOT_UNBUNDLED})
|
||||||
|
|
||||||
if (USE_INTERNAL_ZSTD_LIBRARY AND NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/zstd/lib/zstd.h")
|
if(NOT EXISTS "${ClickHouse_SOURCE_DIR}/contrib/zstd/lib/zstd.h")
|
||||||
|
if(USE_INTERNAL_ZSTD_LIBRARY)
|
||||||
message(WARNING "submodule contrib/zstd is missing. to fix try run: \n git submodule update --init --recursive")
|
message(WARNING "submodule contrib/zstd is missing. to fix try run: \n git submodule update --init --recursive")
|
||||||
|
endif()
|
||||||
set(USE_INTERNAL_ZSTD_LIBRARY 0)
|
set(USE_INTERNAL_ZSTD_LIBRARY 0)
|
||||||
|
set(MISSING_INTERNAL_ZSTD_LIBRARY 1)
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
if (NOT USE_INTERNAL_ZSTD_LIBRARY)
|
if (NOT USE_INTERNAL_ZSTD_LIBRARY)
|
||||||
@ -11,7 +14,7 @@ if (NOT USE_INTERNAL_ZSTD_LIBRARY)
|
|||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (ZSTD_LIBRARY AND ZSTD_INCLUDE_DIR)
|
if (ZSTD_LIBRARY AND ZSTD_INCLUDE_DIR)
|
||||||
else ()
|
elseif (NOT MISSING_INTERNAL_ZSTD_LIBRARY)
|
||||||
set (USE_INTERNAL_ZSTD_LIBRARY 1)
|
set (USE_INTERNAL_ZSTD_LIBRARY 1)
|
||||||
set (ZSTD_LIBRARY zstd)
|
set (ZSTD_LIBRARY zstd)
|
||||||
set (ZSTD_INCLUDE_DIR ${ClickHouse_SOURCE_DIR}/contrib/zstd/lib)
|
set (ZSTD_INCLUDE_DIR ${ClickHouse_SOURCE_DIR}/contrib/zstd/lib)
|
||||||
|
|||||||
@ -1,6 +1,9 @@
|
|||||||
include (CheckCXXSourceCompiles)
|
include (CheckCXXSourceCompiles)
|
||||||
include (CMakePushCheckState)
|
include (CMakePushCheckState)
|
||||||
|
|
||||||
|
set(THREADS_PREFER_PTHREAD_FLAG ON)
|
||||||
|
find_package(Threads)
|
||||||
|
|
||||||
cmake_push_check_state ()
|
cmake_push_check_state ()
|
||||||
|
|
||||||
if (CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
|
if (CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
|
||||||
|
|||||||
@ -11,36 +11,11 @@ if (OS_LINUX AND COMPILER_CLANG)
|
|||||||
set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}")
|
set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}")
|
||||||
|
|
||||||
option (USE_LIBCXX "Use libc++ and libc++abi instead of libstdc++ (only make sense on Linux with Clang)" ${HAVE_LIBCXX})
|
option (USE_LIBCXX "Use libc++ and libc++abi instead of libstdc++ (only make sense on Linux with Clang)" ${HAVE_LIBCXX})
|
||||||
set (LIBCXX_PATH "" CACHE STRING "Use custom path for libc++. It should be used for MSan.")
|
|
||||||
|
|
||||||
if (USE_LIBCXX)
|
if (USE_LIBCXX)
|
||||||
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -stdlib=libc++") # Ok for clang6, for older can cause 'not used option' warning
|
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -stdlib=libc++") # Ok for clang6, for older can cause 'not used option' warning
|
||||||
set (CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -D_LIBCPP_DEBUG=0") # More checks in debug build.
|
set (CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -D_LIBCPP_DEBUG=0") # More checks in debug build.
|
||||||
if (MAKE_STATIC_LIBRARIES)
|
|
||||||
execute_process (COMMAND ${CMAKE_CXX_COMPILER} --print-file-name=libclang_rt.builtins-${CMAKE_SYSTEM_PROCESSOR}.a OUTPUT_VARIABLE BUILTINS_LIB_PATH OUTPUT_STRIP_TRAILING_WHITESPACE)
|
|
||||||
link_libraries (-nodefaultlibs -Wl,-Bstatic -stdlib=libc++ c++ c++abi gcc_eh ${BUILTINS_LIB_PATH} rt -Wl,-Bdynamic dl pthread m c)
|
|
||||||
else ()
|
|
||||||
link_libraries (-stdlib=libc++ c++ c++abi)
|
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (LIBCXX_PATH)
|
|
||||||
include_directories (SYSTEM BEFORE "${LIBCXX_PATH}/include" "${LIBCXX_PATH}/include/c++/v1")
|
|
||||||
link_directories ("${LIBCXX_PATH}/lib")
|
|
||||||
endif ()
|
|
||||||
endif ()
|
|
||||||
endif ()
|
|
||||||
|
|
||||||
if (USE_LIBCXX)
|
|
||||||
set (STATIC_STDLIB_FLAGS "")
|
|
||||||
else ()
|
|
||||||
set (STATIC_STDLIB_FLAGS "-static-libgcc -static-libstdc++")
|
|
||||||
endif ()
|
|
||||||
|
|
||||||
if (MAKE_STATIC_LIBRARIES AND NOT APPLE AND NOT (COMPILER_CLANG AND OS_FREEBSD))
|
|
||||||
set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${STATIC_STDLIB_FLAGS}")
|
|
||||||
|
|
||||||
# Along with executables, we also build example of shared library for "library dictionary source"; and it also should be self-contained.
|
|
||||||
set (CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${STATIC_STDLIB_FLAGS}")
|
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (USE_STATIC_LIBRARIES AND HAVE_NO_PIE)
|
if (USE_STATIC_LIBRARIES AND HAVE_NO_PIE)
|
||||||
|
|||||||
31
contrib/CMakeLists.txt
vendored
31
contrib/CMakeLists.txt
vendored
@ -1,10 +1,10 @@
|
|||||||
# Third-party libraries may have substandard code.
|
# Third-party libraries may have substandard code.
|
||||||
|
|
||||||
if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
|
if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
|
||||||
set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-unused-function -Wno-unused-variable -Wno-unused-but-set-variable -Wno-unused-result -Wno-deprecated-declarations -Wno-maybe-uninitialized -Wno-format -Wno-misleading-indentation -Wno-stringop-overflow -Wno-implicit-function-declaration -Wno-return-type -Wno-array-bounds -Wno-bool-compare -Wno-int-conversion -Wno-switch")
|
set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-unused-function -Wno-unused-variable -Wno-unused-but-set-variable -Wno-unused-result -Wno-deprecated-declarations -Wno-maybe-uninitialized -Wno-format -Wno-misleading-indentation -Wno-stringop-overflow -Wno-implicit-function-declaration -Wno-return-type -Wno-array-bounds -Wno-bool-compare -Wno-int-conversion -Wno-switch -Wno-stringop-truncation")
|
||||||
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-old-style-cast -Wno-unused-function -Wno-unused-variable -Wno-unused-but-set-variable -Wno-unused-result -Wno-deprecated-declarations -Wno-non-virtual-dtor -Wno-maybe-uninitialized -Wno-format -Wno-misleading-indentation -Wno-implicit-fallthrough -Wno-class-memaccess -Wno-sign-compare -std=c++1z")
|
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-old-style-cast -Wno-unused-function -Wno-unused-variable -Wno-unused-but-set-variable -Wno-unused-result -Wno-deprecated-declarations -Wno-non-virtual-dtor -Wno-maybe-uninitialized -Wno-format -Wno-misleading-indentation -Wno-implicit-fallthrough -Wno-class-memaccess -Wno-sign-compare -Wno-array-bounds -Wno-missing-attributes -Wno-stringop-truncation -std=c++1z")
|
||||||
elseif (CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
|
elseif (CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
|
||||||
set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-unused-function -Wno-unused-variable -Wno-unused-result -Wno-deprecated-declarations -Wno-format -Wno-parentheses-equality -Wno-tautological-constant-compare -Wno-tautological-constant-out-of-range-compare -Wno-implicit-function-declaration -Wno-return-type -Wno-pointer-bool-conversion -Wno-enum-conversion -Wno-int-conversion -Wno-switch")
|
set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-unused-function -Wno-unused-variable -Wno-unused-result -Wno-deprecated-declarations -Wno-format -Wno-parentheses-equality -Wno-tautological-constant-compare -Wno-tautological-constant-out-of-range-compare -Wno-implicit-function-declaration -Wno-return-type -Wno-pointer-bool-conversion -Wno-enum-conversion -Wno-int-conversion -Wno-switch -Wno-string-plus-int")
|
||||||
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-old-style-cast -Wno-unused-function -Wno-unused-variable -Wno-unused-result -Wno-deprecated-declarations -Wno-non-virtual-dtor -Wno-format -Wno-inconsistent-missing-override -std=c++1z")
|
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-old-style-cast -Wno-unused-function -Wno-unused-variable -Wno-unused-result -Wno-deprecated-declarations -Wno-non-virtual-dtor -Wno-format -Wno-inconsistent-missing-override -std=c++1z")
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
@ -120,19 +120,26 @@ if (USE_INTERNAL_SSL_LIBRARY)
|
|||||||
target_include_directories(${OPENSSL_CRYPTO_LIBRARY} SYSTEM PUBLIC ${OPENSSL_INCLUDE_DIR})
|
target_include_directories(${OPENSSL_CRYPTO_LIBRARY} SYSTEM PUBLIC ${OPENSSL_INCLUDE_DIR})
|
||||||
target_include_directories(${OPENSSL_SSL_LIBRARY} SYSTEM PUBLIC ${OPENSSL_INCLUDE_DIR})
|
target_include_directories(${OPENSSL_SSL_LIBRARY} SYSTEM PUBLIC ${OPENSSL_INCLUDE_DIR})
|
||||||
set (POCO_SKIP_OPENSSL_FIND 1)
|
set (POCO_SKIP_OPENSSL_FIND 1)
|
||||||
|
|
||||||
|
add_library(OpenSSL::Crypto ALIAS ${OPENSSL_CRYPTO_LIBRARY})
|
||||||
|
add_library(OpenSSL::SSL ALIAS ${OPENSSL_SSL_LIBRARY})
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (ENABLE_MYSQL AND USE_INTERNAL_MYSQL_LIBRARY)
|
if (ENABLE_MYSQL AND USE_INTERNAL_MYSQL_LIBRARY)
|
||||||
add_subdirectory (mariadb-connector-c-cmake)
|
add_subdirectory (mariadb-connector-c-cmake)
|
||||||
target_include_directories(mysqlclient BEFORE PRIVATE ${ZLIB_INCLUDE_DIR})
|
target_include_directories(mysqlclient BEFORE PRIVATE ${ZLIB_INCLUDE_DIR})
|
||||||
|
if(OPENSSL_INCLUDE_DIR)
|
||||||
target_include_directories(mysqlclient BEFORE PRIVATE ${OPENSSL_INCLUDE_DIR})
|
target_include_directories(mysqlclient BEFORE PRIVATE ${OPENSSL_INCLUDE_DIR})
|
||||||
endif()
|
endif()
|
||||||
|
endif ()
|
||||||
|
|
||||||
if (USE_INTERNAL_RDKAFKA_LIBRARY)
|
if (USE_INTERNAL_RDKAFKA_LIBRARY)
|
||||||
add_subdirectory (librdkafka-cmake)
|
add_subdirectory (librdkafka-cmake)
|
||||||
target_include_directories(rdkafka BEFORE PRIVATE ${ZLIB_INCLUDE_DIR})
|
target_include_directories(rdkafka BEFORE PRIVATE ${ZLIB_INCLUDE_DIR})
|
||||||
|
if(OPENSSL_INCLUDE_DIR)
|
||||||
target_include_directories(rdkafka BEFORE PRIVATE ${OPENSSL_INCLUDE_DIR})
|
target_include_directories(rdkafka BEFORE PRIVATE ${OPENSSL_INCLUDE_DIR})
|
||||||
endif()
|
endif()
|
||||||
|
endif ()
|
||||||
|
|
||||||
if (USE_RDKAFKA)
|
if (USE_RDKAFKA)
|
||||||
add_subdirectory (cppkafka-cmake)
|
add_subdirectory (cppkafka-cmake)
|
||||||
@ -140,6 +147,7 @@ endif()
|
|||||||
|
|
||||||
if (ENABLE_ODBC AND USE_INTERNAL_ODBC_LIBRARY)
|
if (ENABLE_ODBC AND USE_INTERNAL_ODBC_LIBRARY)
|
||||||
add_subdirectory (unixodbc-cmake)
|
add_subdirectory (unixodbc-cmake)
|
||||||
|
add_library(ODBC::ODBC ALIAS ${ODBC_LIBRARIES})
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (USE_INTERNAL_CAPNP_LIBRARY)
|
if (USE_INTERNAL_CAPNP_LIBRARY)
|
||||||
@ -219,7 +227,7 @@ if (USE_INTERNAL_POCO_LIBRARY)
|
|||||||
set (ENABLE_TESTS 0)
|
set (ENABLE_TESTS 0)
|
||||||
set (POCO_ENABLE_TESTS 0)
|
set (POCO_ENABLE_TESTS 0)
|
||||||
set (CMAKE_DISABLE_FIND_PACKAGE_ZLIB 1)
|
set (CMAKE_DISABLE_FIND_PACKAGE_ZLIB 1)
|
||||||
if (MSVC)
|
if (MSVC OR NOT USE_POCO_DATAODBC)
|
||||||
set (ENABLE_DATA_ODBC 0 CACHE INTERNAL "") # TODO (build fail)
|
set (ENABLE_DATA_ODBC 0 CACHE INTERNAL "") # TODO (build fail)
|
||||||
endif ()
|
endif ()
|
||||||
add_subdirectory (poco)
|
add_subdirectory (poco)
|
||||||
@ -280,12 +288,17 @@ endif ()
|
|||||||
|
|
||||||
if (USE_INTERNAL_BROTLI_LIBRARY)
|
if (USE_INTERNAL_BROTLI_LIBRARY)
|
||||||
add_subdirectory(brotli-cmake)
|
add_subdirectory(brotli-cmake)
|
||||||
|
target_compile_definitions(brotli PRIVATE BROTLI_BUILD_PORTABLE=1)
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (USE_INTERNAL_PROTOBUF_LIBRARY)
|
if (USE_INTERNAL_PROTOBUF_LIBRARY)
|
||||||
set(protobuf_BUILD_TESTS OFF CACHE INTERNAL "" FORCE)
|
if (MAKE_STATIC_LIBRARIES)
|
||||||
set(protobuf_BUILD_SHARED_LIBS OFF CACHE INTERNAL "" FORCE)
|
set(protobuf_BUILD_SHARED_LIBS OFF CACHE INTERNAL "" FORCE)
|
||||||
|
else ()
|
||||||
|
set(protobuf_BUILD_SHARED_LIBS ON CACHE INTERNAL "" FORCE)
|
||||||
|
endif ()
|
||||||
set(protobuf_WITH_ZLIB 0 CACHE INTERNAL "" FORCE) # actually will use zlib, but skip find
|
set(protobuf_WITH_ZLIB 0 CACHE INTERNAL "" FORCE) # actually will use zlib, but skip find
|
||||||
|
set(protobuf_BUILD_TESTS OFF CACHE INTERNAL "" FORCE)
|
||||||
add_subdirectory(protobuf/cmake)
|
add_subdirectory(protobuf/cmake)
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
@ -296,3 +309,11 @@ endif ()
|
|||||||
if (USE_BASE64)
|
if (USE_BASE64)
|
||||||
add_subdirectory (base64-cmake)
|
add_subdirectory (base64-cmake)
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
|
if (USE_INTERNAL_HYPERSCAN_LIBRARY)
|
||||||
|
add_subdirectory (hyperscan)
|
||||||
|
endif()
|
||||||
|
|
||||||
|
if (USE_SIMDJSON)
|
||||||
|
add_subdirectory (simdjson-cmake)
|
||||||
|
endif()
|
||||||
|
|||||||
@ -41,7 +41,7 @@ set( thriftcpp_threads_SOURCES
|
|||||||
${LIBRARY_DIR}/src/thrift/concurrency/Monitor.cpp
|
${LIBRARY_DIR}/src/thrift/concurrency/Monitor.cpp
|
||||||
${LIBRARY_DIR}/src/thrift/concurrency/Mutex.cpp
|
${LIBRARY_DIR}/src/thrift/concurrency/Mutex.cpp
|
||||||
)
|
)
|
||||||
add_library(${THRIFT_LIBRARY} ${LINK_MODE} ${thriftcpp_SOURCES} ${thriftcpp_threads_SOURCES})
|
add_library(${THRIFT_LIBRARY} ${thriftcpp_SOURCES} ${thriftcpp_threads_SOURCES})
|
||||||
set_target_properties(${THRIFT_LIBRARY} PROPERTIES CXX_STANDARD 14) # REMOVE after https://github.com/apache/thrift/pull/1641
|
set_target_properties(${THRIFT_LIBRARY} PROPERTIES CXX_STANDARD 14) # REMOVE after https://github.com/apache/thrift/pull/1641
|
||||||
target_include_directories(${THRIFT_LIBRARY} SYSTEM PUBLIC ${ClickHouse_SOURCE_DIR}/contrib/thrift/lib/cpp/src PRIVATE ${Boost_INCLUDE_DIRS})
|
target_include_directories(${THRIFT_LIBRARY} SYSTEM PUBLIC ${ClickHouse_SOURCE_DIR}/contrib/thrift/lib/cpp/src PRIVATE ${Boost_INCLUDE_DIRS})
|
||||||
|
|
||||||
@ -149,7 +149,7 @@ if (ARROW_WITH_ZSTD)
|
|||||||
endif()
|
endif()
|
||||||
|
|
||||||
|
|
||||||
add_library(${ARROW_LIBRARY} ${LINK_MODE} ${ARROW_SRCS})
|
add_library(${ARROW_LIBRARY} ${ARROW_SRCS})
|
||||||
target_include_directories(${ARROW_LIBRARY} SYSTEM PUBLIC ${ClickHouse_SOURCE_DIR}/contrib/arrow/cpp/src PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/cpp/src ${Boost_INCLUDE_DIRS})
|
target_include_directories(${ARROW_LIBRARY} SYSTEM PUBLIC ${ClickHouse_SOURCE_DIR}/contrib/arrow/cpp/src PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/cpp/src ${Boost_INCLUDE_DIRS})
|
||||||
target_link_libraries(${ARROW_LIBRARY} PRIVATE ${DOUBLE_CONVERSION_LIBRARIES} Threads::Threads)
|
target_link_libraries(${ARROW_LIBRARY} PRIVATE ${DOUBLE_CONVERSION_LIBRARIES} Threads::Threads)
|
||||||
if (ARROW_WITH_LZ4)
|
if (ARROW_WITH_LZ4)
|
||||||
@ -195,7 +195,7 @@ list(APPEND PARQUET_SRCS
|
|||||||
${CMAKE_CURRENT_SOURCE_DIR}/cpp/src/parquet/parquet_constants.cpp
|
${CMAKE_CURRENT_SOURCE_DIR}/cpp/src/parquet/parquet_constants.cpp
|
||||||
${CMAKE_CURRENT_SOURCE_DIR}/cpp/src/parquet/parquet_types.cpp
|
${CMAKE_CURRENT_SOURCE_DIR}/cpp/src/parquet/parquet_types.cpp
|
||||||
)
|
)
|
||||||
add_library(${PARQUET_LIBRARY} ${LINK_MODE} ${PARQUET_SRCS})
|
add_library(${PARQUET_LIBRARY} ${PARQUET_SRCS})
|
||||||
target_include_directories(${PARQUET_LIBRARY} SYSTEM PUBLIC ${ClickHouse_SOURCE_DIR}/contrib/arrow/cpp/src ${CMAKE_CURRENT_SOURCE_DIR}/cpp/src)
|
target_include_directories(${PARQUET_LIBRARY} SYSTEM PUBLIC ${ClickHouse_SOURCE_DIR}/contrib/arrow/cpp/src ${CMAKE_CURRENT_SOURCE_DIR}/cpp/src)
|
||||||
include(${ClickHouse_SOURCE_DIR}/contrib/thrift/build/cmake/ConfigureChecks.cmake) # makes config.h
|
include(${ClickHouse_SOURCE_DIR}/contrib/thrift/build/cmake/ConfigureChecks.cmake) # makes config.h
|
||||||
target_link_libraries(${PARQUET_LIBRARY} PUBLIC ${ARROW_LIBRARY} PRIVATE ${THRIFT_LIBRARY} ${Boost_REGEX_LIBRARY})
|
target_link_libraries(${PARQUET_LIBRARY} PUBLIC ${ARROW_LIBRARY} PRIVATE ${THRIFT_LIBRARY} ${Boost_REGEX_LIBRARY})
|
||||||
|
|||||||
@ -24,7 +24,7 @@ endif ()
|
|||||||
|
|
||||||
configure_file(config.h.in ${CMAKE_CURRENT_BINARY_DIR}/config.h)
|
configure_file(config.h.in ${CMAKE_CURRENT_BINARY_DIR}/config.h)
|
||||||
|
|
||||||
add_library(base64 ${LINK_MODE}
|
add_library(base64
|
||||||
${LIBRARY_DIR}/lib/lib.c
|
${LIBRARY_DIR}/lib/lib.c
|
||||||
${LIBRARY_DIR}/lib/codec_choose.c
|
${LIBRARY_DIR}/lib/codec_choose.c
|
||||||
${LIBRARY_DIR}/lib/arch/avx/codec.c
|
${LIBRARY_DIR}/lib/arch/avx/codec.c
|
||||||
|
|||||||
2
contrib/boost
vendored
2
contrib/boost
vendored
@ -1 +1 @@
|
|||||||
Subproject commit 6a96e8b59f76148eb8ad54a9d15259f8ce84c606
|
Subproject commit 830e51edb59c4f37a8638138581e1e56c29ac44f
|
||||||
@ -20,7 +20,7 @@ endif()
|
|||||||
|
|
||||||
macro(add_boost_lib lib_name)
|
macro(add_boost_lib lib_name)
|
||||||
add_headers_and_sources(boost_${lib_name} ${LIBRARY_DIR}/libs/${lib_name}/src)
|
add_headers_and_sources(boost_${lib_name} ${LIBRARY_DIR}/libs/${lib_name}/src)
|
||||||
add_library(boost_${lib_name}_internal ${LINK_MODE} ${boost_${lib_name}_sources})
|
add_library(boost_${lib_name}_internal ${boost_${lib_name}_sources})
|
||||||
target_include_directories(boost_${lib_name}_internal SYSTEM BEFORE PUBLIC ${Boost_INCLUDE_DIRS})
|
target_include_directories(boost_${lib_name}_internal SYSTEM BEFORE PUBLIC ${Boost_INCLUDE_DIRS})
|
||||||
target_compile_definitions(boost_${lib_name}_internal PUBLIC BOOST_SYSTEM_NO_DEPRECATED)
|
target_compile_definitions(boost_${lib_name}_internal PUBLIC BOOST_SYSTEM_NO_DEPRECATED)
|
||||||
endmacro()
|
endmacro()
|
||||||
|
|||||||
@ -1,5 +1,5 @@
|
|||||||
set(BROTLI_SOURCE_DIR ${CMAKE_SOURCE_DIR}/contrib/brotli/c)
|
set(BROTLI_SOURCE_DIR ${ClickHouse_SOURCE_DIR}/contrib/brotli/c)
|
||||||
set(BROTLI_BINARY_DIR ${CMAKE_BINARY_DIR}/contrib/brotli/c)
|
set(BROTLI_BINARY_DIR ${ClickHouse_BINARY_DIR}/contrib/brotli/c)
|
||||||
|
|
||||||
set(SRCS
|
set(SRCS
|
||||||
${BROTLI_SOURCE_DIR}/dec/bit_reader.c
|
${BROTLI_SOURCE_DIR}/dec/bit_reader.c
|
||||||
@ -28,6 +28,6 @@ set(SRCS
|
|||||||
${BROTLI_SOURCE_DIR}/common/transform.c
|
${BROTLI_SOURCE_DIR}/common/transform.c
|
||||||
)
|
)
|
||||||
|
|
||||||
add_library(brotli ${LINK_MODE} ${SRCS})
|
add_library(brotli ${SRCS})
|
||||||
|
|
||||||
target_include_directories(brotli PUBLIC ${BROTLI_SOURCE_DIR}/include)
|
target_include_directories(brotli PUBLIC ${BROTLI_SOURCE_DIR}/include)
|
||||||
|
|||||||
@ -1,6 +1,6 @@
|
|||||||
SET(LIBRARY_DIR ${ClickHouse_SOURCE_DIR}/contrib/cctz)
|
SET(LIBRARY_DIR ${ClickHouse_SOURCE_DIR}/contrib/cctz)
|
||||||
|
|
||||||
add_library(cctz ${LINK_MODE}
|
add_library(cctz
|
||||||
${LIBRARY_DIR}/src/civil_time_detail.cc
|
${LIBRARY_DIR}/src/civil_time_detail.cc
|
||||||
${LIBRARY_DIR}/src/time_zone_fixed.cc
|
${LIBRARY_DIR}/src/time_zone_fixed.cc
|
||||||
${LIBRARY_DIR}/src/time_zone_format.cc
|
${LIBRARY_DIR}/src/time_zone_format.cc
|
||||||
|
|||||||
2
contrib/cppkafka
vendored
2
contrib/cppkafka
vendored
@ -1 +1 @@
|
|||||||
Subproject commit 860c90e92eee6690aa74a2ca7b7c5c6930dffecd
|
Subproject commit 9b184d881c15cc50784b28688c7c99d3d764db24
|
||||||
@ -1,4 +1,4 @@
|
|||||||
set(CPPKAFKA_DIR ${CMAKE_SOURCE_DIR}/contrib/cppkafka)
|
set(CPPKAFKA_DIR ${ClickHouse_SOURCE_DIR}/contrib/cppkafka)
|
||||||
|
|
||||||
set(SRCS
|
set(SRCS
|
||||||
${CPPKAFKA_DIR}/src/configuration.cpp
|
${CPPKAFKA_DIR}/src/configuration.cpp
|
||||||
@ -23,7 +23,7 @@ set(SRCS
|
|||||||
${CPPKAFKA_DIR}/src/consumer.cpp
|
${CPPKAFKA_DIR}/src/consumer.cpp
|
||||||
)
|
)
|
||||||
|
|
||||||
add_library(cppkafka ${LINK_MODE} ${SRCS})
|
add_library(cppkafka ${SRCS})
|
||||||
|
|
||||||
target_link_libraries(cppkafka PRIVATE ${RDKAFKA_LIBRARY})
|
target_link_libraries(cppkafka PRIVATE ${RDKAFKA_LIBRARY})
|
||||||
target_include_directories(cppkafka PRIVATE ${CPPKAFKA_DIR}/include/cppkafka)
|
target_include_directories(cppkafka PRIVATE ${CPPKAFKA_DIR}/include/cppkafka)
|
||||||
|
|||||||
@ -1,6 +1,6 @@
|
|||||||
add_library(roaring
|
add_library(roaring
|
||||||
roaring.c
|
roaring.c
|
||||||
roaring.h
|
roaring/roaring.h
|
||||||
roaring.hh)
|
roaring/roaring.hh)
|
||||||
|
|
||||||
target_include_directories (roaring PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
|
target_include_directories (roaring PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
|
||||||
|
|||||||
@ -1,5 +1,5 @@
|
|||||||
/* auto-generated on Tue Dec 18 09:42:59 CST 2018. Do not edit! */
|
/* auto-generated on Tue Dec 18 09:42:59 CST 2018. Do not edit! */
|
||||||
#include "roaring.h"
|
#include "roaring/roaring.h"
|
||||||
|
|
||||||
/* used for http://dmalloc.com/ Dmalloc - Debug Malloc Library */
|
/* used for http://dmalloc.com/ Dmalloc - Debug Malloc Library */
|
||||||
#ifdef DMALLOC
|
#ifdef DMALLOC
|
||||||
|
|||||||
1
contrib/hyperscan
vendored
Submodule
1
contrib/hyperscan
vendored
Submodule
@ -0,0 +1 @@
|
|||||||
|
Subproject commit 01e6b83f9fbdb4020cd68a5287bf3a0471eeb272
|
||||||
@ -1,4 +1,4 @@
|
|||||||
set(JEMALLOC_SOURCE_DIR ${CMAKE_SOURCE_DIR}/contrib/jemalloc)
|
set(JEMALLOC_SOURCE_DIR ${ClickHouse_SOURCE_DIR}/contrib/jemalloc)
|
||||||
|
|
||||||
set(SRCS
|
set(SRCS
|
||||||
${JEMALLOC_SOURCE_DIR}/src/arena.c
|
${JEMALLOC_SOURCE_DIR}/src/arena.c
|
||||||
@ -40,6 +40,10 @@ if(CMAKE_SYSTEM_NAME MATCHES "Darwin")
|
|||||||
list(APPEND SRCS ${JEMALLOC_SOURCE_DIR}/src/zone.c)
|
list(APPEND SRCS ${JEMALLOC_SOURCE_DIR}/src/zone.c)
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
|
if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
|
||||||
|
set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -w")
|
||||||
|
endif ()
|
||||||
|
|
||||||
add_library(jemalloc STATIC ${SRCS})
|
add_library(jemalloc STATIC ${SRCS})
|
||||||
|
|
||||||
target_include_directories(jemalloc PUBLIC
|
target_include_directories(jemalloc PUBLIC
|
||||||
|
|||||||
1820
contrib/lfalloc/src/lf_allocX64.h
Normal file
1820
contrib/lfalloc/src/lf_allocX64.h
Normal file
File diff suppressed because it is too large
Load Diff
23
contrib/lfalloc/src/lfmalloc.h
Normal file
23
contrib/lfalloc/src/lfmalloc.h
Normal file
@ -0,0 +1,23 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <string.h>
|
||||||
|
#include <stdlib.h>
|
||||||
|
#include "util/system/compiler.h"
|
||||||
|
|
||||||
|
namespace NMalloc {
|
||||||
|
volatile inline bool IsAllocatorCorrupted = false;
|
||||||
|
|
||||||
|
static inline void AbortFromCorruptedAllocator() {
|
||||||
|
IsAllocatorCorrupted = true;
|
||||||
|
abort();
|
||||||
|
}
|
||||||
|
|
||||||
|
struct TAllocHeader {
|
||||||
|
void* Block;
|
||||||
|
size_t AllocSize;
|
||||||
|
void Y_FORCE_INLINE Encode(void* block, size_t size, size_t signature) {
|
||||||
|
Block = block;
|
||||||
|
AllocSize = size | signature;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
33
contrib/lfalloc/src/util/README.md
Normal file
33
contrib/lfalloc/src/util/README.md
Normal file
@ -0,0 +1,33 @@
|
|||||||
|
Style guide for the util folder is a stricter version of general style guide (mostly in terms of ambiguity resolution).
|
||||||
|
|
||||||
|
* all {} must be in K&R style
|
||||||
|
* &, * tied closer to a type, not to variable
|
||||||
|
* always use `using` not `typedef`
|
||||||
|
* even a single line block must be in braces {}:
|
||||||
|
```
|
||||||
|
if (A) {
|
||||||
|
B();
|
||||||
|
}
|
||||||
|
```
|
||||||
|
* _ at the end of private data member of a class - `First_`, `Second_`
|
||||||
|
* every .h file must be accompanied with corresponding .cpp to avoid a leakage and check that it is self contained
|
||||||
|
* prohibited to use `printf`-like functions
|
||||||
|
|
||||||
|
|
||||||
|
Things declared in the general style guide, which sometimes are missed:
|
||||||
|
|
||||||
|
* `template <`, not `template<`
|
||||||
|
* `noexcept`, not `throw ()` nor `throw()`, not required for destructors
|
||||||
|
* indents inside `namespace` same as inside `class`
|
||||||
|
|
||||||
|
|
||||||
|
Requirements for a new code (and for corrections in an old code which involves change of behaviour) in util:
|
||||||
|
|
||||||
|
* presence of UNIT-tests
|
||||||
|
* presence of comments in Doxygen style
|
||||||
|
* accessors without Get prefix (`Length()`, but not `GetLength()`)
|
||||||
|
|
||||||
|
This guide is not a mandatory as there is the general style guide.
|
||||||
|
Nevertheless if it is not followed, then a next `ya style .` run in the util folder will undeservedly update authors of some lines of code.
|
||||||
|
|
||||||
|
Thus before a commit it is recommended to run `ya style .` in the util folder.
|
||||||
51
contrib/lfalloc/src/util/system/atomic.h
Normal file
51
contrib/lfalloc/src/util/system/atomic.h
Normal file
@ -0,0 +1,51 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include "defaults.h"
|
||||||
|
|
||||||
|
using TAtomicBase = intptr_t;
|
||||||
|
using TAtomic = volatile TAtomicBase;
|
||||||
|
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#include "atomic_gcc.h"
|
||||||
|
#elif defined(_MSC_VER)
|
||||||
|
#include "atomic_win.h"
|
||||||
|
#else
|
||||||
|
#error unsupported platform
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(ATOMIC_COMPILER_BARRIER)
|
||||||
|
#define ATOMIC_COMPILER_BARRIER()
|
||||||
|
#endif
|
||||||
|
|
||||||
|
static inline TAtomicBase AtomicSub(TAtomic& a, TAtomicBase v) {
|
||||||
|
return AtomicAdd(a, -v);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline TAtomicBase AtomicGetAndSub(TAtomic& a, TAtomicBase v) {
|
||||||
|
return AtomicGetAndAdd(a, -v);
|
||||||
|
}
|
||||||
|
|
||||||
|
#if defined(USE_GENERIC_SETGET)
|
||||||
|
static inline TAtomicBase AtomicGet(const TAtomic& a) {
|
||||||
|
return a;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline void AtomicSet(TAtomic& a, TAtomicBase v) {
|
||||||
|
a = v;
|
||||||
|
}
|
||||||
|
#endif
|
||||||
|
|
||||||
|
static inline bool AtomicTryLock(TAtomic* a) {
|
||||||
|
return AtomicCas(a, 1, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline bool AtomicTryAndTryLock(TAtomic* a) {
|
||||||
|
return (AtomicGet(*a) == 0) && AtomicTryLock(a);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline void AtomicUnlock(TAtomic* a) {
|
||||||
|
ATOMIC_COMPILER_BARRIER();
|
||||||
|
AtomicSet(*a, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
#include "atomic_ops.h"
|
||||||
90
contrib/lfalloc/src/util/system/atomic_gcc.h
Normal file
90
contrib/lfalloc/src/util/system/atomic_gcc.h
Normal file
@ -0,0 +1,90 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#define ATOMIC_COMPILER_BARRIER() __asm__ __volatile__("" \
|
||||||
|
: \
|
||||||
|
: \
|
||||||
|
: "memory")
|
||||||
|
|
||||||
|
static inline TAtomicBase AtomicGet(const TAtomic& a) {
|
||||||
|
TAtomicBase tmp;
|
||||||
|
#if defined(_arm64_)
|
||||||
|
__asm__ __volatile__(
|
||||||
|
"ldar %x[value], %[ptr] \n\t"
|
||||||
|
: [value] "=r"(tmp)
|
||||||
|
: [ptr] "Q"(a)
|
||||||
|
: "memory");
|
||||||
|
#else
|
||||||
|
__atomic_load(&a, &tmp, __ATOMIC_ACQUIRE);
|
||||||
|
#endif
|
||||||
|
return tmp;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline void AtomicSet(TAtomic& a, TAtomicBase v) {
|
||||||
|
#if defined(_arm64_)
|
||||||
|
__asm__ __volatile__(
|
||||||
|
"stlr %x[value], %[ptr] \n\t"
|
||||||
|
: [ptr] "=Q"(a)
|
||||||
|
: [value] "r"(v)
|
||||||
|
: "memory");
|
||||||
|
#else
|
||||||
|
__atomic_store(&a, &v, __ATOMIC_RELEASE);
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicIncrement(TAtomic& p) {
|
||||||
|
return __atomic_add_fetch(&p, 1, __ATOMIC_SEQ_CST);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndIncrement(TAtomic& p) {
|
||||||
|
return __atomic_fetch_add(&p, 1, __ATOMIC_SEQ_CST);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicDecrement(TAtomic& p) {
|
||||||
|
return __atomic_sub_fetch(&p, 1, __ATOMIC_SEQ_CST);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndDecrement(TAtomic& p) {
|
||||||
|
return __atomic_fetch_sub(&p, 1, __ATOMIC_SEQ_CST);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicAdd(TAtomic& p, intptr_t v) {
|
||||||
|
return __atomic_add_fetch(&p, v, __ATOMIC_SEQ_CST);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndAdd(TAtomic& p, intptr_t v) {
|
||||||
|
return __atomic_fetch_add(&p, v, __ATOMIC_SEQ_CST);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicSwap(TAtomic* p, intptr_t v) {
|
||||||
|
(void)p; // disable strange 'parameter set but not used' warning on gcc
|
||||||
|
intptr_t ret;
|
||||||
|
__atomic_exchange(p, &v, &ret, __ATOMIC_SEQ_CST);
|
||||||
|
return ret;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline bool AtomicCas(TAtomic* a, intptr_t exchange, intptr_t compare) {
|
||||||
|
(void)a; // disable strange 'parameter set but not used' warning on gcc
|
||||||
|
return __atomic_compare_exchange(a, &compare, &exchange, false, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndCas(TAtomic* a, intptr_t exchange, intptr_t compare) {
|
||||||
|
(void)a; // disable strange 'parameter set but not used' warning on gcc
|
||||||
|
__atomic_compare_exchange(a, &compare, &exchange, false, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
|
||||||
|
return compare;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicOr(TAtomic& a, intptr_t b) {
|
||||||
|
return __atomic_or_fetch(&a, b, __ATOMIC_SEQ_CST);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicXor(TAtomic& a, intptr_t b) {
|
||||||
|
return __atomic_xor_fetch(&a, b, __ATOMIC_SEQ_CST);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicAnd(TAtomic& a, intptr_t b) {
|
||||||
|
return __atomic_and_fetch(&a, b, __ATOMIC_SEQ_CST);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline void AtomicBarrier() {
|
||||||
|
__sync_synchronize();
|
||||||
|
}
|
||||||
189
contrib/lfalloc/src/util/system/atomic_ops.h
Normal file
189
contrib/lfalloc/src/util/system/atomic_ops.h
Normal file
@ -0,0 +1,189 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <type_traits>
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TAtomic* AsAtomicPtr(T volatile* target) {
|
||||||
|
return reinterpret_cast<TAtomic*>(target);
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline const TAtomic* AsAtomicPtr(T const volatile* target) {
|
||||||
|
return reinterpret_cast<const TAtomic*>(target);
|
||||||
|
}
|
||||||
|
|
||||||
|
// integral types
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
struct TAtomicTraits {
|
||||||
|
enum {
|
||||||
|
Castable = std::is_integral<T>::value && sizeof(T) == sizeof(TAtomicBase) && !std::is_const<T>::value,
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
template <typename T, typename TT>
|
||||||
|
using TEnableIfCastable = std::enable_if_t<TAtomicTraits<T>::Castable, TT>;
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicGet(T const volatile& target) {
|
||||||
|
return static_cast<T>(AtomicGet(*AsAtomicPtr(&target)));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, void> AtomicSet(T volatile& target, TAtomicBase value) {
|
||||||
|
AtomicSet(*AsAtomicPtr(&target), value);
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicIncrement(T volatile& target) {
|
||||||
|
return static_cast<T>(AtomicIncrement(*AsAtomicPtr(&target)));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicGetAndIncrement(T volatile& target) {
|
||||||
|
return static_cast<T>(AtomicGetAndIncrement(*AsAtomicPtr(&target)));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicDecrement(T volatile& target) {
|
||||||
|
return static_cast<T>(AtomicDecrement(*AsAtomicPtr(&target)));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicGetAndDecrement(T volatile& target) {
|
||||||
|
return static_cast<T>(AtomicGetAndDecrement(*AsAtomicPtr(&target)));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicAdd(T volatile& target, TAtomicBase value) {
|
||||||
|
return static_cast<T>(AtomicAdd(*AsAtomicPtr(&target), value));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicGetAndAdd(T volatile& target, TAtomicBase value) {
|
||||||
|
return static_cast<T>(AtomicGetAndAdd(*AsAtomicPtr(&target), value));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicSub(T volatile& target, TAtomicBase value) {
|
||||||
|
return static_cast<T>(AtomicSub(*AsAtomicPtr(&target), value));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicGetAndSub(T volatile& target, TAtomicBase value) {
|
||||||
|
return static_cast<T>(AtomicGetAndSub(*AsAtomicPtr(&target), value));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicSwap(T volatile* target, TAtomicBase exchange) {
|
||||||
|
return static_cast<T>(AtomicSwap(AsAtomicPtr(target), exchange));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, bool> AtomicCas(T volatile* target, TAtomicBase exchange, TAtomicBase compare) {
|
||||||
|
return AtomicCas(AsAtomicPtr(target), exchange, compare);
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicGetAndCas(T volatile* target, TAtomicBase exchange, TAtomicBase compare) {
|
||||||
|
return static_cast<T>(AtomicGetAndCas(AsAtomicPtr(target), exchange, compare));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, bool> AtomicTryLock(T volatile* target) {
|
||||||
|
return AtomicTryLock(AsAtomicPtr(target));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, bool> AtomicTryAndTryLock(T volatile* target) {
|
||||||
|
return AtomicTryAndTryLock(AsAtomicPtr(target));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, void> AtomicUnlock(T volatile* target) {
|
||||||
|
AtomicUnlock(AsAtomicPtr(target));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicOr(T volatile& target, TAtomicBase value) {
|
||||||
|
return static_cast<T>(AtomicOr(*AsAtomicPtr(&target), value));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicAnd(T volatile& target, TAtomicBase value) {
|
||||||
|
return static_cast<T>(AtomicAnd(*AsAtomicPtr(&target), value));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline TEnableIfCastable<T, T> AtomicXor(T volatile& target, TAtomicBase value) {
|
||||||
|
return static_cast<T>(AtomicXor(*AsAtomicPtr(&target), value));
|
||||||
|
}
|
||||||
|
|
||||||
|
// pointer types
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline T* AtomicGet(T* const volatile& target) {
|
||||||
|
return reinterpret_cast<T*>(AtomicGet(*AsAtomicPtr(&target)));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline void AtomicSet(T* volatile& target, T* value) {
|
||||||
|
AtomicSet(*AsAtomicPtr(&target), reinterpret_cast<TAtomicBase>(value));
|
||||||
|
}
|
||||||
|
|
||||||
|
using TNullPtr = decltype(nullptr);
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline void AtomicSet(T* volatile& target, TNullPtr) {
|
||||||
|
AtomicSet(*AsAtomicPtr(&target), 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline T* AtomicSwap(T* volatile* target, T* exchange) {
|
||||||
|
return reinterpret_cast<T*>(AtomicSwap(AsAtomicPtr(target), reinterpret_cast<TAtomicBase>(exchange)));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline T* AtomicSwap(T* volatile* target, TNullPtr) {
|
||||||
|
return reinterpret_cast<T*>(AtomicSwap(AsAtomicPtr(target), 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline bool AtomicCas(T* volatile* target, T* exchange, T* compare) {
|
||||||
|
return AtomicCas(AsAtomicPtr(target), reinterpret_cast<TAtomicBase>(exchange), reinterpret_cast<TAtomicBase>(compare));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline T* AtomicGetAndCas(T* volatile* target, T* exchange, T* compare) {
|
||||||
|
return reinterpret_cast<T*>(AtomicGetAndCas(AsAtomicPtr(target), reinterpret_cast<TAtomicBase>(exchange), reinterpret_cast<TAtomicBase>(compare)));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline bool AtomicCas(T* volatile* target, T* exchange, TNullPtr) {
|
||||||
|
return AtomicCas(AsAtomicPtr(target), reinterpret_cast<TAtomicBase>(exchange), 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline T* AtomicGetAndCas(T* volatile* target, T* exchange, TNullPtr) {
|
||||||
|
return reinterpret_cast<T*>(AtomicGetAndCas(AsAtomicPtr(target), reinterpret_cast<TAtomicBase>(exchange), 0));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline bool AtomicCas(T* volatile* target, TNullPtr, T* compare) {
|
||||||
|
return AtomicCas(AsAtomicPtr(target), 0, reinterpret_cast<TAtomicBase>(compare));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline T* AtomicGetAndCas(T* volatile* target, TNullPtr, T* compare) {
|
||||||
|
return reinterpret_cast<T*>(AtomicGetAndCas(AsAtomicPtr(target), 0, reinterpret_cast<TAtomicBase>(compare)));
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline bool AtomicCas(T* volatile* target, TNullPtr, TNullPtr) {
|
||||||
|
return AtomicCas(AsAtomicPtr(target), 0, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename T>
|
||||||
|
inline T* AtomicGetAndCas(T* volatile* target, TNullPtr, TNullPtr) {
|
||||||
|
return reinterpret_cast<T*>(AtomicGetAndCas(AsAtomicPtr(target), 0, 0));
|
||||||
|
}
|
||||||
114
contrib/lfalloc/src/util/system/atomic_win.h
Normal file
114
contrib/lfalloc/src/util/system/atomic_win.h
Normal file
@ -0,0 +1,114 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <intrin.h>
|
||||||
|
|
||||||
|
#define USE_GENERIC_SETGET
|
||||||
|
|
||||||
|
#if defined(_i386_)
|
||||||
|
|
||||||
|
#pragma intrinsic(_InterlockedIncrement)
|
||||||
|
#pragma intrinsic(_InterlockedDecrement)
|
||||||
|
#pragma intrinsic(_InterlockedExchangeAdd)
|
||||||
|
#pragma intrinsic(_InterlockedExchange)
|
||||||
|
#pragma intrinsic(_InterlockedCompareExchange)
|
||||||
|
|
||||||
|
static inline intptr_t AtomicIncrement(TAtomic& a) {
|
||||||
|
return _InterlockedIncrement((volatile long*)&a);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndIncrement(TAtomic& a) {
|
||||||
|
return _InterlockedIncrement((volatile long*)&a) - 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicDecrement(TAtomic& a) {
|
||||||
|
return _InterlockedDecrement((volatile long*)&a);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndDecrement(TAtomic& a) {
|
||||||
|
return _InterlockedDecrement((volatile long*)&a) + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicAdd(TAtomic& a, intptr_t b) {
|
||||||
|
return _InterlockedExchangeAdd((volatile long*)&a, b) + b;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndAdd(TAtomic& a, intptr_t b) {
|
||||||
|
return _InterlockedExchangeAdd((volatile long*)&a, b);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicSwap(TAtomic* a, intptr_t b) {
|
||||||
|
return _InterlockedExchange((volatile long*)a, b);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline bool AtomicCas(TAtomic* a, intptr_t exchange, intptr_t compare) {
|
||||||
|
return _InterlockedCompareExchange((volatile long*)a, exchange, compare) == compare;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndCas(TAtomic* a, intptr_t exchange, intptr_t compare) {
|
||||||
|
return _InterlockedCompareExchange((volatile long*)a, exchange, compare);
|
||||||
|
}
|
||||||
|
|
||||||
|
#else // _x86_64_
|
||||||
|
|
||||||
|
#pragma intrinsic(_InterlockedIncrement64)
|
||||||
|
#pragma intrinsic(_InterlockedDecrement64)
|
||||||
|
#pragma intrinsic(_InterlockedExchangeAdd64)
|
||||||
|
#pragma intrinsic(_InterlockedExchange64)
|
||||||
|
#pragma intrinsic(_InterlockedCompareExchange64)
|
||||||
|
|
||||||
|
static inline intptr_t AtomicIncrement(TAtomic& a) {
|
||||||
|
return _InterlockedIncrement64((volatile __int64*)&a);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndIncrement(TAtomic& a) {
|
||||||
|
return _InterlockedIncrement64((volatile __int64*)&a) - 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicDecrement(TAtomic& a) {
|
||||||
|
return _InterlockedDecrement64((volatile __int64*)&a);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndDecrement(TAtomic& a) {
|
||||||
|
return _InterlockedDecrement64((volatile __int64*)&a) + 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicAdd(TAtomic& a, intptr_t b) {
|
||||||
|
return _InterlockedExchangeAdd64((volatile __int64*)&a, b) + b;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndAdd(TAtomic& a, intptr_t b) {
|
||||||
|
return _InterlockedExchangeAdd64((volatile __int64*)&a, b);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicSwap(TAtomic* a, intptr_t b) {
|
||||||
|
return _InterlockedExchange64((volatile __int64*)a, b);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline bool AtomicCas(TAtomic* a, intptr_t exchange, intptr_t compare) {
|
||||||
|
return _InterlockedCompareExchange64((volatile __int64*)a, exchange, compare) == compare;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicGetAndCas(TAtomic* a, intptr_t exchange, intptr_t compare) {
|
||||||
|
return _InterlockedCompareExchange64((volatile __int64*)a, exchange, compare);
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicOr(TAtomic& a, intptr_t b) {
|
||||||
|
return _InterlockedOr64(&a, b) | b;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicAnd(TAtomic& a, intptr_t b) {
|
||||||
|
return _InterlockedAnd64(&a, b) & b;
|
||||||
|
}
|
||||||
|
|
||||||
|
static inline intptr_t AtomicXor(TAtomic& a, intptr_t b) {
|
||||||
|
return _InterlockedXor64(&a, b) ^ b;
|
||||||
|
}
|
||||||
|
|
||||||
|
#endif // _x86_
|
||||||
|
|
||||||
|
//TODO
|
||||||
|
static inline void AtomicBarrier() {
|
||||||
|
TAtomic val = 0;
|
||||||
|
|
||||||
|
AtomicSwap(&val, 0);
|
||||||
|
}
|
||||||
617
contrib/lfalloc/src/util/system/compiler.h
Normal file
617
contrib/lfalloc/src/util/system/compiler.h
Normal file
@ -0,0 +1,617 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
// useful cross-platfrom definitions for compilers
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_FUNC_SIGNATURE
|
||||||
|
*
|
||||||
|
* Use this macro to get pretty function name (see example).
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* void Hi() {
|
||||||
|
* Cout << Y_FUNC_SIGNATURE << Endl;
|
||||||
|
* }
|
||||||
|
|
||||||
|
* template <typename T>
|
||||||
|
* void Do() {
|
||||||
|
* Cout << Y_FUNC_SIGNATURE << Endl;
|
||||||
|
* }
|
||||||
|
|
||||||
|
* int main() {
|
||||||
|
* Hi(); // void Hi()
|
||||||
|
* Do<int>(); // void Do() [T = int]
|
||||||
|
* Do<TString>(); // void Do() [T = TString]
|
||||||
|
* }
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#define Y_FUNC_SIGNATURE __PRETTY_FUNCTION__
|
||||||
|
#elif defined(_MSC_VER)
|
||||||
|
#define Y_FUNC_SIGNATURE __FUNCSIG__
|
||||||
|
#else
|
||||||
|
#define Y_FUNC_SIGNATURE ""
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifdef __GNUC__
|
||||||
|
#define Y_PRINTF_FORMAT(n, m) __attribute__((__format__(__printf__, n, m)))
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifndef Y_PRINTF_FORMAT
|
||||||
|
#define Y_PRINTF_FORMAT(n, m)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__clang__)
|
||||||
|
#define Y_NO_SANITIZE(...) __attribute__((no_sanitize(__VA_ARGS__)))
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(Y_NO_SANITIZE)
|
||||||
|
#define Y_NO_SANITIZE(...)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_DECLARE_UNUSED
|
||||||
|
*
|
||||||
|
* Macro is needed to silence compiler warning about unused entities (e.g. function or argument).
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* Y_DECLARE_UNUSED int FunctionUsedSolelyForDebugPurposes();
|
||||||
|
* assert(FunctionUsedSolelyForDebugPurposes() == 42);
|
||||||
|
*
|
||||||
|
* void Foo(const int argumentUsedOnlyForDebugPurposes Y_DECLARE_UNUSED) {
|
||||||
|
* assert(argumentUsedOnlyForDebugPurposes == 42);
|
||||||
|
* // however you may as well omit `Y_DECLARE_UNUSED` and use `UNUSED` macro instead
|
||||||
|
* Y_UNUSED(argumentUsedOnlyForDebugPurposes);
|
||||||
|
* }
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#ifdef __GNUC__
|
||||||
|
#define Y_DECLARE_UNUSED __attribute__((unused))
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifndef Y_DECLARE_UNUSED
|
||||||
|
#define Y_DECLARE_UNUSED
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#define Y_LIKELY(Cond) __builtin_expect(!!(Cond), 1)
|
||||||
|
#define Y_UNLIKELY(Cond) __builtin_expect(!!(Cond), 0)
|
||||||
|
#define Y_PREFETCH_READ(Pointer, Priority) __builtin_prefetch((const void*)(Pointer), 0, Priority)
|
||||||
|
#define Y_PREFETCH_WRITE(Pointer, Priority) __builtin_prefetch((const void*)(Pointer), 1, Priority)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_FORCE_INLINE
|
||||||
|
*
|
||||||
|
* Macro to use in place of 'inline' in function declaration/definition to force
|
||||||
|
* it to be inlined.
|
||||||
|
*/
|
||||||
|
#if !defined(Y_FORCE_INLINE)
|
||||||
|
#if defined(CLANG_COVERAGE)
|
||||||
|
#/* excessive __always_inline__ might significantly slow down compilation of an instrumented unit */
|
||||||
|
#define Y_FORCE_INLINE inline
|
||||||
|
#elif defined(_MSC_VER)
|
||||||
|
#define Y_FORCE_INLINE __forceinline
|
||||||
|
#elif defined(__GNUC__)
|
||||||
|
#/* Clang also defines __GNUC__ (as 4) */
|
||||||
|
#define Y_FORCE_INLINE inline __attribute__((__always_inline__))
|
||||||
|
#else
|
||||||
|
#define Y_FORCE_INLINE inline
|
||||||
|
#endif
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_NO_INLINE
|
||||||
|
*
|
||||||
|
* Macro to use in place of 'inline' in function declaration/definition to
|
||||||
|
* prevent it from being inlined.
|
||||||
|
*/
|
||||||
|
#if !defined(Y_NO_INLINE)
|
||||||
|
#if defined(_MSC_VER)
|
||||||
|
#define Y_NO_INLINE __declspec(noinline)
|
||||||
|
#elif defined(__GNUC__) || defined(__INTEL_COMPILER)
|
||||||
|
#/* Clang also defines __GNUC__ (as 4) */
|
||||||
|
#define Y_NO_INLINE __attribute__((__noinline__))
|
||||||
|
#else
|
||||||
|
#define Y_NO_INLINE
|
||||||
|
#endif
|
||||||
|
#endif
|
||||||
|
|
||||||
|
//to cheat compiler about strict aliasing or similar problems
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#define Y_FAKE_READ(X) \
|
||||||
|
do { \
|
||||||
|
__asm__ __volatile__("" \
|
||||||
|
: \
|
||||||
|
: "m"(X)); \
|
||||||
|
} while (0)
|
||||||
|
|
||||||
|
#define Y_FAKE_WRITE(X) \
|
||||||
|
do { \
|
||||||
|
__asm__ __volatile__("" \
|
||||||
|
: "=m"(X)); \
|
||||||
|
} while (0)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(Y_FAKE_READ)
|
||||||
|
#define Y_FAKE_READ(X)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(Y_FAKE_WRITE)
|
||||||
|
#define Y_FAKE_WRITE(X)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifndef Y_PREFETCH_READ
|
||||||
|
#define Y_PREFETCH_READ(Pointer, Priority) (void)(const void*)(Pointer), (void)Priority
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifndef Y_PREFETCH_WRITE
|
||||||
|
#define Y_PREFETCH_WRITE(Pointer, Priority) (void)(const void*)(Pointer), (void)Priority
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifndef Y_LIKELY
|
||||||
|
#define Y_LIKELY(Cond) (Cond)
|
||||||
|
#define Y_UNLIKELY(Cond) (Cond)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifdef __GNUC__
|
||||||
|
#define _packed __attribute__((packed))
|
||||||
|
#else
|
||||||
|
#define _packed
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#define Y_WARN_UNUSED_RESULT __attribute__((warn_unused_result))
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifndef Y_WARN_UNUSED_RESULT
|
||||||
|
#define Y_WARN_UNUSED_RESULT
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#define Y_HIDDEN __attribute__((visibility("hidden")))
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(Y_HIDDEN)
|
||||||
|
#define Y_HIDDEN
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#define Y_PUBLIC __attribute__((visibility("default")))
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(Y_PUBLIC)
|
||||||
|
#define Y_PUBLIC
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(Y_UNUSED) && !defined(__cplusplus)
|
||||||
|
#define Y_UNUSED(var) (void)(var)
|
||||||
|
#endif
|
||||||
|
#if !defined(Y_UNUSED) && defined(__cplusplus)
|
||||||
|
template <class... Types>
|
||||||
|
constexpr Y_FORCE_INLINE int Y_UNUSED(Types&&...) {
|
||||||
|
return 0;
|
||||||
|
};
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_ASSUME
|
||||||
|
*
|
||||||
|
* Macro that tells the compiler that it can generate optimized code
|
||||||
|
* as if the given expression will always evaluate true.
|
||||||
|
* The behavior is undefined if it ever evaluates false.
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* // factored into a function so that it's testable
|
||||||
|
* inline int Avg(int x, int y) {
|
||||||
|
* if (x >= 0 && y >= 0) {
|
||||||
|
* return (static_cast<unsigned>(x) + static_cast<unsigned>(y)) >> 1;
|
||||||
|
* } else {
|
||||||
|
* // a slower implementation
|
||||||
|
* }
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* // we know that xs and ys are non-negative from domain knowledge,
|
||||||
|
* // but we can't change the types of xs and ys because of API constrains
|
||||||
|
* int Foo(const TVector<int>& xs, const TVector<int>& ys) {
|
||||||
|
* TVector<int> avgs;
|
||||||
|
* avgs.resize(xs.size());
|
||||||
|
* for (size_t i = 0; i < xs.size(); ++i) {
|
||||||
|
* auto x = xs[i];
|
||||||
|
* auto y = ys[i];
|
||||||
|
* Y_ASSUME(x >= 0);
|
||||||
|
* Y_ASSUME(y >= 0);
|
||||||
|
* xs[i] = Avg(x, y);
|
||||||
|
* }
|
||||||
|
* }
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#define Y_ASSUME(condition) ((condition) ? (void)0 : __builtin_unreachable())
|
||||||
|
#elif defined(_MSC_VER)
|
||||||
|
#define Y_ASSUME(condition) __assume(condition)
|
||||||
|
#else
|
||||||
|
#define Y_ASSUME(condition) Y_UNUSED(condition)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifdef __cplusplus
|
||||||
|
[[noreturn]]
|
||||||
|
#endif
|
||||||
|
Y_HIDDEN void _YandexAbort();
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_UNREACHABLE
|
||||||
|
*
|
||||||
|
* Macro that marks the rest of the code branch unreachable.
|
||||||
|
* The behavior is undefined if it's ever reached.
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* switch (i % 3) {
|
||||||
|
* case 0:
|
||||||
|
* return foo;
|
||||||
|
* case 1:
|
||||||
|
* return bar;
|
||||||
|
* case 2:
|
||||||
|
* return baz;
|
||||||
|
* default:
|
||||||
|
* Y_UNREACHABLE();
|
||||||
|
* }
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__GNUC__) || defined(_MSC_VER)
|
||||||
|
#define Y_UNREACHABLE() Y_ASSUME(0)
|
||||||
|
#else
|
||||||
|
#define Y_UNREACHABLE() _YandexAbort()
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(undefined_sanitizer_enabled)
|
||||||
|
#define _ubsan_enabled_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifdef __clang__
|
||||||
|
|
||||||
|
#if __has_feature(thread_sanitizer)
|
||||||
|
#define _tsan_enabled_
|
||||||
|
#endif
|
||||||
|
#if __has_feature(memory_sanitizer)
|
||||||
|
#define _msan_enabled_
|
||||||
|
#endif
|
||||||
|
#if __has_feature(address_sanitizer)
|
||||||
|
#define _asan_enabled_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#else
|
||||||
|
|
||||||
|
#if defined(thread_sanitizer_enabled) || defined(__SANITIZE_THREAD__)
|
||||||
|
#define _tsan_enabled_
|
||||||
|
#endif
|
||||||
|
#if defined(memory_sanitizer_enabled)
|
||||||
|
#define _msan_enabled_
|
||||||
|
#endif
|
||||||
|
#if defined(address_sanitizer_enabled) || defined(__SANITIZE_ADDRESS__)
|
||||||
|
#define _asan_enabled_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(_asan_enabled_) || defined(_msan_enabled_) || defined(_tsan_enabled_) || defined(_ubsan_enabled_)
|
||||||
|
#define _san_enabled_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(_MSC_VER)
|
||||||
|
#define __PRETTY_FUNCTION__ __FUNCSIG__
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#define Y_WEAK __attribute__((weak))
|
||||||
|
#else
|
||||||
|
#define Y_WEAK
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__CUDACC_VER_MAJOR__)
|
||||||
|
#define Y_CUDA_AT_LEAST(x, y) (__CUDACC_VER_MAJOR__ > x || (__CUDACC_VER_MAJOR__ == x && __CUDACC_VER_MINOR__ >= y))
|
||||||
|
#else
|
||||||
|
#define Y_CUDA_AT_LEAST(x, y) 0
|
||||||
|
#endif
|
||||||
|
|
||||||
|
// NVidia CUDA C++ Compiler did not know about noexcept keyword until version 9.0
|
||||||
|
#if !Y_CUDA_AT_LEAST(9, 0)
|
||||||
|
#if defined(__CUDACC__) && !defined(noexcept)
|
||||||
|
#define noexcept throw ()
|
||||||
|
#endif
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#define Y_COLD __attribute__((cold))
|
||||||
|
#define Y_LEAF __attribute__((leaf))
|
||||||
|
#define Y_WRAPPER __attribute__((artificial))
|
||||||
|
#else
|
||||||
|
#define Y_COLD
|
||||||
|
#define Y_LEAF
|
||||||
|
#define Y_WRAPPER
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_PRAGMA
|
||||||
|
*
|
||||||
|
* Macro for use in other macros to define compiler pragma
|
||||||
|
* See below for other usage examples
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* #if defined(__clang__) || defined(__GNUC__)
|
||||||
|
* #define Y_PRAGMA_NO_WSHADOW \
|
||||||
|
* Y_PRAGMA("GCC diagnostic ignored \"-Wshadow\"")
|
||||||
|
* #elif defined(_MSC_VER)
|
||||||
|
* #define Y_PRAGMA_NO_WSHADOW \
|
||||||
|
* Y_PRAGMA("warning(disable:4456 4457")
|
||||||
|
* #else
|
||||||
|
* #define Y_PRAGMA_NO_WSHADOW
|
||||||
|
* #endif
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__clang__) || defined(__GNUC__)
|
||||||
|
#define Y_PRAGMA(x) _Pragma(x)
|
||||||
|
#elif defined(_MSC_VER)
|
||||||
|
#define Y_PRAGMA(x) __pragma(x)
|
||||||
|
#else
|
||||||
|
#define Y_PRAGMA(x)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_PRAGMA_DIAGNOSTIC_PUSH
|
||||||
|
*
|
||||||
|
* Cross-compiler pragma to save diagnostic settings
|
||||||
|
*
|
||||||
|
* @see
|
||||||
|
* GCC: https://gcc.gnu.org/onlinedocs/gcc/Diagnostic-Pragmas.html
|
||||||
|
* MSVC: https://msdn.microsoft.com/en-us/library/2c8f766e.aspx
|
||||||
|
* Clang: https://clang.llvm.org/docs/UsersManual.html#controlling-diagnostics-via-pragmas
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* Y_PRAGMA_DIAGNOSTIC_PUSH
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__clang__) || defined(__GNUC__)
|
||||||
|
#define Y_PRAGMA_DIAGNOSTIC_PUSH \
|
||||||
|
Y_PRAGMA("GCC diagnostic push")
|
||||||
|
#elif defined(_MSC_VER)
|
||||||
|
#define Y_PRAGMA_DIAGNOSTIC_PUSH \
|
||||||
|
Y_PRAGMA(warning(push))
|
||||||
|
#else
|
||||||
|
#define Y_PRAGMA_DIAGNOSTIC_PUSH
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_PRAGMA_DIAGNOSTIC_POP
|
||||||
|
*
|
||||||
|
* Cross-compiler pragma to restore diagnostic settings
|
||||||
|
*
|
||||||
|
* @see
|
||||||
|
* GCC: https://gcc.gnu.org/onlinedocs/gcc/Diagnostic-Pragmas.html
|
||||||
|
* MSVC: https://msdn.microsoft.com/en-us/library/2c8f766e.aspx
|
||||||
|
* Clang: https://clang.llvm.org/docs/UsersManual.html#controlling-diagnostics-via-pragmas
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* Y_PRAGMA_DIAGNOSTIC_POP
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__clang__) || defined(__GNUC__)
|
||||||
|
#define Y_PRAGMA_DIAGNOSTIC_POP \
|
||||||
|
Y_PRAGMA("GCC diagnostic pop")
|
||||||
|
#elif defined(_MSC_VER)
|
||||||
|
#define Y_PRAGMA_DIAGNOSTIC_POP \
|
||||||
|
Y_PRAGMA(warning(pop))
|
||||||
|
#else
|
||||||
|
#define Y_PRAGMA_DIAGNOSTIC_POP
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_PRAGMA_NO_WSHADOW
|
||||||
|
*
|
||||||
|
* Cross-compiler pragma to disable warnings about shadowing variables
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* Y_PRAGMA_DIAGNOSTIC_PUSH
|
||||||
|
* Y_PRAGMA_NO_WSHADOW
|
||||||
|
*
|
||||||
|
* // some code which use variable shadowing, e.g.:
|
||||||
|
*
|
||||||
|
* for (int i = 0; i < 100; ++i) {
|
||||||
|
* Use(i);
|
||||||
|
*
|
||||||
|
* for (int i = 42; i < 100500; ++i) { // this i is shadowing previous i
|
||||||
|
* AnotherUse(i);
|
||||||
|
* }
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* Y_PRAGMA_DIAGNOSTIC_POP
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__clang__) || defined(__GNUC__)
|
||||||
|
#define Y_PRAGMA_NO_WSHADOW \
|
||||||
|
Y_PRAGMA("GCC diagnostic ignored \"-Wshadow\"")
|
||||||
|
#elif defined(_MSC_VER)
|
||||||
|
#define Y_PRAGMA_NO_WSHADOW \
|
||||||
|
Y_PRAGMA(warning(disable : 4456 4457))
|
||||||
|
#else
|
||||||
|
#define Y_PRAGMA_NO_WSHADOW
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @ def Y_PRAGMA_NO_UNUSED_FUNCTION
|
||||||
|
*
|
||||||
|
* Cross-compiler pragma to disable warnings about unused functions
|
||||||
|
*
|
||||||
|
* @see
|
||||||
|
* GCC: https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
|
||||||
|
* Clang: https://clang.llvm.org/docs/DiagnosticsReference.html#wunused-function
|
||||||
|
* MSVC: there is no such warning
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* Y_PRAGMA_DIAGNOSTIC_PUSH
|
||||||
|
* Y_PRAGMA_NO_UNUSED_FUNCTION
|
||||||
|
*
|
||||||
|
* // some code which introduces a function which later will not be used, e.g.:
|
||||||
|
*
|
||||||
|
* void Foo() {
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* int main() {
|
||||||
|
* return 0; // Foo() never called
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* Y_PRAGMA_DIAGNOSTIC_POP
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__clang__) || defined(__GNUC__)
|
||||||
|
#define Y_PRAGMA_NO_UNUSED_FUNCTION \
|
||||||
|
Y_PRAGMA("GCC diagnostic ignored \"-Wunused-function\"")
|
||||||
|
#else
|
||||||
|
#define Y_PRAGMA_NO_UNUSED_FUNCTION
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @ def Y_PRAGMA_NO_UNUSED_PARAMETER
|
||||||
|
*
|
||||||
|
* Cross-compiler pragma to disable warnings about unused function parameters
|
||||||
|
*
|
||||||
|
* @see
|
||||||
|
* GCC: https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
|
||||||
|
* Clang: https://clang.llvm.org/docs/DiagnosticsReference.html#wunused-parameter
|
||||||
|
* MSVC: https://msdn.microsoft.com/en-us/library/26kb9fy0.aspx
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* Y_PRAGMA_DIAGNOSTIC_PUSH
|
||||||
|
* Y_PRAGMA_NO_UNUSED_PARAMETER
|
||||||
|
*
|
||||||
|
* // some code which introduces a function with unused parameter, e.g.:
|
||||||
|
*
|
||||||
|
* void foo(int a) {
|
||||||
|
* // a is not referenced
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* int main() {
|
||||||
|
* foo(1);
|
||||||
|
* return 0;
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* Y_PRAGMA_DIAGNOSTIC_POP
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__clang__) || defined(__GNUC__)
|
||||||
|
#define Y_PRAGMA_NO_UNUSED_PARAMETER \
|
||||||
|
Y_PRAGMA("GCC diagnostic ignored \"-Wunused-parameter\"")
|
||||||
|
#elif defined(_MSC_VER)
|
||||||
|
#define Y_PRAGMA_NO_UNUSED_PARAMETER \
|
||||||
|
Y_PRAGMA(warning(disable : 4100))
|
||||||
|
#else
|
||||||
|
#define Y_PRAGMA_NO_UNUSED_PARAMETER
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_PRAGMA_NO_DEPRECATED
|
||||||
|
*
|
||||||
|
* Cross compiler pragma to disable warnings and errors about deprecated
|
||||||
|
*
|
||||||
|
* @see
|
||||||
|
* GCC: https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
|
||||||
|
* Clang: https://clang.llvm.org/docs/DiagnosticsReference.html#wdeprecated
|
||||||
|
* MSVC: https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-3-c4996?view=vs-2017
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* Y_PRAGMA_DIAGNOSTIC_PUSH
|
||||||
|
* Y_PRAGMA_NO_DEPRECATED
|
||||||
|
*
|
||||||
|
* [deprecated] void foo() {
|
||||||
|
* // ...
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* int main() {
|
||||||
|
* foo();
|
||||||
|
* return 0;
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* Y_PRAGMA_DIAGNOSTIC_POP
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__clang__) || defined(__GNUC__)
|
||||||
|
#define Y_PRAGMA_NO_DEPRECATED \
|
||||||
|
Y_PRAGMA("GCC diagnostic ignored \"-Wdeprecated\"")
|
||||||
|
#elif defined(_MSC_VER)
|
||||||
|
#define Y_PRAGMA_NO_DEPRECATED \
|
||||||
|
Y_PRAGMA(warning(disable : 4996))
|
||||||
|
#else
|
||||||
|
#define Y_PRAGMA_NO_DEPRECATED
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__clang__) || defined(__GNUC__)
|
||||||
|
/**
|
||||||
|
* @def Y_CONST_FUNCTION
|
||||||
|
methods and functions, marked with this method are promised to:
|
||||||
|
1. do not have side effects
|
||||||
|
2. this method do not read global memory
|
||||||
|
NOTE: this attribute can't be set for methods that depend on data, pointed by this
|
||||||
|
this allow compilers to do hard optimization of that functions
|
||||||
|
NOTE: in common case this attribute can't be set if method have pointer-arguments
|
||||||
|
NOTE: as result there no any reason to discard result of such method
|
||||||
|
*/
|
||||||
|
#define Y_CONST_FUNCTION [[gnu::const]]
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(Y_CONST_FUNCTION)
|
||||||
|
#define Y_CONST_FUNCTION
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__clang__) || defined(__GNUC__)
|
||||||
|
/**
|
||||||
|
* @def Y_PURE_FUNCTION
|
||||||
|
methods and functions, marked with this method are promised to:
|
||||||
|
1. do not have side effects
|
||||||
|
2. result will be the same if no global memory changed
|
||||||
|
this allow compilers to do hard optimization of that functions
|
||||||
|
NOTE: as result there no any reason to discard result of such method
|
||||||
|
*/
|
||||||
|
#define Y_PURE_FUNCTION [[gnu::pure]]
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(Y_PURE_FUNCTION)
|
||||||
|
#define Y_PURE_FUNCTION
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @ def Y_HAVE_INT128
|
||||||
|
*
|
||||||
|
* Defined when the compiler supports __int128 extension
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
*
|
||||||
|
* #if defined(Y_HAVE_INT128)
|
||||||
|
* __int128 myVeryBigInt = 12345678901234567890;
|
||||||
|
* #endif
|
||||||
|
*
|
||||||
|
* @endcode
|
||||||
|
*/
|
||||||
|
#if defined(__SIZEOF_INT128__)
|
||||||
|
#define Y_HAVE_INT128 1
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* XRAY macro must be passed to compiler if XRay is enabled.
|
||||||
|
*
|
||||||
|
* Define everything XRay-specific as a macro so that it doesn't cause errors
|
||||||
|
* for compilers that doesn't support XRay.
|
||||||
|
*/
|
||||||
|
#if defined(XRAY) && defined(__cplusplus)
|
||||||
|
#include <xray/xray_interface.h>
|
||||||
|
#define Y_XRAY_ALWAYS_INSTRUMENT [[clang::xray_always_instrument]]
|
||||||
|
#define Y_XRAY_NEVER_INSTRUMENT [[clang::xray_never_instrument]]
|
||||||
|
#define Y_XRAY_CUSTOM_EVENT(__string, __length) \
|
||||||
|
do { \
|
||||||
|
__xray_customevent(__string, __length); \
|
||||||
|
} while (0)
|
||||||
|
#else
|
||||||
|
#define Y_XRAY_ALWAYS_INSTRUMENT
|
||||||
|
#define Y_XRAY_NEVER_INSTRUMENT
|
||||||
|
#define Y_XRAY_CUSTOM_EVENT(__string, __length) \
|
||||||
|
do { \
|
||||||
|
} while (0)
|
||||||
|
#endif
|
||||||
168
contrib/lfalloc/src/util/system/defaults.h
Normal file
168
contrib/lfalloc/src/util/system/defaults.h
Normal file
@ -0,0 +1,168 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include "platform.h"
|
||||||
|
|
||||||
|
#if defined _unix_
|
||||||
|
#define LOCSLASH_C '/'
|
||||||
|
#define LOCSLASH_S "/"
|
||||||
|
#else
|
||||||
|
#define LOCSLASH_C '\\'
|
||||||
|
#define LOCSLASH_S "\\"
|
||||||
|
#endif // _unix_
|
||||||
|
|
||||||
|
#if defined(__INTEL_COMPILER) && defined(__cplusplus)
|
||||||
|
#include <new>
|
||||||
|
#endif
|
||||||
|
|
||||||
|
// low and high parts of integers
|
||||||
|
#if !defined(_win_)
|
||||||
|
#include <sys/param.h>
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(BSD) || defined(_android_)
|
||||||
|
|
||||||
|
#if defined(BSD)
|
||||||
|
#include <machine/endian.h>
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(_android_)
|
||||||
|
#include <endian.h>
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if (BYTE_ORDER == LITTLE_ENDIAN)
|
||||||
|
#define _little_endian_
|
||||||
|
#elif (BYTE_ORDER == BIG_ENDIAN)
|
||||||
|
#define _big_endian_
|
||||||
|
#else
|
||||||
|
#error unknown endian not supported
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#elif (defined(_sun_) && !defined(__i386__)) || defined(_hpux_) || defined(WHATEVER_THAT_HAS_BIG_ENDIAN)
|
||||||
|
#define _big_endian_
|
||||||
|
#else
|
||||||
|
#define _little_endian_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
// alignment
|
||||||
|
#if (defined(_sun_) && !defined(__i386__)) || defined(_hpux_) || defined(__alpha__) || defined(__ia64__) || defined(WHATEVER_THAT_NEEDS_ALIGNING_QUADS)
|
||||||
|
#define _must_align8_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if (defined(_sun_) && !defined(__i386__)) || defined(_hpux_) || defined(__alpha__) || defined(__ia64__) || defined(WHATEVER_THAT_NEEDS_ALIGNING_LONGS)
|
||||||
|
#define _must_align4_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if (defined(_sun_) && !defined(__i386__)) || defined(_hpux_) || defined(__alpha__) || defined(__ia64__) || defined(WHATEVER_THAT_NEEDS_ALIGNING_SHORTS)
|
||||||
|
#define _must_align2_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__GNUC__)
|
||||||
|
#define alias_hack __attribute__((__may_alias__))
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifndef alias_hack
|
||||||
|
#define alias_hack
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#include "types.h"
|
||||||
|
|
||||||
|
#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L)
|
||||||
|
#define PRAGMA(x) _Pragma(#x)
|
||||||
|
#define RCSID(idstr) PRAGMA(comment(exestr, idstr))
|
||||||
|
#else
|
||||||
|
#define RCSID(idstr) static const char rcsid[] = idstr
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#include "compiler.h"
|
||||||
|
|
||||||
|
#ifdef _win_
|
||||||
|
#include <malloc.h>
|
||||||
|
#elif defined(_sun_)
|
||||||
|
#include <alloca.h>
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#ifdef NDEBUG
|
||||||
|
#define Y_IF_DEBUG(X)
|
||||||
|
#else
|
||||||
|
#define Y_IF_DEBUG(X) X
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @def Y_ARRAY_SIZE
|
||||||
|
*
|
||||||
|
* This macro is needed to get number of elements in a statically allocated fixed size array. The
|
||||||
|
* expression is a compile-time constant and therefore can be used in compile time computations.
|
||||||
|
*
|
||||||
|
* @code
|
||||||
|
* enum ENumbers {
|
||||||
|
* EN_ONE,
|
||||||
|
* EN_TWO,
|
||||||
|
* EN_SIZE
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* const char* NAMES[] = {
|
||||||
|
* "one",
|
||||||
|
* "two"
|
||||||
|
* }
|
||||||
|
*
|
||||||
|
* static_assert(Y_ARRAY_SIZE(NAMES) == EN_SIZE, "you should define `NAME` for each enumeration");
|
||||||
|
* @endcode
|
||||||
|
*
|
||||||
|
* This macro also catches type errors. If you see a compiler error like "warning: division by zero
|
||||||
|
* is undefined" when using `Y_ARRAY_SIZE` then you are probably giving it a pointer.
|
||||||
|
*
|
||||||
|
* Since all of our code is expected to work on a 64 bit platform where pointers are 8 bytes we may
|
||||||
|
* falsefully accept pointers to types of sizes that are divisors of 8 (1, 2, 4 and 8).
|
||||||
|
*/
|
||||||
|
#if defined(__cplusplus)
|
||||||
|
namespace NArraySizePrivate {
|
||||||
|
template <class T>
|
||||||
|
struct TArraySize;
|
||||||
|
|
||||||
|
template <class T, size_t N>
|
||||||
|
struct TArraySize<T[N]> {
|
||||||
|
enum {
|
||||||
|
Result = N
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
template <class T, size_t N>
|
||||||
|
struct TArraySize<T (&)[N]> {
|
||||||
|
enum {
|
||||||
|
Result = N
|
||||||
|
};
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
#define Y_ARRAY_SIZE(arr) ((size_t)::NArraySizePrivate::TArraySize<decltype(arr)>::Result)
|
||||||
|
#else
|
||||||
|
#undef Y_ARRAY_SIZE
|
||||||
|
#define Y_ARRAY_SIZE(arr) \
|
||||||
|
((sizeof(arr) / sizeof((arr)[0])) / static_cast<size_t>(!(sizeof(arr) % sizeof((arr)[0]))))
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#undef Y_ARRAY_BEGIN
|
||||||
|
#define Y_ARRAY_BEGIN(arr) (arr)
|
||||||
|
|
||||||
|
#undef Y_ARRAY_END
|
||||||
|
#define Y_ARRAY_END(arr) ((arr) + Y_ARRAY_SIZE(arr))
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Concatenates two symbols, even if one of them is itself a macro.
|
||||||
|
*/
|
||||||
|
#define Y_CAT(X, Y) Y_CAT_I(X, Y)
|
||||||
|
#define Y_CAT_I(X, Y) Y_CAT_II(X, Y)
|
||||||
|
#define Y_CAT_II(X, Y) X##Y
|
||||||
|
|
||||||
|
#define Y_STRINGIZE(X) UTIL_PRIVATE_STRINGIZE_AUX(X)
|
||||||
|
#define UTIL_PRIVATE_STRINGIZE_AUX(X) #X
|
||||||
|
|
||||||
|
#if defined(__COUNTER__)
|
||||||
|
#define Y_GENERATE_UNIQUE_ID(N) Y_CAT(N, __COUNTER__)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(Y_GENERATE_UNIQUE_ID)
|
||||||
|
#define Y_GENERATE_UNIQUE_ID(N) Y_CAT(N, __LINE__)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#define NPOS ((size_t)-1)
|
||||||
242
contrib/lfalloc/src/util/system/platform.h
Normal file
242
contrib/lfalloc/src/util/system/platform.h
Normal file
@ -0,0 +1,242 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
// What OS ?
|
||||||
|
// our definition has the form _{osname}_
|
||||||
|
|
||||||
|
#if defined(_WIN64)
|
||||||
|
#define _win64_
|
||||||
|
#define _win32_
|
||||||
|
#elif defined(__WIN32__) || defined(_WIN32) // _WIN32 is also defined by the 64-bit compiler for backward compatibility
|
||||||
|
#define _win32_
|
||||||
|
#else
|
||||||
|
#define _unix_
|
||||||
|
#if defined(__sun__) || defined(sun) || defined(sparc) || defined(__sparc)
|
||||||
|
#define _sun_
|
||||||
|
#endif
|
||||||
|
#if defined(__hpux__)
|
||||||
|
#define _hpux_
|
||||||
|
#endif
|
||||||
|
#if defined(__linux__)
|
||||||
|
#define _linux_
|
||||||
|
#endif
|
||||||
|
#if defined(__FreeBSD__)
|
||||||
|
#define _freebsd_
|
||||||
|
#endif
|
||||||
|
#if defined(__CYGWIN__)
|
||||||
|
#define _cygwin_
|
||||||
|
#endif
|
||||||
|
#if defined(__APPLE__)
|
||||||
|
#define _darwin_
|
||||||
|
#endif
|
||||||
|
#if defined(__ANDROID__)
|
||||||
|
#define _android_
|
||||||
|
#endif
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__IOS__)
|
||||||
|
#define _ios_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(_linux_)
|
||||||
|
#if defined(_musl_)
|
||||||
|
//nothing to do
|
||||||
|
#elif defined(_android_)
|
||||||
|
#define _bionic_
|
||||||
|
#else
|
||||||
|
#define _glibc_
|
||||||
|
#endif
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(_darwin_)
|
||||||
|
#define unix
|
||||||
|
#define __unix__
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(_win32_) || defined(_win64_)
|
||||||
|
#define _win_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__arm__) || defined(__ARM__) || defined(__ARM_NEON) || defined(__aarch64__) || defined(_M_ARM)
|
||||||
|
#if defined(__arm64) || defined(__arm64__) || defined(__aarch64__)
|
||||||
|
#define _arm64_
|
||||||
|
#else
|
||||||
|
#define _arm32_
|
||||||
|
#endif
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(_arm64_) || defined(_arm32_)
|
||||||
|
#define _arm_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/* __ia64__ and __x86_64__ - defined by GNU C.
|
||||||
|
* _M_IA64, _M_X64, _M_AMD64 - defined by Visual Studio.
|
||||||
|
*
|
||||||
|
* Microsoft can define _M_IX86, _M_AMD64 (before Visual Studio 8)
|
||||||
|
* or _M_X64 (starting in Visual Studio 8).
|
||||||
|
*/
|
||||||
|
#if defined(__x86_64__) || defined(_M_X64) || defined(_M_AMD64)
|
||||||
|
#define _x86_64_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__i386__) || defined(_M_IX86)
|
||||||
|
#define _i386_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__ia64__) || defined(_M_IA64)
|
||||||
|
#define _ia64_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__powerpc__)
|
||||||
|
#define _ppc_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__powerpc64__)
|
||||||
|
#define _ppc64_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if !defined(sparc) && !defined(__sparc) && !defined(__hpux__) && !defined(__alpha__) && !defined(_ia64_) && !defined(_x86_64_) && !defined(_arm_) && !defined(_i386_) && !defined(_ppc_) && !defined(_ppc64_)
|
||||||
|
#error "platform not defined, please, define one"
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(_x86_64_) || defined(_i386_)
|
||||||
|
#define _x86_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__MIC__)
|
||||||
|
#define _mic_
|
||||||
|
#define _k1om_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
// stdio or MessageBox
|
||||||
|
#if defined(__CONSOLE__) || defined(_CONSOLE)
|
||||||
|
#define _console_
|
||||||
|
#endif
|
||||||
|
#if (defined(_win_) && !defined(_console_))
|
||||||
|
#define _windows_
|
||||||
|
#elif !defined(_console_)
|
||||||
|
#define _console_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__SSE__) || defined(SSE_ENABLED)
|
||||||
|
#define _sse_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__SSE2__) || defined(SSE2_ENABLED)
|
||||||
|
#define _sse2_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__SSE3__) || defined(SSE3_ENABLED)
|
||||||
|
#define _sse3_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__SSSE3__) || defined(SSSE3_ENABLED)
|
||||||
|
#define _ssse3_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(POPCNT_ENABLED)
|
||||||
|
#define _popcnt_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(__DLL__) || defined(_DLL)
|
||||||
|
#define _dll_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
// 16, 32 or 64
|
||||||
|
#if defined(__sparc_v9__) || defined(_x86_64_) || defined(_ia64_) || defined(_arm64_) || defined(_ppc64_)
|
||||||
|
#define _64_
|
||||||
|
#else
|
||||||
|
#define _32_
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/* All modern 64-bit Unix systems use scheme LP64 (long, pointers are 64-bit).
|
||||||
|
* Microsoft uses a different scheme: LLP64 (long long, pointers are 64-bit).
|
||||||
|
*
|
||||||
|
* Scheme LP64 LLP64
|
||||||
|
* char 8 8
|
||||||
|
* short 16 16
|
||||||
|
* int 32 32
|
||||||
|
* long 64 32
|
||||||
|
* long long 64 64
|
||||||
|
* pointer 64 64
|
||||||
|
*/
|
||||||
|
|
||||||
|
#if defined(_32_)
|
||||||
|
#define SIZEOF_PTR 4
|
||||||
|
#elif defined(_64_)
|
||||||
|
#define SIZEOF_PTR 8
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#define PLATFORM_DATA_ALIGN SIZEOF_PTR
|
||||||
|
|
||||||
|
#if !defined(SIZEOF_PTR)
|
||||||
|
#error todo
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#define SIZEOF_CHAR 1
|
||||||
|
#define SIZEOF_UNSIGNED_CHAR 1
|
||||||
|
#define SIZEOF_SHORT 2
|
||||||
|
#define SIZEOF_UNSIGNED_SHORT 2
|
||||||
|
#define SIZEOF_INT 4
|
||||||
|
#define SIZEOF_UNSIGNED_INT 4
|
||||||
|
|
||||||
|
#if defined(_32_)
|
||||||
|
#define SIZEOF_LONG 4
|
||||||
|
#define SIZEOF_UNSIGNED_LONG 4
|
||||||
|
#elif defined(_64_)
|
||||||
|
#if defined(_win_)
|
||||||
|
#define SIZEOF_LONG 4
|
||||||
|
#define SIZEOF_UNSIGNED_LONG 4
|
||||||
|
#else
|
||||||
|
#define SIZEOF_LONG 8
|
||||||
|
#define SIZEOF_UNSIGNED_LONG 8
|
||||||
|
#endif // _win_
|
||||||
|
#endif // _32_
|
||||||
|
|
||||||
|
#if !defined(SIZEOF_LONG)
|
||||||
|
#error todo
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#define SIZEOF_LONG_LONG 8
|
||||||
|
#define SIZEOF_UNSIGNED_LONG_LONG 8
|
||||||
|
|
||||||
|
#undef SIZEOF_SIZE_T // in case we include <Python.h> which defines it, too
|
||||||
|
#define SIZEOF_SIZE_T SIZEOF_PTR
|
||||||
|
|
||||||
|
#if defined(__INTEL_COMPILER)
|
||||||
|
#pragma warning(disable 1292)
|
||||||
|
#pragma warning(disable 1469)
|
||||||
|
#pragma warning(disable 193)
|
||||||
|
#pragma warning(disable 271)
|
||||||
|
#pragma warning(disable 383)
|
||||||
|
#pragma warning(disable 424)
|
||||||
|
#pragma warning(disable 444)
|
||||||
|
#pragma warning(disable 584)
|
||||||
|
#pragma warning(disable 593)
|
||||||
|
#pragma warning(disable 981)
|
||||||
|
#pragma warning(disable 1418)
|
||||||
|
#pragma warning(disable 304)
|
||||||
|
#pragma warning(disable 810)
|
||||||
|
#pragma warning(disable 1029)
|
||||||
|
#pragma warning(disable 1419)
|
||||||
|
#pragma warning(disable 177)
|
||||||
|
#pragma warning(disable 522)
|
||||||
|
#pragma warning(disable 858)
|
||||||
|
#pragma warning(disable 111)
|
||||||
|
#pragma warning(disable 1599)
|
||||||
|
#pragma warning(disable 411)
|
||||||
|
#pragma warning(disable 304)
|
||||||
|
#pragma warning(disable 858)
|
||||||
|
#pragma warning(disable 444)
|
||||||
|
#pragma warning(disable 913)
|
||||||
|
#pragma warning(disable 310)
|
||||||
|
#pragma warning(disable 167)
|
||||||
|
#pragma warning(disable 180)
|
||||||
|
#pragma warning(disable 1572)
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(_MSC_VER)
|
||||||
|
#undef _WINSOCKAPI_
|
||||||
|
#define _WINSOCKAPI_
|
||||||
|
#undef NOMINMAX
|
||||||
|
#define NOMINMAX
|
||||||
|
#endif
|
||||||
117
contrib/lfalloc/src/util/system/types.h
Normal file
117
contrib/lfalloc/src/util/system/types.h
Normal file
@ -0,0 +1,117 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
// DO_NOT_STYLE
|
||||||
|
|
||||||
|
#include "platform.h"
|
||||||
|
|
||||||
|
#include <inttypes.h>
|
||||||
|
|
||||||
|
typedef int8_t i8;
|
||||||
|
typedef int16_t i16;
|
||||||
|
typedef uint8_t ui8;
|
||||||
|
typedef uint16_t ui16;
|
||||||
|
|
||||||
|
typedef int yssize_t;
|
||||||
|
#define PRIYSZT "d"
|
||||||
|
|
||||||
|
#if defined(_darwin_) && defined(_32_)
|
||||||
|
typedef unsigned long ui32;
|
||||||
|
typedef long i32;
|
||||||
|
#else
|
||||||
|
typedef uint32_t ui32;
|
||||||
|
typedef int32_t i32;
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#if defined(_darwin_) && defined(_64_)
|
||||||
|
typedef unsigned long ui64;
|
||||||
|
typedef long i64;
|
||||||
|
#else
|
||||||
|
typedef uint64_t ui64;
|
||||||
|
typedef int64_t i64;
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#define LL(number) INT64_C(number)
|
||||||
|
#define ULL(number) UINT64_C(number)
|
||||||
|
|
||||||
|
// Macro for size_t and ptrdiff_t types
|
||||||
|
#if defined(_32_)
|
||||||
|
# if defined(_darwin_)
|
||||||
|
# define PRISZT "lu"
|
||||||
|
# undef PRIi32
|
||||||
|
# define PRIi32 "li"
|
||||||
|
# undef SCNi32
|
||||||
|
# define SCNi32 "li"
|
||||||
|
# undef PRId32
|
||||||
|
# define PRId32 "li"
|
||||||
|
# undef SCNd32
|
||||||
|
# define SCNd32 "li"
|
||||||
|
# undef PRIu32
|
||||||
|
# define PRIu32 "lu"
|
||||||
|
# undef SCNu32
|
||||||
|
# define SCNu32 "lu"
|
||||||
|
# undef PRIx32
|
||||||
|
# define PRIx32 "lx"
|
||||||
|
# undef SCNx32
|
||||||
|
# define SCNx32 "lx"
|
||||||
|
# elif !defined(_cygwin_)
|
||||||
|
# define PRISZT PRIu32
|
||||||
|
# else
|
||||||
|
# define PRISZT "u"
|
||||||
|
# endif
|
||||||
|
# define SCNSZT SCNu32
|
||||||
|
# define PRIPDT PRIi32
|
||||||
|
# define SCNPDT SCNi32
|
||||||
|
# define PRITMT PRIi32
|
||||||
|
# define SCNTMT SCNi32
|
||||||
|
#elif defined(_64_)
|
||||||
|
# if defined(_darwin_)
|
||||||
|
# define PRISZT "lu"
|
||||||
|
# undef PRIu64
|
||||||
|
# define PRIu64 PRISZT
|
||||||
|
# undef PRIx64
|
||||||
|
# define PRIx64 "lx"
|
||||||
|
# undef PRIX64
|
||||||
|
# define PRIX64 "lX"
|
||||||
|
# undef PRId64
|
||||||
|
# define PRId64 "ld"
|
||||||
|
# undef PRIi64
|
||||||
|
# define PRIi64 "li"
|
||||||
|
# undef SCNi64
|
||||||
|
# define SCNi64 "li"
|
||||||
|
# undef SCNu64
|
||||||
|
# define SCNu64 "lu"
|
||||||
|
# undef SCNx64
|
||||||
|
# define SCNx64 "lx"
|
||||||
|
# else
|
||||||
|
# define PRISZT PRIu64
|
||||||
|
# endif
|
||||||
|
# define SCNSZT SCNu64
|
||||||
|
# define PRIPDT PRIi64
|
||||||
|
# define SCNPDT SCNi64
|
||||||
|
# define PRITMT PRIi64
|
||||||
|
# define SCNTMT SCNi64
|
||||||
|
#else
|
||||||
|
# error "Unsupported platform"
|
||||||
|
#endif
|
||||||
|
|
||||||
|
// SUPERLONG
|
||||||
|
#if !defined(DONT_USE_SUPERLONG) && !defined(SUPERLONG_MAX)
|
||||||
|
#define SUPERLONG_MAX ~LL(0)
|
||||||
|
typedef i64 SUPERLONG;
|
||||||
|
#endif
|
||||||
|
|
||||||
|
// UNICODE
|
||||||
|
// UCS-2, native byteorder
|
||||||
|
typedef ui16 wchar16;
|
||||||
|
// internal symbol type: UTF-16LE
|
||||||
|
typedef wchar16 TChar;
|
||||||
|
typedef ui32 wchar32;
|
||||||
|
|
||||||
|
#if defined(_MSC_VER)
|
||||||
|
#include <basetsd.h>
|
||||||
|
typedef SSIZE_T ssize_t;
|
||||||
|
#define HAVE_SSIZE_T 1
|
||||||
|
#include <wchar.h>
|
||||||
|
#endif
|
||||||
|
|
||||||
|
#include <sys/types.h>
|
||||||
2
contrib/libhdfs3
vendored
2
contrib/libhdfs3
vendored
@ -1 +1 @@
|
|||||||
Subproject commit bd6505cbb0c130b0db695305b9a38546fa880e5a
|
Subproject commit e2131aa752d7e95441e08f9a18304c1445f2576a
|
||||||
@ -9,7 +9,7 @@ endif()
|
|||||||
|
|
||||||
SET(WITH_KERBEROS false)
|
SET(WITH_KERBEROS false)
|
||||||
# project and source dir
|
# project and source dir
|
||||||
set(HDFS3_ROOT_DIR ${CMAKE_SOURCE_DIR}/contrib/libhdfs3)
|
set(HDFS3_ROOT_DIR ${ClickHouse_SOURCE_DIR}/contrib/libhdfs3)
|
||||||
set(HDFS3_SOURCE_DIR ${HDFS3_ROOT_DIR}/src)
|
set(HDFS3_SOURCE_DIR ${HDFS3_ROOT_DIR}/src)
|
||||||
set(HDFS3_COMMON_DIR ${HDFS3_SOURCE_DIR}/common)
|
set(HDFS3_COMMON_DIR ${HDFS3_SOURCE_DIR}/common)
|
||||||
|
|
||||||
@ -183,7 +183,7 @@ set(SRCS
|
|||||||
)
|
)
|
||||||
|
|
||||||
# target
|
# target
|
||||||
add_library(hdfs3 STATIC ${SRCS} ${PROTO_SOURCES} ${PROTO_HEADERS})
|
add_library(hdfs3 ${SRCS} ${PROTO_SOURCES} ${PROTO_HEADERS})
|
||||||
|
|
||||||
if (USE_INTERNAL_PROTOBUF_LIBRARY)
|
if (USE_INTERNAL_PROTOBUF_LIBRARY)
|
||||||
add_dependencies(hdfs3 protoc)
|
add_dependencies(hdfs3 protoc)
|
||||||
@ -208,7 +208,8 @@ target_link_libraries(hdfs3 ${LIBXML2_LIBRARY})
|
|||||||
# inherit from parent cmake
|
# inherit from parent cmake
|
||||||
target_include_directories(hdfs3 PRIVATE ${Boost_INCLUDE_DIRS})
|
target_include_directories(hdfs3 PRIVATE ${Boost_INCLUDE_DIRS})
|
||||||
target_include_directories(hdfs3 PRIVATE ${Protobuf_INCLUDE_DIR})
|
target_include_directories(hdfs3 PRIVATE ${Protobuf_INCLUDE_DIR})
|
||||||
target_include_directories(hdfs3 PRIVATE ${OPENSSL_INCLUDE_DIR})
|
|
||||||
|
|
||||||
target_link_libraries(hdfs3 ${Protobuf_LIBRARY})
|
target_link_libraries(hdfs3 ${Protobuf_LIBRARY})
|
||||||
|
if(OPENSSL_INCLUDE_DIR AND OPENSSL_LIBRARIES)
|
||||||
|
target_include_directories(hdfs3 PRIVATE ${OPENSSL_INCLUDE_DIR})
|
||||||
target_link_libraries(hdfs3 ${OPENSSL_LIBRARIES})
|
target_link_libraries(hdfs3 ${OPENSSL_LIBRARIES})
|
||||||
|
endif()
|
||||||
|
|||||||
@ -18,6 +18,7 @@
|
|||||||
#define METROHASH_PLATFORM_H
|
#define METROHASH_PLATFORM_H
|
||||||
|
|
||||||
#include <stdint.h>
|
#include <stdint.h>
|
||||||
|
#include <cstring>
|
||||||
|
|
||||||
// rotate right idiom recognized by most compilers
|
// rotate right idiom recognized by most compilers
|
||||||
inline static uint64_t rotate_right(uint64_t v, unsigned k)
|
inline static uint64_t rotate_right(uint64_t v, unsigned k)
|
||||||
@ -25,20 +26,28 @@ inline static uint64_t rotate_right(uint64_t v, unsigned k)
|
|||||||
return (v >> k) | (v << (64 - k));
|
return (v >> k) | (v << (64 - k));
|
||||||
}
|
}
|
||||||
|
|
||||||
// unaligned reads, fast and safe on Nehalem and later microarchitectures
|
|
||||||
inline static uint64_t read_u64(const void * const ptr)
|
inline static uint64_t read_u64(const void * const ptr)
|
||||||
{
|
{
|
||||||
return static_cast<uint64_t>(*reinterpret_cast<const uint64_t*>(ptr));
|
uint64_t result;
|
||||||
|
// Assignment like `result = *reinterpret_cast<const uint64_t *>(ptr)` here would mean undefined behaviour (unaligned read),
|
||||||
|
// so we use memcpy() which is the most portable. clang & gcc usually translates `memcpy()` into a single `load` instruction
|
||||||
|
// when hardware supports it, so using memcpy() is efficient too.
|
||||||
|
memcpy(&result, ptr, sizeof(result));
|
||||||
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
inline static uint64_t read_u32(const void * const ptr)
|
inline static uint64_t read_u32(const void * const ptr)
|
||||||
{
|
{
|
||||||
return static_cast<uint64_t>(*reinterpret_cast<const uint32_t*>(ptr));
|
uint32_t result;
|
||||||
|
memcpy(&result, ptr, sizeof(result));
|
||||||
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
inline static uint64_t read_u16(const void * const ptr)
|
inline static uint64_t read_u16(const void * const ptr)
|
||||||
{
|
{
|
||||||
return static_cast<uint64_t>(*reinterpret_cast<const uint16_t*>(ptr));
|
uint16_t result;
|
||||||
|
memcpy(&result, ptr, sizeof(result));
|
||||||
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
inline static uint64_t read_u8 (const void * const ptr)
|
inline static uint64_t read_u8 (const void * const ptr)
|
||||||
|
|||||||
@ -81,7 +81,7 @@
|
|||||||
#define PCG_128BIT_CONSTANT(high,low) \
|
#define PCG_128BIT_CONSTANT(high,low) \
|
||||||
((pcg128_t(high) << 64) + low)
|
((pcg128_t(high) << 64) + low)
|
||||||
#else
|
#else
|
||||||
#include "pcg_uint128.hpp" // Y_IGNORE
|
#include "pcg_uint128.hpp"
|
||||||
namespace pcg_extras {
|
namespace pcg_extras {
|
||||||
typedef pcg_extras::uint_x4<uint32_t,uint64_t> pcg128_t;
|
typedef pcg_extras::uint_x4<uint32_t,uint64_t> pcg128_t;
|
||||||
}
|
}
|
||||||
|
|||||||
2
contrib/librdkafka
vendored
2
contrib/librdkafka
vendored
@ -1 +1 @@
|
|||||||
Subproject commit 363dcad5a23dc29381cc626620e68ae418b3af19
|
Subproject commit 8695b9d63ac0fe1b891b511d5b36302ffc84d4e2
|
||||||
@ -1,4 +1,4 @@
|
|||||||
set(RDKAFKA_SOURCE_DIR ${CMAKE_SOURCE_DIR}/contrib/librdkafka/src)
|
set(RDKAFKA_SOURCE_DIR ${ClickHouse_SOURCE_DIR}/contrib/librdkafka/src)
|
||||||
|
|
||||||
set(SRCS
|
set(SRCS
|
||||||
${RDKAFKA_SOURCE_DIR}/crc32c.c
|
${RDKAFKA_SOURCE_DIR}/crc32c.c
|
||||||
@ -33,6 +33,7 @@ set(SRCS
|
|||||||
${RDKAFKA_SOURCE_DIR}/rdkafka_roundrobin_assignor.c
|
${RDKAFKA_SOURCE_DIR}/rdkafka_roundrobin_assignor.c
|
||||||
${RDKAFKA_SOURCE_DIR}/rdkafka_sasl.c
|
${RDKAFKA_SOURCE_DIR}/rdkafka_sasl.c
|
||||||
${RDKAFKA_SOURCE_DIR}/rdkafka_sasl_plain.c
|
${RDKAFKA_SOURCE_DIR}/rdkafka_sasl_plain.c
|
||||||
|
${RDKAFKA_SOURCE_DIR}/rdkafka_sasl_scram.c
|
||||||
${RDKAFKA_SOURCE_DIR}/rdkafka_subscription.c
|
${RDKAFKA_SOURCE_DIR}/rdkafka_subscription.c
|
||||||
${RDKAFKA_SOURCE_DIR}/rdkafka_timer.c
|
${RDKAFKA_SOURCE_DIR}/rdkafka_timer.c
|
||||||
${RDKAFKA_SOURCE_DIR}/rdkafka_topic.c
|
${RDKAFKA_SOURCE_DIR}/rdkafka_topic.c
|
||||||
@ -54,8 +55,11 @@ set(SRCS
|
|||||||
${RDKAFKA_SOURCE_DIR}/rdgz.c
|
${RDKAFKA_SOURCE_DIR}/rdgz.c
|
||||||
)
|
)
|
||||||
|
|
||||||
add_library(rdkafka ${LINK_MODE} ${SRCS})
|
add_library(rdkafka ${SRCS})
|
||||||
target_include_directories(rdkafka SYSTEM PUBLIC include)
|
target_include_directories(rdkafka SYSTEM PUBLIC include)
|
||||||
target_include_directories(rdkafka SYSTEM PUBLIC ${RDKAFKA_SOURCE_DIR}) # Because weird logic with "include_next" is used.
|
target_include_directories(rdkafka SYSTEM PUBLIC ${RDKAFKA_SOURCE_DIR}) # Because weird logic with "include_next" is used.
|
||||||
target_include_directories(rdkafka SYSTEM PRIVATE ${ZSTD_INCLUDE_DIR}/common) # Because wrong path to "zstd_errors.h" is used.
|
target_include_directories(rdkafka SYSTEM PRIVATE ${ZSTD_INCLUDE_DIR}/common) # Because wrong path to "zstd_errors.h" is used.
|
||||||
target_link_libraries(rdkafka PUBLIC ${ZLIB_LIBRARIES} ${ZSTD_LIBRARY} ${LZ4_LIBRARY} ${OPENSSL_SSL_LIBRARY} ${OPENSSL_CRYPTO_LIBRARY})
|
target_link_libraries(rdkafka PUBLIC ${ZLIB_LIBRARIES} ${ZSTD_LIBRARY} ${LZ4_LIBRARY} ${LIBGSASL_LIBRARY})
|
||||||
|
if(OPENSSL_SSL_LIBRARY AND OPENSSL_CRYPTO_LIBRARY)
|
||||||
|
target_link_libraries(rdkafka PUBLIC ${OPENSSL_SSL_LIBRARY} ${OPENSSL_CRYPTO_LIBRARY})
|
||||||
|
endif()
|
||||||
|
|||||||
@ -2,6 +2,8 @@
|
|||||||
#ifndef _CONFIG_H_
|
#ifndef _CONFIG_H_
|
||||||
#define _CONFIG_H_
|
#define _CONFIG_H_
|
||||||
#define ARCH "x86_64"
|
#define ARCH "x86_64"
|
||||||
|
#define BUILT_WITH "GCC GXX PKGCONFIG OSXLD LIBDL PLUGINS ZLIB SSL SASL_CYRUS ZSTD HDRHISTOGRAM LZ4_EXT SNAPPY SOCKEM SASL_SCRAM CRC32C_HW"
|
||||||
|
|
||||||
#define CPU "generic"
|
#define CPU "generic"
|
||||||
#define WITHOUT_OPTIMIZATION 0
|
#define WITHOUT_OPTIMIZATION 0
|
||||||
#define ENABLE_DEVEL 0
|
#define ENABLE_DEVEL 0
|
||||||
@ -10,7 +12,7 @@
|
|||||||
#define ENABLE_SHAREDPTR_DEBUG 0
|
#define ENABLE_SHAREDPTR_DEBUG 0
|
||||||
#define ENABLE_LZ4_EXT 1
|
#define ENABLE_LZ4_EXT 1
|
||||||
#define ENABLE_SSL 1
|
#define ENABLE_SSL 1
|
||||||
//#define ENABLE_SASL 1
|
#define ENABLE_SASL 1
|
||||||
#define MKL_APP_NAME "librdkafka"
|
#define MKL_APP_NAME "librdkafka"
|
||||||
#define MKL_APP_DESC_ONELINE "The Apache Kafka C/C++ library"
|
#define MKL_APP_DESC_ONELINE "The Apache Kafka C/C++ library"
|
||||||
// distro
|
// distro
|
||||||
@ -60,7 +62,7 @@
|
|||||||
// libssl
|
// libssl
|
||||||
#define WITH_SSL 1
|
#define WITH_SSL 1
|
||||||
// WITH_SASL_SCRAM
|
// WITH_SASL_SCRAM
|
||||||
//#define WITH_SASL_SCRAM 1
|
#define WITH_SASL_SCRAM 1
|
||||||
// crc32chw
|
// crc32chw
|
||||||
#if !defined(__PPC__)
|
#if !defined(__PPC__)
|
||||||
#define WITH_CRC32C_HW 1
|
#define WITH_CRC32C_HW 1
|
||||||
|
|||||||
@ -1,5 +1,5 @@
|
|||||||
#if __has_include(<rdkafka.h>) // maybe bundled
|
#if __has_include(<rdkafka.h>) // maybe bundled
|
||||||
# include_next <rdkafka.h> // Y_IGNORE
|
# include_next <rdkafka.h>
|
||||||
#else // system
|
#else // system
|
||||||
# include_next <librdkafka/rdkafka.h>
|
# include_next <librdkafka/rdkafka.h>
|
||||||
#endif
|
#endif
|
||||||
|
|||||||
@ -1,62 +1,56 @@
|
|||||||
set(LIBXML2_SOURCE_DIR ${CMAKE_SOURCE_DIR}/contrib/libxml2)
|
set(LIBXML2_SOURCE_DIR ${ClickHouse_SOURCE_DIR}/contrib/libxml2)
|
||||||
set(LIBXML2_BINARY_DIR ${CMAKE_BINARY_DIR}/contrib/libxml2)
|
set(LIBXML2_BINARY_DIR ${ClickHouse_BINARY_DIR}/contrib/libxml2)
|
||||||
|
|
||||||
|
|
||||||
set(SRCS
|
set(SRCS
|
||||||
${LIBXML2_SOURCE_DIR}/parser.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/HTMLparser.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/buf.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xzlib.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xmlregexp.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/entities.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/rngparser.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/encoding.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/legacy.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/error.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/debugXML.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xpointer.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/DOCBparser.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xmlcatalog.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/c14n.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xmlreader.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xmlstring.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/dict.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xpath.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/tree.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/trionan.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/pattern.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/globals.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xmllint.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/chvalid.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/relaxng.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/list.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xinclude.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xmlIO.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/triostr.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/hash.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xmlsave.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/HTMLtree.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/SAX.c
|
${LIBXML2_SOURCE_DIR}/SAX.c
|
||||||
${LIBXML2_SOURCE_DIR}/xmlschemas.c
|
${LIBXML2_SOURCE_DIR}/entities.c
|
||||||
${LIBXML2_SOURCE_DIR}/SAX2.c
|
${LIBXML2_SOURCE_DIR}/encoding.c
|
||||||
${LIBXML2_SOURCE_DIR}/threads.c
|
${LIBXML2_SOURCE_DIR}/error.c
|
||||||
${LIBXML2_SOURCE_DIR}/runsuite.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/catalog.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/uri.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xmlmodule.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/xlink.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/parserInternals.c
|
${LIBXML2_SOURCE_DIR}/parserInternals.c
|
||||||
${LIBXML2_SOURCE_DIR}/xmlwriter.c
|
${LIBXML2_SOURCE_DIR}/parser.c
|
||||||
${LIBXML2_SOURCE_DIR}/xmlunicode.c
|
${LIBXML2_SOURCE_DIR}/tree.c
|
||||||
${LIBXML2_SOURCE_DIR}/runxmlconf.c
|
${LIBXML2_SOURCE_DIR}/hash.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/list.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xmlIO.c
|
||||||
${LIBXML2_SOURCE_DIR}/xmlmemory.c
|
${LIBXML2_SOURCE_DIR}/xmlmemory.c
|
||||||
${LIBXML2_SOURCE_DIR}/nanoftp.c
|
${LIBXML2_SOURCE_DIR}/uri.c
|
||||||
${LIBXML2_SOURCE_DIR}/xmlschemastypes.c
|
|
||||||
${LIBXML2_SOURCE_DIR}/valid.c
|
${LIBXML2_SOURCE_DIR}/valid.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xlink.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/HTMLparser.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/HTMLtree.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/debugXML.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xpath.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xpointer.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xinclude.c
|
||||||
${LIBXML2_SOURCE_DIR}/nanohttp.c
|
${LIBXML2_SOURCE_DIR}/nanohttp.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/nanoftp.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/DOCBparser.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/catalog.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/globals.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/threads.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/c14n.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xmlstring.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/buf.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xmlregexp.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xmlschemas.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xmlschemastypes.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xmlunicode.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/triostr.c
|
||||||
|
#${LIBXML2_SOURCE_DIR}/trio.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xmlreader.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/relaxng.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/dict.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/SAX2.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xmlwriter.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/legacy.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/chvalid.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/pattern.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xmlsave.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xmlmodule.c
|
||||||
${LIBXML2_SOURCE_DIR}/schematron.c
|
${LIBXML2_SOURCE_DIR}/schematron.c
|
||||||
|
${LIBXML2_SOURCE_DIR}/xzlib.c
|
||||||
)
|
)
|
||||||
add_library(libxml2 STATIC ${SRCS})
|
add_library(libxml2 ${SRCS})
|
||||||
|
|
||||||
target_link_libraries(libxml2 ${ZLIB_LIBRARIES})
|
target_link_libraries(libxml2 ${ZLIB_LIBRARIES})
|
||||||
|
|
||||||
|
|||||||
2
contrib/lz4
vendored
2
contrib/lz4
vendored
@ -1 +1 @@
|
|||||||
Subproject commit c10863b98e1503af90616ae99725ecd120265dfb
|
Subproject commit 7a4e3b1fac5cd9d4ec7c8d0091329ba107ec2131
|
||||||
@ -9,8 +9,7 @@ add_library (lz4
|
|||||||
${LIBRARY_DIR}/xxhash.h
|
${LIBRARY_DIR}/xxhash.h
|
||||||
|
|
||||||
${LIBRARY_DIR}/lz4.h
|
${LIBRARY_DIR}/lz4.h
|
||||||
${LIBRARY_DIR}/lz4hc.h
|
${LIBRARY_DIR}/lz4hc.h)
|
||||||
${LIBRARY_DIR}/lz4opt.h)
|
|
||||||
|
|
||||||
target_compile_definitions(lz4 PUBLIC LZ4_DISABLE_DEPRECATE_WARNINGS=1)
|
target_compile_definitions(lz4 PUBLIC LZ4_DISABLE_DEPRECATE_WARNINGS=1)
|
||||||
|
|
||||||
|
|||||||
@ -1,8 +1,8 @@
|
|||||||
set(MARIADB_CLIENT_SOURCE_DIR ${CMAKE_SOURCE_DIR}/contrib/mariadb-connector-c)
|
set(MARIADB_CLIENT_SOURCE_DIR ${ClickHouse_SOURCE_DIR}/contrib/mariadb-connector-c)
|
||||||
set(MARIADB_CLIENT_BINARY_DIR ${CMAKE_BINARY_DIR}/contrib/mariadb-connector-c)
|
set(MARIADB_CLIENT_BINARY_DIR ${ClickHouse_BINARY_DIR}/contrib/mariadb-connector-c)
|
||||||
|
|
||||||
set(SRCS
|
set(SRCS
|
||||||
${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/bmove_upp.c
|
#${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/bmove_upp.c
|
||||||
${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/get_password.c
|
${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/get_password.c
|
||||||
${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/ma_alloc.c
|
${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/ma_alloc.c
|
||||||
${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/ma_array.c
|
${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/ma_array.c
|
||||||
@ -33,7 +33,6 @@ ${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/ma_time.c
|
|||||||
${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/ma_tls.c
|
${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/ma_tls.c
|
||||||
#${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/secure/gnutls.c
|
#${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/secure/gnutls.c
|
||||||
#${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/secure/ma_schannel.c
|
#${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/secure/ma_schannel.c
|
||||||
${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/secure/openssl.c
|
|
||||||
#${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/secure/schannel.c
|
#${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/secure/schannel.c
|
||||||
#${MARIADB_CLIENT_SOURCE_DIR}/plugins/auth/auth_gssapi_client.c
|
#${MARIADB_CLIENT_SOURCE_DIR}/plugins/auth/auth_gssapi_client.c
|
||||||
#${MARIADB_CLIENT_SOURCE_DIR}/plugins/auth/dialog.c
|
#${MARIADB_CLIENT_SOURCE_DIR}/plugins/auth/dialog.c
|
||||||
@ -55,12 +54,19 @@ ${MARIADB_CLIENT_SOURCE_DIR}/plugins/pvio/pvio_socket.c
|
|||||||
${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64/libmariadb/ma_client_plugin.c
|
${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64/libmariadb/ma_client_plugin.c
|
||||||
)
|
)
|
||||||
|
|
||||||
add_library(mysqlclient STATIC ${SRCS})
|
if(OPENSSL_LIBRARIES)
|
||||||
|
list(APPEND SRCS ${MARIADB_CLIENT_SOURCE_DIR}/libmariadb/secure/openssl.c)
|
||||||
|
endif()
|
||||||
|
|
||||||
target_link_libraries(mysqlclient ${OPENSSL_LIBRARIES})
|
add_library(mysqlclient ${SRCS})
|
||||||
|
|
||||||
|
if(OPENSSL_LIBRARIES)
|
||||||
|
target_link_libraries(mysqlclient PRIVATE ${OPENSSL_LIBRARIES})
|
||||||
|
target_compile_definitions(mysqlclient PRIVATE -D HAVE_OPENSSL -D HAVE_TLS)
|
||||||
|
endif()
|
||||||
|
|
||||||
target_include_directories(mysqlclient PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64/include)
|
target_include_directories(mysqlclient PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64/include)
|
||||||
target_include_directories(mysqlclient PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/common/include)
|
target_include_directories(mysqlclient PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/common/include)
|
||||||
target_include_directories(mysqlclient PUBLIC ${MARIADB_CLIENT_SOURCE_DIR}/include)
|
target_include_directories(mysqlclient PUBLIC ${MARIADB_CLIENT_SOURCE_DIR}/include)
|
||||||
|
|
||||||
target_compile_definitions(mysqlclient PRIVATE -D THREAD -D HAVE_OPENSSL -D HAVE_TLS)
|
target_compile_definitions(mysqlclient PRIVATE -D THREAD)
|
||||||
|
|||||||
2
contrib/poco
vendored
2
contrib/poco
vendored
@ -1 +1 @@
|
|||||||
Subproject commit fe5505e56c27b6ecb0dcbc40c49dc2caf4e9637f
|
Subproject commit 29439cf7fa32c1a2d62d925bb6d6a3f14668a4a2
|
||||||
1
contrib/rapidjson
vendored
Submodule
1
contrib/rapidjson
vendored
Submodule
@ -0,0 +1 @@
|
|||||||
|
Subproject commit 01950eb7acec78818d68b762efc869bba2420d82
|
||||||
1
contrib/simdjson
vendored
Submodule
1
contrib/simdjson
vendored
Submodule
@ -0,0 +1 @@
|
|||||||
|
Subproject commit 2151ad7f34cf773a23f086e941d661f8a8873144
|
||||||
18
contrib/simdjson-cmake/CMakeLists.txt
Normal file
18
contrib/simdjson-cmake/CMakeLists.txt
Normal file
@ -0,0 +1,18 @@
|
|||||||
|
if (NOT HAVE_AVX2)
|
||||||
|
message (FATAL_ERROR "No AVX2 support")
|
||||||
|
endif ()
|
||||||
|
set(SIMDJSON_INCLUDE_DIR "${ClickHouse_SOURCE_DIR}/contrib/simdjson/include")
|
||||||
|
set(SIMDJSON_SRC_DIR "${SIMDJSON_INCLUDE_DIR}/../src")
|
||||||
|
set(SIMDJSON_SRC
|
||||||
|
${SIMDJSON_SRC_DIR}/jsonioutil.cpp
|
||||||
|
${SIMDJSON_SRC_DIR}/jsonminifier.cpp
|
||||||
|
${SIMDJSON_SRC_DIR}/jsonparser.cpp
|
||||||
|
${SIMDJSON_SRC_DIR}/stage1_find_marks.cpp
|
||||||
|
${SIMDJSON_SRC_DIR}/stage2_build_tape.cpp
|
||||||
|
${SIMDJSON_SRC_DIR}/parsedjson.cpp
|
||||||
|
${SIMDJSON_SRC_DIR}/parsedjsoniterator.cpp
|
||||||
|
)
|
||||||
|
|
||||||
|
add_library(${SIMDJSON_LIBRARY} ${SIMDJSON_SRC})
|
||||||
|
target_include_directories(${SIMDJSON_LIBRARY} PUBLIC "${SIMDJSON_INCLUDE_DIR}")
|
||||||
|
target_compile_options(${SIMDJSON_LIBRARY} PRIVATE -mavx2 -mbmi -mbmi2 -mpclmul)
|
||||||
@ -1,5 +1,5 @@
|
|||||||
set(ODBC_SOURCE_DIR ${CMAKE_SOURCE_DIR}/contrib/unixodbc)
|
set(ODBC_SOURCE_DIR ${ClickHouse_SOURCE_DIR}/contrib/unixodbc)
|
||||||
set(ODBC_BINARY_DIR ${CMAKE_BINARY_DIR}/contrib/unixodbc)
|
set(ODBC_BINARY_DIR ${ClickHouse_BINARY_DIR}/contrib/unixodbc)
|
||||||
|
|
||||||
|
|
||||||
set(SRCS
|
set(SRCS
|
||||||
@ -23,7 +23,7 @@ ${ODBC_SOURCE_DIR}/libltdl/loaders/preopen.c
|
|||||||
${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64/libltdl/libltdlcS.c
|
${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64/libltdl/libltdlcS.c
|
||||||
)
|
)
|
||||||
|
|
||||||
add_library(ltdl ${LINK_MODE} ${SRCS})
|
add_library(ltdl ${SRCS})
|
||||||
|
|
||||||
target_include_directories(ltdl PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64/libltdl)
|
target_include_directories(ltdl PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64/libltdl)
|
||||||
target_include_directories(ltdl PUBLIC ${ODBC_SOURCE_DIR}/libltdl)
|
target_include_directories(ltdl PUBLIC ${ODBC_SOURCE_DIR}/libltdl)
|
||||||
@ -273,15 +273,15 @@ ${ODBC_SOURCE_DIR}/lst/lstSetFreeFunc.c
|
|||||||
${ODBC_SOURCE_DIR}/lst/_lstVisible.c
|
${ODBC_SOURCE_DIR}/lst/_lstVisible.c
|
||||||
)
|
)
|
||||||
|
|
||||||
add_library(unixodbc ${LINK_MODE} ${SRCS})
|
add_library(unixodbc ${SRCS})
|
||||||
|
|
||||||
target_link_libraries(unixodbc ltdl)
|
target_link_libraries(unixodbc PRIVATE ltdl)
|
||||||
|
|
||||||
# SYSTEM_FILE_PATH was changed to /etc
|
# SYSTEM_FILE_PATH was changed to /etc
|
||||||
|
|
||||||
target_include_directories(unixodbc PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64/private)
|
target_include_directories(unixodbc SYSTEM PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64/private)
|
||||||
target_include_directories(unixodbc PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64)
|
target_include_directories(unixodbc SYSTEM PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/linux_x86_64)
|
||||||
target_include_directories(unixodbc PUBLIC ${ODBC_SOURCE_DIR}/include)
|
target_include_directories(unixodbc SYSTEM PUBLIC ${ODBC_SOURCE_DIR}/include)
|
||||||
|
|
||||||
target_compile_definitions(unixodbc PRIVATE -DHAVE_CONFIG_H)
|
target_compile_definitions(unixodbc PRIVATE -DHAVE_CONFIG_H)
|
||||||
|
|
||||||
|
|||||||
@ -125,6 +125,6 @@ IF (ZSTD_LEGACY_SUPPORT)
|
|||||||
${LIBRARY_LEGACY_DIR}/zstd_v07.h)
|
${LIBRARY_LEGACY_DIR}/zstd_v07.h)
|
||||||
ENDIF (ZSTD_LEGACY_SUPPORT)
|
ENDIF (ZSTD_LEGACY_SUPPORT)
|
||||||
|
|
||||||
ADD_LIBRARY(zstd ${LINK_MODE} ${Sources} ${Headers})
|
ADD_LIBRARY(zstd ${Sources} ${Headers})
|
||||||
|
|
||||||
target_include_directories (zstd PUBLIC ${LIBRARY_DIR})
|
target_include_directories (zstd PUBLIC ${LIBRARY_DIR})
|
||||||
|
|||||||
@ -20,7 +20,7 @@ set (CONFIG_VERSION ${CMAKE_CURRENT_BINARY_DIR}/src/Common/config_version.h)
|
|||||||
set (CONFIG_COMMON ${CMAKE_CURRENT_BINARY_DIR}/src/Common/config.h)
|
set (CONFIG_COMMON ${CMAKE_CURRENT_BINARY_DIR}/src/Common/config.h)
|
||||||
|
|
||||||
include (cmake/version.cmake)
|
include (cmake/version.cmake)
|
||||||
message (STATUS "Will build ${VERSION_FULL} revision ${VERSION_REVISION}")
|
message (STATUS "Will build ${VERSION_FULL} revision ${VERSION_REVISION} ${VERSION_OFFICIAL}")
|
||||||
configure_file (src/Common/config.h.in ${CONFIG_COMMON})
|
configure_file (src/Common/config.h.in ${CONFIG_COMMON})
|
||||||
configure_file (src/Common/config_version.h.in ${CONFIG_VERSION})
|
configure_file (src/Common/config_version.h.in ${CONFIG_VERSION})
|
||||||
|
|
||||||
@ -49,19 +49,20 @@ if (CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
|
|||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 7)
|
if (NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 7)
|
||||||
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
|
|
||||||
|
|
||||||
if (WEVERYTHING)
|
if (WEVERYTHING)
|
||||||
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-return-std-move-in-c++11")
|
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-return-std-move-in-c++11")
|
||||||
endif ()
|
endif ()
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 8)
|
if (NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 8)
|
||||||
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wextra-semi-stmt -Wshadow-field -Wstring-plus-int")
|
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wshadow-field -Wstring-plus-int")
|
||||||
|
if(NOT APPLE)
|
||||||
|
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wextra-semi-stmt -Wempty-init-stmt")
|
||||||
|
endif()
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9)
|
if (NOT CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9)
|
||||||
if (WEVERYTHING)
|
if (WEVERYTHING AND NOT APPLE)
|
||||||
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-ctad-maybe-unsupported")
|
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-ctad-maybe-unsupported")
|
||||||
endif ()
|
endif ()
|
||||||
endif ()
|
endif ()
|
||||||
@ -134,7 +135,7 @@ list (APPEND dbms_headers src/TableFunctions/ITableFunction.h src/TableFunctio
|
|||||||
list (APPEND dbms_sources src/Dictionaries/DictionaryFactory.cpp src/Dictionaries/DictionarySourceFactory.cpp src/Dictionaries/DictionaryStructure.cpp)
|
list (APPEND dbms_sources src/Dictionaries/DictionaryFactory.cpp src/Dictionaries/DictionarySourceFactory.cpp src/Dictionaries/DictionaryStructure.cpp)
|
||||||
list (APPEND dbms_headers src/Dictionaries/DictionaryFactory.h src/Dictionaries/DictionarySourceFactory.h src/Dictionaries/DictionaryStructure.h)
|
list (APPEND dbms_headers src/Dictionaries/DictionaryFactory.h src/Dictionaries/DictionarySourceFactory.h src/Dictionaries/DictionaryStructure.h)
|
||||||
|
|
||||||
add_library(clickhouse_common_io ${LINK_MODE} ${clickhouse_common_io_headers} ${clickhouse_common_io_sources})
|
add_library(clickhouse_common_io ${clickhouse_common_io_headers} ${clickhouse_common_io_sources})
|
||||||
|
|
||||||
if (OS_FREEBSD)
|
if (OS_FREEBSD)
|
||||||
target_compile_definitions (clickhouse_common_io PUBLIC CLOCK_MONOTONIC_COARSE=CLOCK_MONOTONIC_FAST)
|
target_compile_definitions (clickhouse_common_io PUBLIC CLOCK_MONOTONIC_COARSE=CLOCK_MONOTONIC_FAST)
|
||||||
@ -155,7 +156,6 @@ if (USE_EMBEDDED_COMPILER)
|
|||||||
target_include_directories (dbms SYSTEM BEFORE PUBLIC ${LLVM_INCLUDE_DIRS})
|
target_include_directories (dbms SYSTEM BEFORE PUBLIC ${LLVM_INCLUDE_DIRS})
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
|
|
||||||
if (CMAKE_BUILD_TYPE_UC STREQUAL "RELEASE" OR CMAKE_BUILD_TYPE_UC STREQUAL "RELWITHDEBINFO" OR CMAKE_BUILD_TYPE_UC STREQUAL "MINSIZEREL")
|
if (CMAKE_BUILD_TYPE_UC STREQUAL "RELEASE" OR CMAKE_BUILD_TYPE_UC STREQUAL "RELWITHDEBINFO" OR CMAKE_BUILD_TYPE_UC STREQUAL "MINSIZEREL")
|
||||||
# Won't generate debug info for files with heavy template instantiation to achieve faster linking and lower size.
|
# Won't generate debug info for files with heavy template instantiation to achieve faster linking and lower size.
|
||||||
set_source_files_properties(
|
set_source_files_properties(
|
||||||
@ -184,19 +184,27 @@ target_link_libraries (clickhouse_common_io
|
|||||||
string_utils
|
string_utils
|
||||||
widechar_width
|
widechar_width
|
||||||
${LINK_LIBRARIES_ONLY_ON_X86_64}
|
${LINK_LIBRARIES_ONLY_ON_X86_64}
|
||||||
|
PUBLIC
|
||||||
${DOUBLE_CONVERSION_LIBRARIES}
|
${DOUBLE_CONVERSION_LIBRARIES}
|
||||||
pocoext
|
|
||||||
PUBLIC
|
PUBLIC
|
||||||
${Poco_Net_LIBRARY}
|
${Poco_Net_LIBRARY}
|
||||||
${Poco_Util_LIBRARY}
|
${Poco_Util_LIBRARY}
|
||||||
${Poco_Foundation_LIBRARY}
|
${Poco_Foundation_LIBRARY}
|
||||||
${RE2_LIBRARY}
|
)
|
||||||
${RE2_ST_LIBRARY}
|
|
||||||
|
if(RE2_LIBRARY)
|
||||||
|
target_link_libraries(clickhouse_common_io PUBLIC ${RE2_LIBRARY})
|
||||||
|
endif()
|
||||||
|
if(RE2_ST_LIBRARY)
|
||||||
|
target_link_libraries(clickhouse_common_io PUBLIC ${RE2_ST_LIBRARY})
|
||||||
|
endif()
|
||||||
|
|
||||||
|
target_link_libraries(clickhouse_common_io
|
||||||
|
PUBLIC
|
||||||
${CITYHASH_LIBRARIES}
|
${CITYHASH_LIBRARIES}
|
||||||
PRIVATE
|
PRIVATE
|
||||||
${ZLIB_LIBRARIES}
|
${ZLIB_LIBRARIES}
|
||||||
${EXECINFO_LIBRARY}
|
${EXECINFO_LIBRARIES}
|
||||||
${ELF_LIBRARY}
|
|
||||||
PUBLIC
|
PUBLIC
|
||||||
${Boost_SYSTEM_LIBRARY}
|
${Boost_SYSTEM_LIBRARY}
|
||||||
PRIVATE
|
PRIVATE
|
||||||
@ -210,7 +218,13 @@ target_link_libraries (clickhouse_common_io
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if(RE2_INCLUDE_DIR)
|
||||||
target_include_directories(clickhouse_common_io SYSTEM BEFORE PUBLIC ${RE2_INCLUDE_DIR})
|
target_include_directories(clickhouse_common_io SYSTEM BEFORE PUBLIC ${RE2_INCLUDE_DIR})
|
||||||
|
endif()
|
||||||
|
|
||||||
|
if (USE_LFALLOC)
|
||||||
|
target_include_directories (clickhouse_common_io SYSTEM BEFORE PUBLIC ${LFALLOC_INCLUDE_DIR})
|
||||||
|
endif ()
|
||||||
|
|
||||||
if(CPUID_LIBRARY)
|
if(CPUID_LIBRARY)
|
||||||
target_link_libraries(clickhouse_common_io PRIVATE ${CPUID_LIBRARY})
|
target_link_libraries(clickhouse_common_io PRIVATE ${CPUID_LIBRARY})
|
||||||
@ -221,8 +235,9 @@ if(CPUINFO_LIBRARY)
|
|||||||
endif()
|
endif()
|
||||||
|
|
||||||
target_link_libraries (dbms
|
target_link_libraries (dbms
|
||||||
PRIVATE
|
PUBLIC
|
||||||
clickhouse_compression
|
clickhouse_compression
|
||||||
|
PRIVATE
|
||||||
clickhouse_parsers
|
clickhouse_parsers
|
||||||
clickhouse_common_config
|
clickhouse_common_config
|
||||||
PUBLIC
|
PUBLIC
|
||||||
@ -230,7 +245,6 @@ target_link_libraries (dbms
|
|||||||
PRIVATE
|
PRIVATE
|
||||||
clickhouse_dictionaries_embedded
|
clickhouse_dictionaries_embedded
|
||||||
PUBLIC
|
PUBLIC
|
||||||
pocoext
|
|
||||||
${MYSQLXX_LIBRARY}
|
${MYSQLXX_LIBRARY}
|
||||||
PRIVATE
|
PRIVATE
|
||||||
${BTRIE_LIBRARIES}
|
${BTRIE_LIBRARIES}
|
||||||
@ -255,8 +269,8 @@ if (USE_POCO_SQLODBC)
|
|||||||
target_link_libraries (clickhouse_common_io PRIVATE ${Poco_SQL_LIBRARY})
|
target_link_libraries (clickhouse_common_io PRIVATE ${Poco_SQL_LIBRARY})
|
||||||
target_link_libraries (dbms PRIVATE ${Poco_SQLODBC_LIBRARY} ${Poco_SQL_LIBRARY})
|
target_link_libraries (dbms PRIVATE ${Poco_SQLODBC_LIBRARY} ${Poco_SQL_LIBRARY})
|
||||||
if (NOT USE_INTERNAL_POCO_LIBRARY)
|
if (NOT USE_INTERNAL_POCO_LIBRARY)
|
||||||
target_include_directories (clickhouse_common_io SYSTEM PRIVATE ${ODBC_INCLUDE_DIRECTORIES} ${Poco_SQL_INCLUDE_DIR})
|
target_include_directories (clickhouse_common_io SYSTEM PRIVATE ${ODBC_INCLUDE_DIRS} ${Poco_SQL_INCLUDE_DIR})
|
||||||
target_include_directories (dbms SYSTEM PRIVATE ${ODBC_INCLUDE_DIRECTORIES} ${Poco_SQLODBC_INCLUDE_DIR} PUBLIC ${Poco_SQL_INCLUDE_DIR})
|
target_include_directories (dbms SYSTEM PRIVATE ${ODBC_INCLUDE_DIRS} ${Poco_SQLODBC_INCLUDE_DIR} SYSTEM PUBLIC ${Poco_SQL_INCLUDE_DIR})
|
||||||
endif()
|
endif()
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
@ -269,7 +283,7 @@ if (USE_POCO_DATAODBC)
|
|||||||
target_link_libraries (clickhouse_common_io PRIVATE ${Poco_Data_LIBRARY})
|
target_link_libraries (clickhouse_common_io PRIVATE ${Poco_Data_LIBRARY})
|
||||||
target_link_libraries (dbms PRIVATE ${Poco_DataODBC_LIBRARY})
|
target_link_libraries (dbms PRIVATE ${Poco_DataODBC_LIBRARY})
|
||||||
if (NOT USE_INTERNAL_POCO_LIBRARY)
|
if (NOT USE_INTERNAL_POCO_LIBRARY)
|
||||||
target_include_directories (dbms SYSTEM PRIVATE ${ODBC_INCLUDE_DIRECTORIES} ${Poco_DataODBC_INCLUDE_DIR})
|
target_include_directories (dbms SYSTEM PRIVATE ${ODBC_INCLUDE_DIRS} ${Poco_DataODBC_INCLUDE_DIR})
|
||||||
endif()
|
endif()
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
@ -307,7 +321,10 @@ if (USE_PARQUET)
|
|||||||
endif ()
|
endif ()
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
target_link_libraries(dbms PRIVATE ${OPENSSL_CRYPTO_LIBRARY} Threads::Threads)
|
if(OPENSSL_CRYPTO_LIBRARY)
|
||||||
|
target_link_libraries(dbms PRIVATE ${OPENSSL_CRYPTO_LIBRARY})
|
||||||
|
endif ()
|
||||||
|
target_link_libraries(dbms PRIVATE Threads::Threads)
|
||||||
|
|
||||||
target_include_directories (dbms SYSTEM BEFORE PRIVATE ${DIVIDE_INCLUDE_DIR})
|
target_include_directories (dbms SYSTEM BEFORE PRIVATE ${DIVIDE_INCLUDE_DIR})
|
||||||
target_include_directories (dbms SYSTEM BEFORE PRIVATE ${SPARCEHASH_INCLUDE_DIR})
|
target_include_directories (dbms SYSTEM BEFORE PRIVATE ${SPARCEHASH_INCLUDE_DIR})
|
||||||
@ -331,7 +348,7 @@ if (USE_JEMALLOC)
|
|||||||
target_include_directories (dbms SYSTEM BEFORE PRIVATE ${JEMALLOC_INCLUDE_DIR}) # used in Interpreters/AsynchronousMetrics.cpp
|
target_include_directories (dbms SYSTEM BEFORE PRIVATE ${JEMALLOC_INCLUDE_DIR}) # used in Interpreters/AsynchronousMetrics.cpp
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
target_include_directories (dbms PUBLIC ${DBMS_INCLUDE_DIR})
|
target_include_directories (dbms PUBLIC ${DBMS_INCLUDE_DIR} PRIVATE ${CMAKE_CURRENT_BINARY_DIR}/src/Formats/include)
|
||||||
target_include_directories (clickhouse_common_io PUBLIC ${DBMS_INCLUDE_DIR})
|
target_include_directories (clickhouse_common_io PUBLIC ${DBMS_INCLUDE_DIR})
|
||||||
target_include_directories (clickhouse_common_io SYSTEM PUBLIC ${PCG_RANDOM_INCLUDE_DIR})
|
target_include_directories (clickhouse_common_io SYSTEM PUBLIC ${PCG_RANDOM_INCLUDE_DIR})
|
||||||
target_include_directories (clickhouse_common_io SYSTEM BEFORE PUBLIC ${DOUBLE_CONVERSION_INCLUDE_DIR})
|
target_include_directories (clickhouse_common_io SYSTEM BEFORE PUBLIC ${DOUBLE_CONVERSION_INCLUDE_DIR})
|
||||||
@ -351,6 +368,6 @@ if (ENABLE_TESTS)
|
|||||||
# attach all dbms gtest sources
|
# attach all dbms gtest sources
|
||||||
grep_gtest_sources(${ClickHouse_SOURCE_DIR}/dbms dbms_gtest_sources)
|
grep_gtest_sources(${ClickHouse_SOURCE_DIR}/dbms dbms_gtest_sources)
|
||||||
add_executable(unit_tests_dbms ${dbms_gtest_sources})
|
add_executable(unit_tests_dbms ${dbms_gtest_sources})
|
||||||
target_link_libraries(unit_tests_dbms PRIVATE gtest_main dbms clickhouse_common_zookeeper)
|
target_link_libraries(unit_tests_dbms PRIVATE ${GTEST_BOTH_LIBRARIES} clickhouse_functions clickhouse_parsers dbms clickhouse_common_zookeeper)
|
||||||
add_check(unit_tests_dbms)
|
add_check(unit_tests_dbms)
|
||||||
endif ()
|
endif ()
|
||||||
|
|||||||
@ -1,11 +1,11 @@
|
|||||||
# This strings autochanged from release_lib.sh:
|
# This strings autochanged from release_lib.sh:
|
||||||
set(VERSION_REVISION 54415)
|
set(VERSION_REVISION 54422)
|
||||||
set(VERSION_MAJOR 19)
|
set(VERSION_MAJOR 19)
|
||||||
set(VERSION_MINOR 3)
|
set(VERSION_MINOR 10)
|
||||||
set(VERSION_PATCH 4)
|
set(VERSION_PATCH 1)
|
||||||
set(VERSION_GITHASH 263e69e861b769eae7e2bcc79d87673e3a08d376)
|
set(VERSION_GITHASH 864a5830f5b5fa67c6b7126e8f2615d03ee890ce)
|
||||||
set(VERSION_DESCRIBE v19.3.4-testing)
|
set(VERSION_DESCRIBE v19.10.1.1-prestable)
|
||||||
set(VERSION_STRING 19.3.4)
|
set(VERSION_STRING 19.10.1.1)
|
||||||
# end of autochange
|
# end of autochange
|
||||||
|
|
||||||
set(VERSION_EXTRA "" CACHE STRING "")
|
set(VERSION_EXTRA "" CACHE STRING "")
|
||||||
@ -24,3 +24,7 @@ set (VERSION_FULL "${VERSION_NAME} ${VERSION_STRING}")
|
|||||||
set (VERSION_SO "${VERSION_STRING}")
|
set (VERSION_SO "${VERSION_STRING}")
|
||||||
|
|
||||||
math (EXPR VERSION_INTEGER "${VERSION_PATCH} + ${VERSION_MINOR}*1000 + ${VERSION_MAJOR}*1000000")
|
math (EXPR VERSION_INTEGER "${VERSION_PATCH} + ${VERSION_MINOR}*1000 + ${VERSION_MAJOR}*1000000")
|
||||||
|
|
||||||
|
if(YANDEX_OFFICIAL_BUILD)
|
||||||
|
set(VERSION_OFFICIAL " (official build)")
|
||||||
|
endif()
|
||||||
|
|||||||
@ -22,7 +22,6 @@ endif()
|
|||||||
configure_file (config_tools.h.in ${CMAKE_CURRENT_BINARY_DIR}/config_tools.h)
|
configure_file (config_tools.h.in ${CMAKE_CURRENT_BINARY_DIR}/config_tools.h)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
macro(clickhouse_target_link_split_lib target name)
|
macro(clickhouse_target_link_split_lib target name)
|
||||||
if(NOT CLICKHOUSE_ONE_SHARED)
|
if(NOT CLICKHOUSE_ONE_SHARED)
|
||||||
target_link_libraries(${target} PRIVATE clickhouse-${name}-lib)
|
target_link_libraries(${target} PRIVATE clickhouse-${name}-lib)
|
||||||
@ -45,7 +44,7 @@ macro(clickhouse_program_add_library name)
|
|||||||
set(CLICKHOUSE_${name_uc}_INCLUDE ${CLICKHOUSE_${name_uc}_INCLUDE} PARENT_SCOPE)
|
set(CLICKHOUSE_${name_uc}_INCLUDE ${CLICKHOUSE_${name_uc}_INCLUDE} PARENT_SCOPE)
|
||||||
|
|
||||||
if(NOT CLICKHOUSE_ONE_SHARED)
|
if(NOT CLICKHOUSE_ONE_SHARED)
|
||||||
add_library(clickhouse-${name}-lib ${LINK_MODE} ${CLICKHOUSE_${name_uc}_SOURCES})
|
add_library(clickhouse-${name}-lib ${CLICKHOUSE_${name_uc}_SOURCES})
|
||||||
|
|
||||||
set(_link ${CLICKHOUSE_${name_uc}_LINK}) # can't use ${} in if()
|
set(_link ${CLICKHOUSE_${name_uc}_LINK}) # can't use ${} in if()
|
||||||
if(_link)
|
if(_link)
|
||||||
@ -91,9 +90,10 @@ endif ()
|
|||||||
|
|
||||||
if (CLICKHOUSE_ONE_SHARED)
|
if (CLICKHOUSE_ONE_SHARED)
|
||||||
add_library(clickhouse-lib SHARED ${CLICKHOUSE_SERVER_SOURCES} ${CLICKHOUSE_CLIENT_SOURCES} ${CLICKHOUSE_LOCAL_SOURCES} ${CLICKHOUSE_BENCHMARK_SOURCES} ${CLICKHOUSE_PERFORMANCE_TEST_SOURCES} ${CLICKHOUSE_COPIER_SOURCES} ${CLICKHOUSE_EXTRACT_FROM_CONFIG_SOURCES} ${CLICKHOUSE_COMPRESSOR_SOURCES} ${CLICKHOUSE_FORMAT_SOURCES} ${CLICKHOUSE_OBFUSCATOR_SOURCES} ${CLICKHOUSE_COMPILER_SOURCES} ${CLICKHOUSE_ODBC_BRIDGE_SOURCES})
|
add_library(clickhouse-lib SHARED ${CLICKHOUSE_SERVER_SOURCES} ${CLICKHOUSE_CLIENT_SOURCES} ${CLICKHOUSE_LOCAL_SOURCES} ${CLICKHOUSE_BENCHMARK_SOURCES} ${CLICKHOUSE_PERFORMANCE_TEST_SOURCES} ${CLICKHOUSE_COPIER_SOURCES} ${CLICKHOUSE_EXTRACT_FROM_CONFIG_SOURCES} ${CLICKHOUSE_COMPRESSOR_SOURCES} ${CLICKHOUSE_FORMAT_SOURCES} ${CLICKHOUSE_OBFUSCATOR_SOURCES} ${CLICKHOUSE_COMPILER_SOURCES} ${CLICKHOUSE_ODBC_BRIDGE_SOURCES})
|
||||||
target_link_libraries(clickhouse-lib PUBLIC ${CLICKHOUSE_SERVER_LINK} ${CLICKHOUSE_CLIENT_LINK} ${CLICKHOUSE_LOCAL_LINK} ${CLICKHOUSE_BENCHMARK_LINK} ${CLICKHOUSE_PERFORMANCE_TEST_LINK} ${CLICKHOUSE_COPIER_LINK} ${CLICKHOUSE_EXTRACT_FROM_CONFIG_LINK} ${CLICKHOUSE_COMPRESSOR_LINK} ${CLICKHOUSE_FORMAT_LINK} ${CLICKHOUSE_OBFUSCATOR_LINK} ${CLICKHOUSE_COMPILER_LINK} ${CLICKHOUSE_ODBC_BRIDGE_LINK})
|
target_link_libraries(clickhouse-lib ${CLICKHOUSE_SERVER_LINK} ${CLICKHOUSE_CLIENT_LINK} ${CLICKHOUSE_LOCAL_LINK} ${CLICKHOUSE_BENCHMARK_LINK} ${CLICKHOUSE_PERFORMANCE_TEST_LINK} ${CLICKHOUSE_COPIER_LINK} ${CLICKHOUSE_EXTRACT_FROM_CONFIG_LINK} ${CLICKHOUSE_COMPRESSOR_LINK} ${CLICKHOUSE_FORMAT_LINK} ${CLICKHOUSE_OBFUSCATOR_LINK} ${CLICKHOUSE_COMPILER_LINK} ${CLICKHOUSE_ODBC_BRIDGE_LINK})
|
||||||
set_target_properties(clickhouse-lib PROPERTIES SOVERSION ${VERSION_MAJOR}.${VERSION_MINOR} VERSION ${VERSION_SO} OUTPUT_NAME clickhouse)
|
|
||||||
target_include_directories(clickhouse-lib ${CLICKHOUSE_SERVER_INCLUDE} ${CLICKHOUSE_CLIENT_INCLUDE} ${CLICKHOUSE_LOCAL_INCLUDE} ${CLICKHOUSE_BENCHMARK_INCLUDE} ${CLICKHOUSE_PERFORMANCE_TEST_INCLUDE} ${CLICKHOUSE_COPIER_INCLUDE} ${CLICKHOUSE_EXTRACT_FROM_CONFIG_INCLUDE} ${CLICKHOUSE_COMPRESSOR_INCLUDE} ${CLICKHOUSE_FORMAT_INCLUDE} ${CLICKHOUSE_OBFUSCATOR_INCLUDE} ${CLICKHOUSE_COMPILER_INCLUDE} ${CLICKHOUSE_ODBC_BRIDGE_INCLUDE})
|
target_include_directories(clickhouse-lib ${CLICKHOUSE_SERVER_INCLUDE} ${CLICKHOUSE_CLIENT_INCLUDE} ${CLICKHOUSE_LOCAL_INCLUDE} ${CLICKHOUSE_BENCHMARK_INCLUDE} ${CLICKHOUSE_PERFORMANCE_TEST_INCLUDE} ${CLICKHOUSE_COPIER_INCLUDE} ${CLICKHOUSE_EXTRACT_FROM_CONFIG_INCLUDE} ${CLICKHOUSE_COMPRESSOR_INCLUDE} ${CLICKHOUSE_FORMAT_INCLUDE} ${CLICKHOUSE_OBFUSCATOR_INCLUDE} ${CLICKHOUSE_COMPILER_INCLUDE} ${CLICKHOUSE_ODBC_BRIDGE_INCLUDE})
|
||||||
|
set_target_properties(clickhouse-lib PROPERTIES SOVERSION ${VERSION_MAJOR}.${VERSION_MINOR} VERSION ${VERSION_SO} OUTPUT_NAME clickhouse DEBUG_POSTFIX "")
|
||||||
|
install (TARGETS clickhouse-lib LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT clickhouse)
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
if (CLICKHOUSE_SPLIT_BINARY)
|
if (CLICKHOUSE_SPLIT_BINARY)
|
||||||
@ -112,6 +112,8 @@ if (CLICKHOUSE_SPLIT_BINARY)
|
|||||||
|
|
||||||
add_custom_target (clickhouse-bundle ALL DEPENDS ${CLICKHOUSE_ALL_TARGETS})
|
add_custom_target (clickhouse-bundle ALL DEPENDS ${CLICKHOUSE_ALL_TARGETS})
|
||||||
add_custom_target (clickhouse ALL DEPENDS clickhouse-bundle)
|
add_custom_target (clickhouse ALL DEPENDS clickhouse-bundle)
|
||||||
|
|
||||||
|
install(PROGRAMS clickhouse-split-helper DESTINATION ${CMAKE_INSTALL_BINDIR} RENAME clickhouse COMPONENT clickhouse)
|
||||||
else ()
|
else ()
|
||||||
if (USE_EMBEDDED_COMPILER)
|
if (USE_EMBEDDED_COMPILER)
|
||||||
# before add_executable !
|
# before add_executable !
|
||||||
@ -123,34 +125,34 @@ else ()
|
|||||||
target_include_directories (clickhouse PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
|
target_include_directories (clickhouse PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
|
||||||
|
|
||||||
if (ENABLE_CLICKHOUSE_SERVER)
|
if (ENABLE_CLICKHOUSE_SERVER)
|
||||||
target_link_libraries (clickhouse PRIVATE clickhouse-server-lib)
|
clickhouse_target_link_split_lib(clickhouse server)
|
||||||
endif ()
|
endif ()
|
||||||
if (ENABLE_CLICKHOUSE_CLIENT)
|
if (ENABLE_CLICKHOUSE_CLIENT)
|
||||||
target_link_libraries (clickhouse PRIVATE clickhouse-client-lib)
|
clickhouse_target_link_split_lib(clickhouse client)
|
||||||
endif ()
|
endif ()
|
||||||
if (ENABLE_CLICKHOUSE_LOCAL)
|
if (ENABLE_CLICKHOUSE_LOCAL)
|
||||||
target_link_libraries (clickhouse PRIVATE clickhouse-local-lib)
|
clickhouse_target_link_split_lib(clickhouse local)
|
||||||
endif ()
|
endif ()
|
||||||
if (ENABLE_CLICKHOUSE_BENCHMARK)
|
if (ENABLE_CLICKHOUSE_BENCHMARK)
|
||||||
target_link_libraries (clickhouse PRIVATE clickhouse-benchmark-lib)
|
clickhouse_target_link_split_lib(clickhouse benchmark)
|
||||||
endif ()
|
endif ()
|
||||||
if (ENABLE_CLICKHOUSE_PERFORMANCE_TEST)
|
if (ENABLE_CLICKHOUSE_PERFORMANCE_TEST)
|
||||||
target_link_libraries (clickhouse PRIVATE clickhouse-performance-test-lib)
|
clickhouse_target_link_split_lib(clickhouse performance-test)
|
||||||
endif ()
|
endif ()
|
||||||
if (ENABLE_CLICKHOUSE_COPIER)
|
if (ENABLE_CLICKHOUSE_COPIER)
|
||||||
target_link_libraries (clickhouse PRIVATE clickhouse-copier-lib)
|
clickhouse_target_link_split_lib(clickhouse copier)
|
||||||
endif ()
|
endif ()
|
||||||
if (ENABLE_CLICKHOUSE_EXTRACT_FROM_CONFIG)
|
if (ENABLE_CLICKHOUSE_EXTRACT_FROM_CONFIG)
|
||||||
target_link_libraries (clickhouse PRIVATE clickhouse-extract-from-config-lib)
|
clickhouse_target_link_split_lib(clickhouse extract-from-config)
|
||||||
endif ()
|
endif ()
|
||||||
if (ENABLE_CLICKHOUSE_COMPRESSOR)
|
if (ENABLE_CLICKHOUSE_COMPRESSOR)
|
||||||
target_link_libraries (clickhouse PRIVATE clickhouse-compressor-lib)
|
clickhouse_target_link_split_lib(clickhouse compressor)
|
||||||
endif ()
|
endif ()
|
||||||
if (ENABLE_CLICKHOUSE_FORMAT)
|
if (ENABLE_CLICKHOUSE_FORMAT)
|
||||||
target_link_libraries (clickhouse PRIVATE clickhouse-format-lib)
|
clickhouse_target_link_split_lib(clickhouse format)
|
||||||
endif ()
|
endif ()
|
||||||
if (ENABLE_CLICKHOUSE_OBFUSCATOR)
|
if (ENABLE_CLICKHOUSE_OBFUSCATOR)
|
||||||
target_link_libraries (clickhouse PRIVATE clickhouse-obfuscator-lib)
|
clickhouse_target_link_split_lib(clickhouse obfuscator)
|
||||||
endif ()
|
endif ()
|
||||||
if (USE_EMBEDDED_COMPILER)
|
if (USE_EMBEDDED_COMPILER)
|
||||||
target_link_libraries(clickhouse PRIVATE clickhouse-compiler-lib)
|
target_link_libraries(clickhouse PRIVATE clickhouse-compiler-lib)
|
||||||
@ -208,11 +210,9 @@ else ()
|
|||||||
list(APPEND CLICKHOUSE_BUNDLE clickhouse-obfuscator)
|
list(APPEND CLICKHOUSE_BUNDLE clickhouse-obfuscator)
|
||||||
endif ()
|
endif ()
|
||||||
if(ENABLE_CLICKHOUSE_ODBC_BRIDGE)
|
if(ENABLE_CLICKHOUSE_ODBC_BRIDGE)
|
||||||
# just to be able to run integration tests
|
list(APPEND CLICKHOUSE_BUNDLE clickhouse-odbc-bridge)
|
||||||
add_custom_target (clickhouse-odbc-bridge-copy ALL COMMAND ${CMAKE_COMMAND} -E create_symlink ${CMAKE_CURRENT_BINARY_DIR}/odbc-bridge/clickhouse-odbc-bridge clickhouse-odbc-bridge DEPENDS clickhouse-odbc-bridge)
|
|
||||||
endif()
|
endif()
|
||||||
|
|
||||||
|
|
||||||
# install always because depian package want this files:
|
# install always because depian package want this files:
|
||||||
add_custom_target (clickhouse-clang ALL COMMAND ${CMAKE_COMMAND} -E create_symlink clickhouse clickhouse-clang DEPENDS clickhouse)
|
add_custom_target (clickhouse-clang ALL COMMAND ${CMAKE_COMMAND} -E create_symlink clickhouse clickhouse-clang DEPENDS clickhouse)
|
||||||
add_custom_target (clickhouse-lld ALL COMMAND ${CMAKE_COMMAND} -E create_symlink clickhouse clickhouse-lld DEPENDS clickhouse)
|
add_custom_target (clickhouse-lld ALL COMMAND ${CMAKE_COMMAND} -E create_symlink clickhouse clickhouse-lld DEPENDS clickhouse)
|
||||||
|
|||||||
@ -54,10 +54,10 @@ public:
|
|||||||
const String & host_, UInt16 port_, bool secure_, const String & default_database_,
|
const String & host_, UInt16 port_, bool secure_, const String & default_database_,
|
||||||
const String & user_, const String & password_, const String & stage,
|
const String & user_, const String & password_, const String & stage,
|
||||||
bool randomize_, size_t max_iterations_, double max_time_,
|
bool randomize_, size_t max_iterations_, double max_time_,
|
||||||
const String & json_path_, const ConnectionTimeouts & timeouts, const Settings & settings_)
|
const String & json_path_, const Settings & settings_)
|
||||||
:
|
:
|
||||||
concurrency(concurrency_), delay(delay_), queue(concurrency),
|
concurrency(concurrency_), delay(delay_), queue(concurrency),
|
||||||
connections(concurrency, host_, port_, default_database_, user_, password_, timeouts, "benchmark", Protocol::Compression::Enable, secure_ ? Protocol::Secure::Enable : Protocol::Secure::Disable),
|
connections(concurrency, host_, port_, default_database_, user_, password_, "benchmark", Protocol::Compression::Enable, secure_ ? Protocol::Secure::Enable : Protocol::Secure::Disable),
|
||||||
randomize(randomize_), max_iterations(max_iterations_), max_time(max_time_),
|
randomize(randomize_), max_iterations(max_iterations_), max_time(max_time_),
|
||||||
json_path(json_path_), settings(settings_), global_context(Context::createGlobal()), pool(concurrency)
|
json_path(json_path_), settings(settings_), global_context(Context::createGlobal()), pool(concurrency)
|
||||||
{
|
{
|
||||||
@ -240,7 +240,8 @@ private:
|
|||||||
std::uniform_int_distribution<size_t> distribution(0, queries.size() - 1);
|
std::uniform_int_distribution<size_t> distribution(0, queries.size() - 1);
|
||||||
|
|
||||||
for (size_t i = 0; i < concurrency; ++i)
|
for (size_t i = 0; i < concurrency; ++i)
|
||||||
pool.schedule(std::bind(&Benchmark::thread, this, connections.get()));
|
pool.schedule(std::bind(&Benchmark::thread, this,
|
||||||
|
connections.get(ConnectionTimeouts::getTCPTimeoutsWithoutFailover(settings))));
|
||||||
|
|
||||||
InterruptListener interrupt_listener;
|
InterruptListener interrupt_listener;
|
||||||
info_per_interval.watch.restart();
|
info_per_interval.watch.restart();
|
||||||
@ -310,7 +311,9 @@ private:
|
|||||||
void execute(ConnectionPool::Entry & connection, Query & query)
|
void execute(ConnectionPool::Entry & connection, Query & query)
|
||||||
{
|
{
|
||||||
Stopwatch watch;
|
Stopwatch watch;
|
||||||
RemoteBlockInputStream stream(*connection, query, {}, global_context, &settings, nullptr, Tables(), query_processing_stage);
|
RemoteBlockInputStream stream(
|
||||||
|
*connection,
|
||||||
|
query, {}, global_context, &settings, nullptr, Tables(), query_processing_stage);
|
||||||
|
|
||||||
Progress progress;
|
Progress progress;
|
||||||
stream.setProgressCallback([&progress](const Progress & value) { progress.incrementPiecewiseAtomically(value); });
|
stream.setProgressCallback([&progress](const Progress & value) { progress.incrementPiecewiseAtomically(value); });
|
||||||
@ -325,8 +328,8 @@ private:
|
|||||||
double seconds = watch.elapsedSeconds();
|
double seconds = watch.elapsedSeconds();
|
||||||
|
|
||||||
std::lock_guard lock(mutex);
|
std::lock_guard lock(mutex);
|
||||||
info_per_interval.add(seconds, progress.rows, progress.bytes, info.rows, info.bytes);
|
info_per_interval.add(seconds, progress.read_rows, progress.read_bytes, info.rows, info.bytes);
|
||||||
info_total.add(seconds, progress.rows, progress.bytes, info.rows, info.bytes);
|
info_total.add(seconds, progress.read_rows, progress.read_bytes, info.rows, info.bytes);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@ -439,7 +442,7 @@ int mainEntryClickHouseBenchmark(int argc, char ** argv)
|
|||||||
("help", "produce help message")
|
("help", "produce help message")
|
||||||
("concurrency,c", value<unsigned>()->default_value(1), "number of parallel queries")
|
("concurrency,c", value<unsigned>()->default_value(1), "number of parallel queries")
|
||||||
("delay,d", value<double>()->default_value(1), "delay between intermediate reports in seconds (set 0 to disable reports)")
|
("delay,d", value<double>()->default_value(1), "delay between intermediate reports in seconds (set 0 to disable reports)")
|
||||||
("stage", value<std::string>()->default_value("complete"), "request query processing up to specified stage")
|
("stage", value<std::string>()->default_value("complete"), "request query processing up to specified stage: complete,fetch_columns,with_mergeable_state")
|
||||||
("iterations,i", value<size_t>()->default_value(0), "amount of queries to be executed")
|
("iterations,i", value<size_t>()->default_value(0), "amount of queries to be executed")
|
||||||
("timelimit,t", value<double>()->default_value(0.), "stop launch of queries after specified time limit")
|
("timelimit,t", value<double>()->default_value(0.), "stop launch of queries after specified time limit")
|
||||||
("randomize,r", value<bool>()->default_value(false), "randomize order of execution")
|
("randomize,r", value<bool>()->default_value(false), "randomize order of execution")
|
||||||
@ -451,14 +454,14 @@ int mainEntryClickHouseBenchmark(int argc, char ** argv)
|
|||||||
("password", value<std::string>()->default_value(""), "")
|
("password", value<std::string>()->default_value(""), "")
|
||||||
("database", value<std::string>()->default_value("default"), "")
|
("database", value<std::string>()->default_value("default"), "")
|
||||||
("stacktrace", "print stack traces of exceptions")
|
("stacktrace", "print stack traces of exceptions")
|
||||||
|
|
||||||
#define DECLARE_SETTING(TYPE, NAME, DEFAULT, DESCRIPTION) (#NAME, boost::program_options::value<std::string> (), DESCRIPTION)
|
|
||||||
APPLY_FOR_SETTINGS(DECLARE_SETTING)
|
|
||||||
#undef DECLARE_SETTING
|
|
||||||
;
|
;
|
||||||
|
|
||||||
|
Settings settings;
|
||||||
|
settings.addProgramOptions(desc);
|
||||||
|
|
||||||
boost::program_options::variables_map options;
|
boost::program_options::variables_map options;
|
||||||
boost::program_options::store(boost::program_options::parse_command_line(argc, argv, desc), options);
|
boost::program_options::store(boost::program_options::parse_command_line(argc, argv, desc), options);
|
||||||
|
boost::program_options::notify(options);
|
||||||
|
|
||||||
if (options.count("help"))
|
if (options.count("help"))
|
||||||
{
|
{
|
||||||
@ -469,15 +472,6 @@ int mainEntryClickHouseBenchmark(int argc, char ** argv)
|
|||||||
|
|
||||||
print_stacktrace = options.count("stacktrace");
|
print_stacktrace = options.count("stacktrace");
|
||||||
|
|
||||||
/// Extract `settings` and `limits` from received `options`
|
|
||||||
Settings settings;
|
|
||||||
|
|
||||||
#define EXTRACT_SETTING(TYPE, NAME, DEFAULT, DESCRIPTION) \
|
|
||||||
if (options.count(#NAME)) \
|
|
||||||
settings.set(#NAME, options[#NAME].as<std::string>());
|
|
||||||
APPLY_FOR_SETTINGS(EXTRACT_SETTING)
|
|
||||||
#undef EXTRACT_SETTING
|
|
||||||
|
|
||||||
UseSSL use_ssl;
|
UseSSL use_ssl;
|
||||||
|
|
||||||
Benchmark benchmark(
|
Benchmark benchmark(
|
||||||
@ -494,7 +488,6 @@ int mainEntryClickHouseBenchmark(int argc, char ** argv)
|
|||||||
options["iterations"].as<size_t>(),
|
options["iterations"].as<size_t>(),
|
||||||
options["timelimit"].as<double>(),
|
options["timelimit"].as<double>(),
|
||||||
options["json"].as<std::string>(),
|
options["json"].as<std::string>(),
|
||||||
ConnectionTimeouts::getTCPTimeoutsWithoutFailover(settings),
|
|
||||||
settings);
|
settings);
|
||||||
return benchmark.run();
|
return benchmark.run();
|
||||||
}
|
}
|
||||||
|
|||||||
@ -26,8 +26,8 @@ elseif (EXISTS ${INTERNAL_COMPILER_BIN_ROOT}${INTERNAL_COMPILER_EXECUTABLE})
|
|||||||
set (COPY_HEADERS_COMPILER "${INTERNAL_COMPILER_BIN_ROOT}${INTERNAL_COMPILER_EXECUTABLE}")
|
set (COPY_HEADERS_COMPILER "${INTERNAL_COMPILER_BIN_ROOT}${INTERNAL_COMPILER_EXECUTABLE}")
|
||||||
endif ()
|
endif ()
|
||||||
|
|
||||||
if (COPY_HEADERS_COMPILER AND OS_LINUX)
|
if (COPY_HEADERS_COMPILER)
|
||||||
add_custom_target (copy-headers [ -f ${TMP_HEADERS_DIR}/dbms/src/Interpreters/SpecializedAggregator.h ] || env CLANG=${COPY_HEADERS_COMPILER} BUILD_PATH=${ClickHouse_BINARY_DIR} DESTDIR=${ClickHouse_SOURCE_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/copy_headers.sh ${ClickHouse_SOURCE_DIR} ${TMP_HEADERS_DIR} DEPENDS ${COPY_HEADERS_DEPENDS} WORKING_DIRECTORY ${ClickHouse_SOURCE_DIR} SOURCES copy_headers.sh)
|
add_custom_target (copy-headers [ -f ${TMP_HEADERS_DIR}/dbms/src/Interpreters/SpecializedAggregator.h ] || env CLANG=${COPY_HEADERS_COMPILER} BUILD_PATH=${ClickHouse_BINARY_DIR} DESTDIR=${ClickHouse_SOURCE_DIR} CMAKE_CXX_COMPILER_VERSION=${CMAKE_CXX_COMPILER_VERSION} ${CMAKE_CURRENT_SOURCE_DIR}/copy_headers.sh ${ClickHouse_SOURCE_DIR} ${TMP_HEADERS_DIR} DEPENDS ${COPY_HEADERS_DEPENDS} WORKING_DIRECTORY ${ClickHouse_SOURCE_DIR} SOURCES copy_headers.sh)
|
||||||
|
|
||||||
if (USE_INTERNAL_LLVM_LIBRARY)
|
if (USE_INTERNAL_LLVM_LIBRARY)
|
||||||
set (CLANG_HEADERS_DIR "${ClickHouse_SOURCE_DIR}/contrib/llvm/clang/lib/Headers")
|
set (CLANG_HEADERS_DIR "${ClickHouse_SOURCE_DIR}/contrib/llvm/clang/lib/Headers")
|
||||||
|
|||||||
@ -2,7 +2,7 @@ add_definitions(-Wno-error -Wno-unused-parameter -Wno-non-virtual-dtor -U_LIBCPP
|
|||||||
|
|
||||||
link_directories(${LLVM_LIBRARY_DIRS})
|
link_directories(${LLVM_LIBRARY_DIRS})
|
||||||
|
|
||||||
add_library(clickhouse-compiler-lib ${LINK_MODE}
|
add_library(clickhouse-compiler-lib
|
||||||
driver.cpp
|
driver.cpp
|
||||||
cc1_main.cpp
|
cc1_main.cpp
|
||||||
cc1as_main.cpp
|
cc1as_main.cpp
|
||||||
@ -46,7 +46,7 @@ LLVMSupport
|
|||||||
#PollyISL
|
#PollyISL
|
||||||
#PollyPPCG
|
#PollyPPCG
|
||||||
|
|
||||||
PUBLIC ${ZLIB_LIBRARIES} ${EXECINFO_LIBRARY} Threads::Threads
|
PUBLIC ${ZLIB_LIBRARIES} ${EXECINFO_LIBRARIES} Threads::Threads
|
||||||
${MALLOC_LIBRARIES}
|
${MALLOC_LIBRARIES}
|
||||||
${GLIBC_COMPATIBILITY_LIBRARIES}
|
${GLIBC_COMPATIBILITY_LIBRARIES}
|
||||||
${MEMCPY_LIBRARIES}
|
${MEMCPY_LIBRARIES}
|
||||||
|
|||||||
@ -1,8 +1,9 @@
|
|||||||
|
|
||||||
add_definitions(-Wno-error -Wno-unused-parameter -Wno-non-virtual-dtor -U_LIBCPP_DEBUG)
|
add_definitions(-Wno-error -Wno-unused-parameter -Wno-non-virtual-dtor -U_LIBCPP_DEBUG)
|
||||||
|
|
||||||
link_directories(${LLVM_LIBRARY_DIRS})
|
link_directories(${LLVM_LIBRARY_DIRS})
|
||||||
|
|
||||||
add_library(clickhouse-compiler-lib ${LINK_MODE}
|
add_library(clickhouse-compiler-lib
|
||||||
driver.cpp
|
driver.cpp
|
||||||
cc1_main.cpp
|
cc1_main.cpp
|
||||||
cc1as_main.cpp
|
cc1as_main.cpp
|
||||||
@ -46,7 +47,7 @@ ${REQUIRED_LLVM_LIBRARIES}
|
|||||||
#PollyISL
|
#PollyISL
|
||||||
#PollyPPCG
|
#PollyPPCG
|
||||||
|
|
||||||
PUBLIC ${ZLIB_LIBRARIES} ${EXECINFO_LIBRARY} Threads::Threads
|
PUBLIC ${ZLIB_LIBRARIES} ${EXECINFO_LIBRARIES} Threads::Threads
|
||||||
${MALLOC_LIBRARIES}
|
${MALLOC_LIBRARIES}
|
||||||
${GLIBC_COMPATIBILITY_LIBRARIES}
|
${GLIBC_COMPATIBILITY_LIBRARIES}
|
||||||
${MEMCPY_LIBRARIES}
|
${MEMCPY_LIBRARIES}
|
||||||
|
|||||||
@ -2,7 +2,7 @@ add_definitions(-Wno-error -Wno-unused-parameter -Wno-non-virtual-dtor -U_LIBCPP
|
|||||||
|
|
||||||
link_directories(${LLVM_LIBRARY_DIRS})
|
link_directories(${LLVM_LIBRARY_DIRS})
|
||||||
|
|
||||||
add_library(clickhouse-compiler-lib ${LINK_MODE}
|
add_library(clickhouse-compiler-lib
|
||||||
driver.cpp
|
driver.cpp
|
||||||
cc1_main.cpp
|
cc1_main.cpp
|
||||||
cc1gen_reproducer_main.cpp
|
cc1gen_reproducer_main.cpp
|
||||||
@ -42,7 +42,7 @@ lldCore
|
|||||||
|
|
||||||
${REQUIRED_LLVM_LIBRARIES}
|
${REQUIRED_LLVM_LIBRARIES}
|
||||||
|
|
||||||
PUBLIC ${ZLIB_LIBRARIES} ${EXECINFO_LIBRARY} Threads::Threads
|
PUBLIC ${ZLIB_LIBRARIES} ${EXECINFO_LIBRARIES} Threads::Threads
|
||||||
${MALLOC_LIBRARIES}
|
${MALLOC_LIBRARIES}
|
||||||
${GLIBC_COMPATIBILITY_LIBRARIES}
|
${GLIBC_COMPATIBILITY_LIBRARIES}
|
||||||
${MEMCPY_LIBRARIES}
|
${MEMCPY_LIBRARIES}
|
||||||
|
|||||||
@ -2,7 +2,7 @@ add_definitions(-Wno-error -Wno-unused-parameter -Wno-non-virtual-dtor -U_LIBCPP
|
|||||||
|
|
||||||
link_directories(${LLVM_LIBRARY_DIRS})
|
link_directories(${LLVM_LIBRARY_DIRS})
|
||||||
|
|
||||||
add_library(clickhouse-compiler-lib ${LINK_MODE}
|
add_library(clickhouse-compiler-lib
|
||||||
driver.cpp
|
driver.cpp
|
||||||
cc1_main.cpp
|
cc1_main.cpp
|
||||||
cc1as_main.cpp
|
cc1as_main.cpp
|
||||||
@ -42,7 +42,7 @@ lldCore
|
|||||||
|
|
||||||
${REQUIRED_LLVM_LIBRARIES}
|
${REQUIRED_LLVM_LIBRARIES}
|
||||||
|
|
||||||
PUBLIC ${ZLIB_LIBRARIES} ${EXECINFO_LIBRARY} Threads::Threads
|
PUBLIC ${ZLIB_LIBRARIES} ${EXECINFO_LIBRARIES} Threads::Threads
|
||||||
${MALLOC_LIBRARIES}
|
${MALLOC_LIBRARIES}
|
||||||
${GLIBC_COMPATIBILITY_LIBRARIES}
|
${GLIBC_COMPATIBILITY_LIBRARIES}
|
||||||
${MEMCPY_LIBRARIES}
|
${MEMCPY_LIBRARIES}
|
||||||
|
|||||||
@ -38,26 +38,28 @@ for header in $START_HEADERS; do
|
|||||||
START_HEADERS_INCLUDE+="-include $header "
|
START_HEADERS_INCLUDE+="-include $header "
|
||||||
done
|
done
|
||||||
|
|
||||||
# Опция -mcx16 для того, чтобы выбиралось больше заголовочных файлов (с запасом).
|
|
||||||
# The latter options are the same that are added while building packages.
|
|
||||||
|
|
||||||
# TODO: Does not work on macos:
|
|
||||||
GCC_ROOT=`$CLANG -v 2>&1 | grep "Selected GCC installation"| sed -n -e 's/^.*: //p'`
|
GCC_ROOT=`$CLANG -v 2>&1 | grep "Selected GCC installation"| sed -n -e 's/^.*: //p'`
|
||||||
|
|
||||||
for src_file in $(echo | $CLANG -M -xc++ -std=c++1z -Wall -Werror -msse4 -mcx16 -mpopcnt -O3 -g -fPIC -fstack-protector -D_FORTIFY_SOURCE=2 \
|
# TODO: Does not work on macos?
|
||||||
|
GCC_ROOT=${GCC_ROOT:=/usr/lib/clang/${CMAKE_CXX_COMPILER_VERSION}}
|
||||||
|
|
||||||
|
# Опция -mcx16 для того, чтобы выбиралось больше заголовочных файлов (с запасом).
|
||||||
|
# The latter options are the same that are added while building packages.
|
||||||
|
for src_file in $(echo | $CLANG -M -xc++ -std=c++1z -Wall -Werror -msse2 -msse4 -mcx16 -mpopcnt -O3 -g -fPIC -fstack-protector -D_FORTIFY_SOURCE=2 \
|
||||||
-I $GCC_ROOT/include \
|
-I $GCC_ROOT/include \
|
||||||
-I $GCC_ROOT/include-fixed \
|
-I $GCC_ROOT/include-fixed \
|
||||||
$(cat "$BUILD_PATH/include_directories.txt") \
|
$(cat "$BUILD_PATH/include_directories.txt") \
|
||||||
$START_HEADERS_INCLUDE \
|
$START_HEADERS_INCLUDE \
|
||||||
- |
|
- |
|
||||||
tr -d '\\' |
|
tr -d '\\' |
|
||||||
sed --posix -E -e 's/^-\.o://');
|
sed -E -e 's/^-\.o://');
|
||||||
do
|
do
|
||||||
dst_file=$src_file;
|
dst_file=$src_file;
|
||||||
[ -n $BUILD_PATH ] && dst_file=$(echo $dst_file | sed --posix -E -e "s!^$BUILD_PATH!!")
|
[ -n $BUILD_PATH ] && dst_file=$(echo $dst_file | sed -E -e "s!^$BUILD_PATH!!")
|
||||||
[ -n $DESTDIR ] && dst_file=$(echo $dst_file | sed --posix -E -e "s!^$DESTDIR!!")
|
[ -n $DESTDIR ] && dst_file=$(echo $dst_file | sed -E -e "s!^$DESTDIR!!")
|
||||||
dst_file=$(echo $dst_file | sed --posix -E -e 's/build\///') # for simplicity reasons, will put generated headers near the rest.
|
dst_file=$(echo $dst_file | sed -E -e 's/build\///') # for simplicity reasons, will put generated headers near the rest.
|
||||||
mkdir -p "$DST/$(echo $dst_file | sed --posix -E -e 's/\/[^/]*$/\//')";
|
mkdir -p "$DST/$(echo $dst_file | sed -E -e 's/\/[^/]*$/\//')";
|
||||||
cp "$src_file" "$DST/$dst_file";
|
cp "$src_file" "$DST/$dst_file";
|
||||||
done
|
done
|
||||||
|
|
||||||
@ -68,9 +70,9 @@ done
|
|||||||
for src_file in $(ls -1 $($CLANG -v -xc++ - <<<'' 2>&1 | grep '^ /' | grep 'include' | grep -E '/lib/clang/|/include/clang/')/*.h | grep -vE 'arm|altivec|Intrin');
|
for src_file in $(ls -1 $($CLANG -v -xc++ - <<<'' 2>&1 | grep '^ /' | grep 'include' | grep -E '/lib/clang/|/include/clang/')/*.h | grep -vE 'arm|altivec|Intrin');
|
||||||
do
|
do
|
||||||
dst_file=$src_file;
|
dst_file=$src_file;
|
||||||
[ -n $BUILD_PATH ] && dst_file=$(echo $dst_file | sed --posix -E -e "s!^$BUILD_PATH!!")
|
[ -n $BUILD_PATH ] && dst_file=$(echo $dst_file | sed -E -e "s!^$BUILD_PATH!!")
|
||||||
[ -n $DESTDIR ] && dst_file=$(echo $dst_file | sed --posix -E -e "s!^$DESTDIR!!")
|
[ -n $DESTDIR ] && dst_file=$(echo $dst_file | sed -E -e "s!^$DESTDIR!!")
|
||||||
mkdir -p "$DST/$(echo $dst_file | sed --posix -E -e 's/\/[^/]*$/\//')";
|
mkdir -p "$DST/$(echo $dst_file | sed -E -e 's/\/[^/]*$/\//')";
|
||||||
cp "$src_file" "$DST/$dst_file";
|
cp "$src_file" "$DST/$dst_file";
|
||||||
done
|
done
|
||||||
|
|
||||||
@ -79,9 +81,9 @@ if [ -d "$SOURCE_PATH/contrib/boost/libs/smart_ptr/include/boost/smart_ptr/detai
|
|||||||
for src_file in $(ls -1 $SOURCE_PATH/contrib/boost/libs/smart_ptr/include/boost/smart_ptr/detail/*);
|
for src_file in $(ls -1 $SOURCE_PATH/contrib/boost/libs/smart_ptr/include/boost/smart_ptr/detail/*);
|
||||||
do
|
do
|
||||||
dst_file=$src_file;
|
dst_file=$src_file;
|
||||||
[ -n $BUILD_PATH ] && dst_file=$(echo $dst_file | sed --posix -E -e "s!^$BUILD_PATH!!")
|
[ -n $BUILD_PATH ] && dst_file=$(echo $dst_file | sed -E -e "s!^$BUILD_PATH!!")
|
||||||
[ -n $DESTDIR ] && dst_file=$(echo $dst_file | sed --posix -E -e "s!^$DESTDIR!!")
|
[ -n $DESTDIR ] && dst_file=$(echo $dst_file | sed -E -e "s!^$DESTDIR!!")
|
||||||
mkdir -p "$DST/$(echo $dst_file | sed --posix -E -e 's/\/[^/]*$/\//')";
|
mkdir -p "$DST/$(echo $dst_file | sed -E -e 's/\/[^/]*$/\//')";
|
||||||
cp "$src_file" "$DST/$dst_file";
|
cp "$src_file" "$DST/$dst_file";
|
||||||
done
|
done
|
||||||
fi
|
fi
|
||||||
@ -90,9 +92,9 @@ if [ -d "$SOURCE_PATH/contrib/boost/boost/smart_ptr/detail" ]; then
|
|||||||
for src_file in $(ls -1 $SOURCE_PATH/contrib/boost/boost/smart_ptr/detail/*);
|
for src_file in $(ls -1 $SOURCE_PATH/contrib/boost/boost/smart_ptr/detail/*);
|
||||||
do
|
do
|
||||||
dst_file=$src_file;
|
dst_file=$src_file;
|
||||||
[ -n $BUILD_PATH ] && dst_file=$(echo $dst_file | sed --posix -E -e "s!^$BUILD_PATH!!")
|
[ -n $BUILD_PATH ] && dst_file=$(echo $dst_file | sed -E -e "s!^$BUILD_PATH!!")
|
||||||
[ -n $DESTDIR ] && dst_file=$(echo $dst_file | sed --posix -E -e "s!^$DESTDIR!!")
|
[ -n $DESTDIR ] && dst_file=$(echo $dst_file | sed -E -e "s!^$DESTDIR!!")
|
||||||
mkdir -p "$DST/$(echo $dst_file | sed --posix -E -e 's/\/[^/]*$/\//')";
|
mkdir -p "$DST/$(echo $dst_file | sed -E -e 's/\/[^/]*$/\//')";
|
||||||
cp "$src_file" "$DST/$dst_file";
|
cp "$src_file" "$DST/$dst_file";
|
||||||
done
|
done
|
||||||
fi
|
fi
|
||||||
|
|||||||
12
dbms/programs/clickhouse-split-helper
Executable file
12
dbms/programs/clickhouse-split-helper
Executable file
@ -0,0 +1,12 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
|
||||||
|
# Helper for split build mode.
|
||||||
|
# Allows to run commands like
|
||||||
|
# clickhouse client
|
||||||
|
# clickhouse server
|
||||||
|
# ...
|
||||||
|
|
||||||
|
set -e
|
||||||
|
CMD=$1
|
||||||
|
shift
|
||||||
|
clickhouse-$CMD $*
|
||||||
@ -1,6 +1,19 @@
|
|||||||
set(CLICKHOUSE_CLIENT_SOURCES ${CMAKE_CURRENT_SOURCE_DIR}/Client.cpp)
|
set(CLICKHOUSE_CLIENT_SOURCES
|
||||||
|
${CMAKE_CURRENT_SOURCE_DIR}/Client.cpp
|
||||||
|
${CMAKE_CURRENT_SOURCE_DIR}/ConnectionParameters.cpp
|
||||||
|
)
|
||||||
|
|
||||||
set(CLICKHOUSE_CLIENT_LINK PRIVATE clickhouse_common_config clickhouse_functions clickhouse_aggregate_functions clickhouse_common_io ${LINE_EDITING_LIBS} ${Boost_PROGRAM_OPTIONS_LIBRARY})
|
set(CLICKHOUSE_CLIENT_LINK PRIVATE clickhouse_common_config clickhouse_functions clickhouse_aggregate_functions clickhouse_common_io ${LINE_EDITING_LIBS} ${Boost_PROGRAM_OPTIONS_LIBRARY})
|
||||||
set(CLICKHOUSE_CLIENT_INCLUDE SYSTEM PRIVATE ${READLINE_INCLUDE_DIR})
|
set(CLICKHOUSE_CLIENT_INCLUDE SYSTEM PRIVATE ${READLINE_INCLUDE_DIR} PRIVATE ${CMAKE_CURRENT_BINARY_DIR}/include)
|
||||||
|
|
||||||
|
include(CheckSymbolExists)
|
||||||
|
check_symbol_exists(readpassphrase readpassphrase.h HAVE_READPASSPHRASE)
|
||||||
|
configure_file(config_client.h.in ${CMAKE_CURRENT_BINARY_DIR}/include/config_client.h)
|
||||||
|
|
||||||
|
if(NOT HAVE_READPASSPHRASE)
|
||||||
|
add_subdirectory(readpassphrase)
|
||||||
|
list(APPEND CLICKHOUSE_CLIENT_LINK PRIVATE readpassphrase)
|
||||||
|
endif()
|
||||||
|
|
||||||
clickhouse_program_add(client)
|
clickhouse_program_add(client)
|
||||||
|
|
||||||
|
|||||||
@ -42,6 +42,7 @@
|
|||||||
#include <IO/ReadBufferFromString.h>
|
#include <IO/ReadBufferFromString.h>
|
||||||
#include <IO/ReadHelpers.h>
|
#include <IO/ReadHelpers.h>
|
||||||
#include <IO/WriteHelpers.h>
|
#include <IO/WriteHelpers.h>
|
||||||
|
#include <IO/Operators.h>
|
||||||
#include <IO/UseSSL.h>
|
#include <IO/UseSSL.h>
|
||||||
#include <DataStreams/AsynchronousBlockInputStream.h>
|
#include <DataStreams/AsynchronousBlockInputStream.h>
|
||||||
#include <DataStreams/AddingDefaultsBlockInputStream.h>
|
#include <DataStreams/AddingDefaultsBlockInputStream.h>
|
||||||
@ -64,9 +65,10 @@
|
|||||||
#include <AggregateFunctions/registerAggregateFunctions.h>
|
#include <AggregateFunctions/registerAggregateFunctions.h>
|
||||||
#include <Common/Config/configReadClient.h>
|
#include <Common/Config/configReadClient.h>
|
||||||
#include <Storages/ColumnsDescription.h>
|
#include <Storages/ColumnsDescription.h>
|
||||||
|
#include <common/argsToConfig.h>
|
||||||
|
|
||||||
#if USE_READLINE
|
#if USE_READLINE
|
||||||
#include "Suggest.h" // Y_IGNORE
|
#include "Suggest.h"
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
#ifndef __clang__
|
#ifndef __clang__
|
||||||
@ -101,6 +103,7 @@ namespace ErrorCodes
|
|||||||
extern const int LOGICAL_ERROR;
|
extern const int LOGICAL_ERROR;
|
||||||
extern const int CANNOT_SET_SIGNAL_HANDLER;
|
extern const int CANNOT_SET_SIGNAL_HANDLER;
|
||||||
extern const int CANNOT_READLINE;
|
extern const int CANNOT_READLINE;
|
||||||
|
extern const int SYSTEM_ERROR;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@ -201,7 +204,6 @@ private:
|
|||||||
|
|
||||||
ConnectionParameters connection_parameters;
|
ConnectionParameters connection_parameters;
|
||||||
|
|
||||||
|
|
||||||
void initialize(Poco::Util::Application & self)
|
void initialize(Poco::Util::Application & self)
|
||||||
{
|
{
|
||||||
Poco::Util::Application::initialize(self);
|
Poco::Util::Application::initialize(self);
|
||||||
@ -215,11 +217,12 @@ private:
|
|||||||
context.setApplicationType(Context::ApplicationType::CLIENT);
|
context.setApplicationType(Context::ApplicationType::CLIENT);
|
||||||
|
|
||||||
/// settings and limits could be specified in config file, but passed settings has higher priority
|
/// settings and limits could be specified in config file, but passed settings has higher priority
|
||||||
#define EXTRACT_SETTING(TYPE, NAME, DEFAULT, DESCRIPTION) \
|
for (auto && setting : context.getSettingsRef())
|
||||||
if (config().has(#NAME) && !context.getSettingsRef().NAME.changed) \
|
{
|
||||||
context.setSetting(#NAME, config().getString(#NAME));
|
const String & name = setting.getName().toString();
|
||||||
APPLY_FOR_SETTINGS(EXTRACT_SETTING)
|
if (config().has(name) && !setting.isChanged())
|
||||||
#undef EXTRACT_SETTING
|
setting.setValue(config().getString(name));
|
||||||
|
}
|
||||||
|
|
||||||
/// Set path for format schema files
|
/// Set path for format schema files
|
||||||
if (config().has("format_schema_path"))
|
if (config().has("format_schema_path"))
|
||||||
@ -295,7 +298,6 @@ private:
|
|||||||
/// The value of the option is used as the text of query (or of multiple queries).
|
/// The value of the option is used as the text of query (or of multiple queries).
|
||||||
/// If stdin is not a terminal, INSERT data for the first query is read from it.
|
/// If stdin is not a terminal, INSERT data for the first query is read from it.
|
||||||
/// - stdin is not a terminal. In this case queries are read from it.
|
/// - stdin is not a terminal. In this case queries are read from it.
|
||||||
stdin_is_not_tty = !isatty(STDIN_FILENO);
|
|
||||||
if (stdin_is_not_tty || config().has("query"))
|
if (stdin_is_not_tty || config().has("query"))
|
||||||
is_interactive = false;
|
is_interactive = false;
|
||||||
|
|
||||||
@ -334,7 +336,7 @@ private:
|
|||||||
DateLUT::instance();
|
DateLUT::instance();
|
||||||
if (!context.getSettingsRef().use_client_time_zone)
|
if (!context.getSettingsRef().use_client_time_zone)
|
||||||
{
|
{
|
||||||
const auto & time_zone = connection->getServerTimezone();
|
const auto & time_zone = connection->getServerTimezone(connection_parameters.timeouts);
|
||||||
if (!time_zone.empty())
|
if (!time_zone.empty())
|
||||||
{
|
{
|
||||||
try
|
try
|
||||||
@ -433,7 +435,7 @@ private:
|
|||||||
#if USE_READLINE
|
#if USE_READLINE
|
||||||
int res = read_history(history_file.c_str());
|
int res = read_history(history_file.c_str());
|
||||||
if (res)
|
if (res)
|
||||||
throwFromErrno("Cannot read history from file " + history_file, ErrorCodes::CANNOT_READ_HISTORY);
|
std::cerr << "Cannot read history from file " + history_file + ": "+ errnoToString(ErrorCodes::CANNOT_READ_HISTORY);
|
||||||
#endif
|
#endif
|
||||||
}
|
}
|
||||||
else /// Create history file.
|
else /// Create history file.
|
||||||
@ -478,6 +480,17 @@ private:
|
|||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
|
/// This is intended for testing purposes.
|
||||||
|
if (config().getBool("always_load_suggestion_data", false))
|
||||||
|
{
|
||||||
|
#if USE_READLINE
|
||||||
|
SCOPE_EXIT({ Suggest::instance().finalize(); });
|
||||||
|
Suggest::instance().load(connection_parameters, config().getInt("suggestion_limit"));
|
||||||
|
#else
|
||||||
|
throw Exception("Command line suggestions cannot work without readline", ErrorCodes::BAD_ARGUMENTS);
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
query_id = config().getString("query_id", "");
|
query_id = config().getString("query_id", "");
|
||||||
nonInteractive();
|
nonInteractive();
|
||||||
|
|
||||||
@ -507,7 +520,6 @@ private:
|
|||||||
connection_parameters.default_database,
|
connection_parameters.default_database,
|
||||||
connection_parameters.user,
|
connection_parameters.user,
|
||||||
connection_parameters.password,
|
connection_parameters.password,
|
||||||
connection_parameters.timeouts,
|
|
||||||
"client",
|
"client",
|
||||||
connection_parameters.compression,
|
connection_parameters.compression,
|
||||||
connection_parameters.security);
|
connection_parameters.security);
|
||||||
@ -523,11 +535,14 @@ private:
|
|||||||
connection->setThrottler(throttler);
|
connection->setThrottler(throttler);
|
||||||
}
|
}
|
||||||
|
|
||||||
connection->getServerVersion(server_name, server_version_major, server_version_minor, server_version_patch, server_revision);
|
connection->getServerVersion(connection_parameters.timeouts,
|
||||||
|
server_name, server_version_major, server_version_minor, server_version_patch, server_revision);
|
||||||
|
|
||||||
server_version = toString(server_version_major) + "." + toString(server_version_minor) + "." + toString(server_version_patch);
|
server_version = toString(server_version_major) + "." + toString(server_version_minor) + "." + toString(server_version_patch);
|
||||||
|
|
||||||
if (server_display_name = connection->getServerDisplayName(); server_display_name.length() == 0)
|
if (
|
||||||
|
server_display_name = connection->getServerDisplayName(connection_parameters.timeouts);
|
||||||
|
server_display_name.length() == 0)
|
||||||
{
|
{
|
||||||
server_display_name = config().getString("host", "localhost");
|
server_display_name = config().getString("host", "localhost");
|
||||||
}
|
}
|
||||||
@ -599,7 +614,7 @@ private:
|
|||||||
|
|
||||||
#if USE_READLINE && HAVE_READLINE_HISTORY
|
#if USE_READLINE && HAVE_READLINE_HISTORY
|
||||||
if (!history_file.empty() && append_history(1, history_file.c_str()))
|
if (!history_file.empty() && append_history(1, history_file.c_str()))
|
||||||
throwFromErrno("Cannot append history to file " + history_file, ErrorCodes::CANNOT_APPEND_HISTORY);
|
std::cerr << "Cannot append history to file " + history_file + ": " + errnoToString(ErrorCodes::CANNOT_APPEND_HISTORY);
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
prev_input = input;
|
prev_input = input;
|
||||||
@ -610,9 +625,6 @@ private:
|
|||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
/// Determine the terminal size.
|
|
||||||
ioctl(0, TIOCGWINSZ, &terminal_size);
|
|
||||||
|
|
||||||
if (!process(input))
|
if (!process(input))
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
@ -704,7 +716,7 @@ private:
|
|||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
ASTInsertQuery * insert = typeid_cast<ASTInsertQuery *>(ast.get());
|
auto * insert = ast->as<ASTInsertQuery>();
|
||||||
|
|
||||||
if (insert && insert->data)
|
if (insert && insert->data)
|
||||||
{
|
{
|
||||||
@ -741,7 +753,7 @@ private:
|
|||||||
}
|
}
|
||||||
|
|
||||||
if (!test_hint.checkActual(actual_server_error, actual_client_error, got_exception, last_exception))
|
if (!test_hint.checkActual(actual_server_error, actual_client_error, got_exception, last_exception))
|
||||||
connection->forceConnected();
|
connection->forceConnected(connection_parameters.timeouts);
|
||||||
|
|
||||||
if (got_exception && !ignore_error)
|
if (got_exception && !ignore_error)
|
||||||
{
|
{
|
||||||
@ -799,22 +811,37 @@ private:
|
|||||||
written_progress_chars = 0;
|
written_progress_chars = 0;
|
||||||
written_first_block = false;
|
written_first_block = false;
|
||||||
|
|
||||||
const ASTSetQuery * set_query = typeid_cast<const ASTSetQuery *>(&*parsed_query);
|
{
|
||||||
const ASTUseQuery * use_query = typeid_cast<const ASTUseQuery *>(&*parsed_query);
|
/// Temporarily apply query settings to context.
|
||||||
|
std::optional<Settings> old_settings;
|
||||||
|
SCOPE_EXIT({ if (old_settings) context.setSettings(*old_settings); });
|
||||||
|
auto apply_query_settings = [&](const IAST & settings_ast)
|
||||||
|
{
|
||||||
|
if (!old_settings)
|
||||||
|
old_settings.emplace(context.getSettingsRef());
|
||||||
|
context.applySettingsChanges(settings_ast.as<ASTSetQuery>()->changes);
|
||||||
|
};
|
||||||
|
const auto * insert = parsed_query->as<ASTInsertQuery>();
|
||||||
|
if (insert && insert->settings_ast)
|
||||||
|
apply_query_settings(*insert->settings_ast);
|
||||||
|
/// FIXME: try to prettify this cast using `as<>()`
|
||||||
|
const auto * with_output = dynamic_cast<const ASTQueryWithOutput *>(parsed_query.get());
|
||||||
|
if (with_output && with_output->settings_ast)
|
||||||
|
apply_query_settings(*with_output->settings_ast);
|
||||||
|
|
||||||
|
connection->forceConnected(connection_parameters.timeouts);
|
||||||
|
|
||||||
/// INSERT query for which data transfer is needed (not an INSERT SELECT) is processed separately.
|
/// INSERT query for which data transfer is needed (not an INSERT SELECT) is processed separately.
|
||||||
const ASTInsertQuery * insert = typeid_cast<const ASTInsertQuery *>(&*parsed_query);
|
|
||||||
|
|
||||||
connection->forceConnected();
|
|
||||||
|
|
||||||
if (insert && !insert->select)
|
if (insert && !insert->select)
|
||||||
processInsertQuery();
|
processInsertQuery();
|
||||||
else
|
else
|
||||||
processOrdinaryQuery();
|
processOrdinaryQuery();
|
||||||
|
}
|
||||||
|
|
||||||
/// Do not change context (current DB, settings) in case of an exception.
|
/// Do not change context (current DB, settings) in case of an exception.
|
||||||
if (!got_exception)
|
if (!got_exception)
|
||||||
{
|
{
|
||||||
if (set_query)
|
if (const auto * set_query = parsed_query->as<ASTSetQuery>())
|
||||||
{
|
{
|
||||||
/// Save all changes in settings to avoid losing them if the connection is lost.
|
/// Save all changes in settings to avoid losing them if the connection is lost.
|
||||||
for (const auto & change : set_query->changes)
|
for (const auto & change : set_query->changes)
|
||||||
@ -822,11 +849,11 @@ private:
|
|||||||
if (change.name == "profile")
|
if (change.name == "profile")
|
||||||
current_profile = change.value.safeGet<String>();
|
current_profile = change.value.safeGet<String>();
|
||||||
else
|
else
|
||||||
context.setSetting(change.name, change.value);
|
context.applySettingChange(change);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if (use_query)
|
if (const auto * use_query = parsed_query->as<ASTUseQuery>())
|
||||||
{
|
{
|
||||||
const String & new_database = use_query->database;
|
const String & new_database = use_query->database;
|
||||||
/// If the client initiates the reconnection, it takes the settings from the config.
|
/// If the client initiates the reconnection, it takes the settings from the config.
|
||||||
@ -841,7 +868,7 @@ private:
|
|||||||
std::cout << std::endl
|
std::cout << std::endl
|
||||||
<< processed_rows << " rows in set. Elapsed: " << watch.elapsedSeconds() << " sec. ";
|
<< processed_rows << " rows in set. Elapsed: " << watch.elapsedSeconds() << " sec. ";
|
||||||
|
|
||||||
if (progress.rows >= 1000)
|
if (progress.read_rows >= 1000)
|
||||||
writeFinalProgress();
|
writeFinalProgress();
|
||||||
|
|
||||||
std::cout << std::endl << std::endl;
|
std::cout << std::endl << std::endl;
|
||||||
@ -858,7 +885,7 @@ private:
|
|||||||
/// Convert external tables to ExternalTableData and send them using the connection.
|
/// Convert external tables to ExternalTableData and send them using the connection.
|
||||||
void sendExternalTables()
|
void sendExternalTables()
|
||||||
{
|
{
|
||||||
auto * select = typeid_cast<const ASTSelectWithUnionQuery *>(&*parsed_query);
|
const auto * select = parsed_query->as<ASTSelectWithUnionQuery>();
|
||||||
if (!select && !external_tables.empty())
|
if (!select && !external_tables.empty())
|
||||||
throw Exception("External tables could be sent only with select query", ErrorCodes::BAD_ARGUMENTS);
|
throw Exception("External tables could be sent only with select query", ErrorCodes::BAD_ARGUMENTS);
|
||||||
|
|
||||||
@ -873,7 +900,7 @@ private:
|
|||||||
/// Process the query that doesn't require transferring data blocks to the server.
|
/// Process the query that doesn't require transferring data blocks to the server.
|
||||||
void processOrdinaryQuery()
|
void processOrdinaryQuery()
|
||||||
{
|
{
|
||||||
connection->sendQuery(query, query_id, QueryProcessingStage::Complete, &context.getSettingsRef(), nullptr, true);
|
connection->sendQuery(connection_parameters.timeouts, query, query_id, QueryProcessingStage::Complete, &context.getSettingsRef(), nullptr, true);
|
||||||
sendExternalTables();
|
sendExternalTables();
|
||||||
receiveResult();
|
receiveResult();
|
||||||
}
|
}
|
||||||
@ -883,7 +910,7 @@ private:
|
|||||||
void processInsertQuery()
|
void processInsertQuery()
|
||||||
{
|
{
|
||||||
/// Send part of query without data, because data will be sent separately.
|
/// Send part of query without data, because data will be sent separately.
|
||||||
const ASTInsertQuery & parsed_insert_query = typeid_cast<const ASTInsertQuery &>(*parsed_query);
|
const auto & parsed_insert_query = parsed_query->as<ASTInsertQuery &>();
|
||||||
String query_without_data = parsed_insert_query.data
|
String query_without_data = parsed_insert_query.data
|
||||||
? query.substr(0, parsed_insert_query.data - query.data())
|
? query.substr(0, parsed_insert_query.data - query.data())
|
||||||
: query;
|
: query;
|
||||||
@ -891,7 +918,7 @@ private:
|
|||||||
if (!parsed_insert_query.data && (is_interactive || (stdin_is_not_tty && std_in.eof())))
|
if (!parsed_insert_query.data && (is_interactive || (stdin_is_not_tty && std_in.eof())))
|
||||||
throw Exception("No data to insert", ErrorCodes::NO_DATA_TO_INSERT);
|
throw Exception("No data to insert", ErrorCodes::NO_DATA_TO_INSERT);
|
||||||
|
|
||||||
connection->sendQuery(query_without_data, query_id, QueryProcessingStage::Complete, &context.getSettingsRef(), nullptr, true);
|
connection->sendQuery(connection_parameters.timeouts, query_without_data, query_id, QueryProcessingStage::Complete, &context.getSettingsRef(), nullptr, true);
|
||||||
sendExternalTables();
|
sendExternalTables();
|
||||||
|
|
||||||
/// Receive description of table structure.
|
/// Receive description of table structure.
|
||||||
@ -940,7 +967,7 @@ private:
|
|||||||
void sendData(Block & sample, const ColumnsDescription & columns_description)
|
void sendData(Block & sample, const ColumnsDescription & columns_description)
|
||||||
{
|
{
|
||||||
/// If INSERT data must be sent.
|
/// If INSERT data must be sent.
|
||||||
const ASTInsertQuery * parsed_insert_query = typeid_cast<const ASTInsertQuery *>(&*parsed_query);
|
const auto * parsed_insert_query = parsed_query->as<ASTInsertQuery>();
|
||||||
if (!parsed_insert_query)
|
if (!parsed_insert_query)
|
||||||
return;
|
return;
|
||||||
|
|
||||||
@ -965,18 +992,16 @@ private:
|
|||||||
String current_format = insert_format;
|
String current_format = insert_format;
|
||||||
|
|
||||||
/// Data format can be specified in the INSERT query.
|
/// Data format can be specified in the INSERT query.
|
||||||
if (ASTInsertQuery * insert = typeid_cast<ASTInsertQuery *>(&*parsed_query))
|
if (const auto * insert = parsed_query->as<ASTInsertQuery>())
|
||||||
{
|
{
|
||||||
if (!insert->format.empty())
|
if (!insert->format.empty())
|
||||||
current_format = insert->format;
|
current_format = insert->format;
|
||||||
if (insert->settings_ast)
|
|
||||||
InterpreterSetQuery(insert->settings_ast, context).executeForCurrentContext();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
BlockInputStreamPtr block_input = context.getInputFormat(
|
BlockInputStreamPtr block_input = context.getInputFormat(
|
||||||
current_format, buf, sample, insert_format_max_block_size);
|
current_format, buf, sample, insert_format_max_block_size);
|
||||||
|
|
||||||
const auto & column_defaults = columns_description.defaults;
|
const auto & column_defaults = columns_description.getDefaults();
|
||||||
if (!column_defaults.empty())
|
if (!column_defaults.empty())
|
||||||
block_input = std::make_shared<AddingDefaultsBlockInputStream>(block_input, column_defaults, context);
|
block_input = std::make_shared<AddingDefaultsBlockInputStream>(block_input, column_defaults, context);
|
||||||
|
|
||||||
@ -1040,7 +1065,7 @@ private:
|
|||||||
bool cancelled = false;
|
bool cancelled = false;
|
||||||
|
|
||||||
// TODO: get the poll_interval from commandline.
|
// TODO: get the poll_interval from commandline.
|
||||||
const auto receive_timeout = connection->getTimeouts().receive_timeout;
|
const auto receive_timeout = connection_parameters.timeouts.receive_timeout;
|
||||||
constexpr size_t default_poll_interval = 1000000; /// in microseconds
|
constexpr size_t default_poll_interval = 1000000; /// in microseconds
|
||||||
constexpr size_t min_poll_interval = 5000; /// in microseconds
|
constexpr size_t min_poll_interval = 5000; /// in microseconds
|
||||||
const size_t poll_interval
|
const size_t poll_interval
|
||||||
@ -1231,12 +1256,14 @@ private:
|
|||||||
String current_format = format;
|
String current_format = format;
|
||||||
|
|
||||||
/// The query can specify output format or output file.
|
/// The query can specify output format or output file.
|
||||||
if (ASTQueryWithOutput * query_with_output = dynamic_cast<ASTQueryWithOutput *>(&*parsed_query))
|
/// FIXME: try to prettify this cast using `as<>()`
|
||||||
|
if (const auto * query_with_output = dynamic_cast<const ASTQueryWithOutput *>(parsed_query.get()))
|
||||||
{
|
{
|
||||||
if (query_with_output->out_file != nullptr)
|
if (query_with_output->out_file)
|
||||||
{
|
{
|
||||||
const auto & out_file_node = typeid_cast<const ASTLiteral &>(*query_with_output->out_file);
|
const auto & out_file_node = query_with_output->out_file->as<ASTLiteral &>();
|
||||||
const auto & out_file = out_file_node.value.safeGet<std::string>();
|
const auto & out_file = out_file_node.value.safeGet<std::string>();
|
||||||
|
|
||||||
out_file_buf.emplace(out_file, DBMS_DEFAULT_BUFFER_SIZE, O_WRONLY | O_EXCL | O_CREAT);
|
out_file_buf.emplace(out_file, DBMS_DEFAULT_BUFFER_SIZE, O_WRONLY | O_EXCL | O_CREAT);
|
||||||
out_buf = &*out_file_buf;
|
out_buf = &*out_file_buf;
|
||||||
|
|
||||||
@ -1248,13 +1275,9 @@ private:
|
|||||||
{
|
{
|
||||||
if (has_vertical_output_suffix)
|
if (has_vertical_output_suffix)
|
||||||
throw Exception("Output format already specified", ErrorCodes::CLIENT_OUTPUT_FORMAT_SPECIFIED);
|
throw Exception("Output format already specified", ErrorCodes::CLIENT_OUTPUT_FORMAT_SPECIFIED);
|
||||||
const auto & id = typeid_cast<const ASTIdentifier &>(*query_with_output->format);
|
const auto & id = query_with_output->format->as<ASTIdentifier &>();
|
||||||
current_format = id.name;
|
current_format = id.name;
|
||||||
}
|
}
|
||||||
if (query_with_output->settings_ast)
|
|
||||||
{
|
|
||||||
InterpreterSetQuery(query_with_output->settings_ast, context).executeForCurrentContext();
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if (has_vertical_output_suffix)
|
if (has_vertical_output_suffix)
|
||||||
@ -1318,6 +1341,9 @@ private:
|
|||||||
|
|
||||||
/// Received data block is immediately displayed to the user.
|
/// Received data block is immediately displayed to the user.
|
||||||
block_out_stream->flush();
|
block_out_stream->flush();
|
||||||
|
|
||||||
|
/// Restore progress bar after data block.
|
||||||
|
writeProgress();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@ -1357,8 +1383,8 @@ private:
|
|||||||
|
|
||||||
void clearProgress()
|
void clearProgress()
|
||||||
{
|
{
|
||||||
std::cerr << RESTORE_CURSOR_POSITION CLEAR_TO_END_OF_LINE;
|
|
||||||
written_progress_chars = 0;
|
written_progress_chars = 0;
|
||||||
|
std::cerr << RESTORE_CURSOR_POSITION CLEAR_TO_END_OF_LINE;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@ -1367,6 +1393,9 @@ private:
|
|||||||
if (!need_render_progress)
|
if (!need_render_progress)
|
||||||
return;
|
return;
|
||||||
|
|
||||||
|
/// Output all progress bar commands to stderr at once to avoid flicker.
|
||||||
|
WriteBufferFromFileDescriptor message(STDERR_FILENO, 1024);
|
||||||
|
|
||||||
static size_t increment = 0;
|
static size_t increment = 0;
|
||||||
static const char * indicators[8] =
|
static const char * indicators[8] =
|
||||||
{
|
{
|
||||||
@ -1381,34 +1410,35 @@ private:
|
|||||||
};
|
};
|
||||||
|
|
||||||
if (written_progress_chars)
|
if (written_progress_chars)
|
||||||
clearProgress();
|
message << RESTORE_CURSOR_POSITION CLEAR_TO_END_OF_LINE;
|
||||||
else
|
else
|
||||||
std::cerr << SAVE_CURSOR_POSITION;
|
message << SAVE_CURSOR_POSITION;
|
||||||
|
|
||||||
|
message << DISABLE_LINE_WRAPPING;
|
||||||
|
|
||||||
|
size_t prefix_size = message.count();
|
||||||
|
|
||||||
std::stringstream message;
|
|
||||||
message << indicators[increment % 8]
|
message << indicators[increment % 8]
|
||||||
<< std::fixed << std::setprecision(3)
|
|
||||||
<< " Progress: ";
|
<< " Progress: ";
|
||||||
|
|
||||||
message
|
message
|
||||||
<< formatReadableQuantity(progress.rows) << " rows, "
|
<< formatReadableQuantity(progress.read_rows) << " rows, "
|
||||||
<< formatReadableSizeWithDecimalSuffix(progress.bytes);
|
<< formatReadableSizeWithDecimalSuffix(progress.read_bytes);
|
||||||
|
|
||||||
size_t elapsed_ns = watch.elapsed();
|
size_t elapsed_ns = watch.elapsed();
|
||||||
if (elapsed_ns)
|
if (elapsed_ns)
|
||||||
message << " ("
|
message << " ("
|
||||||
<< formatReadableQuantity(progress.rows * 1000000000.0 / elapsed_ns) << " rows/s., "
|
<< formatReadableQuantity(progress.read_rows * 1000000000.0 / elapsed_ns) << " rows/s., "
|
||||||
<< formatReadableSizeWithDecimalSuffix(progress.bytes * 1000000000.0 / elapsed_ns) << "/s.) ";
|
<< formatReadableSizeWithDecimalSuffix(progress.read_bytes * 1000000000.0 / elapsed_ns) << "/s.) ";
|
||||||
else
|
else
|
||||||
message << ". ";
|
message << ". ";
|
||||||
|
|
||||||
written_progress_chars = message.str().size() - (increment % 8 == 7 ? 10 : 13);
|
written_progress_chars = message.count() - prefix_size - (increment % 8 == 7 ? 10 : 13); /// Don't count invisible output (escape sequences).
|
||||||
std::cerr << DISABLE_LINE_WRAPPING << message.rdbuf();
|
|
||||||
|
|
||||||
/// If the approximate number of rows to process is known, we can display a progress bar and percentage.
|
/// If the approximate number of rows to process is known, we can display a progress bar and percentage.
|
||||||
if (progress.total_rows > 0)
|
if (progress.total_rows_to_read > 0)
|
||||||
{
|
{
|
||||||
size_t total_rows_corrected = std::max(progress.rows, progress.total_rows);
|
size_t total_rows_corrected = std::max(progress.read_rows, progress.total_rows_to_read);
|
||||||
|
|
||||||
/// To avoid flicker, display progress bar only if .5 seconds have passed since query execution start
|
/// To avoid flicker, display progress bar only if .5 seconds have passed since query execution start
|
||||||
/// and the query is less than halfway done.
|
/// and the query is less than halfway done.
|
||||||
@ -1416,7 +1446,7 @@ private:
|
|||||||
if (elapsed_ns > 500000000)
|
if (elapsed_ns > 500000000)
|
||||||
{
|
{
|
||||||
/// Trigger to start displaying progress bar. If query is mostly done, don't display it.
|
/// Trigger to start displaying progress bar. If query is mostly done, don't display it.
|
||||||
if (progress.rows * 2 < total_rows_corrected)
|
if (progress.read_rows * 2 < total_rows_corrected)
|
||||||
show_progress_bar = true;
|
show_progress_bar = true;
|
||||||
|
|
||||||
if (show_progress_bar)
|
if (show_progress_bar)
|
||||||
@ -1424,34 +1454,36 @@ private:
|
|||||||
ssize_t width_of_progress_bar = static_cast<ssize_t>(terminal_size.ws_col) - written_progress_chars - strlen(" 99%");
|
ssize_t width_of_progress_bar = static_cast<ssize_t>(terminal_size.ws_col) - written_progress_chars - strlen(" 99%");
|
||||||
if (width_of_progress_bar > 0)
|
if (width_of_progress_bar > 0)
|
||||||
{
|
{
|
||||||
std::string bar = UnicodeBar::render(UnicodeBar::getWidth(progress.rows, 0, total_rows_corrected, width_of_progress_bar));
|
std::string bar = UnicodeBar::render(UnicodeBar::getWidth(progress.read_rows, 0, total_rows_corrected, width_of_progress_bar));
|
||||||
std::cerr << "\033[0;32m" << bar << "\033[0m";
|
message << "\033[0;32m" << bar << "\033[0m";
|
||||||
if (width_of_progress_bar > static_cast<ssize_t>(bar.size() / UNICODE_BAR_CHAR_SIZE))
|
if (width_of_progress_bar > static_cast<ssize_t>(bar.size() / UNICODE_BAR_CHAR_SIZE))
|
||||||
std::cerr << std::string(width_of_progress_bar - bar.size() / UNICODE_BAR_CHAR_SIZE, ' ');
|
message << std::string(width_of_progress_bar - bar.size() / UNICODE_BAR_CHAR_SIZE, ' ');
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Underestimate percentage a bit to avoid displaying 100%.
|
/// Underestimate percentage a bit to avoid displaying 100%.
|
||||||
std::cerr << ' ' << (99 * progress.rows / total_rows_corrected) << '%';
|
message << ' ' << (99 * progress.read_rows / total_rows_corrected) << '%';
|
||||||
}
|
}
|
||||||
|
|
||||||
std::cerr << ENABLE_LINE_WRAPPING;
|
message << ENABLE_LINE_WRAPPING;
|
||||||
++increment;
|
++increment;
|
||||||
|
|
||||||
|
message.next();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
void writeFinalProgress()
|
void writeFinalProgress()
|
||||||
{
|
{
|
||||||
std::cout << "Processed "
|
std::cout << "Processed "
|
||||||
<< formatReadableQuantity(progress.rows) << " rows, "
|
<< formatReadableQuantity(progress.read_rows) << " rows, "
|
||||||
<< formatReadableSizeWithDecimalSuffix(progress.bytes);
|
<< formatReadableSizeWithDecimalSuffix(progress.read_bytes);
|
||||||
|
|
||||||
size_t elapsed_ns = watch.elapsed();
|
size_t elapsed_ns = watch.elapsed();
|
||||||
if (elapsed_ns)
|
if (elapsed_ns)
|
||||||
std::cout << " ("
|
std::cout << " ("
|
||||||
<< formatReadableQuantity(progress.rows * 1000000000.0 / elapsed_ns) << " rows/s., "
|
<< formatReadableQuantity(progress.read_rows * 1000000000.0 / elapsed_ns) << " rows/s., "
|
||||||
<< formatReadableSizeWithDecimalSuffix(progress.bytes * 1000000000.0 / elapsed_ns) << "/s.) ";
|
<< formatReadableSizeWithDecimalSuffix(progress.read_bytes * 1000000000.0 / elapsed_ns) << "/s.) ";
|
||||||
else
|
else
|
||||||
std::cout << ". ";
|
std::cout << ". ";
|
||||||
}
|
}
|
||||||
@ -1504,7 +1536,7 @@ private:
|
|||||||
|
|
||||||
void showClientVersion()
|
void showClientVersion()
|
||||||
{
|
{
|
||||||
std::cout << DBMS_NAME << " client version " << VERSION_STRING << "." << std::endl;
|
std::cout << DBMS_NAME << " client version " << VERSION_STRING << VERSION_OFFICIAL << "." << std::endl;
|
||||||
}
|
}
|
||||||
|
|
||||||
public:
|
public:
|
||||||
@ -1519,7 +1551,7 @@ public:
|
|||||||
* where possible args are file, name, format, structure, types.
|
* where possible args are file, name, format, structure, types.
|
||||||
* Split these groups before processing.
|
* Split these groups before processing.
|
||||||
*/
|
*/
|
||||||
using Arguments = std::vector<const char *>;
|
using Arguments = std::vector<std::string>;
|
||||||
|
|
||||||
Arguments common_arguments{""}; /// 0th argument is ignored.
|
Arguments common_arguments{""}; /// 0th argument is ignored.
|
||||||
std::vector<Arguments> external_tables_arguments;
|
std::vector<Arguments> external_tables_arguments;
|
||||||
@ -1569,7 +1601,7 @@ public:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
ioctl(0, TIOCGWINSZ, &terminal_size);
|
stdin_is_not_tty = !isatty(STDIN_FILENO);
|
||||||
|
|
||||||
namespace po = boost::program_options;
|
namespace po = boost::program_options;
|
||||||
|
|
||||||
@ -1577,12 +1609,14 @@ public:
|
|||||||
unsigned min_description_length = line_length / 2;
|
unsigned min_description_length = line_length / 2;
|
||||||
if (!stdin_is_not_tty)
|
if (!stdin_is_not_tty)
|
||||||
{
|
{
|
||||||
line_length = std::max(3U, static_cast<unsigned>(terminal_size.ws_col));
|
if (ioctl(STDIN_FILENO, TIOCGWINSZ, &terminal_size))
|
||||||
|
throwFromErrno("Cannot obtain terminal window size (ioctl TIOCGWINSZ)", ErrorCodes::SYSTEM_ERROR);
|
||||||
|
line_length = std::max(
|
||||||
|
static_cast<unsigned>(strlen("--http_native_compression_disable_checksumming_on_decompress ")),
|
||||||
|
static_cast<unsigned>(terminal_size.ws_col));
|
||||||
min_description_length = std::min(min_description_length, line_length - 2);
|
min_description_length = std::min(min_description_length, line_length - 2);
|
||||||
}
|
}
|
||||||
|
|
||||||
#define DECLARE_SETTING(TYPE, NAME, DEFAULT, DESCRIPTION) (#NAME, po::value<std::string>(), DESCRIPTION)
|
|
||||||
|
|
||||||
/// Main commandline options related to client functionality and all parameters from Settings.
|
/// Main commandline options related to client functionality and all parameters from Settings.
|
||||||
po::options_description main_description("Main options", line_length, min_description_length);
|
po::options_description main_description("Main options", line_length, min_description_length);
|
||||||
main_description.add_options()
|
main_description.add_options()
|
||||||
@ -1606,6 +1640,7 @@ public:
|
|||||||
("database,d", po::value<std::string>(), "database")
|
("database,d", po::value<std::string>(), "database")
|
||||||
("pager", po::value<std::string>(), "pager")
|
("pager", po::value<std::string>(), "pager")
|
||||||
("disable_suggestion,A", "Disable loading suggestion data. Note that suggestion data is loaded asynchronously through a second connection to ClickHouse server. Also it is reasonable to disable suggestion if you want to paste a query with TAB characters. Shorthand option -A is for those who get used to mysql client.")
|
("disable_suggestion,A", "Disable loading suggestion data. Note that suggestion data is loaded asynchronously through a second connection to ClickHouse server. Also it is reasonable to disable suggestion if you want to paste a query with TAB characters. Shorthand option -A is for those who get used to mysql client.")
|
||||||
|
("always_load_suggestion_data", "Load suggestion data even if clickhouse-client is run in non-interactive mode. Used for testing.")
|
||||||
("suggestion_limit", po::value<int>()->default_value(10000),
|
("suggestion_limit", po::value<int>()->default_value(10000),
|
||||||
"Suggestion limit for how many databases, tables and columns to fetch.")
|
"Suggestion limit for how many databases, tables and columns to fetch.")
|
||||||
("multiline,m", "multiline")
|
("multiline,m", "multiline")
|
||||||
@ -1624,9 +1659,9 @@ public:
|
|||||||
("compression", po::value<bool>(), "enable or disable compression")
|
("compression", po::value<bool>(), "enable or disable compression")
|
||||||
("log-level", po::value<std::string>(), "client log level")
|
("log-level", po::value<std::string>(), "client log level")
|
||||||
("server_logs_file", po::value<std::string>(), "put server logs into specified file")
|
("server_logs_file", po::value<std::string>(), "put server logs into specified file")
|
||||||
APPLY_FOR_SETTINGS(DECLARE_SETTING)
|
|
||||||
;
|
;
|
||||||
#undef DECLARE_SETTING
|
|
||||||
|
context.getSettingsRef().addProgramOptions(main_description);
|
||||||
|
|
||||||
/// Commandline options related to external tables.
|
/// Commandline options related to external tables.
|
||||||
po::options_description external_description("External tables options");
|
po::options_description external_description("External tables options");
|
||||||
@ -1638,10 +1673,11 @@ public:
|
|||||||
("types", po::value<std::string>(), "types")
|
("types", po::value<std::string>(), "types")
|
||||||
;
|
;
|
||||||
/// Parse main commandline options.
|
/// Parse main commandline options.
|
||||||
po::parsed_options parsed = po::command_line_parser(
|
po::parsed_options parsed = po::command_line_parser(common_arguments).options(main_description).run();
|
||||||
common_arguments.size(), common_arguments.data()).options(main_description).run();
|
|
||||||
po::variables_map options;
|
po::variables_map options;
|
||||||
po::store(parsed, options);
|
po::store(parsed, options);
|
||||||
|
po::notify(options);
|
||||||
|
|
||||||
if (options.count("version") || options.count("V"))
|
if (options.count("version") || options.count("V"))
|
||||||
{
|
{
|
||||||
showClientVersion();
|
showClientVersion();
|
||||||
@ -1670,8 +1706,7 @@ public:
|
|||||||
for (size_t i = 0; i < external_tables_arguments.size(); ++i)
|
for (size_t i = 0; i < external_tables_arguments.size(); ++i)
|
||||||
{
|
{
|
||||||
/// Parse commandline options related to external tables.
|
/// Parse commandline options related to external tables.
|
||||||
po::parsed_options parsed_tables = po::command_line_parser(
|
po::parsed_options parsed_tables = po::command_line_parser(external_tables_arguments[i]).options(external_description).run();
|
||||||
external_tables_arguments[i].size(), external_tables_arguments[i].data()).options(external_description).run();
|
|
||||||
po::variables_map external_options;
|
po::variables_map external_options;
|
||||||
po::store(parsed_tables, external_options);
|
po::store(parsed_tables, external_options);
|
||||||
|
|
||||||
@ -1692,15 +1727,14 @@ public:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Extract settings from the options.
|
/// Copy settings-related program options to config.
|
||||||
#define EXTRACT_SETTING(TYPE, NAME, DEFAULT, DESCRIPTION) \
|
/// TODO: Is this code necessary?
|
||||||
if (options.count(#NAME)) \
|
for (const auto & setting : context.getSettingsRef())
|
||||||
{ \
|
{
|
||||||
context.setSetting(#NAME, options[#NAME].as<std::string>()); \
|
const String name = setting.getName().toString();
|
||||||
config().setString(#NAME, options[#NAME].as<std::string>()); \
|
if (options.count(name))
|
||||||
|
config().setString(name, options[name].as<std::string>());
|
||||||
}
|
}
|
||||||
APPLY_FOR_SETTINGS(EXTRACT_SETTING)
|
|
||||||
#undef EXTRACT_SETTING
|
|
||||||
|
|
||||||
if (options.count("config-file") && options.count("config"))
|
if (options.count("config-file") && options.count("config"))
|
||||||
throw Exception("Two or more configuration files referenced in arguments", ErrorCodes::BAD_ARGUMENTS);
|
throw Exception("Two or more configuration files referenced in arguments", ErrorCodes::BAD_ARGUMENTS);
|
||||||
@ -1759,8 +1793,18 @@ public:
|
|||||||
server_logs_file = options["server_logs_file"].as<std::string>();
|
server_logs_file = options["server_logs_file"].as<std::string>();
|
||||||
if (options.count("disable_suggestion"))
|
if (options.count("disable_suggestion"))
|
||||||
config().setBool("disable_suggestion", true);
|
config().setBool("disable_suggestion", true);
|
||||||
|
if (options.count("always_load_suggestion_data"))
|
||||||
|
{
|
||||||
|
if (options.count("disable_suggestion"))
|
||||||
|
throw Exception("Command line parameters disable_suggestion (-A) and always_load_suggestion_data cannot be specified simultaneously",
|
||||||
|
ErrorCodes::BAD_ARGUMENTS);
|
||||||
|
config().setBool("always_load_suggestion_data", true);
|
||||||
|
}
|
||||||
if (options.count("suggestion_limit"))
|
if (options.count("suggestion_limit"))
|
||||||
config().setInt("suggestion_limit", options["suggestion_limit"].as<int>());
|
config().setInt("suggestion_limit", options["suggestion_limit"].as<int>());
|
||||||
|
|
||||||
|
argsToConfig(common_arguments, config(), 100);
|
||||||
|
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
63
dbms/programs/client/ConnectionParameters.cpp
Normal file
63
dbms/programs/client/ConnectionParameters.cpp
Normal file
@ -0,0 +1,63 @@
|
|||||||
|
#include "ConnectionParameters.h"
|
||||||
|
#include <fstream>
|
||||||
|
#include <iostream>
|
||||||
|
#include <Core/Defines.h>
|
||||||
|
#include <Core/Protocol.h>
|
||||||
|
#include <Core/Types.h>
|
||||||
|
#include <IO/ConnectionTimeouts.h>
|
||||||
|
#include <Poco/Util/AbstractConfiguration.h>
|
||||||
|
#include <Common/Exception.h>
|
||||||
|
#include <common/setTerminalEcho.h>
|
||||||
|
#include <ext/scope_guard.h>
|
||||||
|
#include <readpassphrase.h>
|
||||||
|
|
||||||
|
namespace DB
|
||||||
|
{
|
||||||
|
|
||||||
|
namespace ErrorCodes
|
||||||
|
{
|
||||||
|
extern const int BAD_ARGUMENTS;
|
||||||
|
}
|
||||||
|
|
||||||
|
ConnectionParameters::ConnectionParameters(const Poco::Util::AbstractConfiguration & config)
|
||||||
|
{
|
||||||
|
bool is_secure = config.getBool("secure", false);
|
||||||
|
security = is_secure ? Protocol::Secure::Enable : Protocol::Secure::Disable;
|
||||||
|
|
||||||
|
host = config.getString("host", "localhost");
|
||||||
|
port = config.getInt(
|
||||||
|
"port", config.getInt(is_secure ? "tcp_port_secure" : "tcp_port", is_secure ? DBMS_DEFAULT_SECURE_PORT : DBMS_DEFAULT_PORT));
|
||||||
|
|
||||||
|
default_database = config.getString("database", "");
|
||||||
|
/// changed the default value to "default" to fix the issue when the user in the prompt is blank
|
||||||
|
user = config.getString("user", "default");
|
||||||
|
bool password_prompt = false;
|
||||||
|
if (config.getBool("ask-password", false))
|
||||||
|
{
|
||||||
|
if (config.has("password"))
|
||||||
|
throw Exception("Specified both --password and --ask-password. Remove one of them", ErrorCodes::BAD_ARGUMENTS);
|
||||||
|
password_prompt = true;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
password = config.getString("password", "");
|
||||||
|
/// if the value of --password is omitted, the password will be set implicitly to "\n"
|
||||||
|
if (password == "\n")
|
||||||
|
password_prompt = true;
|
||||||
|
}
|
||||||
|
if (password_prompt)
|
||||||
|
{
|
||||||
|
std::string prompt{"Password for user (" + user + "): "};
|
||||||
|
char buf[1000] = {};
|
||||||
|
if (auto result = readpassphrase(prompt.c_str(), buf, sizeof(buf), 0))
|
||||||
|
password = result;
|
||||||
|
}
|
||||||
|
compression = config.getBool("compression", true) ? Protocol::Compression::Enable : Protocol::Compression::Disable;
|
||||||
|
|
||||||
|
timeouts = ConnectionTimeouts(
|
||||||
|
Poco::Timespan(config.getInt("connect_timeout", DBMS_DEFAULT_CONNECT_TIMEOUT_SEC), 0),
|
||||||
|
Poco::Timespan(config.getInt("send_timeout", DBMS_DEFAULT_SEND_TIMEOUT_SEC), 0),
|
||||||
|
Poco::Timespan(config.getInt("receive_timeout", DBMS_DEFAULT_RECEIVE_TIMEOUT_SEC), 0),
|
||||||
|
Poco::Timespan(config.getInt("tcp_keep_alive_timeout", 0), 0));
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -1,90 +1,30 @@
|
|||||||
#pragma once
|
#pragma once
|
||||||
|
|
||||||
#include <iostream>
|
#include <string>
|
||||||
|
|
||||||
#include <Core/Types.h>
|
|
||||||
#include <Core/Protocol.h>
|
#include <Core/Protocol.h>
|
||||||
#include <Core/Defines.h>
|
|
||||||
#include <Common/Exception.h>
|
|
||||||
#include <IO/ConnectionTimeouts.h>
|
#include <IO/ConnectionTimeouts.h>
|
||||||
|
|
||||||
#include <common/setTerminalEcho.h>
|
namespace Poco::Util
|
||||||
#include <ext/scope_guard.h>
|
{
|
||||||
|
class AbstractConfiguration;
|
||||||
#include <Poco/Util/AbstractConfiguration.h>
|
}
|
||||||
|
|
||||||
|
|
||||||
namespace DB
|
namespace DB
|
||||||
{
|
{
|
||||||
|
|
||||||
namespace ErrorCodes
|
|
||||||
{
|
|
||||||
extern const int BAD_ARGUMENTS;
|
|
||||||
}
|
|
||||||
|
|
||||||
struct ConnectionParameters
|
struct ConnectionParameters
|
||||||
{
|
{
|
||||||
String host;
|
std::string host;
|
||||||
UInt16 port{};
|
UInt16 port{};
|
||||||
String default_database;
|
std::string default_database;
|
||||||
String user;
|
std::string user;
|
||||||
String password;
|
std::string password;
|
||||||
Protocol::Secure security = Protocol::Secure::Disable;
|
Protocol::Secure security = Protocol::Secure::Disable;
|
||||||
Protocol::Compression compression = Protocol::Compression::Enable;
|
Protocol::Compression compression = Protocol::Compression::Enable;
|
||||||
ConnectionTimeouts timeouts;
|
ConnectionTimeouts timeouts;
|
||||||
|
|
||||||
ConnectionParameters() {}
|
ConnectionParameters() {}
|
||||||
|
|
||||||
ConnectionParameters(const Poco::Util::AbstractConfiguration & config)
|
ConnectionParameters(const Poco::Util::AbstractConfiguration & config);
|
||||||
{
|
|
||||||
bool is_secure = config.getBool("secure", false);
|
|
||||||
security = is_secure
|
|
||||||
? Protocol::Secure::Enable
|
|
||||||
: Protocol::Secure::Disable;
|
|
||||||
|
|
||||||
host = config.getString("host", "localhost");
|
|
||||||
port = config.getInt("port",
|
|
||||||
config.getInt(is_secure ? "tcp_port_secure" : "tcp_port",
|
|
||||||
is_secure ? DBMS_DEFAULT_SECURE_PORT : DBMS_DEFAULT_PORT));
|
|
||||||
|
|
||||||
default_database = config.getString("database", "");
|
|
||||||
/// changed the default value to "default" to fix the issue when the user in the prompt is blank
|
|
||||||
user = config.getString("user", "default");
|
|
||||||
bool password_prompt = false;
|
|
||||||
if (config.getBool("ask-password", false))
|
|
||||||
{
|
|
||||||
if (config.has("password"))
|
|
||||||
throw Exception("Specified both --password and --ask-password. Remove one of them", ErrorCodes::BAD_ARGUMENTS);
|
|
||||||
password_prompt = true;
|
|
||||||
}
|
|
||||||
else
|
|
||||||
{
|
|
||||||
password = config.getString("password", "");
|
|
||||||
/// if the value of --password is omitted, the password will be set implicitly to "\n"
|
|
||||||
if (password == "\n")
|
|
||||||
password_prompt = true;
|
|
||||||
}
|
|
||||||
if (password_prompt)
|
|
||||||
{
|
|
||||||
std::cout << "Password for user (" << user << "): ";
|
|
||||||
setTerminalEcho(false);
|
|
||||||
|
|
||||||
SCOPE_EXIT({
|
|
||||||
setTerminalEcho(true);
|
|
||||||
});
|
|
||||||
std::getline(std::cin, password);
|
|
||||||
std::cout << std::endl;
|
|
||||||
}
|
|
||||||
compression = config.getBool("compression", true)
|
|
||||||
? Protocol::Compression::Enable
|
|
||||||
: Protocol::Compression::Disable;
|
|
||||||
|
|
||||||
timeouts = ConnectionTimeouts(
|
|
||||||
Poco::Timespan(config.getInt("connect_timeout", DBMS_DEFAULT_CONNECT_TIMEOUT_SEC), 0),
|
|
||||||
Poco::Timespan(config.getInt("send_timeout", DBMS_DEFAULT_SEND_TIMEOUT_SEC), 0),
|
|
||||||
Poco::Timespan(config.getInt("receive_timeout", DBMS_DEFAULT_RECEIVE_TIMEOUT_SEC), 0),
|
|
||||||
Poco::Timespan(config.getInt("tcp_keep_alive_timeout", 0), 0));
|
|
||||||
}
|
|
||||||
};
|
};
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|||||||
@ -14,6 +14,7 @@
|
|||||||
#include <Common/typeid_cast.h>
|
#include <Common/typeid_cast.h>
|
||||||
#include <Columns/ColumnString.h>
|
#include <Columns/ColumnString.h>
|
||||||
#include <Client/Connection.h>
|
#include <Client/Connection.h>
|
||||||
|
#include <IO/ConnectionTimeouts.h>
|
||||||
|
|
||||||
|
|
||||||
namespace DB
|
namespace DB
|
||||||
@ -39,10 +40,10 @@ private:
|
|||||||
"DATABASES", "LIKE", "PROCESSLIST", "CASE", "WHEN", "THEN", "ELSE", "END", "DESCRIBE", "DESC", "USE", "SET", "OPTIMIZE", "FINAL", "DEDUPLICATE",
|
"DATABASES", "LIKE", "PROCESSLIST", "CASE", "WHEN", "THEN", "ELSE", "END", "DESCRIBE", "DESC", "USE", "SET", "OPTIMIZE", "FINAL", "DEDUPLICATE",
|
||||||
"INSERT", "VALUES", "SELECT", "DISTINCT", "SAMPLE", "ARRAY", "JOIN", "GLOBAL", "LOCAL", "ANY", "ALL", "INNER", "LEFT", "RIGHT", "FULL", "OUTER",
|
"INSERT", "VALUES", "SELECT", "DISTINCT", "SAMPLE", "ARRAY", "JOIN", "GLOBAL", "LOCAL", "ANY", "ALL", "INNER", "LEFT", "RIGHT", "FULL", "OUTER",
|
||||||
"CROSS", "USING", "PREWHERE", "WHERE", "GROUP", "BY", "WITH", "TOTALS", "HAVING", "ORDER", "COLLATE", "LIMIT", "UNION", "AND", "OR", "ASC", "IN",
|
"CROSS", "USING", "PREWHERE", "WHERE", "GROUP", "BY", "WITH", "TOTALS", "HAVING", "ORDER", "COLLATE", "LIMIT", "UNION", "AND", "OR", "ASC", "IN",
|
||||||
"KILL", "QUERY", "SYNC", "ASYNC", "TEST", "BETWEEN"
|
"KILL", "QUERY", "SYNC", "ASYNC", "TEST", "BETWEEN", "TRUNCATE"
|
||||||
};
|
};
|
||||||
|
|
||||||
/// Words are fetched asynchonously.
|
/// Words are fetched asynchronously.
|
||||||
std::thread loading_thread;
|
std::thread loading_thread;
|
||||||
std::atomic<bool> ready{false};
|
std::atomic<bool> ready{false};
|
||||||
|
|
||||||
@ -71,7 +72,7 @@ private:
|
|||||||
return word;
|
return word;
|
||||||
}
|
}
|
||||||
|
|
||||||
void loadImpl(Connection & connection, size_t suggestion_limit)
|
void loadImpl(Connection & connection, const ConnectionTimeouts & timeouts, size_t suggestion_limit)
|
||||||
{
|
{
|
||||||
std::stringstream query;
|
std::stringstream query;
|
||||||
query << "SELECT DISTINCT arrayJoin(extractAll(name, '[\\\\w_]{2,}')) AS res FROM ("
|
query << "SELECT DISTINCT arrayJoin(extractAll(name, '[\\\\w_]{2,}')) AS res FROM ("
|
||||||
@ -104,12 +105,12 @@ private:
|
|||||||
|
|
||||||
query << ") WHERE notEmpty(res)";
|
query << ") WHERE notEmpty(res)";
|
||||||
|
|
||||||
fetch(connection, query.str());
|
fetch(connection, timeouts, query.str());
|
||||||
}
|
}
|
||||||
|
|
||||||
void fetch(Connection & connection, const std::string & query)
|
void fetch(Connection & connection, const ConnectionTimeouts & timeouts, const std::string & query)
|
||||||
{
|
{
|
||||||
connection.sendQuery(query);
|
connection.sendQuery(timeouts, query);
|
||||||
|
|
||||||
while (true)
|
while (true)
|
||||||
{
|
{
|
||||||
@ -175,12 +176,11 @@ public:
|
|||||||
connection_parameters.default_database,
|
connection_parameters.default_database,
|
||||||
connection_parameters.user,
|
connection_parameters.user,
|
||||||
connection_parameters.password,
|
connection_parameters.password,
|
||||||
connection_parameters.timeouts,
|
|
||||||
"client",
|
"client",
|
||||||
connection_parameters.compression,
|
connection_parameters.compression,
|
||||||
connection_parameters.security);
|
connection_parameters.security);
|
||||||
|
|
||||||
loadImpl(connection, suggestion_limit);
|
loadImpl(connection, connection_parameters.timeouts, suggestion_limit);
|
||||||
}
|
}
|
||||||
catch (...)
|
catch (...)
|
||||||
{
|
{
|
||||||
|
|||||||
3
dbms/programs/client/config_client.h.in
Normal file
3
dbms/programs/client/config_client.h.in
Normal file
@ -0,0 +1,3 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#cmakedefine HAVE_READPASSPHRASE
|
||||||
10
dbms/programs/client/readpassphrase/CMakeLists.txt
Normal file
10
dbms/programs/client/readpassphrase/CMakeLists.txt
Normal file
@ -0,0 +1,10 @@
|
|||||||
|
|
||||||
|
# wget https://raw.githubusercontent.com/openssh/openssh-portable/master/openbsd-compat/readpassphrase.c
|
||||||
|
# wget https://raw.githubusercontent.com/openssh/openssh-portable/master/openbsd-compat/readpassphrase.h
|
||||||
|
|
||||||
|
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-unused-result -Wno-reserved-id-macro")
|
||||||
|
|
||||||
|
configure_file(includes.h.in ${CMAKE_CURRENT_BINARY_DIR}/include/includes.h)
|
||||||
|
add_library(readpassphrase ${CMAKE_CURRENT_SOURCE_DIR}/readpassphrase.c)
|
||||||
|
# . to allow #include <readpassphrase.h>
|
||||||
|
target_include_directories(readpassphrase PUBLIC . ${CMAKE_CURRENT_BINARY_DIR}/include ${CMAKE_CURRENT_BINARY_DIR}/../include)
|
||||||
9
dbms/programs/client/readpassphrase/includes.h.in
Normal file
9
dbms/programs/client/readpassphrase/includes.h.in
Normal file
@ -0,0 +1,9 @@
|
|||||||
|
#pragma once
|
||||||
|
|
||||||
|
#cmakedefine HAVE_READPASSPHRASE
|
||||||
|
|
||||||
|
#if !defined(HAVE_READPASSPHRASE)
|
||||||
|
# ifndef _PATH_TTY
|
||||||
|
# define _PATH_TTY "/dev/tty"
|
||||||
|
# endif
|
||||||
|
#endif
|
||||||
211
dbms/programs/client/readpassphrase/readpassphrase.c
Normal file
211
dbms/programs/client/readpassphrase/readpassphrase.c
Normal file
@ -0,0 +1,211 @@
|
|||||||
|
/* $OpenBSD: readpassphrase.c,v 1.26 2016/10/18 12:47:18 millert Exp $ */
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Copyright (c) 2000-2002, 2007, 2010
|
||||||
|
* Todd C. Miller <Todd.Miller@courtesan.com>
|
||||||
|
*
|
||||||
|
* Permission to use, copy, modify, and distribute this software for any
|
||||||
|
* purpose with or without fee is hereby granted, provided that the above
|
||||||
|
* copyright notice and this permission notice appear in all copies.
|
||||||
|
*
|
||||||
|
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||||
|
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||||
|
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||||
|
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||||
|
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||||
|
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||||
|
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||||
|
*
|
||||||
|
* Sponsored in part by the Defense Advanced Research Projects
|
||||||
|
* Agency (DARPA) and Air Force Research Laboratory, Air Force
|
||||||
|
* Materiel Command, USAF, under agreement number F39502-99-1-0512.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/* OPENBSD ORIGINAL: lib/libc/gen/readpassphrase.c */
|
||||||
|
|
||||||
|
#include "includes.h"
|
||||||
|
|
||||||
|
#ifndef HAVE_READPASSPHRASE
|
||||||
|
|
||||||
|
#include <termios.h>
|
||||||
|
#include <signal.h>
|
||||||
|
#include <ctype.h>
|
||||||
|
#include <fcntl.h>
|
||||||
|
#include <readpassphrase.h>
|
||||||
|
#include <errno.h>
|
||||||
|
#include <string.h>
|
||||||
|
#include <unistd.h>
|
||||||
|
|
||||||
|
#ifndef TCSASOFT
|
||||||
|
/* If we don't have TCSASOFT define it so that ORing it it below is a no-op. */
|
||||||
|
# define TCSASOFT 0
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/* SunOS 4.x which lacks _POSIX_VDISABLE, but has VDISABLE */
|
||||||
|
#if !defined(_POSIX_VDISABLE) && defined(VDISABLE)
|
||||||
|
# define _POSIX_VDISABLE VDISABLE
|
||||||
|
#endif
|
||||||
|
|
||||||
|
static volatile sig_atomic_t signo[_NSIG];
|
||||||
|
|
||||||
|
static void handler(int);
|
||||||
|
|
||||||
|
char *
|
||||||
|
readpassphrase(const char *prompt, char *buf, size_t bufsiz, int flags)
|
||||||
|
{
|
||||||
|
ssize_t nr;
|
||||||
|
int input, output, save_errno, i, need_restart;
|
||||||
|
char ch, *p, *end;
|
||||||
|
struct termios term, oterm;
|
||||||
|
struct sigaction sa, savealrm, saveint, savehup, savequit, saveterm;
|
||||||
|
struct sigaction savetstp, savettin, savettou, savepipe;
|
||||||
|
|
||||||
|
/* I suppose we could alloc on demand in this case (XXX). */
|
||||||
|
if (bufsiz == 0) {
|
||||||
|
errno = EINVAL;
|
||||||
|
return(NULL);
|
||||||
|
}
|
||||||
|
|
||||||
|
restart:
|
||||||
|
for (i = 0; i < _NSIG; i++)
|
||||||
|
signo[i] = 0;
|
||||||
|
nr = -1;
|
||||||
|
save_errno = 0;
|
||||||
|
need_restart = 0;
|
||||||
|
/*
|
||||||
|
* Read and write to /dev/tty if available. If not, read from
|
||||||
|
* stdin and write to stderr unless a tty is required.
|
||||||
|
*/
|
||||||
|
if ((flags & RPP_STDIN) ||
|
||||||
|
(input = output = open(_PATH_TTY, O_RDWR)) == -1) {
|
||||||
|
if (flags & RPP_REQUIRE_TTY) {
|
||||||
|
errno = ENOTTY;
|
||||||
|
return(NULL);
|
||||||
|
}
|
||||||
|
input = STDIN_FILENO;
|
||||||
|
output = STDERR_FILENO;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Turn off echo if possible.
|
||||||
|
* If we are using a tty but are not the foreground pgrp this will
|
||||||
|
* generate SIGTTOU, so do it *before* installing the signal handlers.
|
||||||
|
*/
|
||||||
|
if (input != STDIN_FILENO && tcgetattr(input, &oterm) == 0) {
|
||||||
|
memcpy(&term, &oterm, sizeof(term));
|
||||||
|
if (!(flags & RPP_ECHO_ON))
|
||||||
|
term.c_lflag &= ~(ECHO | ECHONL);
|
||||||
|
#ifdef VSTATUS
|
||||||
|
if (term.c_cc[VSTATUS] != _POSIX_VDISABLE)
|
||||||
|
term.c_cc[VSTATUS] = _POSIX_VDISABLE;
|
||||||
|
#endif
|
||||||
|
(void)tcsetattr(input, TCSAFLUSH|TCSASOFT, &term);
|
||||||
|
} else {
|
||||||
|
memset(&term, 0, sizeof(term));
|
||||||
|
term.c_lflag |= ECHO;
|
||||||
|
memset(&oterm, 0, sizeof(oterm));
|
||||||
|
oterm.c_lflag |= ECHO;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Catch signals that would otherwise cause the user to end
|
||||||
|
* up with echo turned off in the shell. Don't worry about
|
||||||
|
* things like SIGXCPU and SIGVTALRM for now.
|
||||||
|
*/
|
||||||
|
sigemptyset(&sa.sa_mask);
|
||||||
|
sa.sa_flags = 0; /* don't restart system calls */
|
||||||
|
sa.sa_handler = handler;
|
||||||
|
(void)sigaction(SIGALRM, &sa, &savealrm);
|
||||||
|
(void)sigaction(SIGHUP, &sa, &savehup);
|
||||||
|
(void)sigaction(SIGINT, &sa, &saveint);
|
||||||
|
(void)sigaction(SIGPIPE, &sa, &savepipe);
|
||||||
|
(void)sigaction(SIGQUIT, &sa, &savequit);
|
||||||
|
(void)sigaction(SIGTERM, &sa, &saveterm);
|
||||||
|
(void)sigaction(SIGTSTP, &sa, &savetstp);
|
||||||
|
(void)sigaction(SIGTTIN, &sa, &savettin);
|
||||||
|
(void)sigaction(SIGTTOU, &sa, &savettou);
|
||||||
|
|
||||||
|
if (!(flags & RPP_STDIN))
|
||||||
|
(void)write(output, prompt, strlen(prompt));
|
||||||
|
end = buf + bufsiz - 1;
|
||||||
|
p = buf;
|
||||||
|
while ((nr = read(input, &ch, 1)) == 1 && ch != '\n' && ch != '\r') {
|
||||||
|
if (p < end) {
|
||||||
|
if ((flags & RPP_SEVENBIT))
|
||||||
|
ch &= 0x7f;
|
||||||
|
if (isalpha((unsigned char)ch)) {
|
||||||
|
if ((flags & RPP_FORCELOWER))
|
||||||
|
ch = (char)tolower((unsigned char)ch);
|
||||||
|
if ((flags & RPP_FORCEUPPER))
|
||||||
|
ch = (char)toupper((unsigned char)ch);
|
||||||
|
}
|
||||||
|
*p++ = ch;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
*p = '\0';
|
||||||
|
save_errno = errno;
|
||||||
|
if (!(term.c_lflag & ECHO))
|
||||||
|
(void)write(output, "\n", 1);
|
||||||
|
|
||||||
|
/* Restore old terminal settings and signals. */
|
||||||
|
if (memcmp(&term, &oterm, sizeof(term)) != 0) {
|
||||||
|
const int sigttou = signo[SIGTTOU];
|
||||||
|
|
||||||
|
/* Ignore SIGTTOU generated when we are not the fg pgrp. */
|
||||||
|
while (tcsetattr(input, TCSAFLUSH|TCSASOFT, &oterm) == -1 &&
|
||||||
|
errno == EINTR && !signo[SIGTTOU])
|
||||||
|
continue;
|
||||||
|
signo[SIGTTOU] = sigttou;
|
||||||
|
}
|
||||||
|
(void)sigaction(SIGALRM, &savealrm, NULL);
|
||||||
|
(void)sigaction(SIGHUP, &savehup, NULL);
|
||||||
|
(void)sigaction(SIGINT, &saveint, NULL);
|
||||||
|
(void)sigaction(SIGQUIT, &savequit, NULL);
|
||||||
|
(void)sigaction(SIGPIPE, &savepipe, NULL);
|
||||||
|
(void)sigaction(SIGTERM, &saveterm, NULL);
|
||||||
|
(void)sigaction(SIGTSTP, &savetstp, NULL);
|
||||||
|
(void)sigaction(SIGTTIN, &savettin, NULL);
|
||||||
|
(void)sigaction(SIGTTOU, &savettou, NULL);
|
||||||
|
if (input != STDIN_FILENO)
|
||||||
|
(void)close(input);
|
||||||
|
|
||||||
|
/*
|
||||||
|
* If we were interrupted by a signal, resend it to ourselves
|
||||||
|
* now that we have restored the signal handlers.
|
||||||
|
*/
|
||||||
|
for (i = 0; i < _NSIG; i++) {
|
||||||
|
if (signo[i]) {
|
||||||
|
kill(getpid(), i);
|
||||||
|
switch (i) {
|
||||||
|
case SIGTSTP:
|
||||||
|
case SIGTTIN:
|
||||||
|
case SIGTTOU:
|
||||||
|
need_restart = 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (need_restart)
|
||||||
|
goto restart;
|
||||||
|
|
||||||
|
if (save_errno)
|
||||||
|
errno = save_errno;
|
||||||
|
return(nr == -1 ? NULL : buf);
|
||||||
|
}
|
||||||
|
//DEF_WEAK(readpassphrase);
|
||||||
|
|
||||||
|
#if 0
|
||||||
|
char *
|
||||||
|
getpass(const char *prompt)
|
||||||
|
{
|
||||||
|
static char buf[_PASSWORD_LEN + 1];
|
||||||
|
|
||||||
|
return(readpassphrase(prompt, buf, sizeof(buf), RPP_ECHO_OFF));
|
||||||
|
}
|
||||||
|
#endif
|
||||||
|
|
||||||
|
static void handler(int s)
|
||||||
|
{
|
||||||
|
|
||||||
|
signo[s] = 1;
|
||||||
|
}
|
||||||
|
#endif /* HAVE_READPASSPHRASE */
|
||||||
56
dbms/programs/client/readpassphrase/readpassphrase.h
Normal file
56
dbms/programs/client/readpassphrase/readpassphrase.h
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
// /* $OpenBSD: readpassphrase.h,v 1.5 2003/06/17 21:56:23 millert Exp $ */
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Copyright (c) 2000, 2002 Todd C. Miller <Todd.Miller@courtesan.com>
|
||||||
|
*
|
||||||
|
* Permission to use, copy, modify, and distribute this software for any
|
||||||
|
* purpose with or without fee is hereby granted, provided that the above
|
||||||
|
* copyright notice and this permission notice appear in all copies.
|
||||||
|
*
|
||||||
|
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
|
||||||
|
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
|
||||||
|
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
|
||||||
|
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
|
||||||
|
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
|
||||||
|
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
|
||||||
|
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||||
|
*
|
||||||
|
* Sponsored in part by the Defense Advanced Research Projects
|
||||||
|
* Agency (DARPA) and Air Force Research Laboratory, Air Force
|
||||||
|
* Materiel Command, USAF, under agreement number F39502-99-1-0512.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/* OPENBSD ORIGINAL: include/readpassphrase.h */
|
||||||
|
|
||||||
|
#pragma once
|
||||||
|
// #ifndef _READPASSPHRASE_H_
|
||||||
|
// #define _READPASSPHRASE_H_
|
||||||
|
|
||||||
|
//#include "includes.h"
|
||||||
|
#include "config_client.h"
|
||||||
|
|
||||||
|
#ifndef HAVE_READPASSPHRASE
|
||||||
|
|
||||||
|
# ifdef __cplusplus
|
||||||
|
extern "C" {
|
||||||
|
# endif
|
||||||
|
|
||||||
|
|
||||||
|
# define RPP_ECHO_OFF 0x00 /* Turn off echo (default). */
|
||||||
|
# define RPP_ECHO_ON 0x01 /* Leave echo on. */
|
||||||
|
# define RPP_REQUIRE_TTY 0x02 /* Fail if there is no tty. */
|
||||||
|
# define RPP_FORCELOWER 0x04 /* Force input to lower case. */
|
||||||
|
# define RPP_FORCEUPPER 0x08 /* Force input to upper case. */
|
||||||
|
# define RPP_SEVENBIT 0x10 /* Strip the high bit from input. */
|
||||||
|
# define RPP_STDIN 0x20 /* Read from stdin, not /dev/tty */
|
||||||
|
|
||||||
|
char * readpassphrase(const char *, char *, size_t, int);
|
||||||
|
|
||||||
|
# ifdef __cplusplus
|
||||||
|
}
|
||||||
|
# endif
|
||||||
|
|
||||||
|
|
||||||
|
#endif /* HAVE_READPASSPHRASE */
|
||||||
|
|
||||||
|
// #endif /* !_READPASSPHRASE_H_ */
|
||||||
@ -1,6 +1,7 @@
|
|||||||
#include <iostream>
|
#include <iostream>
|
||||||
#include <optional>
|
#include <optional>
|
||||||
#include <boost/program_options.hpp>
|
#include <boost/program_options.hpp>
|
||||||
|
#include <boost/algorithm/string/join.hpp>
|
||||||
|
|
||||||
#include <Common/Exception.h>
|
#include <Common/Exception.h>
|
||||||
#include <IO/WriteBufferFromFileDescriptor.h>
|
#include <IO/WriteBufferFromFileDescriptor.h>
|
||||||
@ -9,6 +10,8 @@
|
|||||||
#include <Compression/CompressedReadBuffer.h>
|
#include <Compression/CompressedReadBuffer.h>
|
||||||
#include <IO/WriteHelpers.h>
|
#include <IO/WriteHelpers.h>
|
||||||
#include <IO/copyData.h>
|
#include <IO/copyData.h>
|
||||||
|
#include <Parsers/parseQuery.h>
|
||||||
|
#include <Parsers/ExpressionElementParsers.h>
|
||||||
|
|
||||||
#include <Compression/CompressionFactory.h>
|
#include <Compression/CompressionFactory.h>
|
||||||
|
|
||||||
@ -64,7 +67,7 @@ int mainEntryClickHouseCompressor(int argc, char ** argv)
|
|||||||
("hc", "use LZ4HC instead of LZ4")
|
("hc", "use LZ4HC instead of LZ4")
|
||||||
("zstd", "use ZSTD instead of LZ4")
|
("zstd", "use ZSTD instead of LZ4")
|
||||||
("codec", boost::program_options::value<std::vector<std::string>>()->multitoken(), "use codecs combination instead of LZ4")
|
("codec", boost::program_options::value<std::vector<std::string>>()->multitoken(), "use codecs combination instead of LZ4")
|
||||||
("level", boost::program_options::value<std::vector<int>>()->multitoken(), "compression levels for codecs specified via --codec")
|
("level", boost::program_options::value<int>(), "compression level for codecs spicified via flags")
|
||||||
("none", "use no compression instead of LZ4")
|
("none", "use no compression instead of LZ4")
|
||||||
("stat", "print block statistics of compressed data")
|
("stat", "print block statistics of compressed data")
|
||||||
;
|
;
|
||||||
@ -94,6 +97,9 @@ int mainEntryClickHouseCompressor(int argc, char ** argv)
|
|||||||
if ((use_lz4hc || use_zstd || use_none) && !codecs.empty())
|
if ((use_lz4hc || use_zstd || use_none) && !codecs.empty())
|
||||||
throw DB::Exception("Wrong options, codec flags like --zstd and --codec options are mutually exclusive", DB::ErrorCodes::BAD_ARGUMENTS);
|
throw DB::Exception("Wrong options, codec flags like --zstd and --codec options are mutually exclusive", DB::ErrorCodes::BAD_ARGUMENTS);
|
||||||
|
|
||||||
|
if (!codecs.empty() && options.count("level"))
|
||||||
|
throw DB::Exception("Wrong options, --level is not compatible with --codec list", DB::ErrorCodes::BAD_ARGUMENTS);
|
||||||
|
|
||||||
std::string method_family = "LZ4";
|
std::string method_family = "LZ4";
|
||||||
|
|
||||||
if (use_lz4hc)
|
if (use_lz4hc)
|
||||||
@ -103,28 +109,22 @@ int mainEntryClickHouseCompressor(int argc, char ** argv)
|
|||||||
else if (use_none)
|
else if (use_none)
|
||||||
method_family = "NONE";
|
method_family = "NONE";
|
||||||
|
|
||||||
std::vector<int> levels;
|
std::optional<int> level = std::nullopt;
|
||||||
if (options.count("level"))
|
if (options.count("level"))
|
||||||
levels = options["level"].as<std::vector<int>>();
|
level = options["level"].as<int>();
|
||||||
|
|
||||||
|
|
||||||
DB::CompressionCodecPtr codec;
|
DB::CompressionCodecPtr codec;
|
||||||
if (!codecs.empty())
|
if (!codecs.empty())
|
||||||
{
|
{
|
||||||
if (levels.size() > codecs.size())
|
DB::ParserCodec codec_parser;
|
||||||
throw DB::Exception("Specified more levels than codecs", DB::ErrorCodes::BAD_ARGUMENTS);
|
|
||||||
|
|
||||||
std::vector<DB::CodecNameWithLevel> codec_names;
|
std::string codecs_line = boost::algorithm::join(codecs, ",");
|
||||||
for (size_t i = 0; i < codecs.size(); ++i)
|
auto ast = DB::parseQuery(codec_parser, "(" + codecs_line + ")", 0);
|
||||||
{
|
codec = DB::CompressionCodecFactory::instance().get(ast, nullptr);
|
||||||
if (i < levels.size())
|
|
||||||
codec_names.emplace_back(codecs[i], levels[i]);
|
|
||||||
else
|
|
||||||
codec_names.emplace_back(codecs[i], std::nullopt);
|
|
||||||
}
|
|
||||||
codec = DB::CompressionCodecFactory::instance().get(codec_names);
|
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
codec = DB::CompressionCodecFactory::instance().get(method_family, levels.empty() ? std::nullopt : std::optional<int>(levels.back()));
|
codec = DB::CompressionCodecFactory::instance().get(method_family, level);
|
||||||
|
|
||||||
|
|
||||||
DB::ReadBufferFromFileDescriptor rb(STDIN_FILENO);
|
DB::ReadBufferFromFileDescriptor rb(STDIN_FILENO);
|
||||||
|
|||||||
@ -17,11 +17,11 @@ $ ./clickhouse-compressor --decompress < input_file > output_file
|
|||||||
Compress data with ZSTD at level 5:
|
Compress data with ZSTD at level 5:
|
||||||
|
|
||||||
```
|
```
|
||||||
$ ./clickhouse-compressor --codec ZSTD --level 5 < input_file > output_file
|
$ ./clickhouse-compressor --codec 'ZSTD(5)' < input_file > output_file
|
||||||
```
|
```
|
||||||
|
|
||||||
Compress data with ZSTD level 10, LZ4HC level 7 and LZ4.
|
Compress data with Delta of four bytes and ZSTD level 10.
|
||||||
|
|
||||||
```
|
```
|
||||||
$ ./clickhouse-compressor --codec ZSTD --level 5 --codec LZ4HC --level 7 --codec LZ4 < input_file > output_file
|
$ ./clickhouse-compressor --codec 'Delta(4)' --codec 'ZSTD(10)' < input_file > output_file
|
||||||
```
|
```
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user