Merge branch 'master' into filimonov-quorum-dedup-issue

.gitattributes

@@ -1,2 +1,4 @@
 contrib/* linguist-vendored
 *.h linguist-language=C++
+# to avoid frequent conflicts
+tests/queries/0_stateless/arcadia_skip_list.txt text merge=union

.github/ISSUE_TEMPLATE/10_question.md

@@ -0,0 +1,12 @@
+---
+name: Question
+about: Ask a question about ClickHouse
+title: ''
+labels: question
+assignees: ''
+
+---
+
+Make sure to check documentation https://clickhouse.yandex/docs/en/ first. If the question is concise and probably has a short answer, asking it in Telegram chat https://telegram.me/clickhouse_en is probably the fastest way to find the answer. For more complicated questions, consider asking them on StackOverflow with "clickhouse" tag https://stackoverflow.com/questions/tagged/clickhouse 
+
+If you still prefer GitHub issues, remove all this text and ask your question here.

.github/ISSUE_TEMPLATE/30_unexpected-behaviour.md

@@ -0,0 +1,30 @@
+---
+name: Unexpected behaviour
+about: Some feature is working in non-obvious way
+title: ''
+labels: unexpected behaviour
+assignees: ''
+
+---
+
+(you don't have to strictly follow this form)
+
+**Describe the unexpected behaviour**
+A clear and concise description of what works not as it is supposed to.
+
+**How to reproduce**
+* Which ClickHouse server version to use
+* Which interface to use, if matters
+* Non-default settings, if any
+* `CREATE TABLE` statements for all tables involved
+* Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+* Queries to run that lead to unexpected result
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Error message and/or stacktrace**
+If applicable, add screenshots to help explain your problem.
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/35_incomplete_implementation.md

@@ -0,0 +1,30 @@
+---
+name: Incomplete implementation
+about: Implementation of existing feature is not finished
+title: ''
+labels: unfinished code
+assignees: ''
+
+---
+
+(you don't have to strictly follow this form)
+
+**Describe the unexpected behaviour**
+A clear and concise description of what works not as it is supposed to.
+
+**How to reproduce**
+* Which ClickHouse server version to use
+* Which interface to use, if matters
+* Non-default settings, if any
+* `CREATE TABLE` statements for all tables involved
+* Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+* Queries to run that lead to unexpected result
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Error message and/or stacktrace**
+If applicable, add screenshots to help explain your problem.
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/40_bug-report.md

@@ -0,0 +1,33 @@
+---
+name: Bug report
+about: Create a report to help us improve ClickHouse
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+(you don't have to strictly follow this form)
+
+**Describe the bug**
+A clear and concise description of what works not as it is supposed to.
+
+**Does it reproduce on recent release?**
+[The list of releases](https://github.com/ClickHouse/ClickHouse/blob/master/utils/list-versions/version_date.tsv)
+
+**How to reproduce**
+* Which ClickHouse server version to use
+* Which interface to use, if matters
+* Non-default settings, if any
+* `CREATE TABLE` statements for all tables involved
+* Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+* Queries to run that lead to unexpected result
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Error message and/or stacktrace**
+If applicable, add screenshots to help explain your problem.
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/45_usability-issue.md

@@ -0,0 +1,30 @@
+---
+name: Usability issue
+about: Report something can be made more convenient to use
+title: ''
+labels: usability
+assignees: ''
+
+---
+
+(you don't have to strictly follow this form)
+
+**Describe the issue**
+A clear and concise description of what works not as it is supposed to.
+
+**How to reproduce**
+* Which ClickHouse server version to use
+* Which interface to use, if matters
+* Non-default settings, if any
+* `CREATE TABLE` statements for all tables involved
+* Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+* Queries to run that lead to unexpected result
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Error message and/or stacktrace**
+If applicable, add screenshots to help explain your problem.
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/80_backward-compatibility.md

@@ -0,0 +1,27 @@
+---
+name: Backward compatibility issue
+about: Report the case when the behaviour of a new version can break existing use cases
+title: ''
+labels: backward compatibility
+assignees: ''
+
+---
+
+(you don't have to strictly follow this form)
+
+**Describe the issue**
+A clear and concise description of what works not as it is supposed to.
+
+**How to reproduce**
+* Which ClickHouse server versions are incompatible
+* Which interface to use, if matters
+* Non-default settings, if any
+* `CREATE TABLE` statements for all tables involved
+* Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
+* Queries to run that lead to unexpected result
+
+**Error message and/or stacktrace**
+If applicable, add screenshots to help explain your problem.
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/90_fuzzing-report.md

@@ -0,0 +1,16 @@
+---
+name: Assertion found via fuzzing
+about: Potential issue has been found via Fuzzer or Stress tests
+title: ''
+labels: fuzz
+assignees: ''
+
+---
+
+(you don't have to strictly follow this form)
+
+**Describe the bug**
+A link to the report
+
+**How to reproduce**
+Try to reproduce the report and copy the tables and queries involved.

.github/ISSUE_TEMPLATE/95_sanitizer-report.md

@@ -0,0 +1,19 @@
+---
+name: Sanitizer alert
+about: Potential issue has been found by special code instrumentation
+title: ''
+labels: testing
+assignees: ''
+
+---
+
+(you don't have to strictly follow this form)
+
+**Describe the bug**
+A link to the report
+
+**How to reproduce**
+Try to reproduce the report and copy the tables and queries involved.
+
+**Error message and/or stacktrace**
+You can find additional information in server logs.

.github/ISSUE_TEMPLATE/backward-compatibility.md

@@ -1,27 +0,0 @@
----
-name: Backward compatibility issue
-about: Create a report to help us improve ClickHouse
-title: ''
-labels: backward compatibility
-assignees: ''
-
----
-
-(you don't have to strictly follow this form)
-
-**Describe the issue**
-A clear and concise description of what works not as it is supposed to.
-
-**How to reproduce**
-* Which ClickHouse server versions are incompatible
-* Which interface to use, if matters
-* Non-default settings, if any
-* `CREATE TABLE` statements for all tables involved
-* Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
-* Queries to run that lead to unexpected result
-
-**Error message and/or stacktrace**
-If applicable, add screenshots to help explain your problem.
-
-**Additional context**
-Add any other context about the problem here.

.github/ISSUE_TEMPLATE/bug-report.md

@@ -1,30 +0,0 @@
----
-name: Bug report
-about: Create a report to help us improve ClickHouse
-title: ''
-labels: bug
-assignees: ''
-
----
-
-(you don't have to strictly follow this form)
-
-**Describe the bug**
-A clear and concise description of what works not as it is supposed to.
-
-**How to reproduce**
-* Which ClickHouse server version to use
-* Which interface to use, if matters
-* Non-default settings, if any
-* `CREATE TABLE` statements for all tables involved
-* Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
-* Queries to run that lead to unexpected result
-
-**Expected behavior**
-A clear and concise description of what you expected to happen.
-
-**Error message and/or stacktrace**
-If applicable, add screenshots to help explain your problem.
-
-**Additional context**
-Add any other context about the problem here.

.github/ISSUE_TEMPLATE/question.md

@@ -1,12 +0,0 @@
----
-name: Question
-about: Ask question about ClickHouse
-title: ''
-labels: question
-assignees: ''
-
----
-
-Make sure to check documentation https://clickhouse.yandex/docs/en/ first. If the question is concise and probably has a short answer, asking it in Telegram chat https://telegram.me/clickhouse_en is probably the fastest way to find the answer. For more complicated questions, consider asking them on StackOverflow with "clickhouse" tag https://stackoverflow.com/questions/tagged/clickhouse 
-
-If you still prefer GitHub issues, remove all this text and ask your question here.

.github/ISSUE_TEMPLATE/unexpected-behaviour.md

@@ -1,30 +0,0 @@
----
-name: Unexpected behaviour
-about: Create a report to help us improve ClickHouse
-title: ''
-labels: unexpected behaviour
-assignees: ''
-
----
-
-(you don't have to strictly follow this form)
-
-**Describe the unexpected behaviour**
-A clear and concise description of what works not as it is supposed to.
-
-**How to reproduce**
-* Which ClickHouse server version to use
-* Which interface to use, if matters
-* Non-default settings, if any
-* `CREATE TABLE` statements for all tables involved
-* Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
-* Queries to run that lead to unexpected result
-
-**Expected behavior**
-A clear and concise description of what you expected to happen.
-
-**Error message and/or stacktrace**
-If applicable, add screenshots to help explain your problem.
-
-**Additional context**
-Add any other context about the problem here.

.github/ISSUE_TEMPLATE/usability-issue.md

@@ -1,30 +0,0 @@
----
-name: Usability issue
-about: Create a report to help us improve ClickHouse
-title: ''
-labels: usability
-assignees: ''
-
----
-
-(you don't have to strictly follow this form)
-
-**Describe the issue**
-A clear and concise description of what works not as it is supposed to.
-
-**How to reproduce**
-* Which ClickHouse server version to use
-* Which interface to use, if matters
-* Non-default settings, if any
-* `CREATE TABLE` statements for all tables involved
-* Sample data for all these tables, use [clickhouse-obfuscator](https://github.com/ClickHouse/ClickHouse/blob/master/programs/obfuscator/Obfuscator.cpp#L42-L80) if necessary
-* Queries to run that lead to unexpected result
-
-**Expected behavior**
-A clear and concise description of what you expected to happen.
-
-**Error message and/or stacktrace**
-If applicable, add screenshots to help explain your problem.
-
-**Additional context**
-Add any other context about the problem here.

.github/codecov.yml

@@ -1,5 +1,5 @@
 codecov:
-  max_report_age: off
+  max_report_age: "off"
   strict_yaml_branch: "master"
 
 ignore:
@@ -14,4 +14,4 @@ ignore:
 comment: false
 
 github_checks:
-  annotations: false
+  annotations: false

.github/workflows/anchore-analysis.yml

@@ -1,16 +1,16 @@
 # This workflow checks out code, performs an Anchore container image
 # vulnerability and compliance scan, and integrates the results with
-# GitHub Advanced Security code scanning feature.  For more information on
+# GitHub Advanced Security code scanning feature. For more information on
 # the Anchore scan action usage and parameters, see
-# https://github.com/anchore/scan-action.  For more information on
+# https://github.com/anchore/scan-action. For more information on
 # Anchore container image scanning in general, see
 # https://docs.anchore.com.
 
 name: Docker Container Scan (clickhouse-server)
 
-on:
+"on":
   pull_request:
-    paths: 
+    paths:
       - docker/server/Dockerfile
       - .github/workflows/anchore-analysis.yml
   schedule:
@@ -20,26 +20,20 @@ jobs:
   Anchore-Build-Scan:
     runs-on: ubuntu-latest
     steps:
-    - name: Checkout the code
-      uses: actions/checkout@v2
-    - name: Build the Docker image
-      run: |
-        cd docker/server
-        perl -pi -e 's|=\$version||g' Dockerfile
-        docker build . --file Dockerfile --tag localbuild/testimage:latest      
-    - name: Run the local Anchore scan action itself with GitHub Advanced Security code scanning integration enabled
-      uses: anchore/scan-action@master
-      with:
-        image-reference: "localbuild/testimage:latest"
-        dockerfile-path: "docker/server/Dockerfile"
-        acs-report-enable: true
-        fail-build: true
-    - name: Upload artifact
-      uses: actions/upload-artifact@v1.0.0
-      with:
-        name: AnchoreReports
-        path: ./anchore-reports/
-    - name: Upload Anchore Scan Report
-      uses: github/codeql-action/upload-sarif@v1
-      with:
-        sarif_file: results.sarif
+      - name: Checkout the code
+        uses: actions/checkout@v2
+      - name: Build the Docker image
+        run: |
+          cd docker/server
+          perl -pi -e 's|=\$version||g' Dockerfile
+          docker build . --file Dockerfile --tag localbuild/testimage:latest
+      - name: Run the local Anchore scan action itself with GitHub Advanced Security code scanning integration enabled
+        uses: anchore/scan-action@v2
+        id: scan
+        with:
+          image: "localbuild/testimage:latest"
+          acs-report-enable: true
+      - name: Upload Anchore Scan Report
+        uses: github/codeql-action/upload-sarif@v1
+        with:
+          sarif_file: ${{ steps.scan.outputs.sarif }}

.github/workflows/codeql-analysis.yml

@@ -1,33 +0,0 @@
-name: "CodeQL Scanning"
-
-on:
-  schedule:
-    - cron: '0 19 * * *'
-jobs:
-  CodeQL-Build:
-
-    runs-on: self-hosted
-    timeout-minutes: 1440
-
-    steps:
-    - name: Checkout repository
-      uses: actions/checkout@v2
-      with:
-        fetch-depth: 2
-        submodules: 'recursive'
-
-    - run: git checkout HEAD^2
-      if: ${{ github.event_name == 'pull_request' }}
-      
-    - name: Initialize CodeQL
-      uses: github/codeql-action/init@v1
-
-      with:
-        languages: cpp
-
-    - run: sudo apt-get update && sudo apt-get install -y git cmake python ninja-build gcc-9 g++-9 && mkdir build
-    - run: cd build && CC=gcc-9 CXX=g++-9 cmake ..
-    - run: cd build && ninja
-
-    - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@v1

.gitignore

@@ -137,3 +137,9 @@ website/package-lock.json
 /prof
 
 *.iml
+
+# data store
+/programs/server/data
+/programs/server/metadata
+/programs/server/store
+

.gitmodules

@@ -53,7 +53,8 @@
 	url = https://github.com/ClickHouse-Extras/Turbo-Base64.git
 [submodule "contrib/arrow"]
 	path = contrib/arrow
-	url = https://github.com/apache/arrow
+	url = https://github.com/ClickHouse-Extras/arrow
+	branch = clickhouse-arrow-2.0.0
 [submodule "contrib/thrift"]
 	path = contrib/thrift
 	url = https://github.com/apache/thrift.git
@@ -83,7 +84,7 @@
 	url = https://github.com/google/brotli.git
 [submodule "contrib/h3"]
 	path = contrib/h3
-	url = https://github.com/uber/h3
+	url = https://github.com/ClickHouse-Extras/h3
 [submodule "contrib/hyperscan"]
 	path = contrib/hyperscan
 	url = https://github.com/ClickHouse-Extras/hyperscan.git
@@ -92,7 +93,7 @@
 	url = https://github.com/ClickHouse-Extras/libunwind.git
 [submodule "contrib/simdjson"]
 	path = contrib/simdjson
-	url = https://github.com/ClickHouse-Extras/simdjson.git
+	url = https://github.com/simdjson/simdjson.git
 [submodule "contrib/rapidjson"]
 	path = contrib/rapidjson
 	url = https://github.com/ClickHouse-Extras/rapidjson
@@ -124,9 +125,6 @@
 [submodule "contrib/curl"]
 	path = contrib/curl
 	url = https://github.com/curl/curl.git
-[submodule "contrib/openssl"]
-	path = contrib/openssl
-	url = https://github.com/ClickHouse-Extras/openssl.git
 [submodule "contrib/icudata"]
 	path = contrib/icudata
 	url = https://github.com/ClickHouse-Extras/icudata.git
@@ -186,7 +184,7 @@
 	url = https://github.com/ClickHouse-Extras/krb5
 [submodule "contrib/cyrus-sasl"]
 	path = contrib/cyrus-sasl
-	url = https://github.com/cyrusimap/cyrus-sasl
+	url = https://github.com/ClickHouse-Extras/cyrus-sasl
 	branch = cyrus-sasl-2.1
 [submodule "contrib/croaring"]
 	path = contrib/croaring
@@ -211,3 +209,18 @@
 [submodule "contrib/fast_float"]
 	path = contrib/fast_float
 	url = https://github.com/fastfloat/fast_float
+[submodule "contrib/libpqxx"]
+	path = contrib/libpqxx
+	url = https://github.com/jtv/libpqxx
+[submodule "contrib/libpq"]
+	path = contrib/libpq
+	url = https://github.com/ClickHouse-Extras/libpq
+[submodule "contrib/boringssl"]
+	path = contrib/boringssl
+	url = https://github.com/ClickHouse-Extras/boringssl.git
+[submodule "contrib/NuRaft"]
+	path = contrib/NuRaft
+	url = https://github.com/ClickHouse-Extras/NuRaft.git
+[submodule "contrib/datasketches-cpp"]
+	path = contrib/datasketches-cpp
+	url = https://github.com/ClickHouse-Extras/datasketches-cpp.git

.potato.yml

@@ -14,14 +14,14 @@ handlers:
         # The trigger for creating the Yandex.Tracker issue. When the specified event occurs, it transfers PR data to Yandex.Tracker.
         github:pullRequest:labeled:
           data:
-              # The Yandex.Tracker queue to create the issue in. Each issue in Tracker belongs to one of the project queues.
-          	queue: CLICKHOUSEDOCS
-              # The issue title.
-          	summary: '[Potato] Pull Request #{{pullRequest.number}}'
-              # The issue description.
-          	description: >
+          # The Yandex.Tracker queue to create the issue in. Each issue in Tracker belongs to one of the project queues.
+          queue: CLICKHOUSEDOCS
+          # The issue title.
+          summary: '[Potato] Pull Request #{{pullRequest.number}}'
+          # The issue description.
+          description: >
                 {{pullRequest.description}}
 
                 Ссылка на Pull Request: {{pullRequest.webUrl}}
-            # The condition for creating the Yandex.Tracker issue.
-        	condition: eventPayload.labels.filter(label => ['pr-feature'].includes(label.name)).length
+          # The condition for creating the Yandex.Tracker issue.
+          condition: eventPayload.labels.filter(label => ['pr-feature'].includes(label.name)).length

.pylintrc

@@ -0,0 +1,45 @@
+# vim: ft=config
+
+[BASIC]
+max-module-lines=2000
+# due to SQL
+max-line-length=200
+# Drop/decrease them one day:
+max-branches=50
+max-nested-blocks=10
+max-statements=200
+
+[FORMAT]
+ignore-long-lines = (# )?<?https?://\S+>?$
+
+[MESSAGES CONTROL]
+disable = bad-continuation,
+          missing-docstring,
+          bad-whitespace,
+          too-few-public-methods,
+          invalid-name,
+          too-many-arguments,
+          keyword-arg-before-vararg,
+          too-many-locals,
+          too-many-instance-attributes,
+          cell-var-from-loop,
+          fixme,
+          too-many-public-methods,
+          wildcard-import,
+          unused-wildcard-import,
+          singleton-comparison,
+          # pytest.mark.parametrize is not callable (not-callable)
+          not-callable,
+          # https://github.com/PyCQA/pylint/issues/3882
+          # [Python 3.9] Value 'Optional' is unsubscriptable (unsubscriptable-object) (also Union)
+          unsubscriptable-object,
+          # Drop them one day:
+          redefined-outer-name,
+          broad-except,
+          bare-except,
+          no-else-return,
+          global-statement
+
+[SIMILARITIES]
+# due to SQL
+min-similarity-lines=1000

.yamllint

@@ -0,0 +1,15 @@
+# vi: ft=yaml
+extends: default
+
+rules:
+    indentation:
+        level: warning
+        indent-sequences: consistent
+    line-length:
+        # there are some bash -c "", so this is OK
+        max: 300
+        level: warning
+    comments:
+        min-spaces-from-content: 1
+    document-start:
+        present: false

CMakeLists.txt

@@ -39,6 +39,8 @@ else()
     set(RECONFIGURE_MESSAGE_LEVEL STATUS)
 endif()
 
+enable_language(C CXX ASM)
+
 include (cmake/arch.cmake)
 include (cmake/target.cmake)
 include (cmake/tools.cmake)
@@ -66,9 +68,30 @@ endif ()
 
 include (cmake/find/ccache.cmake)
 
-option(ENABLE_CHECK_HEAVY_BUILDS "Don't allow C++ translation units to compile too long or to take too much memory while compiling" OFF)
+# Take care to add prlimit in command line before ccache, or else ccache thinks that
+# prlimit is compiler, and clang++ is its input file, and refuses to work  with
+# multiple inputs, e.g in ccache log:
+# [2021-03-31T18:06:32.655327 36900] Command line: /usr/bin/ccache prlimit --as=10000000000 --data=5000000000 --cpu=600 /usr/bin/clang++-11 - ...... std=gnu++2a -MD -MT src/CMakeFiles/dbms.dir/Storages/MergeTree/IMergeTreeDataPart.cpp.o -MF src/CMakeFiles/dbms.dir/Storages/MergeTree/IMergeTreeDataPart.cpp.o.d -o src/CMakeFiles/dbms.dir/Storages/MergeTree/IMergeTreeDataPart.cpp.o -c ../src/Storages/MergeTree/IMergeTreeDataPart.cpp
+#
+# [2021-03-31T18:06:32.656704 36900] Multiple input files: /usr/bin/clang++-11 and ../src/Storages/MergeTree/IMergeTreeDataPart.cpp
+#
+# Another way would be to use --ccache-skip option before clang++-11 to make
+# ccache ignore it.
+option(ENABLE_CHECK_HEAVY_BUILDS "Don't allow C++ translation units to compile too long or to take too much memory while compiling." OFF)
 if (ENABLE_CHECK_HEAVY_BUILDS)
-    set (CMAKE_CXX_COMPILER_LAUNCHER prlimit --rss=10000000 --cpu=600)
+    # set DATA (since RSS does not work since 2.6.x+) to 2G
+    set (RLIMIT_DATA 5000000000)
+    # set VIRT (RLIMIT_AS) to 10G (DATA*10)
+    set (RLIMIT_AS 10000000000)
+    # set CPU time limit to 600 seconds
+    set (RLIMIT_CPU 600)
+
+    # gcc10/gcc10/clang -fsanitize=memory is too heavy
+    if (SANITIZE STREQUAL "memory" OR COMPILER_GCC)
+       set (RLIMIT_DATA 10000000000)
+    endif()
+
+    set (CMAKE_CXX_COMPILER_LAUNCHER prlimit --as=${RLIMIT_AS} --data=${RLIMIT_DATA} --cpu=${RLIMIT_CPU} ${CMAKE_CXX_COMPILER_LAUNCHER})
 endif ()
 
 if (NOT CMAKE_BUILD_TYPE OR CMAKE_BUILD_TYPE STREQUAL "None")
@@ -112,6 +135,12 @@ if (ENABLE_FUZZING)
     set (FUZZER "libfuzzer")
 endif()
 
+# Global libraries
+# See:
+# - default_libs.cmake
+# - sanitize.cmake
+add_library(global-libs INTERFACE)
+
 include (cmake/fuzzer.cmake)
 include (cmake/sanitize.cmake)
 
@@ -141,7 +170,6 @@ option(ENABLE_TESTS "Provide unit_test_dbms target with Google.Test unit tests"
 
 if (OS_LINUX AND NOT UNBUNDLED AND MAKE_STATIC_LIBRARIES AND NOT SPLIT_SHARED_LIBRARIES AND CMAKE_VERSION VERSION_GREATER "3.9.0")
     # Only for Linux, x86_64.
-    # Implies ${ENABLE_FASTMEMCPY}
     option(GLIBC_COMPATIBILITY "Enable compatibility with older glibc libraries." ON)
 elseif(GLIBC_COMPATIBILITY)
     message (${RECONFIGURE_MESSAGE_LEVEL} "Glibc compatibility cannot be enabled in current configuration")
@@ -155,7 +183,7 @@ endif ()
 set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -rdynamic")
 
 if (OS_LINUX)
-    find_program (OBJCOPY_PATH NAMES "llvm-objcopy" "llvm-objcopy-11" "llvm-objcopy-10" "llvm-objcopy-9" "llvm-objcopy-8" "objcopy")
+    find_program (OBJCOPY_PATH NAMES "llvm-objcopy" "llvm-objcopy-12" "llvm-objcopy-11" "llvm-objcopy-10" "llvm-objcopy-9" "llvm-objcopy-8" "objcopy")
     if (OBJCOPY_PATH)
         message(STATUS "Using objcopy: ${OBJCOPY_PATH}.")
 
@@ -181,13 +209,14 @@ endif ()
 option(ADD_GDB_INDEX_FOR_GOLD "Add .gdb-index to resulting binaries for gold linker.")
 
 if (NOT CMAKE_BUILD_TYPE_UC STREQUAL "RELEASE")
-    if (LINKER_NAME STREQUAL "lld")
+    # Can be lld or ld-lld.
+    if (LINKER_NAME MATCHES "lld$")
         set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--gdb-index")
         set (CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--gdb-index")
         message (STATUS "Adding .gdb-index via --gdb-index linker option.")
     # we use another tool for gdb-index, because gold linker removes section .debug_aranges, which used inside clickhouse stacktraces
     # http://sourceware-org.1504.n7.nabble.com/gold-No-debug-aranges-section-when-linking-with-gdb-index-td540965.html#a556932
-    elseif (LINKER_NAME STREQUAL "gold" AND ADD_GDB_INDEX_FOR_GOLD)
+    elseif (LINKER_NAME MATCHES "gold$" AND ADD_GDB_INDEX_FOR_GOLD)
         find_program (GDB_ADD_INDEX_EXE NAMES "gdb-add-index" DOC "Path to gdb-add-index executable")
         if (NOT GDB_ADD_INDEX_EXE)
             set (USE_GDB_ADD_INDEX 0)
@@ -199,6 +228,19 @@ if (NOT CMAKE_BUILD_TYPE_UC STREQUAL "RELEASE")
     endif ()
 endif()
 
+# Create BuildID when using lld. For other linkers it is created by default.
+if (LINKER_NAME MATCHES "lld$")
+    # SHA1 is not cryptographically secure but it is the best what lld is offering.
+    set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--build-id=sha1")
+endif ()
+
+# Add a section with the hash of the compiled machine code for integrity checks.
+# Only for official builds, because adding a section can be time consuming (rewrite of several GB).
+# And cross compiled binaries are not supported (since you cannot execute clickhouse hash-binary)
+if (OBJCOPY_PATH AND YANDEX_OFFICIAL_BUILD AND (NOT CMAKE_TOOLCHAIN_FILE))
+    set (USE_BINARY_HASH 1)
+endif ()
+
 cmake_host_system_information(RESULT AVAILABLE_PHYSICAL_MEMORY QUERY AVAILABLE_PHYSICAL_MEMORY) # Not available under freebsd
 
 
@@ -213,9 +255,7 @@ else()
     message(STATUS "Disabling compiler -pipe option (have only ${AVAILABLE_PHYSICAL_MEMORY} mb of memory)")
 endif()
 
-if(NOT DISABLE_CPU_OPTIMIZE)
-    include(cmake/cpu_features.cmake)
-endif()
+include(cmake/cpu_features.cmake)
 
 option(ARCH_NATIVE "Add -march=native compiler flag")
 
@@ -223,19 +263,27 @@ if (ARCH_NATIVE)
     set (COMPILER_FLAGS "${COMPILER_FLAGS} -march=native")
 endif ()
 
-if (COMPILER_GCC OR COMPILER_CLANG)
-    # to make numeric_limits<__int128> works with GCC
-    set (_CXX_STANDARD "gnu++2a")
-else()
-    set (_CXX_STANDARD "c++2a")
-endif()
+if (${CMAKE_VERSION} VERSION_LESS "3.12.4")
+    # CMake < 3.12 doesn't support setting 20 as a C++ standard version.
+    # We will add C++ standard controlling flag in CMAKE_CXX_FLAGS manually for now.
 
-# cmake < 3.12 doesn't support 20. We'll set CMAKE_CXX_FLAGS for now
-# set (CMAKE_CXX_STANDARD 20)
-set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=${_CXX_STANDARD}")
+    if (COMPILER_GCC OR COMPILER_CLANG)
+        # to make numeric_limits<__int128> works with GCC
+        set (_CXX_STANDARD "gnu++2a")
+    else ()
+        set (_CXX_STANDARD "c++2a")
+    endif ()
 
-set (CMAKE_CXX_EXTENSIONS 0) # https://cmake.org/cmake/help/latest/prop_tgt/CXX_EXTENSIONS.html#prop_tgt:CXX_EXTENSIONS
-set (CMAKE_CXX_STANDARD_REQUIRED ON)
+    set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=${_CXX_STANDARD}")
+else ()
+    set (CMAKE_CXX_STANDARD 20)
+    set (CMAKE_CXX_EXTENSIONS ON) # Same as gnu++2a (ON) vs c++2a (OFF): https://cmake.org/cmake/help/latest/prop_tgt/CXX_EXTENSIONS.html
+    set (CMAKE_CXX_STANDARD_REQUIRED ON)
+endif ()
+
+set (CMAKE_C_STANDARD 11)
+set (CMAKE_C_EXTENSIONS ON)
+set (CMAKE_C_STANDARD_REQUIRED ON)
 
 if (COMPILER_GCC OR COMPILER_CLANG)
     # Enable C++14 sized global deallocation functions. It should be enabled by setting -std=c++14 but I'm not sure.
@@ -303,7 +351,7 @@ if (COMPILER_CLANG)
     endif ()
 
     # Always prefer llvm tools when using clang. For instance, we cannot use GNU ar when llvm LTO is enabled
-    find_program (LLVM_AR_PATH NAMES "llvm-ar" "llvm-ar-11" "llvm-ar-10" "llvm-ar-9" "llvm-ar-8")
+    find_program (LLVM_AR_PATH NAMES "llvm-ar" "llvm-ar-12" "llvm-ar-11" "llvm-ar-10" "llvm-ar-9" "llvm-ar-8")
 
     if (LLVM_AR_PATH)
         message(STATUS "Using llvm-ar: ${LLVM_AR_PATH}.")
@@ -312,7 +360,7 @@ if (COMPILER_CLANG)
         message(WARNING "Cannot find llvm-ar. System ar will be used instead. It does not work with ThinLTO.")
     endif ()
 
-    find_program (LLVM_RANLIB_PATH NAMES "llvm-ranlib" "llvm-ranlib-11" "llvm-ranlib-10" "llvm-ranlib-9" "llvm-ranlib-8")
+    find_program (LLVM_RANLIB_PATH NAMES "llvm-ranlib" "llvm-ranlib-12" "llvm-ranlib-11" "llvm-ranlib-10" "llvm-ranlib-9" "llvm-ranlib-8")
 
     if (LLVM_RANLIB_PATH)
         message(STATUS "Using llvm-ranlib: ${LLVM_RANLIB_PATH}.")
@@ -429,6 +477,7 @@ find_contrib_lib(double-conversion) # Must be before parquet
 include (cmake/find/ssl.cmake)
 include (cmake/find/ldap.cmake) # after ssl
 include (cmake/find/icu.cmake)
+include (cmake/find/xz.cmake)
 include (cmake/find/zlib.cmake)
 include (cmake/find/zstd.cmake)
 include (cmake/find/ltdl.cmake) # for odbc
@@ -462,6 +511,8 @@ include (cmake/find/rapidjson.cmake)
 include (cmake/find/fastops.cmake)
 include (cmake/find/odbc.cmake)
 include (cmake/find/rocksdb.cmake)
+include (cmake/find/libpqxx.cmake)
+include (cmake/find/nuraft.cmake)
 
 
 if(NOT USE_INTERNAL_PARQUET_LIBRARY)
@@ -474,6 +525,7 @@ include (cmake/find/msgpack.cmake)
 include (cmake/find/cassandra.cmake)
 include (cmake/find/sentry.cmake)
 include (cmake/find/stats.cmake)
+include (cmake/find/datasketches.cmake)
 
 set (USE_INTERNAL_CITYHASH_LIBRARY ON CACHE INTERNAL "")
 find_contrib_lib(cityhash)
@@ -506,15 +558,18 @@ macro (add_executable target)
     # explicitly acquire and interpose malloc symbols by clickhouse_malloc
     # if GLIBC_COMPATIBILITY is ON and ENABLE_THINLTO is on than provide memcpy symbol explicitly to neutrialize thinlto's libcall generation.
     if (GLIBC_COMPATIBILITY AND ENABLE_THINLTO)
-        _add_executable (${ARGV} $<TARGET_OBJECTS:clickhouse_malloc> $<TARGET_OBJECTS:clickhouse_memcpy>)
+        _add_executable (${ARGV} $<TARGET_OBJECTS:clickhouse_malloc> $<TARGET_OBJECTS:memcpy>)
     else ()
         _add_executable (${ARGV} $<TARGET_OBJECTS:clickhouse_malloc>)
     endif ()
 
     get_target_property (type ${target} TYPE)
     if (${type} STREQUAL EXECUTABLE)
-        # operator::new/delete for executables (MemoryTracker stuff)
-        target_link_libraries (${target} PRIVATE clickhouse_new_delete ${MALLOC_LIBRARIES})
+        # disabled for TSAN and gcc since libtsan.a provides overrides too
+        if (TARGET clickhouse_new_delete)
+            # operator::new/delete for executables (MemoryTracker stuff)
+            target_link_libraries (${target} PRIVATE clickhouse_new_delete ${MALLOC_LIBRARIES})
+        endif()
     endif()
 endmacro()
 
@@ -525,8 +580,8 @@ include_directories(${ConfigIncludePath})
 include (cmake/warnings.cmake)
 
 add_subdirectory (base)
-add_subdirectory (programs)
 add_subdirectory (src)
+add_subdirectory (programs)
 add_subdirectory (tests)
 add_subdirectory (utils)
 

LICENSE

@@ -1,4 +1,4 @@
-Copyright 2016-2020 Yandex LLC
+Copyright 2016-2021 Yandex LLC
 
                                  Apache License
                            Version 2.0, January 2004
@@ -188,7 +188,7 @@ Copyright 2016-2020 Yandex LLC
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright 2016-2020 Yandex LLC
+   Copyright 2016-2021 Yandex LLC
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

PreLoad.cmake

@@ -12,7 +12,7 @@
 # https://youtrack.jetbrains.com/issue/CPP-2659
 # https://youtrack.jetbrains.com/issue/CPP-870
 
-if (NOT DEFINED ENV{CLION_IDE})
+if (NOT DEFINED ENV{CLION_IDE} AND NOT DEFINED ENV{XCODE_IDE})
     find_program(NINJA_PATH ninja)
     if (NINJA_PATH)
         set(CMAKE_GENERATOR "Ninja" CACHE INTERNAL "" FORCE)

README.md

@@ -8,12 +8,8 @@ ClickHouse® is an open-source column-oriented database management system that a
 * [Tutorial](https://clickhouse.tech/docs/en/getting_started/tutorial/) shows how to set up and query small ClickHouse cluster.
 * [Documentation](https://clickhouse.tech/docs/en/) provides more in-depth information.
 * [YouTube channel](https://www.youtube.com/c/ClickHouseDB) has a lot of content about ClickHouse in video format.
-* [Slack](https://join.slack.com/t/clickhousedb/shared_invite/zt-d2zxkf9e-XyxDa_ucfPxzuH4SJIm~Ng) and [Telegram](https://telegram.me/clickhouse_en) allow to chat with ClickHouse users in real-time.
+* [Slack](https://join.slack.com/t/clickhousedb/shared_invite/zt-nwwakmk4-xOJ6cdy0sJC3It8j348~IA) and [Telegram](https://telegram.me/clickhouse_en) allow to chat with ClickHouse users in real-time.
 * [Blog](https://clickhouse.yandex/blog/en/) contains various ClickHouse-related articles, as well as announcements and reports about events.
 * [Code Browser](https://clickhouse.tech/codebrowser/html_report/ClickHouse/index.html) with syntax highlight and navigation.
-* [Yandex.Messenger channel](https://yandex.ru/chat/#/join/20e380d9-c7be-4123-ab06-e95fb946975e) shares announcements and useful links in Russian.
 * [Contacts](https://clickhouse.tech/#contacts) can help to get your questions answered if there are any.
 * You can also [fill this form](https://clickhouse.tech/#meet) to meet Yandex ClickHouse team in person.
-
-## Upcoming Events
-* [SF Bay Area ClickHouse Virtual Office Hours (online)](https://www.meetup.com/San-Francisco-Bay-Area-ClickHouse-Meetup/events/274273549/) on 20 January 2020.

SECURITY.md

@@ -15,9 +15,13 @@ currently being supported with security updates:
 | 20.4   | :x: |
 | 20.5   | :x: |
 | 20.6   | :x: |
-| 20.7   | :white_check_mark: |
+| 20.7   | :x: |
 | 20.8   | :white_check_mark: |
-| 20.9   | :white_check_mark: |
+| 20.9   | :x: |
+| 20.10  | :x: |
+| 20.11  | :white_check_mark: |
+| 20.12  | :white_check_mark: |
+| 21.1   | :white_check_mark: |
 
 ## Reporting a Vulnerability
 

base/CMakeLists.txt

@@ -8,6 +8,7 @@ add_subdirectory (loggers)
 add_subdirectory (pcg-random)
 add_subdirectory (widechar_width)
 add_subdirectory (readpassphrase)
+add_subdirectory (bridge)
 
 if (USE_MYSQL)
     add_subdirectory (mysqlxx)

base/bridge/CMakeLists.txt

@@ -0,0 +1,7 @@
+add_library (bridge
+    IBridge.cpp
+)
+
+target_include_directories (daemon PUBLIC ..)
+target_link_libraries (bridge PRIVATE daemon dbms Poco::Data Poco::Data::ODBC)
+

base/bridge/IBridge.cpp

@@ -0,0 +1,238 @@
+#include "IBridge.h"
+
+#include <IO/ReadHelpers.h>
+#include <boost/program_options.hpp>
+#include <Poco/Net/NetException.h>
+#include <Poco/Util/HelpFormatter.h>
+#include <Common/StringUtils/StringUtils.h>
+#include <Formats/registerFormats.h>
+#include <common/logger_useful.h>
+#include <Common/SensitiveDataMasker.h>
+#include <Server/HTTP/HTTPServer.h>
+
+#if USE_ODBC
+#    include <Poco/Data/ODBC/Connector.h>
+#endif
+
+
+namespace DB
+{
+
+namespace ErrorCodes
+{
+    extern const int ARGUMENT_OUT_OF_BOUND;
+}
+
+namespace
+{
+    Poco::Net::SocketAddress makeSocketAddress(const std::string & host, UInt16 port, Poco::Logger * log)
+    {
+        Poco::Net::SocketAddress socket_address;
+        try
+        {
+            socket_address = Poco::Net::SocketAddress(host, port);
+        }
+        catch (const Poco::Net::DNSException & e)
+        {
+            const auto code = e.code();
+            if (code == EAI_FAMILY
+#if defined(EAI_ADDRFAMILY)
+                || code == EAI_ADDRFAMILY
+#endif
+            )
+            {
+                LOG_ERROR(log, "Cannot resolve listen_host ({}), error {}: {}. If it is an IPv6 address and your host has disabled IPv6, then consider to specify IPv4 address to listen in <listen_host> element of configuration file. Example: <listen_host>0.0.0.0</listen_host>", host, e.code(), e.message());
+            }
+
+            throw;
+        }
+        return socket_address;
+    }
+
+    Poco::Net::SocketAddress socketBindListen(Poco::Net::ServerSocket & socket, const std::string & host, UInt16 port, Poco::Logger * log)
+    {
+        auto address = makeSocketAddress(host, port, log);
+#if POCO_VERSION < 0x01080000
+        socket.bind(address, /* reuseAddress = */ true);
+#else
+        socket.bind(address, /* reuseAddress = */ true, /* reusePort = */ false);
+#endif
+
+        socket.listen(/* backlog = */ 64);
+
+        return address;
+    }
+}
+
+
+void IBridge::handleHelp(const std::string &, const std::string &)
+{
+    Poco::Util::HelpFormatter help_formatter(options());
+    help_formatter.setCommand(commandName());
+    help_formatter.setHeader("HTTP-proxy for odbc requests");
+    help_formatter.setUsage("--http-port <port>");
+    help_formatter.format(std::cerr);
+
+    stopOptionsProcessing();
+}
+
+
+void IBridge::defineOptions(Poco::Util::OptionSet & options)
+{
+    options.addOption(
+        Poco::Util::Option("http-port", "", "port to listen").argument("http-port", true) .binding("http-port"));
+
+    options.addOption(
+        Poco::Util::Option("listen-host", "", "hostname or address to listen, default 127.0.0.1").argument("listen-host").binding("listen-host"));
+
+    options.addOption(
+        Poco::Util::Option("http-timeout", "", "http timeout for socket, default 1800").argument("http-timeout").binding("http-timeout"));
+
+    options.addOption(
+        Poco::Util::Option("max-server-connections", "", "max connections to server, default 1024").argument("max-server-connections").binding("max-server-connections"));
+
+    options.addOption(
+        Poco::Util::Option("keep-alive-timeout", "", "keepalive timeout, default 10").argument("keep-alive-timeout").binding("keep-alive-timeout"));
+
+    options.addOption(
+        Poco::Util::Option("log-level", "", "sets log level, default info") .argument("log-level").binding("logger.level"));
+
+    options.addOption(
+        Poco::Util::Option("log-path", "", "log path for all logs, default console").argument("log-path").binding("logger.log"));
+
+    options.addOption(
+        Poco::Util::Option("err-log-path", "", "err log path for all logs, default no").argument("err-log-path").binding("logger.errorlog"));
+
+    options.addOption(
+        Poco::Util::Option("stdout-path", "", "stdout log path, default console").argument("stdout-path").binding("logger.stdout"));
+
+    options.addOption(
+        Poco::Util::Option("stderr-path", "", "stderr log path, default console").argument("stderr-path").binding("logger.stderr"));
+
+    using Me = std::decay_t<decltype(*this)>;
+
+    options.addOption(
+        Poco::Util::Option("help", "", "produce this help message").binding("help").callback(Poco::Util::OptionCallback<Me>(this, &Me::handleHelp)));
+
+    ServerApplication::defineOptions(options); // NOLINT Don't need complex BaseDaemon's .xml config
+}
+
+
+void IBridge::initialize(Application & self)
+{
+    BaseDaemon::closeFDs();
+    is_help = config().has("help");
+
+    if (is_help)
+        return;
+
+    config().setString("logger", bridgeName());
+
+    /// Redirect stdout, stderr to specified files.
+    /// Some libraries and sanitizers write to stderr in case of errors.
+    const auto stdout_path = config().getString("logger.stdout", "");
+    if (!stdout_path.empty())
+    {
+        if (!freopen(stdout_path.c_str(), "a+", stdout))
+            throw Poco::OpenFileException("Cannot attach stdout to " + stdout_path);
+
+        /// Disable buffering for stdout.
+        setbuf(stdout, nullptr);
+    }
+    const auto stderr_path = config().getString("logger.stderr", "");
+    if (!stderr_path.empty())
+    {
+        if (!freopen(stderr_path.c_str(), "a+", stderr))
+            throw Poco::OpenFileException("Cannot attach stderr to " + stderr_path);
+
+        /// Disable buffering for stderr.
+        setbuf(stderr, nullptr);
+    }
+
+    buildLoggers(config(), logger(), self.commandName());
+
+    BaseDaemon::logRevision();
+
+    log = &logger();
+    hostname = config().getString("listen-host", "127.0.0.1");
+    port = config().getUInt("http-port");
+    if (port > 0xFFFF)
+        throw Exception("Out of range 'http-port': " + std::to_string(port), ErrorCodes::ARGUMENT_OUT_OF_BOUND);
+
+    http_timeout = config().getUInt("http-timeout", DEFAULT_HTTP_READ_BUFFER_TIMEOUT);
+    max_server_connections = config().getUInt("max-server-connections", 1024);
+    keep_alive_timeout = config().getUInt("keep-alive-timeout", 10);
+
+    initializeTerminationAndSignalProcessing();
+
+#if USE_ODBC
+    if (bridgeName() == "ODBCBridge")
+        Poco::Data::ODBC::Connector::registerConnector();
+#endif
+
+    ServerApplication::initialize(self); // NOLINT
+}
+
+
+void IBridge::uninitialize()
+{
+    BaseDaemon::uninitialize();
+}
+
+
+int IBridge::main(const std::vector<std::string> & /*args*/)
+{
+    if (is_help)
+        return Application::EXIT_OK;
+
+    registerFormats();
+    LOG_INFO(log, "Starting up {} on host: {}, port: {}", bridgeName(), hostname, port);
+
+    Poco::Net::ServerSocket socket;
+    auto address = socketBindListen(socket, hostname, port, log);
+    socket.setReceiveTimeout(http_timeout);
+    socket.setSendTimeout(http_timeout);
+
+    Poco::ThreadPool server_pool(3, max_server_connections);
+
+    Poco::Net::HTTPServerParams::Ptr http_params = new Poco::Net::HTTPServerParams;
+    http_params->setTimeout(http_timeout);
+    http_params->setKeepAliveTimeout(keep_alive_timeout);
+
+    auto shared_context = Context::createShared();
+    Context context(Context::createGlobal(shared_context.get()));
+    context.makeGlobalContext();
+
+    if (config().has("query_masking_rules"))
+        SensitiveDataMasker::setInstance(std::make_unique<SensitiveDataMasker>(config(), "query_masking_rules"));
+
+    auto server = HTTPServer(
+        context,
+        getHandlerFactoryPtr(context),
+        server_pool,
+        socket,
+        http_params);
+
+    SCOPE_EXIT({
+        LOG_DEBUG(log, "Received termination signal.");
+        LOG_DEBUG(log, "Waiting for current connections to close.");
+
+        server.stop();
+
+        for (size_t count : ext::range(1, 6))
+        {
+            if (server.currentConnections() == 0)
+                break;
+            LOG_DEBUG(log, "Waiting for {} connections, try {}", server.currentConnections(), count);
+            std::this_thread::sleep_for(std::chrono::milliseconds(1000));
+        }
+    });
+
+    server.start();
+    LOG_INFO(log, "Listening http://{}", address.toString());
+
+    waitForTerminationRequest();
+    return Application::EXIT_OK;
+}
+
+}

base/bridge/IBridge.h

@@ -0,0 +1,50 @@
+#pragma once
+
+#include <Interpreters/Context.h>
+#include <Server/HTTP/HTTPRequestHandlerFactory.h>
+#include <Poco/Util/ServerApplication.h>
+#include <Poco/Logger.h>
+#include <daemon/BaseDaemon.h>
+
+
+namespace DB
+{
+
+/// Class represents base for clickhouse-odbc-bridge and clickhouse-library-bridge servers.
+/// Listens to incoming HTTP POST and GET requests on specified port and host.
+/// Has two handlers '/' for all incoming POST requests and /ping for GET request about service status.
+class IBridge : public BaseDaemon
+{
+
+public:
+    /// Define command line arguments
+    void defineOptions(Poco::Util::OptionSet & options) override;
+
+protected:
+    using HandlerFactoryPtr = std::shared_ptr<HTTPRequestHandlerFactory>;
+
+    void initialize(Application & self) override;
+
+    void uninitialize() override;
+
+    int main(const std::vector<std::string> & args) override;
+
+    virtual const std::string bridgeName() const = 0;
+
+    virtual HandlerFactoryPtr getHandlerFactoryPtr(Context & context) const = 0;
+
+    size_t keep_alive_timeout;
+
+private:
+    void handleHelp(const std::string &, const std::string &);
+
+    bool is_help;
+    std::string hostname;
+    size_t port;
+    std::string log_level;
+    size_t max_server_connections;
+    size_t http_timeout;
+
+    Poco::Logger * log;
+};
+}

base/common/CMakeLists.txt

@@ -47,6 +47,10 @@ endif()
 
 target_include_directories(common PUBLIC .. ${CMAKE_CURRENT_BINARY_DIR}/..)
 
+if (OS_DARWIN AND NOT MAKE_STATIC_LIBRARIES)
+    target_link_libraries(common PUBLIC -Wl,-U,_inside_main)
+endif()
+
 # Allow explicit fallback to readline
 if (NOT ENABLE_REPLXX AND ENABLE_READLINE)
     message (STATUS "Attempt to fallback to readline explicitly")
@@ -74,7 +78,6 @@ target_link_libraries (common
         ${CITYHASH_LIBRARIES}
         boost::headers_only
         boost::system
-        FastMemcpy
         Poco::Net
         Poco::Net::SSL
         Poco::Util

base/common/DateLUT.cpp

@@ -152,7 +152,7 @@ const DateLUTImpl & DateLUT::getImplementation(const std::string & time_zone) co
 
     auto it = impls.emplace(time_zone, nullptr).first;
     if (!it->second)
-        it->second = std::make_unique<DateLUTImpl>(time_zone);
+        it->second = std::unique_ptr<DateLUTImpl>(new DateLUTImpl(time_zone));
 
     return *it->second;
 }

base/common/DateLUT.h

@@ -32,7 +32,6 @@ public:
 
         return date_lut.getImplementation(time_zone);
     }
-
     static void setDefaultTimezone(const std::string & time_zone)
     {
         auto & date_lut = getInstance();

base/common/DateLUTImpl.cpp

@@ -46,24 +46,41 @@ DateLUTImpl::DateLUTImpl(const std::string & time_zone_)
     if (&inside_main)
         assert(inside_main);
 
-    size_t i = 0;
-    time_t start_of_day = 0;
-
     cctz::time_zone cctz_time_zone;
     if (!cctz::load_time_zone(time_zone, &cctz_time_zone))
         throw Poco::Exception("Cannot load time zone " + time_zone_);
 
-    cctz::time_zone::absolute_lookup start_of_epoch_lookup = cctz_time_zone.lookup(std::chrono::system_clock::from_time_t(start_of_day));
-    offset_at_start_of_epoch = start_of_epoch_lookup.offset;
-    offset_is_whole_number_of_hours_everytime = true;
+    constexpr cctz::civil_day epoch{1970, 1, 1};
+    constexpr cctz::civil_day lut_start{DATE_LUT_MIN_YEAR, 1, 1};
+    time_t start_of_day;
 
-    cctz::civil_day date{1970, 1, 1};
+    /// Note: it's validated against all timezones in the system.
+    static_assert((epoch - lut_start) == daynum_offset_epoch);
 
+    offset_at_start_of_epoch = cctz_time_zone.lookup(cctz_time_zone.lookup(epoch).pre).offset;
+    offset_at_start_of_lut = cctz_time_zone.lookup(cctz_time_zone.lookup(lut_start).pre).offset;
+    offset_is_whole_number_of_hours_during_epoch = true;
+
+    cctz::civil_day date = lut_start;
+
+    UInt32 i = 0;
     do
     {
         cctz::time_zone::civil_lookup lookup = cctz_time_zone.lookup(date);
 
-        start_of_day = std::chrono::system_clock::to_time_t(lookup.pre);    /// Ambiguity is possible.
+        /// Ambiguity is possible if time was changed backwards at the midnight
+        /// or after midnight time has been changed back to midnight, for example one hour backwards at 01:00
+        /// or after midnight time has been changed to the previous day, for example two hours backwards at 01:00
+        /// Then midnight appears twice. Usually time change happens exactly at 00:00 or 01:00.
+
+        /// If transition did not involve previous day, we should use the first midnight as the start of the day,
+        /// otherwise it's better to use the second midnight.
+
+        std::chrono::time_point start_of_day_time_point = lookup.trans < lookup.post
+            ? lookup.post /* Second midnight appears after transition, so there was a piece of previous day after transition */
+            : lookup.pre;
+
+        start_of_day = std::chrono::system_clock::to_time_t(start_of_day_time_point);
 
         Values & values = lut[i];
         values.year = date.year();
@@ -72,7 +89,7 @@ DateLUTImpl::DateLUTImpl(const std::string & time_zone_)
         values.day_of_week = getDayOfWeek(date);
         values.date = start_of_day;
 
-        assert(values.year >= DATE_LUT_MIN_YEAR && values.year <= DATE_LUT_MAX_YEAR);
+        assert(values.year >= DATE_LUT_MIN_YEAR && values.year <= DATE_LUT_MAX_YEAR + 1);
         assert(values.month >= 1 && values.month <= 12);
         assert(values.day_of_month >= 1 && values.day_of_month <= 31);
         assert(values.day_of_week >= 1 && values.day_of_week <= 7);
@@ -85,50 +102,42 @@ DateLUTImpl::DateLUTImpl(const std::string & time_zone_)
         else
             values.days_in_month = i != 0 ? lut[i - 1].days_in_month : 31;
 
-        values.time_at_offset_change = 0;
-        values.amount_of_offset_change = 0;
+        values.time_at_offset_change_value = 0;
+        values.amount_of_offset_change_value = 0;
 
-        if (start_of_day % 3600)
-            offset_is_whole_number_of_hours_everytime = false;
+        if (offset_is_whole_number_of_hours_during_epoch && start_of_day > 0 && start_of_day % 3600)
+            offset_is_whole_number_of_hours_during_epoch = false;
 
-        /// If UTC offset was changed in previous day.
-        if (i != 0)
+        /// If UTC offset was changed this day.
+        /// Change in time zone without transition is possible, e.g. Moscow 1991 Sun, 31 Mar, 02:00 MSK to EEST
+        cctz::time_zone::civil_transition transition{};
+        if (cctz_time_zone.next_transition(start_of_day_time_point - std::chrono::seconds(1), &transition)
+            && (cctz::civil_day(transition.from) == date || cctz::civil_day(transition.to) == date)
+            && transition.from != transition.to)
         {
-            auto amount_of_offset_change_at_prev_day = 86400 - (lut[i].date - lut[i - 1].date);
-            if (amount_of_offset_change_at_prev_day)
-            {
-                lut[i - 1].amount_of_offset_change = amount_of_offset_change_at_prev_day;
+            values.time_at_offset_change_value = (transition.from - cctz::civil_second(date)) / Values::OffsetChangeFactor;
+            values.amount_of_offset_change_value = (transition.to - transition.from) / Values::OffsetChangeFactor;
 
-                const auto utc_offset_at_beginning_of_day = cctz_time_zone.lookup(std::chrono::system_clock::from_time_t(lut[i - 1].date)).offset;
+//            std::cerr << time_zone << ", " << date << ": change from " << transition.from << " to " << transition.to << "\n";
+//            std::cerr << time_zone << ", " << date << ": change at " << values.time_at_offset_change() << " with " << values.amount_of_offset_change() << "\n";
 
-                /// Find a time (timestamp offset from beginning of day),
-                ///  when UTC offset was changed. Search is performed with 15-minute granularity, assuming it is enough.
+            /// We don't support too large changes.
+            if (values.amount_of_offset_change_value > 24 * 4)
+                values.amount_of_offset_change_value = 24 * 4;
+            else if (values.amount_of_offset_change_value < -24 * 4)
+                values.amount_of_offset_change_value = -24 * 4;
 
-                time_t time_at_offset_change = 900;
-                while (time_at_offset_change < 86400)
-                {
-                    auto utc_offset_at_current_time = cctz_time_zone.lookup(std::chrono::system_clock::from_time_t(
-                        lut[i - 1].date + time_at_offset_change)).offset;
-
-                    if (utc_offset_at_current_time != utc_offset_at_beginning_of_day)
-                        break;
-
-                    time_at_offset_change += 900;
-                }
-
-                lut[i - 1].time_at_offset_change = time_at_offset_change;
-
-                /// We doesn't support cases when time change results in switching to previous day.
-                if (static_cast<int>(lut[i - 1].time_at_offset_change) + static_cast<int>(lut[i - 1].amount_of_offset_change) < 0)
-                    lut[i - 1].time_at_offset_change = -lut[i - 1].amount_of_offset_change;
-            }
+            /// We don't support cases when time change results in switching to previous day.
+            /// Shift the point of time change later.
+            if (values.time_at_offset_change_value + values.amount_of_offset_change_value < 0)
+                values.time_at_offset_change_value = -values.amount_of_offset_change_value;
         }
 
         /// Going to next day.
         ++date;
         ++i;
     }
-    while (start_of_day <= DATE_LUT_MAX && i <= DATE_LUT_MAX_DAY_NUM);
+    while (i < DATE_LUT_SIZE && lut[i - 1].year <= DATE_LUT_MAX_YEAR);
 
     /// Fill excessive part of lookup table. This is needed only to simplify handling of overflow cases.
     while (i < DATE_LUT_SIZE)

base/common/DayNum.h

@@ -7,3 +7,8 @@
   * See DateLUTImpl for usage examples.
   */
 STRONG_TYPEDEF(UInt16, DayNum)
+
+/** Represent number of days since 1970-01-01 but in extended range,
+ * for dates before 1970-01-01 and after 2105
+ */
+STRONG_TYPEDEF(Int32, ExtendedDayNum)

base/common/LocalDate.h

@@ -92,20 +92,10 @@ public:
     LocalDate(const LocalDate &) noexcept = default;
     LocalDate & operator= (const LocalDate &) noexcept = default;
 
-    LocalDate & operator= (time_t time)
-    {
-        init(time);
-        return *this;
-    }
-
-    operator time_t() const
-    {
-        return DateLUT::instance().makeDate(m_year, m_month, m_day);
-    }
-
     DayNum getDayNum() const
     {
-        return DateLUT::instance().makeDayNum(m_year, m_month, m_day);
+        const auto & lut = DateLUT::instance();
+        return DayNum(lut.makeDayNum(m_year, m_month, m_day).toUnderType());
     }
 
     operator DayNum() const
@@ -166,20 +156,3 @@ public:
 };
 
 static_assert(sizeof(LocalDate) == 4);
-
-
-inline std::ostream & operator<< (std::ostream & ostr, const LocalDate & date)
-{
-    return ostr << date.year()
-        << '-' << (date.month() / 10) << (date.month() % 10)
-        << '-' << (date.day() / 10) << (date.day() % 10);
-}
-
-
-namespace std
-{
-inline string to_string(const LocalDate & date)
-{
-    return date.toString();
-}
-}

base/common/LocalDateTime.h

@@ -29,29 +29,16 @@ private:
     /// NOTE We may use attribute packed instead, but it is less portable.
     unsigned char pad = 0;
 
-    void init(time_t time)
+    void init(time_t time, const DateLUTImpl & time_zone)
     {
-        if (unlikely(time > DATE_LUT_MAX || time == 0))
-        {
-            m_year = 0;
-            m_month = 0;
-            m_day = 0;
-            m_hour = 0;
-            m_minute = 0;
-            m_second = 0;
+        DateLUTImpl::DateTimeComponents components = time_zone.toDateTimeComponents(time);
 
-            return;
-        }
-
-        const auto & date_lut = DateLUT::instance();
-        const auto & values = date_lut.getValues(time);
-
-        m_year = values.year;
-        m_month = values.month;
-        m_day = values.day_of_month;
-        m_hour = date_lut.toHour(time);
-        m_minute = date_lut.toMinute(time);
-        m_second = date_lut.toSecond(time);
+        m_year = components.date.year;
+        m_month = components.date.month;
+        m_day = components.date.day;
+        m_hour = components.time.hour;
+        m_minute = components.time.minute;
+        m_second = components.time.second;
 
         (void)pad;  /// Suppress unused private field warning.
     }
@@ -73,9 +60,9 @@ private:
     }
 
 public:
-    explicit LocalDateTime(time_t time)
+    explicit LocalDateTime(time_t time, const DateLUTImpl & time_zone = DateLUT::instance())
     {
-        init(time);
+        init(time, time_zone);
     }
 
     LocalDateTime(unsigned short year_, unsigned char month_, unsigned char day_,
@@ -104,19 +91,6 @@ public:
     LocalDateTime(const LocalDateTime &) noexcept = default;
     LocalDateTime & operator= (const LocalDateTime &) noexcept = default;
 
-    LocalDateTime & operator= (time_t time)
-    {
-        init(time);
-        return *this;
-    }
-
-    operator time_t() const
-    {
-        return m_year == 0
-            ? 0
-            : DateLUT::instance().makeDateTime(m_year, m_month, m_day, m_hour, m_minute, m_second);
-    }
-
     unsigned short year() const { return m_year; }
     unsigned char month() const { return m_month; }
     unsigned char day() const { return m_day; }
@@ -132,8 +106,30 @@ public:
     void second(unsigned char x) { m_second = x; }
 
     LocalDate toDate() const { return LocalDate(m_year, m_month, m_day); }
+    LocalDateTime toStartOfDate() const { return LocalDateTime(m_year, m_month, m_day, 0, 0, 0); }
 
-    LocalDateTime toStartOfDate() { return LocalDateTime(m_year, m_month, m_day, 0, 0, 0); }
+    std::string toString() const
+    {
+        std::string s{"0000-00-00 00:00:00"};
+
+        s[0] += m_year / 1000;
+        s[1] += (m_year / 100) % 10;
+        s[2] += (m_year / 10) % 10;
+        s[3] += m_year % 10;
+        s[5] += m_month / 10;
+        s[6] += m_month % 10;
+        s[8] += m_day / 10;
+        s[9] += m_day % 10;
+
+        s[11] += m_hour / 10;
+        s[12] += m_hour % 10;
+        s[14] += m_minute / 10;
+        s[15] += m_minute % 10;
+        s[17] += m_second / 10;
+        s[18] += m_second % 10;
+
+        return s;
+    }
 
     bool operator< (const LocalDateTime & other) const
     {
@@ -167,28 +163,3 @@ public:
 };
 
 static_assert(sizeof(LocalDateTime) == 8);
-
-
-inline std::ostream & operator<< (std::ostream & ostr, const LocalDateTime & datetime)
-{
-    ostr << std::setfill('0') << std::setw(4) << datetime.year();
-
-    ostr << '-' << (datetime.month() / 10) << (datetime.month() % 10)
-        << '-' << (datetime.day() / 10) << (datetime.day() % 10)
-        << ' ' << (datetime.hour() / 10) << (datetime.hour() % 10)
-        << ':' << (datetime.minute() / 10) << (datetime.minute() % 10)
-        << ':' << (datetime.second() / 10) << (datetime.second() % 10);
-
-    return ostr;
-}
-
-
-namespace std
-{
-inline string to_string(const LocalDateTime & datetime)
-{
-    stringstream str;
-    str << datetime;
-    return str.str();
-}
-}

base/common/ReplxxLineReader.cpp

@@ -6,6 +6,14 @@
 #include <unistd.h>
 #include <functional>
 #include <sys/file.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <csignal>
+#include <dlfcn.h>
+#include <fcntl.h>
+#include <fstream>
+#include <fmt/format.h>
+
 
 namespace
 {
@@ -83,6 +91,8 @@ ReplxxLineReader::ReplxxLineReader(
     /// it also binded to M-p/M-n).
     rx.bind_key(Replxx::KEY::meta('N'), [this](char32_t code) { return rx.invoke(Replxx::ACTION::COMPLETE_NEXT, code); });
     rx.bind_key(Replxx::KEY::meta('P'), [this](char32_t code) { return rx.invoke(Replxx::ACTION::COMPLETE_PREVIOUS, code); });
+
+    rx.bind_key(Replxx::KEY::meta('E'), [this](char32_t) { openEditor(); return Replxx::ACTION_RESULT::CONTINUE; });
 }
 
 ReplxxLineReader::~ReplxxLineReader()
@@ -127,7 +137,114 @@ void ReplxxLineReader::addToHistory(const String & line)
         rx.print("Unlock of history file failed: %s\n", errnoToString(errno).c_str());
 }
 
+int ReplxxLineReader::execute(const std::string & command)
+{
+    std::vector<char> argv0("sh", &("sh"[3]));
+    std::vector<char> argv1("-c", &("-c"[3]));
+    std::vector<char> argv2(command.data(), command.data() + command.size() + 1);
+
+    const char * filename = "/bin/sh";
+    char * const argv[] = {argv0.data(), argv1.data(), argv2.data(), nullptr};
+
+    static void * real_vfork = dlsym(RTLD_DEFAULT, "vfork");
+    if (!real_vfork)
+    {
+        rx.print("Cannot find symbol vfork in myself: %s\n", errnoToString(errno).c_str());
+        return -1;
+    }
+
+    pid_t pid = reinterpret_cast<pid_t (*)()>(real_vfork)();
+
+    if (-1 == pid)
+    {
+        rx.print("Cannot vfork: %s\n", errnoToString(errno).c_str());
+        return -1;
+    }
+
+    if (0 == pid)
+    {
+        sigset_t mask;
+        sigemptyset(&mask);
+        sigprocmask(0, nullptr, &mask);
+        sigprocmask(SIG_UNBLOCK, &mask, nullptr);
+
+        execv(filename, argv);
+        _exit(-1);
+    }
+
+    int status = 0;
+    if (-1 == waitpid(pid, &status, 0))
+    {
+        rx.print("Cannot waitpid: %s\n", errnoToString(errno).c_str());
+        return -1;
+    }
+    return status;
+}
+
+void ReplxxLineReader::openEditor()
+{
+    char filename[] = "clickhouse_replxx_XXXXXX.sql";
+    int fd = ::mkstemps(filename, 4);
+    if (-1 == fd)
+    {
+        rx.print("Cannot create temporary file to edit query: %s\n", errnoToString(errno).c_str());
+        return;
+    }
+
+    const char * editor = std::getenv("EDITOR");
+    if (!editor || !*editor)
+        editor = "vim";
+
+    replxx::Replxx::State state(rx.get_state());
+
+    size_t bytes_written = 0;
+    const char * begin = state.text();
+    size_t offset = strlen(state.text());
+    while (bytes_written != offset)
+    {
+        ssize_t res = ::write(fd, begin + bytes_written, offset - bytes_written);
+        if ((-1 == res || 0 == res) && errno != EINTR)
+        {
+            rx.print("Cannot write to temporary query file %s: %s\n", filename, errnoToString(errno).c_str());
+            break;
+        }
+        bytes_written += res;
+    }
+
+    if (0 != ::close(fd))
+    {
+        rx.print("Cannot close temporary query file %s: %s\n", filename, errnoToString(errno).c_str());
+        return;
+    }
+
+    if (0 == execute(fmt::format("{} {}", editor, filename)))
+    {
+        try
+        {
+            std::ifstream t(filename);
+            std::string str;
+            t.seekg(0, std::ios::end);
+            str.reserve(t.tellg());
+            t.seekg(0, std::ios::beg);
+            str.assign((std::istreambuf_iterator<char>(t)), std::istreambuf_iterator<char>());
+            rx.set_state(replxx::Replxx::State(str.c_str(), str.size()));
+        }
+        catch (...)
+        {
+            rx.print("Cannot read from temporary query file %s: %s\n", filename, errnoToString(errno).c_str());
+            return;
+        }
+    }
+
+    if (bracketed_paste_enabled)
+        enableBracketedPaste();
+
+    if (0 != ::unlink(filename))
+        rx.print("Cannot remove temporary query file %s: %s\n", filename, errnoToString(errno).c_str());
+}
+
 void ReplxxLineReader::enableBracketedPaste()
 {
+    bracketed_paste_enabled = true;
     rx.enable_bracketed_paste();
 };

base/common/ReplxxLineReader.h

@@ -22,10 +22,13 @@ public:
 private:
     InputStatus readOneLine(const String & prompt) override;
     void addToHistory(const String & line) override;
+    int execute(const std::string & command);
+    void openEditor();
 
     replxx::Replxx rx;
     replxx::Replxx::highlighter_callback_t highlighter;
 
     // used to call flock() to synchronize multiple clients using same history file
     int history_file_fd = -1;
+    bool bracketed_paste_enabled = false;
 };

base/common/arithmeticOverflow.h

@@ -1,9 +1,36 @@
 #pragma once
 
 #include <common/extended_types.h>
+#include <common/defines.h>
+
 
 namespace common
 {
+    /// Multiply and ignore overflow.
+    template <typename T1, typename T2>
+    inline auto NO_SANITIZE_UNDEFINED mulIgnoreOverflow(T1 x, T2 y)
+    {
+        return x * y;
+    }
+
+    template <typename T1, typename T2>
+    inline auto NO_SANITIZE_UNDEFINED addIgnoreOverflow(T1 x, T2 y)
+    {
+        return x + y;
+    }
+
+    template <typename T1, typename T2>
+    inline auto NO_SANITIZE_UNDEFINED subIgnoreOverflow(T1 x, T2 y)
+    {
+        return x - y;
+    }
+
+    template <typename T>
+    inline auto NO_SANITIZE_UNDEFINED negateIgnoreOverflow(T x)
+    {
+        return -x;
+    }
+
     template <typename T>
     inline bool addOverflow(T x, T y, T & res)
     {
@@ -33,14 +60,14 @@ namespace common
     {
         static constexpr __int128 min_int128 = minInt128();
         static constexpr __int128 max_int128 = maxInt128();
-        res = x + y;
+        res = addIgnoreOverflow(x, y);
         return (y > 0 && x > max_int128 - y) || (y < 0 && x < min_int128 - y);
     }
 
     template <>
     inline bool addOverflow(wInt256 x, wInt256 y, wInt256 & res)
     {
-        res = x + y;
+        res = addIgnoreOverflow(x, y);
         return (y > 0 && x > std::numeric_limits<wInt256>::max() - y) ||
             (y < 0 && x < std::numeric_limits<wInt256>::min() - y);
     }
@@ -48,7 +75,7 @@ namespace common
     template <>
     inline bool addOverflow(wUInt256 x, wUInt256 y, wUInt256 & res)
     {
-        res = x + y;
+        res = addIgnoreOverflow(x, y);
         return x > std::numeric_limits<wUInt256>::max() - y;
     }
 
@@ -81,14 +108,14 @@ namespace common
     {
         static constexpr __int128 min_int128 = minInt128();
         static constexpr __int128 max_int128 = maxInt128();
-        res = x - y;
+        res = subIgnoreOverflow(x, y);
         return (y < 0 && x > max_int128 + y) || (y > 0 && x < min_int128 + y);
     }
 
     template <>
     inline bool subOverflow(wInt256 x, wInt256 y, wInt256 & res)
     {
-        res = x - y;
+        res = subIgnoreOverflow(x, y);
         return (y < 0 && x > std::numeric_limits<wInt256>::max() + y) ||
             (y > 0 && x < std::numeric_limits<wInt256>::min() + y);
     }
@@ -96,7 +123,7 @@ namespace common
     template <>
     inline bool subOverflow(wUInt256 x, wUInt256 y, wUInt256 & res)
     {
-        res = x - y;
+        res = subIgnoreOverflow(x, y);
         return x < y;
     }
 
@@ -127,33 +154,33 @@ namespace common
     template <>
     inline bool mulOverflow(__int128 x, __int128 y, __int128 & res)
     {
-        res = static_cast<unsigned __int128>(x) * static_cast<unsigned __int128>(y);    /// Avoid signed integer overflow.
+        res = mulIgnoreOverflow(x, y);
         if (!x || !y)
             return false;
 
         unsigned __int128 a = (x > 0) ? x : -x;
         unsigned __int128 b = (y > 0) ? y : -y;
-        return (a * b) / b != a;
+        return mulIgnoreOverflow(a, b) / b != a;
     }
 
     template <>
     inline bool mulOverflow(wInt256 x, wInt256 y, wInt256 & res)
     {
-        res = x * y;
+        res = mulIgnoreOverflow(x, y);
         if (!x || !y)
             return false;
 
         wInt256 a = (x > 0) ? x : -x;
         wInt256 b = (y > 0) ? y : -y;
-        return (a * b) / b != a;
+        return mulIgnoreOverflow(a, b) / b != a;
     }
 
     template <>
     inline bool mulOverflow(wUInt256 x, wUInt256 y, wUInt256 & res)
     {
-        res = x * y;
+        res = mulIgnoreOverflow(x, y);
         if (!x || !y)
             return false;
-        return (x * y) / y != x;
+        return res / y != x;
     }
 }

base/common/defines.h

@@ -1,5 +1,20 @@
 #pragma once
 
+/// __has_feature supported only by clang.
+///
+/// But libcxx/libcxxabi overrides it to 0,
+/// thus the checks for __has_feature will be wrong.
+///
+/// NOTE:
+/// - __has_feature cannot be simply undefined,
+///   since this will be broken if some C++ header will be included after
+///   including <common/defines.h>
+/// - it should not have fallback to 0,
+///   since this may create false-positive detection (common problem)
+#if defined(__clang__) && defined(__has_feature)
+#    define ch_has_feature __has_feature
+#endif
+
 #if defined(_MSC_VER)
 #   if !defined(likely)
 #      define likely(x)   (x)
@@ -32,8 +47,8 @@
 
 /// Check for presence of address sanitizer
 #if !defined(ADDRESS_SANITIZER)
-#    if defined(__has_feature)
-#        if __has_feature(address_sanitizer)
+#    if defined(ch_has_feature)
+#        if ch_has_feature(address_sanitizer)
 #            define ADDRESS_SANITIZER 1
 #        endif
 #    elif defined(__SANITIZE_ADDRESS__)
@@ -42,8 +57,8 @@
 #endif
 
 #if !defined(THREAD_SANITIZER)
-#    if defined(__has_feature)
-#        if __has_feature(thread_sanitizer)
+#    if defined(ch_has_feature)
+#        if ch_has_feature(thread_sanitizer)
 #            define THREAD_SANITIZER 1
 #        endif
 #    elif defined(__SANITIZE_THREAD__)
@@ -52,8 +67,8 @@
 #endif
 
 #if !defined(MEMORY_SANITIZER)
-#    if defined(__has_feature)
-#        if __has_feature(memory_sanitizer)
+#    if defined(ch_has_feature)
+#        if ch_has_feature(memory_sanitizer)
 #            define MEMORY_SANITIZER 1
 #        endif
 #    elif defined(__MEMORY_SANITIZER__)
@@ -61,6 +76,30 @@
 #    endif
 #endif
 
+#if !defined(UNDEFINED_BEHAVIOR_SANITIZER)
+#    if defined(__has_feature)
+#        if __has_feature(undefined_behavior_sanitizer)
+#            define UNDEFINED_BEHAVIOR_SANITIZER 1
+#        endif
+#    elif defined(__UNDEFINED_BEHAVIOR_SANITIZER__)
+#        define UNDEFINED_BEHAVIOR_SANITIZER 1
+#    endif
+#endif
+
+#if defined(ADDRESS_SANITIZER)
+#    define BOOST_USE_ASAN 1
+#    define BOOST_USE_UCONTEXT 1
+#endif
+
+#if defined(THREAD_SANITIZER)
+#    define BOOST_USE_TSAN 1
+#    define BOOST_USE_UCONTEXT 1
+#endif
+
+#if defined(ARCADIA_BUILD) && defined(BOOST_USE_UCONTEXT)
+#    undef BOOST_USE_UCONTEXT
+#endif
+
 /// TODO: Strange enough, there is no way to detect UB sanitizer.
 
 /// Explicitly allow undefined behaviour for certain functions. Use it as a function attribute.
@@ -70,12 +109,16 @@
 #    define NO_SANITIZE_UNDEFINED __attribute__((__no_sanitize__("undefined")))
 #    define NO_SANITIZE_ADDRESS __attribute__((__no_sanitize__("address")))
 #    define NO_SANITIZE_THREAD __attribute__((__no_sanitize__("thread")))
+#    define ALWAYS_INLINE_NO_SANITIZE_UNDEFINED __attribute__((__always_inline__, __no_sanitize__("undefined")))
 #else  /// It does not work in GCC. GCC 7 cannot recognize this attribute and GCC 8 simply ignores it.
 #    define NO_SANITIZE_UNDEFINED
 #    define NO_SANITIZE_ADDRESS
 #    define NO_SANITIZE_THREAD
+#    define ALWAYS_INLINE_NO_SANITIZE_UNDEFINED ALWAYS_INLINE
 #endif
 
-/// A macro for suppressing warnings about unused variables or function results.
-/// Useful for structured bindings which have no standard way to declare this.
-#define UNUSED(...) (void)(__VA_ARGS__)
+/// A template function for suppressing warnings about unused variables or function results.
+template <typename... Args>
+constexpr void UNUSED(Args &&... args [[maybe_unused]])
+{
+}

base/common/extended_types.h

@@ -104,8 +104,3 @@ template <> struct is_big_int<wUInt256> { static constexpr bool value = true; };
 template <typename T>
 inline constexpr bool is_big_int_v = is_big_int<T>::value;
 
-template <typename To, typename From>
-inline To bigint_cast(const From & x [[maybe_unused]])
-{
-    return static_cast<To>(x);
-}

base/common/phdr_cache.cpp

@@ -15,11 +15,11 @@
 #endif
 
 #define __msan_unpoison(X, Y) // NOLINT
-#if defined(__has_feature)
-#   if __has_feature(memory_sanitizer)
-#       undef __msan_unpoison
-#       include <sanitizer/msan_interface.h>
-#   endif
+#if defined(ch_has_feature)
+#    if ch_has_feature(memory_sanitizer)
+#        undef __msan_unpoison
+#        include <sanitizer/msan_interface.h>
+#    endif
 #endif
 
 #include <link.h>

base/common/setTerminalEcho.cpp

@@ -1,45 +1,28 @@
-// https://stackoverflow.com/questions/1413445/reading-a-password-from-stdcin
-
 #include <common/setTerminalEcho.h>
 #include <common/errnoToString.h>
 #include <stdexcept>
 #include <cstring>
 #include <string>
-
-#ifdef WIN32
-#include <windows.h>
-#else
 #include <termios.h>
 #include <unistd.h>
-#include <errno.h>
-#endif
+
 
 void setTerminalEcho(bool enable)
 {
-#ifdef WIN32
-    auto handle = GetStdHandle(STD_INPUT_HANDLE);
-    DWORD mode;
-    if (!GetConsoleMode(handle, &mode))
-        throw std::runtime_error(std::string("setTerminalEcho failed get: ") + std::to_string(GetLastError()));
+    /// Obtain terminal attributes,
+    /// toggle the ECHO flag
+    /// and set them back.
 
-    if (!enable)
-        mode &= ~ENABLE_ECHO_INPUT;
-    else
-        mode |= ENABLE_ECHO_INPUT;
+    struct termios tty{};
 
-    if (!SetConsoleMode(handle, mode))
-        throw std::runtime_error(std::string("setTerminalEcho failed set: ") + std::to_string(GetLastError()));
-#else
-    struct termios tty;
-    if (tcgetattr(STDIN_FILENO, &tty))
+    if (0 != tcgetattr(STDIN_FILENO, &tty))
         throw std::runtime_error(std::string("setTerminalEcho failed get: ") + errnoToString(errno));
-    if (!enable)
-        tty.c_lflag &= ~ECHO;
-    else
-        tty.c_lflag |= ECHO;
 
-    auto ret = tcsetattr(STDIN_FILENO, TCSANOW, &tty);
-    if (ret)
+    if (enable)
+        tty.c_lflag |= ECHO;
+    else
+        tty.c_lflag &= ~ECHO;
+
+    if (0 != tcsetattr(STDIN_FILENO, TCSANOW, &tty))
         throw std::runtime_error(std::string("setTerminalEcho failed set: ") + errnoToString(errno));
-#endif
 }

base/common/strong_typedef.h

@@ -12,6 +12,7 @@ private:
     T t;
 
 public:
+    using UnderlyingType = T;
     template <class Enable = typename std::is_copy_constructible<T>::type>
     explicit StrongTypedef(const T & t_) : t(t_) {}
     template <class Enable = typename std::is_move_constructible<T>::type>

base/common/tests/CMakeLists.txt

@@ -1,25 +1,2 @@
-include (${ClickHouse_SOURCE_DIR}/cmake/add_check.cmake)
-
-add_executable (date_lut2 date_lut2.cpp)
-add_executable (date_lut3 date_lut3.cpp)
-add_executable (date_lut_default_timezone date_lut_default_timezone.cpp)
-add_executable (local_date_time_comparison local_date_time_comparison.cpp)
-add_executable (realloc-perf allocator.cpp)
-
-set(PLATFORM_LIBS ${CMAKE_DL_LIBS})
-
-target_link_libraries (date_lut2 PRIVATE common ${PLATFORM_LIBS})
-target_link_libraries (date_lut3 PRIVATE common ${PLATFORM_LIBS})
-target_link_libraries (date_lut_default_timezone PRIVATE common ${PLATFORM_LIBS})
-target_link_libraries (local_date_time_comparison PRIVATE common)
-target_link_libraries (realloc-perf PRIVATE common)
-add_check(local_date_time_comparison)
-
-if(USE_GTEST)
-    add_executable(unit_tests_libcommon gtest_json_test.cpp gtest_strong_typedef.cpp gtest_find_symbols.cpp)
-    target_link_libraries(unit_tests_libcommon PRIVATE common ${GTEST_MAIN_LIBRARIES} ${GTEST_LIBRARIES})
-    add_check(unit_tests_libcommon)
-endif()
-
 add_executable (dump_variable dump_variable.cpp)
 target_link_libraries (dump_variable PRIVATE clickhouse_common_io)

base/common/tests/allocator.cpp

@@ -1,47 +0,0 @@
-#include <cstdlib>
-#include <cstring>
-#include <vector>
-#include <thread>
-
-
-void thread_func()
-{
-    for (size_t i = 0; i < 100; ++i)
-    {
-        size_t size = 4096;
-
-        void * buf = malloc(size);
-        if (!buf)
-            abort();
-        memset(buf, 0, size);
-
-        while (size < 1048576)
-        {
-            size_t next_size = size * 4;
-
-            void * new_buf = realloc(buf, next_size);
-            if (!new_buf)
-                abort();
-            buf = new_buf;
-
-            memset(reinterpret_cast<char*>(buf) + size, 0, next_size - size);
-            size = next_size;
-        }
-
-        free(buf);
-    }
-}
-
-
-int main(int, char **)
-{
-    std::vector<std::thread> threads(16);
-    for (size_t i = 0; i < 1000; ++i)
-    {
-        for (auto & thread : threads)
-            thread = std::thread(thread_func);
-        for (auto & thread : threads)
-            thread.join();
-    }
-    return 0;
-}

base/common/tests/date_lut2.cpp

@@ -1,53 +0,0 @@
-#include <iostream>
-#include <cstring>
-
-#include <common/DateLUT.h>
-
-
-static std::string toString(time_t Value)
-{
-    struct tm tm;
-    char buf[96];
-
-    localtime_r(&Value, &tm);
-    snprintf(buf, sizeof(buf), "%04d-%02d-%02d %02d:%02d:%02d",
-             tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec);
-
-    return buf;
-}
-
-static time_t orderedIdentifierToDate(unsigned value)
-{
-    struct tm tm;
-
-    memset(&tm, 0, sizeof(tm));
-
-    tm.tm_year = value / 10000 - 1900;
-    tm.tm_mon = (value % 10000) / 100 - 1;
-    tm.tm_mday = value % 100;
-    tm.tm_isdst = -1;
-
-    return mktime(&tm);
-}
-
-
-void loop(time_t begin, time_t end, int step)
-{
-    const auto & date_lut = DateLUT::instance();
-
-    for (time_t t = begin; t < end; t += step)
-        std::cout << toString(t)
-            << ", " << toString(date_lut.toTime(t))
-            << ", " << date_lut.toHour(t)
-            << std::endl;
-}
-
-
-int main(int, char **)
-{
-    loop(orderedIdentifierToDate(20101031), orderedIdentifierToDate(20101101), 15 * 60);
-    loop(orderedIdentifierToDate(20100328), orderedIdentifierToDate(20100330), 15 * 60);
-    loop(orderedIdentifierToDate(20141020), orderedIdentifierToDate(20141106), 15 * 60);
-
-    return 0;
-}

base/common/tests/date_lut3.cpp

@@ -1,62 +0,0 @@
-#include <iostream>
-#include <cstring>
-
-#include <Poco/Exception.h>
-
-#include <common/DateLUT.h>
-
-
-static std::string toString(time_t Value)
-{
-    struct tm tm;
-    char buf[96];
-
-    localtime_r(&Value, &tm);
-    snprintf(buf, sizeof(buf), "%04d-%02d-%02d %02d:%02d:%02d",
-             tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec);
-
-    return buf;
-}
-
-static time_t orderedIdentifierToDate(unsigned value)
-{
-    struct tm tm;
-
-    memset(&tm, 0, sizeof(tm));
-
-    tm.tm_year = value / 10000 - 1900;
-    tm.tm_mon = (value % 10000) / 100 - 1;
-    tm.tm_mday = value % 100;
-    tm.tm_isdst = -1;
-
-    return mktime(&tm);
-}
-
-
-void loop(time_t begin, time_t end, int step)
-{
-    const auto & date_lut = DateLUT::instance();
-
-    for (time_t t = begin; t < end; t += step)
-    {
-        time_t t2 = date_lut.makeDateTime(date_lut.toYear(t), date_lut.toMonth(t), date_lut.toDayOfMonth(t),
-            date_lut.toHour(t), date_lut.toMinute(t), date_lut.toSecond(t));
-
-        std::string s1 = toString(t);
-        std::string s2 = toString(t2);
-
-        std::cerr << s1 << ", " << s2 << std::endl;
-
-        if (s1 != s2)
-            throw Poco::Exception("Test failed.");
-    }
-}
-
-
-int main(int, char **)
-{
-    loop(orderedIdentifierToDate(20101031), orderedIdentifierToDate(20101101), 15 * 60);
-    loop(orderedIdentifierToDate(20100328), orderedIdentifierToDate(20100330), 15 * 60);
-
-    return 0;
-}

base/common/tests/date_lut_default_timezone.cpp

@@ -1,31 +0,0 @@
-#include <iostream>
-#include <common/DateLUT.h>
-#include <Poco/Exception.h>
-
-int main(int, char **)
-{
-    try
-    {
-        const auto & date_lut = DateLUT::instance();
-        std::cout << "Detected default timezone: `" << date_lut.getTimeZone() << "'" << std::endl;
-        time_t now = time(nullptr);
-        std::cout << "Current time: " << date_lut.timeToString(now)
-                  << ", UTC: " << DateLUT::instance("UTC").timeToString(now) << std::endl;
-    }
-    catch (const Poco::Exception & e)
-    {
-        std::cerr << e.displayText() << std::endl;
-        return 1;
-    }
-    catch (std::exception & e)
-    {
-        std::cerr << "std::exception: " << e.what() << std::endl;
-        return 2;
-    }
-    catch (...)
-    {
-        std::cerr << "Some exception" << std::endl;
-        return 3;
-    }
-    return 0;
-}

base/common/tests/gtest_json_test.cpp

@@ -1,656 +0,0 @@
-#include <vector>
-#include <string>
-#include <exception>
-#include <common/JSON.h>
-
-#include <boost/range/irange.hpp>
-
-using namespace std::literals::string_literals;
-
-#include <gtest/gtest.h>
-
-enum class ResultType
-{
-    Return,
-    Throw
-};
-
-struct GetStringTestRecord
-{
-    const char * input;
-    ResultType result_type;
-    const char * result;
-};
-
-TEST(JSONSuite, SimpleTest)
-{
-    std::vector<GetStringTestRecord> test_data =
-    {
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Вафельница Vitek WX-1102 FL")", ResultType::Return, "Вафельница Vitek WX-1102 FL" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("184509")", ResultType::Return, "184509" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("Все для детей/Детская техника/Vitek")", ResultType::Return, "Все для детей/Детская техника/Vitek" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("В наличии")", ResultType::Return, "В наличии" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("2390.00")", ResultType::Return, "2390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("Карточка")", ResultType::Return, "Карточка" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("detail")", ResultType::Return, "detail" },
-        { R"("actionField")", ResultType::Return, "actionField" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("http://www.techport.ru/q/?t=вафельница&sort=price&sdim=asc")", ResultType::Return, "http://www.techport.ru/q/?t=вафельница&sort=price&sdim=asc" },
-        { R"("action")", ResultType::Return, "action" },
-        { R"("detail")", ResultType::Return, "detail" },
-        { R"("products")", ResultType::Return, "products" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Вафельница Vitek WX-1102 FL")", ResultType::Return, "Вафельница Vitek WX-1102 FL" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("184509")", ResultType::Return, "184509" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("2390.00")", ResultType::Return, "2390.00" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("Vitek")", ResultType::Return, "Vitek" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("Все для детей/Детская техника/Vitek")", ResultType::Return, "Все для детей/Детская техника/Vitek" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("В наличии")", ResultType::Return, "В наличии" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("experiments")", ResultType::Return, "experiments" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("los_portal")", ResultType::Return, "los_portal" },
-        { R"("los_level")", ResultType::Return, "los_level" },
-        { R"("none")", ResultType::Return, "none" },
-        { R"("isAuthorized")", ResultType::Return, "isAuthorized" },
-        { R"("isSubscriber")", ResultType::Return, "isSubscriber" },
-        { R"("postType")", ResultType::Return, "postType" },
-        { R"("Новости")", ResultType::Return, "Новости" },
-        { R"("experiments")", ResultType::Return, "experiments" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("los_portal")", ResultType::Return, "los_portal" },
-        { R"("los_level")", ResultType::Return, "los_level" },
-        { R"("none")", ResultType::Return, "none" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("Электроплита GEFEST Брест ЭПНД 5140-01 0001")", ResultType::Return, "Электроплита GEFEST Брест ЭПНД 5140-01 0001" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("currencyCode")", ResultType::Return, "currencyCode" },
-        { R"("RUB")", ResultType::Return, "RUB" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("experiments")", ResultType::Return, "experiments" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("los_portal")", ResultType::Return, "los_portal" },
-        { R"("los_level")", ResultType::Return, "los_level" },
-        { R"("none")", ResultType::Return, "none" },
-        { R"("trash_login")", ResultType::Return, "trash_login" },
-        { R"("novikoff")", ResultType::Return, "novikoff" },
-        { R"("trash_cat_link")", ResultType::Return, "trash_cat_link" },
-        { R"("progs")", ResultType::Return, "progs" },
-        { R"("trash_parent_link")", ResultType::Return, "trash_parent_link" },
-        { R"("content")", ResultType::Return, "content" },
-        { R"("trash_posted_parent")", ResultType::Return, "trash_posted_parent" },
-        { R"("content.01.2016")", ResultType::Return, "content.01.2016" },
-        { R"("trash_posted_cat")", ResultType::Return, "trash_posted_cat" },
-        { R"("progs.01.2016")", ResultType::Return, "progs.01.2016" },
-        { R"("trash_virus_count")", ResultType::Return, "trash_virus_count" },
-        { R"("trash_is_android")", ResultType::Return, "trash_is_android" },
-        { R"("trash_is_wp8")", ResultType::Return, "trash_is_wp8" },
-        { R"("trash_is_ios")", ResultType::Return, "trash_is_ios" },
-        { R"("trash_posted")", ResultType::Return, "trash_posted" },
-        { R"("01.2016")", ResultType::Return, "01.2016" },
-        { R"("experiments")", ResultType::Return, "experiments" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("los_portal")", ResultType::Return, "los_portal" },
-        { R"("los_level")", ResultType::Return, "los_level" },
-        { R"("none")", ResultType::Return, "none" },
-        { R"("merchantId")", ResultType::Return, "merchantId" },
-        { R"("13694_49246")", ResultType::Return, "13694_49246" },
-        { R"("cps-source")", ResultType::Return, "cps-source" },
-        { R"("wargaming")", ResultType::Return, "wargaming" },
-        { R"("cps_provider")", ResultType::Return, "cps_provider" },
-        { R"("default")", ResultType::Return, "default" },
-        { R"("errorReason")", ResultType::Return, "errorReason" },
-        { R"("no errors")", ResultType::Return, "no errors" },
-        { R"("scid")", ResultType::Return, "scid" },
-        { R"("isAuthPayment")", ResultType::Return, "isAuthPayment" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("rubric")", ResultType::Return, "rubric" },
-        { R"("")", ResultType::Return, "" },
-        { R"("rubric")", ResultType::Return, "rubric" },
-        { R"("Мир")", ResultType::Return, "Мир" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("experiments")", ResultType::Return, "experiments" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("los_portal")", ResultType::Return, "los_portal" },
-        { R"("los_level")", ResultType::Return, "los_level" },
-        { R"("none")", ResultType::Return, "none" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("__ym")", ResultType::Return, "__ym" },
-        { R"("ecommerce")", ResultType::Return, "ecommerce" },
-        { R"("impressions")", ResultType::Return, "impressions" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("863813")", ResultType::Return, "863813" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Happy, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Happy, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("863839")", ResultType::Return, "863839" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Pretty kitten, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Pretty kitten, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("863847")", ResultType::Return, "863847" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Little tiger, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Little tiger, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911480")", ResultType::Return, "911480" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Puppy, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Puppy, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911484")", ResultType::Return, "911484" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Little bears, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Little bears, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911489")", ResultType::Return, "911489" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Dolphin, возраст 2-4 года, трикотаж")", ResultType::Return, "Футболка детская 3D Dolphin, возраст 2-4 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911496")", ResultType::Return, "911496" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Pretty, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Pretty, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911504")", ResultType::Return, "911504" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Fairytale, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Fairytale, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911508")", ResultType::Return, "911508" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Kittens, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Kittens, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911512")", ResultType::Return, "911512" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Sunshine, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Sunshine, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911516")", ResultType::Return, "911516" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Dog in bag, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Dog in bag, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911520")", ResultType::Return, "911520" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Cute puppy, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Cute puppy, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911524")", ResultType::Return, "911524" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Rabbit, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Rabbit, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("911528")", ResultType::Return, "911528" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Футболка детская 3D Turtle, возраст 1-2 года, трикотаж")", ResultType::Return, "Футболка детская 3D Turtle, возраст 1-2 года, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("390.00")", ResultType::Return, "390.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("888616")", ResultType::Return, "888616" },
-        { R"("name")", ResultType::Return, "name" },
-        { "\"3Д Футболка мужская \\\"Collorista\\\" Светлое завтра р-р XL(52-54), 100% хлопок, трикотаж\"", ResultType::Return, "3Д Футболка мужская \"Collorista\" Светлое завтра р-р XL(52-54), 100% хлопок, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Одежда и обувь/Мужская одежда/Футболки/")", ResultType::Return, "/Одежда и обувь/Мужская одежда/Футболки/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("406.60")", ResultType::Return, "406.60" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("913361")", ResultType::Return, "913361" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("3Д Футболка детская World р-р 8-10, 100% хлопок, трикотаж")", ResultType::Return, "3Д Футболка детская World р-р 8-10, 100% хлопок, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("470.00")", ResultType::Return, "470.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("913364")", ResultType::Return, "913364" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("3Д Футболка детская Force р-р 8-10, 100% хлопок, трикотаж")", ResultType::Return, "3Д Футболка детская Force р-р 8-10, 100% хлопок, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("470.00")", ResultType::Return, "470.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("913367")", ResultType::Return, "913367" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("3Д Футболка детская Winter tale р-р 8-10, 100% хлопок, трикотаж")", ResultType::Return, "3Д Футболка детская Winter tale р-р 8-10, 100% хлопок, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("470.00")", ResultType::Return, "470.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("913385")", ResultType::Return, "913385" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("3Д Футболка детская Moonshine р-р 8-10, 100% хлопок, трикотаж")", ResultType::Return, "3Д Футболка детская Moonshine р-р 8-10, 100% хлопок, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("470.00")", ResultType::Return, "470.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("913391")", ResultType::Return, "913391" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("3Д Футболка детская Shaman р-р 8-10, 100% хлопок, трикотаж")", ResultType::Return, "3Д Футболка детская Shaman р-р 8-10, 100% хлопок, трикотаж" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("/Летние товары/Летний текстиль/")", ResultType::Return, "/Летние товары/Летний текстиль/" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("")", ResultType::Return, "" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("470.00")", ResultType::Return, "470.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("/retailrocket/")", ResultType::Return, "/retailrocket/" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/")", ResultType::Return, "/911488/futbolka-detskaya-3d-dolphin-vozrast-1-2-goda-trikotazh/" },
-        { R"("usertype")", ResultType::Return, "usertype" },
-        { R"("visitor")", ResultType::Return, "visitor" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("__ym")", ResultType::Return, "__ym" },
-        { R"("ecommerce")", ResultType::Return, "ecommerce" },
-        { R"("impressions")", ResultType::Return, "impressions" },
-        { R"("experiments")", ResultType::Return, "experiments" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("los_portal")", ResultType::Return, "los_portal" },
-        { R"("los_level")", ResultType::Return, "los_level" },
-        { R"("none")", ResultType::Return, "none" },
-        { R"("experiments")", ResultType::Return, "experiments" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("los_portal")", ResultType::Return, "los_portal" },
-        { R"("los_level")", ResultType::Return, "los_level" },
-        { R"("none")", ResultType::Return, "none" },
-        { R"("experiments")", ResultType::Return, "experiments" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("los_portal")", ResultType::Return, "los_portal" },
-        { R"("los_level")", ResultType::Return, "los_level" },
-        { R"("none")", ResultType::Return, "none" },
-        { R"("experiments")", ResultType::Return, "experiments" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("los_portal")", ResultType::Return, "los_portal" },
-        { R"("los_level")", ResultType::Return, "los_level" },
-        { R"("none")", ResultType::Return, "none" },
-        { R"("experiments")", ResultType::Return, "experiments" },
-        { R"("lang")", ResultType::Return, "lang" },
-        { R"("ru")", ResultType::Return, "ru" },
-        { R"("los_portal")", ResultType::Return, "los_portal" },
-        { R"("los_level")", ResultType::Return, "los_level" },
-        { R"("none")", ResultType::Return, "none" },
-        { R"("__ym")", ResultType::Return, "__ym" },
-        { R"("ecommerce")", ResultType::Return, "ecommerce" },
-        { R"("currencyCode")", ResultType::Return, "currencyCode" },
-        { R"("RUR")", ResultType::Return, "RUR" },
-        { R"("impressions")", ResultType::Return, "impressions" },
-        { R"("name")", ResultType::Return, "name" },
-        { R"("Чайник электрический Mystery MEK-1627, белый")", ResultType::Return, "Чайник электрический Mystery MEK-1627, белый" },
-        { R"("brand")", ResultType::Return, "brand" },
-        { R"("Mystery")", ResultType::Return, "Mystery" },
-        { R"("id")", ResultType::Return, "id" },
-        { R"("187180")", ResultType::Return, "187180" },
-        { R"("category")", ResultType::Return, "category" },
-        { R"("Мелкая бытовая техника/Мелкие кухонные приборы/Чайники электрические/Mystery")", ResultType::Return, "Мелкая бытовая техника/Мелкие кухонные приборы/Чайники электрические/Mystery" },
-        { R"("variant")", ResultType::Return, "variant" },
-        { R"("В наличии")", ResultType::Return, "В наличии" },
-        { R"("price")", ResultType::Return, "price" },
-        { R"("1630.00")", ResultType::Return, "1630.00" },
-        { R"("list")", ResultType::Return, "list" },
-        { R"("Карточка")", ResultType::Return, "Карточка" },
-        { R"("position")", ResultType::Return, "position" },
-        { R"("detail")", ResultType::Return, "detail" },
-        { R"("actionField")", ResultType::Return, "actionField" },
-        { R"("list")", ResultType::Return, "list" },
-        { "\0\"", ResultType::Throw, "JSON: expected \", got \0" },
-        { "\"/igrushki/konstruktory\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/1290414/komplekt-zhenskiy-dzhemper-plusbryuki-m-254-09-malina-plustemno-siniy-\0a", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Творчество/Рисование/Инструменты и кра\0a", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Строительство и ремонт/Силовая техника/Зарядные устройства для автомобильных аккумуляторов/Пуско-зарядные устр\xD0\0a", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Строительство и ремонт/Силовая техника/Зарядные устройств\xD0\0t", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Строительство и ремонт/Силовая техника/Зарядные устройства для автомобиль\0k", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\0t", ResultType::Throw, "JSON: expected \", got \0" },
-        { "\"/Хозтовары/Хранение вещей и организа\xD1\0t", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Хозтовары/Товары для стир\0a", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"li\0a", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/734859/samolet-radioupravlyaemyy-istrebitel-rabotaet-o\0k", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/kosmetika-i-parfyum/parfyumeriya/mu\0t", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/ko\0\x04", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "", ResultType::Throw, "JSON: begin >= end." },
-        { "\"/stroitelstvo-i-remont/stroit\0t", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/stroitelstvo-i-remont/stroitelnyy-instrument/av\0k", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/s\0a", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Строительство и ремонт/Строительный инструмент/Изм\0e", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/avto/soputstvuy\0l", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/str\0\xD0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Отвертка 2 в 1 \\\"TUNDRA basic\\\" 5х75 мм (+,-) \0\xFF", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/stroitelstvo-i-remont/stroitelnyy-instrument/avtoinstrumen\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Мелкая бытовая техника/Мелки\xD0\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Пряжа \\\"Бамбук стрейч\\0\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Карандаш чёрнографитны\xD0\0\xD0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Творчество/Рукоделие, аппликации/Пряжа и шерсть для \xD0\0l", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/1071547/karandash-chernografitnyy-volshebstvo-nv-kruglyy-d-7-2mm-dl-176mm-plast-tuba/\0e", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"ca\0e", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"ca\0e", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/1165424/chipbord-vyrubnoy-dlya-skrapbukinga-malyshi-mikki-maus-disney-bebi\0t", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/posuda/kuhonnye-prinadlezhnosti-i-i\0d", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Канцтовары/Ежедневники и блокн\xD0\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/kanctovary/ezhednevniki-i-blok\0a", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Стакан \xD0\0a", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Набор бумаги для скрапбукинга \\\"Мои первый годик\\\": Микки Маус, Дисней бэби, 12 листов 29.5 х 29.5 см, 160\0\x80", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"c\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Органайзер для хранения аксессуаров, \0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"quantity\00", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Сменный блок для тетрадей на кольцах А5, 160 листов клетка, офсет \xE2\x84\0=", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Сувениры/Ф\xD0\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"\0\"", ResultType::Return, "\0" },
-        { "\"\0\x04", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"va\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"ca\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"В \0\x04", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/letnie-tovary/z\0\x04", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Посудомоечная машина Ha\0=", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Крупная бытов\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Полочная акустическая система Magnat Needl\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"brand\00", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"\0d", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"pos\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"c\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"var\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Телевизоры и видеотехника/Всё для домашних кинотеатр\0=", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Флеш-диск Transcend JetFlash 620 8GB (TS8GJF62\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Табурет Мег\0\xD0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"variant\0\x04", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Катал\xD0\0\"", ResultType::Return, "Катал\xD0\0" },
-        { "\"К\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Полочная акустическая система Magnat Needl\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"brand\00", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"\0d", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"pos\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"c\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"17\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/igrushki/razvivayusc\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Ключница \\\"\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Игр\xD1\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Игрушки/Игрушки для девочек/Игровые модули дл\xD1\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Крупная бытовая техника/Стиральные машины/С фронт\xD0\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\0 ", ResultType::Throw, "JSON: expected \", got \0" },
-        { "\"Светодиодная лента SMD3528, 5 м. IP33, 60LED, зеленый, 4,8W/мет\xD1\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Сантехника/Мебель для ванных комнат/Стол\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\0o", ResultType::Throw, "JSON: expected \", got \0" },
-        { "\"/igrushki/konstruktory\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/posuda/kuhonnye-prinadlezhnosti-i-instrumenty/kuhonnye-pr\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/1290414/komplekt-zhenskiy-dzhemper-plusbryuki-m-254-09-malina-plustemno-siniy-\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Творчество/Рисование/Инструменты и кра\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Строительство и ремонт/Силовая техника/Зарядные устройства для автомобильных аккумуляторов/Пуско-зарядные устр\xD0\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Строительство и ремонт/Силовая техника/Зарядные устройств\xD0\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Строительство и ремонт/Силовая техника/Зарядные устройства для автомобиль\0d", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\0 ", ResultType::Throw, "JSON: expected \", got \0" },
-        { "\"/Хозтовары/Хранение вещей и организа\xD1\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Хозтовары/Товары для стир\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"li\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/igrushki/igrus\0d", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/734859/samolet-radioupravlyaemyy-istrebitel-rabotaet-o\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/kosmetika-i-parfyum/parfyumeriya/mu\00", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/ko\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/avto/avtomobilnyy\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/stroitelstvo-i-remont/stroit\00", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/stroitelstvo-i-remont/stroitelnyy-instrument/av\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/s\0d", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Строительство и ремонт/Строительный инструмент/Изм\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/avto/soputstvuy\0\"", ResultType::Return, "/avto/soputstvuy\0" },
-        { "\"/str\0k", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Отвертка 2 в 1 \\\"TUNDRA basic\\\" 5х75 мм (+,-) \0\xD0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/stroitelstvo-i-remont/stroitelnyy-instrument/avtoinstrumen\0=", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Чайник электрический Vitesse\0=", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Мелкая бытовая техника/Мелки\xD0\0\xD0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Пряжа \\\"Бамбук стрейч\\0о", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Карандаш чёрнографитны\xD0\0k", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Творчество/Рукоделие, аппликации/Пряжа и шерсть для \xD0\0\"", ResultType::Return, "/Творчество/Рукоделие, аппликации/Пряжа и шерсть для \xD0\0" },
-        { "\"/1071547/karandash-chernografitnyy-volshebstvo-nv-kruglyy-d-7-2mm-dl-176mm-plast-tuba/\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"ca\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Подаро\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Средство для прочис\xD1\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"i\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/p\0\"", ResultType::Return, "/p\0" },
-        { "\"/Сувениры/Магниты, н\xD0\0k", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Дерев\xD0\0=", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/prazdniki/svadba/svadebnaya-c\0\xD0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Канцт\0d", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Праздники/То\xD0\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"v\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Косметика \xD0\0d", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Спорт и отдых/Настольные игры/Покер, руле\xD1\0\xD0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"categ\0=", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/retailr\0k", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/retailrocket\0k", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Ежедневник недат А5 140л кл,ляссе,обл пв\0=", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/432809/ezhednevnik-organayzer-sredniy-s-remeshkom-na-knopke-v-oblozhke-kalkulyator-kalendar-do-\0\xD0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/1165424/chipbord-vyrubnoy-dlya-skrapbukinga-malyshi-mikki-maus-disney-bebi\0d", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/posuda/kuhonnye-prinadlezhnosti-i-i\0 ", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/Канцтовары/Ежедневники и блокн\xD0\0o", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"/kanctovary/ezhednevniki-i-blok\00", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Стакан \xD0\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"Набор бумаги для скрапбукинга \\\"Мои первый годик\\\": Микки Маус, Дисней бэби, 12 листов 29.5 х 29.5 см, 160\0\0", ResultType::Throw, "JSON: incorrect syntax (expected end of string, found end of JSON)." },
-        { "\"c\0\"", ResultType::Return, "c\0" },
-    };
-
-    for (auto i : boost::irange(0, 1/*00000*/))
-    {
-        static_cast<void>(i);
-
-        for (auto & r : test_data)
-        {
-            try
-            {
-                JSON j(r.input, r.input + strlen(r.input));
-
-                ASSERT_EQ(j.getString(), r.result);
-                ASSERT_TRUE(r.result_type == ResultType::Return);
-            }
-            catch (JSONException & e)
-            {
-                ASSERT_TRUE(r.result_type == ResultType::Throw);
-                ASSERT_EQ(e.message(), r.result);
-            }
-        }
-    }
-}

base/common/tests/local_date_time_comparison.cpp

@@ -1,29 +0,0 @@
-#include <iostream>
-#include <stdexcept>
-
-#include <common/LocalDateTime.h>
-
-
-void fillStackWithGarbage()
-{
-    volatile uint64_t a = 0xAABBCCDDEEFF0011ULL;
-    volatile uint64_t b = 0x2233445566778899ULL;
-    std::cout << a + b << '\n';
-}
-
-void checkComparison()
-{
-    LocalDateTime a("2018-07-18 01:02:03");
-    LocalDateTime b("2018-07-18 01:02:03");
-
-    if (a != b)
-        throw std::runtime_error("Test failed");
-}
-
-
-int main(int, char **)
-{
-    fillStackWithGarbage();
-    checkComparison();
-    return 0;
-}

base/common/wide_integer_impl.h

@@ -5,9 +5,11 @@
 /// (See at http://www.boost.org/LICENSE_1_0.txt)
 
 #include "throwError.h"
+#include <cmath>
 #include <cfloat>
-#include <limits>
 #include <cassert>
+#include <limits>
+
 
 namespace wide
 {
@@ -239,15 +241,23 @@ struct integer<Bits, Signed>::_impl
     template <class T>
     constexpr static void set_multiplier(integer<Bits, Signed> & self, T t) noexcept {
         constexpr uint64_t max_int = std::numeric_limits<uint64_t>::max();
-        const T alpha = t / max_int;
 
-        if (alpha <= max_int)
+        /// Implementation specific behaviour on overflow (if we don't check here, stack overflow will triggered in bigint_cast).
+        if (!std::isfinite(t))
+        {
+            self = 0;
+            return;
+        }
+
+        const T alpha = t / static_cast<T>(max_int);
+
+        if (alpha <= static_cast<T>(max_int))
             self = static_cast<uint64_t>(alpha);
         else // max(double) / 2^64 will surely contain less than 52 precision bits, so speed up computations.
             set_multiplier<double>(self, alpha);
 
         self *= max_int;
-        self += static_cast<uint64_t>(t - alpha * max_int); // += b_i
+        self += static_cast<uint64_t>(t - alpha * static_cast<T>(max_int)); // += b_i
     }
 
     constexpr static void wide_integer_from_bultin(integer<Bits, Signed>& self, double rhs) noexcept {
@@ -265,7 +275,7 @@ struct integer<Bits, Signed>::_impl
             "On your system long double has less than 64 precision bits,"
             "which may result in UB when initializing double from int64_t");
 
-        if ((rhs > 0 && rhs < max_int) || (rhs < 0 && rhs > min_int))
+        if ((rhs > 0 && rhs < static_cast<long double>(max_int)) || (rhs < 0 && rhs > static_cast<long double>(min_int)))
         {
             self = static_cast<int64_t>(rhs);
             return;

base/daemon/BaseDaemon.cpp

@@ -4,6 +4,11 @@
 #include <sys/stat.h>
 #include <sys/types.h>
 #include <sys/time.h>
+#include <sys/wait.h>
+#include <sys/resource.h>
+#if defined(__linux__)
+    #include <sys/prctl.h>
+#endif
 #include <fcntl.h>
 #include <errno.h>
 #include <string.h>
@@ -12,7 +17,6 @@
 #include <unistd.h>
 
 #include <typeinfo>
-#include <sys/resource.h>
 #include <iostream>
 #include <fstream>
 #include <sstream>
@@ -22,7 +26,6 @@
 #include <Poco/Observer.h>
 #include <Poco/AutoPtr.h>
 #include <Poco/PatternFormatter.h>
-#include <Poco/TaskManager.h>
 #include <Poco/File.h>
 #include <Poco/Path.h>
 #include <Poco/Message.h>
@@ -53,6 +56,9 @@
 #include <Common/Config/ConfigProcessor.h>
 #include <Common/MemorySanitizer.h>
 #include <Common/SymbolIndex.h>
+#include <Common/getExecutablePath.h>
+#include <Common/getHashOfLoadedBinary.h>
+#include <Common/Elf.h>
 
 #if !defined(ARCADIA_BUILD)
 #   include <Common/config_version.h>
@@ -77,16 +83,6 @@ static void call_default_signal_handler(int sig)
     raise(sig);
 }
 
-const char * msan_strsignal(int sig)
-{
-    // Apparently strsignal is not instrumented by MemorySanitizer, so we
-    // have to unpoison it to avoid msan reports inside fmt library when we
-    // print it.
-    const char * signal_name = strsignal(sig);
-    __msan_unpoison_string(signal_name);
-    return signal_name;
-}
-
 static constexpr size_t max_query_id_size = 127;
 
 static const size_t signal_pipe_buf_size =
@@ -116,11 +112,13 @@ static void writeSignalIDtoSignalPipe(int sig)
 /** Signal handler for HUP / USR1 */
 static void closeLogsSignalHandler(int sig, siginfo_t *, void *)
 {
+    DENY_ALLOCATIONS_IN_SCOPE;
     writeSignalIDtoSignalPipe(sig);
 }
 
 static void terminateRequestedSignalHandler(int sig, siginfo_t *, void *)
 {
+    DENY_ALLOCATIONS_IN_SCOPE;
     writeSignalIDtoSignalPipe(sig);
 }
 
@@ -129,6 +127,7 @@ static void terminateRequestedSignalHandler(int sig, siginfo_t *, void *)
   */
 static void signalHandler(int sig, siginfo_t * info, void * context)
 {
+    DENY_ALLOCATIONS_IN_SCOPE;
     auto saved_errno = errno;   /// We must restore previous value of errno in signal handler.
 
     char buf[signal_pipe_buf_size];
@@ -153,7 +152,7 @@ static void signalHandler(int sig, siginfo_t * info, void * context)
     if (sig != SIGTSTP) /// This signal is used for debugging.
     {
         /// The time that is usually enough for separate thread to print info into log.
-        sleepForSeconds(10);
+        sleepForSeconds(20);  /// FIXME: use some feedback from threads that process stacktrace
         call_default_signal_handler(sig);
     }
 
@@ -231,10 +230,10 @@ public:
             }
             else
             {
-                siginfo_t info;
-                ucontext_t context;
+                siginfo_t info{};
+                ucontext_t context{};
                 StackTrace stack_trace(NoCapture{});
-                UInt32 thread_num;
+                UInt32 thread_num{};
                 std::string query_id;
                 DB::ThreadStatus * thread_ptr{};
 
@@ -291,13 +290,13 @@ private:
         {
             LOG_FATAL(log, "(version {}{}, {}) (from thread {}) (no query) Received signal {} ({})",
                 VERSION_STRING, VERSION_OFFICIAL, daemon.build_id_info,
-                thread_num, msan_strsignal(sig), sig);
+                thread_num, strsignal(sig), sig);
         }
         else
         {
             LOG_FATAL(log, "(version {}{}, {}) (from thread {}) (query_id: {}) Received signal {} ({})",
                 VERSION_STRING, VERSION_OFFICIAL, daemon.build_id_info,
-                thread_num, query_id, msan_strsignal(sig), sig);
+                thread_num, query_id, strsignal(sig), sig);
         }
 
         String error_message;
@@ -312,7 +311,8 @@ private:
         if (stack_trace.getSize())
         {
             /// Write bare stack trace (addresses) just in case if we will fail to print symbolized stack trace.
-            /// NOTE This still require memory allocations and mutex lock inside logger. BTW we can also print it to stderr using write syscalls.
+            /// NOTE: This still require memory allocations and mutex lock inside logger.
+            ///       BTW we can also print it to stderr using write syscalls.
 
             std::stringstream bare_stacktrace;
             bare_stacktrace << "Stack trace:";
@@ -325,6 +325,32 @@ private:
         /// Write symbolized stack trace line by line for better grep-ability.
         stack_trace.toStringEveryLine([&](const std::string & s) { LOG_FATAL(log, s); });
 
+#if defined(OS_LINUX)
+        /// Write information about binary checksum. It can be difficult to calculate, so do it only after printing stack trace.
+        String calculated_binary_hash = getHashOfLoadedBinaryHex();
+        if (daemon.stored_binary_hash.empty())
+        {
+            LOG_FATAL(log, "Calculated checksum of the binary: {}."
+                " There is no information about the reference checksum.", calculated_binary_hash);
+        }
+        else if (calculated_binary_hash == daemon.stored_binary_hash)
+        {
+            LOG_FATAL(log, "Checksum of the binary: {}, integrity check passed.", calculated_binary_hash);
+        }
+        else
+        {
+            LOG_FATAL(log, "Calculated checksum of the ClickHouse binary ({0}) does not correspond"
+                " to the reference checksum stored in the binary ({1})."
+                " It may indicate one of the following:"
+                " - the file was changed just after startup;"
+                " - the file is damaged on disk due to faulty hardware;"
+                " - the loaded executable is damaged in memory due to faulty hardware;"
+                " - the file was intentionally modified;"
+                " - logical error in code."
+                , calculated_binary_hash, daemon.stored_binary_hash);
+        }
+#endif
+
         /// Write crash to system.crash_log table if available.
         if (collectCrashLog)
             collectCrashLog(sig, thread_num, query_id, stack_trace);
@@ -390,7 +416,9 @@ static void sanitizerDeathCallback()
     else
         log_message = "Terminate called without an active exception";
 
-    static const size_t buf_size = 1024;
+    /// POSIX.1 says that write(2)s of less than PIPE_BUF bytes must be atomic - man 7 pipe
+    /// And the buffer should not be too small because our exception messages can be large.
+    static constexpr size_t buf_size = PIPE_BUF;
 
     if (log_message.size() > buf_size - 16)
         log_message.resize(buf_size - 16);
@@ -470,7 +498,6 @@ BaseDaemon::~BaseDaemon()
 
 void BaseDaemon::terminate()
 {
-    getTaskManager().cancelAll();
     if (::raise(SIGTERM) != 0)
         throw Poco::SystemException("cannot terminate process");
 }
@@ -478,20 +505,10 @@ void BaseDaemon::terminate()
 void BaseDaemon::kill()
 {
     dumpCoverageReportIfPossible();
-    pid.reset();
-    if (::raise(SIGKILL) != 0)
-        throw Poco::SystemException("cannot kill process");
-}
-
-void BaseDaemon::sleep(double seconds)
-{
-    wakeup_event.reset();
-    wakeup_event.tryWait(seconds * 1000);
-}
-
-void BaseDaemon::wakeup()
-{
-    wakeup_event.set();
+    pid_file.reset();
+    /// Exit with the same code as it is usually set by shell when process is terminated by SIGKILL.
+    /// It's better than doing 'raise' or 'kill', because they have no effect for 'init' process (with pid = 0, usually in Docker).
+    _exit(128 + SIGKILL);
 }
 
 std::string BaseDaemon::getDefaultCorePath() const
@@ -547,6 +564,7 @@ void debugIncreaseOOMScore()
     {
         DB::WriteBufferFromFile buf("/proc/self/oom_score_adj");
         buf.write(new_score.c_str(), new_score.size());
+        buf.close();
     }
     catch (const Poco::Exception & e)
     {
@@ -564,7 +582,6 @@ void BaseDaemon::initialize(Application & self)
 {
     closeFDs();
 
-    task_manager = std::make_unique<Poco::TaskManager>();
     ServerApplication::initialize(self);
 
     /// now highest priority (lowest value) is PRIO_APPLICATION = -100, we want higher!
@@ -648,10 +665,6 @@ void BaseDaemon::initialize(Application & self)
             throw Poco::OpenFileException("Cannot attach stdout to " + stdout_path);
     }
 
-    /// Create pid file.
-    if (config().has("pid"))
-        pid.emplace(config().getString("pid"), DB::StatusFile::write_pid);
-
     /// Change path for logging.
     if (!log_path.empty())
     {
@@ -667,9 +680,17 @@ void BaseDaemon::initialize(Application & self)
             throw Poco::Exception("Cannot change directory to /tmp");
     }
 
-    // sensitive data masking rules are not used here
+    /// sensitive data masking rules are not used here
     buildLoggers(config(), logger(), self.commandName());
 
+    /// After initialized loggers but before initialized signal handling.
+    if (should_setup_watchdog)
+        setupWatchdog();
+
+    /// Create pid file.
+    if (config().has("pid"))
+        pid_file.emplace(config().getString("pid"), DB::StatusFile::write_pid);
+
     if (is_daemon)
     {
         /** Change working directory to the directory to write core dumps.
@@ -704,54 +725,71 @@ void BaseDaemon::initialize(Application & self)
 }
 
 
+static void addSignalHandler(const std::vector<int> & signals, signal_function handler, std::vector<int> * out_handled_signals)
+{
+    struct sigaction sa;
+    memset(&sa, 0, sizeof(sa));
+    sa.sa_sigaction = handler;
+    sa.sa_flags = SA_SIGINFO;
+
+#if defined(OS_DARWIN)
+    sigemptyset(&sa.sa_mask);
+    for (auto signal : signals)
+        sigaddset(&sa.sa_mask, signal);
+#else
+    if (sigemptyset(&sa.sa_mask))
+        throw Poco::Exception("Cannot set signal handler.");
+
+    for (auto signal : signals)
+        if (sigaddset(&sa.sa_mask, signal))
+            throw Poco::Exception("Cannot set signal handler.");
+#endif
+
+    for (auto signal : signals)
+        if (sigaction(signal, &sa, nullptr))
+            throw Poco::Exception("Cannot set signal handler.");
+
+    if (out_handled_signals)
+        std::copy(signals.begin(), signals.end(), std::back_inserter(*out_handled_signals));
+};
+
+
+static void blockSignals(const std::vector<int> & signals)
+{
+    sigset_t sig_set;
+
+#if defined(OS_DARWIN)
+    sigemptyset(&sig_set);
+    for (auto signal : signals)
+        sigaddset(&sig_set, signal);
+#else
+    if (sigemptyset(&sig_set))
+        throw Poco::Exception("Cannot block signal.");
+
+    for (auto signal : signals)
+        if (sigaddset(&sig_set, signal))
+            throw Poco::Exception("Cannot block signal.");
+#endif
+
+    if (pthread_sigmask(SIG_BLOCK, &sig_set, nullptr))
+        throw Poco::Exception("Cannot block signal.");
+};
+
+
 void BaseDaemon::initializeTerminationAndSignalProcessing()
 {
     SentryWriter::initialize(config());
     std::set_terminate(terminate_handler);
 
     /// We want to avoid SIGPIPE when working with sockets and pipes, and just handle return value/errno instead.
-    {
-        sigset_t sig_set;
-        if (sigemptyset(&sig_set) || sigaddset(&sig_set, SIGPIPE) || pthread_sigmask(SIG_BLOCK, &sig_set, nullptr))
-            throw Poco::Exception("Cannot block signal.");
-    }
+    blockSignals({SIGPIPE});
 
     /// Setup signal handlers.
-    auto add_signal_handler =
-        [this](const std::vector<int> & signals, signal_function handler)
-        {
-            struct sigaction sa;
-            memset(&sa, 0, sizeof(sa));
-            sa.sa_sigaction = handler;
-            sa.sa_flags = SA_SIGINFO;
-
-            {
-#if defined(OS_DARWIN)
-                sigemptyset(&sa.sa_mask);
-                for (auto signal : signals)
-                    sigaddset(&sa.sa_mask, signal);
-#else
-                if (sigemptyset(&sa.sa_mask))
-                    throw Poco::Exception("Cannot set signal handler.");
-
-                for (auto signal : signals)
-                    if (sigaddset(&sa.sa_mask, signal))
-                        throw Poco::Exception("Cannot set signal handler.");
-#endif
-
-                for (auto signal : signals)
-                    if (sigaction(signal, &sa, nullptr))
-                        throw Poco::Exception("Cannot set signal handler.");
-
-                std::copy(signals.begin(), signals.end(), std::back_inserter(handled_signals));
-            }
-        };
-
     /// SIGTSTP is added for debugging purposes. To output a stack trace of any running thread at anytime.
 
-    add_signal_handler({SIGABRT, SIGSEGV, SIGILL, SIGBUS, SIGSYS, SIGFPE, SIGPIPE, SIGTSTP}, signalHandler);
-    add_signal_handler({SIGHUP, SIGUSR1}, closeLogsSignalHandler);
-    add_signal_handler({SIGINT, SIGQUIT, SIGTERM}, terminateRequestedSignalHandler);
+    addSignalHandler({SIGABRT, SIGSEGV, SIGILL, SIGBUS, SIGSYS, SIGFPE, SIGPIPE, SIGTSTP, SIGTRAP}, signalHandler, &handled_signals);
+    addSignalHandler({SIGHUP, SIGUSR1}, closeLogsSignalHandler, &handled_signals);
+    addSignalHandler({SIGINT, SIGQUIT, SIGTERM}, terminateRequestedSignalHandler, &handled_signals);
 
 #if defined(SANITIZER)
     __sanitizer_set_death_callback(sanitizerDeathCallback);
@@ -776,6 +814,13 @@ void BaseDaemon::initializeTerminationAndSignalProcessing()
 #else
     build_id_info = "no build id";
 #endif
+
+#if defined(__linux__)
+    std::string executable_path = getExecutablePath();
+
+    if (!executable_path.empty())
+        stored_binary_hash = DB::Elf(executable_path).getBinaryHash();
+#endif
 }
 
 void BaseDaemon::logRevision() const
@@ -786,23 +831,6 @@ void BaseDaemon::logRevision() const
         + ", PID " + std::to_string(getpid()));
 }
 
-/// Makes server shutdown if at least one Poco::Task have failed.
-void BaseDaemon::exitOnTaskError()
-{
-    Poco::Observer<BaseDaemon, Poco::TaskFailedNotification> obs(*this, &BaseDaemon::handleNotification);
-    getTaskManager().addObserver(obs);
-}
-
-/// Used for exitOnTaskError()
-void BaseDaemon::handleNotification(Poco::TaskFailedNotification *_tfn)
-{
-    task_failed = true;
-    Poco::AutoPtr<Poco::TaskFailedNotification> fn(_tfn);
-    Poco::Logger * lg = &(logger());
-    LOG_ERROR(lg, "Task '{}' failed. Daemon is shutting down. Reason - {}", fn->task()->name(), fn->reason().displayText());
-    ServerApplication::terminate();
-}
-
 void BaseDaemon::defineOptions(Poco::Util::OptionSet & new_options)
 {
     new_options.addOption(
@@ -852,24 +880,161 @@ void BaseDaemon::handleSignal(int signal_id)
         onInterruptSignals(signal_id);
     }
     else
-        throw DB::Exception(std::string("Unsupported signal: ") + msan_strsignal(signal_id), 0);
+        throw DB::Exception(std::string("Unsupported signal: ") + strsignal(signal_id), 0);
 }
 
 void BaseDaemon::onInterruptSignals(int signal_id)
 {
     is_cancelled = true;
-    LOG_INFO(&logger(), "Received termination signal ({})", msan_strsignal(signal_id));
+    LOG_INFO(&logger(), "Received termination signal ({})", strsignal(signal_id));
 
     if (sigint_signals_counter >= 2)
     {
         LOG_INFO(&logger(), "Received second signal Interrupt. Immediately terminate.");
-        kill();
+        call_default_signal_handler(signal_id);
+        /// If the above did not help.
+        _exit(128 + signal_id);
     }
 }
 
 
 void BaseDaemon::waitForTerminationRequest()
 {
+    /// NOTE: as we already process signals via pipe, we don't have to block them with sigprocmask in threads
     std::unique_lock<std::mutex> lock(signal_handler_mutex);
     signal_event.wait(lock, [this](){ return terminate_signals_counter > 0; });
 }
+
+
+void BaseDaemon::shouldSetupWatchdog(char * argv0_)
+{
+    should_setup_watchdog = true;
+    argv0 = argv0_;
+}
+
+
+void BaseDaemon::setupWatchdog()
+{
+    /// Initialize in advance to avoid double initialization in forked processes.
+    DateLUT::instance();
+
+    std::string original_process_name;
+    if (argv0)
+        original_process_name = argv0;
+
+    while (true)
+    {
+        static pid_t pid = -1;
+        pid = fork();
+
+        if (-1 == pid)
+            throw Poco::Exception("Cannot fork");
+
+        if (0 == pid)
+        {
+            logger().information("Forked a child process to watch");
+#if defined(__linux__)
+            if (0 != prctl(PR_SET_PDEATHSIG, SIGKILL))
+                logger().warning("Cannot do prctl to ask termination with parent.");
+#endif
+            return;
+        }
+
+        /// Change short thread name and process name.
+        setThreadName("clckhouse-watch");   /// 15 characters
+
+        if (argv0)
+        {
+            const char * new_process_name = "clickhouse-watchdog";
+            memset(argv0, 0, original_process_name.size());
+            memcpy(argv0, new_process_name, std::min(strlen(new_process_name), original_process_name.size()));
+        }
+
+        logger().information(fmt::format("Will watch for the process with pid {}", pid));
+
+        /// Forward signals to the child process.
+        addSignalHandler(
+            {SIGHUP, SIGUSR1, SIGINT, SIGQUIT, SIGTERM},
+            [](int sig, siginfo_t *, void *)
+            {
+                /// Forward all signals except INT as it can be send by terminal to the process group when user press Ctrl+C,
+                /// and we process double delivery of this signal as immediate termination.
+                if (sig == SIGINT)
+                    return;
+
+                const char * error_message = "Cannot forward signal to the child process.\n";
+                if (0 != ::kill(pid, sig))
+                {
+                    auto res = write(STDERR_FILENO, error_message, strlen(error_message));
+                    (void)res;
+                }
+            },
+            nullptr);
+
+        int status = 0;
+        do
+        {
+            if (-1 != waitpid(pid, &status, WUNTRACED | WCONTINUED) || errno == ECHILD)
+            {
+                if (WIFSTOPPED(status))
+                    logger().warning(fmt::format("Child process was stopped by signal {}.", WSTOPSIG(status)));
+                else if (WIFCONTINUED(status))
+                    logger().warning(fmt::format("Child process was continued."));
+                else
+                    break;
+            }
+            else if (errno != EINTR)
+                throw Poco::Exception("Cannot waitpid, errno: " + std::string(strerror(errno)));
+        } while (true);
+
+        if (errno == ECHILD)
+        {
+            logger().information("Child process no longer exists.");
+            _exit(WEXITSTATUS(status));
+        }
+
+        if (WIFEXITED(status))
+        {
+            logger().information(fmt::format("Child process exited normally with code {}.", WEXITSTATUS(status)));
+            _exit(WEXITSTATUS(status));
+        }
+
+        if (WIFSIGNALED(status))
+        {
+            int sig = WTERMSIG(status);
+
+            if (sig == SIGKILL)
+            {
+                logger().fatal(fmt::format("Child process was terminated by signal {} (KILL)."
+                    " If it is not done by 'forcestop' command or manually,"
+                    " the possible cause is OOM Killer (see 'dmesg' and look at the '/var/log/kern.log' for the details).", sig));
+            }
+            else
+            {
+                logger().fatal(fmt::format("Child process was terminated by signal {}.", sig));
+
+                if (sig == SIGINT || sig == SIGTERM || sig == SIGQUIT)
+                    _exit(128 + sig);
+            }
+        }
+        else
+        {
+            logger().fatal("Child process was not exited normally by unknown reason.");
+        }
+
+        /// Automatic restart is not enabled but you can play with it.
+#if 1
+        _exit(WEXITSTATUS(status));
+#else
+        logger().information("Will restart.");
+        if (argv0)
+            memcpy(argv0, original_process_name.c_str(), original_process_name.size());
+#endif
+    }
+}
+
+
+String BaseDaemon::getStoredBinaryHash() const
+{
+    return stored_binary_hash;
+}

base/daemon/BaseDaemon.h

@@ -12,7 +12,6 @@
 #include <chrono>
 #include <Poco/Process.h>
 #include <Poco/ThreadPool.h>
-#include <Poco/TaskNotification.h>
 #include <Poco/Util/Application.h>
 #include <Poco/Util/ServerApplication.h>
 #include <Poco/Net/SocketAddress.h>
@@ -26,9 +25,6 @@
 #include <loggers/Loggers.h>
 
 
-namespace Poco { class TaskManager; }
-
-
 /// \brief Base class for applications that can run as daemons.
 ///
 /// \code
@@ -52,31 +48,26 @@ public:
     BaseDaemon();
     ~BaseDaemon() override;
 
-    /// Загружает конфигурацию и "строит" логгеры на запись в файлы
+    /// Load configuration, prepare loggers, etc.
     void initialize(Poco::Util::Application &) override;
 
-    /// Читает конфигурацию
     void reloadConfiguration();
 
-    /// Определяет параметр командной строки
+    /// Process command line parameters
     void defineOptions(Poco::Util::OptionSet & new_options) override;
 
-    /// Заставляет демон завершаться, если хотя бы одна задача завершилась неудачно
-    void exitOnTaskError();
+    /// Graceful shutdown
+    static void terminate();
 
-    /// Завершение демона ("мягкое")
-    void terminate();
+    /// Forceful shutdown
+    [[noreturn]] void kill();
 
-    /// Завершение демона ("жёсткое")
-    void kill();
-
-    /// Получен ли сигнал на завершение?
+    /// Cancellation request has been received.
     bool isCancelled() const
     {
         return is_cancelled;
     }
 
-    /// Получение ссылки на экземпляр демона
     static BaseDaemon & instance()
     {
         return dynamic_cast<BaseDaemon &>(Poco::Util::Application::instance());
@@ -85,12 +76,6 @@ public:
     /// return none if daemon doesn't exist, reference to the daemon otherwise
     static std::optional<std::reference_wrapper<BaseDaemon>> tryGetInstance() { return tryGetInstance<BaseDaemon>(); }
 
-    /// Спит заданное количество секунд или до события wakeup
-    void sleep(double seconds);
-
-    /// Разбудить
-    void wakeup();
-
     /// В Graphite компоненты пути(папки) разделяются точкой.
     /// У нас принят путь формата root_path.hostname_yandex_ru.key
     /// root_path по умолчанию one_min
@@ -98,7 +83,7 @@ public:
     template <class T>
     void writeToGraphite(const std::string & key, const T & value, const std::string & config_name = DEFAULT_GRAPHITE_CONFIG_NAME, time_t timestamp = 0, const std::string & custom_root_path = "")
     {
-        auto writer = getGraphiteWriter(config_name);
+        auto *writer = getGraphiteWriter(config_name);
         if (writer)
             writer->write(key, value, timestamp, custom_root_path);
     }
@@ -106,7 +91,7 @@ public:
     template <class T>
     void writeToGraphite(const GraphiteWriter::KeyValueVector<T> & key_vals, const std::string & config_name = DEFAULT_GRAPHITE_CONFIG_NAME, time_t timestamp = 0, const std::string & custom_root_path = "")
     {
-        auto writer = getGraphiteWriter(config_name);
+        auto *writer = getGraphiteWriter(config_name);
         if (writer)
             writer->write(key_vals, timestamp, custom_root_path);
     }
@@ -114,7 +99,7 @@ public:
     template <class T>
     void writeToGraphite(const GraphiteWriter::KeyValueVector<T> & key_vals, const std::chrono::system_clock::time_point & current_time, const std::string & custom_root_path)
     {
-        auto writer = getGraphiteWriter();
+        auto *writer = getGraphiteWriter();
         if (writer)
             writer->write(key_vals, std::chrono::system_clock::to_time_t(current_time), custom_root_path);
     }
@@ -131,24 +116,26 @@ public:
     /// also doesn't close global internal pipes for signal handling
     static void closeFDs();
 
+    /// If this method is called after initialization and before run,
+    /// will fork child process and setup watchdog that will print diagnostic info, if the child terminates.
+    /// argv0 is needed to change process name (consequently, it is needed for scripts involving "pgrep", "pidof" to work correctly).
+    void shouldSetupWatchdog(char * argv0_);
+
+    /// Hash of the binary for integrity checks.
+    String getStoredBinaryHash() const;
+
 protected:
-    /// Возвращает TaskManager приложения
-    /// все методы task_manager следует вызывать из одного потока
-    /// иначе возможен deadlock, т.к. joinAll выполняется под локом, а любой метод тоже берет лок
-    Poco::TaskManager & getTaskManager() { return *task_manager; }
-
     virtual void logRevision() const;
 
-    /// Используется при exitOnTaskError()
-    void handleNotification(Poco::TaskFailedNotification *);
-
     /// thread safe
     virtual void handleSignal(int signal_id);
 
     /// initialize termination process and signal handlers
     virtual void initializeTerminationAndSignalProcessing();
 
-    /// реализация обработки сигналов завершения через pipe не требует блокировки сигнала с помощью sigprocmask во всех потоках
+    /// fork the main process and watch if it was killed
+    void setupWatchdog();
+
     void waitForTerminationRequest()
 #if defined(POCO_CLICKHOUSE_PATCH) || POCO_VERSION >= 0x02000000 // in old upstream poco not vitrual
     override
@@ -162,21 +149,13 @@ protected:
 
     virtual std::string getDefaultCorePath() const;
 
-    std::unique_ptr<Poco::TaskManager> task_manager;
-
-    std::optional<DB::StatusFile> pid;
+    std::optional<DB::StatusFile> pid_file;
 
     std::atomic_bool is_cancelled{false};
 
-    /// Флаг устанавливается по сообщению из Task (при аварийном завершении).
-    bool task_failed = false;
-
     bool log_to_console = false;
 
-    /// Событие, чтобы проснуться во время ожидания
-    Poco::Event wakeup_event;
-
-    /// Поток, в котором принимается сигнал HUP/USR1 для закрытия логов.
+    /// A thread that acts on HUP and USR1 signal (close logs).
     Poco::Thread signal_listener_thread;
     std::unique_ptr<Poco::Runnable> signal_listener;
 
@@ -192,8 +171,12 @@ protected:
     Poco::Util::AbstractConfiguration * last_configuration = nullptr;
 
     String build_id_info;
+    String stored_binary_hash;
 
     std::vector<int> handled_signals;
+
+    bool should_setup_watchdog = false;
+    char * argv0 = nullptr;
 };
 
 

base/daemon/CMakeLists.txt

@@ -5,6 +5,11 @@ add_library (daemon
 )
 
 target_include_directories (daemon PUBLIC ..)
+
+if (OS_DARWIN AND NOT MAKE_STATIC_LIBRARIES)
+    target_link_libraries (daemon PUBLIC -Wl,-undefined,dynamic_lookup)
+endif()
+
 target_link_libraries (daemon PUBLIC loggers PRIVATE clickhouse_common_io clickhouse_common_config common ${EXECINFO_LIBRARIES})
 
 if (USE_SENTRY)

base/ext/scope_guard_safe.h

@@ -0,0 +1,66 @@
+#pragma once
+
+#include <ext/scope_guard.h>
+#include <common/logger_useful.h>
+#include <Common/MemoryTracker.h>
+
+/// Same as SCOPE_EXIT() but block the MEMORY_LIMIT_EXCEEDED errors.
+///
+/// Typical example of SCOPE_EXIT_MEMORY() usage is when code under it may do
+/// some tiny allocations, that may fail under high memory pressure or/and low
+/// max_memory_usage (and related limits).
+///
+/// NOTE: it should be used with caution.
+#define SCOPE_EXIT_MEMORY(...) SCOPE_EXIT(                    \
+    MemoryTracker::LockExceptionInThread lock_memory_tracker; \
+    __VA_ARGS__;                                              \
+)
+
+/// Same as SCOPE_EXIT() but try/catch/tryLogCurrentException any exceptions.
+///
+/// SCOPE_EXIT_SAFE() should be used in case the exception during the code
+/// under SCOPE_EXIT() is not "that fatal" and error message in log is enough.
+///
+/// Good example is calling CurrentThread::detachQueryIfNotDetached().
+///
+/// Anti-pattern is calling WriteBuffer::finalize() under SCOPE_EXIT_SAFE()
+/// (since finalize() can do final write and it is better to fail abnormally
+/// instead of ignoring write error).
+///
+/// NOTE: it should be used with double caution.
+#define SCOPE_EXIT_SAFE(...) SCOPE_EXIT(             \
+    try                                              \
+    {                                                \
+        __VA_ARGS__;                                 \
+    }                                                \
+    catch (...)                                      \
+    {                                                \
+        tryLogCurrentException(__PRETTY_FUNCTION__); \
+    }                                                \
+)
+
+/// Same as SCOPE_EXIT() but:
+/// - block the MEMORY_LIMIT_EXCEEDED errors,
+/// - try/catch/tryLogCurrentException any exceptions.
+///
+/// SCOPE_EXIT_MEMORY_SAFE() can be used when the error can be ignored, and in
+/// addition to SCOPE_EXIT_SAFE() it will also lock MEMORY_LIMIT_EXCEEDED to
+/// avoid such exceptions.
+///
+/// It does exists as a separate helper, since you do not need to lock
+/// MEMORY_LIMIT_EXCEEDED always (there are cases when code under SCOPE_EXIT does
+/// not do any allocations, while LockExceptionInThread increment atomic
+/// variable).
+///
+/// NOTE: it should be used with triple caution.
+#define SCOPE_EXIT_MEMORY_SAFE(...) SCOPE_EXIT(                   \
+    try                                                           \
+    {                                                             \
+        MemoryTracker::LockExceptionInThread lock_memory_tracker; \
+        __VA_ARGS__;                                              \
+    }                                                             \
+    catch (...)                                                   \
+    {                                                             \
+        tryLogCurrentException(__PRETTY_FUNCTION__);              \
+    }                                                             \
+)

base/glibc-compatibility/CMakeLists.txt

@@ -1,5 +1,8 @@
 if (GLIBC_COMPATIBILITY)
-    set (ENABLE_FASTMEMCPY ON)
+    add_subdirectory(memcpy)
+    if(TARGET memcpy)
+        set(MEMCPY_LIBRARY memcpy)
+    endif()
 
     enable_language(ASM)
     include(CheckIncludeFile)
@@ -27,10 +30,6 @@ if (GLIBC_COMPATIBILITY)
         list(APPEND glibc_compatibility_sources musl/getentropy.c)
     endif()
 
-    add_library (clickhouse_memcpy OBJECT
-        ${ClickHouse_SOURCE_DIR}/contrib/FastMemcpy/memcpy_wrapper.c
-    )
-
     # Need to omit frame pointers to match the performance of glibc
     set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fomit-frame-pointer")
 
@@ -48,15 +47,16 @@ if (GLIBC_COMPATIBILITY)
         target_compile_options(glibc-compatibility PRIVATE -fPIC)
     endif ()
 
-    target_link_libraries(global-libs INTERFACE glibc-compatibility)
+    target_link_libraries(global-libs INTERFACE glibc-compatibility ${MEMCPY_LIBRARY})
 
     install(
-        TARGETS glibc-compatibility
+        TARGETS glibc-compatibility ${MEMCPY_LIBRARY}
         EXPORT global
         ARCHIVE DESTINATION lib
     )
 
     message (STATUS "Some symbols from glibc will be replaced for compatibility")
+
 elseif (YANDEX_OFFICIAL_BUILD)
     message (WARNING "Option GLIBC_COMPATIBILITY must be turned on for production builds.")
 endif ()

base/glibc-compatibility/memcpy/CMakeLists.txt

@@ -0,0 +1,8 @@
+if (ARCH_AMD64)
+    add_library(memcpy STATIC memcpy.cpp)
+
+    # We allow to include memcpy.h from user code for better inlining.
+    target_include_directories(memcpy PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>)
+
+    target_compile_options(memcpy PRIVATE -fno-builtin-memcpy)
+endif ()

base/glibc-compatibility/memcpy/memcpy.cpp

@@ -0,0 +1,6 @@
+#include "memcpy.h"
+
+extern "C" void * memcpy(void * __restrict dst, const void * __restrict src, size_t size)
+{
+    return inline_memcpy(dst, src, size);
+}

base/glibc-compatibility/memcpy/memcpy.h

@@ -0,0 +1,217 @@
+#include <cstddef>
+
+#include <emmintrin.h>
+
+
+/** Custom memcpy implementation for ClickHouse.
+  * It has the following benefits over using glibc's implementation:
+  * 1. Avoiding dependency on specific version of glibc's symbol, like memcpy@@GLIBC_2.14 for portability.
+  * 2. Avoiding indirect call via PLT due to shared linking, that can be less efficient.
+  * 3. It's possible to include this header and call inline_memcpy directly for better inlining or interprocedural analysis.
+  * 4. Better results on our performance tests on current CPUs: up to 25% on some queries and up to 0.7%..1% in average across all queries.
+  *
+  * Writing our own memcpy is extremely difficult for the following reasons:
+  * 1. The optimal variant depends on the specific CPU model.
+  * 2. The optimal variant depends on the distribution of size arguments.
+  * 3. It depends on the number of threads copying data concurrently.
+  * 4. It also depends on how the calling code is using the copied data and how the different memcpy calls are related to each other.
+  * Due to vast range of scenarios it makes proper testing especially difficult.
+  * When writing our own memcpy there is a risk to overoptimize it
+  * on non-representative microbenchmarks while making real-world use cases actually worse.
+  *
+  * Most of the benchmarks for memcpy on the internet are wrong.
+  *
+  * Let's look at the details:
+  *
+  * For small size, the order of branches in code is important.
+  * There are variants with specific order of branches (like here or in glibc)
+  * or with jump table (in asm code see example from Cosmopolitan libc:
+  * https://github.com/jart/cosmopolitan/blob/de09bec215675e9b0beb722df89c6f794da74f3f/libc/nexgen32e/memcpy.S#L61)
+  * or with Duff device in C (see https://github.com/skywind3000/FastMemcpy/)
+  *
+  * It's also important how to copy uneven sizes.
+  * Almost every implementation, including this, is using two overlapping movs.
+  *
+  * It is important to disable -ftree-loop-distribute-patterns when compiling memcpy implementation,
+  * otherwise the compiler can replace internal loops to a call to memcpy that will lead to infinite recursion.
+  *
+  * For larger sizes it's important to choose the instructions used:
+  * - SSE or AVX or AVX-512;
+  * - rep movsb;
+  * Performance will depend on the size threshold, on the CPU model, on the "erms" flag
+  * ("Enhansed Rep MovS" - it indicates that performance of "rep movsb" is decent for large sizes)
+  * https://stackoverflow.com/questions/43343231/enhanced-rep-movsb-for-memcpy
+  *
+  * Using AVX-512 can be bad due to throttling.
+  * Using AVX can be bad if most code is using SSE due to switching penalty
+  * (it also depends on the usage of "vzeroupper" instruction).
+  * But in some cases AVX gives a win.
+  *
+  * It also depends on how many times the loop will be unrolled.
+  * We are unrolling the loop 8 times (by the number of available registers), but it not always the best.
+  *
+  * It also depends on the usage of aligned or unaligned loads/stores.
+  * We are using unaligned loads and aligned stores.
+  *
+  * It also depends on the usage of prefetch instructions. It makes sense on some Intel CPUs but can slow down performance on AMD.
+  * Setting up correct offset for prefetching is non-obvious.
+  *
+  * Non-temporary (cache bypassing) stores can be used for very large sizes (more than a half of L3 cache).
+  * But the exact threshold is unclear - when doing memcpy from multiple threads the optimal threshold can be lower,
+  * because L3 cache is shared (and L2 cache is partially shared).
+  *
+  * Very large size of memcpy typically indicates suboptimal (not cache friendly) algorithms in code or unrealistic scenarios,
+  * so we don't pay attention to using non-temporary stores.
+  *
+  * On recent Intel CPUs, the presence of "erms" makes "rep movsb" the most benefitial,
+  * even comparing to non-temporary aligned unrolled stores even with the most wide registers.
+  *
+  * memcpy can be written in asm, C or C++. The latter can also use inline asm.
+  * The asm implementation can be better to make sure that compiler won't make the code worse,
+  * to ensure the order of branches, the code layout, the usage of all required registers.
+  * But if it is located in separate translation unit, inlining will not be possible
+  * (inline asm can be used to overcome this limitation).
+  * Sometimes C or C++ code can be further optimized by compiler.
+  * For example, clang is capable replacing SSE intrinsics to AVX code if -mavx is used.
+  *
+  * Please note that compiler can replace plain code to memcpy and vice versa.
+  * - memcpy with compile-time known small size is replaced to simple instructions without a call to memcpy;
+  *   it is controlled by -fbuiltin-memcpy and can be manually ensured by calling __builtin_memcpy.
+  *   This is often used to implement unaligned load/store without undefined behaviour in C++.
+  * - a loop with copying bytes can be recognized and replaced by a call to memcpy;
+  *   it is controlled by -ftree-loop-distribute-patterns.
+  * - also note that a loop with copying bytes can be unrolled, peeled and vectorized that will give you
+  *   inline code somewhat similar to a decent implementation of memcpy.
+  *
+  * This description is up to date as of Mar 2021.
+  *
+  * How to test the memcpy implementation for performance:
+  * 1. Test on real production workload.
+  * 2. For synthetic test, see utils/memcpy-bench, but make sure you will do the best to exhaust the wide range of scenarios.
+  *
+  * TODO: Add self-tuning memcpy with bayesian bandits algorithm for large sizes.
+  * See https://habr.com/en/company/yandex/blog/457612/
+  */
+
+
+static inline void * inline_memcpy(void * __restrict dst_, const void * __restrict src_, size_t size)
+{
+    /// We will use pointer arithmetic, so char pointer will be used.
+    /// Note that __restrict makes sense (otherwise compiler will reload data from memory
+    /// instead of using the value of registers due to possible aliasing).
+    char * __restrict dst = reinterpret_cast<char * __restrict>(dst_);
+    const char * __restrict src = reinterpret_cast<const char * __restrict>(src_);
+
+    /// Standard memcpy returns the original value of dst. It is rarely used but we have to do it.
+    /// If you use memcpy with small but non-constant sizes, you can call inline_memcpy directly
+    /// for inlining and removing this single instruction.
+    void * ret = dst;
+
+tail:
+    /// Small sizes and tails after the loop for large sizes.
+    /// The order of branches is important but in fact the optimal order depends on the distribution of sizes in your application.
+    /// This order of branches is from the disassembly of glibc's code.
+    /// We copy chunks of possibly uneven size with two overlapping movs.
+    /// Example: to copy 5 bytes [0, 1, 2, 3, 4] we will copy tail [1, 2, 3, 4] first and then head [0, 1, 2, 3].
+    if (size <= 16)
+    {
+        if (size >= 8)
+        {
+            /// Chunks of 8..16 bytes.
+            __builtin_memcpy(dst + size - 8, src + size - 8, 8);
+            __builtin_memcpy(dst, src, 8);
+        }
+        else if (size >= 4)
+        {
+            /// Chunks of 4..7 bytes.
+            __builtin_memcpy(dst + size - 4, src + size - 4, 4);
+            __builtin_memcpy(dst, src, 4);
+        }
+        else if (size >= 2)
+        {
+            /// Chunks of 2..3 bytes.
+            __builtin_memcpy(dst + size - 2, src + size - 2, 2);
+            __builtin_memcpy(dst, src, 2);
+        }
+        else if (size >= 1)
+        {
+            /// A single byte.
+            *dst = *src;
+        }
+        /// No bytes remaining.
+    }
+    else
+    {
+        /// Medium and large sizes.
+        if (size <= 128)
+        {
+            /// Medium size, not enough for full loop unrolling.
+
+            /// We will copy the last 16 bytes.
+            _mm_storeu_si128(reinterpret_cast<__m128i *>(dst + size - 16), _mm_loadu_si128(reinterpret_cast<const __m128i *>(src + size - 16)));
+
+            /// Then we will copy every 16 bytes from the beginning in a loop.
+            /// The last loop iteration will possibly overwrite some part of already copied last 16 bytes.
+            /// This is Ok, similar to the code for small sizes above.
+            while (size > 16)
+            {
+                _mm_storeu_si128(reinterpret_cast<__m128i *>(dst), _mm_loadu_si128(reinterpret_cast<const __m128i *>(src)));
+                dst += 16;
+                src += 16;
+                size -= 16;
+            }
+        }
+        else
+        {
+            /// Large size with fully unrolled loop.
+
+            /// Align destination to 16 bytes boundary.
+            size_t padding = (16 - (reinterpret_cast<size_t>(dst) & 15)) & 15;
+
+            /// If not aligned - we will copy first 16 bytes with unaligned stores.
+            if (padding > 0)
+            {
+                __m128i head = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src));
+                _mm_storeu_si128(reinterpret_cast<__m128i*>(dst), head);
+                dst += padding;
+                src += padding;
+                size -= padding;
+            }
+
+            /// Aligned unrolled copy. We will use half of available SSE registers.
+            /// It's not possible to have both src and dst aligned.
+            /// So, we will use aligned stores and unaligned loads.
+            __m128i c0, c1, c2, c3, c4, c5, c6, c7;
+
+            while (size >= 128)
+            {
+                c0 = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src) + 0);
+                c1 = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src) + 1);
+                c2 = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src) + 2);
+                c3 = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src) + 3);
+                c4 = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src) + 4);
+                c5 = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src) + 5);
+                c6 = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src) + 6);
+                c7 = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src) + 7);
+                src += 128;
+                _mm_store_si128((reinterpret_cast<__m128i*>(dst) + 0), c0);
+                _mm_store_si128((reinterpret_cast<__m128i*>(dst) + 1), c1);
+                _mm_store_si128((reinterpret_cast<__m128i*>(dst) + 2), c2);
+                _mm_store_si128((reinterpret_cast<__m128i*>(dst) + 3), c3);
+                _mm_store_si128((reinterpret_cast<__m128i*>(dst) + 4), c4);
+                _mm_store_si128((reinterpret_cast<__m128i*>(dst) + 5), c5);
+                _mm_store_si128((reinterpret_cast<__m128i*>(dst) + 6), c6);
+                _mm_store_si128((reinterpret_cast<__m128i*>(dst) + 7), c7);
+                dst += 128;
+
+                size -= 128;
+            }
+
+            /// The latest remaining 0..127 bytes will be processed as usual.
+            goto tail;
+        }
+    }
+
+    return ret;
+}
+

base/glibc-compatibility/musl/__polevll.c

@@ -0,0 +1,93 @@
+/* origin: OpenBSD /usr/src/lib/libm/src/polevll.c */
+/*
+ * Copyright (c) 2008 Stephen L. Moshier <steve@moshier.net>
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+/*
+ *      Evaluate polynomial
+ *
+ *
+ * SYNOPSIS:
+ *
+ * int N;
+ * long double x, y, coef[N+1], polevl[];
+ *
+ * y = polevll( x, coef, N );
+ *
+ *
+ * DESCRIPTION:
+ *
+ * Evaluates polynomial of degree N:
+ *
+ *                     2          N
+ * y  =  C  + C x + C x  +...+ C x
+ *        0    1     2          N
+ *
+ * Coefficients are stored in reverse order:
+ *
+ * coef[0] = C  , ..., coef[N] = C  .
+ *            N                   0
+ *
+ *  The function p1evll() assumes that coef[N] = 1.0 and is
+ * omitted from the array.  Its calling arguments are
+ * otherwise the same as polevll().
+ *
+ *
+ * SPEED:
+ *
+ * In the interest of speed, there are no checks for out
+ * of bounds arithmetic.  This routine is used by most of
+ * the functions in the library.  Depending on available
+ * equipment features, the user may wish to rewrite the
+ * program in microcode or assembly language.
+ *
+ */
+
+#include "libm.h"
+
+#if LDBL_MANT_DIG == 53 && LDBL_MAX_EXP == 1024
+#else
+/*
+ * Polynomial evaluator:
+ *  P[0] x^n  +  P[1] x^(n-1)  +  ...  +  P[n]
+ */
+long double __polevll(long double x, const long double *P, int n)
+{
+	long double y;
+
+	y = *P++;
+	do {
+		y = y * x + *P++;
+	} while (--n);
+
+	return y;
+}
+
+/*
+ * Polynomial evaluator:
+ *  x^n  +  P[0] x^(n-1)  +  P[1] x^(n-2)  +  ...  +  P[n]
+ */
+long double __p1evll(long double x, const long double *P, int n)
+{
+	long double y;
+
+	n -= 1;
+	y = x + *P++;
+	do {
+		y = y * x + *P++;
+	} while (--n);
+
+	return y;
+}
+#endif

base/glibc-compatibility/musl/lgamma.c

@@ -258,11 +258,3 @@ double lgamma_r(double x, int *signgamp)
 		r = nadj - r;
 	return r;
 }
-
-
-int signgam;
-
-double lgamma(double x)
-{
-	return lgamma_r(x, &signgam);
-}

base/glibc-compatibility/musl/lgammal.c

@@ -38,7 +38,7 @@
  *                          = log(6.3*5.3) + lgamma(5.3)
  *                          = log(6.3*5.3*4.3*3.3*2.3) + lgamma(2.3)
  *   2. Polynomial approximation of lgamma around its
- *      minimun ymin=1.461632144968362245 to maintain monotonicity.
+ *      minimum ymin=1.461632144968362245 to maintain monotonicity.
  *      On [ymin-0.23, ymin+0.27] (i.e., [1.23164,1.73163]), use
  *              Let z = x-ymin;
  *              lgamma(x) = -1.214862905358496078218 + z^2*poly(z)
@@ -328,12 +328,3 @@ long double lgammal_r(long double x, int *sg)
 	return lgamma_r(x, sg);
 }
 #endif
-
-
-int signgam_lgammal;
-
-long double lgammal(long double x)
-{
-	return lgammal_r(x, &signgam_lgammal);
-}
-

base/glibc-compatibility/musl/mkstemps.c

@@ -0,0 +1,44 @@
+#include <errno.h>
+#include <fcntl.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+
+/* This assumes that a check for the
+   template size has already been made */
+static char * __randname(char * template)
+{
+    int i;
+    struct timespec ts;
+    unsigned long r;
+
+    clock_gettime(CLOCK_REALTIME, &ts);
+    r = (ts.tv_nsec * 65537) ^ ((((intptr_t)(&ts)) / 16) + ((intptr_t)template));
+    for (i = 0; i < 6; i++, r >>= 5)
+        template[i] = 'A' + (r & 15) + (r & 16) * 2;
+
+    return template;
+}
+
+int mkstemps(char * template, int len)
+{
+    size_t l = strlen(template);
+    if (l < 6 || len > l - 6 || memcmp(template + l - len - 6, "XXXXXX", 6))
+    {
+        errno = EINVAL;
+        return -1;
+    }
+
+    int fd, retries = 100;
+    do
+    {
+        __randname(template + l - len - 6);
+        if ((fd = open(template, O_RDWR | O_CREAT | O_EXCL, 0600)) >= 0)
+            return fd;
+    } while (--retries && errno == EEXIST);
+
+    memcpy(template + l - len - 6, "XXXXXX", 6);
+    return -1;
+}

base/glibc-compatibility/musl/powf.c

@@ -0,0 +1,185 @@
+/*
+ * Copyright (c) 2017-2018, Arm Limited.
+ * SPDX-License-Identifier: MIT
+ */
+
+#include <math.h>
+#include <stdint.h>
+#include "libm.h"
+#include "exp2f_data.h"
+#include "powf_data.h"
+
+/*
+POWF_LOG2_POLY_ORDER = 5
+EXP2F_TABLE_BITS = 5
+
+ULP error: 0.82 (~ 0.5 + relerr*2^24)
+relerr: 1.27 * 2^-26 (Relative error ~= 128*Ln2*relerr_log2 + relerr_exp2)
+relerr_log2: 1.83 * 2^-33 (Relative error of logx.)
+relerr_exp2: 1.69 * 2^-34 (Relative error of exp2(ylogx).)
+*/
+
+#define N (1 << POWF_LOG2_TABLE_BITS)
+#define T __powf_log2_data.tab
+#define A __powf_log2_data.poly
+#define OFF 0x3f330000
+
+/* Subnormal input is normalized so ix has negative biased exponent.
+   Output is multiplied by N (POWF_SCALE) if TOINT_INTRINICS is set.  */
+static inline double_t log2_inline(uint32_t ix)
+{
+	double_t z, r, r2, r4, p, q, y, y0, invc, logc;
+	uint32_t iz, top, tmp;
+	int k, i;
+
+	/* x = 2^k z; where z is in range [OFF,2*OFF] and exact.
+	   The range is split into N subintervals.
+	   The ith subinterval contains z and c is near its center.  */
+	tmp = ix - OFF;
+	i = (tmp >> (23 - POWF_LOG2_TABLE_BITS)) % N;
+	top = tmp & 0xff800000;
+	iz = ix - top;
+	k = (int32_t)top >> (23 - POWF_SCALE_BITS); /* arithmetic shift */
+	invc = T[i].invc;
+	logc = T[i].logc;
+	z = (double_t)asfloat(iz);
+
+	/* log2(x) = log1p(z/c-1)/ln2 + log2(c) + k */
+	r = z * invc - 1;
+	y0 = logc + (double_t)k;
+
+	/* Pipelined polynomial evaluation to approximate log1p(r)/ln2.  */
+	r2 = r * r;
+	y = A[0] * r + A[1];
+	p = A[2] * r + A[3];
+	r4 = r2 * r2;
+	q = A[4] * r + y0;
+	q = p * r2 + q;
+	y = y * r4 + q;
+	return y;
+}
+
+#undef N
+#undef T
+#define N (1 << EXP2F_TABLE_BITS)
+#define T __exp2f_data.tab
+#define SIGN_BIAS (1 << (EXP2F_TABLE_BITS + 11))
+
+/* The output of log2 and thus the input of exp2 is either scaled by N
+   (in case of fast toint intrinsics) or not.  The unscaled xd must be
+   in [-1021,1023], sign_bias sets the sign of the result.  */
+static inline float exp2_inline(double_t xd, uint32_t sign_bias)
+{
+	uint64_t ki, ski, t;
+	double_t kd, z, r, r2, y, s;
+
+#if TOINT_INTRINSICS
+#define C __exp2f_data.poly_scaled
+	/* N*x = k + r with r in [-1/2, 1/2] */
+	kd = roundtoint(xd); /* k */
+	ki = converttoint(xd);
+#else
+#define C __exp2f_data.poly
+#define SHIFT __exp2f_data.shift_scaled
+	/* x = k/N + r with r in [-1/(2N), 1/(2N)] */
+	kd = eval_as_double(xd + SHIFT);
+	ki = asuint64(kd);
+	kd -= SHIFT; /* k/N */
+#endif
+	r = xd - kd;
+
+	/* exp2(x) = 2^(k/N) * 2^r ~= s * (C0*r^3 + C1*r^2 + C2*r + 1) */
+	t = T[ki % N];
+	ski = ki + sign_bias;
+	t += ski << (52 - EXP2F_TABLE_BITS);
+	s = asdouble(t);
+	z = C[0] * r + C[1];
+	r2 = r * r;
+	y = C[2] * r + 1;
+	y = z * r2 + y;
+	y = y * s;
+	return eval_as_float(y);
+}
+
+/* Returns 0 if not int, 1 if odd int, 2 if even int.  The argument is
+   the bit representation of a non-zero finite floating-point value.  */
+static inline int checkint(uint32_t iy)
+{
+	int e = iy >> 23 & 0xff;
+	if (e < 0x7f)
+		return 0;
+	if (e > 0x7f + 23)
+		return 2;
+	if (iy & ((1 << (0x7f + 23 - e)) - 1))
+		return 0;
+	if (iy & (1 << (0x7f + 23 - e)))
+		return 1;
+	return 2;
+}
+
+static inline int zeroinfnan(uint32_t ix)
+{
+	return 2 * ix - 1 >= 2u * 0x7f800000 - 1;
+}
+
+float powf(float x, float y)
+{
+	uint32_t sign_bias = 0;
+	uint32_t ix, iy;
+
+	ix = asuint(x);
+	iy = asuint(y);
+	if (predict_false(ix - 0x00800000 >= 0x7f800000 - 0x00800000 ||
+			  zeroinfnan(iy))) {
+		/* Either (x < 0x1p-126 or inf or nan) or (y is 0 or inf or nan).  */
+		if (predict_false(zeroinfnan(iy))) {
+			if (2 * iy == 0)
+				return issignalingf_inline(x) ? x + y : 1.0f;
+			if (ix == 0x3f800000)
+				return issignalingf_inline(y) ? x + y : 1.0f;
+			if (2 * ix > 2u * 0x7f800000 ||
+			    2 * iy > 2u * 0x7f800000)
+				return x + y;
+			if (2 * ix == 2 * 0x3f800000)
+				return 1.0f;
+			if ((2 * ix < 2 * 0x3f800000) == !(iy & 0x80000000))
+				return 0.0f; /* |x|<1 && y==inf or |x|>1 && y==-inf.  */
+			return y * y;
+		}
+		if (predict_false(zeroinfnan(ix))) {
+			float_t x2 = x * x;
+			if (ix & 0x80000000 && checkint(iy) == 1)
+				x2 = -x2;
+			/* Without the barrier some versions of clang hoist the 1/x2 and
+			   thus division by zero exception can be signaled spuriously.  */
+			return iy & 0x80000000 ? fp_barrierf(1 / x2) : x2;
+		}
+		/* x and y are non-zero finite.  */
+		if (ix & 0x80000000) {
+			/* Finite x < 0.  */
+			int yint = checkint(iy);
+			if (yint == 0)
+				return __math_invalidf(x);
+			if (yint == 1)
+				sign_bias = SIGN_BIAS;
+			ix &= 0x7fffffff;
+		}
+		if (ix < 0x00800000) {
+			/* Normalize subnormal x so exponent becomes negative.  */
+			ix = asuint(x * 0x1p23f);
+			ix &= 0x7fffffff;
+			ix -= 23 << 23;
+		}
+	}
+	double_t logx = log2_inline(ix);
+	double_t ylogx = y * logx; /* cannot overflow, y is single prec.  */
+	if (predict_false((asuint64(ylogx) >> 47 & 0xffff) >=
+			  asuint64(126.0 * POWF_SCALE) >> 47)) {
+		/* |y*log(x)| >= 126.  */
+		if (ylogx > 0x1.fffffffd1d571p+6 * POWF_SCALE)
+			return __math_oflowf(sign_bias);
+		if (ylogx <= -150.0 * POWF_SCALE)
+			return __math_uflowf(sign_bias);
+	}
+	return exp2_inline(ylogx, sign_bias);
+}

base/glibc-compatibility/musl/powf_data.c

@@ -0,0 +1,34 @@
+/*
+ * Data definition for powf.
+ *
+ * Copyright (c) 2017-2018, Arm Limited.
+ * SPDX-License-Identifier: MIT
+ */
+
+#include "powf_data.h"
+
+const struct powf_log2_data __powf_log2_data = {
+  .tab = {
+  { 0x1.661ec79f8f3bep+0, -0x1.efec65b963019p-2 * POWF_SCALE },
+  { 0x1.571ed4aaf883dp+0, -0x1.b0b6832d4fca4p-2 * POWF_SCALE },
+  { 0x1.49539f0f010bp+0, -0x1.7418b0a1fb77bp-2 * POWF_SCALE },
+  { 0x1.3c995b0b80385p+0, -0x1.39de91a6dcf7bp-2 * POWF_SCALE },
+  { 0x1.30d190c8864a5p+0, -0x1.01d9bf3f2b631p-2 * POWF_SCALE },
+  { 0x1.25e227b0b8eap+0, -0x1.97c1d1b3b7afp-3 * POWF_SCALE },
+  { 0x1.1bb4a4a1a343fp+0, -0x1.2f9e393af3c9fp-3 * POWF_SCALE },
+  { 0x1.12358f08ae5bap+0, -0x1.960cbbf788d5cp-4 * POWF_SCALE },
+  { 0x1.0953f419900a7p+0, -0x1.a6f9db6475fcep-5 * POWF_SCALE },
+  { 0x1p+0, 0x0p+0 * POWF_SCALE },
+  { 0x1.e608cfd9a47acp-1, 0x1.338ca9f24f53dp-4 * POWF_SCALE },
+  { 0x1.ca4b31f026aap-1, 0x1.476a9543891bap-3 * POWF_SCALE },
+  { 0x1.b2036576afce6p-1, 0x1.e840b4ac4e4d2p-3 * POWF_SCALE },
+  { 0x1.9c2d163a1aa2dp-1, 0x1.40645f0c6651cp-2 * POWF_SCALE },
+  { 0x1.886e6037841edp-1, 0x1.88e9c2c1b9ff8p-2 * POWF_SCALE },
+  { 0x1.767dcf5534862p-1, 0x1.ce0a44eb17bccp-2 * POWF_SCALE },
+  },
+  .poly = {
+  0x1.27616c9496e0bp-2 * POWF_SCALE, -0x1.71969a075c67ap-2 * POWF_SCALE,
+  0x1.ec70a6ca7baddp-2 * POWF_SCALE, -0x1.7154748bef6c8p-1 * POWF_SCALE,
+  0x1.71547652ab82bp0 * POWF_SCALE,
+  }
+};

base/glibc-compatibility/musl/powf_data.h

@@ -0,0 +1,26 @@
+/*
+ * Copyright (c) 2017-2018, Arm Limited.
+ * SPDX-License-Identifier: MIT
+ */
+#ifndef _POWF_DATA_H
+#define _POWF_DATA_H
+
+#include "libm.h"
+#include "exp2f_data.h"
+
+#define POWF_LOG2_TABLE_BITS 4
+#define POWF_LOG2_POLY_ORDER 5
+#if TOINT_INTRINSICS
+#define POWF_SCALE_BITS EXP2F_TABLE_BITS
+#else
+#define POWF_SCALE_BITS 0
+#endif
+#define POWF_SCALE ((double)(1 << POWF_SCALE_BITS))
+extern hidden const struct powf_log2_data {
+	struct {
+		double invc, logc;
+	} tab[1 << POWF_LOG2_TABLE_BITS];
+	double poly[POWF_LOG2_POLY_ORDER];
+} __powf_log2_data;
+
+#endif

base/glibc-compatibility/musl/powl.c

@@ -0,0 +1,525 @@
+/* origin: OpenBSD /usr/src/lib/libm/src/ld80/e_powl.c */
+/*
+ * Copyright (c) 2008 Stephen L. Moshier <steve@moshier.net>
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+/*                                                      powl.c
+ *
+ *      Power function, long double precision
+ *
+ *
+ * SYNOPSIS:
+ *
+ * long double x, y, z, powl();
+ *
+ * z = powl( x, y );
+ *
+ *
+ * DESCRIPTION:
+ *
+ * Computes x raised to the yth power.  Analytically,
+ *
+ *      x**y  =  exp( y log(x) ).
+ *
+ * Following Cody and Waite, this program uses a lookup table
+ * of 2**-i/32 and pseudo extended precision arithmetic to
+ * obtain several extra bits of accuracy in both the logarithm
+ * and the exponential.
+ *
+ *
+ * ACCURACY:
+ *
+ * The relative error of pow(x,y) can be estimated
+ * by   y dl ln(2),   where dl is the absolute error of
+ * the internally computed base 2 logarithm.  At the ends
+ * of the approximation interval the logarithm equal 1/32
+ * and its relative error is about 1 lsb = 1.1e-19.  Hence
+ * the predicted relative error in the result is 2.3e-21 y .
+ *
+ *                      Relative error:
+ * arithmetic   domain     # trials      peak         rms
+ *
+ *    IEEE     +-1000       40000      2.8e-18      3.7e-19
+ * .001 < x < 1000, with log(x) uniformly distributed.
+ * -1000 < y < 1000, y uniformly distributed.
+ *
+ *    IEEE     0,8700       60000      6.5e-18      1.0e-18
+ * 0.99 < x < 1.01, 0 < y < 8700, uniformly distributed.
+ *
+ *
+ * ERROR MESSAGES:
+ *
+ *   message         condition      value returned
+ * pow overflow     x**y > MAXNUM      INFINITY
+ * pow underflow   x**y < 1/MAXNUM       0.0
+ * pow domain      x<0 and y noninteger  0.0
+ *
+ */
+
+#include "libm.h"
+
+#if LDBL_MANT_DIG == 53 && LDBL_MAX_EXP == 1024
+long double powl(long double x, long double y)
+{
+	return pow(x, y);
+}
+#elif LDBL_MANT_DIG == 64 && LDBL_MAX_EXP == 16384
+
+/* Table size */
+#define NXT 32
+
+/* log(1+x) =  x - .5x^2 + x^3 *  P(z)/Q(z)
+ * on the domain  2^(-1/32) - 1  <=  x  <=  2^(1/32) - 1
+ */
+static const long double P[] = {
+ 8.3319510773868690346226E-4L,
+ 4.9000050881978028599627E-1L,
+ 1.7500123722550302671919E0L,
+ 1.4000100839971580279335E0L,
+};
+static const long double Q[] = {
+/* 1.0000000000000000000000E0L,*/
+ 5.2500282295834889175431E0L,
+ 8.4000598057587009834666E0L,
+ 4.2000302519914740834728E0L,
+};
+/* A[i] = 2^(-i/32), rounded to IEEE long double precision.
+ * If i is even, A[i] + B[i/2] gives additional accuracy.
+ */
+static const long double A[33] = {
+ 1.0000000000000000000000E0L,
+ 9.7857206208770013448287E-1L,
+ 9.5760328069857364691013E-1L,
+ 9.3708381705514995065011E-1L,
+ 9.1700404320467123175367E-1L,
+ 8.9735453750155359320742E-1L,
+ 8.7812608018664974155474E-1L,
+ 8.5930964906123895780165E-1L,
+ 8.4089641525371454301892E-1L,
+ 8.2287773907698242225554E-1L,
+ 8.0524516597462715409607E-1L,
+ 7.8799042255394324325455E-1L,
+ 7.7110541270397041179298E-1L,
+ 7.5458221379671136985669E-1L,
+ 7.3841307296974965571198E-1L,
+ 7.2259040348852331001267E-1L,
+ 7.0710678118654752438189E-1L,
+ 6.9195494098191597746178E-1L,
+ 6.7712777346844636413344E-1L,
+ 6.6261832157987064729696E-1L,
+ 6.4841977732550483296079E-1L,
+ 6.3452547859586661129850E-1L,
+ 6.2092890603674202431705E-1L,
+ 6.0762367999023443907803E-1L,
+ 5.9460355750136053334378E-1L,
+ 5.8186242938878875689693E-1L,
+ 5.6939431737834582684856E-1L,
+ 5.5719337129794626814472E-1L,
+ 5.4525386633262882960438E-1L,
+ 5.3357020033841180906486E-1L,
+ 5.2213689121370692017331E-1L,
+ 5.1094857432705833910408E-1L,
+ 5.0000000000000000000000E-1L,
+};
+static const long double B[17] = {
+ 0.0000000000000000000000E0L,
+ 2.6176170809902549338711E-20L,
+-1.0126791927256478897086E-20L,
+ 1.3438228172316276937655E-21L,
+ 1.2207982955417546912101E-20L,
+-6.3084814358060867200133E-21L,
+ 1.3164426894366316434230E-20L,
+-1.8527916071632873716786E-20L,
+ 1.8950325588932570796551E-20L,
+ 1.5564775779538780478155E-20L,
+ 6.0859793637556860974380E-21L,
+-2.0208749253662532228949E-20L,
+ 1.4966292219224761844552E-20L,
+ 3.3540909728056476875639E-21L,
+-8.6987564101742849540743E-22L,
+-1.2327176863327626135542E-20L,
+ 0.0000000000000000000000E0L,
+};
+
+/* 2^x = 1 + x P(x),
+ * on the interval -1/32 <= x <= 0
+ */
+static const long double R[] = {
+ 1.5089970579127659901157E-5L,
+ 1.5402715328927013076125E-4L,
+ 1.3333556028915671091390E-3L,
+ 9.6181291046036762031786E-3L,
+ 5.5504108664798463044015E-2L,
+ 2.4022650695910062854352E-1L,
+ 6.9314718055994530931447E-1L,
+};
+
+#define MEXP (NXT*16384.0L)
+/* The following if denormal numbers are supported, else -MEXP: */
+#define MNEXP (-NXT*(16384.0L+64.0L))
+/* log2(e) - 1 */
+#define LOG2EA 0.44269504088896340735992L
+
+#define F W
+#define Fa Wa
+#define Fb Wb
+#define G W
+#define Ga Wa
+#define Gb u
+#define H W
+#define Ha Wb
+#define Hb Wb
+
+static const long double MAXLOGL = 1.1356523406294143949492E4L;
+static const long double MINLOGL = -1.13994985314888605586758E4L;
+static const long double LOGE2L = 6.9314718055994530941723E-1L;
+static const long double huge = 0x1p10000L;
+/* XXX Prevent gcc from erroneously constant folding this. */
+static const volatile long double twom10000 = 0x1p-10000L;
+
+static long double reducl(long double);
+static long double powil(long double, int);
+
+long double __polevll(long double x, const long double *P, int n);
+long double __p1evll(long double x, const long double *P, int n);
+
+long double powl(long double x, long double y)
+{
+	/* double F, Fa, Fb, G, Ga, Gb, H, Ha, Hb */
+	int i, nflg, iyflg, yoddint;
+	long e;
+	volatile long double z=0;
+	long double w=0, W=0, Wa=0, Wb=0, ya=0, yb=0, u=0;
+
+	/* make sure no invalid exception is raised by nan comparison */
+	if (isnan(x)) {
+		if (!isnan(y) && y == 0.0)
+			return 1.0;
+		return x;
+	}
+	if (isnan(y)) {
+		if (x == 1.0)
+			return 1.0;
+		return y;
+	}
+	if (x == 1.0)
+		return 1.0; /* 1**y = 1, even if y is nan */
+	if (x == -1.0 && !isfinite(y))
+		return 1.0; /* -1**inf = 1 */
+	if (y == 0.0)
+		return 1.0; /* x**0 = 1, even if x is nan */
+	if (y == 1.0)
+		return x;
+	if (y >= LDBL_MAX) {
+		if (x > 1.0 || x < -1.0)
+			return INFINITY;
+		if (x != 0.0)
+			return 0.0;
+	}
+	if (y <= -LDBL_MAX) {
+		if (x > 1.0 || x < -1.0)
+			return 0.0;
+		if (x != 0.0 || y == -INFINITY)
+			return INFINITY;
+	}
+	if (x >= LDBL_MAX) {
+		if (y > 0.0)
+			return INFINITY;
+		return 0.0;
+	}
+
+	w = floorl(y);
+
+	/* Set iyflg to 1 if y is an integer. */
+	iyflg = 0;
+	if (w == y)
+		iyflg = 1;
+
+	/* Test for odd integer y. */
+	yoddint = 0;
+	if (iyflg) {
+		ya = fabsl(y);
+		ya = floorl(0.5 * ya);
+		yb = 0.5 * fabsl(w);
+		if( ya != yb )
+			yoddint = 1;
+	}
+
+	if (x <= -LDBL_MAX) {
+		if (y > 0.0) {
+			if (yoddint)
+				return -INFINITY;
+			return INFINITY;
+		}
+		if (y < 0.0) {
+			if (yoddint)
+				return -0.0;
+			return 0.0;
+		}
+	}
+	nflg = 0; /* (x<0)**(odd int) */
+	if (x <= 0.0) {
+		if (x == 0.0) {
+			if (y < 0.0) {
+				if (signbit(x) && yoddint)
+					/* (-0.0)**(-odd int) = -inf, divbyzero */
+					return -1.0/0.0;
+				/* (+-0.0)**(negative) = inf, divbyzero */
+				return 1.0/0.0;
+			}
+			if (signbit(x) && yoddint)
+				return -0.0;
+			return 0.0;
+		}
+		if (iyflg == 0)
+			return (x - x) / (x - x); /* (x<0)**(non-int) is NaN */
+		/* (x<0)**(integer) */
+		if (yoddint)
+			nflg = 1; /* negate result */
+		x = -x;
+	}
+	/* (+integer)**(integer)  */
+	if (iyflg && floorl(x) == x && fabsl(y) < 32768.0) {
+		w = powil(x, (int)y);
+		return nflg ? -w : w;
+	}
+
+	/* separate significand from exponent */
+	x = frexpl(x, &i);
+	e = i;
+
+	/* find significand in antilog table A[] */
+	i = 1;
+	if (x <= A[17])
+		i = 17;
+	if (x <= A[i+8])
+		i += 8;
+	if (x <= A[i+4])
+		i += 4;
+	if (x <= A[i+2])
+		i += 2;
+	if (x >= A[1])
+		i = -1;
+	i += 1;
+
+	/* Find (x - A[i])/A[i]
+	 * in order to compute log(x/A[i]):
+	 *
+	 * log(x) = log( a x/a ) = log(a) + log(x/a)
+	 *
+	 * log(x/a) = log(1+v),  v = x/a - 1 = (x-a)/a
+	 */
+	x -= A[i];
+	x -= B[i/2];
+	x /= A[i];
+
+	/* rational approximation for log(1+v):
+	 *
+	 * log(1+v)  =  v  -  v**2/2  +  v**3 P(v) / Q(v)
+	 */
+	z = x*x;
+	w = x * (z * __polevll(x, P, 3) / __p1evll(x, Q, 3));
+	w = w - 0.5*z;
+
+	/* Convert to base 2 logarithm:
+	 * multiply by log2(e) = 1 + LOG2EA
+	 */
+	z = LOG2EA * w;
+	z += w;
+	z += LOG2EA * x;
+	z += x;
+
+	/* Compute exponent term of the base 2 logarithm. */
+	w = -i;
+	w /= NXT;
+	w += e;
+	/* Now base 2 log of x is w + z. */
+
+	/* Multiply base 2 log by y, in extended precision. */
+
+	/* separate y into large part ya
+	 * and small part yb less than 1/NXT
+	 */
+	ya = reducl(y);
+	yb = y - ya;
+
+	/* (w+z)(ya+yb)
+	 * = w*ya + w*yb + z*y
+	 */
+	F = z * y  +  w * yb;
+	Fa = reducl(F);
+	Fb = F - Fa;
+
+	G = Fa + w * ya;
+	Ga = reducl(G);
+	Gb = G - Ga;
+
+	H = Fb + Gb;
+	Ha = reducl(H);
+	w = (Ga + Ha) * NXT;
+
+	/* Test the power of 2 for overflow */
+	if (w > MEXP)
+		return huge * huge;  /* overflow */
+	if (w < MNEXP)
+		return twom10000 * twom10000;  /* underflow */
+
+	e = w;
+	Hb = H - Ha;
+
+	if (Hb > 0.0) {
+		e += 1;
+		Hb -= 1.0/NXT;  /*0.0625L;*/
+	}
+
+	/* Now the product y * log2(x)  =  Hb + e/NXT.
+	 *
+	 * Compute base 2 exponential of Hb,
+	 * where -0.0625 <= Hb <= 0.
+	 */
+	z = Hb * __polevll(Hb, R, 6);  /*  z = 2**Hb - 1  */
+
+	/* Express e/NXT as an integer plus a negative number of (1/NXT)ths.
+	 * Find lookup table entry for the fractional power of 2.
+	 */
+	if (e < 0)
+		i = 0;
+	else
+		i = 1;
+	i = e/NXT + i;
+	e = NXT*i - e;
+	w = A[e];
+	z = w * z;  /*  2**-e * ( 1 + (2**Hb-1) )  */
+	z = z + w;
+	z = scalbnl(z, i);  /* multiply by integer power of 2 */
+
+	if (nflg)
+		z = -z;
+	return z;
+}
+
+
+/* Find a multiple of 1/NXT that is within 1/NXT of x. */
+static long double reducl(long double x)
+{
+	long double t;
+
+	t = x * NXT;
+	t = floorl(t);
+	t = t / NXT;
+	return t;
+}
+
+/*
+ *      Positive real raised to integer power, long double precision
+ *
+ *
+ * SYNOPSIS:
+ *
+ * long double x, y, powil();
+ * int n;
+ *
+ * y = powil( x, n );
+ *
+ *
+ * DESCRIPTION:
+ *
+ * Returns argument x>0 raised to the nth power.
+ * The routine efficiently decomposes n as a sum of powers of
+ * two. The desired power is a product of two-to-the-kth
+ * powers of x.  Thus to compute the 32767 power of x requires
+ * 28 multiplications instead of 32767 multiplications.
+ *
+ *
+ * ACCURACY:
+ *
+ *                      Relative error:
+ * arithmetic   x domain   n domain  # trials      peak         rms
+ *    IEEE     .001,1000  -1022,1023  50000       4.3e-17     7.8e-18
+ *    IEEE        1,2     -1022,1023  20000       3.9e-17     7.6e-18
+ *    IEEE     .99,1.01     0,8700    10000       3.6e-16     7.2e-17
+ *
+ * Returns MAXNUM on overflow, zero on underflow.
+ */
+
+static long double powil(long double x, int nn)
+{
+	long double ww, y;
+	long double s;
+	int n, e, sign, lx;
+
+	if (nn == 0)
+		return 1.0;
+
+	if (nn < 0) {
+		sign = -1;
+		n = -nn;
+	} else {
+		sign = 1;
+		n = nn;
+	}
+
+	/* Overflow detection */
+
+	/* Calculate approximate logarithm of answer */
+	s = x;
+	s = frexpl( s, &lx);
+	e = (lx - 1)*n;
+	if ((e == 0) || (e > 64) || (e < -64)) {
+		s = (s - 7.0710678118654752e-1L) / (s +  7.0710678118654752e-1L);
+		s = (2.9142135623730950L * s - 0.5 + lx) * nn * LOGE2L;
+	} else {
+		s = LOGE2L * e;
+	}
+
+	if (s > MAXLOGL)
+		return huge * huge;  /* overflow */
+
+	if (s < MINLOGL)
+		return twom10000 * twom10000;  /* underflow */
+	/* Handle tiny denormal answer, but with less accuracy
+	 * since roundoff error in 1.0/x will be amplified.
+	 * The precise demarcation should be the gradual underflow threshold.
+	 */
+	if (s < -MAXLOGL+2.0) {
+		x = 1.0/x;
+		sign = -sign;
+	}
+
+	/* First bit of the power */
+	if (n & 1)
+		y = x;
+	else
+		y = 1.0;
+
+	ww = x;
+	n >>= 1;
+	while (n) {
+		ww = ww * ww;   /* arg to the 2-to-the-kth power */
+		if (n & 1)     /* if that bit is set, then include in product */
+			y *= ww;
+		n >>= 1;
+	}
+
+	if (sign < 0)
+		y = 1.0/y;
+	return y;
+}
+#elif LDBL_MANT_DIG == 113 && LDBL_MAX_EXP == 16384
+// TODO: broken implementation to make things compile
+long double powl(long double x, long double y)
+{
+	return pow(x, y);
+}
+#endif

base/glibc-compatibility/musl/sched_getcpu.c

@@ -4,6 +4,12 @@
 #include "syscall.h"
 #include "atomic.h"
 
+#if defined(__has_feature)
+#if __has_feature(memory_sanitizer)
+#include <sanitizer/msan_interface.h>
+#endif
+#endif
+
 #ifdef VDSO_GETCPU_SYM
 
 static void *volatile vdso_func;
@@ -25,7 +31,7 @@ static void *volatile vdso_func = (void *)getcpu_init;
 int sched_getcpu(void)
 {
 	int r;
-	unsigned cpu;
+	unsigned cpu = 0;
 
 #ifdef VDSO_GETCPU_SYM
 	getcpu_f f = (getcpu_f)vdso_func;
@@ -37,6 +43,13 @@ int sched_getcpu(void)
 #endif
 
 	r = __syscall(SYS_getcpu, &cpu, 0, 0);
-	if (!r) return cpu;
+	if (!r) {
+#if defined(__has_feature)
+#if __has_feature(memory_sanitizer)
+        __msan_unpoison(&cpu, sizeof(cpu));
+#endif
+#endif
+        return cpu;
+    }
 	return __syscall_ret(r);
 }

base/glibc-compatibility/musl/strsignal.c

@@ -0,0 +1,125 @@
+#include <signal.h>
+#include <string.h>
+
+#if (SIGHUP == 1) && (SIGINT == 2) && (SIGQUIT == 3) && (SIGILL == 4) \
+ && (SIGTRAP == 5) && (SIGABRT == 6) && (SIGBUS == 7) && (SIGFPE == 8) \
+ && (SIGKILL == 9) && (SIGUSR1 == 10) && (SIGSEGV == 11) && (SIGUSR2 == 12) \
+ && (SIGPIPE == 13) && (SIGALRM == 14) && (SIGTERM == 15) && (SIGSTKFLT == 16) \
+ && (SIGCHLD == 17) && (SIGCONT == 18) && (SIGSTOP == 19) && (SIGTSTP == 20) \
+ && (SIGTTIN == 21) && (SIGTTOU == 22) && (SIGURG == 23) && (SIGXCPU == 24) \
+ && (SIGXFSZ == 25) && (SIGVTALRM == 26) && (SIGPROF == 27) && (SIGWINCH == 28) \
+ && (SIGPOLL == 29) && (SIGPWR == 30) && (SIGSYS == 31)
+
+#define sigmap(x) x
+
+#else
+
+static const char map[] = {
+	[SIGHUP]    = 1,
+	[SIGINT]    = 2,
+	[SIGQUIT]   = 3,
+	[SIGILL]    = 4,
+	[SIGTRAP]   = 5,
+	[SIGABRT]   = 6,
+	[SIGBUS]    = 7,
+	[SIGFPE]    = 8,
+	[SIGKILL]   = 9,
+	[SIGUSR1]   = 10,
+	[SIGSEGV]   = 11,
+	[SIGUSR2]   = 12,
+	[SIGPIPE]   = 13,
+	[SIGALRM]   = 14,
+	[SIGTERM]   = 15,
+#if defined(SIGSTKFLT)
+	[SIGSTKFLT] = 16,
+#elif defined(SIGEMT)
+	[SIGEMT]    = 16,
+#endif
+	[SIGCHLD]   = 17,
+	[SIGCONT]   = 18,
+	[SIGSTOP]   = 19,
+	[SIGTSTP]   = 20,
+	[SIGTTIN]   = 21,
+	[SIGTTOU]   = 22,
+	[SIGURG]    = 23,
+	[SIGXCPU]   = 24,
+	[SIGXFSZ]   = 25,
+	[SIGVTALRM] = 26,
+	[SIGPROF]   = 27,
+	[SIGWINCH]  = 28,
+	[SIGPOLL]   = 29,
+	[SIGPWR]    = 30,
+	[SIGSYS]    = 31
+};
+
+#define sigmap(x) ((x) >= sizeof map ? (x) : map[(x)])
+
+#endif
+
+static const char strings[] =
+	"Unknown signal\0"
+	"Hangup\0"
+	"Interrupt\0"
+	"Quit\0"
+	"Illegal instruction\0"
+	"Trace/breakpoint trap\0"
+	"Aborted\0"
+	"Bus error\0"
+	"Arithmetic exception\0"
+	"Killed\0"
+	"User defined signal 1\0"
+	"Segmentation fault\0"
+	"User defined signal 2\0"
+	"Broken pipe\0"
+	"Alarm clock\0"
+	"Terminated\0"
+#if defined(SIGSTKFLT)
+	"Stack fault\0"
+#elif defined(SIGEMT)
+	"Emulator trap\0"
+#else
+	"Unknown signal\0"
+#endif
+	"Child process status\0"
+	"Continued\0"
+	"Stopped (signal)\0"
+	"Stopped\0"
+	"Stopped (tty input)\0"
+	"Stopped (tty output)\0"
+	"Urgent I/O condition\0"
+	"CPU time limit exceeded\0"
+	"File size limit exceeded\0"
+	"Virtual timer expired\0"
+	"Profiling timer expired\0"
+	"Window changed\0"
+	"I/O possible\0"
+	"Power failure\0"
+	"Bad system call\0"
+	"RT32"
+	"\0RT33\0RT34\0RT35\0RT36\0RT37\0RT38\0RT39\0RT40"
+	"\0RT41\0RT42\0RT43\0RT44\0RT45\0RT46\0RT47\0RT48"
+	"\0RT49\0RT50\0RT51\0RT52\0RT53\0RT54\0RT55\0RT56"
+	"\0RT57\0RT58\0RT59\0RT60\0RT61\0RT62\0RT63\0RT64"
+#if _NSIG > 65
+	"\0RT65\0RT66\0RT67\0RT68\0RT69\0RT70\0RT71\0RT72"
+	"\0RT73\0RT74\0RT75\0RT76\0RT77\0RT78\0RT79\0RT80"
+	"\0RT81\0RT82\0RT83\0RT84\0RT85\0RT86\0RT87\0RT88"
+	"\0RT89\0RT90\0RT91\0RT92\0RT93\0RT94\0RT95\0RT96"
+	"\0RT97\0RT98\0RT99\0RT100\0RT101\0RT102\0RT103\0RT104"
+	"\0RT105\0RT106\0RT107\0RT108\0RT109\0RT110\0RT111\0RT112"
+	"\0RT113\0RT114\0RT115\0RT116\0RT117\0RT118\0RT119\0RT120"
+	"\0RT121\0RT122\0RT123\0RT124\0RT125\0RT126\0RT127\0RT128"
+#endif
+	"";
+
+char *strsignal(int signum)
+{
+	const char *s = strings;
+
+	signum = sigmap(signum);
+	if (signum - 1U >= _NSIG-1) signum = 0;
+
+	for (; signum--; s++) for (; *s; s++);
+
+	return (char *)s;
+}

base/glibc-compatibility/musl/timerfd.c

@@ -0,0 +1,17 @@
+#include <sys/timerfd.h>
+#include "syscall.h"
+
+int timerfd_create(int clockid, int flags)
+{
+    return syscall(SYS_timerfd_create, clockid, flags);
+}
+
+int timerfd_settime(int fd, int flags, const struct itimerspec *new, struct itimerspec *old)
+{
+    return syscall(SYS_timerfd_settime, fd, flags, new, old);
+}
+
+int timerfd_gettime(int fd, struct itimerspec *cur)
+{
+    return syscall(SYS_timerfd_gettime, fd, cur);
+}

base/harmful/CMakeLists.txt

@@ -0,0 +1,2 @@
+add_library(harmful harmful.c)
+install(TARGETS harmful EXPORT global ARCHIVE DESTINATION lib)

base/harmful/README.md

@@ -0,0 +1 @@
+A library that traps whenever harmful functions from libc are called.

base/harmful/harmful.c

@@ -0,0 +1,248 @@
+/** This library provides runtime instrumentation (hardening)
+  * that ensures no "harmful" functions from libc are called
+  * (by terminating the program immediately).
+  */
+
+/// It is only enabled in debug build (its intended use is for CI checks).
+#if !defined(NDEBUG)
+
+#if defined(__clang__)
+    #pragma clang diagnostic ignored "-Wincompatible-library-redeclaration"
+#else
+    #pragma GCC diagnostic ignored "-Wbuiltin-declaration-mismatch"
+#endif
+
+/// We cannot use libc headers here.
+long write(int, const void *, unsigned long);
+#define TRAP(func) void func() { write(2, #func "\n", __builtin_strlen(#func) + 1); __builtin_trap(); }
+
+/// Trap all non thread-safe functions:
+/// nm -D /lib/x86_64-linux-gnu/{libc.so.6,libdl.so.2,libm.so.6,libpthread.so.0,librt.so.1,libnss_dns.so.2,libresolv.so.2} | grep -P '_r@?$' | awk '{ print $3 }' | sed -r -e 's/_r//' | grep -vP '^_'
+
+/// See also https://reviews.llvm.org/D90944
+
+/// You can edit this list and even comment out some functions.
+/// The only purpose of the library is to force you to pay attention.
+
+TRAP(argp_error)
+TRAP(argp_help)
+TRAP(argp_parse)
+TRAP(argp_state_help)
+TRAP(argp_usage)
+TRAP(asctime)
+TRAP(clearenv)
+TRAP(crypt)
+TRAP(ctime)
+TRAP(cuserid)
+TRAP(drand48)
+TRAP(ecvt)
+TRAP(encrypt)
+TRAP(endfsent)
+TRAP(endgrent)
+TRAP(endhostent)
+TRAP(endnetent)
+TRAP(endnetgrent)
+TRAP(endprotoent)
+TRAP(endpwent)
+TRAP(endservent)
+TRAP(endutent)
+TRAP(endutxent)
+TRAP(erand48)
+TRAP(error_at_line)
+///TRAP(exit)
+TRAP(fcloseall)
+TRAP(fcvt)
+TRAP(fgetgrent)
+TRAP(fgetpwent)
+TRAP(gammal)
+TRAP(getchar_unlocked)
+TRAP(getdate)
+TRAP(getfsent)
+TRAP(getfsfile)
+TRAP(getfsspec)
+TRAP(getgrent)
+TRAP(getgrent_r)
+TRAP(getgrgid)
+TRAP(getgrnam)
+TRAP(gethostbyaddr)
+TRAP(gethostbyname)
+TRAP(gethostbyname2)
+TRAP(gethostent)
+TRAP(getlogin)
+TRAP(getmntent)
+TRAP(getnetbyaddr)
+TRAP(getnetbyname)
+TRAP(getnetent)
+TRAP(getnetgrent)
+TRAP(getnetgrent_r)
+TRAP(getopt)
+TRAP(getopt_long)
+TRAP(getopt_long_only)
+TRAP(getpass)
+TRAP(getprotobyname)
+TRAP(getprotobynumber)
+TRAP(getprotoent)
+TRAP(getpwent)
+TRAP(getpwent_r)
+TRAP(getpwnam)
+TRAP(getpwuid)
+TRAP(getservbyname)
+TRAP(getservbyport)
+TRAP(getservent)
+TRAP(getutent)
+TRAP(getutent_r)
+TRAP(getutid)
+TRAP(getutid_r)
+TRAP(getutline)
+TRAP(getutline_r)
+TRAP(getutxent)
+TRAP(getutxid)
+TRAP(getutxline)
+TRAP(getwchar_unlocked)
+//TRAP(glob)
+//TRAP(glob64)
+TRAP(gmtime)
+TRAP(hcreate)
+TRAP(hdestroy)
+TRAP(hsearch)
+TRAP(innetgr)
+TRAP(jrand48)
+TRAP(l64a)
+TRAP(lcong48)
+TRAP(lgammafNx)
+TRAP(localeconv)
+TRAP(localtime)
+TRAP(login)
+TRAP(login_tty)
+TRAP(logout)
+TRAP(logwtmp)
+TRAP(lrand48)
+TRAP(mallinfo)
+#if !defined(SANITIZER)
+TRAP(mallopt) // Used by tsan
+#endif
+TRAP(mblen)
+TRAP(mbrlen)
+TRAP(mbrtowc)
+TRAP(mbsnrtowcs)
+TRAP(mbsrtowcs)
+//TRAP(mbtowc) // Used by Standard C++ library
+TRAP(mcheck)
+TRAP(mprobe)
+TRAP(mrand48)
+TRAP(mtrace)
+TRAP(muntrace)
+TRAP(nrand48)
+TRAP(__ppc_get_timebase_freq)
+TRAP(ptsname)
+TRAP(putchar_unlocked)
+TRAP(putenv)
+TRAP(pututline)
+TRAP(pututxline)
+TRAP(putwchar_unlocked)
+TRAP(qecvt)
+TRAP(qfcvt)
+TRAP(register_printf_function)
+TRAP(seed48)
+//TRAP(setenv)
+TRAP(setfsent)
+TRAP(setgrent)
+TRAP(sethostent)
+TRAP(sethostid)
+TRAP(setkey)
+//TRAP(setlocale) // Used by replxx at startup
+TRAP(setlogmask)
+TRAP(setnetent)
+TRAP(setnetgrent)
+TRAP(setprotoent)
+TRAP(setpwent)
+TRAP(setservent)
+TRAP(setutent)
+TRAP(setutxent)
+TRAP(siginterrupt)
+TRAP(sigpause)
+//TRAP(sigprocmask)
+TRAP(sigsuspend)
+TRAP(sleep)
+TRAP(srand48)
+//TRAP(strerror) // Used by RocksDB and many other libraries, unfortunately.
+//TRAP(strsignal) // This function is imported from Musl and is thread safe.
+TRAP(strtok)
+TRAP(tcflow)
+TRAP(tcsendbreak)
+TRAP(tmpnam)
+TRAP(ttyname)
+TRAP(unsetenv)
+TRAP(updwtmp)
+TRAP(utmpname)
+TRAP(utmpxname)
+//TRAP(valloc)
+TRAP(vlimit)
+//TRAP(wcrtomb) // Used by Standard C++ library
+TRAP(wcsnrtombs)
+TRAP(wcsrtombs)
+TRAP(wctomb)
+TRAP(wordexp)
+TRAP(basename)
+TRAP(catgets)
+TRAP(dbm_clearerr)
+TRAP(dbm_close)
+TRAP(dbm_delete)
+TRAP(dbm_error)
+TRAP(dbm_fetch)
+TRAP(dbm_firstkey)
+TRAP(dbm_nextkey)
+TRAP(dbm_open)
+TRAP(dbm_store)
+TRAP(dirname)
+#if !defined(SANITIZER)
+TRAP(dlerror) // Used by tsan
+#endif
+TRAP(ftw)
+TRAP(getc_unlocked)
+//TRAP(getenv) // Ok at program startup
+TRAP(inet_ntoa)
+TRAP(lgamma)
+TRAP(lgammaf)
+TRAP(lgammal)
+TRAP(nftw)
+TRAP(nl_langinfo)
+TRAP(putc_unlocked)
+TRAP(rand)
+/** In  the current POSIX.1 specification (POSIX.1-2008), readdir() is not required to be thread-safe.  However, in modern
+  * implementations (including the glibc implementation), concurrent calls to readdir() that specify different directory streams
+  * are thread-safe.  In cases where multiple threads must read from the same directory stream, using readdir() with external
+  * synchronization is still preferable to the use of the deprecated readdir_r(3)  function. It is expected that a future
+  * version of POSIX.1 will require that readdir() be thread-safe when concurrently employed on different directory streams.
+  * - man readdir
+  */
+//TRAP(readdir)
+TRAP(system)
+TRAP(wcstombs)
+TRAP(ether_aton)
+TRAP(ether_ntoa)
+TRAP(fgetsgent)
+TRAP(fgetspent)
+TRAP(getaliasbyname)
+TRAP(getaliasent)
+TRAP(getrpcbyname)
+TRAP(getrpcbynumber)
+TRAP(getrpcent)
+TRAP(getsgent)
+TRAP(getsgnam)
+TRAP(getspent)
+TRAP(getspnam)
+TRAP(initstate)
+TRAP(random)
+TRAP(setstate)
+TRAP(sgetsgent)
+TRAP(sgetspent)
+TRAP(srandom)
+TRAP(twalk)
+TRAP(lgammaf128)
+TRAP(lgammaf32)
+TRAP(lgammaf32x)
+TRAP(lgammaf64)
+TRAP(lgammaf64x)
+
+#endif

base/mysqlxx/CMakeLists.txt

@@ -3,7 +3,6 @@ add_library (mysqlxx
     Exception.cpp
     Query.cpp
     ResultBase.cpp
-    StoreQueryResult.cpp
     UseQueryResult.cpp
     Row.cpp
     Value.cpp

base/mysqlxx/Connection.cpp

@@ -51,10 +51,11 @@ Connection::Connection(
     const char* ssl_key,
     unsigned timeout,
     unsigned rw_timeout,
-    bool enable_local_infile)
+    bool enable_local_infile,
+    bool opt_reconnect)
     : Connection()
 {
-    connect(db, server, user, password, port, socket, ssl_ca, ssl_cert, ssl_key, timeout, rw_timeout, enable_local_infile);
+    connect(db, server, user, password, port, socket, ssl_ca, ssl_cert, ssl_key, timeout, rw_timeout, enable_local_infile, opt_reconnect);
 }
 
 Connection::Connection(const std::string & config_name)
@@ -80,7 +81,8 @@ void Connection::connect(const char* db,
     const char * ssl_key,
     unsigned timeout,
     unsigned rw_timeout,
-    bool enable_local_infile)
+    bool enable_local_infile,
+    bool opt_reconnect)
 {
     if (is_connected)
         disconnect();
@@ -104,9 +106,8 @@ void Connection::connect(const char* db,
     if (mysql_options(driver.get(), MYSQL_OPT_LOCAL_INFILE, &enable_local_infile_arg))
         throw ConnectionFailed(errorMessage(driver.get()), mysql_errno(driver.get()));
 
-    /// Enables auto-reconnect.
-    bool reconnect = true;
-    if (mysql_options(driver.get(), MYSQL_OPT_RECONNECT, reinterpret_cast<const char *>(&reconnect)))
+    /// See C API Developer Guide: Automatic Reconnection Control
+    if (mysql_options(driver.get(), MYSQL_OPT_RECONNECT, reinterpret_cast<const char *>(&opt_reconnect)))
         throw ConnectionFailed(errorMessage(driver.get()), mysql_errno(driver.get()));
 
     /// Specifies particular ssl key and certificate if it needs
@@ -116,8 +117,8 @@ void Connection::connect(const char* db,
     if (!mysql_real_connect(driver.get(), server, user, password, db, port, ifNotEmpty(socket), driver->client_flag))
         throw ConnectionFailed(errorMessage(driver.get()), mysql_errno(driver.get()));
 
-    /// Sets UTF-8 as default encoding.
-    if (mysql_set_character_set(driver.get(), "UTF8"))
+    /// Sets UTF-8 as default encoding. See https://mariadb.com/kb/en/mysql_set_character_set/
+    if (mysql_set_character_set(driver.get(), "utf8mb4"))
         throw ConnectionFailed(errorMessage(driver.get()), mysql_errno(driver.get()));
 
     is_connected = true;

base/mysqlxx/Connection.h

@@ -14,6 +14,8 @@
 
 /// Disable LOAD DATA LOCAL INFILE because it is insecure
 #define MYSQLXX_DEFAULT_ENABLE_LOCAL_INFILE false
+/// See https://dev.mysql.com/doc/c-api/5.7/en/c-api-auto-reconnect.html
+#define MYSQLXX_DEFAULT_MYSQL_OPT_RECONNECT true
 
 
 namespace mysqlxx
@@ -39,7 +41,6 @@ private:
 /** MySQL connection.
   * Usage:
   *        mysqlxx::Connection connection("Test", "127.0.0.1", "root", "qwerty", 3306);
-  *        std::cout << connection.query("SELECT 'Hello, World!'").store().at(0).at(0).getString() << std::endl;
   *
   * Or with Poco library configuration:
   *        mysqlxx::Connection connection("mysql_params");
@@ -77,7 +78,8 @@ public:
         const char * ssl_key = "",
         unsigned timeout = MYSQLXX_DEFAULT_TIMEOUT,
         unsigned rw_timeout = MYSQLXX_DEFAULT_RW_TIMEOUT,
-        bool enable_local_infile = MYSQLXX_DEFAULT_ENABLE_LOCAL_INFILE);
+        bool enable_local_infile = MYSQLXX_DEFAULT_ENABLE_LOCAL_INFILE,
+        bool opt_reconnect = MYSQLXX_DEFAULT_MYSQL_OPT_RECONNECT);
 
     /// Creates connection. Can be used if Poco::Util::Application is using.
     /// All settings will be got from config_name section of configuration.
@@ -97,7 +99,8 @@ public:
         const char* ssl_key,
         unsigned timeout = MYSQLXX_DEFAULT_TIMEOUT,
         unsigned rw_timeout = MYSQLXX_DEFAULT_RW_TIMEOUT,
-        bool enable_local_infile = MYSQLXX_DEFAULT_ENABLE_LOCAL_INFILE);
+        bool enable_local_infile = MYSQLXX_DEFAULT_ENABLE_LOCAL_INFILE,
+        bool opt_reconnect = MYSQLXX_DEFAULT_MYSQL_OPT_RECONNECT);
 
     void connect(const std::string & config_name)
     {
@@ -113,6 +116,7 @@ public:
         std::string ssl_cert = cfg.getString(config_name + ".ssl_cert", "");
         std::string ssl_key = cfg.getString(config_name + ".ssl_key", "");
         bool enable_local_infile = cfg.getBool(config_name + ".enable_local_infile", MYSQLXX_DEFAULT_ENABLE_LOCAL_INFILE);
+        bool opt_reconnect = cfg.getBool(config_name + ".opt_reconnect", MYSQLXX_DEFAULT_MYSQL_OPT_RECONNECT);
 
         unsigned timeout =
             cfg.getInt(config_name + ".connect_timeout",
@@ -136,7 +140,8 @@ public:
                 ssl_key.c_str(),
                 timeout,
                 rw_timeout,
-                enable_local_infile);
+                enable_local_infile,
+                opt_reconnect);
     }
 
     /// If MySQL connection was established.

base/mysqlxx/Exception.h

@@ -26,6 +26,15 @@ struct ConnectionFailed : public Exception
 };
 
 
+/// Connection to MySQL server was lost
+struct ConnectionLost : public Exception
+{
+    ConnectionLost(const std::string & msg, int code = 0) : Exception(msg, code) {}
+    const char * name() const throw() override { return "mysqlxx::ConnectionLost"; }
+    const char * className() const throw() override { return "mysqlxx::ConnectionLost"; }
+};
+
+
 /// Erroneous query.
 struct BadQuery : public Exception
 {

base/mysqlxx/Pool.cpp

@@ -10,7 +10,6 @@
 
 #include <common/sleep.h>
 
-#include <Poco/Util/Application.h>
 #include <Poco/Util/LayeredConfiguration.h>
 
 
@@ -41,7 +40,9 @@ void Pool::Entry::decrementRefCount()
 Pool::Pool(const Poco::Util::AbstractConfiguration & cfg, const std::string & config_name,
      unsigned default_connections_, unsigned max_connections_,
      const char * parent_config_name_)
-    : default_connections(default_connections_), max_connections(max_connections_)
+    : logger(Poco::Logger::get("mysqlxx::Pool"))
+    , default_connections(default_connections_)
+    , max_connections(max_connections_)
 {
     server = cfg.getString(config_name + ".host");
 
@@ -78,6 +79,9 @@ Pool::Pool(const Poco::Util::AbstractConfiguration & cfg, const std::string & co
 
         enable_local_infile = cfg.getBool(config_name + ".enable_local_infile",
             cfg.getBool(parent_config_name + ".enable_local_infile", MYSQLXX_DEFAULT_ENABLE_LOCAL_INFILE));
+
+        opt_reconnect = cfg.getBool(config_name + ".opt_reconnect",
+            cfg.getBool(parent_config_name + ".opt_reconnect", MYSQLXX_DEFAULT_MYSQL_OPT_RECONNECT));
     }
     else
     {
@@ -96,6 +100,8 @@ Pool::Pool(const Poco::Util::AbstractConfiguration & cfg, const std::string & co
 
         enable_local_infile = cfg.getBool(
             config_name + ".enable_local_infile", MYSQLXX_DEFAULT_ENABLE_LOCAL_INFILE);
+
+        opt_reconnect = cfg.getBool(config_name + ".opt_reconnect", MYSQLXX_DEFAULT_MYSQL_OPT_RECONNECT);
     }
 
     connect_timeout = cfg.getInt(config_name + ".connect_timeout",
@@ -125,20 +131,30 @@ Pool::Entry Pool::get()
     initialize();
     for (;;)
     {
+        logger.trace("(%s): Iterating through existing MySQL connections", getDescription());
+
         for (auto & connection : connections)
         {
             if (connection->ref_count == 0)
                 return Entry(connection, this);
         }
 
+        logger.trace("(%s): Trying to allocate a new connection.", getDescription());
         if (connections.size() < static_cast<size_t>(max_connections))
         {
             Connection * conn = allocConnection();
             if (conn)
                 return Entry(conn, this);
+
+            logger.trace("(%s): Unable to create a new connection: Allocation failed.", getDescription());
+        }
+        else
+        {
+            logger.trace("(%s): Unable to create a new connection: Max number of connections has been reached.", getDescription());
         }
 
         lock.unlock();
+        logger.trace("(%s): Sleeping for %d seconds.", getDescription(), MYSQLXX_POOL_SLEEP_ON_CONNECT_FAIL);
         sleepForSeconds(MYSQLXX_POOL_SLEEP_ON_CONNECT_FAIL);
         lock.lock();
     }
@@ -158,12 +174,13 @@ Pool::Entry Pool::tryGet()
         /// Fixme: There is a race condition here b/c we do not synchronize with Pool::Entry's copy-assignment operator
         if (connection_ptr->ref_count == 0)
         {
-            Entry res(connection_ptr, this);
-            if (res.tryForceConnected())  /// Tries to reestablish connection as well
-                return res;
+            {
+                Entry res(connection_ptr, this);
+                if (res.tryForceConnected())  /// Tries to reestablish connection as well
+                    return res;
+            }
 
-            auto & logger = Poco::Util::Application::instance().logger();
-            logger.information("Idle connection to mysql server cannot be recovered, dropping it.");
+            logger.debug("(%s): Idle connection to MySQL server cannot be recovered, dropping it.", getDescription());
 
             /// This one is disconnected, cannot be reestablished and so needs to be disposed of.
             connection_it = connections.erase(connection_it);
@@ -186,6 +203,8 @@ Pool::Entry Pool::tryGet()
 
 void Pool::removeConnection(Connection* connection)
 {
+    logger.trace("(%s): Removing connection.", getDescription());
+
     std::lock_guard<std::mutex> lock(mutex);
     if (connection)
     {
@@ -210,8 +229,6 @@ void Pool::Entry::forceConnected() const
     if (data == nullptr)
         throw Poco::RuntimeException("Tried to access NULL database connection.");
 
-    Poco::Util::Application & app = Poco::Util::Application::instance();
-
     bool first = true;
     while (!tryForceConnected())
     {
@@ -220,7 +237,7 @@ void Pool::Entry::forceConnected() const
         else
             sleepForSeconds(MYSQLXX_POOL_SLEEP_ON_CONNECT_FAIL);
 
-        app.logger().information("MYSQL: Reconnecting to " + pool->description);
+        pool->logger.debug("Entry: Reconnecting to MySQL server %s", pool->description);
         data->conn.connect(
             pool->db.c_str(),
             pool->server.c_str(),
@@ -233,7 +250,8 @@ void Pool::Entry::forceConnected() const
             pool->ssl_key.c_str(),
             pool->connect_timeout,
             pool->rw_timeout,
-            pool->enable_local_infile);
+            pool->enable_local_infile,
+            pool->opt_reconnect);
     }
 }
 
@@ -242,18 +260,22 @@ bool Pool::Entry::tryForceConnected() const
 {
     auto * const mysql_driver = data->conn.getDriver();
     const auto prev_connection_id = mysql_thread_id(mysql_driver);
+
+    pool->logger.trace("Entry(connection %lu): sending PING to check if it is alive.", prev_connection_id);
     if (data->conn.ping())  /// Attempts to reestablish lost connection
     {
         const auto current_connection_id = mysql_thread_id(mysql_driver);
         if (prev_connection_id != current_connection_id)
         {
-            auto & logger = Poco::Util::Application::instance().logger();
-            logger.information("Connection to mysql server has been reestablished. Connection id changed: %lu -> %lu",
-                                prev_connection_id, current_connection_id);
+            pool->logger.debug("Entry(connection %lu): Reconnected to MySQL server. Connection id changed: %lu -> %lu",
+                                current_connection_id, prev_connection_id, current_connection_id);
         }
+
+        pool->logger.trace("Entry(connection %lu): PING ok.", current_connection_id);
         return true;
     }
 
+    pool->logger.trace("Entry(connection %lu): PING failed.", prev_connection_id);
     return false;
 }
 
@@ -274,15 +296,13 @@ void Pool::initialize()
 
 Pool::Connection * Pool::allocConnection(bool dont_throw_if_failed_first_time)
 {
-    Poco::Util::Application & app = Poco::Util::Application::instance();
-
-    std::unique_ptr<Connection> conn(new Connection);
+    std::unique_ptr<Connection> conn_ptr{new Connection};
 
     try
     {
-        app.logger().information("MYSQL: Connecting to " + description);
+        logger.debug("Connecting to %s", description);
 
-        conn->conn.connect(
+        conn_ptr->conn.connect(
             db.c_str(),
             server.c_str(),
             user.c_str(),
@@ -294,29 +314,29 @@ Pool::Connection * Pool::allocConnection(bool dont_throw_if_failed_first_time)
             ssl_key.c_str(),
             connect_timeout,
             rw_timeout,
-            enable_local_infile);
+            enable_local_infile,
+            opt_reconnect);
     }
     catch (mysqlxx::ConnectionFailed & e)
     {
+        logger.error(e.what());
+
         if ((!was_successful && !dont_throw_if_failed_first_time)
             || e.errnum() == ER_ACCESS_DENIED_ERROR
             || e.errnum() == ER_DBACCESS_DENIED_ERROR
             || e.errnum() == ER_BAD_DB_ERROR)
         {
-            app.logger().error(e.what());
             throw;
         }
         else
         {
-            app.logger().error(e.what());
             return nullptr;
         }
     }
 
+    connections.push_back(conn_ptr.get());
     was_successful = true;
-    auto * connection = conn.release();
-    connections.push_back(connection);
-    return connection;
+    return conn_ptr.release();
 }
 
 }

base/mysqlxx/Pool.h

@@ -6,6 +6,8 @@
 #include <atomic>
 
 #include <Poco/Exception.h>
+#include <Poco/Logger.h>
+
 #include <mysqlxx/Connection.h>
 
 
@@ -165,19 +167,21 @@ public:
          unsigned rw_timeout_ = MYSQLXX_DEFAULT_RW_TIMEOUT,
          unsigned default_connections_ = MYSQLXX_POOL_DEFAULT_START_CONNECTIONS,
          unsigned max_connections_ = MYSQLXX_POOL_DEFAULT_MAX_CONNECTIONS,
-         unsigned enable_local_infile_ = MYSQLXX_DEFAULT_ENABLE_LOCAL_INFILE)
-    : default_connections(default_connections_), max_connections(max_connections_),
-    db(db_), server(server_), user(user_), password(password_), port(port_), socket(socket_),
-    connect_timeout(connect_timeout_), rw_timeout(rw_timeout_), enable_local_infile(enable_local_infile_) {}
+         unsigned enable_local_infile_ = MYSQLXX_DEFAULT_ENABLE_LOCAL_INFILE,
+         bool opt_reconnect_ = MYSQLXX_DEFAULT_MYSQL_OPT_RECONNECT)
+    : logger(Poco::Logger::get("mysqlxx::Pool")), default_connections(default_connections_),
+    max_connections(max_connections_), db(db_), server(server_), user(user_), password(password_), port(port_), socket(socket_),
+    connect_timeout(connect_timeout_), rw_timeout(rw_timeout_), enable_local_infile(enable_local_infile_),
+    opt_reconnect(opt_reconnect_) {}
 
     Pool(const Pool & other)
-        : default_connections{other.default_connections},
+        : logger(other.logger), default_connections{other.default_connections},
           max_connections{other.max_connections},
           db{other.db}, server{other.server},
           user{other.user}, password{other.password},
           port{other.port}, socket{other.socket},
           connect_timeout{other.connect_timeout}, rw_timeout{other.rw_timeout},
-          enable_local_infile{other.enable_local_infile}
+          enable_local_infile{other.enable_local_infile}, opt_reconnect(other.opt_reconnect)
     {}
 
     Pool & operator=(const Pool &) = delete;
@@ -201,6 +205,8 @@ public:
     void removeConnection(Connection * connection);
 
 protected:
+    Poco::Logger & logger;
+
     /// Number of MySQL connections which are created at launch.
     unsigned default_connections;
     /// Maximum possible number of connections
@@ -231,6 +237,7 @@ private:
     std::string ssl_cert;
     std::string ssl_key;
     bool enable_local_infile;
+    bool opt_reconnect;
 
     /// True if connection was established at least once.
     bool was_successful{false};

base/mysqlxx/PoolWithFailover.cpp

@@ -1,3 +1,7 @@
+#include <algorithm>
+#include <ctime>
+#include <random>
+#include <thread>
 #include <mysqlxx/PoolWithFailover.h>
 
 
@@ -10,9 +14,12 @@ static bool startsWith(const std::string & s, const char * prefix)
 
 using namespace mysqlxx;
 
-PoolWithFailover::PoolWithFailover(const Poco::Util::AbstractConfiguration & config_,
-                                   const std::string & config_name_, const unsigned default_connections_,
-                                   const unsigned max_connections_, const size_t max_tries_)
+PoolWithFailover::PoolWithFailover(
+        const Poco::Util::AbstractConfiguration & config_,
+        const std::string & config_name_,
+        const unsigned default_connections_,
+        const unsigned max_connections_,
+        const size_t max_tries_)
     : max_tries(max_tries_)
 {
     shareable = config_.getBool(config_name_ + ".share_connection", false);
@@ -33,6 +40,19 @@ PoolWithFailover::PoolWithFailover(const Poco::Util::AbstractConfiguration & con
                     std::make_shared<Pool>(config_, replica_name, default_connections_, max_connections_, config_name_.c_str()));
             }
         }
+
+        /// PoolWithFailover objects are stored in a cache inside PoolFactory.
+        /// This cache is reset by ExternalDictionariesLoader after every SYSTEM RELOAD DICTIONAR{Y|IES}
+        /// which triggers massive re-constructing of connection pools.
+        /// The state of PRNGs like std::mt19937 is considered to be quite heavy
+        /// thus here we attempt to optimize its construction.
+        static thread_local std::mt19937 rnd_generator(
+                std::hash<std::thread::id>{}(std::this_thread::get_id()) + std::clock());
+        for (auto & [_, replicas] : replicas_by_priority)
+        {
+            if (replicas.size() > 1)
+                std::shuffle(replicas.begin(), replicas.end(), rnd_generator);
+        }
     }
     else
     {
@@ -41,16 +61,38 @@ PoolWithFailover::PoolWithFailover(const Poco::Util::AbstractConfiguration & con
     }
 }
 
-PoolWithFailover::PoolWithFailover(const std::string & config_name_, const unsigned default_connections_,
-    const unsigned max_connections_, const size_t max_tries_)
-    : PoolWithFailover{
-        Poco::Util::Application::instance().config(), config_name_,
-        default_connections_, max_connections_, max_tries_}
+
+PoolWithFailover::PoolWithFailover(
+        const std::string & config_name_,
+        const unsigned default_connections_,
+        const unsigned max_connections_,
+        const size_t max_tries_)
+    : PoolWithFailover{Poco::Util::Application::instance().config(),
+            config_name_, default_connections_, max_connections_, max_tries_}
 {
 }
 
+
+PoolWithFailover::PoolWithFailover(
+        const std::string & database,
+        const RemoteDescription & addresses,
+        const std::string & user,
+        const std::string & password,
+        size_t max_tries_)
+    : max_tries(max_tries_)
+    , shareable(false)
+{
+    /// Replicas have the same priority, but traversed replicas are moved to the end of the queue.
+    for (const auto & [host, port] : addresses)
+    {
+        replicas_by_priority[0].emplace_back(std::make_shared<Pool>(database, host, user, password, port));
+    }
+}
+
+
 PoolWithFailover::PoolWithFailover(const PoolWithFailover & other)
-    : max_tries{other.max_tries}, shareable{other.shareable}
+    : max_tries{other.max_tries}
+    , shareable{other.shareable}
 {
     if (shareable)
     {

base/mysqlxx/PoolWithFailover.h

@@ -11,6 +11,8 @@
 namespace mysqlxx
 {
     /** MySQL connection pool with support of failover.
+      *
+      * For dictionary source:
       * Have information about replicas and their priorities.
       * Tries to connect to replica in an order of priority. When equal priority, choose replica with maximum time without connections.
       *
@@ -68,42 +70,58 @@ namespace mysqlxx
         using PoolPtr = std::shared_ptr<Pool>;
         using Replicas = std::vector<PoolPtr>;
 
-        /// [priority][index] -> replica.
+        /// [priority][index] -> replica. Highest priority is 0.
         using ReplicasByPriority = std::map<int, Replicas>;
-
         ReplicasByPriority replicas_by_priority;
 
         /// Number of connection tries.
         size_t max_tries;
         /// Mutex for set of replicas.
         std::mutex mutex;
-
         /// Can the Pool be shared
         bool shareable;
 
     public:
         using Entry = Pool::Entry;
+        using RemoteDescription = std::vector<std::pair<std::string, uint16_t>>;
 
         /**
-         * config_name           Name of parameter in configuration file.
+         * * Mysql dictionary sourse related params:
+         * config_name           Name of parameter in configuration file for dictionary source.
+         *
+         * * Mysql storage related parameters:
+         * replicas_description
+         *
+         * * Mutual parameters:
          * default_connections   Number of connection in pool to each replica at start.
          * max_connections       Maximum number of connections in pool to each replica.
          * max_tries_            Max number of connection tries.
          */
-        PoolWithFailover(const std::string & config_name_,
+        PoolWithFailover(
+            const std::string & config_name_,
             unsigned default_connections_ = MYSQLXX_POOL_WITH_FAILOVER_DEFAULT_START_CONNECTIONS,
             unsigned max_connections_ = MYSQLXX_POOL_WITH_FAILOVER_DEFAULT_MAX_CONNECTIONS,
             size_t max_tries_ = MYSQLXX_POOL_WITH_FAILOVER_DEFAULT_MAX_TRIES);
 
-        PoolWithFailover(const Poco::Util::AbstractConfiguration & config_,
+        PoolWithFailover(
+            const Poco::Util::AbstractConfiguration & config_,
             const std::string & config_name_,
             unsigned default_connections_ = MYSQLXX_POOL_WITH_FAILOVER_DEFAULT_START_CONNECTIONS,
             unsigned max_connections_ = MYSQLXX_POOL_WITH_FAILOVER_DEFAULT_MAX_CONNECTIONS,
             size_t max_tries_ = MYSQLXX_POOL_WITH_FAILOVER_DEFAULT_MAX_TRIES);
 
+        PoolWithFailover(
+            const std::string & database,
+            const RemoteDescription & addresses,
+            const std::string & user,
+            const std::string & password,
+            size_t max_tries_ = MYSQLXX_POOL_WITH_FAILOVER_DEFAULT_MAX_TRIES);
+
         PoolWithFailover(const PoolWithFailover & other);
 
         /** Allocates a connection to use. */
         Entry get();
     };
+
+    using PoolWithFailoverPtr = std::shared_ptr<PoolWithFailover>;
 }

base/mysqlxx/Query.cpp

@@ -1,11 +1,16 @@
 #if __has_include(<mysql.h>)
+#include <errmsg.h>
 #include <mysql.h>
 #else
+#include <mysql/errmsg.h>
 #include <mysql/mysql.h>
 #endif
 
+#include <Poco/Logger.h>
+
 #include <mysqlxx/Connection.h>
 #include <mysqlxx/Query.h>
+#include <mysqlxx/Types.h>
 
 
 namespace mysqlxx
@@ -57,8 +62,24 @@ void Query::reset()
 void Query::executeImpl()
 {
     std::string query_string = query_buf.str();
-    if (mysql_real_query(conn->getDriver(), query_string.data(), query_string.size()))
-        throw BadQuery(errorMessage(conn->getDriver()), mysql_errno(conn->getDriver()));
+
+    MYSQL* mysql_driver = conn->getDriver();
+
+    auto & logger = Poco::Logger::get("mysqlxx::Query");
+    logger.trace("Running MySQL query using connection %lu", mysql_thread_id(mysql_driver));
+    if (mysql_real_query(mysql_driver, query_string.data(), query_string.size()))
+    {
+        const auto err_no = mysql_errno(mysql_driver);
+        switch (err_no)
+        {
+        case CR_SERVER_GONE_ERROR:
+            [[fallthrough]];
+        case CR_SERVER_LOST:
+            throw ConnectionLost(errorMessage(mysql_driver), err_no);
+        default:
+            throw BadQuery(errorMessage(mysql_driver), err_no);
+        }
+    }
 }
 
 UseQueryResult Query::use()
@@ -71,16 +92,6 @@ UseQueryResult Query::use()
     return UseQueryResult(res, conn, this);
 }
 
-StoreQueryResult Query::store()
-{
-    executeImpl();
-    MYSQL_RES * res = mysql_store_result(conn->getDriver());
-    if (!res)
-        checkError(conn->getDriver());
-
-    return StoreQueryResult(res, conn, this);
-}
-
 void Query::execute()
 {
     executeImpl();

base/mysqlxx/Query.h

@@ -3,7 +3,6 @@
 #include <sstream>
 
 #include <mysqlxx/UseQueryResult.h>
-#include <mysqlxx/StoreQueryResult.h>
 
 
 namespace mysqlxx
@@ -46,11 +45,6 @@ public:
       */
     UseQueryResult use();
 
-    /** Выполнить запрос с загрузкой на клиента всех строк.
-      * Требуется оперативка, чтобы вместить весь результат, зато к строкам можно обращаться в произвольном порядке.
-      */
-    StoreQueryResult store();
-
     /// Значение auto increment после последнего INSERT-а.
     UInt64 insertID();
 

base/mysqlxx/ResultBase.h

@@ -9,7 +9,7 @@ class Connection;
 class Query;
 
 
-/** Базовый класс для UseQueryResult и StoreQueryResult.
+/** Базовый класс для UseQueryResult.
   * Содержит общую часть реализации,
   * Ссылается на Connection. Если уничтожить Connection, то пользоваться ResultBase и любым результатом нельзя.
   * Использовать объект можно только для результата одного запроса!

base/mysqlxx/Row.h

@@ -35,7 +35,7 @@ public:
     {
     }
 
-    /** Для того, чтобы создать Row, используйте соответствующие методы UseQueryResult или StoreQueryResult. */
+    /** Для того, чтобы создать Row, используйте соответствующие методы UseQueryResult. */
     Row(MYSQL_ROW row_, ResultBase * res_, MYSQL_LENGTHS lengths_)
         : row(row_), res(res_), lengths(lengths_)
     {

base/mysqlxx/StoreQueryResult.cpp

@@ -1,30 +0,0 @@
-#if __has_include(<mysql.h>)
-#include <mysql.h>
-#else
-#include <mysql/mysql.h>
-#endif
-
-#include <mysqlxx/Connection.h>
-#include <mysqlxx/StoreQueryResult.h>
-
-
-namespace mysqlxx
-{
-
-StoreQueryResult::StoreQueryResult(MYSQL_RES * res_, Connection * conn_, const Query * query_) : ResultBase(res_, conn_, query_)
-{
-    UInt64 rows = mysql_num_rows(res);
-    reserve(rows);
-    lengths.resize(rows * num_fields);
-
-    for (UInt64 i = 0; MYSQL_ROW row = mysql_fetch_row(res); ++i)
-    {
-        MYSQL_LENGTHS lengths_for_row = mysql_fetch_lengths(res);
-        memcpy(&lengths[i * num_fields], lengths_for_row, sizeof(lengths[0]) * num_fields);
-
-        push_back(Row(row, this, &lengths[i * num_fields]));
-    }
-    checkError(conn->getDriver());
-}
-
-}

base/mysqlxx/StoreQueryResult.h

@@ -1,45 +0,0 @@
-#pragma once
-
-#include <vector>
-
-#include <mysqlxx/ResultBase.h>
-#include <mysqlxx/Row.h>
-
-
-namespace mysqlxx
-{
-
-class Connection;
-
-
-/** Результат выполнения запроса, загруженный полностью на клиента.
-  * Это требует оперативку, чтобы вместить весь результат,
-  *  но зато реализует произвольный доступ к строкам по индексу.
-  * Если размер результата большой - используйте лучше UseQueryResult.
-  * Объект содержит ссылку на Connection.
-  * Если уничтожить Connection, то объект становится некорректным и все строки результата - тоже.
-  * Если задать следующий запрос в соединении, то объект и все строки тоже становятся некорректными.
-  * Использовать объект можно только для результата одного запроса!
-  * (При попытке присвоить объекту результат следующего запроса - UB.)
-  */
-class StoreQueryResult : public std::vector<Row>, public ResultBase
-{
-public:
-    StoreQueryResult(MYSQL_RES * res_, Connection * conn_, const Query * query_);
-
-    size_t num_rows() const { return size(); }
-
-private:
-
-    /** Не смотря на то, что весь результат выполнения запроса загружается на клиента,
-      *  и все указатели MYSQL_ROW на отдельные строки различные,
-      *  при этом функция mysql_fetch_lengths() возвращает длины
-      *  для текущей строки по одному и тому же адресу.
-      * То есть, чтобы можно было пользоваться несколькими Row одновременно,
-      *  необходимо заранее куда-то сложить все длины.
-      */
-    using Lengths = std::vector<MYSQL_LENGTH>;
-    Lengths lengths;
-};
-
-}

base/mysqlxx/UseQueryResult.h

@@ -12,8 +12,7 @@ class Connection;
 
 /** Результат выполнения запроса, предназначенный для чтения строк, одна за другой.
   * В памяти при этом хранится только одна, текущая строка.
-  * В отличие от StoreQueryResult, произвольный доступ к строкам невозможен,
-  *  а также, при чтении следующей строки, предыдущая становится некорректной.
+  * При чтении следующей строки, предыдущая становится некорректной.
   * Вы обязаны прочитать все строки из результата
   *  (вызывать функцию fetch(), пока она не вернёт значение, преобразующееся к false),
   *  иначе при следующем запросе будет выкинуто исключение с текстом "Commands out of sync".

base/mysqlxx/Value.h

@@ -25,7 +25,7 @@ class ResultBase;
 
 /** Represents a single value read from MySQL.
   * It doesn't owns the value. It's just a wrapper of a pair (const char *, size_t).
-  * If the UseQueryResult/StoreQueryResult or Connection is destroyed,
+  * If the UseQueryResult or Connection is destroyed,
   *  or you have read the next Row while using UseQueryResult, then the object is invalidated.
   * Allows to transform (parse) the value to various data types:
   * - with getUInt(), getString(), ... (recommended);

base/mysqlxx/tests/CMakeLists.txt

@@ -1,5 +1,2 @@
-add_executable (mysqlxx_test mysqlxx_test.cpp)
-target_link_libraries (mysqlxx_test PRIVATE mysqlxx)
-
 add_executable (mysqlxx_pool_test mysqlxx_pool_test.cpp)
 target_link_libraries (mysqlxx_pool_test PRIVATE mysqlxx)

base/mysqlxx/tests/failover.xml

@@ -1,21 +0,0 @@
-<?xml version = '1.0' encoding = 'utf-8'?>
-<yandex>
-    <mysql_goals>
-        <port>3306</port>
-        <user>root</user>
-        <db>Metrica</db>
-        <password>qwerty</password>
-        <replica>
-            <host>example02t</host>
-            <priority>0</priority>
-        </replica>
-        <replica>
-            <host>example02t</host>
-            <port>3306</port>
-            <user>root</user>
-            <password>qwerty</password>
-            <db>Metrica</db>
-            <priority>1</priority>
-        </replica>
-    </mysql_goals>
-</yandex>