Commit Graph

1444 Commits

Author SHA1 Message Date
Mikhail f. Shiryaev
dd2a0ed755
Improve speed significantly by using cached users and reuse repos 2022-07-21 15:46:39 +02:00
Alexey Milovidov
ad77b84b82 Update changelog 2022-07-21 13:32:34 +02:00
Mikhail f. Shiryaev
aef212d552
Add ref commits to the chengelog headers 2022-07-21 09:00:31 +02:00
Yakov Olkhovskiy
bdd11f671d different path of endian.h for FreeBSD 2022-07-21 02:08:07 -04:00
Yakov Olkhovskiy
e7438cc433
Merge branch 'master' into cmake-self-extracting-executable 2022-07-21 00:25:08 -04:00
Yakov Olkhovskiy
29ed2083d1 endian.h in different path on __APPLE__ and __FreeBSD__ 2022-07-21 00:20:23 -04:00
Yakov Olkhovskiy
b766552481 add native build for cross-compilation 2022-07-20 23:09:05 -04:00
Mikhail f. Shiryaev
59501150d0
Use github_helper in changelog, use a robot token in CI 2022-07-20 17:28:46 +02:00
Robert Schulze
81ef1099cc
Even less usage of StringRef
--> see #39300
2022-07-19 07:01:06 +00:00
Jordi Villar
c63512d255 Use CH custom platform macro for OS Darwin 2022-07-18 09:59:02 +02:00
Jordi Villar
d86abba2c6 Fix macosx compilation due to endian.h 2022-07-17 14:58:27 +02:00
Yakov Olkhovskiy
e5f165d909
Merge branch 'master' into cmake-self-extracting-executable 2022-07-13 16:09:18 -04:00
Yakov Olkhovskiy
2079699f75
Merge pull request #39065 from ClickHouse/utils-self-extracting-decompressor
Parameter --decompressor added to utils/self-extracting-executable/compressor
2022-07-13 01:50:12 -04:00
Alexey Milovidov
e2684447b4
Merge pull request #38437 from ClickHouse/compatible-benchmark
Compatible and reproducible DBMS benchmark
2022-07-12 21:15:53 +03:00
Yakov Olkhovskiy
7c437c3f9b some optimizations 2022-07-12 09:56:09 -04:00
Alexey Milovidov
87ebf51cdc Fix style 2022-07-11 22:26:50 +02:00
Robert Schulze
c2e4fde0e2
Link only necessessary stuff for self-extracting-executable 2022-07-11 19:48:49 +02:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_"
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
2022-07-11 19:36:18 +02:00
Yakov Olkhovskiy
3432768dc3 Parameter --decompressor added, architecture-agnostic metadata, bugs fixed, some improvements 2022-07-09 16:05:12 -04:00
Azat Khuzhin
53d019a94b check-style: improve ZooKeeper path check for Replicated.*MergeTree tables
- detech all Replicated.*MergeTree tables
- allow default_path_test/uuid in zookeeper path
- improve pattern for filtering tests

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-08 13:19:24 +03:00
robot-clickhouse
7cef35da70 Update version_date.tsv after v22.4.6.53-stable 2022-07-07 20:05:14 +00:00
Yakov Olkhovskiy
c6db15458a build utils/self-extracting-executable/compressor whenever we want to build compressed binary 2022-07-06 20:40:41 -04:00
Robert Schulze
ee19c218f3
Teach ClickHouse's spellcheck "ClickHouse's" 2022-07-05 14:24:18 +02:00
alesapin
bd8a3ee841
Merge pull request #38338 from ClickHouse/zookeeper-add-extra-list-argument
Extend ZooKeeper list request with support for filtering persistent or ephemeral nodes only
2022-07-05 12:53:51 +02:00
Kruglov Pavel
4080f055b6
Merge pull request #38477 from Avogar/sql-insert-format
Add SQLInsert output format
2022-07-04 15:06:33 +02:00
Mikhail f. Shiryaev
7a58a8c19b
Merge pull request #38647 from ClickHouse/clickhouse-diagnostics
Clickhouse diagnostics
2022-07-04 12:14:33 +02:00
Mikhail f. Shiryaev
446ead1af4
Fix typos issues it clickhouse-diagnostics 2022-07-04 10:59:39 +02:00
Azat Khuzhin
e4b81e6de9 Add exp_internal for expect tests
This is to catch possible issues like 01565_reconnect_after_client_error in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/38417/e33f236d502fe9f7906392f4ceffaee44e3c4ce4/stateless_tests__thread__actions__[3/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-02 17:08:06 +03:00
robot-clickhouse
656d0c0f04 Update version_date.tsv and changelogs after v22.6.2.12-stable 2022-06-29 11:24:30 +00:00
mergify[bot]
9482c99ab8
Merge branch 'master' into sql-insert-format 2022-06-29 11:03:07 +00:00
Antonio Andelic
7a8fbbf132 Merge branch 'master' into zookeeper-add-extra-list-argument 2022-06-29 08:31:03 +00:00
Robert Schulze
c22038d48b
More clang-tidy fixes 2022-06-28 11:50:05 +00:00
avogar
3b4ecc93bc Fix test and style 2022-06-28 10:48:20 +00:00
Mikhail f. Shiryaev
1b945307ea
Merge pull request #38238 from ClickHouse/changelog-attrs
Changelog attrs
2022-06-27 14:30:19 +02:00
Antonio Andelic
c06776ce0d Merge branch 'master' into zookeeper-add-extra-list-argument 2022-06-27 06:53:29 +00:00
San
cbcf515306
Update SECURITY.md.sh
Update Security.md generation code to include bugcrowd program information.
2022-06-27 12:06:16 +10:00
Robert Schulze
2c828338f4
Replace hyperscan by vectorscan
This commit migrates ClickHouse to Vectorscan. The first 10 min of
[0] explain the reasons for it.

(*) Addresses (but does not resolve) #38046

(*) Config parameter names (e.g. "max_hyperscan_regexp_length") are
    preserved for compatibility. Likewise, error codes (e.g.
    "ErrorCodes::HYPERSCAN_CANNOT_SCAN_TEXT") and function/class names (e.g.
    "HyperscanDeleter") are preserved as vectorscan aims to be a drop-in
    replacement.

[0] https://www.youtube.com/watch?v=KlZWmmflW6M
2022-06-24 10:47:52 +02:00
Antonio Andelic
3a71b63b5d Add list request type 2022-06-23 10:28:12 +00:00
Mikhail f. Shiryaev
d2f3c5b836
Add sleeping after received second rate limit exceeded 2022-06-23 12:13:07 +02:00
Robert Schulze
fa6deaa0f7
Merge pull request #38294 from ClickHouse/fix_system_dot_licences_on_mac
Fix system.licenses on mac
2022-06-23 09:33:07 +02:00
Mikhail f. Shiryaev
779b83262f
Add not for changelog category with titles as entries 2022-06-22 23:50:38 +02:00
Mikhail f. Shiryaev
3ab58fd14f
Add docusaurus header to generated changelogs 2022-06-22 23:47:53 +02:00
Robert Schulze
ec7ef2686b
P -> D (should fix test "01276_system_licenses") 2022-06-22 09:21:12 +00:00
Robert Schulze
287a7f6e98
Let CMake check the presence of GNU utils
In list-licenses.sh it's too late ... the build simply continues if
list-licenses.sh returns with a non-zero exit code.
2022-06-21 23:49:29 +02:00
Robert Schulze
76bd2533a3
Fix system.licenses on mac
- list-licenses.sh assumed GNU versions (e.g. the "-printf" flag of
  find) but Mac ships with a non-GNU version

- this led to error

  [8677/8942] Generating StorageSystemLicenses.generated.cpp
  find: -printf: unknown primary or operator
  find: -printf: unknown primary or operator
  find: -printf: unknown primary or operator
  find: -printf: unknown primary or operator

  during build and as a result, an empty system.licenses table

- As a fix, force the GNU versions of find and grep on Mac
2022-06-21 23:22:27 +02:00
Sergei Trifonov
0a14f3168d
Merge pull request #38287 from ClickHouse/trace-viz-fix
fix trace-viz zoom anomalies
2022-06-21 20:59:35 +02:00
Sergei Trifonov
56a3fc5401 fix trace-viz zoom anomalies 2022-06-21 20:04:52 +02:00
robot-clickhouse
4996236430 Update version_date.tsv after v22.3.7.28-lts 2022-06-20 14:41:14 +00:00
Sergei Trifonov
c0a275984f
Merge pull request #38169 from ClickHouse/trace-visualizer
improve trace-visualizer UX
2022-06-20 12:05:38 +02:00
Alexey Milovidov
457f49a43e Add a word into spelling dictionary 2022-06-19 05:58:45 +02:00
Sergei Trifonov
85dd3afa2b more improvements 2022-06-17 18:02:03 +02:00
Sergei Trifonov
1d9bcb5993 improve trace-visualizer UX 2022-06-17 09:30:08 +02:00
Mikhail f. Shiryaev
7ac4b9b935
New changelog and versions updated 2022-06-16 17:37:41 +02:00
Alexey Milovidov
c737bc2bcf
Update README.md 2022-06-16 12:41:02 +03:00
Alexey Milovidov
d9e558720f Fix typos 2022-06-16 11:38:54 +02:00
Alexey Milovidov
cf6f865f80
Revert "Revert "add d3js based trace visualizer as gantt chart"" 2022-06-16 12:35:13 +03:00
Robert Schulze
8fef6e7fa3
Add missing ":" to error msg 2022-06-15 14:37:58 +02:00
Robert Schulze
50748a94bd
Fix a compile errors in self-extracting-executable (de)compressor
E.g.

utils/self-extracting-executable/compressor.cpp:257:31: format specifies type 'ptrdiff_t' (aka 'long') but the argument has type 'off_t' (aka 'long long') [-Werror,-Wformat]
        printf("Size: %td\n", info_in.st_size);
                      ~~~     ^~~~~~~~~~~~~~~
                      %lld

Not sure though if it's a hard requirement to use only C.

Avoided usage of fmt::format() to keep link dependencies to a minimum.
Also not using C++20 std::format() as it's only available in Clang >=14.
2022-06-15 13:41:28 +02:00
Alexander Tokmakov
afc3f1858b
Revert "add d3js based trace visualizer as gantt chart" 2022-06-14 11:27:49 +03:00
Sergei Trifonov
94b33498ae
Merge pull request #37810 from ClickHouse/trace-d3-gantt 2022-06-14 07:41:48 +02:00
Yakov Olkhovskiy
539100fc4a some refactoring, cleaning, types fixed in fprintf 2022-06-13 20:25:19 -04:00
Yakov Olkhovskiy
ce395dc68d resolving conflicts 2022-06-13 11:41:32 -04:00
xiedeyantu
9f4489ac03
Fix MacOS build faild (#38007) 2022-06-13 12:14:21 +02:00
Yakov Olkhovskiy
0bae6c037a bugs fixed, process EINTR, rename to self-extracting-executable 2022-06-12 23:27:56 -04:00
KinderRiven
18f8f6c61e fix 2022-06-12 17:31:33 +08:00
KinderRiven
b456f657b3 fix 2022-06-12 13:24:08 +08:00
KinderRiven
160e5aa91a fix 2022-06-12 13:23:27 +08:00
KinderRiven
6fa12d02dd fix 2022-06-12 13:17:14 +08:00
KinderRiven
8efd0c8026 restart 2022-06-11 18:07:28 +08:00
KinderRiven
e505df5198 fix bug 2022-06-11 13:18:28 +08:00
Sergei Trifonov
52dc54f0da change trace json format parsing 2022-06-10 21:06:17 +02:00
Sergei Trifonov
139f6035c4 add attribution 2022-06-10 18:57:25 +02:00
Robert Schulze
5f5732a2c4
Merge pull request #37969 from ClickHouse/consistent-macro-usage
More consistent use of platform macros
2022-06-10 14:10:01 +02:00
Yakov Olkhovskiy
5a59957f8b
Merge pull request #35775 from FArthur-cmd/add_self_exctr_exec
Add self extracting executable
2022-06-10 07:59:21 -04:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.

__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
2022-06-10 10:22:31 +02:00
Yakov Olkhovskiy
4787ead19d
typo 2022-06-09 18:34:39 -04:00
Sergei Trifonov
90472569cf add README.md 2022-06-09 15:37:28 +02:00
mergify[bot]
15bcf66763
Merge branch 'master' into add_self_exctr_exec 2022-06-08 20:43:17 +00:00
vdimir
dc772335ee
Doc typos, dict 2022-06-08 14:31:01 +02:00
vdimir
226fcbdac7
Better check-doc-aspell 2022-06-08 14:28:45 +02:00
vdimir
27ebad89c5
Add doc_spell_output.txt to process_style_check_result.py 2022-06-08 14:28:44 +02:00
vdimir
e820291a02
Fix utils/check-style/check-typos 2022-06-08 14:28:44 +02:00
vdimir
94bddec0bc
More typos checked and fixed 2022-06-08 14:28:41 +02:00
vdimir
251403407f
Update spell check 2022-06-08 14:26:02 +02:00
vdimir
13f03ddb65
Update docspell check 2022-06-08 14:26:02 +02:00
vdimir
abce81058e
Add doc aspell 2022-06-08 14:25:56 +02:00
Anton Popov
df6882d2b9
Revert "Fix errors of CheckTriviallyCopyableMove type" 2022-06-07 13:53:10 +02:00
Yakov Olkhovskiy
3d8f00a55f
Update utils/self-extr-exec/decompressor.cpp
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2022-06-07 07:04:57 -04:00
Yakov Olkhovskiy
16a28bf87a
Update utils/self-extr-exec/decompressor.cpp
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2022-06-07 07:04:39 -04:00
Sergei Trifonov
b920a460aa remove limits on input data size 2022-06-03 20:50:13 +02:00
Sergei Trifonov
a7ec8d6389 add d3js based trace visualizer as gantt chart 2022-06-03 01:41:37 +02:00
Alexey Milovidov
eef6a5ec96
Revert "Remove resursive submodules" 2022-06-02 09:41:12 +03:00
HeenaBansal2009
10990402ac Removed move ctor from EventBase hierarchy 2022-06-01 06:01:22 -07:00
Yakov Olkhovskiy
12e77705ee c++ stuff removed, printout fixed, usage printout improved, some minor optimizations 2022-05-30 14:48:48 -04:00
Yakov Olkhovskiy
8730e5410e respect path for decompression, preserve env for exec, some minor bug fixes 2022-05-29 20:39:42 -04:00
Yakov Olkhovskiy
6a134adc40 preserve files permissions, add types header 2022-05-29 14:37:02 -04:00
Yakov Olkhovskiy
e6a6259d2e include decompressor into compressor on post-build step, use included decompressor for compressed file 2022-05-28 21:28:56 -04:00
Yakov Olkhovskiy
cb8f8710ca compressor output prepended with decompressor, some minor fixes 2022-05-28 16:34:26 -04:00
Alexey Milovidov
19b8e1324a Forbid recursive submodules 2022-05-28 04:05:10 +02:00
Antonio Andelic
2ca5b7812d Ignore reusable workflow error 2022-05-27 14:15:34 +00:00
Alexey Milovidov
e1a76e51fb
Merge pull request #37575 from ClickHouse/security-generator
Add security generator
2022-05-27 02:22:33 +03:00
Alexey Milovidov
aeacfa0d7e Readability 2022-05-26 22:23:37 +02:00
Alexey Milovidov
434d8729de Readability 2022-05-26 22:22:14 +02:00
Alexey Milovidov
359e36f421 Readability 2022-05-26 22:21:49 +02:00
Alexey Milovidov
3074be8d17 Add security generator 2022-05-26 22:19:15 +02:00
Sergei Trifonov
417296481e
fix root CMakeLists.txt search 2022-05-26 04:39:02 +02:00
Mikhail f. Shiryaev
0e494c9ee7
Add caching for GitHub PR objects 2022-05-25 00:05:53 +02:00
Mikhail f. Shiryaev
4b28ea92ca
Use commit date for cherry-picked commits (later) 2022-05-25 00:05:52 +02:00
Mikhail f. Shiryaev
fbe16a1d33
Don't compare tags to testing by default 2022-05-25 00:05:50 +02:00
Alexey Milovidov
b3ee8114d9 Minor change 2022-05-22 22:33:41 +02:00
Alexey Milovidov
c34f492293 Merge remote-tracking branch 'origin/master' into changelog-22.5 2022-05-19 16:37:39 +02:00
Alexey Milovidov
775b3b6a05
Update format-changelog.py 2022-05-19 15:51:01 +03:00
Mikhail f. Shiryaev
c3fd892e26
Update version_date.tsv and changelogs after v22.5.1.2079-stable 2022-05-19 10:48:46 +02:00
Alexey Milovidov
08d01e0b41 Revert unrelated changes 2022-05-19 05:22:28 +02:00
Alexey Milovidov
bb431e0a98 Revert README.md 2022-05-19 05:21:51 +02:00
Alexey Milovidov
16513f2a10 Add changelog for 22.5 2022-05-19 05:19:44 +02:00
Mikhail f. Shiryaev
ae0d64d8e1
Merge pull request #37249 from ClickHouse/changelog-script
Improve changelog script
2022-05-18 19:30:31 +02:00
Mikhail f. Shiryaev
e6e0f0589d
Update run-check.py to match PR template, add comments 2022-05-17 21:43:22 +02:00
Mikhail f. Shiryaev
082b30c4a2
Simplify and improve commit validadion for tags 2022-05-16 14:45:27 +02:00
Alexander Tokmakov
c23bc01196 Merge branch 'master' into zookeeper_client_fault_injection 2022-05-11 17:20:42 +02:00
Mikhail f. Shiryaev
d78709c656
Add dependencies and CI for changelogs generator 2022-05-10 11:25:38 +02:00
Mikhail f. Shiryaev
6cdab10d29
Rewrite changelog generator to pure python 2022-05-10 11:25:36 +02:00
robot-clickhouse
6aa870ebb6 Update version_date.tsv after v22.4.5.9-stable 2022-05-06 18:59:00 +00:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase
Reorganize source files so that base won't depend on Common
2022-04-30 09:40:58 +03:00
robot-clickhouse
e57f8e0c68 Update version_date.tsv after v22.3.5.5-lts 2022-04-29 15:17:41 +00:00
Amos Bird
20f63a6fa4
Fix style check 2022-04-29 15:32:53 +08:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
alesapin
33c021702a Black 2022-04-28 18:16:10 +02:00
alesapin
3107b2bcd6 Add small script for stupid keeper check 2022-04-28 18:13:54 +02:00
snyk-bot
326be13c05
fix: utils/clickhouse-diagnostics/requirements.txt to reduce vulnerabilities
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-NUMPY-2321964
- https://snyk.io/vuln/SNYK-PYTHON-NUMPY-2321966
- https://snyk.io/vuln/SNYK-PYTHON-NUMPY-2321969
- https://snyk.io/vuln/SNYK-PYTHON-NUMPY-2321970
2022-04-28 02:30:08 +00:00
Alexey Milovidov
20281c25b3
Merge pull request #36653 from ramazanpolat/patch-2
add missing pandas package
2022-04-27 05:26:53 +03:00
robot-clickhouse
a96276aa09 Update version_date.tsv after v22.3.4.20-lts 2022-04-26 10:47:39 +00:00
Ramazan Polat
07fcb57729
add missing pandas package 2022-04-26 10:29:21 +03:00
robot-clickhouse
b761d5bf3a Update version_date.tsv after v22.4.3.3-stable 2022-04-26 06:51:23 +00:00
mergify[bot]
705d5af3a0
Merge branch 'master' into to_start_of_five_minutes 2022-04-24 22:24:24 +00:00
robot-clickhouse
f0fecb5071 Update version_date.tsv after v22.4.2.1-stable 2022-04-22 09:35:11 +00:00
Memo
956d525840 fix conflict 2022-04-22 11:22:50 +08:00
Memo
39aadf0975 replaced toStartOfFiveMinute to toStartOfFiveMinutes 2022-04-22 10:49:59 +08:00
FArthur-cmd
1f8fd1678b improve compressor 2022-04-21 15:52:32 +00:00
Filatenkov Artur
0f07daf266
fix typo 2022-04-19 21:51:44 +03:00
Filatenkov Artur
3c9881bee2
Delete make_compression.sh 2022-04-19 20:06:08 +03:00
Filatenkov Artur
bff407c139
Fix style 2022-04-19 20:04:04 +03:00
Filatenkov Artur
0398508acc
Fix style 2022-04-19 20:03:27 +03:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Alexey Milovidov
7de4f2b6c1 Merge branch 'master' into remove-arcadia 2022-04-17 16:30:09 +02:00
Mikhail f. Shiryaev
b6a39e78d0
Clean out old packaging 2022-04-16 10:34:40 +02:00
Alexey Milovidov
40357637ec Remove Arcadia 2022-04-16 00:33:32 +02:00
Alexey Milovidov
e128d89957 Remove Arcadia 2022-04-16 00:24:58 +02:00
FArthur-cmd
0baa4f97ec add parallel decompression, forward arguments and fix compression level 2022-04-14 11:43:40 +00:00
Kruglov Pavel
d7b88d7683
Merge pull request #35582 from Avogar/improve-schema-inference
Improve schema inference and add some fixes
2022-04-08 13:44:52 +02:00