thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
85,624 Commits 9,326 Branches 1,783 Tags 2.1 GiB
24d72a18d0
Commit Graph

3129 Commits

Author SHA1 Message Date
kssenii
36a41ac146 Better 2022-01-22 20:42:39 +03:00
Mikhail f. Shiryaev
13f7f56c5b
Merge pull request #33342 from JaySon-Huang/get_cpu_cores_under_cgroup 
Fix cpu count with cgroup limits
 2022-01-22 11:32:08 +01:00
alexey-milovidov
f3053d3727
Merge pull request #33706 from hanqf-git/avx512_memcmpSmall 
Avx512 memcmp small
 2022-01-22 13:02:47 +03:00
alexey-milovidov
eb6849f7c7
Merge pull request #33842 from azat/cmake-contrib-fixes 
More cmake external modules cleanups
 2022-01-22 10:34:54 +03:00
Nikolai Kochetov
b4311cf3c9 Fix cmake for mac. 2022-01-21 22:08:42 +03:00
Anton Popov
e8ce091e68 Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-21 20:11:18 +03:00
kssenii
1858f69f5a Make space reservation incremental 2022-01-21 18:39:34 +03:00
tavplubix
71d69d813c
Merge pull request #33860 from azat/fix-zk-lib 
Fix zookeeper library dependency from interpreters (by marking library STATIC)
 2022-01-21 16:21:48 +03:00
JaySon-Huang
e155057cc5 Fix cpu count with cgroup limits 
Signed-off-by: JaySon-Huang <tshent@qq.com>
 2022-01-21 21:03:23 +08:00
Kruglov Pavel
daba57a933
Merge pull request #33737 from Avogar/bool-type 
Some improvements and fixes for Bool data type
 2022-01-21 15:24:58 +03:00
Azat Khuzhin
d463031105 Fix zookeeper library dependency from interpreters (by marking library STATIC) 
This had been introduced in #33534, and an attempt to fix this had been
made in #33844 but it was not strictly correct, since there is undefined
reference to `DB::SystemLog<DB::ZooKeeperLogElement>::add`:

    $ nm -D ./src/Common/ZooKeeper/libclickhouse_common_zookeeperd.so | fgrep SystemLog | c++filt
          U DB::SystemLog<DB::ZooKeeperLogElement>::add(DB::ZooKeeperLogElement const&)

That patch works, because default is not
`-Wl,--unresolved-symbols=report-all`, so I guess shared build on OSX
may after #33844.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-21 11:10:03 +03:00
Azat Khuzhin
d25b59803e contrib/abseil: add cmake ALIAS library 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-21 10:11:06 +03:00
zzsmdfj
411f43ec4d to MaterializeMySQL_support_set_and_other_dataType 2022-01-21 12:24:12 +08:00
Maksim Kita
8567c3a20e
Merge pull request #33848 from kitaisreal/fix-build-with-examples 
Fix build with ENABLE_EXAMPLES
 2022-01-20 23:18:47 +01:00
Maksim Kita
e2fe8dc3a7 Fix build with ENABLE_EXAMPLES 2022-01-20 19:29:21 +00:00
tavplubix
d26d139974
Update CMakeLists.txt 2022-01-20 20:27:14 +03:00
Maksim Kita
502c1637d5
Merge pull request #33827 from kitaisreal/range-hashed-dictionary-handle-invalid-intervals 
RangeHashedDictionary handle invalid intervals
 2022-01-20 17:16:49 +01:00
Kruglov Pavel
7bfb1231b9
Merge branch 'master' into formats-with-suffixes 2022-01-20 14:47:17 +03:00
Maksim Kita
844eb4ccdc RangeHashedDictionary handle invalid intervals 2022-01-20 11:16:18 +00:00
avogar
ba54a5ff80 Some improvements and fixes for Bool data type 2022-01-20 14:16:11 +03:00
Azat Khuzhin
a6d482d3e1 Cleanup jemalloc contrib 2022-01-20 10:02:03 +03:00
Azat Khuzhin
9926f336e3 Cleanup cpuid contrib 2022-01-20 10:02:03 +03:00
Azat Khuzhin
3e58094bcb Fix build w/o replxx 2022-01-20 10:02:02 +03:00
Azat Khuzhin
e0e81b340d Fix w/o ODBC build 2022-01-20 10:02:02 +03:00
Azat Khuzhin
e160a20769 Remove USE_OPENCL (removed in #31744) 2022-01-20 10:02:02 +03:00
Azat Khuzhin
a75b748fee Remove unbundled mysql support 2022-01-20 10:02:01 +03:00
Azat Khuzhin
7420bd9b18 Move pdqsort 2022-01-20 10:01:59 +03:00
Azat Khuzhin
a773e7ff01 Remove unbundled libpqxx support 2022-01-20 10:01:59 +03:00
Azat Khuzhin
592bc1e97d Remove unbundled yaml-cpp support 2022-01-20 10:01:59 +03:00
Azat Khuzhin
f1cc63d900 Remove unbundled cityhash support 2022-01-20 10:01:12 +03:00
Azat Khuzhin
8ede97925e Remove unbundled sparsehash support 2022-01-20 10:01:11 +03:00
Azat Khuzhin
789dfd9f3b Remove unbundled re2 support 
v2: preserve re2_st name to make PVS check pass (since docker image
update fails)
 2022-01-20 10:00:49 +03:00
Azat Khuzhin
aef6668cb4 Remove unbundled hdfs3 support 2022-01-20 08:47:16 +03:00
Azat Khuzhin
cf30669cc9 Remove unbundled openssl/boringssl (crypo/ssl) support 2022-01-20 08:47:16 +03:00
Azat Khuzhin
313716be94 Remove unbundled thrift support 2022-01-20 08:47:16 +03:00
Azat Khuzhin
ca8525a733 Remove unbundled zlib support 2022-01-20 08:47:16 +03:00
mergify[bot]
cbb87e762c
Merge branch 'master' into formats-with-suffixes 2022-01-19 17:21:18 +00:00
Maksim Kita
41a6cd54aa
Merge pull request #33516 from kitaisreal/range-hashed-dictionary-interval-tree 
RangeHashedDictionary use interval tree
 2022-01-19 16:30:31 +01:00
mergify[bot]
e2f92cf18f
Merge branch 'master' into avx512_memcmpSmall 2022-01-19 12:25:04 +00:00
Maksim Kita
1f72bde52f Fix performance tests 2022-01-19 10:41:58 +00:00
Nikolay Degterinsky
8d03099c91 Merge branch 'master' into classification 2022-01-19 09:21:31 +00:00
alexey-milovidov
d222cb9771
Merge pull request #33695 from amosbird/buildimprove 
Some build improvement
 2022-01-19 03:19:22 +03:00
kssenii
a566099759 Local cache for remote filesystem 2022-01-19 00:42:11 +03:00
Nikolay Degterinsky
9d45025c7d Merge branch 'master' into classification 2022-01-18 21:41:03 +00:00
Dmitry Novik
4e612d0e1a
Merge branch 'master' into memory-overcommit 2022-01-18 23:33:29 +03:00
alesapin
e5ec1aa446
Merge pull request #33642 from ClickHouse/add_integration_test_for_zero_duplication 
Fix bug with background moves in zero copy replication
 2022-01-18 18:32:16 +03:00
Maksim Kita
821c5dd989 Fix build 2022-01-18 14:46:19 +00:00
Kruglov Pavel
2295a07066
Merge pull request #33534 from azat/fwd-decl 
RFC: Split headers, move SystemLog into module, more forward declarations
 2022-01-18 17:22:49 +03:00
Maksim Kita
562098779f Fixed style check 2022-01-18 13:26:23 +00:00
Maksim Kita
60bcf88228 Added IntervalTree documentation 2022-01-18 13:20:43 +00:00
Dmitry Novik
c0970b75ee
Merge branch 'master' into memory-overcommit 2022-01-18 15:30:24 +03:00
Dmitry Novik
83c663e2d6 Cleanup after code review 2022-01-18 12:21:59 +00:00
mergify[bot]
b318f9b5db
Merge branch 'master' into formats-with-suffixes 2022-01-18 12:17:07 +00:00
alesapin
eff85d3352 Review fixes 2022-01-18 11:27:01 +03:00
Amos Bird
6d62060e16
Build improvement 2022-01-17 22:36:27 +08:00
Maksim Kita
f311960b00 Added tests 2022-01-17 14:22:01 +00:00
alesapin
4caef03e6a Add ephemeral lock in zookeeper 2022-01-17 14:52:51 +03:00
alesapin
b2271cc2d9
Merge pull request #33288 from JackyWoo/add_lower_bound_session_timeout_to_keeper 
Add lower bound session timeout to keeper
 2022-01-17 14:06:15 +03:00
Maksim Kita
ea78c0b33c Fixed style 2022-01-16 16:45:33 +00:00
Maksim Kita
c3d0e8e6f1 Fix style check 2022-01-16 12:23:40 +00:00
Maksim Kita
1076ec5539 Added IntervalTree example 2022-01-16 12:23:40 +00:00
Maksim Kita
0626720893 Added IntervalTree 2022-01-16 12:23:40 +00:00
Kruglov Pavel
5a908e8edd
Merge branch 'master' into formats-with-suffixes 2022-01-14 16:45:20 +03:00
avogar
eeec8c6c11 Fix error codes order 2022-01-14 16:36:12 +03:00
hanqf-git
46f3a699df Add x86 feature avx512 support for memcmpSmall 2022-01-14 20:21:22 +08:00
alesapin
d3a93f5d7e
Merge pull request #33602 from ClickHouse/interpret_keeper_error_in_our_client 
Make ZooKeeper client better interpret keeper server connection reject
 2022-01-14 11:32:27 +03:00
alesapin
30180ddc19 Better text 2022-01-13 19:12:01 +03:00
alesapin
ec6840f49c Make ZooKeeper client better interpret keeper server connection reject 2022-01-13 17:12:10 +03:00
mergify[bot]
085492614b
Merge branch 'master' into add_lower_bound_session_timeout_to_keeper 2022-01-13 12:21:46 +00:00
taiyang-li
964f92f2c1 Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-13 18:29:38 +08:00
taiyang-li
82a3024712 Merge remote-tracking branch 'ck/master' into bigo_hive_table 2022-01-13 18:11:23 +08:00
taiyang-li
6567cd7abc fix all the stateless test 2022-01-13 15:27:41 +08:00
lgbo-ustc
5c71d3687a fixed some bugs 
1. interagtion test for test_hive_query failed
2. nullptr reference in arrowSchemaToCHHeader
 2022-01-12 17:01:05 +08:00
taiyang-li
66813a3aa9 merge master 2022-01-12 16:56:29 +08:00
Azat Khuzhin
003cdbcf60 Introduce clickhouse_common_config_no_zookeeper_log 
Otherwise incorrect zookeeper (w/o log) will be used in clickhouse
server itself.
 2022-01-12 07:17:56 +03:00
Azat Khuzhin
4a2bc64989 Introduce clickhouse_common_zookeeper_no_log (w/o extra deps for examples) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-11 23:51:41 +03:00
Azat Khuzhin
248b879fd5 Split TraceCollector between Common and Interpreters 
This is required to leave TraceCollector::collect() inside main for
MemoryTrackier.
 2022-01-11 22:30:55 +03:00
Maksim Kita
457a38a1ad HashTable constant fix 2022-01-11 18:45:41 +03:00
taiyang-li
6751c8552a merge master and resolve conflict 2022-01-11 15:21:56 +08:00
taiyang-li
d31871d253 commit again 2022-01-11 15:17:00 +08:00
Nikolay Degterinsky
85b8985df2 Better 2022-01-10 20:30:26 +00:00
Mikhail f. Shiryaev
1cf46f378f
Merge pull request #33494 from msaf1980/rollup_rules_type 
GraphiteMergeTree Rollup rules type
 2022-01-10 20:53:20 +01:00
Azat Khuzhin
cb70544dfe Move LockMemoryExceptionInThread and MemoryTrackerBlockerInThread 2022-01-10 22:39:10 +03:00
Azat Khuzhin
c1dea66907 Move TraceCollector into Interpreters 
Since now it relies on SystemLog that is in Interpreters, and it cannot
be moved into Common, since it has lots of dependencies.
 2022-01-10 22:35:42 +03:00
Azat Khuzhin
aee034a597 Use explicit template instantiation for SystemLog 
- Move some code into module part to avoid dependency from IStorage in SystemLog
- Remove extra headers from SystemLog.h
- Rewrite some code that was relying on headers that was included by SystemLog.h

v2: rebase
v3: squash move into module part with explicit template instantiation
    (to make each commit self compilable after rebase)
 2022-01-10 22:01:41 +03:00
msaf1980
780a1b2abe graphite: split tagged/plain rollup rules (for merges perfomance) 2022-01-10 16:34:16 +05:00
Kseniia Sumarokova
e6ea7c361d
Merge pull request #32836 from stigsb/explain-table-override 
Add `EXPLAIN TABLE OVERRIDE` for pre-validating overrides.
 2022-01-10 09:58:16 +03:00
taiyang-li
2bb7ec8f72 Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-06 10:44:17 +08:00
lgbo-ustc
f78564491c code style 2022-01-05 14:40:02 +08:00
lgbo-ustc
624da11ee2 fix a bug : cache files are released in wrong way 2022-01-05 14:29:34 +08:00
alexey-milovidov
48eec3d8f8
Merge pull request #33366 from ClickHouse/aggregate-dependencies 
Aggregate functions to calculate dependencies between columns
 2022-01-05 02:50:25 +03:00
taiyang-li
14bce62cc4 fix building 2022-01-04 15:34:34 +08:00
李扬
2df2442ad0
Merge branch 'master' into hive_table 2022-01-04 01:26:16 -06:00
lgbo-ustc
45c86757bf change LRUCache to LRUResourceCache 2022-01-04 15:19:09 +08:00
taiyang-li
3f6d830536 Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-04 10:00:34 +08:00
Alexey Milovidov
4ebc8918a4 Fix style 2022-01-03 02:11:47 +03:00
Amos Bird
0b7a94c2be Saturate date/datetime to zero 2022-01-03 02:07:08 +03:00
Alexey Milovidov
cf66716cd6 Make it less wrong 2022-01-02 22:42:56 +03:00
Alexey Milovidov
4a094c2efd Merging contingency coefficients 2022-01-02 21:50:41 +03:00
taiyang-li
1e102bc1b2 merge master 2022-01-01 09:01:06 +08:00
alexey-milovidov
3f1810c049
Merge pull request #33327 from ClickHouse/rename_committed_to_active 
Rename Committed data part state to Active
 2021-12-31 14:40:35 +03:00
alexey-milovidov
6941b072f4
Merge pull request #33330 from cmsxbc/support-self-closing-tag-for-cluster-secure 
support `<secure/>` in cluster configuration.
 2021-12-31 14:36:19 +03:00
Kseniia Sumarokova
a3b30dce32
Merge pull request #33334 from kssenii/follow-up-for-33098 
Tiny follow-up to pr 33098
 2021-12-31 11:11:02 +03:00
SuperDJY
4322fd4bdb
avoid underscore in gtest name 2021-12-31 10:19:03 +08:00
cmsxbc
fd9f1193d8
use .empty() instead of .size() == 0 2021-12-31 08:30:21 +08:00
Alexander Tokmakov
187c43eba8 rename Committed state to Active 2021-12-30 23:45:38 +03:00
alexey-milovidov
42b8678785
Merge pull request #33271 from ClickHouse/always-display-resource-usage-in-client 
Always display resource usage in client
 2021-12-30 23:36:49 +03:00
kssenii
ae2813a2d5 Fix 2021-12-30 23:31:12 +03:00
Kseniia Sumarokova
a703bcb0c5
Merge pull request #33098 from bigo-sg/lrucache 
Add alternative LRUCache version
 2021-12-30 23:10:28 +03:00
cmsxbc
f0e82e2cb6
add comment for ConfigHelper::getBool 2021-12-31 00:47:01 +08:00
cmsxbc
c28fcd5a2f
support <secure/> for cluster 2021-12-30 23:51:05 +08:00
lgbo-ustc
9c32723dcf fix a bug in LRUResourceCache::getImpl() 2021-12-30 21:48:54 +08:00
lgbo-ustc
e6528ca0ef fix a bug in LRUCache::remove() 2021-12-30 20:38:58 +08:00
lgbo-ustc
77084f5349 add token holder 2021-12-30 18:43:12 +08:00
JackyWoo
569ce62e8d use session_timeout as session timeout uper bound 2021-12-30 17:18:51 +08:00
Stig Bakken
6fbb9f5e87 Implement EXPLAIN TABLE OVERRIDE for pre-validating overrides. 
Example:

```sql
EXPLAIN TABLE OVERRIDE mysql('127.0.0.1:3306', 'db', 'table', 'user', 'pw')
    PARTITION BY tuple(toYYYYMM(created), id % 8)
```

Validations done:

 * check that storage overrides do not reference unknown or nullable columns
 * check that default specifier is not modified for columns
 2021-12-30 09:02:27 +01:00
lgbo-ustc
e8c9079bb8 rename test cases 2021-12-30 14:34:08 +08:00
lgbo-ustc
38605d6be3 rename test cases 2021-12-30 12:20:14 +08:00
Nikolay Degterinsky
b9a75bc39c Fix build 2021-12-30 03:15:21 +00:00
Nikolay Degterinsky
2bdee0b0b8 Merge branch 'master' into classification 2021-12-30 02:17:47 +00:00
lgbo-ustc
54d78bde0d update comments 2021-12-30 10:15:56 +08:00
Nikolay Degterinsky
ba0083736e Change DetectLanguageMixed output type 2021-12-30 02:14:57 +00:00
lgbo-ustc
88383b6715 some change 2021-12-30 10:08:33 +08:00
avogar
97788b9c21 Allow to create new files on insert for File/S3/HDFS engines 2021-12-29 21:19:13 +03:00
alexey-milovidov
3655965b39
Merge pull request #33264 from azat/symbols-from-resources-fix 
Fix parsing symbols from resources (for shared builds)
 2021-12-29 21:06:51 +03:00
JackyWoo
ad7f206840 change max_session_timeout_ms default value from 30000 to 100000 2021-12-29 21:05:27 +08:00
JackyWoo
d35e5f8319 add lower bound session timeout to keeper 2021-12-29 20:59:01 +08:00
mergify[bot]
5070784282
Merge branch 'master' into memory-overcommit 2021-12-29 10:35:32 +00:00
lgbo-ustc
a412738264 format code style 2021-12-29 18:27:52 +08:00
lgbo-ustc
6b6a82f3b9 add MappedHolder to get cache values 2021-12-29 18:18:38 +08:00
Kruglov Pavel
dd7f61b337
Merge branch 'master' into schema-inference 2021-12-29 12:59:05 +03:00
avogar
8112a71233 Implement schema inference for most input formats 2021-12-29 12:18:56 +03:00
lgbo-ustc
5a3b215f24 fixed code style 2021-12-29 16:00:57 +08:00
lgbo-ustc
6aaeb285b0 update destructor 2021-12-29 15:44:24 +08:00
lgbo-ustc
0d474069b5 add a new test case 2021-12-29 15:38:58 +08:00
lgbo-ustc
5f85f77266 add a new test case 2021-12-29 15:34:20 +08:00
lgbo-ustc
5a419a356b fix a bug 2021-12-29 15:29:59 +08:00
lgbo-ustc
59cbd76880 Add LRUResourceCache 
1. add LRUResourceCache for managing resource cache in lru policy
2. rollback LRUCache to the original version
3. add remove() in LRUCache
4. add unit tests for LRUResourceCache and LRUCache
 2021-12-29 15:25:33 +08:00
Azat Khuzhin
a23bb79479 Fix parsing symbols from resources (for shared builds) 
In case of shared builds the symbol can be represented in multiple
shared objects, and so if one SO contains _end symbol firstly, then it
it's address can subtracted from the symbol from another SO, and this
give overflow.

Here is an example of such symbol:

    $ llvm-readelf -a programs/server/libclickhouse-server-libd.so | fgrep schemata_sql
       552: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size
      1271: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
      3627: 000000000010b480     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start
      5822: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
      5828: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size
      5831: 000000000010b480     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start

    $ llvm-readelf -W -a src/TableFunctions/libclickhouse_table_functionsd.* | fgrep schemata_sql
      1709: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
      5107: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size
      6090: 00000000001fc1b8     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start
     10975: 00000000001fc1b8     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start
     10976: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
     10977: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size

To address this, add base address of the Elf into the resources to
compare with.

Follow-up for: #33134 (cc @alexey-milovidov)
 2021-12-29 09:56:59 +03:00
Alexey Milovidov
d456c5ddda Always display resource usage in client 2021-12-29 01:26:58 +03:00
Alexey Milovidov
e8e19b8b0d Always display resource usage in client 2021-12-29 01:20:34 +03:00
Maksim Kita
d1db3c9f42 Updated ShellCommandSource 2021-12-28 22:55:31 +03:00
Maksim Kita
5590cfa3aa Updated executable function integration tests 2021-12-28 22:55:30 +03:00
taiyang-li
aa97634d9b merge master 2021-12-28 18:38:50 +08:00
taiyang-li
2a1fe52b9f rename symbols && fix uts && add setting use_metadata_cache 2021-12-28 18:06:13 +08:00
李扬
91334ecf50
Merge branch 'master' into rocksdb_metacache 2021-12-27 23:20:41 -06:00
lgbo-ustc
1ac697cad2 some modifications 2021-12-28 11:26:39 +08:00
lgbo-ustc
413fb290bb remove comments 2021-12-28 09:19:21 +08:00
lgbo-ustc
26e61061b0 update setImpl() 2021-12-27 20:46:51 +08:00
lgbo-ustc
922dd40c05 optimization 2021-12-27 20:33:25 +08:00
Vitaly Baranov
f80a3e3247
Merge pull request #30549 from aiven/kmichel-server-reload 
Start/stop servers when `listen_host`/`*_port` changes
 2021-12-27 14:51:00 +03:00
lgbo-ustc
f86e3ef1ae update the evict action when overwrite a key 2021-12-27 18:12:40 +08:00
lgbo-ustc
54b2a42530 update variable names 2021-12-27 18:02:02 +08:00
lgbo-ustc
3367755dcc update comment 2021-12-27 16:21:10 +08:00
lgbo-ustc
ef1d7142f5 remove getOrTrySet 2021-12-27 16:12:39 +08:00
lgbo-ustc
0a39273fee fixed code style 2021-12-27 15:47:22 +08:00
taiyang-li
2d6eb40879 fix style 2021-12-27 15:31:24 +08:00
taiyang-li
dd882085d0 Merge branch 'hive_table' of https://github.com/bigo-sg/ClickHouse into bigo_hive_table 2021-12-27 15:14:43 +08:00
taiyang-li
9036b18c2f merge master 2021-12-27 15:12:48 +08:00
lgbo-ustc
078521496a modifications based on pr review 2021-12-27 15:10:48 +08:00
taiyang-li
e7401d2a5e wrap rocksdb metacache related code with USE_ROCKSDB 2021-12-27 11:50:59 +08:00
lgbo-ustc
51e43ae619 some modification in LRUCache 2021-12-27 10:20:07 +08:00
taiyang-li
94a7a09598 Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2021-12-27 10:14:24 +08:00
Alexey Milovidov
bbd8e055a1 Fix clang-tidy 2021-12-24 18:25:59 +03:00
Alexey Milovidov
c8b8cc74fc Fix clang-tidy 2021-12-24 18:15:04 +03:00
Alexey Milovidov
b7799dcc45 Remove even more useless code 2021-12-24 12:25:27 +03:00
Alexey Milovidov
ce5d63510f Remove even more useless code 2021-12-24 12:25:27 +03:00
Alexey Milovidov
edb50b5844 Remove unused code 2021-12-24 12:25:27 +03:00
Alexey Milovidov
a96b4d50d6 Fix error 2021-12-24 12:25:27 +03:00
Alexey Milovidov
003ee51c6d Fix buildId 2021-12-24 12:25:27 +03:00
Alexey Milovidov
ae61cf66b5 Fix resources 2021-12-24 12:25:27 +03:00
Alexey Milovidov
29d28c531f Move code around to avoid dlsym on Musl 2021-12-24 12:25:27 +03:00
Alexey Milovidov
d6ece47b7f Allow vfork 2021-12-24 12:25:27 +03:00
Kevin Michel
ffc1fca296
Start/stop servers when listen_host/*_port changes 
This allows starting and stopping separately each protocol server
without restarting ClickHouse.

This also allows adding or removing `listen_host` entries, which
start and stops servers for all enabled ports.

When stopping a server, the listening socket is immediately closed
(and available for another server).

Protocols with persistent connections try to wait for any currently
running query to finish before closing the connection, but idle
connection are closed quickly (depending on how often the protocol
is polled).

An extra ProfileEvent is added, `MainConfigLoads`, it is
incremented every time the configuration is reloaded. This helps
when trying to assess whether the new configuration was applied.
 2021-12-24 08:26:02 +01:00
Nikolay Degterinsky
52b18b0c56 Refactor 2021-12-23 14:10:12 +00:00
lgbo-ustc
5a9bd7fd41 fixe typos 2021-12-23 18:13:56 +08:00
lgbo-ustc
e25b1f7361 update comments 2021-12-23 18:05:52 +08:00
lgbo-ustc
fe4ef69fc7 remove unused header 2021-12-23 18:01:58 +08:00
lgbo-ustc
59bc87e409 remove unused errorcode 2021-12-23 17:53:19 +08:00
lgbo-ustc
9fdb061b7e Add evict policy in LRUCache 2021-12-23 17:36:41 +08:00
lgbo-ustc
abe79fb1c2 move Exception in LRUCache::set() 2021-12-23 16:33:43 +08:00
lgbo-ustc
d29a4d0527 add trySet() in LRUCache 2021-12-23 16:16:33 +08:00
lgbo-ustc
f0b22d2882 fixed code-style 2021-12-23 15:15:39 +08:00
alexey-milovidov
12fb830181
Merge pull request #33061 from ClickHouse/merge-33024 
Merge #33024
 2021-12-23 07:50:11 +03:00
lgbo-ustc
c3736859ac update some comments 2021-12-23 11:56:54 +08:00
lgbo-ustc
d205f9ecab codes refactor 2021-12-23 11:50:26 +08:00
Alexey Milovidov
53c83e4813 Merge branch 'Issue77' of github.com:DevTeamBK/ClickHouse into merge-33025 2021-12-23 01:36:46 +03:00
Alexey Milovidov
3e059a095b Corrections 2021-12-23 01:25:35 +03:00
Alexey Milovidov
ba77993943 Merge branch 'Issue20' of github.com:DevTeamBK/ClickHouse into merge-33024 2021-12-23 01:23:47 +03:00
Rajkumar
a75d3d1c83 Review comments 2021-12-22 13:39:45 -08:00
Rajkumar
afe9e6a325 Addressing review comment 2021-12-22 11:42:22 -08:00
Rajkumar
852889f995 Addressing review comment 2021-12-22 08:54:33 -08:00
Maksim Kita
5d4b0f8f1a
Merge pull request #33026 from ClickHouse/remove-old-code-config-processor 
Remove obsolete code from ConfigProcessor.
 2021-12-22 13:34:53 +03:00
lgbo-ustc
2d226e287a update comment in LRUCache::set 2021-12-22 15:11:39 +08:00
Kseniia Sumarokova
484a4d62dd
Merge pull request #32835 from kssenii/fising-s3 
Fix optimization with lazy seek for async reads from remote fs
 2021-12-22 09:38:23 +03:00
Meena-Renganathan
91036e729c Fixed the indentation 
Signed-off-by: Meena-Renganathan <Meena.Renganathan@ibm.com>
 2021-12-21 16:07:24 -08:00
Boris Kuschel
c452b3fad9 Modifed the code to handle the sysconf() fialure 
Signed-off-by: Boris Kuschel <Boris.Kuschel@ibm.com>
 2021-12-21 18:33:00 -05:00
Alexey Milovidov
088d14b885 Remove obsolete code from ConfigProcessor. 2021-12-22 02:13:31 +03:00
Boris Kuschel
68de95afc9 Native Format Parsing - Integer Overflow Resizing Arrays 
Signed-off-by: Boris Kuschel <Boris.Kuschel@ibm.com>
 2021-12-21 18:10:51 -05:00
taiyang-li
2597925724 merge master 2021-12-21 15:55:39 +08:00
lgbo-ustc
44c0f0e680 rewrite comments 2021-12-21 10:59:23 +08:00
lgbo-ustc
fb1214efeb fixed code style 2021-12-21 10:56:57 +08:00
lgbo-ustc
eb9e9bd47a fixed a bug 2021-12-21 10:50:25 +08:00
lgbo-ustc
2e2354be40 1. add tryDel and getOrTrySet in LRUCache 
2. fixed bugs in LRUCache for evicting elements
 2021-12-21 10:39:59 +08:00
lgbo-ustc
2c67516432 1. fixed code style 
2. fixed bugs in lru cache
 2021-12-21 09:44:04 +08:00
alexey-milovidov
04fb8aeaa5
Merge pull request #32946 from antonio2368/feature/hints-for-settings 
Hints for invalid settings in query and HTTP requests
 2021-12-21 00:52:42 +03:00
Maksim Kita
dd0d3de050
Merge pull request #32970 from kitaisreal/loops-remove-postfix-increment 
Loops remove postfix increment
 2021-12-20 19:51:07 +03:00
Antonio Andelic
bc52758e4d Explicitly default move/copy constructors 2021-12-20 17:05:08 +01:00
Maksim Kita
51477adf1b Updated additional cases 2021-12-20 15:55:07 +03:00
Azat Khuzhin
6aebc3e94c Do not loose ProfileEvents in case of thread destroyed before 
v2: drop std::move and add copy ctor for ProfileEvents::Counter::Snapshot
v2: remove std::move
 2021-12-19 23:43:05 +03:00
Azat Khuzhin
909ce68b5a Do not suppress exception in ThreadStatus::~ThreadStatus() 2021-12-19 23:43:05 +03:00
liyang
37ba8004ff Speep up mergetree starting up process 2021-12-18 16:39:59 +08:00
Anton Popov
99ebabd822 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-17 19:02:29 +03:00
kssenii
ed42620530 Rename BlobStorage to AzureBlobStorage 2021-12-17 12:24:05 +03:00
taiyang-li
d033fc4c24 merge master and fix conflict 2021-12-17 15:11:21 +08:00
kssenii
ca283a4bb8 Fix optimization with lazy seek 2021-12-17 00:18:28 +03:00
mergify[bot]
cc3176d437
Merge branch 'master' into memory-overcommit 2021-12-16 12:44:50 +00:00
alesapin
2753fe01a3 Merge branch 'master' into fix_special_build_check 2021-12-15 10:35:28 +03:00
mergify[bot]
dfdf886976
Merge branch 'master' into memory-overcommit 2021-12-15 00:00:28 +00:00
Dmitry Novik
de432f9270 Fix possible NPE when thread_group is not set 2021-12-14 16:39:53 +03:00
alesapin
9974f10631 Fix clang tidy 2021-12-14 15:34:20 +03:00
lgbo-ustc
248e933fc4 Add test for local cache 2021-12-14 16:06:30 +08:00
Azat Khuzhin
a7dc6f309f Remove SparseHashMap.h 
It was added only for arcadia build, and used only in one place, no need
to have a separate typedef for it.
 2021-12-14 10:07:14 +03:00
Azat Khuzhin
e16891713d Fix sparse_hashed dict performance with sequential keys (wrong hash function) 
In #27152 the hash function for sparse_hash_map had been changed to
std::hash<> switch it back to DefaultHash<> (ClickHouse builtin), since
std::hash<> for numeric keys returns itself and this does not works
great with sparse_hash_map.

I've tried the example from #32480 and using some hash fixes the
performance of sparse_hashed layout.

Fixes: #32480

v2: Add comments for SparseHashMap
 2021-12-14 10:07:14 +03:00
taiyang-li
8234d1176f merge master 2021-12-14 10:39:21 +08:00
Kseniia Sumarokova
78ed383f15
Merge pull request #31505 from ContentSquare/azure_blob_storage_dependency 
Azure Blob Storage Disks
 2021-12-13 18:28:48 +03:00
mergify[bot]
b81d600c1e
Merge branch 'master' into memory-overcommit 2021-12-12 23:01:56 +00:00
李扬
8675086104
Merge branch 'master' into hive_table 2021-12-12 09:01:46 -06:00
Jakub Kuklis
60dd22dd4e Checking for success of file removal operation 2021-12-10 22:04:37 +01:00
Jakub Kuklis
43f420cba8 Adding a flag to use Azure Blob Storage 2021-12-10 22:04:36 +01:00
kssenii
f5a77fca39 Done 2021-12-09 14:44:25 +00:00
Nikolay Degterinsky
2b2ffcc8ea Merge branch 'master' into classification 2021-12-09 11:22:01 +00:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
Mikhail f. Shiryaev
03927f5fc6
Revert "graphite: split tagged/plain rollup rules (for merges perfomance)" 
This reverts commit 303552f515.
 2021-12-08 15:49:21 +01:00
lgbo
38020a33e4 some refinement 2021-12-08 10:07:04 +08:00
Dmitry Novik
a956a468f7 Remove log entry with data race 2021-12-07 13:03:51 +03:00
mergify[bot]
907a042af3
Merge branch 'master' into memory-overcommit 2021-12-07 09:50:37 +00:00
Mikhail f. Shiryaev
5fc20b3e6a
Merge pull request #25122 from msaf1980/rollup_rules_type 
GraphiteMergeTree Rollup rules type
 2021-12-07 10:18:08 +01:00
lgbo-ustc
54cc65ede8 fixed code style 2021-12-07 17:12:26 +08:00
lgbo-ustc
fcea033a07 fixed code style 2021-12-07 12:17:47 +08:00
lgbo-ustc
0c1733817b update LRUCache set() 2021-12-07 10:57:05 +08:00
mergify[bot]
600dcb749a
Merge branch 'master' into memory-overcommit 2021-12-07 00:40:20 +00:00
Dmitry Novik
366b105c27 Fix data race 2021-12-07 01:16:09 +03:00
Dmitry Novik
697ab52387 Add logs to GlobalOvercommitTracker 2021-12-07 01:06:00 +03:00
Dmitry Novik
12101d82aa Fix overcommit ratio comparison and race condition 2021-12-06 21:34:52 +03:00
msaf1980
303552f515
graphite: split tagged/plain rollup rules (for merges perfomance) 2021-12-06 16:49:08 +01:00
lgbo-ustc
3c56a5deac refector 
1. Make the lru cache pilicy in RemoteReadBufferCache into Common/UnreleasableLRUCache.h
2. If try to use local cached file fail, the RemoteReadBuffer::create() return the original ReadBuffer
3. Remove LocalFileReader
4. RemoteCacheController does not hold a context now, and the download process starts in RemoteReadBufferCache
   once a new RemoteCacheController is create successly
5. Make an abstract class RemoteFileMetaDataBase for descripting a remote file meta data.
6. Remote file meta data need to  a version, can cache multi-version for a file now
 2021-12-06 16:17:32 +08:00
Azat Khuzhin
abab7108e4 Fix QueryProfiler building under osx 
Fixes: #32165
 2021-12-04 09:19:34 +03:00
Azat Khuzhin
22a74dc68d Make jemalloc under osx even more reliable 
- explicitly call zone_register() again
- explicitly call malloc(free()) to initialize jemalloc()
 2021-12-04 09:19:34 +03:00
Azat Khuzhin
0b2de32228 Fix QueryProfiler (query_profiler_{cpu,real}_time_period_ns) reset 
Even after timer_delete() the signal can be fired.
Reproducer:

    $ clickhouse-server & # with configured trace_log
    $ clickhouse-benchmark -c2 --query 'select * from numbers(1e6)' --query_profiler_cpu_time_period_ns=1 &
    ...
    2021.12.02 14:28:01.320288 [ 24885 ] {} <Debug> TCPHandler: Processed in 177.055205644 sec.
    User defined signal 2

CI failures:
- https://s3.amazonaws.com/clickhouse-test-reports/32067/8dbc7a8dae17090a18778f29629d8746a1bb9b72/stateful_tests__debug__actions_.html
- https://s3.amazonaws.com/clickhouse-test-reports/32064/c07450a7dce363b7a4c5ca3ab0e833c25e3d46c0/stateful_tests__debug__actions_.html

Fix this by do not reset the signal back, and introduce a flag to ignore
signals after disabling the timer.

Fixes: #31740
 2021-12-03 01:33:08 +03:00
taiyang-li
2f4e7e1d4e merge master 2021-12-02 19:48:21 +08:00
Dmitry Novik
2ab4fc734d
Merge pull request #31740 from azat/fix-disabling-query-profiler 
Fix disabling query profiler
 2021-12-01 18:15:50 +03:00
Anton Popov
6f4d9a53b2 Merge remote-tracking branch 'origin/sparse-serialization' into HEAD 2021-12-01 15:54:33 +03:00
alexey-milovidov
96ec92c7cd
Merge pull request #31155 from ClickHouse/incremental-profile-events 
Send incremental profile events to client
 2021-12-01 04:04:41 +03:00
Kseniia Sumarokova
3c348a2998
Merge pull request #8331 from Vxider/window-view 
Support for Stream Processing
 2021-11-30 18:20:12 +03:00
alexey-milovidov
e943be340a
Merge pull request #31566 from arenadata/ADQM-388 
get Build ID via Section headers first
 2021-11-30 05:52:11 +03:00
Kseniia Sumarokova
c7a8ad69c4
Merge pull request #31825 from azat/query_views_log-test 
Cover query_views_log
 2021-11-29 16:10:56 +03:00
taiyang-li
df68b5b1a1
Update ErrorCodes.cpp 
remove blank
 2021-11-29 10:46:47 +08:00
taiyang-li
d35e2a1c83
Merge branch 'master' into hive_table 2021-11-26 11:44:50 +08:00
Azat Khuzhin
ee0258e128 Add sanity check into ThreadStatus::disableProfiling() 2021-11-26 01:13:55 +03:00
Azat Khuzhin
2840405c8a Fix typo in ThreadStatus::query_profiler_enabled 2021-11-26 01:13:55 +03:00
kssenii
223048d395 Merge branch 'master' of github.com:ClickHouse/ClickHouse into vxider-window-view 2021-11-26 01:06:06 +03:00
Alexey Milovidov
85cec911be Fix strange code in TypeList 2021-11-25 23:55:02 +03:00
taiyang-li
58c0357436
Update ErrorCodes.cpp 
fix typo
 2021-11-25 19:16:43 +08:00
Kruglov Pavel
f28130193a
Merge branch 'master' into fix-whole-text-serialization 2021-11-25 14:00:20 +03:00
taiyang-li
72f60cceb9
Merge branch 'master' into hive_table 2021-11-25 17:33:26 +08:00
Kseniia Sumarokova
93cf66df12
Merge pull request #30936 from kssenii/seekable-read-buffers 
Reduce memory usage for some formats when reading with s3/url/hdfs
 2021-11-25 11:19:24 +03:00
Alexey Milovidov
2373456866 Fix build 2021-11-25 04:18:54 +03:00
Azat Khuzhin
df67af0c88 Fix disabling query profiler 
Before there was two incorrect checks:
- timer_id != nullptr, but first timer is 0x0 and the comparison does
  not work
- previous_handler should also be reseted regardless it was nullptr or
  not, plus it also incorrectly uses 'sigaction * previous_handler'
  before.

Which leads to permanent query profiler enabled for particular thread,
this is especially visible if you have query profiler disabled and you
are running some query with profiler enabled, i.e.:

  select 1 settings query_profiler_real_time_period_ns=1

After, QueryPipelineEx will eat lots of CPU due because of profiler.
 2021-11-24 23:28:20 +03:00
Alexey Milovidov
21f112b30e Merge branch 'better-crosscompiling-freebsd' into hermetic-builds 2021-11-24 22:43:30 +03:00
Alexey Milovidov
f1b8e2a2c8 Fix build 2021-11-24 22:43:12 +03:00
Alexey Milovidov
8e4c7fccc3 Fix build 2021-11-24 20:19:52 +03:00
Alexey Milovidov
438619389f Fix build 2021-11-24 16:53:51 +03:00
mergify[bot]
2ab091a85b
Merge branch 'master' into fix-whole-text-serialization 2021-11-24 12:44:53 +00:00
Alexey Milovidov
ecf7822153 Fix build 2021-11-24 15:28:54 +03:00
alesapin
646cf38213
Merge pull request #28981 from JackyWoo/add_4_letter_words_commands 
Add four letter commands to keeper
 2021-11-24 13:32:36 +03:00
taiyang-li
89dcef69d5 merge master 2021-11-24 14:38:04 +08:00
Alexey Milovidov
e59e833cc6 Fix build 2021-11-24 03:11:22 +03:00
Ilya Golshtein
ae15126d20 just more verbose comment 2021-11-23 12:56:14 +00:00
Ilya Golshtein
e27ce34cf2 get Build ID via Section headers first 2021-11-23 12:56:14 +00:00
alexey-milovidov
341f705419
Merge pull request #30599 from ClickHouse/zlib-ng-own-cmake 
Use our own CMakeLists for `zlib-ng`, `cassandra`, `mariadb-connector-c` and `xz`, `re2`, `sentry`, `gsasl`, `arrow`, `protobuf`.
 2021-11-23 13:33:29 +03:00
tavplubix
7a43a87f5b
Use shard and replica name from Replicated database arguments (#31488) 
* fix another issue

* use shard and replica name from Replicated database

* fix

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-11-23 12:41:54 +03:00
lgbo
996d7125c0
Merge branch 'master' into hive_table 2021-11-23 10:19:02 +08:00
Vitaly Baranov
8eae84b8cf
Merge pull request #31607 from vitlibar/stop-reloading-all-configs-on-shutdown-early 
Stop all periodic reloading of all the configuration files on shutdown earlier
 2021-11-22 22:45:14 +03:00
alesapin
35de9e42d9
Merge branch 'master' into add_4_letter_words_commands 2021-11-22 18:02:02 +03:00
Anton Popov
ccd78e3838 Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-22 17:19:35 +03:00
avogar
9bee988e16 Fix tests 2021-11-22 14:26:48 +03:00
Vitaly Baranov
2f8c829395 Stop all periodic reloading of all the configuration files on shutdown earlier. 2021-11-22 12:26:05 +03:00
taiyang-li
d22854e82b add compile swiych USE_HIVE 2021-11-22 10:52:10 +08:00
Alexey Milovidov
c5fcef03c3 Remove trash 2021-11-21 18:12:05 +03:00
Alexey Milovidov
6246cec409 Fix build on AArch64 2021-11-21 14:30:04 +03:00
Alexander Tokmakov
856502fa81 log long operations in Keeper 2021-11-20 20:39:31 +03:00
kssenii
ff969b4605 Merge branch 'master' of github.com:ClickHouse/ClickHouse into seekable-read-buffers 2021-11-20 15:03:13 +03:00
kssenii
29978f9707 Merge branch 'master' of github.com:ClickHouse/ClickHouse into vxider-window-view 2021-11-20 12:13:39 +03:00
alesapin
3f8f08e816 Slightly better 2021-11-19 12:30:58 +03:00
taiyang-li
e8644807fe merge master and solve conflict 2021-11-19 15:01:58 +08:00
kssenii
6a39d871b2 Merge branch 'master' of github.com:ClickHouse/ClickHouse into vxider-window-view 2021-11-18 17:29:11 +03:00
Nikolay Degterinsky
f2060d48ac Merge branch 'master' 2021-11-17 22:23:41 +00:00
alesapin
45cd535839 Fix my own wrong command usage 2021-11-17 15:44:07 +03:00
alesapin
5f3eca4cd1 Merge branch 'master' into JackyWoo_add_4_letter_words_commands 2021-11-17 12:59:20 +03:00