thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
81,252 Commits 9,319 Branches 1,783 Tags 2.1 GiB
eff85d3352
Commit Graph

2707 Commits

Author SHA1 Message Date
lgbo-ustc
0d474069b5 add a new test case 2021-12-29 15:38:58 +08:00
lgbo-ustc
5f85f77266 add a new test case 2021-12-29 15:34:20 +08:00
lgbo-ustc
5a419a356b fix a bug 2021-12-29 15:29:59 +08:00
lgbo-ustc
59cbd76880 Add LRUResourceCache 
1. add LRUResourceCache for managing resource cache in lru policy
2. rollback LRUCache to the original version
3. add remove() in LRUCache
4. add unit tests for LRUResourceCache and LRUCache
 2021-12-29 15:25:33 +08:00
Azat Khuzhin
a23bb79479 Fix parsing symbols from resources (for shared builds) 
In case of shared builds the symbol can be represented in multiple
shared objects, and so if one SO contains _end symbol firstly, then it
it's address can subtracted from the symbol from another SO, and this
give overflow.

Here is an example of such symbol:

    $ llvm-readelf -a programs/server/libclickhouse-server-libd.so | fgrep schemata_sql
       552: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size
      1271: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
      3627: 000000000010b480     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start
      5822: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
      5828: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size
      5831: 000000000010b480     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start

    $ llvm-readelf -W -a src/TableFunctions/libclickhouse_table_functionsd.* | fgrep schemata_sql
      1709: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
      5107: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size
      6090: 00000000001fc1b8     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start
     10975: 00000000001fc1b8     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start
     10976: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
     10977: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size

To address this, add base address of the Elf into the resources to
compare with.

Follow-up for: #33134 (cc @alexey-milovidov)
 2021-12-29 09:56:59 +03:00
Alexey Milovidov
d456c5ddda Always display resource usage in client 2021-12-29 01:26:58 +03:00
Alexey Milovidov
e8e19b8b0d Always display resource usage in client 2021-12-29 01:20:34 +03:00
Maksim Kita
d1db3c9f42 Updated ShellCommandSource 2021-12-28 22:55:31 +03:00
Maksim Kita
5590cfa3aa Updated executable function integration tests 2021-12-28 22:55:30 +03:00
taiyang-li
aa97634d9b merge master 2021-12-28 18:38:50 +08:00
lgbo-ustc
1ac697cad2 some modifications 2021-12-28 11:26:39 +08:00
lgbo-ustc
413fb290bb remove comments 2021-12-28 09:19:21 +08:00
lgbo-ustc
26e61061b0 update setImpl() 2021-12-27 20:46:51 +08:00
lgbo-ustc
922dd40c05 optimization 2021-12-27 20:33:25 +08:00
Vitaly Baranov
f80a3e3247
Merge pull request #30549 from aiven/kmichel-server-reload 
Start/stop servers when `listen_host`/`*_port` changes
 2021-12-27 14:51:00 +03:00
lgbo-ustc
f86e3ef1ae update the evict action when overwrite a key 2021-12-27 18:12:40 +08:00
lgbo-ustc
54b2a42530 update variable names 2021-12-27 18:02:02 +08:00
lgbo-ustc
3367755dcc update comment 2021-12-27 16:21:10 +08:00
lgbo-ustc
ef1d7142f5 remove getOrTrySet 2021-12-27 16:12:39 +08:00
lgbo-ustc
0a39273fee fixed code style 2021-12-27 15:47:22 +08:00
taiyang-li
2d6eb40879 fix style 2021-12-27 15:31:24 +08:00
taiyang-li
dd882085d0 Merge branch 'hive_table' of https://github.com/bigo-sg/ClickHouse into bigo_hive_table 2021-12-27 15:14:43 +08:00
taiyang-li
9036b18c2f merge master 2021-12-27 15:12:48 +08:00
lgbo-ustc
078521496a modifications based on pr review 2021-12-27 15:10:48 +08:00
lgbo-ustc
51e43ae619 some modification in LRUCache 2021-12-27 10:20:07 +08:00
Alexey Milovidov
bbd8e055a1 Fix clang-tidy 2021-12-24 18:25:59 +03:00
Alexey Milovidov
c8b8cc74fc Fix clang-tidy 2021-12-24 18:15:04 +03:00
Alexey Milovidov
b7799dcc45 Remove even more useless code 2021-12-24 12:25:27 +03:00
Alexey Milovidov
ce5d63510f Remove even more useless code 2021-12-24 12:25:27 +03:00
Alexey Milovidov
edb50b5844 Remove unused code 2021-12-24 12:25:27 +03:00
Alexey Milovidov
a96b4d50d6 Fix error 2021-12-24 12:25:27 +03:00
Alexey Milovidov
003ee51c6d Fix buildId 2021-12-24 12:25:27 +03:00
Alexey Milovidov
ae61cf66b5 Fix resources 2021-12-24 12:25:27 +03:00
Alexey Milovidov
29d28c531f Move code around to avoid dlsym on Musl 2021-12-24 12:25:27 +03:00
Alexey Milovidov
d6ece47b7f Allow vfork 2021-12-24 12:25:27 +03:00
Kevin Michel
ffc1fca296
Start/stop servers when listen_host/*_port changes 
This allows starting and stopping separately each protocol server
without restarting ClickHouse.

This also allows adding or removing `listen_host` entries, which
start and stops servers for all enabled ports.

When stopping a server, the listening socket is immediately closed
(and available for another server).

Protocols with persistent connections try to wait for any currently
running query to finish before closing the connection, but idle
connection are closed quickly (depending on how often the protocol
is polled).

An extra ProfileEvent is added, `MainConfigLoads`, it is
incremented every time the configuration is reloaded. This helps
when trying to assess whether the new configuration was applied.
 2021-12-24 08:26:02 +01:00
lgbo-ustc
5a9bd7fd41 fixe typos 2021-12-23 18:13:56 +08:00
lgbo-ustc
e25b1f7361 update comments 2021-12-23 18:05:52 +08:00
lgbo-ustc
fe4ef69fc7 remove unused header 2021-12-23 18:01:58 +08:00
lgbo-ustc
59bc87e409 remove unused errorcode 2021-12-23 17:53:19 +08:00
lgbo-ustc
9fdb061b7e Add evict policy in LRUCache 2021-12-23 17:36:41 +08:00
lgbo-ustc
abe79fb1c2 move Exception in LRUCache::set() 2021-12-23 16:33:43 +08:00
lgbo-ustc
d29a4d0527 add trySet() in LRUCache 2021-12-23 16:16:33 +08:00
lgbo-ustc
f0b22d2882 fixed code-style 2021-12-23 15:15:39 +08:00
alexey-milovidov
12fb830181
Merge pull request #33061 from ClickHouse/merge-33024 
Merge #33024
 2021-12-23 07:50:11 +03:00
lgbo-ustc
c3736859ac update some comments 2021-12-23 11:56:54 +08:00
lgbo-ustc
d205f9ecab codes refactor 2021-12-23 11:50:26 +08:00
Alexey Milovidov
53c83e4813 Merge branch 'Issue77' of github.com:DevTeamBK/ClickHouse into merge-33025 2021-12-23 01:36:46 +03:00
Alexey Milovidov
3e059a095b Corrections 2021-12-23 01:25:35 +03:00
Alexey Milovidov
ba77993943 Merge branch 'Issue20' of github.com:DevTeamBK/ClickHouse into merge-33024 2021-12-23 01:23:47 +03:00
Rajkumar
a75d3d1c83 Review comments 2021-12-22 13:39:45 -08:00
Rajkumar
afe9e6a325 Addressing review comment 2021-12-22 11:42:22 -08:00
Rajkumar
852889f995 Addressing review comment 2021-12-22 08:54:33 -08:00
Maksim Kita
5d4b0f8f1a
Merge pull request #33026 from ClickHouse/remove-old-code-config-processor 
Remove obsolete code from ConfigProcessor.
 2021-12-22 13:34:53 +03:00
lgbo-ustc
2d226e287a update comment in LRUCache::set 2021-12-22 15:11:39 +08:00
Kseniia Sumarokova
484a4d62dd
Merge pull request #32835 from kssenii/fising-s3 
Fix optimization with lazy seek for async reads from remote fs
 2021-12-22 09:38:23 +03:00
Meena-Renganathan
91036e729c Fixed the indentation 
Signed-off-by: Meena-Renganathan <Meena.Renganathan@ibm.com>
 2021-12-21 16:07:24 -08:00
Boris Kuschel
c452b3fad9 Modifed the code to handle the sysconf() fialure 
Signed-off-by: Boris Kuschel <Boris.Kuschel@ibm.com>
 2021-12-21 18:33:00 -05:00
Alexey Milovidov
088d14b885 Remove obsolete code from ConfigProcessor. 2021-12-22 02:13:31 +03:00
Boris Kuschel
68de95afc9 Native Format Parsing - Integer Overflow Resizing Arrays 
Signed-off-by: Boris Kuschel <Boris.Kuschel@ibm.com>
 2021-12-21 18:10:51 -05:00
taiyang-li
2597925724 merge master 2021-12-21 15:55:39 +08:00
lgbo-ustc
44c0f0e680 rewrite comments 2021-12-21 10:59:23 +08:00
lgbo-ustc
fb1214efeb fixed code style 2021-12-21 10:56:57 +08:00
lgbo-ustc
eb9e9bd47a fixed a bug 2021-12-21 10:50:25 +08:00
lgbo-ustc
2e2354be40 1. add tryDel and getOrTrySet in LRUCache 
2. fixed bugs in LRUCache for evicting elements
 2021-12-21 10:39:59 +08:00
lgbo-ustc
2c67516432 1. fixed code style 
2. fixed bugs in lru cache
 2021-12-21 09:44:04 +08:00
alexey-milovidov
04fb8aeaa5
Merge pull request #32946 from antonio2368/feature/hints-for-settings 
Hints for invalid settings in query and HTTP requests
 2021-12-21 00:52:42 +03:00
Maksim Kita
dd0d3de050
Merge pull request #32970 from kitaisreal/loops-remove-postfix-increment 
Loops remove postfix increment
 2021-12-20 19:51:07 +03:00
Antonio Andelic
bc52758e4d Explicitly default move/copy constructors 2021-12-20 17:05:08 +01:00
Maksim Kita
51477adf1b Updated additional cases 2021-12-20 15:55:07 +03:00
Azat Khuzhin
6aebc3e94c Do not loose ProfileEvents in case of thread destroyed before 
v2: drop std::move and add copy ctor for ProfileEvents::Counter::Snapshot
v2: remove std::move
 2021-12-19 23:43:05 +03:00
Azat Khuzhin
909ce68b5a Do not suppress exception in ThreadStatus::~ThreadStatus() 2021-12-19 23:43:05 +03:00
kssenii
ed42620530 Rename BlobStorage to AzureBlobStorage 2021-12-17 12:24:05 +03:00
taiyang-li
d033fc4c24 merge master and fix conflict 2021-12-17 15:11:21 +08:00
kssenii
ca283a4bb8 Fix optimization with lazy seek 2021-12-17 00:18:28 +03:00
alesapin
2753fe01a3 Merge branch 'master' into fix_special_build_check 2021-12-15 10:35:28 +03:00
alesapin
9974f10631 Fix clang tidy 2021-12-14 15:34:20 +03:00
lgbo-ustc
248e933fc4 Add test for local cache 2021-12-14 16:06:30 +08:00
Azat Khuzhin
a7dc6f309f Remove SparseHashMap.h 
It was added only for arcadia build, and used only in one place, no need
to have a separate typedef for it.
 2021-12-14 10:07:14 +03:00
Azat Khuzhin
e16891713d Fix sparse_hashed dict performance with sequential keys (wrong hash function) 
In #27152 the hash function for sparse_hash_map had been changed to
std::hash<> switch it back to DefaultHash<> (ClickHouse builtin), since
std::hash<> for numeric keys returns itself and this does not works
great with sparse_hash_map.

I've tried the example from #32480 and using some hash fixes the
performance of sparse_hashed layout.

Fixes: #32480

v2: Add comments for SparseHashMap
 2021-12-14 10:07:14 +03:00
taiyang-li
8234d1176f merge master 2021-12-14 10:39:21 +08:00
Kseniia Sumarokova
78ed383f15
Merge pull request #31505 from ContentSquare/azure_blob_storage_dependency 
Azure Blob Storage Disks
 2021-12-13 18:28:48 +03:00
李扬
8675086104
Merge branch 'master' into hive_table 2021-12-12 09:01:46 -06:00
Jakub Kuklis
60dd22dd4e Checking for success of file removal operation 2021-12-10 22:04:37 +01:00
Jakub Kuklis
43f420cba8 Adding a flag to use Azure Blob Storage 2021-12-10 22:04:36 +01:00
kssenii
f5a77fca39 Done 2021-12-09 14:44:25 +00:00
Nikita Mikhaylov
dbf5091016
Parallel reading from replicas (#29279) 2021-12-09 13:39:28 +03:00
Mikhail f. Shiryaev
03927f5fc6
Revert "graphite: split tagged/plain rollup rules (for merges perfomance)" 
This reverts commit 303552f515.
 2021-12-08 15:49:21 +01:00
lgbo
38020a33e4 some refinement 2021-12-08 10:07:04 +08:00
Mikhail f. Shiryaev
5fc20b3e6a
Merge pull request #25122 from msaf1980/rollup_rules_type 
GraphiteMergeTree Rollup rules type
 2021-12-07 10:18:08 +01:00
lgbo-ustc
54cc65ede8 fixed code style 2021-12-07 17:12:26 +08:00
lgbo-ustc
fcea033a07 fixed code style 2021-12-07 12:17:47 +08:00
lgbo-ustc
0c1733817b update LRUCache set() 2021-12-07 10:57:05 +08:00
msaf1980
303552f515
graphite: split tagged/plain rollup rules (for merges perfomance) 2021-12-06 16:49:08 +01:00
lgbo-ustc
3c56a5deac refector 
1. Make the lru cache pilicy in RemoteReadBufferCache into Common/UnreleasableLRUCache.h
2. If try to use local cached file fail, the RemoteReadBuffer::create() return the original ReadBuffer
3. Remove LocalFileReader
4. RemoteCacheController does not hold a context now, and the download process starts in RemoteReadBufferCache
   once a new RemoteCacheController is create successly
5. Make an abstract class RemoteFileMetaDataBase for descripting a remote file meta data.
6. Remote file meta data need to  a version, can cache multi-version for a file now
 2021-12-06 16:17:32 +08:00
Azat Khuzhin
abab7108e4 Fix QueryProfiler building under osx 
Fixes: #32165
 2021-12-04 09:19:34 +03:00
Azat Khuzhin
22a74dc68d Make jemalloc under osx even more reliable 
- explicitly call zone_register() again
- explicitly call malloc(free()) to initialize jemalloc()
 2021-12-04 09:19:34 +03:00
Azat Khuzhin
0b2de32228 Fix QueryProfiler (query_profiler_{cpu,real}_time_period_ns) reset 
Even after timer_delete() the signal can be fired.
Reproducer:

    $ clickhouse-server & # with configured trace_log
    $ clickhouse-benchmark -c2 --query 'select * from numbers(1e6)' --query_profiler_cpu_time_period_ns=1 &
    ...
    2021.12.02 14:28:01.320288 [ 24885 ] {} <Debug> TCPHandler: Processed in 177.055205644 sec.
    User defined signal 2

CI failures:
- https://s3.amazonaws.com/clickhouse-test-reports/32067/8dbc7a8dae17090a18778f29629d8746a1bb9b72/stateful_tests__debug__actions_.html
- https://s3.amazonaws.com/clickhouse-test-reports/32064/c07450a7dce363b7a4c5ca3ab0e833c25e3d46c0/stateful_tests__debug__actions_.html

Fix this by do not reset the signal back, and introduce a flag to ignore
signals after disabling the timer.

Fixes: #31740
 2021-12-03 01:33:08 +03:00
taiyang-li
2f4e7e1d4e merge master 2021-12-02 19:48:21 +08:00
Dmitry Novik
2ab4fc734d
Merge pull request #31740 from azat/fix-disabling-query-profiler 
Fix disabling query profiler
 2021-12-01 18:15:50 +03:00