millb
a00089f0b4
Renamed StorageOfAllowedURL -> RemoteHostFilter
2019-10-10 15:58:06 +03:00
millb
a97739f67f
Created class StorageOfAllowedURL
...
Created test for table function remote and remoteSecure
Created check allowed url in remote and remoteSecure
Fixed tests
2019-10-09 23:29:41 +03:00
millb
fec1c6deba
Fixed Error message and error code
2019-10-09 14:16:14 +03:00
Alexey Milovidov
9f8d562543
Add function "getMacro" #7239
2019-10-09 04:14:57 +03:00
alexey-milovidov
5c2d478543
Merge pull request #7212 from ClickHouse/aku/field-includes
...
Include Field.h and FieldVisitor.h into fewer files.
2019-10-08 22:40:30 +03:00
Alexander Kuzmenkov
2b30f2696c
Include Field.h in less files.
2019-10-08 16:23:24 +03:00
Azat Khuzhin
92c88248a1
Fix uniqCombined() result for >UINT_MAX values (return UInt64 to avoid overflow)
...
uniqCombined() return type is UInt64, but uniqCombined() uses
CombinedCardinalityEstimator, and CombinedCardinalityEstimator::size()
return type is UInt32, while the underlying HyperLogLog::size() is
UInt64.
So after this patch uniqCombined() can be used for >UINT_MAX values, the
outcome is not ideal (#2073 ) but at least sane.
2019-10-07 21:38:58 +03:00
Alexander Kuzmenkov
367c537106
Do not include Field.h everywhere through SettingsCommon.h.
...
Move the methods of SettingsCollection<> that are dependent on Field to
a separate file, and include it once for each instantiation. This allows
to work on Field without always recompiling the entire project.
2019-10-07 20:23:05 +03:00
proller
594c535bd4
Build fixes ( #7063 )
...
* Build fixes
* fix
* Fix
* fix
* Fix install(export..
* Freebsd fixes
* Freebsd fixes
* Fix warning
* fix
* More PRIVATE linking
* Fix review
* Timmy
* fix
* fix
2019-10-05 22:25:31 +03:00
Alexander Kuzmenkov
8e245d4e8c
Merge pull request #7135 from ClickHouse/aku/msan-field
...
Fix some MemorySanitizer warnings for Fields.
2019-10-03 10:33:44 +03:00
chertus
3b0d870804
rename function
2019-10-02 15:06:34 +03:00
chertus
c46cc3ff1a
refactoring: remove copy-paste
2019-10-01 21:51:33 +03:00
millb
9246f258e8
Created ability to configure allowed URL in config.xml without tests.
2019-09-30 18:29:05 +03:00
Alexander Kuzmenkov
ec2d9a083c
Remove excessive includes.
2019-09-30 15:31:14 +03:00
alexey-milovidov
a2fc7e2e9e
Merge pull request #7093 from millb/merges_counter_metric
...
Add merge metric
2019-09-26 21:49:43 +03:00
alexey-milovidov
87a6d3b222
Update ProfileEvents.cpp
2019-09-26 21:49:30 +03:00
millb
259be751ee
Fixed metric name.
...
Add metric description.
Add test.
2019-09-26 15:06:52 +03:00
millb
97a2ae176b
Add CountOfMerges metric
2019-09-25 13:37:43 +03:00
Alexey Milovidov
ff07ffbbae
Merge branch 'merging_redis' of https://github.com/CurtizJ/ClickHouse into CurtizJ-merging_redis
2019-09-25 02:27:05 +03:00
Alexander Kuzmenkov
b949cc232b
Mark lookupResultGetKey/Mapped functions ALWAYS_INLINE.
...
They only do pointer arithmetics, so it makes sense to always inline
them.
2019-09-24 22:03:10 +03:00
Alexander Kuzmenkov
f3bde19b74
Do not use iterators in find() and emplace() methods of hash tables.
...
Instead, these methods return a pointer to the required data as they are
stored inside the hash table. The caller uses overloaded functions to
get the key and "mapped" values from this pointer. Such an interface
avoids the need for constructing iterator-like wrapper objects, which is
especially important for compound hash tables such as the future
StringHashMap.
2019-09-24 17:44:35 +03:00
Alexey Milovidov
dcd40d1d07
Better log messages about disks
2019-09-24 03:45:40 +03:00
alexey-milovidov
2f195ddb03
Merge pull request #7028 from abyss7/issue-6459
...
Implement cross-compilation support Linux → Darwin
2019-09-24 02:20:17 +03:00
alexey-milovidov
e9bfe99dd8
Merge pull request #7072 from silviucpp/silviu-osx-stack
...
Get proper stack address on osx
2019-09-24 02:09:24 +03:00
alexey-milovidov
e2cc551b22
Merge pull request #6914 from maqroll/master
...
+ Redirect URL Storage
2019-09-24 02:07:06 +03:00
Silviu Caragea
d187b5ed42
Get proper stack address on osx
2019-09-23 22:47:12 +03:00
Alexey Milovidov
8579c26efb
Changed repository URL
2019-09-23 19:18:19 +03:00
Ivan Lezhankin
4b143c3e0f
Merge remote-tracking branch 'upstream/master' into issue-6459
2019-09-23 16:07:19 +03:00
maqroll
a6e7519f9d
Merge remote-tracking branch 'upstream/master'
2019-09-23 07:55:07 +00:00
alexey-milovidov
d446205eec
Update QueryProfiler.cpp
2019-09-22 04:53:19 +03:00
Silviu Caragea
1fbd2fb6a8
Fix osx build
2019-09-21 21:30:01 +03:00
Ivan Lezhankin
9e46118edd
Merge remote-tracking branch 'upstream/master' into issue-6459
2019-09-20 21:25:55 +03:00
Ivan Lezhankin
8cb0e58bc7
Fix build for all targets
2019-09-20 21:13:51 +03:00
Ivan Lezhankin
967e00a8da
It works!
...
if compiled with:
CC=clang CXX=clang++ cmake -Wno-dev . -Bbuild -GNinja -DCMAKE_AR:FILEPATH=x86_64-apple-darwin-ar -DCMAKE_RANLIB:FILEPATH=x86_64-apple-darwin-ranlib -DCMAKE_SYSTEM_NAME=Darwin -DSDK_PATH=MacOSX10.14.sdk -DLINKER_NAME=x86_64-apple-darwin-ld -DUSE_SNAPPY=OFF -DENABLE_SSL=OFF -DENABLE_PROTOBUF=OFF -DENABLE_PARQUET=OFF -DENABLE_READLINE=OFF -DENABLE_ICU=OFF -DENABLE_FASTOPS=OFF
2019-09-20 20:09:22 +03:00
alexey-milovidov
7325f4a77b
Merge pull request #6974 from yandex/aku/mmap-linux
...
Use MAP_POPULATE only on Linux.
2019-09-19 01:58:40 +03:00
alexey-milovidov
bc1ecb0be5
Merge pull request #6951 from yandex/aku/alloc-continue
...
Make the quadratic Arena::allocContinue less bad.
2019-09-19 01:57:26 +03:00
root
ea5480fd72
merge
...
Committer: maqroll <loteroc@gmail.com>
2019-09-18 13:13:40 +00:00
Alexander Kuzmenkov
e3a9863260
Use MAP_POPULATE only on Linux.
2019-09-18 15:32:23 +03:00
Alexander Kuzmenkov
e1898ca89a
Make the quadratic Arena::allocContinue less bad.
...
Scenarios that use Arena::allocContinue may waste quadratically many
memory and perform quadratically many copying, when the memory range
size reaches Arena's linear allocation threshold. To alleviate this,
make sure that the next memory chunk allocated by allocContinue is at
least linear_growth_threshold bytes bigger than the previous one, so
that we don't reallocate and copy that often.
2019-09-18 15:04:56 +03:00
CurtizJ
651f5b0e9f
merging with master
2019-09-18 13:21:10 +03:00
CurtizJ
79a08ef8ea
Merge remote-tracking branch 'upstream/master' into merging_redis
2019-09-18 12:58:22 +03:00
alexey-milovidov
fe9e110a6a
Merge pull request #6965 from azat-archive/sparsehash-c11
...
Replace libsparsehash with sparsehash-c11
2019-09-18 12:46:13 +03:00
Azat Khuzhin
780341060f
Replace libsparsehash with sparsehash-c11
...
- use sparsehash-c11 over libsparsehash
- fix typos in find_sparsehash and users of the vars (s/SPARCE/SPARSE/)
- drop libsparsehash-dev from docker images (but keep for unbunlded build)
- use ::google over GOOGLE_NAMESPACE
2019-09-18 00:08:20 +03:00
CurtizJ
7ccf04440a
better code in Redis external dictionary
2019-09-17 20:57:48 +03:00
CurtizJ
4480e97f9f
fix build
2019-09-17 17:16:07 +03:00
akuzm
75dc7d331e
Merge pull request #6937 from amosbird/ubmemfix
...
In hash tables, properly initialize zero key cell.
2019-09-17 14:57:38 +03:00
Amos Bird
1cf47da60d
ubsan/memsan fix.
2019-09-16 21:20:17 +08:00
Ivan Lezhankin
01fdb802d4
Useful changes
2019-09-15 13:35:12 +03:00
Vitaly Baranov
0fb234c63e
Merge branch 'master' into dict_allow_databases
2019-09-15 01:51:44 +03:00
akuzm
e765733a26
Merge pull request #6925 from yandex/aku/msan
...
Some MemorySanitizer fixes
2019-09-13 20:26:48 +03:00
Ivan Lezhankin
5b60053ef7
[WIP]
2019-09-13 17:56:27 +03:00
Ivan Lezhankin
e692a88196
CMake now works
...
Example: -DCMAKE_SYSTEM_NAME=Darwin -DSDK_PATH=${HOME}/stuff/MacOSX10.12.sdk -DLINKER_NAME=${HOME}/.local/bin/ld64
2019-09-13 17:56:27 +03:00
Ivan Lezhankin
1b5192706e
Some fixes for OS X build.
2019-09-13 17:56:26 +03:00
Alexander Kuzmenkov
f4da0399c0
Some MemorySanitizer fixes.
2019-09-13 14:49:29 +03:00
alesapin
02cff38fef
Merge with master
2019-09-13 13:58:23 +03:00
CurtizJ
fa416dc941
fix Redis dictionary
2019-09-12 17:48:28 +03:00
alexey-milovidov
c5cf4fa4f4
Merge pull request #6878 from azat-archive/build--fixes
...
Trivial build fixes (mostly for -DSPLIT_SHARED_LIBRARIES=ON)
2019-09-12 03:25:19 +03:00
Amos Bird
93c0880ca7
Fix non-ssl build.
2019-09-12 04:20:27 +08:00
l
adb4a58092
Style checks & security considerations added to setting description.
2019-09-11 22:18:22 +02:00
alesapin
b0fce77c3d
Fix path to data on disk
2019-09-11 20:17:10 +03:00
l
cdbcfc2c90
+ Redirect
2019-09-11 18:39:30 +02:00
alexey-milovidov
3ca084ec57
Merge pull request #6854 from Akazz/rwlock-cumulative-update-1
...
Rwlock sanitary update 1: removed weak_ptrs from implementation + added strong exception safety guarantee
2019-09-11 15:40:28 +03:00
Guillaume Tassery
5f1621faa2
Add database right for dictionaries
2019-09-11 13:14:55 +02:00
alesapin
0b8aec469f
Make absolute path on disks start from /clickhouse/data, fix freeze and rename, add tests
2019-09-11 13:57:32 +03:00
Azat Khuzhin
6c646a6978
Missing <chrono> for gtest_sensitive_data_masker
...
I guess that <chrono> had been included somewhere before in primary env.
2019-09-11 02:04:11 +03:00
alexey-milovidov
254102c5bd
Merge pull request #6602 from amosbird/object-library
...
Split libdbms.so using object library
2019-09-11 02:00:13 +03:00
Akazz
7b65dfb867
Build fixed
2019-09-11 01:21:08 +03:00
Akazz
40a9495267
Some formatting + minor change
2019-09-10 22:58:04 +03:00
Amos Bird
8f28a36c17
Use internal iteration over hash tables in Aggregator.
...
For compound hash tables such as the future StringHashMap, an
iterator-based API might be inefficient for iterating over a table or
for merging two tables, because:
1) the key has to be converted to a general format from a from a
component-specific format, which may differ between the components;
2) the information about the component of the compound hash table to
which the value belongs is lost, and has to be recalculated if the
value is reinserted.
A more efficient approach is to use internal iteration, that is,
map-like functions, which avoids unnecessary conversions when iterating,
and allows to use an efficient component-wise approach when merging.
2019-09-10 21:52:53 +03:00
Amos Bird
a1310ffbe6
Split libdbms.so using object library
...
Now the linking time of incremental builds are around 1-2 seconds
2019-09-11 01:56:01 +08:00
Alexander Kuzmenkov
160e0faf77
Do not use iterator::getValue() in Aggregator.
...
Use separate key and "mapped" value references instead. This is
important for hash tables that do not store the key/"mapped" pair
directly, and cannot provide this interface without some runtime
overhead.
2019-09-10 20:12:30 +03:00
Alexander Kuzmenkov
8e9a8584e0
Do not provide mutable key access in hash map cells.
...
We don't need it anymore after we changed the hash table key memory
management to use callbacks. Removing this interface is important for
hash maps that do not store the key, such as FixedHashMap or the
prospective compound StringHashMap.
2019-09-10 20:12:30 +03:00
akazz
33f462d011
Dropped some garbage lines
2019-09-10 15:37:14 +03:00
akazz
4f71fe715c
Strong exception safety guarantee:
...
Based on std::list<>::emplace_back() and std::unordered_map<>::emplace()
provide strong exception safety, RWLockImpl is now changed to provide
the same level of exception safety.
2019-09-10 15:33:45 +03:00
alesapin
193049293a
More comments, more readable code
2019-09-10 14:21:59 +03:00
alesapin
4702e40ed7
Fix bug with b_free and small refactoring
2019-09-10 11:56:27 +03:00
alesapin
1bc0aaeb35
Less code, more comments
2019-09-09 17:20:07 +03:00
alesapin
9f88baebb9
More constants, better tests
2019-09-09 16:50:19 +03:00
alesapin
ef08d6f10b
More tests
2019-09-09 15:28:28 +03:00
alesapin
27419c965e
Merge with master
2019-09-09 14:13:36 +03:00
CurtizJ
3b3657ff1e
Merge branch 'add-ext-dict-redis' of https://github.com/comunodi/ClickHouse into merging_redis
2019-09-09 13:51:50 +03:00
alexey-milovidov
8389200229
Merge pull request #6832 from CurtizJ/merging_input_function
...
Merging #5450 (table function input).
2019-09-07 20:16:45 +03:00
alexey-milovidov
25de2e12a2
Merge pull request #6399 from Enmk/replica_recovery_interval
...
Improvements for failover of Distributed queries
2019-09-07 03:21:30 +03:00
alexey-milovidov
b7cbd33886
Merge pull request #6729 from yandex/aku/key-holder
...
Key memory management for compound hash tables.
2019-09-07 03:09:18 +03:00
alexey-milovidov
b3e30d51c3
Update HashTableKeyHolder.h
2019-09-07 03:09:07 +03:00
alexey-milovidov
46e7b4f925
Merge pull request #6853 from yandex/pipe-capacity-very-old-kernels
...
Returned support for very old Linux kernels (that lack of F_GETPIPE_SZ fcntl)
2019-09-07 03:01:42 +03:00
alexey-milovidov
00edf4fac3
Update HashTable.h
2019-09-06 21:19:50 +03:00
alexey-milovidov
805bd31641
Update dbms/src/Common/HashTable/HashTable.h
...
Co-Authored-By: akuzm <36882414+akuzm@users.noreply.github.com>
2019-09-06 21:12:31 +03:00
filimonov
526afd44ed
Make SensitiveDataMasker a singleton. ( #6810 )
...
Among other things, it is used to filter logs, which are being written even after the global server context is deinitialized, so we can't keep masker there.
2019-09-06 20:48:27 +03:00
alexey-milovidov
71ed9477dc
Update FixedHashTable.h
2019-09-06 20:20:19 +03:00
Alexey Milovidov
1c3fab8f2c
Returned support for very old Linux kernels (that lack of F_GETPIPE_SZ fcntl)
2019-09-06 20:10:28 +03:00
akazz
0f9b91a663
Cosmetic stuff
2019-09-06 18:45:49 +03:00
akazz
e7235cdcfd
Minor changes
2019-09-06 18:13:22 +03:00
alesapin
cfd753a1f3
Add test and found bugs
2019-09-06 18:09:20 +03:00
Alexander Kuzmenkov
cacf6d3282
Key memory management for compound hash tables.
...
Some aggregation methods initially emplace a temporary StringRef key
into a hash table. Then, if the key was not seen before, they make a
persistent copy of the key and update the hash table with it. This
approach is not suitable for compound hash tables, because the logic of
when the persistent key is needed is more complex, and is contained
within the hash table itself.
In this commit, we switch to managing key memory with callbacks passed
to the hash table, that allow it to request a persistent copy of the key
if it is needed. This should be more appropriate for compound hash
tables.
This commit prepares for StringHashMap PR #5417 .
2019-09-06 18:05:28 +03:00
Alexander Kuzmenkov
05095111a1
Cosmetic change: remove redundant functions from FixedHashTable
2019-09-06 17:47:32 +03:00
akazz
13631d2da3
Removed some garbage
2019-09-06 16:58:37 +03:00
Anton Popov
69da4f30b3
Merge branch 'master' into merging_input_function
2019-09-06 16:20:00 +03:00
alesapin
ea8e543b1a
Add protocol version to cookies
2019-09-06 15:18:56 +03:00
alesapin
0c9a9dee1c
Merge with master
2019-09-06 11:53:32 +03:00
Olga Khvostikova
4129d8cf60
Merge pull request #6092 from yandex/globs-in-file-one-more-attempt
...
Globs in storage file and HDFS
2019-09-05 23:05:37 +03:00
Olga Khvostikova
f90642ce23
Merge pull request #6610 from CurtizJ/merging_with_ties_and_with_fill
...
WITH TIES modifier for LIMIT and WITH FILL modifier for ORDER BY. (continuation of #5069 )
2019-09-05 22:45:48 +03:00
akazz
32c77cd933
Dropped weak_ptrs!
2019-09-05 21:09:33 +03:00
akazz
9b55e66755
WIP - About to give up storing weak_ptrs!
2019-09-05 18:19:36 +03:00
akazz
ab6ae7c82b
Refactored Clients -> Referers
2019-09-05 18:14:17 +03:00
CurtizJ
b04cea7463
minor fixes of table function input
2019-09-05 16:17:01 +03:00
akazz
d374a6fb58
Better
2019-09-05 15:49:20 +03:00
akazz
838957f0e2
Cosmetics
2019-09-05 15:45:44 +03:00
akazz
54b4db36eb
Changed FastPath + better diagnostics
2019-09-05 15:20:10 +03:00
alesapin
b71faee8cd
Merge branch 'master' into ObjatieGroba-feature_multiple_disks
2019-09-05 13:19:05 +03:00
stavrolia
5d6959173c
Improvement
2019-09-04 22:55:56 +03:00
Alexey Milovidov
f2c2a2a9ea
Fixed unit test
2019-09-04 19:13:05 +03:00
alesapin
13bbae5860
Remove trash
2019-09-04 19:00:20 +03:00
alesapin
3ffa9fa972
Resolve conflict
2019-09-04 15:44:12 +03:00
alesapin
8b6abab486
Merge branch 'master' into ObjatieGroba-feature_multiple_disks
2019-09-04 15:41:27 +03:00
Alexey Milovidov
843f8303b0
Minor modifications
2019-09-03 23:27:46 +03:00
Alexey Milovidov
9959e8d3e1
Fixed build
2019-09-03 23:07:27 +03:00
alexey-milovidov
b2e80b0cd9
Merge branch 'master' into query_masking
2019-09-03 22:58:15 +03:00
akuzm
357f59cc48
Merge pull request #6667 from yandex/aku/mmap-populate
...
Pre-fault pages when allocating memory with mmap().
2019-09-03 21:41:53 +03:00
alexey-milovidov
845a612b95
Merge pull request #6773 from infinivision/fix_issue_6575
...
fix DataTypeAggregateFunction deserialization
2019-09-03 21:19:41 +03:00
stavrolia
97ac979645
Add docs for hdfs and fix some review comments
2019-09-03 17:23:51 +03:00
CurtizJ
22badef1a2
Merge remote-tracking branch 'upstream/master' into merging_with_ties_and_with_fill
2019-09-03 13:47:18 +03:00
CurtizJ
6d821c4877
fix build
2019-09-03 13:07:31 +03:00
alesapin
f0c5ee5877
Merge with master
2019-09-02 20:34:18 +03:00
CurtizJ
4fea3e9688
style fixes
2019-09-02 20:18:44 +03:00
Alexander Kuzmenkov
fd00889981
Pre-fault mmapped buffers for hash tables.
...
Mmaped pages are copy-on-write references to the zero page, and are
faulted in on the first write. Hash table resize touches most of the
newly allocated buffer, so it's faster to pre-fault all pages after
allocation, instead of having the resize loop interrupted by page
faults.
2019-09-02 18:38:29 +03:00
Alexander Kuzmenkov
ce120ebc0d
Cosmetic change: Remove unused template parameters from Allocator.
2019-09-02 18:38:29 +03:00
CurtizJ
e122d54dcb
update comments
2019-09-02 18:22:27 +03:00
Alexey Milovidov
7bf411548f
Merge branch 'master' of github.com:yandex/ClickHouse into add-test-for-deadlock-system-tables
2019-09-02 18:04:15 +03:00
CurtizJ
de5e2b4d6b
Merge remote-tracking branch 'upstream/master' into merging_with_ties_and_with_fill
2019-09-02 17:34:49 +03:00
CurtizJ
da11fb86b7
implement 'with fill' modifier with processors
2019-09-02 17:29:51 +03:00
Vasily Nemkov
84fc4ba4c8
Replica recovery fixes
...
* Added a limit on how many errors can replica accumulate
* Decreased default error halving time to 60 seconds
* Made both configurable via settings
* Showing errors count and estimated recovery time for each replica in system.clusters
2019-09-02 17:26:25 +03:00
alexey-milovidov
0967ecf03b
Merge pull request #6727 from yandex/merging_template_format
...
Merging Template format
2019-09-02 17:25:43 +03:00
Alexey Milovidov
dcdd8c6e3c
Merge branch 'master' into add-test-for-deadlock-system-tables
2019-09-02 16:39:26 +03:00
Zhichang Yu
9e587fb139
fix DataTypeAggregateFunction deserialization
2019-09-02 20:09:40 +08:00
Alexander Tokmakov
e2928481a3
Merge branch 'master' into merging_template_format
2019-09-02 12:41:34 +03:00
alesapin
e7ee9b7a67
Merge master
2019-09-02 11:46:00 +03:00
Alexey Milovidov
85f45747a5
Fixed test
2019-09-02 04:04:41 +03:00
Alexey Milovidov
951079c457
Updated test
2019-09-02 04:00:58 +03:00
Alexey Milovidov
32adf456d8
Remove support for recursive-by-thread RWLocks because it makes no sense
2019-09-02 03:59:27 +03:00
Alexey Milovidov
2d331fb992
Fixed possible deadlock in RWLock
2019-09-02 03:12:01 +03:00
Alexey Milovidov
de44330cfa
Updated exception message
2019-09-01 22:21:00 +03:00
Alexey Milovidov
2b6d27d587
Merge branch 'add-test-for-deadlock-system-tables' of github.com:yandex/ClickHouse into add-test-for-deadlock-system-tables
2019-09-01 15:16:54 +03:00
Alexey Milovidov
27d753eab7
Fixed error
2019-09-01 15:16:31 +03:00
alexey-milovidov
7ababf2644
Merge branch 'master' into add-test-for-deadlock-system-tables
2019-09-01 04:35:10 +03:00
Alexey Milovidov
6f02cd145f
Added a kludge (experimental)
2019-09-01 04:32:44 +03:00
alexey-milovidov
2e86aece86
Merge pull request #6763 from yandex/add-os-thread-ids-in-system-tables
...
Add "os_thread_ids" column to system tables
2019-09-01 04:30:16 +03:00
Alexey Milovidov
cb79e2371e
Add "os_thread_ids" column to system tables
2019-09-01 00:47:15 +03:00
alesapin
2647d4ca15
Rename immutable to readonly
2019-08-30 23:12:26 +03:00
stavrolia
f51901bb3f
fix comment
2019-08-30 16:27:05 +03:00