Vitaly Baranov
133675a86a
Simplify format of the header of data sending to a shard in a distributed query.
2019-12-05 13:34:25 +03:00
Mikhail Korotov
88e37020e0
JSONCompactEachRow and JSONCompactEachRowWithNamesAndTypes ( #7841 )
...
* Research commit
* Created Output EachRow Format
* Fixed bugs
* Created output format JSONCompactEachRowWithNamesAndTypes without totals
* Fixed bugs
* Fixed bugs
* Totals for JSONCompactEachRowWithNamesAndTypes
* Deleted needless debug
* Working commit
* Working commit
* Working commit
* Working commit
* Working commit
* Working commit
* Working commit
* Working commit
* Working commit
* Working commit
* Working commit
* Working commit
* Tests added
* Input Format for JSONCompactEachRow
* Fixed bugs for JSONCompactEachRow Input format
* Fixed bugs for JSONCompactEachRowRowInputFormat.cpp
* JSONCompactEachRow and JSONCompactEachRowWithNamesAndTypes united
* Created input format for both formats
* fixed bugs
* fixed bugs
* Working commit
* Working commit
* Working commit
* Working commit
* tests
* Working commit
* Final tests
* Performance tests added
2019-12-05 13:13:40 +03:00
Nikita Mikhaylov
b6413d4a04
better
2019-12-05 12:52:49 +03:00
Nikita Mikhaylov
9d005d1565
true changes
2019-12-05 12:50:42 +03:00
Vitaly Baranov
2cc78413ed
Merge pull request #7914 from azat/dist-send-extra-checksum
...
Add checksum for extra info/query in distributed sends
2019-12-05 11:43:08 +03:00
alesapin
a524e82a7f
Merge pull request #8029 from ClickHouse/fix_dict_partially_update_failover
...
Fix external dictionaries failover, when they have invalidate_query.
2019-12-05 11:12:17 +03:00
chertus
d3d2e39955
fix SEMI RIGHT JOIN
2019-12-04 23:23:07 +03:00
Azat Khuzhin
3a72e1c12a
Add checksum for extra info/query in distributed sends
...
This extras includes:
- server revision
- query settings
- query
Otherwise the code can try to interpret data, and got for instance
std::length_error exception, which is not catched (to mark the part as
broken). Also this will protect from the corruptions on disk.
And add a simple test, since dbms/tests/integration/test_insert_into_distributed too complex.
Also simplify the code by using readStringBinary() over
readVarUInt()+b.readStrict() (this also gains additional checks that
string is not bigger then 1GB).
Refs: #4852 (8ef7f3589a
)
v2: avoid ABI breakage (suggested by @vitlibar)
v3: minor code fixes (suggested by @vitlibar) and as a consequence clang-8 build fix
v4: drop DBMS_MIN_REVISION_WITH_EXTRAS_CHECKSUM_IN_DIST_BATCH and also
revert some renames to make the patch cleaner
2019-12-04 22:54:38 +03:00
Nikolai Kochetov
715d7a6455
Disable processors by default.
2019-12-04 20:24:53 +03:00
alesapin
7b8443710d
Better code
2019-12-04 19:20:24 +03:00
alesapin
40c6966223
Don't check dictionary modification if it's already have an exception.
2019-12-04 18:23:18 +03:00
alesapin
3128b62f0e
Create dictionary if not exists
2019-12-04 18:11:36 +03:00
alesapin
a41764cccc
Reload dictionary with invalidate query if it has exception
2019-12-04 17:45:56 +03:00
Nikolai Kochetov
3ac103f21c
Fix the number of rows in MergingSortedTransform.
2019-12-04 16:45:42 +03:00
Nikolai Kochetov
901a6b4627
Style fix.
2019-12-04 12:56:08 +03:00
Nikolai Kochetov
efe1616fda
Fix optimization for 'WHERE column in (empty set)' case.
2019-12-04 12:55:01 +03:00
Nikolai Kochetov
81b5b17828
Fix pvs studio varning.
2019-12-04 12:55:01 +03:00
Nikolai Kochetov
fa98587c19
Init update_info in port.
2019-12-04 12:55:01 +03:00
Nikolai Kochetov
4337fc20b5
Enable processors by default.
2019-12-04 12:55:01 +03:00
Nikolai Kochetov
8524a75580
Check once if filter column can be always false in filter transform.
2019-12-04 12:55:01 +03:00
Nikolai Kochetov
9465183408
Add info about affected ports after prepare.
2019-12-04 12:55:01 +03:00
Nikolai Kochetov
4f2f474fde
Add info about affected ports to IProcessor::prepare
2019-12-04 12:55:01 +03:00
alesapin
eb17d51dd4
Remove unused function
2019-12-04 12:49:54 +03:00
alesapin
d1d3bac744
Better
2019-12-04 12:48:08 +03:00
alesapin
9e5f13e7af
Merge branch 'fix_metric_suffix_in_settings_value' of https://github.com/millb/ClickHouse into millb-fix_metric_suffix_in_settings_value
2019-12-04 12:22:29 +03:00
alesapin
8877129d60
Merge pull request #8012 from ClickHouse/millb-url_engine_problem
...
Merging [#7154 ]
2019-12-04 12:13:15 +03:00
alesapin
fa4728d44f
Fix style
2019-12-03 20:57:00 +03:00
Nikolai Kochetov
7b3c56990d
Merge pull request #8005 from ClickHouse/add-more-checks-to-chunk
...
Add forgotten rows number check to chunk constructor.
2019-12-03 20:26:19 +03:00
alesapin
2a43daec5c
Buildable code
2019-12-03 20:15:16 +03:00
alesapin
20e4e92c49
Fix some conflicts
2019-12-03 19:32:41 +03:00
chertus
3829df0e7b
fix comment
2019-12-03 17:38:13 +03:00
chertus
e160d35e54
more Join.addJoinedBlock() refactoring
2019-12-03 17:30:51 +03:00
Nikolai Kochetov
31d5c66867
Fix num rows in IRowInputFormat
2019-12-03 16:43:40 +03:00
Alexander Burmak
108c792fd9
Fixed typos in ExternalLoader.h and CrossToInnerJoinVisitor.cpp
2019-12-03 16:39:42 +03:00
chertus
46b27e4dd1
perf: move heavy addJoinedBlock() operations out of lock
2019-12-03 16:31:52 +03:00
chertus
3dc6ef421e
perf: do not make row filter if not needed
2019-12-03 15:55:20 +03:00
alexey-milovidov
1b2297bcee
Merge pull request #7999 from litao91/typo_fix
...
Fix a minor typo on formatting UNION ALL AST
2019-12-03 15:29:49 +03:00
alexey-milovidov
aadf7c689c
Merge pull request #7993 from ClickHouse/merge-great-circle-distance
...
Merging #7307
2019-12-03 15:23:38 +03:00
alexey-milovidov
f09c29a2b6
Merge pull request #7992 from ClickHouse/excitoon-favorites-s3auth
...
Merging #7623
2019-12-03 15:21:12 +03:00
Nikolai Kochetov
448d755a3b
Add forgotten rows number check to chunk constructor.
2019-12-03 14:56:51 +03:00
litao91
d9607c50fe
Fix a minor typo on formatting UNION ALL AST
2019-12-03 16:58:00 +08:00
Alexey Milovidov
40b0f12755
Removed tons of garbage from "greatCircleDistance" function. But algorithm is still unclear.
2019-12-03 05:25:23 +03:00
alexey-milovidov
0c904f1484
Merge pull request #7721 from 4ertus2/decimal
...
Basic GatherUtils support for Decimals
2019-12-03 05:15:38 +03:00
alexey-milovidov
0734d4695b
Merge pull request #7724 from 4ertus2/some
...
Missing array functions for Decimals
2019-12-03 05:14:33 +03:00
alexey-milovidov
812df3aef3
Merge pull request #7755 from nikitamikhaylov/storage_set
...
Using the Set engine happen Exception: Method read is not supported by storage Set
2019-12-03 05:09:47 +03:00
alexey-milovidov
ef66c453c4
Update ExpressionAnalyzer.h
2019-12-03 05:09:05 +03:00
alexey-milovidov
cdaa913d1f
Update ExpressionAnalyzer.h
2019-12-03 05:08:41 +03:00
Alexey Milovidov
2474cdfa09
Removed tons of garbage from "greatCircleDistance" function. But algorithm is still unclear.
2019-12-03 04:55:46 +03:00
alexey-milovidov
f5e88f8cd5
Merge pull request #7307 from ClickHouse/new-branch-for-new-geodist
...
Speed up greatCircleDistance function with test
2019-12-03 04:34:35 +03:00
alexey-milovidov
dbc95e27be
Update greatCircleDistance.cpp
2019-12-03 04:33:36 +03:00
Alexey Milovidov
49d25e06a8
Added a test
2019-12-03 04:29:19 +03:00
Alexey Milovidov
93ef18501f
Added a test
2019-12-03 04:27:48 +03:00
Alexey Milovidov
199a209c82
Added a test
2019-12-03 04:23:17 +03:00
Alexey Milovidov
8fa05a212b
Fixed bug in #7623
2019-12-03 04:22:25 +03:00
alexey-milovidov
edede563c9
Merge pull request #7747 from hczhcz/patch-1113
...
Fix empty array handling in arraySplit
2019-12-03 04:00:31 +03:00
alexey-milovidov
87adf355e8
Merge pull request #7935 from filimonov/kafka_perf
...
[wip] attempt to improve kafka parsing performance
2019-12-03 03:52:40 +03:00
alexey-milovidov
b6edda194f
Update KafkaBlockInputStream.cpp
2019-12-03 03:52:02 +03:00
alexey-milovidov
6fdfdcfd4e
Update ValuesBlockInputFormat.cpp
2019-12-03 03:51:10 +03:00
alexey-milovidov
4d65ad4034
Update ValuesBlockInputFormat.cpp
2019-12-03 03:50:50 +03:00
alexey-milovidov
4a69082fbb
Update ProtobufRowInputFormat.h
2019-12-03 03:49:50 +03:00
alexey-milovidov
a7521b97f9
Update IInputFormat.cpp
2019-12-03 03:49:05 +03:00
Alexey Milovidov
d70b73f033
Merge branch 's3auth' of https://github.com/excitoon-favorites/ClickHouse into excitoon-favorites-s3auth
2019-12-03 03:06:01 +03:00
alexey-milovidov
fe05565cec
Update StorageS3.cpp
2019-12-03 03:03:44 +03:00
Alexey Milovidov
fc174dce6e
More simple
2019-12-03 02:50:53 +03:00
Alexey Milovidov
9d099bde8e
Merge branch 'master' into fix_check_table_with_empty_pk
2019-12-03 02:36:21 +03:00
alexey-milovidov
3355492f02
Merge pull request #7983 from ClickHouse/fix_dictionary_lifetime
...
Fix bug with minimal dictionary lifetime equals zero
2019-12-03 02:32:38 +03:00
Alexey Milovidov
e86047bd72
Merge branch 'sigrtmin' of https://github.com/abyss7/ClickHouse into abyss7-sigrtmin
2019-12-03 02:27:53 +03:00
alexey-milovidov
e44dc0a6c8
Merge pull request #7981 from ClickHouse/fix_number_of_streams_for_storages_with_globs
...
Limit number of streams for read from StorageFile and StorageHDFS
2019-12-03 02:23:10 +03:00
alexey-milovidov
41f1224d59
Merge pull request #7987 from achulkov2/dict-small-refactoring
...
Very small code refactoring for external dictionaries
2019-12-03 02:13:31 +03:00
alexey-milovidov
8338be47dd
Merge pull request #7975 from yurriy/mysql-protocol-text
...
Support more column types in MySQL compatibility server
2019-12-03 02:08:24 +03:00
chertus
a179c5025f
Merge branch 'master' into joins
2019-12-02 21:08:39 +03:00
chertus
5ce65546e9
semi & anti join
2019-12-02 21:08:04 +03:00
Ivan Lezhankin
2118b33d9a
Replace SIGRTMIN with SIGTSTP
2019-12-02 20:29:19 +03:00
achulkov2
6c6db598b1
Very small refactoring for external dictionaries
2019-12-02 18:59:45 +03:00
alesapin
7410d17892
Fix bug with minimal dictionary lifetime equals zero
2019-12-02 16:05:43 +03:00
Vitaly Baranov
b2685a3710
Merge pull request #7727 from kekekekule/master
...
Added lifetime to system.dictionaries #6820
2019-12-02 15:15:44 +03:00
chertus
0251f4e442
perf: move applyLazyDefaults() out of cycle for ALL JOIN
2019-12-02 14:45:21 +03:00
Ivan Lezhankin
0e276ad887
Fix build
2019-12-02 14:29:52 +03:00
alesapin
08dfd1d1c4
Limit number of streams for read from StorageFile and StorageHDFS
2019-12-02 14:25:41 +03:00
Ivan Lezhankin
4f56e03729
Use SIGRTMIN instead of SIGPROF for user debugging purposes
2019-12-02 13:50:57 +03:00
alesapin
a4ca30b5fd
Fix bug in check table for tables without primary key
2019-12-02 13:30:04 +03:00
alesapin
af85eb8f80
Merge pull request #7916 from azat/dict-avoid-RELOAD-on-CREATE-DATABASE
...
Do not reload *all* dictionaries on CREATE DATABASE
2019-12-02 11:50:44 +03:00
Yuriy
211e1364f6
fixed style
2019-12-02 06:12:27 +03:00
Yuriy
3677d1dcfa
implemented ProtocolText
2019-12-01 15:17:49 +03:00
Vladimir Chebotarev
d0760bc169
Fixed tests and logic of authorization in S3.
2019-12-01 14:24:55 +03:00
Vladimir Chebotarev
0a19b4fbd6
Attempt to add S3 authentication.
2019-12-01 10:56:11 +03:00
Vladimir Chebotarev
94684c07bf
Added syntax for access_key_id
and secret_access_key
in S3 table function and storage.
2019-12-01 10:56:11 +03:00
Артем Стрельцов
25aa2114b2
Deleted redundant empty lines
2019-12-01 03:49:11 +03:00
Nikolay Degterinsky
6b39253b71
Fixed naming
2019-11-29 17:15:12 +03:00
Nikolay Degterinsky
59002ddb91
Merge branch 'master' of https://github.com/kekekekule/ClickHouse into dict_lifetime
2019-11-29 17:06:38 +03:00
Mikhail Filimonov
ad136ec62d
get rid of cloneWithColumns in internal loop and some other trash
2019-11-29 11:46:25 +01:00
Nikolai Kochetov
7a32ca057d
Merge pull request #7670 from excitoon-favorites/move_pool
...
Separated pool for background moves
2019-11-29 11:08:51 +03:00
hcz
dd7051bd98
Merge remote-tracking branch 'origin/master' into patch-2
2019-11-29 15:07:53 +08:00
hcz
5e45a4f3cd
Add comment on getValue() function
2019-11-29 14:50:36 +08:00
Mikhail Filimonov
4cc0d0ca5d
make virtual_columns local
2019-11-29 07:21:31 +01:00
Mikhail Filimonov
07722dfa93
more progress - cleanup internal cycles
2019-11-29 00:06:03 +01:00
alexey-milovidov
2e34f02a36
Merge pull request #7936 from ClickHouse/branch-for-slightly-improvement-of-usability-of-exception-messages
...
The rest of PR #7897 about improvement of exception message
2019-11-28 22:07:29 +03:00
Mikhail Filimonov
6217cb51f6
Get rid of InputStream layer for Kafka, use InputStream directly. Reopening port during reset.
2019-11-28 17:43:06 +01:00
stavrolia
295633b7ef
Another try
2019-11-28 17:31:35 +03:00
Anton Popov
9fe987c8e1
Merge pull request #7920 from excitoon-favorites/datetimettl
...
Removed check for using `Date` or `DateTime` column from TTL expressions
2019-11-27 19:55:57 +03:00