Maksim Kita
b72ea982b0
Remove unnecessary includes of InterpreterSelectQuery header
2023-02-16 12:17:03 +01:00
Maksim Kita
4ddf1b0f48
Planner filter push down optimization fix
2023-02-16 12:17:03 +01:00
Maksim Kita
80af0666ea
Analyzer support Materialized View
2023-02-16 12:17:03 +01:00
Maksim Kita
077555bd5d
Analyzer support build pushing to views chain
2023-02-16 12:17:03 +01:00
Maksim Kita
f8442b2a8d
Analyzer support LiveView
2023-02-16 12:17:03 +01:00
Kseniia Sumarokova
947753bd4c
Merge pull request #46409 from kssenii/follow-up-to-46168
...
Follow-up to #46168
2023-02-16 11:26:54 +01:00
Nikolay Degterinsky
ebec5b8050
Merge pull request #46422 from evillique/move-sources
...
Move MongoDB and PostgreSQL sources to Sources folder
2023-02-16 10:51:45 +01:00
flynn
ecc39978d7
fix conflict
2023-02-16 02:23:55 +00:00
Alexander Tokmakov
53e32b19e6
Update AggregatingTransform.cpp
2023-02-16 02:13:18 +03:00
Igor Nikonov
cb89fccac6
Disable debug output
2023-02-15 21:26:00 +00:00
Igor Nikonov
766f04b04c
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-02-15 21:24:25 +00:00
Nikolai Kochetov
f524dae924
Fix more tests.
2023-02-15 19:17:24 +00:00
alexX512
26fd12e0c7
Add checking for source processors in executing graph
2023-02-15 15:40:42 +00:00
Vladimir C
3e77463125
Merge pull request #46256 from bigo-sg/window_functions
2023-02-15 15:36:41 +01:00
Kruglov Pavel
e56dead5aa
Merge branch 'master' into fix-kafka-protobuf
2023-02-15 15:16:25 +01:00
Vladimir C
645fee3d94
Merge pull request #46009 from ClickHouse/vdimir/fix_read_in_order_final_desc
2023-02-15 11:23:07 +01:00
Nikolay Degterinsky
bcc4adf73d
Merge branch 'master' into move-sources
2023-02-15 08:46:09 +01:00
alexX512
5aabfe8644
Hard cancel for pipeline executor
2023-02-15 06:50:33 +00:00
lgbo-ustc
bfbd880475
fixed binary_tidy build failure
2023-02-15 08:44:10 +08:00
Nikolay Degterinsky
6e4b660033
Move MongoDB and PostgreSQL sources to Sources folder
2023-02-14 22:35:10 +00:00
Igor Nikonov
3eef3bf643
Simplify aggregation pass
...
always checking AggregatingStep and skipping Rollup, Cube, TotalsHaving
2023-02-14 21:49:56 +00:00
alexX512
e424ad9ba8
Delete unimportant logs
2023-02-14 21:24:33 +00:00
alexX512
b466855e04
Add hard_cancel mod
2023-02-14 21:20:01 +00:00
Igor Nikonov
425b99afca
GROUP BY WITH TOTALS
2023-02-14 21:14:41 +00:00
Nikolai Kochetov
62bce41d3d
Fix more tests.
2023-02-14 19:31:20 +00:00
Kseniia Sumarokova
fe155bb210
Merge branch 'master' into follow-up-to-46168
2023-02-14 18:28:46 +01:00
Geoff Genz
f3bcf26959
Merge branch 'ClickHouse:master' into http_client_version
2023-02-14 09:31:29 -07:00
kssenii
5bda358fb7
Follow-up to #46168
2023-02-14 15:45:38 +01:00
flynn
e05b4e9745
fix conflict
2023-02-14 13:49:28 +00:00
Nikita Taranov
cbd10c4493
Fix mbm with aio on top of query plan ( #45892 )
2023-02-14 14:02:54 +01:00
Igor Nikonov
49763f7dc2
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-02-14 12:13:22 +00:00
Igor Nikonov
7e24b5f059
Fix tests & review comment
2023-02-14 12:10:02 +00:00
Geoff Genz
2857b16562
Merge branch 'ClickHouse:master' into http_client_version
2023-02-14 03:41:17 -07:00
robot-ch-test-poll4
7c420f0982
Merge pull request #46350 from CurtizJ/fix-format-values
...
Fix `LOGICAL_ERROR` in async inserts with invalid data in format `VALUES`
2023-02-14 10:12:08 +01:00
Alexey Milovidov
307551f857
Update NativeFormat.cpp
2023-02-14 11:08:33 +03:00
lgbo-ustc
74d6ccad4f
fixed
...
change window specification as (partition by a order by b rows between unbounded preceding and unbounded following)
2023-02-14 15:49:12 +08:00
lgbo-ustc
0b32a655a8
fixed
2023-02-14 15:49:12 +08:00
lgbo-ustc
9a496aea6a
add new window function, ntile
2023-02-14 15:49:11 +08:00
Nikolai Kochetov
522a39f93f
Normal Projections analysis using query plan [In progress]
2023-02-13 19:56:31 +00:00
Nikolai Kochetov
e6e486c4a1
Not crash with minmax projection.
2023-02-13 17:10:10 +00:00
Nikolai Kochetov
d6ea566b20
Mostly support minmax projection.
2023-02-13 16:52:21 +00:00
Geoff Genz
be8bf3a6a3
Merge branch 'master' into http_client_version
2023-02-13 08:43:59 -07:00
Anton Popov
7f31196606
fix LOGICAL_ERROR in async inserts
2023-02-13 14:59:14 +00:00
Kruglov Pavel
e139070043
Merge branch 'master' into fix-kafka-protobuf
2023-02-13 12:38:18 +01:00
Vladimir C
89cb55aa1f
whitespace
2023-02-13 12:19:31 +01:00
Vladimir C
1a89ecd386
Apply suggestions from code review
...
Co-authored-by: Nikita Taranov <nikita.taranov@clickhouse.com>
2023-02-13 12:17:56 +01:00
vdimir
e4c0afcc3e
Do not build filtering set for sorting join with const columns
2023-02-13 12:17:56 +01:00
Alexander Gololobov
8fc08c74f0
Pass action_settings to getReadTaskColumns()
2023-02-12 21:55:53 +01:00
Alexander Gololobov
7634b9f56e
Added settings to enable the new logic
2023-02-12 21:54:23 +01:00
kssenii
3067c1d723
Merge remote-tracking branch 'upstream/master' into resubmit-prefetches
2023-02-11 11:36:23 +01:00
Alexander Gololobov
f2a975f548
Merge pull request #46270 from Avogar/fix-arrow-lc
...
Fix possible out of bounds error while reading LowCardinality(Nullable) in Arrow format
2023-02-11 10:03:00 +01:00
Vladimir C
ea670acb72
Add comment to requestReadingInOrder
2023-02-10 20:05:18 +01:00
Kruglov Pavel
8bbf62508f
Merge branch 'master' into fix-kafka-protobuf
2023-02-10 18:32:25 +01:00
Kruglov Pavel
2a3cb8b4ee
Merge pull request #45340 from Avogar/parquet-fixed-binary
...
Support FixedSizeBinary type in Parquet/Arrow
2023-02-10 18:31:20 +01:00
avogar
04cf144edc
Fix TSKV, update docs
2023-02-10 17:20:51 +00:00
Alexey Milovidov
d5d87bc6b7
Merge pull request #46152 from ClickHouse/with-fill-bigint
...
More convenient usage of big integers and ORDER BY WITH FILL.
2023-02-10 20:05:32 +03:00
avogar
d1efd02480
Extend setting input_format_null_as_default for more formats
2023-02-10 16:41:09 +00:00
avogar
deda940d87
Fix possible out of bounds error while reading LowCardinality(Nullable) in Arrow format
2023-02-10 12:57:47 +00:00
Geoff Genz
99c3ff53c5
Merge remote-tracking branch 'origin/master' into http_client_version
...
# Conflicts:
# src/Interpreters/Context.cpp
# src/Interpreters/Context.h
2023-02-10 04:35:53 -07:00
Geoff Genz
db982695fa
Simplify NativeOutputFormat constructors
2023-02-10 04:30:08 -07:00
Geoff Genz
7ed8ed0284
Add support for client_protocol_version sent with HTTP
2023-02-10 03:47:06 -07:00
flynn
37a850bd36
Merge branch 'master' of github.com:ClickHouse/ClickHouse into iceberg
2023-02-10 08:47:14 +00:00
flynn
db15634a01
fix conflict
2023-02-10 08:41:04 +00:00
avogar
ba761e04d7
Handle unknown exceptions
2023-02-09 12:31:01 +00:00
Kruglov Pavel
c3c0356fdd
Merge branch 'master' into fix-kafka-protobuf
2023-02-09 13:04:55 +01:00
Igor Nikonov
df23731061
Removing DISTINCT on top of aggregation if possible
2023-02-09 11:37:15 +00:00
vdimir
45f7ef6c8f
Do not apply read in reverse order for queries with final
2023-02-09 11:00:22 +00:00
vdimir
a179528ea4
Revert "Fix read in order optimization for DESC sorting with FINAL"
...
This reverts commit cf762347d5
2023-02-09 10:47:45 +00:00
Robert Schulze
e490ec91d9
Merge branch 'master' into rs/fix-fragile-linking
2023-02-09 11:33:59 +01:00
Kruglov Pavel
b5f90c608a
Merge branch 'master' into parquet-fixed-binary
2023-02-09 11:26:38 +01:00
vdimir
cf762347d5
Fix read in order optimization for DESC sorting with FINAL
2023-02-09 10:26:37 +00:00
Vladimir C
0042c20770
Merge pull request #46054 from alexX512/bug_fix_pulling_pipeline
2023-02-09 10:50:46 +01:00
Alexey Milovidov
3a75ede60a
Implement #16733
2023-02-09 08:41:47 +01:00
Alexey Milovidov
cb847172ea
Fix something
2023-02-09 08:39:46 +01:00
Alexey Milovidov
a40ef2bf18
Make it better
2023-02-09 08:09:19 +01:00
Alexey Milovidov
292a5abbc9
Merge remote-tracking branch 'origin/master' into with-fill-bigint
2023-02-09 01:47:51 +01:00
Alexander Tokmakov
8101b044fa
Merge pull request #46091 from azat/sanity-assertions
...
Sanity assertions for closing file descriptors
2023-02-09 01:02:03 +03:00
kssenii
b0b865c32e
Resubmit prefetches
2023-02-08 21:26:24 +01:00
avogar
c3e8dd8984
Fix low cardinality case
2023-02-08 19:14:28 +00:00
Nikolai Kochetov
e7dba2a85b
Partly support minmax projection.
2023-02-08 18:34:22 +00:00
Nikolai Kochetov
d04b3a30cc
Fix 01710_projections_partial_optimize_aggregation_in_order
2023-02-08 14:26:20 +00:00
Nikolai Kochetov
f30d1efe2f
Fix test 01710_projections.
2023-02-08 13:32:34 +00:00
Robert Schulze
6ff232d782
Merge branch 'master' into rs/fix-fragile-linking
2023-02-08 12:51:12 +01:00
Kruglov Pavel
4e2918cee3
Merge branch 'master' into parquet-fixed-binary
2023-02-08 12:31:13 +01:00
Igor Nikonov
72c393e7c4
Merge pull request #46014 from ClickHouse/inorder-optimization-update-sorting-properties
...
Update sorting properties after reading in order applied
2023-02-08 10:19:47 +01:00
Alexey Milovidov
a9ec73a209
Fix strange trash
2023-02-08 08:26:27 +01:00
Igor Nikonov
5709c9832c
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-02-07 11:49:25 +00:00
Igor Nikonov
14f8eff290
Fixes: checking if steps between DISTINCTs can generate new rows
2023-02-07 11:48:23 +00:00
Robert Schulze
08c1f8346e
Merge remote-tracking branch 'origin/master' into rs/fix-fragile-linking
2023-02-07 11:22:00 +00:00
Azat Khuzhin
8cc41b7f41
Check return value of ::close()
...
Note, that according close(2), EINTR should not be retriable for close()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-07 11:28:22 +01:00
Anton Popov
3ad5a085da
Merge pull request #45987 from CurtizJ/fix-sparse-columns-crash
...
Fix `INTERSECT` and `EXCEPT` with sparse columns
2023-02-06 23:01:02 +01:00
Nikolai Kochetov
b2cc2332f5
Fix 01710_force_use_projection.
2023-02-06 19:20:03 +00:00
Nikolai Kochetov
a542626fa3
Make 01710_aggregate_projections work.
2023-02-06 18:42:58 +00:00
Alexey Perevyshin
7a4d658a46
Merge branch 'master' into bug_fix_pulling_pipeline
2023-02-06 14:24:26 +03:00
flynn
0dd8a61a8e
fix conflict
2023-02-06 03:25:12 +00:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking
...
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848 ) but neither did @Algunenano nor me at first
really understand why the definition is in the header.
Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b
2023-02-05 20:49:34 +00:00
Igor Nikonov
089a0009ad
Polishing
...
+ try to stabilize distinct in order perf test
2023-02-05 13:38:20 +00:00
alexX512
ea07f08f75
Fix bug with wrong checking of execution finish in PullingAsyncPipeline
2023-02-05 13:23:26 +00:00
flynn
f89a6cf68d
Improve format JSONColumns when result is empty
2023-02-05 13:13:21 +00:00
alexX512
3482960f90
Merge branch 'master' of github.com:ClickHouse/ClickHouse
2023-02-05 13:08:49 +00:00
alexX512
b31decc15a
Fix bug with wrong checking of execution finish in PullingAsyncPipeline
2023-02-05 12:54:50 +00:00
Nikita Taranov
1f1be6195e
fix
2023-02-04 21:54:48 +00:00
Nikita Taranov
750933d265
fix
2023-02-04 21:09:44 +00:00
Nikita Taranov
b983b363f8
Merge branch 'master' into aggr_partitions_independently
2023-02-04 18:24:31 +00:00
Nikita Taranov
5183cc6ee5
fix
2023-02-04 16:14:53 +00:00
Nikolai Kochetov
310a4c69af
Aggregate Projections analysis using query plan [In progress]
2023-02-03 18:30:06 +00:00
Igor Nikonov
f49f5d7091
Update tests
...
+ updadte sorting properties after applying aggregation in order
2023-02-03 17:10:31 +00:00
Igor Nikonov
38a0ef8743
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-02-03 15:13:06 +00:00
Igor Nikonov
82e50fc211
Fix review comments
2023-02-03 15:12:20 +00:00
Anton Popov
88f2068bfb
Merge branch 'master' into fix-sparse-columns-crash
2023-02-03 16:01:11 +01:00
Nikita Mikhaylov
33877b5e00
Parallel replicas. Part [2] ( #43772 )
2023-02-03 14:34:18 +01:00
Igor Nikonov
ed00db7580
Update sorting properties after reading in order applied
2023-02-03 13:15:06 +00:00
Igor Nikonov
7b922784e8
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-02-03 10:29:37 +00:00
Robert Schulze
6908093787
Merge pull request #45973 from ClickHouse/final-codecs
...
Add "final" specifier to some classes
2023-02-03 10:52:59 +01:00
Anton Popov
ca20b1e32f
fix INTERSECT and EXCEPT with sparse columns
2023-02-03 00:17:55 +00:00
Igor Nikonov
46ed757472
Fix review comments
2023-02-02 20:07:47 +00:00
Robert Schulze
3472ed4f0d
Make some classes 'final'
2023-02-02 15:59:39 +00:00
Antonio Andelic
a39e4e24c6
Merge branch 'master' into optimize_parquet_reader
2023-02-02 14:18:00 +01:00
Igor Nikonov
32ab4dc693
Fix review comments
2023-02-02 11:46:13 +00:00
Igor Nikonov
996b833faa
Some polishing
2023-02-02 11:14:26 +00:00
Alexey Milovidov
97b6934ed6
Update ParquetBlockInputFormat.cpp
2023-02-02 02:42:21 +03:00
Nikita Taranov
4fbbca075b
review fixes
2023-02-01 23:01:02 +00:00
Igor Nikonov
8f873d5682
Disable optimize_duplicate_order_by_and_distinct
...
+ tests
2023-02-01 21:13:44 +00:00
Nikolai Kochetov
ae420b30e5
Aggregate Projections analysis using query plan [In progress]
2023-02-01 17:55:06 +00:00
Igor Nikonov
fc71acc768
Fix DISTINCT with intersect/except
2023-02-01 13:06:34 +00:00
Igor Nikonov
d3d5e67a36
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-02-01 12:03:06 +00:00
liuneng
17fc22a21e
add parquet max_block_size setting
2023-02-01 18:29:20 +08:00
Robert Schulze
b512316586
Merge pull request #45682 from ClickHouse/rename-qrc-to-qc
...
Rename "Query Result Cache" to "Query Cache"
2023-02-01 11:23:29 +01:00
liuneng
cda9b0beea
optimize parquet reader
2023-02-01 15:54:10 +08:00
Nikolai Kochetov
e2c32ccbca
Aggregate Projections analysis using query plan [In progress]
2023-01-31 20:33:01 +00:00
Alexey Milovidov
075dfe9005
Merge pull request #45750 from ucasfl/arrow-duration
...
Arrow support duration type
2023-01-31 20:34:38 +03:00
flynn
ffddc0dcce
fix conflict
2023-01-31 10:28:58 +00:00
Robert Schulze
325c6bdf3d
Renaming: "Query Result Cache" --> "Query Cache"
...
Reasons:
- The cache will at some point store intermediate results as opposed to
only query results. We should change the terminology now without
having to worry about backward compat.
- Equivalent caches in MySQL (1) and Starrocks (2) are called "query
cache".
- The new name is ca. 13.8% more catchy.
(1) https://dev.mysql.com/doc/refman/5.6/en/query-cache.html
(2) https://docs.starrocks.io/en-us/2.5/using_starrocks/query_cache
2023-01-31 09:54:34 +00:00
Igor Nikonov
f8bcf78593
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-01-30 23:12:09 +00:00
Igor Nikonov
fdb9336b02
Respect aliases when compare DISTINCT columns
2023-01-30 23:08:26 +00:00
Nikita Taranov
8223fc6ab9
stash
2023-01-30 23:02:27 +00:00
Nikolai Kochetov
f09f8f80af
Aggregate Projections analysis using query plan [In progress]
2023-01-30 20:09:58 +00:00
Nikita Taranov
8b51bd84c0
improve perf
2023-01-30 17:11:56 +00:00
Nikita Taranov
3e3d1d15a8
properly handle case when there is nondeterministic functions
2023-01-30 17:11:56 +00:00
Nikita Taranov
581f31ad3d
better
2023-01-30 17:11:56 +00:00
Nikita Taranov
a18343773f
improve perf
2023-01-30 17:11:56 +00:00
Nikita Taranov
0d393c7a7f
cosmetics
2023-01-30 17:11:56 +00:00
Nikita Taranov
47f57b7a5d
few heuristics to disable optimisation
2023-01-30 17:11:56 +00:00
Nikita Taranov
3ce725023c
fix style & build
2023-01-30 17:11:56 +00:00
Nikita Taranov
52fe7edbd9
better key analysis
2023-01-30 17:11:56 +00:00
Nikita Taranov
962f163dbd
add settings
2023-01-30 17:11:52 +00:00
Nikita Taranov
b59ba628a9
do not group buckets w/o need
2023-01-30 17:10:45 +00:00
Nikita Taranov
abe9ab2dff
small fix for matchTrees
2023-01-30 17:10:45 +00:00
Nikita Taranov
d7e1e6314c
move matchTrees to separate file
2023-01-30 17:10:45 +00:00
Nikita Taranov
2057db68a2
cosmetics
2023-01-30 17:10:45 +00:00
Nikita Taranov
1d45cce03c
support for aggr in order
2023-01-30 17:10:45 +00:00
Nikita Taranov
a2c9aeb7c9
stash
2023-01-30 17:10:45 +00:00
flynn
801d6db486
Arrow support duration type
2023-01-29 10:55:31 +00:00
flynn
9b517cdc76
fix conflict
2023-01-29 08:34:56 +00:00
Alexey Milovidov
9ad87f9917
Fix style and typo
2023-01-29 03:51:42 +01:00
Alexey Milovidov
54bb8b1ceb
Merge branch 'master' into fix-kafka-protobuf
2023-01-29 00:59:40 +03:00
Igor Nikonov
101a0cc90b
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-01-27 19:08:38 +00:00
Nikolai Kochetov
286a58801e
Aggregate Projections analysis using query plan [In progress]
2023-01-27 18:38:14 +00:00
Igor Nikonov
300f78df96
Merge pull request #45567 from ClickHouse/enable-remove-redundant-sorting
...
Enable query_plan_remove_redundant_sorting optimization by default
2023-01-27 19:14:36 +01:00
avogar
0a99d421f5
Fix formats parser resetting, test processing bad messages in kafka
2023-01-27 13:22:50 +00:00
Igor Nikonov
4480d8ae60
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-01-26 22:16:24 +00:00
Igor Nikonov
60a69d4689
Fix distinct after JOIN or UNION
2023-01-26 21:53:08 +00:00
exX512
1793730df8
Change behaviour of cancel in PipelineExecutor
2023-01-26 21:30:36 +00:00
Igor Nikonov
211edc054e
Fix check for sum*(Float)
2023-01-26 19:26:53 +00:00
Alexander Tokmakov
067b1f5f13
Merge branch 'master' into exception_message_patterns4
2023-01-26 15:20:58 +01:00
Igor Nikonov
105d72e299
Fix: sum*() with Nullable(Float)
2023-01-26 13:17:06 +00:00
Igor Nikonov
e1302e1a32
Fix: sum*() with Floats
...
sum(Float) depends on order, do not remove sorting in such case
2023-01-25 22:39:28 +00:00
Igor Nikonov
eea076cee0
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-01-25 20:16:41 +00:00
Igor Nikonov
dd5acf9caf
Update
2023-01-25 17:09:26 +00:00
Azat Khuzhin
71286e779e
Extend assertion in buildPushingToViewsChain() to respect is_detached
...
Follow-up for: #45566 (@tavplubix)
Refs: #43161 (@AlfVII)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-25 15:42:06 +01:00
Alexander Tokmakov
d1baa7300c
reformat ParsingException
2023-01-24 23:21:29 +01:00
Alexander Tokmakov
dd57215934
Merge branch 'master' into exception_message_patterns4
2023-01-24 17:03:12 +01:00
Kruglov Pavel
23c12ac8ee
Merge branch 'master' into parquet-fixed-binary
2023-01-24 16:51:05 +01:00
Kruglov Pavel
689fbea759
Merge pull request #45478 from Avogar/fix-arrow-abort
...
Fix possible aborts in arrow lib
2023-01-24 16:37:44 +01:00
Kruglov Pavel
4bd3f0e5ef
Merge pull request #44953 from Avogar/tsv-csv-detect-header
...
Detect header in CSV/TSV/CustomSeparated files automatically
2023-01-24 15:13:52 +01:00
Alexander Tokmakov
6ecae8388e
Merge branch 'master' into exception_message_patterns4
2023-01-24 14:42:36 +01:00
Igor Nikonov
7108189b45
Merge remote-tracking branch 'origin/master' into revert-45414-revert-43905-igor/remove_redundant_order_by
2023-01-24 11:46:24 +00:00
Alexey Milovidov
851f6bf910
Merge pull request #45448 from azat/fix-query-hang
...
Fix possible (likely distributed) query hung
2023-01-24 13:00:37 +03:00
Alexander Tokmakov
c6910f39b9
fix
2023-01-24 01:11:58 +01:00
Kruglov Pavel
cd1cd904a7
Merge branch 'master' into tsv-csv-detect-header
2023-01-23 23:49:56 +01:00
Alexander Tokmakov
bb4c8e169f
check number of parameters in format string
2023-01-23 23:16:16 +01:00
Alexander Tokmakov
3f6594f4c6
forbid old ctor of Exception
2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages ( #45449 )
...
* save format string for NetException
* format exceptions
* format exceptions 2
* format exceptions 3
* format exceptions 4
* format exceptions 5
* format exceptions 6
* fix
* format exceptions 7
* format exceptions 8
* Update MergeTreeIndexGin.cpp
* Update AggregateFunctionMap.cpp
* Update AggregateFunctionMap.cpp
* fix
2023-01-24 00:13:58 +03:00
Kruglov Pavel
478a552a0a
Merge branch 'master' into tsv-csv-detect-header
2023-01-23 21:47:17 +01:00
Robert Schulze
0ab7ae6c17
Merge pull request #43797 from ClickHouse/query-result-cache
...
Query result cache [experimental]
2023-01-23 19:54:35 +01:00
Maksim Kita
7b48c75e82
Merge pull request #45485 from kitaisreal/remove-unnecessary-get-total-row-count-function-calls
...
Remove unnecessary getTotalRowCount function calls
2023-01-23 21:02:51 +03:00
Kruglov Pavel
af2c1bac6a
Fix typo
2023-01-23 17:13:16 +01:00
Kruglov Pavel
01ddf326ac
Merge branch 'master' into parquet-fixed-binary
2023-01-23 15:31:45 +01:00
Kruglov Pavel
84200be7d2
Better comment
2023-01-23 15:31:07 +01:00
Anton Popov
f181254fb0
fix race in destructor of ParallelParsingInputFormat
2023-01-23 01:18:58 +00:00
Igor Nikonov
f807b4b315
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2023-01-22 18:30:30 +00:00
Robert Schulze
340f406553
Merge branch 'master' into query-result-cache
2023-01-22 13:21:36 +01:00
Maksim Kita
47385a19e7
Remove unnecessary getTotalRowCount function calls
2023-01-21 11:27:07 +01:00
Azat Khuzhin
a64f6b5f3e
Fix possible (likely distributed) query hung
...
Recently I saw the following, the client executed long distributed query
and terminated the connection, and in this case query cancellation will
be done from PullingAsyncPipelineExecutor dtor, but during cancellation
one of nodes sent ECONNRESET, and this leads to an exception from
PullingAsyncPipelineExecutor::cancel(), and this leads to a deadlock
when multiple threads waits each others, because cancel() for
LazyOutputFormat wasn't called.
Here is as relevant portion of logs:
2023.01.04 08:26:09.236208 [ 37968 ] {f2ed6149-146d-4a3d-874a-b0b751c7b567} <Debug> executeQuery: (from 10.61.13.253:44266, user: default) TooLongDistributedQueryToPost
...
2023.01.04 08:26:09.262424 [ 37968 ] {f2ed6149-146d-4a3d-874a-b0b751c7b567} <Trace> MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part 9_330_538_18, approx. 61440 rows starting from 0
2023.01.04 08:26:09.266399 [ 26788 ] {f2ed6149-146d-4a3d-874a-b0b751c7b567} <Trace> Connection (s4.ch:9000): Connecting. Database: (not specified). User: default
2023.01.04 08:26:09.266849 [ 26788 ] {f2ed6149-146d-4a3d-874a-b0b751c7b567} <Trace> Connection (s4.ch:9000): Connected to ClickHouse server version 22.10.1.
2023.01.04 08:26:09.267165 [ 26788 ] {f2ed6149-146d-4a3d-874a-b0b751c7b567} <Debug> Connection (s4.ch:9000): Sent data for 2 scalars, total 2 rows in 3.1587e-05 sec., 62635 rows/sec., 68.00 B (2.03 MiB/sec.), compressed 0.4594594594594595 times to 148.00 B (4.41 MiB/sec.)
2023.01.04 08:39:13.047170 [ 37968 ] {f2ed6149-146d-4a3d-874a-b0b751c7b567} <Error> PullingAsyncPipelineExecutor: Code: 210. DB::NetException: Connection reset by peer, while writing to socket (10.7.142.115:9000). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):
0. ./.build/./contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x1818234c in /usr/lib/debug/usr/bin/clickhouse.debug
1. ./.build/./src/Common/Exception.cpp:69: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x1004fbda in /usr/lib/debug/usr/bin/clickhouse.debug
2. ./.build/./src/Common/NetException.h:12: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x14e352f3 in /usr/lib/debug/usr/bin/clickhouse.debug
3. ./.build/./src/IO/BufferBase.h:39: DB::Connection::sendCancel() @ 0x15c21e6b in /usr/lib/debug/usr/bin/clickhouse.debug
4. ./.build/./src/Client/MultiplexedConnections.cpp:0: DB::MultiplexedConnections::sendCancel() @ 0x15c4d5b7 in /usr/lib/debug/usr/bin/clickhouse.debug
5. ./.build/./src/QueryPipeline/RemoteQueryExecutor.cpp:627: DB::RemoteQueryExecutor::tryCancel(char const*, std::__1::unique_ptr<DB::RemoteQueryExecutorReadContext, std::__1::default_delete<DB::RemoteQueryExecutorReadContext> >*) @ 0x14446c09 in /usr/lib/debug/usr/bin/clickhouse.debug
6. ./.build/./contrib/libcxx/include/__iterator/wrap_iter.h💯 DB::ExecutingGraph::cancel() @ 0x15d2c0de in /usr/lib/debug/usr/bin/clickhouse.debug
7. ./.build/./contrib/libcxx/include/__memory/unique_ptr.h:300: DB::PullingAsyncPipelineExecutor::cancel() @ 0x15d32055 in /usr/lib/debug/usr/bin/clickhouse.debug
8. ./.build/./contrib/libcxx/include/__memory/unique_ptr.h:312: DB::PullingAsyncPipelineExecutor::~PullingAsyncPipelineExecutor() @ 0x15d31f4f in /usr/lib/debug/usr/bin/clickhouse.debug
9. ./.build/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::processOrdinaryQueryWithProcessors() @ 0x15cde919 in /usr/lib/debug/usr/bin/clickhouse.debug
10. ./.build/./src/Server/TCPHandler.cpp:0: DB::TCPHandler::runImpl() @ 0x15cd8554 in /usr/lib/debug/usr/bin/clickhouse.debug
11. ./.build/./src/Server/TCPHandler.cpp:1904: DB::TCPHandler::run() @ 0x15ce6479 in /usr/lib/debug/usr/bin/clickhouse.debug
12. ./.build/./contrib/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x18074f07 in /usr/lib/debug/usr/bin/clickhouse.debug
13. ./.build/./contrib/libcxx/include/__memory/unique_ptr.h:54: Poco::Net::TCPServerDispatcher::run() @ 0x180753ed in /usr/lib/debug/usr/bin/clickhouse.debug
14. ./.build/./contrib/poco/Foundation/src/ThreadPool.cpp:213: Poco::PooledThread::run() @ 0x181e3807 in /usr/lib/debug/usr/bin/clickhouse.debug
15. ./.build/./contrib/poco/Foundation/include/Poco/SharedPtr.h:156: Poco::ThreadImpl::runnableEntry(void*) @ 0x181e1483 in /usr/lib/debug/usr/bin/clickhouse.debug
16. ? @ 0x7ffff7e55fd4 in ?
17. ? @ 0x7ffff7ed666c in ?
(version 22.10.1.1)
And here is the state of the threads:
<details>
<summary>system.stack_trace</summary>
```sql
SELECT
arrayStringConcat(arrayMap(x -> demangle(addressToSymbol(x)), trace), '\n') AS sym
FROM system.stack_trace
WHERE query_id = 'f2ed6149-146d-4a3d-874a-b0b751c7b567'
SETTINGS allow_introspection_functions=1
Row 1:
──────
sym:
pthread_cond_wait
std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&)
bool ConcurrentBoundedQueue<DB::Chunk>::emplaceImpl<DB::Chunk>(std::__1::optional<unsigned long>, DB::Chunk&&)
DB::IOutputFormat::work()
DB::ExecutionThreadContext::executeTask()
DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic<bool>*)
Row 2:
──────
sym:
pthread_cond_wait
Poco::EventImpl::waitImpl()
DB::PipelineExecutor::joinThreads()
DB::PipelineExecutor::executeImpl(unsigned long)
DB::PipelineExecutor::execute(unsigned long)
Row 3:
──────
sym:
pthread_cond_wait
Poco::EventImpl::waitImpl()
DB::PullingAsyncPipelineExecutor::Data::~Data()
DB::PullingAsyncPipelineExecutor::~PullingAsyncPipelineExecutor()
DB::TCPHandler::processOrdinaryQueryWithProcessors()
DB::TCPHandler::runImpl()
DB::TCPHandler::run()
Poco::Net::TCPServerConnection::start()
Poco::Net::TCPServerDispatcher::run()
Poco::PooledThread::run()
Poco::ThreadImpl::runnableEntry(void*)
```
</details>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-21 08:05:56 +01:00
Azat Khuzhin
e2fcf0f072
Catch exception on query cancellation
...
Since we still want to join the thread, yes it will be done in dtor, but
this looks better.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-21 08:05:56 +01:00
Azat Khuzhin
0566f72d36
Cleanup PullingAsyncPipelineExecutor::cancel()
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-21 08:05:56 +01:00
avogar
86336940f8
Better comment
2023-01-20 16:41:59 +00:00
avogar
4432ee9927
Fix aborts in arrow lib
2023-01-20 16:40:33 +00:00
avogar
550a703fbc
Make a bit better
2023-01-20 14:58:39 +00:00
Kruglov Pavel
28ddcc2432
Merge branch 'master' into tsv-csv-detect-header
2023-01-20 15:08:38 +01:00
Nikolai Kochetov
134ac9b2dd
Aggregate Projections analysis using query plan [In progress]
2023-01-19 19:53:22 +00:00
avogar
c34c0aa22e
Fix comments
2023-01-19 16:03:46 +00:00
Kruglov Pavel
9820beae68
Apply suggestions from code review
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2023-01-19 16:11:13 +01:00
Igor Nikonov
d0ce804bfc
Fix: dynamic_cast -> typeid_cast for SortingStep
2023-01-19 13:40:21 +00:00
Igor Nikonov
df3776d24b
Make test stable
...
+ disable debug logging
2023-01-19 11:43:40 +00:00
Ilya Yatsishin
00962b7ad5
Merge pull request #45424 from Avogar/fix-json-import-nested
2023-01-19 10:31:40 +01:00
Igor Nikonov
57d2fd300a
Fix: correct update of data stream sorting properties after removing
...
sorting
2023-01-19 00:11:58 +00:00
avogar
a8f20363f4
Fix JSON/BSONEachRow parsing with HTTP
2023-01-18 22:49:03 +00:00
Igor Nikonov
1866f990de
Revert "Revert "Remove redundant sorting""
2023-01-18 20:12:34 +01:00
Igor Nikonov
7ed8fec94f
Revert "Remove redundant sorting"
2023-01-18 18:38:25 +01:00
flynn
4dcd3cc4f6
fix style
2023-01-18 08:46:06 +00:00
flynn
a8e1363151
implement storage iceberg
2023-01-18 03:43:56 +00:00
Robert Schulze
4f90824347
Merge remote-tracking branch 'origin/master' into query-result-cache
2023-01-17 22:49:53 +00:00
Igor Nikonov
0db9bf38a2
Merge branch 'master' into igor/remove_redundant_order_by
2023-01-17 22:26:24 +01:00
Nikolai Kochetov
9c6c6d9844
Aggregate Projections analysis using query plan [In progress]
2023-01-17 20:12:34 +00:00
Alexander Tokmakov
8b13b85ea0
Merge pull request #44543 from ClickHouse/text_log_add_pattern
...
Add a column with a message pattern to text_log
2023-01-17 20:19:32 +03:00
Igor Nikonov
0cfa08df7a
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2023-01-17 16:28:17 +00:00
Igor Nikonov
9855504403
Rename source file according to implementation
2023-01-17 16:24:51 +00:00
Kruglov Pavel
96bb99f864
Merge branch 'master' into tsv-csv-detect-header
2023-01-17 15:33:02 +01:00
Igor Nikonov
6328e02f22
Fix: update input/output stream properties
...
After removing sorting step we need to update sorting properties of
input/ouput streams
2023-01-17 13:39:18 +00:00
Sema Checherinda
35431e91e3
Merge pull request #45276 from ucasfl/avro-fix
...
Fix some avro reading bugs
2023-01-17 12:48:44 +01:00
avogar
5bf4704e7a
Support FixedSizeBinary type in Parquet/Arrow
2023-01-16 21:01:31 +00:00
avogar
1c0941d72a
Add docs and examples
2023-01-16 16:46:41 +00:00
avogar
3ea80b0f54
Merge branch 'master' of github.com:ClickHouse/ClickHouse into tsv-csv-detect-header
2023-01-16 15:14:25 +00:00
Igor Nikonov
a34991cb65
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2023-01-16 12:14:02 +00:00
Robert Schulze
099e30ef2a
Merge remote-tracking branch 'origin/master' into query-result-cache
2023-01-16 08:04:49 +00:00
Robert Schulze
27fe7ebd93
Cosmetics
2023-01-15 16:12:48 +00:00
flynn
29eb30b49f
Fix some reading avro format bugs
...
fix
2023-01-14 18:05:26 +00:00
Alexander Tokmakov
2d7773fccc
Merge branch 'master' into text_log_add_pattern
2023-01-13 20:33:46 +01:00
avogar
e2470dd670
Fix tests
2023-01-13 17:03:53 +00:00
Robert Schulze
d7d3f61c73
Cleanup SourceFromChunks a bit
2023-01-13 10:57:31 +00:00
Robert Schulze
15e11741cb
Cosmetics
2023-01-13 00:00:23 +00:00
Robert Schulze
1b53307375
Refcount cache entries to improve lookup performance
2023-01-12 23:38:39 +00:00
Robert Schulze
0c528fca29
Introduce SourceFromChunks to avoid unnecessary partial result chunk concatenation
2023-01-12 23:01:00 +00:00
avogar
b461935374
Better
2023-01-12 13:11:04 +00:00
Kruglov Pavel
05a11ff4a4
Merge branch 'master' into tsv-csv-detect-header
2023-01-12 12:35:18 +01:00
Maksim Kita
a140d6c5b1
Fixed code review issues
2023-01-12 12:07:58 +01:00
Maksim Kita
47f4159909
Analyzer support distributed queries processing
2023-01-12 12:07:58 +01:00
avogar
e4d774d906
Better naming
2023-01-11 22:57:14 +00:00
avogar
26cd56d113
Fix tests, make better
2023-01-11 22:52:15 +00:00
Alexander Gololobov
6adf1c025f
Merge pull request #45165 from ClickHouse/more_logging_for_replicated_ttl
...
More logging to facilitate debugging of flaky test_ttl_replicated
2023-01-11 23:24:48 +01:00
avogar
3b45863d15
Make better implementation, fix tests
2023-01-11 17:12:56 +00:00
Alexander Gololobov
659fa96365
More logging to facilitate debugging
2023-01-11 13:06:38 +01:00
Nikolai Kochetov
5e7a6ac619
Merge pull request #45122 from ClickHouse/revert-45121-revert-44653-custom-reading-for-mutation
...
Revert "Revert "Custom reading for mutation""
2023-01-11 12:37:32 +01:00
avogar
6312b75f44
Fix style
2023-01-10 16:28:52 +00:00
avogar
615fe4cecb
Fix tests
2023-01-10 16:27:23 +00:00
Maksim Kita
fbba28b31e
Analyzer aggregation without column fix
2023-01-10 16:49:55 +01:00
Nikolai Kochetov
4673b3fe1d
Revert "Revert "Custom reading for mutation""
2023-01-10 16:31:01 +01:00
Alexander Tokmakov
c8ec130be4
Revert "Custom reading for mutation"
2023-01-10 17:51:30 +03:00
Nikolai Kochetov
11418963c0
Merge pull request #44653 from ClickHouse/custom-reading-for-mutation
...
Custom reading for mutation
2023-01-10 12:16:24 +01:00
Robert Schulze
923fa2c15a
Fix review comments, pt. II
2023-01-10 10:21:08 +00:00
Robert Schulze
0c3b034887
Merge remote-tracking branch 'origin/master' into query-result-cache
2023-01-10 09:02:41 +00:00
Alexey Milovidov
1229a20fb3
Merge pull request #45047 from ClickHouse/fix-buffer-overflow
...
Fix buffer overflow in parser
2023-01-10 05:06:03 +03:00
Yakov Olkhovskiy
4f32f3b8cb
Merge pull request #44484 from bigo-sg/arrow_struct_field
...
Optimization for reading struct fields in parquet/orc files
2023-01-09 15:36:26 -05:00
Nikolai Kochetov
3e430b1b6e
Add comments. Remove unused flag from Context.
2023-01-09 19:52:11 +00:00
Kseniia Sumarokova
119501f1d9
Merge pull request #44698 from Avogar/parquet-bool
...
Support Bool type in Arrow/Parquet/ORC
2023-01-09 12:56:28 +01:00
lgbo-ustc
b639bcabc0
some fixes
2023-01-09 18:13:00 +08:00
lgbo-ustc
f127b3a60a
update ArrowFieldIndexUtil
2023-01-09 18:13:00 +08:00
lgbo-ustc
a3bdfddc9d
support nested table
2023-01-09 18:13:00 +08:00
lgbo-ustc
4f3f781b85
fixed test case
2023-01-09 18:13:00 +08:00
lgbo-ustc
755f03db4e
fixed
2023-01-09 18:13:00 +08:00
lgbo-ustc
f6850d96cb
fixed missing columns
2023-01-09 18:13:00 +08:00
lgbo-ustc
4cf6beee27
fixed
2023-01-09 18:13:00 +08:00
lgbo-ustc
81e2832133
fixed
2023-01-09 18:13:00 +08:00
lgbo-ustc
77cea49cec
fixed including header failure
2023-01-09 18:13:00 +08:00
lgbo-ustc
8f8f6f966b
Optimization for reading struct fields in parquet/orc files
2023-01-09 18:13:00 +08:00
Igor Nikonov
7298b39a18
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2023-01-09 10:08:48 +00:00
Igor Nikonov
cec7dc5a5a
Renaming according to setting
2023-01-09 10:06:47 +00:00
Igor Nikonov
e29cd5d640
Rename settings to remove_redundant_sorting
2023-01-09 09:50:00 +00:00
Alexey Milovidov
eea5a2c521
Merge branch 'master' into fix-buffer-overflow
2023-01-09 09:15:29 +03:00
Alexey Milovidov
0d39d26a34
Don't fix parallel formatting
2023-01-09 06:15:20 +01:00
Alexey Milovidov
d331f0ce82
Fix buffer overflow in parser
2023-01-09 03:31:12 +01:00
Alexey Milovidov
0d8d6eca67
Non significant changes
2023-01-09 01:51:05 +01:00
Alexey Milovidov
797dfdd42e
Non significant changes
2023-01-09 01:50:19 +01:00
Alexey Milovidov
3430df2f8c
Non significant changes
2023-01-09 01:48:57 +01:00
avogar
ee72799121
Fix tests, make better
2023-01-06 20:46:43 +00:00
Robert Schulze
45dbcf88e5
Merge remote-tracking branch 'origin/master' into query-result-cache
2023-01-06 20:42:48 +00:00
Igor Nikonov
2187bdd4cc
Disable diagnostics
...
+ cleanup
+ disable optimization in sort performance test since it removes sorting
at all
2023-01-06 17:00:05 +00:00
Igor Nikonov
d2d5a09802
Fix typo
2023-01-06 15:01:31 +00:00
Igor Nikonov
48b58a9fed
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2023-01-06 13:55:26 +00:00
Igor Nikonov
d886b56338
Fix review comments
2023-01-06 13:54:28 +00:00
avogar
7fcdb08ec6
Detect header in CSV/TSV/CustomSeparated files automatically
2023-01-05 22:57:25 +00:00
Robert Schulze
4fcead15c3
Fix handling of exceptions during query runtime
2023-01-04 22:33:04 +00:00
Yakov Olkhovskiy
7a5a36cbed
Merge branch 'master' into refactoring-ip-types
2023-01-04 11:11:06 -05:00
Kruglov Pavel
b9bdf62bf3
Merge branch 'master' into parquet-bool
2023-01-04 14:49:41 +01:00
Kruglov Pavel
59263f3ae1
Merge pull request #44501 from Avogar/validate-types
...
Validate data types according to settings.
2023-01-04 14:48:09 +01:00
Kruglov Pavel
90ae405033
Merge pull request #44876 from Avogar/fix-perf-tests
...
Revert some changes from #42777 to fix performance tests
2023-01-04 14:27:17 +01:00
Kruglov Pavel
0c7d39ac7f
Merge pull request #44832 from ucasfl/row-number
...
Fix output_format_pretty_row_numbers does not preserve the counter across the blocks
2023-01-04 14:15:47 +01:00
Igor Nikonov
1edfdfda4d
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2023-01-04 12:34:40 +00:00
Kruglov Pavel
4e261ab230
Fix JSONCompactEachRow
2023-01-03 21:16:38 +01:00
Kruglov Pavel
314d95fd71
Fix special build
2023-01-03 20:34:30 +01:00
avogar
28eb2dbd4c
Revert some changes from #42777
2023-01-03 18:53:03 +00:00
Alexey Milovidov
e855d3519a
Merge branch 'master' into refactoring-ip-types
2023-01-02 21:58:53 +03:00
Kruglov Pavel
4a7c399076
Merge branch 'master' into parquet-bool
2023-01-02 16:33:42 +01:00
Kruglov Pavel
0a43976977
Merge branch 'master' into validate-types
2023-01-02 16:10:14 +01:00
Kruglov Pavel
1c2dc05d6e
Merge pull request #44446 from Avogar/arrow-nullables
...
Respect setting settings.schema_inference_make_columns_nullable in Parquet/ORC/Arrow formats
2023-01-02 16:05:57 +01:00
Kruglov Pavel
966f57ef68
Merge pull request #42777 from Avogar/improve-streaming-engines
...
Refactor and Improve streaming engines Kafka/RabbitMQ/NATS and data formats
2023-01-02 15:59:06 +01:00
flynn
7780fae9db
fix
2023-01-02 12:38:51 +00:00
flynn
7b487dd923
fix
2023-01-02 12:37:43 +00:00
flynn
3a1dd045dd
Fix output_format_pretty_row_numbers does not preserve the counter across the blocks
2023-01-02 09:27:37 +00:00
Robert Schulze
e9e04166d9
Merge remote-tracking branch 'origin/master' into query-result-cache
2023-01-02 08:20:27 +00:00
Kruglov Pavel
8479615c48
Merge pull request #44684 from Avogar/avro-bool
...
Input/ouptut avro bool type as ClickHouse bool type
2022-12-30 17:56:36 +01:00
Kruglov Pavel
4982d132fb
Merge branch 'master' into validate-types
2022-12-30 17:52:13 +01:00
Nikolay Degterinsky
dfe93b5d82
Merge pull request #42284 from Algunenano/perf_experiment
...
Performance experiment
2022-12-30 03:14:22 +01:00
Kruglov Pavel
894726bd8f
Merge branch 'master' into improve-streaming-engines
2022-12-29 22:59:45 +01:00
Nikolai Kochetov
3c02e208c8
Fix more tests.
2022-12-29 17:52:31 +00:00
Nikita Mikhaylov
ffaff7167b
Fix flaky tests ( #44690 )
2022-12-29 14:32:10 +01:00
Alexey Milovidov
875797ee63
Merge pull request #44685 from ClickHouse/remove-old-code-4
...
Remove old code
2022-12-29 15:43:59 +03:00
Alexey Milovidov
9378194320
Merge pull request #44688 from ClickHouse/improve-odbc-test
...
Improve odbc test
2022-12-29 15:40:47 +03:00
avogar
a0db1dd1ea
Support Bool type in Arrow/Parquet/ORC
2022-12-28 22:58:28 +00:00
Alexey Milovidov
e524d4bcb1
Merge pull request #44673 from ClickHouse/logical-error-with-join-subquery-union-and-totals
...
Fix LOGICAL_ERROR for JOIN with subquery, UNION and totals.
2022-12-29 01:56:15 +03:00
Raúl Marín
5de11979ce
Unify query elapsed time measurements ( #43455 )
...
* Unify query elapsed time reporting
* add-test: Make shell tests executable
* Add some tests around query elapsed time
* Style and ubsan
2022-12-28 21:01:41 +01:00
Raúl Marín
e915ce1e95
Merge remote-tracking branch 'blessed/master' into perf_experiment
2022-12-28 20:15:43 +01:00
Raúl Marín
f6428964cc
Better and common error handling
2022-12-28 20:15:27 +01:00
Alexey Milovidov
c3eff78f20
Don't be kind with third-party code
2022-12-28 20:07:26 +01:00
Alexey Milovidov
cf594a659d
Update MongoDBSource.cpp
2022-12-28 21:26:42 +03:00
Alexey Milovidov
33bcd07be5
Remove old code
2022-12-28 19:02:06 +01:00
avogar
f1191bbbc6
Input/ouptut avro bool type as ClickHouse bool type
2022-12-28 17:38:58 +00:00
Nikolai Kochetov
2d84ef2891
Fix tests and update IntersectOrExceptStep.
2022-12-28 14:44:45 +00:00
Nikolai Kochetov
21e93cd41d
Fix LOGICAL_ERROR for JOIN with subquery, UNION and totals.
2022-12-28 13:00:32 +00:00
avogar
411f98306a
Merge branch 'master' of github.com:ClickHouse/ClickHouse into validate-types
2022-12-27 19:24:15 +00:00
Kruglov Pavel
6dea7336f7
Merge pull request #44405 from Avogar/fix-parquet-orc
...
Fix reading columns that are not presented in input data in Parquet/ORC formats
2022-12-27 16:58:35 +01:00
Kruglov Pavel
e981b2e78b
Merge pull request #43813 from Avogar/fix-sum-if
...
Better handling of NULL in aggregate combinators, fix possible segfault/logical error
2022-12-27 16:50:48 +01:00
Alexey Milovidov
1b21cc018e
Merge pull request #44342 from ClickHouse/add-test-29883
...
Add a test for #29883
2022-12-27 14:55:14 +03:00
Raúl Marín
fc1fa82a39
Merge branch 'master' into perf_experiment
2022-12-27 10:51:58 +01:00
Alexander Tokmakov
ca989e9212
less runtime format strings
2022-12-23 19:50:34 +01:00
Nikolai Kochetov
5030c92373
Merge branch 'master' into fix-wrong-condition-for-async-reading-from-mt
2022-12-23 14:18:08 +01:00
Nikolai Kochetov
4f84becfa0
Fix wrong condition for enabling asyn creading from MergeTree.
2022-12-23 13:00:23 +00:00
Igor Nikonov
e2e898b707
Remove incorrect assert
2022-12-23 12:12:25 +00:00
Vladimir C
7482ea54ab
Merge pull request #43972 from ClickHouse/vdimir/tmp-data-in-fs-cache-2
2022-12-23 11:59:27 +01:00
Dmitry Novik
cff882d506
Merge remote-tracking branch 'origin/master' into refector-function-node
2022-12-22 21:34:29 +00:00
Igor Nikonov
95c3703bbf
Remove incorrect assert
...
ExpressionStep is not always under SortingStep
It can be ReadFromRemote in distributed query
2022-12-22 19:32:41 +00:00
Kruglov Pavel
6a017a6586
Merge pull request #43379 from Avogar/better-capn-proto
...
Add small improvements in CapnProto format
2022-12-22 14:50:10 +01:00
vdimir
182b34c11e
Fixes
2022-12-22 10:22:57 +00:00
Yakov Olkhovskiy
a8cb29da4b
Merge branch 'master' into refactoring-ip-types
2022-12-21 23:56:24 -05:00
avogar
4ab3e90382
Validate types in table function arguments/CAST function arguments/JSONAsObject schema inference
2022-12-21 21:21:30 +00:00
Igor Nikonov
c7650850fe
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-21 15:56:08 +00:00
Igor Nikonov
919fa9b84e
If aggregation step is in order, - do not remove underneath sorting step
2022-12-21 12:55:42 +00:00
Igor Nikonov
fdb277245c
Remove current sorting only if next is full sorting
2022-12-21 12:21:35 +00:00
Kruglov Pavel
5e01a3d74e
Merge branch 'master' into improve-streaming-engines
2022-12-21 10:51:50 +01:00
Kruglov Pavel
09ab5832b1
Merge pull request #44382 from Avogar/fix-bson-object-id
...
Fix reading ObjectId in BSON schema inference
2022-12-21 10:48:50 +01:00
Igor Nikonov
1624647f07
Fix: distributed query with cluster() table function
2022-12-20 23:27:01 +00:00
Alexey Milovidov
a6f50edde3
Merge branch 'master' into add-test-29883
2022-12-21 01:45:18 +03:00
Igor Nikonov
5cd4767838
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-20 17:56:59 +00:00
avogar
c49638e3a9
Respect setting settings.schema_inference_make_columns_nullable in Parquet/ORC/Arrow formats
2022-12-20 17:46:42 +00:00
Dmitry Novik
4793412887
Merge branch 'master' into refector-function-node
2022-12-20 18:26:19 +01:00
Kruglov Pavel
643a35bed1
Merge pull request #44019 from Avogar/refactor-schema-inference
...
Refactor and improve schema inference for text formats
2022-12-20 17:29:03 +01:00
Kruglov Pavel
c0b17ca0af
Merge branch 'master' into fix-bson-object-id
2022-12-20 17:18:10 +01:00
Kruglov Pavel
a0ee1c1227
Merge branch 'master' into fix-sum-if
2022-12-20 15:20:23 +01:00
Kruglov Pavel
fe28faa32d
Fix style
2022-12-20 14:49:39 +01:00
Kruglov Pavel
3f1e40aacd
Merge branch 'master' into fix-orc
2022-12-20 13:32:46 +01:00
Igor Nikonov
15bb4cb65c
Fix distributed query check
2022-12-20 10:21:32 +00:00
Raúl Marín
45d27f461b
Merge branch 'master' into perf_experiment
2022-12-20 09:07:48 +00:00
Igor Nikonov
2164b18485
Disable the optimization for distributed queries
...
+ based on ReadFromRemote
- improve by disabling optimization only between Union and ReadFromRemote
2022-12-19 23:24:35 +00:00
avogar
e262e375dc
Fix reading columns that are not presented in input data in Parquet/ORC formats
2022-12-19 20:30:54 +00:00
avogar
0c406adce2
Fix reading Map type in ORC format
2022-12-19 18:23:07 +00:00
Igor Nikonov
1e53b3be38
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-19 16:44:22 +00:00
Nikolai Kochetov
860afd1832
Add a very bad fix.
2022-12-19 16:33:10 +00:00
avogar
21cdf6e6ae
Fix reading ObjectId in BSON schema inference
2022-12-19 14:13:42 +00:00
Robert Schulze
b93c56323c
Merge remote-tracking branch 'origin/master' into query-result-cache
2022-12-19 09:18:36 +00:00
Alexey Milovidov
3e8dcdc155
Add earlier check
2022-12-18 08:01:43 +01:00
avogar
22e70fa71e
Try fix tests
2022-12-16 18:09:42 +00:00
Dmitry Novik
875a24a650
Merge remote-tracking branch 'origin/master' into refector-function-node
2022-12-16 16:07:30 +00:00
avogar
291e51c533
Merge branch 'better-capn-proto' of github.com:Avogar/ClickHouse into better-capn-proto
2022-12-16 14:43:06 +00:00
avogar
4a51bdce86
Fix comments
2022-12-16 13:58:54 +00:00
Kruglov Pavel
3fad5c7f1f
Merge branch 'master' into refactor-schema-inference
2022-12-16 14:24:51 +01:00
avogar
34274be8df
Fix use after free
2022-12-16 13:20:36 +00:00
Igor Nikonov
319158f32f
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2022-12-16 11:46:27 +00:00
Robert Schulze
4db33f16e3
Make DoS attack on cache less likely
2022-12-15 21:42:51 +00:00
Igor Nikonov
ea0bd38ab1
Avoid to remove sorting expression for now
2022-12-15 21:11:02 +00:00
Robert Schulze
bcc54ab9b4
Add cfg for min query duration for caching
2022-12-15 20:10:37 +00:00
avogar
cfcb444699
Merge branch 'master' of github.com:ClickHouse/ClickHouse into better-capn-proto
2022-12-15 20:04:43 +00:00
Kruglov Pavel
25f199dd89
Merge pull request #43332 from Avogar/csv-custom-delimiter
...
Improve reading CSV field in CustomSeparated/Template format
2022-12-15 21:03:29 +01:00
Igor Nikonov
629f317d7d
Fix for full sorting merge join
2022-12-15 20:01:50 +00:00
avogar
f26b5b57ad
Better comment
2022-12-15 19:52:57 +00:00
avogar
5b4ea53fed
Fix fields order
2022-12-15 19:52:25 +00:00
avogar
755b08a49e
Fix comments
2022-12-15 19:47:10 +00:00
Kruglov Pavel
c5b2e4cc23
Merge branch 'master' into improve-streaming-engines
2022-12-15 18:44:35 +01:00
Robert Schulze
537c64aad2
Merge remote-tracking branch 'origin/master' into query-result-cache
...
Conflicts:
- tests/queries/0_stateless/02117_show_create_table_system.reference
2022-12-15 12:17:58 +00:00
Robert Schulze
f1e367179e
Make maximum record count per item configurable
2022-12-15 11:16:44 +00:00
Robert Schulze
e52dfaf45c
Make number of cached items configurable
2022-12-15 10:08:28 +00:00
Igor Nikonov
bde3e43d3b
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_distinct
2022-12-15 00:36:06 +00:00
Igor Nikonov
6dd2849ae6
Fix for distributed queries
...
+ 01952_optimize_distributed_group_by_sharding_key
2022-12-14 21:54:20 +00:00
avogar
a94a0d9c85
Fix tests, fix bugs
2022-12-14 21:17:00 +00:00
Igor Nikonov
b5de352861
Move checks to functions
2022-12-14 17:51:02 +00:00
Igor Nikonov
6ca0773516
Fix for distributed query
...
- do not apply optimization in case of UnionStep on top of Sorting
01104_distributed_one_test.sql
2022-12-14 16:10:52 +00:00
Dmitry Novik
1df038e39c
Merge remote-tracking branch 'origin/master' into refector-function-node
2022-12-14 14:55:30 +00:00
Igor Nikonov
03f7bd1192
Fix for distributed queries
2022-12-14 13:54:42 +00:00
Igor Nikonov
e25053dec0
Query plan visitor with debug logs
2022-12-13 22:46:39 +00:00
Igor Nikonov
1cbdce8eea
Fix window function case
2022-12-13 16:21:14 +00:00
Nikolay Degterinsky
9b6d31b95d
Merge branch 'master' into perf_experiment
2022-12-13 17:15:07 +01:00
Igor Nikonov
1bed5064e7
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-13 10:55:50 +00:00
Igor Nikonov
6f5fc6645b
Fix: remove ORDER BY before "mergeExpressions" optimization
...
When removing ORDER BY we remove corresponding expression,
so to remove only ORDER BY expression, do it before "mergeExpressions"
optimization is applied
2022-12-13 10:52:18 +00:00
Robert Schulze
8fc23262c5
Merge pull request #44125 from ClickHouse/minor-renaming
...
Minor file renaming
2022-12-13 08:35:26 +01:00
Igor Nikonov
db6e1acc14
Remove redundant DISTINCT
...
- it doesn't consider prefixes yet
2022-12-12 22:40:44 +00:00
avogar
739ad23b1f
Make better, fix bugs, improve error messages
2022-12-12 22:00:45 +00:00
Dmitry Novik
3d2fccab87
Merge branch 'master' into refector-function-node
2022-12-12 21:36:39 +01:00
Igor Nikonov
69d1b39ef8
Merge remote-tracking branch 'origin/master' into igor/remove_redundant_order_by
2022-12-12 13:57:48 +00:00
Kruglov Pavel
d8d7385e68
Merge pull request #43161 from AlfVII/fix-race-condition-between-inserts-and-dropping-mvs
...
Fixed race condition between inserts and dropping MVs
2022-12-12 13:53:36 +01:00
Robert Schulze
b78652e138
Query result cache
2022-12-12 10:37:10 +00:00
