thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-22 15:42:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
112,761 Commits 9,152 Branches 1,781 Tags 2.1 GiB
06087b3bf0
Commit Graph

198 Commits

Author SHA1 Message Date
Raúl Marín
b2cfa70541 Reduce dependencies on ASTFunction.h 
481 -> 230
 2021-11-26 18:21:54 +01:00
Raúl Marín
051dddd8df Reduce dependencies on ASTIdentifier.h 
Goes from rebuilding 483 objects to 165 when it's modified
 2021-11-26 16:49:40 +01:00
Nikolay Degterinsky
ba50e8387d Merge branch 'master' into query_parameters 2021-11-15 19:55:54 +00:00
Azat Khuzhin
48e9456e75 Disable log_queries in clickhouse-copier 
This will avoid calling Context::addQueryFactoriesInfo()

v2: disable it in ClusterCopier::reloadTaskDescription() too
v3: just create local context and pass it to the ClusterCopier instead
of global context
 2021-11-15 10:32:30 +03:00
Azat Khuzhin
6274c7bfc2 Do not use global context for query execution in clickhouse-copier 2021-11-15 10:32:19 +03:00
Nikolay Degterinsky
d76976e272 Merge branch 'master' into query_parameters 2021-10-18 17:34:20 +03:00
Nikolai Kochetov
a08c98d760 Move some files. 2021-10-16 17:03:50 +03:00
Nikolai Kochetov
067eaadadd Merge branch 'master' into removing-data-streams-folder 2021-10-16 09:46:05 +03:00
Nikolai Kochetov
c668696047
Merge pull request #30171 from ClickHouse/remove-stream-interfaces 
Remove stream interfaces
 2021-10-16 09:34:01 +03:00
Nikolai Kochetov
fd14faeae2 Remove DataStreams folder. 2021-10-15 23:18:20 +03:00
Nikolai Kochetov
700504e7d6
Update Internals.cpp 2021-10-14 22:09:44 +03:00
Nikolai Kochetov
ab28c6c855 Remove BlockInputStream interfaces. 2021-10-14 13:25:43 +03:00
Nikolai Kochetov
3d3e143c29
Merge pull request #30001 from ClickHouse/remove-streams-from-formats 
Remove streams from formats.
 2021-10-14 12:39:07 +03:00
Nikolay Degterinsky
49c1a52f34 Merge branch 'master' into query_parameters 2021-10-13 03:04:40 +03:00
Nikolai Kochetov
ec18340351 Remove streams from formats. 2021-10-11 19:11:50 +03:00
Nikita Mikhaylov
32d77aafe2
Merge pull request #29921 from azat/copier-fixes 
copier: add ability to configure retries and delays between them
 2021-10-11 12:11:38 +03:00
Azat Khuzhin
452b3b443d copier: add ability to configure retries and delays between them 
The following options had been introduced:
- max-table-tries
- max-shard-partition-tries
- max-shard-partition-piece-tries-for-alter
- retry-delay-ms
 2021-10-10 08:58:35 +03:00
Nikolai Kochetov
340b53ef85 Remove some more streams. 2021-10-08 17:03:54 +03:00
Nikolai Kochetov
78e1db209f
Remove more data streams (#29491) 
* Remove more streams.

* Fixing build.

* Fixing build.

* Rename files.

* Fix fast test.

* Fix StorageKafka.

* Try fix kafka test.

* Move createBuffer to KafkaSource ctor.

* Revert "Move createBuffer to KafkaSource ctor."

This reverts commit 81fa94d27e.

* Revert "Try fix kafka test."

This reverts commit 2107e54969.

* Comment some rows in test.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2021-10-07 11:26:08 +03:00
Nikolay Degterinsky
e03bf0c526 Merge branch 'master' into query_parameters 2021-10-04 17:23:17 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Nikolay Degterinsky
7f3d395bf3 Merge branch 'master' into query_parameters 2021-09-30 02:01:33 +03:00
Nikolai Kochetov
998d29ebc7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-23 13:10:27 +03:00
Nikolay Degterinsky
1b1d90deb8 Merge branch 'master' into query_parameters 2021-09-22 13:53:56 +03:00
Dmitry Novik
fdd2e5d530
Update programs/copier/ClusterCopier.cpp 2021-09-21 01:19:40 +03:00
Eugene Konkov
130552330f
Fix error message 
Fix typo in error message. Better wording.
 2021-09-21 00:58:22 +03:00
Nikolai Kochetov
a790d391c0 Merge branch 'master' into rewrite-pushing-to-views 2021-09-20 19:43:15 +03:00
Nikolai Kochetov
341553febd Fix build. 2021-09-16 20:40:42 +03:00
Nikolai Kochetov
b997214620 Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 20:48:18 +03:00
Azat Khuzhin
c99bd06195 copier: fix partition_key alias using WITH 
After those changes it is not possible to add an alias for the tuple
element anymore, i.e.:

  $ clickhouse-local -q 'select ((1,2) = (1,2) as foo), foo'
  1       1
  $ clickhouse-upstream local -q 'select ((1,2) = ((1,2) as foo)), foo'
  1       (1,2)

But these can be done using WITH statement.
 2021-09-12 00:15:31 +03:00
Nikolay Degterinsky
22f59e1714 Add support for table and database query parameters 2021-09-07 01:19:02 +03:00
Nikolai Kochetov
66a76ab70f Rewrite PushingToViewsBlockOutputStream part 6 2021-09-03 20:29:36 +03:00
Raúl Marín
34b0bb6b7f Move formatBlock to its own file 2021-07-29 16:39:42 +02:00
Amos Bird
dbfb699690
Asynchronously drain connections. 2021-07-19 21:53:29 +08:00
Nikita Mikhaylov
0d0ea91a0f fix 2021-06-17 15:58:05 +00:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Nikita Mikhaylov
f6932026d5 delete partition from original table in case of errors while moving 2021-06-04 14:21:47 +00:00
Nikita Mikhaylov
68c92cd93e style 2021-06-03 23:58:47 +00:00
Nikita Mikhaylov
b978405f4a fix 2021-06-03 15:50:44 +00:00
Nikita Mikhaylov
4382aac25d support for ReplicatedTable creation without arguments 2021-06-02 15:46:37 +03:00
Nikita Mikhaylov
312bb96eeb Merge branch 'master' of github.com:ClickHouse/ClickHouse into copier-stuck 2021-06-02 01:04:47 +03:00
Nikita Mikhaylov
f57863b974 fix tests 2021-06-02 01:03:08 +03:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs 
Poco::File to std::filesystem
 2021-05-31 23:17:02 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Nikita Mikhaylov
33dcebbb24 style 2021-05-31 17:41:35 +03:00
Nikita Mikhaylov
01511d0cbe better 2021-05-31 17:40:07 +03:00
Nikita Mikhaylov
1c548058e6 add status json 2021-05-31 17:38:55 +03:00
Nikita Mikhaylov
ac072243d6 style + enable trivial test 2021-05-31 17:38:55 +03:00
Nikita Mikhaylov
6d19dea761 better 2021-05-31 17:38:20 +03:00
Nikita Mikhaylov
e9f66a0045 delete alter table optimize 2021-05-31 17:38:20 +03:00
Nikita Mikhaylov
fa22b41d5a added allow_to_drop_target_partitions with test 2021-05-31 17:38:20 +03:00
Nikita Mikhaylov
9022646633 support copying of tables with data skipping indices 2021-05-31 17:38:20 +03:00
Nikita Mikhaylov
24af47063e allow to copy materialized and alias 2021-05-31 17:37:10 +03:00
Nikita Mikhaylov
c41cc36046 support TTL on columns 2021-05-31 17:37:10 +03:00
Nikita Mikhaylov
90ab394769 better 2021-05-31 17:37:10 +03:00
Nikita Mikhaylov
2017d2f918 save 2021-05-31 17:37:10 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
kssenii
31107816bb fs::absolute to fs::canonical 2021-05-24 20:37:54 +03:00
kssenii
2124113aa2 Update programs/* 2021-05-17 01:06:09 +03:00
Amos Bird
264cff6415
Projections 
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
 2021-05-11 18:12:23 +08:00
Alexey Milovidov
4ff812db7f Maybe better support for paths with whitespaces 2021-04-24 22:47:52 +03:00
Egor O'Sten
c77dd36ca9 Readability of error message for ORDER BY 2021-04-11 20:40:00 -04:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT 
It executes the code in the dtor, that should never throw.
 2021-04-06 21:10:37 +03:00
Nikita Mikhaylov
18dc213cee
Merge pull request #21912 from songenjie/clickhouse-copier-create-destination-once 
[ClickHouse][Copier] Improve copier work
 2021-03-30 14:42:00 +03:00
turbo jason
22c9cb1aa3 fix code style 2021-03-23 15:35:15 +08:00
turbo jason
f4be05ca25 fix code style 2021-03-23 15:04:25 +08:00
turbo jason
6aa89d0ee3 remove duplicated code 2021-03-23 11:00:06 +08:00
turbo jason
1b1d425e42 remove duplicated code 2021-03-22 10:27:32 +08:00
turbo jason
a4d3732091 [ClickHouse][Copier] Improve copier work 2021-03-19 20:51:29 +08:00
Anton Popov
bc417cf54a refactoring of serializations 2021-03-09 17:46:52 +03:00
madianjun
198a8d3b27 Drop helping tables when the whole table is done in clickhouse-copier 2021-02-08 10:54:03 +08:00
alexey-milovidov
8c0ec5105b
Add a patch from @FishermanZzhang, #19952 2021-02-02 06:53:11 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers 
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
 2020-12-12 17:43:10 +03:00
Nikita Mikhaylov
d6dd133bee
Merge pull request #17363 from nikitamikhaylov/copier-uninitialized-var 
Initialize unitialized variable
 2020-11-25 15:12:42 +03:00
chenqi
302cd55f45 Fix #15235. When clickhouse-copier handle non-partitioned table, throw segfault error. 2020-11-25 11:52:00 +08:00
nikitamikhaylov
ebd51db987 better 2020-11-24 21:22:50 +03:00
nikitamikhaylov
31505fb279 done 2020-11-24 17:40:38 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Alexey Milovidov
2e0a979e3a Fix inconsistency in FormatFactory 2020-10-29 06:39:43 +03:00
Ivan
1d170f5745
ASTTableIdentifier Part #1: improve internal representation of ASTIdentifier name (#16149) 
* Use only |name_parts| as primary name source

* Restore legacy logic for table restoration

* Fix build

* Fix tests

* Add pytest server config

* Fix tests

* Fixes due to review
 2020-10-24 21:46:10 +03:00
Nikita Mikhaylov
dbb607573b
Update Internals.cpp 2020-10-10 02:31:33 +03:00
nikitamikhaylov
b45a4a2cd1 better 2020-10-09 20:44:54 +03:00
nikitamikhaylov
d0b610c6b2 better backticks 2020-10-09 19:58:45 +03:00
Nikita Mikhaylov
a4832ed5ce
Merge branch 'master' into HyaZz-master 2020-09-28 13:55:45 +03:00
Alexander Tokmakov
f0f91f84af Merge branch 'master' into enable_atomic_database_by_default 2020-09-23 21:50:15 +03:00
Alexander Tokmakov
a8266a3d3c fix DDDL with cross-replication and Atomic 2020-09-23 21:28:59 +03:00
Alexander Tokmakov
13e41cdd20 try enable atomic database by default 2020-09-21 13:55:34 +03:00
Alexey Milovidov
08668de2c1 Remove previous modification 2020-09-17 17:39:22 +03:00
Alexey Milovidov
5ba1ad73d0 Merge branch 'back_quote_in_copier' into HyaZz-master 2020-09-17 17:38:48 +03:00
Alexey Milovidov
4cf131256a Merge branch 'master' of https://github.com/HyaZz/ClickHouse into HyaZz-master 2020-09-17 17:38:41 +03:00
Alexey Milovidov
c1e9ce2cf2 Remove questionable code 2020-09-17 17:38:06 +03:00
Alexander Kuzmenkov
fb64cf210a straighten the protocol version 2020-09-17 17:37:29 +03:00
alexey-milovidov
e101c912aa
Update Internals.cpp 2020-09-17 17:19:10 +03:00
20018712
88738cd610 when using copier，column name ,as sort key,with digital number as beginning,will cause a syntax error 2020-09-17 07:27:55 +08:00
Azat Khuzhin
0a34ae85a3 Use terminal width for the formatted help text of the clickhouse-copier 
Like other utils, clickhouse-client and so on (since copier uses Poco,
while other uses boost).
Otherwise clickhouse-copier --help is unreadable.
 2020-08-25 21:17:21 +03:00
Alexey Milovidov
0cbbe153cd Fix typos, the last 1% 2020-08-08 04:21:04 +03:00
Alexey Milovidov
1ae43b8beb Merge branch 'stack-frams-size-limit' of github.com:yandex/ClickHouse into stack-frams-size-limit 2020-08-02 04:36:12 +03:00
Alexey Milovidov
d3c0635fc0 Merge branch 'master' into stack-frams-size-limit 2020-08-02 04:35:09 +03:00
alexey-milovidov
e4f923097e
Update ClusterCopier.cpp 2020-08-02 01:43:43 +03:00