Commit Graph

43 Commits

Author SHA1 Message Date
Yatsishin Ilya
b5b65d2149 Merge remote-tracking branch 'origin' into clickhouse-help 2023-04-11 11:24:48 +00:00
Antonio Andelic
a4d1cd514d Refactor 2023-03-23 08:58:56 +00:00
Antonio Andelic
65b99ec6a7 Merge branch 'master' into alternative-keeper-configs 2023-03-22 12:22:57 +00:00
Yatsishin Ilya
80857638c1 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-02-22 13:58:17 +00:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
2023-02-05 20:49:34 +00:00
Yatsishin Ilya
98edb9f06b Update help for clickhouse tools and add test 2023-01-31 12:19:37 +00:00
李扬
2405291612
Update programs/copier/ClusterCopierApp.cpp
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2022-12-28 15:07:06 +08:00
taiyang-li
437c31eca3 merge master and fix conflicts 2022-12-28 15:02:37 +08:00
Azat Khuzhin
44f23c2568 Make disks checks only for clickhouse-server
This will fix clickhouse-disks

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
taiyang-li
64d7087084 merge master and fix conflict 2022-11-15 10:33:10 +08:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
taiyang-li
d78493eeb2 commit again 2022-03-08 16:41:16 +08:00
taiyang-li
e21ab223c5 finish dev 2022-03-08 16:11:21 +08:00
Azat Khuzhin
48e9456e75 Disable log_queries in clickhouse-copier
This will avoid calling Context::addQueryFactoriesInfo()

v2: disable it in ClusterCopier::reloadTaskDescription() too
v3: just create local context and pass it to the ClusterCopier instead
of global context
2021-11-15 10:32:30 +03:00
Azat Khuzhin
452b3b443d copier: add ability to configure retries and delays between them
The following options had been introduced:
- max-table-tries
- max-shard-partition-tries
- max-shard-partition-piece-tries-for-alter
- retry-delay-ms
2021-10-10 08:58:35 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Nikita Mikhaylov
312bb96eeb Merge branch 'master' of github.com:ClickHouse/ClickHouse into copier-stuck 2021-06-02 01:04:47 +03:00
Nikita Mikhaylov
1c548058e6 add status json 2021-05-31 17:38:55 +03:00
Nikita Mikhaylov
6d19dea761 better 2021-05-31 17:38:20 +03:00
Nikita Mikhaylov
90ab394769 better 2021-05-31 17:37:10 +03:00
Nikita Mikhaylov
2017d2f918 save 2021-05-31 17:37:10 +03:00
kssenii
31107816bb fs::absolute to fs::canonical 2021-05-24 20:37:54 +03:00
kssenii
2124113aa2 Update programs/* 2021-05-17 01:06:09 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT
It executes the code in the dtor, that should never throw.
2021-04-06 21:10:37 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
2020-12-12 17:43:10 +03:00
Alexey Milovidov
2e0a979e3a Fix inconsistency in FormatFactory 2020-10-29 06:39:43 +03:00
Alexander Kuzmenkov
fb64cf210a straighten the protocol version 2020-09-17 17:37:29 +03:00
Azat Khuzhin
0a34ae85a3 Use terminal width for the formatted help text of the clickhouse-copier
Like other utils, clickhouse-client and so on (since copier uses Poco,
while other uses boost).
Otherwise clickhouse-copier --help is unreadable.
2020-08-25 21:17:21 +03:00
Vladimir Chebotarev
a9e0a5d95b Test fix. 2020-07-26 21:27:36 +03:00
Alexey Milovidov
176a7f2f72 Normalize "pid" file handling #3501 2020-07-04 16:54:24 +03:00
Alexander Tokmakov
3cff070f89 add context to IDatabase methods 2020-05-28 23:10:45 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
18febd7b97 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | while read file; do perl -pne 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/${1}_FORMATTED(${2}, "${3}{}${5}{}${7}{}${9}{}${11}", ${4}, ${6}, ${8}, ${10});/' $file > ${file}.tmp; mv ${file}.tmp $file; done 2020-05-23 22:56:05 +03:00
Alexey Milovidov
f13ccfc8e8 Progress on task 2020-05-23 22:43:13 +03:00
Alexey Milovidov
a2ad11897f Remove duplicate whitespaces (preparation) 2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
f54435e7fd Fix clang-tidy 2020-05-18 11:08:55 +03:00
Nikita Mikhaylov
f1ee08778f
Merge pull request #10414 from nikitamikhaylov/copier-sample-offset
cluster copier with sample offset
2020-04-22 20:31:17 +04:00
Nikita Mikhaylov
15de7e4fa7 first try 2020-04-21 20:37:40 +03:00
Nikolai Kochetov
d51a9c551b Remove shared_ptr to SharedContext from Context. 2020-04-17 12:47:40 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up (#9974)
* Move some code outside dbms/src folder
* Fix paths
2020-04-02 02:51:21 +03:00