turbo jason
a4d3732091
[ClickHouse][Copier] Improve copier work
2021-03-19 20:51:29 +08:00
Anton Popov
bc417cf54a
refactoring of serializations
2021-03-09 17:46:52 +03:00
madianjun
198a8d3b27
Drop helping tables when the whole table is done in clickhouse-copier
2021-02-08 10:54:03 +08:00
alexey-milovidov
8c0ec5105b
Add a patch from @FishermanZzhang, #19952
2021-02-02 06:53:11 +03:00
Azat Khuzhin
5b3ab48861
More forward declaration for generic headers
...
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)
And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)
Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).
Approx results (How much units will be recompiled after changing file X?):
- ConnectionTimeouts.h
- mainline: 100
- Context.h:
- mainline: ~800
- patched: 415
- Settings.h:
- mainline: 900-1K
- patched: 440 (most of them because of the Context.h)
2020-12-12 17:43:10 +03:00
Nikita Mikhaylov
d6dd133bee
Merge pull request #17363 from nikitamikhaylov/copier-uninitialized-var
...
Initialize unitialized variable
2020-11-25 15:12:42 +03:00
chenqi
302cd55f45
Fix #15235 . When clickhouse-copier handle non-partitioned table, throw segfault error.
2020-11-25 11:52:00 +08:00
nikitamikhaylov
ebd51db987
better
2020-11-24 21:22:50 +03:00
nikitamikhaylov
31505fb279
done
2020-11-24 17:40:38 +03:00
Alexander Tokmakov
b94cc5c4e5
remove more stringstreams
2020-11-10 21:22:26 +03:00
Alexey Milovidov
2e0a979e3a
Fix inconsistency in FormatFactory
2020-10-29 06:39:43 +03:00
Ivan
1d170f5745
ASTTableIdentifier Part #1 : improve internal representation of ASTIdentifier name ( #16149 )
...
* Use only |name_parts| as primary name source
* Restore legacy logic for table restoration
* Fix build
* Fix tests
* Add pytest server config
* Fix tests
* Fixes due to review
2020-10-24 21:46:10 +03:00
Nikita Mikhaylov
dbb607573b
Update Internals.cpp
2020-10-10 02:31:33 +03:00
nikitamikhaylov
b45a4a2cd1
better
2020-10-09 20:44:54 +03:00
nikitamikhaylov
d0b610c6b2
better backticks
2020-10-09 19:58:45 +03:00
Nikita Mikhaylov
a4832ed5ce
Merge branch 'master' into HyaZz-master
2020-09-28 13:55:45 +03:00
Alexander Tokmakov
f0f91f84af
Merge branch 'master' into enable_atomic_database_by_default
2020-09-23 21:50:15 +03:00
Alexander Tokmakov
a8266a3d3c
fix DDDL with cross-replication and Atomic
2020-09-23 21:28:59 +03:00
Alexander Tokmakov
13e41cdd20
try enable atomic database by default
2020-09-21 13:55:34 +03:00
Alexey Milovidov
08668de2c1
Remove previous modification
2020-09-17 17:39:22 +03:00
Alexey Milovidov
5ba1ad73d0
Merge branch 'back_quote_in_copier' into HyaZz-master
2020-09-17 17:38:48 +03:00
Alexey Milovidov
4cf131256a
Merge branch 'master' of https://github.com/HyaZz/ClickHouse into HyaZz-master
2020-09-17 17:38:41 +03:00
Alexey Milovidov
c1e9ce2cf2
Remove questionable code
2020-09-17 17:38:06 +03:00
Alexander Kuzmenkov
fb64cf210a
straighten the protocol version
2020-09-17 17:37:29 +03:00
alexey-milovidov
e101c912aa
Update Internals.cpp
2020-09-17 17:19:10 +03:00
20018712
88738cd610
when using copier,column name ,as sort key,with digital number as beginning,will cause a syntax error
2020-09-17 07:27:55 +08:00
Azat Khuzhin
0a34ae85a3
Use terminal width for the formatted help text of the clickhouse-copier
...
Like other utils, clickhouse-client and so on (since copier uses Poco,
while other uses boost).
Otherwise clickhouse-copier --help is unreadable.
2020-08-25 21:17:21 +03:00
Alexey Milovidov
0cbbe153cd
Fix typos, the last 1%
2020-08-08 04:21:04 +03:00
Alexey Milovidov
1ae43b8beb
Merge branch 'stack-frams-size-limit' of github.com:yandex/ClickHouse into stack-frams-size-limit
2020-08-02 04:36:12 +03:00
Alexey Milovidov
d3c0635fc0
Merge branch 'master' into stack-frams-size-limit
2020-08-02 04:35:09 +03:00
alexey-milovidov
e4f923097e
Update ClusterCopier.cpp
2020-08-02 01:43:43 +03:00
Alexey Milovidov
47e9ec3d58
Smaller stack frames
2020-08-01 17:55:20 +03:00
Alexey Milovidov
4fa0b16eaa
Revert "Smaller stack frames"
...
This reverts commit 73c349fd7f
.
2020-08-01 17:54:51 +03:00
Alexey Milovidov
73c349fd7f
Smaller stack frames
2020-08-01 00:23:16 +03:00
Vladimir Chebotarev
a9e0a5d95b
Test fix.
2020-07-26 21:27:36 +03:00
Nikita Mikhaylov
d31ed58f01
done
2020-07-06 17:33:31 +03:00
Alexey Milovidov
176a7f2f72
Normalize "pid" file handling #3501
2020-07-04 16:54:24 +03:00
alesapin
5d3de747cd
Merge remote-tracking branch 'origin/master' into consistent_metadata4
2020-06-13 11:50:27 +03:00
Alexey Milovidov
72257061d5
Avoid errors due to implicit int<->bool conversions when using ZK API
2020-06-12 18:09:12 +03:00
alesapin
8be957ecb5
Better checks around metadata
2020-06-10 14:16:31 +03:00
alexey-milovidov
3de876b2bc
Merge pull request #11313 from ClickHouse/fix-race-condition-cluster-copier
...
Fix data race in clickhouse-copier
2020-05-31 17:00:54 +03:00
Alexey Milovidov
26100b64de
Remove function parameter
2020-05-30 23:30:08 +03:00
alexey-milovidov
3eea042d16
Merge pull request #11243 from ClickHouse/remove-experimental-use-processors-flag-4
...
Remove some code.
2020-05-30 21:05:14 +03:00
Alexey Milovidov
ff8a1c04fb
Fix data race in clickhouse-copier
2020-05-30 20:53:55 +03:00
Nikolai Kochetov
134f9f2b5b
Fix copier.
2020-05-30 11:07:12 +03:00
Nikolai Kochetov
96568566ee
Fix copier.
2020-05-30 10:50:45 +03:00
Nikolai Kochetov
9a3d449da0
Fix copier.
2020-05-30 10:36:00 +03:00
Nikolai Kochetov
998a9f4c0b
Try fix copier tests.
2020-05-29 12:52:30 +03:00
Alexander Tokmakov
3cff070f89
add context to IDatabase methods
2020-05-28 23:10:45 +03:00
Alexey Milovidov
b99dd3661c
Fix bad whitespace
2020-05-24 01:25:25 +03:00
Alexey Milovidov
7e1813825b
Return old names of macros
2020-05-24 01:24:01 +03:00
Alexey Milovidov
61cc605ee7
Remove old method
2020-05-24 01:21:29 +03:00
Alexey Milovidov
18febd7b97
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | while read file; do perl -pne 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/${1}_FORMATTED(${2}, "${3}{}${5}{}${7}{}${9}{}${11}", ${4}, ${6}, ${8}, ${10});/' $file > ${file}.tmp; mv ${file}.tmp $file; done
2020-05-23 22:56:05 +03:00
Alexey Milovidov
cd6d3c7509
Progress on task
2020-05-23 22:55:06 +03:00
Alexey Milovidov
f13ccfc8e8
Progress on task
2020-05-23 22:43:13 +03:00
Alexey Milovidov
d5b7883451
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_[^\_(]+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_[^\_(]+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7{}\9", \4, \6, \8);/'
2020-05-23 22:41:08 +03:00
Alexey Milovidov
7e2fb9ad65
Apply all transformations again
2020-05-23 22:38:30 +03:00
Alexey Milovidov
29762240de
Remove duplicate whitespaces (preparation)
2020-05-23 22:31:54 +03:00
Alexey Milovidov
9d2a0d2dd7
Apply all transformations again
2020-05-23 21:59:49 +03:00
Alexey Milovidov
a2ad11897f
Remove duplicate whitespaces (preparation)
2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65
Make all LOG in single line (preparation)
2020-05-23 21:31:37 +03:00
Alexey Milovidov
533f86278a
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5{}\7", \4, \6);/'
2020-05-23 20:00:41 +03:00
Alexey Milovidov
8042e5febe
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/'
2020-05-23 19:58:15 +03:00
Alexey Milovidov
e391b77d81
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/'
2020-05-23 19:56:05 +03:00
Alexey Milovidov
ee4ffbc332
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/'
2020-05-23 19:47:56 +03:00
Alexey Milovidov
8d2e80a5e2
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/'
2020-05-23 19:42:39 +03:00
Alexey Milovidov
f54435e7fd
Fix clang-tidy
2020-05-18 11:08:55 +03:00
Alexey Milovidov
a576a4fbcd
Enable extra warnings for base, utils, programs
2020-05-10 01:59:34 +03:00
Azat Khuzhin
150dd81e3b
Fix build with unbundled Poco
2020-05-09 01:22:12 +03:00
Ivan
85d783c247
Poco contrib refactoring ( #10396 )
...
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
2020-05-08 17:11:19 +03:00
Nikita Mikhaylov
f1ee08778f
Merge pull request #10414 from nikitamikhaylov/copier-sample-offset
...
cluster copier with sample offset
2020-04-22 20:31:17 +04:00
alexey-milovidov
8b177b45ab
Update ClusterCopier.cpp
2020-04-22 07:38:39 +03:00
Nikita Mikhaylov
15de7e4fa7
first try
2020-04-21 20:37:40 +03:00
Nikolai Kochetov
84faa9af26
Merge branch 'master' into shared-context-lifetime
2020-04-17 19:13:13 +03:00
Nikolai Kochetov
d51a9c551b
Remove shared_ptr to SharedContext from Context.
2020-04-17 12:47:40 +03:00
alexey-milovidov
17e7d4d88a
Merge pull request #10307 from abyss7/arcadia-4
...
Changes for auto-sync with Arcadia
2020-04-17 05:08:34 +03:00
Alexey Milovidov
cdeda4ab91
Fix usage of max_parser_depth setting; remove harmful default function arguments
2020-04-16 04:06:10 +03:00
Ivan Lezhankin
910ab8a265
Move contrib/libpcg-random → base/pcg-random
2020-04-13 18:42:58 +03:00
Ivan
97f2a2213e
Move all folders inside /dbms one level up ( #9974 )
...
* Move some code outside dbms/src folder
* Fix paths
2020-04-02 02:51:21 +03:00