Commit Graph

582 Commits

Author SHA1 Message Date
kssenii
4c4f765aba Globs for url table function 2021-04-21 12:32:57 +00:00
alexey-milovidov
cbbff1bc4b
Merge pull request #23319 from ClickHouse/mysql-insert-without-database
Allow to INSERT into remote MySQL storage with empty database #19281
2021-04-20 19:30:44 +03:00
Nikita Mikhaylov
f55eacee7d
Merge pull request #22988 from excitoon-favorites/s3connectionresetretries
Retries on HTTP connection drops during reads from S3
2021-04-20 14:59:21 +03:00
Kseniia Sumarokova
b43656bf89
Merge pull request #22697 from TCeason/feature/add_MaterializeMySQL_sync_bytes_judgment
Add MySQL read history data bytes judgment
2021-04-20 10:26:42 +03:00
Alexey Milovidov
ac68dd8e97 Fix bad code; allow to INSERT into MySQL without database name 2021-04-20 00:29:37 +03:00
Vladimir Chebotarev
9d4295f261 Removed infinite retries. 2021-04-19 11:02:36 +03:00
TCeason
472c131420 Add MySQL read history data bytes judgment 2021-04-19 10:51:50 +08:00
kssenii
88b50621d4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-18 14:40:37 +00:00
ruct
303813c1d8 Maybe 2021-04-16 16:07:56 +03:00
ruct
adbd2d65c2 Try 2021-04-16 16:07:56 +03:00
Alexander Tokmakov
ebf0a3119e fix data_type_default_nullable on attach 2021-04-16 13:29:33 +03:00
Vladimir Chebotarev
f3e152739b Rebase fix. 2021-04-15 14:00:59 +03:00
Vladimir Chebotarev
10bf56e540 Attempt to add retries to S3 reads. 2021-04-15 09:47:19 +03:00
ruct
2192ad4ca1 Init commit 2021-04-14 20:51:55 +03:00
Nikita Mikhaylov
2a86d76ccd style 2021-04-13 23:19:04 +03:00
Nikita Mikhaylov
ec35a878d3 simplify storages3cluster 2021-04-13 23:17:25 +03:00
Nikita Mikhaylov
5aac762d9c apply suggestion from Maksim Polyanskiy 2021-04-13 22:57:01 +03:00
Nikita Mikhaylov
024374a2ec review fixes 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
f36a715c32 delete unused code 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
a15757a9c9 better renaming 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
09a62e713a rename to s3Cluster 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
a743442a17 build fixes 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
75230e3701 better 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
7a68820342 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
c333c3dedb review fixes 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
36a8419f60 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
8a4b5a586e fixed uuid 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
7276b40556 better 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
587fbdd10d better 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4843f86329 use only one connection between initiator and worker 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
b3094412b1 better 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4e4b383214 added hash of itiator address 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
2549468c14 better 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
64b4cd0e63 save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
31b4f9b17f save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
44ca65a9a4 save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
081ea84a41 save 2021-04-13 22:39:41 +03:00
kssenii
c1adece94f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-11 06:00:18 +00:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
kssenii
8586565ce9 Move connection-related filed to Core/PostgreSQL 2021-04-08 12:43:02 +00:00
kssenii
9fa21a6272 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-08 12:18:54 +00:00
alesapin
86a843bb51
Merge pull request #21509 from kssenii/library-bridge
clickhouse-library-bridge for library dictionary source
2021-04-06 12:26:08 +03:00
kssenii
9dba529494 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-04-05 13:14:51 +00:00
kssenii
2de6b99f84 Better names 2021-04-02 16:12:14 +00:00
kssenii
522d170588 Add default port 2021-04-01 11:30:20 +00:00
kssenii
747967a9bb Better 2021-04-01 10:27:24 +00:00
kssenii
ba5ddd5c32 Do not parse replicas in PoolWithFailover 2021-03-31 14:03:52 +00:00
kssenii
22b515fbc9 Add namespace, simplify names 2021-03-27 20:14:02 +00:00
kssenii
f40c582e7a Replicas/shards for postgres storage 2021-03-27 20:11:48 +00:00
kssenii
ae868208c2 Use pool with failover in mysql storage 2021-03-27 14:39:45 +00:00
tavplubix
ae62bee939
Merge pull request #21913 from ClickHouse/fix_context_ref
Fix possibly dangling reference to Context
2021-03-26 10:39:13 +03:00
kssenii
d9326cc229 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-24 18:28:55 +00:00
Alexander Tokmakov
c938f4f2fe fix 2021-03-23 14:29:29 +03:00
Alexander Kuzmenkov
2f5dbf57b6
Merge pull request #21839 from kssenii/add-postgres-connection-pool
Add connection pool for postgres engine
2021-03-22 19:49:51 +03:00
Alexander Tokmakov
fc3e11a068 fix 2021-03-22 17:09:38 +03:00
Maksim Kita
849a980644 Updated ExternalDictionariesLoader interface with context parameter 2021-03-20 18:02:09 +03:00
kssenii
6bb81630e5 Small fixes 2021-03-20 10:32:11 +00:00
kssenii
619f0f51fd Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-20 09:47:05 +00:00
Maksim Kita
32831e37ba Added table function dictionary 2021-03-19 15:47:27 +03:00
kssenii
ae64a24844 Add connection pool 2021-03-17 13:55:47 +00:00
kssenii
34dea63248 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-17 13:32:43 +00:00
kssenii
fb5c871140 Merge branch 'non-default-schema' of https://github.com/kssenii/ClickHouse into non-default-schema 2021-03-16 11:17:45 +00:00
kssenii
f8a99804c1 Add double quotes 2021-03-16 11:15:43 +00:00
tavplubix
9e8ebb5e2b
Update TableFunctionPostgreSQL.cpp 2021-03-15 20:07:36 +03:00
kssenii
797063ae0f Support non-default table schema for postgres storage/table-function 2021-03-14 10:35:10 +00:00
kssenii
323fb54a8e Fix split build finally 2021-03-12 21:12:34 +00:00
kssenii
70f8f90511 Fix split build 2021-03-12 08:04:20 +00:00
kssenii
61d8e27ea7 Common base for bridges part 2 2021-03-07 11:21:49 +00:00
kssenii
5bc0010f94 Better 2021-02-12 21:14:36 +00:00
Azat Khuzhin
5001b19613 Accept arbitrary numeric types for numbers() arguments (for scientific notation)
This is to make the syntax simpler, i.e. avoid explicit cast to UInt64
if you want to use scientific notation (i.e. 1e9 over 1 000 000 000).

v2: use plain evaluateConstantExpression() over
evaluateConstantExpressionOrIdentifierAsLiteral() since identifier will
not work anyway
2021-02-10 22:32:32 +03:00
Amos Bird
824aee4020
better cte and with global 2021-01-26 10:29:11 +08:00
alexey-milovidov
ba3e064a63
Merge pull request #19371 from kssenii/test-coverage-with-factories
test coverage with factories
2021-01-25 07:11:09 +03:00
kssenii
a98c6d333c Make sure not to log non-existent objects 2021-01-23 12:05:37 +00:00
kssenii
827ccc5395 Fix failing stateless test (tsan) 2021-01-22 12:19:06 +00:00
kssenii
daab2c91bb Better 2021-01-21 21:15:11 +00:00
kssenii
c1702f34ee Add factories info into system.query_log 2021-01-21 15:46:37 +00:00
Alexey Milovidov
56012402e2 Fix potential nullptr dereference in table function VALUES 2021-01-21 15:19:00 +03:00
Alexander Kuzmenkov
d22c04568d
Merge pull request #13405 from excitoon-favorites/s3keepalive
Connection pools for S3
2021-01-18 12:52:37 +03:00
alexey-milovidov
7de745ce77
Merge pull request #18554 from kssenii/pg2ch
Add PostgreSQL table function, dictionary source, database engine
2021-01-16 23:55:05 +03:00
Alexey Milovidov
67f207d8ca More safe connection string 2021-01-16 00:18:51 +03:00
Alexey Milovidov
ef319bfa75 Slightly better 2021-01-15 23:33:04 +03:00
Alexey Milovidov
ba1921f380 Minor changes 2021-01-15 22:59:49 +03:00
feng lv
dbb3c89b50 Merge branch 'master' of github.com:ClickHouse/ClickHouse into insert-cluster 2021-01-15 15:37:56 +00:00
Alexey Milovidov
a19e7edd14 Merge branch 'master' into kssenii-pg2ch 2021-01-15 17:33:19 +03:00
kssenii
38a9cba850 Fix 2021-01-11 10:55:38 +00:00
kssenii
d952b0897e Minor adjustments 2021-01-10 12:06:18 +00:00
kssenii
0088d66d59 Hopefully better array conversions for insert query 2021-01-09 22:42:42 +00:00
Azat Khuzhin
b5ace27014 Add fsync support for Distributed engine.
Two new settings (by analogy with MergeTree family) has been added:

- `fsync_after_insert` - Do fsync for every inserted. Will decreases
  performance of inserts.

- `fsync_tmp_directory` - Do fsync for temporary directory (that is used
  for async INSERT only) after all part operations (writes, renames,
  etc.).

Refs: #17380 (p1)
2021-01-09 11:31:32 +03:00
Alexey Milovidov
8af19c3251 Fix Arcadia 2021-01-07 15:29:02 +03:00
Alexey Milovidov
60d4db421c Fix Arcadia 2021-01-07 06:45:12 +03:00
Vladimir Chebotarev
5cbc25c647 Added global setting s3_max_connections. 2021-01-05 13:34:55 +03:00
kssenii
9b25890674 Generate ya.make, fix fast test 2020-12-28 12:54:52 +00:00
kssenii
8f8920a7ee Add table cache, better drop table 2020-12-27 15:52:15 +00:00
kssenii
00a37404ca Add postgresql database engine 2020-12-27 12:26:29 +00:00
kssenii
aa3484515d Better 2020-12-27 12:17:23 +00:00
kssenii
69f6714461 Add table function 2020-12-27 12:16:46 +00:00
feng lv
257ca664ed fix 2020-12-21 11:32:44 +00:00
feng lv
b7d434a0ea try fix
fix

fix

fix
2020-12-21 10:42:21 +00:00
feng lv
8f17b3cb88 fix 2020-12-20 15:49:37 +00:00
feng lv
8ec695f024 add sharding_key for remote table function
update docs

fix
2020-12-20 15:29:46 +00:00
feng lv
c520d0efe3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into insert-cluster 2020-12-20 13:29:10 +00:00
feng lv
5a6a6991f1 support insert into cluster function 2020-12-20 07:32:44 +00:00
Alexey Milovidov
55604bde47 Whitespaces 2020-12-18 08:07:09 +03:00
alesapin
799997db35
Merge pull request #16621 from arenadata/ADQM-148
HDFS configuration and Kerberos Authenticaiton
2020-12-17 23:16:58 +03:00
Vitaly Baranov
da18af96a3 Fix access rights required for the merge() table function. 2020-12-17 12:29:07 +03:00
Ilya Golshtein
7e97814859 cleanup, fixes, new submodules, ShellCommand, WriteBufferFromString 2020-12-16 14:48:33 +03:00
alexey-milovidov
22aba554c0
Revert "Fix access rights required for the merge() table function." 2020-12-15 18:07:20 +03:00
Vitaly Baranov
5a8c750b0b
Merge pull request #17983 from vitlibar/fix-access-rights-for-merge-table-function
Fix access rights required for the merge() table function.
2020-12-15 09:17:52 +03:00
Pavel Kovalenko
346c241c20 Merge remote-tracking branch 'origin/master' into s3-adaptive-single-part-upload
# Conflicts:
#	src/Disks/S3/DiskS3.cpp
2020-12-14 11:00:30 +03:00
Vitaly Baranov
9310437920 Fix access rights required for the merge() table function. 2020-12-13 23:34:53 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
2020-12-12 17:43:10 +03:00
Pavel Kovalenko
db7a87089b Adaptive choose of single/multi part upload in WriteBufferFromS3. 2020-12-09 17:09:04 +03:00
Amos Bird
1d9d586e20
Make global_context consistent. 2020-11-20 18:23:14 +08:00
Alexey Milovidov
24f4fa6edf Follow Arcadia ya.make rules 2020-11-17 00:16:50 +03:00
Alexey Milovidov
3df04ce0c2 Follow Arcadia ya.make rules 2020-11-16 21:24:58 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
tavplubix
67099f28ac
Merge pull request #16591 from ClickHouse/aku/create-file
Support `SETTINGS` clause for File engine
2020-11-09 14:15:42 +03:00
Alexander Kuzmenkov
79f759fb6c Merge remote-tracking branch 'origin/master' into HEAD 2020-11-07 11:54:11 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
Alexander Kuzmenkov
14f31f5117 add support of settings to URL, fix url and file table functions 2020-11-05 14:28:20 +03:00
Alexey Milovidov
b56486510f Improve Arcadia 2020-11-05 12:55:01 +03:00
Alexander Kuzmenkov
548ac9d57b Merge remote-tracking branch 'origin/master' into HEAD 2020-11-02 17:37:30 +03:00
Alexander Kuzmenkov
99ee127620 Support SETTINGS clause for File engine
Accept the usual user settings related to file formats.

Most of the diff are the mechanistic code changes required to allow
providing the required FormatSettings to the format factory. The File
engine then extracts these settings from the `CREATE` query, and specifies
them when creating the format parser.
2020-11-02 10:50:38 +03:00
Maxim Akhmedov
3627fabfb9 Remove -g0 form Arcadia build settings. 2020-10-29 17:37:23 +03:00
Alexander Tokmakov
62a0ee8255 fix assertion in table function file() 2020-10-20 15:29:46 +03:00
Alexander Tokmakov
72b1339656 Revert "Revert "Write structure of table functions to metadata""
This reverts commit c65d1e5c70.
2020-10-14 15:19:29 +03:00
tavplubix
c65d1e5c70
Revert "Write structure of table functions to metadata" 2020-10-14 13:59:29 +03:00
Alexander Tokmakov
1ca9a92b21 Merge branch 'master' into write_structure_of_table_functions 2020-09-18 21:09:23 +03:00
tavplubix
1762535ffc
Merge pull request #14797 from Vxider/add_tablefunction_null
Add table function null
2020-09-18 20:34:35 +03:00
tavplubix
1f54fdfb7c
Update TableFunctionNull.cpp 2020-09-18 14:52:30 +03:00
Alexander Tokmakov
e1cf60990a minor fixes 2020-09-18 14:06:30 +03:00
Vxider
3da806f8f6 Throws LOGICAL_ERROR if argument is not a literal 2020-09-17 22:49:27 +08:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
Vxider
a913be9201 add table function null 2020-09-14 19:36:14 +08:00
alexey-milovidov
f60267a383
Merge pull request #14611 from amosbird/bf2
Treat query as function argument.
2020-09-09 21:20:33 +03:00
Alexander Tokmakov
d278d849e4 Merge branch 'master' into write_structure_of_table_functions 2020-09-09 17:12:35 +03:00
Vasily Nemkov
3973a17530
MySql datatypes dateTime64 and decimal (#11512) 2020-09-09 15:18:02 +03:00
Amos Bird
b40998ca00
Treat query as function argument. 2020-09-09 15:41:38 +08:00
Alexey Milovidov
e3924b8057 Fix "Arcadia" 2020-09-08 01:14:13 +03:00
Alexander Tokmakov
c5df4647e6 add workaround for memory leak 2020-09-07 14:30:12 +03:00
Alexander Tokmakov
c233be29c5 fix 2020-09-04 16:29:43 +03:00
Alexander Tokmakov
3cb73de059 try fix fast build 2020-09-04 01:23:27 +03:00
Alexander Tokmakov
df09bf3cb6 add suppressions for leak sanitizer 2020-09-03 22:50:26 +03:00
Alexander Tokmakov
9d98effa5e write structure for TableFunctionView 2020-09-02 20:39:37 +03:00
Alexander Tokmakov
b1d6ac0d14 Merge branch 'master' into write_structure_of_table_functions 2020-09-02 20:04:50 +03:00
Alexander Tokmakov
ee7b8a797a better code, fix Distributed format of StorageFile 2020-09-02 18:07:53 +03:00
Amos Bird
6bd753d85d
TableFunction view. 2020-09-02 16:02:10 +08:00
Alexander Tokmakov
3a9779adb4 slightly better code 2020-09-01 17:41:49 +03:00
Alexander Tokmakov
b6093d9a86 fix query header 2020-09-01 13:37:42 +03:00
Alexander Tokmakov
f0a5f19dae try use another wrapper for table functions 2020-08-31 22:03:26 +03:00
Alexander Tokmakov
b521ca9b18 save structure for all table functions 2020-08-28 00:05:52 +03:00
Alexander Tokmakov
969940b4c9 write table tructure for table function remote(...) 2020-08-26 23:55:40 +03:00
Alexey Milovidov
edd89a8610 Fix half of typos 2020-08-08 03:47:03 +03:00
Alexey Milovidov
6f690b7c0d Normalize ya.make files, fix "Arcadia" build 2020-08-02 16:57:38 +03:00
Peng Jian
2fc58e8f10 Fix code sytle check & build check 2020-07-17 11:33:29 +08:00
Peng Jian
7579d65ebc Registering StorageS3 and TableFunctionS3 under two different names (S3 and COSN). 2020-07-13 22:13:30 +08:00
Peng Jian
958bb1f3aa COS (Tencent Cloud Object Storage) integration
To allow import and export to COS.
2020-07-10 15:26:55 +08:00
Alexey Milovidov
de4459c25a Fix error 2020-07-06 04:25:45 +03:00
Alexey Milovidov
3ae9d3c9f3 Change exception code from LOGICAL_ERROR to BAD_ARGUMENTS when the name of remote table is empty 2020-07-06 02:13:44 +03:00
Alexander Tokmakov
a6b7010531 fix some logical errors 2020-06-24 23:51:17 +03:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
alexey-milovidov
e07d02015c
Merge pull request #11661 from ClickHouse/return-not-nullable-from-count-distinct-2
Return non-Nullable results from COUNT(DISTINCT), more complete.
2020-06-17 09:25:45 +03:00
Alexey Milovidov
6c93016c91 Add suggestion from @alesap 2020-06-16 22:16:01 +03:00
Alexey Milovidov
186d336e31 Use proper timeouts when communicating with xdbc-bridge 2020-06-16 01:35:15 +03:00
Alexey Milovidov
394fb64a9c Better way of implementation 2020-06-14 20:42:11 +03:00
Alexey Milovidov
63f2d92eff Better exception message when cannot parse columns declaration list #10403 2020-06-09 01:49:19 +03:00
Alexander Kuzmenkov
fafa953ee4 fixup 2020-06-06 04:23:21 +03:00
Alexander Kuzmenkov
3fd40098c4 fix same error in values() as well 2020-06-05 23:45:15 +03:00
Alexander Kuzmenkov
fc69811c7f generateRandom: check that all arguments are literals 2020-06-05 21:45:48 +03:00
Alexey Milovidov
a6bf4b5d70 Fix error with Lazy database 2020-06-02 05:06:16 +03:00
Alexander Tokmakov
3cff070f89 add context to IDatabase methods 2020-05-28 23:10:45 +03:00
Azat Khuzhin
d93b9a57f6 Forward declaration for Context as much as possible.
Now after changing Context.h 488 modules will be recompiled instead of 582.
2020-05-21 01:53:18 +03:00
Ivan
85d783c247
Poco contrib refactoring (#10396)
* Remove config_common.h
* Refactor libcpuid contrib
* Remove support for libcpuinfo
* Define USE_CPUID in Arcadia
* Refactor Poco libraries
2020-05-08 17:11:19 +03:00
Alexey Milovidov
99c18c5a09 Check for suspicious codecs #4966 2020-05-04 03:11:49 +03:00
Alexander Tokmakov
b29bddac12 Merge branch 'master' into database_atomic 2020-04-20 14:09:09 +03:00
alexey-milovidov
17e7d4d88a
Merge pull request #10307 from abyss7/arcadia-4
Changes for auto-sync with Arcadia
2020-04-17 05:08:34 +03:00
Ivan Lezhankin
e230632645 Changes required for auto-sync with Arcadia 2020-04-16 15:31:57 +03:00
Alexey Milovidov
cdeda4ab91 Fix usage of max_parser_depth setting; remove harmful default function arguments 2020-04-16 04:06:10 +03:00
Alexander Tokmakov
dd1590830b Merge branch 'master' into database_atomic 2020-04-08 22:00:46 +03:00
Vitaly Baranov
e573549945 Rework access rights for table functions. 2020-04-07 23:31:59 +03:00
Alexander Tokmakov
08bae4668d Merge branch 'master' into database_atomic 2020-04-06 16:18:07 +03:00
Ivan Lezhankin
06446b4f08 dbms/ → src/ 2020-04-03 18:14:31 +03:00