thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-10-04 23:50:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
49,029 Commits 8,976 Branches 1,768 Tags 1.9 GiB
f6f33168aa
Commit Graph

64 Commits

Author SHA1 Message Date
Alexander Kuzmenkov
fb64cf210a straighten the protocol version 2020-09-17 17:37:29 +03:00
Azat Khuzhin
0159c74f21 Secure inter-cluster query execution (with initial_user as current query user) [v3] 
Add inter-server cluster secret, it is used for Distributed queries
inside cluster, you can configure in the configuration file:

  <remote_servers>
      <logs>
          <shard>
              <secret>foobar</secret> <!-- empty -- works as before -->
              ...
          </shard>
      </logs>
  </remote_servers>

And this will allow clickhouse to make sure that the query was not
faked, and was issued from the node that knows the secret. And since
trust appeared it can use initial_user for query execution, this will
apply correct *_for_user (since with inter-server secret enabled, the
query will be executed from the same user on the shards as on initator,
unlike "default" user w/o it).

v2: Change user to the initial_user for Distributed queries if secret match
v3: Add Protocol::Cluster package
v4: Drop Protocol::Cluster and use plain Protocol::Hello + user marker
v5: Do not use user from Hello for cluster-secure (superfluous)
 2020-09-15 01:36:28 +03:00
Azat Khuzhin
a588947fe2 Fix DistributedFilesToInsert metric (zeroed when it should not) 
CurrentMetrics::Increment add amount for specified metric only for the
lifetime of the object, but this is not the intention, since
DistributedFilesToInsert is a gauge and after #10263 it can exit from
the callback (and enter again later, for example after SYSTEM STOP
DISTRIBUTED SEND it will always exit from it, until SYSTEM START
DISTRIBUTED SEND).

So make Increment member of a class (this will also fix possible issues
with substructing value on DROP TABLE).
 2020-08-27 00:43:00 +03:00
Alexander Tokmakov
dd4b8b9663 fix lock order inversion when renaming distributed table 2020-08-20 16:36:22 +03:00
Alexey Milovidov
12f66fa82c Fix 99% of typos 2020-08-08 04:01:47 +03:00
Vitaly Baranov
56665a15f7 Rework and rename the template class SettingsCollection => BaseSettings. 2020-07-31 20:54:18 +03:00
Vladimir Chebotarev
faedb04722 Minor fixes. 2020-07-28 19:45:46 +03:00
Vladimir Chebotarev
1b3f5c99f5 Real fix of test. 2020-07-26 21:27:36 +03:00
Alexey Milovidov
73a5c38398 Fix potential overflow in integer division #12119 2020-07-05 03:29:03 +03:00
alesapin
ebb36bec8a Merge branch 'master' into atomic_metadata5 2020-06-18 11:57:16 +03:00
alesapin
1ddeb3d149 Buildable getSampleBlock in StorageInMemoryMetadata 2020-06-16 18:51:29 +03:00
Azat Khuzhin
c139a05370 Forward declaration in StorageDistributed 2020-06-14 01:09:21 +03:00
Azat Khuzhin
d2383f0f5d Fix async INSERT into Distributed for prefer_localhost_replica=0 and w/o internal_replication 2020-06-08 21:58:56 +03:00
Azat Khuzhin
86c5465bf8 Rewrite StorageSystemDistributionQueue interfaces 2020-06-04 03:04:32 +03:00
Azat Khuzhin
f0050adc51 Make system.distribution_queue metrics non racy 2020-06-04 02:36:16 +03:00
Azat Khuzhin
09c3ca9c6c Add last_exception into system.distribution_queue 2020-06-04 02:36:16 +03:00
Azat Khuzhin
389f78ceee Add system.distribution_queue 
system.distribution_queue contains the following columns:
- database
- table
- data_path
- is_blocked
- error_count
- data_files
- data_compressed_bytes
 2020-06-04 02:36:16 +03:00
alesapin
6253e9b97b Revert disabled tests 2020-06-02 21:41:29 +03:00
Alexey Milovidov
25f941020b Remove namespace pollution 2020-05-31 00:57:37 +03:00
Alexey Milovidov
146370934a Keep the value of DistributedFilesToInsert metric on exceptions 2020-05-27 13:07:38 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
ce0619dabf Progress on task 2020-05-24 00:26:45 +03:00
Alexey Milovidov
2d7d5a1547 Apply all transformations again 2020-05-24 00:16:27 +03:00
Alexey Milovidov
bab24879e9 Progress on task 2020-05-24 00:16:05 +03:00
Alexey Milovidov
e1695feb7f Apply all transformations again 2020-05-23 23:40:32 +03:00
Alexey Milovidov
85f84550ba Progress on task 2020-05-23 23:37:37 +03:00
Alexey Milovidov
7e2fb9ad65 Apply all transformations again 2020-05-23 22:38:30 +03:00
Alexey Milovidov
eacff92d0e Progress on task 2020-05-23 22:35:08 +03:00
Alexey Milovidov
29762240de Remove duplicate whitespaces (preparation) 2020-05-23 22:31:54 +03:00
Alexey Milovidov
7fed65cbe2 Remove duplicate whitespaces (preparation) 2020-05-23 22:14:58 +03:00
Alexey Milovidov
ab0562a574 Make all LOG in single line (preparation) 2020-05-23 22:05:41 +03:00
Alexey Milovidov
a2ad11897f Remove duplicate whitespaces (preparation) 2020-05-23 21:53:58 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
8042e5febe find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/' 2020-05-23 19:58:15 +03:00
Alexey Milovidov
e391b77d81 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/' 2020-05-23 19:56:05 +03:00
Alexey Milovidov
8d2e80a5e2 find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/' 2020-05-23 19:42:39 +03:00
Azat Khuzhin
d93b9a57f6 Forward declaration for Context as much as possible. 
Now after changing Context.h 488 modules will be recompiled instead of 582.
 2020-05-21 01:53:18 +03:00
alexey-milovidov
7cf3538840
Merge pull request #10270 from ClickHouse/quota-key-in-client 
Support quota_key for Native client
 2020-05-17 14:09:40 +03:00
alexey-milovidov
7ee35f102d
Merge pull request #10867 from azat/dist-INSERT-load-balancing 
Respect prefer_localhost_replica/load_balancing on INSERT into Distributed
 2020-05-17 11:11:35 +03:00
Alexey Milovidov
397859ccb8 Fix error 2020-05-17 08:45:20 +03:00
Azat Khuzhin
2498041cc1 Avoid sending partially written files by the DistributedBlockOutputStream 2020-05-16 01:00:42 +03:00
Azat Khuzhin
52d73c7f45 Fix prefer_localhost_replica=0 and load_balancing for Distributed INSERT 2020-05-14 03:29:03 +03:00
Azat Khuzhin
e1d4837753 Fix list of possible nodes for Distributed INSERT for internal_replication=0 2020-05-14 03:28:08 +03:00
Azat Khuzhin
cdf3845e43 Respect load_balancing in DirectoryMonitor, to fix w/o internal_replication 2020-05-14 01:33:25 +03:00
Azat Khuzhin
085bafad05 Handle prefer_localhost_replica on INSERT into Distributed 
Right now it will issue remote send even if finally the local replica
will be selected - not good I guess.

This should also fix load_balancing.
 2020-05-13 01:38:03 +03:00
Azat Khuzhin
889f54b549 Fix ENOENT exception on current_batch.txt in DirectoryMonitor 
current_batch.txt will not exist if there was no send, this is the case
when all batches that was pending has been marked as pending.
 2020-05-13 01:23:18 +03:00
alexey-milovidov
ddc84163a7
Merge pull request #10486 from azat/dist-send-on-INSERT 
Fix distributed send that are scheduled by INSERT query
 2020-05-11 06:28:35 +03:00
Azat Khuzhin
5c89cdbe61 Fix distributed send retries on distributed_directory_monitor_{max_,}sleep_time_ms > 5min 
In this case error_count can be decreased before checking it for
rescheduling send.

And actually this can be a problem not only when
distributed_directory_monitor_{max_,}sleep_time_ms > 5min, because all
threads can be occupated and the real timeout between sends will be > 5min.
 2020-05-10 12:37:38 +03:00
Gleb Novikov
c637d99e07 Volumes and storages refactoring: 
1. Moved Volume to separate file
  2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
  3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
  4. Removed old Volume completely
  5. Moved StoragePolicy to separated files
  6. Moved DiskSelector to separated files
  7. Removed DiskSpaceMonitor file
 2020-05-04 23:15:38 +03:00
Azat Khuzhin
6ffdd53b6a Share auto-increment for first batch and tmp file in DistributedBlockOutputStream 2020-05-03 14:47:59 +03:00