thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
59,264 Commits 9,326 Branches 1,783 Tags 2.1 GiB
831b90f272
Commit Graph

3223 Commits

Author SHA1 Message Date
tavplubix
9801bfc0ee
Merge branch 'master' into fix_18063 2021-01-22 17:28:05 +03:00
alexey-milovidov
6e1d660e36
Merge pull request #19390 from ClickHouse/http_referer 
Add http_referer to client info
 2021-01-22 17:25:09 +03:00
alexey-milovidov
9818bd319a
Merge pull request #19381 from azat/parts-types-metrics 
Add metrics for MergeTree parts (Wide/Compact/InMemory) types
 2021-01-22 17:24:50 +03:00
Alexander Kuzmenkov
b58a696b24
Merge pull request #19379 from azat/Buffer-less-lock-contention 
Reduce lock contention for multiple layers of the Buffer engine
 2021-01-22 13:17:34 +03:00
alexey-milovidov
4e11e7cfa9
Merge pull request #18772 from azat/optimize-memory-tracking-fix 
[RFC] Fix memory tracking for OPTIMIZE TABLE/merges
 2021-01-22 03:48:06 +03:00
Alexey Milovidov
afc6f2fd8e Add http_referer to client info 2021-01-22 01:55:45 +03:00
kssenii
daab2c91bb Better 2021-01-21 21:15:11 +00:00
Azat Khuzhin
b0a80af888 Reduce lock contention for multiple layers of the Buffer engine 
Otherwise you can see something like this for the following query:

    ```sql
    WITH
        arrayMap(x -> demangle(addressToSymbol(x)), s.trace) AS trace_array,
        arrayStringConcat(trace_array, '\n') AS trace_string
    SELECT
        p.thread_id,
        p.query_id,
        p.query,
        trace_string
    FROM
    (
        SELECT
            query_id,
            query,
            arrayJoin(thread_ids) AS thread_id
        FROM system.processes
    ) AS p
    INNER JOIN system.stack_trace AS s ON p.thread_id = s.thread_id
    ORDER BY p.query_id ASC
    SETTINGS enable_global_with_statement = 0, allow_introspection_functions = 1
    FORMAT PrettyCompactNoEscapes
    ```

Lots of the following:

    ```sql
    INSERT INTO buffer (...) VALUES

    __lll_lock_wait
    pthread_mutex_lock
    std::__1::mutex::lock()
    DB::StorageBuffer::reschedule()
    DB::BufferBlockOutputStream::write(DB::Block const&)
    ```

That will wait one of this:

    ```
    INSERT INTO buffer (...) VALUES

    ...
    DB::PushingToViewsBlockOutputStream::write(DB::Block const&)
    DB::AddingDefaultBlockOutputStream::write(DB::Block const&)
    DB::SquashingBlockOutputStream::finalize()
    DB::SquashingBlockOutputStream::writeSuffix()
    DB::PushingToViewsBlockOutputStream::writeSuffix()
    DB::StorageBuffer::writeBlockToDestination(DB::Block const&, std::__1::shared_ptr<DB::IStorage>)
    DB::StorageBuffer::flushBuffer(DB::StorageBuffer::Buffer&, bool, bool, bool)
    ```

P.S. we cannot simply unlock the buffer during flushing, see comments in
the code
 2021-01-21 22:09:24 +03:00
Azat Khuzhin
cb951c2116 Add metrics for MergeTree parts types 
- PartsWide
- PartsCompact
- PartsInMemory
 2021-01-21 21:17:00 +03:00
Azat Khuzhin
c68f7cd5b1 Measure time that spend during flush of the Buffer to the underlying 2021-01-21 21:11:39 +03:00
Pavel Kruglov
900580af02 Add parallel select when there is one part with level>0 in select final 2021-01-21 20:34:50 +03:00
kssenii
c1702f34ee Add factories info into system.query_log 2021-01-21 15:46:37 +00:00
alexey-milovidov
062f00aa5d
Merge pull request #19290 from azat/dist-broken-on-EOF-fix 
Do not mark file for distributed send as broken on EOF
 2021-01-21 17:00:36 +03:00
tavplubix
ac50b1b231
Update StorageMaterializedView.cpp 2021-01-21 15:57:18 +03:00
Anton Popov
ac3de63a71 fix several cases, while reading subcolumns 2021-01-21 15:34:11 +03:00
alesapin
fe6b964b32 Revert "Revert "Auto version update to [21.2.1.1] [54446]"" 
This reverts commit 42f63e14b5.
 2021-01-21 12:39:46 +03:00
Azat Khuzhin
5f3059555a Fix system.parts _state column 
There was LOGICAL_ERROR when querying this column, due to incorrect order:

    SELECT
        *,
        _state
    FROM system.parts

    2021.01.21 10:22:57.731556 [ 22851 ] {02a07c6d-467d-4681-9203-4dc11cc6fbee} <Fatal> : Logical error: 'Invalid Field get from type String to type UInt64'.
 2021-01-21 10:41:23 +03:00
Alexander Tokmakov
6560ec3ed5 fix segfault on aggregation when MV has unexpected structure 2021-01-20 19:36:18 +03:00
Anton Ivashkin
357d98eb36 Merge master 2021-01-20 12:23:03 +03:00
Azat Khuzhin
8a00816396 Do not mark file for distributed send as broken on EOF 
- the sender will got ATTEMPT_TO_READ_AFTER_EOF (added in
  946c275dfb) when the client just go
  away, i.e. server had been restarted, and this is incorrect to mark the
  file as broken in this case.

- since #18853 the file will be checked on the sender locally, and
  in case the file was truncated CANNOT_READ_ALL_DATA will be thrown.
  But before #18853 the sender will not receive
  ATTEMPT_TO_READ_AFTER_EOF from the client in case of file was truncated
  on the sender, since the client will just wait for more data, IOW just hang.

- and I don't see how ATTEMPT_TO_READ_AFTER_EOF can be received while
  reading local file.
 2021-01-20 01:10:17 +03:00
alesapin
70d9c7ea03
Merge pull request #19237 from ClickHouse/fix_wait_for_log_entries 
Don't wait forever for log update after table was dropped
 2021-01-19 10:22:52 +03:00
Anton Ivashkin
eba98b04b0 Zero copy replication over S3: Hybrid storage support 2021-01-18 19:16:45 +03:00
Alexander Tokmakov
7f97a11c84 Merge branch 'master' into database_replicated 2021-01-18 17:09:39 +03:00
alesapin
4ee96869a2 Don't wait forever for log update after table was dropped 2021-01-18 15:15:07 +03:00
Pavel Kovalenko
1e3a059f64 Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata 
# Conflicts:
#	src/Disks/DiskCacheWrapper.cpp
#	src/Disks/S3/DiskS3.cpp
 2021-01-18 13:39:49 +03:00
Alexander Kuzmenkov
d22c04568d
Merge pull request #13405 from excitoon-favorites/s3keepalive 
Connection pools for S3
 2021-01-18 12:52:37 +03:00
alexey-milovidov
1444ccb85e
Merge pull request #19206 from azat/dist-pull 
Tiny changes in DistributedBlockOutputStream
 2021-01-18 01:04:50 +03:00
alexey-milovidov
15f4ae26c2
Merge pull request #17310 from CurtizJ/multiple-nested 
Allow nested with multiple nesting and subcolumns of complex types
 2021-01-17 15:00:26 +03:00
Azat Khuzhin
a6631287a7 DistributedBlockOutputStream: add more comments 2021-01-17 12:50:37 +03:00
Azat Khuzhin
b725e1d131 DistributedBlockOutputStream: Remove superfluous brackets for string construction 2021-01-17 12:48:51 +03:00
Azat Khuzhin
2955e25e83 Fix inserted blocks accounting for insert_distributed_one_random_shard=1 
It is tricky due to block splitting

Refs: https://github.com/ClickHouse/ClickHouse/pull/18294
 2021-01-17 12:45:42 +03:00
Azat Khuzhin
858f07c796 Update comment for query AST cloning during inesrt into multiple local shards 
Refs: https://github.com/ClickHouse/ClickHouse/pull/18264#discussion_r558839456
 2021-01-17 12:40:41 +03:00
alexey-milovidov
7de745ce77
Merge pull request #18554 from kssenii/pg2ch 
Add PostgreSQL table function, dictionary source, database engine
 2021-01-16 23:55:05 +03:00
Alexey Milovidov
029302d766 Merge with master 2021-01-16 17:09:44 +03:00
Alexey Milovidov
24c8e53440 Merge branch 'master' into multiple-nested 2021-01-16 16:28:40 +03:00
alexey-milovidov
e67e4588e7
Update PostgreSQLConnection.h 2021-01-16 15:26:21 +03:00
alexey-milovidov
a15092eeb7
Merge pull request #18264 from ucasFL/insert-cluster 
Support insert into table function cluster
 2021-01-16 13:22:49 +03:00
alexey-milovidov
a5a19de878
Update DistributedBlockOutputStream.cpp 2021-01-16 13:22:25 +03:00
alexey-milovidov
4efc7a7dc3
Update MergeTreeIOSettings.h 2021-01-16 13:07:58 +03:00
alexey-milovidov
e32b1e3fe6
Update StorageTinyLog.cpp 2021-01-16 11:53:28 +03:00
Alexey Milovidov
4ee9c57190 Add TODO 2021-01-16 11:43:59 +03:00
Alexey Milovidov
b25e334313 Add assert 2021-01-16 11:30:35 +03:00
alexey-milovidov
2e2988e5d8
Merge pull request #19146 from azat/server-memory-limit-blocking 
MemoryTracker: Do not ignore server memory limits during blocking by default
 2021-01-16 11:09:19 +03:00
alesapin
67fd381034
Merge pull request #19123 from ClickHouse/additional_check_in_writer 
Fix max granules size in MergeTreeDataWriter
 2021-01-16 10:17:44 +03:00
alexey-milovidov
5f189c5756
Merge pull request #19122 from ClickHouse/data-part-better-code 
Add metrics for part number in MergeTree in ClickHouse
 2021-01-16 00:20:15 +03:00
alexey-milovidov
e09ec9fed5
Merge pull request #18373 from amosbird/fix-18364 
Fix 2-arg functions with constant in PK analysis
 2021-01-16 00:20:03 +03:00
Alexey Milovidov
67f207d8ca More safe connection string 2021-01-16 00:18:51 +03:00
Alexey Milovidov
52cfc1d110 Minor changes 2021-01-15 23:09:31 +03:00
Alexey Milovidov
ba1921f380 Minor changes 2021-01-15 22:59:49 +03:00
Azat Khuzhin
61b2d0ce42 MemoryTracker: Do not ignore server memory limits during blocking by default 2021-01-15 22:46:58 +03:00
Alexey Milovidov
b293d56a63 Merge branch 'master' into kssenii-pg2ch 2021-01-15 22:32:03 +03:00
alesapin
184dbedb06 Fix stupid error 2021-01-15 21:50:30 +03:00
alexey-milovidov
b97beea22a
Merge pull request #19101 from ClickHouse/check_compression_codec_read 
Fix compression codec read for empty files
 2021-01-15 20:55:58 +03:00
alexey-milovidov
971ff2ee0a
Merge pull request #19086 from ClickHouse/faster-parts-removal 
Faster parts removal, more safe and efficient interface of IDisk
 2021-01-15 20:37:35 +03:00
feng lv
9829c09720 fix 2021-01-15 15:54:35 +00:00
Alexey Milovidov
a19e7edd14 Merge branch 'master' into kssenii-pg2ch 2021-01-15 17:33:19 +03:00
alexey-milovidov
ca825f36f1
Merge pull request #18853 from azat/dist-send-checksums 
Improve checksum checks for async INSERT into Distributed on the sender
 2021-01-15 17:12:54 +03:00
alesapin
8ccaa6ede9 Additional check for huge granules in MergeTreeDataWriter 2021-01-15 15:40:37 +03:00
Alexey Milovidov
e238fd64ac Add part metrics 2021-01-15 15:28:53 +03:00
Alexey Milovidov
6a2a5e53ed Slightly better code of IMergeTreeDataPart #18955 2021-01-15 15:15:13 +03:00
alexey-milovidov
78fff6bc39
Merge branch 'master' into multiple-nested 2021-01-15 14:54:27 +03:00
Alexey Milovidov
4bae04d500 Merge branch 'master' into amosbird/fix-18364 2021-01-15 14:37:35 +03:00
alexey-milovidov
4a71971b43
Update KeyCondition.cpp 2021-01-15 14:36:07 +03:00
alexey-milovidov
8d58ce532a
Merge pull request #19064 from CurtizJ/restrict-modify-ttl 
Restrict MODIFY TTL for tables created in old syntax
 2021-01-15 14:09:47 +03:00
Alexey Milovidov
d553e46a06 Merge branch 'master' into faster-parts-removal 2021-01-15 13:25:20 +03:00
alexey-milovidov
461d370e8d
Merge pull request #19020 from ClickHouse/fix_18879 
Fix duplicate UUIDs of LiveView on server startup
 2021-01-15 13:04:08 +03:00
alesapin
d601faa669
Merge pull request #18935 from fastio/bugfix_attach_partition_does_not_reset_mutation 
Bugfix: attach partition should reset the mutation
 2021-01-15 12:21:16 +03:00
alesapin
e106df2ad0 Fix comment 2021-01-15 12:10:03 +03:00
alesapin
0662d6bd7d Fix compression codec read for empty files 2021-01-15 12:04:23 +03:00
Azat Khuzhin
ecae6c1c60 Avoid reading the distributed batch just to read the block header 
Before this patch batched mode of the DirectoryMonitor is 2x slower then
non-batched, after it should be more or less the same as non-batched.
 2021-01-14 22:38:46 +03:00
Alexey Milovidov
8276a1c8d2 Faster parts removal, more safe and efficient interface of IDisk 2021-01-14 19:24:13 +03:00
alesapin
09e825e28c
Merge pull request #19040 from ClickHouse/fix_redundant_exception_while_dropping_part 
Avoid redundant exception while dropping part
 2021-01-14 18:59:59 +03:00
Anton Popov
ac426c3da6 restrict MODIFY TTL for tables created in old syntax 2021-01-14 15:32:20 +03:00
Alexander Tokmakov
64975c08e7 fix 2021-01-14 13:33:39 +03:00
alexey-milovidov
2ca185c9f9
Merge pull request #16995 from sundy-li/alias_column_partition_prune 
Add optimize_respect_aliases
 2021-01-14 11:30:45 +03:00
Nikolai Kochetov
7b33ad5e44
Merge pull request #19010 from ClickHouse/fix-insert-lc-to-tiny-log 
Fix insert LC to TinyLog
 2021-01-14 11:25:38 +03:00
alesapin
bfc27254b2 Avoid redundant exception while dropping part 2021-01-14 11:07:13 +03:00
alexey-milovidov
3b02a3ef02
Merge pull request #18976 from ClickHouse/fix_terminate_when_not_enough_memory 
Finally get rid of allocations in ThreadPool::worker(...)
 2021-01-13 23:48:46 +03:00
Alexander Tokmakov
5f0f8ae3cd fix duplicate UUIDs of LiveView on server startup 2021-01-13 21:13:46 +03:00
Anton Popov
0e903552a0 fix TTLs with WHERE 2021-01-13 17:04:27 +03:00
Nikolai Kochetov
da0cb8d47f Added comment. 2021-01-13 16:32:27 +03:00
ygrek
8f2a830d83
add zstd long range option (#17184) 
* add zstd long compression option

* tests: add zstd long read-write test

Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com>
Co-authored-by: ip <igor@ahrefs.com>
 2021-01-13 16:22:59 +03:00
Nikolai Kochetov
12f3b22623 Do not skip streams after serializeBinaryBulkStatePrefix. 2021-01-13 15:20:10 +03:00
Alexander Tokmakov
a6510cc4b7 Merge branch 'master' into fix_terminate_when_not_enough_memory 2021-01-13 14:05:54 +03:00
alesapin
c97469773d
Merge pull request #18969 from ClickHouse/fix_drop_part_deduplication 
Fix DROP PART query break deduplication
 2021-01-13 11:50:26 +03:00
alesapin
73e536a074
Merge pull request #18928 from ClickHouse/more_checks_in_writer_wide 
More checks in merge tree writer wide
 2021-01-13 09:59:45 +03:00
Anton Popov
d7200ee2ed minor changes 2021-01-13 02:20:32 +03:00
Mike Kot
6b949109f1 marked places to edit 2021-01-12 21:46:03 +03:00
Pavel Kovalenko
b09862b7b9 Ability to backup-restore metadata files for DiskS3 (fixes and tests) 2021-01-12 20:18:40 +03:00
Anton Popov
15ead18673 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-12 19:46:10 +03:00
Anton Popov
60b88986bf minor changes near TTL computation 2021-01-12 19:42:49 +03:00
Alexander Tokmakov
ff6c3c75c2 add protection from unsafe allocations 2021-01-12 18:41:24 +03:00
Anton Popov
aed8c78d0d better check for existence of aggregate function 2021-01-12 18:35:07 +03:00
Anton Popov
58b9ef5a10 fix TTL info serialization 2021-01-12 17:14:47 +03:00
alesapin
dead1016d6 Fix deduplication block names parsing 2021-01-12 13:55:02 +03:00
sundy-li
ad01534bd5 Merge branch 'master' into alias_column_partition_prune 2021-01-12 18:28:30 +08:00
alexey-milovidov
2c71b997de
Merge pull request #18464 from hexiaoting/dev_fp_as_pk 
Disallow floating point as partition key
 2021-01-12 13:08:41 +03:00
alesapin
f3e55183ad Better test and check 2021-01-12 11:46:31 +03:00
tavplubix
1d6bfe8ad7
Merge pull request #18944 from ClickHouse/fix_mutation_commands_escaping 
Fix mutations text serialization
 2021-01-12 11:29:24 +03:00
alesapin
5ae31b8068 Slightly relax check 2021-01-12 10:59:14 +03:00
Anton Popov
61d6a323dd multiple TTL with GROUP BY 2021-01-12 03:40:07 +03:00
Anton Popov
5822ee1f01 allow multiple rows TTL with WHERE expression 2021-01-12 02:07:21 +03:00
Azat Khuzhin
05608687d6 Get back memory tracking blocker in calculateAndSerializePrimaryIndex() 
But reduce scope, to avoid leaking too much memory, since there are old
values in last_block_index_columns.

The scope of the MemoryTracker::BlockerInThread has been increased in #8290
 2021-01-12 01:00:37 +03:00
Azat Khuzhin
82edbfb581 Account query memory limits and sampling for OPTIMIZE TABLE/merges 2021-01-12 00:49:41 +03:00
Azat Khuzhin
bd05d9db2f Fix memory tracking for OPTIMIZE TABLE queries 
Because of BlockerInThread in
MergeTreeDataPartWriterOnDisk::calculateAndSerializePrimaryIndex memory
was accounted incorrectly and grows constantly.

And IIUC there is no need in that blocker, since INSERT SELECT shares
the same thread group.
 2021-01-12 00:49:40 +03:00
Pavel Kovalenko
0856b2c514 Ability to backup-restore metadata files for DiskS3 (fixes and tests) 2021-01-11 20:37:08 +03:00
alesapin
e3a50d13a5 Add missed header 2021-01-11 17:51:23 +03:00
alesapin
b1c4e5b9f2 Fix mutations text escaping 2021-01-11 17:43:05 +03:00
fastio
8dde70b937 Bugfix: attach partition should reset the mutation 2021-01-11 21:26:43 +08:00
alesapin
c5df8f324c More checks in writer wide 2021-01-11 15:03:00 +03:00
tavplubix
213c653ef8
Merge pull request #17656 from bharatnc/ncb/ddl-worker-queue-table 
add  system.distributed_ddl_queue table
 2021-01-11 14:59:26 +03:00
Anton Popov
91dc347ff3 add some comments 2021-01-11 14:36:31 +03:00
kssenii
38a9cba850 Fix 2021-01-11 10:55:38 +00:00
Anton Popov
36ae0e4d35 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-11 13:51:12 +03:00
Alexey Milovidov
42f63e14b5 Revert "Auto version update to [21.2.1.1] [54446]" 
This reverts commit bc2593688e.
 2021-01-11 11:32:52 +03:00
Nikolai Kochetov
ee094ed7fd
Merge pull request #18896 from amosbird/keyconditionalias 
correct index analysis of WITH aliases
 2021-01-11 11:13:28 +03:00
robot-clickhouse
bc2593688e Auto version update to [21.2.1.1] [54446] 2021-01-11 11:12:11 +03:00
Alexey Milovidov
c7d4d12f72 Auto version update to [21.1.0] [54445] 2021-01-11 03:51:08 +03:00
Azat Khuzhin
56475774d3 Fix readability-static-definition-in-anonymous-namespace in DirectoryMonitor 2021-01-10 23:57:40 +03:00
Azat Khuzhin
2565d2ac44 Verify compressed headers while sending distributed batches 
Before this patch the DirectoryMonitor was checking the compressed file
by reading it one more time (since w/o this receiver may stuck on
truncated file), while this is ineffective and we can just check the
checksums before sending.

But note that this may decrease batch size that is used for sending over
network.
 2021-01-10 21:23:42 +03:00
kssenii
6ec59f1304 Update libpq, tiny fix 2021-01-10 15:38:46 +00:00
Azat Khuzhin
819b9d7d56 Add more metadata into distributed .bin files to avoid doing the same on sending 
Before this patch StorageDistributedDirectoryMonitor reading .bin files
in batch mode, just to calculate number of bytes/rows, this is very
ineffective, let's just store them in the header (rows/bytes).
 2021-01-10 18:17:15 +03:00
Azat Khuzhin
fce8b6b5ef Refactoring distributed header parsing 2021-01-10 18:17:15 +03:00
Azat Khuzhin
676bc83c6d Check per-block checksum of the distributed batch on the sender before sending 
This is already done for distributed_directory_monitor_batch_inserts=1,
so let's do the same for the non batched mode, since otherwise in case
the file will be truncated the receiver will just stuck (since it will
wait for the block, but the sender will not send it).
 2021-01-10 18:17:14 +03:00
kssenii
c6c6b2d23c Fix style and types check 2021-01-10 13:01:15 +00:00
kssenii
d952b0897e Minor adjustments 2021-01-10 12:06:18 +00:00
alexey-milovidov
1921494bc9
Merge pull request #18864 from azat/dist-fsync 
Add fsync support for Distributed engine.
 2021-01-10 13:16:13 +03:00
alexey-milovidov
571e37188a
Merge pull request #18886 from ClickHouse/remove-useless-code-4 
Remove useless code
 2021-01-10 13:13:17 +03:00
Amos Bird
44758935df
correct index analysis of WITH aliases 2021-01-10 17:40:47 +08:00
Alexey Milovidov
c38dca155c Fix clang-tidy 2021-01-10 05:51:54 +03:00
Alexey Milovidov
76149947ef Remove useless headers 2021-01-10 05:48:57 +03:00
Alexey Milovidov
6eb5a5f4d9 Remove useless code 2021-01-10 03:28:59 +03:00
kssenii
0088d66d59 Hopefully better array conversions for insert query 2021-01-09 22:42:42 +00:00
alexey-milovidov
d593b1faf2
Merge pull request #18876 from ClickHouse/external-queries-in-with-table 
External queries: fix the case of expr IN table #9756
 2021-01-10 00:10:56 +03:00
Azat Khuzhin
471deab63a Rename fsync_tmp_directory to fsync_directories for Distributed engine 2021-01-09 17:51:30 +03:00
Azat Khuzhin
ae0b15455f Add fsync_tmp_directory support into DirectoryMonitor 2021-01-09 16:31:52 +03:00
Azat Khuzhin
2e55bd2285 Accept IDisk in DirectoryMonitor (for further fsync) 2021-01-09 16:31:42 +03:00
Azat Khuzhin
dd669cb2b6 Add fsync support for Distributed/DirectoryMonitor 
Note that there is no fsync_tmp_directory support in DirectoryMonitor
since you cannot propagate the error to user anyway.
 2021-01-09 15:26:25 +03:00
Azat Khuzhin
fbe5df809b Sync other temporary directories for Distributed fsync_tmp_directories 2021-01-09 11:36:04 +03:00
Azat Khuzhin
b5ace27014 Add fsync support for Distributed engine. 
Two new settings (by analogy with MergeTree family) has been added:

- `fsync_after_insert` - Do fsync for every inserted. Will decreases
  performance of inserts.

- `fsync_tmp_directory` - Do fsync for temporary directory (that is used
  for async INSERT only) after all part operations (writes, renames,
  etc.).

Refs: #17380 (p1)
 2021-01-09 11:31:32 +03:00
alexey-milovidov
b2577327f1
Merge pull request #18881 from ClickHouse/remove-useless-headers-2 
Remove useless headers
 2021-01-09 11:01:07 +03:00
alexey-milovidov
b60a37afc7
Merge pull request #18870 from azat/storage-features 
Storage features improvements
 2021-01-09 10:57:45 +03:00
alexey-milovidov
a7eccd2200
Merge pull request #18839 from fastio/expand_macros_for_fetchPartition 
Expand macros for fetchPartition
 2021-01-09 08:35:03 +03:00
Alexey Milovidov
8a57f8f42a Remove useless headers 2021-01-09 08:32:10 +03:00
Alexey Milovidov
8fb64472f2 External queries: fix the case of expr IN table #9756 2021-01-09 06:28:54 +03:00
alexey-milovidov
6dcb887eed
Merge pull request #18857 from amosbird/fixmaxpartitionremotesetting 
Correctly override default settings remotely
 2021-01-08 22:25:43 +03:00
Azat Khuzhin
714d5a067a Expose supports_parallel_insert via system.table_engines 2021-01-08 14:57:24 +03:00
Azat Khuzhin
c7d0c2d075 Add some comments for StorageFeatures 2021-01-08 14:36:42 +03:00
Azat Khuzhin
64cb6405ac Drop IStorage::supportsSettings() (replaced with StorageFeatures::supports_settings) 2021-01-08 14:36:26 +03:00
fastio
a1d0c04e68 fix build 2021-01-08 13:10:00 +08:00
Amos Bird
a3d19fa64d
Correctly override default settings remotely 2021-01-08 12:28:09 +08:00
bharatnc
1e579ac375 Fixes to code, build and style checks 2021-01-07 11:10:49 -08:00
bharatnc
6f0009ff52 extract cluster_name from DDLQuery 2021-01-07 11:10:41 -08:00
bharatnc
f0ea07b493 DDLLogEntry.parse() to populate query and initiator 2021-01-07 11:10:41 -08:00
bharatnc
9cddd22a7a review fixes 2021-01-07 11:10:41 -08:00
bharatnc
24d67082a9 few more changes and simplification to integration test 2021-01-07 11:10:41 -08:00
bharatnc
c796b0c1cb add exception_status column 2021-01-07 11:10:41 -08:00
bharatnc
9004d4384d Add query_start|finish_time, query_dureation_ms 2021-01-07 11:10:41 -08:00
bharatnc
6da923cbe1 Use tryGetChildren() for Zookeeper 2021-01-07 11:10:41 -08:00
bharatnc
57126d1901 Add values column and fix tests 2021-01-07 11:10:41 -08:00
bharatnc
2a122905f1 Review - initial round of changes 2021-01-07 11:10:41 -08:00
bharatnc
c0d32dee37 DDLWorkerQueueTable - make loop vars a const ref 2021-01-07 11:10:41 -08:00
bharatnc
cc568a1d2c DDLWorkerQueueTable - load distributed_ddl.path from server config 2021-01-07 11:10:40 -08:00
bharatnc
56a8532781 DDLWorkerQueueTable - populate active and finished columns 2021-01-07 11:10:40 -08:00
bharatnc
da174a63d1 DDLWorkerQueueTable - initial attempt to fetch entries from zk 2021-01-07 11:10:40 -08:00
bharatnc
2c1f9e2a77 DDLWorkerQueueTable - add intial scaffolding 2021-01-07 11:10:40 -08:00
Azat Khuzhin
b1f08f5c27 Rename FileSyncGuard to DirectorySyncGuard 2021-01-07 20:26:18 +03:00
Azat Khuzhin
513a824f30 Fix fsync_part_directory for parts renames 2021-01-07 19:30:25 +03:00
Alexey Milovidov
e7a7f557d1 Fix Arcadia 2021-01-07 18:40:54 +03:00
Alexey Milovidov
afb4cc8d3a Merge branch 'master' of github.com:yandex/ClickHouse into kssenii-pg2ch 2021-01-07 18:40:12 +03:00
Alexey Milovidov
159891573f Merge branch 'master' into kssenii-pg2ch 2021-01-07 18:39:44 +03:00
fastio
ea047b951f Expand macros for fetchPartition 2021-01-07 22:13:17 +08:00
alexey-milovidov
a08db94343
Revert "Add metrics for part number in MergeTree in ClickHouse" 2021-01-07 16:40:52 +03:00
Alexey Milovidov
36e1361cf8 Miscellaneous 2021-01-07 15:29:34 +03:00
alexey-milovidov
f91626e7ff
Merge pull request #17838 from weeds085490/dev/add_metrics_for_parts 
Add metrics for part number in MergeTree in ClickHouse
 2021-01-07 15:27:04 +03:00
alexey-milovidov
703e16db5a
Update StorageMergeTree.cpp 2021-01-07 06:38:05 +03:00
alexey-milovidov
72b142a00a
Merge branch 'master' into pg2ch 2021-01-06 23:18:59 +03:00
alexey-milovidov
13e4579052
Merge pull request #18712 from amosbird/limitpartition 
add max_partitions_to_read setting
 2021-01-06 21:29:58 +03:00
alexey-milovidov
5b98767652
Update MergeTreeSettings.h 2021-01-06 21:29:45 +03:00
alexey-milovidov
417e685830
Merge pull request #18775 from ClickHouse/dont-insert-empty-blocks-distributed-sync 
Do not insert empty blocks on sync Distributed INSERT
 2021-01-06 20:06:35 +03:00
weeds085490
5f5b86b485 Merge remote-tracking branch 'origin' into dev/add_metrics_for_parts 2021-01-06 17:32:45 +08:00
Amos Bird
0260953a47
better 2021-01-06 17:18:48 +08:00
Alexey Milovidov
1572d2122b Respect network_compression_method in async INSERT into Distributed table 2021-01-06 03:41:34 +03:00
Alexey Milovidov
190402b7d5 Do not insert empty blocks on sync Distributed INSERT 2021-01-06 02:54:22 +03:00
alexey-milovidov
176358f0b4
Merge pull request #18758 from ClickHouse/remove-useless-headers 
Remove some headers
 2021-01-06 00:34:46 +03:00
Vladimir Chebotarev
5cbc25c647 Added global setting s3_max_connections. 2021-01-05 13:34:55 +03:00
Alexey Milovidov
dab4719aac Remove some headers 2021-01-05 06:22:06 +03:00
Alexey Milovidov
5368b20a71 Fix error 2021-01-05 04:49:15 +03:00
Alexey Milovidov
cbc54e2dd7 Merge branch 'master' into revert-15259-revert-14962-log-avoid-deadlock 2021-01-05 04:27:32 +03:00
alexey-milovidov
a4f8a5390e
Merge pull request #18717 from sundy-li/hotfix-log 
Fix Logger with unmatched arg size
 2021-01-04 18:43:26 +03:00
sundy-li
6cc0668af4 Add one more argument 2021-01-04 16:21:04 +08:00
sundy-li
8d7fe410cd Fix Logger with unmatched arg size 2021-01-04 16:15:13 +08:00
Amos Bird
a157a5b3b3
add max_partitions_to_read setting 2021-01-04 12:40:48 +08:00
Alexey Milovidov
c59deb4bca Allow Replicated tables in Arcadia 2021-01-03 19:20:09 +03:00
Alexey Milovidov
9d5b2023e8 Fix tests 2021-01-03 19:12:22 +03:00
Alexey Milovidov
2881b3cf30 Fix errors 2021-01-03 00:29:49 +03:00
sundyli
f8815f233c
Merge branch 'master' into alias_column_partition_prune 2021-01-02 11:23:55 +08:00
Alexey Milovidov
4b3ae495d6 Merge branch 'master' into CurtizJ-multiple-nested 2021-01-02 00:25:16 +03:00
Alexey Milovidov
dd47b94fd0 Fix trivial mistake 2021-01-01 23:16:06 +03:00
Alexey Milovidov
1a90c569cb Merge branch 'master' into revert-15259-revert-14962-log-avoid-deadlock 2021-01-01 21:39:08 +03:00
alexey-milovidov
19e0e1a403
Merge pull request #17646 from azat/Buffer-memory-tracking 
Do not ignore server memory limits during Buffer flush
 2021-01-01 18:58:52 +03:00
Azat Khuzhin
ea4c25e7dd Do not ignore server memory limits during Buffer flush 
But ignore them during rollback, since it is better to account memory
incorrectly them terminating the server.
 2020-12-31 22:10:39 +03:00
Nikita Mikhaylov
4165a58ce4
Merge pull request #11617 from nikitamikhaylov/parallel-parsing-input-format 
Parallel formatting
 2020-12-31 21:22:16 +04:00
alexey-milovidov
c2f0451e39
Merge pull request #18362 from ClickHouse/allow_low_cardinality_key_alter 
Add ability to change some types for primary key
 2020-12-31 15:27:52 +03:00
kssenii
e100aec086 Fix typos check 2020-12-31 11:15:05 +00:00
Amos Bird
f93e30bed6
Fix warning 2020-12-31 11:06:15 +08:00
Nikita Mikhaylov
f27aa148e0
Merge branch 'master' into parallel-parsing-input-format 2020-12-31 06:16:43 +04:00
alexey-milovidov
84d77bfc38
Merge pull request #18385 from ClickHouse/set-join-storage-tables-tsan 
Fix TSan report in StorageSet and StorageJoin
 2020-12-31 01:58:53 +03:00
alexey-milovidov
a8f9eade19
Merge pull request #18095 from spongedu/support_show_create_view 
Support show create view syntax
 2020-12-30 23:15:12 +03:00
Alexey Milovidov
83cff84182 Merge branch 'master' into revert-15259-revert-14962-log-avoid-deadlock 2020-12-30 22:52:12 +03:00
Nikita Mikhailov
8e31ce4a5c Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-30 18:22:11 +03:00
alexey-milovidov
9bc571eacc
Update KeyCondition.cpp 2020-12-30 17:58:43 +03:00
alexey-milovidov
1bcdf37c36
Merge pull request #18614 from CurtizJ/fix-empty-parts 
Fix removing of empty parts in tables with old syntax
 2020-12-30 17:20:31 +03:00
Nikita Mikhailov
cc794b7a83 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-30 16:55:15 +03:00
Alexey Milovidov
203af8b7ec Fix error 2020-12-30 16:52:37 +03:00
Nikita Mikhailov
ec0f70b8ef cleanup 2020-12-30 08:39:08 +03:00
Nikita Mikhailov
2f04cb5ebe abort() instead of std::terminate() + cleanup 2020-12-30 08:31:45 +03:00
Nikita Mikhailov
60b4a36c4a arcadia fix + live view fix + cleanup 2020-12-30 07:50:58 +03:00
Nikita Mikhailov
c5f92e5096 better formatfactory 2020-12-30 06:07:30 +03:00
Nikita Mikhailov
2b90b4e01d Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-29 22:47:06 +03:00
Anton Popov
6336fbf1df fix removing of empty parts in tables with old syntax 2020-12-29 20:16:57 +03:00
Anton Popov
a8f1786d95 fix TTL with GROUP BY 2020-12-29 18:19:11 +03:00
kssenii
c1eced8993 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg2ch 2020-12-29 15:09:53 +00:00
Yatsishin Ilya
9b4b2fa441 fix typo 2020-12-29 17:57:44 +03:00
Yatsishin Ilya
a25fa1996b fix style checks 2020-12-29 17:57:12 +03:00
alesapin
00836a7c52
Merge pull request #18518 from ClickHouse/fix_bug_with_alter_hung 
Control alter sequential execution more correctly
 2020-12-29 12:08:21 +03:00
hexiaoting
e6c9361559 Add a MergeTree setting: allow_floating_point_partition_key 2020-12-29 11:12:02 +08:00
spongedc
494c9ac809 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-29 10:20:06 +08:00
Nikita Mikhailov
c0eef84c50 better 2020-12-29 04:15:26 +03:00
Nikita Mikhailov
d293ae5e8e style 2020-12-28 20:18:53 +03:00
kssenii
9b25890674 Generate ya.make, fix fast test 2020-12-28 12:54:52 +00:00
Nikita Mikhailov
c3288c3fbf Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-28 15:09:37 +03:00
kssenii
8f8920a7ee Add table cache, better drop table 2020-12-27 15:52:15 +00:00
kssenii
00a37404ca Add postgresql database engine 2020-12-27 12:26:29 +00:00
kssenii
d95e8e2d74 Better 2020-12-27 12:18:59 +00:00
kssenii
aa3484515d Better 2020-12-27 12:17:23 +00:00
kssenii
69f6714461 Add table function 2020-12-27 12:16:46 +00:00
kssenii
42023f4b95 Support insert into table 2020-12-27 12:16:27 +00:00
kssenii
2f6cb7f2f5 Add storage PostgreSQL with read support 2020-12-27 12:15:57 +00:00
Anton Popov
11283e3d81 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-25 21:25:59 +03:00
alesapin
0950c16fb2 Fix unexpected change 2020-12-25 18:49:29 +03:00
alesapin
2c1d443944 Control alter sequential execution more correctly 2020-12-25 18:47:17 +03:00
Anton Popov
b60c00ba74 refactoring of TTL stream 2020-12-25 18:46:13 +03:00
alesapin
a50615a22b More correct error code 2020-12-25 16:38:04 +03:00
alexey-milovidov
aba7b5f60d
Merge pull request #18493 from ClickHouse/fix-flaky-test-buffer 
Fix flaky test 01584_distributed_buffer_cannot_find_column
 2020-12-25 12:07:03 +03:00
hexiaoting
a47e177b3e Allow float point as partition key when attach table 2020-12-25 11:05:16 +08:00
alexey-milovidov
491f481713
Merge pull request #18481 from ClickHouse/disable_write_with_aio 
Disable write with AIO even for big merges
 2020-12-25 04:24:43 +03:00
Alexey Milovidov
a671f13595 Fix flaky test 01584_distributed_buffer_cannot_find_column 2020-12-25 04:20:09 +03:00
alesapin
5d94b7eec0 Remove non working code 2020-12-24 19:30:53 +03:00
Nikita Mikhaylov
6818b24fe5
Merge pull request #18381 from CurtizJ/part-types-merge 
Restrict merges from wide to compact parts
 2020-12-24 17:47:14 +03:00
Anton Popov
42725b27c0 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-24 15:18:15 +03:00
hexiaoting
b3b08a035e Disallow floating point as partition key 2020-12-24 18:30:20 +08:00
Yatsishin Ilya
ce1e2b18a3 Fix codespell warnings. Split style checks. Update style checks docker 2020-12-24 13:17:52 +03:00
alexey-milovidov
c4bbb95b0f
Merge pull request #18405 from ClickHouse/add_test_for_compact_parts 
Add test for already working code
 2020-12-24 01:24:09 +03:00
alesapin
f3f5466d23
Merge pull request #18306 from amosbird/restrictsettings 
Extend mergetree setting alter restriction
 2020-12-23 23:09:59 +03:00
Anton Popov
84ec3930ba
Merge pull request #18223 from ClickHouse/delayed_last_mark_for_wide_parts 
Better adaptive granularity calculation for wide parts
 2020-12-23 19:37:25 +03:00
Amos Bird
6fc225e676
Distributed insertion to one random shard (#18294) 
* Distributed insertion to one random shard

* add some tests

* add some documentation

* Respect shards' weights

* fine locking

Co-authored-by: Ivan Lezhankin <ilezhankin@yandex-team.ru>
 2020-12-23 19:04:05 +03:00
alexey-milovidov
2c7979dd64
Merge pull request #18379 from vitlibar/fix-filling-table-system-settings-profile-elements 
Fix filling table system.settings_profile_elements.
 2020-12-23 15:34:00 +03:00
Anton Popov
57857dda63 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-23 15:16:43 +03:00
Anton Popov
5f9c47787a fix subcolumns with some storages 2020-12-23 15:16:26 +03:00
alesapin
fb1c6c4eb4
Update MergeTreeSettings.h 2020-12-23 15:03:55 +03:00
alesapin
54455b4740 Add test for already working code 2020-12-23 14:53:49 +03:00
Anton Popov
518cac6150 fix for review 2020-12-23 13:11:24 +03:00
Nikolai Kochetov
af7f5c9518
Merge pull request #17868 from ClickHouse/async-read-from-socket 
Async read from socket
 2020-12-23 12:20:42 +03:00
Alexander Kuzmenkov
22a0edc596
Update IStorage.cpp 2020-12-23 09:44:44 +03:00
Alexander Kuzmenkov
51b2329295
Merge branch 'master' into aku/window-prototype 2020-12-23 09:26:11 +03:00
Amos Bird
8b5714b2ac
Fix 2-arg functions with constant in PK analysis 2020-12-23 12:29:29 +08:00
spongedc
a277308ac6 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-23 11:56:46 +08:00
alexey-milovidov
8e48a25607
Merge pull request #18299 from ClickHouse/enable_thread_fuzzer_for_flaky_check 
Trying to enable thread fuzzer in flaky check
 2020-12-23 05:32:19 +03:00
Nikita Mikhailov
dcfbe782c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-23 05:20:22 +03:00
Nikita Mikhaylov
dae1e9ba31
Merge pull request #17857 from fastio/feature_hints_for_column_names 
hints for column names
 2020-12-23 04:55:41 +03:00
alexey-milovidov
c72a480d89
Merge pull request #17630 from amosbird/stacktracefix 
Fix system.stack_trace when running in daemon mode
 2020-12-23 04:40:50 +03:00
alexey-milovidov
aedb1874af
Fix clang-tidy 2020-12-23 04:40:23 +03:00
roverxu
e38ecee30e fix style 2020-12-23 09:20:45 +08:00
Anton Popov
78c4b9f2bc do not allow merge from wide to compact parts 2020-12-23 03:16:15 +03:00
Anton Popov
5101804e30
Merge pull request #18366 from CurtizJ/remove-unused-code 
Remove unused code
 2020-12-23 02:45:33 +03:00
Vitaly Baranov
9e31800ebb Fix filling table system.settings_profile_elements. 2020-12-22 23:24:59 +03:00
Anton Popov
cc6df719b8 fix optimize deduplicate 2020-12-22 21:03:30 +03:00
Anton Popov
1be39fddac fix subcolumns with some storages 2020-12-22 19:42:37 +03:00
Anton Popov
b6ff6300b2 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-22 18:06:21 +03:00
Anton Popov
40b5ffe3b5 minor renames 2020-12-22 18:06:12 +03:00
alesapin
d1d139a168 Fix comment 2020-12-22 16:48:10 +03:00
alesapin
23156fe6e4 Fix accident stop merges for merge tree 2020-12-22 16:46:09 +03:00
Anton Popov
371050a16e remove unused code 2020-12-22 15:22:54 +03:00
徐炘
81f9623e2e
Merge branch 'master' into dev/add_metrics_for_parts 2020-12-22 19:04:28 +08:00
alesapin
35413635e7 Add ability to change some types for primary key 2020-12-22 13:41:12 +03:00
flynn
4e580f7b7c
Add asynchronous metrics on MergeTree tables data volume (#17639) 
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
 2020-12-22 13:34:35 +03:00
root
d6a69ee164 remove code 2020-12-22 18:12:28 +08:00
roverxu
b339b9dfd0 fix consistence 2020-12-22 17:38:15 +08:00
Alexey Milovidov
b9df795595 Another code for TSan 2020-12-22 11:16:36 +03:00
Alexey Milovidov
af8022a0b1 Merge branch 'master' into amosbird-stacktracefix 2020-12-22 10:48:28 +03:00
alexey-milovidov
b81e067cd4
Update IStorage.h 2020-12-22 09:51:09 +03:00
alexey-milovidov
c1bd8d5df5
Merge pull request #18329 from azat/buffer-rollback-fix 
Fix NULL dereference in Buffer rollback
 2020-12-22 09:30:53 +03:00
Azat Khuzhin
6acefa7624 Suppress bugprone-use-after-move in Buffer rollback 2020-12-22 01:22:36 +03:00
alexey-milovidov
69719f743d
Merge branch 'master' into revert-15259-revert-14962-log-avoid-deadlock 2020-12-22 00:41:52 +03:00
Azat Khuzhin
ba57c418d0 Fix NULL dereference in Buffer rollback 
<details>

Stacktrace:

```
(gdb) bt
0  DB::appendBlock (from=..., to=...) at ../src/Storages/StorageBuffer.cpp:411
1  DB::BufferBlockOutputStream::insertIntoBuffer (this=<optimized out>, block=..., buffer=...) at ../src/Storages/StorageBuffer.cpp:541
2  0x000000000f2e9d5f in DB::BufferBlockOutputStream::write (this=<optimized out>, block=...) at ../src/Storages/StorageBuffer.cpp:508
3  0x000000000ec426c4 in DB::PushingToViewsBlockOutputStream::write (this=0x7f74660faa18, block=...) at ../src/DataStreams/PushingToViewsBlockOutputStream.cpp:160
4  0x000000000ec49633 in DB::AddingDefaultBlockOutputStream::write (this=0x7f74660f1b18, block=...) at ../src/DataStreams/AddingDefaultBlockOutputStream.cpp:10
5  0x000000000ec483ac in DB::SquashingBlockOutputStream::finalize (this=0x7f74660f1d18) at ../src/DataStreams/SquashingBlockOutputStream.cpp:30
6  0x000000000ec48429 in DB::SquashingBlockOutputStream::writeSuffix (this=0x7f74660f1d18) at ../src/DataStreams/SquashingBlockOutputStream.cpp:50
7  0x000000000ec43f8f in DB::PushingToViewsBlockOutputStream::writeSuffix (this=0x7f74660f8258) at ../src/DataStreams/PushingToViewsBlockOutputStream.cpp:280
8  0x000000000ec43f8f in DB::PushingToViewsBlockOutputStream::writeSuffix (this=0x7f74b7ddea18) at ../src/DataStreams/PushingToViewsBlockOutputStream.cpp:280
9  0x000000000f2e6748 in DB::StorageBuffer::writeBlockToDestination (this=<optimized out>, block=..., table=...) at ../src/Storages/StorageBuffer.cpp:820
10 0x000000000f2ea00b in DB::BufferBlockOutputStream::write (this=0x7f7574e11748, block=...) at ../src/Storages/StorageBuffer.cpp:469
11 0x000000000ec426c4 in DB::PushingToViewsBlockOutputStream::write (this=0x7f7574ed3658, block=...) at ../src/DataStreams/PushingToViewsBlockOutputStream.cpp:160
12 0x000000000ec49633 in DB::AddingDefaultBlockOutputStream::write (this=0x7f7574e84518, block=...) at ../src/DataStreams/AddingDefaultBlockOutputStream.cpp:10
13 0x000000000ec482f4 in DB::SquashingBlockOutputStream::write (this=0x7f7574e84718, block=...) at ../src/DataStreams/SquashingBlockOutputStream.cpp:17
14 0x000000000ebe8bce in DB::CountingBlockOutputStream::write (this=0x7f7574ed3720, block=...) at ../src/DataStreams/CountingBlockOutputStream.cpp:17
15 0x000000000f68e834 in DB::TCPHandler::receiveData (this=<optimized out>, scalar=<optimized out>) at ../src/Server/TCPHandler.cpp:1168
16 0x000000000f68737c in DB::TCPHandler::receivePacket (this=0x7f7574f17000) at ../src/Server/TCPHandler.cpp:918
17 0x000000000f688d2f in DB::TCPHandler::readDataNext (this=0x7f7574f17000, poll_interval=@0x7f6f1dff1f78: 10000000, receive_timeout=@0x7f6f1dff1f68: 300) at ../src/Server/TCPHandler.cpp:460
18 0x000000000f6878be in DB::TCPHandler::readData (this=0x7f7574f17000, connection_settings=...) at ../src/Server/TCPHandler.cpp:490
19 DB::TCPHandler::processInsertQuery (this=0x7f7574f17000, connection_settings=...) at ../src/Server/TCPHandler.cpp:519
20 0x000000000f680ab9 in DB::TCPHandler::runImpl (this=0x7f7574f17000) at ../src/Server/TCPHandler.cpp:268
21 0x000000000f68f297 in DB::TCPHandler::run (this=0x7f7574f17000) at ../src/Server/TCPHandler.cpp:1414
22 0x0000000011fb81cf in Poco::Net::TCPServerConnection::start (this=0x0) at ../contrib/poco/Net/src/TCPServerConnection.cpp:43
23 0x0000000011fb9be1 in Poco::Net::TCPServerDispatcher::run (this=0x7f752ab5fd00) at ../contrib/poco/Net/src/TCPServerDispatcher.cpp:112
24 0x00000000120e71c9 in Poco::PooledThread::run (this=0x7f747d3a4580) at ../contrib/poco/Foundation/src/ThreadPool.cpp:199
25 0x00000000120e315a in Poco::ThreadImpl::runnableEntry (pThread=<optimized out>) at ../contrib/poco/Foundation/src/Thread_POSIX.cpp:345
26 0x00007f760620aea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
27 0x00007f760613aeaf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

(gdb) p to.data.__end_-to.data.__begin_
$17 = 10
(gdb) p to.data.__begin_[9].column.px
$19 = (const DB::IColumn *) 0x7f7328392720
(gdb) p to.data.__begin_[8].column.px
$20 = (const DB::IColumn *) 0x0
(gdb) p to.data.__begin_[7].column.px
$21 = (const DB::IColumn *) 0x7f746f33d360
```

Line numbers matched with this version -
f0e7cb16a7/src/Storages/StorageBuffer.cpp (L411)

</details>
 2020-12-21 23:56:56 +03:00
Amos Bird
7a81e0a0f5
Extend mergetree setting alter restriction 2020-12-21 18:45:39 +08:00
roverxu
f3fef24d30 fix deadlock 2020-12-21 16:30:27 +08:00
alesapin
07b5c03c54 Better adjustment of the last mark 2020-12-21 11:24:52 +03:00
Alexey Milovidov
571520964c Remove useless code 2020-12-21 10:48:15 +03:00
Alexander Kuzmenkov
85c1bc1253 Merge remote-tracking branch 'origin/master' into tmp 2020-12-21 10:46:21 +03:00
roverxu
8e72b6d31b Merge remote-tracking branch 'origin' into dev/add_metrics_for_parts 2020-12-21 09:25:54 +08:00
Alexey Milovidov
838b146b41 Merge branch 'master' into amosbird-stacktracefix 2020-12-21 04:22:31 +03:00
alesapin
23eebb3f90 Merge branch 'master' into delayed_last_mark_for_wide_parts 2020-12-20 23:45:44 +03:00
alesapin
0041c293ab Fix gcc-10 build 2020-12-20 11:01:39 +03:00
Alexey Milovidov
9be5fa9ef2 Merge branch 'master' into Enmk-Optimize_deduplicate 2020-12-20 09:57:10 +03:00
alesapin
ca6708e594 Fix check and granule adjust for non adaptive parts 2020-12-19 21:59:20 +03:00
Vasily Nemkov
e5ec81f7cd Single quotes around column names 2020-12-19 20:58:23 +02:00
alesapin
b2d53d802d Fix for non-adaptive granularity 2020-12-18 23:32:52 +03:00
Anton Popov
25ddd78976 fix nested 2020-12-18 23:09:34 +03:00
Alexander Kuzmenkov
132a0b42eb cleanup 2020-12-18 20:13:28 +03:00
Nikolai Kochetov
d7fc426458
Merge pull request #18216 from ClickHouse/fix-18137 
Fix 18137
 2020-12-18 18:49:02 +03:00
alesapin
cd598034cd Fix add tests 2020-12-18 17:44:31 +03:00
alesapin
c1e7d5d3c1 Working code 2020-12-18 16:49:45 +03:00
Anton Popov
009efeff40 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-18 16:14:09 +03:00
Anton Popov
31e6956ac8 fix alters of nested 2020-12-18 15:29:57 +03:00
Vasily Nemkov
e166aae3f9 Using CSV-like strings for list of columns to deduplicate by instead of JSON-like notation. 2020-12-18 13:44:56 +02:00
alesapin
26637bd7ee
Merge pull request #17943 from ClickHouse/try_rewrite_writer 
Small simplification of MergeTreeDataWriter
 2020-12-18 12:41:16 +03:00
Nikolai Kochetov
eef58c85f0 Put thread back to set if it had stolen all tasks of backoff thread. 2020-12-18 11:33:55 +03:00
Alexander Kuzmenkov
5e19eaf2f0 Merge remote-tracking branch 'origin/master' into tmp 2020-12-18 03:49:59 +03:00
Nikita Mikhailov
6a55e018e4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-18 02:25:32 +03:00
alexey-milovidov
6fc3ca8b7b
Merge pull request #16724 from kitaisreal/added-function-accurate-cast-or-null 
Added function accurateCastOrNull, allow different types inside IN subquery
 2020-12-18 00:45:27 +03:00
alesapin
8140fbd0fb
Merge pull request #18200 from ClickHouse/more_logs_during_quorum 
More logs during quorum insert
 2020-12-18 00:22:19 +03:00
alesapin
799997db35
Merge pull request #16621 from arenadata/ADQM-148 
HDFS configuration and Kerberos Authenticaiton
 2020-12-17 23:16:58 +03:00
Vitaly Baranov
b9c537ac37
Merge pull request #18104 from vitlibar/fix-access-rights-for-merge-table-function-2 
Fix access rights required for the merge() table function #2.
 2020-12-17 22:59:33 +03:00
Vitaly Baranov
d9125a4822
Merge pull request #18014 from vitlibar/fix-showing-temporary-tables-in-system-tables 
Fix showing temporary tables in system tables
 2020-12-17 22:59:09 +03:00
alexey-milovidov
f82274a722
Merge pull request #16527 from kitaisreal/mmap-read-buffer-from-file-descriptor-use-getpagesize 
MMapReadBufferFromFileDescriptor use getpagesize
 2020-12-17 22:34:05 +03:00
alexey-milovidov
dd78c9dc75
Merge pull request #17566 from ClickHouse/relax-too-many-parts-1 
Relax "Too many parts" threshold, part 1
 2020-12-17 20:44:44 +03:00
Anton Popov
a42b00c9aa Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-17 20:43:23 +03:00
Anton Popov
6de8b05b7c improvements in subcolumns reading 2020-12-17 20:10:33 +03:00
alesapin
aecdd02be4 Better message 2020-12-17 19:14:42 +03:00
alesapin
f45993cd1e More logs during quorum insert 2020-12-17 19:13:01 +03:00
tavplubix
7e73b6acd8
Merge pull request #17642 from filimonov/detach-permanently 
DETACH TABLE PERMANENTLY
 2020-12-17 14:26:51 +03:00
alesapin
911a1cd48f Suppress gdb warning 2020-12-17 13:25:31 +03:00
Vitaly Baranov
da18af96a3 Fix access rights required for the merge() table function. 2020-12-17 12:29:07 +03:00
Vitaly Baranov
af3ccb49f1 Show create query for temporary tables in system.tables 2020-12-17 11:42:25 +03:00
Vitaly Baranov
ede755d3bc Show temporary tables in system.columns as tables with empty database. 2020-12-17 11:42:25 +03:00
Vitaly Baranov
d7a3cc8c90 Don't show the "_temporary_and_external_tables" database in system tables 
system.databases, system.tables, system.columns.
 2020-12-17 11:42:25 +03:00
alesapin
b74e7accf1 Review fixes 2020-12-17 11:17:39 +03:00
alesapin
f2957d6f05 Merge branch 'master' into try_rewrite_writer 2020-12-17 10:40:25 +03:00
sundy-li
84807e9791 rename settings 2020-12-17 02:17:59 +00:00
Alexey Milovidov
b25f126569 Added a test 2020-12-17 04:52:27 +03:00
Alexey Milovidov
9a017b77df Merge branch 'master' into relax-too-many-parts-1 2020-12-17 04:40:39 +03:00
alexey-milovidov
d2653f91e1
Merge pull request #18112 from GrigoryPervakov/join_set_s3 
Use IDisk in Set and Join storages
 2020-12-17 03:40:13 +03:00
alexey-milovidov
484f20f77d
Merge pull request #18140 from ClickHouse/fix_flaky_test 
Trying to fix 00620_optimize_on_nonleader_test
 2020-12-17 03:34:39 +03:00
Alexander Kuzmenkov
a1326414ff Merge remote-tracking branch 'origin/master' into tmp 2020-12-17 00:46:14 +03:00
Alexander Kuzmenkov
fef24c81e3 some grammar 2020-12-17 00:44:05 +03:00
Anton Ivashkin
0f0500ca0c Merge master 2020-12-16 18:31:13 +03:00
Maksim Kita
484e6e7de4 Removed getPageSize from small files 2020-12-16 17:38:17 +03:00
alesapin
849db47f8a Better exception messages 2020-12-16 17:31:17 +03:00
roverxu
1065ddef65 fix bug when drop data 2020-12-16 22:02:34 +08:00
Pervakov Grigorii
4edbb42ba1 Use IDisk in Set and Join storages 2020-12-16 14:58:44 +03:00
Ilya Golshtein
09d1e89674 yandex/clickhouse-kerberized-hadoop + missed header 2020-12-16 14:48:33 +03:00
Ilya Golshtein
407565b47f fix - wrong error code 2020-12-16 14:48:33 +03:00
Ilya Golshtein
9701d6a2d5 style fix per code review, doc improvement, params consistency check 2020-12-16 14:48:33 +03:00
Ilya Golshtein
815856bc3d pass config, not context to createHDFSBuilder 2020-12-16 14:48:33 +03:00
Ilya Golshtein
7e97814859 cleanup, fixes, new submodules, ShellCommand, WriteBufferFromString 2020-12-16 14:48:33 +03:00
Ilya Golshtein
d9166a0645 HADOOP_SECURE_DN_USER way, kinit thread, docker capabilities 2020-12-16 14:48:33 +03:00
Ilya Golshtein
185ac744b1 kerberized hdfs compiled 2020-12-16 14:48:33 +03:00
Maksim Kita
7d23451e81 Replaced usages of 4096 constant with getPageSize 2020-12-16 13:46:28 +03:00
sundy-li
5c9f84b2f4 Merge branch 'master' into alias_column_partition_prune 2020-12-16 18:44:00 +08:00
alesapin
2a4a6b22b1 Fix remove TTL for column 2020-12-16 10:26:18 +03:00
Alexey Milovidov
37363ae89d Comment update 2020-12-16 09:43:16 +03:00
spongedc
38c0827881 Merge remote-tracking branch 'origin/master' into support_show_create_view 2020-12-16 12:26:56 +08:00
Alexey Milovidov
4bd5d3b662 Comment update 2020-12-16 06:36:43 +03:00
Alexey Milovidov
88c5031dfa Add a commit #16595 2020-12-16 06:27:28 +03:00
spongedc
81b0fa2989 1. rename is_view to is_ordinary_view 2. add more tests 2020-12-16 11:19:38 +08:00
Alexey Milovidov
ca9ccf4850 Comment update 2020-12-16 06:03:43 +03:00
Alexey Milovidov
7ca86f176c Fix TSan warning in system.stack_trace 2020-12-16 01:43:07 +03:00
Alexey Milovidov
13ac1bee7a Merge branch 'master' into amosbird-stacktracefix 2020-12-15 22:12:49 +03:00
Nikolai Kochetov
4905201985 Merge branch 'master' into async-read-from-socket 2020-12-15 21:04:47 +03:00
alesapin
1bd2f49fb5 Merge branch 'master' into try_rewrite_writer 2020-12-15 19:05:04 +03:00
alexey-milovidov
22aba554c0
Revert "Fix access rights required for the merge() table function." 2020-12-15 18:07:20 +03:00
Nikolai Kochetov
6defcbb662
Merge branch 'master' into optimize-data-on-insert 2020-12-15 16:50:42 +03:00
fastio
e7bce63d3a hints for column names 2020-12-15 20:23:31 +08:00
alesapin
8670836573 Add some comments 2020-12-15 13:34:28 +03:00
alesapin
74c2211510 More clear 2020-12-15 12:54:48 +03:00
sundy-li
cf087002db Merge branch 'master' into alias_column_partition_prune 2020-12-15 16:37:00 +08:00
sundy-li
d72d8ee7d8 apply review advices 2020-12-15 16:35:19 +08:00
alesapin
a50187c995 Slightly better 2020-12-15 11:04:37 +03:00
Vitaly Baranov
5a8c750b0b
Merge pull request #17983 from vitlibar/fix-access-rights-for-merge-table-function 
Fix access rights required for the merge() table function.
 2020-12-15 09:17:52 +03:00
Alexander Kuzmenkov
a75ca3cb79 Merge remote-tracking branch 'origin/master' into tmp 2020-12-15 03:36:43 +03:00
Alexander Kuzmenkov
6d5b23de67 something works 2020-12-15 03:36:03 +03:00
nikitamikhaylov
12e624fd9a fix tests 2020-12-15 00:56:48 +03:00
nikitamikhaylov
f7ac8bf542 rebase and fix tests 2020-12-15 00:56:48 +03:00
Nikita Mikhaylov
5a47928431 save 2020-12-15 00:56:47 +03:00
Nikita Mikhaylov
f40f3ced2a fix JSONEachRowArray 2020-12-15 00:56:46 +03:00
Nikita Mikhaylov
e0addac6fc save changes 2020-12-15 00:56:46 +03:00
alexey-milovidov
363ff79cca
Merge pull request #17934 from Jokser/s3-adaptive-single-part-upload 
Adaptive choose of single/multi part upload in WriteBufferFromS3
 2020-12-14 23:00:43 +03:00
Maksim Kita
18dc118298 Fixed compile issues 2020-12-14 22:12:15 +03:00
Maksim Kita
0464859cfe Updated usage of different types during IN query 
1. Added accurateCast function.
2. Use accurateCast in Set during execute.
3. Added accurateCast tests.
4. Updated select_in_different_types tests.
 2020-12-14 22:12:15 +03:00
Maksim Kita
f4b8e8ef99 Allow different types inside IN subquery 2020-12-14 22:12:15 +03:00
Nikolai Kochetov
8de5cd5bc7 Merge branch 'master' into async-read-from-socket 2020-12-14 17:45:38 +03:00
Mikhail Filimonov
2aeb72c09a
Fix for tiny issue #13850 2020-12-14 15:31:10 +01:00
alesapin
34d1ad1aca Fix writer compact 2020-12-14 17:27:39 +03:00
tavplubix
dd2ae6926d
Merge pull request #14849 from ClickHouse/allow_atomic_database_inside_materialize_mysql 
Allow Atomic database inside MaterializeMySQL
 2020-12-14 16:27:13 +03:00
alesapin
35d2a689dd One more time 2020-12-14 16:06:11 +03:00
alesapin
2a2b02d395 One more time 2020-12-14 16:05:48 +03:00
alesapin
9e98cdad5a Accident changes 2020-12-14 16:05:07 +03:00
alesapin
47e3f619e5 Remove redundant code 2020-12-14 16:01:01 +03:00