Commit Graph

992 Commits

Author SHA1 Message Date
Alexey Milovidov
01e8adbfff Removed "experimental" namespace [#CLICKHOUSE-2]. 2017-11-20 07:15:43 +03:00
Alexey Milovidov
6c8b5e29d8 Additions after merge [#CLICKHOUSE-2]. 2017-11-20 00:17:58 +03:00
alexey-milovidov
fc0a51f1b9
Merge branch 'master' into fix4 2017-11-19 23:57:44 +03:00
Alexey Milovidov
94d3b49e99 BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2]. 2017-11-19 03:27:43 +03:00
Alexey Milovidov
9b92c0fecd BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2]. 2017-11-19 03:22:13 +03:00
Silviu Caragea
dfecd0b2f2 Merge with master 2017-11-18 10:16:06 +02:00
Alexey Milovidov
729b114c15 Removed unsupported functionality [#CLICKHOUSE-2]. 2017-11-18 05:34:00 +03:00
Alexey Milovidov
076e0ae014 More clear [#CLICKHOUSE-2]. 2017-11-18 00:20:28 +03:00
Alexey Milovidov
7c9ba15ba1 Style [#CLICKHOUSE-2]. 2017-11-17 23:42:03 +03:00
proller
79b3ef2f57 Merge remote-tracking branch 'upstream/master' into fix4 2017-11-17 15:31:13 +03:00
Silviu Caragea
64619c614f Basic idea is :
- We add BackgroundSchedulePool which can execute a function at a specific point in time. Basically all tasks are added in a queue and precessed by worker threads.
- The most important difference between this and BackgroundProcessingPool is that we have the guarantee that the same function is not executed from many workers in the same time
- Each of the following classes instead starting a thread will register a task in BackgroundSchedulePool and when they need to run will call schedule or scheduleAfter(duration) functions

This commit is moving all threads created by ReplicatedMergeTree to BackgroundSchedulePool tasks

NOTE: I did a minimum number of changes to be much simple to review the code
2017-11-17 10:58:35 +02:00
alexey-milovidov
dbee90ba5b Update ReplicatedMergeTreeBlockOutputStream.cpp 2017-11-15 23:13:10 +03:00
Alexey Zatelepin
99b2d135df clear deduplication blocks when doing DROP PARTITION [#CLICKHOUSE-3208] 2017-11-15 23:13:10 +03:00
Alexey Milovidov
2846c7560f Fixed wording [#CLICKHOUSE-2]. 2017-11-15 22:47:49 +03:00
proller
468e2ab044 Merge remote-tracking branch 'upstream/master' into fix4 2017-11-14 14:50:19 +03:00
Alexey Milovidov
ac1b69247c Addition to prev. revision [#CLICKHOUSE-3402]. 2017-11-09 22:25:21 +03:00
Alexey Milovidov
783e812eb7 Fixed error [#CLICKHOUSE-2] 2017-11-04 19:46:14 +03:00
proller
c974f0e88e Merge remote-tracking branch 'upstream/master' into fix4 2017-11-02 17:42:36 +03:00
Vitaliy Lyudvichenko
9cf14d2c61 Remove obsolete code. [#CLICKHOUSE-2] 2017-11-02 17:35:12 +03:00
proller
6da4f5a661 add locks 2017-11-01 22:56:07 +03:00
proller
3b58e9cd67 Fix atomic usage 2017-11-01 20:15:05 +03:00
proller
c5dcde0c78 fix atomic usage 2017-11-01 16:58:02 +03:00
Alexey Milovidov
6aff58c747 Added backward compatibility for tables created with wrong sampling key; fixed tiny error [#CLICKHOUSE-2]. 2017-11-01 04:45:10 +03:00
alexey-milovidov
3b99b723d7
Merge pull request #1245 from yandex/CLICKHOUSE-3178
Atomic parts addition and removing
2017-10-27 23:03:33 +03:00
Vitaliy Lyudvichenko
a0d06c5ee0 Don't deduplicate a block inserted from MV if the source block is Ok. [#CLICKHOUSE-3381] 2017-10-26 17:58:00 +03:00
Vitaliy Lyudvichenko
602560cb2e Update system.parts table. [#CLICKHOUSE-3178] 2017-10-26 17:20:21 +03:00
Vitaliy Lyudvichenko
483dfd1b44 Fixed "No part error" in replicated sender. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
397b8bcf92 Smarter parts deletion. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
d6ea9c3dcf Fixed awful copypaste. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
b5c7eaa80e Force data parts cleanup before repeat fetch. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
db3a67a421 Add clearer RangeFiltered implementation. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
e2a12d1088 Introduced logic for MergeTree parts states. [#CLICKHOUSE-3178] 2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
5787e8b257 Add state for MergeTree parts. [#CLICKHOUSE-3178]
And Removed obsolete code.
2017-10-26 17:16:06 +03:00
Alexey Milovidov
65982a1642 Fixed exception messages [#CLICKHOUSE-2]. 2017-10-25 22:17:37 +03:00
proller
4ffbca2d8c Fix warnings (#1406)
* Log query id in executeQuery; Better type mismatch error; change format in report tool

* Better log query_id

* fix message

* Use c++11 thread_local instaed of gcc's __thread

* lock mutex before notifying waiting thread in sync insertion into distributed [#CLICKHOUSE-3379]

* Cmake: fix build without downloaded submodules (#1379)

* fix

* ZooKeeper: fixed stack smashing with tryGet()

The tryGet() operation creates a 1MB buffer on stack. This may or
may not work depending on the default stack size for threads,
whether the stack protector is enabled or not, recursion depth,
and the actual value size.

This is probably going to slow down some ZK operations, but I don't
see how else this could work reliably with the existing API.

* increased timeout for test_insertion_sync_fails_with_timeout

* Update CHANGELOG_RU.md

* Update ZooKeeper.cpp

* Fix warnings

* Fixes

* Dont strip debug info from  asan, tsan and other builds except releases

* Fix asan error causd by test 00144

* Fix empty log message (#CLICKHOUSE-3378)
2017-10-25 21:39:10 +03:00
Alexey Zatelepin
b6b5d1cf25 store rows count for part on disk to allow tables without fixed size columns [#CLICKHOUSE-3000] 2017-10-24 17:11:53 +03:00
Alexey Zatelepin
c61d4106e8 allow loading all MergeTree settings from a query [#CLICKHOUSE-3000] 2017-10-20 21:28:45 +03:00
Alexey Zatelepin
a7c85e0c89 save index_granularity to ATTACH query if it wasn't set explicitly [#CLICKHOUSE-3000] 2017-10-20 21:28:44 +03:00
Alexey Zatelepin
75c65c7b59 use extended storage definition syntax for MergeTree engines [#CLICKHOUSE-3000] 2017-10-20 21:28:42 +03:00
Alexey Zatelepin
e51f95872f fix backward compatibility for new part format: [#CLICKHOUSE-3000]
* do not write out partition and minmax index files for old MergeTreeData format
* download the part from another replica if there are missing or extra files
2017-10-19 21:24:30 +03:00
Marek Vavruša
2ac5a743cc dbms: fix build with OS X built-in clang
The built-in clang doesn’t support value() for
`std::experimental::optional`. It however supports
dereference operator, which is basically the
same thing:

```
/clickhouse/dbms/src/DataStreams/NullableAdapterBlockInputStream.cpp:83:67: error: call to unavailable member function 'value':
                    res.insert({elem.column, elem.type, rename[i].value()});
                                                        ~~~~~~~~~~^~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/experimental/optional:547:17: note: candidate function has been explicitly made unavailable
    value_type& value()
                ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/experimental/optional:539:33: note: candidate function has been explicitly made unavailable
    constexpr value_type const& value() const
```
2017-10-13 18:12:29 -07:00
proller
49f8080e52 Small debud improvement 2017-10-13 21:44:26 +03:00
Amos Bird
f30841d5ab Fix SummingMergeTree argument checking logic.
This patch fixes the Nested Column Name checking logic, which allows nested
columns be explicitly specified in SummingMergeTree engine.
2017-10-13 15:42:32 +03:00
Vladimir Chebotarev
1e0241e262 dbms: CompressionSettingsSelector. [#METR-21516] 2017-10-13 04:08:23 +03:00
Vladimir Chebotarev
4e313f3732 dbms: Added compression level for ZSTD. [#METR-26742] 2017-10-13 04:08:23 +03:00
Alexey Milovidov
3a6705af0f Improvement [#CLICKHOUSE-2]. 2017-10-12 23:34:01 +03:00
Vitaliy Lyudvichenko
62ea1133dd Disable part sendings and fetches before ALTER. [#CLICKHOUSE-3343] 2017-10-12 23:26:54 +03:00
KochetovNicolai
83925f1d5e Fix array writing (#1314)
* changed MergedBlockOutputStream [#CLICKHOUSE-3341]

* fix build [#CLICKHOUSE-3341]

* fix build [#CLICKHOUSE-3341]

* fix build [#CLICKHOUSE-3341]

* fix build [#CLICKHOUSE-3341]

* fix build [#CLICKHOUSE-3341]

* fix build [#CLICKHOUSE-3341]

* fix build [#CLICKHOUSE-3341]

* fix build [#CLICKHOUSE-3341]

* fix build [#CLICKHOUSE-3341]

* disabled checkNoMultidimensionalArrays [#CLICKHOUSE-3341]

* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]

* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]

* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]

* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]

* added test [#CLICKHOUSE-3341]

* fixed test [#CLICKHOUSE-3341]

* refactoring and comments [#CLICKHOUSE-3341]

* fix build [#CLICKHOUSE-3341]

* Update ColumnArray.h
2017-10-12 23:13:26 +03:00
alexey-milovidov
c1285c7d6b Update MergeTreeDataMerger.cpp 2017-10-12 22:28:24 +03:00
Vitaliy Lyudvichenko
cabf6d7799 Add better logging if OPTIMIZE cannot be executed. [#CLICKHOUSE-2] 2017-10-12 22:28:24 +03:00