Commit Graph

1022 Commits

Author SHA1 Message Date
l
688b0d91a0 Style checks 2019-09-11 22:53:53 +02:00
Alexander Tokmakov
f323476111 Merge branch 'master' into merging_values_with_expressions 2019-09-11 23:31:37 +03:00
l
adb4a58092 Style checks & security considerations added to setting description. 2019-09-11 22:18:22 +02:00
Alexander Tokmakov
29fe481b06 optimize expression template deduction 2019-09-11 22:55:28 +03:00
l
cdbcfc2c90 + Redirect 2019-09-11 18:39:30 +02:00
Amos Bird
a1310ffbe6 Split libdbms.so using object library
Now the linking time of incremental builds are around 1-2 seconds
2019-09-11 01:56:01 +08:00
alesapin
193049293a More comments, more readable code 2019-09-10 14:21:59 +03:00
Alexander Tokmakov
3643ecc9cd parse expressions near the end of buffer correctly 2019-09-09 20:06:22 +03:00
alesapin
ea8e543b1a Add protocol version to cookies 2019-09-06 15:18:56 +03:00
Olga Khvostikova
4129d8cf60
Merge pull request #6092 from yandex/globs-in-file-one-more-attempt
Globs in storage file and HDFS
2019-09-05 23:05:37 +03:00
stavrolia
587a780ba2 Better 2019-09-05 17:42:17 +03:00
Ivan
ab7df6b6dd
Fix kafka tests (#6805)
* Commit offsets more precisely
* Get rid of DelimitedReadBuffer since read buffers don't line up well
* Increase timeouts
2019-09-05 00:25:33 +03:00
Vladimir Chebotarev
c891590709 Added even better warning. 2019-09-04 13:39:54 +00:00
Vladimir Chebotarev
ab456262d9 Fixed multipart mechanism and added a warning about 10k parts. 2019-09-04 13:39:54 +00:00
Vladimir Chebotarev
97b7635c8a Minimum block size to configuration. 2019-09-04 13:39:54 +00:00
Vladimir Chebotarev
395560df1b Fixed multipart uploads and 100-continue. 2019-09-04 13:39:54 +00:00
Vladimir Chebotarev
fa0de006d0 More style fixes. 2019-09-04 13:39:54 +00:00
Vladimir Chebotarev
a9153b2fb3 Style fixes. 2019-09-04 13:39:54 +00:00
Vladimir Chebotarev
96d093f9ad Clang compatibility fixes. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
248e26d59f Attempt to make multipart uploads. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
10a7f80af1 Stubs for multipart uploads. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
dd32c92f2a Minor fixes. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
63164db019 Fixed S3 PUT redirects. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
634f82d0ad Minor fix. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
ff69112919 Attempt to make S3 PUT redirects (wip). 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
7236ae0d8d Fixed 411 error when putting to S3. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
0504eb58be Correct S3 PUT requests in WriteBufferFromS3. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
1944ff1a48 POST to PUT, refactoring. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
062db0ec14 Removed templateness of ReadWriteBufferFromS3. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
531460396d POST S3 requests. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
caeacafb76 Fixed GET redirects. 2019-09-04 13:39:53 +00:00
Vladimir Chebotarev
78f57c5f2a First attempt to make redirects. 2019-09-04 13:39:53 +00:00
Alexey Milovidov
9cf0c780fb Better code 2019-09-03 21:23:13 +03:00
alexey-milovidov
845a612b95
Merge pull request #6773 from infinivision/fix_issue_6575
fix DataTypeAggregateFunction deserialization
2019-09-03 21:19:41 +03:00
Zhichang Yu
9e587fb139 fix DataTypeAggregateFunction deserialization 2019-09-02 20:09:40 +08:00
Alexander Tokmakov
e2928481a3 Merge branch 'master' into merging_template_format 2019-09-02 12:41:34 +03:00
Alexander Tokmakov
cd5d733f75 rewrite unit test with gtest 2019-08-30 22:08:23 +03:00
Alexander Tokmakov
4c744af113 minor style fixes 2019-08-30 20:25:35 +03:00
alexey-milovidov
ce73e17ff6
Merge pull request #6730 from yandex/fallback_o_direct
Fallback from O_DIRECT.
2019-08-30 16:08:19 +03:00
Alexander Tokmakov
d95d53b4e4 better error messages 2019-08-29 22:35:59 +03:00
Alexey Milovidov
a894288fa0 Fallback from O_DIRECT. 2019-08-29 18:48:00 +03:00
Alexander Tokmakov
03c83169bd Merge branch 'master' into merging_template_format 2019-08-29 16:34:25 +03:00
Alexander Tokmakov
d6ca9fee63 unit test for PeekableReadBuffer 2019-08-29 16:30:43 +03:00
Alexey Milovidov
39d50b5144 Remove mimalloc 2019-08-28 22:01:52 +03:00
Weiqing Xu
b7fdfcc797 fix HDFS HA can't work on DEBUG mode
The Describe the bug
when the using HDFS HA nameserivce as the uri, the port will be 0.
hdfsBuilderSetNameNodePort will be called to set the port.
hdfsBuilderSetNameNodePort call asset to check if the port is greater
than 0.
So in Release mode, it works OK. In the Debug mode, the asset will fail.

How to reproduce
when compiler the Clickhouse, use DEBUG mode, it will throw error when
using HDFS HA nameservice url
2019-08-24 17:27:36 +08:00
Alexander Tokmakov
734a0f23a9 Merge branch 'master' into merging_template_format 2019-08-23 15:32:35 +03:00
alex-zaitsev
1e6437e4ee WIP: Fix MacOS build (#6429)
* Fix broken MacOS build

* Fix MacOS build

* Update ReadBufferFromFile.cpp

* Update WriteBufferFromFile.cpp

* Update Elf.h

* Update Elf.h
2019-08-21 03:35:41 +03:00
Nikolai Kochetov
aef03fd9bf
Merge pull request #6425 from yandex/vzakaznikov-liveview
Merge LIVE VIEW
2019-08-20 17:13:42 +03:00
Ivan
a502424c33
Implement support for insertion into Kafka tables (#6012)
* Add write-callback on each row for RowOutputStream
* Fix build of new rdkafka library
* Poll messages if Kafka outgoing queue is full
* Add test
* Add test producer-consumer
* Truncate delimiter from last row in message
2019-08-20 14:17:57 +03:00
Nikolai Kochetov
cdf1ce3171 Merge branch 'master' into vzakaznikov-liveview 2019-08-20 10:41:31 +03:00
stavrolia
6055c61e7f wip hdfs 2019-08-19 21:12:50 +03:00
alexey-milovidov
f36d20915b
Merge pull request #6529 from yandex/better-signals
Better debug-ability of fault signals.
2019-08-18 16:44:54 +03:00
proller
d09c5c871b Build fixes (#6491)
* Fix build

* cmake: fix cpuinfo

* Fix includes after processors merge

Conflicts:
	dbms/src/Processors/Formats/Impl/CapnProtoRowInputFormat.cpp
	dbms/src/Processors/Formats/Impl/ParquetBlockOutputFormat.cpp
	dbms/src/Processors/Formats/Impl/ProtobufRowInputFormat.cpp
	dbms/src/Processors/Formats/Impl/ProtobufRowOutputFormat.cpp

* Fix build in gcc8

* fix test link

* fix test link

* Fix test link

* link fix

* Fix includes after processors merge 2

Conflicts:
	dbms/src/Processors/Formats/Impl/ParquetBlockInputFormat.cpp

* Fix includes after processors merge 3

* link fix

* Fix likely/unlikely conflict with cython

* Fix conflict with protobuf/stubs/atomicops.h

* remove unlikely.h

* Fix macos build (do not use timer_t)
2019-08-18 01:53:46 +03:00
Alexey Milovidov
2cd669fa1d Minor improvement 2019-08-17 23:33:50 +03:00
Nikolai Kochetov
631bbde420 Merged with master. 2019-08-12 13:19:13 +03:00
kreuzerkrieg
112fc71276 adding -Wshadow for GCC 2019-08-09 23:58:16 +03:00
Nikolai Kochetov
2ae3db7920 Merged with master. 2019-08-09 18:32:44 +03:00
Alexey Milovidov
40a3f77917 Removed duplicate include 2019-08-08 19:01:42 +03:00
alexey-milovidov
99fad0904b
Merge pull request #6314 from infinivision/add_bitmapRange
added bitmapRange function
2019-08-08 04:51:48 +03:00
alexey-milovidov
355b1e5594
Merge pull request #6352 from yandex/no_space_left_on_device_message
Print extra info in exception message for "no space left on device"
2019-08-07 22:28:38 +03:00
Alexander Tokmakov
5cf183cbe3 fixes after review 2019-08-07 15:55:27 +03:00
Alexander Tokmakov
aefc664819 refactor throwFromErrno 2019-08-06 21:54:06 +03:00
Nikolai Kochetov
599f3ddf6e Merged with master. 2019-08-04 15:35:21 +03:00
proller
77bd2624ea Build fixes (#6016)
* Fix building without submodules

* Fix more gcc9 warnings

* was wrong!
../dbms/src/IO/WriteBufferAIO.cpp:277:54: error: result of comparison 'ssize_t' (aka 'long') > 9223372036854775807 is always false [-Werror,-Wtautological-type-limit-compare]
    if ((static_cast<ssize_t>(flush_buffer.offset()) > std::numeric_limits<off_t>::max()) ||
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* mimalloc off MI_OVERRIDE

* Fix freebsd build

* Fix shared build

* Fix build in gcc9

* Fix split build

* fix

* fix

* fix

* fix

* clean

* zstd

* Update CMakeLists.txt

* Update Defines.h

* Try fix -Wsign-compare

* Freebsd fix

* Add missing libs

* Fix double-conversion include for copy-headers

* Fix zlib link

* Fix includes for arcadia

* Fix includes for arcadia

* Fix includes for arcadia

* Freebsd fix

* Arcadia fixes

* Update QueryProfiler.cpp

* Freebsd fix

* clean

* 19.11: Fixes for arcadia porting (#6223)

* fix glibc-compatibility

* M_LIBRARY fix

* wip

* Fix

* Fix arm build

* unwind fix

* Update CMakeLists.txt
2019-08-04 03:19:03 +03:00
Zhichang Yu
4f146eaa7e added bitmapRange function 2019-08-03 23:07:46 +08:00
alexey-milovidov
e9c2077c71
Merge pull request #6208 from vitlibar/write-buffer-call-next-impl-even-if-zero-offset
Fix formula for new_size in WriteBufferFromVector(AppendModeTag).
2019-08-03 06:27:31 +03:00
Nikolai Kochetov
ab914010e1 Remove TSKVRowInputStream. 2019-08-02 19:56:16 +03:00
Vitaly Baranov
d7891ec9cf Fix formula for new_size in WriteBufferFromVector(AppendModeTag). 2019-08-01 15:27:32 +03:00
Alexey Milovidov
f3921ce3e0 Changed boost::filesystem to std::filesystem 2019-08-01 01:37:41 +03:00
Alexey Milovidov
f03df37b57 Fixed error #6195 2019-07-28 20:16:48 +03:00
Yuriy
383fa25c51 resolved conflict 2019-07-28 02:00:15 +03:00
alexey-milovidov
7219e167d2
Merge pull request #6026 from bopohaa/fix-kafka-unclean-stream
Clear Kafka's buffer if an invalid message is found.
2019-07-25 04:30:37 +03:00
Alexander Tokmakov
7e9c3f2022 PeekableReadBufferCheckpoint 2019-07-22 03:12:11 +03:00
Alexander Tokmakov
456a8c4b17 Fix buffer padding 2019-07-22 03:10:13 +03:00
Alexander Tokmakov
23fddc8688 Fixes 2019-07-22 03:09:35 +03:00
Alexander Tokmakov
4d0a6bad5e PeekableReadBuffer 2019-07-22 03:07:55 +03:00
Alexander Tokmakov
f43f7c43a5 Fixes 2019-07-22 02:59:31 +03:00
Alexander Tokmakov
c92a0f3ac5 Fixes 2019-07-22 02:58:39 +03:00
Alexander Tokmakov
892b67492d Checking for suffix, not for eof 2019-07-22 01:03:29 +03:00
Yuriy
fa2dfcd71d better 2019-07-19 21:43:52 +03:00
Weiqing Xu
ec3c5a369d support reading from HA mode HDSF
the driver libhdfs3 can support HA mode. When the uri doesn't contain
port num, the libhdfs3 will handle it in HA mode, treat the uri as
nameservice name and read the real host and port from configuration file
as the origin Java client.
the default configuration file is hdfs-client.xml in the working
directory, it also can be set in env variable "LIBHDFS3_CONF".
the format of the configuration file is same with hdfs-site.xml.
2019-07-19 07:32:59 +08:00
Nik
4a30eba32f Fix style check 2019-07-16 22:34:47 +04:00
Nik
922c3eb22e Clear Kafka's buffer if an invalid message is found. 2019-07-16 19:27:42 +04:00
Yuriy
1d6e31be4d merged master 2019-07-16 10:33:25 +03:00
alexey-milovidov
5215d13437
Merge pull request #5998 from Enmk/double_delta_fix_2
Fixed DoubleDelta encoding cases for random Int32 and Int64.
2019-07-16 00:54:22 +03:00
Vasily Nemkov
0f4075478e Fixed DoubleDelta encoding cases for random Int32 and Int64. 2019-07-15 08:17:14 +03:00
Yuriy
baa62f2aba style check 2019-07-15 01:53:30 +03:00
Yuriy
0ed77453d3 reading packets in parts 2019-07-15 01:13:56 +03:00
Yuriy
a6bd5bccff tiny fix 2019-07-14 11:27:58 +03:00
Nikolai Kochetov
e9f3d4d6b8
Merge pull request #4914 from yandex/processors
Processors
2019-07-11 19:22:02 +03:00
Alexey Milovidov
499d798b34 Remove "clickhouse_compression" library because it is not a library (cyclic dependency via IDataType) 2019-07-10 00:37:43 +03:00
alexey-milovidov
f6df911307
Update HDFSCommon.cpp 2019-07-09 20:35:47 +03:00
akonyaev
54d890b984 add user parsing in HDFS URI (rewrite split boost->std::string) 2019-07-09 17:09:56 +03:00
akonyaev
c0dc8fc9d3 add user parsing in HDFS URI (fixes after review, style fixes) 2019-07-09 16:16:04 +03:00
akonyaev
74b2440f66 add user parsing in HDFS URI 2019-07-09 13:26:06 +03:00
Alexey Milovidov
e07235e294 Addition to prev. revision 2019-07-08 21:55:06 +03:00
Nikolai Kochetov
209793c981 Merged with master. 2019-07-08 16:00:54 +03:00
Alexey Milovidov
ac20c515ab Better code in unit tests 2019-07-08 04:58:31 +03:00
Alexey Milovidov
03712aabec Add two more warnings from -Weverything 2019-07-08 03:16:39 +03:00
Vitaliy Zakaznikov
a363189dfc Merge branch 'master' of https://github.com/vzakaznikov/ClickHouse into liveview 2019-07-01 18:05:14 -04:00