thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-23 18:20:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
78,243 Commits 8,938 Branches 1,766 Tags 2 GiB
f76149a1ec
Commit Graph

208 Commits

Author SHA1 Message Date
Alexey Milovidov
d604cf5573 Remove the infamous "unbundled" build 2021-11-21 13:58:26 +03:00
Alexander Tokmakov
d095cfe4c1 disable check for zk session uptime by default 2021-10-29 14:45:09 +03:00
Azat Khuzhin
e3074cdd5a clickhouse-test: increase delay for initial SELECT 1 check 
CI report [1]:

<details>

    2021-10-24 04:43:13 Server is not responding. Cannot execute 'SELECT 1' query.             If you are using split build, you have to specify -c option.
    2021-10-24 04:43:13 Cannot get server pid with pidof -s clickhouse-server, got : Command 'pidof -s clickhouse-server' returned non-zero exit status 1.

    2021-10-24 04:43:13 Thread 1 (Thread 0x7f401edd7340 (LWP 370)):
    2021-10-24 04:43:13 0  0x0000000009f9a807 in __sanitizer::StackDepotBase<>::Put(__sanitizer::StackTrace, bool*) ()
    2021-10-24 04:43:13 1  0x0000000009f9a6f7 in __sanitizer::StackDepotPut(__sanitizer::StackTrace) ()
    2021-10-24 04:43:13 2  0x0000000009f2b0e7 in __msan::MsanDeallocate(__sanitizer::StackTrace*, void*) ()
    2021-10-24 04:43:13 3  0x0000000009fa77df in operator delete(void*, unsigned long) ()
    2021-10-24 04:43:13 4  0x000000004185ffc2 in std::__1::__libcpp_operator_delete<void*, unsigned long> () at ../contrib/libcxx/include/new:245
    2021-10-24 04:43:13 5  std::__1::__do_deallocate_handle_size<>(void*, unsigned long) () at ../contrib/libcxx/include/new:271
    2021-10-24 04:43:13 6  std::__1::__libcpp_deallocate () at ../contrib/libcxx/include/new:285
    2021-10-24 04:43:13 7  std::__1::allocator<char>::deallocate () at ../contrib/libcxx/include/memory:849
    2021-10-24 04:43:13 8  std::__1::allocator_traits<std::__1::allocator<char> >::deallocate () at ../contrib/libcxx/include/__memory/allocator_traits.h:476
    2021-10-24 04:43:13 9  std::__1::basic_string<>::~basic_string (this=0x70400000efe8) at ../contrib/libcxx/include/string:2219
    2021-10-24 04:43:13 10 Poco::XML::ContextLocator::~ContextLocator (this=0x70400000efc0) at ../contrib/poco/XML/src/ParserEngine.cpp:53
    2021-10-24 04:43:13 11 0x000000004186010d in Poco::XML::ContextLocator::~ContextLocator (this=0x70400000efc0) at ../contrib/poco/XML/src/ParserEngine.cpp:52
    2021-10-24 04:43:13 12 0x000000004184af75 in Poco::XML::ParserEngine::popContext (this=<optimized out>) at ../contrib/poco/XML/src/ParserEngine.cpp:599
    2021-10-24 04:43:13 13 Poco::XML::ParserEngine::parse (this=<optimized out>, pInputSource=0x706000003720) at ../contrib/poco/XML/src/ParserEngine.cpp:234
    2021-10-24 04:43:13 14 0x000000004184880e in Poco::XML::SAXParser::parse (this=<optimized out>, systemId=...) at ../contrib/poco/XML/src/SAXParser.cpp:201
    2021-10-24 04:43:13 15 0x00000000417f3eeb in Poco::XML::DOMBuilder::parse (this=<optimized out>, uri=...) at ../contrib/poco/XML/src/DOMBuilder.cpp:69
    2021-10-24 04:43:13 16 0x00000000417f2946 in Poco::XML::DOMParser::parse (this=<optimized out>, uri=...) at ../contrib/poco/XML/src/DOMParser.cpp:102
    2021-10-24 04:43:13 17 0x000000003741dd6c in DB::ConfigProcessor::processConfig (this=this@entry=0x7fff89847130, has_zk_includes=has_zk_includes@entry=0x7fff898470f7, zk_node_cache=zk_node_cache@entry=0x0, zk_changed_event=...) at ../src/Common/Config/ConfigProcessor.cpp:570
    2021-10-24 04:43:13 18 0x0000000037427bb7 in DB::ConfigProcessor::loadConfig (this=<optimized out>, allow_zk_includes=true) at ../src/Common/Config/ConfigProcessor.cpp:657
    2021-10-24 04:43:13 19 0x000000003743fb76 in DB::ConfigReloader::reloadIfNewer (this=this@entry=0x712000002700, force=true, throw_on_error=true, fallback_to_preprocessed=true, initial_loading=true) at ../src/Common/Config/ConfigReloader.cpp:96
    2021-10-24 04:43:13 20 0x000000003743d85e in DB::ConfigReloader::ConfigReloader() () at ../src/Common/Config/ConfigReloader.cpp:33
    2021-10-24 04:43:13 21 0x0000000009fd3efb in std::__1::make_unique<>() (__args=<optimized out>, __args=<optimized out>, __args=<optimized out>, __args=<optimized out>, __args=<optimized out>, __args=<optimized out>, __args=<optimized out>) at ../contrib/libcxx/include/memory:2068
    2021-10-24 04:43:13 22 DB::Server::main (this=<optimized out>) at ../programs/server/Server.cpp:803
    2021-10-24 04:43:13 23 0x000000004175c5f9 in Poco::Util::Application::run (this=<optimized out>) at ../contrib/poco/Util/src/Application.cpp:334
    2021-10-24 04:43:13 24 0x0000000009fb5d7f in DB::Server::run (this=<optimized out>) at ../programs/server/Server.cpp:405

</details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/30611/5ff6c5536558821824d5fdf25a75729e5b82060d/functional_stateless_tests_(memory).html#fail1
 2021-10-24 19:04:36 +03:00
Azat Khuzhin
09b782a52e Increase default wait of the server start in clickhouse-test 
Set --server-check-retries to 90 (and this is ~45 seconds), since right
now sometimes it is not enough [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/30191/0e34a9d550cfe6924fe575871f36c44dd44acdaa/functional_stateless_tests_(memory).html#fail1

And the reason I guess is clickhouse-test had been rewritten to
http.client in #30065, and since now it does not need to execute
clickhouse-client binary, which in debug/sanitizers builds can take also
sometime.
That said that with clickhouse-client for hung check it was not 15
seconds, but more (each clickhouse-client requires 0.6sec with
sanitizers for simple SELECT 1, while w/o 0.1second, also too much
should be optimized)
 2021-10-18 10:46:02 +03:00
Azat Khuzhin
612a21cc97 clickhouse-test: do not propagate CLICKHOUSE_PORT_HTTP to clickhouse-client 2021-10-16 17:59:56 +03:00
Azat Khuzhin
c6e61e9497 clickhouse-test: use splitlines() over split('\n') 2021-10-16 17:59:56 +03:00
Azat Khuzhin
3102d24872 Fix --hung-check in clickhouse-test 
JSONEachRow cannot be parsed with a simple json.loads(), instead it
should be passed to json.loads() line by line.

Fixes: #30065
 2021-10-16 17:59:56 +03:00
Azat Khuzhin
e5bc573250 clickhouse-test: fix hung check in stress test by using system database 2021-10-13 00:19:16 +03:00
Azat Khuzhin
8d7798fa2e clickhouse-test: do not guard CREATE DATABASE with try/catch 
Since it will not configure testcase args and fail eventually, and later
we have a try/catch anyway, this should be enough.
 2021-10-12 22:39:43 +03:00
Azat Khuzhin
40d210367f clickhouse-test: fix catching of timeouts 2021-10-12 22:33:41 +03:00
Azat Khuzhin
427c428a27 clickhouse-test: process some options regardless --client 2021-10-12 22:30:34 +03:00
Azat Khuzhin
8dc8674298 clickhouse-test: process --client-option 2021-10-12 22:30:16 +03:00
Azat Khuzhin
fe90c979b5 clickhouse-test: fix redirect to stderr 
Before #29856 `CREATE DATABASE` overwrites it.

Reported-by: @amosbird
 2021-10-12 22:18:39 +03:00
Azat Khuzhin
50b95bd89d clickhouse-test: passthrough log_comment for DROP DATABASE too 2021-10-12 21:09:48 +03:00
Azat Khuzhin
e2e62ce273 clickhouse-test: replace clickhouse-driver with http interface (via http.client) 
Cons of clickhouse-driver:
- it is one more extra dependency
- it does not have correct timeouts (only for socket operations, and
  this is not the same, so we need to set timeout by ourself)
- it is one more thing which can break (@alesapin)
 2021-10-12 21:08:33 +03:00
alesapin
3ae960e04b Review fixes 2021-10-11 16:40:12 +03:00
alesapin
59a78830f9 Better timeouts in clickhouse-test 2021-10-11 14:46:01 +03:00
Azat Khuzhin
42ca2b4bb2 clickhouse-test: remove not existing options for pylint 
https://clickhouse-test-reports.s3.yandex.net/29856/e2d6698244d43979b3fe2478dfdcd8dc3a91a0fd/style_check/test_run.txt.out.log
 2021-10-09 01:43:00 +03:00
Azat Khuzhin
5d6da023bb clickhouse-test: fix hung check under stress tests 
https://clickhouse-test-reports.s3.yandex.net/29856/e2d6698244d43979b3fe2478dfdcd8dc3a91a0fd/stress_test_(address).html#fail1
 2021-10-09 01:42:08 +03:00
Azat Khuzhin
e2d6698244 clickhouse-test: do not use persistent connection for simplicity (due to threads) 2021-10-08 00:09:37 +03:00
Azat Khuzhin
df129d7efc Rewrite clickhouse-test to use python clickhouse_driver 
Pros:
- Using native protocol over executing binaries is always better
- `clickhouse-client` in debug build takes almost a second to execute simple `SELECT 1`
  and `clickhouse-test` requires ~5 queries at start (determine some
  flags, zk, alive, create database)

Notes:
- `FORMAT Vertical` had been replaced with printing of `pandas.DataFrame`

And after this patch tiny tests work with the speed of the test, and
does not requires +-5 seconds of bootstrapping.
 2021-10-08 00:09:37 +03:00
Azat Khuzhin
9dd0fca1ed Suppress some existed warnings in clickhouse-test (will be fixed separately) 2021-10-08 00:09:37 +03:00
tavplubix
75be60ff22
Merge pull request #29452 from azat/clickhouse-test-fix-term 
clickhouse-test: fix shared list object (by fixing manager lifetime)
 2021-09-28 22:34:21 +03:00
Azat Khuzhin
985e8ee061 clickhouse-test: fix shared list object (by fixing manager lifetime) 
Right now it is possible to get the following error:

    Having 20 errors! 0 tests passed. 0 tests skipped. 57.37 s elapsed (MainProcess).
    Won't run stateful tests because test data wasn't loaded.
    Traceback (most recent call last):
      File "/usr/lib/python3.9/multiprocessing/managers.py", line 802, in _callmethod
        conn = self._tls.connection
    AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/src/ch/clickhouse/.cmake/../tests/clickhouse-test", line 1462, in <module>
        main(args)
      File "/src/ch/clickhouse/.cmake/../tests/clickhouse-test", line 1261, in main
        if len(restarted_tests) > 0:
      File "<string>", line 2, in __len__
      File "/usr/lib/python3.9/multiprocessing/managers.py", line 806, in _callmethod
        self._connect()
      File "/usr/lib/python3.9/multiprocessing/managers.py", line 793, in _connect
        conn = self._Client(self._token.address, authkey=self._authkey)
      File "/usr/lib/python3.9/multiprocessing/connection.py", line 507, in Client
        c = SocketClient(address)
      File "/usr/lib/python3.9/multiprocessing/connection.py", line 635, in SocketClient
        s.connect(address)
    ConnectionRefusedError: [Errno 111] Connection refused

The reason behind this is that manager's thread got terminated:

    ipdb> p restarted_tests._manager._process
    <ForkProcess name='SyncManager-1' pid=25125 parent=24939 stopped exitcode=-SIGTERM>

Refs: #29259 (cc: @vdimir)
Follow-up for: #29197 (cc: @tavplubix)
 2021-09-27 21:10:59 +03:00
Azat Khuzhin
8be6a59e65 clickhouse-test: fix long tag check for flaky check (--test-runs > 1) 
Otherwise it will mark the test as failed, like in [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/29369/2320be204397bbb9f9a673cf093b7967fd37a107/functional_stateless_tests_flaky_check_(address).html#fail1

Follow-up for: #28909 (Cc: @vitlibar)
 2021-09-27 21:10:14 +03:00
tavplubix
1c9778603f
Update clickhouse-test 2021-09-27 11:43:00 +03:00
Vladimir C
c83e2f6fd4
Merge pull request #29259 from vdimir/test-global-multiproc-fix 2021-09-22 17:47:08 +03:00
tavplubix
7712d0e9df
Update clickhouse-test 2021-09-22 16:17:51 +03:00
vdimir
3a42d11b65
Initialize global variables for multiproc in __main__ in clickhouse-test 2021-09-22 16:00:59 +03:00
tavplubix
e0ce179aa8
Minor improvements in clickhouse-test (#29197) 
* minor improvements in clickhouse-test

* use enums

* Update clickhouse-test
 2021-09-22 11:42:46 +03:00
tavplubix
66bc619c3e
Merge pull request #28983 from ClickHouse/func_zookeeper_session_uptime 
Add function zookeeperSessionUptime(), fix some flaky tests
 2021-09-20 14:47:49 +03:00
tavplubix
5fdd2b67ee
Update clickhouse-test 2021-09-19 01:05:17 +03:00
Alexander Tokmakov
5c605189ac Merge branch 'master' into func_zookeeper_session_uptime 2021-09-15 21:10:06 +03:00
Alexander Tokmakov
be45ed5fde better check for session expiration in clickhouse-test 2021-09-13 16:36:05 +03:00
Vitaly Baranov
bbb192ee85 Convert skip_list.json into first line comments. 2021-09-12 17:15:23 +03:00
Nikolai Kochetov
0e0c136d43 Fix error code of clickhouse-test if server is dead. 2021-09-09 15:31:29 +03:00
mergify[bot]
02b4e56e6a
Merge branch 'master' into break_some_tests 2021-08-20 13:24:17 +00:00
tavplubix
64bfe21a1b
Fix test 00443_preferred_block_size_bytes.sh (#27846) 
* Update 00443_preferred_block_size_bytes.sh

* Update clickhouse-test

* Update clickhouse-test

* Update database_replicated.xml
 2021-08-20 00:25:14 +03:00
Alexander Tokmakov
21f2da6d33 Merge branch 'master' into break_some_tests 2021-08-15 22:02:36 +03:00
Alexander Tokmakov
9dd742cc29 more debug info 2021-08-12 13:39:06 +03:00
Alexander Tokmakov
0256e313b3 fix 2021-08-12 00:29:37 +03:00
Raúl Marín
4ebdb5917a clickhouse-test: Implement @@SKIP@@ directive in tests 2021-08-11 15:22:31 +02:00
Alexander Kuzmenkov
2259387f0c make it possible to cancel window functions on ctrl+c 2021-08-09 20:04:42 +03:00
Mike Kot
a308c8f842 Extracted test skipping functions 2021-08-06 17:39:44 +03:00
Mike Kot
af536b1b5e Using formatted string literals, extracted sort funcs in tester 2021-08-06 17:39:44 +03:00
Amos Bird
b541b5eca3
Normalize hostname in CI 2021-08-03 15:00:57 +08:00
Alexander Tokmakov
381634960e fix client options in stress test 2021-07-29 15:16:34 +03:00
tavplubix
858bef8bef
Update clickhouse-test 2021-07-26 19:19:47 +03:00
Alexey Milovidov
d815023700 Fix one source of flaky tests 2021-07-21 03:58:01 +03:00
vdimir
1b5de5de0d
Fix style in clickhouse-test removesuffix 2021-07-20 17:51:22 +03:00