Commit Graph

176 Commits

Author SHA1 Message Date
Antonio Andelic
67567fcff4 Fix build 2024-07-24 19:51:22 +02:00
Raúl Marín
e559899193 Random missing headers 2024-07-12 00:00:47 +02:00
Raúl Marín
9e599576ab Hide Settings object from Context.h 2024-07-11 14:00:05 +02:00
Kseniia Sumarokova
dd9bf6d792
Merge pull request #65738 from RodolpheDuge/fix_odbc_nullable_fields
fix ODBC table with nullable fields
2024-07-03 09:43:47 +00:00
Nikita Taranov
b8e803218c remote shart 2024-06-27 13:32:23 +01:00
Rodolphe Dugé de Bernonville
c8bca3135d fix odbc and nullable fields 2024-06-27 09:55:24 +02:00
Nikita Taranov
89a8925f80 impl 2024-06-26 12:02:15 +01:00
Robert Schulze
2909e6451b
Move StringUtils.h/cpp back to Common/ 2024-05-19 09:39:36 +00:00
Alexey Milovidov
224e1940ae Useless changes 2024-05-09 03:58:34 +02:00
Alexey Milovidov
0d14a2c67e Useless changes 2024-05-09 03:11:02 +02:00
Alexey Milovidov
89cee0a3d6 Move bridges to separate packages 2024-03-31 01:59:36 +01:00
Alexey Milovidov
aca3978f2d Remove DataStreams (2) 2024-03-24 17:21:53 +01:00
Alexey Milovidov
4373d5ba16 Merge branch 'master' into split-cast-overload-resolver 2024-03-11 03:01:50 +01:00
Alexey Milovidov
157adefc70 Fix error 2024-03-10 09:17:29 +01:00
Alexey Milovidov
1a2d403f7b Miscellaneae 2024-03-10 09:16:21 +01:00
Alexey Milovidov
9ddec346a0 Miscellaneous 2024-03-10 09:02:34 +01:00
Alexey Milovidov
3ac4f56cfa Fix tests 2024-03-09 18:53:31 +01:00
Alexey Milovidov
e9ab3ed2dd Even better 2024-03-09 09:27:11 +01:00
Alexey Milovidov
47b308d234 Simplify bridges 2024-03-09 08:42:33 +01:00
Alexey Milovidov
574d486322 Something 2024-03-09 07:55:59 +01:00
Alexey Milovidov
6d45eecdad Remove garbage 2024-03-09 06:11:34 +01:00
Alexey Milovidov
2be09581dd Split CastOverloadResolver translation unit 2024-03-09 05:48:52 +01:00
Robert Schulze
3b18eb4f17
Fix clang-tidy in some headers 2024-02-28 22:47:34 +00:00
Maksim Kita
2a327107b6 Updated implementation 2024-01-25 14:31:49 +03:00
Yakov Olkhovskiy
85f03478ef
Revert "Revert "Use CH Buffer for HTTP out stream, add metrics for interfaces"" 2024-01-03 11:47:15 -05:00
Raúl Marín
d491758939
Revert "Use CH Buffer for HTTP out stream, add metrics for interfaces" 2024-01-03 10:42:15 +01:00
Yakov Olkhovskiy
db97764e98 fix tests, some refactoring 2023-12-31 12:56:37 +00:00
Yakov Olkhovskiy
001a38048f use ProfileEvents instead of CurrentMetrics 2023-12-15 19:17:42 +00:00
Yakov Olkhovskiy
4f11132ea2 fix clang tidy 2023-11-04 00:00:14 +00:00
Yakov Olkhovskiy
0cf851316c use CH Buffer for HTTP out stream, add metrics for interfaces 2023-10-27 02:38:36 +00:00
vdimir
aae3894c23
Throw an exception in odbc-bridge if more than one table matched a single query 2023-10-16 10:43:36 +00:00
vdimir
cf1deb7bd5
Fix 'Invalid cursor state' in odbc interacting with MS SQL Server 2023-10-16 10:43:33 +00:00
Alexey Milovidov
d3c3d8b8e4 Remove export of dynamic symbols 2023-05-06 23:52:16 +02:00
Robert Schulze
f8980c582e
CMake: More removal of gold linker (follow-up to #47660)
+ fix a linker warning
2023-03-17 11:01:46 +00:00
Azat Khuzhin
a72647690d Change error code in case of columns definitions was empty in ODBC
CI reports [1]:

    2023.03.14 00:29:07.031349 [ 166170 ] {110f8654-7d7d-4b47-b6b0-3ce83414a80f} <Error> ReadWriteBufferFromHTTP: HTTP request to `http://127.0.0.1:9018/columns_info?use_connection_pooling=1&version=1&connection_string=DSN%3D%7BClickHouse%20DSN%20%28ANSI%29%7D&schema=test_15&table=t&external_table_functions_use_nulls=1` failed at try 1/1 with bytes read: 0/unknown. Error: DB::HTTPException: Received error from remote server /columns_info?use_connection_pooling=1&version=1&connection_string=DSN%3D%7BClickHouse%20DSN%20%28ANSI%29%7D&schema=test_15&table=t&external_table_functions_use_nulls=1. HTTP status code: 500 Internal Server Error, body: Error getting columns from ODBC 'Code: 49. DB::Exception: Columns definition was not returned. (LOGICAL_ERROR) (version 23.2.4.12 (official build))'

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/47541/3d247b8635da44bccfdeb5fcd53be7130b8d0a32/upgrade_check__msan_.html

Here the problem is that system.columns has cached value for number of
total table to iterate, and so it can skip something.

But anyway, this should be LOGICAL_ERROR, since ODBC bridge does two
queries:
- to system.tables and
- to system.columns

And if between this two queries the table will be removed, them there
will be no columns

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-14 14:24:11 +01:00
Bharat Nallan
2ef8fcb318
Merge branch 'master' into ncb/odbc-connection-pool-fixes 2023-01-24 21:27:20 -08:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
Bharat Nallan Chakravarthy
16a2585d55 fixes to odbc connection pooling 2023-01-18 16:38:56 -08:00
Azat Khuzhin
905a95e166 Review fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 22:40:13 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH}
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
Robert Schulze
810221baf2
Assume unversioned server has version=0 and use tryParse() instead of from_chars() 2022-08-10 07:39:32 +00:00
Robert Schulze
e0d5020a92
Add simple versioning to the *-bridge-to-server protocol
- In general, it is expected that clickhouse-*-bridges and
  clickhouse-server were build from the same source version (e.g. are
  upgraded "atomically"). If that is not the case, we should at least
  be able to detect the mismatch and abort.

- This commit adds a URL parameter "version", defined in a header shared
  by the server and bridges. The bridge returns an error in case of
  mismatch.

- The version is *not* send and checked for "ping" requests (used for
  handshake), only for regular requests send after handshake. This is
  because the internally thrown server-side exception due to HTTP
  failure does not propagate the exact HTTP error (it only stores the
  error as text), and as a result, the server-side handshake code
  simply retries in case of error with exponential backoff and finally
  fails with a "timeout error". This is reasonable as pings typically
  fail due to time out. However, without a rework of HTTP exceptions,
  version mismatch during ping would also appear as "timeout" which is
  too misleading. The behavior may be changed later if needed.

- Note that introducing a version parameter does not represent a
  protocol upgrade itself. Bridges older than the server will simply
  ignore the field. Only servers older than the bridges receive an error
  but such a situation should never occur in practice.
2022-08-08 19:40:37 +00:00
Robert Schulze
ea73b98fb9
Prepare library-bridge for catboost integration
- Rename generic file and identifier names in library-bridge to
  something more dictionary-specific. This is needed because later on,
  catboost will be integrated into library-bridge.

- Also: Some smaller fixes like typos and un-inlining non-performance
  critical code.

- The logic remains unchanged in this commit.
2022-08-04 19:26:51 +00:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_"
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
2022-07-11 19:36:18 +02:00
Robert Schulze
bb358617e1
Better naming for stuff related to splitted debug symbols
The previous name was slightly misleading, e.g. it is not about
"intalling stripped binaries" but about splitting debug symbols from the
binary.
2022-06-30 23:41:27 +02:00
kssenii
dc73042d62 Better error messafe 2022-06-24 01:05:33 +02:00
Robert Schulze
5a4f21c50f
Support for Clang Thread Safety Analysis (TSA)
- TSA is a static analyzer build by Google which finds race conditions
  and deadlocks at compile time.

- It works by associating a shared member variable with a
  synchronization primitive that protects it. The compiler can then
  check at each access if proper locking happened before. A good
  introduction are [0] and [1].

- TSA requires some help by the programmer via annotations. Luckily,
  LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
  std::shared_mutex and std::scoped_lock. This commit enables them
  (--> contrib/libcxx-cmake/CMakeLists.txt).

- Further, this commit adds convenience macros for the low-level
  annotations for use in ClickHouse (--> base/defines.h). For
  demonstration, they are leveraged in a few places.

- As we compile with "-Wall -Wextra -Weverything", the required compiler
  flag "-Wthread-safety-analysis" was already enabled. Negative checks
  are an experimental feature of TSA and disabled
  (--> cmake/warnings.cmake). Compile times did not increase noticeably.

- TSA is used in a few places with simple locking. I tried TSA also
  where locking is more complex. The problem was usually that it is
  unclear which data is protected by which lock :-(. But there was
  definitely some weird code where locking looked broken. So there is
  some potential to find bugs.

*** Limitations of TSA besides the ones listed in [1]:

- The programmer needs to know which lock protects which piece of shared
  data. This is not always easy for large classes.

- Two synchronization primitives used in ClickHouse are not annotated in
  libcxx:
  (1) std::unique_lock: A releaseable lock handle often together with
      std::condition_variable, e.g. in solve producer-consumer problems.
  (2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
      considered a design flaw + typically it is slower than a standard
      mutex. In this commit, one std::recursive_mutex was converted to
      std::mutex and annotated with TSA.

- For free-standing functions (e.g. helper functions) which are passed
  shared data members, it can be tricky to specify the associated lock.
  This is because the annotations use the normal C++ rules for symbol
  resolution.

[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
2022-06-20 16:13:25 +02:00
Anton Kozlov
e9d645168e Renamed ODBCConnectionFactory to ODBCPooledConntionFactory 2022-06-01 09:00:39 +00:00
Anton Kozlov
3576625647 CLICKHOUSE-2131 Add an option to disable connection pooling in ODBC bridge 2022-05-31 16:26:08 +00:00