ClickHouse/programs/odbc-bridge/ODBCPooledConnectionFactory.h

178 lines
4.9 KiB
C++
Raw Normal View History

2021-03-31 12:41:12 +00:00
#pragma once
2022-04-27 15:05:45 +00:00
#include <Common/logger_useful.h>
2021-03-31 12:41:12 +00:00
#include <nanodbc/nanodbc.h>
#include <mutex>
2021-10-02 07:13:14 +00:00
#include <base/BorrowedObjectPool.h>
Support for Clang Thread Safety Analysis (TSA) - TSA is a static analyzer build by Google which finds race conditions and deadlocks at compile time. - It works by associating a shared member variable with a synchronization primitive that protects it. The compiler can then check at each access if proper locking happened before. A good introduction are [0] and [1]. - TSA requires some help by the programmer via annotations. Luckily, LLVM's libcxx already has annotations for std::mutex, std::lock_guard, std::shared_mutex and std::scoped_lock. This commit enables them (--> contrib/libcxx-cmake/CMakeLists.txt). - Further, this commit adds convenience macros for the low-level annotations for use in ClickHouse (--> base/defines.h). For demonstration, they are leveraged in a few places. - As we compile with "-Wall -Wextra -Weverything", the required compiler flag "-Wthread-safety-analysis" was already enabled. Negative checks are an experimental feature of TSA and disabled (--> cmake/warnings.cmake). Compile times did not increase noticeably. - TSA is used in a few places with simple locking. I tried TSA also where locking is more complex. The problem was usually that it is unclear which data is protected by which lock :-(. But there was definitely some weird code where locking looked broken. So there is some potential to find bugs. *** Limitations of TSA besides the ones listed in [1]: - The programmer needs to know which lock protects which piece of shared data. This is not always easy for large classes. - Two synchronization primitives used in ClickHouse are not annotated in libcxx: (1) std::unique_lock: A releaseable lock handle often together with std::condition_variable, e.g. in solve producer-consumer problems. (2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be considered a design flaw + typically it is slower than a standard mutex. In this commit, one std::recursive_mutex was converted to std::mutex and annotated with TSA. - For free-standing functions (e.g. helper functions) which are passed shared data members, it can be tricky to specify the associated lock. This is because the annotations use the normal C++ rules for symbol resolution. [0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html [1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
2022-06-14 22:35:55 +00:00
#include <base/defines.h>
2021-03-31 12:41:12 +00:00
#include <unordered_map>
2021-05-07 10:37:11 +00:00
namespace DB
{
namespace ErrorCodes
{
extern const int NO_FREE_CONNECTION;
}
}
2021-03-31 12:41:12 +00:00
namespace nanodbc
{
2021-05-07 10:37:11 +00:00
using ConnectionPtr = std::unique_ptr<nanodbc::connection>;
2021-03-31 12:41:12 +00:00
using Pool = BorrowedObjectPool<ConnectionPtr>;
using PoolPtr = std::shared_ptr<Pool>;
2021-06-07 18:09:16 +00:00
static constexpr inline auto ODBC_CONNECT_TIMEOUT = 100;
2021-03-31 12:41:12 +00:00
class ConnectionHolder
{
public:
2021-06-07 18:09:16 +00:00
ConnectionHolder(PoolPtr pool_,
ConnectionPtr connection_,
const String & connection_string_)
2021-06-07 18:09:16 +00:00
: pool(pool_)
, connection(std::move(connection_))
, connection_string(connection_string_)
{
}
2021-03-31 12:41:12 +00:00
explicit ConnectionHolder(const String & connection_string_)
: pool(nullptr)
, connection()
, connection_string(connection_string_)
{
updateConnection();
}
2021-05-07 10:37:11 +00:00
ConnectionHolder(const ConnectionHolder & other) = delete;
2021-03-31 12:41:12 +00:00
2021-06-07 12:15:24 +00:00
~ConnectionHolder()
{
if (pool != nullptr)
pool->returnObject(std::move(connection));
2021-06-07 12:15:24 +00:00
}
2021-03-31 12:41:12 +00:00
2021-05-07 10:37:11 +00:00
nanodbc::connection & get() const
{
assert(connection != nullptr);
2021-03-31 12:41:12 +00:00
return *connection;
}
2021-06-07 18:09:16 +00:00
void updateConnection()
{
connection = std::make_unique<nanodbc::connection>(connection_string, ODBC_CONNECT_TIMEOUT);
}
2021-03-31 12:41:12 +00:00
private:
PoolPtr pool;
ConnectionPtr connection;
String connection_string;
2021-03-31 12:41:12 +00:00
};
2021-05-07 11:18:49 +00:00
2021-06-07 18:09:16 +00:00
using ConnectionHolderPtr = std::shared_ptr<ConnectionHolder>;
2021-03-31 12:41:12 +00:00
}
namespace DB
{
2021-05-07 10:37:11 +00:00
static constexpr inline auto ODBC_CONNECT_TIMEOUT = 100;
static constexpr inline auto ODBC_POOL_WAIT_TIMEOUT = 10000;
2021-06-07 18:09:16 +00:00
template <typename T>
T execute(nanodbc::ConnectionHolderPtr connection_holder, std::function<T(nanodbc::connection &)> query_func)
2021-03-31 12:41:12 +00:00
{
2021-06-07 18:09:16 +00:00
try
2021-03-31 12:41:12 +00:00
{
2021-06-07 18:09:16 +00:00
return query_func(connection_holder->get());
2021-03-31 12:41:12 +00:00
}
2021-06-07 18:09:16 +00:00
catch (const nanodbc::database_error & e)
2021-06-07 12:15:24 +00:00
{
2022-06-23 23:05:33 +00:00
LOG_ERROR(
&Poco::Logger::get("ODBCConnection"),
"ODBC query failed with error: {}, state: {}, native code: {}",
e.what(), e.state(), e.native());
2021-09-29 09:43:58 +00:00
/// SQLState, connection related errors start with 08 (main: 08S01), cursor invalid state is 24000.
/// Invalid cursor state is a retriable error.
2021-09-29 13:18:24 +00:00
/// Invalid transaction state 25000. Truncate to 2 letters on purpose.
2021-09-29 09:43:58 +00:00
/// https://docs.microsoft.com/ru-ru/sql/odbc/reference/appendixes/appendix-a-odbc-error-codes?view=sql-server-ver15
2021-09-29 13:18:24 +00:00
if (e.state().starts_with("08") || e.state().starts_with("24") || e.state().starts_with("25"))
2021-06-07 12:15:24 +00:00
{
2021-06-07 18:09:16 +00:00
connection_holder->updateConnection();
return query_func(connection_holder->get());
2021-06-07 12:15:24 +00:00
}
/// psqlodbc driver error handling is incomplete and under some scenarious
/// it doesn't propagate correct errors to the caller.
/// As a quick workaround we run a quick "ping" query over the connection
/// on generic errors.
/// If "ping" fails, recycle the connection and try the query once more.
if (e.state().starts_with("HY00"))
{
try
{
just_execute(connection_holder->get(), "SELECT 1");
}
catch (...)
{
connection_holder->updateConnection();
return query_func(connection_holder->get());
}
}
2021-06-07 18:09:16 +00:00
throw;
}
}
2021-06-07 12:15:24 +00:00
class ODBCPooledConnectionFactory final : private boost::noncopyable
2021-06-07 18:09:16 +00:00
{
public:
static ODBCPooledConnectionFactory & instance()
2021-06-07 18:09:16 +00:00
{
static ODBCPooledConnectionFactory ret;
2021-06-07 18:09:16 +00:00
return ret;
2021-06-07 12:15:24 +00:00
}
2021-05-07 11:18:49 +00:00
nanodbc::ConnectionHolderPtr get(const std::string & connection_string, size_t pool_size)
2021-03-31 12:41:12 +00:00
{
std::lock_guard lock(mutex);
if (!factory.count(connection_string))
factory.emplace(std::make_pair(connection_string, std::make_shared<nanodbc::Pool>(pool_size)));
2021-05-07 10:37:11 +00:00
auto & pool = factory[connection_string];
nanodbc::ConnectionPtr connection;
auto connection_available = pool->tryBorrowObject(connection, []() { return nullptr; }, ODBC_POOL_WAIT_TIMEOUT);
if (!connection_available)
throw Exception(ErrorCodes::NO_FREE_CONNECTION, "Unable to fetch connection within the timeout");
2021-05-07 10:37:11 +00:00
try
{
2021-06-07 18:09:16 +00:00
if (!connection)
2021-05-07 10:37:11 +00:00
connection = std::make_unique<nanodbc::connection>(connection_string, ODBC_CONNECT_TIMEOUT);
}
catch (...)
{
pool->returnObject(std::move(connection));
2021-06-07 12:15:24 +00:00
throw;
2021-05-07 10:37:11 +00:00
}
2021-06-07 18:09:16 +00:00
return std::make_unique<nanodbc::ConnectionHolder>(factory[connection_string], std::move(connection), connection_string);
2021-03-31 12:41:12 +00:00
}
private:
2021-04-06 18:59:34 +00:00
/// [connection_settings_string] -> [connection_pool]
2021-03-31 12:41:12 +00:00
using PoolFactory = std::unordered_map<std::string, nanodbc::PoolPtr>;
Support for Clang Thread Safety Analysis (TSA) - TSA is a static analyzer build by Google which finds race conditions and deadlocks at compile time. - It works by associating a shared member variable with a synchronization primitive that protects it. The compiler can then check at each access if proper locking happened before. A good introduction are [0] and [1]. - TSA requires some help by the programmer via annotations. Luckily, LLVM's libcxx already has annotations for std::mutex, std::lock_guard, std::shared_mutex and std::scoped_lock. This commit enables them (--> contrib/libcxx-cmake/CMakeLists.txt). - Further, this commit adds convenience macros for the low-level annotations for use in ClickHouse (--> base/defines.h). For demonstration, they are leveraged in a few places. - As we compile with "-Wall -Wextra -Weverything", the required compiler flag "-Wthread-safety-analysis" was already enabled. Negative checks are an experimental feature of TSA and disabled (--> cmake/warnings.cmake). Compile times did not increase noticeably. - TSA is used in a few places with simple locking. I tried TSA also where locking is more complex. The problem was usually that it is unclear which data is protected by which lock :-(. But there was definitely some weird code where locking looked broken. So there is some potential to find bugs. *** Limitations of TSA besides the ones listed in [1]: - The programmer needs to know which lock protects which piece of shared data. This is not always easy for large classes. - Two synchronization primitives used in ClickHouse are not annotated in libcxx: (1) std::unique_lock: A releaseable lock handle often together with std::condition_variable, e.g. in solve producer-consumer problems. (2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be considered a design flaw + typically it is slower than a standard mutex. In this commit, one std::recursive_mutex was converted to std::mutex and annotated with TSA. - For free-standing functions (e.g. helper functions) which are passed shared data members, it can be tricky to specify the associated lock. This is because the annotations use the normal C++ rules for symbol resolution. [0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html [1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
2022-06-14 22:35:55 +00:00
PoolFactory factory TSA_GUARDED_BY(mutex);
2021-03-31 12:41:12 +00:00
std::mutex mutex;
};
}