thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-30 19:42:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into safer-cut-and-concat

Browse Source
This commit is contained in:
Nikolai Kochetov 2023-04-26 12:49:14 +02:00 committed by GitHub
commit c0e680bac7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
391 changed files with 8977 additions and 10787 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
.github/workflows/pull_request.yml vendored
View File

@ -1308,6 +1308,40 @@ jobs:
docker ps --quiet | xargs --no-run-if-empty docker kill ||:
docker ps --all --quiet | xargs --no-run-if-empty docker rm -f ||:
sudo rm -fr "$TEMP_PATH"
FunctionalStatelessTestReleaseAnalyzer:
needs: [BuilderDebRelease]
runs-on: [self-hosted, func-tester]
steps:
- name: Set envs
run: |
cat >> "$GITHUB_ENV" << 'EOF'
TEMP_PATH=${{runner.temp}}/stateless_analyzer
REPORTS_PATH=${{runner.temp}}/reports_dir
CHECK_NAME=Stateless tests (release, analyzer)
REPO_COPY=${{runner.temp}}/stateless_analyzer/ClickHouse
KILL_TIMEOUT=10800
EOF
- name: Download json reports
uses: actions/download-artifact@v3
with:
path: ${{ env.REPORTS_PATH }}
- name: Check out repository code
uses: ClickHouse/checkout@v1
with:
clear-repository: true
- name: Functional test
run: |
sudo rm -fr "$TEMP_PATH"
mkdir -p "$TEMP_PATH"
cp -r "$GITHUB_WORKSPACE" "$TEMP_PATH"
cd "$REPO_COPY/tests/ci"
python3 functional_test_check.py "$CHECK_NAME" "$KILL_TIMEOUT"
- name: Cleanup
if: always()
run: |
docker ps --quiet | xargs --no-run-if-empty docker kill ||:
docker ps --all --quiet | xargs --no-run-if-empty docker rm -f ||:
sudo rm -fr "$TEMP_PATH"
FunctionalStatelessTestReleaseS3_0:
needs: [BuilderDebRelease]
runs-on: [self-hosted, func-tester]
@ -4755,6 +4789,7 @@ jobs:
- FunctionalStatelessTestReleaseDatabaseReplicated2
- FunctionalStatelessTestReleaseDatabaseReplicated3
- FunctionalStatelessTestReleaseWideParts
- FunctionalStatelessTestReleaseAnalyzer
- FunctionalStatelessTestAarch64
- FunctionalStatelessTestAsan0
- FunctionalStatelessTestAsan1

150
CHANGELOG.md
View File

@ -1,4 +1,5 @@
### Table of Contents
**[ClickHouse release v23.4, 2023-04-26](#234)**<br/>
**[ClickHouse release v23.3 LTS, 2023-03-30](#233)**<br/>
**[ClickHouse release v23.2, 2023-02-23](#232)**<br/>
**[ClickHouse release v23.1, 2023-01-25](#231)**<br/>
@ -6,6 +7,155 @@
# 2023 Changelog
### <a id="234"></a> ClickHouse release 23.4 LTS, 2023-04-26
#### Backward Incompatible Change
* Formatter '%M' in function formatDateTime() now prints the month name instead of the minutes. This makes the behavior consistent with MySQL. The previous behavior can be restored using setting "formatdatetime_parsedatetime_m_is_month_name = 0". [#47246](https://github.com/ClickHouse/ClickHouse/pull/47246) ([Robert Schulze](https://github.com/rschu1ze)).
* This change makes sense only if you are using the virtual filesystem cache. If `path` in the virtual filesystem cache configuration is not empty and is not an absolute path, then it will be put in `<clickhouse server data directory>/caches/<path_from_cache_config>`. [#48784](https://github.com/ClickHouse/ClickHouse/pull/48784) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Primary/secondary indices and sorting keys with identical expressions are now rejected. This behavior can be disabled using setting `allow_suspicious_indices`. [#48536](https://github.com/ClickHouse/ClickHouse/pull/48536) ([凌涛](https://github.com/lingtaolf)).
#### New Feature
* Support new aggregate function `quantileGK`/`quantilesGK`, like [approx_percentile](https://spark.apache.org/docs/latest/api/sql/index.html#approx_percentile) in spark. Greenwald-Khanna algorithm refer to http://infolab.stanford.edu/~datar/courses/cs361a/papers/quantiles.pdf. [#46428](https://github.com/ClickHouse/ClickHouse/pull/46428) ([李扬](https://github.com/taiyang-li)).
* Add a statement `SHOW COLUMNS` which shows distilled information from system.columns. [#48017](https://github.com/ClickHouse/ClickHouse/pull/48017) ([Robert Schulze](https://github.com/rschu1ze)).
* Added `LIGHTWEIGHT` and `PULL` modifiers for `SYSTEM SYNC REPLICA` query. `LIGHTWEIGHT` version waits for fetches and drop-ranges only (merges and mutations are ignored). `PULL` version pulls new entries from ZooKeeper and does not wait for them. Fixes [#47794](https://github.com/ClickHouse/ClickHouse/issues/47794). [#48085](https://github.com/ClickHouse/ClickHouse/pull/48085) ([Alexander Tokmakov](https://github.com/tavplubix)).
* Add `kafkaMurmurHash` function for compatibility with Kafka DefaultPartitioner. Closes [#47834](https://github.com/ClickHouse/ClickHouse/issues/47834). [#48185](https://github.com/ClickHouse/ClickHouse/pull/48185) ([Nikolay Degterinsky](https://github.com/evillique)).
* Allow to easily create a user with the same grants as the current user by using `GRANT CURRENT GRANTS`. [#48262](https://github.com/ClickHouse/ClickHouse/pull/48262) ([pufit](https://github.com/pufit)).
* Add statistical aggregate function `kolmogorovSmirnovTest`. Close [#48228](https://github.com/ClickHouse/ClickHouse/issues/48228). [#48325](https://github.com/ClickHouse/ClickHouse/pull/48325) ([FFFFFFFHHHHHHH](https://github.com/FFFFFFFHHHHHHH)).
* Added a `lost_part_count` column to the `system.replicas` table. The column value shows the total number of lost parts in the corresponding table. Value is stored in zookeeper and can be used instead of not persistent `ReplicatedDataLoss` profile event for monitoring. [#48526](https://github.com/ClickHouse/ClickHouse/pull/48526) ([Sergei Trifonov](https://github.com/serxa)).
* Add `soundex` function for compatibility. Closes [#39880](https://github.com/ClickHouse/ClickHouse/issues/39880). [#48567](https://github.com/ClickHouse/ClickHouse/pull/48567) ([FriendLey](https://github.com/FriendLey)).
* Support `Map` type for JSONExtract. [#48629](https://github.com/ClickHouse/ClickHouse/pull/48629) ([李扬](https://github.com/taiyang-li)).
* Add `PrettyJSONEachRow` format to output pretty JSON with new line delimieters and 4 space indents. [#48898](https://github.com/ClickHouse/ClickHouse/pull/48898) ([Kruglov Pavel](https://github.com/Avogar)).
* Add `ParquetMetadata` input format to read Parquet file metadata. [#48911](https://github.com/ClickHouse/ClickHouse/pull/48911) ([Kruglov Pavel](https://github.com/Avogar)).
* Add `extractKeyValuePairs` function to extract key value pairs from strings. Input strings might contain noise (i.e log files / do not need to be 100% formatted in key-value-pair format), the algorithm will look for key value pairs matching the arguments passed to the function. As of now, function accepts the following arguments: `data_column` (mandatory), `key_value_pair_delimiter` (defaults to `:`), `pair_delimiters` (defaults to `\space \, \;`) and `quoting_character` (defaults to double quotes). [#43606](https://github.com/ClickHouse/ClickHouse/pull/43606) ([Arthur Passos](https://github.com/arthurpassos)).
* Functions replaceOne(), replaceAll(), replaceRegexpOne() and replaceRegexpAll() can now be called with non-const pattern and replacement arguments. [#46589](https://github.com/ClickHouse/ClickHouse/pull/46589) ([Robert Schulze](https://github.com/rschu1ze)).
* Added functions to work with columns of type `Map`: `mapConcat`, `mapSort`, `mapExists`. [#48071](https://github.com/ClickHouse/ClickHouse/pull/48071) ([Anton Popov](https://github.com/CurtizJ)).
#### Performance Improvement
* Reading files in `Parquet` format is now much faster. IO and decoding are parallelized (controlled by `max_threads` setting), and only required data ranges are read. [#47964](https://github.com/ClickHouse/ClickHouse/pull/47964) ([Michael Kolupaev](https://github.com/al13n321)).
* If we run a mutation with IN (subquery) like this: `ALTER TABLE t UPDATE col='new value' WHERE id IN (SELECT id FROM huge_table)` and the table `t` has multiple parts than for each part a set for subquery `SELECT id FROM huge_table` is built in memory. And if there are many parts then this might consume a lot of memory (and lead to an OOM) and CPU. The solution is to introduce a short-lived cache of sets that are currently being built by mutation tasks. If another task of the same mutation is executed concurrently it can lookup the set in the cache, wait for it to be built and reuse it. [#46835](https://github.com/ClickHouse/ClickHouse/pull/46835) ([Alexander Gololobov](https://github.com/davenger)).
* Only check dependencies if necessary when applying `ALTER TABLE` queries. [#48062](https://github.com/ClickHouse/ClickHouse/pull/48062) ([Raúl Marín](https://github.com/Algunenano)).
* Optimize function `mapUpdate`. [#48118](https://github.com/ClickHouse/ClickHouse/pull/48118) ([Anton Popov](https://github.com/CurtizJ)).
* Now an internal query to local replica is sent explicitly and data from it received through loopback interface. Setting `prefer_localhost_replica` is not respected for parallel replicas. This is needed for better scheduling and makes the code cleaner: the initiator is only responsible for coordinating of the reading process and merging results, continiously answering for requests while all the secondary queries read the data. Note: Using loopback interface is not so performant, otherwise some replicas could starve for tasks which could lead to even slower query execution and not utilizing all possible resources. The initialization of the coordinator is now even more lazy. All incoming requests contain the information about the reading algorithm we initialize the coordinator with it when first request comes. If any replica will decide to read with different algorithm - an exception will be thrown and a query will be aborted. [#48246](https://github.com/ClickHouse/ClickHouse/pull/48246) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)).
* Do not build set for the right side of `IN` clause with subquery when it is used only for analysis of skip indexes and they are disabled by setting (`use_skip_indexes=0`). Previously it might affect the performance of queries. [#48299](https://github.com/ClickHouse/ClickHouse/pull/48299) ([Anton Popov](https://github.com/CurtizJ)).
* Query processing is parallelized right after reading `FROM file(...)`. Related to [#38755](https://github.com/ClickHouse/ClickHouse/issues/38755). [#48525](https://github.com/ClickHouse/ClickHouse/pull/48525) ([Igor Nikonov](https://github.com/devcrafter)).
* Query processing is parallelized right after reading from a data source. Affected data sources are mostly simple or external storages like table functions `url`, `file`. [#48727](https://github.com/ClickHouse/ClickHouse/pull/48727) ([Igor Nikonov](https://github.com/devcrafter)).
* Lowered contention of ThreadPool mutex (may increase performance for a huge amount of small jobs). [#48750](https://github.com/ClickHouse/ClickHouse/pull/48750) ([Sergei Trifonov](https://github.com/serxa)).
* Reduce memory usage for multiple `ALTER DELETE` mutations. [#48522](https://github.com/ClickHouse/ClickHouse/pull/48522) ([Nikolai Kochetov](https://github.com/KochetovNicolai)).
* Remove the excessive connection attempts if the `skip_unavailable_shards` setting is enabled. [#48771](https://github.com/ClickHouse/ClickHouse/pull/48771) ([Azat Khuzhin](https://github.com/azat)).
#### Experimental Feature
* Entries in the query cache are now squashed to max_block_size and compressed. [#45912](https://github.com/ClickHouse/ClickHouse/pull/45912) ([Robert Schulze](https://github.com/rschu1ze)).
* It is now possible to define per-user quotas in the query cache. [#48284](https://github.com/ClickHouse/ClickHouse/pull/48284) ([Robert Schulze](https://github.com/rschu1ze)).
* Some fixes for parallel replicas [#48433](https://github.com/ClickHouse/ClickHouse/pull/48433) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)).
* Implement zero-copy-replication (an experimental feature) on encrypted disks. [#48741](https://github.com/ClickHouse/ClickHouse/pull/48741) ([Vitaly Baranov](https://github.com/vitlibar)).
#### Improvement
* Increase default value for `connect_timeout_with_failover_ms` to 1000 ms (because of adding async connections in https://github.com/ClickHouse/ClickHouse/pull/47229) . Closes [#5188](https://github.com/ClickHouse/ClickHouse/issues/5188). [#49009](https://github.com/ClickHouse/ClickHouse/pull/49009) ([Kruglov Pavel](https://github.com/Avogar)).
* Several improvements around data lakes: - Make `Iceberg` work with non-partitioned data. - Support `Iceberg` format version v2 (previously only v1 was supported) - Support reading partitioned data for `DeltaLake`/`Hudi` - Faster reading of `DeltaLake` metadata by using Delta's checkpoint files - Fixed incorrect `Hudi` reads: previously it incorrectly chose which data to read and therefore was able to read correctly only small size tables - Made these engines to pickup updates of changed data (previously the state was set on table creation) - Make proper testing for `Iceberg`/`DeltaLake`/`Hudi` using spark. [#47307](https://github.com/ClickHouse/ClickHouse/pull/47307) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Add async connection to socket and async writing to socket. Make creating connections and sending query/external tables async across shards. Refactor code with fibers. Closes [#46931](https://github.com/ClickHouse/ClickHouse/issues/46931). We will be able to increase `connect_timeout_with_failover_ms` by default after this PR (https://github.com/ClickHouse/ClickHouse/issues/5188). [#47229](https://github.com/ClickHouse/ClickHouse/pull/47229) ([Kruglov Pavel](https://github.com/Avogar)).
* Support config sections `keeper`/`keeper_server` as an alternative to `zookeeper`. Close [#34766](https://github.com/ClickHouse/ClickHouse/issues/34766) , [#34767](https://github.com/ClickHouse/ClickHouse/issues/34767). [#35113](https://github.com/ClickHouse/ClickHouse/pull/35113) ([李扬](https://github.com/taiyang-li)).
* It is possible to set _secure_ flag in named_collections for a dictionary with a ClickHouse table source. Addresses [#38450](https://github.com/ClickHouse/ClickHouse/issues/38450) . [#46323](https://github.com/ClickHouse/ClickHouse/pull/46323) ([Ilya Golshtein](https://github.com/ilejn)).
* `bitCount` function support `FixedString` and `String` data type. [#49044](https://github.com/ClickHouse/ClickHouse/pull/49044) ([flynn](https://github.com/ucasfl)).
* Added configurable retries for all operations with [Zoo]Keeper for Backup queries. [#47224](https://github.com/ClickHouse/ClickHouse/pull/47224) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)).
* Enable `use_environment_credentials` for S3 by default, so the entire provider chain is constructed by default. [#47397](https://github.com/ClickHouse/ClickHouse/pull/47397) ([Antonio Andelic](https://github.com/antonio2368)).
* Currently, the JSON_VALUE function is similar as spark's get_json_object function, which support to get value from json string by a path like '$.key'. But still has something different - 1. in spark's get_json_object will return null while the path is not exist, but in JSON_VALUE will return empty string; - 2. in spark's get_json_object will return a complext type value, such as a json object/array value, but in JSON_VALUE will return empty string. [#47494](https://github.com/ClickHouse/ClickHouse/pull/47494) ([KevinyhZou](https://github.com/KevinyhZou)).
* For `use_structure_from_insertion_table_in_table_functions` more flexible insert table structure propagation to table function. Fixed an issue with name mapping and using virtual columns. No more need for 'auto' setting. [#47962](https://github.com/ClickHouse/ClickHouse/pull/47962) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
* Do not continue retrying to connect to ZK if the query is killed or over limits. [#47985](https://github.com/ClickHouse/ClickHouse/pull/47985) ([Raúl Marín](https://github.com/Algunenano)).
* Support Enum output/input in `BSONEachRow`, allow all map key types and avoid extra calculations on output. [#48122](https://github.com/ClickHouse/ClickHouse/pull/48122) ([Kruglov Pavel](https://github.com/Avogar)).
* Support more ClickHouse types in `ORC`/`Arrow`/`Parquet` formats: Enum(8|16), (U)Int(128|256), Decimal256 (for ORC), allow reading IPv4 from Int32 values (ORC outputs IPv4 as Int32 and we couldn't read it back), fix reading Nullable(IPv6) from binary data for `ORC`. [#48126](https://github.com/ClickHouse/ClickHouse/pull/48126) ([Kruglov Pavel](https://github.com/Avogar)).
* Add columns `perform_ttl_move_on_insert`, `load_balancing` for table `system.storage_policies`, modify column `volume_type` type to `Enum8`. [#48167](https://github.com/ClickHouse/ClickHouse/pull/48167) ([lizhuoyu5](https://github.com/lzydmxy)).
* Added support for `BACKUP ALL` command which backups all tables and databases, including temporary and system ones. [#48189](https://github.com/ClickHouse/ClickHouse/pull/48189) ([Vitaly Baranov](https://github.com/vitlibar)).
* Function mapFromArrays supports `Map` type as an input. [#48207](https://github.com/ClickHouse/ClickHouse/pull/48207) ([李扬](https://github.com/taiyang-li)).
* The output of some SHOW PROCESSLIST is now sorted. [#48241](https://github.com/ClickHouse/ClickHouse/pull/48241) ([Robert Schulze](https://github.com/rschu1ze)).
* Per-query/per-server throttling for remote IO/local IO/BACKUPs (server settings: `max_remote_read_network_bandwidth_for_server`, `max_remote_write_network_bandwidth_for_server`, `max_local_read_bandwidth_for_server`, `max_local_write_bandwidth_for_server`, `max_backup_bandwidth_for_server`, settings: `max_remote_read_network_bandwidth`, `max_remote_write_network_bandwidth`, `max_local_read_bandwidth`, `max_local_write_bandwidth`, `max_backup_bandwidth`). [#48242](https://github.com/ClickHouse/ClickHouse/pull/48242) ([Azat Khuzhin](https://github.com/azat)).
* Support more types in `CapnProto` format: Map, (U)Int(128|256), Decimal(128|256). Allow integer conversions during input/output. [#48257](https://github.com/ClickHouse/ClickHouse/pull/48257) ([Kruglov Pavel](https://github.com/Avogar)).
* Don't throw CURRENT_WRITE_BUFFER_IS_EXHAUSTED for normal behaviour. [#48288](https://github.com/ClickHouse/ClickHouse/pull/48288) ([Raúl Marín](https://github.com/Algunenano)).
* Add new setting `keeper_map_strict_mode` which enforces extra guarantees on operations made on top of `KeeperMap` tables. [#48293](https://github.com/ClickHouse/ClickHouse/pull/48293) ([Antonio Andelic](https://github.com/antonio2368)).
* Check primary key type for simple dictionary is native unsigned integer type Add setting `check_dictionary_primary_key ` for compatibility(set `check_dictionary_primary_key =false` to disable checking). [#48335](https://github.com/ClickHouse/ClickHouse/pull/48335) ([lizhuoyu5](https://github.com/lzydmxy)).
* Don't replicate mutations for `KeeperMap` because it's unnecessary. [#48354](https://github.com/ClickHouse/ClickHouse/pull/48354) ([Antonio Andelic](https://github.com/antonio2368)).
* Allow write/read unnamed tuple as nested Message in Protobuf format. Tuple elements and Message fields are mathced by position. [#48390](https://github.com/ClickHouse/ClickHouse/pull/48390) ([Kruglov Pavel](https://github.com/Avogar)).
* Support `additional_table_filters` and `additional_result_filter` settings in the new planner. Also, add a documentation entry for `additional_result_filter`. [#48405](https://github.com/ClickHouse/ClickHouse/pull/48405) ([Dmitry Novik](https://github.com/novikd)).
* `parseDateTime` now understands format string '%f' (fractional seconds). [#48420](https://github.com/ClickHouse/ClickHouse/pull/48420) ([Robert Schulze](https://github.com/rschu1ze)).
* Format string "%f" in formatDateTime() now prints "000000" if the formatted value has no fractional seconds, the previous behavior (single zero) can be restored using setting "formatdatetime_f_prints_single_zero = 1". [#48422](https://github.com/ClickHouse/ClickHouse/pull/48422) ([Robert Schulze](https://github.com/rschu1ze)).
* Don't replicate DELETE and TRUNCATE for KeeperMap. [#48434](https://github.com/ClickHouse/ClickHouse/pull/48434) ([Antonio Andelic](https://github.com/antonio2368)).
* Generate valid Decimals and Bools in generateRandom function. [#48436](https://github.com/ClickHouse/ClickHouse/pull/48436) ([Kruglov Pavel](https://github.com/Avogar)).
* Allow trailing commas in expression list of SELECT query, for example `SELECT a, b, c, FROM table`. Closes [#37802](https://github.com/ClickHouse/ClickHouse/issues/37802). [#48438](https://github.com/ClickHouse/ClickHouse/pull/48438) ([Nikolay Degterinsky](https://github.com/evillique)).
* Override `CLICKHOUSE_USER` and `CLICKHOUSE_PASSWORD` environment variables with `--user` and `--password` client parameters. Closes [#38909](https://github.com/ClickHouse/ClickHouse/issues/38909). [#48440](https://github.com/ClickHouse/ClickHouse/pull/48440) ([Nikolay Degterinsky](https://github.com/evillique)).
* Added retries to loading of data parts in `MergeTree` tables in case of retryable errors. [#48442](https://github.com/ClickHouse/ClickHouse/pull/48442) ([Anton Popov](https://github.com/CurtizJ)).
* Add support for `Date`, `Date32`, `DateTime`, `DateTime64` data types to `arrayMin`, `arrayMax`, `arrayDifference` functions. Closes [#21645](https://github.com/ClickHouse/ClickHouse/issues/21645). [#48445](https://github.com/ClickHouse/ClickHouse/pull/48445) ([Nikolay Degterinsky](https://github.com/evillique)).
* Add support for `{server_uuid}` macro. It is useful for identifying replicas in autoscaled clusters when new replicas are constantly added and removed in runtime. This closes [#48554](https://github.com/ClickHouse/ClickHouse/issues/48554). [#48563](https://github.com/ClickHouse/ClickHouse/pull/48563) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The installation script will create a hard link instead of copying if it is possible. [#48578](https://github.com/ClickHouse/ClickHouse/pull/48578) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Support `SHOW TABLE` syntax meaning the same as `SHOW CREATE TABLE`. Closes [#48580](https://github.com/ClickHouse/ClickHouse/issues/48580). [#48591](https://github.com/ClickHouse/ClickHouse/pull/48591) ([flynn](https://github.com/ucasfl)).
* HTTP temporary buffers now support working by evicting data from the virtual filesystem cache. [#48664](https://github.com/ClickHouse/ClickHouse/pull/48664) ([Vladimir C](https://github.com/vdimir)).
* Make Schema inference works for `CREATE AS SELECT`. Closes [#47599](https://github.com/ClickHouse/ClickHouse/issues/47599). [#48679](https://github.com/ClickHouse/ClickHouse/pull/48679) ([flynn](https://github.com/ucasfl)).
* Added a `replicated_max_mutations_in_one_entry` setting for `ReplicatedMergeTree` that allows limiting the number of mutation commands per one `MUTATE_PART` entry (default is 10000). [#48731](https://github.com/ClickHouse/ClickHouse/pull/48731) ([Alexander Tokmakov](https://github.com/tavplubix)).
* In AggregateFunction types, don't count unused arena bytes as `read_bytes`. [#48745](https://github.com/ClickHouse/ClickHouse/pull/48745) ([Raúl Marín](https://github.com/Algunenano)).
* Fix some MySQL-related settings not being handled with the MySQL dictionary source + named collection. Closes [#48402](https://github.com/ClickHouse/ClickHouse/issues/48402). [#48759](https://github.com/ClickHouse/ClickHouse/pull/48759) ([Kseniia Sumarokova](https://github.com/kssenii)).
* If a user set `max_single_part_upload_size` to a very large value, it can lead to a crash due to a bug in the AWS S3 SDK. This fixes [#47679](https://github.com/ClickHouse/ClickHouse/issues/47679). [#48816](https://github.com/ClickHouse/ClickHouse/pull/48816) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix data race in `RabbitMQ` ([report](https://pastila.nl/?004f7100/de1505289ab5bb355e67ebe6c7cc8707)), refactor the code. [#48845](https://github.com/ClickHouse/ClickHouse/pull/48845) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Add aliases `name` and `part_name` form `system.parts` and `system.part_log`. Closes [#48718](https://github.com/ClickHouse/ClickHouse/issues/48718). [#48850](https://github.com/ClickHouse/ClickHouse/pull/48850) ([sichenzhao](https://github.com/sichenzhao)).
* Functions "arrayDifferenceSupport()", "arrayCumSum()" and "arrayCumSumNonNegative()" now support input arrays of wide integer types (U)Int128/256. [#48866](https://github.com/ClickHouse/ClickHouse/pull/48866) ([cluster](https://github.com/infdahai)).
* Multi-line history in clickhouse-client is now no longer padded. This makes pasting more natural. [#48870](https://github.com/ClickHouse/ClickHouse/pull/48870) ([Joanna Hulboj](https://github.com/jh0x)).
* Implement a slight improvement for the rare case when ClickHouse is run inside LXC and LXCFS is used. The LXCFS has an issue: sometimes it returns an error "Transport endpoint is not connected" on reading from the file inside `/proc`. This error was correctly logged into ClickHouse's server log. We have additionally workaround this issue by reopening a file. This is a minuscule change. [#48922](https://github.com/ClickHouse/ClickHouse/pull/48922) ([Real](https://github.com/RunningXie)).
* Improve memory accounting for prefetches. Randomise prefetch settings In CI. [#48973](https://github.com/ClickHouse/ClickHouse/pull/48973) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Correctly set headers for native copy operations on GCS. [#48981](https://github.com/ClickHouse/ClickHouse/pull/48981) ([Antonio Andelic](https://github.com/antonio2368)).
* Add support for specifying setting names in the command line with dashes instead of underscores, for example, `--max-threads` instead of `--max_threads`. Additionally, support Unicode dash characters like `—` instead of `--` - this is useful when you communicate with a team in another company, and a manager from that team copy-pasted code from MS Word. [#48985](https://github.com/ClickHouse/ClickHouse/pull/48985) ([alekseygolub](https://github.com/alekseygolub)).
* Add fallback to password authentication when authentication with SSL user certificate has failed. Closes [#48974](https://github.com/ClickHouse/ClickHouse/issues/48974). [#48989](https://github.com/ClickHouse/ClickHouse/pull/48989) ([Nikolay Degterinsky](https://github.com/evillique)).
* Improve the embedded dashboard. Close [#46671](https://github.com/ClickHouse/ClickHouse/issues/46671). [#49036](https://github.com/ClickHouse/ClickHouse/pull/49036) ([Kevin Zhang](https://github.com/Kinzeng)).
* Add profile events for log messages, so you can easily see the count of log messages by severity. [#49042](https://github.com/ClickHouse/ClickHouse/pull/49042) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* In previous versions, the `LineAsString` format worked inconsistently when the parallel parsing was enabled or not, in presence of DOS or MacOS Classic line breaks. This closes [#49039](https://github.com/ClickHouse/ClickHouse/issues/49039). [#49052](https://github.com/ClickHouse/ClickHouse/pull/49052) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* The exception message about the unparsed query parameter will also tell about the name of the parameter. Reimplement [#48878](https://github.com/ClickHouse/ClickHouse/issues/48878). Close [#48772](https://github.com/ClickHouse/ClickHouse/issues/48772). [#49061](https://github.com/ClickHouse/ClickHouse/pull/49061) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
#### Build/Testing/Packaging Improvement
* Update time zones. The following were updated: Africa/Cairo, Africa/Casablanca, Africa/El_Aaiun, America/Bogota, America/Cambridge_Bay, America/Ciudad_Juarez, America/Godthab, America/Inuvik, America/Iqaluit, America/Nuuk, America/Ojinaga, America/Pangnirtung, America/Rankin_Inlet, America/Resolute, America/Whitehorse, America/Yellowknife, Asia/Gaza, Asia/Hebron, Asia/Kuala_Lumpur, Asia/Singapore, Canada/Yukon, Egypt, Europe/Kirov, Europe/Volgograd, Singapore. [#48572](https://github.com/ClickHouse/ClickHouse/pull/48572) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Reduce the number of dependencies in the header files to speed up the build. [#47984](https://github.com/ClickHouse/ClickHouse/pull/47984) ([Dmitry Novik](https://github.com/novikd)).
* Randomize compression of marks and indices in tests. [#48286](https://github.com/ClickHouse/ClickHouse/pull/48286) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Bump internal ZSTD from 1.5.4 to 1.5.5. [#46797](https://github.com/ClickHouse/ClickHouse/pull/46797) ([Robert Schulze](https://github.com/rschu1ze)).
* Randomize vertical merges from compact to wide parts in tests. [#48287](https://github.com/ClickHouse/ClickHouse/pull/48287) ([Raúl Marín](https://github.com/Algunenano)).
* Support for CRC32 checksum in HDFS. Fix performance issues. [#48614](https://github.com/ClickHouse/ClickHouse/pull/48614) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Remove remainders of GCC support. [#48671](https://github.com/ClickHouse/ClickHouse/pull/48671) ([Robert Schulze](https://github.com/rschu1ze)).
* Add CI run with new analyzer infrastructure enabled. [#48719](https://github.com/ClickHouse/ClickHouse/pull/48719) ([Dmitry Novik](https://github.com/novikd)).
#### Bug Fix (user-visible misbehavior in an official stable release)
* Fix system.query_views_log for MVs that are pushed from background threads [#46668](https://github.com/ClickHouse/ClickHouse/pull/46668) ([Azat Khuzhin](https://github.com/azat)).
* Fix several `RENAME COLUMN` bugs [#46946](https://github.com/ClickHouse/ClickHouse/pull/46946) ([alesapin](https://github.com/alesapin)).
* Fix minor hiliting issues in clickhouse-format [#47610](https://github.com/ClickHouse/ClickHouse/pull/47610) ([Natasha Murashkina](https://github.com/murfel)).
* Fix a bug in LLVM's libc++ leading to a crash for uploading parts to S3 which size is greater then INT_MAX [#47693](https://github.com/ClickHouse/ClickHouse/pull/47693) ([Azat Khuzhin](https://github.com/azat)).
* Fix overflow in the `sparkbar` function [#48121](https://github.com/ClickHouse/ClickHouse/pull/48121) ([Vladimir C](https://github.com/vdimir)).
* Fix race in S3 [#48190](https://github.com/ClickHouse/ClickHouse/pull/48190) ([Anton Popov](https://github.com/CurtizJ)).
* Disable JIT for aggregate functions due to inconsistent behavior [#48195](https://github.com/ClickHouse/ClickHouse/pull/48195) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix alter formatting (minor) [#48289](https://github.com/ClickHouse/ClickHouse/pull/48289) ([Natasha Murashkina](https://github.com/murfel)).
* Fix cpu usage in RabbitMQ (was worsened in 23.2 after [#44404](https://github.com/ClickHouse/ClickHouse/issues/44404)) [#48311](https://github.com/ClickHouse/ClickHouse/pull/48311) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Fix crash in EXPLAIN PIPELINE for Merge over Distributed [#48320](https://github.com/ClickHouse/ClickHouse/pull/48320) ([Azat Khuzhin](https://github.com/azat)).
* Fix serializing LowCardinality as Arrow dictionary [#48361](https://github.com/ClickHouse/ClickHouse/pull/48361) ([Kruglov Pavel](https://github.com/Avogar)).
* Reset downloader for cache file segment in TemporaryFileStream [#48386](https://github.com/ClickHouse/ClickHouse/pull/48386) ([Vladimir C](https://github.com/vdimir)).
* Fix possible SYSTEM SYNC REPLICA stuck in case of DROP/REPLACE PARTITION [#48391](https://github.com/ClickHouse/ClickHouse/pull/48391) ([Azat Khuzhin](https://github.com/azat)).
* Fix a startup error when loading a distributed table that depends on a dictionary [#48419](https://github.com/ClickHouse/ClickHouse/pull/48419) ([MikhailBurdukov](https://github.com/MikhailBurdukov)).
* Don't check dependencies when renaming system tables automatically [#48431](https://github.com/ClickHouse/ClickHouse/pull/48431) ([Raúl Marín](https://github.com/Algunenano)).
* Update only affected rows in KeeperMap storage [#48435](https://github.com/ClickHouse/ClickHouse/pull/48435) ([Antonio Andelic](https://github.com/antonio2368)).
* Fix possible segfault in the VFS cache [#48469](https://github.com/ClickHouse/ClickHouse/pull/48469) ([Kseniia Sumarokova](https://github.com/kssenii)).
* `toTimeZone` function throws an error when no constant string is provided [#48471](https://github.com/ClickHouse/ClickHouse/pull/48471) ([Jordi Villar](https://github.com/jrdi)).
* Fix logical error with IPv4 in Protobuf, add support for Date32 [#48486](https://github.com/ClickHouse/ClickHouse/pull/48486) ([Kruglov Pavel](https://github.com/Avogar)).
* "changed" flag in system.settings was calculated incorrectly for settings with multiple values [#48516](https://github.com/ClickHouse/ClickHouse/pull/48516) ([MikhailBurdukov](https://github.com/MikhailBurdukov)).
* Fix storage `Memory` with enabled compression [#48517](https://github.com/ClickHouse/ClickHouse/pull/48517) ([Anton Popov](https://github.com/CurtizJ)).
* Fix bracketed-paste mode messing up password input in the event of client reconnection [#48528](https://github.com/ClickHouse/ClickHouse/pull/48528) ([Michael Kolupaev](https://github.com/al13n321)).
* Fix nested map for keys of IP and UUID types [#48556](https://github.com/ClickHouse/ClickHouse/pull/48556) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
* Fix an uncaught exception in case of parallel loader for hashed dictionaries [#48571](https://github.com/ClickHouse/ClickHouse/pull/48571) ([Azat Khuzhin](https://github.com/azat)).
* The `groupArray` aggregate function correctly works for empty result over nullable types [#48593](https://github.com/ClickHouse/ClickHouse/pull/48593) ([lgbo](https://github.com/lgbo-ustc)).
* Fix bug in Keeper when a node is not created with scheme `auth` in ACL sometimes. [#48595](https://github.com/ClickHouse/ClickHouse/pull/48595) ([Aleksei Filatov](https://github.com/aalexfvk)).
* Allow IPv4 comparison operators with UInt [#48611](https://github.com/ClickHouse/ClickHouse/pull/48611) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
* Fix possible error from cache [#48636](https://github.com/ClickHouse/ClickHouse/pull/48636) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Async inserts with empty data will no longer throw exception. [#48663](https://github.com/ClickHouse/ClickHouse/pull/48663) ([Anton Popov](https://github.com/CurtizJ)).
* Fix table dependencies in case of failed RENAME TABLE [#48683](https://github.com/ClickHouse/ClickHouse/pull/48683) ([Azat Khuzhin](https://github.com/azat)).
* If the primary key has duplicate columns (which is only possible for projections), in previous versions it might lead to a bug [#48838](https://github.com/ClickHouse/ClickHouse/pull/48838) ([Amos Bird](https://github.com/amosbird)).
* Fix for a race condition in ZooKeeper when joining send_thread/receive_thread [#48849](https://github.com/ClickHouse/ClickHouse/pull/48849) ([Alexander Gololobov](https://github.com/davenger)).
* Fix unexpected part name error when trying to drop a ignored detached part with zero copy replication [#48862](https://github.com/ClickHouse/ClickHouse/pull/48862) ([Michael Lex](https://github.com/mlex)).
* Fix reading `Date32` Parquet/Arrow column into not a `Date32` column [#48864](https://github.com/ClickHouse/ClickHouse/pull/48864) ([Kruglov Pavel](https://github.com/Avogar)).
* Fix `UNKNOWN_IDENTIFIER` error while selecting from table with row policy and column with dots [#48976](https://github.com/ClickHouse/ClickHouse/pull/48976) ([Kruglov Pavel](https://github.com/Avogar)).
* Fix aggregation by empty nullable strings [#48999](https://github.com/ClickHouse/ClickHouse/pull/48999) ([LiuNeng](https://github.com/liuneng1994)).
### <a id="233"></a> ClickHouse release 23.3 LTS, 2023-03-30
#### Upgrade Notes

7
CMakeLists.txt
View File

@ -421,8 +421,11 @@ endif ()
set (CMAKE_POSTFIX_VARIABLE "CMAKE_${CMAKE_BUILD_TYPE_UC}_POSTFIX")
set (CMAKE_POSITION_INDEPENDENT_CODE OFF)
if (OS_LINUX AND NOT (ARCH_AARCH64 OR ARCH_S390X))
if (NOT SANITIZE)
set (CMAKE_POSITION_INDEPENDENT_CODE OFF)
endif()
if (OS_LINUX AND NOT (ARCH_AARCH64 OR ARCH_S390X) AND NOT SANITIZE)
# Slightly more efficient code can be generated
# It's disabled for ARM because otherwise ClickHouse cannot run on Android.
set (CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO} -fno-pie")

1
README.md
View File

@ -23,6 +23,7 @@ curl https://clickhouse.com/ | sh
## Upcoming Events
* [**ClickHouse Spring Meetup in Manhattan**](https://www.meetup.com/clickhouse-new-york-user-group/events/292517734) - April 26 - It's spring, and it's time to meet again in the city! Talks include: "Building a domain specific query language on top of Clickhouse", "A Galaxy of Information", "Our Journey to ClickHouse Cloud from Redshift", and a ClickHouse update!
* [**v23.4 Release Webinar**](https://clickhouse.com/company/events/v23-4-release-webinar?utm_source=github&utm_medium=social&utm_campaign=release-webinar-2023-04) - April 26 - 23.4 is rapidly approaching. Original creator, co-founder, and CTO of ClickHouse Alexey Milovidov will walk us through the highlights of the release.
* [**ClickHouse Meetup in Berlin**](https://www.meetup.com/clickhouse-berlin-user-group/events/292892466) - May 16 - Save the date! ClickHouse is coming back to Berlin. We’re excited to announce an upcoming ClickHouse Meetup that you won’t want to miss. Join us as we gather together to discuss the latest in the world of ClickHouse and share user stories.
## Recent Recordings
* **Recent Meetup Videos**: [Meetup Playlist](https://www.youtube.com/playlist?list=PL0Z2YDlm0b3iNDUzpY1S3L_iV4nARda_U) Whenever possible recordings of the ClickHouse Community Meetups are edited and presented as individual talks. Current featuring "Modern SQL in 2023", "Fast, Concurrent, and Consistent Asynchronous INSERTS in ClickHouse", and "Full-Text Indices: Design and Experiments"

26
base/base/argsToConfig.cpp
View File

@ -3,13 +3,29 @@
#include <Poco/Util/LayeredConfiguration.h>
#include <Poco/Util/MapConfiguration.h>
void argsToConfig(const Poco::Util::Application::ArgVec & argv, Poco::Util::LayeredConfiguration & config, int priority)
void argsToConfig(const Poco::Util::Application::ArgVec & argv,
Poco::Util::LayeredConfiguration & config,
int priority,
const std::unordered_set<std::string>* alias_names)
{
/// Parsing all args and converting to config layer
/// Test: -- --1=1 --1=2 --3 5 7 8 -9 10 -11=12 14= 15== --16==17 --=18 --19= --20 21 22 --23 --24 25 --26 -27 28 ---29=30 -- ----31 32 --33 3-4
Poco::AutoPtr<Poco::Util::MapConfiguration> map_config = new Poco::Util::MapConfiguration;
std::string key;
auto add_arg = [&map_config, &alias_names](const std::string & k, const std::string & v)
{
map_config->setString(k, v);
if (alias_names && !alias_names->contains(k))
{
std::string alias_key = k;
std::replace(alias_key.begin(), alias_key.end(), '-', '_');
if (alias_names->contains(alias_key))
map_config->setString(alias_key, v);
}
};
for (const auto & arg : argv)
{
auto key_start = arg.find_first_not_of('-');
@ -19,7 +35,7 @@ void argsToConfig(const Poco::Util::Application::ArgVec & argv, Poco::Util::Laye
// old saved '--key', will set to some true value "1"
if (!key.empty() && pos_minus != std::string::npos && pos_minus < key_start)
{
map_config->setString(key, "1");
add_arg(key, "1");
key = "";
}
@ -29,7 +45,7 @@ void argsToConfig(const Poco::Util::Application::ArgVec & argv, Poco::Util::Laye
{
if (pos_minus == std::string::npos || pos_minus > key_start)
{
map_config->setString(key, arg);
add_arg(key, arg);
}
key = "";
}
@ -55,7 +71,7 @@ void argsToConfig(const Poco::Util::Application::ArgVec & argv, Poco::Util::Laye
if (arg.size() > pos_eq)
value = arg.substr(pos_eq + 1);
map_config->setString(key, value);
add_arg(key, value);
key = "";
}

7
base/base/argsToConfig.h
View File

@ -1,6 +1,8 @@
#pragma once
#include <Poco/Util/Application.h>
#include <string>
#include <unordered_set>
namespace Poco::Util
{
@ -8,4 +10,7 @@ class LayeredConfiguration; // NOLINT(cppcoreguidelines-virtual-class-destructor
}
/// Import extra command line arguments to configuration. These are command line arguments after --.
void argsToConfig(const Poco::Util::Application::ArgVec & argv, Poco::Util::LayeredConfiguration & config, int priority);
void argsToConfig(const Poco::Util::Application::ArgVec & argv,
Poco::Util::LayeredConfiguration & config,
int priority,
const std::unordered_set<std::string>* registered_alias_names = nullptr);

170
base/base/find_symbols.h
View File

@ -34,10 +34,52 @@
* If no such characters, returns nullptr.
*/
struct SearchSymbols
{
static constexpr auto BUFFER_SIZE = 16;
SearchSymbols() = default;
explicit SearchSymbols(std::string in)
: str(std::move(in))
{
#if defined(__SSE4_2__)
if (str.size() > BUFFER_SIZE)
{
throw std::runtime_error("SearchSymbols can contain at most " + std::to_string(BUFFER_SIZE) + " symbols and " + std::to_string(str.size()) + " was provided\n");
}
char tmp_safety_buffer[BUFFER_SIZE] = {0};
memcpy(tmp_safety_buffer, str.data(), str.size());
simd_vector = _mm_loadu_si128(reinterpret_cast<const __m128i *>(tmp_safety_buffer));
#endif
}
#if defined(__SSE4_2__)
__m128i simd_vector;
#endif
std::string str;
};
namespace detail
{
template <char ...chars> constexpr bool is_in(char x) { return ((x == chars) || ...); } // NOLINT(misc-redundant-expression)
static bool is_in(char c, const char * symbols, size_t num_chars)
{
for (size_t i = 0u; i < num_chars; ++i)
{
if (c == symbols[i])
{
return true;
}
}
return false;
}
#if defined(__SSE2__)
template <char s0>
inline __m128i mm_is_in(__m128i bytes)
@ -53,6 +95,43 @@ inline __m128i mm_is_in(__m128i bytes)
__m128i eq = mm_is_in<s1, tail...>(bytes);
return _mm_or_si128(eq0, eq);
}
inline __m128i mm_is_in(__m128i bytes, const char * symbols, size_t num_chars)
{
__m128i accumulator = _mm_setzero_si128();
for (size_t i = 0; i < num_chars; ++i)
{
__m128i eq = _mm_cmpeq_epi8(bytes, _mm_set1_epi8(symbols[i]));
accumulator = _mm_or_si128(accumulator, eq);
}
return accumulator;
}
inline std::array<__m128i, 16u> mm_is_in_prepare(const char * symbols, size_t num_chars)
{
std::array<__m128i, 16u> result {};
for (size_t i = 0; i < num_chars; ++i)
{
result[i] = _mm_set1_epi8(symbols[i]);
}
return result;
}
inline __m128i mm_is_in_execute(__m128i bytes, const std::array<__m128i, 16u> & needles)
{
__m128i accumulator = _mm_setzero_si128();
for (const auto & needle : needles)
{
__m128i eq = _mm_cmpeq_epi8(bytes, needle);
accumulator = _mm_or_si128(accumulator, eq);
}
return accumulator;
}
#endif
template <bool positive>
@ -99,6 +178,32 @@ inline const char * find_first_symbols_sse2(const char * const begin, const char
return return_mode == ReturnMode::End ? end : nullptr;
}
template <bool positive, ReturnMode return_mode>
inline const char * find_first_symbols_sse2(const char * const begin, const char * const end, const char * symbols, size_t num_chars)
{
const char * pos = begin;
#if defined(__SSE2__)
const auto needles = mm_is_in_prepare(symbols, num_chars);
for (; pos + 15 < end; pos += 16)
{
__m128i bytes = _mm_loadu_si128(reinterpret_cast<const __m128i *>(pos));
__m128i eq = mm_is_in_execute(bytes, needles);
uint16_t bit_mask = maybe_negate<positive>(uint16_t(_mm_movemask_epi8(eq)));
if (bit_mask)
return pos + __builtin_ctz(bit_mask);
}
#endif
for (; pos < end; ++pos)
if (maybe_negate<positive>(is_in(*pos, symbols, num_chars)))
return pos;
return return_mode == ReturnMode::End ? end : nullptr;
}
template <bool positive, ReturnMode return_mode, char... symbols>
inline const char * find_last_symbols_sse2(const char * const begin, const char * const end)
@ -179,6 +284,41 @@ inline const char * find_first_symbols_sse42(const char * const begin, const cha
return return_mode == ReturnMode::End ? end : nullptr;
}
template <bool positive, ReturnMode return_mode>
inline const char * find_first_symbols_sse42(const char * const begin, const char * const end, const SearchSymbols & symbols)
{
const char * pos = begin;
const auto num_chars = symbols.str.size();
#if defined(__SSE4_2__)
constexpr int mode = _SIDD_UBYTE_OPS | _SIDD_CMP_EQUAL_ANY | _SIDD_LEAST_SIGNIFICANT;
const __m128i set = symbols.simd_vector;
for (; pos + 15 < end; pos += 16)
{
__m128i bytes = _mm_loadu_si128(reinterpret_cast<const __m128i *>(pos));
if constexpr (positive)
{
if (_mm_cmpestrc(set, num_chars, bytes, 16, mode))
return pos + _mm_cmpestri(set, num_chars, bytes, 16, mode);
}
else
{
if (_mm_cmpestrc(set, num_chars, bytes, 16, mode | _SIDD_NEGATIVE_POLARITY))
return pos + _mm_cmpestri(set, num_chars, bytes, 16, mode | _SIDD_NEGATIVE_POLARITY);
}
}
#endif
for (; pos < end; ++pos)
if (maybe_negate<positive>(is_in(*pos, symbols.str.data(), num_chars)))
return pos;
return return_mode == ReturnMode::End ? end : nullptr;
}
/// NOTE No SSE 4.2 implementation for find_last_symbols_or_null. Not worth to do.
@ -194,6 +334,17 @@ inline const char * find_first_symbols_dispatch(const char * begin, const char *
return find_first_symbols_sse2<positive, return_mode, symbols...>(begin, end);
}
template <bool positive, ReturnMode return_mode>
inline const char * find_first_symbols_dispatch(const std::string_view haystack, const SearchSymbols & symbols)
{
#if defined(__SSE4_2__)
if (symbols.str.size() >= 5)
return find_first_symbols_sse42<positive, return_mode>(haystack.begin(), haystack.end(), symbols);
else
#endif
return find_first_symbols_sse2<positive, return_mode>(haystack.begin(), haystack.end(), symbols.str.data(), symbols.str.size());
}
}
@ -211,6 +362,11 @@ inline char * find_first_symbols(char * begin, char * end)
return const_cast<char *>(detail::find_first_symbols_dispatch<true, detail::ReturnMode::End, symbols...>(begin, end));
}
inline const char * find_first_symbols(std::string_view haystack, const SearchSymbols & symbols)
{
return detail::find_first_symbols_dispatch<true, detail::ReturnMode::End>(haystack, symbols);
}
template <char... symbols>
inline const char * find_first_not_symbols(const char * begin, const char * end)
{
@ -223,6 +379,11 @@ inline char * find_first_not_symbols(char * begin, char * end)
return const_cast<char *>(detail::find_first_symbols_dispatch<false, detail::ReturnMode::End, symbols...>(begin, end));
}
inline const char * find_first_not_symbols(std::string_view haystack, const SearchSymbols & symbols)
{
return detail::find_first_symbols_dispatch<false, detail::ReturnMode::End>(haystack, symbols);
}
template <char... symbols>
inline const char * find_first_symbols_or_null(const char * begin, const char * end)
{
@ -235,6 +396,11 @@ inline char * find_first_symbols_or_null(char * begin, char * end)
return const_cast<char *>(detail::find_first_symbols_dispatch<true, detail::ReturnMode::Nullptr, symbols...>(begin, end));
}
inline const char * find_first_symbols_or_null(std::string_view haystack, const SearchSymbols & symbols)
{
return detail::find_first_symbols_dispatch<true, detail::ReturnMode::Nullptr>(haystack, symbols);
}
template <char... symbols>
inline const char * find_first_not_symbols_or_null(const char * begin, const char * end)
{
@ -247,6 +413,10 @@ inline char * find_first_not_symbols_or_null(char * begin, char * end)
return const_cast<char *>(detail::find_first_symbols_dispatch<false, detail::ReturnMode::Nullptr, symbols...>(begin, end));
}
inline const char * find_first_not_symbols_or_null(std::string_view haystack, const SearchSymbols & symbols)
{
return detail::find_first_symbols_dispatch<false, detail::ReturnMode::Nullptr>(haystack, symbols);
}
template <char... symbols>
inline const char * find_last_symbols_or_null(const char * begin, const char * end)

105
base/base/unaligned.h
View File

@ -5,44 +5,6 @@
#include <bit>
inline void reverseMemcpy(void * dst, const void * src, size_t size)
{
uint8_t * uint_dst = reinterpret_cast<uint8_t *>(dst);
const uint8_t * uint_src = reinterpret_cast<const uint8_t *>(src);
uint_dst += size;
while (size)
{
--uint_dst;
*uint_dst = *uint_src;
++uint_src;
--size;
}
}
template <typename T>
inline T unalignedLoadLE(const void * address)
{
T res {};
if constexpr (std::endian::native == std::endian::little)
memcpy(&res, address, sizeof(res));
else
reverseMemcpy(&res, address, sizeof(res));
return res;
}
template <typename T>
inline void unalignedStoreLE(void * address,
const typename std::enable_if<true, T>::type & src)
{
static_assert(std::is_trivially_copyable_v<T>);
if constexpr (std::endian::native == std::endian::little)
memcpy(address, &src, sizeof(src));
else
reverseMemcpy(address, &src, sizeof(src));
}
template <typename T>
inline T unalignedLoad(const void * address)
{
@ -62,3 +24,70 @@ inline void unalignedStore(void * address,
static_assert(std::is_trivially_copyable_v<T>);
memcpy(address, &src, sizeof(src));
}
inline void reverseMemcpy(void * dst, const void * src, size_t size)
{
uint8_t * uint_dst = reinterpret_cast<uint8_t *>(dst);
const uint8_t * uint_src = reinterpret_cast<const uint8_t *>(src);
uint_dst += size;
while (size)
{
--uint_dst;
*uint_dst = *uint_src;
++uint_src;
--size;
}
}
template <std::endian endian, typename T>
inline T unalignedLoadEndian(const void * address)
{
T res {};
if constexpr (std::endian::native == endian)
memcpy(&res, address, sizeof(res));
else
reverseMemcpy(&res, address, sizeof(res));
return res;
}
template <std::endian endian, typename T>
inline void unalignedStoreEndian(void * address, T & src)
{
static_assert(std::is_trivially_copyable_v<T>);
if constexpr (std::endian::native == endian)
memcpy(address, &src, sizeof(src));
else
reverseMemcpy(address, &src, sizeof(src));
}
template <typename T>
inline T unalignedLoadLittleEndian(const void * address)
{
return unalignedLoadEndian<std::endian::little, T>(address);
}
template <typename T>
inline void unalignedStoreLittleEndian(void * address,
const typename std::enable_if<true, T>::type & src)
{
unalignedStoreEndian<std::endian::little>(address, src);
}
template <typename T>
inline T unalignedLoadBigEndian(const void * address)
{
return unalignedLoadEndian<std::endian::big, T>(address);
}
template <typename T>
inline void unalignedStoreBigEndian(void * address,
const typename std::enable_if<true, T>::type & src)
{
unalignedStoreEndian<std::endian::big>(address, src);
}

11
base/glibc-compatibility/glibc-compatibility.c
View File

@ -235,6 +235,17 @@ ssize_t getrandom(void *buf, size_t buflen, unsigned flags)
return syscall(SYS_getrandom, buf, buflen, flags);
}
/* Structure for scatter/gather I/O. */
struct iovec
{
void *iov_base; /* Pointer to data. */
size_t iov_len; /* Length of data. */
};
ssize_t preadv(int __fd, const struct iovec *__iovec, int __count, __off_t __offset)
{
return syscall(SYS_preadv, __fd, __iovec, __count, (long)(__offset), (long)(__offset>>32));
}
#include <errno.h>
#include <limits.h>

30
base/poco/Crypto/include/Poco/Crypto/ECKeyImpl.h
View File

@ -90,20 +90,6 @@ namespace Crypto
std::string groupName() const;
/// Returns the EC key group name.
void save(const std::string & publicKeyFile, const std::string & privateKeyFile = "", const std::string & privateKeyPassphrase = "")
const;
/// Exports the public and private keys to the given files.
///
/// If an empty filename is specified, the corresponding key
/// is not exported.
void
save(std::ostream * pPublicKeyStream, std::ostream * pPrivateKeyStream = 0, const std::string & privateKeyPassphrase = "") const;
/// Exports the public and private key to the given streams.
///
/// If a null pointer is passed for a stream, the corresponding
/// key is not exported.
static std::string getCurveName(int nid = -1);
/// Returns elliptical curve name corresponding to
/// the given nid; if nid is not found, returns
@ -150,22 +136,6 @@ namespace Crypto
{
return OBJ_nid2sn(groupId());
}
inline void
ECKeyImpl::save(const std::string & publicKeyFile, const std::string & privateKeyFile, const std::string & privateKeyPassphrase) const
{
EVPPKey(_pEC).save(publicKeyFile, privateKeyFile, privateKeyPassphrase);
}
inline void
ECKeyImpl::save(std::ostream * pPublicKeyStream, std::ostream * pPrivateKeyStream, const std::string & privateKeyPassphrase) const
{
EVPPKey(_pEC).save(pPublicKeyStream, pPrivateKeyStream, privateKeyPassphrase);
}
}
} // namespace Poco::Crypto

33
base/poco/Crypto/include/Poco/Crypto/KeyPair.h
View File

@ -56,24 +56,6 @@ namespace Crypto
virtual int size() const;
/// Returns the RSA modulus size.
virtual void save(
const std::string & publicKeyPairFile,
const std::string & privateKeyPairFile = "",
const std::string & privateKeyPairPassphrase = "") const;
/// Exports the public and private keys to the given files.
///
/// If an empty filename is specified, the corresponding key
/// is not exported.
virtual void save(
std::ostream * pPublicKeyPairStream,
std::ostream * pPrivateKeyPairStream = 0,
const std::string & privateKeyPairPassphrase = "") const;
/// Exports the public and private key to the given streams.
///
/// If a null pointer is passed for a stream, the corresponding
/// key is not exported.
KeyPairImpl::Ptr impl() const;
/// Returns the impl object.
@ -97,21 +79,6 @@ namespace Crypto
return _pImpl->size();
}
inline void
KeyPair::save(const std::string & publicKeyFile, const std::string & privateKeyFile, const std::string & privateKeyPassphrase) const
{
_pImpl->save(publicKeyFile, privateKeyFile, privateKeyPassphrase);
}
inline void
KeyPair::save(std::ostream * pPublicKeyStream, std::ostream * pPrivateKeyStream, const std::string & privateKeyPassphrase) const
{
_pImpl->save(pPublicKeyStream, pPrivateKeyStream, privateKeyPassphrase);
}
inline const std::string & KeyPair::name() const
{
return _pImpl->name();

16
base/poco/Crypto/include/Poco/Crypto/KeyPairImpl.h
View File

@ -55,22 +55,6 @@ namespace Crypto
virtual int size() const = 0;
/// Returns the key size.
virtual void save(
const std::string & publicKeyFile,
const std::string & privateKeyFile = "",
const std::string & privateKeyPassphrase = "") const = 0;
/// Exports the public and private keys to the given files.
///
/// If an empty filename is specified, the corresponding key
/// is not exported.
virtual void save(
std::ostream * pPublicKeyStream, std::ostream * pPrivateKeyStream = 0, const std::string & privateKeyPassphrase = "") const = 0;
/// Exports the public and private key to the given streams.
///
/// If a null pointer is passed for a stream, the corresponding
/// key is not exported.
const std::string & name() const;
/// Returns key pair name

16
base/poco/Crypto/include/Poco/Crypto/RSAKeyImpl.h
View File

@ -96,20 +96,6 @@ namespace Crypto
ByteVec decryptionExponent() const;
/// Returns the RSA decryption exponent.
void save(const std::string & publicKeyFile, const std::string & privateKeyFile = "", const std::string & privateKeyPassphrase = "")
const;
/// Exports the public and private keys to the given files.
///
/// If an empty filename is specified, the corresponding key
/// is not exported.
void
save(std::ostream * pPublicKeyStream, std::ostream * pPrivateKeyStream = 0, const std::string & privateKeyPassphrase = "") const;
/// Exports the public and private key to the given streams.
///
/// If a null pointer is passed for a stream, the corresponding
/// key is not exported.
private:
RSAKeyImpl();
@ -139,4 +125,4 @@ namespace Crypto
} // namespace Poco::Crypto
#endif // Crypto_RSAKeyImplImpl_INCLUDED
#endif // Crypto_RSAKeyImplImpl_INCLUDED

99
base/poco/Crypto/src/RSAKeyImpl.cpp
View File

@ -269,103 +269,6 @@ RSAKeyImpl::ByteVec RSAKeyImpl::decryptionExponent() const
}
void RSAKeyImpl::save(const std::string& publicKeyFile,
const std::string& privateKeyFile,
const std::string& privateKeyPassphrase) const
{
if (!publicKeyFile.empty())
{
BIO* bio = BIO_new(BIO_s_file());
if (!bio) throw Poco::IOException("Cannot create BIO for writing public key file", publicKeyFile);
try
{
if (BIO_write_filename(bio, const_cast<char*>(publicKeyFile.c_str())))
{
if (!PEM_write_bio_RSAPublicKey(bio, _pRSA))
throw Poco::WriteFileException("Failed to write public key to file", publicKeyFile);
}
else throw Poco::CreateFileException("Cannot create public key file");
}
catch (...)
{
BIO_free(bio);
throw;
}
BIO_free(bio);
}
if (!privateKeyFile.empty())
{
BIO* bio = BIO_new(BIO_s_file());
if (!bio) throw Poco::IOException("Cannot create BIO for writing private key file", privateKeyFile);
try
{
if (BIO_write_filename(bio, const_cast<char*>(privateKeyFile.c_str())))
{
int rc = 0;
if (privateKeyPassphrase.empty())
rc = PEM_write_bio_RSAPrivateKey(bio, _pRSA, 0, 0, 0, 0, 0);
else
rc = PEM_write_bio_RSAPrivateKey(bio, _pRSA, EVP_des_ede3_cbc(),
reinterpret_cast<unsigned char*>(const_cast<char*>(privateKeyPassphrase.c_str())),
static_cast<int>(privateKeyPassphrase.length()), 0, 0);
if (!rc) throw Poco::FileException("Failed to write private key to file", privateKeyFile);
}
else throw Poco::CreateFileException("Cannot create private key file", privateKeyFile);
}
catch (...)
{
BIO_free(bio);
throw;
}
BIO_free(bio);
}
}
void RSAKeyImpl::save(std::ostream* pPublicKeyStream,
std::ostream* pPrivateKeyStream,
const std::string& privateKeyPassphrase) const
{
if (pPublicKeyStream)
{
BIO* bio = BIO_new(BIO_s_mem());
if (!bio) throw Poco::IOException("Cannot create BIO for writing public key");
if (!PEM_write_bio_RSAPublicKey(bio, _pRSA))
{
BIO_free(bio);
throw Poco::WriteFileException("Failed to write public key to stream");
}
char* pData;
long size = BIO_get_mem_data(bio, &pData);
pPublicKeyStream->write(pData, static_cast<std::streamsize>(size));
BIO_free(bio);
}
if (pPrivateKeyStream)
{
BIO* bio = BIO_new(BIO_s_mem());
if (!bio) throw Poco::IOException("Cannot create BIO for writing public key");
int rc = 0;
if (privateKeyPassphrase.empty())
rc = PEM_write_bio_RSAPrivateKey(bio, _pRSA, 0, 0, 0, 0, 0);
else
rc = PEM_write_bio_RSAPrivateKey(bio, _pRSA, EVP_des_ede3_cbc(),
reinterpret_cast<unsigned char*>(const_cast<char*>(privateKeyPassphrase.c_str())),
static_cast<int>(privateKeyPassphrase.length()), 0, 0);
if (!rc)
{
BIO_free(bio);
throw Poco::FileException("Failed to write private key to stream");
}
char* pData;
long size = BIO_get_mem_data(bio, &pData);
pPrivateKeyStream->write(pData, static_cast<std::streamsize>(size));
BIO_free(bio);
}
}
RSAKeyImpl::ByteVec RSAKeyImpl::convertToByteVec(const BIGNUM* bn)
{
int numBytes = BN_num_bytes(bn);
@ -383,4 +286,4 @@ RSAKeyImpl::ByteVec RSAKeyImpl::convertToByteVec(const BIGNUM* bn)
}
} } // namespace Poco::Crypto
} } // namespace Poco::Crypto

62
base/poco/Data/ODBC/include/Poco/Data/ODBC/Unicode_WIN32.h
View File

@ -1,62 +0,0 @@
//
// Unicode.h
//
// Library: Data/ODBC
// Package: ODBC
// Module: Unicode
//
// Definition of Unicode_WIN32.
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Data_ODBC_Unicode_WIN32_INCLUDED
#define Data_ODBC_Unicode_WIN32_INCLUDED
namespace Poco
{
namespace Data
{
namespace ODBC
{
inline void makeUTF16(SQLCHAR * pSQLChar, SQLINTEGER length, std::wstring & target)
/// Utility function for conversion from UTF-8 to UTF-16
{
int len = length;
if (SQL_NTS == len)
len = (int)std::strlen((const char *)pSQLChar);
UnicodeConverter::toUTF16((const char *)pSQLChar, len, target);
}
inline void makeUTF8(Poco::Buffer<wchar_t> & buffer, SQLINTEGER length, SQLPOINTER pTarget, SQLINTEGER targetLength)
/// Utility function for conversion from UTF-16 to UTF-8. Length is in bytes.
{
if (buffer.sizeBytes() < length)
throw InvalidArgumentException("Specified length exceeds available length.");
else if ((length % 2) != 0)
throw InvalidArgumentException("Length must be an even number.");
length /= sizeof(wchar_t);
std::string result;
UnicodeConverter::toUTF8(buffer.begin(), length, result);
std::memset(pTarget, 0, targetLength);
std::strncpy((char *)pTarget, result.c_str(), result.size() < targetLength ? result.size() : targetLength);
}
}
}
} // namespace Poco::Data::ODBC
#endif // Data_ODBC_Unicode_WIN32_INCLUDED

761
base/poco/Data/ODBC/src/Unicode_WIN32.cpp
View File

@ -1,761 +0,0 @@
//
// Unicode.cpp
//
// Library: Data/ODBC
// Package: ODBC
// Module: Unicode
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Data/ODBC/ODBC.h"
#include "Poco/Data/ODBC/Utility.h"
#include "Poco/Data/ODBC/Unicode_WIN32.h"
#include "Poco/Buffer.h"
#include "Poco/Exception.h"
using Poco::Buffer;
using Poco::InvalidArgumentException;
using Poco::NotImplementedException;
namespace Poco {
namespace Data {
namespace ODBC {
SQLRETURN SQLColAttribute(SQLHSTMT hstmt,
SQLUSMALLINT iCol,
SQLUSMALLINT iField,
SQLPOINTER pCharAttr,
SQLSMALLINT cbCharAttrMax,
SQLSMALLINT* pcbCharAttr,
NumAttrPtrType pNumAttr)
{
if (isString(pCharAttr, cbCharAttrMax))
{
Buffer<wchar_t> buffer(stringLength(pCharAttr, cbCharAttrMax));
SQLRETURN rc = SQLColAttributeW(hstmt,
iCol,
iField,
buffer.begin(),
(SQLSMALLINT) buffer.sizeBytes(),
pcbCharAttr,
pNumAttr);
makeUTF8(buffer, *pcbCharAttr, pCharAttr, cbCharAttrMax);
return rc;
}
return SQLColAttributeW(hstmt,
iCol,
iField,
pCharAttr,
cbCharAttrMax,
pcbCharAttr,
pNumAttr);
}
SQLRETURN SQLColAttributes(SQLHSTMT hstmt,
SQLUSMALLINT icol,
SQLUSMALLINT fDescType,
SQLPOINTER rgbDesc,
SQLSMALLINT cbDescMax,
SQLSMALLINT* pcbDesc,
SQLLEN* pfDesc)
{
return SQLColAttribute(hstmt,
icol,
fDescType,
rgbDesc,
cbDescMax,
pcbDesc,
pfDesc);
}
SQLRETURN SQLConnect(SQLHDBC hdbc,
SQLCHAR* szDSN,
SQLSMALLINT cbDSN,
SQLCHAR* szUID,
SQLSMALLINT cbUID,
SQLCHAR* szAuthStr,
SQLSMALLINT cbAuthStr)
{
std::wstring sqlDSN;
makeUTF16(szDSN, cbDSN, sqlDSN);
std::wstring sqlUID;
makeUTF16(szUID, cbUID, sqlUID);
std::wstring sqlPWD;
makeUTF16(szAuthStr, cbAuthStr, sqlPWD);
return SQLConnectW(hdbc,
(SQLWCHAR*) sqlDSN.c_str(),
(SQLSMALLINT) sqlDSN.size(),
(SQLWCHAR*) sqlUID.c_str(),
(SQLSMALLINT) sqlUID.size(),
(SQLWCHAR*) sqlPWD.c_str(),
(SQLSMALLINT) sqlPWD.size());
}
SQLRETURN SQLDescribeCol(SQLHSTMT hstmt,
SQLUSMALLINT icol,
SQLCHAR* szColName,
SQLSMALLINT cbColNameMax,
SQLSMALLINT* pcbColName,
SQLSMALLINT* pfSqlType,
SQLULEN* pcbColDef,
SQLSMALLINT* pibScale,
SQLSMALLINT* pfNullable)
{
Buffer<wchar_t> buffer(cbColNameMax);
SQLRETURN rc = SQLDescribeColW(hstmt,
icol,
(SQLWCHAR*) buffer.begin(),
(SQLSMALLINT) buffer.size(),
pcbColName,
pfSqlType,
pcbColDef,
pibScale,
pfNullable);
makeUTF8(buffer, *pcbColName * sizeof(wchar_t), szColName, cbColNameMax);
return rc;
}
SQLRETURN SQLError(SQLHENV henv,
SQLHDBC hdbc,
SQLHSTMT hstmt,
SQLCHAR* szSqlState,
SQLINTEGER* pfNativeError,
SQLCHAR* szErrorMsg,
SQLSMALLINT cbErrorMsgMax,
SQLSMALLINT* pcbErrorMsg)
{
throw NotImplementedException("SQLError is obsolete. "
"Use SQLGetDiagRec instead.");
}
SQLRETURN SQLExecDirect(SQLHSTMT hstmt,
SQLCHAR* szSqlStr,
SQLINTEGER cbSqlStr)
{
std::wstring sqlStr;
makeUTF16(szSqlStr, cbSqlStr, sqlStr);
return SQLExecDirectW(hstmt,
(SQLWCHAR*) sqlStr.c_str(),
(SQLINTEGER) sqlStr.size());
}
SQLRETURN SQLGetConnectAttr(SQLHDBC hdbc,
SQLINTEGER fAttribute,
SQLPOINTER rgbValue,
SQLINTEGER cbValueMax,
SQLINTEGER* pcbValue)
{
if (isString(rgbValue, cbValueMax))
{
Buffer<wchar_t> buffer(stringLength(rgbValue, cbValueMax));
SQLRETURN rc = SQLGetConnectAttrW(hdbc,
fAttribute,
buffer.begin(),
(SQLINTEGER) buffer.sizeBytes(),
pcbValue);
makeUTF8(buffer, *pcbValue, rgbValue, cbValueMax);
return rc;
}
return SQLGetConnectAttrW(hdbc,
fAttribute,
rgbValue,
cbValueMax,
pcbValue);
}
SQLRETURN SQLGetCursorName(SQLHSTMT hstmt,
SQLCHAR* szCursor,
SQLSMALLINT cbCursorMax,
SQLSMALLINT* pcbCursor)
{
throw NotImplementedException("Not implemented");
}
SQLRETURN SQLSetDescField(SQLHDESC hdesc,
SQLSMALLINT iRecord,
SQLSMALLINT iField,
SQLPOINTER rgbValue,
SQLINTEGER cbValueMax)
{
if (isString(rgbValue, cbValueMax))
{
std::wstring str;
makeUTF16((SQLCHAR*) rgbValue, cbValueMax, str);
SQLRETURN rc = SQLSetDescFieldW(hdesc,
iRecord,
iField,
(SQLPOINTER) str.c_str(),
(SQLINTEGER) str.size() * sizeof(std::wstring::value_type));
return rc;
}
return SQLSetDescFieldW(hdesc,
iRecord,
iField,
rgbValue,
cbValueMax);
}
SQLRETURN SQLGetDescField(SQLHDESC hdesc,
SQLSMALLINT iRecord,
SQLSMALLINT iField,
SQLPOINTER rgbValue,
SQLINTEGER cbValueMax,
SQLINTEGER* pcbValue)
{
if (isString(rgbValue, cbValueMax))
{
Buffer<wchar_t> buffer(stringLength(rgbValue, cbValueMax));
SQLRETURN rc = SQLGetDescFieldW(hdesc,
iRecord,
iField,
buffer.begin(),
(SQLINTEGER) buffer.sizeBytes(),
pcbValue);
makeUTF8(buffer, *pcbValue, rgbValue, cbValueMax);
return rc;
}
return SQLGetDescFieldW(hdesc,
iRecord,
iField,
rgbValue,
cbValueMax,
pcbValue);
}
SQLRETURN SQLGetDescRec(SQLHDESC hdesc,
SQLSMALLINT iRecord,
SQLCHAR* szName,
SQLSMALLINT cbNameMax,
SQLSMALLINT* pcbName,
SQLSMALLINT* pfType,
SQLSMALLINT* pfSubType,
SQLLEN* pLength,
SQLSMALLINT* pPrecision,
SQLSMALLINT* pScale,
SQLSMALLINT* pNullable)
{
throw NotImplementedException();
}
SQLRETURN SQLGetDiagField(SQLSMALLINT fHandleType,
SQLHANDLE handle,
SQLSMALLINT iRecord,
SQLSMALLINT fDiagField,
SQLPOINTER rgbDiagInfo,
SQLSMALLINT cbDiagInfoMax,
SQLSMALLINT* pcbDiagInfo)
{
if (isString(rgbDiagInfo, cbDiagInfoMax))
{
Buffer<wchar_t> buffer(stringLength(rgbDiagInfo, cbDiagInfoMax));
SQLRETURN rc = SQLGetDiagFieldW(fHandleType,
handle,
iRecord,
fDiagField,
buffer.begin(),
(SQLSMALLINT) buffer.sizeBytes(),
pcbDiagInfo);
makeUTF8(buffer, *pcbDiagInfo, rgbDiagInfo, cbDiagInfoMax);
return rc;
}
return SQLGetDiagFieldW(fHandleType,
handle,
iRecord,
fDiagField,
rgbDiagInfo,
cbDiagInfoMax,
pcbDiagInfo);
}
SQLRETURN SQLGetDiagRec(SQLSMALLINT fHandleType,
SQLHANDLE handle,
SQLSMALLINT iRecord,
SQLCHAR* szSqlState,
SQLINTEGER* pfNativeError,
SQLCHAR* szErrorMsg,
SQLSMALLINT cbErrorMsgMax,
SQLSMALLINT* pcbErrorMsg)
{
const SQLINTEGER stateLen = SQL_SQLSTATE_SIZE + 1;
Buffer<wchar_t> bufState(stateLen);
Buffer<wchar_t> bufErr(cbErrorMsgMax);
SQLRETURN rc = SQLGetDiagRecW(fHandleType,
handle,
iRecord,
bufState.begin(),
pfNativeError,
bufErr.begin(),
(SQLSMALLINT) bufErr.size(),
pcbErrorMsg);
makeUTF8(bufState, stateLen * sizeof(wchar_t), szSqlState, stateLen);
makeUTF8(bufErr, *pcbErrorMsg * sizeof(wchar_t), szErrorMsg, cbErrorMsgMax);
return rc;
}
SQLRETURN SQLPrepare(SQLHSTMT hstmt,
SQLCHAR* szSqlStr,
SQLINTEGER cbSqlStr)
{
std::wstring sqlStr;
makeUTF16(szSqlStr, cbSqlStr, sqlStr);
return SQLPrepareW(hstmt,
(SQLWCHAR*) sqlStr.c_str(),
(SQLINTEGER) sqlStr.size());
}
SQLRETURN SQLSetConnectAttr(SQLHDBC hdbc,
SQLINTEGER fAttribute,
SQLPOINTER rgbValue,
SQLINTEGER cbValue)
{
if (isString(rgbValue, cbValue))
{
std::wstring str;
makeUTF16((SQLCHAR*) rgbValue, cbValue, str);
return SQLSetConnectAttrW(hdbc,
fAttribute,
(SQLWCHAR*) str.c_str(),
(SQLINTEGER) str.size() * sizeof(std::wstring::value_type));
}
return SQLSetConnectAttrW(hdbc,
fAttribute,
rgbValue,
cbValue);
}
SQLRETURN SQLSetCursorName(SQLHSTMT hstmt,
SQLCHAR* szCursor,
SQLSMALLINT cbCursor)
{
throw NotImplementedException("Not implemented");
}
SQLRETURN SQLSetStmtAttr(SQLHSTMT hstmt,
SQLINTEGER fAttribute,
SQLPOINTER rgbValue,
SQLINTEGER cbValueMax)
{
if (isString(rgbValue, cbValueMax))
{
std::wstring str;
makeUTF16((SQLCHAR*) rgbValue, cbValueMax, str);
return SQLSetStmtAttrW(hstmt,
fAttribute,
(SQLPOINTER) str.c_str(),
(SQLINTEGER) str.size());
}
return SQLSetStmtAttrW(hstmt,
fAttribute,
rgbValue,
cbValueMax);
}
SQLRETURN SQLGetStmtAttr(SQLHSTMT hstmt,
SQLINTEGER fAttribute,
SQLPOINTER rgbValue,
SQLINTEGER cbValueMax,
SQLINTEGER* pcbValue)
{
if (isString(rgbValue, cbValueMax))
{
Buffer<wchar_t> buffer(stringLength(rgbValue, cbValueMax));
return SQLGetStmtAttrW(hstmt,
fAttribute,
(SQLPOINTER) buffer.begin(),
(SQLINTEGER) buffer.sizeBytes(),
pcbValue);
}
return SQLGetStmtAttrW(hstmt,
fAttribute,
rgbValue,
cbValueMax,
pcbValue);
}
SQLRETURN SQLColumns(SQLHSTMT hstmt,
SQLCHAR* szCatalogName,
SQLSMALLINT cbCatalogName,
SQLCHAR* szSchemaName,
SQLSMALLINT cbSchemaName,
SQLCHAR* szTableName,
SQLSMALLINT cbTableName,
SQLCHAR* szColumnName,
SQLSMALLINT cbColumnName)
{
throw NotImplementedException();
}
SQLRETURN SQLGetConnectOption(SQLHDBC hdbc,
SQLUSMALLINT fOption,
SQLPOINTER pvParam)
{
throw NotImplementedException();
}
SQLRETURN SQLGetInfo(SQLHDBC hdbc,
SQLUSMALLINT fInfoType,
SQLPOINTER rgbInfoValue,
SQLSMALLINT cbInfoValueMax,
SQLSMALLINT* pcbInfoValue)
{
if (cbInfoValueMax)
{
Buffer<wchar_t> buffer(cbInfoValueMax);
SQLRETURN rc = SQLGetInfoW(hdbc,
fInfoType,
(SQLPOINTER) buffer.begin(),
(SQLSMALLINT) buffer.sizeBytes(),
pcbInfoValue);
makeUTF8(buffer, *pcbInfoValue, rgbInfoValue, cbInfoValueMax);
return rc;
}
return SQLGetInfoW(hdbc,
fInfoType,
rgbInfoValue,
cbInfoValueMax,
pcbInfoValue);
}
SQLRETURN SQLGetTypeInfo(SQLHSTMT StatementHandle, SQLSMALLINT DataType)
{
return SQLGetTypeInfoW(StatementHandle, DataType);
}
SQLRETURN SQLSetConnectOption(SQLHDBC hdbc,
SQLUSMALLINT fOption,
SQLULEN vParam)
{
throw NotImplementedException();
}
SQLRETURN SQLSpecialColumns(SQLHSTMT hstmt,
SQLUSMALLINT fColType,
SQLCHAR* szCatalogName,
SQLSMALLINT cbCatalogName,
SQLCHAR* szSchemaName,
SQLSMALLINT cbSchemaName,
SQLCHAR* szTableName,
SQLSMALLINT cbTableName,
SQLUSMALLINT fScope,
SQLUSMALLINT fNullable)
{
throw NotImplementedException();
}
SQLRETURN SQLStatistics(SQLHSTMT hstmt,
SQLCHAR* szCatalogName,
SQLSMALLINT cbCatalogName,
SQLCHAR* szSchemaName,
SQLSMALLINT cbSchemaName,
SQLCHAR* szTableName,
SQLSMALLINT cbTableName,
SQLUSMALLINT fUnique,
SQLUSMALLINT fAccuracy)
{
throw NotImplementedException();
}
SQLRETURN SQLTables(SQLHSTMT hstmt,
SQLCHAR* szCatalogName,
SQLSMALLINT cbCatalogName,
SQLCHAR* szSchemaName,
SQLSMALLINT cbSchemaName,
SQLCHAR* szTableName,
SQLSMALLINT cbTableName,
SQLCHAR* szTableType,
SQLSMALLINT cbTableType)
{
throw NotImplementedException();
}
SQLRETURN SQLDataSources(SQLHENV henv,
SQLUSMALLINT fDirection,
SQLCHAR* szDSN,
SQLSMALLINT cbDSNMax,
SQLSMALLINT* pcbDSN,
SQLCHAR* szDesc,
SQLSMALLINT cbDescMax,
SQLSMALLINT* pcbDesc)
{
Buffer<wchar_t> bufDSN(cbDSNMax);
Buffer<wchar_t> bufDesc(cbDescMax);
SQLRETURN rc = SQLDataSourcesW(henv,
fDirection,
bufDSN.begin(),
(SQLSMALLINT) bufDSN.size(),
pcbDSN,
bufDesc.begin(),
(SQLSMALLINT) bufDesc.size(),
pcbDesc);
makeUTF8(bufDSN, *pcbDSN * sizeof(wchar_t), szDSN, cbDSNMax);
makeUTF8(bufDesc, *pcbDesc * sizeof(wchar_t), szDesc, cbDescMax);
return rc;
}
SQLRETURN SQLDriverConnect(SQLHDBC hdbc,
SQLHWND hwnd,
SQLCHAR* szConnStrIn,
SQLSMALLINT cbConnStrIn,
SQLCHAR* szConnStrOut,
SQLSMALLINT cbConnStrOutMax,
SQLSMALLINT* pcbConnStrOut,
SQLUSMALLINT fDriverCompletion)
{
std::wstring connStrIn;
int len = cbConnStrIn;
if (SQL_NTS == len)
len = (int) std::strlen((const char*) szConnStrIn);
Poco::UnicodeConverter::toUTF16((const char *) szConnStrIn, len, connStrIn);
Buffer<wchar_t> bufOut(cbConnStrOutMax);
SQLRETURN rc = SQLDriverConnectW(hdbc,
hwnd,
(SQLWCHAR*) connStrIn.c_str(),
(SQLSMALLINT) connStrIn.size(),
bufOut.begin(),
(SQLSMALLINT) bufOut.size(),
pcbConnStrOut,
fDriverCompletion);
if (!Utility::isError(rc))
makeUTF8(bufOut, *pcbConnStrOut * sizeof(wchar_t), szConnStrOut, cbConnStrOutMax);
return rc;
}
SQLRETURN SQLBrowseConnect(SQLHDBC hdbc,
SQLCHAR* szConnStrIn,
SQLSMALLINT cbConnStrIn,
SQLCHAR* szConnStrOut,
SQLSMALLINT cbConnStrOutMax,
SQLSMALLINT* pcbConnStrOut)
{
std::wstring str;
makeUTF16(szConnStrIn, cbConnStrIn, str);
Buffer<wchar_t> bufConnStrOut(cbConnStrOutMax);
SQLRETURN rc = SQLBrowseConnectW(hdbc,
(SQLWCHAR*) str.c_str(),
(SQLSMALLINT) str.size(),
bufConnStrOut.begin(),
(SQLSMALLINT) bufConnStrOut.size(),
pcbConnStrOut);
makeUTF8(bufConnStrOut, *pcbConnStrOut * sizeof(wchar_t), szConnStrOut, cbConnStrOutMax);
return rc;
}
SQLRETURN SQLColumnPrivileges(SQLHSTMT hstmt,
SQLCHAR* szCatalogName,
SQLSMALLINT cbCatalogName,
SQLCHAR* szSchemaName,
SQLSMALLINT cbSchemaName,
SQLCHAR* szTableName,
SQLSMALLINT cbTableName,
SQLCHAR* szColumnName,
SQLSMALLINT cbColumnName)
{
throw NotImplementedException();
}
SQLRETURN SQLForeignKeys(SQLHSTMT hstmt,
SQLCHAR* szPkCatalogName,
SQLSMALLINT cbPkCatalogName,
SQLCHAR* szPkSchemaName,
SQLSMALLINT cbPkSchemaName,
SQLCHAR* szPkTableName,
SQLSMALLINT cbPkTableName,
SQLCHAR* szFkCatalogName,
SQLSMALLINT cbFkCatalogName,
SQLCHAR* szFkSchemaName,
SQLSMALLINT cbFkSchemaName,
SQLCHAR* szFkTableName,
SQLSMALLINT cbFkTableName)
{
throw NotImplementedException();
}
SQLRETURN SQLNativeSql(SQLHDBC hdbc,
SQLCHAR* szSqlStrIn,
SQLINTEGER cbSqlStrIn,
SQLCHAR* szSqlStr,
SQLINTEGER cbSqlStrMax,
SQLINTEGER* pcbSqlStr)
{
std::wstring str;
makeUTF16(szSqlStrIn, cbSqlStrIn, str);
Buffer<wchar_t> bufSQLOut(cbSqlStrMax);
SQLRETURN rc = SQLNativeSqlW(hdbc,
(SQLWCHAR*) str.c_str(),
(SQLINTEGER) str.size(),
bufSQLOut.begin(),
(SQLINTEGER) bufSQLOut.size(),
pcbSqlStr);
makeUTF8(bufSQLOut, *pcbSqlStr * sizeof(wchar_t), szSqlStr, cbSqlStrMax);
return rc;
}
SQLRETURN SQLPrimaryKeys(SQLHSTMT hstmt,
SQLCHAR* szCatalogName,
SQLSMALLINT cbCatalogName,
SQLCHAR* szSchemaName,
SQLSMALLINT cbSchemaName,
SQLCHAR* szTableName,
SQLSMALLINT cbTableName)
{
throw NotImplementedException();
}
SQLRETURN SQLProcedureColumns(SQLHSTMT hstmt,
SQLCHAR* szCatalogName,
SQLSMALLINT cbCatalogName,
SQLCHAR* szSchemaName,
SQLSMALLINT cbSchemaName,
SQLCHAR* szProcName,
SQLSMALLINT cbProcName,
SQLCHAR* szColumnName,
SQLSMALLINT cbColumnName)
{
throw NotImplementedException();
}
SQLRETURN SQLProcedures(SQLHSTMT hstmt,
SQLCHAR* szCatalogName,
SQLSMALLINT cbCatalogName,
SQLCHAR* szSchemaName,
SQLSMALLINT cbSchemaName,
SQLCHAR* szProcName,
SQLSMALLINT cbProcName)
{
throw NotImplementedException();
}
SQLRETURN SQLTablePrivileges(SQLHSTMT hstmt,
SQLCHAR* szCatalogName,
SQLSMALLINT cbCatalogName,
SQLCHAR* szSchemaName,
SQLSMALLINT cbSchemaName,
SQLCHAR* szTableName,
SQLSMALLINT cbTableName)
{
throw NotImplementedException();
}
SQLRETURN SQLDrivers(SQLHENV henv,
SQLUSMALLINT fDirection,
SQLCHAR* szDriverDesc,
SQLSMALLINT cbDriverDescMax,
SQLSMALLINT* pcbDriverDesc,
SQLCHAR* szDriverAttributes,
SQLSMALLINT cbDrvrAttrMax,
SQLSMALLINT* pcbDrvrAttr)
{
Buffer<wchar_t> bufDriverDesc(cbDriverDescMax);
Buffer<wchar_t> bufDriverAttr(cbDrvrAttrMax);
SQLRETURN rc = SQLDriversW(henv,
fDirection,
bufDriverDesc.begin(),
(SQLSMALLINT) bufDriverDesc.size(),
pcbDriverDesc,
bufDriverAttr.begin(),
(SQLSMALLINT) bufDriverAttr.size(),
pcbDrvrAttr);
makeUTF8(bufDriverDesc, *pcbDriverDesc * sizeof(wchar_t), szDriverDesc, cbDriverDescMax);
makeUTF8(bufDriverAttr, *pcbDrvrAttr * sizeof(wchar_t), szDriverAttributes, cbDrvrAttrMax);
return rc;
}
} } } // namespace Poco::Data::ODBC

37
base/poco/Data/include/Poco/Data/AutoTransaction.h
View File

@ -1,37 +0,0 @@
//
// AutoTransaction.h
//
// Library: Data
// Package: DataCore
// Module: AutoTransaction
//
// Forward header for the Transaction class.
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Data_AutoTransaction_INCLUDED
#define Data_AutoTransaction_INCLUDED
#include "Poco/Data/Transaction.h"
namespace Poco
{
namespace Data
{
typedef Transaction AutoTransaction;
}
} // namespace Poco::Data
#endif // Data_AutoTransaction_INCLUDED

54
base/poco/Data/include/Poco/Data/DynamicLOB.h
View File

@ -1,54 +0,0 @@
//
// DynamicLOB.h
//
// Library: Data
// Package: DataCore
// Module: DynamicLOB
//
// Definition of the Poco::Dynamic::Var LOB cast operators.
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Data_DynamicLOB_INCLUDED
#define Data_DynamicLOB_INCLUDED
#include "Poco/Data/Data.h"
#include "Poco/Data/LOB.h"
#include "Poco/Dynamic/Var.h"
namespace Poco
{
namespace Data
{
template <typename T>
class LOB;
typedef LOB<unsigned char> BLOB;
typedef LOB<char> CLOB;
}
} // namespace Poco::Data
namespace Poco
{
namespace Dynamic
{
template <>
Data_API Var::operator Poco::Data::CLOB() const;
template <>
Data_API Var::operator Poco::Data::BLOB() const;
}
} // namespace Poco::Dynamic
#endif // Data_DynamicLOB_INCLUDED

149
base/poco/Data/include/Poco/Data/LOBStream.h
View File

@ -1,149 +0,0 @@
//
// LOBStream.h
//
// Library: Data
// Package: DataCore
// Module: LOBStream
//
// Definition of the LOBStream class.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Data_LOBStream_INCLUDED
#define Data_LOBStream_INCLUDED
#include <istream>
#include <ostream>
#include "Poco/Data/LOB.h"
#include "Poco/Foundation.h"
#include "Poco/UnbufferedStreamBuf.h"
namespace Poco
{
namespace Data
{
template <typename T>
class LOBStreamBuf : public BasicUnbufferedStreamBuf<T, std::char_traits<T>>
/// This is the streambuf class used for reading from and writing to a LOB.
{
public:
LOBStreamBuf(LOB<T> & lob) : _lob(lob), _it(_lob.begin())
/// Creates LOBStreamBuf.
{
}
~LOBStreamBuf()
/// Destroys LOBStreamBuf.
{
}
protected:
typedef std::char_traits<T> TraitsType;
typedef BasicUnbufferedStreamBuf<T, TraitsType> BaseType;
typename BaseType::int_type readFromDevice()
{
if (_it != _lob.end())
return BaseType::charToInt(*_it++);
else
return -1;
}
typename BaseType::int_type writeToDevice(T c)
{
_lob.appendRaw(&c, 1);
return 1;
}
private:
LOB<T> & _lob;
typename LOB<T>::Iterator _it;
};
template <typename T>
class LOBIOS : public virtual std::ios
/// The base class for LOBInputStream and
/// LOBOutputStream.
///
/// This class is needed to ensure the correct initialization
/// order of the stream buffer and base classes.
{
public:
LOBIOS(LOB<T> & lob, openmode mode) : _buf(lob)
/// Creates the LOBIOS with the given LOB.
{
poco_ios_init(&_buf);
}
~LOBIOS()
/// Destroys the LOBIOS.
{
}
LOBStreamBuf<T> * rdbuf()
/// Returns a pointer to the internal LOBStreamBuf.
{
return &_buf;
}
protected:
LOBStreamBuf<T> _buf;
};
template <typename T>
class LOBOutputStream : public LOBIOS<T>, public std::basic_ostream<T, std::char_traits<T>>
/// An output stream for writing to a LOB.
{
public:
LOBOutputStream(LOB<T> & lob) : LOBIOS<T>(lob, std::ios::out), std::ostream(LOBIOS<T>::rdbuf())
/// Creates the LOBOutputStream with the given LOB.
{
}
~LOBOutputStream()
/// Destroys the LOBOutputStream.
{
}
};
template <typename T>
class LOBInputStream : public LOBIOS<T>, public std::basic_istream<T, std::char_traits<T>>
/// An input stream for reading from a LOB.
{
public:
LOBInputStream(LOB<T> & lob) : LOBIOS<T>(lob, std::ios::in), std::istream(LOBIOS<T>::rdbuf())
/// Creates the LOBInputStream with the given LOB.
{
}
~LOBInputStream()
/// Destroys the LOBInputStream.
{
}
};
typedef LOBOutputStream<unsigned char> BLOBOutputStream;
typedef LOBOutputStream<char> CLOBOutputStream;
typedef LOBInputStream<unsigned char> BLOBInputStream;
typedef LOBInputStream<char> CLOBInputStream;
}
} // namespace Poco::Data
#endif // Data_LOBStream_INCLUDED

74
base/poco/Data/src/DynamicLOB.cpp
View File

@ -1,74 +0,0 @@
//
// DynamicLOB.cpp
//
// Library: Data
// Package: DataCore
// Module: DynamicLOB
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifdef __GNUC__
// TODO: determine g++ version able to do the right thing without these specializations
#include "Poco/Data/DynamicLOB.h"
#include "Poco/Data/LOB.h"
#include "Poco/Dynamic/Var.h"
namespace Poco {
namespace Dynamic {
using Poco::Data::CLOB;
using Poco::Data::BLOB;
template <>
Var::operator CLOB () const
{
VarHolder* pHolder = content();
if (!pHolder)
throw InvalidAccessException("Can not convert empty value.");
if (typeid(CLOB) == pHolder->type())
return extract<CLOB>();
else
{
std::string result;
pHolder->convert(result);
return CLOB(result);
}
}
template <>
Var::operator BLOB () const
{
VarHolder* pHolder = content();
if (!pHolder)
throw InvalidAccessException("Can not convert empty value.");
if (typeid(BLOB) == pHolder->type())
return extract<BLOB>();
else
{
std::string result;
pHolder->convert(result);
return BLOB(reinterpret_cast<const unsigned char*>(result.data()),
result.size());
}
}
} } // namespace Poco::Data
#endif // __GNUC__

13
base/poco/Foundation/CMakeLists.txt
View File

@ -31,8 +31,6 @@ set (SRCS
src/ASCIIEncoding.cpp
src/AsyncChannel.cpp
src/AtomicCounter.cpp
src/Base32Decoder.cpp
src/Base32Encoder.cpp
src/Base64Decoder.cpp
src/Base64Encoder.cpp
src/BinaryReader.cpp
@ -81,9 +79,6 @@ set (SRCS
src/HexBinaryEncoder.cpp
src/InflatingStream.cpp
src/JSONString.cpp
src/Latin1Encoding.cpp
src/Latin2Encoding.cpp
src/Latin9Encoding.cpp
src/LineEndingConverter.cpp
src/LocalDateTime.cpp
src/LogFile.cpp
@ -91,8 +86,6 @@ set (SRCS
src/LoggingFactory.cpp
src/LoggingRegistry.cpp
src/LogStream.cpp
src/Manifest.cpp
src/MD4Engine.cpp
src/MD5Engine.cpp
src/MemoryPool.cpp
src/MemoryStream.cpp
@ -113,7 +106,6 @@ set (SRCS
src/PatternFormatter.cpp
src/Pipe.cpp
src/PipeImpl.cpp
src/PipeStream.cpp
src/PriorityNotificationQueue.cpp
src/Process.cpp
src/PurgeStrategy.cpp
@ -136,10 +128,8 @@ set (SRCS
src/StreamChannel.cpp
src/StreamConverter.cpp
src/StreamCopier.cpp
src/StreamTokenizer.cpp
src/String.cpp
src/StringTokenizer.cpp
src/SynchronizedObject.cpp
src/SyslogChannel.cpp
src/Task.cpp
src/TaskManager.cpp
@ -175,9 +165,6 @@ set (SRCS
src/VarHolder.cpp
src/VarIterator.cpp
src/Void.cpp
src/Windows1250Encoding.cpp
src/Windows1251Encoding.cpp
src/Windows1252Encoding.cpp
)
add_library (_poco_foundation ${SRCS})

105
base/poco/Foundation/include/Poco/Base32Decoder.h
View File

@ -1,105 +0,0 @@
//
// Base32Decoder.h
//
// Library: Foundation
// Package: Streams
// Module: Base32
//
// Definition of class Base32Decoder.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_Base32Decoder_INCLUDED
#define Foundation_Base32Decoder_INCLUDED
#include <istream>
#include "Poco/Foundation.h"
#include "Poco/UnbufferedStreamBuf.h"
namespace Poco
{
class Foundation_API Base32DecoderBuf : public UnbufferedStreamBuf
/// This streambuf base32-decodes all data read
/// from the istream connected to it.
///
/// Note: For performance reasons, the characters
/// are read directly from the given istream's
/// underlying streambuf, so the state
/// of the istream will not reflect that of
/// its streambuf.
{
public:
Base32DecoderBuf(std::istream & istr);
~Base32DecoderBuf();
private:
int readFromDevice();
int readOne();
unsigned char _group[8];
int _groupLength;
int _groupIndex;
std::streambuf & _buf;
static unsigned char IN_ENCODING[256];
static bool IN_ENCODING_INIT;
private:
Base32DecoderBuf(const Base32DecoderBuf &);
Base32DecoderBuf & operator=(const Base32DecoderBuf &);
};
class Foundation_API Base32DecoderIOS : public virtual std::ios
/// The base class for Base32Decoder.
///
/// This class is needed to ensure the correct initialization
/// order of the stream buffer and base classes.
{
public:
Base32DecoderIOS(std::istream & istr);
~Base32DecoderIOS();
Base32DecoderBuf * rdbuf();
protected:
Base32DecoderBuf _buf;
private:
Base32DecoderIOS(const Base32DecoderIOS &);
Base32DecoderIOS & operator=(const Base32DecoderIOS &);
};
class Foundation_API Base32Decoder : public Base32DecoderIOS, public std::istream
/// This istream base32-decodes all data
/// read from the istream connected to it.
///
/// Note: For performance reasons, the characters
/// are read directly from the given istream's
/// underlying streambuf, so the state
/// of the istream will not reflect that of
/// its streambuf.
{
public:
Base32Decoder(std::istream & istr);
~Base32Decoder();
private:
Base32Decoder(const Base32Decoder &);
Base32Decoder & operator=(const Base32Decoder &);
};
} // namespace Poco
#endif // Foundation_Base32Decoder_INCLUDED

111
base/poco/Foundation/include/Poco/Base32Encoder.h
View File

@ -1,111 +0,0 @@
//
// Base32Encoder.h
//
// Library: Foundation
// Package: Streams
// Module: Base32
//
// Definition of class Base32Encoder.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_Base32Encoder_INCLUDED
#define Foundation_Base32Encoder_INCLUDED
#include <ostream>
#include "Poco/Foundation.h"
#include "Poco/UnbufferedStreamBuf.h"
namespace Poco
{
class Foundation_API Base32EncoderBuf : public UnbufferedStreamBuf
/// This streambuf base32-encodes all data written
/// to it and forwards it to a connected
/// ostream.
///
/// Note: The characters are directly written
/// to the ostream's streambuf, thus bypassing
/// the ostream. The ostream's state is therefore
/// not updated to match the buffer's state.
{
public:
Base32EncoderBuf(std::ostream & ostr, bool padding = true);
~Base32EncoderBuf();
int close();
/// Closes the stream buffer.
private:
int writeToDevice(char c);
unsigned char _group[5];
int _groupLength;
std::streambuf & _buf;
bool _doPadding;
static const unsigned char OUT_ENCODING[32];
friend class Base32DecoderBuf;
Base32EncoderBuf(const Base32EncoderBuf &);
Base32EncoderBuf & operator=(const Base32EncoderBuf &);
};
class Foundation_API Base32EncoderIOS : public virtual std::ios
/// The base class for Base32Encoder.
///
/// This class is needed to ensure the correct initialization
/// order of the stream buffer and base classes.
{
public:
Base32EncoderIOS(std::ostream & ostr, bool padding = true);
~Base32EncoderIOS();
int close();
Base32EncoderBuf * rdbuf();
protected:
Base32EncoderBuf _buf;
private:
Base32EncoderIOS(const Base32EncoderIOS &);
Base32EncoderIOS & operator=(const Base32EncoderIOS &);
};
class Foundation_API Base32Encoder : public Base32EncoderIOS, public std::ostream
/// This ostream base32-encodes all data
/// written to it and forwards it to
/// a connected ostream.
/// Always call close() when done
/// writing data, to ensure proper
/// completion of the encoding operation.
///
/// Note: The characters are directly written
/// to the ostream's streambuf, thus bypassing
/// the ostream. The ostream's state is therefore
/// not updated to match the buffer's state.
{
public:
Base32Encoder(std::ostream & ostr, bool padding = true);
~Base32Encoder();
private:
Base32Encoder(const Base32Encoder &);
Base32Encoder & operator=(const Base32Encoder &);
};
} // namespace Poco
#endif // Foundation_Base32Encoder_INCLUDED

92
base/poco/Foundation/include/Poco/ClassLibrary.h
View File

@ -1,92 +0,0 @@
//
// ClassLibrary.h
//
// Library: Foundation
// Package: SharedLibrary
// Module: ClassLoader
//
// Definitions for class libraries.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_ClassLibrary_INCLUDED
#define Foundation_ClassLibrary_INCLUDED
#include <typeinfo>
#include "Poco/Foundation.h"
#include "Poco/Manifest.h"
# define POCO_LIBRARY_API
//
// the entry points for every class library
//
extern "C" {
bool POCO_LIBRARY_API pocoBuildManifest(Poco::ManifestBase * pManifest);
void POCO_LIBRARY_API pocoInitializeLibrary();
void POCO_LIBRARY_API pocoUninitializeLibrary();
}
//
// additional support for named manifests
//
#define POCO_DECLARE_NAMED_MANIFEST(name) \
extern "C" { \
bool POCO_LIBRARY_API POCO_JOIN(pocoBuildManifest, name)(Poco::ManifestBase * pManifest); \
}
//
// Macros to automatically implement pocoBuildManifest
//
// usage:
//
// POCO_BEGIN_MANIFEST(MyBaseClass)
// POCO_EXPORT_CLASS(MyFirstClass)
// POCO_EXPORT_CLASS(MySecondClass)
// ...
// POCO_END_MANIFEST
//
#define POCO_BEGIN_MANIFEST_IMPL(fnName, base) \
bool fnName(Poco::ManifestBase * pManifest_) \
{ \
typedef base _Base; \
typedef Poco::Manifest<_Base> _Manifest; \
std::string requiredType(typeid(_Manifest).name()); \
std::string actualType(pManifest_->className()); \
if (requiredType == actualType) \
{ \
Poco::Manifest<_Base> * pManifest = static_cast<_Manifest *>(pManifest_);
#define POCO_BEGIN_MANIFEST(base) POCO_BEGIN_MANIFEST_IMPL(pocoBuildManifest, base)
#define POCO_BEGIN_NAMED_MANIFEST(name, base) \
POCO_DECLARE_NAMED_MANIFEST(name) \
POCO_BEGIN_MANIFEST_IMPL(POCO_JOIN(pocoBuildManifest, name), base)
#define POCO_END_MANIFEST \
return true; \
} \
else return false; \
}
#define POCO_EXPORT_CLASS(cls) pManifest->insert(new Poco::MetaObject<cls, _Base>(#cls));
#define POCO_EXPORT_SINGLETON(cls) pManifest->insert(new Poco::MetaSingleton<cls, _Base>(#cls));
#endif // Foundation_ClassLibrary_INCLUDED

355
base/poco/Foundation/include/Poco/ClassLoader.h
View File

@ -1,355 +0,0 @@
//
// ClassLoader.h
//
// Library: Foundation
// Package: SharedLibrary
// Module: ClassLoader
//
// Definition of the ClassLoader class.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_ClassLoader_INCLUDED
#define Foundation_ClassLoader_INCLUDED
#include <map>
#include "Poco/Exception.h"
#include "Poco/Foundation.h"
#include "Poco/Manifest.h"
#include "Poco/MetaObject.h"
#include "Poco/Mutex.h"
#include "Poco/SharedLibrary.h"
namespace Poco
{
template <class Base>
class ClassLoader
/// The ClassLoader loads C++ classes from shared libraries
/// at runtime. It must be instantiated with a root class
/// of the loadable classes.
/// For a class to be loadable from a library, the library
/// must provide a Manifest of all the classes it contains.
/// The Manifest for a shared library can be easily built
/// with the help of the macros in the header file
/// "Foundation/ClassLibrary.h".
///
/// Starting with POCO release 1.3, a class library can
/// export multiple manifests. In addition to the default
/// (unnamed) manifest, multiple named manifests can
/// be exported, each having a different base class.
///
/// There is one important restriction: one instance of
/// ClassLoader can only load one manifest from a class
/// library.
{
public:
typedef AbstractMetaObject<Base> Meta;
typedef Manifest<Base> Manif;
typedef void (*InitializeLibraryFunc)();
typedef void (*UninitializeLibraryFunc)();
typedef bool (*BuildManifestFunc)(ManifestBase *);
struct LibraryInfo
{
SharedLibrary * pLibrary;
const Manif * pManifest;
int refCount;
};
typedef std::map<std::string, LibraryInfo> LibraryMap;
class Iterator
/// The ClassLoader's very own iterator class.
{
public:
typedef std::pair<std::string, const Manif *> Pair;
Iterator(const typename LibraryMap::const_iterator & it) { _it = it; }
Iterator(const Iterator & it) { _it = it._it; }
~Iterator() { }
Iterator & operator=(const Iterator & it)
{
_it = it._it;
return *this;
}
inline bool operator==(const Iterator & it) const { return _it == it._it; }
inline bool operator!=(const Iterator & it) const { return _it != it._it; }
Iterator & operator++() // prefix
{
++_it;
return *this;
}
Iterator operator++(int) // postfix
{
Iterator result(_it);
++_it;
return result;
}
inline const Pair * operator*() const
{
_pair.first = _it->first;
_pair.second = _it->second.pManifest;
return &_pair;
}
inline const Pair * operator->() const
{
_pair.first = _it->first;
_pair.second = _it->second.pManifest;
return &_pair;
}
private:
typename LibraryMap::const_iterator _it;
mutable Pair _pair;
};
ClassLoader()
/// Creates the ClassLoader.
{
}
virtual ~ClassLoader()
/// Destroys the ClassLoader.
{
for (typename LibraryMap::const_iterator it = _map.begin(); it != _map.end(); ++it)
{
delete it->second.pLibrary;
delete it->second.pManifest;
}
}
void loadLibrary(const std::string & path, const std::string & manifest)
/// Loads a library from the given path, using the given manifest.
/// Does nothing if the library is already loaded.
/// Throws a LibraryLoadException if the library
/// cannot be loaded or does not have a Manifest.
/// If the library exports a function named "pocoInitializeLibrary",
/// this function is executed.
/// If called multiple times for the same library,
/// the number of calls to unloadLibrary() must be the same
/// for the library to become unloaded.
{
FastMutex::ScopedLock lock(_mutex);
typename LibraryMap::iterator it = _map.find(path);
if (it == _map.end())
{
LibraryInfo li;
li.pLibrary = 0;
li.pManifest = 0;
li.refCount = 1;
try
{
li.pLibrary = new SharedLibrary(path);
li.pManifest = new Manif();
std::string pocoBuildManifestSymbol("pocoBuildManifest");
pocoBuildManifestSymbol.append(manifest);
if (li.pLibrary->hasSymbol("pocoInitializeLibrary"))
{
InitializeLibraryFunc initializeLibrary = (InitializeLibraryFunc)li.pLibrary->getSymbol("pocoInitializeLibrary");
initializeLibrary();
}
if (li.pLibrary->hasSymbol(pocoBuildManifestSymbol))
{
BuildManifestFunc buildManifest = (BuildManifestFunc)li.pLibrary->getSymbol(pocoBuildManifestSymbol);
if (buildManifest(const_cast<Manif *>(li.pManifest)))
_map[path] = li;
else
throw LibraryLoadException(std::string("Manifest class mismatch in ") + path, manifest);
}
else
throw LibraryLoadException(std::string("No manifest in ") + path, manifest);
}
catch (...)
{
delete li.pLibrary;
delete li.pManifest;
throw;
}
}
else
{
++it->second.refCount;
}
}
void loadLibrary(const std::string & path)
/// Loads a library from the given path. Does nothing
/// if the library is already loaded.
/// Throws a LibraryLoadException if the library
/// cannot be loaded or does not have a Manifest.
/// If the library exports a function named "pocoInitializeLibrary",
/// this function is executed.
/// If called multiple times for the same library,
/// the number of calls to unloadLibrary() must be the same
/// for the library to become unloaded.
///
/// Equivalent to loadLibrary(path, "").
{
loadLibrary(path, "");
}
void unloadLibrary(const std::string & path)
/// Unloads the given library.
/// Be extremely cautious when unloading shared libraries.
/// If objects from the library are still referenced somewhere,
/// a total crash is very likely.
/// If the library exports a function named "pocoUninitializeLibrary",
/// this function is executed before it is unloaded.
/// If loadLibrary() has been called multiple times for the same
/// library, the number of calls to unloadLibrary() must be the same
/// for the library to become unloaded.
{
FastMutex::ScopedLock lock(_mutex);
typename LibraryMap::iterator it = _map.find(path);
if (it != _map.end())
{
if (--it->second.refCount == 0)
{
if (it->second.pLibrary->hasSymbol("pocoUninitializeLibrary"))
{
UninitializeLibraryFunc uninitializeLibrary
= (UninitializeLibraryFunc)it->second.pLibrary->getSymbol("pocoUninitializeLibrary");
uninitializeLibrary();
}
delete it->second.pManifest;
it->second.pLibrary->unload();
delete it->second.pLibrary;
_map.erase(it);
}
}
else
throw NotFoundException(path);
}
const Meta * findClass(const std::string & className) const
/// Returns a pointer to the MetaObject for the given
/// class, or a null pointer if the class is not known.
{
FastMutex::ScopedLock lock(_mutex);
for (typename LibraryMap::const_iterator it = _map.begin(); it != _map.end(); ++it)
{
const Manif * pManif = it->second.pManifest;
typename Manif::Iterator itm = pManif->find(className);
if (itm != pManif->end())
return *itm;
}
return 0;
}
const Meta & classFor(const std::string & className) const
/// Returns a reference to the MetaObject for the given
/// class. Throws a NotFoundException if the class
/// is not known.
{
const Meta * pMeta = findClass(className);
if (pMeta)
return *pMeta;
else
throw NotFoundException(className);
}
Base * create(const std::string & className) const
/// Creates an instance of the given class.
/// Throws a NotFoundException if the class
/// is not known.
{
return classFor(className).create();
}
Base & instance(const std::string & className) const
/// Returns a reference to the sole instance of
/// the given class. The class must be a singleton,
/// otherwise an InvalidAccessException will be thrown.
/// Throws a NotFoundException if the class
/// is not known.
{
return classFor(className).instance();
}
bool canCreate(const std::string & className) const
/// Returns true if create() can create new instances
/// of the class.
{
return classFor(className).canCreate();
}
void destroy(const std::string & className, Base * pObject) const
/// Destroys the object pObject points to.
/// Does nothing if object is not found.
{
classFor(className).destroy(pObject);
}
bool isAutoDelete(const std::string & className, Base * pObject) const
/// Returns true if the object is automatically
/// deleted by its meta object.
{
return classFor(className).isAutoDelete(pObject);
}
const Manif * findManifest(const std::string & path) const
/// Returns a pointer to the Manifest for the given
/// library, or a null pointer if the library has not been loaded.
{
FastMutex::ScopedLock lock(_mutex);
typename LibraryMap::const_iterator it = _map.find(path);
if (it != _map.end())
return it->second.pManifest;
else
return 0;
}
const Manif & manifestFor(const std::string & path) const
/// Returns a reference to the Manifest for the given library
/// Throws a NotFoundException if the library has not been loaded.
{
const Manif * pManif = findManifest(path);
if (pManif)
return *pManif;
else
throw NotFoundException(path);
}
bool isLibraryLoaded(const std::string & path) const
/// Returns true if the library with the given name
/// has already been loaded.
{
return findManifest(path) != 0;
}
Iterator begin() const
{
FastMutex::ScopedLock lock(_mutex);
return Iterator(_map.begin());
}
Iterator end() const
{
FastMutex::ScopedLock lock(_mutex);
return Iterator(_map.end());
}
private:
LibraryMap _map;
mutable FastMutex _mutex;
};
} // namespace Poco
#endif // Foundation_ClassLoader_INCLUDED

102
base/poco/Foundation/include/Poco/EventLogChannel.h
View File

@ -1,102 +0,0 @@
//
// EventLogChannel.h
//
// Library: Foundation
// Package: Logging
// Module: EventLogChannel
//
// Definition of the EventLogChannel class specific to WIN32.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_EventLogChannel_INCLUDED
#define Foundation_EventLogChannel_INCLUDED
#include "Poco/Channel.h"
#include "Poco/Foundation.h"
#include "Poco/UnWindows.h"
namespace Poco
{
class Foundation_API EventLogChannel : public Channel
/// This Windows-only channel works with the Windows NT Event Log
/// service.
///
/// To work properly, the EventLogChannel class requires that either
/// the PocoFoundation.dll or the PocoMsg.dll Dynamic Link Library
/// containing the message definition resources can be found in $PATH.
{
public:
EventLogChannel();
/// Creates the EventLogChannel.
/// The name of the current application (or more correctly,
/// the name of its executable) is taken as event source name.
EventLogChannel(const std::string & name);
/// Creates the EventLogChannel with the given event source name.
EventLogChannel(const std::string & name, const std::string & host);
/// Creates an EventLogChannel with the given event source
/// name that routes messages to the given host.
void open();
/// Opens the EventLogChannel. If necessary, the
/// required registry entries to register a
/// message resource DLL are made.
void close();
/// Closes the EventLogChannel.
void log(const Message & msg);
/// Logs the given message to the Windows Event Log.
///
/// The message type and priority are mapped to
/// appropriate values for Event Log type and category.
void setProperty(const std::string & name, const std::string & value);
/// Sets or changes a configuration property.
///
/// The following properties are supported:
///
/// * name: The name of the event source.
/// * loghost: The name of the host where the Event Log service is running.
/// The default is "localhost".
/// * host: same as host.
/// * logfile: The name of the log file. The default is "Application".
std::string getProperty(const std::string & name) const;
/// Returns the value of the given property.
static const std::string PROP_NAME;
static const std::string PROP_HOST;
static const std::string PROP_LOGHOST;
static const std::string PROP_LOGFILE;
protected:
~EventLogChannel();
static int getType(const Message & msg);
static int getCategory(const Message & msg);
void setUpRegistry() const;
static std::string findLibrary(const char * name);
private:
std::string _name;
std::string _host;
std::string _logFile;
HANDLE _h;
};
} // namespace Poco
#endif // Foundation_EventLogChannel_INCLUDED

126
base/poco/Foundation/include/Poco/FPEnvironment_DUMMY.h
View File

@ -1,126 +0,0 @@
//
// FPEnvironment_DUMMY.h
//
// Library: Foundation
// Package: Core
// Module: FPEnvironment
//
// Definition of class FPEnvironmentImpl for platforms that do not
// support IEEE 754 extensions.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_FPEnvironment_DUMMY_INCLUDED
#define Foundation_FPEnvironment_DUMMY_INCLUDED
#include <cmath>
#include "Poco/Foundation.h"
namespace Poco
{
class Foundation_API FPEnvironmentImpl
{
protected:
enum RoundingModeImpl
{
FP_ROUND_DOWNWARD_IMPL,
FP_ROUND_UPWARD_IMPL,
FP_ROUND_TONEAREST_IMPL,
FP_ROUND_TOWARDZERO_IMPL
};
enum FlagImpl
{
FP_DIVIDE_BY_ZERO_IMPL,
FP_INEXACT_IMPL,
FP_OVERFLOW_IMPL,
FP_UNDERFLOW_IMPL,
FP_INVALID_IMPL
};
FPEnvironmentImpl();
FPEnvironmentImpl(const FPEnvironmentImpl & env);
~FPEnvironmentImpl();
FPEnvironmentImpl & operator=(const FPEnvironmentImpl & env);
void keepCurrentImpl();
static void clearFlagsImpl();
static bool isFlagImpl(FlagImpl flag);
static void setRoundingModeImpl(RoundingModeImpl mode);
static RoundingModeImpl getRoundingModeImpl();
static bool isInfiniteImpl(float value);
static bool isInfiniteImpl(double value);
static bool isInfiniteImpl(long double value);
static bool isNaNImpl(float value);
static bool isNaNImpl(double value);
static bool isNaNImpl(long double value);
static float copySignImpl(float target, float source);
static double copySignImpl(double target, double source);
static long double copySignImpl(long double target, long double source);
private:
static RoundingModeImpl _roundingMode;
};
//
// inlines
//
inline bool FPEnvironmentImpl::isInfiniteImpl(float value)
{
return std::isinf(value) != 0;
}
inline bool FPEnvironmentImpl::isInfiniteImpl(double value)
{
return std::isinf(value) != 0;
}
inline bool FPEnvironmentImpl::isInfiniteImpl(long double value)
{
return std::isinf((double)value) != 0;
}
inline bool FPEnvironmentImpl::isNaNImpl(float value)
{
return std::isnan(value) != 0;
}
inline bool FPEnvironmentImpl::isNaNImpl(double value)
{
return std::isnan(value) != 0;
}
inline bool FPEnvironmentImpl::isNaNImpl(long double value)
{
return std::isnan((double)value) != 0;
}
inline float FPEnvironmentImpl::copySignImpl(float target, float source)
{
return copysignf(target, source);
}
inline double FPEnvironmentImpl::copySignImpl(double target, double source)
{
return copysign(target, source);
}
} // namespace Poco
#endif // Foundation_FPEnvironment_DUMMY_INCLUDED

72
base/poco/Foundation/include/Poco/FileStream_WIN32.h
View File

@ -1,72 +0,0 @@
//
// FileStream_WIN32.h
//
// Library: Foundation
// Package: Streams
// Module: FileStream
//
// Definition of the FileStreamBuf, FileInputStream and FileOutputStream classes.
//
// Copyright (c) 2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_FileStream_WIN32_INCLUDED
#define Foundation_FileStream_WIN32_INCLUDED
#include "Poco/BufferedBidirectionalStreamBuf.h"
#include "Poco/Foundation.h"
#include "Poco/UnWindows.h"
namespace Poco
{
class Foundation_API FileStreamBuf : public BufferedBidirectionalStreamBuf
/// This stream buffer handles Fileio
{
public:
FileStreamBuf();
/// Creates a FileStreamBuf.
~FileStreamBuf();
/// Destroys the FileStream.
void open(const std::string & path, std::ios::openmode mode);
/// Opens the given file in the given mode.
bool close();
/// Closes the File stream buffer. Returns true if successful,
/// false otherwise.
std::streampos seekoff(std::streamoff off, std::ios::seekdir dir, std::ios::openmode mode = std::ios::in | std::ios::out);
/// change position by offset, according to way and mode
std::streampos seekpos(std::streampos pos, std::ios::openmode mode = std::ios::in | std::ios::out);
/// change to specified position, according to mode
protected:
enum
{
BUFFER_SIZE = 4096
};
int readFromDevice(char * buffer, std::streamsize length);
int writeToDevice(const char * buffer, std::streamsize length);
private:
std::string _path;
HANDLE _handle;
UInt64 _pos;
};
} // namespace Poco
#endif // Foundation_FileStream_WIN32_INCLUDED

176
base/poco/Foundation/include/Poco/HashSet.h
View File

@ -1,176 +0,0 @@
//
// HashSet.h
//
// Library: Foundation
// Package: Hashing
// Module: HashSet
//
// Definition of the HashSet class.
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_HashSet_INCLUDED
#define Foundation_HashSet_INCLUDED
#include "Poco/Foundation.h"
#include "Poco/LinearHashTable.h"
namespace Poco
{
template <class Value, class HashFunc = Hash<Value>>
class HashSet
/// This class implements a set using a LinearHashTable.
///
/// A HashSet can be used just like a std::set.
{
public:
typedef Value ValueType;
typedef Value & Reference;
typedef const Value & ConstReference;
typedef Value * Pointer;
typedef const Value * ConstPointer;
typedef HashFunc Hash;
typedef LinearHashTable<ValueType, Hash> HashTable;
typedef typename HashTable::Iterator Iterator;
typedef typename HashTable::ConstIterator ConstIterator;
HashSet()
/// Creates an empty HashSet.
{
}
HashSet(std::size_t initialReserve) : _table(initialReserve)
/// Creates the HashSet, using the given initialReserve.
{
}
HashSet(const HashSet & set) : _table(set._table)
/// Creates the HashSet by copying another one.
{
}
~HashSet()
/// Destroys the HashSet.
{
}
HashSet & operator=(const HashSet & table)
/// Assigns another HashSet.
{
HashSet tmp(table);
swap(tmp);
return *this;
}
void swap(HashSet & set)
/// Swaps the HashSet with another one.
{
_table.swap(set._table);
}
ConstIterator begin() const
/// Returns an iterator pointing to the first entry, if one exists.
{
return _table.begin();
}
ConstIterator end() const
/// Returns an iterator pointing to the end of the table.
{
return _table.end();
}
Iterator begin()
/// Returns an iterator pointing to the first entry, if one exists.
{
return _table.begin();
}
Iterator end()
/// Returns an iterator pointing to the end of the table.
{
return _table.end();
}
ConstIterator find(const ValueType & value) const
/// Finds an entry in the table.
{
return _table.find(value);
}
Iterator find(const ValueType & value)
/// Finds an entry in the table.
{
return _table.find(value);
}
std::size_t count(const ValueType & value) const
/// Returns the number of elements with the given
/// value, with is either 1 or 0.
{
return _table.count(value);
}
std::pair<Iterator, bool> insert(const ValueType & value)
/// Inserts an element into the set.
///
/// If the element already exists in the set,
/// a pair(iterator, false) with iterator pointing to the
/// existing element is returned.
/// Otherwise, the element is inserted an a
/// pair(iterator, true) with iterator
/// pointing to the new element is returned.
{
return _table.insert(value);
}
void erase(Iterator it)
/// Erases the element pointed to by it.
{
_table.erase(it);
}
void erase(const ValueType & value)
/// Erases the element with the given value, if it exists.
{
_table.erase(value);
}
void clear()
/// Erases all elements.
{
_table.clear();
}
std::size_t size() const
/// Returns the number of elements in the table.
{
return _table.size();
}
bool empty() const
/// Returns true iff the table is empty.
{
return _table.empty();
}
private:
HashTable _table;
};
} // namespace Poco
#endif // Foundation_HashSet_INCLUDED

352
base/poco/Foundation/include/Poco/HashTable.h
View File

@ -1,352 +0,0 @@
//
// HashTable.h
//
// Library: Foundation
// Package: Hashing
// Module: HashTable
//
// Definition of the HashTable class.
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_HashTable_INCLUDED
#define Foundation_HashTable_INCLUDED
#include <cstddef>
#include <cstring>
#include <map>
#include <vector>
#include "Poco/Exception.h"
#include "Poco/Foundation.h"
#include "Poco/HashFunction.h"
#include "Poco/HashStatistic.h"
namespace Poco
{
//@ deprecated
template <class Key, class Value, class KeyHashFunction = HashFunction<Key>>
class HashTable
/// A HashTable stores a key value pair that can be looked up via a hashed key.
///
/// Collision handling is done via overflow maps(!). With small hash tables performance of this
/// data struct will be closer to that a map than a hash table, i.e. slower. On the plus side,
/// this class offers remove operations. Also HashTable full errors are not possible. If a fast
/// HashTable implementation is needed and the remove operation is not required, use SimpleHashTable
/// instead.
///
/// This class is NOT thread safe.
{
public:
typedef std::map<Key, Value> HashEntryMap;
typedef HashEntryMap ** HashTableVector;
typedef typename HashEntryMap::const_iterator ConstIterator;
typedef typename HashEntryMap::iterator Iterator;
HashTable(UInt32 initialSize = 251) : _entries(0), _size(0), _maxCapacity(initialSize)
/// Creates the HashTable.
{
_entries = new HashEntryMap *[initialSize];
memset(_entries, '\0', sizeof(HashEntryMap *) * initialSize);
}
HashTable(const HashTable & ht) : _entries(new HashEntryMap *[ht._maxCapacity]), _size(ht._size), _maxCapacity(ht._maxCapacity)
{
for (UInt32 i = 0; i < _maxCapacity; ++i)
{
if (ht._entries[i])
_entries[i] = new HashEntryMap(ht._entries[i]->begin(), ht._entries[i]->end());
else
_entries[i] = 0;
}
}
~HashTable()
/// Destroys the HashTable.
{
clear();
}
HashTable & operator=(const HashTable & ht)
{
if (this != &ht)
{
clear();
_maxCapacity = ht._maxCapacity;
poco_assert_dbg(_entries == 0);
_entries = new HashEntryMap *[_maxCapacity];
_size = ht._size;
for (UInt32 i = 0; i < _maxCapacity; ++i)
{
if (ht._entries[i])
_entries[i] = new HashEntryMap(ht._entries[i]->begin(), ht._entries[i]->end());
else
_entries[i] = 0;
}
}
return *this;
}
void clear()
{
if (!_entries)
return;
for (UInt32 i = 0; i < _maxCapacity; ++i)
{
delete _entries[i];
}
delete[] _entries;
_entries = 0;
_size = 0;
_maxCapacity = 0;
}
UInt32 insert(const Key & key, const Value & value)
/// Returns the hash value of the inserted item.
/// Throws an exception if the entry was already inserted
{
UInt32 hsh = hash(key);
insertRaw(key, hsh, value);
return hsh;
}
Value & insertRaw(const Key & key, UInt32 hsh, const Value & value)
/// Returns the hash value of the inserted item.
/// Throws an exception if the entry was already inserted
{
if (!_entries[hsh])
_entries[hsh] = new HashEntryMap();
std::pair<typename HashEntryMap::iterator, bool> res(_entries[hsh]->insert(std::make_pair(key, value)));
if (!res.second)
throw InvalidArgumentException("HashTable::insert, key already exists.");
_size++;
return res.first->second;
}
UInt32 update(const Key & key, const Value & value)
/// Returns the hash value of the inserted item.
/// Replaces an existing entry if it finds one
{
UInt32 hsh = hash(key);
updateRaw(key, hsh, value);
return hsh;
}
void updateRaw(const Key & key, UInt32 hsh, const Value & value)
/// Returns the hash value of the inserted item.
/// Replaces an existing entry if it finds one
{
if (!_entries[hsh])
_entries[hsh] = new HashEntryMap();
std::pair<Iterator, bool> res = _entries[hsh]->insert(std::make_pair(key, value));
if (res.second == false)
res.first->second = value;
else
_size++;
}
void remove(const Key & key)
{
UInt32 hsh = hash(key);
removeRaw(key, hsh);
}
void removeRaw(const Key & key, UInt32 hsh)
/// Performance version, allows to specify the hash value
{
if (_entries[hsh])
{
_size -= _entries[hsh]->erase(key);
}
}
UInt32 hash(const Key & key) const { return _hash(key, _maxCapacity); }
const Value & get(const Key & key) const
/// Throws an exception if the value does not exist
{
UInt32 hsh = hash(key);
return getRaw(key, hsh);
}
const Value & getRaw(const Key & key, UInt32 hsh) const
/// Throws an exception if the value does not exist
{
if (!_entries[hsh])
throw InvalidArgumentException("key not found");
ConstIterator it = _entries[hsh]->find(key);
if (it == _entries[hsh]->end())
throw InvalidArgumentException("key not found");
return it->second;
}
Value & get(const Key & key)
/// Throws an exception if the value does not exist
{
UInt32 hsh = hash(key);
return const_cast<Value &>(getRaw(key, hsh));
}
const Value & operator[](const Key & key) const { return get(key); }
Value & operator[](const Key & key)
{
UInt32 hsh = hash(key);
if (!_entries[hsh])
return insertRaw(key, hsh, Value());
ConstIterator it = _entries[hsh]->find(key);
if (it == _entries[hsh]->end())
return insertRaw(key, hsh, Value());
return it->second;
}
const Key & getKeyRaw(const Key & key, UInt32 hsh)
/// Throws an exception if the key does not exist. returns a reference to the internally
/// stored key. Useful when someone does an insert and wants for performance reason only to store
/// a pointer to the key in another collection
{
if (!_entries[hsh])
throw InvalidArgumentException("key not found");
ConstIterator it = _entries[hsh]->find(key);
if (it == _entries[hsh]->end())
throw InvalidArgumentException("key not found");
return it->first;
}
bool get(const Key & key, Value & v) const
/// Sets v to the found value, returns false if no value was found
{
UInt32 hsh = hash(key);
return getRaw(key, hsh, v);
}
bool getRaw(const Key & key, UInt32 hsh, Value & v) const
/// Sets v to the found value, returns false if no value was found
{
if (!_entries[hsh])
return false;
ConstIterator it = _entries[hsh]->find(key);
if (it == _entries[hsh]->end())
return false;
v = it->second;
return true;
}
bool exists(const Key & key)
{
UInt32 hsh = hash(key);
return existsRaw(key, hsh);
}
bool existsRaw(const Key & key, UInt32 hsh) { return _entries[hsh] && (_entries[hsh]->end() != _entries[hsh]->find(key)); }
std::size_t size() const
/// Returns the number of elements already inserted into the HashTable
{
return _size;
}
UInt32 maxCapacity() const { return _maxCapacity; }
void resize(UInt32 newSize)
/// Resizes the hashtable, rehashes all existing entries. Expensive!
{
if (_maxCapacity != newSize)
{
HashTableVector cpy = _entries;
_entries = 0;
UInt32 oldSize = _maxCapacity;
_maxCapacity = newSize;
_entries = new HashEntryMap *[_maxCapacity];
memset(_entries, '\0', sizeof(HashEntryMap *) * _maxCapacity);
if (_size == 0)
{
// no data was yet inserted
delete[] cpy;
return;
}
_size = 0;
for (UInt32 i = 0; i < oldSize; ++i)
{
if (cpy[i])
{
ConstIterator it = cpy[i]->begin();
ConstIterator itEnd = cpy[i]->end();
for (; it != itEnd; ++it)
{
insert(it->first, it->second);
}
delete cpy[i];
}
}
delete[] cpy;
}
}
HashStatistic currentState(bool details = false) const
/// Returns the current internal state
{
UInt32 numberOfEntries = (UInt32)_size;
UInt32 numZeroEntries = 0;
UInt32 maxEntriesPerHash = 0;
std::vector<UInt32> detailedEntriesPerHash;
#ifdef _DEBUG
UInt32 totalSize = 0;
#endif
for (UInt32 i = 0; i < _maxCapacity; ++i)
{
if (_entries[i])
{
UInt32 size = (UInt32)_entries[i]->size();
poco_assert_dbg(size != 0);
if (size > maxEntriesPerHash)
maxEntriesPerHash = size;
if (details)
detailedEntriesPerHash.push_back(size);
#ifdef _DEBUG
totalSize += size;
#endif
}
else
{
numZeroEntries++;
if (details)
detailedEntriesPerHash.push_back(0);
}
}
#ifdef _DEBUG
poco_assert_dbg(totalSize == numberOfEntries);
#endif
return HashStatistic(_maxCapacity, numberOfEntries, numZeroEntries, maxEntriesPerHash, detailedEntriesPerHash);
}
private:
HashTableVector _entries;
std::size_t _size;
UInt32 _maxCapacity;
KeyHashFunction _hash;
};
} // namespace Poco
#endif // Foundation_HashTable_INCLUDED

52
base/poco/Foundation/include/Poco/Latin1Encoding.h
View File

@ -1,52 +0,0 @@
//
// Latin1Encoding.h
//
// Library: Foundation
// Package: Text
// Module: Latin1Encoding
//
// Definition of the Latin1Encoding class.
//
// Copyright (c) 2004-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_Latin1Encoding_INCLUDED
#define Foundation_Latin1Encoding_INCLUDED
#include "Poco/Foundation.h"
#include "Poco/TextEncoding.h"
namespace Poco
{
class Foundation_API Latin1Encoding : public TextEncoding
/// ISO Latin-1 (8859-1) text encoding.
{
public:
Latin1Encoding();
~Latin1Encoding();
const char * canonicalName() const;
bool isA(const std::string & encodingName) const;
const CharacterMap & characterMap() const;
int convert(const unsigned char * bytes) const;
int convert(int ch, unsigned char * bytes, int length) const;
int queryConvert(const unsigned char * bytes, int length) const;
int sequenceLength(const unsigned char * bytes, int length) const;
private:
static const char * _names[];
static const CharacterMap _charMap;
};
} // namespace Poco
#endif // Foundation_Latin1Encoding_INCLUDED

55
base/poco/Foundation/include/Poco/Latin2Encoding.h
View File

@ -1,55 +0,0 @@
//
// Latin2Encoding.h
//
// Library: Foundation
// Package: Text
// Module: Latin2Encoding
//
// Definition of the Latin2Encoding class.
//
// Copyright (c) 2004-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_Latin2Encoding_INCLUDED
#define Foundation_Latin2Encoding_INCLUDED
#include "Poco/Foundation.h"
#include "Poco/TextEncoding.h"
namespace Poco
{
class Foundation_API Latin2Encoding : public TextEncoding
/// ISO Latin-2 (8859-2) text encoding.
///
/// Latin-2 is basically Latin-1 with the EURO sign plus
/// some other minor changes.
{
public:
Latin2Encoding();
virtual ~Latin2Encoding();
const char * canonicalName() const;
bool isA(const std::string & encodingName) const;
const CharacterMap & characterMap() const;
int convert(const unsigned char * bytes) const;
int convert(int ch, unsigned char * bytes, int length) const;
int queryConvert(const unsigned char * bytes, int length) const;
int sequenceLength(const unsigned char * bytes, int length) const;
private:
static const char * _names[];
static const CharacterMap _charMap;
};
} // namespace Poco
#endif // Foundation_Latin2Encoding_INCLUDED

55
base/poco/Foundation/include/Poco/Latin9Encoding.h
View File

@ -1,55 +0,0 @@
//
// Latin9Encoding.h
//
// Library: Foundation
// Package: Text
// Module: Latin9Encoding
//
// Definition of the Latin9Encoding class.
//
// Copyright (c) 2004-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_Latin9Encoding_INCLUDED
#define Foundation_Latin9Encoding_INCLUDED
#include "Poco/Foundation.h"
#include "Poco/TextEncoding.h"
namespace Poco
{
class Foundation_API Latin9Encoding : public TextEncoding
/// ISO Latin-9 (8859-15) text encoding.
///
/// Latin-9 is basically Latin-1 with the EURO sign plus
/// some other minor changes.
{
public:
Latin9Encoding();
~Latin9Encoding();
const char * canonicalName() const;
bool isA(const std::string & encodingName) const;
const CharacterMap & characterMap() const;
int convert(const unsigned char * bytes) const;
int convert(int ch, unsigned char * bytes, int length) const;
int queryConvert(const unsigned char * bytes, int length) const;
int sequenceLength(const unsigned char * bytes, int length) const;
private:
static const char * _names[];
static const CharacterMap _charMap;
};
} // namespace Poco
#endif // Foundation_Latin9Encoding_INCLUDED

96
base/poco/Foundation/include/Poco/MD4Engine.h
View File

@ -1,96 +0,0 @@
//
// MD4Engine.h
//
// Library: Foundation
// Package: Crypt
// Module: MD4Engine
//
// Definition of class MD4Engine.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
//
// MD4 (RFC 1320) algorithm:
// Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
// rights reserved.
//
// License to copy and use this software is granted provided that it
// is identified as the "RSA Data Security, Inc. MD4 Message-Digest
// Algorithm" in all material mentioning or referencing this software
// or this function.
//
// License is also granted to make and use derivative works provided
// that such works are identified as "derived from the RSA Data
// Security, Inc. MD4 Message-Digest Algorithm" in all material
// mentioning or referencing the derived work.
//
// RSA Data Security, Inc. makes no representations concerning either
// the merchantability of this software or the suitability of this
// software for any particular purpose. It is provided "as is"
// without express or implied warranty of any kind.
//
// These notices must be retained in any copies of any part of this
// documentation and/or software.
//
#ifndef Foundation_MD4Engine_INCLUDED
#define Foundation_MD4Engine_INCLUDED
#include "Poco/DigestEngine.h"
#include "Poco/Foundation.h"
namespace Poco
{
class Foundation_API MD4Engine : public DigestEngine
/// This class implements the MD4 message digest algorithm,
/// described in RFC 1320.
{
public:
enum
{
BLOCK_SIZE = 64,
DIGEST_SIZE = 16
};
MD4Engine();
~MD4Engine();
std::size_t digestLength() const;
void reset();
const DigestEngine::Digest & digest();
protected:
void updateImpl(const void * data, std::size_t length);
private:
static void transform(UInt32 state[4], const unsigned char block[64]);
static void encode(unsigned char * output, const UInt32 * input, std::size_t len);
static void decode(UInt32 * output, const unsigned char * input, std::size_t len);
struct Context
{
UInt32 state[4]; // state (ABCD)
UInt32 count[2]; // number of bits, modulo 2^64 (lsb first)
unsigned char buffer[64]; // input buffer
};
Context _context;
DigestEngine::Digest _digest;
MD4Engine(const MD4Engine &);
MD4Engine & operator=(const MD4Engine &);
};
} // namespace Poco
#endif // Foundation_MD5Engine_INCLUDED

152
base/poco/Foundation/include/Poco/Manifest.h
View File

@ -1,152 +0,0 @@
//
// Manifest.h
//
// Library: Foundation
// Package: SharedLibrary
// Module: ClassLoader
//
// Definition of the Manifest class.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_Manifest_INCLUDED
#define Foundation_Manifest_INCLUDED
#include <map>
#include <typeinfo>
#include "Poco/Foundation.h"
#include "Poco/MetaObject.h"
namespace Poco
{
class Foundation_API ManifestBase
/// ManifestBase is a common base class for
/// all instantiations of Manifest.
{
public:
ManifestBase();
virtual ~ManifestBase();
virtual const char * className() const = 0;
/// Returns the type name of the manifest's class.
};
template <class B>
class Manifest : public ManifestBase
/// A Manifest maintains a list of all classes
/// contained in a dynamically loadable class
/// library.
/// Internally, the information is held
/// in a map. An iterator is provided to
/// iterate over all the classes in a Manifest.
{
public:
typedef AbstractMetaObject<B> Meta;
typedef std::map<std::string, const Meta *> MetaMap;
class Iterator
/// The Manifest's very own iterator class.
{
public:
Iterator(const typename MetaMap::const_iterator & it) { _it = it; }
Iterator(const Iterator & it) { _it = it._it; }
~Iterator() { }
Iterator & operator=(const Iterator & it)
{
_it = it._it;
return *this;
}
inline bool operator==(const Iterator & it) const { return _it == it._it; }
inline bool operator!=(const Iterator & it) const { return _it != it._it; }
Iterator & operator++() // prefix
{
++_it;
return *this;
}
Iterator operator++(int) // postfix
{
Iterator result(_it);
++_it;
return result;
}
inline const Meta * operator*() const { return _it->second; }
inline const Meta * operator->() const { return _it->second; }
private:
typename MetaMap::const_iterator _it;
};
Manifest()
/// Creates an empty Manifest.
{
}
virtual ~Manifest()
/// Destroys the Manifest.
{
clear();
}
Iterator find(const std::string & className) const
/// Returns an iterator pointing to the MetaObject
/// for the given class. If the MetaObject cannot
/// be found, the iterator points to end().
{
return Iterator(_metaMap.find(className));
}
Iterator begin() const { return Iterator(_metaMap.begin()); }
Iterator end() const { return Iterator(_metaMap.end()); }
bool insert(const Meta * pMeta)
/// Inserts a MetaObject. Returns true if insertion
/// was successful, false if a class with the same
/// name already exists.
{
return _metaMap.insert(typename MetaMap::value_type(pMeta->name(), pMeta)).second;
}
void clear()
/// Removes all MetaObjects from the manifest.
{
for (typename MetaMap::iterator it = _metaMap.begin(); it != _metaMap.end(); ++it)
{
delete it->second;
}
_metaMap.clear();
}
int size() const
/// Returns the number of MetaObjects in the Manifest.
{
return int(_metaMap.size());
}
bool empty() const
/// Returns true iff the Manifest does not contain any MetaObjects.
{
return _metaMap.empty();
}
const char * className() const { return typeid(*this).name(); }
private:
MetaMap _metaMap;
};
} // namespace Poco
#endif // Foundation_Manifest_INCLUDED

50
base/poco/Foundation/include/Poco/PipeImpl_DUMMY.h
View File

@ -1,50 +0,0 @@
//
// PipeImpl_DUMMY.h
//
// Library: Foundation
// Package: Processes
// Module: PipeImpl
//
// Definition of the PipeImpl_DUMMY class.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_PipeImpl_DUMMY_INCLUDED
#define Foundation_PipeImpl_DUMMY_INCLUDED
#include "Poco/Foundation.h"
#include "Poco/RefCountedObject.h"
namespace Poco
{
class Foundation_API PipeImpl : public RefCountedObject
/// A dummy implementation of PipeImpl for platforms
/// that do not support pipes.
{
public:
typedef int Handle;
PipeImpl();
~PipeImpl();
int writeBytes(const void * buffer, int length);
int readBytes(void * buffer, int length);
Handle readHandle() const;
Handle writeHandle() const;
void closeRead();
void closeWrite();
};
} // namespace Poco
#endif // Foundation_PipeImpl_DUMMY_INCLUDED

121
base/poco/Foundation/include/Poco/PipeStream.h
View File

@ -1,121 +0,0 @@
//
// PipeStream.h
//
// Library: Foundation
// Package: Processes
// Module: PipeStream
//
// Definition of the PipeStream class.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_PipeStream_INCLUDED
#define Foundation_PipeStream_INCLUDED
#include <istream>
#include <ostream>
#include "Poco/BufferedStreamBuf.h"
#include "Poco/Foundation.h"
#include "Poco/Pipe.h"
namespace Poco
{
class Foundation_API PipeStreamBuf : public BufferedStreamBuf
/// This is the streambuf class used for reading from and writing to a Pipe.
{
public:
typedef BufferedStreamBuf::openmode openmode;
PipeStreamBuf(const Pipe & pipe, openmode mode);
/// Creates a PipeStreamBuf with the given Pipe.
~PipeStreamBuf();
/// Destroys the PipeStreamBuf.
void close();
/// Closes the pipe.
protected:
int readFromDevice(char * buffer, std::streamsize length);
int writeToDevice(const char * buffer, std::streamsize length);
private:
enum
{
STREAM_BUFFER_SIZE = 1024
};
Pipe _pipe;
};
class Foundation_API PipeIOS : public virtual std::ios
/// The base class for PipeInputStream and
/// PipeOutputStream.
///
/// This class is needed to ensure the correct initialization
/// order of the stream buffer and base classes.
{
public:
PipeIOS(const Pipe & pipe, openmode mode);
/// Creates the PipeIOS with the given Pipe.
~PipeIOS();
/// Destroys the PipeIOS.
///
/// Flushes the buffer, but does not close the pipe.
PipeStreamBuf * rdbuf();
/// Returns a pointer to the internal PipeStreamBuf.
void close();
/// Flushes the stream and closes the pipe.
protected:
PipeStreamBuf _buf;
};
class Foundation_API PipeOutputStream : public PipeIOS, public std::ostream
/// An output stream for writing to a Pipe.
{
public:
PipeOutputStream(const Pipe & pipe);
/// Creates the PipeOutputStream with the given Pipe.
~PipeOutputStream();
/// Destroys the PipeOutputStream.
///
/// Flushes the buffer, but does not close the pipe.
};
class Foundation_API PipeInputStream : public PipeIOS, public std::istream
/// An input stream for reading from a Pipe.
///
/// Using formatted input from a PipeInputStream
/// is not recommended, due to the read-ahead behavior of
/// istream with formatted reads.
{
public:
PipeInputStream(const Pipe & pipe);
/// Creates the PipeInputStream with the given Pipe.
~PipeInputStream();
/// Destroys the PipeInputStream.
};
} // namespace Poco
#endif // Foundation_PipeStream_INCLUDED

89
base/poco/Foundation/include/Poco/SharedMemory_DUMMY.h
View File

@ -1,89 +0,0 @@
//
// SharedMemoryImpl.h
//
// Library: Foundation
// Package: Processes
// Module: SharedMemoryImpl
//
// Definition of the SharedMemoryImpl class.
//
// Copyright (c) 2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_SharedMemoryImpl_INCLUDED
#define Foundation_SharedMemoryImpl_INCLUDED
#include "Poco/Foundation.h"
#include "Poco/RefCountedObject.h"
#include "Poco/SharedMemory.h"
namespace Poco
{
class Foundation_API SharedMemoryImpl : public RefCountedObject
/// A dummy implementation of shared memory, for systems
/// that do not have shared memory support.
{
public:
SharedMemoryImpl(const std::string & id, std::size_t size, SharedMemory::AccessMode mode, const void * addr, bool server);
/// Creates or connects to a shared memory object with the given name.
///
/// For maximum portability, name should be a valid Unix filename and not
/// contain any slashes or backslashes.
///
/// An address hint can be passed to the system, specifying the desired
/// start address of the shared memory area. Whether the hint
/// is actually honored is, however, up to the system. Windows platform
/// will generally ignore the hint.
SharedMemoryImpl(const Poco::File & aFile, SharedMemory::AccessMode mode, const void * addr);
/// Maps the entire contents of file into a shared memory segment.
///
/// An address hint can be passed to the system, specifying the desired
/// start address of the shared memory area. Whether the hint
/// is actually honored is, however, up to the system. Windows platform
/// will generally ignore the hint.
char * begin() const;
/// Returns the start address of the shared memory segment.
char * end() const;
/// Returns the one-past-end end address of the shared memory segment.
protected:
~SharedMemoryImpl();
/// Destroys the SharedMemoryImpl.
private:
SharedMemoryImpl();
SharedMemoryImpl(const SharedMemoryImpl &);
SharedMemoryImpl & operator=(const SharedMemoryImpl &);
};
//
// inlines
//
inline char * SharedMemoryImpl::begin() const
{
return 0;
}
inline char * SharedMemoryImpl::end() const
{
return 0;
}
} // namespace Poco
#endif // Foundation_SharedMemoryImpl_INCLUDED

387
base/poco/Foundation/include/Poco/SimpleHashTable.h
View File

@ -1,387 +0,0 @@
//
// SimpleHashTable.h
//
// Library: Foundation
// Package: Hashing
// Module: SimpleHashTable
//
// Definition of the SimpleHashTable class.
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_SimpleHashTable_INCLUDED
#define Foundation_SimpleHashTable_INCLUDED
#include <algorithm>
#include <cstddef>
#include <map>
#include <vector>
#include "Poco/Exception.h"
#include "Poco/Foundation.h"
#include "Poco/HashFunction.h"
#include "Poco/HashStatistic.h"
namespace Poco
{
//@ deprecated
template <class Key, class Value, class KeyHashFunction = HashFunction<Key>>
class SimpleHashTable
/// A SimpleHashTable stores a key value pair that can be looked up via a hashed key.
///
/// In comparison to a HashTable, this class handles collisions by sequentially searching the next
/// free location. This also means that the maximum size of this table is limited, i.e. if the hash table
/// is full, it will throw an exception and that this class does not support remove operations.
/// On the plus side it is faster than the HashTable.
///
/// This class is NOT thread safe.
{
public:
class HashEntry
{
public:
Key key;
Value value;
HashEntry(const Key k, const Value v) : key(k), value(v) { }
};
typedef std::vector<HashEntry *> HashTableVector;
SimpleHashTable(UInt32 capacity = 251) : _entries(capacity, 0), _size(0), _capacity(capacity)
/// Creates the SimpleHashTable.
{
}
SimpleHashTable(const SimpleHashTable & ht) : _size(ht._size), _capacity(ht._capacity)
{
_entries.reserve(ht._capacity);
for (typename HashTableVector::iterator it = ht._entries.begin(); it != ht._entries.end(); ++it)
{
if (*it)
_entries.push_back(new HashEntry(*it));
else
_entries.push_back(0);
}
}
~SimpleHashTable()
/// Destroys the SimpleHashTable.
{
clear();
}
SimpleHashTable & operator=(const SimpleHashTable & ht)
{
if (this != &ht)
{
SimpleHashTable tmp(ht);
swap(tmp);
}
return *this;
}
void swap(SimpleHashTable & ht)
{
using std::swap;
swap(_entries, ht._entries);
swap(_size, ht._size);
swap(_capacity, ht._capacity);
}
void clear()
{
for (typename HashTableVector::iterator it = _entries.begin(); it != _entries.end(); ++it)
{
delete *it;
*it = 0;
}
_size = 0;
}
UInt32 insert(const Key & key, const Value & value)
/// Returns the hash value of the inserted item.
/// Throws an exception if the entry was already inserted
{
UInt32 hsh = hash(key);
insertRaw(key, hsh, value);
return hsh;
}
Value & insertRaw(const Key & key, UInt32 hsh, const Value & value)
/// Returns the hash value of the inserted item.
/// Throws an exception if the entry was already inserted
{
UInt32 pos = hsh;
if (!_entries[pos])
_entries[pos] = new HashEntry(key, value);
else
{
UInt32 origHash = hsh;
while (_entries[hsh % _capacity])
{
if (_entries[hsh % _capacity]->key == key)
throw ExistsException();
if (hsh - origHash > _capacity)
throw PoolOverflowException("SimpleHashTable full");
hsh++;
}
pos = hsh % _capacity;
_entries[pos] = new HashEntry(key, value);
}
_size++;
return _entries[pos]->value;
}
UInt32 update(const Key & key, const Value & value)
/// Returns the hash value of the inserted item.
/// Replaces an existing entry if it finds one
{
UInt32 hsh = hash(key);
updateRaw(key, hsh, value);
return hsh;
}
void updateRaw(const Key & key, UInt32 hsh, const Value & value)
/// Returns the hash value of the inserted item.
/// Replaces an existing entry if it finds one
{
if (!_entries[hsh])
_entries[hsh] = new HashEntry(key, value);
else
{
UInt32 origHash = hsh;
while (_entries[hsh % _capacity])
{
if (_entries[hsh % _capacity]->key == key)
{
_entries[hsh % _capacity]->value = value;
return;
}
if (hsh - origHash > _capacity)
throw PoolOverflowException("SimpleHashTable full");
hsh++;
}
_entries[hsh % _capacity] = new HashEntry(key, value);
}
_size++;
}
UInt32 hash(const Key & key) const { return _hash(key, _capacity); }
const Value & get(const Key & key) const
/// Throws an exception if the value does not exist
{
UInt32 hsh = hash(key);
return getRaw(key, hsh);
}
const Value & getRaw(const Key & key, UInt32 hsh) const
/// Throws an exception if the value does not exist
{
UInt32 origHash = hsh;
while (true)
{
if (_entries[hsh % _capacity])
{
if (_entries[hsh % _capacity]->key == key)
{
return _entries[hsh % _capacity]->value;
}
}
else
throw InvalidArgumentException("value not found");
if (hsh - origHash > _capacity)
throw InvalidArgumentException("value not found");
hsh++;
}
}
Value & get(const Key & key)
/// Throws an exception if the value does not exist
{
UInt32 hsh = hash(key);
return const_cast<Value &>(getRaw(key, hsh));
}
const Value & operator[](const Key & key) const { return get(key); }
Value & operator[](const Key & key)
{
UInt32 hsh = hash(key);
UInt32 origHash = hsh;
while (true)
{
if (_entries[hsh % _capacity])
{
if (_entries[hsh % _capacity]->key == key)
{
return _entries[hsh % _capacity]->value;
}
}
else
return insertRaw(key, hsh, Value());
if (hsh - origHash > _capacity)
return insertRaw(key, hsh, Value());
hsh++;
}
}
const Key & getKeyRaw(const Key & key, UInt32 hsh)
/// Throws an exception if the key does not exist. returns a reference to the internally
/// stored key. Useful when someone does an insert and wants for performance reason only to store
/// a pointer to the key in another collection
{
UInt32 origHash = hsh;
while (true)
{
if (_entries[hsh % _capacity])
{
if (_entries[hsh % _capacity]->key == key)
{
return _entries[hsh % _capacity]->key;
}
}
else
throw InvalidArgumentException("key not found");
if (hsh - origHash > _capacity)
throw InvalidArgumentException("key not found");
hsh++;
}
}
bool get(const Key & key, Value & v) const
/// Sets v to the found value, returns false if no value was found
{
UInt32 hsh = hash(key);
return getRaw(key, hsh, v);
}
bool getRaw(const Key & key, UInt32 hsh, Value & v) const
/// Sets v to the found value, returns false if no value was found
{
UInt32 origHash = hsh;
while (true)
{
if (_entries[hsh % _capacity])
{
if (_entries[hsh % _capacity]->key == key)
{
v = _entries[hsh % _capacity]->value;
return true;
}
}
else
return false;
if (hsh - origHash > _capacity)
return false;
hsh++;
}
}
bool exists(const Key & key) const
{
UInt32 hsh = hash(key);
return existsRaw(key, hsh);
}
bool existsRaw(const Key & key, UInt32 hsh) const
{
UInt32 origHash = hsh;
while (true)
{
if (_entries[hsh % _capacity])
{
if (_entries[hsh % _capacity]->key == key)
{
return true;
}
}
else
return false;
if (hsh - origHash > _capacity)
return false;
hsh++;
}
}
std::size_t size() const
/// Returns the number of elements already inserted into the SimpleHashTable
{
return _size;
}
UInt32 capacity() const { return _capacity; }
void resize(UInt32 newSize)
/// Resizes the hashtable, rehashes all existing entries. Expensive!
{
if (_capacity != newSize)
{
SimpleHashTable tmp(newSize);
swap(tmp);
for (typename HashTableVector::const_iterator it = tmp._entries.begin(); it != tmp._entries.end(); ++it)
{
if (*it)
{
insertRaw((*it)->key, hash((*it)->key), (*it)->value);
}
}
}
}
HashStatistic currentState(bool details = false) const
/// Returns the current internal state
{
UInt32 numberOfEntries = (UInt32)_size;
UInt32 numZeroEntries = 0;
UInt32 maxEntriesPerHash = 0;
std::vector<UInt32> detailedEntriesPerHash;
#ifdef _DEBUG
UInt32 totalSize = 0;
#endif
for (int i = 0; i < _capacity; ++i)
{
if (_entries[i])
{
maxEntriesPerHash = 1;
UInt32 size = 1;
if (details)
detailedEntriesPerHash.push_back(size);
#ifdef _DEBUG
totalSize += size;
#endif
}
else
{
numZeroEntries++;
if (details)
detailedEntriesPerHash.push_back(0);
}
}
#ifdef _DEBUG
poco_assert_dbg(totalSize == numberOfEntries);
#endif
return HashStatistic(_capacity, numberOfEntries, numZeroEntries, maxEntriesPerHash, detailedEntriesPerHash);
}
private:
HashTableVector _entries;
std::size_t _size;
UInt32 _capacity;
KeyHashFunction _hash;
};
} // namespace Poco
#endif // Foundation_HashTable_INCLUDED

98
base/poco/Foundation/include/Poco/StreamTokenizer.h
View File

@ -1,98 +0,0 @@
//
// StreamTokenizer.h
//
// Library: Foundation
// Package: Streams
// Module: StreamTokenizer
//
// Definition of the StreamTokenizer class.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_StreamTokenizer_INCLUDED
#define Foundation_StreamTokenizer_INCLUDED
#include <istream>
#include <vector>
#include "Poco/Foundation.h"
#include "Poco/Token.h"
namespace Poco
{
class Foundation_API StreamTokenizer
/// A stream tokenizer splits an input stream
/// into a sequence of tokens of different kinds.
/// Various token kinds can be registered with
/// the tokenizer.
{
public:
StreamTokenizer();
/// Creates a StreamTokenizer with no attached stream.
StreamTokenizer(std::istream & istr);
/// Creates a StreamTokenizer with no attached stream.
virtual ~StreamTokenizer();
/// Destroys the StreamTokenizer and deletes all
/// registered tokens.
void attachToStream(std::istream & istr);
/// Attaches the tokenizer to an input stream.
void addToken(Token * pToken);
/// Adds a token class to the tokenizer. The
/// tokenizer takes ownership of the token and
/// deletes it when no longer needed. Comment
/// and whitespace tokens will be marked as
/// ignorable, which means that next() will not
/// return them.
void addToken(Token * pToken, bool ignore);
/// Adds a token class to the tokenizer. The
/// tokenizer takes ownership of the token and
/// deletes it when no longer needed.
/// If ignore is true, the token will be marked
/// as ignorable, which means that next() will
/// not return it.
const Token * next();
/// Extracts the next token from the input stream.
/// Returns a pointer to an EOFToken if there are
/// no more characters to read.
/// Returns a pointer to an InvalidToken if an
/// invalid character is encountered.
/// If a token is marked as ignorable, it will not
/// be returned, and the next token will be
/// examined.
/// Never returns a NULL pointer.
/// You must not delete the token returned by next().
private:
struct TokenInfo
{
Token * pToken;
bool ignore;
};
typedef std::vector<TokenInfo> TokenVec;
TokenVec _tokens;
std::istream * _pIstr;
InvalidToken _invalidToken;
EOFToken _eofToken;
};
} // namespace Poco
#endif // Foundation_StreamTokenizer_INCLUDED

132
base/poco/Foundation/include/Poco/SynchronizedObject.h
View File

@ -1,132 +0,0 @@
//
// SynchronizedObject.h
//
// Library: Foundation
// Package: Threading
// Module: SynchronizedObject
//
// Definition of the SynchronizedObject class.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_SynchronizedObject_INCLUDED
#define Foundation_SynchronizedObject_INCLUDED
#include "Poco/Event.h"
#include "Poco/Foundation.h"
#include "Poco/Mutex.h"
namespace Poco
{
class Foundation_API SynchronizedObject
/// This class aggregates a Mutex and an Event
/// and can act as a base class for all objects
/// requiring synchronization in a multithreaded
/// scenario.
{
public:
typedef Poco::ScopedLock<SynchronizedObject> ScopedLock;
SynchronizedObject();
/// Creates the object.
virtual ~SynchronizedObject();
/// Destroys the object.
void lock() const;
/// Locks the object. Blocks if the object
/// is locked by another thread.
bool tryLock() const;
/// Tries to lock the object. Returns false immediately
/// if the object is already locked by another thread
/// Returns true if the object was successfully locked.
void unlock() const;
/// Unlocks the object so that it can be locked by
/// other threads.
void notify() const;
/// Signals the object.
/// Exactly only one thread waiting for the object
/// can resume execution.
void wait() const;
/// Waits for the object to become signalled.
void wait(long milliseconds) const;
/// Waits for the object to become signalled.
/// Throws a TimeoutException if the object
/// does not become signalled within the specified
/// time interval.
bool tryWait(long milliseconds) const;
/// Waits for the object to become signalled.
/// Returns true if the object
/// became signalled within the specified
/// time interval, false otherwise.
private:
mutable Mutex _mutex;
mutable Event _event;
};
//
// inlines
//
inline void SynchronizedObject::lock() const
{
_mutex.lock();
}
inline bool SynchronizedObject::tryLock() const
{
return _mutex.tryLock();
}
inline void SynchronizedObject::unlock() const
{
_mutex.unlock();
}
inline void SynchronizedObject::notify() const
{
_event.set();
}
inline void SynchronizedObject::wait() const
{
_event.wait();
}
inline void SynchronizedObject::wait(long milliseconds) const
{
_event.wait(milliseconds);
}
inline bool SynchronizedObject::tryWait(long milliseconds) const
{
return _event.tryWait(milliseconds);
}
} // namespace Poco
#endif // Foundation_SynchronizedObject_INCLUDED

135
base/poco/Foundation/include/Poco/UnWindows.h
View File

@ -1,135 +0,0 @@
//
// UnWindows.h
//
// Library: Foundation
// Package: Core
// Module: UnWindows
//
// A wrapper around the <windows.h> header file that #undef's some
// of the macros for function names defined by <windows.h> that
// are a frequent source of conflicts (e.g., GetUserName).
//
// Remember, that most of the WIN32 API functions come in two variants,
// an Unicode variant (e.g., GetUserNameA) and an ASCII variant (GetUserNameW).
// There is also a macro (GetUserName) that's either defined to be the Unicode
// name or the ASCII name, depending on whether the UNICODE macro is #define'd
// or not. POCO always calls the Unicode or ASCII functions directly (depending
// on whether POCO_WIN32_UTF8 is #define'd or not), so the macros are not ignored.
//
// These macro definitions are a frequent case of problems and naming conflicts,
// especially for C++ programmers. Say, you define a class with a member function named
// GetUserName. Depending on whether "Poco/UnWindows.h" has been included by a particular
// translation unit or not, this might be changed to GetUserNameA/GetUserNameW, or not.
// While, due to naming conventions used, this is less of a problem in POCO, some
// of the users of POCO might use a different naming convention where this can become
// a problem.
//
// To disable the #undef's, compile POCO with the POCO_NO_UNWINDOWS macro #define'd.
//
// Copyright (c) 2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_UnWindows_INCLUDED
#define Foundation_UnWindows_INCLUDED
// Reduce bloat
// Microsoft Visual C++ includes copies of the Windows header files
// that were current at the time Visual C++ was released.
// The Windows header files use macros to indicate which versions
// of Windows support many programming elements. Therefore, you must
// define these macros to use new functionality introduced in each
// major operating system release. (Individual header files may use
// different macros; therefore, if compilation problems occur, check
// the header file that contains the definition for conditional
// definitions.) For more information, see SdkDdkVer.h.
# if defined(_WIN32_WINNT)
# if (_WIN32_WINNT < 0x0502)
# error Unsupported Windows version.
# endif
# elif defined(NTDDI_VERSION)
# if (NTDDI_VERSION < 0x05020000)
# error Unsupported Windows version.
# endif
# elif !defined(_WIN32_WINNT)
// Define minimum supported version.
// This can be changed, if needed.
// If allowed (see POCO_MIN_WINDOWS_OS_SUPPORT
// below), Platform_WIN32.h will do its
// best to determine the appropriate values
// and may redefine these. See Platform_WIN32.h
// for details.
# define _WIN32_WINNT 0x0502
# define NTDDI_VERSION 0x05020000
# endif
// To prevent Platform_WIN32.h to modify version defines,
// uncomment this, otherwise versions will be automatically
// discovered in Platform_WIN32.h.
// #define POCO_FORCE_MIN_WINDOWS_OS_SUPPORT
#include <windows.h>
#if !defined(POCO_NO_UNWINDOWS)
// A list of annoying macros to #undef.
// Extend as required.
# undef GetBinaryType
# undef GetShortPathName
# undef GetLongPathName
# undef GetEnvironmentStrings
# undef SetEnvironmentStrings
# undef FreeEnvironmentStrings
# undef FormatMessage
# undef EncryptFile
# undef DecryptFile
# undef CreateMutex
# undef OpenMutex
# undef CreateEvent
# undef OpenEvent
# undef CreateSemaphore
# undef OpenSemaphore
# undef LoadLibrary
# undef GetModuleFileName
# undef CreateProcess
# undef GetCommandLine
# undef GetEnvironmentVariable
# undef SetEnvironmentVariable
# undef ExpandEnvironmentStrings
# undef OutputDebugString
# undef FindResource
# undef UpdateResource
# undef FindAtom
# undef AddAtom
# undef GetSystemDirectory
# undef GetTempPath
# undef GetTempFileName
# undef SetCurrentDirectory
# undef GetCurrentDirectory
# undef CreateDirectory
# undef RemoveDirectory
# undef CreateFile
# undef DeleteFile
# undef SearchPath
# undef CopyFile
# undef MoveFile
# undef ReplaceFile
# undef GetComputerName
# undef SetComputerName
# undef GetUserName
# undef LogonUser
# undef GetVersion
# undef GetObject
#endif // POCO_NO_UNWINDOWS
#endif // Foundation_UnWindows_INCLUDED

53
base/poco/Foundation/include/Poco/Windows1250Encoding.h
View File

@ -1,53 +0,0 @@
//
// Windows1250Encoding.h
//
// Library: Foundation
// Package: Text
// Module: Windows1250Encoding
//
// Definition of the Windows1250Encoding class.
//
// Copyright (c) 2005-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_Windows1250Encoding_INCLUDED
#define Foundation_Windows1250Encoding_INCLUDED
#include "Poco/Foundation.h"
#include "Poco/TextEncoding.h"
namespace Poco
{
class Foundation_API Windows1250Encoding : public TextEncoding
/// Windows Codepage 1250 text encoding.
/// Based on: http://msdn.microsoft.com/en-us/goglobal/cc305143
{
public:
Windows1250Encoding();
~Windows1250Encoding();
const char * canonicalName() const;
bool isA(const std::string & encodingName) const;
const CharacterMap & characterMap() const;
int convert(const unsigned char * bytes) const;
int convert(int ch, unsigned char * bytes, int length) const;
int queryConvert(const unsigned char * bytes, int length) const;
int sequenceLength(const unsigned char * bytes, int length) const;
private:
static const char * _names[];
static const CharacterMap _charMap;
};
} // namespace Poco
#endif // Foundation_Windows1250Encoding_INCLUDED

53
base/poco/Foundation/include/Poco/Windows1251Encoding.h
View File

@ -1,53 +0,0 @@
//
// Windows1251Encoding.h
//
// Library: Foundation
// Package: Text
// Module: Windows1251Encoding
//
// Definition of the Windows1251Encoding class.
//
// Copyright (c) 2005-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_Windows1251Encoding_INCLUDED
#define Foundation_Windows1251Encoding_INCLUDED
#include "Poco/Foundation.h"
#include "Poco/TextEncoding.h"
namespace Poco
{
class Foundation_API Windows1251Encoding : public TextEncoding
/// Windows Codepage 1251 text encoding.
/// Based on: http://msdn.microsoft.com/en-us/goglobal/cc305144
{
public:
Windows1251Encoding();
~Windows1251Encoding();
const char * canonicalName() const;
bool isA(const std::string & encodingName) const;
const CharacterMap & characterMap() const;
int convert(const unsigned char * bytes) const;
int convert(int ch, unsigned char * bytes, int length) const;
int queryConvert(const unsigned char * bytes, int length) const;
int sequenceLength(const unsigned char * bytes, int length) const;
private:
static const char * _names[];
static const CharacterMap _charMap;
};
} // namespace Poco
#endif // Foundation_Windows1251Encoding_INCLUDED

52
base/poco/Foundation/include/Poco/Windows1252Encoding.h
View File

@ -1,52 +0,0 @@
//
// Windows1252Encoding.h
//
// Library: Foundation
// Package: Text
// Module: Windows1252Encoding
//
// Definition of the Windows1252Encoding class.
//
// Copyright (c) 2005-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_Windows1252Encoding_INCLUDED
#define Foundation_Windows1252Encoding_INCLUDED
#include "Poco/Foundation.h"
#include "Poco/TextEncoding.h"
namespace Poco
{
class Foundation_API Windows1252Encoding : public TextEncoding
/// Windows Codepage 1252 text encoding.
{
public:
Windows1252Encoding();
~Windows1252Encoding();
const char * canonicalName() const;
bool isA(const std::string & encodingName) const;
const CharacterMap & characterMap() const;
int convert(const unsigned char * bytes) const;
int convert(int ch, unsigned char * bytes, int length) const;
int queryConvert(const unsigned char * bytes, int length) const;
int sequenceLength(const unsigned char * bytes, int length) const;
private:
static const char * _names[];
static const CharacterMap _charMap;
};
} // namespace Poco
#endif // Foundation_Windows1252Encoding_INCLUDED

184
base/poco/Foundation/include/Poco/WindowsConsoleChannel.h
View File

@ -1,184 +0,0 @@
//
// WindowsConsoleChannel.h
//
// Library: Foundation
// Package: Logging
// Module: WindowsConsoleChannel
//
// Definition of the WindowsConsoleChannel class.
//
// Copyright (c) 2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Foundation_WindowsConsoleChannel_INCLUDED
#define Foundation_WindowsConsoleChannel_INCLUDED
#include "Poco/Channel.h"
#include "Poco/Foundation.h"
#include "Poco/Mutex.h"
#include "Poco/UnWindows.h"
namespace Poco
{
class Foundation_API WindowsConsoleChannel : public Channel
/// A channel that writes to the Windows console.
///
/// Only the message's text is written, followed
/// by a newline.
///
/// If POCO has been compiled with POCO_WIN32_UTF8,
/// log messages are assumed to be UTF-8 encoded, and
/// are converted to UTF-16 prior to writing them to the
/// console. This is the main difference to the ConsoleChannel
/// class, which cannot handle UTF-8 encoded messages on Windows.
///
/// Chain this channel to a FormattingChannel with an
/// appropriate Formatter to control what is contained
/// in the text.
///
/// Only available on Windows platforms.
{
public:
WindowsConsoleChannel();
/// Creates the WindowsConsoleChannel.
void log(const Message & msg);
/// Logs the given message to the channel's stream.
protected:
~WindowsConsoleChannel();
private:
HANDLE _hConsole;
bool _isFile;
};
class Foundation_API WindowsColorConsoleChannel : public Channel
/// A channel that writes to the Windows console.
///
/// Only the message's text is written, followed
/// by a newline.
///
/// If POCO has been compiled with POCO_WIN32_UTF8,
/// log messages are assumed to be UTF-8 encoded, and
/// are converted to UTF-16 prior to writing them to the
/// console. This is the main difference to the ConsoleChannel
/// class, which cannot handle UTF-8 encoded messages on Windows.
///
/// Messages can be colored depending on priority.
///
/// To enable message coloring, set the "enableColors"
/// property to true (default). Furthermore, colors can be
/// configured by setting the following properties
/// (default values are given in parenthesis):
///
/// * traceColor (gray)
/// * debugColor (gray)
/// * informationColor (default)
/// * noticeColor (default)
/// * warningColor (yellow)
/// * errorColor (lightRed)
/// * criticalColor (lightRed)
/// * fatalColor (lightRed)
///
/// The following color values are supported:
///
/// * default
/// * black
/// * red
/// * green
/// * brown
/// * blue
/// * magenta
/// * cyan
/// * gray
/// * darkgray
/// * lightRed
/// * lightGreen
/// * yellow
/// * lightBlue
/// * lightMagenta
/// * lightCyan
/// * white
///
/// Chain this channel to a FormattingChannel with an
/// appropriate Formatter to control what is contained
/// in the text.
///
/// Only available on Windows platforms.
{
public:
WindowsColorConsoleChannel();
/// Creates the WindowsConsoleChannel.
void log(const Message & msg);
/// Logs the given message to the channel's stream.
void setProperty(const std::string & name, const std::string & value);
/// Sets the property with the given name.
///
/// The following properties are supported:
/// * enableColors: Enable or disable colors.
/// * traceColor: Specify color for trace messages.
/// * debugColor: Specify color for debug messages.
/// * informationColor: Specify color for information messages.
/// * noticeColor: Specify color for notice messages.
/// * warningColor: Specify color for warning messages.
/// * errorColor: Specify color for error messages.
/// * criticalColor: Specify color for critical messages.
/// * fatalColor: Specify color for fatal messages.
///
/// See the class documentation for a list of supported color values.
std::string getProperty(const std::string & name) const;
/// Returns the value of the property with the given name.
/// See setProperty() for a description of the supported
/// properties.
protected:
enum Color
{
CC_BLACK = 0x0000,
CC_RED = 0x0004,
CC_GREEN = 0x0002,
CC_BROWN = 0x0006,
CC_BLUE = 0x0001,
CC_MAGENTA = 0x0005,
CC_CYAN = 0x0003,
CC_GRAY = 0x0007,
CC_DARKGRAY = 0x0008,
CC_LIGHTRED = 0x000C,
CC_LIGHTGREEN = 0x000A,
CC_YELLOW = 0x000E,
CC_LIGHTBLUE = 0x0009,
CC_LIGHTMAGENTA = 0x000D,
CC_LIGHTCYAN = 0x000B,
CC_WHITE = 0x000F
};
~WindowsColorConsoleChannel();
WORD parseColor(const std::string & color) const;
std::string formatColor(WORD color) const;
void initColors();
private:
bool _enableColors;
HANDLE _hConsole;
bool _isFile;
WORD _colors[9];
};
} // namespace Poco
#endif // Foundation_WindowsConsoleChannel_INCLUDED

160
base/poco/Foundation/src/Base32Decoder.cpp
View File

@ -1,160 +0,0 @@
//
// Base32Decoder.cpp
//
// Library: Foundation
// Package: Streams
// Module: Base32
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Base32Decoder.h"
#include "Poco/Base32Encoder.h"
#include "Poco/Exception.h"
#include "Poco/Mutex.h"
#include <cstring>
namespace Poco {
unsigned char Base32DecoderBuf::IN_ENCODING[256];
bool Base32DecoderBuf::IN_ENCODING_INIT = false;
namespace
{
static FastMutex mutex;
}
Base32DecoderBuf::Base32DecoderBuf(std::istream& istr):
_groupLength(0),
_groupIndex(0),
_buf(*istr.rdbuf())
{
FastMutex::ScopedLock lock(mutex);
if (!IN_ENCODING_INIT)
{
for (unsigned i = 0; i < sizeof(IN_ENCODING); i++)
{
IN_ENCODING[i] = 0xFF;
}
for (unsigned i = 0; i < sizeof(Base32EncoderBuf::OUT_ENCODING); i++)
{
IN_ENCODING[Base32EncoderBuf::OUT_ENCODING[i]] = i;
}
IN_ENCODING[static_cast<unsigned char>('=')] = '\0';
IN_ENCODING_INIT = true;
}
}
Base32DecoderBuf::~Base32DecoderBuf()
{
}
int Base32DecoderBuf::readFromDevice()
{
if (_groupIndex < _groupLength)
{
return _group[_groupIndex++];
}
else
{
unsigned char buffer[8];
std::memset(buffer, '=', sizeof(buffer));
int c;
// per RFC-4648, Section 6, permissible block lengths are:
// 2, 4, 5, 7, and 8 bytes. Any other length is malformed.
//
do {
if ((c = readOne()) == -1) return -1;
buffer[0] = (unsigned char) c;
if (IN_ENCODING[buffer[0]] == 0xFF) throw DataFormatException();
if ((c = readOne()) == -1) throw DataFormatException();
buffer[1] = (unsigned char) c;
if (IN_ENCODING[buffer[1]] == 0xFF) throw DataFormatException();
if ((c = readOne()) == -1) break;
buffer[2] = (unsigned char) c;
if (IN_ENCODING[buffer[2]] == 0xFF) throw DataFormatException();
if ((c = readOne()) == -1) throw DataFormatException();
buffer[3] = (unsigned char) c;
if (IN_ENCODING[buffer[3]] == 0xFF) throw DataFormatException();
if ((c = readOne()) == -1) break;
buffer[4] = (unsigned char) c;
if (IN_ENCODING[buffer[4]] == 0xFF) throw DataFormatException();
if ((c = readOne()) == -1) break;
buffer[5] = (unsigned char) c;
if (IN_ENCODING[buffer[5]] == 0xFF) throw DataFormatException();
if ((c = readOne()) == -1) throw DataFormatException();
buffer[6] = (unsigned char) c;
if (IN_ENCODING[buffer[6]] == 0xFF) throw DataFormatException();
if ((c = readOne()) == -1) break;
buffer[7] = (unsigned char) c;
if (IN_ENCODING[buffer[7]] == 0xFF) throw DataFormatException();
} while (false);
_group[0] = (IN_ENCODING[buffer[0]] << 3) | (IN_ENCODING[buffer[1]] >> 2);
_group[1] = ((IN_ENCODING[buffer[1]] & 0x03) << 6) | (IN_ENCODING[buffer[2]] << 1) | (IN_ENCODING[buffer[3]] >> 4);
_group[2] = ((IN_ENCODING[buffer[3]] & 0x0F) << 4) | (IN_ENCODING[buffer[4]] >> 1);
_group[3] = ((IN_ENCODING[buffer[4]] & 0x01) << 7) | (IN_ENCODING[buffer[5]] << 2) | (IN_ENCODING[buffer[6]] >> 3);
_group[4] = ((IN_ENCODING[buffer[6]] & 0x07) << 5) | IN_ENCODING[buffer[7]];
if (buffer[2] == '=')
_groupLength = 1;
else if (buffer[4] == '=')
_groupLength = 2;
else if (buffer[5] == '=')
_groupLength = 3;
else if (buffer[7] == '=')
_groupLength = 4;
else
_groupLength = 5;
_groupIndex = 1;
return _group[0];
}
}
int Base32DecoderBuf::readOne()
{
int ch = _buf.sbumpc();
return ch;
}
Base32DecoderIOS::Base32DecoderIOS(std::istream& istr): _buf(istr)
{
poco_ios_init(&_buf);
}
Base32DecoderIOS::~Base32DecoderIOS()
{
}
Base32DecoderBuf* Base32DecoderIOS::rdbuf()
{
return &_buf;
}
Base32Decoder::Base32Decoder(std::istream& istr): Base32DecoderIOS(istr), std::istream(&_buf)
{
}
Base32Decoder::~Base32Decoder()
{
}
} // namespace Poco

202
base/poco/Foundation/src/Base32Encoder.cpp
View File

@ -1,202 +0,0 @@
//
// Base32Encoder.cpp
//
// Library: Foundation
// Package: Streams
// Module: Base32
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Base32Encoder.h"
namespace Poco {
const unsigned char Base32EncoderBuf::OUT_ENCODING[32] =
{
'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H',
'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P',
'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X',
'Y', 'Z', '2', '3', '4', '5', '6', '7',
};
Base32EncoderBuf::Base32EncoderBuf(std::ostream& ostr, bool padding):
_groupLength(0),
_buf(*ostr.rdbuf()),
_doPadding(padding)
{
}
Base32EncoderBuf::~Base32EncoderBuf()
{
try
{
close();
}
catch (...)
{
}
}
int Base32EncoderBuf::writeToDevice(char c)
{
static const int eof = std::char_traits<char>::eof();
_group[_groupLength++] = (unsigned char) c;
if (_groupLength == 5)
{
unsigned char idx;
idx = _group[0] >> 3;
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[0] & 0x07) << 2) | (_group[1] >> 6);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[1] & 0x3E) >> 1);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[1] & 0x01) << 4) | (_group[2] >> 4);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[2] & 0x0F) << 1) | (_group[3] >> 7);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[3] & 0x7C) >> 2);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[3] & 0x03) << 3) | (_group[4] >> 5);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = (_group[4] & 0x1F);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
_groupLength = 0;
}
return charToInt(c);
}
int Base32EncoderBuf::close()
{
static const int eof = std::char_traits<char>::eof();
if (sync() == eof) return eof;
if (_groupLength == 1)
{
_group[1] = 0;
unsigned char idx;
idx = _group[0] >> 3;
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[0] & 0x07) << 2);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
if (_doPadding) {
if (_buf.sputc('=') == eof) return eof;
if (_buf.sputc('=') == eof) return eof;
if (_buf.sputc('=') == eof) return eof;
if (_buf.sputc('=') == eof) return eof;
if (_buf.sputc('=') == eof) return eof;
if (_buf.sputc('=') == eof) return eof;
}
}
else if (_groupLength == 2)
{
_group[2] = 0;
unsigned char idx;
idx = _group[0] >> 3;
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[0] & 0x07) << 2) | (_group[1] >> 6);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[1] & 0x3E) >> 1);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[1] & 0x01) << 4);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
if (_doPadding) {
if (_buf.sputc('=') == eof) return eof;
if (_buf.sputc('=') == eof) return eof;
if (_buf.sputc('=') == eof) return eof;
if (_buf.sputc('=') == eof) return eof;
}
}
else if (_groupLength == 3)
{
_group[3] = 0;
unsigned char idx;
idx = _group[0] >> 3;
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[0] & 0x07) << 2) | (_group[1] >> 6);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[1] & 0x3E) >> 1);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[1] & 0x01) << 4) | (_group[2] >> 4);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[2] & 0x0F) << 1);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
if (_doPadding) {
if (_buf.sputc('=') == eof) return eof;
if (_buf.sputc('=') == eof) return eof;
if (_buf.sputc('=') == eof) return eof;
}
}
else if (_groupLength == 4)
{
_group[4] = 0;
unsigned char idx;
idx = _group[0] >> 3;
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[0] & 0x07) << 2) | (_group[1] >> 6);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[1] & 0x3E) >> 1);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[1] & 0x01) << 4) | (_group[2] >> 4);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[2] & 0x0F) << 1) | (_group[3] >> 7);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[3] & 0x7C) >> 2);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
idx = ((_group[3] & 0x03) << 3);
if (_buf.sputc(OUT_ENCODING[idx]) == eof) return eof;
if (_doPadding && _buf.sputc('=') == eof) return eof;
}
_groupLength = 0;
return _buf.pubsync();
}
Base32EncoderIOS::Base32EncoderIOS(std::ostream& ostr, bool padding):
_buf(ostr, padding)
{
poco_ios_init(&_buf);
}
Base32EncoderIOS::~Base32EncoderIOS()
{
}
int Base32EncoderIOS::close()
{
return _buf.close();
}
Base32EncoderBuf* Base32EncoderIOS::rdbuf()
{
return &_buf;
}
Base32Encoder::Base32Encoder(std::ostream& ostr, bool padding):
Base32EncoderIOS(ostr, padding), std::ostream(&_buf)
{
}
Base32Encoder::~Base32Encoder()
{
}
} // namespace Poco

221
base/poco/Foundation/src/EventLogChannel.cpp
View File

@ -1,221 +0,0 @@
//
// EventLogChannel.cpp
//
// Library: Foundation
// Package: Logging
// Module: EventLogChannel
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/EventLogChannel.h"
#include "Poco/Message.h"
#include "Poco/String.h"
#include "pocomsg.h"
namespace Poco {
const std::string EventLogChannel::PROP_NAME = "name";
const std::string EventLogChannel::PROP_HOST = "host";
const std::string EventLogChannel::PROP_LOGHOST = "loghost";
const std::string EventLogChannel::PROP_LOGFILE = "logfile";
EventLogChannel::EventLogChannel():
_logFile("Application"),
_h(0)
{
const DWORD maxPathLen = MAX_PATH + 1;
char name[maxPathLen];
int n = GetModuleFileNameA(NULL, name, maxPathLen);
if (n > 0)
{
char* end = name + n - 1;
while (end > name && *end != '\\') --end;
if (*end == '\\') ++end;
_name = end;
}
}
EventLogChannel::EventLogChannel(const std::string& name):
_name(name),
_logFile("Application"),
_h(0)
{
}
EventLogChannel::EventLogChannel(const std::string& name, const std::string& host):
_name(name),
_host(host),
_logFile("Application"),
_h(0)
{
}
EventLogChannel::~EventLogChannel()
{
try
{
close();
}
catch (...)
{
poco_unexpected();
}
}
void EventLogChannel::open()
{
setUpRegistry();
_h = RegisterEventSource(_host.empty() ? NULL : _host.c_str(), _name.c_str());
if (!_h) throw SystemException("cannot register event source");
}
void EventLogChannel::close()
{
if (_h) DeregisterEventSource(_h);
_h = 0;
}
void EventLogChannel::log(const Message& msg)
{
if (!_h) open();
const char* pMsg = msg.getText().c_str();
ReportEvent(_h, getType(msg), getCategory(msg), POCO_MSG_LOG, NULL, 1, 0, &pMsg, NULL);
}
void EventLogChannel::setProperty(const std::string& name, const std::string& value)
{
if (icompare(name, PROP_NAME) == 0)
_name = value;
else if (icompare(name, PROP_HOST) == 0)
_host = value;
else if (icompare(name, PROP_LOGHOST) == 0)
_host = value;
else if (icompare(name, PROP_LOGFILE) == 0)
_logFile = value;
else
Channel::setProperty(name, value);
}
std::string EventLogChannel::getProperty(const std::string& name) const
{
if (icompare(name, PROP_NAME) == 0)
return _name;
else if (icompare(name, PROP_HOST) == 0)
return _host;
else if (icompare(name, PROP_LOGHOST) == 0)
return _host;
else if (icompare(name, PROP_LOGFILE) == 0)
return _logFile;
else
return Channel::getProperty(name);
}
int EventLogChannel::getType(const Message& msg)
{
switch (msg.getPriority())
{
case Message::PRIO_TRACE:
case Message::PRIO_DEBUG:
case Message::PRIO_INFORMATION:
return EVENTLOG_INFORMATION_TYPE;
case Message::PRIO_NOTICE:
case Message::PRIO_WARNING:
return EVENTLOG_WARNING_TYPE;
default:
return EVENTLOG_ERROR_TYPE;
}
}
int EventLogChannel::getCategory(const Message& msg)
{
switch (msg.getPriority())
{
case Message::PRIO_TRACE:
return POCO_CTG_TRACE;
case Message::PRIO_DEBUG:
return POCO_CTG_DEBUG;
case Message::PRIO_INFORMATION:
return POCO_CTG_INFORMATION;
case Message::PRIO_NOTICE:
return POCO_CTG_NOTICE;
case Message::PRIO_WARNING:
return POCO_CTG_WARNING;
case Message::PRIO_ERROR:
return POCO_CTG_ERROR;
case Message::PRIO_CRITICAL:
return POCO_CTG_CRITICAL;
case Message::PRIO_FATAL:
return POCO_CTG_FATAL;
default:
return 0;
}
}
void EventLogChannel::setUpRegistry() const
{
std::string key = "SYSTEM\\CurrentControlSet\\Services\\EventLog\\";
key.append(_logFile);
key.append("\\");
key.append(_name);
HKEY hKey;
DWORD disp;
DWORD rc = RegCreateKeyEx(HKEY_LOCAL_MACHINE, key.c_str(), 0, NULL, REG_OPTION_NON_VOLATILE, KEY_ALL_ACCESS, NULL, &hKey, &disp);
if (rc != ERROR_SUCCESS) return;
if (disp == REG_CREATED_NEW_KEY)
{
std::string path;
if (path.empty())
path = findLibrary("PocoMsg.dll");
if (!path.empty())
{
DWORD count = 8;
DWORD types = 7;
RegSetValueEx(hKey, "CategoryMessageFile", 0, REG_SZ, (const BYTE*) path.c_str(), static_cast<DWORD>(path.size() + 1));
RegSetValueEx(hKey, "EventMessageFile", 0, REG_SZ, (const BYTE*) path.c_str(), static_cast<DWORD>(path.size() + 1));
RegSetValueEx(hKey, "CategoryCount", 0, REG_DWORD, (const BYTE*) &count, static_cast<DWORD>(sizeof(count)));
RegSetValueEx(hKey, "TypesSupported", 0, REG_DWORD, (const BYTE*) &types, static_cast<DWORD>(sizeof(types)));
}
}
RegCloseKey(hKey);
}
std::string EventLogChannel::findLibrary(const char* name)
{
std::string path;
HMODULE dll = LoadLibraryA(name);
if (dll)
{
const DWORD maxPathLen = MAX_PATH + 1;
char name[maxPathLen];
int n = GetModuleFileNameA(dll, name, maxPathLen);
if (n > 0) path = name;
FreeLibrary(dll);
}
return path;
}
} // namespace Poco

144
base/poco/Foundation/src/FPEnvironment_DEC.cpp
View File

@ -1,144 +0,0 @@
//
// FPEnvironment_DEC.cpp
//
// Library: Foundation
// Package: Core
// Module: FPEnvironment
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
//
// _XOPEN_SOURCE disables the ieee fp functions
// in <math.h>, therefore we undefine it for this file.
//
#undef _XOPEN_SOURCE
#include <math.h>
#include <fp.h>
#include <fp_class.h>
#include "Poco/FPEnvironment_DEC.h"
namespace Poco {
FPEnvironmentImpl::FPEnvironmentImpl()
{
_env = ieee_get_fp_control();
}
FPEnvironmentImpl::FPEnvironmentImpl(const FPEnvironmentImpl& env)
{
_env = env._env;
}
FPEnvironmentImpl::~FPEnvironmentImpl()
{
ieee_set_fp_control(_env);
}
FPEnvironmentImpl& FPEnvironmentImpl::operator = (const FPEnvironmentImpl& env)
{
_env = env._env;
return *this;
}
bool FPEnvironmentImpl::isInfiniteImpl(float value)
{
int cls = fp_classf(value);
return cls == FP_POS_INF || cls == FP_NEG_INF;
}
bool FPEnvironmentImpl::isInfiniteImpl(double value)
{
int cls = fp_class(value);
return cls == FP_POS_INF || cls == FP_NEG_INF;
}
bool FPEnvironmentImpl::isInfiniteImpl(long double value)
{
int cls = fp_classl(value);
return cls == FP_POS_INF || cls == FP_NEG_INF;
}
bool FPEnvironmentImpl::isNaNImpl(float value)
{
return isnanf(value) != 0;
}
bool FPEnvironmentImpl::isNaNImpl(double value)
{
return isnan(value) != 0;
}
bool FPEnvironmentImpl::isNaNImpl(long double value)
{
return isnanl(value) != 0;
}
float FPEnvironmentImpl::copySignImpl(float target, float source)
{
return copysignf(target, source);
}
double FPEnvironmentImpl::copySignImpl(double target, double source)
{
return copysign(target, source);
}
long double FPEnvironmentImpl::copySignImpl(long double target, long double source)
{
return copysignl(target, source);
}
void FPEnvironmentImpl::keepCurrentImpl()
{
ieee_set_fp_control(_env);
}
void FPEnvironmentImpl::clearFlagsImpl()
{
ieee_set_fp_control(0);
}
bool FPEnvironmentImpl::isFlagImpl(FlagImpl flag)
{
return (ieee_get_fp_control() & flag) != 0;
}
void FPEnvironmentImpl::setRoundingModeImpl(RoundingModeImpl mode)
{
// not supported
}
FPEnvironmentImpl::RoundingModeImpl FPEnvironmentImpl::getRoundingModeImpl()
{
// not supported
return FPEnvironmentImpl::RoundingModeImpl(0);
}
} // namespace Poco

79
base/poco/Foundation/src/FPEnvironment_DUMMY.cpp
View File

@ -1,79 +0,0 @@
//
// FPEnvironment_C99.cpp
//
// Library: Foundation
// Package: Core
// Module: FPEnvironment
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/FPEnvironment_DUMMY.h"
namespace Poco {
FPEnvironmentImpl::RoundingModeImpl FPEnvironmentImpl::_roundingMode;
FPEnvironmentImpl::FPEnvironmentImpl()
{
}
FPEnvironmentImpl::FPEnvironmentImpl(const FPEnvironmentImpl& env)
{
}
FPEnvironmentImpl::~FPEnvironmentImpl()
{
}
FPEnvironmentImpl& FPEnvironmentImpl::operator = (const FPEnvironmentImpl& env)
{
return *this;
}
void FPEnvironmentImpl::keepCurrentImpl()
{
}
void FPEnvironmentImpl::clearFlagsImpl()
{
}
bool FPEnvironmentImpl::isFlagImpl(FlagImpl flag)
{
return false;
}
void FPEnvironmentImpl::setRoundingModeImpl(RoundingModeImpl mode)
{
_roundingMode = mode;
}
FPEnvironmentImpl::RoundingModeImpl FPEnvironmentImpl::getRoundingModeImpl()
{
return _roundingMode;
}
long double FPEnvironmentImpl::copySignImpl(long double target, long double source)
{
return (source >= 0 && target >= 0) || (source < 0 && target < 0) ? target : -target;
}
} // namespace Poco

82
base/poco/Foundation/src/FPEnvironment_QNX.cpp
View File

@ -1,82 +0,0 @@
//
// FPEnvironment_QNX.cpp
//
// Library: Foundation
// Package: Core
// Module: FPEnvironment
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/FPEnvironment_QNX.h"
namespace Poco {
FPEnvironmentImpl::FPEnvironmentImpl()
{
fegetenv(&_env);
}
FPEnvironmentImpl::FPEnvironmentImpl(const FPEnvironmentImpl& env)
{
_env = env._env;
}
FPEnvironmentImpl::~FPEnvironmentImpl()
{
fesetenv(&_env);
}
FPEnvironmentImpl& FPEnvironmentImpl::operator = (const FPEnvironmentImpl& env)
{
_env = env._env;
return *this;
}
void FPEnvironmentImpl::keepCurrentImpl()
{
fegetenv(&_env);
}
void FPEnvironmentImpl::clearFlagsImpl()
{
feclearexcept(FE_ALL_EXCEPT);
}
bool FPEnvironmentImpl::isFlagImpl(FlagImpl flag)
{
return fetestexcept(flag) != 0;
}
void FPEnvironmentImpl::setRoundingModeImpl(RoundingModeImpl mode)
{
fesetround(mode);
}
FPEnvironmentImpl::RoundingModeImpl FPEnvironmentImpl::getRoundingModeImpl()
{
return (RoundingModeImpl) fegetround();
}
long double FPEnvironmentImpl::copySignImpl(long double target, long double source)
{
return (source >= 0 && target >= 0) || (source < 0 && target < 0) ? target : -target;
}
} // namespace Poco

119
base/poco/Foundation/src/Latin1Encoding.cpp
View File

@ -1,119 +0,0 @@
//
// Latin1Encoding.cpp
//
// Library: Foundation
// Package: Text
// Module: Latin1Encoding
//
// Copyright (c) 2004-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Latin1Encoding.h"
#include "Poco/String.h"
namespace Poco {
const char* Latin1Encoding::_names[] =
{
"ISO-8859-1",
"Latin1",
"Latin-1",
NULL
};
const TextEncoding::CharacterMap Latin1Encoding::_charMap =
{
/* 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f */
/* 00 */ 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
/* 10 */ 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
/* 20 */ 0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f,
/* 30 */ 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f,
/* 40 */ 0x40, 0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49, 0x4a, 0x4b, 0x4c, 0x4d, 0x4e, 0x4f,
/* 50 */ 0x50, 0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x5a, 0x5b, 0x5c, 0x5d, 0x5e, 0x5f,
/* 60 */ 0x60, 0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x6a, 0x6b, 0x6c, 0x6d, 0x6e, 0x6f,
/* 70 */ 0x70, 0x71, 0x72, 0x73, 0x74, 0x75, 0x76, 0x77, 0x78, 0x79, 0x7a, 0x7b, 0x7c, 0x7d, 0x7e, 0x7f,
/* 80 */ 0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87, 0x88, 0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f,
/* 90 */ 0x90, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97, 0x98, 0x99, 0x9a, 0x9b, 0x9c, 0x9d, 0x9e, 0x9f,
/* a0 */ 0xa0, 0xa1, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7, 0xa8, 0xa9, 0xaa, 0xab, 0xac, 0xad, 0xae, 0xaf,
/* b0 */ 0xb0, 0xb1, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7, 0xb8, 0xb9, 0xba, 0xbb, 0xbc, 0xbd, 0xbe, 0xbf,
/* c0 */ 0xc0, 0xc1, 0xc2, 0xc3, 0xc4, 0xc5, 0xc6, 0xc7, 0xc8, 0xc9, 0xca, 0xcb, 0xcc, 0xcd, 0xce, 0xcf,
/* d0 */ 0xd0, 0xd1, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6, 0xd7, 0xd8, 0xd9, 0xda, 0xdb, 0xdc, 0xdd, 0xde, 0xdf,
/* e0 */ 0xe0, 0xe1, 0xe2, 0xe3, 0xe4, 0xe5, 0xe6, 0xe7, 0xe8, 0xe9, 0xea, 0xeb, 0xec, 0xed, 0xee, 0xef,
/* f0 */ 0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8, 0xf9, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff,
};
Latin1Encoding::Latin1Encoding()
{
}
Latin1Encoding::~Latin1Encoding()
{
}
const char* Latin1Encoding::canonicalName() const
{
return _names[0];
}
bool Latin1Encoding::isA(const std::string& encodingName) const
{
for (const char** name = _names; *name; ++name)
{
if (Poco::icompare(encodingName, *name) == 0)
return true;
}
return false;
}
const TextEncoding::CharacterMap& Latin1Encoding::characterMap() const
{
return _charMap;
}
int Latin1Encoding::convert(const unsigned char* bytes) const
{
return *bytes;
}
int Latin1Encoding::convert(int ch, unsigned char* bytes, int length) const
{
if (ch >= 0 && ch <= 255)
{
if (bytes && length >= 1)
*bytes = (unsigned char) ch;
return 1;
}
else return 0;
}
int Latin1Encoding::queryConvert(const unsigned char* bytes, int length) const
{
if (1 <= length)
return *bytes;
else
return -1;
}
int Latin1Encoding::sequenceLength(const unsigned char* bytes, int length) const
{
return 1;
}
} // namespace Poco

179
base/poco/Foundation/src/Latin2Encoding.cpp
View File

@ -1,179 +0,0 @@
//
// Latin2Encoding.cpp
//
// Library: Foundation
// Package: Text
// Module: Latin2Encoding
//
// Copyright (c) 2004-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Latin2Encoding.h"
#include "Poco/String.h"
namespace Poco {
const char* Latin2Encoding::_names[] =
{
"ISO-8859-2",
"Latin2",
"Latin-2",
NULL
};
const TextEncoding::CharacterMap Latin2Encoding::_charMap =
{
/* 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f */
/* 00 */ 0x0000, 0x0001, 0x0002, 0x0003, 0x0004, 0x0005, 0x0006, 0x0007, 0x0008, 0x0009, 0x000a, 0x000b, 0x000c, 0x000d, 0x000e, 0x000f,
/* 10 */ 0x0010, 0x0011, 0x0012, 0x0013, 0x0014, 0x0015, 0x0016, 0x0017, 0x0018, 0x0019, 0x001a, 0x001b, 0x001c, 0x001d, 0x001e, 0x001f,
/* 20 */ 0x0020, 0x0021, 0x0022, 0x0023, 0x0024, 0x0025, 0x0026, 0x0027, 0x0028, 0x0029, 0x002a, 0x002b, 0x002c, 0x002d, 0x002e, 0x002f,
/* 30 */ 0x0030, 0x0031, 0x0032, 0x0033, 0x0034, 0x0035, 0x0036, 0x0037, 0x0038, 0x0039, 0x003a, 0x003b, 0x003c, 0x003d, 0x003e, 0x003f,
/* 40 */ 0x0040, 0x0041, 0x0042, 0x0043, 0x0044, 0x0045, 0x0046, 0x0047, 0x0048, 0x0049, 0x004a, 0x004b, 0x004c, 0x004d, 0x004e, 0x004f,
/* 50 */ 0x0050, 0x0051, 0x0052, 0x0053, 0x0054, 0x0055, 0x0056, 0x0057, 0x0058, 0x0059, 0x005a, 0x005b, 0x005c, 0x005d, 0x005e, 0x005f,
/* 60 */ 0x0060, 0x0061, 0x0062, 0x0063, 0x0064, 0x0065, 0x0066, 0x0067, 0x0068, 0x0069, 0x006a, 0x006b, 0x006c, 0x006d, 0x006e, 0x006f,
/* 70 */ 0x0070, 0x0071, 0x0072, 0x0073, 0x0074, 0x0075, 0x0076, 0x0077, 0x0078, 0x0079, 0x007a, 0x007b, 0x007c, 0x007d, 0x007e, 0x007f,
/* 80 */ 0x0080, 0x0081, 0x0082, 0x0083, 0x0084, 0x0085, 0x0086, 0x0087, 0x0088, 0x0089, 0x008a, 0x008b, 0x008c, 0x008d, 0x008e, 0x008f,
/* 90 */ 0x0090, 0x0091, 0x0092, 0x0093, 0x0094, 0x0095, 0x0096, 0x0097, 0x0098, 0x0099, 0x009a, 0x009b, 0x009c, 0x009d, 0x009e, 0x009f,
/* a0 */ 0x00a0, 0x0104, 0x02d8, 0x0141, 0x00a4, 0x013d, 0x015a, 0x00a7, 0x00a8, 0x0160, 0x015e, 0x0164, 0x0179, 0x00ad, 0x017d, 0x017b,
/* b0 */ 0x00b0, 0x0105, 0x02db, 0x0142, 0x00b4, 0x013e, 0x015b, 0x02c7, 0x00b8, 0x0161, 0x015f, 0x0165, 0x017a, 0x02dd, 0x017e, 0x017c,
/* c0 */ 0x0154, 0x00c1, 0x00c2, 0x0102, 0x00c4, 0x0139, 0x0106, 0x00c7, 0x010c, 0x00c9, 0x0118, 0x00cb, 0x011a, 0x00cd, 0x00ce, 0x010e,
/* d0 */ 0x0110, 0x0143, 0x0147, 0x00d3, 0x00d4, 0x0150, 0x00d6, 0x00d7, 0x0158, 0x016e, 0x00da, 0x0170, 0x00dc, 0x00dd, 0x0162, 0x00df,
/* e0 */ 0x0155, 0x00e1, 0x00e2, 0x0103, 0x00e4, 0x013a, 0x0107, 0x00e7, 0x010d, 0x00e9, 0x0119, 0x00eb, 0x011b, 0x00ed, 0x00ee, 0x010f,
/* f0 */ 0x0111, 0x0144, 0x0148, 0x00f3, 0x00f4, 0x0151, 0x00f6, 0x00f7, 0x0159, 0x016f, 0x00fa, 0x0171, 0x00fc, 0x00fd, 0x0163, 0x02d9,
};
Latin2Encoding::Latin2Encoding()
{
}
Latin2Encoding::~Latin2Encoding()
{
}
const char* Latin2Encoding::canonicalName() const
{
return _names[0];
}
bool Latin2Encoding::isA(const std::string& encodingName) const
{
for (const char** name = _names; *name; ++name)
{
if (Poco::icompare(encodingName, *name) == 0)
return true;
}
return false;
}
const TextEncoding::CharacterMap& Latin2Encoding::characterMap() const
{
return _charMap;
}
int Latin2Encoding::convert(const unsigned char* bytes) const
{
return _charMap[*bytes];
}
int Latin2Encoding::convert(int ch, unsigned char* bytes, int length) const
{
if (ch >= 0 && ch <= 255 && _charMap[ch] == ch)
{
if (bytes && length >= 1)
*bytes = (unsigned char) ch;
return 1;
}
switch(ch)
{
case 0x0104: if (bytes && length >= 1) *bytes = 0xa1; return 1;
case 0x02d8: if (bytes && length >= 1) *bytes = 0xa2; return 1;
case 0x0141: if (bytes && length >= 1) *bytes = 0xa3; return 1;
case 0x013d: if (bytes && length >= 1) *bytes = 0xa5; return 1;
case 0x015a: if (bytes && length >= 1) *bytes = 0xa6; return 1;
case 0x0160: if (bytes && length >= 1) *bytes = 0xa9; return 1;
case 0x015e: if (bytes && length >= 1) *bytes = 0xaa; return 1;
case 0x0164: if (bytes && length >= 1) *bytes = 0xab; return 1;
case 0x0179: if (bytes && length >= 1) *bytes = 0xac; return 1;
case 0x017d: if (bytes && length >= 1) *bytes = 0xae; return 1;
case 0x017b: if (bytes && length >= 1) *bytes = 0xaf; return 1;
case 0x0105: if (bytes && length >= 1) *bytes = 0xb1; return 1;
case 0x02db: if (bytes && length >= 1) *bytes = 0xb2; return 1;
case 0x0142: if (bytes && length >= 1) *bytes = 0xb3; return 1;
case 0x013e: if (bytes && length >= 1) *bytes = 0xb5; return 1;
case 0x015b: if (bytes && length >= 1) *bytes = 0xb6; return 1;
case 0x02c7: if (bytes && length >= 1) *bytes = 0xb7; return 1;
case 0x0161: if (bytes && length >= 1) *bytes = 0xb9; return 1;
case 0x015f: if (bytes && length >= 1) *bytes = 0xba; return 1;
case 0x0165: if (bytes && length >= 1) *bytes = 0xbb; return 1;
case 0x017a: if (bytes && length >= 1) *bytes = 0xbc; return 1;
case 0x02dd: if (bytes && length >= 1) *bytes = 0xbd; return 1;
case 0x017e: if (bytes && length >= 1) *bytes = 0xbe; return 1;
case 0x017c: if (bytes && length >= 1) *bytes = 0xbf; return 1;
case 0x0154: if (bytes && length >= 1) *bytes = 0xc0; return 1;
case 0x0102: if (bytes && length >= 1) *bytes = 0xc3; return 1;
case 0x0139: if (bytes && length >= 1) *bytes = 0xc5; return 1;
case 0x0106: if (bytes && length >= 1) *bytes = 0xc6; return 1;
case 0x010c: if (bytes && length >= 1) *bytes = 0xc8; return 1;
case 0x0118: if (bytes && length >= 1) *bytes = 0xca; return 1;
case 0x011a: if (bytes && length >= 1) *bytes = 0xcc; return 1;
case 0x010e: if (bytes && length >= 1) *bytes = 0xcf; return 1;
case 0x0110: if (bytes && length >= 1) *bytes = 0xd0; return 1;
case 0x0143: if (bytes && length >= 1) *bytes = 0xd1; return 1;
case 0x0147: if (bytes && length >= 1) *bytes = 0xd2; return 1;
case 0x0150: if (bytes && length >= 1) *bytes = 0xd5; return 1;
case 0x0158: if (bytes && length >= 1) *bytes = 0xd8; return 1;
case 0x016e: if (bytes && length >= 1) *bytes = 0xd9; return 1;
case 0x0170: if (bytes && length >= 1) *bytes = 0xdb; return 1;
case 0x0162: if (bytes && length >= 1) *bytes = 0xde; return 1;
case 0x0155: if (bytes && length >= 1) *bytes = 0xe0; return 1;
case 0x0103: if (bytes && length >= 1) *bytes = 0xe3; return 1;
case 0x013a: if (bytes && length >= 1) *bytes = 0xe5; return 1;
case 0x0107: if (bytes && length >= 1) *bytes = 0xe6; return 1;
case 0x010d: if (bytes && length >= 1) *bytes = 0xe8; return 1;
case 0x0119: if (bytes && length >= 1) *bytes = 0xea; return 1;
case 0x011b: if (bytes && length >= 1) *bytes = 0xec; return 1;
case 0x010f: if (bytes && length >= 1) *bytes = 0xef; return 1;
case 0x0111: if (bytes && length >= 1) *bytes = 0xf0; return 1;
case 0x0144: if (bytes && length >= 1) *bytes = 0xf1; return 1;
case 0x0148: if (bytes && length >= 1) *bytes = 0xf2; return 1;
case 0x0151: if (bytes && length >= 1) *bytes = 0xf5; return 1;
case 0x0159: if (bytes && length >= 1) *bytes = 0xf8; return 1;
case 0x016f: if (bytes && length >= 1) *bytes = 0xf9; return 1;
case 0x0171: if (bytes && length >= 1) *bytes = 0xfb; return 1;
case 0x0163: if (bytes && length >= 1) *bytes = 0xfe; return 1;
case 0x02d9: if (bytes && length >= 1) *bytes = 0xff; return 1;
default: return 0;
}
}
int Latin2Encoding::queryConvert(const unsigned char* bytes, int length) const
{
if (1 <= length)
return _charMap[*bytes];
else
return -1;
}
int Latin2Encoding::sequenceLength(const unsigned char* bytes, int length) const
{
return 1;
}
} // namespace Poco

130
base/poco/Foundation/src/Latin9Encoding.cpp
View File

@ -1,130 +0,0 @@
//
// Latin9Encoding.cpp
//
// Library: Foundation
// Package: Text
// Module: Latin9Encoding
//
// Copyright (c) 2004-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Latin9Encoding.h"
#include "Poco/String.h"
namespace Poco {
const char* Latin9Encoding::_names[] =
{
"ISO-8859-15",
"Latin9",
"Latin-9",
NULL
};
const TextEncoding::CharacterMap Latin9Encoding::_charMap =
{
/* 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f */
/* 00 */ 0x0000, 0x0001, 0x0002, 0x0003, 0x0004, 0x0005, 0x0006, 0x0007, 0x0008, 0x0009, 0x000a, 0x000b, 0x000c, 0x000d, 0x000e, 0x000f,
/* 10 */ 0x0010, 0x0011, 0x0012, 0x0013, 0x0014, 0x0015, 0x0016, 0x0017, 0x0018, 0x0019, 0x001a, 0x001b, 0x001c, 0x001d, 0x001e, 0x001f,
/* 20 */ 0x0020, 0x0021, 0x0022, 0x0023, 0x0024, 0x0025, 0x0026, 0x0027, 0x0028, 0x0029, 0x002a, 0x002b, 0x002c, 0x002d, 0x002e, 0x002f,
/* 30 */ 0x0030, 0x0031, 0x0032, 0x0033, 0x0034, 0x0035, 0x0036, 0x0037, 0x0038, 0x0039, 0x003a, 0x003b, 0x003c, 0x003d, 0x003e, 0x003f,
/* 40 */ 0x0040, 0x0041, 0x0042, 0x0043, 0x0044, 0x0045, 0x0046, 0x0047, 0x0048, 0x0049, 0x004a, 0x004b, 0x004c, 0x004d, 0x004e, 0x004f,
/* 50 */ 0x0050, 0x0051, 0x0052, 0x0053, 0x0054, 0x0055, 0x0056, 0x0057, 0x0058, 0x0059, 0x005a, 0x005b, 0x005c, 0x005d, 0x005e, 0x005f,
/* 60 */ 0x0060, 0x0061, 0x0062, 0x0063, 0x0064, 0x0065, 0x0066, 0x0067, 0x0068, 0x0069, 0x006a, 0x006b, 0x006c, 0x006d, 0x006e, 0x006f,
/* 70 */ 0x0070, 0x0071, 0x0072, 0x0073, 0x0074, 0x0075, 0x0076, 0x0077, 0x0078, 0x0079, 0x007a, 0x007b, 0x007c, 0x007d, 0x007e, 0x007f,
/* 80 */ 0x0080, 0x0081, 0x0082, 0x0083, 0x0084, 0x0085, 0x0086, 0x0087, 0x0088, 0x0089, 0x008a, 0x008b, 0x008c, 0x008d, 0x008e, 0x008f,
/* 90 */ 0x0090, 0x0091, 0x0092, 0x0093, 0x0094, 0x0095, 0x0096, 0x0097, 0x0098, 0x0099, 0x009a, 0x009b, 0x009c, 0x009d, 0x009e, 0x009f,
/* a0 */ 0x00a0, 0x00a1, 0x00a2, 0x00a3, 0x20ac, 0x00a5, 0x0160, 0x00a7, 0x0161, 0x00a9, 0x00aa, 0x00ab, 0x00ac, 0x00ad, 0x00ae, 0x00af,
/* b0 */ 0x00b0, 0x00b1, 0x00b2, 0x00b3, 0x017d, 0x00b5, 0x00b6, 0x00b7, 0x017e, 0x00b9, 0x00ba, 0x00bb, 0x0152, 0x0153, 0x0178, 0x00bf,
/* c0 */ 0x00c0, 0x00c1, 0x00c2, 0x00c3, 0x00c4, 0x00c5, 0x00c6, 0x00c7, 0x00c8, 0x00c9, 0x00ca, 0x00cb, 0x00cc, 0x00cd, 0x00ce, 0x00cf,
/* d0 */ 0x00d0, 0x00d1, 0x00d2, 0x00d3, 0x00d4, 0x00d5, 0x00d6, 0x00d7, 0x00d8, 0x00d9, 0x00da, 0x00db, 0x00dc, 0x00dd, 0x00de, 0x00df,
/* e0 */ 0x00e0, 0x00e1, 0x00e2, 0x00e3, 0x00e4, 0x00e5, 0x00e6, 0x00e7, 0x00e8, 0x00e9, 0x00ea, 0x00eb, 0x00ec, 0x00ed, 0x00ee, 0x00ef,
/* f0 */ 0x00f0, 0x00f1, 0x00f2, 0x00f3, 0x00f4, 0x00f5, 0x00f6, 0x00f7, 0x00f8, 0x00f9, 0x00fa, 0x00fb, 0x00fc, 0x00fd, 0x00fe, 0x00ff,
};
Latin9Encoding::Latin9Encoding()
{
}
Latin9Encoding::~Latin9Encoding()
{
}
const char* Latin9Encoding::canonicalName() const
{
return _names[0];
}
bool Latin9Encoding::isA(const std::string& encodingName) const
{
for (const char** name = _names; *name; ++name)
{
if (Poco::icompare(encodingName, *name) == 0)
return true;
}
return false;
}
const TextEncoding::CharacterMap& Latin9Encoding::characterMap() const
{
return _charMap;
}
int Latin9Encoding::convert(const unsigned char* bytes) const
{
return _charMap[*bytes];
}
int Latin9Encoding::convert(int ch, unsigned char* bytes, int length) const
{
if (ch >= 0 && ch <= 255 && _charMap[ch] == ch)
{
if (bytes && length >= 1)
*bytes = ch;
return 1;
}
else switch (ch)
{
case 0x0152: if (bytes && length >= 1) *bytes = 0xbc; return 1;
case 0x0153: if (bytes && length >= 1) *bytes = 0xbd; return 1;
case 0x0160: if (bytes && length >= 1) *bytes = 0xa6; return 1;
case 0x0161: if (bytes && length >= 1) *bytes = 0xa8; return 1;
case 0x017d: if (bytes && length >= 1) *bytes = 0xb4; return 1;
case 0x017e: if (bytes && length >= 1) *bytes = 0xb8; return 1;
case 0x0178: if (bytes && length >= 1) *bytes = 0xbe; return 1;
case 0x20ac: if (bytes && length >= 1) *bytes = 0xa4; return 1;
default: return 0;
}
}
int Latin9Encoding::queryConvert(const unsigned char* bytes, int length) const
{
if (1 <= length)
return _charMap[*bytes];
else
return -1;
}
int Latin9Encoding::sequenceLength(const unsigned char* bytes, int length) const
{
return 1;
}
} // namespace Poco

278
base/poco/Foundation/src/MD4Engine.cpp
View File

@ -1,278 +0,0 @@
//
// MD4Engine.cpp
//
// Library: Foundation
// Package: Crypt
// Module: MD4Engine
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
//
// MD4 (RFC 1320) algorithm:
// Copyright (C) 1991-2, RSA Data Security, Inc. Created 1991. All
// rights reserved.
//
// License to copy and use this software is granted provided that it
// is identified as the "RSA Data Security, Inc. MD4 Message-Digest
// Algorithm" in all material mentioning or referencing this software
// or this function.
//
// License is also granted to make and use derivative works provided
// that such works are identified as "derived from the RSA Data
// Security, Inc. MD4 Message-Digest Algorithm" in all material
// mentioning or referencing the derived work.
//
// RSA Data Security, Inc. makes no representations concerning either
// the merchantability of this software or the suitability of this
// software for any particular purpose. It is provided "as is"
// without express or implied warranty of any kind.
//
// These notices must be retained in any copies of any part of this
// documentation and/or software.
//
#include "Poco/MD4Engine.h"
#include <cstring>
namespace Poco {
MD4Engine::MD4Engine()
{
_digest.reserve(16);
reset();
}
MD4Engine::~MD4Engine()
{
reset();
}
void MD4Engine::updateImpl(const void* input_, std::size_t inputLen)
{
const unsigned char* input = (const unsigned char*) input_;
unsigned int i, index, partLen;
/* Compute number of bytes mod 64 */
index = (unsigned int)((_context.count[0] >> 3) & 0x3F);
/* Update number of bits */
if ((_context.count[0] += ((UInt32) inputLen << 3)) < ((UInt32) inputLen << 3))
_context.count[1]++;
_context.count[1] += ((UInt32) inputLen >> 29);
partLen = 64 - index;
/* Transform as many times as possible. */
if (inputLen >= partLen)
{
std::memcpy(&_context.buffer[index], input, partLen);
transform(_context.state, _context.buffer);
for (i = partLen; i + 63 < inputLen; i += 64)
transform(_context.state, &input[i]);
index = 0;
}
else i = 0;
/* Buffer remaining input */
std::memcpy(&_context.buffer[index], &input[i], inputLen-i);
}
std::size_t MD4Engine::digestLength() const
{
return DIGEST_SIZE;
}
void MD4Engine::reset()
{
std::memset(&_context, 0, sizeof(_context));
_context.count[0] = _context.count[1] = 0;
_context.state[0] = 0x67452301;
_context.state[1] = 0xefcdab89;
_context.state[2] = 0x98badcfe;
_context.state[3] = 0x10325476;
}
const DigestEngine::Digest& MD4Engine::digest()
{
static const unsigned char PADDING[64] =
{
0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
};
unsigned char bits[8];
unsigned int index, padLen;
/* Save number of bits */
encode(bits, _context.count, 8);
/* Pad out to 56 mod 64. */
index = (unsigned int)((_context.count[0] >> 3) & 0x3f);
padLen = (index < 56) ? (56 - index) : (120 - index);
update(PADDING, padLen);
/* Append length (before padding) */
update(bits, 8);
/* Store state in digest */
unsigned char digest[16];
encode(digest, _context.state, 16);
_digest.clear();
_digest.insert(_digest.begin(), digest, digest + sizeof(digest));
/* Zeroize sensitive information. */
std::memset(&_context, 0, sizeof (_context));
reset();
return _digest;
}
/* Constants for MD4Transform routine. */
#define S11 3
#define S12 7
#define S13 11
#define S14 19
#define S21 3
#define S22 5
#define S23 9
#define S24 13
#define S31 3
#define S32 9
#define S33 11
#define S34 15
/* F, G and H are basic MD4 functions. */
#define F(x, y, z) (((x) & (y)) | ((~x) & (z)))
#define G(x, y, z) (((x) & (y)) | ((x) & (z)) | ((y) & (z)))
#define H(x, y, z) ((x) ^ (y) ^ (z))
/* ROTATE_LEFT rotates x left n bits. */
#define ROTATE_LEFT(x, n) (((x) << (n)) | ((x) >> (32-(n))))
/* FF, GG and HH are transformations for rounds 1, 2 and 3 */
/* Rotation is separate from addition to prevent recomputation */
#define FF(a, b, c, d, x, s) { \
(a) += F ((b), (c), (d)) + (x); \
(a) = ROTATE_LEFT ((a), (s)); \
}
#define GG(a, b, c, d, x, s) { \
(a) += G ((b), (c), (d)) + (x) + (UInt32)0x5a827999; \
(a) = ROTATE_LEFT ((a), (s)); \
}
#define HH(a, b, c, d, x, s) { \
(a) += H ((b), (c), (d)) + (x) + (UInt32)0x6ed9eba1; \
(a) = ROTATE_LEFT ((a), (s)); \
}
void MD4Engine::transform (UInt32 state[4], const unsigned char block[64])
{
UInt32 a = state[0], b = state[1], c = state[2], d = state[3], x[16];
decode(x, block, 64);
/* Round 1 */
FF (a, b, c, d, x[ 0], S11); /* 1 */
FF (d, a, b, c, x[ 1], S12); /* 2 */
FF (c, d, a, b, x[ 2], S13); /* 3 */
FF (b, c, d, a, x[ 3], S14); /* 4 */
FF (a, b, c, d, x[ 4], S11); /* 5 */
FF (d, a, b, c, x[ 5], S12); /* 6 */
FF (c, d, a, b, x[ 6], S13); /* 7 */
FF (b, c, d, a, x[ 7], S14); /* 8 */
FF (a, b, c, d, x[ 8], S11); /* 9 */
FF (d, a, b, c, x[ 9], S12); /* 10 */
FF (c, d, a, b, x[10], S13); /* 11 */
FF (b, c, d, a, x[11], S14); /* 12 */
FF (a, b, c, d, x[12], S11); /* 13 */
FF (d, a, b, c, x[13], S12); /* 14 */
FF (c, d, a, b, x[14], S13); /* 15 */
FF (b, c, d, a, x[15], S14); /* 16 */
/* Round 2 */
GG (a, b, c, d, x[ 0], S21); /* 17 */
GG (d, a, b, c, x[ 4], S22); /* 18 */
GG (c, d, a, b, x[ 8], S23); /* 19 */
GG (b, c, d, a, x[12], S24); /* 20 */
GG (a, b, c, d, x[ 1], S21); /* 21 */
GG (d, a, b, c, x[ 5], S22); /* 22 */
GG (c, d, a, b, x[ 9], S23); /* 23 */
GG (b, c, d, a, x[13], S24); /* 24 */
GG (a, b, c, d, x[ 2], S21); /* 25 */
GG (d, a, b, c, x[ 6], S22); /* 26 */
GG (c, d, a, b, x[10], S23); /* 27 */
GG (b, c, d, a, x[14], S24); /* 28 */
GG (a, b, c, d, x[ 3], S21); /* 29 */
GG (d, a, b, c, x[ 7], S22); /* 30 */
GG (c, d, a, b, x[11], S23); /* 31 */
GG (b, c, d, a, x[15], S24); /* 32 */
/* Round 3 */
HH (a, b, c, d, x[ 0], S31); /* 33 */
HH (d, a, b, c, x[ 8], S32); /* 34 */
HH (c, d, a, b, x[ 4], S33); /* 35 */
HH (b, c, d, a, x[12], S34); /* 36 */
HH (a, b, c, d, x[ 2], S31); /* 37 */
HH (d, a, b, c, x[10], S32); /* 38 */
HH (c, d, a, b, x[ 6], S33); /* 39 */
HH (b, c, d, a, x[14], S34); /* 40 */
HH (a, b, c, d, x[ 1], S31); /* 41 */
HH (d, a, b, c, x[ 9], S32); /* 42 */
HH (c, d, a, b, x[ 5], S33); /* 43 */
HH (b, c, d, a, x[13], S34); /* 44 */
HH (a, b, c, d, x[ 3], S31); /* 45 */
HH (d, a, b, c, x[11], S32); /* 46 */
HH (c, d, a, b, x[ 7], S33); /* 47 */
HH (b, c, d, a, x[15], S34); /* 48 */
state[0] += a;
state[1] += b;
state[2] += c;
state[3] += d;
/* Zeroize sensitive information. */
std::memset(x, 0, sizeof(x));
}
void MD4Engine::encode(unsigned char* output, const UInt32* input, std::size_t len)
{
unsigned int i, j;
for (i = 0, j = 0; j < len; i++, j += 4)
{
output[j] = (unsigned char)(input[i] & 0xff);
output[j+1] = (unsigned char)((input[i] >> 8) & 0xff);
output[j+2] = (unsigned char)((input[i] >> 16) & 0xff);
output[j+3] = (unsigned char)((input[i] >> 24) & 0xff);
}
}
void MD4Engine::decode(UInt32* output, const unsigned char* input, std::size_t len)
{
unsigned int i, j;
for (i = 0, j = 0; j < len; i++, j += 4)
output[i] = ((UInt32)input[j]) | (((UInt32)input[j+1]) << 8) |
(((UInt32)input[j+2]) << 16) | (((UInt32)input[j+3]) << 24);
}
} // namespace Poco

31
base/poco/Foundation/src/Manifest.cpp
View File

@ -1,31 +0,0 @@
//
// Manifest.cpp
//
// Library: Foundation
// Package: SharedLibrary
// Module: ClassLoader
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Manifest.h"
namespace Poco {
ManifestBase::ManifestBase()
{
}
ManifestBase::~ManifestBase()
{
}
} // namespace Poco

65
base/poco/Foundation/src/PipeImpl_DUMMY.cpp
View File

@ -1,65 +0,0 @@
//
// PipeImpl_DUMMY.cpp
//
// Library: Foundation
// Package: Processes
// Module: PipeImpl
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/PipeImpl_DUMMY.h"
namespace Poco {
PipeImpl::PipeImpl()
{
}
PipeImpl::~PipeImpl()
{
}
int PipeImpl::writeBytes(const void* buffer, int length)
{
return 0;
}
int PipeImpl::readBytes(void* buffer, int length)
{
return 0;
}
PipeImpl::Handle PipeImpl::readHandle() const
{
return 0;
}
PipeImpl::Handle PipeImpl::writeHandle() const
{
return 0;
}
void PipeImpl::closeRead()
{
}
void PipeImpl::closeWrite()
{
}
} // namespace Poco

127
base/poco/Foundation/src/PipeStream.cpp
View File

@ -1,127 +0,0 @@
//
// PipeStream.cpp
//
// Library: Foundation
// Package: Processes
// Module: PipeStream
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/PipeStream.h"
namespace Poco {
//
// PipeStreamBuf
//
PipeStreamBuf::PipeStreamBuf(const Pipe& pipe, openmode mode):
BufferedStreamBuf(STREAM_BUFFER_SIZE, mode),
_pipe(pipe)
{
}
PipeStreamBuf::~PipeStreamBuf()
{
}
int PipeStreamBuf::readFromDevice(char* buffer, std::streamsize length)
{
return _pipe.readBytes(buffer, (int) length);
}
int PipeStreamBuf::writeToDevice(const char* buffer, std::streamsize length)
{
return _pipe.writeBytes(buffer, (int) length);
}
void PipeStreamBuf::close()
{
_pipe.close(Pipe::CLOSE_BOTH);
}
//
// PipeIOS
//
PipeIOS::PipeIOS(const Pipe& pipe, openmode mode):
_buf(pipe, mode)
{
poco_ios_init(&_buf);
}
PipeIOS::~PipeIOS()
{
try
{
_buf.sync();
}
catch (...)
{
}
}
PipeStreamBuf* PipeIOS::rdbuf()
{
return &_buf;
}
void PipeIOS::close()
{
_buf.sync();
_buf.close();
}
//
// PipeOutputStream
//
PipeOutputStream::PipeOutputStream(const Pipe& pipe):
PipeIOS(pipe, std::ios::out),
std::ostream(&_buf)
{
}
PipeOutputStream::~PipeOutputStream()
{
}
//
// PipeInputStream
//
PipeInputStream::PipeInputStream(const Pipe& pipe):
PipeIOS(pipe, std::ios::in),
std::istream(&_buf)
{
}
PipeInputStream::~PipeInputStream()
{
}
} // namespace Poco

52
base/poco/Foundation/src/Semaphore_VX.cpp
View File

@ -1,52 +0,0 @@
//
// Semaphore_VX.cpp
//
// Library: Foundation
// Package: Threading
// Module: Semaphore
//
// Copyright (c) 2004-2011, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Semaphore_VX.h"
#include <sysLib.h>
namespace Poco {
SemaphoreImpl::SemaphoreImpl(int n, int max)
{
poco_assert (n >= 0 && max > 0 && n <= max);
_sem = semCCreate(SEM_Q_PRIORITY, n);
if (_sem == 0)
throw Poco::SystemException("cannot create semaphore");
}
SemaphoreImpl::~SemaphoreImpl()
{
semDelete(_sem);
}
void SemaphoreImpl::waitImpl()
{
if (semTake(_sem, WAIT_FOREVER) != OK)
throw SystemException("cannot wait for semaphore");
}
bool SemaphoreImpl::waitImpl(long milliseconds)
{
int ticks = milliseconds*sysClkRateGet()/1000;
return semTake(_sem, ticks) == OK;
}
} // namespace Poco

65
base/poco/Foundation/src/Semaphore_WIN32.cpp
View File

@ -1,65 +0,0 @@
//
// Semaphore_WIN32.cpp
//
// Library: Foundation
// Package: Threading
// Module: Semaphore
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Semaphore_WIN32.h"
namespace Poco {
SemaphoreImpl::SemaphoreImpl(int n, int max)
{
poco_assert (n >= 0 && max > 0 && n <= max);
_sema = CreateSemaphoreW(NULL, n, max, NULL);
if (!_sema)
{
throw SystemException("cannot create semaphore");
}
}
SemaphoreImpl::~SemaphoreImpl()
{
CloseHandle(_sema);
}
void SemaphoreImpl::waitImpl()
{
switch (WaitForSingleObject(_sema, INFINITE))
{
case WAIT_OBJECT_0:
return;
default:
throw SystemException("wait for semaphore failed");
}
}
bool SemaphoreImpl::waitImpl(long milliseconds)
{
switch (WaitForSingleObject(_sema, milliseconds + 1))
{
case WAIT_TIMEOUT:
return false;
case WAIT_OBJECT_0:
return true;
default:
throw SystemException("wait for semaphore failed");
}
}
} // namespace Poco

36
base/poco/Foundation/src/SharedMemory_DUMMY.cpp
View File

@ -1,36 +0,0 @@
//
// SharedMemoryImpl.cpp
//
// Library: Foundation
// Package: Processes
// Module: SharedMemoryImpl
//
// Copyright (c) 2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/SharedMemory_DUMMY.h"
namespace Poco {
SharedMemoryImpl::SharedMemoryImpl(const std::string&, std::size_t, SharedMemory::AccessMode, const void*, bool)
{
}
SharedMemoryImpl::SharedMemoryImpl(const Poco::File&, SharedMemory::AccessMode, const void*)
{
}
SharedMemoryImpl::~SharedMemoryImpl()
{
}
} // namespace Poco

105
base/poco/Foundation/src/StreamTokenizer.cpp
View File

@ -1,105 +0,0 @@
//
// StreamTokenizer.cpp
//
// Library: Foundation
// Package: Streams
// Module: StreamTokenizer
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/StreamTokenizer.h"
namespace Poco {
StreamTokenizer::StreamTokenizer():
_pIstr(0)
{
}
StreamTokenizer::StreamTokenizer(std::istream& istr):
_pIstr(&istr)
{
}
StreamTokenizer::~StreamTokenizer()
{
for (TokenVec::iterator it = _tokens.begin(); it != _tokens.end(); ++it)
{
delete it->pToken;
}
}
void StreamTokenizer::attachToStream(std::istream& istr)
{
_pIstr = &istr;
}
void StreamTokenizer::addToken(Token* pToken)
{
poco_check_ptr (pToken);
TokenInfo ti;
ti.pToken = pToken;
ti.ignore = (pToken->tokenClass() == Token::COMMENT_TOKEN || pToken->tokenClass() == Token::WHITESPACE_TOKEN);
_tokens.push_back(ti);
}
void StreamTokenizer::addToken(Token* pToken, bool ignore)
{
poco_check_ptr (pToken);
TokenInfo ti;
ti.pToken = pToken;
ti.ignore = ignore;
_tokens.push_back(ti);
}
const Token* StreamTokenizer::next()
{
poco_check_ptr (_pIstr);
static const int eof = std::char_traits<char>::eof();
int first = _pIstr->get();
TokenVec::const_iterator it = _tokens.begin();
while (first != eof && it != _tokens.end())
{
const TokenInfo& ti = *it;
if (ti.pToken->start((char) first, *_pIstr))
{
ti.pToken->finish(*_pIstr);
if (ti.ignore)
{
first = _pIstr->get();
it = _tokens.begin();
}
else return ti.pToken;
}
else ++it;
}
if (first == eof)
{
return &_eofToken;
}
else
{
_invalidToken.start((char) first, *_pIstr);
return &_invalidToken;
}
}
} // namespace Poco

31
base/poco/Foundation/src/SynchronizedObject.cpp
View File

@ -1,31 +0,0 @@
//
// SynchronizedObject.cpp
//
// Library: Foundation
// Package: Threading
// Module: SynchronizedObject
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/SynchronizedObject.h"
namespace Poco {
SynchronizedObject::SynchronizedObject()
{
}
SynchronizedObject::~SynchronizedObject()
{
}
} // namespace Poco

12
base/poco/Foundation/src/TextEncoding.cpp
View File

@ -16,15 +16,9 @@
#include "Poco/Exception.h"
#include "Poco/String.h"
#include "Poco/ASCIIEncoding.h"
#include "Poco/Latin1Encoding.h"
#include "Poco/Latin2Encoding.h"
#include "Poco/Latin9Encoding.h"
#include "Poco/UTF32Encoding.h"
#include "Poco/UTF16Encoding.h"
#include "Poco/UTF8Encoding.h"
#include "Poco/Windows1250Encoding.h"
#include "Poco/Windows1251Encoding.h"
#include "Poco/Windows1252Encoding.h"
#include "Poco/RWLock.h"
#include "Poco/SingletonHolder.h"
#include <map>
@ -47,15 +41,9 @@ public:
add(pUtf8Encoding, TextEncoding::GLOBAL);
add(new ASCIIEncoding);
add(new Latin1Encoding);
add(new Latin2Encoding);
add(new Latin9Encoding);
add(pUtf8Encoding);
add(new UTF16Encoding);
add(new UTF32Encoding);
add(new Windows1250Encoding);
add(new Windows1251Encoding);
add(new Windows1252Encoding);
}
~TextEncodingManager()

78
base/poco/Foundation/src/Timezone_VX.cpp
View File

@ -1,78 +0,0 @@
//
// Timezone_VXX.cpp
//
// Library: Foundation
// Package: DateTime
// Module: Timezone
//
// Copyright (c) 2004-2011, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Timezone.h"
#include "Poco/Exception.h"
#include "Poco/Environment.h"
#include <ctime>
namespace Poco {
int Timezone::utcOffset()
{
std::time_t now = std::time(NULL);
struct std::tm t;
gmtime_r(&now, &t);
std::time_t utc = std::mktime(&t);
return now - utc;
}
int Timezone::dst()
{
std::time_t now = std::time(NULL);
struct std::tm t;
if (localtime_r(&now, &t) != OK)
throw Poco::SystemException("cannot get local time DST offset");
return t.tm_isdst == 1 ? 3600 : 0;
}
bool Timezone::isDst(const Timestamp& timestamp)
{
std::time_t time = timestamp.epochTime();
struct std::tm* tms = std::localtime(&time);
if (!tms) throw Poco::SystemException("cannot get local time DST flag");
return tms->tm_isdst > 0;
}
std::string Timezone::name()
{
// format of TIMEZONE environment variable:
// name_of_zone:<(unused)>:time_in_minutes_from_UTC:daylight_start:daylight_end
std::string tz = Environment::get("TIMEZONE", "UTC");
std::string::size_type pos = tz.find(':');
if (pos != std::string::npos)
return tz.substr(0, pos);
else
return tz;
}
std::string Timezone::standardName()
{
return name();
}
std::string Timezone::dstName()
{
return name();
}
} // namespace Poco

237
base/poco/Foundation/src/Windows1251Encoding.cpp
View File

@ -1,237 +0,0 @@
//
// Windows1251Encoding.cpp
//
// Library: Foundation
// Package: Text
// Module: Windows1251Encoding
//
// Copyright (c) 2005-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Windows1251Encoding.h"
#include "Poco/String.h"
namespace Poco {
const char* Windows1251Encoding::_names[] =
{
"windows-1251",
"Windows-1251",
"cp1251",
"CP1251",
NULL
};
const TextEncoding::CharacterMap Windows1251Encoding::_charMap =
{
/* 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f */
/* 00 */ 0x0000, 0x0001, 0x0002, 0x0003, 0x0004, 0x0005, 0x0006, 0x0007, 0x0008, 0x0009, 0x000a, 0x000b, 0x000c, 0x000d, 0x000e, 0x000f,
/* 10 */ 0x0010, 0x0011, 0x0012, 0x0013, 0x0014, 0x0015, 0x0016, 0x0017, 0x0018, 0x0019, 0x001a, 0x001b, 0x001c, 0x001d, 0x001e, 0x001f,
/* 20 */ 0x0020, 0x0021, 0x0022, 0x0023, 0x0024, 0x0025, 0x0026, 0x0027, 0x0028, 0x0029, 0x002a, 0x002b, 0x002c, 0x002d, 0x002e, 0x002f,
/* 30 */ 0x0030, 0x0031, 0x0032, 0x0033, 0x0034, 0x0035, 0x0036, 0x0037, 0x0038, 0x0039, 0x003a, 0x003b, 0x003c, 0x003d, 0x003e, 0x003f,
/* 40 */ 0x0040, 0x0041, 0x0042, 0x0043, 0x0044, 0x0045, 0x0046, 0x0047, 0x0048, 0x0049, 0x004a, 0x004b, 0x004c, 0x004d, 0x004e, 0x004f,
/* 50 */ 0x0050, 0x0051, 0x0052, 0x0053, 0x0054, 0x0055, 0x0056, 0x0057, 0x0058, 0x0059, 0x005a, 0x005b, 0x005c, 0x005d, 0x005e, 0x005f,
/* 60 */ 0x0060, 0x0061, 0x0062, 0x0063, 0x0064, 0x0065, 0x0066, 0x0067, 0x0068, 0x0069, 0x006a, 0x006b, 0x006c, 0x006d, 0x006e, 0x006f,
/* 70 */ 0x0070, 0x0071, 0x0072, 0x0073, 0x0074, 0x0075, 0x0076, 0x0077, 0x0078, 0x0079, 0x007a, 0x007b, 0x007c, 0x007d, 0x007e, 0x007f,
/* 80 */ 0x0402, 0x0403, 0x201a, 0x0453, 0x201e, 0x2026, 0x2020, 0x2021, 0x20ac, 0x2030, 0x0409, 0x2039, 0x040a, 0x040c, 0x040b, 0x040f,
/* 90 */ 0x0452, 0x2018, 0x2019, 0x201c, 0x201d, 0x2022, 0x2013, 0x2014, 0xfffe, 0x2122, 0x0459, 0x203a, 0x045a, 0x045c, 0x045b, 0x045f,
/* a0 */ 0x00a0, 0x040e, 0x045e, 0x0408, 0x00a4, 0x0490, 0x00a6, 0x00a7, 0x0401, 0x00a9, 0x0404, 0x00ab, 0x00ac, 0x00ad, 0x00ae, 0x0407,
/* b0 */ 0x00b0, 0x00b1, 0x0406, 0x0456, 0x0491, 0x00b5, 0x00b6, 0x00b7, 0x0451, 0x2116, 0x0454, 0x00bb, 0x0458, 0x0405, 0x0455, 0x0457,
/* c0 */ 0x0410, 0x0411, 0x0412, 0x0413, 0x0414, 0x0415, 0x0416, 0x0417, 0x0418, 0x0419, 0x041a, 0x041b, 0x041c, 0x041d, 0x041e, 0x041f,
/* d0 */ 0x0420, 0x0421, 0x0422, 0x0423, 0x0424, 0x0425, 0x0426, 0x0427, 0x0428, 0x0429, 0x042a, 0x042b, 0x042c, 0x042d, 0x042e, 0x042f,
/* e0 */ 0x0430, 0x0431, 0x0432, 0x0433, 0x0434, 0x0435, 0x0436, 0x0437, 0x0438, 0x0439, 0x043a, 0x043b, 0x043c, 0x043d, 0x043e, 0x043f,
/* f0 */ 0x0440, 0x0441, 0x0442, 0x0443, 0x0444, 0x0445, 0x0446, 0x0447, 0x0448, 0x0449, 0x044a, 0x044b, 0x044c, 0x044d, 0x044e, 0x044f,
};
Windows1251Encoding::Windows1251Encoding()
{
}
Windows1251Encoding::~Windows1251Encoding()
{
}
const char* Windows1251Encoding::canonicalName() const
{
return _names[0];
}
bool Windows1251Encoding::isA(const std::string& encodingName) const
{
for (const char** name = _names; *name; ++name)
{
if (Poco::icompare(encodingName, *name) == 0)
return true;
}
return false;
}
const TextEncoding::CharacterMap& Windows1251Encoding::characterMap() const
{
return _charMap;
}
int Windows1251Encoding::convert(const unsigned char* bytes) const
{
return _charMap[*bytes];
}
int Windows1251Encoding::convert(int ch, unsigned char* bytes, int length) const
{
if (ch >= 0 && ch <= 255 && _charMap[ch] == ch)
{
if (bytes && length >= 1)
*bytes = (unsigned char) ch;
return 1;
}
else switch(ch)
{
case 0x0402: if (bytes && length >= 1) *bytes = 0x80; return 1;
case 0x0403: if (bytes && length >= 1) *bytes = 0x81; return 1;
case 0x201a: if (bytes && length >= 1) *bytes = 0x82; return 1;
case 0x0453: if (bytes && length >= 1) *bytes = 0x83; return 1;
case 0x201e: if (bytes && length >= 1) *bytes = 0x84; return 1;
case 0x2026: if (bytes && length >= 1) *bytes = 0x85; return 1;
case 0x2020: if (bytes && length >= 1) *bytes = 0x86; return 1;
case 0x2021: if (bytes && length >= 1) *bytes = 0x87; return 1;
case 0x20ac: if (bytes && length >= 1) *bytes = 0x88; return 1;
case 0x2030: if (bytes && length >= 1) *bytes = 0x89; return 1;
case 0x0409: if (bytes && length >= 1) *bytes = 0x8a; return 1;
case 0x2039: if (bytes && length >= 1) *bytes = 0x8b; return 1;
case 0x040a: if (bytes && length >= 1) *bytes = 0x8c; return 1;
case 0x040c: if (bytes && length >= 1) *bytes = 0x8d; return 1;
case 0x040b: if (bytes && length >= 1) *bytes = 0x8e; return 1;
case 0x040f: if (bytes && length >= 1) *bytes = 0x8f; return 1;
case 0x0452: if (bytes && length >= 1) *bytes = 0x90; return 1;
case 0x2018: if (bytes && length >= 1) *bytes = 0x91; return 1;
case 0x2019: if (bytes && length >= 1) *bytes = 0x92; return 1;
case 0x201c: if (bytes && length >= 1) *bytes = 0x93; return 1;
case 0x201d: if (bytes && length >= 1) *bytes = 0x94; return 1;
case 0x2022: if (bytes && length >= 1) *bytes = 0x95; return 1;
case 0x2013: if (bytes && length >= 1) *bytes = 0x96; return 1;
case 0x2014: if (bytes && length >= 1) *bytes = 0x97; return 1;
case 0xfffe: if (bytes && length >= 1) *bytes = 0x98; return 1;
case 0x2122: if (bytes && length >= 1) *bytes = 0x99; return 1;
case 0x0459: if (bytes && length >= 1) *bytes = 0x9a; return 1;
case 0x203a: if (bytes && length >= 1) *bytes = 0x9b; return 1;
case 0x045a: if (bytes && length >= 1) *bytes = 0x9c; return 1;
case 0x045c: if (bytes && length >= 1) *bytes = 0x9d; return 1;
case 0x045b: if (bytes && length >= 1) *bytes = 0x9e; return 1;
case 0x045f: if (bytes && length >= 1) *bytes = 0x9f; return 1;
case 0x040e: if (bytes && length >= 1) *bytes = 0xa1; return 1;
case 0x045e: if (bytes && length >= 1) *bytes = 0xa2; return 1;
case 0x0408: if (bytes && length >= 1) *bytes = 0xa3; return 1;
case 0x0490: if (bytes && length >= 1) *bytes = 0xa5; return 1;
case 0x0401: if (bytes && length >= 1) *bytes = 0xa8; return 1;
case 0x0404: if (bytes && length >= 1) *bytes = 0xaa; return 1;
case 0x0407: if (bytes && length >= 1) *bytes = 0xaf; return 1;
case 0x0406: if (bytes && length >= 1) *bytes = 0xb2; return 1;
case 0x0456: if (bytes && length >= 1) *bytes = 0xb3; return 1;
case 0x0491: if (bytes && length >= 1) *bytes = 0xb4; return 1;
case 0x0451: if (bytes && length >= 1) *bytes = 0xb8; return 1;
case 0x2116: if (bytes && length >= 1) *bytes = 0xb9; return 1;
case 0x0454: if (bytes && length >= 1) *bytes = 0xba; return 1;
case 0x0458: if (bytes && length >= 1) *bytes = 0xbc; return 1;
case 0x0405: if (bytes && length >= 1) *bytes = 0xbd; return 1;
case 0x0455: if (bytes && length >= 1) *bytes = 0xbe; return 1;
case 0x0457: if (bytes && length >= 1) *bytes = 0xbf; return 1;
case 0x0410: if (bytes && length >= 1) *bytes = 0xc0; return 1;
case 0x0411: if (bytes && length >= 1) *bytes = 0xc1; return 1;
case 0x0412: if (bytes && length >= 1) *bytes = 0xc2; return 1;
case 0x0413: if (bytes && length >= 1) *bytes = 0xc3; return 1;
case 0x0414: if (bytes && length >= 1) *bytes = 0xc4; return 1;
case 0x0415: if (bytes && length >= 1) *bytes = 0xc5; return 1;
case 0x0416: if (bytes && length >= 1) *bytes = 0xc6; return 1;
case 0x0417: if (bytes && length >= 1) *bytes = 0xc7; return 1;
case 0x0418: if (bytes && length >= 1) *bytes = 0xc8; return 1;
case 0x0419: if (bytes && length >= 1) *bytes = 0xc9; return 1;
case 0x041a: if (bytes && length >= 1) *bytes = 0xca; return 1;
case 0x041b: if (bytes && length >= 1) *bytes = 0xcb; return 1;
case 0x041c: if (bytes && length >= 1) *bytes = 0xcc; return 1;
case 0x041d: if (bytes && length >= 1) *bytes = 0xcd; return 1;
case 0x041e: if (bytes && length >= 1) *bytes = 0xce; return 1;
case 0x041f: if (bytes && length >= 1) *bytes = 0xcf; return 1;
case 0x0420: if (bytes && length >= 1) *bytes = 0xd0; return 1;
case 0x0421: if (bytes && length >= 1) *bytes = 0xd1; return 1;
case 0x0422: if (bytes && length >= 1) *bytes = 0xd2; return 1;
case 0x0423: if (bytes && length >= 1) *bytes = 0xd3; return 1;
case 0x0424: if (bytes && length >= 1) *bytes = 0xd4; return 1;
case 0x0425: if (bytes && length >= 1) *bytes = 0xd5; return 1;
case 0x0426: if (bytes && length >= 1) *bytes = 0xd6; return 1;
case 0x0427: if (bytes && length >= 1) *bytes = 0xd7; return 1;
case 0x0428: if (bytes && length >= 1) *bytes = 0xd8; return 1;
case 0x0429: if (bytes && length >= 1) *bytes = 0xd9; return 1;
case 0x042a: if (bytes && length >= 1) *bytes = 0xda; return 1;
case 0x042b: if (bytes && length >= 1) *bytes = 0xdb; return 1;
case 0x042c: if (bytes && length >= 1) *bytes = 0xdc; return 1;
case 0x042d: if (bytes && length >= 1) *bytes = 0xdd; return 1;
case 0x042e: if (bytes && length >= 1) *bytes = 0xde; return 1;
case 0x042f: if (bytes && length >= 1) *bytes = 0xdf; return 1;
case 0x0430: if (bytes && length >= 1) *bytes = 0xe0; return 1;
case 0x0431: if (bytes && length >= 1) *bytes = 0xe1; return 1;
case 0x0432: if (bytes && length >= 1) *bytes = 0xe2; return 1;
case 0x0433: if (bytes && length >= 1) *bytes = 0xe3; return 1;
case 0x0434: if (bytes && length >= 1) *bytes = 0xe4; return 1;
case 0x0435: if (bytes && length >= 1) *bytes = 0xe5; return 1;
case 0x0436: if (bytes && length >= 1) *bytes = 0xe6; return 1;
case 0x0437: if (bytes && length >= 1) *bytes = 0xe7; return 1;
case 0x0438: if (bytes && length >= 1) *bytes = 0xe8; return 1;
case 0x0439: if (bytes && length >= 1) *bytes = 0xe9; return 1;
case 0x043a: if (bytes && length >= 1) *bytes = 0xea; return 1;
case 0x043b: if (bytes && length >= 1) *bytes = 0xeb; return 1;
case 0x043c: if (bytes && length >= 1) *bytes = 0xec; return 1;
case 0x043d: if (bytes && length >= 1) *bytes = 0xed; return 1;
case 0x043e: if (bytes && length >= 1) *bytes = 0xee; return 1;
case 0x043f: if (bytes && length >= 1) *bytes = 0xef; return 1;
case 0x0440: if (bytes && length >= 1) *bytes = 0xf0; return 1;
case 0x0441: if (bytes && length >= 1) *bytes = 0xf1; return 1;
case 0x0442: if (bytes && length >= 1) *bytes = 0xf2; return 1;
case 0x0443: if (bytes && length >= 1) *bytes = 0xf3; return 1;
case 0x0444: if (bytes && length >= 1) *bytes = 0xf4; return 1;
case 0x0445: if (bytes && length >= 1) *bytes = 0xf5; return 1;
case 0x0446: if (bytes && length >= 1) *bytes = 0xf6; return 1;
case 0x0447: if (bytes && length >= 1) *bytes = 0xf7; return 1;
case 0x0448: if (bytes && length >= 1) *bytes = 0xf8; return 1;
case 0x0449: if (bytes && length >= 1) *bytes = 0xf9; return 1;
case 0x044a: if (bytes && length >= 1) *bytes = 0xfa; return 1;
case 0x044b: if (bytes && length >= 1) *bytes = 0xfb; return 1;
case 0x044c: if (bytes && length >= 1) *bytes = 0xfc; return 1;
case 0x044d: if (bytes && length >= 1) *bytes = 0xfd; return 1;
case 0x044e: if (bytes && length >= 1) *bytes = 0xfe; return 1;
case 0x044f: if (bytes && length >= 1) *bytes = 0xff; return 1;
default: return 0;
}
}
int Windows1251Encoding::queryConvert(const unsigned char* bytes, int length) const
{
if (1 <= length)
return _charMap[*bytes];
else
return -1;
}
int Windows1251Encoding::sequenceLength(const unsigned char* bytes, int length) const
{
return 1;
}
} // namespace Poco

151
base/poco/Foundation/src/Windows1252Encoding.cpp
View File

@ -1,151 +0,0 @@
//
// Windows1252Encoding.cpp
//
// Library: Foundation
// Package: Text
// Module: Windows1252Encoding
//
// Copyright (c) 2005-2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Windows1252Encoding.h"
#include "Poco/String.h"
#include <map>
namespace Poco {
const char* Windows1252Encoding::_names[] =
{
"windows-1252",
"Windows-1252",
"cp1252",
"CP1252",
NULL
};
const TextEncoding::CharacterMap Windows1252Encoding::_charMap =
{
/* 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f */
/* 00 */ 0x0000, 0x0001, 0x0002, 0x0003, 0x0004, 0x0005, 0x0006, 0x0007, 0x0008, 0x0009, 0x000a, 0x000b, 0x000c, 0x000d, 0x000e, 0x000f,
/* 10 */ 0x0010, 0x0011, 0x0012, 0x0013, 0x0014, 0x0015, 0x0016, 0x0017, 0x0018, 0x0019, 0x001a, 0x001b, 0x001c, 0x001d, 0x001e, 0x001f,
/* 20 */ 0x0020, 0x0021, 0x0022, 0x0023, 0x0024, 0x0025, 0x0026, 0x0027, 0x0028, 0x0029, 0x002a, 0x002b, 0x002c, 0x002d, 0x002e, 0x002f,
/* 30 */ 0x0030, 0x0031, 0x0032, 0x0033, 0x0034, 0x0035, 0x0036, 0x0037, 0x0038, 0x0039, 0x003a, 0x003b, 0x003c, 0x003d, 0x003e, 0x003f,
/* 40 */ 0x0040, 0x0041, 0x0042, 0x0043, 0x0044, 0x0045, 0x0046, 0x0047, 0x0048, 0x0049, 0x004a, 0x004b, 0x004c, 0x004d, 0x004e, 0x004f,
/* 50 */ 0x0050, 0x0051, 0x0052, 0x0053, 0x0054, 0x0055, 0x0056, 0x0057, 0x0058, 0x0059, 0x005a, 0x005b, 0x005c, 0x005d, 0x005e, 0x005f,
/* 60 */ 0x0060, 0x0061, 0x0062, 0x0063, 0x0064, 0x0065, 0x0066, 0x0067, 0x0068, 0x0069, 0x006a, 0x006b, 0x006c, 0x006d, 0x006e, 0x006f,
/* 70 */ 0x0070, 0x0071, 0x0072, 0x0073, 0x0074, 0x0075, 0x0076, 0x0077, 0x0078, 0x0079, 0x007a, 0x007b, 0x007c, 0x007d, 0x007e, 0x007f,
/* 80 */ 0x20ac, 0x0081, 0x201a, 0x0192, 0x201e, 0x2026, 0x2020, 0x2021, 0x02c6, 0x2030, 0x0160, 0x2039, 0x0152, 0x008d, 0x017d, 0x008f,
/* 90 */ 0x0090, 0x2018, 0x2019, 0x201c, 0x201d, 0x2022, 0x2013, 0x2014, 0x02dc, 0x2122, 0x0161, 0x203a, 0x0153, 0x009d, 0x017e, 0x0178,
/* a0 */ 0x00a0, 0x00a1, 0x00a2, 0x00a3, 0x00a4, 0x00a5, 0x00a6, 0x00a7, 0x00a8, 0x00a9, 0x00aa, 0x00ab, 0x00ac, 0x00ad, 0x00ae, 0x00af,
/* b0 */ 0x00b0, 0x00b1, 0x00b2, 0x00b3, 0x00b4, 0x00b5, 0x00b6, 0x00b7, 0x00b8, 0x00b9, 0x00ba, 0x00bb, 0x00bc, 0x00bd, 0x00be, 0x00bf,
/* c0 */ 0x00c0, 0x00c1, 0x00c2, 0x00c3, 0x00c4, 0x00c5, 0x00c6, 0x00c7, 0x00c8, 0x00c9, 0x00ca, 0x00cb, 0x00cc, 0x00cd, 0x00ce, 0x00cf,
/* d0 */ 0x00d0, 0x00d1, 0x00d2, 0x00d3, 0x00d4, 0x00d5, 0x00d6, 0x00d7, 0x00d8, 0x00d9, 0x00da, 0x00db, 0x00dc, 0x00dd, 0x00de, 0x00df,
/* e0 */ 0x00e0, 0x00e1, 0x00e2, 0x00e3, 0x00e4, 0x00e5, 0x00e6, 0x00e7, 0x00e8, 0x00e9, 0x00ea, 0x00eb, 0x00ec, 0x00ed, 0x00ee, 0x00ef,
/* f0 */ 0x00f0, 0x00f1, 0x00f2, 0x00f3, 0x00f4, 0x00f5, 0x00f6, 0x00f7, 0x00f8, 0x00f9, 0x00fa, 0x00fb, 0x00fc, 0x00fd, 0x00fe, 0x00ff,
};
Windows1252Encoding::Windows1252Encoding()
{
}
Windows1252Encoding::~Windows1252Encoding()
{
}
const char* Windows1252Encoding::canonicalName() const
{
return _names[0];
}
bool Windows1252Encoding::isA(const std::string& encodingName) const
{
for (const char** name = _names; *name; ++name)
{
if (Poco::icompare(encodingName, *name) == 0)
return true;
}
return false;
}
const TextEncoding::CharacterMap& Windows1252Encoding::characterMap() const
{
return _charMap;
}
int Windows1252Encoding::convert(const unsigned char* bytes) const
{
return _charMap[*bytes];
}
int Windows1252Encoding::convert(int ch, unsigned char* bytes, int length) const
{
if (ch >= 0 && ch <= 255 && _charMap[ch] == ch)
{
if (bytes && length >= 1)
*bytes = ch;
return 1;
}
else switch (ch)
{
case 0x20ac: if (bytes && length >= 1) *bytes = 0x80; return 1;
case 0x201a: if (bytes && length >= 1) *bytes = 0x82; return 1;
case 0x0192: if (bytes && length >= 1) *bytes = 0x83; return 1;
case 0x201e: if (bytes && length >= 1) *bytes = 0x84; return 1;
case 0x2026: if (bytes && length >= 1) *bytes = 0x85; return 1;
case 0x2020: if (bytes && length >= 1) *bytes = 0x86; return 1;
case 0x2021: if (bytes && length >= 1) *bytes = 0x87; return 1;
case 0x02c6: if (bytes && length >= 1) *bytes = 0x88; return 1;
case 0x2030: if (bytes && length >= 1) *bytes = 0x89; return 1;
case 0x0160: if (bytes && length >= 1) *bytes = 0x8a; return 1;
case 0x2039: if (bytes && length >= 1) *bytes = 0x8b; return 1;
case 0x0152: if (bytes && length >= 1) *bytes = 0x8c; return 1;
case 0x017d: if (bytes && length >= 1) *bytes = 0x8e; return 1;
case 0x2018: if (bytes && length >= 1) *bytes = 0x91; return 1;
case 0x2019: if (bytes && length >= 1) *bytes = 0x92; return 1;
case 0x201c: if (bytes && length >= 1) *bytes = 0x93; return 1;
case 0x201d: if (bytes && length >= 1) *bytes = 0x94; return 1;
case 0x2022: if (bytes && length >= 1) *bytes = 0x95; return 1;
case 0x2013: if (bytes && length >= 1) *bytes = 0x96; return 1;
case 0x2014: if (bytes && length >= 1) *bytes = 0x97; return 1;
case 0x02dc: if (bytes && length >= 1) *bytes = 0x98; return 1;
case 0x2122: if (bytes && length >= 1) *bytes = 0x99; return 1;
case 0x0161: if (bytes && length >= 1) *bytes = 0x9a; return 1;
case 0x203a: if (bytes && length >= 1) *bytes = 0x9b; return 1;
case 0x0153: if (bytes && length >= 1) *bytes = 0x9c; return 1;
case 0x017e: if (bytes && length >= 1) *bytes = 0x9e; return 1;
case 0x0178: if (bytes && length >= 1) *bytes = 0x9f; return 1;
default: return 0;
}
}
int Windows1252Encoding::queryConvert(const unsigned char* bytes, int length) const
{
if (1 <= length)
return _charMap[*bytes];
else
return -1;
}
int Windows1252Encoding::sequenceLength(const unsigned char* bytes, int length) const
{
return 1;
}
} // namespace Poco

269
base/poco/Foundation/src/WindowsConsoleChannel.cpp
View File

@ -1,269 +0,0 @@
//
// WindowsConsoleChannel.cpp
//
// Library: Foundation
// Package: Logging
// Module: WindowsConsoleChannel
//
// Copyright (c) 2007, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/WindowsConsoleChannel.h"
#include "Poco/Message.h"
#include "Poco/String.h"
#include "Poco/Exception.h"
namespace Poco {
WindowsConsoleChannel::WindowsConsoleChannel():
_isFile(false),
_hConsole(INVALID_HANDLE_VALUE)
{
_hConsole = GetStdHandle(STD_OUTPUT_HANDLE);
// check whether the console has been redirected
DWORD mode;
_isFile = (GetConsoleMode(_hConsole, &mode) == 0);
}
WindowsConsoleChannel::~WindowsConsoleChannel()
{
}
void WindowsConsoleChannel::log(const Message& msg)
{
std::string text = msg.getText();
text += "\r\n";
DWORD written;
WriteFile(_hConsole, text.data(), text.size(), &written, NULL);
}
WindowsColorConsoleChannel::WindowsColorConsoleChannel():
_enableColors(true),
_isFile(false),
_hConsole(INVALID_HANDLE_VALUE)
{
_hConsole = GetStdHandle(STD_OUTPUT_HANDLE);
// check whether the console has been redirected
DWORD mode;
_isFile = (GetConsoleMode(_hConsole, &mode) == 0);
initColors();
}
WindowsColorConsoleChannel::~WindowsColorConsoleChannel()
{
}
void WindowsColorConsoleChannel::log(const Message& msg)
{
std::string text = msg.getText();
text += "\r\n";
if (_enableColors && !_isFile)
{
WORD attr = _colors[0];
attr &= 0xFFF0;
attr |= _colors[msg.getPriority()];
SetConsoleTextAttribute(_hConsole, attr);
}
DWORD written;
WriteFile(_hConsole, text.data(), text.size(), &written, NULL);
if (_enableColors && !_isFile)
{
SetConsoleTextAttribute(_hConsole, _colors[0]);
}
}
void WindowsColorConsoleChannel::setProperty(const std::string& name, const std::string& value)
{
if (name == "enableColors")
{
_enableColors = icompare(value, "true") == 0;
}
else if (name == "traceColor")
{
_colors[Message::PRIO_TRACE] = parseColor(value);
}
else if (name == "debugColor")
{
_colors[Message::PRIO_DEBUG] = parseColor(value);
}
else if (name == "informationColor")
{
_colors[Message::PRIO_INFORMATION] = parseColor(value);
}
else if (name == "noticeColor")
{
_colors[Message::PRIO_NOTICE] = parseColor(value);
}
else if (name == "warningColor")
{
_colors[Message::PRIO_WARNING] = parseColor(value);
}
else if (name == "errorColor")
{
_colors[Message::PRIO_ERROR] = parseColor(value);
}
else if (name == "criticalColor")
{
_colors[Message::PRIO_CRITICAL] = parseColor(value);
}
else if (name == "fatalColor")
{
_colors[Message::PRIO_FATAL] = parseColor(value);
}
else
{
Channel::setProperty(name, value);
}
}
std::string WindowsColorConsoleChannel::getProperty(const std::string& name) const
{
if (name == "enableColors")
{
return _enableColors ? "true" : "false";
}
else if (name == "traceColor")
{
return formatColor(_colors[Message::PRIO_TRACE]);
}
else if (name == "debugColor")
{
return formatColor(_colors[Message::PRIO_DEBUG]);
}
else if (name == "informationColor")
{
return formatColor(_colors[Message::PRIO_INFORMATION]);
}
else if (name == "noticeColor")
{
return formatColor(_colors[Message::PRIO_NOTICE]);
}
else if (name == "warningColor")
{
return formatColor(_colors[Message::PRIO_WARNING]);
}
else if (name == "errorColor")
{
return formatColor(_colors[Message::PRIO_ERROR]);
}
else if (name == "criticalColor")
{
return formatColor(_colors[Message::PRIO_CRITICAL]);
}
else if (name == "fatalColor")
{
return formatColor(_colors[Message::PRIO_FATAL]);
}
else
{
return Channel::getProperty(name);
}
}
WORD WindowsColorConsoleChannel::parseColor(const std::string& color) const
{
if (icompare(color, "default") == 0)
return _colors[0];
else if (icompare(color, "black") == 0)
return CC_BLACK;
else if (icompare(color, "red") == 0)
return CC_RED;
else if (icompare(color, "green") == 0)
return CC_GREEN;
else if (icompare(color, "brown") == 0)
return CC_BROWN;
else if (icompare(color, "blue") == 0)
return CC_BLUE;
else if (icompare(color, "magenta") == 0)
return CC_MAGENTA;
else if (icompare(color, "cyan") == 0)
return CC_CYAN;
else if (icompare(color, "gray") == 0)
return CC_GRAY;
else if (icompare(color, "darkGray") == 0)
return CC_DARKGRAY;
else if (icompare(color, "lightRed") == 0)
return CC_LIGHTRED;
else if (icompare(color, "lightGreen") == 0)
return CC_LIGHTGREEN;
else if (icompare(color, "yellow") == 0)
return CC_YELLOW;
else if (icompare(color, "lightBlue") == 0)
return CC_LIGHTBLUE;
else if (icompare(color, "lightMagenta") == 0)
return CC_LIGHTMAGENTA;
else if (icompare(color, "lightCyan") == 0)
return CC_LIGHTCYAN;
else if (icompare(color, "white") == 0)
return CC_WHITE;
else throw InvalidArgumentException("Invalid color value", color);
}
std::string WindowsColorConsoleChannel::formatColor(WORD color) const
{
switch (color)
{
case CC_BLACK: return "black";
case CC_RED: return "red";
case CC_GREEN: return "green";
case CC_BROWN: return "brown";
case CC_BLUE: return "blue";
case CC_MAGENTA: return "magenta";
case CC_CYAN: return "cyan";
case CC_GRAY: return "gray";
case CC_DARKGRAY: return "darkGray";
case CC_LIGHTRED: return "lightRed";
case CC_LIGHTGREEN: return "lightGreen";
case CC_YELLOW: return "yellow";
case CC_LIGHTBLUE: return "lightBlue";
case CC_LIGHTMAGENTA: return "lightMagenta";
case CC_LIGHTCYAN: return "lightCyan";
case CC_WHITE: return "white";
default: return "invalid";
}
}
void WindowsColorConsoleChannel::initColors()
{
if (!_isFile)
{
CONSOLE_SCREEN_BUFFER_INFO csbi;
GetConsoleScreenBufferInfo(_hConsole, &csbi);
_colors[0] = csbi.wAttributes;
}
else
{
_colors[0] = CC_WHITE;
}
_colors[Message::PRIO_FATAL] = CC_LIGHTRED;
_colors[Message::PRIO_CRITICAL] = CC_LIGHTRED;
_colors[Message::PRIO_ERROR] = CC_LIGHTRED;
_colors[Message::PRIO_WARNING] = CC_YELLOW;
_colors[Message::PRIO_NOTICE] = _colors[0];
_colors[Message::PRIO_INFORMATION] = _colors[0];
_colors[Message::PRIO_DEBUG] = CC_GRAY;
_colors[Message::PRIO_TRACE] = CC_GRAY;
}
} // namespace Poco

109
base/poco/Net/include/Poco/Net/SMTPChannel.h
View File

@ -1,109 +0,0 @@
//
// SMTPChannel.h
//
// Library: Net
// Package: Logging
// Module: SMTPChannel
//
// Definition of the SMTPChannel class.
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Net_SMTPChannel_INCLUDED
#define Net_SMTPChannel_INCLUDED
#include "Poco/Channel.h"
#include "Poco/Net/Net.h"
#include "Poco/String.h"
namespace Poco
{
namespace Net
{
class Net_API SMTPChannel : public Poco::Channel
/// This Channel implements SMTP (email) logging.
{
public:
SMTPChannel();
/// Creates a SMTPChannel.
SMTPChannel(const std::string & mailhost, const std::string & sender, const std::string & recipient);
/// Creates a SMTPChannel with the given target mailhost, sender, and recipient.
void open();
/// Opens the SMTPChannel.
void close();
/// Closes the SMTPChannel.
void log(const Message & msg);
/// Sends the message's text to the recipient.
void setProperty(const std::string & name, const std::string & value);
/// Sets the property with the given value.
///
/// The following properties are supported:
/// * mailhost: The SMTP server. Default is "localhost".
/// * sender: The sender address.
/// * recipient: The recipient address.
/// * local: If true, local time is used. Default is true.
/// * attachment: Filename of the file to attach.
/// * type: Content type of the file to attach.
/// * delete: Boolean value indicating whether to delete
/// the attachment file after sending.
/// * throw: Boolean value indicating whether to throw
/// exception upon failure.
std::string getProperty(const std::string & name) const;
/// Returns the value of the property with the given name.
static void registerChannel();
/// Registers the channel with the global LoggingFactory.
static const std::string PROP_MAILHOST;
static const std::string PROP_SENDER;
static const std::string PROP_RECIPIENT;
static const std::string PROP_LOCAL;
static const std::string PROP_ATTACHMENT;
static const std::string PROP_TYPE;
static const std::string PROP_DELETE;
static const std::string PROP_THROW;
protected:
~SMTPChannel();
private:
bool isTrue(const std::string & value) const;
std::string _mailHost;
std::string _sender;
std::string _recipient;
bool _local;
std::string _attachment;
std::string _type;
bool _delete;
bool _throw;
};
inline bool SMTPChannel::isTrue(const std::string & value) const
{
return (
(0 == icompare(value, "true")) || (0 == icompare(value, "t")) || (0 == icompare(value, "yes")) || (0 == icompare(value, "y")));
}
}
} // namespace Poco::Net
#endif // Net_SMTPChannel_INCLUDED

210
base/poco/Net/src/SMTPChannel.cpp
View File

@ -1,210 +0,0 @@
//
// SMTPChannel.cpp
//
// Library: Net
// Package: Logging
// Module: SMTPChannel
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Net/SMTPChannel.h"
#include "Poco/Net/MailMessage.h"
#include "Poco/Net/MailRecipient.h"
#include "Poco/Net/SMTPClientSession.h"
#include "Poco/Net/StringPartSource.h"
#include "Poco/Message.h"
#include "Poco/DateTimeFormatter.h"
#include "Poco/DateTimeFormat.h"
#include "Poco/LocalDateTime.h"
#include "Poco/LoggingFactory.h"
#include "Poco/Instantiator.h"
#include "Poco/NumberFormatter.h"
#include "Poco/FileStream.h"
#include "Poco/File.h"
#include "Poco/Environment.h"
namespace Poco {
namespace Net {
const std::string SMTPChannel::PROP_MAILHOST("mailhost");
const std::string SMTPChannel::PROP_SENDER("sender");
const std::string SMTPChannel::PROP_RECIPIENT("recipient");
const std::string SMTPChannel::PROP_LOCAL("local");
const std::string SMTPChannel::PROP_ATTACHMENT("attachment");
const std::string SMTPChannel::PROP_TYPE("type");
const std::string SMTPChannel::PROP_DELETE("delete");
const std::string SMTPChannel::PROP_THROW("throw");
SMTPChannel::SMTPChannel():
_mailHost("localhost"),
_local(true),
_type("text/plain"),
_delete(false),
_throw(false)
{
}
SMTPChannel::SMTPChannel(const std::string& mailhost, const std::string& sender, const std::string& recipient):
_mailHost(mailhost),
_sender(sender),
_recipient(recipient),
_local(true),
_type("text/plain"),
_delete(false),
_throw(false)
{
}
SMTPChannel::~SMTPChannel()
{
try
{
close();
}
catch (...)
{
poco_unexpected();
}
}
void SMTPChannel::open()
{
}
void SMTPChannel::close()
{
}
void SMTPChannel::log(const Message& msg)
{
try
{
MailMessage message;
message.setSender(_sender);
message.addRecipient(MailRecipient(MailRecipient::PRIMARY_RECIPIENT, _recipient));
message.setSubject("Log Message from " + _sender);
std::stringstream content;
content << "Log Message\r\n"
<< "===========\r\n\r\n"
<< "Host: " << Environment::nodeName() << "\r\n"
<< "Logger: " << msg.getSource() << "\r\n";
if (_local)
{
DateTime dt(msg.getTime());
content << "Timestamp: " << DateTimeFormatter::format(LocalDateTime(dt), DateTimeFormat::RFC822_FORMAT) << "\r\n";
}
else
content << "Timestamp: " << DateTimeFormatter::format(msg.getTime(), DateTimeFormat::RFC822_FORMAT) << "\r\n";
content << "Priority: " << NumberFormatter::format(msg.getPriority()) << "\r\n"
<< "Process ID: " << NumberFormatter::format(msg.getPid()) << "\r\n"
<< "Thread: " << msg.getThread() << " (ID: " << msg.getTid() << ")\r\n"
<< "Message text: " << msg.getText() << "\r\n\r\n";
message.addContent(new StringPartSource(content.str()));
if (!_attachment.empty())
{
{
Poco::FileInputStream fis(_attachment, std::ios::in | std::ios::binary | std::ios::ate);
if (fis.good())
{
typedef std::allocator<std::string::value_type>::size_type SST;
std::streamoff size = fis.tellg();
poco_assert (std::numeric_limits<unsigned int>::max() >= size);
poco_assert (std::numeric_limits<SST>::max() >= size);
char* pMem = new char [static_cast<unsigned int>(size)];
fis.seekg(std::ios::beg);
fis.read(pMem, size);
message.addAttachment(_attachment,
new StringPartSource(std::string(pMem, static_cast<SST>(size)),
_type,
_attachment));
delete [] pMem;
}
}
if (_delete) File(_attachment).remove();
}
SMTPClientSession session(_mailHost);
session.login();
session.sendMessage(message);
session.close();
}
catch (Exception&)
{
if (_throw) throw;
}
}
void SMTPChannel::setProperty(const std::string& name, const std::string& value)
{
if (name == PROP_MAILHOST)
_mailHost = value;
else if (name == PROP_SENDER)
_sender = value;
else if (name == PROP_RECIPIENT)
_recipient = value;
else if (name == PROP_LOCAL)
_local = isTrue(value);
else if (name == PROP_ATTACHMENT)
_attachment = value;
else if (name == PROP_TYPE)
_type = value;
else if (name == PROP_DELETE)
_delete = isTrue(value);
else if (name == PROP_THROW)
_throw = isTrue(value);
else
Channel::setProperty(name, value);
}
std::string SMTPChannel::getProperty(const std::string& name) const
{
if (name == PROP_MAILHOST)
return _mailHost;
else if (name == PROP_SENDER)
return _sender;
else if (name == PROP_RECIPIENT)
return _recipient;
else if (name == PROP_LOCAL)
return _local ? "true" : "false";
else if (name == PROP_ATTACHMENT)
return _attachment;
else if (name == PROP_TYPE)
return _type;
else if (name == PROP_DELETE)
return _delete ? "true" : "false";
else if (name == PROP_THROW)
return _throw ? "true" : "false";
else
return Channel::getProperty(name);
}
void SMTPChannel::registerChannel()
{
Poco::LoggingFactory::defaultFactory().registerChannelClass("SMTPChannel",
new Poco::Instantiator<SMTPChannel, Poco::Channel>);
}
} } // namespace Poco::Net

97
base/poco/Util/include/Poco/Util/ConfigurationMapper.h
View File

@ -1,97 +0,0 @@
//
// ConfigurationMapper.h
//
// Library: Util
// Package: Configuration
// Module: ConfigurationMapper
//
// Definition of the ConfigurationMapper class.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Util_ConfigurationMapper_INCLUDED
#define Util_ConfigurationMapper_INCLUDED
#include "Poco/Util/AbstractConfiguration.h"
#include "Poco/Util/Util.h"
namespace Poco
{
namespace Util
{
class Util_API ConfigurationMapper : public AbstractConfiguration
/// This configuration maps a property hierarchy into another
/// hierarchy.
///
/// For example, given a configuration with the following properties:
/// config.value1
/// config.value2
/// config.sub.value1
/// config.sub.value2
/// and a ConfigurationView with fromPrefix == "config" and toPrefix == "root.conf", then
/// the above properties will be available via the mapper as
/// root.conf.value1
/// root.conf.value2
/// root.conf.sub.value1
/// root.conf.sub.value2
///
/// FromPrefix can be empty, in which case, and given toPrefix == "root",
/// the properties will be available as
/// root.config.value1
/// root.config.value2
/// root.config.sub.value1
/// root.config.sub.value2
///
/// This is equivalent to the functionality of the ConfigurationView class.
///
/// Similarly, toPrefix can also be empty. Given fromPrefix == "config" and
/// toPrefix == "", the properties will be available as
/// value1
/// value2
/// sub.value1
/// sub.value2
///
/// If both fromPrefix and toPrefix are empty, no mapping is performed.
///
/// A ConfigurationMapper is most useful in combination with a
/// LayeredConfiguration.
{
public:
ConfigurationMapper(const std::string & fromPrefix, const std::string & toPrefix, AbstractConfiguration * pConfig);
/// Creates the ConfigurationMapper. The ConfigurationMapper does not take
/// ownership of the passed configuration.
protected:
bool getRaw(const std::string & key, std::string & value) const;
void setRaw(const std::string & key, const std::string & value);
void enumerate(const std::string & key, Keys & range) const;
void removeRaw(const std::string & key);
std::string translateKey(const std::string & key) const;
~ConfigurationMapper();
private:
ConfigurationMapper(const ConfigurationMapper &);
ConfigurationMapper & operator=(const ConfigurationMapper &);
std::string _fromPrefix;
std::string _toPrefix;
AbstractConfiguration * _pConfig;
};
}
} // namespace Poco::Util
#endif // Util_ConfigurationMapper_INCLUDED

75
base/poco/Util/include/Poco/Util/WinRegistryConfiguration.h
View File

@ -1,75 +0,0 @@
//
// WinRegistryConfiguration.h
//
// Library: Util
// Package: Windows
// Module: WinRegistryConfiguration
//
// Definition of the WinRegistryConfiguration class.
//
// Copyright (c) 2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Util_WinRegistryConfiguration_INCLUDED
#define Util_WinRegistryConfiguration_INCLUDED
#include "Poco/String.h"
#include "Poco/Util/AbstractConfiguration.h"
#include "Poco/Util/Util.h"
namespace Poco
{
namespace Util
{
class Util_API WinRegistryConfiguration : public AbstractConfiguration
/// An implementation of AbstractConfiguration that stores configuration data
/// in the Windows registry.
///
/// Removing key is not supported. An attempt to remove a key results
/// in a NotImplementedException being thrown.
{
public:
WinRegistryConfiguration(const std::string & rootPath, REGSAM extraSam = 0);
/// Creates the WinRegistryConfiguration.
/// The rootPath must start with one of the root key names
/// like HKEY_CLASSES_ROOT, e.g. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services.
/// All further keys are relative to the root path and can be
/// dot separated, e.g. the path MyService.ServiceName will be converted to
/// HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MyService\ServiceName.
/// The extraSam parameter will be passed along to WinRegistryKey, to control
/// registry virtualization for example.
protected:
~WinRegistryConfiguration();
/// Destroys the WinRegistryConfiguration.
bool getRaw(const std::string & key, std::string & value) const;
void setRaw(const std::string & key, const std::string & value);
void enumerate(const std::string & key, Keys & range) const;
void removeRaw(const std::string & key);
std::string convertToRegFormat(const std::string & key, std::string & keyName) const;
/// Takes a key in the format of A.B.C and converts it to
/// registry format A\B\C, the last entry is the keyName, the rest is returned as path
friend class WinConfigurationTest;
private:
std::string _rootPath;
REGSAM _extraSam;
};
}
} // namespace Poco::Util
#endif // Util_WinRegistryConfiguration_INCLUDED

199
base/poco/Util/include/Poco/Util/WinRegistryKey.h
View File

@ -1,199 +0,0 @@
//
// WinRegistryKey.h
//
// Library: Util
// Package: Windows
// Module: WinRegistryKey
//
// Definition of the WinRegistryKey class.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Util_WinRegistryKey_INCLUDED
#define Util_WinRegistryKey_INCLUDED
#include <vector>
#include "Poco/UnWindows.h"
#include "Poco/Util/Util.h"
namespace Poco
{
namespace Util
{
class Util_API WinRegistryKey
/// This class implements a convenient interface to the
/// Windows Registry.
///
/// This class is only available on Windows platforms.
{
public:
typedef std::vector<std::string> Keys;
typedef std::vector<std::string> Values;
enum Type
{
REGT_NONE = 0,
REGT_STRING = 1,
REGT_STRING_EXPAND = 2,
REGT_BINARY = 3,
REGT_DWORD = 4,
REGT_DWORD_BIG_ENDIAN = 5,
REGT_LINK = 6,
REGT_MULTI_STRING = 7,
REGT_RESOURCE_LIST = 8,
REGT_FULL_RESOURCE_DESCRIPTOR = 9,
REGT_RESOURCE_REQUIREMENTS_LIST = 10,
REGT_QWORD = 11
};
WinRegistryKey(const std::string & key, bool readOnly = false, REGSAM extraSam = 0);
/// Creates the WinRegistryKey.
///
/// The key must start with one of the root key names
/// like HKEY_CLASSES_ROOT, e.g. HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services.
///
/// If readOnly is true, then only read access to the registry
/// is available and any attempt to write to the registry will
/// result in an exception.
///
/// extraSam is used to pass extra flags (in addition to KEY_READ and KEY_WRITE)
/// to the samDesired argument of RegOpenKeyEx() or RegCreateKeyEx().
WinRegistryKey(HKEY hRootKey, const std::string & subKey, bool readOnly = false, REGSAM extraSam = 0);
/// Creates the WinRegistryKey.
///
/// If readOnly is true, then only read access to the registry
/// is available and any attempt to write to the registry will
/// result in an exception.
///
/// extraSam is used to pass extra flags (in addition to KEY_READ and KEY_WRITE)
/// to the samDesired argument of RegOpenKeyEx() or RegCreateKeyEx().
~WinRegistryKey();
/// Destroys the WinRegistryKey.
void setString(const std::string & name, const std::string & value);
/// Sets the string value (REG_SZ) with the given name.
/// An empty name denotes the default value.
std::string getString(const std::string & name);
/// Returns the string value (REG_SZ) with the given name.
/// An empty name denotes the default value.
///
/// Throws a NotFoundException if the value does not exist.
void setStringExpand(const std::string & name, const std::string & value);
/// Sets the expandable string value (REG_EXPAND_SZ) with the given name.
/// An empty name denotes the default value.
std::string getStringExpand(const std::string & name);
/// Returns the string value (REG_EXPAND_SZ) with the given name.
/// An empty name denotes the default value.
/// All references to environment variables (%VAR%) in the string
/// are expanded.
///
/// Throws a NotFoundException if the value does not exist.
void setBinary(const std::string & name, const std::vector<char> & value);
/// Sets the string value (REG_BINARY) with the given name.
/// An empty name denotes the default value.
std::vector<char> getBinary(const std::string & name);
/// Returns the string value (REG_BINARY) with the given name.
/// An empty name denotes the default value.
///
/// Throws a NotFoundException if the value does not exist.
void setInt(const std::string & name, int value);
/// Sets the numeric (REG_DWORD) value with the given name.
/// An empty name denotes the default value.
int getInt(const std::string & name);
/// Returns the numeric value (REG_DWORD) with the given name.
/// An empty name denotes the default value.
///
/// Throws a NotFoundException if the value does not exist.
void setInt64(const std::string & name, Poco::Int64 value);
/// Sets the numeric (REG_QWORD) value with the given name.
/// An empty name denotes the default value.
Poco::Int64 getInt64(const std::string & name);
/// Returns the numeric value (REG_QWORD) with the given name.
/// An empty name denotes the default value.
///
/// Throws a NotFoundException if the value does not exist.
void deleteValue(const std::string & name);
/// Deletes the value with the given name.
///
/// Throws a NotFoundException if the value does not exist.
void deleteKey();
/// Recursively deletes the key and all subkeys.
bool exists();
/// Returns true iff the key exists.
Type type(const std::string & name);
/// Returns the type of the key value.
bool exists(const std::string & name);
/// Returns true iff the given value exists under that key.
void subKeys(Keys & keys);
/// Appends all subKey names to keys.
void values(Values & vals);
/// Appends all value names to vals;
bool isReadOnly() const;
/// Returns true iff the key has been opened for read-only access only.
protected:
void open();
void close();
std::string key() const;
std::string key(const std::string & valueName) const;
HKEY handle();
void handleSetError(const std::string & name);
static HKEY handleFor(const std::string & rootKey);
private:
WinRegistryKey();
WinRegistryKey(const WinRegistryKey &);
WinRegistryKey & operator=(const WinRegistryKey &);
HKEY _hRootKey;
std::string _subKey;
HKEY _hKey;
bool _readOnly;
REGSAM _extraSam;
};
//
// inlines
//
inline bool WinRegistryKey::isReadOnly() const
{
return _readOnly;
}
}
} // namespace Poco::Util
#endif // Util_WinRegistryKey_INCLUDED

140
base/poco/Util/include/Poco/Util/WinService.h
View File

@ -1,140 +0,0 @@
//
// WinService.h
//
// Library: Util
// Package: Windows
// Module: WinService
//
// Definition of the WinService class.
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#ifndef Util_WinService_INCLUDED
#define Util_WinService_INCLUDED
#include "Poco/UnWindows.h"
#include "Poco/Util/Util.h"
# define POCO_LPQUERY_SERVICE_CONFIG LPQUERY_SERVICE_CONFIGA
namespace Poco
{
namespace Util
{
class Util_API WinService
/// This class provides an object-oriented interface to
/// the Windows Service Control Manager for registering,
/// unregistering, configuring, starting and stopping
/// services.
///
/// This class is only available on Windows platforms.
{
public:
enum Startup
{
SVC_AUTO_START,
SVC_MANUAL_START,
SVC_DISABLED
};
WinService(const std::string & name);
/// Creates the WinService, using the given service name.
~WinService();
/// Destroys the WinService.
const std::string & name() const;
/// Returns the service name.
std::string displayName() const;
/// Returns the service's display name.
std::string path() const;
/// Returns the path to the service executable.
///
/// Throws a NotFoundException if the service has not been registered.
void registerService(const std::string & path, const std::string & displayName);
/// Creates a Windows service with the executable specified by path
/// and the given displayName.
///
/// Throws a ExistsException if the service has already been registered.
void registerService(const std::string & path);
/// Creates a Windows service with the executable specified by path
/// and the given displayName. The service name is used as display name.
///
/// Throws a ExistsException if the service has already been registered.
void unregisterService();
/// Deletes the Windows service.
///
/// Throws a NotFoundException if the service has not been registered.
bool isRegistered() const;
/// Returns true if the service has been registered with the Service Control Manager.
bool isRunning() const;
/// Returns true if the service is currently running.
void start();
/// Starts the service.
/// Does nothing if the service is already running.
///
/// Throws a NotFoundException if the service has not been registered.
void stop();
/// Stops the service.
/// Does nothing if the service is not running.
///
/// Throws a NotFoundException if the service has not been registered.
void setStartup(Startup startup);
/// Sets the startup mode for the service.
Startup getStartup() const;
/// Returns the startup mode for the service.
void setDescription(const std::string & description);
/// Sets the service description in the registry.
std::string getDescription() const;
/// Returns the service description from the registry.
static const int STARTUP_TIMEOUT;
protected:
static const std::string REGISTRY_KEY;
static const std::string REGISTRY_DESCRIPTION;
private:
void open() const;
bool tryOpen() const;
void close() const;
POCO_LPQUERY_SERVICE_CONFIG config() const;
WinService();
WinService(const WinService &);
WinService & operator=(const WinService &);
std::string _name;
SC_HANDLE _scmHandle;
mutable SC_HANDLE _svcHandle;
};
}
} // namespace Poco::Util
#endif // Util_WinService_INCLUDED

101
base/poco/Util/src/ConfigurationMapper.cpp
View File

@ -1,101 +0,0 @@
//
// ConfigurationMapper.cpp
//
// Library: Util
// Package: Configuration
// Module: ConfigurationMapper
//
// Copyright (c) 2004-2006, Applied Informatics Software Engineering GmbH.
// and Contributors.
//
// SPDX-License-Identifier: BSL-1.0
//
#include "Poco/Util/ConfigurationMapper.h"
namespace Poco {
namespace Util {
ConfigurationMapper::ConfigurationMapper(const std::string& fromPrefix, const std::string& toPrefix, AbstractConfiguration* pConfig):
_fromPrefix(fromPrefix),
_toPrefix(toPrefix),
_pConfig(pConfig)
{
poco_check_ptr (pConfig);
if (!_fromPrefix.empty()) _fromPrefix += '.';
if (!_toPrefix.empty()) _toPrefix += '.';
_pConfig->duplicate();
}
ConfigurationMapper::~ConfigurationMapper()
{
_pConfig->release();
}
bool ConfigurationMapper::getRaw(const std::string& key, std::string& value) const
{
std::string translatedKey = translateKey(key);
return _pConfig->getRaw(translatedKey, value);
}
void ConfigurationMapper::setRaw(const std::string& key, const std::string& value)
{
std::string translatedKey = translateKey(key);
_pConfig->setRaw(translatedKey, value);
}
void ConfigurationMapper::enumerate(const std::string& key, Keys& range) const
{
std::string cKey(key);
if (!cKey.empty()) cKey += '.';
std::string::size_type keyLen = cKey.length();
if (keyLen < _toPrefix.length())
{
if (_toPrefix.compare(0, keyLen, cKey) == 0)
{
std::string::size_type pos = _toPrefix.find_first_of('.', keyLen);
poco_assert_dbg(pos != std::string::npos);
range.push_back(_toPrefix.substr(keyLen, pos - keyLen));
}
}
else
{
std::string translatedKey;
if (cKey == _toPrefix)
{
translatedKey = _fromPrefix;
if (!translatedKey.empty())
translatedKey.resize(translatedKey.length() - 1);
}
else translatedKey = translateKey(key);
_pConfig->enumerate(translatedKey, range);
}
}
void ConfigurationMapper::removeRaw(const std::string& key)
{
std::string translatedKey = translateKey(key);
_pConfig->remove(translatedKey);
}
std::string ConfigurationMapper::translateKey(const std::string& key) const
{
std::string result(key);
if (result.compare(0, _toPrefix.size(), _toPrefix) == 0)
result.replace(0, _toPrefix.size(), _fromPrefix);
return result;
}
} } // namespace Poco::Util

3
cmake/sanitize.cmake
View File

@ -33,8 +33,7 @@ if (SANITIZE)
# RelWithDebInfo, and downgrade optimizations to -O1 but not to -Og, to
# keep the binary size down.
# TODO: try compiling with -Og and with ld.gold.
set (MSAN_FLAGS "-fsanitize=memory -fsanitize-memory-use-after-dtor -fsanitize-memory-track-origins -fno-optimize-sibling-calls -fsanitize-blacklist=${CMAKE_SOURCE_DIR}/tests/msan_suppressions.txt")
set (MSAN_FLAGS "-fsanitize=memory -fsanitize-memory-use-after-dtor -fsanitize-memory-track-origins -fno-optimize-sibling-calls -fPIC -fpie -fsanitize-blacklist=${CMAKE_SOURCE_DIR}/tests/msan_suppressions.txt")
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${SAN_FLAGS} ${MSAN_FLAGS}")
set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${SAN_FLAGS} ${MSAN_FLAGS}")

2
contrib/sysroot vendored

@ -1 +1 @@
Subproject commit f0081b2649b94837855f3bc7d05ef326b100bad8
Subproject commit e0d1b64da666afbfaa6f1ee0487c33f3fd2cd5cb

1
docker/docs/builder/Dockerfile
View File

@ -1,4 +1,3 @@
# rebuild in #36968
# docker build -t clickhouse/docs-builder .
# nodejs 17 prefers ipv6 and is broken in our environment
FROM node:16-alpine

2
docker/keeper/Dockerfile
View File

@ -32,7 +32,7 @@ RUN arch=${TARGETARCH:-amd64} \
esac
ARG REPOSITORY="https://s3.amazonaws.com/clickhouse-builds/22.4/31c367d3cd3aefd316778601ff6565119fe36682/package_release"
ARG VERSION="23.3.1.2823"
ARG VERSION="23.3.2.37"
ARG PACKAGES="clickhouse-keeper"
# user/group precreated explicitly with fixed uid/gid on purpose.

2
docker/server/Dockerfile.alpine
View File

@ -33,7 +33,7 @@ RUN arch=${TARGETARCH:-amd64} \
# lts / testing / prestable / etc
ARG REPO_CHANNEL="stable"
ARG REPOSITORY="https://packages.clickhouse.com/tgz/${REPO_CHANNEL}"
ARG VERSION="23.3.1.2823"
ARG VERSION="23.3.2.37"
ARG PACKAGES="clickhouse-client clickhouse-server clickhouse-common-static"
# user/group precreated explicitly with fixed uid/gid on purpose.

2
docker/server/Dockerfile.ubuntu
View File

@ -22,7 +22,7 @@ RUN sed -i "s|http://archive.ubuntu.com|${apt_archive}|g" /etc/apt/sources.list
ARG REPO_CHANNEL="stable"
ARG REPOSITORY="deb https://packages.clickhouse.com/deb ${REPO_CHANNEL} main"
ARG VERSION="23.3.1.2823"
ARG VERSION="23.3.2.37"
ARG PACKAGES="clickhouse-client clickhouse-server clickhouse-common-static"
# set non-empty deb_location_url url to create a docker image

41
docker/test/util/process_functional_tests_result.py
View File

@ -18,7 +18,7 @@ SUCCESS_FINISH_SIGNS = ["All tests have finished", "No tests were run"]
RETRIES_SIGN = "Some tests were restarted"
def process_test_log(log_path):
def process_test_log(log_path, broken_tests):
total = 0
skipped = 0
unknown = 0
@ -62,8 +62,12 @@ def process_test_log(log_path):
failed += 1
test_results.append((test_name, "Timeout", test_time, []))
elif FAIL_SIGN in line:
failed += 1
test_results.append((test_name, "FAIL", test_time, []))
if test_name in broken_tests:
success += 1
test_results.append((test_name, "OK", test_time, []))
else:
failed += 1
test_results.append((test_name, "FAIL", test_time, []))
elif UNKNOWN_SIGN in line:
unknown += 1
test_results.append((test_name, "FAIL", test_time, []))
@ -71,8 +75,21 @@ def process_test_log(log_path):
skipped += 1
test_results.append((test_name, "SKIPPED", test_time, []))
else:
success += int(OK_SIGN in line)
test_results.append((test_name, "OK", test_time, []))
if OK_SIGN in line and test_name in broken_tests:
failed += 1
test_results.append(
(
test_name,
"FAIL",
test_time,
[
"Test is expected to fail! Please, update broken_tests.txt!\n"
],
)
)
else:
success += int(OK_SIGN in line)
test_results.append((test_name, "OK", test_time, []))
test_end = False
elif (
len(test_results) > 0 and test_results[-1][1] == "FAIL" and not test_end
@ -110,7 +127,7 @@ def process_test_log(log_path):
)
def process_result(result_path):
def process_result(result_path, broken_tests):
test_results = []
state = "success"
description = ""
@ -134,7 +151,7 @@ def process_result(result_path):
success_finish,
retries,
test_results,
) = process_test_log(result_path)
) = process_test_log(result_path, broken_tests)
is_flacky_check = 1 < int(os.environ.get("NUM_TRIES", 1))
logging.info("Is flaky check: %s", is_flacky_check)
# If no tests were run (success == 0) it indicates an error (e.g. server did not start or crashed immediately)
@ -186,9 +203,17 @@ if __name__ == "__main__":
parser.add_argument("--in-results-dir", default="/test_output/")
parser.add_argument("--out-results-file", default="/test_output/test_results.tsv")
parser.add_argument("--out-status-file", default="/test_output/check_status.tsv")
parser.add_argument("--broken-tests", default="/broken_tests.txt")
args = parser.parse_args()
state, description, test_results = process_result(args.in_results_dir)
broken_tests = list()
if os.path.exists(args.broken_tests):
logging.info(f"File {args.broken_tests} with broken tests found")
with open(args.broken_tests) as f:
broken_tests = f.read().splitlines()
logging.info(f"Broken tests in the list: {len(broken_tests)}")
state, description, test_results = process_result(args.in_results_dir, broken_tests)
logging.info("Result parsed")
status = (state, description)
write_results(args.out_results_file, args.out_status_file, test_results, status)

22
docs/changelogs/v22.8.17.17-lts.md Normal file
View File

@ -0,0 +1,22 @@
---
sidebar_position: 1
sidebar_label: 2023
---
# 2023 Changelog
### ClickHouse release v22.8.17.17-lts (df7f2ef0b41) FIXME as compared to v22.8.16.32-lts (7c4be737bd0)
#### Improvement
* Backported in [#48157](https://github.com/ClickHouse/ClickHouse/issues/48157): Fixed `UNKNOWN_TABLE` exception when attaching to a materialized view that has dependent tables that are not available. This might be useful when trying to restore state from a backup. [#47975](https://github.com/ClickHouse/ClickHouse/pull/47975) ([MikhailBurdukov](https://github.com/MikhailBurdukov)).
#### Build/Testing/Packaging Improvement
* Backported in [#48957](https://github.com/ClickHouse/ClickHouse/issues/48957): After the recent update, the `dockerd` requires `--tlsverify=false` together with the http port explicitly. [#48924](https://github.com/ClickHouse/ClickHouse/pull/48924) ([Mikhail f. Shiryaev](https://github.com/Felixoid)).
#### Bug Fix (user-visible misbehavior in an official stable release)
* Fix explain graph with projection [#47473](https://github.com/ClickHouse/ClickHouse/pull/47473) ([flynn](https://github.com/ucasfl)).
* Remove a feature [#48195](https://github.com/ClickHouse/ClickHouse/pull/48195) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix possible segfault in cache [#48469](https://github.com/ClickHouse/ClickHouse/pull/48469) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Fix bug in Keeper when a node is not created with scheme `auth` in ACL sometimes. [#48595](https://github.com/ClickHouse/ClickHouse/pull/48595) ([Aleksei Filatov](https://github.com/aalexfvk)).

28
docs/changelogs/v23.1.7.30-stable.md Normal file
View File

@ -0,0 +1,28 @@
---
sidebar_position: 1
sidebar_label: 2023
---
# 2023 Changelog
### ClickHouse release v23.1.7.30-stable (c94dba6e023) FIXME as compared to v23.1.6.42-stable (783ddf67991)
#### Improvement
* Backported in [#48161](https://github.com/ClickHouse/ClickHouse/issues/48161): Fixed `UNKNOWN_TABLE` exception when attaching to a materialized view that has dependent tables that are not available. This might be useful when trying to restore state from a backup. [#47975](https://github.com/ClickHouse/ClickHouse/pull/47975) ([MikhailBurdukov](https://github.com/MikhailBurdukov)).
#### Build/Testing/Packaging Improvement
* Backported in [#48585](https://github.com/ClickHouse/ClickHouse/issues/48585): Update time zones. The following were updated: Africa/Cairo, Africa/Casablanca, Africa/El_Aaiun, America/Bogota, America/Cambridge_Bay, America/Ciudad_Juarez, America/Godthab, America/Inuvik, America/Iqaluit, America/Nuuk, America/Ojinaga, America/Pangnirtung, America/Rankin_Inlet, America/Resolute, America/Whitehorse, America/Yellowknife, Asia/Gaza, Asia/Hebron, Asia/Kuala_Lumpur, Asia/Singapore, Canada/Yukon, Egypt, Europe/Kirov, Europe/Volgograd, Singapore. [#48572](https://github.com/ClickHouse/ClickHouse/pull/48572) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Backported in [#48958](https://github.com/ClickHouse/ClickHouse/issues/48958): After the recent update, the `dockerd` requires `--tlsverify=false` together with the http port explicitly. [#48924](https://github.com/ClickHouse/ClickHouse/pull/48924) ([Mikhail f. Shiryaev](https://github.com/Felixoid)).
#### Bug Fix (user-visible misbehavior in an official stable release)
* Fix race in grace hash join with limit [#47153](https://github.com/ClickHouse/ClickHouse/pull/47153) ([Vladimir C](https://github.com/vdimir)).
* Fix explain graph with projection [#47473](https://github.com/ClickHouse/ClickHouse/pull/47473) ([flynn](https://github.com/ucasfl)).
* Fix crash in polygonsSymDifferenceCartesian [#47702](https://github.com/ClickHouse/ClickHouse/pull/47702) ([pufit](https://github.com/pufit)).
* Remove a feature [#48195](https://github.com/ClickHouse/ClickHouse/pull/48195) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* ClickHouse startup error when loading a distributed table that depends on a dictionary [#48419](https://github.com/ClickHouse/ClickHouse/pull/48419) ([MikhailBurdukov](https://github.com/MikhailBurdukov)).
* Fix possible segfault in cache [#48469](https://github.com/ClickHouse/ClickHouse/pull/48469) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Fix nested map for keys of IP and UUID types [#48556](https://github.com/ClickHouse/ClickHouse/pull/48556) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
* Fix bug in Keeper when a node is not created with scheme `auth` in ACL sometimes. [#48595](https://github.com/ClickHouse/ClickHouse/pull/48595) ([Aleksei Filatov](https://github.com/aalexfvk)).
* Fix IPv4 comparable with UInt [#48611](https://github.com/ClickHouse/ClickHouse/pull/48611) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).

29
docs/changelogs/v23.2.6.34-stable.md Normal file
View File

@ -0,0 +1,29 @@
---
sidebar_position: 1
sidebar_label: 2023
---
# 2023 Changelog
### ClickHouse release v23.2.6.34-stable (570190045b0) FIXME as compared to v23.2.5.46-stable (b50faecbb12)
#### Improvement
* Backported in [#48709](https://github.com/ClickHouse/ClickHouse/issues/48709): Formatter '%M' in function formatDateTime() now prints the month name instead of the minutes. This makes the behavior consistent with MySQL. The previous behavior can be restored using setting "formatdatetime_parsedatetime_m_is_month_name = 0". [#47246](https://github.com/ClickHouse/ClickHouse/pull/47246) ([Robert Schulze](https://github.com/rschu1ze)).
#### Build/Testing/Packaging Improvement
* Backported in [#48587](https://github.com/ClickHouse/ClickHouse/issues/48587): Update time zones. The following were updated: Africa/Cairo, Africa/Casablanca, Africa/El_Aaiun, America/Bogota, America/Cambridge_Bay, America/Ciudad_Juarez, America/Godthab, America/Inuvik, America/Iqaluit, America/Nuuk, America/Ojinaga, America/Pangnirtung, America/Rankin_Inlet, America/Resolute, America/Whitehorse, America/Yellowknife, Asia/Gaza, Asia/Hebron, Asia/Kuala_Lumpur, Asia/Singapore, Canada/Yukon, Egypt, Europe/Kirov, Europe/Volgograd, Singapore. [#48572](https://github.com/ClickHouse/ClickHouse/pull/48572) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Backported in [#48959](https://github.com/ClickHouse/ClickHouse/issues/48959): After the recent update, the `dockerd` requires `--tlsverify=false` together with the http port explicitly. [#48924](https://github.com/ClickHouse/ClickHouse/pull/48924) ([Mikhail f. Shiryaev](https://github.com/Felixoid)).
#### Bug Fix (user-visible misbehavior in an official stable release)
* Fix race in grace hash join with limit [#47153](https://github.com/ClickHouse/ClickHouse/pull/47153) ([Vladimir C](https://github.com/vdimir)).
* Fix explain graph with projection [#47473](https://github.com/ClickHouse/ClickHouse/pull/47473) ([flynn](https://github.com/ucasfl)).
* Fix crash in polygonsSymDifferenceCartesian [#47702](https://github.com/ClickHouse/ClickHouse/pull/47702) ([pufit](https://github.com/pufit)).
* Remove a feature [#48195](https://github.com/ClickHouse/ClickHouse/pull/48195) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix cpu usage in rabbitmq (was worsened in 23.2 after [#44404](https://github.com/ClickHouse/ClickHouse/issues/44404)) [#48311](https://github.com/ClickHouse/ClickHouse/pull/48311) ([Kseniia Sumarokova](https://github.com/kssenii)).
* ClickHouse startup error when loading a distributed table that depends on a dictionary [#48419](https://github.com/ClickHouse/ClickHouse/pull/48419) ([MikhailBurdukov](https://github.com/MikhailBurdukov)).
* Fix possible segfault in cache [#48469](https://github.com/ClickHouse/ClickHouse/pull/48469) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Fix nested map for keys of IP and UUID types [#48556](https://github.com/ClickHouse/ClickHouse/pull/48556) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
* Fix bug in Keeper when a node is not created with scheme `auth` in ACL sometimes. [#48595](https://github.com/ClickHouse/ClickHouse/pull/48595) ([Aleksei Filatov](https://github.com/aalexfvk)).
* Fix IPv4 comparable with UInt [#48611](https://github.com/ClickHouse/ClickHouse/pull/48611) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).

35
docs/changelogs/v23.3.2.37-lts.md Normal file
View File

@ -0,0 +1,35 @@
---
sidebar_position: 1
sidebar_label: 2023
---
# 2023 Changelog
### ClickHouse release v23.3.2.37-lts (1b144bcd101) FIXME as compared to v23.3.1.2823-lts (46e85357ce2)
#### Improvement
* Backported in [#48459](https://github.com/ClickHouse/ClickHouse/issues/48459): Formatter '%M' in function formatDateTime() now prints the month name instead of the minutes. This makes the behavior consistent with MySQL. The previous behavior can be restored using setting "formatdatetime_parsedatetime_m_is_month_name = 0". [#47246](https://github.com/ClickHouse/ClickHouse/pull/47246) ([Robert Schulze](https://github.com/rschu1ze)).
* Backported in [#48842](https://github.com/ClickHouse/ClickHouse/issues/48842): Fix some mysql related settings not being handled with mysql dictionary source + named collection. Closes [#48402](https://github.com/ClickHouse/ClickHouse/issues/48402). [#48759](https://github.com/ClickHouse/ClickHouse/pull/48759) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Backported in [#49035](https://github.com/ClickHouse/ClickHouse/issues/49035): Add fallback to password authentication when authentication with SSL user certificate has failed. Closes [#48974](https://github.com/ClickHouse/ClickHouse/issues/48974). [#48989](https://github.com/ClickHouse/ClickHouse/pull/48989) ([Nikolay Degterinsky](https://github.com/evillique)).
#### Build/Testing/Packaging Improvement
* Backported in [#48589](https://github.com/ClickHouse/ClickHouse/issues/48589): Update time zones. The following were updated: Africa/Cairo, Africa/Casablanca, Africa/El_Aaiun, America/Bogota, America/Cambridge_Bay, America/Ciudad_Juarez, America/Godthab, America/Inuvik, America/Iqaluit, America/Nuuk, America/Ojinaga, America/Pangnirtung, America/Rankin_Inlet, America/Resolute, America/Whitehorse, America/Yellowknife, Asia/Gaza, Asia/Hebron, Asia/Kuala_Lumpur, Asia/Singapore, Canada/Yukon, Egypt, Europe/Kirov, Europe/Volgograd, Singapore. [#48572](https://github.com/ClickHouse/ClickHouse/pull/48572) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Backported in [#48960](https://github.com/ClickHouse/ClickHouse/issues/48960): After the recent update, the `dockerd` requires `--tlsverify=false` together with the http port explicitly. [#48924](https://github.com/ClickHouse/ClickHouse/pull/48924) ([Mikhail f. Shiryaev](https://github.com/Felixoid)).
#### Bug Fix (user-visible misbehavior in an official stable release)
* Remove a feature [#48195](https://github.com/ClickHouse/ClickHouse/pull/48195) ([Alexey Milovidov](https://github.com/alexey-milovidov)).
* Fix cpu usage in rabbitmq (was worsened in 23.2 after [#44404](https://github.com/ClickHouse/ClickHouse/issues/44404)) [#48311](https://github.com/ClickHouse/ClickHouse/pull/48311) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Fix ThreadPool for DistributedSink and use StrongTypedef for CurrentMetrics/ProfileEvents/StatusInfo to avoid further errors [#48314](https://github.com/ClickHouse/ClickHouse/pull/48314) ([Azat Khuzhin](https://github.com/azat)).
* Reset downloader for cache file segment in TemporaryFileStream [#48386](https://github.com/ClickHouse/ClickHouse/pull/48386) ([Vladimir C](https://github.com/vdimir)).
* ClickHouse startup error when loading a distributed table that depends on a dictionary [#48419](https://github.com/ClickHouse/ClickHouse/pull/48419) ([MikhailBurdukov](https://github.com/MikhailBurdukov)).
* Fix possible segfault in cache [#48469](https://github.com/ClickHouse/ClickHouse/pull/48469) ([Kseniia Sumarokova](https://github.com/kssenii)).
* Fix nested map for keys of IP and UUID types [#48556](https://github.com/ClickHouse/ClickHouse/pull/48556) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
* Fix bug in Keeper when a node is not created with scheme `auth` in ACL sometimes. [#48595](https://github.com/ClickHouse/ClickHouse/pull/48595) ([Aleksei Filatov](https://github.com/aalexfvk)).
* Fix IPv4 comparable with UInt [#48611](https://github.com/ClickHouse/ClickHouse/pull/48611) ([Yakov Olkhovskiy](https://github.com/yakov-olkhovskiy)).
#### NOT FOR CHANGELOG / INSIGNIFICANT
* Batch fix for projections analysis with analyzer. [#48357](https://github.com/ClickHouse/ClickHouse/pull/48357) ([Nikolai Kochetov](https://github.com/KochetovNicolai)).
* Fix a confusing warning about interserver mode [#48793](https://github.com/ClickHouse/ClickHouse/pull/48793) ([Alexander Tokmakov](https://github.com/tavplubix)).

2
docs/en/development/build-cross-s390x.md
View File

@ -90,7 +90,7 @@ Process 1 stopped
## Visual Studio Code integration
- (CodeLLDB extension)[https://github.com/vadimcn/vscode-lldb] is required for visual debugging, the (Command Variable)[https://github.com/rioj7/command-variable] extension can help dynamic launches if using (cmake variants)[https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/variants.md].
- [CodeLLDB extension](https://github.com/vadimcn/vscode-lldb) is required for visual debugging, the [Command Variable](https://github.com/rioj7/command-variable) extension can help dynamic launches if using [cmake variants](https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/variants.md).
- Make sure to set the backend to your llvm installation eg. `"lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-15.so"`
- Launcher:
```json

91
docs/en/development/build.md
View File

@ -13,9 +13,11 @@ Supported platforms:
- AArch64
- Power9 (experimental)
## Normal Build for Development on Ubuntu
## Building on Ubuntu
The following tutorial is based on the Ubuntu Linux system. With appropriate changes, it should also work on any other Linux distribution.
The following tutorial is based on Ubuntu Linux.
With appropriate changes, it should also work on any other Linux distribution.
The minimum recommended Ubuntu version for development is 22.04 LTS.
### Install Prerequisites {#install-prerequisites}
@ -23,13 +25,11 @@ The following tutorial is based on the Ubuntu Linux system. With appropriate cha
sudo apt-get install git cmake ccache python3 ninja-build yasm gawk
```
Or cmake3 instead of cmake on older systems.
### Install and Use the Clang compiler
### Install the latest clang (recommended)
On Ubuntu/Debian you can use LLVM's automatic installation script, see [here](https://apt.llvm.org/).
On Ubuntu/Debian you can use the automatic installation script (check [official webpage](https://apt.llvm.org/))
```bash
``` bash
sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
```
@ -40,19 +40,17 @@ sudo apt-get install software-properties-common
sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
```
For other Linux distribution - check the availability of the [prebuild packages](https://releases.llvm.org/download.html) or build clang [from sources](https://clang.llvm.org/get_started.html).
For other Linux distribution - check the availability of LLVM's [prebuild packages](https://releases.llvm.org/download.html).
#### Use the latest clang for Builds
As of April 2023, any version of Clang >= 15 will work.
GCC as a compiler is not supported
To build with a specific Clang version:
``` bash
export CC=clang-15
export CXX=clang++-15
```
In this example we use version 15 that is the latest as of Sept 2022.
Gcc cannot be used.
### Checkout ClickHouse Sources {#checkout-clickhouse-sources}
``` bash
@ -70,79 +68,46 @@ git clone --recursive --shallow-submodules https://github.com/ClickHouse/ClickHo
``` bash
cd ClickHouse
mkdir build
cd build
cmake ..
ninja
cmake -S . -B build
cmake --build build # or: `cd build; ninja`
```
To create an executable, run `ninja clickhouse`.
This will create the `programs/clickhouse` executable, which can be used with `client` or `server` arguments.
To create an executable, run `cmake --build --target clickhouse` (or: `cd build; ninja clickhouse`).
This will create executable `build/programs/clickhouse` which can be used with `client` or `server` arguments.
## How to Build ClickHouse on Any Linux {#how-to-build-clickhouse-on-any-linux}
## Building on Any Linux {#how-to-build-clickhouse-on-any-linux}
The build requires the following components:
- Git (is used only to checkout the sources, it’s not needed for the build)
- CMake 3.15 or newer
- Git (used to checkout the sources, not needed for the build)
- CMake 3.20 or newer
- Compiler: Clang 15 or newer
- Linker: lld 15 or newer
- Ninja
- C++ compiler: clang-15 or newer
- Linker: lld
- Yasm
- Gawk
If all the components are installed, you may build in the same way as the steps above.
Example for Ubuntu Eoan:
``` bash
sudo apt update
sudo apt install git cmake ninja-build clang++ python yasm gawk
git clone --recursive https://github.com/ClickHouse/ClickHouse.git
mkdir build && cd build
cmake ../ClickHouse
ninja
```
Example for OpenSUSE Tumbleweed:
``` bash
sudo zypper install git cmake ninja clang-c++ python lld yasm gawk
git clone --recursive https://github.com/ClickHouse/ClickHouse.git
mkdir build && cd build
cmake ../ClickHouse
ninja
mkdir build
cmake -S . -B build
cmake --build build
```
Example for Fedora Rawhide:
``` bash
sudo yum update
sudo yum --nogpg install git cmake make clang python3 ccache yasm gawk
git clone --recursive https://github.com/ClickHouse/ClickHouse.git
mkdir build && cd build
cmake ../ClickHouse
make -j $(nproc)
```
Here is an example of how to build `clang` and all the llvm infrastructure from sources:
```
git clone git@github.com:llvm/llvm-project.git
mkdir llvm-build && cd llvm-build
cmake -DCMAKE_BUILD_TYPE:STRING=Release -DLLVM_ENABLE_PROJECTS=all ../llvm-project/llvm/
make -j16
sudo make install
hash clang
clang --version
```
You can install the older clang like clang-11 from packages and then use it to build the new clang from sources.
Here is an example of how to install the new `cmake` from the official website:
```
wget https://github.com/Kitware/CMake/releases/download/v3.22.2/cmake-3.22.2-linux-x86_64.sh
chmod +x cmake-3.22.2-linux-x86_64.sh
./cmake-3.22.2-linux-x86_64.sh
export PATH=/home/milovidov/work/cmake-3.22.2-linux-x86_64/bin/:${PATH}
hash cmake
mkdir build
cmake -S . -B build
cmake --build build
```
## You Don’t Have to Build ClickHouse {#you-dont-have-to-build-clickhouse}

2
docs/en/interfaces/cli.md
View File

@ -119,7 +119,7 @@ When processing a query, the client shows:
1. Progress, which is updated no more than 10 times per second (by default). For quick queries, the progress might not have time to be displayed.
2. The formatted query after parsing, for debugging.
3. The result in the specified format.
4. The number of lines in the result, the time passed, and the average speed of query processing.
4. The number of lines in the result, the time passed, and the average speed of query processing. All data amounts refer to uncompressed data.
You can cancel a long query by pressing Ctrl+C. However, you will still need to wait for a little for the server to abort the request. It is not possible to cancel a query at certain stages. If you do not wait and press Ctrl+C a second time, the client will exit.

2
docs/en/interfaces/formats.md
View File

@ -2241,7 +2241,7 @@ $ clickhouse-client --query="SELECT * FROM {some_table} FORMAT Arrow" > {filenam
- [input_format_arrow_allow_missing_columns](/docs/en/operations/settings/settings-formats.md/#input_format_arrow_allow_missing_columns) - allow missing columns while reading Arrow data. Default value - `false`.
- [input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference](/docs/en/operations/settings/settings-formats.md/#input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference) - allow skipping columns with unsupported types while schema inference for Arrow format. Default value - `false`.
- [output_format_arrow_fixed_string_as_fixed_byte_array](/docs/en/operations/settings/settings-formats.md/#output_format_arrow_fixed_string_as_fixed_byte_array) - use Arrow FIXED_SIZE_BINARY type instead of Binary/String for FixedString columns. Default value - `true`.
- [output_format_arrow_compression_method](/docs/en/operations/settings/settings-formats.md/#output_format_arrow_compression_method) - compression method used in output Arrow format. Default value - `none`.
- [output_format_arrow_compression_method](/docs/en/operations/settings/settings-formats.md/#output_format_arrow_compression_method) - compression method used in output Arrow format. Default value - `lz4_frame`.
## ArrowStream {#data-format-arrow-stream}

27
docs/en/operations/query-cache.md
View File

@ -88,6 +88,33 @@ If the query was aborted due to an exception or user cancellation, no entry is w
The size of the query cache in bytes, the maximum number of cache entries and the maximum size of individual cache entries (in bytes and in
records) can be configured using different [server configuration options](server-configuration-parameters/settings.md#server_configuration_parameters_query-cache).
It is also possible to limit the cache usage of individual users using [settings profiles](settings/settings-profiles.md) and [settings
constraints](settings/constraints-on-settings.md). More specifically, you can restrict the maximum amount of memory (in bytes) a user may
allocate in the query cache and the the maximum number of stored query results. For that, first provide configurations
[query_cache_max_size_in_bytes](settings/settings.md#query-cache-max-size-in-bytes) and
[query_cache_max_entries](settings/settings.md#query-cache-size-max-items) in a user profile in `users.xml`, then make both settings
readonly:
``` xml
<profiles>
<default>
<!-- The maximum cache size in bytes for user/profile 'default' -->
<query_cache_max_size_in_bytes>10000</query_cache_max_size_in_bytes>
<!-- The maximum number of SELECT query results stored in the cache for user/profile 'default' -->
<query_cache_max_entries>100</query_cache_max_entries>
<!-- Make both settings read-only so the user cannot change them -->
<constraints>
<query_cache_max_size_in_bytes>
<readonly/>
</query_cache_max_size_in_bytes>
<query_cache_max_entries>
<readonly/>
<query_cache_max_entries>
</constraints>
</default>
</profiles>
```
To define how long a query must run at least such that its result can be cached, you can use setting
[query_cache_min_query_duration](settings/settings.md#query-cache-min-query-duration). For example, the result of query

14
docs/en/operations/server-configuration-parameters/settings.md
View File

@ -1382,25 +1382,25 @@ If the table does not exist, ClickHouse will create it. If the structure of the
The following settings are available:
- `max_size`: The maximum cache size in bytes. 0 means the query cache is disabled. Default value: `1073741824` (1 GiB).
- `max_size_in_bytes`: The maximum cache size in bytes. 0 means the query cache is disabled. Default value: `1073741824` (1 GiB).
- `max_entries`: The maximum number of `SELECT` query results stored in the cache. Default value: `1024`.
- `max_entry_size`: The maximum size in bytes `SELECT` query results may have to be saved in the cache. Default value: `1048576` (1 MiB).
- `max_entry_rows`: The maximum number of rows `SELECT` query results may have to be saved in the cache. Default value: `30000000` (30 mil).
- `max_entry_size_in_bytes`: The maximum size in bytes `SELECT` query results may have to be saved in the cache. Default value: `1048576` (1 MiB).
- `max_entry_size_in_rows`: The maximum number of rows `SELECT` query results may have to be saved in the cache. Default value: `30000000` (30 mil).
Changed settings take effect immediately.
:::note
Data for the query cache is allocated in DRAM. If memory is scarce, make sure to set a small value for `max_size` or disable the query cache altogether.
Data for the query cache is allocated in DRAM. If memory is scarce, make sure to set a small value for `max_size_in_bytes` or disable the query cache altogether.
:::
**Example**
```xml
<query_cache>
<max_size>1073741824</max_size>
<max_size_in_bytes>1073741824</max_size_in_bytes>
<max_entries>1024</max_entries>
<max_entry_size>1048576</max_entry_size>
<max_entry_rows>30000000</max_entry_rows>
<max_entry_size_in_bytes>1048576</max_entry_size_in_bytes>
<max_entry_size_in_rows>30000000</max_entry_size_in_rows>
</query_cache>
```

2
docs/en/operations/settings/constraints-on-settings.md
View File

@ -40,7 +40,7 @@ If the user tries to violate the constraints an exception is thrown and the sett
There are supported few types of constraints: `min`, `max`, `readonly` (with alias `const`) and `changeable_in_readonly`. The `min` and `max` constraints specify upper and lower boundaries for a numeric setting and can be used in combination. The `readonly` or `const` constraint specifies that the user cannot change the corresponding setting at all. The `changeable_in_readonly` constraint type allows user to change the setting within `min`/`max` range even if `readonly` setting is set to 1, otherwise settings are not allow to be changed in `readonly=1` mode. Note that `changeable_in_readonly` is supported only if `settings_constraints_replace_previous` is enabled:
``` xml
<access_control_improvements>
<settings_constraints_replace_previous>true<settings_constraints_replace_previous>
<settings_constraints_replace_previous>true</settings_constraints_replace_previous>
</access_control_improvements>
```

26
docs/en/operations/settings/settings.md
View File

@ -890,7 +890,7 @@ Write time that processor spent during execution/waiting for data to `system.pro
See also:
- [`system.processors_profile_log`](../../operations/system-tables/processors_profile_log.md#system-processors_profile_log)
- [`system.processors_profile_log`](../../operations/system-tables/processors_profile_log.md)
- [`EXPLAIN PIPELINE`](../../sql-reference/statements/explain.md#explain-pipeline)
## max_insert_block_size {#settings-max_insert_block_size}
@ -1123,7 +1123,7 @@ Could be used for throttling speed when replicating the data to add or replace n
The timeout in milliseconds for connecting to a remote server for a Distributed table engine, if the ‘shard’ and ‘replica’ sections are used in the cluster definition.
If unsuccessful, several attempts are made to connect to various replicas.
Default value: 50.
Default value: 1000.
## connection_pool_max_wait_ms {#connection-pool-max-wait-ms}
@ -1512,6 +1512,26 @@ Possible values:
Default value: `0`.
## query_cache_max_size_in_bytes {#query-cache-max-size-in-bytes}
The maximum amount of memory (in bytes) the current user may allocate in the query cache. 0 means unlimited.
Possible values:
- Positive integer >= 0.
Default value: 0 (no restriction).
## query_cache_max_entries {#query-cache-max-entries}
The maximum number of query results the current user may store in the query cache. 0 means unlimited.
Possible values:
- Positive integer >= 0.
Default value: 0 (no restriction).
## insert_quorum {#settings-insert_quorum}
Enables the quorum writes.
@ -4102,7 +4122,7 @@ Enabled by default.
If we can't establish connection with replica after this timeout in hedged requests, we start working with the next replica without cancelling connection to the previous.
Timeout value is in milliseconds.
Default value: `100`.
Default value: `50`.
## receive_data_timeout {#receive_data_timeout}

583
docs/en/operations/system-tables/asynchronous_metrics.md
View File

@ -32,6 +32,589 @@ SELECT * FROM system.asynchronous_metrics LIMIT 10
└─────────────────────────────────────────┴────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```
## Metric descriptions
### AsynchronousHeavyMetricsCalculationTimeSpent
Time in seconds spent for calculation of asynchronous heavy (tables related) metrics (this is the overhead of asynchronous metrics).
### AsynchronousHeavyMetricsUpdateInterval
Heavy (tables related) metrics update interval
### AsynchronousMetricsCalculationTimeSpent
Time in seconds spent for calculation of asynchronous metrics (this is the overhead of asynchronous metrics).
### AsynchronousMetricsUpdateInterval
Metrics update interval
### BlockActiveTime_*name*
Time in seconds the block device had the IO requests queued. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockDiscardBytes_*name*
Number of discarded bytes on the block device. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockDiscardMerges_*name*
Number of discard operations requested from the block device and merged together by the OS IO scheduler. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockDiscardOps_*name*
Number of discard operations requested from the block device. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockDiscardTime_*name*
Time in seconds spend in discard operations requested from the block device, summed across all the operations. These operations are relevant for SSD. Discard operations are not used by ClickHouse, but can be used by other processes on the system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockInFlightOps_*name*
This value counts the number of I/O requests that have been issued to the device driver but have not yet completed. It does not include IO requests that are in the queue but not yet issued to the device driver. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockQueueTime_*name*
This value counts the number of milliseconds that IO requests have waited on this block device. If there are multiple IO requests waiting, this value will increase as the product of the number of milliseconds times the number of requests waiting. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockReadBytes_*name*
Number of bytes read from the block device. It can be lower than the number of bytes read from the filesystem due to the usage of the OS page cache, that saves IO. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockReadMerges_*name*
Number of read operations requested from the block device and merged together by the OS IO scheduler. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockReadOps_*name*
Number of read operations requested from the block device. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockReadTime_*name*
Time in seconds spend in read operations requested from the block device, summed across all the operations. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockWriteBytes_*name*
Number of bytes written to the block device. It can be lower than the number of bytes written to the filesystem due to the usage of the OS page cache, that saves IO. A write to the block device may happen later than the corresponding write to the filesystem due to write-through caching. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockWriteMerges_*name*
Number of write operations requested from the block device and merged together by the OS IO scheduler. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockWriteOps_*name*
Number of write operations requested from the block device. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### BlockWriteTime_*name*
Time in seconds spend in write operations requested from the block device, summed across all the operations. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Source: `/sys/block`. See https://www.kernel.org/doc/Documentation/block/stat.txt
### CPUFrequencyMHz_*name*
The current frequency of the CPU, in MHz. Most of the modern CPUs adjust the frequency dynamically for power saving and Turbo Boosting.
### CompiledExpressionCacheBytes
Total bytes used for the cache of JIT-compiled code.
### CompiledExpressionCacheCount
Total entries in the cache of JIT-compiled code.
### DiskAvailable_*name*
Available bytes on the disk (virtual filesystem). Remote filesystems can show a large value like 16 EiB.
### DiskTotal_*name*
The total size in bytes of the disk (virtual filesystem). Remote filesystems can show a large value like 16 EiB.
### DiskUnreserved_*name*
Available bytes on the disk (virtual filesystem) without the reservations for merges, fetches, and moves. Remote filesystems can show a large value like 16 EiB.
### DiskUsed_*name*
Used bytes on the disk (virtual filesystem). Remote filesystems not always provide this information.
### FilesystemCacheBytes
Total bytes in the `cache` virtual filesystem. This cache is hold on disk.
### FilesystemCacheFiles
Total number of cached file segments in the `cache` virtual filesystem. This cache is hold on disk.
### FilesystemLogsPathAvailableBytes
Available bytes on the volume where ClickHouse logs path is mounted. If this value approaches zero, you should tune the log rotation in the configuration file.
### FilesystemLogsPathAvailableINodes
The number of available inodes on the volume where ClickHouse logs path is mounted.
### FilesystemLogsPathTotalBytes
The size of the volume where ClickHouse logs path is mounted, in bytes. It's recommended to have at least 10 GB for logs.
### FilesystemLogsPathTotalINodes
The total number of inodes on the volume where ClickHouse logs path is mounted.
### FilesystemLogsPathUsedBytes
Used bytes on the volume where ClickHouse logs path is mounted.
### FilesystemLogsPathUsedINodes
The number of used inodes on the volume where ClickHouse logs path is mounted.
### FilesystemMainPathAvailableBytes
Available bytes on the volume where the main ClickHouse path is mounted.
### FilesystemMainPathAvailableINodes
The number of available inodes on the volume where the main ClickHouse path is mounted. If it is close to zero, it indicates a misconfiguration, and you will get 'no space left on device' even when the disk is not full.
### FilesystemMainPathTotalBytes
The size of the volume where the main ClickHouse path is mounted, in bytes.
### FilesystemMainPathTotalINodes
The total number of inodes on the volume where the main ClickHouse path is mounted. If it is less than 25 million, it indicates a misconfiguration.
### FilesystemMainPathUsedBytes
Used bytes on the volume where the main ClickHouse path is mounted.
### FilesystemMainPathUsedINodes
The number of used inodes on the volume where the main ClickHouse path is mounted. This value mostly corresponds to the number of files.
### HTTPThreads
Number of threads in the server of the HTTP interface (without TLS).
### InterserverThreads
Number of threads in the server of the replicas communication protocol (without TLS).
### Jitter
The difference in time the thread for calculation of the asynchronous metrics was scheduled to wake up and the time it was in fact, woken up. A proxy-indicator of overall system latency and responsiveness.
### LoadAverage_*N*
The whole system load, averaged with exponential smoothing over 1 minute. The load represents the number of threads across all the processes (the scheduling entities of the OS kernel), that are currently running by CPU or waiting for IO, or ready to run but not being scheduled at this point of time. This number includes all the processes, not only clickhouse-server. The number can be greater than the number of CPU cores, if the system is overloaded, and many processes are ready to run but waiting for CPU or IO.
### MMapCacheCells
The number of files opened with `mmap` (mapped in memory). This is used for queries with the setting `local_filesystem_read_method` set to `mmap`. The files opened with `mmap` are kept in the cache to avoid costly TLB flushes.
### MarkCacheBytes
Total size of mark cache in bytes
### MarkCacheFiles
Total number of mark files cached in the mark cache
### MaxPartCountForPartition
Maximum number of parts per partition across all partitions of all tables of MergeTree family. Values larger than 300 indicates misconfiguration, overload, or massive data loading.
### MemoryCode
The amount of virtual memory mapped for the pages of machine code of the server process, in bytes.
### MemoryDataAndStack
The amount of virtual memory mapped for the use of stack and for the allocated memory, in bytes. It is unspecified whether it includes the per-thread stacks and most of the allocated memory, that is allocated with the 'mmap' system call. This metric exists only for completeness reasons. I recommend to use the `MemoryResident` metric for monitoring.
### MemoryResident
The amount of physical memory used by the server process, in bytes.
### MemoryShared
The amount of memory used by the server process, that is also shared by another processes, in bytes. ClickHouse does not use shared memory, but some memory can be labeled by OS as shared for its own reasons. This metric does not make a lot of sense to watch, and it exists only for completeness reasons.
### MemoryVirtual
The size of the virtual address space allocated by the server process, in bytes. The size of the virtual address space is usually much greater than the physical memory consumption, and should not be used as an estimate for the memory consumption. The large values of this metric are totally normal, and makes only technical sense.
### MySQLThreads
Number of threads in the server of the MySQL compatibility protocol.
### NetworkReceiveBytes_*name*
Number of bytes received via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### NetworkReceiveDrop_*name*
Number of bytes a packet was dropped while received via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### NetworkReceiveErrors_*name*
Number of times error happened receiving via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### NetworkReceivePackets_*name*
Number of network packets received via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### NetworkSendBytes_*name*
Number of bytes sent via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### NetworkSendDrop_*name*
Number of times a packed was dropped while sending via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### NetworkSendErrors_*name*
Number of times error (e.g. TCP retransmit) happened while sending via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### NetworkSendPackets_*name*
Number of network packets sent via the network interface. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### NumberOfDatabases
Total number of databases on the server.
### NumberOfDetachedByUserParts
The total number of parts detached from MergeTree tables by users with the `ALTER TABLE DETACH` query (as opposed to unexpected, broken or ignored parts). The server does not care about detached parts and they can be removed.
### NumberOfDetachedParts
The total number of parts detached from MergeTree tables. A part can be detached by a user with the `ALTER TABLE DETACH` query or by the server itself it the part is broken, unexpected or unneeded. The server does not care about detached parts and they can be removed.
### NumberOfTables
Total number of tables summed across the databases on the server, excluding the databases that cannot contain MergeTree tables. The excluded database engines are those who generate the set of tables on the fly, like `Lazy`, `MySQL`, `PostgreSQL`, `SQlite`.
### OSContextSwitches
The number of context switches that the system underwent on the host machine. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSGuestNiceTime
The ratio of time spent running a virtual CPU for guest operating systems under the control of the Linux kernel, when a guest was set to a higher priority (See `man procfs`). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This metric is irrelevant for ClickHouse, but still exists for completeness. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSGuestNiceTimeCPU_*N*
The ratio of time spent running a virtual CPU for guest operating systems under the control of the Linux kernel, when a guest was set to a higher priority (See `man procfs`). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This metric is irrelevant for ClickHouse, but still exists for completeness. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSGuestNiceTimeNormalized
The value is similar to `OSGuestNiceTime` but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.
### OSGuestTime
The ratio of time spent running a virtual CPU for guest operating systems under the control of the Linux kernel (See `man procfs`). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This metric is irrelevant for ClickHouse, but still exists for completeness. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSGuestTimeCPU_*N*
The ratio of time spent running a virtual CPU for guest operating systems under the control of the Linux kernel (See `man procfs`). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This metric is irrelevant for ClickHouse, but still exists for completeness. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSGuestTimeNormalized
The value is similar to `OSGuestTime` but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.
### OSIOWaitTime
The ratio of time the CPU core was not running the code but when the OS kernel did not run any other process on this CPU as the processes were waiting for IO. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSIOWaitTimeCPU_*N*
The ratio of time the CPU core was not running the code but when the OS kernel did not run any other process on this CPU as the processes were waiting for IO. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSIOWaitTimeNormalized
The value is similar to `OSIOWaitTime` but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.
### OSIdleTime
The ratio of time the CPU core was idle (not even ready to run a process waiting for IO) from the OS kernel standpoint. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This does not include the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls, branch mispredictions, running another SMT core). The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSIdleTimeCPU_*N*
The ratio of time the CPU core was idle (not even ready to run a process waiting for IO) from the OS kernel standpoint. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This does not include the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls, branch mispredictions, running another SMT core). The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSIdleTimeNormalized
The value is similar to `OSIdleTime` but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.
### OSInterrupts
The number of interrupts on the host machine. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSIrqTime
The ratio of time spent for running hardware interrupt requests on the CPU. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. A high number of this metric may indicate hardware misconfiguration or a very high network load. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSIrqTimeCPU_*N*
The ratio of time spent for running hardware interrupt requests on the CPU. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. A high number of this metric may indicate hardware misconfiguration or a very high network load. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSIrqTimeNormalized
The value is similar to `OSIrqTime` but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.
### OSMemoryAvailable
The amount of memory available to be used by programs, in bytes. This is very similar to the `OSMemoryFreePlusCached` metric. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSMemoryBuffers
The amount of memory used by OS kernel buffers, in bytes. This should be typically small, and large values may indicate a misconfiguration of the OS. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSMemoryCached
The amount of memory used by the OS page cache, in bytes. Typically, almost all available memory is used by the OS page cache - high values of this metric are normal and expected. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSMemoryFreePlusCached
The amount of free memory plus OS page cache memory on the host system, in bytes. This memory is available to be used by programs. The value should be very similar to `OSMemoryAvailable`. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSMemoryFreeWithoutCached
The amount of free memory on the host system, in bytes. This does not include the memory used by the OS page cache memory, in bytes. The page cache memory is also available for usage by programs, so the value of this metric can be confusing. See the `OSMemoryAvailable` metric instead. For convenience we also provide the `OSMemoryFreePlusCached` metric, that should be somewhat similar to OSMemoryAvailable. See also https://www.linuxatemyram.com/. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSMemoryTotal
The total amount of memory on the host system, in bytes.
### OSNiceTime
The ratio of time the CPU core was running userspace code with higher priority. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSNiceTimeCPU_*N*
The ratio of time the CPU core was running userspace code with higher priority. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSNiceTimeNormalized
The value is similar to `OSNiceTime` but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.
### OSOpenFiles
The total number of opened files on the host machine. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSProcessesBlocked
Number of threads blocked waiting for I/O to complete (`man procfs`). This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSProcessesCreated
The number of processes created. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSProcessesRunning
The number of runnable (running or ready to run) threads by the operating system. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server.
### OSSoftIrqTime
The ratio of time spent for running software interrupt requests on the CPU. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. A high number of this metric may indicate inefficient software running on the system. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSSoftIrqTimeCPU_*N*
The ratio of time spent for running software interrupt requests on the CPU. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. A high number of this metric may indicate inefficient software running on the system. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSSoftIrqTimeNormalized
The value is similar to `OSSoftIrqTime` but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.
### OSStealTime
The ratio of time spent in other operating systems by the CPU when running in a virtualized environment. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Not every virtualized environments present this metric, and most of them don't. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSStealTimeCPU_*N*
The ratio of time spent in other operating systems by the CPU when running in a virtualized environment. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. Not every virtualized environments present this metric, and most of them don't. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSStealTimeNormalized
The value is similar to `OSStealTime` but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.
### OSSystemTime
The ratio of time the CPU core was running OS kernel (system) code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSSystemTimeCPU_*N*
The ratio of time the CPU core was running OS kernel (system) code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSSystemTimeNormalized
The value is similar to `OSSystemTime` but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.
### OSThreadsRunnable
The total number of 'runnable' threads, as the OS kernel scheduler seeing it.
### OSThreadsTotal
The total number of threads, as the OS kernel scheduler seeing it.
### OSUptime
The uptime of the host server (the machine where ClickHouse is running), in seconds.
### OSUserTime
The ratio of time the CPU core was running userspace code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This includes also the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls, branch mispredictions, running another SMT core). The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSUserTimeCPU_*N*
The ratio of time the CPU core was running userspace code. This is a system-wide metric, it includes all the processes on the host machine, not just clickhouse-server. This includes also the time when the CPU was under-utilized due to the reasons internal to the CPU (memory loads, pipeline stalls, branch mispredictions, running another SMT core). The value for a single CPU core will be in the interval [0..1]. The value for all CPU cores is calculated as a sum across them [0..num cores].
### OSUserTimeNormalized
The value is similar to `OSUserTime` but divided to the number of CPU cores to be measured in the [0..1] interval regardless of the number of cores. This allows you to average the values of this metric across multiple servers in a cluster even if the number of cores is non-uniform, and still get the average resource utilization metric.
### PostgreSQLThreads
Number of threads in the server of the PostgreSQL compatibility protocol.
### ReplicasMaxAbsoluteDelay
Maximum difference in seconds between the most fresh replicated part and the most fresh data part still to be replicated, across Replicated tables. A very high value indicates a replica with no data.
### ReplicasMaxInsertsInQueue
Maximum number of INSERT operations in the queue (still to be replicated) across Replicated tables.
### ReplicasMaxMergesInQueue
Maximum number of merge operations in the queue (still to be applied) across Replicated tables.
### ReplicasMaxQueueSize
Maximum queue size (in the number of operations like get, merge) across Replicated tables.
### ReplicasMaxRelativeDelay
Maximum difference between the replica delay and the delay of the most up-to-date replica of the same table, across Replicated tables.
### ReplicasSumInsertsInQueue
Sum of INSERT operations in the queue (still to be replicated) across Replicated tables.
### ReplicasSumMergesInQueue
Sum of merge operations in the queue (still to be applied) across Replicated tables.
### ReplicasSumQueueSize
Sum queue size (in the number of operations like get, merge) across Replicated tables.
### TCPThreads
Number of threads in the server of the TCP protocol (without TLS).
### Temperature_*N*
The temperature of the corresponding device in ℃. A sensor can return an unrealistic value. Source: `/sys/class/thermal`
### Temperature_*name*
The temperature reported by the corresponding hardware monitor and the corresponding sensor in ℃. A sensor can return an unrealistic value. Source: `/sys/class/hwmon`
### TotalBytesOfMergeTreeTables
Total amount of bytes (compressed, including data and indices) stored in all tables of MergeTree family.
### TotalPartsOfMergeTreeTables
Total amount of data parts in all tables of MergeTree family. Numbers larger than 10 000 will negatively affect the server startup time and it may indicate unreasonable choice of the partition key.
### TotalRowsOfMergeTreeTables
Total amount of rows (records) stored in all tables of MergeTree family.
### UncompressedCacheBytes
Total size of uncompressed cache in bytes. Uncompressed cache does not usually improve the performance and should be mostly avoided.
### UncompressedCacheCells
Total number of entries in the uncompressed cache. Each entry represents a decompressed block of data. Uncompressed cache does not usually improve performance and should be mostly avoided.
### Uptime
The server uptime in seconds. It includes the time spent for server initialization before accepting connections.
### jemalloc.active
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.allocated
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.arenas.all.dirty_purged
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.arenas.all.muzzy_purged
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.arenas.all.pactive
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.arenas.all.pdirty
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.arenas.all.pmuzzy
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.background_thread.num_runs
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.background_thread.num_threads
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.background_thread.run_intervals
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.epoch
An internal incremental update number of the statistics of jemalloc (Jason Evans' memory allocator), used in all other `jemalloc` metrics.
### jemalloc.mapped
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.metadata
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.metadata_thp
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.resident
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
### jemalloc.retained
An internal metric of the low-level memory allocator (jemalloc). See https://jemalloc.net/jemalloc.3.html
**See Also**
- [Monitoring](../../operations/monitoring.md) — Base concepts of ClickHouse monitoring.

714
docs/en/operations/system-tables/metrics.md
View File

@ -34,6 +34,720 @@ SELECT * FROM system.metrics LIMIT 10
└──────────────────────────────────────┴───────┴────────────────────────────────────────────────────────────────────────┘
```
## Metric descriptions
### AggregatorThreads
Number of threads in the Aggregator thread pool.
### AggregatorThreadsActive
Number of threads in the Aggregator thread pool running a task.
### AsyncInsertCacheSize
Number of async insert hash id in cache
### AsynchronousInsertThreads
Number of threads in the AsynchronousInsert thread pool.
### AsynchronousInsertThreadsActive
Number of threads in the AsynchronousInsert thread pool running a task.
### AsynchronousReadWait
Number of threads waiting for asynchronous read.
### BackgroundBufferFlushSchedulePoolSize
Limit on number of tasks in BackgroundBufferFlushSchedulePool
### BackgroundBufferFlushSchedulePoolTask
Number of active tasks in BackgroundBufferFlushSchedulePool. This pool is used for periodic Buffer flushes
### BackgroundCommonPoolSize
Limit on number of tasks in an associated background pool
### BackgroundCommonPoolTask
Number of active tasks in an associated background pool
### BackgroundDistributedSchedulePoolSize
Limit on number of tasks in BackgroundDistributedSchedulePool
### BackgroundDistributedSchedulePoolTask
Number of active tasks in BackgroundDistributedSchedulePool. This pool is used for distributed sends that is done in background.
### BackgroundFetchesPoolSize
Limit on number of simultaneous fetches in an associated background pool
### BackgroundFetchesPoolTask
Number of active fetches in an associated background pool
### BackgroundMergesAndMutationsPoolSize
Limit on number of active merges and mutations in an associated background pool
### BackgroundMergesAndMutationsPoolTask
Number of active merges and mutations in an associated background pool
### BackgroundMessageBrokerSchedulePoolSize
Limit on number of tasks in BackgroundProcessingPool for message streaming
### BackgroundMessageBrokerSchedulePoolTask
Number of active tasks in BackgroundProcessingPool for message streaming
### BackgroundMovePoolSize
Limit on number of tasks in BackgroundProcessingPool for moves
### BackgroundMovePoolTask
Number of active tasks in BackgroundProcessingPool for moves
### BackgroundSchedulePoolSize
Limit on number of tasks in BackgroundSchedulePool. This pool is used for periodic ReplicatedMergeTree tasks, like cleaning old data parts, altering data parts, replica re-initialization, etc.
### BackgroundSchedulePoolTask
Number of active tasks in BackgroundSchedulePool. This pool is used for periodic ReplicatedMergeTree tasks, like cleaning old data parts, altering data parts, replica re-initialization, etc.
### BackupsIOThreads
Number of threads in the BackupsIO thread pool.
### BackupsIOThreadsActive
Number of threads in the BackupsIO thread pool running a task.
### BackupsThreads
Number of threads in the thread pool for BACKUP.
### BackupsThreadsActive
Number of threads in thread pool for BACKUP running a task.
### BrokenDistributedFilesToInsert
Number of files for asynchronous insertion into Distributed tables that has been marked as broken. This metric will starts from 0 on start. Number of files for every shard is summed.
### CacheDetachedFileSegments
Number of existing detached cache file segments
### CacheDictionaryThreads
Number of threads in the CacheDictionary thread pool.
### CacheDictionaryThreadsActive
Number of threads in the CacheDictionary thread pool running a task.
### CacheDictionaryUpdateQueueBatches
Number of 'batches' (a set of keys) in update queue in CacheDictionaries.
### CacheDictionaryUpdateQueueKeys
Exact number of keys in update queue in CacheDictionaries.
### CacheFileSegments
Number of existing cache file segments
### ContextLockWait
Number of threads waiting for lock in Context. This is global lock.
### DDLWorkerThreads
Number of threads in the DDLWorker thread pool for ON CLUSTER queries.
### DDLWorkerThreadsActive
Number of threads in the DDLWORKER thread pool for ON CLUSTER queries running a task.
### DatabaseCatalogThreads
Number of threads in the DatabaseCatalog thread pool.
### DatabaseCatalogThreadsActive
Number of threads in the DatabaseCatalog thread pool running a task.
### DatabaseOnDiskThreads
Number of threads in the DatabaseOnDisk thread pool.
### DatabaseOnDiskThreadsActive
Number of threads in the DatabaseOnDisk thread pool running a task.
### DatabaseOrdinaryThreads
Number of threads in the Ordinary database thread pool.
### DatabaseOrdinaryThreadsActive
Number of threads in the Ordinary database thread pool running a task.
### DelayedInserts
Number of INSERT queries that are throttled due to high number of active data parts for partition in a MergeTree table.
### DestroyAggregatesThreads
Number of threads in the thread pool for destroy aggregate states.
### DestroyAggregatesThreadsActive
Number of threads in the thread pool for destroy aggregate states running a task.
### DictCacheRequests
Number of requests in fly to data sources of dictionaries of cache type.
### DiskObjectStorageAsyncThreads
Number of threads in the async thread pool for DiskObjectStorage.
### DiskObjectStorageAsyncThreadsActive
Number of threads in the async thread pool for DiskObjectStorage running a task.
### DiskSpaceReservedForMerge
Disk space reserved for currently running background merges. It is slightly more than the total size of currently merging parts.
### DistributedFilesToInsert
Number of pending files to process for asynchronous insertion into Distributed tables. Number of files for every shard is summed.
### DistributedSend
Number of connections to remote servers sending data that was INSERTed into Distributed tables. Both synchronous and asynchronous mode.
### EphemeralNode
Number of ephemeral nodes hold in ZooKeeper.
### FilesystemCacheElements
Filesystem cache elements (file segments)
### FilesystemCacheReadBuffers
Number of active cache buffers
### FilesystemCacheSize
Filesystem cache size in bytes
### GlobalThread
Number of threads in global thread pool.
### GlobalThreadActive
Number of threads in global thread pool running a task.
### HTTPConnection
Number of connections to HTTP server
### HashedDictionaryThreads
Number of threads in the HashedDictionary thread pool.
### HashedDictionaryThreadsActive
Number of threads in the HashedDictionary thread pool running a task.
### IOPrefetchThreads
Number of threads in the IO prefertch thread pool.
### IOPrefetchThreadsActive
Number of threads in the IO prefetch thread pool running a task.
### IOThreads
Number of threads in the IO thread pool.
### IOThreadsActive
Number of threads in the IO thread pool running a task.
### IOUringInFlightEvents
Number of io_uring SQEs in flight
### IOUringPendingEvents
Number of io_uring SQEs waiting to be submitted
### IOWriterThreads
Number of threads in the IO writer thread pool.
### IOWriterThreadsActive
Number of threads in the IO writer thread pool running a task.
### InterserverConnection
Number of connections from other replicas to fetch parts
### KafkaAssignedPartitions
Number of partitions Kafka tables currently assigned to
### KafkaBackgroundReads
Number of background reads currently working (populating materialized views from Kafka)
### KafkaConsumers
Number of active Kafka consumers
### KafkaConsumersInUse
Number of consumers which are currently used by direct or background reads
### KafkaConsumersWithAssignment
Number of active Kafka consumers which have some partitions assigned.
### KafkaLibrdkafkaThreads
Number of active librdkafka threads
### KafkaProducers
Number of active Kafka producer created
### KafkaWrites
Number of currently running inserts to Kafka
### KeeperAliveConnections
Number of alive connections
### KeeperOutstandingRequets
Number of outstanding requests
### LocalThread
Number of threads in local thread pools. The threads in local thread pools are taken from the global thread pool.
### LocalThreadActive
Number of threads in local thread pools running a task.
### MMappedAllocBytes
Sum bytes of mmapped allocations
### MMappedAllocs
Total number of mmapped allocations
### MMappedFileBytes
Sum size of mmapped file regions.
### MMappedFiles
Total number of mmapped files.
### MarksLoaderThreads
Number of threads in thread pool for loading marks.
### MarksLoaderThreadsActive
Number of threads in the thread pool for loading marks running a task.
### MaxDDLEntryID
Max processed DDL entry of DDLWorker.
### MaxPushedDDLEntryID
Max DDL entry of DDLWorker that pushed to zookeeper.
### MemoryTracking
Total amount of memory (bytes) allocated by the server.
### Merge
Number of executing background merges
### MergeTreeAllRangesAnnouncementsSent
The current number of announcement being sent in flight from the remote server to the initiator server about the set of data parts (for MergeTree tables). Measured on the remote server side.
### MergeTreeBackgroundExecutorThreads
Number of threads in the MergeTreeBackgroundExecutor thread pool.
### MergeTreeBackgroundExecutorThreadsActive
Number of threads in the MergeTreeBackgroundExecutor thread pool running a task.
### MergeTreeDataSelectExecutorThreads
Number of threads in the MergeTreeDataSelectExecutor thread pool.
### MergeTreeDataSelectExecutorThreadsActive
Number of threads in the MergeTreeDataSelectExecutor thread pool running a task.
### MergeTreePartsCleanerThreads
Number of threads in the MergeTree parts cleaner thread pool.
### MergeTreePartsCleanerThreadsActive
Number of threads in the MergeTree parts cleaner thread pool running a task.
### MergeTreePartsLoaderThreads
Number of threads in the MergeTree parts loader thread pool.
### MergeTreePartsLoaderThreadsActive
Number of threads in the MergeTree parts loader thread pool running a task.
### MergeTreeReadTaskRequestsSent
The current number of callback requests in flight from the remote server back to the initiator server to choose the read task (for MergeTree tables). Measured on the remote server side.
### Move
Number of currently executing moves
### MySQLConnection
Number of client connections using MySQL protocol
### NetworkReceive
Number of threads receiving data from network. Only ClickHouse-related network interaction is included, not by 3rd party libraries.
### NetworkSend
Number of threads sending data to network. Only ClickHouse-related network interaction is included, not by 3rd party libraries.
### OpenFileForRead
Number of files open for reading
### OpenFileForWrite
Number of files open for writing
### ParallelFormattingOutputFormatThreads
Number of threads in the ParallelFormattingOutputFormatThreads thread pool.
### ParallelFormattingOutputFormatThreadsActive
Number of threads in the ParallelFormattingOutputFormatThreads thread pool running a task.
### ParallelParsingInputFormatThreads
Number of threads in the ParallelParsingInputFormat thread pool.
### ParallelParsingInputFormatThreadsActive
Number of threads in the ParallelParsingInputFormat thread pool running a task.
### PartMutation
Number of mutations (ALTER DELETE/UPDATE)
### PartsActive
Active data part, used by current and upcoming SELECTs.
### PartsCommitted
Deprecated. See PartsActive.
### PartsCompact
Compact parts.
### PartsDeleteOnDestroy
Part was moved to another disk and should be deleted in own destructor.
### PartsDeleting
Not active data part with identity refcounter, it is deleting right now by a cleaner.
### PartsInMemory
In-memory parts.
### PartsOutdated
Not active data part, but could be used by only current SELECTs, could be deleted after SELECTs finishes.
### PartsPreActive
The part is in data_parts, but not used for SELECTs.
### PartsPreCommitted
Deprecated. See PartsPreActive.
### PartsTemporary
The part is generating now, it is not in data_parts list.
### PartsWide
Wide parts.
### PendingAsyncInsert
Number of asynchronous inserts that are waiting for flush.
### PostgreSQLConnection
Number of client connections using PostgreSQL protocol
### Query
Number of executing queries
### QueryPreempted
Number of queries that are stopped and waiting due to 'priority' setting.
### QueryThread
Number of query processing threads
### RWLockActiveReaders
Number of threads holding read lock in a table RWLock.
### RWLockActiveWriters
Number of threads holding write lock in a table RWLock.
### RWLockWaitingReaders
Number of threads waiting for read on a table RWLock.
### RWLockWaitingWriters
Number of threads waiting for write on a table RWLock.
### Read
Number of read (read, pread, io_getevents, etc.) syscalls in fly
### ReadTaskRequestsSent
The current number of callback requests in flight from the remote server back to the initiator server to choose the read task (for s3Cluster table function and similar). Measured on the remote server side.
### ReadonlyReplica
Number of Replicated tables that are currently in readonly state due to re-initialization after ZooKeeper session loss or due to startup without ZooKeeper configured.
### RemoteRead
Number of read with remote reader in fly
### ReplicatedChecks
Number of data parts checking for consistency
### ReplicatedFetch
Number of data parts being fetched from replica
### ReplicatedSend
Number of data parts being sent to replicas
### RestartReplicaThreads
Number of threads in the RESTART REPLICA thread pool.
### RestartReplicaThreadsActive
Number of threads in the RESTART REPLICA thread pool running a task.
### RestoreThreads
Number of threads in the thread pool for RESTORE.
### RestoreThreadsActive
Number of threads in the thread pool for RESTORE running a task.
### Revision
Revision of the server. It is a number incremented for every release or release candidate except patch releases.
### S3Requests
S3 requests
### SendExternalTables
Number of connections that are sending data for external tables to remote servers. External tables are used to implement GLOBAL IN and GLOBAL JOIN operators with distributed subqueries.
### SendScalars
Number of connections that are sending data for scalars to remote servers.
### StartupSystemTablesThreads
Number of threads in the StartupSystemTables thread pool.
### StartupSystemTablesThreadsActive
Number of threads in the StartupSystemTables thread pool running a task.
### StorageBufferBytes
Number of bytes in buffers of Buffer tables
### StorageBufferRows
Number of rows in buffers of Buffer tables
### StorageDistributedThreads
Number of threads in the StorageDistributed thread pool.
### StorageDistributedThreadsActive
Number of threads in the StorageDistributed thread pool running a task.
### StorageHiveThreads
Number of threads in the StorageHive thread pool.
### StorageHiveThreadsActive
Number of threads in the StorageHive thread pool running a task.
### StorageS3Threads
Number of threads in the StorageS3 thread pool.
### StorageS3ThreadsActive
Number of threads in the StorageS3 thread pool running a task.
### SystemReplicasThreads
Number of threads in the system.replicas thread pool.
### SystemReplicasThreadsActive
Number of threads in the system.replicas thread pool running a task.
### TCPConnection
Number of connections to TCP server (clients with native interface), also included server-server distributed query connections
### TablesLoaderThreads
Number of threads in the tables loader thread pool.
### TablesLoaderThreadsActive
Number of threads in the tables loader thread pool running a task.
### TablesToDropQueueSize
Number of dropped tables, that are waiting for background data removal.
### TemporaryFilesForAggregation
Number of temporary files created for external aggregation
### TemporaryFilesForJoin
Number of temporary files created for JOIN
### TemporaryFilesForSort
Number of temporary files created for external sorting
### TemporaryFilesUnknown
Number of temporary files created without known purpose
### ThreadPoolFSReaderThreads
Number of threads in the thread pool for local_filesystem_read_method=threadpool.
### ThreadPoolFSReaderThreadsActive
Number of threads in the thread pool for local_filesystem_read_method=threadpool running a task.
### ThreadPoolRemoteFSReaderThreads
Number of threads in the thread pool for remote_filesystem_read_method=threadpool.
### ThreadPoolRemoteFSReaderThreadsActive
Number of threads in the thread pool for remote_filesystem_read_method=threadpool running a task.
### ThreadsInOvercommitTracker
Number of waiting threads inside of OvercommitTracker
### TotalTemporaryFiles
Number of temporary files created
### VersionInteger
Version of the server in a single integer number in base-1000. For example, version 11.22.33 is translated to 11022033.
### Write
Number of write (write, pwrite, io_getevents, etc.) syscalls in fly
### ZooKeeperRequest
Number of requests to ZooKeeper in fly.
### ZooKeeperSession
Number of sessions (connections) to ZooKeeper. Should be no more than one, because using more than one connection to ZooKeeper may lead to bugs due to lack of linearizability (stale reads) that ZooKeeper consistency model allows.
### ZooKeeperWatch
Number of watches (event subscriptions) in ZooKeeper.
**See Also**
- [system.asynchronous_metrics](../../operations/system-tables/asynchronous_metrics.md#system_tables-asynchronous_metrics) — Contains periodically calculated metrics.

4
docs/en/operations/system-tables/processors_profile_log.md
View File

@ -1,4 +1,4 @@
# system.processors_profile_log {#system-processors_profile_log}
# processors_profile_log
This table contains profiling on processors level (that you can find in [`EXPLAIN PIPELINE`](../../sql-reference/statements/explain.md#explain-pipeline)).
@ -73,4 +73,4 @@ Here you can see:
**See Also**
- [`EXPLAIN PIPELINE`](../../sql-reference/statements/explain.md#explain-pipeline)
- [`EXPLAIN PIPELINE`](../../sql-reference/statements/explain.md#explain-pipeline)

4
docs/en/sql-reference/functions/array-functions.md
View File

@ -646,7 +646,7 @@ SELECT arraySlice([1, 2, NULL, 4, 5], 2, 3) AS res;
Array elements set to `NULL` are handled as normal values.
## arraySort(\[func,\] arr, …)
## arraySort(\[func,\] arr, …) {#array_functions-sort}
Sorts the elements of the `arr` array in ascending order. If the `func` function is specified, sorting order is determined by the result of the `func` function applied to the elements of the array. If `func` accepts multiple arguments, the `arraySort` function is passed several arrays that the arguments of `func` will correspond to. Detailed examples are shown at the end of `arraySort` description.
@ -751,7 +751,7 @@ To improve sorting efficiency, the [Schwartzian transform](https://en.wikipedia.
Same as `arraySort` with additional `limit` argument allowing partial sorting. Returns an array of the same size as the original array where elements in range `[1..limit]` are sorted in ascending order. Remaining elements `(limit..N]` shall contain elements in unspecified order.
## arrayReverseSort(\[func,\] arr, …)
## arrayReverseSort(\[func,\] arr, …) {#array_functions-reverse-sort}
Sorts the elements of the `arr` array in descending order. If the `func` function is specified, `arr` is sorted according to the result of the `func` function applied to the elements of the array, and then the sorted array is reversed. If `func` accepts multiple arguments, the `arrayReverseSort` function is passed several arrays that the arguments of `func` will correspond to. Detailed examples are shown at the end of `arrayReverseSort` description.

93
docs/en/sql-reference/functions/string-functions.md
View File

@ -1215,3 +1215,96 @@ Result:
│ A240 │
└──────────────────┘
```
## extractKeyValuePairs
Extracts key-value pairs from any string. The string does not need to be 100% structured in a key value pair format;
It can contain noise (e.g. log files). The key-value pair format to be interpreted should be specified via function arguments.
A key-value pair consists of a key followed by a `key_value_delimiter` and a value. Quoted keys and values are also supported. Key value pairs must be separated by pair delimiters.
**Syntax**
``` sql
extractKeyValuePairs(data, [key_value_delimiter], [pair_delimiter], [quoting_character])
```
**Arguments**
- `data` - String to extract key-value pairs from. [String](../../sql-reference/data-types/string.md) or [FixedString](../../sql-reference/data-types/fixedstring.md).
- `key_value_delimiter` - Character to be used as delimiter between the key and the value. Defaults to `:`. [String](../../sql-reference/data-types/string.md) or [FixedString](../../sql-reference/data-types/fixedstring.md).
- `pair_delimiters` - Set of character to be used as delimiters between pairs. Defaults to `\space`, `,` and `;`. [String](../../sql-reference/data-types/string.md) or [FixedString](../../sql-reference/data-types/fixedstring.md).
- `quoting_character` - Character to be used as quoting character. Defaults to `"`. [String](../../sql-reference/data-types/string.md) or [FixedString](../../sql-reference/data-types/fixedstring.md).
**Returned values**
- The extracted key-value pairs in a Map(String, String).
**Examples**
Query:
**Simple case**
``` sql
arthur :) select extractKeyValuePairs('name:neymar, age:31 team:psg,nationality:brazil') as kv
SELECT extractKeyValuePairs('name:neymar, age:31 team:psg,nationality:brazil') as kv
Query id: f9e0ca6f-3178-4ee2-aa2c-a5517abb9cee
┌─kv──────────────────────────────────────────────────────────────────────┐
│ {'name':'neymar','age':'31','team':'psg','nationality':'brazil'} │
└─────────────────────────────────────────────────────────────────────────┘
```
**Single quote as quoting character**
``` sql
arthur :) select extractKeyValuePairs('name:\'neymar\';\'age\':31;team:psg;nationality:brazil,last_key:last_value', ':', ';,', '\'') as kv
SELECT extractKeyValuePairs('name:\'neymar\';\'age\':31;team:psg;nationality:brazil,last_key:last_value', ':', ';,', '\'') as kv
Query id: 0e22bf6b-9844-414a-99dc-32bf647abd5e
┌─kv───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ {'name':'neymar','age':'31','team':'psg','nationality':'brazil','last_key':'last_value'} │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
```
**Escape sequences without escape sequences support**
``` sql
arthur :) select extractKeyValuePairs('age:a\\x0A\\n\\0') as kv
SELECT extractKeyValuePairs('age:a\\x0A\\n\\0') AS kv
Query id: e9fd26ee-b41f-4a11-b17f-25af6fd5d356
┌─kv─────────────────────┐
│ {'age':'a\\x0A\\n\\0'} │
└────────────────────────┘
```
## extractKeyValuePairsWithEscaping
Same as `extractKeyValuePairs` but with escaping support.
Escape sequences supported: `\x`, `\N`, `\a`, `\b`, `\e`, `\f`, `\n`, `\r`, `\t`, `\v` and `\0`.
Non standard escape sequences are returned as it is (including the backslash) unless they are one of the following:
`\\`, `'`, `"`, `backtick`, `/`, `=` or ASCII control characters (c <= 31).
This function will satisfy the use case where pre-escaping and post-escaping are not suitable. For instance, consider the following
input string: `a: "aaaa\"bbb"`. The expected output is: `a: aaaa\"bbbb`.
- Pre-escaping: Pre-escaping it will output: `a: "aaaa"bbb"` and `extractKeyValuePairs` will then output: `a: aaaa`
- Post-escaping: `extractKeyValuePairs` will output `a: aaaa\` and post-escaping will keep it as it is.
Leading escape sequences will be skipped in keys and will be considered invalid for values.
**Escape sequences with escape sequence support turned on**
``` sql
arthur :) select extractKeyValuePairsWithEscaping('age:a\\x0A\\n\\0') as kv
SELECT extractKeyValuePairsWithEscaping('age:a\\x0A\\n\\0') AS kv
Query id: 44c114f0-5658-4c75-ab87-4574de3a1645
┌─kv────────────────┐
│ {'age':'a\n\n\0'} │
└───────────────────┘
```

Some files were not shown because too many files have changed in this diff Show More