2023-01-24 22:19:50 +00:00
---
sidebar_label: Format Settings
sidebar_position: 52
slug: /en/operations/settings/formats
toc_max_heading_level: 2
---
# Format settings {#format-settings}
2024-10-06 03:12:59 +00:00
## bool_false_representation {#bool_false_representation}
2023-04-28 12:42:23 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-04-28 12:42:23 +00:00
2024-10-06 03:12:59 +00:00
Default value: false
2023-04-28 12:42:23 +00:00
2024-10-06 03:12:59 +00:00
Text to represent false bool value in TSV/CSV/Vertical/Pretty formats.
2023-04-28 12:42:23 +00:00
2024-10-06 03:12:59 +00:00
## bool_true_representation {#bool_true_representation}
2023-04-28 12:42:23 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-04-28 12:42:23 +00:00
2024-10-06 03:12:59 +00:00
Default value: true
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Text to represent true bool value in TSV/CSV/Vertical/Pretty formats.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## column_names_for_schema_inference {#column_names_for_schema_inference}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The list of column names to use in schema inference for formats without column names. The format: 'column1,column2,column3,...'
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## cross_to_inner_join_rewrite {#cross_to_inner_join_rewrite}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use inner join instead of comma/cross join if there are joining expressions in the WHERE section. Values: 0 - no rewrite, 1 - apply if possible for comma/cross, 2 - force rewrite all comma joins, cross - if possible
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## date_time_input_format {#date_time_input_format}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: DateTimeInputFormat
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: basic
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Allows choosing a parser of the text representation of date and time.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The setting does not apply to [date and time functions ](../../sql-reference/functions/date-time-functions.md ).
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
- `'best_effort'` — Enables extended parsing.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
ClickHouse can parse the basic `YYYY-MM-DD HH:MM:SS` format and all [ISO 8601 ](https://en.wikipedia.org/wiki/ISO_8601 ) date and time formats. For example, `'2018-06-08T01:02:03.000Z'` .
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
- `'basic'` — Use basic parser.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
ClickHouse can parse only the basic `YYYY-MM-DD HH:MM:SS` or `YYYY-MM-DD` format. For example, `2019-08-20 10:18:56` or `2019-08-20` .
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Cloud default value: `'best_effort'` .
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
See also:
- [DateTime data type. ](../../sql-reference/data-types/datetime.md )
- [Functions for working with dates and times. ](../../sql-reference/functions/date-time-functions.md )
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## date_time_output_format {#date_time_output_format}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: DateTimeOutputFormat
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: simple
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Allows choosing different output formats of the text representation of date and time.
2023-01-24 22:19:50 +00:00
Possible values:
2024-10-06 03:12:59 +00:00
- `simple` - Simple output format.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
ClickHouse output date and time `YYYY-MM-DD hh:mm:ss` format. For example, `2019-08-20 10:18:56` . The calculation is performed according to the data type's time zone (if present) or server time zone.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
- `iso` - ISO output format.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
ClickHouse output date and time in [ISO 8601 ](https://en.wikipedia.org/wiki/ISO_8601 ) `YYYY-MM-DDThh:mm:ssZ` format. For example, `2019-08-20T10:18:56Z` . Note that output is in UTC (`Z` means UTC).
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
- `unix_timestamp` - Unix timestamp output format.
2023-02-10 17:20:51 +00:00
2024-10-06 03:12:59 +00:00
ClickHouse output date and time in [Unix timestamp ](https://en.wikipedia.org/wiki/Unix_time ) format. For example `1566285536` .
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
See also:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
- [DateTime data type. ](../../sql-reference/data-types/datetime.md )
- [Functions for working with dates and times. ](../../sql-reference/functions/date-time-functions.md )
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## date_time_overflow_behavior {#date_time_overflow_behavior}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: DateTimeOverflowBehavior
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: ignore
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Overflow mode for Date, Date32, DateTime, DateTime64 types. Possible values: 'ignore', 'throw', 'saturate'.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## dictionary_use_async_executor {#dictionary_use_async_executor}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Execute a pipeline for reading dictionary source in several threads. It's supported only by dictionaries with local CLICKHOUSE source.
2023-06-05 15:22:04 +00:00
2024-10-06 03:12:59 +00:00
## errors_output_format {#errors_output_format}
2023-06-05 15:22:04 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-06-05 15:22:04 +00:00
2024-10-06 03:12:59 +00:00
Default value: CSV
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Method to write Errors to text output.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## exact_rows_before_limit {#exact_rows_before_limit}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When enabled, ClickHouse will provide exact value for rows_before_limit_at_least statistic, but with the cost that the data before limit will have to be read completely
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_avro_schema_registry_url {#format_avro_schema_registry_url}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: URI
2023-02-27 13:02:10 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
For AvroConfluent format: Confluent Schema Registry URL.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_binary_max_array_size {#format_binary_max_array_size}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1073741824
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The maximum allowed size for Array in RowBinary format. It prevents allocating large amount of memory in case of corrupted data. 0 means there is no limit
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_binary_max_string_size {#format_binary_max_string_size}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1073741824
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The maximum allowed size for String in RowBinary format. It prevents allocating large amount of memory in case of corrupted data. 0 means there is no limit
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_capn_proto_enum_comparising_mode {#format_capn_proto_enum_comparising_mode}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: CapnProtoEnumComparingMode
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: by_values
2024-06-11 10:14:36 +00:00
2024-10-06 03:12:59 +00:00
How to map ClickHouse Enum and CapnProto Enum
2024-06-11 10:14:36 +00:00
2024-10-06 03:12:59 +00:00
## format_capn_proto_use_autogenerated_schema {#format_capn_proto_use_autogenerated_schema}
2024-06-11 10:14:36 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2024-06-11 10:14:36 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2024-06-11 10:14:36 +00:00
2024-10-06 03:12:59 +00:00
Use autogenerated CapnProto schema when format_schema is not set
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_csv_allow_double_quotes {#format_csv_allow_double_quotes}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
If it is set to true, allow strings in double quotes.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_csv_allow_single_quotes {#format_csv_allow_single_quotes}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
If it is set to true, allow strings in single quotes.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_csv_delimiter {#format_csv_delimiter}
2024-02-09 02:16:44 +00:00
2024-10-06 03:12:59 +00:00
Type: Char
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: ,
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The character to be considered as a delimiter in CSV data. If setting with a string, a string has to have a length of 1.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_csv_null_representation {#format_csv_null_representation}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: \N
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Custom NULL representation in CSV format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_custom_escaping_rule {#format_custom_escaping_rule}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: EscapingRule
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: Escaped
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Field escaping rule (for CustomSeparated format)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_custom_field_delimiter {#format_custom_field_delimiter}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Delimiter between fields (for CustomSeparated format)
2023-06-26 18:15:19 +00:00
2024-10-06 03:12:59 +00:00
## format_custom_result_after_delimiter {#format_custom_result_after_delimiter}
2023-06-26 18:15:19 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-06-26 18:15:19 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-06-26 18:15:19 +00:00
2024-10-06 03:12:59 +00:00
Suffix after result set (for CustomSeparated format)
2023-06-26 18:15:19 +00:00
2024-10-06 03:12:59 +00:00
## format_custom_result_before_delimiter {#format_custom_result_before_delimiter}
2023-06-26 18:15:19 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-06-26 18:15:19 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-06-26 18:15:19 +00:00
2024-10-06 03:12:59 +00:00
Prefix before result set (for CustomSeparated format)
2023-06-26 18:15:19 +00:00
2024-10-06 03:12:59 +00:00
## format_custom_row_after_delimiter {#format_custom_row_after_delimiter}
2023-06-26 18:15:19 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Delimiter after field of the last column (for CustomSeparated format)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_custom_row_before_delimiter {#format_custom_row_before_delimiter}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Delimiter before field of the first column (for CustomSeparated format)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_custom_row_between_delimiter {#format_custom_row_between_delimiter}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Delimiter between rows (for CustomSeparated format)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_display_secrets_in_show_and_select {#format_display_secrets_in_show_and_select}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Enables or disables showing secrets in `SHOW` and `SELECT` queries for tables, databases,
table functions, and dictionaries.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
User wishing to see secrets must also have
[`display_secrets_in_show_and_select` server setting ](../server-configuration-parameters/settings#display_secrets_in_show_and_select )
turned on and a
[`displaySecretsInShowAndSelect` ](../../sql-reference/statements/grant#display-secrets ) privilege.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
- 0 — Disabled.
- 1 — Enabled.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_json_object_each_row_column_for_object_name {#format_json_object_each_row_column_for_object_name}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The name of column that will be used for storing/writing object names in [JSONObjectEachRow ](../../interfaces/formats.md/#jsonobjecteachrow ) format.
Column type should be String. If value is empty, default names `row_{i}` will be used for object names.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
### input_format_json_compact_allow_variable_number_of_columns {#input_format_json_compact_allow_variable_number_of_columns}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Allow variable number of columns in rows in JSONCompact/JSONCompactEachRow input formats.
Ignore extra columns in rows with more columns than expected and treat missing columns as default values.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Disabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
### output_format_markdown_escape_special_characters {#output_format_markdown_escape_special_characters}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When enabled, escape special characters in Markdown.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
[Common Mark ](https://spec.commonmark.org/0.30/#example-12 ) defines the following special characters that can be escaped by \:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
```
! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \ ] ^ _ ` { | } ~
```
2023-07-27 18:54:41 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-07-27 18:54:41 +00:00
2024-10-06 03:12:59 +00:00
+ 0 — Disable.
+ 1 — Enable.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
### input_format_json_empty_as_default {#input_format_json_empty_as_default}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When enabled, replace empty input fields in JSON with default values. For complex default expressions `input_format_defaults_for_omitted_fields` must be enabled too.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
+ 0 — Disable.
+ 1 — Enable.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_protobuf_use_autogenerated_schema {#format_protobuf_use_autogenerated_schema}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use autogenerated Protobuf when format_schema is not set
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_regexp {#format_regexp}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Regular expression (for Regexp format)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_regexp_escaping_rule {#format_regexp_escaping_rule}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: EscapingRule
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: Raw
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Field escaping rule (for Regexp format)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_regexp_skip_unmatched {#format_regexp_skip_unmatched}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Skip lines unmatched by regular expression (for Regexp format)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## format_schema {#format_schema}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
This parameter is useful when you are using formats that require a schema definition, such as [Cap’ n Proto ](https://capnproto.org/ ) or [Protobuf ](https://developers.google.com/protocol-buffers/ ). The value depends on the format.
## format_template_resultset {#format_template_resultset}
Type: String
Default value:
Path to file which contains format string for result set (for Template format)
## format_template_resultset_format {#format_template_resultset_format}
Type: String
Default value:
Format string for result set (for Template format)
## format_template_row {#format_template_row}
Type: String
Default value:
Path to file which contains format string for rows (for Template format)
## format_template_row_format {#format_template_row_format}
Type: String
Default value:
Format string for rows (for Template format)
## format_template_rows_between_delimiter {#format_template_rows_between_delimiter}
Type: String
Default value:
Delimiter between rows (for Template format)
## format_tsv_null_representation {#format_tsv_null_representation}
Type: String
Default value: \N
Custom NULL representation in TSV format
## input_format_allow_errors_num {#input_format_allow_errors_num}
Type: UInt64
Default value: 0
Sets the maximum number of acceptable errors when reading from text formats (CSV, TSV, etc.).
The default value is 0.
Always pair it with `input_format_allow_errors_ratio` .
If an error occurred while reading rows but the error counter is still less than `input_format_allow_errors_num` , ClickHouse ignores the row and moves on to the next one.
If both `input_format_allow_errors_num` and `input_format_allow_errors_ratio` are exceeded, ClickHouse throws an exception.
## input_format_allow_errors_ratio {#input_format_allow_errors_ratio}
Type: Float
Default value: 0
Sets the maximum percentage of errors allowed when reading from text formats (CSV, TSV, etc.).
The percentage of errors is set as a floating-point number between 0 and 1.
The default value is 0.
Always pair it with `input_format_allow_errors_num` .
If an error occurred while reading rows but the error counter is still less than `input_format_allow_errors_ratio` , ClickHouse ignores the row and moves on to the next one.
If both `input_format_allow_errors_num` and `input_format_allow_errors_ratio` are exceeded, ClickHouse throws an exception.
## input_format_allow_seeks {#input_format_allow_seeks}
Type: Bool
Default value: 1
Allow seeks while reading in ORC/Parquet/Arrow input formats.
Enabled by default.
## input_format_arrow_allow_missing_columns {#input_format_arrow_allow_missing_columns}
Type: Bool
Default value: 1
Allow missing columns while reading Arrow input formats
## input_format_arrow_case_insensitive_column_matching {#input_format_arrow_case_insensitive_column_matching}
Type: Bool
Default value: 0
Ignore case when matching Arrow columns with CH columns.
## input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference {#input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference}
Type: Bool
Default value: 0
Skip columns with unsupported types while schema inference for format Arrow
## input_format_avro_allow_missing_fields {#input_format_avro_allow_missing_fields}
Type: Bool
Default value: 0
For Avro/AvroConfluent format: when field is not found in schema use default value instead of error
## input_format_avro_null_as_default {#input_format_avro_null_as_default}
Type: Bool
Default value: 0
For Avro/AvroConfluent format: insert default in case of null and non Nullable column
## input_format_binary_decode_types_in_binary_format {#input_format_binary_decode_types_in_binary_format}
Type: Bool
Default value: 0
Read data types in binary format instead of type names in RowBinaryWithNamesAndTypes input format
2024-10-07 04:29:51 +00:00
### input_format_binary_read_json_as_string {#input_format_binary_read_json_as_string}
Type: Bool
Default value: 0
Read values of [JSON ](../../sql-reference/data-types/newjson.md ) data type as JSON [String ](../../sql-reference/data-types/string.md ) values in RowBinary input format.
2024-10-06 03:12:59 +00:00
## input_format_bson_skip_fields_with_unsupported_types_in_schema_inference {#input_format_bson_skip_fields_with_unsupported_types_in_schema_inference}
Type: Bool
Default value: 0
Skip fields with unsupported types while schema inference for format BSON.
## input_format_capn_proto_skip_fields_with_unsupported_types_in_schema_inference {#input_format_capn_proto_skip_fields_with_unsupported_types_in_schema_inference}
Type: Bool
Default value: 0
Skip columns with unsupported types while schema inference for format CapnProto
## input_format_csv_allow_cr_end_of_line {#input_format_csv_allow_cr_end_of_line}
Type: Bool
Default value: 0
If it is set true, \\r will be allowed at end of line not followed by \\n
## input_format_csv_allow_variable_number_of_columns {#input_format_csv_allow_variable_number_of_columns}
Type: Bool
Default value: 0
Ignore extra columns in CSV input (if file has more columns than expected) and treat missing fields in CSV input as default values
## input_format_csv_allow_whitespace_or_tab_as_delimiter {#input_format_csv_allow_whitespace_or_tab_as_delimiter}
Type: Bool
Default value: 0
Allow to use spaces and tabs(\\t) as field delimiter in the CSV strings
## input_format_csv_arrays_as_nested_csv {#input_format_csv_arrays_as_nested_csv}
Type: Bool
Default value: 0
When reading Array from CSV, expect that its elements were serialized in nested CSV and then put into string. Example: \"[\"\"Hello\"\", \"\"world\"\", \"\"42\"\"\"\" TV\"\"]\". Braces around array can be omitted.
## input_format_csv_deserialize_separate_columns_into_tuple {#input_format_csv_deserialize_separate_columns_into_tuple}
Type: Bool
Default value: 1
If it set to true, then separate columns written in CSV format can be deserialized to Tuple column.
## input_format_csv_detect_header {#input_format_csv_detect_header}
Type: Bool
Default value: 1
Automatically detect header with names and types in CSV format
## input_format_csv_empty_as_default {#input_format_csv_empty_as_default}
Type: Bool
Default value: 1
Treat empty fields in CSV input as default values.
## input_format_csv_enum_as_number {#input_format_csv_enum_as_number}
Type: Bool
Default value: 0
Treat inserted enum values in CSV formats as enum indices
## input_format_csv_skip_first_lines {#input_format_csv_skip_first_lines}
Type: UInt64
Default value: 0
Skip specified number of lines at the beginning of data in CSV format
## input_format_csv_skip_trailing_empty_lines {#input_format_csv_skip_trailing_empty_lines}
Type: Bool
Default value: 0
Skip trailing empty lines in CSV format
## input_format_csv_trim_whitespaces {#input_format_csv_trim_whitespaces}
Type: Bool
Default value: 1
Trims spaces and tabs (\\t) characters at the beginning and end in CSV strings
## input_format_csv_try_infer_numbers_from_strings {#input_format_csv_try_infer_numbers_from_strings}
Type: Bool
Default value: 0
If enabled, during schema inference ClickHouse will try to infer numbers from string fields.
It can be useful if CSV data contains quoted UInt64 numbers.
Disabled by default.
## input_format_csv_try_infer_strings_from_quoted_tuples {#input_format_csv_try_infer_strings_from_quoted_tuples}
Type: Bool
Default value: 1
Interpret quoted tuples in the input data as a value of type String.
## input_format_csv_use_best_effort_in_schema_inference {#input_format_csv_use_best_effort_in_schema_inference}
Type: Bool
Default value: 1
Use some tweaks and heuristics to infer schema in CSV format
## input_format_csv_use_default_on_bad_values {#input_format_csv_use_default_on_bad_values}
Type: Bool
Default value: 0
Allow to set default value to column when CSV field deserialization failed on bad value
## input_format_custom_allow_variable_number_of_columns {#input_format_custom_allow_variable_number_of_columns}
Type: Bool
Default value: 0
Ignore extra columns in CustomSeparated input (if file has more columns than expected) and treat missing fields in CustomSeparated input as default values
## input_format_custom_detect_header {#input_format_custom_detect_header}
Type: Bool
Default value: 1
Automatically detect header with names and types in CustomSeparated format
## input_format_custom_skip_trailing_empty_lines {#input_format_custom_skip_trailing_empty_lines}
Type: Bool
Default value: 0
Skip trailing empty lines in CustomSeparated format
## input_format_defaults_for_omitted_fields {#input_format_defaults_for_omitted_fields}
Type: Bool
Default value: 1
When performing `INSERT` queries, replace omitted input column values with default values of the respective columns. This option applies to [JSONEachRow ](../../interfaces/formats.md/#jsoneachrow ) (and other JSON formats), [CSV ](../../interfaces/formats.md/#csv ), [TabSeparated ](../../interfaces/formats.md/#tabseparated ), [TSKV ](../../interfaces/formats.md/#tskv ), [Parquet ](../../interfaces/formats.md/#parquet ), [Arrow ](../../interfaces/formats.md/#arrow ), [Avro ](../../interfaces/formats.md/#avro ), [ORC ](../../interfaces/formats.md/#orc ), [Native ](../../interfaces/formats.md/#native ) formats and formats with `WithNames` /`WithNamesAndTypes` suffixes.
:::note
When this option is enabled, extended table metadata are sent from server to client. It consumes additional computing resources on the server and can reduce performance.
:::
Possible values:
- 0 — Disabled.
- 1 — Enabled.
## input_format_force_null_for_omitted_fields {#input_format_force_null_for_omitted_fields}
Type: Bool
Default value: 0
Force initialize omitted fields with null values
## input_format_hive_text_allow_variable_number_of_columns {#input_format_hive_text_allow_variable_number_of_columns}
Type: Bool
Default value: 1
Ignore extra columns in Hive Text input (if file has more columns than expected) and treat missing fields in Hive Text input as default values
## input_format_hive_text_collection_items_delimiter {#input_format_hive_text_collection_items_delimiter}
Type: Char
Default value:
Delimiter between collection(array or map) items in Hive Text File
## input_format_hive_text_fields_delimiter {#input_format_hive_text_fields_delimiter}
Type: Char
Default value:
Delimiter between fields in Hive Text File
## input_format_hive_text_map_keys_delimiter {#input_format_hive_text_map_keys_delimiter}
Type: Char
Default value:
Delimiter between a pair of map key/values in Hive Text File
## input_format_import_nested_json {#input_format_import_nested_json}
Type: Bool
Default value: 0
Enables or disables the insertion of JSON data with nested objects.
Supported formats:
- [JSONEachRow ](../../interfaces/formats.md/#jsoneachrow )
Possible values:
- 0 — Disabled.
- 1 — Enabled.
See also:
- [Usage of Nested Structures ](../../interfaces/formats.md/#jsoneachrow-nested ) with the `JSONEachRow` format.
## input_format_ipv4_default_on_conversion_error {#input_format_ipv4_default_on_conversion_error}
Type: Bool
Default value: 0
Deserialization of IPv4 will use default values instead of throwing exception on conversion error.
Disabled by default.
## input_format_ipv6_default_on_conversion_error {#input_format_ipv6_default_on_conversion_error}
Type: Bool
Default value: 0
Deserialization of IPV6 will use default values instead of throwing exception on conversion error.
Disabled by default.
## input_format_json_compact_allow_variable_number_of_columns {#input_format_json_compact_allow_variable_number_of_columns}
Type: Bool
Default value: 0
Ignore extra columns in JSONCompact(EachRow) input (if file has more columns than expected) and treat missing fields in JSONCompact(EachRow) input as default values
## input_format_json_defaults_for_missing_elements_in_named_tuple {#input_format_json_defaults_for_missing_elements_in_named_tuple}
Type: Bool
Default value: 1
Insert default values for missing elements in JSON object while parsing named tuple.
This setting works only when setting `input_format_json_named_tuples_as_objects` is enabled.
Enabled by default.
## input_format_json_empty_as_default {#input_format_json_empty_as_default}
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Treat empty fields in JSON input as default values.
2024-01-05 20:33:30 +00:00
2024-10-06 03:12:59 +00:00
## input_format_json_ignore_unknown_keys_in_named_tuple {#input_format_json_ignore_unknown_keys_in_named_tuple}
2024-01-05 20:33:30 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2024-01-05 20:33:30 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Ignore unknown keys in json object for named tuples.
2023-01-24 22:19:50 +00:00
2023-09-25 15:42:59 +00:00
Enabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_json_ignore_unnecessary_fields {#input_format_json_ignore_unnecessary_fields}
2023-09-28 13:45:44 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-09-28 13:45:44 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-09-28 13:45:44 +00:00
2024-10-06 03:12:59 +00:00
Ignore unnecessary fields and not parse them. Enabling this may not throw exceptions on json strings of invalid format or with duplicated fields
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_json_infer_incomplete_types_as_strings {#input_format_json_infer_incomplete_types_as_strings}
2023-09-28 13:45:44 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-09-28 13:45:44 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Allow to use String type for JSON keys that contain only `Null` /`{}`/`[]` in data sample during schema inference.
In JSON formats any value can be read as String, and we can avoid errors like `Cannot determine type for column 'column_name' by first 25000 rows of data, most likely this column contains only Nulls or empty Arrays/Maps` during schema inference
by using String type for keys with unknown types.
2023-01-24 22:19:50 +00:00
Example:
```sql
2024-10-06 03:12:59 +00:00
SET input_format_json_infer_incomplete_types_as_strings = 1, input_format_json_try_infer_named_tuples_from_objects = 1;
DESCRIBE format(JSONEachRow, '{"obj" : {"a" : [1,2,3], "b" : "hello", "c" : null, "d" : {}, "e" : []}}');
SELECT * FROM format(JSONEachRow, '{"obj" : {"a" : [1,2,3], "b" : "hello", "c" : null, "d" : {}, "e" : []}}');
2023-01-24 22:19:50 +00:00
```
Result:
```
2024-10-06 03:12:59 +00:00
┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐
│ obj │ Tuple(a Array(Nullable(Int64)), b Nullable(String), c Nullable(String), d Nullable(String), e Array(Nullable(String))) │ │ │ │ │ │
└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘
┌─obj────────────────────────────┐
│ ([1,2,3],'hello',NULL,'{}',[]) │
└────────────────────────────────┘
2023-01-24 22:19:50 +00:00
```
2023-09-07 19:41:19 +00:00
Enabled by default.
2024-10-06 03:12:59 +00:00
## input_format_json_max_depth {#input_format_json_max_depth}
2023-09-07 19:41:19 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-09-07 19:41:19 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1000
2023-09-07 19:41:19 +00:00
2024-10-06 03:12:59 +00:00
Maximum depth of a field in JSON. This is not a strict limit, it does not have to be applied precisely.
2023-09-07 19:41:19 +00:00
2024-10-06 03:12:59 +00:00
## input_format_json_named_tuples_as_objects {#input_format_json_named_tuples_as_objects}
2023-09-07 19:41:19 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
Default value: 1
Parse named tuple columns as JSON objects.
2023-09-07 19:41:19 +00:00
2023-09-25 15:42:59 +00:00
Enabled by default.
## input_format_json_read_arrays_as_strings {#input_format_json_read_arrays_as_strings}
2024-10-06 03:12:59 +00:00
Type: Bool
Default value: 1
2023-09-25 15:42:59 +00:00
Allow parsing JSON arrays as strings in JSON input formats.
Example:
```sql
SET input_format_json_read_arrays_as_strings = 1;
SELECT arr, toTypeName(arr), JSONExtractArrayRaw(arr)[3] from format(JSONEachRow, 'arr String', '{"arr" : [1, "Hello", [1,2,3]]}');
```
Result:
```
┌─arr───────────────────┬─toTypeName(arr)─┬─arrayElement(JSONExtractArrayRaw(arr), 3)─┐
│ [1, "Hello", [1,2,3]] │ String │ [1,2,3] │
└───────────────────────┴─────────────────┴───────────────────────────────────────────┘
```
Enabled by default.
2024-10-06 03:12:59 +00:00
## input_format_json_read_bools_as_numbers {#input_format_json_read_bools_as_numbers}
2023-09-25 15:42:59 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
Default value: 1
Allow parsing bools as numbers in JSON input formats.
Enabled by default.
## input_format_json_read_bools_as_strings {#input_format_json_read_bools_as_strings}
Type: Bool
Default value: 1
Allow parsing bools as strings in JSON input formats.
Enabled by default.
## input_format_json_read_numbers_as_strings {#input_format_json_read_numbers_as_strings}
Type: Bool
Default value: 1
Allow parsing numbers as strings in JSON input formats.
Enabled by default.
## input_format_json_read_objects_as_strings {#input_format_json_read_objects_as_strings}
Type: Bool
Default value: 1
Allow parsing JSON objects as strings in JSON input formats.
2023-09-25 15:42:59 +00:00
Example:
```sql
2024-10-06 03:12:59 +00:00
SET input_format_json_read_objects_as_strings = 1;
CREATE TABLE test (id UInt64, obj String, date Date) ENGINE=Memory();
INSERT INTO test FORMAT JSONEachRow {"id" : 1, "obj" : {"a" : 1, "b" : "Hello"}, "date" : "2020-01-01"};
SELECT * FROM test;
2023-09-25 15:42:59 +00:00
```
Result:
2024-10-06 03:12:59 +00:00
```
┌─id─┬─obj──────────────────────┬───────date─┐
│ 1 │ {"a" : 1, "b" : "Hello"} │ 2020-01-01 │
└────┴──────────────────────────┴────────────┘
2023-09-25 15:42:59 +00:00
```
Enabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_json_throw_on_bad_escape_sequence {#input_format_json_throw_on_bad_escape_sequence}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
Default value: 1
Throw an exception if JSON string contains bad escape sequence in JSON input formats. If disabled, bad escape sequences will remain as is in the data.
2023-01-24 22:19:50 +00:00
Enabled by default.
2024-10-06 03:12:59 +00:00
## input_format_json_try_infer_named_tuples_from_objects {#input_format_json_try_infer_named_tuples_from_objects}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
If enabled, during schema inference ClickHouse will try to infer named Tuple from JSON objects.
The resulting named Tuple will contain all elements from all corresponding JSON objects from sample data.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Example:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
```sql
SET input_format_json_try_infer_named_tuples_from_objects = 1;
DESC format(JSONEachRow, '{"obj" : {"a" : 42, "b" : "Hello"}}, {"obj" : {"a" : 43, "c" : [1, 2, 3]}}, {"obj" : {"d" : {"e" : 42}}}')
```
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Result:
```
┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐
│ obj │ Tuple(a Nullable(Int64), b Nullable(String), c Array(Nullable(Int64)), d Tuple(e Nullable(Int64))) │ │ │ │ │ │
└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘
```
Enabled by default.
## input_format_json_try_infer_numbers_from_strings {#input_format_json_try_infer_numbers_from_strings}
Type: Bool
Default value: 0
If enabled, during schema inference ClickHouse will try to infer numbers from string fields.
It can be useful if JSON data contains quoted UInt64 numbers.
2023-01-24 22:19:50 +00:00
Disabled by default.
2024-10-06 03:12:59 +00:00
## input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects {#input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use String type instead of an exception in case of ambiguous paths in JSON objects during named tuples inference
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_json_validate_types_from_metadata {#input_format_json_validate_types_from_metadata}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
For JSON/JSONCompact/JSONColumnsWithMetadata input formats, if this setting is set to 1,
the types from metadata in input data will be compared with the types of the corresponding columns from the table.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Enabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_max_bytes_to_read_for_schema_inference {#input_format_max_bytes_to_read_for_schema_inference}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 33554432
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The maximum amount of data in bytes to read for automatic schema inference.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_max_rows_to_read_for_schema_inference {#input_format_max_rows_to_read_for_schema_inference}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 25000
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The maximum rows of data to read for automatic schema inference.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_msgpack_number_of_columns {#input_format_msgpack_number_of_columns}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The number of columns in inserted MsgPack data. Used for automatic schema inference from data.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_mysql_dump_map_column_names {#input_format_mysql_dump_map_column_names}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Match columns from table in MySQL dump and columns from ClickHouse table by names
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_mysql_dump_table_name {#input_format_mysql_dump_table_name}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Name of the table in MySQL dump from which to read data
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_native_allow_types_conversion {#input_format_native_allow_types_conversion}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-26 15:47:56 +00:00
2024-10-06 03:12:59 +00:00
Allow data types conversion in Native input format
2023-01-26 15:47:56 +00:00
2024-10-06 03:12:59 +00:00
## input_format_native_decode_types_in_binary_format {#input_format_native_decode_types_in_binary_format}
2023-01-26 15:47:56 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Read data types in binary format instead of type names in Native input format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_null_as_default {#input_format_null_as_default}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2024-03-25 21:58:53 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2024-03-25 21:58:53 +00:00
2024-10-06 03:12:59 +00:00
Enables or disables the initialization of [NULL ](../../sql-reference/syntax.md/#null-literal ) fields with [default values ](../../sql-reference/statements/create/table.md/#create-default-values ), if data type of these fields is not [nullable ](../../sql-reference/data-types/nullable.md/#data_type-nullable ).
If column type is not nullable and this setting is disabled, then inserting `NULL` causes an exception. If column type is nullable, then `NULL` values are inserted as is, regardless of this setting.
2024-03-25 21:58:53 +00:00
2024-10-06 03:12:59 +00:00
This setting is applicable for most input formats.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
For complex default expressions `input_format_defaults_for_omitted_fields` must be enabled too.
2023-01-24 22:19:50 +00:00
Possible values:
2024-10-06 03:12:59 +00:00
- 0 — Inserting `NULL` into a not nullable column causes an exception.
- 1 — `NULL` fields are initialized with default column values.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_orc_allow_missing_columns {#input_format_orc_allow_missing_columns}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Allow missing columns while reading ORC input formats
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_orc_case_insensitive_column_matching {#input_format_orc_case_insensitive_column_matching}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Ignore case when matching ORC columns with CH columns.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_orc_filter_push_down {#input_format_orc_filter_push_down}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When reading ORC files, skip whole stripes or row groups based on the WHERE/PREWHERE expressions, min/max statistics or bloom filter in the ORC metadata.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_orc_reader_time_zone_name {#input_format_orc_reader_time_zone_name}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: GMT
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The time zone name for ORC row reader, the default ORC row reader's time zone is GMT.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_orc_row_batch_size {#input_format_orc_row_batch_size}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Int64
2023-07-27 18:02:29 +00:00
2024-10-06 03:12:59 +00:00
Default value: 100000
2023-07-27 18:02:29 +00:00
2024-10-06 03:12:59 +00:00
Batch size when reading ORC stripes.
2023-07-27 18:02:29 +00:00
2024-10-06 03:12:59 +00:00
## input_format_orc_skip_columns_with_unsupported_types_in_schema_inference {#input_format_orc_skip_columns_with_unsupported_types_in_schema_inference}
2023-08-27 16:10:33 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-08-27 16:10:33 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-08-29 15:54:40 +00:00
2024-10-06 03:12:59 +00:00
Skip columns with unsupported types while schema inference for format ORC
2023-08-29 15:54:40 +00:00
2024-10-06 03:12:59 +00:00
## input_format_orc_use_fast_decoder {#input_format_orc_use_fast_decoder}
2023-08-27 16:10:33 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-08-27 16:10:33 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-08-27 16:10:33 +00:00
2024-10-06 03:12:59 +00:00
Use a faster ORC decoder implementation.
2024-07-17 01:19:07 +00:00
2024-10-06 03:12:59 +00:00
## input_format_parquet_allow_missing_columns {#input_format_parquet_allow_missing_columns}
2024-07-17 01:19:07 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2024-07-17 01:19:07 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2024-07-17 01:19:07 +00:00
2024-10-06 03:12:59 +00:00
Allow missing columns while reading Parquet input formats
2024-07-17 01:19:07 +00:00
2024-10-06 03:12:59 +00:00
## input_format_parquet_case_insensitive_column_matching {#input_format_parquet_case_insensitive_column_matching}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Ignore case when matching Parquet columns with CH columns.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_parquet_filter_push_down {#input_format_parquet_filter_push_down}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When reading Parquet files, skip whole row groups based on the WHERE/PREWHERE expressions and min/max statistics in the Parquet metadata.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_parquet_local_file_min_bytes_for_seek {#input_format_parquet_local_file_min_bytes_for_seek}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 8192
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Min bytes required for local read (file) to do seek, instead of read with ignore in Parquet input format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_parquet_max_block_size {#input_format_parquet_max_block_size}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 65409
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Max block size for parquet reader.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_parquet_prefer_block_bytes {#input_format_parquet_prefer_block_bytes}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 16744704
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Average block bytes output by parquet reader
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_parquet_preserve_order {#input_format_parquet_preserve_order}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Avoid reordering rows when reading from Parquet files. Usually makes it much slower.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_parquet_skip_columns_with_unsupported_types_in_schema_inference {#input_format_parquet_skip_columns_with_unsupported_types_in_schema_inference}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Skip columns with unsupported types while schema inference for format Parquet
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_parquet_use_native_reader {#input_format_parquet_use_native_reader}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When reading Parquet files, to use native reader instead of arrow reader.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_protobuf_flatten_google_wrappers {#input_format_protobuf_flatten_google_wrappers}
2024-01-31 20:03:06 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2024-01-31 20:03:06 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2024-01-31 20:03:06 +00:00
2024-10-06 03:12:59 +00:00
Enable Google wrappers for regular non-nested columns, e.g. google.protobuf.StringValue 'str' for String column 'str'. For Nullable columns empty wrappers are recognized as defaults, and missing as nulls
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_protobuf_skip_fields_with_unsupported_types_in_schema_inference {#input_format_protobuf_skip_fields_with_unsupported_types_in_schema_inference}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Skip fields with unsupported types while schema inference for format Protobuf
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_record_errors_file_path {#input_format_record_errors_file_path}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Path of the file used to record errors while reading text formats (CSV, TSV).
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_skip_unknown_fields {#input_format_skip_unknown_fields}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Enables or disables skipping insertion of extra data.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When writing data, ClickHouse throws an exception if input data contain columns that do not exist in the target table. If skipping is enabled, ClickHouse does not insert extra data and does not throw an exception.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Supported formats:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
- [JSONEachRow ](../../interfaces/formats.md/#jsoneachrow ) (and other JSON formats)
- [BSONEachRow ](../../interfaces/formats.md/#bsoneachrow ) (and other JSON formats)
- [TSKV ](../../interfaces/formats.md/#tskv )
- All formats with suffixes WithNames/WithNamesAndTypes
- [MySQLDump ](../../interfaces/formats.md/#mysqldump )
- [Native ](../../interfaces/formats.md/#native )
2023-06-06 17:33:05 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-06-06 17:33:05 +00:00
2024-10-06 03:12:59 +00:00
- 0 — Disabled.
- 1 — Enabled.
2023-06-06 17:33:05 +00:00
2024-10-06 03:12:59 +00:00
## input_format_try_infer_dates {#input_format_try_infer_dates}
2023-07-27 18:02:29 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-07-27 18:02:29 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-07-27 18:02:29 +00:00
2024-10-06 03:12:59 +00:00
If enabled, ClickHouse will try to infer type `Date` from string fields in schema inference for text formats. If all fields from a column in input data were successfully parsed as dates, the result type will be `Date` , if at least one field was not parsed as date, the result type will be `String` .
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Enabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_try_infer_datetimes {#input_format_try_infer_datetimes}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
If enabled, ClickHouse will try to infer type `DateTime64` from string fields in schema inference for text formats. If all fields from a column in input data were successfully parsed as datetimes, the result type will be `DateTime64` , if at least one field was not parsed as datetime, the result type will be `String` .
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Enabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_try_infer_datetimes_only_datetime64 {#input_format_try_infer_datetimes_only_datetime64}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When input_format_try_infer_datetimes is enabled, infer only DateTime64 but not DateTime types
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_try_infer_exponent_floats {#input_format_try_infer_exponent_floats}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-09-06 03:28:53 +00:00
2024-10-06 03:12:59 +00:00
Try to infer floats in exponential notation while schema inference in text formats (except JSON, where exponent numbers are always inferred)
2023-09-06 03:28:53 +00:00
2024-10-06 03:12:59 +00:00
## input_format_try_infer_integers {#input_format_try_infer_integers}
2023-09-06 03:28:53 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
If enabled, ClickHouse will try to infer integers instead of floats in schema inference for text formats. If all numbers in the column from input data are integers, the result type will be `Int64` , if at least one number is float, the result type will be `Float64` .
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Enabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_try_infer_variants {#input_format_try_infer_variants}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
If enabled, ClickHouse will try to infer type [`Variant` ](../../sql-reference/data-types/variant.md ) in schema inference for text formats when there is more than one possible type for column/array elements.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
- 0 — Disabled.
- 1 — Enabled.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_tsv_allow_variable_number_of_columns {#input_format_tsv_allow_variable_number_of_columns}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Ignore extra columns in TSV input (if file has more columns than expected) and treat missing fields in TSV input as default values
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_tsv_crlf_end_of_line {#input_format_tsv_crlf_end_of_line}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
If it is set true, file function will read TSV format with \\r\\n instead of \\n.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_tsv_detect_header {#input_format_tsv_detect_header}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Automatically detect header with names and types in TSV format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_tsv_empty_as_default {#input_format_tsv_empty_as_default}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Treat empty fields in TSV input as default values.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_tsv_enum_as_number {#input_format_tsv_enum_as_number}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Treat inserted enum values in TSV formats as enum indices.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_tsv_skip_first_lines {#input_format_tsv_skip_first_lines}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Skip specified number of lines at the beginning of data in TSV format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_tsv_skip_trailing_empty_lines {#input_format_tsv_skip_trailing_empty_lines}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Skip trailing empty lines in TSV format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_tsv_use_best_effort_in_schema_inference {#input_format_tsv_use_best_effort_in_schema_inference}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use some tweaks and heuristics to infer schema in TSV format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## input_format_values_accurate_types_of_literals {#input_format_values_accurate_types_of_literals}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
For Values format: when parsing and interpreting expressions using template, check actual type of literal to avoid possible overflow and precision issues.
2023-06-06 17:33:05 +00:00
2024-10-06 03:12:59 +00:00
## input_format_values_deduce_templates_of_expressions {#input_format_values_deduce_templates_of_expressions}
2023-06-06 17:33:05 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-06-06 17:33:05 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
For Values format: if the field could not be parsed by streaming parser, run SQL parser, deduce template of the SQL expression, try to parse all rows using template and then interpret expression for all rows.
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
## input_format_values_interpret_expressions {#input_format_values_interpret_expressions}
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
For Values format: if the field could not be parsed by streaming parser, run SQL parser and try to interpret it as SQL expression.
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
## input_format_with_names_use_header {#input_format_with_names_use_header}
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
Enables or disables checking the column order when inserting data.
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
To improve insert performance, we recommend disabling this check if you are sure that the column order of the input data is the same as in the target table.
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
Supported formats:
2023-06-14 16:44:31 +00:00
2024-10-06 03:12:59 +00:00
- [CSVWithNames ](../../interfaces/formats.md/#csvwithnames )
- [CSVWithNamesAndTypes ](../../interfaces/formats.md/#csvwithnamesandtypes )
- [TabSeparatedWithNames ](../../interfaces/formats.md/#tabseparatedwithnames )
- [TabSeparatedWithNamesAndTypes ](../../interfaces/formats.md/#tabseparatedwithnamesandtypes )
- [JSONCompactEachRowWithNames ](../../interfaces/formats.md/#jsoncompacteachrowwithnames )
- [JSONCompactEachRowWithNamesAndTypes ](../../interfaces/formats.md/#jsoncompacteachrowwithnamesandtypes )
- [JSONCompactStringsEachRowWithNames ](../../interfaces/formats.md/#jsoncompactstringseachrowwithnames )
- [JSONCompactStringsEachRowWithNamesAndTypes ](../../interfaces/formats.md/#jsoncompactstringseachrowwithnamesandtypes )
- [RowBinaryWithNames ](../../interfaces/formats.md/#rowbinarywithnames )
- [RowBinaryWithNamesAndTypes ](../../interfaces/formats.md/#rowbinarywithnamesandtypes )
- [CustomSeparatedWithNames ](../../interfaces/formats.md/#customseparatedwithnames )
- [CustomSeparatedWithNamesAndTypes ](../../interfaces/formats.md/#customseparatedwithnamesandtypes )
2023-07-27 18:02:29 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-06-14 16:44:31 +00:00
2024-10-06 03:12:59 +00:00
- 0 — Disabled.
- 1 — Enabled.
2023-06-14 16:44:31 +00:00
2024-10-06 03:12:59 +00:00
## input_format_with_types_use_header {#input_format_with_types_use_header}
2023-05-25 07:51:32 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
Controls whether format parser should check if data types from the input data match data types from the target table.
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
Supported formats:
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
- [CSVWithNamesAndTypes ](../../interfaces/formats.md/#csvwithnamesandtypes )
- [TabSeparatedWithNamesAndTypes ](../../interfaces/formats.md/#tabseparatedwithnamesandtypes )
- [JSONCompactEachRowWithNamesAndTypes ](../../interfaces/formats.md/#jsoncompacteachrowwithnamesandtypes )
- [JSONCompactStringsEachRowWithNamesAndTypes ](../../interfaces/formats.md/#jsoncompactstringseachrowwithnamesandtypes )
- [RowBinaryWithNamesAndTypes ](../../interfaces/formats.md/#rowbinarywithnamesandtypes-rowbinarywithnamesandtypes )
- [CustomSeparatedWithNamesAndTypes ](../../interfaces/formats.md/#customseparatedwithnamesandtypes )
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
- 0 — Disabled.
- 1 — Enabled.
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
## insert_distributed_one_random_shard {#insert_distributed_one_random_shard}
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
Enables or disables random shard insertion into a [Distributed ](../../engines/table-engines/special/distributed.md/#distributed ) table when there is no distributed key.
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
By default, when inserting data into a `Distributed` table with more than one shard, the ClickHouse server will reject any insertion request if there is no distributed key. When `insert_distributed_one_random_shard = 1` , insertions are allowed and data is forwarded randomly among all shards.
2023-06-12 03:57:52 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-07-19 12:51:11 +00:00
2024-10-06 03:12:59 +00:00
- 0 — Insertion is rejected if there are multiple shards and no distributed key is given.
- 1 — Insertion is done randomly among all available shards when no distributed key is given.
2023-07-19 12:51:11 +00:00
2024-10-06 03:12:59 +00:00
## interval_output_format {#interval_output_format}
2023-07-19 12:51:11 +00:00
2024-10-06 03:12:59 +00:00
Type: IntervalOutputFormat
2023-07-19 12:51:11 +00:00
2024-10-06 03:12:59 +00:00
Default value: numeric
2023-07-19 12:51:11 +00:00
2024-10-06 03:12:59 +00:00
Allows choosing different output formats of the text representation of interval types.
2023-07-19 12:51:11 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-07-19 12:51:11 +00:00
2024-10-06 03:12:59 +00:00
- `kusto` - KQL-style output format.
2023-07-19 12:51:11 +00:00
2024-10-06 03:12:59 +00:00
ClickHouse outputs intervals in [KQL format ](https://learn.microsoft.com/en-us/dotnet/standard/base-types/standard-timespan-format-strings#the-constant-c-format-specifier ). For example, `toIntervalDay(2)` would be formatted as `2.00:00:00` . Please note that for interval types of varying length (ie. `IntervalMonth` and `IntervalYear` ) the average number of seconds per interval is taken into account.
2023-11-16 13:31:04 +00:00
2024-10-06 03:12:59 +00:00
- `numeric` - Numeric output format.
2023-11-16 13:31:04 +00:00
2024-10-06 03:12:59 +00:00
ClickHouse outputs intervals as their underlying numeric representation. For example, `toIntervalDay(2)` would be formatted as `2` .
2023-11-16 13:31:04 +00:00
2024-10-06 03:12:59 +00:00
See also:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
- [Interval ](../../sql-reference/data-types/special-data-types/interval.md )
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_arrow_compression_method {#output_format_arrow_compression_method}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: ArrowCompression
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: lz4_frame
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Compression method for Arrow output format. Supported codecs: lz4_frame, zstd, none (uncompressed)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_arrow_fixed_string_as_fixed_byte_array {#output_format_arrow_fixed_string_as_fixed_byte_array}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use Arrow FIXED_SIZE_BINARY type instead of Binary for FixedString columns.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_arrow_low_cardinality_as_dictionary {#output_format_arrow_low_cardinality_as_dictionary}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Enable output LowCardinality type as Dictionary Arrow type
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_arrow_string_as_string {#output_format_arrow_string_as_string}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use Arrow String type instead of Binary for String columns
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_arrow_use_64_bit_indexes_for_dictionary {#output_format_arrow_use_64_bit_indexes_for_dictionary}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Always use 64 bit integers for dictionary indexes in Arrow format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_arrow_use_signed_indexes_for_dictionary {#output_format_arrow_use_signed_indexes_for_dictionary}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use signed integers for dictionary indexes in Arrow format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_avro_codec {#output_format_avro_codec}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Compression codec used for output. Possible values: 'null', 'deflate', 'snappy', 'zstd'.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_avro_rows_in_file {#output_format_avro_rows_in_file}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Max rows in a file (if permitted by storage)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_avro_string_column_pattern {#output_format_avro_string_column_pattern}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
For Avro format: regexp of String columns to select as AVRO string.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_avro_sync_interval {#output_format_avro_sync_interval}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 16384
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Sync interval in bytes.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_binary_encode_types_in_binary_format {#output_format_binary_encode_types_in_binary_format}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-07 04:29:51 +00:00
Write data types in binary format instead of type names in RowBinaryWithNamesAndTypes output format
2023-01-24 22:19:50 +00:00
2024-10-07 04:29:51 +00:00
### output_format_binary_write_json_as_string {#output_format_binary_write_json_as_string}
2023-01-24 22:19:50 +00:00
2024-10-07 04:29:51 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-07 04:29:51 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-07 04:29:51 +00:00
Write values of [JSON ](../../sql-reference/data-types/newjson.md ) data type as JSON [String ](../../sql-reference/data-types/string.md ) values in RowBinary output format.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_bson_string_as_string {#output_format_bson_string_as_string}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use BSON String type instead of Binary for String columns.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_csv_crlf_end_of_line {#output_format_csv_crlf_end_of_line}
2024-01-04 17:50:53 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2024-01-04 17:50:53 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2024-01-04 17:50:53 +00:00
2024-10-06 03:12:59 +00:00
If it is set true, end of line in CSV format will be \\r\\n instead of \\n.
2024-01-04 17:50:53 +00:00
2024-10-06 03:12:59 +00:00
## output_format_csv_serialize_tuple_into_separate_columns {#output_format_csv_serialize_tuple_into_separate_columns}
2024-01-04 17:50:53 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2024-01-04 17:50:53 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2024-01-04 17:50:53 +00:00
2024-10-06 03:12:59 +00:00
If it set to true, then Tuples in CSV format are serialized as separate columns (that is, their nesting in the tuple is lost)
2024-01-04 17:50:53 +00:00
2024-10-06 03:12:59 +00:00
## output_format_decimal_trailing_zeros {#output_format_decimal_trailing_zeros}
2024-01-04 17:50:53 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2024-01-04 17:50:53 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Output trailing zeros when printing Decimal values. E.g. 1.230000 instead of 1.23.
2023-01-24 22:19:50 +00:00
Disabled by default.
2024-10-06 03:12:59 +00:00
## output_format_enable_streaming {#output_format_enable_streaming}
2023-02-08 11:32:26 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-02-08 11:32:26 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-02-08 11:32:26 +00:00
2024-10-06 03:12:59 +00:00
Enable streaming in output formats that support it.
2023-03-02 10:36:07 +00:00
2024-10-06 03:12:59 +00:00
Disabled by default.
2023-03-02 10:36:07 +00:00
2024-10-06 03:12:59 +00:00
## output_format_json_array_of_rows {#output_format_json_array_of_rows}
2023-03-02 10:36:07 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Enables the ability to output all rows as a JSON array in the [JSONEachRow ](../../interfaces/formats.md/#jsoneachrow ) format.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
- 1 — ClickHouse outputs all rows as an array, each row in the `JSONEachRow` format.
- 0 — ClickHouse outputs each row separately in the `JSONEachRow` format.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
**Example of a query with the enabled setting**
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Query:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
```sql
SET output_format_json_array_of_rows = 1;
SELECT number FROM numbers(3) FORMAT JSONEachRow;
```
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Result:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
```text
[
{"number":"0"},
{"number":"1"},
{"number":"2"}
]
```
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
**Example of a query with the disabled setting**
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Query:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
```sql
SET output_format_json_array_of_rows = 0;
SELECT number FROM numbers(3) FORMAT JSONEachRow;
```
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Result:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
```text
{"number":"0"}
{"number":"1"}
{"number":"2"}
```
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_json_escape_forward_slashes {#output_format_json_escape_forward_slashes}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-03-02 10:36:07 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-03-02 10:36:07 +00:00
2024-10-06 03:12:59 +00:00
Controls escaping forward slashes for string outputs in JSON output format. This is intended for compatibility with JavaScript. Don't confuse with backslashes that are always escaped.
2023-03-02 10:36:07 +00:00
2024-10-06 03:12:59 +00:00
Enabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_json_named_tuples_as_objects {#output_format_json_named_tuples_as_objects}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Serialize named tuple columns as JSON objects.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Enabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_json_quote_64bit_floats {#output_format_json_quote_64bit_floats}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Controls quoting of 64-bit [floats ](../../sql-reference/data-types/float.md ) when they are output in JSON* formats.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Disabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_json_quote_64bit_integers {#output_format_json_quote_64bit_integers}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-08-17 12:28:01 +00:00
2024-10-06 03:12:59 +00:00
Controls quoting of 64-bit or bigger [integers ](../../sql-reference/data-types/int-uint.md ) (like `UInt64` or `Int128` ) when they are output in a [JSON ](../../interfaces/formats.md/#json ) format.
Such integers are enclosed in quotes by default. This behavior is compatible with most JavaScript implementations.
2023-08-17 12:28:01 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-08-17 12:28:01 +00:00
2024-10-06 03:12:59 +00:00
- 0 — Integers are output without quotes.
- 1 — Integers are enclosed in quotes.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_json_quote_decimals {#output_format_json_quote_decimals}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-02-08 11:32:26 +00:00
2024-10-06 03:12:59 +00:00
Controls quoting of decimals in JSON output formats.
2023-02-08 11:32:26 +00:00
2024-10-06 03:12:59 +00:00
Disabled by default.
2023-02-08 11:32:26 +00:00
2024-10-06 03:12:59 +00:00
## output_format_json_quote_denormals {#output_format_json_quote_denormals}
2023-02-23 16:14:10 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-02-23 16:14:10 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-02-23 16:14:10 +00:00
2024-10-06 03:12:59 +00:00
Enables `+nan` , `-nan` , `+inf` , `-inf` outputs in [JSON ](../../interfaces/formats.md/#json ) output format.
2023-03-02 10:36:07 +00:00
2024-10-06 03:12:59 +00:00
Possible values:
2023-03-02 10:36:07 +00:00
2024-10-06 03:12:59 +00:00
- 0 — Disabled.
- 1 — Enabled.
2023-03-02 10:36:07 +00:00
2024-10-06 03:12:59 +00:00
**Example**
2024-05-31 02:31:14 +00:00
2024-10-06 03:12:59 +00:00
Consider the following table `account_orders` :
2024-05-31 02:31:14 +00:00
2024-10-06 03:12:59 +00:00
```text
┌─id─┬─name───┬─duration─┬─period─┬─area─┐
│ 1 │ Andrew │ 20 │ 0 │ 400 │
│ 2 │ John │ 40 │ 0 │ 0 │
│ 3 │ Bob │ 15 │ 0 │ -100 │
└────┴────────┴──────────┴────────┴──────┘
```
2024-05-31 02:31:14 +00:00
2024-10-06 03:12:59 +00:00
When `output_format_json_quote_denormals = 0` , the query returns `null` values in output:
2024-05-31 02:31:14 +00:00
2024-10-06 03:12:59 +00:00
```sql
SELECT area/period FROM account_orders FORMAT JSON;
```
2024-06-20 09:43:13 +00:00
2024-10-06 03:12:59 +00:00
```json
{
"meta":
[
{
"name": "divide(area, period)",
"type": "Float64"
}
],
2024-06-20 09:43:13 +00:00
2024-10-06 03:12:59 +00:00
"data":
[
{
"divide(area, period)": null
},
{
"divide(area, period)": null
},
{
"divide(area, period)": null
}
],
2024-06-20 09:43:13 +00:00
2024-10-06 03:12:59 +00:00
"rows": 3,
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
"statistics":
{
"elapsed": 0.003648093,
"rows_read": 3,
"bytes_read": 24
}
}
```
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When `output_format_json_quote_denormals = 1` , the query returns:
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
```json
{
"meta":
[
{
"name": "divide(area, period)",
"type": "Float64"
}
],
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
"data":
[
{
"divide(area, period)": "inf"
},
{
"divide(area, period)": "-nan"
},
{
"divide(area, period)": "-inf"
}
],
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
"rows": 3,
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
"statistics":
{
"elapsed": 0.000070241,
"rows_read": 3,
"bytes_read": 24
}
}
```
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_json_skip_null_value_in_named_tuples {#output_format_json_skip_null_value_in_named_tuples}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Skip key value pairs with null value when serialize named tuple columns as JSON objects. It is only valid when output_format_json_named_tuples_as_objects is true.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_json_validate_utf8 {#output_format_json_validate_utf8}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Controls validation of UTF-8 sequences in JSON output formats, doesn't impact formats JSON/JSONCompact/JSONColumnsWithMetadata, they always validate UTF-8.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Disabled by default.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_markdown_escape_special_characters {#output_format_markdown_escape_special_characters}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Escape special characters in Markdown
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_msgpack_uuid_representation {#output_format_msgpack_uuid_representation}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: MsgPackUUIDRepresentation
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: ext
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The way how to output UUID in MsgPack format.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_native_encode_types_in_binary_format {#output_format_native_encode_types_in_binary_format}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-07-27 18:54:41 +00:00
2024-10-06 03:12:59 +00:00
Write data types in binary format instead of type names in Native output format
2023-07-27 18:54:41 +00:00
2024-10-07 04:31:47 +00:00
### output_format_native_write_json_as_string {#output_format_native_write_json_as_string}
Type: Bool
Default value: 0
Write data of [JSON ](../../sql-reference/data-types/newjson.md ) column as [String ](../../sql-reference/data-types/string.md ) column containing JSON strings instead of default native JSON serialization.
2024-10-06 03:12:59 +00:00
## output_format_orc_compression_method {#output_format_orc_compression_method}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: ORCCompression
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: zstd
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Compression method for ORC output format. Supported codecs: lz4, snappy, zlib, zstd, none (uncompressed)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_orc_dictionary_key_size_threshold {#output_format_orc_dictionary_key_size_threshold}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Double
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
For a string column in ORC output format, if the number of distinct values is greater than this fraction of the total number of non-null rows, turn off dictionary encoding. Otherwise dictionary encoding is enabled
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_orc_row_index_stride {#output_format_orc_row_index_stride}
2023-05-08 20:49:43 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-05-08 20:49:43 +00:00
2024-10-06 03:12:59 +00:00
Default value: 10000
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Target row index stride in ORC output format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_orc_string_as_string {#output_format_orc_string_as_string}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use ORC String type instead of Binary for String columns
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_batch_size {#output_format_parquet_batch_size}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1024
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Check page size every this many rows. Consider decreasing if you have columns with average values size above a few KBs.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_compliant_nested_types {#output_format_parquet_compliant_nested_types}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
In parquet file schema, use name 'element' instead of 'item' for list elements. This is a historical artifact of Arrow library implementation. Generally increases compatibility, except perhaps with some old versions of Arrow.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_compression_method {#output_format_parquet_compression_method}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: ParquetCompression
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: zstd
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Compression method for Parquet output format. Supported codecs: snappy, lz4, brotli, zstd, gzip, none (uncompressed)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_data_page_size {#output_format_parquet_data_page_size}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1048576
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Target page size in bytes, before compression.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_fixed_string_as_fixed_byte_array {#output_format_parquet_fixed_string_as_fixed_byte_array}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use Parquet FIXED_LENGTH_BYTE_ARRAY type instead of Binary for FixedString columns.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_parallel_encoding {#output_format_parquet_parallel_encoding}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Do Parquet encoding in multiple threads. Requires output_format_parquet_use_custom_encoder.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_row_group_size {#output_format_parquet_row_group_size}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1000000
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Target row group size in rows.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_row_group_size_bytes {#output_format_parquet_row_group_size_bytes}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2024-01-12 17:46:03 +00:00
2024-10-06 03:12:59 +00:00
Default value: 536870912
2024-01-12 17:46:03 +00:00
2024-10-06 03:12:59 +00:00
Target row group size in bytes, before compression.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_string_as_string {#output_format_parquet_string_as_string}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use Parquet String type instead of Binary for String columns.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_use_custom_encoder {#output_format_parquet_use_custom_encoder}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use a faster Parquet encoder implementation.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_version {#output_format_parquet_version}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: ParquetVersion
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 2.latest
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Parquet format version for output format. Supported versions: 1.0, 2.4, 2.6 and 2.latest (default)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_parquet_write_page_index {#output_format_parquet_write_page_index}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2024-02-26 12:19:53 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2024-02-26 12:19:53 +00:00
2024-10-06 03:12:59 +00:00
Add a possibility to write page index into parquet files.
2024-02-26 12:19:53 +00:00
2024-10-06 03:12:59 +00:00
## output_format_pretty_color {#output_format_pretty_color}
2024-02-26 12:19:53 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64Auto
2024-02-26 12:19:53 +00:00
2024-10-06 03:12:59 +00:00
Default value: auto
2024-02-26 12:19:53 +00:00
2024-10-06 03:12:59 +00:00
Use ANSI escape sequences in Pretty formats. 0 - disabled, 1 - enabled, 'auto' - enabled if a terminal.
2024-02-26 12:19:53 +00:00
2024-10-06 03:12:59 +00:00
## output_format_pretty_display_footer_column_names {#output_format_pretty_display_footer_column_names}
2024-02-26 12:19:53 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2024-02-26 12:19:53 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2024-06-11 13:13:36 +00:00
2024-06-12 07:47:57 +00:00
Display column names in the footer if there are many table rows.
2024-06-11 13:13:36 +00:00
Possible values:
- 0 — No column names are displayed in the footer.
2024-06-12 07:47:57 +00:00
- 1 — Column names are displayed in the footer if row count is greater than or equal to the threshold value set by [output_format_pretty_display_footer_column_names_min_rows ](#output_format_pretty_display_footer_column_names_min_rows ) (50 by default).
2024-06-12 07:45:36 +00:00
2024-06-11 13:13:36 +00:00
**Example**
Query:
```sql
SELECT *, toTypeName(* ) FROM (SELECT * FROM system.numbers LIMIT 1000);
```
Result:
```response
┌─number─┬─toTypeName(number)─┐
1. │ 0 │ UInt64 │
2. │ 1 │ UInt64 │
3. │ 2 │ UInt64 │
...
999. │ 998 │ UInt64 │
1000. │ 999 │ UInt64 │
└─number─┴─toTypeName(number)─┘
```
2024-10-06 03:12:59 +00:00
## output_format_pretty_display_footer_column_names_min_rows {#output_format_pretty_display_footer_column_names_min_rows}
Type: UInt64
Default value: 50
2024-06-11 13:13:36 +00:00
2024-06-12 07:45:36 +00:00
Sets the minimum number of rows for which a footer with column names will be displayed if setting [output_format_pretty_display_footer_column_names ](#output_format_pretty_display_footer_column_names ) is enabled.
2024-06-11 13:13:36 +00:00
2024-10-06 03:12:59 +00:00
## output_format_pretty_grid_charset {#output_format_pretty_grid_charset}
2024-06-11 13:13:36 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: UTF-8
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Charset for printing grid borders. Available charsets: ASCII, UTF-8 (default one).
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_pretty_highlight_digit_groups {#output_format_pretty_highlight_digit_groups}
2024-01-29 20:25:58 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2024-01-29 20:25:58 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
If enabled and if output is a terminal, highlight every digit corresponding to the number of thousands, millions, etc. with underline.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_pretty_max_column_pad_width {#output_format_pretty_max_column_pad_width}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 250
2024-01-18 19:53:56 +00:00
2024-10-06 03:12:59 +00:00
Maximum width to pad all values in a column in Pretty formats.
2024-01-18 19:53:56 +00:00
2024-10-06 03:12:59 +00:00
## output_format_pretty_max_rows {#output_format_pretty_max_rows}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 10000
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Rows limit for Pretty formats.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_pretty_max_value_width {#output_format_pretty_max_value_width}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 10000
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Maximum width of value to display in Pretty formats. If greater - it will be cut.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_pretty_max_value_width_apply_for_single_value {#output_format_pretty_max_value_width_apply_for_single_value}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Only cut values (see the `output_format_pretty_max_value_width` setting) when it is not a single value in a block. Otherwise output it entirely, which is useful for the `SHOW CREATE TABLE` query.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_pretty_row_numbers {#output_format_pretty_row_numbers}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Add row numbers before each row for pretty output format
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_pretty_single_large_number_tip_threshold {#output_format_pretty_single_large_number_tip_threshold}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1000000
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Print a readable number tip on the right side of the table if the block consists of a single number which exceeds this value (except 0)
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_protobuf_nullables_with_google_wrappers {#output_format_protobuf_nullables_with_google_wrappers}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When serializing Nullable columns with Google wrappers, serialize default values as empty wrappers. If turned off, default and null values are not serialized
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_schema {#output_format_schema}
2023-06-06 17:33:05 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-06-06 17:33:05 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2023-06-06 17:33:05 +00:00
2024-10-06 03:12:59 +00:00
The path to the file where the automatically generated schema will be saved in [Cap’ n Proto ](../../interfaces/formats.md#capnproto-capnproto ) or [Protobuf ](../../interfaces/formats.md#protobuf-protobuf ) formats.
2023-07-27 18:02:29 +00:00
2024-10-06 03:12:59 +00:00
## output_format_sql_insert_include_column_names {#output_format_sql_insert_include_column_names}
2023-07-27 18:02:29 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-07-27 18:02:29 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Include column names in INSERT query
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_sql_insert_max_batch_size {#output_format_sql_insert_max_batch_size}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 65409
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The maximum number of rows in one INSERT statement.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_sql_insert_quote_names {#output_format_sql_insert_quote_names}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Quote column names with '`' characters
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_sql_insert_table_name {#output_format_sql_insert_table_name}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: table
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
The name of table in the output INSERT query
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_sql_insert_use_replace {#output_format_sql_insert_use_replace}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-07-27 18:54:41 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-07-27 18:54:41 +00:00
2024-10-06 03:12:59 +00:00
Use REPLACE statement instead of INSERT
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_tsv_crlf_end_of_line {#output_format_tsv_crlf_end_of_line}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
If it is set true, end of line in TSV format will be \\r\\n instead of \\n.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_values_escape_quote_with_quote {#output_format_values_escape_quote_with_quote}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
If true escape ' with '', otherwise quoted with \\'
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## output_format_write_statistics {#output_format_write_statistics}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Write statistics about read rows, bytes, time elapsed in suitable output formats.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Enabled by default
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## precise_float_parsing {#precise_float_parsing}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Prefer more precise (but slower) float parsing algorithm
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## regexp_dict_allow_hyperscan {#regexp_dict_allow_hyperscan}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Allow regexp_tree dictionary using Hyperscan library.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## regexp_dict_flag_case_insensitive {#regexp_dict_flag_case_insensitive}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Use case-insensitive matching for a regexp_tree dictionary. Can be overridden in individual expressions with (?i) and (?-i).
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## regexp_dict_flag_dotall {#regexp_dict_flag_dotall}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Allow '.' to match newline characters for a regexp_tree dictionary.
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## rows_before_aggregation {#rows_before_aggregation}
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Type: Bool
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
Default value: 0
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
When enabled, ClickHouse will provide exact value for rows_before_aggregation statistic, represents the number of rows read before aggregation
2023-01-24 22:19:50 +00:00
2024-10-06 03:12:59 +00:00
## schema_inference_hints {#schema_inference_hints}
2023-02-27 19:28:19 +00:00
2024-10-06 03:12:59 +00:00
Type: String
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Default value:
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
The list of column names and types to use as hints in schema inference for formats without schema.
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Example:
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Query:
```sql
desc format(JSONEachRow, '{"x" : 1, "y" : "String", "z" : "0.0.0.0" }') settings schema_inference_hints='x UInt8, z IPv4';
```
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Result:
```sql
x UInt8
y Nullable(String)
z IPv4
```
:::note
2024-10-03 00:49:54 +00:00
If the `schema_inference_hints` is not formatted properly, or if there is a typo or a wrong datatype, etc... the whole schema_inference_hints will be ignored.
2024-10-06 03:12:59 +00:00
:::
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
## schema_inference_make_columns_nullable {#schema_inference_make_columns_nullable}
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Type: UInt64Auto
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Controls making inferred types `Nullable` in schema inference.
If the setting is enabled, all inferred type will be `Nullable` , if disabled, the inferred type will never be `Nullable` , if set to `auto` , the inferred type will be `Nullable` only if the column contains `NULL` in a sample that is parsed during schema inference or file metadata contains information about column nullability.
2023-02-27 19:28:19 +00:00
2024-10-06 03:12:59 +00:00
## schema_inference_mode {#schema_inference_mode}
2023-02-27 19:28:19 +00:00
2024-10-06 03:12:59 +00:00
Type: SchemaInferenceMode
2023-02-27 19:28:19 +00:00
2024-10-06 03:12:59 +00:00
Default value: default
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Mode of schema inference. 'default' - assume that all files have the same schema and schema can be inferred from any file, 'union' - files can have different schemas and the resulting schema should be the a union of schemas of all files
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
## show_create_query_identifier_quoting_rule {#show_create_query_identifier_quoting_rule}
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Type: IdentifierQuotingRule
Default value: when_necessary
Set the quoting rule for identifiers in SHOW CREATE query
## show_create_query_identifier_quoting_style {#show_create_query_identifier_quoting_style}
Type: IdentifierQuotingStyle
Default value: Backticks
Set the quoting style for identifiers in SHOW CREATE query
## type_json_skip_duplicated_paths {#type_json_skip_duplicated_paths}
Type: Bool
Default value: 0
When enabled, during parsing JSON object into JSON type duplicated paths will be ignored and only the first one will be inserted instead of an exception
## validate_experimental_and_suspicious_types_inside_nested_types {#validate_experimental_and_suspicious_types_inside_nested_types}
Type: Bool
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Default value: 1
2024-06-21 18:20:35 +00:00
2024-10-06 03:12:59 +00:00
Validate usage of experimental and suspicious types inside nested types like Array/Map/Tuple