thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-02 04:22:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into debug_in_ci

Browse Source
This commit is contained in:
Nikita Taranov 2024-01-17 13:48:39 +01:00 committed by GitHub
commit c291a11e5d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
56 changed files with 847 additions and 145 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

207
docs/en/operations/allocation-profiling.md Normal file
View File

@ -0,0 +1,207 @@
---
slug: /en/operations/allocation-profiling
sidebar_label: "Allocation profiling"
title: "Allocation profiling"
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Allocation profiling
ClickHouse uses [jemalloc](https://github.com/jemalloc/jemalloc) as its global allocator that comes with some tools for allocation sampling and profiling.
To make allocation profiling more convenient, `SYSTEM` commands are provided along 4LW commands in Keeper.
## Sampling allocations and flushing heap profiles
If we want to sample and profile allocations in `jemalloc`, we need to start ClickHouse/Keeper with profiling enabled using environment variable `MALLOC_CONF`.
```sh
MALLOC_CONF=background_thread:true,prof:true
```
`jemalloc` will sample allocation and store the information internally.
We can tell `jemalloc` to flush current profile by running:
<Tabs groupId="binary">
<TabItem value="clickhouse" label="ClickHouse">
SYSTEM JEMALLOC FLUSH PROFILE
</TabItem>
<TabItem value="keeper" label="Keeper">
echo jmfp | nc localhost 9181
</TabItem>
</Tabs>
By default, heap profile file will be generated in `/tmp/jemalloc_clickhouse._pid_._seqnum_.heap` where `_pid_` is the PID of ClickHouse and `_seqnum_` is the global sequence number for the current heap profile.
For Keeper, the default file is `/tmp/jemalloc_keeper._pid_._seqnum_.heap` following the same rules.
A different location can be defined by appending the `MALLOC_CONF` environment variable with `prof_prefix` option.
For example, if we want to generate profiles in `/data` folder where the prefix for filename will be `my_current_profile` we can run ClickHouse/Keeper with following environment variable:
```sh
MALLOC_CONF=background_thread:true,prof:true,prof_prefix:/data/my_current_profile
```
Generated file will append to prefix PID and sequence number.
## Analyzing heap profiles
After we generated heap profiles, we need to analyze them.
For that, we need to use `jemalloc`'s tool called [jeprof](https://github.com/jemalloc/jemalloc/blob/dev/bin/jeprof.in) which can be installed in multiple ways:
- installing `jemalloc` using system's package manager
- cloning [jemalloc repo](https://github.com/jemalloc/jemalloc) and running autogen.sh from the root folder that will provide you with `jeprof` script inside the `bin` folder
:::note
`jeprof` uses `addr2line` to generate stacktraces which can be really slow.
If that’s the case, we recommend installing an [alternative implementation](https://github.com/gimli-rs/addr2line) of the tool.
```
git clone https://github.com/gimli-rs/addr2line
cd addr2line
cargo b --examples -r
cp ./target/release/examples/addr2line path/to/current/addr2line
```
:::
There are many different formats to generate from the heap profile using `jeprof`.
We recommend to run `jeprof --help` to check usage and many different options the tool provides.
In general, `jeprof` command will look like this:
```sh
jeprof path/to/binary path/to/heap/profile --output_format [ > output_file]
```
If we want to compare which allocations happened between 2 profiles we can set the base argument:
```sh
jeprof path/to/binary --base path/to/first/heap/profile path/to/second/heap/profile --output_format [ > output_file]
```
For example:
- if we want to generate a text file with each procedure written per line:
```sh
jeprof path/to/binary path/to/heap/profile --text > result.txt
```
- if we want to generate a PDF file with call-graph:
```sh
jeprof path/to/binary path/to/heap/profile --pdf > result.pdf
```
### Generating flame graph
`jeprof` allows us to generate collapsed stacks for building flame graphs.
We need to use `--collapsed` argument:
```sh
jeprof path/to/binary path/to/heap/profile --collapsed > result.collapsed
```
After that, we can use many different tools to visualize collapsed stacks.
Most popular would be [FlameGraph](https://github.com/brendangregg/FlameGraph) which contains a script called `flamegraph.pl`:
```sh
cat result.collapsed | /path/to/FlameGraph/flamegraph.pl --color=mem --title="Allocation Flame Graph" --width 2400 > result.svg
```
Another interesting tool is [speedscope](https://www.speedscope.app/) that allows you to analyze collected stacks in a more interactive way.
## Controlling allocation profiler during runtime
If ClickHouse/Keeper were started with enabled profiler, they support additional commands for disabling/enabling allocation profiling during runtime.
Using those commands, it's easier to profile only specific intervals.
Disable profiler:
<Tabs groupId="binary">
<TabItem value="clickhouse" label="ClickHouse">
SYSTEM JEMALLOC DISABLE PROFILE
</TabItem>
<TabItem value="keeper" label="Keeper">
echo jmdp | nc localhost 9181
</TabItem>
</Tabs>
Enable profiler:
<Tabs groupId="binary">
<TabItem value="clickhouse" label="ClickHouse">
SYSTEM JEMALLOC ENABLE PROFILE
</TabItem>
<TabItem value="keeper" label="Keeper">
echo jmep | nc localhost 9181
</TabItem>
</Tabs>
It's also possible to control the initial state of the profiler by setting `prof_active` option which is enabled by default.
For example, if we don't want to sample allocations during startup but only after we enable the profiler, we can start ClickHouse/Keeper with following environment variable:
```sh
MALLOC_CONF=background_thread:true,prof:true,prof_active:false
```
and enable profiler at a later point.
## Additional options for profiler
`jemalloc` has many different options available related to profiler which can be controlled by modifying `MALLOC_CONF` environment variable.
For example, interval between allocation samples can be controlled with `lg_prof_sample`.
If you want to dump heap profile every N bytes you can enable it using `lg_prof_interval`.
We recommend to check `jemalloc`s [reference page](https://jemalloc.net/jemalloc.3.html) for such options.
## Other resources
ClickHouse/Keeper expose `jemalloc` related metrics in many different ways.
:::warning Warning
It's important to be aware that none of these metrics are synchronized with each other and values may drift.
:::
### System table `asynchronous_metrics`
```sql
SELECT *
FROM system.asynchronous_metrics
WHERE metric ILIKE '%jemalloc%'
FORMAT Vertical
```
[Reference](/en/operations/system-tables/asynchronous_metrics)
### System table `jemalloc_bins`
Contains information about memory allocations done via jemalloc allocator in different size classes (bins) aggregated from all arenas.
[Reference](/en/operations/system-tables/jemalloc_bins)
### Prometheus
All `jemalloc` related metrics from `asynchronous_metrics` are also exposed using Prometheus endpoint in both ClickHouse and Keeper.
[Reference](/en/operations/server-configuration-parameters/settings#prometheus)
### `jmst` 4LW command in Keeper
Keeper supports `jmst` 4LW command which returns [basic allocator statistics](https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Basic-Allocator-Statistics).
Example:
```sh
echo jmst | nc localhost 9181
```

8
docs/en/operations/settings/settings-formats.md
View File

@ -1597,7 +1597,13 @@ Result:
Use ANSI escape sequences to paint colors in Pretty formats.
Enabled by default.
possible values:
- `0` — Disabled. Pretty formats do not use ANSI escape sequences.
- `1` — Enabled. Pretty formats will use ANSI escape sequences except for `NoEscapes` formats.
- `auto` - Enabled if `stdout` is a terminal except for `NoEscapes` formats.
Default value is `auto`.
### output_format_pretty_grid_charset {#output_format_pretty_grid_charset}

11
docs/ru/operations/settings/settings.md
View File

@ -2796,6 +2796,17 @@ SELECT TOP 3 name, value FROM system.settings;
3. │ max_block_size │ 65505 │
└─────────────────────────┴─────────┘
```
### output_format_pretty_color {#output_format_pretty_color}
Включает/выключает управляющие последовательности ANSI в форматах Pretty.
Возможные значения:
- `0` — выключена. Не исползует ANSI последовательности в форматах Pretty.
- `1` — включена. Исползует ANSI последовательности с исключением форматов `NoEscapes`.
- `auto` - включена если `stdout` является терминалом с исключением форматов `NoEscapes`.
Значение по умолчанию: `auto`
## system_events_show_zero_values {#system_events_show_zero_values}

2
src/Common/AsyncLoader.cpp
View File

@ -36,7 +36,7 @@ static constexpr size_t PRINT_MESSAGE_EACH_N_SECONDS = 5;
void logAboutProgress(Poco::Logger * log, size_t processed, size_t total, AtomicStopwatch & watch)
{
if (processed % PRINT_MESSAGE_EACH_N_OBJECTS == 0 || watch.compareAndRestart(PRINT_MESSAGE_EACH_N_SECONDS))
if (total && (processed % PRINT_MESSAGE_EACH_N_OBJECTS == 0 || watch.compareAndRestart(PRINT_MESSAGE_EACH_N_SECONDS)))
{
LOG_INFO(log, "Processed: {}%", processed * 100.0 / total);
watch.restart();

127
src/Common/StackTrace.cpp
View File

@ -32,6 +32,10 @@ std::atomic<bool> show_addresses = true;
bool shouldShowAddress(const void * addr)
{
/// Likely inline frame
if (!addr)
return false;
/// If the address is less than 4096, most likely it is a nullptr dereference with offset,
/// and showing this offset is secure nevertheless.
/// NOTE: 4096 is the page size on x86 and it can be different on other systems,
@ -203,20 +207,24 @@ static void * getCallerAddress(const ucontext_t & context)
#endif
}
// FIXME: looks like this is used only for Sentry but duplicates the whole algo, maybe replace?
void StackTrace::symbolize(
const StackTrace::FramePointers & frame_pointers, [[maybe_unused]] size_t offset, size_t size, StackTrace::Frames & frames)
void StackTrace::forEachFrame(
const StackTrace::FramePointers & frame_pointers,
size_t offset,
size_t size,
std::function<void(const Frame &)> callback,
bool fatal)
{
#if defined(__ELF__) && !defined(OS_FREEBSD)
const DB::SymbolIndex & symbol_index = DB::SymbolIndex::instance();
std::unordered_map<std::string, DB::Dwarf> dwarfs;
for (size_t i = 0; i < offset; ++i)
frames[i].virtual_addr = frame_pointers[i];
using enum DB::Dwarf::LocationInfoMode;
const auto mode = fatal ? FULL_WITH_INLINE : FAST;
for (size_t i = offset; i < size; ++i)
{
StackTrace::Frame & current_frame = frames[i];
StackTrace::Frame current_frame;
std::vector<DB::Dwarf::SymbolizedFrame> inline_frames;
current_frame.virtual_addr = frame_pointers[i];
const auto * object = symbol_index.findObject(current_frame.virtual_addr);
uintptr_t virtual_offset = object ? uintptr_t(object->address_begin) : 0;
@ -230,26 +238,41 @@ void StackTrace::symbolize(
auto dwarf_it = dwarfs.try_emplace(object->name, object->elf).first;
DB::Dwarf::LocationInfo location;
std::vector<DB::Dwarf::SymbolizedFrame> inline_frames;
if (dwarf_it->second.findAddress(
uintptr_t(current_frame.physical_addr), location, DB::Dwarf::LocationInfoMode::FAST, inline_frames))
uintptr_t(current_frame.physical_addr), location, mode, inline_frames))
{
current_frame.file = location.file.toString();
current_frame.line = location.line;
}
}
}
else
current_frame.object = "?";
if (const auto * symbol = symbol_index.findSymbol(current_frame.virtual_addr))
current_frame.symbol = demangle(symbol->name);
else
current_frame.symbol = "?";
for (const auto & frame : inline_frames)
{
StackTrace::Frame current_inline_frame;
const String file_for_inline_frame = frame.location.file.toString();
current_inline_frame.file = "inlined from " + file_for_inline_frame;
current_inline_frame.line = frame.location.line;
current_inline_frame.symbol = frame.name;
callback(current_inline_frame);
}
callback(current_frame);
}
#else
for (size_t i = 0; i < size; ++i)
frames[i].virtual_addr = frame_pointers[i];
UNUSED(fatal);
for (size_t i = offset; i < size; ++i)
{
StackTrace::Frame current_frame;
current_frame.virtual_addr = frame_pointers[i];
callback(current_frame);
}
#endif
}
@ -349,72 +372,52 @@ toStringEveryLineImpl([[maybe_unused]] bool fatal, const StackTraceRefTriple & s
if (stack_trace.size == 0)
return callback("<Empty trace>");
size_t frame_index = stack_trace.offset;
#if defined(__ELF__) && !defined(OS_FREEBSD)
using enum DB::Dwarf::LocationInfoMode;
const auto mode = fatal ? FULL_WITH_INLINE : FAST;
const DB::SymbolIndex & symbol_index = DB::SymbolIndex::instance();
std::unordered_map<String, DB::Dwarf> dwarfs;
for (size_t i = stack_trace.offset; i < stack_trace.size; ++i)
size_t inline_frame_index = 0;
auto callback_wrapper = [&](const StackTrace::Frame & frame)
{
std::vector<DB::Dwarf::SymbolizedFrame> inline_frames;
const void * virtual_addr = stack_trace.pointers[i];
const auto * object = symbol_index.findObject(virtual_addr);
uintptr_t virtual_offset = object ? uintptr_t(object->address_begin) : 0;
const void * physical_addr = reinterpret_cast<const void *>(uintptr_t(virtual_addr) - virtual_offset);
DB::WriteBufferFromOwnString out;
out << i << ". ";
String file;
if (std::error_code ec; object && std::filesystem::exists(object->name, ec) && !ec)
/// Inline frame
if (!frame.virtual_addr)
{
auto dwarf_it = dwarfs.try_emplace(object->name, object->elf).first;
DB::Dwarf::LocationInfo location;
if (dwarf_it->second.findAddress(uintptr_t(physical_addr), location, mode, inline_frames))
{
file = location.file.toString();
out << file << ":" << location.line << ": ";
}
out << frame_index << "." << inline_frame_index++ << ". ";
}
else
{
out << frame_index++ << ". ";
inline_frame_index = 0;
}
if (const auto * const symbol = symbol_index.findSymbol(virtual_addr))
out << demangleAndCollapseNames(file, symbol->name);
if (frame.file.has_value() && frame.line.has_value())
out << *frame.file << ':' << *frame.line << ": ";
if (frame.symbol.has_value() && frame.file.has_value())
out << demangleAndCollapseNames(*frame.file, frame.symbol->data());
else
out << "?";
if (shouldShowAddress(physical_addr))
if (shouldShowAddress(frame.physical_addr))
{
out << " @ ";
DB::writePointerHex(physical_addr, out);
DB::writePointerHex(frame.physical_addr, out);
}
out << " in " << (object ? object->name : "?");
for (size_t j = 0; j < inline_frames.size(); ++j)
{
const auto & frame = inline_frames[j];
const String file_for_inline_frame = frame.location.file.toString();
callback(fmt::format(
"{}.{}. inlined from {}:{}: {}",
i,
j + 1,
file_for_inline_frame,
frame.location.line,
demangleAndCollapseNames(file_for_inline_frame, frame.name)));
}
if (frame.object.has_value())
out << " in " << *frame.object;
callback(out.str());
}
};
#else
for (size_t i = stack_trace.offset; i < stack_trace.size; ++i)
if (const void * const addr = stack_trace.pointers[i]; shouldShowAddress(addr))
callback(fmt::format("{}. {}", i, addr));
auto callback_wrapper = [&](const StackTrace::Frame & frame)
{
if (frame.virtual_addr && shouldShowAddress(frame.virtual_addr))
callback(fmt::format("{}. {}", frame_index++, frame.virtual_addr));
};
#endif
StackTrace::forEachFrame(stack_trace.pointers, stack_trace.offset, stack_trace.size, callback_wrapper, fatal);
}
void StackTrace::toStringEveryLine(std::function<void(std::string_view)> callback) const

9
src/Common/StackTrace.h
View File

@ -62,7 +62,14 @@ public:
static std::string toString(void ** frame_pointers, size_t offset, size_t size);
static void dropCache();
static void symbolize(const FramePointers & frame_pointers, size_t offset, size_t size, StackTrace::Frames & frames);
/// @param fatal - if true, will process inline frames (slower)
static void forEachFrame(
const FramePointers & frame_pointers,
size_t offset,
size_t size,
std::function<void(const Frame &)> callback,
bool fatal);
void toStringEveryLine(std::function<void(std::string_view)> callback) const;
static void toStringEveryLine(const FramePointers & frame_pointers, std::function<void(std::string_view)> callback);

4
src/Core/Settings.h
View File

@ -810,7 +810,7 @@ class IColumn;
M(Bool, parallelize_output_from_storages, true, "Parallelize output for reading step from storage. It allows parallelizing query processing right after reading from storage if possible", 0) \
M(String, insert_deduplication_token, "", "If not empty, used for duplicate detection instead of data digest", 0) \
M(Bool, count_distinct_optimization, false, "Rewrite count distinct to subquery of group by", 0) \
M(Bool, throw_if_no_data_to_insert, true, "Enables or disables empty INSERTs, enabled by default", 0) \
M(Bool, throw_if_no_data_to_insert, true, "Allows or forbids empty INSERTs, enabled by default (throws an error on an empty insert)", 0) \
M(Bool, compatibility_ignore_auto_increment_in_create_table, false, "Ignore AUTO_INCREMENT keyword in column declaration if true, otherwise return error. It simplifies migration from MySQL", 0) \
M(Bool, multiple_joins_try_to_keep_original_names, false, "Do not add aliases to top level expression list on multiple joins rewrite", 0) \
M(Bool, optimize_sorting_by_input_stream_properties, true, "Optimize sorting by sorting properties of input stream", 0) \
@ -1048,7 +1048,7 @@ class IColumn;
M(UInt64, output_format_pretty_max_rows, 10000, "Rows limit for Pretty formats.", 0) \
M(UInt64, output_format_pretty_max_column_pad_width, 250, "Maximum width to pad all values in a column in Pretty formats.", 0) \
M(UInt64, output_format_pretty_max_value_width, 10000, "Maximum width of value to display in Pretty formats. If greater - it will be cut.", 0) \
M(Bool, output_format_pretty_color, true, "Use ANSI escape sequences to paint colors in Pretty formats", 0) \
M(UInt64Auto, output_format_pretty_color, "auto", "Use ANSI escape sequences in Pretty formats. 0 - disabled, 1 - enabled, 'auto' - enabled if a terminal.", 0) \
M(String, output_format_pretty_grid_charset, "UTF-8", "Charset for printing grid borders. Available charsets: ASCII, UTF-8 (default one).", 0) \
M(UInt64, output_format_parquet_row_group_size, 1000000, "Target row group size in rows.", 0) \
M(UInt64, output_format_parquet_row_group_size_bytes, 512 * 1024 * 1024, "Target row group size in bytes, before compression.", 0) \

3
src/Core/SettingsChangesHistory.h
View File

@ -87,7 +87,8 @@ static std::map<ClickHouseVersion, SettingsChangesHistory::SettingsChanges> sett
{"23.12", {{"allow_suspicious_ttl_expressions", true, false, "It is a new setting, and in previous versions the behavior was equivalent to allowing."},
{"input_format_parquet_allow_missing_columns", false, true, "Allow missing columns in Parquet files by default"},
{"input_format_orc_allow_missing_columns", false, true, "Allow missing columns in ORC files by default"},
{"input_format_arrow_allow_missing_columns", false, true, "Allow missing columns in Arrow files by default"}}},
{"input_format_arrow_allow_missing_columns", false, true, "Allow missing columns in Arrow files by default"},
{"output_format_pretty_color", true, "auto", "Setting is changed to allow also for auto value, disabling ANSI escapes if output is not a tty"}}},
{"23.9", {{"optimize_group_by_constant_keys", false, true, "Optimize group by constant keys by default"},
{"input_format_json_try_infer_named_tuples_from_objects", false, true, "Try to infer named Tuples from JSON objects by default"},
{"input_format_json_read_numbers_as_strings", false, true, "Allow to read numbers as strings in JSON formats by default"},

8
src/Daemon/SentryWriter.cpp
View File

@ -169,11 +169,9 @@ void SentryWriter::onFault(int sig, const std::string & error_message, const Sta
};
StackTrace::Frames frames;
StackTrace::symbolize(stack_trace.getFramePointers(), offset, stack_size, frames);
for (ssize_t i = stack_size - 1; i >= offset; --i)
auto sentry_add_stack_trace = [&](const StackTrace::Frame & current_frame)
{
const StackTrace::Frame & current_frame = frames[i];
sentry_value_t sentry_frame = sentry_value_new_object();
UInt64 frame_ptr = reinterpret_cast<UInt64>(current_frame.virtual_addr);
@ -190,7 +188,9 @@ void SentryWriter::onFault(int sig, const std::string & error_message, const Sta
sentry_value_set_by_key(sentry_frame, "lineno", sentry_value_new_int32(static_cast<int32_t>(current_frame.line.value())));
sentry_value_append(sentry_frames, sentry_frame);
}
};
StackTrace::forEachFrame(stack_trace.getFramePointers(), offset, stack_size, sentry_add_stack_trace, /* fatal= */ true);
}
/// Prepare data for https://develop.sentry.dev/sdk/event-payloads/threads/

6
src/Disks/IO/createReadBufferFromFileBase.cpp
View File

@ -36,8 +36,7 @@ std::unique_ptr<ReadBufferFromFileBase> createReadBufferFromFileBase(
std::optional<size_t> file_size,
int flags,
char * existing_memory,
size_t alignment,
bool use_external_buffer)
size_t alignment)
{
if (file_size.has_value() && !*file_size)
return std::make_unique<ReadBufferFromEmptyFile>();
@ -149,8 +148,7 @@ std::unique_ptr<ReadBufferFromFileBase> createReadBufferFromFileBase(
existing_memory,
buffer_alignment,
file_size,
settings.local_throttler,
use_external_buffer);
settings.local_throttler);
}
else
throw Exception(ErrorCodes::LOGICAL_ERROR, "Unknown read method");

3
src/Disks/IO/createReadBufferFromFileBase.h
View File

@ -21,6 +21,5 @@ std::unique_ptr<ReadBufferFromFileBase> createReadBufferFromFileBase(
std::optional<size_t> file_size = {},
int flags_ = -1,
char * existing_memory = nullptr,
size_t alignment = 0,
bool use_external_buffer = false);
size_t alignment = 0);
}

3
src/Formats/FormatFactory.cpp
View File

@ -5,6 +5,7 @@
#include <Interpreters/Context.h>
#include <Interpreters/ProcessList.h>
#include <IO/SharedThreadPools.h>
#include <IO/WriteHelpers.h>
#include <Processors/Formats/IRowInputFormat.h>
#include <Processors/Formats/IRowOutputFormat.h>
#include <Processors/Formats/Impl/MySQLOutputFormat.h>
@ -451,6 +452,7 @@ OutputFormatPtr FormatFactory::getOutputFormatParallelIfPossible(
throw Exception(ErrorCodes::FORMAT_IS_NOT_SUITABLE_FOR_OUTPUT, "Format {} is not suitable for output", name);
auto format_settings = _format_settings ? *_format_settings : getFormatSettings(context);
format_settings.is_writing_to_terminal = isWritingToTerminal(buf);
const Settings & settings = context->getSettingsRef();
@ -492,6 +494,7 @@ OutputFormatPtr FormatFactory::getOutputFormat(
auto format_settings = _format_settings ? *_format_settings : getFormatSettings(context);
format_settings.max_threads = context->getSettingsRef().max_threads;
format_settings.is_writing_to_terminal = format_settings.is_writing_to_terminal = isWritingToTerminal(buf);
/** TODO: Materialization is needed, because formats can use the functions `IDataType`,
* which only work with full columns.

5
src/Formats/FormatSettings.h
View File

@ -4,7 +4,7 @@
#include <Core/Defines.h>
#include <base/types.h>
#include <base/unit.h>
#include <Core/SettingsFields.h>
namespace DB
{
@ -34,6 +34,7 @@ struct FormatSettings
bool null_as_default = true;
bool decimal_trailing_zeros = false;
bool defaults_for_omitted_fields = true;
bool is_writing_to_terminal = false;
bool seekable_read = true;
UInt64 max_rows_to_read_for_schema_inference = 25000;
@ -272,7 +273,7 @@ struct FormatSettings
UInt64 max_rows = 10000;
UInt64 max_column_pad_width = 250;
UInt64 max_value_width = 10000;
bool color = true;
SettingFieldUInt64Auto color{"auto"};
bool output_format_pretty_row_numbers = false;

6
src/IO/AsynchronousReadBufferFromFile.h
View File

@ -67,10 +67,8 @@ public:
char * existing_memory = nullptr,
size_t alignment = 0,
std::optional<size_t> file_size_ = std::nullopt,
ThrottlerPtr throttler_ = {},
bool use_external_buffer_ = false)
: AsynchronousReadBufferFromFileDescriptor(
reader_, priority_, -1, buf_size, existing_memory, alignment, file_size_, throttler_, use_external_buffer_)
ThrottlerPtr throttler_ = {})
: AsynchronousReadBufferFromFileDescriptor(reader_, priority_, -1, buf_size, existing_memory, alignment, file_size_, throttler_)
, file_name(file_name_)
{
file = OpenedFileCache::instance().get(file_name, flags);

17
src/IO/AsynchronousReadBufferFromFileDescriptor.cpp
View File

@ -97,11 +97,7 @@ bool AsynchronousReadBufferFromFileDescriptor::nextImpl()
/// No pending request. Do synchronous read.
ProfileEventTimeIncrement<Microseconds> watch(ProfileEvents::SynchronousReadWaitMicroseconds);
if (!use_external_buffer)
result = asyncReadInto(memory.data(), memory.size(), DEFAULT_PREFETCH_PRIORITY).get();
else
/// External buffer will be substituted in place of internal_buffer (see CachedOnDiskReadBufferFromFile)
result = asyncReadInto(internal_buffer.begin(), internal_buffer.size(), DEFAULT_PREFETCH_PRIORITY).get();
result = asyncReadInto(memory.data(), memory.size(), DEFAULT_PREFETCH_PRIORITY).get();
}
chassert(result.size >= result.offset);
@ -114,9 +110,8 @@ bool AsynchronousReadBufferFromFileDescriptor::nextImpl()
if (bytes_read)
{
/// Adjust the working buffer so that it ignores `offset` bytes.
if (!use_external_buffer)
internal_buffer = Buffer(memory.data(), memory.data() + memory.size());
working_buffer = Buffer(internal_buffer.begin() + result.offset, internal_buffer.begin() + result.size);
internal_buffer = Buffer(memory.data(), memory.data() + memory.size());
working_buffer = Buffer(memory.data() + result.offset, memory.data() + result.size);
pos = working_buffer.begin();
}
@ -142,15 +137,13 @@ AsynchronousReadBufferFromFileDescriptor::AsynchronousReadBufferFromFileDescript
char * existing_memory,
size_t alignment,
std::optional<size_t> file_size_,
ThrottlerPtr throttler_,
bool use_external_buffer_)
ThrottlerPtr throttler_)
: ReadBufferFromFileBase(buf_size, existing_memory, alignment, file_size_)
, reader(reader_)
, base_priority(priority_)
, required_alignment(alignment)
, fd(fd_)
, throttler(throttler_)
, use_external_buffer(use_external_buffer_)
{
if (required_alignment > buf_size)
throw Exception(
@ -228,7 +221,7 @@ off_t AsynchronousReadBufferFromFileDescriptor::seek(off_t offset, int whence)
file_offset_of_buffer_end = seek_pos;
bytes_to_ignore = new_pos - seek_pos;
if (bytes_to_ignore >= internal_buffer.size() && !(use_external_buffer && internal_buffer.empty()))
if (bytes_to_ignore >= internal_buffer.size())
throw Exception(ErrorCodes::LOGICAL_ERROR,
"Logical error in AsynchronousReadBufferFromFileDescriptor, bytes_to_ignore ({}"
") >= internal_buffer.size() ({})", bytes_to_ignore, internal_buffer.size());

4
src/IO/AsynchronousReadBufferFromFileDescriptor.h
View File

@ -29,7 +29,6 @@ protected:
size_t bytes_to_ignore = 0; /// How many bytes should we ignore upon a new read request.
int fd;
ThrottlerPtr throttler;
bool use_external_buffer;
bool nextImpl() override;
@ -47,8 +46,7 @@ public:
char * existing_memory = nullptr,
size_t alignment = 0,
std::optional<size_t> file_size_ = std::nullopt,
ThrottlerPtr throttler_ = {},
bool use_external_buffer_ = false);
ThrottlerPtr throttler_ = {});
~AsynchronousReadBufferFromFileDescriptor() override;

8
src/IO/WriteHelpers.h
View File

@ -29,6 +29,7 @@
#include <Common/Exception.h>
#include <Common/StringUtils/StringUtils.h>
#include <Common/NaNUtils.h>
#include <Common/typeid_cast.h>
#include <IO/CompressionMethod.h>
#include <IO/WriteBuffer.h>
@ -36,6 +37,7 @@
#include <IO/VarInt.h>
#include <IO/DoubleConverter.h>
#include <IO/WriteBufferFromString.h>
#include <IO/WriteBufferFromFileDescriptor.h>
#ifdef __clang__
#pragma clang diagnostic push
@ -1405,6 +1407,12 @@ void writePointerHex(const void * ptr, WriteBuffer & buf);
String fourSpaceIndent(size_t indent);
bool inline isWritingToTerminal(const WriteBuffer & buf)
{
const auto * write_buffer_to_descriptor = typeid_cast<const WriteBufferFromFileDescriptor *>(&buf);
return write_buffer_to_descriptor && write_buffer_to_descriptor->getFD() == STDOUT_FILENO && isatty(STDOUT_FILENO);
}
}
template<>

4
src/Interpreters/Aggregator.cpp
View File

@ -1797,8 +1797,8 @@ void Aggregator::writeToTemporaryFile(AggregatedDataVariants & data_variants, si
rows,
ReadableSize(uncompressed_size),
ReadableSize(compressed_size),
static_cast<double>(uncompressed_size) / rows,
static_cast<double>(compressed_size) / rows,
rows ? static_cast<double>(uncompressed_size) / rows : 0.0,
rows ? static_cast<double>(compressed_size) / rows : 0.0,
static_cast<double>(uncompressed_size) / compressed_size,
static_cast<double>(rows) / elapsed_seconds,
ReadableSize(static_cast<double>(uncompressed_size) / elapsed_seconds),

1
src/Interpreters/Cache/SLRUFileCachePriority.cpp
View File

@ -264,7 +264,6 @@ SLRUFileCachePriority::SLRUIterator::SLRUIterator(
SLRUFileCachePriority::EntryPtr SLRUFileCachePriority::SLRUIterator::getEntry() const
{
chassert(entry == lru_iterator.getEntry());
return entry;
}

10
src/Processors/Formats/Impl/PrettyBlockOutputFormat.cpp
View File

@ -12,8 +12,8 @@ namespace DB
{
PrettyBlockOutputFormat::PrettyBlockOutputFormat(
WriteBuffer & out_, const Block & header_, const FormatSettings & format_settings_, bool mono_block_)
: IOutputFormat(header_, out_), format_settings(format_settings_), serializations(header_.getSerializations()), mono_block(mono_block_)
WriteBuffer & out_, const Block & header_, const FormatSettings & format_settings_, bool mono_block_, bool color_)
: IOutputFormat(header_, out_), format_settings(format_settings_), serializations(header_.getSerializations()), color(color_), mono_block(mono_block_)
{
}
@ -237,7 +237,7 @@ void PrettyBlockOutputFormat::writeChunk(const Chunk & chunk, PortKind port_kind
const auto & col = header.getByPosition(i);
if (format_settings.pretty.color)
if (color)
writeCString("\033[1m", out);
if (col.type->shouldAlignRightInPrettyFormats())
@ -255,7 +255,7 @@ void PrettyBlockOutputFormat::writeChunk(const Chunk & chunk, PortKind port_kind
writeChar(' ', out);
}
if (format_settings.pretty.color)
if (color)
writeCString("\033[0m", out);
}
writeCString(" ", out);
@ -335,7 +335,7 @@ void PrettyBlockOutputFormat::writeValueWithPadding(
reinterpret_cast<const UInt8 *>(serialized_value.data()), serialized_value.size(), 0, 1 + format_settings.pretty.max_value_width));
const char * ellipsis = format_settings.pretty.charset == FormatSettings::Pretty::Charset::UTF8 ? "⋯" : "~";
if (format_settings.pretty.color)
if (color)
{
serialized_value += "\033[31;1m";
serialized_value += ellipsis;

16
src/Processors/Formats/Impl/PrettyBlockOutputFormat.h
View File

@ -5,7 +5,6 @@
#include <Formats/FormatSettings.h>
#include <Formats/FormatFactory.h>
namespace DB
{
@ -19,10 +18,8 @@ class PrettyBlockOutputFormat : public IOutputFormat
{
public:
/// no_escapes - do not use ANSI escape sequences - to display in the browser, not in the console.
PrettyBlockOutputFormat(WriteBuffer & out_, const Block & header_, const FormatSettings & format_settings_, bool mono_block_);
PrettyBlockOutputFormat(WriteBuffer & out_, const Block & header_, const FormatSettings & format_settings_, bool mono_block_, bool color_);
String getName() const override { return "PrettyBlockOutputFormat"; }
protected:
void consume(Chunk) override;
void consumeTotals(Chunk) override;
@ -57,6 +54,8 @@ protected:
total_rows = 0;
}
bool color;
private:
bool mono_block;
/// For mono_block == true only
@ -73,13 +72,8 @@ void registerPrettyFormatWithNoEscapesAndMonoBlock(FormatFactory & factory, cons
const Block & sample,
const FormatSettings & format_settings)
{
if (no_escapes)
{
FormatSettings changed_settings = format_settings;
changed_settings.pretty.color = false;
return std::make_shared<OutputFormat>(buf, sample, changed_settings, mono_block);
}
return std::make_shared<OutputFormat>(buf, sample, format_settings, mono_block);
bool color = !no_escapes && format_settings.pretty.color.valueOr(format_settings.is_writing_to_terminal);
return std::make_shared<OutputFormat>(buf, sample, format_settings, mono_block, color);
});
if (!mono_block)
factory.markOutputFormatSupportsParallelFormatting(name);

12
src/Processors/Formats/Impl/PrettyCompactBlockOutputFormat.cpp
View File

@ -48,8 +48,8 @@ GridSymbols ascii_grid_symbols {
}
PrettyCompactBlockOutputFormat::PrettyCompactBlockOutputFormat(WriteBuffer & out_, const Block & header, const FormatSettings & format_settings_, bool mono_block_)
: PrettyBlockOutputFormat(out_, header, format_settings_, mono_block_)
PrettyCompactBlockOutputFormat::PrettyCompactBlockOutputFormat(WriteBuffer & out_, const Block & header, const FormatSettings & format_settings_, bool mono_block_, bool color_)
: PrettyBlockOutputFormat(out_, header, format_settings_, mono_block_, color_)
{
}
@ -87,18 +87,18 @@ void PrettyCompactBlockOutputFormat::writeHeader(
for (size_t k = 0; k < max_widths[i] - name_widths[i]; ++k)
writeCString(grid_symbols.dash, out);
if (format_settings.pretty.color)
if (color)
writeCString("\033[1m", out);
writeString(col.name, out);
if (format_settings.pretty.color)
if (color)
writeCString("\033[0m", out);
}
else
{
if (format_settings.pretty.color)
if (color)
writeCString("\033[1m", out);
writeString(col.name, out);
if (format_settings.pretty.color)
if (color)
writeCString("\033[0m", out);
for (size_t k = 0; k < max_widths[i] - name_widths[i]; ++k)

2
src/Processors/Formats/Impl/PrettyCompactBlockOutputFormat.h
View File

@ -13,7 +13,7 @@ namespace DB
class PrettyCompactBlockOutputFormat : public PrettyBlockOutputFormat
{
public:
PrettyCompactBlockOutputFormat(WriteBuffer & out_, const Block & header, const FormatSettings & format_settings_, bool mono_block_);
PrettyCompactBlockOutputFormat(WriteBuffer & out_, const Block & header, const FormatSettings & format_settings_, bool mono_block_, bool color);
String getName() const override { return "PrettyCompactBlockOutputFormat"; }
private:

8
src/Processors/Formats/Impl/PrettySpaceBlockOutputFormat.cpp
View File

@ -48,18 +48,18 @@ void PrettySpaceBlockOutputFormat::writeChunk(const Chunk & chunk, PortKind port
for (ssize_t k = 0; k < std::max(0z, static_cast<ssize_t>(max_widths[i] - name_widths[i])); ++k)
writeChar(' ', out);
if (format_settings.pretty.color)
if (color)
writeCString("\033[1m", out);
writeString(col.name, out);
if (format_settings.pretty.color)
if (color)
writeCString("\033[0m", out);
}
else
{
if (format_settings.pretty.color)
if (color)
writeCString("\033[1m", out);
writeString(col.name, out);
if (format_settings.pretty.color)
if (color)
writeCString("\033[0m", out);
for (ssize_t k = 0; k < std::max(0z, static_cast<ssize_t>(max_widths[i] - name_widths[i])); ++k)

4
src/Processors/Formats/Impl/PrettySpaceBlockOutputFormat.h
View File

@ -11,8 +11,8 @@ namespace DB
class PrettySpaceBlockOutputFormat : public PrettyBlockOutputFormat
{
public:
PrettySpaceBlockOutputFormat(WriteBuffer & out_, const Block & header, const FormatSettings & format_settings_, bool mono_block_)
: PrettyBlockOutputFormat(out_, header, format_settings_, mono_block_) {}
PrettySpaceBlockOutputFormat(WriteBuffer & out_, const Block & header, const FormatSettings & format_settings_, bool mono_block_, bool color_)
: PrettyBlockOutputFormat(out_, header, format_settings_, mono_block_, color_) {}
String getName() const override { return "PrettySpaceBlockOutputFormat"; }

20
src/Storages/MergeTree/RPNBuilder.cpp
View File

@ -14,7 +14,9 @@
#include <Columns/ColumnConst.h>
#include <Columns/ColumnSet.h>
#include <Functions/indexHint.h>
#include <Functions/IFunction.h>
#include <Functions/IFunctionAdaptors.h>
#include <Storages/KeyDescription.h>
@ -390,6 +392,15 @@ size_t RPNBuilderFunctionTreeNode::getArgumentsSize() const
}
else
{
// indexHint arguments are stored inside of `FunctionIndexHint` class,
// because they are used only for index analysis.
if (dag_node->function_base->getName() == "indexHint")
{
const auto * adaptor = typeid_cast<const FunctionToFunctionBaseAdaptor *>(dag_node->function_base.get());
const auto * index_hint = typeid_cast<const FunctionIndexHint *>(adaptor->getFunction().get());
return index_hint->getActions()->getOutputs().size();
}
return dag_node->children.size();
}
}
@ -409,6 +420,15 @@ RPNBuilderTreeNode RPNBuilderFunctionTreeNode::getArgumentAt(size_t index) const
}
else
{
// indexHint arguments are stored inside of `FunctionIndexHint` class,
// because they are used only for index analysis.
if (dag_node->function_base->getName() == "indexHint")
{
const auto * adaptor = typeid_cast<const FunctionToFunctionBaseAdaptor *>(dag_node->function_base.get());
const auto * index_hint = typeid_cast<const FunctionIndexHint *>(adaptor->getFunction().get());
return RPNBuilderTreeNode(index_hint->getActions()->getOutputs()[index], tree_context);
}
return RPNBuilderTreeNode(dag_node->children[index], tree_context);
}
}

2
tests/integration/test_storage_kafka/test.py
View File

@ -4450,7 +4450,7 @@ def test_block_based_formats_1(kafka_cluster):
kafka_group_name = '{topic}',
kafka_format = 'PrettySpace';
INSERT INTO test.kafka SELECT number * 10 as key, number * 100 as value FROM numbers(5) settings max_block_size=2, optimize_trivial_insert_select=0;
INSERT INTO test.kafka SELECT number * 10 as key, number * 100 as value FROM numbers(5) settings max_block_size=2, optimize_trivial_insert_select=0, output_format_pretty_color=1;
"""
)

2
tests/integration/test_storage_rabbitmq/test.py
View File

@ -3156,7 +3156,7 @@ def test_block_based_formats_1(rabbitmq_cluster):
)
instance.query(
"INSERT INTO test.rabbitmq SELECT number * 10 as key, number * 100 as value FROM numbers(5) settings max_block_size=2, optimize_trivial_insert_select=0;"
"INSERT INTO test.rabbitmq SELECT number * 10 as key, number * 100 as value FROM numbers(5) settings max_block_size=2, optimize_trivial_insert_select=0, output_format_pretty_color=1;"
)
insert_messages = []

1
tests/queries/0_stateless/00085_visible_width_of_tuple_of_dates.sql
View File

@ -1 +1,2 @@
SET output_format_pretty_color=1;
SELECT (toDate('2000-01-01'), toDate('2000-01-01')) AS x FORMAT PrettyCompact;

2
tests/queries/0_stateless/00089_group_by_arrays_of_fixed.sql
View File

@ -1 +1 @@
SELECT arr, count() AS c FROM (SELECT arrayMap(x -> x % 2, groupArray(number)) AS arr FROM (SELECT number FROM system.numbers LIMIT 10000) GROUP BY number % ((number * 0xABCDEF0123456789 % 1234) + 1)) GROUP BY arr ORDER BY c DESC, arr ASC;
SELECT arr, count() AS c FROM (SELECT arrayMap(x -> x % 2, arraySort(groupArray(number))) AS arr FROM (SELECT number FROM system.numbers LIMIT 10000) GROUP BY number % ((number * 0xABCDEF0123456789 % 1234) + 1)) GROUP BY arr ORDER BY c DESC, arr ASC;

1
tests/queries/0_stateless/00098_k_union_all.sql
View File

@ -1,3 +1,4 @@
SET output_format_pretty_color=1;
SELECT 1 FORMAT PrettySpace;
SELECT 1 UNION ALL SELECT 1 FORMAT PrettySpace;
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 FORMAT PrettySpace;

1
tests/queries/0_stateless/00298_enum_width_and_cast.sql
View File

@ -1,5 +1,6 @@
DROP TABLE IF EXISTS enum;
SET output_format_pretty_color=1;
CREATE TABLE enum (x Enum8('Hello' = -100, '\\' = 0, '\t\\t' = 111), y UInt8) ENGINE = TinyLog;
INSERT INTO enum (y) VALUES (0);
SELECT * FROM enum ORDER BY x, y FORMAT PrettyCompact;

8
tests/queries/0_stateless/00405_PrettyCompactMonoBlock.sh
View File

@ -5,10 +5,10 @@ CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
. "$CURDIR"/../shell_config.sh
echo 'one block'
${CLICKHOUSE_LOCAL} --query="SELECT * FROM numbers(2)" --format PrettyCompactMonoBlock
${CLICKHOUSE_LOCAL} --query="SELECT * FROM numbers(2) SETTINGS output_format_pretty_color=1" --format PrettyCompactMonoBlock
echo 'two blocks'
${CLICKHOUSE_LOCAL} --query="SELECT * FROM numbers(1) UNION ALL SELECT * FROM numbers(1)" --format PrettyCompactMonoBlock
${CLICKHOUSE_LOCAL} --query="SELECT * FROM numbers(1) UNION ALL SELECT * FROM numbers(1) SETTINGS output_format_pretty_color=1" --format PrettyCompactMonoBlock
echo 'extremes'
${CLICKHOUSE_LOCAL} --query="SELECT * FROM numbers(3)" --format PrettyCompactMonoBlock --extremes=1
${CLICKHOUSE_LOCAL} --query="SELECT * FROM numbers(3) SETTINGS output_format_pretty_color=1" --format PrettyCompactMonoBlock --extremes=1
echo 'totals'
${CLICKHOUSE_LOCAL} --query="SELECT sum(number) FROM numbers(3) GROUP BY number%2 WITH TOTALS ORDER BY number%2" --format PrettyCompactMonoBlock
${CLICKHOUSE_LOCAL} --query="SELECT sum(number) FROM numbers(3) GROUP BY number%2 WITH TOTALS ORDER BY number%2 SETTINGS output_format_pretty_color=1" --format PrettyCompactMonoBlock

363
tests/queries/0_stateless/00405_output_format_pretty_color.reference Normal file
View File

@ -0,0 +1,363 @@
0
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 1 │ 1 │ (1,'1') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 2 │ 2 │ (2,'2') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 5 │ 5 │ (5,'5') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 7 │ 7 │ (7,'7') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 8 │ 8 │ (8,'8') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
│ 1 │ 1 │ (1,'1') │ 1 │
│ 2 │ 2 │ (2,'2') │ 2 │
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 5 │ 5 │ (5,'5') │ 2 │
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
│ 7 │ 7 │ (7,'7') │ 1 │
│ 8 │ 8 │ (8,'8') │ 2 │
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
hello world tuple sometimes_nulls
0 0 (0,'0') ᴺᵁᴸᴸ
1 1 (1,'1') 1
2 2 (2,'2') 2
3 3 (3,'3') ᴺᵁᴸᴸ
4 4 (4,'4') 1
hello world tuple sometimes_nulls
5 5 (5,'5') 2
6 6 (6,'6') ᴺᵁᴸᴸ
7 7 (7,'7') 1
8 8 (8,'8') 2
9 9 (9,'9') ᴺᵁᴸᴸ
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
│ 1 │ 1 │ (1,'1') │ 1 │
│ 2 │ 2 │ (2,'2') │ 2 │
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
│ 4 │ 4 │ (4,'4') │ 1 │
│ 5 │ 5 │ (5,'5') │ 2 │
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
│ 7 │ 7 │ (7,'7') │ 1 │
│ 8 │ 8 │ (8,'8') │ 2 │
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 1 │ 1 │ (1,'1') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 2 │ 2 │ (2,'2') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 5 │ 5 │ (5,'5') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 7 │ 7 │ (7,'7') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 8 │ 8 │ (8,'8') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
│ 1 │ 1 │ (1,'1') │ 1 │
│ 2 │ 2 │ (2,'2') │ 2 │
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 5 │ 5 │ (5,'5') │ 2 │
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
│ 7 │ 7 │ (7,'7') │ 1 │
│ 8 │ 8 │ (8,'8') │ 2 │
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
hello world tuple sometimes_nulls
0 0 (0,'0') ᴺᵁᴸᴸ
1 1 (1,'1') 1
2 2 (2,'2') 2
3 3 (3,'3') ᴺᵁᴸᴸ
4 4 (4,'4') 1
hello world tuple sometimes_nulls
5 5 (5,'5') 2
6 6 (6,'6') ᴺᵁᴸᴸ
7 7 (7,'7') 1
8 8 (8,'8') 2
9 9 (9,'9') ᴺᵁᴸᴸ
1
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple  ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 1 │ 1 │ (1,'1') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 2 │ 2 │ (2,'2') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple  ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 5 │ 5 │ (5,'5') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 7 │ 7 │ (7,'7') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 8 │ 8 │ (8,'8') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
│ 1 │ 1 │ (1,'1') │ 1 │
│ 2 │ 2 │ (2,'2') │ 2 │
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 5 │ 5 │ (5,'5') │ 2 │
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
│ 7 │ 7 │ (7,'7') │ 1 │
│ 8 │ 8 │ (8,'8') │ 2 │
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
hello world tuple sometimes_nulls
0 0 (0,'0') ᴺᵁᴸᴸ
1 1 (1,'1') 1
2 2 (2,'2') 2
3 3 (3,'3') ᴺᵁᴸᴸ
4 4 (4,'4') 1
hello world tuple sometimes_nulls
5 5 (5,'5') 2
6 6 (6,'6') ᴺᵁᴸᴸ
7 7 (7,'7') 1
8 8 (8,'8') 2
9 9 (9,'9') ᴺᵁᴸᴸ
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
│ 1 │ 1 │ (1,'1') │ 1 │
│ 2 │ 2 │ (2,'2') │ 2 │
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
│ 4 │ 4 │ (4,'4') │ 1 │
│ 5 │ 5 │ (5,'5') │ 2 │
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
│ 7 │ 7 │ (7,'7') │ 1 │
│ 8 │ 8 │ (8,'8') │ 2 │
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 1 │ 1 │ (1,'1') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 2 │ 2 │ (2,'2') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 5 │ 5 │ (5,'5') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 7 │ 7 │ (7,'7') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 8 │ 8 │ (8,'8') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
│ 1 │ 1 │ (1,'1') │ 1 │
│ 2 │ 2 │ (2,'2') │ 2 │
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 5 │ 5 │ (5,'5') │ 2 │
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
│ 7 │ 7 │ (7,'7') │ 1 │
│ 8 │ 8 │ (8,'8') │ 2 │
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
hello world tuple sometimes_nulls
0 0 (0,'0') ᴺᵁᴸᴸ
1 1 (1,'1') 1
2 2 (2,'2') 2
3 3 (3,'3') ᴺᵁᴸᴸ
4 4 (4,'4') 1
hello world tuple sometimes_nulls
5 5 (5,'5') 2
6 6 (6,'6') ᴺᵁᴸᴸ
7 7 (7,'7') 1
8 8 (8,'8') 2
9 9 (9,'9') ᴺᵁᴸᴸ
auto
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 1 │ 1 │ (1,'1') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 2 │ 2 │ (2,'2') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 5 │ 5 │ (5,'5') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 7 │ 7 │ (7,'7') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 8 │ 8 │ (8,'8') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
│ 1 │ 1 │ (1,'1') │ 1 │
│ 2 │ 2 │ (2,'2') │ 2 │
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 5 │ 5 │ (5,'5') │ 2 │
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
│ 7 │ 7 │ (7,'7') │ 1 │
│ 8 │ 8 │ (8,'8') │ 2 │
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
hello world tuple sometimes_nulls
0 0 (0,'0') ᴺᵁᴸᴸ
1 1 (1,'1') 1
2 2 (2,'2') 2
3 3 (3,'3') ᴺᵁᴸᴸ
4 4 (4,'4') 1
hello world tuple sometimes_nulls
5 5 (5,'5') 2
6 6 (6,'6') ᴺᵁᴸᴸ
7 7 (7,'7') 1
8 8 (8,'8') 2
9 9 (9,'9') ᴺᵁᴸᴸ
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
│ 1 │ 1 │ (1,'1') │ 1 │
│ 2 │ 2 │ (2,'2') │ 2 │
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
│ 4 │ 4 │ (4,'4') │ 1 │
│ 5 │ 5 │ (5,'5') │ 2 │
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
│ 7 │ 7 │ (7,'7') │ 1 │
│ 8 │ 8 │ (8,'8') │ 2 │
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 1 │ 1 │ (1,'1') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 2 │ 2 │ (2,'2') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ hello ┃ world ┃ tuple ┃ sometimes_nulls ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 5 │ 5 │ (5,'5') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
├───────┼───────┼─────────┼─────────────────┤
│ 7 │ 7 │ (7,'7') │ 1 │
├───────┼───────┼─────────┼─────────────────┤
│ 8 │ 8 │ (8,'8') │ 2 │
├───────┼───────┼─────────┼─────────────────┤
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 0 │ 0 │ (0,'0') │ ᴺᵁᴸᴸ │
│ 1 │ 1 │ (1,'1') │ 1 │
│ 2 │ 2 │ (2,'2') │ 2 │
│ 3 │ 3 │ (3,'3') │ ᴺᵁᴸᴸ │
│ 4 │ 4 │ (4,'4') │ 1 │
└───────┴───────┴─────────┴─────────────────┘
┌─hello─┬─world─┬─tuple───┬─sometimes_nulls─┐
│ 5 │ 5 │ (5,'5') │ 2 │
│ 6 │ 6 │ (6,'6') │ ᴺᵁᴸᴸ │
│ 7 │ 7 │ (7,'7') │ 1 │
│ 8 │ 8 │ (8,'8') │ 2 │
│ 9 │ 9 │ (9,'9') │ ᴺᵁᴸᴸ │
└───────┴───────┴─────────┴─────────────────┘
hello world tuple sometimes_nulls
0 0 (0,'0') ᴺᵁᴸᴸ
1 1 (1,'1') 1
2 2 (2,'2') 2
3 3 (3,'3') ᴺᵁᴸᴸ
4 4 (4,'4') 1
hello world tuple sometimes_nulls
5 5 (5,'5') 2
6 6 (6,'6') ᴺᵁᴸᴸ
7 7 (7,'7') 1
8 8 (8,'8') 2
9 9 (9,'9') ᴺᵁᴸᴸ

32
tests/queries/0_stateless/00405_output_format_pretty_color.sql Normal file
View File

@ -0,0 +1,32 @@
SET output_format_pretty_color = 0;
SHOW SETTING output_format_pretty_color;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT Pretty;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyCompact;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettySpace;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyCompactMonoBlock;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyNoEscapes;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyCompactNoEscapes;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettySpaceNoEscapes;
SET output_format_pretty_color = 1;
SHOW SETTING output_format_pretty_color;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT Pretty;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyCompact;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettySpace;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyCompactMonoBlock;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyNoEscapes;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyCompactNoEscapes;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettySpaceNoEscapes;
SET output_format_pretty_color = 'auto';
SHOW SETTING output_format_pretty_color;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT Pretty;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyCompact;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettySpace;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyCompactMonoBlock;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyNoEscapes;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyCompactNoEscapes;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettySpaceNoEscapes;

2
tests/queries/0_stateless/00405_pretty_formats.sql
View File

@ -1,3 +1,5 @@
SET output_format_pretty_color = 1;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT Pretty;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettyCompact;
SELECT number AS hello, toString(number) AS world, (hello, world) AS tuple, nullIf(hello % 3, 0) AS sometimes_nulls FROM system.numbers LIMIT 10 SETTINGS max_block_size = 5 FORMAT PrettySpace;

1
tests/queries/0_stateless/00476_pretty_formats_and_widths.sql
View File

@ -1,3 +1,4 @@
SET output_format_pretty_color=1;
SELECT toUInt64(round(exp10(number))) AS x, toString(x) AS s FROM system.numbers LIMIT 10 FORMAT Pretty;
SELECT toUInt64(round(exp10(number))) AS x, toString(x) AS s FROM system.numbers LIMIT 10 FORMAT PrettyCompact;
SELECT toUInt64(round(exp10(number))) AS x, toString(x) AS s FROM system.numbers LIMIT 10 FORMAT PrettySpace;

1
tests/queries/0_stateless/00818_inner_join_bug_3567.sql
View File

@ -1,3 +1,4 @@
SET output_format_pretty_color = 1;
SET allow_experimental_analyzer = 1;
DROP TABLE IF EXISTS table1;

1
tests/queries/0_stateless/01018_ambiguous_column.sql
View File

@ -1,3 +1,4 @@
SET output_format_pretty_color=1;
SET allow_experimental_analyzer = 1;
select * from system.one cross join system.one;

4
tests/queries/0_stateless/01074_partial_revokes.sql
View File

@ -43,7 +43,7 @@ REVOKE SELECT ON db.* FROM test_user_01074;
GRANT SELECT ON db.table TO test_user_01074;
REVOKE SELECT(col1) ON db.table FROM test_user_01074;
SHOW GRANTS FOR test_user_01074;
SELECT * FROM system.grants WHERE user_name = 'test_user_01074' format Pretty;
SELECT * FROM system.grants WHERE user_name = 'test_user_01074' SETTINGS output_format_pretty_color=1 FORMAT Pretty;
SELECT '--cleanup';
REVOKE SELECT ON *.* FROM test_user_01074;
@ -73,7 +73,7 @@ SELECT '--grant option 1';
GRANT SELECT ON *.* TO test_user_01074 WITH GRANT OPTION;
REVOKE GRANT OPTION FOR SELECT(col1) ON db.table FROM test_user_01074;
SHOW GRANTS FOR test_user_01074;
SELECT * FROM system.grants WHERE user_name = 'test_user_01074' format Pretty;
SELECT * FROM system.grants WHERE user_name = 'test_user_01074' SETTINGS output_format_pretty_color=1 FORMAT Pretty;
SELECT '--cleanup';
REVOKE SELECT ON *.* FROM test_user_01074;

1
tests/queries/0_stateless/01293_pretty_max_value_width.sql
View File

@ -1,3 +1,4 @@
SET output_format_pretty_color=1;
SELECT 'привет' AS x, 'мир' AS y FORMAT Pretty;
SET output_format_pretty_max_value_width = 5;

2
tests/queries/0_stateless/01472_many_rows_in_totals.sql
View File

@ -1,3 +1,5 @@
set output_format_pretty_color=1;
-- Disable external aggregation because it may produce several blocks instead of one.
set max_bytes_before_external_group_by = 0;
set output_format_write_statistics = 0;

1
tests/queries/0_stateless/01509_output_format_pretty_row_numbers.sql
View File

@ -1,3 +1,4 @@
SET output_format_pretty_color=1;
SELECT * FROM numbers(10) FORMAT Pretty;
SELECT * FROM numbers(10) FORMAT PrettyCompact;
SELECT * FROM numbers(10) FORMAT PrettyCompactMonoBlock;

1
tests/queries/0_stateless/01670_neighbor_lc_bug.sql
View File

@ -40,6 +40,7 @@ FROM
ORDER BY val_string, rowNr
)
ORDER BY rowNr, val_string, str_m1, str_p1, val_low, low_m1, low_p1
SETTINGS output_format_pretty_color=1
format PrettyCompact;
drop table if exists neighbor_test;

1
tests/queries/0_stateless/01671_merge_join_and_constants.sql
View File

@ -1,3 +1,4 @@
SET output_format_pretty_color=1;
SET allow_experimental_analyzer = 1;
DROP TABLE IF EXISTS table1;

2
tests/queries/0_stateless/02152_bool_type.sql
View File

@ -1,3 +1,5 @@
SET output_format_pretty_color=1;
SELECT CAST('True', 'Bool');
SELECT CAST('TrUe', 'Bool');
SELECT CAST('true', 'Bool');

2
tests/queries/0_stateless/02375_pretty_formats.sql.j2
View File

@ -3,6 +3,6 @@
'PrettySpaceNoEscapesMonoBlock'] -%}
select '{{ format }}';
select number as x, number + 1 as y from numbers(4) settings max_block_size=2 format {{ format }};
select number as x, number + 1 as y from numbers(4) settings max_block_size=2, output_format_pretty_color=1 format {{ format }};
{% endfor -%}

4
tests/queries/0_stateless/02418_do_not_return_empty_blocks_from_ConvertingAggregatedToChunksTransform.sh
View File

@ -13,7 +13,7 @@ ${CLICKHOUSE_CURL} \
from numbers_mt(1e6)
where number = 42
group by number
settings max_threads = 10, max_bytes_before_external_group_by = 1, group_by_two_level_threshold = 1
settings max_threads = 10, max_bytes_before_external_group_by = 1, group_by_two_level_threshold = 1, output_format_pretty_color=1
format PrettyCompact"
${CLICKHOUSE_CURL} \
@ -24,5 +24,5 @@ ${CLICKHOUSE_CURL} \
from numbers_mt(1e6)
where number = 42
group by number
settings max_threads = 10, max_bytes_before_external_group_by = 0, group_by_two_level_threshold = 1
settings max_threads = 10, max_bytes_before_external_group_by = 0, group_by_two_level_threshold = 1, output_format_pretty_color=1
format PrettyCompact"

1
tests/queries/0_stateless/02421_formats_with_totals_and_extremes.sql.j2
View File

@ -1,5 +1,6 @@
-- Tags: no-fasttest
set output_format_pretty_color=1;
set output_format_write_statistics=0;
{% for format in ['CSV', 'TSV', 'XML', 'Vertical', 'Pretty', 'JSON', 'JSONCompact'] -%}

1
tests/queries/0_stateless/02798_explain_settings_not_applied_bug.sql
View File

@ -1,3 +1,4 @@
SET output_format_pretty_color=1;
SET read_in_order_two_level_merge_threshold=1000000;
DROP TABLE IF EXISTS t;

4
tests/queries/0_stateless/02896_union_distinct_http_format.sh
View File

@ -4,5 +4,5 @@ CURDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=../shell_config.sh
. "$CURDIR"/../shell_config.sh
curl -d@- -sS "${CLICKHOUSE_URL}" <<< 'SELECT 1 UNION DISTINCT SELECT 1 FORMAT PrettyCompactMonoBlock'
curl -d@- -sS "${CLICKHOUSE_URL}" <<< 'SELECT * FROM (SELECT 1 as a UNION DISTINCT SELECT 2 as a) ORDER BY a FORMAT PrettyCompactMonoBlock'
curl -d@- -sS "${CLICKHOUSE_URL}" <<< 'SELECT 1 UNION DISTINCT SELECT 1 SETTINGS output_format_pretty_color=1 FORMAT PrettyCompactMonoBlock'
curl -d@- -sS "${CLICKHOUSE_URL}" <<< 'SELECT * FROM (SELECT 1 as a UNION DISTINCT SELECT 2 as a) ORDER BY a SETTINGS output_format_pretty_color=1 FORMAT PrettyCompactMonoBlock'

0
tests/queries/0_stateless/02962_analyzer_resolve_group_by_on_shards.reference Normal file
View File

20
tests/queries/0_stateless/02962_analyzer_resolve_group_by_on_shards.sql Normal file
View File

@ -0,0 +1,20 @@
SELECT NULL AND (toDate(-2147483647, NULL) AND NULL)
FROM remote('127.0.0.{1,2}', view(
SELECT
NULL AND NULL,
NULL,
toDate(toDate('0.0001048577', toDate(NULL, 10 AND (toDate(257, 9223372036854775807, NULL) AND NULL AND NULL) AND NULL, 7, NULL), NULL, NULL) AND NULL AND -2147483648, NULL, NULL) AND NULL
FROM system.one
WHERE toDate(toDate(NULL, NULL, NULL), NULL)
GROUP BY
GROUPING SETS ((NULL))
));
SELECT NULL AND (toDate(-2147483647, NULL) AND NULL)
FROM remote('127.0.0.{1,2}', view(
SELECT NULL
FROM system.one
WHERE toDate(toDate(NULL, NULL, NULL), NULL)
GROUP BY
GROUPING SETS ((''))
));

0
tests/queries/0_stateless/02962_indexHint_rpn_construction.reference Normal file
View File

20
tests/queries/0_stateless/02962_indexHint_rpn_construction.sql Normal file
View File

@ -0,0 +1,20 @@
CREATE TABLE tab
(
`foo` Array(LowCardinality(String)),
INDEX idx foo TYPE bloom_filter GRANULARITY 1
)
ENGINE = MergeTree
PRIMARY KEY tuple();
INSERT INTO tab SELECT if(number % 2, ['value'], [])
FROM system.numbers
LIMIT 10000;
SELECT *
FROM tab
PREWHERE indexHint(indexHint(-1, 0.))
WHERE has(foo, 'b');
SELECT *
FROM tab
PREWHERE indexHint(0);

3
utils/check-style/aspell-ignore/en/aspell-dict.txt
View File

@ -282,6 +282,7 @@ FilesystemMainPathTotalINodes
FilesystemMainPathUsedBytes
FilesystemMainPathUsedINodes
FixedString
FlameGraph
Flink
ForEach
FreeBSD
@ -1130,6 +1131,7 @@ authenticators
autocompletion
autodetect
autodetected
autogen
autogenerate
autogenerated
autogeneration
@ -1718,6 +1720,7 @@ javaHashUTF
jbod
jdbc
jemalloc
jeprof
joinGet
json
jsonMergePatch