ClickHouse/docs/en/operations/system-tables/trace_log.md

slug
/en/operations/system-tables/trace_log

trace_log

Contains stack traces collected by the sampling query profiler.

ClickHouse creates this table when the trace_log server configuration section is set. Also see settings: query_profiler_real_time_period_ns, query_profiler_cpu_time_period_ns, memory_profiler_step, memory_profiler_sample_probability, trace_profile_events.

To analyze logs, use the addressToLine, addressToLineWithInlines, addressToSymbol and demangle introspection functions.

Columns:

• hostname (LowCardinality(String)) — Hostname of the server executing the query.

• event_date (Date) — Date of sampling moment.

• event_time (DateTime) — Timestamp of the sampling moment.

• event_time_microseconds (DateTime64) — Timestamp of the sampling moment with microseconds precision.

• timestamp_ns (UInt64) — Timestamp of the sampling moment in nanoseconds.

• revision (UInt32) — ClickHouse server build revision.

  When connecting to the server by clickhouse-client, you see the string similar to Connected to ClickHouse server version 19.18.1.. This field contains the revision, but not the version of a server.

• trace_type (Enum8) — Trace type:

  • Real represents collecting stack traces by wall-clock time.
  • CPU represents collecting stack traces by CPU time.
  • Memory represents collecting allocations and deallocations when memory allocation exceeds the subsequent watermark.
  • MemorySample represents collecting random allocations and deallocations.
  • MemoryPeak represents collecting updates of peak memory usage.
  • ProfileEvent represents collecting of increments of profile events.

• thread_id (UInt64) — Thread identifier.

• query_id (String) — Query identifier that can be used to get details about a query that was running from the query_log system table.

• trace (Array(UInt64)) — Stack trace at the moment of sampling. Each element is a virtual memory address inside ClickHouse server process.

• size (Int64) - For trace types Memory, MemorySample or MemoryPeak is the amount of memory allocated, for other trace types is 0.

• event (LowCardinality(String)) - For trace type ProfileEvent is the name of updated profile event, for other trace types is an empty string.

• increment (UInt64) - For trace type ProfileEvent is the amount of increment of profile event, for other trace types is 0.

Example

SELECT * FROM system.trace_log LIMIT 1 \G

Row 1:
──────
hostname:                clickhouse.eu-central1.internal
event_date:              2020-09-10
event_time:              2020-09-10 11:23:09
event_time_microseconds: 2020-09-10 11:23:09.872924
timestamp_ns:            1599762189872924510
revision:                54440
trace_type:              Memory
thread_id:               564963
query_id:
trace:                   [371912858,371912789,371798468,371799717,371801313,371790250,624462773,566365041,566440261,566445834,566460071,566459914,566459842,566459580,566459469,566459389,566459341,566455774,371993941,371988245,372158848,372187428,372187309,372187093,372185478,140222123165193,140222122205443]
size:                    5244400