ClickHouse/docs
Arthur Passos 24b8bbe9fa
Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition (#39507)
* temp commit

* temp commit

* draft impl for feedback

* fix weird style changes

* fix weird style changes

* fix weird style changes

* fix weird style changes

* fix weird style changes

* aa

* aa

* Add integ tests and remove partition key restriction

* fix small incosistency in partition id

* style fix

* style fix

* style fix

* use existing DataPartStorageBuilder instead of new one

* Refactor part clone to make it more readable and maintainable

* Add MergeTreeDataPartCloner docs

* define ErrorCodes::BAD_ARGUMENTS

* Rebase

* camel case methods

* address some comments

* yet another rebase?

* Move from integ tests to stateless tests

* address more comments

* add finalize on min_max_idx files

* Add sync option to DistinctPartitionExpCloner

* just a temp test

* revert temp change

* Use echoOn to distinguish test queries

* remove comment

* fix build issue during rebase

* atempt to fix build after rebase

* finally fix build

* clear minmaxidx hyperrectangle before loading it

* Fix error on min_max files deletion where it was being assumed that partition expression contained all columns

* get it to the state it was previously

* add missing include

* getting functional?

* refactoring and renaming

* some more refactoring

* extern bad arguments

* try to fix style

* improvements and docs

* remove duplicate includes

* fix crash

* make tests more stable by ordering

* rebase once again..

* fix

* make ci happy?

* fix rebase issues

* docs

* rebase, but prolly needs to be improved

* refactor out from nasty inheritance to static methods

* fix style

* work around optional

* refactor & integrate some changes

* update column_type

* add tests by dencrane

* set utc

* fix ref file

* fix tests

* use MergeTree instead of SummingMergeTree

* mark MergeTreeDataPart::getBlock as const

* address a few comments

* compute module function name size at compile time

* simplify branching in getPartitionAstFieldsCount

* remove column_indexes argument

* merge getBlock with buildBlock

* add some const specifiers

* small adjustments

* remove no longer needed isNull check

* use std::min and max to update global min max idx

* add some assertions

* forward declare some symbols

* fix grammar

* forward decl

* try to fix build..

* remove IFunction forward decl

* Revert "use std::min and max to update global min max idx"

This reverts commit b2fe79dda7.

* Revert "remove no longer needed isNull check"

This reverts commit 129db2610f.

* Revert "Revert "remove no longer needed isNull check""

This reverts commit 9416087dd8.

* Revert "Revert "use std::min and max to update global min max idx""

This reverts commit 20246d4416.

* remove some comments

* partial use of MonotonicityCheckMatcher

* ranges

* remove KeyDescriptionMonotonicityChecker

* remove duplication of applyfunction

* move functions to anonymous namespace

* move functions to cpp

* Relax partition compatibility requirements by accepting subset, add tests from partitioned to unpartitioned

* updte reference file

* Support for partition by a, b, c to partition by a, b

* refactoring part 1

* refactoring part 2, use hyperrectangle, still not complete

* refactoring part 3, build hyperrectangle with intersection of source & destination min max columns

* Support attaching to table with partition expression of multiple expressions

* add tests

* rename method

* remove some code duplication

* draft impl for replicatedmergetree, need to dive deeper

* ship ref file

* fix impl for replicatedmergetree..

* forbid attach empty partition replicatedmergetree

* Add replicated merge tree integration tests

* add test missing files

* fix black

* do not check for monotonicity of empty partition

* add empty tests & fix replicated

* remove no longer needed buildBlockWithMinMaxINdexes

* remove column logic in buildHyperrectangle

* simplify implementation by using existing methods

* further simplify implementation

* move all MergeTreeDataPartClone private methods to .cpp file

* decrease decomposition

* use different namespaces

* reduce code duplication

* fix style

* address a few comments

* add chassert to assert arguments size on MonotonicityCheckVisitor

* remove deleteMinMaxFiles method

* remove useless checks from sanitycheck

* add tests for attach partition (not id)

* Remove sanityCheckASTPartition and bring back conditional getPartitionIDFromQuery

* remove empty block comment

* small fixes

* fix formatting

* add missing include

* remove duplicate iuncludes

* trigger ci

* reduce some code duplication

* use updated partition id on replicatedmergetree

* fix build

* fix build

* small refactor

* do not use insert increment on fetch part

* remove duplicate includes

* add one more integ test

* black

* black

* rely on partition exp instead of partition id on replicated part fetch to decide if it is a different partition exp

* add one more integ test

* add order by clause

* fix black

---------

Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2024-01-22 21:00:04 +01:00
..
_description_templates add missing doc templates back 2023-07-03 14:15:20 -04:00
_includes Update comment in universal.sh 2023-11-16 10:59:17 +00:00
changelogs Merge pull request #58549 from ClickHouse/auto/v23.3.19.32-lts 2024-01-05 18:38:53 +01:00
en Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition (#39507) 2024-01-22 21:00:04 +01:00
ru Merge pull request #57599 from Khatskevich/khatskevich/fix_docs 2024-01-22 12:54:57 +01:00
zh Update docs/zh/faq/operations/delete-old-data.md 2024-01-22 00:29:06 +01:00
.gitignore Add a script to clone or update clickhouse-docs 2023-03-28 19:52:51 +02:00
clean
get-clickhouse-docs.sh Update docs/get-clickhouse-docs.sh 2023-03-30 15:45:33 +03:00
mkdocs.yml
pull-clickhouse-docs-hook.sh improve script for updating clickhouse-docs 2023-03-29 00:55:03 +02:00
README.md Update named collections related docs 2023-07-27 09:42:38 +03:00
redirects.txt move redirects to docusaurus config 2023-06-22 08:29:32 -04:00

Contributing to ClickHouse Documentation

Why Do You Need to Document ClickHouse

The main reason is that ClickHouse is an open source project, and if you don't write the docs, nobody does. "Incomplete or Confusing Documentation" is the top complaint about open source software by the results of a Github Open Source Survey of 2017. Documentation is highly valued but often overlooked. One of the most important contributions someone can make to an open source repository is a documentation update.

Many developers can say that the code is the best docs by itself, and they are right. But, ClickHouse is not a project for C++ developers. Most of its users don't know C++, and they can't understand the code quickly. ClickHouse is large enough to absorb almost any change without a noticeable trace. Nobody will find your very useful function, or an important setting, or a very informative new column in a system table if it is not referenced in the documentation.

If you want to help ClickHouse with documentation you can face, for example, the following questions:

  • "I don't know how to write."

    We have prepared some recommendations for you.

  • "I know what I want to write, but I don't know how to contribute to docs."

    Here are some tips.

Writing the docs is extremely useful for project's users and developers, and grows your karma.

Contents

What is the ClickHouse Documentation

The documentation contains information about all the aspects of the ClickHouse lifecycle: developing, testing, installing, operating, and using. The base language of the documentation is English. The English version is the most actual. All other languages are supported as much as they can by contributors from different countries.

At the moment, documentation exists in English, Russian, and Chinese. We store the reference documentation besides the ClickHouse source code in the GitHub repository, and user guides in a separate repo Clickhouse/clickhouse-docs.

To get the latter launch the get-clickhouse-docs.sh script.

Each language lies in the corresponding folder. Files that are not translated from English are symbolic links to the English ones.

How to Contribute to ClickHouse Documentation

You can contribute to the documentation in many ways, for example:

Contribute all new information in English language. Other languages are translations from English.

Markdown Dialect Cheatsheet

  • Headings: Place them on a separate line and start with # , ## or ### . Use the Title Case for them. Example:

    # The First Obligatory Title on a Page.
    
  • Bold text: **asterisks** or __underlines__.

  • Links: [link text](uri). Examples:

    • External link: [ClickHouse repo](https://github.com/ClickHouse/ClickHouse)
    • Cross link: [How to build docs](tools/README.md)
  • Images: ![Exclamation sign](uri). You can refer to local images as well as remote in internet.

  • Lists: Lists can be of two types:

    • - unordered: Each item starts from the -.
    • 1. ordered: Each item starts from the number.

    A list must be separated from the text by an empty line. Nested lists must be indented with 4 spaces.

  • Inline code: `in backticks`.

  • Multiline code blocks:

    ```lang_name
      code
      lines
      ```
  • Note:

    !!! info "Header"
        4 spaces indented text.
    
  • Warning:

    !!! warning "Header"
        4 spaces indented text.
    
  • Text hidden behind a cut (single sting that opens on click):

    <details markdown="1"> <summary>Visible text</summary>
        Hidden content.
    </details>`.
    
  • Colored text: <span style="color: red;">text</span>.

  • Heading anchor to be linked to: # Title {#anchor-name}.

  • Table:

    | Header    1 | Header    2 | Header    3 |
    | ----------- | ----------- | ----------- |
    | Cell     A1 | Cell     A2 | Cell     A3 |
    | Cell     B1 | Cell     B2 | Cell     B3 |
    | Cell     C1 | Cell     C2 | Cell     C3 |
    

Adding a New File

{## When adding a new file:

  • Make symbolic links for all other languages. You can use the following commands:

    $ cd /ClickHouse/clone/directory/docs
    $ ln -sr en/new/file.md lang/new/file.md
    

##}

Adding a New Language

  1. Create a new docs subfolder named using the ISO-639-1 language code.
  2. Add Markdown files with the translation, mirroring the folder structure of other languages.
  3. Commit and open a pull-request with the new content.

When everything is ready, we will add the new language to the website.

How to Write Content for ClickHouse Documentation

Documentation for Different Audiences

When writing documentation, think about the people who read it. Each audience has specific requirements for terms they use in communications.

ClickHouse documentation can be divided up by the audience for the following parts:

  • Conceptual topics like tutorials and overviews.

    These topics are for the most common audience. When editing text in them, use the most common terms that are comfortable for the audience with basic technical skills.

  • Query language reference and related topics.

    These parts of the documentation are dedicated to those who use ClickHouse for data analysis. Carefully describe syntax, input, and output data for expressions. Don't forget the examples.

  • Description of table engines and operation details.

    Operation engineers who help data analysts to solve their tasks should know how to install/update a ClickHouse server, maintain the ClickHouse cluster, how to integrate it with other tools and systems, how to get the maximum performance of their entire environment.

  • Developer's guides.

    The documentation provides code writers with information about how to write code for ClickHouse and how to build it in different environments.

Common Recommendations

  • When searching for a position for your text, try to place it in the most anticipated place.
  • Group entities. For example, if several functions solve similar tasks or belong to a specific group by use case or an application type, place them together.
  • Try to avoid slang. Use the most common and specific terms possible. If some terms are used as synonyms, state this explicitly.
  • Add examples for all the functionality. Add basic examples to show how the function works by itself. Add use case examples to show how the function participates in solving specific tasks.
  • Any text concerning politics, religion, or other social related themes are strictly prohibited in all the ClickHouse texts.
  • Proofread your text before publishing. Look for typos, missing punctuation, or repetitions that could be avoided.

Description Templates

When writing docs, you can use prepared templates. Copy the code of a template and use it in your contribution. Sometimes you just need to change level of headers.

Templates:

How to Build Documentation

You can build your documentation manually by following the instructions in the docs repo contrib-writing-guide. Also, our CI runs the documentation build after the documentation label is added to PR. You can see the results of a build in the GitHub interface. If you have no permissions to add labels, a reviewer of your PR will add it.