switch from mkdocs to Docusaurus

docs/README.md

@@ -38,9 +38,9 @@ Writing the docs is extremely useful for project's users and developers, and gro
 
 The documentation contains information about all the aspects of the ClickHouse lifecycle: developing, testing, installing, operating, and using. The base language of the documentation is English. The English version is the most actual. All other languages are supported as much as they can by contributors from different countries.
 
-At the moment, [documentation](https://clickhouse.com/docs) exists in English, Russian, Chinese, Japanese. We store the documentation besides the ClickHouse source code in the [GitHub repository](https://github.com/ClickHouse/ClickHouse/tree/master/docs).
+At the moment, [documentation](https://clickhouse.com/docs) exists in English, Russian, and Chinese. We store the reference documentation besides the ClickHouse source code in the [GitHub repository](https://github.com/ClickHouse/ClickHouse/tree/master/docs), and user guides in a separate repo [Clickhouse/clickhouse-docs](https://github.com/ClickHouse/clickhouse-docs).
 
-Each language lays in the corresponding folder. Files that are not translated from English are the symbolic links to the English ones.
+Each language lies in the corresponding folder. Files that are not translated from English are symbolic links to the English ones.
 
 <a name="how-to-contribute"/>
 
@@ -48,9 +48,9 @@ Each language lays in the corresponding folder. Files that are not translated fr
 
 You can contribute to the documentation in many ways, for example:
 
-- Fork the ClickHouse repository, edit, commit, push, and open a pull request.
+- Fork the ClickHouse and ClickHouse-docs repositories, edit, commit, push, and open a pull request.
 
-    Add the `documentation` label to this pull request for proper automatic checks applying. If you have no permissions for adding labels, the reviewer of your PR adds it.
+    Add the `pr-documentation` label to this pull request for proper automatic checks applying. If you do not have permission to add labels, then the reviewer of your PR will add it.
 
 - Open a required file in the ClickHouse repository and edit it from the GitHub web interface.
 
@@ -158,15 +158,15 @@ When everything is ready, we will add the new language to the website.
 
 <a name="target-audience"/>
 
-### Documentation for Different Audience
+### Documentation for Different Audiences
 
-When writing documentation, think about people who read it. Each audience has specific requirements for terms they use in communications.
+When writing documentation, think about the people who read it. Each audience has specific requirements for terms they use in communications.
 
-ClickHouse documentation can be divided by the audience for the following parts:
+ClickHouse documentation can be divided up by the audience for the following parts:
 
-- Conceptual topics in [Introduction](https://clickhouse.com/docs/en/), tutorials and overviews, changelog.
+- Conceptual topics like tutorials and overviews.
 
-    These topics are for the most common auditory. When editing text in them, use the most common terms that are comfortable for the audience with basic technical skills.
+    These topics are for the most common audience. When editing text in them, use the most common terms that are comfortable for the audience with basic technical skills.
 
 - Query language reference and related topics.
 

docs/tools/README.md

@@ -1,50 +1,94 @@
-## How ClickHouse documentation is generated? {#how-clickhouse-documentation-is-generated}
+## Generating ClickHouse documentation {#how-clickhouse-documentation-is-generated}
 
-ClickHouse documentation is built using [build.py](build.py) script that uses [mkdocs](https://www.mkdocs.org) library and it’s dependencies to separately build all version of documentations (all languages in either single and multi page mode) as static HTMLs for each single page version. The results are then put in the correct directory structure. It is recommended to use Python 3.7 to run this script.
+ClickHouse documentation is built using [Docusaurus](https://docusaurus.io). 
 
-[release.sh](release.sh) also pulls static files needed for [official ClickHouse website](https://clickhouse.com) from [../../website](../../website) folder then pushes to specified GitHub repo to be served via [GitHub Pages](https://pages.github.com).
+## Check the look of your documentation changes {#how-to-check-if-the-documentation-will-look-fine}
 
-## How to check if the documentation will look fine? {#how-to-check-if-the-documentation-will-look-fine}
+There are a few options that are all useful depending on how large or complex your edits are.
 
-There are few options that are all useful depending on how large or complex your edits are.
+### Use the GitHub web interface to edit
 
-### Use GitHub web interface to edit
+Every page in the docs has an **Edit this page** link that opens the page in the GitHub editor.  GitHub has Markdown support with a preview feature. The details of GitHub Markdown and the documentation Markdown are a bit different but generally this is close enough, and the person merging your PR will build the docs and check them.
 
-GitHub has Markdown support with preview feature, but the details of GitHub Markdown dialect are a bit different in ClickHouse documentation.
+### Install a Markdown editor or plugin for your IDE {#install-markdown-editor-or-plugin-for-your-ide}
 
-### Install Markdown editor or plugin for your IDE {#install-markdown-editor-or-plugin-for-your-ide}
+Usually, these plugins provide a preview of how the markdown will render, and they catch basic errors like unclosed tags very early.
 
-Usually those also have some way to preview how Markdown will look like, which allows to catch basic errors like unclosed tags very early.
 
-### Use build.py {#use-build-py}
+## Build the docs locally {#use-build-py}
 
-It’ll take some effort to go through, but the result will be very close to production documentation.
+You can build the docs locally.  It takes a few minutes to set up, but once you have done it the first time, the process is very simple.
 
-For the first time you’ll need to:
+### Clone the repos
 
-#### 1. Set up virtualenv
+The documentation is in two repos, clone both of them:
+- [ClickHouse/ClickHouse](https://github.com/ClickHouse/ClickHouse)
+- [ClickHouse/ClickHouse-docs](https://github.com/ClickHouse/clickhouse-docs)
 
-``` bash
-$ cd ClickHouse/docs/tools
-$ mkdir venv
-$ virtualenv -p $(which python3) venv
-$ source venv/bin/activate
-$ pip3 install -r requirements.txt
+### Install Node.js
+
+The documentation is built with Docusaurus, which requires Node.js.  We recommend version 16. Install [Node.js](https://nodejs.org/en/download/).
+
+### Copy files into place
+
+Docusaurus expects all of the markdown files to be located in the directory tree `clickhouse-docs/docs/`.  This is not the way our repos are set up, so some copying of files is needed to build the docs:
+
+```bash
+# from the parent directory of both the ClickHouse/ClickHouse and ClickHouse-clickhouse-docs repos:
+cp -r ClickHouse/docs/en/development     clickhouse-docs/docs/en/
+cp -r ClickHouse/docs/en/engines         clickhouse-docs/docs/en/
+cp -r ClickHouse/docs/en/getting-started clickhouse-docs/docs/en/
+cp -r ClickHouse/docs/en/interfaces      clickhouse-docs/docs/en/
+cp -r ClickHouse/docs/en/operations      clickhouse-docs/docs/en/
+cp -r ClickHouse/docs/en/sql-reference   clickhouse-docs/docs/en/
+
+cp -r ClickHouse/docs/ru/*               clickhouse-docs/docs/ru/
+cp -r ClickHouse/docs/zh                 clickhouse-docs/docs/
 ```
 
-#### 2. Run build.py
+#### Note: Symlinks will not work.
+### Setup Docusaurus
 
-When all prerequisites are installed, running `build.py` without args (there are some, check `build.py --help`) will generate `ClickHouse/docs/build` folder with complete static html website.
+There are two commands that you may need to use with Docusaurus:
+- `yarn install`
+- `yarn start`
 
-The easiest way to see the result is to use `--livereload=8888` argument of build.py. Alternatively, you can manually launch a HTTP server to serve the docs, for example by running `cd ClickHouse/docs/build && python3 -m http.server 8888`. Then go to http://localhost:8888 in browser. Feel free to use any other port instead of 8888.
+#### Install Docusaurus and its dependencies:
+
+```bash
+cd clickhouse-docs
+yarn install
+```
+
+#### Start a development Docusaurus environment
+
+This command will start Docusaurus in development mode, which means that as you edit source (for example, `.md` files) files the changes will be rendered into HTML files and served by the Docusaurus development server.
+
+```bash
+yarn start
+```
+
+### Make your changes to the markdown files
+
+Edit your files.  Remember that if you are editing files in the `ClickHouse/ClickHouse` repo then you should edit them
+in that repo and then copy the edited file into the `ClickHouse/clickhouse-docs/` directory structure so that they are updated in your develoment environment.
+
+`yarn start` probably opened a browser for you when you ran it; if not, open a browser to `http://localhost:3000/docs/en/intro` and navigate to the documentation that you are changing.  If you have already made the changes, you can verify them here; if not, make them, and you will see the page update as you save the changes.  
 
 ## How to change code highlighting? {#how-to-change-code-hl}
 
-ClickHouse does not use mkdocs `highlightjs` feature. It uses modified pygments styles instead.
-If you want to change code highlighting, edit the `website/css/highlight.css` file.
-Currently, an [eighties](https://github.com/idleberg/base16-pygments/blob/master/css/base16-eighties.dark.css) theme
-is used.
+Code highlighting is based on the language chosen for your code blocks.  Specify the language when you start the code block:
 
+<pre lang="no-highlight"><code>```sql
+SELECT firstname from imdb.actors;
+```
+</code></pre>
+
+```sql
+SELECT firstname from imdb.actors;
+```
+
+If you need a language supported then open an issue in [ClickHouse-docs](https://github.com/ClickHouse/clickhouse-docs/issues).
 ## How to subscribe on documentation changes? {#how-to-subscribe-on-documentation-changes}
 
 At the moment there’s no easy way to do just that, but you can consider: