Merge pull request #42939 from DanRoscigno/add-vis-to-cell-towers

Add vis to cell towers

docs/en/getting-started/example-datasets/cell-towers.md

@@ -4,25 +4,39 @@ sidebar_label: Cell Towers
 sidebar_position: 3
 title: "Cell Towers"
 ---
+import ConnectionDetails from '@site/docs/en/_snippets/_gather_your_details_http.mdx';
 
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 import CodeBlock from '@theme/CodeBlock';
 import ActionsMenu from '@site/docs/en/_snippets/_service_actions_menu.md';
 import SQLConsoleDetail from '@site/docs/en/_snippets/_launch_sql_console.md';
+import SupersetDocker from '@site/docs/en/_snippets/_add_superset_detail.md';
 
-This dataset is from [OpenCellid](https://www.opencellid.org/) - The world's largest Open Database of Cell Towers.
+## Goal
+
+In this guide you will learn how to:
+- Load the OpenCelliD data in Clickhouse
+- Connect Apache Superset to ClickHouse
+- Build a dashboard based on data available in the dataset
+
+Here is a preview of the dashboard created in this guide:
+
+![Dashboard of cell towers by radio type in mcc 204](@site/docs/en/getting-started/example-datasets/images/superset-cell-tower-dashboard.png)
+
+## Get the Dataset {#get-the-dataset}
+
+This dataset is from [OpenCelliD](https://www.opencellid.org/) - The world's largest Open Database of Cell Towers.
 
 As of 2021, it contains more than 40 million records about cell towers (GSM, LTE, UMTS, etc.) around the world with their geographical coordinates and metadata (country code, network, etc).
 
 OpenCelliD Project is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License, and we redistribute a snapshot of this dataset under the terms of the same license. The up-to-date version of the dataset is available to download after sign in.
 
-
-## Get the Dataset {#get-the-dataset}
-
 <Tabs groupId="deployMethod">
 <TabItem value="serverless" label="ClickHouse Cloud" default>
 
+### Load the sample data
+
 ClickHouse Cloud provides an easy-button for uploading this dataset from S3.  Log in to your ClickHouse Cloud organization, or create a free trial at [ClickHouse.cloud](https://clickhouse.cloud).
 <ActionsMenu menu="Load Data" />
 
@@ -30,13 +44,33 @@ Choose the **Cell Towers** dataset from the **Sample data** tab, and **Load data
 
 ![Load cell towers dataset](@site/docs/en/_snippets/images/cloud-load-data-sample.png)
 
-Examine the schema of the cell_towers table:
+### Examine the schema of the cell_towers table
 ```sql
 DESCRIBE TABLE cell_towers
 ```
 
 <SQLConsoleDetail />
 
+This is the output of `DESCRIBE`.  Down further in this guide the field type choices will be described.
+```response
+┌─name──────────┬─type──────────────────────────────────────────────────────────────────┬
+│ radio         │ Enum8('' = 0, 'CDMA' = 1, 'GSM' = 2, 'LTE' = 3, 'NR' = 4, 'UMTS' = 5) │
+│ mcc           │ UInt16                                                                │
+│ net           │ UInt16                                                                │
+│ area          │ UInt16                                                                │
+│ cell          │ UInt64                                                                │
+│ unit          │ Int16                                                                 │
+│ lon           │ Float64                                                               │
+│ lat           │ Float64                                                               │
+│ range         │ UInt32                                                                │
+│ samples       │ UInt32                                                                │
+│ changeable    │ UInt8                                                                 │
+│ created       │ DateTime                                                              │
+│ updated       │ DateTime                                                              │
+│ averageSignal │ UInt8                                                                 │
+└───────────────┴───────────────────────────────────────────────────────────────────────┴
+```
+
 </TabItem>
 <TabItem value="selfmanaged" label="Self-managed">
 
@@ -86,7 +120,7 @@ clickhouse-client --query "INSERT INTO cell_towers FORMAT CSVWithNames" < cell_t
 </TabItem>
 </Tabs>
 
-## Example queries {#examples}
+## Run some example queries {#examples}
 
 1. A number of cell towers by type:
 
@@ -127,13 +161,13 @@ SELECT mcc, count() FROM cell_towers GROUP BY mcc ORDER BY count() DESC LIMIT 10
 10 rows in set. Elapsed: 0.019 sec. Processed 43.28 million rows, 86.55 MB (2.33 billion rows/s., 4.65 GB/s.)
 ```
 
-So, the top countries are: the USA, Germany, and Russia.
+Based on the above query and the [MCC list](https://en.wikipedia.org/wiki/Mobile_country_code), the countries with the most cell towers are: the USA, Germany, and Russia.
 
 You may want to create an [External Dictionary](../../sql-reference/dictionaries/external-dictionaries/external-dicts.md) in ClickHouse to decode these values.
 
 ## Use case: Incorporate geo data {#use-case}
 
-Using `pointInPolygon` function.
+Using the [`pointInPolygon`](/docs/en/sql-reference/functions/geo/coordinates.md/#pointinpolygon) function.
 
 1. Create a table where we will store polygons:
 
@@ -224,6 +258,110 @@ WHERE pointInPolygon((lon, lat), (SELECT * FROM moscow))
 1 rows in set. Elapsed: 0.067 sec. Processed 43.28 million rows, 692.42 MB (645.83 million rows/s., 10.33 GB/s.)
 ```
 
-The data is also available for interactive queries in the [Playground](https://play.clickhouse.com/play?user=play), [example](https://play.clickhouse.com/play?user=play#U0VMRUNUIG1jYywgY291bnQoKSBGUk9NIGNlbGxfdG93ZXJzIEdST1VQIEJZIG1jYyBPUkRFUiBCWSBjb3VudCgpIERFU0M=).
+## Review of the schema
 
-Although you cannot create temporary tables there.
+Before building visualizations in Superset have a look at the columns that you will use. This dataset primarily provides the location (Longitude and Latitude) and radio types at mobile cellular towers worldwide. The column descriptions can be found in the [community forum](https://community.opencellid.org/t/documenting-the-columns-in-the-downloadable-cells-database-csv/186).  The columns used in the visualizations that will be built are described below
+
+Here is a description of the columns taken from the OpenCelliD forum:
+
+| Column       | Description                                            |
+|--------------|--------------------------------------------------------|
+| radio        | Technology generation: CDMA, GSM, UMTS, 5G NR          |
+| mcc          | Mobile Country Code: `204` is The Netherlands          |
+| lon          | Longitude: With Latitude, approximate tower location   |
+| lat          | Latitude: With Longitude, approximate tower location   |
+
+:::tip mcc
+To find your MCC check [Mobile network codes](https://en.wikipedia.org/wiki/Mobile_country_code), and use the three digits in the **Mobile country code** column.
+:::
+
+The schema for this table was designed for compact storage on disk and query speed.  
+- The `radio` data is stored as an `Enum8` (`UInt8`) rather than a string.
+- `mcc` or Mobile country code, is stored as a `UInt16` as we know the range is 1 - 999.
+- `lon` and `lat` are `Float64`.
+
+None of the other fields are used in the queries or visualizations in this guide, but they are described in the forum linked above if you are interested.
+
+## Build visualizations with Apache Superset
+
+Superset is easy to run from Docker.  If you already have Superset running, all you need to do is add ClickHouse Connect with `pip install clickhouse-connect`.  If you need to install Superset open the **Launch Apache Superset in Docker** directly below.
+
+<SupersetDocker />
+
+To build a Superset dashboard using the OpenCelliD dataset you should:
+- Add your ClickHouse service as a Superset **database**
+- Add the table **cell_towers** as a Superset **dataset**
+- Create some **charts**
+- Add the charts to a **dashboard**
+
+### Add your ClickHouse service as a Superset database
+
+<ConnectionDetails />
+
+  In Superset a database can be added by choosing the database type, and then providing the connection details.  Open Superset and look for the **+**, it has a menu with **Data** and then **Connect database** options.
+
+  ![Add a database](@site/docs/en/getting-started/example-datasets/images/superset-add.png)
+
+  Choose **ClickHouse Connect** from the list:
+
+  ![Choose clickhouse connect as database type](@site/docs/en/getting-started/example-datasets/images/superset-choose-a-database.png)
+
+:::note
+  If **ClickHouse Connect** is not one of your options, then you will need to install it.  The comand is `pip install clickhouse-connect`, and more info is [available here](https://pypi.org/project/clickhouse-connect/).
+:::
+
+#### Add your connection details:
+
+:::tip
+  Make sure that you set **SSL** on when connecting to ClickHouse Cloud or other ClickHouse systems that enforce the use of SSL.
+:::
+
+  ![Add ClickHouse as a Superset datasource](@site/docs/en/getting-started/example-datasets/images/superset-connect-a-database.png)
+
+### Add the table **cell_towers** as a Superset **dataset**
+
+  In Superset a **dataset** maps to a table within a database.  Click on add a dataset and choose your ClickHouse service, the database containing your table (`default`), and choose the `cell_towers` table:
+
+![Add cell_towers table as a dataset](@site/docs/en/getting-started/example-datasets/images/superset-add-dataset.png)
+
+### Create some **charts**
+
+When you choose to add a chart in Superset you have to specify the dataset (`cell_towers`) and the chart type.  Since the OpenCelliD dataset provides longitude and latitude coordinates for cell towers we will create a **Map** chart.  The **deck.gL Scatterplot** type is suited to this dataset as it works well with dense data points on a map.
+
+![Create a map in Superset](@site/docs/en/getting-started/example-datasets/images/superset-create-map.png)
+
+#### Specify the query used for the map
+
+A deck.gl Scatterplot requires a longitude and latitude, and one or more filters can also be applied to the query.  In this example two filters are applied, one for cell towers with UMTS radios, and one for the Mobile country code assigned to The Netherlands.
+
+The fields `lon` and `lat` contain the longitude and latitude:
+
+![Specify longitude and latitude fields](@site/docs/en/getting-started/example-datasets/images/superset-lon-lat.png)
+
+Add a filter with `mcc` = `204` (or substitute any other `mcc` value):
+
+![Filter on MCC 204](@site/docs/en/getting-started/example-datasets/images/superset-mcc-204.png)
+
+Add a filter with `radio` = `'UMTS'` (or substitute any other `radio` value, you can see the choices in the output of `DESCRIBE TABLE cell_towers`):
+
+![Filter on radio = UMTS](@site/docs/en/getting-started/example-datasets/images/superset-radio-umts.png)
+
+This is the full configuration for the chart that filters on `radio = 'UMTS'` and `mcc = 204`:
+
+![Chart for UMTS radios in MCC 204](@site/docs/en/getting-started/example-datasets/images/superset-umts-netherlands.png)
+
+Click on **UPDATE CHART** to render the visualization.
+
+### Add the charts to a **dashboard**
+
+This screenshot shows cell tower locations with LTE, UMTS, and GSM radios.  The charts are all created in the same way and they are added to a dashboard.
+
+  ![Dashboard of cell towers by radio type in mcc 204](@site/docs/en/getting-started/example-datasets/images/superset-cell-tower-dashboard.png)
+
+:::tip
+The data is also available for interactive queries in the [Playground](https://play.clickhouse.com/play?user=play).
+
+This [example](https://play.clickhouse.com/play?user=play#U0VMRUNUIG1jYywgY291bnQoKSBGUk9NIGNlbGxfdG93ZXJzIEdST1VQIEJZIG1jYyBPUkRFUiBCWSBjb3VudCgpIERFU0M=) will populate the username and even the query for you.
+
+Although you cannot create tables in the Playground, you can run all of the queries and even use Superset (adjust the hostname and port number).
+:::

docs/en/getting-started/example-datasets/recipes.md

@@ -4,7 +4,7 @@ sidebar_label: Recipes Dataset
 title: "Recipes Dataset"
 ---
 
-RecipeNLG dataset is available for download [here](https://recipenlg.cs.put.poznan.pl/dataset). It contains 2.2 million recipes. The size is slightly less than 1 GB.
+The RecipeNLG dataset is available for download [here](https://recipenlg.cs.put.poznan.pl/dataset). It contains 2.2 million recipes. The size is slightly less than 1 GB.
 
 ## Download and Unpack the Dataset
 

docs/en/sql-reference/functions/geo/index.md

@@ -8,70 +8,69 @@ title: "Geo Functions"
 
 ## Geographical Coordinates Functions
 
--   [greatCircleDistance](./coordinates.md#greatCircleDistance)
--   [geoDistance](./coordinates.md#geoDistance)
--   [greatCircleAngle](./coordinates.md#greatCircleAngle)
--   [pointInEllipses](./coordinates.md#pointInEllipses)
--   [pointInPolygon](./coordinates.md#pointInPolygon)
+-   [greatCircleDistance](./coordinates.md#greatcircledistance)
+-   [geoDistance](./coordinates.md#geodistance)
+-   [greatCircleAngle](./coordinates.md#greatcircleangle)
+-   [pointInEllipses](./coordinates.md#pointinellipses)
+-   [pointInPolygon](./coordinates.md#pointinpolygon)
 
 ## Geohash Functions
--   [geohashEncode](./geohash.md#geohashEncode)
--   [geohashDecode](./geohash.md#geohashDecode)
--   [geohashesInBox](./geohash.md#geohashesInBox)
+-   [geohashEncode](./geohash.md#geohashencode)
+-   [geohashDecode](./geohash.md#geohashdecode)
+-   [geohashesInBox](./geohash.md#geohashesinbox)
 
 ## H3 Indexes Functions
 
--   [h3IsValid](./h3.md#h3IsValid)
--   [h3GetResolution](./h3.md#h3GetResolution)
--   [h3EdgeAngle](./h3.md#h3EdgeAngle)
--   [h3EdgeLengthM​](./h3.md#h3EdgeLengthM​)
--   [h3EdgeLengthKm](./h3.md#h3EdgeLengthKm)
--   [geoToH3](./h3.md#geoToH3)
--   [h3ToGeo](./h3.md#h3ToGeo)
--   [h3ToGeoBoundary](./h3.md#h3ToGeoBoundary)
--   [h3kRing](./h3.md#h3kRing)
--   [h3GetBaseCell](./h3.md#h3GetBaseCell)
--   [h3HexAreaM2](./h3.md#h3HexAreaM2)
--   [h3HexAreaKm2](./h3.md#h3HexAreaKm2)
--   [h3IndexesAreNeighbors](./h3.md#h3IndexesAreNeighbors)
--   [h3ToChildren](./h3.md#h3ToChildren)
--   [h3ToParent](./h3.md#h3ToParent)
--   [h3ToString](./h3.md#h3ToString)
--   [stringToH3](./h3.md#stringToH3)
--   [h3GetResolution](./h3.md#h3GetResolution)
--   [h3IsResClassIII](./h3.md#h3IsResClassIII)
--   [h3IsPentagon](./h3.md#h3IsPentagon)
--   [h3GetFaces](./h3.md#h3GetFaces)
--   [h3CellAreaM2](./h3.md#h3CellAreaM2)
--   [h3CellAreaRads2](./h3.md#h3CellAreaRads2)
--   [h3ToCenterChild](./h3.md#h3ToCenterChild)
--   [h3ExactEdgeLengthM](./h3.md#h3ExactEdgeLengthM)
--   [h3ExactEdgeLengthKm](./h3.md#h3ExactEdgeLengthKm)
--   [h3ExactEdgeLengthRads](./h3.md#h3ExactEdgeLengthRads)
--   [h3NumHexagons](./h3.md#h3NumHexagons)
--   [h3Line](./h3.md#h3Line)
--   [h3Distance](./h3.md#h3Distance)
--   [h3HexRing](./h3.md#h3HexRing)
--   [h3GetUnidirectionalEdge](./h3.md#h3GetUnidirectionalEdge)
--   [h3UnidirectionalEdgeIsValid](./h3.md#h3UnidirectionalEdgeIsValid)
--   [h3GetOriginIndexFromUnidirectionalEdge](./h3.md#h3GetOriginIndexFromUnidirectionalEdge)
--   [h3GetDestinationIndexFromUnidirectionalEdge](./h3.md#h3GetDestinationIndexFromUnidirectionalEdge)
--   [h3GetIndexesFromUnidirectionalEdge](./h3.md#h3GetIndexesFromUnidirectionalEdge)
--   [h3GetUnidirectionalEdgesFromHexagon](./h3.md#h3GetUnidirectionalEdgesFromHexagon)
--   [h3GetUnidirectionalEdgeBoundary](./h3.md#h3GetUnidirectionalEdgeBoundary)
+-   [h3IsValid](./h3.md#h3isvalid)
+-   [h3GetResolution](./h3.md#h3getresolution)
+-   [h3EdgeAngle](./h3.md#h3edgeangle)
+-   [h3EdgeLengthM](./h3.md#h3edgelengthm)
+-   [h3EdgeLengthKm](./h3.md#h3edgelengthkm)
+-   [geoToH3](./h3.md#geotoh3)
+-   [h3ToGeo](./h3.md#h3togeo)
+-   [h3ToGeoBoundary](./h3.md#h3togeoboundary)
+-   [h3kRing](./h3.md#h3kring)
+-   [h3GetBaseCell](./h3.md#h3getbasecell)
+-   [h3HexAreaM2](./h3.md#h3hexaream2)
+-   [h3HexAreaKm2](./h3.md#h3hexareakm2)
+-   [h3IndexesAreNeighbors](./h3.md#h3indexesareneighbors)
+-   [h3ToChildren](./h3.md#h3tochildren)
+-   [h3ToParent](./h3.md#h3toparent)
+-   [h3ToString](./h3.md#h3tostring)
+-   [stringToH3](./h3.md#stringtoh3)
+-   [h3GetResolution](./h3.md#h3getresolution)
+-   [h3IsResClassIII](./h3.md#h3isresclassiii)
+-   [h3IsPentagon](./h3.md#h3ispentagon)
+-   [h3GetFaces](./h3.md#h3getfaces)
+-   [h3CellAreaM2](./h3.md#h3cellaream2)
+-   [h3CellAreaRads2](./h3.md#h3cellarearads2)
+-   [h3ToCenterChild](./h3.md#h3tocenterchild)
+-   [h3ExactEdgeLengthM](./h3.md#h3exactedgelengthm)
+-   [h3ExactEdgeLengthKm](./h3.md#h3exactedgelengthkm)
+-   [h3ExactEdgeLengthRads](./h3.md#h3exactedgelengthrads)
+-   [h3NumHexagons](./h3.md#h3numhexagons)
+-   [h3Line](./h3.md#h3line)
+-   [h3Distance](./h3.md#h3distance)
+-   [h3HexRing](./h3.md#h3hexring)
+-   [h3GetUnidirectionalEdge](./h3.md#h3getunidirectionaledge)
+-   [h3UnidirectionalEdgeIsValid](./h3.md#h3unidirectionaledgeisvalid)
+-   [h3GetOriginIndexFromUnidirectionalEdge](./h3.md#h3getoriginindexfromunidirectionaledge)
+-   [h3GetDestinationIndexFromUnidirectionalEdge](./h3.md#h3getdestinationindexfromunidirectionaledge)
+-   [h3GetIndexesFromUnidirectionalEdge](./h3.md#h3getindexesfromunidirectionaledge)
+-   [h3GetUnidirectionalEdgesFromHexagon](./h3.md#h3getunidirectionaledgesfromhexagon)
+-   [h3GetUnidirectionalEdgeBoundary](./h3.md#h3getunidirectionaledgeboundary)
 
 ## S2 Index Functions
 
--   [geoToS2](./s2.md#geoToS2)
--   [s2ToGeo](./s2.md#s2ToGeo)
--   [s2GetNeighbors](./s2.md#s2GetNeighbors)
--   [s2CellsIntersect](./s2.md#s2CellsIntersect)
--   [s2CapContains](./s2.md#s2CapContains)
--   [s2CapUnion](./s2.md#s2CapUnion)
--   [s2RectAdd](./s2.md#s2RectAdd)
--   [s2RectContains](./s2.md#s2RectContains)
--   [s2RectUinion](./s2.md#s2RectUinion)
--   [s2RectIntersection](./s2.md#s2RectIntersection)
+-   [geoToS2](./s2.md#geotos2)
+-   [s2ToGeo](./s2.md#s2togeo)
+-   [s2GetNeighbors](./s2.md#s2getneighbors)
+-   [s2CellsIntersect](./s2.md#s2cellsintersect)
+-   [s2CapContains](./s2.md#s2capcontains)
+-   [s2CapUnion](./s2.md#s2capunion)
+-   [s2RectAdd](./s2.md#s2rectadd)
+-   [s2RectContains](./s2.md#s2rectcontains)
+-   [s2RectUnion](./s2.md#s2rectunion)
+-   [s2RectIntersection](./s2.md#s2rectintersection)
 
 
-[Original article](https://clickhouse.com/docs/en/sql-reference/functions/geo/) <!--hide-->