Commit Graph

23 Commits

Author SHA1 Message Date
Raúl Marín
20458b3e69 Completely disable rust build for FreeBSD 2024-04-10 12:39:44 +02:00
Raúl Marín
e274bd7b64 Use a rust workspace
This fixes recompilation and avoids duplication of dependencies
2024-04-09 00:50:40 +02:00
Raúl Marín
54d75be48c Separate builds and registries 2024-04-05 13:17:25 +02:00
Raúl Marín
f77822e732 Vendor rust dependencies 2024-04-04 19:22:38 +02:00
Azat Khuzhin
e926253664 Check is Rust avaiable for build, if not, suggest a way to disable Rust support
Rust requires internet connection, so let's detect if it works, and if
not, suggest building with -DENABLE_RUST=OFF, here is an example of the
output in case of error:

    $ docker run --network=none ...
    -- Copy skim to /root/rust/skim
    CMake Error at rust/CMakeLists.txt:112 (message):
      Rust (/rust/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo)
      support is not available (likely there is no internet connectivity):

          Updating git repository `https://github.com/azat-rust/tuikit.git`

      warning: spurious network error (3 tries remaining): failed to resolve
      address for github.com: Temporary failure in name resolution; class=Net
      (12)

      warning: spurious network error (2 tries remaining): failed to resolve
      address for github.com: Temporary failure in name resolution; class=Net
      (12)

      warning: spurious network error (1 tries remaining): failed to resolve
      address for github.com: Temporary failure in name resolution; class=Net
      (12)

      error: failed to load source for dependency `tuikit`

      Caused by:

        Unable to update https://github.com/azat-rust/tuikit.git?rev=e1994c0e03ff02c49cf1471f0cc3cbf185ce0104

      Caused by:

        failed to clone into: /rust/cargo/git/db/tuikit-c3ca927b4dbcf00d

      Caused by:

        network failure seems to have happened
        if a proxy or similar is necessary `net.git-fetch-with-cli` may help here
        https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli

      Caused by:

        failed to resolve address for github.com: Temporary failure in name resolution; class=Net (12)

      You can disable Rust support with -DENABLE_RUST=OFF
    Call Stack (most recent call first):
      rust/CMakeLists.txt:129 (add_rust_subdirectory)

    -- Configuring incomplete, errors occurred!

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-03-26 15:20:49 +01:00
Azat Khuzhin
a3938e7df0 Respect CMAKE_OSX_DEPLOYMENT_TARGET for Rust targets
This should fix the following warnings:

    ld: warning: object file (rust/prql/Debug/lib_ch_rust_prql.a[74](aarch_aapcs64.o)) was built for newer 'macOS' version (14.0) than being linked (11.0)
    ld: warning: object file (rust/skim/Debug/lib_ch_rust_skim_rust.a[40](c0e230e4543ad627-lib.rs.o)) was built for newer 'macOS' version (14.0) than being linked (11.0)
    ld: warning: object file (rust/skim/Debug/lib_ch_rust_skim_rust.a[46](6c3a6ae2099d4914-cxx.o)) was built for newer 'macOS' version (14.0) than being linked (11.0)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-02-11 10:55:14 +01:00
Raúl Marín
99d1659b4c Replace rust's BLAKE3 with llvm's implementation 2023-12-18 15:29:01 +01:00
Azat Khuzhin
1e0d8e098f Fix building Rust with Musl
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-11 20:49:19 +01:00
Mikhail f. Shiryaev
ed7ade255a
Use sccache for rust builds 2023-08-10 22:09:20 +02:00
Azat Khuzhin
0b258dda4e Reproducible builds for Rust
From now on cargo will not download anything from the internet during
builds. This step had been moved for docker image builds (via cargo
vendor).

And now cargo inside docker.io/clickhouse/binary-builder will not use
any crates from the internet, so we don't need to add --offline for
cargo commands in cmake (corrosion_import_crate()).

Also the docker build command had been adjusted to allow following
symlinks inside build context, by using tar, this is required for Rust
packages.

Note, that to make proper Cargo.lock that could be vendored I did the
following:
- per-project locks had been removed (since there is no automatic way to
  sync the workspace Cargo.lock with per-project Cargo.lock, since cargo
  update/generate-lockfile will use only per-project Cargo.toml files
  apparently, -Z minimal-versions does not helps either)
- and to generate Cargo.lock with less changes I've pinned version in
  the Cargo.toml strictly, i.e. not 'foo = "0.1"' but 'foo = "=0.1"'
  then the Cargo.lock for workspace had been generated and afterwards
  I've reverted this part.

Plus I have to update the dependencies afterwards, since otherwise there
are conflicts with dependencies for std library. Non trivial.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-07-22 22:46:22 +02:00
János Benjamin Antal
e74acda53e
PRQL integration ()
* Added prql-lib

* Add PRQL parser

* Extend stateless tests

* Add unit tests for `ParserPRQL`

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-37-24.eu-central-1.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-10-10-10-195.eu-central-1.compute.internal>
Co-authored-by: Александр Нам <47687537+seshWCS@users.noreply.github.com>
2023-07-20 12:54:42 +02:00
Azat Khuzhin
bf127f4e1e MSan support for Rust
Previously you have to unpoison memory from the Rust, however Rust does
supports MSan, so let's simply use it.

But for this we need nightly Rust and recompile standard library.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-06-04 23:04:55 +02:00
Azat Khuzhin
1107988a82 Improve performance of BLAKE3 by 11% by enabling LTO for Rust
LTO in Rust produces multiple definition of `rust_eh_personality' (and
few others), and to overcome this --allow-multiple-definition has been
added.

Query for benchmark:

    SELECT ignore(BLAKE3(materialize('Lorem ipsum dolor sit amet, consectetur adipiscing elit'))) FROM numbers(1000000000) FORMAT `Null`

  upstream           : Elapsed: 2.494 sec. Processed 31.13 million rows, 249.08 MB (12.48 million rows/s., 99.86 MB/s.)
  upstream + rust lto: Elapsed: 13.56 sec. Processed 191.9 million rows, 1.5400 GB (14.15 million rows/s., 113.22 MB/s.)
  llvm BLAKE3        : Elapsed: 3.053 sec. Processed 43.24 million rows, 345.88 MB (14.16 million rows/s., 113.28 MB/s.)

Note, I thought about simply replacing it with BLAKE3 from LLVM, but:
- this will not solve LTO issues for Rust (and in future more libraries
  could be added)
- it makes integrating_rust_libraries.md useless (and there is even blog
  post)

So instead I've decided to add this quirk (--allow-multiple-definition)
to fix builds.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-05-06 22:28:56 +02:00
Azat Khuzhin
9f9989ea72 Properly detect changes in Rust code and recompile Rust libraries
Due to we need to substitude config for Rust builds (see  for
details), the source dir is copied to the binary dir, but this is done
only once, so to detect changes in sources you need to run cmake,
without this patch.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-02 18:33:15 +01:00
Azat Khuzhin
89c071e291 Build rust modules from the binary directory
Our crates has configuration files:
- config for cargo (see config.toml.in)
- and possibly config for build (build.rs.in)

Previously it uses source directory, and it will overrides files in the
source directory, which will require recompile of crates in case of
multiple builds for one source directory.

To avoid overlaps different builds for one source directory, crate will
be copied from source directory to the binary directory.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-30 18:15:41 +01:00
Azat Khuzhin
e7c5b48d84 rust: fix buidling modules with CMAKE_BUILD_TYPE in a different case
Before this patch corrosion requires that CMAKE_BUILD_TYPE matches the
CMAKE_CONFIGURATION_TYPES, which is
"RelWithDebInfo;Debug;Release;MinSizeRel", so that said, that if you
were using CMAKE_BUILD_TYPE=debug, it will not work.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-14 20:58:34 +01:00
Azat Khuzhin
f8c17d4a66 rust: reuse RUST_CXXFLAGS for skim
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-14 20:58:17 +01:00
Azat Khuzhin
82aaad61aa Integrate skim into the client/local
Note, that it can the fail the client if the skim itself will fail,
however I haven't seen it panicd, so let's try.

P.S. about adding USE_SKIM into configure header instead of just compile
option for target, it is better, because it allows not to recompile lots
of C++ headers, since we have to add skim library as PUBLIC. But anyway
this will be resolved in a different way, but separatelly.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-14 20:57:41 +01:00
Azat Khuzhin
28737a503c Configure rustc compiler properly
Right now it works for host platforms because of gcc package, that
includes gcc-cross sysroot.

Use bundled sysroot from contrib instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-14 20:57:29 +01:00
Azat Khuzhin
67fa185611
Revert "Builtin skim" 2022-12-14 17:17:19 +03:00
Azat Khuzhin
de58e9c02d Integrate skim into the client/local
Note, that it can the fail the client if the skim itself will fail,
however I haven't seen it panicd, so let's try.

P.S. about adding USE_SKIM into configure header instead of just compile
option for target, it is better, because it allows not to recompile lots
of C++ headers, since we have to add skim library as PUBLIC.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-11 15:52:00 +01:00
Azat Khuzhin
8570f15688 Configure rustc compiler properly
Right now it works for host platforms because of gcc package, that
includes gcc-cross sysroot.

Use bundled sysroot from contrib instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-11 15:52:00 +01:00
BoloniniD
147dfac11e Try using Corrosion 2022-09-12 23:05:41 +03:00