Ditto Rust SDK

Overview

Describes the different layers of the Ditto Rust SDK.

Ditto's core codebase is written in Rust. Currently, however, Rust does not have a stable ABI suitable for directly linking. Therefore the Rust SDK, like the other Ditto SDKs, is exposed through an external interface that uses the C ABI calling conventions. This core library is compiled as a both a static and dynamic library for a variety of architectures. The dittolive-ditto-sys crate contains Rust bindings to this C ABI library. The build.rs script will also attempt to identify the proper library binary for the host environment, download it from Ditto, and link to it. The Rust SDK includes this -sys crate as a dependency and then exposes an idiomatic Rust interface on top of this library, along with documentation and example apps.

Building an App with the Rust SDK

The following outlines the general process of getting started with a new App based on the Ditto Rust SDK. Rust currently does not support a stable ABI. To work around this, the Ditto Rust SDK is distributed in two parts: an ergonomic, open-source crate and a closed-source pre-compiled library. It is essential to have the correct library for both your development system and ultimate production target.

There are also some key terms you will need to know, especially for cross-compiling the RustSDK.

Using a pre-built binary for your RustSDK-based App

The easiest way to get started with the Ditto Rust SDK is to use a pre-built Ditto library for your target architecture, where available.

  1. Obtain a Ditto License.
  2. Install the nightly tool chain for your current development machine using [rustup]https://rustup.rs
  3. Create a new Rust repository using github or cargo new
  4. In the root of the newly created project directory, edit the Cargo.toml file. Add a dittolive-ditto as a dependency. This is the crate for the Ditto Rust SDK.
  5. Test building the automatically generated Hello World app by running cargo build. This will trigger linking against the Ditto SDK for the current host system. If Ditto is not present for the current host system, this step will report an error when run with --verbose logging. If curl is present on the host system and the binary component of the Ditto SDK is absent, the build.rs script will attempt to download the appropriate library for the compilation target automatically. This step will also create a TARGET_DIR target directory (ie. target/debug) in the project root. This is a good location for putting the binary component of the Ditto SDK, especially if the development and production hosts are different.
  6. For cross-compilation (ie. development on MacOS for a Linux target), the Rust build script and machinery such as pkg_config may not automatically find the correct library for your target platform. In this event the Ditto library should be downloaded manually. Manually prefetching this library may also be desirable for offline CI pipelines and other situations where network access is not desired.
  7. The URL for each target's Ditto SDK is https://software.ditto.live/rust/Ditto/<version>/<target-tripple>/<profile>/<library-name> For example, on an x86_64 MacOS developer machine, one would use https://software.ditto.live/rust/Ditto/1.0.3-alpha1/x86_64-apple-darwin/release/libdittoffi.dylib.
  8. The environment variable DITTOFFI_SEARCH_PATH may be used to manually set a directory to be used to find the binary component of the Ditto SDK. This is especially helpful when cross-compiling for different target architectures.
  9. With the binary component now in place, verify that cargo run executes successfully and doesn't throw a linker error.
  10. On some platforms, similar operations are required for other libraries such as libdbus for Linux Bluetooth Transport and various MacOS Frameworks (ie. CoreFoundation, Security). The cargo documentation provides guidance of various ways to provide these libraries to each platforms linker.

Compiling from libditto from source

If you have access to a copy of the full Ditto source code, you can compile the binary component of the Rust SDK, libdittoffi yourself. The compilation host machine will need 4GB of RAM (or swap file) for linking purposes. If your target machine doesn't have this, you'll need to cross-compile (see section below).

  1. First, clone the Ditto source tree. We will refer to the absolute path to the root of the Ditto source tree as DITTO_ROOT going forward.
  2. Select a target system. This can be the same as your host system, or another target triple (cross-compilation). The full list of supported target systems is available here. Note that you will need a valid C toolchain (eg. CC, ld) for both the host and target system. If you are compiling on Mac OS for Linux, this will require more set up (described in a following section). bash export DITTO_TARGET=x86_64-unknown-linux-gnu
  3. Build libdittoffi for your target platform. This can take a while. bash (cd ffi && cargo build --release --target $DITTO_TARGET)
  4. Confirm your build is successful. You should see libdittoffi.a and libdittoffi.so in $DITTO_ROOT/target/$DITTO_TARGET/release/. This path is known as your TARGET_DIR and is configured by a file .cargo/config.toml in DITTO_ROOT. Each target and profile (release, debug) will get its own sub-directory. Note that on Windows and Mac OS, the library file extensions will be different (.dll and .dylib, respectively).
  5. Optionally, you can now install libdittoffi.so into a cannonical search location for the target's linker to find. When compiling on the execution device, we recommend a symlink to /usr/local/lib or /opt/ditto/lib. This will make it easy for any apps outside of DITTO_ROOT to find libdittoffi.so. bash sudo ln -sf "$DITTO_ROOT/target/$DITTO_TARGET/release/libdittoffi.so" /usr/local/bin
  6. Alternatively, you can specify your TARGETDIR as the DITTOFFISEARCH_PATH that your app will search for this library. bash export DITTOFFI_SEARCH_PATH="$DITTO_ROOT/target/$DITTO_TARGET/release"
  7. Create your new Ditto-powered App. We'll refer to the root of this app as APP_ROOT going forward. bash cargo new myapp
  8. Add the ditto Rust SDK as a dependency by editing your app's Cargo.toml file. This should be the Path to the Rust SDK not libdittoffi.so. Cargo.toml [dependencies] dittolive-ditto.path = "$DITTO_ROOT/rust"
  9. Test building your app from within the your apps root directory. Your app should build dittolive-ditto (the Rust SDK) which in turn will build dittolive-ditto-sys which links to libdittoffi.so or libdittoffi.a. bash cargo build
  10. If you want to force static linking of your app to libdittoffi.a you can set the LIBDITTO_STATIC=1 env var. However, this may result in missing symbol errors on some platforms where shared system libraries are otherwise linked in by default. For example, on linux you may see errors about missing libdbus symbols and on Mac OS Darwin, missing system framework symbols, because these symbols are only available as shared libraries. Your app will need to be configured to tell the linker how to source these symbols. bash LIBDITTO_STATIC=1 cargo build
  11. Finally, to build a release of your app bash cargo build --release --target $DITTO_TARGET
  12. Your final executable can be run as follows. This may be copied to another system, but be sure to also provide copies of libdittoffi.so and any other dynamically linked libraries. bash $APP_ROOT/target/$DITTO_TARGET/release/myapp

How is the Ditto Library located

The build.rs script for dittolive-ditto-sys makes a best effort attempt to locate, and if absent download, libdittoffi in spite of each target OS having distinct linkers and conventions for managing shared libraries. The absolute location of these files depends on whether the SDK is built from source or downloaded from crates.io as a dependency. The search order is as follows:

  1. If DITTOFFI_SEARCH_PATH env var is set to a valid directory, use this. This takes priority over all other methods.
  2. Look in the CARGOBUILDTARGET_DIR (ie. target/$DITTO_TARGET/{debug,release}). This is automatic for Cargo.
  3. Look in the deps folder for the CARGOBUILDTARGET (ie. target/$DITTO_TARGET/{debug,release}/deps). This is automatic for Cargo.
  4. Look in the CARGOMANIFESTDIR.
  5. The current working directory of cargo.
  6. Common POSIX paths defined in the Filesystem Hierarchy Standard: /usr, /lib, /usr/local/lib, /opt.
  7. If on a linux system, try using pkg_config. Note that Raspberry Pi OS does not ship with pkg_config.
  8. If on a windows system, try using vcpkg.
  9. Download pre-built version from Ditto's S3 bucket

The end result of this search is to ensure that two key arguments are passed to the target linker when building your app: * -ldittoffi - Your app should be linked against the ditto library * -L DITTOFFI_SEARCH_PATH - The directory where the ditto library for the target system can be found You can see this process by building with cargo build -vv.

If desired, you can explicitly provide these two arguments using cargo rustc to build your app, or otherwise configuring your apps build system to provide these values when linking. We recommend explicitly specifying the search path for the ditto library, especially when cross-compiling, to ensure the correct DITTO_TARGET architecture is used.

Cross-compiling for IOT devices

Many IOT devices make poor compilation hosts for Rust, and thus it is better to cross-compile on a developer machine with more resources and then execute on the target IOT device.

Method 1 - Use a rustembedded/cross Docker and VS Studio Code "devContainer" extension to build from source

  1. Install docker on your HOST machine
  2. Install Visual Studio Code on your Host Machine
  3. Install the VS Code "Remote - Containers" extension.
  4. Configure your DITTO_TARGET so that you can ssh onto it and copy files using a utility such as scp, sftp, or rsync.
  5. Check out the Ditto source tree in VS Code, and then in the root of the project edit .devcontainer/devontainer.json's build.Dockerfile key to point to the docker image for your DITTO_TARGET. In this case Dockerfile.armv-unknown-linux-gnueabihf.
  6. Use the "Remote - Containers" plug in to "Reload in Container" (the green >< icon in the lower left corner). This will download the image, build it, mount the Ditto repo into /workspaces/ditto, which will be our effective DITTO_ROOT going forward.
  7. Open a terminal in the container and build libdittoffi. The machining target architecture will be automatically selected. bash (cd ffi && cargo build --release)
  8. You may now copy the built artifact to your target device (on your host) bash scp $DITTO_ROOT/target/$DITTO_TARGET/release/libdittoffi.so pi@raspberrypi:/home/pi
  9. SSH onto your Raspberry Pi and Symlink libdittoffi.so into a linker search directory bash sudo ln -sf /home/pi/libdittoffi.so /usr/local/lib/libdittoffi.so
  10. Repeat steps 1 through 8 for your app. If your app lives outside of the DITTO_ROOT, you will need to set up a distinct VS Code project for it. Alternatively, you can compile your app in place on the Raspberry Pi device.
  11. You can verify your app on the target device with the following bash ldd ./myapp This should verify the target devices linker can resolve all the shared libraries (including libdittoffi.so) on the target device

Configuring the RUST SDK

The following env vars are commonly used to configure the Ditto SDK.

Common config patterns

The following are common patterns used to configure test and example apps