Unit 1.1 - Introduction

Slides

On these pages you will find open source Rust training material designed and maintained by the Rust experts working at Tweede golf. It comprises several modules that we use in our multi-day (embedded) training courses.

It is made available to you under the CC-BY-SA 4.0 license, which means you are free to use and share it as long as you give us appropriate credit and share any modifications you make with others under that same license (which also means that we can use it to improve our training material!).

The source for it can be found on our GitHub repository. If you see any mistake, we welcome issues and pull requests!

Let us train you!

Going through this material on your own is good, but what's even better: our experts guiding your through the material and associated exercise during an interactive, tailor-made multi-day program!

To learn more, visit our professional training page.

Unit 1.2 - Introduction

Exercise 1.2.1: Setup Your Installation

In this file you'll find instructions on how to install the tools we'll use during the course.

All of these tools are available for Linux, macOS and Windows users. We'll need the tools to write and compile our Rust code, and allow for remote mentoring. Important: these instructions are to be followed at home, before the start of the first tutorial. If you have any problems with installation, contact the lecturers! We won't be addressing installation problems during the first tutorial.

Rust and Cargo

First we'll need rustc, the standard Rust compiler. rustc is generally not invoked directly, but through cargo, the Rust package manager. rustup takes care of installing rustc and cargo.

This part is easy: go to https://rustup.rs and follow the instructions. Please make sure you're installing the latest default toolchain. Once done, run

rustc -V && cargo -V

The output should be something like this:

rustc 1.79.0 (129f3b996 2024-06-10)
cargo 1.79.0 (ffa9cf99a 2024-06-03)

Using Rustup, you can install Rust toolchains and components. More info:

• https://rust-lang.github.io/rustup
• https://doc.rust-lang.org/cargo

Rustfmt and Clippy

To avoid discussions, Rust provides its own formatting tool, Rustfmt. We'll also be using Clippy, a collection of lints to analyze your code, that catches common mistakes for you. You'll find that Rusts Clippy can be a very helpful companion. Both Rustfmt and Clippy are installed by Rustup by default.

To run Rustfmt on your project, execute:

cargo fmt

To run clippy:

cargo clippy

More info:

• Rustfmt: https://github.com/rust-lang/rustfmt
• Clippy: https://github.com/rust-lang/rust-clippy

Visual Studio Code

During the course, we will use Visual Studio Code (vscode) to write code in. Of course, you're free to use your favorite editor, but if you encounter problems, you can't rely on support from us. Also, we'll use VSCode to allow for remote collaboration and mentoring during remote training sessions.

You can find the installation instructions here: https://code.visualstudio.com/.

We will install some plugins as well. The first one is Rust-Analyzer. Installation instructions can be found here https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer. Rust-Analyzer provides a lot of help during development and in indispensable when getting started with Rust.

Another plugin we'll use is CodeLLDB. This plugin enables debugging Rust code from within vscode. You can find instructions here: https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb.

If you're following the training remotely, install the Live Share plugin as well. We will use the plugin to share code and provide help during remote tutorial sessions. Installation instructions can be found here: https://marketplace.visualstudio.com/items?itemName=MS-vsliveshare.vsliveshare

More info:

• https://rust-analyzer.github.io/
• https://code.visualstudio.com/learn/collaboration/live-share

Git

During the trainings, you'll need the Git version control tool. If you haven't installed Git already, you can find instructions here: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git. If you're new to Git, you'll also appreciate GitHubs intro to Git https://docs.github.com/en/get-started/using-git/about-git and the Git intro with vscode, which you can find here: https://www.youtube.com/watch?v=i_23KUAEtUM.

More info: https://www.youtube.com/playlist?list=PLg7s6cbtAD15G8lNyoaYDuKZSKyJrgwB-

Course code

Now that everything is installed, you can clone the source code repository using Git. The repository can be found here: https://github.com/tweedegolf/rust-training.

Instructions on cloning the repository can be found here: https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls

Trying it out

Now that you've got the code on your machine, navigate to it using your favorite terminal and run:

cd exercises/1-course-introduction/1-introduction/1-setup-your-installation
cargo run

This command may take a while to run the first time, as Cargo will first fetch the crate index from the registry. It will compile and run the intro package, which you can find in exercises/1-course-introduction/1-introduction/1-setup-your-installation. If everything goes well, you should see some output:

   Compiling intro v0.1.0 ([/path/to/rust-workshop]/exercises/1-course-introduction/1-introduction/1-setup-your-installation)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
     Running `target/debug/intro`
🦀 Hello, world! 🦀
You've successfully compiled and run your first Rust project!
X: 2; Y: 2

If Rust-Analyzer is set up correctly, you can also click the '▶️ Run'-button that is shown in exercises/1-course-introduction/1-introduction/1-setup-your-installation/src/main.rs. With CodeLLDB installed correctly, you can also start a debug session by clicking 'Debug', right next to the '▶️ Run'-button. Play a little with setting breakpoints by clicking on a line number, making a red circle appear and stepping over/into/out of functions using the controls. You can view variable values by hovering over them while execution is paused, or by expanding the 'Local' view under 'Variables' in the left panel during a debug session.

Instructions for embedded

This part is relevant only if you're partaking in one of the workshops on embedded Rust.

Hardware

You should have a [BBC micro:bit](https://microbit.org/buy/bbc-microbit-single/ V2) available.

You'll also need a Micro-USB cable, but we're sure you've got one to spare.

Please check that everything is complete. If not, please contact us.

Software

Then, we'll install some tools needed to flash the mcu and inspect the code.

Install the thumbv7em-none-eabihf toolchain with the following command:

rustup target add thumbv7em-none-eabihf

We'll also install a couple of tools that let us inspect our binaries:

rustup component add llvm-tools
cargo install cargo-binutils

Now, let's install probe-rs. Follow the installation instructions. Probe-rs talks with the debug interface on the micro:bit, to flash your application, log messages, or even set breakpoints and read out device memory.

If you're on linux, you'll need to update your udev rules. On ubuntu or fedora, run the following inside the workshop folder you just cloned;

sudo cp 99-microbit-v2.rules /etc/udev/rules.d
sudo udevadm control --reload-rules
sudo udevadm trigger

If you're on windows, we need to install a generic WinUSB driver. You can use Zadig to select the usb device that uses the jlink driver and install WinUSB on it. This will uninstall the official driver, which means that the official Segger tools will not work anymore after this. To revert, go to device manager and uninstall the usb device. The jlink driver will then be used again for that usb connection.

Then, unplug the USB cable and plug it in again.

Trying it out

Before we begin, we need to test our hardware. We'll be testing the nRF52833 microcontroller and the LSM303AGR motion sensor, that are present on the micro:bit V2. Make sure you have checked out the latest version of the workshop source.

Running the test

To test the hardware, please connect the micro:bit V2 to your pc, switch it on, and run

cd ./exercises/1-course-introduction/1-introduction/2-embedded
cargo run --release

If everything works correctly, you should now see the accelerometer samples being printed on the display. If not, don't worry and contact us.

Docs

Datasheets, manuals, and schematics of the parts we are using in the embedded workshops.

BBC micro:bit V2

• Hardware doc
• Schematic

nRF52833

• nRF52833 product specification

LSM303AGR

• Datsheet

Unit 2.1 - Basic Syntax

Slides

Exercise 2.1.1: Basic Syntax

Open exercises/2-foundations-of-rust/1-basic-syntax/1-basic-syntax in your editor. This folder contains a number of exercises with which you can practise basic Rust syntax.

While inside the exercises/2-foundations-of-rust/1-basic-syntax/1-basic-syntax folder, to get started, run:

cargo run --bin 01

This will try to compile exercise 1. Try and get the example to run, and continue on with the next exercise by replacing the number of the exercise in the cargo run command.

Some exercises contain unit tests. To run the test in src/bin/01.rs, run

cargo test --bin 01

Make sure all tests pass!

Unit 2.2 - Ownership and References

Slides

Exercise 2.2.1: Move Semantics

This exercise is adapted from the move semantics exercise from Rustlings

While inside the exercises/2-foundations-of-rust/2-ownership-and-references/1-move-semantics folder, to get started, run:

cargo run --bin 01

This will try to compile exercise 1. Try and get the example to run, and continue on with the next exercise by replacing the number of the exercise in the cargo run command.

Some exercises contain unit tests. To run the test in src/bin/01.rs, run

cargo test --bin 01

Make sure all tests pass!

01.rs should compile as is, but you'll have to make sure the others compile as well. For some exercises, instructions are included as doc comments at the top of the file. Make sure to adhere to them.

Exercise 2.2.2: Borrowing

Fix the two examples in the exercises/2-foundations-of-rust/2-ownership-and-references/2-borrowing crate! Don't forget you can run individual binaries by using cargo run --bin 01 in that directory! Make sure to follow the instructions that are in the comments!

Unit 2.3 - Advanced Syntax

Slides

Exercise 2.3.1: Error propagation

Follow the instructions in the comments of exercises/2-foundations-of-rust/3-advanced-syntax/1-error-propagation/src/main.rs!

Exercise 2.3.2: Error handling

Follow the instructions in the comments of exercises/2-foundations-of-rust/3-advanced-syntax/2-error-handling/src/main.rs!

Exercise 2.3.3: Slices

Follow the instructions in the comments of exercises/2-foundations-of-rust/3-advanced-syntax/3-slices/src/main.rs! Don't take too much time on the extra assignment, instead come back later once you've done the rest of the excercises.

Exercise 2.3.4: Ring Buffer

This is a bonus exercise! Follow the instructions in the comments of exercises/2-foundations-of-rust/3-advanced-syntax/4-ring-buffer/src/main.rs!

Exercise 2.3.5: Boxed Data

Follow the instructions in the comments of exercises/2-foundations-of-rust/3-advanced-syntax/5-boxed-data/src/main.rs!

Unit 2.4 - Traits and Generics

Slides

Exercise 2.4.1: Local Storage Vec

In this exercise, we'll create a type called LocalStorageVec, which is generic list of items that resides either on the stack or the heap, depending on its size. If its size is small enough for items to be put on the stack, the LocalStorageVec buffer is backed by an array. LocalStorageVec is not only generic over the type (T) of items in the list, but also by the size (N) of this stack-located array using a relatively new feature called "const generics". Once the LocalStorageVec contains more items than fit in the array, a heap based Vec is allocated as space for the items to reside in.

Within this exercise, the objectives are annotated with a number of stars (⭐), indicating the difficulty. You are likely not to be able to finish all exercises during the tutorial session

Questions

1. When is such a data structure more efficient than a standard Vec?
2. What are the downsides, compared to just using a Vec?

Open the exercises/2-foundations-of-rust/4-traits-and-generics/1-local-storage-vec crate. It contains a src/lib.rs file, meaning this crate is a library. lib.rs contains a number of tests, which can be run by calling cargo test. Don't worry if they don't pass or even compile right now: it's your job to fix that in this exercise. Most of the tests are commented out right now, to enable a step-by-step approach. Before you begin, have a look at the code and the comments in there, they contain various helpful clues.

2.4.1.A Defining the type ⭐

Currently, the LocalStorageVec enum is incomplete. Give it two variants: Stack and Heap. Stack contains two named fields, buf and len. buf will be the array with a capacity to hold N items of type T; len is a field of type usize that will denote the amount of items actually stored. The Heap variant has an unnamed field containing a Vec<T>. If you've defined the LocalStorageVec variants correctly, running cargo test should output something like

running 1 test
test test::it_compiles ... ignored, This test is just to validate the definition of `LocalStorageVec`. If it compiles, all is OK

test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s

This test does (and should) not run, but is just there for checking your variant definition.

#![allow(unused)]
fn main() {
// Define an enum `LocalStorageVec` that is generic over
// type `T` and a constant `N` of type `usize`
pub enum LocalStorageVec<T, const N: usize> {
    // Define a struct-like variant called `Stack` containing two named fields:
    // - `buf` is an array with elements of `T` of size `N`
    // - `len` is a field of type `usize`
    Stack { buf: [T; N], len: usize },
    // Define a tuple-like variant called `Heap`, containing a single field
    // of type `Vec<T>`, which is a heap-based growable, contiguous list of `T`
    Heap(Vec<T>),
}
}

2.4.1.B impl-ing From<Vec<T>> ⭐

Uncomment the test it_from_vecs, and add an implementation for From<Vec<T>> to LocalStorageVec<T>. To do so, copy the following code in your lib.rs file and replace the todo! macro invocation with your code that creates a heap-based LocalStorageVec containing the passed Vec<T>.

#![allow(unused)]
fn main() {
impl<T, const N: usize> From<Vec<T>> for LocalStorageVec<T, N> {
    fn from(v: Vec<T>) -> Self {
        todo!("Implement me");
    }
}
}

Question

1. How would you pronounce the first line of the code you just copied in English?*

Run cargo test to validate your implementation.

2.4.1.C impl LocalStorageVec ⭐⭐

To make the LocalStorageVec more useful, we'll add more methods to it. Create an impl-block for LocalStorageVec. Don't forget to declare and provide the generic parameters. For now, to make implementations easier, we will add a bound T, requiring that it implements Copy and Default. First off, uncomment the test called it_constructs. Make it compile and pass by creating a associated function called new on LocalStorageVec that creates a new, empty LocalStorageVec instance without heap allocation.

The next methods we'll implement are len, push, pop, insert, remove and clear:

• len returns the length of the LocalStorageVec
• push appends an item to the end of the LocalStorageVec and increments its length. Possibly moves the contents to the heap if they no longer fit on the stack.
• pop removes an item from the end of the LocalStorageVec, optionally returns it and decrements its length. If the length is 0, pop returns None
• insert inserts an item at the given index and increments the length of the LocalStorageVec
• remove removes an item at the given index and returns it.
• clear resets the length of the LocalStorageVec to 0.

Uncomment the corresponding test cases and make them compile and pass. Be sure to have a look at the methods provided for slices [T] and Vec<T> Specifically, [T]::copy_within and Vec::extend_from_slice can be of use.

2.4.1.E Iterator and IntoIterator ⭐⭐

Our LocalStorageVec can be used in the real world now, but we still shouldn't be satisfied. There are various traits in the standard library that we can implement for our LocalStorageVec that would make users of our crate happy.

First off, we will implement the IntoIterator and Iterator traits. Go ahead and uncomment the it_iters test case. Let's define a new type:

#![allow(unused)]
fn main() {
pub struct LocalStorageVecIter<T, const N: usize> {
    vec: LocalStorageVec<T, N>,
    counter: usize,
}
}

This is the type we'll implement the Iterator trait on. You'll need to specify the item this Iterator implementation yields, as well as an implementation for Iterator::next, which yields the next item. You'll be able to make this easier by bounding T to Default when implementing the Iterator trait, as then you can use the std::mem::take function to take an item from the LocalStorageVec and replace it with the default value for T.

Take a look at the list of methods under the 'provided methods' section. In there, lots of useful methods that come free with the implementation of the Iterator trait are defined, and implemented in terms of the next method. Knowing in the back of your head what methods there are, greatly helps in improving your efficiency in programming with Rust. Which of the provided methods can you override in order to make the implementation of LocalStorageVecIter more efficient, given that we can access the fields and methods of LocalStorageVec?

Now to instantiate a LocalStorageVecIter, implement the [IntoIter] trait for it, in such a way that calling into_iter yields a LocalStorageVecIter.

2.4.1.F Index ⭐⭐

To allow users of the LocalStorageVec to read items or slices from its buffer, we can implement the Index trait. This trait is generic over the type of the item used for indexing. In order to make our LocalStorageVec versatile, we should implement:

• Index<usize>, allowing us to get a single item by calling vec[1];
• Index<RangeTo<usize>>, allowing us to get the first n items (excluding item n) by calling vec[..n];
• Index<RangeFrom<usize>>, allowing us to get the last n items by calling vec[n..];
• Index<Range<usize>>, allowing us to get the items between n and m items (excluding item m) by calling vec[n..m];

Each of these implementations can be implemented in terms of the as_ref implementation, as slices [T] all support indexing by the previous types. That is, [T] also implements Index for those types. Uncomment the it_indexes test case and run cargo test in order to validate your implementation.

2.4.1.G Removing bounds ⭐⭐

When we implemented the borrowing Iterator, we saw that it's possible to define methods in separate impl blocks with different type bounds. Some of the functionality you wrote used the assumption that T is both Copy and Default. However, this means that each of those methods are only defined for LocalStorageVecs containing items of type T that in fact do implement Copy and Default, which is not ideal. How many methods can you rewrite having one or both of these bounds removed?

2.4.1.H Borrowing Iterator ⭐⭐⭐

We've already got an iterator for LocalStorageVec, though it has the limitation that in order to construct it, the LocalStorageVec needs to be consumed. What if we only want to iterate over the items, and not consume them? We will need another iterator type, one that contains an immutable reference to the LocalStorageVec and that will thus need a lifetime annotation. Add a method called iter to LocalStorageVec that takes a shared &self reference, and instantiates the borrowing iterator. Implement the Iterator trait with the appropriate Item reference type for your borrowing iterator. To validate your code, uncomment and run the it_borrowing_iters test case.

Note that this time, the test won't compile if you require the items of LocalStorageVec be Copy! That means you'll have to define LocalStorageVec::iter in a new impl block that does not put this bound on T:

#![allow(unused)]
fn main() {
impl<T: Default + Copy, const N: usize> LocalStorageVec<T, N> {
    // Methods you've implemented so far
}

impl<T: const N: usize> LocalStorageVec<T, N> {
    pub fn iter(&self) -> /* TODO */
}
}

Defining methods in separate impl blocks means some methods are not available for certain instances of the generic type. In our case, the new method is only available for LocalStorageVecs containing items of type T that implement both Copy and Default, but iter is available for all LocalStorageVecs.

2.4.1.I Generic Index ⭐⭐⭐⭐

You've probably duplicated a lot of code in exercise 2.4.1.F. We can reduce the boilerplate by defining an empty trait:

#![allow(unused)]
fn main() {
trait LocalStorageVecIndex {}
}

First, implement this trait for usize, RangeTo<usize>, RangeFrom<usize>, and Range<usize>.

Next, replace the multiple implementations of Index with a single implementation. In English:

"For each type T, I and constant N of type usize, implement Index<I> for LocalStorageVec<T, N>, where I implements LocalStorageVecIndex and [T] implements Index<I>"

If you've done this correctly, it_indexes should again compile and pass.

2.4.1.J Deref and DerefMut ⭐⭐⭐⭐

The next trait that makes our LocalStorageVec more flexible in use are Deref and DerefMut that utilize the 'deref coercion' feature of Rust to allow types to be treated as if they were some type they look like. That would allow us to use any method that is defined on [T] by calling them on a LocalStorageVec. Before continuing, read the section 'Treating a Type Like a Reference by Implementing the Deref Trait' from The Rust Programming Language (TRPL). Don't confuse deref coercion with any kind of inheritance! Using Deref and DerefMut for inheritance is frowned upon in Rust.

Below, an implementation of Deref and DerefMut is provided in terms of the AsRef and AsMut implementations. Notice the specific way in which as_ref and as_mut are called.

#![allow(unused)]
fn main() {
impl<T, const N: usize> Deref for LocalStorageVec<T, N> {
    type Target = [T];

    fn deref(&self) -> &Self::Target {
        <Self as AsRef<[T]>>::as_ref(self)
    }
}

impl<T, const N: usize> DerefMut for LocalStorageVec<T, N> {
    fn deref_mut(&mut self) -> &mut Self::Target {
        <Self as AsMut<[T]>>::as_mut(self)
    }
}
}

Question

• Replacing the implementation of deref with self.as_ref() results in a stack overflow when running an unoptimized version. Why? (Hint: deref coercion)

Unit 2.5 - Closures and Dynamic dispatch

Slides

Exercise 2.5.1: Config Reader

In this exercise, you'll work with dynamic dispatch to deserialize with serde_json or serde_yaml, depending on the file extension. The starter code is in exercises/2-foundations-of-rust/5-closures-and-dynamic-dispatch/1-config-reader. Fix the todo's in there.

To run the program, you'll need to pass the file to deserialize to the binary, not to Cargo. To do this, run

cargo run -- <FILE_PATH>

Deserializing both config.json and config.yml should result in the Config being printed correctly.

Unit 3.1 - Crate Engineering

Slides

Exercise 3.1.1: My Serde App

This exercise is adapted from the serde_lifetimes exercise by Ferrous Systems

Open exercises/3-crate-engineering/1-crate-engineering/1-my-serde-app/src/main.rs. In there, you'll find some Rust code we will do this exercise with.

We used todo!() macros to mark places where you should put code to make the program run. Look at the serde_json api for help.

How come main returns an anyhow::Result<()>? By having main return a result, we can bubble errors up all the way to runtime. You can find more information about it in Rust By Example. The anyhow::Result is a more flexible type of Result, which allows for easy conversion of error types.

What is that r#"... thing?
r in front of a string literal means it's a "raw" string. Escape sequences (\n, \", etc.) don't work, and thus they are very convenient for things like regular expressions, JSON literals, etc.

Optionally r can be followed by one or more symbols (like # in our case), and then your string ends when there's a closing double quote followed by the same number of the same symbols. This is great for cases when you want to have double quotes inside your string literal. For our example r#" ... "# works great for JSON. In rare cases you'd want to put two or more pound signs. Like, when you store CSS color values in your JSON strings:

#![allow(unused)]
fn main() {
// here `"#` would not terminate the string
r##"
    {
        "color": "#ff00ff"
    }
"##
}

Exercise 3.1.2: Quizzer

In this exercise, you will create a Rust crate that adheres to the guidelines that were pointed out during the lecture. Additionally, you will add and use dependencies, create unit tests, and create some documentation. You can view this exercise as a stepping stone to the final project.

This exercise should be done in groups of 2 people

3.1.2.A Setting up ⭐

Create a new project using cargo new --name quizzer. Make sure it acts as both a binary and a library. That means there will be both a src/lib.rs and a src/bin/quizzer/main.rs file in your crate, where quizzer is the name of the binary:

$ tree
.
├── Cargo.toml
├── quiz.json
└── src
    ├── bin
    │   └── quizzer
    │       └── main.rs
    └── lib.rs

Add the following dependencies to your Cargo.toml file. Below items contain links to their page on lib.rs. Make sure you get a general idea of what these crates are for and how they can be used. Don't dive too deep just yet.

• anyhow 1.0
• clap 4.0 Also, skim over https://docs.rs/clap/latest/clap/_derive/_tutorial/index.html
• serde-json 1.0
• serde 1.0

Your Cargo.toml should look like this:

[package]
name = "quizzer"
version = "0.1.0"
edition = "2021"

### See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
anyhow = "1.0.66"
clap = { version = "4.0.18", features = ["derive"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0.87"

For clap and serde, the non-standard derive feature of each these crates is enabled. For clap, it allows us to derive the Parser trait, which greatly simplifies creating a CLI. The derive feaure from serde allows us to derive the Serialize and Deserialize traits on any struct we wish to serialize or deserialize using serde and its backends, in our case serde_json.

3.1.2.B Quizzer ⭐⭐⭐

This exercise is about both design and finding information. You'll have to figure out a model to represent your quiz questions, as well as a means to store them into a JSON file, and load them yourself. Also, you will have to find out how to parse the program arguments.

We will use the project we just set up to write a quiz game creator and player. You may add other dependencies as needed. It has the following functional requirements:

• It runs as a command-line tool in your terminal.
• It has two modes: question-entering mode and quiz mode. The mode is selected with a subcommand, passed as the first argument to the program.
  • Question-entering mode: Allows for entering multiple-choice quiz questions, with 4 possible answers each, exactly 1 of them being correct. The questions are stored on disk as a JSON file.
  • Quiz mode: Loads stored questions from the JSON file, presents the questions one-by-one to the player, reads and verifies the player input, and presents the score at the end of the game.
• Errors are correctly handled, i.e. your application does not panic if it encounters any unexpected situation. Use anywhow and the question-mark (?) operator to make error-bubbling concise. You can read about the ?-operator here: https://doc.rust-lang.org/reference/expressions/operator-expr.html#the-question-mark-operator
• Logic concerning creating, storing, and loading quiz questions is defined in the library part of your crate.
• Functionality regarding user input (arg parsing, reading from stdin) is defined in the application code, not in your library.
• Logical units of your crate are divided up into modules.

Before you start coding, make sure you've listed all open questions and found answers to them. You're also encouraged to draw a simple diagram of the module structure of your application, annotating each module with its responsibilities.

Exercise 3.1.3: BSN

The BSN (Burgerservicennummer) is a Dutch personal identification number that somewhat resembles the US Social Security Number in its use. The BSN is a number that adheres to some rules. In this exercise, we will create a Rust type that guarantees that it represents a valid BSN.

3.1.3.A Newtype ⭐⭐

In this part we will implement the BSN number validation, as well as a fallible constructor.

A BSN is valid if and only if it matches the following criteria:

• It consists of 8 or 9 digits
• It passes a variant of the 11 check (elfproef (Dutch)):

For 8-digit BSNs, we concatenate a 0 to the end. The digits of the number are labeled as ABCDEFGHI. For example: for BSN 123456789, A = 1, B = 2, C = 3, and so forth until I.

Then, (9 × A) + (8 × B) + (7 × C) + (6 × D) + (5 × E) + (4 × F) + (3 × G) + (2 × H) + (-1 × I) must be a multiple of 11

Open exercises/3-crate-engineering/1-crate-engineering/3-bsn in your editor. You'll find the scaffolding code there, along with two files:

• valid_bsns.in containing a list of valid BSNs
• invalid_bsns.in containing a list of invalid BSNs.

In src/lib.rs, implement Bsn::validate to make the test_validation test case pass. Implement Bsn::try_from_string as well. To try just the test_validation test case, run:

cargo test -- test_validation

3.1.3.B Visitor with Serde ⭐⭐⭐

Next up is implementing the serde::Serialize and serde::Deserialize traits, to support serialization and deserialization of Bsns. In this case, simply deriving those traits won't suffice, as we want to represent the BSN as a string after serialization. We also want to deserialize strings directly into Bsns, while still upholding the guarantee that an instantiated Bsn represents a valid BSN. Therefore, you have to incorporate Bsn::validate into the implementation of the deserialization visitor.

More information on implementing the traits:

• serde::Serialize: https://serde.rs/impl-serialize.html
• serde::Deserialize: https://serde.rs/impl-deserialize.html

If everything works out, all tests should pass.

Exercise 3.1.4: 3D Printer

An imaginary 3D printer uses filament to create all kinds of things. Its states can be represented with the following state diagram:

                   ┌─────────────────┐
                   │                 │
                   │                 │   Reset
                   │      Idle       │◄────────────────────────────┐
         ┌────────►│                 │                             │
         │         │                 │                             │
         │         │                 │                             │
         │         └────────┬────────┘                             │
         │                  │                                      │
         │                  │                                      │
         │                  │ Start                                │
         │                  │                                      │
         │                  ▼                                      │
         │         ┌─────────────────┐                    ┌────────┴────────┐
         │         │                 │                    │                 │
         │         │                 │   Out of filament  │                 │
Product  │         │    Printing     ├──────────────────► │      Error      │
retrieved│         │                 │                    │                 │
         │         │                 │                    │                 │
         │         │                 │                    │                 │
         │         └────────┬────────┘                    └─────────────────┘
         │                  │
         │                  │ Product ready
         │                  │
         │                  ▼
         │         ┌─────────────────┐
         │         │                 │
         │         │                 │
         │         │  Product Ready  │
         └─────────┤                 │
                   │                 │
                   │                 │
                   └─────────────────┘

The printer boots in Idle state. Once a job is started, the printer enters the Printing state. In printing state, it keeps on printing the product until either it is ready or the printer is out of filament. If the printer is out of filament, the printer goes into Error state, which it can only come out of upon device reset. If the product is ready, the printer goes to Product Ready state, and once the user retrieves the product, the printer goes back to Idle.

The printer can be represented in Rust using the typestate pattern as described during the lecture. This allows you to write a simple 3D printer driver. In exercises/3-crate-engineering/1-crate-engineering/4-3d-printer, a Printer3D struct is instantiated. Add methods corresponding to each of the traits, that simulate the state transitions by printing the state. A method simulating checking if the printer is out of filament is provided.

Of course, to make the printer more realistic, you can add more states and transitions.

Exercise 3.1.5: FizzBuzz

In this exercise, you will practise writing a unit test, and use Rusts benchmarking functionality to help you optimize a FizzBuzz app. You will need cargo-criterion, a tool that runs benchmarks and creates nice reports. You can install it by running

cargo install cargo-criterion --version=1.1.0

3.1.5.A Testing Fizz Buzz ⭐

Open exercises/3-crate-engineering/1-crate-engineering/5-fizzbuzz/src/lib.rs. Create a unit test that verifies the correctness of the fizz_buzz function. You can use the include_str macro to include exercises/3-crate-engineering/1-crate-engineering/5-fizzbuzz/fizzbuzz.out as a &str into your binary. Each line of fizzbuzz.out contains the expected output of the fizz_buzz function given the line number as input. You can run the test with

cargo test

By default, Rusts test harness captures all output and discards it, If you like to debug your test code using print statements, you can run

cargo test -- --nocapture

to prevent the harness from capturing output.

3.1.5.B Benchmarking Fizz Buzz ⭐⭐

You'll probably have noticed the fizz_buzz implementation is not very optimized. We will use criterion to help us benchmark fizz_buzz. To run a benchmark, run the following command when in the exercises/3-crate-engineering/1-crate-engineering/5-fizzbuzz/ directory:

cargo criterion

This command will run the benchmarks, and report some statistics to your terminal. It also generates HTML reports including graphs that you can find under target/criterion/reports. For instance, target/criterion/reports/index.html is a summary of all benchmark. Open it with your browser and have a look.

Your job is to do some optimization of the fizz_buzz function, and use cargo-criterion to measure the impact of your changes. Don't be afraid to change the signature of fizz_buzz, if, for instance, you want to minimize the number of allocations done by this function. However, make sure that the function is able to correctly produce the output. How fast can you FizzBuzz?

Unit 4.1 - Introduction to Multitasking

Slides

There are no exercises for this unit

Unit 4.2 - Parallel Multitasking

Slides

Exercise 4.2.1: TF-IDF

Follow the instructions in the comments of exercises/4-multitasking/2-parallel-multitasking/1-tf-idf/src/main.rs!

Exercise 4.2.2: Mutex

The basic mutex performs a spin-loop while waiting to take the lock. That is terribly inefficient. Luckily, your operating system is able to wait until the lock becomes available, and will just put the thread to sleep in the meantime.

This functionality is exposed in the atomic_wait crate. The section on implementing a mutex from "Rust Atomics and Locks" explains how to use it.

• change the AtomicBool for a AtomicU32
• implement lock. Be careful about spurious wakes: after wait returns, you must stil check the condition
• implement unlocking (Drop for MutexGuard<T> using wake_one.

The linked chapter goes on to further optimize the mutex. This is technically out of scope for this course, but we won't stop you if you try (and will still try to help if you get stuck)!

Unit 4.3 - Asynchronous Multitasking

Slides

Exercise 4.3.1: Async Channels

Channels are a very useful way to communicate between threads and async tasks. They allow for decoupling your application into many tasks. You'll see how that can come in nicely in exercise E.2. In this exercise, you'll implement two variants: a oneshot channel and a multi-producer-single-consumer (MPSC) channel. If you're up for a challenge, you can write a broadcast channel as well.

4.3.1.A MPSC channel ⭐⭐

A multi-producer-single-consumer (MPSC) channel is a channel that allows for multiple Senders to send many messages to a single Receiver.

Open exercises/4-multitasking/3-asynchronous-multitasking/1-async-channels in your editor. You'll find the scaffolding code there. For part A, you'll work in src/mpsc.rs. Fix the todo!s in that file in order to make the test pass. To test, run:

cargo test -- mpsc

If your tests are stuck, probably either your implementation does not use the Waker correctly, or it returns Poll::Pending where it shouldn't.

4.3.1.B Oneshot channel ⭐⭐⭐

A oneshot is a channel that allows for one Sender to send exactly one message to a single Receiver.

For part B, you'll work in src/broadcast.rs. This time, you'll have to do more yourself. Intended behavior:

• Receiver implements Future. It returns Poll::Ready(Ok(T)) if inner.data is Some(T), Poll::Pending if inner.data is None, and Poll::Ready(Err(Error::SenderDropped)) if the Sender was dropped.
• Receiver::poll replaces inner.waker with the one from the Context.
• Sender consumes self on send, allowing the it to be used no more than once. Sending sets inner.data to Some(T). It returns Err(Error::ReceiverDropped(T)) if the Receiver was dropped before sending.
• Sender::send wakes inner.waker after putting the data in inner.data
• Once the Sender is dropped, it marks itself dropped with inner
• Once the Receiver is dropped, it marks itself dropped with inner
• Upon succesfully sending the message, the consumed Sender is not marked as dropped. Instead std::mem::forget is used to avoid running the destructor.

To test, run:

cargo test -- broadcast

4.3.1.C Broadcast channel (bonus) ⭐⭐⭐⭐

A Broadcast channel is a channel that supports multiple senders and receivers. Each message that is sent by any of the senders, is received by every receiver. Therefore, the implemenentation has to hold on to messages until they have been sent to every receiver that has not yet been dropped. This furthermore implies that the message shoud be cloned upon broadcasting.

For this bonus exercise, we provide no scaffolding. Take your inspiration from the mpsc and oneshot modules, and implement a broadcast module yourself.

Exercise 4.3.2: Async Chat

In this exercise, you'll write a simple chat server and client based on Tokio. Open exercises/4-multitasking/3-asynchronous-multitasking/2-async-chat in your editor. The project contains a lib.rs file, in which a type Message resides. This Message defines the data the chat server and clients use to communicate.

4.3.2.A Server ⭐⭐⭐

The chat server, which resides in src/bin/server.rs listens for incoming TCP connections on port 8000, and spawns two tasks (futures):

• handle_incoming: reads lines coming in from the TCP connection. It reads the username the client provides, and broadcasts incoming Messages, possibly after some modification.
• handle_outgoing: sends messages that were broadcasted by the handle_incoming tasks to the client over TCP.

Both handle_incoming and handle_outgoing contain a number to todos. Fix them.

To start the server, run

cargo run --bin server

4.3.2.B Client ⭐⭐

The chat client, residing in src/bin/client.rs contains some todo's as well. Fix them to allow for registration and sending Messages to the server.

To start the client, run

cargo run --bin client

If everything works well, you should be able to run multiple clients and see messages sent from each client in every other.

Unit 5.1 - Rust for Web

Slides

Exercise 5.1.1: Pastebin

This exercise is about writing a simple pastebin web server. Like the quizzer app, you will need to set up the project yourself. This webserver will be powered by axum.

• Data is kept in memory. Bonus if you use a database or sqlite, but first make the app function properly without.
• Expose a route to which a POST request can be sent, that accepts some plain text, and stores it along with a freshly generated UUID. The UUID is sent in the response. You can use the uuid crate to generate UUIDs.
• Expose a route to which a GET request can be sent, that accepts a UUID and returns the plain text corresponding to the UUID, or a 404 error if it doesn't exist.
• Expose a route to which a DELETE request can be sent, that accepts a UUID and deletes the plain text corresonding to that UUID.

Unit 6.1 - Foreign Function Interface

Slides

Exercise 6.1.1: CRC in C

Use a CRC checksum function written in C in a Rust program

prerequisites

• A C compiler

Steps

1. Add the cc build dependency, by adding to Crate.toml the lines:

   [build-dependencies]
   cc = "1.0"
   

2. Create build.rs with contents

   extern crate cc;
   
   fn main() {
       println!("cargo:rerun-if-changed=crc32.h");
       println!("cargo:rerun-if-changed=crc32.c");
       cc::Build::new().file("crc32.c").compile("crc32.a");
   }

   This will find your c code, compile it, and link it into the executable rust produces

3. In main.rs, define an extern (fill in the argument and return types)

   #![allow(unused)]
   fn main() {
   extern "C" {
       fn CRC32( ... ) -> ...; // hint: https://doc.rust-lang.org/std/os/raw
   }
   }

4. Now, create a rust wrapper that calls the extern function

   #![allow(unused)]
   fn main() {
   fn crc32( ... ) -> ... { 
       ... // (hints: `unsafe`, `.as_ptr()`, `.len()`)
   }
   }

5. Call our wrapper on some example input

   fn main() {
       println!("{:#x}", crc32(b"12345678"));
   }

   In the above example, the correct output is 0x9ae0daaf

Exercise 6.1.2: CRC in Rust

Use a CRC checksum function written in Rust in a C program

Requirements

• A C compiler

Steps

1. Change Cargo.toml to

   [package]
   name = "crc-in-rust"
   version = "0.1.0"
   edition = "2021"
   
   [lib]
   name = "crc_in_rust"
   crate-type = ["dylib"]
   
   # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
   
   [dependencies]
   

2. Expose an extern rust function

   #![allow(unused)]
   fn main() {
   #[no_mangle]
   pub extern "C" fn crc32(...) -> ... {
   
       ...
   
       crc32_rust(...)
   }
   }

3. Create a C header file crc_in_rust.h

   #include <inttypes.h> // uint32_t, uint8_t
   #include <stddef.h> // size_t
   
   uint32_t crc32(const uint8_t data[], size_t data_length);
   

4. Use the rust crc32 function in C

   #include <inttypes.h> // uint32_t, uint8_t
   #include <stddef.h> // size_t
   #include <stdio.h> // printf
   #include "crc_in_rust.h"
   
   int main() { 
       uint8_t data[] = { 0,1,2,3,4,5,6 };
       size_t data_length = 7;
   
       uint32_t hash = crc32(data, data_length);
   
       printf("Hash: 0x%d\n", hash);
   
       return 0;
   }
   

5. compile and run

   $ clang main.c target/debug/libcrc_in_rust.so -omain
   $ ./main
   Hash: -1386739207
   

Exercise 6.1.3: TweetNaCl Bindgen

Use cargo bindgen to generate the FFI bindings. Bindgen will look at a C header file, and generate rust functions, types and constants based on the C definitions.

But the generated code is ugly and non-idiomatic. To wrap a C library properly, good API design and documentation is needed.

tweetnacl-bindgen

Making rust bindings for the tweetnacl C library

Exercise: implement crypto_hash_sha256_tweet

Below you find instructions for using bindgen and wrapping crypto_hash_sha512_tweet. Follow the instructions, then repeat the steps for crypto_hash_sha256_tweet

Instructions

Prerequisites:

• a C compiler is installed on the system
• bindgen, install with cargo install bindgen-cli

Steps

1. Create the rust bindings: bindgen tweetnacl.h -o src/bindings.rs

2. Use build.rs to compile and link tweetnacl.c. Create build.rs and insert

   fn main() {
       cc::Build::new()
           .file("tweetnacl.c")
           .compile("tweetnacl");   // outputs `libtweetnacl.a`
   }

   And add this section to your Cargo.toml

   [build-dependencies]
   cc = "1"
   

3. Create src/lib.rs with the contents pub mod bindings;. This will make the bindings module available in main.rs.

4. Run cargo check to verify everything is compiling correctly.

5. By default building will generate a bunch of warnings. we can turn those off by replacing our build.rs with

   fn main() {
       cc::Build::new()
           .warnings(false)
           .extra_warnings(false)
           .file("tweetnacl.c")
           .compile("tweetnacl"); // outputs `libtweetnacl.a`
   }

   and adding this line at the top of src/bindings.rs:

   #![allow(unused)]
   #![allow(non_upper_case_globals)]
   fn main() {
   }

Inspecting our bindings

In the generated bindings.rs file we find this signature for the crypto_hash_sha512_tweet C function from tweetNaCl:

#![allow(unused)]
fn main() {
extern "C" {
    pub fn crypto_hash_sha512_tweet(
        arg1: *mut ::std::os::raw::c_uchar,
        arg2: *const ::std::os::raw::c_uchar,
        arg3: ::std::os::raw::c_ulonglong,
    ) -> ::std::os::raw::c_int;
}
}

Some observations

• The definition is inside of an extern "C" block, and has no body. Therefore this function is marked as an extern, and rust expects it to be linked in.
• The function is marked pub, meaning we can import and use it in other modules (like main.rs in our case)
• We can deduce the behavior from the type signature:
  • arg1 is the output: a mutable pointer to a sequence of bytes
  • arg2 is the input: a constant pointer to a sequence of bytes
  • arg3 is a length (unclear of what)
  • the return value is probably an error code
• These are raw C types, which makes it a hassle to call directly from rust.

We will deal with the last point by writing some nice rust wrappers around the generated bindings.

In rust we bundle a pointer to a sequence of elements and its length in a slice. We could write the signature of our own rust wrapper function as:

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(out: &mut [u8], data: &[u8]) -> i32 {
    todo!()
}
}

Modelling with types

But by looking at the tweetNaCl source code we can see that the contract is a bit stronger:

• the output is always 64 bytes wide (64 * 8 = 512)
• we only ever return 0

int crypto_hash(u8 *out,const u8 *m,u64 n)
{
  u8 h[64],x[256];
  u64 i,b = n;

  FOR(i,64) h[i] = iv[i];

  crypto_hashblocks(h,m,n);
  m += n;
  n &= 127;
  m -= n;

  FOR(i,256) x[i] = 0;
  FOR(i,n) x[i] = m[i];
  x[n] = 128;

  n = 256-128*(n<112);
  x[n-9] = b >> 61;
  ts64(x+n-8,b<<3);
  crypto_hashblocks(h,x,n);

  FOR(i,64) out[i] = h[i];

  return 0;
}

The rust type system can model these invariants: We can explicitly make the output 64 elements long by using a reference to an array. Furthermore we can drop the return type if there is nothing useful to return.

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(out: &mut [u8; 64], data: &[u8]) {
    todo!()
}
}

But even better, we can return the output array directly:

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] {
    todo!()
}
}

The compiler will turn this signature into the one we had before under the hood. Returning the value is more idiomatic and convenient in rust, and with modern compilers there is no performance penalty.

In detail: The C ABI mandates that any return value larger than those that fit in a register (typically 128 bits nowadays) are allocated on the caller's stack. The first argument to the function is the pointer to write the result into. LLVM, the backend used by the rust compiler has specific optimizations to make sure the function result is written directly into this pointer.

Writing our implementation

Allright, with the signature worked out, we can write the actual implementation.

We can reach the bindings from main.rs with e.g.

#![allow(unused)]
fn main() {
tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(a,b,c);
}

Here tweetnacl_bindgen is the name of the project, specified in the package section of the Cargo.toml

[package]
name = "tweetnacl-bindgen"

Then bindings is the module name (the file src/bindings.rs is implicitly also a module) and finally crypto_hash_sha512_tweet is the function name from the original C library.

On to the implmentation. Extern functions are considered unsafe in rust, so we will need an unsafe block to call ours.

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] {
    unsafe {
        tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(
            todo!(),
            todo!(),
            todo!(),
        );
    }
}
}

Next we can pass our argument: we turn the slice into a pointer with .as_ptr(), and get the length with len(). The length needs to be cast to the right type. In this case we can use as _ where rust will infer the right type to cast to.

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] {
    unsafe {
        tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(
            todo!(),
            data.as_ptr(),
            data.len() as _,
        );
    }
}
}

Next we create an array for the return value, pass a mutable pointer to this memory to our extern functin, and return the array.

#![allow(unused)]
fn main() {
pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] {
    let mut result = [ 0; 64 ];

    unsafe {
        tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(
            &mut result as *mut _,
            data.as_ptr(),
            data.len() as _,
        );
    }

    result
}
}

And we're done: an idiomatic rust wrapper around the crypto_hash_sha512_tweet!

Uninitialized memory

There is one more trick: our current function initializes and zeroes out the memory for result. That is wasteful because the extern function will overwrite these zeroes. Because the extern function is linked in, the compiler likely does not have enough information to optimize the zeroing out away.

The solution is MaybeUninit:

#![allow(unused)]
fn main() {
use std::mem::MaybeUninit;

pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] {
    let mut result : MaybeUninit<[u8; 64]> = MaybeUninit::uninit();

    unsafe {
        tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(
            result.as_mut_ptr() as *mut _,
            data.as_ptr(),
            data.len() as _,
        );

        result.assume_init()
    }
}
}

The std::mem::MaybeUninit type is an abstraction for uninitialized memory. The .uninit() method gives a chunk of uninitialized memory big enough to store a value of the desired type (in our case [u8; 64] will be inferred).

We can look at the LLVM IR to verify that 1) the initialization with zeroes is not optimized away and 2) using MaybeUninit does not initialize the array.

Below is a call site of our crypto_hash_sha512_tweet function that zeroes out the memory. Indeed, we see a memset that sets all the bytes to 0. (also not that our wrapper function actually got inlined)

%result.i = alloca <64 x i8>, align 1
%0 = getelementptr inbounds <64 x i8>, <64 x i8>* %result.i, i64 0, i64 0
call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(64) %0, i8 0, i64 64, i1 false), !alias.scope !8, !noalias !11
%_2.i = call i32 @bindings::crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull "foobarbaz", i64 9)

In constrast, the version with MaybeUninit just calls our extern function without touching the memory at all:

%result.i = alloca <64 x i8>, align 1
%0 = getelementptr inbounds <64 x i8>, <64 x i8>* %result.i, i64 0, i64 0

%_3.i = call i32 @bindings::crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull "foobarbaz", i64 9), !noalias !6

define i8 @call_with_maybeuninit() unnamed_addr #1 personality i32 (i32, i32, i64, %"unwind::libunwind::_Unwind_Exception"*, %"unwind::libunwind::_Unwind_Context"*)* @rust_eh_personality {
start:
  %result.i = alloca <64 x i8>, align 1
  %0 = getelementptr inbounds <64 x i8>, <64 x i8>* %result.i, i64 0, i64 0
  call void @llvm.lifetime.start.p0i8(i64 64, i8* nonnull %0), !noalias !2
  %_3.i = call i32 @crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull getelementptr inbounds (<{ [9 x i8] }>, <{ [9 x i8] }>* @alloc1, i64 0, i32 0, i64 0), i64 9), !noalias !6
  %1 = load <64 x i8>, <64 x i8>* %result.i, align 1, !noalias !7
  call void @llvm.lifetime.end.p0i8(i64 64, i8* nonnull %0), !noalias !2
  %2 = call i8 @llvm.vector.reduce.add.v64i8(<64 x i8> %1)
  ret i8 %2
}

define i8 @call_without_maybeuninit() unnamed_addr #1 personality i32 (i32, i32, i64, %"unwind::libunwind::_Unwind_Exception"*, %"unwind::libunwind::_Unwind_Context"*)* @rust_eh_personality {
start:
  %_4 = alloca <64 x i8>, align 1
  %0 = getelementptr inbounds <64 x i8>, <64 x i8>* %_4, i64 0, i64 0
  call void @llvm.lifetime.start.p0i8(i64 64, i8* nonnull %0)
  call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(64) %0, i8 0, i64 64, i1 false), !alias.scope !8, !noalias !11
  %_2.i = call i32 @crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull getelementptr inbounds (<{ [9 x i8] }>, <{ [9 x i8] }>* @alloc1, i64 0, i32 0, i64 0), i64 9)
  %1 = load <64 x i8>, <64 x i8>* %_4, align 1
  %2 = call i8 @llvm.vector.reduce.add.v64i8(<64 x i8> %1)
  call void @llvm.lifetime.end.p0i8(i64 64, i8* nonnull %0)
  ret i8 %2
}

Unit 7.1 - Rust from Python

Slides

Exercise 7.1.1: Test your environment

7.1.1.A: Set up the tools ⭐

Follow the instructions to instal the Maturin build tool: https://pyo3.rs/v0.21.2/getting-started. We recommend using pyenv: https://github.com/pyenv/pyenv, but pyenv only supports UNIX and WSL. Of course, you can use your favourite venv manager, too.

Navigate to the exercises/7-rust-for-data-science/1-rust-from-python/1-hello-world/ folder in your terminal. Setup and activate your virtual environment, and install the things we need. If you're using pyenv, that means running the following commands:

$ pyenv activate pyo3
$ pip install maturin
$ pip install asyncio
$ pip install aiofiles

7.1.1.B: Trying it out ⭐

If everything went well, you should now be able to build this project into a Python extension:

$ maturin develop
$ python test.py

The test.py script ends in a call to a function that unconditionally raises, so your script is expected to result in an uncaught exception.

Now, try running test_async.py as well and observe its behavior.

7.1.1.C: Running futures concurrently ⭐⭐

You can use asyncio.gather in your python script to execute multiple awaitables. Spawn a number of print_sleep futures, each with different sleep durations using asyncio.gather and observe the behavior.

Exercise 7.1.2: Streaming JSON

Here's a cool crate: Struson. It's a crate that allows for (de)serialization of JSON objects in a streaming fashion, meaning that you don't need to hold all of the data in memory. This contrasts a bit with the great library serde_json, which is very, very ergonomic, but does not allow for easy streaming (de)serialization of data. We're going build something around Struson, that we can use from Python.

Now, let's say we have a stream of JSON. Could be from someone on the internet or some other process running on the machine. Or, you know, a file. The stream consists of data that looks like this:

[
    {
        "lhs": {
            "d":[
                1, 2, 3, 4,
                5, 6, 7, 8,
                9, 10, 11, 12
            ],
            "n": 3
        },
        
        "op": [
            {
                "code": "dot",
                "rhs": {
                    "d": [
                        13, 14, 15, 16,
                        17, 18, 19, 20,
                        21, 22, 23, 24
                    ],
                    "n": 3
                }
            }
        ]
    }
]

This JSON is an array of objects that each represent the following:

• lhs: a matrix with m rows and n columns. m can be derived from n and the length of d. In this case m = len / n = 12 / 3 = 4.
• op: a sequence of operations that need to take place given lhs and, if the operation takes two operands, its rhs field. x corresponds to the operation that should be executed, and should more or less correspond to the methods provided on nalgebra's Matrix type. In this case, the Matrix::dot method should be run, which, for probably good reason, is describead as 'the dot product between two vectors or matrices (seen as vectors)'.

Our library should streamingly deserialize each incoming object from an asyncio stream of bytes, apply the given operation, and pass on the result.

Exercise 7.1.2.A: Give it a go ⭐

The scaffolding code in exercises/7-rust-for-data-science/1-rust-from-python/2-strompy/ implements what we want in Rust, and also provides a synchronous implementation. Have a look around in the code, and try to make sense of it. https://docs.rs is your friend if you want to know more about the dependencies that are being used. Note that for struson, we import a fork that supports deserializing asynchronously.

You can run cargo test to build and run the tests in src/lib.rs.

Exercise 7.1.2.B: The sync way ⭐⭐

During the next exercises, it's good to keep the PyO3 User Guide and the PyO3 API docs at hand.

This exercise is about making strompy_test.py run. Therefore, fix the todo!() in the exec function in src/lib.rs. Don't forget to add the function to the exposed module!

If you've done it correctly, you should be able to run it with the following commands:

$ maturin develop
$ python strompy_test.py

7.1.2.B: The async way ⭐⭐⭐

This time, we'll make strompy_test_async.py work. Implement feed_bytes and add an async method to the StrompyJsonReader type that yields a PyResult<Option<Vec<Vec<f64>>>> and is exposed with the name 'next'.

If you've done it correctly, you should be able to run it with the following commands:

$ maturin develop
$ python strompy_test_async.py

7.1.2.C: More features (Bonus) ⭐⭐⭐

Can you make strompy support more operations from Nalgebra?

7.1.2.D: Tidying up (Bonus) ⭐⭐⭐⭐

Representing the vectors as Vec<Vec<f64>> works, but it's not great. Make the StrompyJsonReader produce PyResult<Option<Py<PyList>>>s that constists of PyLists instead.

This is quite a hard exercise, as it involves working with PyO3s smart pointers: Py<T>, Bound<'py, T>, and GIL tokens: Python

Unit 8.1 - The Rust embedded ecosystem

Slides

Exercise 8.1.1: LSM303AGR ID

Use our newly gained knowledge to get our first application running and read out the ID of the LSM303AGR accelerometer. We can communicate with the LSM303AGR using the I2C that is present on the micro:bit board. Note that the nRF52833 supports I2C with its TWIM (Two-Wire Interface Master) peripheral.

To get started we'll setup the i2c peripheral on our development kit and read out the ID register of the LSM303AGR accelerometer. The starting point can be found in the file at exercises/8-embedded/1-embedded-ecosystem/src/main.rs in the repository.

Try to run the existing project and then fill in the functionality as instructed by the comments.

To use that project, you can use the following commands from inside that folder using the terminal:

• cargo build: Builds the project
• cargo run: Builds the project, flashes it to the device and listens for any logs which it will display in the terminal. (This uses the probe-rs run tool)

In both cases you can add the --release flag to turn on optimizations.

Unit 8.2 - Portable Rust drivers

Slides

Exercise 8.2.1: LSM303AGR Driver

When you really want to use a device, you want to have a driver. Let's write an actual portable device driver for the accelerometer we've got.

Go to the assignment in exercises/8-embedded/2-portable-drivers/1-lsm303agr-driver and implement the lsm303agr module. The goal is to use embedded-hal for our hardware definitions, so try not to use any nRF-specific types in that module.

Use the driver to read data from the sensor in src/main.rs

As a bonus exercise, support reading the magnetometer as well

Unit 8.3 - Async on Embedded

Slides

Exercise 8.3.1: Compass

In this exercise, we'll use the lsm303agr driver. Although the documentation doesn't show it, it supports async if you enable its async feature

Have a look at the examples in the lsm303agr-rs repository to get an idea of how to use this driver.

Using the lsm303agr driver, implement a compass. You can use the dial module to indicate the north pole's direction. You'll find a couple of todo!()'s with instructions.

Compiling the starter code yields a bunch of warnings. They'll be gone once your done.

Exercise 8.3.2: Blinky compass

The channel sender - receiver example in the embassy repository shows how to spawn separate tasks, and how to use channels to communicate between tasks. Using that knowledge, make the indicator LED in the dial module blink while magnetometer measurements are taken at the same time.

As we're not using defmt in this exercise, replace the unwrap!(<expr>) macro invocations with <expr>.unwrap() calls.

Unit 8.4 - The Embassy Framework

Slides

Exercise 8.4.1: Embassy project

This exercise is a bit bigger than you're used to: we're going to do a custom project using Embassy and the micro:bit V2. Work together in teams of 2 to 3 people during this exercise.

We have set up a crate for you to work on, with a simpl blinky application. You're free to adapt anything about it, it's just meant as a starting point. The crate resides in exercises/8-embedded/4-embassy-framework/1-embassy-project. Try it out to ensure the intial setup works.

You can choose from any of below projects:

• Build a snake game that uses the LED matrix as game display. You can use the buttons as input, or use the accelerometer or magnetometer. Or a combination.
• Create a light banner that reads text from UART, using the virtual COM port that is exposed by the interface MCU and shows it on the LED matrix.
• Create an audio recorder that reads samples from the on-board microphone and replays it using the on-board speaker. Additionally, you can show and FFT chart or a volume meter on the display
• Write a driver for the capacative touch sensor
• Build a button masher multiplayer game. Who is the fastest button presser? Either have it work with the two on-board buttons, or connect multiple boards together somehow. Show the score on the LED matrix.

Wrap-up

Slides

Evaluation form

Thank you for taking the time to help us improve the training!

You can find the evaluation from here