You can find the full source code here: https://github.com/caveofprogramming/rust/

To do this I’m implementing a data block recycling mechanism, so that after some initial buffering, the consumer passes used data blocks back to the producer, which fills in new data and adds them to the queue as soon as the consumer has read more data from the queue.

Here I’m using crossbeam 0.8, so my cargo.toml has this in it:

My imports look like this:

I’m using a bounded crossbeam channel. This is a channel which will only accept a specified number of data blocks. This is what constrains the speed of my producer; the producer cannot add new data blocks into the channel, once the channel’s full, until the consumer reads a block from the channel.

I’m using Duration here just to limit the speed of the consumer, to simulate processing video or audio at a fixed pace.

Then I have some definitions:

BLOCK_SIZE is the size of a single block of data. More typically it might be 512, 1024 or whatever.

N_BLOCKS is the size of the crossbeam channel.

Then BLOCK is the block definition itself. A block in this case is an array of BLOCK_SIZE elements, of type f64 (but you can use whatever type is useful). I wrap this in a Box smart pointer just so that these blocks exist on the heap, not the stack, and the channel actually contains references to them.

My two channels are created like this:

I have one channel for sending data to the consumer from the producer, and a second for ‘recycling’ the blocks by sending them from the consumer back to the producer.

In the producer I use send() to send the blocks and recv() to receive them. These methods both wait until (in the case of send()) there is space on the queue to send, or until (in the case of recv()) a block is available to be received.

The consumer is as follows. Now I’m using try_recv() to receive data. This returns immediately and will return an error if there’s no data to receive, so I’m relying on the producer successfully keeping up.

Then I use send to return the data block back to the producer. I think this should be OK, but we could also consider making the return queue unbounded (or just really large) to avoid any possible delay.

Finally, we wait for the threads to finish.

The output from running the program looks like this:

The most frustrating thing about Rust that I’ve found so far is the problem of how to refer to persistent entities that have to maintain their own state, across multiple threads.

In this video I’ll show you how to tackle the problem using Arc and Mutex.

However, this isn’t suitable for every situation.

For example, if your persistent entity happens to manage a collection of software instruments which have to be modified from a particular thread, while your audio thread separately has to read audio samples from those instruments, and the persistent entity must remain modifiable—then probably Mutexes aren’t the answer.

Locking mutexes in audio threads, or real-time threads generally, is widely considered a bad idea.

The best way I’ve found so far of dealing with this, after trying out a bunch of different things, is to use channels, which we covered earlier. These may not be fast enough to send one audio sample at a time, but we can send small blocks of data (maybe 512 samples) at a time, and recycle the blocks to and from the producer and consumer threads. That’s a topic for a future post!

Git repo: https://github.com/caveofprogramming/rust/

For many of us, writing Rust in an object-oriented style would seem natural, since we’re used to Java and Python and what-not.

But Rust does make you jump through a few hoops, for the sake of efficiency.

In this post we’ll see how to get object oriented behaviour in Rust, even when we want to be able to clone a vector of objects, which is sometimes very useful.

First let’s start with a simple trait.

Anything that implements this trait must have a get method that returns a float.

Now we’ll create a struct that implements the trait.

I’ve implemented the trait for Pi, and I’ve also added a constructor.

Now we can use Pi like this:

This prints:

I can also add Pi to vectors:

This works fine.

However, there’s a problem if I change the type of the vector to Vec<Value>, intending to put other things that implement the Value trait into the vector.

This doesn’t work:

The problem is that Rust, for efficiency, wants to know the size of everything you put in a vector, and here it doesn’t know that. We could create other things of difference sizes that implement Value, even at runtime, and put them in the vector.

The values in the vector will just be stored in contiguous blocks of memory, which can’t work if Rust doesn’t even know the size of them.

Furthermore, we don’t get the polymorphic behaviour we expect. We want the correct get method to be called, depending on what objects are found in the vector.

We can fix the first issue by storing an array of pointers, so that the actual objects are stored on the heap. To make polymorphism work, we need to use the dyn keyword.

We end up with the following working code:

By wrapping the vector elements in a Box and by using the dyn keyword, we get the result we expect.

Now suppose we want to clone the entire vector for some reason.

The following code still doesn’t work.

Rust knows how to clone the vector; it just doesn’t know how to clone

Box<dyn Value>

To fix this, we have to implement Clone for Box<dyn Value>.

The following implementation would work if we only ever expected Pi to implement Value (which is obviously a bit silly).

Now the program runs, but the problem is that when we clone a Vector of Value objects, we’re always going to get a Vector of Pi.

The solution is to give each implementation of Value a method which returns a box containing the cloned version of itself.

An easy way to make Pi cloneable is just to add #[derive(Clone)]at the top of it. But we could alternatively implement Clone for Pi however we want.

Now we add a method to the Value trait called clone_box.

Then we implement this for Pi.

Finally, we use this in our implementation of Clone for Box<dyn Value>.

Now our program works properly.

We can create something else that implements Value.

Now we can add Surprise to our vector and everything still works correctly.

When we run the program, the contents of both the original vector and the cloned vector behave appropriately.

Sending and Receiving Messages

Rust’s ownership rules can make it difficult to have multiple threads reading and writing to some data model.

Inter-thread messaging is a very elegant alternative.

For this example you need crossbeam-channel = “0.5” in your dependencies.

In this program I use crossbeam to create an “unbounded” channel, then I send a message consisting of the value 123 down the channel. At the other end of the channel, a receiver receives the value, and we print it.

Note: the channel is “unbounded” in the sense that we can send as many messages as we like, even if they haven’t been received yet. We can also create bounded channels that limit the number of messages that can be sent when they are not being received.

You can send vectors, structs, enums and complex data structures through a crossbeam channel.

The real value of this is:

• The sender and receiver do not have to exist in the same thread. They can be in different threads, and thread synchronisation problems are thereby avoided, since we’re not reading or writing shared data.

• With crossbeam you can clone the sender and receiver as many times as you want, and pass them around to different functions and to different threads, just as you like. A function or thread can take ownership of the clone, but it still refers to the same channel.

For example, you can have an “update engine” thread which solely deals with a data model, and fields requests for bits of data or applies updates to the model on the basis of messages sent to it.

We can also create a “bounded” channel.

This channel will only let us send messages one at a time; each message must be received before a new one can be sent. This channel is apparently a little more lightweight, and can be used to send replies to messages containing data.

I cover both in the video, and we also see how this looks in the (more realistic) context of using threads.

What is Dynamic Dispatch?

In object-oriented languages we can normally create subclasses of a superclass type, then have a variable of the superclass type which can be made to refer “dynamically” at runtime to whichever of the subclasses we like.

For example, we could have an Animal superclass, and Dog and Cat subclasses.

Dog and Cat are then kinds of Animal.

Suppose Animal has a speak() method. Dog and Cat then inherit this method, but they can override it, so that each has its own version of speak() with different functionality.

It’s then possible to write a program which, for example, asks the user what kind of Animal to create, and sets a variable of type Animal to refer to whatever they’ve chosen.

When the speak() method of this method is called, the correct version of it must then by called by the program at runtime. If the user chose to create a Dog, the Dog speak() method is called; if they chose a Cat, the Cat speak() method is called.

This is known as dynamic dispatch. The appropriate method is dynamically dispatched at runtime.

How This Works In Rust

Rust is a kind of semi-object-oriented language. It doesn’t have inheritance, so we can’t create a superclass object and then have subclasses of it.

However, it does have traits, which define functionality. So it is possible to have an Animal trait and then Dog and Cat structs which implement this trait.

Then we implement the trait for these classes. If Animal defines a speak() method, Dog and Cat must have one.

It is then possible, in Rust, to do dynamic dispatch, even just using ordinary references.

However, I ran into some problems with this when I wanted to create objects in the new() function of some other struct, as you’ll see in the video.

These problems can be solved using the reference counter, Rc.

In addition, when you specify a trait as the type of a variable, you must make the variable a reference and use the dyn keyword, which is specifically intended to allow dynamic dispatch, apparently. Otherwise your program won’t compile.

Dynamic Dispatch with Ordinary References

First let’s take a look at dynamic dispatch using only references.

Here’s our trait.

Dog and Cat are then just structs which implement this trait.

Give this, the following actually works perfectly well:

The problem is that this doesn’t work when we want to create the struct instances in a method, such as new(), and store references in struct fields, because then we end up with dangling references to objects that have gone out of scope.

Then we need reference counters, and I cover this in the video.

Let’s take a look here at a simple example of dynamic dispatch using reference counters.

Dynamic Dispatch with Reference Counters.

We’re going to need

at the top of the file.

Then we can do the following, which can also be made to work with struct methods and fields.

Why Don’t Plain References Work with Struct Fields?

This issue I had with plain references was that I wanted to have two struct fields referring to a Dog and a Cat then a third, “pet” that refers to one of the above, and I wanted to arrange this in a function called new() that’s associated with the struct.

One you’ve directly initialised a struct field with a Dog, for example, you can’t then make pet refer to that field via self, because self doesn’t exist yet.

You can’t create dog and cat local variables and just use references to those either, because then you’re left with dangling references.

Try it and you’ll see what I mean!

However, Rc fixes the problem.

If you need your objects to be mutable, you can use RefCell for interior mutability and use types like Rc<RefCell<dyn Animal».

If you need to pass objects between threads you could also consider

Arc<Mutex<dyn Animal»

or

Rc<RwLock<dyn Animal».

One major problem I’ve faced was, MVC in Rust just doesn’t work like I thought.

It appears you can’t really just have a model and some views, and then have a controller class that creates these and passes parts of the model to the views.

In this video I’ll show you the kinds of problems I ran into. Basically they come down to problems with borrowing; you can’t have two mutable references to the same thing at the same time, so your controller just freely pass around mutable links to views.

Passing different parts of a model to different views would require multiple mutable links to different parts of the same data structure, and I couldn’t find a way to get this to work.

However, it can be done, and the trick is to use RefCell (for interior mutability) and Rc (for multiple ownership).

Full code is here: https://github.com/caveofprogramming/rust/

I’ll probably also make videos on the software synth I’m working on, as soon as it can play an actual melody.

As programmers familiar with multithreading will know, you can’t just write to shared data using multiple different threads. Rust even stops you from doing this via its ownership system.

For example, in C (or practically any other language), even if two threads try to increment an integer, this creates problems, because even this isn’t an atomic (single step) operation.

Behind the scenes, the value of the integer gets copied to a register, incremented, and then copied back to the original memory location. Two threads doing this at the same time will sometimes both copy the original value before incrementing and copying back, and this means some increments will get skipped.

One way to deal with this is by using thread-safe integers, in this particular case. Rust has AtomicI32 for example.

Another, more general but also trickier method, is to use mutexes (locks) to ensure that two threads cannot write to the same variable at the same time.

The examples in this page are all built around the Rust documentation at https://doc.rust-lang.org/book/ch16-03-shared-state.html

A Simple Mutex Example

The new method of Mutex expects the initial value of the type that we want to protect.

In this example, a Mutex wraps an integer. To access the integer for writing, we have to lock the mutex.

The lock() method returns a Result which wraps either a MutexGuard or a PoisonError. MutexGuard is a kind of smart pointer which implements the Deref trait, so we can dereference it to get at the actual value.

The key point about a mutex is that two threads (or the same thread) cannot lock it twice at the same time.

The following program just hangs, because the main thread waits indefinitely on the second lock attempt. It’s waiting for the lock to be unlocked by another thread, which never happens.

Using Mutexes to Increment a Value

In the following program I create 10 threads and, in each of them, I increment the value guarded by the mutex.

When you try to do this naively, the first problem that comes up is you cannot access the mutex variable from the closure, because then the closure code might outlive the variable.

You are forced to add move to the closer declaration, to move the variable into the closure. But then, you can’t move it into subsequent closures in further iterations of the loop.

You can try to fix this by using Rc, with clone, but then the problem is that Rc isn’t threadsafe, so you still get an error and the code won’t compile.

However, if we swap Rc for Arc (atomic reference counter), which is thread safe, the code now works and finally prints 10.

Read more

Read more

Read more

Read more

Read more

Read more

Read more

Read more

Read more

Read more

Read more

Read more

Read more