Rust Concurrency and Parallelism: The Basics

In this article, we will learn the basic of concurrency, parallelism, and how they work in Rust.

Concurrency vs Parallelism

Concurrency is working at multiple task at the same time. Consider a chef in a restaurant. He managed to do all the work like cooking, chopping vegetables, then checking the oven. Only one action happens at a time.

Parallelism is doing multiple thing at the exact same time. There are multiple chef handling multiple tasks at the same moment. It’s an actual simultaneous work. We need multiple cores or CPU to do true parallelism.

Core vs Thread

We need to understand how concurrency works in core and threads. Modern CPU has two threads per core. Back with the example of a kitchen, consider a chef is working on two pads at once. A chef is a representation of a core, and a pad is executed in a thread. In pad 1, the chef has to boil water for 4 minutes. While waiting for the water to boil, the chef is chopping vegetables for pad 2.

When the water starts boiling, the CPU hardware notices that Thread 1 is stalled (waiting). The chef leaves and start working to chop vegetables, which the task will be executed in thread 2.

The real-world example of “boiling water” is I/O operation such as network calls or reading from storage/database. I/O operation is a delay where the CPU has to wait from outside its chip.

On the other side, chopping vegetables is an example for CPU is doing raw, heavy mathematical calculations and zero-waiting around. The data it needs (vegetable) is already sitting xbeside it in L1/L2 caches.

Rust Concurrency

Handling concurrency while ensuring memory safety is one of Rust major advantage. Historically, problems such as data race and deadlocks are very common when implementing concurrency. By leveraging ownership and type checking, many concurrency errors are prevented in compile-time rather than runtime errors.

To achieve concurrency in Rust, we can utilize the most popular asynchronous runtime, tokio. Consider this example:

use tokio::time::{Duration, sleep};

async fn run_task(task_name: &str) {
    println!("task {} start", task_name);
    sleep(Duration::from_millis(100)).await;
    println!("task {} end", task_name);
}

#[tokio::main]
async fn main() {
    tokio::join!(run_task("A"), run_task("B"), run_task("C"));
}

In above code, we have an async function run_task which executes sleep function inside. See below output:

task "A" start
task "B" start
task "C" start
task "B" end
task "C" end
task "A" end

Notice that the task B starts when task A is sleeping. Task C starts when task B is sleeping. During that 100ms sleep time, tokio reschedules the tasks and pick whatever the CPU scheduling picks. The result could be different between executions, depends on how the operating system schedules this thread.

Look at the blue arrow above. It yields when there is an .await, meaning the thread is allowed to do another task while waiting.

For those who are familiar with JavaScript, it is equivalent to Promise.all(run_task("A"), run_task("B"), run_task("C")).

Tokio also provides spawn function that immediately spawn a thread to run the work inside an async block. Let’s rewrite the example above to use tokio::spawn!

#[tokio::main]
async fn main() {
    let (result1, result2, result3) = tokio::join!(
        tokio::spawn(run_task("A")),
        tokio::spawn(run_task("B")),
        tokio::spawn(run_task("C"))
    );
}

tokio::join! handle multiple async tasks in a single thread. Like a multitasking chef from earlier examples. When we use tokio::spawn for each task, we are summoning new chef to handle the task. In this example, three threads are spawned.

If there is no .await point in the task, the thread will be blocked to focus on that task. The chef has to focus on that single task, he can’t switch to another task while working on it. Check out this example:

tokio::spawn(async {
    for i in 0..1_000_000_000 {
        heavy_math(i);
    }
});

We spawn a thread to do a long and heavy task that will block the thread. This is bad because the thread can’t yield to do another task while working on that long task.

Better way to put this is to use tokio::spawn_blocking which spawn a separate thread to work do CPU-heavy or blocking synchronous task so it won’t block the main async executor.

The task is no longer blocking the thread as it’s moved to a separate dedicated thread.