Rust is an amazing language. It is one of the best potential alternatives to C and has been elected two times in a row as the most promising language of the year (by me, :P). However, because its strict compile-time memory correctness enforcement and because it is a low-level language, it is not the fastest way to build a prototype. But don’t worry! Rust is the perfect language for embedding fast-binary libraries in Python! In this way we can get the best of both worlds!

Writing Rust code that can be executed in Python is stupidly easy. Obviously, you have to well design the interface between the two languages, but this will improve your Python code in two ways: 1) You can execute CPU-intensive algorithms at binary speed and 2) use real threads instead of the Python “simulated” ones (and because Rust is designed to be memory safe, writing thread safe routines is much easier). Let’s see!

Writing a simple Rust Library

We will start with a very basic example taken from the official Rust guide. At first, we need to create a new Rust package (crate) in which we will write the CPU-intensive code in Rust.

$ cargo new rustbomb
$ cd rustbomb

In the src\lib.rs file we will write a simple multi-threading algorithm: it starts 10 threads and each thread will loop 5 million times increasing a counter. No input. No output. It is a “heat generator”.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

use std::thread;

fn process() {
    let handles: Vec = (0..10).map(|_| {
        thread::spawn(|| {
            let mut x = 0;
            for _ in 0..5_000_000 {
                x += 1
            }
            x
        })
    }).collect();

    for h in handles {
        println!("Thread finished with count={}",
        h.join().map_err(|_| "Could not join a thread!").unwrap());
    }
}

Nothing to add here. In order to make this piece of code working as an external library we need to change the function definition with the following lines:

1
2

#[no_mangle]
pub extern fn process() {

Don’t worry about the meaning of this for now. Just note that extern means that we want to call this function from the outside, from a C-like interface, and  pub means that the function is “public”.

The second change we need is to add these two lines at the end of the Cargo.toml file.

1
2
3

[lib]
name = "rustbomb"
crate-type = ["dylib"]

This say to the compiler to compile the library in a “standard” way so that can be used as a C library, and not in the “rusty” way. We can now compile this crate and we are ready to embed this library in a Python script.

$ cargo build --release

Writing the Python side

We need now to write the Python part of the application. Create a new file client.py (for instance) in the crate root. To attach the Python script to the library we will use the standard interface ctypes.

1
2
3
4
5
6
7
8

import ctypes

# Wind DLL get as parameter the path to the brand new compiled library!
lib = ctypes.WinDLL(".\\target\\release\\rustbomb.dll")

lib.process()

print("done")

Because I’m using Windows I’m using WinDLL. If you are using Linux try with CDLL.

We finished in just 3 lines of code. Try to run this Python script and enjoy some rusty threads running on your cores!

Passing Parameters to Rust functions

Cool! We want more! The example in the official guide is a bit simplistic. It is uncommon that we call no input/no output function in an external library. What if we want to pass as parameter the number of threads we want to spawn?

The first naive implementation will be to just add the parameters in Rust in the standard way.

1
2
3
4
5

#[no_mangle]
pub extern fn process2(threads_number: i32) {
    let handles: Vec<_> = (0..threads_number).map(|_| {
        thread::spawn(|| {
...

And then call this new function with a number. And… Well… It works! However, is not a wise to use “Rust’s specific data types” when we are writing an FFI interface. We need something more C-friendly.

Let’s add this to Cargo.toml

1
2

[dependencies]
libc = "0.2.0"

This will include the libc crate that will add a lot of C-friendly types you can use. Then we need to add these two lines at the beginning of the Rust library.

1
2
3

extern crate libc;

use libc::{size_t,int32_t};

Now we are ready to write the function declaration with the right type:

1
2
3
4
5

#[no_mangle]
pub extern fn process2(threads_number: int32_t) {
    let handles: Vec<_> = (0..threads_number).map(|_| {
        thread::spawn(|| {
...

Note that we are using int32_t instead of i32. It is the same thing, but now your code is bulletproof, even if you change architecture or implementations. It is formally correct and it safer to use.

What we will see next

The topic requires more space than I originally thought. For this reason I’m going to split this article in several parts. For now we have just barely scratched the surface. We are able to create a simple function that gets an integer and return nothing.

In the next part we are going to see how to pass lists, tuples, strings and other complex Python data-types without irritating the Rust’s Gods.

Extra:

All the code in this and future articles will be available in this GitHub repository. Enjoy!

(Cover image by oOIsiusOo)