Why is my Rust code blocking???
Rust is a low-level programming language known for its performance and safety. We'll look at why my first attempt to use std::process
resulted in blocking.
Problem
The code snippet below is an example of a Rust program that can potentially block. It uses the Command
struct from the std::process
module to spawn a new process that runs the ffmpeg
command. It reads an audio file in mp3
format from standard input (stdin
), passes it to ffmpeg
as input, convert the audio file to ogg
format, and then reads the output from ffmpeg
and writes it to standard output (stdout
).
fn main() {
let mut cmd = Command::new("ffmpeg")
.arg("-i")
.arg("-")
.arg("-c:a")
.arg("libopus")
.arg("-f")
.arg("ogg")
.arg("-")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.unwrap();
let mut input_data = Vec::new();
stdin().read_to_end(&mut input_data).unwrap();
let mut output_data = Vec::new();
let mut stdin = cmd.stdin.take().unwrap();
let mut stdout = cmd.stdout.take().unwrap();
stdin.write_all(&input_data).unwrap();
stdout.read_to_end(&mut output_data).unwrap();
let exit_status = cmd.wait().unwrap();
if !exit_status.success() {
panic!("ffmpeg failed");
}
}
In this scenario, the Rust program is using the write_all
method to write data to the stdin
pipe, which is connected to the ffmpeg
child process. The ffmpeg
process is writing data to its stdout
pipe, but nobody is consuming that data.
When the stdout
pipe buffer fills up, the ffmpeg
process will block while trying to write further data. This means it will also stop reading data from stdin
, causing the stdin
pipe buffer to fill up. When the stdin
pipe buffer is full, the parent Rust process will also block while trying to write further data.
Solution
To avoid this issue, the program can use another thread to write the data to stdin
, allowing the ffmpeg
process to read from stdin
and the parent process to write to it concurrently.
fn main() {
let mut cmd = Command::new("ffmpeg")
.arg("-i")
.arg("-")
.arg("-c:a")
.arg("libopus")
.arg("-f")
.arg("ogg")
.arg("-")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.unwrap();
let mut input_data = Vec::new();
stdin().read_to_end(&mut input_data).unwrap();
let mut output_data = Vec::new();
let mut stdin = cmd.stdin.take().unwrap();
let mut stdout = cmd.stdout.take().unwrap();
// Use a thread to write to stdin
let writer = thread::spawn(move || {
stdin.write_all(&input_data).unwrap();
});
stdout.read_to_end(&mut output_data).unwrap();
writer.join().unwrap();
let exit_status = cmd.wait().unwrap();
if !exit_status.success() {
panic!("ffmpeg failed");
}
}
Why am I doing this?
I'm working on a Telegram bot that can send voice messages.
Since the sendVoice API in Telegram only accepts .OGG file encoded with OPUS, I created this simple Rust program wrapped in an Actix Web server.
Source code: https://github.com/lzm0/mpeg2opus