Rewriting scripts with Rust

It's becoming common to see some tools rewrote with Rust like bat, dust, tokei & co.

In my case, it's rewriting some Bash and Python mostly to extend them more easily.

Bash to Rust

Bash script can become a real pain, for example when I'm writing it on macOS (bash and zsh are installed) and needs to execute it on some OS with different shells like (t)csh, ksh.

Just exporting an environment variable is a rewrite from export NAME=VALUE to setenv NAME VALUE, now I let you imagine when it's directly logic with finding available tools like checking if curl or wget is available and some logic about potential failures.

With Rust, I can do a strict "no external dependency" so the binary built will always work as expected. In addition, the type checking permits avoiding a lot of "bargains" specific to the Bash script.

Python to Rust

Python script needs to have Python installed. Sometimes, it's impossible to have it (ITIL, specific environment requirements & co.) so I need to provide a virtual env. Providing a 50 MB virtual env for a 60 kB script, it's unoptimized and mostly a waste of resources/time.

The binary version (Rust): 3.2 MB… so a 93.6% decrease. I can use scratch image as the base for my containers because the binary built is statically linked and doesn't need an external system like alpine or busybox.

The other advantage is for creating an OCI image. Python-based containers are heavysized and need some environment variables to work correctly:

• PYTHONDONTWRITEBYTECODE: If this is set to a non-empty string, Python won’t try to write .pyc files on the import of source modules.
• PYTHONUNBUFFERED: Force the stdout and stderr streams to be unbuffered. (needed for Python older than 3.7)

Finally, when I develop, Rust is the game changer because the compiler outputs every warning and error instead of partial runs with Python where it'll fail during the execution.

Do we still need Bash/Python scripts

Honestly, I can continue to need them for specific cases:

• Bash: executed by Terraform or Ansible or some CI/CD (running autoconf and make for example) or to create an OCI image using buildah
• Python: for a quick PoC but not for the production version where it's important to have the best performance/cost ratio with the budget in mind

Now, Python is like Java before GraalVM and more exactly GraalVM - Native Image.

My biggest "garbage" in Python was Pandas… I replaced it with Polars (Rust library with Python bindings) otherwise I'll use (and cost) a big compute node for nothing because Pandas is still unoptimized.