Technical notes › Unclutter $HOME with command wrappers

You know what’s not the worst problem in the world, but is still kind of annoying? The proliferation of dotfiles and dotfolders in your home directory, like .vim or .lesshst or .sqlite_history or .wget-hsts.

There’s a standard for which directories should be used by programs for things like scratch space, or configuration files, or cache — the XDG Base Directory Specification. The standard originated in 2003, but seems to have only started being adopted by programs in the early-to-mid 2010s. This means that a lot of older programs that pre-date the standard — including common programs that I use every day, like Vim or OpenSSH — had to pick their own directory instead.

This isn’t just an aesthetic thing; there are good reasons to adopt the standard. It enforces the separation between where users should put configuration files, where programs should place their own state files, and where programs should put their own temporary cache files. This is something that’s not obvious when each program has its own directory, like ~/.vim or ~/.bundle, which usually share multiple classes of files. Which directories should you sync between computers, which directories should you leave alone, and which directories can you safely delete to free up disk space?

Under the XDG specification, you should sync everything in ~/.config, you should leave everything in ~/.local alone, and you can safely delete everything in ~/.cache. The only problem is that programs aren’t using them.

Wrapper scripts #

The way I prefer to deal with this problem is to create a folder full of wrapper scripts, each of which exports a couple of environment variables before calling the program it’s named after, possibly with an extra argument or two. I then add this folder to the start of my $PATH.

For example, even though my main shell is Fish, I will still run bash from time to time, to run a piece of common shell script without having to test it, or something like that. It used to be the case that every time I ran bash, it would create its history file in my home folder; I also have a few lines of configuration, which it would also read from my home folder. But no more, as I configure bash to look in the ~/.local and ~/.config directories for these files:

Similarly, every time I’d run wget to fetch something from a site with the Strict-Transport-Security header set, it would cache the domain name locally to avoid a plaintext HTTP request from being made in the future. This is the correct behaviour for the header, and would be a useful feature if I ran wget regularly — but less useful if I run it rarely. The HSTS history file is cached data that I can safely delete, so I put it in the ~/.cache folder instead:

These Fish scripts work by exporting the relevant environment variables with set -x, which makes them available to any child processes. I then add any extra command-line arguments that are necessary. Because I install both Bash and Wget from Homebrew, I know exactly where the “parent” executable lives (you can run brew --prefix to get the directory), I include the full path because otherwise the wrapper script would end up calling itself, and the result would not be good.

The most comprehensive guide to which programs’ paths are available for configuration, and how you can go about doing that, is over at the ArchWiki’s “XDG Base Directory” page. It lists programs that support the XDG folders natively, programs where you can customise the paths by modifying environment variables, and programs that (sadly) have their paths hard-coded. If you want to start dealing with these files and folders, this page should be your first resource.

The Danger #

The worst part about this setup is that if anything happens to run the base executable, rather than the wrapper script, it will act as though the files it is looking for no longer exist, which may do something unexpected. If something runs, say, /usr/local/bin/wget, it will re-create the ~/.wget-hsts file — as from its perspective, it is being run for the first time ever.

Here’s the most annoying instance of this happening to me: one of the first wrapper scripts I created was for Vagrant, the development virtual machine manager. Vagrant needs somewhere to store its plugins and VM images, and by default it uses the ~/.vagrant.d directory as scratch space to store them. I had overridden this directory to be ~/.local/share/vagrant instead, so I could have one fewer entry in my home directory listing.

However, I soon discovered that the ~/.vagrant.d directory was actually being re-created every few minutes. My SwiftBar script that periodically runs vagrant global-status --machine-readable was running the vagrant executable, and as SwiftBar has nothing to do with my Fish configuration files, it was not getting its environment variables exported like it would when run from a terminal. Not only did this mean that the same program was giving me two different answers depending on where it was run from — my VMs were present in the terminal but absent in the menu bar — but I hadn’t actually uncluttered my home directory, which was the entire point of this exercise!

An alternate approach: do nothing #

This is why I need to finish this tech note by saying that there’s a perfectly reasonable alternative to dealing with this problem: letting them pile up and doing nothing at all. That way you’ll never have any issues, as long as you know which files you need to synchronise and back up and which files you don’t. Going with the flow will always be easier and will avoid problems you’ll encounter trying to go against it — for one final example, IntelliJ-Rust could no longer auto-detect my Rust and Cargo directories after I had moved them out of $HOME. Normal people wouldn’t run into that problem, but I did.

Most of the configuring, tweaking, and scripting that goes on on my computer, I do for productivity reasons: I’d like to script this to avoid making mistakes I’d make when doing it by hand; I’d like to automate that to save myself time and manual work. Uncluttering my home directory is neither of those things — it’s giving myself more problems, in exchange for some sense of cleanliness or order. The only reason I do it anyway is to remind myself I shouldn’t be 100% productive 100% of the time. Maybe you have other priorities. Embark upon this quest if these dotfiles and dotfolders truly bug you, but bear this in mind.∎