Passwords in scripts: why I switched to keys
This week my NAS raised an alert that I took for an attack at first: over the older interface, 2715 requests had signed in within 24 hours, all of them from a single machine. The machine was mine, the requests came from my own scripts. Along the way I learned something about my homelab that I should have known long ago.
This week my NAS raised an alert that I took for an attack at first: over the older interface, 2715 requests had signed in within 24 hours, all of them from a single machine. The machine was mine, the requests came from my own scripts. Along the way I learned something about my homelab that I should have known long ago.
The alert I did not understand at first
The alert contained a number, and it was so high that I first took it for a counting error. 2715 sign-ins in a single day. So I looked up who was signing in: always the same address, always the same account name, spread across the whole day, without a pause. It was the pattern of a script in a loop that signs in again on every pass. The script was mine, the alert was justified, and the attacker was me.
Why every query was a sign-in
The reason is how the older interface is built. It has no session that survives between two calls. Every request carries a user name and a password, gets checked, gets answered, and is forgotten. For the other side, every single query is therefore a complete sign-in, even though from my point of view I only pick up a number.
For a human who signs in once a day through the browser, that is no problem at all. For anything that runs automatically, it is a bad idea. Automation means repetition, and repetition becomes a stream of data in the log. In my case it looked like this:
- A status query that fetches the service state every few minutes.
- A storage display that collects the free space.
- A monitor that asks every minute.
- A backup that checks whether its target is reachable.
- A few tiles on an overview page, each asking by itself.
The arithmetic that convinced me
I did the maths, and it was disarmingly simple. 2715 sign-ins a day is around 113 per hour, on average almost two per minute, around the clock. A single monitor running every minute adds up to 1440 sign-ins a day. Over a year that is almost a million sign-ins, for numbers I already had.
What bothers me is not the sign-in itself, it is the risk of confusion. A log that constantly shows thousands of sign-ins from your own machine is a log in which a real anomaly disappears. If all you ever see is noise, you stop recognising the signal.
The switch: one key per service
The solution was unspectacular. Instead of a user name and a password, every script gets its own key, which I create in the application itself. A key can be revoked on its own, it can be limited to a role, and it does not carry the password I also use to sign in through the browser. One key per service and task, not one for everything.
The second half of the solution matters just as much: the key does not belong in the script. No password, no key value in the source code, not even commented out, and not even because nobody reads that directory anyway. In my case the value comes from an environment variable, and only the service that needs it knows the file holding the values. What is not in the script cannot travel along either.
The newer interface: one connection instead of many sign-ins
At the same time I moved the scripts to the newer interface wherever one exists. The difference is exactly the one this post is about: instead of creating a new sign-in for every question, it keeps a persistent connection and sends changes by itself. 1440 sign-ins a day become one connection that stays open. That takes load off both sides, and my log no longer looks like an attack.
The fallback that saved me
One thing I learned that I would have called unnecessary before: the newer interface cannot always do everything the older one could. One service delivered only part of the values I needed over the new connection. So for that case the old method stayed, but with a key instead of a password. I kept this fallback on purpose, because without it I would have switched off half of the display.
The password itself is now only the emergency exit. It appears in no script and in no schedule. When I need it, it is once by hand, to create a new key.
What happens when the key expires
A key can expire, and that is exactly what happened to me. One morning a monitor was silent. Not broken in the sense of an error message, just silent. The key had expired, the script no longer received data and dutifully kept writing to its file, only without content. The worst part was that the alert meant to wake me ran over the same path as the data. A failure that takes its own alarm with it never reports.
Since then: a script that receives no data must make noise. Better one alert too many at three in the morning than a display that stays empty for weeks and looks friendly while doing so. For keys that can expire, I keep a date in the calendar, a few days ahead.
What I take from this
- Read the log, not just for errors but for patterns.
- One key per service and task, with few permissions.
- No credentials in the script, not even in a private file.
- Use the newer interface when it offers a persistent connection.
- Keep the fallback, but only for the case that needs it.
- Fail loudly when no data arrives any more.
- Put expiring keys in the calendar in time.