ebde01ddeb
The script posted alerts to https://ntfy.silverwal.com/certbot, which is served by the same nginx whose certificates it monitors. A TLS failure on this host would make curl -fsS fail verification and drop the alert describing that failure, leaving only a line in a log nobody reads. ntfy is already bound to 127.0.0.1:2586, so post there instead.
77 lines
2.1 KiB
Markdown
77 lines
2.1 KiB
Markdown
# Silver Server Ops
|
|
|
|
Server-wide maintenance scripts and operational notes for the Silver Cloud Hetzner host.
|
|
|
|
This repo is for host-level improvements that are not tied to one specific app or
|
|
service. SSL expiry alerts are the first check in the repo; more server-wide
|
|
maintenance checks can be added here over time.
|
|
|
|
## Implemented Checks
|
|
|
|
- SSL certificate expiry alerts
|
|
|
|
## SSL Certificate Expiry Alerts
|
|
|
|
`scripts/check-cert-expiry.sh` checks every Let's Encrypt certificate under
|
|
`/etc/letsencrypt/live` and sends an ntfy alert when any certificate expires in
|
|
less than 14 days.
|
|
|
|
### Install
|
|
|
|
```bash
|
|
sudo install -m 0755 scripts/check-cert-expiry.sh /opt/scripts/check-cert-expiry.sh
|
|
```
|
|
|
|
### Root cron
|
|
|
|
```cron
|
|
15 8 * * * /opt/scripts/check-cert-expiry.sh >>/var/log/cert-expiry-check.log 2>&1
|
|
```
|
|
|
|
### Defaults
|
|
|
|
```bash
|
|
CERT_DIR=/etc/letsencrypt/live
|
|
EXPIRY_DAYS=14
|
|
NTFY_URL=http://127.0.0.1:2586/certbot
|
|
ALERT_ON_NO_CERTS=true
|
|
```
|
|
|
|
### Overrides
|
|
|
|
```bash
|
|
EXPIRY_DAYS=21 /opt/scripts/check-cert-expiry.sh
|
|
```
|
|
|
|
If the ntfy topic is protected with an access token:
|
|
|
|
```bash
|
|
NTFY_TOKEN=your-token /opt/scripts/check-cert-expiry.sh
|
|
```
|
|
|
|
### Manual test
|
|
|
|
```bash
|
|
sudo /opt/scripts/check-cert-expiry.sh
|
|
```
|
|
|
|
The script exits `0` when all certificates are healthy and `1` when it sends an
|
|
alert or cannot run the check correctly.
|
|
|
|
## Notification Notes
|
|
|
|
ntfy is a good default for this server because it is already self-hosted, simple
|
|
to call from shell scripts, and supports useful alert metadata such as title,
|
|
priority, and tags.
|
|
|
|
Alerts go to ntfy over `http://127.0.0.1:2586` rather than the public
|
|
`https://ntfy.silverwal.com` URL. The public URL is served by the same nginx
|
|
whose certificates this script watches, so an expired or broken certificate on
|
|
this host would fail curl's TLS verification and silently drop the very alert
|
|
that reports it. The loopback address removes DNS, nginx, and TLS from the
|
|
alerting path.
|
|
|
|
For jobs where silence is also a failure, pair ntfy with a dead man's switch such
|
|
as Healthchecks. ntfy tells you what the script found; Healthchecks tells you when
|
|
the script did not run at all.
|