mirror of
https://github.com/NickoScope/nickol-knx-mcp.git
synced 2026-10-22 06:20:48 +02:00
feat(skills): ship ha-git-backup — two-circuit HA backup ops-companion skill
Circuit 1: real git in /config (deploy key, pre-commit secret scanner, meaningful commits). Circuit 2: age-encrypted native HA backups in GitHub Releases with rotation and a mandatory monthly restore drill. Ships install/sync/scan/offsite/restore scripts, HA automations/snippets, setup + architecture + incident + runbook references, EN + RU skill docs. README/README.ru gain an ops-companion bullet in Scenario 3; CHANGELOG [Unreleased] Added entry (no version bump). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
74ac2ac830
commit
1aea8a6d9b
@@ -8,6 +8,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Added
|
||||
|
||||
- **`skills/ha-git-backup`** — an ops-companion skill for the engineer package: a two-circuit
|
||||
Home Assistant backup system (real git in `/config` with a deploy key and a pre-commit secret
|
||||
scanner + age-encrypted full backups in GitHub Releases), with install/sync/scan/offsite/restore
|
||||
scripts, HA automations, a restore runbook and a mandatory monthly drill. Field-tested;
|
||||
the secret scanner ships canary-tested (BusyBox `grep -e` lesson recorded).
|
||||
|
||||
- **Manufacturer names from the archive itself** (`appprog_parser.py`): `parse_devices_from_project`
|
||||
now reads vendor names from the `knx_master.xml` shipped inside every `.knxproj`/`.knxprod`
|
||||
(e.g. M-0073 → HDL, M-00B6 → Ekinex S.p.A.) instead of relying on a hardcoded id map that can
|
||||
|
||||
@@ -160,6 +160,9 @@ spec encodes.
|
||||
KNX IoT (Turtle/RDF) semantic export.
|
||||
- Live control of the house stays in the official Home Assistant integration (layer 1) — this
|
||||
server only prepares its configuration.
|
||||
- **Ops companion**: [`skills/ha-git-backup`](skills/ha-git-backup) — the life of your config
|
||||
*after* deploy: a real git history of `/config` (deploy key + pre-commit secret scanner) plus
|
||||
encrypted offsite backups in GitHub Releases, with a monthly restore drill.
|
||||
|
||||
### 🧩 The foundation — a growing device library
|
||||
|
||||
|
||||
@@ -138,6 +138,9 @@ KNX Community в мае 2026 прямо просит такую интеграц
|
||||
семантический экспорт KNX IoT (Turtle/RDF).
|
||||
- Живое управление домом остаётся в официальной интеграции Home Assistant (слой 1) — этот сервер
|
||||
только готовит её конфигурацию.
|
||||
- **Ops-компаньон**: [`skills/ha-git-backup`](skills/ha-git-backup) — жизнь конфига *после*
|
||||
деплоя: настоящая git-история `/config` (deploy key + pre-commit сканер секретов) плюс
|
||||
шифрованные offsite-бэкапы в GitHub Releases и ежемесячный restore drill.
|
||||
|
||||
### 🧩 Фундамент — растущая библиотека устройств
|
||||
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
# ha-git-backup — the ops companion skill
|
||||
|
||||
Our Scenario 3 ends with `generate_ha_package` handing you a Home Assistant KNX YAML. This
|
||||
skill covers what happens **after you deploy it**: the life of your `/config`.
|
||||
|
||||
**Two circuits, two different jobs:**
|
||||
|
||||
1. **History** — a *real* git repo inside `/config`, pushed to a private GitHub repo over a
|
||||
**deploy key** (blast radius: one repo). Every change is a meaningful commit
|
||||
(`[daily] 3 file(s) | HA 2026.7.1`); a **pre-commit secret scanner** blocks any commit whose
|
||||
diff contains a token, password or private key. Answers "what broke my automation last night"
|
||||
in one `git diff`.
|
||||
2. **Recovery** — full native HA backups (including `.storage`: entity registry, auth, UI
|
||||
dashboards) **age-encrypted** and uploaded to GitHub Releases with rotation. The private age
|
||||
key never touches the HA host — a compromised host cannot read its own backups. Answers
|
||||
"the SSD died" with a 30-minute restore.
|
||||
|
||||
Plus: HA automations (daily commit at 03:30, a pre-update tag, a sync button, a failure alert),
|
||||
a health sensor, a restore runbook and a **mandatory monthly restore drill** — because a backup
|
||||
you have never restored is a hypothesis, not a backup.
|
||||
|
||||
## Quick start
|
||||
|
||||
```text
|
||||
1. Create an empty PRIVATE GitHub repo (e.g. ha-config).
|
||||
2. Follow references/setup.md: deploy key → scripts to /config/.git-sync/bin → install.sh.
|
||||
3. Add assets/configuration_snippet.yaml + assets/automations.yaml, restart HA (full restart).
|
||||
4. Circuit 2: age key + fine-grained PAT (Contents RW, ONE repo) → backup_offsite.sh.
|
||||
5. Run the first restore drill immediately (references/restore-runbook.md).
|
||||
```
|
||||
|
||||
Start with [SKILL.md](SKILL.md) — it is written as a Claude skill (drop it into
|
||||
`.claude/skills/` and Claude will walk you through installation, incidents and drills), but it
|
||||
reads just as well as plain documentation for humans.
|
||||
|
||||
> Field-tested: the secret scanner ships canary-tested (see LESSON-05 — a BusyBox grep quirk
|
||||
> silently disabled the private-key pattern until a planted-secret test caught it).
|
||||
@@ -0,0 +1,102 @@
|
||||
---
|
||||
name: ha-git-backup
|
||||
description: >-
|
||||
Two-circuit backup & GitHub sync system for Home Assistant. Circuit 1 — real git inside
|
||||
/config with a deploy key, a pre-commit secret scanner and meaningful commits; Circuit 2 —
|
||||
full encrypted HA backups in GitHub Releases. Use EVERY time the task involves: backing up
|
||||
Home Assistant, syncing its config, git in /config, restoring a config, rolling back a YAML
|
||||
change, "what broke my automation", a leaked secret, setting up a deploy key, migrating HA
|
||||
to new hardware, GitHub Releases for backups — and BEFORE any HA Core/OS update or AFTER any
|
||||
incident with config loss. Also trigger on: backup, restore, gitleaks, secrets.yaml, 3-2-1.
|
||||
The skill ships ready-made scripts (install, sync, secret-scan, offsite, restore) and a
|
||||
mandatory protocol — do NOT reinvent sync logic, use scripts/.
|
||||
---
|
||||
|
||||
# HA Git Backup — a two-circuit system
|
||||
|
||||
Philosophy: **git gives you history, a backup gives you recovery. These are different jobs —
|
||||
different circuits solve them.** Projects like GithubConfigSync mix the two and implement git
|
||||
on top of the Contents API — we use real git.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
CIRCUIT 1: CONFIG HISTORY (what changed and when)
|
||||
/config (git repo) ──deploy key──▶ GitHub private repo (ha-config)
|
||||
├── triggers: daily 03:30 / pre-update / button / HA restart
|
||||
├── secret protection: .gitignore → pre-commit scan → age mirror
|
||||
└── commit: "[trigger] N files | HA 2026.7.1" + file list
|
||||
|
||||
CIRCUIT 2: RECOVERY (full state, including .storage)
|
||||
HA native backup (.tar) ──age──▶ GitHub Releases (ha-config)
|
||||
├── schedule: weekly + before updates
|
||||
├── rotation: 8 releases
|
||||
└── restore drill: 1st of the month (mandatory)
|
||||
```
|
||||
|
||||
The 3-2-1 rule: the config lives (1) on the HA host, (2) in the GitHub git repo, (3) as a full
|
||||
tarball in Releases — plus a local NAS copy if you have one.
|
||||
|
||||
## IRON RULES
|
||||
|
||||
1. **Deploy key, not a PAT.** Fine-grained WRITE access to ONE repository. A classic token with
|
||||
the `repo` scope = access to all your private repos → forbidden.
|
||||
2. **A secret in the diff blocks the commit.** The pre-commit hook runs `secret_scan.sh`.
|
||||
Bypassing with `--no-verify` — only deliberately, with the reason logged.
|
||||
3. **`.storage/` does NOT go to git** (auth, tokens), except the `lovelace*` whitelist —
|
||||
dashboards. The full `.storage` lives only in the encrypted Circuit-2 tarball.
|
||||
4. **A backup without a restore drill is not a backup.** Monthly: download the release →
|
||||
`age -d` → `tar -t` → verify `.storage/core.config_entries` is inside.
|
||||
5. **Do not rewrite the scripts from scratch.** The logic (lock, retry, notify, rotation) is
|
||||
already in `scripts/` — read and use them.
|
||||
|
||||
## Quick start (fresh install)
|
||||
|
||||
1. Create an empty private repo `ha-config` on GitHub.
|
||||
2. Read `references/setup.md` — step by step: deploy key, install via the SSH add-on.
|
||||
3. Run `scripts/install.sh` inside HA — it does git init, .gitignore, hook, remote, first dry-run.
|
||||
4. Add the contents of `assets/configuration_snippet.yaml` to configuration.yaml and
|
||||
`assets/automations.yaml` to your automations. **A full HA restart is required**
|
||||
(`shell_command` and `command_line` are NOT loaded by a quick reload).
|
||||
5. Circuit 2: `scripts/backup_offsite.sh` needs a fine-grained PAT + an age key.
|
||||
See `references/setup.md` §4.
|
||||
6. Run the first restore drill IMMEDIATELY — before the system is ever needed.
|
||||
|
||||
## Symptom → action map
|
||||
|
||||
| Symptom | Action |
|
||||
|---|---|
|
||||
| "Worked yesterday, broken today" | `git log --oneline -10`, `git diff HEAD~1` in /config — see what changed |
|
||||
| Broke YAML, HA won't start | `git checkout -- <file>` or `git reset --hard <good_sha>`, restart |
|
||||
| Secret leaked into the repo | `references/incident-secret-leak.md` — rotate the secret FIRST, history second |
|
||||
| Push silent/failing | `/config/.git-sync/log`; typical: read-only deploy key, stale known_hosts |
|
||||
| Migrating to new hardware | `scripts/restore.sh` — the Circuit-2 tarball, NOT the git repo (git has no .storage) |
|
||||
| Sync sensor = error | Check the log; the automation already sends a persistent_notification |
|
||||
| Repo bloated | Check binaries (db, tar) are gitignored; history is cleaned with `git gc` |
|
||||
|
||||
## What's in the skill
|
||||
|
||||
- `scripts/install.sh` — bootstrap the git repo in /config
|
||||
- `scripts/ha_git_sync.sh` — the sync engine: lock → scan → commit → push with retry → status file
|
||||
- `scripts/secret_scan.sh` — diff-based secret scanner (keys, tokens, private keys, passwords not via !secret)
|
||||
- `scripts/backup_offsite.sh` — Circuit 2: age encryption + upload to GitHub Releases + rotation
|
||||
- `scripts/restore.sh` — interactive restore with a checklist
|
||||
- `assets/gitignore.template`, `assets/configuration_snippet.yaml`, `assets/automations.yaml`
|
||||
- `references/setup.md` — full installation from scratch (deploy key, age, PAT)
|
||||
- `references/architecture.md` — design rationale, comparison with alternatives
|
||||
- `references/incident-secret-leak.md` — the secret-leak incident protocol
|
||||
- `references/restore-runbook.md` — restore runbook and the monthly drill
|
||||
|
||||
## LESSONS (recorded — do not repeat)
|
||||
|
||||
- **LESSON-01**: the GithubConfigSync approach (files one by one via the Contents API) = a commit
|
||||
per file, rate limits, snapshot duplicates in the git repo. Real git solves all of that for free.
|
||||
- **LESSON-02**: .gitignore is NOT protection. A file added before the rule keeps being tracked.
|
||||
Hence the mandatory pre-commit scan.
|
||||
- **LESSON-03**: a config git repo ≠ a backup: without .storage a restore gives you a bare HA
|
||||
with no entity registry, no auth, no UI dashboards.
|
||||
- **LESSON-04**: a restore drill in calm times costs 10 minutes. A first-ever restore during an
|
||||
outage without a drill costs an evening and your nerves.
|
||||
- **LESSON-05**: BusyBox grep treats patterns starting with `-` as options — always pass scan
|
||||
patterns with `grep -E -e "$pattern"`. The scanner was silently skipping its private-key
|
||||
pattern until this was caught by a canary test. Test your scanner with planted secrets.
|
||||
@@ -0,0 +1,76 @@
|
||||
# HA Git Backup — двухконтурная система (русская версия; каноническая — [SKILL.md](SKILL.md))
|
||||
|
||||
Философия: **git даёт историю, бэкап даёт восстановление. Это разные задачи — их решают разные
|
||||
контуры.** Проекты типа GithubConfigSync смешивают их и реализуют git поверх Contents API — мы
|
||||
используем настоящий git.
|
||||
|
||||
## Архитектура
|
||||
|
||||
```
|
||||
КОНТУР 1: ИСТОРИЯ КОНФИГА (что изменилось и когда)
|
||||
/config (git repo) ──deploy key──▶ GitHub private repo (ha-config)
|
||||
├── триггеры: daily 03:30 / pre-update / кнопка / рестарт HA
|
||||
├── защита секретов: .gitignore → pre-commit scan → age-зеркало
|
||||
└── коммит: "[trigger] N files | HA 2026.7.1" + список файлов
|
||||
|
||||
КОНТУР 2: ВОССТАНОВЛЕНИЕ (полное состояние, включая .storage)
|
||||
HA native backup (.tar) ──age──▶ GitHub Releases (ha-config)
|
||||
├── расписание: еженедельно + перед обновлениями
|
||||
├── ротация: 8 релизов
|
||||
└── restore drill: 1-е число месяца (обязательный)
|
||||
```
|
||||
|
||||
Правило 3-2-1: конфиг живёт (1) на хосте HA, (2) в git-репо GitHub, (3) полный тарбол в
|
||||
Releases + локальный NAS при наличии.
|
||||
|
||||
## ЖЕЛЕЗНЫЕ ПРАВИЛА
|
||||
|
||||
1. **Deploy key, не PAT.** Fine-grained доступ на ЗАПИСЬ в ОДИН репозиторий. Classic-токен со
|
||||
scope `repo` = доступ ко всем приватным репо → запрещён.
|
||||
2. **Секрет в diff = коммит блокируется.** Pre-commit hook вызывает `secret_scan.sh`. Обход
|
||||
через `--no-verify` — только осознанно и с записью причины в лог.
|
||||
3. **`.storage/` НЕ идёт в git** (auth, токены), кроме whitelist `lovelace*` — дашборды.
|
||||
Полное `.storage` живёт только в шифрованном тарболе Контура 2.
|
||||
4. **Бэкап без restore drill — не бэкап.** Раз в месяц: скачать релиз → `age -d` → `tar -t` →
|
||||
проверить наличие `.storage/core.config_entries`.
|
||||
5. **Не переписывать скрипты с нуля.** Логика (lock, retry, notify, ротация) уже в `scripts/`
|
||||
— читать и использовать их.
|
||||
|
||||
## Быстрый старт (новая установка)
|
||||
|
||||
1. Создать приватный репо `ha-config` на GitHub (пустой).
|
||||
2. Прочитать `references/setup.md` — там пошагово: deploy key, установка через SSH add-on.
|
||||
3. Запустить `scripts/install.sh` внутри HA — он делает git init, .gitignore, hook, remote,
|
||||
первый dry-run.
|
||||
4. Добавить `assets/configuration_snippet.yaml` в configuration.yaml и `assets/automations.yaml`
|
||||
в автоматизации. **Нужен ПОЛНЫЙ перезапуск HA** (quick-reload не грузит
|
||||
`shell_command`/`command_line`).
|
||||
5. Контур 2: `scripts/backup_offsite.sh` требует fine-grained PAT + age-ключ.
|
||||
См. `references/setup.md` §4.
|
||||
6. Провести первый restore drill СРАЗУ — до того, как система понадобится.
|
||||
|
||||
## Карта симптомов → действия
|
||||
|
||||
| Симптом | Действие |
|
||||
|---|---|
|
||||
| «Вчера работало, сегодня нет» | `git log --oneline -10`, `git diff HEAD~1` в /config |
|
||||
| Сломал YAML, HA не стартует | `git checkout -- <file>` или `git reset --hard <good_sha>`, рестарт |
|
||||
| Утёк секрет в репо | `references/incident-secret-leak.md` — ротация секрета ПЕРВОЙ |
|
||||
| Push молчит/падает | `/config/.git-sync/log`; типовое: deploy key read-only, протух known_hosts |
|
||||
| Переезд на новое железо | `scripts/restore.sh` — тарбол Контура 2, НЕ git-репо (в git нет .storage) |
|
||||
| Sync-сенсор = error | Смотреть лог; автоматизация шлёт persistent_notification |
|
||||
| Репо распух | Бинарники (db, tar) в .gitignore; история чистится `git gc` |
|
||||
|
||||
## УРОКИ (зафиксированы, не повторять)
|
||||
|
||||
- **УРОК-01**: GithubConfigSync-подход (файлы по одному через Contents API) = коммит на файл,
|
||||
rate limits, снапшоты-дубли. Настоящий git решает всё это бесплатно.
|
||||
- **УРОК-02**: .gitignore — НЕ защита. Файл, добавленный до правила, продолжает трекаться.
|
||||
Отсюда обязательный pre-commit scan.
|
||||
- **УРОК-03**: git-репо конфига ≠ бэкап: без .storage восстановление даёт голый HA без entity
|
||||
registry, авторизаций и дашбордов из UI.
|
||||
- **УРОК-04**: restore drill в спокойное время стоит 10 минут. Первое восстановление во время
|
||||
аварии без drill стоит вечер и нервы.
|
||||
- **УРОК-05**: BusyBox-grep принимает паттерн, начинающийся с `-`, за опцию — паттерны сканера
|
||||
передавать только как `grep -E -e "$pattern"`. Сканер молча пропускал проверку приватных
|
||||
ключей, пока это не поймал тест с подсадной уткой. Проверяй сканер подсадными секретами.
|
||||
@@ -0,0 +1,70 @@
|
||||
# ha-git-backup: automations. Add to automations.yaml or via the UI.
|
||||
|
||||
- alias: "GitSync - daily commit"
|
||||
triggers:
|
||||
- trigger: time
|
||||
at: "03:30:00"
|
||||
actions:
|
||||
- action: shell_command.git_sync_daily
|
||||
|
||||
- alias: "GitSync - button"
|
||||
triggers:
|
||||
- trigger: state
|
||||
entity_id: input_button.git_sync_now
|
||||
actions:
|
||||
- action: shell_command.git_sync_manual
|
||||
|
||||
- alias: "GitSync - tag before a Core update"
|
||||
triggers:
|
||||
- trigger: state
|
||||
entity_id: update.home_assistant_core_update
|
||||
to: "on"
|
||||
actions:
|
||||
- action: shell_command.git_sync_pre_update
|
||||
|
||||
- alias: "GitSync - commit after restart (capture the surviving state)"
|
||||
triggers:
|
||||
- trigger: homeassistant
|
||||
event: start
|
||||
actions:
|
||||
- delay: "00:03:00"
|
||||
- action: shell_command.git_sync_manual
|
||||
|
||||
- alias: "GitSync - alert on failure"
|
||||
triggers:
|
||||
- trigger: state
|
||||
entity_id: sensor.git_sync_status
|
||||
conditions:
|
||||
- condition: template
|
||||
value_template: "{{ 'error' in states('sensor.git_sync_status') }}"
|
||||
actions:
|
||||
- action: persistent_notification.create
|
||||
data:
|
||||
title: "⚠️ Git Sync FAILED"
|
||||
message: "{{ states('sensor.git_sync_status') }} — see /config/.git-sync/log"
|
||||
# + add your notify.mobile_app_* here
|
||||
|
||||
- alias: "Backup - weekly full + offsite"
|
||||
triggers:
|
||||
- trigger: time
|
||||
at: "04:00:00"
|
||||
conditions:
|
||||
- condition: time
|
||||
weekday: [sun]
|
||||
actions:
|
||||
- action: backup.create # native HA backup (including .storage)
|
||||
- delay: "00:10:00"
|
||||
- action: shell_command.backup_offsite
|
||||
|
||||
- alias: "Backup - restore drill reminder"
|
||||
triggers:
|
||||
- trigger: time
|
||||
at: "10:00:00"
|
||||
conditions:
|
||||
- condition: template
|
||||
value_template: "{{ now().day == 1 }}"
|
||||
actions:
|
||||
- action: persistent_notification.create
|
||||
data:
|
||||
title: "📋 Restore drill"
|
||||
message: "1st of the month: verify recovery (restore.sh full, DRILL mode)."
|
||||
@@ -0,0 +1,21 @@
|
||||
# ha-git-backup: add to configuration.yaml
|
||||
# Paths assume scripts live in /config/.git-sync/bin — adjust otherwise.
|
||||
# NOTE: shell_command and command_line load only on a FULL HA restart.
|
||||
|
||||
shell_command:
|
||||
git_sync_manual: "/config/.git-sync/bin/ha_git_sync.sh manual"
|
||||
git_sync_daily: "/config/.git-sync/bin/ha_git_sync.sh daily"
|
||||
git_sync_pre_update: "/config/.git-sync/bin/ha_git_sync.sh pre-update"
|
||||
backup_offsite: "GH_REPO={{ 'YOUR_USER/ha-config' }} /config/.git-sync/bin/backup_offsite.sh"
|
||||
|
||||
# Health sensor: reads the status file written by ha_git_sync.sh
|
||||
command_line:
|
||||
- sensor:
|
||||
name: git_sync_status
|
||||
command: "cat /config/.git-sync/last_status 2>/dev/null || echo 'never'"
|
||||
scan_interval: 300
|
||||
|
||||
input_button:
|
||||
git_sync_now:
|
||||
name: "Git Sync Now"
|
||||
icon: mdi:source-branch-sync
|
||||
@@ -0,0 +1,43 @@
|
||||
# ha-git-backup: echelon 1 of protection. Do NOT loosen without reading SKILL.md
|
||||
# --- secrets and state ---
|
||||
secrets.yaml
|
||||
*.pem
|
||||
*.key
|
||||
known_devices.yaml
|
||||
.storage/*
|
||||
!.storage/lovelace*
|
||||
.cloud/
|
||||
.google.token
|
||||
|
||||
# --- databases and runtime junk (no place in git; Circuit 2 covers them) ---
|
||||
*.db
|
||||
*.db-*
|
||||
*.log
|
||||
*.log.*
|
||||
home-assistant.log*
|
||||
OZW_Log.txt
|
||||
deps/
|
||||
tts/
|
||||
image/
|
||||
www/community/
|
||||
__pycache__/
|
||||
.cache/
|
||||
.ha_run.lock
|
||||
|
||||
# --- backups are not stored in the git repo ---
|
||||
backups/
|
||||
*.tar
|
||||
*.tar.age
|
||||
|
||||
# --- embedded git repos keep their own history (content is covered by Circuit 2) ---
|
||||
esphome/
|
||||
|
||||
# --- this system's own runtime files ---
|
||||
.git-sync/log
|
||||
.git-sync/last_status
|
||||
.git-sync/last_offsite
|
||||
.git-sync/gh_token
|
||||
.git-sync/age_recipient
|
||||
.git-sync/deploy_key
|
||||
.git-sync/deploy_key.pub
|
||||
.HA_VERSION
|
||||
@@ -0,0 +1,41 @@
|
||||
# Architecture: design rationale
|
||||
|
||||
## Why two circuits, not one
|
||||
Git answers "what changed and when" (diffs, blame, file rollback in seconds). A backup answers
|
||||
"how do I bring the system up from nothing" (.storage: entity registry, auth, config entries,
|
||||
the zigbee network). Mixing the two jobs produces a system that does both badly — the main
|
||||
lesson from analysing GithubConfigSync.
|
||||
|
||||
## Comparison with alternatives
|
||||
| Solution | History | Full recovery | Secrets | Verdict |
|
||||
|---|---|---|---|---|
|
||||
| **This system** | real git | Releases + age | 3 echelons | ✅ |
|
||||
| GithubConfigSync | pseudo-git (Contents API, commit per file) | no (.storage excluded) | .gitignore only | history — yes, restore — no |
|
||||
| Git Pull add-on (official) | pull only | no | — | a different job (deploy) |
|
||||
| Google Drive Backup add-on | no | yes | HA encryption | great as an ADDITION to Circuit 2 (second offsite copy = full 3-2-1) |
|
||||
| Bare cron+git | yes | no | manual | that IS Circuit 1, but without scan/notify/lock |
|
||||
|
||||
## Key decisions
|
||||
1. **Deploy key instead of a PAT for git** — blast radius of one repo.
|
||||
2. **Fine-grained PAT only for the Releases API** (Contents RW, one repo) — Releases can't be
|
||||
pushed over SSH.
|
||||
3. **The pre-commit scan is mandatory**: .gitignore does not protect against a secret written
|
||||
INTO automations.yaml as a literal (a real, frequent case: a Telegram bot token in a
|
||||
rest_command).
|
||||
4. **Whitelist `.storage/lovelace*`**: dashboards created in the UI live in .storage — without
|
||||
them the config history is incomplete. The rest of .storage holds auth → encrypted Circuit 2
|
||||
only.
|
||||
5. **age, not GPG**: a single binary, no keyring state, the key is one line. There is NO private
|
||||
key on the HA host — a compromised host ≠ readable backups.
|
||||
6. **GitHub Releases, not git-lfs and not files in the repo**: binaries in git bloat history
|
||||
forever; Releases is flat storage with a 2 GB/file limit and trivial rotation.
|
||||
7. **A status file + command_line sensor, not push notifications from the script**: the script
|
||||
does not depend on HA API tokens; observability is done by HA itself.
|
||||
8. **Locking via flock**: an HA restart plus the daily trigger cannot race.
|
||||
|
||||
## System boundaries (deliberate)
|
||||
- The 2 GB Releases limit: exclude media/large DBs from the native backup (partial backup).
|
||||
- The git repo in /config is visible to every add-on with config access — threat model:
|
||||
trusted add-ons.
|
||||
- Git history keeps deleted files forever: a leaked secret requires the
|
||||
incident-secret-leak.md protocol, not a simple delete.
|
||||
@@ -0,0 +1,32 @@
|
||||
# Incident: a secret landed in the git repo
|
||||
|
||||
The order is STRICT. Step 1 comes before everything else.
|
||||
|
||||
## 1. ROTATE THE SECRET (immediately)
|
||||
A leaked token/password/key counts as compromised from the moment of push, even in a private
|
||||
repo. Revoke and reissue it at the provider (GitHub token → revoke; a service API key →
|
||||
regenerate; a password → change). Only then move on to cleanup.
|
||||
|
||||
## 2. Assess the exposure
|
||||
- `git log --all -S '<secret fragment>' --oneline` — which commits carry it.
|
||||
- Is the repo private? Any forks/clones? Who had access?
|
||||
|
||||
## 3. Rewrite history
|
||||
```bash
|
||||
# on your workstation, not on HA:
|
||||
git clone git@github.com:YOUR_USER/ha-config.git && cd ha-config
|
||||
pip install git-filter-repo
|
||||
git filter-repo --replace-text <(echo '<secret>==>REDACTED')
|
||||
git push --force --all && git push --force --tags
|
||||
```
|
||||
On HA afterwards: `cd /config && git fetch && git reset --hard origin/main`.
|
||||
|
||||
## 4. GitHub specifics
|
||||
A force-push does NOT delete objects instantly: commits stay reachable by SHA via cache/API.
|
||||
For a full purge — GitHub Support (a request to delete unreachable objects). One more reason
|
||||
step 1 is primary: history may never be fully scrubbed; rotation closes the risk with certainty.
|
||||
|
||||
## 5. Close the gap
|
||||
- Why didn't the scan catch it? → add the pattern to `secret_scan.sh` PATTERNS.
|
||||
- The value should have lived in secrets.yaml behind `!secret` — move it.
|
||||
- Record a LESSON in SKILL.md.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Restore runbook
|
||||
|
||||
## Scenario A: a broken file/config (frequent) — Circuit 1
|
||||
```bash
|
||||
cd /config
|
||||
git log --oneline -10 # find the last known-good commit
|
||||
git diff <sha> --stat # what changed
|
||||
git checkout <sha> -- automations.yaml # surgical rollback
|
||||
ha core check && ha core restart
|
||||
```
|
||||
Time: minutes. If HA won't start at all — same commands via the SSH add-on (it lives
|
||||
independently of Core).
|
||||
|
||||
## Scenario B: dead SD/SSD/hardware — Circuit 2
|
||||
1. Fresh HA OS install on the new medium.
|
||||
2. On your workstation: download the latest `backup/*` release from `ha-config`, decrypt:
|
||||
`age -d -i age_key.txt -o restored.tar ha-backup-*.tar.age`
|
||||
3. New HA onboarding → "Restore from backup" → upload restored.tar.
|
||||
4. After start verify: entity registry intact, auth alive, dashboards present, ESPHome nodes
|
||||
reconnected, zigbee/z-wave network up.
|
||||
5. Verify Circuit 1: `cd /config && git status` (the repo arrives inside the backup, .git and all).
|
||||
|
||||
## Scenario C: need a year-old config
|
||||
Circuit 1: `git log --before="2025-07-01" -1` → checkout the needed file from that SHA.
|
||||
|
||||
## The monthly DRILL (1st of the month, reminder ships in automations.yaml)
|
||||
The goal — prove the "Releases → age → tar" chain is alive WITHOUT a real restore:
|
||||
1. Download the latest release.
|
||||
2. `age -d` with the private key (this doubles as "the key isn't lost" check!).
|
||||
3. `tar -tf restored.tar | grep -c .` — the archive reads.
|
||||
4. `tar -tf restored.tar | grep core.config_entries` — .storage is inside.
|
||||
5. `date +%F > /config/.git-sync/last_drill`
|
||||
Any step failing = an incident: fix now, not when the SSD burns.
|
||||
|
||||
## System health metrics
|
||||
- `sensor.git_sync_status` = ok, age < 26 h
|
||||
- `.git-sync/last_offsite` — age < 8 days
|
||||
- `.git-sync/last_drill` — age < 35 days
|
||||
@@ -0,0 +1,61 @@
|
||||
# Installation from scratch
|
||||
|
||||
## §1. Prerequisites
|
||||
- HA OS / Supervised with the **Advanced SSH & Web Terminal** add-on (it ships git and ssh).
|
||||
- A private repo `ha-config` on GitHub (empty, no README).
|
||||
- Copy the skill's scripts to `/config/.git-sync/bin/` and the gitignore template to
|
||||
`/config/.git-sync/assets/`, then `chmod +x` the scripts.
|
||||
- Note: with `sftp: false` in the add-on config, `scp` won't work — use a tar pipe:
|
||||
`tar cz scripts | ssh <your-ha-host> 'cd /tmp && tar xz'`.
|
||||
|
||||
## §2. Deploy key (Circuit 1) — NOT a PAT
|
||||
```bash
|
||||
ssh-keygen -t ed25519 -f /config/.git-sync/deploy_key -N "" -C "ha-git-backup"
|
||||
cat /config/.git-sync/deploy_key.pub
|
||||
```
|
||||
GitHub → repo `ha-config` → Settings → Deploy keys → Add key → **Allow write access**.
|
||||
|
||||
SSH config so git picks exactly this key:
|
||||
```bash
|
||||
mkdir -p ~/.ssh && cat >> ~/.ssh/config <<'EOF'
|
||||
Host github.com
|
||||
IdentityFile /config/.git-sync/deploy_key
|
||||
IdentitiesOnly yes
|
||||
EOF
|
||||
```
|
||||
Why a deploy key: it grants access to ONE repo. A compromised key ≠ a compromised GitHub
|
||||
account (unlike a classic PAT with the `repo` scope).
|
||||
|
||||
## §3. Circuit 1: install
|
||||
```bash
|
||||
/config/.git-sync/bin/install.sh git@github.com:YOUR_USER/ha-config.git
|
||||
# review the dry-run file list for secrets, then:
|
||||
/config/.git-sync/bin/ha_git_sync.sh manual
|
||||
```
|
||||
Add `assets/configuration_snippet.yaml` and `assets/automations.yaml`, **fully restart HA**
|
||||
(quick reload does not load `shell_command`/`command_line`), check `sensor.git_sync_status`.
|
||||
|
||||
## §4. Circuit 2: offsite to GitHub Releases
|
||||
1. **age key** (generate NOT on HA but on your workstation):
|
||||
`age-keygen -o age_key.txt` → store the private key offline (password manager / paper).
|
||||
Only the public half goes to HA:
|
||||
`echo "age1..." > /config/.git-sync/age_recipient`
|
||||
2. **Fine-grained PAT**: GitHub → Settings → Developer settings → Fine-grained tokens →
|
||||
access to `ha-config` only, permission **Contents: Read and write**. 1-year expiry,
|
||||
calendar reminder to rotate.
|
||||
`echo "github_pat_..." > /config/.git-sync/gh_token && chmod 600 /config/.git-sync/gh_token`
|
||||
3. `age` inside the SSH add-on container: add `age` to the add-on's `packages:` list (persists
|
||||
across restarts) or `apk add age`.
|
||||
4. Test: `GH_REPO=YOUR_USER/ha-config /config/.git-sync/bin/backup_offsite.sh`
|
||||
|
||||
## §5. Optional: an age mirror of secrets.yaml (echelon 3)
|
||||
To make the git repo self-sufficient for restoring the YAML part:
|
||||
```bash
|
||||
age -r "$(cat /config/.git-sync/age_recipient)" -o /config/secrets.yaml.age /config/secrets.yaml
|
||||
```
|
||||
`secrets.yaml.age` is NOT gitignored → it goes to git. Add the command to ha_git_sync.sh before
|
||||
`git add` if you enable this. Decryption requires only the private key, which is not on HA.
|
||||
|
||||
## §6. Final — the mandatory first drill
|
||||
Right after installation run `restore.sh full` in DRILL mode (steps 1–4). Record the date in
|
||||
`/config/.git-sync/last_drill`.
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
#!/usr/bin/env bash
|
||||
# backup_offsite.sh — Circuit 2: freshest native HA backup → age encryption →
|
||||
# GitHub Release in the ha-config repo. Rotation: keep the KEEP latest releases.
|
||||
# Requires: age, curl, a fine-grained PAT (Contents RW of ONE repo) in
|
||||
# /config/.git-sync/gh_token, the public age key in /config/.git-sync/age_recipient
|
||||
set -euo pipefail
|
||||
|
||||
CONFIG_DIR="${CONFIG_DIR:-/config}"
|
||||
BACKUP_DIR="${BACKUP_DIR:-/backup}" # native backup path on supervised/HAOS
|
||||
REPO="${GH_REPO:?export GH_REPO=user/ha-config}"
|
||||
KEEP="${KEEP:-8}"
|
||||
TOKEN="$(cat "$CONFIG_DIR/.git-sync/gh_token")"
|
||||
RECIPIENT="$(cat "$CONFIG_DIR/.git-sync/age_recipient")"
|
||||
API="https://api.github.com/repos/$REPO"
|
||||
AUTH=(-H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.github+json")
|
||||
LOG="$CONFIG_DIR/.git-sync/log"
|
||||
log(){ echo "$(date '+%F %T') [offsite] $*" >> "$LOG"; }
|
||||
|
||||
# 1. The freshest native backup (creating it is the HA automation's job, before this call)
|
||||
LATEST="$(ls -t "$BACKUP_DIR"/*.tar 2>/dev/null | head -1)"
|
||||
[ -n "$LATEST" ] || { log "FAIL: no .tar in $BACKUP_DIR"; exit 1; }
|
||||
|
||||
# 2. Encrypt (the HA tarball may already be password-encrypted — age on top is fine)
|
||||
STAMP="$(date +%Y-%m-%d_%H%M)"
|
||||
ENC="/tmp/ha-backup-$STAMP.tar.age"
|
||||
age -r "$RECIPIENT" -o "$ENC" "$LATEST"
|
||||
SIZE=$(du -m "$ENC" | cut -f1)
|
||||
[ "$SIZE" -lt 1900 ] || { log "FAIL: $SIZE MB > GitHub Releases limit (2GB) — exclude media from the backup"; rm -f "$ENC"; exit 1; }
|
||||
|
||||
# 3. Create the release
|
||||
TAG="backup/$STAMP"
|
||||
REL_JSON="$(curl -sf "${AUTH[@]}" -X POST "$API/releases" \
|
||||
-d "{\"tag_name\":\"$TAG\",\"name\":\"HA backup $STAMP\",\"body\":\"age-encrypted, $SIZE MB\"}")"
|
||||
REL_ID="$(echo "$REL_JSON" | grep -o '"id": *[0-9]*' | head -1 | grep -o '[0-9]*')"
|
||||
[ -n "$REL_ID" ] || { log "FAIL: release create"; rm -f "$ENC"; exit 1; }
|
||||
|
||||
# 4. Upload the asset
|
||||
curl -sf "${AUTH[@]}" -H "Content-Type: application/octet-stream" \
|
||||
--data-binary @"$ENC" \
|
||||
"https://uploads.github.com/repos/$REPO/releases/$REL_ID/assets?name=$(basename "$ENC")" >/dev/null \
|
||||
|| { log "FAIL: asset upload"; rm -f "$ENC"; exit 1; }
|
||||
rm -f "$ENC"
|
||||
log "OK: $TAG uploaded ($SIZE MB)"
|
||||
|
||||
# 5. Rotation: delete backup/* releases beyond KEEP (and their tags)
|
||||
curl -sf "${AUTH[@]}" "$API/releases?per_page=100" \
|
||||
| grep -oE '"tag_name": *"backup/[^"]*"|"id": *[0-9]*' \
|
||||
| paste - - | grep 'backup/' | sort -r | tail -n +$((KEEP+1)) \
|
||||
| while read -r idline tagline; do
|
||||
RID="$(echo "$idline" | grep -o '[0-9]*')"
|
||||
RTAG="$(echo "$tagline" | sed 's/.*"backup/backup/;s/"//g')"
|
||||
curl -sf "${AUTH[@]}" -X DELETE "$API/releases/$RID" >/dev/null || true
|
||||
curl -sf "${AUTH[@]}" -X DELETE "$API/git/refs/tags/$RTAG" >/dev/null || true
|
||||
log "rotated out: $RTAG"
|
||||
done
|
||||
echo "ok: offsite $TAG" > "$CONFIG_DIR/.git-sync/last_offsite"
|
||||
Executable
+65
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env bash
|
||||
# ha_git_sync.sh — Circuit 1: git sync /config → GitHub
|
||||
# Usage: ha_git_sync.sh [trigger] trigger: manual|daily|pre-update|restart
|
||||
# Exit: 0 = ok/nothing-to-do, 1 = error (recorded in the status file)
|
||||
set -uo pipefail
|
||||
|
||||
CONFIG_DIR="${CONFIG_DIR:-/config}"
|
||||
SYNC_DIR="$CONFIG_DIR/.git-sync"
|
||||
LOG="$SYNC_DIR/log"
|
||||
STATUS="$SYNC_DIR/last_status" # read by the HA command_line sensor
|
||||
LOCKFILE="/tmp/ha_git_sync.lock"
|
||||
TRIGGER="${1:-manual}"
|
||||
BRANCH="${GIT_BRANCH:-main}"
|
||||
RETRIES=3
|
||||
|
||||
mkdir -p "$SYNC_DIR"
|
||||
log() { echo "$(date '+%F %T') [$TRIGGER] $*" >> "$LOG"; }
|
||||
fail() { echo "error: $*" > "$STATUS"; log "FAIL: $*"; exit 1; }
|
||||
ok() { echo "ok: $*" > "$STATUS"; log "OK: $*"; exit 0; }
|
||||
|
||||
# --- lock: no parallel runs ---------------------------------------------------
|
||||
exec 9>"$LOCKFILE"
|
||||
flock -n 9 || fail "another sync is running"
|
||||
|
||||
cd "$CONFIG_DIR" || fail "no $CONFIG_DIR"
|
||||
[ -d .git ] || fail "not a git repo — run install.sh first"
|
||||
|
||||
# --- any changes? --------------------------------------------------------------
|
||||
git add -A
|
||||
if git diff --cached --quiet; then
|
||||
ok "clean, nothing to commit"
|
||||
fi
|
||||
|
||||
# --- secret scanner (echelon 2) -----------------------------------------------
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
if ! "$SCRIPT_DIR/secret_scan.sh"; then
|
||||
git reset -q # unstage everything, commit nothing
|
||||
fail "SECRET DETECTED in staged diff — commit blocked, see $LOG"
|
||||
fi
|
||||
|
||||
# --- a meaningful commit --------------------------------------------------------
|
||||
HA_VER="unknown"
|
||||
[ -f .HA_VERSION ] && HA_VER="$(cat .HA_VERSION)"
|
||||
NFILES=$(git diff --cached --name-only | wc -l | tr -d ' ')
|
||||
SUMMARY="[$TRIGGER] $NFILES file(s) | HA $HA_VER"
|
||||
BODY="$(git diff --cached --stat | tail -20)"
|
||||
|
||||
git commit -q -m "$SUMMARY" -m "$BODY" || fail "commit failed"
|
||||
log "committed: $SUMMARY"
|
||||
|
||||
# --- tag before an update -------------------------------------------------------
|
||||
if [ "$TRIGGER" = "pre-update" ]; then
|
||||
TAG="pre-update/$(date +%Y-%m-%d_%H%M)"
|
||||
git tag -f "$TAG" && log "tagged $TAG"
|
||||
fi
|
||||
|
||||
# --- push with retry and exponential backoff ------------------------------------
|
||||
n=0
|
||||
until git push -q origin "$BRANCH" --tags 2>>"$LOG"; do
|
||||
n=$((n+1))
|
||||
[ $n -ge $RETRIES ] && fail "push failed after $RETRIES attempts (commit is local, next run will push it)"
|
||||
sleep $((5 * 2 ** n))
|
||||
done
|
||||
|
||||
ok "$SUMMARY pushed"
|
||||
Executable
+57
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env bash
|
||||
# install.sh — Circuit 1 bootstrap: a git repo in /config.
|
||||
# Run ONCE from the SSH add-on. Requires: git, ssh, a created private repo and
|
||||
# a deploy key (see references/setup.md §2). Idempotent.
|
||||
set -euo pipefail
|
||||
|
||||
CONFIG_DIR="${CONFIG_DIR:-/config}"
|
||||
REMOTE="${1:-}" # git@github.com:YOUR_USER/ha-config.git
|
||||
BRANCH="${GIT_BRANCH:-main}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
[ -n "$REMOTE" ] || { echo "Usage: install.sh git@github.com:YOUR_USER/ha-config.git"; exit 1; }
|
||||
command -v git >/dev/null || { echo "git not found — use the Advanced SSH & Web Terminal add-on"; exit 1; }
|
||||
|
||||
cd "$CONFIG_DIR"
|
||||
mkdir -p .git-sync
|
||||
|
||||
# 1. The repository
|
||||
if [ ! -d .git ]; then
|
||||
git init -q -b "$BRANCH"
|
||||
echo "[+] git init ($BRANCH)"
|
||||
fi
|
||||
git config user.name "ha-git-backup"
|
||||
git config user.email "ha-git-backup@config.local"
|
||||
git remote get-url origin >/dev/null 2>&1 && git remote set-url origin "$REMOTE" || git remote add origin "$REMOTE"
|
||||
|
||||
# 2. .gitignore (echelon 1) — never overwrite an existing one
|
||||
if [ ! -f .gitignore ]; then
|
||||
cp "$SCRIPT_DIR/../assets/gitignore.template" .gitignore
|
||||
echo "[+] .gitignore installed"
|
||||
else
|
||||
echo "[i] .gitignore already exists — diff it against assets/gitignore.template manually"
|
||||
fi
|
||||
|
||||
# 3. pre-commit hook (echelon 2)
|
||||
mkdir -p .git/hooks
|
||||
cat > .git/hooks/pre-commit <<EOF
|
||||
#!/usr/bin/env bash
|
||||
exec "$SCRIPT_DIR/secret_scan.sh"
|
||||
EOF
|
||||
chmod +x .git/hooks/pre-commit .git-sync 2>/dev/null || true
|
||||
chmod +x "$SCRIPT_DIR"/*.sh
|
||||
echo "[+] pre-commit secret scan wired up"
|
||||
|
||||
# 4. SSH access check (deploy key)
|
||||
if ssh -o BatchMode=yes -o StrictHostKeyChecking=accept-new -T git@github.com 2>&1 | grep -q "successfully authenticated"; then
|
||||
echo "[+] deploy key works"
|
||||
else
|
||||
echo "[!] SSH to GitHub failed — check the deploy key (references/setup.md §2)"
|
||||
fi
|
||||
|
||||
# 5. First pass: show WHAT would enter the repo, before any real commit
|
||||
echo "[i] DRY-RUN — these files would land in the first commit:"
|
||||
git add -A -n | head -50
|
||||
echo "..."
|
||||
echo "Review the list above for secrets. If it looks right, run:"
|
||||
echo " $SCRIPT_DIR/ha_git_sync.sh manual"
|
||||
Executable
+35
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env bash
|
||||
# restore.sh — recovery. Two scenarios:
|
||||
# restore.sh file — roll back files from git (Circuit 1)
|
||||
# restore.sh full — full restore from a GitHub Release (Circuit 2)
|
||||
# Also used for the monthly restore drill (references/restore-runbook.md)
|
||||
set -euo pipefail
|
||||
MODE="${1:-help}"
|
||||
|
||||
case "$MODE" in
|
||||
file)
|
||||
cd "${CONFIG_DIR:-/config}"
|
||||
echo "Recent commits:"; git log --oneline -15
|
||||
echo
|
||||
echo "Roll back one file: git checkout <sha> -- <file>"
|
||||
echo "Roll back everything: git reset --hard <sha> (DESTRUCTIVE: wipes uncommitted changes)"
|
||||
echo "What changed: git diff <sha> --stat"
|
||||
echo "After rollback: validate the config (ha core check) and restart HA."
|
||||
;;
|
||||
full)
|
||||
REPO="${GH_REPO:?export GH_REPO=user/ha-config}"
|
||||
KEYFILE="${AGE_KEY:?export AGE_KEY=/path/to/age_key.txt}"
|
||||
echo "=== FULL RESTORE (Circuit 2) ==="
|
||||
echo "1. List backups: gh release list -R $REPO (or the /releases API)"
|
||||
echo "2. Download asset: gh release download <backup/TAG> -R $REPO -D /tmp"
|
||||
echo "3. Decrypt: age -d -i $KEYFILE -o restored.tar /tmp/ha-backup-*.tar.age"
|
||||
echo "4. Verify: tar -tf restored.tar | grep -c . && tar -tf restored.tar | grep core.config_entries"
|
||||
echo "5. HAOS: place restored.tar in /backup and restore via the UI"
|
||||
echo " (Settings → System → Backups), or onboarding-restore on a fresh install."
|
||||
echo "6. After start: verify entity registry, auth, dashboards, ESPHome nodes."
|
||||
echo
|
||||
echo "DRILL mode: run steps 1-4 WITHOUT step 5, record the date in .git-sync/last_drill"
|
||||
;;
|
||||
*)
|
||||
echo "Usage: restore.sh file|full"; exit 1;;
|
||||
esac
|
||||
Executable
+56
@@ -0,0 +1,56 @@
|
||||
#!/usr/bin/env bash
|
||||
# secret_scan.sh — echelon 2 of protection: scans the STAGED diff for secrets.
|
||||
# Exit: 0 = clean, 1 = secret found (the commit must be blocked).
|
||||
# Also installed as the pre-commit hook (install.sh wires it up).
|
||||
set -uo pipefail
|
||||
|
||||
CONFIG_DIR="${CONFIG_DIR:-/config}"
|
||||
LOG="$CONFIG_DIR/.git-sync/log"
|
||||
ALLOWLIST="$CONFIG_DIR/.git-sync/scan_allowlist" # exclusion lines (one per line)
|
||||
|
||||
# Only ADDED diff lines, without headers
|
||||
DIFF="$(git diff --cached --unified=0 | grep -E '^\+' | grep -vE '^\+\+\+' || true)"
|
||||
[ -z "$DIFF" ] && exit 0
|
||||
|
||||
# Apply the allowlist (false positives are recorded THERE, not by disabling the scanner)
|
||||
if [ -f "$ALLOWLIST" ]; then
|
||||
while IFS= read -r line; do
|
||||
[ -n "$line" ] && DIFF="$(echo "$DIFF" | grep -vF "$line" || true)"
|
||||
done < "$ALLOWLIST"
|
||||
fi
|
||||
|
||||
PATTERNS=(
|
||||
# private keys
|
||||
'-----BEGIN (RSA |EC |OPENSSH |DSA |PGP )?PRIVATE KEY'
|
||||
# cloud / API tokens of known formats
|
||||
'AKIA[0-9A-Z]{16}' # AWS access key
|
||||
'ghp_[A-Za-z0-9]{36}' # GitHub classic PAT
|
||||
'github_pat_[A-Za-z0-9_]{22,}' # GitHub fine-grained PAT
|
||||
'xox[baprs]-[A-Za-z0-9-]{10,}' # Slack
|
||||
'AIza[0-9A-Za-z_-]{35}' # Google API key
|
||||
'sk-[A-Za-z0-9_-]{20,}' # OpenAI/Anthropic-style
|
||||
'eyJhbGciOi[A-Za-z0-9_-]{20,}' # JWT / HA long-lived token
|
||||
# a secret-looking field assigned a literal value NOT via !secret
|
||||
'(password|api_key|token|client_secret|bearer)["'"'"']?\s*[:=]\s*["'"'"']?[A-Za-z0-9_\-\.\+/=]{8,}'
|
||||
)
|
||||
|
||||
HIT=""
|
||||
for p in "${PATTERNS[@]}"; do
|
||||
# NOTE: "-e" is mandatory — BusyBox grep treats a pattern starting with "-" as an
|
||||
# option and silently skips it otherwise (LESSON-05, caught by a canary test).
|
||||
M="$(echo "$DIFF" | grep -inE -e "$p" | grep -viE '!secret|REDACTED|example|changeme|password_here' || true)"
|
||||
[ -n "$M" ] && HIT="$HIT
|
||||
[$p]
|
||||
$M"
|
||||
done
|
||||
|
||||
if [ -n "$HIT" ]; then
|
||||
{
|
||||
echo "$(date '+%F %T') SECRET-SCAN BLOCKED COMMIT:"
|
||||
echo "$HIT" | head -40
|
||||
echo "--- False positive? Add the exact line to $ALLOWLIST"
|
||||
} >> "$LOG" 2>/dev/null || true
|
||||
echo "SECRET DETECTED — commit blocked. Details: $LOG" >&2
|
||||
exit 1
|
||||
fi
|
||||
exit 0
|
||||
Reference in New Issue
Block a user