MrBlake/Netbox-Store
1
0
Fork 0
Code Issues Pull Requests Actions 13 Packages Projects Releases Wiki Activity

feat: add NetBox plugin store
CI / php-store (push) Waiting to run
Details
CI / python-components (push) Waiting to run
Details

Browse Source
This commit is contained in:
MrBlake
2026-08-24 20:51:25 +02:00
commit f36d6be511
135 changed files with 15160 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+47
View File
@@ -0,0 +1,47 @@
# Direkter Apache-Port. Der sichere lokale Standard ist nur auf dem Host erreichbar.
STORE_BIND_ADDRESS=127.0.0.1
STORE_HTTP_PORT=8080
STORE_PUBLIC_URL=http://localhost:8080
STORE_COOKIE_SECURE=false
STORE_TRUST_PROXY=false
STORE_TRUSTED_PROXY_IPS=127.0.0.1,::1
# Es gibt absichtlich keine Zugangsdaten. Admin wird nur aktiviert, wenn alle
# drei Werte gesetzt sind. Argon2id-Hashes wegen der $-Zeichen einfach quoten.
STORE_ADMIN_USERNAME=
STORE_ADMIN_PASSWORD_HASH=''
STORE_SESSION_SECRET=
STORE_ADMIN_SESSION_TTL=28800
STORE_LOGIN_MAX_ATTEMPTS=5
STORE_LOGIN_WINDOW_SECONDS=900
# JSON ist der Standard. STORE_DB_DRIVER wird vom MariaDB-Overlay überschrieben.
STORE_DB_DRIVER=json
# Nur für `compose.mariadb.yaml`; unbedingt eigene, unterschiedliche Werte setzen.
MARIADB_DATABASE=netbox_store
MARIADB_USER=netbox_store
MARIADB_PASSWORD=
MARIADB_ROOT_PASSWORD=
# Quellzugriff. Tokens dürfen leer bleiben, solange nur öffentliche Repositories
# synchronisiert werden. Interne/private Ziele müssen bewusst freigeschaltet werden.
GITEA_TOKEN=
GITHUB_TOKEN=
STORE_ALLOWED_SOURCE_HOSTS=git.mrblake.cc,github.com,api.github.com,*.github.com,*.githubusercontent.com
STORE_ALLOW_PRIVATE_NETWORKS=false
STORE_DEFAULT_BASE_URL=https://git.mrblake.cc
STORE_DEFAULT_API_URL=https://git.mrblake.cc/api/v1
STORE_DEFAULT_OWNER=MrBlake
STORE_DEFAULT_OWNER_KIND=user
STORE_DEFAULT_TOPIC=netbox-plugin
# Synchronisierung
STORE_SCHEDULER_ENABLED=true
STORE_SYNC_INTERVAL_SECONDS=900
# Harte Gesamtgrenzen je Lauf; ein paralleler Lauf wird zusätzlich abgewiesen.
STORE_SYNC_MAX_SECONDS=900
STORE_SYNC_MAX_REQUESTS=2500
STORE_SYNC_MAX_BYTES=1073741824
STORE_SYNC_MAX_REPOSITORIES=2000
STORE_SYNC_MAX_RELEASES=1000
.gitattributes
+4
View File
@@ -0,0 +1,4 @@
*.sh text eol=lf
store/bin/console text eol=lf
host_agent/systemd/*.service text eol=lf
host_agent/systemd/*.socket text eol=lf
.gitea/workflows/ci.yml
+47
View File
@@ -0,0 +1,47 @@
name: CI
on:
push:
pull_request:
jobs:
php-store:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: shivammathur/setup-php@v2
with:
php-version: "8.4"
extensions: curl, dom, intl, mbstring, pdo_mysql
coverage: none
- name: Validate Composer metadata
working-directory: store
run: composer validate --strict
- name: Install PHP dependencies
working-directory: store
run: composer install --no-interaction --no-progress --prefer-dist
- name: Audit locked PHP dependencies
working-directory: store
run: composer audit --locked --no-interaction
- name: Run Store tests
working-directory: store
run: composer test
python-components:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip
cache-dependency-path: |
host_agent/pyproject.toml
netbox_plugin/pyproject.toml
- name: Install host-agent test dependencies
run: python -m pip install -e "./host_agent[dev]" packaging
- name: Test host agent
run: pytest host_agent/tests
- name: Test NetBox plugin core
working-directory: netbox_plugin
run: python -m unittest discover -s tests -v
.gitignore
+44
View File
@@ -0,0 +1,44 @@
# Python
__pycache__/
*.py[cod]
*.egg-info/
.pytest_cache/
.coverage
htmlcov/
build/
dist/
# Virtual environments and local configuration
.venv/
.venv-*/
venv/
.env
.env.*
!.env.example
# Runtime files
*.sqlite3
staticfiles/
media/
# Editors and operating systems
.idea/
.vscode/
.DS_Store
Thumbs.db
# Tool caches
.ruff_cache/
.mypy_cache/
.tox/
# PHP/Composer and Store runtime data
vendor/
composer.phar
.phpunit.cache/
.phpunit.result.cache
store/data/*
!store/data/.gitkeep
# Local build verification output
*.whl
README.md
+319
View File
@@ -0,0 +1,319 @@
# MrBlake NetBox Plugin Store
Dieses Repository besteht aus drei getrennten Komponenten:
| Pfad | Aufgabe |
| --- | --- |
| `store/` | Eigenständiger PHP-Store, direkt durch Apache ausgeliefert. Er synchronisiert Forgejo-/GitHub-Repositories, rendert README-Dateien und stellt den kuratierten API-Katalog bereit. |
| `netbox_plugin/` | NetBox-Oberfläche für Katalog, Installation und Plugin-Lebenszyklus. Unterstützt NetBox 4.6.5 bis 4.6.8. |
| `host_agent/` | Kleine privilegierte Linux-Komponente, die freigegebene Wheel-Dateien prüft und die eigentlichen Änderungen am NetBox-Host ausführt. |
Der Store verwendet standardmäßig eine lokale JSON-Datei. MariaDB ist optional. Die konfigurierte Standardquelle gilt als vom Betreiber freigegeben; weitere Quellen werden im Admin-Bereich zunächst ausstehend angelegt. Automatisch erkannte Plugins und Releases erscheinen erst nach einer ausdrücklichen Freigabe im öffentlichen Katalog.
## Native Installation mit Apache und JSON
Docker ist nicht erforderlich. Der primäre Betriebsweg ist eine normale PHP-Anwendung unter Apache. Benötigt werden:
- PHP 8.3 oder neuer mit CLI und Apache-Modul/FPM (getestet mit PHP 8.4);
- die PHP-Erweiterungen cURL, DOM/XML, intl, JSON, mbstring und PDO; für MariaDB zusätzlich `pdo_mysql`;
- Composer 2;
- Apache 2.4 mit `mod_rewrite`; `mod_headers` wird für die zusätzlichen Header aus `.htaccess` empfohlen.
Unter Debian/Ubuntu sind die wesentlichen Pakete beispielsweise:
```text
sudo apt install apache2 composer libapache2-mod-php8.3 php8.3-cli \
php8.3-curl php8.3-intl php8.3-mbstring php8.3-mysql php8.3-xml
sudo a2enmod rewrite headers expires
```
Die Paketnamen anderer Distributionen unterscheiden sich. Bei PHP-FPM muss Apache stattdessen mit dem passenden FPM-Handler konfiguriert werden.
### Anwendung und Konfiguration
Dieses Beispiel verwendet `/var/www/netbox-plugin-store` als Repository-Pfad. Nur `store/public` darf durch Apache veröffentlicht werden. Das Repository kann zunächst unter einem normalen Deployment-Benutzer ausgecheckt und danach an diesen Pfad verschoben werden:
```text
git clone https://git.mrblake.cc/MrBlake/Netbox-Store.git netbox-plugin-store
sudo mv netbox-plugin-store /var/www/netbox-plugin-store
sudo chown -R "$(id -un):$(id -gn)" /var/www/netbox-plugin-store
cd /var/www/netbox-plugin-store/store
composer install --no-dev --no-interaction --prefer-dist --optimize-autoloader
cp .env.example .env
```
Zuerst werden ein eigenes Admin-Passwort und ein unabhängiges Session-Geheimnis erzeugt:
```text
php bin/console hash-password 'EIN-LANGES-ZUFÄLLIGES-PASSWORT'
php -r 'echo bin2hex(random_bytes(32)), PHP_EOL;'
```
Das Klartextpasswort sollte in der Praxis aus einer temporären, nicht protokollierten Shell-Variablen übergeben werden. In `store/.env` werden mindestens die folgenden Werte ersetzt. Der Argon2id-Hash steht in einfachen Anführungszeichen, damit seine `$`-Zeichen unverändert bleiben:
```dotenv
APP_ENV=production
STORE_PUBLIC_URL=https://plugins.example.internal
STORE_COOKIE_SECURE=true
STORE_ADMIN_USERNAME=mein-admin
STORE_ADMIN_PASSWORD_HASH='$argon2id$...'
STORE_SESSION_SECRET=...
STORE_DB_DRIVER=json
STORE_JSON_PATH=/var/www/netbox-plugin-store/store/data/store.json
```
Bei einem reinen HTTP-Testsystem müssen URL und `STORE_COOKIE_SECURE=false` zusammenpassen. Ein extern erreichbarer Produktivbetrieb sollte TLS direkt in Apache terminieren.
Der Anwendungscode bleibt root-owned; nur `.env` ist für die Apache-Gruppe lesbar und nur `store/data` ist für den Web-/Scheduler-Benutzer schreibbar:
```text
sudo chown -R root:root /var/www/netbox-plugin-store
sudo chown root:www-data /var/www/netbox-plugin-store/store/.env
sudo chmod 0640 /var/www/netbox-plugin-store/store/.env
sudo install -d -o www-data -g www-data -m 0750 \
/var/www/netbox-plugin-store/store/data
sudo chown -R www-data:www-data /var/www/netbox-plugin-store/store/data
```
### Apache VirtualHost
Der VirtualHost zeigt direkt auf `store/public`. `AllowOverride All` ist erforderlich, damit die mitgelieferte `.htaccess` das Front-Controller-Rewrite aktivieren kann:
```apache
<VirtualHost *:80>
ServerName plugins.example.internal
DocumentRoot /var/www/netbox-plugin-store/store/public
<Directory /var/www/netbox-plugin-store/store/public>
Options -Indexes
AllowOverride All
Require all granted
</Directory>
ErrorLog ${APACHE_LOG_DIR}/netbox-plugin-store-error.log
CustomLog ${APACHE_LOG_DIR}/netbox-plugin-store-access.log combined
</VirtualHost>
```
Die Konfiguration wird zum Beispiel als `/etc/apache2/sites-available/netbox-plugin-store.conf` gespeichert und aktiviert:
```text
sudo a2ensite netbox-plugin-store.conf
sudo apache2ctl configtest
sudo systemctl reload apache2
```
Für Produktion wird derselbe DocumentRoot in einem TLS-VirtualHost verwendet. Weder das Repository-Wurzelverzeichnis noch `store/` selbst dürfen als DocumentRoot dienen, da dort Konfiguration und Laufzeitdaten liegen.
### Initialisierung und erster Import
CLI und Apache müssen den JSON-Zustand mit derselben UID verwalten. Deshalb laufen Bootstrap und Sync als Apache-Benutzer:
```text
cd /var/www/netbox-plugin-store/store
sudo -u www-data php bin/console bootstrap
sudo -u www-data php bin/console sync
```
Danach sind Website und `/healthz` erreichbar. Die Admin-Anmeldung liegt unter `/admin`. Importierte Kandidaten bleiben bis zur manuellen Prüfung und Freigabe unsichtbar für den öffentlichen API-Katalog.
### Regelmäßige Synchronisierung
Bevorzugt läuft der Watch-Modus als eigener systemd-Dienst. Beispiel für `/etc/systemd/system/netbox-plugin-store-sync.service`:
```ini
[Unit]
Description=NetBox Plugin Store repository sync
After=network-online.target apache2.service
Wants=network-online.target
[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/var/www/netbox-plugin-store/store
ExecStart=/usr/bin/php /var/www/netbox-plugin-store/store/bin/console sync --watch --interval=900
Restart=on-failure
RestartSec=10
UMask=0077
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ReadWritePaths=/var/www/netbox-plugin-store/store/data
[Install]
WantedBy=multi-user.target
```
```text
sudo systemctl daemon-reload
sudo systemctl enable --now netbox-plugin-store-sync.service
sudo systemctl status netbox-plugin-store-sync.service
```
Alternativ kann `www-data` den einmaligen Befehl per Cron aufrufen. Dazu `sudo crontab -u www-data -e` öffnen und die folgende Zeile eintragen. `flock` verhindert überlappende Läufe:
```cron
*/15 * * * * /usr/bin/flock -n /var/www/netbox-plugin-store/store/data/sync-cron.lock /bin/sh -c 'cd /var/www/netbox-plugin-store/store && /usr/bin/php bin/console sync' 2>&1 | /usr/bin/logger -t netbox-plugin-store-sync
```
Der Store selbst weist zusätzlich jeden parallelen Sync-Lauf ab. Pro Lauf gelten standardmäßig 900 Sekunden, 2.500 HTTP-Requests, 1 GiB Downloadvolumen, 2.000 Repositories und 1.000 Releases als gemeinsame Obergrenzen. Diese Werte lassen sich mit `STORE_SYNC_MAX_SECONDS`, `STORE_SYNC_MAX_REQUESTS`, `STORE_SYNC_MAX_BYTES`, `STORE_SYNC_MAX_REPOSITORIES` und `STORE_SYNC_MAX_RELEASES` in `store/.env` an die eigene Instanz anpassen.
### Native MariaDB-Option
JSON und MariaDB sind alternative Zustands-Speicher. Ein Wechsel des Treibers migriert bestehende Freigaben nicht automatisch. Nach dem Anlegen einer eigenen Datenbank und eines eingeschränkten Datenbankbenutzers werden diese Werte in `store/.env` gesetzt:
```dotenv
STORE_DB_DRIVER=mariadb
STORE_MARIADB_DSN=mysql:host=127.0.0.1;port=3306;dbname=netbox_store;charset=utf8mb4
STORE_MARIADB_USER=netbox_store
STORE_MARIADB_PASSWORD=EIN-EIGENES-ZUFÄLLIGES-PASSWORT
```
Die erforderliche Tabelle wird bei Bootstrap, Sync oder dem ersten Web-Aufruf angelegt. Das Datenbankkonto benötigt dafür Rechte auf genau diese Datenbank. `STORE_JSON_PATH` wird im MariaDB-Modus nicht verwendet.
## Optional: Docker Compose
Alternativ kann derselbe Apache/PHP-Store in einem Container laufen. Der Standard-Stack bindet seinen Port absichtlich nur an localhost und verwendet ein benanntes JSON-Volume:
```text
cp .env.example .env
docker compose build store
```
Unter Windows kann statt `cp` der Befehl `copy .env.example .env` verwendet werden. Vor dem Start werden in der Root-`.env` eigene Admin-Zugangsdaten gesetzt. Ohne vollständige Admin-Konfiguration bleibt die Anmeldung deaktiviert; es gibt kein Standardpasswort.
Hash und Session-Geheimnis lassen sich mit dem gebauten Image erzeugen:
```text
docker compose run --rm --no-deps -e STORE_SCHEDULER_ENABLED=false store php bin/console hash-password 'EIN-LANGES-ZUFÄLLIGES-PASSWORT'
docker compose run --rm --no-deps -e STORE_SCHEDULER_ENABLED=false store php -r 'echo bin2hex(random_bytes(32)), PHP_EOL;'
```
Danach:
```text
docker compose up -d --build
```
Der im Container aktivierte Scheduler führt den ersten Import unmittelbar aus. Mit `docker compose logs -f store` lässt sich der Lauf verfolgen. Für einen ausschließlich manuellen Sync wird vorher `STORE_SCHEDULER_ENABLED=false` gesetzt und anschließend `docker compose exec store php bin/console sync` aufgerufen.
Der Store ist standardmäßig auf `http://127.0.0.1:8080` erreichbar. Für MariaDB werden in `.env` mindestens `MARIADB_PASSWORD` und `MARIADB_ROOT_PASSWORD` auf unterschiedliche, zufällige Werte gesetzt und das Overlay zugeschaltet:
```text
docker compose -f compose.yaml -f compose.mariadb.yaml up -d --build
```
MariaDB erhält keinen Host-Port und ist nur im internen Compose-Netz erreichbar. Der Store wartet auf den Datenbank-Healthcheck.
## Synchronisierung und Freigabe
Der voreingestellte Forgejo-Import liest Repositories von `https://git.mrblake.cc`. GitHub ist bereits als zweiter Provider implementiert und kann im Admin-Bereich über dieselbe Quellenverwaltung hinzugefügt werden.
Mit einem Token lassen sich auch Metadaten und README privater GitHub-Repositories synchronisieren. Deren private Release-Assets bleiben in API v1 bewusst nicht installierbar, weil der Host-Agent keine Provider-Zugangsdaten erhält; für automatische Installationen muss das Wheel als öffentliches Release-Asset oder auf einem anderen freigegebenen HTTPS-Host veröffentlicht werden.
Der normale Ablauf ist:
1. Der Scheduler oder `php bin/console sync` liest Repository-Metadaten, Manifest/Packaging-Konfiguration, Releases und die README eines festen Commit-SHA ein.
2. Neue oder in installrelevanten Feldern geänderte Datensätze erhalten den Status `pending`.
3. Ein Administrator prüft Quelle, Paket-/Importname, NetBox-Kompatibilität, Artifact-URL, Größe und SHA-256.
4. Quelle, Plugin und Release werden getrennt freigegeben. Eine Freigabe ist keine automatische Folge der Synchronisierung.
5. Nur aktive, vollständige und freigegebene Datensätze erscheinen unter `/api/v1/plugins/`.
Die Detailseite wird aus der gespeicherten README erzeugt. Relative Links und Bilder werden auf den synchronisierten Commit aufgelöst; HTML wird vor der Ausgabe bereinigt. Verändert sich ein Artifact oder ein installrelevantes Metadatum, muss das Release erneut geprüft werden.
## NetBox-Plugin installieren
Das Plugin unter `netbox_plugin/` ist auf NetBox 4.6.5–4.6.8 begrenzt. Zuerst wird ein Wheel gebaut und in die NetBox-Virtualenv installiert:
```text
cd netbox_plugin
python -m build
/opt/netbox/venv/bin/pip install dist/netbox_plugin_store-0.1.0-py3-none-any.whl
```
Das Paket muss außerdem in `/opt/netbox/local_requirements.txt` festgehalten werden. In `configuration.py` wird es zunächst sicher im Dry-run-Modus eingerichtet:
```python
PLUGINS = [
"netbox_plugin_store",
]
PLUGINS_CONFIG = {
"netbox_plugin_store": {
"store_url": "https://plugins.example.internal",
"allowed_store_urls": ["https://plugins.example.internal"],
"allowed_artifact_urls": [
"https://plugins.example.internal",
"https://git.mrblake.cc",
],
"execution_mode": "dry_run",
},
}
```
Danach wird der übliche NetBox-Upgrade-Ablauf mit Migration, `collectstatic` und Neustart von Web- und RQ-Dienst ausgeführt. Details und alle Einstellungen stehen in [`netbox_plugin/README.md`](netbox_plugin/README.md).
## Produktion: Host-Agent
Für echte Installationen ist `execution_mode = "agent"` die vorgesehene Trennung: Der NetBox-Webprozess bleibt unprivilegiert und sendet einen kleinen Auftrag über den Unix-Socket `/run/netbox-store-agent/agent.sock`. Der Agent ruft Plugin und Release erneut aus dem Store ab, vergleicht die Freigabemarkierung und prüft Wheel, Dateigröße und SHA-256 vor jeder Änderung.
Kurzablauf auf dem Linux-NetBox-Host:
1. Das Paket aus `host_agent/` in eine administrative Python-Umgebung installieren.
2. `host_agent/examples/agent.toml` nach `/etc/netbox-store-agent/agent.toml` kopieren, alle Hosts, Pfade, UIDs/GIDs und die NetBox-Version anpassen und die Datei root-owned mit Modus `0600` schützen.
3. Die beiden Dateien aus `host_agent/systemd/` nach `/etc/systemd/system/` kopieren. Insbesondere `SocketGroup` muss zur NetBox-Servicegruppe passen.
4. Die vom Agent verwaltete Plugin-Liste einmalig in `configuration.py` einbinden:
```python
from store_plugins import STORE_PLUGINS
PLUGINS += STORE_PLUGINS
```
5. Socket aktivieren und zunächst im voreingestellten Dry-run-Modus testen:
```text
systemctl daemon-reload
systemctl enable --now netbox-store-agent.socket
netbox-store-agent capabilities
```
6. Erst nach erfolgreichen Katalog- und Lifecycle-Tests `dry_run = false` in der root-geschützten Agent-Konfiguration setzen und den Dienst neu starten.
Die NetBox-Konfiguration wird dann ergänzt:
```python
PLUGINS_CONFIG["netbox_plugin_store"].update({
"execution_mode": "agent",
"agent_socket_path": "/run/netbox-store-agent/agent.sock",
"agent_timeout": 30,
})
```
Der Agent installiert ausschließlich freigegebene, unveränderliche Wheel-Dateien. Abhängigkeiten müssen im aktuellen MVP bereits durch den Betreiber bereitgestellt sein. Vollständige Sicherheits- und Recovery-Hinweise stehen in [`host_agent/README.md`](host_agent/README.md).
## API v1
Der öffentliche, nur freigegebene Katalog stellt diese Endpunkte bereit:
```text
GET /api/v1/plugins/
GET /api/v1/plugins/{slug}/
GET /api/v1/plugins/{slug}/releases/{version}/
```
Ein Plugin enthält unter anderem `package_name`, `import_name`, `min_netbox_version`, `max_netbox_version` und `releases`. Ein installierbares Release enthält `download_url`, Artifact-`sha256`, `artifact_size`, `immutable` und die opake Freigabemarkierung `approved_payload_sha256`. Der genaue Datenfluss und die Vertrauensgrenzen sind in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) beschrieben.
## Lokale Prüfungen
Die Komponenten bringen getrennte Prüfungen mit; sie können ohne produktive NetBox- oder Host-Änderungen ausgeführt werden:
```text
cd store && composer install && composer test
cd host_agent && python -m pip install -e ".[dev]" && pytest
cd netbox_plugin && python -m unittest discover -s tests -v
```
compose.mariadb.yaml
+38
View File
@@ -0,0 +1,38 @@
services:
store:
environment:
STORE_DB_DRIVER: mariadb
STORE_MARIADB_DSN: "mysql:host=mariadb;port=3306;dbname=${MARIADB_DATABASE:-netbox_store};charset=utf8mb4"
STORE_MARIADB_USER: "${MARIADB_USER:-netbox_store}"
STORE_MARIADB_PASSWORD: "${MARIADB_PASSWORD:?MARIADB_PASSWORD must be set}"
depends_on:
mariadb:
condition: service_healthy
mariadb:
image: mariadb:11.4
restart: unless-stopped
environment:
MARIADB_DATABASE: "${MARIADB_DATABASE:-netbox_store}"
MARIADB_USER: "${MARIADB_USER:-netbox_store}"
MARIADB_PASSWORD: "${MARIADB_PASSWORD:?MARIADB_PASSWORD must be set}"
MARIADB_ROOT_PASSWORD: "${MARIADB_ROOT_PASSWORD:?MARIADB_ROOT_PASSWORD must be set}"
volumes:
- mariadb-data:/var/lib/mysql
healthcheck:
test:
- CMD
- healthcheck.sh
- --connect
- --innodb_initialized
interval: 10s
timeout: 5s
retries: 10
start_period: 20s
security_opt:
- no-new-privileges:true
networks:
- store-internal
volumes:
mariadb-data:
compose.yaml
+65
View File
@@ -0,0 +1,65 @@
name: mrblake-netbox-plugin-store
services:
store:
build:
context: ./store
image: mrblake/netbox-plugin-store:local
restart: unless-stopped
ports:
- "${STORE_BIND_ADDRESS:-127.0.0.1}:${STORE_HTTP_PORT:-8080}:80"
environment:
APP_ENV: "${APP_ENV:-production}"
STORE_PUBLIC_URL: "${STORE_PUBLIC_URL:-http://localhost:8080}"
STORE_COOKIE_SECURE: "${STORE_COOKIE_SECURE:-false}"
STORE_TRUST_PROXY: "${STORE_TRUST_PROXY:-false}"
STORE_TRUSTED_PROXY_IPS: "${STORE_TRUSTED_PROXY_IPS:-127.0.0.1,::1}"
STORE_SESSION_NAME: "${STORE_SESSION_NAME:-netbox_plugin_store}"
STORE_ADMIN_USERNAME: "${STORE_ADMIN_USERNAME:-}"
STORE_ADMIN_PASSWORD_HASH: "${STORE_ADMIN_PASSWORD_HASH:-}"
STORE_SESSION_SECRET: "${STORE_SESSION_SECRET:-}"
STORE_ADMIN_SESSION_TTL: "${STORE_ADMIN_SESSION_TTL:-28800}"
STORE_LOGIN_MAX_ATTEMPTS: "${STORE_LOGIN_MAX_ATTEMPTS:-5}"
STORE_LOGIN_WINDOW_SECONDS: "${STORE_LOGIN_WINDOW_SECONDS:-900}"
STORE_DB_DRIVER: "${STORE_DB_DRIVER:-json}"
STORE_JSON_PATH: "/var/www/html/data/store.json"
STORE_ALLOWED_SOURCE_HOSTS: "${STORE_ALLOWED_SOURCE_HOSTS:-git.mrblake.cc,github.com,api.github.com,*.github.com,*.githubusercontent.com}"
STORE_ALLOW_PRIVATE_NETWORKS: "${STORE_ALLOW_PRIVATE_NETWORKS:-false}"
STORE_HTTP_TIMEOUT_SECONDS: "${STORE_HTTP_TIMEOUT_SECONDS:-20}"
STORE_MAX_METADATA_BYTES: "${STORE_MAX_METADATA_BYTES:-2097152}"
STORE_MAX_ARTIFACT_BYTES: "${STORE_MAX_ARTIFACT_BYTES:-536870912}"
STORE_USER_AGENT: "${STORE_USER_AGENT:-MrBlake-NetBox-Plugin-Store/1.0}"
STORE_DEFAULT_PROVIDER: "${STORE_DEFAULT_PROVIDER:-forgejo}"
STORE_DEFAULT_SOURCE_NAME: "${STORE_DEFAULT_SOURCE_NAME:-MrBlake Forgejo}"
STORE_DEFAULT_SOURCE_SLUG: "${STORE_DEFAULT_SOURCE_SLUG:-mrblake-forgejo}"
STORE_DEFAULT_BASE_URL: "${STORE_DEFAULT_BASE_URL:-https://git.mrblake.cc}"
STORE_DEFAULT_API_URL: "${STORE_DEFAULT_API_URL:-https://git.mrblake.cc/api/v1}"
STORE_DEFAULT_OWNER: "${STORE_DEFAULT_OWNER:-MrBlake}"
STORE_DEFAULT_OWNER_KIND: "${STORE_DEFAULT_OWNER_KIND:-user}"
STORE_DEFAULT_TOPIC: "${STORE_DEFAULT_TOPIC:-netbox-plugin}"
STORE_DEFAULT_TOKEN_ENV: "${STORE_DEFAULT_TOKEN_ENV:-GITEA_TOKEN}"
GITEA_TOKEN: "${GITEA_TOKEN:-}"
GITHUB_TOKEN: "${GITHUB_TOKEN:-}"
STORE_SCHEDULER_ENABLED: "${STORE_SCHEDULER_ENABLED:-true}"
STORE_SYNC_INTERVAL_SECONDS: "${STORE_SYNC_INTERVAL_SECONDS:-900}"
STORE_SYNC_MAX_SECONDS: "${STORE_SYNC_MAX_SECONDS:-900}"
STORE_SYNC_MAX_REQUESTS: "${STORE_SYNC_MAX_REQUESTS:-2500}"
STORE_SYNC_MAX_BYTES: "${STORE_SYNC_MAX_BYTES:-1073741824}"
STORE_SYNC_MAX_REPOSITORIES: "${STORE_SYNC_MAX_REPOSITORIES:-2000}"
STORE_SYNC_MAX_RELEASES: "${STORE_SYNC_MAX_RELEASES:-1000}"
STORE_PAGE_SIZE: "${STORE_PAGE_SIZE:-12}"
STORE_API_PAGE_SIZE: "${STORE_API_PAGE_SIZE:-50}"
STORE_API_MAX_PAGE_SIZE: "${STORE_API_MAX_PAGE_SIZE:-100}"
volumes:
- store-data:/var/www/html/data
security_opt:
- no-new-privileges:true
networks:
- store-internal
volumes:
store-data:
networks:
store-internal:
driver: bridge
docs/ARCHITECTURE.md
+145
View File
@@ -0,0 +1,145 @@
# Architektur und Vertrauensgrenzen
## Komponenten
```text
Forgejo / GitHub
|
| HTTPS, fest erlaubte Hosts, commit-gebundene Metadaten
v
+-----------------------+ JSON-Datei oder MariaDB
| PHP Store auf Apache |------------------------------+
| Sync + Admin + API v1 | |
+-----------------------+ |
| freigegebener Katalog |
v |
+-----------------------+ Unix-Socket +--------------------------+
| NetBox Store Plugin |--------------->| privilegierter Host-Agent|
| unprivilegierte UI | | erneute Prüfung + Lifecycle|
+-----------------------+ +--------------------------+
|
v
NetBox-Virtualenv, Konfiguration,
Migrationen, Static Files, Dienste
```
Der Store ist eine eigenständige PHP-Anwendung. Apache liefert HTML, Assets und API direkt aus. NetBox benötigt keinen Zugriff auf die Store-Datenbank, und der Store erhält keinen Schreibzugriff auf einen NetBox-Host.
## Zustandsmodell
Der persistierte Zustand enthält mindestens diese fachlichen Ebenen:
- **Source:** Provider, API-/Basis-URL, Owner/Organisation und Synchronisierungsstatus.
- **Plugin:** Anzeige- und Installationsmetadaten, Repository, README-Snapshot, Paket-/Importname und NetBox-Kompatibilität.
- **Release:** Version, Download-URL, SHA-256, Größe, Commit und Kompatibilität. Da ein Upstream-Asset trotz gleicher URL ersetzt werden kann, wird es bei jedem erfolgreichen Sync erneut gehasht.
- **Audit/Admin-Zustand:** Freigaben, Ablehnungen und sicherheitsrelevante Änderungen.
Die JSON-Implementierung speichert denselben logischen Zustand wie der MariaDB-Adapter. Sie ist für eine einzelne Store-Instanz gedacht und schützt Schreibvorgänge mit Sperre und atomischem Austausch. MariaDB erlaubt einen extern verwalteten, transaktionalen Persistenzdienst. Ein gleichzeitiger Wechsel zwischen beiden Treibern ist kein Replikationsmechanismus.
## Import-Pipeline
1. Eine Quelle wird serverseitig konfiguriert; Tokens werden über Umgebungsvariablen referenziert und nicht als Katalogdaten gespeichert.
2. Der Provider-Adapter listet Repositories und Releases. Netzwerkzugriffe sind auf HTTPS und eine explizite Hostliste begrenzt. Private/reservierte Adressen sind standardmäßig gesperrt.
3. Vor dem Lesen von README oder Packaging-Metadaten wird ein Commit-SHA festgehalten. Relative README-Ressourcen werden an diesen Stand gebunden.
4. Kandidaten werden aus Topic, `pyproject.toml`/vergleichbarer Packaging-Konfiguration und der NetBox-`PluginConfig` erkannt. Fehlende oder widersprüchliche Installationsfelder verhindern eine Freigabe.
5. Externes Markdown wird in HTML umgewandelt und sanitisiert. Unsichere Schemata und nicht erlaubtes HTML werden nicht in die Store-Seite übernommen.
6. Neue Datensätze oder installrelevante Änderungen landen wieder im Freigabezustand `pending`.
GitHub verwendet dasselbe interne Source-/Plugin-/Release-Modell. Provider-spezifisch bleiben nur Authentifizierung, Pagination und API-Formate.
## Freigabegatter
Source, Plugin und Release werden getrennt bewertet. Ein Plugin darf auch ohne installierbares Release sichtbar sein; dann liefert die API `releases: []` und `latest_version: null`. Für einen sichtbaren Plugin-Eintrag gelten:
- Quelle und Plugin sind aktiv und freigegeben.
- Das Plugin besitzt vollständige Paket-, Import- und Kompatibilitätsdaten.
Ein Release wird diesem Plugin nur dann öffentlich beigefügt, wenn zusätzlich gilt:
- Das Release ist freigegeben und als unveränderlich markiert.
- Artifact-URL, SHA-256 und positive Dateigröße sind vorhanden.
- Die gespeicherte Freigabemarkierung gehört exakt zum aktuell freigegebenen Payload.
Die Freigabemarkierung heißt aus Protokollkompatibilitätsgründen `approved_payload_sha256`. Clients behandeln den 64-stelligen Hexwert als opaken Store-Marker. Er ersetzt nicht den separaten SHA-256 der heruntergeladenen Artifact-Datei.
## API-Vertrag
`GET /api/v1/plugins/` liefert eine paginierte Antwort:
```json
{
"api_version": "v1",
"count": 1,
"page": 1,
"page_size": 50,
"next": null,
"previous": null,
"results": []
}
```
`GET /api/v1/plugins/{slug}/` liefert das Plugin direkt. Sein Kernschema ist:
```json
{
"api_version": "v1",
"slug": "netbox-example",
"name": "Example",
"summary": "Example plugin",
"description": "Description",
"repository_url": "https://git.example/repo",
"latest_version": "1.2.3",
"package_name": "netbox-example",
"import_name": "netbox_example",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
"approved": true,
"status": "approved",
"releases": []
}
```
`GET /api/v1/plugins/{slug}/releases/{version}/` liefert ein Release direkt:
```json
{
"version": "1.2.3",
"download_url": "https://artifacts.example/netbox_example-1.2.3-py3-none-any.whl",
"sha256": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"artifact_size": 12345,
"commit_sha": "",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
"published_at": "2026-08-24T12:00:00Z",
"approved": true,
"status": "approved",
"immutable": true,
"approved_payload_sha256": "abcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcd"
}
```
Unbekannte, nicht freigegebene und unvollständige Datensätze werden gegenüber öffentlichen Clients nicht wie installierbare Einträge behandelt.
## Lifecycle-Sequenz
1. Ein berechtigter NetBox-Benutzer wählt eine Aktion; echte Mutationen erfordern zusätzlich Superuser-Rechte, CSRF-Prüfung und die Eingabe des exakten Slugs.
2. Das NetBox-Plugin liest den freigegebenen Katalog und sendet nur Aktion, Slug, Version, Actor/Request-ID und Store-Freigabemarker an den Unix-Socket.
3. Der Agent akzeptiert den Auftrag idempotent und speichert seinen Zustand in SQLite. NetBox wartet nicht auf einen möglichen eigenen Neustart.
4. Der Agent ruft Plugin und Release selbst erneut aus dem Store ab. Er vertraut nicht allein auf die vom Webprozess gelieferten Metadaten.
5. Er erzwingt Host-/DNS-Regeln, NetBox-Version, Wheel-Dateiname, Paket-/Importname, Dateigröße, Artifact-SHA-256 und den exakten Freigabemarker.
6. Host-Änderungen laufen serialisiert unter einem globalen Lock. Installationen bleiben zunächst deaktiviert; Aktivierung ist eine eigene Aktion.
7. Der Audit-Eintrag in NetBox fragt den Agent-Status später ab und gleicht Erfolg, Fehler oder manuell zu bereinigenden Zustand ab.
## Sicherheitsgrenzen
- Admin-Zugangsdaten existieren nicht als Standardwerte. Alle drei Credential-Variablen müssen gemeinsam gesetzt sein.
- Admin-Aktionen verwenden serverseitige Sessions mit geschütztem Cookie, HMAC-gebundene CSRF-Tokens und Login-Ratenbegrenzung.
- Source-Tokens dürfen nur an ihren konfigurierten Ursprung gesendet werden. Artifact-Hosts erhalten keine Store-Zugangsdaten.
- Der öffentliche Store-Prozess ist nicht privilegiert gegenüber NetBox. Nur der separate, root-konfigurierte Agent darf die Virtualenv, verwaltete Include-/Requirements-Dateien und Dienste verändern.
- Der Unix-Socket benötigt Dateisystemrechte und Linux-Peer-Credential-Prüfung. Die NetBox-Service-UID oder -GID muss explizit erlaubt sein.
- Der Agent startet mit `dry_run = true`, blockiert Selbstverwaltung und verwendet feste Argumentlisten statt Shell-Kommandos.
- Das aktuelle Verfahren ist kein vollständiges Supply-Chain-Framework. Artifact-Build-Provenienz, Signierung, Abhängigkeitsbereitstellung und Recovery nach bereits angewandten Migrationen bleiben Betreiberaufgaben.
## Betriebsgrenzen
Der JSON-Treiber ist für genau eine Store-Installation auf einem lokalen Dateisystem ausgelegt. Apache-Worker und der lokale Scheduler koordinieren sich dabei über die Dateisperre. Bei mehreren Hosts/Replikaten ist MariaDB zu verwenden, und Scheduler/Sync müssen so betrieben werden, dass nicht mehrere Instanzen dieselbe Quelle gleichzeitig bearbeiten. Store-Daten, Agent-Journal, Backups, Konfiguration und Token-Dateien gehören in die reguläre Backup- und Restore-Planung.
host_agent/.gitignore
+6
View File
@@ -0,0 +1,6 @@
__pycache__/
*.py[cod]
*.egg-info/
.buildcheck/
build/
dist/
host_agent/README.md
+176
View File
@@ -0,0 +1,176 @@
# NetBox Store host agent (MVP)
This package is the deliberately small privileged boundary between the NetBox Store plugin and a
NetBox 4.6.5–4.6.8 Linux host. It does not import the Store implementation or assume Django: every
decision is revalidated against the configured JSON API immediately before a lifecycle action.
The daemon is **dry-run by default**. It serializes operations, persists idempotency and state in
SQLite, accepts one bounded JSON line per Unix-stream connection, verifies Linux peer credentials,
downloads only an approved immutable wheel, and invokes subprocesses only as fixed argument arrays
with `shell=False`.
## Install and one-time operator setup
Build/install into a dedicated administrative environment, copy
[`examples/agent.toml`](examples/agent.toml) to `/etc/netbox-store-agent/agent.toml`, replace all
site-specific paths, hosts, UIDs/GIDs and versions, then make the file root-owned and mode `0600`.
The optional bearer-token file must also be root-owned `0600`. Store credentials are sent only to
the exact scheme/host/port origin of `store.base_url`, never to a separately allow-listed artifact
host.
`allow_private_addresses = false` is intentionally fail-closed. If the Store or an approved
artifact host consciously runs on RFC1918/ULA infrastructure, set it to `true` only after pinning
every expected hostname in `allowed_hosts` and deploying correctly verified TLS (including the
private CA via `ca_file` when needed). The host allow-list and exact-origin credential rule remain
active; this switch only permits private/reserved DNS results.
The agent owns only these two configured files:
- `paths.include_path`, which contains only `STORE_PLUGINS = [...]`;
- `paths.requirements_path`, which contains the locked direct wheel references.
In the operator-owned NetBox `configuration.py`, add once, after the normal `PLUGINS` declaration:
```python
from store_plugins import STORE_PLUGINS
PLUGINS += STORE_PLUGINS
```
Place `store_plugins.py` where that import resolves in your deployment. Optionally add a one-time
`-r /opt/netbox/local_requirements_store.txt` line to the operator-owned
`/opt/netbox/local_requirements.txt` for reproducible maintenance installs. The agent never edits
either operator-owned file.
Install the example systemd units, review their `ReadWritePaths`, `SocketGroup`, and `ExecStart`, then:
```text
systemctl daemon-reload
systemctl enable --now netbox-store-agent.socket
netbox-store-agent capabilities
```
The canonical socket path used by the unit, example config, CLI default, and NetBox Store plugin is
`/run/netbox-store-agent/agent.sock`.
With peer checks enabled (the production default), Linux `SO_PEERCRED` must be available and either
the peer's effective UID must appear in `allowed_peer_uids` or its effective GID in
`allowed_peer_gids`; Unix-socket filesystem permissions still apply. Prefer allow-listing the exact
NetBox service UID, especially when socket access is granted through a supplementary group.
Keep `dry_run = true` through catalog and lifecycle acceptance tests. Enabling real mutation is an
explicit operator configuration change.
## Store JSON contract
Both endpoint templates may be configured with or without a trailing slash. A 404 is retried once
using the alternate form. Path parameters are URL-quoted. The agent accepts exactly these fields.
`GET /api/v1/plugins/{slug}/`:
```json
{
"api_version": "v1",
"slug": "netbox-example",
"name": "Example",
"summary": "Example plugin",
"description": "Description",
"repository_url": "https://git.example/repo",
"latest_version": "1.2.3",
"package_name": "netbox-example",
"import_name": "netbox_example",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
"approved": true,
"status": "approved",
"releases": []
}
```
`GET /api/v1/plugins/{slug}/releases/{version}/` returns a release object directly:
```json
{
"version": "1.2.3",
"download_url": "https://store.example/artifacts/netbox_example-1.2.3-py3-none-any.whl",
"sha256": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"artifact_size": 12345,
"commit_sha": "",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
"published_at": "2026-08-24T12:00:00Z",
"approved": true,
"status": "approved",
"immutable": true,
"approved_payload_sha256": "abcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcd"
}
```
If release detail returns 404, exactly one matching object in plugin `releases[]` is accepted. The
historic field `approved_payload_sha256` is protocol-v1 naming: clients treat its lowercase 64-hex
value as an opaque Store marker, and the agent compares it exactly with the freshly fetched value.
Artifact `sha256`, in contrast, is always a lowercase 64-character SHA-256.
The plugin and release must be approved, the release immutable and compatible with the configured
NetBox version. The artifact must be a valid wheel whose filename distribution and version match
the catalog, and its byte count, digest, host, scheme and DNS addresses are checked while streaming.
Redirects, source distributions, private/reserved DNS targets (unless explicitly enabled for a test
environment), and catalog additions outside the v1 schema fail closed.
## Client protocol
Transport is Unix `SOCK_STREAM`, UTF-8 JSON-lines, exactly one request and one response per
connection, maximum 64 KiB. Response shape is always
`{"protocol_version":1,"status":...,"body":{...}}`.
```json
{"protocol_version":1,"method":"GET","path":"/v1/capabilities"}
```
```json
{"protocol_version":1,"method":"POST","path":"/v1/operations","idempotency_key":"20f4274f-d4e5-42bf-9164-967b1a774481","body":{"request_id":"eea17d87-8944-4ee2-a076-363338ab746d","action":"install","plugin_slug":"netbox-example","version":"1.2.3","approved_payload_sha256":"abcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcd","requested_by":"netbox:alice"}}
```
```json
{"protocol_version":1,"method":"GET","path":"/v1/operations/20f4274f-d4e5-42bf-9164-967b1a774481"}
```
`POST` returns 202. Repeating the same idempotency UUID with the identical body returns the existing
operation (`created:false`); a different body returns 409. States are `queued`, `running`,
`dry_run`, `succeeded`, `failed`, or `manual_recovery`.
CLI examples:
```text
netbox-store-agent submit install netbox-example --version 1.2.3 \
--approved-payload-sha256 abcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdefabcd
netbox-store-agent status 20f4274f-d4e5-42bf-9164-967b1a774481
```
## Lifecycle and fail-closed boundaries
- `install` and `update` re-fetch plugin and release, exactly match the approval token, download and
validate the wheel, then use `pip --no-index --no-deps --only-binary=:all: --require-hashes`.
- New installs remain disabled. Updating an enabled plugin runs NetBox `migrate`, `collectstatic`,
and restarts every configured service (`netbox` and `netbox-rq` in the example).
- `enable` revalidates its installed release, writes the include, migrates, collects static files,
and restarts both services. `disable` writes the include and restarts both. `uninstall` is allowed
only after disable and revalidates current approved plugin identity first.
- Self-management slugs are denied for every action. Package/import names and all executable paths,
service units, managed paths, Store origins and NetBox compatibility are configuration/catalog
policy—not caller-controlled command fragments.
- A root-owned global file lock prevents simultaneous host mutations. Interrupted `running`
operations become `manual_recovery` at startup. Managed files are backed up per operation and
atomically replaced; best-effort restoration does not claim package/service rollback.
This MVP intentionally has no TUF metadata, dependency-wheel set, transactional virtualenv switch,
or reliable rollback for package installation/database migrations/service restarts. `--no-deps`
means an approved plugin's dependencies must already be provisioned by the operator/base image.
Any failure after host mutation is marked `manual_recovery`; inspect the journal, backup directory,
installed distributions, migrations, both services and both managed files before retrying. If the
Store is unavailable or an entry is no longer approved, lifecycle calls fail closed; emergency
manual recovery remains an operator procedure outside this API.
The security boundary still depends on root ownership and permissions of the daemon executable,
configuration, token, journal/state directories, socket and NetBox paths; TLS/CA integrity; Store
approval operations; artifact build provenance; and a correctly restricted NetBox service account.
host_agent/examples/agent.toml
+52
View File
@@ -0,0 +1,52 @@
[agent]
socket_path = "/run/netbox-store-agent/agent.sock"
journal_path = "/var/lib/netbox-store-agent/journal.sqlite3"
lock_path = "/run/lock/netbox-store-agent.lifecycle.lock"
backup_dir = "/var/lib/netbox-store-agent/backups"
dry_run = true
require_root = true
require_peer_credentials = true
# Replace/add the numeric UID or effective GID used by the NetBox service.
allowed_peer_uids = [0]
allowed_peer_gids = [0]
socket_mode = 0o660
socket_uid = 0
socket_gid = 0
max_request_bytes = 65536
connection_timeout_seconds = 5
worker_threads = 1
[store]
base_url = "https://store.example.invalid"
plugin_endpoint_template = "/api/v1/plugins/{plugin_slug}/"
release_endpoint_template = "/api/v1/plugins/{plugin_slug}/releases/{version}/"
timeout_seconds = 10
max_catalog_bytes = 1048576
max_artifact_bytes = 268435456
allow_private_addresses = false
allow_http_for_testing = false
# Add an artifact origin only when releases intentionally use that origin.
allowed_hosts = ["store.example.invalid"]
# bearer_token_file = "/etc/netbox-store-agent/store.token"
# ca_file = "/etc/ssl/certs/internal-store-ca.pem"
[paths]
allowed_root = "/opt/netbox"
include_path = "/opt/netbox/netbox/netbox/store_plugins.py"
requirements_path = "/opt/netbox/local_requirements_store.txt"
temp_dir = "/var/lib/netbox-store-agent/tmp"
[commands]
python_path = "/opt/netbox/venv/bin/python"
manage_path = "/opt/netbox/netbox/manage.py"
systemctl_path = "/usr/bin/systemctl"
services = ["netbox", "netbox-rq"]
command_timeout_seconds = 900
[policy]
netbox_version = "4.6.8"
min_supported_netbox = "4.6.5"
max_supported_netbox = "4.6.8"
self_plugin_slugs = ["netbox-store", "netbox-plugin-store", "netbox_plugin_store"]
allow_prereleases = false
require_release_for_enable = true
host_agent/pyproject.toml
+42
View File
@@ -0,0 +1,42 @@
[build-system]
requires = ["setuptools>=75"]
build-backend = "setuptools.build_meta"
[project]
name = "mrblake-netbox-store-agent"
version = "0.1.0"
description = "Fail-closed host agent for curated NetBox plugin lifecycle operations"
readme = "README.md"
requires-python = ">=3.11"
license = {text = "MIT"}
authors = [{name = "MrBlake"}]
dependencies = ["packaging>=24,<27"]
classifiers = [
"Development Status :: 3 - Alpha",
"Environment :: No Input/Output (Daemon)",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Topic :: System :: Systems Administration",
]
[project.optional-dependencies]
dev = ["pytest>=8,<9", "ruff>=0.9,<1"]
[project.scripts]
netbox-store-agent = "netbox_store_agent.cli:main"
[tool.setuptools]
package-dir = {"" = "src"}
[tool.setuptools.packages.find]
where = ["src"]
[tool.ruff]
line-length = 100
target-version = "py311"
[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B", "S"]
ignore = ["S101"]
host_agent/src/netbox_store_agent/__init__.py
+5
View File
@@ -0,0 +1,5 @@
"""NetBox Store privileged host agent."""
from .constants import AGENT_VERSION, PROTOCOL_VERSION
__all__ = ["AGENT_VERSION", "PROTOCOL_VERSION"]
host_agent/src/netbox_store_agent/__main__.py
+3
View File
@@ -0,0 +1,3 @@
from .cli import main
raise SystemExit(main())
host_agent/src/netbox_store_agent/catalog.py
+497
View File
@@ -0,0 +1,497 @@
from __future__ import annotations
import hashlib
import http.client
import ipaddress
import json
import os
import socket
import ssl
import stat
import zipfile
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Protocol
from urllib.parse import quote, urlsplit
from packaging.utils import canonicalize_name, parse_wheel_filename
from packaging.version import InvalidVersion, Version
from .config import Config, StoreSettings
from .errors import CatalogError
from .util import DIST_NAME_RE, IMPORT_NAME_RE, SHA256_RE, SLUG_RE
class HttpStatusError(CatalogError):
def __init__(self, status: int, message: str):
super().__init__(message)
self.http_status = status
class Transport(Protocol):
def get_json(self, url: str, max_bytes: int) -> Any: ...
def download(
self,
url: str,
destination: Path,
*,
max_bytes: int,
expected_size: int,
expected_sha256: str,
) -> None: ...
class _PinnedHTTPConnection(http.client.HTTPConnection):
def __init__(self, host: str, port: int, address: str, timeout: float):
super().__init__(host, port=port, timeout=timeout)
self._address = address
def connect(self) -> None:
self.sock = socket.create_connection((self._address, self.port), self.timeout)
class _PinnedHTTPSConnection(http.client.HTTPSConnection):
def __init__(
self, host: str, port: int, address: str, timeout: float, context: ssl.SSLContext
):
super().__init__(host, port=port, timeout=timeout, context=context)
self._address = address
def connect(self) -> None:
raw_socket = socket.create_connection((self._address, self.port), self.timeout)
try:
self.sock = self._context.wrap_socket(raw_socket, server_hostname=self.host)
except Exception:
raw_socket.close()
raise
class SecureHTTPTransport:
"""Small no-redirect HTTP transport with a DNS-pinned connection."""
def __init__(self, settings: StoreSettings):
self.settings = settings
self._ssl_context = ssl.create_default_context(cafile=str(settings.ca_file) if settings.ca_file else None)
def _token(self) -> str | None:
path = self.settings.bearer_token_file
if path is None:
return None
try:
metadata = path.lstat()
except OSError as exc:
raise CatalogError("bearer token file cannot be read") from exc
if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISREG(metadata.st_mode):
raise CatalogError("bearer token file must be a regular file, not a symlink")
if metadata.st_mode & (stat.S_IRWXG | stat.S_IRWXO):
raise CatalogError("bearer token file must not be accessible by group or other")
if hasattr(metadata, "st_uid") and metadata.st_uid != 0:
raise CatalogError("bearer token file must be owned by root")
try:
token = path.read_text(encoding="utf-8").strip()
except (OSError, UnicodeDecodeError) as exc:
raise CatalogError("bearer token file cannot be read as UTF-8") from exc
if not 1 <= len(token) <= 4096 or any(ord(char) < 33 or ord(char) > 126 for char in token):
raise CatalogError("bearer token file contains an invalid token")
return token
@staticmethod
def _is_public(address: str) -> bool:
ip = ipaddress.ip_address(address)
return not (
ip.is_private
or ip.is_loopback
or ip.is_link_local
or ip.is_multicast
or ip.is_reserved
or ip.is_unspecified
)
def _connection(self, url: str) -> tuple[http.client.HTTPConnection, str]:
parsed = urlsplit(url)
schemes = {"https", "http"} if self.settings.allow_http_for_testing else {"https"}
if parsed.scheme not in schemes or not parsed.hostname:
raise CatalogError("Store returned a URL with a forbidden scheme or missing host")
if parsed.username or parsed.password or parsed.fragment:
raise CatalogError("Store URL may not contain credentials or a fragment")
host = parsed.hostname.lower()
if host not in self.settings.allowed_hosts:
raise CatalogError("Store URL hostname is not allow-listed")
port = parsed.port or (443 if parsed.scheme == "https" else 80)
try:
records = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)
except OSError as exc:
raise CatalogError("Store hostname could not be resolved") from exc
addresses = list(dict.fromkeys(record[4][0] for record in records))
if not addresses:
raise CatalogError("Store hostname did not resolve to an address")
if not self.settings.allow_private_addresses and any(
not self._is_public(address) for address in addresses
):
raise CatalogError("Store hostname resolves to a non-public address")
address = addresses[0]
if parsed.scheme == "https":
connection: http.client.HTTPConnection = _PinnedHTTPSConnection(
host, port, address, self.settings.timeout_seconds, self._ssl_context
)
else:
connection = _PinnedHTTPConnection(host, port, address, self.settings.timeout_seconds)
target = parsed.path or "/"
if parsed.query:
target += "?" + parsed.query
return connection, target
def _headers(self, url: str) -> dict[str, str]:
headers = {"Accept": "application/json", "User-Agent": "netbox-store-agent/0.1"}
requested = urlsplit(url)
configured = urlsplit(self.settings.base_url)
requested_origin = (
requested.scheme,
requested.hostname.lower() if requested.hostname else "",
requested.port or (443 if requested.scheme == "https" else 80),
)
configured_origin = (
configured.scheme,
configured.hostname.lower() if configured.hostname else "",
configured.port or (443 if configured.scheme == "https" else 80),
)
# Artifact hosts may be separately allow-listed, but Store credentials
# are never delegated to them.
if requested_origin == configured_origin:
token = self._token()
if token:
headers["Authorization"] = f"Bearer {token}"
return headers
def _response(self, url: str) -> tuple[http.client.HTTPConnection, http.client.HTTPResponse]:
connection, target = self._connection(url)
headers = self._headers(url)
try:
connection.request("GET", target, headers=headers)
response = connection.getresponse()
except (OSError, http.client.HTTPException) as exc:
connection.close()
raise CatalogError("Store request failed") from exc
if response.status != 200:
response.read(4096)
connection.close()
raise HttpStatusError(response.status, f"Store returned HTTP {response.status}")
return connection, response
@staticmethod
def _declared_length(response: http.client.HTTPResponse, max_bytes: int) -> int | None:
raw = response.getheader("Content-Length")
if raw is None:
return None
try:
length = int(raw)
except ValueError as exc:
raise CatalogError("Store returned an invalid Content-Length") from exc
if length < 0 or length > max_bytes:
raise CatalogError("Store response exceeds the configured size limit")
return length
def get_json(self, url: str, max_bytes: int) -> Any:
connection, response = self._response(url)
try:
declared = self._declared_length(response, max_bytes)
body = response.read(max_bytes + 1)
if len(body) > max_bytes or (declared is not None and declared != len(body)):
raise CatalogError("Store response has an invalid or excessive size")
finally:
connection.close()
try:
return json.loads(body.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise CatalogError("Store returned invalid UTF-8 JSON") from exc
def download(
self,
url: str,
destination: Path,
*,
max_bytes: int,
expected_size: int,
expected_sha256: str,
) -> None:
connection, response = self._response(url)
descriptor: int | None = None
try:
declared = self._declared_length(response, max_bytes)
if declared is not None and declared != expected_size:
raise CatalogError("artifact Content-Length does not match the approved size")
flags = os.O_CREAT | os.O_EXCL | os.O_WRONLY | getattr(os, "O_NOFOLLOW", 0)
descriptor = os.open(destination, flags, 0o600)
digest = hashlib.sha256()
total = 0
while True:
chunk = response.read(1024 * 1024)
if not chunk:
break
total += len(chunk)
if total > max_bytes or total > expected_size:
raise CatalogError("artifact exceeds its approved size")
digest.update(chunk)
os.write(descriptor, chunk)
os.fsync(descriptor)
if total != expected_size:
raise CatalogError("artifact size does not match the approved size")
if digest.hexdigest() != expected_sha256:
raise CatalogError("artifact SHA-256 does not match the approved digest")
finally:
if descriptor is not None:
os.close(descriptor)
connection.close()
@dataclass(frozen=True)
class PluginMetadata:
slug: str
package_name: str
import_name: str
min_netbox_version: str
max_netbox_version: str
releases: tuple[dict[str, Any], ...]
@dataclass(frozen=True)
class ReleasePlan:
plugin: PluginMetadata
version: str
download_url: str
filename: str
artifact_sha256: str
artifact_size: int
approved_payload_sha256: str
def requirement(self) -> dict[str, Any]:
return {
"package_name": self.plugin.package_name,
"version": self.version,
"download_url": self.download_url,
"filename": self.filename,
"sha256": self.artifact_sha256,
"size": self.artifact_size,
}
def _object(value: Any, context: str) -> dict[str, Any]:
if not isinstance(value, dict) or not all(isinstance(key, str) for key in value):
raise CatalogError(f"{context} must be a JSON object")
return value
def _strict_fields(data: dict[str, Any], required: set[str], context: str) -> None:
missing = sorted(required - set(data))
unknown = sorted(set(data) - required)
if missing:
raise CatalogError(f"{context} is missing fields: {', '.join(missing)}")
if unknown:
raise CatalogError(f"{context} contains unknown fields: {', '.join(unknown)}")
def _bounded_string(value: Any, field: str, maximum: int = 4096, *, empty: bool = False) -> str:
if not isinstance(value, str) or len(value) > maximum or (not empty and not value):
raise CatalogError(f"{field} must be a bounded string")
if any(ord(char) < 32 for char in value):
raise CatalogError(f"{field} contains control characters")
return value
class StoreClient:
PLUGIN_FIELDS = {
"api_version",
"slug",
"name",
"summary",
"description",
"repository_url",
"latest_version",
"package_name",
"import_name",
"min_netbox_version",
"max_netbox_version",
"approved",
"status",
"releases",
}
RELEASE_FIELDS = {
"version",
"download_url",
"sha256",
"artifact_size",
"commit_sha",
"min_netbox_version",
"max_netbox_version",
"published_at",
"approved",
"status",
"immutable",
"approved_payload_sha256",
}
def __init__(self, config: Config, transport: Transport | None = None):
self.config = config
self.transport = transport or SecureHTTPTransport(config.store)
def _url(self, template: str, **values: str) -> str:
quoted = {key: quote(value, safe="") for key, value in values.items()}
return self.config.store.base_url + template.format(**quoted)
def _get_with_slash_fallback(self, url: str) -> Any:
candidates = (url, url[:-1] if url.endswith("/") else url + "/")
for index, candidate in enumerate(dict.fromkeys(candidates)):
try:
return self.transport.get_json(candidate, self.config.store.max_catalog_bytes)
except HttpStatusError as exc:
if exc.http_status == 404 and index == 0:
continue
raise
raise CatalogError("Store resource was not found")
@staticmethod
def _version(value: Any, field: str) -> Version:
text = _bounded_string(value, field, 100)
try:
return Version(text)
except InvalidVersion as exc:
raise CatalogError(f"{field} is not a valid version") from exc
def _check_compatibility(self, minimum: Any, maximum: Any, context: str) -> tuple[str, str]:
minimum_text = _bounded_string(minimum, f"{context}.min_netbox_version", 100)
maximum_text = _bounded_string(maximum, f"{context}.max_netbox_version", 100)
minimum_version = self._version(minimum_text, f"{context}.min_netbox_version")
maximum_version = self._version(maximum_text, f"{context}.max_netbox_version")
current = Version(self.config.policy.netbox_version)
if minimum_version > maximum_version or not minimum_version <= current <= maximum_version:
raise CatalogError(f"{context} is incompatible with configured NetBox")
return minimum_text, maximum_text
def get_plugin(self, slug: str) -> PluginMetadata:
if not SLUG_RE.fullmatch(slug):
raise CatalogError("plugin slug is invalid")
raw = self._get_with_slash_fallback(
self._url(self.config.store.plugin_endpoint_template, plugin_slug=slug)
)
data = _object(raw, "plugin")
_strict_fields(data, self.PLUGIN_FIELDS, "plugin")
if data["api_version"] != "v1" or data["slug"] != slug:
raise CatalogError("plugin identity or API version does not match the request")
if data["approved"] is not True or data["status"] != "approved":
raise CatalogError("plugin is not approved")
package_name = _bounded_string(data["package_name"], "plugin.package_name", 200)
import_name = _bounded_string(data["import_name"], "plugin.import_name", 200)
if not DIST_NAME_RE.fullmatch(package_name) or not IMPORT_NAME_RE.fullmatch(import_name):
raise CatalogError("plugin package_name or import_name is invalid")
minimum, maximum = self._check_compatibility(
data["min_netbox_version"], data["max_netbox_version"], "plugin"
)
for field in ("name", "summary", "description", "repository_url"):
_bounded_string(data[field], f"plugin.{field}", 65535, empty=field in {"summary", "description"})
if data["latest_version"] is not None:
self._version(data["latest_version"], "plugin.latest_version")
releases = data["releases"]
if not isinstance(releases, list) or len(releases) > 1000:
raise CatalogError("plugin.releases must be a bounded array")
release_objects = tuple(_object(item, "plugin.releases[]") for item in releases)
for release in release_objects:
_strict_fields(release, self.RELEASE_FIELDS, "plugin.releases[]")
return PluginMetadata(slug, package_name, import_name, minimum, maximum, release_objects)
def _parse_release(
self, plugin: PluginMetadata, raw: Any, requested_version: str
) -> ReleasePlan:
data = _object(raw, "release")
_strict_fields(data, self.RELEASE_FIELDS, "release")
version_text = _bounded_string(data["version"], "release.version", 100)
version = self._version(version_text, "release.version")
if version_text != requested_version:
raise CatalogError("release version does not match the request")
if version.is_prerelease and not self.config.policy.allow_prereleases:
raise CatalogError("prerelease versions are forbidden by policy")
if data["approved"] is not True or data["status"] != "approved" or data["immutable"] is not True:
raise CatalogError("release is not approved and immutable")
self._check_compatibility(
data["min_netbox_version"], data["max_netbox_version"], "release"
)
download_url = _bounded_string(data["download_url"], "release.download_url", 8192)
sha256 = _bounded_string(data["sha256"], "release.sha256", 64)
if not SHA256_RE.fullmatch(sha256):
raise CatalogError("release.sha256 must be lowercase SHA-256")
size = data["artifact_size"]
if isinstance(size, bool) or not isinstance(size, int) or not 1 <= size <= self.config.store.max_artifact_bytes:
raise CatalogError("release.artifact_size is invalid")
token = _bounded_string(
data["approved_payload_sha256"], "release.approved_payload_sha256", 64
)
if not SHA256_RE.fullmatch(token):
raise CatalogError("release.approved_payload_sha256 must be lowercase SHA-256")
commit = _bounded_string(data["commit_sha"], "release.commit_sha", 64, empty=True)
if commit and (len(commit) != 40 or any(char not in "0123456789abcdef" for char in commit)):
raise CatalogError("release.commit_sha is invalid")
if data["published_at"] is not None:
_bounded_string(data["published_at"], "release.published_at", 100)
parsed_url = urlsplit(download_url)
filename = Path(parsed_url.path).name
if not filename or not filename.endswith(".whl") or len(filename) > 255:
raise CatalogError("approved artifact must be a wheel with a safe filename")
# URL host/scheme/DNS are revalidated by the transport at download time.
return ReleasePlan(plugin, version_text, download_url, filename, sha256, size, token)
def get_release(self, slug: str, version: str) -> ReleasePlan:
plugin = self.get_plugin(slug)
url = self._url(
self.config.store.release_endpoint_template, plugin_slug=slug, version=version
)
try:
raw = self._get_with_slash_fallback(url)
except HttpStatusError as exc:
if exc.http_status != 404:
raise
matches = [item for item in plugin.releases if item.get("version") == version]
if len(matches) != 1:
raise CatalogError("approved release was not found") from exc
raw = matches[0]
return self._parse_release(plugin, raw, version)
def download_release(self, plan: ReleasePlan, directory: Path) -> Path:
destination = directory / plan.filename
self.transport.download(
plan.download_url,
destination,
max_bytes=self.config.store.max_artifact_bytes,
expected_size=plan.artifact_size,
expected_sha256=plan.artifact_sha256,
)
try:
distribution, wheel_version, _build, _tags = parse_wheel_filename(plan.filename)
except (InvalidVersion, ValueError) as exc:
raise CatalogError("artifact filename is not a valid wheel filename") from exc
if canonicalize_name(distribution) != canonicalize_name(plan.plugin.package_name):
raise CatalogError("wheel distribution does not match approved package_name")
if wheel_version != Version(plan.version):
raise CatalogError("wheel version does not match approved release version")
try:
with zipfile.ZipFile(destination) as wheel:
members = wheel.infolist()
if len(members) > 10_000:
raise CatalogError("wheel contains too many archive members")
expanded = 0
for member in members:
parts = Path(member.filename).parts
if (
not member.filename
or member.filename.startswith(("/", "\\"))
or ".." in parts
or "\x00" in member.filename
):
raise CatalogError("wheel contains an unsafe archive member")
expanded += member.file_size
if expanded > min(self.config.store.max_artifact_bytes * 20, 2 * 1024**3):
raise CatalogError("wheel expands beyond the configured safety limit")
if wheel.testzip() is not None:
raise CatalogError("wheel archive failed its integrity check")
except (OSError, zipfile.BadZipFile) as exc:
raise CatalogError("artifact is not a valid wheel archive") from exc
return destination
host_agent/src/netbox_store_agent/cli.py
+100
View File
@@ -0,0 +1,100 @@
from __future__ import annotations
import argparse
import getpass
import json
import signal
import sys
import threading
import uuid
from pathlib import Path
from typing import Any
from .client import exchange
from .config import load_config
from .constants import ACTIONS, DEFAULT_CONFIG_PATH, PROTOCOL_VERSION
from .daemon import serve
from .errors import AgentError
def _print_response(value: dict[str, Any]) -> int:
print(json.dumps(value, indent=2, ensure_ascii=False, sort_keys=True))
return 0 if int(value.get("status", 500)) < 400 else 1
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="netbox-store-agent")
parser.add_argument("--config", default=DEFAULT_CONFIG_PATH)
parser.add_argument("--socket", default="/run/netbox-store-agent/agent.sock")
subcommands = parser.add_subparsers(dest="command", required=True)
subcommands.add_parser("daemon", help="run the Unix-socket daemon")
subcommands.add_parser("validate-config", help="validate the root-owned TOML configuration")
subcommands.add_parser("capabilities", help="query daemon capabilities")
status = subcommands.add_parser("status", help="query one operation")
status.add_argument("operation_id")
submit = subcommands.add_parser("submit", help="submit one lifecycle operation")
submit.add_argument("action", choices=sorted(ACTIONS))
submit.add_argument("plugin_slug")
submit.add_argument("--version")
submit.add_argument("--approved-payload-sha256")
submit.add_argument("--idempotency-key", default=None)
submit.add_argument("--request-id", default=None)
submit.add_argument("--requested-by", default=f"cli:{getpass.getuser()}")
return parser
def main(argv: list[str] | None = None) -> int:
arguments = build_parser().parse_args(argv)
try:
if arguments.command == "validate-config":
load_config(arguments.config)
print("configuration is valid")
return 0
if arguments.command == "daemon":
config = load_config(arguments.config)
stop = threading.Event()
def request_stop(_signum: int, _frame: object) -> None:
stop.set()
signal.signal(signal.SIGTERM, request_stop)
signal.signal(signal.SIGINT, request_stop)
serve(config, stop)
return 0
if arguments.command == "capabilities":
request = {
"protocol_version": PROTOCOL_VERSION,
"method": "GET",
"path": "/v1/capabilities",
}
elif arguments.command == "status":
request = {
"protocol_version": PROTOCOL_VERSION,
"method": "GET",
"path": f"/v1/operations/{arguments.operation_id}",
}
else:
request = {
"protocol_version": PROTOCOL_VERSION,
"method": "POST",
"path": "/v1/operations",
"idempotency_key": arguments.idempotency_key or str(uuid.uuid4()),
"body": {
"request_id": arguments.request_id or str(uuid.uuid4()),
"action": arguments.action,
"plugin_slug": arguments.plugin_slug,
"version": arguments.version,
"approved_payload_sha256": arguments.approved_payload_sha256,
"requested_by": arguments.requested_by,
},
}
return _print_response(exchange(Path(arguments.socket), request))
except (AgentError, OSError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 2
if __name__ == "__main__": # pragma: no cover
raise SystemExit(main())
host_agent/src/netbox_store_agent/client.py
+47
View File
@@ -0,0 +1,47 @@
from __future__ import annotations
import json
import socket
from pathlib import Path
from typing import Any
from .constants import DEFAULT_MAX_REQUEST_BYTES, PROTOCOL_VERSION
from .errors import ValidationError
def exchange(socket_path: str | Path, request: dict[str, Any], timeout: float = 10) -> dict[str, Any]:
payload = (json.dumps(request, separators=(",", ":"), ensure_ascii=False) + "\n").encode("utf-8")
if len(payload) > DEFAULT_MAX_REQUEST_BYTES:
raise ValidationError("request exceeds the protocol size limit")
connection = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
connection.settimeout(timeout)
try:
connection.connect(str(socket_path))
connection.sendall(payload)
buffer = bytearray()
while b"\n" not in buffer:
chunk = connection.recv(4096)
if not chunk:
raise ValidationError("agent closed without a complete response")
buffer.extend(chunk)
if len(buffer) > DEFAULT_MAX_REQUEST_BYTES:
raise ValidationError("agent response exceeds the protocol size limit")
line, separator, trailing = bytes(buffer).partition(b"\n")
if not separator or trailing:
raise ValidationError("agent returned more than one JSON line")
finally:
connection.close()
try:
response = json.loads(line.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ValidationError("agent returned invalid JSON") from exc
if (
not isinstance(response, dict)
or set(response) != {"protocol_version", "status", "body"}
or response.get("protocol_version") != PROTOCOL_VERSION
or isinstance(response.get("status"), bool)
or not isinstance(response.get("status"), int)
or not isinstance(response.get("body"), dict)
):
raise ValidationError("agent returned an invalid protocol response")
return response
host_agent/src/netbox_store_agent/config.py
+383
View File
@@ -0,0 +1,383 @@
from __future__ import annotations
import os
import stat
import string
import tomllib
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from urllib.parse import urlsplit
from packaging.version import InvalidVersion, Version
from .constants import DEFAULT_MAX_REQUEST_BYTES
from .errors import PolicyError, ValidationError
from .util import (
SERVICE_NAME_RE,
is_relative_to,
reject_unknown,
require_absolute_path,
require_mapping,
)
@dataclass(frozen=True)
class AgentSettings:
socket_path: Path
journal_path: Path
lock_path: Path
backup_dir: Path
dry_run: bool
require_root: bool
require_peer_credentials: bool
allowed_peer_uids: tuple[int, ...]
allowed_peer_gids: tuple[int, ...]
socket_mode: int
socket_uid: int
socket_gid: int
max_request_bytes: int
connection_timeout_seconds: float
worker_threads: int
@dataclass(frozen=True)
class StoreSettings:
base_url: str
plugin_endpoint_template: str
release_endpoint_template: str
timeout_seconds: float
max_catalog_bytes: int
max_artifact_bytes: int
allow_private_addresses: bool
allow_http_for_testing: bool
allowed_hosts: tuple[str, ...]
bearer_token_file: Path | None
ca_file: Path | None
@dataclass(frozen=True)
class PathSettings:
allowed_root: Path
include_path: Path
requirements_path: Path
temp_dir: Path
@dataclass(frozen=True)
class CommandSettings:
python_path: Path
manage_path: Path
systemctl_path: Path
services: tuple[str, ...]
command_timeout_seconds: int
@dataclass(frozen=True)
class PolicySettings:
netbox_version: str
min_supported_netbox: str
max_supported_netbox: str
self_plugin_slugs: tuple[str, ...]
allow_prereleases: bool
require_release_for_enable: bool
@dataclass(frozen=True)
class Config:
agent: AgentSettings
store: StoreSettings
paths: PathSettings
commands: CommandSettings
policy: PolicySettings
def _table(root: dict[str, Any], name: str, allowed: set[str]) -> dict[str, Any]:
value = require_mapping(root.get(name), name)
reject_unknown(value, allowed, name)
return value
def _bool(data: dict[str, Any], key: str, default: bool) -> bool:
value = data.get(key, default)
if not isinstance(value, bool):
raise ValidationError(f"{key} must be a boolean")
return value
def _int(data: dict[str, Any], key: str, default: int, minimum: int, maximum: int) -> int:
value = data.get(key, default)
if isinstance(value, bool) or not isinstance(value, int) or not minimum <= value <= maximum:
raise ValidationError(f"{key} must be an integer between {minimum} and {maximum}")
return value
def _float(data: dict[str, Any], key: str, default: float, minimum: float, maximum: float) -> float:
value = data.get(key, default)
if isinstance(value, bool) or not isinstance(value, (int, float)):
raise ValidationError(f"{key} must be numeric")
result = float(value)
if not minimum <= result <= maximum:
raise ValidationError(f"{key} must be between {minimum} and {maximum}")
return result
def _string(data: dict[str, Any], key: str, default: str | None = None) -> str:
value = data.get(key, default)
if not isinstance(value, str) or not value:
raise ValidationError(f"{key} must be a non-empty string")
if "\x00" in value:
raise ValidationError(f"{key} contains a NUL byte")
return value
def _int_tuple(data: dict[str, Any], key: str, default: tuple[int, ...]) -> tuple[int, ...]:
value = data.get(key, list(default))
if not isinstance(value, list) or not value:
raise ValidationError(f"{key} must be a non-empty array")
result: list[int] = []
for item in value:
if isinstance(item, bool) or not isinstance(item, int) or item < 0:
raise ValidationError(f"{key} must contain non-negative integers")
result.append(item)
return tuple(sorted(set(result)))
def _string_tuple(data: dict[str, Any], key: str, default: tuple[str, ...]) -> tuple[str, ...]:
value = data.get(key, list(default))
if not isinstance(value, list) or not value or not all(isinstance(item, str) and item for item in value):
raise ValidationError(f"{key} must be a non-empty string array")
return tuple(dict.fromkeys(value))
def _optional_path(data: dict[str, Any], key: str) -> Path | None:
value = data.get(key)
if value in (None, ""):
return None
return require_absolute_path(value, key)
def _validate_endpoint_template(value: str, field: str, expected: set[str]) -> None:
if not value.startswith("/") or value.startswith("//") or "?" in value or "#" in value:
raise ValidationError(f"{field} must be an absolute URL path without query or fragment")
fields = {name for _, name, _, _ in string.Formatter().parse(value) if name is not None}
if fields != expected:
raise ValidationError(f"{field} placeholders must be exactly: {', '.join(sorted(expected))}")
if ".." in value.split("/"):
raise ValidationError(f"{field} may not contain '..'")
def _validate_config_file(path: Path, require_root_owner: bool) -> None:
try:
metadata = path.lstat()
except FileNotFoundError as exc:
raise ValidationError(f"configuration file does not exist: {path}") from exc
if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISREG(metadata.st_mode):
raise ValidationError("configuration file must be a regular file, not a symlink")
if os.name == "posix" and metadata.st_mode & (stat.S_IWGRP | stat.S_IWOTH):
raise ValidationError("configuration file must not be group/world writable")
if (
os.name == "posix"
and require_root_owner
and hasattr(metadata, "st_uid")
and metadata.st_uid != 0
):
raise ValidationError("configuration file must be owned by root")
def load_config(path: str | Path, *, allow_insecure_owner: bool = False) -> Config:
config_path = Path(path)
_validate_config_file(config_path, require_root_owner=not allow_insecure_owner)
try:
with config_path.open("rb") as handle:
root = tomllib.load(handle)
except (OSError, tomllib.TOMLDecodeError) as exc:
raise ValidationError(f"cannot read configuration: {exc}") from exc
reject_unknown(root, {"agent", "store", "paths", "commands", "policy"}, "configuration")
missing_tables = sorted({"agent", "store", "paths", "commands", "policy"} - set(root))
if missing_tables:
raise ValidationError(f"configuration is missing tables: {', '.join(missing_tables)}")
agent_data = _table(
root,
"agent",
{
"socket_path",
"journal_path",
"lock_path",
"backup_dir",
"dry_run",
"require_root",
"require_peer_credentials",
"allowed_peer_uids",
"allowed_peer_gids",
"socket_mode",
"socket_uid",
"socket_gid",
"max_request_bytes",
"connection_timeout_seconds",
"worker_threads",
},
)
store_data = _table(
root,
"store",
{
"base_url",
"plugin_endpoint_template",
"release_endpoint_template",
"timeout_seconds",
"max_catalog_bytes",
"max_artifact_bytes",
"allow_private_addresses",
"allow_http_for_testing",
"allowed_hosts",
"bearer_token_file",
"ca_file",
},
)
paths_data = _table(root, "paths", {"allowed_root", "include_path", "requirements_path", "temp_dir"})
commands_data = _table(
root,
"commands",
{"python_path", "manage_path", "systemctl_path", "services", "command_timeout_seconds"},
)
policy_data = _table(
root,
"policy",
{
"netbox_version",
"min_supported_netbox",
"max_supported_netbox",
"self_plugin_slugs",
"allow_prereleases",
"require_release_for_enable",
},
)
agent = AgentSettings(
socket_path=require_absolute_path(_string(agent_data, "socket_path"), "socket_path"),
journal_path=require_absolute_path(_string(agent_data, "journal_path"), "journal_path"),
lock_path=require_absolute_path(_string(agent_data, "lock_path"), "lock_path"),
backup_dir=require_absolute_path(_string(agent_data, "backup_dir"), "backup_dir"),
dry_run=_bool(agent_data, "dry_run", True),
require_root=_bool(agent_data, "require_root", True),
require_peer_credentials=_bool(agent_data, "require_peer_credentials", True),
allowed_peer_uids=_int_tuple(agent_data, "allowed_peer_uids", (0,)),
allowed_peer_gids=_int_tuple(agent_data, "allowed_peer_gids", (0,)),
socket_mode=_int(agent_data, "socket_mode", 0o660, 0, 0o777),
socket_uid=_int(agent_data, "socket_uid", 0, 0, 2**31 - 1),
socket_gid=_int(agent_data, "socket_gid", 0, 0, 2**31 - 1),
max_request_bytes=_int(
agent_data, "max_request_bytes", DEFAULT_MAX_REQUEST_BYTES, 1024, DEFAULT_MAX_REQUEST_BYTES
),
connection_timeout_seconds=_float(agent_data, "connection_timeout_seconds", 5, 0.1, 60),
worker_threads=_int(agent_data, "worker_threads", 1, 1, 4),
)
allow_http = _bool(store_data, "allow_http_for_testing", False)
base_url = _string(store_data, "base_url").rstrip("/")
parsed_url = urlsplit(base_url)
allowed_schemes = {"https", "http"} if allow_http else {"https"}
if parsed_url.scheme not in allowed_schemes or not parsed_url.hostname:
raise ValidationError("store.base_url must use HTTPS and contain a hostname")
if parsed_url.username or parsed_url.password or parsed_url.query or parsed_url.fragment:
raise ValidationError("store.base_url may not contain credentials, query, or fragment")
allowed_hosts = tuple(host.lower() for host in _string_tuple(store_data, "allowed_hosts", (parsed_url.hostname,)))
if parsed_url.hostname.lower() not in allowed_hosts:
raise ValidationError("store.base_url hostname must be present in store.allowed_hosts")
plugin_template = _string(
store_data, "plugin_endpoint_template", "/api/v1/plugins/{plugin_slug}"
)
release_template = _string(
store_data,
"release_endpoint_template",
"/api/v1/plugins/{plugin_slug}/releases/{version}",
)
_validate_endpoint_template(plugin_template, "plugin_endpoint_template", {"plugin_slug"})
_validate_endpoint_template(
release_template, "release_endpoint_template", {"plugin_slug", "version"}
)
store = StoreSettings(
base_url=base_url,
plugin_endpoint_template=plugin_template,
release_endpoint_template=release_template,
timeout_seconds=_float(store_data, "timeout_seconds", 10, 0.1, 120),
max_catalog_bytes=_int(store_data, "max_catalog_bytes", 1024 * 1024, 1024, 8 * 1024 * 1024),
max_artifact_bytes=_int(
store_data, "max_artifact_bytes", 256 * 1024 * 1024, 1024, 2 * 1024 * 1024 * 1024
),
allow_private_addresses=_bool(store_data, "allow_private_addresses", False),
allow_http_for_testing=allow_http,
allowed_hosts=allowed_hosts,
bearer_token_file=_optional_path(store_data, "bearer_token_file"),
ca_file=_optional_path(store_data, "ca_file"),
)
allowed_root = require_absolute_path(_string(paths_data, "allowed_root"), "allowed_root").resolve()
include_path = require_absolute_path(_string(paths_data, "include_path"), "include_path")
requirements_path = require_absolute_path(
_string(paths_data, "requirements_path"), "requirements_path"
)
for field, path_value in (("include_path", include_path), ("requirements_path", requirements_path)):
if not is_relative_to(path_value.resolve(strict=False), allowed_root):
raise ValidationError(f"paths.{field} must be below paths.allowed_root")
if include_path.resolve(strict=False) == requirements_path.resolve(strict=False):
raise ValidationError("paths.include_path and paths.requirements_path must be distinct")
if include_path.name == requirements_path.name:
raise ValidationError("managed files must have distinct basenames for unambiguous backups")
paths = PathSettings(
allowed_root=allowed_root,
include_path=include_path,
requirements_path=requirements_path,
temp_dir=require_absolute_path(_string(paths_data, "temp_dir"), "temp_dir"),
)
services = _string_tuple(commands_data, "services", ("netbox", "netbox-rq"))
if any(not SERVICE_NAME_RE.fullmatch(service) for service in services):
raise ValidationError("commands.services contains an invalid systemd unit name")
commands = CommandSettings(
python_path=require_absolute_path(_string(commands_data, "python_path"), "python_path"),
manage_path=require_absolute_path(_string(commands_data, "manage_path"), "manage_path"),
systemctl_path=require_absolute_path(_string(commands_data, "systemctl_path"), "systemctl_path"),
services=services,
command_timeout_seconds=_int(commands_data, "command_timeout_seconds", 900, 1, 7200),
)
versions: dict[str, str] = {}
for key, default in (
("netbox_version", "4.6.8"),
("min_supported_netbox", "4.6.5"),
("max_supported_netbox", "4.6.8"),
):
value = _string(policy_data, key, default)
try:
Version(value)
except InvalidVersion as exc:
raise ValidationError(f"policy.{key} is not a valid version") from exc
versions[key] = value
if not Version(versions["min_supported_netbox"]) <= Version(versions["netbox_version"]) <= Version(
versions["max_supported_netbox"]
):
raise PolicyError("configured NetBox version is outside the agent support range")
policy = PolicySettings(
netbox_version=versions["netbox_version"],
min_supported_netbox=versions["min_supported_netbox"],
max_supported_netbox=versions["max_supported_netbox"],
self_plugin_slugs=tuple(
slug.lower()
for slug in _string_tuple(
policy_data,
"self_plugin_slugs",
("netbox-store", "netbox-plugin-store", "netbox_plugin_store"),
)
),
allow_prereleases=_bool(policy_data, "allow_prereleases", False),
require_release_for_enable=_bool(policy_data, "require_release_for_enable", True),
)
return Config(agent=agent, store=store, paths=paths, commands=commands, policy=policy)
def enforce_runtime_identity(config: Config) -> None:
if config.agent.require_root and hasattr(os, "geteuid") and os.geteuid() != 0:
raise PolicyError("daemon must run as root")
host_agent/src/netbox_store_agent/constants.py
+9
View File
@@ -0,0 +1,9 @@
from __future__ import annotations
AGENT_VERSION = "0.1.0"
PROTOCOL_VERSION = 1
DEFAULT_CONFIG_PATH = "/etc/netbox-store-agent/agent.toml"
DEFAULT_MAX_REQUEST_BYTES = 64 * 1024
ACTIONS = frozenset({"install", "update", "enable", "disable", "uninstall"})
TERMINAL_STATES = frozenset({"succeeded", "failed", "manual_recovery", "dry_run"})
host_agent/src/netbox_store_agent/daemon.py
+137
View File
@@ -0,0 +1,137 @@
from __future__ import annotations
import os
import socket
import stat
import struct
import threading
from pathlib import Path
from .config import Config, enforce_runtime_identity
from .errors import AgentError, AuthenticationError, ValidationError
from .journal import Journal
from .protocol import decode_request, encode_response, error_response
from .service import AgentService
def _authorize_peer(connection: socket.socket, config: Config) -> None:
if not config.agent.require_peer_credentials:
return
option = getattr(socket, "SO_PEERCRED", None)
if option is None:
raise AuthenticationError("platform does not expose Unix peer credentials")
try:
raw = connection.getsockopt(socket.SOL_SOCKET, option, struct.calcsize("3i"))
_pid, uid, gid = struct.unpack("3i", raw)
except (OSError, struct.error) as exc:
raise AuthenticationError("Unix peer credentials could not be verified") from exc
if uid not in config.agent.allowed_peer_uids and gid not in config.agent.allowed_peer_gids:
raise AuthenticationError("Unix peer is not allow-listed")
def _read_frame(connection: socket.socket, maximum: int) -> bytes:
buffer = bytearray()
while True:
chunk = connection.recv(min(4096, maximum + 1 - len(buffer)))
if not chunk:
raise ValidationError("connection closed before the JSON-line terminator")
buffer.extend(chunk)
if len(buffer) > maximum:
raise ValidationError("request exceeds the configured size limit")
newline = buffer.find(b"\n")
if newline >= 0:
if newline >= maximum or newline != len(buffer) - 1:
raise ValidationError("connection must contain exactly one JSON line")
return bytes(buffer[:newline])
def handle_connection(connection: socket.socket, config: Config, service: AgentService) -> None:
try:
connection.settimeout(config.agent.connection_timeout_seconds)
_authorize_peer(connection, config)
request = decode_request(_read_frame(connection, config.agent.max_request_bytes), config.agent.max_request_bytes)
result = service.handle(request)
except AgentError as exc:
result = error_response(exc)
except (OSError, TimeoutError):
result = error_response(ValidationError("connection timed out or failed"))
except Exception:
result = error_response(
AgentError("unexpected internal error", code="unexpected_error", status=500)
)
try:
try:
connection.shutdown(socket.SHUT_RD)
except OSError:
pass
connection.sendall(encode_response(result))
except OSError:
pass
finally:
connection.close()
def _systemd_socket() -> socket.socket | None:
try:
listen_pid = int(os.environ.get("LISTEN_PID", "0"))
listen_fds = int(os.environ.get("LISTEN_FDS", "0"))
except ValueError:
return None
if listen_pid != os.getpid() or listen_fds != 1:
return None
inherited = socket.socket(fileno=3)
if inherited.family != socket.AF_UNIX or inherited.type & socket.SOCK_STREAM != socket.SOCK_STREAM:
inherited.close()
raise ValidationError("systemd passed an unexpected socket type")
os.environ.pop("LISTEN_PID", None)
os.environ.pop("LISTEN_FDS", None)
os.environ.pop("LISTEN_FDNAMES", None)
return inherited
def _bind_socket(config: Config) -> socket.socket:
path = config.agent.socket_path
path.parent.mkdir(parents=True, exist_ok=True, mode=0o750)
if path.exists() or path.is_symlink():
metadata = path.lstat()
if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISSOCK(metadata.st_mode):
raise ValidationError("configured socket path exists and is not a Unix socket")
path.unlink()
listener = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
listener.bind(str(path))
os.chmod(path, config.agent.socket_mode)
os.chown(path, config.agent.socket_uid, config.agent.socket_gid)
listener.listen(32)
except Exception:
listener.close()
raise
return listener
def serve(config: Config, stop_event: threading.Event | None = None) -> None:
enforce_runtime_identity(config)
stop = stop_event or threading.Event()
listener = _systemd_socket()
owns_socket = listener is None
if listener is None:
listener = _bind_socket(config)
journal = Journal(
config.agent.journal_path, require_root_owner=config.agent.require_root
)
service = AgentService(config, journal)
listener.settimeout(1.0)
try:
while not stop.is_set():
try:
connection, _address = listener.accept()
except socket.timeout:
continue
handle_connection(connection, config, service)
finally:
listener.close()
service.close()
if owns_socket:
path = Path(config.agent.socket_path)
if path.exists() and stat.S_ISSOCK(path.lstat().st_mode):
path.unlink()
host_agent/src/netbox_store_agent/errors.py
+55
View File
@@ -0,0 +1,55 @@
from __future__ import annotations
class AgentError(Exception):
"""Base error carrying a stable machine-readable code and HTTP-like status."""
code = "agent_error"
status = 500
def __init__(self, message: str, *, code: str | None = None, status: int | None = None):
super().__init__(message)
self.message = message
if code is not None:
self.code = code
if status is not None:
self.status = status
class ValidationError(AgentError):
code = "invalid_request"
status = 400
class AuthenticationError(AgentError):
code = "peer_not_allowed"
status = 403
class NotFoundError(AgentError):
code = "not_found"
status = 404
class ConflictError(AgentError):
code = "conflict"
status = 409
class CatalogError(AgentError):
code = "catalog_rejected"
status = 422
class PolicyError(AgentError):
code = "policy_rejected"
status = 422
class ExecutionError(AgentError):
code = "execution_failed"
status = 500
class ManualRecoveryRequired(ExecutionError):
code = "manual_recovery_required"
host_agent/src/netbox_store_agent/executor.py
+353
View File
@@ -0,0 +1,353 @@
from __future__ import annotations
import secrets
import tempfile
from pathlib import Path
from typing import Callable
from .catalog import PluginMetadata, ReleasePlan, StoreClient
from .config import Config
from .errors import (
AgentError,
ConflictError,
ManualRecoveryRequired,
NotFoundError,
PolicyError,
)
from .journal import Journal, ManagedPlugin
from .locking import GlobalFileLock
from .managed_files import FileSnapshot, ManagedFiles
from .protocol import OperationRequest
from .runner import Runner, SubprocessRunner
class OperationProcessor:
def __init__(
self,
config: Config,
journal: Journal,
*,
store: StoreClient | None = None,
files: ManagedFiles | None = None,
runner: Runner | None = None,
lock_factory: Callable[[Path], GlobalFileLock] = GlobalFileLock,
):
self.config = config
self.journal = journal
self.store = store or StoreClient(config)
self.files = files or ManagedFiles(config)
self.runner = runner or SubprocessRunner(config)
self.lock_factory = lock_factory
self._mutated_operations: set[str] = set()
self._snapshots: dict[str, FileSnapshot] = {}
def _event(self, operation_id: str, step: str, message: str) -> None:
self.journal.add_event(operation_id, "info", step, message)
self.journal.transition(operation_id, "running", step=step)
def _assert_not_self(self, request: OperationRequest) -> None:
if request.plugin_slug.lower() in self.config.policy.self_plugin_slugs:
raise PolicyError("self-management is forbidden")
@staticmethod
def _same_identity(existing: ManagedPlugin, plugin: PluginMetadata) -> None:
if existing.package_name != plugin.package_name or existing.import_name != plugin.import_name:
raise PolicyError("approved package/import identity changed for an existing plugin")
@staticmethod
def _same_release(existing: ManagedPlugin, plan: ReleasePlan) -> None:
if existing.version != plan.version or existing.requirements != (plan.requirement(),):
raise PolicyError(
"managed artifact lock differs from the current immutable Store release"
)
def _plugins_with(
self, replacement: ManagedPlugin | None = None, *, delete_slug: str | None = None
) -> list[ManagedPlugin]:
plugins = []
replaced = False
for item in self.journal.list_managed_plugins():
if item.slug == delete_slug:
continue
if replacement is not None and item.slug == replacement.slug:
plugins.append(replacement)
replaced = True
else:
plugins.append(item)
if replacement is not None and not replaced:
plugins.append(replacement)
return plugins
def _run(self, operation_id: str, step: str, argv: list[str]) -> None:
self._event(operation_id, step, f"Executing fixed {step} command")
self.runner.run(argv)
def _pip_install(
self, operation_id: str, directory: Path, plugin: ManagedPlugin, wheel: Path
) -> None:
requirements = self.files.make_local_requirements(directory, plugin, wheel)
self._run(
operation_id,
"pip_install",
[
str(self.config.commands.python_path),
"-m",
"pip",
"install",
"--no-input",
"--disable-pip-version-check",
"--no-index",
"--no-deps",
"--only-binary=:all:",
"--require-hashes",
"-r",
str(requirements),
],
)
def _pip_uninstall(self, operation_id: str, package_name: str) -> None:
self._run(
operation_id,
"pip_uninstall",
[
str(self.config.commands.python_path),
"-m",
"pip",
"uninstall",
"--yes",
package_name,
],
)
def _netbox_prepare(self, operation_id: str) -> None:
python = str(self.config.commands.python_path)
manage = str(self.config.commands.manage_path)
self._run(operation_id, "migrate", [python, manage, "migrate", "--no-input"])
self._run(
operation_id,
"collectstatic",
[python, manage, "collectstatic", "--no-input"],
)
def _restart(self, operation_id: str) -> None:
self._run(
operation_id,
"restart",
[
str(self.config.commands.systemctl_path),
"restart",
*self.config.commands.services,
],
)
def _write(self, operation_id: str, plugins: list[ManagedPlugin]) -> FileSnapshot:
self._event(operation_id, "managed_files", "Writing managed include and requirements")
snapshot = self.files.write(operation_id, plugins)
self._snapshots[operation_id] = snapshot
return snapshot
def _mark_mutated(self, operation_id: str) -> None:
self._mutated_operations.add(operation_id)
def _release_for_request(self, request: OperationRequest) -> ReleasePlan:
assert request.version is not None
plan = self.store.get_release(request.plugin_slug, request.version)
if request.approved_payload_sha256 is None or not secrets.compare_digest(
request.approved_payload_sha256, plan.approved_payload_sha256
):
raise PolicyError("approval payload token no longer matches the Store")
return plan
def _dry_result(self, request: OperationRequest, **extra: object) -> dict[str, object]:
return {
"dry_run": True,
"action": request.action,
"plugin_slug": request.plugin_slug,
**extra,
}
def _execute(
self, operation_id: str, request: OperationRequest
) -> tuple[dict[str, object], bool]:
self._assert_not_self(request)
existing = self.journal.get_managed_plugin(request.plugin_slug)
action = request.action
host_mutated = False
if action in {"install", "update"}:
if action == "install" and existing is not None:
raise ConflictError("plugin is already managed; use update")
if action == "update" and existing is None:
raise NotFoundError("plugin is not managed; use install")
self._event(operation_id, "catalog", "Fetching approved release from Store")
plan = self._release_for_request(request)
if existing is not None:
self._same_identity(existing, plan.plugin)
enabled = existing.enabled if existing else False
managed = ManagedPlugin(
slug=request.plugin_slug,
package_name=plan.plugin.package_name,
import_name=plan.plugin.import_name,
version=plan.version,
enabled=enabled,
requirements=(plan.requirement(),),
)
temp_root = self.config.paths.temp_dir
if temp_root.is_symlink():
raise PolicyError("temporary directory may not be a symlink")
temp_root.mkdir(parents=True, exist_ok=True, mode=0o700)
with tempfile.TemporaryDirectory(prefix="operation-", dir=temp_root) as temporary:
self._event(operation_id, "artifact", "Downloading and verifying approved wheel")
wheel = self.store.download_release(plan, Path(temporary))
if self.config.agent.dry_run:
return self._dry_result(request, version=plan.version, artifact_verified=True), False
self._mark_mutated(operation_id)
host_mutated = True
self._pip_install(operation_id, Path(temporary), managed, wheel)
self._write(operation_id, self._plugins_with(managed))
if enabled:
self._netbox_prepare(operation_id)
self._restart(operation_id)
self.journal.upsert_managed_plugin(managed)
return {
"dry_run": False,
"action": action,
"plugin_slug": request.plugin_slug,
"version": plan.version,
"enabled": enabled,
}, host_mutated
if existing is None:
raise NotFoundError("plugin is not managed")
self._event(operation_id, "catalog", "Re-fetching approved plugin metadata")
if action == "enable" and self.config.policy.require_release_for_enable:
plan = self.store.get_release(existing.slug, existing.version)
plugin = plan.plugin
self._same_release(existing, plan)
else:
plugin = self.store.get_plugin(existing.slug)
self._same_identity(existing, plugin)
if action == "enable":
if existing.enabled:
raise ConflictError("plugin is already enabled")
replacement = ManagedPlugin(
existing.slug,
existing.package_name,
existing.import_name,
existing.version,
True,
existing.requirements,
)
if self.config.agent.dry_run:
return self._dry_result(request, version=existing.version, enabled=True), False
self._write(operation_id, self._plugins_with(replacement))
host_mutated = True
self._mark_mutated(operation_id)
self._netbox_prepare(operation_id)
self._restart(operation_id)
self.journal.upsert_managed_plugin(replacement)
return self._dry_result(request, dry_run=False, version=existing.version, enabled=True), True
if action == "disable":
if not existing.enabled:
raise ConflictError("plugin is already disabled")
replacement = ManagedPlugin(
existing.slug,
existing.package_name,
existing.import_name,
existing.version,
False,
existing.requirements,
)
if self.config.agent.dry_run:
return self._dry_result(request, version=existing.version, enabled=False), False
self._write(operation_id, self._plugins_with(replacement))
host_mutated = True
self._mark_mutated(operation_id)
self._restart(operation_id)
self.journal.upsert_managed_plugin(replacement)
return self._dry_result(request, dry_run=False, version=existing.version, enabled=False), True
if action == "uninstall":
if existing.enabled:
raise ConflictError("disable the plugin before uninstalling it")
if self.config.agent.dry_run:
return self._dry_result(request, version=existing.version, removed=True), False
self._mark_mutated(operation_id)
host_mutated = True
self._pip_uninstall(operation_id, existing.package_name)
self._write(operation_id, self._plugins_with(delete_slug=existing.slug))
self.journal.delete_managed_plugin(existing.slug)
return self._dry_result(request, dry_run=False, version=existing.version, removed=True), True
raise PolicyError("unsupported action")
def process(self, operation_id: str) -> None:
if not self.journal.claim(operation_id):
return
self.journal.add_event(operation_id, "info", "starting", "Operation worker started")
host_mutated = False
try:
request = self.journal.get_request(operation_id)
with self.lock_factory(self.config.agent.lock_path):
result, host_mutated = self._execute(operation_id, request)
state = "dry_run" if self.config.agent.dry_run else "succeeded"
self.journal.transition(operation_id, state, step="complete", result=result, finished=True)
self.journal.add_event(operation_id, "info", "complete", f"Operation {state}")
except AgentError as exc:
mutated = host_mutated or operation_id in self._mutated_operations
snapshot = self._snapshots.get(operation_id)
if mutated and snapshot is not None:
try:
self.files.restore(snapshot)
self.journal.add_event(
operation_id,
"warning",
"rollback",
"Managed files restored; package/service state still requires inspection.",
)
except AgentError:
exc = ManualRecoveryRequired("managed-file rollback failed")
state = (
"manual_recovery"
if mutated or isinstance(exc, ManualRecoveryRequired)
else "failed"
)
message = exc.message[:2000]
self.journal.transition(
operation_id,
state,
step="failed",
error_code=exc.code,
error_message=message,
finished=True,
)
self.journal.add_event(operation_id, "error", "failed", message)
except Exception:
mutated = host_mutated or operation_id in self._mutated_operations
snapshot = self._snapshots.get(operation_id)
if mutated and snapshot is not None:
try:
self.files.restore(snapshot)
except AgentError:
pass
state = "manual_recovery" if mutated else "failed"
self.journal.transition(
operation_id,
state,
step="failed",
error_code="unexpected_error",
error_message="Unexpected internal error; inspect root-owned service logs.",
finished=True,
)
self.journal.add_event(
operation_id,
"error",
"failed",
"Unexpected internal error; details were withheld from the client.",
)
finally:
self._mutated_operations.discard(operation_id)
self._snapshots.pop(operation_id, None)
host_agent/src/netbox_store_agent/journal.py
+352
View File
@@ -0,0 +1,352 @@
from __future__ import annotations
import json
import os
import sqlite3
import stat
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from .errors import ConflictError, NotFoundError, ValidationError
from .protocol import OperationRequest
from .util import canonical_json, payload_sha256, utc_now
class _ClosingConnection(sqlite3.Connection):
"""sqlite context manager which also releases the OS handle on exit."""
def __exit__(self, exc_type: object, exc_value: object, traceback: object) -> bool:
try:
return super().__exit__(exc_type, exc_value, traceback)
finally:
self.close()
@dataclass(frozen=True)
class ManagedPlugin:
slug: str
package_name: str
import_name: str
version: str
enabled: bool
requirements: tuple[dict[str, Any], ...]
class Journal:
def __init__(self, path: str | Path, *, require_root_owner: bool = False):
self.path = Path(path)
self.require_root_owner = require_root_owner
self._prepare_path()
self._initialize()
def _prepare_path(self) -> None:
self.path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
if self.path.exists() or self.path.is_symlink():
metadata = self.path.lstat()
if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISREG(metadata.st_mode):
raise ValidationError("journal path must be a regular file, not a symlink")
if os.name == "posix" and metadata.st_mode & (stat.S_IRWXG | stat.S_IRWXO):
raise ValidationError("journal file must not be accessible by group or other")
if os.name == "posix" and self.require_root_owner and metadata.st_uid != 0:
raise ValidationError("journal file must be owned by root")
return
flags = os.O_CREAT | os.O_EXCL | os.O_WRONLY
flags |= getattr(os, "O_NOFOLLOW", 0)
descriptor = os.open(self.path, flags, 0o600)
os.close(descriptor)
try:
os.chmod(self.path, 0o600)
except OSError:
pass
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(
self.path, timeout=30, isolation_level=None, factory=_ClosingConnection
)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA synchronous=FULL")
return connection
def _initialize(self) -> None:
with self._connect() as connection:
connection.execute("PRAGMA journal_mode=WAL")
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS operations (
operation_id TEXT PRIMARY KEY,
request_hash TEXT NOT NULL,
request_json TEXT NOT NULL,
request_id TEXT NOT NULL,
action TEXT NOT NULL,
plugin_slug TEXT NOT NULL,
version TEXT,
approved_payload_sha256 TEXT,
requested_by TEXT NOT NULL,
state TEXT NOT NULL,
current_step TEXT NOT NULL DEFAULT '',
submitted_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
finished_at TEXT,
error_code TEXT,
error_message TEXT,
result_json TEXT
);
CREATE INDEX IF NOT EXISTS operations_state_idx ON operations(state, submitted_at);
CREATE TABLE IF NOT EXISTS operation_events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
operation_id TEXT NOT NULL REFERENCES operations(operation_id) ON DELETE CASCADE,
created_at TEXT NOT NULL,
level TEXT NOT NULL,
step TEXT NOT NULL,
message TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS operation_events_operation_idx
ON operation_events(operation_id, id);
CREATE TABLE IF NOT EXISTS managed_plugins (
slug TEXT PRIMARY KEY,
package_name TEXT NOT NULL,
import_name TEXT NOT NULL,
version TEXT NOT NULL,
enabled INTEGER NOT NULL CHECK(enabled IN (0, 1)),
requirements_json TEXT NOT NULL,
updated_at TEXT NOT NULL
);
"""
)
@staticmethod
def _operation_dict(row: sqlite3.Row, events: list[dict[str, Any]] | None = None) -> dict[str, Any]:
result = json.loads(row["result_json"]) if row["result_json"] else None
operation = {
"operation_id": row["operation_id"],
"request_id": row["request_id"],
"action": row["action"],
"plugin_slug": row["plugin_slug"],
"version": row["version"],
"approved_payload_sha256": row["approved_payload_sha256"],
"requested_by": row["requested_by"],
"state": row["state"],
"current_step": row["current_step"],
"submitted_at": row["submitted_at"],
"updated_at": row["updated_at"],
"finished_at": row["finished_at"],
"error": (
{"code": row["error_code"], "message": row["error_message"]}
if row["error_code"]
else None
),
"result": result,
}
if events is not None:
operation["events"] = events
return operation
def submit(self, operation_id: str, request: OperationRequest) -> tuple[dict[str, Any], bool]:
request_data = request.as_dict()
request_json = canonical_json(request_data)
request_hash = payload_sha256(request_data)
now = utc_now()
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute(
"SELECT * FROM operations WHERE operation_id = ?", (operation_id,)
).fetchone()
if existing is not None:
connection.execute("COMMIT")
if existing["request_hash"] != request_hash:
raise ConflictError("idempotency_key is already bound to a different request")
return self._operation_dict(existing), False
connection.execute(
"""
INSERT INTO operations (
operation_id, request_hash, request_json, request_id, action, plugin_slug,
version, approved_payload_sha256, requested_by, state, submitted_at, updated_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, 'queued', ?, ?)
""",
(
operation_id,
request_hash,
request_json,
request.request_id,
request.action,
request.plugin_slug,
request.version,
request.approved_payload_sha256,
request.requested_by,
now,
now,
),
)
connection.execute("COMMIT")
self.add_event(operation_id, "info", "queued", "Operation accepted")
return self.get_operation(operation_id), True
def get_operation(self, operation_id: str, *, include_events: bool = True) -> dict[str, Any]:
with self._connect() as connection:
row = connection.execute(
"SELECT * FROM operations WHERE operation_id = ?", (operation_id,)
).fetchone()
if row is None:
raise NotFoundError("operation not found")
events = None
if include_events:
event_rows = connection.execute(
"""
SELECT created_at, level, step, message FROM operation_events
WHERE operation_id = ? ORDER BY id ASC LIMIT 40
""",
(operation_id,),
).fetchall()
events = [dict(event) for event in event_rows]
return self._operation_dict(row, events)
def get_request(self, operation_id: str) -> OperationRequest:
with self._connect() as connection:
row = connection.execute(
"SELECT request_json FROM operations WHERE operation_id = ?", (operation_id,)
).fetchone()
if row is None:
raise NotFoundError("operation not found")
data = json.loads(row["request_json"])
return OperationRequest(**data)
def claim(self, operation_id: str) -> bool:
"""Atomically move a queued operation to running exactly once."""
now = utc_now()
with self._connect() as connection:
cursor = connection.execute(
"""
UPDATE operations SET state = 'running', current_step = 'starting', updated_at = ?
WHERE operation_id = ? AND state = 'queued'
""",
(now, operation_id),
)
return cursor.rowcount == 1
def transition(
self,
operation_id: str,
state: str,
*,
step: str = "",
error_code: str | None = None,
error_message: str | None = None,
result: dict[str, Any] | None = None,
finished: bool = False,
) -> None:
now = utc_now()
result_json = canonical_json(result) if result is not None else None
with self._connect() as connection:
cursor = connection.execute(
"""
UPDATE operations
SET state = ?, current_step = ?, updated_at = ?,
finished_at = CASE WHEN ? THEN ? ELSE finished_at END,
error_code = ?, error_message = ?, result_json = ?
WHERE operation_id = ?
""",
(
state,
step,
now,
1 if finished else 0,
now,
error_code,
error_message,
result_json,
operation_id,
),
)
if cursor.rowcount != 1:
raise NotFoundError("operation not found")
def add_event(self, operation_id: str, level: str, step: str, message: str) -> None:
safe_level = level if level in {"debug", "info", "warning", "error"} else "info"
safe_step = step[:80]
safe_message = "".join(char for char in message if char == "\t" or ord(char) >= 32)[:1000]
with self._connect() as connection:
connection.execute(
"""
INSERT INTO operation_events(operation_id, created_at, level, step, message)
VALUES (?, ?, ?, ?, ?)
""",
(operation_id, utc_now(), safe_level, safe_step, safe_message),
)
def queued_operations(self) -> list[str]:
with self._connect() as connection:
rows = connection.execute(
"SELECT operation_id FROM operations WHERE state = 'queued' ORDER BY submitted_at"
).fetchall()
return [row["operation_id"] for row in rows]
def recover_interrupted(self) -> int:
now = utc_now()
with self._connect() as connection:
cursor = connection.execute(
"""
UPDATE operations SET state = 'manual_recovery', current_step = 'agent_restart',
updated_at = ?, finished_at = ?, error_code = 'agent_restarted',
error_message = 'Agent restarted while the operation was running; inspect host state.'
WHERE state = 'running'
""",
(now, now),
)
return cursor.rowcount
@staticmethod
def _managed_from_row(row: sqlite3.Row) -> ManagedPlugin:
requirements = json.loads(row["requirements_json"])
return ManagedPlugin(
slug=row["slug"],
package_name=row["package_name"],
import_name=row["import_name"],
version=row["version"],
enabled=bool(row["enabled"]),
requirements=tuple(requirements),
)
def get_managed_plugin(self, slug: str) -> ManagedPlugin | None:
with self._connect() as connection:
row = connection.execute(
"SELECT * FROM managed_plugins WHERE slug = ?", (slug,)
).fetchone()
return self._managed_from_row(row) if row else None
def list_managed_plugins(self) -> list[ManagedPlugin]:
with self._connect() as connection:
rows = connection.execute("SELECT * FROM managed_plugins ORDER BY slug").fetchall()
return [self._managed_from_row(row) for row in rows]
def upsert_managed_plugin(self, plugin: ManagedPlugin) -> None:
requirements_json = canonical_json(list(plugin.requirements))
with self._connect() as connection:
connection.execute(
"""
INSERT INTO managed_plugins(
slug, package_name, import_name, version, enabled, requirements_json, updated_at
) VALUES (?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(slug) DO UPDATE SET
package_name=excluded.package_name,
import_name=excluded.import_name,
version=excluded.version,
enabled=excluded.enabled,
requirements_json=excluded.requirements_json,
updated_at=excluded.updated_at
""",
(
plugin.slug,
plugin.package_name,
plugin.import_name,
plugin.version,
int(plugin.enabled),
requirements_json,
utc_now(),
),
)
def delete_managed_plugin(self, slug: str) -> None:
with self._connect() as connection:
connection.execute("DELETE FROM managed_plugins WHERE slug = ?", (slug,))
host_agent/src/netbox_store_agent/locking.py
+80
View File
@@ -0,0 +1,80 @@
from __future__ import annotations
import os
import stat
from pathlib import Path
from types import TracebackType
from .errors import ConflictError, ValidationError
try:
import fcntl
except ImportError: # pragma: no cover - Linux is the production target.
fcntl = None
try:
import msvcrt
except ImportError: # pragma: no cover - Windows-only test fallback.
msvcrt = None
class GlobalFileLock:
def __init__(self, path: str | Path):
self.path = Path(path)
self._descriptor: int | None = None
def __enter__(self) -> "GlobalFileLock":
self.path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
if self.path.is_symlink():
raise ValidationError("lock path may not be a symlink")
flags = os.O_CREAT | os.O_RDWR | getattr(os, "O_NOFOLLOW", 0)
descriptor = os.open(self.path, flags, 0o600)
metadata = os.fstat(descriptor)
if not stat.S_ISREG(metadata.st_mode):
os.close(descriptor)
raise ValidationError("lock path must be a regular file")
if os.name == "posix" and metadata.st_mode & (stat.S_IWGRP | stat.S_IWOTH):
os.close(descriptor)
raise ValidationError("lock file must not be group/world writable")
if (
os.name == "posix"
and hasattr(os, "geteuid")
and os.geteuid() == 0
and metadata.st_uid != 0
):
os.close(descriptor)
raise ValidationError("lock file must be owned by root")
try:
if fcntl is not None:
fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
elif msvcrt is not None: # pragma: no cover - exercised on Windows CI only.
os.lseek(descriptor, 0, os.SEEK_SET)
if os.fstat(descriptor).st_size == 0:
os.write(descriptor, b"0")
os.lseek(descriptor, 0, os.SEEK_SET)
msvcrt.locking(descriptor, msvcrt.LK_NBLCK, 1)
else: # pragma: no cover
raise ValidationError("platform has no supported file locking primitive")
except (BlockingIOError, OSError) as exc:
os.close(descriptor)
raise ConflictError("another lifecycle operation holds the global lock") from exc
self._descriptor = descriptor
return self
def __exit__(
self,
exc_type: type[BaseException] | None,
exc_value: BaseException | None,
traceback: TracebackType | None,
) -> None:
if self._descriptor is None:
return
try:
if fcntl is not None:
fcntl.flock(self._descriptor, fcntl.LOCK_UN)
elif msvcrt is not None: # pragma: no cover
os.lseek(self._descriptor, 0, os.SEEK_SET)
msvcrt.locking(self._descriptor, msvcrt.LK_UNLCK, 1)
finally:
os.close(self._descriptor)
self._descriptor = None
host_agent/src/netbox_store_agent/managed_files.py
+224
View File
@@ -0,0 +1,224 @@
from __future__ import annotations
import json
import os
import stat
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Iterable
from urllib.parse import urlsplit
from packaging.utils import canonicalize_name
from .config import Config
from .errors import ManualRecoveryRequired, PolicyError, ValidationError
from .journal import ManagedPlugin
from .util import DIST_NAME_RE, IMPORT_NAME_RE, SHA256_RE, is_relative_to, require_uuid
@dataclass(frozen=True)
class PreviousFile:
path: Path
content: bytes | None
mode: int
@dataclass(frozen=True)
class FileSnapshot:
files: tuple[PreviousFile, ...]
class ManagedFiles:
MAX_MANAGED_FILE_BYTES = 2 * 1024 * 1024
def __init__(self, config: Config):
self.config = config
def _validate_target(self, path: Path) -> None:
root = self.config.paths.allowed_root
if path.is_symlink():
raise PolicyError(f"managed path may not be a symlink: {path}")
resolved = path.resolve(strict=False)
if not is_relative_to(resolved, root):
raise PolicyError(f"managed path escaped the configured root: {path}")
if path.exists() and not path.is_file():
raise PolicyError(f"managed path must be a regular file: {path}")
@staticmethod
def _validate_backup_root(path: Path) -> None:
current = path
while not current.exists() and current != current.parent:
current = current.parent
if current.is_symlink():
raise PolicyError("backup directory ancestry may not be a symlink")
def _read_previous(self, path: Path) -> PreviousFile:
self._validate_target(path)
if not path.exists():
return PreviousFile(path, None, 0o640)
metadata = path.stat()
if metadata.st_size > self.MAX_MANAGED_FILE_BYTES:
raise PolicyError("managed file exceeds the safe backup limit")
return PreviousFile(path, path.read_bytes(), stat.S_IMODE(metadata.st_mode))
def _atomic_write(self, path: Path, content: bytes, mode: int = 0o640) -> None:
self._validate_target(path)
path.parent.mkdir(parents=True, exist_ok=True, mode=0o750)
# Re-check after creating the parent to narrow symlink races.
self._validate_target(path)
descriptor, temporary_name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
temporary = Path(temporary_name)
try:
os.fchmod(descriptor, mode)
with os.fdopen(descriptor, "wb", closefd=True) as handle:
handle.write(content)
handle.flush()
os.fsync(handle.fileno())
descriptor = -1
os.replace(temporary, path)
if os.name == "posix":
directory_descriptor = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(directory_descriptor)
finally:
os.close(directory_descriptor)
finally:
if descriptor >= 0:
os.close(descriptor)
if temporary.exists():
temporary.unlink()
def backup(self, operation_id: str) -> FileSnapshot:
require_uuid(operation_id, "operation_id")
root = self.config.agent.backup_dir
self._validate_backup_root(root)
root.mkdir(parents=True, exist_ok=True, mode=0o700)
operation_dir = root / operation_id
operation_dir.mkdir(mode=0o700)
previous = tuple(
self._read_previous(path)
for path in (self.config.paths.include_path, self.config.paths.requirements_path)
)
for item in previous:
if item.content is None:
(operation_dir / f"{item.path.name}.absent").touch(mode=0o600, exist_ok=False)
else:
backup_path = operation_dir / item.path.name
descriptor = os.open(
backup_path,
os.O_CREAT | os.O_EXCL | os.O_WRONLY | getattr(os, "O_NOFOLLOW", 0),
0o600,
)
try:
os.write(descriptor, item.content)
os.fsync(descriptor)
finally:
os.close(descriptor)
return FileSnapshot(previous)
def restore(self, snapshot: FileSnapshot) -> None:
try:
for previous in snapshot.files:
if previous.content is None:
self._validate_target(previous.path)
if previous.path.exists():
previous.path.unlink()
else:
self._atomic_write(previous.path, previous.content, previous.mode)
except OSError as exc:
raise ManualRecoveryRequired("managed-file rollback failed") from exc
@staticmethod
def render_include(plugins: Iterable[ManagedPlugin]) -> bytes:
imports = sorted({plugin.import_name for plugin in plugins if plugin.enabled})
if any(not IMPORT_NAME_RE.fullmatch(name) for name in imports):
raise ValidationError("managed plugin state contains an invalid import name")
values = json.dumps(imports, ensure_ascii=True, indent=2)
text = (
"# Generated by netbox-store-agent. Do not edit.\n"
f"STORE_PLUGINS = {values}\n"
)
return text.encode("utf-8")
def render_requirements(self, plugins: Iterable[ManagedPlugin]) -> bytes:
requirements: dict[str, dict[str, object]] = {}
schemes = {"https", "http"} if self.config.store.allow_http_for_testing else {"https"}
for plugin in plugins:
for raw in plugin.requirements:
if not isinstance(raw, dict):
raise ValidationError("managed requirement must be an object")
required = {
"package_name",
"version",
"download_url",
"filename",
"sha256",
"size",
}
if set(raw) != required:
raise ValidationError("managed requirement has an invalid schema")
package = raw["package_name"]
url = raw["download_url"]
digest = raw["sha256"]
if not isinstance(package, str) or not DIST_NAME_RE.fullmatch(package):
raise ValidationError("managed requirement package is invalid")
if not isinstance(url, str) or any(char in url for char in "\r\n\t "):
raise ValidationError("managed requirement URL is invalid")
parsed = urlsplit(url)
if (
parsed.scheme not in schemes
or not parsed.hostname
or parsed.hostname.lower() not in self.config.store.allowed_hosts
or parsed.username
or parsed.password
or parsed.fragment
):
raise ValidationError("managed requirement URL violates Store policy")
if not isinstance(digest, str) or not SHA256_RE.fullmatch(digest):
raise ValidationError("managed requirement digest is invalid")
key = canonicalize_name(package)
existing = requirements.get(key)
if existing is not None and existing != raw:
raise PolicyError(f"conflicting locked requirements for {package}")
requirements[key] = raw
lines = ["# Generated by netbox-store-agent. Do not edit."]
for key in sorted(requirements):
item = requirements[key]
lines.append(
f"{item['package_name']} @ {item['download_url']} "
f"--hash=sha256:{item['sha256']}"
)
return ("\n".join(lines) + "\n").encode("utf-8")
def write(self, operation_id: str, plugins: Iterable[ManagedPlugin]) -> FileSnapshot:
plugin_list = tuple(plugins)
include = self.render_include(plugin_list)
requirements = self.render_requirements(plugin_list)
snapshot = self.backup(operation_id)
try:
self._atomic_write(self.config.paths.include_path, include)
self._atomic_write(self.config.paths.requirements_path, requirements)
except Exception:
self.restore(snapshot)
raise
return snapshot
def make_local_requirements(self, directory: Path, plugin: ManagedPlugin, wheel: Path) -> Path:
requirement = plugin.requirements[0]
path = directory / "install-requirements.txt"
content = (
f"{plugin.package_name} @ {wheel.as_uri()} "
f"--hash=sha256:{requirement['sha256']}\n"
).encode("utf-8")
descriptor = os.open(
path,
os.O_CREAT | os.O_EXCL | os.O_WRONLY | getattr(os, "O_NOFOLLOW", 0),
0o600,
)
try:
os.write(descriptor, content)
os.fsync(descriptor)
finally:
os.close(descriptor)
return path
host_agent/src/netbox_store_agent/protocol.py
+171
View File
@@ -0,0 +1,171 @@
from __future__ import annotations
import json
import re
from dataclasses import dataclass
from typing import Any
from .constants import ACTIONS, DEFAULT_MAX_REQUEST_BYTES, PROTOCOL_VERSION
from .errors import AgentError, ValidationError
from .util import SHA256_RE, SLUG_RE, reject_unknown, require_mapping, require_uuid
STATUS_PATH_RE = re.compile(r"^/v1/operations/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$")
@dataclass(frozen=True)
class OperationRequest:
request_id: str
action: str
plugin_slug: str
version: str | None
approved_payload_sha256: str | None
requested_by: str
def as_dict(self) -> dict[str, Any]:
return {
"request_id": self.request_id,
"action": self.action,
"plugin_slug": self.plugin_slug,
"version": self.version,
"approved_payload_sha256": self.approved_payload_sha256,
"requested_by": self.requested_by,
}
@dataclass(frozen=True)
class Request:
protocol_version: int
method: str
path: str
idempotency_key: str | None = None
operation: OperationRequest | None = None
def _parse_operation_body(value: Any) -> OperationRequest:
body = require_mapping(value, "body")
allowed = {
"request_id",
"action",
"plugin_slug",
"version",
"approved_payload_sha256",
"requested_by",
}
reject_unknown(body, allowed, "body")
missing = sorted(allowed - set(body))
if missing:
raise ValidationError(f"body is missing fields: {', '.join(missing)}")
request_id = require_uuid(body["request_id"], "body.request_id")
action = body["action"]
if not isinstance(action, str) or action not in ACTIONS:
raise ValidationError(f"body.action must be one of: {', '.join(sorted(ACTIONS))}")
slug = body["plugin_slug"]
if not isinstance(slug, str) or not SLUG_RE.fullmatch(slug):
raise ValidationError("body.plugin_slug is invalid")
version = body["version"]
if version is not None and (
not isinstance(version, str)
or not 1 <= len(version) <= 100
or any(char.isspace() for char in version)
or "\x00" in version
):
raise ValidationError("body.version is invalid")
digest = body["approved_payload_sha256"]
# Semantically opaque: the agent never recomputes this Store approval
# marker. Protocol v1 nevertheless fixes its wire encoding to lowercase
# SHA-256 so malformed values are rejected before persistence.
if digest is not None and (not isinstance(digest, str) or not SHA256_RE.fullmatch(digest)):
raise ValidationError("body.approved_payload_sha256 must be lowercase SHA-256 or null")
requested_by = body["requested_by"]
if not isinstance(requested_by, str) or not 1 <= len(requested_by) <= 200:
raise ValidationError("body.requested_by must be a non-empty string up to 200 characters")
if not requested_by.isprintable():
raise ValidationError("body.requested_by contains control characters")
if action in {"install", "update"}:
if version is None or digest is None:
raise ValidationError(f"{action} requires version and approved_payload_sha256")
elif version is not None or digest is not None:
raise ValidationError(f"{action} requires version and approved_payload_sha256 to be null")
return OperationRequest(
request_id=request_id,
action=action,
plugin_slug=slug,
version=version,
approved_payload_sha256=digest,
requested_by=requested_by,
)
def parse_request(value: Any) -> Request:
data = require_mapping(value, "request")
base_allowed = {"protocol_version", "method", "path"}
if data.get("method") == "POST" and data.get("path") == "/v1/operations":
allowed = base_allowed | {"idempotency_key", "body"}
else:
allowed = base_allowed
reject_unknown(data, allowed, "request")
missing = sorted(base_allowed - set(data))
if missing:
raise ValidationError(f"request is missing fields: {', '.join(missing)}")
if data["protocol_version"] != PROTOCOL_VERSION:
raise ValidationError(f"unsupported protocol_version; expected {PROTOCOL_VERSION}")
method = data["method"]
path = data["path"]
if method not in {"GET", "POST"} or not isinstance(path, str):
raise ValidationError("invalid method or path")
if method == "GET" and path == "/v1/capabilities":
return Request(PROTOCOL_VERSION, method, path)
if method == "POST" and path == "/v1/operations":
if "idempotency_key" not in data or "body" not in data:
raise ValidationError("submit requires idempotency_key and body")
key = require_uuid(data["idempotency_key"], "idempotency_key")
return Request(
PROTOCOL_VERSION,
method,
path,
idempotency_key=key,
operation=_parse_operation_body(data["body"]),
)
match = STATUS_PATH_RE.fullmatch(path) if method == "GET" else None
if match:
operation_id = require_uuid(match.group(1), "operation id")
return Request(PROTOCOL_VERSION, method, path, idempotency_key=operation_id)
raise ValidationError("unknown method/path")
def decode_request(raw: bytes, max_bytes: int) -> Request:
if not raw or len(raw) > max_bytes:
raise ValidationError(f"request must contain 1 to {max_bytes} bytes")
try:
text = raw.decode("utf-8")
except UnicodeDecodeError as exc:
raise ValidationError("request must be UTF-8") from exc
if "\n" in text or "\r" in text:
raise ValidationError("request frame must contain exactly one JSON line")
try:
value = json.loads(text)
except json.JSONDecodeError as exc:
raise ValidationError("request contains invalid JSON") from exc
return parse_request(value)
def response(status: int, body: dict[str, Any]) -> dict[str, Any]:
return {"protocol_version": PROTOCOL_VERSION, "status": status, "body": body}
def error_response(error: AgentError) -> dict[str, Any]:
return response(error.status, {"error": {"code": error.code, "message": error.message}})
def encode_response(value: dict[str, Any]) -> bytes:
encoded = (json.dumps(value, separators=(",", ":"), ensure_ascii=False) + "\n").encode("utf-8")
if len(encoded) <= DEFAULT_MAX_REQUEST_BYTES:
return encoded
fallback = error_response(
AgentError("response exceeded protocol limit", code="response_too_large", status=500)
)
return (json.dumps(fallback, separators=(",", ":")) + "\n").encode("utf-8")
host_agent/src/netbox_store_agent/runner.py
+70
View File
@@ -0,0 +1,70 @@
from __future__ import annotations
import os
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import Protocol, Sequence
from .config import Config
from .errors import ExecutionError, ValidationError
@dataclass(frozen=True)
class CommandResult:
argv: tuple[str, ...]
stdout: str
stderr: str
class Runner(Protocol):
def run(self, argv: Sequence[str]) -> CommandResult: ...
class SubprocessRunner:
MAX_CAPTURE_CHARS = 16_000
def __init__(self, config: Config):
self.config = config
def run(self, argv: Sequence[str]) -> CommandResult:
command = tuple(argv)
if not command or not all(isinstance(item, str) and item for item in command):
raise ValidationError("command argv must contain non-empty strings")
if any("\x00" in item or "\r" in item or "\n" in item for item in command):
raise ValidationError("command argv contains a forbidden control character")
if not Path(command[0]).is_absolute():
raise ValidationError("command executable must be an absolute path")
environment = os.environ.copy()
for key in tuple(environment):
if key.startswith("PIP_") or key in {"PYTHONPATH", "PYTHONHOME"}:
environment.pop(key, None)
environment.update(
{
"PIP_DISABLE_PIP_VERSION_CHECK": "1",
"PIP_NO_INPUT": "1",
"PYTHONNOUSERSITE": "1",
}
)
try:
completed = subprocess.run(
command,
shell=False,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
errors="replace",
timeout=self.config.commands.command_timeout_seconds,
check=False,
env=environment,
)
except (OSError, subprocess.TimeoutExpired) as exc:
raise ExecutionError(f"command could not complete: {command[0]}") from exc
stdout = completed.stdout[-self.MAX_CAPTURE_CHARS :]
stderr = completed.stderr[-self.MAX_CAPTURE_CHARS :]
if completed.returncode != 0:
raise ExecutionError(
f"command failed with exit code {completed.returncode}: {command[0]}"
)
return CommandResult(command, stdout, stderr)
host_agent/src/netbox_store_agent/service.py
+56
View File
@@ -0,0 +1,56 @@
from __future__ import annotations
from concurrent.futures import ThreadPoolExecutor
from typing import Any
from .config import Config
from .constants import ACTIONS, AGENT_VERSION, PROTOCOL_VERSION
from .executor import OperationProcessor
from .journal import Journal
from .protocol import Request, response
class AgentService:
def __init__(
self,
config: Config,
journal: Journal,
processor: OperationProcessor | None = None,
):
self.config = config
self.journal = journal
self.processor = processor or OperationProcessor(config, journal)
# Lifecycle mutations are deliberately serialized in-process; the
# root-owned file lock also protects against a second process.
self.pool = ThreadPoolExecutor(max_workers=1, thread_name_prefix="plugin-operation")
self.journal.recover_interrupted()
for operation_id in self.journal.queued_operations():
self.pool.submit(self.processor.process, operation_id)
def capabilities(self) -> dict[str, Any]:
return {
"agent_version": AGENT_VERSION,
"protocol_version": PROTOCOL_VERSION,
"supported_actions": sorted(ACTIONS),
"dry_run": self.config.agent.dry_run,
"netbox_version": self.config.policy.netbox_version,
"min_supported_netbox": self.config.policy.min_supported_netbox,
"max_supported_netbox": self.config.policy.max_supported_netbox,
"max_request_bytes": self.config.agent.max_request_bytes,
"execution": "serialized-host-lifecycle",
}
def handle(self, request: Request) -> dict[str, Any]:
if request.method == "GET" and request.path == "/v1/capabilities":
return response(200, self.capabilities())
if request.method == "POST" and request.path == "/v1/operations":
assert request.idempotency_key is not None and request.operation is not None
operation, created = self.journal.submit(request.idempotency_key, request.operation)
if created:
self.pool.submit(self.processor.process, request.idempotency_key)
return response(202, {"created": created, "operation": operation})
assert request.idempotency_key is not None
return response(200, {"operation": self.journal.get_operation(request.idempotency_key)})
def close(self) -> None:
self.pool.shutdown(wait=True, cancel_futures=False)
host_agent/src/netbox_store_agent/util.py
+74
View File
@@ -0,0 +1,74 @@
from __future__ import annotations
import hashlib
import json
import re
import uuid
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
from .errors import ValidationError
SLUG_RE = re.compile(r"^[a-z0-9](?:[a-z0-9._-]{0,198}[a-z0-9])?$")
SHA256_RE = re.compile(r"^[a-f0-9]{64}$")
IMPORT_NAME_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*$")
DIST_NAME_RE = re.compile(r"^[A-Za-z0-9](?:[A-Za-z0-9._-]{0,198}[A-Za-z0-9])?$")
SERVICE_NAME_RE = re.compile(r"^[A-Za-z0-9_.@-]{1,128}$")
def utc_now() -> str:
return datetime.now(UTC).isoformat().replace("+00:00", "Z")
def canonical_json(value: Any) -> str:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
def payload_sha256(value: Any) -> str:
return hashlib.sha256(canonical_json(value).encode("utf-8")).hexdigest()
def require_uuid(value: Any, field: str) -> str:
if not isinstance(value, str):
raise ValidationError(f"{field} must be a UUID string")
try:
parsed = uuid.UUID(value)
except (ValueError, AttributeError) as exc:
raise ValidationError(f"{field} must be a valid UUID") from exc
if str(parsed) != value:
raise ValidationError(f"{field} must use canonical UUID notation")
return str(parsed)
def reject_unknown(data: dict[str, Any], allowed: set[str], context: str) -> None:
unknown = sorted(set(data) - allowed)
if unknown:
raise ValidationError(f"{context} contains unknown fields: {', '.join(unknown)}")
def require_mapping(value: Any, context: str) -> dict[str, Any]:
if not isinstance(value, dict):
raise ValidationError(f"{context} must be a JSON object")
if not all(isinstance(k, str) for k in value):
raise ValidationError(f"{context} keys must be strings")
return value
def require_absolute_path(value: Any, field: str) -> Path:
if not isinstance(value, str) or not value:
raise ValidationError(f"{field} must be a non-empty absolute path")
path = Path(value)
if not path.is_absolute():
raise ValidationError(f"{field} must be absolute")
if "\x00" in value:
raise ValidationError(f"{field} contains a NUL byte")
return path
def is_relative_to(path: Path, parent: Path) -> bool:
try:
path.relative_to(parent)
except ValueError:
return False
return True
host_agent/systemd/netbox-store-agent.service
+29
View File
@@ -0,0 +1,29 @@
[Unit]
Description=Privileged NetBox Store host agent
Requires=netbox-store-agent.socket
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=root
Group=root
ExecStart=/usr/local/bin/netbox-store-agent --config /etc/netbox-store-agent/agent.toml daemon
StateDirectory=netbox-store-agent
StateDirectoryMode=0700
RuntimeDirectory=netbox-store-agent
RuntimeDirectoryMode=0750
UMask=0077
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ReadWritePaths=/opt/netbox /var/lib/netbox-store-agent /run/netbox-store-agent /run/lock
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
LockPersonality=true
SystemCallArchitectures=native
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target
host_agent/systemd/netbox-store-agent.socket
+14
View File
@@ -0,0 +1,14 @@
[Unit]
Description=NetBox Store host-agent socket
[Socket]
ListenStream=/run/netbox-store-agent/agent.sock
SocketMode=0660
SocketUser=root
# Replace with the group of the NetBox service.
SocketGroup=netbox
DirectoryMode=0750
RemoveOnStop=true
[Install]
WantedBy=sockets.target
host_agent/tests/support.py
+215
View File
@@ -0,0 +1,215 @@
from __future__ import annotations
import hashlib
import json
import sys
import zipfile
from pathlib import Path
from typing import Any
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
from netbox_store_agent.catalog import HttpStatusError, PluginMetadata, ReleasePlan
from netbox_store_agent.config import (
AgentSettings,
CommandSettings,
Config,
PathSettings,
PolicySettings,
StoreSettings,
)
from netbox_store_agent.errors import ExecutionError
from netbox_store_agent.runner import CommandResult
def make_config(root: Path, *, dry_run: bool = True, require_peers: bool = False) -> Config:
root = root.resolve()
managed_root = root / "netbox"
managed_root.mkdir(parents=True, exist_ok=True)
state = root / "state"
return Config(
agent=AgentSettings(
socket_path=root / "agent.sock",
journal_path=state / "journal.sqlite3",
lock_path=state / "lifecycle.lock",
backup_dir=state / "backups",
dry_run=dry_run,
require_root=False,
require_peer_credentials=require_peers,
allowed_peer_uids=(0,),
allowed_peer_gids=(0,),
socket_mode=0o660,
socket_uid=0,
socket_gid=0,
max_request_bytes=65536,
connection_timeout_seconds=1,
worker_threads=1,
),
store=StoreSettings(
base_url="http://store.test",
plugin_endpoint_template="/api/v1/plugins/{plugin_slug}",
release_endpoint_template="/api/v1/plugins/{plugin_slug}/releases/{version}",
timeout_seconds=1,
max_catalog_bytes=1024 * 1024,
max_artifact_bytes=1024 * 1024,
allow_private_addresses=True,
allow_http_for_testing=True,
allowed_hosts=("store.test", "artifacts.test"),
bearer_token_file=None,
ca_file=None,
),
paths=PathSettings(
allowed_root=managed_root,
include_path=managed_root / "store_plugins.py",
requirements_path=managed_root / "store_requirements.txt",
temp_dir=state / "tmp",
),
commands=CommandSettings(
python_path=root / "bin" / "python",
manage_path=managed_root / "manage.py",
systemctl_path=root / "bin" / "systemctl",
services=("netbox", "netbox-rq"),
command_timeout_seconds=10,
),
policy=PolicySettings(
netbox_version="4.6.8",
min_supported_netbox="4.6.5",
max_supported_netbox="4.6.8",
self_plugin_slugs=("netbox-store", "netbox-plugin-store", "netbox_plugin_store"),
allow_prereleases=False,
require_release_for_enable=True,
),
)
def release_json(version: str = "1.2.3", **overrides: Any) -> dict[str, Any]:
result: dict[str, Any] = {
"version": version,
"download_url": f"http://artifacts.test/demo_plugin-{version}-py3-none-any.whl",
"sha256": "a" * 64,
"artifact_size": 100,
"commit_sha": "",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
"published_at": None,
"approved": True,
"status": "approved",
"immutable": True,
"approved_payload_sha256": "c" * 64,
}
result.update(overrides)
return result
def plugin_json(releases: list[dict[str, Any]] | None = None, **overrides: Any) -> dict[str, Any]:
result: dict[str, Any] = {
"api_version": "v1",
"slug": "demo-plugin",
"name": "Demo",
"summary": "Summary",
"description": "Description",
"repository_url": "https://git.test/demo",
"latest_version": "1.2.3",
"package_name": "demo-plugin",
"import_name": "demo_plugin",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
"approved": True,
"status": "approved",
"releases": releases or [],
}
result.update(overrides)
return result
class FakeTransport:
def __init__(self, responses: dict[str, Any], artifact: bytes | None = None):
self.responses = responses
self.artifact = artifact or b""
self.json_urls: list[str] = []
self.download_urls: list[str] = []
def get_json(self, url: str, max_bytes: int) -> Any:
self.json_urls.append(url)
if url not in self.responses:
raise HttpStatusError(404, "missing")
value = self.responses[url]
if isinstance(value, Exception):
raise value
assert len(json.dumps(value)) <= max_bytes
return value
def download(
self,
url: str,
destination: Path,
*,
max_bytes: int,
expected_size: int,
expected_sha256: str,
) -> None:
self.download_urls.append(url)
if len(self.artifact) != expected_size:
raise AssertionError("fixture size mismatch")
if hashlib.sha256(self.artifact).hexdigest() != expected_sha256:
raise AssertionError("fixture digest mismatch")
destination.write_bytes(self.artifact)
def wheel_bytes(path: Path, version: str = "1.2.3") -> bytes:
wheel = path / f"demo_plugin-{version}-py3-none-any.whl"
with zipfile.ZipFile(wheel, "w") as archive:
archive.writestr("demo_plugin/__init__.py", "")
archive.writestr(
f"demo_plugin-{version}.dist-info/WHEEL",
"Wheel-Version: 1.0\nGenerator: tests\nRoot-Is-Purelib: true\nTag: py3-none-any\n",
)
return wheel.read_bytes()
def plan(version: str = "1.2.3") -> ReleasePlan:
plugin = PluginMetadata(
"demo-plugin", "demo-plugin", "demo_plugin", "4.6.5", "4.6.8", ()
)
return ReleasePlan(
plugin,
version,
f"http://artifacts.test/demo_plugin-{version}-py3-none-any.whl",
f"demo_plugin-{version}-py3-none-any.whl",
"b" * 64,
1,
"c" * 64,
)
class FakeStore:
def __init__(self, release: ReleasePlan | None = None):
self.release = release or plan()
self.calls: list[tuple[str, ...]] = []
def get_plugin(self, slug: str) -> PluginMetadata:
self.calls.append(("plugin", slug))
return self.release.plugin
def get_release(self, slug: str, version: str) -> ReleasePlan:
self.calls.append(("release", slug, version))
return self.release
def download_release(self, release: ReleasePlan, directory: Path) -> Path:
self.calls.append(("download", release.version))
target = directory / release.filename
target.write_bytes(b"x")
return target
class FakeRunner:
def __init__(self, fail_step: str | None = None):
self.commands: list[tuple[str, ...]] = []
self.fail_step = fail_step
def run(self, argv: list[str]) -> CommandResult:
command = tuple(argv)
self.commands.append(command)
if self.fail_step and self.fail_step in command:
raise ExecutionError("injected command failure")
return CommandResult(command, "", "")
host_agent/tests/test_catalog.py
+124
View File
@@ -0,0 +1,124 @@
from __future__ import annotations
import hashlib
import tempfile
import unittest
from pathlib import Path
from support import FakeTransport, make_config, plugin_json, release_json, wheel_bytes
from netbox_store_agent.catalog import CatalogError, SecureHTTPTransport, StoreClient
from netbox_store_agent.config import StoreSettings
class CatalogTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.root = Path(self.temporary.name)
self.config = make_config(self.root)
def tearDown(self) -> None:
self.temporary.cleanup()
def test_trailing_slash_fallback_and_valid_wheel(self) -> None:
artifact = wheel_bytes(self.root)
release = release_json(
sha256=hashlib.sha256(artifact).hexdigest(), artifact_size=len(artifact)
)
responses = {
"http://store.test/api/v1/plugins/demo-plugin/": plugin_json(),
"http://store.test/api/v1/plugins/demo-plugin/releases/1.2.3/": release,
}
transport = FakeTransport(responses, artifact)
client = StoreClient(self.config, transport)
plan = client.get_release("demo-plugin", "1.2.3")
# Destination directory is caller-owned; use an existing operation directory.
operation_dir = self.root / "operation"
operation_dir.mkdir()
downloaded = client.download_release(plan, operation_dir)
self.assertTrue(downloaded.is_file())
self.assertEqual(transport.json_urls[0][-1], "n")
self.assertEqual(transport.json_urls[1][-1], "/")
def test_release_falls_back_to_plugin_releases_array(self) -> None:
release = release_json()
transport = FakeTransport(
{"http://store.test/api/v1/plugins/demo-plugin": plugin_json([release])}
)
plan = StoreClient(self.config, transport).get_release("demo-plugin", "1.2.3")
self.assertEqual(plan.version, "1.2.3")
def test_unapproved_or_mutable_release_rejected(self) -> None:
for change in ({"approved": False}, {"immutable": False}, {"status": "pending"}):
with self.subTest(change=change):
transport = FakeTransport(
{
"http://store.test/api/v1/plugins/demo-plugin": plugin_json(),
"http://store.test/api/v1/plugins/demo-plugin/releases/1.2.3": release_json(
**change
),
}
)
with self.assertRaises(CatalogError):
StoreClient(self.config, transport).get_release("demo-plugin", "1.2.3")
def test_unknown_catalog_field_fails_closed(self) -> None:
payload = plugin_json()
payload["internal_id"] = 7
transport = FakeTransport({"http://store.test/api/v1/plugins/demo-plugin": payload})
with self.assertRaises(CatalogError):
StoreClient(self.config, transport).get_plugin("demo-plugin")
def test_netbox_incompatibility_rejected(self) -> None:
transport = FakeTransport(
{
"http://store.test/api/v1/plugins/demo-plugin": plugin_json(
min_netbox_version="4.7.0"
)
}
)
with self.assertRaises(CatalogError):
StoreClient(self.config, transport).get_plugin("demo-plugin")
def test_wheel_distribution_must_match(self) -> None:
artifact = wheel_bytes(self.root)
release = release_json(
sha256=hashlib.sha256(artifact).hexdigest(), artifact_size=len(artifact)
)
payload = plugin_json(package_name="another-package")
transport = FakeTransport(
{
"http://store.test/api/v1/plugins/demo-plugin": payload,
"http://store.test/api/v1/plugins/demo-plugin/releases/1.2.3": release,
},
artifact,
)
client = StoreClient(self.config, transport)
plan = client.get_release("demo-plugin", "1.2.3")
directory = self.root / "mismatch"
directory.mkdir()
with self.assertRaises(CatalogError):
client.download_release(plan, directory)
def test_bearer_token_is_not_sent_to_artifact_origin(self) -> None:
settings = StoreSettings(
**{
**self.config.store.__dict__,
"base_url": "https://store.test:8443",
"allow_http_for_testing": False,
}
)
transport = SecureHTTPTransport(settings)
transport._token = lambda: "secret" # type: ignore[method-assign]
self.assertEqual(
transport._headers("https://store.test:8443/api")["Authorization"],
"Bearer secret",
)
self.assertNotIn(
"Authorization", transport._headers("https://artifacts.test/release.whl")
)
self.assertNotIn("Authorization", transport._headers("https://store.test/api"))
if __name__ == "__main__":
unittest.main()
host_agent/tests/test_config.py
+132
View File
@@ -0,0 +1,132 @@
from __future__ import annotations
import json
import tempfile
import unittest
from pathlib import Path
from support import * # noqa: F403
from netbox_store_agent.config import load_config
from netbox_store_agent.errors import ValidationError
class ConfigTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.root = Path(self.temporary.name).resolve()
def tearDown(self) -> None:
self.temporary.cleanup()
@staticmethod
def q(path: Path) -> str:
return json.dumps(path.as_posix())
def config_text(self, **changes: str) -> str:
root = self.root
values = {
"agent_extra": "",
"store_extra": "",
"paths_extra": "",
"commands_extra": "",
"policy_extra": "",
"base_url": '"https://store.test"',
"include": self.q(root / "netbox" / "store_plugins.py"),
}
values.update(changes)
return f"""
[agent]
socket_path = {self.q(root / 'agent.sock')}
journal_path = {self.q(root / 'state' / 'journal.sqlite3')}
lock_path = {self.q(root / 'state' / 'lock')}
backup_dir = {self.q(root / 'state' / 'backups')}
require_root = false
require_peer_credentials = false
{values['agent_extra']}
[store]
base_url = {values['base_url']}
allowed_hosts = ["store.test"]
{values['store_extra']}
[paths]
allowed_root = {self.q(root / 'netbox')}
include_path = {values['include']}
requirements_path = {self.q(root / 'netbox' / 'requirements.txt')}
temp_dir = {self.q(root / 'state' / 'tmp')}
{values['paths_extra']}
[commands]
python_path = {self.q(root / 'bin' / 'python')}
manage_path = {self.q(root / 'netbox' / 'manage.py')}
systemctl_path = {self.q(root / 'bin' / 'systemctl')}
{values['commands_extra']}
[policy]
{values['policy_extra']}
"""
def write(self, text: str) -> Path:
path = self.root / "agent.toml"
path.write_text(text, encoding="utf-8")
path.chmod(0o600)
return path
def test_defaults_are_dry_run_and_block_all_self_slugs(self) -> None:
config = load_config(self.write(self.config_text()), allow_insecure_owner=True)
self.assertTrue(config.agent.dry_run)
self.assertIn("netbox-plugin-store", config.policy.self_plugin_slugs)
self.assertIn("netbox_plugin_store", config.policy.self_plugin_slugs)
def test_unknown_setting_rejected(self) -> None:
with self.assertRaises(ValidationError):
load_config(
self.write(self.config_text(agent_extra="surprise = true")),
allow_insecure_owner=True,
)
def test_http_requires_explicit_testing_switch(self) -> None:
with self.assertRaises(ValidationError):
load_config(
self.write(self.config_text(base_url='"http://store.test"')),
allow_insecure_owner=True,
)
config = load_config(
self.write(
self.config_text(
base_url='"http://store.test"', store_extra="allow_http_for_testing = true"
)
),
allow_insecure_owner=True,
)
self.assertTrue(config.store.allow_http_for_testing)
def test_managed_path_escape_rejected(self) -> None:
with self.assertRaises(ValidationError):
load_config(
self.write(self.config_text(include=self.q(self.root / "outside.py"))),
allow_insecure_owner=True,
)
def test_protocol_size_cannot_exceed_64k(self) -> None:
with self.assertRaises(ValidationError):
load_config(
self.write(self.config_text(agent_extra="max_request_bytes = 65537")),
allow_insecure_owner=True,
)
def test_endpoint_placeholders_are_exact(self) -> None:
with self.assertRaises(ValidationError):
load_config(
self.write(
self.config_text(
store_extra='release_endpoint_template = "/api/{plugin_slug}/{other}"'
)
),
allow_insecure_owner=True,
)
if __name__ == "__main__":
unittest.main()
host_agent/tests/test_daemon.py
+80
View File
@@ -0,0 +1,80 @@
from __future__ import annotations
import json
import socket
import tempfile
import unittest
from pathlib import Path
from support import make_config
from netbox_store_agent.daemon import handle_connection
from netbox_store_agent.protocol import response
class FakeService:
def __init__(self) -> None:
self.requests = []
def handle(self, request: object) -> dict[str, object]:
self.requests.append(request)
return response(200, {"ok": True})
class DaemonFramingTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.config = make_config(Path(self.temporary.name), require_peers=False)
def tearDown(self) -> None:
self.temporary.cleanup()
def exchange(self, payload: bytes) -> dict[str, object]:
server, client = socket.socketpair()
try:
client.sendall(payload)
service = FakeService()
handle_connection(server, self.config, service) # type: ignore[arg-type]
data = bytearray()
while b"\n" not in data:
chunk = client.recv(4096)
if not chunk:
break
data.extend(chunk)
self.assertEqual(data.count(b"\n"), 1)
return json.loads(bytes(data).decode())
finally:
client.close()
def test_one_request_one_response(self) -> None:
result = self.exchange(
b'{"protocol_version":1,"method":"GET","path":"/v1/capabilities"}\n'
)
self.assertEqual(result["status"], 200)
self.assertEqual(set(result), {"protocol_version", "status", "body"})
def test_second_json_line_rejected(self) -> None:
result = self.exchange(
b'{"protocol_version":1,"method":"GET","path":"/v1/capabilities"}\n{}\n'
)
self.assertEqual(result["status"], 400)
def test_missing_newline_rejected_when_client_half_closes(self) -> None:
server, client = socket.socketpair()
try:
client.sendall(b"{}")
client.shutdown(socket.SHUT_WR)
service = FakeService()
handle_connection(server, self.config, service) # type: ignore[arg-type]
result = json.loads(client.recv(4096).decode())
self.assertEqual(result["status"], 400)
finally:
client.close()
def test_oversized_frame_rejected(self) -> None:
result = self.exchange(b"x" * 65536 + b"\n")
self.assertEqual(result["status"], 400)
if __name__ == "__main__":
unittest.main()
host_agent/tests/test_executor.py
+177
View File
@@ -0,0 +1,177 @@
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from support import FakeRunner, FakeStore, make_config, plan
from netbox_store_agent.executor import OperationProcessor
from netbox_store_agent.journal import Journal, ManagedPlugin
from netbox_store_agent.protocol import OperationRequest
class ExecutorTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.root = Path(self.temporary.name)
self.key = "20f4274f-d4e5-42bf-9164-967b1a774481"
def tearDown(self) -> None:
self.temporary.cleanup()
def request(self, action: str, *, version: str | None = None, token: str | None = None) -> OperationRequest:
return OperationRequest(
"eea17d87-8944-4ee2-a076-363338ab746d",
action,
"demo-plugin",
version,
token,
"alice",
)
def seed(self, journal: Journal, *, enabled: bool) -> ManagedPlugin:
release = plan()
plugin = ManagedPlugin(
"demo-plugin",
"demo-plugin",
"demo_plugin",
"1.2.3",
enabled,
(release.requirement(),),
)
journal.upsert_managed_plugin(plugin)
return plugin
def test_dry_run_verifies_artifact_without_runner_or_state(self) -> None:
config = make_config(self.root, dry_run=True)
journal = Journal(config.agent.journal_path)
store = FakeStore()
runner = FakeRunner()
journal.submit(self.key, self.request("install", version="1.2.3", token="c" * 64))
OperationProcessor(config, journal, store=store, runner=runner).process(self.key)
operation = journal.get_operation(self.key)
self.assertEqual(operation["state"], "dry_run")
self.assertIn(("download", "1.2.3"), store.calls)
self.assertEqual(runner.commands, [])
self.assertIsNone(journal.get_managed_plugin("demo-plugin"))
self.assertFalse(config.paths.include_path.exists())
def test_install_is_disabled_and_uses_fixed_pip_argv(self) -> None:
config = make_config(self.root, dry_run=False)
journal = Journal(config.agent.journal_path)
runner = FakeRunner()
journal.submit(self.key, self.request("install", version="1.2.3", token="c" * 64))
OperationProcessor(config, journal, store=FakeStore(), runner=runner).process(self.key)
operation = journal.get_operation(self.key)
self.assertEqual(operation["state"], "succeeded")
managed = journal.get_managed_plugin("demo-plugin")
self.assertIsNotNone(managed)
self.assertFalse(managed.enabled)
self.assertEqual(len(runner.commands), 1)
command = runner.commands[0]
self.assertIn("--no-index", command)
self.assertIn("--no-deps", command)
self.assertIn("--require-hashes", command)
def test_approval_token_mismatch_fails_before_download(self) -> None:
config = make_config(self.root, dry_run=False)
journal = Journal(config.agent.journal_path)
store = FakeStore()
runner = FakeRunner()
journal.submit(self.key, self.request("install", version="1.2.3", token="d" * 64))
OperationProcessor(config, journal, store=store, runner=runner).process(self.key)
self.assertEqual(journal.get_operation(self.key)["state"], "failed")
self.assertFalse(any(call[0] == "download" for call in store.calls))
self.assertEqual(runner.commands, [])
def test_enable_runs_migrate_collectstatic_and_both_service_restart(self) -> None:
config = make_config(self.root, dry_run=False)
journal = Journal(config.agent.journal_path)
self.seed(journal, enabled=False)
runner = FakeRunner()
journal.submit(self.key, self.request("enable"))
OperationProcessor(config, journal, store=FakeStore(), runner=runner).process(self.key)
self.assertEqual(journal.get_operation(self.key)["state"], "succeeded")
self.assertTrue(journal.get_managed_plugin("demo-plugin").enabled)
flattened = [item for command in runner.commands for item in command]
self.assertIn("migrate", flattened)
self.assertIn("collectstatic", flattened)
self.assertIn("netbox", flattened)
self.assertIn("netbox-rq", flattened)
def test_uninstall_requires_disabled(self) -> None:
config = make_config(self.root, dry_run=False)
journal = Journal(config.agent.journal_path)
self.seed(journal, enabled=True)
runner = FakeRunner()
journal.submit(self.key, self.request("uninstall"))
OperationProcessor(config, journal, store=FakeStore(), runner=runner).process(self.key)
self.assertEqual(journal.get_operation(self.key)["state"], "failed")
self.assertEqual(runner.commands, [])
def test_disable_restarts_services_and_persists_disabled_state(self) -> None:
config = make_config(self.root, dry_run=False)
journal = Journal(config.agent.journal_path)
self.seed(journal, enabled=True)
runner = FakeRunner()
journal.submit(self.key, self.request("disable"))
OperationProcessor(config, journal, store=FakeStore(), runner=runner).process(self.key)
self.assertEqual(journal.get_operation(self.key)["state"], "succeeded")
self.assertFalse(journal.get_managed_plugin("demo-plugin").enabled)
self.assertEqual(len(runner.commands), 1)
self.assertIn("restart", runner.commands[0])
def test_uninstall_disabled_plugin_uses_fixed_package_and_deletes_state(self) -> None:
config = make_config(self.root, dry_run=False)
journal = Journal(config.agent.journal_path)
self.seed(journal, enabled=False)
runner = FakeRunner()
journal.submit(self.key, self.request("uninstall"))
OperationProcessor(config, journal, store=FakeStore(), runner=runner).process(self.key)
self.assertEqual(journal.get_operation(self.key)["state"], "succeeded")
self.assertIsNone(journal.get_managed_plugin("demo-plugin"))
self.assertEqual(len(runner.commands), 1)
self.assertEqual(runner.commands[0][-1], "demo-plugin")
def test_failed_pip_attempt_is_conservatively_manual_recovery(self) -> None:
config = make_config(self.root, dry_run=False)
journal = Journal(config.agent.journal_path)
runner = FakeRunner(fail_step="install")
journal.submit(self.key, self.request("install", version="1.2.3", token="c" * 64))
OperationProcessor(config, journal, store=FakeStore(), runner=runner).process(self.key)
self.assertEqual(journal.get_operation(self.key)["state"], "manual_recovery")
def test_failure_after_pip_requires_manual_recovery_and_restores_files(self) -> None:
config = make_config(self.root, dry_run=False)
config.paths.include_path.write_text("old include", encoding="utf-8")
config.paths.requirements_path.write_text("old requirements", encoding="utf-8")
journal = Journal(config.agent.journal_path)
self.seed(journal, enabled=True)
runner = FakeRunner(fail_step="restart")
journal.submit(self.key, self.request("update", version="1.2.3", token="c" * 64))
OperationProcessor(config, journal, store=FakeStore(), runner=runner).process(self.key)
self.assertEqual(journal.get_operation(self.key)["state"], "manual_recovery")
self.assertEqual(config.paths.include_path.read_text(), "old include")
self.assertEqual(config.paths.requirements_path.read_text(), "old requirements")
def test_self_management_fails_before_store_access(self) -> None:
config = make_config(self.root, dry_run=True)
journal = Journal(config.agent.journal_path)
store = FakeStore()
request = OperationRequest(
"eea17d87-8944-4ee2-a076-363338ab746d",
"install",
"netbox-plugin-store",
"1.2.3",
"c" * 64,
"alice",
)
journal.submit(self.key, request)
OperationProcessor(config, journal, store=store, runner=FakeRunner()).process(self.key)
self.assertEqual(journal.get_operation(self.key)["state"], "failed")
self.assertEqual(store.calls, [])
if __name__ == "__main__":
unittest.main()
host_agent/tests/test_journal.py
+72
View File
@@ -0,0 +1,72 @@
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from support import * # noqa: F403
from netbox_store_agent.errors import ConflictError
from netbox_store_agent.journal import Journal, ManagedPlugin
from netbox_store_agent.protocol import OperationRequest
class JournalTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.journal = Journal(Path(self.temporary.name) / "journal.sqlite3")
self.key = "20f4274f-d4e5-42bf-9164-967b1a774481"
self.request = OperationRequest(
"eea17d87-8944-4ee2-a076-363338ab746d",
"install",
"demo-plugin",
"1.2.3",
"opaque",
"alice",
)
def tearDown(self) -> None:
self.temporary.cleanup()
def test_idempotency_same_payload_returns_existing(self) -> None:
first, created = self.journal.submit(self.key, self.request)
second, created_again = self.journal.submit(self.key, self.request)
self.assertTrue(created)
self.assertFalse(created_again)
self.assertEqual(first["operation_id"], second["operation_id"])
def test_idempotency_conflict(self) -> None:
self.journal.submit(self.key, self.request)
changed = OperationRequest(**{**self.request.as_dict(), "requested_by": "mallory"})
with self.assertRaises(ConflictError):
self.journal.submit(self.key, changed)
def test_claim_is_exactly_once(self) -> None:
self.journal.submit(self.key, self.request)
self.assertTrue(self.journal.claim(self.key))
self.assertFalse(self.journal.claim(self.key))
def test_running_recovery_is_manual(self) -> None:
self.journal.submit(self.key, self.request)
self.journal.claim(self.key)
self.assertEqual(self.journal.recover_interrupted(), 1)
operation = self.journal.get_operation(self.key)
self.assertEqual(operation["state"], "manual_recovery")
def test_managed_plugin_round_trip(self) -> None:
plugin = ManagedPlugin(
"demo-plugin",
"demo-plugin",
"demo_plugin",
"1.2.3",
False,
({"sha256": "a" * 64},),
)
self.journal.upsert_managed_plugin(plugin)
self.assertEqual(self.journal.get_managed_plugin("demo-plugin"), plugin)
self.journal.delete_managed_plugin("demo-plugin")
self.assertIsNone(self.journal.get_managed_plugin("demo-plugin"))
if __name__ == "__main__":
unittest.main()
host_agent/tests/test_managed_files.py
+87
View File
@@ -0,0 +1,87 @@
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from support import make_config, plan
from netbox_store_agent.errors import PolicyError
from netbox_store_agent.journal import ManagedPlugin
from netbox_store_agent.managed_files import ManagedFiles
class ManagedFilesTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary = tempfile.TemporaryDirectory()
self.root = Path(self.temporary.name)
self.config = make_config(self.root, dry_run=False)
self.files = ManagedFiles(self.config)
release = plan()
self.plugin = ManagedPlugin(
"demo-plugin",
"demo-plugin",
"demo_plugin",
"1.2.3",
True,
(release.requirement(),),
)
def tearDown(self) -> None:
self.temporary.cleanup()
def test_include_exports_constant_without_referencing_plugins(self) -> None:
text = self.files.render_include([self.plugin]).decode()
self.assertIn('STORE_PLUGINS = [\n "demo_plugin"\n]', text)
self.assertNotIn("PLUGINS = list", text)
def test_atomic_write_backup_and_restore(self) -> None:
self.config.paths.include_path.write_text("old include", encoding="utf-8")
self.config.paths.requirements_path.write_text("old requirements", encoding="utf-8")
snapshot = self.files.write(
"20f4274f-d4e5-42bf-9164-967b1a774481", [self.plugin]
)
self.assertIn("STORE_PLUGINS", self.config.paths.include_path.read_text())
self.assertIn("--hash=sha256:", self.config.paths.requirements_path.read_text())
self.files.restore(snapshot)
self.assertEqual(self.config.paths.include_path.read_text(), "old include")
self.assertEqual(self.config.paths.requirements_path.read_text(), "old requirements")
def test_conflicting_locked_distribution_rejected(self) -> None:
other_requirement = {**self.plugin.requirements[0], "version": "2.0.0"}
other = ManagedPlugin(
"other", "demo-plugin", "other_plugin", "2.0.0", False, (other_requirement,)
)
with self.assertRaises(PolicyError):
self.files.render_requirements([self.plugin, other])
def test_requirement_host_must_be_allowlisted(self) -> None:
requirement = {**self.plugin.requirements[0], "download_url": "http://evil.test/x.whl"}
plugin = ManagedPlugin(
"demo-plugin", "demo-plugin", "demo_plugin", "1.2.3", False, (requirement,)
)
with self.assertRaises(Exception):
self.files.render_requirements([plugin])
def test_path_escape_rejected(self) -> None:
escaped = self.root / "outside.py"
changed = self.config.__class__(
self.config.agent,
self.config.store,
self.config.paths.__class__(
self.config.paths.allowed_root,
escaped,
self.config.paths.requirements_path,
self.config.paths.temp_dir,
),
self.config.commands,
self.config.policy,
)
with self.assertRaises(PolicyError):
ManagedFiles(changed).write(
"20f4274f-d4e5-42bf-9164-967b1a774481", [self.plugin]
)
if __name__ == "__main__":
unittest.main()
host_agent/tests/test_protocol.py
+83
View File
@@ -0,0 +1,83 @@
from __future__ import annotations
import json
import unittest
from support import * # noqa: F403
from netbox_store_agent.errors import ValidationError
from netbox_store_agent.protocol import decode_request, encode_response, parse_request, response
class ProtocolTests(unittest.TestCase):
def submit(self, **body_overrides: object) -> dict[str, object]:
body: dict[str, object] = {
"request_id": "eea17d87-8944-4ee2-a076-363338ab746d",
"action": "install",
"plugin_slug": "demo-plugin",
"version": "1.2.3",
"approved_payload_sha256": "c" * 64,
"requested_by": "netbox:alice",
}
body.update(body_overrides)
return {
"protocol_version": 1,
"method": "POST",
"path": "/v1/operations",
"idempotency_key": "20f4274f-d4e5-42bf-9164-967b1a774481",
"body": body,
}
def test_exact_submit_contract_and_opaque_token(self) -> None:
request = parse_request(self.submit())
self.assertEqual(request.operation.approved_payload_sha256, "c" * 64)
def test_unknown_top_level_field_rejected(self) -> None:
value = self.submit(extra=True)
value["unexpected"] = True
with self.assertRaises(ValidationError):
parse_request(value)
def test_unknown_body_field_rejected(self) -> None:
with self.assertRaises(ValidationError):
parse_request(self.submit(extra=True))
def test_non_lifecycle_fields_must_be_null(self) -> None:
with self.assertRaises(ValidationError):
parse_request(self.submit(action="disable"))
request = parse_request(
self.submit(action="disable", version=None, approved_payload_sha256=None)
)
self.assertEqual(request.operation.action, "disable")
def test_noncanonical_uuid_rejected(self) -> None:
with self.assertRaises(ValidationError):
parse_request(self.submit(request_id="EEA17D87-8944-4EE2-A076-363338AB746D"))
def test_multiple_lines_and_oversize_rejected(self) -> None:
raw = json.dumps(self.submit()).encode()
with self.assertRaises(ValidationError):
decode_request(raw + b"\n{}", 65536)
with self.assertRaises(ValidationError):
decode_request(b"x" * 10, 9)
def test_capabilities_and_status_paths(self) -> None:
cap = parse_request({"protocol_version": 1, "method": "GET", "path": "/v1/capabilities"})
self.assertEqual(cap.path, "/v1/capabilities")
status = parse_request(
{
"protocol_version": 1,
"method": "GET",
"path": "/v1/operations/20f4274f-d4e5-42bf-9164-967b1a774481",
}
)
self.assertEqual(status.idempotency_key, "20f4274f-d4e5-42bf-9164-967b1a774481")
def test_response_shape_and_single_line(self) -> None:
encoded = encode_response(response(200, {"ok": True}))
self.assertEqual(encoded.count(b"\n"), 1)
self.assertEqual(set(json.loads(encoded)), {"protocol_version", "status", "body"})
if __name__ == "__main__":
unittest.main()
netbox_plugin/.gitignore
+6
View File
@@ -0,0 +1,6 @@
__pycache__/
*.py[cod]
build/
dist/
*.egg-info/
.pytest_cache/
netbox_plugin/LICENSE
+13
View File
@@ -0,0 +1,13 @@
Copyright 2026 MrBlake
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
netbox_plugin/MANIFEST.in
+3
View File
@@ -0,0 +1,3 @@
recursive-include netbox_plugin_store/templates *.html
recursive-include netbox_plugin_store/migrations *.py
include README.md LICENSE
netbox_plugin/README.md
+109
View File
@@ -0,0 +1,109 @@
# NetBox Plugin Store
`netbox-plugin-store` is the NetBox-side client for an approved internal plugin catalog. It supports NetBox **4.6.5 through 4.6.8** and provides catalog, status, confirmation, and redacted audit pages.
The safe default is intentionally non-mutating. Installing the UI alone never grants the NetBox web process permission to modify its Python environment.
## Security model
- Catalog, status, audit, and dry-run access require an authenticated user with `netbox_plugin_store.manage_plugin` (superusers implicitly have it).
- Every state-changing POST is checked again in the view and requires a **superuser**. A menu permission alone is never sufficient.
- Lifecycle endpoints accept POST only, use Django CSRF protection, and require the user to type the exact plugin slug on a separate confirmation page.
- Package, import, slug, version, URL, and SHA-256 fields are validated. The Store plugin cannot update, disable, or uninstall itself.
- Installation/update requires an explicitly approved plugin and an explicitly approved, immutable release with a 64-character **artifact** SHA-256. A Git commit SHA is not an artifact hash and is rejected.
- Downloads go to a private temporary directory and are SHA-256 verified before pip sees the file. Redirects and pagination remain inside configured URL allowlists.
- Pip and maintenance commands use argv arrays with `shell=False`; direct mode uses `--no-index` by default and runs `python -m pip check` after install/update.
- `configuration.py` and `local_requirements.txt` are changed via same-directory temporary files plus `os.replace()`. Existing files receive timestamped `0600` backups. A cross-process file lock serializes operations.
- Audit input/output is bounded and redacted for common token, password, authorization, and credential patterns.
- Install only installs and pins a package; it remains disabled. Enable is a separate action which edits `PLUGINS`, runs migrations/collectstatic, and requires restart. Uninstall is accepted only after an explicit disable.
## Installation
Install the wheel into NetBox's virtual environment and persist it in `/opt/netbox/local_requirements.txt`:
```text
netbox-plugin-store==0.1.0
```
Add the plugin to `configuration.py`:
```python
PLUGINS = [
"netbox_plugin_store",
]
PLUGINS_CONFIG = {
"netbox_plugin_store": {
# Required: this is the separate Store service, not the Forgejo host.
"store_url": "https://store.example.internal",
"allowed_store_urls": ["https://store.example.internal"],
# Include every origin from which approved immutable artifacts are served.
"allowed_artifact_urls": ["https://store.example.internal", "https://git.mrblake.cc"],
"api_token": "", # Prefer a read-only catalog token if authentication is required.
"execution_mode": "dry_run",
}
}
```
Then use NetBox's supported upgrade flow (normally `/opt/netbox/upgrade.sh`) or run migrate/collectstatic and restart the web and RQ services. The service user needs read access to the Store and configuration. Dry-run mode needs no venv/config write permission.
The Store API contract is:
- `GET /api/v1/plugins/` (plain list or `{ "results": [...] }` pagination)
- `GET /api/v1/plugins/<slug>/`
- `GET /api/v1/plugins/<slug>/releases/<version>/` (used by the host agent; returns one release object)
- plugin fields: `slug`, `name`, `summary`, `description`, `repository_url`, `latest_version`, `package_name`, `import_name`, `min_netbox_version`, `max_netbox_version`, `approved`/`status`, and `releases`
- release fields: `version`, `download_url`, `sha256`, `artifact_size`, `commit_sha`, `min_netbox_version`, `max_netbox_version`, `published_at`, `approved`, `status`, `immutable`, and opaque `approved_payload_sha256`
## Execution modes
### `dry_run` (default)
Only validates and plans. `default_dry_run` also defaults to `True`. Dry-runs may use NetBox's default RQ queue; real operations never run inside `netbox-rq`, because restarting the same worker would leave its job state inconsistent.
### `agent` (recommended for production)
Use a separately privileged host agent. NetBox connects to an absolute Unix socket and sends one JSON line per connection (maximum 64 KiB):
```python
"execution_mode": "agent",
"agent_socket_path": "/run/netbox-store-agent/agent.sock",
"agent_timeout": 30,
```
Protocol version 1 uses `GET /v1/capabilities`, `POST /v1/operations`, and `GET /v1/operations/<uuid>`. The POST includes a UUID idempotency key and only the action, slug, version, actor, request ID, and the Store's opaque `approved_payload_sha256`. The agent re-fetches the approved catalog record itself.
The mutating HTTP request stops immediately after the agent accepts the operation. It does not poll while the agent might restart NetBox. Opening the audit detail page performs one status query and reconciles a completed/failed operation.
The Unix socket should be owned by the agent group, writable only by the NetBox service account/group, and placed in a non-world-writable directory.
### `direct` (development/controlled installations only)
Direct mode additionally requires `allow_lifecycle_mutations=True`. The NetBox service account must be able to write the venv, `configuration.py`, `local_requirements.txt`, backup directory, and lock file. This is often inappropriate for production.
```python
"execution_mode": "direct",
"allow_lifecycle_mutations": True,
"configuration_path": "/opt/netbox/netbox/netbox/configuration.py",
"requirements_path": "/opt/netbox/local_requirements.txt",
"manage_path": "/opt/netbox/netbox/manage.py",
"allow_package_index": False,
```
With `allow_package_index=False`, pip receives `--no-index`; therefore all transitive dependencies must already be installed or available in the artifact. Enabling `allow_package_index` is an explicit supply-chain policy decision.
Direct mode never performs automatic restart. Enable, disable, and updating an enabled plugin are marked `restart-required`; restart NetBox web services and workers out of band. `auto_restart=True` is rejected in direct mode.
For agent-managed restarts, `restart_commands` must match `restart_allowlist` token-for-token. Do not use a shell command string. The host agent must enforce its own independent command policy.
## Persistence and rollback
Install/update writes an immutable PEP 508 direct URL plus `#sha256=...` into `local_requirements.txt`; uninstall removes it. This keeps plugins present across NetBox's supported upgrade flow. Config/requirements writes are atomic and backed up under `plugin-store-backups` by default. The direct executor performs compensating rollback where safe; an interrupted pip upgrade or already-applied database migration can still require operator recovery, which is recorded as failed/restart-required.
## Test
The core tests use fake downloads, subprocesses, repositories, and sockets; they never invoke real pip, NetBox restarts, or maintenance commands:
```bash
python -m unittest discover -s tests -v
```
netbox_plugin/netbox_plugin_store/__init__.py
+55
View File
@@ -0,0 +1,55 @@
from netbox.plugins import PluginConfig
from .version import __version__
class NetBoxPluginStoreConfig(PluginConfig):
name = "netbox_plugin_store"
verbose_name = "NetBox Plugin Store"
description = "Install and manage approved plugins from the MrBlake store."
version = __version__
author = "MrBlake"
base_url = "plugin-store"
min_version = "4.6.5"
max_version = "4.6.8"
required_settings = ["store_url", "allowed_store_urls", "allowed_artifact_urls"]
default_settings = {
"api_token": "",
"request_timeout": 15,
"download_timeout": 120,
"max_download_bytes": 268_435_456,
"configuration_path": None,
"requirements_path": None,
"manage_requirements_file": True,
"allow_lifecycle_mutations": False,
"default_dry_run": True,
"execution_mode": "dry_run",
"agent_socket_path": None,
"agent_timeout": 30,
"agent_poll_interval": 1.0,
"manage_path": None,
"lock_path": None,
"backup_dir": None,
"lock_timeout": 30,
"operation_timeout": 900,
"pip_extra_args": [],
"allow_package_index": False,
"run_migrations": True,
"collect_static": True,
"background_jobs": True,
"synchronous_fallback": True,
"job_queue": "default",
"auto_restart": False,
"restart_commands": [],
"restart_allowlist": [],
"backups_to_keep": 25,
}
def ready(self):
super().ready()
# Import so NetBox can deserialize the JobRunner by dotted path.
from .jobs import PluginLifecycleJob # noqa: F401
config = NetBoxPluginStoreConfig
netbox_plugin/netbox_plugin_store/access.py
+18
View File
@@ -0,0 +1,18 @@
from __future__ import annotations
from typing import Protocol
class PermissionUser(Protocol):
is_authenticated: bool
is_superuser: bool
def has_perm(self, permission_name: str) -> bool: ...
def has_store_access(user: PermissionUser, permission_name: str) -> bool:
"""Authorize NetBox users without relying on the removed ``is_staff`` field."""
return bool(
user.is_authenticated
and (user.is_superuser or user.has_perm(permission_name))
)
netbox_plugin/netbox_plugin_store/agent.py
+129
View File
@@ -0,0 +1,129 @@
from __future__ import annotations
import json
import socket
import time
from pathlib import Path
from typing import Any
from uuid import UUID, uuid4
from .redaction import redact_text
class AgentError(RuntimeError):
pass
class AgentClient:
PROTOCOL_VERSION = 1
MAX_MESSAGE_BYTES = 64 * 1024
def __init__(self, socket_path: Path, *, timeout: int = 30):
self.socket_path = Path(socket_path)
self.timeout = timeout
def request(self, method: str, path: str, **values: Any) -> dict[str, Any]:
request = {
"protocol_version": self.PROTOCOL_VERSION,
"method": method,
"path": path,
**values,
}
encoded = json.dumps(request, separators=(",", ":"), ensure_ascii=True).encode("utf-8") + b"\n"
if len(encoded) > self.MAX_MESSAGE_BYTES:
raise AgentError("Agent request exceeds 64 KiB.")
if not hasattr(socket, "AF_UNIX"):
raise AgentError("Unix sockets are unavailable on this platform.")
try:
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client:
client.settimeout(self.timeout)
client.connect(str(self.socket_path))
client.sendall(encoded)
client.shutdown(socket.SHUT_WR)
response = bytearray()
while b"\n" not in response:
chunk = client.recv(4096)
if not chunk:
break
response.extend(chunk)
if len(response) > self.MAX_MESSAGE_BYTES:
raise AgentError("Agent response exceeds 64 KiB.")
except (OSError, TimeoutError) as exc:
raise AgentError(f"Agent connection failed: {type(exc).__name__}") from exc
line, separator, remainder = bytes(response).partition(b"\n")
if not separator or remainder:
raise AgentError("Agent must return exactly one JSON line.")
try:
payload = json.loads(line.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise AgentError("Agent returned invalid JSON.") from exc
if not isinstance(payload, dict) or payload.get("protocol_version") != self.PROTOCOL_VERSION:
raise AgentError("Agent returned an incompatible protocol response.")
status = payload.get("status")
body = payload.get("body")
if not isinstance(status, int) or not isinstance(body, dict):
raise AgentError("Agent returned a malformed response.")
if status < 200 or status >= 300:
message = body.get("error") or body.get("detail") or f"Agent returned status {status}."
raise AgentError(redact_text(message, limit=2_000))
return body
def capabilities(self) -> dict[str, Any]:
return self.request("GET", "/v1/capabilities")
def submit_operation(
self,
*,
request_id: str,
action: str,
plugin_slug: str,
version: str | None,
approved_payload_sha256: str | None,
requested_by: str,
) -> UUID:
idempotency_key = uuid4()
body = self.request(
"POST",
"/v1/operations",
idempotency_key=str(idempotency_key),
body={
"request_id": request_id,
"action": action,
"plugin_slug": plugin_slug,
"version": version,
"approved_payload_sha256": approved_payload_sha256,
"requested_by": requested_by,
},
)
raw_id = body.get("operation_id") or body.get("id")
try:
return UUID(str(raw_id))
except (ValueError, TypeError) as exc:
raise AgentError("Agent did not return a valid operation UUID.") from exc
def wait_for_operation(
self,
operation_id: UUID,
*,
overall_timeout: int,
poll_interval: float,
) -> dict[str, Any]:
deadline = time.monotonic() + overall_timeout
while True:
body = self.get_operation(operation_id)
state = body.get("state") or body.get("status")
if state in {"succeeded", "completed"}:
result = body.get("result", body)
if not isinstance(result, dict):
raise AgentError("Agent operation result is malformed.")
return result
if state in {"failed", "errored", "cancelled"}:
raise AgentError(redact_text(body.get("error") or f"Agent operation {state}."))
if state not in {"queued", "pending", "running", "accepted"}:
raise AgentError("Agent returned an unknown operation state.")
if time.monotonic() >= deadline:
raise AgentError("Timed out waiting for the host agent operation.")
time.sleep(poll_interval)
def get_operation(self, operation_id: UUID) -> dict[str, Any]:
return self.request("GET", f"/v1/operations/{operation_id}")
netbox_plugin/netbox_plugin_store/client.py
+294
View File
@@ -0,0 +1,294 @@
from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.parse import quote, urljoin, urlsplit, urlunsplit
from urllib.request import HTTPRedirectHandler, Request, build_opener
from packaging.version import InvalidVersion, Version
from .validation import (
ValidationError,
validate_distribution_name,
validate_import_name,
validate_public_url,
validate_sha256,
validate_slug,
)
class StoreClientError(RuntimeError):
pass
class URLPolicy:
"""Exact scheme/host/port allowlist with an optional path prefix."""
def __init__(self, allowed_roots: tuple[str, ...] | list[str]):
if not allowed_roots:
raise ValueError("At least one allowed URL is required.")
if any(urlsplit(root).query or urlsplit(root).fragment for root in allowed_roots):
raise ValueError("Allowlist roots may not contain a query string or fragment.")
self._roots = tuple(self._normalize(validate_public_url(root)) for root in allowed_roots)
@staticmethod
def _normalize(value: str) -> tuple[str, str, int, str]:
parsed = urlsplit(value)
scheme = parsed.scheme.lower()
default_port = 443 if scheme == "https" else 80
port = parsed.port or default_port
prefix = parsed.path.rstrip("/") or "/"
return scheme, parsed.hostname.lower(), port, prefix
def check(self, value: str) -> str:
validate_public_url(value)
parsed = urlsplit(value)
candidate = self._normalize(value)
scheme, host, port, path = candidate
for allowed_scheme, allowed_host, allowed_port, prefix in self._roots:
path_ok = prefix == "/" or path == prefix or path.startswith(prefix + "/")
if (scheme, host, port) == (allowed_scheme, allowed_host, allowed_port) and path_ok:
return value
raise StoreClientError("URL is outside the configured allowlist.")
class _PolicyRedirectHandler(HTTPRedirectHandler):
def __init__(self, policy: URLPolicy):
self.policy = policy
super().__init__()
def redirect_request(self, req, fp, code, msg, headers, newurl):
self.policy.check(newurl)
return super().redirect_request(req, fp, code, msg, headers, newurl)
@dataclass(frozen=True, slots=True)
class Release:
version: str
download_url: str
sha256: str
min_netbox_version: str
max_netbox_version: str
published_at: str
approved: bool
immutable: bool
approved_payload_sha256: str
@classmethod
def from_mapping(cls, value: dict[str, Any]) -> "Release":
version = str(value.get("version", "")).strip()
try:
Version(version)
except InvalidVersion as exc:
raise StoreClientError("Store returned an invalid release version.") from exc
digest = str(value.get("sha256") or "").strip()
if digest:
digest = validate_sha256(digest)
approved_payload_sha256 = str(value.get("approved_payload_sha256") or "").strip()
if approved_payload_sha256:
approved_payload_sha256 = validate_sha256(approved_payload_sha256)
download_url = str(value.get("download_url") or value.get("artifact_url") or "").strip()
if download_url:
validate_public_url(download_url)
return cls(
version=version,
download_url=download_url,
sha256=digest,
min_netbox_version=str(value.get("min_netbox_version") or "").strip(),
max_netbox_version=str(value.get("max_netbox_version") or "").strip(),
published_at=str(value.get("published_at") or "").strip(),
approved=value.get("approved") is True or value.get("status") == "approved",
immutable=value.get("immutable") is True,
approved_payload_sha256=approved_payload_sha256,
)
def supports(self, netbox_version: str, plugin: "CatalogPlugin") -> bool:
try:
current = Version(netbox_version)
minimum = Version(self.min_netbox_version or plugin.min_netbox_version or "0")
maximum = Version(self.max_netbox_version or plugin.max_netbox_version or "999999")
except InvalidVersion:
return False
return minimum <= current <= maximum
@dataclass(frozen=True, slots=True)
class CatalogPlugin:
slug: str
name: str
summary: str
description: str
repository_url: str
latest_version: str
package_name: str
import_name: str
min_netbox_version: str
max_netbox_version: str
approved: bool
releases: tuple[Release, ...]
@classmethod
def from_mapping(cls, value: dict[str, Any]) -> "CatalogPlugin":
try:
slug = validate_slug(str(value.get("slug", "")))
package_name = validate_distribution_name(str(value.get("package_name", "")))
import_name = validate_import_name(str(value.get("import_name", "")))
except ValidationError as exc:
raise StoreClientError(str(exc)) from exc
releases_raw = value.get("releases") or []
if not isinstance(releases_raw, list):
raise StoreClientError("Store returned an invalid releases collection.")
releases = tuple(Release.from_mapping(item) for item in releases_raw if isinstance(item, dict))
repository_url = str(value.get("repository_url") or "").strip()
if repository_url:
validate_public_url(repository_url)
return cls(
slug=slug,
name=str(value.get("name") or slug)[:200],
summary=str(value.get("summary") or "")[:2_000],
description=str(value.get("description") or "")[:250_000],
repository_url=repository_url,
latest_version=str(value.get("latest_version") or "").strip(),
package_name=package_name,
import_name=import_name,
min_netbox_version=str(value.get("min_netbox_version") or "").strip(),
max_netbox_version=str(value.get("max_netbox_version") or "").strip(),
approved=value.get("approved") is True or value.get("status") == "approved",
releases=releases,
)
def select_release(self, netbox_version: str, requested_version: str = "") -> Release:
compatible = [release for release in self.releases if release.supports(netbox_version, self)]
if requested_version:
compatible = [release for release in compatible if release.version == requested_version]
elif self.latest_version:
latest = [release for release in compatible if release.version == self.latest_version]
if latest:
compatible = latest
if not compatible:
raise StoreClientError("No compatible release is available for this NetBox version.")
return max(compatible, key=lambda release: Version(release.version))
class StoreClient:
API_LIMIT = 5 * 1024 * 1024
MAX_PAGES = 50
def __init__(
self,
store_url: str,
allowed_store_urls: tuple[str, ...],
allowed_artifact_urls: tuple[str, ...],
*,
api_token: str = "",
timeout: int = 15,
):
self.store_policy = URLPolicy(allowed_store_urls)
self.artifact_policy = URLPolicy(allowed_artifact_urls)
if urlsplit(store_url).query or urlsplit(store_url).fragment:
raise StoreClientError("Store base URL may not contain a query string or fragment.")
self.store_url = self.store_policy.check(store_url.rstrip("/"))
# API authentication may only follow redirects below the configured base URL.
self.api_policy = URLPolicy([self.store_url])
self.api_token = api_token
self.timeout = timeout
def _request_json(self, url: str) -> Any:
self.api_policy.check(url)
headers = {"Accept": "application/json", "User-Agent": "netbox-plugin-store/0.1"}
if self.api_token:
headers["Authorization"] = f"Bearer {self.api_token}"
request = Request(url, headers=headers, method="GET")
opener = build_opener(_PolicyRedirectHandler(self.api_policy))
try:
with opener.open(request, timeout=self.timeout) as response:
length = response.headers.get("Content-Length")
if length and int(length) > self.API_LIMIT:
raise StoreClientError("Store response exceeds the size limit.")
body = response.read(self.API_LIMIT + 1)
except (HTTPError, URLError, TimeoutError, OSError) as exc:
raise StoreClientError(f"Store request failed: {type(exc).__name__}") from exc
if len(body) > self.API_LIMIT:
raise StoreClientError("Store response exceeds the size limit.")
try:
return json.loads(body.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise StoreClientError("Store returned invalid JSON.") from exc
def list_plugins(self) -> list[CatalogPlugin]:
next_url: str | None = self.store_url + "/api/v1/plugins/"
plugins: list[CatalogPlugin] = []
pages = 0
while next_url:
pages += 1
if pages > self.MAX_PAGES:
raise StoreClientError("Store pagination limit exceeded.")
payload = self._request_json(next_url)
if isinstance(payload, list):
items = payload
next_url = None
elif isinstance(payload, dict):
items = payload.get("results", payload.get("plugins", []))
raw_next = payload.get("next")
next_url = urljoin(next_url, str(raw_next)) if raw_next else None
if next_url:
self.api_policy.check(next_url)
else:
raise StoreClientError("Store returned an invalid catalog payload.")
if not isinstance(items, list):
raise StoreClientError("Store returned an invalid plugin collection.")
plugins.extend(CatalogPlugin.from_mapping(item) for item in items if isinstance(item, dict))
return plugins
def get_plugin(self, slug: str) -> CatalogPlugin:
slug = validate_slug(slug)
payload = self._request_json(self.store_url + f"/api/v1/plugins/{quote(slug)}/")
if not isinstance(payload, dict):
raise StoreClientError("Store returned an invalid plugin payload.")
return CatalogPlugin.from_mapping(payload)
def download_artifact(
self,
url: str,
destination: Path,
*,
expected_sha256: str,
timeout: int,
max_bytes: int,
) -> tuple[str, int]:
self.artifact_policy.check(url)
expected = validate_sha256(expected_sha256)
request = Request(
url,
headers={"Accept": "application/octet-stream", "User-Agent": "netbox-plugin-store/0.1"},
method="GET",
)
opener = build_opener(_PolicyRedirectHandler(self.artifact_policy))
digest = hashlib.sha256()
written = 0
try:
with opener.open(request, timeout=timeout) as response, destination.open("xb") as target:
length = response.headers.get("Content-Length")
if length and int(length) > max_bytes:
raise StoreClientError("Artifact exceeds the configured size limit.")
while chunk := response.read(1024 * 1024):
written += len(chunk)
if written > max_bytes:
raise StoreClientError("Artifact exceeds the configured size limit.")
digest.update(chunk)
target.write(chunk)
except StoreClientError:
destination.unlink(missing_ok=True)
raise
except (HTTPError, URLError, TimeoutError, OSError) as exc:
destination.unlink(missing_ok=True)
raise StoreClientError(f"Artifact download failed: {type(exc).__name__}") from exc
actual = digest.hexdigest()
if actual != expected:
destination.unlink(missing_ok=True)
raise StoreClientError("Artifact SHA-256 verification failed.")
return actual, written
netbox_plugin/netbox_plugin_store/commands.py
+69
View File
@@ -0,0 +1,69 @@
from __future__ import annotations
import os
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import Sequence
from .redaction import redact_text
class CommandExecutionError(RuntimeError):
def __init__(self, message: str, *, output: str = ""):
super().__init__(message)
self.output = redact_text(output)
@dataclass(frozen=True, slots=True)
class CommandResult:
returncode: int
stdout: str
stderr: str
@property
def output(self) -> str:
return redact_text("\n".join(part for part in (self.stdout, self.stderr) if part))
class SubprocessRunner:
"""Injectable no-shell subprocess boundary."""
def run(
self,
argv: Sequence[str],
*,
timeout: int,
cwd: Path | None = None,
input_text: str | None = None,
) -> CommandResult:
if not argv or any(not isinstance(arg, str) or "\x00" in arg for arg in argv):
raise CommandExecutionError("Refusing to execute invalid argv.")
env = os.environ.copy()
env.update({"PIP_NO_INPUT": "1", "PYTHONUNBUFFERED": "1"})
try:
completed = subprocess.run(
list(argv),
cwd=str(cwd) if cwd else None,
env=env,
shell=False,
check=False,
text=True,
input=input_text,
stdin=subprocess.DEVNULL if input_text is None else None,
capture_output=True,
timeout=timeout,
)
except subprocess.TimeoutExpired as exc:
output = "\n".join(
str(value) for value in (getattr(exc, "stdout", ""), getattr(exc, "stderr", "")) if value
)
raise CommandExecutionError("Command timed out.", output=output) from exc
except OSError as exc:
raise CommandExecutionError(f"Unable to start command: {type(exc).__name__}") from exc
result = CommandResult(completed.returncode, completed.stdout or "", completed.stderr or "")
if completed.returncode != 0:
raise CommandExecutionError(
f"Command failed with exit code {completed.returncode}.", output=result.output
)
return result
netbox_plugin/netbox_plugin_store/editors.py
+237
View File
@@ -0,0 +1,237 @@
from __future__ import annotations
import ast
import hashlib
import os
import stat
import tempfile
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from uuid import uuid4
from packaging.requirements import InvalidRequirement, Requirement
from packaging.utils import canonicalize_name
from .validation import validate_distribution_name, validate_import_name
class EditorError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class MutationReceipt:
path: Path
changed: bool
backup_path: Path | None
existed_before: bool
previous_text: str
before_sha256: str
after_sha256: str
class AtomicTextEditor:
def __init__(self, path: Path, backup_dir: Path, backups_to_keep: int):
raw_path = Path(path).expanduser()
if raw_path.exists():
if not raw_path.is_file():
raise EditorError(f"Managed path is not a regular file: {raw_path}")
self.path = raw_path.resolve(strict=True)
else:
self.path = raw_path.parent.resolve(strict=True) / raw_path.name
self.backup_dir = Path(backup_dir)
self.backups_to_keep = backups_to_keep
def read(self, *, required: bool = False) -> str:
if not self.path.exists():
if required:
raise EditorError(f"Managed file does not exist: {self.path}")
return ""
try:
return self.path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError) as exc:
raise EditorError(f"Unable to read managed file: {self.path.name}") from exc
def apply(self, new_text: str, *, required: bool = False) -> MutationReceipt:
old_text = self.read(required=required)
existed = self.path.exists()
before = hashlib.sha256(old_text.encode()).hexdigest()
after = hashlib.sha256(new_text.encode()).hexdigest()
if old_text == new_text:
return MutationReceipt(self.path, False, None, existed, old_text, before, after)
backup = self._backup(old_text) if existed else None
self._atomic_write(new_text)
self._prune_backups()
return MutationReceipt(self.path, True, backup, existed, old_text, before, after)
def rollback(self, receipt: MutationReceipt) -> None:
if not receipt.changed:
return
if receipt.existed_before:
self._atomic_write(receipt.previous_text)
else:
self.path.unlink(missing_ok=True)
def _backup(self, text: str) -> Path:
self.backup_dir.mkdir(mode=0o700, parents=True, exist_ok=True)
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%S.%fZ")
backup = self.backup_dir / f"{self.path.name}.{stamp}.{uuid4().hex}.bak"
try:
with backup.open("x", encoding="utf-8", newline="") as handle:
handle.write(text)
handle.flush()
os.fsync(handle.fileno())
os.chmod(backup, 0o600)
except OSError as exc:
backup.unlink(missing_ok=True)
raise EditorError(f"Unable to create backup for {self.path.name}") from exc
return backup
def _atomic_write(self, text: str) -> None:
existing_stat = self.path.stat() if self.path.exists() else None
mode = stat.S_IMODE(existing_stat.st_mode) if existing_stat else 0o640
fd, temp_name = tempfile.mkstemp(prefix=f".{self.path.name}.", dir=self.path.parent)
temp_path = Path(temp_name)
try:
with os.fdopen(fd, "w", encoding="utf-8", newline="") as handle:
handle.write(text)
handle.flush()
os.fsync(handle.fileno())
os.chmod(temp_path, mode)
if existing_stat is not None and hasattr(os, "chown"):
try:
os.chown(temp_path, existing_stat.st_uid, existing_stat.st_gid)
except PermissionError:
pass
os.replace(temp_path, self.path)
if os.name != "nt":
directory_fd = os.open(self.path.parent, os.O_RDONLY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
except OSError as exc:
temp_path.unlink(missing_ok=True)
raise EditorError(f"Unable to atomically update {self.path.name}") from exc
def _prune_backups(self) -> None:
pattern = f"{self.path.name}.*.bak"
backups = sorted(self.backup_dir.glob(pattern), key=lambda path: path.stat().st_mtime, reverse=True)
for old_backup in backups[self.backups_to_keep :]:
old_backup.unlink(missing_ok=True)
class PluginConfigurationEditor(AtomicTextEditor):
def enabled_plugins(self) -> list[str]:
text = self.read(required=True)
_, plugins = self._find_plugins_assignment(text)
return plugins
def preview(self, import_name: str, enabled: bool) -> tuple[str, list[str], bool]:
import_name = validate_import_name(import_name)
text = self.read(required=True)
node, plugins = self._find_plugins_assignment(text)
changed = False
if enabled and import_name not in plugins:
plugins.append(import_name)
changed = True
elif not enabled and import_name in plugins:
plugins = [plugin for plugin in plugins if plugin != import_name]
changed = True
if not changed:
return text, plugins, False
newline = "\r\n" if "\r\n" in text else "\n"
lines = text.splitlines(keepends=True)
if node.lineno == node.end_lineno and ";" in lines[node.lineno - 1]:
raise EditorError("PLUGINS assignment sharing a line cannot be safely edited.")
replacement = [f"PLUGINS = [{newline}"]
replacement.extend(f" {plugin!r},{newline}" for plugin in plugins)
replacement.append(f"]{newline}")
new_text = "".join(lines[: node.lineno - 1] + replacement + lines[node.end_lineno :])
return new_text, plugins, True
def set_enabled(self, import_name: str, enabled: bool) -> MutationReceipt:
new_text, _, _ = self.preview(import_name, enabled)
return self.apply(new_text, required=True)
@staticmethod
def _find_plugins_assignment(text: str) -> tuple[ast.Assign | ast.AnnAssign, list[str]]:
try:
tree = ast.parse(text)
except SyntaxError as exc:
raise EditorError("configuration.py is not valid Python; refusing to edit it.") from exc
matches: list[ast.Assign | ast.AnnAssign] = []
for node in tree.body:
if isinstance(node, ast.Assign) and any(
isinstance(target, ast.Name) and target.id == "PLUGINS" for target in node.targets
):
matches.append(node)
elif isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name) and node.target.id == "PLUGINS":
matches.append(node)
if len(matches) != 1:
raise EditorError("configuration.py must contain exactly one static PLUGINS assignment.")
node = matches[0]
try:
value = ast.literal_eval(node.value)
except (ValueError, TypeError, SyntaxError) as exc:
raise EditorError("PLUGINS must be a literal list or tuple of import names.") from exc
if not isinstance(value, (list, tuple)) or any(not isinstance(item, str) for item in value):
raise EditorError("PLUGINS must be a literal list or tuple of import names.")
plugins = [validate_import_name(item) for item in value]
if len(set(plugins)) != len(plugins):
raise EditorError("PLUGINS contains duplicate entries; refusing to edit it.")
return node, plugins
class RequirementsEditor(AtomicTextEditor):
def preview(self, package_name: str, requirement_line: str | None) -> tuple[str, bool]:
package_name = validate_distribution_name(package_name)
wanted = canonicalize_name(package_name)
text = self.read(required=False)
lines = text.splitlines(keepends=True)
indexes: list[int] = []
for index, line in enumerate(lines):
stripped = line.strip()
if not stripped or stripped.startswith("#") or stripped.startswith(("-r", "--", "-e")):
continue
candidate = stripped.split(" #", 1)[0].rstrip()
try:
requirement = Requirement(candidate)
except InvalidRequirement:
continue
if canonicalize_name(requirement.name) == wanted:
indexes.append(index)
if len(indexes) > 1:
raise EditorError("local_requirements.txt contains duplicate entries for this package.")
newline = "\r\n" if "\r\n" in text else "\n"
replacement = f"{requirement_line}{newline}" if requirement_line else None
if requirement_line:
try:
parsed = Requirement(requirement_line)
except InvalidRequirement as exc:
raise EditorError("Generated requirement is invalid.") from exc
if canonicalize_name(parsed.name) != wanted:
raise EditorError("Generated requirement targets the wrong distribution.")
if indexes:
index = indexes[0]
if replacement is None:
del lines[index]
elif lines[index].rstrip("\r\n") == requirement_line:
return text, False
else:
lines[index] = replacement
elif replacement is not None:
if text and not text.endswith(("\n", "\r")):
lines.append(newline)
lines.append(replacement)
else:
return text, False
return "".join(lines), True
def set_requirement(self, package_name: str, requirement_line: str | None) -> MutationReceipt:
new_text, _ = self.preview(package_name, requirement_line)
return self.apply(new_text, required=False)
netbox_plugin/netbox_plugin_store/forms.py
+69
View File
@@ -0,0 +1,69 @@
from __future__ import annotations
from django import forms
from .client import CatalogPlugin
from .runtime import RuntimeSettings
class LifecycleConfirmForm(forms.Form):
confirmation = forms.CharField(
max_length=64,
label="Plugin-Slug zur Bestätigung",
help_text="Diese Eingabe verhindert versehentliche Lifecycle-Aktionen.",
)
version = forms.ChoiceField(required=False, label="Version")
dry_run = forms.BooleanField(
required=False,
initial=True,
label="Nur prüfen (Dry-Run)",
help_text="Plant und validiert den Vorgang, ohne Dateien oder Prozesse zu verändern.",
)
def __init__(
self,
*args,
plugin: CatalogPlugin,
action: str,
runtime: RuntimeSettings,
**kwargs,
):
super().__init__(*args, **kwargs)
self.plugin = plugin
self.action = action
self.runtime = runtime
self.fields["confirmation"].widget.attrs.update({"autocomplete": "off", "placeholder": plugin.slug})
self.fields["dry_run"].initial = runtime.default_dry_run
if action in {"install", "update"}:
choices = [
(release.version, release.version)
for release in plugin.releases
if (
release.supports(runtime.netbox_version, plugin)
and release.approved
and release.immutable
and bool(release.sha256)
and (runtime.execution_mode != "agent" or bool(release.approved_payload_sha256))
)
]
choices.sort(reverse=True)
self.fields["version"].choices = choices
self.fields["version"].required = True
self.fields["version"].initial = plugin.latest_version
else:
self.fields.pop("version")
for field in self.fields.values():
if not isinstance(field.widget, forms.CheckboxInput):
field.widget.attrs.setdefault("class", "form-control")
def clean_confirmation(self) -> str:
value = self.cleaned_data["confirmation"]
if value != self.plugin.slug:
raise forms.ValidationError("Der Slug stimmt nicht exakt überein.")
return value
def clean(self):
cleaned = super().clean()
if self.runtime.execution_mode == "dry_run" and not cleaned.get("dry_run", False):
self.add_error("dry_run", "Reale Aktionen sind in execution_mode=dry_run deaktiviert.")
return cleaned
netbox_plugin/netbox_plugin_store/jobs.py
+36
View File
@@ -0,0 +1,36 @@
from __future__ import annotations
from core.exceptions import JobFailed
from netbox.jobs import JobRunner
from .lifecycle import LifecycleRequest, build_service
class PluginLifecycleJob(JobRunner):
"""Background execution is intentionally restricted to non-mutating plans."""
class Meta:
name = "Plugin Store dry-run"
def run(
self,
*,
audit_id: int,
slug: str,
action: str,
version: str = "",
dry_run: bool = True,
actor_id: int | None = None,
**kwargs,
):
if not dry_run:
raise JobFailed("Mutating lifecycle actions may not run inside netbox-rq.")
try:
build_service().execute(
LifecycleRequest(slug=slug, action=action, version=version, dry_run=True),
actor_id=actor_id,
audit_id=audit_id,
)
except Exception as exc:
self.logger.error("Plugin Store dry-run failed: %s", exc)
raise JobFailed(str(exc)) from exc
netbox_plugin/netbox_plugin_store/lifecycle.py
+537
View File
@@ -0,0 +1,537 @@
from __future__ import annotations
import importlib.metadata
import re
import tempfile
from dataclasses import asdict, dataclass, field
from pathlib import Path
from typing import Any, Callable, Protocol
from urllib.parse import urlsplit, urlunsplit
from uuid import uuid4
from packaging.version import InvalidVersion, Version
from .agent import AgentClient
from .client import CatalogPlugin, Release, StoreClient
from .commands import SubprocessRunner
from .editors import MutationReceipt, PluginConfigurationEditor, RequirementsEditor
from .locking import FileLock
from .redaction import redact_data, redact_text
from .runtime import RuntimeSettings
from .validation import ensure_not_self, validate_slug
ACTIONS = ("install", "update", "enable", "disable", "uninstall")
MUTATING_ACTIONS = frozenset(ACTIONS)
class LifecycleError(RuntimeError):
pass
class LifecycleRepository(Protocol):
def create_audit(self, request: "LifecycleRequest", actor_id: int | None) -> int: ...
def mark_running(self, audit_id: int) -> None: ...
def mark_success(self, audit_id: int, result: dict[str, Any]) -> None: ...
def mark_handed_off(self, audit_id: int, operation_id: str, result: dict[str, Any]) -> None: ...
def mark_failed(self, audit_id: int, error: str, result: dict[str, Any]) -> None: ...
def update_status(self, plugin: CatalogPlugin, **values: Any) -> None: ...
def has_other_pending_mutation(self, slug: str, audit_id: int) -> bool: ...
@dataclass(frozen=True, slots=True)
class LifecycleRequest:
slug: str
action: str
version: str = ""
dry_run: bool = True
def validate(self) -> None:
validate_slug(self.slug)
if self.action not in ACTIONS:
raise LifecycleError("Unknown lifecycle action.")
if self.version:
try:
Version(self.version)
except InvalidVersion as exc:
raise LifecycleError("Invalid requested version.") from exc
@dataclass(slots=True)
class LifecycleResult:
slug: str
action: str
state: str
dry_run: bool
installed_version: str = ""
enabled: bool = False
restart_required: bool = False
handed_to_agent: bool = False
operation_id: str = ""
plan: list[str] = field(default_factory=list)
output: str = ""
def safe_dict(self) -> dict[str, Any]:
return redact_data(asdict(self))
def installed_distribution_version(package_name: str) -> str:
try:
return importlib.metadata.version(package_name)
except importlib.metadata.PackageNotFoundError:
return ""
def runtime_plugin_is_loaded(import_name: str) -> bool:
try:
from django.conf import settings as django_settings
if not django_settings.configured:
return False
return import_name in django_settings.PLUGINS
except (ImportError, AttributeError, RuntimeError):
return False
class LifecycleService:
def __init__(
self,
settings: RuntimeSettings,
client: StoreClient,
repository: LifecycleRepository,
*,
runner: SubprocessRunner | None = None,
version_provider: Callable[[str], str] = installed_distribution_version,
runtime_active_provider: Callable[[str], bool] = runtime_plugin_is_loaded,
):
self.settings = settings
self.client = client
self.repository = repository
self.runner = runner or SubprocessRunner()
self.version_provider = version_provider
self.runtime_active_provider = runtime_active_provider
def execute(
self,
request: LifecycleRequest,
*,
actor_id: int | None = None,
audit_id: int | None = None,
) -> LifecycleResult:
request.validate()
if audit_id is None:
audit_id = self.repository.create_audit(request, actor_id)
self.repository.mark_running(audit_id)
result: LifecycleResult | None = None
try:
if request.dry_run:
result = self._execute_locked(request, mutate=False, requested_by=actor_id)
else:
if self.settings.execution_mode == "dry_run":
raise LifecycleError(
"Real lifecycle changes are disabled while execution_mode is dry_run."
)
if self.settings.execution_mode == "direct" and not self.settings.allow_lifecycle_mutations:
raise LifecycleError(
"Direct lifecycle changes require the explicit allow_lifecycle_mutations opt-in."
)
with FileLock(self.settings.lock_path, self.settings.lock_timeout):
if self.repository.has_other_pending_mutation(request.slug, audit_id):
raise LifecycleError("Another lifecycle operation for this plugin is still pending.")
result = self._execute_locked(request, mutate=True, requested_by=actor_id)
if result.handed_to_agent:
self.repository.mark_handed_off(audit_id, result.operation_id, result.safe_dict())
else:
self.repository.mark_success(audit_id, result.safe_dict())
return result
except Exception as exc:
safe_error = redact_text(exc)
safe_result = result.safe_dict() if result else {"slug": request.slug, "action": request.action}
self.repository.mark_failed(audit_id, safe_error, safe_result)
raise
def _execute_locked(
self, request: LifecycleRequest, *, mutate: bool, requested_by: int | None
) -> LifecycleResult:
plugin = self.client.get_plugin(request.slug)
ensure_not_self(plugin.package_name, plugin.import_name)
installed = self.version_provider(plugin.package_name)
config_editor = PluginConfigurationEditor(
self.settings.configuration_path,
self.settings.backup_dir,
self.settings.backups_to_keep,
)
requirements_editor = RequirementsEditor(
self.settings.requirements_path,
self.settings.backup_dir,
self.settings.backups_to_keep,
)
enabled = plugin.import_name in config_editor.enabled_plugins()
runtime_active = self.runtime_active_provider(plugin.import_name)
release: Release | None = None
if request.action in {"install", "update"}:
release = plugin.select_release(self.settings.netbox_version, request.version)
self._assert_installable(plugin, release)
self._assert_state(request.action, installed, enabled, runtime_active)
plan = self._build_plan(request.action, plugin, release, enabled)
if not mutate:
return LifecycleResult(
slug=plugin.slug,
action=request.action,
state="dry-run",
dry_run=True,
installed_version=installed,
enabled=enabled,
restart_required=(
request.action in {"enable", "disable"}
or (request.action == "update" and enabled)
),
plan=plan,
)
self.repository.update_status(
plugin,
state="running",
installed_version=installed,
enabled=enabled,
restart_required=False,
last_error="",
)
try:
if self.settings.execution_mode == "agent":
result = self._execute_agent(request, plugin, release, installed, enabled, plan, requested_by)
else:
result = self._execute_direct(
request, plugin, release, installed, enabled, plan, config_editor, requirements_editor
)
except Exception as exc:
self.repository.update_status(
plugin,
state="failed",
installed_version=self.version_provider(plugin.package_name) or installed,
enabled=enabled,
restart_required=True,
last_error=redact_text(exc, limit=4_000),
)
raise
self.repository.update_status(
plugin,
state=result.state,
installed_version=result.installed_version,
enabled=result.enabled,
restart_required=result.restart_required,
last_error="",
)
return result
@staticmethod
def _assert_installable(plugin: CatalogPlugin, release: Release) -> None:
if not plugin.approved:
raise LifecycleError("Plugin is not explicitly approved by the Store.")
if not release.approved or not release.immutable:
raise LifecycleError("Release is not both approved and immutable.")
if not release.download_url:
raise LifecycleError("Approved release has no artifact URL.")
if not release.sha256:
raise LifecycleError("Approved release has no artifact SHA-256. Commit hashes are not accepted.")
@staticmethod
def _assert_state(action: str, installed: str, enabled: bool, runtime_active: bool) -> None:
if action == "install" and installed:
raise LifecycleError("Plugin is already installed; use update.")
if action in {"update", "enable", "disable", "uninstall"} and not installed:
raise LifecycleError("Plugin is not installed.")
if action == "enable" and enabled:
raise LifecycleError("Plugin is already enabled.")
if action == "disable" and not enabled:
raise LifecycleError("Plugin is already disabled.")
if action == "uninstall" and (enabled or runtime_active):
raise LifecycleError("Disable the plugin and restart NetBox before uninstalling it.")
def _build_plan(
self, action: str, plugin: CatalogPlugin, release: Release | None, currently_enabled: bool
) -> list[str]:
plan: list[str] = []
if release:
plan.extend(
[
f"Download immutable release {release.version} from an allowlisted origin.",
f"Verify artifact SHA-256 {release.sha256} before executing pip.",
]
)
if action == "install":
plan.extend(["Install the verified local artifact with pip.", "Pin local_requirements.txt atomically."])
plan.append("Keep the newly installed plugin disabled until an explicit enable action.")
elif action == "update":
plan.extend(["Update the persistent requirement pin atomically.", "Upgrade from the verified local artifact."])
elif action == "enable":
plan.append(f"Add {plugin.import_name} to the static PLUGINS list atomically.")
elif action == "disable":
plan.append(f"Remove {plugin.import_name} from the static PLUGINS list atomically.")
elif action == "uninstall":
plan.extend(["Remove the persistent requirement pin atomically.", "Uninstall the distribution with pip."])
if action == "enable" or (action == "update" and currently_enabled):
if self.settings.run_migrations:
plan.append("Run NetBox database migrations without interaction.")
if self.settings.collect_static:
plan.append("Collect NetBox static files without interaction.")
if action in {"enable", "disable"} or (action == "update" and currently_enabled):
plan.append("Require a NetBox/web and worker restart before the new state is fully active.")
return plan
def _artifact_suffix(self, release: Release) -> str:
filename = Path(urlsplit(release.download_url).path).name
suffixes = Path(filename).suffixes[-2:]
suffix = "".join(suffixes) or ".artifact"
return suffix if re.fullmatch(r"\.[A-Za-z0-9.]{1,16}", suffix) else ".artifact"
def _download(self, release: Release, directory: Path) -> Path:
artifact = directory / f"artifact{self._artifact_suffix(release)}"
self.client.download_artifact(
release.download_url,
artifact,
expected_sha256=release.sha256,
timeout=self.settings.download_timeout,
max_bytes=self.settings.max_download_bytes,
)
return artifact
def _execute_agent(
self,
request: LifecycleRequest,
plugin: CatalogPlugin,
release: Release | None,
installed: str,
enabled: bool,
plan: list[str],
requested_by: int | None,
) -> LifecycleResult:
if self.settings.agent_socket_path is None:
raise LifecycleError("Host agent socket is not configured.")
if release is not None and not release.approved_payload_sha256:
raise LifecycleError("Store release is missing its opaque approved_payload_sha256 marker.")
agent = AgentClient(self.settings.agent_socket_path, timeout=self.settings.agent_timeout)
operation_id = agent.submit_operation(
request_id=str(uuid4()),
action=request.action,
plugin_slug=plugin.slug,
version=release.version if release else (request.version or None),
approved_payload_sha256=release.approved_payload_sha256 if release else None,
requested_by=f"netbox-user:{requested_by}" if requested_by is not None else "netbox-system",
)
return LifecycleResult(
slug=plugin.slug,
action=request.action,
state="handed-off",
dry_run=False,
installed_version=installed,
enabled=enabled,
restart_required=(
request.action in {"enable", "disable"} or (request.action == "update" and enabled)
),
handed_to_agent=True,
operation_id=str(operation_id),
plan=plan,
output="Operation accepted by the host agent.",
)
def _execute_direct(
self,
request: LifecycleRequest,
plugin: CatalogPlugin,
release: Release | None,
installed: str,
enabled: bool,
plan: list[str],
config_editor: PluginConfigurationEditor,
requirements_editor: RequirementsEditor,
) -> LifecycleResult:
output: list[str] = []
with tempfile.TemporaryDirectory(prefix="netbox-plugin-store-") as temp_name:
artifact = self._download(release, Path(temp_name)) if release else None
if request.action == "install":
new_version, new_enabled = self._direct_install(
plugin, release, artifact, requirements_editor, output
)
elif request.action == "update":
new_version, new_enabled = self._direct_update(
plugin, release, artifact, enabled, requirements_editor, output
)
elif request.action == "enable":
self._direct_enable(plugin, config_editor, output)
new_version, new_enabled = installed, True
elif request.action == "disable":
config_editor.set_enabled(plugin.import_name, False)
new_version, new_enabled = installed, False
else:
self._direct_uninstall(plugin, enabled, config_editor, requirements_editor, output)
new_version, new_enabled = "", False
return LifecycleResult(
slug=plugin.slug,
action=request.action,
state=(
"restart-required"
if request.action in {"enable", "disable"} or (request.action == "update" and enabled)
else ("installed" if new_version else "unknown")
),
dry_run=False,
installed_version=new_version,
enabled=new_enabled,
restart_required=(
request.action in {"enable", "disable"} or (request.action == "update" and enabled)
),
plan=plan,
output=redact_text("\n".join(output)),
)
def _pip_install(self, artifact: Path, *, upgrade: bool) -> str:
argv = [
self.settings.python_executable,
"-m",
"pip",
"install",
"--disable-pip-version-check",
"--no-input",
*self.settings.pip_extra_args,
]
if upgrade:
argv.append("--upgrade")
if not self.settings.allow_package_index:
argv.append("--no-index")
argv.append(str(artifact))
return self.runner.run(argv, timeout=self.settings.operation_timeout).output
def _pip_uninstall(self, package_name: str) -> str:
argv = [self.settings.python_executable, "-m", "pip", "uninstall", "--yes", package_name]
return self.runner.run(argv, timeout=self.settings.operation_timeout).output
def _pip_check(self) -> str:
argv = [self.settings.python_executable, "-m", "pip", "check"]
return self.runner.run(argv, timeout=self.settings.operation_timeout).output
@staticmethod
def _requirement_line(plugin: CatalogPlugin, release: Release) -> str:
parsed = urlsplit(release.download_url)
fragment = f"sha256={release.sha256}"
if parsed.fragment:
fragment = parsed.fragment + "&" + fragment
pinned_url = urlunsplit((parsed.scheme, parsed.netloc, parsed.path, parsed.query, fragment))
return f"{plugin.package_name} @ {pinned_url}"
def _manage(self, command: str) -> str:
if not self.settings.manage_path.is_file():
raise LifecycleError("NetBox manage.py path does not exist; configure manage_path.")
argv = [self.settings.python_executable, str(self.settings.manage_path), command, "--no-input"]
return self.runner.run(
argv, timeout=self.settings.operation_timeout, cwd=self.settings.manage_path.parent
).output
def _post_enable_steps(self, output: list[str]) -> None:
if self.settings.run_migrations:
output.append(self._manage("migrate"))
if self.settings.collect_static:
output.append(self._manage("collectstatic"))
def _direct_install(
self,
plugin: CatalogPlugin,
release: Release,
artifact: Path,
requirements_editor: RequirementsEditor,
output: list[str],
) -> tuple[str, bool]:
requirement_receipt: MutationReceipt | None = None
pip_installed = False
try:
output.append(self._pip_install(artifact, upgrade=False))
pip_installed = True
output.append(self._pip_check())
if self.settings.manage_requirements_file:
requirement_receipt = requirements_editor.set_requirement(
plugin.package_name, self._requirement_line(plugin, release)
)
except Exception:
if requirement_receipt:
requirements_editor.rollback(requirement_receipt)
if pip_installed:
try:
self._pip_uninstall(plugin.package_name)
except Exception:
pass
raise
return release.version, False
def _direct_update(
self,
plugin: CatalogPlugin,
release: Release,
artifact: Path,
enabled: bool,
requirements_editor: RequirementsEditor,
output: list[str],
) -> tuple[str, bool]:
receipt: MutationReceipt | None = None
pip_updated = False
try:
if self.settings.manage_requirements_file:
receipt = requirements_editor.set_requirement(
plugin.package_name, self._requirement_line(plugin, release)
)
output.append(self._pip_install(artifact, upgrade=True))
pip_updated = True
output.append(self._pip_check())
if enabled:
self._post_enable_steps(output)
except Exception:
if receipt and not pip_updated:
requirements_editor.rollback(receipt)
raise
return release.version, enabled
def _direct_enable(
self, plugin: CatalogPlugin, config_editor: PluginConfigurationEditor, output: list[str]
) -> None:
receipt = config_editor.set_enabled(plugin.import_name, True)
try:
self._post_enable_steps(output)
except Exception:
config_editor.rollback(receipt)
raise
def _direct_uninstall(
self,
plugin: CatalogPlugin,
enabled: bool,
config_editor: PluginConfigurationEditor,
requirements_editor: RequirementsEditor,
output: list[str],
) -> None:
config_receipt: MutationReceipt | None = None
requirement_receipt: MutationReceipt | None = None
try:
if self.settings.manage_requirements_file:
requirement_receipt = requirements_editor.set_requirement(plugin.package_name, None)
output.append(self._pip_uninstall(plugin.package_name))
except Exception:
if config_receipt:
config_editor.rollback(config_receipt)
if requirement_receipt:
requirements_editor.rollback(requirement_receipt)
raise
def build_service(*, repository: LifecycleRepository | None = None) -> LifecycleService:
settings = RuntimeSettings.from_django()
client = StoreClient(
settings.store_url,
settings.allowed_store_urls,
settings.allowed_artifact_urls,
api_token=settings.api_token,
timeout=settings.request_timeout,
)
if repository is None:
from .repository import DjangoLifecycleRepository
repository = DjangoLifecycleRepository()
return LifecycleService(settings, client, repository)
netbox_plugin/netbox_plugin_store/locking.py
+65
View File
@@ -0,0 +1,65 @@
from __future__ import annotations
import os
import time
from pathlib import Path
class LockTimeoutError(TimeoutError):
pass
class FileLock:
"""Small cross-platform inter-process exclusive lock."""
def __init__(self, path: Path, timeout: float = 30):
self.path = Path(path)
self.timeout = timeout
self._file = None
def __enter__(self):
self.path.parent.mkdir(parents=True, exist_ok=True)
self._file = self.path.open("a+b")
deadline = time.monotonic() + self.timeout
while True:
try:
self._acquire()
return self
except (BlockingIOError, OSError):
if time.monotonic() >= deadline:
self._file.close()
self._file = None
raise LockTimeoutError("Another plugin lifecycle operation is still running.")
time.sleep(0.1)
def _acquire(self) -> None:
if os.name == "nt":
import msvcrt
self._file.seek(0)
if self._file.read(1) == b"":
self._file.write(b"\0")
self._file.flush()
self._file.seek(0)
msvcrt.locking(self._file.fileno(), msvcrt.LK_NBLCK, 1)
else:
import fcntl
fcntl.flock(self._file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
def __exit__(self, exc_type, exc, tb):
if self._file is None:
return
try:
if os.name == "nt":
import msvcrt
self._file.seek(0)
msvcrt.locking(self._file.fileno(), msvcrt.LK_UNLCK, 1)
else:
import fcntl
fcntl.flock(self._file.fileno(), fcntl.LOCK_UN)
finally:
self._file.close()
self._file = None
netbox_plugin/netbox_plugin_store/migrations/0001_initial.py
+101
View File
@@ -0,0 +1,101 @@
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
initial = True
dependencies = [migrations.swappable_dependency(settings.AUTH_USER_MODEL)]
operations = [
migrations.CreateModel(
name="ManagedPlugin",
fields=[
("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
("slug", models.SlugField(max_length=64, unique=True)),
("name", models.CharField(max_length=200)),
("package_name", models.CharField(max_length=128)),
("import_name", models.CharField(max_length=128)),
("repository_url", models.URLField(blank=True, max_length=500)),
("installed_version", models.CharField(blank=True, max_length=64)),
("available_version", models.CharField(blank=True, max_length=64)),
("enabled", models.BooleanField(default=False)),
("restart_required", models.BooleanField(default=False)),
(
"state",
models.CharField(
choices=[
("unknown", "Unknown"),
("running", "Operation running"),
("installed", "Installed"),
("enabled", "Enabled"),
("disabled", "Disabled"),
("restart-required", "Restart required"),
("handed-off", "Handed to agent"),
("failed", "Failed"),
],
default="unknown",
max_length=32,
),
),
("last_error", models.TextField(blank=True)),
("last_checked", models.DateTimeField(blank=True, null=True)),
("created", models.DateTimeField(auto_now_add=True)),
("updated", models.DateTimeField(auto_now=True)),
],
options={
"ordering": ("name", "slug"),
"permissions": (("manage_plugin", "Can execute Plugin Store lifecycle actions"),),
},
),
migrations.CreateModel(
name="LifecycleAudit",
fields=[
("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
("slug", models.SlugField(max_length=64)),
("action", models.CharField(max_length=16)),
("requested_version", models.CharField(blank=True, max_length=64)),
("dry_run", models.BooleanField(default=True)),
(
"status",
models.CharField(
choices=[
("queued", "Queued"),
("running", "Running"),
("succeeded", "Succeeded"),
("failed", "Failed"),
("handed-off", "Handed to host agent"),
],
default="queued",
max_length=16,
),
),
("request_data", models.JSONField(blank=True, default=dict)),
("result_data", models.JSONField(blank=True, default=dict)),
("error", models.TextField(blank=True)),
("external_operation_id", models.UUIDField(blank=True, null=True, unique=True)),
("created", models.DateTimeField(auto_now_add=True)),
("started", models.DateTimeField(blank=True, null=True)),
("completed", models.DateTimeField(blank=True, null=True)),
(
"actor",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="+",
to=settings.AUTH_USER_MODEL,
),
),
],
options={
"ordering": ("-created",),
"permissions": (("execute_plugin_lifecycle", "Can request Plugin Store lifecycle execution"),),
},
),
migrations.AddIndex(
model_name="lifecycleaudit",
index=models.Index(fields=["slug", "-created"], name="nbps_audit_slug_created"),
),
]
netbox_plugin/netbox_plugin_store/migrations/__init__.py
+1
View File
@@ -0,0 +1 @@
netbox_plugin/netbox_plugin_store/models.py
+79
View File
@@ -0,0 +1,79 @@
from __future__ import annotations
from django.conf import settings
from django.db import models
class ManagedPlugin(models.Model):
class State(models.TextChoices):
UNKNOWN = "unknown", "Unknown"
RUNNING = "running", "Operation running"
INSTALLED = "installed", "Installed"
ENABLED = "enabled", "Enabled"
DISABLED = "disabled", "Disabled"
RESTART_REQUIRED = "restart-required", "Restart required"
HANDED_OFF = "handed-off", "Handed to agent"
FAILED = "failed", "Failed"
slug = models.SlugField(max_length=64, unique=True)
name = models.CharField(max_length=200)
package_name = models.CharField(max_length=128)
import_name = models.CharField(max_length=128)
repository_url = models.URLField(max_length=500, blank=True)
installed_version = models.CharField(max_length=64, blank=True)
available_version = models.CharField(max_length=64, blank=True)
enabled = models.BooleanField(default=False)
restart_required = models.BooleanField(default=False)
state = models.CharField(max_length=32, choices=State.choices, default=State.UNKNOWN)
last_error = models.TextField(blank=True)
last_checked = models.DateTimeField(null=True, blank=True)
created = models.DateTimeField(auto_now_add=True)
updated = models.DateTimeField(auto_now=True)
_netbox_private = True
class Meta:
ordering = ("name", "slug")
permissions = (("manage_plugin", "Can execute Plugin Store lifecycle actions"),)
def __str__(self) -> str:
return self.name
class LifecycleAudit(models.Model):
class Status(models.TextChoices):
QUEUED = "queued", "Queued"
RUNNING = "running", "Running"
SUCCEEDED = "succeeded", "Succeeded"
FAILED = "failed", "Failed"
HANDED_OFF = "handed-off", "Handed to host agent"
actor = models.ForeignKey(
settings.AUTH_USER_MODEL,
on_delete=models.SET_NULL,
related_name="+",
null=True,
blank=True,
)
slug = models.SlugField(max_length=64)
action = models.CharField(max_length=16)
requested_version = models.CharField(max_length=64, blank=True)
dry_run = models.BooleanField(default=True)
status = models.CharField(max_length=16, choices=Status.choices, default=Status.QUEUED)
request_data = models.JSONField(default=dict, blank=True)
result_data = models.JSONField(default=dict, blank=True)
error = models.TextField(blank=True)
external_operation_id = models.UUIDField(null=True, blank=True, unique=True)
created = models.DateTimeField(auto_now_add=True)
started = models.DateTimeField(null=True, blank=True)
completed = models.DateTimeField(null=True, blank=True)
_netbox_private = True
class Meta:
ordering = ("-created",)
indexes = (models.Index(fields=("slug", "-created"), name="nbps_audit_slug_created"),)
permissions = (("execute_plugin_lifecycle", "Can request Plugin Store lifecycle execution"),)
def __str__(self) -> str:
return f"{self.action} {self.slug} ({self.status})"
netbox_plugin/netbox_plugin_store/navigation.py
+34
View File
@@ -0,0 +1,34 @@
from netbox.plugins import PluginMenu, PluginMenuItem
_permission = "netbox_plugin_store.manage_plugin"
menu = PluginMenu(
label="Plugin Store",
icon_class="mdi mdi-store",
groups=(
(
"Store",
(
PluginMenuItem(
link="plugins:netbox_plugin_store:catalog",
link_text="Katalog",
auth_required=True,
permissions=[_permission],
),
PluginMenuItem(
link="plugins:netbox_plugin_store:status",
link_text="Installierte Plugins",
auth_required=True,
permissions=[_permission],
),
PluginMenuItem(
link="plugins:netbox_plugin_store:audit-list",
link_text="Audit-Protokoll",
auth_required=True,
permissions=[_permission],
),
),
),
),
)
netbox_plugin/netbox_plugin_store/redaction.py
+35
View File
@@ -0,0 +1,35 @@
from __future__ import annotations
import re
from collections.abc import Mapping, Sequence
_SECRET_KEYS = re.compile(r"token|secret|password|authorization|cookie|api[_-]?key", re.I)
_BEARER = re.compile(r"(?i)\bBearer\s+[A-Za-z0-9._~+\-/]+=*")
_URL_CREDENTIALS = re.compile(r"(?i)(https?://)([^/@\s:]+):([^/@\s]+)@")
_ASSIGNMENT = re.compile(
r"(?i)\b(token|secret|password|authorization|api[_-]?key)\s*([=:])\s*([^\s,;]+)"
)
def redact_text(value: object, *, limit: int = 8_000) -> str:
text = str(value)
text = _BEARER.sub("Bearer [REDACTED]", text)
text = _URL_CREDENTIALS.sub(r"\1[REDACTED]@", text)
text = _ASSIGNMENT.sub(r"\1\2[REDACTED]", text)
if len(text) > limit:
return text[:limit] + "\n...[truncated]"
return text
def redact_data(value: object) -> object:
if isinstance(value, Mapping):
return {
str(key): "[REDACTED]" if _SECRET_KEYS.search(str(key)) else redact_data(item)
for key, item in value.items()
}
if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
return [redact_data(item) for item in value]
if isinstance(value, str):
return redact_text(value)
return value
netbox_plugin/netbox_plugin_store/repository.py
+86
View File
@@ -0,0 +1,86 @@
from __future__ import annotations
from typing import Any
from django.db import transaction
from django.utils import timezone
from .client import CatalogPlugin
from .lifecycle import LifecycleRequest
from .models import LifecycleAudit, ManagedPlugin
from .redaction import redact_data, redact_text
class DjangoLifecycleRepository:
def has_other_pending_mutation(self, slug: str, audit_id: int) -> bool:
return LifecycleAudit.objects.filter(
slug=slug,
dry_run=False,
status=LifecycleAudit.Status.HANDED_OFF,
).exclude(pk=audit_id).exists()
def create_audit(self, request: LifecycleRequest, actor_id: int | None) -> int:
audit = LifecycleAudit.objects.create(
actor_id=actor_id,
slug=request.slug,
action=request.action,
requested_version=request.version,
dry_run=request.dry_run,
request_data={
"slug": request.slug,
"action": request.action,
"version": request.version,
"dry_run": request.dry_run,
},
)
return audit.pk
def mark_running(self, audit_id: int) -> None:
LifecycleAudit.objects.filter(pk=audit_id).update(
status=LifecycleAudit.Status.RUNNING,
started=timezone.now(),
error="",
)
def mark_success(self, audit_id: int, result: dict[str, Any]) -> None:
LifecycleAudit.objects.filter(pk=audit_id).update(
status=LifecycleAudit.Status.SUCCEEDED,
result_data=redact_data(result),
completed=timezone.now(),
)
def mark_handed_off(self, audit_id: int, operation_id: str, result: dict[str, Any]) -> None:
LifecycleAudit.objects.filter(pk=audit_id).update(
status=LifecycleAudit.Status.HANDED_OFF,
external_operation_id=operation_id,
result_data=redact_data(result),
)
def mark_failed(self, audit_id: int, error: str, result: dict[str, Any]) -> None:
LifecycleAudit.objects.filter(pk=audit_id).update(
status=LifecycleAudit.Status.FAILED,
error=redact_text(error, limit=8_000),
result_data=redact_data(result),
completed=timezone.now(),
)
@transaction.atomic
def update_status(self, plugin: CatalogPlugin, **values: Any) -> None:
safe_values = {
"name": plugin.name,
"package_name": plugin.package_name,
"import_name": plugin.import_name,
"repository_url": plugin.repository_url,
"available_version": plugin.latest_version,
"last_checked": timezone.now(),
**values,
}
if "last_error" in safe_values:
safe_values["last_error"] = redact_text(safe_values["last_error"], limit=4_000)
obj = ManagedPlugin.objects.select_for_update().filter(slug=plugin.slug).first()
if obj is None:
ManagedPlugin.objects.create(slug=plugin.slug, **safe_values)
else:
for key, value in safe_values.items():
setattr(obj, key, value)
obj.save(update_fields=tuple(safe_values) + ("updated",))
netbox_plugin/netbox_plugin_store/runtime.py
+199
View File
@@ -0,0 +1,199 @@
from __future__ import annotations
import os
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any
class RuntimeConfigurationError(RuntimeError):
pass
def _argv_list(value: object, setting: str) -> tuple[tuple[str, ...], ...]:
if not isinstance(value, (list, tuple)):
raise RuntimeConfigurationError(f"{setting} must be a list of argv lists.")
commands: list[tuple[str, ...]] = []
for command in value:
if (
not isinstance(command, (list, tuple))
or not command
or any(not isinstance(arg, str) or "\x00" in arg for arg in command)
):
raise RuntimeConfigurationError(f"Each {setting} entry must be a non-empty argv list.")
commands.append(tuple(command))
return tuple(commands)
def _string_list(value: object, setting: str) -> tuple[str, ...]:
if not isinstance(value, (list, tuple)) or any(not isinstance(v, str) for v in value):
raise RuntimeConfigurationError(f"{setting} must be a list of strings.")
return tuple(value)
@dataclass(frozen=True, slots=True)
class RuntimeSettings:
store_url: str
allowed_store_urls: tuple[str, ...]
allowed_artifact_urls: tuple[str, ...]
api_token: str
request_timeout: int
download_timeout: int
max_download_bytes: int
configuration_path: Path
requirements_path: Path
manage_requirements_file: bool
allow_lifecycle_mutations: bool
default_dry_run: bool
execution_mode: str
agent_socket_path: Path | None
agent_timeout: int
agent_poll_interval: float
manage_path: Path
lock_path: Path
backup_dir: Path
lock_timeout: int
operation_timeout: int
pip_extra_args: tuple[str, ...]
allow_package_index: bool
run_migrations: bool
collect_static: bool
background_jobs: bool
synchronous_fallback: bool
job_queue: str
auto_restart: bool
restart_commands: tuple[tuple[str, ...], ...]
restart_allowlist: tuple[tuple[str, ...], ...]
backups_to_keep: int
python_executable: str
netbox_version: str
@classmethod
def from_mapping(
cls,
raw: dict[str, Any],
*,
configuration_dir: str | os.PathLike[str],
netbox_root: str | os.PathLike[str],
base_dir: str | os.PathLike[str],
netbox_version: str,
) -> "RuntimeSettings":
configuration_dir = Path(configuration_dir)
netbox_root = Path(netbox_root)
base_dir = Path(base_dir)
config_path = Path(raw.get("configuration_path") or configuration_dir / "configuration.py")
requirements_path = Path(raw.get("requirements_path") or netbox_root / "local_requirements.txt")
configured_manage = raw.get("manage_path")
if configured_manage:
manage_path = Path(configured_manage)
else:
candidates = (base_dir / "manage.py", netbox_root / "netbox" / "manage.py", netbox_root / "manage.py")
manage_path = next((candidate for candidate in candidates if candidate.is_file()), candidates[0])
lock_path = Path(raw.get("lock_path") or configuration_dir / ".plugin-store.lock")
backup_dir = Path(raw.get("backup_dir") or configuration_dir / "plugin-store-backups")
restart_commands = _argv_list(raw.get("restart_commands", []), "restart_commands")
restart_allowlist = _argv_list(raw.get("restart_allowlist", []), "restart_allowlist")
if raw.get("auto_restart", False):
if not restart_commands:
raise RuntimeConfigurationError("auto_restart requires at least one restart command.")
denied = [command for command in restart_commands if command not in restart_allowlist]
if denied:
raise RuntimeConfigurationError("Every restart command must exactly match restart_allowlist.")
if str(raw.get("execution_mode", "dry_run")) == "direct":
raise RuntimeConfigurationError("auto_restart is forbidden in direct execution mode; use the host agent.")
pip_extra_args = _string_list(raw.get("pip_extra_args", []), "pip_extra_args")
if any("\x00" in arg for arg in pip_extra_args):
raise RuntimeConfigurationError("pip_extra_args contains an invalid NUL byte.")
settings = cls(
store_url=str(raw.get("store_url") or "").rstrip("/"),
allowed_store_urls=_string_list(raw.get("allowed_store_urls", []), "allowed_store_urls"),
allowed_artifact_urls=_string_list(raw.get("allowed_artifact_urls", []), "allowed_artifact_urls"),
api_token=str(raw.get("api_token", "")),
request_timeout=int(raw.get("request_timeout", 15)),
download_timeout=int(raw.get("download_timeout", 120)),
max_download_bytes=int(raw.get("max_download_bytes", 268_435_456)),
configuration_path=config_path,
requirements_path=requirements_path,
manage_requirements_file=bool(raw.get("manage_requirements_file", True)),
allow_lifecycle_mutations=bool(raw.get("allow_lifecycle_mutations", False)),
default_dry_run=bool(raw.get("default_dry_run", True)),
execution_mode=str(raw.get("execution_mode", "dry_run")),
agent_socket_path=Path(raw["agent_socket_path"]) if raw.get("agent_socket_path") else None,
agent_timeout=int(raw.get("agent_timeout", 30)),
agent_poll_interval=float(raw.get("agent_poll_interval", 1.0)),
manage_path=manage_path,
lock_path=lock_path,
backup_dir=backup_dir,
lock_timeout=int(raw.get("lock_timeout", 30)),
operation_timeout=int(raw.get("operation_timeout", 900)),
pip_extra_args=pip_extra_args,
allow_package_index=bool(raw.get("allow_package_index", False)),
run_migrations=bool(raw.get("run_migrations", True)),
collect_static=bool(raw.get("collect_static", True)),
background_jobs=bool(raw.get("background_jobs", True)),
synchronous_fallback=bool(raw.get("synchronous_fallback", True)),
job_queue=str(raw.get("job_queue", "default")),
auto_restart=bool(raw.get("auto_restart", False)),
restart_commands=restart_commands,
restart_allowlist=restart_allowlist,
backups_to_keep=int(raw.get("backups_to_keep", 25)),
python_executable=sys.executable,
netbox_version=str(netbox_version).split("-")[0],
)
settings.validate()
return settings
@classmethod
def from_django(cls) -> "RuntimeSettings":
from django.conf import settings as django_settings
plugin_settings = dict(django_settings.PLUGINS_CONFIG.get("netbox_plugin_store", {}))
config_cls = __import__("netbox_plugin_store", fromlist=["config"]).config
defaults = dict(config_cls.default_settings)
defaults.update(plugin_settings)
release = getattr(django_settings, "RELEASE", None)
version = getattr(release, "version", None) or django_settings.VERSION
return cls.from_mapping(
defaults,
configuration_dir=django_settings.CONFIGURATION_DIR,
netbox_root=getattr(django_settings, "NETBOX_ROOT", django_settings.BASE_DIR.parent),
base_dir=django_settings.BASE_DIR,
netbox_version=version,
)
def validate(self) -> None:
from .client import URLPolicy
URLPolicy(self.allowed_store_urls).check(self.store_url)
if not self.allowed_artifact_urls:
raise RuntimeConfigurationError("allowed_artifact_urls cannot be empty.")
if self.execution_mode not in {"dry_run", "direct", "agent"}:
raise RuntimeConfigurationError("execution_mode must be 'dry_run', 'direct', or 'agent'.")
if self.execution_mode == "agent":
if self.agent_socket_path is None or not self.agent_socket_path.is_absolute():
raise RuntimeConfigurationError("agent mode requires an absolute agent_socket_path.")
if not 1 <= self.agent_timeout <= 300:
raise RuntimeConfigurationError("agent_timeout must be between 1 and 300 seconds.")
if not 0.1 <= self.agent_poll_interval <= 30:
raise RuntimeConfigurationError("agent_poll_interval must be between 0.1 and 30 seconds.")
if not 1 <= self.request_timeout <= 300:
raise RuntimeConfigurationError("request_timeout must be between 1 and 300 seconds.")
if not 1 <= self.download_timeout <= 3_600:
raise RuntimeConfigurationError("download_timeout must be between 1 and 3600 seconds.")
if not 1_024 <= self.max_download_bytes <= 2_147_483_648:
raise RuntimeConfigurationError("max_download_bytes is outside the accepted range.")
if not 1 <= self.operation_timeout <= 7_200:
raise RuntimeConfigurationError("operation_timeout must be between 1 and 7200 seconds.")
if not 1 <= self.lock_timeout <= 300:
raise RuntimeConfigurationError("lock_timeout must be between 1 and 300 seconds.")
if not 1 <= self.backups_to_keep <= 500:
raise RuntimeConfigurationError("backups_to_keep must be between 1 and 500.")
if not self.job_queue or any(char.isspace() for char in self.job_queue):
raise RuntimeConfigurationError("job_queue is invalid.")
netbox_plugin/netbox_plugin_store/templates/netbox_plugin_store/audit_detail.html
+28
View File
@@ -0,0 +1,28 @@
{% extends 'base/layout.html' %}
{% block title %}Audit #{{ audit.pk }}{% endblock %}
{% block content %}
<a href="{% url 'plugins:netbox_plugin_store:audit-list' %}">← Zum Audit-Protokoll</a>
<h1 class="mt-2">Audit #{{ audit.pk }}</h1>
<div class="card mb-3">
<div class="card-body">
<dl class="row mb-0">
<dt class="col-3">Plugin</dt><dd class="col-9">{{ audit.slug }}</dd>
<dt class="col-3">Aktion</dt><dd class="col-9">{{ audit.action }}</dd>
<dt class="col-3">Version</dt><dd class="col-9">{{ audit.requested_version|default:"–" }}</dd>
<dt class="col-3">Benutzer</dt><dd class="col-9">{{ audit.actor|default:"System" }}</dd>
<dt class="col-3">Status</dt><dd class="col-9">{{ audit.get_status_display }}</dd>
<dt class="col-3">Dry-Run</dt><dd class="col-9">{{ audit.dry_run|yesno:"Ja,Nein" }}</dd>
<dt class="col-3">Beginn</dt><dd class="col-9">{{ audit.started|default:"–" }}</dd>
<dt class="col-3">Ende</dt><dd class="col-9">{{ audit.completed|default:"–" }}</dd>
{% if audit.external_operation_id %}<dt class="col-3">Agent-Operation</dt><dd class="col-9"><code>{{ audit.external_operation_id }}</code></dd>{% endif %}
</dl>
</div>
</div>
{% if audit.error %}<div class="alert alert-danger"><pre class="mb-0 text-wrap">{{ audit.error }}</pre></div>{% endif %}
<div class="card">
<div class="card-header"><h2 class="card-title">Redigiertes Ergebnis</h2></div>
<div class="card-body"><pre class="mb-0 text-wrap">{{ audit.result_data }}</pre></div>
</div>
{% endblock %}
netbox_plugin/netbox_plugin_store/templates/netbox_plugin_store/audit_list.html
+24
View File
@@ -0,0 +1,24 @@
{% extends 'base/layout.html' %}
{% block title %}Plugin Store Audit{% endblock %}
{% block content %}
<h1>Plugin Store Audit</h1>
<div class="card">
<div class="table-responsive">
<table class="table table-vcenter card-table">
<thead><tr><th>Zeit</th><th>Plugin</th><th>Aktion</th><th>Benutzer</th><th>Dry-Run</th><th>Status</th></tr></thead>
<tbody>
{% for audit in audits %}
<tr>
<td><a href="{% url 'plugins:netbox_plugin_store:audit-detail' pk=audit.pk %}">{{ audit.created }}</a></td>
<td>{{ audit.slug }}</td><td>{{ audit.action }}</td>
<td>{{ audit.actor|default:"System" }}</td>
<td>{{ audit.dry_run|yesno:"Ja,Nein" }}</td><td>{{ audit.get_status_display }}</td>
</tr>
{% empty %}<tr><td colspan="6" class="text-secondary">Keine Einträge.</td></tr>{% endfor %}
</tbody>
</table>
</div>
</div>
{% endblock %}
netbox_plugin/netbox_plugin_store/templates/netbox_plugin_store/catalog.html
+58
View File
@@ -0,0 +1,58 @@
{% extends 'base/layout.html' %}
{% block title %}Plugin Store{% endblock %}
{% block content %}
<div class="d-flex justify-content-between align-items-center mb-3">
<div>
<h1 class="mb-1">NetBox Plugin Store</h1>
<p class="text-secondary mb-0">Freigegebene Plugins für diese NetBox-Instanz.</p>
</div>
{% if runtime %}
<span class="badge {% if runtime.execution_mode == 'dry_run' %}bg-yellow text-dark{% else %}bg-green{% endif %}">
Modus: {{ runtime.execution_mode }}
</span>
{% endif %}
</div>
{% if store_error %}
<div class="alert alert-danger" role="alert">
<strong>Store nicht erreichbar:</strong> {{ store_error }}
</div>
{% endif %}
<div class="row row-cards">
{% for card in cards %}
<div class="col-sm-6 col-xl-4">
<div class="card h-100">
<div class="card-body">
<div class="d-flex justify-content-between">
<h3 class="card-title">{{ card.plugin.name }}</h3>
{% if card.plugin.approved %}
<span class="badge bg-green">Freigegeben</span>
{% else %}
<span class="badge bg-red">Nicht freigegeben</span>
{% endif %}
</div>
<p class="text-secondary">{{ card.plugin.summary|default:"Keine Zusammenfassung vorhanden." }}</p>
<dl class="row mb-0">
<dt class="col-5">Verfügbar</dt><dd class="col-7">{{ card.plugin.latest_version|default:"–" }}</dd>
<dt class="col-5">Installiert</dt><dd class="col-7">{{ card.installed_version|default:"–" }}</dd>
<dt class="col-5">Status</dt>
<dd class="col-7">
{% if card.enabled %}aktiv{% elif card.installed %}deaktiviert{% else %}nicht installiert{% endif %}
</dd>
</dl>
</div>
<div class="card-footer">
<a class="btn btn-primary" href="{% url 'plugins:netbox_plugin_store:plugin-detail' slug=card.plugin.slug %}">
Details
</a>
</div>
</div>
</div>
{% empty %}
{% if not store_error %}<p class="text-secondary">Der Store enthält noch keine freigegebenen Plugins.</p>{% endif %}
{% endfor %}
</div>
{% endblock %}
netbox_plugin/netbox_plugin_store/templates/netbox_plugin_store/confirm.html
+40
View File
@@ -0,0 +1,40 @@
{% extends 'base/layout.html' %}
{% block title %}{{ action }}: {{ plugin.name }}{% endblock %}
{% block content %}
<div class="row justify-content-center">
<div class="col-lg-7">
<div class="card">
<div class="card-header">
<h1 class="card-title">Lifecycle-Aktion bestätigen</h1>
</div>
<div class="card-body">
<div class="alert alert-warning">
Aktion <strong>{{ action }}</strong> für <strong>{{ plugin.name }}</strong>.
Reale Änderungen können Python-Pakete, <code>local_requirements.txt</code> und die NetBox-Konfiguration verändern.
</div>
{% if runtime.execution_mode == 'dry_run' %}
<div class="alert alert-info">Diese Instanz erlaubt ausschließlich Dry-Runs.</div>
{% endif %}
<form method="post" action="{% url 'plugins:netbox_plugin_store:lifecycle-action' slug=plugin.slug action=action %}">
{% csrf_token %}
{{ form.non_field_errors }}
{% for field in form %}
<div class="mb-3">
<label class="form-label" for="{{ field.id_for_label }}">{{ field.label }}</label>
{{ field }}
{% if field.help_text %}<div class="form-hint">{{ field.help_text }}</div>{% endif %}
{% for error in field.errors %}<div class="text-danger">{{ error }}</div>{% endfor %}
</div>
{% endfor %}
<div class="d-flex justify-content-between">
<a class="btn btn-outline-secondary" href="{% url 'plugins:netbox_plugin_store:plugin-detail' slug=plugin.slug %}">Abbrechen</a>
<button class="btn btn-danger" type="submit">Bestätigen</button>
</div>
</form>
</div>
</div>
</div>
</div>
{% endblock %}
netbox_plugin/netbox_plugin_store/templates/netbox_plugin_store/detail.html
+70
View File
@@ -0,0 +1,70 @@
{% extends 'base/layout.html' %}
{% block title %}{{ plugin.name }}{% endblock %}
{% block content %}
<div class="d-flex justify-content-between align-items-start mb-3">
<div>
<a href="{% url 'plugins:netbox_plugin_store:catalog' %}">← Zurück zum Katalog</a>
<h1 class="mt-2 mb-1">{{ plugin.name }}</h1>
<p class="text-secondary">{{ plugin.summary }}</p>
</div>
<div class="btn-list">
{% for action in actions %}
<a class="btn {% if action == 'uninstall' or action == 'disable' %}btn-outline-danger{% else %}btn-primary{% endif %}"
href="{% url 'plugins:netbox_plugin_store:lifecycle-confirm' slug=plugin.slug action=action %}">
{% if action == 'install' %}Installieren{% elif action == 'update' %}Aktualisieren{% elif action == 'enable' %}Aktivieren{% elif action == 'disable' %}Deaktivieren{% else %}Deinstallieren{% endif %}
</a>
{% endfor %}
</div>
</div>
{% if local_status and local_status.restart_required %}
<div class="alert alert-warning">Ein Neustart von NetBox und den Workern ist erforderlich.</div>
{% endif %}
<div class="row">
<div class="col-lg-8">
<div class="card mb-3">
<div class="card-header"><h2 class="card-title">README / Beschreibung</h2></div>
<div class="card-body">
<div class="text-break">{{ plugin.description|default:plugin.summary|linebreaksbr }}</div>
</div>
</div>
</div>
<div class="col-lg-4">
<div class="card mb-3">
<div class="card-header"><h2 class="card-title">Paket</h2></div>
<div class="card-body">
<dl class="row mb-0">
<dt class="col-5">Distribution</dt><dd class="col-7"><code>{{ plugin.package_name }}</code></dd>
<dt class="col-5">Import</dt><dd class="col-7"><code>{{ plugin.import_name }}</code></dd>
<dt class="col-5">Installiert</dt><dd class="col-7">{{ state.installed_version|default:"–" }}</dd>
<dt class="col-5">Aktiv</dt><dd class="col-7">{{ state.enabled|yesno:"Ja,Nein" }}</dd>
<dt class="col-5">NetBox</dt><dd class="col-7">{{ plugin.min_netbox_version|default:"–" }} – {{ plugin.max_netbox_version|default:"–" }}</dd>
</dl>
{% if plugin.repository_url %}
<a class="btn btn-outline-secondary mt-3" href="{{ plugin.repository_url }}" target="_blank" rel="noopener noreferrer">Repository öffnen</a>
{% endif %}
</div>
</div>
<div class="card">
<div class="card-header"><h2 class="card-title">Releases</h2></div>
<div class="list-group list-group-flush">
{% for item in releases %}
<div class="list-group-item d-flex justify-content-between">
<span>{{ item.release.version }}</span>
{% if item.compatible and item.release.approved and item.release.immutable and item.release.sha256 %}
<span class="badge bg-green">installierbar</span>
{% else %}
<span class="badge bg-secondary">gesperrt</span>
{% endif %}
</div>
{% empty %}
<div class="list-group-item text-secondary">Keine Releases</div>
{% endfor %}
</div>
</div>
</div>
</div>
{% endblock %}
netbox_plugin/netbox_plugin_store/templates/netbox_plugin_store/status.html
+29
View File
@@ -0,0 +1,29 @@
{% extends 'base/layout.html' %}
{% block title %}Installierte Plugins{% endblock %}
{% block content %}
<h1>Installierte Plugins</h1>
<div class="card">
<div class="table-responsive">
<table class="table table-vcenter card-table">
<thead><tr><th>Plugin</th><th>Installiert</th><th>Verfügbar</th><th>Aktiv</th><th>Status</th><th>Neustart</th><th>Aktualisiert</th></tr></thead>
<tbody>
{% for status in statuses %}
<tr>
<td><a href="{% url 'plugins:netbox_plugin_store:plugin-detail' slug=status.slug %}">{{ status.name }}</a></td>
<td>{{ status.installed_version|default:"–" }}</td>
<td>{{ status.available_version|default:"–" }}</td>
<td>{{ status.enabled|yesno:"Ja,Nein" }}</td>
<td>{{ status.get_state_display }}</td>
<td>{{ status.restart_required|yesno:"Erforderlich,Nein" }}</td>
<td>{{ status.updated }}</td>
</tr>
{% empty %}
<tr><td colspan="7" class="text-secondary">Noch keine Lifecycle-Aktion protokolliert.</td></tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
{% endblock %}
netbox_plugin/netbox_plugin_store/urls.py
+22
View File
@@ -0,0 +1,22 @@
from django.urls import path
from . import views
urlpatterns = [
path("", views.CatalogView.as_view(), name="catalog"),
path("installed/", views.InstalledStatusView.as_view(), name="status"),
path("audit/", views.AuditListView.as_view(), name="audit-list"),
path("audit/<int:pk>/", views.AuditDetailView.as_view(), name="audit-detail"),
path("plugins/<slug:slug>/", views.PluginDetailView.as_view(), name="plugin-detail"),
path(
"plugins/<slug:slug>/confirm/<str:action>/",
views.LifecycleConfirmView.as_view(),
name="lifecycle-confirm",
),
path(
"plugins/<slug:slug>/actions/<str:action>/",
views.LifecycleActionView.as_view(),
name="lifecycle-action",
),
]
netbox_plugin/netbox_plugin_store/validation.py
+57
View File
@@ -0,0 +1,57 @@
from __future__ import annotations
import re
from urllib.parse import urlsplit
from packaging.utils import canonicalize_name
SLUG_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]{0,62}[a-z0-9])?$")
DISTRIBUTION_RE = re.compile(r"^[A-Za-z0-9](?:[A-Za-z0-9._-]{0,126}[A-Za-z0-9])?$")
IMPORT_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]{0,127}$")
SHA256_RE = re.compile(r"^[a-fA-F0-9]{64}$")
SELF_IMPORT_NAME = "netbox_plugin_store"
SELF_DISTRIBUTION_NAME = canonicalize_name("netbox-plugin-store")
class ValidationError(ValueError):
pass
def validate_slug(value: str) -> str:
if not isinstance(value, str) or not SLUG_RE.fullmatch(value):
raise ValidationError("Invalid plugin slug.")
return value
def validate_distribution_name(value: str) -> str:
if not isinstance(value, str) or not DISTRIBUTION_RE.fullmatch(value):
raise ValidationError("Invalid Python distribution name.")
return value
def validate_import_name(value: str) -> str:
if not isinstance(value, str) or not IMPORT_RE.fullmatch(value):
raise ValidationError("Invalid Python import name.")
return value
def validate_sha256(value: str) -> str:
if not isinstance(value, str) or not SHA256_RE.fullmatch(value):
raise ValidationError("Invalid SHA-256 digest.")
return value.lower()
def ensure_not_self(package_name: str, import_name: str) -> None:
if canonicalize_name(package_name) == SELF_DISTRIBUTION_NAME or import_name == SELF_IMPORT_NAME:
raise ValidationError("The Plugin Store cannot manage its own lifecycle.")
def validate_public_url(value: str) -> str:
parsed = urlsplit(value)
if parsed.scheme not in {"https", "http"} or not parsed.hostname:
raise ValidationError("URL must be an absolute HTTP(S) URL.")
if parsed.username is not None or parsed.password is not None:
raise ValidationError("URLs containing credentials are not accepted.")
return value
netbox_plugin/netbox_plugin_store/version.py
+1
View File
@@ -0,0 +1 @@
__version__ = "0.1.0"
netbox_plugin/netbox_plugin_store/views.py
+267
View File
@@ -0,0 +1,267 @@
from __future__ import annotations
from django.contrib import messages
from django.contrib.auth.mixins import LoginRequiredMixin
from django.core.exceptions import PermissionDenied
from django.http import Http404
from django.shortcuts import get_object_or_404, redirect, render
from django.utils import timezone
from django.utils.decorators import method_decorator
from django.views import View
from django.views.decorators.csrf import csrf_protect
from .agent import AgentClient
from .access import has_store_access
from .client import StoreClient, StoreClientError
from .forms import LifecycleConfirmForm
from .jobs import PluginLifecycleJob
from .lifecycle import ACTIONS, LifecycleRequest, LifecycleService, installed_distribution_version
from .models import LifecycleAudit, ManagedPlugin
from .redaction import redact_data, redact_text
from .repository import DjangoLifecycleRepository
from .runtime import RuntimeSettings
class StorePermissionMixin(LoginRequiredMixin):
permission_name = "netbox_plugin_store.manage_plugin"
def dispatch(self, request, *args, **kwargs):
if not has_store_access(request.user, self.permission_name):
raise PermissionDenied
return super().dispatch(request, *args, **kwargs)
def _components() -> tuple[RuntimeSettings, StoreClient, DjangoLifecycleRepository]:
runtime = RuntimeSettings.from_django()
client = StoreClient(
runtime.store_url,
runtime.allowed_store_urls,
runtime.allowed_artifact_urls,
api_token=runtime.api_token,
timeout=runtime.request_timeout,
)
return runtime, client, DjangoLifecycleRepository()
def _plugin_state(plugin, active_plugins: set[str]) -> dict:
installed_version = installed_distribution_version(plugin.package_name)
enabled = plugin.import_name in active_plugins
return {
"installed_version": installed_version,
"enabled": enabled,
"installed": bool(installed_version),
}
class CatalogView(StorePermissionMixin, View):
def get(self, request):
from django.conf import settings as django_settings
try:
runtime, client, _ = _components()
plugins = client.list_plugins()
active_plugins = set(django_settings.PLUGINS)
cards = [{"plugin": plugin, **_plugin_state(plugin, active_plugins)} for plugin in plugins]
error = ""
except Exception as exc:
runtime, cards = None, []
error = redact_text(exc)
return render(
request,
"netbox_plugin_store/catalog.html",
{"cards": cards, "store_error": error, "runtime": runtime},
)
class PluginDetailView(StorePermissionMixin, View):
def get(self, request, slug: str):
from django.conf import settings as django_settings
try:
runtime, client, _ = _components()
plugin = client.get_plugin(slug)
except StoreClientError as exc:
raise Http404(redact_text(exc)) from exc
state = _plugin_state(plugin, set(django_settings.PLUGINS))
actions: list[str] = []
if not state["installed"]:
if plugin.approved:
actions.append("install")
else:
if plugin.approved:
actions.append("update")
if state["enabled"]:
actions.append("disable")
else:
actions.extend(("enable", "uninstall"))
releases = [
{"release": release, "compatible": release.supports(runtime.netbox_version, plugin)}
for release in plugin.releases
]
local_status = ManagedPlugin.objects.filter(slug=slug).first()
return render(
request,
"netbox_plugin_store/detail.html",
{
"plugin": plugin,
"state": state,
"actions": actions,
"releases": releases,
"local_status": local_status,
},
)
class LifecycleConfirmView(StorePermissionMixin, View):
def get(self, request, slug: str, action: str):
if action not in ACTIONS:
raise Http404
runtime, client, _ = _components()
plugin = client.get_plugin(slug)
form = LifecycleConfirmForm(plugin=plugin, action=action, runtime=runtime)
return render(
request,
"netbox_plugin_store/confirm.html",
{"plugin": plugin, "action": action, "form": form, "runtime": runtime},
)
@method_decorator(csrf_protect, name="dispatch")
class LifecycleActionView(StorePermissionMixin, View):
http_method_names = ["post"]
def post(self, request, slug: str, action: str):
if action not in ACTIONS:
raise Http404
runtime, client, repository = _components()
plugin = client.get_plugin(slug)
form = LifecycleConfirmForm(
request.POST,
plugin=plugin,
action=action,
runtime=runtime,
)
if not form.is_valid():
return render(
request,
"netbox_plugin_store/confirm.html",
{"plugin": plugin, "action": action, "form": form, "runtime": runtime},
status=400,
)
operation = LifecycleRequest(
slug=slug,
action=action,
version=form.cleaned_data.get("version", ""),
dry_run=form.cleaned_data["dry_run"],
)
if not operation.dry_run and not request.user.is_superuser:
raise PermissionDenied("Reale Plugin-Lifecycle-Aktionen erfordern einen Superuser.")
audit_id = repository.create_audit(operation, request.user.pk)
if operation.dry_run and runtime.background_jobs:
try:
PluginLifecycleJob.enqueue(
user=request.user,
queue_name=runtime.job_queue,
audit_id=audit_id,
slug=slug,
action=action,
version=operation.version,
dry_run=True,
actor_id=request.user.pk,
)
messages.success(request, "Dry-Run wurde in die NetBox-Jobqueue gestellt.")
return redirect("plugins:netbox_plugin_store:audit-detail", pk=audit_id)
except Exception as exc:
if not runtime.synchronous_fallback:
repository.mark_failed(audit_id, redact_text(exc), {"queue_failed": True})
messages.error(request, "Dry-Run konnte nicht eingeplant werden.")
return redirect("plugins:netbox_plugin_store:audit-detail", pk=audit_id)
service = LifecycleService(runtime, client, repository)
try:
result = service.execute(operation, actor_id=request.user.pk, audit_id=audit_id)
except Exception as exc:
messages.error(request, f"Lifecycle-Aktion fehlgeschlagen: {redact_text(exc, limit=1_000)}")
else:
label = "Dry-Run abgeschlossen" if result.dry_run else "Lifecycle-Aktion abgeschlossen"
messages.success(request, label + ".")
return redirect("plugins:netbox_plugin_store:audit-detail", pk=audit_id)
class InstalledStatusView(StorePermissionMixin, View):
def get(self, request):
statuses = ManagedPlugin.objects.all()
return render(request, "netbox_plugin_store/status.html", {"statuses": statuses})
class AuditListView(StorePermissionMixin, View):
def get(self, request):
audits = LifecycleAudit.objects.select_related("actor")[:200]
return render(request, "netbox_plugin_store/audit_list.html", {"audits": audits})
class AuditDetailView(StorePermissionMixin, View):
def get(self, request, pk: int):
audit = get_object_or_404(LifecycleAudit.objects.select_related("actor"), pk=pk)
if audit.status == LifecycleAudit.Status.HANDED_OFF and audit.external_operation_id:
self._refresh_agent_status(request, audit)
audit.refresh_from_db()
return render(request, "netbox_plugin_store/audit_detail.html", {"audit": audit})
@staticmethod
def _refresh_agent_status(request, audit: LifecycleAudit) -> None:
try:
runtime = RuntimeSettings.from_django()
if runtime.execution_mode != "agent" or runtime.agent_socket_path is None:
return
body = AgentClient(runtime.agent_socket_path, timeout=runtime.agent_timeout).get_operation(
audit.external_operation_id
)
state = body.get("state") or body.get("status")
if state in {"succeeded", "completed"}:
result = body.get("result", body)
if not isinstance(result, dict):
result = {"message": "Host agent completed without a structured result."}
LifecycleAudit.objects.filter(pk=audit.pk).update(
status=LifecycleAudit.Status.SUCCEEDED,
result_data=redact_data(result),
error="",
completed=timezone.now(),
)
status_values = {}
if "installed_version" in result:
status_values["installed_version"] = str(result["installed_version"] or "")[:64]
if "enabled" in result:
status_values["enabled"] = result["enabled"] is True
if "restart_required" in result:
status_values["restart_required"] = result["restart_required"] is True
if status_values:
if status_values.get("restart_required"):
status_values["state"] = ManagedPlugin.State.RESTART_REQUIRED
elif status_values.get("installed_version"):
status_values["state"] = (
ManagedPlugin.State.ENABLED
if status_values.get("enabled")
else ManagedPlugin.State.DISABLED
)
else:
status_values["state"] = ManagedPlugin.State.UNKNOWN
status_values.update({"last_error": "", "last_checked": timezone.now()})
ManagedPlugin.objects.filter(slug=audit.slug).update(**status_values)
elif state in {"failed", "errored", "cancelled"}:
error = redact_text(body.get("error") or f"Host agent operation {state}.")
LifecycleAudit.objects.filter(pk=audit.pk).update(
status=LifecycleAudit.Status.FAILED,
result_data=redact_data(body),
error=error,
completed=timezone.now(),
)
ManagedPlugin.objects.filter(slug=audit.slug).update(
state=ManagedPlugin.State.FAILED,
last_error=error,
restart_required=True,
last_checked=timezone.now(),
)
elif state in {"queued", "pending", "running", "accepted"}:
LifecycleAudit.objects.filter(pk=audit.pk).update(result_data=redact_data(body))
except Exception as exc:
messages.warning(request, f"Host-Agent-Status konnte nicht aktualisiert werden: {redact_text(exc)}")
netbox_plugin/pyproject.toml
+39
View File
@@ -0,0 +1,39 @@
[build-system]
requires = ["setuptools>=75", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "netbox-plugin-store"
version = "0.1.0"
description = "A secure NetBox 4.6 plugin lifecycle client for the MrBlake Plugin Store"
readme = "README.md"
requires-python = ">=3.12"
license = { text = "Apache-2.0" }
authors = [{ name = "MrBlake" }]
dependencies = [
"packaging>=24.0",
]
classifiers = [
"Development Status :: 3 - Alpha",
"Framework :: Django",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: System :: Systems Administration",
]
[project.urls]
Homepage = "https://git.mrblake.cc"
[tool.setuptools.packages.find]
include = ["netbox_plugin_store*"]
[tool.setuptools.package-data]
netbox_plugin_store = [
"templates/netbox_plugin_store/*.html",
"migrations/*.py",
]
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-q"
netbox_plugin/tests/_bootstrap.py
+41
View File
@@ -0,0 +1,41 @@
import sys
import types
if "netbox.plugins" not in sys.modules:
netbox = types.ModuleType("netbox")
plugins = types.ModuleType("netbox.plugins")
class PluginConfig:
def ready(self):
return None
class PluginMenu:
def __init__(self, label, groups, icon_class=None):
self.label = label
self.groups = groups
self.icon_class = icon_class
class PluginMenuItem:
def __init__(
self,
link,
link_text,
auth_required=False,
staff_only=False,
permissions=None,
buttons=None,
):
self.link = link
self.link_text = link_text
self.auth_required = auth_required
self.staff_only = staff_only
self.permissions = permissions or []
self.buttons = buttons or []
plugins.PluginConfig = PluginConfig
plugins.PluginMenu = PluginMenu
plugins.PluginMenuItem = PluginMenuItem
netbox.plugins = plugins
sys.modules["netbox"] = netbox
sys.modules["netbox.plugins"] = plugins
netbox_plugin/tests/netbox_test_configuration.py
+34
View File
@@ -0,0 +1,34 @@
ALLOWED_HOSTS = ["localhost"]
SECRET_KEY = "netbox-plugin-store-compatibility-test-key-2026-08-24!"
API_TOKEN_PEPPERS = {
1: "netbox-plugin-store-compatibility-test-pepper-2026-08-24!",
}
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": "netbox_compatibility_test",
"USER": "netbox",
"PASSWORD": "unused",
"HOST": "127.0.0.1",
"PORT": 5432,
"CONN_MAX_AGE": 0,
}
}
REDIS = {
"tasks": {"HOST": "127.0.0.1", "PORT": 6379, "DATABASE": 0},
"caching": {"HOST": "127.0.0.1", "PORT": 6379, "DATABASE": 1},
}
PLUGINS = ["netbox_plugin_store"]
PLUGINS_CONFIG = {
"netbox_plugin_store": {
"store_url": "https://store.example",
"allowed_store_urls": ["https://store.example"],
"allowed_artifact_urls": ["https://store.example"],
}
}
CENSUS_REPORTING_ENABLED = False
RELEASE_CHECK_URL = None
netbox_plugin/tests/official_netbox_smoke.py
+130
View File
@@ -0,0 +1,130 @@
"""Smoke tests executed in an environment containing an official NetBox tag."""
from __future__ import annotations
import inspect
import os
import sys
import unittest
from pathlib import Path
from unittest.mock import patch
NETBOX_SOURCE = Path(os.environ["NETBOX_SOURCE"]).resolve()
EXPECTED_VERSION = os.environ["NETBOX_EXPECTED_VERSION"]
sys.path.insert(0, str(NETBOX_SOURCE / "netbox"))
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "netbox.settings")
os.environ.setdefault("NETBOX_CONFIGURATION", "netbox_test_configuration")
import django # noqa: E402
django.setup()
from django.apps import apps # noqa: E402
from django.db.migrations.autodetector import MigrationAutodetector # noqa: E402
from django.db.migrations.loader import MigrationLoader # noqa: E402
from django.db.migrations.questioner import MigrationQuestioner # noqa: E402
from django.db.migrations.state import ProjectState # noqa: E402
from django.template.loader import get_template # noqa: E402
from django.urls import resolve, reverse # noqa: E402
from core.models import Job # noqa: E402
from netbox.jobs import JobRunner # noqa: E402
from netbox_plugin_store import config # noqa: E402
from netbox_plugin_store.jobs import PluginLifecycleJob # noqa: E402
from netbox_plugin_store.navigation import menu # noqa: E402
from users.models import User # noqa: E402
class OfficialNetBoxCompatibilityTests(unittest.TestCase):
def test_exact_netbox_and_django_runtime(self):
from django.conf import settings
self.assertEqual(settings.RELEASE.version, EXPECTED_VERSION)
self.assertGreaterEqual(sys.version_info, (3, 12))
self.assertEqual(django.get_version(), "6.0.7" if EXPECTED_VERSION == "4.6.5" else "6.0.8")
def test_plugin_config_is_registered(self):
app = apps.get_app_config("netbox_plugin_store")
self.assertIsInstance(app, config)
self.assertEqual(config.min_version, "4.6.5")
self.assertEqual(config.max_version, "4.6.8")
def test_jobrunner_queue_metadata_and_run_arguments_are_compatible(self):
self.assertIn("queue_name", inspect.signature(Job.enqueue).parameters)
self.assertTrue(issubclass(PluginLifecycleJob, JobRunner))
with patch.object(Job, "enqueue", return_value="queued") as enqueue:
result = PluginLifecycleJob.enqueue(
user="operator",
queue_name="compatibility",
audit_id=12,
slug="example-plugin",
action="install",
version="1.0.0",
dry_run=True,
actor_id=34,
)
self.assertEqual(result, "queued")
args, kwargs = enqueue.call_args
self.assertIs(args[0].__func__, PluginLifecycleJob.handle.__func__)
self.assertEqual(kwargs["queue_name"], "compatibility")
self.assertEqual(kwargs["audit_id"], 12)
self.assertEqual(kwargs["actor_id"], 34)
def test_navigation_urls_and_permission_semantics(self):
self.assertFalse(hasattr(User(), "is_staff"))
urls = {
"catalog": reverse("plugins:netbox_plugin_store:catalog"),
"status": reverse("plugins:netbox_plugin_store:status"),
"audit-list": reverse("plugins:netbox_plugin_store:audit-list"),
"audit-detail": reverse("plugins:netbox_plugin_store:audit-detail", kwargs={"pk": 7}),
"plugin-detail": reverse(
"plugins:netbox_plugin_store:plugin-detail", kwargs={"slug": "example-plugin"}
),
"lifecycle-confirm": reverse(
"plugins:netbox_plugin_store:lifecycle-confirm",
kwargs={"slug": "example-plugin", "action": "install"},
),
"lifecycle-action": reverse(
"plugins:netbox_plugin_store:lifecycle-action",
kwargs={"slug": "example-plugin", "action": "install"},
),
}
for name, url in urls.items():
self.assertEqual(resolve(url).url_name, name)
items = [item for group in menu.groups for item in group.items]
self.assertEqual(len(items), 3)
for item in items:
self.assertTrue(item.auth_required)
self.assertFalse(item.staff_only)
self.assertEqual(item.permissions, ["netbox_plugin_store.manage_plugin"])
def test_templates_resolve_and_compile_from_installed_wheel(self):
for name in (
"audit_detail.html",
"audit_list.html",
"catalog.html",
"confirm.html",
"detail.html",
"status.html",
):
template = get_template(f"netbox_plugin_store/{name}")
self.assertIsNotNone(template.template)
self.assertIsNotNone(get_template("base/layout.html").template)
def test_migration_graph_and_model_state_have_no_drift(self):
loader = MigrationLoader(None, ignore_no_migrations=True)
self.assertIn(("netbox_plugin_store", "0001_initial"), loader.disk_migrations)
from_state = loader.project_state()
to_state = ProjectState.from_apps(apps)
changes = MigrationAutodetector(
from_state,
to_state,
MigrationQuestioner(specified_apps={"netbox_plugin_store"}),
).changes(graph=loader.graph, trim_to_apps={"netbox_plugin_store"})
self.assertNotIn("netbox_plugin_store", changes)
if __name__ == "__main__":
unittest.main(verbosity=2)
netbox_plugin/tests/test_agent_and_commands.py
+81
View File
@@ -0,0 +1,81 @@
import json
import subprocess
import unittest
from pathlib import Path
from unittest.mock import patch
from uuid import UUID
import _bootstrap # noqa: F401
from netbox_plugin_store.agent import AgentClient
from netbox_plugin_store.commands import SubprocessRunner
class FakeSocket:
def __init__(self, response):
self.response = response
self.sent = b""
def __enter__(self):
return self
def __exit__(self, *args):
return None
def settimeout(self, timeout):
self.timeout = timeout
def connect(self, path):
self.path = path
def sendall(self, value):
self.sent += value
def shutdown(self, how):
return None
def recv(self, size):
value, self.response = self.response, b""
return value
class AgentTests(unittest.TestCase):
def test_json_line_operation_protocol(self):
operation_id = "12345678-1234-5678-1234-567812345678"
response = (
json.dumps({"protocol_version": 1, "status": 202, "body": {"operation_id": operation_id}}).encode()
+ b"\n"
)
fake = FakeSocket(response)
with (
patch("netbox_plugin_store.agent.socket.AF_UNIX", 1, create=True),
patch("netbox_plugin_store.agent.socket.SOCK_STREAM", 1),
patch("netbox_plugin_store.agent.socket.socket", return_value=fake),
):
result = AgentClient(Path("/run/store.sock")).submit_operation(
request_id="request-id",
action="install",
plugin_slug="example-plugin",
version="1.0.0",
approved_payload_sha256="c" * 64,
requested_by="netbox-user:1",
)
self.assertEqual(result, UUID(operation_id))
request = json.loads(fake.sent.decode().strip())
self.assertEqual(request["method"], "POST")
self.assertEqual(request["path"], "/v1/operations")
self.assertEqual(request["body"]["plugin_slug"], "example-plugin")
UUID(request["idempotency_key"])
class CommandTests(unittest.TestCase):
def test_subprocess_boundary_never_uses_shell(self):
completed = subprocess.CompletedProcess(["python", "-V"], 0, "Python", "")
with patch("netbox_plugin_store.commands.subprocess.run", return_value=completed) as run:
result = SubprocessRunner().run(["python", "-V"], timeout=5)
self.assertEqual(result.stdout, "Python")
self.assertIs(run.call_args.kwargs["shell"], False)
if __name__ == "__main__":
unittest.main()
netbox_plugin/tests/test_client.py
+122
View File
@@ -0,0 +1,122 @@
import hashlib
import io
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
import _bootstrap # noqa: F401
from netbox_plugin_store.client import CatalogPlugin, StoreClient, StoreClientError, URLPolicy
def plugin_payload():
artifact = b"verified artifact"
return {
"slug": "example-plugin",
"name": "Example",
"summary": "Example plugin",
"description": "README",
"repository_url": "https://git.mrblake.cc/team/example",
"latest_version": "1.2.0",
"package_name": "netbox-example",
"import_name": "netbox_example",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
"approved": True,
"releases": [
{
"version": "1.2.0",
"download_url": "https://store.example/artifacts/example.whl",
"sha256": hashlib.sha256(artifact).hexdigest(),
"approved_payload_sha256": "a" * 64,
"approved": True,
"immutable": True,
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
}
],
}, artifact
class URLPolicyTests(unittest.TestCase):
def test_exact_origin_and_path_prefix(self):
policy = URLPolicy(["https://store.example/internal"])
self.assertEqual(
policy.check("https://store.example/internal/api/v1/plugins/"),
"https://store.example/internal/api/v1/plugins/",
)
with self.assertRaises(StoreClientError):
policy.check("https://store.example.evil/internal")
with self.assertRaises(StoreClientError):
policy.check("https://store.example/other")
def test_release_selection_checks_netbox_version(self):
payload, _ = plugin_payload()
plugin = CatalogPlugin.from_mapping(payload)
self.assertEqual(plugin.select_release("4.6.8").version, "1.2.0")
with self.assertRaises(StoreClientError):
plugin.select_release("4.6.9")
class _Response(io.BytesIO):
def __init__(self, body):
super().__init__(body)
self.headers = {"Content-Length": str(len(body))}
def __enter__(self):
return self
def __exit__(self, *args):
self.close()
class _Opener:
def __init__(self, body):
self.body = body
def open(self, request, timeout):
return _Response(self.body)
class DownloadTests(unittest.TestCase):
def test_download_is_hashed_before_use(self):
_, artifact = plugin_payload()
client = StoreClient(
"https://store.example", ("https://store.example",), ("https://store.example",)
)
with tempfile.TemporaryDirectory() as temp_name:
target = Path(temp_name) / "plugin.whl"
with patch("netbox_plugin_store.client.build_opener", return_value=_Opener(artifact)):
actual, size = client.download_artifact(
"https://store.example/artifacts/example.whl",
target,
expected_sha256=hashlib.sha256(artifact).hexdigest(),
timeout=10,
max_bytes=1_000,
)
self.assertEqual(actual, hashlib.sha256(artifact).hexdigest())
self.assertEqual(size, len(artifact))
self.assertEqual(target.read_bytes(), artifact)
def test_mismatched_hash_removes_download(self):
_, artifact = plugin_payload()
client = StoreClient(
"https://store.example", ("https://store.example",), ("https://store.example",)
)
with tempfile.TemporaryDirectory() as temp_name:
target = Path(temp_name) / "plugin.whl"
with patch("netbox_plugin_store.client.build_opener", return_value=_Opener(artifact)):
with self.assertRaises(StoreClientError):
client.download_artifact(
"https://store.example/artifacts/example.whl",
target,
expected_sha256="0" * 64,
timeout=10,
max_bytes=1_000,
)
self.assertFalse(target.exists())
if __name__ == "__main__":
unittest.main()
netbox_plugin/tests/test_editors.py
+52
View File
@@ -0,0 +1,52 @@
import tempfile
import unittest
from pathlib import Path
import _bootstrap # noqa: F401
from netbox_plugin_store.editors import EditorError, PluginConfigurationEditor, RequirementsEditor
class ConfigurationEditorTests(unittest.TestCase):
def test_atomic_change_backup_and_rollback(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
config = root / "configuration.py"
original = "SECRET_KEY = 'unchanged'\nPLUGINS = [\n 'netbox_plugin_store',\n]\n"
config.write_text(original, encoding="utf-8")
editor = PluginConfigurationEditor(config, root / "backups", 5)
receipt = editor.set_enabled("netbox_example", True)
self.assertTrue(receipt.changed)
self.assertTrue(receipt.backup_path.is_file())
self.assertIn("'netbox_example'", config.read_text(encoding="utf-8"))
editor.rollback(receipt)
self.assertEqual(config.read_text(encoding="utf-8"), original)
def test_dynamic_plugins_assignment_is_rejected(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
config = root / "configuration.py"
config.write_text("PLUGINS = load_plugins()\n", encoding="utf-8")
editor = PluginConfigurationEditor(config, root / "backups", 5)
with self.assertRaises(EditorError):
editor.set_enabled("netbox_example", True)
class RequirementsEditorTests(unittest.TestCase):
def test_pin_and_remove_preserve_other_lines(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
requirements = root / "local_requirements.txt"
requirements.write_text("# managed manually\nother-package==2\n", encoding="utf-8")
editor = RequirementsEditor(requirements, root / "backups", 5)
line = "netbox-example @ https://store.example/example.whl#sha256=" + "a" * 64
editor.set_requirement("netbox-example", line)
text = requirements.read_text(encoding="utf-8")
self.assertIn(line, text)
self.assertIn("other-package==2", text)
editor.set_requirement("netbox-example", None)
self.assertNotIn("netbox-example", requirements.read_text(encoding="utf-8"))
if __name__ == "__main__":
unittest.main()
netbox_plugin/tests/test_lifecycle.py
+169
View File
@@ -0,0 +1,169 @@
import hashlib
import tempfile
import unittest
from pathlib import Path
import _bootstrap # noqa: F401
from netbox_plugin_store.client import CatalogPlugin
from netbox_plugin_store.commands import CommandResult
from netbox_plugin_store.lifecycle import LifecycleError, LifecycleRequest, LifecycleService
from netbox_plugin_store.runtime import RuntimeSettings
ARTIFACT = b"approved wheel bytes"
def catalog_plugin():
return CatalogPlugin.from_mapping(
{
"slug": "example-plugin",
"name": "Example",
"package_name": "netbox-example",
"import_name": "netbox_example",
"latest_version": "1.0.0",
"approved": True,
"releases": [
{
"version": "1.0.0",
"download_url": "https://store.example/example.whl",
"sha256": hashlib.sha256(ARTIFACT).hexdigest(),
"approved_payload_sha256": "b" * 64,
"approved": True,
"immutable": True,
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
}
],
}
)
class FakeClient:
def __init__(self, plugin):
self.plugin = plugin
self.downloads = 0
def get_plugin(self, slug):
return self.plugin
def download_artifact(self, url, destination, **kwargs):
self.downloads += 1
destination.write_bytes(ARTIFACT)
return hashlib.sha256(ARTIFACT).hexdigest(), len(ARTIFACT)
class FakeRepository:
def __init__(self):
self.audits = {}
self.statuses = []
def create_audit(self, request, actor_id):
self.audits[1] = {"status": "queued"}
return 1
def mark_running(self, audit_id):
self.audits[audit_id]["status"] = "running"
def mark_success(self, audit_id, result):
self.audits[audit_id].update(status="succeeded", result=result)
def mark_handed_off(self, audit_id, operation_id, result):
self.audits[audit_id].update(status="handed-off", operation_id=operation_id, result=result)
def mark_failed(self, audit_id, error, result):
self.audits[audit_id].update(status="failed", error=error)
def update_status(self, plugin, **values):
self.statuses.append(values)
def has_other_pending_mutation(self, slug, audit_id):
return False
class FakeRunner:
def __init__(self):
self.calls = []
def run(self, argv, **kwargs):
self.calls.append((list(argv), kwargs))
return CommandResult(0, "ok", "")
def runtime(root: Path, *, execution_mode="direct", allow=True):
config = root / "configuration.py"
config.write_text("PLUGINS = ['netbox_plugin_store']\n", encoding="utf-8")
return RuntimeSettings.from_mapping(
{
"store_url": "https://store.example",
"allowed_store_urls": ["https://store.example"],
"allowed_artifact_urls": ["https://store.example"],
"configuration_path": str(config),
"requirements_path": str(root / "local_requirements.txt"),
"manage_path": str(root / "manage.py"),
"lock_path": str(root / "operation.lock"),
"backup_dir": str(root / "backups"),
"execution_mode": execution_mode,
"allow_lifecycle_mutations": allow,
"run_migrations": False,
"collect_static": False,
"allow_package_index": False,
},
configuration_dir=root,
netbox_root=root,
base_dir=root,
netbox_version="4.6.8",
)
class LifecycleTests(unittest.TestCase):
def test_dry_run_has_no_download_or_subprocess(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
client = FakeClient(catalog_plugin())
repository = FakeRepository()
runner = FakeRunner()
service = LifecycleService(runtime(root), client, repository, runner=runner, version_provider=lambda _: "")
result = service.execute(LifecycleRequest("example-plugin", "install", "1.0.0", True))
self.assertEqual(result.state, "dry-run")
self.assertEqual(client.downloads, 0)
self.assertEqual(runner.calls, [])
def test_direct_install_is_disabled_and_uses_no_index_and_pip_check(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
client = FakeClient(catalog_plugin())
repository = FakeRepository()
runner = FakeRunner()
service = LifecycleService(runtime(root), client, repository, runner=runner, version_provider=lambda _: "")
result = service.execute(LifecycleRequest("example-plugin", "install", "1.0.0", False))
self.assertFalse(result.enabled)
self.assertFalse(result.restart_required)
self.assertNotIn("netbox_example", (root / "configuration.py").read_text(encoding="utf-8"))
requirement = (root / "local_requirements.txt").read_text(encoding="utf-8")
self.assertIn("#sha256=", requirement)
self.assertIn("--no-index", runner.calls[0][0])
self.assertEqual(runner.calls[1][0][-2:], ["pip", "check"])
def test_uninstall_requires_explicit_disable_first(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
settings = runtime(root)
settings.configuration_path.write_text(
"PLUGINS = ['netbox_plugin_store', 'netbox_example']\n", encoding="utf-8"
)
runner = FakeRunner()
service = LifecycleService(
settings,
FakeClient(catalog_plugin()),
FakeRepository(),
runner=runner,
version_provider=lambda _: "1.0.0",
)
with self.assertRaises(LifecycleError):
service.execute(LifecycleRequest("example-plugin", "uninstall", dry_run=False))
self.assertEqual(runner.calls, [])
if __name__ == "__main__":
unittest.main()
netbox_plugin/tests/test_netbox_compat.py
+50
View File
@@ -0,0 +1,50 @@
import unittest
import _bootstrap # noqa: F401
from netbox_plugin_store.access import has_store_access
from netbox_plugin_store.navigation import menu
PERMISSION = "netbox_plugin_store.manage_plugin"
class NetBoxUser:
"""NetBox 4.6 users deliberately have no is_staff attribute."""
def __init__(self, *, authenticated=True, superuser=False, permissions=()):
self.is_authenticated = authenticated
self.is_superuser = superuser
self.permissions = set(permissions)
def has_perm(self, permission_name):
return permission_name in self.permissions
class AccessCompatibilityTests(unittest.TestCase):
def test_permission_user_without_is_staff_is_authorized(self):
user = NetBoxUser(permissions=(PERMISSION,))
self.assertTrue(has_store_access(user, PERMISSION))
def test_unauthenticated_and_unprivileged_users_are_rejected(self):
self.assertFalse(
has_store_access(NetBoxUser(authenticated=False, permissions=(PERMISSION,)), PERMISSION)
)
self.assertFalse(has_store_access(NetBoxUser(), PERMISSION))
def test_superuser_is_authorized_without_explicit_permission(self):
self.assertTrue(has_store_access(NetBoxUser(superuser=True), PERMISSION))
class NavigationCompatibilityTests(unittest.TestCase):
def test_menu_uses_permission_not_staff_only(self):
items = [item for _label, group_items in menu.groups for item in group_items]
self.assertEqual(len(items), 3)
for item in items:
self.assertTrue(item.auth_required)
self.assertFalse(item.staff_only)
self.assertEqual(item.permissions, [PERMISSION])
if __name__ == "__main__":
unittest.main()
store/.dockerignore
+12
View File
@@ -0,0 +1,12 @@
.git
.env
.env.*
!.env.example
data/*
!data/.gitkeep
vendor/
tests/
test/
.phpunit.cache/
coverage/
*.log
store/.env.example
+59
View File
@@ -0,0 +1,59 @@
# HTTP / sessions
APP_ENV=production
STORE_PUBLIC_URL=https://plugins.example.com
# Nur aktivieren, wenn direkte Zugriffe per Firewall ausgeschlossen sind und
# ausschließlich ein vertrauenswürdiger Proxy X-Forwarded-For setzt.
STORE_TRUST_PROXY=false
STORE_TRUSTED_PROXY_IPS=127.0.0.1,::1
STORE_COOKIE_SECURE=true
STORE_SESSION_NAME=netbox_plugin_store
# Admin: generate with `php bin/console hash-password 'a-long-random-password'`.
# There are deliberately no default credentials. All three values are required.
STORE_ADMIN_USERNAME=admin
STORE_ADMIN_PASSWORD_HASH=$argon2id$REPLACE_WITH_GENERATED_HASH
STORE_SESSION_SECRET=replace-with-at-least-32-random-characters
STORE_ADMIN_SESSION_TTL=28800
STORE_LOGIN_MAX_ATTEMPTS=5
STORE_LOGIN_WINDOW_SECONDS=900
# Storage: zero-setup JSON or MariaDB via PDO. JSON uses flock + fsync + rename.
STORE_DB_DRIVER=json
STORE_JSON_PATH=./data/store.json
STORE_MARIADB_DSN=mysql:host=mariadb;port=3306;dbname=netbox_store;charset=utf8mb4
STORE_MARIADB_USER=netbox_store
STORE_MARIADB_PASSWORD=replace-me
# Strict outbound policy. Add internal hosts explicitly and only enable private
# networks when the deployment intentionally uses an internal Forgejo.
STORE_ALLOWED_SOURCE_HOSTS=git.mrblake.cc,github.com,api.github.com,*.github.com,*.githubusercontent.com
STORE_ALLOW_PRIVATE_NETWORKS=false
STORE_HTTP_TIMEOUT_SECONDS=20
STORE_MAX_METADATA_BYTES=2097152
STORE_MAX_ARTIFACT_BYTES=536870912
STORE_USER_AGENT=MrBlake-NetBox-Plugin-Store/1.0
# Zero-setup Forgejo source (created on first web/CLI start).
STORE_DEFAULT_PROVIDER=forgejo
STORE_DEFAULT_SOURCE_NAME=MrBlake Forgejo
STORE_DEFAULT_SOURCE_SLUG=mrblake-forgejo
STORE_DEFAULT_BASE_URL=https://git.mrblake.cc
STORE_DEFAULT_API_URL=https://git.mrblake.cc/api/v1
STORE_DEFAULT_OWNER=MrBlake
STORE_DEFAULT_OWNER_KIND=user
STORE_DEFAULT_TOPIC=netbox-plugin
STORE_DEFAULT_TOKEN_ENV=GITEA_TOKEN
GITEA_TOKEN=
GITHUB_TOKEN=
# In-container scheduler. CLI alternative: php bin/console sync --watch.
STORE_SCHEDULER_ENABLED=true
STORE_SYNC_INTERVAL_SECONDS=900
STORE_SYNC_MAX_SECONDS=900
STORE_SYNC_MAX_REQUESTS=2500
STORE_SYNC_MAX_BYTES=1073741824
STORE_SYNC_MAX_REPOSITORIES=2000
STORE_SYNC_MAX_RELEASES=1000
STORE_PAGE_SIZE=12
STORE_API_PAGE_SIZE=50
STORE_API_MAX_PAGE_SIZE=100
store/.gitignore
+7
View File
@@ -0,0 +1,7 @@
/vendor/
/.env
/data/*.json
/data/*.lock
/data/*.tmp-*
!/data/.gitkeep
composer.phar
store/Dockerfile
+27
View File
@@ -0,0 +1,27 @@
FROM composer:2 AS vendor
WORKDIR /app
COPY composer.json composer.lock ./
RUN composer install --no-dev --no-interaction --no-progress --prefer-dist --optimize-autoloader \
--ignore-platform-req=ext-curl --ignore-platform-req=ext-dom --ignore-platform-req=ext-iconv --ignore-platform-req=ext-intl \
--ignore-platform-req=ext-mbstring --ignore-platform-req=ext-pdo
FROM php:8.4-apache
RUN apt-get update \
&& apt-get install -y --no-install-recommends libcurl4-openssl-dev libicu-dev libonig-dev libxml2-dev util-linux \
&& docker-php-ext-install -j"$(nproc)" curl dom intl mbstring opcache pcntl pdo_mysql \
&& a2enmod rewrite headers expires \
&& rm -rf /var/lib/apt/lists/*
COPY deploy/apache-docker.conf /etc/apache2/sites-available/000-default.conf
COPY deploy/php-production.ini /usr/local/etc/php/conf.d/store-production.ini
WORKDIR /var/www/html
COPY --from=vendor /app/vendor ./vendor
COPY . .
RUN mkdir -p data \
&& chown -R www-data:www-data data \
&& chmod +x bin/console docker-entrypoint-store.sh
EXPOSE 80
VOLUME ["/var/www/html/data"]
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD php -r '$c=@file_get_contents("http://127.0.0.1/healthz"); if ($c===false) exit(1);'
ENTRYPOINT ["/var/www/html/docker-entrypoint-store.sh"]
CMD ["apache2-foreground"]
store/README.md
+362
View File
@@ -0,0 +1,362 @@
# NetBox Plugin Store
Der Store ist eine eigenständige PHP-Anwendung für einen kuratierten NetBox-Plugin-Katalog. Er liest Repositories von Forgejo/Gitea (standardmäßig `git.mrblake.cc`) ein, erkennt Plugin-Metadaten, rendert die README der exakt synchronisierten Commit-Revision und veröffentlicht erst nach einer getrennten Admin-Freigabe einen API-v1-Katalog.
Die native Installation mit PHP und Apache ist der primäre Betriebsweg. Docker ist optional.
## Funktionsumfang
- responsive deutschsprachige Store-, Such-, Filter- und Plugin-Detailseiten;
- vollständige Forgejo-/Gitea- und GitHub-Discovery für Benutzer und Organisationen;
- Erkennung über Topic, `netbox-plugin.json|yaml`, `pyproject.toml`, `setup.py` und `PluginConfig`;
- commitgenaue Reads für Manifest, Projektmetadaten, Tree, PluginConfig und README;
- CommonMark-Rendering ohne eingebettetes HTML; relative Links und Bilder werden auf die Raw-URL des Commit-SHA umgeschrieben;
- eigener Admin-Login mit Argon2id, CSRF-Schutz, Session-Härtung und persistentem Login-Rate-Limit;
- Freigabe/Ablehnung von Quellen, Plugins und jedem Release-Artefakt; Admin-Metadatenkorrekturen und Audit-Log;
- tatsächlicher Download jedes öffentlichen Wheel-Artefakts mit SHA-256 und Größe; sicherheitsrelevante Änderungen setzen die Freigabe zurück;
- öffentlicher, freigegebener JSON-Katalog unter `/api/v1/`;
- atomar geschriebener JSON-Datastore als Zero-Setup-Standard oder optional MariaDB über PDO;
- einmaliger Sync sowie signalverträglicher Watch-Modus ohne Cron/Celery;
- pro Source eine prozessübergreifende exklusive Sync-Lease sowie aggregierte Limits für Laufzeit, Requests, Bytes, Repositories und Releases.
Wichtig: Ein eingelesenes und freigegebenes Repository ist noch nicht automatisch installierbar. API v1 veröffentlicht ausschließlich separat freigegebene `.whl`-Artefakte. Fehlt ein Wheel, bleibt das Plugin als Katalogeintrag sichtbar und `latest_version` ist `null`; die UI zeigt „Kein installierbares Release“.
## 1. Native Installation (empfohlen)
### Voraussetzungen
- PHP 8.3 oder neuer (getestet mit PHP 8.4);
- Apache 2.4 mit `mod_rewrite` und `mod_headers`;
- Composer 2;
- PHP-Erweiterungen: `curl`, `dom`, `iconv`, `intl`, `json`, `mbstring`, `pdo` und für MariaDB zusätzlich `pdo_mysql`; für einen graceful beendbaren dauerhaften `--watch`-Prozess wird `pcntl` empfohlen;
- Schreibzugriff des Apache-/Sync-Benutzers auf `store/data`.
Beispiel für Debian/Ubuntu (Paketnamen können je Distribution/PHP-Repository abweichen):
```bash
sudo apt update
sudo apt install apache2 libapache2-mod-php php-cli php-curl php-xml php-intl php-mbstring php-mysql composer
sudo a2enmod rewrite headers
```
### Anwendung installieren
```bash
sudo mkdir -p /opt/netbox-plugin-store
sudo chown "$USER":"$USER" /opt/netbox-plugin-store
git clone <REPOSITORY-URL> /opt/netbox-plugin-store
cd /opt/netbox-plugin-store/store
composer install --no-dev --no-interaction --prefer-dist --classmap-authoritative
cp .env.example .env
mkdir -p data
sudo chown -R www-data:www-data data
sudo chmod 750 data
```
Der JSON-Pfad in `.env.example` ist relativ zum `store/`-Verzeichnis und funktioniert nativ ohne Änderung:
```dotenv
STORE_DB_DRIVER=json
STORE_JSON_PATH=./data/store.json
```
Der Datastore verwendet einen prozessübergreifenden `flock`, schreibt in eine temporäre Datei, synchronisiert sie und ersetzt anschließend die alte Datei atomar. Eine separate nicht-blockierende Lease-Datei hält zusätzlich jeden Source-Sync für seine gesamte Laufzeit exklusiv; ein Prozessabbruch gibt die Kernel-Sperre frei. Verwende ein lokales Dateisystem; geteilte/NFS-Dateisysteme bieten nicht immer die benötigten Lock-/Rename-Garantien. Für mehrere Web-Hosts ist MariaDB die bessere Wahl; dort übernimmt `GET_LOCK` die Sync-Lease.
### Admin-Zugang sicher konfigurieren
Es gibt absichtlich keine Standard-Zugangsdaten. Erzeuge einen langen Zufallswert als Session-Secret und einen Argon2id-Hash:
```bash
php -r 'echo bin2hex(random_bytes(32)), PHP_EOL;'
php bin/console hash-password 'ein-langes-zufaelliges-passwort'
```
Trage Benutzername, ausgegebenen Hash und Secret vollständig in `.env` ein:
```dotenv
STORE_ADMIN_USERNAME=admin
STORE_ADMIN_PASSWORD_HASH=$argon2id$...
STORE_SESSION_SECRET=<mindestens-32-zeichen>
```
Sind alle drei Werte leer, bleibt `/admin` deaktiviert. Ist nur ein Teil gesetzt, bricht die Anwendung fail-closed mit einer Konfigurationsmeldung ab. Vermeide Passwörter in der Shell-History; alternativ liest `hash-password` den Wert aus `STORE_ADMIN_PASSWORD_TO_HASH`.
Setze außerdem die öffentliche URL. Sichere Cookies richten sich standardmäßig nach ihrem Protokoll:
```dotenv
APP_ENV=production
STORE_PUBLIC_URL=https://plugins.example.com
# Optionaler expliziter Override:
STORE_COOKIE_SECURE=true
```
Ein Production-Quickstart über `http://localhost` funktioniert mit `STORE_PUBLIC_URL=http://localhost:3000` und `STORE_COOKIE_SECURE=false`; für einen öffentlichen Betrieb ist HTTPS erforderlich.
### Apache konfigurieren
Nur `store/public` darf DocumentRoot sein. Passe Pfad und Servernamen in [deploy/apache-vhost.conf.example](deploy/apache-vhost.conf.example) an:
```bash
sudo cp deploy/apache-vhost.conf.example /etc/apache2/sites-available/netbox-plugin-store.conf
sudo editor /etc/apache2/sites-available/netbox-plugin-store.conf
sudo a2ensite netbox-plugin-store
sudo apache2ctl configtest
sudo systemctl reload apache2
```
Die Beispielkonfiguration erlaubt `.htaccess` ausschließlich im Store-DocumentRoot. Alternativ können Rewrite- und Header-Regeln aus `public/.htaccess` direkt in den vHost übernommen und `AllowOverride None` gesetzt werden. Für Produktion HTTPS direkt mit Apache `mod_ssl`/ACME konfigurieren und `STORE_PUBLIC_URL` auf `https://…` setzen; ein Reverse Proxy ist nicht erforderlich.
Übernimm für den produktiven PHP-SAPI außerdem die sicherheitsrelevanten Werte aus `deploy/php-production.ini` (insbesondere `display_errors=Off`, `log_errors=On` und `expose_php=Off`) in deine PHP-Konfiguration und lade Apache neu. Der Front Controller deaktiviert die Fehlerausgabe zusätzlich selbst.
### Bootstrap und erster Sync
```bash
cd /opt/netbox-plugin-store/store
sudo -u www-data php bin/console bootstrap
sudo -u www-data php bin/console sync
```
Wenn noch keine Quelle existiert, legt `bootstrap` (und standardmäßig auch `sync`) aus den `STORE_DEFAULT_*`-Werten eine aktive, freigegebene Forgejo-Quelle an. Standardwerte:
- Provider `forgejo`;
- Basis `https://git.mrblake.cc`;
- API `https://git.mrblake.cc/api/v1`;
- Benutzer `MrBlake` – entsprechend `/api/v1/users/MrBlake/repos`.
Mit `--no-bootstrap` wird die automatische Anlage unterdrückt. Einzelne Quellen lassen sich mit `--source=slug` synchronisieren; `--fail-fast` beendet den Lauf beim ersten Repositoryfehler.
Jeder Source-Lauf ist mit konservativen, konfigurierbaren Gesamtbudgets begrenzt. Eine Überschreitung beendet ihn fail-closed als `failed`; neue Freigaben entstehen dabei nicht. Die Defaults stehen auch in `.env.example`:
```dotenv
STORE_SYNC_MAX_SECONDS=900
STORE_SYNC_MAX_REQUESTS=2500
STORE_SYNC_MAX_BYTES=1073741824
STORE_SYNC_MAX_REPOSITORIES=2000
STORE_SYNC_MAX_RELEASES=1000
```
Die Limits zählen Redirects und Retries als weitere Requests sowie Metadaten und Artefakte gemeinsam gegen das Byte-Budget. Repository- und Release-Zähler werden nach jeder validierten Provider-API-Seite vor deren Übernahme in den Akkumulator geprüft; dadurch kann höchstens die gerade empfangene Seite zusätzlich im Speicher liegen. `STORE_SYNC_MAX_RELEASES` ist zugleich auf die API-v1-Grenze von 1.000 begrenzt. Ein zweiter Lauf derselben Source wird unabhängig vom Alter eines sichtbaren Run-Eintrags abgewiesen; nach einem Prozessabbruch wird der verwaiste Eintrag erst nach erfolgreichem Erwerb der exklusiven Lease geschlossen.
### Regelmäßiger Sync mit systemd
Die mitgelieferte Unit ist ein `oneshot`-Dienst mit einem 15-Minuten-Timer. Passe gegebenenfalls `/opt/netbox-plugin-store` und `www-data` an:
```bash
sudo cp deploy/netbox-plugin-store-sync.service /etc/systemd/system/
sudo cp deploy/netbox-plugin-store-sync.timer /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now netbox-plugin-store-sync.timer
systemctl list-timers netbox-plugin-store-sync.timer
journalctl -u netbox-plugin-store-sync.service
```
Alternativ kann ein dauerhafter Prozess verwendet werden:
```bash
sudo -u www-data php bin/console sync --watch --interval=900
```
Oder per Cron (keine überlappenden Läufe; der Store blockiert parallele Syncs zusätzlich):
```cron
*/15 * * * * cd /opt/netbox-plugin-store/store && /usr/bin/php bin/console sync >>/var/log/netbox-plugin-store-sync.log 2>&1
```
## 2. MariaDB statt JSON (optional)
Der MariaDB-Adapter nutzt dieselbe Repository-/State-Schnittstelle. Er legt beim ersten Start automatisch eine einzelne, per InnoDB-Transaktion und `SELECT … FOR UPDATE` geschützte State-Zeile an. Das äquivalente Schema liegt unter `config/schema.sql`.
```dotenv
STORE_DB_DRIVER=mariadb
STORE_MARIADB_DSN=mysql:host=127.0.0.1;port=3306;dbname=netbox_store;charset=utf8mb4
STORE_MARIADB_USER=netbox_store
STORE_MARIADB_PASSWORD=<starkes-passwort>
```
Der Datenbankbenutzer benötigt `CREATE`, `SELECT`, `INSERT` und `UPDATE` auf der Store-Datenbank. Sichere die Datenbank wie andere Produktionsdaten regelmäßig.
## 3. Quellen und automatische Erkennung
Neue Quellen werden in `/admin` zunächst als `pending` angelegt und müssen per POST/CSRF freigegeben werden. Verfügbare Provider:
- `forgejo` für Forgejo und Gitea;
- `github` für GitHub-User oder -Organisationen.
Tokens werden nur aus dem in der Source hinterlegten Namen einer Umgebungsvariable gelesen, beispielsweise `GITEA_TOKEN` oder `GITHUB_TOKEN`; Tokenwerte werden nie im Datastore gespeichert. Ein Authorization-Header wird nur gesendet, wenn die Ziel-Origin exakt der `apiUrl`-Origin entspricht, und nach jedem Redirect neu bewertet.
Private GitHub-Repositories dürfen mit einem entsprechend berechtigten Token für Metadaten und README eingelesen werden. Ihre Release-Assets werden in v1 bewusst fail-closed übersprungen: Der Store betreibt keinen Artefakt-Cache, und der Host-Agent besitzt keine GitHub-Credentials für einen späteren Download. Der Sync zeigt dafür einen Moderationshinweis und zieht eventuell ältere installierbare Releases dieses Repositorys zurück. Installierbar sind von GitHub ausschließlich öffentliche `.whl`-Assets über deren `browser_download_url`; Provider-Tokens werden niemals an Asset-Hosts weitergereicht.
Die zuverlässigste Discovery gelingt mit einem Manifest im Repository-Root:
```json
{
"name": "Mein NetBox Plugin",
"summary": "Kurze Beschreibung",
"package_name": "netbox-mein-plugin",
"import_name": "netbox_mein_plugin",
"version": "1.2.3",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8"
}
```
Alternativ werden PEP-621/Poetry/Setuptools-Metadaten, Entry Points, `setup.py` und `PluginConfig` ausgewertet. Dynamische Setuptools-Versionen über `tool.setuptools.dynamic.version.attr` werden aus dem am gleichen Commit gepinnten Modul gelesen. Bleiben Paket, Top-Level-Import oder Kompatibilitätsgrenzen unbekannt, blockiert die Admin-Seite die Freigabe mit einer klaren Meldung. Dort können die Felder korrigiert werden; diese Overrides bleiben bei späteren Syncs erhalten.
## 4. Ein Wheel veröffentlichen und freigeben
Die aktuell eingelesenen MrBlake-Repositories besitzen überwiegend keine Forgejo-Release-Wheels. Sie können nach Plugin-Freigabe im Store erscheinen, sind aber nicht automatisch installierbar.
Empfohlener Release-Ablauf im Plugin-Repository:
```bash
python -m pip install --upgrade build
python -m build
```
1. Einen unveränderlichen Git-Tag setzen.
2. In Forgejo einen Release für diesen Tag erstellen.
3. Das erzeugte `dist/*.whl` als Release-Asset hochladen.
4. Den Store synchronisieren.
5. Im Admin zunächst fehlende Plugin-Metadaten korrigieren/freigeben und anschließend genau das neue Release-Artefakt freigeben.
Beim Sync lädt der Store das Wheel tatsächlich, berechnet SHA-256 und `artifact_size` und speichert den Commit-SHA. Neue Releases erben niemals die Plugin-Freigabe. Ändern sich URL, Hash, Größe, Commit, Paket-/Importname oder Kompatibilitätsgrenzen, werden Plugin bzw. Release wieder `pending`. Entfernte Upstream-Releases werden als zurückgezogen markiert und verschwinden sofort aus dem öffentlichen Katalog.
Ein Git-Tag oder Quellcode-Archiv allein erzeugt kein installierbares Release. Der Store erzeugt keine Ersatzartefakte; NetBox-Client und Host-Agent installieren ausschließlich separat veröffentlichte und freigegebene Wheels.
## 5. Moderationsablauf
Unter `/admin` stehen ausschließlich POST-Aktionen mit CSRF-Schutz bereit:
1. Source prüfen und freigeben;
2. Source synchronisieren;
3. Plugin-Metadaten prüfen/korrigieren und Plugin freigeben;
4. Wheel-Hash, Größe, Version und Kompatibilität prüfen und Release separat freigeben.
Der Store erzeugt beim Release-Approval einen kanonischen `approved_payload_sha256`. Die öffentliche API liefert ein Artefakt nur, wenn der aktuelle Payload weiterhin exakt zu diesem Approval-Hash passt. Das ist unabhängig von den sichtbaren Statusfeldern eine zusätzliche Defense-in-Depth-Prüfung.
## 6. API v1
Alle Routen tolerieren einen abschließenden Slash:
```text
GET /api/v1/plugins/
GET /api/v1/plugins/:slug/
GET /api/v1/plugins/:slug/releases/:version/
```
Die Liste ist paginiert (`page`, `page_size`) und unterstützt `q`, `source` und `netbox_version`. Sie enthält ausschließlich aktive, vollständig validierte und freigegebene Source-/Plugin-Ketten.
Plugin-Detail (gekürzt auf ein Release):
```json
{
"api_version": "v1",
"slug": "demo-plugin",
"name": "Demo Plugin",
"summary": "Kurzbeschreibung",
"description": "Beschreibung",
"repository_url": "https://git.example.com/team/demo",
"latest_version": "1.2.3",
"package_name": "netbox-demo",
"import_name": "netbox_demo",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
"approved": true,
"status": "approved",
"releases": [
{
"version": "1.2.3",
"download_url": "https://git.example.com/assets/netbox_demo-1.2.3-py3-none-any.whl",
"sha256": "<64-hex>",
"artifact_size": 12345,
"commit_sha": "<git-sha>",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
"published_at": "2026-08-20T10:00:00+00:00",
"approved": true,
"status": "approved",
"immutable": true,
"approved_payload_sha256": "<64-hex>"
}
]
}
```
Die Release-Detailroute liefert exakt das flache Release-Objekt aus `releases[]`, ohne interne Admin-ID. Ohne freigegebenes Wheel ist `releases` leer und `latest_version` `null`. Ein Plugin-Detail enthält höchstens 1.000 Releases. `commit_sha` ist entweder leer oder exakt 40 kleingeschriebene Hex-Zeichen. Vor der Freigabe validiert der Store außerdem den Wheel-Dateinamen und verlangt, dass dessen normalisierte Distribution und Version exakt zu `package_name` und `version` passen; damit scheitert ein Artefakt nicht erst später im Host-Agent.
## 7. Netzwerk- und SSRF-Schutz
Ausgehende Ziele müssen credential-freies HTTPS verwenden und in `STORE_ALLOWED_SOURCE_HOSTS` stehen. Jeder DNS-A/AAAA-Wert wird geprüft; Loopback, private, Link-Local- und reservierte Adressen sind standardmäßig gesperrt. cURL wird an die geprüfte Adresse gepinnt, Redirects werden einzeln erneut aufgelöst/geprüft, und Provider-Tokens bleiben auf der exakten API-Origin.
```dotenv
STORE_ALLOWED_SOURCE_HOSTS=git.mrblake.cc,github.com,api.github.com,*.github.com,*.githubusercontent.com
STORE_ALLOW_PRIVATE_NETWORKS=false
```
Für ein bewusst internes Forgejo muss dessen Host explizit allowgelistet und `STORE_ALLOW_PRIVATE_NETWORKS=true` gesetzt werden. Diese Ausnahme erweitert den SSRF-Radius und sollte nur in einem kontrollierten Netz verwendet werden.
`STORE_TRUST_PROXY=true` darf nur genutzt werden, wenn direkter Zugriff auf Apache per Firewall ausgeschlossen ist. Zusätzlich werden Forwarded-IPs nur von exakt gelisteten Peers akzeptiert:
```dotenv
STORE_TRUST_PROXY=true
STORE_TRUSTED_PROXY_IPS=10.20.0.10,2001:db8::10
```
## 8. Optionale Docker-Installation
Das Image enthält Apache und PHP 8.4; ein Proxy ist nicht erforderlich. Der Scheduler läuft im Container als `www-data`, damit Webprozess und Watcher dieselben geschützten JSON-Dateien lesen können.
```bash
cd store
docker build -t netbox-plugin-store .
docker volume create netbox-plugin-store-data
docker run -d --name netbox-plugin-store \
-p 3000:80 \
--env-file .env \
-e STORE_PUBLIC_URL=http://localhost:3000 \
-e STORE_COOKIE_SECURE=false \
-e STORE_JSON_PATH=/var/www/html/data/store.json \
-e STORE_SCHEDULER_ENABLED=true \
-v netbox-plugin-store-data:/var/www/html/data \
netbox-plugin-store
```
Der Healthcheck ist `GET /healthz`. Für Produktion HTTPS konfigurieren und `STORE_PUBLIC_URL`/Cookie-Einstellung entsprechend setzen.
## 9. Update, Backup und Diagnose
Native Aktualisierung:
```bash
cd /opt/netbox-plugin-store
git pull --ff-only
cd store
composer install --no-dev --no-interaction --prefer-dist --classmap-authoritative
sudo systemctl reload apache2
sudo -u www-data php bin/console sync
```
Vor Updates den JSON-Datastore oder MariaDB sichern. Der JSON-Store darf nur kopiert werden, während kein Schreibvorgang läuft; am einfachsten Apache/Timer kurz stoppen oder ein konsistentes dateisystemseitiges Backup-Verfahren verwenden.
Nützliche Prüfungen:
```bash
php -l public/index.php
composer validate --strict
composer test
curl -fsS http://127.0.0.1/healthz
curl -fsS http://127.0.0.1/api/v1/plugins/
```
Sync-Ergebnisse stehen im Admin-Dashboard und werden als JSON vom CLI ausgegeben. Einzelne Repository-Fehler führen zu einem `partial`-Lauf und archivieren den letzten bekannten Eintrag nicht; eine erfolgreich festgestellte Nicht-Kandidatur oder ein erfolgreich festgestellter Release-Rückzug wird dagegen fail-closed aus dem öffentlichen Katalog entfernt.
## Tests
Der zero-dependency Runner nutzt PHPs Laufzeit direkt; die Anwendungsabhängigkeiten müssen per Composer installiert sein:
```bash
composer install
composer test
```
Abgedeckt sind unter anderem API-Feldvertrag und 1.000er-Grenze, Approval-Payload, Wheel-Distribution/-Version/-Dateiname, 40-stellige Commit-SHAs, strikte Versions-/Metadatenvalidierung, README-Sanitizing und Commit-Pinning, Origin-Token-Isolation, private GitHub-Assets, private-IP-SSRF, atomare JSON-Grenzen und exklusive Leases, aggregierte Sync-Budgets, Trusted-Proxy-Auswertung, sticky Admin-Overrides, Rehash bei ersetzten Assets, Withdrawal und Nicht-Kandidaten-Archivierung.
store/bin/console
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env php
<?php
declare(strict_types=1);
use NetBoxStore\Config;
use NetBoxStore\Database\RepositoryFactory;
use NetBoxStore\Security\Auth;
use NetBoxStore\Security\HttpClient;
use NetBoxStore\Security\SsrfGuard;
use NetBoxStore\Sync\SyncService;
$root = dirname(__DIR__);
require $root . '/vendor/autoload.php';
$command = $argv[1] ?? '';
if ($command === 'hash-password') {
try {
$password = $argv[2] ?? getenv('STORE_ADMIN_PASSWORD_TO_HASH') ?: '';
if ($password === '') {
throw new RuntimeException("Usage: php bin/console hash-password 'a-long-random-password'");
}
fwrite(STDOUT, Auth::passwordHash($password) . PHP_EOL);
exit(0);
} catch (Throwable $exception) {
fwrite(STDERR, $exception->getMessage() . PHP_EOL);
exit(1);
}
}
if (!in_array($command, ['sync', 'bootstrap'], true)) {
fwrite(STDERR, "Usage: php bin/console <hash-password|bootstrap|sync> [--watch] [--source=slug] [--interval=900] [--fail-fast]" . PHP_EOL);
exit(1);
}
try {
$config = Config::load($root);
$repository = RepositoryFactory::create($config);
$repository->initialize();
$guard = new SsrfGuard($config);
$http = new HttpClient($config, $guard);
$service = new SyncService($repository, $config, $http, $guard);
if (!in_array('--no-bootstrap', $argv, true)) {
$bootstrapped = $service->ensureDefaultSource();
if ($bootstrapped['created']) {
fwrite(STDOUT, 'Default source created: ' . $bootstrapped['source']['slug'] . PHP_EOL);
}
}
if ($command === 'bootstrap') {
exit(0);
}
$watch = in_array('--watch', $argv, true);
$failFast = in_array('--fail-fast', $argv, true);
$sources = [];
$interval = $config->scheduler['interval'];
foreach ($argv as $argument) {
if (str_starts_with($argument, '--source=')) {
$sources[] = substr($argument, 9);
} elseif (str_starts_with($argument, '--interval=')) {
$interval = max(30, min(604_800, (int) substr($argument, 11)));
}
}
$stopped = false;
if (function_exists('pcntl_async_signals')) {
pcntl_async_signals(true);
pcntl_signal(SIGINT, static function () use (&$stopped): void { $stopped = true; });
pcntl_signal(SIGTERM, static function () use (&$stopped): void { $stopped = true; });
}
do {
try {
$runs = $service->syncAll($sources === [] ? null : $sources, $failFast, $watch ? 'scheduled' : 'command');
fwrite(STDOUT, json_encode($runs, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE | JSON_THROW_ON_ERROR) . PHP_EOL);
if (!$watch && array_filter($runs, static fn (array $run): bool => $run['status'] === 'failed')) {
exit(1);
}
} catch (Throwable $exception) {
fwrite(STDERR, 'Sync failed: ' . $exception->getMessage() . PHP_EOL);
if (!$watch) {
exit(1);
}
}
if ($watch && !$stopped) {
for ($second = 0; $second < $interval && !$stopped; $second++) {
sleep(1);
}
}
} while ($watch && !$stopped);
exit(0);
} catch (Throwable $exception) {
fwrite(STDERR, $exception->getMessage() . PHP_EOL);
exit(1);
}
store/composer.json
+32
View File
@@ -0,0 +1,32 @@
{
"name": "mrblake/netbox-plugin-store",
"description": "Curated, approval-gated NetBox plugin catalogue",
"type": "project",
"license": "proprietary",
"require": {
"php": ">=8.3 <9.0",
"ext-curl": "*",
"ext-dom": "*",
"ext-iconv": "*",
"ext-intl": "*",
"ext-json": "*",
"ext-mbstring": "*",
"ext-pdo": "*",
"devium/toml": "^1.1",
"league/commonmark": "^2.7",
"symfony/yaml": "^7.2"
},
"autoload": {
"psr-4": {
"NetBoxStore\\": "src/"
}
},
"scripts": {
"test": "php tests/run.php"
},
"config": {
"allow-plugins": {},
"optimize-autoloader": true,
"sort-packages": true
}
}
store/composer.lock
Generated
+1023
View File
File diff suppressed because it is too large Load Diff
store/config/schema.sql
+10
View File
@@ -0,0 +1,10 @@
CREATE TABLE IF NOT EXISTS store_state (
id TINYINT UNSIGNED NOT NULL PRIMARY KEY,
document LONGTEXT NOT NULL,
updated_at DATETIME(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3)
ON UPDATE CURRENT_TIMESTAMP(3),
CONSTRAINT store_state_document_json CHECK (JSON_VALID(document))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
INSERT IGNORE INTO store_state (id, document)
VALUES (1, '{"schemaVersion":1,"sources":[],"plugins":[],"releases":[],"syncRuns":[],"auditLog":[],"authAttempts":[],"meta":{}}');
store/data/.gitkeep
+1
View File
@@ -0,0 +1 @@
store/deploy/apache-docker.conf
+16
View File
@@ -0,0 +1,16 @@
<VirtualHost *:80>
ServerName localhost
DocumentRoot /var/www/html/public
<Directory /var/www/html/public>
Options -Indexes
AllowOverride All
Require all granted
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
ServerTokens Prod
ServerSignature Off
store/deploy/apache-vhost.conf.example
+19
View File
@@ -0,0 +1,19 @@
<VirtualHost *:80>
ServerName plugins.example.com
DocumentRoot /opt/netbox-plugin-store/store/public
<Directory /opt/netbox-plugin-store/store/public>
Options -Indexes
AllowOverride All
Require all granted
</Directory>
ErrorLog ${APACHE_LOG_DIR}/netbox-plugin-store-error.log
CustomLog ${APACHE_LOG_DIR}/netbox-plugin-store-access.log combined
</VirtualHost>
ServerTokens Prod
ServerSignature Off
# Für Produktion HTTPS direkt mit mod_ssl konfigurieren (oder den vHost durch
# certbot ergänzen lassen). STORE_PUBLIC_URL muss anschließend https:// nutzen.
store/deploy/netbox-plugin-store-sync.service
+19
View File
@@ -0,0 +1,19 @@
[Unit]
Description=NetBox Plugin Store synchronization
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
User=www-data
Group=www-data
WorkingDirectory=/opt/netbox-plugin-store/store
ExecStart=/usr/bin/php /opt/netbox-plugin-store/store/bin/console sync
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
ProtectSystem=strict
ReadWritePaths=/opt/netbox-plugin-store/store/data
[Install]
WantedBy=multi-user.target
store/deploy/netbox-plugin-store-sync.timer
+12
View File
@@ -0,0 +1,12 @@
[Unit]
Description=Synchronize NetBox Plugin Store every 15 minutes
[Timer]
OnBootSec=2min
OnUnitActiveSec=15min
AccuracySec=30s
Persistent=true
Unit=netbox-plugin-store-sync.service
[Install]
WantedBy=timers.target
store/deploy/php-production.ini
+6
View File
@@ -0,0 +1,6 @@
display_errors=Off
display_startup_errors=Off
log_errors=On
expose_php=Off
session.cookie_httponly=1
session.use_strict_mode=1
store/docker-entrypoint-store.sh
+15
View File
@@ -0,0 +1,15 @@
#!/bin/sh
set -eu
mkdir -p /var/www/html/data
chown -R www-data:www-data /var/www/html/data
if [ "${STORE_SCHEDULER_ENABLED:-false}" = "true" ]; then
interval="${STORE_SYNC_INTERVAL_SECONDS:-900}"
case "$interval" in
*[!0-9]*) interval=900 ;;
esac
runuser -u www-data -- php /var/www/html/bin/console sync --watch --interval="$interval" &
fi
exec "$@"
store/public/.htaccess
+16
View File
@@ -0,0 +1,16 @@
Options -Indexes
DirectoryIndex index.php
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^ index.php [QSA,L]
</IfModule>
<IfModule mod_headers.c>
Header always set X-Content-Type-Options "nosniff"
Header always set X-Frame-Options "DENY"
Header always set Referrer-Policy "strict-origin-when-cross-origin"
Header always set Permissions-Policy "camera=(), microphone=(), geolocation=()"
</IfModule>
store/public/assets/app.css
+274
View File
@@ -0,0 +1,274 @@
:root {
--ink: #102a43;
--ink-soft: #334e68;
--muted: #627d98;
--line: #d9e2ec;
--surface: #fff;
--surface-soft: #f4f7fa;
--navy: #102a43;
--blue: #1769e0;
--blue-dark: #1254b5;
--cyan: #21b6c7;
--green: #147d64;
--green-bg: #e6f6f1;
--amber: #9a6700;
--amber-bg: #fff5d6;
--red: #b42318;
--red-bg: #ffebe9;
--shadow: 0 16px 40px rgba(16, 42, 67, .09);
--radius: 18px;
font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
color: var(--ink);
background: #f8fafc;
font-synthesis: none;
}
* { box-sizing: border-box; }
html { scroll-behavior: smooth; }
body { margin: 0; min-height: 100vh; background: #f8fafc; color: var(--ink); line-height: 1.55; }
a { color: var(--blue); text-decoration: none; }
a:hover { color: var(--blue-dark); }
button, input, select, textarea { font: inherit; }
button { cursor: pointer; }
button:disabled { cursor: not-allowed; opacity: .48; }
code { border-radius: 6px; background: #edf2f7; color: #243b53; padding: .12rem .35rem; font: .88em ui-monospace, SFMono-Regular, Consolas, monospace; overflow-wrap: anywhere; }
.shell { width: min(1180px, calc(100% - 40px)); margin-inline: auto; }
.skip-link { position: fixed; top: -100px; left: 16px; z-index: 100; padding: 10px 16px; background: #fff; border-radius: 8px; }
.skip-link:focus { top: 12px; }
.site-header { position: sticky; top: 0; z-index: 20; border-bottom: 1px solid rgba(217,226,236,.9); background: rgba(255,255,255,.92); backdrop-filter: blur(14px); }
.header-inner { min-height: 76px; display: flex; align-items: center; justify-content: space-between; gap: 24px; }
.brand { display: inline-flex; align-items: center; gap: 11px; color: var(--ink); }
.brand:hover { color: var(--ink); }
.brand-mark { display: grid; place-items: center; width: 38px; height: 38px; border-radius: 11px; color: #fff; background: linear-gradient(145deg, var(--blue), var(--cyan)); font-weight: 800; box-shadow: 0 7px 18px rgba(23,105,224,.24); }
.brand span:last-child { display: grid; line-height: 1.05; }
.brand small { margin-top: 4px; color: var(--muted); font-size: .68rem; font-weight: 700; letter-spacing: .12em; text-transform: uppercase; }
.main-nav { display: flex; align-items: center; gap: 5px; }
.main-nav a { padding: 9px 13px; border-radius: 9px; color: var(--ink-soft); font-size: .92rem; font-weight: 650; }
.main-nav a:hover, .main-nav a.active { color: var(--blue); background: #edf5ff; }
.hero { overflow: hidden; color: #fff; background: radial-gradient(circle at 80% 10%, rgba(33,182,199,.28), transparent 34%), linear-gradient(130deg, #0b2035, #123c64 62%, #15566f); }
.hero-grid { min-height: 435px; display: grid; grid-template-columns: minmax(0, 1.8fr) minmax(240px, .55fr); align-items: center; gap: 70px; padding-block: 70px; }
.eyebrow { display: block; margin-bottom: 12px; color: #50b9ff; font-size: .72rem; font-weight: 800; letter-spacing: .16em; text-transform: uppercase; }
.hero h1 { max-width: 760px; margin: 0; font-size: clamp(2.6rem, 6vw, 5.2rem); line-height: .98; letter-spacing: -.055em; }
.hero h1 span { color: #6bd7e1; }
.hero-copy { max-width: 730px; margin: 27px 0 0; color: #c8d9e8; font-size: clamp(1rem, 1.5vw, 1.18rem); }
.hero-stat { position: relative; display: grid; padding: 34px; border: 1px solid rgba(255,255,255,.16); border-radius: 22px; background: rgba(255,255,255,.08); box-shadow: inset 0 1px 0 rgba(255,255,255,.12); }
.hero-stat strong { font-size: 4.4rem; line-height: 1; letter-spacing: -.06em; }
.hero-stat span { margin-top: 8px; font-weight: 700; }
.hero-stat small { margin-top: 22px; color: #b7d1e5; }
.catalog-section { padding-block: 0 80px; }
.filter-panel { position: relative; z-index: 2; display: grid; grid-template-columns: 2fr 1fr 1fr auto auto; align-items: end; gap: 12px; margin-top: -35px; padding: 20px; border: 1px solid var(--line); border-radius: 16px; background: #fff; box-shadow: var(--shadow); }
label { display: grid; gap: 6px; color: var(--ink-soft); font-size: .82rem; font-weight: 700; }
input, select, textarea { width: 100%; min-height: 43px; padding: 9px 12px; border: 1px solid #bcccdc; border-radius: 9px; outline: none; background: #fff; color: var(--ink); transition: border-color .15s, box-shadow .15s; }
textarea { resize: vertical; }
input:focus, select:focus, textarea:focus { border-color: var(--blue); box-shadow: 0 0 0 3px rgba(23,105,224,.12); }
.button { min-height: 42px; display: inline-flex; align-items: center; justify-content: center; gap: 7px; padding: 9px 15px; border: 1px solid transparent; border-radius: 9px; font-weight: 750; font-size: .88rem; white-space: nowrap; }
.button.primary { border-color: var(--blue); color: #fff; background: var(--blue); }
.button.primary:hover { border-color: var(--blue-dark); color: #fff; background: var(--blue-dark); }
.button.secondary { border-color: #b6c6d8; color: var(--ink); background: #fff; }
.button.quiet { color: var(--ink-soft); background: #edf2f7; }
.button.danger { color: var(--red); background: var(--red-bg); }
.button.wide { width: 100%; }
.notice { margin: 20px 0; padding: 13px 16px; border: 1px solid; border-radius: 10px; font-weight: 650; }
.notice.error { border-color: #f3b7b2; color: var(--red); background: var(--red-bg); }
.notice.success { border-color: #9dd8c8; color: var(--green); background: var(--green-bg); }
.catalog-heading { display: flex; align-items: end; justify-content: space-between; gap: 20px; margin: 55px 0 22px; }
.catalog-heading h2 { margin: 0; font-size: 2rem; letter-spacing: -.035em; }
.catalog-heading .eyebrow { margin-bottom: 5px; color: var(--blue); }
.muted { color: var(--muted); }
.plugin-grid { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 18px; }
.plugin-card { min-width: 0; display: flex; flex-direction: column; min-height: 305px; padding: 23px; border: 1px solid var(--line); border-radius: var(--radius); background: #fff; box-shadow: 0 3px 12px rgba(16,42,67,.035); transition: transform .18s, box-shadow .18s, border-color .18s; }
.plugin-card:hover { transform: translateY(-3px); border-color: #b6cce1; box-shadow: var(--shadow); }
.card-topline, .release-tags, .summary-status { display: flex; align-items: center; flex-wrap: wrap; gap: 7px; }
.card-topline { justify-content: space-between; }
.provider-pill, .status-pill { display: inline-flex; align-items: center; width: fit-content; padding: 4px 8px; border-radius: 999px; font-size: .66rem; font-weight: 800; letter-spacing: .04em; text-transform: uppercase; }
.provider-pill { color: #526d82; background: #edf2f7; }
.status-pill.success { color: var(--green); background: var(--green-bg); }
.status-pill.warning { color: var(--amber); background: var(--amber-bg); }
.status-pill.danger { color: var(--red); background: var(--red-bg); }
.status-pill.neutral { color: #526d82; background: #edf2f7; }
.plugin-card h3 { margin: 22px 0 9px; font-size: 1.3rem; letter-spacing: -.025em; }
.plugin-card h3 a { color: var(--ink); }
.plugin-card > p { flex: 1; margin: 0; color: var(--muted); font-size: .92rem; }
.card-meta { display: flex; gap: 24px; margin: 22px 0 18px; }
.card-meta div { display: grid; }
.card-meta dt { color: var(--muted); font-size: .67rem; font-weight: 800; letter-spacing: .09em; text-transform: uppercase; }
.card-meta dd { margin: 3px 0 0; color: var(--ink-soft); font-size: .86rem; font-weight: 700; }
.card-footer { display: flex; align-items: center; justify-content: space-between; gap: 12px; padding-top: 16px; border-top: 1px solid #e9eff5; }
.card-footer code { max-width: 62%; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.arrow-link { font-size: .85rem; font-weight: 750; }
.empty-state, .error-page { display: grid; justify-items: center; padding: 80px 30px; border: 1px dashed #bcccdc; border-radius: var(--radius); text-align: center; background: #fff; }
.empty-state h2, .error-page h1 { margin: 12px 0 4px; }
.empty-state p, .error-page p { max-width: 600px; color: var(--muted); }
.empty-icon { font-size: 3rem; color: var(--blue); }
.pagination { display: flex; justify-content: center; align-items: center; gap: 20px; margin-top: 38px; }
.pagination a { padding: 8px 12px; border-radius: 8px; background: #fff; font-weight: 700; }
.pagination span { color: var(--muted); font-size: .88rem; }
.detail-hero { padding: 56px 0 62px; color: #fff; background: linear-gradient(130deg, #0c253d, #174f73); }
.back-link { display: inline-block; margin-bottom: 36px; color: #a8d8fb; font-size: .86rem; font-weight: 700; }
.back-link:hover { color: #fff; }
.detail-title-row { display: flex; align-items: end; justify-content: space-between; gap: 40px; }
.detail-title-row h1 { margin: 0; font-size: clamp(2.35rem, 5vw, 4.3rem); line-height: 1; letter-spacing: -.05em; }
.detail-title-row p { max-width: 730px; margin: 18px 0 0; color: #c6d9e9; font-size: 1.08rem; }
.detail-actions { display: flex; flex-wrap: wrap; gap: 9px; }
.detail-actions .secondary { border-color: rgba(255,255,255,.28); color: #fff; background: rgba(255,255,255,.08); }
.detail-layout { display: grid; grid-template-columns: minmax(0, 1fr) 330px; align-items: start; gap: 24px; padding-block: 34px 80px; }
.readme-card, .side-card { border: 1px solid var(--line); border-radius: var(--radius); background: #fff; }
.section-label { padding: 14px 23px; border-bottom: 1px solid var(--line); color: var(--muted); font-size: .68rem; font-weight: 800; letter-spacing: .1em; text-transform: uppercase; }
.readme-content { padding: clamp(24px, 5vw, 48px); color: #243b53; overflow-wrap: anywhere; }
.readme-content > :first-child { margin-top: 0; }
.readme-content > :last-child { margin-bottom: 0; }
.readme-content h1, .readme-content h2, .readme-content h3 { margin-top: 1.7em; color: var(--ink); line-height: 1.2; letter-spacing: -.025em; }
.readme-content h1 { padding-bottom: .35em; border-bottom: 1px solid var(--line); font-size: 2rem; }
.readme-content h2 { padding-bottom: .3em; border-bottom: 1px solid #e9eff5; font-size: 1.5rem; }
.readme-content pre { max-width: 100%; padding: 17px; overflow: auto; border-radius: 11px; background: #0e2438; color: #e3edf5; }
.readme-content pre code { padding: 0; background: transparent; color: inherit; }
.readme-content img { max-width: 100%; height: auto; }
.readme-content blockquote { margin-inline: 0; padding: 2px 18px; border-left: 4px solid var(--cyan); color: var(--muted); }
.readme-content table { width: 100%; border-collapse: collapse; }
.readme-content th, .readme-content td { padding: 8px 10px; border: 1px solid var(--line); text-align: left; }
.detail-sidebar { display: grid; gap: 18px; }
.side-card { padding: 21px; }
.side-card h2 { margin: 0 0 17px; font-size: 1rem; }
.side-list { margin: 0; }
.side-list div { display: grid; gap: 3px; padding: 12px 0; border-top: 1px solid #e9eff5; }
.side-list dt { color: var(--muted); font-size: .69rem; font-weight: 800; letter-spacing: .08em; text-transform: uppercase; }
.side-list dd { margin: 0; font-size: .88rem; font-weight: 650; }
.side-heading { display: flex; align-items: center; justify-content: space-between; }
.side-heading span { display: grid; place-items: center; width: 25px; height: 25px; border-radius: 50%; background: #edf2f7; font-size: .75rem; font-weight: 800; }
.release-warning { padding: 13px; border: 1px solid #f1d38a; border-radius: 10px; color: #714b00; background: var(--amber-bg); }
.release-warning p { margin: 6px 0 0; font-size: .82rem; }
.release-list { margin: 0; padding: 0; list-style: none; }
.release-list li { display: grid; gap: 9px; padding: 14px 0; border-top: 1px solid #e9eff5; }
.release-list li > div:first-child { display: flex; justify-content: space-between; gap: 8px; }
.release-list span { color: var(--muted); font-size: .75rem; }
.hash { max-width: 100%; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.site-footer { padding: 38px 0; border-top: 1px solid var(--line); color: #829ab1; background: #0b2035; }
.footer-inner { display: flex; justify-content: space-between; gap: 30px; }
.footer-inner strong { color: #fff; }
.footer-inner p { margin: 6px 0 0; font-size: .85rem; }
.footer-links { display: flex; align-items: center; gap: 20px; }
.footer-links a { color: #b8d3e6; font-size: .86rem; }
.error-page { min-height: 500px; margin-block: 45px; align-content: center; }
.error-code { color: var(--blue); font-size: 4.4rem; font-weight: 850; line-height: 1; }
.login-page { min-height: 680px; display: grid; place-items: center; padding-block: 60px; }
.login-card { width: min(440px, 100%); padding: 38px; border: 1px solid var(--line); border-radius: 20px; background: #fff; box-shadow: var(--shadow); }
.login-card h1 { margin: 0; letter-spacing: -.035em; }
.login-card > p { color: var(--muted); }
.stack-form { display: grid; gap: 17px; margin-top: 25px; }
.admin-hero { padding: 40px 0; border-bottom: 1px solid #214761; color: #fff; background: #102a43; }
.admin-title-row { display: flex; align-items: center; justify-content: space-between; gap: 20px; }
.admin-title-row h1 { margin: 0; font-size: 2.25rem; letter-spacing: -.04em; }
.admin-title-row p { margin: 7px 0 0; color: #b8cfdf; }
.admin-layout { padding-block: 28px 80px; }
.admin-stats { display: grid; grid-template-columns: repeat(4, 1fr); gap: 13px; margin-bottom: 24px; }
.admin-stats div { display: grid; padding: 19px; border: 1px solid var(--line); border-radius: 13px; background: #fff; }
.admin-stats strong { font-size: 1.9rem; line-height: 1; }
.admin-stats span { margin-top: 7px; color: var(--muted); font-size: .76rem; font-weight: 750; text-transform: uppercase; }
.admin-section { margin-top: 20px; border: 1px solid var(--line); border-radius: 15px; background: #fff; box-shadow: 0 2px 8px rgba(16,42,67,.03); }
.admin-section-heading { min-height: 72px; display: flex; align-items: center; justify-content: space-between; gap: 20px; padding: 15px 20px; border-bottom: 1px solid var(--line); }
.admin-section-heading > div { display: flex; align-items: baseline; gap: 12px; }
.admin-section-heading .eyebrow { margin: 0; color: var(--blue); }
.admin-section-heading h2 { margin: 0; font-size: 1.18rem; }
.admin-section-heading > p { margin: 0; color: var(--muted); font-size: .82rem; }
.create-source { position: relative; }
.create-source > summary { list-style: none; }
.create-source > summary::-webkit-details-marker { display: none; }
.popover-form { position: absolute; top: calc(100% + 10px); right: 0; z-index: 10; width: min(650px, calc(100vw - 50px)); padding: 20px; border: 1px solid var(--line); border-radius: 13px; background: #fff; box-shadow: 0 20px 50px rgba(16,42,67,.18); }
.form-grid { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 14px; }
.span-2 { grid-column: 1 / -1; }
.check-field { display: flex; grid-template-columns: auto 1fr; align-items: center; align-self: end; min-height: 43px; }
.check-field input { width: 18px; min-height: 18px; }
.form-actions { display: flex; align-items: center; gap: 12px; }
.form-actions small { color: var(--muted); }
.table-wrap { width: 100%; overflow-x: auto; }
.admin-table { width: 100%; border-collapse: collapse; font-size: .84rem; }
.admin-table th { padding: 11px 15px; color: var(--muted); background: #f6f8fa; font-size: .67rem; letter-spacing: .08em; text-align: left; text-transform: uppercase; }
.admin-table td { padding: 14px 15px; border-top: 1px solid #e8eef4; vertical-align: middle; }
.admin-table tbody tr:hover { background: #fbfdff; }
.admin-table td > strong, .admin-table td > small { display: block; }
.admin-table td > small { margin-top: 3px; color: var(--muted); }
.action-row { display: flex; align-items: center; flex-wrap: wrap; gap: 5px; }
.action-row form { margin: 0; }
.mini-button { padding: 5px 8px; border: 1px solid #b8c7d5; border-radius: 7px; color: var(--ink-soft); background: #fff; font-size: .72rem; font-weight: 750; }
.mini-button.approve { border-color: #8bcab9; color: var(--green); background: var(--green-bg); }
.mini-button.reject { border-color: #efb1aa; color: var(--red); background: var(--red-bg); }
.moderation-list { display: grid; }
.moderation-item { border-bottom: 1px solid var(--line); }
.moderation-item:last-child { border-bottom: 0; }
.moderation-item > summary { display: flex; align-items: center; justify-content: space-between; gap: 20px; padding: 17px 20px; cursor: pointer; list-style: none; }
.moderation-item > summary::-webkit-details-marker { display: none; }
.moderation-item > summary::before { content: "+"; order: 3; display: grid; place-items: center; width: 26px; height: 26px; border-radius: 50%; color: var(--blue); background: #edf5ff; font-size: 1.15rem; }
.moderation-item[open] > summary::before { content: "−"; }
.moderation-item > summary > span:first-of-type { display: grid; }
.moderation-item summary small { color: var(--muted); }
.moderation-body { padding: 20px; border-top: 1px solid #e8eef4; background: #fbfcfe; }
.validation-box { margin-bottom: 17px; padding: 12px 15px; border-left: 4px solid var(--amber); border-radius: 6px; color: #714b00; background: var(--amber-bg); }
.validation-box ul { margin: 6px 0 0; padding-left: 20px; font-size: .82rem; }
.moderation-actions { display: flex; flex-wrap: wrap; gap: 8px; margin-top: 18px; padding-top: 17px; border-top: 1px solid var(--line); }
.moderation-actions form { margin: 0; }
.release-admin-table { min-width: 960px; }
.artifact-details { max-width: 330px; margin-top: 7px; }
.artifact-details summary { color: var(--blue); cursor: pointer; font-size: .72rem; font-weight: 750; }
.artifact-details a { display: block; margin-top: 5px; font-size: .7rem; }
.artifact-details strong { display: block; margin-top: 7px; color: var(--muted); font-size: .64rem; text-transform: uppercase; }
.break-value, .full-hash { white-space: normal; overflow-wrap: anywhere; word-break: break-all; }
.full-hash { display: block; max-width: 270px; font-size: .7rem; }
.text-danger { color: var(--red) !important; }
.empty-row { padding: 30px !important; color: var(--muted); text-align: center !important; }
.admin-columns { display: grid; grid-template-columns: 1fr 1fr; gap: 20px; }
.compact-section { min-width: 0; }
.timeline { max-height: 430px; margin: 0; padding: 7px 20px 18px; overflow: auto; list-style: none; }
.timeline li { display: grid; grid-template-columns: 13px minmax(0, 1fr); gap: 12px; padding: 14px 0; border-bottom: 1px solid #e8eef4; }
.timeline li:last-child { border-bottom: 0; }
.timeline-dot { width: 9px; height: 9px; margin-top: 6px; border-radius: 50%; background: var(--muted); }
.timeline-dot.success, .timeline-dot.audit { background: var(--green); }
.timeline-dot.partial { background: var(--amber); }
.timeline-dot.failed { background: var(--red); }
.timeline strong { font-size: .84rem; text-transform: capitalize; }
.timeline p { margin: 2px 0; color: var(--ink-soft); font-size: .8rem; }
.timeline small { color: var(--muted); font-size: .72rem; }
.sync-errors { margin-top: 7px; color: var(--ink-soft); font-size: .72rem; }
.sync-errors summary { color: var(--blue); cursor: pointer; font-weight: 750; }
.sync-errors ul { margin: 7px 0 0; padding-left: 18px; }
.sync-errors li { display: list-item; padding: 3px 0; border: 0; overflow-wrap: anywhere; }
@media (max-width: 980px) {
.hero-grid { grid-template-columns: 1fr; gap: 35px; }
.hero-stat { width: min(350px, 100%); }
.filter-panel { grid-template-columns: 2fr 1fr; }
.plugin-grid { grid-template-columns: repeat(2, minmax(0, 1fr)); }
.detail-layout { grid-template-columns: 1fr; }
.detail-sidebar { grid-template-columns: 1fr 1fr; }
.admin-columns { grid-template-columns: 1fr; }
}
@media (max-width: 680px) {
.shell { width: min(100% - 24px, 1180px); }
.header-inner { min-height: 66px; }
.brand small { display: none; }
.main-nav a { padding: 8px; font-size: .82rem; }
.hero-grid { min-height: 0; padding-block: 60px 75px; }
.hero h1 { font-size: 2.65rem; }
.hero-stat { padding: 23px; }
.filter-panel { grid-template-columns: 1fr; margin-top: -25px; }
.plugin-grid { grid-template-columns: 1fr; }
.catalog-heading, .detail-title-row, .admin-title-row, .footer-inner { align-items: flex-start; flex-direction: column; }
.detail-actions { width: 100%; }
.detail-actions .button { flex: 1; }
.detail-sidebar { grid-template-columns: 1fr; }
.readme-content { padding: 22px; }
.footer-links { flex-direction: column; align-items: flex-start; gap: 8px; }
.admin-stats { grid-template-columns: 1fr 1fr; }
.admin-section-heading { align-items: flex-start; flex-direction: column; }
.create-source { width: 100%; }
.create-source > summary { width: 100%; }
.popover-form { position: static; width: 100%; margin-top: 10px; box-shadow: none; }
.form-grid { grid-template-columns: 1fr; }
.span-2 { grid-column: auto; }
.moderation-item > summary { align-items: flex-start; }
.summary-status { display: none; }
}
@media (prefers-reduced-motion: reduce) {
*, *::before, *::after { scroll-behavior: auto !important; transition: none !important; }
}
store/public/assets/favicon.svg
+10
View File
@@ -0,0 +1,10 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64">
<defs>
<linearGradient id="g" x1="8" y1="4" x2="56" y2="60" gradientUnits="userSpaceOnUse">
<stop stop-color="#1769e0"/>
<stop offset="1" stop-color="#21b6c7"/>
</linearGradient>
</defs>
<rect x="4" y="4" width="56" height="56" rx="15" fill="url(#g)"/>
<path d="M18 46V18h7l14 18V18h7v28h-7L25 28v18z" fill="#fff"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 412 B

store/public/index.php
+40
View File
@@ -0,0 +1,40 @@
<?php
declare(strict_types=1);
// Web responses never expose PHP diagnostics; errors are logged and the
// application returns a generic error page/JSON envelope.
ini_set('display_errors', '0');
ini_set('display_startup_errors', '0');
ini_set('log_errors', '1');
header_remove('X-Powered-By');
use NetBoxStore\Config;
use NetBoxStore\Database\RepositoryFactory;
use NetBoxStore\Http\Application;
use NetBoxStore\Http\Request;
use NetBoxStore\Security\Auth;
use NetBoxStore\Security\HttpClient;
use NetBoxStore\Security\SsrfGuard;
use NetBoxStore\Sync\SyncService;
$root = dirname(__DIR__);
require $root . '/vendor/autoload.php';
$config = Config::load($root);
$repository = RepositoryFactory::create($config);
$repository->initialize();
$request = Request::fromGlobals($config);
$guard = new SsrfGuard($config);
$http = new HttpClient($config, $guard);
$sync = new SyncService($repository, $config, $http, $guard);
$sync->ensureDefaultSource();
$auth = new Auth($config, $repository);
$adminPath = $request->path === '/admin' || str_starts_with($request->path, '/admin/');
if ($auth->enabled() && $adminPath) {
// Public catalog/API/health traffic must not allocate anonymous sessions.
$auth->startSession();
}
$application = new Application($config, $repository, $auth, $sync, $guard);
$application->handle($request)->send();
store/src/Config.php
+197
View File
@@ -0,0 +1,197 @@
<?php
declare(strict_types=1);
namespace NetBoxStore;
use RuntimeException;
final class Config
{
/** @param list<string> $allowedHosts */
private function __construct(
public readonly string $root,
public readonly string $environment,
public readonly string $publicUrl,
public readonly bool $trustProxy,
public readonly bool $secureCookies,
public readonly string $sessionName,
public readonly array $admin,
public readonly array $database,
public readonly array $network,
public readonly array $defaults,
public readonly array $scheduler,
public readonly array $syncLimits,
public readonly array $pagination,
) {
}
public static function load(string $root): self
{
self::loadDotEnv($root . '/.env');
$publicUrl = self::env('STORE_PUBLIC_URL', 'http://localhost');
$publicParts = parse_url($publicUrl);
if (filter_var($publicUrl, FILTER_VALIDATE_URL) === false || !is_array($publicParts)
|| !in_array($publicParts['scheme'] ?? '', ['http', 'https'], true)
|| isset($publicParts['user']) || isset($publicParts['pass'])) {
throw new RuntimeException('STORE_PUBLIC_URL is invalid.');
}
$environment = self::env('APP_ENV', 'development');
if ($environment === 'production' && ($publicParts['scheme'] ?? '') !== 'https') {
error_log('WARNING: APP_ENV=production is using an HTTP STORE_PUBLIC_URL; secure transport is strongly recommended.');
}
$sessionName = self::env('STORE_SESSION_NAME', 'netbox_plugin_store');
if (!preg_match('/^[A-Za-z0-9_-]{1,64}$/', $sessionName)) {
throw new RuntimeException('STORE_SESSION_NAME is invalid.');
}
$admin = [
'username' => self::env('STORE_ADMIN_USERNAME'),
'passwordHash' => self::env('STORE_ADMIN_PASSWORD_HASH'),
'sessionSecret' => self::env('STORE_SESSION_SECRET'),
'sessionTtl' => self::integer('STORE_ADMIN_SESSION_TTL', 28_800, 900, 86_400),
'maxAttempts' => self::integer('STORE_LOGIN_MAX_ATTEMPTS', 5, 2, 50),
'attemptWindow' => self::integer('STORE_LOGIN_WINDOW_SECONDS', 900, 60, 86_400),
];
$configured = count(array_filter(array_slice($admin, 0, 3), static fn (mixed $v): bool => $v !== ''));
if ($configured > 0 && $configured < 3) {
throw new RuntimeException('All admin credential environment variables must be set together.');
}
$admin['enabled'] = $configured === 3;
if ($admin['enabled'] && strlen($admin['sessionSecret']) < 32) {
throw new RuntimeException('STORE_SESSION_SECRET must contain at least 32 characters.');
}
if ($admin['enabled'] && (password_get_info($admin['passwordHash'])['algoName'] ?? 'unknown') !== 'argon2id') {
throw new RuntimeException('STORE_ADMIN_PASSWORD_HASH must be a valid Argon2id hash.');
}
$driver = strtolower(self::env('STORE_DB_DRIVER', 'json'));
if (!in_array($driver, ['json', 'mariadb'], true)) {
throw new RuntimeException('STORE_DB_DRIVER must be json or mariadb.');
}
$jsonPath = self::env('STORE_JSON_PATH', $root . '/data/store.json');
if (!str_starts_with($jsonPath, '/') && !preg_match('/^[A-Za-z]:[\\\\\/]/', $jsonPath)) {
$jsonPath = $root . '/' . ltrim($jsonPath, '/');
}
return new self(
root: $root,
environment: $environment,
publicUrl: rtrim($publicUrl, '/'),
trustProxy: self::boolean('STORE_TRUST_PROXY'),
secureCookies: getenv('STORE_COOKIE_SECURE') === false
? str_starts_with($publicUrl, 'https://')
: self::boolean('STORE_COOKIE_SECURE'),
sessionName: $sessionName,
admin: $admin,
database: [
'driver' => $driver,
'jsonPath' => $jsonPath,
'dsn' => self::env('STORE_MARIADB_DSN', 'mysql:host=127.0.0.1;dbname=netbox_store;charset=utf8mb4'),
'user' => self::env('STORE_MARIADB_USER', 'netbox_store'),
'password' => self::env('STORE_MARIADB_PASSWORD'),
],
network: [
'allowedHosts' => self::list('STORE_ALLOWED_SOURCE_HOSTS', 'git.mrblake.cc,github.com,api.github.com,*.github.com,*.githubusercontent.com'),
'allowPrivate' => self::boolean('STORE_ALLOW_PRIVATE_NETWORKS'),
'trustedProxyIps' => self::ipList('STORE_TRUSTED_PROXY_IPS', '127.0.0.1,::1'),
'timeout' => self::integer('STORE_HTTP_TIMEOUT_SECONDS', 20, 2, 120),
'maxMetadataBytes' => self::integer('STORE_MAX_METADATA_BYTES', 2 * 1024 * 1024, 65_536, 20 * 1024 * 1024),
'maxArtifactBytes' => self::integer('STORE_MAX_ARTIFACT_BYTES', 512 * 1024 * 1024, 1_024, 2 * 1024 * 1024 * 1024),
'userAgent' => self::env('STORE_USER_AGENT', 'MrBlake-NetBox-Plugin-Store/1.0'),
],
defaults: [
'provider' => strtolower(self::env('STORE_DEFAULT_PROVIDER', 'forgejo')),
'name' => self::env('STORE_DEFAULT_SOURCE_NAME', 'MrBlake Forgejo'),
'slug' => self::env('STORE_DEFAULT_SOURCE_SLUG', 'mrblake-forgejo'),
'baseUrl' => rtrim(self::env('STORE_DEFAULT_BASE_URL', 'https://git.mrblake.cc'), '/'),
'apiUrl' => rtrim(self::env('STORE_DEFAULT_API_URL', 'https://git.mrblake.cc/api/v1'), '/'),
'owner' => self::env('STORE_DEFAULT_OWNER', 'MrBlake'),
'ownerKind' => strtolower(self::env('STORE_DEFAULT_OWNER_KIND', 'user')),
'topic' => self::env('STORE_DEFAULT_TOPIC', 'netbox-plugin'),
'tokenEnv' => self::env('STORE_DEFAULT_TOKEN_ENV', 'GITEA_TOKEN'),
],
scheduler: [
'enabled' => self::boolean('STORE_SCHEDULER_ENABLED'),
'interval' => self::integer('STORE_SYNC_INTERVAL_SECONDS', 900, 30, 604_800),
],
syncLimits: [
'seconds' => self::integer('STORE_SYNC_MAX_SECONDS', 900, 30, 7_200),
'requests' => self::integer('STORE_SYNC_MAX_REQUESTS', 2_500, 50, 20_000),
'bytes' => self::integer('STORE_SYNC_MAX_BYTES', 1_073_741_824, 16_777_216, 17_179_869_184),
'repositories' => self::integer('STORE_SYNC_MAX_REPOSITORIES', 2_000, 1, 10_000),
'releases' => self::integer('STORE_SYNC_MAX_RELEASES', 1_000, 1, 1_000),
],
pagination: [
'pageSize' => self::integer('STORE_PAGE_SIZE', 12, 1, 100),
'apiPageSize' => self::integer('STORE_API_PAGE_SIZE', 50, 1, 100),
'apiMaxPageSize' => self::integer('STORE_API_MAX_PAGE_SIZE', 100, 1, 250),
],
);
}
private static function env(string $name, string $default = ''): string
{
$value = getenv($name);
return $value === false ? $default : trim($value);
}
private static function boolean(string $name, bool $default = false): bool
{
$value = getenv($name);
if ($value === false) {
return $default;
}
return in_array(strtolower(trim($value)), ['1', 'true', 'yes', 'on'], true);
}
private static function integer(string $name, int $default, int $minimum, int $maximum): int
{
$value = filter_var(getenv($name), FILTER_VALIDATE_INT);
return max($minimum, min($maximum, $value === false ? $default : $value));
}
/** @return list<string> */
private static function list(string $name, string $default): array
{
return array_values(array_filter(array_map(
static fn (string $part): string => strtolower(trim($part)),
explode(',', self::env($name, $default)),
)));
}
/** @return list<string> */
private static function ipList(string $name, string $default): array
{
$values = self::list($name, $default);
foreach ($values as $value) {
if (filter_var($value, FILTER_VALIDATE_IP) === false) {
throw new RuntimeException($name . ' must contain exact IP addresses.');
}
}
return $values;
}
private static function loadDotEnv(string $path): void
{
if (!is_file($path)) {
return;
}
foreach (file($path, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES) ?: [] as $line) {
$line = trim($line);
if ($line === '' || str_starts_with($line, '#') || !str_contains($line, '=')) {
continue;
}
[$name, $value] = explode('=', $line, 2);
$name = trim($name);
if (getenv($name) !== false || !preg_match('/^[A-Z][A-Z0-9_]*$/', $name)) {
continue;
}
$value = trim($value);
if (strlen($value) >= 2 && in_array($value[0], ['"', "'"], true) && $value[-1] === $value[0]) {
$value = substr($value, 1, -1);
}
putenv($name . '=' . $value);
$_ENV[$name] = $value;
}
}
}

Some files were not shown because too many files have changed in this diff Show More