feat: add NetBox plugin store
CI / php-store (push) Waiting to run
CI / python-components (push) Waiting to run

This commit is contained in:
2026-08-24 20:51:25 +02:00
commit f36d6be511
135 changed files with 15160 additions and 0 deletions
+6
View File
@@ -0,0 +1,6 @@
__pycache__/
*.py[cod]
build/
dist/
*.egg-info/
.pytest_cache/
+13
View File
@@ -0,0 +1,13 @@
Copyright 2026 MrBlake
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+3
View File
@@ -0,0 +1,3 @@
recursive-include netbox_plugin_store/templates *.html
recursive-include netbox_plugin_store/migrations *.py
include README.md LICENSE
+109
View File
@@ -0,0 +1,109 @@
# NetBox Plugin Store
`netbox-plugin-store` is the NetBox-side client for an approved internal plugin catalog. It supports NetBox **4.6.5 through 4.6.8** and provides catalog, status, confirmation, and redacted audit pages.
The safe default is intentionally non-mutating. Installing the UI alone never grants the NetBox web process permission to modify its Python environment.
## Security model
- Catalog, status, audit, and dry-run access require an authenticated user with `netbox_plugin_store.manage_plugin` (superusers implicitly have it).
- Every state-changing POST is checked again in the view and requires a **superuser**. A menu permission alone is never sufficient.
- Lifecycle endpoints accept POST only, use Django CSRF protection, and require the user to type the exact plugin slug on a separate confirmation page.
- Package, import, slug, version, URL, and SHA-256 fields are validated. The Store plugin cannot update, disable, or uninstall itself.
- Installation/update requires an explicitly approved plugin and an explicitly approved, immutable release with a 64-character **artifact** SHA-256. A Git commit SHA is not an artifact hash and is rejected.
- Downloads go to a private temporary directory and are SHA-256 verified before pip sees the file. Redirects and pagination remain inside configured URL allowlists.
- Pip and maintenance commands use argv arrays with `shell=False`; direct mode uses `--no-index` by default and runs `python -m pip check` after install/update.
- `configuration.py` and `local_requirements.txt` are changed via same-directory temporary files plus `os.replace()`. Existing files receive timestamped `0600` backups. A cross-process file lock serializes operations.
- Audit input/output is bounded and redacted for common token, password, authorization, and credential patterns.
- Install only installs and pins a package; it remains disabled. Enable is a separate action which edits `PLUGINS`, runs migrations/collectstatic, and requires restart. Uninstall is accepted only after an explicit disable.
## Installation
Install the wheel into NetBox's virtual environment and persist it in `/opt/netbox/local_requirements.txt`:
```text
netbox-plugin-store==0.1.0
```
Add the plugin to `configuration.py`:
```python
PLUGINS = [
"netbox_plugin_store",
]
PLUGINS_CONFIG = {
"netbox_plugin_store": {
# Required: this is the separate Store service, not the Forgejo host.
"store_url": "https://store.example.internal",
"allowed_store_urls": ["https://store.example.internal"],
# Include every origin from which approved immutable artifacts are served.
"allowed_artifact_urls": ["https://store.example.internal", "https://git.mrblake.cc"],
"api_token": "", # Prefer a read-only catalog token if authentication is required.
"execution_mode": "dry_run",
}
}
```
Then use NetBox's supported upgrade flow (normally `/opt/netbox/upgrade.sh`) or run migrate/collectstatic and restart the web and RQ services. The service user needs read access to the Store and configuration. Dry-run mode needs no venv/config write permission.
The Store API contract is:
- `GET /api/v1/plugins/` (plain list or `{ "results": [...] }` pagination)
- `GET /api/v1/plugins/<slug>/`
- `GET /api/v1/plugins/<slug>/releases/<version>/` (used by the host agent; returns one release object)
- plugin fields: `slug`, `name`, `summary`, `description`, `repository_url`, `latest_version`, `package_name`, `import_name`, `min_netbox_version`, `max_netbox_version`, `approved`/`status`, and `releases`
- release fields: `version`, `download_url`, `sha256`, `artifact_size`, `commit_sha`, `min_netbox_version`, `max_netbox_version`, `published_at`, `approved`, `status`, `immutable`, and opaque `approved_payload_sha256`
## Execution modes
### `dry_run` (default)
Only validates and plans. `default_dry_run` also defaults to `True`. Dry-runs may use NetBox's default RQ queue; real operations never run inside `netbox-rq`, because restarting the same worker would leave its job state inconsistent.
### `agent` (recommended for production)
Use a separately privileged host agent. NetBox connects to an absolute Unix socket and sends one JSON line per connection (maximum 64 KiB):
```python
"execution_mode": "agent",
"agent_socket_path": "/run/netbox-store-agent/agent.sock",
"agent_timeout": 30,
```
Protocol version 1 uses `GET /v1/capabilities`, `POST /v1/operations`, and `GET /v1/operations/<uuid>`. The POST includes a UUID idempotency key and only the action, slug, version, actor, request ID, and the Store's opaque `approved_payload_sha256`. The agent re-fetches the approved catalog record itself.
The mutating HTTP request stops immediately after the agent accepts the operation. It does not poll while the agent might restart NetBox. Opening the audit detail page performs one status query and reconciles a completed/failed operation.
The Unix socket should be owned by the agent group, writable only by the NetBox service account/group, and placed in a non-world-writable directory.
### `direct` (development/controlled installations only)
Direct mode additionally requires `allow_lifecycle_mutations=True`. The NetBox service account must be able to write the venv, `configuration.py`, `local_requirements.txt`, backup directory, and lock file. This is often inappropriate for production.
```python
"execution_mode": "direct",
"allow_lifecycle_mutations": True,
"configuration_path": "/opt/netbox/netbox/netbox/configuration.py",
"requirements_path": "/opt/netbox/local_requirements.txt",
"manage_path": "/opt/netbox/netbox/manage.py",
"allow_package_index": False,
```
With `allow_package_index=False`, pip receives `--no-index`; therefore all transitive dependencies must already be installed or available in the artifact. Enabling `allow_package_index` is an explicit supply-chain policy decision.
Direct mode never performs automatic restart. Enable, disable, and updating an enabled plugin are marked `restart-required`; restart NetBox web services and workers out of band. `auto_restart=True` is rejected in direct mode.
For agent-managed restarts, `restart_commands` must match `restart_allowlist` token-for-token. Do not use a shell command string. The host agent must enforce its own independent command policy.
## Persistence and rollback
Install/update writes an immutable PEP 508 direct URL plus `#sha256=...` into `local_requirements.txt`; uninstall removes it. This keeps plugins present across NetBox's supported upgrade flow. Config/requirements writes are atomic and backed up under `plugin-store-backups` by default. The direct executor performs compensating rollback where safe; an interrupted pip upgrade or already-applied database migration can still require operator recovery, which is recorded as failed/restart-required.
## Test
The core tests use fake downloads, subprocesses, repositories, and sockets; they never invoke real pip, NetBox restarts, or maintenance commands:
```bash
python -m unittest discover -s tests -v
```
@@ -0,0 +1,55 @@
from netbox.plugins import PluginConfig
from .version import __version__
class NetBoxPluginStoreConfig(PluginConfig):
name = "netbox_plugin_store"
verbose_name = "NetBox Plugin Store"
description = "Install and manage approved plugins from the MrBlake store."
version = __version__
author = "MrBlake"
base_url = "plugin-store"
min_version = "4.6.5"
max_version = "4.6.8"
required_settings = ["store_url", "allowed_store_urls", "allowed_artifact_urls"]
default_settings = {
"api_token": "",
"request_timeout": 15,
"download_timeout": 120,
"max_download_bytes": 268_435_456,
"configuration_path": None,
"requirements_path": None,
"manage_requirements_file": True,
"allow_lifecycle_mutations": False,
"default_dry_run": True,
"execution_mode": "dry_run",
"agent_socket_path": None,
"agent_timeout": 30,
"agent_poll_interval": 1.0,
"manage_path": None,
"lock_path": None,
"backup_dir": None,
"lock_timeout": 30,
"operation_timeout": 900,
"pip_extra_args": [],
"allow_package_index": False,
"run_migrations": True,
"collect_static": True,
"background_jobs": True,
"synchronous_fallback": True,
"job_queue": "default",
"auto_restart": False,
"restart_commands": [],
"restart_allowlist": [],
"backups_to_keep": 25,
}
def ready(self):
super().ready()
# Import so NetBox can deserialize the JobRunner by dotted path.
from .jobs import PluginLifecycleJob # noqa: F401
config = NetBoxPluginStoreConfig
@@ -0,0 +1,18 @@
from __future__ import annotations
from typing import Protocol
class PermissionUser(Protocol):
is_authenticated: bool
is_superuser: bool
def has_perm(self, permission_name: str) -> bool: ...
def has_store_access(user: PermissionUser, permission_name: str) -> bool:
"""Authorize NetBox users without relying on the removed ``is_staff`` field."""
return bool(
user.is_authenticated
and (user.is_superuser or user.has_perm(permission_name))
)
+129
View File
@@ -0,0 +1,129 @@
from __future__ import annotations
import json
import socket
import time
from pathlib import Path
from typing import Any
from uuid import UUID, uuid4
from .redaction import redact_text
class AgentError(RuntimeError):
pass
class AgentClient:
PROTOCOL_VERSION = 1
MAX_MESSAGE_BYTES = 64 * 1024
def __init__(self, socket_path: Path, *, timeout: int = 30):
self.socket_path = Path(socket_path)
self.timeout = timeout
def request(self, method: str, path: str, **values: Any) -> dict[str, Any]:
request = {
"protocol_version": self.PROTOCOL_VERSION,
"method": method,
"path": path,
**values,
}
encoded = json.dumps(request, separators=(",", ":"), ensure_ascii=True).encode("utf-8") + b"\n"
if len(encoded) > self.MAX_MESSAGE_BYTES:
raise AgentError("Agent request exceeds 64 KiB.")
if not hasattr(socket, "AF_UNIX"):
raise AgentError("Unix sockets are unavailable on this platform.")
try:
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client:
client.settimeout(self.timeout)
client.connect(str(self.socket_path))
client.sendall(encoded)
client.shutdown(socket.SHUT_WR)
response = bytearray()
while b"\n" not in response:
chunk = client.recv(4096)
if not chunk:
break
response.extend(chunk)
if len(response) > self.MAX_MESSAGE_BYTES:
raise AgentError("Agent response exceeds 64 KiB.")
except (OSError, TimeoutError) as exc:
raise AgentError(f"Agent connection failed: {type(exc).__name__}") from exc
line, separator, remainder = bytes(response).partition(b"\n")
if not separator or remainder:
raise AgentError("Agent must return exactly one JSON line.")
try:
payload = json.loads(line.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise AgentError("Agent returned invalid JSON.") from exc
if not isinstance(payload, dict) or payload.get("protocol_version") != self.PROTOCOL_VERSION:
raise AgentError("Agent returned an incompatible protocol response.")
status = payload.get("status")
body = payload.get("body")
if not isinstance(status, int) or not isinstance(body, dict):
raise AgentError("Agent returned a malformed response.")
if status < 200 or status >= 300:
message = body.get("error") or body.get("detail") or f"Agent returned status {status}."
raise AgentError(redact_text(message, limit=2_000))
return body
def capabilities(self) -> dict[str, Any]:
return self.request("GET", "/v1/capabilities")
def submit_operation(
self,
*,
request_id: str,
action: str,
plugin_slug: str,
version: str | None,
approved_payload_sha256: str | None,
requested_by: str,
) -> UUID:
idempotency_key = uuid4()
body = self.request(
"POST",
"/v1/operations",
idempotency_key=str(idempotency_key),
body={
"request_id": request_id,
"action": action,
"plugin_slug": plugin_slug,
"version": version,
"approved_payload_sha256": approved_payload_sha256,
"requested_by": requested_by,
},
)
raw_id = body.get("operation_id") or body.get("id")
try:
return UUID(str(raw_id))
except (ValueError, TypeError) as exc:
raise AgentError("Agent did not return a valid operation UUID.") from exc
def wait_for_operation(
self,
operation_id: UUID,
*,
overall_timeout: int,
poll_interval: float,
) -> dict[str, Any]:
deadline = time.monotonic() + overall_timeout
while True:
body = self.get_operation(operation_id)
state = body.get("state") or body.get("status")
if state in {"succeeded", "completed"}:
result = body.get("result", body)
if not isinstance(result, dict):
raise AgentError("Agent operation result is malformed.")
return result
if state in {"failed", "errored", "cancelled"}:
raise AgentError(redact_text(body.get("error") or f"Agent operation {state}."))
if state not in {"queued", "pending", "running", "accepted"}:
raise AgentError("Agent returned an unknown operation state.")
if time.monotonic() >= deadline:
raise AgentError("Timed out waiting for the host agent operation.")
time.sleep(poll_interval)
def get_operation(self, operation_id: UUID) -> dict[str, Any]:
return self.request("GET", f"/v1/operations/{operation_id}")
+294
View File
@@ -0,0 +1,294 @@
from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.parse import quote, urljoin, urlsplit, urlunsplit
from urllib.request import HTTPRedirectHandler, Request, build_opener
from packaging.version import InvalidVersion, Version
from .validation import (
ValidationError,
validate_distribution_name,
validate_import_name,
validate_public_url,
validate_sha256,
validate_slug,
)
class StoreClientError(RuntimeError):
pass
class URLPolicy:
"""Exact scheme/host/port allowlist with an optional path prefix."""
def __init__(self, allowed_roots: tuple[str, ...] | list[str]):
if not allowed_roots:
raise ValueError("At least one allowed URL is required.")
if any(urlsplit(root).query or urlsplit(root).fragment for root in allowed_roots):
raise ValueError("Allowlist roots may not contain a query string or fragment.")
self._roots = tuple(self._normalize(validate_public_url(root)) for root in allowed_roots)
@staticmethod
def _normalize(value: str) -> tuple[str, str, int, str]:
parsed = urlsplit(value)
scheme = parsed.scheme.lower()
default_port = 443 if scheme == "https" else 80
port = parsed.port or default_port
prefix = parsed.path.rstrip("/") or "/"
return scheme, parsed.hostname.lower(), port, prefix
def check(self, value: str) -> str:
validate_public_url(value)
parsed = urlsplit(value)
candidate = self._normalize(value)
scheme, host, port, path = candidate
for allowed_scheme, allowed_host, allowed_port, prefix in self._roots:
path_ok = prefix == "/" or path == prefix or path.startswith(prefix + "/")
if (scheme, host, port) == (allowed_scheme, allowed_host, allowed_port) and path_ok:
return value
raise StoreClientError("URL is outside the configured allowlist.")
class _PolicyRedirectHandler(HTTPRedirectHandler):
def __init__(self, policy: URLPolicy):
self.policy = policy
super().__init__()
def redirect_request(self, req, fp, code, msg, headers, newurl):
self.policy.check(newurl)
return super().redirect_request(req, fp, code, msg, headers, newurl)
@dataclass(frozen=True, slots=True)
class Release:
version: str
download_url: str
sha256: str
min_netbox_version: str
max_netbox_version: str
published_at: str
approved: bool
immutable: bool
approved_payload_sha256: str
@classmethod
def from_mapping(cls, value: dict[str, Any]) -> "Release":
version = str(value.get("version", "")).strip()
try:
Version(version)
except InvalidVersion as exc:
raise StoreClientError("Store returned an invalid release version.") from exc
digest = str(value.get("sha256") or "").strip()
if digest:
digest = validate_sha256(digest)
approved_payload_sha256 = str(value.get("approved_payload_sha256") or "").strip()
if approved_payload_sha256:
approved_payload_sha256 = validate_sha256(approved_payload_sha256)
download_url = str(value.get("download_url") or value.get("artifact_url") or "").strip()
if download_url:
validate_public_url(download_url)
return cls(
version=version,
download_url=download_url,
sha256=digest,
min_netbox_version=str(value.get("min_netbox_version") or "").strip(),
max_netbox_version=str(value.get("max_netbox_version") or "").strip(),
published_at=str(value.get("published_at") or "").strip(),
approved=value.get("approved") is True or value.get("status") == "approved",
immutable=value.get("immutable") is True,
approved_payload_sha256=approved_payload_sha256,
)
def supports(self, netbox_version: str, plugin: "CatalogPlugin") -> bool:
try:
current = Version(netbox_version)
minimum = Version(self.min_netbox_version or plugin.min_netbox_version or "0")
maximum = Version(self.max_netbox_version or plugin.max_netbox_version or "999999")
except InvalidVersion:
return False
return minimum <= current <= maximum
@dataclass(frozen=True, slots=True)
class CatalogPlugin:
slug: str
name: str
summary: str
description: str
repository_url: str
latest_version: str
package_name: str
import_name: str
min_netbox_version: str
max_netbox_version: str
approved: bool
releases: tuple[Release, ...]
@classmethod
def from_mapping(cls, value: dict[str, Any]) -> "CatalogPlugin":
try:
slug = validate_slug(str(value.get("slug", "")))
package_name = validate_distribution_name(str(value.get("package_name", "")))
import_name = validate_import_name(str(value.get("import_name", "")))
except ValidationError as exc:
raise StoreClientError(str(exc)) from exc
releases_raw = value.get("releases") or []
if not isinstance(releases_raw, list):
raise StoreClientError("Store returned an invalid releases collection.")
releases = tuple(Release.from_mapping(item) for item in releases_raw if isinstance(item, dict))
repository_url = str(value.get("repository_url") or "").strip()
if repository_url:
validate_public_url(repository_url)
return cls(
slug=slug,
name=str(value.get("name") or slug)[:200],
summary=str(value.get("summary") or "")[:2_000],
description=str(value.get("description") or "")[:250_000],
repository_url=repository_url,
latest_version=str(value.get("latest_version") or "").strip(),
package_name=package_name,
import_name=import_name,
min_netbox_version=str(value.get("min_netbox_version") or "").strip(),
max_netbox_version=str(value.get("max_netbox_version") or "").strip(),
approved=value.get("approved") is True or value.get("status") == "approved",
releases=releases,
)
def select_release(self, netbox_version: str, requested_version: str = "") -> Release:
compatible = [release for release in self.releases if release.supports(netbox_version, self)]
if requested_version:
compatible = [release for release in compatible if release.version == requested_version]
elif self.latest_version:
latest = [release for release in compatible if release.version == self.latest_version]
if latest:
compatible = latest
if not compatible:
raise StoreClientError("No compatible release is available for this NetBox version.")
return max(compatible, key=lambda release: Version(release.version))
class StoreClient:
API_LIMIT = 5 * 1024 * 1024
MAX_PAGES = 50
def __init__(
self,
store_url: str,
allowed_store_urls: tuple[str, ...],
allowed_artifact_urls: tuple[str, ...],
*,
api_token: str = "",
timeout: int = 15,
):
self.store_policy = URLPolicy(allowed_store_urls)
self.artifact_policy = URLPolicy(allowed_artifact_urls)
if urlsplit(store_url).query or urlsplit(store_url).fragment:
raise StoreClientError("Store base URL may not contain a query string or fragment.")
self.store_url = self.store_policy.check(store_url.rstrip("/"))
# API authentication may only follow redirects below the configured base URL.
self.api_policy = URLPolicy([self.store_url])
self.api_token = api_token
self.timeout = timeout
def _request_json(self, url: str) -> Any:
self.api_policy.check(url)
headers = {"Accept": "application/json", "User-Agent": "netbox-plugin-store/0.1"}
if self.api_token:
headers["Authorization"] = f"Bearer {self.api_token}"
request = Request(url, headers=headers, method="GET")
opener = build_opener(_PolicyRedirectHandler(self.api_policy))
try:
with opener.open(request, timeout=self.timeout) as response:
length = response.headers.get("Content-Length")
if length and int(length) > self.API_LIMIT:
raise StoreClientError("Store response exceeds the size limit.")
body = response.read(self.API_LIMIT + 1)
except (HTTPError, URLError, TimeoutError, OSError) as exc:
raise StoreClientError(f"Store request failed: {type(exc).__name__}") from exc
if len(body) > self.API_LIMIT:
raise StoreClientError("Store response exceeds the size limit.")
try:
return json.loads(body.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise StoreClientError("Store returned invalid JSON.") from exc
def list_plugins(self) -> list[CatalogPlugin]:
next_url: str | None = self.store_url + "/api/v1/plugins/"
plugins: list[CatalogPlugin] = []
pages = 0
while next_url:
pages += 1
if pages > self.MAX_PAGES:
raise StoreClientError("Store pagination limit exceeded.")
payload = self._request_json(next_url)
if isinstance(payload, list):
items = payload
next_url = None
elif isinstance(payload, dict):
items = payload.get("results", payload.get("plugins", []))
raw_next = payload.get("next")
next_url = urljoin(next_url, str(raw_next)) if raw_next else None
if next_url:
self.api_policy.check(next_url)
else:
raise StoreClientError("Store returned an invalid catalog payload.")
if not isinstance(items, list):
raise StoreClientError("Store returned an invalid plugin collection.")
plugins.extend(CatalogPlugin.from_mapping(item) for item in items if isinstance(item, dict))
return plugins
def get_plugin(self, slug: str) -> CatalogPlugin:
slug = validate_slug(slug)
payload = self._request_json(self.store_url + f"/api/v1/plugins/{quote(slug)}/")
if not isinstance(payload, dict):
raise StoreClientError("Store returned an invalid plugin payload.")
return CatalogPlugin.from_mapping(payload)
def download_artifact(
self,
url: str,
destination: Path,
*,
expected_sha256: str,
timeout: int,
max_bytes: int,
) -> tuple[str, int]:
self.artifact_policy.check(url)
expected = validate_sha256(expected_sha256)
request = Request(
url,
headers={"Accept": "application/octet-stream", "User-Agent": "netbox-plugin-store/0.1"},
method="GET",
)
opener = build_opener(_PolicyRedirectHandler(self.artifact_policy))
digest = hashlib.sha256()
written = 0
try:
with opener.open(request, timeout=timeout) as response, destination.open("xb") as target:
length = response.headers.get("Content-Length")
if length and int(length) > max_bytes:
raise StoreClientError("Artifact exceeds the configured size limit.")
while chunk := response.read(1024 * 1024):
written += len(chunk)
if written > max_bytes:
raise StoreClientError("Artifact exceeds the configured size limit.")
digest.update(chunk)
target.write(chunk)
except StoreClientError:
destination.unlink(missing_ok=True)
raise
except (HTTPError, URLError, TimeoutError, OSError) as exc:
destination.unlink(missing_ok=True)
raise StoreClientError(f"Artifact download failed: {type(exc).__name__}") from exc
actual = digest.hexdigest()
if actual != expected:
destination.unlink(missing_ok=True)
raise StoreClientError("Artifact SHA-256 verification failed.")
return actual, written
@@ -0,0 +1,69 @@
from __future__ import annotations
import os
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import Sequence
from .redaction import redact_text
class CommandExecutionError(RuntimeError):
def __init__(self, message: str, *, output: str = ""):
super().__init__(message)
self.output = redact_text(output)
@dataclass(frozen=True, slots=True)
class CommandResult:
returncode: int
stdout: str
stderr: str
@property
def output(self) -> str:
return redact_text("\n".join(part for part in (self.stdout, self.stderr) if part))
class SubprocessRunner:
"""Injectable no-shell subprocess boundary."""
def run(
self,
argv: Sequence[str],
*,
timeout: int,
cwd: Path | None = None,
input_text: str | None = None,
) -> CommandResult:
if not argv or any(not isinstance(arg, str) or "\x00" in arg for arg in argv):
raise CommandExecutionError("Refusing to execute invalid argv.")
env = os.environ.copy()
env.update({"PIP_NO_INPUT": "1", "PYTHONUNBUFFERED": "1"})
try:
completed = subprocess.run(
list(argv),
cwd=str(cwd) if cwd else None,
env=env,
shell=False,
check=False,
text=True,
input=input_text,
stdin=subprocess.DEVNULL if input_text is None else None,
capture_output=True,
timeout=timeout,
)
except subprocess.TimeoutExpired as exc:
output = "\n".join(
str(value) for value in (getattr(exc, "stdout", ""), getattr(exc, "stderr", "")) if value
)
raise CommandExecutionError("Command timed out.", output=output) from exc
except OSError as exc:
raise CommandExecutionError(f"Unable to start command: {type(exc).__name__}") from exc
result = CommandResult(completed.returncode, completed.stdout or "", completed.stderr or "")
if completed.returncode != 0:
raise CommandExecutionError(
f"Command failed with exit code {completed.returncode}.", output=result.output
)
return result
@@ -0,0 +1,237 @@
from __future__ import annotations
import ast
import hashlib
import os
import stat
import tempfile
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from uuid import uuid4
from packaging.requirements import InvalidRequirement, Requirement
from packaging.utils import canonicalize_name
from .validation import validate_distribution_name, validate_import_name
class EditorError(RuntimeError):
pass
@dataclass(frozen=True, slots=True)
class MutationReceipt:
path: Path
changed: bool
backup_path: Path | None
existed_before: bool
previous_text: str
before_sha256: str
after_sha256: str
class AtomicTextEditor:
def __init__(self, path: Path, backup_dir: Path, backups_to_keep: int):
raw_path = Path(path).expanduser()
if raw_path.exists():
if not raw_path.is_file():
raise EditorError(f"Managed path is not a regular file: {raw_path}")
self.path = raw_path.resolve(strict=True)
else:
self.path = raw_path.parent.resolve(strict=True) / raw_path.name
self.backup_dir = Path(backup_dir)
self.backups_to_keep = backups_to_keep
def read(self, *, required: bool = False) -> str:
if not self.path.exists():
if required:
raise EditorError(f"Managed file does not exist: {self.path}")
return ""
try:
return self.path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError) as exc:
raise EditorError(f"Unable to read managed file: {self.path.name}") from exc
def apply(self, new_text: str, *, required: bool = False) -> MutationReceipt:
old_text = self.read(required=required)
existed = self.path.exists()
before = hashlib.sha256(old_text.encode()).hexdigest()
after = hashlib.sha256(new_text.encode()).hexdigest()
if old_text == new_text:
return MutationReceipt(self.path, False, None, existed, old_text, before, after)
backup = self._backup(old_text) if existed else None
self._atomic_write(new_text)
self._prune_backups()
return MutationReceipt(self.path, True, backup, existed, old_text, before, after)
def rollback(self, receipt: MutationReceipt) -> None:
if not receipt.changed:
return
if receipt.existed_before:
self._atomic_write(receipt.previous_text)
else:
self.path.unlink(missing_ok=True)
def _backup(self, text: str) -> Path:
self.backup_dir.mkdir(mode=0o700, parents=True, exist_ok=True)
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%S.%fZ")
backup = self.backup_dir / f"{self.path.name}.{stamp}.{uuid4().hex}.bak"
try:
with backup.open("x", encoding="utf-8", newline="") as handle:
handle.write(text)
handle.flush()
os.fsync(handle.fileno())
os.chmod(backup, 0o600)
except OSError as exc:
backup.unlink(missing_ok=True)
raise EditorError(f"Unable to create backup for {self.path.name}") from exc
return backup
def _atomic_write(self, text: str) -> None:
existing_stat = self.path.stat() if self.path.exists() else None
mode = stat.S_IMODE(existing_stat.st_mode) if existing_stat else 0o640
fd, temp_name = tempfile.mkstemp(prefix=f".{self.path.name}.", dir=self.path.parent)
temp_path = Path(temp_name)
try:
with os.fdopen(fd, "w", encoding="utf-8", newline="") as handle:
handle.write(text)
handle.flush()
os.fsync(handle.fileno())
os.chmod(temp_path, mode)
if existing_stat is not None and hasattr(os, "chown"):
try:
os.chown(temp_path, existing_stat.st_uid, existing_stat.st_gid)
except PermissionError:
pass
os.replace(temp_path, self.path)
if os.name != "nt":
directory_fd = os.open(self.path.parent, os.O_RDONLY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
except OSError as exc:
temp_path.unlink(missing_ok=True)
raise EditorError(f"Unable to atomically update {self.path.name}") from exc
def _prune_backups(self) -> None:
pattern = f"{self.path.name}.*.bak"
backups = sorted(self.backup_dir.glob(pattern), key=lambda path: path.stat().st_mtime, reverse=True)
for old_backup in backups[self.backups_to_keep :]:
old_backup.unlink(missing_ok=True)
class PluginConfigurationEditor(AtomicTextEditor):
def enabled_plugins(self) -> list[str]:
text = self.read(required=True)
_, plugins = self._find_plugins_assignment(text)
return plugins
def preview(self, import_name: str, enabled: bool) -> tuple[str, list[str], bool]:
import_name = validate_import_name(import_name)
text = self.read(required=True)
node, plugins = self._find_plugins_assignment(text)
changed = False
if enabled and import_name not in plugins:
plugins.append(import_name)
changed = True
elif not enabled and import_name in plugins:
plugins = [plugin for plugin in plugins if plugin != import_name]
changed = True
if not changed:
return text, plugins, False
newline = "\r\n" if "\r\n" in text else "\n"
lines = text.splitlines(keepends=True)
if node.lineno == node.end_lineno and ";" in lines[node.lineno - 1]:
raise EditorError("PLUGINS assignment sharing a line cannot be safely edited.")
replacement = [f"PLUGINS = [{newline}"]
replacement.extend(f" {plugin!r},{newline}" for plugin in plugins)
replacement.append(f"]{newline}")
new_text = "".join(lines[: node.lineno - 1] + replacement + lines[node.end_lineno :])
return new_text, plugins, True
def set_enabled(self, import_name: str, enabled: bool) -> MutationReceipt:
new_text, _, _ = self.preview(import_name, enabled)
return self.apply(new_text, required=True)
@staticmethod
def _find_plugins_assignment(text: str) -> tuple[ast.Assign | ast.AnnAssign, list[str]]:
try:
tree = ast.parse(text)
except SyntaxError as exc:
raise EditorError("configuration.py is not valid Python; refusing to edit it.") from exc
matches: list[ast.Assign | ast.AnnAssign] = []
for node in tree.body:
if isinstance(node, ast.Assign) and any(
isinstance(target, ast.Name) and target.id == "PLUGINS" for target in node.targets
):
matches.append(node)
elif isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name) and node.target.id == "PLUGINS":
matches.append(node)
if len(matches) != 1:
raise EditorError("configuration.py must contain exactly one static PLUGINS assignment.")
node = matches[0]
try:
value = ast.literal_eval(node.value)
except (ValueError, TypeError, SyntaxError) as exc:
raise EditorError("PLUGINS must be a literal list or tuple of import names.") from exc
if not isinstance(value, (list, tuple)) or any(not isinstance(item, str) for item in value):
raise EditorError("PLUGINS must be a literal list or tuple of import names.")
plugins = [validate_import_name(item) for item in value]
if len(set(plugins)) != len(plugins):
raise EditorError("PLUGINS contains duplicate entries; refusing to edit it.")
return node, plugins
class RequirementsEditor(AtomicTextEditor):
def preview(self, package_name: str, requirement_line: str | None) -> tuple[str, bool]:
package_name = validate_distribution_name(package_name)
wanted = canonicalize_name(package_name)
text = self.read(required=False)
lines = text.splitlines(keepends=True)
indexes: list[int] = []
for index, line in enumerate(lines):
stripped = line.strip()
if not stripped or stripped.startswith("#") or stripped.startswith(("-r", "--", "-e")):
continue
candidate = stripped.split(" #", 1)[0].rstrip()
try:
requirement = Requirement(candidate)
except InvalidRequirement:
continue
if canonicalize_name(requirement.name) == wanted:
indexes.append(index)
if len(indexes) > 1:
raise EditorError("local_requirements.txt contains duplicate entries for this package.")
newline = "\r\n" if "\r\n" in text else "\n"
replacement = f"{requirement_line}{newline}" if requirement_line else None
if requirement_line:
try:
parsed = Requirement(requirement_line)
except InvalidRequirement as exc:
raise EditorError("Generated requirement is invalid.") from exc
if canonicalize_name(parsed.name) != wanted:
raise EditorError("Generated requirement targets the wrong distribution.")
if indexes:
index = indexes[0]
if replacement is None:
del lines[index]
elif lines[index].rstrip("\r\n") == requirement_line:
return text, False
else:
lines[index] = replacement
elif replacement is not None:
if text and not text.endswith(("\n", "\r")):
lines.append(newline)
lines.append(replacement)
else:
return text, False
return "".join(lines), True
def set_requirement(self, package_name: str, requirement_line: str | None) -> MutationReceipt:
new_text, _ = self.preview(package_name, requirement_line)
return self.apply(new_text, required=False)
@@ -0,0 +1,69 @@
from __future__ import annotations
from django import forms
from .client import CatalogPlugin
from .runtime import RuntimeSettings
class LifecycleConfirmForm(forms.Form):
confirmation = forms.CharField(
max_length=64,
label="Plugin-Slug zur Bestätigung",
help_text="Diese Eingabe verhindert versehentliche Lifecycle-Aktionen.",
)
version = forms.ChoiceField(required=False, label="Version")
dry_run = forms.BooleanField(
required=False,
initial=True,
label="Nur prüfen (Dry-Run)",
help_text="Plant und validiert den Vorgang, ohne Dateien oder Prozesse zu verändern.",
)
def __init__(
self,
*args,
plugin: CatalogPlugin,
action: str,
runtime: RuntimeSettings,
**kwargs,
):
super().__init__(*args, **kwargs)
self.plugin = plugin
self.action = action
self.runtime = runtime
self.fields["confirmation"].widget.attrs.update({"autocomplete": "off", "placeholder": plugin.slug})
self.fields["dry_run"].initial = runtime.default_dry_run
if action in {"install", "update"}:
choices = [
(release.version, release.version)
for release in plugin.releases
if (
release.supports(runtime.netbox_version, plugin)
and release.approved
and release.immutable
and bool(release.sha256)
and (runtime.execution_mode != "agent" or bool(release.approved_payload_sha256))
)
]
choices.sort(reverse=True)
self.fields["version"].choices = choices
self.fields["version"].required = True
self.fields["version"].initial = plugin.latest_version
else:
self.fields.pop("version")
for field in self.fields.values():
if not isinstance(field.widget, forms.CheckboxInput):
field.widget.attrs.setdefault("class", "form-control")
def clean_confirmation(self) -> str:
value = self.cleaned_data["confirmation"]
if value != self.plugin.slug:
raise forms.ValidationError("Der Slug stimmt nicht exakt überein.")
return value
def clean(self):
cleaned = super().clean()
if self.runtime.execution_mode == "dry_run" and not cleaned.get("dry_run", False):
self.add_error("dry_run", "Reale Aktionen sind in execution_mode=dry_run deaktiviert.")
return cleaned
+36
View File
@@ -0,0 +1,36 @@
from __future__ import annotations
from core.exceptions import JobFailed
from netbox.jobs import JobRunner
from .lifecycle import LifecycleRequest, build_service
class PluginLifecycleJob(JobRunner):
"""Background execution is intentionally restricted to non-mutating plans."""
class Meta:
name = "Plugin Store dry-run"
def run(
self,
*,
audit_id: int,
slug: str,
action: str,
version: str = "",
dry_run: bool = True,
actor_id: int | None = None,
**kwargs,
):
if not dry_run:
raise JobFailed("Mutating lifecycle actions may not run inside netbox-rq.")
try:
build_service().execute(
LifecycleRequest(slug=slug, action=action, version=version, dry_run=True),
actor_id=actor_id,
audit_id=audit_id,
)
except Exception as exc:
self.logger.error("Plugin Store dry-run failed: %s", exc)
raise JobFailed(str(exc)) from exc
@@ -0,0 +1,537 @@
from __future__ import annotations
import importlib.metadata
import re
import tempfile
from dataclasses import asdict, dataclass, field
from pathlib import Path
from typing import Any, Callable, Protocol
from urllib.parse import urlsplit, urlunsplit
from uuid import uuid4
from packaging.version import InvalidVersion, Version
from .agent import AgentClient
from .client import CatalogPlugin, Release, StoreClient
from .commands import SubprocessRunner
from .editors import MutationReceipt, PluginConfigurationEditor, RequirementsEditor
from .locking import FileLock
from .redaction import redact_data, redact_text
from .runtime import RuntimeSettings
from .validation import ensure_not_self, validate_slug
ACTIONS = ("install", "update", "enable", "disable", "uninstall")
MUTATING_ACTIONS = frozenset(ACTIONS)
class LifecycleError(RuntimeError):
pass
class LifecycleRepository(Protocol):
def create_audit(self, request: "LifecycleRequest", actor_id: int | None) -> int: ...
def mark_running(self, audit_id: int) -> None: ...
def mark_success(self, audit_id: int, result: dict[str, Any]) -> None: ...
def mark_handed_off(self, audit_id: int, operation_id: str, result: dict[str, Any]) -> None: ...
def mark_failed(self, audit_id: int, error: str, result: dict[str, Any]) -> None: ...
def update_status(self, plugin: CatalogPlugin, **values: Any) -> None: ...
def has_other_pending_mutation(self, slug: str, audit_id: int) -> bool: ...
@dataclass(frozen=True, slots=True)
class LifecycleRequest:
slug: str
action: str
version: str = ""
dry_run: bool = True
def validate(self) -> None:
validate_slug(self.slug)
if self.action not in ACTIONS:
raise LifecycleError("Unknown lifecycle action.")
if self.version:
try:
Version(self.version)
except InvalidVersion as exc:
raise LifecycleError("Invalid requested version.") from exc
@dataclass(slots=True)
class LifecycleResult:
slug: str
action: str
state: str
dry_run: bool
installed_version: str = ""
enabled: bool = False
restart_required: bool = False
handed_to_agent: bool = False
operation_id: str = ""
plan: list[str] = field(default_factory=list)
output: str = ""
def safe_dict(self) -> dict[str, Any]:
return redact_data(asdict(self))
def installed_distribution_version(package_name: str) -> str:
try:
return importlib.metadata.version(package_name)
except importlib.metadata.PackageNotFoundError:
return ""
def runtime_plugin_is_loaded(import_name: str) -> bool:
try:
from django.conf import settings as django_settings
if not django_settings.configured:
return False
return import_name in django_settings.PLUGINS
except (ImportError, AttributeError, RuntimeError):
return False
class LifecycleService:
def __init__(
self,
settings: RuntimeSettings,
client: StoreClient,
repository: LifecycleRepository,
*,
runner: SubprocessRunner | None = None,
version_provider: Callable[[str], str] = installed_distribution_version,
runtime_active_provider: Callable[[str], bool] = runtime_plugin_is_loaded,
):
self.settings = settings
self.client = client
self.repository = repository
self.runner = runner or SubprocessRunner()
self.version_provider = version_provider
self.runtime_active_provider = runtime_active_provider
def execute(
self,
request: LifecycleRequest,
*,
actor_id: int | None = None,
audit_id: int | None = None,
) -> LifecycleResult:
request.validate()
if audit_id is None:
audit_id = self.repository.create_audit(request, actor_id)
self.repository.mark_running(audit_id)
result: LifecycleResult | None = None
try:
if request.dry_run:
result = self._execute_locked(request, mutate=False, requested_by=actor_id)
else:
if self.settings.execution_mode == "dry_run":
raise LifecycleError(
"Real lifecycle changes are disabled while execution_mode is dry_run."
)
if self.settings.execution_mode == "direct" and not self.settings.allow_lifecycle_mutations:
raise LifecycleError(
"Direct lifecycle changes require the explicit allow_lifecycle_mutations opt-in."
)
with FileLock(self.settings.lock_path, self.settings.lock_timeout):
if self.repository.has_other_pending_mutation(request.slug, audit_id):
raise LifecycleError("Another lifecycle operation for this plugin is still pending.")
result = self._execute_locked(request, mutate=True, requested_by=actor_id)
if result.handed_to_agent:
self.repository.mark_handed_off(audit_id, result.operation_id, result.safe_dict())
else:
self.repository.mark_success(audit_id, result.safe_dict())
return result
except Exception as exc:
safe_error = redact_text(exc)
safe_result = result.safe_dict() if result else {"slug": request.slug, "action": request.action}
self.repository.mark_failed(audit_id, safe_error, safe_result)
raise
def _execute_locked(
self, request: LifecycleRequest, *, mutate: bool, requested_by: int | None
) -> LifecycleResult:
plugin = self.client.get_plugin(request.slug)
ensure_not_self(plugin.package_name, plugin.import_name)
installed = self.version_provider(plugin.package_name)
config_editor = PluginConfigurationEditor(
self.settings.configuration_path,
self.settings.backup_dir,
self.settings.backups_to_keep,
)
requirements_editor = RequirementsEditor(
self.settings.requirements_path,
self.settings.backup_dir,
self.settings.backups_to_keep,
)
enabled = plugin.import_name in config_editor.enabled_plugins()
runtime_active = self.runtime_active_provider(plugin.import_name)
release: Release | None = None
if request.action in {"install", "update"}:
release = plugin.select_release(self.settings.netbox_version, request.version)
self._assert_installable(plugin, release)
self._assert_state(request.action, installed, enabled, runtime_active)
plan = self._build_plan(request.action, plugin, release, enabled)
if not mutate:
return LifecycleResult(
slug=plugin.slug,
action=request.action,
state="dry-run",
dry_run=True,
installed_version=installed,
enabled=enabled,
restart_required=(
request.action in {"enable", "disable"}
or (request.action == "update" and enabled)
),
plan=plan,
)
self.repository.update_status(
plugin,
state="running",
installed_version=installed,
enabled=enabled,
restart_required=False,
last_error="",
)
try:
if self.settings.execution_mode == "agent":
result = self._execute_agent(request, plugin, release, installed, enabled, plan, requested_by)
else:
result = self._execute_direct(
request, plugin, release, installed, enabled, plan, config_editor, requirements_editor
)
except Exception as exc:
self.repository.update_status(
plugin,
state="failed",
installed_version=self.version_provider(plugin.package_name) or installed,
enabled=enabled,
restart_required=True,
last_error=redact_text(exc, limit=4_000),
)
raise
self.repository.update_status(
plugin,
state=result.state,
installed_version=result.installed_version,
enabled=result.enabled,
restart_required=result.restart_required,
last_error="",
)
return result
@staticmethod
def _assert_installable(plugin: CatalogPlugin, release: Release) -> None:
if not plugin.approved:
raise LifecycleError("Plugin is not explicitly approved by the Store.")
if not release.approved or not release.immutable:
raise LifecycleError("Release is not both approved and immutable.")
if not release.download_url:
raise LifecycleError("Approved release has no artifact URL.")
if not release.sha256:
raise LifecycleError("Approved release has no artifact SHA-256. Commit hashes are not accepted.")
@staticmethod
def _assert_state(action: str, installed: str, enabled: bool, runtime_active: bool) -> None:
if action == "install" and installed:
raise LifecycleError("Plugin is already installed; use update.")
if action in {"update", "enable", "disable", "uninstall"} and not installed:
raise LifecycleError("Plugin is not installed.")
if action == "enable" and enabled:
raise LifecycleError("Plugin is already enabled.")
if action == "disable" and not enabled:
raise LifecycleError("Plugin is already disabled.")
if action == "uninstall" and (enabled or runtime_active):
raise LifecycleError("Disable the plugin and restart NetBox before uninstalling it.")
def _build_plan(
self, action: str, plugin: CatalogPlugin, release: Release | None, currently_enabled: bool
) -> list[str]:
plan: list[str] = []
if release:
plan.extend(
[
f"Download immutable release {release.version} from an allowlisted origin.",
f"Verify artifact SHA-256 {release.sha256} before executing pip.",
]
)
if action == "install":
plan.extend(["Install the verified local artifact with pip.", "Pin local_requirements.txt atomically."])
plan.append("Keep the newly installed plugin disabled until an explicit enable action.")
elif action == "update":
plan.extend(["Update the persistent requirement pin atomically.", "Upgrade from the verified local artifact."])
elif action == "enable":
plan.append(f"Add {plugin.import_name} to the static PLUGINS list atomically.")
elif action == "disable":
plan.append(f"Remove {plugin.import_name} from the static PLUGINS list atomically.")
elif action == "uninstall":
plan.extend(["Remove the persistent requirement pin atomically.", "Uninstall the distribution with pip."])
if action == "enable" or (action == "update" and currently_enabled):
if self.settings.run_migrations:
plan.append("Run NetBox database migrations without interaction.")
if self.settings.collect_static:
plan.append("Collect NetBox static files without interaction.")
if action in {"enable", "disable"} or (action == "update" and currently_enabled):
plan.append("Require a NetBox/web and worker restart before the new state is fully active.")
return plan
def _artifact_suffix(self, release: Release) -> str:
filename = Path(urlsplit(release.download_url).path).name
suffixes = Path(filename).suffixes[-2:]
suffix = "".join(suffixes) or ".artifact"
return suffix if re.fullmatch(r"\.[A-Za-z0-9.]{1,16}", suffix) else ".artifact"
def _download(self, release: Release, directory: Path) -> Path:
artifact = directory / f"artifact{self._artifact_suffix(release)}"
self.client.download_artifact(
release.download_url,
artifact,
expected_sha256=release.sha256,
timeout=self.settings.download_timeout,
max_bytes=self.settings.max_download_bytes,
)
return artifact
def _execute_agent(
self,
request: LifecycleRequest,
plugin: CatalogPlugin,
release: Release | None,
installed: str,
enabled: bool,
plan: list[str],
requested_by: int | None,
) -> LifecycleResult:
if self.settings.agent_socket_path is None:
raise LifecycleError("Host agent socket is not configured.")
if release is not None and not release.approved_payload_sha256:
raise LifecycleError("Store release is missing its opaque approved_payload_sha256 marker.")
agent = AgentClient(self.settings.agent_socket_path, timeout=self.settings.agent_timeout)
operation_id = agent.submit_operation(
request_id=str(uuid4()),
action=request.action,
plugin_slug=plugin.slug,
version=release.version if release else (request.version or None),
approved_payload_sha256=release.approved_payload_sha256 if release else None,
requested_by=f"netbox-user:{requested_by}" if requested_by is not None else "netbox-system",
)
return LifecycleResult(
slug=plugin.slug,
action=request.action,
state="handed-off",
dry_run=False,
installed_version=installed,
enabled=enabled,
restart_required=(
request.action in {"enable", "disable"} or (request.action == "update" and enabled)
),
handed_to_agent=True,
operation_id=str(operation_id),
plan=plan,
output="Operation accepted by the host agent.",
)
def _execute_direct(
self,
request: LifecycleRequest,
plugin: CatalogPlugin,
release: Release | None,
installed: str,
enabled: bool,
plan: list[str],
config_editor: PluginConfigurationEditor,
requirements_editor: RequirementsEditor,
) -> LifecycleResult:
output: list[str] = []
with tempfile.TemporaryDirectory(prefix="netbox-plugin-store-") as temp_name:
artifact = self._download(release, Path(temp_name)) if release else None
if request.action == "install":
new_version, new_enabled = self._direct_install(
plugin, release, artifact, requirements_editor, output
)
elif request.action == "update":
new_version, new_enabled = self._direct_update(
plugin, release, artifact, enabled, requirements_editor, output
)
elif request.action == "enable":
self._direct_enable(plugin, config_editor, output)
new_version, new_enabled = installed, True
elif request.action == "disable":
config_editor.set_enabled(plugin.import_name, False)
new_version, new_enabled = installed, False
else:
self._direct_uninstall(plugin, enabled, config_editor, requirements_editor, output)
new_version, new_enabled = "", False
return LifecycleResult(
slug=plugin.slug,
action=request.action,
state=(
"restart-required"
if request.action in {"enable", "disable"} or (request.action == "update" and enabled)
else ("installed" if new_version else "unknown")
),
dry_run=False,
installed_version=new_version,
enabled=new_enabled,
restart_required=(
request.action in {"enable", "disable"} or (request.action == "update" and enabled)
),
plan=plan,
output=redact_text("\n".join(output)),
)
def _pip_install(self, artifact: Path, *, upgrade: bool) -> str:
argv = [
self.settings.python_executable,
"-m",
"pip",
"install",
"--disable-pip-version-check",
"--no-input",
*self.settings.pip_extra_args,
]
if upgrade:
argv.append("--upgrade")
if not self.settings.allow_package_index:
argv.append("--no-index")
argv.append(str(artifact))
return self.runner.run(argv, timeout=self.settings.operation_timeout).output
def _pip_uninstall(self, package_name: str) -> str:
argv = [self.settings.python_executable, "-m", "pip", "uninstall", "--yes", package_name]
return self.runner.run(argv, timeout=self.settings.operation_timeout).output
def _pip_check(self) -> str:
argv = [self.settings.python_executable, "-m", "pip", "check"]
return self.runner.run(argv, timeout=self.settings.operation_timeout).output
@staticmethod
def _requirement_line(plugin: CatalogPlugin, release: Release) -> str:
parsed = urlsplit(release.download_url)
fragment = f"sha256={release.sha256}"
if parsed.fragment:
fragment = parsed.fragment + "&" + fragment
pinned_url = urlunsplit((parsed.scheme, parsed.netloc, parsed.path, parsed.query, fragment))
return f"{plugin.package_name} @ {pinned_url}"
def _manage(self, command: str) -> str:
if not self.settings.manage_path.is_file():
raise LifecycleError("NetBox manage.py path does not exist; configure manage_path.")
argv = [self.settings.python_executable, str(self.settings.manage_path), command, "--no-input"]
return self.runner.run(
argv, timeout=self.settings.operation_timeout, cwd=self.settings.manage_path.parent
).output
def _post_enable_steps(self, output: list[str]) -> None:
if self.settings.run_migrations:
output.append(self._manage("migrate"))
if self.settings.collect_static:
output.append(self._manage("collectstatic"))
def _direct_install(
self,
plugin: CatalogPlugin,
release: Release,
artifact: Path,
requirements_editor: RequirementsEditor,
output: list[str],
) -> tuple[str, bool]:
requirement_receipt: MutationReceipt | None = None
pip_installed = False
try:
output.append(self._pip_install(artifact, upgrade=False))
pip_installed = True
output.append(self._pip_check())
if self.settings.manage_requirements_file:
requirement_receipt = requirements_editor.set_requirement(
plugin.package_name, self._requirement_line(plugin, release)
)
except Exception:
if requirement_receipt:
requirements_editor.rollback(requirement_receipt)
if pip_installed:
try:
self._pip_uninstall(plugin.package_name)
except Exception:
pass
raise
return release.version, False
def _direct_update(
self,
plugin: CatalogPlugin,
release: Release,
artifact: Path,
enabled: bool,
requirements_editor: RequirementsEditor,
output: list[str],
) -> tuple[str, bool]:
receipt: MutationReceipt | None = None
pip_updated = False
try:
if self.settings.manage_requirements_file:
receipt = requirements_editor.set_requirement(
plugin.package_name, self._requirement_line(plugin, release)
)
output.append(self._pip_install(artifact, upgrade=True))
pip_updated = True
output.append(self._pip_check())
if enabled:
self._post_enable_steps(output)
except Exception:
if receipt and not pip_updated:
requirements_editor.rollback(receipt)
raise
return release.version, enabled
def _direct_enable(
self, plugin: CatalogPlugin, config_editor: PluginConfigurationEditor, output: list[str]
) -> None:
receipt = config_editor.set_enabled(plugin.import_name, True)
try:
self._post_enable_steps(output)
except Exception:
config_editor.rollback(receipt)
raise
def _direct_uninstall(
self,
plugin: CatalogPlugin,
enabled: bool,
config_editor: PluginConfigurationEditor,
requirements_editor: RequirementsEditor,
output: list[str],
) -> None:
config_receipt: MutationReceipt | None = None
requirement_receipt: MutationReceipt | None = None
try:
if self.settings.manage_requirements_file:
requirement_receipt = requirements_editor.set_requirement(plugin.package_name, None)
output.append(self._pip_uninstall(plugin.package_name))
except Exception:
if config_receipt:
config_editor.rollback(config_receipt)
if requirement_receipt:
requirements_editor.rollback(requirement_receipt)
raise
def build_service(*, repository: LifecycleRepository | None = None) -> LifecycleService:
settings = RuntimeSettings.from_django()
client = StoreClient(
settings.store_url,
settings.allowed_store_urls,
settings.allowed_artifact_urls,
api_token=settings.api_token,
timeout=settings.request_timeout,
)
if repository is None:
from .repository import DjangoLifecycleRepository
repository = DjangoLifecycleRepository()
return LifecycleService(settings, client, repository)
@@ -0,0 +1,65 @@
from __future__ import annotations
import os
import time
from pathlib import Path
class LockTimeoutError(TimeoutError):
pass
class FileLock:
"""Small cross-platform inter-process exclusive lock."""
def __init__(self, path: Path, timeout: float = 30):
self.path = Path(path)
self.timeout = timeout
self._file = None
def __enter__(self):
self.path.parent.mkdir(parents=True, exist_ok=True)
self._file = self.path.open("a+b")
deadline = time.monotonic() + self.timeout
while True:
try:
self._acquire()
return self
except (BlockingIOError, OSError):
if time.monotonic() >= deadline:
self._file.close()
self._file = None
raise LockTimeoutError("Another plugin lifecycle operation is still running.")
time.sleep(0.1)
def _acquire(self) -> None:
if os.name == "nt":
import msvcrt
self._file.seek(0)
if self._file.read(1) == b"":
self._file.write(b"\0")
self._file.flush()
self._file.seek(0)
msvcrt.locking(self._file.fileno(), msvcrt.LK_NBLCK, 1)
else:
import fcntl
fcntl.flock(self._file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
def __exit__(self, exc_type, exc, tb):
if self._file is None:
return
try:
if os.name == "nt":
import msvcrt
self._file.seek(0)
msvcrt.locking(self._file.fileno(), msvcrt.LK_UNLCK, 1)
else:
import fcntl
fcntl.flock(self._file.fileno(), fcntl.LOCK_UN)
finally:
self._file.close()
self._file = None
@@ -0,0 +1,101 @@
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
initial = True
dependencies = [migrations.swappable_dependency(settings.AUTH_USER_MODEL)]
operations = [
migrations.CreateModel(
name="ManagedPlugin",
fields=[
("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
("slug", models.SlugField(max_length=64, unique=True)),
("name", models.CharField(max_length=200)),
("package_name", models.CharField(max_length=128)),
("import_name", models.CharField(max_length=128)),
("repository_url", models.URLField(blank=True, max_length=500)),
("installed_version", models.CharField(blank=True, max_length=64)),
("available_version", models.CharField(blank=True, max_length=64)),
("enabled", models.BooleanField(default=False)),
("restart_required", models.BooleanField(default=False)),
(
"state",
models.CharField(
choices=[
("unknown", "Unknown"),
("running", "Operation running"),
("installed", "Installed"),
("enabled", "Enabled"),
("disabled", "Disabled"),
("restart-required", "Restart required"),
("handed-off", "Handed to agent"),
("failed", "Failed"),
],
default="unknown",
max_length=32,
),
),
("last_error", models.TextField(blank=True)),
("last_checked", models.DateTimeField(blank=True, null=True)),
("created", models.DateTimeField(auto_now_add=True)),
("updated", models.DateTimeField(auto_now=True)),
],
options={
"ordering": ("name", "slug"),
"permissions": (("manage_plugin", "Can execute Plugin Store lifecycle actions"),),
},
),
migrations.CreateModel(
name="LifecycleAudit",
fields=[
("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
("slug", models.SlugField(max_length=64)),
("action", models.CharField(max_length=16)),
("requested_version", models.CharField(blank=True, max_length=64)),
("dry_run", models.BooleanField(default=True)),
(
"status",
models.CharField(
choices=[
("queued", "Queued"),
("running", "Running"),
("succeeded", "Succeeded"),
("failed", "Failed"),
("handed-off", "Handed to host agent"),
],
default="queued",
max_length=16,
),
),
("request_data", models.JSONField(blank=True, default=dict)),
("result_data", models.JSONField(blank=True, default=dict)),
("error", models.TextField(blank=True)),
("external_operation_id", models.UUIDField(blank=True, null=True, unique=True)),
("created", models.DateTimeField(auto_now_add=True)),
("started", models.DateTimeField(blank=True, null=True)),
("completed", models.DateTimeField(blank=True, null=True)),
(
"actor",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="+",
to=settings.AUTH_USER_MODEL,
),
),
],
options={
"ordering": ("-created",),
"permissions": (("execute_plugin_lifecycle", "Can request Plugin Store lifecycle execution"),),
},
),
migrations.AddIndex(
model_name="lifecycleaudit",
index=models.Index(fields=["slug", "-created"], name="nbps_audit_slug_created"),
),
]
@@ -0,0 +1 @@
@@ -0,0 +1,79 @@
from __future__ import annotations
from django.conf import settings
from django.db import models
class ManagedPlugin(models.Model):
class State(models.TextChoices):
UNKNOWN = "unknown", "Unknown"
RUNNING = "running", "Operation running"
INSTALLED = "installed", "Installed"
ENABLED = "enabled", "Enabled"
DISABLED = "disabled", "Disabled"
RESTART_REQUIRED = "restart-required", "Restart required"
HANDED_OFF = "handed-off", "Handed to agent"
FAILED = "failed", "Failed"
slug = models.SlugField(max_length=64, unique=True)
name = models.CharField(max_length=200)
package_name = models.CharField(max_length=128)
import_name = models.CharField(max_length=128)
repository_url = models.URLField(max_length=500, blank=True)
installed_version = models.CharField(max_length=64, blank=True)
available_version = models.CharField(max_length=64, blank=True)
enabled = models.BooleanField(default=False)
restart_required = models.BooleanField(default=False)
state = models.CharField(max_length=32, choices=State.choices, default=State.UNKNOWN)
last_error = models.TextField(blank=True)
last_checked = models.DateTimeField(null=True, blank=True)
created = models.DateTimeField(auto_now_add=True)
updated = models.DateTimeField(auto_now=True)
_netbox_private = True
class Meta:
ordering = ("name", "slug")
permissions = (("manage_plugin", "Can execute Plugin Store lifecycle actions"),)
def __str__(self) -> str:
return self.name
class LifecycleAudit(models.Model):
class Status(models.TextChoices):
QUEUED = "queued", "Queued"
RUNNING = "running", "Running"
SUCCEEDED = "succeeded", "Succeeded"
FAILED = "failed", "Failed"
HANDED_OFF = "handed-off", "Handed to host agent"
actor = models.ForeignKey(
settings.AUTH_USER_MODEL,
on_delete=models.SET_NULL,
related_name="+",
null=True,
blank=True,
)
slug = models.SlugField(max_length=64)
action = models.CharField(max_length=16)
requested_version = models.CharField(max_length=64, blank=True)
dry_run = models.BooleanField(default=True)
status = models.CharField(max_length=16, choices=Status.choices, default=Status.QUEUED)
request_data = models.JSONField(default=dict, blank=True)
result_data = models.JSONField(default=dict, blank=True)
error = models.TextField(blank=True)
external_operation_id = models.UUIDField(null=True, blank=True, unique=True)
created = models.DateTimeField(auto_now_add=True)
started = models.DateTimeField(null=True, blank=True)
completed = models.DateTimeField(null=True, blank=True)
_netbox_private = True
class Meta:
ordering = ("-created",)
indexes = (models.Index(fields=("slug", "-created"), name="nbps_audit_slug_created"),)
permissions = (("execute_plugin_lifecycle", "Can request Plugin Store lifecycle execution"),)
def __str__(self) -> str:
return f"{self.action} {self.slug} ({self.status})"
@@ -0,0 +1,34 @@
from netbox.plugins import PluginMenu, PluginMenuItem
_permission = "netbox_plugin_store.manage_plugin"
menu = PluginMenu(
label="Plugin Store",
icon_class="mdi mdi-store",
groups=(
(
"Store",
(
PluginMenuItem(
link="plugins:netbox_plugin_store:catalog",
link_text="Katalog",
auth_required=True,
permissions=[_permission],
),
PluginMenuItem(
link="plugins:netbox_plugin_store:status",
link_text="Installierte Plugins",
auth_required=True,
permissions=[_permission],
),
PluginMenuItem(
link="plugins:netbox_plugin_store:audit-list",
link_text="Audit-Protokoll",
auth_required=True,
permissions=[_permission],
),
),
),
),
)
@@ -0,0 +1,35 @@
from __future__ import annotations
import re
from collections.abc import Mapping, Sequence
_SECRET_KEYS = re.compile(r"token|secret|password|authorization|cookie|api[_-]?key", re.I)
_BEARER = re.compile(r"(?i)\bBearer\s+[A-Za-z0-9._~+\-/]+=*")
_URL_CREDENTIALS = re.compile(r"(?i)(https?://)([^/@\s:]+):([^/@\s]+)@")
_ASSIGNMENT = re.compile(
r"(?i)\b(token|secret|password|authorization|api[_-]?key)\s*([=:])\s*([^\s,;]+)"
)
def redact_text(value: object, *, limit: int = 8_000) -> str:
text = str(value)
text = _BEARER.sub("Bearer [REDACTED]", text)
text = _URL_CREDENTIALS.sub(r"\1[REDACTED]@", text)
text = _ASSIGNMENT.sub(r"\1\2[REDACTED]", text)
if len(text) > limit:
return text[:limit] + "\n...[truncated]"
return text
def redact_data(value: object) -> object:
if isinstance(value, Mapping):
return {
str(key): "[REDACTED]" if _SECRET_KEYS.search(str(key)) else redact_data(item)
for key, item in value.items()
}
if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
return [redact_data(item) for item in value]
if isinstance(value, str):
return redact_text(value)
return value
@@ -0,0 +1,86 @@
from __future__ import annotations
from typing import Any
from django.db import transaction
from django.utils import timezone
from .client import CatalogPlugin
from .lifecycle import LifecycleRequest
from .models import LifecycleAudit, ManagedPlugin
from .redaction import redact_data, redact_text
class DjangoLifecycleRepository:
def has_other_pending_mutation(self, slug: str, audit_id: int) -> bool:
return LifecycleAudit.objects.filter(
slug=slug,
dry_run=False,
status=LifecycleAudit.Status.HANDED_OFF,
).exclude(pk=audit_id).exists()
def create_audit(self, request: LifecycleRequest, actor_id: int | None) -> int:
audit = LifecycleAudit.objects.create(
actor_id=actor_id,
slug=request.slug,
action=request.action,
requested_version=request.version,
dry_run=request.dry_run,
request_data={
"slug": request.slug,
"action": request.action,
"version": request.version,
"dry_run": request.dry_run,
},
)
return audit.pk
def mark_running(self, audit_id: int) -> None:
LifecycleAudit.objects.filter(pk=audit_id).update(
status=LifecycleAudit.Status.RUNNING,
started=timezone.now(),
error="",
)
def mark_success(self, audit_id: int, result: dict[str, Any]) -> None:
LifecycleAudit.objects.filter(pk=audit_id).update(
status=LifecycleAudit.Status.SUCCEEDED,
result_data=redact_data(result),
completed=timezone.now(),
)
def mark_handed_off(self, audit_id: int, operation_id: str, result: dict[str, Any]) -> None:
LifecycleAudit.objects.filter(pk=audit_id).update(
status=LifecycleAudit.Status.HANDED_OFF,
external_operation_id=operation_id,
result_data=redact_data(result),
)
def mark_failed(self, audit_id: int, error: str, result: dict[str, Any]) -> None:
LifecycleAudit.objects.filter(pk=audit_id).update(
status=LifecycleAudit.Status.FAILED,
error=redact_text(error, limit=8_000),
result_data=redact_data(result),
completed=timezone.now(),
)
@transaction.atomic
def update_status(self, plugin: CatalogPlugin, **values: Any) -> None:
safe_values = {
"name": plugin.name,
"package_name": plugin.package_name,
"import_name": plugin.import_name,
"repository_url": plugin.repository_url,
"available_version": plugin.latest_version,
"last_checked": timezone.now(),
**values,
}
if "last_error" in safe_values:
safe_values["last_error"] = redact_text(safe_values["last_error"], limit=4_000)
obj = ManagedPlugin.objects.select_for_update().filter(slug=plugin.slug).first()
if obj is None:
ManagedPlugin.objects.create(slug=plugin.slug, **safe_values)
else:
for key, value in safe_values.items():
setattr(obj, key, value)
obj.save(update_fields=tuple(safe_values) + ("updated",))
@@ -0,0 +1,199 @@
from __future__ import annotations
import os
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any
class RuntimeConfigurationError(RuntimeError):
pass
def _argv_list(value: object, setting: str) -> tuple[tuple[str, ...], ...]:
if not isinstance(value, (list, tuple)):
raise RuntimeConfigurationError(f"{setting} must be a list of argv lists.")
commands: list[tuple[str, ...]] = []
for command in value:
if (
not isinstance(command, (list, tuple))
or not command
or any(not isinstance(arg, str) or "\x00" in arg for arg in command)
):
raise RuntimeConfigurationError(f"Each {setting} entry must be a non-empty argv list.")
commands.append(tuple(command))
return tuple(commands)
def _string_list(value: object, setting: str) -> tuple[str, ...]:
if not isinstance(value, (list, tuple)) or any(not isinstance(v, str) for v in value):
raise RuntimeConfigurationError(f"{setting} must be a list of strings.")
return tuple(value)
@dataclass(frozen=True, slots=True)
class RuntimeSettings:
store_url: str
allowed_store_urls: tuple[str, ...]
allowed_artifact_urls: tuple[str, ...]
api_token: str
request_timeout: int
download_timeout: int
max_download_bytes: int
configuration_path: Path
requirements_path: Path
manage_requirements_file: bool
allow_lifecycle_mutations: bool
default_dry_run: bool
execution_mode: str
agent_socket_path: Path | None
agent_timeout: int
agent_poll_interval: float
manage_path: Path
lock_path: Path
backup_dir: Path
lock_timeout: int
operation_timeout: int
pip_extra_args: tuple[str, ...]
allow_package_index: bool
run_migrations: bool
collect_static: bool
background_jobs: bool
synchronous_fallback: bool
job_queue: str
auto_restart: bool
restart_commands: tuple[tuple[str, ...], ...]
restart_allowlist: tuple[tuple[str, ...], ...]
backups_to_keep: int
python_executable: str
netbox_version: str
@classmethod
def from_mapping(
cls,
raw: dict[str, Any],
*,
configuration_dir: str | os.PathLike[str],
netbox_root: str | os.PathLike[str],
base_dir: str | os.PathLike[str],
netbox_version: str,
) -> "RuntimeSettings":
configuration_dir = Path(configuration_dir)
netbox_root = Path(netbox_root)
base_dir = Path(base_dir)
config_path = Path(raw.get("configuration_path") or configuration_dir / "configuration.py")
requirements_path = Path(raw.get("requirements_path") or netbox_root / "local_requirements.txt")
configured_manage = raw.get("manage_path")
if configured_manage:
manage_path = Path(configured_manage)
else:
candidates = (base_dir / "manage.py", netbox_root / "netbox" / "manage.py", netbox_root / "manage.py")
manage_path = next((candidate for candidate in candidates if candidate.is_file()), candidates[0])
lock_path = Path(raw.get("lock_path") or configuration_dir / ".plugin-store.lock")
backup_dir = Path(raw.get("backup_dir") or configuration_dir / "plugin-store-backups")
restart_commands = _argv_list(raw.get("restart_commands", []), "restart_commands")
restart_allowlist = _argv_list(raw.get("restart_allowlist", []), "restart_allowlist")
if raw.get("auto_restart", False):
if not restart_commands:
raise RuntimeConfigurationError("auto_restart requires at least one restart command.")
denied = [command for command in restart_commands if command not in restart_allowlist]
if denied:
raise RuntimeConfigurationError("Every restart command must exactly match restart_allowlist.")
if str(raw.get("execution_mode", "dry_run")) == "direct":
raise RuntimeConfigurationError("auto_restart is forbidden in direct execution mode; use the host agent.")
pip_extra_args = _string_list(raw.get("pip_extra_args", []), "pip_extra_args")
if any("\x00" in arg for arg in pip_extra_args):
raise RuntimeConfigurationError("pip_extra_args contains an invalid NUL byte.")
settings = cls(
store_url=str(raw.get("store_url") or "").rstrip("/"),
allowed_store_urls=_string_list(raw.get("allowed_store_urls", []), "allowed_store_urls"),
allowed_artifact_urls=_string_list(raw.get("allowed_artifact_urls", []), "allowed_artifact_urls"),
api_token=str(raw.get("api_token", "")),
request_timeout=int(raw.get("request_timeout", 15)),
download_timeout=int(raw.get("download_timeout", 120)),
max_download_bytes=int(raw.get("max_download_bytes", 268_435_456)),
configuration_path=config_path,
requirements_path=requirements_path,
manage_requirements_file=bool(raw.get("manage_requirements_file", True)),
allow_lifecycle_mutations=bool(raw.get("allow_lifecycle_mutations", False)),
default_dry_run=bool(raw.get("default_dry_run", True)),
execution_mode=str(raw.get("execution_mode", "dry_run")),
agent_socket_path=Path(raw["agent_socket_path"]) if raw.get("agent_socket_path") else None,
agent_timeout=int(raw.get("agent_timeout", 30)),
agent_poll_interval=float(raw.get("agent_poll_interval", 1.0)),
manage_path=manage_path,
lock_path=lock_path,
backup_dir=backup_dir,
lock_timeout=int(raw.get("lock_timeout", 30)),
operation_timeout=int(raw.get("operation_timeout", 900)),
pip_extra_args=pip_extra_args,
allow_package_index=bool(raw.get("allow_package_index", False)),
run_migrations=bool(raw.get("run_migrations", True)),
collect_static=bool(raw.get("collect_static", True)),
background_jobs=bool(raw.get("background_jobs", True)),
synchronous_fallback=bool(raw.get("synchronous_fallback", True)),
job_queue=str(raw.get("job_queue", "default")),
auto_restart=bool(raw.get("auto_restart", False)),
restart_commands=restart_commands,
restart_allowlist=restart_allowlist,
backups_to_keep=int(raw.get("backups_to_keep", 25)),
python_executable=sys.executable,
netbox_version=str(netbox_version).split("-")[0],
)
settings.validate()
return settings
@classmethod
def from_django(cls) -> "RuntimeSettings":
from django.conf import settings as django_settings
plugin_settings = dict(django_settings.PLUGINS_CONFIG.get("netbox_plugin_store", {}))
config_cls = __import__("netbox_plugin_store", fromlist=["config"]).config
defaults = dict(config_cls.default_settings)
defaults.update(plugin_settings)
release = getattr(django_settings, "RELEASE", None)
version = getattr(release, "version", None) or django_settings.VERSION
return cls.from_mapping(
defaults,
configuration_dir=django_settings.CONFIGURATION_DIR,
netbox_root=getattr(django_settings, "NETBOX_ROOT", django_settings.BASE_DIR.parent),
base_dir=django_settings.BASE_DIR,
netbox_version=version,
)
def validate(self) -> None:
from .client import URLPolicy
URLPolicy(self.allowed_store_urls).check(self.store_url)
if not self.allowed_artifact_urls:
raise RuntimeConfigurationError("allowed_artifact_urls cannot be empty.")
if self.execution_mode not in {"dry_run", "direct", "agent"}:
raise RuntimeConfigurationError("execution_mode must be 'dry_run', 'direct', or 'agent'.")
if self.execution_mode == "agent":
if self.agent_socket_path is None or not self.agent_socket_path.is_absolute():
raise RuntimeConfigurationError("agent mode requires an absolute agent_socket_path.")
if not 1 <= self.agent_timeout <= 300:
raise RuntimeConfigurationError("agent_timeout must be between 1 and 300 seconds.")
if not 0.1 <= self.agent_poll_interval <= 30:
raise RuntimeConfigurationError("agent_poll_interval must be between 0.1 and 30 seconds.")
if not 1 <= self.request_timeout <= 300:
raise RuntimeConfigurationError("request_timeout must be between 1 and 300 seconds.")
if not 1 <= self.download_timeout <= 3_600:
raise RuntimeConfigurationError("download_timeout must be between 1 and 3600 seconds.")
if not 1_024 <= self.max_download_bytes <= 2_147_483_648:
raise RuntimeConfigurationError("max_download_bytes is outside the accepted range.")
if not 1 <= self.operation_timeout <= 7_200:
raise RuntimeConfigurationError("operation_timeout must be between 1 and 7200 seconds.")
if not 1 <= self.lock_timeout <= 300:
raise RuntimeConfigurationError("lock_timeout must be between 1 and 300 seconds.")
if not 1 <= self.backups_to_keep <= 500:
raise RuntimeConfigurationError("backups_to_keep must be between 1 and 500.")
if not self.job_queue or any(char.isspace() for char in self.job_queue):
raise RuntimeConfigurationError("job_queue is invalid.")
@@ -0,0 +1,28 @@
{% extends 'base/layout.html' %}
{% block title %}Audit #{{ audit.pk }}{% endblock %}
{% block content %}
<a href="{% url 'plugins:netbox_plugin_store:audit-list' %}">← Zum Audit-Protokoll</a>
<h1 class="mt-2">Audit #{{ audit.pk }}</h1>
<div class="card mb-3">
<div class="card-body">
<dl class="row mb-0">
<dt class="col-3">Plugin</dt><dd class="col-9">{{ audit.slug }}</dd>
<dt class="col-3">Aktion</dt><dd class="col-9">{{ audit.action }}</dd>
<dt class="col-3">Version</dt><dd class="col-9">{{ audit.requested_version|default:"" }}</dd>
<dt class="col-3">Benutzer</dt><dd class="col-9">{{ audit.actor|default:"System" }}</dd>
<dt class="col-3">Status</dt><dd class="col-9">{{ audit.get_status_display }}</dd>
<dt class="col-3">Dry-Run</dt><dd class="col-9">{{ audit.dry_run|yesno:"Ja,Nein" }}</dd>
<dt class="col-3">Beginn</dt><dd class="col-9">{{ audit.started|default:"" }}</dd>
<dt class="col-3">Ende</dt><dd class="col-9">{{ audit.completed|default:"" }}</dd>
{% if audit.external_operation_id %}<dt class="col-3">Agent-Operation</dt><dd class="col-9"><code>{{ audit.external_operation_id }}</code></dd>{% endif %}
</dl>
</div>
</div>
{% if audit.error %}<div class="alert alert-danger"><pre class="mb-0 text-wrap">{{ audit.error }}</pre></div>{% endif %}
<div class="card">
<div class="card-header"><h2 class="card-title">Redigiertes Ergebnis</h2></div>
<div class="card-body"><pre class="mb-0 text-wrap">{{ audit.result_data }}</pre></div>
</div>
{% endblock %}
@@ -0,0 +1,24 @@
{% extends 'base/layout.html' %}
{% block title %}Plugin Store Audit{% endblock %}
{% block content %}
<h1>Plugin Store Audit</h1>
<div class="card">
<div class="table-responsive">
<table class="table table-vcenter card-table">
<thead><tr><th>Zeit</th><th>Plugin</th><th>Aktion</th><th>Benutzer</th><th>Dry-Run</th><th>Status</th></tr></thead>
<tbody>
{% for audit in audits %}
<tr>
<td><a href="{% url 'plugins:netbox_plugin_store:audit-detail' pk=audit.pk %}">{{ audit.created }}</a></td>
<td>{{ audit.slug }}</td><td>{{ audit.action }}</td>
<td>{{ audit.actor|default:"System" }}</td>
<td>{{ audit.dry_run|yesno:"Ja,Nein" }}</td><td>{{ audit.get_status_display }}</td>
</tr>
{% empty %}<tr><td colspan="6" class="text-secondary">Keine Einträge.</td></tr>{% endfor %}
</tbody>
</table>
</div>
</div>
{% endblock %}
@@ -0,0 +1,58 @@
{% extends 'base/layout.html' %}
{% block title %}Plugin Store{% endblock %}
{% block content %}
<div class="d-flex justify-content-between align-items-center mb-3">
<div>
<h1 class="mb-1">NetBox Plugin Store</h1>
<p class="text-secondary mb-0">Freigegebene Plugins für diese NetBox-Instanz.</p>
</div>
{% if runtime %}
<span class="badge {% if runtime.execution_mode == 'dry_run' %}bg-yellow text-dark{% else %}bg-green{% endif %}">
Modus: {{ runtime.execution_mode }}
</span>
{% endif %}
</div>
{% if store_error %}
<div class="alert alert-danger" role="alert">
<strong>Store nicht erreichbar:</strong> {{ store_error }}
</div>
{% endif %}
<div class="row row-cards">
{% for card in cards %}
<div class="col-sm-6 col-xl-4">
<div class="card h-100">
<div class="card-body">
<div class="d-flex justify-content-between">
<h3 class="card-title">{{ card.plugin.name }}</h3>
{% if card.plugin.approved %}
<span class="badge bg-green">Freigegeben</span>
{% else %}
<span class="badge bg-red">Nicht freigegeben</span>
{% endif %}
</div>
<p class="text-secondary">{{ card.plugin.summary|default:"Keine Zusammenfassung vorhanden." }}</p>
<dl class="row mb-0">
<dt class="col-5">Verfügbar</dt><dd class="col-7">{{ card.plugin.latest_version|default:"" }}</dd>
<dt class="col-5">Installiert</dt><dd class="col-7">{{ card.installed_version|default:"" }}</dd>
<dt class="col-5">Status</dt>
<dd class="col-7">
{% if card.enabled %}aktiv{% elif card.installed %}deaktiviert{% else %}nicht installiert{% endif %}
</dd>
</dl>
</div>
<div class="card-footer">
<a class="btn btn-primary" href="{% url 'plugins:netbox_plugin_store:plugin-detail' slug=card.plugin.slug %}">
Details
</a>
</div>
</div>
</div>
{% empty %}
{% if not store_error %}<p class="text-secondary">Der Store enthält noch keine freigegebenen Plugins.</p>{% endif %}
{% endfor %}
</div>
{% endblock %}
@@ -0,0 +1,40 @@
{% extends 'base/layout.html' %}
{% block title %}{{ action }}: {{ plugin.name }}{% endblock %}
{% block content %}
<div class="row justify-content-center">
<div class="col-lg-7">
<div class="card">
<div class="card-header">
<h1 class="card-title">Lifecycle-Aktion bestätigen</h1>
</div>
<div class="card-body">
<div class="alert alert-warning">
Aktion <strong>{{ action }}</strong> für <strong>{{ plugin.name }}</strong>.
Reale Änderungen können Python-Pakete, <code>local_requirements.txt</code> und die NetBox-Konfiguration verändern.
</div>
{% if runtime.execution_mode == 'dry_run' %}
<div class="alert alert-info">Diese Instanz erlaubt ausschließlich Dry-Runs.</div>
{% endif %}
<form method="post" action="{% url 'plugins:netbox_plugin_store:lifecycle-action' slug=plugin.slug action=action %}">
{% csrf_token %}
{{ form.non_field_errors }}
{% for field in form %}
<div class="mb-3">
<label class="form-label" for="{{ field.id_for_label }}">{{ field.label }}</label>
{{ field }}
{% if field.help_text %}<div class="form-hint">{{ field.help_text }}</div>{% endif %}
{% for error in field.errors %}<div class="text-danger">{{ error }}</div>{% endfor %}
</div>
{% endfor %}
<div class="d-flex justify-content-between">
<a class="btn btn-outline-secondary" href="{% url 'plugins:netbox_plugin_store:plugin-detail' slug=plugin.slug %}">Abbrechen</a>
<button class="btn btn-danger" type="submit">Bestätigen</button>
</div>
</form>
</div>
</div>
</div>
</div>
{% endblock %}
@@ -0,0 +1,70 @@
{% extends 'base/layout.html' %}
{% block title %}{{ plugin.name }}{% endblock %}
{% block content %}
<div class="d-flex justify-content-between align-items-start mb-3">
<div>
<a href="{% url 'plugins:netbox_plugin_store:catalog' %}">← Zurück zum Katalog</a>
<h1 class="mt-2 mb-1">{{ plugin.name }}</h1>
<p class="text-secondary">{{ plugin.summary }}</p>
</div>
<div class="btn-list">
{% for action in actions %}
<a class="btn {% if action == 'uninstall' or action == 'disable' %}btn-outline-danger{% else %}btn-primary{% endif %}"
href="{% url 'plugins:netbox_plugin_store:lifecycle-confirm' slug=plugin.slug action=action %}">
{% if action == 'install' %}Installieren{% elif action == 'update' %}Aktualisieren{% elif action == 'enable' %}Aktivieren{% elif action == 'disable' %}Deaktivieren{% else %}Deinstallieren{% endif %}
</a>
{% endfor %}
</div>
</div>
{% if local_status and local_status.restart_required %}
<div class="alert alert-warning">Ein Neustart von NetBox und den Workern ist erforderlich.</div>
{% endif %}
<div class="row">
<div class="col-lg-8">
<div class="card mb-3">
<div class="card-header"><h2 class="card-title">README / Beschreibung</h2></div>
<div class="card-body">
<div class="text-break">{{ plugin.description|default:plugin.summary|linebreaksbr }}</div>
</div>
</div>
</div>
<div class="col-lg-4">
<div class="card mb-3">
<div class="card-header"><h2 class="card-title">Paket</h2></div>
<div class="card-body">
<dl class="row mb-0">
<dt class="col-5">Distribution</dt><dd class="col-7"><code>{{ plugin.package_name }}</code></dd>
<dt class="col-5">Import</dt><dd class="col-7"><code>{{ plugin.import_name }}</code></dd>
<dt class="col-5">Installiert</dt><dd class="col-7">{{ state.installed_version|default:"" }}</dd>
<dt class="col-5">Aktiv</dt><dd class="col-7">{{ state.enabled|yesno:"Ja,Nein" }}</dd>
<dt class="col-5">NetBox</dt><dd class="col-7">{{ plugin.min_netbox_version|default:"" }} {{ plugin.max_netbox_version|default:"" }}</dd>
</dl>
{% if plugin.repository_url %}
<a class="btn btn-outline-secondary mt-3" href="{{ plugin.repository_url }}" target="_blank" rel="noopener noreferrer">Repository öffnen</a>
{% endif %}
</div>
</div>
<div class="card">
<div class="card-header"><h2 class="card-title">Releases</h2></div>
<div class="list-group list-group-flush">
{% for item in releases %}
<div class="list-group-item d-flex justify-content-between">
<span>{{ item.release.version }}</span>
{% if item.compatible and item.release.approved and item.release.immutable and item.release.sha256 %}
<span class="badge bg-green">installierbar</span>
{% else %}
<span class="badge bg-secondary">gesperrt</span>
{% endif %}
</div>
{% empty %}
<div class="list-group-item text-secondary">Keine Releases</div>
{% endfor %}
</div>
</div>
</div>
</div>
{% endblock %}
@@ -0,0 +1,29 @@
{% extends 'base/layout.html' %}
{% block title %}Installierte Plugins{% endblock %}
{% block content %}
<h1>Installierte Plugins</h1>
<div class="card">
<div class="table-responsive">
<table class="table table-vcenter card-table">
<thead><tr><th>Plugin</th><th>Installiert</th><th>Verfügbar</th><th>Aktiv</th><th>Status</th><th>Neustart</th><th>Aktualisiert</th></tr></thead>
<tbody>
{% for status in statuses %}
<tr>
<td><a href="{% url 'plugins:netbox_plugin_store:plugin-detail' slug=status.slug %}">{{ status.name }}</a></td>
<td>{{ status.installed_version|default:"" }}</td>
<td>{{ status.available_version|default:"" }}</td>
<td>{{ status.enabled|yesno:"Ja,Nein" }}</td>
<td>{{ status.get_state_display }}</td>
<td>{{ status.restart_required|yesno:"Erforderlich,Nein" }}</td>
<td>{{ status.updated }}</td>
</tr>
{% empty %}
<tr><td colspan="7" class="text-secondary">Noch keine Lifecycle-Aktion protokolliert.</td></tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
{% endblock %}
+22
View File
@@ -0,0 +1,22 @@
from django.urls import path
from . import views
urlpatterns = [
path("", views.CatalogView.as_view(), name="catalog"),
path("installed/", views.InstalledStatusView.as_view(), name="status"),
path("audit/", views.AuditListView.as_view(), name="audit-list"),
path("audit/<int:pk>/", views.AuditDetailView.as_view(), name="audit-detail"),
path("plugins/<slug:slug>/", views.PluginDetailView.as_view(), name="plugin-detail"),
path(
"plugins/<slug:slug>/confirm/<str:action>/",
views.LifecycleConfirmView.as_view(),
name="lifecycle-confirm",
),
path(
"plugins/<slug:slug>/actions/<str:action>/",
views.LifecycleActionView.as_view(),
name="lifecycle-action",
),
]
@@ -0,0 +1,57 @@
from __future__ import annotations
import re
from urllib.parse import urlsplit
from packaging.utils import canonicalize_name
SLUG_RE = re.compile(r"^[a-z0-9](?:[a-z0-9-]{0,62}[a-z0-9])?$")
DISTRIBUTION_RE = re.compile(r"^[A-Za-z0-9](?:[A-Za-z0-9._-]{0,126}[A-Za-z0-9])?$")
IMPORT_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]{0,127}$")
SHA256_RE = re.compile(r"^[a-fA-F0-9]{64}$")
SELF_IMPORT_NAME = "netbox_plugin_store"
SELF_DISTRIBUTION_NAME = canonicalize_name("netbox-plugin-store")
class ValidationError(ValueError):
pass
def validate_slug(value: str) -> str:
if not isinstance(value, str) or not SLUG_RE.fullmatch(value):
raise ValidationError("Invalid plugin slug.")
return value
def validate_distribution_name(value: str) -> str:
if not isinstance(value, str) or not DISTRIBUTION_RE.fullmatch(value):
raise ValidationError("Invalid Python distribution name.")
return value
def validate_import_name(value: str) -> str:
if not isinstance(value, str) or not IMPORT_RE.fullmatch(value):
raise ValidationError("Invalid Python import name.")
return value
def validate_sha256(value: str) -> str:
if not isinstance(value, str) or not SHA256_RE.fullmatch(value):
raise ValidationError("Invalid SHA-256 digest.")
return value.lower()
def ensure_not_self(package_name: str, import_name: str) -> None:
if canonicalize_name(package_name) == SELF_DISTRIBUTION_NAME or import_name == SELF_IMPORT_NAME:
raise ValidationError("The Plugin Store cannot manage its own lifecycle.")
def validate_public_url(value: str) -> str:
parsed = urlsplit(value)
if parsed.scheme not in {"https", "http"} or not parsed.hostname:
raise ValidationError("URL must be an absolute HTTP(S) URL.")
if parsed.username is not None or parsed.password is not None:
raise ValidationError("URLs containing credentials are not accepted.")
return value
@@ -0,0 +1 @@
__version__ = "0.1.0"
+267
View File
@@ -0,0 +1,267 @@
from __future__ import annotations
from django.contrib import messages
from django.contrib.auth.mixins import LoginRequiredMixin
from django.core.exceptions import PermissionDenied
from django.http import Http404
from django.shortcuts import get_object_or_404, redirect, render
from django.utils import timezone
from django.utils.decorators import method_decorator
from django.views import View
from django.views.decorators.csrf import csrf_protect
from .agent import AgentClient
from .access import has_store_access
from .client import StoreClient, StoreClientError
from .forms import LifecycleConfirmForm
from .jobs import PluginLifecycleJob
from .lifecycle import ACTIONS, LifecycleRequest, LifecycleService, installed_distribution_version
from .models import LifecycleAudit, ManagedPlugin
from .redaction import redact_data, redact_text
from .repository import DjangoLifecycleRepository
from .runtime import RuntimeSettings
class StorePermissionMixin(LoginRequiredMixin):
permission_name = "netbox_plugin_store.manage_plugin"
def dispatch(self, request, *args, **kwargs):
if not has_store_access(request.user, self.permission_name):
raise PermissionDenied
return super().dispatch(request, *args, **kwargs)
def _components() -> tuple[RuntimeSettings, StoreClient, DjangoLifecycleRepository]:
runtime = RuntimeSettings.from_django()
client = StoreClient(
runtime.store_url,
runtime.allowed_store_urls,
runtime.allowed_artifact_urls,
api_token=runtime.api_token,
timeout=runtime.request_timeout,
)
return runtime, client, DjangoLifecycleRepository()
def _plugin_state(plugin, active_plugins: set[str]) -> dict:
installed_version = installed_distribution_version(plugin.package_name)
enabled = plugin.import_name in active_plugins
return {
"installed_version": installed_version,
"enabled": enabled,
"installed": bool(installed_version),
}
class CatalogView(StorePermissionMixin, View):
def get(self, request):
from django.conf import settings as django_settings
try:
runtime, client, _ = _components()
plugins = client.list_plugins()
active_plugins = set(django_settings.PLUGINS)
cards = [{"plugin": plugin, **_plugin_state(plugin, active_plugins)} for plugin in plugins]
error = ""
except Exception as exc:
runtime, cards = None, []
error = redact_text(exc)
return render(
request,
"netbox_plugin_store/catalog.html",
{"cards": cards, "store_error": error, "runtime": runtime},
)
class PluginDetailView(StorePermissionMixin, View):
def get(self, request, slug: str):
from django.conf import settings as django_settings
try:
runtime, client, _ = _components()
plugin = client.get_plugin(slug)
except StoreClientError as exc:
raise Http404(redact_text(exc)) from exc
state = _plugin_state(plugin, set(django_settings.PLUGINS))
actions: list[str] = []
if not state["installed"]:
if plugin.approved:
actions.append("install")
else:
if plugin.approved:
actions.append("update")
if state["enabled"]:
actions.append("disable")
else:
actions.extend(("enable", "uninstall"))
releases = [
{"release": release, "compatible": release.supports(runtime.netbox_version, plugin)}
for release in plugin.releases
]
local_status = ManagedPlugin.objects.filter(slug=slug).first()
return render(
request,
"netbox_plugin_store/detail.html",
{
"plugin": plugin,
"state": state,
"actions": actions,
"releases": releases,
"local_status": local_status,
},
)
class LifecycleConfirmView(StorePermissionMixin, View):
def get(self, request, slug: str, action: str):
if action not in ACTIONS:
raise Http404
runtime, client, _ = _components()
plugin = client.get_plugin(slug)
form = LifecycleConfirmForm(plugin=plugin, action=action, runtime=runtime)
return render(
request,
"netbox_plugin_store/confirm.html",
{"plugin": plugin, "action": action, "form": form, "runtime": runtime},
)
@method_decorator(csrf_protect, name="dispatch")
class LifecycleActionView(StorePermissionMixin, View):
http_method_names = ["post"]
def post(self, request, slug: str, action: str):
if action not in ACTIONS:
raise Http404
runtime, client, repository = _components()
plugin = client.get_plugin(slug)
form = LifecycleConfirmForm(
request.POST,
plugin=plugin,
action=action,
runtime=runtime,
)
if not form.is_valid():
return render(
request,
"netbox_plugin_store/confirm.html",
{"plugin": plugin, "action": action, "form": form, "runtime": runtime},
status=400,
)
operation = LifecycleRequest(
slug=slug,
action=action,
version=form.cleaned_data.get("version", ""),
dry_run=form.cleaned_data["dry_run"],
)
if not operation.dry_run and not request.user.is_superuser:
raise PermissionDenied("Reale Plugin-Lifecycle-Aktionen erfordern einen Superuser.")
audit_id = repository.create_audit(operation, request.user.pk)
if operation.dry_run and runtime.background_jobs:
try:
PluginLifecycleJob.enqueue(
user=request.user,
queue_name=runtime.job_queue,
audit_id=audit_id,
slug=slug,
action=action,
version=operation.version,
dry_run=True,
actor_id=request.user.pk,
)
messages.success(request, "Dry-Run wurde in die NetBox-Jobqueue gestellt.")
return redirect("plugins:netbox_plugin_store:audit-detail", pk=audit_id)
except Exception as exc:
if not runtime.synchronous_fallback:
repository.mark_failed(audit_id, redact_text(exc), {"queue_failed": True})
messages.error(request, "Dry-Run konnte nicht eingeplant werden.")
return redirect("plugins:netbox_plugin_store:audit-detail", pk=audit_id)
service = LifecycleService(runtime, client, repository)
try:
result = service.execute(operation, actor_id=request.user.pk, audit_id=audit_id)
except Exception as exc:
messages.error(request, f"Lifecycle-Aktion fehlgeschlagen: {redact_text(exc, limit=1_000)}")
else:
label = "Dry-Run abgeschlossen" if result.dry_run else "Lifecycle-Aktion abgeschlossen"
messages.success(request, label + ".")
return redirect("plugins:netbox_plugin_store:audit-detail", pk=audit_id)
class InstalledStatusView(StorePermissionMixin, View):
def get(self, request):
statuses = ManagedPlugin.objects.all()
return render(request, "netbox_plugin_store/status.html", {"statuses": statuses})
class AuditListView(StorePermissionMixin, View):
def get(self, request):
audits = LifecycleAudit.objects.select_related("actor")[:200]
return render(request, "netbox_plugin_store/audit_list.html", {"audits": audits})
class AuditDetailView(StorePermissionMixin, View):
def get(self, request, pk: int):
audit = get_object_or_404(LifecycleAudit.objects.select_related("actor"), pk=pk)
if audit.status == LifecycleAudit.Status.HANDED_OFF and audit.external_operation_id:
self._refresh_agent_status(request, audit)
audit.refresh_from_db()
return render(request, "netbox_plugin_store/audit_detail.html", {"audit": audit})
@staticmethod
def _refresh_agent_status(request, audit: LifecycleAudit) -> None:
try:
runtime = RuntimeSettings.from_django()
if runtime.execution_mode != "agent" or runtime.agent_socket_path is None:
return
body = AgentClient(runtime.agent_socket_path, timeout=runtime.agent_timeout).get_operation(
audit.external_operation_id
)
state = body.get("state") or body.get("status")
if state in {"succeeded", "completed"}:
result = body.get("result", body)
if not isinstance(result, dict):
result = {"message": "Host agent completed without a structured result."}
LifecycleAudit.objects.filter(pk=audit.pk).update(
status=LifecycleAudit.Status.SUCCEEDED,
result_data=redact_data(result),
error="",
completed=timezone.now(),
)
status_values = {}
if "installed_version" in result:
status_values["installed_version"] = str(result["installed_version"] or "")[:64]
if "enabled" in result:
status_values["enabled"] = result["enabled"] is True
if "restart_required" in result:
status_values["restart_required"] = result["restart_required"] is True
if status_values:
if status_values.get("restart_required"):
status_values["state"] = ManagedPlugin.State.RESTART_REQUIRED
elif status_values.get("installed_version"):
status_values["state"] = (
ManagedPlugin.State.ENABLED
if status_values.get("enabled")
else ManagedPlugin.State.DISABLED
)
else:
status_values["state"] = ManagedPlugin.State.UNKNOWN
status_values.update({"last_error": "", "last_checked": timezone.now()})
ManagedPlugin.objects.filter(slug=audit.slug).update(**status_values)
elif state in {"failed", "errored", "cancelled"}:
error = redact_text(body.get("error") or f"Host agent operation {state}.")
LifecycleAudit.objects.filter(pk=audit.pk).update(
status=LifecycleAudit.Status.FAILED,
result_data=redact_data(body),
error=error,
completed=timezone.now(),
)
ManagedPlugin.objects.filter(slug=audit.slug).update(
state=ManagedPlugin.State.FAILED,
last_error=error,
restart_required=True,
last_checked=timezone.now(),
)
elif state in {"queued", "pending", "running", "accepted"}:
LifecycleAudit.objects.filter(pk=audit.pk).update(result_data=redact_data(body))
except Exception as exc:
messages.warning(request, f"Host-Agent-Status konnte nicht aktualisiert werden: {redact_text(exc)}")
+39
View File
@@ -0,0 +1,39 @@
[build-system]
requires = ["setuptools>=75", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "netbox-plugin-store"
version = "0.1.0"
description = "A secure NetBox 4.6 plugin lifecycle client for the MrBlake Plugin Store"
readme = "README.md"
requires-python = ">=3.12"
license = { text = "Apache-2.0" }
authors = [{ name = "MrBlake" }]
dependencies = [
"packaging>=24.0",
]
classifiers = [
"Development Status :: 3 - Alpha",
"Framework :: Django",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: System :: Systems Administration",
]
[project.urls]
Homepage = "https://git.mrblake.cc"
[tool.setuptools.packages.find]
include = ["netbox_plugin_store*"]
[tool.setuptools.package-data]
netbox_plugin_store = [
"templates/netbox_plugin_store/*.html",
"migrations/*.py",
]
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-q"
+41
View File
@@ -0,0 +1,41 @@
import sys
import types
if "netbox.plugins" not in sys.modules:
netbox = types.ModuleType("netbox")
plugins = types.ModuleType("netbox.plugins")
class PluginConfig:
def ready(self):
return None
class PluginMenu:
def __init__(self, label, groups, icon_class=None):
self.label = label
self.groups = groups
self.icon_class = icon_class
class PluginMenuItem:
def __init__(
self,
link,
link_text,
auth_required=False,
staff_only=False,
permissions=None,
buttons=None,
):
self.link = link
self.link_text = link_text
self.auth_required = auth_required
self.staff_only = staff_only
self.permissions = permissions or []
self.buttons = buttons or []
plugins.PluginConfig = PluginConfig
plugins.PluginMenu = PluginMenu
plugins.PluginMenuItem = PluginMenuItem
netbox.plugins = plugins
sys.modules["netbox"] = netbox
sys.modules["netbox.plugins"] = plugins
@@ -0,0 +1,34 @@
ALLOWED_HOSTS = ["localhost"]
SECRET_KEY = "netbox-plugin-store-compatibility-test-key-2026-08-24!"
API_TOKEN_PEPPERS = {
1: "netbox-plugin-store-compatibility-test-pepper-2026-08-24!",
}
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": "netbox_compatibility_test",
"USER": "netbox",
"PASSWORD": "unused",
"HOST": "127.0.0.1",
"PORT": 5432,
"CONN_MAX_AGE": 0,
}
}
REDIS = {
"tasks": {"HOST": "127.0.0.1", "PORT": 6379, "DATABASE": 0},
"caching": {"HOST": "127.0.0.1", "PORT": 6379, "DATABASE": 1},
}
PLUGINS = ["netbox_plugin_store"]
PLUGINS_CONFIG = {
"netbox_plugin_store": {
"store_url": "https://store.example",
"allowed_store_urls": ["https://store.example"],
"allowed_artifact_urls": ["https://store.example"],
}
}
CENSUS_REPORTING_ENABLED = False
RELEASE_CHECK_URL = None
@@ -0,0 +1,130 @@
"""Smoke tests executed in an environment containing an official NetBox tag."""
from __future__ import annotations
import inspect
import os
import sys
import unittest
from pathlib import Path
from unittest.mock import patch
NETBOX_SOURCE = Path(os.environ["NETBOX_SOURCE"]).resolve()
EXPECTED_VERSION = os.environ["NETBOX_EXPECTED_VERSION"]
sys.path.insert(0, str(NETBOX_SOURCE / "netbox"))
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "netbox.settings")
os.environ.setdefault("NETBOX_CONFIGURATION", "netbox_test_configuration")
import django # noqa: E402
django.setup()
from django.apps import apps # noqa: E402
from django.db.migrations.autodetector import MigrationAutodetector # noqa: E402
from django.db.migrations.loader import MigrationLoader # noqa: E402
from django.db.migrations.questioner import MigrationQuestioner # noqa: E402
from django.db.migrations.state import ProjectState # noqa: E402
from django.template.loader import get_template # noqa: E402
from django.urls import resolve, reverse # noqa: E402
from core.models import Job # noqa: E402
from netbox.jobs import JobRunner # noqa: E402
from netbox_plugin_store import config # noqa: E402
from netbox_plugin_store.jobs import PluginLifecycleJob # noqa: E402
from netbox_plugin_store.navigation import menu # noqa: E402
from users.models import User # noqa: E402
class OfficialNetBoxCompatibilityTests(unittest.TestCase):
def test_exact_netbox_and_django_runtime(self):
from django.conf import settings
self.assertEqual(settings.RELEASE.version, EXPECTED_VERSION)
self.assertGreaterEqual(sys.version_info, (3, 12))
self.assertEqual(django.get_version(), "6.0.7" if EXPECTED_VERSION == "4.6.5" else "6.0.8")
def test_plugin_config_is_registered(self):
app = apps.get_app_config("netbox_plugin_store")
self.assertIsInstance(app, config)
self.assertEqual(config.min_version, "4.6.5")
self.assertEqual(config.max_version, "4.6.8")
def test_jobrunner_queue_metadata_and_run_arguments_are_compatible(self):
self.assertIn("queue_name", inspect.signature(Job.enqueue).parameters)
self.assertTrue(issubclass(PluginLifecycleJob, JobRunner))
with patch.object(Job, "enqueue", return_value="queued") as enqueue:
result = PluginLifecycleJob.enqueue(
user="operator",
queue_name="compatibility",
audit_id=12,
slug="example-plugin",
action="install",
version="1.0.0",
dry_run=True,
actor_id=34,
)
self.assertEqual(result, "queued")
args, kwargs = enqueue.call_args
self.assertIs(args[0].__func__, PluginLifecycleJob.handle.__func__)
self.assertEqual(kwargs["queue_name"], "compatibility")
self.assertEqual(kwargs["audit_id"], 12)
self.assertEqual(kwargs["actor_id"], 34)
def test_navigation_urls_and_permission_semantics(self):
self.assertFalse(hasattr(User(), "is_staff"))
urls = {
"catalog": reverse("plugins:netbox_plugin_store:catalog"),
"status": reverse("plugins:netbox_plugin_store:status"),
"audit-list": reverse("plugins:netbox_plugin_store:audit-list"),
"audit-detail": reverse("plugins:netbox_plugin_store:audit-detail", kwargs={"pk": 7}),
"plugin-detail": reverse(
"plugins:netbox_plugin_store:plugin-detail", kwargs={"slug": "example-plugin"}
),
"lifecycle-confirm": reverse(
"plugins:netbox_plugin_store:lifecycle-confirm",
kwargs={"slug": "example-plugin", "action": "install"},
),
"lifecycle-action": reverse(
"plugins:netbox_plugin_store:lifecycle-action",
kwargs={"slug": "example-plugin", "action": "install"},
),
}
for name, url in urls.items():
self.assertEqual(resolve(url).url_name, name)
items = [item for group in menu.groups for item in group.items]
self.assertEqual(len(items), 3)
for item in items:
self.assertTrue(item.auth_required)
self.assertFalse(item.staff_only)
self.assertEqual(item.permissions, ["netbox_plugin_store.manage_plugin"])
def test_templates_resolve_and_compile_from_installed_wheel(self):
for name in (
"audit_detail.html",
"audit_list.html",
"catalog.html",
"confirm.html",
"detail.html",
"status.html",
):
template = get_template(f"netbox_plugin_store/{name}")
self.assertIsNotNone(template.template)
self.assertIsNotNone(get_template("base/layout.html").template)
def test_migration_graph_and_model_state_have_no_drift(self):
loader = MigrationLoader(None, ignore_no_migrations=True)
self.assertIn(("netbox_plugin_store", "0001_initial"), loader.disk_migrations)
from_state = loader.project_state()
to_state = ProjectState.from_apps(apps)
changes = MigrationAutodetector(
from_state,
to_state,
MigrationQuestioner(specified_apps={"netbox_plugin_store"}),
).changes(graph=loader.graph, trim_to_apps={"netbox_plugin_store"})
self.assertNotIn("netbox_plugin_store", changes)
if __name__ == "__main__":
unittest.main(verbosity=2)
@@ -0,0 +1,81 @@
import json
import subprocess
import unittest
from pathlib import Path
from unittest.mock import patch
from uuid import UUID
import _bootstrap # noqa: F401
from netbox_plugin_store.agent import AgentClient
from netbox_plugin_store.commands import SubprocessRunner
class FakeSocket:
def __init__(self, response):
self.response = response
self.sent = b""
def __enter__(self):
return self
def __exit__(self, *args):
return None
def settimeout(self, timeout):
self.timeout = timeout
def connect(self, path):
self.path = path
def sendall(self, value):
self.sent += value
def shutdown(self, how):
return None
def recv(self, size):
value, self.response = self.response, b""
return value
class AgentTests(unittest.TestCase):
def test_json_line_operation_protocol(self):
operation_id = "12345678-1234-5678-1234-567812345678"
response = (
json.dumps({"protocol_version": 1, "status": 202, "body": {"operation_id": operation_id}}).encode()
+ b"\n"
)
fake = FakeSocket(response)
with (
patch("netbox_plugin_store.agent.socket.AF_UNIX", 1, create=True),
patch("netbox_plugin_store.agent.socket.SOCK_STREAM", 1),
patch("netbox_plugin_store.agent.socket.socket", return_value=fake),
):
result = AgentClient(Path("/run/store.sock")).submit_operation(
request_id="request-id",
action="install",
plugin_slug="example-plugin",
version="1.0.0",
approved_payload_sha256="c" * 64,
requested_by="netbox-user:1",
)
self.assertEqual(result, UUID(operation_id))
request = json.loads(fake.sent.decode().strip())
self.assertEqual(request["method"], "POST")
self.assertEqual(request["path"], "/v1/operations")
self.assertEqual(request["body"]["plugin_slug"], "example-plugin")
UUID(request["idempotency_key"])
class CommandTests(unittest.TestCase):
def test_subprocess_boundary_never_uses_shell(self):
completed = subprocess.CompletedProcess(["python", "-V"], 0, "Python", "")
with patch("netbox_plugin_store.commands.subprocess.run", return_value=completed) as run:
result = SubprocessRunner().run(["python", "-V"], timeout=5)
self.assertEqual(result.stdout, "Python")
self.assertIs(run.call_args.kwargs["shell"], False)
if __name__ == "__main__":
unittest.main()
+122
View File
@@ -0,0 +1,122 @@
import hashlib
import io
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
import _bootstrap # noqa: F401
from netbox_plugin_store.client import CatalogPlugin, StoreClient, StoreClientError, URLPolicy
def plugin_payload():
artifact = b"verified artifact"
return {
"slug": "example-plugin",
"name": "Example",
"summary": "Example plugin",
"description": "README",
"repository_url": "https://git.mrblake.cc/team/example",
"latest_version": "1.2.0",
"package_name": "netbox-example",
"import_name": "netbox_example",
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
"approved": True,
"releases": [
{
"version": "1.2.0",
"download_url": "https://store.example/artifacts/example.whl",
"sha256": hashlib.sha256(artifact).hexdigest(),
"approved_payload_sha256": "a" * 64,
"approved": True,
"immutable": True,
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
}
],
}, artifact
class URLPolicyTests(unittest.TestCase):
def test_exact_origin_and_path_prefix(self):
policy = URLPolicy(["https://store.example/internal"])
self.assertEqual(
policy.check("https://store.example/internal/api/v1/plugins/"),
"https://store.example/internal/api/v1/plugins/",
)
with self.assertRaises(StoreClientError):
policy.check("https://store.example.evil/internal")
with self.assertRaises(StoreClientError):
policy.check("https://store.example/other")
def test_release_selection_checks_netbox_version(self):
payload, _ = plugin_payload()
plugin = CatalogPlugin.from_mapping(payload)
self.assertEqual(plugin.select_release("4.6.8").version, "1.2.0")
with self.assertRaises(StoreClientError):
plugin.select_release("4.6.9")
class _Response(io.BytesIO):
def __init__(self, body):
super().__init__(body)
self.headers = {"Content-Length": str(len(body))}
def __enter__(self):
return self
def __exit__(self, *args):
self.close()
class _Opener:
def __init__(self, body):
self.body = body
def open(self, request, timeout):
return _Response(self.body)
class DownloadTests(unittest.TestCase):
def test_download_is_hashed_before_use(self):
_, artifact = plugin_payload()
client = StoreClient(
"https://store.example", ("https://store.example",), ("https://store.example",)
)
with tempfile.TemporaryDirectory() as temp_name:
target = Path(temp_name) / "plugin.whl"
with patch("netbox_plugin_store.client.build_opener", return_value=_Opener(artifact)):
actual, size = client.download_artifact(
"https://store.example/artifacts/example.whl",
target,
expected_sha256=hashlib.sha256(artifact).hexdigest(),
timeout=10,
max_bytes=1_000,
)
self.assertEqual(actual, hashlib.sha256(artifact).hexdigest())
self.assertEqual(size, len(artifact))
self.assertEqual(target.read_bytes(), artifact)
def test_mismatched_hash_removes_download(self):
_, artifact = plugin_payload()
client = StoreClient(
"https://store.example", ("https://store.example",), ("https://store.example",)
)
with tempfile.TemporaryDirectory() as temp_name:
target = Path(temp_name) / "plugin.whl"
with patch("netbox_plugin_store.client.build_opener", return_value=_Opener(artifact)):
with self.assertRaises(StoreClientError):
client.download_artifact(
"https://store.example/artifacts/example.whl",
target,
expected_sha256="0" * 64,
timeout=10,
max_bytes=1_000,
)
self.assertFalse(target.exists())
if __name__ == "__main__":
unittest.main()
+52
View File
@@ -0,0 +1,52 @@
import tempfile
import unittest
from pathlib import Path
import _bootstrap # noqa: F401
from netbox_plugin_store.editors import EditorError, PluginConfigurationEditor, RequirementsEditor
class ConfigurationEditorTests(unittest.TestCase):
def test_atomic_change_backup_and_rollback(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
config = root / "configuration.py"
original = "SECRET_KEY = 'unchanged'\nPLUGINS = [\n 'netbox_plugin_store',\n]\n"
config.write_text(original, encoding="utf-8")
editor = PluginConfigurationEditor(config, root / "backups", 5)
receipt = editor.set_enabled("netbox_example", True)
self.assertTrue(receipt.changed)
self.assertTrue(receipt.backup_path.is_file())
self.assertIn("'netbox_example'", config.read_text(encoding="utf-8"))
editor.rollback(receipt)
self.assertEqual(config.read_text(encoding="utf-8"), original)
def test_dynamic_plugins_assignment_is_rejected(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
config = root / "configuration.py"
config.write_text("PLUGINS = load_plugins()\n", encoding="utf-8")
editor = PluginConfigurationEditor(config, root / "backups", 5)
with self.assertRaises(EditorError):
editor.set_enabled("netbox_example", True)
class RequirementsEditorTests(unittest.TestCase):
def test_pin_and_remove_preserve_other_lines(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
requirements = root / "local_requirements.txt"
requirements.write_text("# managed manually\nother-package==2\n", encoding="utf-8")
editor = RequirementsEditor(requirements, root / "backups", 5)
line = "netbox-example @ https://store.example/example.whl#sha256=" + "a" * 64
editor.set_requirement("netbox-example", line)
text = requirements.read_text(encoding="utf-8")
self.assertIn(line, text)
self.assertIn("other-package==2", text)
editor.set_requirement("netbox-example", None)
self.assertNotIn("netbox-example", requirements.read_text(encoding="utf-8"))
if __name__ == "__main__":
unittest.main()
+169
View File
@@ -0,0 +1,169 @@
import hashlib
import tempfile
import unittest
from pathlib import Path
import _bootstrap # noqa: F401
from netbox_plugin_store.client import CatalogPlugin
from netbox_plugin_store.commands import CommandResult
from netbox_plugin_store.lifecycle import LifecycleError, LifecycleRequest, LifecycleService
from netbox_plugin_store.runtime import RuntimeSettings
ARTIFACT = b"approved wheel bytes"
def catalog_plugin():
return CatalogPlugin.from_mapping(
{
"slug": "example-plugin",
"name": "Example",
"package_name": "netbox-example",
"import_name": "netbox_example",
"latest_version": "1.0.0",
"approved": True,
"releases": [
{
"version": "1.0.0",
"download_url": "https://store.example/example.whl",
"sha256": hashlib.sha256(ARTIFACT).hexdigest(),
"approved_payload_sha256": "b" * 64,
"approved": True,
"immutable": True,
"min_netbox_version": "4.6.5",
"max_netbox_version": "4.6.8",
}
],
}
)
class FakeClient:
def __init__(self, plugin):
self.plugin = plugin
self.downloads = 0
def get_plugin(self, slug):
return self.plugin
def download_artifact(self, url, destination, **kwargs):
self.downloads += 1
destination.write_bytes(ARTIFACT)
return hashlib.sha256(ARTIFACT).hexdigest(), len(ARTIFACT)
class FakeRepository:
def __init__(self):
self.audits = {}
self.statuses = []
def create_audit(self, request, actor_id):
self.audits[1] = {"status": "queued"}
return 1
def mark_running(self, audit_id):
self.audits[audit_id]["status"] = "running"
def mark_success(self, audit_id, result):
self.audits[audit_id].update(status="succeeded", result=result)
def mark_handed_off(self, audit_id, operation_id, result):
self.audits[audit_id].update(status="handed-off", operation_id=operation_id, result=result)
def mark_failed(self, audit_id, error, result):
self.audits[audit_id].update(status="failed", error=error)
def update_status(self, plugin, **values):
self.statuses.append(values)
def has_other_pending_mutation(self, slug, audit_id):
return False
class FakeRunner:
def __init__(self):
self.calls = []
def run(self, argv, **kwargs):
self.calls.append((list(argv), kwargs))
return CommandResult(0, "ok", "")
def runtime(root: Path, *, execution_mode="direct", allow=True):
config = root / "configuration.py"
config.write_text("PLUGINS = ['netbox_plugin_store']\n", encoding="utf-8")
return RuntimeSettings.from_mapping(
{
"store_url": "https://store.example",
"allowed_store_urls": ["https://store.example"],
"allowed_artifact_urls": ["https://store.example"],
"configuration_path": str(config),
"requirements_path": str(root / "local_requirements.txt"),
"manage_path": str(root / "manage.py"),
"lock_path": str(root / "operation.lock"),
"backup_dir": str(root / "backups"),
"execution_mode": execution_mode,
"allow_lifecycle_mutations": allow,
"run_migrations": False,
"collect_static": False,
"allow_package_index": False,
},
configuration_dir=root,
netbox_root=root,
base_dir=root,
netbox_version="4.6.8",
)
class LifecycleTests(unittest.TestCase):
def test_dry_run_has_no_download_or_subprocess(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
client = FakeClient(catalog_plugin())
repository = FakeRepository()
runner = FakeRunner()
service = LifecycleService(runtime(root), client, repository, runner=runner, version_provider=lambda _: "")
result = service.execute(LifecycleRequest("example-plugin", "install", "1.0.0", True))
self.assertEqual(result.state, "dry-run")
self.assertEqual(client.downloads, 0)
self.assertEqual(runner.calls, [])
def test_direct_install_is_disabled_and_uses_no_index_and_pip_check(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
client = FakeClient(catalog_plugin())
repository = FakeRepository()
runner = FakeRunner()
service = LifecycleService(runtime(root), client, repository, runner=runner, version_provider=lambda _: "")
result = service.execute(LifecycleRequest("example-plugin", "install", "1.0.0", False))
self.assertFalse(result.enabled)
self.assertFalse(result.restart_required)
self.assertNotIn("netbox_example", (root / "configuration.py").read_text(encoding="utf-8"))
requirement = (root / "local_requirements.txt").read_text(encoding="utf-8")
self.assertIn("#sha256=", requirement)
self.assertIn("--no-index", runner.calls[0][0])
self.assertEqual(runner.calls[1][0][-2:], ["pip", "check"])
def test_uninstall_requires_explicit_disable_first(self):
with tempfile.TemporaryDirectory() as temp_name:
root = Path(temp_name)
settings = runtime(root)
settings.configuration_path.write_text(
"PLUGINS = ['netbox_plugin_store', 'netbox_example']\n", encoding="utf-8"
)
runner = FakeRunner()
service = LifecycleService(
settings,
FakeClient(catalog_plugin()),
FakeRepository(),
runner=runner,
version_provider=lambda _: "1.0.0",
)
with self.assertRaises(LifecycleError):
service.execute(LifecycleRequest("example-plugin", "uninstall", dry_run=False))
self.assertEqual(runner.calls, [])
if __name__ == "__main__":
unittest.main()
+50
View File
@@ -0,0 +1,50 @@
import unittest
import _bootstrap # noqa: F401
from netbox_plugin_store.access import has_store_access
from netbox_plugin_store.navigation import menu
PERMISSION = "netbox_plugin_store.manage_plugin"
class NetBoxUser:
"""NetBox 4.6 users deliberately have no is_staff attribute."""
def __init__(self, *, authenticated=True, superuser=False, permissions=()):
self.is_authenticated = authenticated
self.is_superuser = superuser
self.permissions = set(permissions)
def has_perm(self, permission_name):
return permission_name in self.permissions
class AccessCompatibilityTests(unittest.TestCase):
def test_permission_user_without_is_staff_is_authorized(self):
user = NetBoxUser(permissions=(PERMISSION,))
self.assertTrue(has_store_access(user, PERMISSION))
def test_unauthenticated_and_unprivileged_users_are_rejected(self):
self.assertFalse(
has_store_access(NetBoxUser(authenticated=False, permissions=(PERMISSION,)), PERMISSION)
)
self.assertFalse(has_store_access(NetBoxUser(), PERMISSION))
def test_superuser_is_authorized_without_explicit_permission(self):
self.assertTrue(has_store_access(NetBoxUser(superuser=True), PERMISSION))
class NavigationCompatibilityTests(unittest.TestCase):
def test_menu_uses_permission_not_staff_only(self):
items = [item for _label, group_items in menu.groups for item in group_items]
self.assertEqual(len(items), 3)
for item in items:
self.assertTrue(item.auth_required)
self.assertFalse(item.staff_only)
self.assertEqual(item.permissions, [PERMISSION])
if __name__ == "__main__":
unittest.main()