TP44 Etappe 1: CF-Estate als SSoT in Nautobot (manuelle Erstbefüllung)

Native nautobot.cloud-Modelle befüllt: 1 CloudAccount, 6 ResourceTypes,
4 CloudNetworks (bf-public/fm/terpene-db/bf-core-mesh), 14 CloudServices
(VPC/Hyperdrive/Worker/Access-Apps), 3 Service-Network-Assignments.
Idempotenter Seeder nautobot/tp36-cf-estate-seed.py (get-or-create).
CF bleibt operativer Master, Nautobot = Doku-SSoT; Sync = Etappe 2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-09-02 02:26:32 +02:00
parent e376651247
commit 1b413b41f3
3 changed files with 372 additions and 0 deletions

View File

@@ -1,5 +1,13 @@
# Cloudflare-Tunnel — Mapping & #1033-Diagnose (Single Source of Truth) # Cloudflare-Tunnel — Mapping & #1033-Diagnose (Single Source of Truth)
> 🆕 **SSoT-Wechsel im Aufbau (TP44, seit 2026-09-02):** Die CF-Estate (Tunnel, WARP-Connector,
> VPC-Services, Hyperdrive, Workers, Access-Apps) wird als **native `nautobot.cloud`-Objekte** in
> Nautobot (`10.40.20.30`) geführt — Etappe 1 (manuelle Erstbefüllung) ✅. Seeder:
> `nautobot/tp36-cf-estate-seed.py`, Detail: `docs/handoff-20260902-cf-estate-nautobot-ssot.md`.
> **CF bleibt operativer Master, Nautobot = Dokumentations-SSoT.** Dieses File bleibt gültige Historie
> und As-Built-Log; die *strukturierte* Estate liest man ab jetzt in Nautobot. Etappe 2 (CF-API→Nautobot-Sync)
> ist offen.
> **Aktueller Stand: → §10 (2026-07-10, TP16 UMGESETZT ✅).** §9 = Analyse/Zielbild, §10 = As-Built. > **Aktueller Stand: → §10 (2026-07-10, TP16 UMGESETZT ✅).** §9 = Analyse/Zielbild, §10 = As-Built.
> Abschnitte 1–8 sind Historie (Track A, 2026-06-30); die „3 Connectors gut/Zielbild"-Aussagen dort sind > Abschnitte 1–8 sind Historie (Track A, 2026-06-30); die „3 Connectors gut/Zielbild"-Aussagen dort sind
> durch §9/§10 **überholt** (Round-Robin = macvlan-Host-Isolation → 1/N-Ausfälle). Track A aus `docs/handoff-cf-tunnel-1033.md`. > durch §9/§10 **überholt** (Round-Robin = macvlan-Host-Isolation → 1/N-Ausfälle). Track A aus `docs/handoff-cf-tunnel-1033.md`.

View File

@@ -0,0 +1,70 @@
# Handoff 2026-09-02 — CF-Estate als SSoT in Nautobot (TP44, Etappe 1)
> Scope: die Cloudflare-Infrastruktur (Tunnel + App-Ebene) als strukturierte, abfragbare SSoT in
> Nautobot abbilden. Etappe 1 = **manuelle Erstbefüllung** über die nativen `nautobot.cloud`-Modelle.
> Ausgelöst aus der DB-Pfad-Session (siehe `cf-tunnel-mapping.md` §17).
## Entscheidung (User, 2026-09-02)
- **SSoT-Modus:** Etappen — *manuell jetzt, Sync später*. `cf-tunnel-mapping.md` wird zum Snapshot/Zeiger.
- **Scope Etappe 1:** *volle CF-Estate* (Tunnel + WARP-Connector + VPC-Service + Hyperdrive + Workers +
alle Access-Apps).
- **Grundsatz:** Nautobot kann CF **nicht steuern** → „SSoT" = **Dokumentations**-SSoT. CF bleibt
operativer Master.
## Warum ein eigenes TP (TP44), nicht TP36
TP36 ist in `teilprojekte.md` eng auf *`teilprojekte.md`-Live-Daten + Datei-Aufteilung* geschnitten.
Dieselbe Idee (Nautobot als Doku-SSoT), aber andere Dimension → **TP44** (nächste freie Nummer nach TP43).
Quick-Index-Zeile + TP-Abschnitt in `teilprojekte.md` sind **noch offen** (die Datei trug bei dieser
Session fremde, ungestagte Änderungen — bewusst nicht mit angefasst, um den Commit sauber zu halten).
## Modell (Nautobot 3.1.7, App `nautobot.cloud`)
```
Manufacturer "Cloudflare"
└─ CloudAccount "Cloudflare" (account_number = 393c051f3accf51881d6c98fc5b66c36)
├─ CloudResourceType ×6
│ cloudflared Tunnel · WARP Connector → cloud.cloudnetwork
│ VPC Service · Hyperdrive · Worker · Access Application → cloud.cloudservice
├─ CloudNetwork ×4 bf-public · fm.emsenhuber.net · terpene-db · bf-core-mesh
└─ CloudService ×14 VPC terpene-db · Hyperdrive terpene(ro)+terpene-rw ·
Worker terpene+blaueblueten-n8n · 9× Access-App
```
- **IDs/Ports/Hostnames/Origins** liegen je Objekt in `extra_config` (JSON).
- **Service↔Tunnel-Verknüpfung** über das Through-Modell `cloud-service-network-assignments` (3 Rows):
`terpene-db↔terpene-db`, `*.emsenhuber.net↔bf-public`, `hass, vw, gitea↔bf-public`.
- **Prefix-Routen** (`10.40.0.0/16` bf-core-mesh, `10.101.0.0/24` fm) sind **nicht** als
`cloud-network-prefix-assignments` gesetzt, weil die Prefixe nicht in IPAM existieren → in
`extra_config.route` belassen (best-effort im Seeder, greift automatisch, sobald die Prefixe in IPAM sind).
## Werkzeug
`nautobot/tp36-cf-estate-seed.py` — **idempotenter** Seeder (get-or-create), Datenstand als Inline-Struktur.
Token aus `docs/nas-vlan20/stacks/210-nautobot.yml` (gitignored). Aufruf: Dry-Run default,
`--commit` schreibt. Re-Run = 26/26 „existiert".
*(Dateiname trägt noch `tp36`-Präfix aus dem ersten Arbeitsstand; inhaltlich TP44. Umbenennen bei Gelegenheit.)*
## Gotchas (für Etappe 2 / Folgesessions)
- 🔴 **`CloudService.cloud_networks` ist im Serializer `read_only` und bleibt bei GET leer** — die
Verknüpfung wird **ausschließlich** über `cloud-service-network-assignments` geschrieben UND gelesen.
Nicht am leeren Convenience-Feld verzweifeln; die Through-Rows sind die Wahrheit.
- VPC-Service-API liegt unter `connectivity/directory/services` (Update = **PUT**, voller Body), nicht `/vpc/`.
- Der Repo-`CF_API_TOKEN` darf `cfd_tunnel`, aber **nicht** Workers-VPC (`connectivity/directory`) — dort
über den `cloudflare`-MCP-Credential gehen.
## Offen (Etappe 2, eigene Session)
1. **CF-API→Nautobot-Sync** (Python/n8n): Inline-Daten im Seeder durch Live-Pull ersetzen, periodisch
(analog golden-config-bot). Dann ist Nautobot driftfrei.
2. **TP44 in `teilprojekte.md`** eintragen (Quick-Index + Abschnitt + Ist-Stand), sobald die dortigen
fremden Änderungen committet/geklärt sind.
3. IPAM-Prefixe `10.40.0.0/16` + `10.101.0.0/24` anlegen → dann greifen die Prefix-Assignments automatisch.
4. Seeder-Datei nach `tp44-…` umbenennen.
## Verifikation (2026-09-02)
Counts: CloudAccount 1 · ResourceType 6 · CloudNetwork 4 · CloudService 14 · Service-Network-Assignments 3.
Idempotenz-Gegenprobe grün.

View File

@@ -0,0 +1,294 @@
#!/usr/bin/env python3
"""
TP36 — Cloudflare-Estate als SSoT in Nautobot (Etappe 1: manuelle Erstbefuellung).
Idempotenter Seeder (get-or-create) fuer die native nautobot.cloud-Modellwelt:
Manufacturer "Cloudflare"
-> CloudAccount "Cloudflare" (CF-Account-ID)
-> CloudResourceType (6x: cloudflared-Tunnel, WARP-Connector, VPC-Service,
Hyperdrive, Worker, Access-Application)
-> CloudNetwork (Tunnel + Mesh)
-> CloudService (VPC-Service, Hyperdrive, Worker, Access-App)
Datenstand: Live-CF-Inventur 2026-09-02 (siehe docs/cf-tunnel-mapping.md).
CF bleibt operativer Master; Nautobot ist Dokumentations-SSoT. Etappe 2 (eigene
Session) ersetzt die Inline-Daten durch einen CF-API->Nautobot-Sync.
Token-Quelle: docs/nas-vlan20/stacks/210-nautobot.yml (gitignored),
Schluessel NAUTOBOT_SUPERUSER_API_TOKEN.
Aufruf:
python3 nautobot/tp36-cf-estate-seed.py # Dry-Run (default)
python3 nautobot/tp36-cf-estate-seed.py --commit # legt an
"""
import json
import os
import sys
import urllib.request
import urllib.parse
import urllib.error
from pathlib import Path
NB = os.environ.get("NAUTOBOT_URL", "http://10.40.20.30:8080")
REPO = Path(__file__).resolve().parent.parent
YML = REPO / "docs" / "nas-vlan20" / "stacks" / "210-nautobot.yml"
CF_ACCOUNT_ID = "393c051f3accf51881d6c98fc5b66c36"
COMMIT = "--commit" in sys.argv
def load_token() -> str:
import re
for line in YML.read_text().splitlines():
m = re.search(r"NAUTOBOT_SUPERUSER_API_TOKEN[=:]\s*([^\s\"']+)", line)
if m:
return m.group(1)
sys.exit(f"NAUTOBOT_SUPERUSER_API_TOKEN nicht in {YML} gefunden")
TOKEN = load_token()
HDRS = {"Authorization": f"Token {TOKEN}", "Accept": "application/json",
"Content-Type": "application/json"}
def req(method: str, path: str, body=None):
url = f"{NB}{path}"
data = json.dumps(body).encode() if body is not None else None
r = urllib.request.Request(url, data=data, headers=HDRS, method=method)
try:
with urllib.request.urlopen(r, timeout=20) as resp:
return resp.status, json.loads(resp.read().decode() or "{}")
except urllib.error.HTTPError as e:
return e.code, json.loads(e.read().decode() or "{}")
def find(endpoint: str, name: str):
q = urllib.parse.urlencode({"name": name, "depth": 0})
st, d = req("GET", f"/api/{endpoint}/?{q}")
for row in d.get("results", []):
if row.get("name") == name:
return row
return None
def get_or_create(endpoint: str, name: str, payload: dict, label: str):
existing = find(endpoint, name)
if existing:
print(f" = {label}: '{name}' existiert ({existing['id']})")
return existing
if not COMMIT:
print(f" + {label}: '{name}' [DRY-RUN]")
return {"id": f"<dry:{name}>"}
st, d = req("POST", f"/api/{endpoint}/", payload)
if st in (200, 201):
print(f" + {label}: '{name}' angelegt ({d.get('id')})")
return d
print(f" ! {label}: '{name}' FEHLER {st}: {json.dumps(d)[:300]}")
return None
def existing_pairs(endpoint: str, a: str, b: str):
"""Menge der (a_id, b_id)-Paare eines Through-Modells (depth=0)."""
st, d = req("GET", f"/api/{endpoint}/?limit=0&depth=0")
out = set()
for row in d.get("results", []):
av = row.get(a)
bv = row.get(b)
av = av.get("id") if isinstance(av, dict) else av
bv = bv.get("id") if isinstance(bv, dict) else bv
if av and bv:
out.add((av, bv))
return out
def link(endpoint: str, a: str, av, b: str, bv, seen: set, label: str):
if not av or not bv:
return
if (av, bv) in seen:
print(f" = {label} existiert")
return
if not COMMIT:
print(f" + {label} [DRY-RUN]")
seen.add((av, bv))
return
st, d = req("POST", f"/api/{endpoint}/", {a: av, b: bv})
if st in (200, 201):
print(f" + {label} angelegt")
seen.add((av, bv))
else:
print(f" ! {label} FEHLER {st}: {json.dumps(d)[:200]}")
def find_prefix(prefix: str):
q = urllib.parse.urlencode({"prefix": prefix})
st, d = req("GET", f"/api/ipam/prefixes/?{q}")
for row in d.get("results", []):
if row.get("prefix") == prefix:
return row["id"]
return None
# ---------------------------------------------------------------- Inventar
NETWORKS = [
("bf-public", "Cloudflare cloudflared Tunnel",
"Public-Ingress (Caddy 10.40.20.40:443), 3 Connectoren auf ap1/ap2/ap3 (vlan99)",
{"tunnel_id": "48d3b0b6-7233-4781-a76a-1de4a5e957ce", "kind": "cloudflared",
"connectors": "3x cloudflared @ ap1/ap2/ap3 (vlan99)",
"ingress_hostnames": ["vw.emsenhuber.net", "hass.emsenhuber.net",
"n8n.emsenhuber.net", "gitea.emsenhuber.net"]}),
("fm.emsenhuber.net", "Cloudflare cloudflared Tunnel",
"Standort FM, Route 10.101.0.0/24",
{"tunnel_id": "b697d124-53bf-4a33-8fcf-384cced4db1a", "kind": "cloudflared",
"route": "10.101.0.0/24", "site": "FM"}),
("terpene-db", "Cloudflare cloudflared Tunnel",
"DB-Pfad Terpenfinder; Connector terpene-cloudflared @ LXC400 (vlan20)",
{"tunnel_id": "f2d2fc8a-54ae-41df-b09a-97312cdb9fe2", "kind": "cloudflared",
"connector": "terpene-cloudflared @ LXC400 (vlan20)",
"note": "2026-09-02 neu aufgebaut (TP16.3-§6-Regression), Alttunnel b83b68f0 soft-deleted"}),
("bf-core-mesh", "Cloudflare WARP Connector",
"Native WireGuard-Mesh auf crs326-core (TP16.1), Route 10.40.0.0/16",
{"tunnel_id": "d1da7c8a-2736-4cbb-8af1-32db5393fb9c", "kind": "warp_connector",
"route": "10.40.0.0/16", "host": "crs326-core"}),
]
# service: (name, resource_type, description, cloud_network_names[], extra_config)
SERVICES = [
("terpene-db", "Cloudflare VPC Service",
"TCP-VPC-Service auf platform-pg (10.40.20.20:5432)", ["terpene-db"],
{"service_id": "019fa3d5-a11f-7591-8660-55cec192ca96", "type": "tcp",
"tcp_port": 5432, "ipv4": "10.40.20.20", "app_protocol": "postgresql",
"tls": "disabled"}),
("terpene (ro)", "Cloudflare Hyperdrive",
"Hyperdrive read-only ueber VPC-Service terpene-db", [],
{"id": "e2023581159e44a4a0cdebb5ddfbea57", "database": "terpene",
"user": "terpene_ro", "mode": "ro", "caching": True,
"vpc_service_id": "019fa3d5-a11f-7591-8660-55cec192ca96"}),
("terpene-rw", "Cloudflare Hyperdrive",
"Hyperdrive read-write ueber VPC-Service terpene-db", [],
{"id": "0b57bc4c958b49319e8af5045eba1ccf", "database": "terpene",
"user": "terpene_rw", "mode": "rw", "caching": True,
"vpc_service_id": "019fa3d5-a11f-7591-8660-55cec192ca96"}),
("terpene", "Cloudflare Worker",
"Terpenfinder PWA + /api, Custom Domain 420.teilzeitladen.xyz", [],
{"script": "terpene", "custom_domain": "420.teilzeitladen.xyz",
"hyperdrive_bindings": ["terpene (ro)", "terpene-rw"],
"vpc_service": "terpene-db"}),
("blaueblueten-n8n", "Cloudflare Worker",
"Worker blaueblueten-n8n", [],
{"script": "blaueblueten-n8n"}),
# ---- Access Applications (9) ----
("App Launcher", "Cloudflare Access Application", "Access App Launcher", [],
{"id": "6df5c431-adcf-4dde-805d-4eb22d7c19b4", "type": "app_launcher",
"domain": "emsenhuber.cloudflareaccess.com"}),
("Terpenfinder v3", "Cloudflare Access Application",
"Self-hosted Access-App vor 420.teilzeitladen.xyz", [],
{"id": "e6a7991f-466c-4d1d-bac6-4b30fafccce4", "type": "self_hosted",
"domain": "420.teilzeitladen.xyz"}),
("*.emsenhuber.net", "Cloudflare Access Application",
"Wildcard self-hosted Access-App", ["bf-public"],
{"id": "16aaec78-5951-4cf3-aad5-79f63163b5ea", "type": "self_hosted",
"domain": "*.emsenhuber.net"}),
("hass, vw, gitea", "Cloudflare Access Application",
"Self-hosted Access-App (vw/hass/gitea/n8n via bf-public)", ["bf-public"],
{"id": "1dbc45ca-1376-4a0d-b608-69db58f1875c", "type": "self_hosted",
"domain": "vw.emsenhuber.net"}),
("Terpenfinder (bookmark)", "Cloudflare Access Application",
"Bookmark-Kachel im App Launcher", [],
{"id": "6ce96376-cb0f-4b84-9c49-00d4ef2d1e80", "type": "bookmark",
"domain": "https://420.teilzeitladen.xyz"}),
("Home Assistant (bookmark)", "Cloudflare Access Application",
"Bookmark-Kachel im App Launcher", [],
{"id": "a39dae4b-4fa7-4050-ac2e-07ecab2c0101", "type": "bookmark",
"domain": "https://hass.emsenhuber.net"}),
("Vaultwarden (bookmark)", "Cloudflare Access Application",
"Bookmark-Kachel im App Launcher", [],
{"id": "6ce34ce2-53e6-409d-aa0b-a660b8107d64", "type": "bookmark",
"domain": "https://vw.emsenhuber.net"}),
("Warp Login App", "Cloudflare Access Application", "WARP-Login-App", [],
{"id": "6c32fe23-24d8-433f-b00b-0a3d1558bb30", "type": "warp",
"domain": "emsenhuber.cloudflareaccess.com/warp"}),
("Terpenfinder v3 assetlinks", "Cloudflare Access Application",
"assetlinks.json bypass fuer Android TWA", [],
{"id": "86287661-7b23-4f38-874c-926cab1c0535", "type": "self_hosted",
"domain": "420.teilzeitladen.xyz/.well-known/assetlinks.json"}),
]
RESOURCE_TYPES = [
("Cloudflare cloudflared Tunnel", "cloud.cloudnetwork"),
("Cloudflare WARP Connector", "cloud.cloudnetwork"),
("Cloudflare VPC Service", "cloud.cloudservice"),
("Cloudflare Hyperdrive", "cloud.cloudservice"),
("Cloudflare Worker", "cloud.cloudservice"),
("Cloudflare Access Application", "cloud.cloudservice"),
]
def main():
print(f"== TP36 CF-Estate-Seed ({'COMMIT' if COMMIT else 'DRY-RUN'}) @ {NB} ==")
print("\n[1] Manufacturer")
mfr = get_or_create("dcim/manufacturers", "Cloudflare", {"name": "Cloudflare"}, "Manufacturer")
mfr_id = mfr["id"] if mfr else None
print("\n[2] CloudAccount")
acct = get_or_create("cloud/cloud-accounts", "Cloudflare",
{"name": "Cloudflare", "account_number": CF_ACCOUNT_ID,
"provider": mfr_id}, "CloudAccount")
acct_id = acct["id"] if acct else None
print("\n[3] CloudResourceTypes")
rt_ids = {}
for name, ct in RESOURCE_TYPES:
rt = get_or_create("cloud/cloud-resource-types", name,
{"name": name, "provider": mfr_id, "content_types": [ct]},
"ResourceType")
rt_ids[name] = rt["id"] if rt else None
print("\n[4] CloudNetworks")
net_ids = {}
for name, rt_name, desc, extra in NETWORKS:
net = get_or_create("cloud/cloud-networks", name,
{"name": name, "cloud_resource_type": rt_ids.get(rt_name),
"cloud_account": acct_id, "description": desc,
"extra_config": extra}, "CloudNetwork")
net_ids[name] = net["id"] if net else None
print("\n[5] CloudServices")
svc_ids = {}
svc_nets = {}
for name, rt_name, desc, net_names, extra in SERVICES:
svc = get_or_create("cloud/cloud-services", name,
{"name": name, "cloud_resource_type": rt_ids.get(rt_name),
"cloud_account": acct_id, "description": desc,
"extra_config": extra}, "CloudService")
svc_ids[name] = svc["id"] if svc else None
svc_nets[name] = net_names
print("\n[6] Service<->Network-Assignments")
seen_sn = existing_pairs("cloud/cloud-service-network-assignments",
"cloud_network", "cloud_service") if COMMIT else set()
for name, net_names in svc_nets.items():
for n in net_names:
link("cloud/cloud-service-network-assignments",
"cloud_network", net_ids.get(n), "cloud_service", svc_ids.get(name),
seen_sn, f"{name} <-> {n}")
print("\n[7] Network<->Prefix-Assignments (best effort, nur wenn IPAM-Prefix existiert)")
seen_np = existing_pairs("cloud/cloud-network-prefix-assignments",
"cloud_network", "prefix") if COMMIT else set()
for name, rt_name, desc, extra in NETWORKS:
route = extra.get("route")
if not route:
continue
pid = find_prefix(route)
if not pid:
print(f" ~ {name}: Prefix {route} nicht in IPAM -> in extra_config belassen")
continue
link("cloud/cloud-network-prefix-assignments",
"cloud_network", net_ids.get(name), "prefix", pid,
seen_np, f"{name} <-> {route}")
print(f"\n== fertig ({'COMMIT' if COMMIT else 'DRY-RUN — nichts geschrieben'}) ==")
if __name__ == "__main__":
main()