Document canonical plugin architecture
This commit is contained in:
Lyncoln
@@ -2,19 +2,374 @@
|
||||
|
||||
A plugin-based DNS reconciliation service.
|
||||
|
||||
## Architecture
|
||||
|
||||
dns-sync separates generic reconciliation from application-specific endpoint
|
||||
behavior.
|
||||
|
||||
native endpoint
|
||||
|
|
||||
v
|
||||
<App>Client
|
||||
|
|
||||
| native objects
|
||||
v
|
||||
Plugin
|
||||
|
|
||||
| DiscoveredObject
|
||||
v
|
||||
dns-sync core
|
||||
|
|
||||
| normalization
|
||||
| observations
|
||||
| desired state
|
||||
| persistence
|
||||
| reconciliation
|
||||
| planning
|
||||
v
|
||||
PlanAction
|
||||
|
|
||||
v
|
||||
Plugin
|
||||
|
|
||||
| native translation
|
||||
v
|
||||
<App>Client
|
||||
|
|
||||
v
|
||||
native endpoint
|
||||
|
||||
The core owns orchestration truth.
|
||||
|
||||
Plugins own application semantics.
|
||||
|
||||
Clients own endpoint-native communication.
|
||||
|
||||
## Plugin contract
|
||||
|
||||
The reconciliation runtime is adapter-based.
|
||||
|
||||
Adapter plugins expose:
|
||||
Every adapter package must export:
|
||||
|
||||
- `Plugin`
|
||||
- `configured_endpoints(config)`
|
||||
- `Plugin`
|
||||
|
||||
Each adapter declares object-specific discovery, apply, and verification
|
||||
capabilities. Built-in adapters live under `dns_sync/plugins/adapters/`.
|
||||
External adapters are loaded from `DNS_SYNC_PLUGIN_PATH` and must be
|
||||
explicitly enabled through `ENABLED_PLUGINS`.
|
||||
The registry calls:
|
||||
|
||||
configured_endpoints(config)
|
||||
|
||||
to obtain the endpoints owned by the plugin.
|
||||
|
||||
For each configured endpoint, the registry constructs:
|
||||
|
||||
Plugin(config, endpoint)
|
||||
|
||||
Every endpoint returned by `configured_endpoints(config)` must identify the
|
||||
plugin that returned it. The registry rejects cross-plugin endpoint ownership.
|
||||
|
||||
Plugins declare object-specific capabilities using the generic
|
||||
`Capability` model from `dns_sync.adapters`.
|
||||
|
||||
Capabilities currently describe:
|
||||
|
||||
- discovery
|
||||
- apply
|
||||
- verification
|
||||
|
||||
for a particular generic object type.
|
||||
|
||||
Built-in adapters live under:
|
||||
|
||||
dns_sync/plugins/adapters/
|
||||
|
||||
External adapters are loaded from:
|
||||
|
||||
DNS_SYNC_PLUGIN_PATH
|
||||
|
||||
and must be explicitly enabled through:
|
||||
|
||||
ENABLED_PLUGINS
|
||||
|
||||
## Canonical plugin structure
|
||||
|
||||
Application integrations should use this package structure:
|
||||
|
||||
<plugin_name>/
|
||||
├── __init__.py
|
||||
└── client.py
|
||||
|
||||
The repository contains a copyable example under:
|
||||
|
||||
examples/plugin_template/
|
||||
|
||||
The example is documentation only and is not loaded from that location.
|
||||
|
||||
### Adapter boundary: __init__.py
|
||||
|
||||
`__init__.py` is the dns-sync-facing plugin boundary.
|
||||
|
||||
It owns:
|
||||
|
||||
- endpoint configuration through `configured_endpoints(config)`
|
||||
- generic capability declarations
|
||||
- validation of the supplied endpoint
|
||||
- native-to-generic discovery translation
|
||||
- generic-to-native apply translation
|
||||
- verification
|
||||
- optional retirement behavior
|
||||
- application-specific semantic validation
|
||||
|
||||
The adapter communicates with the core using generic types such as:
|
||||
|
||||
- `Endpoint`
|
||||
- `DiscoveredObject`
|
||||
- `NormalizedObject`
|
||||
- `ObjectReference`
|
||||
- `PlanAction`
|
||||
- `ApplyResult`
|
||||
- `RetireRequest`
|
||||
|
||||
### Native boundary: client.py
|
||||
|
||||
`client.py` is private implementation machinery for that application.
|
||||
|
||||
It owns:
|
||||
|
||||
- authentication
|
||||
- HTTP sessions, sockets, commands, or other transport
|
||||
- application-native request and response handling
|
||||
- application-native models
|
||||
- endpoint-local mutation primitives
|
||||
- endpoint-specific verification mechanics
|
||||
|
||||
A client represents one explicit `Endpoint`.
|
||||
|
||||
The client should not independently rediscover which endpoint it is supposed
|
||||
to contact from another global endpoint setting.
|
||||
|
||||
There is deliberately no generic dns-sync `Client` base class. Different
|
||||
applications may communicate through completely different mechanisms. The
|
||||
shared contract is the adapter plugin boundary, not the transport layer.
|
||||
|
||||
Application-specific models belong inside the plugin package rather than the
|
||||
dns-sync core unless they represent a genuinely application-independent core
|
||||
concept.
|
||||
|
||||
## Discovery
|
||||
|
||||
`Plugin.discover(object_type)` reads native endpoint state through the private
|
||||
client and returns `DiscoveredObject` instances.
|
||||
|
||||
A discovered object carries:
|
||||
|
||||
object_type
|
||||
local_key
|
||||
payload
|
||||
references
|
||||
|
||||
`local_key` is endpoint-local identity. It is not assumed to be portable to
|
||||
another endpoint.
|
||||
|
||||
The plugin owns translation from the native application's representation into
|
||||
this generic discovery representation.
|
||||
|
||||
Unsupported object types should fail explicitly rather than silently returning
|
||||
an empty or misleading result.
|
||||
|
||||
## Normalized state
|
||||
|
||||
The core converts discoveries into `NormalizedObject` instances and owns the
|
||||
reconciliation state built from them.
|
||||
|
||||
A normalized object carries:
|
||||
|
||||
object_type
|
||||
logical_key
|
||||
payload
|
||||
local_key
|
||||
references
|
||||
|
||||
`logical_key` represents portable identity.
|
||||
|
||||
`local_key` represents endpoint-local identity.
|
||||
|
||||
Plugins do not persist their own reconciliation truth. The core owns:
|
||||
|
||||
- observations
|
||||
- observation status
|
||||
- desired state
|
||||
- applied and verified state
|
||||
- reconciliation
|
||||
- planning
|
||||
- dependency resolution
|
||||
|
||||
## Object references
|
||||
|
||||
Dependencies between objects use `ObjectReference`.
|
||||
|
||||
A reference identifies another object by exactly one of:
|
||||
|
||||
local_key
|
||||
|
||||
or:
|
||||
|
||||
logical_key
|
||||
|
||||
but never both.
|
||||
|
||||
Endpoint discovery may expose local references because those are the identities
|
||||
native applications actually use.
|
||||
|
||||
Central reconciliation can convert those into portable logical references.
|
||||
|
||||
During execution, dns-sync resolves portable references back to the correct
|
||||
endpoint-local identity before invoking the target adapter.
|
||||
|
||||
Plugins should never treat an endpoint-local ID from one endpoint as portable
|
||||
desired state for another endpoint.
|
||||
|
||||
## Apply
|
||||
|
||||
`Plugin.apply(action)` receives a generic `PlanAction`.
|
||||
|
||||
Before mutation, the adapter should validate that the action belongs to its
|
||||
endpoint.
|
||||
|
||||
Shared validation helpers are available in:
|
||||
|
||||
dns_sync.adapter_support
|
||||
|
||||
including:
|
||||
|
||||
require_endpoint_plugin(...)
|
||||
require_action_endpoint(...)
|
||||
require_desired(...)
|
||||
|
||||
`NOOP` and `SKIP` actions must not mutate the endpoint.
|
||||
|
||||
For a mutating operation, the adapter should:
|
||||
|
||||
1. validate the requested operation;
|
||||
2. validate the desired normalized object;
|
||||
3. translate desired state into an application-native object;
|
||||
4. invoke the private client;
|
||||
5. re-fetch or rediscover the actual resulting native object;
|
||||
6. obtain its real endpoint-local identity;
|
||||
7. return an `ApplyResult`.
|
||||
|
||||
Do not guess or predict endpoint-local IDs when the endpoint assigns them.
|
||||
|
||||
Unsupported operations should fail explicitly.
|
||||
|
||||
## Apply results
|
||||
|
||||
A successful mutation returns an `ApplyResult` whose normalized object contains
|
||||
the actual endpoint-local key.
|
||||
|
||||
Conceptually:
|
||||
|
||||
ApplyResult(
|
||||
object=normalized_applied_object,
|
||||
detail=...,
|
||||
)
|
||||
|
||||
The returned object's `local_key` is required.
|
||||
|
||||
An apply result may also include deferred `RetireRequest` objects when a
|
||||
replacement workflow requires the old object to remain temporarily while
|
||||
dependencies are rebound.
|
||||
|
||||
## Verification
|
||||
|
||||
`Plugin.verify(object)` checks that the actual endpoint state matches the
|
||||
normalized object dns-sync expects to exist.
|
||||
|
||||
Verification is intentionally separate from mutation.
|
||||
|
||||
A successful API response is not sufficient evidence that endpoint state
|
||||
converged.
|
||||
|
||||
The executor requires supported mutations to pass verification before the
|
||||
result is treated as successfully applied.
|
||||
|
||||
## Retirement
|
||||
|
||||
`retire()` is optional.
|
||||
|
||||
A plugin should implement retirement only when it can safely remove or retire
|
||||
endpoint-local objects.
|
||||
|
||||
Retirement is used by deletion and replacement workflows.
|
||||
|
||||
The executor performs preflight checks before mutation. If execution requires
|
||||
retirement and the adapter does not provide it, the operation fails before the
|
||||
unsafe mutation begins.
|
||||
|
||||
Retirement should use the endpoint-local identity supplied by dns-sync and fail
|
||||
closed when the object cannot be identified safely.
|
||||
|
||||
## Fail-closed behavior
|
||||
|
||||
Plugins should reject ambiguous or unsupported behavior.
|
||||
|
||||
In particular:
|
||||
|
||||
- reject endpoints owned by another plugin;
|
||||
- reject actions intended for another endpoint;
|
||||
- reject unsupported object types;
|
||||
- reject unsupported plan operations;
|
||||
- reject malformed native state;
|
||||
- reject ambiguous endpoint-local identities;
|
||||
- never guess missing IDs;
|
||||
- never report successful application until the resulting object can be
|
||||
identified;
|
||||
- never treat mutation success alone as verification.
|
||||
|
||||
The generic executor adds additional preflight and dependency safety checks.
|
||||
|
||||
## Configuration ownership
|
||||
|
||||
Configuration should live at the lowest layer that actually understands it.
|
||||
|
||||
Use this rule:
|
||||
|
||||
Does generic reconciliation need to understand this value?
|
||||
|
||||
yes -> core configuration may be appropriate
|
||||
no -> plugin/client configuration
|
||||
|
||||
Examples of generic reconciliation policy include:
|
||||
|
||||
DNS_TARGET_IP
|
||||
DOMAIN_MATCHING
|
||||
DOMAIN
|
||||
|
||||
Examples of plugin/client configuration include:
|
||||
|
||||
NPM_ENDPOINTS
|
||||
PROXY_API_PORT
|
||||
PROXY_API_SCHEME
|
||||
NPM_IDENTITY
|
||||
NPM_SECRET
|
||||
VERIZON_ENDPOINTS
|
||||
ROUTER_PASSWORD
|
||||
ROUTER_TIMEOUT
|
||||
|
||||
`configured_endpoints(config)` owns endpoint discovery for the plugin.
|
||||
|
||||
The client receives the resulting explicit `Endpoint`; it should not implement
|
||||
a second independent endpoint-selection mechanism.
|
||||
|
||||
## Secret handling
|
||||
|
||||
Secret material should remain below the normalized and persistence boundary
|
||||
unless the core has an explicit safe representation for it.
|
||||
|
||||
Plugins must not place credentials, authentication tokens, private keys, or
|
||||
other secret material into generic persisted payloads.
|
||||
|
||||
Runtime credentials should normally be provided through environment variables,
|
||||
container secrets, or another external secret-management mechanism.
|
||||
|
||||
## Plugins
|
||||
|
||||
@@ -38,6 +393,125 @@ The Verizon adapter lives under `external_plugins/verizon` and is deliberately
|
||||
not copied into the dns-sync container image. The external plugin directory is
|
||||
mounted read-only at runtime.
|
||||
|
||||
## Plugin loading
|
||||
|
||||
Only explicitly enabled plugins are loadable.
|
||||
|
||||
The registry first checks for a built-in adapter under:
|
||||
|
||||
dns_sync/plugins/adapters/<plugin_name>/
|
||||
|
||||
If no built-in adapter exists, it looks for an external package at:
|
||||
|
||||
$DNS_SYNC_PLUGIN_PATH/<plugin_name>/__init__.py
|
||||
|
||||
External plugins are loaded as packages, so they may use relative imports for
|
||||
private modules such as:
|
||||
|
||||
from .client import ExampleClient
|
||||
|
||||
Imports from dns-sync itself should remain absolute, for example:
|
||||
|
||||
from dns_sync.adapters import Endpoint
|
||||
from dns_sync.adapter_support import require_endpoint_plugin
|
||||
|
||||
The external plugin directory does not need to be added globally to
|
||||
`PYTHONPATH`.
|
||||
|
||||
## Adding a new plugin
|
||||
|
||||
Start from:
|
||||
|
||||
examples/plugin_template/
|
||||
|
||||
For an external plugin:
|
||||
|
||||
1. copy the template into the configured external plugin root;
|
||||
2. rename the directory to the plugin name;
|
||||
3. rename `ExampleClient` and `ExampleRecord`;
|
||||
4. implement `configured_endpoints(config)`;
|
||||
5. implement native authentication and transport in `client.py`;
|
||||
6. declare only the generic capabilities actually supported;
|
||||
7. implement native-to-`DiscoveredObject` translation;
|
||||
8. implement `NormalizedObject`-to-native translation;
|
||||
9. implement `apply()` and `verify()`;
|
||||
10. implement `retire()` only when safe retirement is supported;
|
||||
11. add focused client and adapter tests;
|
||||
12. enable the plugin through `ENABLED_PLUGINS`;
|
||||
13. run read-only reconciliation against the real endpoint before permitting
|
||||
mutations.
|
||||
|
||||
Do not begin a new integration by adding application-specific models or
|
||||
transport behavior to the core.
|
||||
|
||||
## Reference implementations
|
||||
|
||||
Use both existing plugins when designing a new integration.
|
||||
|
||||
### NPM
|
||||
|
||||
The NPM plugin demonstrates:
|
||||
|
||||
- a built-in adapter;
|
||||
- multiple configured endpoints;
|
||||
- proxy-host and certificate object types;
|
||||
- endpoint-local object IDs;
|
||||
- cross-object certificate references;
|
||||
- portable logical references;
|
||||
- replacement workflows;
|
||||
- deferred retirement;
|
||||
- certificate material handling.
|
||||
|
||||
Its package lives at:
|
||||
|
||||
dns_sync/plugins/adapters/npm/
|
||||
|
||||
### Verizon
|
||||
|
||||
The Verizon plugin demonstrates:
|
||||
|
||||
- an external adapter;
|
||||
- plugin-owned endpoint configuration;
|
||||
- a private native record model;
|
||||
- a private endpoint client;
|
||||
- DNS record discovery;
|
||||
- create/update operations;
|
||||
- verification;
|
||||
- a read-only mounted external plugin package.
|
||||
|
||||
Its package lives at:
|
||||
|
||||
external_plugins/verizon/
|
||||
|
||||
Together they demonstrate the preferred boundary:
|
||||
|
||||
dns-sync core
|
||||
|
|
||||
+-- generic state and orchestration
|
||||
|
|
||||
+-- Plugin
|
||||
|
|
||||
+-- native / normalized translation
|
||||
|
|
||||
+-- <App>Client
|
||||
|
|
||||
+-- application-native implementation
|
||||
|
||||
## Plugin development validation
|
||||
|
||||
Before committing a plugin change, run at least:
|
||||
|
||||
git diff --check
|
||||
python3 -m compileall -q dns_sync external_plugins tests examples
|
||||
python3 -m unittest discover -s tests -v
|
||||
|
||||
Changes involving plugin loading, native clients, endpoint configuration, or
|
||||
mutation behavior should also be validated with a real read-only container
|
||||
reconciliation against the intended endpoints.
|
||||
|
||||
A clean read-only plan does not by itself authorize mutation. Review unexpected
|
||||
planned changes before enabling apply mode.
|
||||
|
||||
## Container runtime
|
||||
|
||||
The container periodically executes the generic reconciliation engine:
|
||||
|
||||
Reference in New Issue
Block a user