dns-sync/README.md

# dns-sync modular

A plugin-based DNS reconciliation service.

## Architecture

dns-sync separates generic reconciliation from application-specific endpoint
behavior.

    native endpoint
          |
          v
    <App>Client
          |
          | native objects
          v
    Plugin
          |
          | DiscoveredObject
          v
    dns-sync core
          |
          | normalization
          | observations
          | desired state
          | persistence
          | reconciliation
          | planning
          v
    PlanAction
          |
          v
    Plugin
          |
          | native translation
          v
    <App>Client
          |
          v
    native endpoint

The core owns orchestration truth.

Plugins own application semantics.

Clients own endpoint-native communication.

## Plugin contract

The reconciliation runtime is adapter-based.

Every adapter package must export:

- `configured_endpoints(config)`
- `Plugin`

The registry calls:

    configured_endpoints(config)

to obtain the endpoints owned by the plugin.

For each configured endpoint, the registry constructs:

    Plugin(config, endpoint)

Every endpoint returned by `configured_endpoints(config)` must identify the
plugin that returned it. The registry rejects cross-plugin endpoint ownership.

Plugins declare object-specific capabilities using the generic
`Capability` model from `dns_sync.adapters`.

Capabilities currently describe:

- discovery
- apply
- verification

for a particular generic object type.

Built-in adapters live under:

    dns_sync/plugins/adapters/

External adapters are loaded from:

    DNS_SYNC_PLUGIN_PATH

and must be explicitly enabled through:

    ENABLED_PLUGINS

## Canonical plugin structure

Application integrations should use this package structure:

    <plugin_name>/
    ├── __init__.py
    └── client.py

The repository contains a copyable example under:

    examples/plugin_template/

The example is documentation only and is not loaded from that location.

### Adapter boundary: __init__.py

`__init__.py` is the dns-sync-facing plugin boundary.

It owns:

- endpoint configuration through `configured_endpoints(config)`
- generic capability declarations
- validation of the supplied endpoint
- native-to-generic discovery translation
- generic-to-native apply translation
- verification
- optional retirement behavior
- application-specific semantic validation

The adapter communicates with the core using generic types such as:

- `Endpoint`
- `DiscoveredObject`
- `NormalizedObject`
- `ObjectReference`
- `PlanAction`
- `ApplyResult`
- `RetireRequest`

### Native boundary: client.py

`client.py` is private implementation machinery for that application.

It owns:

- authentication
- HTTP sessions, sockets, commands, or other transport
- application-native request and response handling
- application-native models
- endpoint-local mutation primitives
- endpoint-specific verification mechanics

A client represents one explicit `Endpoint`.

The client should not independently rediscover which endpoint it is supposed
to contact from another global endpoint setting.

There is deliberately no generic dns-sync `Client` base class. Different
applications may communicate through completely different mechanisms. The
shared contract is the adapter plugin boundary, not the transport layer.

Application-specific models belong inside the plugin package rather than the
dns-sync core unless they represent a genuinely application-independent core
concept.

## Discovery

`Plugin.discover(object_type)` reads native endpoint state through the private
client and returns `DiscoveredObject` instances.

A discovered object carries:

    object_type
    local_key
    payload
    references

`local_key` is endpoint-local identity. It is not assumed to be portable to
another endpoint.

The plugin owns translation from the native application's representation into
this generic discovery representation.

Unsupported object types should fail explicitly rather than silently returning
an empty or misleading result.

## Normalized state

The core converts discoveries into `NormalizedObject` instances and owns the
reconciliation state built from them.

A normalized object carries:

    object_type
    logical_key
    payload
    local_key
    references

`logical_key` represents portable identity.

`local_key` represents endpoint-local identity.

Plugins do not persist their own reconciliation truth. The core owns:

- observations
- observation status
- desired state
- applied and verified state
- reconciliation
- planning
- dependency resolution

## Object references

Dependencies between objects use `ObjectReference`.

A reference identifies another object by exactly one of:

    local_key

or:

    logical_key

but never both.

Endpoint discovery may expose local references because those are the identities
native applications actually use.

Central reconciliation can convert those into portable logical references.

During execution, dns-sync resolves portable references back to the correct
endpoint-local identity before invoking the target adapter.

Plugins should never treat an endpoint-local ID from one endpoint as portable
desired state for another endpoint.

## Apply

`Plugin.apply(action)` receives a generic `PlanAction`.

Before mutation, the adapter should validate that the action belongs to its
endpoint.

Shared validation helpers are available in:

    dns_sync.adapter_support

including:

    require_endpoint_plugin(...)
    require_action_endpoint(...)
    require_desired(...)

`NOOP` and `SKIP` actions must not mutate the endpoint.

For a mutating operation, the adapter should:

1. validate the requested operation;
2. validate the desired normalized object;
3. translate desired state into an application-native object;
4. invoke the private client;
5. re-fetch or rediscover the actual resulting native object;
6. obtain its real endpoint-local identity;
7. return an `ApplyResult`.

Do not guess or predict endpoint-local IDs when the endpoint assigns them.

Unsupported operations should fail explicitly.

## Apply results

A successful mutation returns an `ApplyResult` whose normalized object contains
the actual endpoint-local key.

Conceptually:

    ApplyResult(
        object=normalized_applied_object,
        detail=...,
    )

The returned object's `local_key` is required.

An apply result may also include deferred `RetireRequest` objects when a
replacement workflow requires the old object to remain temporarily while
dependencies are rebound.

## Verification

`Plugin.verify(object)` checks that the actual endpoint state matches the
normalized object dns-sync expects to exist.

Verification is intentionally separate from mutation.

A successful API response is not sufficient evidence that endpoint state
converged.

The executor requires supported mutations to pass verification before the
result is treated as successfully applied.

## Retirement

`retire()` is optional.

A plugin should implement retirement only when it can safely remove or retire
endpoint-local objects.

Retirement is used by deletion and replacement workflows.

The executor performs preflight checks before mutation. If execution requires
retirement and the adapter does not provide it, the operation fails before the
unsafe mutation begins.

Retirement should use the endpoint-local identity supplied by dns-sync and fail
closed when the object cannot be identified safely.

## Fail-closed behavior

Plugins should reject ambiguous or unsupported behavior.

In particular:

- reject endpoints owned by another plugin;
- reject actions intended for another endpoint;
- reject unsupported object types;
- reject unsupported plan operations;
- reject malformed native state;
- reject ambiguous endpoint-local identities;
- never guess missing IDs;
- never report successful application until the resulting object can be
  identified;
- never treat mutation success alone as verification.

The generic executor adds additional preflight and dependency safety checks.

## Configuration ownership

Configuration should live at the lowest layer that actually understands it.

Use this rule:

    Does generic reconciliation need to understand this value?

    yes -> core configuration may be appropriate
    no  -> plugin/client configuration

Examples of generic reconciliation policy include:

    DNS_TARGET_IP
    DOMAIN_MATCHING
    DOMAIN

Examples of plugin/client configuration include:

    NPM_ENDPOINTS
    PROXY_API_PORT
    PROXY_API_SCHEME
    NPM_IDENTITY
    NPM_SECRET
    VERIZON_ENDPOINTS
    ROUTER_PASSWORD
    ROUTER_TIMEOUT

`configured_endpoints(config)` owns endpoint discovery for the plugin.

The client receives the resulting explicit `Endpoint`; it should not implement
a second independent endpoint-selection mechanism.

## Secret handling

Secret material should remain below the normalized and persistence boundary
unless the core has an explicit safe representation for it.

Plugins must not place credentials, authentication tokens, private keys, or
other secret material into generic persisted payloads.

Runtime credentials should normally be provided through environment variables,
container secrets, or another external secret-management mechanism.

## Plugins

The generic reconciliation engine uses adapter plugins.

Built-in adapters:

- `npm` - Nginx Proxy Manager proxy hosts and certificates

External adapters are loaded from `DNS_SYNC_PLUGIN_PATH` and must also be
explicitly enabled through `ENABLED_PLUGINS`.

Example:

```text
ENABLED_PLUGINS=npm,verizon
DNS_SYNC_PLUGIN_PATH=/plugins
```

The Verizon adapter lives under `external_plugins/verizon` and is deliberately
not copied into the dns-sync container image. The external plugin directory is
mounted read-only at runtime.

## Plugin loading

Only explicitly enabled plugins are loadable.

The registry first checks for a built-in adapter under:

    dns_sync/plugins/adapters/<plugin_name>/

If no built-in adapter exists, it looks for an external package at:

    $DNS_SYNC_PLUGIN_PATH/<plugin_name>/__init__.py

External plugins are loaded as packages, so they may use relative imports for
private modules such as:

    from .client import ExampleClient

Imports from dns-sync itself should remain absolute, for example:

    from dns_sync.adapters import Endpoint
    from dns_sync.adapter_support import require_endpoint_plugin

The external plugin directory does not need to be added globally to
`PYTHONPATH`.

## Adding a new plugin

Start from:

    examples/plugin_template/

For an external plugin:

1. copy the template into the configured external plugin root;
2. rename the directory to the plugin name;
3. rename `ExampleClient` and `ExampleRecord`;
4. implement `configured_endpoints(config)`;
5. implement native authentication and transport in `client.py`;
6. declare only the generic capabilities actually supported;
7. implement native-to-`DiscoveredObject` translation;
8. implement `NormalizedObject`-to-native translation;
9. implement `apply()` and `verify()`;
10. implement `retire()` only when safe retirement is supported;
11. add focused client and adapter tests;
12. enable the plugin through `ENABLED_PLUGINS`;
13. run read-only reconciliation against the real endpoint before permitting
    mutations.

Do not begin a new integration by adding application-specific models or
transport behavior to the core.

## Reference implementations

Use both existing plugins when designing a new integration.

### NPM

The NPM plugin demonstrates:

- a built-in adapter;
- multiple configured endpoints;
- proxy-host and certificate object types;
- endpoint-local object IDs;
- cross-object certificate references;
- portable logical references;
- replacement workflows;
- deferred retirement;
- certificate material handling.

Its package lives at:

    dns_sync/plugins/adapters/npm/

### Verizon

The Verizon plugin demonstrates:

- an external adapter;
- plugin-owned endpoint configuration;
- a private native record model;
- a private endpoint client;
- DNS record discovery;
- create/update operations;
- verification;
- a read-only mounted external plugin package.

Its package lives at:

    external_plugins/verizon/

Together they demonstrate the preferred boundary:

    dns-sync core
        |
        +-- generic state and orchestration
        |
        +-- Plugin
              |
              +-- native / normalized translation
              |
              +-- <App>Client
                    |
                    +-- application-native implementation

## Plugin development validation

Before committing a plugin change, run at least:

    git diff --check
    python3 -m compileall -q dns_sync external_plugins tests examples
    python3 -m unittest discover -s tests -v

Changes involving plugin loading, native clients, endpoint configuration, or
mutation behavior should also be validated with a real read-only container
reconciliation against the intended endpoints.

A clean read-only plan does not by itself authorize mutation. Review unexpected
planned changes before enabling apply mode.

## Container runtime

The container periodically executes the generic reconciliation engine:

```text
python -m dns_sync reconcile
```

The default container mode is read-only.

```text
SYNC_INTERVAL=1
DNS_SYNC_APPLY=false
DNS_SYNC_STATE_DB=/var/lib/dns-sync/state.sqlite3
DNS_SYNC_PLUGIN_PATH=/plugins
ENABLED_PLUGINS=npm,verizon
```

`SYNC_INTERVAL` is expressed in minutes. The interval begins after each
reconciliation completes, so runs do not overlap.

To permit planned changes to be applied and verified:

```text
DNS_SYNC_APPLY=true
```

An optional mutation budget can be specified:

```text
DNS_SYNC_MAX_CHANGES=10
```

When `DNS_SYNC_APPLY=false`, reconciliation still performs discovery,
normalization, desired-state reconciliation, planning, and state persistence,
but no endpoint mutations are executed.

The SQLite database should be stored on persistent storage.

External plugins should be mounted read-only at `/plugins`.

## Main environment

```text
NPM_ENDPOINTS=192.168.1.3,192.168.1.4,192.168.1.5
PROXY_API_PORT=81
PROXY_API_SCHEME=http
DNS_TARGET_IP=192.168.1.101

VERIZON_ENDPOINTS=192.168.1.1

DOMAIN_MATCHING=false
DOMAIN=example.com

NPM_IDENTITY=...
NPM_SECRET=...
ROUTER_PASSWORD=...
```

## Domain filtering

Filtering is performed centrally after adapter discovery and before
reconciliation, so endpoint-specific adapters get identical behavior.

```text
DOMAIN_MATCHING=true
DOMAIN=example.com
```

This includes the apex and every subdomain:

```text
example.com
app.example.com
deep.app.example.com
*.example.com
```

It excludes unrelated names such as:

```text
example.net
notexample.com
example.com.evil.net
```

When `DOMAIN_MATCHING=false`, `DOMAIN` is ignored and all discovered records are
eligible for reconciliation.

When `DOMAIN_MATCHING=true`, `DOMAIN` is required.