backend-core Service Catalog and commons-core Boundary

Context and Problem Statement

backend-core is being cleansed into the untool.ai data/model runtime plane. It already owns concrete runtime surfaces such as UDA connectors, LLM Gateway, Agent Gateway, credentials broker entrypoints, OTel bootstrap, auth/RBAC enforcement, SSRF checks, secret resolution, and contract export tooling.

At the same time, commons-core is becoming the cross-cutting substrate for shared rules, contracts, SDK/MCP surfaces, validators, and forged client types. If backend-core continues to define fleet-wide rules locally, every second spoke will duplicate auth, privacy, error, identity, finops, contract, and security semantics.

The question: which parts of the backend-core service catalog stay here, which parts are consumed from commons-core, and which parts split runtime from contract/rule ownership?

Decision Drivers

• backend-core should stay focused on provider-differentiated runtime: data plane, model/AI gateways, graph/vector execution, ingest, egress enforcement, and runtime telemetry.
• commons-core should own reusable rules, contracts, validators, SDKs, MCP surfaces, and forged typed clients.
• Cross-spoke behavior must not drift across auth, audit, error, privacy, identity, finops, NATS, contract, and telemetry guardrails.
• Enterprise posture needs policy, audit, privacy, budget, and identity enforcement at runtime without duplicating policy definitions.
• OSS posture needs open interfaces, swappable providers, public contracts, conformance kits, and low-friction generated clients.
• Accelerators should make data faster and more open: embed-on-land, semantic cache, provenance everywhere, self-describing APIs, and forge-emitted clients.

Decision Outcome

Adopt the backend-core service catalog with three ownership tags:

Tag	Meaning
[Own]	backend-core expands and operates this runtime.
[Consume]	backend-core uses a commons-core SDK/MCP/tooling surface and keeps only a thin client or call site.
[Split]	runtime/enforcement stays in backend-core; contract/rule/schema/client lives in commons-core.

The anchoring rule is:

backend-core owns provider-differentiated runtime. commons-core owns cross-cutting rules, contracts, validators, SDK/MCP surfaces, and forged typed clients.

Service Catalog Boundary

Lane	Service family	Boundary
Data plane and UDA	[Own] UDA connector runtime, cross-modal semantic search, CDC/stream ingest, object/blob service; [Split] ingest/parsing contracts.	backend-core runs data movement and source-specific execution; commons-core owns connector/parser contracts and conformance.
AI and model gateways	[Own] LLM Gateway, embeddings, RAG orchestration; [Split] Agent Gateway contracts, web search/tool contracts, guardrail rule packs.	backend-core runs provider routing and gateway enforcement; commons-core owns vendor-neutral contracts and policy/rule definitions.
Semantic and ontology layer	[Split] ontology sift/sieve runtime and knowledge graph contracts; [Consume] schema abstraction/adapter builder.	backend-core executes against data/graph state; commons-core owns validators, proof formats, and forge surfaces.
Platform integration and messaging	[Split] platform capability messaging; [Consume] central memory, distributed locks, runbook triggers.	backend-core emits/consumes runtime events; commons-core owns shared schemas, validators, and orchestration clients.
Security, identity, governance	[Split] auth/RBAC, credentials broker; [Own] audit trail runtime and SSRF/egress guard; [Consume] privacy/PII, error validator, identity broker, security broker.	backend-core enforces per-request behavior; commons-core owns rule definitions, schemas, and clients.
Ops, observability, FinOps, sustainability	[Own] observability bootstrap; [Split] health endpoints; [Consume] DB diagnostics, finops guardrails, ESG/carbon.	backend-core emits live runtime data; commons-core owns shared checks, budgets, diagnostic and carbon models.
Developer experience and openness	[Own] contract-first pipeline, test console, public capability catalog, sample-data/notebooks, data marketplace publisher.	backend-core exposes self-describing runtime surfaces; commons-core receives forged clients and shared contract packages.

Architecture Rules

• Do not move UDA connectors, provider routers, parser execution, graph/vector query execution, or gateway enforcement out of backend-core unless the runtime itself becomes a separate platform service.
• Do move shared contracts, validators, rule definitions, generated client types, and conformance kits into commons-core.
• Do not let backend-core depend on private implementation details inside commons-core. Consume stable SDK/MCP/tool contracts.
• Do not let commons-core depend on backend-core runtime modules.
• A [Split] service must name both sides explicitly in code/docs: backend runtime owner and commons rule/contract owner.
• Every new backend-core route or provider capability must be contract-first and forge-compatible.

Consequences

Positive:

• backend-core stays lean and runtime-focused.
• Other spokes can consume the same rules/contracts instead of reimplementing them.
• Enterprise controls become consistent across data, model, graph, and agent surfaces.
• OSS users get open contracts, conformance kits, swappable providers, and generated clients.
• The platform can ship accelerators such as embed-on-land, semantic cache, citation graphs, and self-describing APIs without muddling ownership.

Negative / trade-offs:

• The first extraction needs compatibility wrappers and duplicated tests for a short period.
• [Split] services require more discipline than local-only code because each change may touch backend-core and commons-core.
• Commons-core package boundaries must be cleaned before publishing; generated/dist artifacts in the current worktree are noisy and should not define the source package shape.

Implementation Notes

The detailed migration table lives in:

.agent/plans/backend-core-service-catalog-cleansing.md

Start with shared plumbing extraction because it is already duplicated by design:

1. audit, errors, ssrf, secrets, env, and otel move to commons-owned package surfaces with backend-core wrappers.
2. auth follows after exact compatibility tests cover JWT/JWKS, role extraction, disabled-auth local principal, and denial audit emission.
3. Credentials broker splits into service runtime in backend-core and shared client/contracts in commons-core.
4. Contract export/drift tooling migrates toward commons governance commands while backend-core keeps route-specific exporters.

Validation

Backend-core validation:

python -m pytest -q tests/test_env_bootstrap.py tests/test_secret_resolution.py tests/test_ssrf.py
python -m pytest -q tests/test_connections_api.py tests/test_pipelines_api.py tests/test_credentials.py
python -m pytest -q tests/test_agent_gateway.py tests/test_llm_gateway.py
python scripts/export_openapi.py
python scripts/export_agent_gateway_contract.py
python scripts/export_llm_gateway_contract.py
python scripts/check_tier_separation.py

commons-core validation after package extraction:

python -m pytest -q
python -m compileall sdk/python/untool_common

Decision Log

Date	Decision
2026-06-01	Ratify backend-core as provider-differentiated runtime owner and commons-core as shared rule/contract/SDK/MCP owner.
2026-06-01	Adopt [Own], [Consume], and [Split] tags for every backend-core service family.
2026-06-01	Start cleansing with shared plumbing extraction through compatibility wrappers before broad service expansion.