protos

protogate

Migration tool and delegation platform for extracting bounded slices from legacy systems with minimal coupling. Built on SUMD + DOQL + testql + taskfile ecosystem.

Package-level operational guide: see protogate/README.md.

Architecture

c2004 owns (c2004-first):

protogate owns:

protogate is not the source-of-truth for migration contracts, discovery logic, or planning artifacts.

Each delegated module follows a vertical-slice template:

Quick Start

# Install protogate CLI
pip install -e .

# Generate all artifacts from contracts
protogate generate all

# Run specific generator
protogate generate python
protogate generate zod

# Schema registry operations
protogate registry check contracts/user/v1/user.proto
protogate registry list

# Run gateway
protogate gateway

# Run full CI pipeline
protogate ci

Using Makefile (Legacy)

# Install dependencies
pip install -r requirements.txt

# Run gateway (development mode)
make gateway

# Run full CI pipeline
make ci

API Overview

Core Endpoints

Method Endpoint Description
GET /health Platform health + module aggregation
GET /health/modules All delegated slices status
GET /health/modules/{slice} Specific slice health
GET /delegation/slices List all delegated slices
GET /delegation/slices/{slice} Slice details & metadata

User Module (Live)

Method Endpoint Description
POST /commands/user/create Create user
POST /commands/user/dual-create Dual-write with idempotency
POST /commands/user/{id}/change-email Change email
POST /commands/user/{id}/deactivate Deactivate user
GET /queries/user/{id} Get user state
GET /events Event stream

Search Module (Phase-1)

Method Endpoint Description
POST /commands/search/index Index entry
GET /queries/search?q={query} Full-text search

Code Generation

The project provides multiple code generators from Protobuf contracts:

Makefile Targets

Target Description
make proto Generate gRPC stubs via buf (requires buf CLI)
make zod Generate TypeScript Zod schemas
make python Generate Pydantic Python models
make json Generate JSON Schema (draft-07)
make sql Generate SQL DDL
make proto-all Run all generators (proto + zod + python + json + sql)
make proto-changed Detect changed proto files against main branch
make generate-incremental Incremental regeneration (only changed proto files)
make clean Remove all generated artifacts

Schema Registry

Manage schema versions and compatibility:

Target Description
make registry-register Register proto file in schema registry
make registry-check Check compatibility without registering
make registry-list List all schemas in registry

Legacy Bridge

Legacy schema migration and synchronization:

Target Description
make legacy-register Register legacy JSON schema + proto mapping
make diff-legacy Diff legacy vs proto schemas
make legacy-report Generate detailed migration report
make legacy-list List all legacy schemas
make sync-check Full sync check (fails if readiness < 1.0)
make bootstrap-legacy Bootstrap EventStore from legacy DB

CI Pipeline

Target Description
make ci Full CI: lint → generate → test → registry check

Contract Enum Cross-Check

protogate codegen registry cross-checks CQRS contract JSON files (*.command.json, *.query.json, *.event.json) against Pydantic Literal[...] annotations found in layers.python. It catches the drift class that broke ADR-012 Wave 2 in c2004, where a server could return an enum value the contract did not advertise, crashing the client decoder.

CLI

# 1. Report-only gate (CI default)
protogate codegen registry contracts/ --check --cross-check-pydantic

# 2. Auto-fix warnings (safe; touches JSON only)
protogate codegen registry contracts/ --cross-check-pydantic --fix-safe

# 3. Auto-fix warnings + expand output enums (opt-in; may bless server bugs)
protogate codegen registry contracts/ --cross-check-pydantic --fix-safe --auto-expand-output

Directional rules

Direction Rule Verdict
output/payload pydantic ⊆ contract compatible
output/payload pydantic ⊋ contract error — client may crash on undeclared value
output/payload contract ⊋ pydantic warning — dead code paths on client
input contract ⊆ pydantic compatible
input contract ⊋ pydantic error — server rejects valid-per-contract input (HTTP 422)
input pydantic ⊋ contract compatible (intentional API restriction)

Pydantic source is never modified; all fixes apply to contract JSON only. Input-direction errors are never auto-fixed (require human decision between narrowing the contract or loosening Pydantic).

See docs/contract-cross-check.md for the full reference, the c2004 Makefile integration, and the Wave 2 post-mortem that motivated the feature.

Delegation Workflow

  1. Generate candidate report in c2004 (detect_migration_candidates.py)

  2. Generate delegation plan in protogate:

    python scripts/legacy_bridge/generate_delegation_plan.py \
  --input /path/to/c2004/module-candidates.json \
  --clusters /path/to/c2004/cqrs-pattern-clusters.json \
  --output-dir docs

  3. (Recommended) Run full discovery pipeline in protogate:

    python scripts/legacy_bridge/run_arch_migration_discovery.py \
  --repo-root /path/to/c2004 \
  --output-dir reports/migration-discovery \
  --delegation-limit 30
  4. Pick top module from Phase-1
  5. Implement full vertical slice in protogate
  6. Switch c2004 route to iframe host
  7. Validate parity & archive legacy

Project Structure

protogate/
├── contracts/              # Protobuf contracts per slice
│   ├── user/v{1,2}/
│   ├── search/v1/
│   └── legacy_bridge/
├── gateway/                # FastAPI gateway
│   ├── main.py            # Entry point & routes
│   ├── delegation.py      # Slice registry & health
│   ├── user_handler.py    # User CQRS handlers
│   ├── search_handler.py  # Search CQRS handlers
│   └── static/            # Delegated UI assets
├── adapters/              # Legacy ↔ Proto adapters
├── scripts/               # Code generation & migration
│   ├── generate_zod.py    # TypeScript Zod generator
│   ├── generate_pydantic.py # Python Pydantic generator
│   ├── generate_json_schema.py # JSON Schema generator
│   ├── generate_sql.py    # SQL DDL generator
│   ├── generate_incremental.py # Incremental regeneration
│   ├── schema_registry.py # Proto schema registry
│   ├── legacy_registry.py # Legacy schema registry
│   ├── event_store.py     # CQRS event store
│   ├── conflict_resolver.py # Event conflict resolution
│   ├── dual_writer.py     # Dual-write pattern
│   ├── idempotency_store.py # Idempotency tracking
│   ├── vector_clock.py    # Vector clock for ordering
│   └── legacy_bridge/     # Migration tooling
│       ├── run_arch_migration_discovery.py # Full orchestrator
│       ├── detect_migration_candidates.py # Module scoring
│       ├── analyze_service_boundaries.py # Frontend/backend analysis
│       ├── detect_cqrs_pattern_clusters.py # CQRS pattern detection
│       ├── generate_migration_wave_plan.py # Wave planning
│       ├── delegation_plan.py # Delegation plan logic
│       ├── generate_delegation_plan.py # Plan generator
│       ├── migrator.py # Legacy to EventStore migration
│       ├── sync_check.py # Sync validation
│       └── diff_engine.py # Schema diffing
├── tests/                 # Test suite
└── docs/                  # Generated plans
    ├── delegation-plan.generated.json
    └── delegation-plan.generated.md

Key Components

Code Generators

Schema Registry

SQLite-backed schema registry with compatibility enforcement (scripts/schema_registry.py):

Legacy Bridge

Comprehensive migration tooling for legacy systems:

CQRS Infrastructure

DelegatedSlice Registry

Runtime model for slice metadata in gateway/delegation.py:

DelegatedSlice(
    name="search",
    phase="phase-1",           # phase-1 | phase-2 | live
    backend="delegated",
    frontend="static",         # none | static | planned
    contract_paths=("contracts/search/v1/search.proto",),
    command_routes=("/commands/search/index",),
    query_routes=("/queries/search",),
    smoke_checks=("/health", "/queries/search?q=test"),
)

Health Checks

Per-slice health validates:

Returns ok or degraded with missing requirements listed.

Deployment

Docker Compose

# Build and run all services
docker-compose up

# Services:
# - generator: Proto code generation
# - gateway: FastAPI gateway (port 8080)

Gateway Docker

# Build gateway image
docker build -f gateway/Dockerfile -t semcod-gateway .

# Run gateway container
docker run --rm -p 8080:8080 semcod-gateway

Or use Makefile:

make gateway-docker

Environment Variables

Variable Default Description
OPENROUTER_API_KEY *(not set)* OpenRouter API key (https://openrouter.ai/keys)
LLM_MODEL openrouter/qwen/qwen3-coder-next LLM model for AI-assisted features
PFIX_AUTO_APPLY true Apply fixes without asking
PFIX_AUTO_INSTALL_DEPS true Auto pip/uv install dependencies
PFIX_AUTO_RESTART false Restart after fix
PFIX_MAX_RETRIES 3 Max retry attempts
PFIX_ENABLED true Enable auto-fix features
PFIX_GIT_COMMIT false Auto-commit fixes
PFIX_GIT_PREFIX pfix: Commit message prefix

Testing

TestQL Scenarios

Auto-generated API smoke tests in testql-scenarios/generated-api-smoke.testql.toon.yaml:

Pytest

# Run all tests
pytest tests/ -v

# Run specific test
pytest tests/test_event_store.py -v

Release Management

AI Cost Tracking

PyPI Version Python License AI Cost Human Time Model

Generated on 2026-04-26 using openrouter/qwen/qwen3-coder-next

License

Licensed under Apache-2.0.

Status

Last updated by taskill at 2026-04-25 13:43 UTC

Metric Value
HEAD 3d1c764
Coverage —
Failing tests —
Commits in last cycle 40

Various tests, docs, and refactors plus new features for codegen/registry and protogate CLI. Additions include directional subset checks, a –cross-check-pydantic flag, migration-analysis CLI commands, and schema registry improvements (conflict resolution, vector clocks, v2 proto).

This site is open source. Improve this page.