beaver_project/docs/superpowers/specs/2026-06-02-external-sidecar-connectors-design.md

External Sidecar Connectors Design

Date: 2026-06-02

Goal

Add real Weixin personal-account QR login and Feishu/Lark plugin onboarding to Beaver through a docker-compose predeclared sidecar service, without binding Beaver's connector layer to one vendor runtime. Beaver must not dynamically create containers or require Docker socket access.

This design implements the next connector layer after docs/superpowers/plans/2026-06-02-channel-connectors-foundation.md.

Design Corrections

This design intentionally fixes four architecture constraints before implementation:

• The sidecar is generic. Beaver depends on a connector HTTP contract, not on one vendor runtime.
• Pairing is modeled as a broader ConnectorSession, because Feishu/Lark install/link flows are not only QR pairing.
• Bridge events include eventId, timestamp, and deliveryAttempt, and Beaver dedupes bridge events before they can trigger duplicate agent replies.
• Bridge authentication is service-level in the first version. The shared connector token lives in environment variables, not per-connection credentials.
• Outbound sidecar sends include a required requestId so sidecar retries are idempotent.
• Connected sessions dynamically register runtime channels. A successful Weixin or Feishu/Lark connection must not require a Beaver restart.

Scope

Included:

• A repo-local external-connector sidecar service.
• A docker-compose service declaration for the sidecar.
• A sidecar ConnectorProvider abstraction.
• A production VendorCliProvider that runs the real vendor CLI/plugin commands required for Weixin personal-account QR login and Feishu/Lark plugin onboarding.
• Sidecar HTTP API for health, connector metadata, connector sessions, logout/remove, outbound send, and inbound event forwarding.
• Beaver WeixinConnector and FeishuConnector objects registered in ChannelConnectorRegistry.
• Beaver connector bridge endpoints that accept normalized sidecar inbound events and submit them to ChannelRuntime.accept_inbound().
• MessageDedupeStore for connector bridge event idempotency.
• ExternalConnectorChannel runtime object for sidecar-backed outbound sends.
• ChannelRuntime.add_channel() and ChannelRuntime.remove_channel() for dynamic runtime activation.
• Web UI connection wizard for Weixin QR login and Feishu/Lark plugin onboarding.
• Unit tests using fake sidecar providers and bridge events.

Excluded:

• Dynamic Docker container creation from Beaver.
• Docker socket mounts in Beaver.
• Reimplementing Weixin iLink or Feishu/Lark plugin protocols inside Beaver.
• Building a generic plugin marketplace.
• Multi-user enterprise permission governance beyond local connector ownership and bridge token validation.

Architecture

Use one predeclared sidecar for external connector providers:

Beaver backend
-> Connector HTTP client
-> external-connector sidecar
-> ConnectorProvider
-> provider-specific runtime or CLI
-> Weixin / Feishu / future platform

Beaver owns:

• connection state in ChannelConnectionStore
• credential references in CredentialStore
• connector session state exposed to the web UI
• service-level connector authentication
• bridge event dedupe
• normalized runtime message admission
• runtime channel lifecycle
• runtime dedupe/session identity
• outbound dispatch into sidecar /send

The sidecar owns:

• provider runtime state
• provider install/update commands
• Weixin QR login and login-state persistence
• Feishu/Lark plugin install, bot creation/linking, and provider-side verification
• platform receive loops
• sidecar-to-Beaver inbound event delivery

ConnectorProvider

The sidecar must isolate provider-specific behavior behind a provider contract. Beaver must not know which provider implementation is active.

interface ConnectorProvider {
  providerId: string;
  connectors(): ConnectorDescriptor[];
  health(): Promise<ProviderHealth>;
  startSession(input: StartConnectorSessionInput): Promise<ConnectorSessionView>;
  getSession(sessionId: string): Promise<ConnectorSessionView>;
  cancelSession(sessionId: string): Promise<void>;
  logout(connectionId: string): Promise<void>;
  send(input: SendMessageInput): Promise<SendMessageResult>;
}

Initial provider:

• VendorCliProvider: runs the real CLI/plugin commands required by the current Weixin and Feishu/Lark vendor flows.

Future providers can be added without changing Beaver runtime code:

• WechatyProvider
• NapcatProvider
• OneBotProvider
• EnterpriseWeixinProvider

Provider choice is sidecar configuration, not Beaver architecture. ExternalConnectorChannel only calls the sidecar HTTP contract.

Runtime Flow

Inbound:

platform event
-> ConnectorProvider inside sidecar
-> sidecar normalized bridge event
-> POST Beaver /api/channel-connector-bridge/events
-> MessageDedupeStore
-> ChannelRuntime.accept_inbound()
-> MessageBus
-> AgentService

Outbound:

AgentService
-> MessageBus outbound
-> ChannelManager.dispatch_outbound()
-> ExternalConnectorChannel.send()
-> POST sidecar /send
-> ConnectorProvider.send()
-> platform

ExternalConnectorChannel implements the existing runtime channel protocol:

channel_id: str
kind: str
mode: str

async def start() -> None
async def stop() -> None
async def send(message: OutboundMessage) -> None

It is not a platform protocol adapter. It is a generic HTTP bridge to a sidecar.

Runtime materialization for sidecar-backed connections always emits:

ChannelConfig(
    enabled=True,
    kind="external_connector",
    mode="http",
    account_id=spec.account_id,
    display_name=spec.display_name,
    config={
        "platformKind": "weixin",
        "connectionId": "conn_...",
        "sidecarBaseUrl": "http://external-connector:8787",
    },
    secrets={},
)

The original ChannelConnection.kind remains weixin or feishu; only the runtime transport kind is generic.

ExternalConnectorChannel authenticates outbound calls with the service-level connector token configured in Beaver's process environment, not with a per-channel secret. The same first-version deployment may use one shared token value for both directions, exposed as EXTERNAL_CONNECTOR_TOKEN to Beaver and BEAVER_BRIDGE_TOKEN to the sidecar.

Dynamic Runtime Activation

A connected connector session must activate without restarting Beaver.

Add runtime methods:

async def add_channel(self, channel_id: str, config: ChannelConfig) -> None:
    ...

async def remove_channel(self, channel_id: str) -> None:
    ...

add_channel() must run under a runtime lifecycle lock and has deterministic duplicate semantics:

• Same channel_id and same effective ChannelConfig: no-op.
• Same channel_id and changed effective ChannelConfig: build and start the replacement adapter before swapping it into the manager; after the swap succeeds, stop the old adapter.
• Replacement start failure: keep the old adapter registered and running, and return the failure to the caller.
• First registration after runtime start: build the adapter, register it, then start only that adapter.

remove_channel() must also run under the lifecycle lock. Missing channel ids are no-op; existing channels are stopped and unregistered.

When a connector session reaches connected:

Connector session connected
-> connector updates ChannelConnection
-> registry materializes ChannelConfig
-> ChannelRuntime.add_channel(channel_id, config)
-> ChannelManager.register(adapter)
-> adapter.start()
-> channel status becomes running

This is a hard requirement for Weixin and Feishu/Lark onboarding. Manual backend restart is not an acceptable success path for this feature.

remove_channel() is used when a user logs out or revokes a sidecar connection:

logout / revoke
-> sidecar logout
-> ChannelRuntime.remove_channel(channel_id)
-> connection status revoked or disconnected

Sidecar Deployment

Add a sidecar service that can be enabled in deployment:

services:
  external-connector:
    build: ./external-connector
    restart: unless-stopped
    environment:
      BEAVER_BRIDGE_BASE_URL: http://app-instance:8080
      BEAVER_BRIDGE_TOKEN: ${BEAVER_BRIDGE_TOKEN}
      CONNECTOR_API_TOKEN: ${EXTERNAL_CONNECTOR_TOKEN}
      CONNECTOR_HOME: /var/lib/external-connector
      CONNECTOR_PROVIDER: vendor_cli
    volumes:
      - external-connector-state:/var/lib/external-connector

For the current create-instance.sh-style deployment, the implementation adds:

• docker-compose.external-connectors.yml for local/development sidecar tests.
• documentation for attaching external-connector to the same Docker network as the target app instance.
• instance environment EXTERNAL_CONNECTOR_BASE_URL=http://external-connector:8787.
• instance environment EXTERNAL_CONNECTOR_TOKEN=<service-level shared secret>.

The implementation must not depend on Beaver mounting /var/run/docker.sock.

Sidecar HTTP API

All sidecar requests and responses are JSON. The sidecar listens on port 8787.

GET  /health
GET  /connectors
POST /connector-sessions
GET  /connector-sessions/{session_id}
POST /connector-sessions/{session_id}/cancel
POST /connections/{connection_id}/logout
POST /send

GET /connectors returns:

[
  {
    "kind": "weixin",
    "displayName": "Weixin",
    "authType": "qr",
    "providerId": "vendor_cli",
    "capabilities": ["receive_text", "send_text", "receive_media", "direct_messages"]
  },
  {
    "kind": "feishu",
    "displayName": "Feishu/Lark",
    "authType": "plugin_install",
    "providerId": "vendor_cli",
    "capabilities": ["receive_text", "send_text", "receive_media", "groups"]
  }
]

POST /connector-sessions request:

{
  "kind": "weixin",
  "connectionId": "conn_...",
  "channelId": "weixin-main",
  "displayName": "Weixin Main",
  "callbackBaseUrl": "http://app-instance:8080",
  "options": {}
}

The sidecar authenticates the connector-session request with Authorization: Bearer <EXTERNAL_CONNECTOR_TOKEN>. It already has BEAVER_BRIDGE_TOKEN from its environment, so Beaver does not send bridge tokens in connector-session bodies.

For Feishu/Lark, kind is feishu and options may include domain, mode, and optional app credentials when linking an existing bot. If using the official plugin installer to create a bot, the sidecar starts that installer flow and reports QR, instruction, or action status back to Beaver.

GET /connector-sessions/{session_id} response:

{
  "sessionId": "cs_...",
  "kind": "weixin",
  "status": "qr_ready",
  "qrCode": "weixin://...",
  "qrImage": "data:image/png;base64,...",
  "instructions": [],
  "accountId": null,
  "displayName": null,
  "error": null,
  "metadata": {}
}

Allowed connector session statuses:

• pending
• qr_ready
• scanned
• confirmed
• installing
• waiting_for_user
• connected
• expired
• error
• cancelled

POST /send request:

{
  "requestId": "out_...",
  "connectionId": "conn_...",
  "channelId": "weixin-main",
  "kind": "weixin",
  "target": {
    "peerId": "wx_user_or_chat_id",
    "peerType": "dm",
    "threadId": null
  },
  "content": "reply text",
  "metadata": {
    "contextToken": "optional"
  }
}

requestId is required. Beaver must generate a stable request id for each outbound delivery attempt from the outbound message identity, and must reuse the same requestId if the same outbound delivery is retried. The sidecar dedupes connectionId + requestId; duplicate requests return the original send result and must not send a second platform message.

Beaver Bridge API

Add a backend bridge endpoint for sidecar inbound messages:

POST /api/channel-connector-bridge/events

The sidecar must authenticate every bridge request using the service-level bearer token from BEAVER_BRIDGE_TOKEN. Beaver rejects missing or invalid bridge tokens. Bridge tokens are deployment secrets, not connection records.

Bridge event body:

{
  "eventId": "provider-event-id",
  "timestamp": "2026-06-02T09:30:00Z",
  "deliveryAttempt": 1,
  "connectionId": "conn_...",
  "channelId": "weixin-main",
  "kind": "weixin",
  "accountId": "weixin:...",
  "peerId": "wx_user_or_chat_id",
  "peerType": "dm",
  "userId": "wx_sender",
  "threadId": null,
  "messageId": "platform-message-id",
  "messageType": "text",
  "content": "hello",
  "metadata": {
    "contextToken": "optional"
  }
}

The bridge endpoint must:

1. validate bearer token
2. load ChannelConnection
3. reject unknown or revoked connections
4. dedupe by connectionId + eventId through MessageDedupeStore
5. construct ChannelIdentity
6. construct InboundMessage
7. call ChannelRuntime.accept_inbound()
8. mark bridge event completed or failed

MessageDedupeStore

Add a JSON-backed MessageDedupeStore under:

<workspace>/state/channel_connections/message_dedupe.json

It stores:

@dataclass
class ConnectorMessageDedupeRecord:
    dedupe_key: str
    connection_id: str
    event_id: str
    status: str
    first_seen_at: str
    updated_at: str
    delivery_attempts: int
    message_id: str | None
    last_error: str | None

status values:

• processing
• completed
• failed

Duplicate handling:

• completed: return idempotent success and do not call ChannelRuntime.accept_inbound() again.
• processing updated less than 60 seconds ago: return 409 Conflict with {"retryAfterSeconds": 5} so the sidecar retries later.
• processing updated 60 seconds or more ago: treat the record as stale, increment delivery_attempts, update updated_at, and reprocess the event.
• failed: allow reprocessing on the next delivery attempt, increment delivery_attempts, and clear last_error before calling runtime.

This store is separate from runtime session dedupe. Runtime dedupe still protects platform message identity, while bridge dedupe protects connector retries.

Beaver Connectors

WeixinConnector

Responsibilities:

• discover sidecar health
• start Weixin connector session through sidecar /connector-sessions
• poll sidecar connector session status
• create or update ChannelConnection
• store sidecar connection state reference in CredentialStore when the provider returns one
• validate by checking sidecar connection status
• materialize runtime config for ExternalConnectorChannel
• activate runtime via ChannelRuntime.add_channel() when connected
• revoke/logout by calling sidecar /connections/{connection_id}/logout
• deactivate runtime via ChannelRuntime.remove_channel() on logout/revoke

FeishuConnector

Responsibilities:

• discover sidecar health
• start Feishu/Lark plugin install/link connector session
• optionally pass appId/appSecret/domain/mode for existing bot linking
• poll installer/session status
• create or update ChannelConnection
• validate by sidecar session or connection status
• materialize runtime config for ExternalConnectorChannel
• activate runtime via ChannelRuntime.add_channel() when connected
• revoke/remove plugin connection by calling sidecar logout/remove API
• deactivate runtime via ChannelRuntime.remove_channel() on logout/revoke

Feishu is sidecar-backed in this design because the user's supplied Feishu article describes the official plugin flow, not only a static bot-credential runtime adapter.

Frontend

Replace the old static Weixin and Feishu fields with connector-driven UI:

• fetch GET /api/channel-connectors
• show Telegram, Weixin, and Feishu/Lark as connector options
• for Weixin:
  • start connector session
  • show QR image
  • poll status until connected/expired/error
  • show connected account and logout
• for Feishu/Lark:
  • choose create bot or link existing bot
  • collect domain and optional app credentials
  • start sidecar connector session
  • show QR/instructions/status returned by sidecar
  • show connected account and logout

The old /api/channels static config editor may remain for advanced runtime config, but connector onboarding should not rely on manual JSON editing or direct token entry for Weixin/Feishu.

Error Handling

• Sidecar unavailable: show connector as unavailable; do not create a running connection.
• Provider install command fails: status error, with redacted stderr summary.
• QR expired: status expired, user can start a new connector session.
• Bridge token invalid: reject with 401, record event without platform secret values.
• Unknown connection id in bridge event: reject with 404.
• Duplicate completed bridge event: return idempotent success and do not call runtime again.
• Duplicate in-flight bridge event: return 409 Conflict until the 60-second processing TTL expires, then allow one reprocess.
• Outbound send failure: mark outbound delivery failed and record connector error.
• Duplicate outbound send requestId: sidecar returns the original send result and does not send a second platform message.
• Sidecar restart: persisted provider state should survive through sidecar volume.

Security

• Beaver never logs raw tokens, app secrets, bridge tokens, or sidecar connection tokens.
• Bridge authentication uses a service-level token from environment variables. It is not stored per connection and is never returned by APIs.
• Sidecar can only call bridge endpoints with the service-level bridge token.
• Beaver can only call sidecar control and send endpoints with the service-level connector token.
• Sidecar state volume contains login state and must be treated as sensitive.
• Feishu user-identity mode has stronger privacy risk than bot-identity mode; UI must label it clearly if exposed.

Testing

Backend unit tests:

• sidecar client fake for Weixin connector session start/status/logout/send
• sidecar client fake for Feishu connector session start/status/logout/send
• ExternalConnectorChannel.send() target mapping
• ExternalConnectorChannel.send() includes stable requestId and connector bearer auth
• ChannelRuntime.add_channel() dynamically starts and registers a channel
• ChannelRuntime.add_channel() no-ops for identical config, replaces changed config, and keeps the old channel if replacement start fails
• ChannelRuntime.remove_channel() stops and unregisters a channel
• bridge endpoint accepts valid events
• bridge endpoint rejects invalid token and unknown connection id
• bridge endpoint dedupes repeated eventId and calls runtime once
• bridge endpoint returns 409 Conflict for non-stale processing duplicates and reprocesses stale records
• registry lists telegram, weixin, and feishu
• materialized sidecar connections produce ChannelConfig(kind="external_connector", mode="http") compatible with runtime factory

Sidecar tests:

• HTTP API shape for health/connectors/connector-sessions/send
• fake provider status transitions
• provider command runner error redaction
• send idempotency for duplicate connectionId + requestId

Frontend tests:

• Weixin connector option opens QR modal
• polling reaches connected state
• expired/error states are visible
• Feishu flow starts install/link and shows returned instructions/status

Manual verification:

• Build app and sidecar Docker images.
• Start docker-compose sidecar setup.
• In terminaltest, open Weixin connector, scan QR, observe connected status without restarting Beaver.
• Send a Weixin text message and verify Beaver receives it once.
• Force sidecar retry of the same event and verify Beaver does not produce a duplicate agent reply.
• Send a Beaver reply and verify sidecar /send path.
• Start Feishu connector flow using the official Feishu/Lark plugin install path and verify the vendor-provided start command.

Rollout

Implement in this order:

1. Sidecar HTTP contract with fake provider.
2. MessageDedupeStore.
3. Beaver ExternalConnectorChannel and bridge endpoint.
4. ChannelRuntime.add_channel() and ChannelRuntime.remove_channel().
5. Weixin connector against fake sidecar client.
6. Feishu connector against fake sidecar client.
7. Frontend connector UI.
8. Production VendorCliProvider that shells out to real vendor CLI/plugin commands.
9. Docker build/compose integration.
10. Manual live verification.

The fake provider is test-only. The production provider must use the real vendor CLI/plugin commands for Weixin and Feishu/Lark; the fake provider only makes Beaver and frontend tests deterministic while the live provider handles non-deterministic external login and install flows.