docs: decouple external connector sidecar design
This commit is contained in:
@ -0,0 +1,538 @@
|
|||||||
|
# External Sidecar Connectors Design
|
||||||
|
|
||||||
|
Date: 2026-06-02
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Add real Weixin personal-account QR login and Feishu/Lark plugin onboarding to Beaver through a docker-compose predeclared sidecar service, without binding Beaver's connector layer to one vendor runtime. Beaver must not dynamically create containers or require Docker socket access.
|
||||||
|
|
||||||
|
This design implements the next connector layer after `docs/superpowers/plans/2026-06-02-channel-connectors-foundation.md`.
|
||||||
|
|
||||||
|
## Design Corrections
|
||||||
|
|
||||||
|
This design intentionally fixes four architecture constraints before implementation:
|
||||||
|
|
||||||
|
- The sidecar is generic. Beaver depends on a connector HTTP contract, not on one vendor runtime.
|
||||||
|
- Pairing is modeled as a broader `ConnectorSession`, because Feishu/Lark install/link flows are not only QR pairing.
|
||||||
|
- Bridge events include `eventId`, `timestamp`, and `deliveryAttempt`, and Beaver dedupes bridge events before they can trigger duplicate agent replies.
|
||||||
|
- Connected sessions dynamically register runtime channels. A successful Weixin or Feishu/Lark connection must not require a Beaver restart.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
Included:
|
||||||
|
|
||||||
|
- A repo-local `external-connector` sidecar service.
|
||||||
|
- A docker-compose service declaration for the sidecar.
|
||||||
|
- A sidecar `ConnectorProvider` abstraction.
|
||||||
|
- A production `VendorCliProvider` that runs the real vendor CLI/plugin commands required for Weixin personal-account QR login and Feishu/Lark plugin onboarding.
|
||||||
|
- Sidecar HTTP API for health, connector metadata, connector sessions, logout/remove, outbound send, and inbound event forwarding.
|
||||||
|
- Beaver `WeixinConnector` and `FeishuConnector` objects registered in `ChannelConnectorRegistry`.
|
||||||
|
- Beaver connector bridge endpoints that accept normalized sidecar inbound events and submit them to `ChannelRuntime.accept_inbound()`.
|
||||||
|
- `MessageDedupeStore` for connector bridge event idempotency.
|
||||||
|
- `ExternalConnectorChannel` runtime object for sidecar-backed outbound sends.
|
||||||
|
- `ChannelRuntime.add_channel()` and `ChannelRuntime.remove_channel()` for dynamic runtime activation.
|
||||||
|
- Web UI connection wizard for Weixin QR login and Feishu/Lark plugin onboarding.
|
||||||
|
- Unit tests using fake sidecar providers and bridge events.
|
||||||
|
|
||||||
|
Excluded:
|
||||||
|
|
||||||
|
- Dynamic Docker container creation from Beaver.
|
||||||
|
- Docker socket mounts in Beaver.
|
||||||
|
- Reimplementing Weixin iLink or Feishu/Lark plugin protocols inside Beaver.
|
||||||
|
- Building a generic plugin marketplace.
|
||||||
|
- Multi-user enterprise permission governance beyond local connector ownership and bridge token validation.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
Use one predeclared sidecar for external connector providers:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Beaver backend
|
||||||
|
-> Connector HTTP client
|
||||||
|
-> external-connector sidecar
|
||||||
|
-> ConnectorProvider
|
||||||
|
-> provider-specific runtime or CLI
|
||||||
|
-> Weixin / Feishu / future platform
|
||||||
|
```
|
||||||
|
|
||||||
|
Beaver owns:
|
||||||
|
|
||||||
|
- connection state in `ChannelConnectionStore`
|
||||||
|
- credential references in `CredentialStore`
|
||||||
|
- connector session state exposed to the web UI
|
||||||
|
- bridge endpoint authentication
|
||||||
|
- bridge event dedupe
|
||||||
|
- normalized runtime message admission
|
||||||
|
- runtime channel lifecycle
|
||||||
|
- runtime dedupe/session identity
|
||||||
|
- outbound dispatch into sidecar `/send`
|
||||||
|
|
||||||
|
The sidecar owns:
|
||||||
|
|
||||||
|
- provider runtime state
|
||||||
|
- provider install/update commands
|
||||||
|
- Weixin QR login and login-state persistence
|
||||||
|
- Feishu/Lark plugin install, bot creation/linking, and provider-side verification
|
||||||
|
- platform receive loops
|
||||||
|
- sidecar-to-Beaver inbound event delivery
|
||||||
|
|
||||||
|
## ConnectorProvider
|
||||||
|
|
||||||
|
The sidecar must isolate provider-specific behavior behind a provider contract. Beaver must not know which provider implementation is active.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
interface ConnectorProvider {
|
||||||
|
providerId: string;
|
||||||
|
connectors(): ConnectorDescriptor[];
|
||||||
|
health(): Promise<ProviderHealth>;
|
||||||
|
startSession(input: StartConnectorSessionInput): Promise<ConnectorSessionView>;
|
||||||
|
getSession(sessionId: string): Promise<ConnectorSessionView>;
|
||||||
|
cancelSession(sessionId: string): Promise<void>;
|
||||||
|
logout(connectionId: string): Promise<void>;
|
||||||
|
send(input: SendMessageInput): Promise<SendMessageResult>;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Initial provider:
|
||||||
|
|
||||||
|
- `VendorCliProvider`: runs the real CLI/plugin commands required by the current Weixin and Feishu/Lark vendor flows.
|
||||||
|
|
||||||
|
Future providers can be added without changing Beaver runtime code:
|
||||||
|
|
||||||
|
- `WechatyProvider`
|
||||||
|
- `NapcatProvider`
|
||||||
|
- `OneBotProvider`
|
||||||
|
- `EnterpriseWeixinProvider`
|
||||||
|
|
||||||
|
Provider choice is sidecar configuration, not Beaver architecture. `ExternalConnectorChannel` only calls the sidecar HTTP contract.
|
||||||
|
|
||||||
|
## Runtime Flow
|
||||||
|
|
||||||
|
Inbound:
|
||||||
|
|
||||||
|
```text
|
||||||
|
platform event
|
||||||
|
-> ConnectorProvider inside sidecar
|
||||||
|
-> sidecar normalized bridge event
|
||||||
|
-> POST Beaver /api/channel-connector-bridge/events
|
||||||
|
-> MessageDedupeStore
|
||||||
|
-> ChannelRuntime.accept_inbound()
|
||||||
|
-> MessageBus
|
||||||
|
-> AgentService
|
||||||
|
```
|
||||||
|
|
||||||
|
Outbound:
|
||||||
|
|
||||||
|
```text
|
||||||
|
AgentService
|
||||||
|
-> MessageBus outbound
|
||||||
|
-> ChannelManager.dispatch_outbound()
|
||||||
|
-> ExternalConnectorChannel.send()
|
||||||
|
-> POST sidecar /send
|
||||||
|
-> ConnectorProvider.send()
|
||||||
|
-> platform
|
||||||
|
```
|
||||||
|
|
||||||
|
`ExternalConnectorChannel` implements the existing runtime channel protocol:
|
||||||
|
|
||||||
|
```python
|
||||||
|
channel_id: str
|
||||||
|
kind: str
|
||||||
|
mode: str
|
||||||
|
|
||||||
|
async def start() -> None
|
||||||
|
async def stop() -> None
|
||||||
|
async def send(message: OutboundMessage) -> None
|
||||||
|
```
|
||||||
|
|
||||||
|
It is not a platform protocol adapter. It is a generic HTTP bridge to a sidecar.
|
||||||
|
|
||||||
|
Runtime materialization for sidecar-backed connections always emits:
|
||||||
|
|
||||||
|
```python
|
||||||
|
ChannelConfig(
|
||||||
|
enabled=True,
|
||||||
|
kind="external_connector",
|
||||||
|
mode="http",
|
||||||
|
account_id=spec.account_id,
|
||||||
|
display_name=spec.display_name,
|
||||||
|
config={
|
||||||
|
"platformKind": "weixin",
|
||||||
|
"connectionId": "conn_...",
|
||||||
|
"sidecarBaseUrl": "http://external-connector:8787",
|
||||||
|
},
|
||||||
|
secrets={"bridgeToken": "..."},
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
The original `ChannelConnection.kind` remains `weixin` or `feishu`; only the runtime transport kind is generic.
|
||||||
|
|
||||||
|
## Dynamic Runtime Activation
|
||||||
|
|
||||||
|
A connected connector session must activate without restarting Beaver.
|
||||||
|
|
||||||
|
Add runtime methods:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def add_channel(self, channel_id: str, config: ChannelConfig) -> None:
|
||||||
|
...
|
||||||
|
|
||||||
|
async def remove_channel(self, channel_id: str) -> None:
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
When a connector session reaches `connected`:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Connector session connected
|
||||||
|
-> connector updates ChannelConnection
|
||||||
|
-> registry materializes ChannelConfig
|
||||||
|
-> ChannelRuntime.add_channel(channel_id, config)
|
||||||
|
-> ChannelManager.register(adapter)
|
||||||
|
-> adapter.start()
|
||||||
|
-> channel status becomes running
|
||||||
|
```
|
||||||
|
|
||||||
|
This is a hard requirement for Weixin and Feishu/Lark onboarding. Manual backend restart is not an acceptable success path for this feature.
|
||||||
|
|
||||||
|
`remove_channel()` is used when a user logs out or revokes a sidecar connection:
|
||||||
|
|
||||||
|
```text
|
||||||
|
logout / revoke
|
||||||
|
-> sidecar logout
|
||||||
|
-> ChannelRuntime.remove_channel(channel_id)
|
||||||
|
-> connection status revoked or disconnected
|
||||||
|
```
|
||||||
|
|
||||||
|
## Sidecar Deployment
|
||||||
|
|
||||||
|
Add a sidecar service that can be enabled in deployment:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
external-connector:
|
||||||
|
build: ./external-connector
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
BEAVER_BRIDGE_BASE_URL: http://app-instance:8080
|
||||||
|
BEAVER_BRIDGE_TOKEN: ${BEAVER_BRIDGE_TOKEN}
|
||||||
|
CONNECTOR_HOME: /var/lib/external-connector
|
||||||
|
CONNECTOR_PROVIDER: vendor_cli
|
||||||
|
volumes:
|
||||||
|
- external-connector-state:/var/lib/external-connector
|
||||||
|
```
|
||||||
|
|
||||||
|
For the current `create-instance.sh`-style deployment, the implementation adds:
|
||||||
|
|
||||||
|
- `docker-compose.external-connectors.yml` for local/development sidecar tests.
|
||||||
|
- documentation for attaching `external-connector` to the same Docker network as the target app instance.
|
||||||
|
- instance environment `EXTERNAL_CONNECTOR_BASE_URL=http://external-connector:8787`.
|
||||||
|
|
||||||
|
The implementation must not depend on Beaver mounting `/var/run/docker.sock`.
|
||||||
|
|
||||||
|
## Sidecar HTTP API
|
||||||
|
|
||||||
|
All sidecar requests and responses are JSON. The sidecar listens on port `8787`.
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /health
|
||||||
|
GET /connectors
|
||||||
|
POST /connector-sessions
|
||||||
|
GET /connector-sessions/{session_id}
|
||||||
|
POST /connector-sessions/{session_id}/cancel
|
||||||
|
POST /connections/{connection_id}/logout
|
||||||
|
POST /send
|
||||||
|
```
|
||||||
|
|
||||||
|
`GET /connectors` returns:
|
||||||
|
|
||||||
|
```json
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"kind": "weixin",
|
||||||
|
"displayName": "Weixin",
|
||||||
|
"authType": "qr",
|
||||||
|
"providerId": "vendor_cli",
|
||||||
|
"capabilities": ["receive_text", "send_text", "receive_media", "direct_messages"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"kind": "feishu",
|
||||||
|
"displayName": "Feishu/Lark",
|
||||||
|
"authType": "plugin_install",
|
||||||
|
"providerId": "vendor_cli",
|
||||||
|
"capabilities": ["receive_text", "send_text", "receive_media", "groups"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
`POST /connector-sessions` request:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"kind": "weixin",
|
||||||
|
"connectionId": "conn_...",
|
||||||
|
"channelId": "weixin-main",
|
||||||
|
"displayName": "Weixin Main",
|
||||||
|
"callbackBaseUrl": "http://app-instance:8080",
|
||||||
|
"bridgeToken": "...",
|
||||||
|
"options": {}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
For Feishu/Lark, `kind` is `feishu` and `options` may include `domain`, `mode`, and optional app credentials when linking an existing bot. If using the official plugin installer to create a bot, the sidecar starts that installer flow and reports QR, instruction, or action status back to Beaver.
|
||||||
|
|
||||||
|
`GET /connector-sessions/{session_id}` response:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"sessionId": "cs_...",
|
||||||
|
"kind": "weixin",
|
||||||
|
"status": "qr_ready",
|
||||||
|
"qrCode": "weixin://...",
|
||||||
|
"qrImage": "data:image/png;base64,...",
|
||||||
|
"instructions": [],
|
||||||
|
"accountId": null,
|
||||||
|
"displayName": null,
|
||||||
|
"error": null,
|
||||||
|
"metadata": {}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Allowed connector session statuses:
|
||||||
|
|
||||||
|
- `pending`
|
||||||
|
- `qr_ready`
|
||||||
|
- `scanned`
|
||||||
|
- `confirmed`
|
||||||
|
- `installing`
|
||||||
|
- `waiting_for_user`
|
||||||
|
- `connected`
|
||||||
|
- `expired`
|
||||||
|
- `error`
|
||||||
|
- `cancelled`
|
||||||
|
|
||||||
|
`POST /send` request:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"connectionId": "conn_...",
|
||||||
|
"channelId": "weixin-main",
|
||||||
|
"kind": "weixin",
|
||||||
|
"target": {
|
||||||
|
"peerId": "wx_user_or_chat_id",
|
||||||
|
"peerType": "dm",
|
||||||
|
"threadId": null
|
||||||
|
},
|
||||||
|
"content": "reply text",
|
||||||
|
"metadata": {
|
||||||
|
"contextToken": "optional"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Beaver Bridge API
|
||||||
|
|
||||||
|
Add a backend bridge endpoint for sidecar inbound messages:
|
||||||
|
|
||||||
|
```text
|
||||||
|
POST /api/channel-connector-bridge/events
|
||||||
|
```
|
||||||
|
|
||||||
|
The sidecar must authenticate every bridge request using a bearer token scoped to the connector service. Beaver rejects missing or invalid bridge tokens.
|
||||||
|
|
||||||
|
Bridge event body:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"eventId": "provider-event-id",
|
||||||
|
"timestamp": "2026-06-02T09:30:00Z",
|
||||||
|
"deliveryAttempt": 1,
|
||||||
|
"connectionId": "conn_...",
|
||||||
|
"channelId": "weixin-main",
|
||||||
|
"kind": "weixin",
|
||||||
|
"accountId": "weixin:...",
|
||||||
|
"peerId": "wx_user_or_chat_id",
|
||||||
|
"peerType": "dm",
|
||||||
|
"userId": "wx_sender",
|
||||||
|
"threadId": null,
|
||||||
|
"messageId": "platform-message-id",
|
||||||
|
"messageType": "text",
|
||||||
|
"content": "hello",
|
||||||
|
"metadata": {
|
||||||
|
"contextToken": "optional"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The bridge endpoint must:
|
||||||
|
|
||||||
|
1. validate bearer token
|
||||||
|
2. load `ChannelConnection`
|
||||||
|
3. reject unknown or revoked connections
|
||||||
|
4. dedupe by `connectionId + eventId` through `MessageDedupeStore`
|
||||||
|
5. construct `ChannelIdentity`
|
||||||
|
6. construct `InboundMessage`
|
||||||
|
7. call `ChannelRuntime.accept_inbound()`
|
||||||
|
8. mark bridge event completed or failed
|
||||||
|
|
||||||
|
## MessageDedupeStore
|
||||||
|
|
||||||
|
Add a JSON-backed `MessageDedupeStore` under:
|
||||||
|
|
||||||
|
```text
|
||||||
|
<workspace>/state/channel_connections/message_dedupe.json
|
||||||
|
```
|
||||||
|
|
||||||
|
It stores:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@dataclass
|
||||||
|
class ConnectorMessageDedupeRecord:
|
||||||
|
dedupe_key: str
|
||||||
|
connection_id: str
|
||||||
|
event_id: str
|
||||||
|
status: str
|
||||||
|
first_seen_at: str
|
||||||
|
updated_at: str
|
||||||
|
delivery_attempts: int
|
||||||
|
message_id: str | None
|
||||||
|
last_error: str | None
|
||||||
|
```
|
||||||
|
|
||||||
|
`status` values:
|
||||||
|
|
||||||
|
- `processing`
|
||||||
|
- `completed`
|
||||||
|
- `failed`
|
||||||
|
|
||||||
|
If a duplicate bridge event arrives while the record is `processing` or `completed`, Beaver returns an idempotent success response and does not call `ChannelRuntime.accept_inbound()` again.
|
||||||
|
|
||||||
|
This store is separate from runtime session dedupe. Runtime dedupe still protects platform message identity, while bridge dedupe protects connector retries.
|
||||||
|
|
||||||
|
## Beaver Connectors
|
||||||
|
|
||||||
|
### WeixinConnector
|
||||||
|
|
||||||
|
Responsibilities:
|
||||||
|
|
||||||
|
- discover sidecar health
|
||||||
|
- start Weixin connector session through sidecar `/connector-sessions`
|
||||||
|
- poll sidecar connector session status
|
||||||
|
- create or update `ChannelConnection`
|
||||||
|
- store bridge token and sidecar connection state reference in `CredentialStore`
|
||||||
|
- validate by checking sidecar connection status
|
||||||
|
- materialize runtime config for `ExternalConnectorChannel`
|
||||||
|
- activate runtime via `ChannelRuntime.add_channel()` when connected
|
||||||
|
- revoke/logout by calling sidecar `/connections/{connection_id}/logout`
|
||||||
|
- deactivate runtime via `ChannelRuntime.remove_channel()` on logout/revoke
|
||||||
|
|
||||||
|
### FeishuConnector
|
||||||
|
|
||||||
|
Responsibilities:
|
||||||
|
|
||||||
|
- discover sidecar health
|
||||||
|
- start Feishu/Lark plugin install/link connector session
|
||||||
|
- optionally pass appId/appSecret/domain/mode for existing bot linking
|
||||||
|
- poll installer/session status
|
||||||
|
- create or update `ChannelConnection`
|
||||||
|
- validate by sidecar session or connection status
|
||||||
|
- materialize runtime config for `ExternalConnectorChannel`
|
||||||
|
- activate runtime via `ChannelRuntime.add_channel()` when connected
|
||||||
|
- revoke/remove plugin connection by calling sidecar logout/remove API
|
||||||
|
- deactivate runtime via `ChannelRuntime.remove_channel()` on logout/revoke
|
||||||
|
|
||||||
|
Feishu is sidecar-backed in this design because the user's supplied Feishu article describes the official plugin flow, not only a static bot-credential runtime adapter.
|
||||||
|
|
||||||
|
## Frontend
|
||||||
|
|
||||||
|
Replace the old static Weixin and Feishu fields with connector-driven UI:
|
||||||
|
|
||||||
|
- fetch `GET /api/channel-connectors`
|
||||||
|
- show Telegram, Weixin, and Feishu/Lark as connector options
|
||||||
|
- for Weixin:
|
||||||
|
- start connector session
|
||||||
|
- show QR image
|
||||||
|
- poll status until connected/expired/error
|
||||||
|
- show connected account and logout
|
||||||
|
- for Feishu/Lark:
|
||||||
|
- choose create bot or link existing bot
|
||||||
|
- collect domain and optional app credentials
|
||||||
|
- start sidecar connector session
|
||||||
|
- show QR/instructions/status returned by sidecar
|
||||||
|
- show connected account and logout
|
||||||
|
|
||||||
|
The old `/api/channels` static config editor may remain for advanced runtime config, but connector onboarding should not rely on manual JSON editing or direct token entry for Weixin/Feishu.
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
- Sidecar unavailable: show connector as `unavailable`; do not create a running connection.
|
||||||
|
- Provider install command fails: status `error`, with redacted stderr summary.
|
||||||
|
- QR expired: status `expired`, user can start a new connector session.
|
||||||
|
- Bridge token invalid: reject with `401`, record event without platform secret values.
|
||||||
|
- Unknown connection id in bridge event: reject with `404`.
|
||||||
|
- Duplicate bridge event: return idempotent success and do not call runtime again.
|
||||||
|
- Outbound send failure: mark outbound delivery failed and record connector error.
|
||||||
|
- Sidecar restart: persisted provider state should survive through sidecar volume.
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
- Beaver never logs raw tokens, app secrets, bridge tokens, or sidecar connection tokens.
|
||||||
|
- Bridge token is generated by Beaver and stored behind `credentials_ref`.
|
||||||
|
- Sidecar can only call bridge endpoints with its bridge token.
|
||||||
|
- Sidecar state volume contains login state and must be treated as sensitive.
|
||||||
|
- Feishu user-identity mode has stronger privacy risk than bot-identity mode; UI must label it clearly if exposed.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
Backend unit tests:
|
||||||
|
|
||||||
|
- sidecar client fake for Weixin connector session start/status/logout/send
|
||||||
|
- sidecar client fake for Feishu connector session start/status/logout/send
|
||||||
|
- `ExternalConnectorChannel.send()` target mapping
|
||||||
|
- `ChannelRuntime.add_channel()` dynamically starts and registers a channel
|
||||||
|
- `ChannelRuntime.remove_channel()` stops and unregisters a channel
|
||||||
|
- bridge endpoint accepts valid events
|
||||||
|
- bridge endpoint rejects invalid token and unknown connection id
|
||||||
|
- bridge endpoint dedupes repeated `eventId` and calls runtime once
|
||||||
|
- registry lists `telegram`, `weixin`, and `feishu`
|
||||||
|
- materialized sidecar connections produce `ChannelConfig(kind="external_connector", mode="http")` compatible with runtime factory
|
||||||
|
|
||||||
|
Sidecar tests:
|
||||||
|
|
||||||
|
- HTTP API shape for health/connectors/connector-sessions/send
|
||||||
|
- fake provider status transitions
|
||||||
|
- provider command runner error redaction
|
||||||
|
|
||||||
|
Frontend tests:
|
||||||
|
|
||||||
|
- Weixin connector option opens QR modal
|
||||||
|
- polling reaches connected state
|
||||||
|
- expired/error states are visible
|
||||||
|
- Feishu flow starts install/link and shows returned instructions/status
|
||||||
|
|
||||||
|
Manual verification:
|
||||||
|
|
||||||
|
- Build app and sidecar Docker images.
|
||||||
|
- Start docker-compose sidecar setup.
|
||||||
|
- In `terminaltest`, open Weixin connector, scan QR, observe connected status without restarting Beaver.
|
||||||
|
- Send a Weixin text message and verify Beaver receives it once.
|
||||||
|
- Force sidecar retry of the same event and verify Beaver does not produce a duplicate agent reply.
|
||||||
|
- Send a Beaver reply and verify sidecar `/send` path.
|
||||||
|
- Start Feishu connector flow using the official Feishu/Lark plugin install path and verify the vendor-provided start command.
|
||||||
|
|
||||||
|
## Rollout
|
||||||
|
|
||||||
|
Implement in this order:
|
||||||
|
|
||||||
|
1. Sidecar HTTP contract with fake provider.
|
||||||
|
2. `MessageDedupeStore`.
|
||||||
|
3. Beaver `ExternalConnectorChannel` and bridge endpoint.
|
||||||
|
4. `ChannelRuntime.add_channel()` and `ChannelRuntime.remove_channel()`.
|
||||||
|
5. Weixin connector against fake sidecar client.
|
||||||
|
6. Feishu connector against fake sidecar client.
|
||||||
|
7. Frontend connector UI.
|
||||||
|
8. Production `VendorCliProvider` that shells out to real vendor CLI/plugin commands.
|
||||||
|
9. Docker build/compose integration.
|
||||||
|
10. Manual live verification.
|
||||||
|
|
||||||
|
The fake provider is test-only. The production provider must use the real vendor CLI/plugin commands for Weixin and Feishu/Lark; the fake provider only makes Beaver and frontend tests deterministic while the live provider handles non-deterministic external login and install flows.
|
||||||
@ -1,396 +0,0 @@
|
|||||||
# OpenClaw Sidecar Connectors Design
|
|
||||||
|
|
||||||
Date: 2026-06-02
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Add real Weixin personal-account QR login and Feishu/Lark OpenClaw plugin onboarding to Beaver through a docker-compose predeclared sidecar service. Beaver must not dynamically create containers or require Docker socket access.
|
|
||||||
|
|
||||||
This design implements the next connector layer after `docs/superpowers/plans/2026-06-02-channel-connectors-foundation.md`.
|
|
||||||
|
|
||||||
## Sources
|
|
||||||
|
|
||||||
- Tencent `openclaw-weixin` supports Weixin QR login and persists login credentials after scan confirmation: https://github.com/Tencent/openclaw-weixin
|
|
||||||
- The Weixin ClawBot install flow shown by the user uses `npx -y @tencent-weixin/openclaw-weixin-cli@latest install`.
|
|
||||||
- Feishu's OpenClaw article documents the official Lark/Feishu plugin install command `npx -y @larksuite/openclaw-lark install`, bot creation/linking, `/feishu start` verification, and user-vs-bot identity modes: https://www.feishu.cn/content/article/7613711414611463386
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
Included:
|
|
||||||
|
|
||||||
- A repo-local `openclaw-connector` sidecar service.
|
|
||||||
- A docker-compose service declaration for the sidecar.
|
|
||||||
- Sidecar HTTP API for health, connector metadata, pairing/install status, logout/remove, outbound send, and inbound event forwarding.
|
|
||||||
- Beaver `WeixinConnector` and `FeishuConnector` objects registered in `ChannelConnectorRegistry`.
|
|
||||||
- Beaver connector bridge endpoints that accept normalized sidecar inbound events and submit them to `ChannelRuntime.accept_inbound()`.
|
|
||||||
- `ExternalConnectorChannel` runtime object for sidecar-backed outbound sends.
|
|
||||||
- Web UI connection wizard for Weixin QR login and Feishu/Lark plugin onboarding.
|
|
||||||
- Unit tests using fake sidecar clients and bridge events.
|
|
||||||
|
|
||||||
Excluded:
|
|
||||||
|
|
||||||
- Dynamic Docker container creation from Beaver.
|
|
||||||
- Docker socket mounts in Beaver.
|
|
||||||
- Reimplementing Weixin iLink or Feishu OpenClaw protocol inside Beaver.
|
|
||||||
- Building a generic plugin marketplace.
|
|
||||||
- Hot-swapping running adapters without backend restart. This phase may create/update connections and require a Beaver app restart for runtime materialization unless explicitly handled by the existing runtime.
|
|
||||||
- Multi-user enterprise permission governance beyond local connector ownership and bridge token validation.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
Use one predeclared sidecar for OpenClaw-backed platform connectors:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Beaver backend
|
|
||||||
-> OpenClawConnector HTTP client
|
|
||||||
-> openclaw-connector sidecar
|
|
||||||
-> OpenClaw CLI/runtime
|
|
||||||
-> @tencent-weixin/openclaw-weixin
|
|
||||||
-> @larksuite/openclaw-lark
|
|
||||||
```
|
|
||||||
|
|
||||||
Beaver owns:
|
|
||||||
|
|
||||||
- connection state in `ChannelConnectionStore`
|
|
||||||
- credential references in `CredentialStore`
|
|
||||||
- bridge endpoint authentication
|
|
||||||
- normalized runtime message admission
|
|
||||||
- runtime dedupe/session identity
|
|
||||||
- outbound dispatch into sidecar `/send`
|
|
||||||
|
|
||||||
The sidecar owns:
|
|
||||||
|
|
||||||
- OpenClaw installation/runtime state
|
|
||||||
- plugin install/update commands
|
|
||||||
- Weixin QR login and login-state persistence
|
|
||||||
- Feishu/Lark plugin install, bot creation/linking, and OpenClaw-side verification
|
|
||||||
- platform receive loops
|
|
||||||
- sidecar-to-Beaver inbound event delivery
|
|
||||||
|
|
||||||
## Runtime Flow
|
|
||||||
|
|
||||||
Inbound:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Weixin or Feishu/Lark platform event
|
|
||||||
-> OpenClaw plugin inside sidecar
|
|
||||||
-> sidecar normalized event
|
|
||||||
-> POST Beaver /api/channel-connector-bridge/events
|
|
||||||
-> ChannelRuntime.accept_inbound()
|
|
||||||
-> MessageBus
|
|
||||||
-> AgentService
|
|
||||||
```
|
|
||||||
|
|
||||||
Outbound:
|
|
||||||
|
|
||||||
```text
|
|
||||||
AgentService
|
|
||||||
-> MessageBus outbound
|
|
||||||
-> ChannelManager.dispatch_outbound()
|
|
||||||
-> ExternalConnectorChannel.send()
|
|
||||||
-> POST sidecar /send
|
|
||||||
-> OpenClaw plugin send path
|
|
||||||
-> Weixin or Feishu/Lark platform
|
|
||||||
```
|
|
||||||
|
|
||||||
`ExternalConnectorChannel` implements the existing runtime channel protocol:
|
|
||||||
|
|
||||||
```python
|
|
||||||
channel_id: str
|
|
||||||
kind: str
|
|
||||||
mode: str
|
|
||||||
|
|
||||||
async def start() -> None
|
|
||||||
async def stop() -> None
|
|
||||||
async def send(message: OutboundMessage) -> None
|
|
||||||
```
|
|
||||||
|
|
||||||
It is not a platform protocol adapter. It is a generic HTTP bridge to a sidecar.
|
|
||||||
|
|
||||||
Runtime materialization for sidecar-backed connections always emits:
|
|
||||||
|
|
||||||
```python
|
|
||||||
ChannelConfig(
|
|
||||||
enabled=True,
|
|
||||||
kind="external_connector",
|
|
||||||
mode="http",
|
|
||||||
account_id=spec.account_id,
|
|
||||||
display_name=spec.display_name,
|
|
||||||
config={
|
|
||||||
"platformKind": "weixin",
|
|
||||||
"connectionId": "conn_...",
|
|
||||||
"sidecarBaseUrl": "http://openclaw-connector:8787",
|
|
||||||
},
|
|
||||||
secrets={"bridgeToken": "..."},
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
The original `ChannelConnection.kind` remains `weixin` or `feishu`; only the runtime transport kind is generic.
|
|
||||||
|
|
||||||
## Sidecar Deployment
|
|
||||||
|
|
||||||
Add a sidecar service that can be enabled in deployment:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
openclaw-connector:
|
|
||||||
build: ./openclaw-connector
|
|
||||||
restart: unless-stopped
|
|
||||||
environment:
|
|
||||||
BEAVER_BRIDGE_BASE_URL: http://app-instance:8080
|
|
||||||
BEAVER_BRIDGE_TOKEN: ${BEAVER_BRIDGE_TOKEN}
|
|
||||||
OPENCLAW_HOME: /var/lib/openclaw
|
|
||||||
volumes:
|
|
||||||
- openclaw-connector-state:/var/lib/openclaw
|
|
||||||
```
|
|
||||||
|
|
||||||
For the current `create-instance.sh`-style deployment, the implementation adds:
|
|
||||||
|
|
||||||
- `docker-compose.openclaw.yml` for local/development sidecar tests.
|
|
||||||
- documentation for attaching `openclaw-connector` to the same Docker network as the target app instance.
|
|
||||||
- instance environment `OPENCLAW_CONNECTOR_BASE_URL=http://openclaw-connector:8787`.
|
|
||||||
|
|
||||||
The implementation must not depend on Beaver mounting `/var/run/docker.sock`.
|
|
||||||
|
|
||||||
## Sidecar HTTP API
|
|
||||||
|
|
||||||
All sidecar requests and responses are JSON. The sidecar listens on port `8787`.
|
|
||||||
|
|
||||||
```text
|
|
||||||
GET /health
|
|
||||||
GET /connectors
|
|
||||||
POST /pairings
|
|
||||||
GET /pairings/{pairing_id}
|
|
||||||
POST /pairings/{pairing_id}/cancel
|
|
||||||
POST /connections/{connection_id}/logout
|
|
||||||
POST /send
|
|
||||||
```
|
|
||||||
|
|
||||||
`GET /connectors` returns:
|
|
||||||
|
|
||||||
```json
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"kind": "weixin",
|
|
||||||
"displayName": "Weixin",
|
|
||||||
"authType": "qr",
|
|
||||||
"capabilities": ["receive_text", "send_text", "receive_media", "direct_messages"]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"kind": "feishu",
|
|
||||||
"displayName": "Feishu/Lark",
|
|
||||||
"authType": "openclaw_plugin",
|
|
||||||
"capabilities": ["receive_text", "send_text", "receive_media", "groups"]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
`POST /pairings` request:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"kind": "weixin",
|
|
||||||
"connectionId": "conn_...",
|
|
||||||
"channelId": "weixin-main",
|
|
||||||
"displayName": "Weixin Main",
|
|
||||||
"callbackBaseUrl": "http://app-instance:8080",
|
|
||||||
"bridgeToken": "..."
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
For Feishu/Lark, `kind` is `feishu` and the request may include `domain`, `mode`, and optional app credentials when linking an existing bot. If using the OpenClaw official installer to create a bot, the sidecar starts that installer flow and reports QR or action status back to Beaver.
|
|
||||||
|
|
||||||
`GET /pairings/{pairing_id}` response:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"pairingId": "pair_...",
|
|
||||||
"kind": "weixin",
|
|
||||||
"status": "pending",
|
|
||||||
"qrCode": "weixin://...",
|
|
||||||
"qrImage": "data:image/png;base64,...",
|
|
||||||
"accountId": null,
|
|
||||||
"displayName": null,
|
|
||||||
"error": null,
|
|
||||||
"metadata": {}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Allowed pairing statuses:
|
|
||||||
|
|
||||||
- `pending`
|
|
||||||
- `qr_ready`
|
|
||||||
- `scanned`
|
|
||||||
- `confirmed`
|
|
||||||
- `connected`
|
|
||||||
- `expired`
|
|
||||||
- `error`
|
|
||||||
- `cancelled`
|
|
||||||
|
|
||||||
`POST /send` request:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"connectionId": "conn_...",
|
|
||||||
"channelId": "weixin-main",
|
|
||||||
"kind": "weixin",
|
|
||||||
"target": {
|
|
||||||
"peerId": "wx_user_or_chat_id",
|
|
||||||
"peerType": "dm",
|
|
||||||
"threadId": null
|
|
||||||
},
|
|
||||||
"content": "reply text",
|
|
||||||
"metadata": {
|
|
||||||
"contextToken": "optional"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Beaver Bridge API
|
|
||||||
|
|
||||||
Add a backend bridge endpoint for sidecar inbound messages:
|
|
||||||
|
|
||||||
```text
|
|
||||||
POST /api/channel-connector-bridge/events
|
|
||||||
```
|
|
||||||
|
|
||||||
The sidecar must authenticate every bridge request using a bearer token scoped to the connector service. Beaver rejects missing or invalid bridge tokens.
|
|
||||||
|
|
||||||
Bridge event body:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"connectionId": "conn_...",
|
|
||||||
"channelId": "weixin-main",
|
|
||||||
"kind": "weixin",
|
|
||||||
"accountId": "weixin:...",
|
|
||||||
"peerId": "wx_user_or_chat_id",
|
|
||||||
"peerType": "dm",
|
|
||||||
"userId": "wx_sender",
|
|
||||||
"threadId": null,
|
|
||||||
"messageId": "platform-message-id",
|
|
||||||
"messageType": "text",
|
|
||||||
"content": "hello",
|
|
||||||
"metadata": {
|
|
||||||
"contextToken": "optional"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The bridge endpoint constructs `ChannelIdentity`, then `InboundMessage`, then calls `ChannelRuntime.accept_inbound()`.
|
|
||||||
|
|
||||||
## Beaver Connectors
|
|
||||||
|
|
||||||
### WeixinConnector
|
|
||||||
|
|
||||||
Responsibilities:
|
|
||||||
|
|
||||||
- discover sidecar health
|
|
||||||
- start Weixin pairing through sidecar `/pairings`
|
|
||||||
- poll sidecar pairing status
|
|
||||||
- create or update `ChannelConnection`
|
|
||||||
- store sidecar connection token or state reference in `CredentialStore`
|
|
||||||
- validate by checking sidecar connection status
|
|
||||||
- materialize runtime config for `ExternalConnectorChannel`
|
|
||||||
- revoke/logout by calling sidecar `/connections/{connection_id}/logout`
|
|
||||||
|
|
||||||
### FeishuConnector
|
|
||||||
|
|
||||||
Responsibilities:
|
|
||||||
|
|
||||||
- discover sidecar health
|
|
||||||
- start Feishu/Lark OpenClaw plugin install/link flow
|
|
||||||
- optionally pass appId/appSecret/domain/mode for existing bot linking
|
|
||||||
- poll installer/pairing status
|
|
||||||
- create or update `ChannelConnection`
|
|
||||||
- validate by sidecar `/pairings/{id}` or connector status
|
|
||||||
- materialize runtime config for `ExternalConnectorChannel`
|
|
||||||
- revoke/remove plugin connection by calling sidecar logout/remove API
|
|
||||||
|
|
||||||
Feishu is sidecar-backed in this design because the user's supplied Feishu article describes the official OpenClaw plugin flow, not only a static bot-credential runtime adapter.
|
|
||||||
|
|
||||||
## Frontend
|
|
||||||
|
|
||||||
Replace the old static Weixin fields with connector-driven UI:
|
|
||||||
|
|
||||||
- fetch `GET /api/channel-connectors`
|
|
||||||
- show Telegram, Weixin, and Feishu/Lark as connector options
|
|
||||||
- for Weixin:
|
|
||||||
- start pairing
|
|
||||||
- show QR image
|
|
||||||
- poll status until connected/expired/error
|
|
||||||
- show connected account and logout
|
|
||||||
- for Feishu/Lark:
|
|
||||||
- choose create bot or link existing bot
|
|
||||||
- collect domain and optional app credentials
|
|
||||||
- start sidecar pairing/install
|
|
||||||
- show QR/instructions/status returned by sidecar
|
|
||||||
- show connected account and logout
|
|
||||||
|
|
||||||
The old `/api/channels` static config editor may remain for advanced runtime config, but connector onboarding should not rely on manual JSON editing or direct token entry for Weixin/Feishu.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
|
|
||||||
- Sidecar unavailable: show connector as `unavailable`; do not create a running connection.
|
|
||||||
- OpenClaw install command fails: status `error`, with redacted stderr summary.
|
|
||||||
- QR expired: status `expired`, user can start a new pairing.
|
|
||||||
- Bridge token invalid: reject with `401`, record event without platform secret values.
|
|
||||||
- Unknown connection id in bridge event: reject with `404`.
|
|
||||||
- Outbound send failure: mark outbound delivery failed and record connector error.
|
|
||||||
- Sidecar restart: persisted OpenClaw state should survive through sidecar volume.
|
|
||||||
|
|
||||||
## Security
|
|
||||||
|
|
||||||
- Beaver never logs raw tokens, app secrets, or sidecar connection tokens.
|
|
||||||
- Bridge token is generated by Beaver and stored behind `credentials_ref`.
|
|
||||||
- Sidecar can only call bridge endpoints with its bridge token.
|
|
||||||
- Sidecar state volume contains OpenClaw login state and must be treated as sensitive.
|
|
||||||
- Feishu user-identity mode has stronger privacy risk than bot-identity mode; UI must label it clearly if exposed.
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
Backend unit tests:
|
|
||||||
|
|
||||||
- sidecar client fake for Weixin pairing start/status/logout/send
|
|
||||||
- sidecar client fake for Feishu pairing start/status/logout/send
|
|
||||||
- `ExternalConnectorChannel.send()` target mapping
|
|
||||||
- bridge endpoint accepts valid events and rejects invalid token/connection id
|
|
||||||
- registry lists `telegram`, `weixin`, and `feishu`
|
|
||||||
- materialized sidecar connections produce `ChannelConfig(kind="external_connector", mode="http")` compatible with runtime factory
|
|
||||||
|
|
||||||
Sidecar tests:
|
|
||||||
|
|
||||||
- HTTP API shape for health/connectors/pairings/send
|
|
||||||
- fake OpenClaw provider status transitions
|
|
||||||
- command runner error redaction
|
|
||||||
|
|
||||||
Frontend tests:
|
|
||||||
|
|
||||||
- Weixin connector option opens QR modal
|
|
||||||
- polling reaches connected state
|
|
||||||
- expired/error states are visible
|
|
||||||
- Feishu flow starts install/link and shows returned instructions/status
|
|
||||||
|
|
||||||
Manual verification:
|
|
||||||
|
|
||||||
- Build app and sidecar Docker images.
|
|
||||||
- Start docker-compose sidecar setup.
|
|
||||||
- In `terminaltest`, open Weixin connector, scan QR, observe connected status.
|
|
||||||
- Send a Weixin text message and verify Beaver receives it.
|
|
||||||
- Send a Beaver reply and verify sidecar `/send` path.
|
|
||||||
- Start Feishu connector flow using official OpenClaw Lark plugin install path and verify `/feishu start`.
|
|
||||||
|
|
||||||
## Rollout
|
|
||||||
|
|
||||||
Implement in this order:
|
|
||||||
|
|
||||||
1. Sidecar HTTP contract with fake provider.
|
|
||||||
2. Beaver `ExternalConnectorChannel` and bridge endpoint.
|
|
||||||
3. Weixin connector against fake sidecar client.
|
|
||||||
4. Feishu connector against fake sidecar client.
|
|
||||||
5. Frontend connector UI.
|
|
||||||
6. Real sidecar provider that shells out to OpenClaw/OpenClaw plugin commands.
|
|
||||||
7. Docker build/compose integration.
|
|
||||||
8. Manual live verification.
|
|
||||||
|
|
||||||
The fake provider is test-only. The production sidecar provider must use real OpenClaw plugin commands for Weixin and Feishu/Lark; the fake provider only makes Beaver and frontend tests deterministic while the live provider handles the non-deterministic external login flow.
|
|
||||||
Reference in New Issue
Block a user