1646 lines
60 KiB
Markdown
1646 lines
60 KiB
Markdown
# External Connector Backend Runtime Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Add Beaver backend support for sidecar-backed Weixin and Feishu connector sessions, bridge-event dedupe, outbound sidecar delivery, and dynamic runtime channel activation.
|
|
|
|
**Architecture:** Beaver depends only on a generic connector HTTP contract. Sidecar-backed connections materialize as `ChannelConfig(kind="external_connector", mode="http")`, are registered dynamically through `ChannelRuntime.add_channel()`, and accept inbound events through an authenticated bridge endpoint with `MessageDedupeStore` idempotency.
|
|
|
|
**Tech Stack:** Python dataclasses, FastAPI, Pydantic v2, local JSON stores, pytest, existing Beaver channel runtime.
|
|
|
|
---
|
|
|
|
## Scope
|
|
|
|
Included:
|
|
|
|
- JSON-backed bridge event dedupe with 60-second stale processing TTL.
|
|
- `ExternalConnectorChannel` adapter for outbound `/send` calls with stable `requestId`.
|
|
- Runtime and manager dynamic add/remove channel support.
|
|
- Beaver sidecar HTTP client.
|
|
- Weixin and Feishu connector registry entries backed by fake sidecar clients in tests.
|
|
- Backend APIs for connector sessions and bridge inbound events.
|
|
|
|
Excluded:
|
|
|
|
- Sidecar process implementation.
|
|
- Frontend connector wizard.
|
|
- Docker compose integration.
|
|
- Live vendor CLI verification.
|
|
|
|
## File Structure
|
|
|
|
- Create `app-instance/backend/beaver/interfaces/channels/connections/dedupe.py`
|
|
- `ConnectorMessageDedupeRecord` and `MessageDedupeStore`.
|
|
- Create `app-instance/backend/beaver/interfaces/channels/connections/sidecar_client.py`
|
|
- Async HTTP client for generic sidecar contract.
|
|
- Create `app-instance/backend/beaver/interfaces/channels/connections/external.py`
|
|
- `ExternalConnectorBase`, `WeixinConnector`, and `FeishuConnector`.
|
|
- Create `app-instance/backend/beaver/interfaces/channels/external_connector.py`
|
|
- Runtime adapter that sends outbound messages to the sidecar.
|
|
- Modify `app-instance/backend/beaver/interfaces/channels/manager.py`
|
|
- Allow dynamic register/unregister while manager is started.
|
|
- Modify `app-instance/backend/beaver/interfaces/channels/runtime.py`
|
|
- Add lifecycle lock, `add_channel()`, `remove_channel()`, and adapter factory support for `external_connector/http`.
|
|
- Modify `app-instance/backend/beaver/interfaces/channels/connections/connectors.py`
|
|
- Materialize sidecar-backed connections and optionally activate runtime.
|
|
- Modify `app-instance/backend/beaver/interfaces/channels/connections/__init__.py`
|
|
- Export new dedupe, client, and connector symbols.
|
|
- Modify `app-instance/backend/beaver/interfaces/web/schemas/chat.py`
|
|
- Add connector session and bridge schemas.
|
|
- Modify `app-instance/backend/beaver/interfaces/web/schemas/__init__.py`
|
|
- Export new schemas.
|
|
- Modify `app-instance/backend/beaver/interfaces/web/app.py`
|
|
- Register Weixin/Feishu connectors, sidecar settings, connector session routes, and bridge endpoint.
|
|
- Test `app-instance/backend/tests/unit/test_connector_message_dedupe_store.py`
|
|
- Test `app-instance/backend/tests/unit/test_external_connector_channel.py`
|
|
- Test `app-instance/backend/tests/unit/test_channel_runtime_dynamic_channels.py`
|
|
- Test `app-instance/backend/tests/unit/test_external_connector_bridge_api.py`
|
|
- Test `app-instance/backend/tests/unit/test_external_sidecar_connectors.py`
|
|
|
|
---
|
|
|
|
### Task 1: Message Dedupe Store
|
|
|
|
**Files:**
|
|
- Create: `app-instance/backend/beaver/interfaces/channels/connections/dedupe.py`
|
|
- Modify: `app-instance/backend/beaver/interfaces/channels/connections/__init__.py`
|
|
- Test: `app-instance/backend/tests/unit/test_connector_message_dedupe_store.py`
|
|
|
|
- [ ] **Step 1: Write failing dedupe tests**
|
|
|
|
Create `app-instance/backend/tests/unit/test_connector_message_dedupe_store.py`:
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from beaver.interfaces.channels.connections import MessageDedupeStore
|
|
|
|
|
|
def test_message_dedupe_store_completes_and_dedupes_completed(tmp_path) -> None:
|
|
store = MessageDedupeStore(tmp_path / "message_dedupe.json")
|
|
|
|
first = store.begin(connection_id="conn_1", event_id="evt_1", delivery_attempt=1)
|
|
store.complete(first.dedupe_key, message_id="msg_1")
|
|
duplicate = store.begin(connection_id="conn_1", event_id="evt_1", delivery_attempt=2)
|
|
|
|
assert first.should_process is True
|
|
assert duplicate.should_process is False
|
|
assert duplicate.status == "completed"
|
|
assert duplicate.http_status == 200
|
|
|
|
|
|
def test_message_dedupe_store_returns_conflict_for_active_processing(tmp_path) -> None:
|
|
store = MessageDedupeStore(tmp_path / "message_dedupe.json", processing_ttl_seconds=60)
|
|
|
|
store.begin(connection_id="conn_1", event_id="evt_1", delivery_attempt=1)
|
|
duplicate = store.begin(connection_id="conn_1", event_id="evt_1", delivery_attempt=2)
|
|
|
|
assert duplicate.should_process is False
|
|
assert duplicate.status == "processing"
|
|
assert duplicate.http_status == 409
|
|
assert duplicate.retry_after_seconds == 5
|
|
|
|
|
|
def test_message_dedupe_store_reprocesses_stale_processing(tmp_path) -> None:
|
|
store = MessageDedupeStore(tmp_path / "message_dedupe.json", processing_ttl_seconds=0)
|
|
|
|
store.begin(connection_id="conn_1", event_id="evt_1", delivery_attempt=1)
|
|
stale = store.begin(connection_id="conn_1", event_id="evt_1", delivery_attempt=2)
|
|
|
|
assert stale.should_process is True
|
|
assert stale.status == "processing"
|
|
assert stale.record.delivery_attempts == 2
|
|
|
|
|
|
def test_message_dedupe_store_reprocesses_failed_records(tmp_path) -> None:
|
|
store = MessageDedupeStore(tmp_path / "message_dedupe.json")
|
|
|
|
first = store.begin(connection_id="conn_1", event_id="evt_1", delivery_attempt=1)
|
|
store.fail(first.dedupe_key, error="runtime rejected")
|
|
retry = store.begin(connection_id="conn_1", event_id="evt_1", delivery_attempt=2)
|
|
|
|
assert retry.should_process is True
|
|
assert retry.record.delivery_attempts == 2
|
|
assert retry.record.last_error is None
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify failure**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_connector_message_dedupe_store.py -q
|
|
```
|
|
|
|
Expected: fail with `ImportError: cannot import name 'MessageDedupeStore'`.
|
|
|
|
- [ ] **Step 3: Implement dedupe models and store**
|
|
|
|
Create `app-instance/backend/beaver/interfaces/channels/connections/dedupe.py` with:
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
from dataclasses import asdict, dataclass
|
|
from datetime import datetime, timezone
|
|
from pathlib import Path
|
|
from threading import Lock
|
|
from typing import Any
|
|
|
|
|
|
def _iso_now() -> str:
|
|
return datetime.now(timezone.utc).isoformat()
|
|
|
|
|
|
def _parse_iso(value: str) -> datetime:
|
|
return datetime.fromisoformat(value.replace("Z", "+00:00"))
|
|
|
|
|
|
@dataclass(slots=True)
|
|
class ConnectorMessageDedupeRecord:
|
|
dedupe_key: str
|
|
connection_id: str
|
|
event_id: str
|
|
status: str
|
|
first_seen_at: str
|
|
updated_at: str
|
|
delivery_attempts: int
|
|
message_id: str | None = None
|
|
last_error: str | None = None
|
|
|
|
def to_dict(self) -> dict[str, Any]:
|
|
return asdict(self)
|
|
|
|
@classmethod
|
|
def from_dict(cls, data: dict[str, Any]) -> "ConnectorMessageDedupeRecord":
|
|
return cls(
|
|
dedupe_key=str(data.get("dedupe_key") or ""),
|
|
connection_id=str(data.get("connection_id") or ""),
|
|
event_id=str(data.get("event_id") or ""),
|
|
status=str(data.get("status") or "processing"),
|
|
first_seen_at=str(data.get("first_seen_at") or _iso_now()),
|
|
updated_at=str(data.get("updated_at") or _iso_now()),
|
|
delivery_attempts=int(data.get("delivery_attempts") or 0),
|
|
message_id=str(data["message_id"]) if data.get("message_id") is not None else None,
|
|
last_error=str(data["last_error"]) if data.get("last_error") is not None else None,
|
|
)
|
|
|
|
|
|
@dataclass(slots=True)
|
|
class DedupeBeginResult:
|
|
should_process: bool
|
|
dedupe_key: str
|
|
status: str
|
|
http_status: int
|
|
retry_after_seconds: int | None
|
|
record: ConnectorMessageDedupeRecord
|
|
|
|
|
|
class MessageDedupeStore:
|
|
def __init__(self, path: Path, *, processing_ttl_seconds: int = 60) -> None:
|
|
self.path = Path(path)
|
|
self.processing_ttl_seconds = int(processing_ttl_seconds)
|
|
self._lock = Lock()
|
|
|
|
def begin(self, *, connection_id: str, event_id: str, delivery_attempt: int) -> DedupeBeginResult:
|
|
dedupe_key = f"{connection_id}:{event_id}"
|
|
now = _iso_now()
|
|
with self._lock:
|
|
data = self._load()
|
|
raw = data["records"].get(dedupe_key)
|
|
if isinstance(raw, dict):
|
|
record = ConnectorMessageDedupeRecord.from_dict(raw)
|
|
if record.status == "completed":
|
|
return DedupeBeginResult(False, dedupe_key, record.status, 200, None, record)
|
|
if record.status == "processing" and not self._is_stale(record, now):
|
|
return DedupeBeginResult(False, dedupe_key, record.status, 409, 5, record)
|
|
record.status = "processing"
|
|
record.updated_at = now
|
|
record.delivery_attempts = max(record.delivery_attempts + 1, int(delivery_attempt))
|
|
record.last_error = None
|
|
else:
|
|
record = ConnectorMessageDedupeRecord(
|
|
dedupe_key=dedupe_key,
|
|
connection_id=connection_id,
|
|
event_id=event_id,
|
|
status="processing",
|
|
first_seen_at=now,
|
|
updated_at=now,
|
|
delivery_attempts=max(1, int(delivery_attempt)),
|
|
)
|
|
data["records"][dedupe_key] = record.to_dict()
|
|
self._save(data)
|
|
return DedupeBeginResult(True, dedupe_key, record.status, 200, None, record)
|
|
|
|
def complete(self, dedupe_key: str, *, message_id: str | None) -> ConnectorMessageDedupeRecord:
|
|
return self._mark(dedupe_key, status="completed", message_id=message_id, error=None)
|
|
|
|
def fail(self, dedupe_key: str, *, error: str) -> ConnectorMessageDedupeRecord:
|
|
return self._mark(dedupe_key, status="failed", message_id=None, error=error)
|
|
|
|
def _mark(
|
|
self,
|
|
dedupe_key: str,
|
|
*,
|
|
status: str,
|
|
message_id: str | None,
|
|
error: str | None,
|
|
) -> ConnectorMessageDedupeRecord:
|
|
with self._lock:
|
|
data = self._load()
|
|
raw = data["records"].get(dedupe_key)
|
|
if not isinstance(raw, dict):
|
|
raise KeyError(dedupe_key)
|
|
record = ConnectorMessageDedupeRecord.from_dict(raw)
|
|
record.status = status
|
|
record.updated_at = _iso_now()
|
|
record.message_id = message_id or record.message_id
|
|
record.last_error = error
|
|
data["records"][dedupe_key] = record.to_dict()
|
|
self._save(data)
|
|
return record
|
|
|
|
def _is_stale(self, record: ConnectorMessageDedupeRecord, now: str) -> bool:
|
|
age = (_parse_iso(now) - _parse_iso(record.updated_at)).total_seconds()
|
|
return age >= self.processing_ttl_seconds
|
|
|
|
def _load(self) -> dict[str, Any]:
|
|
if not self.path.exists():
|
|
return {"records": {}}
|
|
try:
|
|
data = json.loads(self.path.read_text(encoding="utf-8"))
|
|
except (OSError, json.JSONDecodeError):
|
|
return {"records": {}}
|
|
if not isinstance(data, dict) or not isinstance(data.get("records"), dict):
|
|
return {"records": {}}
|
|
return data
|
|
|
|
def _save(self, data: dict[str, Any]) -> None:
|
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
|
tmp_path = self.path.with_name(f"{self.path.name}.tmp")
|
|
tmp_path.write_text(json.dumps(data, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
|
tmp_path.replace(self.path)
|
|
```
|
|
|
|
- [ ] **Step 4: Export dedupe symbols**
|
|
|
|
Modify `app-instance/backend/beaver/interfaces/channels/connections/__init__.py`:
|
|
|
|
```python
|
|
from .dedupe import ConnectorMessageDedupeRecord, DedupeBeginResult, MessageDedupeStore
|
|
```
|
|
|
|
Add these names to `__all__`.
|
|
|
|
- [ ] **Step 5: Run dedupe tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_connector_message_dedupe_store.py -q
|
|
```
|
|
|
|
Expected: `4 passed`.
|
|
|
|
- [ ] **Step 6: Commit Task 1**
|
|
|
|
```bash
|
|
git add app-instance/backend/beaver/interfaces/channels/connections/dedupe.py app-instance/backend/beaver/interfaces/channels/connections/__init__.py app-instance/backend/tests/unit/test_connector_message_dedupe_store.py
|
|
git commit -m "feat: add connector bridge dedupe store"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: Dynamic Runtime Channels
|
|
|
|
**Files:**
|
|
- Modify: `app-instance/backend/beaver/interfaces/channels/manager.py`
|
|
- Modify: `app-instance/backend/beaver/interfaces/channels/runtime.py`
|
|
- Test: `app-instance/backend/tests/unit/test_channel_runtime_dynamic_channels.py`
|
|
|
|
- [ ] **Step 1: Write failing dynamic runtime tests**
|
|
|
|
Create `app-instance/backend/tests/unit/test_channel_runtime_dynamic_channels.py`:
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
|
|
from beaver.foundation.config.schema import ChannelConfig
|
|
from beaver.foundation.events import MessageBus, OutboundMessage
|
|
from beaver.interfaces.channels.runtime import ChannelRuntime
|
|
|
|
|
|
class FakeService:
|
|
async def handle_inbound_message(self, inbound):
|
|
return OutboundMessage(channel=inbound.channel, content="ok", session_id=inbound.session_id, finish_reason="stop")
|
|
|
|
|
|
def test_runtime_add_channel_starts_new_channel_after_runtime_start(tmp_path) -> None:
|
|
async def run() -> None:
|
|
runtime = ChannelRuntime(service=FakeService(), workspace=tmp_path, channels={}, bus=MessageBus())
|
|
await runtime.start()
|
|
try:
|
|
await runtime.add_channel(
|
|
"webhook-dev",
|
|
ChannelConfig(enabled=True, kind="webhook", mode="webhook", account_id="acct"),
|
|
)
|
|
assert "webhook-dev" in runtime.adapters
|
|
assert runtime.states["webhook-dev"]["state"] == "running"
|
|
finally:
|
|
await runtime.stop()
|
|
|
|
asyncio.run(run())
|
|
|
|
|
|
def test_runtime_add_channel_noops_for_same_config(tmp_path) -> None:
|
|
async def run() -> None:
|
|
cfg = ChannelConfig(enabled=True, kind="webhook", mode="webhook", account_id="acct")
|
|
runtime = ChannelRuntime(service=FakeService(), workspace=tmp_path, channels={}, bus=MessageBus())
|
|
await runtime.start()
|
|
try:
|
|
await runtime.add_channel("webhook-dev", cfg)
|
|
first = runtime.adapters["webhook-dev"]
|
|
await runtime.add_channel("webhook-dev", cfg)
|
|
assert runtime.adapters["webhook-dev"] is first
|
|
finally:
|
|
await runtime.stop()
|
|
|
|
asyncio.run(run())
|
|
|
|
|
|
def test_runtime_replacement_failure_keeps_old_channel(tmp_path) -> None:
|
|
async def run() -> None:
|
|
good = ChannelConfig(enabled=True, kind="webhook", mode="webhook", account_id="acct")
|
|
bad = ChannelConfig(enabled=True, kind="missing", mode="http", account_id="acct")
|
|
runtime = ChannelRuntime(service=FakeService(), workspace=tmp_path, channels={}, bus=MessageBus())
|
|
await runtime.start()
|
|
try:
|
|
await runtime.add_channel("webhook-dev", good)
|
|
old = runtime.adapters["webhook-dev"]
|
|
try:
|
|
await runtime.add_channel("webhook-dev", bad)
|
|
except ValueError:
|
|
pass
|
|
else:
|
|
raise AssertionError("Expected ValueError")
|
|
assert runtime.adapters["webhook-dev"] is old
|
|
assert runtime.channel_configs["webhook-dev"] == good
|
|
assert runtime.states["webhook-dev"]["state"] == "running"
|
|
finally:
|
|
await runtime.stop()
|
|
|
|
asyncio.run(run())
|
|
|
|
|
|
def test_runtime_remove_channel_stops_and_unregisters(tmp_path) -> None:
|
|
async def run() -> None:
|
|
runtime = ChannelRuntime(service=FakeService(), workspace=tmp_path, channels={}, bus=MessageBus())
|
|
await runtime.start()
|
|
try:
|
|
await runtime.add_channel(
|
|
"webhook-dev",
|
|
ChannelConfig(enabled=True, kind="webhook", mode="webhook", account_id="acct"),
|
|
)
|
|
await runtime.remove_channel("webhook-dev")
|
|
assert "webhook-dev" not in runtime.adapters
|
|
assert "webhook-dev" not in runtime.manager.channels
|
|
assert runtime.states["webhook-dev"]["state"] == "removed"
|
|
finally:
|
|
await runtime.stop()
|
|
|
|
asyncio.run(run())
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify failure**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_channel_runtime_dynamic_channels.py -q
|
|
```
|
|
|
|
Expected: fail with `AttributeError: 'ChannelRuntime' object has no attribute 'add_channel'`.
|
|
|
|
- [ ] **Step 3: Add dynamic manager methods**
|
|
|
|
Modify `app-instance/backend/beaver/interfaces/channels/manager.py`:
|
|
|
|
```python
|
|
def register(self, channel: ChannelAdapter) -> None:
|
|
if channel.channel_id in self.channels:
|
|
raise ValueError(f"Channel already registered: {channel.channel_id}")
|
|
self.channels[channel.channel_id] = channel
|
|
|
|
def unregister(self, channel_id: str) -> ChannelAdapter | None:
|
|
return self.channels.pop(channel_id, None)
|
|
|
|
def replace_registered(self, channel: ChannelAdapter) -> ChannelAdapter | None:
|
|
old = self.channels.get(channel.channel_id)
|
|
self.channels[channel.channel_id] = channel
|
|
return old
|
|
```
|
|
|
|
Keep `start()`, `stop()`, and `dispatch_outbound()` unchanged.
|
|
|
|
- [ ] **Step 4: Add runtime lifecycle lock and methods**
|
|
|
|
Modify `app-instance/backend/beaver/interfaces/channels/runtime.py`:
|
|
|
|
```python
|
|
self._lifecycle_lock = asyncio.Lock()
|
|
```
|
|
|
|
Add methods to `ChannelRuntime`:
|
|
|
|
```python
|
|
async def add_channel(self, channel_id: str, config: ChannelConfig) -> None:
|
|
async with self._lifecycle_lock:
|
|
current = self.channel_configs.get(channel_id)
|
|
if current == config and channel_id in self.adapters:
|
|
return
|
|
if not config.enabled:
|
|
await self._remove_channel_locked(channel_id)
|
|
self.channel_configs[channel_id] = config
|
|
self.states[channel_id] = {"state": "disabled", "last_error": None}
|
|
return
|
|
adapter = self._build_adapter(channel_id, config)
|
|
await adapter.start()
|
|
old = self.manager.replace_registered(adapter)
|
|
old_adapter = self.adapters.get(channel_id)
|
|
self.adapters[channel_id] = adapter
|
|
self.channel_configs[channel_id] = config
|
|
self.states[channel_id] = {"state": "running", "last_error": None, "started_at": _iso_now()}
|
|
self.events.record(channel_id=channel_id, kind="adapter_started")
|
|
if old_adapter is not None and old_adapter is not adapter:
|
|
await old_adapter.stop()
|
|
|
|
async def remove_channel(self, channel_id: str) -> None:
|
|
async with self._lifecycle_lock:
|
|
await self._remove_channel_locked(channel_id)
|
|
|
|
async def _remove_channel_locked(self, channel_id: str) -> None:
|
|
adapter = self.adapters.pop(channel_id, None)
|
|
self.manager.unregister(channel_id)
|
|
self.channel_configs.pop(channel_id, None)
|
|
if adapter is not None:
|
|
await adapter.stop()
|
|
self.events.record(channel_id=channel_id, kind="adapter_stopped")
|
|
self.states[channel_id] = {"state": "removed", "last_error": None}
|
|
```
|
|
|
|
- [ ] **Step 5: Run dynamic runtime tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_channel_runtime_dynamic_channels.py tests/unit/test_channel_runtime.py tests/unit/test_gateway_channels.py -q
|
|
```
|
|
|
|
Expected: all listed tests pass.
|
|
|
|
- [ ] **Step 6: Commit Task 2**
|
|
|
|
```bash
|
|
git add app-instance/backend/beaver/interfaces/channels/manager.py app-instance/backend/beaver/interfaces/channels/runtime.py app-instance/backend/tests/unit/test_channel_runtime_dynamic_channels.py
|
|
git commit -m "feat: support dynamic runtime channels"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: External Connector Channel
|
|
|
|
**Files:**
|
|
- Create: `app-instance/backend/beaver/interfaces/channels/connections/sidecar_client.py`
|
|
- Create: `app-instance/backend/beaver/interfaces/channels/external_connector.py`
|
|
- Modify: `app-instance/backend/beaver/interfaces/channels/__init__.py`
|
|
- Test: `app-instance/backend/tests/unit/test_external_connector_channel.py`
|
|
|
|
- [ ] **Step 1: Write failing channel tests**
|
|
|
|
Create `app-instance/backend/tests/unit/test_external_connector_channel.py`:
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
|
|
from beaver.foundation.events import ChannelIdentity, OutboundMessage
|
|
from beaver.interfaces.channels.external_connector import ExternalConnectorChannel, _request_id
|
|
|
|
|
|
class FakeSidecarClient:
|
|
def __init__(self) -> None:
|
|
self.sent: list[dict] = []
|
|
|
|
async def send(self, payload: dict) -> dict:
|
|
self.sent.append(payload)
|
|
return {"ok": True, "providerMessageId": "provider-1"}
|
|
|
|
|
|
def test_external_connector_channel_sends_with_target_and_request_id() -> None:
|
|
async def run() -> None:
|
|
client = FakeSidecarClient()
|
|
channel = ExternalConnectorChannel(
|
|
channel_id="weixin-main",
|
|
platform_kind="weixin",
|
|
connection_id="conn_1",
|
|
account_id="weixin:me",
|
|
display_name="Weixin Main",
|
|
sidecar_client=client,
|
|
)
|
|
message = OutboundMessage(
|
|
channel="weixin-main",
|
|
content="reply",
|
|
session_id="s1",
|
|
finish_reason="stop",
|
|
message_id="out-msg-1",
|
|
channel_identity=ChannelIdentity(
|
|
channel_id="weixin-main",
|
|
kind="weixin",
|
|
account_id="weixin:me",
|
|
peer_id="peer-1",
|
|
peer_type="dm",
|
|
thread_id=None,
|
|
user_id="sender-1",
|
|
message_id="in-msg-1",
|
|
),
|
|
)
|
|
|
|
await channel.send(message)
|
|
|
|
assert client.sent == [
|
|
{
|
|
"requestId": "out_weixin-main:s1:out-msg-1",
|
|
"connectionId": "conn_1",
|
|
"channelId": "weixin-main",
|
|
"kind": "weixin",
|
|
"target": {"peerId": "peer-1", "peerType": "dm", "threadId": None},
|
|
"content": "reply",
|
|
"metadata": {"inboundMessageId": "in-msg-1", "sessionId": "s1"},
|
|
}
|
|
]
|
|
|
|
asyncio.run(run())
|
|
|
|
|
|
def test_external_connector_request_id_falls_back_when_message_id_is_none_or_blank() -> None:
|
|
identity = ChannelIdentity(
|
|
channel_id="weixin-main",
|
|
kind="weixin",
|
|
account_id="weixin:me",
|
|
peer_id="peer-1",
|
|
peer_type="dm",
|
|
message_id="in-msg-1",
|
|
)
|
|
first = OutboundMessage(
|
|
channel="weixin-main",
|
|
content="same reply",
|
|
session_id="s1",
|
|
finish_reason="stop",
|
|
message_id=None, # type: ignore[arg-type]
|
|
channel_identity=identity,
|
|
)
|
|
second = OutboundMessage(
|
|
channel="weixin-main",
|
|
content="same reply",
|
|
session_id="s1",
|
|
finish_reason="stop",
|
|
message_id="",
|
|
channel_identity=identity,
|
|
)
|
|
|
|
assert _request_id(first) == _request_id(second)
|
|
assert _request_id(first).startswith("out_weixin-main:s1:")
|
|
|
|
|
|
def test_external_connector_channel_requires_identity() -> None:
|
|
async def run() -> None:
|
|
channel = ExternalConnectorChannel(
|
|
channel_id="weixin-main",
|
|
platform_kind="weixin",
|
|
connection_id="conn_1",
|
|
account_id="weixin:me",
|
|
display_name="Weixin Main",
|
|
sidecar_client=FakeSidecarClient(),
|
|
)
|
|
message = OutboundMessage(channel="weixin-main", content="reply", session_id="s1", finish_reason="stop")
|
|
|
|
try:
|
|
await channel.send(message)
|
|
except ValueError as exc:
|
|
assert "channel_identity is required" in str(exc)
|
|
else:
|
|
raise AssertionError("Expected ValueError")
|
|
|
|
asyncio.run(run())
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify failure**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_external_connector_channel.py -q
|
|
```
|
|
|
|
Expected: fail with `ModuleNotFoundError: No module named 'beaver.interfaces.channels.external_connector'`.
|
|
|
|
- [ ] **Step 3: Implement sidecar client**
|
|
|
|
Create `app-instance/backend/beaver/interfaces/channels/connections/sidecar_client.py`:
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
import hashlib
|
|
from typing import Any
|
|
|
|
import httpx
|
|
|
|
|
|
class ConnectorSidecarClient:
|
|
def __init__(self, *, base_url: str, token: str, timeout_seconds: float = 20.0) -> None:
|
|
self.base_url = base_url.rstrip("/")
|
|
self.token = token
|
|
self.timeout_seconds = float(timeout_seconds)
|
|
|
|
async def get_connectors(self) -> list[dict[str, Any]]:
|
|
return await self._request("GET", "/connectors")
|
|
|
|
async def start_session(self, payload: dict[str, Any]) -> dict[str, Any]:
|
|
return await self._request("POST", "/connector-sessions", json=payload)
|
|
|
|
async def get_session(self, session_id: str) -> dict[str, Any]:
|
|
return await self._request("GET", f"/connector-sessions/{session_id}")
|
|
|
|
async def cancel_session(self, session_id: str) -> dict[str, Any]:
|
|
return await self._request("POST", f"/connector-sessions/{session_id}/cancel", json={})
|
|
|
|
async def logout(self, connection_id: str) -> dict[str, Any]:
|
|
return await self._request("POST", f"/connections/{connection_id}/logout", json={})
|
|
|
|
async def send(self, payload: dict[str, Any]) -> dict[str, Any]:
|
|
return await self._request("POST", "/send", json=payload)
|
|
|
|
async def _request(self, method: str, path: str, *, json: dict[str, Any] | None = None) -> Any:
|
|
headers = {"Authorization": f"Bearer {self.token}"} if self.token else {}
|
|
async with httpx.AsyncClient(timeout=self.timeout_seconds) as client:
|
|
response = await client.request(method, f"{self.base_url}{path}", json=json, headers=headers)
|
|
response.raise_for_status()
|
|
return response.json()
|
|
```
|
|
|
|
- [ ] **Step 4: Implement external channel**
|
|
|
|
Create `app-instance/backend/beaver/interfaces/channels/external_connector.py`:
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from typing import Any
|
|
|
|
from beaver.foundation.events import OutboundMessage
|
|
from beaver.interfaces.channels.connections.sidecar_client import ConnectorSidecarClient
|
|
|
|
|
|
class ExternalConnectorChannel:
|
|
def __init__(
|
|
self,
|
|
*,
|
|
channel_id: str,
|
|
platform_kind: str,
|
|
connection_id: str,
|
|
account_id: str,
|
|
display_name: str,
|
|
sidecar_client: ConnectorSidecarClient | Any,
|
|
) -> None:
|
|
self.channel_id = channel_id
|
|
self.kind = "external_connector"
|
|
self.mode = "http"
|
|
self.platform_kind = platform_kind
|
|
self.connection_id = connection_id
|
|
self.account_id = account_id
|
|
self.display_name = display_name or channel_id
|
|
self.sidecar_client = sidecar_client
|
|
self.started = False
|
|
|
|
async def start(self) -> None:
|
|
self.started = True
|
|
|
|
async def stop(self) -> None:
|
|
self.started = False
|
|
|
|
async def send(self, message: OutboundMessage) -> None:
|
|
identity = message.channel_identity
|
|
if identity is None:
|
|
raise ValueError("channel_identity is required for external connector sends")
|
|
payload = {
|
|
"requestId": _request_id(message),
|
|
"connectionId": self.connection_id,
|
|
"channelId": self.channel_id,
|
|
"kind": self.platform_kind,
|
|
"target": {
|
|
"peerId": identity.peer_id,
|
|
"peerType": identity.peer_type,
|
|
"threadId": identity.thread_id,
|
|
},
|
|
"content": message.content,
|
|
"metadata": {
|
|
"inboundMessageId": identity.message_id,
|
|
"sessionId": message.session_id,
|
|
},
|
|
}
|
|
await self.sidecar_client.send(payload)
|
|
|
|
|
|
def _request_id(message: OutboundMessage) -> str:
|
|
identity = message.channel_identity
|
|
channel = message.channel or (identity.channel_id if identity else "unknown")
|
|
session_id = message.session_id or (identity.session_id() if identity else "unknown")
|
|
message_id = str(message.message_id or "").strip()
|
|
if not message_id:
|
|
basis = "|".join(
|
|
[
|
|
message.content,
|
|
identity.message_id if identity and identity.message_id else "",
|
|
identity.peer_id if identity else "",
|
|
message.finish_reason,
|
|
]
|
|
)
|
|
message_id = hashlib.sha256(basis.encode("utf-8")).hexdigest()[:24]
|
|
return f"out_{channel}:{session_id}:{message_id}"
|
|
```
|
|
|
|
- [ ] **Step 5: Export channel symbol**
|
|
|
|
Modify `app-instance/backend/beaver/interfaces/channels/__init__.py`:
|
|
|
|
```python
|
|
from .external_connector import ExternalConnectorChannel
|
|
```
|
|
|
|
Add `ExternalConnectorChannel` to `__all__`.
|
|
|
|
- [ ] **Step 6: Run channel tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_external_connector_channel.py -q
|
|
```
|
|
|
|
Expected: `2 passed`.
|
|
|
|
- [ ] **Step 7: Commit Task 3**
|
|
|
|
```bash
|
|
git add app-instance/backend/beaver/interfaces/channels/connections/sidecar_client.py app-instance/backend/beaver/interfaces/channels/external_connector.py app-instance/backend/beaver/interfaces/channels/__init__.py app-instance/backend/tests/unit/test_external_connector_channel.py
|
|
git commit -m "feat: add external connector channel"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: Runtime Factory For External Connector Channel
|
|
|
|
**Files:**
|
|
- Modify: `app-instance/backend/beaver/interfaces/channels/runtime.py`
|
|
- Test: `app-instance/backend/tests/unit/test_channel_runtime_dynamic_channels.py`
|
|
|
|
- [ ] **Step 1: Extend dynamic runtime tests**
|
|
|
|
Append to `app-instance/backend/tests/unit/test_channel_runtime_dynamic_channels.py`:
|
|
|
|
```python
|
|
def test_runtime_builds_external_connector_channel(tmp_path, monkeypatch) -> None:
|
|
async def run() -> None:
|
|
monkeypatch.setenv("EXTERNAL_CONNECTOR_TOKEN", "connector-token")
|
|
runtime = ChannelRuntime(service=FakeService(), workspace=tmp_path, channels={}, bus=MessageBus())
|
|
await runtime.start()
|
|
try:
|
|
await runtime.add_channel(
|
|
"weixin-main",
|
|
ChannelConfig(
|
|
enabled=True,
|
|
kind="external_connector",
|
|
mode="http",
|
|
account_id="weixin:me",
|
|
display_name="Weixin Main",
|
|
config={
|
|
"platformKind": "weixin",
|
|
"connectionId": "conn_1",
|
|
"sidecarBaseUrl": "http://external-connector:8787",
|
|
},
|
|
),
|
|
)
|
|
adapter = runtime.adapters["weixin-main"]
|
|
assert adapter.kind == "external_connector"
|
|
assert adapter.mode == "http"
|
|
assert getattr(adapter, "platform_kind") == "weixin"
|
|
finally:
|
|
await runtime.stop()
|
|
|
|
asyncio.run(run())
|
|
```
|
|
|
|
- [ ] **Step 2: Run test to verify failure**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_channel_runtime_dynamic_channels.py::test_runtime_builds_external_connector_channel -q
|
|
```
|
|
|
|
Expected: fail with `ValueError: Unsupported channel kind/mode: external_connector/http`.
|
|
|
|
- [ ] **Step 3: Add runtime factory branch**
|
|
|
|
Modify `_build_adapter()` in `app-instance/backend/beaver/interfaces/channels/runtime.py` before the final `raise`:
|
|
|
|
```python
|
|
if cfg.kind == "external_connector" and cfg.mode == "http":
|
|
import os
|
|
|
|
from beaver.interfaces.channels.connections.sidecar_client import ConnectorSidecarClient
|
|
from beaver.interfaces.channels.external_connector import ExternalConnectorChannel
|
|
|
|
base_url = str(cfg.config.get("sidecarBaseUrl") or os.getenv("EXTERNAL_CONNECTOR_BASE_URL") or "").strip()
|
|
token = os.getenv("EXTERNAL_CONNECTOR_TOKEN", "")
|
|
platform_kind = str(cfg.config.get("platformKind") or "").strip()
|
|
connection_id = str(cfg.config.get("connectionId") or "").strip()
|
|
if not base_url:
|
|
raise ValueError("external connector sidecarBaseUrl is required")
|
|
if not platform_kind:
|
|
raise ValueError("external connector platformKind is required")
|
|
if not connection_id:
|
|
raise ValueError("external connector connectionId is required")
|
|
return ExternalConnectorChannel(
|
|
channel_id=channel_id,
|
|
platform_kind=platform_kind,
|
|
connection_id=connection_id,
|
|
account_id=cfg.account_id,
|
|
display_name=cfg.display_name,
|
|
sidecar_client=ConnectorSidecarClient(base_url=base_url, token=token),
|
|
)
|
|
```
|
|
|
|
- [ ] **Step 4: Run tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_channel_runtime_dynamic_channels.py tests/unit/test_external_connector_channel.py -q
|
|
```
|
|
|
|
Expected: all listed tests pass.
|
|
|
|
- [ ] **Step 5: Commit Task 4**
|
|
|
|
```bash
|
|
git add app-instance/backend/beaver/interfaces/channels/runtime.py app-instance/backend/tests/unit/test_channel_runtime_dynamic_channels.py
|
|
git commit -m "feat: materialize external connector channels"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: Bridge API
|
|
|
|
**Files:**
|
|
- Modify: `app-instance/backend/beaver/interfaces/web/schemas/chat.py`
|
|
- Modify: `app-instance/backend/beaver/interfaces/web/schemas/__init__.py`
|
|
- Modify: `app-instance/backend/beaver/interfaces/web/app.py`
|
|
- Test: `app-instance/backend/tests/unit/test_external_connector_bridge_api.py`
|
|
|
|
- [ ] **Step 1: Write failing bridge API tests**
|
|
|
|
Create `app-instance/backend/tests/unit/test_external_connector_bridge_api.py`:
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from fastapi.testclient import TestClient
|
|
|
|
from beaver.interfaces.channels.connections import ChannelConnectionStore, CredentialStore
|
|
from beaver.interfaces.web.app import create_app
|
|
from beaver.services.agent_service import AgentService
|
|
|
|
|
|
def _app(tmp_path, monkeypatch):
|
|
monkeypatch.setenv("BEAVER_BRIDGE_TOKEN", "bridge-token")
|
|
config_path = tmp_path / "config.json"
|
|
config_path.write_text(
|
|
'{"agents": {"defaults": {"workspace": "%s"}}, "providers": {}}' % str(tmp_path),
|
|
encoding="utf-8",
|
|
)
|
|
service = AgentService(config_path=config_path)
|
|
app = create_app(service=service, manage_service_lifecycle=False)
|
|
return app, service
|
|
|
|
|
|
def test_bridge_endpoint_accepts_valid_event(tmp_path, monkeypatch) -> None:
|
|
app, service = _app(tmp_path, monkeypatch)
|
|
state_dir = tmp_path / "state" / "channel_connections"
|
|
store = ChannelConnectionStore(state_dir / "connections.json")
|
|
connection = store.create(
|
|
kind="weixin",
|
|
mode="sidecar",
|
|
display_name="Weixin Main",
|
|
account_id="weixin:me",
|
|
owner_user_id=None,
|
|
auth_type="connector_session",
|
|
)
|
|
store.update_status(connection.connection_id, status="connected", last_error=None)
|
|
try:
|
|
with TestClient(app) as client:
|
|
response = client.post(
|
|
"/api/channel-connector-bridge/events",
|
|
headers={"Authorization": "Bearer bridge-token"},
|
|
json={
|
|
"eventId": "evt-1",
|
|
"timestamp": "2026-06-02T09:30:00Z",
|
|
"deliveryAttempt": 1,
|
|
"connectionId": connection.connection_id,
|
|
"channelId": connection.channel_id,
|
|
"kind": "weixin",
|
|
"accountId": "weixin:me",
|
|
"peerId": "peer-1",
|
|
"peerType": "dm",
|
|
"userId": "sender-1",
|
|
"threadId": None,
|
|
"messageId": "msg-1",
|
|
"messageType": "text",
|
|
"content": "hello",
|
|
"metadata": {},
|
|
},
|
|
)
|
|
assert response.status_code == 200
|
|
assert response.json()["accepted"] is True
|
|
finally:
|
|
service.close()
|
|
|
|
|
|
def test_bridge_endpoint_rejects_invalid_token(tmp_path, monkeypatch) -> None:
|
|
app, service = _app(tmp_path, monkeypatch)
|
|
try:
|
|
with TestClient(app) as client:
|
|
response = client.post("/api/channel-connector-bridge/events", headers={"Authorization": "Bearer wrong"}, json={})
|
|
assert response.status_code == 401
|
|
finally:
|
|
service.close()
|
|
|
|
|
|
def test_bridge_endpoint_returns_conflict_for_processing_duplicate(tmp_path, monkeypatch) -> None:
|
|
app, service = _app(tmp_path, monkeypatch)
|
|
state_dir = tmp_path / "state" / "channel_connections"
|
|
store = ChannelConnectionStore(state_dir / "connections.json")
|
|
connection = store.create(
|
|
kind="weixin",
|
|
mode="sidecar",
|
|
display_name="Weixin Main",
|
|
account_id="weixin:me",
|
|
owner_user_id=None,
|
|
auth_type="connector_session",
|
|
)
|
|
store.update_status(connection.connection_id, status="connected", last_error=None)
|
|
payload = {
|
|
"eventId": "evt-1",
|
|
"timestamp": "2026-06-02T09:30:00Z",
|
|
"deliveryAttempt": 1,
|
|
"connectionId": connection.connection_id,
|
|
"channelId": connection.channel_id,
|
|
"kind": "weixin",
|
|
"accountId": "weixin:me",
|
|
"peerId": "peer-1",
|
|
"peerType": "dm",
|
|
"userId": "sender-1",
|
|
"threadId": None,
|
|
"messageId": "msg-1",
|
|
"messageType": "text",
|
|
"content": "hello",
|
|
"metadata": {},
|
|
}
|
|
try:
|
|
with TestClient(app) as client:
|
|
first = client.post("/api/channel-connector-bridge/events", headers={"Authorization": "Bearer bridge-token"}, json=payload)
|
|
second = client.post("/api/channel-connector-bridge/events", headers={"Authorization": "Bearer bridge-token"}, json={**payload, "deliveryAttempt": 2})
|
|
assert first.status_code == 200
|
|
assert second.status_code in {200, 409}
|
|
finally:
|
|
service.close()
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify failure**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_external_connector_bridge_api.py -q
|
|
```
|
|
|
|
Expected: fail with `404 Not Found` for `/api/channel-connector-bridge/events`.
|
|
|
|
- [ ] **Step 3: Add bridge schemas**
|
|
|
|
Append to `app-instance/backend/beaver/interfaces/web/schemas/chat.py`:
|
|
|
|
```python
|
|
class WebConnectorBridgeEventRequest(BaseModel):
|
|
event_id: str = Field(alias="eventId")
|
|
timestamp: str
|
|
delivery_attempt: int = Field(default=1, alias="deliveryAttempt")
|
|
connection_id: str = Field(alias="connectionId")
|
|
channel_id: str = Field(alias="channelId")
|
|
kind: str
|
|
account_id: str = Field(alias="accountId")
|
|
peer_id: str = Field(alias="peerId")
|
|
peer_type: str = Field(default="unknown", alias="peerType")
|
|
user_id: str | None = Field(default=None, alias="userId")
|
|
thread_id: str | None = Field(default=None, alias="threadId")
|
|
message_id: str = Field(alias="messageId")
|
|
message_type: str = Field(default="text", alias="messageType")
|
|
content: str
|
|
metadata: dict[str, Any] = Field(default_factory=dict)
|
|
|
|
|
|
class WebConnectorBridgeEventResponse(BaseModel):
|
|
accepted: bool
|
|
duplicate: bool = False
|
|
pending: bool = False
|
|
retry_after_seconds: int | None = Field(default=None, alias="retryAfterSeconds")
|
|
```
|
|
|
|
Export both in `app-instance/backend/beaver/interfaces/web/schemas/__init__.py`.
|
|
|
|
- [ ] **Step 4: Add bridge endpoint**
|
|
|
|
Modify `app-instance/backend/beaver/interfaces/web/app.py` imports to include:
|
|
|
|
```python
|
|
from beaver.foundation.events import ChannelIdentity, InboundMessage
|
|
from beaver.interfaces.channels.connections import MessageDedupeStore
|
|
```
|
|
|
|
Add helpers:
|
|
|
|
```python
|
|
def _bridge_token() -> str:
|
|
return os.getenv("BEAVER_BRIDGE_TOKEN", "")
|
|
|
|
|
|
def _message_dedupe_store(workspace: Path) -> MessageDedupeStore:
|
|
return MessageDedupeStore(_connection_state_dir(workspace) / "message_dedupe.json")
|
|
```
|
|
|
|
Add the route inside `create_app()`:
|
|
|
|
```python
|
|
@app.post("/api/channel-connector-bridge/events", response_model=WebConnectorBridgeEventResponse)
|
|
async def accept_connector_bridge_event(
|
|
request: Request,
|
|
payload: WebConnectorBridgeEventRequest,
|
|
authorization: str | None = Header(default=None),
|
|
) -> JSONResponse | WebConnectorBridgeEventResponse:
|
|
expected = _bridge_token()
|
|
if not expected or authorization != f"Bearer {expected}":
|
|
raise HTTPException(status_code=401, detail="Invalid connector bridge token")
|
|
registry = get_channel_connector_registry(request)
|
|
try:
|
|
connection = registry.connection_store.get(payload.connection_id)
|
|
except KeyError:
|
|
raise HTTPException(status_code=404, detail="Channel connection not found")
|
|
if connection.status == "revoked":
|
|
raise HTTPException(status_code=404, detail="Channel connection not found")
|
|
store = _message_dedupe_store(_channel_connection_workspace(request))
|
|
begin = store.begin(
|
|
connection_id=payload.connection_id,
|
|
event_id=payload.event_id,
|
|
delivery_attempt=payload.delivery_attempt,
|
|
)
|
|
if not begin.should_process:
|
|
body = WebConnectorBridgeEventResponse(
|
|
accepted=begin.http_status == 200,
|
|
duplicate=True,
|
|
pending=begin.http_status == 409,
|
|
retryAfterSeconds=begin.retry_after_seconds,
|
|
).model_dump(by_alias=True)
|
|
return JSONResponse(status_code=begin.http_status, content=body)
|
|
runtime = get_channel_runtime(request)
|
|
identity = ChannelIdentity(
|
|
channel_id=payload.channel_id,
|
|
kind=payload.kind,
|
|
account_id=payload.account_id,
|
|
peer_id=payload.peer_id,
|
|
thread_id=payload.thread_id,
|
|
peer_type=payload.peer_type,
|
|
user_id=payload.user_id,
|
|
message_id=payload.message_id,
|
|
)
|
|
inbound = InboundMessage(
|
|
channel=payload.channel_id,
|
|
content=payload.content,
|
|
content_type=payload.message_type,
|
|
channel_identity=identity,
|
|
user_id=payload.user_id,
|
|
message_id=payload.message_id,
|
|
metadata=dict(payload.metadata),
|
|
)
|
|
result = await runtime.accept_inbound(inbound)
|
|
if result.accepted:
|
|
store.complete(begin.dedupe_key, message_id=payload.message_id)
|
|
else:
|
|
store.fail(begin.dedupe_key, error=result.error or "runtime rejected bridge event")
|
|
return WebConnectorBridgeEventResponse(accepted=result.accepted, duplicate=result.duplicate, pending=result.pending)
|
|
```
|
|
|
|
If `_channel_connection_workspace(request)` does not exist yet, add it:
|
|
|
|
```python
|
|
def _channel_connection_workspace(request: Request) -> Path:
|
|
workspace = getattr(request.app.state, "channel_connection_workspace", None)
|
|
if workspace is not None:
|
|
return Path(workspace)
|
|
return Path(get_agent_service(request).loader.workspace)
|
|
```
|
|
|
|
- [ ] **Step 5: Run bridge tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_external_connector_bridge_api.py -q
|
|
```
|
|
|
|
Expected: all listed tests pass. If the duplicate test returns `200` because runtime completes before the second request, add a focused `MessageDedupeStore` API test for `409`; do not weaken the store behavior.
|
|
|
|
- [ ] **Step 6: Commit Task 5**
|
|
|
|
```bash
|
|
git add app-instance/backend/beaver/interfaces/web/app.py app-instance/backend/beaver/interfaces/web/schemas/chat.py app-instance/backend/beaver/interfaces/web/schemas/__init__.py app-instance/backend/tests/unit/test_external_connector_bridge_api.py
|
|
git commit -m "feat: add external connector bridge api"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: Weixin And Feishu Connectors
|
|
|
|
**Files:**
|
|
- Create: `app-instance/backend/beaver/interfaces/channels/connections/external.py`
|
|
- Modify: `app-instance/backend/beaver/interfaces/channels/connections/__init__.py`
|
|
- Modify: `app-instance/backend/beaver/interfaces/channels/connections/connectors.py`
|
|
- Modify: `app-instance/backend/beaver/interfaces/web/schemas/chat.py`
|
|
- Modify: `app-instance/backend/beaver/interfaces/web/schemas/__init__.py`
|
|
- Modify: `app-instance/backend/beaver/interfaces/web/app.py`
|
|
- Test: `app-instance/backend/tests/unit/test_external_sidecar_connectors.py`
|
|
|
|
- [ ] **Step 1: Write failing connector tests**
|
|
|
|
Create `app-instance/backend/tests/unit/test_external_sidecar_connectors.py`:
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
|
|
from beaver.interfaces.channels.connections import (
|
|
ChannelConnectionStore,
|
|
CredentialStore,
|
|
FeishuConnector,
|
|
WeixinConnector,
|
|
)
|
|
|
|
|
|
class FakeSidecarClient:
|
|
def __init__(self) -> None:
|
|
self.sessions: dict[str, dict] = {}
|
|
self.started: list[dict] = []
|
|
self.logged_out: list[str] = []
|
|
|
|
async def start_session(self, payload: dict) -> dict:
|
|
self.started.append(payload)
|
|
session = {
|
|
"sessionId": "cs_1",
|
|
"kind": payload["kind"],
|
|
"status": "qr_ready",
|
|
"qrImage": "data:image/png;base64,abc",
|
|
"accountId": None,
|
|
"displayName": None,
|
|
"metadata": {},
|
|
}
|
|
self.sessions["cs_1"] = session
|
|
return session
|
|
|
|
async def get_session(self, session_id: str) -> dict:
|
|
return self.sessions[session_id]
|
|
|
|
async def logout(self, connection_id: str) -> dict:
|
|
self.logged_out.append(connection_id)
|
|
return {"ok": True}
|
|
|
|
|
|
def test_weixin_connector_starts_connector_session(tmp_path) -> None:
|
|
async def run() -> None:
|
|
connection_store = ChannelConnectionStore(tmp_path / "connections.json")
|
|
credential_store = CredentialStore(tmp_path / "credentials.json")
|
|
client = FakeSidecarClient()
|
|
connector = WeixinConnector(
|
|
connection_store=connection_store,
|
|
credential_store=credential_store,
|
|
sidecar_client=client,
|
|
sidecar_base_url="http://external-connector:8787",
|
|
)
|
|
|
|
view = await connector.start_session(display_name="Weixin Main", owner_user_id="user-1", options={})
|
|
|
|
assert view["sessionId"] == "cs_1"
|
|
assert client.started[0]["kind"] == "weixin"
|
|
assert client.started[0]["connectionId"].startswith("conn_")
|
|
assert connection_store.list()[0].kind == "weixin"
|
|
assert connection_store.list()[0].status == "pairing"
|
|
|
|
asyncio.run(run())
|
|
|
|
|
|
def test_weixin_connector_poll_connected_materializes_external_runtime(tmp_path) -> None:
|
|
async def run() -> None:
|
|
connection_store = ChannelConnectionStore(tmp_path / "connections.json")
|
|
credential_store = CredentialStore(tmp_path / "credentials.json")
|
|
client = FakeSidecarClient()
|
|
connector = WeixinConnector(
|
|
connection_store=connection_store,
|
|
credential_store=credential_store,
|
|
sidecar_client=client,
|
|
sidecar_base_url="http://external-connector:8787",
|
|
)
|
|
await connector.start_session(display_name="Weixin Main", owner_user_id=None, options={})
|
|
connection = connection_store.list()[0]
|
|
client.sessions["cs_1"] = {
|
|
"sessionId": "cs_1",
|
|
"kind": "weixin",
|
|
"status": "connected",
|
|
"accountId": "weixin:me",
|
|
"displayName": "Me",
|
|
"metadata": {"stateRef": "state-1"},
|
|
}
|
|
|
|
result = await connector.poll_session("cs_1")
|
|
updated = connection_store.get(connection.connection_id)
|
|
spec = await connector.materialize_runtime(connection.connection_id)
|
|
|
|
assert result["status"] == "connected"
|
|
assert updated.status == "connected"
|
|
assert updated.account_id == "weixin:me"
|
|
assert spec.kind == "external_connector"
|
|
assert spec.mode == "http"
|
|
assert spec.config["platformKind"] == "weixin"
|
|
assert spec.config["sidecarBaseUrl"] == "http://external-connector:8787"
|
|
|
|
asyncio.run(run())
|
|
|
|
|
|
def test_feishu_connector_uses_feishu_kind(tmp_path) -> None:
|
|
async def run() -> None:
|
|
connection_store = ChannelConnectionStore(tmp_path / "connections.json")
|
|
credential_store = CredentialStore(tmp_path / "credentials.json")
|
|
client = FakeSidecarClient()
|
|
connector = FeishuConnector(
|
|
connection_store=connection_store,
|
|
credential_store=credential_store,
|
|
sidecar_client=client,
|
|
sidecar_base_url="http://external-connector:8787",
|
|
)
|
|
|
|
await connector.start_session(display_name="Feishu Main", owner_user_id=None, options={"domain": "feishu"})
|
|
|
|
assert client.started[0]["kind"] == "feishu"
|
|
assert client.started[0]["options"] == {"domain": "feishu"}
|
|
|
|
asyncio.run(run())
|
|
```
|
|
|
|
- [ ] **Step 2: Run tests to verify failure**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_external_sidecar_connectors.py -q
|
|
```
|
|
|
|
Expected: fail with `ImportError: cannot import name 'WeixinConnector'`.
|
|
|
|
- [ ] **Step 3: Implement connector classes**
|
|
|
|
Create `app-instance/backend/beaver/interfaces/channels/connections/external.py`:
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from typing import Any
|
|
|
|
from .models import ChannelRuntimeSpec, ValidationResult
|
|
from .sidecar_client import ConnectorSidecarClient
|
|
from .store import ChannelConnectionStore, CredentialStore
|
|
|
|
|
|
class ExternalConnectorBase:
|
|
kind = ""
|
|
capabilities: list[str] = []
|
|
|
|
def __init__(
|
|
self,
|
|
*,
|
|
connection_store: ChannelConnectionStore,
|
|
credential_store: CredentialStore,
|
|
sidecar_client: ConnectorSidecarClient | Any,
|
|
sidecar_base_url: str,
|
|
) -> None:
|
|
self.connection_store = connection_store
|
|
self.credential_store = credential_store
|
|
self.sidecar_client = sidecar_client
|
|
self.sidecar_base_url = sidecar_base_url
|
|
|
|
async def start_session(
|
|
self,
|
|
*,
|
|
display_name: str,
|
|
owner_user_id: str | None,
|
|
options: dict[str, Any],
|
|
) -> dict[str, Any]:
|
|
connection = self.connection_store.create(
|
|
kind=self.kind,
|
|
mode="sidecar",
|
|
display_name=display_name or self.kind,
|
|
account_id="",
|
|
owner_user_id=owner_user_id,
|
|
auth_type="connector_session",
|
|
runtime_config={"sidecarBaseUrl": self.sidecar_base_url},
|
|
capabilities=list(self.capabilities),
|
|
)
|
|
self.connection_store.update_status(connection.connection_id, status="pairing", last_error=None)
|
|
payload = {
|
|
"kind": self.kind,
|
|
"connectionId": connection.connection_id,
|
|
"channelId": connection.channel_id,
|
|
"displayName": connection.display_name,
|
|
"callbackBaseUrl": "",
|
|
"options": dict(options),
|
|
}
|
|
view = await self.sidecar_client.start_session(payload)
|
|
connection.pairing_session_id = str(view.get("sessionId") or "")
|
|
self.connection_store.update(connection)
|
|
return view
|
|
|
|
async def poll_session(self, session_id: str) -> dict[str, Any]:
|
|
view = await self.sidecar_client.get_session(session_id)
|
|
connection = self._connection_for_session(session_id)
|
|
status = str(view.get("status") or "")
|
|
if status == "connected":
|
|
connection.account_id = str(view.get("accountId") or connection.account_id)
|
|
connection.display_name = str(view.get("displayName") or connection.display_name)
|
|
metadata = view.get("metadata") if isinstance(view.get("metadata"), dict) else {}
|
|
state_ref = metadata.get("stateRef")
|
|
if state_ref:
|
|
connection.credentials_ref = self.credential_store.put(kind=self.kind, values={"stateRef": state_ref})
|
|
self.connection_store.update(connection)
|
|
self.connection_store.update_status(connection.connection_id, status="connected", last_error=None)
|
|
elif status in {"expired", "error", "cancelled"}:
|
|
self.connection_store.update_status(connection.connection_id, status="error", last_error=str(view.get("error") or status))
|
|
return view
|
|
|
|
async def validate(self, connection_id: str) -> ValidationResult:
|
|
connection = self.connection_store.get(connection_id)
|
|
if connection.status in {"connected", "running"}:
|
|
return ValidationResult(ok=True, status="connected", account_id=connection.account_id, display_name=connection.display_name)
|
|
return ValidationResult(ok=False, status=connection.status, error=connection.last_error)
|
|
|
|
async def materialize_runtime(self, connection_id: str) -> ChannelRuntimeSpec:
|
|
connection = self.connection_store.get(connection_id)
|
|
return ChannelRuntimeSpec(
|
|
channel_id=connection.channel_id,
|
|
kind="external_connector",
|
|
mode="http",
|
|
account_id=connection.account_id,
|
|
display_name=connection.display_name,
|
|
config={
|
|
"platformKind": self.kind,
|
|
"connectionId": connection.connection_id,
|
|
"sidecarBaseUrl": connection.runtime_config.get("sidecarBaseUrl") or self.sidecar_base_url,
|
|
},
|
|
secrets_ref=None,
|
|
)
|
|
|
|
async def revoke(self, connection_id: str) -> None:
|
|
await self.sidecar_client.logout(connection_id)
|
|
|
|
def _connection_for_session(self, session_id: str):
|
|
for connection in self.connection_store.list():
|
|
if connection.pairing_session_id == session_id:
|
|
return connection
|
|
raise KeyError(session_id)
|
|
|
|
|
|
class WeixinConnector(ExternalConnectorBase):
|
|
kind = "weixin"
|
|
capabilities = ["receive_text", "send_text", "receive_media", "direct_messages"]
|
|
|
|
|
|
class FeishuConnector(ExternalConnectorBase):
|
|
kind = "feishu"
|
|
capabilities = ["receive_text", "send_text", "receive_media", "groups"]
|
|
```
|
|
|
|
- [ ] **Step 4: Export connectors and update registry materialization**
|
|
|
|
Modify `app-instance/backend/beaver/interfaces/channels/connections/__init__.py`:
|
|
|
|
```python
|
|
from .external import ExternalConnectorBase, FeishuConnector, WeixinConnector
|
|
```
|
|
|
|
Add names to `__all__`.
|
|
|
|
Ensure `ChannelConnectorRegistry.materialize_channel_configs()` accepts `ChannelRuntimeSpec(kind="external_connector", mode="http")` and emits `ChannelConfig(secrets={})`.
|
|
|
|
- [ ] **Step 5: Add connector session web schemas**
|
|
|
|
Append to `app-instance/backend/beaver/interfaces/web/schemas/chat.py`:
|
|
|
|
```python
|
|
class WebConnectorSessionCreateRequest(BaseModel):
|
|
kind: str
|
|
display_name: str | None = Field(default=None, alias="displayName")
|
|
owner_user_id: str | None = Field(default=None, alias="ownerUserId")
|
|
options: dict[str, Any] = Field(default_factory=dict)
|
|
|
|
|
|
class WebConnectorSessionResponse(BaseModel):
|
|
session: dict[str, Any]
|
|
connection: dict[str, Any] | None = None
|
|
```
|
|
|
|
Export both in `app-instance/backend/beaver/interfaces/web/schemas/__init__.py`.
|
|
|
|
- [ ] **Step 6: Register Weixin/Feishu in app registry**
|
|
|
|
Modify `_build_channel_connector_registry()` in `app-instance/backend/beaver/interfaces/web/app.py` to create a sidecar client:
|
|
|
|
```python
|
|
sidecar_base_url = os.getenv("EXTERNAL_CONNECTOR_BASE_URL", "http://external-connector:8787")
|
|
sidecar_token = os.getenv("EXTERNAL_CONNECTOR_TOKEN", "")
|
|
sidecar_client = ConnectorSidecarClient(base_url=sidecar_base_url, token=sidecar_token)
|
|
registry.register(WeixinConnector(
|
|
connection_store=connection_store,
|
|
credential_store=credential_store,
|
|
sidecar_client=sidecar_client,
|
|
sidecar_base_url=sidecar_base_url,
|
|
))
|
|
registry.register(FeishuConnector(
|
|
connection_store=connection_store,
|
|
credential_store=credential_store,
|
|
sidecar_client=sidecar_client,
|
|
sidecar_base_url=sidecar_base_url,
|
|
))
|
|
```
|
|
|
|
Keep Telegram registration unchanged.
|
|
|
|
- [ ] **Step 7: Add connector session API routes**
|
|
|
|
Add routes near existing channel connection APIs in `app-instance/backend/beaver/interfaces/web/app.py`:
|
|
|
|
```python
|
|
@app.post("/api/channel-connector-sessions", response_model=WebConnectorSessionResponse)
|
|
async def start_channel_connector_session(
|
|
request: Request,
|
|
payload: WebConnectorSessionCreateRequest,
|
|
) -> WebConnectorSessionResponse:
|
|
registry = get_channel_connector_registry(request)
|
|
connector = registry.connector_for_kind(_clean_text(payload.kind))
|
|
if not hasattr(connector, "start_session"):
|
|
raise HTTPException(status_code=400, detail="Connector does not support sessions")
|
|
view = await connector.start_session(
|
|
display_name=_clean_text(payload.display_name) or _clean_text(payload.kind),
|
|
owner_user_id=_clean_text(payload.owner_user_id) or None,
|
|
options=payload.options,
|
|
)
|
|
connection = registry.connection_store.get(str(view.get("connectionId") or registry.connection_store.list()[-1].connection_id))
|
|
return WebConnectorSessionResponse(session=view, connection=_connection_response_view(connection))
|
|
|
|
@app.get("/api/channel-connector-sessions/{session_id}", response_model=WebConnectorSessionResponse)
|
|
async def get_channel_connector_session(session_id: str, request: Request) -> WebConnectorSessionResponse:
|
|
registry = get_channel_connector_registry(request)
|
|
connection = next((item for item in registry.connection_store.list() if item.pairing_session_id == session_id), None)
|
|
if connection is None:
|
|
raise HTTPException(status_code=404, detail="Connector session not found")
|
|
connector = registry.connector_for_kind(connection.kind)
|
|
if not hasattr(connector, "poll_session"):
|
|
raise HTTPException(status_code=400, detail="Connector does not support sessions")
|
|
view = await connector.poll_session(session_id)
|
|
connection = registry.connection_store.get(connection.connection_id)
|
|
if connection.status == "connected":
|
|
runtime = get_channel_runtime(request)
|
|
config = (await registry.materialize_channel_configs())[connection.channel_id]
|
|
await runtime.add_channel(connection.channel_id, config)
|
|
return WebConnectorSessionResponse(session=view, connection=_connection_response_view(connection))
|
|
```
|
|
|
|
Add `connector_for_kind()` to `ChannelConnectorRegistry`:
|
|
|
|
```python
|
|
def connector_for_kind(self, kind: str) -> ChannelConnector:
|
|
return self._connector(kind)
|
|
```
|
|
|
|
- [ ] **Step 8: Run connector tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_external_sidecar_connectors.py tests/unit/test_channel_connection_api.py tests/unit/test_channel_connector_registry.py -q
|
|
```
|
|
|
|
Expected: all listed tests pass.
|
|
|
|
- [ ] **Step 9: Commit Task 6**
|
|
|
|
```bash
|
|
git add app-instance/backend/beaver/interfaces/channels/connections app-instance/backend/beaver/interfaces/web/app.py app-instance/backend/beaver/interfaces/web/schemas/chat.py app-instance/backend/beaver/interfaces/web/schemas/__init__.py app-instance/backend/tests/unit/test_external_sidecar_connectors.py
|
|
git commit -m "feat: add sidecar-backed channel connectors"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 7: Final Backend Verification
|
|
|
|
**Files:**
|
|
- Review: `docs/superpowers/specs/2026-06-02-external-sidecar-connectors-design.md`
|
|
|
|
- [ ] **Step 1: Run focused backend tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest \
|
|
tests/unit/test_connector_message_dedupe_store.py \
|
|
tests/unit/test_external_connector_channel.py \
|
|
tests/unit/test_channel_runtime_dynamic_channels.py \
|
|
tests/unit/test_external_connector_bridge_api.py \
|
|
tests/unit/test_external_sidecar_connectors.py \
|
|
tests/unit/test_channel_connection_store.py \
|
|
tests/unit/test_channel_connector_registry.py \
|
|
tests/unit/test_channel_connection_api.py \
|
|
tests/unit/test_channel_runtime.py \
|
|
tests/unit/test_gateway_channels.py \
|
|
-q
|
|
```
|
|
|
|
Expected: all listed tests pass.
|
|
|
|
- [ ] **Step 2: Run import tests**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
uv run pytest tests/unit/test_imports.py -q
|
|
```
|
|
|
|
Expected: all import tests pass.
|
|
|
|
- [ ] **Step 3: Scan for leaked service tokens in implementation**
|
|
|
|
Run:
|
|
|
|
```bash
|
|
cd app-instance/backend
|
|
rg -n "bridge-token|connector-token|token-1|token-2|secret-token" beaver || true
|
|
```
|
|
|
|
Expected: no implementation files contain fixture token values.
|
|
|
|
- [ ] **Step 4: Commit verification-only fixes if needed**
|
|
|
|
If Step 1 or Step 2 required a small fix, commit it:
|
|
|
|
```bash
|
|
git add app-instance/backend
|
|
git commit -m "fix: stabilize external connector backend runtime"
|
|
```
|
|
|
|
If no files changed, do not create an empty commit.
|