# Agentee Documentation — Full Text > Concatenated corpus of every documentation page. Generated by scripts/build-llms.js. --- # What is Agentee? > A multi-tenant **Sales AI platform** that runs conversational agents directly on top of **Twendee ERP** — chat, CRM actions, proposals, and admin-defined workflows, all in one web chat surface. ## Overview **Agentee** (codebase: `TWDAgentsHub`) is a multi-tenant SaaS for Vietnamese SMEs. End users talk to a **Sales Agent** in a web chat that is wired into the Twendee **CRM/ERP** through an MCP tool layer. The agent can look up and mutate sales data, draft branded proposals, and drive multi-step business workflows — with human approval on every irreversible action. It is not a single chatbot. It is an **agent runtime** designed so that many specialised agents (Sale, Delivery Manager, and future profiles) share one platform, one ERP integration, one governance model, and one production deployment. ## What it does | Capability | What the user gets | |---|---| | **Sale Agent** | Streaming chat, file/image reading, vision → lead, episodic memory, follow-up suggestions, approval gates. | | **CRM/ERP integration** | Read and write deals, leads, companies, follow-ups — scoped to the user's own ERP permissions. | | **Proposal Builder** | Draft, section-edit, version, publish and export branded DOCX/PDF proposals tied to a deal. | | **Dynamic Workflow Engine** | Admins author SOP-markdown workflows; agents run them step-by-step with approvals. | | **Delivery Manager** | Second agent profile: decomposes a SOW into a WBS, exports CSV / pushes to Jira. | | **Collaboration** | Read-only session sharing, fork-edit-merge co-build, notifications, @mentions. | | **Multi-provider LLM** | Per-tenant provider + key, smart/fast/vision tiering, token metering. | | **Sub-agents** | Web Research (Tavily), Documentor (export), Agent Skills (sandboxed scripts). | See **[Feature Overview](/feature-overview)** for the full map and release status. ## Why it is built this way Agentee is an agent system for a **production ERP**, so three concerns shape every design decision: - **Isolation** — each tenant's data, credentials, and traces stay separate; every write is scoped to the user's ERP identity. - **Governance** — reads run freely, writes pause for approval; a default-DENY classifier gates every ERP operation. - **Safe rollout** — new agents and features ship *dark* behind feature flags, enabled per tenant. The heart of the platform — **how agents are organized, routed, and governed on top of the ERP** — is covered in the **[Agent organization](/agent-topology)** section. ## Tech stack - **Backend:** NestJS (modular monolith), TypeScript, Vercel **AI SDK v6** (`ToolLoopAgent`). - **Data:** PostgreSQL 16 (Drizzle ORM), Redis 7 + BullMQ (background jobs). - **Frontend:** React + Vite + shadcn/ui (chat-ui SPA). - **Monorepo:** Turborepo + pnpm workspaces (`apps/*`, `packages/*`, `tooling/*`). - **LLM providers:** Anthropic, OpenAI, Google, DeepSeek, GLM, Groq (per-tenant BYO key). ## Where to go next - [Architecture Overview](/architecture-overview) — the tiers, pillars, and data flow. - [Agent Topology](/agent-topology) — front orchestrator + specialists, the core of the platform. - [Environments & Deployment](/environments-deploy) — how Agentee runs in production. --- # Architecture Overview > Agentee is a NestJS modular monolith organized around **three pillars** — the agent runtime, the workflow engine, and sharing/collaboration — sitting on PostgreSQL + Redis, integrated into Twendee ERP over MCP. ## High-level tiers ```mermaid flowchart TB subgraph FE["chat-ui — React / Vite SPA"] UI["Sale-agent chat · workflow output · proposal panel"] end FE -->|HTTP REST + SSE| API subgraph API["@twd/api — NestJS backend"] AUTH["Auth + Tenants + Access Guard"] P1["Pillar 1 — Agent Runtime
dispatcher → orchestrator → sale / delivery-manager
→ sub-agents (researcher, documentor)"] P2["Pillar 2 — Dynamic Workflow Engine
SOP compiler → graph → 4 node executors"] P3["Pillar 3 — Sharing & Collaboration
shared sessions · @mentions · co-build"] SUP["Support: llm/ · integrations/erp (MCP) · me/ memory · notifications/"] AUTH --> P1 --> P2 --> P3 --> SUP end API -->|MCP JSON-RPC + HTTP| ERP["Twendee ERP
CRM · identity · sidebar-menu"] SUP --> DB[("PostgreSQL 16
Drizzle")] SUP --> REDIS[("Redis 7
BullMQ")] P1 -->|per-tenant key| LLM["LLM providers
Anthropic · OpenAI · Google · DeepSeek · GLM · Groq"] ``` The frontend holds no business logic beyond presentation; the backend is authoritative on every decision. All live chat flows through one endpoint: `POST /agents/:agentId/chat/stream` (SSE). ## The three pillars | Pillar | Module(s) | Responsibility | |---|---|---| | **1 — Agent Runtime** | `agents/runtime`, `agents/sale`, `agents/delivery-manager` | Per-request `ToolLoopAgent` (AI SDK v6). A dispatcher routes each turn (sticky session → orchestrator front → specialist). Sub-agents (researcher, documentor) are called in-process. | | **2 — Workflow Engine** | `workflow/` | PostgreSQL-backed state machine. SOP markdown compiles to a JSON graph of 4 node types (`ai_chat`, `form`, `approval`, `automation`). Output stored in `workflow_outputs`, exportable to PDF/DOCX/MD. Gated by `WORKFLOW_ENGINE_ENABLED`. | | **3 — Sharing & Collaboration** | `sharing/`, `mentions/` | Read-only session share-by-link, @mention directory, and fork-edit-push-merge co-build over workflow outputs. | See **[Three Core Pillars](/three-pillars)** for the runtime detail. ## Request path (live chat turn) ```mermaid sequenceDiagram participant U as User (chat-ui) participant C as agent-chat.controller participant G as AgentAccessGuard participant D as turn-dispatcher participant R as agent-turn-runner (ToolLoopAgent) participant T as Tools (ERP MCP · sub-agents · workflow) U->>C: POST /agents/:agentId/chat/stream (SSE) C->>G: check user ↔ agent access G->>D: resolve target agent (sticky → orchestrator front) D->>R: run the turn R->>T: tool calls (reads auto · writes → approval) T-->>R: results (quarantined if from research) R-->>U: streamed tokens + tool previews R->>R: turn-metering + trace capture ``` Details: **[Request Lifecycle](/request-lifecycle)**. ## Runtime strategy (important) Agentee runs on **one always-on runtime**: the Vercel AI SDK `ToolLoopAgent`. A previously explored LangGraph migration was **cancelled and removed** (ADR-001, 2026-07-26) and was never enabled in production. Treat any LangGraph references in older docs as historical. ## Data & persistence - **PostgreSQL 16** via Drizzle — chat, workflow, proposal, sharing, notifications, memory, observability tables. Tenant-scoped by explicit `tenant_id`/`userId` filters (see the [multi-tenancy note](/stack-multitenancy)). - **Redis 7 + BullMQ** — workflow automation jobs, async memory writes, notification fan-out, replay protection. - **Model capabilities/pricing** live in code (`llm/models/model-registry.ts`), not the DB. ## Next - [Agent Topology](/agent-topology) — how the agents are organized. - [Stack & Multi-Tenancy](/stack-multitenancy) — isolation, security, identity bridge. - [Repository Structure](/repo-structure) — where everything lives. --- # Feature Overview > The full Agentee feature map — 10 groups (A–J), an honest release-status legend, and jump-off points into the detailed feature pages. ## Release-status legend | Badge | Meaning | |---|---| | ✅ **Live** | Enabled for end users. | | 🚧 **Dark-launch** | Code-complete but hidden behind a feature flag (default **OFF**), enabled per tenant. | Many workflow and collaboration features ship *dark* — the badge reflects what a user can actually reach, not what exists in code. Full flag table in [Feature Flags](/feature-flags). ## Feature map (groups A–J) | # | Group | What it does | Status | |---|---|---|---| | A | **Sale Agent v2** | Streaming web-chat assistant: attachments/vision, episodic memory, follow-ups, approval gates, context compaction. | ✅ Live | | B | **CRM/ERP integration** | Read/write deals, leads, companies, follow-ups on Twendee ERP; reads free, writes approval-gated; role-based visibility. | ✅ Live | | C | **Proposal Builder** | Draft a proposal from a deal, section-edit, version/publish, handoff, export DOCX/PDF/MD. | ✅ Live | | D | **Dynamic Workflow Engine** | Admin-authored SOP-markdown workflows compiled to a graph; 4 node types; reconciler + kill-switch. | 🚧 Dark-launch | | E | **Delivery Manager agent** | Second agent profile: SOW → WBS tree, CSV export, Jira push. | 🚧 Per-tenant, default OFF | | F | **Collaboration & sharing** | Read-only session share, fork-edit-merge co-build, notifications, @mentions. | 🚧 Partly dark-launch | | G | **Multi-provider LLM platform** | Per-tenant BYO key, smart/fast/vision tiering, vision safety-net, token metering. | ✅ Live | | H | **Sub-agents** | Web Research (Tavily), Documentor (export), Agent Skills (sandboxed scripts). | ✅ Live | | I | **Chat UI** | Streaming chat, command palette, voice input (STT), sessions, settings, workflow editor. | ✅ Live | | J | **Platform** | ERP SSO/bridge, multi-tenant isolation, onboarding, i18n (EN/VI), observability capture. | ✅ Live | ## How the groups relate ```mermaid flowchart TB U[User · web chat] --> SA[A · Sale Agent v2] SA --> CRM[B · CRM/ERP tools] SA --> PB[C · Proposal Builder] SA --> WF[D · Workflow Engine] SA --> SUB[H · Sub-agents] DM[E · Delivery Manager] --> WF WF --> COLLAB[F · Collaboration] PB --> COLLAB SA -.runs on.-> LLM[G · LLM Platform] DM -.runs on.-> LLM subgraph PLAT[I · Chat UI · J · Platform] U end ``` - **A + B** are the everyday surface: chat that reads and mutates ERP sales data. - **C** produces branded deliverables from a deal; **D** generalizes that into any admin-defined process. - **E** is a second specialist agent that reuses the same engine (D). - **F** layers sharing/co-build on top of proposals and workflow outputs. - **G/H/I/J** are the cross-cutting platform every group runs on. ## Status reality check - The agent runtime is **Vercel AI SDK v6 `ToolLoopAgent`** only. An earlier LangGraph migration was **cancelled and removed** ([ADR-001](/architecture-overview)); do not treat it as active. - The **live channel is web chat**. Telegram config/schema fields remain in code but Telegram is **not** an active channel (product decision to drop). - Groups **D and F** are largely gated OFF by default; **E** is per-tenant OFF. Treat their sub-features as dark unless a tenant flag is flipped. ## Detailed feature pages | Page | Covers | |---|---| | [Sale Agent & CRM/ERP](/sale-agent) | Group A + B — the conversational assistant and its ERP tool layer. | | [Proposal & Workflow Engine](/proposal-workflow) | Group C + D + E — deliverable authoring, the workflow graph, Delivery Manager. | | [Collaboration & LLM Platform](/collaboration-llm) | Group F + G + H — sharing, multi-provider LLM, sub-agents. | | [Agent Organization](/agent-topology) | How agents are routed, governed, and added on top of the ERP. | ## Next - [Sale Agent & CRM/ERP](/sale-agent) — start here for the core product. - [Feature Flags](/feature-flags) — the full dark-launch flag table. - [Architecture Overview](/architecture-overview) — the three pillars behind the map. --- # Sale Agent & CRM/ERP > Sale Agent v2 is the everyday web-chat assistant — streaming chat, attachments/vision, episodic memory, approval-gated ERP writes — wired directly into Twendee CRM/ERP. Sale Agent v2 (`agents/sale/`) is a per-request Vercel AI SDK v6 `ToolLoopAgent` with a 40+ tool ToolSet (CRM, proposal, sub-agent delegation). It runs on the shared runtime — see [Agent Topology](/agent-topology). ## Sale Agent v2 capabilities | Capability | What the user gets | Status | |---|---|---| | **Streaming chat (SSE)** | Answers stream in token-by-token, with typing indicator. | ✅ | | **Attachments** | Upload PDF/DOCX/XLSX/CSV/TXT/MD + images (5 files / 10 MB); agent reads content to answer. | ✅ | | **Vision → lead** | Snap a business card or customer list; a vision-tier read extracts structured leads. Extraction is an on-demand, read-only tool (`extract_lead_from_attachment`) — the CRM create still passes the approval gate. | ✅ | | **Bulk lead import** | Normalize + preview a lead list from CSV/image before creating in CRM. | ✅ | | **Follow-up suggestions** | After each turn, 2–3 next questions/actions as clickable chips. | ✅ | | **Auto session titles** | Short chat titles generated automatically. | ✅ | | **Context compaction** | Long history auto-summarized so threads stay under the token budget. | ✅ | | **Episodic memory** | Learns/stores per-user facts + preferences across sessions (pgvector `user_memories`, async writes); user can view/edit/delete. | ✅ | | **Role persona** | Agent tone/behavior adjusts to the sales persona. | ✅ | | **Approval gates** | Irreversible actions (e.g. publish proposal, CRM writes) pause for explicit confirmation before running. | ✅ | | **Auto response language** | Detects the user's language and replies in it (`RESPONSE_LANGUAGE_SYNC_ENABLED`, ON). | ✅ | ### Attachment handling (how bytes reach the LLM) Attachments hydrate **for the current turn only** — historical parts are stripped to `[attachment: ]` stubs so bytes are never re-sent (saves tokens, prevents drift). Hydration is provider-aware: | File type | How it reaches the model | |---|---| | **Images** | Native image part on vision providers. On a non-vision binding, one vision-tier call transcribes it out-of-band (cached), injected as labeled text; only a failed read degrades to a `[image attached: not readable]` stub. | | **PDF** | Native file part **only** on `anthropic` / `google`; extract-to-text everywhere else (incl. OpenAI-compatible gateways). | | **DOCX/XLSX/CSV/TXT/MD** | Extract-to-text labeled part. | Extracted-text budget: **12k tokens/turn aggregate, 4k/file** (truncated). Hydration runs in `AgentTurnRunner` (not a profile hook), so `sale`, `delivery-manager`, and `orchestrator` read attachments identically. ## CRM/ERP integration (Twendee ERP) The agent reaches the ERP through an MCP tool layer. **Reads run free; every write is wrapped in an approval + parameter check.** Live tool calls thread the user's own ERP JWT, so CRM operations run under the user's ERP RBAC — not a tenant-shared token. ```mermaid flowchart LR A[Sale Agent turn] --> C{read or write?} C -->|read| R[CRM read tool
runs immediately] C -->|write| G[approval gate
crm_api_call] G -->|user approves| W[ERP write
under user ERP JWT] G -->|reject| X[no-op] ``` | Area | What the user can do | |---|---| | **CRM lookup (read)** | Find/read companies, leads, deals, follow-ups; view KPI dashboard. | | **Deal management** | Create, update, change status; add/read comments. | | **Lead management** | Create and update leads from chat. | | **Company management** | Create and update company records. | | **Follow-up management** | Create, update, mark-complete customer-care tasks. | | **Undo** | Reverse the CRM write just performed. | | **Generic CRM escape-hatch** | Browse + operate CRM entities beyond deals (campaign, proposal…) via a dynamic operation catalog. | ### Governance & access - **Read free / write approval-gated** — a default-DENY classifier gates every ERP operation; writes surface an approval card in chat. - **Role-based visibility** — a rep sees only records they created/follow; manager/BOD see all. The filter is applied automatically from ERP role. - **Menu-driven access** — the agent exposes only the features a user's real ERP sidebar menu permits; `AgentAccessGuard` gates which agents a user can reach at all. > **Scope note:** beyond the core CRM set (~21 tools), the catalog defines an extended **sales-outreach** group (~18 tools: lead gen, enrich/dedupe, ICP scoring, email/LinkedIn/SMS sequences, human escalation). How fully these are wired into the Sale Agent v2 runtime is **unverified** — treat as definition-level until confirmed. ## Related - [ERP MCP & Governance](/erp-mcp) — the tool layer and approval classifier in depth. - [Access Control](/access-control) — role capabilities and menu-driven gating. - [Proposal & Workflow Engine](/proposal-workflow) — what the agent builds from a deal. - [Collaboration & LLM Platform](/collaboration-llm) — the LLM tiering and sub-agents behind every turn. --- # Proposal & Workflow Engine > How Agentee turns a deal into a deliverable — the Proposal Builder (live), the admin-authored Dynamic Workflow Engine (dark-launch), and the Delivery Manager agent (per-tenant). ## Proposal Builder — draft, version, export Deal-scoped proposal authoring, driven step-by-step through chat. **Live** (legacy `sales/proposal-builder/` path, mid-migration onto the workflow engine via `PROPOSAL_ON_ENGINE`). | Feature | What the user gets | |---|---| | **Create from deal** | Generate a proposal tied to a specific CRM deal. | | **Step-by-step authoring** | Agent walks the steps, locking agreed content before advancing. | | **Real-time A4 preview** | Live document preview updates as data is entered. | | **Section edit** | Replace content manually or ask AI to rewrite a section; manual edits survive regeneration. | | **Revert to AI draft** | Drop manual edits and re-generate a section from source data. | | **Cover-page edit** | Edit "Prepared by / End Client / Version". | | **Template pick** | Choose + lock a presentation template at the final step. | | **Publish + versioning** | Publish locks an immutable version; view/compare/rollback prior versions (Versions tab). | | **Handoff** | Pass the "pen" to a colleague by @username; the recipient becomes the sole editor (`PROPOSAL_ASSIGN_ENABLED`, ON). | | **Assigned list** | List and reopen proposals handed off to you. | | **Export** | Render DOCX / PDF / Markdown (TOC, page breaks) via the Documentor sub-agent. | ## Dynamic Workflow Engine — 🚧 dark-launch Lets **admins define business processes without code** and lets the **agent drive them in chat**. Whole group gated by `WORKFLOW_ENGINE_ENABLED` (default OFF, kill-switch), agent-driving by `WORKFLOW_AGENT_DRIVE_ENABLED`. ```mermaid flowchart LR SOP[SOP markdown
natural-language process] --> C[compiler
LLM + cite-back] C --> G[JSON graph
nodes + edges + schema] G --> GATE{activation gates
well-formed · valid tools · registered actions} GATE -->|pass| ACT[active definition] ACT --> ENG[engine interpreter
node-by-node] ``` ### Authoring - **SOP-markdown** — business writes the process as a natural-language markdown doc (like a `SKILL.md`), no programming. - **Compile SOP → graph** — the compiler turns prose into a node/edge/schema graph, self-repairing missing branches. - **Visual canvas editor** — canvas + node/edge inspector, AI-assisted fixes, diff review before Apply. - **Controlled activation** — a draft only goes "active" after passing gates (well-formedness, valid tools, registered actions, adequate descriptions). - **Author RBAC** — only executive roles (BOD/Admin/Management/CEO) may compile/activate/remove workflows. ### The 4 node types | Node | Behavior | |---|---| | `ai_chat` | Inline LLM Q&A that extracts an agreed data slice and validates it before advancing. | | `form` | Creates a task awaiting a schema-validated form submission. | | `approval` | Human-in-the-loop: pauses for approve/reject; blocks the initiator from self-approving. | | `automation` | Background job via BullMQ — kill-switch, cursor-drift protection, 3 retries. | ### Runtime safety - **Conditional branching** — edges carry fail-closed safe expressions (malicious expressions blocked). - **Action vs Document mode** — document workflows show an output panel + step-chips; action workflows (e.g. create lead) run inline, full-width chat. - **One workflow per session** — at most one live workflow per chat session. - **Reconciler + kill-switch** — a cron auto-recovers stuck instances; flipping the engine flag OFF freezes both running and new instances. - **Cross-user approvals** — approval tasks notify a different approver via an `/approvals` inbox deep-link; agents can approve/reject in-chat. ### Sample workflows (seeded) | Workflow | Purpose | |---|---| | `proposal-builder` | Full proposal-authoring process. | | `crm-create-lead` | Collect → enrich → dedupe → confirm → save lead. | | `crm-create-company` / `crm-create-campaign` | Create CRM company / campaign. | | `work-breakdown-structure` | SOW → EPIC/Story/Task/Subtask tree → PM approval → render. | | `weekly-report-demo` / `proposal-report-demo` | End-to-end engine validation demos. | New tenants are seeded these as drafts. When `FORCE_WORKFLOW_ONLY` is set, every deliverable must pass through an admin-defined workflow. ## Delivery Manager agent — 🚧 per-tenant, default OFF A second agent profile (`agents/delivery-manager/`) in the PM/delivery role, on the same runtime. | Feature | What the user gets | |---|---| | **SOW → WBS** | Accepts a Statement of Work (paste, file, or **Jira ticket link**) and decomposes it into an EPIC/Story/Task/Subtask tree. | | **Read Jira from link** | Paste a ticket link/key; the agent reads it via the user's own Jira OAuth and proposes a WBS. Not connected → returns a connect link. Works from both the front agent and Delivery Manager. | | **Runs on the standard engine** | Uses the shared `wbs-generation` workflow + output store — no bespoke tooling. | | **PM approval** | The WBS stays a draft until a project manager approves; the agent stops at that gate. | | **Export to document** | After approval, export DOCX/PDF/Markdown. | | **Export to CSV (Jira import)** | Flat Epic/Story/Task/Subtask table for Jira import or spreadsheet (direct Jira push is on the roadmap). | | **Multi-agent routing** | The dispatcher picks Sale vs Delivery Manager by @mention/sticky/intent in the same chat UI. | | **Menu access guard** | Visible only to users with PM rights in the ERP sidebar. | ## Related - [Sale Agent & CRM/ERP](/sale-agent) — where a deal and its data come from. - [Feature Flags](/feature-flags) — the workflow/proposal flag states. - [Collaboration & LLM Platform](/collaboration-llm) — co-build merge on workflow outputs. - [Agent Topology](/agent-topology) — how Delivery Manager registers as a profile. --- # Collaboration & LLM Platform > How teammates share and co-build deliverables (mostly dark-launch), the multi-provider LLM platform every turn runs on (live), and the specialised sub-agents. ## Collaboration & sharing Sharing and co-editing on top of proposals and workflow outputs. Sharing/co-build are gated; notifications and @mentions are live. | Feature | What the user gets | Status | |---|---|---| | **Read-only session share** | Share a session's result with a colleague or a link; recipient views `/share/:id` read-only (`shared_sessions` table). | 🚧 `SHARE_SESSION_ENABLED` | | **Co-build: fork → edit → push → merge** | Owner invites a collaborator to fork a private copy, edit, and push back; owner reviews and merges per-section (staged commit). | 🚧 `OUTPUT_COLLAB_ENABLED` | | **3-column merge review** | IDE-style: classifies sections untouched/clean/conflict; shows Current / Incoming / Result before committing. | 🚧 `OUTPUT_COLLAB_ENABLED` | | **Reverse pull (owner → collaborator)** | Collaborator pulls the owner's latest version. | 🚧 `OUTPUT_COLLAB_ENABLED` | | **Cursor sync after merge** | Workflow resumes at the right position post-merge (`COBUILD_MERGE_CURSOR_SYNC`, ON). | 🚧 | | **Real-time notifications** | SSE inbox at `/notifications` with unread count (proposal assigned, share, collab output…). | ✅ | | **@mention agent/user** | Mention an agent (Sale, Web-Research, Documentor) or another user in chat (`PROPOSAL_ASSIGN_ENABLED`, ON). | ✅ | | **Multi-user proposal collaboration** | Several people work one proposal, permissions synced to ERP deal access. | ✅ | ```mermaid flowchart LR OWN[Owner document] -->|invite| FORK[Collaborator fork] FORK -->|edit + push| STAGE[staged_decisions] STAGE --> REV[3-column merge review
Current · Incoming · Result] REV -->|merge per section| OWN OWN -.reverse pull.-> FORK ``` ## Multi-provider LLM platform — ✅ live Per-tenant, tier-aware LLM resolution. Model capabilities + pricing live in code (`llm/models/model-registry.ts`), not the DB. | Feature | Detail | |---|---| | **Multi-provider** | anthropic, openai, google, deepseek, glm, groq, ollama. | | **BYO key per tenant** | Each tenant configures its own providers + API keys; credentials AES-256-GCM encrypted at rest (`tenant_llm_providers`). | | **smart / fast / vision tiering** | Each task requests a tier; strong model for reasoning, fast model for cheap turns, vision model for image/OCR. | | **Vision safety-net** | A tenant with no vision model still gets image→lead via the platform's default vision model. | | **Token metering** | Per-turn token + cost metering (`MAIN_LOOP_METERING_ENABLED`, ON) recorded to `pipeline_events`. | | **Prompt platform / personas** | System prompts assembled by persona, language, and token budget. | ### Tier resolution order 1. **Agent pin** — if the agent pins `llmProviderId`, resolve from that entry only (requested tier, degrading to its other tier); never falls through to other providers. 2. **Tenant walk** — no pin → priority walk across active entries (`priority ASC, created_at ASC`); first with a non-null model for the requested tier wins. smart/fast resolve independently. 3. **Agent override** — `agent.llmModel` is an absolute, tier-agnostic model override. Errors surface as `LlmConfigError` (`agent.llm_provider_unavailable`, `agent.pinned_provider_no_model`, `tenant.no_llm_provider_for_tier`). ## Sub-agents — ✅ live Bounded, single-purpose helpers invoked **in-process** by a specialist (not RPC). | Sub-agent | What the user gets | |---|---| | **Web Research** | Researches companies/people/markets on the public web with cited results (Tavily; opt-in per tenant, audit-logged). | | **URL read (`web_fetch`)** | Reads a public page/document/JSON API by URL; shares the `web-research` tenant permission + query log; blocks internal/metadata addresses. | | **Documentor** | Renders any Markdown or workflow output to a downloadable DOCX / PDF / Markdown file. | | **Agent Skills** | Loads dynamic "skills" to extend capability on demand; runs scripts in a sandbox (L1 catalog → L2 file access → L3 sandboxed execution, progressive disclosure). | ## Related - [Sale Agent & CRM/ERP](/sale-agent) — the specialist that invokes these sub-agents. - [Proposal & Workflow Engine](/proposal-workflow) — the deliverables co-build operates on. - [Feature Flags](/feature-flags) — sharing/co-build flag states. - [Agent Topology](/agent-topology) — the in-process A2A invocation boundary. --- # Agent Topology > Agentee organizes agents as a **front orchestrator + domain specialists**, all running on one shared `ToolLoopAgent` runtime, one endpoint, one dispatcher. This is the load-bearing pattern for running an agent system in a production ERP. ## The shape ```mermaid flowchart TB U[User turn] --> DISP[turn-dispatcher] DISP -->|sticky session?| STICK{sticky specialist held?} STICK -->|yes| SPEC STICK -->|no| FRONT FRONT["orchestrator (FRONT)
FAST tier · routing only
tools: ask_specialist, handoff_to_agent, web_fetch"] FRONT -->|ask_specialist / handoff_to_agent| SPEC subgraph SPEC["Domain specialists"] SALE["sale
CRM/ERP · proposal · skills"] DM["delivery-manager
WBS · Jira · resourcing"] end SALE -->|in-process| SUB DM -->|in-process| SUB subgraph SUB["In-process sub-agents (AgentRegistry)"] RES["researcher
web search (Tavily)"] DOC["documentor
DOCX / PDF / MD export"] end ``` Three roles, three responsibilities: | Role | Who | Job | Model tier | |---|---|---|---| | **Front** | `orchestrator` | Route, small talk, relay `ask_specialist`. Holds only delegation tools. | **fast** (degrades to smart) | | **Specialist** | `sale`, `delivery-manager` | Own a business domain and its tools. Keep the thread while in-scope. | smart / fast per task | | **Sub-agent** | `researcher`, `documentor` | Bounded, single-purpose helpers invoked in-process by a specialist. | fast (escalates) | ## One runtime, one endpoint Every profile shares the same machinery, so adding an agent never forks the platform: - **`agent-chat.controller.ts`** — the single `POST /agents/:agentId/chat/stream` endpoint for all agents. - **`turn-dispatcher.service.ts`** — routes each turn: **sticky-session override first**, otherwise the orchestrator front resolves the target. - **`agent-turn-runner.service.ts`** — the generic `ToolLoopAgent` executor shared by all profiles (attachment reading, metering, and trace capture happen here, identically for every agent). - **`agent-profile-registry.service.ts`** — where profiles register. A new agent is one registered profile. - **`handoff-coordinator.service.ts`** — tracks per-session handoff/handback state. ## Why front + specialists A production ERP has **distinct domains** (sales vs delivery/PM) with different tools, permissions, and risk. Putting everything in one mega-agent leaks tools across domains and bloats the prompt. Agentee instead: 1. Keeps a **thin, fast front** that only routes — cheap entry latency, no domain tools to leak. 2. Gives each **specialist its own tool roster** governed by the [tool-ownership rule](/tool-ownership). 3. Lets specialists **borrow bounded reads** from peers via `ask_specialist` instead of holding foreign tools. ## Routing: sticky, handoff, handback - **Sticky session** — once a specialist owns a thread, follow-up turns stay with it (no re-routing cost). - **Handoff** (`handoff_to_agent`) — always on, no flag. Entering a peer specialist or returning to the front are both available. - **Handback** — **out-of-scope-only and deferred** (2026-08-10): a specialist keeps same-domain follow-ups and hands back only when a request leaves its scope. A handback records a *deferred skip-sticky marker* (Redis, 7-day TTL) consumed on the user's next real turn — avoiding a `specialist → front → specialist` ping-pong that cost two LLM turns per topic cycle. Full rules: **[Tool Ownership & Handoff](/tool-ownership)**. ## Sub-agents = the A2A lift point `researcher` and `documentor` are invoked **in-process** through a generic `AgentRegistry` (`registry.invoke('documentor', …)`). Each is a stateless `run(ctx, input, {signal})` — all per-call state in the closure, because the singletons are shared across concurrent tenants. That `invoke()` boundary is the deliberate **A2A (agent-to-agent) lift point**: in-process today, a network hop later behind the same shape. See [Three Core Pillars](/three-pillars). ## Current agents | Agent | Slug | Status | |---|---|---| | Sales assistant | `sale` | ✅ enabled for end users | | Delivery/PM | `delivery-manager` | 🚧 per-tenant, default OFF | | Front router | `orchestrator` | always present (routing) | | Web research | `researcher` | sub-agent / @mention | | Document export | `documentor` | sub-agent | ## Next - [Tool Ownership & Handoff](/tool-ownership) — the rule that keeps rosters clean. - [ERP MCP & Governance](/erp-mcp) — how specialists reach the ERP safely. - [Adding an Agent (Prod ERP)](/adding-an-agent) — the production playbook. --- # Tool Ownership & Handoff > **A tool belongs to the agent whose DOMAIN owns the action.** This one rule keeps agent rosters clean as the platform grows, and it is enforced by tests — not left to per-agent judgement. ## The ownership rule (2026-07-27) Written down because it never was: every new agent's roster used to be a fresh judgement call, which produced a front agent holding a delivery tool, a sales agent that could not read a ticket, and CRM-named tools doing Jira work. The rule: 1. **The FRONT (`orchestrator`) holds only** the two delegation tools (`ask_specialist`, `handoff_to_agent`) plus reads owned by no domain — `web_fetch` (the public internet is nobody's domain). Its roster is fail-closed and asserted exactly in `orchestrator.profile.spec.ts`; adding to it must be a deliberate diff. 2. **Cross-domain reads go through `ask_specialist`.** That obligation is the point: it forces each profile's `buildDelegateTools` to expose every session-*independent* read a peer might need, instead of the capability leaking onto the front. A tool qualifies only if it works off `tenantId`/`userId`/`userJwt` alone (never a session lookup) and classifies as `read` — otherwise `filterDelegateToolsForSafety` stubs it. 3. **One capability, one tool name.** When two agents need the same capability differently, inject per-agent config at build time (`export_document`'s format sets, `delegate_research`'s use-case hint). Never fork the tool, never copy it. 4. **Shared tool NAMES live in `SHARED_TOOL_NAMES`** (`agents/sale/tool-names.ts`) — preventing drift between "the tool that exists" and "the tool a stage allows". 5. **Two-layer gating always.** A tool is reachable only if it is in BOTH the `ToolSet` and `activeTools`. Tests assert both; passing one proves nothing. 6. **Sub-agent surfaces do not lend sub-agent tools.** `buildDelegateTools` is consumed by `ask_specialist` (itself a sub-agent call), so lending e.g. `delegate_research` there would nest a sub-agent inside a sub-agent and hand the front that capability by the back door. **Cost the rule accepts:** a read the front no longer holds costs two LLM calls instead of one (front → specialist → synthesise). That is the deliberate price of clean ownership. If a path proves too hot, revisit the rule explicitly — do not quietly re-add the tool to the front. ## Current rosters | Tool | orchestrator | sale | delivery-manager | |---|:---:|:---:|:---:| | `ask_specialist` | ✅ | — | — | | `handoff_to_agent` | ✅ | ✅ | ✅ | | `web_fetch` | ✅ | ✅ | ✅ | | `read_jira_issue` | via `ask_specialist` | — | **owns** | | `list_jira_projects` | — | — | **owns** | | `push_wbs_to_jira` | — | — | **owns** | | `delegate_research` | — | ✅ | ✅ | | `export_document` | — | docx/pdf/md/xlsx | docx/pdf/md/csv | | proposal + skill tools | — | **owns** | — | | workflow-drive tools | — | ✅ | ✅ | | CRM/ERP generic call | — | **owns** | — | Note how `export_document` is **one tool with per-agent format config**, not two tools — rule 3 in practice. ## Handoff vs handback ```mermaid stateDiagram-v2 [*] --> Front Front --> Sale: handoff_to_agent (sale) Front --> DM: handoff_to_agent (delivery-manager) Sale --> Sale: same-domain follow-up (sticky) Sale --> Front: handback (OUT OF SCOPE only) DM --> Front: handback (OUT OF SCOPE only) Sale --> DM: specialist → specialist handoff ``` - **Handoff is always on** — no flag gates entering a peer specialist or returning to the front (`agent_slug: "orchestrator"`). - **`handbackOnly`** is a per-tool construction option (`HandoffToolDeps`, default `false`) a profile can set to restrict its roster to the return-to-front target only — e.g. to stop a specialist pushing the user sideways into another specialist. The tool description changes with it, so the model is never offered a target the guard refuses. - **Handback is out-of-scope-only + deferred (2026-08-10).** A specialist keeps the thread for same-domain follow-ups and hands back only when a request falls outside its scope. Mechanically it records a **deferred skip-sticky marker** (`HandoffCoordinator.requestHandback`, Redis `handoff:handback:{sessionId}`, 7-day TTL, fail-soft) instead of a pending handoff. On the user's next real turn the dispatcher consumes the marker and detaches the sticky specialist so the turn falls to the front — unless a client override or pending handoff already routes it. This removed a `specialist → front → specialist` ping-pong that cost two LLM turns per topic cycle. Specialist→specialist and front→specialist handoffs are unchanged (still auto-continued). ## The front runs FAST-tier (2026-08-10) `OrchestratorProfile.resolveTurnBinding` resolves the tenant's **fast** model first, degrading to smart only when no fast binding exists; the trace tier label reflects the tier actually resolved. Routing, small talk, and relaying `ask_specialist` do not need smart-tier, so every front hop and the session's entry latency get cheaper. ## Next - [ERP MCP & Governance](/erp-mcp) — how the owned CRM tool actually reaches the ERP. - [Adding an Agent (Prod ERP)](/adding-an-agent) — applying the rule when you add a profile. --- # ERP MCP & Governance > Agents reach the **full CRM surface generically** through the Twendee ERP's stateless JSON-RPC MCP endpoint — two tools, not one-per-entity — with a **default-DENY** governor that auto-runs reads and pauses writes for approval. ## Two tool sources ```mermaid flowchart LR A[sale agent turn] --> G{tool-governance
classify by HTTP method} A -->|discover| L["list_crm_operations
(read · forces domain=crm)"] A -->|invoke| C["crm_api_call
(any CRM operationId)"] L --> ERP C --> G G -->|GET / read| AUTO[auto-run] G -->|write / unknown| APPROVE[pause → approval gate] AUTO --> ERP["ERP /api/mcp/http
Bearer user JWT"] APPROVE --> ERP A -->|templates / publish| HTTP["ErpHttpClient
listTemplates · publishProposal"] HTTP --> ERP ``` **1. ERP MCP — generic CRM access** (`agents/sale/tools/mcp/`) The agent uses **two generic tools** instead of per-entity named tools: - `list_crm_operations` — read; forces `domain:'crm'`; returns each `operationId` + HTTP method + **input schema**. - `crm_api_call` — invokes any CRM `operationId`; refuses ops outside the discovered CRM allowlist; governs by HTTP method (GET ⇒ auto, write ⇒ approval). Flow: **discover op + schema → call it.** This covers deals / leads / companies / proposals with no per-entity tools. The ERP returns `inputSchema` per op so the model fills parameters correctly. A small always-present **baseline** (`crm_list_deals`, `crm_get_deal`) stays registered — not active — so older history that referenced them still passes `validateUIMessages`. Definitions are cached per `(tenant, domain)` (5-min TTL). **2. Retained HTTP client** (`ErpHttpClient`) — proposal templates + publish (`listTemplates`, `getTemplate`, `publishProposal`), per-user JWT. Proposal-builder tools stay Agentee-local static. ## Transport note The ERP `POST {erpBaseUrl}/api/mcp/http` endpoint is a **minimal stateless JSON-RPC** handler implementing only `tools/list` + `tools/call` — it has **no MCP `initialize` handshake**, so the `@ai-sdk/mcp` Streamable-HTTP client cannot connect. Agentee speaks the endpoint directly via a tiny `ErpMcpClient` (one Bearer POST per call); the `@ai-sdk/mcp` dependency was removed. Each call is stateless — `close()` is a no-op kept for the `{ agent, close }` lifecycle. Discovery / tenant-lookup failures degrade to the baseline, never crash the turn. ## Governance: default-DENY `tool-governance.ts` is a **default-DENY verb classifier**: | Class | Verbs | Behaviour | |---|---|---| | **read** | list / get / find | auto-run | | **write / unknown** | create / update / delete / … | **pause for approval** | - `crm_api_call` decides **per operation at runtime** from the allowlist HTTP method — ERP hints can only tighten, never loosen. - Writes pause via the SDK's native `needsApproval` (static `tool()`, not `dynamicTool`), audited through `ApprovalAuditRepository`. - A small `AUTO_APPROVE_OPERATION_IDS` set covers safe enrich/dedupe/score/generate-outreach ops used by CRM-create workflows. - Typed errors: `ErpToolError` (kinds: `auth | not_found | timeout | http | network`), shared with the HTTP-client path; a backend `401` relays as `kind:'auth'` ("sign in again"). ## Per-tenant + per-user JWT - **Tenant base URL:** `tenants.erp_base_url` → `TWENDEE_ERP_BASE_URL` env → default (then `/api/mcp/http` is appended). - **Per-user JWT:** the user's ERP JWT is forwarded as `Authorization: Bearer` only (no cookie), never cached — caches hold only public tool definitions. So CRM writes run under the **user's own ERP RBAC**, not a tenant-shared token. See [Access Control](/access-control). ## Reusable across domains `ErpMcpToolRegistry.build(ctx, domain)` is **domain-parameterized** — an HR or Manager agent reuses the same registry / governance / error-mapping via its own domain prefix (`hr_*`, …) with no CRM-specific branching. This is why adding an ERP-backed agent does not mean rewriting the tool layer — see [Adding an Agent](/adding-an-agent). ## Approval gate (what the user sees) An irreversible action (e.g. create lead, publish proposal) **stops mid-turn** and asks for confirmation inline in chat, or via the `/approvals` inbox for cross-user approvals. Approval arguments always come from **server state**, never from LLM output or research results (which are quarantined). ## Next - [Access Control](/access-control) — who can reach which agent and act as whom. - [Request Lifecycle](/request-lifecycle) — where governance sits in a turn. --- # Access Control > In production, **the ERP sidebar menu is the source of truth** for who can use which agent. Access is `tenant allows the agent` **AND** `the user's ERP menu grants it`, enforced on the backend and mirrored on the frontend so displayed and enforced access can never disagree. ## Menu-driven dynamic agent access (2026-06-24) The problem it solved: agent access used to be a **frontend-only** hardcoded role allowlist (`['bod','sales','management']`) with **zero backend enforcement**. The ERP already did dynamic per-role permissioning the app never read. ```mermaid flowchart TB LOGIN["/me (login + reload)"] --> FETCH["GET {erpBaseUrl}/ui/sidebar-menu
(scoped by the user's JWT)"] FETCH --> SNAP["permission snapshot
users.erp_menu (jsonb)"] SNAP --> RESOLVE["resolveAccessibleAgents(snapshot, tenantPerms)"] RESOLVE --> FE["FE gating: canAccessChatUi(me)"] RESOLVE --> GUARD["AgentAccessGuard (backend 403)"] TENANT["tenants.agentPermissions
(default OFF)"] --> RESOLVE ``` - On `/me`, the backend fetches the ERP sidebar menu (forwarding the user's raw JWT) and parses it into a minimized **permission snapshot** persisted to `users.erp_menu`. - A declarative in-code **agent-access registry** (`agents/access/agent-access-registry.ts`) maps each agent to the ERP menu keys it requires (`sale` → any of `crmDeals | crmLeads | crmCompanies`). Adding an agent is one array entry. - The single shared resolver `resolveAccessibleAgents(snapshot, tenantPerms)` computes `accessibleAgents` for **both** `/me` (FE gating) and the backend guard. ## Tenant AND user (default-OFF) Final access = **`tenant allows the agent`** (`tenants.agentPermissions[tenantFlagKey]` truthy — note `sale` → the `'sales-agent'` flag) **AND** **`the user's menu grants it`**. `tenantAllows` is **default-OFF**, so a tenant lacking an explicit `{ "sales-agent": true }` is denied — a pre-rollout backfill is required before enabling the guard. ## Backend enforcement `AgentAccessGuard` re-derives the decision from the persisted snapshot and enforces `403` on session + sale-chat routes: - **Fetch-on-miss** — on a snapshot miss (absent OR older than `MENU_SNAPSHOT_MAX_AGE_MS`, default 24h) it does one synchronous fetch + persist, so it never depends on `/me` having run (covers API callers and the deploy window). - **Fail-closed past the ceiling** — a within-ceiling snapshot is served as-is (no lockout on a brief ERP blip); past the ceiling with a failed refresh it **fails closed**, so a revoked grant cannot outlive the ceiling. - Gated behind `AGENT_ACCESS_GUARD_ENABLED` (default OFF) for a dark rollout. **Fail-soft + loud:** any menu fetch failure returns `null` and logs a structured `erp-menu.fetch-failed` warn — `/me` serves the last-known snapshot and never 500s; a wrong-host `404` is visible in logs, never a silent empty snapshot. ## Acting as the user: per-user ERP JWT Live chat/tool calls thread the user's ERP JWT (`userErpJwt`) through `SalesIntegrationFactoryService.bundleFor(ctx)`, so CRM MCP calls run under the **user's own ERP RBAC** rather than a tenant-shared token. A rep sees only records they created/follow; a manager/BOD sees all — the ERP applies the filter, Agentee just forwards identity. See [ERP MCP & Governance](/erp-mcp). ## SSRF guard Because the user JWT is forwarded to `tenant.erpBaseUrl` on every login, the tenant write DTOs (`create-tenant`, `update-tenant-config`) validate `erpBaseUrl` via `erpBaseUrlSchema` — **https-only**, rejecting loopback / link-local / private-network hosts. > ⚠️ **Host caveat:** `/ui/sidebar-menu` is served by the ERP *work host*; the fallback default is the *data API* host. Each tenant's `erpBaseUrl` must point at the host that actually serves the menu. The menu fetch uses a dedicated short timeout `TWENDEE_MENU_TIMEOUT_MS` (default 2500ms), never the 15s data-tool timeout. ## Identity source Twendee ERP is the **single source of identity**. chat-ui delegates login to ERP via `returnUrl` redirect and validates ERP-issued JWTs (shared `JWT_SECRET`); `/me` upserts the `users` row per `(tenant_id, erp_user_id)`. Details: [Stack & Multi-Tenancy](/stack-multitenancy). ## Next - [Adding an Agent (Prod ERP)](/adding-an-agent) — wiring access for a new agent. - [Feature Flags](/feature-flags) — the guard flag and related rollout switches. --- # Adding an Agent (Production ERP) > A production playbook: how a new agent gets **organized** into Agentee — registered, given a clean tool roster, wired to the ERP, gated by access, and rolled out dark — following the platform's own conventions rather than inventing a new path. This is the practical answer to *"how do you organize agents when deploying an agent system in a production ERP?"* Each step maps to a real seam described elsewhere in this section. ## The seven steps ```mermaid flowchart LR S1[1 · Register profile] --> S2[2 · Define roster] S2 --> S3[3 · Wire ERP domain] S3 --> S4[4 · Governance] S4 --> S5[5 · Access mapping] S5 --> S6[6 · Feature-flag] S6 --> S7[7 · Assert in tests] ``` ### 1. Register the profile Add a profile to `agent-profile-registry.service.ts`. It runs on the shared `agent-turn-runner.service.ts` (`ToolLoopAgent`) and is reachable at the existing `POST /agents/:agentId/chat/stream` — **no new endpoint, no new runtime**. Decide its default model tier (the front is fast; a domain specialist is usually smart with fast escalation). ### 2. Define the tool roster (ownership rule) Apply the [tool-ownership rule](/tool-ownership): the agent owns the tools whose **domain it owns**, and nothing else. - Cross-domain reads it needs from a peer → consume them via `ask_specialist` (the peer exposes them in `buildDelegateTools`), don't copy the tool. - Reuse one capability with per-agent config (e.g. `export_document` format set), don't fork a tool. - Put shared tool names in `SHARED_TOOL_NAMES`. ### 3. Wire the ERP domain If the agent is ERP-backed, reuse `ErpMcpToolRegistry.build(ctx, domain)` with its **own domain prefix** (`hr_*`, `manager_*`, …). The generic `list_*_operations` + `*_api_call` pair and the stateless JSON-RPC transport come for free — see [ERP MCP & Governance](/erp-mcp). No per-entity tools, no CRM-specific branching. ### 4. Confirm governance The default-DENY governor already classifies by HTTP method (read ⇒ auto, write ⇒ approval). Only add operationIds to `AUTO_APPROVE_OPERATION_IDS` if they are genuinely safe and needed by a workflow. Writes must pause via the SDK's native `needsApproval` and audit through `ApprovalAuditRepository`. ### 5. Map access to the ERP menu Add one entry to `agents/access/agent-access-registry.ts` mapping the agent to the ERP sidebar menu key(s) it requires. `resolveAccessibleAgents` then gates it on both FE and backend. Remember access is **tenant-allow (default OFF) AND user-menu-grant** — see [Access Control](/access-control). Backfill `tenants.agentPermissions` before enabling the guard. ### 6. Roll out dark Ship behind a feature flag (default OFF), enable per tenant, widen gradually. `delivery-manager` is the reference example — a full second agent shipped per-tenant, default OFF. See [Feature Flags](/feature-flags). ### 7. Assert it in tests Two-layer gating means the roster is asserted in **both** the `ToolSet` and `activeTools`. The front's roster is asserted *exactly* (fail-closed) so an accidental tool leak fails CI. A profile-completeness check (`assertRegistered([...])`) runs at boot. ## Worked example — Delivery Manager The `delivery-manager` profile shows every step in production: | Step | How Delivery Manager did it | |---|---| | Register | Registered in the same runtime; drives the `wbs-generation` workflow. | | Roster | **Owns** `read_jira_issue`, `list_jira_projects`, `push_wbs_to_jira`, WBS CSV export; shares `web_fetch`, `delegate_research`, workflow-drive tools. | | ERP domain | Read-only ERP **resource gateway** for WF1 (deal ingestion / resourcing); roster→ERP push is approval-gated with a ledger (`roster_erp_push_links`). | | Governance | Writes (roster push, Jira push) approval-gated; ledger tables record pushes idempotently. | | Access | Shown only to users with the PM menu key in the ERP sidebar. | | Dark launch | Per-tenant, default OFF. | | Data-first | The WF1 graph is an **admin-authored SOP compiled into a `workflow_definitions` row**, not a `*.definition.ts`; the roster renderer locates its slice by SCHEMA, not a hardcoded node id, so it survives a recompile. | ## Sub-agent vs specialist — which to build - **Specialist** (a full agent) when it owns a business domain, a tool roster, and holds a conversation — e.g. Delivery Manager. - **Sub-agent** (in-process, via `AgentRegistry`) when it is a bounded, single-purpose helper a specialist calls and synthesises — e.g. `researcher`, `documentor`. Sub-agents are stateless (`run(ctx, input, {signal})`), have all state in the closure, and are the platform's **A2A lift point** (in-process now, network hop later behind the same `invoke()` shape). ## Non-negotiables for production ERP - **Never** put a domain tool on the front agent. - **Never** let a write skip the approval gate or run on a tenant-shared token — forward the **user's ERP JWT**. - **Never** enable a new agent tenant-wide before backfilling its tenant permission + verifying the menu-key mapping. - **Always** ship dark and assert the roster in tests. ## Next - [Feature Flags](/feature-flags) · [Request Lifecycle](/request-lifecycle) · [Environments & Deployment](/environments-deploy) --- # Request Lifecycle > End-to-end trace of one live chat turn: from a `chat-ui` SSE POST through the access guard, dispatcher, `ToolLoopAgent` turn-runner, tool calls, and streamed response, to per-turn metering and observability capture. Every live chat turn — for any agent — flows through **one endpoint**, `POST /agents/:agentId/chat/stream` (SSE). The steps below are the same for `sale`, `delivery-manager`, and the `orchestrator` front. See [Agent Topology](/agent-topology) for how the agents are organized. ## The turn, end to end ```mermaid sequenceDiagram participant U as User (chat-ui) participant C as agent-chat.controller participant G as AgentAccessGuard participant D as turn-dispatcher participant H as SaleMessageStore (DB history) participant R as agent-turn-runner (ToolLoopAgent) participant X as agents/attachments (hydrate) participant T as Tools (ERP MCP · sub-agents · workflow) participant M as turn-metering + observability U->>C: POST /agents/:agentId/chat/stream (SSE) — new turn only C->>G: check user ↔ agent access (403 if denied) G->>D: resolve target agent Note over D: sticky specialist? → hold it
else orchestrator front routes D->>H: load server-authoritative history H-->>D: full transcript (client turn merged by id) D->>R: run the turn R->>X: hydrate CURRENT-turn attachments (native vs extract-to-text) R->>R: context compaction (prompt-only) if ≥60% util loop tool loop R->>T: tool call (read → auto · write → approval) T-->>R: result (quarantined if from research) R-->>U: stream tokens + tool previews (SSE) end R-->>U: final assistant message R->>M: token usage + pipeline_events + trace capture ``` ## Step by step | # | Stage | What happens | |---|---|---| | 1 | **SSE POST** | `chat-ui` calls `POST /agents/:agentId/chat/stream`. The body carries **only the new turn** (server-authoritative history — see below), plus `FileUIPart` refs for any attachments already uploaded. | | 2 | **AgentAccessGuard** | Controller-scoped guard re-derives the access decision from the persisted ERP menu snapshot and returns **403** if the user cannot reach this agent. Gated `AGENT_ACCESS_GUARD_ENABLED`. See [Access Control](/access-control). | | 3 | **turn-dispatcher** | `turn-dispatcher.service.ts` resolves the target agent: **sticky-session override first** (a specialist that already owns the thread keeps it), otherwise the **orchestrator front** (FAST tier) routes to a specialist or answers small talk. | | 4 | **History load** | `SaleMessageStore` loads the full transcript from Postgres and merges the client turn by id (`mergeMessagesById`, part-level dedupe). Corrupt/abandoned parts are sanitized (`messageHasParts`, `dropUnresolvedToolParts`) so a bad prior turn can't brick the session. | | 5 | **turn-runner** | `agent-turn-runner.service.ts` — the generic `ToolLoopAgent` executor (Vercel AI SDK v6) shared by every profile. Attachment hydration, compaction, metering, and trace capture all live here, identically for all agents. | | 6 | **Attachment hydration** | Current-turn bytes are injected per provider capability (native vs extract-to-text — see below). Runs for **every** profile as a turn-runner step, not a profile hook. | | 7 | **Context compaction** | Prompt trimmed to the token budget when ≥60% utilized (**prompt-only, DB untouched** — see below). | | 8 | **Tool loop** | The agent calls tools: ERP MCP reads run **auto**, writes **pause for approval**; sub-agent delegation (`ask_specialist`, `delegate_research`, documentor); workflow-drive tools (`list_active_workflows` / `start_workflow`). See [ERP MCP & Governance](/erp-mcp). | | 9 | **Stream** | Tokens and tool previews stream back over SSE as they are produced. A client disconnect (`res` 'close') fires an `AbortController` that cancels in-flight sub-agent / ERP work. | | 10 | **Metering + observability** | `turn-metering.service.ts` records token usage and `pipeline_events`; `observability-capture` writes the per-turn trace to `agent_traces` / `usage_snapshots`, surfaced under admin `/traces` + `/usage`. | ## Tool calls inside the loop - **ERP MCP reads (auto):** GET-classed CRM operations (`list_crm_operations`, GET `crm_api_call`) run without a gate. The user's own ERP JWT is forwarded per call, so reads honor the user's ERP RBAC. - **ERP MCP writes (approval):** create / update / delete pause mid-turn via the SDK's native `needsApproval`. Approval args always come from **server state**, never LLM output or research. Audited through `ApprovalAuditRepository`. - **Sub-agent delegation:** `researcher` and `documentor` are invoked **in-process** through `AgentRegistry.invoke(...)` — not RPC. Research findings return **quarantined** (untrusted-data delimiters). Cross-agent reads go through `ask_specialist`. - **Workflow tools:** `start_workflow` / `list_active_workflows` (gated `WORKFLOW_AGENT_DRIVE_ENABLED`) create a `workflow_instances` row and hand off to the engine interpreter. See [Three Core Pillars](/three-pillars). ## Server-authoritative history (O(1) request body) The client POSTs **only the new turn**. `streamTurn()` loads history from the DB and merges the client turn in, so the request payload stays **O(1)** regardless of thread length. The **full transcript is always preserved in the DB** — the client is never the source of truth for history. ## Context compaction (prompt-only, DB untouched) The `agents/sale/context/` subsystem trims the **model prompt only** — never the stored transcript: - **Budget:** `contextWindow − maxOutput − systemTokens`; compaction **triggers at 60% utilization**. - **Output:** durable facts (schema-extracted) + a rolling fast-tier summary + the recent N turns. - **Injection:** at the `prepareStep` system tail — transparent to the agent. - **Tool-result eviction:** old tool-result **outputs** (not the calls) collapse to a one-line stub, keeping the last 2 verbatim and preserving call↔result pairing. The DB is untouched; a later turn can still replay the whole thread. ## Attachment hydration (native vs extract-to-text) Uploads happen **before** the chat turn (`POST /agents/sale/chat/:sessionId/attachments` → S3 + `message_attachments`, returns light refs). At turn time the shared `agents/attachments/` subsystem hydrates **only the current turn's** bytes, per provider capability. This runs as a **turn-runner step for every profile** (`sale`, `delivery-manager`, `orchestrator`), not a profile hook. | Attachment | Hydration | |---|---| | **Images** | Native AI SDK image part on vision-capable bindings; on a non-vision model, one out-of-band **vision-tier** read transcribes to labeled text (cached in `extracted_text`), degrading to an `[image attached: … not readable]` stub only on failure. | | **PDF** | Native file part **only** on `anthropic` / `google` (reliable native PDF); **extract-to-text** everywhere else (incl. OpenAI-compatible gateways). | | **DOCX / XLSX / CSV / TXT / MD** | Extract-to-text labeled part (no native file support). | **Current-turn-only cost rule:** historical attachment parts are stripped to `[attachment: ]` stubs on every turn (incl. approval-resume), so bytes are never re-sent — saving tokens and preventing drift. Extracted-text budget: **12k/turn aggregate, 4k/file max** (truncated). Native media bounded by upload limits (5 files / 10MB), not the token budget. ## Notes - **One runtime.** The turn executes on the Vercel AI SDK `ToolLoopAgent`. The explored LangGraph migration was **cancelled and removed** (ADR-001) and never enabled in production — treat older LangGraph references as historical. - **Native sub-agent path is unmetered** — a documented gap; the `AgentRegistry.invoke` boundary has no token budget yet. ## Next - [Three Core Pillars](/three-pillars) — the runtime, engine, and collaboration in depth. - [ERP MCP & Governance](/erp-mcp) — where the approval gate sits in the loop. - [Agent Topology](/agent-topology) — front + specialists + sub-agents. --- # Three Core Pillars > Agentee rests on three pillars — the **ToolLoopAgent multi-agent runtime**, the **Dynamic Workflow Engine**, and **Sharing & Collaboration** — each a distinct module group on the same NestJS backend. The explored LangGraph migration was **cancelled and removed** (ADR-001, 2026-07-26) and never ran in production. The single always-on runtime is the Vercel AI SDK v6 `ToolLoopAgent`. Treat older LangGraph references as historical. ## Pillar 1 — ToolLoopAgent Multi-Agent Runtime A per-request Vercel AI SDK v6 `ToolLoopAgent`, with one dispatcher, one endpoint, and pluggable profiles. Modules: `agents/runtime`, `agents/sale`, `agents/delivery-manager`, plus sub-agents. | Piece | File | Role | |---|---|---| | **Dispatcher** | `turn-dispatcher.service.ts` | Routes each turn: **sticky session first**, else the **orchestrator front** resolves the target. | | **Profile registry** | `agent-profile-registry.service.ts` | Holds all agent profiles. Adding an agent is **one registered profile**. | | **Turn-runner** | `agent-turn-runner.service.ts` | The generic `ToolLoopAgent` executor shared by every profile. Attachment hydration, compaction, metering, trace capture all live here. | | **Handoff coordinator** | `handoff-coordinator.service.ts` | Tracks per-session handoff / handback (always on, no flag). | | **Endpoint** | `agent-chat.controller.ts` | The single `POST /agents/:agentId/chat/stream` (SSE) for all agents. | **Profiles:** a thin FAST-tier `orchestrator` front (routing + small talk only, holds only delegation tools), and domain specialists `sale` (CRM / ERP / proposal / skills) and `delivery-manager` (WBS / Jira / resourcing). Specialists own their tool rosters under the [tool-ownership rule](/tool-ownership) and borrow bounded peer reads via `ask_specialist`. **Sub-agents = the A2A lift point.** `researcher` (bounded web search over Tavily, opt-in per tenant) and `documentor` (DOCX / PDF / MD export) are invoked **in-process** through a generic `AgentRegistry` (`registry.invoke('documentor', …)`) — **not RPC**. Each is a stateless `run(ctx, input, {signal})` (all per-call state in the closure; singletons are shared across concurrent tenants). That `invoke()` boundary is the deliberate **A2A (agent-to-agent) lift point**: in-process today, a network hop later behind the same shape. See [Agent Topology](/agent-topology) for the full shape and [Request Lifecycle](/request-lifecycle) for a turn trace. ## Pillar 2 — Dynamic Workflow Engine `workflow/engine/workflow-engine.service.ts` — a **Postgres-backed state-machine interpreter** (gated `WORKFLOW_ENGINE_ENABLED`) that drives admin-authored workflows node by node. **SOP compiler:** workflows are authored as **SOP markdown** and compiled to a JSON graph by `workflow/compiler/workflow-compiler.service.ts` (LLM-assisted + cite-back), persisted as a `workflow_definitions` row. A workflow is **data**, not a `*.definition.ts` file. **4 node types:** | Node | Executor | Behaviour | |---|---|---| | `ai_chat` | inline LLM | conversational step in the turn | | `form` | JSON schema | structured user input | | `approval` | cross-user task queue | routes to another user's `/approvals` inbox | | `automation` | BullMQ | out-of-band background job | **Output surface:** results land in the generic `workflow_outputs` store (`workflow/output/`), rendered per-shape (`proposal-section-renderers.ts`, `wbs-section.renderer.ts`, `generic-section.renderer.ts`) and exportable to **PDF / DOCX / Markdown**. Agents discover and drive workflows via `list_active_workflows` / `start_workflow`, gated `WORKFLOW_AGENT_DRIVE_ENABLED`. **Co-build (collaborative editing):** staged-commit on `workflow_outputs` — **fork → stage edits (`staged_decisions` jsonb) → merge**, gated `OUTPUT_COLLAB_ENABLED`. **Proposal migration (in progress):** the proposal flow is mid-migration from legacy `sales/proposal-builder/` onto this engine, controlled by the 3-state flag `PROPOSAL_ON_ENGINE` (off / canary / on) with `FORCE_WORKFLOW_ONLY` as the eventual kill-switch. **The tenant-wide cutover is not yet flipped** — the two paths currently coexist. See [Feature Flags](/feature-flags). ## Pillar 3 — Sharing & Collaboration Modules `sharing/` + `mentions/`, plus workflow-output co-build. - **Shared sessions** (`sharing/shared-sessions.service.ts`, gated `SHARE_SESSION_ENABLED`) — read-only access to a session + its live workflow output via a token-protected link (tables `shared_sessions`, `shared_session_opens`). A direct/no-notification grant variant is used internally by co-build. - **@Mentions** (`mentions/mentions.service.ts`, gated `PROPOSAL_ASSIGN_ENABLED`) — resolves the @-mention directory (agents + accessible ERP users); backs both hand-off requests and workflow-output collaboration invites. - **Co-build** — the fork-edit-push-merge flow over `workflow_outputs` (shared with Pillar 2's `OUTPUT_COLLAB_ENABLED`), letting multiple users collaborate on one output. ## How the pillars connect ```mermaid flowchart LR P1["Pillar 1
Agent Runtime"] -->|start_workflow / list_active_workflows| P2["Pillar 2
Workflow Engine"] P2 -->|workflow_outputs| P3["Pillar 3
Sharing & Collaboration"] P3 -->|@mention / share link| P1 P2 -->|export| OUT["PDF / DOCX / MD"] ``` The agent runtime **starts** workflows; the engine **produces** outputs; sharing **distributes and co-builds** them, and @mentions route work back to agents and users. ## Next - [Agent Topology](/agent-topology) — front + specialists + sub-agents. - [Feature Flags](/feature-flags) — the gates named above (`WORKFLOW_ENGINE_ENABLED`, `PROPOSAL_ON_ENGINE`, `SHARE_SESSION_ENABLED`, …). - [Request Lifecycle](/request-lifecycle) — one turn end to end. --- # Stack & Multi-Tenancy > Agentee is a NestJS modular monolith on PostgreSQL 16 + Redis 7, integrated into Twendee ERP. Tenant isolation is enforced today by **explicit `tenant_id` / `userId` WHERE clauses** — not row-level security — with AES-256-GCM encryption at rest, a dual JWT auth path, and ERP as the single source of identity. ## Tech stack | Layer | Tech | |---|---| | **Backend** | NestJS (modular monolith), TypeScript, Node 22 | | **Runtime** | Vercel AI SDK v6 `ToolLoopAgent` (single always-on path) | | **Database** | PostgreSQL 16 via Drizzle ORM | | **Cache / jobs** | Redis 7 (ioredis + BullMQ) | | **Frontend** | React + Vite SPA (`chat-ui`), shadcn/ui, react-i18next (EN / VI) | | **Monorepo** | Turborepo + pnpm (5 workspace packages) | | **ERP integration** | Twendee ERP over stateless JSON-RPC MCP + retained HTTP client | | **LLM providers** | Anthropic, OpenAI, Google, DeepSeek, GLM, Groq, Ollama (per-tenant, tier-aware) | ## Multi-tenant isolation (important nuance) The **real isolation boundary today is explicit `tenant_id` / `userId` WHERE clauses** in each service. The path is: **JWT carries `tenantId` → guards extract it → every tenant-owned query filters `WHERE tenant_id = …`**. > **RLS is largely dormant.** PostgreSQL row-level-security policies **exist on many tables**, but they are **not the active enforcement at runtime**: the DB owner role bypasses RLS, and `set_config` for `app.current_tenant_id` is only set inside the **legacy auth transaction**. Do **not** rely on RLS as the live guard. **Every new query — including pgvector / RAG reads — must carry the `tenant_id` / `userId` filter explicitly.** `tenants` is the one system-level table; everything else is tenant-scoped by filter. ## Encryption at rest AES-256-GCM via `AesEncryptionService`, keyed by the **`ENCRYPTION_KEY` env var**, protects: - `tenants.llmApiKey`, `tenants.erp_auth_config` (`{username, password, baseUrl?}`) - `tenant_llm_providers.apiKey` - episodic memory content and other credential / PII fields > **Known gap:** the key is a **static `ENCRYPTION_KEY`** — no managed KMS / key rotation yet. ## Auth: dual JWT path + refresh rotation Two JWT paths coexist (convergence not yet scheduled — a known gap): 1. **Legacy Passport JWT** (HS256). 2. **Platform path** — `JWT_SECRET_CURRENT` / `JWT_SECRET_PREVIOUS` rotation + **Redis replay protection**. **Refresh rotation:** opaque `.`, stored as a **bcrypt hash**, validated and marked revoked on rotate; `POST /auth/refresh` issues a new pair. ## Identity bridge: ERP ↔ chat-ui **Twendee ERP is the single source of identity.** `chat-ui` delegates all login to ERP (via `returnUrl` redirect) and validates ERP-issued JWTs with the shared `JWT_SECRET`. | Aspect | Behaviour | |---|---| | **Token source** | `JwtStrategy` accepts the JWT from the **`twd_auth` cookie** (primary chat-ui path, set by ERP on the shared parent domain) **or** an `Authorization: Bearer` header (programmatic clients). | | **Token shape** | HS256; CUID `userId` as `sub`/`userId`; ad-hoc role strings (`'ADMIN'`, …); optional `tenantId` override. | | **User materialization** | On every `/me` call, `MeService.getMe(jwt)` **upserts the `users` row by `(tenant_id, erp_user_id)`** (partial unique index `uq_users_tenant_erp_user`), keeping email / role / permissions in sync with ERP each request. Falls back to `ERP_DEFAULT_TENANT_ID` when the JWT lacks `tenantId` (single-tenant mode). | | **Role mapping** | `mapErpRole()` normalizes ERP strings (`'ADMIN'` → `'ceo'`, `'MANAGER'` → `'manager'`, …) to an internal `UserRole`; **unknown roles default to `'guest'`** (least privilege). Capabilities are role-derived (`ROLE_CAPABILITIES`); the backend is source-of-truth on every endpoint, FE `useCapability()` gates UX only. | Role propagation follows JWT TTL (no live revocation); the FE `useMe` query has a 1-min staleTime, so a role change is picked up within ~15m (JWT TTL + cache). ## Per-user ERP JWT propagation Live chat / tool calls thread the user's own ERP JWT (`userErpJwt`) through `SalesIntegrationFactoryService.bundleFor(ctx)`, so CRM MCP calls run under the **user's own ERP RBAC**, not a tenant-shared token. The JWT is forwarded as `Authorization: Bearer` only, **never cached** (caches hold only public tool definitions). `AgentAccessGuard` gates which agents a user can reach. See [ERP MCP & Governance](/erp-mcp) and [Access Control](/access-control). The tenant-shared ERP service token is separate: `TwendeeErpAuthService` caches a per-tenant `accessToken` (until `exp − 60s`; a 401 mid-request invalidates and retries once; concurrent callers share one in-flight login). ## Data model (table groups) Tenant-scoped by explicit `tenant_id` / `userId` filters; `tenants` is system-level. Model capabilities + pricing are **not tables** — they live in code (`llm/models/model-registry.ts`). | Group | Tables | |---|---| | **Core** | `users`, `refresh_tokens` | | **Chat** | `chat_sessions`, `messages`, `message_attachments`, `attachments` | | **Legacy proposal-builder** | `proposal_workspaces`, `proposal_versions`, `proposal_sections`, `proposal_revisions`, `proposal_exports`, `proposal_render_jobs`, `proposal_collaboration`, `proposal_workflow_steps`, `document_templates` | | **Dynamic workflow engine** | `workflow_definitions`, `workflow_instances`, `workflow_outputs`, `workflow_tasks`, `workflow_step_events`, `workflow_instance_forks`, `workflow_fork_invitations` | | **Delivery Manager** | `wbs_jira_push_links` | | **Sharing** | `shared_sessions`, `shared_session_opens` | | **Multi-tenant LLM** | `tenant_llm_providers` | | **Memory** | `user_memories` (pgvector-backed episodic memory) | | **Notifications** | `notifications` | | **Observability** | `agent_traces`, `usage_snapshots`, `pipeline_events`, `audit_logs`, `request_payloads` | | **ERP integration** | `erp_mcp_operations` | | **Legacy / still-present** | `platform_channel_links`, `approvals`, `research_audit`, `sale_agent_approval_audit` | > Telegram is **not an active channel**: `platform_channel_links` and related fields remain in schema, but the product decision to drop Telegram is made and removal is a pending follow-up. ## Known gaps - **Static `ENCRYPTION_KEY`** — no KMS. - **Dual auth path** — legacy Passport + platform rotation not yet converged. - **Proposal-on-Engine cutover** — not flipped tenant-wide (`PROPOSAL_ON_ENGINE` / `FORCE_WORKFLOW_ONLY`). ## Next - [Access Control](/access-control) — agent access + who you act as. - [Environments & Deployment](/environments-deploy) — where this runs. - [Feature Flags](/feature-flags) — the gates behind coexisting paths. --- # Environments & Deployment > How the **Agentee app itself** (`apps/api` + `apps/chat-ui`) ships: three branch-driven environments, a Docker Compose stack of `redis + api-1 + api-2 + chat-ui` (ERP Postgres on the host), nginx vhosts, and a GHCR + GitHub Actions build → push → SSH → migrate → rolling-restart flow. This page covers deployment of the **Agentee application**, not the docs site (the docs site deploys separately to Cloudflare Pages). ## Environments | Branch | Env | chat-ui domain | API serving | Deploy workflow | |---|---|---|---|---| | `main` | **dev** | `agentee-dev.twendeesoft.com` | ERP path-mount: `dev-work.twendeesoft.com/agentee-chat/*` (`nginx/twd-api.conf`) | `deploy-dev.yml` (on push) | | `staging` | **staging** | `app.agentee.vn` | same domain, `/api/v1` (`nginx/twd-app.conf`) | `deploy-staging.yml` (on push) | | `production` | **production** | `agentee.twendeesoft.com` | same domain, `/api/v1` (`nginx/twd-app.conf`) | `deploy-production.yml` (**manual only**) | - **dev** keeps the legacy ERP **path-mount** vhost (API mounted under the ERP `dev-work` host at `/agentee-chat/`). - **staging / production** use the **same-domain** `nginx/twd-app.conf` vhost (API + SPA on one host, API under `/api/v1`). - `API_URL` must include the `/api/v1` suffix (NestJS `setGlobalPrefix('api/v1')`); `ERP_URL` is used by chat-ui to redirect to ERP login on 401. ## Docker Compose topology ```mermaid flowchart TB subgraph HOST["VPS host"] direction TB NGINX["nginx (TLS · Let's Encrypt)"] subgraph NET["twd-net (bridge network)"] REDIS[("redis:7
healthcheck")] API1["api-1 :3001"] API2["api-2 :3002"] UI["chat-ui :8080"] MIG["api-migrate
(one-shot · profile=migrate)"] end PG[("ERP postgres :5432
db twd_agents_hub · role twd
ERP-team owned")] end NGINX -->|/agentee-chat/ or /api/v1| API1 NGINX -->|upstream| API2 NGINX -->|SPA| UI REDIS -.depends_on healthy.-> API1 REDIS -.depends_on healthy.-> API2 API1 -->|host.docker.internal:5432| PG API2 -->|host.docker.internal:5432| PG MIG -->|host.docker.internal:5432| PG ``` - **`redis:7`** — in-network, health-gated; `api-1` / `api-2` `depends_on` it healthy. - **`api-1` + `api-2`** — two API replicas behind an nginx upstream (`:3001`, `:3002`) for rolling restart. - **`chat-ui`** — static SPA container (`:8080`), no deps. - **`api-migrate`** — one-shot migration container (compose profile `migrate`), run once per deploy. - **ERP Postgres** lives **on the host at `:5432`** (owned + operated by the ERP team), **outside** `twd-net`. API containers reach it via **`host.docker.internal:5432`** (`extra_hosts: host-gateway` in `docker-compose.prod.yml`). The in-stack Postgres was dropped in the 2026-05 cutover — only `redis` + APIs + `chat-ui` remain. ## nginx vhosts | vhost file | Env | Serves | |---|---|---| | `nginx/twd-chat-ui.conf` | dev | chat-ui SPA on its own domain → `127.0.0.1:8080` | | `nginx/twd-api.conf` | dev | API path-mount block copied into the existing `dev-work` ERP vhost: `upstream { :3001; :3002; }` + `location /agentee-chat/` placed **before** the ERP catch-all `location /`. | | `nginx/twd-app.conf` | staging / production | same-domain vhost — SPA + API `/api/v1` on one host. | ## Deploy flow (GHCR + GitHub Actions) ```mermaid flowchart LR PUSH["push to branch"] --> BUILD["build twd-api ·
twd-chat-ui · twd-api-migrate
(parallel)"] BUILD --> PUSHIMG["push images → GHCR"] PUSHIMG --> SSH["SSH server ·
sync compose + deploy.sh ·
write .env"] SSH --> MIGRATE["api-migrate one-shot"] MIGRATE --> UIDEP["deploy chat-ui"] UIDEP --> R1["rolling-restart api-1
(wait /health)"] R1 --> R2["rolling-restart api-2
(wait /health)"] R2 --> SMOKE["smoke test /health"] ``` Each workflow (`scripts/deploy.sh` on the server): 1. Build `twd-api`, `twd-chat-ui`, `twd-api-migrate` images in parallel → push to GHCR. 2. SSH the server; sync `docker-compose.yml`, `docker-compose.prod.yml`, `scripts/deploy.sh`. 3. Write `/opt/twd-agents-hub/.env` from `ENV_FILE_CONTENT`, appending pinned `API_IMAGE` / `CHAT_UI_IMAGE` / `MIGRATE_IMAGE` tags. 4. Pull images → run **`api-migrate` one-shot** → deploy chat-ui → **rolling-restart `api-1` then `api-2`** (wait for `/health` between replicas). 5. Final **smoke test** against `…/health`. Total ~5 min. GHCR pull uses a one-time `docker login` with a `read:packages` PAT. Model capabilities + pricing live in code (`llm/models/model-registry.ts`) — adding a model is a code edit + redeploy, **no seed step**. ## Rollback - GitHub → Actions → the env's Deploy workflow → **Run workflow** → set `image_tag` to a previous short SHA (e.g. `sha-abc1234`). Build jobs skip (image already on GHCR); the deploy job pulls the older tag and rolls. **< 90 s.** - Migrations are mostly **forward-only** — changing image tags does **not** rewind data state. For a migration rollback see the database migration guide. ## On-call cheatsheet ```bash ssh deploy@ cd /opt/twd-agents-hub # Status + logs for both replicas docker compose -f docker-compose.yml -f docker-compose.prod.yml ps docker compose -f docker-compose.yml -f docker-compose.prod.yml logs -f api-1 api-2 # psql against the ERP-shared host postgres (this stack ships no DB) # Password: /opt/twd-agents-hub/.env → DATABASE_URL PGPASSWORD= psql -h 127.0.0.1 -p 5432 -U twd -d twd_agents_hub # Restart one replica (zero-downtime if the other stays healthy) docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --no-deps --force-recreate api-1 ``` | Symptom | First check | |---|---| | **502 from API** | `docker compose ps` — both replicas up? `logs --tail=200 api-1 api-2` | | **Blank chat-ui** | DevTools network — do `/assets/*.js` load? CSP/CORS in console? `API_URL` var set? | | **Migration failed** | GH Actions `api-migrate` step; on server `docker compose logs api-migrate` | | **Cert renewal fail** | `sudo certbot renew --dry-run` (DNS or HTTP-01 reach) | | **ERP regressed** | `sudo nginx -t` (vhost typo?); `sudo tail /var/log/nginx/error.log` | | **Disk full** | `docker image prune -af --filter "until=720h"` | **Validation:** `bash scripts/smoke-test.sh` after a deploy (target 100% pass); `DURATION=120 bash scripts/zero-downtime-probe.sh` during a deploy (target 0 fail). ## Next - [Stack & Multi-Tenancy](/stack-multitenancy) — what runs inside these containers. - [Feature Flags](/feature-flags) — env-driven gates per environment. --- # Feature Flags > New capabilities **ship "dark"** — coded, merged, and shipped **OFF by default**, then enabled per tenant. This table is the canonical list of the platform's flags: what each controls, its default, and its rollout group. ## The dark-launch model Agentee ships risky or in-progress features behind flags so code can land on `main` without exposing it: - **Ship OFF, enable per tenant.** A new feature merges with its flag `OFF` (or `off`), stays invisible in production, and is turned on one tenant at a time once verified. - **Backend flag + `VITE_` mirror.** UI-visible features carry a paired frontend flag (`VITE_*`) so the client only renders what the backend will honor — displayed and enforced state can't disagree. - **Kill-switch semantics.** A top-level engine flag (`WORKFLOW_ENGINE_ENABLED`) freezes both new starts and in-flight runs when flipped off — not just a UI hide. - **3-state where a canary is needed.** `PROPOSAL_ON_ENGINE` is `off` / `canary` / `on`, not a boolean, so the legacy path can coexist during migration. ## Which agents are live Access is gated separately from features — see [Access Control](/access-control). Agent availability today: | Agent | Status | |---|---| | `sale` (sales assistant) | ✅ enabled for end users | | `delivery-manager` (PM/DM) | 🚧 per-tenant, **default OFF** | | `orchestrator` (front router) | always present (routing only) | | `researcher` | sub-agent / `@mention` | | `documentor` | sub-agent | ## Flags by area ### Workflow engine | Flag | Default | Controls | |---|---|---| | `WORKFLOW_ENGINE_ENABLED` | OFF | Master kill-switch for the [Dynamic Workflow Engine](/proposal-workflow); freezes running + new instances when off | | `WORKFLOW_AGENT_DRIVE_ENABLED` | OFF | Agent-facing workflow tools (`list_active_workflows` / `start_workflow`) — lets an agent discover & drive workflows in chat | | `WORKFLOW_OUTPUT_ENABLED` · `VITE_WORKFLOW_OUTPUT_ENABLED` | OFF | Right-column workflow-output document panel (backend + UI mirror) | | `WORKFLOW_DRIVE_UPLOAD_ENABLED` | OFF | Upload rendered workflow output to Google Drive and return the link | | `FORCE_WORKFLOW_ONLY` | OFF | Force every deliverable (proposal / report / quote) through an admin-defined workflow; eventual kill-switch for the legacy proposal path | ### Proposal-on-engine | Flag | Default | Controls | |---|---|---| | `PROPOSAL_ON_ENGINE` | `off` / `canary` / `on` | Runs the proposal flow on the workflow engine instead of the legacy `sales/proposal-builder/` module; 3-state canary rollout | `FORCE_WORKFLOW_ONLY` (above) is the eventual kill-switch that retires the legacy proposal path once `PROPOSAL_ON_ENGINE=on` is safe tenant-wide. ### Collaboration / co-build | Flag | Default | Controls | |---|---|---| | `OUTPUT_COLLAB_ENABLED` · `VITE_OUTPUT_COLLAB_ENABLED` | OFF | [Co-build](/collaboration-llm) on workflow outputs: fork → edit → push → merge → pull (staged-commit) | | `COBUILD_MERGE_CURSOR_SYNC` | ON | Re-sync the workflow cursor after a merge so the run resumes at the right step | | `PROPOSAL_ASSIGN_ENABLED` | ON | Proposal hand-off + `@mention` directory (agents + accessible ERP users) | ### Sharing | Flag | Default | Controls | |---|---|---| | `SHARE_SESSION_ENABLED` | OFF | Read-only session share-by-link (`/share/:id`, `shared_sessions`) | ### Access guard | Flag | Default | Controls | |---|---|---| | `AGENT_ACCESS_GUARD_ENABLED` | OFF | Backend `AgentAccessGuard` — enforces menu-driven per-agent `403` on session + chat routes (dark rollout; requires a tenant-permission backfill first) | | `VITE_LEGACY_ROLE_GATE` | — | Frontend fallback to the old hardcoded role allowlist | ### Metering | Flag | Default | Controls | |---|---|---| | `MAIN_LOOP_METERING_ENABLED` | ON | Per-turn token + cost metering (`pipeline_events`) | ### Misc | Flag | Default | Controls | |---|---|---| | `RESPONSE_LANGUAGE_SYNC_ENABLED` | ON | Detect the user's language and answer in it | | `VITE_CHANGELOG_ENABLED` | — | Product changelog page in the UI | ## Notes - `VITE_`-prefixed flags are **frontend mirrors** of a backend flag — set both consistently or the UI and backend disagree. - A blank default (`—`) means the flag has no committed platform-wide default; it is set per environment / tenant. - Flags gating **Pillar 2/3** work (workflow engine, co-build, sharing) are the ones most commonly OFF in production today. ## Next - [Repository Structure](/repo-structure) — where the flag-gated modules live. - [Access Control](/access-control) — how `AGENT_ACCESS_GUARD_ENABLED` decides agent reachability. - [Feature Overview](/feature-overview) — what each gated capability does for the user. --- # Repository Structure > Agentee is a **Turborepo + pnpm monorepo**: one NestJS backend (`apps/api`), one React/Vite chat frontend (`apps/chat-ui`), this docs SPA (`apps/agentee-docs`), plus shared `packages/*` and `tooling/*`. The backend is a **modular monolith** — one process, many NestJS modules. ## Directory tree ``` TWDAgentsHub/ ├── apps/ │ ├── api/ @twd/api NestJS modular monolith (backend) │ ├── chat-ui/ @twd/chat-ui Sales Agent v2 frontend (Vite + React + shadcn) │ └── agentee-docs/ static docs SPA (renders these Markdown pages) ├── packages/ │ ├── shared-types/ @twd/shared-types Cross-app TS contracts (UnifiedMessage, AgentTask…) │ ├── ui-kit/ @twd/ui-kit shadcn/Radix component library │ └── web-core/ @twd/web-core axios / auth / query client for chat-ui ├── tooling/ │ ├── tsconfig/ @twd/tsconfig Shared TS base / nestjs / react configs │ └── tailwind-config/ @twd/tailwind-config Shared Tailwind preset ├── docs/ Architecture, codebase summary, team guides ├── plans/ Phase plans, validation logs, research, reports ├── docker-compose.yml postgres:16 + redis:7 (dockerized app — production) ├── docker-compose.dev.yml dev override (host runs API with `pnpm start:dev`) ├── Dockerfile multi-stage Node alpine, runs as non-root ├── turbo.json build/dev/test pipelines (shared-types → apps order) ├── pnpm-workspace.yaml pnpm workspace manifest └── .env.example full list of required env vars ``` ## Workspaces | Workspace | Package | Role | |---|---|---| | `apps/api` | `@twd/api` | Backend — all agent runtime, workflow engine, ERP integration, auth | | `apps/chat-ui` | `@twd/chat-ui` | Live chat UI (`features/sale-agent`), workflow-output panel, proposal panel, admin observability | | `apps/agentee-docs` | — | This static documentation site | | `packages/shared-types` | `@twd/shared-types` | TS contracts shared across apps (`UnifiedMessage`, `AgentTask`, `ERPRequest`…) | | `packages/ui-kit` | `@twd/ui-kit` | shadcn/Radix component library | | `packages/web-core` | `@twd/web-core` | axios/auth/react-query client the chat-ui builds on | | `tooling/tsconfig` | `@twd/tsconfig` | Shared `tsconfig` bases (base / nestjs / react) | | `tooling/tailwind-config` | `@twd/tailwind-config` | Shared Tailwind preset | ## Backend module map (`apps/api/src/`) The backend is organized as NestJS modules by the [three pillars](/three-pillars). Read the module code for full detail. ### Pillar 1 — agent runtime | Module | Responsibility | |---|---| | `agents/runtime/` | The single always-on runtime path: `turn-dispatcher` (sticky session → orchestrator front → target agent), `agent-profile-registry` (all profiles), `agent-turn-runner` (generic ToolLoopAgent executor), `handoff-coordinator`, unified `POST /agents/:agentId/chat/stream` controller | | `agents/sale/` | Sale Agent — AI SDK v6 ToolLoopAgent, 40+ tool ToolSet (proposal / CRM / subagent delegation), dynamic system prompt, per-turn `prepare-step` context binding | | `agents/delivery-manager/` | PM/DM profile — drives the WBS-generation workflow, WBS CSV export, WBS→Jira push (ledger `wbs_jira_push_links`) | | `agents/documentor/` | In-process sub-agent — renders Markdown / model output to DOCX / PDF / Markdown | | `agents/researcher/` | In-process sub-agent — bounded, opt-in web search (Tavily) | | `agents/skills/` | Constrained Anthropic Agent Skills runtime (L1 catalog → L2 file access → L3 sandboxed execution) | ### Pillar 2 — workflow engine | Module | Responsibility | |---|---| | `workflow/` | Gated `WORKFLOW_ENGINE_ENABLED`. `engine/` (PG state-machine interpreter + reconciler), `compiler/` (SOP markdown → JSON graph via LLM + cite-back), `executors/` (4 node types), `output/` (`workflow_outputs` store, renderers, export, co-build), `runtime/` (agent discovery/start tools), `approval/` (cross-user task queue), `catalog/` (definition seeders), `jobs/` (BullMQ automation processor) | ### Pillar 3 — sharing & collaboration | Module | Responsibility | |---|---| | `sharing/` | Gated `SHARE_SESSION_ENABLED` — read-only session share-by-link (`shared_sessions`) | | `mentions/` | `@`-user/agent mention directory (agents + accessible ERP users); gated `PROPOSAL_ASSIGN_ENABLED`; backs hand-off + co-build invites | ### Support services | Module | Responsibility | |---|---| | `integrations/erp/` | Twendee ERP REST adapter + canonical types; per-tenant adapter wiring; CRM reached generically via the stateless JSON-RPC MCP endpoint — see [ERP MCP](/erp-mcp) | | `llm/` | Multi-provider factory + `LlmService`, tier-aware (smart / fast / vision) resolution, `generateObject` wrapper | | `auth/` | Dual JWT path (legacy Passport JWT + platform `JWT_SECRET_CURRENT/PREVIOUS` rotation with Redis replay protection); refresh tokens; ERP user-bridge | | `chat/` | Web chat surface — sessions, history, resume detection, attachments (vision + extraction), auto-titles, follow-up suggestions | | `me/` | `/me` profile + preferences; episodic memory (pgvector-backed `user_memories`, async BullMQ writes) | | `notifications/` | Inbox CRUD + in-memory SSE gateway; proposal-assignment + workflow-task subscribers | | `observability/` + `observability-capture/` | Per-turn agent trace capture → admin `/traces` + `/usage` read surfaces (gated `EXECUTIVE_ADMIN_EMAILS`) | | `tenants/` | Tenant CRUD + encrypted config, multi-tenant context | | `tenant-llm-providers/` | Per-tenant LLM provider CRUD + credential resolution (BYO key) | ## Daily commands | Command | Effect | |---|---| | `pnpm install` | Install all workspace deps (uses `pnpm-lock.yaml`) | | `pnpm build` | `turbo run build` — builds `shared-types` then every app | | `pnpm dev` | `turbo run dev` — runs every app's dev server in parallel | | `pnpm test` | `turbo run test` | | `pnpm typecheck` | `turbo run typecheck` | | `pnpm docker:up` | Boots postgres + redis only (dev mode, host runs API) | | `pnpm docker:down` | Stops the compose stack | | `pnpm --filter @twd/api start:dev` | Watch-mode API | | `pnpm --filter @twd/api build` | Build only the API | | `docker compose --profile full up -d` | Boot the FULL stack including the dockerized API | **Prereqs:** Node 22.12.0 (`nvm use`), pnpm 10.29+, Docker Desktop. Turbo enforces the `shared-types → apps` build order. ## Conventions - **File names:** kebab-case for `.ts`/`.js`/`.py`/`.sh` with long descriptive names. - **File size:** keep files under ~200 LOC; split into focused modules when they grow. - **Commits:** conventional (`feat:`, `fix:`, `refactor:`, `test:`…), no AI references. - **Branches:** `main` protected; feature work on `feat/-slug` branches, PR into `main`. ## Next - [Feature Flags](/feature-flags) — which modules are dark-launched. - [Architecture Overview](/architecture-overview) — how the pillars fit together. - [Adding an Agent (Prod ERP)](/adding-an-agent) — where a new profile plugs in. --- # Glossary > The platform's vocabulary in one place — agent-topology terms, ERP-governance terms, workflow terms, and platform terms. Definitions are short; each links to the page that covers it in depth. ## A **A2A lift point** — The `AgentRegistry.invoke()` boundary through which specialists call sub-agents. In-process today, it is the deliberate seam where an agent-to-agent network hop can be added later behind the same shape. See [Agent Topology](/agent-topology). **Agent Skills** — A constrained Anthropic Agent Skills runtime (`agents/skills/`) letting an agent dynamically load a "skill" to extend its capability, with progressive disclosure (L1 catalog → L2 file access → L3 sandboxed script execution). **Agentee** — Product name for **TWDAgentsHub**: a multi-tenant Sales AI SaaS built on the Twendee ERP, whose core is a conversational Sale Agent in a web chat. See [What is Agentee](/what-is-agentee). **AgentAccessGuard** — Backend guard that enforces per-agent `403`s on session + chat routes, deriving the decision from the user's persisted ERP menu snapshot. Gated by `AGENT_ACCESS_GUARD_ENABLED`. See [Access Control](/access-control). **AgentRegistry** — The generic in-process registry that holds sub-agents (`researcher`, `documentor`) as stateless `run(ctx, input, {signal})` singletons and invokes them via `registry.invoke('documentor', …)`. **approval gate** — The pause where an irreversible action (create lead, publish proposal) stops mid-turn and asks for confirmation — inline in chat, or via the `/approvals` inbox for cross-user approvals. Approval arguments always come from server state, never LLM output. See [ERP MCP](/erp-mcp). **ask_specialist** — The delegation tool the front uses to borrow a bounded, session-independent **read** from a peer specialist instead of holding that foreign tool itself. See [Tool Ownership](/tool-ownership). ## B **BYO key** — "Bring your own key." Each tenant configures its own LLM provider(s) and API key; credentials are stored AES-encrypted. See [Stack & Multi-Tenancy](/stack-multitenancy). ## C **co-build** — Staged-commit collaboration on a workflow output: an owner invites a collaborator to **fork → edit → push → merge → pull**, reviewing changes section-by-section. Gated by `OUTPUT_COLLAB_ENABLED`. See [Collaboration & LLM](/collaboration-llm). **context compaction** — Automatic summarization of long conversation history so a session's token use stays under budget without losing thread. **crm_api_call** — The generic ERP tool that invokes any CRM `operationId` discovered via `list_crm_operations`, refusing ops outside the allowlist and governing by HTTP method (GET auto-runs, writes pause for approval). See [ERP MCP](/erp-mcp). ## D **dark launch / feature flag** — Shipping a feature coded but **OFF by default**, then enabling it per tenant. Controlled by a named flag. See [Feature Flags](/feature-flags). **default-DENY governance** — The `tool-governance.ts` verb classifier: reads (list/get/find) auto-run; writes and unknown verbs pause for approval. ERP hints can only tighten, never loosen. See [ERP MCP](/erp-mcp). **Delivery Manager** — The second end-user agent (`delivery-manager`), a PM-focused profile that decomposes a Statement of Work into a WBS via the WBS-generation workflow. Per-tenant, default OFF. See [Agent Topology](/agent-topology). **Documentor** — In-process sub-agent that renders arbitrary Markdown or workflow output into a downloadable DOCX / PDF / Markdown file. **Dynamic Workflow Engine** — Pillar 2: a PG + BullMQ state-machine that runs admin-authored business processes as graphs, gated by `WORKFLOW_ENGINE_ENABLED`. See [Proposal Workflow](/proposal-workflow). ## E **episodic memory** — Per-user long-term memory: "facts" / preferences learned across sessions, stored in pgvector-backed `user_memories`, viewable and editable by the user. **ERP MCP** — The stateless JSON-RPC endpoint (`POST {erpBaseUrl}/api/mcp/http`, `tools/list` + `tools/call` only) through which agents reach the full CRM surface generically. Agentee speaks it directly via a tiny `ErpMcpClient` (no `initialize` handshake). See [ERP MCP](/erp-mcp). ## H **handback** — A specialist returning the thread to the front, **out-of-scope-only and deferred**: it records a skip-sticky marker (Redis, 7-day TTL) consumed on the user's next turn, avoiding a specialist→front→specialist ping-pong. See [Tool Ownership](/tool-ownership). **handoff** — Routing a turn from the front to a specialist, or between specialists (`handoff_to_agent`). Always on, no flag. See [Agent Topology](/agent-topology). ## I **identity bridge** — Twendee ERP is the single source of identity; chat-ui delegates login to ERP via `returnUrl` redirect and validates ERP-issued JWTs against a shared `JWT_SECRET`, upserting the `users` row per `(tenant_id, erp_user_id)`. See [Stack & Multi-Tenancy](/stack-multitenancy). ## M **menu-driven access** — Access model where the ERP sidebar menu is the source of truth for which agents/features a user may use; the app reads the user's real ERP menu instead of a hardcoded role list. See [Access Control](/access-control). **multi-tenancy** — Isolation of each tenant's data. The real boundary today is **explicit `tenant_id`/`userId` WHERE clauses** in every service (see RLS). See [Stack & Multi-Tenancy](/stack-multitenancy). ## N **node types (ai_chat / form / approval / automation)** — The four workflow node executors: `ai_chat` (inline LLM Q&A + extraction), `form` (JSON-schema data entry), `approval` (cross-user HITL task queue), `automation` (out-of-band BullMQ job). See [Proposal Workflow](/proposal-workflow). ## O **orchestrator / front agent** — The thin, fast-tier router (`orchestrator`) that resolves each turn to a specialist and holds only delegation tools (`ask_specialist`, `handoff_to_agent`) plus `web_fetch`. See [Agent Topology](/agent-topology). ## P **per-user ERP JWT** — The user's own ERP JWT, threaded through to CRM MCP calls so ERP RBAC is applied under the user's identity (a rep sees only their records; a manager sees all) rather than a tenant-shared token. See [Access Control](/access-control). **Proposal Builder** — The step-by-step conversational flow (deal-scoped) for authoring a branded proposal/quote, with section edits, versioning, and DOCX/PDF export. Mid-migration onto the workflow engine via `PROPOSAL_ON_ENGINE`. See [Proposal Workflow](/proposal-workflow). ## R **Researcher** — In-process sub-agent doing bounded, opt-in public web research (Tavily) with cited results; enabled per tenant. **RLS (Row-Level Security)** — Postgres RLS policies exist on many tables but are **largely dormant at runtime** (owner role bypasses them; the tenant `set_config` runs only in the legacy auth transaction). The real isolation boundary is explicit `tenant_id`/`userId` WHERE clauses. See [Stack & Multi-Tenancy](/stack-multitenancy). ## S **SOP** — Standard Operating Procedure: an admin authors a workflow as natural-language Markdown (SKILL.md-like), which the compiler turns into a JSON graph. See [Proposal Workflow](/proposal-workflow). **specialist** — A domain agent (`sale`, `delivery-manager`) that owns a business domain and its tool roster, keeping the thread while a request is in-scope. See [Agent Topology](/agent-topology). **sticky session** — Once a specialist owns a thread, follow-up turns stay routed to it without re-routing through the front. See [Agent Topology](/agent-topology). **sub-agent** — A bounded, single-purpose helper (`researcher`, `documentor`) invoked in-process by a specialist through the `AgentRegistry`, not exposed to end users as a top-level agent. ## T **tier (smart / fast / vision)** — The three model tiers a task resolves to for cost/quality balance: `smart` (strong reasoning), `fast` (cheap/low-latency, e.g. the front), `vision` (image/OCR). Resolved per tenant provider. See [Collaboration & LLM](/collaboration-llm). **ToolLoopAgent** — The Vercel AI SDK v6 agent loop (LLM ↔ tool calls until done) that every profile runs on, executed generically by `agent-turn-runner.service.ts`. See [Request Lifecycle](/request-lifecycle). **tool-ownership rule** — "A tool belongs to the agent whose domain owns the action." Keeps rosters clean; enforced by tests. See [Tool Ownership](/tool-ownership). **turn-dispatcher** — `turn-dispatcher.service.ts`, which routes each turn: sticky-session override first, otherwise the orchestrator front resolves the target agent. See [Request Lifecycle](/request-lifecycle). **two-layer gating (ToolSet + activeTools)** — A tool is reachable only if it is in **both** the profile's `ToolSet` **and** its `activeTools` for the current stage; tests assert both. See [Tool Ownership](/tool-ownership). ## W **WBS** — Work Breakdown Structure: an EPIC / Story / Task / Subtask tree the Delivery Manager decomposes a Statement of Work into via the WBS-generation workflow; exportable to DOCX/PDF/CSV (Jira import). See [Agent Topology](/agent-topology). **workflow_outputs** — The generic store for documents a workflow produces (proposal / WBS / generic), rendered per shape and exportable to PDF/DOCX/Markdown; the surface co-build collaborates on. See [Proposal Workflow](/proposal-workflow). ## Next - [Agent Topology](/agent-topology) · [Tool Ownership](/tool-ownership) · [ERP MCP](/erp-mcp) - [Feature Flags](/feature-flags) · [Repository Structure](/repo-structure)