crm-ai-demo/AGENTS.md

CRM AI Demo - Project Guidelines

Project

B2B stainless steel jewelry wholesaler (Yehwang, NL) CRM with AI-powered sales/service assistant bridge. Chatwoot webhook → LLM agent (tool-use) → Chatwoot reply + Twenty CRM opportunity creation.

Tech Stack

• Bridge: Node.js 24, CommonJS ("type": "commonjs"), pure Node built-ins (http/fetch/crypto) + jose + pg. No Express. No test/lint/build scripts — verify by running the server and hitting /healthz.
• LLM: OpenAI-compatible provider via bridge/llm.config.json. Active: aliyun → DashScope qwen-plus-latest (enableThinking: false). server.js header comment says "glm-5.2/ark" — ignore it, trust llm.config.json.
• Messaging: Chatwoot (self-hosted, external source checkout at ./chatwoot/ — has its own .git/).
• CRM: Twenty CRM (self-hosted, external source checkout at ./twenty/ — may be absent).
• Vector DB: pgvector on shared crm-postgres (pg18). Three DBs: chatwoot, twenty, products. KB RAG table: products.kb_docs.
• Product Search: external recommend API (RECOMMEND_URL, default http://recommend.internal.yw.com).
• Order/Inventory APIs: trading-service (ORDER_API_URL) and eta-service (INVENTORY_API_URL). Mock server available: node mock_services.js (port 9505).
• Embedding: DashScope text-embedding-v4 (dim 1024), configured in kb.config.json. Query translation fallback uses qwen-turbo.

Architecture

Chatwoot (external) ──webhook──► Bridge (:4000) ──LLM tool-use──► Chatwoot reply
                                       │
                                       ├── pgvector (products.kb_docs)   KB RAG            (search_kb)
                                       ├── recommend API                  product catalog    (search_products)
                                       ├── trading-service                order tracking     (search_orders)
                                       ├── eta-service                    inventory/stock    (search_inventory)
                                       └── Twenty CRM GraphQL             create Opportunity

Bridge routes (bridge/server.js):

• GET /healthz (and /) — liveness, returns agentBot: !!AGENT_BOT_SECRET
• GET /conversation/:id and GET /conversation/:id/html — debug views
• POST /agent-bot — Chatwoot AgentBot outgoing webhook (HMAC-signed)
• POST /chatwoot-webhook — Chatwoot message_created webhook (fallback entry)

Agent tool-use loop: LLM autonomously calls search_kb, search_products, search_orders, search_inventory, or escalate_human. Single AGENT_SYSTEM persona — no prompt switching. Per-conversation lock (locks Set) prevents concurrent LLM calls; createdOpps Set prevents duplicate Opportunities.

Startup

# Full stack (requires Docker Desktop):
bash start.sh

# Shared infra only (no Chatwoot/Twenty source needed):
docker compose up -d --build

# Local dev (no Docker):
cd bridge && node server.js  # Node 24+; needs PG_URL etc. as env vars

# Mock business APIs (for dev without real trading/eta services):
node mock_services.js  # Listens on :9505

start.sh conditionally starts Twenty (if ./twenty/docker-compose.yaml exists) and Chatwoot (if ./chatwoot/docker-compose.yml exists — note: currently runs unconditionally, will error if absent). Shared infra always starts.

Logs / health:

docker logs -f crm-bridge
curl http://localhost:4000/healthz

No rebuild needed for code changes — docker-compose.yml volume-mounts bridge/*.js and *.config.json read-only. docker restart crm-bridge picks up edits. Only rebuild the image when changing package.json deps.

Key Files

bridge/
  server.js              Main bridge: routes, LLM agent loop, tool impls, Chatwoot/Twenty API calls (~1250 lines)
  index_kb.js            KB vectorizer: customer-service-kb/*.md → embed → products.kb_docs (DROPS + recreates table each run)
  register_agent_bot.js  Registers Chatwoot AgentBot (idempotent). Run: node register_agent_bot.js
  llm.config.json        LLM provider config. Active provider key: "aliyun"
  kb.config.json         Embedding config (model, dim, translateModel) + DashScope creds
  chatwoot.config.json   Chatwoot API connection (baseURL, accountId, accessToken, inboxId)
  twenty.config.json     Twenty CRM connection (baseURL, workspaceId, apiKey JWT)
  Dockerfile             node:24-alpine, npm install --omit=dev, copies server.js + configs
customer-service-kb/     FAQ markdown (source for index_kb.js). Chunks split on ### Q:. English; AI translates per customer language.
infra/init-db.sh         postgres init: creates DBs chatwoot/twenty/products + enables pgvector on chatwoot & products.
docker-compose.yml       Shared infra: pgvector/pgvector:pg18 + redis:8-alpine + bridge
mock_services.js         Mock trading/eta API server (:9505) for dev without real backends
add_note.js              CLI utility: attach a text note to a Twenty opportunity
create_workflow.js       CLI utility: create Twenty automation workflow (new opp → auto follow-up task)
find_keys.js             CLI utility: find Twenty signing key tables in postgres
gen_apikey.js            CLI utility: generate Twenty API key JWT (needs ENCRYPTION_KEY + PG_DATABASE_URL)
landing.html             Multilingual landing page
start.sh                 One-shot launcher for all compose stacks
docs/twenty-crm-guide.md Twenty CRM integration guide: which GraphQL ops bridge uses, data model, extension ideas

Environment Variables

Env vars always override *.config.json values. Key overrides in server.js:

• CW_BASE_URL — overrides chatwoot.config.json baseURL
• TWENTY_BASE_URL — overrides twenty.config.json baseURL
• PG_URL — postgres connection (default postgres://postgres:postgres@localhost:5432/products)
• AGENT_BOT_SECRET — HMAC signing key for AgentBot webhook verification
• RECOMMEND_URL — product recommendation API base URL
• ORDER_API_URL — trading-service base URL (order queries)
• ORDER_API_KEY — auth key for trading-service
• INVENTORY_API_URL — eta-service base URL (stock/ETA queries)
• INVENTORY_API_KEY — auth key for eta-service
• DASHSCOPE_API_KEY — overrides kb.config.json apiKey (avoids hardcoding in config)
• PORT — bridge listen port (default 4000)

Docker-compose sets these for the bridge container (using container names for inter-service routing, host.docker.internal for host-side services).

Conventions & Gotchas

• Config loading: loadCfg() reads *.config.json from __dirname (the bridge dir). Env vars always win — check process.env.* overrides in server.js before editing config files.
• KB re-indexing: index_kb.js DROPS and recreates kb_docs every run (no incremental update). Any model/dimension change requires a full re-vectorize. Run node index_kb.js after editing any customer-service-kb/*.md file.
• KB doc format: YAML frontmatter (title/category/keywords/source_url) + ### Q:/answer blocks. Embedding text = keywords + Q + A. Source URLs are mapped per-section in index_kb.js SECTION_URL_MAP.
• Agent persona is the source of truth for sales policy (discount tiers, minimum order €30, contacts). AGENT_SYSTEM in server.js encodes business rules; customer-service-kb/ encodes policy FAQs. Keep them consistent.
• Language rule: replies must mirror the customer's language, even when tool results are English. Enforced in AGENT_SYSTEM.
• Multi-language KB recall: when cosine similarity < 0.55 and query isn't English, searchKB auto-translates to English via qwen-turbo and merges results. Rarely triggers — text-embedding-v4 handles most EU languages well.
• Duplicate addChatToOpportunity call: server.js lines ~1086-1088 call this function twice for the same conversation/opportunity. Likely a bug — second call creates a duplicate note.
• start.sh Chatwoot unconditional: the Twenty section checks for docker-compose.yaml existence first, but the Chatwoot section runs docker compose up unconditionally — will fail if ./chatwoot/ is absent.
• Not a git repo: the project root has no .git/. ./chatwoot/ has its own .git/ (separate checkout). ./twenty/ may or may not be present.
• Secrets in configs: *.config.json and docker-compose.yml contain real API keys/tokens (DashScope, Chatwoot access token, Twenty JWT, AGENT_BOT_SECRET). Treat as sensitive; do not commit to public repos.
• Resource limits: postgres 0.5 CPU/512M, redis 0.3 CPU/256M, bridge 0.5 CPU/512M (in docker-compose.yml). LLM/embedding calls are external network — watch latency under load.
• Dockerfile base image: node:24-alpine. The comment says "Node 18+" but FROM node:24-alpine. Node 24 has built-in fetch/FormData/Blob.
• Bridge auto-creates follow-up tasks: after creating an Opportunity, createLeadInCRM auto-creates a Task due in 3 days, linked to the opportunity. This is in bridge code, not Twenty's workflow engine (Twenty API key lacks permission to create workflows via GraphQL).
• Opportunity extraction logic: EXTRACT_SYSTEM requires 3 elements to set ready=true: company name + contact info (email/phone/name) + product/need. Missing any → ready=false, no CRM record created.