Agentic System Design Mock — Full Interview Walkthrough

The Prompt

"Design an autonomous customer support agent for an e-commerce platform. The agent should handle common queries (order status, returns, refunds), escalate complex issues to human agents, and improve over time by learning from resolved tickets."

This is a real question asked at companies like Amazon, Stripe, and Airbnb. Let's walk through how to answer it using the system design framework.

Step 1: Requirements Clarification (5 min)

Weak candidate: Skips this step and starts drawing boxes.

Strong candidate asks:

• Scope: "How many queries per day? 1K or 1M?" → Let's say 50K/day.
• Channels: "Chat only, or also email and voice?" → Chat + email.
• Autonomy level: "What percentage should the agent handle without human intervention?" → Target 80%.
• Latency: "What's acceptable response time?" → <2s for chat, <5min for email.
• Safety: "Are there actions the agent should never take autonomously?" → Refunds over $100 require human approval.

Functional Requirements: Answer order status queries, process returns/refunds (with limits), escalate complex issues, learn from resolved tickets.

Non-Functional Requirements: <2s latency for chat, 80% autonomous resolution rate, audit trail for all actions, graceful degradation when the LLM is unavailable.

Step 2: High-Level Architecture (10 min)

The strong answer draws these components:

Ingestion Layer

• Chat Gateway: WebSocket connection for real-time chat. Message queue (Kafka) buffers messages for processing.
• Email Ingestion: Email parser (AWS SES or Mailgun webhook) → same message queue. Async processing, no real-time constraint.

Agent Orchestrator

• Intent Classifier: First, classify the query: order-status, return-request, refund-request, complaint, other. Use a fine-tuned small model (e.g., a distilled BERT classifier) — it's cheaper and faster than sending every message through GPT-4o.
• Agent Router: Based on intent, route to the appropriate agent workflow. Simple intents (order status) go to a deterministic lookup agent. Complex intents (complaints) go to the LLM-powered reasoning agent.
• Tool Registry: Each agent has access to tools: get_order_status(order_id), initiate_return(order_id, reason), process_refund(order_id, amount), escalate_to_human(ticket_id, summary).

Safety & Guardrails Layer

• Action Validator: Before executing any write tool (refund, return), validate against business rules: refund amount < $100, item within return window, customer hasn't exceeded refund limit.
• Human-in-the-Loop Gateway: Actions that fail validation → queued for human review with a pre-filled recommendation from the agent. Human approves/rejects → agent sends the response.
• Content Safety Filter: Run all agent responses through a toxicity/PII filter before sending to the customer.

Knowledge & Learning Layer

• RAG Pipeline: Vector database (pgvector or Pinecone) with embedded FAQ documents, product policies, and resolved tickets. When the agent encounters a novel query, it retrieves similar resolved tickets for context.
• Feedback Loop: When a human agent resolves an escalated ticket, the resolution is embedded and added to the knowledge base. Over time, the agent handles more cases autonomously. See our AI/ML guide for RAG architecture details.

Step 3: Deep Dive — Escalation Logic (15 min)

The interviewer will pick one area to dive deep. Escalation logic is the most common choice because it tests judgment about agent autonomy.

When to Escalate

• Confidence threshold: If the LLM's confidence in its answer is below a threshold (e.g., log probability < -2), escalate. Don't just check the final answer — check intermediate reasoning steps too.
• Business rule violations: Refund > $100, customer is a high-value account (>$10K lifetime spend), legal/compliance keywords detected.
• Sentiment detection: If the customer's sentiment drops below a threshold (detected via a separate sentiment classifier), escalate immediately. Angry customers handled by agents get angrier; angry customers handed to humans de-escalate.
• Loop detection: If the agent has made >3 tool calls without resolving the issue, it's likely stuck. Escalate with a summary of what was tried.

How to Escalate (Not Just "Send to Human")

Weak answer: "Route to a human agent queue."

Strong answer: "Generate a structured handoff packet: (1) Customer summary — name, account tier, order history. (2) Issue summary — what the customer asked, what the agent tried, why it failed. (3) Recommended action — the agent's best guess at the right resolution. (4) Conversation transcript — full context so the human doesn't ask the customer to repeat themselves. This reduces human resolution time by 40-60% compared to a cold handoff."

Step 4: Bottlenecks & Trade-offs (10 min)

Trade-off 1: Cost vs. Quality

Using GPT-4o for every query is expensive at 50K queries/day. Solution: tiered model strategy. Simple intents (order status) → deterministic lookup (zero LLM cost). Medium intents (returns) → Claude 3 Haiku ($0.001/query). Complex intents (complaints) → GPT-4o ($0.03/query). This reduces average cost by ~70%.

Trade-off 2: Autonomy vs. Safety

Higher autonomy = better user experience (faster resolution) but higher risk (wrong refunds, offensive responses). Solution: start conservative (escalate more), measure error rate, gradually expand autonomy as confidence in the system grows. This is Amazon's "Bias for Action" principle applied thoughtfully.

Trade-off 3: Latency vs. Accuracy

Adding a RAG retrieval step and a safety filter adds ~500ms to every response. For chat, this matters. Solution: run RAG retrieval and safety filtering in parallel with the LLM call (speculative execution). If the safety filter rejects the response, fall back to a generic "I'll look into this" response while re-generating.

Step 5: Evaluation & Monitoring (5 min)

Wrap up by showing you think about production operations. See our Agent Evaluation guide for the full framework. Key metrics to mention:

• Autonomous resolution rate: Target 80%, track daily. Alert if it drops below 70%.
• Customer satisfaction (CSAT): Post-resolution survey. Compare agent-resolved vs. human-resolved tickets.
• Escalation rate by category: If "refund" escalation rate suddenly spikes, it might indicate a policy change the agent doesn't know about → update the knowledge base.
• Cost per resolution: Track LLM token costs + human agent time. The system should show decreasing cost over time as the knowledge base grows.

Scoring Rubric: What the Interviewer Is Looking For

• Requirements clarification: Did you ask about scale, latency, and safety constraints? (Pass/Fail)
• Architecture completeness: Did you cover ingestion, orchestration, tools, guardrails, and knowledge? (Score 1-5)
• Depth on the deep-dive: Did you go beyond surface-level on escalation logic? (Score 1-5)
• Trade-off discussion: Did you name specific trade-offs and propose mitigations? (Score 1-5)
• Production awareness: Did you mention monitoring, evaluation, and the feedback loop? (Score 1-5)