# Human-in-the-Loop ## ⚡We *Care*. Period. {% content-ref url="human-in-the-loop" %} [human-in-the-loop](https://docs.lisaiceland.com/platform+/active-development/human-in-the-loop) {% endcontent-ref %} {% content-ref url="advanced-agent-verifier" %} [advanced-agent-verifier](https://docs.lisaiceland.com/platform+/active-development/advanced-agent-verifier) {% endcontent-ref %} {% content-ref url="../../smarter-ai-learn-more/ai-safety+/guardrails+/ai-safety-guardrails" %} [ai-safety-guardrails](https://docs.lisaiceland.com/smarter-ai-learn-more/ai-safety+/guardrails+/ai-safety-guardrails) {% endcontent-ref %} {% content-ref url="testing" %} [testing](https://docs.lisaiceland.com/platform+/active-development/testing) {% endcontent-ref %} *** ## AI Voice+ ## HITL Protections ![HITL](https://camo.githubusercontent.com/ac7e88dccfa80ee9fcecfcd9ccf6a86cd5ed3cee0fdae54d7561c030296f0794/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4849544c2d362d2d6d656368616e69736d732d677265656e) Last updated: March 24, 2026. [![Tests](https://camo.githubusercontent.com/26641b5a70dea0526ad84e92b8d1dea013f3682c187ef1cac1ac09685e2c31e2/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f74657374732d39373725323070617373696e672d627269676874677265656e)](https://camo.githubusercontent.com/26641b5a70dea0526ad84e92b8d1dea013f3682c187ef1cac1ac09685e2c31e2/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f74657374732d39373725323070617373696e672d627269676874677265656e) [![Vitest](https://camo.githubusercontent.com/75fde65290dbfc7dd2b52c4aa25a9f069d8f342dd65f96e40ddf31bab03b69bc/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f746573746564253230776974682d7669746573742d364539463138)](https://camo.githubusercontent.com/75fde65290dbfc7dd2b52c4aa25a9f069d8f342dd65f96e40ddf31bab03b69bc/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f746573746564253230776974682d7669746573742d364539463138) [![Languages](https://camo.githubusercontent.com/526cf55b8be703ab2d413b92d1ccf65a837f02413ed34f9f3015fc0e07161bf8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c616e6775616765732d33372d626c7565)](https://camo.githubusercontent.com/526cf55b8be703ab2d413b92d1ccf65a837f02413ed34f9f3015fc0e07161bf8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c616e6775616765732d33372d626c7565) [![BYOK Providers](https://camo.githubusercontent.com/991e44d30e63e00f5d26eb53658c85234ebfbfad6a72f2854fddb9ebd5ba80d3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f42594f4b25323070726f7669646572732d31382d6f72616e6765)](https://camo.githubusercontent.com/991e44d30e63e00f5d26eb53658c85234ebfbfad6a72f2854fddb9ebd5ba80d3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f42594f4b25323070726f7669646572732d31382d6f72616e6765) [![Security](https://camo.githubusercontent.com/9848248df8f878d8a375f7a0993b27219c2ed5c209d9a2442e6001064201a7cc/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f73656375726974792d68617264656e65642d637269746963616c)](https://camo.githubusercontent.com/9848248df8f878d8a375f7a0993b27219c2ed5c209d9a2442e6001064201a7cc/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f73656375726974792d68617264656e65642d637269746963616c) ![Moat](https://docs.lisaiceland.com/~gitbook/image?url=https%3A%2F%2Fcamo.githubusercontent.com%2Fcf9284fb15978bad5057ded6dd214f81c456978e71e1a84894a7c1c0203e94db%2F68747470733a2f2f696d672e736869656c64732e696f2f62616467652f636f6d70657469746976652532306d6f61742d3530253230646966666572656e746961746f72732d707572706c65\&width=300\&dpr=3\&quality=100\&sign=f6cd6274\&sv=2) [![Shipped](https://camo.githubusercontent.com/c6d47a185d7feee3e89913188f3f3f27c0dcd3c37348c167e7c76818d565e5d4/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f7368697070656425323066656174757265732d7e3332352d677265656e)](https://camo.githubusercontent.com/c6d47a185d7feee3e89913188f3f3f27c0dcd3c37348c167e7c76818d565e5d4/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f7368697070656425323066656174757265732d7e3332352d677265656e) *** ### What is Human-in-the-Loop? Human-in-the-Loop (HITL) is an AI safety pattern where human judgement, consent, or oversight is required at critical points in an automated pipeline. Instead of letting AI operate with unchecked autonomy, HITL ensures that humans remain the final authority on sensitive decisions. In AI Voice+, HITL does **not** mean a human reviews every single AI response (that would destroy the real-time experience). Instead, it means the system enforces **human checkpoints** at the moments that matter most: consent, identity, safety, and audit. *** ### Our 6 HITL Mechanisms #### 1. Content Moderation Blocks (Fail-Closed) Every AI chat function (`agent-one-chat`, `convo-chat`, `chat-with-data`) runs user input through moderate functions before the AI model ever sees it. If the moderation check flags the content **or** if the moderation service itself is unreachable, the request is **blocked** — not allowed through. * **Code**: in each edge function * **Logging**: Blocks are logged to `ai_usage_logs` with feature tags `agent_one_moderation_block`, `convo_moderation_block`, `chat_data_moderation_block` * **Human element**: The user receives a clear safety notice and can rephrase their request #### 2. Safety Refusals via System Prompt The `SAFETY_PREAMBLE` is injected into every AI conversation. It contains non-negotiable instructions that the model must follow, including: * **No medical, legal, or financial advice** — the AI will politely decline and suggest consulting a professional * **Uncertainty disclosure** — if the AI is not confident, it says so rather than guessing * **Bias protection** — equitable treatment regardless of caller demographics (see BiasProtections.md) * **Code**: `SAFETY_PREAMBLE` constant in `agent-one-chat`, `convo-chat` * **Human element**: The AI actively redirects users to human experts for sensitive topics #### 3. Call Recording Consent Before any voice call is recorded, the caller must provide explicit consent. The `recordConsent` tool captures: * Whether consent was given (`consent_given: boolean`) * The method of consent (`consent_method: string`) * Caller identity (name, number) * Timestamp and metadata If the caller declines, recording does not proceed. Consent records are stored in the `call_consents` table with full audit trail. * **Code**: `Record Consent` tool * **Human element**: The caller — a real human — has the final say on whether their call is recorded #### 4. Identity Verification Before AI agents can perform sensitive actions (accessing account details, making changes), callers must verify their identity through one or more methods: * **Security PIN** — caller provides their PIN * **Date of birth** — caller confirms their DOB * **Account number** — caller provides their account number The `Verify Identity` tool checks these against `client_records` and logs every attempt (successful or not) to the `Identity Verifications` table. * **Code**: `Verify Identity` tool * **Human element**: The caller must prove who they are before the AI proceeds — no verification, no access #### 5. Injection Safe-Wrapping When prompt injection attempts are detected (10 regex patterns covering DAN mode, system prompt extraction, role-play jailbreaks, etc.), the system does **not** silently block them. Instead, it: 1. Wraps the injection in safety markers so the AI model can see it's been flagged 2. Logs the detection to `ai_usage_logs` with feature tag `injection_detected` 3. Allows the conversation to continue safely This approach preserves UX (no mysterious failures) while neutralizing the attack. * **Code**: `Scan For Injection` in `agent-one-safety.ts` and edge functions * **Human element**: The superadmin can review all injection attempts in the audit log and take action if patterns emerge #### 6. Output Moderation AI responses are checked **after** generation but **before** delivery to the user. If the output contains flagged content: * The response is replaced with a safety notice * The event is logged to `ai_usage_logs` with feature tag `convo_output_moderation_block` * The user is informed that the response was filtered * **Code**: Output moderation in `convo-chat` edge function * **Human element**: Harmful content never reaches the end user; the superadmin can review what was blocked *** ### Request Pipeline — Where HITL Checkpoints Sit ``` User Input │ ▼ ┌─────────────────────┐ │ Rate Limiting │ ← IP-based, 30 msg / 15 min └─────────┬───────────┘ │ ▼ ┌─────────────────────┐ │ Injection Scanning │ ← 10 regex patterns, safe-wrap └─────────┬───────────┘ │ ▼ ┌─────────────────────┐ │ Input Moderation │ ← moderateContent(), fail-closed ✦ HITL └─────────┬───────────┘ │ ▼ ┌─────────────────────┐ │ PII Redaction │ ← Email, phone, SSN, NINO patterns └─────────┬───────────┘ │ ▼ ┌─────────────────────┐ │ AI Model + Safety │ ← SAFETY_PREAMBLE enforced ✦ HITL │ Preamble │ └─────────┬───────────┘ │ ▼ ┌─────────────────────┐ │ Output Moderation │ ← Response checked before delivery ✦ HITL └─────────┬───────────┘ │ ▼ Response to User ``` For voice calls, two additional HITL checkpoints apply **before** the conversation begins: ``` Incoming Call │ ▼ ┌─────────────────────┐ │ Consent Flow │ ← Caller must agree to recording ✦ HITL └─────────┬───────────┘ │ ▼ ┌─────────────────────┐ │ Identity Verify │ ← PIN / DOB / Account number ✦ HITL └─────────┬───────────┘ │ ▼ AI Agent Conversation ``` *** ### How Users Benefit | Benefit | How It Works | | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | **Callers are never recorded without consent** | The consent flow is mandatory. No consent = no recording. Period. | | **AI cannot give dangerous advice** | Medical, legal, and financial advice is refused by the safety preamble. The AI redirects to human professionals. | | **Client PII is protected automatically** | Email addresses, phone numbers, SSNs, and NINOs are redacted before they reach the AI model. | | **AI asks instead of guessing** | The safety preamble instructs the AI to disclose uncertainty and ask for clarification rather than hallucinating answers. | | **Harmful content is blocked** | Both input and output moderation catch inappropriate content before it affects the conversation. | | **Identity theft is prevented** | Callers must verify their identity before accessing sensitive account information. | *** ### How Superadmins Benefit | Benefit | Implementation | | ------------------------------- | ------------------------------------------------------------------------------------------------------------ | | **Full audit trail** | Every moderation block, injection detection, and safety event is logged to `ai_usage_logs` with feature tags | | **Attack pattern visibility** | Injection attempts are logged (not silently dropped), making patterns visible over time | | **Moderation metrics** | Block counts per feature tag enable monitoring of content safety trends | | **Consent compliance** | The `Call Consents` table provides a complete record for regulatory compliance | | **Identity verification audit** | The `Identity Verifications` table logs every attempt, including failed ones | *** ### What HITL Does NOT Do | Item | Reason | | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | **Manual review queue** | Would add unacceptable latency to real-time conversations. We use automated moderation instead. | | **Human approval before every response** | Would destroy the conversational UX. The safety preamble and output moderation provide equivalent protection at machine speed. | | **Human-reviewed training data** | We use third-party models (OpenAI, Google). We mitigate via prompts, not training. | | **Escalation to human agents** | Currently out of scope. The transfer-to-phone-number feature provides a manual fallback for complex situations. | *** ### Related Documentation * Admin - need-to-know basis *** ### Framework Alignment Our HITL implementation aligns with: * **NIST AI RMF (Govern function)**: Human oversight is a core requirement of the Govern function. Our consent flows and identity verification satisfy this. * **EU AI Act (Article 14)**: Requires "human oversight measures" for AI systems. Our fail-closed moderation and consent flows provide this. * **ISO/IEC 42001 (Section 6.1.3)**: Requires identification of AI risks and human intervention points. Our 6 mechanisms map directly to identified risk areas. *** These are reference alignments, not certifications. External auditing is recommended for formal compliance. ***