# The learning loop

A human-in-the-loop system without a learning loop is just expensive review. LoopDesk treats every override as structured data, then closes the loop back into your pipeline.

## What gets captured

Every time a reviewer's decision differs from the AI suggestion, LoopDesk writes a row to `learnings` with:

| Field                 | Notes                                 |
| --------------------- | ------------------------------------- |
| `item_id`             | Links back to full input + suggestion |
| `project_id`          | Group by use case                     |
| `original_suggestion` | What the AI proposed                  |
| `corrected_decision`  | What the human shipped                |
| `diff_kind`           | `edit`, `reject`, `escalate_override` |
| `reason`              | Free-text reviewer note (optional)    |
| `guideline_id`        | Which guideline (if any) was invoked  |
| `reviewer_id`         | Who corrected it                      |
| `confidence`          | Snapshot from the original item       |
| `risk_flags`          | Snapshot from the original item       |
| `created_at`          | Timestamp                             |

Approvals where the human shipped the AI suggestion unchanged are *not* learnings — they are reinforcing signal at best, and we do not want to dilute the corrections dataset with them.

## Three ways to close the loop

### 1. Tighten guidelines

The fastest cycle. When you see a cluster of overrides with the same reason ("agent suggested a refund without checking order status"), update the project guideline. Reviewers see the new guideline immediately; future AI suggestions can include it in the prompt.

### 2. Tune the routing policy

When you see overrides clustered in a specific confidence band, your `auto_threshold` is wrong for that project. The [analytics](/hitl-docs/features/analytics.md) calibration view plots override rate against confidence — flat lines mean calibrated, U-shapes mean broken.

### 3. Fine-tune or update prompts

Export `learnings` as CSV or JSONL from `/export` and feed it into:

* Prompt engineering (few-shot examples drawn from real corrections)
* RAG retrieval examples
* SFT / DPO fine-tuning datasets
* Eval suites — every learning is a test case the current model failed

Export format is stable; treat it as a contract.

## Healthy loop metrics

Watch these on the [analytics](/hitl-docs/features/analytics.md) dashboard:

* **Override rate by week** — should trend down as you act on learnings
* **Override rate by confidence band** — should be flat (calibrated) or strictly decreasing
* **Time-to-action on learnings** — how long between capturing a learning and shipping a guideline / prompt update

A queue that is not shrinking after a few iteration cycles means the loop is open somewhere — usually that nobody is reading the learnings.

## Privacy note

Learnings contain whatever was in the original `input`. If your pipeline submits PII, the learnings dataset contains PII. Either:

* Scrub PII before submitting to LoopDesk, or
* Treat the `learnings` table as a PII-bearing system and apply your usual data-handling rules (retention, access controls, export auditing)

The schema does not encrypt `input` at rest beyond Postgres-level encryption.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://hitl-01.gitbook.io/hitl-docs/human-in-the-loop/learning-loop.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.