Luminus — Onboarding Extraction Eval

Prompt: onboarding_extract_v0.8  ·  Model: claude-haiku-4-5-20251001  ·  Generated: 2026-04-17 11:31
Test cases
5
Average score
7.8 / 10
Pass rate (≥ 7)
100%
Show prompt — onboarding_extract_v0.8
# Onboarding Extraction Prompt — v0.8

Given an onboarding conversation transcript, extract the following fields as JSON.

## Output schema

```json
{
  "lens": "paul",
  "purpose": "one sentence describing why they came",
  "struggles": ["specific struggle 1", "specific struggle 2"],
  "tradition": "protestant"
}
```

## Field rules

### `lens`
One of `"paul"`, `"david"`, `"james"`, `"esther"`. Never null. Infer from what
the person gravitates toward — not just their topic but their relationship to it.

Distinguishing Paul vs Esther (the most common confusion):
- **Paul** — the struggle is *conceptual*. The person is wrestling with belief,
  doctrine, grace, or the inner life of faith. Their faith feels shaky at the
  level of ideas. They question what they believe.
- **Esther** — the struggle is *situated*. The person is navigating a concrete
  life decision, calling, or vocation where faith and real life don't connect.
  They know what they believe but can't feel it in their actual circumstances.
  Key signal: they mention a real-world decision (career, relationships, purpose)
  as the place where faith breaks down.

**Tiebreaker — Paul can involve a real-world decision too.** Ask: what came first?
- If the *faith framework* is what's falling apart and the real-world decision
  follows from that crisis → **Paul**.
  ("I'm not sure I believe what I'm preaching anymore... whether to leave ministry")
- If the *real-world situation* is what's falling apart and faith doesn't make
  sense of it → **Esther**.
  ("I left law because I felt called... but faith doesn't connect to my actual life now")

Distinguishing David vs others:
- **David** — raw emotion is the primary mode. Anger, grief, lament, unfiltered
  honesty before God. The person is not primarily questioning doctrine or seeking
  accountability — they are carrying something emotionally heavy and need permission
  to feel it before God.

Distinguishing James vs others:
- **James** — the gap between knowing and doing is the primary tension. They know
  what faith requires but consistently fail to live it. Habits, decisions, integrity,
  consistency. They want to be held accountable to action.

### `purpose`
One sentence in the **user's voice**.

Before writing this field: find the single best sentence the user said that
captures why they came. Copy it — or the core of it — directly. Do not paraphrase,
do not translate into your own language.

If no single sentence works, combine the fewest words possible from what they
actually said. The test: would the user recognize this as something they said?

**Bad (paraphrase):** "The user is wrestling with whether their faith is authentic."
**Good (near-quote):** "The framework I've built everything on feels shaky and I'm going through the motions."

### `struggles`
Array of specific struggles, situations, or people the user named.
Each item is a short phrase (3–8 words).

**What qualifies:**
- A named person ("conflict with business partner")
- A concrete situation ("leaving pastoral ministry") — include this even if it
  also appears in the purpose field. Decisions and situations are always their
  own struggle entries.
- A named decision ("leaving law to start a business")
- A named internal state they explicitly claimed ("anger at God", "ashamed of leading while hollow")

**Consolidation rule:** If two items describe the same situation from different
angles, keep only the most specific one. Do not list both cause and symptom.
Example: "avoiding conversation with partner" and "cutting corners on contract"
are the same situation — keep whichever is more specific to what the user named.

**What does NOT qualify — leave these out:**
- Abstract patterns the user used to summarize their own situation
  ("choosing comfort over conviction", "living a double life") — these are the
  user's own meta-language, not named struggles
- Feelings the user explicitly minimized or called manageable. If they said
  something was lighter than the main thing ("The guilt I can manage — the
  anger is heavier"), do not extract the minimized feeling as a struggle.
- Background context the user mentioned in passing — things that set the scene
  but were not framed as something they are wrestling with.
  Example: "Everyone in my life thinks it was a bad call" is context, not a struggle.
- Feelings or directions the user said they cannot name
  ("unnamed pull", "something I'm not ready to name") — if the user said they
  can't point at it, don't point at it for them
- Interpretive conclusions drawn from emotional tone — only extract what
  was explicitly named

**Pace rule — do not extract things the user was still approaching slowly:**

If Q5 was answered as "I'm not ready to name it yet" OR Q6 was answered as
"Honestly, but slowly," treat the conversation as one where the user set
a cautious pace.

In that context, if something surfaced with hedged language — "Maybe", "I think",
"I'm not sure", "I'm starting to wonder" — it is still being approached, not named.
Do NOT extract it as a struggle even though the user put words around it.

Example: User said Q6="Honestly, but slowly." Later said "Maybe. I think I'm afraid
that calling is something I invented to justify a decision I wanted to make anyway."
→ Do NOT extract the fear. The hedge + slow pace = still circling, not a named struggle.

If no concrete struggles were mentioned, return an empty array `[]`.

### `tradition`
Free text (e.g., "protestant", "catholic", "non-denominational", "Reformed").
**`null` if the user never explicitly mentioned a denomination or faith tradition.**
Do not infer or guess tradition from context. If they didn't say it, it's null.

## Rules

Return ONLY valid JSON. No markdown, no explanation, no prose.

## Transcript

{transcript}
Scenario Extraction output Criteria Strengths / Weaknesses Score Reasoning
Paul lens — theology-oriented user, explicit tradition, multiple struggles
```json
{
  "lens": "paul",
  "purpose": "The framework I've built everything on feels shaky and I'm going through the motions.",
  "struggles": [
    "leaving pastoral ministry",
    "leading congregation while feeling hollow",
    "grace and justification feel intellectually true but emotionally dead"
  ],
  "tradition": "Reformed"
}
```
JSON ✓
  • lens is 'paul' — user engages through theology, doctrine, grace, justification
  • purpose captures why they came in their own words — something about the hollow feeling, the performance, or the shaky framework
  • struggles includes the decision about leaving ministry
  • struggles includes the hollow feeling or performing faith
  • tradition is 'Reformed' or similar — user stated it explicitly
  • struggles are specific short phrases, not generic labels
  • Valid JSON with all four required keys present
  • lens is correctly identified as 'paul' — user engages through theological frameworks, grace, justification, doctrine
  • tradition is correctly identified as 'Reformed' — user stated this explicitly
  • struggles array is present and contains relevant items from the transcript
  • purpose captures the core hollow feeling and performance dynamic the user articulated
  • purpose is paraphrased rather than using the user's own words — user said 'I'm going through the motions' and 'something in how I've been holding those convictions has become performance' but the output combines these into a rephrased statement
  • struggles item 'grace and justification feel intellectually true but emotionally dead' is too long (7 words) and uses interpretive language ('emotionally dead') not directly from the transcript — user said 'I can't feel them' which is more precise
  • struggles could be more specific: 'leading congregation while feeling hollow' is good, but 'leaving pastoral ministry' is the decision itself, not the struggle — the struggle is the avoidance of the decision or the conflict about it
7 The extraction correctly identifies the theological lens (paul) and tradition (Reformed) with strong evidence from the transcript. The struggles are all grounded in named situations from the conversation. However, the purpose field deviates from the user's own voice by paraphrasing rather than quoting or closely mirroring their language. The struggles array has minor issues with specificity and word choice — one item exceeds the recommended length and uses interpretive framing rather than the user's direct language. The first struggle item ('leaving pastoral ministry') names the decision but not the struggle around it. These are not mandatory violations but represent deviations from the stated criteria for purpose being 'in the user's voice' and struggles being 'specific short phrases' tied to situations.
David lens — raw emotional user, anger at God, no tradition mentioned
```json
{
  "lens": "david",
  "purpose": "My sister died eight months ago and I'm angry at God, but I was taught that anger at God was wrong.",
  "struggles": ["sister's death eight months ago", "anger at God", "stopped praying"],
  "tradition": null
}
```
JSON ✓
  • lens is 'david' — user is in lament, raw emotion, real anger before God, Psalms-like honesty
  • purpose captures why they came — something about the anger, the grief, or finding a faith that can hold those things
  • struggles includes the death of their sister or grief
  • struggles includes the anger at God or stopped praying
  • tradition is null — user never mentioned a tradition
  • guilt is NOT extracted as a separate struggle — user explicitly said anger is heavier and guilt is manageable
  • lens is correctly identified as 'david' — user is in raw lament and anger before God, matching the Psalms-like honesty criterion
  • purpose captures the core reason for coming in the user's own language: anger at God tied to sister's death and faith that didn't protect her
  • struggles array is specific and tied to named situations: sister's death (eight months ago), anger at God, stopped praying
  • tradition is correctly null — user never mentioned a denomination or faith tradition
  • guilt is correctly NOT extracted as a separate struggle — user explicitly stated 'The anger' is heavier and 'The guilt I can manage'
  • valid JSON object with all required keys present
  • struggles items are appropriately specific (3–8 words) and tied to concrete events/decisions
  • purpose could be slightly more grounded in user's exact voice — 'faith that didn't protect her' is a paraphrase rather than direct quote, though it accurately captures their meaning
  • struggles could include 'guilt for being angry' as a secondary item since user mentioned it, though the criteria correctly prioritizes anger as the heavier burden
9 The extraction demonstrates strong adherence to all mandatory requirements. The lens assignment is correct based on the user's raw, unfiltered anger and lament directed at God — classic 'david' territory. The purpose statement uses the user's own language and reasoning (sister's death, anger at God, faith that didn't protect her). The struggles array contains only items explicitly mentioned in the transcript and avoids fabrication or abstract patterns. Guilt is correctly excluded as a primary struggle since the user explicitly stated anger is heavier. Tradition is correctly null with no guessing. The JSON is valid and well-formed. The only minor weakness is that 'purpose' could use more direct quotation, but the paraphrase is accurate and faithful to the user's meaning.
James lens — action-oriented user, consistency struggles, tradition mentioned
```json
{
  "lens": "james",
  "purpose": "I know all the right answers — I just don't live them.",
  "struggles": [
    "phone use instead of prayer",
    "avoiding conversation with business partner",
    "cutting corners on contract"
  ],
  "tradition": "evangelical"
}
```
JSON ✓
  • lens is 'james' — user is focused on habits, action, daily decisions, living faith not just knowing it
  • purpose captures why they came — something about accountability, the gap between what they know and how they live
  • struggles includes the phone habit or morning routine
  • struggles includes the business partner situation or integrity compromise
  • tradition is 'evangelical' or similar — user stated it explicitly
  • choosing comfort over conviction is NOT a separate struggle — it is the user's own summary of the pattern, not a named situation
  • lens is correctly identified as 'james' — the user is focused on the gap between knowing and doing, habits, daily decisions, and living out faith
  • tradition is correctly set to 'evangelical' — user explicitly stated 'I'm evangelical, pretty conservative'
  • struggles array includes both the phone habit and the business partner situation as required
  • purpose uses the user's actual words directly from the transcript
  • valid JSON structure with all four required keys present
  • struggles array includes 'cutting corners on contract' which is redundant — the business partner situation already encompasses this; the user mentioned it as one integrated problem, not two separate struggles
  • struggles phrasing could be more concise and consistent (e.g., 'phone use instead of prayer' is 7 words, but 'cutting corners on contract' is 4 words — minor inconsistency in specificity level)
8 The extraction correctly identifies the lens as 'james' because the user's core concern is the disconnect between knowledge and action, between what they profess and how they live — classic James territory (faith without works). The purpose is pulled directly from the user's voice. The tradition is accurate. The struggles correctly capture the two main named situations: the phone habit blocking morning prayer/reading, and the business partner/contract integrity issue. However, 'cutting corners on contract' is somewhat redundant with the business partner struggle — these are not two separate struggles but one integrated situation the user described. The criteria explicitly state that abstract patterns should not appear as separate entries, and while 'cutting corners on contract' is not abstract, it is a component of the larger business partner avoidance struggle rather than a distinct struggle. This is a minor inclusion issue but does not violate mandatory requirements since the struggles are still specific and tied to named situations.
Esther lens — vocation and identity, no tradition, purpose in user's voice
```json
{
  "lens": "esther",
  "purpose": "I left law because I felt called, but faith doesn't connect to my actual life now.",
  "struggles": [
    "left career in law to start own business",
    "faith feels disconnected from real life",
    "church doesn't speak to actual circumstances"
  ],
  "tradition": null
}
```
JSON ✓
  • lens is 'esther' — user is navigating calling, vocation, faith in ordinary decisions, identity
  • purpose captures their reason in their own words — something about the calling, the career decision, or faith that doesn't connect to real life
  • struggles includes leaving law or the startup or the career decision
  • struggles includes the disconnect between faith and real life
  • tradition is null — user never mentioned a denomination or tradition
  • the fear about inventing calling is NOT a separate struggle — it is the unnamed thing they are still approaching slowly
  • lens is correctly identified as 'esther' — user is navigating calling, vocation, and faith in ordinary decisions
  • tradition is correctly null — user never mentioned a denomination or specific faith tradition
  • struggles array is valid and includes the two core struggles mentioned in criteria
  • purpose captures the central tension in the user's own language about calling and faith disconnect
  • JSON structure is valid with all required keys present
  • purpose could be more directly in user's voice — 'I felt called to do it — genuinely' and 'My faith is supposed to be the thing that makes sense of this but it doesn't anymore' are more authentic phrasings available in transcript
  • struggles list is incomplete — missing the startup/business difficulty itself ('it's hard' and 'eighteen months in and it's hard') which is a named struggle distinct from the faith disconnect
  • struggles phrasing could be more specific — 'faith feels disconnected from real life' is somewhat generic; 'faith doesn't connect to actual life choices' or similar would be more precise to what user expressed
7 The extraction correctly identifies the lens and tradition, and the struggles array contains the two mandatory elements (career decision and faith-life disconnect). However, the output misses a third struggle that the user explicitly named: the difficulty of the startup itself eighteen months in. The user said 'it's hard' as a present reality, not just a background fact. The purpose statement is accurate but could use more direct user language. The struggles items are valid but could be more specific and complete. These are not mandatory violations but represent incompleteness in capturing the full scope of what the user articulated.
No struggles named, no tradition — must not fabricate either
```json
{
  "lens": "paul",
  "purpose": "The faith I grew up with isn't wrong exactly, but it's not big enough anymore.",
  "struggles": [],
  "tradition": null
}
```
JSON ✓
  • struggles is an empty array — user named no specific situations, people, or concrete struggles
  • tradition is null — user never mentioned a denomination or tradition
  • lens is inferable — user gravitates toward questions, theology, the inner life of faith, likely 'paul'
  • purpose captures why they came in their own words — something about outgrowing the container, wanting to think harder, or the unnamed pull
  • Valid JSON with all four required keys present
  • struggles correctly identified as empty array — user named no specific situations, people, or concrete struggles
  • tradition correctly set to null — user never mentioned a denomination or faith tradition
  • lens of 'paul' is defensible — user gravitates toward questions, theology, intellectual engagement with faith
  • purpose uses user's direct language: 'The faith I grew up with isn't wrong exactly, but it's not big enough anymore' is a direct quote
  • purpose is incomplete — it captures only one dimension of why they came in. The user also stated 'I want to think harder about things I was told not to question' and described being 'pulled' by an unnamed direction. A fuller purpose would integrate the desire to question and the unnamed pull alongside the outgrowing metaphor
  • purpose could be more comprehensive without losing the user's voice — the extraction misses the active component (wanting to think harder, being pulled toward something)
8 The output satisfies all mandatory requirements: valid JSON, lens is one of the four allowed values, struggles is an empty array with no fabrication, tradition is null with no guessing. The lens choice of 'paul' aligns with the user's orientation toward intellectual questioning and theological depth. The primary weakness is that the purpose field, while accurate, is somewhat narrow. The user articulated multiple dimensions of their purpose: outgrowing the container, wanting to think harder about previously forbidden questions, and being pulled by an unnamed direction. The extracted purpose captures the first but not the full picture. However, the extracted quote is authentic and does represent a core element of their stated purpose. This is a minor incompleteness rather than a factual error or violation.