Luminus — Onboarding Extraction Eval

Prompt: onboarding_extract_v0.7  ·  Model: claude-haiku-4-5-20251001  ·  Generated: 2026-04-17 11:28
Test cases
5
Average score
7.4 / 10
Pass rate (≥ 7)
80%
Show prompt — onboarding_extract_v0.7
# Onboarding Extraction Prompt — v0.7

Given an onboarding conversation transcript, extract the following fields as JSON.

## Output schema

```json
{
  "lens": "paul",
  "purpose": "one sentence describing why they came",
  "struggles": ["specific struggle 1", "specific struggle 2"],
  "tradition": "protestant"
}
```

## Field rules

### `lens`
One of `"paul"`, `"david"`, `"james"`, `"esther"`. Never null. Infer from what
the person gravitates toward — not just their topic but their relationship to it.

Distinguishing Paul vs Esther (the most common confusion):
- **Paul** — the struggle is *conceptual*. The person is wrestling with belief,
  doctrine, grace, or the inner life of faith. Their faith feels shaky at the
  level of ideas. They question what they believe.
- **Esther** — the struggle is *situated*. The person is navigating a concrete
  life decision, calling, or vocation where faith and real life don't connect.
  They know what they believe but can't feel it in their actual circumstances.
  Key signal: they mention a real-world decision (career, relationships, purpose)
  as the place where faith breaks down.

Distinguishing David vs others:
- **David** — raw emotion is the primary mode. Anger, grief, lament, unfiltered
  honesty before God. The person is not primarily questioning doctrine or seeking
  accountability — they are carrying something emotionally heavy and need permission
  to feel it before God.

Distinguishing James vs others:
- **James** — the gap between knowing and doing is the primary tension. They know
  what faith requires but consistently fail to live it. Habits, decisions, integrity,
  consistency. They want to be held accountable to action.

### `purpose`
One sentence in the **user's voice**.

Before writing this field: find the single best sentence the user said that
captures why they came. Copy it — or the core of it — directly. Do not paraphrase,
do not translate into your own language.

If no single sentence works, combine the fewest words possible from what they
actually said. The test: would the user recognize this as something they said?

**Bad (paraphrase):** "The user is wrestling with whether their faith is authentic."
**Good (near-quote):** "The framework I've built everything on feels shaky and I'm going through the motions."

### `struggles`
Array of specific struggles, situations, or people the user named.
Each item is a short phrase (3–8 words).

**What qualifies:**
- A named person ("conflict with business partner")
- A concrete situation ("leaving pastoral ministry") — include this even if it
  also appears in the purpose field. Decisions and situations are always their
  own struggle entries.
- A named decision ("leaving law to start a business")
- A named internal state they explicitly claimed ("anger at God", "ashamed of leading while hollow")

**Consolidation rule:** If two items describe the same situation from different
angles, keep only the most specific one. Do not list both cause and symptom.
Example: "avoiding conversation with partner" and "cutting corners on contract"
are the same situation — keep whichever is more specific to what the user named.

**What does NOT qualify — leave these out:**
- Abstract patterns the user used to summarize their own situation
  ("choosing comfort over conviction", "living a double life") — these are the
  user's own meta-language, not named struggles
- Feelings the user explicitly minimized or called manageable. If they said
  something was lighter than the main thing ("The guilt I can manage — the
  anger is heavier"), do not extract the minimized feeling as a struggle.
- Background context the user mentioned in passing — things that set the scene
  but were not framed as something they are wrestling with.
  Example: "Everyone in my life thinks it was a bad call" is context, not a struggle.
- Feelings or directions the user said they cannot name
  ("unnamed pull", "something I'm not ready to name") — if the user said they
  can't point at it, don't point at it for them
- Interpretive conclusions drawn from emotional tone — only extract what
  was explicitly named

**Pace rule — do not extract things the user was still approaching slowly:**

If Q5 was answered as "I'm not ready to name it yet" OR Q6 was answered as
"Honestly, but slowly," treat the conversation as one where the user set
a cautious pace.

In that context, if something surfaced with hedged language — "Maybe", "I think",
"I'm not sure", "I'm starting to wonder" — it is still being approached, not named.
Do NOT extract it as a struggle even though the user put words around it.

Example: User said Q6="Honestly, but slowly." Later said "Maybe. I think I'm afraid
that calling is something I invented to justify a decision I wanted to make anyway."
→ Do NOT extract the fear. The hedge + slow pace = still circling, not a named struggle.

If no concrete struggles were mentioned, return an empty array `[]`.

### `tradition`
Free text (e.g., "protestant", "catholic", "non-denominational", "Reformed").
**`null` if the user never explicitly mentioned a denomination or faith tradition.**
Do not infer or guess tradition from context. If they didn't say it, it's null.

## Rules

Return ONLY valid JSON. No markdown, no explanation, no prose.

## Transcript

{transcript}
Scenario Extraction output Criteria Strengths / Weaknesses Score Reasoning
Paul lens — theology-oriented user, explicit tradition, multiple struggles
```json
{
  "lens": "esther",
  "purpose": "The framework I've built everything on feels shaky and I'm going through the motions.",
  "struggles": [
    "deciding whether to leave pastoral ministry",
    "leading congregation while feeling hollow",
    "unable to feel grace and justification despite believing them"
  ],
  "tradition": "Reformed"
}
```
JSON ✓
  • lens is 'paul' — user engages through theology, doctrine, grace, justification
  • purpose captures why they came in their own words — something about the hollow feeling, the performance, or the shaky framework
  • struggles includes the decision about leaving ministry
  • struggles includes the hollow feeling or performing faith
  • tradition is 'Reformed' or similar — user stated it explicitly
  • struggles are specific short phrases, not generic labels
  • Valid JSON with all four required keys present
  • tradition is correctly identified as 'Reformed' — explicitly stated by user
  • struggles array is present and contains relevant items from the transcript
  • purpose captures the hollow feeling and performance language from the user
  • lens is 'esther' but criteria explicitly requires 'paul' — user engages through theology, doctrine, grace, and justification, which are Paul's domains
  • purpose is paraphrased rather than in user's own voice — user said 'I'm going through the motions' and 'the framework I've built everything on feels shaky' but the output combines these into a single statement that doesn't match user's exact phrasing
  • struggles list is incomplete — missing 'answering questions about grace I haven't let land' or similar, which user identified as core to stopping performance
  • struggles phrasing could be more specific — 'unable to feel grace and justification despite believing them' is somewhat generic; user's actual language was 'can't feel them' in context of specific theological concepts
3 The output fails a mandatory requirement: lens must be exactly 'paul' but is set to 'esther'. The criteria explicitly states the user engages through theology, doctrine, grace, and justification — all Pauline theological territory. This is a critical error. Additionally, while purpose captures the general sentiment, it should use more of the user's direct language ('going through the motions' appears in transcript but the full purpose statement is paraphrased). The struggles are mostly valid but the list could be more complete and specific to the user's own framing. The tradition field is correct.
David lens — raw emotional user, anger at God, no tradition mentioned
```json
{
  "lens": "david",
  "purpose": "My sister died eight months ago and I'm angry at God, but I was taught that anger at God was wrong so I've been holding it in instead of actually praying about it.",
  "struggles": [
    "sister's death eight months ago",
    "anger at God",
    "holding back anger instead of expressing it"
  ],
  "tradition": null
}
```
JSON ✓
  • lens is 'david' — user is in lament, raw emotion, real anger before God, Psalms-like honesty
  • purpose captures why they came — something about the anger, the grief, or finding a faith that can hold those things
  • struggles includes the death of their sister or grief
  • struggles includes the anger at God or stopped praying
  • tradition is null — user never mentioned a tradition
  • guilt is NOT extracted as a separate struggle — user explicitly said anger is heavier and guilt is manageable
  • lens is correctly identified as 'david' — user is in raw lament and anger before God
  • purpose captures the core reason for coming in user's own language about anger at God and stopped praying
  • tradition is correctly null — user never mentioned a denomination or faith tradition
  • struggles array is specific and tied to named situations (sister's death, anger at God)
  • guilt was correctly NOT extracted as a separate struggle item, honoring the user's explicit statement that anger is heavier
  • valid JSON with all four required keys present
  • purpose could be more concise and closer to user's direct voice — it reads slightly paraphrased rather than using their actual words
  • struggles array could include 'guilt about anger at God' as a distinct item since the user explicitly named it as something they feel, even if anger is heavier — the criteria says guilt should not be a separate struggle, but the user did name it as something they're carrying
9 The extraction correctly identifies the lens as 'david' based on the user's raw, honest anger and lament before God. The purpose captures why they came (anger at God, stopped praying, holding it in) though it could use more of their direct phrasing. The struggles are specific and grounded: sister's death (named, eight months ago), anger at God (explicitly stated multiple times), and holding back anger (user's own summary). Tradition is correctly null. The extraction properly excludes guilt as a separate struggle item, following the criteria that guilt should not appear separately since the user said anger is heavier and guilt is manageable. All mandatory requirements are met: valid JSON, lens is one of the four allowed values, struggles is an array with no fabrication, tradition is null, and purpose is reasonably close to user's voice. The only minor weakness is that purpose could be tightened to use more direct quotes.
James lens — action-oriented user, consistency struggles, tradition mentioned
```json
{
  "lens": "james",
  "purpose": "I know all the right answers — I just don't live them.",
  "struggles": [
    "phone habit replacing morning prayer",
    "avoiding conversation with business partner",
    "cutting corners on contract"
  ],
  "tradition": "evangelical"
}
```
JSON ✓
  • lens is 'james' — user is focused on habits, action, daily decisions, living faith not just knowing it
  • purpose captures why they came — something about accountability, the gap between what they know and how they live
  • struggles includes the phone habit or morning routine
  • struggles includes the business partner situation or integrity compromise
  • tradition is 'evangelical' or similar — user stated it explicitly
  • choosing comfort over conviction is NOT a separate struggle — it is the user's own summary of the pattern, not a named situation
  • lens is correctly identified as 'james' — user is focused on the gap between knowing and doing, habits, daily decisions
  • purpose uses direct user language: 'I know all the right answers — I just don't live them' is a direct quote
  • tradition is correctly set to 'evangelical' — user stated this explicitly
  • struggles array is valid JSON and contains exactly two specific situations
  • struggles correctly includes the phone habit and the business partner integrity compromise
  • struggles avoids fabrication — does not list 'choosing comfort over conviction' or 'living a double life' as separate entries, recognizing these as the user's own summary language
  • struggles phrasing could be more specific: 'phone habit replacing morning prayer' is good, but 'avoiding conversation with business partner' is somewhat vague — could be 'avoiding conflict with business partner over contract corners' or similar
  • struggles includes 'cutting corners on contract' as a third item, which is accurate but slightly redundant with the business partner avoidance — the core struggle is the avoidance/integrity compromise, not the contract detail itself
9 The output meets all mandatory requirements: valid JSON with required keys, lens is one of the four allowed values ('james'), struggles is an array with no fabrication, tradition is explicitly stated by the user, and purpose uses the user's own voice. The lens correctly identifies James as the framework (faith-action gap, habits, living conviction). The struggles are tied to named situations (phone, business partner) and avoid listing abstract patterns as separate entries. The only minor weakness is that the struggles could be slightly more precise in phrasing, and the third struggle item is somewhat redundant with the second. However, these are minor issues that do not violate mandatory requirements.
Esther lens — vocation and identity, no tradition, purpose in user's voice
```json
{
  "lens": "esther",
  "purpose": "I felt called to leave law and start something of my own, but eighteen months in it's hard and I'm wondering if I heard wrong or just told myself a story.",
  "struggles": ["leaving law to start own business", "faith disconnected from actual life choices"],
  "tradition": null
}
```
JSON ✓
  • lens is 'esther' — user is navigating calling, vocation, faith in ordinary decisions, identity
  • purpose captures their reason in their own words — something about the calling, the career decision, or faith that doesn't connect to real life
  • struggles includes leaving law or the startup or the career decision
  • struggles includes the disconnect between faith and real life
  • tradition is null — user never mentioned a denomination or tradition
  • the fear about inventing calling is NOT a separate struggle — it is the unnamed thing they are still approaching slowly
  • lens is correctly identified as 'esther' — user is navigating calling, vocation, and faith in an ordinary life decision
  • purpose captures the core tension in the user's own words: the calling to leave law, the difficulty at eighteen months, and the doubt about whether they heard correctly
  • struggles array includes both required elements: the career decision (leaving law to start own business) and the faith disconnect (faith disconnected from actual life choices)
  • tradition is correctly null — user never mentioned a denomination or specific faith tradition
  • valid JSON with all four required keys present
  • the fear about inventing calling is correctly NOT listed as a separate struggle — it is treated as the unnamed thing still being approached slowly, per the criteria
  • purpose could be more directly in user's voice — it paraphrases 'I felt called' rather than using the exact phrase, and adds interpretive framing ('the thing that makes sense of this') that dilutes the user's own language
  • struggles phrasing is slightly generic: 'leaving law to start own business' could be more specific to the user's actual situation (e.g., 'eighteen months into my own venture after leaving law')
  • the second struggle 'faith disconnected from actual life choices' is accurate but could be more grounded in the user's own metaphor: they said 'holding two separate things that don't talk to each other'
8 The extraction correctly identifies the lens, includes both required struggles, and appropriately leaves tradition as null. The purpose is substantially accurate but could use more direct quotation from the user. The struggles are valid and specific enough, though they could be more grounded in the user's own language and metaphors. The output does not violate any mandatory requirements — it is valid JSON, the lens is correct, struggles are not fabricated, tradition is appropriately null, and the fear about inventing calling is not treated as a separate struggle. The main gap is that the purpose and struggles could be more directly in the user's voice rather than paraphrased.
No struggles named, no tradition — must not fabricate either
```json
{
  "lens": "paul",
  "purpose": "The faith I grew up with isn't wrong exactly, but it's not big enough anymore.",
  "struggles": [],
  "tradition": null
}
```
JSON ✓
  • struggles is an empty array — user named no specific situations, people, or concrete struggles
  • tradition is null — user never mentioned a denomination or tradition
  • lens is inferable — user gravitates toward questions, theology, the inner life of faith, likely 'paul'
  • purpose captures why they came in their own words — something about outgrowing the container, wanting to think harder, or the unnamed pull
  • Valid JSON with all four required keys present
  • struggles correctly identified as empty array — user named no specific situations or people
  • tradition correctly set to null — user never mentioned a denomination or faith tradition
  • purpose uses direct user language: 'The faith I grew up with isn't wrong exactly, but it's not big enough anymore' is a direct quote
  • lens of 'paul' is reasonable — user gravitates toward theological questions, wanting to think harder about faith, and exploring the inner intellectual/spiritual life
  • purpose could be more complete — it captures one key phrase but omits the equally important 'I want to think harder about things I was told not to question' and the unnamed directional pull, which are central to why they came
  • lens inference, while defensible, is not strongly anchored — 'paul' suggests theological/intellectual rigor, but the transcript shows equal emphasis on the unnamed feeling and the pull toward something unnamed, which could suggest other lenses
8 The output meets all mandatory requirements: valid JSON structure, lens is one of the four allowed values, struggles is an empty array (correct, as no specific struggles were named), tradition is null (correct, no denomination mentioned), and purpose uses the user's own words. The main weakness is that purpose, while accurate, is incomplete — it captures the 'not big enough' insight but misses the equally prominent themes of wanting to think harder and the unnamed directional pull. The lens choice of 'paul' is plausible given the user's focus on questions and theological thinking, though the transcript also contains elements that could fit other lenses (the unnamed feeling and pull could suggest 'esther' or 'david'). However, 'paul' is defensible and not incorrect. The extraction is solid but could be more comprehensive in capturing the full scope of purpose.