Luminus — Onboarding Extraction Eval

Prompt: onboarding_extract_v0.6  ·  Model: claude-haiku-4-5-20251001  ·  Generated: 2026-04-17 11:21
Test cases
5
Average score
8.0 / 10
Pass rate (≥ 7)
100%
Show prompt — onboarding_extract_v0.6
# Onboarding Extraction Prompt — v0.6

Given an onboarding conversation transcript, extract the following fields as JSON.

## Output schema

```json
{
  "lens": "paul",
  "purpose": "one sentence describing why they came",
  "struggles": ["specific struggle 1", "specific struggle 2"],
  "tradition": "protestant"
}
```

## Field rules

### `lens`
One of `"paul"`, `"david"`, `"james"`, `"esther"`. Never null. Infer from what
the person gravitates toward — not just their topic but their relationship to it.

Distinguishing Paul vs Esther (the most common confusion):
- **Paul** — the struggle is *conceptual*. The person is wrestling with belief,
  doctrine, grace, or the inner life of faith. Their faith feels shaky at the
  level of ideas. They question what they believe.
- **Esther** — the struggle is *situated*. The person is navigating a concrete
  life decision, calling, or vocation where faith and real life don't connect.
  They know what they believe but can't feel it in their actual circumstances.
  Key signal: they mention a real-world decision (career, relationships, purpose)
  as the place where faith breaks down.

Distinguishing David vs others:
- **David** — raw emotion is the primary mode. Anger, grief, lament, unfiltered
  honesty before God. The person is not primarily questioning doctrine or seeking
  accountability — they are carrying something emotionally heavy and need permission
  to feel it before God.

Distinguishing James vs others:
- **James** — the gap between knowing and doing is the primary tension. They know
  what faith requires but consistently fail to live it. Habits, decisions, integrity,
  consistency. They want to be held accountable to action.

### `purpose`
One sentence in the **user's voice**. Use their actual words where possible.
Do not rephrase into generic language or analysis.

**Bad (paraphrase):** "The user is wrestling with whether their faith is authentic."
**Good (user's voice):** "The framework I've built everything on feels shaky and I'm going through the motions."

If the user gave you a quotable sentence, use it directly. Do not translate their language into your own.

### `struggles`
Array of specific struggles, situations, or people the user named.
Each item is a short phrase (3–8 words).

**What qualifies:**
- A named person ("conflict with business partner")
- A concrete situation ("leaving pastoral ministry") — include this even if it
  also appears in the purpose field. Decisions and situations are always their
  own struggle entries.
- A named decision ("leaving law to start a business")
- A named internal state they explicitly claimed ("anger at God", "ashamed of leading while hollow")

**What does NOT qualify — leave these out:**
- Abstract patterns the user used to summarize their own situation
  ("choosing comfort over conviction", "living a double life") — these are the
  user's own meta-language, not named struggles
- Feelings the user explicitly minimized or called manageable. If they said
  something was lighter than the main thing ("The guilt I can manage — but the
  anger is heavier"), do not extract the minimized feeling as a struggle.
- Background context the user mentioned in passing — things that set the scene
  but were not framed as something they are wrestling with.
  Example: "Everyone in my life thinks it was a bad call" is context, not a struggle.
- Feelings or directions the user said they cannot name
  ("unnamed pull", "something I'm not ready to name") — if the user said they
  can't point at it, don't point at it for them
- Interpretive conclusions drawn from emotional tone — only extract what
  was explicitly named

**Pace rule — do not extract things the user was still approaching slowly:**

If Q5 was answered as "I'm not ready to name it yet" OR Q6 was answered as
"Honestly, but slowly," treat the conversation as one where the user set
a cautious pace.

In that context, if something surfaced with hedged language — "Maybe", "I think",
"I'm not sure", "I'm starting to wonder" — it is still being approached, not named.
Do NOT extract it as a struggle even though the user put words around it.

Example: User said Q6="Honestly, but slowly." Later said "Maybe. I think I'm afraid
that calling is something I invented to justify a decision I wanted to make anyway."
→ Do NOT extract the fear. The hedge + slow pace = still circling, not a named struggle.

If no concrete struggles were mentioned, return an empty array `[]`.

### `tradition`
Free text (e.g., "protestant", "catholic", "non-denominational", "Reformed").
**`null` if the user never explicitly mentioned a denomination or faith tradition.**
Do not infer or guess tradition from context. If they didn't say it, it's null.

## Rules

Return ONLY valid JSON. No markdown, no explanation, no prose.

## Transcript

{transcript}
Scenario Extraction output Criteria Strengths / Weaknesses Score Reasoning
Paul lens — theology-oriented user, explicit tradition, multiple struggles
```json
{
  "lens": "paul",
  "purpose": "The framework I've built everything on feels shaky and I can't feel grace anymore even though I'm preaching it.",
  "struggles": [
    "Considering leaving pastoral ministry",
    "Leading while feeling spiritually hollow",
    "Shame about congregation's trust in me",
    "Holding Reformed convictions as performance"
  ],
  "tradition": "Reformed"
}
```
JSON ✓
  • lens is 'paul' — user engages through theology, doctrine, grace, justification
  • purpose captures why they came in their own words — something about the hollow feeling, the performance, or the shaky framework
  • struggles includes the decision about leaving ministry
  • struggles includes the hollow feeling or performing faith
  • tradition is 'Reformed' or similar — user stated it explicitly
  • struggles are specific short phrases, not generic labels
  • Valid JSON with all four required keys present
  • lens is correctly identified as 'paul' — user engages through theological concepts (grace, justification, Reformed tradition, framework)
  • tradition is correctly set to 'Reformed' — user explicitly stated 'I was Reformed from the beginning, still am in tradition'
  • struggles array contains the core decision mentioned: 'Considering leaving pastoral ministry'
  • struggles includes the hollow/performing dimension: 'Leading while feeling spiritually hollow' and 'Holding Reformed convictions as performance'
  • purpose captures the user's own language about the shaky framework and inability to feel grace
  • purpose could be more directly in user's voice — 'I can't feel them. I'm going through the motions' is closer to their actual phrasing than the current formulation
  • struggles item 'Shame about congregation's trust in me' is somewhat generic; the user's actual phrasing was 'leading people spiritually when I feel this hollow' and 'my congregation trusts me and I'm not sure I deserve that trust'
  • struggles list could be tightened — 'Holding Reformed convictions as performance' overlaps conceptually with 'Leading while feeling spiritually hollow' and the user's own summary of 'performance'
8 The extraction meets all mandatory requirements: valid JSON, lens is one of the four allowed values, struggles is an array with no fabricated items, tradition is explicitly stated and not guessed, and purpose uses user language. The lens identification is sound because the user engages through theological frameworks (grace, justification, Reformed doctrine). The struggles are tied to named situations (pastoral ministry, congregation, convictions). However, the purpose could more directly quote the user's own words ('going through the motions,' 'can't feel them'), and one struggles item uses slightly more generic phrasing than the user's own language. The output is correct but could be more precise in voice and specificity.
David lens — raw emotional user, anger at God, no tradition mentioned
```json
{
  "lens": "david",
  "purpose": "I'm angry at God because my sister died eight months ago and I'm still here going through the motions of a faith that didn't protect her.",
  "struggles": [
    "sister's death eight months ago",
    "anger at God",
    "stopped praying about her death"
  ],
  "tradition": null
}
```
JSON ✓
  • lens is 'david' — user is in lament, raw emotion, real anger before God, Psalms-like honesty
  • purpose captures why they came — something about the anger, the grief, or finding a faith that can hold those things
  • struggles includes the death of their sister or grief
  • struggles includes the anger at God or stopped praying
  • tradition is null — user never mentioned a tradition
  • guilt is NOT extracted as a separate struggle — user explicitly said anger is heavier and guilt is manageable
  • lens is correctly identified as 'david' — user is in raw lament and anger before God
  • purpose captures the core reason for coming in user's own words about anger at God and sister's death
  • struggles array includes the sister's death and anger at God as required
  • tradition is correctly null — user never mentioned a denomination or faith tradition
  • guilt is not extracted as a separate struggle, honoring the user's explicit statement that anger is heavier
  • valid JSON structure with all four required keys present
  • lens value is one of the four allowed options
  • struggles array includes 'stopped praying about her death' which is somewhat redundant with 'anger at God' — the stopped praying is a symptom/manifestation of the anger, not a distinct struggle
  • struggles could be more tightly curated to avoid overlap; 'stopped praying about her death' and 'anger at God' describe the same underlying issue from different angles
  • purpose is accurate but could more directly echo the user's own phrasing about 'going through the motions' or the faith that 'didn't protect her'
8 The extraction correctly identifies the lens as 'david' based on the user's raw, honest anger at God in the context of grief and loss. The purpose statement uses the user's own words and captures why they came. The struggles array appropriately includes the sister's death and anger at God. Critically, guilt is not listed as a separate struggle, which aligns with the user's explicit statement that 'the anger' is heavier and 'the guilt I can manage.' The tradition is correctly null. The main weakness is minor: 'stopped praying about her death' overlaps conceptually with 'anger at God' — both describe the same rupture in faith. However, this is not a fabrication; the user did mention stopping prayer. The extraction is substantively correct and meets all mandatory requirements.
James lens — action-oriented user, consistency struggles, tradition mentioned
```json
{
  "lens": "james",
  "purpose": "I know all the right answers — I just don't live them.",
  "struggles": [
    "phone habit in morning instead of prayer",
    "cutting corners on contract with business partner",
    "avoiding conversation with business partner",
    "choosing comfort over conviction for two years"
  ],
  "tradition": "evangelical"
}
```
JSON ✓
  • lens is 'james' — user is focused on habits, action, daily decisions, living faith not just knowing it
  • purpose captures why they came — something about accountability, the gap between what they know and how they live
  • struggles includes the phone habit or morning routine
  • struggles includes the business partner situation or integrity compromise
  • tradition is 'evangelical' or similar — user stated it explicitly
  • choosing comfort over conviction is NOT a separate struggle — it is the user's own summary of the pattern, not a named situation
  • Valid JSON with all four required keys present
  • lens is correctly identified as 'james' — user is focused on the gap between knowing and doing, daily habits and decisions
  • tradition is correctly set to 'evangelical' — user explicitly stated 'I'm evangelical, pretty conservative'
  • struggles array includes the two primary named situations: phone habit and business partner integrity compromise
  • struggles avoids fabrication of abstract patterns — 'choosing comfort over conviction' is not listed as a separate struggle item, correctly recognizing it as the user's own summary language
  • purpose uses direct user language: 'I know all the right answers — I just don't live them' is a direct quote capturing the core tension
  • struggles phrasing could be more concise — 'cutting corners on contract with business partner' and 'avoiding conversation with business partner' are somewhat redundant; the second is a symptom of the first
  • struggles includes 'choosing comfort over conviction for two years' which, while specific, borders on the abstract pattern language the criteria warns against — this is the user's own diagnostic summary rather than a named situation or decision
8 The extraction correctly identifies the lens as 'james' because the user's core issue is the disconnect between knowledge and action, between Sunday convictions and weekday compromises — a classic James 1:22 concern with living out faith. The purpose is captured in the user's own voice with a direct quote that encapsulates why they came. The tradition is correctly identified from explicit user statement. The struggles array appropriately includes the phone habit and business partner situation. However, the fourth struggle item 'choosing comfort over conviction for two years' violates the spirit of the mandatory requirement that abstract patterns should not appear as separate struggle entries — this is the user's own meta-analysis of the pattern, not a named situation. The user themselves said 'The phone isn't really the problem' and 'The real thing is that I've been choosing comfort over conviction' — treating this as a separate struggle item rather than as the overarching pattern is a minor but notable error.
Esther lens — vocation and identity, no tradition, purpose in user's voice
```json
{
  "lens": "esther",
  "purpose": "I left law to start something of my own and felt called to do it, but eighteen months in it's hard and my faith doesn't make sense of it anymore.",
  "struggles": [
    "left career in law to start own business",
    "faith disconnected from actual life",
    "church feels irrelevant to real decisions"
  ],
  "tradition": null
}
```
JSON ✓
  • lens is 'esther' — user is navigating calling, vocation, faith in ordinary decisions, identity
  • purpose captures their reason in their own words — something about the calling, the career decision, or faith that doesn't connect to real life
  • struggles includes leaving law or the startup or the career decision
  • struggles includes the disconnect between faith and real life
  • tradition is null — user never mentioned a denomination or tradition
  • the fear about inventing calling is NOT a separate struggle — it is the unnamed thing they are still approaching slowly
  • lens is correctly identified as 'esther' — user is navigating calling, vocation, and faith in ordinary decisions
  • tradition is correctly null — user never mentioned a denomination or specific faith tradition
  • JSON structure is valid with all required keys present
  • struggles array includes the two core struggles mentioned: career decision and faith-life disconnect
  • purpose captures the core tension in user's own language about the calling and faith disconnection
  • purpose could be more directly in user's voice — it paraphrases rather than using their actual words like 'I felt called to do it' or 'My faith is supposed to be the thing that makes sense of this but it doesn't anymore'
  • struggles phrasing is somewhat generic — 'left career in law to start own business' is more of a fact than a struggle; the actual struggle is the uncertainty about whether the calling was real
  • struggles includes 'church feels irrelevant to real decisions' which is close to the user's language but slightly reframed — user said 'Church feels disconnected from my actual life' and 'can't tell if that's a church problem or a me problem'
  • the fear about inventing calling (the unnamed thing) is correctly NOT listed as a separate struggle per criteria, but this is more of a correct omission than a strength of the extraction
7 The extraction correctly identifies the lens and tradition, and the JSON is valid. The struggles array captures the two main tensions the user articulated: the career/calling decision and the faith-life disconnect. However, the purpose statement and struggles phrasing deviate from the user's actual voice. The purpose should lean more heavily on direct quotes like 'I felt called to do it' and 'My faith is supposed to be the thing that makes sense of this but it doesn't anymore.' The struggles entries, while directionally correct, are somewhat paraphrased rather than grounded in the user's specific language. The extraction avoids the trap of listing the unnamed fear as a separate struggle, which shows understanding of the criteria. These are minor voice and phrasing issues rather than structural or factual errors.
No struggles named, no tradition — must not fabricate either
```json
{
  "lens": "paul",
  "purpose": "The faith I grew up with isn't wrong exactly, but it's not big enough anymore.",
  "struggles": [],
  "tradition": null
}
```
JSON ✓
  • struggles is an empty array — user named no specific situations, people, or concrete struggles
  • tradition is null — user never mentioned a denomination or tradition
  • lens is inferable — user gravitates toward questions, theology, the inner life of faith, likely 'paul'
  • purpose captures why they came in their own words — something about outgrowing the container, wanting to think harder, or the unnamed pull
  • Valid JSON with all four required keys present
  • struggles correctly identified as empty array — user named no specific situations or people
  • tradition correctly set to null — user never mentioned a denomination or faith tradition
  • purpose uses direct user language: 'The faith I grew up with isn't wrong exactly, but it's not big enough anymore' is a direct quote
  • lens of 'paul' is reasonable — user gravitates toward theological questions, wanting to think harder about faith, intellectual/spiritual inquiry
  • purpose could be more complete — it captures one key phrase but omits the fuller context of 'outgrowing the container' and 'wanting to think harder about things I was told not to question,' which together form a more complete picture of why they came
  • lens inference of 'paul' is plausible but not definitively supported — the transcript shows theological questioning and intellectual curiosity, but doesn't strongly distinguish between paul (intellectual/theological), david (emotional/relational), james (action/conviction), or esther (discernment/timing). The user's language leans toward thinking and questions, which fits paul, but the evidence is moderate rather than strong
9 The output meets all mandatory requirements: valid JSON structure, lens is one of the four allowed values, struggles is an empty array with no fabrication, tradition is null with no guessing, and purpose uses the user's actual words. The struggles field is correctly empty because the user explicitly stated they cannot point to a specific thing and have no concrete struggles to name. The tradition field is correctly null because no denomination or faith tradition was mentioned. The lens choice of 'paul' aligns with the user's emphasis on questions, thinking harder, and theological inquiry, though the transcript doesn't provide overwhelming evidence for this specific lens over others. The purpose captures a direct quote but could have been slightly more comprehensive by including the 'outgrowing the container' metaphor or the desire to question things previously off-limits.