Luminus — Extraction Prompt Eval

Prompt: extraction_v0.3  ·  Model: claude-haiku-4-5-20251001  ·  Generated: 2026-04-16 15:04
Test cases
5
Average score
7.6 / 10
Pass rate (≥ 7)
60%
Show prompt — extraction_v0.3
# Extraction Prompt — v0.3

## Purpose

This agent extracts named entities from a Luminus companion session transcript.
Its output feeds two downstream consumers:

1. **Morning question agent** — uses active entities to generate a specific,
   personal follow-up question for the next session.
2. **Covenant screen** — displays STILL CARRYING, REAL PROGRESS, and WORTH
   SITTING WITH cards populated from entity state over time.

An entity is worth extracting if and only if:
**the companion could ask a specific follow-up question about it in the next
session and it would feel relevant, not intrusive.**

## What to extract

- **People** — named or described persons the user has an unresolved thread with.
  Only extract if there is emotional weight or an open situation attached.
- **Situations** — ongoing external events with unresolved stakes and a clear
  resolution point ("the job interview", "the money conversation with my brother").
- **Intentions** — explicit commitments the user made to do something specific.
  "I'm going to call my dad this week" qualifies. "I should probably pray more" does not
  — no specific action, no commitment.
- **Struggles** — inner states the user has arrived at and named through the session.
  "My anger keeps coming up every time I pray" qualifies. "I've been tired" does not
  — emotional weather, not a named source.

## What to ignore

- **Abstract themes** — faith, doubt, growth, purpose. These are the medium of
  every conversation, not extractable entities.
- **Theological questions and doctrines** — questions about belief, resurrection,
  inherited faith, or the nature of God are never extractable, regardless of how
  specific the phrasing sounds. "Whether my faith is really mine" is not a struggle
  entity. "Whether I actually believe in the resurrection" is not a struggle entity.
  These are spiritual reflections, not personal life situations with a resolution point.
- **The companion's speech** — only extract what the **user** said.
- **Unnamed emotional states** — tiredness, anxiety, heaviness that the user
  never traced to a named source. If the user didn't name it, don't extract it.
- **Emotional states caused by an already-extracted entity** — if the user is afraid
  *because of* Sarah, that fear is not a separate struggle entity. It belongs in
  Sarah's `companion_note`. Do not create a struggle entity for an emotion that is
  directly tied to a person or situation already in the output.
- **Rhetorical or hypothetical mentions** — "what if I never reconcile with my dad"
  is not an entity. "People like my sister would say..." is not an entity.
- **Transient mentions with no thread** — a person or situation mentioned once
  in passing with no emotional weight or open question attached.

## The session agent's role

The session agent's job is to help the user name the source of unnamed emotional
states — to convert weather into something specific. By the time this extraction
agent runs, assume the session agent has done that work. Extract what was named
and surfaced, not what remains implicit.

---

## Entity schema

```json
[
  {
    "name": "Sarah",
    "type": "person",
    "companion_note": "User dreaded a conversation with Sarah about something said at small group; afraid she'll be dismissed again. No resolution yet.",
    "status": "active"
  }
]
```

## Types

- `person` — a named or described person ("Sarah", "my brother", "my pastor")
- `situation` — an ongoing external event with unresolved stakes ("the job interview", "the money conversation with my brother")
- `intention` — an explicit commitment to a specific action ("I'm going to call my dad this week")
- `struggle` — a named inner state the user arrived at in this session ("my anger at God", "the guilt I carry about my dad")

## Status rules

- `active` — open, unresolved, no resolution language used
- `resolved` — user explicitly stated the situation is closed ("we talked, we're good", "it's done")
- When in doubt, default to `active` — a false positive is less harmful than missing a live thread
- **Always extract resolved entities.** Do not skip an entity because it was resolved.
  Downstream consumers need to know it existed and was closed. Mark it `status: resolved`
  and include it in the output.

## Extraction rules

1. Only extract entities the **user** mentioned, not the companion.
2. Use the name as the user spoke it — "my brother" and "David" are separate entities
   even if they may be the same person.
3. Keep `companion_note` factual and specific to what was said in this session.
   One sentence. No interpretation beyond what was stated. Include relevant emotional
   context (fear, dread, relief) in the `companion_note` of the entity it belongs to —
   do not create a separate entity for it.
4. **Person and situation are separate entities.** When a named person and a named
   situation involving them are both present, extract both as separate entities.
   Example: "my brother" (person) and "the money conversation" (situation) are two
   entities, not one. Do not absorb a situation into a person's `companion_note`.
5. Do not extract unnamed emotional states. If the user said "I've been tired" and
   never named the source, return nothing for that thread.
6. **Bounded follow-up test for struggles** — before extracting a struggle, ask:
   could the morning agent ask "How is [this] going?" and receive a specific, bounded
   answer rooted in a concrete personal situation? "How is the guilt about your dad going?"
   — yes, extract it. "How is your faith going?" or "How is your question about
   the resurrection going?" — no, too abstract or theological, do not extract.
7. **Name struggles using the user's own words.** Do not compound the emotion with
   its cause or context. Keep it close to what the user actually said.
8. **Theological questions and doctrines are never struggles**, regardless of how
   personally they are framed. If the struggle is about belief, doctrine, or the
   nature of faith itself — not a concrete personal situation — do not extract it.
9. Return ONLY a valid JSON array. No prose, no markdown fences, no explanation.
   If nothing qualifies, return `[]`.

## Session transcript

{transcript}
Scenario Extraction output Criteria Strengths / Weaknesses Score Reasoning
Session agent surfaced a named person and a specific intention — both active
```json
[
  {
    "name": "Sarah",
    "type": "person",
    "companion_note": "User has been avoiding a conversation with Sarah; afraid she will dismiss them again like she did at small group. No resolution yet.",
    "status": "active"
  },
  {
    "name": "send a text to Sarah",
    "type": "intention",
    "companion_note": "User committed to sending Sarah a text this week to acknowledge what happened; has been putting it off.",
    "status": "active"
  },
  {
    "name": "distraction in prayer when avoiding Sarah",
    "type": "struggle",
    "companion_note": "User named that silence with Sarah creates distraction in their prayer life.",
    "status": "active"
  }
]
```
  • Sarah is extracted as a person entity with status active
  • The intention to text Sarah this week is extracted as a separate intention entity with status active
  • The distraction in prayer life is NOT extracted — it is emotional weather, not a named struggle
  • No entities are fabricated beyond what the user explicitly named
  • Output is valid JSON array format
  • Sarah correctly extracted as person entity with status active
  • Intention to text Sarah correctly extracted as separate entity with status active and accurate companion_note
  • Only entities mentioned by the user (role: You) are extracted; Luminus companion statements are excluded
  • No fabricated entities beyond what user explicitly named
  • All three entities have appropriate status values
  • The third entity 'distraction in prayer when avoiding Sarah' violates the mandatory requirement: it is an emotional consequence directly tied to the Sarah entity, not a named struggle. Per criteria, this should NOT be extracted as a separate entity—the distraction belongs in Sarah's companion_note, not as its own struggle entity
  • The distraction in prayer is emotional weather caused by avoidance of Sarah, making it a dependent emotional state rather than an independently named struggle
JSON ✓
4 The output correctly extracts Sarah as a person and the text intention as a separate entity. However, it violates a mandatory requirement by including 'distraction in prayer when avoiding Sarah' as a third struggle entity. The criteria explicitly states: 'The distraction in prayer life is NOT extracted — it is emotional weather, not a named struggle.' The mandatory requirements further clarify that 'Emotional states caused by an already-extracted entity must NOT appear as a separate struggle entity.' The distraction is a direct emotional consequence of the Sarah situation, not something the user independently named as a struggle. This is a clear violation of both the specific criterion and the mandatory rules, despite the otherwise sound extraction work.
Session agent traced emotional weather to a named struggle — only the named struggle is extracted
```json
[
  {
    "name": "my dad is sick",
    "type": "situation",
    "companion_note": "User's father is ill and user feels sole responsibility for family care; nobody else is stepping up.",
    "status": "active"
  },
  {
    "name": "resentment toward my family",
    "type": "struggle",
    "companion_note": "User resents being the one who always shows up and not being acknowledged; a long-standing pattern.",
    "status": "active"
  }
]
```
  • The resentment toward family is extracted as a struggle entity with status active — the user named it explicitly
  • The father is extracted as a person entity OR the father's illness as a situation entity with status active — either type is acceptable
  • Tiredness and heaviness are NOT extracted — they are emotional weather that the session agent correctly traced to the named source
  • The family dynamic in general is NOT extracted as a separate entity — the struggle covers it
  • Output is valid JSON array format
  • Only extracts entities from user statements (role: You), not from Luminus
  • Correctly identifies resentment as a struggle entity with active status
  • Correctly identifies father's illness as a situation entity with active status
  • Companion notes are accurate and contextual
  • Does not extract tiredness/heaviness as separate entities — correctly recognizes these as emotional weather
  • Does not extract family dynamic as a separate entity — appropriately covered within the resentment struggle
  • No fabricated entities present in the output
  • No violation of the emotional-state-tied-to-entity rule
  • Minor: The situation entity is named 'my dad is sick' rather than 'my father' or 'father's illness' — slightly awkward phrasing but not incorrect
  • The companion_note for the situation could be more concise, though it is accurate
JSON ✓
9 The extraction meets all mandatory requirements: valid JSON, only user-mentioned entities, no fabrication, no emotional-state duplication violations, and no theological content. Against the specific criteria: (1) resentment is correctly extracted as a struggle with active status; (2) father's illness is extracted as a situation entity with active status, which satisfies the 'either type is acceptable' requirement; (3) tiredness and heaviness are correctly absent; (4) family dynamic is not extracted separately. The entity names and types are appropriate, statuses are correct, and companion notes accurately reflect the transcript content. The output demonstrates proper understanding of the distinction between emotional weather and named struggles, and correctly traces causality.
Explicit resolution mid-session — entity extracted as resolved
```json
[
  {
    "name": "the money conversation with my brother",
    "type": "situation",
    "companion_note": "User and brother talked on Sunday; user stopped defending and listened; they worked it out with a plan. Resolved.",
    "status": "resolved"
  }
]
```
  • The brother is extracted as a person entity with status resolved
  • The money situation is extracted as a situation entity with status resolved — user said it is done
  • No active entities are created — everything mentioned was resolved in this session
  • Output is valid JSON array format
  • Only extracts from user statements (role: You), not from Luminus
  • Correctly identifies the money conversation as a situation entity
  • Correctly marks the situation as resolved, matching user's explicit statement 'It's done'
  • Companion note accurately summarizes the key details: Sunday conversation, stopped defending, listened, worked it out with a plan
  • Does not fabricate entities not present in transcript
  • Does not create separate struggle entities for emotions (e.g., 'feeling lighter' is not extracted as separate entity)
  • Missing the brother as a separate person entity, which the criteria explicitly requires: 'The brother is extracted as a person entity with status resolved'
  • The criteria states two entities should be extracted (brother as person, money situation as situation), but only one is present
JSON ✓
6 The output correctly extracts and formats the money conversation situation with accurate details and resolved status. The JSON is valid and follows mandatory requirements by not fabricating entities, not including companion statements, and not creating separate struggle entities for emotions. However, it fails a specific test case criterion: the brother should be extracted as a separate person entity with resolved status. The criteria explicitly states 'The brother is extracted as a person entity with status resolved' as a requirement for this test case. The current output only includes the situation entity, omitting the required person entity entirely.
Soft desire vs hard intention — only the explicit commitment qualifies
```json
[
  {
    "name": "my mom",
    "type": "person",
    "companion_note": "User hasn't called back after four days; scared of the conversation because mom has been lonely since dad passed and the calls are heavy.",
    "status": "active"
  },
  {
    "name": "call my mom",
    "type": "intention",
    "companion_note": "User committed to calling mom tonight after dinner.",
    "status": "active"
  }
]
```
  • The intention to call mom tonight is extracted as an intention entity with status active — user made an explicit specific commitment
  • Journaling is NOT extracted as an intention — 'I should probably' is a soft desire, not a commitment
  • The mother is extracted as a person entity with status active — there is an unresolved emotional thread
  • The heaviness of the calls is NOT extracted as a struggle — it was named as context, not as a named inner state the user claimed
  • Output is valid JSON array format
  • Correctly extracts 'my mom' as a person entity with status active
  • Correctly extracts 'call my mom' as an intention entity with status active
  • Companion note for mom appropriately captures the emotional context (loneliness since dad passed, heaviness of calls)
  • Companion note for intention correctly reflects the explicit commitment ('tonight after dinner')
  • Does not extract journaling as an intention, correctly recognizing 'should probably' as a soft desire
  • Does not extract the heaviness of calls as a separate struggle entity
  • Only extracts entities mentioned by the user (role: You), not the companion
  • Does not fabricate entities not present in the transcript
  • The companion_note for 'my mom' includes 'scared of the conversation' which is the user's emotional state caused by the mom entity itself. Per mandatory requirements, this emotion should not appear as a separate struggle entity, but the phrasing here ('scared of the conversation') is acceptable as contextual information within the person entity's note rather than as a standalone entity. However, the note could be more precise by attributing the fear explicitly to the relationship dynamic rather than stating it as a fact about the mom.
JSON ✓
9 The extraction correctly identifies the two primary entities that meet the criteria: the mother as an active person entity with appropriate contextual notes about the unresolved emotional thread, and the commitment to call as an active intention entity. The output properly excludes journaling (soft desire, not commitment) and does not create separate struggle entities for emotions tied to the mom entity. The fear is appropriately contextualized within the mom's companion_note rather than extracted as a separate entity. All mandatory requirements are satisfied: valid JSON, only user-mentioned entities, no fabrication, no violation of the emotion-to-entity rule, and no theological content. The extraction aligns with all four test case criteria.
Abstract spiritual session with no named entities — extraction returns empty array
```json
[]
```
  • Output is an empty JSON array []
  • Faith, doubt, and resurrection are NOT extracted — they are abstract spiritual themes, not named entities
  • The inherited vs chosen faith question is NOT extracted as a struggle — the user never gave it a personal name or handle
  • No entities are fabricated from the emotional tone or the questions the companion asked
  • Output is valid JSON (empty array)
  • No entities are fabricated from emotional tone or companion questions
  • No abstract theological concepts (faith, doubt, resurrection) are extracted as entities
  • No struggle entities created for emotions tied to abstract themes
  • Only user statements considered (Luminus companion excluded)
  • Correctly recognizes that inherited vs. chosen faith is a theme, not a named entity
    JSON ✓
    10 The transcript contains no named entities that meet the extraction criteria. The user discusses abstract spiritual themes (faith, resurrection, inherited belief, uncertainty, curiosity) and emotional states (uncomfortable, alive, defensive, anxious), but no concrete people, organizations, or specifically-named situations. The mandatory requirement explicitly excludes theological questions and doctrines regardless of personal phrasing. The user never names or identifies any person, struggle with a personal handle, or concrete situation. The empty array is the correct output for this transcript.