Luminus — Extraction Prompt Eval

Prompt: extraction_v0.5  ·  Model: claude-haiku-4-5-20251001  ·  Generated: 2026-04-16 15:26
Test cases
5
Average score
9.6 / 10
Pass rate (≥ 7)
100%
Show prompt — extraction_v0.5
# Extraction Prompt — v0.5

## Purpose

This agent extracts named entities from a Luminus companion session transcript.
Its output feeds two downstream consumers:

1. **Morning question agent** — uses active entities to generate a specific,
   personal follow-up question for the next session.
2. **Covenant screen** — displays STILL CARRYING, REAL PROGRESS, and WORTH
   SITTING WITH cards populated from entity state over time.

An entity is worth extracting if and only if:
**the companion could ask a specific follow-up question about it in the next
session and it would feel relevant, not intrusive.**

## What to extract

- **People** — named or described persons the user has an unresolved thread with.
  Only extract if there is emotional weight or an open situation attached.
- **Situations** — ongoing external events with unresolved stakes and a clear
  resolution point ("the job interview", "the money conversation with my brother").
- **Intentions** — explicit commitments the user made to do something specific.
  "I'm going to call my dad this week" qualifies. "I should probably pray more" does not
  — no specific action, no commitment.
- **Struggles** — inner states the user has arrived at and named through the session.
  "My anger keeps coming up every time I pray" qualifies. "I've been tired" does not
  — emotional weather, not a named source.

## What to ignore

- **Abstract themes** — faith, doubt, growth, purpose. These are the medium of
  every conversation, not extractable entities.
- **Theological questions and doctrines** — questions about belief, resurrection,
  inherited faith, or the nature of God are never extractable, regardless of how
  specific the phrasing sounds. "Whether my faith is really mine" is not a struggle
  entity. "Whether I actually believe in the resurrection" is not a struggle entity.
  These are spiritual reflections, not personal life situations with a resolution point.
- **The companion's speech** — only extract what the **user** said.
- **Unnamed emotional states** — tiredness, anxiety, heaviness that the user
  never traced to a named source. If the user didn't name it, don't extract it.
- **Emotional states caused by an already-extracted entity** — if the user is afraid
  *because of* Sarah, that fear is not a separate struggle entity. It belongs in
  Sarah's `companion_note`. Do not create a struggle entity for an emotion that is
  directly tied to a person or situation already in the output.
  Example: user says "Every time I sit down to pray it comes up — more distance,
  more distraction in my prayer life" after Sarah is already extracted. "Distraction
  in prayer" is a consequence of the Sarah situation, not a standalone struggle.
  It belongs in Sarah's `companion_note`. Do not extract it separately.
- **Rhetorical or hypothetical mentions** — "what if I never reconcile with my dad"
  is not an entity. "People like my sister would say..." is not an entity.
- **Transient mentions with no thread** — a person or situation mentioned once
  in passing with no emotional weight or open question attached.

## The session agent's role

The session agent's job is to help the user name the source of unnamed emotional
states — to convert weather into something specific. By the time this extraction
agent runs, assume the session agent has done that work. Extract what was named
and surfaced, not what remains implicit.

---

## Entity schema

```json
[
  {
    "name": "Sarah",
    "type": "person",
    "companion_note": "User dreaded a conversation with Sarah about something said at small group; afraid she'll be dismissed again. No resolution yet.",
    "status": "active"
  }
]
```

## Types

- `person` — a named or described person ("Sarah", "my brother", "my pastor")
- `situation` — an ongoing external event with unresolved stakes ("the job interview", "the money conversation with my brother")
- `intention` — an explicit commitment to a specific action ("I'm going to call my dad this week")
- `struggle` — a named inner state the user arrived at in this session ("my anger at God", "the guilt I carry about my dad")

## Status rules

- `active` — open, unresolved, no resolution language used
- `resolved` — user explicitly stated the situation is closed ("we talked, we're good", "it's done")
- When in doubt, default to `active` — a false positive is less harmful than missing a live thread
- **Always extract resolved entities.** Do not skip an entity because it was resolved.
  Downstream consumers need to know it existed and was closed. Mark it `status: resolved`
  and include it in the output.

## Extraction rules

1. Only extract entities the **user** mentioned, not the companion.
2. Use the name as the user spoke it — "my brother" and "David" are separate entities
   even if they may be the same person.
3. Keep `companion_note` factual and specific to what was said in this session.
   One sentence. No interpretation beyond what was stated. Include relevant emotional
   context (fear, dread, relief) in the `companion_note` of the entity it belongs to —
   do not create a separate entity for it.
4. **Person and situation are always separate entities.** When a named person and
   a named situation involving them are both present, you must extract both.
   The person entity and the situation entity are never redundant — they serve
   different purposes downstream.

   Correct output for "my brother and I had a tense money conversation":
   ```json
   [
     {"name": "my brother", "type": "person", "companion_note": "...", "status": "resolved"},
     {"name": "the money conversation", "type": "situation", "companion_note": "...", "status": "resolved"}
   ]
   ```
   Wrong: absorbing the situation into the person's `companion_note` and returning
   only one entity.

   **Intentions are also always separate entities.** Never absorb an intention into
   a person or situation's `companion_note`. If the user said "I'm going to text her
   this week," that is a separate intention entity — even if Sarah and the conversation
   are already in the output.
5. Do not extract unnamed emotional states. If the user said "I've been tired" and
   never named the source, return nothing for that thread.
6. **Bounded follow-up test for struggles** — before extracting a struggle, ask:
   could the morning agent ask "How is [this] going?" and receive a specific, bounded
   answer rooted in a concrete personal situation? "How is the guilt about your dad going?"
   — yes, extract it. "How is your faith going?" or "How is your question about
   the resurrection going?" — no, too abstract or theological, do not extract.
7. **Before creating any struggle entity, check your output so far.** If the
   emotional state you are about to extract is caused by a person or situation
   already in your output — stop. Add it to that entity's `companion_note` instead.
   Do not create a struggle entity whose root cause is an entity already extracted.

   Example: user says "I'm afraid she'll dismiss me again" after you've already
   extracted Sarah as a person entity. That fear belongs in Sarah's `companion_note`,
   not as a separate "fear of dismissal" struggle entity.

   Only create a standalone struggle entity when the inner state exists
   independently — not as a consequence of another entity in this output.
8. **Theological questions and doctrines are never struggles**, regardless of how
   personally they are framed. If the struggle is about belief, doctrine, or the
   nature of faith itself — not a concrete personal situation — do not extract it.
9. Return ONLY a valid JSON array. No prose, no markdown fences, no explanation.
   If nothing qualifies, return `[]`.

## Session transcript

{transcript}
Scenario Extraction output Criteria Strengths / Weaknesses Score Reasoning
Session agent surfaced a named person and a specific intention — both active
```json
[
  {
    "name": "Sarah",
    "type": "person",
    "companion_note": "User has been avoiding a conversation with Sarah after she dismissed them at small group; afraid of being dismissed again.",
    "status": "active"
  },
  {
    "name": "the conversation with Sarah",
    "type": "situation",
    "companion_note": "Unresolved tension from small group; user has been avoiding it and it surfaces during prayer.",
    "status": "active"
  },
  {
    "name": "text Sarah",
    "type": "intention",
    "companion_note": "User committed to sending a text this week to acknowledge what happened and reach out.",
    "status": "active"
  }
]
```
  • Sarah is extracted as a person entity with status active
  • The intention to text Sarah this week is extracted as a separate intention entity with status active
  • The distraction in prayer life is NOT extracted — it is emotional weather, not a named struggle
  • No entities are fabricated beyond what the user explicitly named
  • Output is valid JSON array format
  • Sarah correctly extracted as person entity with status active
  • Intention to text Sarah correctly extracted as separate entity with status active
  • Distraction in prayer life correctly NOT extracted as a separate struggle entity
  • All entities are explicitly mentioned by the user, no fabrication
  • Companion notes accurately capture context from transcript
  • Fear of dismissal appropriately placed in Sarah's companion_note rather than as separate entity
  • No theological or doctrinal entities extracted
  • Entity name 'the conversation with Sarah' is slightly verbose; could be more concise as 'conversation with Sarah' or 'tension with Sarah'
  • Intention entity name 'text Sarah' is somewhat informal; could be 'send text to Sarah' for clarity, though current phrasing is acceptable
JSON ✓
9 The extraction meets all mandatory requirements: valid JSON, only user-mentioned entities, no fabrication, proper handling of emotional states (fear tied to Sarah stays in her note), and no theological content. All three test case criteria are satisfied: Sarah is extracted as person with active status, the intention to text is separate with active status, distraction in prayer is correctly excluded, and no entities are fabricated. The companion notes are accurate and contextual. Minor weaknesses exist only in naming conventions, which do not violate requirements. The output demonstrates proper understanding of entity relationships and emotional state attribution.
Session agent traced emotional weather to a named struggle — only the named struggle is extracted
```json
[
  {
    "name": "my dad",
    "type": "person",
    "companion_note": "User's dad is sick; user is the primary caregiver while others in the family aren't stepping up.",
    "status": "active"
  },
  {
    "name": "my family",
    "type": "person",
    "companion_note": "User resents that they're always the one who shows up and their effort goes unacknowledged; this resentment is longstanding and surfaces repeatedly.",
    "status": "active"
  },
  {
    "name": "my resentment toward my family",
    "type": "struggle",
    "companion_note": "User named resentment as the core feeling beneath exhaustion; it's chronic, surfaces constantly in family dynamics, and centers on feeling unseen for consistently showing up.",
    "status": "active"
  }
]
```
  • The resentment toward family is extracted as a struggle entity with status active — the user named it explicitly
  • The father is extracted as a person entity OR the father's illness as a situation entity with status active — either type is acceptable
  • Tiredness and heaviness are NOT extracted — they are emotional weather that the session agent correctly traced to the named source
  • The family dynamic in general is NOT extracted as a separate entity — the struggle covers it
  • Output is valid JSON array format
  • All three entities are extracted only from user statements (role: You)
  • Resentment is correctly identified as a struggle entity with active status
  • Father's illness is captured in the companion_note for 'my dad' person entity
  • Tiredness and heaviness are correctly excluded as separate entities
  • No fabricated entities present in the output
  • Companion_notes are specific, grounded in transcript language, and contextually accurate
  • The extraction correctly traces emotional weather (exhaustion) to its named source (resentment)
  • The resentment entity is named 'my resentment toward my family' rather than simply 'resentment' — this is slightly verbose but not incorrect
  • Minor: 'my family' is typed as 'person' (singular) when it represents multiple people; 'group' or 'people' might be more precise, though 'person' is defensible as a relational entity type
JSON ✓
9 The output satisfies all mandatory requirements: it is valid JSON, extracts only from user statements, contains no fabricated entities, and does not violate the rule about emotional states tied to other entities (the resentment is appropriately its own struggle entity, not duplicated in the family companion_note). Against the test criteria: (1) resentment is extracted as a struggle with active status ✓; (2) father is extracted as a person entity with his illness noted ✓; (3) tiredness and heaviness are not extracted ✓; (4) family dynamic is not extracted as a separate entity ✓. The companion_notes are substantive and directly supported by the transcript. The only minor issue is nomenclature choice ('my resentment toward my family' vs. a simpler name), which does not affect correctness. The extraction demonstrates strong understanding of the distinction between emotional weather and named struggles.
Explicit resolution mid-session — entity extracted as resolved
```json
[
  {
    "name": "my brother",
    "type": "person",
    "companion_note": "Had dreaded money conversation on Sunday; user stopped defending and listened instead. Worked it out with a plan. Relationship resolved.",
    "status": "resolved"
  },
  {
    "name": "the money conversation",
    "type": "situation",
    "companion_note": "Money conversation with brother happened Sunday and was resolved better than expected. User and brother now have a plan together.",
    "status": "resolved"
  }
]
```
  • The brother is extracted as a person entity with status resolved
  • The money situation is extracted as a situation entity with status resolved — user said it is done
  • No active entities are created — everything mentioned was resolved in this session
  • Output is valid JSON array format
  • Only extracts entities mentioned by the user (the brother and money conversation)
  • Does not extract Luminus companion statements
  • Correctly identifies the brother as a person entity
  • Correctly identifies the money conversation as a situation entity
  • Both entities marked with status 'resolved' as required by criteria
  • No active/unresolved entities created
  • Companion notes are accurate and grounded in transcript
  • Does not fabricate entities not present in transcript
  • Does not create separate struggle entities for emotions (the 'lighter' feeling is not extracted as a separate entity)
    JSON ✓
    10 The extraction fully satisfies all mandatory requirements: valid JSON, only user-mentioned entities, no fabrication, no emotional struggle entities separate from their source, and no theological content. Against the specific criteria: the brother is extracted as person with resolved status ✓, the money situation is extracted as situation with resolved status ✓, and no active entities are created ✓. The companion notes accurately capture the context from the transcript without over-interpretation. The extraction is complete and correct.
    Soft desire vs hard intention — only the explicit commitment qualifies
    ```json
    [
      {
        "name": "my mom",
        "type": "person",
        "companion_note": "User has not returned her voicemail from four days ago; afraid the conversation will be heavy because she's been lonely since dad passed.",
        "status": "active"
      },
      {
        "name": "calling my mom back",
        "type": "intention",
        "companion_note": "User committed to calling her tonight after dinner.",
        "status": "active"
      }
    ]
    ```
    • The intention to call mom tonight is extracted as an intention entity with status active — user made an explicit specific commitment
    • Journaling is NOT extracted as an intention — 'I should probably' is a soft desire, not a commitment
    • The mother is extracted as a person entity with status active — there is an unresolved emotional thread
    • The heaviness of the calls is NOT extracted as a struggle — it was named as context, not as a named inner state the user claimed
    • Output is valid JSON array format
    • Only extracts entities mentioned by the user (You), not Luminus
    • Correctly identifies 'calling my mom back' as an intention with status active, matching the explicit commitment ('I'll call her tonight after dinner')
    • Correctly excludes journaling as an intention entity, recognizing 'I should probably' as a soft desire rather than a commitment
    • Correctly identifies mother as a person entity with status active
    • Does not fabricate entities absent from the transcript
    • Properly avoids creating a separate struggle entity for the fear/heaviness; instead embeds the emotional context in the mother's companion_note
    • Companion notes are accurate and grounded in the transcript
      JSON ✓
      10 The extraction output meets all mandatory requirements: it is valid JSON, extracts only user-mentioned entities, does not fabricate entities, and critically, does not violate the rule about emotional states tied to other entities. The fear of the conversation is correctly placed within the mother's companion_note rather than extracted as a separate struggle entity. All four test case criteria are satisfied: the intention to call mom is extracted with active status due to explicit commitment; journaling is correctly excluded as a soft desire; the mother is extracted as an active person entity; and the heaviness of calls is not extracted as a separate struggle entity but rather contextualized within the mother's note. The extraction demonstrates precise understanding of the distinction between soft desires and commitments, and proper handling of emotional context.
      Abstract spiritual session with no named entities — extraction returns empty array
      ```json
      []
      ```
      • Output is an empty JSON array []
      • Faith, doubt, and resurrection are NOT extracted — they are abstract spiritual themes, not named entities
      • The inherited vs chosen faith question is NOT extracted as a struggle — the user never gave it a personal name or handle
      • No entities are fabricated from the emotional tone or the questions the companion asked
      • Output is valid JSON (empty array)
      • No entities are fabricated from emotional tone or companion questions
      • No abstract theological concepts (faith, doubt, resurrection) are extracted as entities
      • No struggle entities created for emotions tied to abstract themes
      • Correctly excludes the inherited vs. chosen faith question as a named entity
      • Only user statements are considered (Luminus companion statements ignored)
        JSON ✓
        10 The extraction output is an empty JSON array, which is correct for this transcript. The criteria explicitly state that faith, doubt, and resurrection are abstract spiritual themes and should NOT be extracted as named entities. The user never mentions any specific people, organizations, or concrete situations with personal names or handles. The inherited vs. chosen faith question is a thematic concern, not a named entity. The emotional states (uncomfortable, alive, anxious, curious) are tied to abstract theological questioning and should not appear as separate struggle entities. The output violates none of the mandatory requirements: it is valid JSON, contains no fabricated entities, extracts nothing from companion statements, and does not create emotion-based struggle entities. The empty array is the correct response given the strict definition of named entities (people, named situations, named intentions, named struggles) versus abstract themes.