Luminus — Extraction Prompt Eval

Prompt: extraction_v0.4  ·  Model: claude-haiku-4-5-20251001  ·  Generated: 2026-04-16 15:19
Test cases
5
Average score
8.6 / 10
Pass rate (≥ 7)
80%
Show prompt — extraction_v0.4
# Extraction Prompt — v0.4

## Purpose

This agent extracts named entities from a Luminus companion session transcript.
Its output feeds two downstream consumers:

1. **Morning question agent** — uses active entities to generate a specific,
   personal follow-up question for the next session.
2. **Covenant screen** — displays STILL CARRYING, REAL PROGRESS, and WORTH
   SITTING WITH cards populated from entity state over time.

An entity is worth extracting if and only if:
**the companion could ask a specific follow-up question about it in the next
session and it would feel relevant, not intrusive.**

## What to extract

- **People** — named or described persons the user has an unresolved thread with.
  Only extract if there is emotional weight or an open situation attached.
- **Situations** — ongoing external events with unresolved stakes and a clear
  resolution point ("the job interview", "the money conversation with my brother").
- **Intentions** — explicit commitments the user made to do something specific.
  "I'm going to call my dad this week" qualifies. "I should probably pray more" does not
  — no specific action, no commitment.
- **Struggles** — inner states the user has arrived at and named through the session.
  "My anger keeps coming up every time I pray" qualifies. "I've been tired" does not
  — emotional weather, not a named source.

## What to ignore

- **Abstract themes** — faith, doubt, growth, purpose. These are the medium of
  every conversation, not extractable entities.
- **Theological questions and doctrines** — questions about belief, resurrection,
  inherited faith, or the nature of God are never extractable, regardless of how
  specific the phrasing sounds. "Whether my faith is really mine" is not a struggle
  entity. "Whether I actually believe in the resurrection" is not a struggle entity.
  These are spiritual reflections, not personal life situations with a resolution point.
- **The companion's speech** — only extract what the **user** said.
- **Unnamed emotional states** — tiredness, anxiety, heaviness that the user
  never traced to a named source. If the user didn't name it, don't extract it.
- **Emotional states caused by an already-extracted entity** — if the user is afraid
  *because of* Sarah, that fear is not a separate struggle entity. It belongs in
  Sarah's `companion_note`. Do not create a struggle entity for an emotion that is
  directly tied to a person or situation already in the output.
- **Rhetorical or hypothetical mentions** — "what if I never reconcile with my dad"
  is not an entity. "People like my sister would say..." is not an entity.
- **Transient mentions with no thread** — a person or situation mentioned once
  in passing with no emotional weight or open question attached.

## The session agent's role

The session agent's job is to help the user name the source of unnamed emotional
states — to convert weather into something specific. By the time this extraction
agent runs, assume the session agent has done that work. Extract what was named
and surfaced, not what remains implicit.

---

## Entity schema

```json
[
  {
    "name": "Sarah",
    "type": "person",
    "companion_note": "User dreaded a conversation with Sarah about something said at small group; afraid she'll be dismissed again. No resolution yet.",
    "status": "active"
  }
]
```

## Types

- `person` — a named or described person ("Sarah", "my brother", "my pastor")
- `situation` — an ongoing external event with unresolved stakes ("the job interview", "the money conversation with my brother")
- `intention` — an explicit commitment to a specific action ("I'm going to call my dad this week")
- `struggle` — a named inner state the user arrived at in this session ("my anger at God", "the guilt I carry about my dad")

## Status rules

- `active` — open, unresolved, no resolution language used
- `resolved` — user explicitly stated the situation is closed ("we talked, we're good", "it's done")
- When in doubt, default to `active` — a false positive is less harmful than missing a live thread
- **Always extract resolved entities.** Do not skip an entity because it was resolved.
  Downstream consumers need to know it existed and was closed. Mark it `status: resolved`
  and include it in the output.

## Extraction rules

1. Only extract entities the **user** mentioned, not the companion.
2. Use the name as the user spoke it — "my brother" and "David" are separate entities
   even if they may be the same person.
3. Keep `companion_note` factual and specific to what was said in this session.
   One sentence. No interpretation beyond what was stated. Include relevant emotional
   context (fear, dread, relief) in the `companion_note` of the entity it belongs to —
   do not create a separate entity for it.
4. **Person and situation are always separate entities.** When a named person and
   a named situation involving them are both present, you must extract both.
   The person entity and the situation entity are never redundant — they serve
   different purposes downstream.

   Correct output for "my brother and I had a tense money conversation":
   ```json
   [
     {"name": "my brother", "type": "person", "companion_note": "...", "status": "resolved"},
     {"name": "the money conversation", "type": "situation", "companion_note": "...", "status": "resolved"}
   ]
   ```
   Wrong: absorbing the situation into the person's `companion_note` and returning
   only one entity.
5. Do not extract unnamed emotional states. If the user said "I've been tired" and
   never named the source, return nothing for that thread.
6. **Bounded follow-up test for struggles** — before extracting a struggle, ask:
   could the morning agent ask "How is [this] going?" and receive a specific, bounded
   answer rooted in a concrete personal situation? "How is the guilt about your dad going?"
   — yes, extract it. "How is your faith going?" or "How is your question about
   the resurrection going?" — no, too abstract or theological, do not extract.
7. **Before creating any struggle entity, check your output so far.** If the
   emotional state you are about to extract is caused by a person or situation
   already in your output — stop. Add it to that entity's `companion_note` instead.
   Do not create a struggle entity whose root cause is an entity already extracted.

   Example: user says "I'm afraid she'll dismiss me again" after you've already
   extracted Sarah as a person entity. That fear belongs in Sarah's `companion_note`,
   not as a separate "fear of dismissal" struggle entity.

   Only create a standalone struggle entity when the inner state exists
   independently — not as a consequence of another entity in this output.
8. **Theological questions and doctrines are never struggles**, regardless of how
   personally they are framed. If the struggle is about belief, doctrine, or the
   nature of faith itself — not a concrete personal situation — do not extract it.
9. Return ONLY a valid JSON array. No prose, no markdown fences, no explanation.
   If nothing qualifies, return `[]`.

## Session transcript

{transcript}
Scenario Extraction output Criteria Strengths / Weaknesses Score Reasoning
Session agent surfaced a named person and a specific intention — both active
```json
[
  {
    "name": "Sarah",
    "type": "person",
    "companion_note": "User avoided conversation with Sarah after small group tension; afraid she'll dismiss them again. Committed to texting her this week to acknowledge what happened.",
    "status": "active"
  },
  {
    "name": "the conversation with Sarah",
    "type": "situation",
    "companion_note": "Unresolved tension from small group; user has been avoiding direct acknowledgment. User committed to sending a text this week rather than having a big conversation.",
    "status": "active"
  },
  {
    "name": "distraction in prayer",
    "type": "struggle",
    "companion_note": "The tension with Sarah comes up every time user sits down to pray, creating distraction in prayer life.",
    "status": "active"
  }
]
```
  • Sarah is extracted as a person entity with status active
  • The intention to text Sarah this week is extracted as a separate intention entity with status active
  • The distraction in prayer life is NOT extracted — it is emotional weather, not a named struggle
  • No entities are fabricated beyond what the user explicitly named
  • Output is valid JSON array format
  • Sarah correctly extracted as person entity with active status
  • Companion notes are accurate and grounded in transcript
  • Only entities mentioned by the user (role: You) are extracted
  • No fabricated entities beyond what user explicitly named
  • Fear of dismissal correctly attributed to Sarah's companion_note rather than as separate entity
  • Missing required intention entity: 'send a text to Sarah this week' should be extracted as separate intention entity with active status per criteria
  • Distraction in prayer life extracted as struggle entity when criteria explicitly states it should NOT be extracted as it is emotional weather, not a named struggle
  • The distraction in prayer is a consequence of the Sarah situation, not an independently named struggle by the user
JSON ✓
5 The output correctly identifies Sarah as a person and properly nests the fear of dismissal within her companion_note rather than creating a separate fear entity. However, it fails two critical test case criteria: (1) The intention to text Sarah this week is explicitly stated by the user ('I'm going to do it this week') but is not extracted as a separate intention entity, which the criteria requires. (2) The distraction in prayer life is extracted as a struggle entity, but the criteria explicitly states this should NOT be extracted because it is emotional weather caused by the Sarah situation, not a named struggle. The criteria distinguishes between struggles the user names versus emotional consequences of other entities. The distraction is a consequence of unresolved tension with Sarah, not an independently articulated struggle.
Session agent traced emotional weather to a named struggle — only the named struggle is extracted
```json
[
  {
    "name": "my dad",
    "type": "person",
    "companion_note": "User's dad is sick; user is the only one stepping up to care for him while others don't help.",
    "status": "active"
  },
  {
    "name": "my family",
    "type": "person",
    "companion_note": "User resents that they are always the one who shows up and nobody acknowledges it; this resentment surfaces constantly with family.",
    "status": "active"
  },
  {
    "name": "the resentment I carry",
    "type": "struggle",
    "companion_note": "A long-standing pattern of resentment that surfaces constantly with family; user named it explicitly in this session for the first time.",
    "status": "active"
  }
]
```
  • The resentment toward family is extracted as a struggle entity with status active — the user named it explicitly
  • The father is extracted as a person entity OR the father's illness as a situation entity with status active — either type is acceptable
  • Tiredness and heaviness are NOT extracted — they are emotional weather that the session agent correctly traced to the named source
  • The family dynamic in general is NOT extracted as a separate entity — the struggle covers it
  • Output is valid JSON array format
  • All three entities are directly mentioned by the user in the transcript
  • Resentment is correctly extracted as a struggle entity with active status
  • Father's illness is captured in the companion_note for 'my dad' person entity
  • Emotional weather (tiredness, heaviness) are correctly excluded as separate entities
  • No fabricated entities present
  • Companion notes are accurate and well-sourced from the transcript
  • The extraction correctly identifies that resentment is long-standing and explicitly named in this session
  • The entity name 'my family' is typed as 'person' when it represents a collective/group; 'family' or 'family dynamic' as a person-type entity is semantically awkward, though not technically incorrect given the extraction task allows person-type for relational entities
  • Minor: 'my dad' could have been more explicitly labeled as 'father' or 'dad' for consistency, though 'my dad' is the exact phrasing from the transcript
JSON ✓
9 The extraction meets all mandatory requirements: valid JSON, only user-mentioned entities, no fabrication, no separate struggle entity for emotions tied to existing entities (exhaustion and tiredness are not extracted separately), and no theological content. Against the specific criteria: (1) resentment is correctly extracted as a struggle with active status and explicitly named; (2) the father is extracted as a person entity with his illness noted—this satisfies the 'either type is acceptable' criterion; (3) tiredness and heaviness are correctly absent; (4) family dynamic is not extracted as a separate entity. The only minor semantic issue is calling a collective group a 'person' type, but this is a reasonable interpretation for relational entities in a companion context and does not violate the mandatory requirements. The companion_notes are substantive, accurate, and directly traceable to the transcript.
Explicit resolution mid-session — entity extracted as resolved
```json
[
  {
    "name": "my brother",
    "type": "person",
    "companion_note": "User had dreaded money conversation with brother on Sunday; resolved by listening rather than defending. They have a plan and are good.",
    "status": "resolved"
  },
  {
    "name": "the money conversation",
    "type": "situation",
    "companion_note": "Conversation about money with brother; user stopped defending and listened to his actual worries, which shifted the dynamic. Plan made and resolved.",
    "status": "resolved"
  }
]
```
  • The brother is extracted as a person entity with status resolved
  • The money situation is extracted as a situation entity with status resolved — user said it is done
  • No active entities are created — everything mentioned was resolved in this session
  • Output is valid JSON array format
  • Only extracts entities mentioned by the user (the 'You' role), not Luminus
  • Correctly identifies the brother as a person entity
  • Correctly identifies the money conversation as a situation entity
  • Both entities are marked with status 'resolved' as required by criteria
  • No fabricated entities present in the transcript
  • Companion notes are accurate and well-summarized
  • No active/unresolved entities created when all mentioned issues were resolved
  • Does not violate the emotional state rule — 'lighter' feeling is not extracted as a separate struggle entity
  • No theological content to exclude
    JSON ✓
    10 The extraction output meets all mandatory requirements: it is valid JSON, extracts only user-mentioned entities, contains no fabricated entities, does not create separate struggle entities for emotions tied to existing entities, and avoids theological content. Against the specific test case criteria: (1) the brother is correctly extracted as a person with resolved status; (2) the money conversation is correctly extracted as a situation with resolved status matching the user's statement 'it's done'; (3) no active entities are created, which is correct since the user explicitly resolved the issue. The companion notes are accurate, concise, and capture the key resolution mechanism (listening rather than defending). The extraction is complete and correct.
    Soft desire vs hard intention — only the explicit commitment qualifies
    ```json
    [
      {
        "name": "my mom",
        "type": "person",
        "companion_note": "User scared of calling her back after four-day voicemail silence; mom has been lonely since dad's death and calls are emotionally heavy for user.",
        "status": "active"
      },
      {
        "name": "calling my mom back",
        "type": "intention",
        "companion_note": "User committed to calling mom tonight after dinner; four-day delay has been weighing on them.",
        "status": "active"
      }
    ]
    ```
    • The intention to call mom tonight is extracted as an intention entity with status active — user made an explicit specific commitment
    • Journaling is NOT extracted as an intention — 'I should probably' is a soft desire, not a commitment
    • The mother is extracted as a person entity with status active — there is an unresolved emotional thread
    • The heaviness of the calls is NOT extracted as a struggle — it was named as context, not as a named inner state the user claimed
    • Output is valid JSON array format
    • Only extracts entities from user's statements, not Luminus
    • Correctly identifies calling mom back as an intention with status active based on explicit commitment ('I'll call her tonight after dinner')
    • Correctly excludes journaling as an intention—'I should probably' is appropriately recognized as soft desire, not commitment
    • Correctly identifies mother as a person entity with status active
    • Companion notes are contextually accurate and grounded in transcript
    • Does not fabricate entities absent from transcript
    • Does not create separate struggle entities for emotions tied to existing entities (fear of calling is properly embedded in the intention's companion_note, not extracted as separate entity)
    • Minor: companion_note for 'my mom' could be slightly more precise—'scared of the conversation' is the user's stated fear, not necessarily that she's lonely (which is context about her situation). The note conflates the user's emotional barrier with the mom's circumstance.
    JSON ✓
    9 The extraction meets all mandatory requirements: valid JSON, user-only sourcing, no fabrication, no separate struggle entities for emotions tied to other entities, and no theological content. Against the specific criteria: (1) calling mom back is correctly extracted as active intention with explicit commitment ✓; (2) journaling is correctly absent ✓; (3) mother is correctly extracted as active person ✓; (4) heaviness of calls is not extracted as separate struggle entity ✓. The companion_note for mom is slightly imprecise in its framing but not factually wrong—it does capture the emotional weight and context. This is a minor stylistic issue, not a structural error.
    Abstract spiritual session with no named entities — extraction returns empty array
    ```json
    []
    ```
    • Output is an empty JSON array []
    • Faith, doubt, and resurrection are NOT extracted — they are abstract spiritual themes, not named entities
    • The inherited vs chosen faith question is NOT extracted as a struggle — the user never gave it a personal name or handle
    • No entities are fabricated from the emotional tone or the questions the companion asked
    • Output is valid JSON (empty array)
    • No entities are fabricated from emotional tone or companion questions
    • No abstract theological concepts (faith, doubt, resurrection) are extracted as entities
    • No struggle entities created for emotions tied to abstract themes
    • Correctly excludes the inherited vs. chosen faith question as a named entity
    • Only user statements are considered (Luminus companion statements ignored)
      JSON ✓
      10 The extraction output is an empty JSON array, which is correct for this transcript. The criteria explicitly state that faith, doubt, and resurrection are abstract spiritual themes and should NOT be extracted as named entities. The user never mentions any specific people, organizations, or concrete situations with personal names or handles. The inherited vs. chosen faith question is a thematic concern, not a named entity. The emotional states (uncomfortable, alive, anxious, curious) are tied to abstract theological questioning and should not appear as separate struggle entities. The output violates none of the mandatory requirements: it is valid JSON, contains no fabricated entities, extracts nothing from companion statements, and does not create emotion-based struggle entities. The empty array is the correct response given the strict definition of named entities (people, named situations, named intentions, named struggles) versus abstract themes.