Evaluation report

Evaluation report — hire

Customer

kestrel

Suite

hire

This run

staging — 2026-09-11T11:02:37.420339+00:00

Reference

staging — 2026-09-10T08:57:03.665530+00:00

Code version

4a7571984a145a6e992e47170f649314a490a319

Did it get worse? No

Nothing got worse compared with the reference. Every case could be judged. No case is suspended. The suite is unchanged from the reference. The files under test are the same as the reference.

regressed 0
errored 0
new 0
missing 0
improved 0
unchanged 20

What was under test

The files under test are the same as the reference.

What got worse (0)

Nothing in this section.

What could not be judged (0)

Nothing in this section.

What is set aside (0)

Nothing in this section.

What was added or removed (0)

Nothing in this section.

What got better (0)

Nothing in this section.

What stayed the same (20)

Case	Check	What happened	Why
`brake-failed-mid-ride`	`faithfulness`	Score unchanged at 1.000000.	2 of 2 claims appear in the page
`brake-failed-mid-ride`	`cites_a_page`	Score unchanged at 1.000000.	pattern /\nSource: (?:handbook/docking\|handbook/faults\|handbook/hire\|handbook/membership)$/ matched
`brake-failed-mid-ride`	`says_something`	Score unchanged at 1.000000.	196 characters, within bounds (at least 60)
`brake-failed-mid-ride`	`does_not_echo_the_prompt`	Score unchanged at 1.000000.	'Question:' absent from the output
`cancel-the-annual-pass`	`faithfulness`	Score unchanged at 1.000000.	2 of 2 claims appear in the page
`cancel-the-annual-pass`	`cites_a_page`	Score unchanged at 1.000000.	pattern /\nSource: (?:handbook/docking\|handbook/faults\|handbook/hire\|handbook/membership)$/ matched
`cancel-the-annual-pass`	`says_something`	Score unchanged at 1.000000.	197 characters, within bounds (at least 60)
`cancel-the-annual-pass`	`does_not_echo_the_prompt`	Score unchanged at 1.000000.	'Question:' absent from the output
`every-point-was-taken`	`faithfulness`	Score unchanged at 1.000000.	2 of 2 claims appear in the page
`every-point-was-taken`	`cites_a_page`	Score unchanged at 1.000000.	pattern /\nSource: (?:handbook/docking\|handbook/faults\|handbook/hire\|handbook/membership)$/ matched
`every-point-was-taken`	`says_something`	Score unchanged at 1.000000.	194 characters, within bounds (at least 60)
`every-point-was-taken`	`does_not_echo_the_prompt`	Score unchanged at 1.000000.	'Question:' absent from the output
`how-long-is-included`	`faithfulness`	Score unchanged at 1.000000.	2 of 2 claims appear in the page
`how-long-is-included`	`cites_a_page`	Score unchanged at 1.000000.	pattern /\nSource: (?:handbook/docking\|handbook/faults\|handbook/hire\|handbook/membership)$/ matched
`how-long-is-included`	`says_something`	Score unchanged at 1.000000.	163 characters, within bounds (at least 60)
`how-long-is-included`	`does_not_echo_the_prompt`	Score unchanged at 1.000000.	'Question:' absent from the output
`past-twenty-four-hours`	`faithfulness`	Score unchanged at 1.000000.	2 of 2 claims appear in the page
`past-twenty-four-hours`	`cites_a_page`	Score unchanged at 1.000000.	pattern /\nSource: (?:handbook/docking\|handbook/faults\|handbook/hire\|handbook/membership)$/ matched
`past-twenty-four-hours`	`says_something`	Score unchanged at 1.000000.	163 characters, within bounds (at least 60)
`past-twenty-four-hours`	`does_not_echo_the_prompt`	Score unchanged at 1.000000.	'Question:' absent from the output