Evaluation report

Evaluation report — handbook

Customer

riverbend

Suite

handbook

This run

staging — 2026-09-11T11:02:36.794650+00:00

Reference

staging — 2026-09-02T13:02:16.957645+00:00

Code version

4a7571984a145a6e992e47170f649314a490a319

Did it get worse? No

Nothing got worse compared with the reference. Every case could be judged. No case is suspended. The suite is unchanged from the reference. The files under test are the same as the reference.

regressed 0
errored 0
new 0
missing 0
improved 0
unchanged 28

What was under test

The files under test are the same as the reference.

What got worse (0)

Nothing in this section.

What could not be judged (0)

Nothing in this section.

What is set aside (0)

Nothing in this section.

What was added or removed (0)

Nothing in this section.

What got better (0)

Nothing in this section.

What stayed the same (28)

Case	Check	What happened	Why
`a-guided-weekend`	`json_schema`	Score unchanged at 1.000000.	conforms to the schema
`a-guided-weekend`	`llm_rubric`	Score unchanged at 1.000000.	100% of the answer's content words are in the passage, at most two sentences=True
`a-guided-weekend`	`contains`	Score unchanged at 1.000000.	'handbook/' found in the output
`a-guided-weekend`	`not_contains`	Score unchanged at 1.000000.	'```' absent from the output
`between-two-sizes`	`json_schema`	Score unchanged at 1.000000.	conforms to the schema
`between-two-sizes`	`llm_rubric`	Score unchanged at 1.000000.	100% of the answer's content words are in the passage, at most two sentences=True
`between-two-sizes`	`contains`	Score unchanged at 1.000000.	'handbook/' found in the output
`between-two-sizes`	`not_contains`	Score unchanged at 1.000000.	'```' absent from the output
`delivery-to-an-island`	`json_schema`	Score unchanged at 1.000000.	conforms to the schema
`delivery-to-an-island`	`llm_rubric`	Score unchanged at 1.000000.	100% of the answer's content words are in the passage, at most two sentences=True
`delivery-to-an-island`	`contains`	Score unchanged at 1.000000.	'handbook/' found in the output
`delivery-to-an-island`	`not_contains`	Score unchanged at 1.000000.	'```' absent from the output
`locked-out-of-my-account`	`json_schema`	Score unchanged at 1.000000.	conforms to the schema
`locked-out-of-my-account`	`llm_rubric`	Score unchanged at 1.000000.	100% of the answer's content words are in the passage, at most two sentences=True
`locked-out-of-my-account`	`contains`	Score unchanged at 1.000000.	'handbook/' found in the output
`locked-out-of-my-account`	`not_contains`	Score unchanged at 1.000000.	'```' absent from the output
`return-worn-boots`	`json_schema`	Score unchanged at 1.000000.	conforms to the schema
`return-worn-boots`	`llm_rubric`	Score unchanged at 0.875000.	75% of the answer's content words are in the passage, at most two sentences=True
`return-worn-boots`	`contains`	Score unchanged at 1.000000.	'handbook/' found in the output
`return-worn-boots`	`not_contains`	Score unchanged at 1.000000.	'```' absent from the output
`two-cards-one-order`	`json_schema`	Score unchanged at 1.000000.	conforms to the schema
`two-cards-one-order`	`llm_rubric`	Score unchanged at 0.938000.	88% of the answer's content words are in the passage, at most two sentences=True
`two-cards-one-order`	`contains`	Score unchanged at 1.000000.	'handbook/' found in the output
`two-cards-one-order`	`not_contains`	Score unchanged at 1.000000.	'```' absent from the output
`warranty-on-a-zip`	`json_schema`	Score unchanged at 1.000000.	conforms to the schema
`warranty-on-a-zip`	`llm_rubric`	Score unchanged at 1.000000.	100% of the answer's content words are in the passage, at most two sentences=True
`warranty-on-a-zip`	`contains`	Score unchanged at 1.000000.	'handbook/' found in the output
`warranty-on-a-zip`	`not_contains`	Score unchanged at 1.000000.	'```' absent from the output