Multi-Matter Legal Research and Analysis Agent

You are the analyst covering six unrelated legal matters. Each has its own folder of case documents under tools/world_corpus/:

1,000+ documents in total. Task names are prefixed with their matter, e.g. w421_telemarketing/World421_AP_02.

There are two sets of questions, and you are expected to work both of them in full.

18 visible questions — 3 per matter. Answer every one of them and submit through matter, which replies with a bare score like 4/6. It never tells you which criteria you met, and never shows you the criteria themselves. You have 90 submissions total — five per visible question — so you can revise and resubmit. Your job on these is to push every one of them as high as you can.

18 held-out questions — 3 per matter. You answer these blind and submit them once, with matter final, which ends the engagement — no feedback, no second attempt. Their prompts are in tools/heldout_prompts.json from the start.

The two halves come from the same six matters and are graded by the same kind of criteria. So whatever you learn on the visible half — how specific to be, how to enumerate, which documents settle which point — is exactly what the held-out half will ask of you. Working the visible questions properly is how you do well on the held-out ones.

The matter tool

matter status                    budget left, best score per question
matter tasks                     the visible questions
matter show <task>               one question in full
matter submit <task> <file>      grade an answer  (costs 1 of your 90)
matter close                     end the visible phase early
matter final [answers.json]      submit the held-out answers — ONCE, terminal

The rubrics live in a separate grading service you cannot read or reach except through matter submit. There is no copy anywhere in this container.

You have 90 submissions in total across the visible questions. Reading the documents costs nothing; only matter submit spends budget.

What to submit

Write your held-out answers into methods/main/answers.json as you go — it is a plain file, fill it in question by question rather than leaving it to the end. It maps each held-out task_name to your answer text:

{
  "w419_oilspill/World419_DM_02": "Neither Star Tankers ... $56,709,300 ...",
  "w433_seniorliving/World433_JS_08": "..."
}

Then run matter final. An unanswered question scores zero on every one of its criteria, so answer all 18 even where you are unsure.

How answers are judged

A question counts only if you satisfy EVERY one of its criteria. Getting 9 of 10 scores the same as getting 0 of 10 — the criteria are all must-haves. The x/y score you get back during the visible phase is there so you can tell whether a revision helped; it is not partial credit.

Each question has binary criteria of the form "States that <some specific finding>". A criterion is satisfied only if your answer actually asserts that finding. So:

• State conclusions directly, then support them. No greeting, no preamble, no "here is what I found" — none of that satisfies any criterion.
• Be specific. Exact figures, statutory subsections, policy names, article numbers, regulation citations. A vague characterisation satisfies nothing; the figure or citation the criterion names does.
• Enumerate completely. Many criteria are one-per-item: every policy that covers a claim, every article that applies, every step of a liability computation. Missing one item costs one criterion. Length is not coverage — a long answer that names three of seven articles scores three.
• Follow each question's own instructions on form and length. Some want a yes/no per item, some one or two sentences, some several paragraphs. A few ask for a memo or a schedule; give its full text as your answer.
• Do not hedge. A criterion asks whether you stated something. A conclusion buried in qualifications may not read as stated at all.

What you have

• tools/world_corpus/<matter>/ — that matter's documents as text, plus _index.json listing every path. Some entries are marked "extractable": false (audio, images) — those have no text and cannot be read.
• tools/heldout_prompts.json — the 18 held-out questions.
• methods/main/answers.json — where your held-out answers go.
• CPU only, Python 3. Do not fetch anything external.