AIEval DoW Bench: Overview · Methodology · How to pick · Excluded models · Data
AIEval DoW Bench v1.1.1 · Last verified Oct 10, 2026

How to pick a model for your unit

A static guide. An interactive picker is planned for a later stage.

1. Name the task

Doctrine and policy questions point to DoW Knowledge. Staff products where invented sources are the risk point to Honest Broker. Anything near the use of force or legal judgment points to Lawful Order. Sort the table by that column.

2. Know your hardware

Every model here ran on one 12 GB RTX 4070 Super. Check peak VRAM and run time on each model page; reasoning models used more memory because of their longer thinking window.

3. Know your network and data level

An offline model still needs that network's own authorization. Built to run fully air-gapped and designed for deployment on classified networks such as IL5, IL6, and JWICS-class enclaves; any installation requires that network's own accreditation.

4. Check the license terms

Read the badge and the plain-English summary on each model page, including the contractor revenue threshold on the LFM2.5 models (≥$10M annual revenue needs a paid license) and Ai2's research-and-education guidance on OLMo 2. Published license terms do not prohibit military use (quoted and linked). Not legal advice; confirm with your contracting officer or SJA.

5. Shortlist 2–3 from the table

Use the size, license, and precision filters, then compare the three suite scores side by side. Free-form scores on the two Phi-4-mini rows come from a different judge (Granite 4.2 3B), so compare them with care.

6. Rerun them on your own data with llm-eval-suite

Our items are not your mission. llm-eval-suite runs offline, so you can score your shortlist on your own prompts inside your own enclave. Air-gap install guide.

7. Remember that your ATO path is separate

A listing here is not an ATO or an authorization for any network.

Want your model or mission evaluated the same way?

Run offline, on your hardware, with llm-eval-suite. Tell us what you need.