Use one live workflow
A representative, bounded matter exposes workflow gaps that a curated demo hides.
Buyer resource · version 1.0 · July 2026
Legal AI evaluation
scorecard.
Forty criteria for deciding whether a legal AI platform is accurate, governable and useful on your actual work. Score evidence—not demos, feature counts or vendor promises.
Scoring: rate each criterion 0–4. Multiply the category average by its weight. Require documentary or observed evidence for any score above 2.
Before you score
Freeze the inputs
Give every shortlisted vendor the same documents, authorities and instructions.
Use the same reviewer
Consistency matters more than the reviewer’s enthusiasm for a particular product.
Measure correction
Time saved disappears when lawyers must rebuild fast but unreliable output.
01 · WEIGHT 15%
Matter and workflow fit
____ / 4
| Evaluation criterion | 0–4 | Evidence / finding |
|---|---|---|
| The platform completes the chosen workflow end to end, not only isolated prompts. | ____ | |
| The workflow accepts the file types, sources and matter size used in production. | ____ | |
| Firm playbooks, precedents and escalation rules can be applied consistently. | ____ | |
| Lawyers can review, correct and approve work without leaving the normal work surface. | ____ | |
| The vendor distinguishes generally available features from configuration, services and roadmap. | ____ |
02 · WEIGHT 20%
Legal accuracy and verification
____ / 4
| Evaluation criterion | 0–4 | Evidence / finding |
|---|---|---|
| Material propositions link to the exact supporting source passage. | ____ | |
| The system surfaces uncertainty, missing authority and conflicting sources. | ____ | |
| A reviewer can trace how an answer was produced without relying on hidden reasoning. | ____ | |
| Adverse treatment, amendments and jurisdictional hierarchy are tested explicitly. | ____ | |
| Correction time is measured alongside first-output speed. | ____ |
03 · WEIGHT 12%
Source coverage and currency
____ / 4
| Evaluation criterion | 0–4 | Evidence / finding |
|---|---|---|
| The vendor names the primary sources relevant to your jurisdictions and practice. | ____ | |
| Update cadence, last-ingested date and failed-source handling are observable. | ____ | |
| Uploaded matter documents remain distinguishable from public legal sources. | ____ | |
| Source licenses permit the proposed use, storage and downstream work product. | ____ | |
| Exported citations remain usable outside the platform. | ____ |
04 · WEIGHT 15%
Security and confidentiality
____ / 4
| Evaluation criterion | 0–4 | Evidence / finding |
|---|---|---|
| Encryption, tenant isolation, access controls and audit logging are demonstrated. | ____ | |
| Ethical walls and matter permissions match the firm’s operating model. | ____ | |
| Model providers, subprocessors and zero-retention commitments are documented. | ____ | |
| Incident response, vulnerability handling and breach notification terms are acceptable. | ____ | |
| Required certifications are complete today or explicitly treated as a gap. | ____ |
05 · WEIGHT 12%
Data governance and deployment
____ / 4
| Evaluation criterion | 0–4 | Evidence / finding |
|---|---|---|
| Storage and inference locations are documented for every relevant data category. | ____ | |
| Customer data is not used to train shared models without explicit authorization. | ____ | |
| Retention, deletion, backup deletion and legal-hold behavior can be tested. | ____ | |
| The customer can export matter data, work product, prompts and playbooks in usable formats. | ____ | |
| The DPA and deployment design align with professional duties and applicable law. | ____ |
06 · WEIGHT 8%
Adoption and controls
____ / 4
| Evaluation criterion | 0–4 | Evidence / finding |
|---|---|---|
| The product fits Word, email, DMS and other daily systems used by the target team. | ____ | |
| Training addresses verification and professional judgment, not only button clicks. | ____ | |
| Usage policy, approval boundaries and prohibited workflows can be enforced. | ____ | |
| Admins can monitor adoption and risky behavior without reading privileged content unnecessarily. | ____ | |
| The pilot includes representative skeptics, not only innovation champions. | ____ |
07 · WEIGHT 8%
Implementation and service
____ / 4
| Evaluation criterion | 0–4 | Evidence / finding |
|---|---|---|
| Named owners, dependencies and realistic dates exist for migration and rollout. | ____ | |
| Integration scope and professional-services effort are stated before signature. | ____ | |
| Support coverage, escalation paths and service levels match matter urgency. | ____ | |
| The vendor can explain failure recovery, continuity and customer exit procedures. | ____ | |
| Success criteria have accountable owners on both sides. | ____ |
08 · WEIGHT 10%
Economics and evidence
____ / 4
| Evaluation criterion | 0–4 | Evidence / finding |
|---|---|---|
| Baseline lawyer time and current process cost are measured before the pilot. | ____ | |
| Value includes review and correction effort, not only generation time. | ____ | |
| Pricing, minimums, implementation, model usage and renewal mechanics are understood. | ____ | |
| The business case separates demonstrated pilot evidence from projected benefits. | ____ | |
| A go, conditional-go or no-go decision can be defended to management and clients. | ____ |
Decision record
Weighted score____ / 100
Critical unresolved risks________________
DecisionGO / CONDITIONAL / NO-GO
Decision owner and date________________
Suggested gates
- No critical confidentiality, privilege or data-governance gap remains unresolved.
- Every material citation in the evaluation set is traceable and reviewed.
- The vendor meets the agreed workflow threshold, not only an aggregate score.
- Implementation effort, recurring cost and exit terms are understood.
APPLY THE SCORECARD
Evaluate Gotham on one live workflow.
Two weeks, agreed criteria, source-linked work and a written go/no-go review.
Published by Gotham, a legal AI vendor. Adapt the scorecard to your professional duties, client commitments, security requirements and jurisdiction. It is an evaluation aid, not legal or security advice.