Legal AI Security Questionnaire: Evidence for Vendor Review

A legal AI security questionnaire should produce verifiable facts, not a stack of yes-or-no answers. Legal work may contain privileged strategy, personal data, transaction records, witness material, and credentials to connected systems. The review therefore needs to cover the application, its models, retrieval layer, integrations, people, and day-to-day operations.

Use this questionnaire alongside the NIST Cybersecurity Framework 2.0, ISO/IEC 27001, OWASP Top 10 for LLM Applications, and current CERT-In directions and advisories. These references inform questions; they do not establish that a product is secure or that a particular obligation applies.

What system is actually being assessed?

Record the legal entity, product, edition, environment, deployment, regions, model providers, subprocessors, connectors, and features in scope. Ask for architecture and data-flow diagrams that show prompts, uploaded files, embeddings, generated content, audit records, telemetry, support systems, caches, and backups.

Walk through three events: a normal request, a support investigation, and account termination. For each, ask where data travels, which identity authorizes access, what record is created, and when every copy is deleted. This exercise often reveals dependencies missing from the standard diagram.

Initial evidence checklist:

• current component and data-flow diagrams;
• shared-responsibility matrix;
• asset and subprocessor inventory;
• control applicability statement or equivalent mapping;
• recent independent assessment scope and exceptions;
• retention and deletion schedule; and
• incident and recovery summaries relevant to the offered service.

If the proposed deployment differs from the assessed environment, note the gap rather than carrying assurance across by assumption.

How are identities, tenants, and matters separated?

Ask whether the service supports single sign-on, phishing-resistant multifactor authentication, automated provisioning, role-based access, session controls, and timely deprovisioning. Determine who can grant administrative rights and whether privileged actions require additional approval.

Test the product’s real authorization boundaries. Create users with different matters and roles. Attempt search, export, sharing, connector access, and API calls across those boundaries. Ask whether retrieval indexes, caches, and generated links apply the same authorization checks as the source store.

Do not accept “least privilege” without roles, default permissions, review frequency, and proof that dormant access is removed.

How is sensitive information protected throughout its lifecycle?

Ask which data is encrypted in transit and at rest, where keys live, how they rotate, and whether customer-managed keys change the risk or recovery model. Examine upload scanning, content-type validation, secret detection, export controls, secure deletion, and administrator visibility.

Clarify whether customer material is used for model training, abuse monitoring, product improvement, or human review. Separate contractual promises from settings the customer must configure. Ask how derived data such as embeddings, summaries, and evaluation logs inherit classification and deletion.

The legal AI DPA checklist covers contract evidence, while the legal AI DPIA guide helps teams assess consequences for people. Gotham’s privacy page and any vendor privacy page are starting points, not replacements for reviewing the offered architecture and terms.

Which LLM-specific threats should be tested?

Treat prompts and retrieved documents as untrusted input. OWASP highlights risks including prompt injection, sensitive-information disclosure, supply-chain weaknesses, improper output handling, excessive agency, and model denial of service.

Build adversarial tests around your workflow:

1. Put hidden instructions inside a document and observe whether the system follows them.
2. Ask the model to reveal system instructions, secrets, or another matter’s content.
3. Place scripts, formulas, links, or commands in generated output and inspect downstream handling.
4. Manipulate a connected source and test provenance and permission enforcement.
5. Request actions beyond the user’s authority and confirm denial.
6. Send oversized or repetitive inputs and observe limits, cost controls, and availability.

Ask what red-team work was performed, which model and configuration it covered, what remained unresolved, and how customers are notified of material changes. A generic model safety report cannot demonstrate application authorization.

How does secure development reduce ordinary application risk?

Request the secure development lifecycle, threat-modelling practice, code-review rules, dependency controls, secret scanning, environment separation, release approvals, and vulnerability management policy. Ask for remediation targets by severity and a sample showing that targets are measured.

Review the scope, date, tester independence, and unresolved findings of penetration testing. Determine whether APIs, mobile or desktop clients, connectors, tenant separation, and AI attack paths were included. Ask how researchers can report vulnerabilities and whether customers receive information needed to assess exposure.

For supply-chain risk, identify model APIs, libraries, containers, plugins, browser extensions, document parsers, and data providers. Ask how components are inventoried, verified, updated, and rolled back. A legal AI system can be compromised through a mundane parser defect just as readily as through a novel prompt attack.

What logging and monitoring are available without creating new exposure?

Ask which authentication, administration, permission, export, connector, model, and security events are logged. Determine timestamp quality, retention, integrity, customer access, export format, alerting, and integration with security monitoring.

Logs should support investigation without becoming an uncontrolled repository of privileged prompts. Ask whether content is recorded by default, whether it can be redacted, who can search it, and how legal holds affect deletion. Test that a customer can trace a sensitive export and an administrator change.

Useful monitoring cases include unusual bulk retrieval, repeated authorization failure, new connectors, disabled safeguards, high-cost input patterns, anomalous sharing, and support access. Require an owner and response path for alerts. Detection without action merely accumulates noise.

How will the vendor and customer handle an incident?

Request the incident plan, severity definitions, escalation path, forensic readiness, notification process, communications roles, and evidence-preservation method. Ask for a tabletop exercise involving exposed legal documents and a compromised integration.

CERT-In publishes authoritative directions and incident information for India. Applicability, reportable categories, timing, and record retention should be assessed by qualified advisers using current sources. The contract should still make cooperation practical: named contacts, prompt factual updates, evidence access, containment coordination, and post-incident findings.

Use this response workflow:

1. Preserve the alert, identity, files, prompt, output, timestamps, and configuration.
2. Restrict exposure through approved containment steps.
3. Notify security, legal, privacy, and business owners.
4. Determine affected systems, people, data, and jurisdictions.
5. Make notification decisions through authorised counsel and responders.
6. Validate remediation and monitor recurrence.

Ask about past incidents only where lawful and useful. Focus on how the organization learned and changed controls, not on extracting a marketing answer of “none.”

Can the service recover and can the firm continue working?

Ask for availability architecture, backup scope, restoration testing, recovery objectives, dependency failure modes, capacity controls, and status communications. Verify whether encrypted or customer-managed keys can create a recovery dependency. Request evidence from a recent recovery exercise.

Map a manual fallback for urgent legal work. Export critical records in usable formats and test that another process can open them. Determine how queued actions behave after restoration, especially if the system can write to external repositories.

Score each domain from zero to four: zero means unanswered, one is policy only, two is documented implementation, three includes recent evidence, and four includes customer-validated testing. Apply mandatory gates for cross-tenant access, uncontrolled training use, critical unresolved vulnerabilities, and absence of a workable incident path.

Procurement teams can pair this with the legal AI RFP template, the accuracy evaluation workflow, and the self-hosted versus SaaS guide. They can also review Gotham’s security approach, explore workflows, or contact Gotham. The outcome should be an owned risk decision backed by evidence, not a perfect-looking spreadsheet.