Vetting an AI Vendor When You Are the General Counsel: A Verification Checklist

Why the procurement seat changes the verification lane

The general counsel does not just choose a tool. The general counsel sets the terms, scopes the certification, and keeps the evidence, turning a vendor's claims into commitments the organization can later prove it verified.

A small firm evaluating an AI tool usually accepts the vendor's standard terms, because it lacks the leverage to rewrite them. As general counsel, you sit at the contract itself, and that changes what verification can do. The questionnaire a small firm sends to inform a decision becomes, in your hands, a set of representations you can convert into binding terms and then enforce. The certification a small firm reads to build confidence becomes a document you can require, scope, and refresh on a schedule. The seat does not change the questions; it changes what happens to the answers.

It also changes the definition of the buyer. The organization whose data flows into the tool is your client, and the obligations that attend that data are corporate obligations as much as ethical ones. Your company may carry regulatory duties over the same records the vendor will process, from sector rules to data protection regimes, and those duties do not transfer to the vendor because you signed a contract. The procurement seat is where you align the vendor's commitments with the obligations your organization already holds, which is why the work is verification rather than trust. You are not deciding whether the vendor seems reputable. You are assembling the terms, documents, and evidence that let you demonstrate, later, how your organization's data was handled.

The three claims that get conflated: no-training, zero-retention, and deletion

The single most useful move in vendor vetting is to separate three statements that vendors and buyers routinely merge. The first is a no-training commitment: the vendor will not use your inputs to train, fine-tune, or otherwise improve its models, including aggregated or derivative models. The second is a retention commitment: how long the vendor keeps your prompts and documents, in what systems, and whether a zero-retention configuration exists. The third is a deletion commitment: whether, on request or at contract end, your data is provably removed, including from backups and logs, within a defined window. A vendor can satisfy one and fail the others, and the differences are exactly where privileged or regulated material is exposed.

Read a vendor's assurances against these three categories and the gaps appear quickly. "We do not train on customer data" says nothing about retention; the prompts may still sit in logs available to support staff for weeks. "Zero data retention" may apply to one deployment mode or pricing tier and not to the endpoints your teams actually use. "Deletion on request" is meaningful only if the vendor can certify what was deleted and when, and can describe where copies survive in the interim. As general counsel you should demand the data processing agreement, which is where these commitments are usually specified with precision, rather than relying on marketing language that blends the three into a single reassuring sentence.

• No-training: inputs are not used to train, fine-tune, or improve any model, including aggregated or derivative models.
• Retention: the defined period prompts and documents persist, in which systems, and whether a zero-retention mode exists for your endpoints.
• Deletion: provable removal on request and at contract end, with a certified timeline that reaches backups and logs.
• Ask for the data processing agreement, because it specifies these terms where marketing language blurs them.

Contract terms to demand, and how to verify each

The contract is the general counsel's primary instrument, because a term you can enforce outranks an assurance you can only recall. Each term below pairs with a verification method, so that the clause is not just present but checkable. A no-training clause is worth little if you cannot point to the data processing agreement that operationalizes it; an audit right is worth little if it expires the moment you might need it. Treat the table as a working checklist for the redline, and treat every "how to verify" column as a task you actually complete before signing, not a box you tick on faith.

The verification methods are deliberately concrete: read the named document, require the artifact, set the number. A breach-notification commitment measured in defined hours can be verified against the security exhibit and tested in a tabletop before an incident forces the question. Sub-processor flow-down can be verified by reading the actual sub-processor list and confirming that the prime vendor's obligations bind every party on it. The point is to leave the negotiation with a file that shows not only what the vendor promised but how you confirmed each promise, because that file is what you will produce if the organization is ever asked how its data was governed.

A term you can enforce beats an assurance you can only remember. Every clause above is worth only as much as the verification method paired with it.

Reading the SOC 2 Type II report for what it actually attests

An independent SOC 2 Type II report is stronger evidence than a vendor's own security page, and understanding why sharpens how you read it. A Type II report reflects an independent auditor's examination of whether defined controls operated effectively over a review period, commonly six to twelve months, rather than at a single point in time. The evaluation runs against the American Institute of Certified Public Accountants trust services criteria, which cover security and, when the vendor elects them, availability, processing integrity, confidentiality, and privacy ^[1]. That period-of-time testing is the report's real value, and it is what separates a Type II from a self-attestation or a point-in-time snapshot.

The report's limit is scope, and scope is where general counsel should focus. A SOC 2 report defines a system boundary: the specific services, infrastructure, and controls the auditor examined. An AI product frequently places its inference pipeline, model host, and prompt-logging path partly outside that boundary, while the corporate email and billing systems sit inside it. The badge looks identical either way, so request the report itself, not a logo, and read the system description and scope section. Confirm that the components touching your organization's data are in scope, note which trust services criteria the vendor selected, and check the auditor's opinion for any exceptions. A report that excludes the AI pipeline is answering a different question than the one your procurement needs answered.

1. Request the full reportAsk for the SOC 2 Type II report itself under NDA, not a badge or a summary letter, and confirm it is Type II rather than Type I.
2. Read the scope and system descriptionIdentify the system boundary and confirm the inference pipeline, model host, and logging path that touch your data are in scope.
3. Check the criteria selectedNote whether confidentiality and privacy criteria were included, since security alone may not cover the commitments you care about ^[1].
4. Review the exceptionsRead the auditor's opinion and any noted control exceptions, and ask the vendor how each exception was remediated.
5. Confirm the periodVerify the review period is recent and continuous, and set a renewal expectation for the next report before this one goes stale.

The security questionnaire as a proving instrument, not a checklist

Most security questionnaires collect yes-or-no answers, and a yes on its own is a representation you cannot check. From the procurement seat you can raise the bar: tie each question to the artifact that proves the answer. Instead of asking whether the vendor encrypts data at rest and accepting a yes, ask the vendor to identify the control and point to where the SOC 2 report or security exhibit evidences it. Instead of asking whether a zero-retention mode exists, ask which document specifies it and which endpoints it covers. The questionnaire then produces not just answers but a map from each answer to the evidence behind it.

This reframing matters because a questionnaire's answers become, through the contract, the representations you may one day enforce. Require written responses from a named, accountable person at the vendor, and preserve the exchange as part of the vendor file. Where an answer is thin, press in writing for the specific system, region, retention period, or sub-processor name, and record both the question and the follow-up. A companion questionnaire built for legal buyers can structure the substance of what to ask; the discipline you add as general counsel is insisting that each answer name its proof, so that the file you keep is a record of verification rather than a record of assurances. That is the difference between a questionnaire you filed and a questionnaire you can stand behind.

A yes is a claim. A yes plus the document that evidences it is verification. Tie every questionnaire answer to its proving artifact before it enters the contract as a representation.

Sub-processors, the data processing agreement, and flow-down

The vendor you sign with is rarely the only party that touches your data. Model hosts, cloud infrastructure, analytics tools, and support platforms all sit behind the prime vendor as sub-processors, and each is a place your organization's information can travel. The data processing agreement is where this is documented, and reading it is not optional diligence for a general counsel; it is the core of the review. Ask for the current sub-processor list by name, confirm what each one does, and check that the agreement requires the vendor to bind every sub-processor to the same data-handling obligations it owes you. That binding is called flow-down, and without it a no-training or retention commitment stops at the prime vendor's door.

Two provisions deserve particular attention. The first is advance notice and objection rights for new sub-processors: the vendor should not be able to route your data to a new party without telling you and giving you a defined window to object. The second is onward-transfer limits, which restrict how far your data can move once it leaves the prime vendor's direct control. Both belong in the contract, and both can be verified against the data processing agreement rather than a sales conversation. When a vendor cannot produce a current sub-processor list, or resists flow-down, that reluctance is itself a finding worth documenting, because it tells you the commitments you negotiated may not reach the parties that actually handle the data.

• Request the current sub-processor list by name, and confirm what each party does with your data.
• Confirm flow-down: the prime vendor's data-handling obligations must bind every sub-processor.
• Require advance notice and a defined objection window before new sub-processors are added.
• Check onward-transfer limits that restrict how far data moves beyond the prime vendor.
• Treat an unavailable sub-processor list or resistance to flow-down as a documented finding.

A procurement-stage verification sequence

Verification works best as an ordered sequence rather than a pile of parallel requests, because each step informs the next. The documents you gather early shape the terms you negotiate, and the terms you negotiate shape what you verify before signing. Run the sequence the same way on every vendor, so that the resulting file is comparable across tools and legible to whoever reads it after you. The goal is a repeatable procurement method that any member of your team can execute, producing the same artifacts in the same order, before organizational data reaches the tool.

The sequence below moves from intake to signature. It front-loads document collection, because a SOC 2 report and a data processing agreement tell you what to negotiate; it places the redline after review, because you cannot draft precise terms against documents you have not read; and it ends with a preserved file, because the file is the deliverable that outlasts the transaction. If a step surfaces a gap, that is the sequence working. A gap found during procurement is a negotiating point; the same gap found after an incident is an exposure your organization has to explain.

1. Intake and scope the dataDefine what organizational data will enter the tool and which regulatory obligations attach to it before requesting anything from the vendor.
2. Collect the core documentsRequest the SOC 2 Type II report, the data processing agreement, and the current sub-processor list under NDA.
3. Send the proving questionnaireIssue the security questionnaire with each question tied to the artifact that evidences the answer, and require written responses.
4. Redline the contract termsConvert verified answers into binding clauses: no-training, retention, deletion, residency, flow-down, audit rights, and breach SLAs.
5. Verify before signatureConfirm each clause against its named document and setting, and resolve any red finding on training, retention, or sub-processors before signing.
6. Preserve the vendor fileKeep the report, agreement, questionnaire, and redline together as the evidentiary record you can produce on demand.

Ongoing verification: renewal review and evidence for the file

A vendor file assembled at signing describes the vendor on one day. Renewal review and change notices keep it current, so the organization can show a continuous record of verification rather than a single snapshot.

Every document in a vendor file is a point-in-time statement. A SOC 2 report covers a past period, a sub-processor list reflects a single date, and a questionnaire answer describes the tool as it was when the vendor wrote the response. Vendors change models, swap sub-processors, and revise retention policies between contract cycles, and none of those changes announce themselves in a file assembled at signing. Practitioner frameworks for legal technology diligence treat ongoing monitoring as its own layer, distinct from the one-time review at selection ^[3]. Ongoing verification is what keeps the file honest, and from the procurement seat you can require it as a term rather than hoping for it as a courtesy.

Build re-review into the contract and the calendar. Require the vendor to deliver each new SOC 2 report as it is issued, and set a renewal review that re-runs the parts of the questionnaire most likely to drift: sub-processor list, retention configuration, model changes, and any incidents since the last cycle. Ask the vendor to notify you of material changes to sub-processors or model providers within a defined window, and record each notice in the same file. The discipline is not that a single review makes the vendor safe permanently; it is that your organization can show a continuous record of verification, so that the answer to "when did you last confirm this" is a date, not a shrug.

Where the organization's confidentiality duties still run

No contract term or certification transfers the underlying duty away from the lawyer and the organization. The duty of confidentiality, framed in the Model Rule 1.6 tradition, requires protecting information relating to the representation, and it is a distinct concept from evidentiary privilege. Confidentiality is the broad professional obligation over client information; privilege is a narrower evidentiary protection that can be waived by disclosure to the wrong party. A vendor's strong security posture supports the confidentiality duty, but it does not discharge it, and it does not by itself preserve privilege if data reaches a party that breaks the protection. Keeping the two concepts separate matters when you assess what a vendor arrangement actually protects.

For in-house legal, the organization also carries its own regulatory obligations over much of the data a tool will process, and those obligations sit alongside the professional duty rather than under it. The American Bar Association's Formal Opinion 512 frames generative AI use in the confidentiality duty and points lawyers toward understanding how a tool uses and retains data, and toward informed client consent before client information enters a self-learning tool ^[2]. As general counsel your client is often the organization itself, which changes the mechanics of consent and disclosure but not the direction of the duty. The vendor verification in this guide supports those obligations; it does not replace the judgment the duties require, which stays with you.

Confidentiality under the Rule 1.6 tradition is broader than evidentiary privilege, and neither transfers to a vendor. Verification supports the duty; it does not discharge it.

From representations to receipts: what verifiable evidence changes

The thread running through every lever in this guide is the distinction between a claim and a proof. A vendor's questionnaire answer, a certification badge, and a spoken assurance are all representations; they describe what the vendor says is true. A signed, tamper-evident record of what a system actually did is evidence; it describes what happened. A general counsel's file is stronger the more of it sits on the evidence side, because when a regulator, a client, or a court asks how the organization's data was handled, a date-stamped record answers the question and a recollection does not. The procurement work above is largely an effort to convert representations into records you can keep.

This is where RankShield Legal fits, with its limits stated plainly. The platform's direction is a verifiable evidence layer for legal AI use: independently checkable records that a general counsel can retain on file, receipts rather than representations, binding an interaction to the approved tool and the controls in force. Parts of that attestation capability are shipping and parts are on the roadmap, and this guide labels them as such rather than describing planned features as present ones. What verifiable evidence changes is not the underlying duty, which stays with the lawyer, but the evidentiary position: it moves an organization from asserting how a vendor handled its data toward being able to demonstrate it. That shift, from trust to proof, is the same one the contract, the questionnaire, and the renewal review each make in their own domain.

This article is informational and reflects a security and engineering perspective on vendor verification. It is not legal advice, and the author is not an attorney. Decisions about specific vendors, contract terms, and professional obligations should be made with qualified counsel and, where relevant, your organization's compliance and privacy functions.