AI Vendor Due Diligence: The Questions to Ask

Every AI tool you adopt extends your attack surface and your data footprint into someone else’s infrastructure and someone else’s model. When you buy an AI product, or when a familiar vendor switches on an AI feature, you inherit their data practices, their security posture and the behaviour of models you cannot see inside. Due diligence is how you understand what you are taking on before it becomes your problem.

This article sets out the questions that matter, grouped by the areas where AI vendors most often fall short. It builds on general cyber due diligence principles and adds the AI-specific layers that standard assessments miss.

Data use and training: the questions that matter most

The first thing to establish is what the vendor does with the data you put in. This is where AI products differ most sharply from traditional software, and where the worst surprises hide.

• Is our input data used to train your models or any third-party foundation model? Get the answer in the contract, not just the sales deck.
• How long are prompts, outputs and uploaded content retained, and can retention be disabled or shortened?
• Is our data logically or physically segregated from other customers, and from your model improvement pipelines?
• If we delete data or terminate, what happens to data already ingested, embedded in vector stores, or used in fine-tuning?

A vendor that cannot give clear, contractual answers to these questions is telling you something important about how they treat customer data. Pay close attention to the difference between a default setting and a contractual guarantee. A vendor may say data is not used for training by default, but if that can be changed by a setting, a policy update or a different pricing tier, it is not a protection you can rely on. The commitment you want is one that cannot be silently reversed.

The model underneath

Most AI products are built on foundation models from a small number of providers. You need to know which, because that provider’s data practices, security and residency become yours by inheritance.

• Which foundation models power the product, and are they self-hosted or accessed through a third-party API?
• What contractual protections exist between you, the vendor, and the foundation model provider?
• How do you handle model updates, deprecations and behaviour changes that could affect our use?
• What controls exist against prompt injection, jailbreaks and the model leaking one customer’s data to another?

The reason this matters so much is that the vendor you contract with may have little real control over the model underneath. If they are reselling access to a third-party foundation model, then the commitments you need on data use, residency and retention have to flow all the way down to that provider. A vendor who has not secured those terms from their own supplier cannot honestly extend them to you, regardless of what their contract says. Ask to see how the chain of commitments holds together, not just the top layer.

Security posture and certifications

AI vendors should meet the same security bar you would expect of any provider handling your data, plus AI-specific controls. Ask for evidence rather than assurances.

• Do you hold recognised certifications such as ISO 27001 or SOC 2, and can we see the current report or certificate?
• How do you secure the AI pipeline specifically, including model access controls, API authentication and logging?
• What is your vulnerability management and penetration testing regime, and does it cover AI-specific attack vectors?
• How are encryption, access management and incident response handled?

For organisations pursuing or holding ISO 27001, a vendor’s certification status and their willingness to share evidence is a strong early signal of maturity. Be aware, though, that a certification covers a defined scope. A vendor may hold ISO 27001 for its corporate environment while the specific AI product, or the foundation model behind it, falls outside the certified boundary. Always check what the certificate actually covers rather than treating its existence as blanket assurance.

It is also worth probing how the vendor secures the AI-specific parts of its stack that traditional certifications were not designed to address. Ask how they prevent training data poisoning, how they protect model weights and fine-tuning data, and how they isolate one customer’s prompts and context from another’s. These are emerging areas, and a thoughtful answer signals a vendor that takes AI security seriously rather than treating it as ordinary software.

Data residency and sovereignty

Many AI services process data through overseas infrastructure, often without making that obvious. For Australian government entities and regulated organisations this can breach data residency or sovereignty requirements outright.

• Where is our data processed and stored, including by any underlying foundation model provider?
• Is Australian data residency available, and is it contractually guaranteed rather than best-effort?
• Which jurisdictions could lawfully compel access to our data, and under what circumstances?

Councils and agencies should map these answers against their obligations. Our guidance on government and council cybersecurity covers how sovereignty requirements shape procurement.

Sub-processors and the supply chain

AI products typically rely on a chain of sub-processors: the foundation model provider, cloud infrastructure, analytics, and often other AI services. Each link is part of your risk. Ask for a complete sub-processor list, how you will be notified of changes, and what diligence the vendor performs on its own suppliers. A vendor that cannot tell you who is in their supply chain cannot secure it.

Accuracy, bias and failure modes

AI vendors should be candid about what their product gets wrong. Ask how the system handles hallucination, what accuracy testing they perform, how they address bias in outputs, and what guardrails exist for high-stakes use. A vendor who claims their AI is never wrong is either naive or selling. The honest answer is always about how errors are detected, bounded and surfaced to the user.

For any product that will inform decisions about people, push harder on bias and explainability. Ask what testing has been done across the populations your organisation serves, how the vendor would help you investigate a complaint that an output was unfair, and whether the system can explain or evidence how it reached a result. Public sector buyers in particular may have to defend AI-influenced decisions to citizens, ombudsmen or courts, and a product that is a complete black box can make those obligations impossible to meet. The vendor’s willingness to engage seriously with these questions is itself a useful signal.

Contractual protections and exit

Finally, make sure the answers are enforceable. Push the key commitments into the contract: no training on your data, retention limits, residency, breach notification timeframes, audit rights and a clean exit. Confirm you can export your data and any derived assets, and that the vendor will delete it on termination. Strong verbal assurances that never make it into the agreement are worth very little when something goes wrong.

Re-assess your existing vendors too

The most overlooked risk is not new purchases but existing tools that quietly add AI features. A SaaS product your staff have used for years may now route content through a third-party model with no assessment having been done. Treat every new AI feature in an existing vendor as a fresh due diligence event.

A pragmatic way to make all of this manageable is to tier your diligence to the risk. A tool that handles no personal information and merely drafts internal text does not need the same scrutiny as one that processes customer records or feeds decisions about people. Define a small set of risk tiers, map your questions to each, and reserve the deepest assessments for the AI products that touch your most sensitive data or your most consequential processes. This keeps diligence proportionate without letting the high-risk cases slip through on a light-touch review.

AI vendor due diligence spans security, privacy, legal and the business, which is why it works best coordinated by someone accountable for the whole picture. CISO Advisory runs AI vendor assessments for Australian government and enterprise clients through our Virtual CISO and due diligence services. To put a vendor through its paces, call 07 2112 8502 or get in touch.