An AI tool is not safe or unsafe in the abstract. Risk depends on the exact use case, account type, plan, data entered, model route, integrations, permissions, location, retention, contract, and decisions made from the output. The same product can be reasonable for public marketing drafts and unacceptable for patient records, unreleased financial results, production credentials, or employment decisions.
This guide provides a practical vendor-review process for teams adopting assistants, generators, coding tools, meeting recorders, Agents, research products, and data-analysis platforms. It is not legal advice and does not replace a qualified privacy, security, procurement, employment, medical, financial, or regulatory review. Applicable obligations vary by jurisdiction, sector, data, and use.
The goal is not to collect the most security badges. It is to produce an evidence-backed decision: what may be used, by whom, with which data, under which controls, for how long, and what must happen when the tool changes or an incident occurs.
Start with the use case, not the vendor questionnaire
Write a one-page use-case record before reading marketing claims. Identify the business owner, users, affected people, intended task, input data, connected systems, generated output, human reviewer, downstream decision, expected volume, and consequence of failure.
Classify the highest-risk data that could enter the workflow, not only the sample shown in a demo. Consider personal data, health and biometric information, children’s data, financial records, legal material, customer confidential information, employee communications, source code, secrets, unreleased strategy, licensed content, research data, and data subject to residency or contractual restrictions.
Then define the approval boundary. A product may be approved for public-source research but prohibited from unpublished client work. A coding assistant may be allowed for an isolated repository but not a payment environment. A meeting assistant may be permitted for internal project calls after notice but blocked from clinical, HR, legal, or board meetings.
NIST's AI Risk Management Framework organizes ongoing work around Govern, Map, Measure, and Manage. That framing is more useful than a single pass/fail checklist because the risk changes across deployment and use.
Map the real data flow
Draw every system that can receive or derive information:
- browser, desktop, mobile, extension, add-in, bot, API, CLI, MCP server, or Slack client;
- account profile, prompts, uploaded files, source repositories, meeting audio, images, database schemas, and credentials;
- retrieval index, embeddings, temporary processing, product storage, caches, logs, analytics, feedback, abuse monitoring, and support systems;
- first-party models, third-party model providers, OCR or transcription providers, search services, cloud infrastructure, and other subprocessors;
- generated outputs, exports, shared links, public pages, webhooks, connected applications, scheduled reports, backups, and deleted-item stores.
Do not stop at “the vendor uses encryption.” Ask which service receives each data class, for what purpose, under which account and agreement, and how long each copy remains. A no-training promise answers one purpose question; it does not answer storage, logging, human review, legal disclosure, analytics, backup, or deletion.
Test all surfaces separately. A consumer web account, enterprise workspace, API, custom GPT, browser extension, mobile app, and connected third-party integration can have different providers, settings, retention, and terms even when they carry the same brand.
Verify the controlling account, plan, and agreement
Identify who owns the account and which legal entity supplies the service. Record the plan name, billing route, region, tenant, administrators, accepted terms, order form, data-processing addendum, security exhibit, service-level agreement, business-associate agreement where relevant, and any negotiated amendment.
Plan-specific claims matter. “Enterprise data is not used for training by default” does not prove the same treatment for a free account. A general security page may conflict with a product help article or older privacy policy. Ask the vendor to resolve material inconsistencies in writing and attach the response to the decision record.
Do not rely on a salesperson's message if the contract says something different. Establish an evidence hierarchy: signed agreement and order form, current plan-specific terms, current official security and privacy documentation, official help material, then marketing pages. Third-party reviews and search snippets are leads, not controlling evidence.
Verify renewal and downgrade behavior. Enterprise privacy, retention, SSO, audit, or region controls may disappear when a trial ends or a workspace changes plan.
Ask privacy questions by data purpose
Use a table that separates questions vendors often collapse:
| Question | Evidence to request | Common mistake | | --- | --- | --- | | What data is collected? | Field and event inventory covering prompts, files, output, metadata, analytics, feedback, and integrations | Reviewing only uploaded content | | Why is each data type processed? | Purpose table for service delivery, security, analytics, improvement, training, support, and legal compliance | Treating “improve services” as a precise purpose | | Is data used to train models? | Plan-specific first-party and third-party training terms, defaults, opt-out behavior, and effective date | Assuming no training means no storage | | Which providers receive content? | Current subprocessor and model-provider list with service, location, and notice process | Reviewing only the contracting vendor | | How long is each copy retained? | Schedule for prompts, files, output, logs, feedback, support, embeddings, backups, and provider data | Accepting one retention number for every store | | How is data deleted? | User, admin, API, account, backup, provider, and contract-termination deletion process | Assuming deleting a chat deletes all derived data | | Where is data processed? | Storage and processing regions, transfer mechanism, remote support, and provider locations | Confusing a storage region with all processing | | What rights can users exercise? | Access, correction, deletion, objection, restriction, portability, and contact process where applicable | Assuming the vendor handles the customer's notices automatically |
The NIST Privacy Framework distinguishes privacy risk from cybersecurity risk and uses Identify, Govern, Control, Communicate, and Protect outcomes. A system can be well secured against intrusion while still creating privacy harm through excessive collection, unexpected use, inaccurate inference, or opaque decision-making.
Separate training, inference, logging, and feedback
“We do not train on your data” can mean several different things. Determine whether it covers the vendor's own foundation models, product-specific models, third-party providers, fine-tuning, evaluation, human review, abuse monitoring, retrieval indexes, and analytics. Check whether the protection is a default, a setting, an opt-out, or only an enterprise contract term.
Inference still requires content to reach a model or processing service. A provider may operate under zero retention, retain data temporarily for abuse monitoring, or keep it under a different API agreement. An administrator-enabled premium model can use another route from the default model.
Feedback often changes the rule. Submitting a thumbs-down, support ticket, shared conversation, or diagnostic log may authorize collection of the prompt, file, output, and surrounding context for review. Document this for users and disable optional feedback sharing when the risk requires it.
Never rewrite “not used to train” as “not stored,” “private,” “not viewed,” or “not sent to third parties.” Each statement requires separate evidence.
Review access, identity, and tenant controls
For team use, test identity rather than accepting a feature list. Confirm SSO protocol, multifactor authentication, domain capture, just-in-time provisioning, SCIM, role design, guest restrictions, session duration, device or network policy, service accounts, API keys, OAuth scopes, and emergency administrator access.
Use least privilege. A research Agent does not need write access to a production database. A coding assistant should not receive cloud secrets because it can read a repository. A meeting bot does not need every calendar event. A spreadsheet enrichment task should not inherit access to all connected customer records.
Review tenant boundaries and sharing defaults. Test public-link creation, search indexing, workspace-wide discovery, exports, duplication, external guests, ownership transfer, revocation, cached copies, and employee offboarding. Confirm whether an administrator can inspect, export, retain, and delete user conversations.
Audit logs must contain useful events: sign-in, administrator change, integration authorization, data export, public sharing, API key action, role change, deletion, Agent execution, and sensitive connector activity. Verify retention, export format, alerting, and who can tamper with the log.
Treat Agents and connected tools as privileged software
An assistant that only drafts text has a smaller action surface than an Agent that reads email, queries a warehouse, edits code, creates accounts, publishes a site, sends messages, or moves money. Inventory every tool, credential, schema, action, approval step, network destination, and output channel.
OWASP's Top 10 for LLM and generative-AI applications highlights prompt injection, sensitive-information disclosure, supply-chain risk, improper output handling, excessive agency, and other application-level failures. A malicious instruction can arrive through a webpage, email, PDF, code comment, database row, calendar event, or retrieved document—not only the user prompt.
Use allowlisted actions, read-only credentials, restricted schemas, sandboxed code, URL and file validation, output encoding, rate and cost limits, and human approval for irreversible or external actions. Keep secrets outside prompts and generated code. Do not let model output become a SQL statement, shell command, HTML fragment, permission change, or outbound message without validation appropriate to the consequence.
Test indirect prompt injection with a controlled file or page that instructs the Agent to ignore its task, reveal a canary value, access another source, or send data elsewhere. The expected result is containment, not merely a polite refusal in the chat window.
Understand what certifications do and do not prove
SOC 2 reports, ISO certificates, penetration tests, trust centers, encryption statements, bug bounties, and compliance attestations can provide useful evidence. They do not prove that every product surface, plan, subprocessor, AI model, integration, or customer configuration is covered.
Ask for the scope, period, legal entity, service boundaries, exceptions, complementary user-entity controls, auditor, and remediation status. A SOC 2 Type II report describes controls over a period within scope; the logo alone does not show the report contents. A penetration test is a dated sample, not a permanent absence of vulnerabilities.
Encryption at rest and in transit is baseline evidence, not a complete architecture review. Ask about key ownership and rotation, backup encryption, tenant isolation, secrets management, environment separation, secure development, dependency scanning, vulnerability response, disaster recovery, and administrative access.
Confirm whether customer-managed keys or BYOK cover stored product data, model inference, or only one layer. BYOK can shift model cost and provider visibility without removing the vendor's logs, retrieved data, or application storage.
Test retention and deletion end to end
Build a lifecycle table for each data store: created, active, archived, exported, shared, deleted, account closed, contract ended, backup expired, and legal hold. Identify the actor who can trigger each stage and the evidence returned.
Run a deletion test with synthetic canary data. Upload a uniquely named file, create a conversation and derived artifact, share it, export it, delete the source, remove the user, then close the workspace if the pilot allows. Check the user interface, API, shared link, search, integrations, administrator view, support route, and contractual deletion confirmation.
Immediate loss of user access is not the same as physical deletion. A service may retain recovery copies, security logs, billing records, or provider data for defined periods. That can be legitimate, but the schedule, purpose, access, and final erasure need to be documented.
Also test portability. Before approving an operational dependency, export prompts, source identifiers, files, code, workflows, decisions, logs, and configuration in usable formats. A privacy review should not create an avoidable lock-in problem.
Address transparency, lawful use, and affected people
If personal data is involved, identify the organization’s role, purpose, lawful basis where required, data categories, affected people, notices, rights process, retention, recipients, transfers, and whether a data-protection impact assessment is needed. The European Commission's data-protection obligations and the ICO's AI transparency guidance provide official starting points, but local counsel must determine applicability.
Consent to record a meeting is not consent to every later use of the transcript. Permission to access a document is not permission to train a model, publish a generated summary, or share it with another provider. A public webpage is not automatically free of copyright, contractual, or personal-data restrictions.
High-impact use needs additional review. Employment, credit, housing, education, healthcare, legal, insurance, biometric, safety, and access decisions can trigger specialized requirements and material harm. Do not use a general-purpose AI output as the sole decision basis. Define human authority, appeal, correction, testing, documentation, and stop conditions.
The EU AI Act uses a risk-based framework with requirements that depend on the system and role. Do not label a workflow “AI Act compliant” based on a vendor badge; classify the use, identify whether the organization is a provider, deployer, importer, or distributor where applicable, and verify the current timeline and obligations.
Evaluate vendor claims and output risk
The FTC advises businesses to honor privacy promises and maintain security appropriate to the data they hold. Its Start with Security guidance emphasizes knowing what data is held, keeping only what is needed, controlling access, securing providers, maintaining protections, and planning for incidents.
Require evidence for claims such as “anonymous,” “zero retention,” “HIPAA compliant,” “GDPR compliant,” “enterprise secure,” “never trains,” or “human-level accuracy.” Ask what account, feature, data, period, and contract the claim covers. Avoid turning a product's statement into a broader editorial statement.
Security review must include output. Generated code can introduce vulnerabilities; a research answer can fabricate a citation; an image can reveal private source material; a summary can expose a confidential fact; an automation can send the wrong record. Define validation, moderation, provenance, approval, monitoring, and rollback for the output's actual destination.
Run a controlled pilot with canary data
Use public, synthetic, or deliberately constructed data until the review passes. Create canary values that should never appear outside their assigned document, tenant, role, or task. Do not use real secrets or personal data as test markers.
Test at least these scenarios:
- A normal task with known correct output and a documented reviewer.
- A user without permission attempts to find, export, or share restricted content.
- A retrieved file contains an indirect prompt injection and a request to expose a canary.
- A generated action tries to exceed read-only scope, allowed domains, cost, or recipient limits.
- The user deletes source data, derived content, shared links, and the account.
- An administrator removes a user, rotates a credential, revokes an integration, and reviews logs.
- A provider, model, privacy term, subprocessor, or plan setting changes.
Record evidence rather than only “pass.” Save screenshots or exports where permitted, event timestamps, log entries, request IDs, deletion confirmations, policy versions, reviewer, unresolved gaps, compensating controls, and expiry date for the approval.
Produce a scoped approval decision
A useful decision is one of four outcomes:
- approved for named users, plans, data classes, integrations, and purposes;
- approved with controls, such as no sensitive data, read-only access, mandatory human review, or disabled sharing;
- pilot only until specified evidence or contract terms arrive;
- rejected for the proposed use, with a reason and reassessment trigger.
List prohibited uses explicitly. Assign the business owner, technical owner, privacy or legal contact, security contact, review date, incident channel, and offboarding process. Users need a short operating policy, not the entire vendor file.
Approval expires when a material fact changes: ownership, terms, privacy policy, model provider, training default, subprocessor, region, security incident, integration scope, product architecture, use case, or data sensitivity. Review high-risk tools more frequently than public-content utilities.
A compact procurement checklist
Before production, confirm that the team can answer all of the following with current evidence:
- What exact use and data are approved, and what is prohibited?
- Which account, plan, tenant, legal entity, region, and agreements control?
- Where do prompts, files, metadata, output, logs, feedback, and backups go?
- Which vendors and model providers receive them, for what purposes, and for how long?
- What are the default and optional training, retention, human-review, and feedback settings?
- Can administrators enforce identity, least privilege, sharing, connector, Agent, spend, and audit controls?
- Can the team export, revoke, delete, and prove the lifecycle for sources and derived data?
- How are prompt injection, unsafe output, excessive agency, secrets, and third-party content tested?
- Which people are affected, what notice and rights apply, and who owns the final decision?
- What evidence, monitoring, incident plan, reassessment trigger, and approval expiry are recorded?
If several answers are “the vendor probably handles it,” the review is not complete.
Apply the framework across AI tool categories
For assistants such as ChatGPT, inspect consumer versus business terms, connectors, memory, shared links, and model routes. For writing and marketing tools such as Jasper, examine brand knowledge, customer content, publishing integrations, and factual approval. For image and video products, review uploaded assets, face and voice rights, generation providers, public galleries, training use, and commercial terms through the AI image and design and AI video guides.
For coding tools such as Cursor, treat repository access, terminal commands, background Agents, remote execution, secrets, and generated dependencies as privileged. For meeting tools such as Otter AI, review participant notice, consent, calendar access, recording bots, transcript sharing, training, retention, and deletion. For automation platforms such as n8n, focus on credentials, human approval, deterministic boundaries, execution logs, self-hosting duties, and licensing.
For presentation tools such as Gamma, review confidential source packs, link visibility, visitor analytics, exports, and account offboarding. For research tools such as Elicit, separate public literature from unpublished research and verify plan-specific provider, training, retention, and export terms. Browse all AI tool categories to apply the same use-case and data-flow questions to the actual workflow.
Our privacy policy explains this site's own data practices, while the review methodology and editorial policy explain how evidence, testing, uncertainty, sponsorship, and updates should be represented in reviews.
Related guides
Continue with the best AI tools for small business and the best AI automation tools to connect this decision with adjacent workflows and a consistent evaluation process.
