What belongs in this category?
AI development products now cover several different jobs. Code-completion assistants predict the next lines inside an existing editor. Chat tools explain a repository or propose a diff. Local agents can read files, edit several modules, run tests, and execute terminal commands. Cloud agents work asynchronously in a remote environment and may open a pull request. Browser app builders create, preview, host, and connect an application without a traditional local setup.
These products should not share one generic “coding quality” score. GitHub Copilot, Tabnine, Amazon Q Developer, and JetBrains AI Assistant can augment an established IDE. Cursor and Windsurf ask teams to adopt an AI-first editor. Amp emphasizes agent work through the terminal and editor integrations. Replit, Bolt.new, and v0 combine generation with a hosted build or deployment path.
Choose the workflow before the model. A developer who wants fast completions in an approved corporate IDE has a different requirement from a founder asking an agent to produce and deploy a complete application. An organization may accept cloud inference for an open-source repository while prohibiting it for unreleased product code, credentials, customer data, or regulated logic.
Permissions are part of the product
An agent that can edit files and run shell commands is more capable than a chat box, but it also has a larger failure radius. Document whether the tool can read the whole repository, traverse outside the workspace, access environment variables, use the network, install packages, call MCP servers, modify infrastructure, commit code, open pull requests, or deploy. Check whether each action is shown before execution and whether administrators can restrict models, tools, repositories, and paid usage.
Start new agents in a disposable branch or isolated environment with test credentials. Use least-privilege tokens, block production secrets, review proposed commands, and keep deployment approval outside the agent. Treat third-party MCP servers and extensions as additional software suppliers: each can receive context or perform actions beyond the base assistant.
Content exclusion deserves a feature-specific check. A provider may offer repository or file exclusions for completion and chat while not applying the same controls to agent mode, code review, symbolic links, remote workspaces, or terminal output. Read the documented limitations instead of assuming one privacy setting covers every surface.
Compare privacy and deployment honestly
Map the data flow for each recurring task. Record which source files, prompts, diffs, logs, dependency manifests, embeddings, telemetry, and feedback leave the machine; which company receives them; which model provider processes them; where they are stored; how long they are retained; whether they may improve models; and how deletion or opt-out works. Individual, business, enterprise, API, and bring-your-own-key plans can have different answers.
Local model support does not automatically make an entire product local. Authentication, routing, repository indexing, telemetry, account metadata, or some agent features can still use vendor infrastructure. Conversely, a cloud product may provide stronger organizational controls, zero-retention agreements, regional processing, audit logs, and indemnity than a lightly governed local installation. Evaluate the complete architecture and contract.
Never put secrets in prompts or source files. Use secret managers and separate development credentials. For sensitive code, test with a sanitized repository until security, legal, privacy, and procurement owners approve the intended feature and plan.
Price the approved change, not the prompt
AI coding pricing may combine seats, completions, chat requests, model multipliers, credits, token processing, agent steps, background environments, code review, deployment, databases, bandwidth, and provider API charges. “Unlimited completion” does not mean unlimited frontier-model agents. A short request can become expensive when the tool repeatedly reads a large repository, runs commands, generates files, or repairs its own failed changes.
Use a representative task set: a small bug, a cross-file feature, a test-writing task, a framework upgrade, and a security-sensitive change. Record agent cost, elapsed time, human review time, rework, tests, rejected diffs, and infrastructure charges. Divide total cost by merged changes that pass review. Set account and organization budgets before enabling automatic or background work.
Hosted builders need a second cost model for the application they create. Include preview environments, compute, database, storage, egress, domains, observability, backups, and migration effort. Confirm whether the project is private, what license applies when it is public, and whether source code and data can be exported to another host.
A practical evaluation workflow
Create a small repository that resembles production without containing confidential data. Include established conventions, an incomplete feature, tests, a dependency with a known issue, a permission boundary, and an ambiguous requirement. Give every candidate the same tasks and approved context.
Review whether the assistant asks clarifying questions, follows repository instructions, limits the diff, preserves architecture, adds meaningful tests, exposes assumptions, handles failures, and stops before consequential actions. Run the normal formatter, type checker, unit and integration tests, static analysis, dependency scan, secret scan, and manual security review. Inspect every generated dependency and license.
For browser builders, also test accessibility, responsive layout, authentication, authorization, data validation, error states, environment-variable handling, backup and rollback, Git synchronization, custom-domain configuration, and export. A successful preview is only the beginning of software delivery.
The final decision should be based on accepted changes, controllability, governance, data handling, total cost, and exit options—not the most impressive one-shot demo. Read the best AI coding assistants guide for a role-based shortlist and the Cursor versus GitHub Copilot comparison for the editor-adoption decision.
