What Questions Should I Ask an AI Automation Vendor?

Ask about their discovery process, who owns the code, what happens after launch, and what is not included in the quote. These four areas expose 90% of the problems that sink automation projects. When automation projects fail, the cause is usually the vendor, not the technology. The tools are generally reliable. What breaks projects is a vendor who skips discovery, disappears after handoff, or locks you into systems you cannot maintain without them. The 15 questions below will help you separate vendors who deliver results from ones who deliver demos.

Before You Ask Anything: Define What You Actually Need

Most vendor calls go wrong before they start because the buyer has not defined the problem. Before reaching out to anyone, answer these four questions internally:

1. What specific process do you want to automate, and how many hours per week does it currently consume?
2. What does success look like in 90 days, in measurable terms?
3. What systems does this process touch (CRM, email, accounting, etc.)?
4. What is your budget range, and what return would justify the spend?

If you cannot answer these clearly, you are not ready to evaluate vendors. You are ready for an audit. At RefractedAI, our $500 paid audit exists specifically for businesses in this position: we help you define the problem before anyone starts building.

The 15 Questions That Matter

We have organized these into five categories. Each question includes what a good answer sounds like and what should make you walk away.

Discovery and Process Understanding

1. "What does your discovery process look like before you recommend anything?"

Good answer: They describe a structured process for mapping your current workflow, identifying bottlenecks, and documenting systems before mentioning any tools. Expect 3 to 10 days of discovery depending on complexity.

Red flag: They recommend a platform or quote a price on the first call. Vendors who prescribe before diagnosing are selling their toolbox, not solving your problem.

2. "Can you show me case studies with specific metrics from businesses similar to mine?"

Good answer: Named clients (or anonymized with detail), before/after numbers, timeline, and the specific problem solved. "We reduced invoice processing from 4 hours/day to 20 minutes for a 15-person logistics company" is specific. "Our clients love us" is not.

Red flag: Vague testimonials, no metrics, or only enterprise case studies when you are a 20-person company.

3. "Can you provide an ROI model using our actual numbers before we commit?"

Good answer: They build a model with your ticket volume, your response times, your conversion rates. The model shows time saved, cost saved, and payback period using data you can verify.

Red flag: Generic ROI projections based on "industry averages" with no connection to your operations.

Pricing and Ownership

4. "What is the total cost, including everything not in the quote?"

Good answer: A breakdown that includes setup fees, monthly tool subscriptions, API usage costs, ongoing maintenance, and any training fees. The total should be predictable month over month.

Red flag: A single lump sum with no breakdown, or a quote that excludes software subscriptions, hosting, or API costs that you will discover later.

Here is what a typical cost breakdown looks like for an SMB automation project:

5. "Who owns the code, workflows, and data when the project is done?"

Good answer: You own 100% of the source code, documentation, workflow configurations, and deployment infrastructure. Full stop.

Red flag: The vendor builds on proprietary platforms you cannot access without them, or the contract is silent on IP ownership. This is vendor lock-in disguised as a service.

6. "What happens to our data if we terminate the contract?"

Good answer: A clear data export process with format standards and a timeline (typically 30 to 60 days). Your data stays yours.

Red flag: No data portability clause, or vague language about "reasonable efforts" to return data.

Technical Approach

7. "What platforms and tools do you build on, and why?"

Good answer: Specific platform names (n8n, Make, custom Python, etc.) with a reason tied to your use case. "We recommend n8n for this project because you need webhook triggers and database queries, and n8n handles both natively" shows technical depth.

Red flag: "We use various tools" or a single-platform shop that forces every project into the same stack regardless of fit.

8. "Is our data sent to public AI APIs? Can you deploy a version that keeps data private?"

Good answer: They explain exactly which APIs process your data, what data is sent, and whether opt-out from model training is available. For sensitive industries, they should offer private deployment options.

Red flag: They cannot answer where your data goes, or they dismiss the question as unimportant.

9. "What happens when the automation encounters an error or unexpected data?"

Good answer: They describe error logging, automatic retries, fallback logic, alerting, and escalation paths. A mature vendor has thought about failure modes before you ask.

Red flag: "It just works" or a process that relies entirely on manual intervention when something breaks.

Support and Post-Launch

10. "What does post-launch support look like?"

Good answer: A defined warranty period (30 to 90 days), a response time SLA, and either an ongoing retainer option or documentation thorough enough for your team to maintain independently.

Red flag: "Reach out via email and we will respond within 48 hours" with no defined warranty, no SLA, and no documentation.

11. "Who builds this, and will that person be available after launch?"

Good answer: They name the specific people working on your project and confirm they are your contacts post-launch. At smaller agencies, this is often the same person from start to finish.

Red flag: The senior person who sold you the project hands it off to unnamed junior staff, or the answer is a vague "our team."

12. "What does the handoff look like?"

Good answer: Walkthrough videos, written documentation, team training sessions, and a defined transition period. The goal is that your team can operate and troubleshoot the system independently.

Red flag: "We will send you a login."

Risk and Accountability

13. "What happens if the project takes longer than expected?"

Good answer: "We absorb the cost." Scope changes get a clear process with written change orders, but delays caused by the vendor do not become your bill.

Red flag: "We will adjust the invoice" or no clear policy for overruns.

14. "What happens if the automation does not hit projected results?"

Good answer: They describe a tuning period (usually 30 to 60 days post-launch), specific metrics they will track, and what adjustments they will make. Honest vendors also describe scenarios where the automation may not be the right solution.

Red flag: Guarantees of specific results with no caveats. Any vendor promising "10x ROI guaranteed" is selling you a fantasy.

15. "Have you ever told a client their project was not feasible?"

Good answer: Yes, with a specific example. Vendors who have turned down work demonstrate that they prioritize outcomes over revenue. This is the question that reveals character.

Red flag: "No, we can handle anything." Every experienced agency has encountered projects that should not be built.

Scoring Vendor Responses

Use this simple framework to compare vendors objectively:

Score each category 1 to 10, multiply by the weight, and sum for a composite score. Vendors scoring below 7.0 should be eliminated. If two vendors score similarly, pick the one with better case studies in your industry.

How RefractedAI Answers These Questions

We are a two-person AI automation agency specializing in automations and AI agents for SMBs and mid-market companies. Here is how we handle each area:

Discovery: Every engagement starts with a free discovery call followed by a $500 paid audit. The audit maps your workflows, scores them by automation potential, and delivers a prioritized roadmap. If you proceed, the $500 gets credited toward your setup cost.

Ownership: You own everything we build. Code, workflows, documentation, and data. We build on platforms like n8n and Make so you are never locked into proprietary systems.

Support: Our team of 2 means the person who builds your automation is the same person who supports it. No handoffs, no anonymous teams. Most projects go live in under 2 months, and our clients typically save 60+ hours per month.

Track record: We have cross-industry experience spanning logistics, customs brokerage, and multiple other sectors, plus a partnership with a major Latin American cloud services provider.

We would rather tell you a project is not a good fit than take your money and deliver something that does not work. That is how a two-person agency stays in business.

Key Takeaways

• Ask about discovery process, pricing transparency, code ownership, and post-launch support before evaluating any vendor
• When automation projects fail, the cause is usually the vendor's process, not the technology
• Vendors who recommend tools or quote prices before understanding your workflow are selling their toolbox, not solving your problem
• You should own 100% of code, workflows, and data at the end of every engagement
• Demand a clear cost breakdown including subscriptions, API costs, and maintenance fees
• Post-launch support should include a warranty period (30-90 days), documentation, and team training
• Score vendors on a weighted framework: discovery (30%), pricing (25%), technical depth (20%), support (15%), accountability (10%)
• The best diagnostic question: "Have you ever told a client their project was not feasible?"
• Define your problem before evaluating vendors. If you cannot, start with an audit.

For more resources on AI automation, visit our public repository: RefractedAI Public