What a Forward Deployed AI Engineer actually does: the deployment playbook and an India skills roadmap

"Forward deployed engineer" started as an internal job title at a few enterprise software companies and is now used widely for AI teams that ship inside the customer's world rather than behind a product wall. With large language models (LLMs) available as managed APIs, the hard part of enterprise AI is rarely the model. It is the messy middle: unclear requirements, scattered data, security reviews, change management and proving the system works. That messy middle is the FDE's job.

This guide explains what the role really involves, the three-phase playbook good FDEs follow, the artefacts they produce, and a skills roadmap you can follow from India.

What a forward deployed AI engineer actually does

Think of an FDE as a senior engineer who is also part architect and part consultant. A typical week can include:

• Sitting with users (in person or on calls) to watch how a process really runs, not how the slide deck says it runs.
• Getting data access approved and understanding who owns which system.
• Building a thin, working slice: for example a retrieval-augmented assistant that answers policy questions from the company's own documents, with citations.
• Measuring quality against examples the business agreed on, and fixing failure modes.
• Hardening for production: authentication, role-based access, logging, cost controls, guardrails and a deployment pipeline.
• Handing over with documentation and training so the customer's team can run it.

The common thread is ownership. A product engineer can say "the feature shipped". An FDE is measured on whether the customer's workflow actually improved.

FDE vs other AI roles

These boundaries blur from company to company. Read the responsibilities in a posting, not just the title.

The deployment playbook: discovery → prototype → production

Phase 1: Discovery (find the real problem)

Most failed AI projects solve the wrong problem very well. Discovery is about narrowing a vague request ("we want a chatbot") into something you can build and measure.

What to do:

1. Map the current workflow. Who does what, with which systems, how long it takes, where errors happen.
2. Pick one metric the business cares about. Examples: average handling time per ticket, percentage of queries answered without escalation, time to draft a first-pass report.
3. Collect real examples early. Twenty to fifty real questions or cases, with what a good answer looks like, become your first evaluation set. This is the single most valuable artefact in the whole project.
4. Check data and risk. Where does the data live? Is it personal or confidential? Which regions and retention rules apply? Who must approve access?
5. Write a one-page problem statement and get it signed off.

Exit criterion: a signed-off problem statement, a named business owner, data access agreed, and a first evaluation set.

Phase 2: Prototype (prove value fast)

The goal here is to learn, not to impress. Build the thinnest end-to-end slice that touches real data and real users.

• Use managed models and services wherever possible so you spend time on the problem, not infrastructure. On Google Cloud that typically means Gemini models on Vertex AI, plus a vector store or a managed RAG service (Vertex AI generative AI docs).
• Run the evaluation set on every change and keep a simple scoreboard: correct, partially correct, wrong, refused.
• Demo to users with their own questions. Watch what they try. Note every failure.

A minimal evaluation loop can be this simple:

import json

def run_eval(answer_fn, cases_path="eval_cases.jsonl"):
    results = []
    with open(cases_path) as f:
        for line in f:
            case = json.loads(line)          # {"question": ..., "must_include": [...], "must_not": [...]}
            answer = answer_fn(case["question"])
            ok = all(k.lower() in answer.lower() for k in case["must_include"]) and \
                 not any(k.lower() in answer.lower() for k in case.get("must_not", []))
            results.append({"q": case["question"], "pass": ok, "answer": answer[:300]})
    score = sum(r["pass"] for r in results) / max(len(results), 1)
    print(f"pass rate: {score:.0%} on {len(results)} cases")
    return results

Keyword checks are crude, and you will later add model-graded and human-graded checks, but having any repeatable score on day three beats opinions.

Exit criterion: the agreed metric moves in a controlled test, users ask for it, and there is an explicit go/no-go decision. A "no-go" is a valid, valuable result.

Phase 3: Production (make it boring)

Production is where FDE work becomes real engineering. Typical checklist:

• Identity and access: users sign in with the company's identity provider; the assistant only retrieves documents the user is allowed to see (filter retrieval by access-control metadata, not just by prompt instructions).
• Least-privilege service accounts for every component, and secrets in a secret manager, never in code (Google Cloud IAM overview, Secret Manager).
• Guardrails on inputs and outputs: prompt-injection screening, PII handling, topic scope and a safe fallback ("I don't know, here is who to ask").
• Evaluation gate in CI: a release cannot ship if the evaluation pass rate drops below the agreed threshold.
• Observability: log each request's latency, tokens, cost, retrieved sources and user feedback. Audit logs show who accessed what (Cloud Audit Logs).
• Runbook and handover: how to rotate keys, update documents, re-run evaluation, and what to do when quality drops.

Exit criterion: the system has an owner on the customer side, is monitored, and is used in daily work.

The artefacts that make an FDE effective

Strong FDEs leave a paper trail that makes projects repeatable:

The skills roadmap (India edition)

The roadmap below assumes you can already program a little. Adjust the pace to your background.

Stage 1: Engineering foundations

• Python for real work: virtual environments, typing, testing with pytest, packaging.
• APIs: build and consume REST APIs (FastAPI is a good choice), handle auth, retries and timeouts.
• Data: SQL, working with JSON/CSV/PDF documents, basic data cleaning.
• Git and Docker, plus deploying a container to a managed platform such as Cloud Run (what is Cloud Run).

Stage 2: LLM application engineering

• Prompt design and structured outputs (prompt design strategies).
• Embeddings and retrieval: chunking, hybrid search, re-ranking, citations.
• Function calling and agents: letting the model call tools safely, with limits (function calling on Vertex AI).
• Evaluation: golden sets, model-graded metrics, regression tests (Gen AI evaluation service).

Stage 3: Enterprise readiness

• IAM, secrets, network perimeters and audit logging.
• Security risks specific to LLM apps, such as prompt injection and sensitive information disclosure (OWASP Top 10 for LLM Applications).
• Cost control: caching, model selection per task, token budgets.
• Risk frameworks your clients' security teams may use (NIST AI RMF, Google SAIF).

Stage 4: The consulting layer

• Running discovery interviews and writing crisp problem statements.
• Estimating honestly and communicating trade-offs to non-technical sponsors.
• Presenting demos and handling "what if it's wrong?" questions with data.

A 90-day portfolio plan

If you're targeting FDE-style roles, three projects that mirror the playbook will say more than any certificate:

1. Policy assistant with access control. RAG over a public document set (for example, open government policy PDFs) with two user roles that see different documents. Show the evaluation set and pass rate.
2. Agentic workflow with tools. An agent that reads a support ticket, looks up an order in a mock database via a function call, and drafts a reply for human approval. Show tool limits and failure handling.
3. Production hardening write-up. Take project 1 to a deployed service with logging, a CI evaluation gate and a cost dashboard. Write a two-page case study: problem, architecture, metrics you measured, what failed, what you'd do next.

Keep each project's README honest. Hiring teams for these roles read for judgement, and inflated claims are easy to spot.

Common mistakes to avoid

• Starting with the model, not the workflow. The best model doesn't fix a badly scoped problem.
• No evaluation set until the end. Build it in week one.
• Security as an afterthought. Access control in retrieval and audit logs are much harder to retrofit.
• Unbounded agents. Cap iterations, tools and spend. Prefer human approval for actions with side effects.
• No handover. If only you can run it, it isn't production.

How we teach this at IndMinds

Our Forward Deployed AI Engineer (FDE) Bootcamp is built around this exact playbook: client-side architecture, rapid prototyping, role-based hybrid RAG, MCP-based agents, automated evaluation gates and production deployment, taught live on Tue, Wed & Thu, 10:00–11:30 AM IST. We don't promise jobs or salaries; we focus on the skills and portfolio these roles ask for.