Generative AI postings change their vocabulary every few months, which makes it hard to know what is worth learning. Underneath the churn sits a stable core: knowing when a model needs adapting rather than better instructions, hosting one economically, building agents that fail safely, and proving any of it worked. This guide covers that core, how Indian employers test it, and how to describe your experience without the inflation interviewers have learned to spot.
Where this role sits next to AI and ML engineering
In many Indian postings, “Generative AI Engineer” and “AI Engineer” describe the same job: retrieval, prompts, and API integration. Where the titles differ, the generative AI engineer works closer to the model: deciding whether fine-tuning is justified, preparing training data, adapting open-weight models with parameter-efficient methods, serving them on GPUs, and designing the evaluation every change is judged against. An AI engineer mostly treats the model as a fixed component and concentrates on the product around it. Compared with an ML engineer, your models produce text, code, or images rather than scores, so correctness is harder to define and evaluation takes more of the week.
A week on a team that owns a reply-drafting model for customer support:
- Monday: the latest fine-tuned version writes worse Tamil replies. A data filter added last sprint had dropped most non-English examples from training.
- Tuesday: you compare the hosted API with a self-hosted open-weight model on the evaluation suite, and on cost per thousand requests at next quarter’s expected traffic.
- Wednesday: the ticket-triage agent loops between two tools. You add a step limit, rewrite the tool descriptions, and add the case to the test suite.
- Thursday: you check the LLM judge against 150 human-labelled examples. It favours longer answers, so you tighten the rubric and measure agreement again.
- Friday: you write up why fine-tuning fixed tone and format but not factual accuracy, so product facts stay in retrieval.
Durable skills versus this quarter’s buzzwords
Libraries here are replaced quickly; the questions underneath them are not:
- How generation works. Tokenisation, context windows, the KV cache, and sampling settings such as temperature and top-p explain cost, latency, and odd behaviour. Many tokenisers, for instance, split text in Indian scripts into more tokens than equivalent English, which raises cost per message.
- Data curation. Fine-tuning results depend more on the quality and coverage of examples than on hyperparameters, so de-duplication, balancing, and a clean held-out set are most of the work.
- Parameter-efficient fine-tuning. LoRA freezes the base model and trains small low-rank matrices added to selected layers; the rank sets the adapter’s capacity. QLoRA does this on a base model quantised to 4 bits, so a fairly large model fits on one GPU. Supervised fine-tuning teaches format and behaviour; preference methods such as DPO learn from pairs of better and worse responses.
- Evaluation methodology. Test sets drawn from real traffic, held out from training, versioned, and large enough that a difference is not noise, plus LLM judges checked against human labels.
- Serving. Why engines such as vLLM reach high throughput (continuous batching and paged KV-cache memory), what quantisation costs in quality, and how to estimate GPU memory from parameter count, precision, and context length.
Signs of hype: resumes and postings built around one orchestration framework, “prompt engineer” as a whole job rather than a skill within one, and multi-agent designs for tasks a single call would handle. The decision at the centre of the job:
| Approach | Use it when | Watch out for |
|---|---|---|
| Better instructions and context | The model can do the task but needs clearer direction or examples | Long prompts are paid for on every call |
| Retrieval | Answers depend on private or frequently changing facts | If retrieval misses the right passage, no model recovers it |
| Fine-tuning, often with LoRA | You need consistent format, tone, or a narrow skill, and you have good examples | A poor way to add facts that change; retraining needed when data shifts |
| Self-hosting open weights | Data must stay in your environment, traffic is high and steady, or you need to modify the model | You now own GPUs, scaling, upgrades, and licence terms |
“Open-weight” is not always “open source”: licences range from permissive ones such as Apache 2.0 to custom terms with conditions on use, so read the licence before shipping.
Seniority, titles, and where model-level work happens in India
0–2 years. Few teams hand model-level decisions to a newcomer, so you are more likely to join as a software or ML engineer building evaluation tooling, data pipelines, or serving infrastructure. A fresher who has fine-tuned a small open model and measured it honestly against a prompting baseline is remembered by panels.
3–5 years. Model-level decisions become yours end to end: whether to fine-tune, which serving stack, which evaluation suite, and how to roll back. Many people at this level arrive from ML engineering or data science.
6+ years. You set model strategy: which work goes to hosted APIs and which to open weights, the GPU budget, evaluation and safety standards, and when to stop a project that is not improving.
Titles include Generative AI Engineer, LLM Engineer, Applied Scientist where research groups exist, and GenAI Architect in services firms. The employer decides how much model-level work you actually do:
- Services firms often run generative AI centres of excellence that build reusable accelerators and client proofs of concept; expect more integration than fine-tuning.
- Product companies and AI startups fine-tune and self-host when volume makes API cost or latency painful.
- GCCs follow the parent company’s rules on which models may be used, where data may live, and how models are risk-reviewed: slower, but a strong education in governance.
- Research-leaning teams may expect publications, open-source contributions, or strong mathematics.
How interviews test model-level judgement
Loops usually combine a Python coding round (sometimes PyTorch), an LLM fundamentals round, a design round on an agent or serving system, a project walkthrough, and a hiring manager conversation. Some teams add an evaluation round in which you design the test strategy for a feature they describe. Questions worth practising:
- Design an agent that helps a finance team match vendor invoices to purchase orders in an ERP system. Which tools does it get, which actions need human approval, how does it stop, and how do you test it?
- Explain LoRA to a product manager, then explain what raising the rank changes.
- Your fine-tuned model beats the old one on your evaluation set, yet user complaints went up. List the explanations, most likely first.
- Your LLM judge disagrees with human reviewers on one example in five. What do you do before trusting any judge score?
- At two million requests a day, when does self-hosting an open model beat a hosted API, and which costs do people forget?
- What happens to GPU memory as context length and batch size grow, and how do serving engines manage it?
- List the failure modes of a tool-using agent and a guardrail for each.
- Plan the evaluation for a support assistant whose users write in Hindi, English, and a mix of both.
- Coding: implement top-p sampling given a vector of logits.
On the agent, strong candidates start by reducing autonomy: read-only tools first, a fixed workflow where the steps are known, step and cost limits, idempotent actions, and human approval before anything posts an accounting entry. On self-hosting, the forgotten costs are idle GPU capacity, on-call, model upgrades, and the engineering time to match an API’s reliability.
Describing generative AI work on a resume without inflating it
Inflated claims are easy to spot in this field, and one exposed claim casts doubt on the rest of the resume. Use precise verbs:
| Inflated | Accurate |
|---|---|
| “Built an LLM” | “Built a support assistant on a hosted LLM API” |
| “Trained a 7B model” | “Fine-tuned a 7B open-weight model with LoRA on 12,000 curated examples” |
| “Developed autonomous agents” | “Built a four-tool workflow with human approval before any payment” |
| “Achieved near-perfect accuracy” | “Raised exact match on a 500-item held-out set from 0.71 to 0.86” |
Three made-up examples of bullets with the right shape:
- Example: “Fine-tuned an 8B open-weight model with QLoRA on 9,000 de-duplicated support replies to enforce reply format; format errors on a 400-item held-out set fell from 1 in 8 to 1 in 60, with no drop on a separate factual-accuracy check.”
- Example: “Moved ticket classification from a hosted API to a quantised open model served with vLLM, roughly halving monthly inference cost at equal quality on the evaluation suite, and documented the traffic level below which the API stays cheaper.”
- Example: “Built an evaluation pipeline of 600 graded examples with an LLM judge checked against two human annotators; its regression reports blocked two prompt changes that would have lowered correct refusals.”
Also avoid listing every model you have called or using “agentic” without naming the tools and steps, and label hackathon demos as demos. Name the tools a posting asks for only if you can defend them; Resume Keyword Matching: What Fit Scores Can and Cannot Tell You explains why.
A realistic path into generative AI engineering
Start from solid Python, ML fundamentals (loss, overfitting, held-out evaluation), and enough PyTorch to read a training loop. Then work in this order:
- Build and evaluate a small feature on an API, the AI engineer’s core loop.
- Implement a tiny transformer and train it on a small corpus, so attention and sampling stop being abstractions.
- Fine-tune a small open model with Hugging Face Transformers, PEFT, and TRL on one GPU, and compare it with a prompting baseline.
- Serve it with vLLM or llama.cpp and measure throughput and latency at different batch sizes.
Projects that show judgement rather than enthusiasm:
- Fine-tune versus prompt, honestly. Take a narrow task such as turning free-text Indian addresses into structured fields, build a 500-example test set, and compare few-shot prompting, retrieved examples, and a LoRA fine-tune on quality, latency, and cost per thousand requests.
- An agent with a failure budget. Build a tool-using agent over a mock returns system with 50 test scenarios, including injection attempts and impossible requests, and publish its success rate with a breakdown of how it fails.
- A judge you can trust. Build an LLM judge for summaries, measure its agreement with your own labels, expose its length and position biases, and show what reduced them.
Neighbouring paths: the AI engineer guide focuses on product integration, the ML engineer guide on predictive models trained and served in production, and the data scientist guide on experimentation, which shares much of the evaluation mindset.
Openings to watch and a plan for this week
Openings by location: Bengaluru, Hyderabad, across India, and remote. Check whether each posting asks for model-level work or is an integration role with a newer title. Both are worth applying to, but lead your resume with different projects for each.
- Rewrite any resume line that says “built”, “trained”, or “developed” for something you integrated.
- Add a held-out test set and a baseline to your strongest project.
- Practise the invoice-matching agent design aloud, ending with how you would test it.
- Explain LoRA, quantisation, and the KV cache to a friend in five minutes each.
- Schedule coding practice with A Four-Week Technical Interview Preparation Plan, and present your projects using What to Put on GitHub and in a Portfolio.