AI Fundamentals: A Practical Guide for Engineers and Interviews
Everything important about modern AI in one calm pass — concepts, vocabulary, and interview-ready intuition without the noise.
This guide is for software engineers who want a solid mental model of artificial intelligence — whether you are refreshing for interviews, joining an AI feature team, or reading production LLM code with less confusion.
Read it top to bottom once (~20–30 minutes). Use the table of contents later as a cheat sheet.
The map in one minute
| Term | Plain meaning |
|---|---|
| AI | Broad umbrella: machines performing tasks that usually need human intelligence |
| Machine learning (ML) | Systems that learn patterns from data instead of only hard-coded rules |
| Deep learning (DL) | ML using multi-layer neural networks |
| Generative AI | Models that produce new content (text, code, images, audio) |
| LLM | Large language model — a generative model specialized for language/tokens |
Interview one-liner: AI ⊃ ML ⊃ DL; modern GenAI/LLMs sit mostly inside deep learning.
What problems does ML solve?
Almost every ML problem is one of these:
| Type | Goal | Examples |
|---|---|---|
| Classification | Predict a label | Spam / not spam, fraud detection |
| Regression | Predict a number | Demand forecast, house price |
| Ranking / recommendation | Order candidates | Feed ranking, “similar items” |
| Clustering | Group without labels | Customer segments |
| Generation | Create content | Chat answers, code completion |
| Anomaly detection | Find rare events | Intrusion, sensor faults |
If you can name the input, output, and success metric, you are already framing the problem correctly.
How learning works (core loop)
- Collect data (features + optional labels)
- Train a model to minimize error on training data
- Validate on held-out data so you catch overfitting
- Ship and monitor live performance
Train / validation / test
- Train: model fits here
- Validation: tune hyperparameters / choose among models
- Test: final honest estimate (touch once)
Overfitting vs underfitting
- Underfit: too simple — poor on train and test
- Overfit: memorizes train — great on train, weak on new data
Fixes for overfitting (say these in interviews): more data, regularization, simpler model, dropout (neural nets), early stopping, better validation.
Supervised, unsupervised, reinforcement
| Paradigm | Signal | Typical use |
|---|---|---|
| Supervised | Labeled examples (x → y) | Most classical industry ML |
| Unsupervised | No labels | Clustering, compression, anomaly cues |
| Self-supervised | Labels invented from data itself | Language masking, next-token prediction |
| Reinforcement | Rewards from actions | Games, some alignment/RLHF flavors |
LLMs are mainly pretrained with self-supervised next-token prediction, then often improved with supervised fine-tuning and preference/reward methods.
Classical ML you should still know
Even in an LLM world, these ideas show up constantly:
- Features: numeric/categorical inputs the model sees
- Baseline: always compare to a simple model or heuristic
- Linear / logistic regression: strong, explainable baselines
- Trees / forests / gradient boosting: tabular data workhorses
- k-NN, SVM: know what they optimize for, not every proof
Metrics cheat sheet
| Problem | Common metrics |
|---|---|
| Binary classification | Precision, recall, F1, ROC-AUC, PR-AUC |
| Multiclass | Macro/micro F1, confusion matrix |
| Regression | MAE, MSE/RMSE, MAPE |
| Ranking | NDCG, MAP, recall@k |
| Generation | Task-specific eval + human prefs (BLEU alone is rarely enough) |
Interview tip: Prefer the metric that matches business cost (false positives vs false negatives), not the one that looks nicest on a slide.
Neural networks (just enough)
A neural net is stacked functions:
input → linear transform → non-linearity → … → output
- Weights are learned by gradient descent (backpropagation)
- Activation functions (ReLU, etc.) make the network non-linear
- Loss measures error (cross-entropy, MSE, …)
- Batching, learning rate, optimizers (SGD, Adam) control training dynamics
Important architectures (names to recognize)
| Architecture | Good at |
|---|---|
| MLP / feed-forward | Tabular-ish learned functions, classifier heads |
| CNN | Grid-like data (images; also some text/history uses) |
| RNN / LSTM | Sequences (mostly overshadowed by transformers for NLP) |
| Transformer | Sequences with attention — foundation of modern LLMs |
| Diffusion / other generative nets | Images and some multimodal generation |
Transformers and LLMs
Tokens
Models do not “read English.” Text is split into tokens (subwords/bytes). Context limits are usually in tokens, not characters.
Attention (intuition)
Attention lets each token look at other tokens and decide what matters for the next prediction. That is why transformers handle long-range dependencies better than older RNNs for many language tasks.
What an LLM is doing
At its core, a typical LLM learns:
Given previous tokens, predict the probability distribution of the next token.
Chat behavior emerges after further training (instruction tuning, preference optimization) and product scaffolding (system prompts, tools, retrieval).
Useful LLM vocabulary
| Term | Meaning |
|---|---|
| Context window | How much prior token context the model can see |
| Temperature | Higher → more random sampling; lower → more deterministic |
| Top-p / top-k | Limits sampling to likely tokens |
| Hallucination | Fluent but false or unsupported output |
| Embeddings | Vector representations of text for search/similarity |
| Tool calling | Model asks your system to run functions (search, SQL, APIs) |
Adapting models: prompt, RAG, fine-tune
Pick the lightest approach that works:
| Approach | When to use | Tradeoff |
|---|---|---|
| Prompting | Fast iteration, general reasoning | Limited private knowledge |
| RAG | Need current/company documents | Retrieval quality becomes the bottleneck |
| Fine-tuning | Stable style/format or specialized behavior | Costly; can go stale; needs good data |
| Agents / tools | Multi-step tasks with external systems | Harder reliability and evaluation |
RAG in one picture
- Chunk and embed your documents
- On each query, retrieve relevant chunks
- Put them into the prompt as evidence
- Generate an answer grounded in that evidence
Interview line: RAG updates knowledge without retraining; fine-tuning changes behavior/style more deeply but is heavier.
Evaluation, safety, and production reality
Shipping AI is more than calling an API.
Evaluate like an engineer
- Offline metrics + curated golden sets
- Human review for nuanced quality
- Online A/B tests / shadow traffic when stakes are high
- Track latency, cost per request, cache hit rate, toxicity/PII leaks
Failure modes worth naming
- Hallucinations and overconfidence
- Prompt injection / untrusted tool output
- Data leakage (training or logs)
- Bias and uneven performance across groups
- Distribution shift (production data drifts from training)
Minimal production checklist
- Clear user job-to-be-done and non-goals
- Grounding strategy (RAG/tools) if facts matter
- Guardrails (input/output filters, authz around tools)
- Observability (prompts, retrieval hits, latency, cost)
- Fallback path when the model is wrong or down
AI system design (interview framing)
When asked to “design an AI feature,” structure the answer:
- Problem & users — what decision/output improves?
- Constraints — latency, cost, privacy, language, offline needs
- Data — sources, labels, refresh cadence, access control
- Model choice — API LLM vs open model vs classical ML
- Architecture — prompt service, retriever, vector DB, cache, tools
- Eval plan — offline + online metrics, red-team cases
- Risks — abuse, compliance, blast radius
That outline alone often scores well even before deep ML theory.
Quick interview Q&A
ML vs deep learning?
DL is ML with deep neural nets; strong for unstructured data (text, images, audio), usually needs more data/compute.
Why not always fine-tune?
Cost, ops complexity, and risk of overfitting to narrow examples. Try prompting/RAG first.
Precision vs recall?
Precision: of predicted positives, how many are right. Recall: of real positives, how many you caught. High-stakes misses → prioritize recall; noisy false alarms → prioritize precision.
What is an embedding?
A vector that places similar meaning nearby in space — used for semantic search, clustering, and RAG retrieval.
Why do LLMs hallucinate?
They optimize for likely next tokens, not guaranteed truth. Without grounding/verification, fluent guessing fills gaps.
Batch vs real-time inference?
Batch for offline scoring/reports; real-time for user-facing apps with latency budgets.
One-page mental checklist
- Define the task type and metric
- Separate data, model, and product scaffolding
- Prefer simple baselines before complex models
- For LLMs: prompt → RAG/tools → fine-tune in that order
- Design for failure: monitoring, evals, guardrails, fallbacks
If you remember only that checklist, you already sound like someone who can ship AI carefully — which is what most technical interviews are probing for.