AI Fundamentals: A Practical Guide for Engineers and Interviews

Artificial intelligence can feel like a wall of vocabulary: machine learning, neural networks, transformers, embeddings, RAG, agents.

The ideas are easier when learned in the order they depend on one another.

We will begin with a small prediction problem, see how a machine learns from examples, and gradually work toward modern language models and production AI systems.

1. Start with a problem, not a model

Suppose we want to detect spam email.

A rule-based version might say:

if subject contains "WIN MONEY":
    mark as spam

This works until spammers change the wording. We can keep adding rules, but language has too many variations to enumerate manually.

A machine-learning approach is different:

Examples of spam and normal email
                ↓
          Learning algorithm
                ↓
             Model
                ↓
New email → probability that it is spam

Instead of writing every rule, we give the system examples and let it learn patterns that help predict the answer.

That is the core idea of machine learning.

2. AI, machine learning, and deep learning

These terms describe nested ideas:

Artificial Intelligence
└── Machine Learning
    └── Deep Learning
        └── Many modern generative models and LLMs

Artificial intelligence

AI is the broad goal of making machines perform tasks associated with human intelligence: planning, recognizing images, understanding language, or making decisions.

An AI system does not have to learn. A chess engine built entirely from search and hand-written rules is still AI.

Machine learning

ML is one way to build AI. A model learns a relationship from data instead of relying only on rules written by a programmer.

Examples:

• spam detection;
• fraud prediction;
• product recommendations;
• demand forecasting; and
• search ranking.

Deep learning

Deep learning is machine learning built from neural networks with many layers. It is especially strong on unstructured data such as text, images, and audio.

Generative AI and LLMs

Generative models create new content rather than only predict a label or number.

Large language models are deep-learning models trained to process and generate sequences of tokens. Chat assistants, code completion, and text summarization are LLM applications.

The useful relationship is:

AI is the broad field. ML learns from data. Deep learning uses neural networks. LLMs are deep-learning models specialized for language.

3. Common machine-learning tasks

Before selecting a model, identify the output you need.

For any ML problem, write down:

Input
Expected output
Success metric
Cost of being wrong

For spam detection:

Input             Email text and metadata
Output            Spam probability
Success metric    Precision and recall
False positive    Important email hidden as spam
False negative    Spam reaches the inbox

The cost of errors determines which metric matters. Model choice comes later.

4. How a model learns

Training is an optimization loop.

For each example:

Email
  ↓
Model predicts 20% spam
  ↓
True label says spam
  ↓
Loss measures the error
  ↓
Training adjusts model parameters

After seeing many examples, the parameters should produce lower error on similar unseen data.

Four pieces are involved:

1. Data: examples the model learns from.
2. Model: a function with adjustable parameters.
3. Loss: a number describing how wrong a prediction is.
4. Optimizer: the procedure that changes parameters to reduce loss.

Training changes parameters. Inference uses the trained parameters to make a prediction.

Training     many examples → learn parameters
Inference    one new input → produce an output

5. Train, validation, and test data

Evaluating a model on the same examples used for training tells us how well it remembers those examples, not how well it handles new ones.

Split the dataset:

Training set      Learn model parameters
Validation set    Choose settings and compare models
Test set          Final unbiased evaluation

The test set should remain untouched until important choices are complete. Repeatedly tuning against it quietly turns it into another validation set.

Underfitting

The model is too simple or insufficiently trained.

Training performance    poor
Validation performance  poor

Possible fixes: better features, a more capable model, or more training.

Overfitting

The model learns the training examples too specifically and fails to generalize.

Training performance    excellent
Validation performance  poor

Possible fixes include more representative data, regularization, a simpler model, dropout, data augmentation, and early stopping.

The goal is not to memorize training data. It is to learn a pattern that survives new data.

6. Ways a model can learn

Supervised learning

Every example includes the expected answer:

Email A → spam
Email B → not spam

Classification and regression commonly use supervised learning.

Unsupervised learning

The data has no answer labels. The model looks for structure:

Customer behavior
        ↓
Groups of similar customers

Clustering and dimensionality reduction are common examples.

Self-supervised learning

The data creates its own training signal.

For language:

"The server returned a 500 ___"
                           ↓
                     predict "error"

No person needs to label each sentence. The next token already exists in the text. Modern LLM pretraining uses this idea at enormous scale.

Reinforcement learning

An agent takes actions and receives rewards:

State → action → new state → reward

It learns behavior that increases long-term reward. Games, robotics, and parts of model alignment use reinforcement-learning ideas.

7. Start with a baseline

A more complex model is not automatically better.

For spam detection, begin with:

Simple keyword rule
        ↓
Logistic regression
        ↓
Tree-based model
        ↓
Neural network, if evidence justifies it

A baseline tells us whether complexity is buying anything.

Useful classical models include:

• linear regression for numeric prediction;
• logistic regression for classification;
• decision trees and gradient-boosted trees for tabular data;
• k-nearest neighbors for local similarity; and
• support-vector machines for margin-based classification.

For structured business data, gradient-boosted trees may outperform a neural network while being cheaper and easier to explain.

8. Evaluation metrics

Accuracy is not enough for every problem.

Imagine 10,000 transactions where only 10 are fraudulent. A model that always predicts “not fraud” is 99.9% accurate and completely useless.

For binary classification:

Precision = Of items predicted positive, how many were correct?
Recall    = Of all real positives, how many did we find?

For spam:

• high precision means legitimate email rarely enters spam;
• high recall means most spam is caught.

Common metrics:

Choose a metric that reflects the cost of errors in the product.

9. Neural networks

A neural network is a stack of learned transformations.

Input
  ↓
Weighted combination
  ↓
Non-linear activation
  ↓
More layers
  ↓
Output

The adjustable numbers are weights.

During training:

1. The network produces an output.
2. A loss function measures the error.
3. Backpropagation calculates how each weight contributed to the error.
4. An optimizer such as SGD or Adam updates the weights.
5. The process repeats over many batches.

Non-linear activation functions such as ReLU matter because a stack of only linear operations would still behave like one linear operation.

Common architectures:

MLP / feed-forward     General learned functions and classifier heads
CNN                    Images and grid-like data
RNN / LSTM             Sequential data, especially before transformers
Transformer            Language and many multimodal tasks
Diffusion model        Image and other generative tasks

10. From words to tokens

Language models do not directly process words or characters. A tokenizer converts text into tokens.

"unbelievable response"
        ↓ tokenizer
["un", "believ", "able", " response"]
        ↓ ids
[431, 9821, 612, 3371]

The exact split depends on the tokenizer. Common words may be one token; unusual words may use several.

The context window is measured in tokens because tokens are what the model receives.

Token ids are then mapped to learned vectors called embeddings:

token id → vector of numbers

Those vectors let the model represent useful relationships in a continuous space.

11. Why transformers matter

Consider:

“The database rejected the write because it was read-only.”

To interpret “it,” the model needs information from earlier words.

Attention lets each token examine other tokens and assign more weight to the ones relevant to its current representation.

Conceptually:

Current token
     │
     ├── compare with earlier tokens
     ├── assign relevance scores
     └── combine useful information

Transformers apply attention across a sequence and then process the result through feed-forward layers. Multiple attention heads can learn different relationships.

Unlike older recurrent networks, transformers can process many sequence positions in parallel during training. That parallelism, combined with scale, made them the foundation of modern LLMs.

Attention is not a database lookup or human-like focus. It is a learned weighted combination of token representations.

12. What an LLM learns

The core pretraining task is simple:

Given previous tokens, predict the next token.

For:

"The API returned status code"

the model produces probabilities:

200    0.35
500    0.22
404    0.18
...

The chosen token is appended, and the process repeats.

Prompt
  ↓
Predict next-token probabilities
  ↓
Choose a token
  ↓
Append it to context
  ↓
Repeat

Predicting tokens across vast amounts of text teaches patterns of language, code, facts, and reasoning behavior. It does not create a guaranteed truth database. The model is optimized to produce likely continuations, which explains both its fluency and its ability to hallucinate.

13. How a base model becomes a chat assistant

Pretraining alone produces a text-completion model.

A chat product usually adds:

Pretraining
    ↓
Instruction tuning
    ↓
Preference/alignment training
    ↓
System prompt + tools + retrieval + safety controls
    ↓
Chat assistant

Instruction tuning

The model learns from examples of instructions and useful responses.

Preference optimization

Human or model feedback teaches which of several responses is more helpful, safe, or appropriate.

Product scaffolding

The application provides system instructions, conversation history, retrieved documents, tool results, output validation, and authorization.

An LLM application is therefore much more than a model endpoint.

14. Inference settings

The model returns a probability distribution over possible next tokens. Decoding settings control how a token is selected.

Temperature

Lower temperature makes high-probability tokens more dominant. Higher temperature produces more variation.

Low temperature     extraction, classification, deterministic formatting
Higher temperature  brainstorming and creative variation

Temperature does not make a model more knowledgeable. It changes sampling.

Top-p and top-k

These restrict sampling to likely candidates:

• top-k keeps the k most likely tokens;
• top-p keeps the smallest set whose cumulative probability reaches p.

Context window

The context window limits how many input and generated tokens the model can consider in one request. A larger window costs more memory and computation and does not guarantee that the model will use every included detail well.

15. Embeddings

An embedding represents an item as a vector:

"reset my password" → [0.12, -0.43, 0.81, ...]

Texts with similar meaning tend to have nearby vectors.

This enables:

• semantic search;
• recommendation;
• clustering;
• duplicate detection; and
• retrieval for LLM applications.

Embedding search does not ask whether two strings share exact words. It asks whether their vectors are close according to a similarity function such as cosine similarity.

16. RAG: giving an LLM external knowledge

Suppose an employee asks:

“How many parental-leave weeks does our current policy allow?”

The model may not know the company’s policy, and the policy may have changed after training.

Retrieval-augmented generation, or RAG, retrieves relevant documents before asking the model to answer.

Preparing documents

Documents
   ↓
Split into chunks
   ↓
Create embeddings
   ↓
Store chunks + vectors

Answering a question

User question
      ↓
Create query embedding
      ↓
Retrieve similar chunks
      ↓
Add chunks to the prompt
      ↓
LLM answers from the evidence

RAG helps with private, current, and source-backed knowledge without retraining the model.

Its quality depends on retrieval. Bad chunking, missing permissions, weak embeddings, or irrelevant results still produce poor answers.

The retriever must apply document authorization before returning chunks. Otherwise the LLM can expose content the user was never allowed to read.

17. Prompting, RAG, tools, or fine-tuning?

Use the lightest method that solves the problem.

Prompting

Use prompting when the base model already has the necessary capability and needs clearer instructions or examples.

Good for      format, tone, task instructions
Trade-off     does not reliably add private or current knowledge

RAG

Use RAG when the answer depends on external documents that change independently of the model.

Good for      policies, product docs, knowledge bases
Trade-off     retrieval quality and latency become critical

Tool calling

Use tools when the task requires an action or exact live data:

Check order status
Run a database query
Create a support ticket
Calculate a price

The model proposes the call; application code validates authorization and executes it. Never let model output bypass normal access controls.

Fine-tuning

Use fine-tuning for stable behavior learned from many examples: specialized style, domain-specific classifications, or consistent output patterns.

Good for      behavior and specialized patterns
Trade-off     training data, evaluation, deployment, and staleness

Fine-tuning is not the first choice for frequently changing facts. RAG or tools usually fit that need better.

18. A simple production LLM application

Consider an internal support assistant.

                           ┌──────────────┐
                           │    User      │
                           └──────┬───────┘
                                  │
                           ┌──────▼───────┐
                           │ AI Gateway   │
                           │ Auth + limits│
                           └──────┬───────┘
                                  │
                           ┌──────▼───────┐
                           │ Orchestrator │
                           └───┬─────┬────┘
                               │     │
                    ┌──────────┘     └──────────┐
                    ▼                           ▼
             ┌────────────┐              ┌────────────┐
             │ Retriever  │              │   Tools    │
             └─────┬──────┘              └────────────┘
                   │
             Vector index
                    \                         /
                     └──────────┬────────────┘
                                ▼
                           ┌──────────┐
                           │   LLM    │
                           └────┬─────┘
                                ▼
                    Validate, log, return

The flow:

1. Authenticate the user and apply limits.
2. Classify the request and decide whether retrieval or tools are needed.
3. Retrieve only documents the user may access.
4. Build the prompt with instructions, evidence, and conversation context.
5. Call the model with a timeout and token budget.
6. Validate tool calls or structured output.
7. Return the answer with citations where facts matter.

The model is one component. Retrieval, permissions, fallbacks, and evaluation make the feature dependable.

19. Evaluating an AI system

Do not wait for production complaints to discover whether the system works.

Build an evaluation set

Collect representative examples:

Normal questions
Ambiguous questions
Missing-information cases
Adversarial prompts
Permission boundaries
Known failure cases

For each example, define what success means.

Evaluate each stage

For RAG:

Retrieval    Did we find the correct document?
Grounding    Is the answer supported by the document?
Answer       Is it correct, complete, and clear?
Safety       Did it respect policy and access controls?

End-to-end answer quality alone can hide the cause of failure.

Offline and online evaluation

Use:

• deterministic tests for schemas and tool calls;
• model-based grading with calibration;
• human review for nuanced quality;
• shadow traffic before risky launches; and
• A/B tests for actual product impact.

Track latency, token usage, cost, fallback rate, retrieval quality, and user corrections alongside quality scores.

20. Common failure modes

Hallucination

The model produces a fluent unsupported answer. Reduce risk with retrieval, citations, constrained outputs, verification tools, and an explicit “I do not know” path.

Prompt injection

Untrusted text tells the model to ignore instructions or reveal data. Treat retrieved content and tool output as data, not trusted instructions. Enforce permissions in code.

Data leakage

Sensitive prompts may enter logs, traces, training datasets, or third-party APIs. Minimize collection, redact data, control retention, and understand provider policies.

Distribution shift

Production inputs change after evaluation. Monitor real examples and retrain or revise prompts when the input distribution or business rules move.

Bias

Aggregate accuracy can hide poor performance for smaller groups. Evaluate slices relevant to the product and investigate uneven errors.

Model or provider outage

Use timeouts, bounded retries, circuit breakers, and a fallback:

Smaller model
Cached answer
Search results without generation
Human handoff
Clear temporary error

The safest fallback depends on the task’s risk.

21. Cost and latency

LLM latency includes:

Queue time
+ retrieval
+ prompt processing
+ token generation
+ tool calls

Useful optimizations:

• choose the smallest model meeting quality requirements;
• keep prompts and retrieved context focused;
• cache safe repeated results;
• stream generated tokens to improve perceived latency;
• run independent retrievals in parallel;
• cap tool loops and output length; and
• batch offline work.

Do not optimize token cost before measuring answer quality. A cheaper answer that users cannot trust has negative value.

22. How to approach an AI design question

Use this sequence:

1. Define the user task. What decision or output should improve?
2. Define mistakes. What happens when the system is wrong?
3. Choose metrics. How will offline and online success be measured?
4. Understand the data. Sources, labels, freshness, privacy, and bias.
5. Build a baseline. Rules or a simple model first.
6. Choose the model. Classical ML, hosted LLM, or self-hosted model.
7. Add retrieval or tools only when required.
8. Design evaluation, monitoring, and fallback paths.
9. Estimate latency and cost.
10. Launch gradually and learn from production.

This keeps the conversation centered on the product problem rather than starting with a fashionable model.

23. Quick interview questions

ML versus deep learning?

Deep learning is a subset of ML that uses multi-layer neural networks. It is especially effective for text, images, and audio, but often needs more data and compute.

Why do LLMs hallucinate?

They are trained to predict likely token sequences, not to guarantee factual truth. When evidence is missing, a plausible continuation may still receive high probability.

Precision versus recall?

Precision asks how many predicted positives were correct. Recall asks how many real positives were found. Choose based on the cost of false alarms versus missed cases.

What is an embedding?

A learned vector representation where items with related meaning tend to be closer. Embeddings support semantic search, retrieval, clustering, and recommendation.

Why not always fine-tune?

Fine-tuning adds data preparation, training, evaluation, deployment, and maintenance. Prompting, retrieval, or tools often solve the need more cheaply and keep changing knowledge current.

RAG versus fine-tuning?

RAG supplies external knowledge at request time. Fine-tuning changes model behavior and learned patterns. Use RAG for changing facts and fine-tuning for stable specialized behavior.

24. The mental model to remember

Problem
  ↓
Data and success metric
  ↓
Simple baseline
  ↓
Model
  ↓
Evaluation
  ↓
Product system: retrieval, tools, safety, fallback
  ↓
Production monitoring and improvement

For modern language applications:

Tokens
  ↓
Embeddings
  ↓
Transformer attention
  ↓
Next-token prediction
  ↓
Instruction-tuned LLM
  ↓
Prompt + RAG + tools + guardrails
  ↓
Useful application

The model is important, but it is not the whole system.

If you remember one principle, remember:

Start with the user problem and the cost of being wrong. Choose the simplest model and system that meet that requirement, then evaluate the complete product—not only the model.

If you already think in APIs and queues and want a two-week schedule instead of a conceptual tour, use AI for Senior Software Engineers.