Week 1

Week 1 — Essential mathematical intuition: learn to read AI formulas

Key questionHow can we translate AI formulas into data, shapes, and code?

Learning objectives

Understand and apply the essential mathematical ideas needed to read AI formulas.

50 min estimated reading time

Week progress: 0 of 26 sections (0%)Course progress: 0 of 321 sections (0%)

This week's problem: how do we turn predicting one number from several inputs into a program we can calculate? You already know functions and lists; no matrix knowledge is required yet. First run a function, then name the numerical structures you have just seen.

python

def predict(x, w, b):
    return sum(value * weight for value, weight in zip(x, w)) + b

x = [1.0, 2.0]
w = [0.5, 0.4]
b = 0.1
print(predict(x, w, b))  # 1.4
print(predict([2.0, 1.0], w, b))  # 1.5

The example above runs independently and assumes x and w have the same length. x is the input for one prediction; w and b are rules we have specified by hand for now. Changing x gives the program a different problem. Changing w or b changes its rule. It can calculate, but it has not yet learned from data.

Scroll horizontally to view all columns.

Course data table
Study session	What you will do
1. Inputs and rules	Read numbers and vectors in their feature order, and explain why exchanging inputs changes the result.
2. Calculate one output	Multiply corresponding values, sum them, and add a bias; translate Σ into a loop.
3. Multiple samples and outputs	Arrange two samples and two outputs in tables; connect the loops to X @ W + b.
4. Work independently	Run course_examples/week01_linear_layer.py, change the input, and predict the result.

By the end of this week, distinguish a scalar, one number; a vector, an ordered set of numbers; and a matrix, numbers arranged in rows and columns. Shape tells you each axis's size, but does not replace an explanation of what the axis means. Split the four study sessions as needed and alternate reading with experiments.

Completion criterion: without the answer, explain how inputs [4,3], weights [3,2], and bias [2] produce outputs [4,2]. Explain why we still need next week's automatic parameter adjustment. For now, locate where exponentials and probabilities will be used; their detailed calculations come in Week 6.

Week 1 learning goals

Scroll horizontally to view all columns.

Course data table
Notation	Read it this way first	Small example
= / ≈	Exactly equal / approximately equal after rounding	1/3 ≈ 0.333; do not treat rounded numbers as exact
x₁、x₂	The first and second entries of a collection; these subscripts are not powers	For x = [3,5], x₂ = 5; Python writes x[1] = 5
x²	x multiplied by itself	(−3)² = 9; Python uses x ** 2
Σ	Add the indicated terms	Σᵢ xᵢ=3+5=8
∈ ℝ	Is a real number; may be fractional or negative	x ∈ ℝ³ means a vector of three real numbers
←	Replace the old value with the result on the right	w ← w−0.1: subtract 0.1 from w

Other symbols appear when needed: η (eta) is a learning step size, θ (theta) collectively denotes parameters, and τ (tau) later denotes generation temperature. You do not need to memorize them now; each is explained when first used.

This course does not ask you to finish an advanced mathematics textbook before starting.

The practical goal is to translate an AI formula into data, code, and tensor shapes.

This week focuses on five things:

What scalars, vectors, and matrices are.
Why shape resembles a type contract in a program.
Why the dot product is a neuron's central computation.
How functions, parameters, and inputs differ.
How to translate a formula into Python step by step.

The main sequence this week is:

Real-world information

↓

Numbers

↓

Vector / Matrix

↓

Weighted calculation

↓

A prediction

Week 2 asks what happens after a prediction is wrong: how can the model automatically adjust its parameters?

How to study each concept

Do not only memorize the names of mathematical objects. Ask four questions:

What data structure is it?
Why does the model need it?
What does it do in the formula?
What are its shape and type in code?

Mathematics here is not a collection of isolated formulas. It is a precise language for describing data flow and relationships between changes in a program.

1. AI starts with numbers

We might read a sentence such as:

This house is close to the city center.

A computer cannot directly matrix-multiply the phrase close to the city center. We first represent information numerically, for example:

Floor area

= 100

Number of bedrooms

= 3

Distance to the city center

= 8

Age of the house

= 12

These numbers are not knowledge by themselves. They are a representation the model can process.

Why represent information? Concepts such as close to the city center, a newer house, or a user's strong preference do not share a numerical interface. We need values that can be compared and combined. The representation determines what information is supplied: a model cannot directly use a distance feature that it was never given, although other correlated features might act as proxies.

The first design question in AI is therefore often not how many layers to use, but:

How can we represent the relevant objects and relationships as numbers without discarding essential information?

Image features such as [0.8,0.6,0.2] are supplied for hand calculation. They assume some preceding processing step; they are not answers attached to a raw image. House-area examples will also use simplified numbers such as 1, 2, 3, and 4. Whenever units or features change, their meanings must be stated too. Do not interpret toy outputs as real property prices.

2. Scalar: one number

A scalar is a single number:

House price = 800000

Temperature = 22.5

Loss = 0.18

In Python:

python

price = 800000
temperature = 22.5
loss = 0.18

Later, a training batch will usually be summarized by one scalar loss. This gives us a specified optimization objective whose derivatives with respect to the parameters we can calculate. Multi-element outputs can also be differentiated, but we must specify how their contributions combine. Week 5 returns to that point; for now we use a mean loss.

A scalar represents one quantity without internal position structure. A model may produce many predictions, but training often combines their errors into a scalar loss, one objective for the update.

This does not erase the examples' contributions. Each example contributes to that scalar objective, and backpropagation calculates how the total loss depends on the parameters.

3. Vector: an ordered collection of numbers

We can put several features of one house into a vector:

x = \begin{bmatrix} 100 \\ 3 \\ 8 \end{bmatrix}

The entries represent:

x₁ = floor area

x₂ = number of bedrooms

x₃ = distance to the city center

In code, start by treating it as a list:

python

x = [100, 3, 8]

Two important properties of this feature vector:

The numbers have a fixed order.
Each position has a defined meaning.

If training uses [area, bedrooms, distance] but prediction receives [distance, area, bedrooms], the program may run without an error even though the meanings are wrong.

Scroll horizontally to view all columns.

Course data table
The same three numbers	Shape	Meaning
[0.8,0.6,0.2]	[3]	A vector stored as a one-axis tensor
[[0.8,0.6,0.2]]	[1,3]	One row and three columns
[[0.8],[0.6],[0.2]]	[3,1]	Three rows and one column

We normally put one sample in each row, giving batch input X with shape [B,D]. A vertically written mathematical vector uses a different display convention. In code, shape [3] is not inherently a row matrix or a column matrix. Write the actual shapes before multiplying.

Knowledge check

Do three features arranged as a row and as a column have the same element count and shape?

Why a vector means more than an ordinary list

A list emphasizes that several values are stored together. A vector also treats those values as one mathematical object that supports operations such as addition, scaling, and dot products.

For example, a customer representation might be:

[purchase frequency, average purchase amount, recent activity]

The combination of the three numbers forms the representation. Vector dimension counts how many coordinates describe the object. More coordinates are not automatically better; what matters is whether the representation can carry useful information for the task and training.

4. Shape: a structural contract for data

The vector above has three entries, so its shape is:

[3]

Put four houses together:

X = \begin{bmatrix} 100 & 3 & 8 \\ 80 & 2 & 15 \\ 140 & 4 & 6 \\ 95 & 3 & 11 \end{bmatrix}

Its shape is:

[4, 3]

This means:

4 rows

= 4 examples

3 columns = 3 features per example

For a software engineer, shape can be treated as part of a data type signature:

Tensor[batch_size, feature_count]

Many AI bugs are shape-contract errors rather than incorrect formulas.

Dimension, axis, and shape

[4,3] is a shape with two axes, sometimes called tensor dimensions. Axis 0 has length 4; axis 1 has length 3. This axis count is distinct from a vector's number of coordinates.

The numbers are not enough; record their meanings too:

X.shape = [4, 3]

axis 0 = examples

axis 1 = features

Two tensors can have the same shape but different meanings. [8,32] could mean eight tokens with 32 features each, or eight independent examples with 32 features each. A framework checks numerical shape compatibility; we must ensure semantic compatibility.

5. Weights: how a model combines different inputs

Suppose the model assigns three weights to three features:

w = \begin{bmatrix} 0.7 \\ 0.5 \\ -0.4 \end{bmatrix}

Intuitively:

Floor area

A positive weight

Number of bedrooms

A positive weight

A negative weight for distance to the city center

Initially, think of a weight as specifying:

The direction and rate of an input's effect on this calculation.

These demonstration weights are supplied by hand; training will later learn weight values from data rather than require a programmer to specify them individually.

A weight's sign tells us whether increasing its input raises or lowers this linear score while other inputs stay fixed. Weight magnitudes are comparable only with appropriate attention to input units and scales. Changing area from square meters to hundreds of square meters requires a corresponding weight change in an equivalent model. A larger weight does not automatically establish a stronger real-world causal effect.

Why introduce weights?

If we merely add all inputs, we impose a fixed equal-coefficient rule. Weights give the model an adjustable coefficient on each input path:

A positive coefficient raises the linear result as that input increases.
A negative coefficient lowers the linear result as that input increases.
A coefficient near zero gives that input little local effect in this calculation.
A larger absolute coefficient gives a larger change in output per unit change in that input.

The interpretation depends on feature scale. Area measured in different units or price measured in dollars versus thousands changes the numerical values. Do not compare two absolute weights without considering their input and output units.

6. Dot product: combine several inputs into one score

The dot product is this week's most important calculation.

It appears repeatedly in neural networks because it combines several input signals and their weights into a single score. Each wᵢxᵢ contributes one term; the sum is the neuron's weighted response to the current input.

Formula:

w \cdot x = w_1x_1 + w_2x_2 + w_3x_3

It performs three straightforward steps:

Multiply each input by its corresponding weight

↓

Add all the products

↓

Obtain one number

For an easy hand calculation, suppose:

x = [0.8, 0.6, 0.2]

w = [0.7, 0.5, -0.4]

Then:

w \cdot x = (0.7)(0.8) + (0.5)(0.6) + (-0.4)(0.2)

= 0.56 + 0.30 - 0.08

= 0.78

Python：

python

x = [0.8, 0.6, 0.2]
w = [0.7, 0.5, -0.4]
score = 0.0

for input_value, weight in zip(x, w):
    score += input_value * weight

print(score)  # 0.78

The dot symbol means multiply corresponding entries and add all the products.

Knowledge check

What is the dot product of inputs [2,3] and weights [0.5,−1]?

Why does a dot product require equal lengths?

Each input needs a corresponding weight:

x₁ ↔w₁

x₂ ↔w₂

x₃ ↔w₃

If x has three features but w has only two weights, the third feature has no specified coefficient. This explains the meaning of the shape mismatch, not merely a matrix-library syntax restriction.

7. Bias: an additional learnable offset

A model often adds a bias:

z = w \cdot x + b

Suppose:

w・x = 0.78

b = 0.10

Then:

z = 0.78 + 0.10 = 0.88

Code:

python

bias = 0.1
z = score + bias

A bias allows a nonzero output when all inputs are zero. Like a weight, it is a parameter the model can learn.

Understand bias through a threshold

Suppose we say a neuron activates when z > 0. Without a bias, the corresponding linear decision boundary passes through the origin. Adding a bias gives:

w \cdot x + b > 0

This lets the model move the threshold. For example, suppose the weighted score is 0.78 but we want activation only above 0.90. A bias of −0.90 implements that threshold. In this linear-boundary interpretation, weights control the boundary's orientation and bias shifts its location; removing either can restrict the available rules.

8. Function: apply a rule to inputs to obtain outputs

A function is a computation with specified inputs and outputs:

y = f(x)

Code:

python

def f(x):
    return 2 * x + 1

For a model:

python

def model(x, w, b):
    return w * x + b

Here:

x is the input.
w and b are parameters.
The function's structure defines the model.
The returned value is the prediction.

These four terms return throughout Week 2.

How functions and models relate

A model's forward computation can be viewed as a function, but an arbitrary function does not necessarily learn. A programmer specifies an ordinary function's rule. In a trainable model, the programmer specifies the structure while training determines some numerical parameters from data.

Fixed code: return w * x + b

Learnable state: w, b

Runtime input: x

Result: prediction

Training changes w and b, rather than rewriting the forward code at every step.

9. Variables, parameters, and hyperparameters

These terms describe different roles and are easy to confuse; they are not mutually exclusive programming data types.

Variable

A value used in a computation, such as an input, prediction, or loss.

Parameter

A model value learned through training, such as a weight or bias.

Hyperparameter

A setting chosen for the model or training procedure, such as learning rate, batch size, or layer count.

Parameter → model learns

Hyperparameter → human chooses

Why are hyperparameters not automatically learned by the same ordinary training step? They often define the learning procedure or architecture: layer count determines the graph's structure, while learning rate controls update size. Outer experiments, search, or specialized algorithms can tune them, but they are not normally among the model parameters updated by the basic forward/backward loop.

10. Matrix: process many vectors together

One predict call currently produces one output. For two output scores from the same inputs, give each output its own weight set. Both samples still share that same parameter set; parameters do not change just because a sample occupies another row.

Scroll horizontally to view all columns.

Course data table
Quantity	Values	Meaning of rows and columns
X	[[1,2],[3,4]]	Two rows represent two samples; each row contains two features in a fixed order.
W	[[0.5,−0.3],[0.4,0.8]]	Each row corresponds to an input feature; each column corresponds to an output.
b	[0.1,−0.2]	One bias per output, shared across all samples.

First calculate output 0 for sample 0: 1×0.5+2×0.4+0.1 = 1.4. Next, output 1 is 1×(−0.3)+2×0.8−0.2 = 1.1. For the second sample, substitute [3,4] without changing parameters, giving [3.2,2.1].

Z=XW+b=\begin{bmatrix}1.4&1.1\\3.2&2.1\end{bmatrix}

python

X = [[1, 2], [3, 4]]
W = [[0.5, -0.3], [0.4, 0.8]]
b = [0.1, -0.2]
Z = [[sum(row[i] * W[i][j] for i in range(2)) + b[j]
      for j in range(2)] for row in X]
print(Z)  # Approximately [[1.4, 1.1], [3.2, 2.1]]

This is independently runnable standard Python. Matrix multiplication expresses these same multiply-and-add calculations together. X's column count must match W's row count so every input feature has its corresponding coefficients. The outer dimensions determine sample count and output count. The next section generalizes this rule to arbitrary shapes.

Knowledge check

EX01: change the first input from [1,2] to [2,1] without changing parameters. What are the two outputs? Then give X, W, b, and Z shapes for four samples, three features, and two outputs.

11. The most important matrix-multiplication shape rule

You do not need to memorize every matrix calculation yet. Start with the shape rule:

[m,n] \times [n,p] \rightarrow [m,p]

The inner dimensions must match.

For example:

[4,3] \times [3,2] \rightarrow [4,2]

This means:

Four examples

Three features per example

Two neurons process each example

Two outputs per example

Connecting the shape rule to its meaning is more useful than memorizing calculation steps without understanding them.

Scroll horizontally to view all columns.

Course data table
The same input x = [0.8,0.6,0.2]	Three products	After adding bias
Output 1; weights [0.7,0.5,−0.4]	0.56+0.30−0.08=0.78	0.78+0.1=0.88
Output 2; weights [0.2,−0.1,0.8]	0.16−0.06+0.16=0.26	0.26−0.2=0.06

x=[0.8,0.6,0.2],\quad W=\begin{bmatrix}0.7&0.2\\0.5&-0.1\\-0.4&0.8\end{bmatrix},\quad b=[0.1,-0.2],\quad xW+b=[0.88,0.06]

Each column of W produces one output, so shape = [input features, output features] = [3,2]. Transposing to Wᵀ exchanges rows and columns, rearranging the same values into [2,3]. It is not inversion and does not change the individual weight values. Week 5's nn.Linear stores each output's weights in a row, W_store = Wᵀ, so its equivalent calculation is X @ W_store.T + b. Both conventions calculate the same dot products.

Knowledge check

For X:[4,3], W:[3,2], and b:[2], what is the output shape?

Why is the output shape [m,p]?

The left matrix supplies m samples and the right matrix supplies p output-weight sets. Every sample takes a dot product with each output's weights, producing:

m samples × p outputs

The shared n dimension indexes features whose products are summed within each dot product. It therefore does not remain as an output axis.

12. The summation symbol Σ

Σ means sum. In the example, i = 1 through 3 selects the first, second, and third terms; it is not asking you to solve for a mysterious variable. Expand x₁w₁+x₂w₂+x₃w₃, then perform three multiplications and two additions. Mathematical examples often index from 1 while Python lists index from 0: mathematical x₁ is x[0] in code.

The following two formulas express the same calculation:

w_1x_1 + w_2x_2 + w_3x_3

\sum_{i=1}^{3} w_i x_i

Σ says: let i take the values 1 through 3 and add the corresponding terms.

The equivalent code is:

python

total = 0
for i in range(3):
    total += w[i] * x[i]

When you see Σ, first translate it into a loop.

Σ writes a repeated pattern compactly; it does not introduce a different arithmetic operation. The lower limit tells you where iteration starts, the upper limit where it ends, and the expression tells you what to add on each iteration.

13. Squares, exponentials, and probabilities: a first intuition

Scroll horizontally to view all columns.

Course data table
Arithmetic tool	Example	Where we use it later
Parentheses and negative signs	(-2)²=4；-2²=-(2²)=-4	Squared error
Fractions and division	1/4=0.25=25%	Means and probabilities
Zeroth powers and negative exponents	e⁰=1；e⁻¹=1/e≈0.368	Softmax
Probability check	0.2+0.3+0.5=1	A complete distribution over mutually exclusive candidates

e ≈ 2.718 is a fixed constant, like π. For now, use a calculator or math.exp to calculate powers of e. The natural logarithm ln asks the reverse question: to what power must e be raised to give this number? Since e⁰ = 1, ln(1) = 0. Week 6 uses −ln(p) to convert probabilities into loss.

These concepts appear later; we do not need their full development this week.

Square

x^2 = x \times x

Week 2 uses squared error to measure prediction error.

Exponential

e^x

The exponential maps any real input to a positive value. Week 6 uses it in Softmax.

Probability

0 \le p \le 1

The closer a model-assigned probability is to 1, the more strongly the model favors that result. A language model produces a whole probability distribution for the next token.

For now, know where these tools will be used; you need not memorize the later formulas.

Why square an error?

Positive and negative errors should not cancel each other when measuring prediction error. Squaring makes both nonnegative and penalizes larger errors more strongly. Week 2 develops this into mean squared error.

Why use exponentials when constructing probabilities?

Softmax converts arbitrary logits into positive weights, then normalizes them to sum to 1. The exponential is positive and preserves the ordering: a higher logit receives a higher weight.

A probability is not an established fact

p = 0.8 is the probability assigned by the model under its current parameters and context. It does not guarantee a calibrated 80% real-world chance or make a statement 80% objectively true. Model probabilities reflect patterns learned from data and can be poorly calibrated or wrong.

14. A fixed order for reading AI formulas

When you encounter a formula such as:

z = xW + b

Read it in this order:

What does each symbol mean?
What is the shape of each object?
In what order are operations performed?
What is the output shape?
Which values are parameters?
Which code line performs the calculation?

For example:

python

# Shape trace
# x: [batch, input_features]
# W: [input_features, neurons]
# b: [neurons]
# z: [batch, neurons]

Then inspect the code:

python

z = x @ W + b

Without checking shapes, it is easy to mistake Wx and xW for the same operation. Without checking meanings, it is easy to mistake a runnable tensor calculation for a correct model. These six questions connect mathematical validity, shape compatibility, and the intended meaning of the data.

15. Week 1 Python experiment

Code preparation: understand these six operations first

Scroll horizontally to view all columns.

Course data table
Python notation	Meaning
x = [1,2]	Store an ordered list
x[0]	Read the first entry, obtaining 1
for value in x:	Process each list item in turn; the indented body belongs to the loop
zip(inputs, weights)	Pair corresponding entries; in this course first ensure the lengths match
def score(...): / return ...	Define a computation / return its result
print(value)	Display the result

Save a complete experiment as a .py file and run python filename.py in a terminal; some systems use python3. If a snippet uses variables from an earlier section, run them in order within the same file. Complete runnable examples are collected in course_examples. In Python, ** means exponentiation, @ means matrix multiplication for supporting array types, and * multiplies ordinary numbers.

python

inputs = [0.8, 0.6, 0.2]
weights = [
    [0.7, 0.5, -0.4],
    [0.2, -0.1, 0.8],
]
biases = [0.1, -0.2]
outputs = []

for neuron_weights, bias in zip(weights, biases):
    z = bias
    for x, w in zip(inputs, neuron_weights):
        z += x * w
    outputs.append(z)

print(outputs)

This code already performs the central linear computation of a neural-network layer:

Inputs

↓

Dot Products

↓

Add Biases

↓

Layer Outputs

We have not added activation functions or training yet. Weeks 2 and 3 develop those next.

16. The six main ideas from Week 1

First represent real-world information numerically.
A feature vector is an ordered set of values describing one example.
A matrix can hold multiple examples or multiple sets of weights.
Shape is a structural data contract.
A dot product multiplies corresponding entries and adds their products.
With row samples, XW+b is the linear computation underlying later neurons, layers, and Transformers. nn.Linear stores weights differently but performs the equivalent XW_storeᵀ+b.

17. Check your understanding

Knowledge check

Given: x = [2, 3] w = [0.5, -0.2] b = 0.1 Calculate:

Review the relevant lesson

z = w \cdot x + b

Answer:

z = (0.5)(2) + (-0.2)(3) + 0.1

= 1 - 0.6 + 0.1 = 0.5

If you can translate this directly into:

python

z = 0.5 * 2 + (-0.2) * 3 + 0.1

you have understood this week's central calculation.

18. Week 1 → Week 2

We can now make predictions using parameters:

\hat{y} = wx + b

But an important question remains:

If the prediction is wrong, how can a computer determine which parameter to change, in which direction, and by how much?

Week 2 answers this through loss, derivatives, gradients, and gradient descent.