LESSON 01 /24Learn & practice

Vectors: coordinates, addition, and scaling

Sunday, October 25, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

Warm-up: calculate $-3 + 5$ and $2 \times (-4)$ .

Show R1 answer

$-3 + 5 = 2$ ; $2 \times (-4) = -8$ .

R2

Warm-up: solve $x + 2 = 5$ .

Show R2 answer

Subtract $2$ from both sides: $x = 3$ .

02

Watch & read

22:40–23:00 · 20 minutes

Watch20 min

3Blue1Brown — Vectors | Chapter 1, Essence of linear algebra

Watch the full video once. Pause to copy one addition and one scaling example. The 20-minute block includes pauses and the mini-lesson below; it is not the video runtime.

LESSON NOTES

A scalar is one number. Here a vector is an ordered list, such as $(2, -1)$ . On a drawing, this means move $2$ units right and $1$ unit down. Use the same scale on both axes. Add or subtract matching coordinates: $(a, b) + (c, d) = (a+c, b+d)$ . Multiply every coordinate when scaling: $3(a, b) = (3a, 3b)$ . An arrow from point P to point Q has coordinates $Q-P$ . We write small vectors horizontally for convenience; later $Ax$ will treat x as a column vector.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L01

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Let $u = (2, -1)$ and $v = (-3, 4)$ . Draw both from the origin and label their endpoints. Then draw $u+v$ using head-to-tail addition.

Show Q1 answer

u ends at $(2, -1)$ ; v ends at $(-3, 4)$ . Move a copy of v so its tail is at u’s endpoint. Its tip is $(2-3, -1+4) = (-1, 3)$ , the endpoint of $u+v$ .

Q2

Calculate $u+v$ , $u-v$ , and $2u + v$ for the same u and v. Show the coordinate arithmetic.

Show Q2 answer

$u+v = (2-3, -1+4) = (-1, 3)$ . $u-v = (2-(-3), -1-4) = (5, -5)$ . $2u+v = (4, -2)+(-3, 4) = (1, 2)$ .

Q3

Calculate $3u$ and $-2v$ . For each, describe the change in length and direction.

Show Q3 answer

$3u = (6, -3)$ , three times the length with the same direction as u. $-2v = (6, -8)$ , twice the length with the opposite direction to v. The minus sign reverses direction; the absolute value scales length.

Q4

An arrow starts at $(1, 2)$ and ends at $(4, 0)$ . Find its vector coordinates and draw an equivalent arrow from the origin.

Show Q4 answer

End minus start gives $(4-1, 0-2) = (3, -2)$ . The equivalent arrow starts at $(0, 0)$ and ends at $(3, -2)$ . Moving an arrow without changing its length or direction preserves the vector.

Q5

A feature vector uses [length, width, height]. Explain whether $[2, 3, 4]$ and $[4, 3, 2]$ describe the same measurements.

Show Q5 answer

They do not. The first has length $2$ and height $4$ ; the second has length $4$ and height $2$ . Positions have assigned meanings, so order matters.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Without notes, let $a = (-1, 2), b = (3, 1)$ . Calculate $2a - b$ and explain the first coordinate.

Show final answer

$2a-b = (-2, 4)-(3, 1) = (-5, 3)$ . The first coordinate is $2\times (-1)-3 = -5$ .

WHEN YOU FINISH

Attempt all five exercises, correct every mismatch, then solve the exit question without copying its answer. You can explain why vector subtraction is coordinatewise.

LESSON 02 /24Learn & practice

Linear combinations, span, and basis

Monday, October 26, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

Calculate $2(1, -2) + (3, 1)$ .

Show R1 answer

$(2, -4)+(3, 1) = (5, -3)$ .

R2

What does multiplying a nonzero vector by $-1$ do?

Show R2 answer

It keeps its length and reverses its direction.

02

Watch & read

22:40–23:00 · 20 minutes

Watch20 min

3Blue1Brown — Linear combinations, span, and basis vectors | Chapter 2

Watch the entire video. Pause once to sketch two parallel vectors and once to sketch two nonparallel vectors. Read the mini-lesson with the remaining time in this 20-minute block.

LESSON NOTES

A linear combination is a sum of scaled vectors, such as $2u - v$ . Their span is every result obtainable by varying the scalars. In the plane, two nonparallel vectors span the plane; parallel nonzero vectors span a line through the origin. A basis spans the space without redundant vectors. To find coefficients, match coordinates: $\alpha (1,1)+\beta (1,-1)=(p,q)$ means $\alpha +\beta =p$ and $\alpha -\beta =q$ . Add these equations to obtain $2\alpha =p+q$ , then use $\beta =p-\alpha$ . Greek letters $\alpha$ and $\beta$ are simply unknown numbers.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L02

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Let $e_1 = (1, 0), e_2 = (0, 1)$ . Write $(3, -2)$ as a linear combination of $e_1$ and $e_2$ . Sketch the result.

Show Q1 answer

$3e_1-2e_2 = (3, 0)+(0, -2) = (3, -2)$ . Draw an arrow from the origin to $(3, -2)$ .

Q2

Let $p = (1, 2), q = (2, 4)$ . Calculate $2p - q$ . Then find two different coefficient pairs that produce $(3, 6)$ .

Show Q2 answer

$2p-q = (2,4)-(2,4) = (0,0)$ , because $q=2p$ . Two representations are $3p+0q$ and $p+q$ : both equal $(3,6)$ . The pairs are $(3,0)$ and $(1,1)$ .

Q3

Can combinations of p and q from Question 2 produce $(1, 0)$ ? Write the general combination to justify your answer.

Show Q3 answer

$\alpha p+\beta q = (\alpha +2\beta , 2\alpha +4\beta ) = (t, 2t)$ , where $t=\alpha +2\beta$ . A first coordinate of $1$ forces the second to be $2$ , so $(1,0)$ is impossible. Their span is the line $y=2x$ .

Q4

Let $r = (1,1), s = (1,-1)$ . Find $\alpha$ and $\beta$ such that $\alpha r+\beta s = (4,2)$ . Check by substitution. Explain whether r and s span the plane.

Show Q4 answer

The coordinate equations are $\alpha +\beta =4$ and $\alpha -\beta =2$ . Adding gives $2\alpha =6$ , so $\alpha =3$ and $\beta =1$ . Check: $3(1,1)+(1,-1)=(4,2)$ . They are nonparallel, so their span is the plane.

Q5

The vector $(4,2)$ has coordinates $(4,2)$ in the standard basis, but coefficients $(3,1)$ in the basis $(r,s)$ from Question 4. Does changing the coefficients here change the geometric vector? Explain.

Show Q5 answer

No. $4e_1+2e_2=(4,2)$ , and $3r+s=(4,2)$ . The coefficients refer to different basis vectors. A coordinate list must be interpreted with its basis.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Let $a=(1,2), b=(1,-2)$ . Find coefficients for $(4,0)$ , and decide whether a,b span a line or the plane.

Show final answer

$\alpha +\beta =4$ and $2\alpha -2\beta =0$ imply $\alpha =\beta =2$ . Thus $2a+2b=(4,0)$ . The vectors are nonparallel and span the plane.

WHEN YOU FINISH

Correct all five exercises and complete the exit question independently, including the explanation of span. Do not count a guessed coefficient pair without a substitution check.

LESSON 03 /24Review & apply

Review: vectors and linear combinations

Wednesday, October 28, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Practice35 min

Break5 min

Correct25 min

Final check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

State how to calculate a vector from a starting point P to an ending point Q.

Show R1 answer

Subtract coordinates: $Q-P$ .

R2

When do two nonzero vectors in the plane span only a line?

Show R2 answer

When one is a scalar multiple of the other, so they point along the same line through the origin.

Review reference notes

LESSON NOTES

No new video or reading. Keep prior lessons and solutions closed during recall and the 35-minute problem block. During the 25-minute review block, compare each answer below, write one sentence explaining each error, then close the key. Use the final 15 minutes for the exit question.

02

Practice · L03

22:40–23:15 · 35 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Let $c=(-1,3), d=(2,1)$ . Calculate $2c-d$ and find x such that $x+c=d$ .

Show Q1 answer

$2c-d=(-2,6)-(2,1)=(-4,5)$ . Rearranging $x+c=d$ gives $x=d-c=(2-(-1),1-3)=(3,-2)$ .

Q2

Draw $c=(-1,3)$ and $-c$ from the origin. Then give the vector from $(2,-1)$ to $(-1,3)$ .

Show Q2 answer

$-c=(1,-3)$ ; the two arrows have equal lengths and opposite directions. The displacement between the stated points is $(-1-2,3-(-1))=(-3,4)$ .

Q3

Do $a=(2,2)$ and $b=(-1,-1)$ span the plane? Can they produce $(6,6)$ ? Can they produce $(0,1)$ ?

Show Q3 answer

$b=-a/2$ , so their combinations have equal coordinates and span only the line $y=x$ . $3a=(6,6)$ . $(0,1)$ is unreachable because its coordinates are unequal.

Q4

Find $\alpha ,\beta$ with $\alpha (1,1)+\beta (1,-1)=(5,1)$ . Show your equations and check.

Show Q4 answer

$\alpha +\beta =5$ ; $\alpha -\beta =1$ . Adding gives $\alpha =3$ , then $\beta =2$ . Check: $(3,3)+(2,-2)=(5,1)$ .

Q5

A classmate says: “Any two different nonzero vectors span the plane.” Give a counterexample and correct the claim.

Show Q5 answer

$(1,2)$ and $(2,4)$ are different and nonzero, but the second is twice the first. They span only a line. Two nonparallel vectors in the plane span the plane.

Q6

A vector is represented by coefficients $(2,-1)$ in the basis $r=(1,1), s=(1,-1)$ . Find its standard coordinates.

Show Q6 answer

Use the basis vectors: $2r-s=(2,2)-(1,-1)=(1,3)$ . The coefficients $(2,-1)$ are not the standard coordinates.

Five-minute break23:15–23:20 · leave the screen

03

Check & correct

23:20–23:45 · 25 minutes

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

04

Final check

23:45–00:00 · 15 minutes

Let $r=(1,0), s=(1,2)$ . Find coefficients for $(3,4)$ and explain why this pair spans the plane.

Show final answer

$\alpha r+\beta s=(\alpha +\beta ,2\beta )=(3,4)$ . Hence $\beta =2$ and $\alpha =1$ , giving $r+2s=(3,4)$ . The vectors are nonparallel, so they span the plane.

WHEN YOU FINISH

Attempt all six problems without notes, correct them, and solve the exit question independently. You must distinguish a line from a plane and interpret coefficients relative to the stated basis.

LESSON 04 /24Learn & practice

Matrices and matrix-vector multiplication

Sunday, November 1, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

Calculate $2(1,3)-(2,1)$ .

Show R1 answer

$(2,6)-(2,1)=(0,5)$ .

R2

Write $(-2,3)$ using $e_1=(1,0), e_2=(0,1)$ .

Show R2 answer

$-2e_1+3e_2$ .

02

Watch & read

22:40–23:00 · 20 minutes

Watch20 min

3Blue1Brown — Linear transformations and matrices | Chapter 3

Watch the entire video and pause to copy one example of a vector expressed as a combination of matrix columns. Read the mini-lesson within the same 20-minute block.

LESSON NOTES

A matrix is a rectangular array. For example,

A=\begin{bmatrix}1&2\\3&4\end{bmatrix}

has rows $(1,2)$ and $(3,4)$ , and columns $(1,3)$ and $(2,4)$ . A matrix with $m$ rows and $n$ columns has shape $m\times n$ . It accepts an $n$ -coordinate column vector and returns $m$ coordinates.

To calculate $Ax$ , weight column $j$ of $A$ by coordinate $j$ of $x$ , then add. Equivalently, each output coordinate is the sum of matching row/input products:

\begin{bmatrix}1&2\\3&4\end{bmatrix}\begin{bmatrix}2\\-1\end{bmatrix}=2\begin{bmatrix}1\\3\end{bmatrix}-\begin{bmatrix}2\\4\end{bmatrix}=\begin{bmatrix}0\\2\end{bmatrix}.

We sometimes write vector coordinates horizontally, such as $x=(2,-1)$ , but $Ax$ still treats $x$ as a column vector. This operation is not multiplying only matching array positions.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L04

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Assigned published exercise8 min within this block

MIT 18.06SC — Geometry of linear equations, Problem 1.2

In the PDF, solve Problem 1.2 only. This replaces worksheet Question 3 below. Compute by row products, then check by combining columns. Use 8 minutes of the existing practice block; do worksheet Questions 1, 2, 4, and 5 as usual. Open the official solution only during feedback.

Show published exercise answer

The product is $(-1,9,11)$ . Row products are $1(3)+2(-2)+0(1)=-1$ , $2(3)+0(-2)+3(1)=9$ , and $4(3)+1(-2)+1(1)=11$ . By columns, $3(1,2,4)-2(2,0,1)+(0,3,1)=(-1,9,11)$ .

Published solutions ↗

Q1

Let $A=\begin{bmatrix}2 & 1\\0 & 3\end{bmatrix}$ . Identify its two columns. Calculate $Ae_1$ and $Ae_2$ , then sketch both results.

Show Q1 answer

The columns are $(2,0)$ and $(1,3)$ . $Ae_1=1(2,0)+0(1,3)=(2,0)$ ; $Ae_2=0(2,0)+1(1,3)=(1,3)$ . Sketch arrows from the origin to those endpoints.

Q2

With A from Question 1 and $x=(2,-1)$ , calculate $Ax$ as a sum of columns. Check with row arithmetic.

Show Q2 answer

$Ax=2(2,0)-(1,3)=(3,-3)$ . Row check: first output $2\times 2+1\times (-1)=3$ ; second output $0\times 2+3\times (-1)=-3$ .

Q3 · Local fallback Only if the published source is unavailable

Let $B=\begin{bmatrix}1 & 0 & 2\\-1 & 3 & 1\end{bmatrix}$ and $x=(2,1,-1)$ . State the shape of B and calculate $Bx$ .

Show Q3 answer

B has shape $2\times 3$ , so the output has two coordinates. $Bx=(1\times 2+0\times 1+2\times (-1), -1\times 2+3\times 1+1\times (-1))=(0,0)$ . The column sum $2(1,-1)+(0,3)-(2,1)$ also gives $(0,0)$ .

Q4

C has shape $3\times 2$ . Which inputs are valid for $Cx$ : $(2,1), (2,1,0)$ , and $(0,0)$ ? For each valid input, how many output coordinates are there?

Show Q4 answer

$(2,1)$ and $(0,0)$ are valid because they each have two coordinates. Each produces three output coordinates. $(2,1,0)$ is invalid: it has three coordinates, but C has two columns.

Q5

A transformation sends $e_1$ to $(1,-1)$ and $e_2$ to $(2,0)$ . Write its matrix and find the image of $(3,2)$ .

Show Q5 answer

Put the images in columns: $M=\begin{bmatrix}1 & 2\\-1 & 0\end{bmatrix}$ . Then $M(3,2)=3(1,-1)+2(2,0)=(7,-3)$ . Putting these images in rows would describe a different transformation.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Let $D=\begin{bmatrix}1 & -1\\2 & 0\\0 & 3\end{bmatrix}$ and $x=(2,1)$ . State the input/output dimensions and calculate $Dx$ .

Show final answer

D is $3\times 2$ : two input coordinates and three output coordinates. $Dx=(2-1,4+0,0+3)=(1,4,3)$ .

WHEN YOU FINISH

Correct all five exercises and solve the exit question without notes. You can identify columns and explain why the number of input coordinates matches the number of columns.

LESSON 05 /24Learn & practice

Matrix multiplication, order, and transpose

Monday, November 2, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

A has shape $2\times 3$ . How many coordinates does $Ax$ require and return?

Show R1 answer

It requires three input coordinates and returns two.

R2

Calculate $\begin{bmatrix}1 & 2\\0 & 1\end{bmatrix}(1,-1)$ .

Show R2 answer

$(1-2,0-1)=(-1,-1)$ .

02

Watch & read

22:40–23:00 · 20 minutes

Watch15 min

3Blue1Brown — Matrix multiplication as composition | Chapter 4

Watch the full video. Use at most 15 minutes of the lesson block, including pauses. Spend the remaining 5 minutes on the mini-lesson below, which supplies rectangular multiplication and transpose.

Included below5 min

Mini-lesson below — rectangular products and transpose

Read the full mini-lesson and copy its transpose example. No extra website or video is assigned.

LESSON NOTES

For $A$ of shape $m\times n$ and $B$ of shape $n\times p$ , the product $AB$ has shape $m\times p$ . Entry $(i,j)$ pairs row $i$ of $A$ with column $j$ of $B$ : multiply matching entries, then add. The shared dimension $n$ is summed over. This also works for rectangular matrices.

For column vectors, $ABx=A(Bx)$ means apply $B$ first, then $A$ .

A transpose swaps rows and columns:

C=\begin{bmatrix}1&2&3\\4&5&6\end{bmatrix},\qquad C^\top=\begin{bmatrix}1&4\\2&5\\3&6\end{bmatrix}.

Their shapes are $2\times3$ and $3\times2$ . Transpose changes positions, not signs: $(C^\top)_{ji}=C_{ij}$ . Transposing twice restores $C$ , because an entry moves from $(i,j)$ to $(j,i)$ and then back to $(i,j)$ .

Five-minute break23:00–23:05 · leave the screen

03

Practice · L05

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Assigned published exercise8 min within this block

Dive into Deep Learning §2.3 — Exercise 1

Exercise 1 replaces Question 5. First transpose its matrix twice, then prove the general claim by tracking entry (i,j). Use 8 practice minutes; complete Questions 1–4.

Show published exercise answer

$A_{ij}$ moves to $(j,i)$ , then back to $(i,j)$ . Every entry returns, proving $(A^\top)^\top=A$ . The numerical check is in the fallback answer below.

Q1

Let $A=\begin{bmatrix}1 & 2\\0 & 1\end{bmatrix}$ and $B=\begin{bmatrix}2 & 0\\1 & 3\end{bmatrix}$ . Compute $AB$ , showing how each entry is formed.

Show Q1 answer

$AB=\begin{bmatrix}1\times 2+2\times 1 & 1\times 0+2\times 3\\0\times 2+1\times 1 & 0\times 0+1\times 3\end{bmatrix}=\begin{bmatrix}4 & 6\\1 & 3\end{bmatrix}$ .

Q2

Compute $BA$ for the same matrices. Are $AB$ and $BA$ equal? For $x=(1,0)$ , compute $ABx$ and $BAx$ to exhibit the difference.

Show Q2 answer

$BA=\begin{bmatrix}2\times 1+0\times 0 & 2\times 2+0\times 1\\1\times 1+3\times 0 & 1\times 2+3\times 1\end{bmatrix}=\begin{bmatrix}2 & 4\\1 & 5\end{bmatrix}$ . They differ. $ABx=(4,1)$ ; $BAx=(2,1)$ . In $ABx$ , $B$ acts first; in $BAx$ , $A$ acts first.

Q3

Let $C=\begin{bmatrix}1 & 0 & 2\\-1 & 3 & 1\end{bmatrix}$ and $D=\begin{bmatrix}1 & 2\\0 & 1\\2 & 0\end{bmatrix}$ . State the product shape, then compute $CD$ .

Show Q3 answer

C is $2\times 3$ and D is $3\times 2$ , so $CD$ is $2\times 2$ . Its entries are $[1\times 1+0\times 0+2\times 2,1\times 2+0\times 1+2\times 0]=[5,2]$ and $[-1\times 1+3\times 0+1\times 2,-1\times 2+3\times 1+1\times 0]=[1,1]$ . Thus $CD=\begin{bmatrix}5 & 2\\1 & 1\end{bmatrix}$ .

Q4

For $A$ of shape $2\times3$ , $B$ of shape $3\times4$ , and $C$ of shape $2\times4$ , classify $AB$ , $BA$ , $AC$ , and $B^\top A^\top$ as valid or invalid. Give each valid output shape.

Show Q4 answer

$AB$ is valid, with shape $2\times4$ . $BA$ is invalid because inner dimensions $4$ and $2$ differ. AC is invalid because $3$ and $2$ differ. $B^\top$ is $4\times 3$ and $A^\top$ is $3\times 2$ , so $B^\top A^\top$ is valid, with shape $4\times2$ .

Q5 · Local fallback Only if the published source is unavailable

For $M=\begin{bmatrix}1 & -2 & 0\\3 & 4 & 5\end{bmatrix}$ , write $M^\top$ and $(M^\top)^\top$ , with their shapes. Explain why multiplying all entries by $-1$ is not transposition.

Show Q5 answer

$M^\top=\begin{bmatrix}1 & 3\\-2 & 4\\0 & 5\end{bmatrix}$ , shape $3\times 2$ . Transposing again returns M, shape $2\times 3$ . A transpose swaps coordinate positions; multiplying by $-1$ changes values and keeps the original shape.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Let $A=\begin{bmatrix}1 & 2\\0 & 1\end{bmatrix}$ and $B=\begin{bmatrix}1\\3\end{bmatrix}$ . Calculate $AB$ and $(AB)^\top$ . State which shape rule makes $AB$ valid.

Show final answer

A is $2\times 2$ and B is $2\times 1$ ; the inner dimensions match. $AB=\begin{bmatrix}1\times 1+2\times 3\\0\times 1+1\times 3\end{bmatrix}=\begin{bmatrix}7\\3\end{bmatrix}$ , shape $2\times 1$ . $(AB)^\top=\begin{bmatrix}7 & 3\end{bmatrix}$ , shape $1\times 2$ .

WHEN YOU FINISH

Correct all five exercises and solve the exit question independently. You can state the shared dimension, distinguish AB from BA, and transpose by moving entries.

LESSON 06 /24Review & apply

Review: vector and matrix calculations

Wednesday, November 4, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Practice35 min

Break5 min

Correct25 min

Final check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

What is the shape of the product of a $3\times 2$ matrix and a $2\times 4$ matrix?

Show R1 answer

$3\times 4$ . The shared inner dimension is $2$ .

R2

In $ABx$ , which matrix acts on x first?

Show R2 answer

B acts first: $ABx=A(Bx)$ .

Review reference notes

LESSON NOTES

No new video. Do recall and the six problems with the solutions closed. In the review block, check the dimensions before checking arithmetic. After correcting the work, close the key and attempt the exit question.

02

Practice · L06

22:40–23:15 · 35 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Assigned published exercise5 min within this block

MIT 18.06SC — Geometry of linear equations, Problem 1.3

Solve Problem 1.3 only and justify your true/false answer with the dimensions being matched. This replaces worksheet Question 5 below. Use 5 minutes of the existing practice block; do Questions 1–4 and 6 as usual. Check the official solution during feedback.

Show published exercise answer

True. The inner dimensions are both 2, and the outer dimensions are 3 and 3. Thus a $3\times2$ matrix multiplied by a $2\times3$ matrix produces a $3\times3$ matrix.

Published solutions ↗

Q1

Let $u=(2,-1), v=(-1,2)$ . Calculate $3u-2v$ and find x such that $x+u=v$ .

Show Q1 answer

$3u-2v=(6,-3)-(-2,4)=(8,-7)$ . $x=v-u=(-1-2,2-(-1))=(-3,3)$ .

Q2

Can $(1,3)$ and $(2,6)$ produce $(-1,-3)$ ? Can they produce $(1,2)$ ? Justify both.

Show Q2 answer

The second vector is twice the first, so all combinations have the form $(t,3t)$ . The first target equals $-1(1,3)$ . The second target violates $y=3x$ , so it is unreachable.

Q3

Let $A=\begin{bmatrix}2 & -1\\1 & 0\end{bmatrix}, x=(3,2)$ . Compute $Ax$ both by rows and as a sum of columns.

Show Q3 answer

Rows: $(2\times 3-1\times 2,1\times 3+0\times 2)=(4,3)$ . Columns: $3(2,1)+2(-1,0)=(6,3)+(-2,0)=(4,3)$ .

Q4

Let $S=\begin{bmatrix}2 & 0\\0 & 3\end{bmatrix}, H=\begin{bmatrix}1 & 2\\0 & 1\end{bmatrix}, x=(1,-1)$ . Compute $Hx$ , $S(Hx)$ , and $SH$ . Which transformation acts first in $SHx$ ?

Show Q4 answer

$Hx=(1-2,-1)=(-1,-1)$ . $S(Hx)=(-2,-3)$ . $SH=\begin{bmatrix}2 & 4\\0 & 3\end{bmatrix}$ , and $SHx=(2-4,-3)=(-2,-3)$ . H acts first; S then scales its output.

Q5 · Local fallback Only if the published source is unavailable

For $P$ of shape $2\times3$ and $Q$ of shape $2\times4$ , is $PQ$ valid? Is $P^\top Q$ valid? Give the valid output shape and the dimension being summed over.

Show Q5 answer

$PQ$ is invalid because $3\ne 2$ . $P^\top Q$ has shapes $3\times 2$ and $2\times 4$ , so it is valid with output $3\times 4$ . The two-entry dimension is summed over.

Q6

Let $M=\begin{bmatrix}0 & 1 & 2\\3 & 4 & 5\end{bmatrix}$ . Write $M^\top$ . A classmate instead writes $\begin{bmatrix}0 & 1\\2 & 3\\4 & 5\end{bmatrix}$ . Explain the error using one entry.

Show Q6 answer

$M^\top=\begin{bmatrix}0 & 3\\1 & 4\\2 & 5\end{bmatrix}$ . The classmate rearranged the sequence into new rows without swapping row/column coordinates. For example, M’s first-row second entry is $1$ , so $M^\top$ ’s second-row first entry must be $1$ , not $2$ .

Five-minute break23:15–23:20 · leave the screen

03

Check & correct

23:20–23:45 · 25 minutes

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

04

Final check

23:45–00:00 · 15 minutes

Let $A=\begin{bmatrix}1 & 0\\2 & 1\end{bmatrix}, B=\begin{bmatrix}0 & 1\\1 & 0\end{bmatrix}, x=(2,3)$ . Compute $Bx$ and $A(Bx)$ , then compute $AB$ to verify.

Show final answer

$Bx=(3,2)$ . $A(Bx)=(3,8)$ . $AB=\begin{bmatrix}0 & 1\\1 & 2\end{bmatrix}$ and $(AB)x=(3,2+6)=(3,8)$ . B acts first.

WHEN YOU FINISH

Correct all six exercises and complete the exit calculation independently. You can explain composition order and identify the invalid product before calculating.

LESSON 07 /24Learn & practice

Dot products, lengths, and summation notation

Sunday, November 8, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

For $A=\begin{bmatrix}1 & 2\\-1 & 0\end{bmatrix}$ , calculate $A(2,1)$ .

Show R1 answer

$(1\times 2+2\times 1,-1\times 2+0\times 1)=(4,-2)$ .

R2

Can $(1,2)$ and $(2,4)$ span the plane?

Show R2 answer

No. The second is twice the first, so their span is one line.

02

Watch & read

22:40–23:00 · 20 minutes

Read10 min

3Blue1Brown — Dot products and duality: selected reading

Read the sections “Numerical Method” and “Geometric Interpretation”, including their small examples. Stop before “Order Doesn't Matter”. Do not watch the full chapter today. Use 10 minutes.

Included below10 min

Mini-lesson below — norm and summation notation

Read the entire mini-lesson. Copy the 3D dot-product example and the norm formula; no code setup is required.

LESSON NOTES

A dot product returns one number. Summation notation abbreviates adding the coordinate products:

u\cdot v=u_1v_1+\cdots+u_nv_n=\sum_{i=1}^{n}u_iv_i.

For $(2,-1,3)$ and $(1,4,0)$ , this is $2\times1+(-1)\times4+3\times0=-2$ . The index $i$ runs through the coordinate positions.

The Euclidean norm is vector length:

\lVert u\rVert=\sqrt{u\cdot u}=\sqrt{u_1^2+\cdots+u_n^2}.

For $(3,4)$ , the length is $\sqrt{9+16}=5$ . A zero dot product means orthogonal. For two nonzero vectors, that means a right angle; the zero vector has no direction, although its dot product with every vector is zero. Positive and negative dot products indicate an angle below or above $90^\circ$ for nonzero vectors.

In $Ax$ , each output coordinate is the dot product of a row of $A$ with $x$ .

Five-minute break23:00–23:05 · leave the screen

03

Practice · L07

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Calculate $(2,-1,3)\cdot (1,4,-2)$ . Expand every term and state whether the result is a scalar or vector.

Show Q1 answer

$2\times 1+(-1)\times 4+3\times (-2)=2-4-6=-8$ . A dot product is a scalar.

Q2

Calculate the norms of $(3,4), (-3,-4)$ , and $(0,0)$ . Explain why the first two agree.

Show Q2 answer

$\sqrt{3^2+4^2}=5$ ; $\sqrt{(-3)^2+(-4)^2}=5$ ; $\sqrt{0}=0$ . Reversing direction preserves length; squaring removes the signs.

Q3

For $u=(1,2)$ , calculate its dot product with $a=(2,-1), b=(2,1)$ , and $c=(-2,-1)$ . Identify the perpendicular pair and the signs of the other two.

Show Q3 answer

$u\cdot a=2-2=0$ , so u and a are perpendicular. $u\cdot b=2+2=4>0$ . $u\cdot c=-2-2=-4<0$ . All four vectors are nonzero, so the geometric angle descriptions apply.

Q4

Let $w=(2,-1,1)$ and $x=(3,2,-2)$ . Expand $\sum_{i=1}^{3}w_ix_i$ . Then calculate $y=wx+5$ when $w$ is a $1\times3$ row matrix and $x$ is a column vector.

Show Q4 answer

The sum is $2\times 3+(-1)\times 2+1\times (-2)=6-2-2=2$ . Hence $wx=2$ and $y=2+5=7$ . The summation notation names the coordinate index being added over.

Q5

A classmate says: “A zero dot product always means two arrows meet at a right angle.” Use the zero vector to refine this statement.

Show Q5 answer

$(0,0)\cdot v=0$ for every v, but the zero vector has no direction, so its angle with v is undefined. A zero dot product implies a right angle when both vectors are nonzero. Algebraically the zero vector is orthogonal to every vector.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

For $u=(2,1), v=(1,-2)$ , calculate $u\cdot v$ and $\lVert u\rVert$ . What can you conclude about their angle?

Show final answer

$u\cdot v=2-2=0$ . $\lVert u\rVert=\sqrt{4+1}=\sqrt{5}$ . Both vectors are nonzero, so their angle is $90^\circ$ .

WHEN YOU FINISH

Correct all five exercises and solve the exit question without notes. You can expand the summation, distinguish a scalar dot product from a vector, and compute a norm.

LESSON 08 /24Learn & practice

Projection onto a line and elementary rank

Monday, November 9, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

Calculate $(1,1)\cdot (1,-1)$ and interpret the result.

Show R1 answer

$1-1=0$ ; the two nonzero vectors are perpendicular.

R2

Find the columns of $\begin{bmatrix}1 & 2\\2 & 4\end{bmatrix}$ . Are they independent?

Show R2 answer

They are $(1,2)$ and $(2,4)$ . The second is twice the first, so they are dependent.

02

Watch & read

22:40–23:00 · 20 minutes

Included below20 min

Mini-lesson below — projection, rank, and lost information

Read the entire mini-lesson twice. On the second pass, reproduce its worked projection without looking at the arithmetic. Use the full 20-minute learning block; no new video is assigned.

LESSON NOTES

To project $x$ onto the line through a nonzero vector $u$ , seek a point $p=cu$ on that line whose residual $r=x-p$ is perpendicular to $u$ . The condition $r\cdot u=0$ gives $(x-cu)\cdot u=0$ , so

c=\frac{x\cdot u}{u\cdot u},\qquad p=\frac{x\cdot u}{u\cdot u}\,u.

The denominator is nonzero because $u\ne0$ . The projection $p$ is a vector; $c$ is a scalar.

Worked example. For $x=(3,1)$ and $u=(1,1)$ , we have $x\cdot u=4$ and $u\cdot u=2$ . Thus $c=2$ , $p=(2,2)$ , and $r=(1,-1)$ . Check: $r\cdot u=1-1=0$ . Draw the line $y=x$ , the point $(3,1)$ , and its perpendicular segment to $(2,2)$ . If $u=(1,0)$ , projection keeps the horizontal coordinate and sets the vertical one to zero.

Rank. Matrix rank is the number of independent directions among the columns, equivalently the dimension of all possible outputs $Ax$ . The zero matrix has rank 0. Nonzero columns all on one line give rank 1. Two nonparallel columns in a two-row matrix give rank 2. Rank cannot exceed either the row count or the column count.

For $A=\begin{bmatrix}1&2\\2&4\end{bmatrix}$ , every output lies on $y=2x$ , so its rank is 1. Distinct inputs can produce the same output: $A(2,0)=A(0,1)=(2,4)$ . The output alone cannot reveal which input was used.

This lesson uses orthogonal projection onto a line. In an ML layer, “projection” can instead mean a general learned linear map.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L08

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Project $x=(3,2)$ onto the line through $u=(1,0)$ . Calculate p and $r=x-p$ , then check $r\cdot u$ .

Show Q1 answer

$c=\frac{x\cdot u}{u\cdot u}=\frac31=3$ . $p=3(1,0)=(3,0)$ . $r=(0,2)$ . $r\cdot u=0$ , confirming perpendicularity.

Q2

Project $x=(4,0)$ onto the line through $u=(1,1)$ . Calculate $c,p,r$ and verify both defining conditions: p is on the line and r is perpendicular to u.

Show Q2 answer

$c=\frac{4\times1+0\times1}{1^2+1^2}=\frac42=2$ . $p=(2,2)$ , which is $2u$ and lies on $y=x$ . $r=(4,0)-(2,2)=(2,-2)$ ; $r\cdot u=2-2=0$ .

Q3

Give the rank of each matrix, using its columns: $A=\begin{bmatrix}1 & 2\\2 & 4\end{bmatrix}, B=\begin{bmatrix}1 & 0\\0 & 1\end{bmatrix}, Z=\begin{bmatrix}0 & 0\\0 & 0\end{bmatrix}$ .

Show Q3 answer

A has nonzero but parallel columns, so rank $1$ . B’s columns are the two nonparallel coordinate directions, so rank $2$ . Z has no nonzero direction among its columns, so rank $0$ .

Q4

For $A=\begin{bmatrix}1 & 2\\2 & 4\end{bmatrix}$ , compare $A(2,0)$ with $A(0,1)$ . Can the output alone distinguish these inputs?

Show Q4 answer

$A(2,0)=2(1,2)=(2,4)$ . $A(0,1)=(2,4)$ too. The inputs differ, so the common output does not tell us which input was used. This demonstrates lost information.

Q5

Let $C=\begin{bmatrix}1 & 0 & 1\\0 & 1 & 1\end{bmatrix}$ . What is its rank? Calculate $C(1,1,0)$ and $C(0,0,1)$ . Does rank $2$ guarantee that a three-coordinate input is recoverable?

Show Q5 answer

The first two columns $(1,0),(0,1)$ are independent, and no third independent direction exists in the two-dimensional output, so rank $2$ . Both stated inputs produce $(1,1)$ . Rank $2$ does not make this three-input map one-to-one; distinct inputs can coincide in its two-coordinate output.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Project $x=(2,4)$ onto the line through $u=(1,1)$ . Check the residual. Then give the rank of $P=\begin{bmatrix}0.5 & 0.5\\0.5 & 0.5\end{bmatrix}$ , which performs this projection.

Show final answer

$c=\frac{2+4}{2}=3$ ; $p=(3,3)$ ; $r=(-1,1)$ ; $r\cdot u=-1+1=0$ . P has identical nonzero columns, so its rank is $1$ .

WHEN YOU FINISH

Correct all five exercises and solve the exit question independently. You can check both projection conditions and explain one pair of distinct inputs with the same output.

LESSON 09 /24Review & apply

Checkpoint before tensors: rows, columns, and geometry

Wednesday, November 11, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Practice35 min

Break5 min

Correct25 min

Final check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

State the formula for projection of x onto the line through nonzero u.

Show R1 answer

$p=\frac{x\cdot u}{u\cdot u}\,u$ . It has the form $cu$ , and $(x-p)\cdot u=0$ .

R2

For $A$ of shape $m\times n$ and x with n coordinates, explain $Ax$ in two ways.

Show R2 answer

It is a weighted sum of A’s columns, with weights from x. Equivalently, each of its m coordinates is the dot product of a row of A with x.

Review reference notes

LESSON NOTES

No new video. Use paper for all calculations. During correction, identify whether each mistake concerns arithmetic, dimensions, column interpretation, or geometry. The final question uses fresh values and must be attempted with the answer key closed.

02

Practice · L09

22:40–23:15 · 35 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Let $a=(1,-1), b=(2,1)$ . Calculate $2a-b$ . Then find $\alpha ,\beta$ with $\alpha a+\beta b=(5,1)$ .

Show Q1 answer

$2a-b=(2,-2)-(2,1)=(0,-3)$ . For the coefficients: $\alpha +2\beta =5$ and $-\alpha +\beta =1$ . Adding gives $3\beta =6$ , so $\beta =2$ and $\alpha =1$ . Check: $a+2b=(1,-1)+(4,2)=(5,1)$ .

Q2

Let $A=\begin{bmatrix}1 & 2 & 0\\-1 & 1 & 3\end{bmatrix}, x=(2,-1,1)$ . Compute $Ax$ using row dot products and a weighted sum of columns.

Show Q2 answer

Rows: $(1\times 2+2\times (-1)+0\times 1,-1\times 2+1\times (-1)+3\times 1)=(0,0)$ . Columns: $2(1,-1)-(2,1)+(0,3)=(2,-2)+(-2,-1)+(0,3)=(0,0)$ .

Q3

Let $B=\begin{bmatrix}1 & 1\\0 & 2\end{bmatrix}, C=\begin{bmatrix}2 & 0\\1 & 1\end{bmatrix}, v=(1,-1)$ . Compute $Cv$ and $B(Cv)$ . Then compute $BC$ to verify the result.

Show Q3 answer

$Cv=(2,0)$ . $B(Cv)=(2,0)$ . $BC=\begin{bmatrix}3 & 1\\2 & 2\end{bmatrix}$ , and $(BC)v=(3-1,2-2)=(2,0)$ . C acts first.

Q4

D has shape $2\times 3$ and E has shape $3\times 4$ . State the shapes of $DE$ and $(DE)^\top$ , and decide whether $D^\top E^\top$ is valid. Write the transpose of $\begin{bmatrix}1 & 0 & -1\\2 & 3 & 4\end{bmatrix}$ .

Show Q4 answer

$DE$ is $2\times 4$ , so $(DE)^\top$ is $4\times 2$ . $D^\top E^\top$ has shapes $3\times 2$ and $4\times 3$ , so it is invalid because $2\ne 4$ . The explicit transpose is $\begin{bmatrix}1 & 2\\0 & 3\\-1 & 4\end{bmatrix}$ .

Q5

For $x=(5,1), u=(1,1)$ , calculate the projection p onto the line through u and the residual r. Calculate $\lVert r\rVert$ and verify orthogonality.

Show Q5 answer

$c=\frac{5+1}{2}=3$ ; $p=(3,3)$ ; $r=(2,-2)$ . $\lVert r\rVert=\sqrt{4+4}=\sqrt{8}=2\sqrt{2}$ . $r\cdot u=2-2=0$ .

Q6

Let $M=\begin{bmatrix}1 & -1\\-2 & 2\end{bmatrix}$ . Determine its rank and find a nonzero input x for which $Mx=(0,0)$ . Explain the resulting ambiguity with the zero input.

Show Q6 answer

The columns $(1,-2)$ and $(-1,2)$ are negatives, so rank $1$ . Choose $x=(1,1)$ : $Mx=(1-1,-2+2)=(0,0)$ . The zero input also gives $(0,0)$ , so the output cannot distinguish these inputs.

Five-minute break23:15–23:20 · leave the screen

03

Check & correct

23:20–23:45 · 25 minutes

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

04

Final check

23:45–00:00 · 15 minutes

Let $A=\begin{bmatrix}2 & 0 & 1\\1 & 1 & 0\end{bmatrix}, x=(1,2,-1)$ . State the dimensions, compute $Ax$ by columns and rows, and decide the rank of A from its columns.

Show final answer

A is $2\times 3$ and x has $3$ coordinates; $Ax$ has $2$ . Columns give $(2,1)+2(0,1)-(1,0)=(1,3)$ . Rows give $(2\times 1+0\times 2+1\times (-1),1\times 1+1\times 2+0\times (-1))=(1,3)$ . Its columns include the nonparallel vectors $(2,1)$ and $(0,1)$ , so its rank is $2$ .

WHEN YOU FINISH

Correct all six problems and complete the exit question without notes. Before proceeding to tensors, explain why both methods for Ax agree and why two independent output directions give rank 2.

LESSON 10 /24Learn & practice

Tensor axes and indexing

Sunday, November 15, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

For A = [[1, 2, 3], [4, 5, 6]], how many rows and columns are there?

Show R1 answer

Two rows and three columns: matrix shape (2, 3).

R2

For $u=(1,2,-1)$ and $v=(3,0,2)$ , calculate the dot product.

Show R2 answer

$u\cdot v=1\times3+2\times0+(-1)\times2=1$ .

02

Watch & read

22:40–23:00 · 20 minutes

Read6 min

D2L §2.1.1 — Getting Started

Select the PyTorch tab. Read from the heading through the x.shape example and its output. Stop before the reshape discussion. Ignore the other framework tabs.

Read6 min

D2L §2.1.2 — Indexing and Slicing

Read the opening paragraph and the X[-1], X[1:3] example. Stop before the paragraph about writing elements. Then read the notation primer below.

Included below8 min

Notation primer in this lesson

Read the entire primer below and trace its small example on paper. No installation or code execution is required.

LESSON NOTES

A tensor is a rectangular array. Its shape lists the length of each axis, in order. For a matrix, axis 0 selects rows and axis 1 selects columns. For T[batch, token, feature], the three axes mean exactly those labels; software does not infer the labels for you.

Python/PyTorch notation used throughout these worksheets:

A = ... gives a value the name A. # ... is a comment.
torch.tensor([[1, 2], [3, 4]]) creates a matrix from a list of rows. torch.tensor(7) is a scalar with shape (); [7] gives a one-element vector with shape (1,).
Indexing starts at 0. A[0, 1] means first row, second column.
: means all positions on that axis. 1:3 includes positions 1 and 2; it excludes position 3. -1 selects the last position.
One integer index removes that axis. A slice keeps that axis, even when its length becomes 1.
A.shape reports the shape; A.numel() counts all entries. print(...) displays a result.

Example: for A = [[1, 2], [3, 4]], A[0] is [1, 2], shape (2,), while A[0:1] is [[1, 2]], shape (1, 2). A[:, 1] is [2, 4], shape (2,).

Treat code in these sheets as precise notation and solve on paper. If PyTorch is already available, use import torch first, create inputs with torch.tensor(...), then print the named expressions after predicting them. Decimal inputs such as 1.0 permit later mean calculations. Do not spend this session installing software.

For the published exercises, len(T) means the length of axis 0, not the total number of entries. A scalar has no axis 0, so len is undefined for it.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L10

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Assigned published exercise6 min within this block

Dive into Deep Learning §2.3 — Exercises 4 and 5

Exercises 4–5 replace Question 5. Work on paper; use this key instead of running code. Allow 6 practice minutes; complete Questions 1–4.

Show published exercise answer

4: len(X) = 2. 5: the first-axis length for every non-scalar tensor. A scalar has no first axis, so len raises an error.

Q1

Write the shape and number of entries of each tensor: (a) torch.tensor(7), (b) torch.tensor([7]), (c) torch.tensor([[7]]), (d) torch.tensor([[1, 2, 3], [4, 5, 6]]). Explain why the first three shapes differ despite each containing one number.

Show Q1 answer

(a) shape (), 1 entry; (b) (1,), 1 entry; (c) (1, 1), 1 entry; (d) (2, 3), 6 entries. A scalar needs no index; the vector needs one index; the matrix needs a row index and a column index.

Q2

Let A = [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]], shape (3, 4), axes [row, column]. Give values and shapes of A[1, 2], A[-1], A[:, 1], and A[1:3, 1:3].

Show Q2 answer

A[1, 2] = 7, shape (). A[-1] = [9, 10, 11, 12], shape (4,). A[:, 1] = [2, 6, 10], shape (3,). A[1:3, 1:3] = [[6, 7], [10, 11]], shape (2, 2): positions 1 and 2 on each axis.

Q3

Let T = [[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]], shape (2, 2, 3), axes [batch, token, feature]. Give values and shapes of T[1, 0, :], T[:, 1, 2], and T[0, :, 1:]. Say in words what T[:, 1, 2] selects.

Show Q3 answer

T[1, 0, :] = [7, 8, 9], shape (3,). T[:, 1, 2] = [6, 12], shape (2,): feature position 2 of token position 1 in each batch. T[0, :, 1:] = [[2, 3], [5, 6]], shape (2, 2).

Q4

Using A from Exercise 2, compare A[0, :] and A[0:1, :]. Give values and shapes. Which preserves the row axis?

Show Q4 answer

A[0, :] = [1, 2, 3, 4], shape (4,). A[0:1, :] = [[1, 2, 3, 4]], shape (1, 4). The slice preserves a row axis of length 1; the integer index removes it.

Q5 · Local fallback Only if the published source is unavailable

Using T from Exercise 3, an instruction says 'select the first token from every batch.' Choose T[:, 0, :] or T[0, :, :]. Write both results and explain why a shape-only check misses the error here.

Show Q5 answer

Correct: T[:, 0, :] = [[1, 2, 3], [7, 8, 9]]. Other expression: T[0, :, :] = [[1, 2, 3], [4, 5, 6]], all tokens from batch 0. Both have shape (2, 3) because batch and token counts happen to be equal; their meanings and values differ.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Without notes, use B = [[2, 4, 6], [8, 10, 12]], shape (2, 3), axes [row, column]. Give the values and shapes of B[:, 1] and B[:, 1:2].

Show final answer

B[:, 1] = [4, 10], shape (2,). B[:, 1:2] = [[4], [10]], shape (2, 1). Integer selection removes the column axis; slicing retains it.

WHEN YOU FINISH

Proceed when you can distinguish an integer index from a slice, and identify which named axis a selection refers to. If the exit problem is wrong, reread the indexing primer and correct Exercises 3–5 before the next new lesson.

LESSON 11 /24Learn & practice

Reshape, transpose, sums, and means

Monday, November 16, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

For A = [[1, 2, 3], [4, 5, 6]], what are A[:, 0] and its shape?

Show R1 answer

[1, 4], shape (2,).

R2

If a matrix has shape (2, 3), what shape does its transpose have, and where does entry (0, 2) move?

Show R2 answer

Shape (3, 2); entry (0, 2) moves to (2, 0).

02

Watch & read

22:40–23:00 · 20 minutes

Included below8 min

Mini-lesson in this worksheet

Read the primer below first; work through its reshape example.

Read8 min

D2L §2.3.6 — Reduction

Select PyTorch. Read the whole subsection, from Reduction through the final A.mean(axis=0) example. Stop at Non-Reduction Sum. The worksheet uses dim for the axis argument.

Read4 min

D2L §2.3.7 — Non-Reduction Sum

Read only the opening paragraph and the sum_A = A.sum(axis=1, keepdims=True) example with its shape. Stop before the division example; broadcasting is next week.

LESSON NOTES

All operations in this lesson use the inputs shown; no preceding exercise changes an input.

A.reshape(r, c) regroups the entries into r rows and c columns. For an ordinary row-list matrix, read the entries row by row, then refill row by row. The number of entries must stay the same. A.reshape(2, -1) asks the library to infer the one missing size.

A.T exchanges rows and columns of a two-dimensional matrix. For A = [[1, 2], [3, 4]], reshaping to (1, 4) gives [[1, 2, 3, 4]], whereas transposing gives [[1, 3], [2, 4]]. With more than two axes, use T.transpose(a, b) to exchange just axes a and b. Do not use .T for higher-order tensors in this plan.

A.sum(dim=0) combines positions along axis 0, leaving one total for each column. A.sum(dim=1) leaves one total for each row. With no dim argument, A.sum() totals everything. mean uses the same axis rule and divides by how many entries were combined. keepdim=True retains each reduced axis with length 1; otherwise that axis disappears. D2L writes axis and keepdims in its examples; our PyTorch notation is dim and keepdim.

For code checks, create inputs with decimal values, such as torch.tensor([[1.0, 2.0], [3.0, 4.0]]), so mean accepts them. Keep full fractions on paper.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L11

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Assigned published exercise6 min within this block

Dive into Deep Learning §2.3 — Exercise 8

Exercise 8 replaces Question 4. Label the surviving axes and use keepdim=False. Allow 6 practice minutes; complete Questions 1–3 and 5.

Show published exercise answer

Axis 0: (3,4). Axis 1: (2,4). Axis 2: (2,3). Each result removes exactly the reduced axis.

Q1

Let A = [[1, 2, 3], [4, 5, 6]], shape (2, 3). Calculate A.reshape(3, 2) and A.T. Both have shape (3, 2); give an entry that distinguishes them. Is A.reshape(4, 2) allowed?

Show Q1 answer

reshape: [[1, 2], [3, 4], [5, 6]]. Transpose: [[1, 4], [2, 5], [3, 6]]. At (0, 1), reshape gives 2 and transpose gives 4. Reshape to (4, 2) is invalid: it would require 8 entries, but A contains 6.

Q2

For the same A, predict values and shapes of A.sum(), A.sum(dim=0), A.sum(dim=1), and A.mean(dim=0). Label the axis that disappears in each axis-specific result.

Show Q2 answer

sum() = 21, shape (). sum(dim=0) = [5, 7, 9], shape (3,), removing the row axis. sum(dim=1) = [6, 15], shape (2,), removing the column axis. mean(dim=0) = [5/2, 7/2, 9/2], shape (3,), removing the row axis and dividing each two-entry column sum by 2.

Q3

For the same A, calculate A.sum(dim=1, keepdim=True) and A.mean(dim=0, keepdim=True), including shapes. Explain what is retained.

Show Q3 answer

Row sums: [[6], [15]], shape (2, 1). Column means: [[5/2, 7/2, 9/2]], shape (1, 3). The reduced axis remains present with length 1; the number of axes stays 2.

Q4 · Local fallback Only if the published source is unavailable

Let T = [[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]], shape (2, 2, 3), axes [batch, token, feature]. Calculate T.sum(dim=0), T.sum(dim=1), and T.sum(dim=2), with shapes and the meaning of each result.

Show Q4 answer

dim=0 gives [[8, 10, 12], [14, 16, 18]], shape (2, 3), totaling batches at each token/feature. dim=1 gives [[5, 7, 9], [17, 19, 21]], shape (2, 3), totaling tokens for each batch/feature. dim=2 gives [[6, 15], [24, 33]], shape (2, 2), totaling features for each batch/token. Equal output shapes for the first two operations do not make them equivalent.

Q5

Let U = [[[1, 2], [3, 4], [5, 6]], [[7, 8], [9, 10], [11, 12]]], shape (2, 3, 2), axes [batch, token, feature]. The desired result has axes [batch, feature, token]. Compare V = U.transpose(1, 2) and R = U.reshape(2, 2, 3). Give their shapes, V[0], R[0], V[1, 1, 0], and R[1, 1, 0]. Which fulfills the request?

Show Q5 answer

Both shapes are (2, 2, 3). V[0] = [[1, 3, 5], [2, 4, 6]]; R[0] = [[1, 2, 3], [4, 5, 6]]. V[1, 1, 0] = 8, but R[1, 1, 0] = 10. Transpose fulfills the request because it maps V[b, f, t] = U[b, t, f]. Reshape only regroups the entries.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Let C = [[2, 0, 4], [1, 3, 5]], shape (2, 3). Calculate C.sum(dim=1, keepdim=True) and C.T. Give shapes and explain why reshape(3, 2) cannot substitute for C.T here.

Show final answer

Row sums are [[6], [9]], shape (2, 1). C.T = [[2, 1], [0, 3], [4, 5]], shape (3, 2). reshape(3, 2) gives [[2, 0], [4, 1], [3, 5]], changing the grouping without swapping the row and column roles.

WHEN YOU FINISH

Proceed when you can predict both values and shapes, explain the reduced axis, and distinguish reshape from transpose on unequal dimensions. Correct any conceptual error in Exercises 1 or 5 before progressing.

LESSON 12 /24Review & apply

Review: tensor axes with matrix foundations

Wednesday, November 18, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Practice35 min

Break5 min

Correct25 min

Final check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

For a tensor shaped (2, 3, 4), what shape results from sum(dim=1, keepdim=True)?

Show R1 answer

(2, 1, 4): axis 1 is retained with length 1.

R2

Does knowing the output shape prove that reshape and transpose performed the same computation?

Show R2 answer

No. They can produce the same shape with different entry arrangements. Check values and the meaning of each axis.

Review reference notes

LESSON NOTES

No new video or reading. Attempt the recall and exercises without notes. During the feedback block, use this lesson's answers; if needed, reread only the relevant primer in Lessons 10–11. Use the final 15 minutes for the exit problem. All arrays are independent and indexed from 0.

02

Practice · L12

22:40–23:15 · 35 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Let $u=(2,-1)$ and $v=(1,2)$ . Calculate $2u-v$ , $u\cdot v$ , and $\lVert u\rVert$ . Explain what the dot product says about their directions.

Show Q1 answer

$2u-v=(4,-2)-(1,2)=(3,-4)$ . Since $u\cdot v=2\times1+(-1)\times2=0$ , the nonzero vectors are perpendicular. The length is $\lVert u\rVert=\sqrt{2^2+(-1)^2}=\sqrt5$ .

Q2

Let $A=\begin{bmatrix}1&2&0\\-1&1&3\end{bmatrix}$ and $x=(1,-1,2)$ . Calculate $Ax$ by row dot products, then as a combination of columns. Give the output shape in tensor notation.

Show Q2 answer

Rows give $1(1)+2(-1)+0(2)=-1$ and $-1(1)+1(-1)+3(2)=4$ . Thus $Ax=(-1,4)$ , with tensor shape (2,). Columns give $1(1,-1)-(2,1)+2(0,3)=(-1,4)$ .

Q3

Let M = [[2, 4], [6, 8], [10, 12]], shape (3, 2). Give values and shapes of M[1:3, 0], M[1:3, 0:1], and M.sum(dim=0).

Show Q3 answer

M[1:3, 0] = [6, 10], shape (2,). M[1:3, 0:1] = [[6], [10]], shape (2, 1). M.sum(dim=0) = [18, 24], shape (2,): add along the row axis.

Q4

Let A = [[1, 4, 7], [2, 5, 8]], shape (2, 3). Someone tries to exchange rows and columns using A.reshape(3, 2). Write this result and the correct result. Identify the first row in each.

Show Q4 answer

reshape gives [[1, 4], [7, 2], [5, 8]], first row [1, 4]. A.T gives [[1, 2], [4, 5], [7, 8]], first row [1, 2]. Only the transpose makes each new row an old column.

Q5

Let T = [[[1, 2], [3, 4]], [[5, 6], [7, 8]]], shape (2, 2, 2), axes [batch, token, feature]. The goal is one feature total per token within each batch. Compare T.sum(dim=2) with T.sum(dim=1): calculate both results, their shapes, and name the correct expression.

Show Q5 answer

T.sum(dim=2) = [[3, 7], [11, 15]], shape (2, 2), axes [batch, token]; this is correct. T.sum(dim=1) = [[4, 6], [12, 14]], shape (2, 2), axes [batch, feature]. The wrong result has the same shape because the token and feature counts are both 2.

Five-minute break23:15–23:20 · leave the screen

03

Check & correct

23:20–23:45 · 25 minutes

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

04

Final check

23:45–00:00 · 15 minutes

Let R = [[[1, 3, 5], [2, 4, 6]]], shape (1, 2, 3), axes [batch, token, feature]. Calculate R.sum(dim=2, keepdim=True) and R.transpose(1, 2), with shapes. State the axis names of the transpose.

Show final answer

Sum = [[[9], [12]]], shape (1, 2, 1). Transpose = [[[1, 2], [3, 4], [5, 6]]], shape (1, 3, 2), axes [batch, feature, token].

WHEN YOU FINISH

Ready for broadcasting when you can complete the exit problem and justify the axis choice in Exercise 5. If not, use the next available slot to redo Exercises 3–5 after checking their explanations; keep the later lessons in order.

LESSON 13 /24Learn & practice

Broadcasting and row versus column operations

Wednesday, November 25, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

For A = [[1, 2, 3], [4, 5, 6]], what are A.sum(dim=1) and A.sum(dim=1, keepdim=True), with shapes?

Show R1 answer

[6, 15], shape (2,), and [[6], [15]], shape (2, 1).

R2

For $u=(2,1)$ and $v=(-1,3)$ , calculate $u+2v$ .

Show R2 answer

$u+2v=(2,1)+(-2,6)=(0,7)$ .

02

Watch & read

22:40–23:00 · 20 minutes

Included below8 min

Broadcasting mini-lesson in this worksheet

Read the primer below and trace the two alignment examples.

Read12 min

PyTorch — Broadcasting semantics

Read General semantics in full, from its heading up to but excluding In-place semantics. Skip the empty-dimension example with shape (0,); every worksheet dimension is positive. For shape rules, focus on the trailing-dimension alignment and resulting-size examples.

LESSON NOTES

For elementwise addition, subtraction, multiplication, and division, align the shapes from the right. Add missing leading axes of length 1. Each aligned pair must have equal lengths or include a 1. A length-1 axis reuses its value along the other operand's axis. We use positive axis sizes only.

Example: shape (2, 3) plus (3,) aligns as (2, 3) plus (1, 3). A three-entry vector therefore adds one value per column, repeated for every row. Shape (2, 3) plus (2, 1) adds one value per row, repeated for every column. Shape (2, 3) plus (2,) fails because the rightmost lengths 3 and 2 disagree.

For A = [[1, 2], [3, 4]], A + [10, 20] gives [[11, 22], [13, 24]], whereas A + [[10], [20]] gives [[11, 12], [23, 24]]. Plain arrays in these expressions stand for tensors.

To divide each row by its own sum, first use A.sum(dim=1, keepdim=True). Its shape is (rows, 1), so each row receives its own denominator. Without keepdim=True, a square matrix can silently use those row totals as column denominators. Write the intended axis meaning before checking whether code runs.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L13

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Assigned published exercise8 min within this block

Dive into Deep Learning §2.3 — Exercise 6

Exercise 6 replaces Question 4. Use A = [[0,1,2],[3,4,5]]. Predict the failure on paper, then repair with keepdim=True. Allow 8 practice minutes; complete Questions 1–3 and 5.

Show published exercise answer

Shapes (2,3) and (2,) conflict. Retaining the axis gives divisor [[3],[12]] and result [[0,1/3,2/3],[1/4,1/3,5/12]]. Both row sums are 1.

Q1

For each elementwise addition, give the output shape or explain why it fails. Inputs contain arbitrary numbers; only their shapes matter: (a) (2, 3) + (3,), (b) (2, 3) + (2,), (c) (2, 3) + (2, 1), (d) (2, 1, 3) + (4, 3).

Show Q1 answer

(a) (2, 3), aligning (3,) as (1, 3). (b) Invalid: the trailing lengths 3 and 2 conflict. (c) (2, 3), reusing each row's single value across columns. (d) (2, 4, 3), aligning the second input as (1, 4, 3).

Q2

Let A = [[1, 2, 3], [4, 5, 6]], shape (2, 3); r = [10, 20, 30], shape (3,); c = [[100], [200]], shape (2, 1). Calculate A + r and A + c. Say which value each original row or column receives.

Show Q2 answer

A + r = [[11, 22, 33], [14, 25, 36]], shape (2, 3); columns 0, 1, 2 receive 10, 20, 30 respectively. A + c = [[101, 102, 103], [204, 205, 206]], shape (2, 3); row 0 receives 100 and row 1 receives 200.

Q3

Let a = [1, 2], shape (2,), and b = [[10], [20]], shape (2, 1). Calculate a + b and its shape. Why is it not a two-entry vector of pairwise sums?

Show Q3 answer

Align a as (1, 2) and b as (2, 1), giving output shape (2, 2): [[11, 12], [21, 22]]. The axes request every row value from b paired with every column value from a; neither input has shape (2,) aligned with another (2,) input.

Q4 · Local fallback Only if the published source is unavailable

Let A = [[1, 1, 2], [2, 1, 3]], shape (2, 3), axes [row, column]. Find S = A.sum(dim=1, keepdim=True) and P = A / S, with shapes. Verify that each row of P sums to 1.

Show Q4 answer

S = [[4], [6]], shape (2, 1). P = [[1/4, 1/4, 1/2], [1/3, 1/6, 1/2]], shape (2, 3). The sums are 1/4+1/4+1/2=1 and 1/3+1/6+1/2=1. Each row is divided by its own total.

Q5

Let A = [[1, 3], [2, 6]], shape (2, 2). The goal is again for each row to sum to 1. The code uses bad = A / A.sum(dim=1), omitting keepdim. Calculate bad, explain why it runs, and write a corrected expression and result.

Show Q5 answer

The denominator is [4, 8], shape (2,), aligned as (1, 2). Thus bad = [[1/4, 3/8], [1/2, 3/4]], shape (2, 2). Its row sums are 5/8 and 5/4. It runs because the two row totals happen to match the number of columns. Correct: A / A.sum(dim=1, keepdim=True) = [[1/4, 3/4], [1/4, 3/4]].

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Let B = [[1, 2], [3, 4], [5, 6]], shape (3, 2), and r = [10, 20, 30], shape (3,). Add 10 to row 0, 20 to row 1, and 30 to row 2. Does B + r work? Write a working expression and all values.

Show final answer

B + r fails because trailing lengths 2 and 3 conflict. B + r.reshape(3, 1) works, giving [[11, 12], [23, 24], [35, 36]], shape (3, 2).

WHEN YOU FINISH

Proceed when you can align shapes from the right and repair Exercise 5 using the intended row meaning. A successful runtime shape check alone is not enough.

LESSON 14 /24Learn & practice

Batched matrix multiplication and XW + b

Sunday, November 29, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

Calculate [[1, 2], [0, 1]] @ [[2], [3]], with shape.

Show R1 answer

[[8], [3]], shape (2, 1).

R2

A result has shape (2, 3, 4). Can a bias of shape (4,) be added to it elementwise? Which axis receives the different bias values?

Show R2 answer

Yes; it aligns as (1, 1, 4). The last axis receives the four bias values, shared across both earlier axes.

02

Watch & read

22:40–23:00 · 20 minutes

Included below12 min

Batched multiplication mini-lesson in this worksheet

Read the entire primer, including the affine-layer example and the index formula.

Read8 min

PyTorch — torch.matmul

Read the opening dimensionality rules, especially the rule for inputs with more than two dimensions and the paragraph distinguishing batch dimensions from matrix dimensions. Then read only the last two Examples, labeled batched matrix x batched matrix and batched matrix x broadcasted matrix. Skip sparse-layout and hardware notes.

LESSON NOTES

A @ B means matrix multiplication; A * B means elementwise multiplication. With two matrices, (m, k) @ (k, n) produces (m, n), using row–column dot products.

For tensors with at least two axes, @ treats the last two axes as matrix axes. Earlier axes select batches. With shapes (b, m, k) @ (b, k, n), pair batch 0 with batch 0, batch 1 with batch 1, and so on; the result is (b, m, n). The batch axis is not summed. With (b, m, k) @ (k, n), the same right-hand matrix is used for every batch. More general batch dimensions follow the broadcasting rules, but the two matrix inner sizes must still match.

For an affine layer, X has axes [batch, token, input_feature], W has axes [input_feature, output_feature], and b has one output-feature axis. Then Y = X @ W + b has axes [batch, token, output_feature]. In indices: Y[bch, t, o] = sum over i of X[bch, t, i] * W[i, o] + b[o]. The same W and b apply to every token in every batch. The bias is added after the multiplication, once per output entry.

Example: one input row [2, 3], W = [[1, 0], [2, 1]], and b = [1, -1] produce [2×1+3×2+1, 2×0+3×1-1] = [9, 2]. The term affine means a linear transformation plus an offset.

Negative axis numbers count backward from the end: dim=-1 means the last axis, dim=-2 the second-to-last. For a shape [batch, token, feature], dim=-1 means the feature axis, regardless of the lengths of the other axes.

Mathematically, each output entry is

Y_{btj}=\sum_{i=0}^{I-1}X_{bti}W_{ij}+b_j.

Here $I$ is the number of input features. The summation combines only input features $i$ . Batch $b$ and token $t$ stay fixed while one output feature $j$ is calculated.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L14

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Give the output shape or identify the incompatibility for each @ operation. Inputs contain arbitrary numbers: (a) (2, 3, 4) @ (4, 2); (b) (2, 3, 4) @ (2, 4, 1); (c) (2, 3, 4) @ (5, 4, 1); (d) (2, 3, 4) @ (4, 3). Label the batch axis when present.

Show Q1 answer

(a) (2, 3, 2), batch length 2, shared right matrix. (b) (2, 3, 1), batch length 2 with paired right matrices. (c) Invalid: matrix inner sizes match, but batch sizes 2 and 5 cannot broadcast. (d) (2, 3, 3), batch length 2; its last two axes represent 3 input rows and 3 output columns, which are distinct roles.

Q2

Let A = [[[1, 0], [0, 1]], [[1, 2], [0, 1]]], shape (2, 2, 2), and B = [[[3], [4]], [[5], [6]]], shape (2, 2, 1). Calculate A @ B. Show the dot product for result entry [1, 0, 0].

Show Q2 answer

Batch 0 gives [[3], [4]]. Batch 1 gives [[17], [6]], so A @ B = [[[3], [4]], [[17], [6]]], shape (2, 2, 1). Entry [1, 0, 0] is 1×5 + 2×6 = 17; it uses only batch 1.

Q3

Let X = [[[1, 0], [0, 1]], [[1, 1], [2, -1]]], shape (2, 2, 2), axes [batch, token, input_feature]; W = [[1, 2, 0], [0, -1, 3]], shape (2, 3); b = [1, 0, -1], shape (3,). Calculate Y = X @ W + b, its shape, and the arithmetic for Y[1, 1, 2].

Show Q3 answer

Y = [[[2, 2, -1], [1, -1, 2]], [[2, 1, 2], [3, 5, -4]]], shape (2, 2, 3), axes [batch, token, output_feature]. For example, Y[1, 1, 2] = 2×0 + (-1)×3 + (-1) = -4. The same W and b are used for all four input rows.

Q4

Let P = [[1, 2], [3, 4]] and Q = [[2, 0], [1, 2]], both shape (2, 2). Calculate P @ Q and P * Q. A test checks only output shape (2, 2). Does it distinguish them? Name one exact expected entry that would.

Show Q4 answer

P @ Q = [[4, 4], [10, 8]]; P * Q = [[2, 0], [3, 8]]. Both are shape (2, 2), so the shape check misses the error. For matrix multiplication, entry [0, 1] must be 1×0+2×2 = 4; elementwise multiplication gives 0.

Q5

Reuse X, W, and b from Exercise 3. A faulty implementation uses bad = X.transpose(1, 2) @ W + b. Explain why it runs. Calculate bad[1, 0, :] and compare it with Y[1, 0, :]. Which roles were exchanged?

Show Q5 answer

Token count and input-feature count are both 2, so the swapped input still has shape (2, 2, 2) and multiplication runs. Its batch-1 first row becomes [1, 2], producing bad[1, 0, :] = [2, 0, 5]. The correct value is [2, 1, 2]. The faulty operation exchanges token and input-feature roles. Unequal token/feature dimensions would make this particular error fail at multiplication.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Let X = [[[2, 1], [0, -1]]], shape (1, 2, 2), axes [batch, token, input_feature]; W = [[1, 0], [2, 1]], shape (2, 2); b = [-1, 2], shape (2,). Calculate X @ W + b, state its shape, and say whether either token uses a different W.

Show final answer

Result = [[[3, 3], [-3, 1]]], shape (1, 2, 2). First row: [2×1+1×2-1, 2×0+1×1+2] = [3, 3]. Second: [0×1-1×2-1, 0×0-1×1+2] = [-3, 1]. Both tokens use the same W.

WHEN YOU FINISH

Proceed when you can calculate the exit result, explain which axis is summed, and identify both shape-valid bugs in Exercises 4–5. The batch and token indices must remain distinct.

LESSON 15 /24Review & apply

Review: shapes, broadcasting, and affine calculations

Monday, November 30, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Practice35 min

Break5 min

Correct25 min

Final check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

For X shaped (2, 3, 4), W shaped (4, 5), and b shaped (5,), what is the shape of X @ W + b? Which X axis is summed?

Show R1 answer

(2, 3, 5). Input-feature axis 2, length 4, is summed against W's axis 0. Batch and token axes remain.

R2

For a matrix shaped (3, 2), what shape should row-specific offsets have to be added to all entries of their own rows?

Show R2 answer

(3, 1). A plain length-3 vector aligns with the trailing axis and cannot broadcast to (3, 2).

Review reference notes

LESSON NOTES

No new material. Use the review-session timetable. For each tensor operation, write the axis labels, predicted shape, and at least one exact value before reading the answer or running code. All questions are self-contained; reset inputs between exercises.

02

Practice · L15

22:40–23:15 · 35 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Let $u=(1,-2)$ , $v=(3,1)$ , and $A=\begin{bmatrix}2&0\\1&-1\end{bmatrix}$ . Calculate $u+2v$ and $A(u+2v)$ . Check the second answer by calculating $Au+2Av$ .

Show Q1 answer

$u+2v=(7,0)$ and $A(7,0)=(14,7)$ . Also $Au=(2,3)$ and $Av=(6,2)$ , so $Au+2Av=(2,3)+(12,4)=(14,7)$ , matching linearity.

Q2

Let T = [[[1, 2], [3, 4], [5, 6]], [[7, 8], [9, 10], [11, 12]]], shape (2, 3, 2), axes [batch, token, feature]. Give values and shapes of T[:, 1, :] and T.sum(dim=1, keepdim=True). Explain what is totaled.

Show Q2 answer

T[:, 1, :] = [[3, 4], [9, 10]], shape (2, 2). The sum is [[[9, 12]], [[27, 30]]], shape (2, 1, 2): for each batch and feature, add across the three tokens.

Q3

Let B = [[2, 4], [3, 9]], shape (2, 2), axes [row, feature]. The goal is to subtract each row's own mean from every entry in that row. Calculate bad = B - B.mean(dim=1), and the correct result using keepdim=True. Which simple expected property detects the bug?

Show Q3 answer

Row means are [3, 6], shape (2,). bad = [[-1, -2], [0, 3]], because broadcasting subtracts 3 from column 0 and 6 from column 1. Correct: B - B.mean(dim=1, keepdim=True) = [[-1, 1], [-3, 3]]. Correct centered rows have sum 0; the bad row sums are -3 and 3.

Q4

Let X = [[[1, 0, 2], [-1, 1, 0]]], shape (1, 2, 3), axes [batch, token, input_feature]; W = [[1, 2], [0, 1], [-1, 0]], shape (3, 2); b = [2, -1], shape (2,). Calculate X @ W + b, with shape. Is X * W a valid substitute?

Show Q4 answer

First row times W is [-1, 2]; adding b gives [1, 1]. Second row times W is [-1, -1]; adding b gives [1, -2]. Result = [[[1, 1], [1, -2]]], shape (1, 2, 2). X * W is invalid here because aligned trailing dimensions 3 and 2 conflict; even when dimensions happen to permit it, elementwise multiplication is a different operation.

Q5

Let X = [[[1, 0], [2, 0]], [[0, 1], [0, 3]]], shape (2, 2, 2), axes [batch, token, feature], and W = [[1], [10]], shape (2, 1). Compare correct = X @ W and bad = X.transpose(0, 1) @ W. Calculate both arrays. Write one exact check that fails for bad, even though both outputs have shape (2, 2, 1).

Show Q5 answer

correct = [[[1], [2]], [[10], [30]]]. bad = [[[1], [10]], [[2], [30]]]. Swapping the equal-length batch and token axes changes which request/token each result belongs to. One check is correct[0, 1, 0] = 2; bad gives 10.

Five-minute break23:15–23:20 · leave the screen

03

Check & correct

23:20–23:45 · 25 minutes

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

04

Final check

23:45–00:00 · 15 minutes

Let H = [[2, 6], [3, 1]], shape (2, 2), axes [row, feature], and W = [[10], [20]], shape (2, 1). First calculate P = H / H.sum(dim=1, keepdim=True), then P @ W. Give both shapes and verify the row sums of P.

Show final answer

The row totals are [[8], [4]], shape (2, 1). P = [[1/4, 3/4], [3/4, 1/4]], shape (2, 2), each row summing to 1. P @ W = [[35/2], [25/2]] = [[17.5], [12.5]], shape (2, 1).

WHEN YOU FINISH

Ready for probability and softmax when you can calculate the exit problem and detect the axis errors in Exercises 3 and 5. If a conceptual error remains, correct that exercise and redo the exit problem at the beginning of the next slot before introducing new material.

LESSON 16 /24Learn & practice

Probabilities and weighted averages

Wednesday, December 2, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

Calculate 0.25 × $\left[4, 0\right]$ + 0.75 × $\left[0, 4\right]$ .

Show R1 answer

$\left[1, 3\right]$ : multiply each coordinate, then add.

R2

A matrix has shape $\left[2, 3\right]$ . What is the shape of its row sums when the summed dimension is retained?

Show R2 answer

$\left[2, 1\right]$ : one sum for each of the two rows.

02

Watch & read

22:40–23:00 · 20 minutes

Read7 min

D2L §2.6.1: A Simple Example: Tossing Coins

Read from this heading to immediately before the first code block, which starts num_tosses = 100. Do not run the chapter's simulation.

Read5 min

D2L §2.6.6: Expectations

Read from the heading through equation (2.6.12). Stop before the sentence beginning 'Likewise, for densities'.

Included below8 min

Primer below

Read the entire primer and work through its two small calculations on paper.

LESSON NOTES

Use a calculator from this lesson onward. A probability vector lists the probabilities of mutually exclusive outcomes covering all possibilities. Each entry must be between 0 and 1, and all entries must sum to 1. For outcomes A, B, C, the vector $\left[0.2, 0.5, 0.3\right]$ is valid.

A sample is one outcome drawn using these probabilities. The most probable outcome is B, but drawing A or C is possible. Over 10 draws, the expected number of B outcomes is $10 \times 0.5 = 5$ ; it is not a promise of exactly five.

A weighted average multiplies each value by its weight and adds the products. With values $\left[2, 8\right]$ and probabilities $\left[0.75, 0.25\right]$ , the expectation is $0.75 \times 2 + 0.25 \times 8$ = 3.5. The result need not be a possible individual outcome. For vector values $\left[2, 0\right]$ and $\left[0, 8\right]$ , apply the same calculation to each coordinate: 0.75 $\left[2, 0\right]$ + 0.25 $\left[0, 8\right]$ = $\left[1.5, 2\right]$ .

Read $P(\text{next token}=\text{cat}\mid\text{context})$ as 'the probability that the next token is cat, given this context'. The vertical bar means 'given'. Different contexts can have different probability vectors. This lesson does not require Bayes' theorem.

In p @ V, a one-dimensional p of shape $\left[k\right]$ supplies one weight per row of a $\left[k, d\right]$ matrix V. The result has shape $\left[d\right]$ : output coordinate j is $\sum_{i=1}^{k}p_i V_{ij}$ . For this operation, think of p as a row of weights; the one-dimensional result omits the extra leading size-1 dimension that a $\left[1, k\right]$ row matrix would retain.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L16

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

For three exhaustive outcomes, decide which vectors are valid probabilities: a = $\left[0.2, 0.5, 0.3\right]$ , b = $\left[0.6, 0.6, -0.2\right]$ , c = $\left[0.1, 0.2, 0.3\right]$ , d = $\left[0, 1, 0\right]$ . State the failed rule for each invalid vector.

Show Q1 answer

a is valid: all entries are nonnegative and sum to 1. b sums to 1 but contains a negative probability. c contains permitted individual values but sums to 0.6. d is valid and assigns certainty to the middle outcome.

Q2

A random value takes 2, 6, or 10 with probabilities 0.25, 0.5, and 0.25. Calculate its expectation. Must a single sample equal the expectation?

Show Q2 answer

$0.25 \times 2 + 0.5 \times 6 + 0.25 \times 10 = 0.5 + 3 + 2.5$ = 6. A sample may be 2, 6, or 10; it is not required to equal 6.

Q3

The rows of V are $\left[2, 0\right]$ , $\left[0, 4\right]$ , and $\left[4, 2\right]$ . Calculate their weighted average using p = $\left[0.5, 0.25, 0.25\right]$ . Give the shape of p @ V when p has shape $\left[3\right]$ and V has shape $\left[3, 2\right]$ .

Show Q3 answer

$0.5[2,0]+0.25[0,4]+0.25[4,2]=[2,1.5]$ . The contributions are $[1,0]$ , $[0,1]$ , and $[1,0.5]$ . The shape is [2].

Q4

A sampler returns red with probability 0.7 and blue with probability 0.3 on each independent draw. What is the expected number of red outcomes in 10 draws? Could a draw be blue? Would observing six reds prove the sampler wrong?

Show Q4 answer

The expected count is $10 \times 0.7$ = 7. A blue draw is possible because its probability is 0.3. Six reds does not prove an error: a random count need not equal its expectation.

Q5

A toy vocabulary contains only cat, dog, and END. Given context A, their probabilities are $\left[0.6, 0.3, 0.1\right]$ ; given context B, they are $\left[0.1, 0.2, 0.7\right]$ . Find $P(\text{cat}\mid A)$ , $P(\text{END}\mid B)$ , and $P(\text{cat or dog}\mid A)$ . Does the larger $P(\text{END}\mid B)$ make END certain?

Show Q5 answer

$P(\text{cat}\mid A)$ = 0.6; $P(\text{END}\mid B)$ = 0.7; $P(\text{cat or dog}\mid A)$ = $0.6 + 0.3 = 0.9$ because those outcomes are mutually exclusive. END is not certain: the other two outcomes have combined probability 0.3.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Without the key, use probabilities $\left[0.2, 0.8\right]$ and values $\left[5, 10\right]$ to calculate the expectation and explain whether that number must be sampled.

Show final answer

$0.2 \times 5 + 0.8 \times 10$ = 9. A sample is 5 or 10, never 9; expectation describes the weighted average.

WHEN YOU FINISH

Attempt all five exercises, check each against the key, and answer the exit question without looking at its answer. Record an incorrect exit for review; do not replace it with extra unassigned homework.

LESSON 17 /24Learn & practice

Exponentials, logarithms, and softmax

Sunday, December 6, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

Why is $\left[0.5, 0.7, -0.2\right]$ not a probability vector even though its entries sum to 1?

Show R1 answer

A probability cannot be negative.

R2

Calculate the weighted average of values $\left[2, 10\right]$ with probabilities $\left[0.25, 0.75\right]$ .

Show R2 answer

$0.25 \times 2 + 0.75 \times 10$ = 8.

02

Watch & read

22:40–23:00 · 20 minutes

Included below12 min

Primer below

Read the entire primer. Enter exp(0), exp(1), and ln(2) on your calculator; keep at least four decimal places.

Read8 min

D2L §4.1.1.2: The Softmax

Read from this heading through equation (4.1.4). Stop before the historical paragraph about Gibbs. Skip references to training losses; this assignment concerns only the conversion from scores to probabilities.

LESSON NOTES

An exponent is a power: $2^3=8,\quad 2^0=1,\quad 2^{-1}=\frac12$ . The number e is approximately 2.71828. $\exp(x)=e^x$ . It is positive for every finite real x; $\exp\left(0\right)$ = 1, $\exp\left(1\right)$ ≈ 2.7183, and $\exp\left(-1\right)$ ≈ 0.3679. Larger x produces larger $\exp\left(x\right)$ .

The natural logarithm ln reverses exp: $\exp(\ln a)=a\quad(a>0)$ , and $\ln(\exp x)=x$ . For example, $\ln\left(1\right)$ = 0, $\ln\left(e\right)$ = 1, and $\ln\left(2\right)$ ≈ 0.6931. The calculator button must be ln, not base-10 log. There is no finite real $\ln\left(0\right)$ , and ln of a negative number is not a real number.

Softmax takes scores, often called logits, and produces probabilities. For each score $z_i$ , calculate $e_i=\exp(z_i)$ , add $S=\sum_{j=1}^{n}e_j$ , and set $p_i=\frac{e_i}{S}$ . Example: scores $\left[0, \ln\left(3\right)\right]$ give exponentials $\left[1, 3\right]$ , total 4, and probabilities $\left[\frac{1}{4}, \frac{3}{4}\right]$ . The largest score gives the largest probability. Softmax does not select a sample.

Two useful identities are $\exp(a+b)=\exp(a)\exp(b)$ and $\ln(ab)=\ln(a)+\ln(b)$ for positive a and b. Adding one constant c to every score multiplies every exponential by $\exp\left(c\right)$ ; this factor cancels when dividing by the sum. Therefore $\operatorname{softmax}(z+c\mathbf{1})=\operatorname{softmax}(z)$ , where $\mathbf{1}$ is a vector of ones.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L17

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Assigned published exercise8 min within this block

D2L §4.1.5 — Exercise 4, part 1

During the worksheet block, do exercise 4, part 1 only: establish why the ordering of two scores is preserved by softmax. Spend at most 8 minutes and then check the answer below. This replaces local question 5; use that question only if the source is unavailable. Skip part 2 and the other chapter exercises.

Show published exercise answer

For scores $a>b$ , the exponential function is increasing, so $e^a>e^b$ . Both probabilities divide by the same positive total $S=e^a+e^b$ , hence $\frac{e^a}{S}>\frac{e^b}{S}$ . Raising one score while holding the other fixed also raises its probability: $p_a=\frac{1}{1+e^{b-a}}$ , whose denominator decreases as $a$ increases. Thus the higher-scored choice is more probable. This is an original worked solution to the linked exercise.

Q1

Calculate $\exp\left(0\right)$ , $\exp\left(\ln\left(5\right)\right)$ , $\ln\left(1\right)$ , and $\ln\left(\exp\left(-2\right)\right)$ . Is $\ln\left(-1\right)$ a real number?

Show Q1 answer

The answers are 1, 5, 0, and -2. $\ln\left(-1\right)$ is not real. The inverse-function calculations use exp and ln undoing each other.

Q2

Compute $\operatorname{softmax}\left(\left[0, \ln\left(2\right), \ln\left(3\right)\right]\right)$ , showing exponentials, their sum, and probabilities. Check the sum and ranking.

Show Q2 answer

Exponentials are $\left[1, 2, 3\right]$ , total 6, and probabilities $\left[\frac{1}{6}, \frac{1}{3}, \frac{1}{2}\right]$ ≈ $\left[0.1667, 0.3333, 0.5000\right]$ . They sum to 1 and retain the ordering of the scores.

Q3

Compute $\operatorname{softmax}\left(\left[-1, 0, 1\right]\right)$ with a calculator, keeping four decimal places. Also compute $\operatorname{softmax}\left(\left[4, 4, 4\right]\right)$ . Explain why the latter needs no calculator.

Show Q3 answer

For $\left[-1, 0, 1\right]$ , exponentials are approximately $\left[0.3679, 1, 2.7183\right]$ , total 4.0862, giving $\left[0.0900, 0.2447, 0.6652\right]$ . Equal scores produce equal exponentials, so $\operatorname{softmax}\left(\left[4, 4, 4\right]\right)$ = $\left[\frac{1}{3}, \frac{1}{3}, \frac{1}{3}\right]$ . Rounding can make displayed probabilities sum slightly below 1.

Q4

Without calculating huge exponentials, compare $\operatorname{softmax}\left(\left[0, \ln\left(3\right)\right]\right)$ and $\operatorname{softmax}\left(\left[100, 100 + \ln\left(3\right)\right]\right)$ . Explain the algebra that makes your answer valid.

Show Q4 answer

Both are $[\tfrac14,\tfrac34]$ . In the second expression the exponentials are $e^{100}[1,3]$ , and the common factor $e^{100}$ cancels from numerator and denominator.

Q5 · Local fallback Only if the published source is unavailable

For scores $\left[-1, 2\right]$ , someone divides by their sum and returns $\left[-1, 2\right]$ . Explain why this is invalid as probabilities and compute the correct softmax.

Show Q5 answer

Their sum is 1, but $-1$ is negative and $2$ exceeds 1. Softmax uses exponentials first: $\frac{[e^{-1},e^2]}{e^{-1}+e^2}\approx[0.0474,0.9526]$ .

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Calculate $\operatorname{softmax}\left(\left[\ln\left(4\right), 0\right]\right)$ and say whether $\operatorname{softmax}\left(\left[\ln\left(4\right)+7, 7\right]\right)$ changes it.

Show final answer

$\frac{[4,1]}{5}=[0.8,0.2]$ . Adding 7 to both scores leaves the result unchanged.

WHEN YOU FINISH

Attempt local questions 1–4 and the linked D2L exercise (or local question 5 if the link is unavailable), correct them, and answer the exit without the key. Explain why a shared exponential factor cancels.

LESSON 18 /24Review & apply

Probability, tensor axes, and softmax review

Monday, December 7, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Practice35 min

Break5 min

Correct25 min

Final check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

State both requirements for a probability vector.

Show R1 answer

Every entry lies between 0 and 1, and the entries sum to 1.

R2

State the steps of softmax and one transformation of the scores that leaves it unchanged.

Show R2 answer

Exponentiate each score, add the exponentials, divide each by the sum. Adding the same constant to every score leaves the probabilities unchanged.

Review reference notes

LESSON NOTES

No new video or reading today. Use the review timetable. During the independent block keep earlier primers and answers closed; a calculator is allowed. Open the key only during the review block. Rowwise means separately across the entries in each row. Columnwise means separately down each column.

02

Practice · L18

22:40–23:15 · 35 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

For T = $\begin{bmatrix}1 & 2 & 3 \\ 4 & 5 & 6\end{bmatrix}$ , calculate T.sum(dim=1, keepdim=True) and T.sum(dim=0, keepdim=True). Give their shapes.

Show Q1 answer

Across columns within each row: $\begin{bmatrix}6 \\ 15\end{bmatrix}$ , shape $\left[2, 1\right]$ . Down rows within each column: $\begin{bmatrix}5 & 7 & 9\end{bmatrix}$ , shape $\left[1, 3\right]$ .

Q2

Let E = $\begin{bmatrix}1 & 2 & 1 \\ 2 & 2 & 4\end{bmatrix}$ . To make each row sum to 1, calculate E / E.sum(dim=1, keepdim=True). What would happen if the denominator were E.sum() with no dimension?

Show Q2 answer

Row sums are $\begin{bmatrix}4 \\ 8\end{bmatrix}$ , giving $\begin{bmatrix}0.25 & 0.5 & 0.25 \\ 0.25 & 0.25 & 0.5\end{bmatrix}$ . E.sum() is 12, producing $\begin{bmatrix}\frac{1}{12} & \frac{2}{12} & \frac{1}{12} \\ \frac{2}{12} & \frac{2}{12} & \frac{4}{12}\end{bmatrix}$ . Those rows sum to $\frac{1}{3}$ and $\frac{2}{3}$ , not 1.

Q3

Values are $\left[-2, 4, 8\right]$ and probabilities are $\left[0.25, 0.5, 0.25\right]$ . Find the expected value. Is it the same as the unweighted mean?

Show Q3 answer

Expectation $=-0.5+2+2=3.5$ . The unweighted mean is $\frac{-2+4+8}{3}=\frac{10}{3}\approx3.3333$ . They differ because the probabilities are unequal.

Q4

Calculate the weighted average of $\left[1, 3\right]$ and $\left[5, -1\right]$ using weights $\left[0.75, 0.25\right]$ . Check that each output coordinate lies between the corresponding input coordinates.

Show Q4 answer

$0.75[1,3]+0.25[5,-1]=[2,2]$ . The first coordinate lies between 1 and 5; the second lies between $-1$ and 3. Nonnegative weights summing to 1 give this property.

Q5

Compute $\operatorname{softmax}\left(\left[\ln\left(2\right), \ln\left(2\right), \ln\left(4\right)\right]\right)$ and compare it with $\operatorname{softmax}\left(\left[5+\ln\left(2\right), 5+\ln\left(2\right), 5+\ln\left(4\right)\right]\right)$ .

Show Q5 answer

The first exponentials are $\left[2, 2, 4\right]$ , total 8, so both results are $\left[0.25, 0.25, 0.5\right]$ . The common $\exp\left(5\right)$ factor cancels in the second calculation.

Q6

A function applies softmax separately to the columns of Z = $\begin{bmatrix}0 & \ln\left(3\right) \\ 0 & \ln\left(3\right)\end{bmatrix}$ , although each row should be one example's distribution. Calculate its output and each row sum. Then give the correct rowwise output.

Show Q6 answer

Each column has equal scores, so the incorrect columnwise result is $\begin{bmatrix}0.5 & 0.5 \\ 0.5 & 0.5\end{bmatrix}$ . Each row happens to sum to 1, so that check alone misses the bug. Correct rowwise probabilities are $\begin{bmatrix}0.25 & 0.75 \\ 0.25 & 0.75\end{bmatrix}$ . This known input checks the intended meaning of the axes.

Five-minute break23:15–23:20 · leave the screen

03

Check & correct

23:20–23:45 · 25 minutes

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

04

Final check

23:45–00:00 · 15 minutes

For Z = $\begin{bmatrix}0 & \ln\left(2\right) & 0 \\ \ln\left(3\right) & 0 & 0\end{bmatrix}$ , compute rowwise softmax. State one check beyond merely checking each row sum.

Show final answer

$\begin{bmatrix}0.25 & 0.5 & 0.25 \\ 0.6 & 0.2 & 0.2\end{bmatrix}$ . Also compare with these hand-calculated reference values or check that the largest score in each row receives its largest probability. Row sums alone do not identify every wrong-axis computation.

WHEN YOU FINISH

Finish attempts at all six exercises; during feedback explain the wrong-axis failure in exercise 6 and complete the exit independently. Keep this example for later code reviews.

LESSON 19 /24Learn & practice

Numerical accuracy, stable softmax, and storage

Wednesday, December 9, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

Compute $\operatorname{softmax}\left(\left[0, \ln\left(2\right)\right]\right)$ .

Show R1 answer

Exponentials $\left[1, 2\right]$ , total 3, probabilities $\left[\frac{1}{3}, \frac{2}{3}\right]$ .

R2

What remains unchanged when the same scalar is subtracted from every score?

Show R2 answer

The softmax probabilities, because the exponential scale factor cancels.

02

Watch & read

22:40–23:00 · 20 minutes

Read4 min

PyTorch: Numerical accuracy

Read only the introduction under Numerical accuracy. Stop before Batched computations or slice computations. Do not read the hardware sections.

Read6 min

D2L §4.5.2: Softmax Revisited

Read from the heading through equation (4.5.1) and the following paragraph explaining that the denominator is between 1 and q. Stop before the paragraph beginning 'In particular, in backpropagation'.

Included below10 min

Primer below

Read the full primer and reproduce the stable-softmax calculation and the storage example.

LESSON NOTES

A floating-point format stores a finite set of numbers. Intermediate arithmetic is rounded; very large results may overflow to infinity, and tiny results may underflow to zero. Identical mathematical expressions can therefore produce slightly different stored values.

For finite scores, stable softmax first subtracts the largest score m. Compute $\exp\left(z_i-m\right)$ and divide by their sum. Example: $\left[1000, 1000+\ln\left(2\right)\right]$ becomes $\left[-\ln\left(2\right), 0\right]$ , whose exponentials are $\left[0.5, 1\right]$ ; probabilities are $\left[\frac{1}{3}, \frac{2}{3}\right]$ . At least one exponential is 1 and none exceeds 1, so this removes large exponentials. It does not make floating-point computation exact; sufficiently tiny probabilities can still round to zero.

For a reference r and computed value c, absolute error is $|c-r|$ . Relative error is $\frac{|c-r|}{|r|}$ when r is nonzero. The bars $|\cdot|$ mean absolute value. When checking a calculation, we will use the explicitly supplied rule $|c-r|\leq \mathrm{atol}+\mathrm{rtol}\,|r|$ . The absolute term matters near zero. The tolerances in these worksheets are exercise settings, not universal production defaults.

Storage for a dense tensor's element payload is the product of its dimensions times bytes per element. float32 uses 4 bytes; float16 uses 2 bytes. For shape $\left[2, 3, 4\right]$ , there are 24 elements and float32 payload is 96 bytes. 1 KiB = 1024 bytes. This excludes object metadata, temporary tensors, allocator overhead, and any other buffers.

Significant digits count from the first nonzero digit. Three significant digits retain three digits of precision: 12.34 rounds to 12.3, 0.01234 rounds to 0.0123, and 1001 rounds to $1.00\times 10^3=1000$ . Trailing zeros in 1.00 indicate the stated precision. In the toy arithmetic exercise, round after every addition, not just at the end.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L19

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

A calculator cannot evaluate $\exp\left(1000\right)$ . Calculate stable $\operatorname{softmax}\left(\left[1000, 1001, 1002\right]\right)$ without doing so. Show the shifted scores and explain why the result equals $\operatorname{softmax}\left(\left[0, 1, 2\right]\right)$ .

Show Q1 answer

Subtract 1002 to get $\left[-2, -1, 0\right]$ . Exponentials are approximately $\left[0.1353, 0.3679, 1\right]$ , total 1.5032, yielding $\left[0.0900, 0.2447, 0.6652\right]$ . The original and $\left[0, 1, 2\right]$ differ by a common constant, which cannot affect softmax.

Q2

Find absolute and relative error for (a) reference 100, result 100.08 and (b) reference 0.001, result 0.00108. Compare the interpretations.

Show Q2 answer

The absolute errors are 0.08 and 0.00008 respectively. Relative errors are $\frac{0.08}{100}=0.0008=0.08\%$ and $\frac{0.00008}{0.001}=0.08=8\%$ . The smaller absolute error in (b) is much larger relative to its reference.

Q3

Use atol = 0.00001 and rtol = 0.001. Apply $|c-r|\leq \mathrm{atol}+\mathrm{rtol}\,|r|$ to both pairs in exercise 2, then to reference 0 and result 0.000008. State pass or fail, showing the allowed error.

Show Q3 answer

(a) allowed error $0.00001+0.001\times 100 = 0.10001$ ; actual 0.08, so pass. (b) allowed error $0.00001+0.001\times 0.001 = 0.000011$ ; actual 0.00008, so fail. At zero the allowed error is 0.00001; actual 0.000008, so pass. Relative error alone would be undefined at zero.

Q4

Calculate element count and payload bytes for shape $\left[2, 4, 8\right]$ in float32 and shape $\left[2, 3, 4, 5\right]$ in float16. Would these figures alone predict total process memory?

Show Q4 answer

First: $2\times 4\times 8 = 64$ elements, $64\times 4 = 256$ bytes. Second: $2\times 3\times 4\times 5 = 120$ elements, $120\times 2 = 240$ bytes. Total process memory also includes metadata, allocations, temporaries, program code, and other data.

Q5

A toy decimal machine rounds every arithmetic result to three significant digits. Let a = 1000, b = 1, c = -1000. Calculate (a+b)+c and a+(b+c), rounding after each addition. Explain the difference from exact arithmetic.

Show Q5 answer

$a+b=1001$ rounds to 1000, then $1000-1000=0$ . In the other order, $b+c=-999$ is exactly representable, then $1000-999=1$ . Exact arithmetic gives 1 in either order; intermediate rounding makes the two evaluation orders differ.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Compute stable $\operatorname{softmax}\left(\left[1000, 1000+\ln\left(4\right)\right]\right)$ and the float16 payload for a $\left[4, 8\right]$ tensor. Explain one reason exact equality is too strict for comparing equivalent floating-point computations.

Show final answer

Shift by 1000+ $\ln\left(4\right)$ : $\left[-\ln\left(4\right), 0\right]$ , exponentials $\left[0.25, 1\right]$ , probabilities $\left[0.2, 0.8\right]$ . Storage = $4\times 8\times 2 = 64$ bytes. Different rounding orders can yield small numerical differences.

WHEN YOU FINISH

Attempt all five exercises; check the numerical work and reproduce the exit calculation independently. Keep the distinction between mathematical equality and a stated numerical tolerance explicit.

LESSON 20 /24Learn & practice

Mean, population variance, and normalization axes

Sunday, December 13, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

Find the equally weighted average of $\left[1, 3, 5\right]$ .

Show R1 answer

( $1+3+5$ )/3 = 3.

R2

For a $\left[2, 3\right]$ matrix, what shape must retained row means have to subtract one mean from each row?

Show R2 answer

$\left[2, 1\right]$ .

02

Watch & read

22:40–23:00 · 20 minutes

Read5 min

D2L §2.6.6: Expectations — scalar variance passage

Within Expectations, begin at the paragraph immediately before equation (2.6.14), which introduces variance as squared differences. Read equation (2.6.14) and the following paragraph defining standard deviation. Stop before 'Lastly, the variance of a function'.

Included below15 min

Primer below

Read the full primer and reproduce its [2, 6] example. Write the mean, deviations, squared deviations, variance, and normalized values on separate lines.

LESSON NOTES

For $n$ values $x_1,\ldots,x_n$ with equal weight, the mean is $\mu=\frac{1}{n}\sum_{i=1}^{n}x_i$ . The population variance is $\sigma^2=\frac{1}{n}\sum_{i=1}^{n}(x_i-\mu)^2$ . Standard deviation is $\sqrt{\sigma^2}$ . Use division by n throughout these worksheets. The alternative division by $n-1$ estimates population variance from a sample and is not the convention used here.

Example: x = $\left[2, 6\right]$ has mean 4, deviations $\left[-2, 2\right]$ , squared deviations $\left[4, 4\right]$ , variance 4, and standard deviation 2. Centering means subtracting the mean. Dividing the centered values by the standard deviation gives $\left[-1, 1\right]$ , with mean 0 and population variance 1.

For a positive number $\varepsilon$ (epsilon), use $y_i=\frac{x_i-\mu}{\sqrt{\sigma^2+\varepsilon}}$ . Epsilon prevents division by zero for constant input. A positive epsilon also means the output variance is $\frac{\sigma^2}{\sigma^2+\varepsilon}$ , not exactly 1. For x = $\left[5, 5\right]$ and epsilon = 0.01, mean is 5, variance is 0, denominator is 0.1, and output is $\left[0, 0\right]$ .

Choose the axis by the meaning of the data. For a matrix with examples in rows and features in columns, per-example normalization calculates one mean and variance for each row. With keepdim=True these have shape [number_of_examples, 1], so subtraction and division broadcast across that row. A column mean answers a different question.

If checking with PyTorch, calculate population variance as ((X - mean)**2).mean(dim=..., keepdim=True), using the same axis as mean. This explicit expression avoids relying on a library's default sample-variance convention.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L20

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

For x = $\left[1, 1, 5, 5\right]$ , calculate mean, deviations, squared deviations, population variance, standard deviation, and normalized values using epsilon = 0.

Show Q1 answer

Mean = $\frac{12}{4} = 3$ ; deviations $\left[-2, -2, 2, 2\right]$ ; squared deviations $\left[4, 4, 4, 4\right]$ ; variance = $\frac{16}{4} = 4$ ; standard deviation = 2; normalized values $\left[-1, -1, 1, 1\right]$ .

Q2

Normalize each row of X = $\begin{bmatrix}1 & 3 \\ 2 & 6\end{bmatrix}$ separately with epsilon = 0. Give the means, population variances, and output. State the shapes when mean and variance retain the reduced dimension.

Show Q2 answer

Means $\begin{bmatrix}2 \\ 4\end{bmatrix}$ and variances $\begin{bmatrix}1 \\ 4\end{bmatrix}$ , both shape $\left[2, 1\right]$ . Centered rows $\left[-1, 1\right]$ and $\left[-2, 2\right]$ , divided by standard deviations 1 and 2, give $\begin{bmatrix}-1 & 1 \\ -1 & 1\end{bmatrix}$ , shape $\left[2, 2\right]$ .

Q3

Now normalize the columns of the same X = $\begin{bmatrix}1 & 3 \\ 2 & 6\end{bmatrix}$ with epsilon = 0. Give the means, variances, output, and one sentence explaining why it differs from exercise 2.

Show Q3 answer

Column means $\begin{bmatrix}1.5 & 4.5\end{bmatrix}$ , variances $\begin{bmatrix}0.25 & 2.25\end{bmatrix}$ , standard deviations $\left[0.5, 1.5\right]$ . The output is $\begin{bmatrix}-1 & -1 \\ 1 & 1\end{bmatrix}$ . Each feature is compared across examples, whereas exercise 2 compared features within each example.

Q4

For x = $\left[7, 7, 7\right]$ , compute $y=\frac{x-\mu}{\sqrt{\sigma^2+\varepsilon}}$ with epsilon = 0.01. What fails if epsilon is zero?

Show Q4 answer

Mean 7, variance 0, denominator $\sqrt{0.01}$ = 0.1, and output $\left[0, 0, 0\right]$ . With epsilon zero, every entry attempts $\frac{0}{0}$ , which is undefined and ordinarily produces NaN in floating-point arithmetic.

Q5

Use x = $\left[1, 3\right]$ and deliberately large epsilon = 3. Calculate normalized values and their population variance. Does adding epsilon preserve variance exactly equal to 1?

Show Q5 answer

Mean 2, variance 1, denominator $\sqrt{1+3}=2$ , output $[-0.5,0.5]$ . Its mean is 0 and variance is $\frac{0.25+0.25}{2}=0.25$ . This matches $\frac{1}{1+3}$ ; positive epsilon changes the scale.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Normalize x = $\left[-2, 2\right]$ with epsilon = 0. State mean, population variance, and output. Then give the output for x = $\left[3, 3\right]$ with epsilon = 0.01.

Show final answer

First: mean 0, variance 4, standard deviation 2, output $\left[-1, 1\right]$ . Second: mean 3, variance 0, output $\left[0, 0\right]$ .

WHEN YOU FINISH

Attempt and correct all five exercises. Complete the exit without the key, and label row versus column normalization explicitly in exercises 2 and 3.

LESSON 21 /24Review & apply

Trace a complete score-to-weighted-sum calculation

Monday, December 14, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Practice35 min

Break5 min

Correct25 min

Final check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

If P has shape $\left[2, 3\right]$ and V has shape $\left[3, 2\right]$ , what is the shape of P @ V?

Show R1 answer

$\left[2, 2\right]$ : two examples, each with a two-coordinate weighted value.

R2

What denominator is used for each row in rowwise softmax?

Show R2 answer

The sum of the exponentials in that same row, not a sum over the entire matrix or down each column.

Review reference notes

LESSON NOTES

No new resource today. Use paper and a calculator; execution is optional. Work through X @ W + b, then rowwise softmax, then P @ V. Each row of X is an example, each column of scores identifies one of three choices, and the rows of V give a two-number value for each choice. This is a mathematical composition; no knowledge of attention architecture is required.

Use X = $\begin{bmatrix}1 & 0 \\ 0 & 1\end{bmatrix}$ , W = $\begin{bmatrix}0 & \ln\left(2\right) & \ln\left(3\right) \\ \ln\left(3\right) & \ln\left(2\right) & 0\end{bmatrix}$ , b = $\left[0, 0, 0\right]$ , and V = $\begin{bmatrix}0 & 2 \\ 3 & 0 \\ 6 & 4\end{bmatrix}$ . Keep fractions exact where convenient.

02

Practice · L21

22:40–23:15 · 35 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Using the inputs in the primer, calculate Z = X @ W + b. Give the shapes of X, W, b, Z, and V, and explain which direction b broadcasts.

Show Q1 answer

X $\left[2, 2\right]$ , W $\left[2, 3\right]$ , b $\left[3\right]$ , Z $\left[2, 3\right]$ , V $\left[3, 2\right]$ . Since X is the identity matrix, Z = $\begin{bmatrix}0 & \ln\left(2\right) & \ln\left(3\right) \\ \ln\left(3\right) & \ln\left(2\right) & 0\end{bmatrix}$ . b is applied across the three columns of every row.

Q2

Calculate P by applying softmax across each row of Z. Show the exponential row sums and check the probability sums.

Show Q2 answer

Exponentials are $\begin{bmatrix}1 & 2 & 3 \\ 3 & 2 & 1\end{bmatrix}$ , each row sums to 6. P = $\begin{bmatrix}\frac{1}{6} & \frac{1}{3} & \frac{1}{2} \\ \frac{1}{2} & \frac{1}{3} & \frac{1}{6}\end{bmatrix}$ , shape $\left[2, 3\right]$ . Each probability row sums to 1.

Q3

Compute O = P @ V. For O[0], show the weighted sum of the three rows of V and verify that each coordinate is within the corresponding minimum and maximum of V.

Show Q3 answer

For O[0], the sum is $\frac16[0,2]+\frac13[3,0]+\frac12[6,4]=[4,\frac73]$ . For O[1], the sum is $\frac12[0,2]+\frac13[3,0]+\frac16[6,4]=[2,\frac53]$ . Output shape [2,2]. First coordinates lie in $[0,6]$ ; second coordinates lie in $[0,4]$ .

Q4

A program uses P_wrong = softmax(Z, dim=0), meaning down each column. Calculate P_wrong, its row sums, and P_wrong @ V. Which test would expose the wrong axis?

Show Q4 answer

Each exponential column sums to 4. P_wrong = $\begin{bmatrix}\frac{1}{4} & \frac{1}{2} & \frac{3}{4} \\ \frac{3}{4} & \frac{1}{2} & \frac{1}{4}\end{bmatrix}$ . Both rows sum to 1.5. P_wrong @ V = $\begin{bmatrix}6 & 3.5 \\ 3 & 2.5\end{bmatrix}$ . Comparing row sums with 1 exposes this case; comparing with the exact reference P in exercise 2 also does.

Q5

Add 1000 to every score in only the first row of Z. Predict the new P and O without evaluating $\exp\left(1000\right)$ . Write how to compute the first row stably.

Show Q5 answer

P and O are unchanged because softmax is applied independently to each row. For the first row subtract its maximum, 1000+ $\ln\left(3\right)$ , giving $\left[-\ln\left(3\right), \ln\left(2\right)-\ln\left(3\right), 0\right]$ ; exponentials $\left[\frac{1}{3}, \frac{2}{3}, 1\right]$ , total 2, giving $\left[\frac{1}{6}, \frac{1}{3}, \frac{1}{2}\right]$ .

Five-minute break23:15–23:20 · leave the screen

03

Check & correct

23:20–23:45 · 25 minutes

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

04

Final check

23:45–00:00 · 15 minutes

For one row with scores $\left[0, \ln\left(3\right)\right]$ and value rows $\begin{bmatrix}2 & 0 \\ 0 & 4\end{bmatrix}$ , calculate its probabilities and weighted output. Explain the meaning of both numbers in the output.

Show final answer

Probabilities $[\tfrac14,\tfrac34]$ ; weighted output $\frac14[2,0]+\frac34[0,4]=[0.5,3]$ . Each coordinate averages the corresponding coordinate of the two value rows using the same probabilities.

WHEN YOU FINISH

Trace all five exercises, correct any axis mistakes, and answer the exit independently. Retain the calculations as a hand-worked reference for later implementations.

LESSON 22 /24Learn & practice

Fixed consolidation: matrices, axes, and numerical checks

Wednesday, December 16, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Learn20 min

Break5 min

Practice40 min

Check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

In A @ x, which dimension of A must equal the length of x?

Show R1 answer

The number of columns of A.

R2

How do you prevent overflow from large positive scores when evaluating softmax?

Show R2 answer

Subtract the largest score in each intended probability group before exponentiation.

02

Watch & read

22:40–23:00 · 20 minutes

Included below20 min

Consolidation primer below

Spend the 20-minute lesson block reproducing all five small examples below, then cover them. No resource selection is needed and no new theory is introduced.

LESSON NOTES

Reproduce these five examples in order.

Matrix-vector product: $\begin{bmatrix}1&2\\3&0\end{bmatrix}\begin{bmatrix}2\\1\end{bmatrix}=\begin{bmatrix}4\\6\end{bmatrix}$ . It is also $2\begin{bmatrix}1\\3\end{bmatrix}+\begin{bmatrix}2\\0\end{bmatrix}$ , using the columns.
Axes: for $\begin{bmatrix}1 & 2 & 3 \\ 4 & 5 & 6\end{bmatrix}$ , retained row sums are $\begin{bmatrix}6 \\ 15\end{bmatrix}$ , whereas retained column sums are $\begin{bmatrix}5 & 7 & 9\end{bmatrix}$ . Transpose gives $\begin{bmatrix}1 & 4 \\ 2 & 5 \\ 3 & 6\end{bmatrix}$ ; reshaping to $\left[3, 2\right]$ instead gives $\begin{bmatrix}1 & 2 \\ 3 & 4 \\ 5 & 6\end{bmatrix}$ .
Softmax: scores $\left[0, \ln\left(3\right)\right]$ give $\left[\frac{1}{4}, \frac{3}{4}\right]$ . With value rows $\begin{bmatrix}4 & 0 \\ 0 & 4\end{bmatrix}$ , their weighted sum is $\left[1, 3\right]$ . Adding 1000 to both scores changes neither result.
Normalization: $\left[2, 6\right]$ has mean 4 and population variance 4. Subtract 4 and divide by $\sqrt{4}$ to get $\left[-1, 1\right]$ .
Verification: float32 shape $\left[2, 3\right]$ contains 6 elements and takes 24 payload bytes. With reference 2, result 2.001, atol 0.001, rtol 0.001, absolute error 0.001 is within $0.001+0.001\times 2$ = 0.003.

These are fixed review tasks for everyone, including anyone who feels caught up. Use the normal learn timetable, with the primer occupying the resource block.

Five-minute break23:00–23:05 · leave the screen

03

Practice · L22

23:05–23:45 · 40 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

Let A = $\begin{bmatrix}2 & 0 \\ 1 & -1\end{bmatrix}$ and x = $\left[3, 2\right]$ . Calculate A @ x using row dot products, then verify with a weighted sum of columns.

Show Q1 answer

Row products give $[2\times3+0\times2,\;1\times3-1\times2]=[6,1]$ . Columns give $3[2,1]+2[0,-1]=[6,3]+[0,-2]=[6,1]$ .

Q2

For T = $\begin{bmatrix}0 & 1 & 2 \\ 3 & 4 & 5\end{bmatrix}$ , write T transposed and T reshaped to $\left[3, 2\right]$ in row-major order. Why does matching output shape not show that the operations are interchangeable?

Show Q2 answer

Transpose is $\begin{bmatrix}0 & 3 \\ 1 & 4 \\ 2 & 5\end{bmatrix}$ . Reshape is $\begin{bmatrix}0 & 1 \\ 2 & 3 \\ 4 & 5\end{bmatrix}$ . Both shapes are $\left[3, 2\right]$ , but transpose swaps row/column indices while reshape regroups the flattened sequence.

Q3

X has shape $\left[2, 3, 4\right]$ , W $\left[4, 5\right]$ , and b $\left[5\right]$ . State the shape of Y = X @ W + b and of Y.mean(dim=-1, keepdim=True). Explain what the mean groups together.

Show Q3 answer

Y has shape $\left[2, 3, 5\right]$ . The retained mean shape is $\left[2, 3, 1\right]$ ; it averages the five output features separately for each batch item and sequence position.

Q4

Find $\operatorname{softmax}\left(\left[0, \ln\left(2\right), 0\right]\right)$ , then use it to average scalar values $\left[2, 4, 10\right]$ .

Show Q4 answer

Exponentials $\left[1, 2, 1\right]$ , total 4, probabilities $\left[\frac{1}{4}, \frac{1}{2}, \frac{1}{4}\right]$ . Weighted value = $0.5+2+2.5$ = 5.

Q5

For x = $\left[2, 6\right]$ , calculate mean, population variance, and normalization with epsilon = 0. Explain the denominator change for a constant input with positive epsilon.

Show Q5 answer

Mean 4, deviations $\left[-2, 2\right]$ , variance 4, denominator 2, output $\left[-1, 1\right]$ . For constant input variance is 0; positive epsilon gives a positive denominator $\sqrt{\varepsilon}$ and centered output zero.

Q6

Calculate the float32 payload of a $\left[2, 3, 5\right]$ tensor. Then compare reference 1 and result 1.0004 with atol = 0.0001 and rtol = 0.001.

Show Q6 answer

The tensor has $2\times3\times5=30$ elements and $30\times4=120$ payload bytes. Absolute error $0.0004$ ; allowed error $0.0001+0.001\times1=0.0011$ . The comparison passes.

04

Check & correct

23:45–00:00 · 15 minutes total

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

05

Final check

Within the final 15-minute block

Use x = $\left[1, 2\right]$ , W = $\begin{bmatrix}1 & 0 \\ 0 & 2\end{bmatrix}$ , and b = $\left[0, -3\right]$ . Compute z = x @ W + b and $\operatorname{softmax}\left(z\right)$ .

Show final answer

x @ W = $\left[1, 4\right]$ ; adding b gives $\left[1, 1\right]$ . Equal scores give probabilities $\left[0.5, 0.5\right]$ .

WHEN YOU FINISH

Complete all six fixed exercises and the exit, recording which answers required help. The next session is the readiness assessment even if today's corrections took the full feedback block.

LESSON 23 /24Review & apply

Readiness assessment

Sunday, December 20, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Practice35 min

Break5 min

Correct25 min

Final check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

Name two checks you can apply to a computed probability vector.

Show R1 answer

Entries are nonnegative and sum to 1. A known reference example is also needed to check that the probabilities correspond to the intended calculation.

R2

Explain what keeping a reduced tensor dimension does.

Show R2 answer

It leaves that axis with size 1, which can support broadcasting back across the original axis.

Review reference notes

LESSON NOTES

Use the review timetable. In the 35-minute independent block, attempt all eight questions with paper and a calculator; keep primers and answer keys closed. Mark your answers final before opening the key. During feedback, award one point per question only when all requested parts and the explanation are correct. An arithmetic slip corrected independently before opening the key is allowed. Record that original score out of eight before correcting mistakes.

If you need more than the independent-work block, take the five-minute break and use the remaining session to continue with solutions closed. Stop at midnight. In the next scheduled slot, finish the unanswered questions before opening the key, then check, correct, and complete the exit question. Score the original answers only once every question has an attempt. Shift later lessons as needed; speed is not part of the readiness criterion.

02

Practice · L23

22:40–23:15 · 35 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

For u = $\left[1, 2\right]$ and v = $\left[2, -1\right]$ , calculate $u\cdot v$ and the length of u. Are u and v perpendicular?

Show Q1 answer

Dot product $1\times2+2\times(-1)=0$ . Length $\sqrt{1^2+2^2}=\sqrt{5}$ . They are perpendicular because both are nonzero and their dot product is zero.

Q2

Let A = $\begin{bmatrix}1 & 2 \\ 0 & -1\end{bmatrix}$ and x = $\left[2, 3\right]$ . Calculate A @ x and show the same result as a combination of A's columns.

Show Q2 answer

A @ x gives $[8,-3]$ . Column calculation: $2[1,0]+3[2,-1]=[8,-3]$ .

Q3

For T = $\begin{bmatrix}1 & 2 & 3 \\ 4 & 5 & 6\end{bmatrix}$ , write its transpose and its retained row sums, including both output shapes.

Show Q3 answer

Transpose $\begin{bmatrix}1 & 4 \\ 2 & 5 \\ 3 & 6\end{bmatrix}$ , shape $\left[3, 2\right]$ . Row sums $\begin{bmatrix}6 \\ 15\end{bmatrix}$ , shape $\left[2, 1\right]$ .

Q4

X has shape $\left[2, 4, 3\right]$ , W $\left[3, 5\right]$ , and b $\left[5\right]$ . Give the shape of Z = X @ W + b. To make a distribution over the five outputs for each sequence position, which softmax axis is correct? What shape results from summing those probabilities with keepdim=True?

Show Q4 answer

Z shape $\left[2, 4, 5\right]$ . Use dim=-1 (equivalently dim=2). Retained sums have shape $\left[2, 4, 1\right]$ and each value should be approximately 1.

Q5

Check whether p = $\left[0.25, 0.75\right]$ is a probability vector. Using values $\left[-2, 6\right]$ , calculate its weighted average and explain why that result need not be a possible sampled value.

Show Q5 answer

It is valid: nonnegative and sums to 1. Weighted average $=0.25\times(-2)+0.75\times6=4$ . Samples are $-2$ or 6; expectation need not be one of them.

Q6

Calculate $\operatorname{softmax}\left(\left[\ln\left(2\right), \ln\left(6\right)\right]\right)$ . Then write the shifted scores you would exponentiate for stable $\operatorname{softmax}\left(\left[1000+\ln\left(2\right), 1000+\ln\left(6\right)\right]\right)$ , and give its probabilities.

Show Q6 answer

Exponentials $\left[2, 6\right]$ give $\left[0.25, 0.75\right]$ . Subtract 1000+ $\ln\left(6\right)$ : shifted scores $\left[\ln\left(2\right)-\ln\left(6\right), 0\right]$ = $\left[\ln\left(\frac{1}{3}\right), 0\right]$ . Exponentials $\left[\frac{1}{3}, 1\right]$ give the same probabilities.

Q7

For $\left[2, 4\right]$ , compute mean, population variance, and normalized values with epsilon = 0. What is the output for $\left[4, 4\right]$ with epsilon = 0.01?

Show Q7 answer

Mean 3, variance $\frac{(-1)^2+1^2}{2}=1$ , output $[-1,1]$ . Constant $[4,4]$ has centered values $[0,0]$ and positive denominator 0.1, so output $[0,0]$ .

Q8

Find the float16 payload bytes for shape $\left[3, 4, 5\right]$ . For reference 2 and result 2.002, determine whether atol = 0.001 and rtol = 0.001 permit the difference.

Show Q8 answer

$3\times 4\times 5\times 2 = 120$ bytes. Error 0.002; allowed error $0.001+0.001\times 2$ = 0.003. It passes.

Five-minute break23:15–23:20 · leave the screen

03

Check & correct

23:20–23:45 · 25 minutes

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

04

Final check

23:45–00:00 · 15 minutes

After reviewing, close the answers. For X shape $\left[3, 2\right]$ , W shape $\left[2, 4\right]$ , and b shape $\left[4\right]$ , state the score shape, softmax axis for four choices per row, and retained probability-sum shape. If a row's scores are all equal, give its probabilities.

Show final answer

Scores $\left[3, 4\right]$ ; softmax dim=1 or -1; retained sums $\left[3, 1\right]$ . An equal-score row gives $\left[0.25, 0.25, 0.25, 0.25\right]$ .

WHEN YOU FINISH

Record the first-attempt score out of eight and correct every answer. Complete the exit without the key. Continue to lesson 24 for a delayed check; use its explicit rule to determine whether to proceed afterward.

LESSON 24 /24Review & apply

Delayed practice and final integration

Monday, December 21, 2026·22:30–00:00 JST·90 minutes

Recall10 min

Practice35 min

Break5 min

Correct25 min

Final check15 min

01

Recall

22:30–22:40 · 10 minutes

Try these without notes. Reveal each answer after your attempt.

R1

What does matrix multiplication combine that elementwise multiplication does not?

Show R1 answer

Each matrix-product entry sums products across the shared inner dimension; elementwise multiplication only multiplies corresponding broadcasted entries.

R2

Why should a test use an example with a known result as well as checking shape?

Show R2 answer

Different operations and wrong axes can produce the same shape while producing incorrect values or meaning.

Review reference notes

LESSON NOTES

No new material. Use the review timetable. In the independent block, answer the five exercises without earlier notes or solutions; use a calculator. Mark your answers final before opening the key. During feedback, award one point per exercise only when every requested part and explanation is correct. Record the original score, then correct the work. Use the last 15 minutes for the fresh exit calculation and readiness decision below.

If you need more than the independent-work block, take the five-minute break and use the remaining session to continue with solutions closed. Stop at midnight. In the next scheduled slot, finish the unanswered questions before opening the key, then check, correct, and complete the exit question. Score the original answers only once every question has an attempt. Shift later lessons as needed; speed is not part of the readiness criterion.

02

Practice · L24

22:40–23:15 · 35 minutes

Write an attempt before opening an answer. Check and correct your work in the next block.

Q1

For A = $\begin{bmatrix}1 & -1 \\ 2 & 0\end{bmatrix}$ and x = $\left[3, 1\right]$ , calculate A @ x and the length of x. Verify the product using A's columns.

Show Q1 answer

A @ x gives $[3-1,6]=[2,6]$ . Length $\sqrt{3^2+1^2}=\sqrt{10}$ . Column calculation $3[1,2]+[-1,0]=[2,6]$ .

Q2

X has shape $\left[2, 3, 2\right]$ , W = $\begin{bmatrix}1 & 0 & 1 \\ 0 & 1 & -1\end{bmatrix}$ , and b = $\left[0, 1, 0\right]$ . Give the shape of Y = X @ W + b. If X[0,0] = $\left[2, 1\right]$ , calculate Y[0,0]. Which axis holds output features, despite two axes having size 3?

Show Q2 answer

Y has shape $\left[2, 3, 3\right]$ . $\left[2, 1\right]$ @ W = $\left[2, 1, 1\right]$ , so adding b gives $\left[2, 2, 1\right]$ . The last axis holds output features; the middle axis still holds sequence positions. Matching axis lengths do not make their meanings interchangeable.

Q3

Compute $\operatorname{softmax}\left(\left[0, \ln\left(2\right), 0\right]\right)$ . Use those probabilities to combine the rows of V = $\begin{bmatrix}0 & 4 \\ 2 & 0 \\ 4 & 4\end{bmatrix}$ . State the output shape and calculation.

Show Q3 answer

Probabilities $[\tfrac14,\tfrac12,\tfrac14]$ . Weighted sum $\frac14[0,4]+\frac12[2,0]+\frac14[4,4]=[2,2]$ . The three contributions are $[0,1]$ , $[1,0]$ , and $[1,1]$ ; output shape [2].

Q4

Normalize each row of X = $\begin{bmatrix}2 & 4 \\ 6 & 10\end{bmatrix}$ using population variance and epsilon = 0. Give retained means, variances, and output. Explain why a single global mean would describe another operation.

Show Q4 answer

Means $\begin{bmatrix}3 \\ 8\end{bmatrix}$ , variances $\begin{bmatrix}1 \\ 4\end{bmatrix}$ , centered values $\begin{bmatrix}-1 & 1 \\ -2 & 2\end{bmatrix}$ , output $\begin{bmatrix}-1 & 1 \\ -1 & 1\end{bmatrix}$ . Per-row statistics treat each example separately; a global mean combines both examples and all features.

Q5

Calculate stable $\operatorname{softmax}\left(\left[1000, 1000+\ln\left(3\right)\right]\right)$ . Compare its second probability with a computed value 0.7502 using atol = 0.0001 and rtol = 0.001. Finally calculate float16 payload bytes for shape $\left[2, 3, 4\right]$ .

Show Q5 answer

Subtract the maximum: $\left[-\ln\left(3\right), 0\right]$ . Exponentials $\left[\frac{1}{3}, 1\right]$ give $\left[0.25, 0.75\right]$ . The second-coordinate error is 0.0002; allowed error $0.0001+0.001\times 0.75 = 0.00085$ , so it passes. Storage is $2\times 3\times 4\times 2 = 48$ bytes.

Five-minute break23:15–23:20 · leave the screen

03

Check & correct

23:20–23:45 · 25 minutes

Reveal the answers above one at a time. Compare the reasoning, correct any errors, and record at most one unresolved issue. Then attempt the final check with answers closed. For a learning session, both steps share the last 15 minutes.

04

Final check

23:45–00:00 · 15 minutes

Without the key, trace the full calculation for x = $\left[1, 1\right]$ , W = $\begin{bmatrix}\ln\left(2\right) & 0 \\ 0 & \ln\left(3\right)\end{bmatrix}$ , b = $\left[0, 0\right]$ , and value rows V = $\begin{bmatrix}5 & 0 \\ 0 & 5\end{bmatrix}$ . Calculate z = x @ W + b, p = $\operatorname{softmax}\left(z\right)$ , and o = p @ V. Label the shape at each step and say how adding 500 to both scores changes o.

Show final answer

$z=[\ln2,\ln3]$ , shape [2]. Exponentials $[2,3]$ , so $p=[0.4,0.6]$ , shape [2]. The output is $o=0.4[5,0]+0.6[0,5]=[2,3]$ , shape [2]. Adding 500 to both scores leaves $p$ and $o$ unchanged; use maximum subtraction to evaluate safely.

WHEN YOU FINISH

Proceed to Transformer foundations if lesson 23's completed initial attempt scores at least 7/8, today's completed initial attempt scores at least 4/5 with exercises 2, 3, and 5 correct, and the exit is correct without the key. If unfinished, continue closed-book in the next scheduled slot before checking answers; assess accuracy only after all questions are attempted. If the thresholds are not met, follow the fixed retry instructions and alternate assessment below. Do not reuse lessons 23 or 24 as a fresh gate.

06

Alternate assessment

Extra sessions only

If the first-attempt completion rule is not met, use your next scheduled slot to redo the fixed consolidation lesson 22 in full. In the following scheduled slot, attempt the five alternate problems below using the review timetable: recall 10 minutes, independent work 35, break 5, feedback 25, and retry/explanation 15. Keep earlier solutions closed and use a calculator. If independent work takes longer, take the five-minute break and use the remaining session to continue with solutions closed. Stop at midnight and finish any unanswered questions in the next scheduled slot before viewing the key. Mark answers final, then score the original answers before correcting them. Speed is not part of the readiness criterion. Award one point per problem only when every requested part and explanation is correct. Proceed with at least 4/5, including correct answers to problems 2 (affine calculation and axes) and 3 (stable softmax and weighted sum). During the final explanation block, explain the first corrected problem aloud, or problem 3 if all were correct. If the alternate check still does not meet the rule, bring the first numbered failed problem and your attempted reasoning to this chat before advancing. Do not loop through the same assessments again as a fresh readiness gate.

Open the alternate assessment

A1

Let A = $\begin{bmatrix}2 & -1 \\ 1 & 3\end{bmatrix}$ and x = $\left[-1, 2\right]$ . Calculate A @ x with row dot products, then show the same result as a weighted sum of the columns of A.

Show A1 answer

Row products give $[2\times(-1)-1\times2,\;1\times(-1)+3\times2]=[-4,5]$ . Using columns: $(-1)[2,1]+2[-1,3]=[-2,-1]+[-2,6]=[-4,5]$ .

A2

X has shape $\left[3, 2, 4\right]$ , W = $\begin{bmatrix}1 & 0 & 2 \\ 0 & 1 & -1 \\ 1 & 1 & 0 \\ 0 & 2 & 1\end{bmatrix}$ , and b = $\left[1, -1, 2\right]$ . Give the shape of Y = X @ W + b. If X[1,0] = $\left[1, 2, 0, -1\right]$ , calculate Y[1,0]. Which axis should softmax use for a distribution over the output features, and what shape results when those probabilities are summed with keepdim=True?

Show A2 answer

Y has shape $\left[3, 2, 3\right]$ . The selected row product is $\left[1, 0, -1\right]$ ; adding b gives $\left[2, -1, 1\right]$ . Softmax uses the last axis, dim=-1 or 2, because its three entries are output features. Retained sums have shape $\left[3, 2, 1\right]$ .

A3

Calculate stable softmax for scores $\left[700+\ln\left(2\right), 700+\ln\left(3\right), 700+\ln\left(5\right)\right]$ , showing shifted scores and exponentials. Then use its probabilities to combine the rows of V = $\begin{bmatrix}1 & 4 \\ 3 & 0 \\ -1 & 2\end{bmatrix}$ .

Show A3 answer

Subtract $700+\ln5$ : shifted scores $[\ln(\tfrac25),\ln(\tfrac35),0]$ , exponentials $[0.4,0.6,1]$ , total 2. Probabilities are $[0.2,0.3,0.5]$ . Weighted output $=0.2[1,4]+0.3[3,0]+0.5[-1,2]=[0.6,1.8]$ .

A4

Normalize each row of X = $\begin{bmatrix}-3 & 1 \\ 4 & 10\end{bmatrix}$ using population variance and epsilon = 0. Give retained means, variances, and output. Also give the output for the constant row $\left[8, 8\right]$ with epsilon = 0.04.

Show A4 answer

Means $\begin{bmatrix}-1 \\ 7\end{bmatrix}$ , variances $\begin{bmatrix}4 \\ 9\end{bmatrix}$ , centered rows $\left[-2, 2\right]$ and $\left[-3, 3\right]$ , so output is $\begin{bmatrix}-1 & 1 \\ -1 & 1\end{bmatrix}$ . For $\left[8, 8\right]$ , mean 8 and variance 0 give denominator $\sqrt{0.04}$ =0.2 and output $\left[0, 0\right]$ .

A5

For reference 0.2 and result 0.2005, check the error using atol = 0.0002 and rtol = 0.002. With the same tolerances, does reference 0 and result 0.0003 pass? Calculate float32 payload bytes for a tensor of shape $\left[4, 5, 6\right]$ .

Show A5 answer

First error = 0.0005; allowance = $0.0002+0.002\times 0.2 = 0.0006$ , so pass. At reference zero the allowance is 0.0002; error 0.0003 fails. Storage = $4\times 5\times 6\times 4 = 480$ payload bytes.