Show HN: The Little Notebook for Learning Linear Algebra with Python

Chapter 1. Vectors, scalars, and geometry 1. Scalars, Vectors, and Coordinate Systems Let’s get our hands dirty! This lab is about playing with the building blocks of linear algebra: scalars and vectors. Think of a scalar as just a plain number, like 3 or -1.5 . A vector is a small list of numbers, which you can picture as an arrow in space. We’ll use Python (with NumPy) to explore them. Don’t worry if this is your first time with NumPy - we’ll go slowly. Set Up Your Lab import numpy as np numpynp That’s it - we’re ready! NumPy is the main tool we’ll use for linear algebra. Step-by-Step Code Walkthrough Scalars are just numbers. a = 5 # a scalar b = - 2.5 # another scalar print (a + b) # add them (ab) print (a * b) # multiply them (ab) 2.5 -12.5 Vectors are lists of numbers. v = np.array([ 2 , 3 ]) # a vector in 2D np.array([]) w = np.array([ 1 , - 1 , 4 ]) # a vector in 3D np.array([]) print (v) (v) print (w) (w) [2 3] [ 1 -1 4] Coordinates tell us where we are. Think of [2, 3] as “go 2 steps in the x-direction, 3 steps in the y-direction.” We can even draw it: import matplotlib.pyplot as plt matplotlib.pyplotplt # plot vector v 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' ) plt.quiver(, v[], v[], angles, scale_units, scale, color 0 , 4 ) plt.xlim( 0 , 4 ) plt.ylim( plt.grid() plt.show() This makes a little arrow from the origin (0,0) to (2,3) . Try It Yourself Change the vector v to [4, 1] . Where does the arrow point now? Try making a 3D vector with 4 numbers, like [1, 2, 3, 4] . What happens? Replace np.array([2,3]) with np.array([0,0]) . What does the arrow look like? 2. Vector Notation, Components, and Arrows In this lab, we’ll practice reading, writing, and visualizing vectors in different ways. A vector can look simple at first - just a list of numbers - but how we write it and how we interpret it really matters. This is where notation and components come into play. A vector has: A symbol (we might call it v , w , or even →AB in geometry). , , or even in geometry). Components (the individual numbers, like 2 and 3 in [2, 3] ). and in ). An arrow picture (a geometric way to see the vector as a directed line segment). Let’s see all three in action with Python. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Writing vectors in Python # Two-dimensional vector v = np.array([ 2 , 3 ]) np.array([]) # Three-dimensional vector w = np.array([ 1 , - 1 , 4 ]) np.array([]) print ( "v =" , v) , v) print ( "w =" , w) , w) v = [2 3] w = [ 1 -1 4] Here v has components (2, 3) and w has components (1, -1, 4) . Accessing components Each number in the vector is a component. We can pick them out using indexing. print ( "First component of v:" , v[ 0 ]) , v[]) print ( "Second component of v:" , v[ 1 ]) , v[]) First component of v: 2 Second component of v: 3 Notice: in Python, indices start at 0 , so v[0] is the first component. Visualizing vectors as arrows In 2D, it’s easy to draw a vector from the origin (0,0) to its endpoint (x,y) . 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' ) plt.quiver(, v[], v[], angles, scale_units, scale, color - 1 , 4 ) plt.xlim( - 2 , 4 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() This shows vector v as a red arrow from (0,0) to (2,3) . Drawing multiple vectors We can plot several arrows at once to compare them. u = np.array([ 3 , 1 ]) np.array([]) z = np.array([ - 1 , 2 ]) np.array([]) # Draw v, u, z in different colors 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' , label = 'v' ) plt.quiver(, v[], v[], angles, scale_units, scale, color, label 0 , 0 , u[ 0 ], u[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' , label = 'u' ) plt.quiver(, u[], u[], angles, scale_units, scale, color, label 0 , 0 , z[ 0 ], z[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'g' , label = 'z' ) plt.quiver(, z[], z[], angles, scale_units, scale, color, label - 2 , 4 ) plt.xlim( - 2 , 4 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() Now you’ll see three arrows starting at the same point, each pointing in a different direction. Try It Yourself Change v to [5, 0] . What does the arrow look like now? Try a vector like [0, -3] . Which axis does it line up with? Make a new vector q = np.array([2, 0, 0]) . What happens if you try to plot it with plt.quiver in 2D? 3. Vector Addition and Scalar Multiplication In this lab, we’ll explore the two most fundamental operations you can perform with vectors: adding them together and scaling them by a number (a scalar). These operations form the basis of everything else in linear algebra, from geometry to machine learning. Understanding how they work, both in code and visually, is key to building intuition. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Vector addition When you add two vectors, you simply add their components one by one. v = np.array([ 2 , 3 ]) np.array([]) u = np.array([ 1 , - 1 ]) np.array([]) = v + u sum_vector print ( "v + u =" , sum_vector) , sum_vector) v + u = [3 2] Here, (2,3) + (1,-1) = (3,2) . Visualizing vector addition (tip-to-tail method) Graphically, vector addition means placing the tail of one vector at the head of the other. The resulting vector goes from the start of the first to the end of the second. 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' , label = 'v' ) plt.quiver(, v[], v[], angles, scale_units, scale, color, label 0 ], v[ 1 ], u[ 0 ], u[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' , label = 'u placed at end of v' ) plt.quiver(v[], v[], u[], u[], angles, scale_units, scale, color, label 0 , 0 , sum_vector[ 0 ], sum_vector[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'g' , label = 'v + u' ) plt.quiver(, sum_vector[], sum_vector[], angles, scale_units, scale, color, label - 1 , 5 ) plt.xlim( - 2 , 5 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() The green arrow is the result of adding v and u . Scalar multiplication Multiplying a vector by a scalar stretches or shrinks it. If the scalar is negative, the vector flips direction. c = 2 = c * v scaled_v print ( "2 * v =" , scaled_v) , scaled_v) d = - 1 = d * v scaled_v_neg print ( "-1 * v =" , scaled_v_neg) , scaled_v_neg) 2 * v = [4 6] -1 * v = [-2 -3] So 2 * (2,3) = (4,6) and -1 * (2,3) = (-2,-3) . Visualizing scalar multiplication 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' , label = 'v' ) plt.quiver(, v[], v[], angles, scale_units, scale, color, label 0 , 0 , scaled_v[ 0 ], scaled_v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' , label = '2 * v' ) plt.quiver(, scaled_v[], scaled_v[], angles, scale_units, scale, color, label 0 , 0 , scaled_v_neg[ 0 ], scaled_v_neg[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'g' , label = '-1 * v' ) plt.quiver(, scaled_v_neg[], scaled_v_neg[], angles, scale_units, scale, color, label - 5 , 5 ) plt.xlim( - 5 , 7 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() Here, the blue arrow is twice as long as the red arrow, while the green arrow points in the opposite direction. Combining both operations We can scale vectors and then add them. This is called a linear combination (and it’s the foundation for the next section). = 3 * v + ( - 2 ) * u combo print ( "3*v - 2*u =" , combo) , combo) 3*v - 2*u = [ 4 11] Try It Yourself Replace c = 2 with c = 0.5 . What happens to the vector? Try adding three vectors: v + u + np.array([-1,2]) . Can you predict the result before printing? Visualize 3*v + 2*u using arrows. How does it compare to just v + u ? 4. Linear Combinations and Span Now that we know how to add vectors and scale them, we can combine these two moves to create linear combinations. A linear combination is just a recipe: multiply vectors by scalars, then add them together. The set of all possible results you can get from such recipes is called the span. This idea is powerful because span tells us what directions and regions of space we can reach using given vectors. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Linear combinations in Python v = np.array([ 2 , 1 ]) np.array([]) u = np.array([ 1 , 3 ]) np.array([]) = 2 * v + 3 * u combo1 = - 1 * v + 4 * u combo2 print ( "2*v + 3*u =" , combo1) , combo1) print ( "-v + 4*u =" , combo2) , combo2) 2*v + 3*u = [ 7 11] -v + 4*u = [ 2 11] Here, we multiplied and added vectors using scalars. Each result is a new vector. Visualizing linear combinations Let’s plot v , u , and their combinations. 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' , label = 'v' ) plt.quiver(, v[], v[], angles, scale_units, scale, color, label 0 , 0 , u[ 0 ], u[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' , label = 'u' ) plt.quiver(, u[], u[], angles, scale_units, scale, color, label 0 , 0 , combo1[ 0 ], combo1[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'g' , label = '2v + 3u' ) plt.quiver(, combo1[], combo1[], angles, scale_units, scale, color, label 0 , 0 , combo2[ 0 ], combo2[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'm' , label = '-v + 4u' ) plt.quiver(, combo2[], combo2[], angles, scale_units, scale, color, label - 5 , 10 ) plt.xlim( - 5 , 10 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() This shows how new arrows can be generated from scaling and adding the original ones. Exploring the span The span of two 2D vectors is either: A line (if one is a multiple of the other). The whole 2D plane (if they are independent). # Generate many combinations = range ( - 5 , 6 ) coeffs = [] points[] for a in coeffs: coeffs: for b in coeffs: coeffs: = a * v + b * u point points.append(point) = np.array(points) pointsnp.array(points) 0 ], points[:, 1 ], s = 10 , color = 'gray' ) plt.scatter(points[:,], points[:,], s, color 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' ) plt.quiver(, v[], v[], angles, scale_units, scale, color 0 , 0 , u[ 0 ], u[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' ) plt.quiver(, u[], u[], angles, scale_units, scale, color - 10 , 10 ) plt.xlim( - 10 , 10 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() The gray dots show all reachable points with combinations of v and u . Special case: dependent vectors w = np.array([ 4 , 2 ]) # notice w = 2*v np.array([]) = range ( - 5 , 6 ) coeffs = [] points[] for a in coeffs: coeffs: for b in coeffs: coeffs: * v + b * w) points.append(aw) = np.array(points) pointsnp.array(points) 0 ], points[:, 1 ], s = 10 , color = 'gray' ) plt.scatter(points[:,], points[:,], s, color 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' ) plt.quiver(, v[], v[], angles, scale_units, scale, color 0 , 0 , w[ 0 ], w[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' ) plt.quiver(, w[], w[], angles, scale_units, scale, color - 10 , 10 ) plt.xlim( - 10 , 10 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() Here, the span collapses to a line because w is just a scaled copy of v . Try It Yourself Replace u = [1,3] with u = [-1,2] . What does the span look like? Try three vectors in 2D (e.g., v, u, w ). Do you get more than the whole plane? Experiment with 3D vectors. Use np.array([x,y,z]) and check whether different vectors span a plane or all of space. 5. Length (Norm) and Distance In this lab, we’ll measure how big a vector is (its length, also called its norm) and how far apart two vectors are (their distance). These ideas connect algebra to geometry: when we compute a norm, we’re measuring the size of an arrow; when we compute a distance, we’re measuring the gap between two points in space. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Vector length (norm) in 2D The length of a vector is computed using the Pythagorean theorem. For a vector (x, y) , the length is sqrt(x² + y²) . v = np.array([ 3 , 4 ]) np.array([]) = np.linalg.norm(v) lengthnp.linalg.norm(v) print ( "Length of v =" , length) , length) Length of v = 5.0 This prints 5.0 , because (3,4) forms a right triangle with sides 3 and 4, and sqrt(3²+4²)=5 . Manual calculation vs NumPy = (v[ 0 ] ** 2 + v[ 1 ] ** 2 ) ** 0.5 manual_length(v[v[ print ( "Manual length =" , manual_length) , manual_length) print ( "NumPy length =" , np.linalg.norm(v)) , np.linalg.norm(v)) Manual length = 5.0 NumPy length = 5.0 Both give the same result. Visualizing vector length 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' ) plt.quiver(, v[], v[], angles, scale_units, scale, color 0 , 5 ) plt.xlim( 0 , 5 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth 0 ] / 2 , v[ 1 ] / 2 , f"Length= { length } " , fontsize = 10 , color = 'blue' ) plt.text(v[, v[length, fontsize, color plt.grid() plt.show() You’ll see the arrow (3,4) with its length labeled. Distance between two vectors The distance between v and another vector u is the length of their difference: ‖v - u‖ . u = np.array([ 0 , 0 ]) # the origin np.array([]) = np.linalg.norm(v - u) distnp.linalg.norm(vu) print ( "Distance between v and u =" , dist) , dist) Distance between v and u = 5.0 Since u is the origin, this is just the length of v . A more interesting distance u = np.array([ 1 , 1 ]) np.array([]) = np.linalg.norm(v - u) distnp.linalg.norm(vu) print ( "Distance between v and u =" , dist) , dist) Distance between v and u = 3.605551275463989 This measures how far (3,4) is from (1,1) . Visualizing distance between points 0 ], u[ 0 ]], [v[ 1 ], u[ 1 ]], color = [ 'red' , 'blue' ]) plt.scatter([v[], u[]], [v[], u[]], color]) 0 ], u[ 0 ]], [v[ 1 ], u[ 1 ]], 'k--' ) plt.plot([v[], u[]], [v[], u[]], 0 ], v[ 1 ], 'v' , fontsize = 12 , color = 'red' ) plt.text(v[], v[],, fontsize, color 0 ], u[ 1 ], 'u' , fontsize = 12 , color = 'blue' ) plt.text(u[], u[],, fontsize, color plt.grid() plt.show() The dashed line shows the distance between the two points. Higher-dimensional vectors Norms and distances work the same way in any dimension: a = np.array([ 1 , 2 , 3 ]) np.array([]) b = np.array([ 4 , 0 , 8 ]) np.array([]) print ( "‖a‖ =" , np.linalg.norm(a)) , np.linalg.norm(a)) print ( "‖b‖ =" , np.linalg.norm(b)) , np.linalg.norm(b)) print ( "Distance between a and b =" , np.linalg.norm(a - b)) , np.linalg.norm(ab)) ‖a‖ = 3.7416573867739413 ‖b‖ = 8.94427190999916 Distance between a and b = 6.164414002968976 Even though we can’t draw 3D easily on paper, the formulas still apply. Try It Yourself Compute the length of np.array([5,12]) . What do you expect? Find the distance between (2,3) and (7,7) . Can you sketch it by hand and check? In 3D, try vectors (1,1,1) and (2,2,2) . Why is the distance exactly sqrt(3) ? 6. Dot Product The dot product is one of the most important operations in linear algebra. It takes two vectors and gives you a single number. That number combines both the lengths of the vectors and how much they point in the same direction. In this lab, we’ll calculate dot products in several ways, see how they relate to geometry, and visualize their meaning. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Algebraic definition The dot product of two vectors is the sum of the products of their components: v = np.array([ 2 , 3 ]) np.array([]) u = np.array([ 4 , - 1 ]) np.array([]) = v[ 0 ] * u[ 0 ] + v[ 1 ] * u[ 1 ] dot_manualv[u[v[u[ = np.dot(v, u) dot_numpynp.dot(v, u) print ( "Manual dot product:" , dot_manual) , dot_manual) print ( "NumPy dot product:" , dot_numpy) , dot_numpy) Manual dot product: 5 NumPy dot product: 5 Here, (2*4) + (3*-1) = 8 - 3 = 5 . Geometric definition The dot product also equals the product of the lengths of the vectors times the cosine of the angle between them: \[ v \cdot u = \|v\| \|u\| \cos \theta \] We can compute the angle: = np.linalg.norm(v) norm_vnp.linalg.norm(v) = np.linalg.norm(u) norm_unp.linalg.norm(u) = np.dot(v, u) / (norm_v * norm_u) cos_thetanp.dot(v, u)(norm_vnorm_u) = np.arccos(cos_theta) thetanp.arccos(cos_theta) print ( "cos(theta) =" , cos_theta) , cos_theta) print ( "theta (in radians) =" , theta) , theta) print ( "theta (in degrees) =" , np.degrees(theta)) , np.degrees(theta)) cos(theta) = 0.33633639699815626 theta (in radians) = 1.2277723863741932 theta (in degrees) = 70.3461759419467 This gives the angle between v and u . Visualizing the dot product Let’s draw the two vectors: 0 , 0 ,v[ 0 ],v[ 1 ],angles = 'xy' ,scale_units = 'xy' ,scale = 1 ,color = 'r' ,label = 'v' ) plt.quiver(,v[],v[],angles,scale_units,scale,color,label 0 , 0 ,u[ 0 ],u[ 1 ],angles = 'xy' ,scale_units = 'xy' ,scale = 1 ,color = 'b' ,label = 'u' ) plt.quiver(,u[],u[],angles,scale_units,scale,color,label - 1 , 5 ) plt.xlim( - 2 , 4 ) plt.ylim( 0 ,color = 'black' ,linewidth = 0.5 ) plt.axhline(,color,linewidth 0 ,color = 'black' ,linewidth = 0.5 ) plt.axvline(,color,linewidth plt.grid() plt.show() The dot product is positive if the angle is less than 90°, negative if greater than 90°, and zero if the vectors are perpendicular. Projections and dot product The dot product lets us compute how much of one vector lies in the direction of another. = np.dot(v, u) / np.linalg.norm(u) proj_lengthnp.dot(v, u)np.linalg.norm(u) print ( "Projection length of v onto u:" , proj_length) , proj_length) Projection length of v onto u: 1.212678125181665 This is the length of the shadow of v onto u . Special cases If vectors point in the same direction, the dot product is large and positive. If vectors are perpendicular, the dot product is zero. If vectors point in opposite directions, the dot product is negative. a = np.array([ 1 , 0 ]) np.array([]) b = np.array([ 0 , 1 ]) np.array([]) c = np.array([ - 1 , 0 ]) np.array([]) print ( "a · b =" , np.dot(a,b)) # perpendicular , np.dot(a,b)) print ( "a · a =" , np.dot(a,a)) # length squared , np.dot(a,a)) print ( "a · c =" , np.dot(a,c)) # opposite , np.dot(a,c)) a · b = 0 a · a = 1 a · c = -1 Try It Yourself Compute the dot product of (3,4) with (4,3) . Is the result larger or smaller than the product of their lengths? Try (1,2,3) · (4,5,6) . Does the geometric meaning still work in 3D? Create two perpendicular vectors (e.g. (2,0) and (0,5) ). Verify the dot product is zero. 7. Angles Between Vectors and Cosine In this lab, we’ll go deeper into the connection between vectors and geometry by calculating angles. Angles tell us how much two vectors “point in the same direction.” The bridge between algebra and geometry here is the cosine formula, which comes directly from the dot product. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Formula for the angle The angle \(\theta\) between two vectors \(v\) and \(u\) is given by: \[ \cos \theta = \frac{v \cdot u}{\|v\| \, \|u\|} \] This means: If \(\cos \theta = 1\) , the vectors point in exactly the same direction. , the vectors point in exactly the same direction. If \(\cos \theta = 0\) , they are perpendicular. , they are perpendicular. If \(\cos \theta = -1\) , they point in opposite directions. Computing the angle in Python v = np.array([ 2 , 3 ]) np.array([]) u = np.array([ 3 , - 1 ]) np.array([]) = np.dot(v, u) dotnp.dot(v, u) = np.linalg.norm(v) norm_vnp.linalg.norm(v) = np.linalg.norm(u) norm_unp.linalg.norm(u) = dot / (norm_v * norm_u) cos_thetadot(norm_vnorm_u) = np.arccos(cos_theta) thetanp.arccos(cos_theta) print ( "cos(theta) =" , cos_theta) , cos_theta) print ( "theta in radians =" , theta) , theta) print ( "theta in degrees =" , np.degrees(theta)) , np.degrees(theta)) cos(theta) = 0.2631174057921088 theta in radians = 1.3045442776439713 theta in degrees = 74.74488129694222 This gives both the cosine value and the actual angle. Visualizing the vectors 0 , 0 ,v[ 0 ],v[ 1 ],angles = 'xy' ,scale_units = 'xy' ,scale = 1 ,color = 'r' ,label = 'v' ) plt.quiver(,v[],v[],angles,scale_units,scale,color,label 0 , 0 ,u[ 0 ],u[ 1 ],angles = 'xy' ,scale_units = 'xy' ,scale = 1 ,color = 'b' ,label = 'u' ) plt.quiver(,u[],u[],angles,scale_units,scale,color,label - 1 , 4 ) plt.xlim( - 2 , 4 ) plt.ylim( 0 ,color = 'black' ,linewidth = 0.5 ) plt.axhline(,color,linewidth 0 ,color = 'black' ,linewidth = 0.5 ) plt.axvline(,color,linewidth plt.grid() plt.show() You can see the angle between v and u as the gap between the red and blue arrows. Checking special cases a = np.array([ 1 , 0 ]) np.array([]) b = np.array([ 0 , 1 ]) np.array([]) c = np.array([ - 1 , 0 ]) np.array([]) print ( "Angle between a and b =" , np.degrees(np.arccos(np.dot(a,b) / (np.linalg.norm(a) * np.linalg.norm(b))))) , np.degrees(np.arccos(np.dot(a,b)(np.linalg.norm(a)np.linalg.norm(b))))) print ( "Angle between a and c =" , np.degrees(np.arccos(np.dot(a,c) / (np.linalg.norm(a) * np.linalg.norm(c))))) , np.degrees(np.arccos(np.dot(a,c)(np.linalg.norm(a)np.linalg.norm(c))))) Angle between a and b = 90.0 Angle between a and c = 180.0 Angle between (1,0) and (0,1) is 90°. and is 90°. Angle between (1,0) and (-1,0) is 180°. Using cosine as a similarity measure In data science and machine learning, people often use cosine similarity instead of raw angles. It’s just the cosine value itself: = np.dot(v,u) / (np.linalg.norm(v) * np.linalg.norm(u)) cosine_similaritynp.dot(v,u)(np.linalg.norm(v)np.linalg.norm(u)) print ( "Cosine similarity =" , cosine_similarity) , cosine_similarity) Cosine similarity = 0.2631174057921088 Values close to 1 mean vectors are aligned, values near 0 mean unrelated, and values near -1 mean opposite. Try It Yourself Create two random vectors with np.random.randn(3) and compute the angle between them. Verify that swapping the vectors gives the same angle (symmetry). Find two vectors where cosine similarity is exactly 0 . Can you come up with an example in 2D? 8. Projections and Decompositions In this lab, we’ll learn how to split one vector into parts: one part that lies along another vector, and one part that is perpendicular. This process is called projection and decomposition. Projections let us measure “how much of a vector points in a given direction,” and decompositions give us a way to break vectors into useful components. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Projection formula The projection of vector \(v\) onto vector \(u\) is: \[ \text{proj}_u(v) = \frac{v \cdot u}{u \cdot u} \, u \] This gives the component of \(v\) that points in the direction of \(u\). Computing projection in Python v = np.array([ 3 , 2 ]) np.array([]) u = np.array([ 2 , 0 ]) np.array([]) = (np.dot(v, u) / np.dot(u, u)) * u proj_u_v(np.dot(v, u)np.dot(u, u)) print ( "Projection of v onto u:" , proj_u_v) , proj_u_v) Projection of v onto u: [3. 0.] Here, \(v = (3,2)\) and \(u = (2,0)\). The projection of v onto u is a vector pointing along the x-axis. Decomposing into parallel and perpendicular parts We can write: \[ v = \text{proj}_u(v) + (v - \text{proj}_u(v)) \] The first part is parallel to u , the second part is perpendicular. = v - proj_u_v perpproj_u_v print ( "Parallel part:" , proj_u_v) , proj_u_v) print ( "Perpendicular part:" , perp) , perp) Parallel part: [3. 0.] Perpendicular part: [0. 2.] Visualizing projection and decomposition 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' , label = 'v' ) plt.quiver(, v[], v[], angles, scale_units, scale, color, label 0 , 0 , u[ 0 ], u[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' , label = 'u' ) plt.quiver(, u[], u[], angles, scale_units, scale, color, label 0 , 0 , proj_u_v[ 0 ], proj_u_v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'g' , label = 'proj_u(v)' ) plt.quiver(, proj_u_v[], proj_u_v[], angles, scale_units, scale, color, label 0 ], proj_u_v[ 1 ], perp[ 0 ], perp[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'm' , label = 'perpendicular' ) plt.quiver(proj_u_v[], proj_u_v[], perp[], perp[], angles, scale_units, scale, color, label - 1 , 5 ) plt.xlim( - 1 , 4 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() You’ll see v (red), u (blue), the projection (green), and the perpendicular remainder (magenta). Projection in higher dimensions This formula works in any dimension: a = np.array([ 1 , 2 , 3 ]) np.array([]) b = np.array([ 0 , 1 , 0 ]) np.array([]) = (np.dot(a,b) / np.dot(b,b)) * b proj(np.dot(a,b)np.dot(b,b)) = a - proj perpproj print ( "Projection of a onto b:" , proj) , proj) print ( "Perpendicular component:" , perp) , perp) Projection of a onto b: [0. 2. 0.] Perpendicular component: [1. 0. 3.] Even in 3D or higher, projections are about splitting into “along” and “across.” Try It Yourself Try projecting (2,3) onto (0,5) . Where does it land? Take a 3D vector like (4,2,6) and project it onto (1,0,0) . What does this give you? Change the base vector u to something not aligned with the axes, like (1,1) . Does the projection still work? 9. Cauchy–Schwarz and Triangle Inequalities This lab introduces two fundamental inequalities in linear algebra. They may look abstract at first, but they provide guarantees that always hold true for vectors. We’ll explore them with small examples in Python to see why they matter. Set Up Your Lab import numpy as np numpynp Step-by-Step Code Walkthrough Cauchy–Schwarz inequality The inequality states: \[ |v \cdot u| \leq \|v\| \, \|u\| \] It means the dot product is never “bigger” than the product of the vector lengths. Equality happens only if the two vectors are pointing in exactly the same (or opposite) direction. v = np.array([ 3 , 4 ]) np.array([]) u = np.array([ 1 , 2 ]) np.array([]) = abs (np.dot(v, u)) lhs(np.dot(v, u)) = np.linalg.norm(v) * np.linalg.norm(u) rhsnp.linalg.norm(v)np.linalg.norm(u) print ( "Left-hand side (|v·u|):" , lhs) , lhs) print ( "Right-hand side (‖v‖‖u‖):" , rhs) , rhs) print ( "Inequality holds?" , lhs <= rhs) , lhsrhs) Left-hand side (|v·u|): 11 Right-hand side (‖v‖‖u‖): 11.180339887498949 Inequality holds? True Testing Cauchy–Schwarz with different vectors = [ pairs 1 , 0 ]), np.array([ 0 , 1 ])), # perpendicular (np.array([]), np.array([])), 2 , 3 ]), np.array([ 4 , 6 ])), # multiples (np.array([]), np.array([])), - 1 , 2 ]), np.array([ 3 , - 6 ])) # opposite multiples (np.array([]), np.array([])) ] for v,u in pairs: v,upairs: = abs (np.dot(v, u)) lhs(np.dot(v, u)) = np.linalg.norm(v) * np.linalg.norm(u) rhsnp.linalg.norm(v)np.linalg.norm(u) print ( f"v= { v } , u= { u } -> |v·u|= { lhs } , ‖v‖‖u‖= { rhs } , holds= { lhs <= rhs } " ) lhsrhslhsrhs v=[1 0], u=[0 1] -> |v·u|=0, ‖v‖‖u‖=1.0, holds=True v=[2 3], u=[4 6] -> |v·u|=26, ‖v‖‖u‖=25.999999999999996, holds=False v=[-1 2], u=[ 3 -6] -> |v·u|=15, ‖v‖‖u‖=15.000000000000002, holds=True Perpendicular vectors give |v·u| = 0 , far less than the product of norms. , far less than the product of norms. Multiples give equality ( lhs = rhs ). Triangle inequality The triangle inequality states: \[ \|v + u\| \leq \|v\| + \|u\| \] Geometrically, the length of one side of a triangle can never be longer than the sum of the other two sides. v = np.array([ 3 , 4 ]) np.array([]) u = np.array([ 1 , 2 ]) np.array([]) = np.linalg.norm(v + u) lhsnp.linalg.norm(vu) = np.linalg.norm(v) + np.linalg.norm(u) rhsnp.linalg.norm(v)np.linalg.norm(u) print ( "‖v+u‖ =" , lhs) , lhs) print ( "‖v‖ + ‖u‖ =" , rhs) , rhs) print ( "Inequality holds?" , lhs <= rhs) , lhsrhs) ‖v+u‖ = 7.211102550927978 ‖v‖ + ‖u‖ = 7.23606797749979 Inequality holds? True Visual demonstration with a triangle import matplotlib.pyplot as plt matplotlib.pyplotplt = np.array([ 0 , 0 ]) originnp.array([]) = np.array([origin, v, v + u, origin]) pointsnp.array([origin, v, vu, origin]) 0 ], points[:, 1 ], 'ro-' ) # triangle outline plt.plot(points[:,], points[:,], 0 ], v[ 1 ], 'v' ) plt.text(v[], v[], 0 ] + u[ 0 ], v[ 1 ] + u[ 1 ], 'v+u' ) plt.text(v[u[], v[u[], 0 ], u[ 1 ], 'u' ) plt.text(u[], u[], plt.grid() 0 ,color = 'black' ,linewidth = 0.5 ) plt.axhline(,color,linewidth 0 ,color = 'black' ,linewidth = 0.5 ) plt.axvline(,color,linewidth 'equal' ) plt.axis( plt.show() This triangle shows why the inequality is called the “triangle” inequality. Testing triangle inequality with random vectors for _ in range ( 5 ): ): v = np.random.randn( 2 ) np.random.randn( u = np.random.randn( 2 ) np.random.randn( = np.linalg.norm(v + u) lhsnp.linalg.norm(vu) = np.linalg.norm(v) + np.linalg.norm(u) rhsnp.linalg.norm(v)np.linalg.norm(u) print ( f"‖v+u‖= { lhs :.3f} , ‖v‖+‖u‖= { rhs :.3f} , holds= { lhs <= rhs } " ) lhsrhslhsrhs ‖v+u‖=0.778, ‖v‖+‖u‖=2.112, holds=True ‖v+u‖=1.040, ‖v‖+‖u‖=2.621, holds=True ‖v+u‖=1.632, ‖v‖+‖u‖=2.482, holds=True ‖v+u‖=1.493, ‖v‖+‖u‖=2.250, holds=True ‖v+u‖=2.653, ‖v‖+‖u‖=2.692, holds=True No matter what vectors you try, the inequality always holds. The Takeaway Cauchy–Schwarz: The dot product is always bounded by the product of vector lengths. Triangle inequality: The length of one side of a triangle can’t exceed the sum of the other two. These inequalities form the backbone of geometry, analysis, and many proofs in linear algebra.