Chapter 1. Vectors, scalars, and geometry
1. Scalars, Vectors, and Coordinate Systems Let’s get our hands dirty! This lab is about playing with the building blocks of linear algebra: scalars and vectors. Think of a scalar as just a plain number, like 3 or -1.5 . A vector is a small list of numbers, which you can picture as an arrow in space. We’ll use Python (with NumPy) to explore them. Don’t worry if this is your first time with NumPy - we’ll go slowly. Set Up Your Lab import numpy as np numpynp That’s it - we’re ready! NumPy is the main tool we’ll use for linear algebra. Step-by-Step Code Walkthrough Scalars are just numbers. a = 5 # a scalar b = - 2.5 # another scalar print (a + b) # add them (ab) print (a * b) # multiply them (ab) 2.5 -12.5 Vectors are lists of numbers. v = np.array([ 2 , 3 ]) # a vector in 2D np.array([]) w = np.array([ 1 , - 1 , 4 ]) # a vector in 3D np.array([]) print (v) (v) print (w) (w) [2 3] [ 1 -1 4] Coordinates tell us where we are. Think of [2, 3] as “go 2 steps in the x-direction, 3 steps in the y-direction.” We can even draw it: import matplotlib.pyplot as plt matplotlib.pyplotplt # plot vector v 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' ) plt.quiver(, v[], v[], angles, scale_units, scale, color 0 , 4 ) plt.xlim( 0 , 4 ) plt.ylim( plt.grid() plt.show() This makes a little arrow from the origin (0,0) to (2,3) . Try It Yourself Change the vector v to [4, 1] . Where does the arrow point now? Try making a 3D vector with 4 numbers, like [1, 2, 3, 4] . What happens? Replace np.array([2,3]) with np.array([0,0]) . What does the arrow look like?
2. Vector Notation, Components, and Arrows In this lab, we’ll practice reading, writing, and visualizing vectors in different ways. A vector can look simple at first - just a list of numbers - but how we write it and how we interpret it really matters. This is where notation and components come into play. A vector has: A symbol (we might call it v , w , or even →AB in geometry).
, , or even in geometry). Components (the individual numbers, like 2 and 3 in [2, 3] ).
and in ). An arrow picture (a geometric way to see the vector as a directed line segment). Let’s see all three in action with Python. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Writing vectors in Python # Two-dimensional vector v = np.array([ 2 , 3 ]) np.array([]) # Three-dimensional vector w = np.array([ 1 , - 1 , 4 ]) np.array([]) print ( "v =" , v) , v) print ( "w =" , w) , w) v = [2 3] w = [ 1 -1 4] Here v has components (2, 3) and w has components (1, -1, 4) . Accessing components Each number in the vector is a component. We can pick them out using indexing. print ( "First component of v:" , v[ 0 ]) , v[]) print ( "Second component of v:" , v[ 1 ]) , v[]) First component of v: 2 Second component of v: 3 Notice: in Python, indices start at 0 , so v[0] is the first component. Visualizing vectors as arrows In 2D, it’s easy to draw a vector from the origin (0,0) to its endpoint (x,y) . 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' ) plt.quiver(, v[], v[], angles, scale_units, scale, color - 1 , 4 ) plt.xlim( - 2 , 4 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() This shows vector v as a red arrow from (0,0) to (2,3) . Drawing multiple vectors We can plot several arrows at once to compare them. u = np.array([ 3 , 1 ]) np.array([]) z = np.array([ - 1 , 2 ]) np.array([]) # Draw v, u, z in different colors 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' , label = 'v' ) plt.quiver(, v[], v[], angles, scale_units, scale, color, label 0 , 0 , u[ 0 ], u[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' , label = 'u' ) plt.quiver(, u[], u[], angles, scale_units, scale, color, label 0 , 0 , z[ 0 ], z[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'g' , label = 'z' ) plt.quiver(, z[], z[], angles, scale_units, scale, color, label - 2 , 4 ) plt.xlim( - 2 , 4 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() Now you’ll see three arrows starting at the same point, each pointing in a different direction. Try It Yourself Change v to [5, 0] . What does the arrow look like now? Try a vector like [0, -3] . Which axis does it line up with? Make a new vector q = np.array([2, 0, 0]) . What happens if you try to plot it with plt.quiver in 2D?
3. Vector Addition and Scalar Multiplication In this lab, we’ll explore the two most fundamental operations you can perform with vectors: adding them together and scaling them by a number (a scalar). These operations form the basis of everything else in linear algebra, from geometry to machine learning. Understanding how they work, both in code and visually, is key to building intuition. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Vector addition When you add two vectors, you simply add their components one by one. v = np.array([ 2 , 3 ]) np.array([]) u = np.array([ 1 , - 1 ]) np.array([]) = v + u sum_vector print ( "v + u =" , sum_vector) , sum_vector) v + u = [3 2] Here, (2,3) + (1,-1) = (3,2) . Visualizing vector addition (tip-to-tail method) Graphically, vector addition means placing the tail of one vector at the head of the other. The resulting vector goes from the start of the first to the end of the second. 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' , label = 'v' ) plt.quiver(, v[], v[], angles, scale_units, scale, color, label 0 ], v[ 1 ], u[ 0 ], u[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' , label = 'u placed at end of v' ) plt.quiver(v[], v[], u[], u[], angles, scale_units, scale, color, label 0 , 0 , sum_vector[ 0 ], sum_vector[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'g' , label = 'v + u' ) plt.quiver(, sum_vector[], sum_vector[], angles, scale_units, scale, color, label - 1 , 5 ) plt.xlim( - 2 , 5 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() The green arrow is the result of adding v and u . Scalar multiplication Multiplying a vector by a scalar stretches or shrinks it. If the scalar is negative, the vector flips direction. c = 2 = c * v scaled_v print ( "2 * v =" , scaled_v) , scaled_v) d = - 1 = d * v scaled_v_neg print ( "-1 * v =" , scaled_v_neg) , scaled_v_neg) 2 * v = [4 6] -1 * v = [-2 -3] So 2 * (2,3) = (4,6) and -1 * (2,3) = (-2,-3) . Visualizing scalar multiplication 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' , label = 'v' ) plt.quiver(, v[], v[], angles, scale_units, scale, color, label 0 , 0 , scaled_v[ 0 ], scaled_v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' , label = '2 * v' ) plt.quiver(, scaled_v[], scaled_v[], angles, scale_units, scale, color, label 0 , 0 , scaled_v_neg[ 0 ], scaled_v_neg[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'g' , label = '-1 * v' ) plt.quiver(, scaled_v_neg[], scaled_v_neg[], angles, scale_units, scale, color, label - 5 , 5 ) plt.xlim( - 5 , 7 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() Here, the blue arrow is twice as long as the red arrow, while the green arrow points in the opposite direction. Combining both operations We can scale vectors and then add them. This is called a linear combination (and it’s the foundation for the next section). = 3 * v + ( - 2 ) * u combo print ( "3*v - 2*u =" , combo) , combo) 3*v - 2*u = [ 4 11] Try It Yourself Replace c = 2 with c = 0.5 . What happens to the vector? Try adding three vectors: v + u + np.array([-1,2]) . Can you predict the result before printing? Visualize 3*v + 2*u using arrows. How does it compare to just v + u ?
4. Linear Combinations and Span Now that we know how to add vectors and scale them, we can combine these two moves to create linear combinations. A linear combination is just a recipe: multiply vectors by scalars, then add them together. The set of all possible results you can get from such recipes is called the span. This idea is powerful because span tells us what directions and regions of space we can reach using given vectors. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Linear combinations in Python v = np.array([ 2 , 1 ]) np.array([]) u = np.array([ 1 , 3 ]) np.array([]) = 2 * v + 3 * u combo1 = - 1 * v + 4 * u combo2 print ( "2*v + 3*u =" , combo1) , combo1) print ( "-v + 4*u =" , combo2) , combo2) 2*v + 3*u = [ 7 11] -v + 4*u = [ 2 11] Here, we multiplied and added vectors using scalars. Each result is a new vector. Visualizing linear combinations Let’s plot v , u , and their combinations. 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' , label = 'v' ) plt.quiver(, v[], v[], angles, scale_units, scale, color, label 0 , 0 , u[ 0 ], u[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' , label = 'u' ) plt.quiver(, u[], u[], angles, scale_units, scale, color, label 0 , 0 , combo1[ 0 ], combo1[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'g' , label = '2v + 3u' ) plt.quiver(, combo1[], combo1[], angles, scale_units, scale, color, label 0 , 0 , combo2[ 0 ], combo2[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'm' , label = '-v + 4u' ) plt.quiver(, combo2[], combo2[], angles, scale_units, scale, color, label - 5 , 10 ) plt.xlim( - 5 , 10 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() This shows how new arrows can be generated from scaling and adding the original ones. Exploring the span The span of two 2D vectors is either: A line (if one is a multiple of the other).
The whole 2D plane (if they are independent). # Generate many combinations = range ( - 5 , 6 ) coeffs = [] points[] for a in coeffs: coeffs: for b in coeffs: coeffs: = a * v + b * u point points.append(point) = np.array(points) pointsnp.array(points) 0 ], points[:, 1 ], s = 10 , color = 'gray' ) plt.scatter(points[:,], points[:,], s, color 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' ) plt.quiver(, v[], v[], angles, scale_units, scale, color 0 , 0 , u[ 0 ], u[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' ) plt.quiver(, u[], u[], angles, scale_units, scale, color - 10 , 10 ) plt.xlim( - 10 , 10 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() The gray dots show all reachable points with combinations of v and u . Special case: dependent vectors w = np.array([ 4 , 2 ]) # notice w = 2*v np.array([]) = range ( - 5 , 6 ) coeffs = [] points[] for a in coeffs: coeffs: for b in coeffs: coeffs: * v + b * w) points.append(aw) = np.array(points) pointsnp.array(points) 0 ], points[:, 1 ], s = 10 , color = 'gray' ) plt.scatter(points[:,], points[:,], s, color 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' ) plt.quiver(, v[], v[], angles, scale_units, scale, color 0 , 0 , w[ 0 ], w[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'b' ) plt.quiver(, w[], w[], angles, scale_units, scale, color - 10 , 10 ) plt.xlim( - 10 , 10 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth plt.grid() plt.show() Here, the span collapses to a line because w is just a scaled copy of v . Try It Yourself Replace u = [1,3] with u = [-1,2] . What does the span look like? Try three vectors in 2D (e.g., v, u, w ). Do you get more than the whole plane? Experiment with 3D vectors. Use np.array([x,y,z]) and check whether different vectors span a plane or all of space.
5. Length (Norm) and Distance In this lab, we’ll measure how big a vector is (its length, also called its norm) and how far apart two vectors are (their distance). These ideas connect algebra to geometry: when we compute a norm, we’re measuring the size of an arrow; when we compute a distance, we’re measuring the gap between two points in space. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Vector length (norm) in 2D The length of a vector is computed using the Pythagorean theorem. For a vector (x, y) , the length is sqrt(x² + y²) . v = np.array([ 3 , 4 ]) np.array([]) = np.linalg.norm(v) lengthnp.linalg.norm(v) print ( "Length of v =" , length) , length) Length of v = 5.0 This prints 5.0 , because (3,4) forms a right triangle with sides 3 and 4, and sqrt(3²+4²)=5 . Manual calculation vs NumPy = (v[ 0 ] ** 2 + v[ 1 ] ** 2 ) ** 0.5 manual_length(v[v[ print ( "Manual length =" , manual_length) , manual_length) print ( "NumPy length =" , np.linalg.norm(v)) , np.linalg.norm(v)) Manual length = 5.0 NumPy length = 5.0 Both give the same result. Visualizing vector length 0 , 0 , v[ 0 ], v[ 1 ], angles = 'xy' , scale_units = 'xy' , scale = 1 , color = 'r' ) plt.quiver(, v[], v[], angles, scale_units, scale, color 0 , 5 ) plt.xlim( 0 , 5 ) plt.ylim( 0 , color = 'black' , linewidth = 0.5 ) plt.axhline(, color, linewidth 0 , color = 'black' , linewidth = 0.5 ) plt.axvline(, color, linewidth 0 ] / 2 , v[ 1 ] / 2 , f"Length= { length } " , fontsize = 10 , color = 'blue' ) plt.text(v[, v[length, fontsize, color plt.grid() plt.show() You’ll see the arrow (3,4) with its length labeled. Distance between two vectors The distance between v and another vector u is the length of their difference: ‖v - u‖ . u = np.array([ 0 , 0 ]) # the origin np.array([]) = np.linalg.norm(v - u) distnp.linalg.norm(vu) print ( "Distance between v and u =" , dist) , dist) Distance between v and u = 5.0 Since u is the origin, this is just the length of v . A more interesting distance u = np.array([ 1 , 1 ]) np.array([]) = np.linalg.norm(v - u) distnp.linalg.norm(vu) print ( "Distance between v and u =" , dist) , dist) Distance between v and u = 3.605551275463989 This measures how far (3,4) is from (1,1) . Visualizing distance between points 0 ], u[ 0 ]], [v[ 1 ], u[ 1 ]], color = [ 'red' , 'blue' ]) plt.scatter([v[], u[]], [v[], u[]], color]) 0 ], u[ 0 ]], [v[ 1 ], u[ 1 ]], 'k--' ) plt.plot([v[], u[]], [v[], u[]], 0 ], v[ 1 ], 'v' , fontsize = 12 , color = 'red' ) plt.text(v[], v[],, fontsize, color 0 ], u[ 1 ], 'u' , fontsize = 12 , color = 'blue' ) plt.text(u[], u[],, fontsize, color plt.grid() plt.show() The dashed line shows the distance between the two points. Higher-dimensional vectors Norms and distances work the same way in any dimension: a = np.array([ 1 , 2 , 3 ]) np.array([]) b = np.array([ 4 , 0 , 8 ]) np.array([]) print ( "‖a‖ =" , np.linalg.norm(a)) , np.linalg.norm(a)) print ( "‖b‖ =" , np.linalg.norm(b)) , np.linalg.norm(b)) print ( "Distance between a and b =" , np.linalg.norm(a - b)) , np.linalg.norm(ab)) ‖a‖ = 3.7416573867739413 ‖b‖ = 8.94427190999916 Distance between a and b = 6.164414002968976 Even though we can’t draw 3D easily on paper, the formulas still apply. Try It Yourself Compute the length of np.array([5,12]) . What do you expect? Find the distance between (2,3) and (7,7) . Can you sketch it by hand and check? In 3D, try vectors (1,1,1) and (2,2,2) . Why is the distance exactly sqrt(3) ?
6. Dot Product The dot product is one of the most important operations in linear algebra. It takes two vectors and gives you a single number. That number combines both the lengths of the vectors and how much they point in the same direction. In this lab, we’ll calculate dot products in several ways, see how they relate to geometry, and visualize their meaning. Set Up Your Lab import numpy as np numpynp import matplotlib.pyplot as plt matplotlib.pyplotplt Step-by-Step Code Walkthrough Algebraic definition The dot product of two vectors is the sum of the products of their components: v = np.array([ 2 , 3 ]) np.array([]) u = np.array([ 4 , - 1 ]) np.array([]) = v[ 0 ] * u[ 0 ] + v[ 1 ] * u[ 1 ] dot_manualv[u[v[u[ = np.dot(v, u) dot_numpynp.dot(v, u) print ( "Manual dot product:" , dot_manual) , dot_manual) print ( "NumPy dot product:" , dot_numpy) , dot_numpy) Manual dot product: 5 NumPy dot product: 5 Here, (2*4) + (3*-1) = 8 - 3 = 5 . Geometric definition The dot product also equals the product of the lengths of the vectors times the cosine of the angle between them: \[ v \cdot u = \|v\| \|u\| \cos \theta \] We can compute the angle: = np.linalg.norm(v) norm_vnp.linalg.norm(v) = np.linalg.norm(u) norm_unp.linalg.norm(u) = np.dot(v, u) / (norm_v * norm_u) cos_thetanp.dot(v, u)(norm_vnorm_u) = np.arccos(cos_theta) thetanp.arccos(cos_theta) print ( "cos(theta) =" , cos_theta) , cos_theta) print ( "theta (in radians) =" , theta) , theta) print ( "theta (in degrees) =" , np.degrees(theta)) , np.degrees(theta)) cos(theta) = 0.33633639699815626 theta (in radians) = 1.2277723863741932 theta (in degrees) = 70.3461759419467 This gives the angle between v and u . Visualizing the dot product Let’s draw the two vectors: 0 , 0 ,v[ 0 ],v[ 1 ],angles = 'xy' ,scale_units = 'xy' ,scale = 1 ,color = 'r' ,label = 'v' ) plt.quiver(,v[],v[],angles,scale_units,scale,color,label 0 , 0 ,u[ 0 ],u[ 1 ],angles = 'xy' ,scale_units = 'xy' ,scale = 1 ,color = 'b' ,label = 'u' ) plt.quiver(,u[],u[],angles,scale_units,scale,color,label - 1 , 5 ) plt.xlim( - 2 , 4 ) plt.ylim( 0 ,color = 'black' ,linewidth = 0.5 ) plt.axhline(,color,linewidth 0 ,color = 'black' ,linewidth = 0.5 ) plt.axvline(,color,linewidth plt.grid() plt.show() The dot product is positive if the angle is less than 90°, negative if greater than 90°, and zero if the vectors are perpendicular. Projections and dot product The dot product lets us compute how much of one vector lies in the direction of another. = np.dot(v, u) / np.linalg.norm(u) proj_lengthnp.dot(v, u)np.linalg.norm(u) print ( "Projection length of v onto u:" , proj_length) , proj_length) Projection length of v onto u: 1.212678125181665 This is the length of the shadow of v onto u . Special cases If vectors point in the same direction, the dot product is large and positive.
... continue reading