Chapter 9 Matrix Multiplication
In this chapter you will learn about scalar-matrix multiplication and matrix-matrix multiplication.
9.1 Scalar–Matrix Multiplication
Any matrix can be multiplied by a scalar. The result is a matrix in which each element in the original matrix is multiplied by the value of the scalar. For example to multiply matrix A (from the previous section) by a scalar \(\lambda = -2\),
\[ \begin{split} -2\mathbf{A} &= -2\begin{bmatrix} 112 & 86 & 0 \\ 134 & 94 & 0 \end{bmatrix} \\[2em] &= \begin{bmatrix} -2(112) & -2(86) & -2(0) \\ -2(134) & -2(94) & -2(0) \end{bmatrix} \\[2em] &= \begin{bmatrix} -224 & -172 & 0 \\ -268 & -188 & 0 \end{bmatrix} \end{split} \] In general the elements of the product matrix, \(\lambda\mathbf{A}\), are \(\lambda (a_{ij})\) for every i and j. Scalar–matrix multiplication is commutative, that is:
\[ \lambda\mathbf{A} = \mathbf{A}\lambda \]
In R we use the * operator to perform scalar–matrix multiplication.
-2 * A [,1] [,2] [,3]
[1,] -224 -172 0
[2,] -268 -188 09.2 Matrix Multiplication
The process of multiplying matrices follows the same basic principle as vector multiplication, where we consider matrices to be a collection of column vectors. When we multiply vectors, they must have the same number of elements because we multiply corresponding elements and add the resulting products, obtaining a scalar (dot) product (i.e., \(\mathbf{a}\bullet\mathbf{b}\)). To multiply matrix X by matrix Y, we are going to compute the dot product between each row in X and each column in Y, that is, \(\mathbf{x}^\intercal_{j}\bullet\mathbf{b}_j\). As an example, consider the following two matrices:
\[ \mathbf{X} = \begin{bmatrix} 2 & 3 \\ 1 & 2 \end{bmatrix} \qquad \mathbf{Y} = \begin{bmatrix} 5 & -2 \\ 4 & -1 \end{bmatrix} \]
To complete the multiplication of XY, we compute the dot product between the first row of X and the first column of Y (i.e., \(\mathbf{X}_1^\intercal\bullet\mathbf{Y}_1\)).
\[ \begin{split} \begin{bmatrix}2 & 3\end{bmatrix}\begin{bmatrix}5 \\ 4\end{bmatrix} &= 2(5) + 3(4)\\[2em] &= 22 \end{split} \]
This gives us the element in the first row and first column of the product matrix, Z. The element in the first row and second column of Z is based on the dot product between the the first row of X and the second column of Y. This continues, finding the dot product between rows of X and columns of Y for each corresponding element in Z.
\[ \begin{split} \mathbf{Z} &= \begin{bmatrix}\begin{bmatrix}2 & 3\end{bmatrix}\begin{bmatrix}5 \\ 4\end{bmatrix} & \begin{bmatrix}2 & 3\end{bmatrix}\begin{bmatrix}-2 \\ -1\end{bmatrix} \\ \begin{bmatrix}1 & 2\end{bmatrix}\begin{bmatrix}5 \\ 4\end{bmatrix} & \begin{bmatrix}1 & 2\end{bmatrix}\begin{bmatrix}-2 \\ -1\end{bmatrix}\end{bmatrix}\\[2em] &= \begin{bmatrix} 22 & -7 \\ 13 & -4 \end{bmatrix} \end{split} \]
This process must always be carefully followed: Finding the dot product between a row of the first matrix and a column of the second matrix. The general rule on matrix multiplication for \(\mathbf{AB} = \mathbf{C}\), for each element in C, \(C_{ij}\) is the dot product of the ith row of A and the jth column of B.
To compute the product between two matrices we use the matrix multiplication operator, %*%.
# Create A
A = matrix(data = c(2, 3, 1, 2), byrow = TRUE, nrow = 2)
# Create B
B = matrix(data = c(5, -2, 4, -1), byrow = TRUE, nrow = 2)
# Matrix multiplication
A %*% B [,1] [,2]
[1,] 22 -7
[2,] 13 -4In contrast to scalar multiplication, matrix multiplication is rarely commutative, that is, it is rare that \(\mathbf{AB}=\mathbf{BA}\). In our previous example, \(\mathbf{AB}\neq\mathbf{BA}\).
# Matrix multiplication
B %*% A [,1] [,2]
[1,] 8 11
[2,] 7 10The resulting product is not the same as computing AB. The order of the multiplication is quite important, so rather than saying that “A is multiplied by B”, we instead specify the order by saying:
- A is postmultiplied by B; or
- B is premultiplied by A.
9.2.1 Conformability
We can also multiply matrices that are not of equal dimensions, as long as they are conformable. To be conformable, the number of columns in the premultiplied (first) matrix must equal the number of rows in the postmultiplied (second) matrix. For example, matrices A and B (below) are conformable since the number of columns in A is equal to the number of rows in B, namely 4.
\[ \underset{2\times4}{\mathbf{A}} = \begin{bmatrix} 5 & 2 & 3 & 4 \\ 5 & 4 & 6 & 1 \end{bmatrix} \qquad \underset{4\times3}{\mathbf{B}} = \begin{bmatrix} 8 & 9 & 4 \\ 6 & 5 & 1 \\ 2 & 3 & 4 \\ 6 & 1 & 2 \end{bmatrix} \]
One way to consider conformability is to ensure that the inner dimensions of the two matrices being multiplied are equal. Here the inner dimensions (shown in red) are both 4.
\[ \underset{2\times\color{red}{4}}{\mathbf{A}}~\underset{{\color{red}{4}} \times3}{\mathbf{B}} \]
# Create A
A = matrix(data = c(5, 2, 3, 4, 5, 4, 6, 1), byrow = TRUE, nrow = 2)
# Create B
B = matrix(data = c(8, 9, 4, 6, 5, 1, 2, 3, 4, 6, 1, 2), byrow = TRUE, nrow = 4)
# Postmultiply A by B
A %*% B [,1] [,2] [,3]
[1,] 82 68 42
[2,] 82 84 50Notice the product AB has dimensions of \(2\times4\), which are the outer dimensions shown in the product (below in red).
\[ \underset{{\color{red}{2}} \times4}{\mathbf{A}}~\underset{4\times\color{red}{3}}{\mathbf{B}} \]
If we tried to postmultiply B by A, that product is not conformable since the inner dimensions would no longer match (i.e., \(3\neq2\)).
\[ \underset{4\times\color{red}{3}}{\mathbf{B}} ~ \underset{{\color{red}{2}}\times4}{\mathbf{A}} \]
# Postmultiply B by A
B %*% AError in B %*% A: non-conformable arguments9.3 Properties of Matrix Multiplication
Although matrix multiplication is rarely commutative, it does have the following properties:
- \(\mathbf{A}(\mathbf{BC})=(\mathbf{AB})\mathbf{C})\) (Associative Property)
- \(\mathbf{A}(\mathbf{B}+\mathbf{C})=\mathbf{AB} +\mathbf{AC})\) (Left Distributive Property)
- \((\mathbf{B}+\mathbf{C})\mathbf{A}=\mathbf{BA} +\mathbf{CA})\) (Right Distributive Property)
so long as the matrices being considered are conformable.
Aside from commutativity, there are also some other properties that matrix multiplication lack. For example, in scalar arithmetic, if \(ab = 0\), then either a or b must be zero. In matrix multiplication, however, this is not always the case. To illustrate, consider the following three matrices:
\[ \mathbf{A} = \begin{bmatrix}-2 & 4 \\-2 & 4 \end{bmatrix} \qquad \mathbf{B} = \begin{bmatrix}0 & 0 \\0 & 0 \end{bmatrix} \qquad \mathbf{C} = \begin{bmatrix}4 & 4 \\2 & 2 \end{bmatrix} \]
Matrix B, in which every element is zero is called the null matrix. (A null matrix is a matrix in which every element is zero.) If we postmultiply matrix A by the null matrix B, we get:
\[ \begin{split} \mathbf{A}\mathbf{B} &= \begin{bmatrix}-2 & 4 \\-2 & 4 \end{bmatrix} \begin{bmatrix}0 & 0 \\0 & 0 \end{bmatrix} \\[2em] &= \begin{bmatrix}0 & 0 \\0 & 0 \end{bmatrix} \end{split} \]
The result of postmultiplying a matrix (so long as it is conformable) by the null matrix is a null matrix. This is also true when we premultiply by a null matrix. Now consider postmultiplying matrix A by matrix C:
\[ \begin{split} \mathbf{A}\mathbf{C} &= \begin{bmatrix}-2 & 4 \\-2 & 4 \end{bmatrix} \begin{bmatrix}4 & 4 \\2 & 2 \end{bmatrix} \\[2em] &= \begin{bmatrix}0 & 0 \\0 & 0 \end{bmatrix} \end{split} \]
We again obtain a null matrix. However, neither A nor C was a null matrix. (In fact, none of the elements of A or C were zero).
Relatedly, in scalar arithmetic, if \(ab = 0\), then \(ba = 0\). This property does not hold in matrix multiplication. For example, we just saw that \(\mathbf{A}\mathbf{C}=\mathbf{0}\). But, if we postmultiply C by A:
\[ \begin{split} \mathbf{C}\mathbf{A} &= \begin{bmatrix}4 & 4 \\2 & 2 \end{bmatrix}\begin{bmatrix}-2 & 4 \\-2 & 4 \end{bmatrix} \\[2em] &= \begin{bmatrix}-16 & 16 \\-8 & 8 \end{bmatrix} \end{split} \]
That is, although \(\mathbf{A}\mathbf{C}=\mathbf{0}\), \(\mathbf{C}\mathbf{A}\neq\mathbf{0}\). Changing the order of the matrix multiplication changes the resulting product.
Drawing from this example we see that since \(\mathbf{AB}=\mathbf{0}\) and \(\mathbf{AC}=\mathbf{0}\), this means that \(\mathbf{AB}=\mathbf{AC}\). However, \(\mathbf{B}\neq\mathbf{C}\). This implies that “cancellation” is not a valid property of matrix multiplication.
9.4 Revisiting Dot Products
It turns out that we can express the dot product between two vectors as matrix multiplication. Consider the column vectors a and b:
\[ \mathbf{a} = \begin{bmatrix}5 \\ 4 \\ 7 \\ 2\end{bmatrix} \qquad \mathbf{b} = \begin{bmatrix}1 \\ 0 \\ -1 \\ 2\end{bmatrix} \]
Remember column vectors are matrices with one column. The dot product between a and b is the same as premultiplying b by the transpose of a, that is:
\[ \mathbf{a}\bullet\mathbf{b} = \underset{1\times4}{\mathbf{a}^\intercal}~\underset{4\times1}{\mathbf{b}} \]
By transposing a, the matrices are conformable, and the resulting product will be a \(1\times1\) matrix (i.e., a scalar).
# Create a
a = matrix(data = c(5, 4, 7, 2), ncol = 1)
# Create b
b = matrix(data = c(1, 0, -1, 2), ncol = 1)
# Postmultiply the transpose of a by b
t(a) %*% b [,1]
[1,] 2# Compute dot product as sum of products
sum(a * b)[1] 2