Matrix multiplication and inverses

Matrix multiplication involves the multiplication of two matrices to produce a third matrix – the matrix product. This allows for the efficient processing of multiple data points or operations simultaneously, rather than computing each one individually. Just as we can multiply matrices, we can also reverse this process with inverse matrices, which act as the multiplicative inverse. This is similar to division in arithmetic.

Use this page to revise the following concepts of matrix multiplication and inverses:

Matrix Multiplication

Matrix multiplication is an operation between two matrices where the rows of the first matrix are multiplied by the columns of the second matrix to produce a new matrix – the matrix product.

Core Condition of Matrix Multiplication

For the multiplication to be valid, the number of columns of the first matrix must be equal to the number of rows of the second matrix.

If matrix \(A\) has an order of \(n\times m\), and matrix \(B\) has an order of \(p\times q\); then \(A\times B\) only exists if \(m=p\).

For matrix \(A\) and \(B\):

\(A=\left[\begin{align}& a_{11}\space\space a_{12}\space\space a_{13}\\& a_{21}\space\space a_{22}\space\space a_{23}\end{align}\right]\)                \(B=\left[\begin{align}& b_{11}\space\space b_{12}\\& b_{21}\space\space b_{22}\\& b_{31}\space\space b_{32}\end{align}\right]\)

\(A\times B\) exists as the number of columns of \(A(3)\) is equal to the number of rows of \(B(3)\).

Resulting Matrix

This produces a new matrix whose order is equal to the number of rows of the first matrix by the number of columns of the second matrix. This is known as the matrix product or the resulting matrix. For the example above, if \(A\times B=C\), then \(C\), the matrix product has an order of \(n\times q\).

Finding the Matrix Product

The elements of matrix \(C\) are formed from the sum of the elements in the rows of matrix \(A\) and the columns of matrix \(B\).

The first calculation is the sum of the first row of \(A\) multiplied by the first column of \(B\)

\(A\times B=\left[\begin{align}& a_{11}\space\space a_{12}\space\space a_{13}\\& a_{21}\space\space a_{22}\space\space a_{23}\end{align}\right]\) \(B=\left[\begin{align}& b_{11}\space\space b_{12}\\& b_{21}\space\space b_{22}\\& b_{31}\space\space b_{32}\end{align}\right]\)

\(\phantom {A\times B}=\left[\begin{align}& a_{11}\times b_{11}+a_{12}\times b_{21}+a_{13}\times b_{31}\ldots\\& \space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\ldots\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\space\ldots\end{align}\right]\)

The same process is repeated, to complete the matrix product. A quick way to think about which product makes each element in the final matrix product is the element reference.

If the matrix product is:

\(C=\left[\begin{align}& c_{11}\space\space c_{12}\\& c_{21}\space\space c_{22}\end{align}\right]\)

then the \(c_{ij}\) can be thought of the \(i\) row of matrix \(A\) multiplied by the \(j\) column of matrix \(B\), for example, \(c_{21}\) is made from the 2nd row of matrix \(A\) multiplied by the 1st column of matrix \(B\).

The complete matrix product from above is as such:

\[A\times B=\left[\begin{align}& a_{11}\times b_{11} +a_{12}\times b_{21} +a_{13}\times b_{31}\space\space\space\space\space a_{11}\times b_{12}+a_{12}\times b_{22}+a_{13}\times b_{32}\\& a_{21}\times b_{11} +a_{22}\times b_{21} +a_{23}\times b_{31}\space\space\space\space\space a_{21}\times b_{12}+a_{22}\times b_{22}+a_{23}\times b_{32}\end{align}\right]\]

Worked Example

For the following matrices:

\(A=\left[\begin{align}& 1\space\space 2\space\space 3\\& 4\space\space 5\space\space 6\end{align}\right]\),          \(B=\left[\begin{align}& 1\space\space 2\\& 3\space\space 4\\& 5\space\space 6\end{align}\right]\)

Find \(A\times B\).

Check to see if \(A\times B\) is valid:

The number of columns of matrix \(A\) is 3, and the number of rows of matrix \(B\) is 3.

Since these numbers are equal, matrix multiplication is valid.


Determine the order of the resulting matrix:

The number of rows of matrix \(A\) is 2, and the number of columns of matrix \(B\) is 2.

Therefore, the resulting matrix should have an order of \(2\times 2\).

\(A\times B=\left[\begin{align}& 1\space\space 2\space\space 3\\& 4\space\space 5\space\space 6\end{align}\right]\) \(B=\left[\begin{align}& 1\space\space 2\\& 3\space\space 4\\& 5\space\space 6\end{align}\right]\)

\(\phantom {A\times B}=\left[\begin{align}& 1\times 1 +2\times 3 +3\times 5\space\space\space\space\space 1\times 2 +2\times 4 +3\times 6\\& 4\times 1+ 5\times 3 +6\times 5\space\space\space\space\space 4\times 2+ 5\times 4+ 6\times 6\end{align}\right]\)

\(\phantom {A\times B}=\left[\begin{align}& 22\space\space 28\\& 49\space\space 64\end{align}\right]\)

Properties of Matrix Multiplication

Powers of Matrices

Matrices can also be raised to a power, resulting in the repeated operation of matrix multiplication. For example:

\[A^{3}=A\times A\times A\]

This operation is only defined for square matrices, and it is one of the special cases where matrix multiplication is commutative because the same matrix is being multiplied.

The zero power of a square matrix results in a special case – the identity matrix.

\[A^{0}=I\]

Negative powers of matrices follow the same rule as exponents, but the matrix must be invertible. A negative exponent can be thought of as taking the inverse of the matrix and then raising it to the positive power.

\[A^{-3}=(A^{-1})^{3}\]

Inverse Matrices

An inverse matrix can be multiplied with another matrix to reverse the effect of matrix multiplication, which can be thought of as a form of “matrix division”.

The matrix equivalent of the number 1 is the identity matrix, denoted as \(I\). Consequently, when a matrix is multiplied by its inverse, the result is the identity matrix:

\[A\times A^{-1}=A^{-1}\times A=I\]

For a \(2\times 2\) matrix:

\[A=\left[\begin{align}& a\space\space b\\& c\space\space d\end{align}\right]\]

its inverse can be calculated using the formula:

\[A^{-1}=\frac{1}{ad-bc}\left[\begin{align}& d\space\space\space -b\\& -c\space\space \space a\end{align}\right]\]

Determinant

The determinant tells us whether the inverse of a matrix can be found. It is denoted as \(\textrm{det}(A)\) and pronounced “determinant of A”.

For a \(2\times 2\) matrix:

\(A=\left[\begin{align}& a\space\space\space b\\& c\space\space\space d\end{align}\right]\)

\[\textrm{det}(A)=ad-bc\]


Note

A square matrix only has an inverse if its determinant is not equal to zero, that is \(\textrm{det}(A)\neq0\).

The determinant is the divisor in the formula for the inverse.

\(A^{-1}=\frac{1}{\textrm{det}(A)}\left[\begin{align}& d\space\space\space -b\\& -c\space\space\space a\end{align}\right]\)

Therefore, if the determinant is zero, the inverse does not exist, as dividing by zero would result in an undefined answer.

Solving Systems of Equations

Matrices can be used to solve systems of equations. This is done by writing them in the matrix form:

\[AX=B\]

Where:

  • \(A=\) coefficient matrix

  • \( X=\) variable matrix

  • \( B=\) constant matrix

    • To solve for \(X\), we pre-multiply both sides of the equation by the inverse of \(A\):

    \[A^{-1}AX=A^{-1}B\]

    Since \(A^{-1}A=I\):

    \[IX=A^{-1}B\]

    \[X=A^{-1}B\]

    Worked Example

    For the system of equations:

    \[3x+2y=16\]

    \[2x-y=-1\]

    It can be expressed in matrix form as:

    \(\left[\begin{align}& 3\space\space\space\space 2\\& 2\space -1\end{align}\right]\) \(\left[\begin{align}& x\\& y\end{align}\right]\) \(=\) \(\left[\begin{align}& 16\\& -1\end{align}\right]\)

    The inverse of matrix \(A\) is:

    \(A^{-1}=-\frac{1}{7}\left[\begin{align}& -1\space\space -2\\& -2\space\space\space\space\space\space 3\end{align}\right]\)

    Performing the matrix multiplication:

    \(\left[\begin{align}& x\\& y\end{align}\right]\) \(=\) \(-\frac{1}{7}\left[\begin{align}& -1\space\space -2\\& -2\space\space\space\space\space\space 3\end{align}\right]\)\(\left[\begin{align}& 16\\& -1\end{align}\right]\)

    \(\phantom{\left[\begin{align}& x\\& y\end{align}\right]}\) \(=\) \(-\frac{1}{7}\) \(\left[\begin{align}& -1\times 16 + (-2)\times (-1)\\& \space\space\space\space\space\space\space -2\times 16 + 3\times -1\end{align}\right]\)

    \(\phantom{\left[\begin{align}& x\\& y\end{align}\right]}\) \(=\) \(-\frac{1}{7}\) \(\left[\begin{align}& -14\\& -35\end{align}\right]\)

    \(\phantom{\left[\begin{align}& x\\& y\end{align}\right]}\) \(=\) \(\left[\begin{align}& 2\\& 5\end{align}\right]\)

    Therefore, the solution to the system of equations is \(x=2, y=5\).

    Permutation Matrices

    Permutation matrices provide a way of rearranging the rows or columns in a matrix. This is done by multiplying a matrix by a square binary matrix of the same size, where each row and column contains exactly one 1 (and all other elements are 0).

    We can start by observing the effect of multiplying by a row or column matrix.

    For example, matrix \(A\):

    \(A=\left[\begin{align}& 1\space\space\space\space 2\space\space\space\space 3\\& 4\space\space\space\space 5\space\space\space\space 6\\& 7\space\space\space\space 8\space\space\space\space 9\end{align}\right]\)

    Consider the following row matrix \(R=[0\space\space\space\space 1\space\space\space\space 0]\)

    By multiplying \(R\) by \(A\):

    \(RA\) \(=\) \([0\space\space\space\space 1\space\space\space\space 0]\) \(\left[\begin{align}& 1\space\space\space 2\space\space\space 3\\& 4\space\space\space 5\space\space\space 6\\& 7\space\space\space 8\space\space\space 9\end{align}\right]\)

    \(\phantom{RA}\) \(=\) \([0\times 1+1\times 4+0\times 7\space\space\space\space 0\times 2+1\times 5+0\times 8\space\space\space\space 0\times 3+1\times 6+0\times 9]\)

    \(\phantom{RA}\) \(=\) \([4\space\space\space\space 5\space\space\space\space 6]\)

    It can be observed that when \(R\) is pre-multiplied (to the left), the position of the \(1\) selects the corresponding row number (the second) from matrix \(A\).

    Consider the following column matrix, \(C=\left[\begin{align}& 0\\& 0\\& 1\end{align}\right]\)

    By multiplying \(A\) by \(C\):

    \(AC\) \(=\) \(\left[\begin{align}& 1\space\space\space 2\space\space\space 3\\& 4\space\space\space 5\space\space\space 6\\& 7\space\space\space 8\space\space\space 9\end{align}\right]\) \(\left[\begin{align}& 0\\& 0\\& 1\end{align}\right]\)

    \(\phantom{AC}\) \(=\) \(\left[\begin{align}& 1\times 0+2\times 0+3\times 1\\& 4\times 0+5\times 0+6\times 1\\& 7\times 0+8\times 0+9\times 1\end{align}\right]\)

    \(\phantom{AC}\) \(=\) \(\left[\begin{align}& 3\\& 6\\& 9\end{align}\right]\)

    It can be observed that when \(C\) is post-multiplied (to the right), the position of the \(1\) selects the corresponding column number (the third) from matrix \(A\).

    A permutation matrix does not isolate a single row or column. Instead, being a square matrix of the same order, it rearranges the rows or columns of the original matrix, resulting in a matrix of the same order. Whether the rows or columns are rearranged depends on whether you pre-multiply or post-multiply by the permutation matrix.

    For example, consider the permutation matrix, \(P\):

    \(P=\left[\begin{align}& 0\space\space\space 0\space\space\space 1\\& 1\space\space\space 0\space\space\space 0\\& 0\space\space\space 1\space\space\space 0\end{align}\right]\)

    To rearrange the rows, you must pre-multiply by \(P\):

    \(PA\) \(=\) \(\left[\begin{align}& 0\space\space\space 0\space\space\space 1\\& 1\space\space\space 0\space\space\space 0\\& 0\space\space\space 1\space\space\space0\end{align}\right]\) \(\left[\begin{align}& 1\space\space\space 2\space\space\space 3\\& 4\space\space\space 5\space\space\space 6\\& 7\space\space\space 8\space\space\space 9\end{align}\right]\)

    • Row 1 of \(P\) has a 1 in the third column, which means that the first row in the resulting matrix will be row 3 of \(A\).
    • Row 2 of \(P\) has a 1 in the first column, which means that the second row in the resulting matrix will be row 1 of \(A\).
    • Row 3 of \(P\) has a 1 in the second column, which means that the third row in the resulting matrix will be row 2 of \(A\).

    \(PA\) \(=\) \(\left[\begin{align}& 7\space\space\space 8\space\space\space 9\\& 1\space\space\space 2\space\space\space 3\\& 4\space\space\space 5\space\space\space 6\end{align}\right]\) \(\begin{align}& \textrm{Row}\space 3\space \textrm{of}\space A\\& \textrm{Row}\space 1\space \textrm{of}\space A\\& \textrm {Row}\space 2\space \textrm{of}\space A\end{align}\)

    Similarly, to rearrange the columns, you must post-multiply by \(P\):

    \(AP\) \(=\) \(\left[\begin{align}& 1\space\space\space 2\space\space\space 3\\& 4\space\space\space 5\space\space\space 6\\& 7\space\space\space 8\space\space\space 9\end{align}\right]\) \(\left[\begin{align}& 0\space\space\space 0\space\space\space 1\\& 1\space\space\space 0\space\space\space 0\\& 0\space\space\space 1\space\space\space 0\end{align}\right]\)

    • Column 1 of \(P\) has a 1 in the second row, which means that the first column in the resulting matrix will be column 2 of \(A\).
    • Column 2 of \(P\) has a 1 in the third row, which means that the second column in the resulting matrix will be column 3 of \(A\).
    • Column 3 of \(P\) has a 1 in the first row, which means that the third row in the resulting matrix will be column 1 of \(A\).

    The matrix AP, a 3 x 3 matrix. The first column has from top to bottom 2, 5, 8 and is labelled Column 2 of A, the second column is 3, 6, 9 and is labelled Column 3 of A and the third column is 1, 4, 7 and labelled Column 1 of A.