Chapter 2 Linear algebra

2.1 Matrices

2.1.1 Definition matrix

A \(m \times n\)-matrix has the form

\[ \begin {pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end {pmatrix} \]

• • \(a_{ij}\) : elements

• • \(m \times n\) : orde (size)

• • \(m\) : number of rows

• • \(n\) : number of columns

• • Notation: \(A\), \(B\), \(C\), ...

• • Short notation: \((a_{ij})\)

• • \(\mathbb {R}^{m \times n}\) : set of all \(m \times n\)-matrices

2.1.2 Special matrices

Zero matrix \(O_{m,n}\) is a \(m \times n\)-matrix with all \(a_{ij}=0\)

Row matrix precisely one row

Column matrix precisely one column

Square matrix equal number of rows as columns, \(m=n\).

• • main diagonal : elements \(a_{11}\), \(a_{22}\), ... , \(a_{nn}\)

• • anti-diagonal : elements \(a_{n\,1}\), \(a_{n-1\,2}\), ... , \(a_{1\,n}\)

Diagonal matrix a square matrix \(a_{ij} = 0\) with \(i \neq j\)

Scalar matrix diagonal matrix with \(a_{11} = a_{22} = \cdots = a_{nn}\)

Upper triangular matrix square matrix with \(a_{ij}=0 \quad (i \gt j)\)

Lower triangular matrix square matrix with \(a_{ij}=0 \quad (i \lt j)\)

Identity matrix \(I_n\) of order \(n\) with \(a_{ii}=1\)

\[ \begin {pmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \\ \end {pmatrix} \]

2.1.3 Sum two matrices

Definition

\[A, B \in \mathbb {R}^{m \times n} \Rightarrow A + B = C \in \mathbb {R}^{m \times n}\]

with

\[(c_{ij}) = (a_{ij} + b_{ij})\]

Group properties

\begin{align*} A + B & = C \in \mathbb {R}^{m \times n}\\ A + (B + C) & = (A + B) + C\\ A + O_{m,n} & = A = O_{m,n} + A\\ A + (-A) & = O_{m,n} = (-A) + A\\ A+B & = B+A \end{align*}

\[\Rightarrow \mathbb {R}^{m \times n}, + \text { is a commutative group}\]

2.1.4 Scalar product matrices

Definition

\[\lambda \in \mathbb {R}, A \in \mathbb {R}^{m \times n}\]

\begin{align*} \Rightarrow \lambda \cdot A &= \lambda \cdot \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end {pmatrix}\\ &= \begin{pmatrix} \lambda \cdot a_{11} & \lambda \cdot a_{12} & \cdots & \lambda \cdot a_{1n} \\ \lambda \cdot a_{21} & \lambda \cdot a_{22} & \cdots & \lambda \cdot a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ \lambda \cdot a_{m1} & \lambda \cdot a_{m2} & \cdots & \lambda \cdot a_{mn} \\ \end {pmatrix} \end{align*}

Properties

\begin{align*} \lambda \cdot A & \in \mathbb {R}^{m \times n} \\ \lambda \cdot (\mu \cdot A) & = (\lambda \mu ) \cdot A \\ \lambda \cdot (A + B) & = \lambda \cdot A + \lambda \cdot B \\ (\lambda + \mu ) \cdot A & = \lambda \cdot A + \mu \cdot A \\ 1 \cdot A & = A \\ 0 \cdot A & = O_{m,n} \end{align*}

\[\Rightarrow \mathbb {R}, \mathbb {R}^{m \times n}, + \text { is a real vector space}\]

2.1.5 Product two matrices

Definition

\[A \in \mathbb {R}^{m \times n}, B \in \mathbb {R}^{n \times p} \Rightarrow A \cdot B = C \in \mathbb {R}^{m \times p}\]

with

\[c_{ij} = \sum _{k=1}^n a_{ik} \cdot b_{kj}\]

Associative

\[A \cdot (B \cdot C) = (A \cdot B) \cdot C\]

Absorbing element

\[A_{m \times n} \cdot O_{n \times p} = O_{m \times p} = O_{m \times n} \cdot B_{n \times p}\]

Neutral element

\[A \cdot I = A = I \cdot A\]

NOT commutative!

2.1.6 Transposed matrix

Definition If \(A=(a_{ij})\) then the transposed matrix \(A^T\) is

\[A^T=(a_{ji})\]

(rows \(\leftrightarrow \) columns)

Properties

\begin{align*} (A^T)^T&=A\\ (\lambda \cdot A)^T&=\lambda \cdot A^T\\ (A + B)^T &= A^T + B^T\\ (A \cdot B)^T &= B^T \cdot A^T\\ \end{align*}

2.1.7 Special square matrices

\begin{align*} A \text { is {\bf symmetric} } &\Leftrightarrow A^T = A\\ A \text { is {\bf anti-symmetric} } &\Leftrightarrow A^T = -A\\ A \text { is {\bf orthogonal} } &\Leftrightarrow A \cdot A^T = I_n = A^T \cdot A\\ A \text { is {\bf idempotent} } &\Leftrightarrow A^2 = A\\ A \text { is {\bf nilpotent} } &\Leftrightarrow \exists n\in \mathbb {N}:A^n=O\\ \text {index } p &=\min \{n\in \mathbb {N}:A^n=O\} \end{align*}

2.1.8 Inverse matrices

\(A^{-1}\) is the inverse matrix of \(A\)

\[\Leftrightarrow A \cdot A^{-1} = I_n = A^{-1} \cdot A\]

2.2 System of linear equations

2.2.1 Definition

System of linear equations

\[\left \{ \begin {aligned} a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n & = b_1 \\ a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n & = b_2 \\ \vdots \qquad \qquad \qquad & = \,\vdots \\ a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n & = b_m \\ \end {aligned} \right .\]

• • \(a_{ij}\) : coefficients

• • \(x_i\) : unknowns

• • \(b_i\) : constant terms (known)

Matrix representation

\[A \cdot X = B\]

• • \(A\) : coefficient-matrix

  \[A = \begin {pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end {pmatrix} \]

• • \(X\) : matrix of unknowns

  \[X = \begin {pmatrix} x_1\\ x_2\\ \vdots \\ x_n\\ \end {pmatrix} \]

• • \(B\) : matrix of constant terms

  \[B = \begin {pmatrix} b_1\\ b_2\\ \vdots \\ b_m\\ \end {pmatrix} \]

Augmented matrix

\[A_B = \begin {pmatrix} a_{11} & a_{12} & \cdots & a_{1n} &|& b_1\\ a_{21} & a_{22} & \cdots & a_{2n} &|& b_2\\ \vdots & \vdots & \ddots & \vdots &\vdots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} &|& b_m\\ \end {pmatrix} \]

2.2.2 Elementary row operations

Swap two rows

\[R_i \leftrightarrow R_j\]

Multiply a row by a nonzero scalar

\[R_i \leftarrow k \cdot R_i \qquad \text { met } k \neq 0\]

Add to one row a multiple of another row

\[R_i \leftarrow R_i + k \cdot R_j \qquad \text { met } i \neq j\]

2.2.3 Echelon form

Reduced row echelon form has

• • all zero rows at the bottom

• • pivot: leading non-zero coefficient in a row is 1

• • all elements above and below the pivot are zero

• • echelon form: each row starts with more zeros than the previous row

\[ \begin {pmatrix} 1 & 0 & a_1 & 0 & b_1 \\ 0 & 1 & a_2 & 0 & b_2 \\ 0 & 0 & 0 & 1 & b_3 \\ 0 & 0 & 0 & 0 & 0 \\ \end {pmatrix} \]

Rang

• • \(\text {rank}(A)\): number of non-zero rows in a reduced row echelon matrix of \(A\)

2.2.4 Homogeneous system

General

\[\left \{ \begin {aligned} a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n &= 0\\ a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n &= 0\\ \vdots \qquad \qquad \qquad &= \,\vdots \\ a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n &= 0\\ \end {aligned} \right .\]

\[\Leftrightarrow A \cdot X = 0\]

Homogeneous \(2 \times 3-\)system

\[\left \{ \begin {aligned} a_1x + b_1y + c_1z &= 0\\ a_2x + b_2y + c_2z &= 0\\ \end {aligned} \right .\]

with rank 2 has infinity many (\(\lambda \in \mathbb {R}\)) solutions:

\[ x = \lambda \cdot \begin {vmatrix}b_1 & c_1 \\ b_2 & c_2\end {vmatrix} \quad y = - \lambda \cdot \begin {vmatrix}a_1 & c_1 \\ a_2 & c_2\end {vmatrix} \quad z = \lambda \cdot \begin {vmatrix}a_1 & b_1 \\ a_2 & b_2\end {vmatrix}\]

2.3 Determinants

2.3.1 Definitions

Determinant of a \(1 \times 1\)-matrix

\[\begin {vmatrix}a_{11}\end {vmatrix} = a_{11}\]

Determinant of a \(2 \times 2\)-matrix

\[\begin {vmatrix}a_{11} & a_{12} \\ a_{21} & a_{22} \end {vmatrix} = a_{11}a_{22} - a_{21}a_{12}\]

Determinant of a \(3 \times 3\)-matrix

Rule of Sarrus

\begin{align*} \begin{vmatrix}a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end {vmatrix} = \;& a_{11}a_{22}a_{33}+a_{12}a_{23}a_{31}+a_{13}a_{21}a_{32}\\ &-a_{31}a_{22}a_{13}-a_{32}a_{23}a_{11}-a_{33}a_{21}a_{12} \end{align*}

Laplace expansion

\begin{align*} \begin{vmatrix}a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end {vmatrix} &=a_{i1}A_{i1} + a_{i2}A_{i2} + a_{i3}A_{i3}\\ &=a_{1j}A_{1j} + a_{2j}A_{2j} + a_{3j}A_{3j} \end{align*}

• • \(A_{ij} = (-1)^{i+j} \cdot M_{ij}\) : cofactor of \(a_{ij}\), product of \((-1)^{i+j}\) en the minor \(M_{ij}\) of \(a_{ij}\)

• • \(M_{ij}\) : minor \(M_{ij}\) of \(a_{ij}\), determinant of the submatrix where the \(i\)-th row and the \(j\)-th column of \(A\) is removed

Determinant of a \(n \times n\)-matrix

\begin{align*} \det (A) &= \sum _{j=1}^{n} a_{ij}A_{ij} \qquad \text {$i$ a random row}\\ &= \sum _{i=1}^{n} a_{ij}A_{ij} \qquad \text {$j$ a random column}\\ \end{align*}

2.3.2 Properties of determinants

Main properties

• • Switch two rows or columns \(\Rightarrow \) change sign of determinant

• • Two equals rows or columns \(\Rightarrow \) determinant \(= 0\)

• • A row/column is the multiple of an other row/column \(\Rightarrow \) determinant \(= 0\)

• • Row/column times a factor \(k\) \(\Rightarrow \) determinant times factor \(k\)

• • Add to a row/column the multiple of an other row/column \(\Rightarrow \) determinant stays the same

Calculations with determinants

\begin{align*} \det (A \cdot B) &= \det (A) \cdot \det (B)\\ \det (k \cdot A) &= k^n \cdot \det (A)\\ \det (A^T) &= \det (A) \end{align*}

Existence of inverse matrix

\[\exists A^{-1} \Leftrightarrow \det (A) \neq 0\]

Calculating the inverse matrix

\[A^{-1} = \dfrac {1}{\det (A)} \cdot \text {adj}(A)\]

• • \(\text {adj}(A)\) : transpose of the cofactor matrix of \(A\)

Cramer’s Rule System

\[A \cdot X = B\]

with \(\det (A) \neq 0\):

\[ x_i = \dfrac {\det (A_i)}{\det (A)} \]

• • \(A_i\): matrix \(A\) with column \(i\) replaced by \(B\).