From the Solving System of Linear Equations, the system of linear equations can be written in the matrix form as:

AX = B

Where:

A is m x n matrix of coefficients.

X is n x 1 matrix of unknowns.

B is m x 1 matrix of constant terms.

So, the system becomes overdetermined when m > n.
To solve this kind of system, we need to find X such that the residual vector

R = B – AX

is as small as possible.

The solution X given be the least squares method minimizes ||R||² = ||B – AX||². This solution is Least-Squares Solution.

Least-Squares Solution with Pseudoinverse

From:

AX = B

When m > n, so:

A^TAX = A^TB

Then

X = (A^TA)^-1A^TB

(A^TA)^-1A^T is called pseudo inverse of A

Solving with MathPACK Solver

MathPACK Solver provides inv or pinv function finding invert matrix. If the input matrix to the function, is not squared matrix, the pseudo invert will be returned from the function.

Function Signature

U = inv(A);

U = pinv(A);

Example

Find the least-squares solution of the following system of equations:

x₁ – x₂ = 2

x₁ + x₂ = 4

2x₁ + x₂ = 8


    »A = [1,-1; 1, 1; 2, 1]; 

a₁₁x₁ + a₁₂x₂ + a₁₃x₃ + … + a_1nx_n = 0

a₂₁x₁ + a₂₂x₂ + a₂₃x₃ + … + a_2nx_n = 0

… … … …

a_m1x₁ + a_m2x₂ + a_m3x₃ + … + a_mnx_n = 0

which can be written in matrix form

AX = 0

Every homogeneous system of linear equations has at least one solution, which is X = 0. It is called zero solution or trivial solution.

• If det(A) is non-zero, then it is the unique solution.
• if det(A) is zero, then there is an infinite number of solutions.

An important technique for solving a homogeneous system of linear equations AX = 0 is to form the augmented matrix M = [ A | 0 ] and reduce M to reduced row echelon form.

Solving with MathPACK Solver

To reduce the augmented matrix M to reduced row echelon form, you can use the function rref() in MathPACK Solver.

Function Signature

U = rref(A)

Example

Solve the homogeneous linear system of equations.

x₁ + x₂ – 2x₃ = 0

3x₁ + 2x₂ + 4x₃ = 0

4x₁ + 3x₂ + 2x₃ = 0

From the given system of linear equations, we can write it in the matrix form

AX = 0

Where:

A = [1, 1, -2; 3, 2, 4; 4, 3, 2];

X = [x₁;x₂;x₃];

So, we can create matrix A and construct augmented matrix M by:


»A = [1, 1, -2; 3, 2, 4; 4, 3, 2]; 

»M = zeros(3, 4); 

»M(1:3,1:3) = A;


Reduce M to reduced row echelon form using rref bundled function


»rref(M) 

ans = 

    1 0 8 0  

    0 1 -10 0 

    0 0 0 0 

Rewrite system in the equations form.

x₁ + 0 + 8x₃ = 0

0 + x₂ – 10x₃ = 0

Thus:

x₁ = – 8x₃

x₂ = 10x₃

Let s=x₃, then

x₁ = – 8s

x₂ = 10s

x₃ = s

The solution vector X will be:

X = [-8; 10; 1]s

For example:

5x + 2y – 3z = 1

-2x – y + 7z = 8

4x – 13y + z = -10

is a system of three equations in the three variables x, y and z.

A general form of system of m linear equations with n unknowns can be written as:

a₁₁x₁ + a₁₂x₂ + a₁₃x₃ + … + a_1nx_n = b₁

a₂₁x₁ + a₂₂x₂ + a₂₃x₃ + … + a_2nx_n = b₂

… … … …

a_m1x₁ + a_m2x₂ + a_m3x₃ + … + a_mnx_n = b_m

From above system of linear equations, x₁, x₂, x₃, …, x_n are unknown, a₁₁, a₁₂, a₁₃, … a_mn are coefficients of the system, and b₁, b₂, b₃, …, b_n are the constant terms. If all constant terms are zero, the system of linear equations is called homogeneous system (see Solving Homogeneous System).

Matrix Equation

The system of linear equations is equivalent to a matrix equation of the form

AX=B

Where:

A is m x n matrix of coefficients. Element of matrix A at i ^th-row and j ^th-column is a_ij.

X is n x 1 matrix (column vector with n entries). Element of matrix X at i ^th-row is x_i.

B is m x 1 matrix (column vector with m entries). Element of matrix B at i ^th-row is b_i.

Solving with MathPACK Solver

MathPACK Solver can solve the system of linear equations for finding the values of unknowns (matrix X) by using linsolve function. The system of linear equations must be written in the form of Matrix Equation described in previous section and the number of linear equations must be equal to number of unknowns (A must be square matrix : m = n).

Function Signature

X = linsolve(A, B);

Example

5x + 2y – 3z = 1

-2x – y + 7z = 8

4x – 13y + z = -10

So, the following expressions must be executed in MathPACK Solver console


»A = [5, 2, -3; -2, -1,7; 4, -13, 1]; 

»B = [1; 8; -10]; 

»X = linsolve(A, B) 

ans = 

    0.6571  

    1.0857  

    1.4857 

 

So, the solution will be:

x = 0.6571

y = 1.0857

z = 1.4857