Skip to content

Contains important notes/definitions/propositions from the book Algebra, Topology, Differential Calculus, and Optimization Theory for Computer Science and Machine Learning Jean Gallier and Jocelyn Quaintance

Notifications You must be signed in to change notification settings

akhauriyash/MathForCS_ML

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

26 Commits
 
 
 
 
 
 

Repository files navigation

Algebra, Topology, Differential Calculus, and Optimization Theory for Computer Science and Machine Learning

Jean Gallier and Jocelyn Quaintance

Contains important notes/definitions/propositions from the book Algebra, Topology, Differential Calculus, and Optimization Theory for Computer Science and Machine Learning Jean Gallier and Jocelyn Quaintance. The book can be found here.

Important chapters (Personally, chapters as indicated by Contents): 2, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 22, 23, 39, 40, 41, 42, 43, 44, 45,46, 47, 48, 49, 50, 51, 52, 54

Table of contents

Introduction

These are just meant to be notes for personal use, you may find them useful. I have very limited background in mathematics, so there may be mistakes here. You will most definitely need to keep the book open on the side as this mentions specific definitions/propositions by reference for brevity.

I may give up on this endeavor at any moment I find better ways to condense knowledge for myself. Honestly, this may be hard to read/understand unless you have a background in mathematics or have read this book atleast roughly.

Groups, Rings, and Fields

def2.1

Abelian: A group G is abelian (or commutative) if: abel

Monoid: Set M with operation MxM -> M and element 'e' satisfying (G1) and (G2) is a monoid. Not necessarily a group.

General Linear Group GL(n, R) or GL(n, C) :

  1. N x N Invertible with Real/Complex Coefficients
  2. Group under MatMul
  3. Identity element In

Special Linear Group SL(n, R) or GL(n, R):

  1. Sub-group of GL with det(matrix) = 1

Orthogonal Group O(n):

  1. Set of N x N invertible matrices (Q) with real coefficients
  2. QQ' = Q'Q = In (Q' is the transpose here.) Special Orthogonal Group SO(n):
  3. Sub-group of O(n) with Det(Q) = 1

Proposition 2.7: For finite group G, subgroup H of G, orcer of H divides order of G.

Rings

def2.16

Fields

def2.22

Characteristic of Field

def2.28

Vector Spaces, Bases, Linear Maps

Matrices and Linear Maps

Haar Bases, Haar Wavelets, Hadamard Matrices

Direct Sums

Determinants

Permutations

def7.1

Transpositions

def7.1b

Signature of permutation

def7.2

Signature of product of transpositions

prop7.2

Properties of Alternating Multilinear Maps:

prop7.3

Important Lemma

lemma7.4a lemma7.4b

Note how:

wowdet

remarkpg193

For the rest of this chapter, K is a commutative ring and when needed a field.

Adjugate and Minor

def7.7

Invertibility of a Matrix

prop7.10

Determinant of a Linear Map

def7.8

Cayley-Hamilton Theorem

theo7.14

Permanents It is a multilinear symmetric form. Refer to book for an execllent interpretation of permanents. permanentspg209

Gaussian Elimination, LU, Cholesky, Echelon Form

This chapter assumes all vector spaces are over the field R. All results that do not rely on the ordering on R or on taking square roots hold for arbitrary fields.

Read the book for background on Bezier curves and curve interpolation. Gaussian Elimination

  1. Computing inverse directly is inefficient.
  2. Solving large linear systems by computing determinants (Cramers formulae) not used.

Solution trivial if A is an upper-triangular matrix. We use Gaussian elimination to iteratively eliminate variables using simple row operations.

theo8.1

def8.1

theo8.5a theo8.5b theo8.5c

theo8.10

Transvections and Dilatations Transvections and Dilatations characterize the linear isomorphisms of a vector space E that leave some vector in some hyperplane fixed. These maps are linear maps represented in some suitable basis by elementary matrices of the form Ei,j;β (Transvections) or Ei,λ (Dilatations).

  1. Transvections generate the group SL(E)
  2. Dilatations generate the group GL(E)

def8.6

def8.7

Vector Norms and Matrix Norms

def9.1

Holders Inequalities

coll9.2

For p = 2, it is the standard Cauchy-Schwarz inequality.

Equivalence of norms

def9.2

Check : Corollary 9.4.

theo9.5

Normal Matrix Mn(C) : AA* = A*A Unitary Matrix Mn(C) : UU* = U*U = I Orthogonal Matrix Mn(R) : QQT = QTQ = I

Characteristic Polynomial, eigenvalues, spectrum and spectral radius. def9.5

Spectral radius of Mn(C) is always smaller-than or equal to any matrix norm.

def9.6

Frobenius Norm :

  1. It is a matrix norm.
  2. Untarily invariant

Proposition 9.8: It says that every linear map on a finite-dimensional space is bounded, impyling that every linear map on a finite-dimensional space is continuous.

Check Section 9.3 (Subordinate Norms) for another method of obtaining matrix norms.

Condition Number For its properties, refer to Proposition 9.17. def9.10

Iterative Methods for Solving Linear Systems

The Dual Space and Duality

Euclidean Spaces

QR-Decomposition for Arbitrary Matrices

Hermitian Spaces

Eigenvectors and Eigenvalues

Unit Quaternions and Rotations in SO(3

Spectral Theorems

Computing Eigenvalues and Eigenvectors

Introduction to The Finite Elements Method

Graphs and Graph Laplacians; Basic Facts

Spectral Graph Drawing

Singular Value Decomposition and Polar Form

Applications of SVD and Pseudo-Inverses

Basics of Affine Geometry

Embedding an Affine Space in a Vector Space

Basics of Projective Geometry

The Cartan–Dieudonn´e Theorem

Isometries of Hermitian Spaces

The Geometry of Bilinear Forms; Witt’s Theorem

Polynomials, Ideals and PID’s

Annihilating Polynomials; Primary Decomposition

UFD’s, Noetherian Rings, Hilbert’s Basis Theorem

Tensor Algebras

Exterior Tensor Powers and Exterior Algebras

Introduction to Modules; Modules over a PID

Normal Forms; The Rational Canonical Form

Topology

A Detour On Fractals

Differential Calculus

Extrema of Real-Valued Functions

Newton’s Method and Its Generalizations

Quadratic Optimization Problems

Schur Complements and Applications

Convex Sets, Cones, H-Polyhedra

Linear Programs

The Simplex Algorithm

Linear Programming and Duality

Basics of Hilbert Spaces

General Results of Optimization Theory

Introduction to Nonlinear Optimization

Subgradients and Subdifferentials

Dual Ascent Methods; ADMM

Ridge Regression and Lasso Regression

Positive Definite Kernels

Soft Margin Support Vector Machines

Total Orthogonal Families in Hilbert Spaces

Zorn’s Lemma; Some Applications

About

Contains important notes/definitions/propositions from the book Algebra, Topology, Differential Calculus, and Optimization Theory for Computer Science and Machine Learning Jean Gallier and Jocelyn Quaintance

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published