Independent Component Analysis (ICA)

ICA is a technique used to separate mixed signals into independent non-Gaussian components.

Used heavily in:

• Audio processing
• Image processing
• Biomedical signals (EEG, ECG)
• Blind source separation

1. What is ICA?

ICA finds a linear transformation that makes the resulting components statistically independent.

Statistical independence:

2. Assumptions of ICA

1. Source signals are statistically independent
2. Sources are non-Gaussian
   • ICA cannot separate Gaussian components
3. Mixing is linear
   • Non-linear mixtures break ICA

3. Mathematical Representation

Observed mixed signals:

$Hidden independent components:

$Linear mixing model:

$Goal of ICA:

$Where:

• = unknown mixing matrix
• = unmixing matrix (to be learned)

ICA tries to find such that components of are as independent as possible.

Independence is measured using a function:

ICA finds that minimizes dependence.

4. Real-World Example (Party Problem)

A room has N speakers talking simultaneously and N microphones placed at different positions.

Each microphone records:

• A mixture of all speakers
• With different intensities

Goal:
Use ICA to recover each speaker’s original voice:

Where:

• = mixed signals
• ​ = independent components

5. Advantages of ICA

1. Separates mixed signals
   • Excellent for blind source separation
2. Unsupervised technique
   • No labeled data needed
3. Useful for feature extraction
   • Finds important independent features

6. Disadvantages of ICA

1. Assumes non-Gaussian sources
   • Fails if sources are Gaussian
2. Assumes linear mixing
   • Ineffective for nonlinear mixtures
3. Computationally expensive
   • Hard to scale to large datasets