How SVD Transforms Image Processing: Compression, Denoising & AI

Unlocking Hidden Dimensions: Singular Value Decomposition (SVD) from Scratch with SageMath

How Netflix, Google, and Spotify Use the Magic of SVD

Ever wondered how Netflix knows what to recommend, how Google ranks web pages, or how Spotify categorizes your favorite music?

The answer often lies in a mathematical powerhouse: Singular Value Decomposition (SVD).

In this blog post, we'll explore the inner workings of SVD by building it from scratch using SageMath, and then connect it to the real-world magic it powers every day—from facial recognition to search engines and audio filtering.

What is Singular Value Decomposition?

SVD decomposes any matrix A into three matrices: \[ A = UΣV^T \]

• U: Left singular vectors (output directions)
• Σ: Diagonal matrix of singular values (importance of features)
• V: Right singular vectors (input directions)

This decomposition helps us understand the structure, energy, and dominant patterns in data.

Let's Build SVD from Scratch: A Hands-on Journey

Step 1: Transpose the Matrix

    A = matrix([[1, 2, 1, 0], [2, 0, 1, 1]])
	B = A.T
	show(B)
    BTB = B.T * B
	show(BTB)
    

Step 2: Find Eigenvalues and Singular Values

    egv = BTB.eigenvalues()
	s1, s2 = sqrt(egv[0]), sqrt(egv[1])
	show(s1, s2)
	S = matrix([[s1, 0, 0, 0], [0, s2, 0, 0]]).T
	show(S)
    

Step 3: Find Right Singular Vectors (V)

    D, P = BTB.eigenmatrix_right()
	v1 = P.columns()[0] / norm(P.columns()[0])
	v2 = P.columns()[1] / norm(P.columns()[1])
	show(v1)
	show(v2)
    

Step 4: Compute Left Singular Vectors (U)

    u1 = (1/s1) * B * v1
	u2 = (1/s2) * B * v2
	show(u1)
	show(u2)
    

Step 5: Complete U Matrix with Orthogonal Vectors

   	B1 = B.left_kernel().basis_matrix()
	u3 = B1.T.columns()[0] / norm(B1.T.columns()[0])
	u4 = B1.T.columns()[1] / norm(B1.T.columns()[1])
	show(u3)
	show(u4)
    

Step 6: Final Decomposition

	U = column_matrix([u1, u2, u3, u4])
	V = column_matrix([v1, v2])
	show(U)
	show(S)
	show(V)
    

Sanity Check:

	(U * S * V.T).T == A
    

Built-in SVD: Quick & Efficient

	U2, S2, V2 = A.change_ring(RDF).SVD()
	show(U2)
	show(S2)
	show(V2)
    

Visualizing the Power of SVD in Images

Image Compression with SVD

	@interact
	def compress_image(rank=(1, 20)):
    	A = matrix(RDF, image_data)
   	 	U, S, V = A.SVD()
    	Ak = sum(S[i]*U.column(i)*V.row(i) for i in range(rank))
    	matrix_plot(Ak).show()
    

Plot Singular Values

	plot(list(S), title="Singular Values", axes_labels=["Index", "Value"])
    

SVD: The Image Whisperer

So how does all this relate to images?
Images are just matrices of pixel values. And here’s where the SVD shines:

🗜 Compression

Keep only the largest 𝑘 singular values and vectors. You can reconstruct a visually similar image with far less data. \[A_k = \sum_{i=1}^{k} \sigma_i u_i v_i^T \approx A \] Like summarizing a photo with its most important features!

🧹 Denoising

Small singular values often represent noise. Removing them leaves behind a cleaner image.

🧠 Feature Extraction

In machine learning, especially in facial recognition, the singular vectors represent the key features of an image. These become powerful inputs for algorithms to recognize or classify faces.

🦄 Real-World Applications of SVD

• 📖 Google PageRank: Uses eigen-decomposition (related to SVD) to rank web pages based on link structure.
• 👩‍💼 Facial Recognition (Eigenfaces):SVD helps decompose faces into key features that aid recognition.
• 🎵 Spotify & Music Analysis:Audio data is decomposed to extract rhythm, tone, and genre components.
• 🧬 Bioinformatics:SVD simplifies massive gene expression matrices for faster analysis.

SVD: A Superpower for AI

SVD isn't just a math trick—it's a lens that reveals the essential features in images, text, audio, and beyond. By mastering it, you're learning to see and manipulate data with deeper clarity.

🚩 TL;DR

• SVD breaks a matrix into its most fundamental parts.
• We can build SVD manually with SageMath, step by step.
• Applications include image compression, search ranking, recommender systems, and bioinformatics.
• Use built-in functions for scale, but understand the mechanics for power.

🧩 What’s Next?

In the next blog, we’ll dive deeper into real image applications of SVD, including:

• 📉 Compression: How to compress images using just a few singular values
• 🧽 Denoising: How to remove noise while preserving image quality

We’ll also explore the rank-one decomposition: \[ A = \sigma_1 u_1 v_1^T + \sigma_2 u_2 v_2^T + \dots + \sigma_r u_r v_r^T \] This lets us see how images can be reconstructed layer by layer — each term adding more detail.

🧠 Final Thought: The Power of Perspective

By viewing data through the lens of SVD, we gain clarity, control, and creativity. Whether it’s unlocking a matrix or compressing an image, SVD helps us see the invisible.

See you in the next chapter — where numbers become pictures, and pictures become stories.

💡 Try It Yourself!

We've included an Active SageMath cell below. You can copy and run the code, or even modify the matrix values and explore how the results change in real-time. It's a hands-on way to deepen your understanding of SVD!