Topological Data Analysis of Large Language Models

Image
  Topological Data Analysis of Large Language Models: A Persistent-Homology Framework for Neural Representation Geometry Abstract Large Language Models (LLMs) have demonstrated remarkable capabilities across a broad spectrum of natural language processing tasks, yet the internal mechanisms governing their representations remain fundamentally opaque. We propose a rigorous methodological framework utilizing Topological Data Analysis (TDA), specifically persistent homology, to characterize the hidden geometric and topological structures of these neural activations. Rather than presenting experimental findings, this paper serves as a comprehensive methodology proposal designed to transition the analysis of LLM embeddings from heuristic geometric approximations to formalized topological invariants. By treating the outputs of self-attention heads and feed-forward networks as dynamic metric spaces, we construct Vietoris-Rips filtrations to trace the birth, persistence, and death of to...

Navigating SageMath’s Interface: Deep Dive with Examples

Meta Description: 

Explore more examples to master the SageMath interface. Dive deeper into calculations, graphing, interactive features, and customizations in this advanced walkthrough.

🔄 A Quick Recap

Previously, we explored how to launch SageMath, navigate its interface, and perform basic calculations. In this post, we’ll expand our skills with practical examples to help you unlock the full potential of SageMath.

🛠️ Beyond Basics: Advanced Features in the Toolbar

Highlight the additional functionalities of SageMath's interface:

Let's Practice more Examples:

1.    Write python codes to input radius of a sphere and print its surface area and volume.

# Input radius from user

r = float(input("Enter the radius of the sphere: "))


# Surface area of a sphere = 4 * pi * r^2

surface_area = 4 * pi * r^2


# Volume of a sphere = (4/3) * pi * r^3

volume = (4/3) * pi * r^3


Output: 

💬 Understanding the Output

  • Surface Area gives the amount of outer space the sphere occupies.

  • Volume tells us how much space is enclosed inside the sphere.

✅ Try entering values like r = 1, r = 5, or even a decimal like r = 2.5 and observe how the results scale.

🎯 Challenge:

Try modifying the code to calculate the area and volume for multiple spheres. Can you store the values in a list or a dictionary?


2.    Input a positive integer n and integer k<=n. Verify the following properties

# Input values

n = int(input("Enter a positive integer n: "))

k = int(input("Enter an integer k (k <= n): "))


# Check for valid input

if n < 0 or k < 0 or k > n:

    print("Invalid input. Please ensure n >= 0 and 0 <= k <= n.")

else:

    # Define binomial coefficient function

    C = binomial


    # Identity i: nCk = nC(n−k)

    lhs1 = C(n, k)

    rhs1 = C(n, n - k)

    print("\ni. nCk = nC(n-k)")

    print(f"{lhs1} = {rhs1} ->", "Verified ✅" if lhs1 == rhs1 else "Not Verified ❌")


    # Identity ii: k * nCk = n * (n-1)C(k-1)

    lhs2 = k * C(n, k)

    rhs2 = n * C(n - 1, k - 1)

    print("\nii. k × nCk = n × (n−1)C(k−1)")

    print(f"{lhs2} = {rhs2} ->", "Verified ✅" if lhs2 == rhs2 else "Not Verified ❌")


    # Identity iii: nC(k−1) + nCk = (n+1)Ck

    lhs3 = C(n, k - 1) + C(n, k)

    rhs3 = C(n + 1, k)

    print("\niii. nC(k−1) + nCk = (n+1)Ck")

    print(f"{lhs3} = {rhs3} ->", "Verified ✅" if lhs3 == rhs3 else "Not Verified ❌")


Output :

💬 Understanding the Output

These are combinatorial identities often used in probability and algebra:

  • The first identity shows symmetry in combinations.

  • The second connects combinations and multiplication.

  • The third is Pascal's Rule, foundational for Pascal’s Triangle.

📸 Visual : Pascal’s Triangle highlighting how nC(k-1) + nCk = (n+1)Ck.


🎯 Challenge:

Can you build Pascal’s Triangle up to n = 10 using a loop? Try printing it in a triangular form!

3.    Input a complex number z and explore some of the properties of z.


SageMath code that takes a complex number

z=a+biz = a + bi  as input and explores the following common properties:

🔍 Explored Properties:

  1. Real part

  2. Imaginary part

  3. Conjugate

  4. Modulus (absolute value)

  5. Argument (angle in radians)

  6. Polar form

  7.   

# Input complex number as string

z_input = input("Enter a complex number (e.g. 3+4i): ")


# Replace 'i' with 'I' to make it SageMath-friendly

z_input = z_input.replace('i', 'I')


# Evaluate the input string as a SageMath expression

z = sage_eval(z_input)


# Display the entered complex number

print(f"\nComplex number z = {z}")


# Real and Imaginary parts

print("Real part of z:", z.real())

print("Imaginary part of z:", z.imag())


# Conjugate

print("Conjugate of z:", z.conjugate())


# Modulus

print("Modulus (|z|):", abs(z))


# Argument (angle in radians)

print("Argument (arg(z)):", arg(z))


# Polar form

r = abs(z)

theta = arg(z)

print(f"Polar form: {r} * (cos({theta}) + i*sin({theta}))")


# Other computations

print("z squared (z^2):", z^2)

print("Reciprocal (1/z):", 1/z)

print("z * conjugate(z):", z * z.conjugate())


Output: 

Enter a complex number (e.g. 3+4i): 6+i

Complex number z = I + 6
Real part of z: 6
Imaginary part of z: 1
Conjugate of z: -I + 6
Modulus (|z|): sqrt(37)
Argument (arg(z)): arctan(1/6)
Polar form: sqrt(37) * (cos(arctan(1/6)) + i*sin(arctan(1/6)))
z squared (z^2): 12*I + 35
Reciprocal (1/z): -1/37*I + 6/37
z * conjugate(z): 37

💬 Understanding the Output

  • Real & Imaginary Parts break down the number.

  • Conjugate reflects the number across the real axis.

  • Modulus gives the distance from origin (like length of a vector).

  • Argument is the angle from the real axis (in radians).

  • Polar form is useful in trigonometry and electrical engineering.

  • 1/z shows the multiplicative inverse.

  • z * conjugate(z) = square of the modulus.

📸 Visual :

A diagram of the complex plane with:

  • Point z plotted

  • Vector from origin

  • Angle (argument)

  • Conjugate as reflected point

🎯 Challenge:

Try plotting the modulus of z = a + bi for various values of a and b (e.g., -5 to 5) and create a 3D surface or contour plot. Share your plots in the comments!

📌 Wrapping Up

With these advanced examples and features, you’re no longer just navigating SageMath—you’re mastering it. Next up, we’ll Built-in Data Structures in Python in SageMath.

Comments

Popular posts from this blog

Heuristic Computation and the Discovery of Mersenne Primes

Understanding the Laplacian of 1/r and the Dirac Delta Function Mathematical Foundations & SageMath Insights

Neural Network Generalization in the Over-Parameterization Regime: Mechanisms, Benefits, and Limitations