Compressing multivariate functions with tree tensor networks

Introduction to the show: ident: Quantum Radio. Generated commentary on the latest quantum physics and condensed matter papers.

Kai: We're looking at the paper: "Compressing multivariate functions with tree tensor networks".

Mira: Here's the gist of the paper. The gist Tensor networks are increasingly being used as a numerical ansatz for continuum functions by “quantizing” the inputs into discrete binary digits, and this work demonstrates the power of more general tree tensor networks (TTNs) for this purpose.

Kai: Constructing Functions as Tree Tensor Networks The paper provides direct constructions of several elementary functions as generic tree tensor networks and interpolative constructions for more complicated functions via a generalization of the tensor cross interpolation algorithm.

Mira: Direct construction shows that certain elementary functions are factorizable as a product of separate functions for each bit, leading to rank-one tensor networks with local tensors T(i,j) = f(i,j)(xi,j) (Page 4).

Kai: Specifically, three classes of rank-one functions are mentioned: Constant functions where f(x) = c = Qn i=1 QL j=1 c 1 nL (Page 4), Exponential functions where f(x) = cek·x+a = Qn i=1 QL j=1 c 1 nL e ki xi,j2j +a with a,c ∈ C and k = (k1, k2, …, kn) ∈ C n (Page 4), and the Dirac delta function f(x) = δ(x−x˜) = Qn i=1 QL j=1 2δxi,j,x˜i,j where δn,m is the Kronecker delta function and x˜i,j is the setting of digit (i, j) in the binary decomposition of x˜ (Page 4).

Title and authors: Kai: So to wrap up this paper, "Compressing multivariate functions with tree tensor networks," the main point is that we need more than just the standard tensor train method when dealing with multi-dimensional data because it's too inefficient for capturing those complex correlations.

Mira: They are proposing tree tensor networks as a more versatile format that handles these higher dimensions better by allowing for structured connections between the tensors.

Lev: What I see here is they are moving toward a formalism where the structure of the network itself helps solve the compression problem, not just relying on sequential ordering like in tensor trains.

Kai: The authors demonstrate how to construct exact representations of certain elementary functions, like polynomials of degree d, on these general tree tensor networks with a bond dimension that doesn't depend on the specific network topology.

Mira: That's a strong point because it means you don't have to worry about the network shape being perfect if you just want to represent a simple function accurately.

Lev: If we can guarantee that, then the overhead of building the structure itself is less of a limiting factor when we are trying to find an efficient representation.

Kai: They also show how they can use tensor cross interpolation to create interpolative constructions for more complicated functions through a generalization of that algorithm.

Mira: So this isn't just about finding one good function, it’s about having a method to dynamically learn and interpolate the space of possible functions.

Lev: That dynamic learning aspect is what makes it interesting for practical use, especially when we think about real-world data that might have complex dependencies.

Kai: These methods allow them to represent things like exponential functions, where the function form itself has several parameters a and c, as a tensor network with specific indices.

Mira: And they show that they can build even more complex things, like hyperbolic functions, by combining the exponential definition with their addition rules for tensor networks.

Lev: Building things piece by piece like that is exactly how we approach many quantum algorithms where you compose smaller operations into larger ones.

The paper's summary: Kai: The core of the paper is demonstrating that tree tensor networks are a more powerful tool for quantifying inputs into discrete binary digits than just one-dimensional tensor trains or matrix product states.

Mira: They show that by using these general tree tensor networks, you can achieve this quantization with much greater flexibility, which is what they call the power of tree tensor networks (TTNs) for this purpose.

Lev: So they are showing that TTNs offer a more expressive way to map continuous variables onto discrete binary strings than the limited rank-three tensors in a tensor train format.

Kai: They provide direct constructions for several elementary functions, including constants, exponentials, and the Dirac delta function using these tree tensor networks.

Mira: And they also show that they can construct polynomials of degree d on any tree tensor network where the bond dimension is set to d plus one.

Lev: That means we't don't have to worry about the rank being too small for a specific function, as long as we use a TTN structure appropriately.

Kai: They also detail how they handle addition and multiplication of these networks, which defines the space of functions you can realize with this new network format.

Mira: Addition just adds up the bond dimensions, chi = chi(one) + chi(two), and multiplication multiplies them down to chi = chi(one) chi(two) <ref:2410.03572#pg1>.

Lev: That algebraic definition of the space is crucial for understanding how complex functions are built from simpler ones.

Kai: They then introduce the tensor cross interpolation algorithm as a way to dynamically adapt the network ranks while moving through this space.

Mira: This allows them to actively minimize the infinity norm error, x f(x) - T(x), by adjusting neighboring pairs of local tensors.

Lev: It’s an active learning approach that lets you "learn general, multi-dimensional target functions f(x) into a TTN format" using pivots determined by the algorithm.

The paper's improvements: Kai: One major improvement is showing that for multivariate functions, structured tree tensor networks offer a significantly more efficient ansatz than the commonly used tensor train.

Mira: And in three dimensions, they found that coupled binary tree tensor networks and comb tree tensor networks are both better ansatzes for the function at hand.

Lev: The comb tree network compresses much more effectively under truncation of internal bonds than tensor trains, which is a key finding for practical applications.

Kai: For instance, the CTTN achieves an error of epsilon around ten-six with a bond dimension of fifteen, whereas tensor trains need thirty to reach that error.

Mira: That difference in rank shows how much better the structured network is at capturing correlations in higher dimensions.

Lev: But they also have to be careful because the paper flags that for real hardware, it's not just about achieving a good theoretical error on a computer; we have to consider the actual physical resources needed.

Kai: And they do address this by showing how the scaling of walltime with bond dimensions follows chi cubed and chi squared respectively for the comb tree and tensor trains.

Mira: That scaling difference in walltime is a concrete way to compare the two methods on physical hardware.

Lev: So, if you're building something, you have to weigh that faster theoretical compression against the practical cost of running it on real hardware.

Conclusion: Kai: To wrap up this paper "Compressing multivariate functions with tree tensor networks," the main implication is that for problems where the solution is a multivariate function with significant inter-dimensional correlations, we should move toward structured tree tensor networks instead of the standard tensor train ansatz.

Mira: The comb tree network seems to be a very effective ansatz for compressing these kinds of functions going forward.

Lev: It sounds like they are pushing us toward using these more structured networks when we think about problems that aren't just simple one-dimensional functions anymore.

Kai: We can use the methods here to build new, more efficient numerical algorithms by leveraging the generality of tensor-network based numerical methods.

Mira: The breadth of what this paper shows is that we can build a wide range of TTN-based numerical algorithms that broaden the potential for these methods significantly.

Lev: It gives us a lot to work with because the tools and methods described in this paper are all contained within the ITensorNumericalAnalysis.jl library, which makes it accessible for building things.