Fundamental Tensor and Related Operations

---

Let $\mathcal{X}\in\mathbb{R}^{I_i\times\cdots\times I_N}$ be a multidimensional array, i.e., a tensor.

Tensor Multiplication

• n-mode product: \(\left(\mathcal{X}\times_n U\right)_{i_1,\dots,i_{n-1},j,i_{n+1},\dots i_N} := \sum_{i_n=1}^{I_n}X_{i_1,\dots,i_N}u_{j,i_n}\) Or, equivalently \(\mathcal{Y} = \mathcal{X}\times_n U\iff Y_{(n)}=UX_{(n)},\) where $X_{(n)}$ is the mode-n matricization of $\mathcal{X}$.

  • Commutativity: $\mathcal{X}\times_mA\times_nB = \mathcal{X}\times_nB\times_mA$
  • Associative: $\mathcal{X}\times_mA\times_mB = \mathcal{X}\times_m(BA)$

Kronecker Product

• $(A\otimes B)\otimes(C\otimes D)=AC\otimes BD$
• $(A\otimes B)^{-1} = A^{-1}\otimes B^{-1}$

Kronecker & Tensor Mode Product

TFAE:

• $\mathcal{Y} = \mathcal{X}\times_1 A^{(1)}\cdots\times_N A^{(N)}$
• $\text{vec}(\mathcal{Y}) = (A^{(N)}\otimes\cdots\otimes A^{(1)})\text{vec}(\mathcal{X})$
• $Y_{(n)} = A^{(n)}X_{(n)}(A^{(N)}\otimes\cdots\otimes A^{(n+1)}\otimes A^{(n-1)}\otimes\cdots\otimes A^{(1)})^\top$

Kronecker & Outer Product

TFAE:

• $\mathcal{X}=a_1\circ\cdots\circ a_N$
• $\text{vec}(\mathcal{X})=a_N\otimes\cdots\otimes a_1$
• $X_{(n)}=a_n(A^{(N)}\otimes\cdots\otimes A^{(n+1)}\otimes A^{(n-1)}\otimes\cdots\otimes A^{(1)})^\top$

vec Operator

• $\text{vec}(AXB) = (B^\top\otimes A)\text{vec}(X)$
  • Corollary: $\text{vec}(AB) = (I\otimes A)\text{vec}(B) = (B^\top\otimes I)\text{vec}(A)$
• $\text{tr}(ABC) = \text{vec}(A^\top)^\top(I\otimes B)\text{vec}(C)$
  • Corollary: $\text{tr}(AB) = \text{vec}(A^\top)^\top\text{vec}(B)$
• $\text{vec}(aa^\top) = a\otimes a$
• $\text{tr}(A^\top BCD^\top) = \text{vec}(A)^\top(D\otimes B)\text{vec}(C)$