PyTorch linalg.svd() method computes the singular value decomposition (SVD) of a matrix. 2D tensors are matrices in PyTorch. This method supports both real and complex-valued matrices (float, double, cfloat, and cdouble dtypes). It takes input a matrix or a batch of matrices and returns decomposition as a named tuple (U, S, VT). I have listed some properties of U, S, and VT as below-
- If input matrix A is a real-valued matrix, both U and VT are orthogonal.
- If input matrix A is a complex-valued matrix, U and VT are unitary.
- S is always real-valued for both real or complex-valued input matrix A. The singular values (elements of S) are arranged in descending order.
Following is the syntax-
Syntax: torch.linalg.svd(A, full_matrices=True, *, out=None)
Parameters:
- A: the input matrix (tensor or batch of tensors).
- full_matrices: an optional boolean value. If True full SVD is computed. If False, reduced SVD is computed.
- out: the output tuple of three tensors.
Return: It returns three tensors as a named tuple (U, S, VT).
Let’s understand the torch.linalg.svd() method with the help of some Python program examples.
Example 1:
In this example, we compute the full singular value decomposition of a real-valued matrix using torch.linalg.svd(). Here, the U and VT matrices are square matrices, and the size of S is min(2,3). Matrices U and VT are orthogonal. You can check the orthogonality of these matrices using U@U.t() and VT@VT.t().
Python3
# Python program to demonstrate torch.linalg.svd() # method for a real-valued matrix # importing torch import torch # creating a real-valued matrix (2-D tensor) Mat = torch.tensor([[1.,2.,3.], [3.,-2,-7]]) # printing the matrix print("Matrix:\n", Mat) # computing SVD of the matrix Mat U, S, VT = torch.linalg.svd(Mat) # printing U, S, and Vh print("U:\n",U) print("Singular Values (S):\n",S) print("VT:\n",VT) print("Shape of U, S and VT:\n", U.shape, S.shape, VT.shape) |
Output:
Matrix:
tensor([[ 1., 2., 3.],
[ 3., -2., -7.]])
U:
tensor([[-0.3625, -0.9320],
[ 0.9320, -0.3625]])
Singular Values (S):
tensor([8.3999, 2.3329])
VT:
tensor([[ 0.2897, -0.3082, -0.9061],
[-0.8657, -0.4882, -0.1107],
[ 0.4082, -0.8165, 0.4082]])
Shape of U, S and VT:
torch.Size([2, 2]) torch.Size([2]) torch.Size([3, 3])
Example 2:
In the example below we compute the reduced singular value decomposition of a real-valued matrix using torch.linalg.svd(). Here, the reduced SVD is computed. Notice that U is square and VT is not, and the size of S is min(3,4). As the input matrix is real-valued matrices U and VT should be orthogonal. You can check the orthogonality of these matrices using U@U.t() and VT@VT.t(). What if the input matrix Mat in the above code is a 4×3 matrix? Yes!!!. VT is square and U is not.
Python3
# Python program to demonstrate torch.linalg.svd() # method for a real-valued matrix with # full_matrices=False # importing torch import torch # creating a real-valued matrix (2-D tensor) Mat = torch.rand(3, 4) # printing the matrix print("Matrix:\n", Mat) # computing SVD of the matrix Mat U, S, VT = torch.linalg.svd(Mat, full_matrices=False) # printing U, S, and Vh print("U:\n", U) print("Singular Values (S):\n", S) print("VT:\n", VT) print("Shape of U, S and VT:\n", U.shape, S.shape, VT.shape) |
Output:
Please note that as we are creating an input matrix using a random number generator, you may get a different matrix from the output-
Matrix:
tensor([[0.5280, 0.3108, 0.1215, 0.8151],
[0.9640, 0.4199, 0.3913, 0.2239],
[0.7886, 0.0702, 0.7123, 0.6120]])
U:
tensor([[-0.4984, -0.7911, -0.3546],
[-0.5810, 0.6083, -0.5407],
[-0.6435, 0.0634, 0.7628]])
Singular Values (S):
tensor([1.8411, 0.5512, 0.4225])
VT:
tensor([[-0.7227, -0.2412, -0.4053, -0.5052],
[ 0.3969, 0.0254, 0.3395, -0.8524],
[-0.2531, -0.6715, 0.6834, 0.1343]])
Shape of U, S and VT:
torch.Size([3, 3]) torch.Size([3]) torch.Size([3, 4])
Example 3:
In the example below we compute the full SVD of a complex-valued matrix. Notice that the S is real-valued and U and VT are complex-valued.
Python3
# Python program to demonstrate torch.linalg.svd() # method for a complex-valued matrix # importing torch import torch # creating a complex-valued matrix (2-D tensor) Mat = torch.randn(3, 2, dtype=torch.cfloat) # printing the matrix print("Matrix:\n", Mat) # computing SVD of the matrix Mat U, S, VT = torch.linalg.svd(Mat) # printing U, S, and Vh print("U:\n", U) print("Singular Values (S):\n", S) print("VT:\n", VT) print("Shape of U, S and VT:\n", U.shape, S.shape, VT.shape) |
Output:
Please note that as we are creating an input matrix using a random number generator, you may get a different matrix from the output below
Matrix:
tensor([[-0.4095-0.8878j, 0.3983+0.5446j],
[-1.3408+1.1268j, 0.3193+0.9775j],
[ 0.8876+0.6970j, -0.9217-0.5416j]])
U:
tensor([[-0.2687-0.3263j, -0.3146+0.2721j, 0.7890+0.1605j],
[-0.6525+0.3324j, -0.1834-0.6499j, 0.0518+0.0713j],
[ 0.5008+0.1853j, 0.3777-0.4778j, 0.5800-0.0863j]])
Singular Values (S):
tensor([2.4929, 1.3183])
VT:
tensor([[ 0.8916+0.0000j, -0.2928-0.3453j],
[-0.4527+0.0000j, -0.5767-0.6800j]])
Shape of U, S and VT:
torch.Size([3, 3]) torch.Size([2]) torch.Size([2, 2])
