PyTorch is an open-source machine learning library developed by Facebook. It is used for deep neural network and natural language processing purposes.
One of the many activation functions is the hyperbolic tangent function (also known as tanh) which is defined asĀ 
.
The hyperbolic tangent function outputs in the range (-1, 1), thus mapping strongly negative inputs to negative values. Unlike the sigmoid function, only near-zero values are mapped to near-zero outputs, and this solves the āvanishing gradientsā problem to some extent. The hyperbolic tangent function is differentiable at every point and its derivative comes out to beĀ 
. Since the expression involves the tanh function, its value can be reused to make the backward propagation faster.
Despite the lower chances of the network getting āstuckā when compared with the sigmoid function, the hyperbolic tangent function still suffers from āvanishing gradientsā. Rectified Linear Unit (ReLU) can be used to overcome this problem.
The function torch.tanh() provides support for the hyperbolic tangent function in PyTorch. It expects the input in radian form and the output is in the range [-ā, ā]. The input type is tensor and if the input contains more than one element, element-wise hyperbolic tangent is computed.
Ā
Syntax: torch.tanh(x, out=None)
Parameters:Ā
x: Input tensorĀ
name (optional): Output tensor
Return type: A tensor with the same type as that of x.Ā
Ā
Code #1:Ā
Python3
# Importing the PyTorch libraryimport torchĀ
# A constant tensor of size 6a = torch.FloatTensor([1.0, -0.5, 3.4, -2.1, 0.0, -6.5])print(a)Ā
# Applying the tanh function and# storing the result in 'b'b = torch.tanh(a)print(b) |
Output:Ā
1.0000 -0.5000 3.4000 -2.1000 0.0000 -6.5000 [torch.FloatTensor of size 6] 0.7616 -0.4621 0.9978 -0.9705 0.0000 -1.0000 [torch.FloatTensor of size 6]
Ā
Code #2: VisualizationĀ
Python3
# Importing the PyTorch libraryimport torchĀ
# Importing the NumPy libraryimport numpy as npĀ
# Importing the matplotlib.pyplot functionimport matplotlib.pyplot as pltĀ
# A vector of size 15 with values from -5 to 5a = np.linspace(-5, 5, 15)Ā
# Applying the hyperbolic tangent function and# storing the result in 'b'b = torch.tanh(torch.FloatTensor(a))Ā
print(b)Ā
# Plottingplt.plot(a, b.numpy(), color = 'red', marker = "o") plt.title("torch.tanh") plt.xlabel("X") plt.ylabel("Y") Ā
plt.show() |
Output:Ā
Ā
-0.9999 -0.9996 -0.9984 -0.9934 -0.9728 -0.8914 -0.6134 0.0000 0.6134 0.8914 0.9728 0.9934 0.9984 0.9996 0.9999 [torch.FloatTensor of size 15]
Ā
Ā
