Softplus

In mathematics and machine learning, the softplus function is

${\displaystyle f(x)=\ln(1+e^{x}).}$

The names softplus^[1][2] and SmoothReLU^[3] are used in machine learning.

It is a smooth approximation (in fact, an analytic function) to the ramp function, which is known as the rectifier or ReLU in machine learning. For large negative ${\displaystyle x}$ it is ${\displaystyle \log(1+e^{x})=\log(1+\epsilon )\gtrapprox \log 1=0}$, so just above 0, while for large positive ${\displaystyle x}$ it is ${\displaystyle \log(1+e^{x})\gtrapprox \log(e^{x})=x}$, so just above ${\displaystyle x}$.