Ref | Activation Function | Output Range | Equation | Â |
---|---|---|---|---|
[121] | Relu | 0 to ( +) Values | \({f\left(x\right)}_{Relu}\text{= Max(0, x)}\) | (7) |
[126] | Sigmoid | 0 to 1 | \(\updelta \left({\text{x}}\right)=\frac{1}{1+{{\text{e}}}^{-{\text{x}}}}\) | (8) |
[127] | Tanh | (-) to ( +) Values | \(Tanh(x)=\frac{\left({e}^{x}-{e}^{-x}\right)}{\left({e}^{x}+{e}^{-x}\right)}\) | (9) |