Example of convex activation function

Question

For a particular task, I need a convex activation function with the following properties:

f''(x) > 0
0 <=    f(x) <=  1
f(x) is monotonic
f(x) is not "exploding" i.e. avoiding functions such as f(x) = x²

The only example I have in mind for this is the softplus activation function. Would you have anything else in mind?

Ben Reiniger · Answer

There is no such function.

A convex function defined on all of $mathbb{R}$ with bounded range is constant.

One Answer