A road between the floor and the ceiling
The easiest way to think about the sigmoid is as a road. Below is a floor at height 0, above is a ceiling at height 1. The road runs left to right: it starts almost on the floor, then rises smoothly and settles almost under the ceiling.
The key word is “almost”. The road comes arbitrarily close to the floor and the ceiling but never touches them. A line that a curve approaches forever without ever reaching is called an asymptote. The sigmoid has two, both horizontal: y = 0 as x → −∞ and y = 1 as x → +∞.
The numbers show just how small that gap becomes.
| x | σ(x) | 1 − σ(x) |
|---|---|---|
| 2 | 0.8808 | 0.12 |
| 5 | 0.9933 | 6.7 × 10−3 |
| 10 | 0.99995 | 4.5 × 10−5 |
| 20 | 0.999999998 | 2.1 × 10−9 |
| 100 | 1 − 3.7 × 10−44 | 3.7 × 10−44 |
At x = 100 the distance to one is a number with forty-three zeros after the decimal point. But it is still not zero, and no x will ever make it zero. That “never” is the whole meaning of the word “asymptote”.
The steepest place
Now walk the road. On the left it is almost flat. Then the climb gets steeper and steeper, peaks exactly in the middle, at x = 0 and height 0.5, and levels off again.
The place where “getting steeper” turns into “getting flatter” is called the inflection point. Formally, the second derivative changes sign there: the curve is convex on the left and concave on the right. The sigmoid has exactly one such point, so the whole curve is a single smooth S-shaped move with no steps or plateaus in the middle.
The derivative
The derivative is remarkably convenient here. It is expressed through the function itself: σ′ = σ(1 − σ). So during backpropagation there is no need to recompute the exponential; the value the network already computed on the forward pass is enough.
Training needs only σ(x); e−x is never recomputed. The derivative is symmetric, σ′(x) = σ′(−x), and the area under it is one: it is the density of the logistic distribution.
What this means for a neural network
Inside the network, an output neuron receives an arbitrary number: −50, 3, 1000. Binary classification needs an answer as a probability between 0 and 1. The sigmoid takes any number and lays it onto the road between the floor and the ceiling.
The asymptotes give boundedness. The output is always strictly between 0 and 1, so it can be read as a probability. The same property is also the main drawback: on the flat parts, large inputs stop being distinguishable. σ(10) and σ(100) differ in the fifth decimal, and the derivative there is practically zero. When activations drift into the tails, no gradient passes through them and the weights stop learning. This is the vanishing gradient, the reason the sigmoid left the hidden layers of deep networks.
The inflection point gives the threshold. x = 0 and σ = 0.5 is the midpoint between “no” and “yes”. The derivative is largest there, so the network is most sensitive: a small shift in the input visibly changes the output. In logistic regression the decision boundary w·x + b = 0 maps exactly to this point. That is why 0.5 became the standard threshold.
A single inflection gives a single transition. The curve is monotonic with no intermediate shelves, so there is exactly one boundary between “no” and “yes”, not several.
In short
- Two horizontal asymptotes, 0 and 1: the output can always be read as a probability, but in the tails the function saturates and the gradient vanishes.
- One inflection point at (0, 0.5): maximum sensitivity, a natural decision threshold and a single boundary between classes.
- The derivative is expressed through the function itself, σ′ = σ(1 − σ), which makes backpropagation cheap.
- For comparison, tanh has the same properties but asymptotes at −1 and 1, an inflection at the origin and a slope of 1 instead of 0.25, so it trains faster in hidden layers. The sigmoid stayed where a probability is exactly what is needed: the output of a binary classifier.