Analysis of the Logistic Funtion

In my post on logistic regression and maximum likelihood, I note the following in relation to the logit model:

Ln ((D_i /(1 – D_i)) = βX      [equation  1a]

D_i = probability y = 1 = e^Xβ / ( 1 + e^Xβ )      [equation 1b ]

Where : D_i / (1 – D_i) = ’odds ratio’ or ’odds’

E[y|X] = Prob(y = 1|X) = p = e^Xβ / (1 + e^Xβ )    [equation 1c]

In other words, the probability that y = 1 can be represented by the logistic function: 

e^Xβ / ( 1 + e^Xβ )         [ equation 1d]  or substituting  t  for XB:

e^t / ( 1 + e^t ) [equation 1e]

Often, we will see the logistic function represented as:

1/ ( 1 + e^-t )     [equation 2]

Algebraically, one could show that these are equivalent, or alternatively, you could evaluate the functions numerically and graphically using R.
 
Graphing the formulation in equations 1b-1e or   e^t / ( 1 + e^t )   in R gives the following:

{code}

# define t-values

t  <-  seq(-4, 4, length=100)

# sigmoid/logistic = exp(t)/(1 + exp(t))= probability y =1

y <- exp(t)/(1 + exp(t))

plot(t, y, type="l", lty=2, xlab="t value",

  ylab="Density", main="Sigmoid/Logistic Distribution exp(t)/(1 + exp(t))")

Graphing 1/ ( 1 + e^-t )  gives the following:

{code}

# sigmoid/logistic = 1/(1 + exp(-t))= probability y =1

y <- 1/(1 + exp(-t))

plot(t, y, type="l", lty=2, xlab="t value",

  ylab="Density", main="Sigmoid/Logistic Distribution 1/(1 + exp(-t))")

You can also show, numerically, that both functions give the same value evaluated at the same level of t:

t = 4  ( or XB = 4)


e⁴ / ( 1 + e⁴ )    = 0.9820138


1 / ( 1 + e^-4 )  =  0.9820138

{ code}

t <-4

exp(t)/(1+ exp(t))

1/(1+exp(-t))

I also defined the likelihood function as follows:

L(β) = ∏f(y, β) = ∏ e^Xβ / (1 + e^Xβ ) ∏ 1/(1 + e^Xβ)      [equation 3]

Which is (probability y=1)*(probability y = 0) or the joint density of y.

We've already worked with probability y = 1;  e^t / ( 1 + e^t ). If 'p' is the probability y = 1, then 1-p would give the probability that y = 0. This would be 1 - [e^t / ( 1 + e^t )]. This turns out to be the term in the second product above in equation 3.

(prob y =0) =  1/ ( 1 + e^t )     [equation 4]

Note, this is different from equation 2 in that the 't' is positive and not negative. We can check that these equations give the correct results numerically as follows:

For t = 3, p(y = 1) :


e⁴ / ( 1 + e⁴ )    = 0.9820138
p(y = 0) = 1/ ( 1 + e⁴ )  = 0.01798621

Note, the simpler calculation (1-p) gives us the same result: 1-0.9820138 = 0.0179862

{code} 

t <-4

1/(1+exp(t))
This can be graphed as well:

{code}

y0 <- 1/(1 + exp(t))
plot(t, y0, type="l", lty=2, xlab="t value",
  ylab="Density", main="Sigmoid/Logistic Distribution 1/(1 + exp(t))")