## Tuesday, January 4, 2011

### Analysis of the Logistic Funtion

In my post on logistic regression and maximum likelihood, I note the following in relation to the logit model:

Ln ((Di /(1 – Di)) = βX      [equation  1a]

Di = probability y = 1 = e / ( 1 + e     [equation 1b ]

Where : Di / (1 – Di) = ’odds ratio’ or ’odds’

E[y|X] = Prob(y = 1|X) = p = e / (1 + e )    [equation 1c]

In other words, the probability that y = 1 can be represented by the logistic function:

e / ( 1 + e        [ equation 1d]  or substituting  t  for XB:

et / ( 1 + et ) [equation 1e]

Often, we will see the logistic function represented as:

1/ ( 1 + e-t )     [equation 2]

Algebraically, one could show that these are equivalent, or alternatively, you could evaluate the functions numerically and graphically using R.

Graphing the formulation in equations 1b-1e or   et / ( 1 + et )   in R gives the following:

{code}

# define t-values

t  <-  seq(-4, 4, length=100)

# sigmoid/logistic = exp(t)/(1 + exp(t))= probability y =1

y <- exp(t)/(1 + exp(t))

plot(t, y, type="l", lty=2, xlab="t value",
ylab="Density", main="Sigmoid/Logistic Distribution exp(t)/(1 + exp(t))")

Graphing 1/ ( 1 + e-t )  gives the following:

{code}

# sigmoid/logistic = 1/(1 + exp(-t))= probability y =1

y <- 1/(1 + exp(-t))
plot(t, y, type="l", lty=2, xlab="t value",
ylab="Density", main="Sigmoid/Logistic Distribution 1/(1 + exp(-t))")

You can also show, numerically, that both functions give the same value evaluated at the same level of t:

t = 4  ( or XB = 4)

e4 / ( 1 + e4 )    = 0.9820138

1 / ( 1 + e-4 )  =  0.9820138

{ code}

t <-4

exp(t)/(1+ exp(t))

1/(1+exp(-t))

I also defined the likelihood function as follows:

L(β) = ∏f(y, β) = ∏ e / (1 + e ) ∏ 1/(1 + e)      [equation 3]

Which is (probability y=1)*(probability y = 0) or the joint density of y.

We've already worked with probability y = 1;  et / ( 1 + et ). If 'p' is the probability y = 1, then 1-p would give the probability that y = 0. This would be 1 - [et / ( 1 + et )]. This turns out to be the term in the second product above in equation 3.

(prob y =0) =  1/ ( 1 + et )     [equation 4]

Note, this is different from equation 2 in that the 't' is positive and not negative. We can check that these equations give the correct results numerically as follows:

For t = 3, p(y = 1) :

e4 / ( 1 + e4 )    = 0.9820138
p(y = 0) = 1/ ( 1 + e4 )  = 0.01798621

Note, the simpler calculation (1-p) gives us the same result: 1-0.9820138 = 0.0179862

{code}

t <-4

1/(1+exp(t))
This can be graphed as well:

{code}

y0 <- 1/(1 + exp(t))
plot(t, y0, type="l", lty=2, xlab="t value",
ylab="Density", main="Sigmoid/Logistic Distribution 1/(1 + exp(t))")