Consider a random variable f(x|θ1….θm) and a sample x1…xn
Define the kth raw population moment as: μk’ =E(xk) which is some function of θ.This is referred to as the population moment condition and is often expressed as an ‘orthogonality’ condition:
μk’ - E(xk) = 0 or more compactly as E[Ψ (xi,θ)] = 0
The sample analog of μk’ =mk’ = (1/n) Σ xik .
The MM estimator is derived by equating the sample moments to the population moments and solving for the parameters of interest.
This gives us the sample analog to the orthogonality condition:
mk’ - μk’ = 0 or (1/n)Σ Ψ (xi’,θ) =0
Let e = y-xb and Ψ (x,θ) = x(y-xb)
Our moment condition is then: E(xe) = 0 (which is one of the familiar assumptions of OLS)
The sample analog:
1/n Σ x(y-xb) = 0
X’(Y-Xb) = 0 (matrix representation)
X’Y-X’Xb = 0
b = (X’X)-1X’Y
Generalized Method of Moments
The sample analog to the moment condition (1/n)Σ Ψ (xi’,θ) often represents a system of ‘q’ equations and ‘p’ parameters. If q =p then we have an identified system and can derive the MM estimator. If q>p then the system is overidentified and the following quadratic form is specified:
min[(1/n)Σ Ψ (xi’,θ)]’W[(1/n)Σ Ψ (xi’,θ)] where the matrix W is a weigthing matrix.
GMM estimates are derived from the above iteratively.
Jeff Thurk provides a very useful set of notes related to GMM on his website: https://webspace.utexas.edu/jmt597/www/teaching.htm
Econometric Analysis, William H. Greene . 1990.