Estimating CES utility (not production) function parameters

This answer closely follows the logic of estimation of translog cost function presented in Section 4.7 of Fumio Hayashi's "Econometrics".

Define for convenience the CES aggregate price index $P:=(sum_i alpha_i^sigma p_i^{1-sigma})^frac{1}{1-sigma}$. Then the Marshallian demand system in log form can be written as a system of linear equations

$$ begin{cases} ln x_1 &= ln alpha_1^sigma -sigmaln p_1 - (1-sigma)ln P + ln M ... ln x_n &= ln alpha_n^sigma -sigmaln p_n - (1-sigma)ln P + ln M. end{cases}$$

Note that we are dealing with a multiple regression model. If we estimate these equations separately, data will necessarily make estimates of $sigma$ vary depending on the equation we estimate, therefore, we have to impose the common coefficient restriction from the start.

Further, a regression model necessarily needs error terms:

$$ begin{cases} ln x_1 &= ln alpha_1^sigma -sigmaln p_1 - (1-sigma)ln P + ln M + epsilon_1 ... ln x_n &= ln alpha_n^sigma -sigmaln p_n - (1-sigma)ln P + ln M + epsilon_n. end{cases}$$

The error terms $epsilon_i$ must have an economic interpretation for us to understand the statistical properties of chosen estimators. In other words, the uncertainty expressed in $epsilon_i$ must be attributed either to $alpha_i$ or to $sigma$ (assuming $x_i,p_i,M$ are free of measurement/observation errors).

The most accessible and common interpretation goes as follows :

In reality the weights $tilde alpha_i$ are not constant across individuals, but may fluctuate a little around the population average $alpha_i$, with $epsilon_isimmathcal{N}(0,sigma_epsilon^2)$ being a proxy for this noise: $$ tilde alpha_i = exp(epsilon_i/sigma) alpha_i. $$

Replacing $alpha_i$ by $tilde alpha_i$ in the initial Marshallian demand system and taking logarithms gives exactly the stated regression equation system.

Remark: relating $epsilon$ to the cross-sectional differences in $sigma$ makes things more complicated, as in this case we get heteroskedasticity of the error term in the regression (after taking logarithms, the error term is necessarily multiplied by some function of $p_i$).

Now, another problem is that without knowledge of $alpha_i,sigma$, $P$ is not really observed. This problem is solved by subtracting equation $j$ from equation $j$, as suggested in the answer by @Bertrand. We get a smaller system of equations ($n(n-1)/2$ against $n$ in the beginning):

$$ ln frac{x_i}{x_j} = sigma ln frac{alpha_i}{alpha_j} -sigmaln frac{p_i}{p_j} + epsilon_i - epsilon_j,quad forall iin{1,..n}, jin{1,..n}setminus {i}$$

Now this new multiple regression model has two new restrictions besides the common coefficient inherited from the original one:

1. correlated error terms, as $mathbb{Cov}(epsilon_i-epsilon_j,epsilon_j-epsilon_k) = -mathbb{V}(epsilon_j)$,
2. consistency restiction on pair effects: $ln{alpha_i/alpha_j}+ln{alpha_j/alpha_k} = ln{alpha_i/alpha_k},quad forall ineq jneq k$

Correlation in the error terms is not a problem for consistency, but it certainly affects your estimate of the covariances and confidence intervals.

Pair effect consistency on the other hand allows us to reduce the number of coefficients to estimate at the model formulation stage.

In vector form our multivariate regression can be represented as

$$ y_t = Z_t delta + u_t $$

where (for $n=3$ without loss of generality)

$$ y_t = begin{bmatrix} ln(frac{x_{1t}}{x_{2t}}) ln(frac{x_{1t}}{x_{3t}}) ln(frac{x_{2t}}{x_{3t}})end{bmatrix},quad Z_t = begin{bmatrix} 1 & 0 & &ln(frac{p_{1t}}{p_{2t}}) 0 & 1 & & ln(frac{p_{1t}}{p_{3t}}) -1 & 1 & & ln(frac{p_{2t}}{p_{3t}}) end{bmatrix}, quad delta = begin{bmatrix} sigma ln frac{alpha_1}{alpha_2} sigma ln frac{alpha_1}{alpha_3} sigma end{bmatrix}, u_t = begin{bmatrix}epsilon_1-epsilon_2 epsilon_1-epsilon_3 epsilon_2-epsilon_3end{bmatrix}.$$

The above system yeilds easily to the random-effects estimation (treated extensively in Section 4.6 of Hayashi).

The estimate of CES elasticity of substitution $hat sigma$ is obtained directly, whereas $hatalpha_i$ can be pinned down with some simple normalizaton (e.g. $sum_i alpha_i = 1$).

In this comment I simply show that

Under certain assumptions the problem is not an estimation problem, there is an exact solution for $sigma$ and a solution for the structural errors $alpha_i$ up to a normalisation.

The demand system considered is described by the Marshall demand function

$$x_k^star(p,M) = left(frac{alpha_j}{p_j}right)^{sigma}frac{M}{sum_j p_j^{1-sigma}alpha_j^sigma},$$

which implies that the expenditure shares are given as

$$s_k(p,I) := frac{p_k x_k^star(p,I)}{sum_j p_j x_j^star(p,I)} = frac{p_k^{1-sigma}alpha_k^sigma}{sum_j p_j^{1-sigma}alpha_j^sigma}.$$

I then define $K(p,alpha) := sum_j p_j^{1-sigma}alpha_j^sigma$ where $p = (p_1,...,p_N)$ and $alpha := (alpha_1,...,alpha_N)$ and log linearize expenditure shares to get

$$(1) log s_k = (1-sigma) p_k + sigma log alpha_k + K(p,alpha)$$

Single cross section of aggregate data: Given data on ${s_k,p_k}_{k=1}^N$ this equation can be estimated using OLS under the assumption that $mathbb E[p_k,ln alpha_k] =0$.

Consider now the situation where two time periods are observed $t=1,2$ and prices are assumed to show some variation while it is assumed $alpha$ the comsumer demand parameters are time constant

$$(1') log s_{kt} = (1-sigma) p_{kt} + sigma log alpha_k + K_t(p_t,alpha),$$

where I have added time index to $K$ because the price-vector is varying across time. It is clear that this is a standard two-way fixed effects regression with no good-time specific error term. Using methods of demeaning know from fixed effects estimation methodology one can find the exact solution for $(1-sigma)$.

First take averages over goods to get

$$frac{1}{N}sum_k log s_{kt} = (1-sigma) frac{1}{N}sum_kp_{kt} + frac{1}{N}sum_k sigma log alpha_k + K_t(p_t,alpha),$$

substract from equation (1) to get

$$(2) ds_{kt} = (1-sigma) dp_{kt} + sigma dalpha_k,$$ where the $d$ in from of the variable simply signifies it has been demeaned in log-version so $ds_{kt} := log s_{kt} - frac{1}{N}sum_k log s_{kt}$.

Second take first differences over time to get

$$(3) ds_{k2} - ds_{k1} = (1-sigma) [dp_{k2} - dp_{k1}]$$

such that

$$(4) forall k: (1-sigma) = frac{ds_{k2} - ds_{k1}}{dp_{k2} - dp_{k1}}$$

having found $sigma$ one can find $alpha_i$'s using equation (2). This means that one has to solve for $dalpha_k = alpha_k$ under normalisation that $frac{1}{N}sum_k log alpha_k = 0$. So the average of the structural errors cannot be recovered.

Under the assumption that $alpha_k$'s are time invariant one with data ${p_{jt},s_{jt}}_{j=1,...N, t=1,2}$ recover $sigma$ using exactly using (4) and the structural errors $alpha_k$ up to a single normalization using (2).

To illustrate the calculations I provide a small simulation:

#set.seed(1)
N <- 50
M <- 100000
sigma <- 4

# Simulate structural errors assumed time-constant
phi <- 2*runif(N)+2
phi <- phi/((prod(phi))^(1/N))

# Simulate prices   
p1 <- 2*runif(N)+2
p2 <- 2*runif(N)+2

s1 <- (p1/phi)^(1-sigma)
s1 <- s1/sum(s1)
E1 <- M*s1

s2 <- (p2/phi)^(1-sigma)
s2 <- s2/sum(s2)
E2 <- M*s2

x1 <- E1/p1
x2 <- E2/p2

library(data.table)
dt <- data.table(good=rep(1:N,2),time=rep(c(1,2),each=N),price=c(p1,p2),x=c(x1,x2))
dt[,E:=sum(price*x),by=time]
dt[,share:=price*x/E]

dt[,ds:=log(share)-mean(log(share)),by=time]
dt[,dp:=log(price)-mean(log(price)),by=time]
1-coef(lm(ds~dp+as.factor(good),data=dt))[2]
1-coef(lm(log(share)~log(price)+as.factor(good)+as.factor(time),data=dt))[2]
sigma
        
    
# Or from a single instead of all N 
temp <- as.matrix(dt[good==2,.(ds,dp)])
1-(temp[2,1] - temp[1,1]) / (temp[2,2]-temp[1,2])

The calculations in this post are based on Stephen J. Redding and David E. Weinstein paper A UNIFIED APPROACH TO ESTIMATING DEMAND AND WELFARE.

These other answers seem to be citing some methods I haven't quite heard about yet however they all touch upon the idea developed by Czech economist named Jan Kmenta which has come to be known as the Kmenta approximation (an in depth explanation as well as detailed derivation can be found in the documentation of the R package,MicEconCES.

A general form of a CES utility function is:

$$u(x_1,...,x_n)=gammaleft(sum_i delta_i x_i^{-rho}right)^frac{v}{rho}$$

To estimate this we have: $$ln y=alpha_0+sum_ialpha_i ln x_i+frac{1}{2}sum_isum_j beta_{ij}ln x_i ln x_j$$ where $y$ is our index of utility (note that we can recover an organic index of utility through estimating an expenditure system and using our principles of duality).

To obtain the parameters of the CES function we have the following forms: $$gamma=exp(alpha_0)$$ $$v=sum_ialpha _i$$ $$delta_i=frac{alpha_i}{sum_i alpha_i}$$ $$rho=frac{beta_{ij}}{alpha_i alpha_j}sum_i alpha_i forall i,j$$

I'd heavily recommend using the R package for any empirical work you may be doing. Hope this helps.