Hi everyone,
I had another thread about this sort of problem, but received good advice about using the cmp command. The context is this. I am estimating a discrete-continuous choice model where individuals choose where to live and, given where they live, how much to work, consume, and "use" their house. Each location has a different amenity (pollution), which also affects their utility.
However, leisure, consumption, and housing are all endogenous, so I also instrument for each of them. cmp is extremely useful to allow for this possibility and still estimate a discrete choice - truly a remarkable command.
However, I got some problems when estimating it that I did not know how to diagnose completely. I am copying a subset of the output (not the estimates) below. But, I will direct attention towards some matrices being ill conditioned and the collinear regressors. However, all these variables work if I'm using reg3 -- there's no reason why they should be ill conditioned or collinear. The only thing I can think of is that it's an extremely tough problem to optimize. I haven't even added the fixed effects in yet...
I had another thread about this sort of problem, but received good advice about using the cmp command. The context is this. I am estimating a discrete-continuous choice model where individuals choose where to live and, given where they live, how much to work, consume, and "use" their house. Each location has a different amenity (pollution), which also affects their utility.
However, leisure, consumption, and housing are all endogenous, so I also instrument for each of them. cmp is extremely useful to allow for this possibility and still estimate a discrete choice - truly a remarkable command.
However, I got some problems when estimating it that I did not know how to diagnose completely. I am copying a subset of the output (not the estimates) below. But, I will direct attention towards some matrices being ill conditioned and the collinear regressors. However, all these variables work if I'm using reg3 -- there's no reason why they should be ill conditioned or collinear. The only thing I can think of is that it's an extremely tough problem to optimize. I haven't even added the fixed effects in yet...
cmp (lwage_hourly = lleisure lcons_nondur lpoll $aqX $aqstX) (lleisure=$ivweather) (lcons_nondur = $ivcons) (lhprice = lcons_house lcons_nondur lpoll $aqX $aqstX) (lcons_house = $ivhouse) (move = lleisure lcons_nondur lcons_house $aqX $aqstX) (location = lleisure lcons_nondur lcons_house $aqX $aqstX) [w=perwt],indicators($cmp_cont $cmp_cont $cmp_cont $cmp_cont $cmp_cont $cmp_probit $cmp_oprobit) cluster(county)
(sampling weights assumed)
Fitting individual models as starting point for full model fit.
Note: For programming reasons, these initial estimates may deviate from your specification.
For exact fits of each equation alone, run cmp separately on each.
-------------------------------------------------------------------------------
Warning: regressor matrix for lwage_hourly equation appears ill-conditioned. (Condition num
> ber = 1526747.4.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
----------------------------------------------------------------------------------
Warning: regressor matrix for lleisure equation appears ill-conditioned. (Condition number
> = 196065.85.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
--------------------------------------------------------------------------------------
Warning: regressor matrix for lcons_nondur equation appears ill-conditioned. (Condition num
> ber = 2253.5663.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
Source | SS df MS Number of obs =235581188
-------------+------------------------------ F( 34,235581153) = .
Model | 32507335.3 34 956098.096 Prob > F = 0.0000
Residual | 99403557.6235581153 .421950382 R-squared = 0.2464
-------------+------------------------------ Adj R-squared = 0.2464
Total | 131910893235581187 .559938145 Root MSE = .64958
-------------------------------------------------------------------------------
Warning: regressor matrix for lhprice equation appears ill-conditioned. (Condition number =
> 1263661.2.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
Source | SS df MS Number of obs =238163538
-------------+------------------------------ F( 1,238163536) =80511.68
Model | 10561.4387 1 10561.4387 Prob > F = 0.0000
Residual | 31242044.7238163536 .131178959 R-squared = 0.0003
-------------+------------------------------ Adj R-squared = 0.0003
Total | 31252606.2238163537 .131223304 Root MSE = .36219
------------------------------------------------------------------------------
lcons_house | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ltrantime | .0088839 .0000313 283.75 0.000 .0088225 .0089453
_cons | 9.577865 .0001006 9.5e+04 0.000 9.577668 9.578062
------------------------------------------------------------------------------
Iteration 0: log likelihood = -1.279e+08
Iteration 1: log likelihood = -1.004e+08
Iteration 2: log likelihood = -98963302
Iteration 3: log likelihood = -98953638
Iteration 4: log likelihood = -98953634
Probit regression Number of obs = 1947523
LR chi2(34) = 5.79e+07
Prob > chi2 = 0.0000
Log likelihood = -98953634 Pseudo R2 = 0.2263
-------------------------------------------------------------------------------
Warning: regressor matrix for moved equation appears ill-conditioned. (Condition number = 1
> 110405.6.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
Iteration 0: log likelihood = -2.679e+08
Iteration 1: log likelihood = -2.405e+08
Iteration 2: log likelihood = -2.392e+08
Iteration 3: log likelihood = -2.392e+08
Iteration 4: log likelihood = -2.392e+08
Note: 14 observations completely determined. Standard errors questionable.
Warning: regressor matrix for _cmp_y7 equation appears ill-conditioned. (Condition number =
> 1110405.6.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
Fitting full model.
cmp_lnL(): 3499 halton2() not found
<istmt>: - function returned error
Mata run-time error
Mata run-time error
(sampling weights assumed)
Fitting individual models as starting point for full model fit.
Note: For programming reasons, these initial estimates may deviate from your specification.
For exact fits of each equation alone, run cmp separately on each.
-------------------------------------------------------------------------------
Warning: regressor matrix for lwage_hourly equation appears ill-conditioned. (Condition num
> ber = 1526747.4.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
----------------------------------------------------------------------------------
Warning: regressor matrix for lleisure equation appears ill-conditioned. (Condition number
> = 196065.85.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
--------------------------------------------------------------------------------------
Warning: regressor matrix for lcons_nondur equation appears ill-conditioned. (Condition num
> ber = 2253.5663.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
Source | SS df MS Number of obs =235581188
-------------+------------------------------ F( 34,235581153) = .
Model | 32507335.3 34 956098.096 Prob > F = 0.0000
Residual | 99403557.6235581153 .421950382 R-squared = 0.2464
-------------+------------------------------ Adj R-squared = 0.2464
Total | 131910893235581187 .559938145 Root MSE = .64958
-------------------------------------------------------------------------------
Warning: regressor matrix for lhprice equation appears ill-conditioned. (Condition number =
> 1263661.2.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
Source | SS df MS Number of obs =238163538
-------------+------------------------------ F( 1,238163536) =80511.68
Model | 10561.4387 1 10561.4387 Prob > F = 0.0000
Residual | 31242044.7238163536 .131178959 R-squared = 0.0003
-------------+------------------------------ Adj R-squared = 0.0003
Total | 31252606.2238163537 .131223304 Root MSE = .36219
------------------------------------------------------------------------------
lcons_house | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ltrantime | .0088839 .0000313 283.75 0.000 .0088225 .0089453
_cons | 9.577865 .0001006 9.5e+04 0.000 9.577668 9.578062
------------------------------------------------------------------------------
Iteration 0: log likelihood = -1.279e+08
Iteration 1: log likelihood = -1.004e+08
Iteration 2: log likelihood = -98963302
Iteration 3: log likelihood = -98953638
Iteration 4: log likelihood = -98953634
Probit regression Number of obs = 1947523
LR chi2(34) = 5.79e+07
Prob > chi2 = 0.0000
Log likelihood = -98953634 Pseudo R2 = 0.2263
-------------------------------------------------------------------------------
Warning: regressor matrix for moved equation appears ill-conditioned. (Condition number = 1
> 110405.6.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
Iteration 0: log likelihood = -2.679e+08
Iteration 1: log likelihood = -2.405e+08
Iteration 2: log likelihood = -2.392e+08
Iteration 3: log likelihood = -2.392e+08
Iteration 4: log likelihood = -2.392e+08
Note: 14 observations completely determined. Standard errors questionable.
Warning: regressor matrix for _cmp_y7 equation appears ill-conditioned. (Condition number =
> 1110405.6.)
This might prevent convergence. If it does, and if you have not done so already, you may ne
> ed to remove nearly
collinear regressors to achieve convergence. Or you may need to add a nrtolerance(#) or non
> rtolerance option to the command line.
See cmp tips.
Fitting full model.
cmp_lnL(): 3499 halton2() not found
<istmt>: - function returned error
Mata run-time error
Mata run-time error