Hi all
I have a question about interpreting results of marginal effects and its first difference correctly.
I ran a three-way interaction effect of x1,x2 and x3 using "xtlogit" and estimated marginal effects of x1 while fixing x2 and x3 at certain values of interest using "margins, dxdy(x1) at(x2=.. x3=...)". x1 and x2 are continuous variables, and x3 is discrete and runs from 1 to 5.
The results are shown below:
Based on the margins results and the graph produced by "marginsplot, x(s) plot(income)" following the margins command, it seems that both income groups (1:the poorest and 5: the richest) show a similar pattern such that as "s" increases the effect of "growthpr" diminishes, and the effect disappears when "s" goes beyond 2 for the rich (income=5) and 2.5 for the poor (income=1).
Then, the conclusion will be that the effect of "growthpr" reduces as "s" grows, and the effect is apparent among both the poor and the rich.
I think, in many cases, scholars interpret the results based on the graph of marginal effects like this and finish there.
However, I went on further to calculate the first difference of marginal effects for each income group to see whether the moderating effect of "s" is statistically significant holding the income group constant, because to show the moderating effect of "s" on the relationship between "growthpr" and y is the focus of my research.
I used both "test" command following the margins estimation and "mlincom" command of the spost13 package. Both generate the same results with the same p-values of the test although the test statistics they report are different (such that test reports Chi-2 and mlincom generates linear combination of estimates from margins). The results of "mlincom" are shown below:
Based on the first difference estimation, for the rich (income=5), the marginal effect of "growthpr" when "s" takes the highest value (=3.5) is statistically different from that at all levels of "s", suggesting that there is a moderating effect of "s" among the richest income group.
However, for the poor (income=1), it seems that the marginal effects of "growthpr" at all levels of "s" are not statistically distinguishable from one another. Then, should I conclude that the moderating effect of "s" exists only among the rich but not among the poor?
I think the answer to this question is yes, but I am still confused because from the results after the "margins" command, we saw even among the poorest group, the marginal effects of "growthpr" at lower values of "s" were positive and statistically significant, but then at higher values of "s" they lost significance.
Maybe I am asking a question that is too obvious, but I want to make sure if the second step of testing the first difference of marginal effects is necessary so that the second conclusion is correct while the first conclusion is wrong.
I look forward to your advice. Thank you all for your time!
I have a question about interpreting results of marginal effects and its first difference correctly.
I ran a three-way interaction effect of x1,x2 and x3 using "xtlogit" and estimated marginal effects of x1 while fixing x2 and x3 at certain values of interest using "margins, dxdy(x1) at(x2=.. x3=...)". x1 and x2 are continuous variables, and x3 is discrete and runs from 1 to 5.
The results are shown below:
Code:
. xtlogit y c.growthpr##c.s##c.income educ soph pidch female married unemployed, nolog
Random-effects logistic regression Number of obs = 72210
Group variable: election1 Number of groups = 77
Random effects u_i ~ Gaussian Obs per group: min = 128
avg = 937.8
max = 3339
Integration method: mvaghermite Integration points = 12
Wald chi2(52) = 15043.21
Log likelihood = -30646.39 Prob > chi2 = 0.0000
------------------------------------------------------------------------------------------
votech | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
growthpr | .1442353 .0596027 2.42 0.016 .0274161 .2610544
s | -.4836696 .1520272 -3.18 0.001 -.7816374 -.1857017
|
c.growthpr#c.s | -.0314627 .0332276 -0.95 0.344 -.0965875 .0336621
|
income | -.0405134 .022002 -1.84 0.066 -.0836366 .0026097
|
c.growthpr#c.income | .0088782 .0061788 1.44 0.151 -.0032321 .0209884
|
c.s#c.income | .0414441 .0136099 3.05 0.002 .0147692 .0681191
|
c.growthpr#c.s#c.income | -.0085141 .003867 -2.20 0.028 -.0160934 -.0009349
|
educ | -.066387 .0070612 -9.40 0.000 -.0802267 -.0525474
soph | -.0872549 .0118639 -7.35 0.000 -.1105077 -.0640021
pidch | 3.603269 .0296561 121.50 0.000 3.545145 3.661394
female | .0767834 .021088 3.64 0.000 .0354517 .1181151
married | .0909665 .0246408 3.69 0.000 .0426713 .1392616
unemployed | -.0806562 .0522876 -1.54 0.123 -.1831381 .0218257
|
_cons | -1.478319 .4122553 -3.59 0.000 -2.286325 -.6703135
-------------------------+----------------------------------------------------------------
/lnsig2u | -2.049376 .1757649 -2.393868 -1.704883
-------------------------+----------------------------------------------------------------
sigma_u | .3589085 .0315418 .302119 .4263727
rho | .0376798 .0063732 .0269956 .052365
------------------------------------------------------------------------------------------
Likelihood-ratio test of rho=0: chibar2(01) = 928.71 Prob >= chibar2 = 0.000
. margins, dydx(growthpr) at(s=(0(.5)3.5) income=(1 5)) predict(pu0) post
Average marginal effects Number of obs = 72210
Model VCE : OIM
Expression : Pr(votech=1 assuming u_i=0), predict(pu0)
dy/dx w.r.t. : growthpr
1._at : s = 0
income = 1
2._at : s = 0
income = 5
3._at : s = .5
income = 1
4._at : s = .5
income = 5
5._at : s = 1
income = 1
6._at : s = 1
income = 5
7._at : s = 1.5
income = 1
8._at : s = 1.5
income = 5
9._at : s = 2
income = 1
10._at : s = 2
income = 5
11._at : s = 2.5
income = 1
12._at : s = 2.5
income = 5
13._at : s = 3
income = 1
14._at : s = 3
income = 5
15._at : s = 3.5
income = 1
16._at : s = 3.5
income = 5
------------------------------------------------------------------------------
| Delta-method
| dy/dx Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
growthpr |
_at |
1 | .0250461 .0098778 2.54 0.011 .0056858 .0444063
2 | .0291296 .0094184 3.09 0.002 .01067 .0475892
3 | .0204341 .0070597 2.89 0.004 .0065974 .0342708
4 | .0226016 .0068806 3.28 0.001 .0091159 .0360872
5 | .0160909 .0045529 3.53 0.000 .0071674 .0250144
6 | .0163199 .0045498 3.59 0.000 .0074024 .0252374
7 | .0121725 .0029369 4.14 0.000 .0064163 .0179287
8 | .0104831 .0030103 3.48 0.000 .004583 .0163831
9 | .0087574 .00289 3.03 0.002 .0030931 .0144216
10 | .0051963 .0029983 1.73 0.083 -.0006803 .0110729
11 | .0058407 .0038104 1.53 0.125 -.0016276 .013309
12 | .0004317 .0040198 0.11 0.914 -.0074469 .0083103
13 | .003362 .0048688 0.69 0.490 -.0061807 .0129048
14 | -.0038948 .0052978 -0.74 0.462 -.0142783 .0064887
15 | .0012408 .005857 0.21 0.832 -.0102386 .0127203
16 | -.0078576 .0065926 -1.19 0.233 -.0207789 .0050638
------------------------------------------------------------------------------
Then, the conclusion will be that the effect of "growthpr" reduces as "s" grows, and the effect is apparent among both the poor and the rich.
I think, in many cases, scholars interpret the results based on the graph of marginal effects like this and finish there.
However, I went on further to calculate the first difference of marginal effects for each income group to see whether the moderating effect of "s" is statistically significant holding the income group constant, because to show the moderating effect of "s" on the relationship between "growthpr" and y is the focus of my research.
I used both "test" command following the margins estimation and "mlincom" command of the spost13 package. Both generate the same results with the same p-values of the test although the test statistics they report are different (such that test reports Chi-2 and mlincom generates linear combination of estimates from margins). The results of "mlincom" are shown below:
Code:
. qui mlincom 1-15, add rowname(Poor1_15)
. qui mlincom 3-15, add rowname(Poor3_15)
. qui mlincom 5-15, add rowname(Poor5_15)
. qui mlincom 7-15, add rowname(Poor7_15)
. qui mlincom 9-15, add rowname(Poor9_15)
. qui mlincom 11-15, add rowname(Poor11_15)
. qui mlincom 13-15, add rowname(Poor13_15)
. qui mlincom 2-16, add rowname(Rich2_16)
. qui mlincom 4-16, add rowname(Rich4_16)
. qui mlincom 6-16, add rowname(Rich6_16)
. qui mlincom 8-16, add rowname(Rich8_16)
. qui mlincom 10-16, add rowname(Rich10_16)
. qui mlincom 12-16, add rowname(Rich12_16)
. mlincom 14-16, add rowname(Rich14_16)
| lincom pvalue ll ul
-------------+----------------------------------------
Poor1_15 | 0.024 0.107 -0.005 0.053
Poor3_15 | 0.019 0.103 -0.004 0.042
Poor5_15 | 0.015 0.095 -0.003 0.032
Poor7_15 | 0.011 0.085 -0.002 0.023
Poor9_15 | 0.008 0.076 -0.001 0.016
Poor11_15 | 0.005 0.070 -0.000 0.010
Poor13_15 | 0.002 0.068 -0.000 0.004
Rich2_16 | 0.037 0.012 0.008 0.066
Rich4_16 | 0.030 0.012 0.007 0.054
Rich6_16 | 0.024 0.011 0.006 0.043
Rich8_16 | 0.018 0.009 0.005 0.032
Rich10_16 | 0.013 0.008 0.003 0.023
Rich12_16 | 0.008 0.008 0.002 0.014
Rich14_16 | 0.004 0.008 0.001 0.007
However, for the poor (income=1), it seems that the marginal effects of "growthpr" at all levels of "s" are not statistically distinguishable from one another. Then, should I conclude that the moderating effect of "s" exists only among the rich but not among the poor?
I think the answer to this question is yes, but I am still confused because from the results after the "margins" command, we saw even among the poorest group, the marginal effects of "growthpr" at lower values of "s" were positive and statistically significant, but then at higher values of "s" they lost significance.
Maybe I am asking a question that is too obvious, but I want to make sure if the second step of testing the first difference of marginal effects is necessary so that the second conclusion is correct while the first conclusion is wrong.
I look forward to your advice. Thank you all for your time!