Calculating d and composite scores: a statistical note

When comparing groups, typically, the following formula is used:

calculating d

The difference between the means is divided by the pooled standard deviations.

Often, sample sizes are not presented directly. But usually, they can be determined. For example, the NAEP explorer gives percentages which when multiplied by 100 can be used as N(a) and N(b) in our formula above. For example, below were the average and subtest Math TIMSS 2007 scores for grade 8:

TIMSS subtest scores

Whites made up 55% of the sample and Blacks made up 12%. The pooled standard deviations then were: SQRT((55*SD^2+12*SD^2)/(55+12). And the standardized differences were (White score – Black score)/pooled SD. These are presented on the right hand side of the figure above.

It will be noticed that the average B/W difference is larger than the average of the subtest differences. This is because the average is a composite score and because composite scores are calculated thusly:

composites

Importantly, the magnitude of the composite score is a function of the correlation between predictors/subtests.

Taking our sample above, there are 7 subtests (Numbers, Algebra …Reasoning). The average magnitude of the B/W difference on these is 0.95. The summed magnitude is 6.66. The average correlation between subtests is about 0.7. So we get:

Composite d = 6.66 / SQRT(7+7(7-1)*0.7)

Which should come out to about to a Cohen’s d of 1.1.

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to Calculating d and composite scores: a statistical note

  1. 猛虎 says:

    This should be “SQRT(7+7(7-1)*0.7)” and not “SQRT(7-7(7-1)*0.7)”, if I may.

    Plus, When I don’t have much time to make my own calculations, I use this program. This is faster, and yields the same result.
    http://www.cognitiveflexibility.org/effectsize/

    By the way, how could you explain the B-W gap of 1.7 in the UK here ? (page 309). I cannot make sense of this figure. Worse. I cannot find this reference : Scott & Anderson (2003).

  2. Chuck says:

    Thanks and thanks.

    As for the UK GMA ds they’re based on a presentation entitled “Ethnic and gender differences in GMA test scores: Findings from the UK.” I contacted the authors of the presentation a while back and they confirmed the magnitude of the ds. I’m not sure why they’re so large. I have speculated that this is possibly because there are a lot of 1st generation Blacks in the sample. And that the tests could very well be culturally biased against them. The situation is curious because adult 1st generation Blacks don’t do that poorly in the US. See here:
    http://occidentalascent.wordpress.com/page/2/
    And for my other discussions:
    http://occidentalascent.wordpress.com/2012/02/07/partially-falsified/

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s