Measurement error

The true value is what we would like to obtain for any variable we measure: What we get, however, is the observed value. The difference between the two is caused by some combination of possible systematic error and some amount of random error. Systematic error refers to any bias or errors affecting all the data obtained. A few of many sources of systematic error are: having invalid or unreliable measures, poor interviewing skills, personal bias of the researcher, or consistent errors in scoring items in a scale or index. Random error includes any error that occurs at a specific time, such an interruption or misunderstanding during a particular interview but not in others or an occasional mistake in scoring an item or in adding item scores. The relation among these three components is shown by the following formula:

Obviously, we want to avoid any systematic errors and minimize the inevitable random errors. We can avoid or mimimize possible systematic errors by developing a detailed, sound research design and having it reviewed by others to identify any bias or potential systematic error. Clear evidence for validity and reliability of measures is a further safeguard against systematic errors. Careful attention to each research task and rechecking every action will protect against random errors.

For a more extensive discussion of measurement error, go to:

Measurement Error

The Role of Measurement Error

Aids

Key terms

Cell Coefficient of reproducibility Composite measure External validity Index Index score Internal validity Item Item analysis Likert scale Mixed types Observed value

Response set Scale Scale types Scoring Social distance scales Systematic error True value Typology Unweighted index Weighted index Weighting indicators

Main points

1. Composite measures are based on a set of indicators. The main forms of composite scores are scales and indexes.
2. Scales and indexes are composed of sets of   indicators, each of which is given a score or value. The total score for the scale or index is the sum of the item scores.
3. Composite measures provide more valid, reliable, and   precise measurement for many variables than can be obtained from use of single items.
4. Scales provide measures of the intensity of responses to a set of items. Frequently used forms of scales are the Likert, social distance, and Guttman scales.
5. Indexes are based on multiple indicators, each of which provides a separate measure of a dimension of the variable being measured.
6. Indicators in scales and indexes are represented by items or statements. Criteria for selection of items includes: demonstrating their face or content validity and their unidimensionality and providing evidence that each item contributes significantly to the total score.
7. Items in a scale or index are scored by assigning numbers to each of the response categories for the items.
8. Items making up a scale or index should be empirically related to one another. This shows that the items are indicators of the same variable or one of its specific dimensions.
9. Scoring items includes making a decision whether to give greater importance or weight to certain items or to leave them all unweighted.
10. Scoring items also requires deciding how to handle missing data. This can be done by dropping cases with missing data; making an "educated" estimate of what the missing value should be; or by assigning the average value in place of the missing data.
11. A typology is a classification system based on two or more characteristics. Typologies produce nominal measurement and should be used only as independent variables.
12. Any measurement involves two kinds of errors: systematic error, which affects every measurement taken for some indicator (poorly phrased questions or bad interviewing style); and random error (something that occurs in a particular setting only, like a mistake in recording a single response). Investigators seek to minimize both forms of errors so that any measurement for any indicator is as close to its true value as possible.