Measurement

I. MEASURES AND MEASUREMENT ERRORS




CONSTRUCTION OF INSTRUMENT

1. Measurement defined

2. Levels of measurement

3. Validity/reliability of measures

4. Simple measures, index, and scale

5. Construction of survey instruments

INTERVIEW/QUESTIONNAIRE

1. difference between questionnaire and interview

2. difference between questionnaire instrument and interview schedule

3. objectives of questionnaire design and construction

4. steps in questionnaire design

5. advantages and disadvantages between questionnaire and interview

6. types of questions

7. question content

8. criteria for evaluating question content

9. response format
- exhaustive set
- mutually-exclusive categories
- unidimensionality

10. questionnaire construction
- structure
- process

MEASUREMENT AND MEASUREMENT PROBLEMS

1) Definition of Measurement
- identification of and differentiation between values or a variable through quantification
- assignment of numerals or numbers to objects, events or properties according to rules
Rules are significant component of the measurement procedure, they determine the quality of measurement

Measurement is meaningless when it is not tied to reality. Function of rules is to tie the measurement procedure to reality.

2) Levels of Measurement

a) nominal
- weakest level of measurement
- numbers of symbols are used to classify objects
- properties of objects in one category are equal to each other but not to anything else in their identical aspect
- symbols or numbers assigned may be interchanged without altering any information

b) ordinal
- designated by greater than > symbol
- number assigned to ranked objects are called ranked values
- ordinal number indicate rank order and nothing more; they do not indicate that the intervals between them are equal nor do they indicate absolute qualities

c) interval level
- able to order a property in terms of the > relation, one also knows the exact distance between each of the object and this distance is constant
- characterized by a common and constant unit of measurement

d) Ratio level
- property of natural zero points
Example: weight
time
length

Four types of relation are operationally possible.
1. equivalence
2. greater than
3. known distance or any two
4. true zero point

Variables that can be measured at a ratio level can also be measured at the interval, ordinal, and nominal levels.

- properties that can be measured at higher levels can be measured at lower level but not vise-versa



Level Equivalence Greater Than Fixed Interval Rational Zero

nominal X ----- ----- -----
ordinal X X ----- -----
interval X X X -----
ratio x x X X



3) Validity
- refers to the question whether researcher is measuring the precise property they intend to measure
- provide evidence so one could be confident that a measuring device does in fact measure what it appears to measure

Three basic kinds of validity:
a) content validity
b) empirical validity
c) construct validity

a) content validity

1. face validity
- rest on investigators evaluation as to the validity of a measuring instrument
- concerns with the extent to which it measures that which it appear to measure according to the researcher’s subjective assessment

Example: measure of ‘social development”
1. indicators of social development or questions which taken together will provide an index to measure the concept
2. researcher reviews each of indicators or questions
3. to ascertain his assessment researcher consults a number of specialists
4. agreement which constitute a conclusion as to the face validity

Problems of face validity:
- no replicable rules in evaluating the measuring instrument
- reliance on subjective judgement


2. Sampling validity
- concerns whether a given population of situation or behavior is adequately sampled by the measuring instrument
- does the content of the instrument adequately represent the content population of the property being measured?


b) Empirical validity
- concerns with the relations between the measuring instrument and the measuring results

- assumed that if a certain instrument is valid, then there should exist a certain empirical relations between the results produced by the instrument and other properties of variables

- existence of a relation is measured by correlation

- to evaluate empirical validity

a) predictive validity
- characterized by prediction to an external measure referred to a criterion and by checking a measuring instrument against some outcome
- ability of a test to identify future differences
- correlation between the results of a given measurement with an external criterion

Example: - validation of an intelligence test
- test scores of the group and grade points average
- performs correlation coefficient
- obtained validity coefficient


c) construct validity (an evaluation of the extent to which an instrument measures the theoretical construct)

- involves relating a measuring instrument to an overall theoretical framework in order to determine whether the instrument is tied to the concept and the theoretical assumption that are employed

- to demonstrate construct of a measuring instrument, an investigator has to show that these relationship do in fact hold
(convergent validity)
(discriminant validity)

4) Reliability
- need not be a concern if measuring instrument is valid
- frequently used method for evaluating an instrument
- an indication of the extent to which a measure contain variable errors, errors that differed from individual to individual during any one measuring instance, ad that varies from time to time for a given individual measured twice by the same instrument
- defined as the ratio of the true-score variance to the variance in the score as measured
- the evaluation of the reliability of any measurement procedure consists in estimating how much of the variation is attributable to chance or random errors.

Reliability – denotes consistency, dependability, accuracy and the extent to which two application of the same measure yield the same results under comparable conditions.


Variables are measured into

1. simple measure
2. index
3. scale


Simple measure is a direct measure of an entire concept, a single indicator that measures the concept as a whole.


An index is formed by combining several simple measures, each of which is a sub-concept or component of an overall concept.


An index can be formed by:
- adding up items
- statistical methods
- ratios
- analytic weighing

A. Definition


1. Questionnaire is a pre-arranged series of questions designed to obtain information from the respondents in written responses

2. Interview is a pre-arranged series of questions designed to obtain information in oral response wherein the interviewer and the person being interviewed are both present as the questions are asked and answered, the interviewer and the respondent have an opportunity to take greater care in communicating questions and eliciting information. Interviewer has also the opportunity to observe both the respondent and the situation.

3. Interview by telephone falls between questionnaire and personal interview. The interviewer can assess both the subject and subject’s situation in which the process of questioning is assured to be valid.


What Is A Questionnaire?

“A questionnaire is (or should be) a standardized set of questions designed to answer research objectives and phrased and ordered in such a way as to elicit the most complete and accurate information possible within the limits of survey.”


Objectives of Questionnaire Design
 produce data required
 facilitate interviewing
 facilitate responding
 facilitate processing


Steps in Questionnaire Design

1. prepare draft tables to identify data needed
2. define terms and concepts
3. flowchart logic and sequence of questions
4. draft questions and response formats
5. draft the physical format of questionnaire
6. test, evaluate, and finalize
7. print

B. Advantages of Questionnaire over Interview

1. Less expensive

2. Requires less skill in administering for it is mailed with a minimum explanation

3. Can be administered simultaneously on large individuals while interview calls for questioning each respondent

4. Questionnaires can be sent through mail

5. Possible to cover a wider area to obtain information

6. Respondents may have greater confidence in their anonymity, thus may freely express their views

7. Less pressure on the respondents

8. Mailed questionnaires can reach special groups of persons e.g. executives, businessmen who may seek refuge behind secretaries


C. Disadvantages of Questionnaire over Interview

1. Wordings may have diverse meaning for different kind of people

2. Comprehensible and incomprehensible to some

3. Appropriate only on population who has a considerable education

4. Lack of motivation and not appropriate to a large population

5. Do not elicit so high completion rate

6. The proportion of returned mailed questionnaires are very low


D. Advantages of Interview

1. Can be used in all segments of population

2. Many people will cooperate when they have to talk

3. Yield much better sample of general population

4. Greater sensitivity to misunderstanding by respondents

5. Appropriate technique for revealing information on complex subjects and sentiments



E. Disadvantages of Interview

1. Respondents may not tell completely what you are asking because of his doubt of his anonymity

2. Personality of interview may affect the measurement situation differently may be less comparable with one another


F. Factors Influencing Questionnaire’s Rate of Return

1. Sponsorship of the questionnaire

2. Attractiveness and clarity of the questionnaire format

3. Length of the questionnaire

4. Nature of the accompanying cover letter requesting cooperation

5. Ease of filling out the questionnaire and mailing it back

6. Inducement offered to reply

7. Interest of the questions to the respondent

8. Nature of the people to whom the questionnaire is sent


G. Respondents Who May Return Questionnaire

1. Less mobile

2. More interested

3. More literate

4. More partisan


H. Advantages of Telephone Interview

1. Low cost or no travel

2. Rapid completion with relatively high responses


I. Disadvantages of Telephone Interview

1. Those who have telephone are not representative of the general population

2. Unlisted number of telephone

3. Conversation usually terminated by the respondent

TYPES OF QUESTIONS


There are three important considerations which must be kept in mind when deciding what type of question to use:

1. the kind of information required – factual or subjective, such as opinions and attitudes.
2. the level of ability and training of data collection staff – are experienced personnel being used? What is their level of education? What kind of training will they received?
3. data processing plans – how many forms will be processed and how much time is available? What kind and how many clerical staff will be available? How much editing and tabulation will be done manually versus by computer?



II. QUESTION CONTENT

A. Aimed to ascertain

1. Facts
Researcher identifies the facts to be obtained going directly to the people who are knowledgeable on the subject and can furnish necessary information.


2. Belief About What the Facts Are
Investigator may wish to learn what people believe to be facts and to provide a picture of his beliefs.


3. Feelings
Emotional reaction
a) fear c) sympathy
b) hate d) admiration


4. Discovering Standard of Action
(a) Components of appropriate action
1. ethical standard of what should be done
2. practical consideration of what is feasible


5. Present and Past Behavior
e.g. Which brand do you like – a year ago? or now?
(retrospective or prospective)


6. Conscious reason for belief, feeling policies or behavior e.g.
Why do you go school?



III. TYPES OF INTERVIEWS AND QUESTIONNAIRES

A. Standardized/Interviews and Questionnaires
Presented with exactly the same wording and the same order to all respondents to ensure all respondents are replying to the same questions

1. Fixed alternative questions
2. Open-ended questions


B. Advantages of Fixed Alternative Questions
1. quick and relatively inexpensive to analyze
2. help make the question clear
3. omission may lead to biases


C. Disadvantages of Fixed Alternative Questions
1. force the respondents to give a statement of opinion
on an issue that respondents does not have
2. discourage the respondent to think


D. Less Structured Interviews
1. Focused Interview
- main function is to focus attention upon a given
experience and its main effects
2. Clinical Interview
- concerns with the broad underlying feeling
3. Non-directive
- interview is more complete in the hands of the
respondent. Questions are not frame or
respondents are made to talk and talk


IV. SOCIOMETRIC METHOD

- concerns with social interaction among the group of people. Data are designed to obtain information about interaction or lack of interaction

Uses of Sociometric

1. Provide information about an individual position in the group
2. Data are used in the study of leadership


V. VISUAL AID IN INTERVIEW

- Use to substitute verbal statement e.g. cards, photographs



CRITERIA FOR EVALUATING QUESTION CONTENT


Sirken mentions three criteria for evaluating question content:


1. relevance - Is the question relevant to the survey objectives? Does it produce the information required – no more and no less?

Example: In a consumer expenditure survey it would be irrelevant to ask about attitudes toward family planning.

2. sensitivity - Is the respondent likely to cooperate in answering the question?

Example: Fertility surveys usually don’t ask unmarried women about how many children they have borne. This is only asked of females who were ever-married.

3. likelihood that respondent is able to answer the question

Is the question within the respondents scope of knowledge?

Example: Respondents cannot reasonably be expected to be able to reliably report medical diagnoses of their illness.

Is the respondent able to remember the information? Questions about status, such as age, sex or property ownership usually present no problem. But episodic questions require recall of some event. The respondent must determine two things: did the event occur and did it occur within the reference period.

Example: Did you visit a dentist in the last twelve months?

We can add two or more to the list of three criteria from Sirken.

4. understandable wording

Is the question worded so that the meaning is clear? The question should not be too long or too short. It should be specific and directed.

5. neutral wording

Is the question worded so as not to lead the respondent or influence him to give one answer rather than another?

CONSTRUCTION OF SCALE (LIKERT-LIKE)

Steps:

1. Assembles a large number of items considered relevant to the attitude being investigated. Items/statements must either clearly favorable or clearly unfavorable (positive and negative statements)

2. These items are administered to a group of individuals representative of those with whom the questionnaire is to be used

3. Obtain the total score by summing the item scores

Scoring Favorable Unfavorable
SA 5 1
A 4 2
U 3 3
D 2 4
SD 1 5

4. Determine the weak items (items that cannot discriminate clearly between high scores and low scores on the total scale)


WAYS TO DETERMINE THE DISCRIMINATIVE POWER OF AN ITEM

a) Pearson product moment correlation

b) item analysis

Application
- administer the instrument to a number of individuals
ex: 40

- obtain total scores. Get the 25% top and 25% low from the total scores
Set aside the 50% at the middle

- calculate the discriminate power of each item
- determine the cut-off point. Those items below the cut-off point are to be discarded


III. INDIRECT TECHNIQUES

A. PROJECTIVE METHODS

1) Rationale
- how an individual organizes a relatively unstructured stimulus reflects his/her perception of the world and response to it

2) Characteristics
- stimuli capable of arousing many different kinds of reactions
- no right or wrong answer
- emphasis is on the respondent’s perception of the natural, the meaning given to it, and the way the respondents organizes or manipulates it.
- responses are not taken at its face-value – but interpreted in terms of some pre-established conceptualization.

3) Projective Methods in the Study of Attitudes

a. pictorial technique

b. verbal technique
• sentence completion
Series of incomplete sentences which an individual is asked to complete usually under some pressure to ensure spontaneity of response
• projective question
• asking other people’s views
• description of persons who would behave in a specified way (used often in market research) imaging or positioning

c. play techniques
used to study children’s attitudes


B. MORE STRUCTURED INDIRECT TEST

1) Basis
Attitudes are likely to influence perception, beliefs, judgment and memory

2) Attitude measured in terms of knowledge
a. information test (multiple-choice test)
b. syllogistic inference
- statement of fact followed by several conclusion that might be drawn
c. bias in learning and memory accuracy of recall and speed of learning reflects one’s attitude

3) Judgment as reflection of attitude
a. judgment of characteristics of individuals
b. judgment of product attributed to different sources
c. judgment of social policies