SET 1 (Scroll down below for Set 2) - Note\; We shall discuss in class those figures, items that are not readable/distorted in this format - thanks and good luck!
DATA ANALYSIS AND REPORT WRITING: WHAT TO KNOW BEFORE YOU START
(source|: Bencha Yoddumneren-Attig)
In collecting, analyzing and reporting research data/information, two main formats most researchers tend to use.
1)researcher makes some sense out of fieldnotes, documents and other data sources after the data collection process, i.e., fieldwork, is completed.
- format is very problematical (in many instances field insights are lacking or forgotten, and the opportunity to clarify social and cultural patterns with supplementary data has long since passed)
2) researcher must know, before fieldwork even commences, exactly how to work the data into an organized and understandable unit, which can then be easily written-up in a logical form.
-data collection and analysis should occur simultaneously during the time spent in the field to continually formulate, test, and/or modify hypotheses concerning the research problem under study using newly obtained field data and the insights of his/her informants
-lets researcher cycle back and forth between thinking about existing data and generating new strategies for collecting ever finer data
-focuses more fully on analytic exploration and development of data which are substantiated through on-going systematic observations and interviews.
Major aims of this lecture:
- to initially familiarize you with qualitative analysis by guiding you in mapping out a plan for effectively sorting, categorizing and classifying raw data into units of analysis as well as changing information from a narrative form into a concrete filing system. Thereafter, specific helpful tools for analysis are described
-to communicate the data by organizing the research report format, since individual report writing styles are dependent on the researcher, himself/herself, his/her project objectives, and the data involved.
I. Data Analysis
As noted, a qualitative analysis is best done as data is collected through its efficient sorting, coding (or indexing- category) and filing. This will aid the researcher in organizing his data, turn the data into concepts, and then concepts into relationships. That is to say, in analyzing qualitative materials, the researcher is trying to find underlying patterns that join together and make sense out of observations and interview cases (Arnold, 1982; Patton, 1980). The purpose of these procedures is to organize information into logical, discrete and comparable units which aid the researcher in repeatedly collecting and analyzing data in an on-going, progressive manner.
Sorting Data
The sorting process usually begins as soon as field data are obtained. Every evening (and lasting sometimes into night), a researcher must write-up his field notes in a systematic form. Each completed write-up should be comprised of at least the following aspects; a) people met as well as events or situations experienced that day and the exact context in which they occurred (even the smallest, seemingly insignificant encounter should be noted in detail since, at a later time, it may become exceedingly relevant); b) the main themes or issues discussed in each interview, described in as complete during the next interview; and, d) the specific types of data sought about these issues (Miles and Huberman, 1984).
The researcher may then do one of two things with his field notes, depending on his specific style. He might immediately analyze them by making clarifications and comments as they appear out of the data, noting specific relationships between variables (e.g. issue items), and linking these with previously collected information. For other researchers, after the field notes are written up, he/she might let them “sit” for an hour or two, and then come back and review them using the same process as in the first case. This latter technique is especially is good when the field notes are lengthy, since it gives the researcher a chance to relax after the arduous task of compilation and then clears his mind for a more objective analysis. Both methods are acceptable in that each aims to give the researcher clear insight into what data he has actually collected, its connection with previous information, and what data still needs to be obtained. In this way, the researcher will gain a more comprehensive understanding of the core variables and key phases in a process, as well as major research issues which may not have been anticipated (Burgess, 1982; Lofland and Lofland, 1984).
Data Coding (or Indexing-Category)
While the researcher is sorting the data, he must simultaneously do the coding or indexing-category. These are mechanisms for organizing and classifying data, so that it can be readily compared between cases (e.g. persons), and patterns can be readily identified. These codes are inductively developed and point to the general domains evident in the fieldnote content. Bogden and Biklen (1982) rightly suggest that the major scheme for coding should include: a) setting/content, e.g., general information or surroundings; b) definition of the situation – how people define the setting surrounding specific topics; c) perspectives – ways of thinking, orientation; d) activities – regularly occurring behaviors and their patterning; e) events or specific happenings; and f) strategies – ways of achieving specific ends. To illustrate, the following is an example of completed field notes and an indexing-category.
Example 1: The following is an adopted study… “Continuity and Change in a Northern Bukidnon Village: Determinants and Consequences of Fertility Decline on Northern Bukidnon Family Structure” (Yoddumnern, 1985).
Fieldnotes Indexing-Category
Mrs. A migrated to this village from barangay imagine, a nearby community. In total, she has six siblings: 2 elder and 3 younger, four of which are female and one male (thus 5 females, 1 male; a family of six).
Mrs. A is the third eldest child, and she has ,worked in her family’s rice fields since she was 14 years old. Before that time, she did household chores.
Mrs. A is now 20 years old, and she has been married for one year. Mrs. A and her husband knew each other for about 5 months before they were married.
The wedding ceremony was a traditional one. In the beginning, the groom’s parents asked for the hand of the bride from her parents. When this proposal was accepted both the groom and bride paid respect to each other’s lineage spirit. This is called the rite of cross-lineage propitiation.
Usually, men then move into their wife’s parent’s house. For Mrs. A, though, this was not the case. Since her husband was an only child, no one would be left to care for his parents. Thus Mrs. A began her married life in her husband’s natal household.
Thereafter, Mrs. A worked with her parents-in-law in their rice field. Whenever possible, though, after they were finished the married couple also went and helped Mrs. A’s family. If they could not go, they hired someone to help Mrs. A’s parents.
After marriage, Mrs. A used contraceptive pills for about one year. Her husband’s mother and sister advised her to do this, since her husband’s temporary job did not provide a steady income. After nine months of marriage, however, her husband’s job became permanent. They both then wanted to have a child, so Mrs. A stopped using the pills two months ago.
1. Prior residence
2. No. of siblings/family size
3. Occupational duties prior to marriage;
4. Age at farm labor initiation
5. Current age
6. Age at marriage
7. Marriage process
8. Sacred/religious rites or beliefs
9. Post-marital residence
10. Labor obtainment
13. Contraceptive use duration
14. Information sources for family planning
15. Usage determinants
16. Contraceptive discontinuation determinants
17. Time of contraceptive discontinuation
Coding or indexing-category, as noted, can be made by designing the required units of analysis (i.e.’ the major relevant headings which arise during data collection). Oftentimes, these units can be divided into roughly three types: enumeration unit, recording unit and context unit (Cartright, 1966). The enumeration unit refers to an exact number, time or age, physical length or temporal duration. In the previous example, indexing-categories numbers 2, 4, 5, 6, 13, and 17 express enumeration units.
The recording unit is the answer to a single question, and is often referred to as an indicator. For example, when there is a question about patterns of post-nuptial residence in a community, the answer might be neolocality. Neolocality is then perceived as the recording unit (subsequent examples to follow).
The content unit (or classes/sub-classes) provides the basis for perceiving the recording unit. For example, in the study of “Population Growth, Society and Culture”, Sipes (1980: 34-36) hypothetically anticipates several factors that correlate with population growth such as kin, social system and relationships; the position of women in the society; marriage and divorce; etc. Each of these factors is considered as one context unit. Under each unit, several recording units can be noted as in Example 2 (e.g., Kin, Social System and Relationships is the context unit, whereas classes 1.1 – 1.16 are the recording units).
Example 2: Context and Recording Units
Context Unit 1. Kin, Social System and Relationships
Recording Units 1.1 Neolocality
1.2 Polygyny
1.3 Inheritance rules
1.4 Sociopolitical usefulness of children
1.5 Results of illegitimacy
1.6 Child labor laws or customs
1.7 Attitudes of children toward parents
1.8 Attitudes of young toward old
1.9 Sociality of care for the aged
1.10 Amount of care given the aged
1.11 Individual achieved versus ascribed status
1.12 Sex-based differences in roles and behavior
1.13 Freeness of conversation between sexes
1.14 Extent of corporate kin group activity
1.15 Economic contribution by children to the house hold
1.16 General importance of family to the society
Context Unit 2. Position of Women in Society
Recording Units 2.1 Preference for a particular sex of child
2.2 Equality of women with men
2.3 Desire to restrict women to the home
Context Unit 3. Marriage and Divorce
Recording Units 3.1 Social drawbacks of the unmarried state
3.2 Parental arrangements of marriages
3.3 Difficulty in acquiring a marriage partner
3.4 Age at first marriage
3.5 Differences in ages of marriage partners
3.6 Percent of people never married
3.7 Frequency of divorce
3.8 Ease of divorce
Context Unit 4. Pregnancy and Parenthood
Recording Units 4.1 Desire to limit the number of children
4.2 Pregnancy viewed as onerous
4.3 Influence of parenthood on sex identity
4.4 Attitude toward abortion and infanticide
Context Unit 5. Sexuality
Recording Units 5.1 Abstinence from sex
5.2 Adolescent’s knowledge about sex
5.3 Discussion of sex in mixed company
5.4 Freedom to resist the advances of the other sex
5.5 Acceptance of nonmarital sex
5.6 Moral decay of society
5.7 Sex as recreation
Context Unit 6. The Supernatural
Recording Units 6.1 Ego’s misfortune due to other persona
6.2 Contribution of deceased kin to living descendants
6.3 Contribution of living kin to deceased ancestors
6.4 Need of a male or female heir for religious rituals
6.5 A religious versus a secular orientation
Context Unit 7. World View and Horizons
Recording Units 7.1 Fatalistic attitude toward the future
7.2 Economic status affected by individual scion
(Source: Sipes, R.G., 1980: 34-35)
Like the sorting procedure, the aim of these units of analysis – be they enumeration, content or recording units – is to organize and classify data. From this procedure, a multitude of different data types and their content (which oftentimes are intricately inter-twined) can be logically separated out. They can then be objectively compared and similar patterns can be identified which link variables on an intra- and inter-case basis (e.g., determinants of post-marital residence patterns, post-partum food habits/taboos).
Establishing Files
In addition to sorting, coding and devising the units of analysis, an efficient data, as mentioned in several chapters, is usually not based on numbers but words, and these are oftentimes difficult to keep organized so that they can be readily accessed and reviewed. The researcher’s challenge, therefore, is to change the data out of its narrative form – as found in the researcher’s fieldnotes, recording or write-ups – and into a storage system where he can easily order and retrieve it for latter use (especially in writing the report).
To do this, three major files are needed, namely, fieldwork, mundane/background, and analytic (Lofland and Lofland, 1984). A fieldwork file contains materials on the process used in conducting the research. It should include the step-by-step procedures used in collecting information, personal experiences, feelings and observations of the researcher himself as these may or may not affect data collection, any logistical problems encountered, and the like. By having a file on this topic already built up, the researcher will find it easier to write-up the final report’s section on research methodology or research strategies. He will also be able to look back on how his personal actions and reactions might have affected, or were affected by, the community itself. For instance, he can assess if his experiences as a participant were actual, normal reflections of natural community and individual behavioral patterns, or whether certain community members changed their behavior because of his presence. (In some cases, researchers keep a personal diary of their intimate thoughts and feelings toward the community and research project as a way of assessing objectivity at a later time.)
A mundane or background file is used to keep track of people, places, organizations, documents and so forth. Mundane files should be organized in such a way that information is grouped under obvious categories so as to facilitate its later retrieval. For instance, when an in-depth interview is conducted, the researcher will almost certainly want to have a folder on this person, in addition to subsequent persons (Lofland and Lofland, 1984). Data related to the community under study – such as its history and development, material resources, family organization, and so forth – should also be filed under specific thematic categories.
And lastly, the analytic file, as its name implies, is the heart of the analysis. When the researcher reviews his notes, he must analyze and interpret the data by discerning patterns of behavior (often through the use of content and recording units) and finding the underlying meanings that were evident in the interviews or observations. This preliminary analysis should be written-up in a brief (or as extended as possible) fashion, to be put along with a copy of the relevant data into labeled file folders, and entered into the analytical file under specific behavioral, cultural or social categories (Lofland and Lofland, 1984).
From my own personal experience, trained research assistants almost inevitably have problems in deciding what to write in an analytic file. Their main concern is whether there will be any repetition between field notes and the analytic file. The answer is NO. Fieldnotes record a picture of the social setting and conversation involved in each interview, event or experience. They serve as a computer – to memorize, sort, code, record and organize data into a written form. The analytic sheets (which comprise the file), on the other hand, reveal the main themes, impressions and summary statements about what actually occurred in the interview. It also includes explanations, speculations, and hypotheses about the community at large as they bear upon the research problem.
When to write-up this analytic sheet is rather flexible. Ideally, it should be done every night after taking two or three interviews, since it aids in updating the researcher’s understanding of what is going on. It also assists in revising and updating the coding scheme. I personally use this preliminary analysis in forming a plan for the next interview.
Preliminary analyses, thus, should be put in one analytic file. If the researcher begins the preliminary analysis at the start of the fieldwork, he will be better able to identify a central thesis or set of fundamental assumptions concerning the research project and its results (Arnold, 1982; Burgess, 1982; Lofland and Lofland, 1984). He can then elaborate on and expand crucial features into an effective outline (or it can be call a ‘Table of Contents’) and reorganize his analytic file according to the outline’s order. As his analysis builds upon itself, the mundane and analytic files are likely to merge.
Tools for Data Analysis
As analysis, therefore, entails the reviewing, indexing, reorganizing, classifying, filing, and refining of data in order to turn it into comparable items, concepts and then relationships for further investigation. In conducting comparative qualitative analysis, several tools exist which can facilitate this process (these are discussed in detail and at great length by Miles and Huberman, 1984, and Scrimshaw and Hurtado, 1987). They include, amongst others, graphic devices, organizational charts, causal networks, taxonomies or ethnoclassifications, conceptually clustered variables, and mapping.
Graphic devices are useful in reflecting trends and aiding in understanding. For example, organizational charts show the relationships between structural or hierarchal levels. Flow charts describe and contrast events. Growth charts, relatedly, show increases in significant variables over time (Miles and Huberman, 1984).
Example 3: Causal Networks Regarding Immunization Services
Taxonomies or ethnoclassifications constitute an extremely useful strategy for organizing qualitative information and interpreting research findings (Spradley, 1979, 1980). In Northeastern Bukidnon, as a case point, there are several local terminologies for infant diarrheal disease, which are different from those assigned for adults. These are developed according to folk knowledge or experiences about symptoms, degree of severity as well as etiological beliefs. For instance, certain episodes of diarrhea are associated with normal child development, they are common to everyone, and thus require no special treatment. In other instances, diarrheal episodes, which do not correlate to a life-cycle stage, are given a different name (taxonomic or ethnoclassification) and readily treated (Premsriratana, 1985: 121-124).
Conceptually clustered variables bring together data which are relatable (Scrimshaw and Hurtado, 1987). In a study concerning the duration of breast-feeding among Northeastern Bukidnon women, seven main variables from a cluster which influences incidences of child/infant malnutrition. These variables are: 1) climate, 2) economic opportunities, 3) female (i.e., mother) migration patterns, 4) child care-taking responsibilities, 5) weaning and food supplements, 6) the cognitive classification of breastmilk, and 7) the color, taste, and texture of sweetened condensed milk. These variables are related based on the following overall scheme. In Northeast Bukidnon, economic opportunities are more restricted than elsewhere in Bukidnon. The main occupation is wet-rice cultivation. But due to the twin conditions of drought versus flood (oftentimes in the same growing season), fairly good crops can only be expected once every three years of so. Hence, both men and women often migrate either daily or seasonally in search of work and supplemental income. In their absence, grandmothers become the major child care-takers so as to facilitate parental migration, and thus early weaning often occurs. In their attempt to find a suitable weaning food, powdered milk formulas are not used since they are expensive and not readily available. Alternatively, they try to find the closest cognitive match between breastmilk (the indigenously recognized ideal food for infants) and a suitable replacement which is inexpensive and accessible. Breastmilk which is thick, sweet, and cloudy white is believed to be of the highest quality. Thin, yellow, and tasteless breastmilk is believed to be rotten. In assessing the foods available in their area, many grandmothers (and some mothers) select sweetened condensed milk as a breastmilk substitute. Sweetened condensed milk has all the sensory characteristics of breastmilk, it is inexpensive, and readily available (Vong-ek,1987). Hence many variables cluster themselves in such a way as to determine the selection and use of certain foods in weaning and their consequences for child nutritional status. Information of this type should be included in the analytic file.
In addition to conceptually clustered variables and the other tools, mapping is also a useful aid to understanding the relationship between the physical environment and human behavior. A map of the community’s settlement pattern can be prepared by the researcher to show the locations of households relative to such resources as agricultural land, water sources (e.g., wells, rivers), roads, and markets. If the settlement is influenced by seasonal migrations, this should also be noted (Johnson, 1978). Maps which identify not only the location of houses but also their occupants are also exceedingly valuable. Once the researcher has become familiar with community members and their social relations, maps can be used to trace social networks and how the community’s spatial arrangements reflects these as well as the selection of house sites. Maps can be placed either in mundane/background files (as in their relation emphasis on the physical environment and settlement patterns) or in analytic files where they demonstrate or illustrate social relationships and inter-personal communication.
II. Report Writing
Upon completion of the fieldwork, the researcher should go through all three sets of files again, reorder them and create a serious outline by working out the component parts of the report (chapters, sections, and so forth). Again, well organized files will tremendously facilitate this process. For example, in my own research on the determinants and consequences of fertility decline on Northern Bukidnon family structure (Yoddumnern, 1985), the analytic file (which is the heart of analysis and report writing) was comprised of 5 major components (or context units): 1) social and cultural continuity in a northern Bukidnon village; 2) socio-economic and demographic change and their processes; 3) determinants of fertility behavior; 4) consequences of fertility decline; and 5) the Northern Bukidnon social and family structure. These five major components form the core of report analyzing and writing. The next step is to order the files according to or centering around the research project’s objectives.
For example, the main objective of this research project was to demonstrate how demographic change has affected the Northern Bukidnon family structure and function. The specific aims of this research were: 1) to examine continuity and change in Northern Bukidnon social and family structure; 2) to determine the factors involved in the process of fertility decline in a Northern Bukidnon village; and 3) to address the consequences of fertility decline on Northern Thai family structure.
Having these purpose in mind and after gone through the analytic files, the researcher (myself) found that there is a process of fertility decline in a Northern Bukidnon village. Therefore, the best way to show a starting point and the process of change in fertility behavior was to divide the Northern Bukidnon social system into three periods – the traditional period, the transitional period and the contemporary period. These period reflected simultaneous changes in patterns of mortality, economics and village/family organization. The data related to each period was then organized into single chapters: one chapter each for the traditional, transitional and contemporary periods. Since the major theme of this report contains two poles, the Northern Bukidnon family structure as opposed to fertility behavior or demographic change, each chapter should then contain these components and sub-components relevant to the topic. For example, the chapter for the traditional period is divisible into 2 major sub-components: family structure and fertility behavior. Under each sub-component, the more specific sections, topics and paragraphs should also be divided. Therefore, the serious outline for this chapter turns out as following. (Suppose this chapter is designed to be chapter six).
Chapter 6
The Transitional Period.
6.1 The Residential Patterns
6.2 Inheritance Patterns
6.2.1 The Transmission of Property
6.2.2 The Transmission of Authority
6.2.3 The Transmission of Kin Group Membership
6.3 The Roles and Duties of Family Members
6.3.1 Maternal Grandparents
6.3.2 Father/Husband and Mother/Wife
6.3.3 Married Daughter and Son-in-law
6.3.4 Role of the Son
6.3.5 Role of the Daughter
6.3.6 Role of Siblings
6.4 Old Age Security
6.5 Fertility Behavior Among the Traditional Northern Bukidnon
6.5.1 Marriage
6.5.2 Age at Marriage
6.5.3 Permanent Celibacy and Widowhood
6.5.4 Value of Children
6.5.5 Fertility Control
6.6 Chapter Summary
As it is now, the researcher has already formed three major chapters for this report designated as chapter six, seven, and eight, which are the core of the data analysis and report presentation.
According the researcher’s preliminary analysis (which is in her analytic files), she feels the need to have one chapter describing the Northern Thai social system itself. It is crucial for the readers to understand such aspects of Northern Thai life as marital behavior, residential and inheritance patterns, recruitment of kin group memberships, the value of children and old age security. Right before the analysis and presentation of family structure and fertility behavior which are chapters six, seven, and eight, she would present the information on Northern Bukidnon social system and designate it as Chapter Five.
However, the completed research report needs other parts also. These are introduction, literature review, theoretical framework, the setting of the community, research methodology and conclusion. Each one of these parts is a chapter in itself. The researcher could then map out his completed outline by adding these parts in and designate them as chapters one, two, three, four, etc. (See details of completed outline or table of contents in Appendix 1)
Up to this point, the researcher has merged the three sets of files together, and reorganized the data and the files according to the completed outline. The writing of the whole report is just begun.
Where to start? Different people have different objectives, styles of thinking and writing. The best way to start is with the topic she feels most comfortable in writing. For example, if a researcher feels most comfortable with the Northern Bukidnon social system, and feels that she is going to loose the grasp of it if she does not write about it right away, she may start with Chapter Five. Some people may feel the need to start from Chapter One and then go on to the end. In doing this, she might be able to see the flow and continuity of the report better. Some people may like writing a little bit on Chapter One and a little bit on Chapter Six. That is fine too. Please, however, note that these people have already created their completed outline. Without it, the researchers will be lost in the jungle, and it will take them a much longer time to find their way back and finish the report.
Conclusion
The main function of qualitative research, and thus the responsibility of each investigator, is to reveal underlying patterns in human behavior by identifying and showing relevant relationships (both direct and indirect) between significant variables be they in the physical, biological, socio-cultural or psychological dimensions. This can only be accomplished by analyzing and interpreting data obtained through observations, formal and informal interviews, as well as other tools, either qualitative or quantitative. An effective analysis rests firmly on the researcher’s ability to efficiently sort, organize, classify and file data, not vice versa. He can then put together the pieces of the puzzle, which means transforming data and experiences into concepts, and these into patterns of relationships and new ideas.
Bibliography
Arnold, D.O. 1982. Qualitative Field Methods. In A Handbook of Social Science Methods, Volume 2 : Qualitative Methods, Smith and Manning , eds, pp. 49-78 Cambridge : Ballenger Publishing Company.
Bogdan, R.C. and S.K. Biklen. 1975. Qualitative Research in Education. Boston : Allyn & Bacon.
Burgess, R.G. 1982. Styles and Data Analysis : Approaches And Implications. In Field Research : A Sourcebook and Manual, R.G. Burgess, ed, pp.107-110 London : George Allen & Unwin.
Cartright, D.P. 1966. Analysis of Qualitative Material . In Research Methods in the Behavioral Sciences, Festinger and Katz, eds, pp. 421-470. New York : Holt, Rhinehart and Winston.
Johnson, A.W. 1978. Quantification Anthropology : An Introduction to Research Design. Stanford : Stanford University Press.
Lofland, J. and L.H. Lofland. 1984. Analysing Social Settings : A Guide to Qualitative Observation Analysis. Belmont : Wadsworth Publishing Company, Inc.
Miles, M.B. and A.M. Huberman. 1984. Qualitative Data Analysis. A Sourcebook of New Methods. Beverly Hills, Ca: Sage Publications.
Patton, M.Q. 1980. Qualitative Evaluation Methods. Beverly Hills, Ca: Sage Publications.
Premsriratana, S. 1985. Ethnoclassifications and Diarrheal Diseases. Ramathibodi Vejasaan, 8:121-125.
Scrimshaw, S.C.M. and E. Hurtado. 1987. Rapid Assessment Procedures for Nutrition and Primary Health Care. Tokyo: The United Nations University.
Sipes, R.G. 1980. Population Growth, Society, and Culture : An Inventory of Cross-Culturally Tested Causal Hypotheses. New Haven : HRAF Press.
Spradley, J.P. 1979. The Ethnographic Interview. New York : Holt, Rhinehart & Winston.
________. 1980. Participant Observation. New York : Holt, Rhinehart & Winston.
Vong – ek, P. 1987. Influence of Beliefs on the Duration of Breastfeeding : A Comparative Study of Northeast and Central Thai Regions. Progress Report to the World Health Organization.
Yoddumnern, B. 1985. Continuity and Change in a Northern Thai Village : Determinants and Consequences of Fertility Decline on Northern Thai Family Structure. Unpublished Ph.D. Dissertation, University of Illinois at Urbana Champaign.
Appendix 1
TABLE OF CONTENTS
CHAPTER
1. INTRODUCTION
1.1 Background
1.2 Methodology
1.3 Organization
2. LITERATURE REVIEW
2.1 Thai Social and Family Structure
Predominant Interpretations
Key Features of Thai Family Structure
Family Structure with Special Reference to Fertility Behavior
Determinants and Consequences of Fertility Decline
Key Factors Involved in Fertility decline
Consequences of Fertility Decline
2.3 Chapter Summary
3. THEORETICAL FRAMEWORK
3.1 Davis and Blake Model of Social Structure and Fertility
3.2 Family Developmental Cycle
3.3 Individual Life Course
3.4 Chapter Summary
4. THE RESEARCH VILLAGE : SOCIAL AND HISTORICAL SETTING
4.1 Location and Brief Description of Village Development
4.2 Ethnohistory of Ban Dawn
4.2.1 Legend of Ban Dawn
4.2.2 Interpretation of the Legend
5. MAJOR CONCEPTS USED IN THE NORTHERN THAI SYSTEM
5.1 The Lineage Spirit and Spirit
5.1.1 The Lineage Spirit and Descent
5.1.2 The Lineage Spirit as Social Control
5.1.3 The Lineage Spirit as a Transition Marker in an Individual’s Life Course
5.1.4 The Lineage Spirit as a Source of Lineage Solidarity and Reciprocity
5.2 The Ritual Officiant, Shaman, and Spirit Festival Organizer
5.3 Hyan Kao (The Original House)
5.4 Hyan Kao (The Original House) Versus Hyan Kao Phii (The House Associated with the Spirit Shrine)
5.5 Chapter Summary
6. THE TRADITIONAL PERIOD (until 1913)
6.1 The Residential Pattern During the Traditional Period
6.2 Inheritance Patterns
6.2.1 The Transmission of Property
6.2.2 The Transmission of Authority
6.2.3 The Transmission of Kin Group Membership
6.3 The Roles and Duties of Family Members
6.3.1 Maternal Grandparents
6.3.2 Father/Husband and Mother/Wife
6.3.3 Married Daughter and Son-in-Law
6.3.4 Role of the Son
6.3.5 Role of the Daughter
6.3.6 Role of Siblings
6.4 Old Age Security
6.5 Fertility Behavior Among the Traditional Northern Thai
6.5.1 Marriage
6.5.2 Age at Marriage
6.5.3 Timing of the First Birth
6.5.4 Permanent Celibacy and Widowhood
6.5.5 Fertility Control
6.5.6 Value of Children
6.5.7 Sex Preference of Children
6.5.8 Desire for a Large Family
6.6 Chapter Summary and Discussion
7. TRANSITIONAL PERIOD (1913-1945)
7.1 Residential Pattern
7.2 Inheritance Pattern
7.3 The Roles and Duties of Family Members
7.4 Old Age Security
7.5 Fertility Behavior in the Transitional Period
7.5.1 Marriage
7.5.2 Age at Marriage
7.5.3 Fertility Control
7.5.4 Value of Children
7.5.5 Family Size
7.6 Chapter Summary and Discussion
8. CONTEMPORARY PERIOD (1945- present)
8.1 Modernization
8.2 Lineage Spirits
8.3 Residential Pattern
8.4 Inheritance Pattern
8.4.1 Transmission of Kin Group Membership
8.4.2 Transmission of Authority
8.4.3 Transmission of Property
8.5 Roles and Duties of Family Member
8.6 Old Age Security
8.7 Fertility Behavior in the Contemporary Period
8.7.1 Marriage
8.7.2 Age at Marriage
8.7.3 Fertility Control and Birth Spacing
8.7.4 Value of Children
8.7.5 Desired Family Size
8.8 Chapter Summary and Discussion
9. CONCLUSION
(Source: Yoddumnern, 1985: viii-xii)
=====================================================================================
I. PRESENTATION OF DATA IN QUANTITATIVE RESEARCH
1. Distribution
a. Frequency Distribution
1. Calculate the range of the data by subtracting the smallest score from the largest score, then add 1.
2. Divide the result obtained by derived number of intervals. This is the width of our interval.
3. tally scores
2. Tables
a. univariate table
b. Bivariate table
- describes two variables
- called contingency tables
- dependent variable – row
- independent variable – column
c. multivariate tables
- difficult to read.
3. Concepts
a. class interval
b. class width – difference between the lowest and highest number in a class interval
d. class limits
e. midpoint
f. open-ended class
4. Rules of Presentation
a. clarity – clear, without ambiguity and confusion
b. simplicity – readable
c. economy of space – no crowding
d. order of variables – dependent and independent variables presented in correct places.
e. appearance
g. accuracy
h. objectivity
5. Visual Presentation
a. graphs – consist of a relation and a body
body differs: circle, bars, column, maps, pictures
format – two lines planed at right angle
origin – point of intersection
coordinate axis
horizontal line – X- axis or Abscissa
vertical line Y axis or Ordinate
value of dependent variable
X-axis Abscissa
Value of independent variable
b. Type of Graphs
1. Frequency Polygon
2. Histogram – bars are adjacent to each other
3. Bar graph
bars are separate – indicating no quantitative relationship between them when independent variable is nominally sided can be vertical or horizontal.
4. scattergram – demonstrate the relationship between two
variables
- denotes the shape of relationship
5. normal curve
- bell shaped
- bilaterally symmetrical
- centered on the mean
- a curve in which tails approximate the abscissa but never touch it
- a curve in which 68.26 percent of the area of curve lie between -1 and +1 standard deviation; 95.44 percent between -2 and +2 standard deviation and 99.74 percent between -3 and +3 standard deviation.
6. pie chart – circle
7. population pyramid
8. cartograph
- use of symbols
- indicate presence, frequency or strength
- shading
9. pictograph
10. stem and leaf display
II. PRESENTATION OF DATA IN QUALITATIVE RESEARCH
a. Matrices
- a type of data presentation resembles equivalent of a table
- contains table, heading cells
- summary table containing verbal information, quotes, summarized text extract from notes
- forms:
• checklist matrix
• time-ordered matrix
• role-ordered matrix
• ______ theme (conceptualizing clustered matrix)
• effort matrix (outcomes)
• site dynamics matrix (processes and outcome)
• event listing – order of events
building matrixes
- construction relates more to personal ingenuity, competence and creativity
Rule-of-Thumb:
a. should be kept to one page display
b. include 15 to 20 variables in rows and columns
b. figures
c. charts
context chart
1. Presentation of Data in Quantitative Form
2. Statistical Measures in Univariate Analysis
a. relations measures
● Rate
- compare figures that are not related to the same variables
● Ratio – describes a relationship between parts of a group with each other
● Percentage – relates two subgroups to each other
percent made in the population
b. Measures of Location (central tendency)
Mean:
Mode: is the category with the largest number of observation
1. unimodal distribution – one mode
2. bimodal distribution – two modes
3. multimodal distribution – more than two modes
Median: point on a distribution that divides the observation into two equal parts
Ungrouped data:
- order scores in an array
- identify the scores but divides the distribution into half. (odd-numbered distribution) for even-numbered distribution the mean of two adjacent middle scores
What measure of location to choose:
a. type of measurement
b. shape of distribution
Guide for Determining Measure of Location
- the mode is chosen if the variable is nominally sided.
- The mean or median is chosen if the variable is ordinal interval or ratio.
- Skewed distribution, median is a better choice
When skewness is extreme and if distribution contains ordinal data, the mode may be a better choice.
- if further measures will be considered (e.g. standard deviation the mean will be preferred)
3. Measures of Dispersion
- how data are spread enough the mean
- show how close to or how far away from the main stream of the data the observation are (e.g. average QPI = 2.8; how low is the lowest and how high is the highest?
a. Variance – mean of the squared deviation of the observation from the mean.
Ungrouped data:
Computational Formula:
Grouped Data
BIVARIATE ANALYSIS
1. Measure of Association
2. Consideration for the Choice of the Measure
- symmetric/asymmetric
- its interpretation
- its sensitivity to confounding factors
3. The interpretation of Measures
- numerical value lies between 0 – 1
- nil association
- perfect association
- intermediate values: depend upon operational definition
4. Confounding Factors
5. Bivariate Procedures
First Variable Second Variable Chi – Square
Nominal
Ordinal
Interval/ ratio
Interval/ratio Nominal
Ordinal
Nominal
Interval ratio Analysis of variance
Kendall’s tau gamma
analysis of variance
correlation/regression
6. Measures of Association (nominal-nominal)
- percent
- cross-product
- chi-square
7. Interval Data
a. points to explore
- presence or absence of correlation
- direction of correlation
- strength of correlation
b. direction, strength and sample of the relationship
c. scatter plot
THE ANALYSIS OF NOMINAL DATA
1. Definition of nominal data/categorical variables
- nominal data are facts, attributes or properties that can be sorted into categories.
- Categories are identified by numbers that are arbitrary assigned.
2. Consideration
- analysis of nominal data requires to first determine which type of measure of association is most desirable.
- categories should be meaningful (materially exhaustive)
- distinguish dependent from variable which is the dependent variable appear
- By convention, the row variable which is the dependent variable appear first, then the column variable and the layer or control variable last.
3. Measures of Association
a. definition
- numerical index summarizing the strength or degree of relationship in a two dimensional classification.
b. considerations (guide to choice of a measure)
- the type of association whether symmetric or asymmetric.
- its interpretation
- its sensitivity to confounding influences.
- its sensitivity to confounding influences.
c. symmetric versus asymmetric measures/reciprocal meaning of relationship
d. The Interpretation of Measures of Association
- summarizes the information in a table
- numerical value lies between 0 and 1.
- zero when variable are completely unrelated and 1.0 if the variable are perfectly associated according to some criterion of nil or perfect association. The meaning of intermediate depends on how the measure is optionally defined.
e. nil association
- two variables association is nil implies that they are statistically independent [loosely speaking, statistical independence means that the probability of the joint occurrence of two events equals the product of probability of their separate occurrence].
Geographic residence
Political preference
Urban
Rural
Semi-rural Liberal
Moderate
conservative
In practical terms, knowledge of person, residence is no help in predicting his value of political preference because the values are unrelated. Measures of association then will be zero or close to zero indicating weak to nil relationship.
Note: A few measures of association are zero even in the absence of statistical independence. Lambda frequently equals to zero when the marginal totals are highly skewed but variables are not independent. (Inherent weakness in a measure).
4. Perfect Association
Ways of conceptualizing perfect association
a. strict perfect association – each value of one variable is uniquely associated with a value of the other.
NOTE: # of categories for X & Y must be equally
Totals 50
0
0
50 0
0
50
50 0
50
0
50
b. implicit perfect association
- one variable has more classes than another
- the members of a column in classification are as homogeneous as possible respect to y in a sense that there is only one nonzero row entry.
y
Totals
0
50
50
50
0
50 X
0
0
50
50
0
50
c. weak perfect association
- categories of X are as homogeneous with respect to by given the difference in the variables marginal totals.
Y
Total
50
50
50
150
0
0
50
50 X
0
0
50
50 Totals
50
50
150
200
5. Intermediate Values
- difficulty in interpreting intermediate values.
- turn to the measure’s operational definitions.
NOTE: chi-square do not have intrinsically appealing interpretation.
- look at the cross classification and examine each measure in order to group its underlying logic and meaning.
CONFOUNDING FACTORS
[chi-square is affected by sample size the greater the number the greater the value]
1. Skewed Margins Distribution : Two problem: skewed marginal distribution and unequal number of rows and columns.
- marginal distributions affect the numerical values of many measures of association.
a.
Y
total X
60.0%
(60)
30.0%
(30)
10%
(10)
100%
(100)
20%
(200)
60%
(600)
20%
(200)
100%
(1000)
10%
(10)
30%
(30)
60%
(60)
100%
(100) Total
270
660
270
1200
b.
Y X
60
(180)
30
(90)
10
(30)
100
(300)
20
(120)
60
(360)
20
(120)
100
(600)
10
(30)
30
(90)
60
(180)
100
(300) Total
330
540
30
1200
Observed: Table a, most of the ____ fall in the middle column. In b, cases are more evenly distributed among the categories of X. There are more variations on X.
Note that column percentage or (relative frequencies) are the same in both tables.
Hence: There is equivalence in the relationship (or measures) by parents. But many measures do not give the same value for both tables.
!!! Second set of data may yield index of strong relationship which analyzes the strong relationship while analyzing the ___ table may yield weaken association even though some statistics is being used. Only few indices are impressions to marginal distribution. When one or both variables are highly skewed, he should --- whether or not the relative absence of victim is substantially meaningful.
What to do: Select a less sensitive measure or adjust the observed data.
Important: One must be owner of the possible confounding effects of marginal distribution.
2. Non-square Tables
- some measures can not attain their maximum and are affected by non-equal rows and columns.
SOLUTIONS TO CONFOUNDING FACTORS
Especially ____ skewed marginal tables
1. Standardize or smooth a table of observed frequencies to inform to any set of derived marginal totals. The easiest method is to compute percents and test percents as though they are now frequencies. Percentaging effectively standardizes a variable because it consumes each category of the independent variable has exactly 100 cases; thereby, removing the effects of the unequal margins. Note: percentaging effects only one variable.
2. Compute a maximum version of a measure
- maximum given the table size or marginal distribution and then divide this ____ with the observed value.
Ex. Maximum value of a particular measure .6
Observed value .3
3/.6 = .5 this adjusted value has eliminated the constraints improved by extraneous factors.
Measures of Associations (2 x 2 tables)
Note: It is not advisable to collapse or reduce larger array into 2 x 2 tables. As collapsing introduce distortion and procedure misleading results.
1. Percents
- one of the easiest ways to measure relationship especially if one is clearly a dependent variable.
- If the distribution of responses changes from one category to another, there is evidence for a relationship.
Political Preferences Urban Semi-urban Rural Total
Liberal
Moderate
Conservative 33
(193)
41
(241)
26
(153)
100 30
(161)
37
(199)
34
(182)
100 11
(461)
33
(134)
54
(229)
100
400
547
567
Total 587 562 409 1538
- A difference in percents can be interpreted as a regression coefficients between two dichotomous variables ( a regression coefficients given the magnitude of a change in y, for a unit change in x).
- Example:
x difference
y .9
(45)
.1
(5) .4
(20)
.6
(30) .5
1.0
(50) 1.0
(50)
A change in one unit of X (from 0 to 1) produces a change of 5. This result indicating a substantial relationship.
- gives a clear interpretation and not sensitive to imbalances in the marginal distribution of X.
MEASURES OF ASSOCIATION
1. What is a measure of association
- numerical index summarizing the strength or degree of relationship in two-dimensional cross classification.
2. Consideration which guide the choice of a measure
a. symmetric/asymmetric
b. its interpretation
c. its sensitivity to confounding influence
3. The interpretation of Measures of Association
- The numerical value of most measures lies between 0 and 1.
- Zero if variables are completely unrelated.
- one if the variables are perfectly associated.
a. nil association
- implies statistical independent (statistical independence means that the probability of the joint occurrence of two events equals the product of the probability of their separate occurrence)
Thus: Knowledge of ____ score X is no help in predicting his value on y because the variables are unrelated to work after.
- values close to zero typically indicate a weak to nil relationship
- few measures of association are zero even in the absence of statistics independence Lambda frequently equals zero when the margins totals are highly skewed – that is – most cases in one category but the variables are not independent.
b. perfect association
- classified variables often represent measurement errors where individuals are ____ together into categories out of convenience or necessity.
WAYS TO CONCEPTUALIZE PERFECT ASSOCIATION
1. Strict perfect association
- each value of one variables is uniquely associated with a value of the other.
x
Y 50
0
0 0
0
50 0
50
0
Totals 50 50 50
Knowledge of a person X category implies perfect prediction of his score or y measure of association equal 1.0
2. Implicit Perfect Association
- one variable frequently has more classes than another (the rows and columns are not equal)
Example:
X
Y
Total 0
50
0
50 50
0
0
50 0
0
50
50 50
0
0
50
The numbers of a column classification are as homogeneous as possible with respect to y in the sense that there is only one non-zero row entry per column. Different X categories are generally associated with different y categories but since the classes on X outnumber those on y, the association is not unique.
3. Weak Perfect Association
- categories of X are as homogeneous as possible with respect to y, given the differences in the variable marginal totals.
Example:
X
Y
Total 50
50
50
150 0
0
50
50 0
0
50
50
4. Intermediate Values
How would one make sense of a values being between 0 and 1.0. Suppose the value of an index is .45, what will be the conclusion about the strength and form of relationship?
Answers lies on measures operational definition. Chi-square in this context do not have intuitively appealing interpretation. Thus each measure has to be examined separately in order to group its underlying logic and meaning.
5. Confounding Factors
- extraneous factors frequently confuse one’s interpretation. It is well known that number of cases affects the magnitude of chi-square statistics: The greater the sample size, the larger the value of chi-square statistics.
Almost all measures of association eliminate the effect of sample sizes but similar types of factors can influence their numerical values. Two most common problems are:
1. Skewed marginal distribution
2. Unequal numbers of rows and columns
a. Skewed Marginal Distribution
Example A
Example B
X X
y 60
30
10 20
60
20 10
30
60 60
30
10 20
60
20 10
30
60
A B
Number
Total 60
30
10
100 200
600
200
1000 10
30
60
100 270
660
270
1200 180
90
30
300 120
360
120
600 30
90
180
300 330
540
330
1200
In table A most of the cases fall in middle column, in table B, they are more evenly distributed among the categories of X. But the column percentages are the same in both tables.
!!! A researcher who computes on index for the second set of data might find a strong relationship while someone analyzing the first table might report a much weaker association, although both use the same statistics.
!! Pay particular attention to marginal table. If one or both variables are highly skewed decide whether or not the relative absence of variation is substantially meaningful.
!!!! Adjust observed data or select a less sensitive measure or the lack of variation may itself be theoretically important.
6. Solutions to confounding factors
a. skewed marginal total
- standardize or smooth a table of observed frequencies to conform to a desired marginal totals. Easiest method is to compute percent and ____ percent as though they are now frequencies. Percentaging effectively standardizes a variable ____ it assumes each category of the independent variable has exactly 100 cases thereby removing the effects of unequal marginals.
b. Compute a “maximum” version of a measure
- maximum given the table size or marginal distribution and divide this version into the observed values.
Ex: for a given set of marginal totals, the maximum value of a particular measure is 6.6. Divide this quantity .6 into the observed value, say .3 to obtain in adjusted value of .5 – the adjusted measure partially eliminate the constraints improved by the extraneous factors.
7. Measures of Association for 2 x 2 tables
- best known and most extensively studied type of cross-classification
- little advantage in collapsing/reducing a larger among into a 2 x 2 table.
Collapsed data frequently introduced distortions. Weak relationship in table larger than 2 x 2 could turn out to be a large association if the variables are dichotomized. Produce misleading results.
a. Percent
- easiest way to measure relationship between two variables especially if one is clearly dependent variable. Compare how people in different categories behave with respect to the classes of another. If the distribution of responses changes from one category to another, there is evidence of a relationship.
Example: Table: Relationship between Ethnicity and Political Analysis
Christian M T
Liberal 193
33 161
30 46
21 400
Moderate 241
41 199
37 134
33 547
Conservative 153
26 182
34 229
56 564
587 542 409 1 538
Percents are particularly useful in 2 x 2 tables. A difference in percents or proportions can be interpreted a regression coefficient between two dichotomous variables. (A regression coefficient gives the magnitude of a change in y for a ___ change in the independent variable.
Consider:
X
y .9
(45)
.1
(5)
1.0
(50) .4
(20)
.6
(30)
1.0
(50)
Difference in proportions with respect to fist row is .5. The same quantity should be obtained if the categories of x and y are coded 0 and 1 and the data substituted into familiar regression formula. A change in one wink of x (from 0 to 1) produces a change 7.5 in y. Given the range of possible values (0 to 1), this result indicates a substandard relationship.
b. cross-product ratio
- called odds ratio
- underlies two popular measure of _____ and has several useful properties
- provides an understanding of log linen analysis; a categories multi-variate technique
Simplified Version for Convenience purpose
Christian Muslim Total
Liberal
Conservative 193
153 46
229 239
382
Total 346 275 621
Obviously the variables are related. But how strongly? Comparing the odds of being liberal.
For Christian, these odds are
193/153 = 1.26
or 1.3 to 1
odds of being liberal among Muslim
46/229 = .20
If ethnicity is unrelated to ideology, the odds of being liberal should be the same for our ethnic groups. The odd of being liberal among Muslim is considerably less than one.
Calculate this ratio:
The ratio of the odds (denoted by ) has a simple interpretation. If they are the same in both categories of ethnicity, their ratio will equal to 1.0
If a hypothetical data:
y x
45 90
15 30
Is
Hence, no relationship. Departure in either direction from 1.0 suggests association. The greater the departure, the stronger the relationship.
c. Properties of odds ratio
The odds ratio range from 0 to as with 1.0 indicating statistical independence. Values less than 1.0 imply a “negative” association while values greater than 1.0 mean a positive relationship.
Examine the following:
A B
100
25 50
200 25
200 100
50
125 250 225 150
The odds ratio for A = 16.0 in B = .0625
The B is simply similar with A. Frequencies are related thus maintaining the same underlying strength of association.
In this since the two tables reflect similarity in the magnitude but not in the direction of the relationship.
The lack of symmetry is truly easily removed by calculating the natural log of .
Properties:
1. Odds ratio are ____ under row and column multiplication
Example:
Y A B
75
10 15
100 750
100 15
100
85 115 200 850 115 965
This insensitivity to marginal distribution is quite useful because the inherent relationship appear essentially equivalent.
2. ____ under ____ of rows and columns
Estimate Variance of Odds Ratio
Measure Based on the Chi-Square
Phi Squared
One reason for not using as a measure of association is that its numerical magnitude depends partly on the size of the sample.
Dividing Chi Square by n corrects for n and leads to a popular measure of association phi squared
varies between 0 and 1. zero when the variable are statistically independent sensitive to marginal totals.
5. Correlation Coefficient
r2 = .16 16% of the variance in ideology is accounted by ethnicity.
- sensitive to skewed marginal distribution equivalence to
Cosines of Associations of I x J tables
1. The odds Ratio in I x J tables
- involve looking at several individual odds ratio
- permit one to examine various subhypothesis
- to locate the precise source of association
- east (the bottom right hand) all of the table is the reference point:
Urban Semi-urban Rural
Liberal
Moderate
Conservative 193
241
153 161
199
182 46
134
229 400
547
564
Total 587 542 409 1 538
Where I = row and J = column
t = (I – 1)(J = 1) 2 x 2 table
t = (3 – 1)(3) 2 x 2 table
= (2) (2) = 4 2 x 2 table
- hence, calculate 4 .
Disadvantage: to ____ for larger table
Corresponding odds ratios:
11 = (193/153) / (46/229) = 6.28
21 = (241/153) / (134/229) = 2.69
12 = (161/153) / (134/229) = 4.42
22 = (199/182) / (134/229) = 1.86
Phi Squared
- does not have an upper ___ except in 2 x 2 tables (where it vanishes between 0 and 1)
- difficult to interpret
2. Contingency Coefficient
- lies between 0 and 1
- sample estimate given by
- does not reach 1.0 even when variables seem completely associated
3. Proportional Reduction in Error Measures (PRE)
- rationale
- properties
1. skewed distribution of y may yield PRE measures on zero even if the variables are not statistically independent.
2. limit is 0 and 1.0 zero when x and y are independent 1.0 when they are completely related.
- intermediate values have clear interpretation.
Thursday, July 30, 2009
Tuesday, July 7, 2009
Data Analysis
Chapter 6 - More about Data Analysis
When the fieldwork is done and the data entry completed, the fun really begins. To illustrate some more principles of data analysis, let us assume that you are analyzing a public opinion poll. The first thing you want to see is the marginal frequencies: the number and percentage of people who have each of the possible responses to each of the questions in the survey. Determining this basic information is not as clear-cut as it sounds, however, and a few policy decisions must be made in advance.
First among them is the problem of dealing with answers of the don't-know, no-opinion, and no-answer variety. Do you leave them in the base for calculating percentages, or do you take them out? It can make a difference. Suppose you ask 500 people, “On the whole, do you approve or disapprove of the way the mayor is handling his job?” and you get the following distribution:
Approve
238
Disapprove 118
Don't know 104
No answer 40
If you base the percentages on the total sample of 500, you find:
Approve
48%
Disapprove 24
Don't know 21
No answer 8 (n = 500)
The total in this case is 101 percent because of rounding errors. No need to be compulsive about that. If survey research were a totally precise and reliable instrument, you might be justified in reporting fractional values. But it isn't, and using decimal points gives a false sense of precision which you may as well avoid.
Now looking at the above percentages, the sensation-seeking beast that lurks in all of us spots an opportunity for an exciting lead: “Mayor Frump has failed to gain the approval of a majority of the adult residents of the city, an exclusive Daily Bugle poll revealed today.”
However, it is possible to give the mayor his majority support by the simple expedient of dropping the “no answers” from the percentage base. Using the same numbers based on the 460 who responded to the question, we find:
Approve
52%
Disapprove 26
Don't know 23 (n = 460)
Mayor Frump suddenly looks better. Much of his looking better, of course, is based on the artificial distinction between a minority and a majority. The four-point difference would not seem nearly as important if the range were, say, from 42 to 46. And since no election is involved here, the question of majority support is not particularly germane. Moreover, the apparent majority or lack of it could be due to sampling error. Artificial as the distinction may be, however, it is one that can quickly catch the reader's eye and one that will be overemphasized despite your best efforts to keep it in perspective. The choice of a base for computing percentage is therefore crucial.
There is yet a third possibility, basing the percentages on the total number of people who have opinions about the mayor:
Approve
67%
Disapprove 33 (n = 356)
Now the mayor looks very good indeed, especially when we consider the likelihood that the “don't know” segment is also the least informed. The public relations staff at City Hall can leap on this and claim, with some justification, that informed citizens approve of the mayor by a ratio of two to one.
Deciding what to count
So here you sit with a survey containing perhaps two hundred questions, and each of them is subject to three different interpretations. You are a writer of news stories, not lengthy scholarly treatises. What do you do? A rule set forth in the previous chapter is so important that it is worth repeating here:
“Don't know” is data.
The soundest procedure is to base your percentages on the nonblank answers, as in the second of the three examples cited above. It is theoretically justifiable because not answering a particular question is in somewhat the same category as not responding to the entire questionnaire. The reasons for no answer are varied: the interviewer may have been careless and failed to mark that question or failed to ask it, or the respondent may have refused to answer. In any case, failure to answer may be treated as not being in the completed sample for that particular question. You should, of course, be on the lookout for items for which the no-answer rate is particularly high. They may be a tipoff to a particularly sensitive or controversial issue worth alerting your readers about; and you will, of course, want to warn the reader whenever you find meaningful responses that are based on considerably less than the total sample.
Usually, however, the no-answer rate will be small enough to be considered trivial, and you can base your percentages on the nonblank answers with a clear conscience and without elaborate explanation.
The don't-know category is quite different. The inability of a respondent to choose between alternatives is important information, and this category should be considered as important data–as important as that furnished by people who can make up their minds. In an election campaign, for example, a high undecided rate is a tipoff that the situation is still unstable. In the example just examined it suggests a substantial lack of interest in or information about the mayor–although these are qualities best measured more directly.
Therefore, you should, as a matter of routine, include the don't-knows in the basic frequency count and report them. When you judge it newsworthy to report percentages based on only the decided response, you can do that, too. But present it as supplementary information: “Among those with opinions, Mayor Frump scored a substantial . . .”
When you do your counting with a computer, it is an easy matter to set it to base the percentages on the nonblank answers and also report the number of blanks. If you are working with SAS or SPSS, the frequency procedures will automatically give you percentages both ways, with the missing data in and out.
Beyond the marginals
Either way, you can quickly size up your results if you enter the percentages on an unused copy of the interview schedule. Before going further, you will want to make some external validity checks. Are males and females fairly equal in number? Does the age distribution fit what you know about the population from other sources, such as census data? Does voting behavior fit the known results (allowing for the expected overrecall in favor of the winning candidates)? With any luck, each of these distributions will fall within the sampling error tolerances. If not, you will have to figure out why, and what to do about it. Once you know the percentage who gave each of the alternative responses to each of the questions, you already have quite a bit to write about. USA Today can produce a newspaper column from three or four questions alone. However, the frequencies – or marginals, as social scientists like to call them – are not the entire story. Often they are not even very interesting or meaningful standing by themselves. If I tell you that 75 percent of the General Social Survey's national sample says the government spends “too little” on improving the environment, it may strike you as mildly interesting at most, but not especially meaningful. To put meaning into that 75 percent figure, I must compare it with something else. If I tell you that in a similar national sample two years earlier, only 68 percent gave that response, and that it was 59 percent four years earlier, you can see that something interesting is going on in the nation. And that is just what the General Social Survey did show in the years 1989, 1987, and 1985. A one-shot survey cannot provide such a comparison, of course. However, if the question has been asked in other surveys of other populations, you can make a comparison that may prove newsworthy. That is one benefit of using questions that have been used before in national samples. For example, a 1969 survey of young people who had been arrested in December 1964 at the University of California sit-in used a question on faith in government taken from a national study by the Michigan Survey Research Center. The resulting comparison showed the former radicals to have much less faith in government than did the nation as a whole.
Internal comparisons
Important opportunities for comparison may also be found within the survey itself. That 75 percent of Miami blacks are in favor of improving their lot through more political power is a fact which takes on new meaning when it is compared to the proportion who favor other measures for improvement. In a list of possible action programs for Miami blacks, encompassing a spectrum from improving education to rioting in the streets, education ranked at the very top, with 96 percent rating it “very important.” Violent behavior ranked quite low on the list.
And this brings us anew to the problem of interpretation raised in the opening chapter of this book. You can report the numbers, pad some words around them, the way wire-service writers in one-person bureaus construct brief stories about high school football games from the box scores, and let it go at that, leaving the reader to figure out what it all means. Or you can do the statistical analog of reporter's leg-work, and dig inside your data to find the meaning there.
One example will suffice to show the need for digging. A lot has been written about generational differences, particularly the contrast between the baby boomers and the rest of the population. And almost any national survey will show that age is a powerful explanatory variable. One of the most dramatic presentations of this kind of data was made by CBS News in a three-part series in May and June of 1969. Survey data gathered by Daniel Yankelovich, Inc., was illustrated by back-to-back interviews with children and their parents expressing opposite points of view. The sample was drawn from two populations: college youth and their parents constituted one population; noncollege youth and their parents the other. Here is just one illustrative comparison: Asked whether “fighting for our honor” was worth having a war, 25 percent of the college youth said yes, compared to 40 percent of their parents, a difference of 15 percentage points.
However, tucked away on page 186 on Yankelovich's 213-page report to CBS, which formed the basis for the broadcasts, was another interesting comparison. Among college-educated parents of college children, only 35 percent thought fighting for our honor was enough to justify a war. By restricting comparison to college-educated people of both generations, the level of education was held constant, and the effect of age, i.e., the generation gap, was reduced to a difference of 10 percentage points.
Yankelovich had an even more interesting comparison back there on page 186. He separated out the noncollege parents of the noncollege kids to see what they thought about having a war over national honor. And 67 percent of them were for it. Therefore, on this one indicator we find a gap of 32 percentage points between college-educated adults with kids in college and their adult peers in noncollege families:
Percent saying "honor" worth fighting a war
College youth
25
10% difference
College parent of
college child
35
32% difference
Noncollege parent of
noncollege child 67
Obviously, a lot more is going on here than just a generation gap. The education and social-class gap is considerably stronger. Yankelovich pursued the matter further by making comparisons within the younger generation. “The intra-generation gap, i.e., the divisions within youth itself,” he told CBS a month before the first broadcast, “is greater in most instances than the division between the generations.”
The same thing has turned up in other surveys. Hold education constant, and the generation gap fades. Hold age constant, and a big social-class gap –a wide divergence of attitudes between the educated and the uneducated–opens up. Therefore, to attribute the divisions in American society to age differences is worse than an oversimplification. It is largely wrong and it obscures recognition of the more important sources of difference. CBS, pressed for time, as most of us usually are in the news business, chose to broadcast and illustrate the superficial data which supported the preconceived, conventional-wisdom thesis of the generation gap.
Hidden effects
Using three-way cross-tabulation to create statistical controls can also bring out effects that were invisible before. When Jimmy Carter ran for president in 1976, the reporters using old-fashioned shoe-leather methods wrote that his religious conviction was helping him among churchgoers. Then the pollsters looked at their numbers and saw that frequent churchgoers were neither more nor less likely to vote for Carter than the sinners who stayed home on Sunday.
These data from a September 1976 Knight-Ridder poll illustrate what was turning up:
Highly Religious
Not So Religious
Carter 42% 38%
Ford 47 52
Not voting or DK 11 10
Total 100 100
Carter support was four points greater among the “highly religious” than the “not so religious” (42 to 38). But the difference was not statistically significant. As it turned out, however, the shoe-leather guys were right. There was a religin effect if you knew where to look for it. Carter had a strong appeal to young people, and young people tend to be less religious. Carter's religiosity did not have much effect on older people whose political beliefs were well established. The religion appeal worked mainly on the young. Variables that conceal effects this way have been called “suppressor and distorter variables” by Morris Rosenberg. The way to find the effect is to look at Carter support by churchgoing behavior within each age group. When that was done, a strong church effect favoring Carter appeared among those aged 18 to 41.
Highly Religious Not So Religious
Carter 49% 38%
Ford 43 52
Not voting or DK 8 9
Total 100 100
The two above examples are rather complicated, and you can't be blamed for scratching your head right now. Let's slow down a bit and poke around a single survey. I like the Miami Herald's pre-riot survey as a case study because of its path-breaking nature, and because the analysis was fairly basic. We shall start with a simple two-way table. A two-way (or bivariate) table simply sorts a sample population into each of the possible combinations of categories. This one uses age and conventional militancy among Miami blacks. In the first pass through the data, age was divided four ways, militancy into three.
AGE
15-24 25-35 36-50 Over 50 Total
Low 23 28 34 45 130
MILITANCY Medium 65 60 65 56 246
High 23 44 38 19 124
Because the marginal totals are unequal, it is hard to grasp any meaning from the table without converting the raw numbers to percentages. Because militancy is the dependent variable, we shall base the percentages on column totals.
AGE
15-24 25-35 36-50 Over 50 Total
Low 21% 21% 25% 37% 26%
MILITANCY Medium 59 45 47 47 49
High 21 33 28 16 25
Percent of N 22 26 27 24 100
The marginal percentages are based on the total 500 cases. Thus we see at a glance that 26 percent are in the low-militancy category, 49 percent in the medium group, and 25 percent in the high group. Age is distributed in nearly equal categories. Looking across the top row of cells, we can also see that the proportion of low militancy tends to increase with age. And the greatest percentage of high militancy is found in the 25-35 group.
There are too many numbers here to throw at your readers. But they mean something (the chi-square value –computed from the raw numbers – is 20, which, with six degrees of freedom, makes it significant at the .003 level). And the meaning, oversimplified–but honestly oversimplified, so we need make no apology–is that older people aren't as militant as younger people. We can say this by writing with words and we can also collapse the cells to make an easier table.
AGE
15-35 Over 35
Low Militancy 21% 31%
Medium and High Militancy 79 69
100% 100%
This table also eliminates the marginal percentages. The sums at the bottom are just to make it clear that the percents are based on column totals.
The problem of figuring which way the percentages run may seem confusing at first, but eventually you will get the hang of it. To make it easier, most of the tables in this book base the percentages on column sums. Thus the dependent variable –the quality being dissected – is listed across the rows. No law of social science requires this arrangement. We could just as logically put the dependent variable in columns and figure percent across rows. In some cases, to clarify a distribution, you may want to base percentage on the table total, i.e., the sum in the corner of the margins. But for now we shall standardize with the dependent variable reported across rows and the percentages based on totals down the columns.
Standardizing your tables
With this standard presentation, you can quickly gain the habit of letting your eye rove from left to right to find the percent difference. The table comparing the proportion of younger and older people among the militants shows older people have half again as many conservatives or low militants among them as younger people have: 31 percent vs. 21 percent. And that is a fairly good way to explain it to the reader.
But if it takes you some time to develop your table-reading skill so that the figures leap off the page at you, bright and meaningful, consider the plight of the reader. Your practice and skill at interpretation do not help him or her, and so you have to reduce things to words or to bare numerical essentials, or both in combination. One way to ease the burden on the reader is to give him or her one-way tables, created by lopping off the less dramatic half of the two-way table. Even then, you should add some words to tell what the numbers are saying. Here is how the relationship between militancy and age could be presented:
MILITANCY AND AGE
Older people tend to be more conservative than younger people.
15-35 years Over 35 years
Percent conservative 21 31
The other half of the table, the percent nonmilitant in each category, is implied. And the table makes clear that the numbers are percentages within age categories.
What about the don't-knows? Shouldn't there be cells in the table for them? Not in this case. The militant was operationally defined as someone who gave militant answers to six of the eight questions in the index. “No answer,” therefore, counted as a nonmilitant answer. Had the number of blanks for any one of the items been unusually high, some theoretical problems could have been raised. They weren't. In a few cases, interviewers failed to record age, and those cases were automatically eliminated from the table.
Now we must reflect a bit on what we have added to the newspaper reader's knowledge here. We have made a comparison between young and old and demonstrated that militancy is found more frequently among the young. That in itself is something of an achievement because it provides a fuller description of the phenomenon of militancy than was available before we produced the table. Age and militancy are related.
Can we go beyond that and assume that the relationship involves causation? Nothing in the numbers themselves demonstrates that the relationship is a causal one. To make the leap to a causal assumption, we have to use some logic, a bit of intuition, and common sense. And that, you may remember from an earlier chapter, is why we keep referring to one variable as the dependent variable and the other as the independent variable. This description is the most parsimonious and conservative one available. We can say that we are looking for evidence that militancy depends on age without committing ourselves to talking about causation. The statement that the amount of militancy depends on age is purely a descriptive one. (Or, if the row comparisons had turned out to be about equal, we could have said that militancy does not depend on age.)
Now let us look at this particular table and bring some logic to bear on it. If there is a relationship between the two variables, then it could be because one of them is a cause of the other. But which way does the causal arrow run? In this case, it is easy to eliminate the hypothesis that militancy causes age. Chronological age, more's the pity, is fixed and unchangeable, so the arrow must run the other way. Other readily measured attributes possess the same advantage in deducing causality: sex, race, birth order within the family are examples. Because they are unchangeable, we can assume that they are causes and not effects, if there is causation.
For illustration, we return to the case of the Miami blacks. Militancy can't cause people to be young, but the reverse proposition, that being young causes them to be militant, also lacks a good deal. What we really want to know is what it is about being young that makes people militant.
The way to find out is to look first for other dimensions where young people differ from old: education, for example. People in all social classes tend to be better educated than their parents. Among Miami blacks more than 24 years old, 25 percent had high school diplomas. Only 12 percent of their fathers were high school graduates. Furthermore, we can expect militancy to increase with education.
Knowing this, we then suspect that it may not be youth per se that causes militancy, but merely the fact that young blacks are better educated, and that better education is the real cause of militancy. To test this idea, we first verify our suspicion that education is related to militancy. And indeed it is. Reducing the table to a newspaper-compatible form produces the following evidence that militancy increases with education.
LEVEL OF EDUCATION
Grade
school Some high
school High school
graduate Beyond
high school
Percent militant 16 23 32 38
(The full table, with seven categories of education and three of militancy, has a chi-square value of 33, significant at the .001 level with 12 degrees of freedom.)
At first glance, education seems to have even more to do with militancy than does age. Perhaps age is not a “real” factor at all. Maybe it is simply a cause of education, which is then the more direct cause of militancy.
To test this idea, we introduce what Morris Rosenberg calls a test factor. Another word for it is control. As in the CBS report, we want to control for age–this time to examine the relationship between militancy and education. Controlling for age means to hold its effect constant by looking at the effect of education on militancy within each age category. This process is analogous to the laboratory scientist who, by repeating his experiment in a variety of temperatures, makes certain that variation in room temperature is not affecting his chemical reaction.
Tables with three dimensions
The result of such an examination is a three-way table: militancy by education by age. In physical printout, it is a two-way table repeated for each educational category–the only way that a three-dimensional table can be reproduced on two-dimensional paper.
Several things might happen:
1. The education-militancy relationship could disappear within each of the age groups. If so, age, not education, is the cause of militancy. Age, in fact, is a common cause of both education and militancy and the apparent education-militancy relationship is spurious.
2. The education-militancy relationship could remain in each of the age categories. In this case, age and militancy are both links in the causal chain. Logic tell us that age, being fixed, comes first. Therefore, youth causes education and education causes militancy.
3. The education-militancy relationship could disappear in some of the age categories and remain the same or become stronger in others. If so, some kind of interaction effect is operating. There are special circumstances in which age and education work together to increase militancy–more, perhaps, than simple addition of their separate effects would account for.
Which of the above really happened in the Miami case?
To make it easy, we'll again collapse the tables to one dimension, looking at the percent militant in each age group, starting with the youngest. Remember, we are looking not at one table but four.
Grade
school Some high
school High school
graduate Beyond
high school
Age 15-24
Percent Militant * 15 25 36
Age 25-35
Percent Militant * 25 39 48
Age 36-50
Percent Militant 17 32 33 *
Age over 50
Percent Militant 14 15 23 *
* Number in cell too small to computer percentage
As you can see, education and militancy are related in all age categories. The second of the three hypotheses is sustained. It is not the ebullience of youth so much that produces militant attitudes. Rather, it is the greater educational opportunity open to young people, which in turn increases their militancy. This finding fits neatly into a larger theoretical framework of rising aspirations: education moves people closer to goals and increases their expectations. Higher expectations, left unfulfilled, produce frustration and militancy.
This example deals with a relationship between what Rosenberg calls a “property” (age) and “disposition” (militancy). Properties are unambiguous, things an individual is or isn't: white, German-born, grammar school graduate, blue-collar worker, 1968 Nixon voter, color-television owner, teetotaler, licensed airplane pilot, pack-a-day cigarette smoker. Dispositions are more difficult to get a grip on because they are qualities that need the presence of special situations in order to be activated. The black militant, i.e., a black person scoring high on the militancy scale, may or may not express his or her disposition by behaving in a militant manner, depending, normally, on the external situation he or she faces at any given moment. But the disposition is there and it is measurable. One of the things that some social scientists try very hard to do is to establish relationships between dispositions and behavior. Another school, however, holds that dispositions –the attitudes, values, personality traits within the person – have much less to do with behavior than does the external situation.
It is generally much easier to establish the relationship between a property and a disposition than to predict actual behavior, as in the case of the age-education-militancy chain. However, you can find news applications of both types, although you may find that you are unable to interest your editors or your readers in attitudinal relationships until dispositions are manifested in newsworthy behavior. We become interested in what blacks or students or public school teachers are thinking after they riot or eject the dean from his office or go on strike. Nevertheless, a news organization that takes the new methods seriously will try to monitor the attitudinal developments before matters reach the overt, obviously newsworthy, man-bites-dog level. If a man biting a dog is news, a man thinking about biting a dog is a potential scoop.
If a disposition is the dependent variable, i.e., the thing being studied, you then can quickly find yourself in the business of searching for relationships between one disposition and another. In the Miami Herald's pre-riot survey of early 1968, one of the important dependent variables was disposition toward violence. It was measured with two questionnaire items, one dealing with general approval of violence to advance black goals and the other asking about the respondent's own intent to participate in such violence should the opportunity arise. The two items were combined to form a four-part index ranging from “violent” –those who both favored violence as an idea and were ready to participate themselves –to those who were opposed to both the concept and the personal act. It was then tested against a number of specific grievances to find out, first, whether grievances really do dispose people toward thoughts of violence, and, if so, which grievances had the most effect.
One of the more clear-cut tables showed quite plainly that disposition toward violence was associated with dissatisfaction over one's personal housing situation.
ATTITUDE TOWARD HOUSING
Happy Unhappy
ATTITUDE TOWARD VIOLENCE Violent 7% 12%
Near violent 8 18
Ambivalent 25 24
Opposed 69 45
100 100 (n = 478)
However, young people tend to be more educated and more discontent and also more violent. The question arises, then, whether discontent with housing really has an effect on feeling toward violence, or whether the two variables are merely common effects of a more complicated bundle of attitudes that involved being young and black. So the table was run again, this time controlling for age. To conserve cases, the four violence categories were collapsed to two and the tables were reduced to one dimension apiece for simplicity.
ATTITUDE TOWARD HOUSING
Happy Unhappy
Age 15-24
Violent or near violent 21% 33%
Age 25-35
Violent or near violent 19 29
Age 36-50
Violent or near violent 12 40
Age over 50
Violent or near violent 12 20
The relationship persists in each age group, though it comes on significantly stronger in the middle-aged, 36-50, group. Therefore, at least two things are probably going on here: first, youth, or something about youth, causes people to feel and express grievances over housing and the grievances in turn stimulate favorable attitudes toward violence. But look at the third age category. Housing is strongest as an explanatory variable for violence in that age group for which it is most relevant: these are people in the middle-to-late child-rearing years for whom being deprived of satisfactory housing can be the most frustrating. Thus bad housing makes these people disposed toward violence despite their age; age, negatively associated with violence (the greater the age, the less the violence) has thus had a suppressor effect in the previous two-way table, which tested for the effect of housing dissatisfaction on violence.
Or, recasting this statement in shorter sentences:
Disposition toward violence decreases with age.
Disposition toward violence increases with housing dissatisfaction.
Among middle-aged people, housing dissatisfaction relates so strongly to disposition toward violence as to outweigh the violence-suppressing effect of age. When you can pinpoint a group in which special circumstances make a relationship stand out with extra power and clarity you have a strong potential for an important news story. In Miami, urban affairs writer Juanita Greene found that housing frustrations for people in the child-rearing years were grounded in objective reality. The gap between housing need and housing availability was large and it was not being closed. This kind of spotting of special situations, by the way, is something that social scientists, in their quest for general theoretical principles, do not often do very well. “Social science has been justly criticized,” says Rosenberg, “for its neglect of situational factors. One may know that general principles obtain, but one does not know whether these principles have predictive value in specific circumstances.” Digging for specific circumstances may fit the instincts of the journalist better than those of the social scientist. Academic reluctance to ferret out the mundane details needn't inhibit us at all as we look for the traditional who, what, when, where, why, and how.
Before leaving the violence-by-housing and satisfaction-by-age tables as compressed above, scan it once more to see how quickly you are becoming accustomed to letting the numbers leap off the page, bearing their message to you. Your eye automatically scans from left to right to show that housing attitudes relate to violence attitudes in all age groups. But you can also scan it vertically to see how attitudes favoring violence tend to fade with advancing age. The percent disposed to violence fades from 21 to 12 among those happy with housing conditions and from 33 to 20 among those unhappy with housing conditions. Furthermore, the exception, that 40 percent violent among the unhappy 36-50 group, cries out for attention and explanation.
If, on the other hand, you still have to stop and puzzle out the meaning, don't worry. You are, after all, in the business of writing with words, not numbers. But facility with the numbers will come with practice, particularly as you apply them to work of your own and not just to examples printed in a book. You may also take comfort from this thought: you are avoiding, so far, the danger of becoming so fluent with numbers that you begin to lose your ability to put their meaning into words that newspaper readers can understand and appreciate. This hazard is well known to social scientists, especially among students getting a grip on quantitative methods for the first time. They sometimes reach a stage in which they resent the fact that they can't simply send the computer printout to the publishers and have its significance instantly appreciated and spread upon the record for all the world to see. It doesn't work that way for them and it especially doesn't work that way for us in journalism. We write with words, but we must learn to read in numbers.
More than one independent variable
We have now seen some of the things that can happen to a simple two-variable relationship when a third variable, related to both, is introduced. One more example deserves some examination. This is the case where the third variable is not really a test or control variable but acts as a second independent variable. In other words, we find two things which relate to the phenomenon under study and their impact appears to be cumulative. There are ways to sort out the relative contribution of the two independent variables. However, in a newspaper story, about all you need to get across is that they exist, that they affect the dependent variable, and that their effect is cumulative.
In the five-year follow-up study of Berkeley arrestees, for example, an effort was made to determine what made some of the former student radicals –a minority as it turned out –become relatively conservative in their political behavior. A political conservative in this context was someone who voted for Hubert Humphrey in 1968 (no one in this group voted for Nixon). A vote for Humphrey represented an act within the political system and not a protest, as a write-in vote for Eugene McCarthy, Dick Gregory, Eldridge Cleaver, Pat Paulsen, or a deliberate nonvote would be.
Several factors were associated with a Humphrey vote, of which two are worth mentioning here as examples: a general low level of self-perceived radicalism, and the acquisition of a spouse since the 1964 arrests.
Among all those arrestees who responded to the mail survey, 33 percent voted for Humphrey. Among those who got married after the sit-in, 43 percent voted for him. Among those who placed themselves in the lower two-thirds on a self-anchoring scale of radicalism, 49 percent voted for Humphrey.
Immediately, the hypothesis suggests itself that the less radical are the more likely to do conventional things like getting married, and these two independent variables –marriage and low self-assessed radicalism – are telling us the same thing. It seems likely that radical students, being dedicated to their causes, would have little time or inclination for such a conventional institution as marriage. In fact, however, there was no significant difference in the marriage rate of the high-radical group and the low-radical group. What difference there was indicated that those who were still single five years later tended to rank somewhat lower in radicalism.
This lack of correlation between the two independent variables means, then, that their effect must be cumulative. The existence of both conditions, low radicalism and marriage, should produce a higher rate of Humphrey voting than either condition separately. It did, and the effect was easily demonstrated with numbers that could be interpreted to readers:
Forty-three percent of the married subjects voted for Humphrey.
Forty-nine percent of the low-radicalism subjects voted for Humphrey.
Sixty-one percent of the married, low-radicalism subjects voted for Humphrey.
From these examples, you can see how the introduction of a third variable to elaborate on what you already know about a two-variable relationship can do three basic things:
1. It can spot a spurious relationship. Example: People who eat candy have a lower death rate than those who do not eat candy. Therefore, eating candy causes longevity? No. Children eat candy, and children have yet to face the death-causing infirmities of old age. Control for age, and the relationship between candy eating and death rate disappears.
2. It can isolate the conditions in which the relationship is strongest and most important. Example: Newspaper readership in a southern city declines among younger age groups, a finding which raises the suspicion that the paper is losing touch with the educated younger generation. But when the relationship is controlled for education, nearly all of the young-readership decline is found to be among the least educated. This discovery opens a whole new line of inquiry, directed at the substitution of television for newspapers among the uneducated young, and a projection of future trends as the number of uneducated young people continues to dwindle.
3. Chains of causation can be sorted out and interactive or cumulative effects discovered. Example: Among Miami blacks in 1968, those scoring highest on a measure of political effectiveness tended to score high on conventional (as opposed to radical) militancy. Both measures also were positively correlated with income. Treating income as a separate independent variable demonstrated that it and political efficacy tended to operate cumulatively: each made a separate contribution to increased militancy.
Until this chapter, when you thought of a variable you probably thought of a single question item in the interview schedule. And most of the time that is exactly what a variable will be. However, you must learn early in the game not to let your imagination be limited by single-item variables. It often makes sense to combine two or more to create an entirely new measure.
New variables from old
One of the classic combined variables in social science literature is status inconsistency. Take two measures of socioeconomic status, education and income, for example. People who rank high in both or low in both are status-consistent. Those who are high in one but low in the other are status-inconsistent. And, research has shown, status-inconsistent people are different. It is a useful variable.
Another way to create new variables from old is in index construction. The new variables are not really “new” in that they involve a different property or disposition. Rather, they provide a more accurate and flexible indicator of what you are trying to measure. To return to the black militancy example, its eight items are a better collective measure of the disposition than any single item. One obvious advantage is that you can rank order the individuals in the sample according to the number of militant answers given. The choice of which of the available intercorrelated items to use in the index may be somewhat arbitrary, but you needn't worry about it. As Paul Lazarsfeld has pointed out, indices measuring the same phenomenon tend to be interchangeable. Shifting combinations of indicators will mean that some people will fall into or out of the defined group with each change. However, the different combinations are not likely to make any substantive change in the results when you look for relationships with another variable. We could take any four of the eight conventional militancy items and use them for a dichotomized index and still find that conventional militancy correlates with education. Then we could take the other four, try again, and again get the same general finding.
For this reason, there is usually little need to use as many as eight items in an index. Two or three will often do quite as well. And, in the case of socioeconomic status, you may often find cases where you will be content to use education as the only indicator. Sometimes the response rate is low on the income question, and occupation involves a difficult (though not insurmountable) coding problem. But both correlate well enough with education that, for many research purposes, you can get by with education alone.
You don't see many newspaper stories based on indices. That's a pity because index construction can add to the power and interpretability of polling data. And you should start to think about ways of analyzing survey data that go well beyond the basic frequency count–how many people gave each answer to each question –and even beyond simple bivariate comparisons – how high-income people compare to low-income people, city dwellers vs. suburbanites, etc. How can such painstaking analysis be made to fit into the time constraints of journalism?
It is easier than you might think. Even though the bulk of what you write from survey data will probably be based on the marginal frequencies and simple two-way relationships, it pays to stay alert to situations where deeper analyis can be useful.
One very basic tactic is to recode your variables so that the numerical values form rank order indices. For example, an item on prospective riot behavior might be originally coded:
Probably would join
1
Probably would not join 2
Not sure 3
To make the numbers stand for a rough approximation of propensity to riot, this item can be recoded:
Probably would join
3
Probably would not join 2
Not sure 1
When all the continuous or roughly continuous variables are thus coded, the computer can produce a correlation matrix – every item correlated with every other item. Statistically, of course, that is unsound in most cases. The assumptions underlying correlation (Pearsonian r) include interval scaling. Aside from such necessarily continuous variables as education, income, and age, most social science variables can only be ordinally scaled at best. But the computer doesn't know what kind of scales or indices your numbers represent. It will produce a correlation matrix which will have enough approximate validity to be useful as a searching device. Your eye can scan the rows and columns of correlation coefficients and when unexpectedly high values turn up, you can ask yourself why, and then run contingency tables to find out what is going on. You would not, of course, consider reporting correlation coefficients in the newspaper. It is simply a tool to alert you to relationships that you might otherwise not have noticed or not thought to have looked for.
Index construction
The correlation matrix can also guide you in the construction of indices out of several variables. If you think that several questionnaire items all tap a common characteristic, such as disposition toward violence, dissatisfaction with local government services, or racial prejudice, then you can get a quick estimate of their validity as an index by seeing whether they intercorrelate.
How do you tell what items are suitable for an index? You want them to have low intercorrelations, in the neighborhood of .2 to .5. If the intercorrelations are too high, the items are redundant, measuring too much the same thing. If they are too low, they are measuring different things. There are several statistical tests to guide you in building indices. Chronbach's Alpha is available in SPSS. It provides an estimate of the extent to which all of the items in an index are measuring the same underlying characteristic. How does it do this? For a direct explanation, you need a statistics test. For most purposes, it is enough to think of it as a measure of internal consistency. A low Alpha score means that you probably have an apples-and-oranges problem, i.e., the items in your index are not really measuring the same thing. The accepted interpretation of Chronbach's Alpha is that an alpha of .7 means that an index is good enough for exploratory research, and if it is .8 or more, you can use it in a confirmatory application. The same SPSS routine that produces it will also tell you how badly you need each item in the index. It looks at each one in turn and tells you how much Alpha will be reduced if the item is dropped. You would not want to bother newspaper readers with this information, but it can be good for your own peace of mind.
This quick search for things that hang together can be carried a step further with a factor analysis program which combs through a correlation matrix and picks out the clusters of variables that stick together with the most mathematical precision. The logic of factor analysis assumes that your variables are surface manifestations of some underlying condition and the optimum alignment of the intercorrelated variables will show what these underlying situations are. The trouble with this particular tool is that it is so powerful that it will cause factors to surface for you whether or not they are real. So you have to look at it with a skeptical eye and ask yourself whether they make any intuitive or theoretical sense. When they do, you can use them as indices to construct variables that will usually work better than single-item indicators.
An example of a case in which this procedure was successful is the 1968 Detroit study of black attitudes in the 1967 riot area. A number of items that dealt with possible courses of actions toward black achievement were intercorrelated and factor analyzed. The program called for orthogonal factors to be extracted –a shorthand way of saying that the items within each factor should correlate with another in that factor but that the separate factors should not be intercorrelated. Thus each factor represents a separate dimension unrelated to the others.
In the Detroit study, the first two factors extracted made good sense. The first was labeled “black power,” and its strongest components were positive responses to statements such as “blacks should get more political power by voting together to get officials who will look out for the Negro people” and “blacks should get more economic power by developing strong businesses and industries that are controlled by blacks.”
The second factor was labeled “black nationalism.” (The computer, it should be noted, does not invent these labels. You have to do it yourself.) Its strongest components included agreement with statements such as “It is very important for blacks to avoid having anything to do with white people as much as possible” and “It is very important for blacks to be ready to fight alongside other blacks and participate in riots if necessary.”
Finally it was shown that a sizable majority of Detroit blacks subscribed to the black power idea as defined in the conservative, self-help sense, and that very few were disposed to black nationalism. That these two dimensions were different and unrelated things was news to many whites who were accustomed to thinking of the extreme forms of militancy as differing only in degree from the black power concept. It is a difference in kind, not in degree.
Although the discovery was accomplished through factor analysis, the proof did not rest on this rather intricate and difficult-to-explain tool. Indices of black power and black nationalism were constructed, collapsed into categories for contingency tables, and run against one another to verify the absence of a relationship. This step was necessary not only for simplification, but as a check against the misuse of factor analysis. For our purposes especially, it should only be used to discover clues to things that can be described in a more straightforward manner.
There are other tricks that can be done with a correlation matrix. When there are several independent variables associated with the dependent variable, it is a shortcut for sorting out the effect of each by computing what is called the partial correlation coefficient. For example, in a survey of voters in Muncie, Indiana, in 1964, the correlation between political interest and income was .212, suggesting that people with money have more at stake in political decisions and therefore pay more attention to politics. On the other hand, income and education were very definitely correlated (r = .408), and there was a small but significant link between education and political interest (r = .181). Using contingency tables, it is possible to test the relationship between political interest and income by looking at it within categories of education. But that means another trip to the computing center. Partial correlation offers a quicker way to estimate the effect of holding education constant.
The correlation matrix has one other special application if you have panel data. It can, sometimes, give you an easy way to spot the direction of causation. Suppose you have interviewed the same people in two projects a year apart. Each survey shows a relationship between interest in local politics and time spent reading the morning paper. The question bothering you is which comes first in the causal chain (if one does come first; it could be a case of mutual causation). Out of the correlation matrix, you find that there is a significantly stronger relationship between interest in politics at time 1 and newspaper reading at time 2 than there is between newspaper reading at time 1 and interest in politics at time 2. The predominant direction of causation, then, is from interest in politics to newspaper reading. See Figure 6A.
You may have noticed by now that you are beginning to see social science methodology in a somewhat new light. With any luck, it should begin to look like something you can do rather than just passively observe and write about.
There is another corner which we have turned in this chapter. We have not, you may have noticed, made a big deal of significance testing in the discussion of survey analysis. And we have put a lot of emphasis on digging and searching procedures which don't quite square with the pure model of hypothesis testing which was presented earlier in this book.
Is this, then, the point of transition from scholarship to journalism? Not exactly. The better, more creative scholars know that the search for probable cause is where the action is in survey research, and statistical testing is trivial by comparison. The tests help you guard against the temptations of overinterpretation. But analysis of tables, with the introduction of third variables to go behind the superficial, two-variable relationships, is your protection against wrong interpretation. It is also your opportunity to discover causal sequences and explanations of the way things work in your community that you didn't suspect existed before.
In classical scientific method, you form a hypothesis, test it, and, if it fails the test, reject it and go on to something else. You don't keep cranking in epicycles as Ptolemy did until you have a clumsy, unparsimonious explanation to fit the observable data. Nevertheless, the rule against making interpretations after the fact, after the data are in and the printed-out tables are on your desk, is by no means ironclad. There is room in social science for serendipity. If the data give you an idea that you didn't have before, you need feel no guilt at all about pursuing it through the tables to see where it leads. Rosenberg notes that cases of serendipitous discoveries are plentiful in both the natural and the social sciences. He cites one of the best and most original concepts to emerge from modern survey research as an example: relative deprivation, uncovered by Samuel Stouffer in his research for The American Soldier.
Stouffer did not enter this survey with the idea that there might be such a thing as relative deprivation. The idea had not occurred to him and the survey was not designed to test it. But numbers came out that were so unexpected and so surprising that it was necessary to invent the idea of relative deprivation in order to live with them. One of the unexpected findings was that Northern blacks stationed in the South, despite their resentment of local racial discrimination, were as well or even better adjusted when compared to those stationed in the North. Another discrepancy turned up in the comparative morale of soldiers in units with high promotion rates and those in units with low chances of promotion: the low-promotion group was the happiest.
One parsimonious concept, relative deprivation, fit both of these situations. The black soldiers stationed in the South compared themselves to the black civilians they saw around them and found themselves better off. The high-promotion units had more soldiers who saw others being promoted and therefore felt more dissatisfaction at not getting ahead than soldiers in units where no one got ahead.
When the apparent discrepancies turned up, did the analysts shout “Eureka” and appreciate their importance in the history of social science? No. Instead, they delayed the report, went over the numbers again and again, hoping that some clerical error or something would show that the black soldiers and the low-promotion soldiers were not so happy after all. There is a moral here for journalists. We are not charged with the awesome responsibility of making original scientific discovery. We do have the responsibility of challenging and testing the conventional wisdom. And if the conventional wisdom says one thing and our data say another, we should, if the data are well and truly collected and analyzed, believe our data.
Rosenberg also has an answer for the methodological purists who say that after-the-fact interpretation is too much like Ptolemaic epicycle building. Accidental discoveries, he points out, are nullifiable. If you find something surprising, you can use your contingency tables to creep up on it from another direction to see if it is still there. Stouffer found something surprising in the attitude of black soldiers, invented the concept, and then tested it elsewhere in his data on the high- and low-promotion units.
A journalistic example is also available. When the Knight Newspapers surveyed Berkeley arrestees five years after the arrests one of the interesting findings was that females who had been radicalized by the Sproul Hall affair tended to hold on to that radicalization more than did males in the ensuing years. This finding, based on one table, led to the hypothesis that for a girl to become a radical involves a more traumatic separation from the values and attitudes of her family than it does for a male, and that she therefore holds on to the radical movement as a family substitute. The theory was testable from other data in the survey. If it were true, females should have a greater proportion of parents who disapproved of the activity that led to their getting arrested. Furthermore, those females with disapproving parents should be more likely to retain their radicalism.
Checking these two propositions required a new two-way table (sex by parent approval) and a three-way table (radical retention by parental approval by sex) and one trip to the computing center. It turned out that there was a small sex difference (though not statistically significant) in favor of males having parental approval. However, the effect of disapproving parents on radical retention was the same for boys and girls. So the theory was, for the purposes of this project at least, nullified.
“The post-factum interpretation,” says Rosenberg, “is thus not the completion of the analysis but only the first step in it. The interpretation is made conditional upon the presence of other evidence to support it.”
Thus it is not necessary to fall back on a journalistic excuse for using the computer as a searching device instead of a hypothesis-testing tool. The journalistic excuse would be that we are in too much of a hurry to be as precise as sociologists, and, besides, our findings are not going to be engraved on tablets of stone. Let us think twice before copping out like that. If we were really backed up against the wall in a methodological argument with scientific purists, we might have to take that last-resort position. Meanwhile, we can make the better argument that we are practical people, just as most sociologists are practical people, and therefore, when we spot the germ of an idea glimmering in our data, we need not shrink from its hot pursuit.
When the fieldwork is done and the data entry completed, the fun really begins. To illustrate some more principles of data analysis, let us assume that you are analyzing a public opinion poll. The first thing you want to see is the marginal frequencies: the number and percentage of people who have each of the possible responses to each of the questions in the survey. Determining this basic information is not as clear-cut as it sounds, however, and a few policy decisions must be made in advance.
First among them is the problem of dealing with answers of the don't-know, no-opinion, and no-answer variety. Do you leave them in the base for calculating percentages, or do you take them out? It can make a difference. Suppose you ask 500 people, “On the whole, do you approve or disapprove of the way the mayor is handling his job?” and you get the following distribution:
Approve
238
Disapprove 118
Don't know 104
No answer 40
If you base the percentages on the total sample of 500, you find:
Approve
48%
Disapprove 24
Don't know 21
No answer 8 (n = 500)
The total in this case is 101 percent because of rounding errors. No need to be compulsive about that. If survey research were a totally precise and reliable instrument, you might be justified in reporting fractional values. But it isn't, and using decimal points gives a false sense of precision which you may as well avoid.
Now looking at the above percentages, the sensation-seeking beast that lurks in all of us spots an opportunity for an exciting lead: “Mayor Frump has failed to gain the approval of a majority of the adult residents of the city, an exclusive Daily Bugle poll revealed today.”
However, it is possible to give the mayor his majority support by the simple expedient of dropping the “no answers” from the percentage base. Using the same numbers based on the 460 who responded to the question, we find:
Approve
52%
Disapprove 26
Don't know 23 (n = 460)
Mayor Frump suddenly looks better. Much of his looking better, of course, is based on the artificial distinction between a minority and a majority. The four-point difference would not seem nearly as important if the range were, say, from 42 to 46. And since no election is involved here, the question of majority support is not particularly germane. Moreover, the apparent majority or lack of it could be due to sampling error. Artificial as the distinction may be, however, it is one that can quickly catch the reader's eye and one that will be overemphasized despite your best efforts to keep it in perspective. The choice of a base for computing percentage is therefore crucial.
There is yet a third possibility, basing the percentages on the total number of people who have opinions about the mayor:
Approve
67%
Disapprove 33 (n = 356)
Now the mayor looks very good indeed, especially when we consider the likelihood that the “don't know” segment is also the least informed. The public relations staff at City Hall can leap on this and claim, with some justification, that informed citizens approve of the mayor by a ratio of two to one.
Deciding what to count
So here you sit with a survey containing perhaps two hundred questions, and each of them is subject to three different interpretations. You are a writer of news stories, not lengthy scholarly treatises. What do you do? A rule set forth in the previous chapter is so important that it is worth repeating here:
“Don't know” is data.
The soundest procedure is to base your percentages on the nonblank answers, as in the second of the three examples cited above. It is theoretically justifiable because not answering a particular question is in somewhat the same category as not responding to the entire questionnaire. The reasons for no answer are varied: the interviewer may have been careless and failed to mark that question or failed to ask it, or the respondent may have refused to answer. In any case, failure to answer may be treated as not being in the completed sample for that particular question. You should, of course, be on the lookout for items for which the no-answer rate is particularly high. They may be a tipoff to a particularly sensitive or controversial issue worth alerting your readers about; and you will, of course, want to warn the reader whenever you find meaningful responses that are based on considerably less than the total sample.
Usually, however, the no-answer rate will be small enough to be considered trivial, and you can base your percentages on the nonblank answers with a clear conscience and without elaborate explanation.
The don't-know category is quite different. The inability of a respondent to choose between alternatives is important information, and this category should be considered as important data–as important as that furnished by people who can make up their minds. In an election campaign, for example, a high undecided rate is a tipoff that the situation is still unstable. In the example just examined it suggests a substantial lack of interest in or information about the mayor–although these are qualities best measured more directly.
Therefore, you should, as a matter of routine, include the don't-knows in the basic frequency count and report them. When you judge it newsworthy to report percentages based on only the decided response, you can do that, too. But present it as supplementary information: “Among those with opinions, Mayor Frump scored a substantial . . .”
When you do your counting with a computer, it is an easy matter to set it to base the percentages on the nonblank answers and also report the number of blanks. If you are working with SAS or SPSS, the frequency procedures will automatically give you percentages both ways, with the missing data in and out.
Beyond the marginals
Either way, you can quickly size up your results if you enter the percentages on an unused copy of the interview schedule. Before going further, you will want to make some external validity checks. Are males and females fairly equal in number? Does the age distribution fit what you know about the population from other sources, such as census data? Does voting behavior fit the known results (allowing for the expected overrecall in favor of the winning candidates)? With any luck, each of these distributions will fall within the sampling error tolerances. If not, you will have to figure out why, and what to do about it. Once you know the percentage who gave each of the alternative responses to each of the questions, you already have quite a bit to write about. USA Today can produce a newspaper column from three or four questions alone. However, the frequencies – or marginals, as social scientists like to call them – are not the entire story. Often they are not even very interesting or meaningful standing by themselves. If I tell you that 75 percent of the General Social Survey's national sample says the government spends “too little” on improving the environment, it may strike you as mildly interesting at most, but not especially meaningful. To put meaning into that 75 percent figure, I must compare it with something else. If I tell you that in a similar national sample two years earlier, only 68 percent gave that response, and that it was 59 percent four years earlier, you can see that something interesting is going on in the nation. And that is just what the General Social Survey did show in the years 1989, 1987, and 1985. A one-shot survey cannot provide such a comparison, of course. However, if the question has been asked in other surveys of other populations, you can make a comparison that may prove newsworthy. That is one benefit of using questions that have been used before in national samples. For example, a 1969 survey of young people who had been arrested in December 1964 at the University of California sit-in used a question on faith in government taken from a national study by the Michigan Survey Research Center. The resulting comparison showed the former radicals to have much less faith in government than did the nation as a whole.
Internal comparisons
Important opportunities for comparison may also be found within the survey itself. That 75 percent of Miami blacks are in favor of improving their lot through more political power is a fact which takes on new meaning when it is compared to the proportion who favor other measures for improvement. In a list of possible action programs for Miami blacks, encompassing a spectrum from improving education to rioting in the streets, education ranked at the very top, with 96 percent rating it “very important.” Violent behavior ranked quite low on the list.
And this brings us anew to the problem of interpretation raised in the opening chapter of this book. You can report the numbers, pad some words around them, the way wire-service writers in one-person bureaus construct brief stories about high school football games from the box scores, and let it go at that, leaving the reader to figure out what it all means. Or you can do the statistical analog of reporter's leg-work, and dig inside your data to find the meaning there.
One example will suffice to show the need for digging. A lot has been written about generational differences, particularly the contrast between the baby boomers and the rest of the population. And almost any national survey will show that age is a powerful explanatory variable. One of the most dramatic presentations of this kind of data was made by CBS News in a three-part series in May and June of 1969. Survey data gathered by Daniel Yankelovich, Inc., was illustrated by back-to-back interviews with children and their parents expressing opposite points of view. The sample was drawn from two populations: college youth and their parents constituted one population; noncollege youth and their parents the other. Here is just one illustrative comparison: Asked whether “fighting for our honor” was worth having a war, 25 percent of the college youth said yes, compared to 40 percent of their parents, a difference of 15 percentage points.
However, tucked away on page 186 on Yankelovich's 213-page report to CBS, which formed the basis for the broadcasts, was another interesting comparison. Among college-educated parents of college children, only 35 percent thought fighting for our honor was enough to justify a war. By restricting comparison to college-educated people of both generations, the level of education was held constant, and the effect of age, i.e., the generation gap, was reduced to a difference of 10 percentage points.
Yankelovich had an even more interesting comparison back there on page 186. He separated out the noncollege parents of the noncollege kids to see what they thought about having a war over national honor. And 67 percent of them were for it. Therefore, on this one indicator we find a gap of 32 percentage points between college-educated adults with kids in college and their adult peers in noncollege families:
Percent saying "honor" worth fighting a war
College youth
25
10% difference
College parent of
college child
35
32% difference
Noncollege parent of
noncollege child 67
Obviously, a lot more is going on here than just a generation gap. The education and social-class gap is considerably stronger. Yankelovich pursued the matter further by making comparisons within the younger generation. “The intra-generation gap, i.e., the divisions within youth itself,” he told CBS a month before the first broadcast, “is greater in most instances than the division between the generations.”
The same thing has turned up in other surveys. Hold education constant, and the generation gap fades. Hold age constant, and a big social-class gap –a wide divergence of attitudes between the educated and the uneducated–opens up. Therefore, to attribute the divisions in American society to age differences is worse than an oversimplification. It is largely wrong and it obscures recognition of the more important sources of difference. CBS, pressed for time, as most of us usually are in the news business, chose to broadcast and illustrate the superficial data which supported the preconceived, conventional-wisdom thesis of the generation gap.
Hidden effects
Using three-way cross-tabulation to create statistical controls can also bring out effects that were invisible before. When Jimmy Carter ran for president in 1976, the reporters using old-fashioned shoe-leather methods wrote that his religious conviction was helping him among churchgoers. Then the pollsters looked at their numbers and saw that frequent churchgoers were neither more nor less likely to vote for Carter than the sinners who stayed home on Sunday.
These data from a September 1976 Knight-Ridder poll illustrate what was turning up:
Highly Religious
Not So Religious
Carter 42% 38%
Ford 47 52
Not voting or DK 11 10
Total 100 100
Carter support was four points greater among the “highly religious” than the “not so religious” (42 to 38). But the difference was not statistically significant. As it turned out, however, the shoe-leather guys were right. There was a religin effect if you knew where to look for it. Carter had a strong appeal to young people, and young people tend to be less religious. Carter's religiosity did not have much effect on older people whose political beliefs were well established. The religion appeal worked mainly on the young. Variables that conceal effects this way have been called “suppressor and distorter variables” by Morris Rosenberg. The way to find the effect is to look at Carter support by churchgoing behavior within each age group. When that was done, a strong church effect favoring Carter appeared among those aged 18 to 41.
Highly Religious Not So Religious
Carter 49% 38%
Ford 43 52
Not voting or DK 8 9
Total 100 100
The two above examples are rather complicated, and you can't be blamed for scratching your head right now. Let's slow down a bit and poke around a single survey. I like the Miami Herald's pre-riot survey as a case study because of its path-breaking nature, and because the analysis was fairly basic. We shall start with a simple two-way table. A two-way (or bivariate) table simply sorts a sample population into each of the possible combinations of categories. This one uses age and conventional militancy among Miami blacks. In the first pass through the data, age was divided four ways, militancy into three.
AGE
15-24 25-35 36-50 Over 50 Total
Low 23 28 34 45 130
MILITANCY Medium 65 60 65 56 246
High 23 44 38 19 124
Because the marginal totals are unequal, it is hard to grasp any meaning from the table without converting the raw numbers to percentages. Because militancy is the dependent variable, we shall base the percentages on column totals.
AGE
15-24 25-35 36-50 Over 50 Total
Low 21% 21% 25% 37% 26%
MILITANCY Medium 59 45 47 47 49
High 21 33 28 16 25
Percent of N 22 26 27 24 100
The marginal percentages are based on the total 500 cases. Thus we see at a glance that 26 percent are in the low-militancy category, 49 percent in the medium group, and 25 percent in the high group. Age is distributed in nearly equal categories. Looking across the top row of cells, we can also see that the proportion of low militancy tends to increase with age. And the greatest percentage of high militancy is found in the 25-35 group.
There are too many numbers here to throw at your readers. But they mean something (the chi-square value –computed from the raw numbers – is 20, which, with six degrees of freedom, makes it significant at the .003 level). And the meaning, oversimplified–but honestly oversimplified, so we need make no apology–is that older people aren't as militant as younger people. We can say this by writing with words and we can also collapse the cells to make an easier table.
AGE
15-35 Over 35
Low Militancy 21% 31%
Medium and High Militancy 79 69
100% 100%
This table also eliminates the marginal percentages. The sums at the bottom are just to make it clear that the percents are based on column totals.
The problem of figuring which way the percentages run may seem confusing at first, but eventually you will get the hang of it. To make it easier, most of the tables in this book base the percentages on column sums. Thus the dependent variable –the quality being dissected – is listed across the rows. No law of social science requires this arrangement. We could just as logically put the dependent variable in columns and figure percent across rows. In some cases, to clarify a distribution, you may want to base percentage on the table total, i.e., the sum in the corner of the margins. But for now we shall standardize with the dependent variable reported across rows and the percentages based on totals down the columns.
Standardizing your tables
With this standard presentation, you can quickly gain the habit of letting your eye rove from left to right to find the percent difference. The table comparing the proportion of younger and older people among the militants shows older people have half again as many conservatives or low militants among them as younger people have: 31 percent vs. 21 percent. And that is a fairly good way to explain it to the reader.
But if it takes you some time to develop your table-reading skill so that the figures leap off the page at you, bright and meaningful, consider the plight of the reader. Your practice and skill at interpretation do not help him or her, and so you have to reduce things to words or to bare numerical essentials, or both in combination. One way to ease the burden on the reader is to give him or her one-way tables, created by lopping off the less dramatic half of the two-way table. Even then, you should add some words to tell what the numbers are saying. Here is how the relationship between militancy and age could be presented:
MILITANCY AND AGE
Older people tend to be more conservative than younger people.
15-35 years Over 35 years
Percent conservative 21 31
The other half of the table, the percent nonmilitant in each category, is implied. And the table makes clear that the numbers are percentages within age categories.
What about the don't-knows? Shouldn't there be cells in the table for them? Not in this case. The militant was operationally defined as someone who gave militant answers to six of the eight questions in the index. “No answer,” therefore, counted as a nonmilitant answer. Had the number of blanks for any one of the items been unusually high, some theoretical problems could have been raised. They weren't. In a few cases, interviewers failed to record age, and those cases were automatically eliminated from the table.
Now we must reflect a bit on what we have added to the newspaper reader's knowledge here. We have made a comparison between young and old and demonstrated that militancy is found more frequently among the young. That in itself is something of an achievement because it provides a fuller description of the phenomenon of militancy than was available before we produced the table. Age and militancy are related.
Can we go beyond that and assume that the relationship involves causation? Nothing in the numbers themselves demonstrates that the relationship is a causal one. To make the leap to a causal assumption, we have to use some logic, a bit of intuition, and common sense. And that, you may remember from an earlier chapter, is why we keep referring to one variable as the dependent variable and the other as the independent variable. This description is the most parsimonious and conservative one available. We can say that we are looking for evidence that militancy depends on age without committing ourselves to talking about causation. The statement that the amount of militancy depends on age is purely a descriptive one. (Or, if the row comparisons had turned out to be about equal, we could have said that militancy does not depend on age.)
Now let us look at this particular table and bring some logic to bear on it. If there is a relationship between the two variables, then it could be because one of them is a cause of the other. But which way does the causal arrow run? In this case, it is easy to eliminate the hypothesis that militancy causes age. Chronological age, more's the pity, is fixed and unchangeable, so the arrow must run the other way. Other readily measured attributes possess the same advantage in deducing causality: sex, race, birth order within the family are examples. Because they are unchangeable, we can assume that they are causes and not effects, if there is causation.
For illustration, we return to the case of the Miami blacks. Militancy can't cause people to be young, but the reverse proposition, that being young causes them to be militant, also lacks a good deal. What we really want to know is what it is about being young that makes people militant.
The way to find out is to look first for other dimensions where young people differ from old: education, for example. People in all social classes tend to be better educated than their parents. Among Miami blacks more than 24 years old, 25 percent had high school diplomas. Only 12 percent of their fathers were high school graduates. Furthermore, we can expect militancy to increase with education.
Knowing this, we then suspect that it may not be youth per se that causes militancy, but merely the fact that young blacks are better educated, and that better education is the real cause of militancy. To test this idea, we first verify our suspicion that education is related to militancy. And indeed it is. Reducing the table to a newspaper-compatible form produces the following evidence that militancy increases with education.
LEVEL OF EDUCATION
Grade
school Some high
school High school
graduate Beyond
high school
Percent militant 16 23 32 38
(The full table, with seven categories of education and three of militancy, has a chi-square value of 33, significant at the .001 level with 12 degrees of freedom.)
At first glance, education seems to have even more to do with militancy than does age. Perhaps age is not a “real” factor at all. Maybe it is simply a cause of education, which is then the more direct cause of militancy.
To test this idea, we introduce what Morris Rosenberg calls a test factor. Another word for it is control. As in the CBS report, we want to control for age–this time to examine the relationship between militancy and education. Controlling for age means to hold its effect constant by looking at the effect of education on militancy within each age category. This process is analogous to the laboratory scientist who, by repeating his experiment in a variety of temperatures, makes certain that variation in room temperature is not affecting his chemical reaction.
Tables with three dimensions
The result of such an examination is a three-way table: militancy by education by age. In physical printout, it is a two-way table repeated for each educational category–the only way that a three-dimensional table can be reproduced on two-dimensional paper.
Several things might happen:
1. The education-militancy relationship could disappear within each of the age groups. If so, age, not education, is the cause of militancy. Age, in fact, is a common cause of both education and militancy and the apparent education-militancy relationship is spurious.
2. The education-militancy relationship could remain in each of the age categories. In this case, age and militancy are both links in the causal chain. Logic tell us that age, being fixed, comes first. Therefore, youth causes education and education causes militancy.
3. The education-militancy relationship could disappear in some of the age categories and remain the same or become stronger in others. If so, some kind of interaction effect is operating. There are special circumstances in which age and education work together to increase militancy–more, perhaps, than simple addition of their separate effects would account for.
Which of the above really happened in the Miami case?
To make it easy, we'll again collapse the tables to one dimension, looking at the percent militant in each age group, starting with the youngest. Remember, we are looking not at one table but four.
Grade
school Some high
school High school
graduate Beyond
high school
Age 15-24
Percent Militant * 15 25 36
Age 25-35
Percent Militant * 25 39 48
Age 36-50
Percent Militant 17 32 33 *
Age over 50
Percent Militant 14 15 23 *
* Number in cell too small to computer percentage
As you can see, education and militancy are related in all age categories. The second of the three hypotheses is sustained. It is not the ebullience of youth so much that produces militant attitudes. Rather, it is the greater educational opportunity open to young people, which in turn increases their militancy. This finding fits neatly into a larger theoretical framework of rising aspirations: education moves people closer to goals and increases their expectations. Higher expectations, left unfulfilled, produce frustration and militancy.
This example deals with a relationship between what Rosenberg calls a “property” (age) and “disposition” (militancy). Properties are unambiguous, things an individual is or isn't: white, German-born, grammar school graduate, blue-collar worker, 1968 Nixon voter, color-television owner, teetotaler, licensed airplane pilot, pack-a-day cigarette smoker. Dispositions are more difficult to get a grip on because they are qualities that need the presence of special situations in order to be activated. The black militant, i.e., a black person scoring high on the militancy scale, may or may not express his or her disposition by behaving in a militant manner, depending, normally, on the external situation he or she faces at any given moment. But the disposition is there and it is measurable. One of the things that some social scientists try very hard to do is to establish relationships between dispositions and behavior. Another school, however, holds that dispositions –the attitudes, values, personality traits within the person – have much less to do with behavior than does the external situation.
It is generally much easier to establish the relationship between a property and a disposition than to predict actual behavior, as in the case of the age-education-militancy chain. However, you can find news applications of both types, although you may find that you are unable to interest your editors or your readers in attitudinal relationships until dispositions are manifested in newsworthy behavior. We become interested in what blacks or students or public school teachers are thinking after they riot or eject the dean from his office or go on strike. Nevertheless, a news organization that takes the new methods seriously will try to monitor the attitudinal developments before matters reach the overt, obviously newsworthy, man-bites-dog level. If a man biting a dog is news, a man thinking about biting a dog is a potential scoop.
If a disposition is the dependent variable, i.e., the thing being studied, you then can quickly find yourself in the business of searching for relationships between one disposition and another. In the Miami Herald's pre-riot survey of early 1968, one of the important dependent variables was disposition toward violence. It was measured with two questionnaire items, one dealing with general approval of violence to advance black goals and the other asking about the respondent's own intent to participate in such violence should the opportunity arise. The two items were combined to form a four-part index ranging from “violent” –those who both favored violence as an idea and were ready to participate themselves –to those who were opposed to both the concept and the personal act. It was then tested against a number of specific grievances to find out, first, whether grievances really do dispose people toward thoughts of violence, and, if so, which grievances had the most effect.
One of the more clear-cut tables showed quite plainly that disposition toward violence was associated with dissatisfaction over one's personal housing situation.
ATTITUDE TOWARD HOUSING
Happy Unhappy
ATTITUDE TOWARD VIOLENCE Violent 7% 12%
Near violent 8 18
Ambivalent 25 24
Opposed 69 45
100 100 (n = 478)
However, young people tend to be more educated and more discontent and also more violent. The question arises, then, whether discontent with housing really has an effect on feeling toward violence, or whether the two variables are merely common effects of a more complicated bundle of attitudes that involved being young and black. So the table was run again, this time controlling for age. To conserve cases, the four violence categories were collapsed to two and the tables were reduced to one dimension apiece for simplicity.
ATTITUDE TOWARD HOUSING
Happy Unhappy
Age 15-24
Violent or near violent 21% 33%
Age 25-35
Violent or near violent 19 29
Age 36-50
Violent or near violent 12 40
Age over 50
Violent or near violent 12 20
The relationship persists in each age group, though it comes on significantly stronger in the middle-aged, 36-50, group. Therefore, at least two things are probably going on here: first, youth, or something about youth, causes people to feel and express grievances over housing and the grievances in turn stimulate favorable attitudes toward violence. But look at the third age category. Housing is strongest as an explanatory variable for violence in that age group for which it is most relevant: these are people in the middle-to-late child-rearing years for whom being deprived of satisfactory housing can be the most frustrating. Thus bad housing makes these people disposed toward violence despite their age; age, negatively associated with violence (the greater the age, the less the violence) has thus had a suppressor effect in the previous two-way table, which tested for the effect of housing dissatisfaction on violence.
Or, recasting this statement in shorter sentences:
Disposition toward violence decreases with age.
Disposition toward violence increases with housing dissatisfaction.
Among middle-aged people, housing dissatisfaction relates so strongly to disposition toward violence as to outweigh the violence-suppressing effect of age. When you can pinpoint a group in which special circumstances make a relationship stand out with extra power and clarity you have a strong potential for an important news story. In Miami, urban affairs writer Juanita Greene found that housing frustrations for people in the child-rearing years were grounded in objective reality. The gap between housing need and housing availability was large and it was not being closed. This kind of spotting of special situations, by the way, is something that social scientists, in their quest for general theoretical principles, do not often do very well. “Social science has been justly criticized,” says Rosenberg, “for its neglect of situational factors. One may know that general principles obtain, but one does not know whether these principles have predictive value in specific circumstances.” Digging for specific circumstances may fit the instincts of the journalist better than those of the social scientist. Academic reluctance to ferret out the mundane details needn't inhibit us at all as we look for the traditional who, what, when, where, why, and how.
Before leaving the violence-by-housing and satisfaction-by-age tables as compressed above, scan it once more to see how quickly you are becoming accustomed to letting the numbers leap off the page, bearing their message to you. Your eye automatically scans from left to right to show that housing attitudes relate to violence attitudes in all age groups. But you can also scan it vertically to see how attitudes favoring violence tend to fade with advancing age. The percent disposed to violence fades from 21 to 12 among those happy with housing conditions and from 33 to 20 among those unhappy with housing conditions. Furthermore, the exception, that 40 percent violent among the unhappy 36-50 group, cries out for attention and explanation.
If, on the other hand, you still have to stop and puzzle out the meaning, don't worry. You are, after all, in the business of writing with words, not numbers. But facility with the numbers will come with practice, particularly as you apply them to work of your own and not just to examples printed in a book. You may also take comfort from this thought: you are avoiding, so far, the danger of becoming so fluent with numbers that you begin to lose your ability to put their meaning into words that newspaper readers can understand and appreciate. This hazard is well known to social scientists, especially among students getting a grip on quantitative methods for the first time. They sometimes reach a stage in which they resent the fact that they can't simply send the computer printout to the publishers and have its significance instantly appreciated and spread upon the record for all the world to see. It doesn't work that way for them and it especially doesn't work that way for us in journalism. We write with words, but we must learn to read in numbers.
More than one independent variable
We have now seen some of the things that can happen to a simple two-variable relationship when a third variable, related to both, is introduced. One more example deserves some examination. This is the case where the third variable is not really a test or control variable but acts as a second independent variable. In other words, we find two things which relate to the phenomenon under study and their impact appears to be cumulative. There are ways to sort out the relative contribution of the two independent variables. However, in a newspaper story, about all you need to get across is that they exist, that they affect the dependent variable, and that their effect is cumulative.
In the five-year follow-up study of Berkeley arrestees, for example, an effort was made to determine what made some of the former student radicals –a minority as it turned out –become relatively conservative in their political behavior. A political conservative in this context was someone who voted for Hubert Humphrey in 1968 (no one in this group voted for Nixon). A vote for Humphrey represented an act within the political system and not a protest, as a write-in vote for Eugene McCarthy, Dick Gregory, Eldridge Cleaver, Pat Paulsen, or a deliberate nonvote would be.
Several factors were associated with a Humphrey vote, of which two are worth mentioning here as examples: a general low level of self-perceived radicalism, and the acquisition of a spouse since the 1964 arrests.
Among all those arrestees who responded to the mail survey, 33 percent voted for Humphrey. Among those who got married after the sit-in, 43 percent voted for him. Among those who placed themselves in the lower two-thirds on a self-anchoring scale of radicalism, 49 percent voted for Humphrey.
Immediately, the hypothesis suggests itself that the less radical are the more likely to do conventional things like getting married, and these two independent variables –marriage and low self-assessed radicalism – are telling us the same thing. It seems likely that radical students, being dedicated to their causes, would have little time or inclination for such a conventional institution as marriage. In fact, however, there was no significant difference in the marriage rate of the high-radical group and the low-radical group. What difference there was indicated that those who were still single five years later tended to rank somewhat lower in radicalism.
This lack of correlation between the two independent variables means, then, that their effect must be cumulative. The existence of both conditions, low radicalism and marriage, should produce a higher rate of Humphrey voting than either condition separately. It did, and the effect was easily demonstrated with numbers that could be interpreted to readers:
Forty-three percent of the married subjects voted for Humphrey.
Forty-nine percent of the low-radicalism subjects voted for Humphrey.
Sixty-one percent of the married, low-radicalism subjects voted for Humphrey.
From these examples, you can see how the introduction of a third variable to elaborate on what you already know about a two-variable relationship can do three basic things:
1. It can spot a spurious relationship. Example: People who eat candy have a lower death rate than those who do not eat candy. Therefore, eating candy causes longevity? No. Children eat candy, and children have yet to face the death-causing infirmities of old age. Control for age, and the relationship between candy eating and death rate disappears.
2. It can isolate the conditions in which the relationship is strongest and most important. Example: Newspaper readership in a southern city declines among younger age groups, a finding which raises the suspicion that the paper is losing touch with the educated younger generation. But when the relationship is controlled for education, nearly all of the young-readership decline is found to be among the least educated. This discovery opens a whole new line of inquiry, directed at the substitution of television for newspapers among the uneducated young, and a projection of future trends as the number of uneducated young people continues to dwindle.
3. Chains of causation can be sorted out and interactive or cumulative effects discovered. Example: Among Miami blacks in 1968, those scoring highest on a measure of political effectiveness tended to score high on conventional (as opposed to radical) militancy. Both measures also were positively correlated with income. Treating income as a separate independent variable demonstrated that it and political efficacy tended to operate cumulatively: each made a separate contribution to increased militancy.
Until this chapter, when you thought of a variable you probably thought of a single question item in the interview schedule. And most of the time that is exactly what a variable will be. However, you must learn early in the game not to let your imagination be limited by single-item variables. It often makes sense to combine two or more to create an entirely new measure.
New variables from old
One of the classic combined variables in social science literature is status inconsistency. Take two measures of socioeconomic status, education and income, for example. People who rank high in both or low in both are status-consistent. Those who are high in one but low in the other are status-inconsistent. And, research has shown, status-inconsistent people are different. It is a useful variable.
Another way to create new variables from old is in index construction. The new variables are not really “new” in that they involve a different property or disposition. Rather, they provide a more accurate and flexible indicator of what you are trying to measure. To return to the black militancy example, its eight items are a better collective measure of the disposition than any single item. One obvious advantage is that you can rank order the individuals in the sample according to the number of militant answers given. The choice of which of the available intercorrelated items to use in the index may be somewhat arbitrary, but you needn't worry about it. As Paul Lazarsfeld has pointed out, indices measuring the same phenomenon tend to be interchangeable. Shifting combinations of indicators will mean that some people will fall into or out of the defined group with each change. However, the different combinations are not likely to make any substantive change in the results when you look for relationships with another variable. We could take any four of the eight conventional militancy items and use them for a dichotomized index and still find that conventional militancy correlates with education. Then we could take the other four, try again, and again get the same general finding.
For this reason, there is usually little need to use as many as eight items in an index. Two or three will often do quite as well. And, in the case of socioeconomic status, you may often find cases where you will be content to use education as the only indicator. Sometimes the response rate is low on the income question, and occupation involves a difficult (though not insurmountable) coding problem. But both correlate well enough with education that, for many research purposes, you can get by with education alone.
You don't see many newspaper stories based on indices. That's a pity because index construction can add to the power and interpretability of polling data. And you should start to think about ways of analyzing survey data that go well beyond the basic frequency count–how many people gave each answer to each question –and even beyond simple bivariate comparisons – how high-income people compare to low-income people, city dwellers vs. suburbanites, etc. How can such painstaking analysis be made to fit into the time constraints of journalism?
It is easier than you might think. Even though the bulk of what you write from survey data will probably be based on the marginal frequencies and simple two-way relationships, it pays to stay alert to situations where deeper analyis can be useful.
One very basic tactic is to recode your variables so that the numerical values form rank order indices. For example, an item on prospective riot behavior might be originally coded:
Probably would join
1
Probably would not join 2
Not sure 3
To make the numbers stand for a rough approximation of propensity to riot, this item can be recoded:
Probably would join
3
Probably would not join 2
Not sure 1
When all the continuous or roughly continuous variables are thus coded, the computer can produce a correlation matrix – every item correlated with every other item. Statistically, of course, that is unsound in most cases. The assumptions underlying correlation (Pearsonian r) include interval scaling. Aside from such necessarily continuous variables as education, income, and age, most social science variables can only be ordinally scaled at best. But the computer doesn't know what kind of scales or indices your numbers represent. It will produce a correlation matrix which will have enough approximate validity to be useful as a searching device. Your eye can scan the rows and columns of correlation coefficients and when unexpectedly high values turn up, you can ask yourself why, and then run contingency tables to find out what is going on. You would not, of course, consider reporting correlation coefficients in the newspaper. It is simply a tool to alert you to relationships that you might otherwise not have noticed or not thought to have looked for.
Index construction
The correlation matrix can also guide you in the construction of indices out of several variables. If you think that several questionnaire items all tap a common characteristic, such as disposition toward violence, dissatisfaction with local government services, or racial prejudice, then you can get a quick estimate of their validity as an index by seeing whether they intercorrelate.
How do you tell what items are suitable for an index? You want them to have low intercorrelations, in the neighborhood of .2 to .5. If the intercorrelations are too high, the items are redundant, measuring too much the same thing. If they are too low, they are measuring different things. There are several statistical tests to guide you in building indices. Chronbach's Alpha is available in SPSS. It provides an estimate of the extent to which all of the items in an index are measuring the same underlying characteristic. How does it do this? For a direct explanation, you need a statistics test. For most purposes, it is enough to think of it as a measure of internal consistency. A low Alpha score means that you probably have an apples-and-oranges problem, i.e., the items in your index are not really measuring the same thing. The accepted interpretation of Chronbach's Alpha is that an alpha of .7 means that an index is good enough for exploratory research, and if it is .8 or more, you can use it in a confirmatory application. The same SPSS routine that produces it will also tell you how badly you need each item in the index. It looks at each one in turn and tells you how much Alpha will be reduced if the item is dropped. You would not want to bother newspaper readers with this information, but it can be good for your own peace of mind.
This quick search for things that hang together can be carried a step further with a factor analysis program which combs through a correlation matrix and picks out the clusters of variables that stick together with the most mathematical precision. The logic of factor analysis assumes that your variables are surface manifestations of some underlying condition and the optimum alignment of the intercorrelated variables will show what these underlying situations are. The trouble with this particular tool is that it is so powerful that it will cause factors to surface for you whether or not they are real. So you have to look at it with a skeptical eye and ask yourself whether they make any intuitive or theoretical sense. When they do, you can use them as indices to construct variables that will usually work better than single-item indicators.
An example of a case in which this procedure was successful is the 1968 Detroit study of black attitudes in the 1967 riot area. A number of items that dealt with possible courses of actions toward black achievement were intercorrelated and factor analyzed. The program called for orthogonal factors to be extracted –a shorthand way of saying that the items within each factor should correlate with another in that factor but that the separate factors should not be intercorrelated. Thus each factor represents a separate dimension unrelated to the others.
In the Detroit study, the first two factors extracted made good sense. The first was labeled “black power,” and its strongest components were positive responses to statements such as “blacks should get more political power by voting together to get officials who will look out for the Negro people” and “blacks should get more economic power by developing strong businesses and industries that are controlled by blacks.”
The second factor was labeled “black nationalism.” (The computer, it should be noted, does not invent these labels. You have to do it yourself.) Its strongest components included agreement with statements such as “It is very important for blacks to avoid having anything to do with white people as much as possible” and “It is very important for blacks to be ready to fight alongside other blacks and participate in riots if necessary.”
Finally it was shown that a sizable majority of Detroit blacks subscribed to the black power idea as defined in the conservative, self-help sense, and that very few were disposed to black nationalism. That these two dimensions were different and unrelated things was news to many whites who were accustomed to thinking of the extreme forms of militancy as differing only in degree from the black power concept. It is a difference in kind, not in degree.
Although the discovery was accomplished through factor analysis, the proof did not rest on this rather intricate and difficult-to-explain tool. Indices of black power and black nationalism were constructed, collapsed into categories for contingency tables, and run against one another to verify the absence of a relationship. This step was necessary not only for simplification, but as a check against the misuse of factor analysis. For our purposes especially, it should only be used to discover clues to things that can be described in a more straightforward manner.
There are other tricks that can be done with a correlation matrix. When there are several independent variables associated with the dependent variable, it is a shortcut for sorting out the effect of each by computing what is called the partial correlation coefficient. For example, in a survey of voters in Muncie, Indiana, in 1964, the correlation between political interest and income was .212, suggesting that people with money have more at stake in political decisions and therefore pay more attention to politics. On the other hand, income and education were very definitely correlated (r = .408), and there was a small but significant link between education and political interest (r = .181). Using contingency tables, it is possible to test the relationship between political interest and income by looking at it within categories of education. But that means another trip to the computing center. Partial correlation offers a quicker way to estimate the effect of holding education constant.
The correlation matrix has one other special application if you have panel data. It can, sometimes, give you an easy way to spot the direction of causation. Suppose you have interviewed the same people in two projects a year apart. Each survey shows a relationship between interest in local politics and time spent reading the morning paper. The question bothering you is which comes first in the causal chain (if one does come first; it could be a case of mutual causation). Out of the correlation matrix, you find that there is a significantly stronger relationship between interest in politics at time 1 and newspaper reading at time 2 than there is between newspaper reading at time 1 and interest in politics at time 2. The predominant direction of causation, then, is from interest in politics to newspaper reading. See Figure 6A.
You may have noticed by now that you are beginning to see social science methodology in a somewhat new light. With any luck, it should begin to look like something you can do rather than just passively observe and write about.
There is another corner which we have turned in this chapter. We have not, you may have noticed, made a big deal of significance testing in the discussion of survey analysis. And we have put a lot of emphasis on digging and searching procedures which don't quite square with the pure model of hypothesis testing which was presented earlier in this book.
Is this, then, the point of transition from scholarship to journalism? Not exactly. The better, more creative scholars know that the search for probable cause is where the action is in survey research, and statistical testing is trivial by comparison. The tests help you guard against the temptations of overinterpretation. But analysis of tables, with the introduction of third variables to go behind the superficial, two-variable relationships, is your protection against wrong interpretation. It is also your opportunity to discover causal sequences and explanations of the way things work in your community that you didn't suspect existed before.
In classical scientific method, you form a hypothesis, test it, and, if it fails the test, reject it and go on to something else. You don't keep cranking in epicycles as Ptolemy did until you have a clumsy, unparsimonious explanation to fit the observable data. Nevertheless, the rule against making interpretations after the fact, after the data are in and the printed-out tables are on your desk, is by no means ironclad. There is room in social science for serendipity. If the data give you an idea that you didn't have before, you need feel no guilt at all about pursuing it through the tables to see where it leads. Rosenberg notes that cases of serendipitous discoveries are plentiful in both the natural and the social sciences. He cites one of the best and most original concepts to emerge from modern survey research as an example: relative deprivation, uncovered by Samuel Stouffer in his research for The American Soldier.
Stouffer did not enter this survey with the idea that there might be such a thing as relative deprivation. The idea had not occurred to him and the survey was not designed to test it. But numbers came out that were so unexpected and so surprising that it was necessary to invent the idea of relative deprivation in order to live with them. One of the unexpected findings was that Northern blacks stationed in the South, despite their resentment of local racial discrimination, were as well or even better adjusted when compared to those stationed in the North. Another discrepancy turned up in the comparative morale of soldiers in units with high promotion rates and those in units with low chances of promotion: the low-promotion group was the happiest.
One parsimonious concept, relative deprivation, fit both of these situations. The black soldiers stationed in the South compared themselves to the black civilians they saw around them and found themselves better off. The high-promotion units had more soldiers who saw others being promoted and therefore felt more dissatisfaction at not getting ahead than soldiers in units where no one got ahead.
When the apparent discrepancies turned up, did the analysts shout “Eureka” and appreciate their importance in the history of social science? No. Instead, they delayed the report, went over the numbers again and again, hoping that some clerical error or something would show that the black soldiers and the low-promotion soldiers were not so happy after all. There is a moral here for journalists. We are not charged with the awesome responsibility of making original scientific discovery. We do have the responsibility of challenging and testing the conventional wisdom. And if the conventional wisdom says one thing and our data say another, we should, if the data are well and truly collected and analyzed, believe our data.
Rosenberg also has an answer for the methodological purists who say that after-the-fact interpretation is too much like Ptolemaic epicycle building. Accidental discoveries, he points out, are nullifiable. If you find something surprising, you can use your contingency tables to creep up on it from another direction to see if it is still there. Stouffer found something surprising in the attitude of black soldiers, invented the concept, and then tested it elsewhere in his data on the high- and low-promotion units.
A journalistic example is also available. When the Knight Newspapers surveyed Berkeley arrestees five years after the arrests one of the interesting findings was that females who had been radicalized by the Sproul Hall affair tended to hold on to that radicalization more than did males in the ensuing years. This finding, based on one table, led to the hypothesis that for a girl to become a radical involves a more traumatic separation from the values and attitudes of her family than it does for a male, and that she therefore holds on to the radical movement as a family substitute. The theory was testable from other data in the survey. If it were true, females should have a greater proportion of parents who disapproved of the activity that led to their getting arrested. Furthermore, those females with disapproving parents should be more likely to retain their radicalism.
Checking these two propositions required a new two-way table (sex by parent approval) and a three-way table (radical retention by parental approval by sex) and one trip to the computing center. It turned out that there was a small sex difference (though not statistically significant) in favor of males having parental approval. However, the effect of disapproving parents on radical retention was the same for boys and girls. So the theory was, for the purposes of this project at least, nullified.
“The post-factum interpretation,” says Rosenberg, “is thus not the completion of the analysis but only the first step in it. The interpretation is made conditional upon the presence of other evidence to support it.”
Thus it is not necessary to fall back on a journalistic excuse for using the computer as a searching device instead of a hypothesis-testing tool. The journalistic excuse would be that we are in too much of a hurry to be as precise as sociologists, and, besides, our findings are not going to be engraved on tablets of stone. Let us think twice before copping out like that. If we were really backed up against the wall in a methodological argument with scientific purists, we might have to take that last-resort position. Meanwhile, we can make the better argument that we are practical people, just as most sociologists are practical people, and therefore, when we spot the germ of an idea glimmering in our data, we need not shrink from its hot pursuit.
Subscribe to:
Posts (Atom)