Table of Contents (100 pages)
2 Scientific Method
3 Experimental vs. Non-Experimental
6 Reliability and Validity
9 Survey & Content Analysis Instruments
10 Collecting Information
11 Interpreting the Mean
12 Presenting Results
13 Research Ethics
14 Mass Media Research
Appendix – Analysis of Means Standard
A student analyses the content of selected academic papers for a literature review; the public is tested for the purpose of marketing a new product; a scientist uses genetic splicing techniques to find a new grain that will give higher yields; and so it goes, on and on. What is happening here? Research is happening here, that’s what…careful investigation using systematic methods that enable analysis of the results.
2 SCIENTIFIC METHOD
There is a scientific method, both in the physical sciences such as chemistry, and the social sciences such as communication. The differences are more a matter of precision than the ways and means of research. The same precision that is possible in the physical sciences (generally using quantitative methods for measurement of variables) is not possible when measuring people in the social sciences (generally using qualitative methods for measurement of values and experiences). Science is public, objective, empirical, systematic, cumulative and predictive.
3 EXPERIMENTAL vs. NON-EXPERIMENTAL RESEARCH
We must differentiate between these two large areas of research because there is quite a difference. In experimental research an independent variable is manipulated in some way to find out how it will affect the dependent variable. In non-experimental research there is no such manipulation. This is one of those times when the more you look at the words such as independent, dependent, and variable, the worse it gets. A few examples will help.
Variance in research has to do with how things differ from one another. It is, in fact, the whole point of doing the research in the first place. We look for variation…the differences between the ratings of different radio stations, the value placed on a course or an instructor by students, and so forth. After all, if everyone felt the same about everything and behaved the same, there would be no need to conduct social research. If we couldn’t find new species in agriculture which increase yields or have better resistance to cold or dry climates, then there would be no point in doing agricultural research.
As error variance decreases,
true variance increases,
making the results more dependable.
The word statistics has more than one meaning. On the one hand it represents a complete branch of mathematics that seems so strange and so difficult that it strikes fear in the hearts of most of us. If any person plans to enter into the field of research as a career, this branch of mathematics must be studied, understood and become an integral part of that person. On the other hand, a simple number that represents the size of a group is by itself a statistic. If someone asks you “How many people will be at the picnic?”…that’s a statistic. We do not always need to think of those strange symbols and impossible formulas every time we hear that feared word.
6 RELIABILITY and VALIDITY
The concepts of reliability and validity are extremely important to any research and both belong to the research instruments used. The reliability of any method of collecting data is a statistically determined coefficient (number), ranging from -1 to +1, and is defined as the ability of a measurement instrument to give the same results when it is repeated using the same subjects. The validity of an instrument is determined primarily in philosophical terms regarding its value or relevance. There is no simple statistical method to determine the validity of an instrument.
7 SAMPLING – How big is big?
If we are going to do a descriptive research job such as an evaluation of an instructor in a course, then the entire population of the class will participate; the entire population of anything will give us the best results. But if we need to find out how many people watch a particular television program in a city of 250,000 people, it becomes obvious that we cannot contact every person, therefore we must use random or systematic procedures to obtain a manageable sample size.
Anything can be measured; in research you need to determine what to measure and how precise the measurement needs to be, or can be. A familiar object such as a penny, provides comparison for determining scale. We can frame a house with an error variance of about 1/4 inch, but must have an error variance of less than 1/32 inch if we are cutting furniture parts. We can count the number of people or things with no error variance, but how do we measure a person’s values or experience?
9 SURVEY and CONTENT ANALYSIS INSTRUMENTS
The development of a good instrument is more of an art than it is a science. The researcher who is developing an instrument which purports to get into the minds of the subjects must, in fact, try to do so before writing the questions. Any confusion on the subjects’ part will introduce error variance that is buried in the data, making the outcome questionable. Most of us have experience frustration in completing a survey because the questions were not understandable. An unintelligible instrument is certainly not operational, and to be operational must be considered the first requirement of good instrumentation.
A continuum Likert scale enables a researcher to capture values using survey questions. Written material can be analysed to identify themes.
Data collection forms must be carefully designed to ensure data collection is not biased by the researcher.
10 COLLECTING INFORMATION
Some historical methods are summarized, plus considerations are presented for a variety of common data collection methods including surveys, interviews, observation, focus groups and diaries/journals and data analysis.
Face-to-face collection methods (interview, focus group) provide the highest quality data, but are time consuming. Surveys and diaries allow for broad sampling, but reliability can be an issue. A blended approach is good.
11 INTERPRETING THE MEAN
The mathematical concept of the mean is a more precise way of determining the average. The mean is the best number that can represent a large series of numbers. There is no particular problem with the interpretation of the mean with an example such as the question, “What is the average age of the students in career programs in the college?” However, if we use a rating scale of any kind and ask the subjects of our research a question of value, how do we interpret the resultant mean? Can we simply place the mean on the same instrument device that we used in the data collection?
Methods for calculating the mean and standard deviation are illustrated in the book, with examples using the respective formulas.
12 PRESENTING THE RESULTS
The results of all the data must finally come together in a research report or academic paper. This document should be complete, concise and well structured so that the research results can be easily understood, are clearly supported and can be repeated. Even though a replication may never happen, it assures that the results can be verified by the reader. The report should be written for the target audience (usually the people who asked for the research in the first place). Ideally, the results are presented in clear, direct language, so the information is accessible to decision makers, other researchers and if appropriate, the general public.
13 RESEARCH ETHICS
People have certain rights which must be acknowledged and not violated. Each research situation has its own problems to solve, but a simple guideline captures the spirit behind ethical decisions and the need for informed consent, “Do unto others as you would have them do unto you.”
The Canadian Interagency Advisory Panel on Research Ethics offers a comprehensive set of general guidelines: TCPS2 Tri-Council Policy Statement, 2nd Ed.: Ethical Conduct for Research Involving Humans
14 MASS MEDIA RESEARCH
TV and radio programs and other advertising-based media offerings are sustained or cancelled based on ratings derived from mass media research (e.g. Nielson TV diaries). Statistically valid results for Canada’ population of 34 million can be derived from 1000 randomly sampled individuals; 2000 for the USA’s 304 million.
APPENDIX – What does the mean mean?
The author proposes a standard for analyzing means, by overlaying a 9-point interpretation scale on a 5-point continuum survey scale to help articulate the meaning of the results.