Last Updated on July 9, 2024 by Karl Thompson
Social Surveys are a quantitative, positivist research method consisting of structured questionnaires and interviews. This post considers the theoretical, practical and ethical advantages and disadvantages of using social surveys in social research.
The strengths and limitations below are mainly based around surveys administered as self-completion questionnaires.
Theoretical Factors
Theoretical strengths of social surveys
Detachment, Objectivity and Validity
Positivists favour questionnaires because they are a detached and objective (unbiased) method, where the sociologist’s personal involvement with respondents is kept to a minimum.
Hypothesis Testing
Questionnaires are particularly useful for testing hypotheses about cause and effect relationships between different variables, because the fact that they are quantifiable allows us to find correlations.
For example, based on government statistics on educational achievement we know that white boys on Free School Meals achieve at a significantly lower level than Chinese girls on Free School Meals. We reasonably hypothesise that this is because differences in parental attitudes – Chinese parents may value education more highly, and they may be stricter with their children when it comes to homework compared to white parents. Good questionnaire design and appropriate sampling would enable us to test out this hypothesis. Good sampling would further allow us to see if those white working class boys who do well have parents with similar attitudes to those Chinese girls who do well.
Representativeness
Questionnaires allow the researcher to collect information from a large number of people, so the results should be more representative of the wider population than with more qualitative methods. However, this all depends on appropriate sampling techniques being used and the researchers having knowledge of how actually completes the questionnaire.
Reliability
Questionnaires are generally seen as one of the more reliable methods of data collection – if repeated by another researcher, then they should give similar results. There are two main reasons for this:
When the research is repeated, it is easy to use the exact same questionnaire meaning the respondents are asked the exact same questions in the same order and they have the same choice of answers.
With self-completion questions, especially those sent by post, there is no researcher present to influence the results.
The reliability of questionnaires means that if we do find differences in answers, then we can be reasonably certain that this is because the opinions of the respondents have changed over time. For this reason, questionnaires are a good method for conducting longitudinal research where change over time is measured.
Theoretical Limitations
Issues affecting validity – Interpretivists make a number of criticisms of questionnaires.
The Imposition Problem
The imposition problem is when the researcher chooses the questions, they are deciding what is important rather than the respondent, and with closed ended questions the respondent has to fit their answers into what’s on offer. The result is that the respondent may not be able to express themselves in the way that want to. The structure of the questionnaire thus distorts the respondents’ meanings and undermines the validity of the data.
Misinterpetation of questions
Interpretivists argue that the detached nature of questionnaires and the lack of close contact between researcher and respondent means that there is no way to guarantee that the respondents are interpreting the questions in the same way as the researcher. This is especially true where very complex topics are involved – If I tick ‘yes’ that I am Christian’ – this could mean a range of things – from my being baptised but not practising or really believing to being a devout Fundamentalist. For this reason Interpretivists typically prefer qualitative methods where researchers are present to clarify meanings and probe deeper.
Researchers may not be present to check whether respondents are giving socially desirable answers, or simply lying, or even to check who is actually completing the questionnaire. At least with interviews researchers are present to check up on these problems (by observing body language or probing further for example).
Issues affecting representativeness
Postal questionnaires in particular can suffer from a low response rate. For example, Shere Hite’s (1991) study of ‘love, passion, and emotional violence’ in America sent out 100, 000 questionnaires but only 4.5% of them were returned.
All self-completion questionnaires also suffer from the problem of a self-selecting sample which makes the research unrepresentative – certain types of people are more likely to complete questionnaires – literate people for example, people with plenty of time, or people who get a positive sense of self-esteem when completing questionnaires.
The Pandemic has reduced representativeness of many government surveys
Several government departments carry out annual surveys of the population using random postcode sampling. With this sampling method, addresses are generated to represent the whole country and advance letters sent out. It is then up to interviewers to convince those selected to take part. With these surveys Interviewers are not allowed to go next door if one address refuses to take part.
For example the department of transport carries out the National Travel Survey every year. This has a sample of over 10 000 surveys. During the Pandemic interviewers door knocked and tried to book video or phone interviews with respondents. In 2020, then main lockdown year the response rate dropped to just 16%. By 2022 it was back up to 31%, but this was a long way down from 54% in 2019.
The same trend has happened with a number of other government surveys such as the Health and Housing surveys. It seems the Pandemic has made people reluctant to take part in these government surveys.
Practical Factors
Practical Strengths of Social Surveys
Questionnaires are a quick and cheap means of gathering large amounts of data from large numbers of people, even if they are widely dispersed geographically if the questionnaire is sent by post or conducted online. It is difficult to see how any other research method could provide 10s of millions of responses as is the case with the UK national census.
In the context of education, Connor and Dewson (2001) posted nearly 4000 questionnaires to students at 14 higher education institutions in their study of the factors which influenced working class decisions to attend university.
With self-completion questionnaires there is no need to recruit and train interviewers, which reduces cost.
The data is quick to analyse once it has been collected. With online questionnaires, pre-coded questions can be updated live.
Practical Limitations
The fact that questionnaires need to be brief means you can only ever get relatively superficial data from them, thus for many topics, they will need to be combined with more qualitative methods to achieve more insight.
Although questionnaires are a relatively cheap form of gathering data, it might be necessary to offer incentives for people to return them.
Structured Interviews are also considerably more expensive than self-completion questionnaires.
Ethical Factors
Ethical strengths of surveys
When a respondent is presented with a questionnaire, it is fairly obvious that research is taken place, so informed consent isn’t normally an issue as long as researchers are honest about the purpose of the research.
It is also a relatively unobtrusive method, given the detachment of the researcher, and it is quite an easy matter for respondents to just ignore questionnaires if they don’t want to complete them.
Ethical Limitations
Social Surveys are best avoided when researching sensitive topics. For example a closed questionnaire about the impacts of domestic violence on victims may be a little cold!
Related Posts
An Introduction to Social Surveys – Definition and Basic Types of Survey
Positivism, Sociology and Social Research – Positivists like the survey method.
Please click here for more posts on research methods.
Hey there, Lots of love and best wishes from India. I have no training in sociology what so ever and is having very crucial exam which will require graduation level command over the subject. With paucity of time somehow found this gem of a site. I almost have never commented online , but this is love. Best wishes….