Social Surveys – Strengths and Limitations

Social surveys, a quantitative research method using structured questionnaires and interviews, have theoretical strengths in detachment, hypothesis testing, representativeness and reliability, but face limitations in imposition, misinterpretation, and representativeness issues. They are practical for gathering large data sets but offer only superficial insights. Ethically, they provide informed consent and unobtrusiveness but may not be suitable for sensitive topics.

Last Updated on July 9, 2024 by Karl Thompson

Social Surveys are a quantitative, positivist research method consisting of structured questionnaires and interviews. This post considers the theoretical, practical and ethical advantages and disadvantages of using social surveys in social research. 

The strengths and limitations below are mainly based around surveys administered as self-completion questionnaires.

Social Surveys.png

Theoretical Factors

slide showing the theoretical strengths and limitations of social surveys

Theoretical strengths of social surveys

Detachment, Objectivity and Validity

Positivists favour questionnaires because they are a detached and objective (unbiased) method, where the sociologist’s personal involvement with respondents is kept to a minimum.

Hypothesis Testing

Questionnaires are particularly useful for testing hypotheses about cause and effect relationships between different variables, because the fact that they are quantifiable allows us to find correlations.

For example, based on government statistics on educational achievement we know that white boys on Free School Meals achieve at a significantly lower level than Chinese girls on Free School Meals. We reasonably hypothesise that this is because differences in parental attitudes – Chinese parents may value education more highly, and they may be stricter with their children when it comes to homework compared to white parents. Good questionnaire design and appropriate sampling would enable us to test out this hypothesis. Good sampling would further allow us to see if those white working class boys who do well have parents with similar attitudes to those Chinese girls who do well.

Representativeness

Questionnaires allow the researcher to collect information from a large number of people, so the results should be more representative of the wider population than with more qualitative methods. However, this all depends on appropriate sampling techniques being used and the researchers having knowledge of how actually completes the questionnaire.

Reliability

Questionnaires are generally seen as one of the more reliable methods of data collection – if repeated by another researcher, then they should give similar results. There are two main reasons for this:

When the research is repeated, it is easy to use the exact same questionnaire meaning the respondents are asked the exact same questions in the same order and they have the same choice of answers.

With self-completion questions, especially those sent by post, there is no researcher present to influence the results.

The reliability of questionnaires means that if we do find differences in answers, then we can be reasonably certain that this is because the opinions of the respondents have changed over time. For this reason, questionnaires are a good method for conducting longitudinal research where change over time is measured.

Social surveys are a good method for measuring change over time.

Theoretical Limitations

Issues affecting validity – Interpretivists make a number of criticisms of questionnaires.

The Imposition Problem

The imposition problem is when the researcher chooses the questions, they are deciding what is important rather than the respondent, and with closed ended questions the respondent has to fit their answers into what’s on offer. The result is that the respondent may not be able to express themselves in the way that want to. The structure of the questionnaire thus distorts the respondents’ meanings and undermines the validity of the data.

Misinterpetation of questions

Interpretivists argue that the detached nature of questionnaires and the lack of close contact between researcher and respondent means that there is no way to guarantee that the respondents are interpreting the questions in the same way as the researcher. This is especially true where very complex topics are involved – If I tick ‘yes’ that I am Christian’ – this could mean a range of things – from my being baptised but not practising or really believing to being a devout Fundamentalist. For this reason Interpretivists typically prefer qualitative methods where researchers are present to clarify meanings and probe deeper.

Researchers may not be present to check whether respondents are giving socially desirable answers, or simply lying, or even to check who is actually completing the questionnaire. At least with interviews researchers are present to check up on these problems (by observing body language or probing further for example).

Issues affecting representativeness

Postal questionnaires in particular can suffer from a low response rate. For example, Shere Hite’s (1991) study of ‘love, passion, and emotional violence’ in America sent out 100, 000 questionnaires but only 4.5% of them were returned.

All self-completion questionnaires also suffer from the problem of a self-selecting sample which makes the research unrepresentative – certain types of people are more likely to complete questionnaires – literate people for example, people with plenty of time, or people who get a positive sense of self-esteem when completing questionnaires.

The Pandemic has reduced representativeness of many government surveys

Several government departments carry out annual surveys of the population using random postcode sampling. With this sampling method, addresses are generated to represent the whole country and advance letters sent out. It is then up to interviewers to convince those selected to take part. With these surveys Interviewers are not allowed to go next door if one address refuses to take part.

For example the department of transport carries out the National Travel Survey every year. This has a sample of over 10 000 surveys. During the Pandemic interviewers door knocked and tried to book video or phone interviews with respondents. In 2020, then main lockdown year the response rate dropped to just 16%. By 2022 it was back up to 31%, but this was a long way down from 54% in 2019.

graph showing declining response rate with the national travel survey 2013 to 2022.

The same trend has happened with a number of other government surveys such as the Health and Housing surveys. It seems the Pandemic has made people reluctant to take part in these government surveys.

Practical Factors

Slide showing the practical strengths and limitations of social surveys.

Practical Strengths of Social Surveys

Questionnaires are a quick and cheap means of gathering large amounts of data from large numbers of people, even if they are widely dispersed geographically if the questionnaire is sent by post or conducted online. It is difficult to see how any other research method could provide 10s of millions of responses as is the case with the UK national census.

In the context of education, Connor and Dewson (2001) posted nearly 4000 questionnaires to students at 14 higher education institutions in their study of the factors which influenced working class decisions to attend university.

With self-completion questionnaires there is no need to recruit and train interviewers, which reduces cost.

The data is quick to analyse once it has been collected. With online questionnaires, pre-coded questions can be updated live.

Practical Limitations

The fact that questionnaires need to be brief means you can only ever get relatively superficial data from them, thus for many topics, they will need to be combined with more qualitative methods to achieve more insight.

Although questionnaires are a relatively cheap form of gathering data, it might be necessary to offer incentives for people to return them.

Structured Interviews are also considerably more expensive than self-completion questionnaires.

Ethical Factors

slide showing the ethical strengths and limtiations of social surveys

Ethical strengths of surveys

When a respondent is presented with a questionnaire, it is fairly obvious that research is taken place, so informed consent isn’t normally an issue as long as researchers are honest about the purpose of the research.

It is also a relatively unobtrusive method, given the detachment of the researcher, and it is quite an easy matter for respondents to just ignore questionnaires if they don’t want to complete them.

Ethical Limitations

Social Surveys are best avoided when researching sensitive topics. For example a closed questionnaire about the impacts of domestic violence on victims may be a little cold!

Related Posts 

An Introduction to Social Surveys – Definition and Basic Types of Survey

Positivism, Sociology and Social Research – Positivists like the survey method.

Please click here for more posts on research methods.

7 thoughts on “Social Surveys – Strengths and Limitations”

  1. Hey there, Lots of love and best wishes from India. I have no training in sociology what so ever and is having very crucial exam which will require graduation level command over the subject. With paucity of time somehow found this gem of a site. I almost have never commented online , but this is love. Best wishes….

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Discover more from ReviseSociology

Subscribe now to keep reading and get access to the full archive.

Continue reading