The YouGov website is a great source for finding examples of social surveys and results from survey data.
YouGov is company which collects mainly survey data on a wide range of topics from people all over the world, and publishes it’s findings on a daily basis.
On their intro page they say ‘YouGov is a community of 4 million people around the world who share their views…. w’ere pretty sure its the largest daily updated database of people’s habits and opinions in the world’ – in addition to the structured survey data, some people also comment on the findings of said data, so you get a more qualitative feel added into the mix.
The data is very easy to access – for example below are YouGov’s latest findings on attitudes towards the children of illegal immigrants:
You can see from the above that we are pretty intolerant of illegal immigrants as a nation, which is one of the advantages of survey data.
You can also ‘drill down’ into the data and find correlations between attitudes and politics/ gender/ age and social class. Below we see that older people are less tolerant than younger people:
The advantages and disadvantages of social surveys
The big strength of this site is that it’s very accessible – you can very easily get some quick ‘facts’ about what people think about a lot of different topics, and you can easily see the correlations between attitudes and other variables such as class and gender.
The information contained in the site is also good for illustrating the limitations of survey data – you don’t really get any depth or explanation of why people hold these views (not even with the comments, because relatively few people comment).
Finally, I really like the fact that you get to see the specific question asked, so you can always bung a particular question, or set of questions on Socrative to check out the reliability with your students!
Social Surveys are a quantitative, positivist research method consisting of structured questionnaires and interviews. This post considers the theoretical, practical and ethical advantages and disadvantages of using social surveys in social research.
The Theoretical Advantages of Social Surveys
Detachment, Objectivity and Validity
Positivists favour questionnaires because they are a detached and objective (unbiased) method, where the sociologist’s personal involvement with respondents is kept to a minimum.
Questionnaires are particularly useful for testing hypotheses about cause and effect relationships between different variables, because the fact that they are quantifiable allows us to find correlations.
For example, based on government statistics on educational achievement we know that white boys on Free School Meals achieve at a significantly lower level than Chinese girls on Free School Meals. We reasonably hypothesise that this is because differences in parental attitudes – Chinese parents may value education more highly, and they may be stricter with their children when it comes to homework compared to white parents. Good questionnaire design and appropriate sampling would enable us to test out this hypothesis. Good sampling would further allow us to see if those white working class boys who do well have parents with similar attitudes to those Chinese girls who do well.
Questionnaires allow the researcher to collect information from a large number of people, so the results should be more representative of the wider population than with more qualitative methods. However, this all depends on appropriate sampling techniques being used and the researchers having knowledge of how actually completes the questionnaire.
Questionnaires are generally seen as one of the more reliable methods of data collection – if repeated by another researcher, then they should give similar results. There are two main reasons for this:
When the research is repeated, it is easy to use the exact same questionnaire meaning the respondents are asked the exact same questions in the same order and they have the same choice of answers.
With self-completion questions, especially those sent by post, there is no researcher present to influence the results.
The reliability of questionnaires means that if we do find differences in answers, then we can be reasonably certain that this is because the opinions of the respondents have changed over time. For this reason, questionnaires are a good method for conducting longitudinal research where change over time is measured.
Questionnaires are a quick and cheap means of gathering large amounts of data from large numbers of people, even if they are widely dispersed geographically if the questionnaire is sent by post or conducted online. It is difficult to see how any other research method could provide 10s of millions of responses as is the case with the UK national census.
In the context of education, Connor and Dewson (2001) posted nearly 4000 questionnaires to students at 14 higher education institutions in their study of the factors which influenced working class decisions to attend university.
With self-completion questionnaires there is no need to recruit and train interviewers, which reduces cost.
The data is quick to analyse once it has been collected. With online questionnaires, pre-coded questions can be updated live.
When a respondent is presented with a questionnaire, it is fairly obvious that research is taken place, so informed consent isn’t normally an issue as long as researchers are honest about the purpose of the research.
It is also a relatively unobtrusive method, given the detachment of the researcher, and it is quite an easy matter for respondents to just ignore questionnaires if they don’t want to complete them.
Theoretical Disadvantages of Questionnaires
Issues affecting validity – Interpretivists make a number of criticisms of questionnaires
Firstly there is the imposition problem – When the researcher chooses the questions, they are deciding what is important rather than the respondent, and with closed ended questions the respondent has to fit their answers into what’s on offer. The result is that the respondent may not be able to express themselves in the way that want to. The structure of the questionnaire thus distorts the respondents’ meanings and undermines the validity of the data
Secondly, Interpretivists argue that the detached nature of questionnaires and the lack of close contact between researcher and respondent means that there is no way to guarantee that the respondents are interpreting the questions in the same way as the researcher. This is especially true where very complex topics are involved – If I tick ‘yes’ that I am Christian’ – this could mean a range of things – from my being baptised but not practising or really believing to being a devout Fundamentalist. For this reason Interpretivists typically prefer qualitative methods where researchers are present to clarify meanings and probe deeper.
Thirdly, researchers may not be present to check whether respondents are giving socially desirable answers, or simply lying, or even to check who is actually completing the questionnaire. At least with interviews researchers are present to check up on these problems (by observing body language or probing further for example)
Issues affecting representativeness
Postal questionnaires in particular can suffer from a low response rate. For example, Shere Hite’s (1991) study of ‘love, passion, and emotional violence’ in the America sent out 100, 000 questionnaires but only 4.5% of them were returned.
All self-completion questionnaires also suffer from the problem of a self-selecting sample which makes the research unrepresentative – certain types of people are more likely to complete questionnaires – literate people for example, people with plenty of time, or people who get a positive sense of self-esteem when completing questionnaires.
Practical Problems with Questionnaires
The fact that questionnaires need to be brief means you can only ever get relatively superficial data from them, thus for many topics, they will need to be combined with more qualitative methods to achieve more insight.
Although questionnaires are a relatively cheap form of gathering data, it might be necessary to offer incentives for people to return them.
Structured Interviews are also considerably more expensive than self-completion questionnaires.
Ethical Issues with questionnaires
They are best avoided when researching sensitive topics.
A Social Survey involves obtaining information in a standardised from large groups of people. The main survey methods are questionnaires and structured interviews.
Two well-known examples of Social Surveys in the United Kingdom include:
The UK National Census – which is sent out to every UK household every ten years and asks basic information about who lives in the household, employment, education, religion and health.
The British Social Attitudes Survey – which has a sample of around 3000 and asks people a range of questions to measure opinions on a range of topics – such as family life, religious belief, immigration and environmental issues.
Surveys are carried out by a wide range of organisations such as government departments, schools and colleges, businesses, charities, and market research and consumer groups. You may well have been stopped in a high street by a market researcher asking your opinion about a new design of chocolate bar wrapper, or phoned by an independent polling company such as Mori asking you to do a brief survey on any number of social issues.
Types of Social Survey and Key Terms
Social Surveys are typically questionnaires designed to collect information from large numbers of people in standardised form. Surveys are prepared in advance of giving them to respondents, and so they have a ‘structure’ to them. Most questionnaires will have a high degree of structure, and it is difficult to see how one could have an ‘unstructured questionnaire’. Because of this questionnaires tend to be a very formal means of collecting data, allowing the researcher little freedom to ‘follow her nose’ unlike other methods such as unstructured interviews or participant observation.
Pre-coded, or closed question questionnaires are those in which the respondent has to choose from a limited range of responses. Two of the most common types of closed questionnaire are the simply yes/no questionnaire and the scaled questionnaire, where respondents are asked to either strongly agree, agree, disagree, or strongly disagree with a particular statement. This later form of scaling is known as a ‘Likert Scale’ (basically a strength of feeling scale).
One of the main problems of this type of questionnaire is the imposition problem, which refers to the risk that the research might be imposing their view, or framework on respondents rather than getting at what they really think about the issue.
Open-ended question questionnaires are less structured than pre-coded questionnaires. Although open-ended questionnaires will still usually have set questions, there is no pre-set choice of answers. Open questions allow individuals to write their own answers or dictate them to interviewers.
Different ways of administering surveys
The researcher has a choice of administering her questionnaire in a number of different ways. The most obvious difference choice is between whether respondents complete the surveys themselves, making it a ‘self-completion questionnaire’, or whether the researcher fills in the information, effectively making it a structured interview.
Some of the more obvious choices for ‘administering’ questionnaires include:
• Sending questionnaires by post, or by email.
• Simply putting the questionnaire online and leaving it to be completed
• Doing a structured interview in person, either on the street, house to house.
• Doing the interview by phone.
Structured interviews with closed questions
One obvious way of improving the response rate to questionnaires is to conduct a face to face interview by paying a researcher to read out the questionnaire to the respondent and writing down their responses on their behalf. Having an interviewer present can also reduce misinterpretation of questions as respondents can ask for clarification where necessary and an interviewer can also target specific groups if necessary, as with much market research.
On the downside, structured interviews are more time consuming. One researcher can only do one interview at a time (although focus groups are an exception to this, they too are limited in terms of the amount of respondents one can deal with in one go) whereas a self completion questionnaire can be administered to hundreds of people within minutes.
Structured Questionnaires and Interviewer Bias
At a more theoretical level, having an interviewer present opens up the possibility of interviewer bias occurring, where the presence of the researcher interferes with the results obtained. The social characteristics of the interviewer may affect the responses, depending on the age, gender and ethnicity of the researcher in relation to the respondent. If one is researching the prevalence of domestic violence against women, for example, one might reasonably expect a female victim to give different responses to a female researcher rather than a male researcher.
Each interviewer will have their own style of interviewing; right from selecting who they ask questions to if they are on the street, to the tone of voice, facial expressions, and pacing of the interview. Each of these nuances may affect the results, meaning the reliability of the research is compromised because it is difficult for another researcher to repeat the exact conditions under which previous interviews took place. To be fair, with closed question, structured interviews, and with trained researchers, such interviewer bias should be kept to a minimum, and such problems are likely to be more exaggerated with more qualitative unstructured interviews, which we will come onto later.