Tag: questionnaires

Surveys on Family Life in the UK

Social Surveys are one of the most common methods for routinely collecting data in sociology and the social sciences more generally. There are lots of examples of where we use social surveys throughout the families and households module in the A level sociology syllabus – so what do they tell us about family life in modern Britain, and what are their strengths and limitations….?

This information should be useful for both families and households and for exploring the strengths and limitations of social surveys for research methods…

Attitudes to non-traditional families

Findings from British Social Attitudes 37 suggests that the British public is becoming more supportive of non-traditional families.

Headline Fact – in 2018/19 only 12% of the UK population disapproved of people having children while cohabiting, down from 22% in 2006/07.

tables of statistics showing changes in attitudes to non traditional families in the UK 2007-2019.

Disapproval for all types of non-traditional families decreased between 2006/7 and 2018/19.

Further analysis demonstrated this was due to a generational affect. In the previous round of surveys (2006/07) 50% of people born before 1927 disapproved of traditional families. However most of these had died by 2018/19 and so their attitudes did not show up in these later survey results.

Back in 1989, seven people in ten (70%) felt that people should be married if they want to have children, compared with less two in ten (17%) who disagreed.

The number of people who think couples should get married before having children has fallen dramatically in the last 40 years. In 1989 70% of people believed couples should get married before having children. This had fallen to 42% in 2012.

Percentage of the UK population who agree that parents should be married before they have children.

Judging by the low 12% disapproval rate for ‘cohabiting with children’ it seems reasonable to say that even fewer people think couples need to get married before having children in 2023!

What are the strengths of this survey

  • I’m tempted to say the validity is probably quite good, as this isn’t a particularly sensitive topic, and the focus of the questions is the ‘generalised other’, so there should be no social desirability.
  • It’s very useful for making comparisons over time – given that the same questions have been asked in pretty much the same way over different years.
  • Representativeness seems to be OK – NatCen sampled a range of ages, and people with different political views, so we can compare all that too – no surprises here btw – the old and the conservatives are more likely to be in favour of marriage.

What are the limitations of this survey?

The question above is so generalised, it might give us a false impression of how liberal people are. I wonder how much the results would change if you made the questions more personal – would you rather your own son/ daughter should be married before they had children? Or just different – ‘all other things being equal, it’s better for children to be brought up by married parents, rather than by non-married-parents’ – and then likehert scale it. Of course that question itself is maybe just a little leading….

Housework Surveys 

Headline ‘fact’ – women still do 60% more housework than men (based on ONS data from 2014-15)

housework UK

Women carry out an overall average of 60% more unpaid work than men, ONS analysis has shown.

Women put in more than double the proportion of unpaid work when it comes to cooking, childcare and housework and on average men do 16 hours a week of such unpaid work compared to the 26 hours of unpaid work done by women a week.

The only area where men put in more unpaid work hours than women is in the provision of transport – this includes driving themselves and others around, as well as commuting to work.

This data is derived from the The UK Time Diary Study (2014-15) – which used a combination of time-use surveys and interviews to collect data from around 9000 people in 4000 households.

It’s worth noting that even though the respondents were merely filling in a few pages worth of diary, this document contains over 200 pages of technical details, mainly advice on how researchers are supposed to code responses.

What are the strengths of this survey?

  • The usual ease of comparison. You can clearly see the differences in hours between men and women – NB the survey also shows differences by age and social class, but I haven’t included that here (to keep things brief).
  • It’s a relatively simply topic, so there’s unlikely to be any validity errors due to interpretation on the part of people completing the surveys: it’s obvious what ‘washing clothes’ means for example.
  • This seems to suggest the continued relevance of Feminism to helping us understand and combat gender inequality in the private sphere.

What are the limitations of this data? 

  • click on the above link and you’ll find that there is only a 50% response rate…. which makes the representativeness of this data questionable. If we take into account social desirability, then surely those couples with more equal housework patterns will more likely to return then, and also the busier the couple, the less likely they are to do the surveys. NO, really not convinced about the representativeness here!
  • this research tells us nothing about why these inequalities exist – to what extent is this situation freely chosen, and to what extent is it down to an ‘oppressive socialisation into traditional gender norms’ or just straightforward coercion?
  • given all of the coding involved, I’m not even convinced that this is really that practically advantageous…. overall this research seems to have taken quite a long time, which is a problem given the first criticism above!

Surveys on Children’s Media Usage

TikTok, Snapchat and Instagram are the most popular social media apps used by teenagers. Around 40% of teenagers use these three apps.

bar chart showing percentages of children who use social media, UK 2023.

Facebook usage among teens has decreased in recent years.

The data which produced these results comes from three separate tracking studies with a combined sample of over 10 000 children and their parents, data collected between March-November 2022. (OFCOM: Children and Parents: Media Use and Attitude Report).

Strengths of this Survey

  • It makes comparisons over time easy, as the same questions are asked over a number of different years.
  • Other than that, I think there are more problems!

Limitations of this Survey

  • There are no details of how the sample was achieved in the methodology – so I can’t comment on the representativeness.
  • There is so much data here it’s difficult to get an overview. For example there are no longer any easily accessible stats on how much total time children spend with media. The figures are broken down by media type, split into TV, gaming and social media for example.
SIGNPOSTING AND RELATED POSTS.

This material is mainly relevant to the families and households module, usually taught as part of the first year in A-level sociology.

Outline and Explain Two Theoretical Problems of Using Social Surveys in Social Research

Firstly, social surveys suffer from the imposition problem, closed questions limits what respondents can say Interpretivists argue respondents have diverse motives and it is unlikely that researchers will think up every possible relevant question and every possible, response, thus questionnaires will lack validity.

This is especially true for more complex topics such as religions belief – ticking the ‘Christian’ box can mean many different things to many different people, for example.

Interpretivists thus say that surveys are socially constructed—they don’t reflect reality, but the interests of researchers

However, this is easily rectified by including a section at the end of questionnaires in which respondents can write their explanations.

Secondly, self-completion surveys can also suffer from poor representativeness…

Postal questionnaires can suffer from a low response rate, and samples might be self-selecting— due to the illiterate or people who might be ashamed/ scared to return questionnaires on sensitive topics.

Also, you can’t check who has filled them in, so surveys may actually misrepresent the target population.

However, it is possible to rectify this with incentives and booster samples.

The above is a suggested response to a possible 10 mark ‘pure methods’ question which might come up on either paper 1 or 3 of the AQA’s A Level Sociology Papers. It follows the basic formula – make a point, develop it twice, and then evaluate it (which to my mind seems to work well for ‘pure methods’ 10 mark questions. 

Theory and Methods A Level Sociology Revision Bundle 

If you like this sort of thing, then you might like my Theory and Methods Revision Bundle – specifically designed to get students through the theory and methods sections of  A level sociology papers 1 and 3.

Contents include:

  • 74 pages of revision notes
  • 15 mind maps on various topics within theory and methods
  • Five theory and methods essays
  • ‘How to write methods in context essays’.

Using self-completion written questionnaires to investigate unauthorised absences from school

An example of a methods in context question, mark scheme, and some thoughts on how to answer the question. The ‘methods in context question’ appears on paper 1 at both AS and A Level, and it’s the same format in both papers.

The item and question below are taken directly from an AQA AS sociology specimen paper, I’ve put in bold the useful ‘hooks’ in the item.

Example of a Methods in Context Question:

(05) Read item B, then answer the question below

Item B

Investigating unauthorised absences from school

There is a close correlation between frequent unauthorised absence from school and educational underachievement. Those pupils who are not doing well at school are more likely to truant. Similarly, those who truant regularly are likely to finish their school career with poor qualifications. Pupils may be absent without authorisation for many reasons, from caring responsibilities at home or dislike of school, to parents arranging family holidays in term time.

Sociologists may use self-completion written questionnaires to study unauthorised absences. These can be distributed easily to large numbers of pupils, parents or teachers. The findings of the questionnaires can also be used to establish patterns and trends in relation to unauthorised absences. However, self-completion questionnaires often have very low response rates, especially when they ask about sensitive issues.

Applying material from Item B and your knowledge of research methods, evaluate the strengths and limitations of using self-completion written questionnaires to investigate unauthorised absences from school (20 marks)

Examples of ‘Top Band’ Statements

If you top and tail this with an intro paragraph about Positivism and the strengths/ limitations of the method (thus show good knowledge and evaluation of the method in general) and a conclusion saying it’s a pretty crap method, then just 3-4 of these statements below should be enough to get you into the top mark band (17-20)

  • An advantage of self-completion questionnaires is that they can be distributed easily to large numbers of pupils, parents, or teachers, HOWEVER there are numerous reasons why pupils who are absent from school without being authorised won’t want to fill in the questionnaires – as the item states, such pupils may not be doing well at school and would be reluctant to fill in a questionnaire about something they don’t like (school), which could result in a low response rate
  • A second reason for a low response rate is, as the item states, because students have caring responsibilities at home, and they may not have time to complete the questionnaire, or they may not see it as important as their caring duties.
  • Another problem is that validity of responses may be low – if unauthorised absences are due to parents arranging holidays in term time, they may not want to admit to this in a questionnaire because they may have lied about this reason to the school to avoid a fine.
  • The item states that self completion questionnaires are a good way of finding trends and you could use them to explore the relationship between unauthorised absences and low qualifications, however, if people have low qualifications they may have low literacy levels, meaning they would not be happy filling in a questionnaire, so a booster sample would be required, or another method for such people, such as structured interviews, but this would reduce the reliability.
  • One advantage of the method is that you can distribute large numbers of questionnaires quickly, and they are usually quick to fill in, so teachers would like them as they have busy schedules, and would also probably be happy to talk about this issue, given its negative effects.
  • One problem with this method is the imposition problem – you need to set questions in advance, and as the item says, there are many reasons for unauthorised absences, they problem is that you may not discover these reasons if you don’t include it in the questionnaire in the first place.
  • This imposition problem would be a problem especially if absences are due to bullying, which is a sensitive issue – even if it is on the questionnaire, it’s quite a cold method and so respondents may not want to discuss it in a ‘tick box’ manner.
  • A final advantage of this method is that it is anonymous, which may outweigh some of the problems above.

Methods in Context CoverIf you like this sort of thing, then why not purchase my handy ‘How to Write Methods in Context Essays‘ hand-out, a bargain at only £1.49, and who knows, it may prevent you from being the victim in a future research study focusing on why certain students fail their A levels… 

It covers the following processes of how to deal with Methods in Context (MIC) questions.

  1. It starts off by looking at an example of a methods in context question and a mark scheme and outlines what you need to do to get into the top three mark bands.
  2. It tells you how to plan methods in context essays.
  3. It tells you how to actually write methods in context essays – presenting a ‘safe’ strategy to get into at least mark band 4 (13-16)
  4. In total it provides three examples of how you might go about answering a three different MIC questions.

The Mark Scheme (top three bands)

Top Mark Band (17-20) – Good knowledge of method and applies the method to the specific topic

‘Students will apply knowledge of a range of relevant strengths and limitations of using self-completion written questionnaires to research issues and characteristics relating to unauthorised absences from school.

These may include some of the following and/or other relevant concerns, though answers do not need to include all of these, even for full marks:

the research characteristics of potential research subjects, eg individual pupils, peer groups, parents, teachers (eg class, ethnic and gender differences; parental literacy skills; teachers’ professionalism, self-interest or stereotypes of pupils)

contexts and settings (eg classrooms; staffrooms)

the sensitivity of researching unauthorised absences from school (eg policy and resource implications for schools; schools’ market and league table position; its impact on achievement or behaviour; stigmatisation; parental consent).’

Fourth Mark Band (13-16) – Good knowledge of method and applies the method to education in general.

‘Application of knowledge will be broadly appropriate but will be applied in a more generalised way or a more restricted way; for example:

applying the method to the study of education in general, not to the specifics of studying unauthorised absences from school, or

specific but undeveloped application to unauthorised absences from school, or

a focus on the research characteristics of unauthorised absences from school, or groups/contexts etc involved in it.’

Middle Mark Band (9-12) – Good knowledge of method, loosely applied to education

‘Largely accurate knowledge but limited range and depth, including a broadly accurate, if basic, account of some of the strengths and/or limitations of self completion written questionnaires.

Understands some limited aspects of the question; superficial understanding of the presented material.

Applying material (possibly in a list-like fashion) on self-completion written questionnaires, but with very limited or non-existent application to either the study of unauthorised absences from school in particular or of education in general. ‘

Related Posts 

Methods in Context Mark Scheme (Pared-Down)

 

Social Surveys – Strengths and Limitations

Social Surveys are a quantitative, positivist research method consisting of structured questionnaires and interviews. This post considers the theoretical, practical and ethical advantages and disadvantages of using social surveys in social research. 

The strengths and limitations below are mainly based around surveys administered as self-completion questionnaires.

Social Surveys.png

Theoretical Factors

slide showing the theoretical strengths and limitations of social surveys

Theoretical strengths of social surveys

Detachment, Objectivity and Validity

Positivists favour questionnaires because they are a detached and objective (unbiased) method, where the sociologist’s personal involvement with respondents is kept to a minimum.

Hypothesis Testing

Questionnaires are particularly useful for testing hypotheses about cause and effect relationships between different variables, because the fact that they are quantifiable allows us to find correlations.

For example, based on government statistics on educational achievement we know that white boys on Free School Meals achieve at a significantly lower level than Chinese girls on Free School Meals. We reasonably hypothesise that this is because differences in parental attitudes – Chinese parents may value education more highly, and they may be stricter with their children when it comes to homework compared to white parents. Good questionnaire design and appropriate sampling would enable us to test out this hypothesis. Good sampling would further allow us to see if those white working class boys who do well have parents with similar attitudes to those Chinese girls who do well.

Representativeness

Questionnaires allow the researcher to collect information from a large number of people, so the results should be more representative of the wider population than with more qualitative methods. However, this all depends on appropriate sampling techniques being used and the researchers having knowledge of how actually completes the questionnaire.

Reliability

Questionnaires are generally seen as one of the more reliable methods of data collection – if repeated by another researcher, then they should give similar results. There are two main reasons for this:

When the research is repeated, it is easy to use the exact same questionnaire meaning the respondents are asked the exact same questions in the same order and they have the same choice of answers.

With self-completion questions, especially those sent by post, there is no researcher present to influence the results.

The reliability of questionnaires means that if we do find differences in answers, then we can be reasonably certain that this is because the opinions of the respondents have changed over time. For this reason, questionnaires are a good method for conducting longitudinal research where change over time is measured.

Social surveys are a good method for measuring change over time.

Theoretical Limitations

Issues affecting validity – Interpretivists make a number of criticisms of questionnaires.

The Imposition Problem

The imposition problem is when the researcher chooses the questions, they are deciding what is important rather than the respondent, and with closed ended questions the respondent has to fit their answers into what’s on offer. The result is that the respondent may not be able to express themselves in the way that want to. The structure of the questionnaire thus distorts the respondents’ meanings and undermines the validity of the data.

Misinterpetation of questions

Interpretivists argue that the detached nature of questionnaires and the lack of close contact between researcher and respondent means that there is no way to guarantee that the respondents are interpreting the questions in the same way as the researcher. This is especially true where very complex topics are involved – If I tick ‘yes’ that I am Christian’ – this could mean a range of things – from my being baptised but not practising or really believing to being a devout Fundamentalist. For this reason Interpretivists typically prefer qualitative methods where researchers are present to clarify meanings and probe deeper.

Researchers may not be present to check whether respondents are giving socially desirable answers, or simply lying, or even to check who is actually completing the questionnaire. At least with interviews researchers are present to check up on these problems (by observing body language or probing further for example).

Issues affecting representativeness

Postal questionnaires in particular can suffer from a low response rate. For example, Shere Hite’s (1991) study of ‘love, passion, and emotional violence’ in the America sent out 100, 000 questionnaires but only 4.5% of them were returned.

All self-completion questionnaires also suffer from the problem of a self-selecting sample which makes the research unrepresentative – certain types of people are more likely to complete questionnaires – literate people for example, people with plenty of time, or people who get a positive sense of self-esteem when completing questionnaires.

Practical Factors

Slide showing the practical strengths and limitations of social surveys.

Practical Strengths of Social Surveys

Questionnaires are a quick and cheap means of gathering large amounts of data from large numbers of people, even if they are widely dispersed geographically if the questionnaire is sent by post or conducted online. It is difficult to see how any other research method could provide 10s of millions of responses as is the case with the UK national census.

In the context of education, Connor and Dewson (2001) posted nearly 4000 questionnaires to students at 14 higher education institutions in their study of the factors which influenced working class decisions to attend university.

With self-completion questionnaires there is no need to recruit and train interviewers, which reduces cost.

The data is quick to analyse once it has been collected. With online questionnaires, pre-coded questions can be updated live.

Practical Limitations

The fact that questionnaires need to be brief means you can only ever get relatively superficial data from them, thus for many topics, they will need to be combined with more qualitative methods to achieve more insight.

Although questionnaires are a relatively cheap form of gathering data, it might be necessary to offer incentives for people to return them.

Structured Interviews are also considerably more expensive than self-completion questionnaires.

Ethical Factors

slide showing the ethical strengths and limtiations of social surveys

Ethical strengths of surveys

When a respondent is presented with a questionnaire, it is fairly obvious that research is taken place, so informed consent isn’t normally an issue as long as researchers are honest about the purpose of the research.

It is also a relatively unobtrusive method, given the detachment of the researcher, and it is quite an easy matter for respondents to just ignore questionnaires if they don’t want to complete them.

Ethical Limitations

They are best avoided when researching sensitive topics.

Related Posts 

An Introduction to Social Surveys – Definition and Basic Types of Survey

Positivism, Sociology and Social Research – Positivists like the survey method.

Please click here for more posts on research methods.