What are the strengths and limitations of using laboratory and field experiments to research education?
As a student you probably would have been the subject of an experiment within your school at some point during your 13 years of formal education.
Experiments conducted by schools themselves are a lot more common than educational researchers conducting their own field experiments within schools, and so such experiments are a rich source of data for students studying Methods in Context for A-Level sociology.
It is quite usual for schools to conduct small scale field experiments to try out new teaching techniques or to evaluate the effectiveness of banding of streaming.
If there are several classes of students of similar abilities doing the same subject, it is relatively easy to keep some classes being taught in the same way as usual but to change one aspect of teaching of other classes, and to measure the effect this has on student behaviour or learning.
Some years ago we conducted the following experiment in my institution, designed to measure the effectiveness of splitting students into ability bands:
In five subjects we deliberately created one class of all higher ability students (those predicted to get Bs or above at A-level based on their GCSE grades), but kept some high ability students in all of the regular mixed ability classes.
So in A-level sociology we had 7 classes at AS, and we ended up with one ‘high ability class’ and then 6 mixed ability classes, each with 2-3 A/B students rather than 3/5 A/B students.
The hypothesis of the Senior Management was that grouping all the higher ability students into one group would lead to improved results.
The control students are the high ability students still in the regular mixed classes.
We let this run for a year and compared the AS exam results at the end (it was a long time ago, when we still had AS exams.
The results in the end were inconclusive when we looked at the results of the ‘top band’ classes across all subjects – there was no signficant evidence that putting them in once class led to them getting better results.
As far as I am aware the students involved to this day have no idea they were subject to this experiment!
This kind of experiment within education is probably more common than you think!
A lot of schools put students into ability bands, and it makes sense that they review their results from time to time and ‘experiment’ to see if mixing up the bands give better results – for example, if you’ve got 6 maths groups in one school year, you could either have 6 discrete bands, or 2 more general bands, so you end up with a wider range of abilities split across 3 classes.
It kind of makes sense for schools to play around with how the split groups up to see if they can improve behaviour our outcome.
If you think about all of the things that schools can do differently, there are a lot of potential variables schools can change in one class, say, but not in others, just to see if changing that one variable makes a difference after a year in one class, rather than risking rolling out a change across the whole year group.
Variables you might change (all possible in-school experiments):
The gender mix of classes.
Seating lay-outs within classes
The length of lessons/ number of lessons in a week
The timing of ‘support lessons’ – before or after school, weekends, holidays.
How support staff are used in classrooms
Wider school policies on uniform, discipline and punishment
The teaching techniques used in lessons.
The ratio of face to face and online learning.
The practical, theoretical and ethical strengths of schools conducting experiments
Don’t kid yourselves, this kind of micro-experiment goes on all the time, but there some good reasons:
Ethically teachers and schools have to provide students with the best education they can – educational theory about what the most effective teaching techniques changes, technology changes, so teachers and schools have to adapt. Doing an experiment for a year with one class can be a useful way of finding out how to implement changes more effectively across the whole school the following year. Or if some experiments don’t work out, at least it’s not all students who suffer.
Practically, the students are there, the school is there, it’s relatively easy for teachers and schools to do experiments, rather than having them done externally.
Theoretically – validity should be very high because one typically doesn’t inform students they are part of an experiment. Reliability should also be good because the conditions are relatively stable over time in most schools.
The practical, theoretical and ethical weaknesses of schools conducting experiments
There is the ethical problem of deception and some students getting treated differently for the period of the experiment, which goes against equal opportunities.
Teaching one class differently to the rest can be stressful and demanding for the teacher.
In very small classes and schools with few classes, it’s hard to get a large enough sample for good representativeness.
Related Posts
For a great example of a really extreme experiment in a school, see ‘Chinese School‘.
For some of the general strengths and limitations of field experiments please see this post.
An autobiography is an account of the life of an individual, written by that individual, sometimes with the assistance of a professional biographer.
One of the most popular UK autobiographies of 2020 was Harry and Meghan’s ‘Finding Freedom’, and it is supposed to ‘dispel rumors about their relationship from both sides of the pond’.
The Amazon critics, however, disagree. The comments ranked at 2 and 3 (accessed 18 August 2020) in order of usefulness both give the book 1 star out of five and comment thus:
“… the reader can only assume a good proportion of [this book is] made up… the reader is left with a very poor impression of the couple. As someone else said – this is very much an ‘own goal’.”
600 people found this helpful
hellsbells123 – 1.0 out of 5 stars Dross of the highest order – all time low for Harry
“Dreadful book full of ridiculous unnecessary detail from a couple who profess to want privacy. This is a book masquerading as a love story but full of bile, hatred and bitterness. “
The strengths and limitations of autobiographies as a source of data
Whether they have a readership of millions or tens, autobiographies are selective in the information they provide about the life of the author.
They thus tell you what the author wants you to know about themselves and their life history.
However, you have no way of knowing whether the events outlined in an autobiography are actually and I wouldn’t even trust an autobiography to give me an accurate view of the authors’ own interpretation of what the most significant events in their life history were.
The author may exaggerate certain events, either because they mis-remember them, or because they want their book to sell, thus they are selecting what they think their audience will want to read.
In some cases, events may even be fabricated altogether.
As a rule, I’d say that the more famous someone is, then the less valid its contents are. An exception to this would be less famous ‘positive thinking’ lifestyle gurus, whose income maybe depends more on their book sales than really famous people, who could possibly afford to be honest in the biographies!
Either way, there are so many reasons why an autobiography might lack validity, I wouldn’t trust the content of any of them – think about it, how honest would you be in your autobiography, if you knew anyone could read it?
Using autobiography sales data may be more useful…
IMO the value of autobiographies lies in telling us what people want to hear, not necessarily in getting to the truth of people’s personal lives.
If want to know what people want to hear, look a the sales volumes – there are really no surprises…..
It is possible to analyse qualitative social media data to reveal social trends in attitudes.
Twitter recently released an analysis of the content of 4 billion tweets made over the past three years, from users based in the United States. (Source)
The fastest growing theme which Twitter users are talking about is ‘creator culture’, with people tweeting about products they create in order to sell to make a living…
They claim that the content of tweets reveal that the U.S. population has become increasingly interested in six major cultural themes over the last 4 years (from 2016 to 2020):
Tweets about ‘Creator Culture’ are up 462% – which includes tweets about creative currency, ‘hustle life’ and connecting through video.
One Planet tweets are up – 285% – includes tweets on the themes of the ethical self, sustainability, and clean corporations
Tweets about Well Being are up 225% – tweets about digital monitoring, holistic health and being well together
Tech Life tweets are up – 166% – on the topics of blended realities, future tech and ‘tech angst’.
‘My Identity’ tweets are up – 167% – fandom, gender redefined and ‘representing me’ are the main themes here.
Tweets about ‘Everyday Wonders’ are up 161% – a theme which includes DIY spirituality, awe of nature and cosmic fascination.
Sustainability is another large twitter conversation growth area.
The 2020 report by twitter (here) was produced for marketing purposes, but nonetheless reveals what twitter users are becoming increasingly interested in, and there are no real surprises here.
The report is broken down into several sections which include the nice infographics I’ve put up in this post, there are many more available in the reports.
Intuitively I’m not surprised to see any of the above trends emerging from this analysis – I’m sure that as a population as a whole, we are generally more interested in all of the above in 2020, compared to 2016.
The limitations of using Twitter data to reveal cultural trends
There may be a lot of data, but there are possible problems with representativeness – twitter users tend to be younger and more educated than the wider population. (Source).
There’s also a problem with the motivations behind the data being collected – this was done for marketing purposes, to be useful to companies wishing to advertise on Twitter – so this analysis wouldn’t show any more negative trends which may have been tweeted about.
A limitation of the way this data is published is that we’re not told the raw numbers – so we know how much more a particular trend is being tweeted about in percentages, but we don’t know about the actual numbers. Some of these may have started from a very low base in 2016, in which case a 250% increase in 4 years still wouldn’t be that signficant!
This analysis paints Twitter as a wholly positive place where people are full of wonder and fascination, and are creative and positive. In reality we all know there’s a darker side to Twitter!
Students really should be considering how valid, reliable and representative twitter data is in terms of what it can tell us about broader cultural, political and economic values.
You may well decide that it’s NOT a valid data source at all, but that’s fine as Twitter gives you something to be critical of, and being critical is all part of A-level sociology!
Personal documents are those which are intended only to be viewed by oneself or intimate relations, namely friends or family. They generally (but not always) not intended to be seen by a wider public audience.
For the purposes of A-level sociology, the two main types of personal document are diaries and personal letters.
Today, I’m inclined to include personal ‘emails’ and certain intimate chat groups – such as circles of close friends chatting on WhatsApp, in this definition, because the data produced here will reveal personal thoughts and feelings, and isn’t intended for wider public consumption.
I think we can also include some personal blogs and vlogs in this definition, as some of these do reveal personal thoughts and feelings, even if they are written to be viewed by the general public – people sharing aspects of their daily lives on YouTube, or people writing more focused blogs about the travel experiences or how they are coping with critical illnesses, all have something of the ‘personal’ about them.
We could also include ‘naughty photos’ intended only to be shared with an intimate partner, but I think I’ll leave an analysis of those kind of documents out of this particular post!
Just a quick not on definitions – you need to be careful with the distinction I think between personal and private documents.
Personal documents = anything written which reveals one’s personal thoughts and feelings. These can either be written for consumption by oneself, by close others, or sometimes for public consumption.
Private documents – these are simply not intended to be viewed by a wider public audience, and can include someone’s personal diary or intimate letters/ photos between two people, but company accounts and strategy can also count as private documents, even if shared by several dozens of people, if not intended for consumption by a wider audience.
As with all definitions, just be clear what you’re talking about.
Certainly to be safe, for the sake of getting marks in an A-level sociology exam question on the topic, personal diaries and ‘intimate letters’ are certainly both types of personal document.
Examples of sociological research using Personal Documents
Thomas and Znaniecki, The Polish Peasant (1918/ 1921)
Ozana Cucu-Oancea argues that this remains the most significant work using personal documents in the history of the social sciences (source).
The study used a range of both personal and public documents, and the former included hundreds of letters between Polish immigrants and their families back home in Poland, as well as several personal diaries.
In all the work consisted of 2,200 pages in five volumes, so it’s pretty extensive, focussing on the cultural consequences of Polish migration.
The documents revealed touched on such themes as crime, prostitution, alcoholism; and the problem of social happiness in general.
What was significant about this study from a theoretical point of view is that it put the individual at the centre of social analysis and stood in contrast to Positivism which was popular at that time.
The limitations of using personal documents in social research
There is a problem of interpretation. The researchers might misinterpret the meaning of the documents. The less contextual information the researchers have, the more likely this is to happen.
Practically it takes a long time to sift through and organise the information.
Who cares? Let’s face it, are you really going to go and read a 2, 200 page work analysing letters from Polish Immigrants, written over 100 years ago?
This post aims to provide some examples to some of the more unusual and interesting experiments that students can explore and evaluate.
I’ve already done a post on ‘seven field experiments‘, that outline seven of the most interesting classic and contemporary experiments which are relevant to various topics within the A-level sociology syllabus, in this post I provide a much fuller list, and try to present some more unusual examples, focusing on contemporary examples with video examples where possible.
The Circle
Channel Four’s ‘The Circle’ is an experiment of sorts – contestants have to stay in one room and can only interact with each other by a bespoke, in-house social media application, competing for popularity. At the end of every day the two-three most popular people get to kick out someone from the least three popular people, then a newbie comes in to replace them.
The Twinstitute
This recent series which aired on BBC2 involves getting identical twins to do the same tasks under different circumstances – to see what the effect of ‘external stimuli’ (independent variables) are on factors such as ‘concentration’.
In one classic, and super easy to relate to example, sets of twins are asked to do a written IQ test – one half are allowed to keep their mobile phones on the table, another have to put them away – all other variables remain the same. The findings are predictable – the group with their phones out get worse scores.
Conclusion – mobile phones are distracting, quite a useful fact to remind students of!
Sleep deprivation makes people less likely to want to socialise with you!
A 2017 experiment measured how respondents perceived tired people. The findings were that respondents were less likely to want to socialise with sleep-deprived people.
25 Participants (aged 18-47) were photographed after normal sleep and again after two days of sleep deprivation.
The two photographs were then rated by 122 raters (aged 18-65), according to how much they would like to socialise with the participants. The raters also rated the photos based on attractiveness, health, sleepiness and trustworthiness.
The raters were less likely to want to socialise with the participants in the ‘sleep-deprived’ photos compared to the photos of them when they’d had normal sleep. They also perceived the ‘sleep-deprived’ versions as less attractive, less health and more sleepy.
There was no difference in the trustworthiness ratings.
You have to think about this to get to what the variables are:
The main dependent variable is the raters’ ‘desire to socialise’ with the people in the photos.
The independent variable is the ‘level of sleep-deprivation’ (measured by photos)
What I like about this experiment is the clear ‘control measure’ – the researchers used photos of the same participants – after regular sleep and sleep-deprivation.
Without that control measure, the experiment would probably fall apart1
Science Professors think female applicants are less competent
In this 2012 experiment researchers sent 127 science professors around the country (both male and female) the exact same application materials from a made-up undergraduate student applying for a lab manager position.
63 of the fake applications were made by a male, named John; the other 64 were made by a female, named Jennifer.
Every other element of the applications were identical.
The researchers also matched the two groups of professors to whom the applications were sent, in terms of age distribution, scientific fields, and tenure status.
The 127 professors were each asked to evaluate the application based on
their overall competency and hireability,
the salary they would offer to the student
the degree of mentoring they felt the student deserved.
The faculty were not told the purpose of the experiment, just that their feedback would be shared with the student.
The results
Both male and female professors consistently regarded the female student applicant as less competent and less hireable than the otherwise identical male student:
The average competency rating for the male applicant was 4.05, compared to 3.33 for the female applicant.
The average salary offered to the female was $26,507.94, while the male was offered $30,238.10.
The professor’s age and sex had insignificant effects on discrimination —old and young, male, and female alike tended to view the female applicants more negatively.
Blind auditions improve the chances of female musicians being recruited to orchestras
A comparative study by Cecilia Rouse, an associate professor in Princeton’s Woodrow Wilson School and Claudia Goldin, a professor of economics at Harvard University, seems to confirm the existence of sex-biased hiring by major symphony orchestras.
Traditionally, women have been underrepresented in American and European orchestras. Renowned conductors have asserted that female musicians have “smaller techniques,” are more temperamental and are simply unsuitable for orchestras, and some European orchestras do not hire women at all.
To overcome bias, most major U.S. orchestras implemented blind auditions in the 1970s to 1980s, in which musicians audition behind a screen that conceals their identities but does not alter sound. However, some kept non-blind auditions.
This provided the context for a nice ‘natural experiment’…
Using data from the audition records, the researchers found that:
– for both blind and non-blind auditions, about 28.6 percent of female musicians and 20.2 percent of male musicians advanced from the preliminary to the final round.
– When preliminary auditions were not blind, only 19.3 percent of the women advanced, along with 22.5 percent of the men.
The researchers calculated that blind auditions increased the probability that a woman would advance from preliminary rounds by 50 percent.
As a result, blind auditions have had a significant impact on the face of symphony orchestras. About 10 percent of orchestra members were female around 1970, compared to about 35 percent in the mid-1990s.
Rouse and Goldin attribute about 30 percent of this gain to the advent of blind auditions.
Their report was published in the September-November issue of the American Economic Review.
The Marshmallow Test
This classic 1971 experiment was designed to measure a child’s level of self-control, or will-power. In sociological terms, this is measuring a child’s ability to ‘defer gratification’.
Researchers put a child in a room with one Marshmallow. The child was informed that they could eat it whenever they wanted, but if they could wait until the researcher returned, they could have two Marshmallows.
The researcher then left and the child was left alone to deal with their temptation for approximately 15 minutes. In the end 2/3rds of children gave into temptation and ate the Marshmallow, the other third resisted.
The researchers then tracked the children through later life and found that those who had more will power/ self control (those who hadn’t eaten the treat) were more likely to do well at school, avoid obesity and generally had a better quality of life.
NB – it’s down to you to do your research on how replicable and valid this experiment is.
Here’s one of the original researchers in 2015 saying how they’ve evolved and replicated the experiment and he’s written a book on the importance of teaching self-control to enhance people’s quality of life:
On the other hand, this is a video which is critical, saying that future studies found that social economic background accounted for around half of life-success, with individual will-power only accounting for half.
(However, this second video appears to be one young guy with no academic credentials, other than the lame bookshelf he’s put in the background, hardly semiotics genius.)
I’ve just released some extensive revision workbooks and Power Points for sale as part of my sociology teaching resources subscription package, available for only £9.99 a month!
This teaching resource bundle contains work books and Power
Points covering the entire content of education and research methods of the
AQA’s A-level sociology specification.
The resources should be enough to cover at least 8-10 revision lessons on education and research methods.
Resources in February’s bundle include
One education workbook in Word – 65 pages
Two education Power Points – over 70 slides
One research methods workbook in Word – 60 pages
One Research Methods Power Point – 60 slides
Short answer questions PDF for education and research methods
Essay plans PDF for education, research methods and methods in context.
NB – These aren’t visual, I have had to remove most the pictures I use personally, for copyright reasons, but I’m sure you can find your own to fit in.
The work packs and Power Points contain various activities such as….
Paired concepts
Short answer practice questions
A-Z word matching tasks
10 Mark practice questions
Essay plans and short answer question PDFs
I’m also throwing in PDFs of my short answer practice questions and Essay plans for education research methods, which I normally sell as part of my revision bundles!
More resources to come…
I’m making resources available every month as part of this teacher resource subscription package. Please click the link to left for details of the schedule of what’s coming in future months!
The 2018 report shows that the overall rate of permanent exclusions was 0.1 per cent of pupil enrolments in 2016/17. The number of exclusions was 7,720.
The report also goes into more detail, for example….
The vast majority of exclusions were from secondary schools >85% of exclusions.
The three main reasons for permanent exclusions (not counting ‘other’) were
Persistent disruptive behaviour
Physical assault against a pupil
Physical assault against an adult.
Certain groups of students are far more likely to be permanently excluded:
Free School Meals (FSM) pupils had a permanent exclusion rate four times higher than non-FSM pupils
FSM pupils accounted for 40.0% of all permanent exclusions
The permanent exclusion rate for boys was over three times higher than that for girls
Over half of all permanent exclusions occur in national curriculum year 9 or above. A quarter of all permanent exclusions were for pupils aged 14
Black Caribbean pupils had a permanent exclusion rate nearly three times higher than the school population as a whole.
Pupils with identified special educational needs (SEN) accounted for around half of all permanent exclusions
The ‘reasons why’ and ‘types of pupil’ data probably hold no surprises, but NB there are quite a few limitations with the above data, and so these stats should be treated with caution!
Limitations of data on permanent exclusions
Validity problems…
According to this Guardian article, the figures do not take into account ‘informal exclusions’ or ‘off-rolling’ – where schools convince parents to withdraw their children without making a formal exclusion order – technically it’s then down to the parents to enrol their child at another institution or home-educate them, but in many cases this doesn’t happen.
According to research conducted by FFT Education Datalab up to 7, 700 students go missing from the school role between year 7 and year 11 when they are supposed to sit their GCSEs…. Equivalent to a 1.4% drop out rate across from first enrolment at secondary school to GCSEs.
Datalabs took their figures from the annual school census and the DfE’s national pupil database. The cohort’s numbers were traced from year seven, the first year of secondary school, up until taking their GCSEs in 2017.
The entire cohort enrolled in year 7 in state schools in England in 2013 was 550,000 children
However, by time of sitting GCSEs:
8,700 pupils were in alternative provision or pupil referral units,
nearly 2,500 had moved to special schools
22,000 had left the state sector (an increase from 20,000 in 2014) Of the 22,000,
3,000 had moved to mainstream private schools
Just under 4,000 were enrolled or sat their GCSEs at a variety of other education institutions.
60% of the remaining 15,000 children were likely to have moved away from England, in some case to other parts of the UK such as Wales (used emigration data by age and internal migration data to estimate that around)
Leaves between 6,000 to 7,700 former pupils unaccounted for, who appear not to have sat any GCSE or equivalent qualifications or been counted in school data.
Working out the percentages this means that by GCSEs, the following percentages of the original year 7 cohort had been ‘moved on’ to other schools.
6% or 32, 000 students in all, 10, 00 of which were moved to ‘state funded alternative provision, e.g. Pupil Referral Units.
4%, or 22K left the mainstream state sector altogether (presumably due to exclusion or ‘coerced withdrawal’ (i.e. off rolling), of which
4%, or 7, 700 cannot be found in any educational records!
This Guardian article provides a decent summary of the research.
Further limitations of data on school exclusions
There is very little detail on why pupils were excluded, other than the ‘main reason’ formally recorded by the head teacher in all school. There is no information at all about the specific act or the broader context. Labelling theorists might have something to say about this!
There is a significant time gap between recording and publication of the data. This data was published in summer 2018 and covers exclusions in the academic year 2016-2017. Given that you might be looking at this in 2019 (data is published annually) and that there is probably a ‘long history’ behind many exclusions (i.e. pupils probably get more than one second chance), this data refers to events that happened 2 or more years ago.
Relevance of this to A-level sociology
This is of obvious relevance to the education module… it might be something of a wake up call that 4% of students leave mainstream secondary education before making it to GCSEs, and than 1.4% seem to end up out of education and not sitting GCSEs!
An example of a full mark answer to a methods in context question from the AQA.
Below I provide an example full mark answer to a methods in context question taken from the AQA’s 2016 Specimen A-level sociology paper 7192 (1) and provide some running commentary on this model answer.
NB – I also outline why the AQA has (IMO) miss-marked this exemplar… I don’t think it should get full marks, because IT DOES NOT do what the mark scheme says it should do to get 20/20. However… it’s still a good answer…!
The question I’m focussing on is as follows:
About Methods in Context Questions:
Methods in Context questions will ask students to evaluate the strengths and limitations of any of the six main research methods for researching a particular topic within the sociology of education, applying material from the item.
Students often struggle with these questions and so it is useful to have exemplars which demonstrate how to answer them.
The Specific Question with Item:
Read Item C below and answer the question that follows.
Item CInvestigating the influence of the family on pupils’ education
Families have an important influence on pupils’ education. For example, the family’s income may be able to pay for educational materials and experiences as well as for comfortable conditions in which to study. Similarly, parents’ own education, their child-rearing and socialisation practices, and their speech codes and cultural background can influence children’s’ attitudes to school and their ability to succeed. In all these respects, there are significant class and ethnic differences in family life and they help to explain differences in the educational experiences of different pupils.
One way of studying the influence of the family on pupils’ education is to use structured interviews. These are a good way of gathering basic data quickly. Structured interviews also allow researchers to establish patterns and make comparisons. However, they may be less useful when dealing with sensitive or private issues.
Applying material from Item C and your knowledge of research methods, evaluate the strengths and limitations of using structured interviews to investigate the influence of the family on pupils’ education (20).
Mark Scheme (top band only: 17-20)
Answers in this band will show accurate, conceptually detailed knowledge and sound understanding of a range of relevant material on structured interviews.
Appropriate material will be applied accurately and with sensitivity to the investigation of the specific issue of the influence of the family on pupils’ education.
Students will apply knowledge of a range of relevant strengths and limitations of using structured interviews to research issues and characteristics relating to the influence of the family on pupils’ education. These may include some of the following and/or other relevant concerns, though answers do not need to include all of these, even for full marks:
the research characteristics of potential research subjects, eg individual pupils, parents, other relatives, teachers (eg class and ethnic differences among parents; teachers’ professionalism or attitudes towards pupils’ families)
the research contexts and settings, eg pupils’ homes, school premises, school gates
the sensitivity of researching influence of the family on pupils’ education, eg families’ material circumstances or child-rearing practices; eligibility for free school meals; stigmatisation; policy and resource implications for schools; parental consent).
Evaluation of the usefulness of structured interviews will be explicit and relevant. Analysis will show clear explanation. Appropriate conclusions will be drawn.
Indicative Content for the strengths and limitations of the method
Strengths and limitations of structured interviews, as applied to the particular issue in education, may include: time, cost, access, hypothesis-testing, quantitative data, factual data, correlation, reliability, sample size, representativeness, generalisability, inflexibility, superficiality, lack of validity, interviewer bias, social desirability effect, status differences, misunderstanding, ethical issues.
Student Answer
KT’s comments in bold and red beneath each paragraph…
Structured interviews are usually closed-ended interviews which produce reliable, quantitative data. They are relatively quick to carry out and require little training. If the school agrees to the research taking place the researcher would be able to get a large sample of pupils. However, these interviews, although preferred by positivists, are limiting because the questions are fixed. The quantitative nature of the interviews means they are ideal for examining cause and effect such as whether parent attending parents’ evening has an impact on the pupils’ education.
This is a good general introductory paragraph about structured interviews, but it’s really only a mark band level 3 response: because you could replace the phrases ‘school’ and ‘pupils’ with (for example) ‘hospitals’ and ‘patients’ and it would be saying the same thing. The same is true with the final sentence. You could say that about ‘eating 5 pieces of fruit a day’ has an impact on ‘patient recovery rates’.
This is a good example of a paragraph where the candidate may think they’ve said something at level 4 or 5, but really it’s down at level 3!
However, when asking parents about how they bring up their children there could be many problems. Most parents will not want to be thought of as bad parents who do not care about their child. These parents will want to show that they are supportive of their child. The formality of a structured interview will increase parents’ fear and this means that parents may give socially desirable answers, especially as they are face-to-face with the interviewer. They may see the interviewer as a teacher in disguise and this will further encourage choosing answers that may not reflect the true situation of their involvement in their child’s education.
This is a solid ‘mark band level 5’ paragraph – the method applied specifically to the topic under investigation.
Another problem with unstructured interviews is they are inflexible. Closed questions with limited responses will only give the options chosen by the researcher and so may miss vital aspects of home life that could have an impact on a child’s achievement such as temporary housing or domestic abuse. This is likely if the parents are working class and the sociologist is middle class and does not have experience of working class life or know the concerns or worries facing working-class families.
Not quite as solid as the first paragraph, but it does pick up on aspects of home life, so should be at least level 4.
Working-class parents may have lower levels of education and speak in restricted speech code. This means they may not understand a question or they may say something the sociologist does not understand. In a structured interview the sociologist cannot ask for clarification of what has been said. The same problem applies if the parent and the sociologist are of a different ethnic background, in this case there may also be a language barrier if the parent does not speak English or it is not their first language.
Seems like a solid level 5 paragraph again.
Many deprived pupils may have a sense of shame or stigma attached to them. Many do not claim free school meals for this reason and if they are asked about this they may not want to tell the truth. They may lie and they are more likely to lie when they do not feel relaxed or comfortable. This is much more likely in a structured interview as there is no chance to gain rapport. Since the interviewer is present there is an increased risk of social desirable answers. There may be an ethical issue of harm linked to the research due to the nature of the topic and the questions that the interviewer may ask about personal circumstances linked to the pupil’s home background.
The link to free school meals at the beginning should just about clarify this a level 5 response.
A problem with structured interviews with pupils is that most of them will be under 18. This means that they are unable to give their consent and this will cause some ethical concerns. Parents will be unlikely to give their consent because they will feel a sense of shame or they just may not want their child to be part of the research which asked them to give personal details about the parent-child relationship.
This should classify as a standard ‘level 4 response’, about pupils in general.
Structured interviews could be used with teachers to assess their views of the impact of home background. Teachers would be more likely to take part in a structured interview as they are less time consuming. As the questions would be related to children’s home backgrounds teachers may not be able to answer all the questions if they did not have all the details of a pupil’s home situation. Teachers may also give answers that suggest that achievement is linked to factors at home rather than in the school as this takes some of the pressure away from their responsibility.
A clear level 5 response… teachers not knowing about home background… one of the clearest level 5 responses in the whole essay.
Examiner commentary
The answer shows a wide range of application. Many of the points are linked explicitly to the issue of the influence of the family on pupils’ education. The answer covers a range of characteristics of research subjects; parents, pupils and teachers. There is some consideration of the school as a research setting. There are a number of points that consider the sensitivity of researching this subject and the problematic nature of the presence of the interviewer when carrying out this research.
20/20 marks awarded
KT’s commentary…
This is a solid answer, HOWEVER… I don’t see how it can get 20/20 because IT DOES NOT DRAW APPROPRIATE CONCLUSIONS. Hence as far as I can see the AQA should have awarded it a maximum of 16/20.
Theory and Methods A Level Sociology Revision Bundle
If you like this sort of thing, then you might like my Theory and Methods Revision Bundle – specifically designed to get students through the theory and methods sections of A level sociology papers 1 and 3.
Contents include:
74 pages of revision notes
15 mind maps on various topics within theory and methods
Five theory and methods essays
‘How to write methods in context essays’.
Signposting
Methods in Context questions will ask students to evaluate the strengths and limitations of any of the six main research methods for researching a particular topic within the sociology of education, applying material from the item.
‘Methods in Context’ questions appear on A Level Sociology Paper 1 (Education with Theory and Methods) and AS Sociology Paper 1 (Education with Methods in Context).
Methods in Context questions will ask students to evaluate the strengths and limitations of any of the six main research methods for researching a particular topic within the sociology of education, applying material from the item.
Students often struggle with these questions and so it is useful to have exemplars which demonstrate how to answer them. Thankfully the AQA has recently released some of these, with examiner commentary, and below I’ve reproduced a top band 18/20 answer to one particular methods in context question!
NB – I’ve take this directly from the AQA’s feedback to the 2017 AS sociology exam series (specific source below), but I’ve repositioned the comments on each paragraph to make them more accessible (at the end of each paragraph, rather than at the end of the whole essay.
The specific question below appeared on the June 2017 AS Sociology Paper 1 – the whole paper is now publically available from the AQA’s web site.
Read Item B below and answer the question that follows.
ITEM B
On average, working-class pupils underachieve in education compared with those from middle-class backgrounds. Some sociologists believe that material deprivation is one factor that causes working-class underachievement.
Other sociologists argue that values and attitudes in working-class homes may cause underachievement. School factors may also affect achievement. Sociologists may use written questionnaires to study working-class educational underachievement.
Using written questionnaires enables the researcher to reach a large number of pupils, parents and teachers. Also, those who complete the questionnaire can usually remain anonymous. However, not all those who receive a questionnaire will complete it.
Applying material from Item B and your knowledge of research methods, evaluate the strengths and limitations of using written questionnaires to investigate working-class educational underachievement.
The Mark Scheme (Top Band Only: 17-20)
Answers in this band will show accurate, conceptually detailed knowledge and good understanding of a range of relevant material on written questionnaires.
Appropriate material will be applied accurately to the investigation of the specific issue of working-class educational underachievement.
Students will apply knowledge of a range of relevant strengths and limitations of using written questionnaires to research issues and characteristics relating to working-class educational underachievement. These may include some of the following and/or other relevant concerns, though answers do not need to include all of these, even for full marks:
the research characteristics of potential research subjects, eg pupils, teachers, parents, (self-esteem; literacy skills; attitude to school)
the research contexts and settings (eg school; classroom; home environment).
the sensitivity of researching working-class underachievement (eg schools’ market position; negative publicity; vulnerability of participants; parental consent; teacher reluctance).
Evaluation of the usefulness of written questionnaires will be explicit and relevant. Analysis will show clear explanation and may draw appropriate conclusions
Student Answer – Awarded 18/20 (AS standard!)
Picture version:
Page 1
Page 2
Text Version:
Paragraphs as in actual student response, numbers added for clarity.
Examiner comments appear in red after each paragraph.
ONE – Written questionnaires are a type of survey where questions are standardised and distributed to large numbers of people. This is useful in an educational setting because it means they can be given to numerous students in numerous schools, something which is very important when investigating working class pupils as there are many regions which are predominantly working class.
First paragraph – general advantages of written questionnaires – standardised and large distribution. Attempt to link to topic
TWO – One major advantage of using questionnaires is that they pose relatively few practical issues. They are fairly cheap to create and distribute and they quick to fill out, especially if all questions are closed ended. This means that access is not usually an issue for the researcher as they will not disrupt lessons as much as other methods such as structured interviews, meaning that the researcher is more likely to received permission from the gatekeeper. Furhtermore, working class pupils are more likely to need to take on paid work and so the quick-nature of questinnaires which are not very time consuming means that they are useful for investigating working class underachievement.
Para 2 – advantage of Wc related to context of research in schools (gatekeepers).
THREE – However, when investigating working class pupils there may be the issue of cultural deprivation, particularly language issues. Berciler and Englemann argue that the language spoken by the working class is deficient, a particular issue when trying to interpret the questions on a written question questionnaire. When coupled with the fact that questionnaires are written in the elaborated code but working class pupils (and parents) tend to speak in the restricted code this can be a major problem in gaining accurate results; unlike with other methods, questions cannot be clarified
Para 3 – good link to topic and WQ re language and speech codes.
FOUR – As well as posing few practical issues, written questionnaires do not pose many ethical issues. This is because the respondent can remain anonymous if they so wish and they can also leave any intrusive or sensitive issues blank. When studying working class underachievement this is a particular advantage because some pupils may be embarrassed to discuss their home lives, particularly if they live in poverty.
Para 4 – ethical issues discussed – anonymity developed with reference to topic
FIVE –Even though there are relatively few ethical uses, the researcher must be aware of harm to respondents. For working class children there may be a stigma attached, and for sensitive issues such as home life, the use of questionnaires can still cause distress. Nevertheless, the fact that respondents are not obligated to respond means this ethical problem is easily overcome.
Para 5 – further developed with reference to topic
SIX – From the perspective of a positivist, written questionnaires are a useful way to investigate working class underachievement because the data produced when using standardised questions is quantitative and high in reliability. This makes questionnaires useful for investigating working class underachievement because it allows cause and effect relationships to be established, for example whether or the not the structure of the education system reproduces working class underachievement, or whether there is a correlation between family background and achievement. However, the nature of written questionnaires can be an issue if the researcher’s meaning is imposed onto the questionnaire so it is another fact that must be taken into account
Para 6 – various positivist concepts – good on usefulness of WC – but not unique to topic
SEVEN – From the point of view of an interpretivist, written questionnaires are not useful when investigating working class underachievement because the data lacks validity. While questionnaires may be able to identify that factors such as material deprivation may influence the achievement of working class pupils, it does not get to the heart of the matter. Written questionnaires do not investigate the meanings that pupils may attach to the reasons they may underachieve, and do not let the respondent communicate their ideas freely. Because of this lack of validity interpretivists do not favour the use of written questionnaires to investigate working class underachievement.
Para 7 – interpretivism and validity – not related to topic specifically (generic)
EIGHT – Ultimately, written questionnaires can be useful to investigate working class underachievement because the data is easy to analyse and compare, which may be useful as the data could be used over time to look at whether government policies put in place to reduce working class underachievement really work. Not only that but they are representative, so generalisations about the wider population can be made in a way that methods favoured by interpretivists cannot.
Para 8 – attempt to relate strengths of WQs to topic
Overall COMMENT – very strong on method with some (2/3) clear links to topic
Theory and Methods A Level Sociology Revision Bundle
If you like this sort of thing, then you might like my Theory and Methods Revision Bundle – specifically designed to get students through the theory and methods sections of A level sociology papers 1 and 3.
Contents include:
74 pages of revision notes
15 mind maps on various topics within theory and methods
Five theory and methods essays
‘How to write methods in context essays’.
Signposting
Methods in Context Questions (possibly better know as ‘applied methods questions’ will appear on the Education with Theory and Methods Paper in the A-level sociology exams.
AQA A-level sociology Papers 1 and 3 will both contain an ‘outline and explain’ 10 mark (no item) question on sociological theories, and/ or methods.
One possible format for this question is what I like to think of as the ‘pure research methods’ format (‘classic’ might be a better word than ‘pure’) in which students are asked to outline and explain two theoretical, practical or ethical advantages or problems of using one of the main research methods.
For example (taken from the AQA’s June 2017 Education with Theory and Methods paper): ‘Outline and explain two problems of using documents in social research’
There are actually 36 possible variations of such ‘pure’ or ‘classic’ research methods questions, as outlined in the flow chart below.
Students may be asked to give two advantages or problems of any of the above methods, or more specific methods (field experiments for example), or they may be asked to give two advantages of using overt compared to covert participant observation, or asked to simply give two ethical problems which you may encounter when doing research more generally.
Then of course, students may be asked to relate methods to theories, or just asked about a pure ‘theoretical’/ perspectives question.
While there is no guarantee that this particular format of question will actually come up on either paper 1 or 3, it’s still good practice for students to work through a number of such questions as revision practice.
