Category: research methods

Zimbardo’s Prison Experiment

In this famously notorious experiment college students volunteered to take on the role of either prison guards or prisoners and spend time in an artificial prison. The Stanford Prison Experiment was meant to last 14 days, it had to be stopped after just six because the ‘guards’ became abusive and the ‘prisoners’ began to show signs of extreme stress and anxiety.

In 1971, psychologist Philip Zimbardo and his colleagues set out to create an experiment that looked at the impact of becoming a prisoner or prison guard. The researchers set up a mock prison in the basement of Standford University’s psychology building, and then selected 24 undergraduate students to play the roles of both prisoners and guards.

The simulated prison included three six by nine foot prison cells. Each cell held three prisoners and included three cots. Other rooms across from the cells were utilized for the prison guards and warden. One very small space was designated as the solitary confinement room, and yet another small room served as the prison yard.

The 24 volunteers were then randomly assigned to either the prisoner group or the guard group. Prisoners were to remain in the mock prison 24-hours a day for the duration of the study. Guards, on the other hand, were assigned to work in three-man teams for eight-hour shifts. After each shift, guards were allowed to return to their homes until their next shift. Researchers were able to observe the behavior of the prisoners and guards using hidden cameras and microphones.

While the prisoners and guards were allowed to interact in any way they wanted, the interactions were generally hostile or even dehumanizing. The guards began to behave in ways that were aggressive and abusive toward the prisoners, while the prisoners became passive and depressed. Five of the prisoners began to experience such severe negative emotions, including crying and acute anxiety, that they had to be released from the study early.

The Stanford Prison Experiment demonstrates the powerful role that the situation can play in human behaviour. Because the guards were placed in a position of power, they began to behave in ways they would not normally act in their everyday lives or in other situations. The prisoners, placed in a situation where they had no real control, became passive and depressed.

Criticisms of Quantitative Research

Bryman (2016) identifies four criticisms of quantitative research:

Quantitative researchers fail to distinguish people and social institutions from the world of nature

Schutz (1962) is the main critique here.

Schutz and other phenomenologists accuse quantitative social researchers of treating the social world as if it were no different from the natural world. In so doing, quantitative researchers tend to ignore the fact that people interpret the world around them, whereas this capacity for self-reflection cannot be found among the objects of the natural sciences.

The measurement process possesses an artificial and spurious sense of precision and accuracy

Cicourel (1964) is the main critique here.

He argues that the connection between the measures developed by social scientists and the concepts they are supposed to be revealing is assumed rather than real – basically measures and concepts are both effectively ‘made up’ by the researchers, rather than being ‘out there’ in reality.

A further problem is that quantitative researchers assume that everyone who answers a survey interprets the questions in the same way – in reality, this simply may not be the case.

The reliance on instruments and procedures hinders the connection between research and everyday life

This issue relates to the question of ecological validity.  

Many methods of quantitative research rely heavily on administering research instruments to participants (such as structured interviews or self-completion questionnaires), or controlling situations to determine effects.

However, these instruments simply do not ‘tap into’ people’s real life experiences – for example, many of the well known lab experiments on the A-level sociology syllabus clearly do not reflect real life, while surveys which ask people about their attitudes towards immigration, or the environment, do not necessarily tell us about how people act towards migrants or the environment on a day to day basis.

The analysis of relationships between variables creates a static view of social life that is independent of people’s lives. 

The main critique here is Blumer (1956).

Blumer (1956) argued that studies that seek to bring out the relationships between variables omit ‘the process of interpretation or definition that goes on in human groups’.

This is a combination of criticisms 1 and 3 above, but adds on an additional problem – that in isolating out variables, quantitative research creates an artificial, fixed and frozen social (un)reality – whereas social reality is (really) alive and constantly being created through processes of interaction by its various members.

In other words, the criticism here is that quantitative research is seen as carrying an objective ontology that reifies the social world.

The above criticisms have lead intepretivists to prefer more qualitative research methods. However, these too have their limitations!

Sources:

Bryman (2016) Social Research Methods

 

The Four Main Concerns of Quantitative Research

Quantitative researchers generally have four main preoccupations: they want their research to be measurable, to focus on causation, to be generalisable, and to be replicable.

These preoccupations reflect epistemological grounded beliefs about what constitutes acceptable knowledge, and can be contrasted with the preoccupations of researchers who prefer a qualitative approach.

Measurement 

It may sound like it’s stating the obvious – but quantitative researchers are primarily interested in collecting numerical data, which means they are essentially concerned with counting social phenomena, which will often require concepts to be operationalised.

Causality 

In most quantitative research there is a strong concern with explanation: qualitative researchers are more concerned with explaining why things are as they are, rather than merely describing them (which tends to be the focus of more qualitative research).

It follows that it is crucial for quantitative researchers to effectively isolate variables in order to establish causal relationships.

Generalisation 

Quantitative researchers tend to want their findings to be representative of wider populations, rather than the just the sample involved in the study, thus there is a concern with making sure appropriate sampling techniques will be used.

Replication

If a study is repeatable then it is possible to check that the original researchers’ own personal biases or characteristics have not influenced the findings: in other words, replication is necessary to test the objectivity of an original piece of research.

Quantitative researchers tend to be keen on making sure studies are repeatable, although most studies are never repeated because there is a lack of status attached to doing so.

Source:

Bryman (2017) Social Research Methods

 

A few thoughts on revising research methods in context/ applied research methods

The ‘applied methods*’ question appears in paper 1 of the AQA’s Education with Theory and Methods exam (paper 7192/1). This is out of 20 marks, and students are expected to apply their understanding of any of the six main research method covered in the A-level sociology specification to any conceivable topic within education.

An example of an ‘applied methods*’ question is as follows:

‘Applying material from item B and elsewhere, evaluate the strengths and limitation of using participant observation to investigate truancy from school’ (20)

Here’s how I revise these questions with my students… NB I don’t introduce the item until later…

Warm up with the method

Firstly, I get students to talk through the theoretical practical and ethical strengths and limitations just of the method. I do this because students need to know they method anyway, and they can get 10/20 just for writing a decent methods essay (without applying it) – see the mark scheme here.

Methods in Context

Warm up with the method generally applied to the topic

Students brainstorm the general ethical, practical and theoretical issues you may encounter when researching this topic with this method… I think it’s good to be as open-minded as possible early on… It’s easiest just to get them to do this on paper. 

Sociology applied methods

Do a plan applying the method to the specific details in the item

I use an A3 sheet for this, with the item and question in the middle, students now read the item. 

Methods in Context

Write a detailed flow-chart

Here I get students to add in analysis and evaluation points to each original lead-point, showing a chain of reasoning (side 2 of A3 sheet).

Applied Research Methods

Repeat stage two with a different topic, to emphasise the difference in answers for the same method applied to a different topic

DO NOT go over the whole process again, once is enough!

Research Methods

Issues with Revising Applied Research Methods 

There’s a very real possibility that students will just not ‘get it’, because they have to be so nit-pickingly overt about relating the method to the specific topic. Drilling this into students is a painful and thankless task, induced solely by the demands of this specific form of the assessment.

There is also the possibility that students may lose the will to live, especially when some past papers have examples that even I find intolerably dull, and I’m actually interested in this stuff!

*These are sometimes referred to as ‘Methods in Context’ questions. This was the term originally used by the AQA for many years, but (much like this question format itself as a means of assessing application skills) it’s pretty clumsy, so the new ‘applied methods’ phrase is IMO much better.  

What is a Likert Scale?

A Likert* scale is a multiple-indicator or multiple-item measure of a set of attitudes relating to a particular area. The goal of a Likert scale is to measure intensity of feelings about the area in question.

A Likert scale about Likert scales!

In its most common format, the Likert scale consists of a statement (e.g. ‘I love Likert scales’) and then a range of ‘strength of feeling’ options which respondents choose from – in the above example, there are five such options ranging from strongly agree to strongly disagree.

Each respondents reply on each item is scored, typically with a high score (5 in the above example) being given for positive feelings and a low score (1 in the above example) for negative feelings.

Once all respondents have completed the questionnaire, the scores from all responses are aggregated to give an overall score, or ‘strength of feeling’ about the issue being measured.

Some examples of sociological research using Likert scales:

The World Values Survey is my favourite example – they use a simple four point scale to measure happiness. The poll below gives you the exact wording used in the survey…

The results on the web site (and below) show you the percentages who answer in each category, but I believe that the researchers also give scores to each response (4 to 1) and then do the same for similar questions, combine the scores and eventually come up with a happiness rating for a country out of 10. I think the USA scores around 7.2 or something like that, it might be more! Look it up if you’re interested….

America’s happiness results

Important points to remember about Likert scales

  • The items must be statements, not questions.
  • The items must all relate to the same object being measured (e.g. happiness, strength of religious belief)
  • The items that make up the scale should be interrelated so as to ensure internal reliability is strong.

*The Likert Scale is named after Rensis Likert, who developed the method.

Sources

Adapted from Bryman’s Social Research Methods

 

Validity in Social Research

Validity refers to the extent to which an indicator (or set of indicators) really measure the concept under investigation. This post outlines five ways in which sociologists and psychologists might determine how valid their indicators are: face validity, concurrent validity, convergent validity, construct validity, and predictive validity. 

Validity refers to the extent to which an indicator (or set of indicators) really measure the concept under investigation. This post outlines five ways in which sociologists and psychologists might determine how valid their indicators are: face validity, concurrent validity, convergent validity, construct validity, and predictive validity.

As with many things in sociology, it makes sense to start with an example to illustrate the general meaning of the concept of validity:

When universities question whether or not BTECs really provide a measure of academic intelligence, they are questioning the validity of BTECs to accurately measure the concept of ‘academic intelligence’.

When academics question the validity of BTECs in this way, they might be suspicious that that BTECs are actually measuring something other than a student’s academic intelligence; rather BTECs might instead actually be measuring a student’s ability to cut and paste and modify just enough to avoid being caught out by plagiarism software.

If this is the case, then we can say that BTECs are not a valid measurement of a student’s academic intelligence.

How can sociologists assess the validity of measures and indicators?

what is validity.png

There are number of ways testing measurement validity in social research:

  • Face validity – on the face of it, does the measure fit the concept? Face validity is simply achieved by asking others with experience in the field whether they think the measure seems to be measuring the concept. This is essentially an intuitive process.
  • Concurrent validity – to establish the concurrent validity of a measure, the researchers simply compare the results of one measure to another which is known to be valid (known as a ‘criterion measure). For example with gamblers, betting accounts give us a valid indication of how much they actually win or lose, but wording of questions designed to measure ‘how much they win or lose in a given period’ can yield vastly different results. Some questions provide results which are closer to the hard-financial statistics, and these can be said to have the highest degree of concurrent validity.
  • Predictive validity – here a researcher uses a future criterion measure to assess the validity of existing measures. For example we might assess the validity of BTECs as measurement of academic intelligence by looking at how well BTEC students do at university compared to A-level students with equivalent grades.
  • Construct validity – here the researcher is encouraged to deduce hypotheses from a theory that is relevant to the concept. However, there are problems with this approach as the theory and the process of deduction might be misguided!
  • Convergent validity – here the researcher compares her measures to measures of the same concept developed through other methods. Probably the most obvious example of this is the British Crime Survey as a test of the ‘validity’ of Police Crime Statistics’. The BCS shows us that different crimes, as measured by PCR have different levels of construct validity – Vehicle Theft is relatively high, vandalism is relatively low, for example.

Source 

Bryman (2016) Social Research Methods

 

 

Britain in Statistics (2017)

Just a look back at what some of the official statistics and opinion polls told us about life in Britain in 2017…selected so they’re relevant to families and households, education and crime and deviance…

  • The proportion of women aged 18 who started university in 2017 was nearly 1/3rd greater than men – 37.1% compared to 27.3%.
  • Family size is declining: about 45% of children today have no siblings.
  • The ageing population: the proportion of people aged 65 and over in work has almost doubled since 1992 – 5.5% to 10.4% – there are now nearly 1.2 million over 65s in work.
  • The downsides of immigration: Of the 8008 people registered homeless in London (2015-16) only 3271 were British, nearly 3000 were from central or eastern Europe and fully 1,546 were Romanian
  • Crime and racial injustice: Young black youths are nine times as likely to be in England and Wales.
  • Class inequality: there are 59 theaters in London’s private schools, but only 42 in the West End.

I had intended to make this an all bells and whistles posts, but time, much like the year, has just about run out!

Happy New Year!

Sources:

Taken from The Week, 23rd December 2017.

Concepts in Quantitative Sociological Research

Concepts are the building blocks of theory, and are the points around which social research is conducted.

Concepts are closely related to the main sociological perspectives, and some of the main concepts developed by different perspectives include:

  • Functionalism – social integration and anomie
  • Marxism – social class and alienation.
  • Feminism – gender and patriarchy
  • Interactionism – labelling and discrimination
  • Postmodernism – identity.

Within sociology, one might even say that there’s a more ‘fundamental’ layer of concepts that lie behind the above – such as ‘society’, ‘culture’ and ‘socialization‘, even ‘sociology’ itself is a concept, as are ‘research’ and ‘knowledge’.

Concepts also include some really ‘obvious’ aspects of social life such as ‘family’, ‘childhood’, ‘religious belief’, ‘educational achievement’ and ‘crime’. Basically, anything that can be said to be ‘socially constructed’ is a concept.

Each concept basically represents a label that researchers give to elements of the social world that strikes them as significant. Bulmer (1984) suggests that concepts are ‘categories for the organisation of ideas and observations’.

Concepts and their measurement in quantitative research 

If a concept is to be employed in quantitative research, a measure will have to be developed for it so it can be quantified.

 

Once they have been converted into measures, concepts can then take the form of independent or dependent variables. In other words, concepts may provide an explanation of a certain aspect of the social world, or they may stand for things we want to explain. A concept such as educational achievement may be used in either capacity – we may explore it as a dependent variable (why some achieve fewer GCSE results than others?) Or: as an independent variable (how do GCSE results affect future earnings?).

Measures also make it easier to compare educational achievement over time and across countries.

As we start to investigate such issues we are likely to formulate theories to help us understand why, for example, educational achievement varies between countries or over time.

This will in turn generate new concepts, as we try to refine our understanding of variations in poverty rates.

Why Measure Concepts?

  1. It allows us to find small differences between individuals – it is usually obvious to spot large differences, for example between the richest 0.1% and the poorest 10%, but smaller once can often only be seen by measuring more precisely – so if we want to see the differences within the poorest 10%, we need precise measurements of income (for example).
  2. Measurement gives us a consistent device, or yardstick for making such distinctions – a measurement device allows us to achieve consistency over time, and thus make historical comparisons, and with other researchers, who can replicate our research using the same measures. This relates to reliability.
  3. Measurement allows for more precise estimates to be made about the correlation between independent and dependent variables.

Indicators in Quantitative Social Research 

Because most concepts are not directly observable in quantitative form (i.e. they do not already appear in society in numerical form),  sociologists need to devise ‘indicators’ to measure most sociological concepts. An indicator is something that stands for a concept and enables (in quantitative research at least) a sociologist to measure that concept.

For example….

  • We might use  ‘Average GCSE score’ as an indicator to measure ‘educational achievement’.
  • We might use the number of social connections an individual has to society to measure ‘social integration’, much like Hirschi did in his ‘bonds of attachment theory‘.
  • We might use the number of barriers women face compared to men in politics and education to measure ‘Patriarchy’ in society.

NB – there is often disagreement within sociology as to the correct indicators to use to measure concepts – before doing research you should be clear about which indicators you are using to measure your concepts, why you are choosing these particular indicators , and be prepared for others to criticize your choice of indicators. 

Direct and Indirect indicators 

Direct indicators are ones which are closely related to the concept being measured. In the example above, it’s probably fair to say that average GCSE score is more directly related to ‘educational achievement’ than ‘bonds of attachment’ are to ‘social integration’, mainly because the later is more abstract.

How sociologists devise indicators:

There are a number of ways indicators can be devised:

  • through a questionnaire
  • through recording behaviour
  • through official statistics
  • through content analysis of documents.

Using multiple-indicator measures

It is often useful to use multiple indicators to measure concepts. The advantages of doing so are three fold:

  • there are often many dimensions to a concept – for example to accurately tap ‘religious belief’ questionnaires often include questions on attitudes and beliefs about ‘God’, ‘the afterlife’, ‘the spirit’, ‘as well as practices – such as church attendance. Generally speaking, the more complex the concept, the more indicators are required to measure it accurately.
  • Some people may not understand some of the questions in a questionnaire, so using multiple questions makes misunderstanding less likely.
  • It enables us to make more nuanced distinctions between respondents.

Measuring the effectiveness of measures in quantitative social research

It is crucial that indicators provide both a valid and reliable measurement of the concepts under investigation.

 

 

What is an Indicator?

An indicator provides a measure of a concept, and is typically used in quantitative research.

It is useful to distinguish between an indicator and a measure:

Measures refer to things that can be relatively unambiguously counted, such as personal income, household income, age, number of children, or number of years spent at school. Measures, in other words, are quantities. If we are interested in some of the changes in personal income, the latter can be quantified in a reasonably direct way (assuming we have access to all the relevant data).

Sociologists use indicators to tap concepts that are less directly quantifiable, such as job satisfaction. If we are interested in the causes of variation of job satisfaction, we will need indicators that stand for the concept of ‘job satisfaction’. These indicators will allow the level of ‘job satisfaction’ to be measured, and we can treat the resulting quantitative information as if it were a measure.

An indicator, then, is something which is devised that is employed as though it were a measure of a concept.

Direct and Indirect indicators 

Direct indicators are ones which are closely related to the concept being measured. For example questions about how much a person earns each much are direct indicators of personal income; but the same question would only be an indirect measurement of the concept of social class background.

 

 

 

 

Are female surgeons better?

New research suggests that women make better surgeons than men. For the study, a team at the University of Toronto compared like for like procedures performed by 3,314 surgeons at a single Canadian based hospital over an eight-year period.

This revealed that the post-operative death rages for female surgeons were 12% lower than for their male counterparts – a figure that equates to one less patient dying per every 230 operations a woman performs. (Clearly the death rates are very low!).

Previous research has also found that women doctors have, on average, slightly better outcomes than male ones and that they are less likely to be struck off.

How might we explain these disparities?

  • Researchers speculate that women may be more better communicators and more cautious than men.
  • However, it may also be that women face greater obstacles to entering a male-dominated profession – with the result that only the most skilled qualify as surgeons.
  • You also have to question the representativeness of the Canadian study – in only one hospital in one country, you can hardly generalise from this!

Sources

The Week, 21 Oct 2017