This post outlines an interesting comparative research study of secondary documents (‘private’ letters and a more public blog) which could be used to get students thinking about the usefulness of such sources in social research.
I’ve taken the summary below straight from Bryman (2016) Social Research Methods:
It is tempting to think that the century and a half that separates a solider writing a military blogs and the letters and diary of a solider in the American civil war will be far apart in tone and content.
Shapiro and Humphreys (2013) compare the military blog of ‘Dadmanly’, who was in the US army for just over four years beginning in August 2004 and who served in Iraq for 18 months, with the letters and diaries of ‘Charlie Mac’, who joined the Union army in 1862, whose writings continued until 1865.
Dadmanly’s blog is looking like a bit of a historical artefact already. with its last update in 2012, but he did make some contributions to the more recent ‘blog of war’ book, which brings together different bloggers from the front-line of war.
There are clear differences between them:
Dadmanly wrote for a general audience the vast majority of whom he would never know
Charlie Mac wrote primarily for his large family, although he seems to have anticipated that that they would passed around to others, as they have a tone which implies they will have a more general readership than just his close family.
However, there are also various common elements:
Both writers show a desire to reassure family and friends about their safety and well-being.
Both expressed opinions about the progress of the war, and offered political commentary on them;
both wrote in large part to maintain contact with their families during the wars,
and the writing was therapeutic for both of them.
Shapiro and Humphries conclude that this comparison is significant because it shows that changes in communications technologies do not necessarily result in changes in the nature of the content of communication.
one question you might like to consider is whether Dadmanly’s blog is any less valid as a source of information about war than Charlie Mac’s letters?
How useful is secondary qualitative data when researching education? This post considers some of the theoretical, practical and ethical limitations of Public and Personal Documents which are produced in the context of education – such as OFSTED reports, school prospectuses, school reports and messages sent between pupils.
Public Documents
Personal Documents
Ofsted and Inspection Reports
School Websites
School prospectuses
School policy documents
School text books
School reports on pupils
Pupils written work
Pupils’ and teachers’ diaries
Notes and text messages passed between pupils
Practical Issues with using documents when researching education
Since the 1988 education act many Public documents on education are freely available to the public – OFSTED reports on schools are easily obtainable, and schools publish a wide variety of information about themselves in their prospectuses and on their websites.
Schools also publish a huge variety of policy documents – such as student codes of conducts, equal opportunities policies and information about how they implement every child matters and safeguarding policies – all of which are likely to be made available to researchers on request, since they are a matter of public record. These are useful as they give an account of the ‘official’ picture of schools in Britain from the perspective of management.
Theoretical Issues with using documents when researching education
In terms of validity, while school web sites and prospectuses can be trusted to provide some basic information about what subjects are on offer, GCSE results and extra-curricular activities, the credibility of such sites is undermined by the fact that they produced to advertise the school in a positive light, and all of these web sites put a positive spin on the school or college. For example, although schools are required to publicise their results, they do have some freedom to emphasise the way they report them so they can portray themselves in the best possible light.
To what extent do school web sites provide a valid picture of school life?
Suggested activity: Visit the web sites of your past school and present college – to what extent do they give you an accurate picture of what life was actually like in that school?
Extension activity: Look at the web site from another school in your area. Pick one that is very different in terms of results and so on. Are the impressions you get of the two schools that different, or are they quite similar, which would suggest that school web sites are designed to a formula and really tell you very little about a particular school.
OFSTED reports may provide greater insight into what’s going on in a school than the statistical snap shot of the yearly GCSE results, but OFSTED inspections only last for three days, and are typically only done every four years, so it is quite easy for a school to put on an act for this short a period and produce a performance which is better than usual.
Might some schools be able to put on a better performance than usual during a brief OFSTED inspection?
Conversely, there are some schools that feel as if they have been harshly judged by OFSTED inspectors, and question the validity of OFSTED reports, feeling that the grade they’ve been awarded does not reflect the reality of school life. This is partly because OFSTED inspectors only really get to see one lesson by each teacher, which is not representative, but also because the focus of different OFSTED inspectors will be different in different schools, raising the prospect that schools are not being judged by the same standard.
Policy documents produced by schools, such as student codes of conduct might be useful for seeing how schools function in an ideal-world, but they lack validity in that they tell you nothing about how many students actually stick to the code of conduct or what’s done with students who break the code of conduct. If you wanted to get more of an insight into this, a researcher would have to gain access to individual reports of each student, which would be more difficult to obtain.
Representativeness and Public Documents in Educational Research
All schools and colleges are required to publish prospectuses and results, so these should cover a 100% sample of educational institutions, but the same cannot be said of OFSTED reports – schools graded outstanding go into ‘light touch’ mode and may not be observed for several years.
Using Personal Documents to Research Education
Suggested starter activity – Have a browse of this interesting blog – ‘Scenes from the Battleground’ – which has had over a million hits and is written by a teacher. What impression does this give you school life? How valid and representative do you think it is?
Personal documents in the context of education include school reports on pupils, pupils written work, pupils’ and teachers’ diaries and Notes and text messages passed between pupils
Practical Issues
For a start, these will be very difficult to access. Things like teacher mark books, records of conversations with students, and disciplinary records may not be available because of the ethical requirement to safeguard children’s privacy. The same could also be true of the written work of pupils.
Where private messages and texts are concerned, it is unlikely that researchers will be allowed access to students personal mobile phones or tablets, and even if they could gain access, threads of conversation may have been deleted shortly after they took place, and the more ‘anti-school’ such messages are, the more likely they are to have been deleted.
Validity and Personal Documents
The kind of personal documents which are readily available are likely to be of a public nature (social media accounts for example) and because they are public, they would have been subjected to impression management so they are acceptable – so while this can give us an insight into what teachers and staff think is socially acceptable, using these to give us a picture of what people actually think about school life is problematic. The more ‘personal’ and private a document is, then the higher the validity is likely to be – however, the number of people who write down in-depth personal accounts of their school experiences is tiny.
Representativeness
If one could gain access to social media accounts and personal messaging services, representativeness should be good as the majority of students have access and make use of these services.
As mentioned above, hardly anyone keeps diaries any more, and so representativeness here is a problem.
If a researcher is lucky enough to gain access to disciplinary records, these may not be representative of the actual underlying patterns of student disobedience – teacher bias may increase the number of certain types of students who have undergone disciplinary procedures.
Ethical Issues when using public and private documents in educational research
There are no particular ethical problems with using publicly produced documents,
When using private or personal documents, there are some ethical concerns. If the researcher is given access to teacher mark books, records of conversations with students, and disciplinary records this won’t necessarily be with the informed consent of the pupils for example.
