Research report – “Assisting listeners to hear words that aren’t there” (Fraser 2018)

In 2018, Helen published a paper in the Australian Journal of Forensic Sciences called ‘Assisting’ listeners to hear words that aren’t there: dangers in using police transcripts of indistinct covert recordings. This paper is highly relevant to our ongoing work in the Research Hub for Language in Forensic Evidence, and we summarise the main findings here.

This is not an open access paper but we can send you a copy (contact us via our email addresses here). In this blog post we refer to this paper as Fraser (2018a) – Fraser (2018b) is another highly recommended article about forensic transcription.

Background

Fraser (2018a) is based on one of the key issues that the Research Hub for Language in Forensic Evidence seeks to address, which is about how police transcripts, often based on (extremely) indistinct forensic audio, can prime listeners to hear what is written in them whether the transcript is correct or not. This is clearly a justice issue, and many of Helen’s publications have talked about this matter (e.g. Fraser & Stevenson 2014, Fraser 2018b, Fraser & Loakes 2020, Fraser & Kinoshita 2021). You can also read more in some of our previous blog posts (see especially this post from June 2021 and this post from Sept 2021). You can also read more here about the aims of the Hub.

Data in the Fraser (2018a) paper is forensic audio used in a murder trial, as well as a phrase used in a police transcript which led to a guilty verdict. Helen explains how in a previous paper (Fraser & Stevenson 2014), it was shown that “first … the detective’s transcription of the phrase is inaccurate, and second … the phrase nevertheless has a strong priming effect on listeners”.

Aim

The aim of Fraser (2018a) is to show how easily listeners can be primed to hear something which is wrong, and also to locate the incorrect phrases in a section of audio.

Intermission!

You can listen to the audio in the experiment here just scroll down to “‘Assisting’ listeners to hear words that aren’t there”. Try and locate the phrase Adelaide bank account while you are listening.

As mentioned in Fraser (2018a), it is best to do this before reading on. The audio is only one minute!

Method

Fraser (2018a) was based on a listening experiment with 76 listeners, using the data mentioned above, which we hope you have had a chance to listen to.

The experiment has two transcribed phrases associated with the audio, the first is “Adelaide Bank Account” and the second is “at the start we made a pact”. The first phrase was made up by Helen for the experiment, and the second was the phrase leading to the guilty verdict.

Participants had to listen to the audio cold, and transcribe it, and they were free to take as long as they liked. After that, the participant pool was divided into two groups and asked to locate the different “target” phrases (group 1 – ADL group Adelaide bank account; Group  2 – PACT).

Findings

When participants transcribed cold, Fraser (2018a: 113) reports that their transcripts “differed wildly” but 93% contained the word “money”. Some examples she gives in the paper are

  • money mortgages?
  • now according to this, he helped me get money, barbeque, Amish
  • we have to get money
  • money, barbecue package
  • do you have like the moneydark and shady

Fraser (2018a) also looked at whether any participants transcribed the target phrases before they were exposed to them. She reports that one person transcribed Adelaide bank account, and 25% of transcripts also contained bank account, but there was some variability about which part of the audio this phrase was heard. She also points out that nobody heard the “pact” phrase at all.

When listeners were primed with the target phrases, Fraser (2018a) reports that 68% of the ADL group felt they definitely heard the phrase Adelaide bank account, while 32% said they thought they heard it (100% of participants). For the PACT group, numbers were lower, but 15% said they definitely heard the phrase at the start we made a pact and 13% said they thought they heard it.

Discussion 

The findings of this paper are stark – 100% of ADL participants “heard” a phrase once they were primed with it, despite the fact it was fabricated for the purposes of the experiment. Fewer participants heard the pact phrase used in the murder trial once primed with it – yet almost half agreed with it to some extent. This is highly problematic – the police transcript that helped convict a man of murder was considerably less plausible to participants than the one made up by the experimenter.

Fraser (2018a) goes through the serious implications of these findings. In particular, in an experiment the stakes are low and participants are not given further contextual information about the real case. However this is not true in court, where juries are in fact exposed to far more priming detail along with transcripts provided to them.

Some of the key take away findings from this article are:

  • “[l]isteners do not need much acoustic evidence to confirm they ‘hear’ a phrase they are ‘listening out for’ in indistinct audio” (Fraser 2018a: 135);
  • some phrases are more plausible than others (also see our June 2021 blog post) but crucially this does not mean they are correct.

Conclusion

Two primary points are made in the conclusion (see Fraser 2018a: 138). The first one is that “evaluation of indistinct covert recordings is not a matter for ‘common knowledge'” despite current legal processes. The second is that transcripts can be “thoroughly unreliable, yet nevertheless highly influential”.

The Hub continues to work on these matters, and we are currently conducting experiments to ensure more just and reliable processes.

References

Fraser, H. (2018a) Assisting’ listeners to hear words that aren’t there: dangers in using police transcripts of indistinct covert recordings, Australian Journal of Forensic Sciences, 50:2, 129-139, DOI: 10.1080/00450618.2017.1340522

Fraser, H. (2018b) Forensic transcription: How confident false beliefs about language and speech threaten the right to a fair trial in Australia Australian Journal of Linguistics 50: 129-139. open access

Fraser, H., & Y. Kinoshita (2021). Injustice arising from the unnoticed power of priming: How lawyers and even judges can be misled by unreliable transcripts of indistinct forensic audio. Criminal Law Journal, 45(3). open access

Fraser, H. and D. Loakes (2020). Acoustic injustice: The experience of listening to indistinct covert recordings presented as evidence in court, Law Text Culture, 24, 405-429. open access

Fraser H & B. Stevenson (2014). The power and persistence of contextual priming: more risks in using police transcripts to aid jurors’ perception of poor quality covert recordings. International Journal of Evidence and Proof. 18, 205–229. DOI: 10.1350/ijep.2014.18.3.453