Conference: International Association for Forensic Phonetics and Acoustics – Zurich

The IAFPA conference in Zurich has just finished – and what a great experience it was. Despite the 35-degree heat, participants minds were fully alert, and the program was packed with excellent presentations from established and emerging scholars, including an unprecedented degree of student participation.

As usual, the Hub’s colleagues at University of Zurich’s Centre for Forensic Phonetics and Acoustics put on a brilliant social program which enabled the conversation to continue into the evening.

Read all about it at the conference website here: iafpa2023.uzh.ch/

As for the Hub’s contribution, you can read our abstract below. Other abstracts are in the conferences Book of Abstracts – well worth a peruse.

Towards accountable evidence-based methods for producing reliable transcripts of indistinct forensic audio

Helen Fraser, Debbie Loakes, Ute Knoch, and Lauren Harrington

Covert recordings provide powerful evidence in criminal trials, but are often of extremely poor quality. Many jurisdictions allow the court to be assisted by a police transcript, but these can be unreliable (French and Fraser, 2018). The law has developed safeguards intended to ensure that triers of fact are not misled by inaccurate transcripts. However, these safeguards are ineffective, as they rely on lawyers and judges checking the transcript against the audio (Fraser and Kinoshita, 2021). Even if experts are consulted, responsibility for evaluating their findings typically rests with lawyers and judges. Multiple cases of actual and potential injustice have been documented. In 2017, Australian linguists raised a Call to Action, asking the judiciary to review and reform the legal handling of indistinct forensic audio.

The present paper starts with a brief update on the progress of the Call to Action, which gives reason to hope that police transcripts will eventually be disallowed. This makes it important for linguists to be able to provide reliable transcripts of indistinct forensic audio via accountable, evidence-based methods – that do not start from a police version (though investigators’ opinions may well be sought at an appropriate point in the process).

The paper then shares results of an experiment which forms part of an investigation into how transcription methods can best be evaluated, drawing on insights from language testing research (Knoch and Macqueen 2020).

Forty participants transcribed a three-minute sample of forensic-like audio, without contextual information. Each transcript was divided into intonation phrases (IPs) and each IP was scored against the reference transcript, with a global rating, and three analysis ratings, showing how much was misinterpreted, missing or added.

Overall scores were relatively low, and highly variable – e.g., exact matches to the reference transcript varied from 47% to 12%. A wide range of demographic data were collected, but the only one to show a significant correlation with score was language background, with L2 speakers of English, on average, scoring lower than L1 speakers – even though all L2 speakers were highly proficient in English (cf. de Boer, 2016). Other factors that might have been expected to correlate with scores (e.g., a background in phonetics or forensic speech science) did not. Participants’ confidence was not a reliable indication of performance.

These results, following those of Tschäpe & Wagner (2012) and Love & Wright (2020), confirm that it is unrealistic to expect individual transcribers with no contextual information to produce demonstrably reliable transcripts (Fraser 2022a). Rather it is necessary for accredited transcribers to follow an evidence-based method, designed and managed by experts – as is done for other responsible forensic sciences. Further, it is important to ensure that transcripts are used in trials in a manner that minimises opportunities for the court to be misled about the content of indistinct forensic audio (see Haworth, 2018; Fraser, 2022b).

The paper concludes by outlining a proposed method for producing and evaluating transcripts, seeking discussion and input from IAFPA members (see Harrington and Rhodes, in prep).

References

  • de Boer, Meike. Expectancy bias in forensic speech transcriptions: Can a forensic context change what is heard in an audio fragment? Masters Thesis, Maastricht
  • Fraser, H. (2022a). Forensic transcription: Legal and scientific perspectives. In C. Bernardasci, et al (Eds.), Speaker Individuality in Phonetics and Speech Sciences: Speech Technology and Forensic Applications (pp. 19–32). Milano: Officinaventuno.
  • Fraser, H. (2022b). A framework for deciding how to create and evaluate transcripts for forensic and other purposes. Frontiers in Communication.
  • Fraser, H., & Kinoshita, Y. (2021). Injustice arising from the unnoticed power of priming: How lawyers and even judges can be misled by unreliable transcripts of indistinct forensic audio. Criminal Law Journal, 45(3), 142–152.
  • Fraser, H., & Loakes, D. (2020). Acoustic injustice: The experience of listening to indistinct covert recordings presented as evidence in court. Law Text Culture, 24, 405–429.
  • French, P., & Fraser, H. (2018). Why “ad hoc experts” should not provide transcripts of indistinct forensic audio, and a proposal for a better approach. Criminal Law Journal, 42, 298–302.
  • Harrington, L. & Rhodes, R. (in prep) Forensic transcription: A survey of expert transcription practices in Europe and North America.
  • Haworth, K. (2018). Tapes, transcripts and trials. International Journal of Evidence and Proof, 22(4), 428–450.
  • Knoch, U., and Macqueen, S. (2020). Assessing English for Professional Purposes. Routledge.
  • Love, R., & Wright, D. (2021). Specifying challenges in transcribing covert recordings: implications for forensic transcription. Frontiers in Communication: 6:797448. doi: 10.3389/fcomm.2021.797448
  • Marzi, T. et al. (2021). Mapping the featural and holistic face processing of bad and good face recognizers. Behavioural Sciences, 11(5), 75.
  • Tschäpe, N., & Wagner, I. (2012). Analysis of disputed utterances: A proficiency test. IAFPA.