AI evaluations 1174/2025

FOI Reference: 1174/2025

Request

I am investigating how AI-assisted evidence analysis is documented and reported within existing Chain of Custody (CoC) and Audit Trail (AT) structures.

For the sake of clarity, I have included a glossary at the end of this document. CoC is used here as an umbrella term encompassing related concepts such as continuity, chain of evidence, and provenance; while AT refers to any documentation system that captures and records the forensic processing of evidence.

1. Has your force conducted any evaluations or assessments regarding the effectiveness of AI systems for evidence processing, including audio, textual, or other forms of evidence?

If yes, please provide copies of these documents or reports.

Glossary

• Artificial Intelligence (AI): This refers to a machine that learns, generalises, or infers meaning from input, thereby reproducing or surpassing human performance. The term AI can also be used loosely to describe a machine's ability to perform repetitive tasks without guidance.
• Audit Trail (AT): Refers to the documentation processes that capture, record, and report the sequence of actions applied to digital evidence. An AT may include details such as model configurations, parameters used, error rates, human interventions, and system-generated logs. It functions as a transparency mechanism, enabling the reconstruction and evaluation of how evidence has been processed, analysed, and interpreted.
• Automatic Speech Recognition (ASR): A subfield of AI that processes spoken human language (audio) and converts it into written text, useful for tasks like transcription and speaker diarisation.
• Chain of Custody (CoC): This is the record of how evidence is handled from the point of collection through processing to presentation in court. This may include the identity of the person handling the evidence, date/time of transfer, storage details, and digital identifiers. Alternative terms include 'continuity', 'chain of evidence' and 'provenance'.
• Forensic Audio evidence: This refers to recorded sound collected, preserved, and analysed for legal or investigative purposes. This often includes spoken language from sources including body-worn cameras, dashcams, police radio communications, and interview or interrogation recordings.
• Forensic Textual evidence: This refers to written or encoded content, stored or transmitted in digital form, collected, preserved, and analysed for legal or investigative purposes. It includes unstructured human language (e.g., transcripts, emails, chat logs, social media posts), semi-structured data (e.g., system or network logs, ASR outputs), and structured data (e.g., metadata, timestamps).
• Large Language Models (LLMs): A type of Natural Language Processing (NLP) system trained on vast amounts of text to perform sophisticated language tasks like summarisation, translation, or text generation.
• Natural Language Processing (NLP): A subfield of AI focused on the interaction between computers and human (natural) language. It involves programming computers to process, analyse, understand, and generate human language. It is useful for tasks like translation,  document generation, summarisation and redaction.
• Paradata: This refers to information generated as a by-product of human interaction with AI systems during evidence processing. This includes prompts, edit histories, data discards, interaction logs, correction logs, reviewer annotations, parameter adjustments, and iterative outputs. Such records provide traceable documentation of how humans and AI jointly contribute to outputs, supporting transparency and accountability in evidential practice.

I can confirm that Dyfed-Powys Police does not hold the information requested

Response

No

It should be noted that as a result of the systems adopted by Dyfed-Powys Police in relation to the recording of such information that the information released may or may not be accurate.

(This is a response under the Freedom of Information Act 2000 and disclosed on 16/12/2025)