DPP v Tuite
[2014] VSC 662
•19 December 2014
| IN THE SUPREME COURT OF VICTORIA | Not Restricted |
AT MELBOURNE
CRIMINAL DIVISION
S CR 2014 0007
| DIRECTOR OF PUBLIC PROSECUTIONS |
| v |
| CLINTON JAMES TUITE |
---
JUDGE: | EMERTON J |
WHERE HELD: | Melbourne |
DATE OF HEARING: | 11, 14-15, 17-18, 21-25, 28-31 July 2014 20-21, 23 October 2014 |
DATE OF RULING: | 19 December 2014 |
CASE MAY BE CITED AS: | R v Tuite |
MEDIUM NEUTRAL CITATION: | [2014] VSC 662 |
---
EVIDENCE – Admissibility – Expert Evidence - DNA evidence – Likelihood ratios - Whether fully-continuous probabilistic statistical methodology for the evaluation of DNA evidence constitutes a new and discrete field of knowledge – Whether admission of DNA evidence would give rise to unfair prejudice to the accused due to its complexity - Evidence Act 2008 (Vic), ss 79(1), s 137.
---
APPEARANCES: | Counsel | Solicitors |
| For the Crown | Dr N Rogers Mr B Sonnet | Office of Public Prosecutions Victoria |
| For the Accused | Mr J Desmond | Doogue O’Brien George |
HER HONOUR:
Background
The accused has been charged with aggravated burglary, rape, indecent assault and intentionally causing injury.
The offending is alleged to have taken place very early in the morning of 13 October 2007. The details of the offending alleged by the Crown are broadly as follows. The complainant lived alone. On the day in question, she returned home from her shift work at about 2.30am. She had something to eat, watched some television and showered. She walked into her bedroom to find a man wearing a black balaclava and dark clothing standing at the foot of her bed. He grabbed her, secured her hands using white cable ties and pushed her face down onto the bed. When she struggled, he struck her on the head with a piece of wood. He wrapped grey duct tape around her neck and mouth, and placed a blindfold over her eyes. When, in the course of struggling, the complainant fell onto the bedroom floor, the man stood on her throat until she passed out. On regaining consciousness, she found that the cable tie from her right wrist had been removed and her hands were no longer tied together, the tape had been removed from her mouth and was hanging loose around her neck and her tracksuit pants had been pulled up. She saw the man standing in the doorway of the bedroom. He said to her, ‘You had fallen asleep’, and left the unit.
When investigating police arrived at the complainant’s address, they found the complainant with a cable tie attached to her left wrist and duct tape wrapped around her neck. Crime scene investigators were called to process the crime scene. They took a series of photographs and seized a number of items for forensic examination.
Among other things, at the entrance to the laundry, the crime scene investigator found and seized a makeshift blindfold made of a piece of dark material, apparently torn from the sleeve of a garment, with a shoelace tie. The crime scene investigator found and seized eight cable ties from the bathroom floor. Outside the house, he located the unused end of a cigarette in a garden bed and two used cigarette butts on the other side of the fence on the neighbouring property. One of the cigarette butts was on the edge of the lawn adjacent to a garden bed; the other was lying on the middle rail of the boundary fence.
These and the other seized items were separately bagged, and the individual bags were placed in a larger bag. The large bag containing the small bags was delivered to the Victoria Police Forensic Science Service (VPFSS) on 15 October 2007.
As a result of DNA analyses on the seized items carried out by VPFSS in 2007, an unknown male profile was identified and placed onto the national DNA database.
On 15 March 2012, the accused provided a DNA sample for placement on the national DNA database as a consequence of a court order for an unrelated conviction. In May 2012, the accused was nominated as a person of interest in the investigation of these offences.
The prosecution seeks to rely at trial on the results of DNA analyses of the following items seized by police at the crime scene:
(a)Item 1 – the makeshift blindfold with the attached shoelace found on the floor outside the laundry;
(b)Item 4 – seven cable ties found on the bathroom floor;
(c)Item 5A – cigarette butt found on the neighbouring property;
(d)Item 5B – second cigarette butt found on the neighbouring property.
In particular, the prosecution seeks to rely on DNA evidence derived from three samples taken from the blindfold fabric and its shoelace tie (referred to by VPFSS as Items 1-1, 1-2 and 1-3), trace samples taken from the entire area of the seven cable ties combined (referred to by VPFSS as Item 4-1), a portion of filter from one cigarette butt (referred to by VPFSS as Item 5A-1) and a portion of filter from the second cigarette butt (referred to by VPFSS as Item 5B-1).
The six samples in issue were initially analysed in 2007 using a 10 marker profiling kit known as ‘Profiler Plus’ (‘P+’). Two of the samples were re-analysed in 2013 using a much more sensitive and discriminating 21 marker profiling kit called ‘PowerPlex 21’ (‘PP21’).
DNA evidence is conventionally presented in the form of a ‘likelihood ratio’. The likelihood ratio compares the likelihood of the correctness of the ‘prosecution hypothesis’ to the likelihood of the correctness of a competing hypothesis. It expresses in mathematical terms the likelihood that a particular person contributed to the detected DNA sample as compared to a person chosen at random from the population.[1] If what is detected in a sample is a mixture of the DNA from more than one person, the assumptions that underlie the expression of that ratio will be altered. Generally, the higher the ratio, the more likely it is that the prosecution hypothesis is correct.
[1]Or the relevant population group, such as the Australian Caucasian population.
The likelihood ratio for each of the Items has been calculated using the recently introduced STRmix program. STRmix is a statistical software package for computing likelihood ratios developed by scientists in Australia and New Zealand. It is used in New Zealand and in all States and Territories in Australia except Tasmania and the ACT. It was introduced into Victoria in March 2013. The defence argues against the admissibility of the DNA evidence on the basis that STRmix has not yet been shown to be sufficiently reliable for use in criminal trials.
Defence challenge
The defence challenges the admissibility of the DNA evidence on the basis that:
(a)it is opinion evidence that the Crown has not established satisfies the requirements of s 79 of the Evidence Act 2006 (Vic); and
(b)its probative value is outweighed by the danger of unfair prejudice to the accused and the Court must exclude it under s 137 of the Evidence Act.
In substance, the defence challenges the use of the STRmix methodology to analyse and generate likelihood ratios for the Items because it is ‘new’ science that is largely untested and has not been generally accepted by the forensic science community, and because it is unsuitable for use with ‘low-template’ DNA due to the likelihood of random or ‘stochastic’ effects in the profiling of such DNA.
The defence says further that the SPURS statistical program previously used by VPFSS is not an appropriate methodology where there is more than one contributor to the DNA profile generated by the sample (which is the case for all Items other than Items 5A-1 and 5B-1).
The DNA evidence
The results of the VPFSS analysis of the six Items are as follows:
(a)Item 1-1 (trace from the outside surface of the blindfold fabric)
Item 1-1 was analysed using P+ (10 markers).
Analysis showed a partial mixed DNA profile from three contributors. The complainant is an assumed contributor.
Using STRmix, it is estimated to be 23 million times more likely that the DNA profile obtained from Item 1-1 would occur if the DNA originated from the accused, the complainant and one unknown person than if it originated from the complainant and two unknown people chosen at random from the Australian Caucasian population.
(b)Item 1-2 (trace from the inside surface of the blindfold fabric)
Item 1-2 was analysed using P+.
Analysis showed a mixed DNA profile from three contributors. The complainant is an assumed contributor.
Using STRmix, it is estimated to be 9.7 million times more likely that the DNA profile obtained from Item 1-2 would occur if the DNA originated from the accused, the complainant and one unknown person than if it originated from the complainant and two unknown people chosen at random from the Australian Caucasian population.
(c)Item 1-3 (trace from the ends of the shoelace combined)
Item 1-3 was analysed using PP21 (21 markers).
The analysis showed a mixed DNA profile from three contributors. The complainant is an assumed contributor.
Using STRmix, it is estimated to be 2.7 sextillion[2] times more likely that the DNA profile obtained from Item 1-3 would occur if the DNA originated from the accused, the complainant and one unknown person than if it originated from the complainant and two unknown people chosen at random from the Australian Caucasian population. This is reported using the default likelihood ratio for PP21 analyses of one hundred billion.
[2]A sextillion in the UK is the sixth power of a million (10 to the power of 36) and in the US is the seventh power of a thousand (10 to the power of 21).
(d)Item 4-1 (trace from the entire area of the seven cable ties combined)
Item 4-1 was analysed using PP21.
The analysis showed a partial mixed DNA profile from two contributors.
Using STRmix, it is estimated to be 35 million times more likely that the DNA profile obtained from Item 4-1 would occur if the DNA originated from the accused and an unknown person than if it originated from two unknown persons chosen at random from the Australian Caucasian population.
(e)Item 5A-1 (portion of filter of one cigarette butt)
Item 5A-1 was analysed using P+.
Analysis showed a single source profile.
Using STRmix, it is estimated to be 29 billion times more likely that the DNA profile would occur if the DNA originated from the accused than if it originated from an unknown person chosen at random from the Australian Caucasian population.
(f)Item 5B-1 (portion of filter of the other cigarette butt)
Item 5B-1 was also analysed using P+.
Analysis showed a single source profile.
It is estimated to be 29 billion times more likely that the DNA profile would occur if the DNA originated from the accused than if it originated from an unknown person chosen at random from the Australian Caucasian population.
It can be seen that:
(a) Only the cigarette butts (Items 5A-1 and 5B-1) produced a single source profile. There were multiple contributors to the DNA profiles on the blindfold and the cable ties, making analysis of these DNA profiles and their comparison with the reference samples of the accused more complex;
(b) Item 1-1 and Item 4-1 produced only partial (or incomplete) profiles, again making analysis more complex.
In addition, the DNA in Item 4-1 is ‘low-template’ DNA, in that the amount analysed is significantly less than the optimal amount of 0.5ngs. Some of the contributors to the DNA on the Items with multiple contributors also individually provided very low amounts of DNA and all of the profiles other than for Items 5A-1 and 5B-1 contain low level or sub-optimal peaks.
If DNA on a crime scene sample is of low quality or quantity, then the profile may be ambiguous. Low-template DNA profiles are more likely to exhibit ‘stochastic’ or random effects, complicating their interpretation. These effects may include ‘drop in’ which are peaks in the profile that are not true alleles[3] and, conversely, alleles may go missing or ‘drop out’ of the profile, producing partial profiles only.
[3]An allele is an alternative form of a gene (one member of a pair) that is located at a specific position on a specific chromosome.
STRmix purports to account for many of these random effects in a DNA profile. It is based on continuous modelling of peak heights, and therefore applies no thresholds below which peaks will not be interpreted.[4] Described as ‘fully-continuous’, it purports to make use of all the information in the profile, including peaks appearing below what other systems used for the evaluation of DNA profiles would designate as a ‘stochastic threshold’.
[4]Other than a very low ‘detection threshold’ in order to eliminate from the analysis any background ‘noise’.
Importantly, STRmix also purports to take into account peaks that are not detected as present in the DNA profile by positing the possibility that alleles have ‘dropped- out’ of the profile and giving them a value. How this is done and whether it is legitimate is a source of strong disagreement between the parties in this case.
As a statistical program, STRmix uses the Markov Chain Monte Carlo (‘MCMC’) process. In seeking to explain the observed profile STRmix randomly chooses genotype sets for all the contributors, as well as a template and a degradation amount for each contributor and locus amplification efficiencies, gradually building up a picture of the components in the observed profile. Through this iterative process, STRmix identifies the genotype combinations that best explain the observed DNA profile, and it weights the genotype combinations according to how well they explain that profile.
At the start of the process, STRmix will therefore list all possible explanations for the observed profile, including genotypes that have drop-in and drop-out. It will also include multiple drop-out scenarios, as well as scenarios that do not require any drop-out or drop-in. It will produce a comprehensive list of all possible explanations for the profile. These possible explanations are then ranked depending on how well they explain the observed profile: the higher the ranking (or weight), the better the explanation they give.
A reference sample (for example, that of the accused) can then be compared to the list of weighted genotype combinations. If the genotypes in the reference sample are among the highest weighted combinations, a large likelihood ratio will be produced; if the genotypes in the reference sample are not among the listed combinations, the reference person will be excluded as a contributor.
However, a person of interest will not be excluded as a contributor even when his or her alleles are not detected at a marker in the observed profile if STRmix has posited the possibility of drop-out at that marker. STRmix may posit the presence of a dropped or ‘Q’ allele, and give a weighting to the genotype set that includes the Q allele. Because the Q allele could be any allele at the relevant marker, the person of interest will not be excluded. However, the weighting of the genotype set that includes the Q allele will be low, because STRmix assigns a negative number to account for drop-out, and the likelihood ratio decreases as a result.
STRmix determines the possibility or probability of drop-out by reference to peak height variability in the observed profile. It does not calculate drop-out empirically based on observed numbers of drop-outs. Instead, STRmix contains an internal model – known as ‘Model Maker’ – that produces a ‘variance value’ based on calibration data entered by the laboratory. Peak height variability in the observed profile is analysed having regard to the peak height variability experienced in the laboratory generally. The variance value is fed into the STRmix calculation to determine the probability of drop-out in the observed profile.[5]
[5]Dr Taylor gave evidence that there are a number of models that describe different DNA profile behaviours sitting within STRmix. There is a model for template amount and how that is related to peak heights; there is a model that describes degradation across the profile; and there is a model that describes the general amplification efficiency of each locus. There are some DNA profile behaviours that have not been modelled because they make up a minor part of the profile, for example, over-stutter. Some artefacts have not been modelled, for example, pull-up or dye blobs in the generation of electropherograms. They have to be removed manually by a typer. One of the essential components of using STRmix is that the typer has to look at the evidence profile first and decide whether or not it’s suitable for analysis in STRmix.
The probabilistic and fully continuous methodology used by STRmix can be contrasted with a classical binary methodology for DNA analysis. Binary methods are rule and threshold based, and a binary determination of the evidence of a match versus a non-match produces probabilities of only one and zero. Using the probabilistic model, the likelihood of the evidence of a match versus a non-match can have any value between zero and one, depending on the weighting. The evidence is evaluated on a continuous scale.
Previously, VPFSS used the ‘SPURS’ methodology to produce likelihood ratios. SPURS is a binary (match or non-match) system. It considers possible genotype combinations from alleles detected as present (based on the application of rules and thresholds) and weights each possible genotype combination equally, whereas STRmix generates numbers of possible genotype combinations and assigns different weightings to them, including where the drop-out of alleles is posited in one or more of the combinations.
An issue with using a probabilistic system like STRmix is that the results of no two analyses will be the same. Because the MCMC process is a random process, it calculates things slightly differently each time and it does not produce a ‘true’ likelihood ratio. This is relatively new to forensic science.[6] Previously, the calculation of a likelihood ratio would produce a single value, and if the analysis was repeated, the same value would result. With STRmix, it is a question of how much variability it to be expected and whether the actual variability is within expectations so as to enable the information that has been produced to be used.
Admissibility under 79 of the Evidence Act
[6]And something the Court was told scientists at VPFSS needed to get used to.
Section 76(1) of the Evidence Act contains the rule against opinion evidence. It provides that evidence of an opinion is not admissible to prove the existence of a fact about the existence of which the opinion is expressed. However, s 79(1) provides that if a person has specialised knowledge based on the person’s training, study or experience, the opinion rule does not apply to evidence of an opinion of that person that is wholly or substantially based on that specialised knowledge.
There are therefore two conditions for the admission of opinion evidence. First, the witness must have ‘specialised knowledge’ based on his or her training, study or experience. Secondly, the opinion given in evidence must be ‘wholly or substantially based on that knowledge’. It follows that it is necessary for the prosecution to establish both that its witnesses have specialised knowledge about the production and evaluation of the DNA evidence (including the generation of likelihood ratios) based on their training, study or experience and that the opinions that they give and upon which the prosecution seeks to rely are based upon that specialised knowledge.
The evidence of the likelihood ratios, as well as the underlying evidence comprising the profiles and case-notes, is proposed to be given by Ms Deborah Scott, who is a Forensic Officer and Senior Case Manager and Unit Leader in the Biological Examination Branch of the VPFSS. Ms Scott has made statements dated 4 October 2013 (‘the first statement’), 13 June 2014 (‘the second statement’) and 8 July 2014 (‘the third statement’). She also gave extensive oral evidence at the preliminary hearing, principally in response to questions in cross-examination.
Relevantly, the first statement sets out the likelihood ratios generated by STRmix for a large number of the crime scene samples, including the Items. In the second statement, Ms Scott comments on the reliability of interpreting DNA profiles when less than optimal amounts of DNA have been amplified.[7]
[7]The second statement also contains the calculated value of the likelihood ratio for Items 5A-1 and 5B-1. These Items were previously reported by reference to the default likelihood ratio of 120 million. The likelihood ratio given in the second statement is 29 billion in favour of the prosecution hypothesis.
The third statement explains an error in removing a peak height of less than 50 rfu in the EPG of Item 1-2 and another (for present purposes irrelevant) item and that the profiles have been modified to remove these peaks.
Ms Scott is an experienced forensic scientist. She has practised as a forensic scientist since May 2000, both in New South Wales and in Victoria. She has a Bachelor of Science degree with honours and a Master of Science in Forensic Science from the University of Strathclyde. She has successfully completed training requirements and is authorised by VPFSS in the interpretation of genetic typing of biological material and the statistical evaluation of DNA profiles.
Given Ms Scott’s qualifications and experience in the interpretation of genetic typing of biological material and the statistical interpretation of DNA profiles, I am satisfied that she has specialised knowledge about these matters, that is, knowledge that is outside that of persons who do not have her training, study and experience. She can explain what was done to produce the crime scene and reference sample DNA profiles and how it was done. She can interpret these profiles and evaluate them statistically, at least to the extent that she uses established statistical tools.
This, so far as I am aware, is not in dispute.
What appears to be in dispute is whether Ms Scott’s specialised knowledge about the statistical evaluation of DNA profiles enables her to give evidence about the likelihood ratios generated by the STRmix methodology specifically.
Ms Scott has received training in the use of STRmix and was able to broadly describe its methodology, including how it generates and weights possible genotype combinations, and produces likelihood ratios. She could and did give evidence of the validation studies carried out by VPFSS on the use of STRmix with both P+ and PP21 profiling kits. At the preliminary hearing, however, the Crown also adduced evidence about STRmix and its application in forensic case-work from two further witnesses: Dr Duncan Taylor, who developed STRmix in conjunction with colleagues in New Zealand; and Ms Lisa Federle, who is a Senior Case Manager within the Biological Examination Branch of VPFSS and who carried out the validation studies for STRmix within the VPFSS.
Ms Federle, when asked, said she considered herself to have sufficient expertise to address the validity and operation of the STRmix program and the reliability of the likelihood ratios. She said she was not necessarily an expert in the mathematics behind STRmix, but she knew about its application to forensic case work and the validation work conducted on its use in by VPFSS. She was involved in the validation studies carried out by VPFSS and had received training that authorised her to train other scientists to use STRmix. She said she was therefore able to give evidence about the mechanics of the program, why it works and why it is sufficient for use in forensic case-work.
However, STRmix involves the application of ‘black box’ technology, and its software is not open source. Neither Ms Scott nor Ms Federle could give evidence about the mathematical and statistical models underpinning STRmix, or the testing or validation of the software.
Evidence of this kind could, however, be given by Dr Taylor. At the preliminary hearing, Dr Taylor gave evidence about the development and operation of STRmix. Dr Taylor has qualifications and experience in the fields of biology and biological statistics. He has a Diploma of Biostatistics from Biostatistics Collaboration of Australia, lectures in DNA statistics in the faculty of Biological Sciences at Flinders University and is the current chair of the Australasian DNA Statistic Advisory Group, of which he has been a member since 2009. He is one of the authors, along with Jo-Anne Bright and John Buckleton, of the paper titled ‘The interpretation of single source and mixed DNA profiles’[8] (the ‘STRmix Research Paper’) that describes the STRmix methodology and sets out the mathematical and biological models that underpin it.
[8]Duncan Taylor, Jo-Anne Bright and John Buckleton, ‘The Interpretation of single source and mixed DNA profiles’ (2013) 7 Forensic Science International: Genetics 516.
The principle complaint made by the defence is not that Ms Scott, Ms Federle or Dr Taylor are unqualified to give evidence about the statistical evaluation of DNA profiles and likelihood ratios, but that the likelihood ratios relied on by the prosecution ought not to be admitted in to evidence because STRmix has not been shown to be a reliable tool for the statistical evaluation of DNA profiles. The defence argues that STRmix has not been properly validated for the use to which it has been put by VPFSS and it is not widely accepted by the forensic science community.
The defence witness, Ms Jane Taupin, gave evidence that there is no consensus in the literature that STRmix works or any description of how it works. While probabilistic statistical methodologies have been introduced in other jurisdictions, these other systems have not been evaluated. Neither the United States ‘Scientific Working Group on DNA Analysis Methods’ (‘SWGDAM’) nor the International Society for Forensic Genetics DNA Commission (‘ISFG’) has published guidelines for the use of a fully continuous probabilistic methodology. In particular, there are no recommendations as to how to calculate the probability of allele drop-out or more generally regarding the use of models based on peak height variation, like STRmix. Furthermore, the way in which STRmix determines the probability of drop-out is embedded in an internal system which is not open to evaluation. In Ms Taupin’s opinion, the inherent unreliability of peak heights in low level DNA means that STRmix should only be used where the DNA is not low-template DNA and there is less variance in peak heights. STRmix might be suitable for good quality DNA profiles, but it should not be used on poor quality profiles, particularly without recommendations for its use from either the ISFG or SWGDAM.
For the purpose of s 79 admissibility, the defence contends that the prosecution has not established that STRmix is a reliable body of knowledge in respect of which evidence based on ‘specialised knowledge’ can be given. This is because:
(a)The particular fully-continuous probabilistic methodology used or applied by STRmix is a discrete and new development in the area of DNA science; it involves a new or novel ‘area’ which does not constitute (at least not yet) a reliable body of knowledge; and
(c)Even if the Court came to the view that the fully continuous probabilistic methodology used and applied by STRmix was appropriate for use with optimum amounts of DNA, there is no reliable body of knowledge for its application in relation to complex mixtures and/or sub-optimal amounts of DNA.
For the DNA evidence to be admissible as expert opinion, the ‘specialised knowledge’ on which it is based must relate to a body of known facts or ideas. In Makita (Australia) Pty Limited v Sprowles,[9] Heydon JA held that if evidence tendered as expert opinion evidence is to be admissible, it must be agreed or demonstrated that it forms part of a ‘field’ of specialised knowledge.[10] In Honeysett v The Queen,[11] the High Court held that s 79(1) of the Evidence Act uses the term ‘knowledge’ in the sense defined in the Macquarie Dictionary, as an ‘acquaintance with facts, truths or principles, as from study or investigation’,[12] which the Court said was consistent with Blackmun J’s formulation in Daubert v Merrell Dow Pharmaceuticals Inc:[13]
The word ‘knowledge’ connotes more than subjective belief or unsupported speculation. … [It] applies to any body of known facts or to any body of ideas inferred from such facts or accepted as truths on good grounds.[14]
[9](2001) 52 NSWLR 705.
[10]Ibid, [85].
[11][2014] HCA 29.
[12]Ibid, [23].
[13]509 US 579 (1993) (‘Daubert’).
[14]Ibid, [590].
The knowledge of Ms Scott, Ms Federle and Dr Taylor about the statistical evaluation of DNA profiles is knowledge that involves acquaintance with a range of facts and principles concerning the nature of DNA profiles and their evaluation. Dr Taylor’s knowledge of statistics in general, and biological statistics in particular, is also based on known facts and recognised principles. These are plainly fields of specialised knowledge that satisfy the test in Daubert.
The defence argument, however, is based on what counsel described as the ‘probabilistic methodology used by STRmix’ being a new and discrete field of knowledge that remains untested and lacks acceptance in the forensic science community. According to the defence, the methodology used by STRmix is not ‘a body of knowledge or experience which is sufficiently organised or recognised to be accepted as a reliable body of knowledge or experience.’[15]
[15]Velevski v The Queen (2002) 187 ALR 233.
I do not accept that ‘the probabilistic methodology used by STRmix’ is either a ‘discrete’ or a ‘new’ field of knowledge as asserted by the defence. Based on the evidence at the preliminary hearing, I have concluded that STRmix and the methodology that it uses, is a development within an established and sophisticated field of knowledge concerned with the evaluation of DNA profiles. It is not a new or discrete body of knowledge. Rather, it is a new development in an existing body of knowledge.
Although the STRmix program is ‘new’ in that it has only recently been developed and put to forensic use, and only a small number of fully-continuous probabilistic systems like STRmix are in use internationally, the development and use of semi-continuous and fully-continuous probabilistic systems has been the subject of discussion and promotion in the forensic scientific community at the highest levels, particularly for the analysis of low-template and complex DNA profiles. Two examples were drawn to the Court’s attention during preliminary argument.
In 2012, the ISFG DNA Commission published a paper on the evaluation of DNA results that may include drop-out and/or drop-in using probabilistic methods.[16] The authors opine that ‘classical binary models’ are largely inferior to the probabilistic approach but observe that the adoption of probabilistic models has been inhibited by the complexity of concepts that are largely outside the experience of case-working forensic scientists, coupled with lack of suitable training opportunities. They confirm that the ISFG DNA Commission strongly supports new initiatives to remedy the problem and continue:
Some laboratories will wish to quickly adopt probabilistic methods ahead of the main-stream forensic community. This ISFG DNA Commission strongly supports this approach, since it will encourage others to follow. In this context, it should be noted that the approach described here still requires rigid assessment of the overall quality of a given DNA profile and its suitability for further analysis based on criteria described in the laboratory’s quality management guidelines.[17]
[16]Peter Gill et al, ‘DNA commission of the International Society of Forensic Genetics: Recommendations of the evaluation of STR typing results that may include drop-out and/or drop-in using probabilistic methods’, (2012) 6 Forensic Science International: Genetics 679.
[17]Ibid 680.
The ‘main-stream forensic community’ is therefore expected to follow the lead of laboratories using probabilistic methods for typing results that include drop-in and drop-out, that is, results that include stochastic effects and are associated with the analysis of low-template or compromised DNA.
Christopher Steele and David Balding from the Genetics Institute at University College, London, have recently published a paper[18] in which they review the main models and software for the statistical evaluation of DNA profile evidence, including STRmix, having outlined the general principles for the statistical evaluation such evidence and the difficulties associated with the analysis of low-template DNA. They state:
We hope that this review will help disseminate current best practices in statistical evaluation of [low template DNA] evidence, spur further developments and advertise to the forensic and wider community that robust methods for [low template DNA] evidence evaluation are now available. We have not verified the software described here, but, where available, we cite validation studies conducted by the authors of each program or package. In our view, courts are now able to avail themselves of the powerful new [low template DNA] profiling technologies, provided that as much care is take with the statistical analysis as is necessary for the collection, handling and analysis of samples.[19]
[18]Christopher Steele and David Balding, ‘Statistical Evaluation of Forensic DNA Profile Evidence’, (2014) 1 Annual Review of Statistics and its Application 361.
[19]Ibid 362.
Although neither article endorses STRmix specifically, or the fully-continuous methodology that it uses more generally, the articles place the fully-continuous probabilistic statistical methodology within the evolving field of knowledge concerned with the statistical evaluation of DNA evidence. This is not to say, however, that there may not be problems associated with fully-continuous systems or that particular DNA scientists might well prefer other models or methodologies.
In the STRmix Research Paper, Dr Taylor and his co-authors also describe the ‘forward movement’ in the late 2000’s to develop and use continuous and semi-continuous statistical models, but acknowledge that progress is still partial, with only a few laboratories worldwide implementing or investigating fully continuous methods. Barriers to the adoption of the methodology include the initial lack of validated software, the fear of complexity, the implications of using ‘black box’ technology and the perceived costs. Nonetheless, validated and commercially available software, known as ‘TrueAllele’, has proceeded through the court processes in the United Kingdom and the United States of America.
In response to the proposition that open source software is highly desirable in a court environment, Dr Taylor said that open source software is desirable but not essential, as long as other measures are taken, such as reporting the models or the mathematics underlying the methodology and validation information is provided. He agreed that few systematic comparisons of the performance of the different programs had been published and said that such comparisons were a high priority now that the field was beginning to mature. He was aware from the results available that the different programs often generated different results when comparing the same hypotheses and agreed that the most important difference was typically between the continuous and discrete algorithms, as the continuous algorithms exploit peak height information which gives different results.
For her part, Ms Federle described STRmix as an ‘incremental change’ to VPFSS’s method for calculating likelihood ratios. She said STRmix modified but was based on the same principles as SPURS. In response to the criticism that STRmix listed countless numbers of genotypes as possible contributors, Ms Federle said that it had ever been thus: there had always been a list of possibilities that could explain the evidence, particularly where there are a number of contributors. STRmix is different in that it weights the possibilities, whereas in the past, the statistical methodology had not been able to weight possibilities differently: if something registered as a possibility, it had to be considered to be just as likely a possibility as something that could potentially be a really good explanation for the profile.
It was put to Ms Federle that using weightings and abandoning thresholds had not been robustly debated in the broader scientific forensic DNA community. She responded that there had been a lot of literature about the different methods and, in the past, there had been recommendations to move to this sort of method, but there had not been the statistical programs to enable a method to be implemented. The continuous method was advocated for dealing with partial profiles and low level profiles, and the debate had proceeded on the basis that this was the way forward.
In my view, this evidence places STRmix and its fully-continuous probabilistic methodology squarely within the field of statistical DNA evaluation, albeit possibly at its leading edge. It has a credible scientific and mathematical basis. However, that is not to say that the results that it produces are inherently reliable.[20] It may not produce results that are as reliable for the purposes of forensic case-work as binary or semi-continuous methods. Its method for determining the possibility or probability of drop-out based on peak height variability may be open to criticism. Its use on compromised profiles may be questionable. But those are matters for competing expert opinion, providing, of course, that STRmix is amenable to scrutiny and independent testing.
[20]Insofar as the defence argument is based on a challenge to the reliability of results generated by STRmix, there is a real question as to whether the reliability of the results is relevant to the admissibility of the DNA evidence under s 79 of the Evidence Act. In R v Tang [2006] NSWCCA 167, [137] the New South Wales Court of Criminal Appeal held that the focus of attention for the purposes of s 79(1) must be on the words ‘specialised knowledge’ and not on ‘an extraneous idea’ such as ‘reliability’.
It is trite that expert opinion must be amenable to being tested in court. In the oft-quoted passage in Makita[21] in which his Honour set out compendiously the requirements of s 79, Heydon JA stressed that the opinion of an expert requires demonstration or examination of the scientific or other intellectual basis of the conclusions reached. The expert’s evidence must explain how the field of ‘specialised knowledge’ in which the witness is expert by reason of ‘training, study or experience’ and on which the opinion is ‘wholly or substantially based’, applies to the facts assumed or observed so as to produce the opinion propounded.
[21](2001) 52 NSWLR 705, [85].
I observe in this context that while STRmix produces quite extensive reports (referred to as ‘output data’) that permit scrutiny of many of its operations, Dr Taylor gave evidence that this output data is incomplete for the reason that there are ‘hundreds of thousands of different data points and calculations’ and it is impossible to include them all in the output data due to their sheer number.
The STRmix Research Paper acknowledges that without an understanding of the underlying mathematics, there is a risk that systems become ‘black boxes’ the workings of which are not understood by the users and that presentation of any statistical analysis in court becomes problematic.
To this end, the STRmix Research Paper describes the mathematics underpinning STRmix along with the practical implementation of the mathematics and the means of calculating the likelihood ratio. It is an attempt by the developers of STRmix to expose the internal workings of STRmix by setting out the mathematical and statistical models on which it is based.[22]
[22]The authors state that they report ‘a synergy of mathematics and biological modelling specifically designed to ameliorate the black box aspect of continuous models’.
The STRmix Research Paper also deals with the issues of reproducibility and describes the validation experiments carried out by the authors. The results of the validation experiments are set out in the appendices. According to the authors, inspection of the results of these experiments suggests that the likelihood ratio assigned by the method is fair and reasonable.
In his evidence, Dr Taylor agreed that it is not feasible to validate a process for producing a likelihood ratio in the way that one might validate a procedure for measuring a physical quantity, because a likelihood ratio has no true value: it expresses uncertainty about an unknown event and depends on modelling assumptions that cannot be expressly verified. However, he said that progress can be made in evaluating the validity and performance of software, and that the courts need these kinds of evaluations to have confidence in the results generated by software-based forensic analysis.
Dr Taylor gave evidence that validation can be done at several levels. The STRmix Research Paper addresses the question of conceptual validation. Development validation of the software involves ascertaining whether the software does what the Research Paper says it does. Then there is laboratory-based validation, which is verification that the software is fit for purpose in the hands of scientists. That involves examination of interpretations of mixtures of known contributors and comparison against other methods and/or human judgment. The STRmix Research Paper details validation of both kinds, including validation studies using known contributors and comparison against other methods and human judgment.
In addition, the STRmix User’s Manual details the verification ‘by hand’ of a number of STRmix functions, including expected allele and stutter heights and expected peak heights of drop or ‘Q’ alleles.[23]
[23]The verifications were carried out on:
Expected allele and stutter heights given mass parameters
Expected peak heights of drop or ‘Q’ alleles given mass parameters
Probabilities given expected and observed peak heights
Locus specific amplification efficiency calculations
Summation of probabilities for each allele in a locus and across a profile
Summation of probabilities across multiple replicate profiles
LR point estimates where there are no assumed contributors
LR point estimates with varying theta values
LR point estimates for scenarios with assumed contributors
LR stratified point estimates
LR HPD interval values
Database search functionality.
The STRmix V1.08 User’s Manual records that the comparison of expected heights, probability and LR values was conducted in Excel or by comparison to results generated in the rHPD package written by Associate Professor James Curran in R . A summary of the development validation results is discussed in turn for each comparison undertaken. This is only a small subset of all the testing activities that were undertaken during the eight month STRmix development project.
VPFSS has carried out validation studies using known contributors and P+ and PP21 mixtures. The validation studies were tendered in court and Ms Federle and Ms Scott gave evidence about them at the preliminary hearing.
The PP21 study is dated April 2013 and tests with two person and three person mixtures. Six known individuals were used in different pairings and at different ratios. A total of 16 two person mixtures were amplified, producing 161 deconvolutions and 307 likelihood ratios. Ten three person mixtures were amplified, producing 124 deconvolutions and 371 likelihood ratios. In addition, three mocked partial profiles were analysed a number of times using STRmix.
The P+ study is dated May 2013. Again, tests were conducted with two person and three person mixtures and six known individuals were used in different pairings and at different ratios. A total of 12 two person mixtures were amplified, 60 deconvolutions performed and 120 likelihood ratios calculated. For the three person mixture study, 11 mixtures were amplified, 60 deconvolutions performed and 180 likelihood ratios calculated.
Both studies contained a variety of other testing and calibration, including the pilot studies for peak height variance using Model Maker required for STRmix.
All of these studies are open to scrutiny.[24]
[24]The STRmix User’s Manual was filed in accordance with an order of the Court.
It is the defence position that the validation studies are inadequate because they did not test a sufficient number of samples and did not use the ratios that are in issue in this case.
In her evidence about these studies, Ms Federle described them as ‘large’ in the sense that each of the different mixture ratios was repeated over and over again to create a lot of data. The ratios used were intended to cover a broad range of different scenarios. According to Ms Federle, there was nothing in the studies to suggest that other mixture ratios would behave any differently from the study ratios. She said that none threw up any issues, so it was possible to extrapolate from them in respect of all sorts of mixtures.
For his part, Dr Taylor commented on the VPFSS testing with PP21 mixtures, observing that there was a reasonably substantial number of calculations and that contributors were combined in different proportions and in different amounts. When it was put to Dr Taylor that no validation study had been done for the particular ratios in the present case, Dr Taylor said that it was possible to extrapolate based on the kinds of ratios that were tested. The aim is to vary the total amount of DNA that goes into the mixtures and the proportions for each contributor to show mixtures in a range of configurations – a major-minor, two equal contributors, two equal contributors of very low amounts of DNA and so forth – to permit extrapolation from these results and to see that STRmix is performing as expected or as intended, based on these mixtures. He gave evidence that he himself had recently published a study in which he looked at two, three and four person mixtures at high and low levels and in different concentrations, and he concluded that STRmix was behaving as would be expected and could therefore be used on evidence profiles.
In cross-examination, Ms Federle was taken to the STRmix validation study that tested with P+ mixtures. She agreed in the two person mixtures, when both of the contributors contributed low levels of DNA, the likelihood ratio was determined to be less than one at four of markers, giving more support for the actual contributor not contributing to the DNA profile at four of the nine markers. It was put that the study did not validate the methodology. Ms Federle responded that this did not prove that the contributor was a non-contributor, but highlighted the issues with low-template DNA. According to Ms Federle, that sort of result was to be expected, given the input amount of the contributors to the profile: when there are really low contributions, STRmix will not calculate a likelihood ratio indicating a contribution that is contrary to the evidence. It will therefore give a likelihood ratio that favours the alternative view. That, says Ms Federle, is a totally reasonable explanation for the evidence.
Ms Federle denied that this showed that the validation group was too small and said it was just one of the effects of low-template DNA. It showed that STRmix was doing the correct thing and not overstating the evidence.
By contrast, Ms Taupin viewed the production of ‘false negatives’ of this kind by STRmix as an indication that it was unreliable.
In my view, this will be an issue for the experts at trial. I do not see it as showing that STRmix lacks an objective foundation or consider that it detracts from the cogency or reliability of the validation studies.
On the basis of the evidence at the preliminary hearing, I have concluded that knowledge concerning the statistical evaluation of DNA profiles, including by a fully continuous probabilistic system such as STRmix, is ‘specialised knowledge’ that could support opinion evidence in the form of likelihood ratios generated by STRmix. The DNA evidence proposed to be given by Ms Scott is admissible as opinion evidence under s 79(1) of the Evidence Act having regard to the supporting evidence of Ms Federle and Dr Taylor.
Section 137
Section 137 of the Evidence Act provides that in a criminal proceeding, the Court must refuse to admit evidence adduced by the prosecutor if its probative value is outweighed by the danger of unfair prejudice to the defendant.
The words ‘probative value’ are defined in the dictionary to mean the extent to which the evidence could rationally affect the assessment of the probability of the existence of a fact in issue. ‘Unfair prejudice’ is not defined.
In Dupas v The Queen,[25] the Court of Appeal considered the application of s 137. It held that the application of that section involves a series of steps: first, an assessment of the probative value of the evidence; second, an assessment of the danger of unfair prejudice to the defendant; and, third, a weighing of the probative value of the evidence with any danger of such prejudice. If the latter outweighs the former, the court must refuse to admit the evidence. In assessing the probative value of the evidence, it is necessary to assume its truthfulness, although not its reliability. In considering ‘unfair prejudice’, the issue is not that the evidence may lead to conviction; rather the issue is whether the evidence may be misused.[26] The risk is that the jury may be distracted from its proper role and give particular evidence more weight than it deserves.
[25](2012) 218 A Crim R 507 (‘Dupas’).
[26]‘Evidence is not unfairly prejudicial merely because it makes it more likely that the defendant will be convicted’; Papakosmas v The Queen (1999) 196 CLR 297, 325 (McHugh J).
The Court of Appeal held that in order to determine the capacity of the evidence to rationally effect the determination of a fact in issue (that is, to be probative), the judge is required to make some assessment of the weight that the jury could, acting reasonably, give to that evidence. Where it is contended that the quality or frailties of the evidence would result in the jury attaching more weight to the evidence than it deserved, the trial judge is obliged to assess the extent of the risk. That does not require the trial judge to anticipate the weight that the jury would or will attach to it. A judge is obliged to assess what probative value the jury could assign to the evidence, against which must be balanced the risk that the jury will give the evidence disproportionate weight.[27]
[27]Dupas, [63].
In respect of expert evidence in particular, the Court observed that to the extent that the expert does not satisfactorily demonstrate that the opinion is based upon specialised knowledge in which the witness is expert, or does not sufficiently identify the facts upon which the opinion is wholly or substantially based, the evidence so far as it is admissible will be of diminished weight. Where the sufficiency of such matters is in doubt, the reliability of the opinion is brought into question, with the risk that the expert evidence will be given disproportionate weight by the tribunal of fact.[28] In this context, the Court referred to the statement of Hunt J in R v Elliott[29] as to the trial judge’s duty:
If scientific testing in the particular case is unreliable or if it has a tendency to produce a misleading or confusing impression to the jury or if the weight to be afforded to the result is so minimal as to preclude the jury being satisfied beyond reasonable doubt that the Crown has established the fact which it seeks to prove, then clearly I have a duty to exclude it from the jury – whether it is a result of ruling that evidence is inadmissible or whether or it is excluded in the exercise of my discretion.[30]
[28]Ibid, [127], referring to Makita (Australia) Pty Ltd v Sprowles (2001) 52 NSWLR 705, 744.
[29]Unreported, Supreme Court, New South Wales, number 70154 of 1989, 6 April 1990.
[30]Ibid.
In this case, the defence submits the probative value of the DNA evidence is weak and that the relevant reliability issues are as follows:
(a) The real risk of contamination of the samples either at the crime scene or at the laboratory;
(b) The lack of consensus in the broad DNA scientific community as to the reliability of STRmix, the lack of comparison with other statistical methodologies or independent review or assessment, and the paucity of validation studies, particularly given the low level amounts of DNA in issue;
(c) The fact that validation studies by the amplification kit manufacturer do not support its use for low copy DNA;
(d) the specific problems Ms Taupin identified with the VPFSS analysis of each of the Items.
These are matters, so the defence contends, that are relevant to assessing the weight that the jury could give to the DNA evidence.
Ms Taupin, called by the defence as an expert at the preliminary hearing, produced a report and gave evidence of her concerns about the way in which VPFSS had produced the DNA profiles and generated the likelihood ratios. Her concerns were principally directed to the reliability of STRmix results, specifically because of:
(a) the lack of guidelines or recommendations by bodies such as ISFG or SWGDAM for the use of methodologies such as STRmix in forensic case-work, particularly for the analysis of low-template DNA;
(b) the lack of proper validation of STRmix;
(c) the failure of STRmix to provide estimates of the probability of drop-out based on validated studies; and
(d) the lack of risk assessment in the application of STRmix. According to Ms Taupin, no statistical analysis should be carried out on a poor quality profile.
Ms Taupin pointed generally to complexities associated with the evaluation of evidential weight for low-template and unresolved mixture DNA profiles. Her evidence was that, as yet, there is no consensus within the forensic biology community as to the interpretation strategy for these types of DNA profiles. The scientific literature is constantly evolving: some approaches make use of the presence or absence of peaks in the profile, or use peak heights in a semi-quantitative fashion; other approaches seek to develop fully quantitative methods based on observed peak heights. Although several methods have been developed, they each suffer from limitations of various kinds and none of the methods has been universally acknowledged as ‘gold standard’.
In addition, Ms Taupin expressed concern about the quality of the profiles that were analysed using STRmix, and the use of peaks that are below the stochastic threshold. She advocated re-amplification or replication in many cases.
It is common ground that the fully-continuous probabilistic methodology used by STRmix is relatively new and is not, as yet, being widely used internationally for forensic case-work; that, in this sense, there is no wide-spread ‘acceptance’ of STRmix in the forensic biology community; that there are, as yet, no internationally recognised protocols or recommendations for the use of fully-continuous probabilistic systems like STRmix; and that there is, as yet, little if any literature reviewing and comparing fully-continuous probabilistic systems like STRmix.
I accept that this is the case. I accept also that the lack of review and comparison with other systems and the lack of recognised protocols or recommendations are matters that go, potentially, to the reliability of the likelihood ratios generated by STRmix.
However, this, like most of the reliability matters raised by the defence, involves a contest between the expert evidence to be called by the prosecution (the DNA evidence and the evidence about the operation and application of STRmix to be given by Ms Scott, Ms Federle and Dr Taylor) and the evidence to be called by the defence (the evidence to be given by Ms Taupin (and, possibly, an expert qualified in biological statistics) about the short-comings of probabilistic methodologies and the need to understand their limitations, as well as specific criticisms of the profiling and analysis of the samples carried out by VPFSS).
Insofar as an assessment of the probative value of the DNA evidence involves deciding whether to prefer the evidence of Ms Taupin to that of Ms Scott, Ms Federle and/or Dr Taylor, this is a matter for the jury. In R v DG,[31] Court of Appeal held that the weight to be accorded to expert opinions is usually a matter ‘entirely for the jury’. The Court said:
It would not be appropriate, in general, for a trial judge, called upon to make an evidentiary ruling, to resolve a conflict between experts by preferring one opinion over another.[32]
[31](2010) 28 VR 127.
[32]Ibid [59].
I do not propose to engage in an analysis of the competing claims in order to decide which evidence I prefer. As the Court of Appeal held in Dupas, a trial judge is obliged to assess what probative value a jury acting reasonably could assign to the evidence. In this case, the issue is the weight that could be assigned to the DNA evidence if the jury accepted the expert evidence of Ms Scott, Ms Federle and Dr Taylor.
The prosecution witnesses acknowledge that there are limitations in the STRmix methodology, a number of which are detailed in the STRmix Research Paper. The STRmix Research Paper describes limitations arising in some cases from conscious choice and in others from the current state of the model development. They include the danger that a large artefact is allowed through the manual review of the electropherogram and what is described as a ‘sub-optimal’ stutter model. It is also recognised that it is difficult to test continuous models for the accuracy of the likelihood ratio produced because the correct answer is unknown and in many cases unknowable.
Importantly, so far as I can tell, none of the prosecution witnesses contend that the forensic scientist can simply ignore the quality of the evidentiary profile and feed any kind of profile into STRmix to produce a reliable likelihood ratio. Although STRmix purports to be specifically suited for the analysis of low-template and partial DNA profiles, there is a need to carefully consider the profile and whether it is suitable for statistical evaluation for use in legal proceedings.
I have found the STRmix methodology to be a development in a recognised field of knowledge concerned with the statistical evaluation of DNA profiles. It is not subjective or speculative or otherwise to be dismissed as lacking an objective basis. In my view, based on the opinions of Dr Taylor, Ms Federle and Ms Scott the limitations identified do not substantially erode the probative worth of the DNA evidence.
However, it is clear that there is scope for competing expert evidence about the reliability of the STRmix methodology. What a lack of international take-up or independent review and assessment of the STRmix methodology means for the weight that should be given to the DNA evidence will be a matter for the jury, based on differing expert evidence on this issue. Likewise, the suitability of STRmix (or any system based on peak height modelling) for the analysis of low-template DNA will be the subject of disagreement between experts upon which the jury will be called to adjudicate. As to the specific problems identified by Ms Taupin in the VPFSS analysis of the Items, notably whether the profile for Item 1-3 shows three or four contributors and whether the accused is excluded as a contributor to Item 4-1 because the PP21 profile does not show a Y allele at the amelogenin marker, these again are matters that can and should be resolved by a jury hearing the expert evidence and deciding which evidence is to be preferred.
The defence further contends that the probative value of the DNA evidence is compromised by its possible contamination at the crime scene and/or at the laboratory. As I understand the argument, it is that DNA from the cigarette butts (Items 5A-1 and/or 5B-1) has possibly contaminated Items 1-1, 1-2, 1-3 and 4-1. Not all of the Items are alleged to suffer from possible contamination and there is no suggestion (again, so far as I can tell) that any of the Items were contaminated by DNA that was already held in the laboratory.
Senior Constable Papeneux, the crime scene investigator, gave evidence at the preliminary hearing about the order in which he collected the exhibits from the house and surrounds, starting with the cigarette butts on the neighbouring property (Items 5A-1 and 5B-1). He told the Court that he picked up the exhibits one by one and placed them in small bags. The small bags were then placed together into a larger bag. According to the defence, Mr Papaneux first touched the cigarette butts and the same gloved hand then had contact with the outside of the small bags so the DNA from the cigarette butt(s) could have been transferred onto the outside of the small bags. From there, it could have been transferred onto the outside of the other small bags containing Items 1-1, 1-2, 1-3 and 4-1. Then, according to the defence, at VPFSS a number of exhibits were taken out of the small bags and sampled on the same bench within hours of each other. If these two opportunities for contamination are combined, there is, so the defence contends, a real possibility of contamination of the exhibits that were then sampled for DNA extraction, amplification and analysis.
In addition, the defence points out (correctly) that contamination protocols at VPFSS were not as strict in 2007 as they are now.
In my view, however, the defence has not demonstrated that there is a real chance that any of the Items were contaminated either before or during the testing process, although I accept that contamination protocols at VPFSS have been improved since 2007.
As to the risk of contamination at the crime scene or before analysis by VPFSS, the evidence was that Senior Constable Papeneux bagged and tagged the relevant exhibits separately, that he did so wearing gloves at all times and that the standard practice was to change gloves after the collection of each exhibit. The bags themselves were stapled closed and there is no evidence that any police officer opened the individual bags before they were received at VPFSS. Even if there was a chance that DNA from one of the cigarette butts was left on the outside of one of the small bags and that bag came into contact with the outside of another small bag when placed in the larger bag, there is no explanation for how the DNA would come to be on the exhibits themselves.
As to the risk of contamination during the analysis of the items at VPFSS, the prosecution has prepared a detailed chronology showing when the different exhibits were sampled. This demonstrates that a transfer of DNA from Items 5A and 5B to Item 4 is unlikely because Items 2 and 3 were sampled in between and DNA results on those Items excluded the accused. Item 1 (the blindfold) was not sampled until the following day.
In addition, there was quite detailed evidence from a number of scientists at VPFSS as to the contamination controls that were in place in 2007. Relevantly, there was evidence of the use of blotting paper on the bench that was changed between samples after the bench had been cleaned, the wearing of face masks and gloves that are regularly changed. All benches and equipment were cleaned before and after use with hypochlorite and UV light, and a reference sample was always analysed after an evidentiary sample as a control mechanism to prevent contamination between samples. There was no touching of samples within the DNA science branch, and officers were trained not to deviate from standard procedures.
On the contamination scenario advanced by the defence generally, there is no explanation for the contamination of only some of the exhibits. The evidence from Senior Constable Papeneux was that he numbered the relevant exhibits in chronological order and seized them in reverse order. However, only Items 1, 4, 7 and 8 (7 and 8 being re-numbered 5A and 5B by VPFSS) provide evidence of a DNA match with the accused. By contrast, the accused is excluded as a contributor to the single source profile in Item 3. It is therefore unlikely that contamination has occurred during the seizure phase, given that some exhibits show no possible contamination at all.
Moreover, the DNA quantification values for the relevant Items do not support contamination of the kind described, whether it is alleged to have occurred in the field or in the laboratory. Ms Scott gave evidence in relation to possible secondary transfer from the cigarette butts (Items 5A and 5B) to Item 1 (the blindfold). She explained that with each single transfer step, less and less DNA would be transferred. However, the quantitation value for the DNA located on each of the cigarette butts (0.332ng and 0.337ng for Items 5A-1 and 5B-1 respectively) is lower than the quantification value for the DNA component matching that of the accused in relation to the blindfold (0.638ng for Item 1-1 and 0.899ng for Item 1-3). Ms Scott expressed the opinion that it was ‘extremely unlikely’ and ‘almost impossible’ for any DNA to have been transferred from the cigarette butt to the blindfold.
In my view, while there may be a theoretical possibility of contamination, it is not a real possibility that could erode the probative value of the DNA evidence. Ultimately, however, this will be a matter for the jury to consider.
In assessing the probative value of the DNA evidence, I take into account that Items 1-1, 1-2, 1-3 and 4-1 (the samples from exhibits found in the house) produced complex profiles of two or more contributors and that Items 1-1 and 4-1 are only partial profiles. Item 4-1 contains a sub-optimal amount of DNA, increasing the chances of random effects in the profile. The analysis of profiles of this kind, as Ms Taupin emphasised in her evidence, is complicated and results are open to challenge.
However, the same cannot be said about the Items 5A-1 and 5B-1, which are single source profiles with good levels of DNA. The majority of peaks are of optimal height and there are matches with the accused at all ten loci. Items 5A-1 and 5B-1 produced likelihood ratios using SPURs of 1 in 30 billion, as opposed to the STRmix likelihood ratios of 1 in 29 billion.
Although Ms Taupin made some criticism of the quality of these profiles, I do not see any argument at all for excluding Items 5A-1 and 5B-1, which would leave it open for a jury to find that the accused was present at the fence line on the neighbouring property. From that location, a person is able to closely view the complainant’s house. The weight to be given to the DNA evidence from the exhibits found inside the complainant’s house has to be assessed on that basis.
In my view, the DNA evidence viewed as a whole is highly probative. It may be used by a jury to put the accused both inside and outside the house on the night in question. This is so, notwithstanding only small amounts of DNA matching that of the accused were found on the relevant items inside the house, and that other people also contributed to the DNA found on these items. The limitations in the STRmix methodology acknowledged by the prosecution witnesses must have some effect on the quality of the DNA evidence. However, I am not persuaded that they erode its probative value to any significant degree. Whilst the amounts of DNA may be small in some cases, the fact that DNA matching the accused’s was found on a number of items both inside and outside the house in my view fortifies the overall probative value of the DNA evidence, which I assess to be high.
It remains to assess the worth of the DNA evidence against the unfair prejudice considerations.
Evidence is not unfairly prejudicial merely because it makes it more likely that the defendant will be convicted.[33] The danger of unfair prejudice to the accused must involve a real (and not just a hypothetical) risk that the DNA evidence will be used by the jury in some unfair way. Unfairness may arise if there is a danger that the jury will adopt an illegitimate form of reasoning or misjudge the weight to be given to particular evidence. An inability to test the reliability of the evidence may carry with it the danger of such misjudgement.[34]
[33]Papakosmas v The Queen (1999) 196 CLR 297, 325 (McHugh J).
[34]Dupas [175].
In this case, the danger of unfair prejudice is said to arise from a particular issue identified by Ms Taupin in the STRmix analysis of Item 1-2, although it has wider consequences as it is a product of the way in which STRmix works generally. Ms Taupin identified, having closely examined the STRmix case-notes, that at two of the ten markers the probability of the evidence given the prosecution hypothesis was very very low, yet the likelihood ratios for the markers favoured the prosecution hypothesis. Ms Taupin pointed out that this means that STRmix produces likelihood ratios strongly favouring the prosecution hypothesis in circumstances where there is only very weak evidence to support that hypothesis. That, in combination with the very high likelihood ratios generated by STRmix, is said to be unfairly prejudicial to the accused and not something that should be allowed in a criminal trial.
I am not persuaded that this constitutes unfair prejudice. The likelihood ratios for the two markers in question favoured the prosecution hypothesis simply because the probability of the evidence given the prosecution hypothesis was higher than the probability of the evidence given the alternative hypothesis. The probability of the evidence given the alternative hypothesis was also very very low, even though it allowed for any other ramdomly selected member of the Australian Caucasian population to have contributed the DNA found on the exhibit rather than the accused. Furthermore, the likelihood ratios for individual markers are not the likelihood ratios that will be considered by the jury. The likelihood ratios sought to be relied on by the prosecution are the product of the likelihood ratios at each of the markers for the relevant Item.
In my view, this feature of STRmix does not give rise to unfair prejudice when considered alone or in the context of the DNA evidence as a whole. It is a simply a function of the way in which likelihood ratios are calculated by STRmix.
However, I have considered generally the effect of the very high likelihood ratios generated by STRmix (especially when analysing profiles using 21 markers) and the difficulty of explaining to a group of laypersons how STRmix works so as to enable them to understand the competing evidence. The STRmix Research Paper acknowledges the complexity of the model and that there has been a ‘realistic’ fear of complexity that has hindered the adoption of continuous probabilistic methods in the past.
On the other hand, the STRmix Research Paper also records that results produced by a similar system, TrueAllele, have been reported in over 75 criminal cases in the United Kingdom and the United States.
I am persuaded, based on the evidence given at the preliminary hearing, that the move away from binary systems to the more complex semi-continuous and continuous models is inexorable, and that juries will increasingly be required to consider and understand such evidence in the future. It is also the case that understanding what a likelihood ratio is may present real challenges for the layperson, whether it is produced using conventional binary methods or the relatively novel and more sophisticated probabilistic methods. Likelihood ratios are complex creatures that are often expressed using very large numbers – numbers which frequently far exceed the population of the town or city in which the crime took place[35]. Yet juries have come to terms with likelihood ratios and they are a regular feature in criminal trials.
[35]With which they in fact bear no relation.
In considering all of these matters, I have had regard to the fact that the DNA evidence forms the bulk of the prosecution’s case against the accused. Although this is not a ‘DNA evidence only’ case as there is other evidence linking the accused to the crimes,[36] the prosecution concedes that the other evidence would not be sufficient to convict the accused. The DNA evidence is therefore ‘front and centre’ in this case, and I must carefully consider whether, to use the words of Hunt J in R v Elliott,[37] it creates a ‘misleading or confusing impression’. In other words, I must consider whether there is a danger that jurors will be bamboozled by the evidence and simply seize upon the large numbers in the likelihood ratios as conclusive of guilt.
[36]The description of the offender provided by the complainant matches the accused in terms of physique, ethnicity and age; he lived within 400 m the crime scene; he smoked the same cigarettes as those found; he admitted that he had used cable ties like the ones found; he admitted to owning a dark blue wind-cheater with fabric like the fabric used for the blindfold.
[37]Unreported, Supreme Court, New South Wales, number 70154 of 1989, 6 April 1990.
The mere fact that expert evidence deals with difficult and highly technical subject-matter does not, in itself, constitute unfair prejudice to the accused. However, if the DNA evidence were simply too complex for a jury to comprehend beyond its conclusions (the very large likelihood ratios), such incomprehensible complexity might amount to a very real prejudice. It is important that the jury understand the testing of the conclusions in cross-examination so that they do not misjudge the weight to be given to the likelihood ratios or adopt an illegitimate form of reasoning regarding their significance based on the appearance of scientific credibility.
In my view, the danger of unfair prejudice of this kind will not arise if the DNA evidence is led logically and sequentially by the prosecution and the jury is given proper assistance. It may be necessary to resort to visual and other aids, and Dr Taylor (or his equivalent) will be required to explain in very clear terms the mathematical and biological models upon which the STRmix methodology is based, and answer questions arising from its application in this particular case, such as (potentially) how weightings for particular genotype sets were arrived at and how allele frequencies were identified and used in the likelihood ratios.
Furthermore, any perceived prejudice that might arise from the complexity of the evidence (and, if relevant, the limited nature of the output data) could be addressed by a strong direction that the jury must not act on the STRmix conclusions unless they are wholly satisfied that they are soundly based in the evidence that they have heard and understood.
I have therefore concluded that the Court is not required to exclude the DNA evidence under s 137 of the Evidence Act.
5
4
0