R v Tuite

[2015] VSC 303

IN THE SUPREME COURT OF VICTORIA Not Restricted

AT MELBOURNE

CRIMINAL DIVISION

S CR 2014 0007

DIRECTOR OF PUBLIC PROSECUTIONS

v

CLINTON JAMES TUITE

---

JUDGE:	EMERTON J
WHERE HELD:	Melbourne
DATE OF HEARING:	6, 20 May and 4 June 2015
DATE OF RULING:	25 June 2015
CASE MAY BE CITED AS:	R v Tuite
MEDIUM NEUTRAL CITATION:	[2015] VSC 303

---

EVIDENCE – Expert Evidence – Whether expert witness has expertise to give opinion evidence on reliability of statistical programme for the interpretation of DNA profiles.

---

APPEARANCES:	Counsel	Solicitors
For the Crown	Dr N Rogers SC	Office of Public Prosecutions Victoria
For the Accused	Mr J Desmond	Doogue O’Brien George

HER HONOUR:

1. The defence has filed an expert forensic biology report by Jane Moira Taupin, a forensic science consultant.  Ms Taupin’s expert report considers the profiling and analysis by the Victoria Police Forensic Science Service (‘VPFSS’) of low-template DNA found on items removed from the crime scene.[1]

   [1]I have previously ruled on the admissibility of the DNA evidence in question in DPP v Tuite [2014] VSC 662. See also [Tuite v The Queen [2015] VSCA 148]. I refer to the description of the items and the evidence derived from them in my ruling, which I will not now repeat.

1. In her report, Ms Taupin has expressed a number of opinions about VPFSS’s analysis of the DNA profiles in question to generate likelihood ratios, which the Crown submits she does not have the expertise to give.

1. Ms Taupin is a highly qualified and experienced biological scientist who has worked as a forensic scientist for over 25 years in both Australia and England, and has been an expert witness in over 140 trials, including over 40 murder trials.  She has reported on likelihood ratios in the statistical evaluation of forensic DNA profiling evidence in Victoria since 1999 and since 2003 in the United Kingdom.

1. In addition, Ms Taupin has written training programs and methods for accreditation for forensic laboratories in the United Kingdom and Australia, delivered training workshops internationally and has been an internal laboratory auditor for many years in Australia.  In 2013, she authored a book ‘An Introduction to DNA Profiling for Criminal Justice Professionals’ which discussed, among other things, various statistical approaches to DNA analysis, with an emphasis on likelihood ratios, as well as the limitations involved in analysing some sub-optimal DNA profiles.  As recently as October 2014, Ms Taupin presented on likelihood ratios in sub-optimal DNA profiles at the International Association of Forensic Sciences Triennial Meeting in Seoul, South Korea.

1. Ms Taupin’s formal qualifications include a Bachelor of Science with Honours in chemistry from the University of Melbourne, a post-graduate Diploma in Criminology and a Masters Degree in Criminology.  She is a member of the American Academy of Forensic Sciences and formerly a member of the scientific working group for materials in the FBI in Washington DC.

1. As explained in my previous ruling, VPFSS interpreted the DNA profiles found on the items using a relatively new statistical programme, STRmix.  STRmix is described as a ‘probabilistic’ programme which represents a fundamental departure from the binary (match versus non-match) methodologies previously used by VPFSS to generate likelihood ratios.  STRmix is based on continuous modelling of peak heights and purports, through this modelling, to account for many of the random or stochastic effects found in a low-template DNA profiles.  STRmix does not apply thresholds to exclude information found in the profile but purports to make use of all the information in the profile.  In addition, it purports to take into account peaks that are not detected as present in the DNA profile by positing the possibility that alleles have ‘dropped-out’ of the observed profile and it includes the dropped-out alleles in the genotype combinations that provide a possible explanation for the observed profile.

1. STRmix is apparently not the only methodology that posits the possibility or probability of alleles having dropped out of the observed profile.  However, STRmix determines the probability of drop-out by reference to peak height variability in the observed profile and not on the basis of observed drop-out.  It contains an internal model – ‘Model Maker’ – that produces a ‘variance value’ based on calibration data entered by the laboratory.  The variance value is fed into the STRmix calculation to determine the probability of drop-out in the observed profile.

1. In my earlier ruling, I described the core of Ms Taupin’s evidence at the preliminary hearing as follows:

The defence witness, Ms Jane Taupin, gave evidence that there is no consensus in the literature that STRmix works or any description of how it works.  While probabilistic statistical methodologies have been introduced in other jurisdictions, these other systems have not been evaluated.  Neither the United States ‘Scientific Working Group on DNA Analysis Methods’ (‘SWGDAM’) nor the International Society for Forensic Genetics DNA Commission (‘ISFG’) has published guidelines for the use of a fully continuous probabilistic methodology. In particular, there are no recommendations as to how to calculate the probability of allele drop-out or more generally regarding the use of models based on peak height variation, like STRmix.  Furthermore, the way in which STRmix determines the probability of drop-out is embedded in an internal system which is not open to evaluation.  In Ms Taupin’s opinion, the inherent unreliability of peak heights in low level DNA means that STRmix should only be used where the DNA is not low-template DNA and there is less variance in peak heights.  STRmix might be suitable for good quality DNA profiles, but it should not be used on poor quality profiles, particularly without recommendations for its use from either the ISFG or SWGDAM.[2]

[2]Ibid [43].

1. Ms Taupin gave evidence that she could not see how drop-out was calculated in STRmix and observed that the formula for the probability of drop-out could not be reviewed because Model Maker was ‘closed source’ software.  She stated variously that the actual probability of drop-out was not determined in STRmix when it should be, that there was a need to state what values for determining drop-out were being used and how they were derived and that quantitative models (such as STRmix) were less effective where there was drop-out because it was not clear how the models behaved when applied to low template DNA.  Ms Taupin was generally sceptical about the reliability of a system based on peak height variability for the analysis of low-template DNA because of the incidence of random or stochastic effects associated with such profiles.

1. The Crown has identified the opinions to which it objects in Ms Taupin’s report and in the evidence that she gave at the preliminary hearing.  The subject-matter of those opinions can be conveniently summarised as follows:

(a)the method used by STRmix for positing drop-out;

(b)the need for a risk assessment (involving a threshold of 2000 rfu) when analysing the profiles, including when using  STRmix;

(c)the use by STRmix of predictions in respect of individual loci that have only a mediocre probability of existence;

(d)the ability of different statistical models to produce different likelihood ratios; and

(e)the suitability of the STRmix analysis for certain mixture proportions.

1. The Crown contends that Taupin has not demonstrated specialised knowledge of statistics to be able to give evidence concerning STRmix and how it deals with features such as drop-out.  According to the Crown, while Ms Taupin has had extensive experience in DNA profiling, she lacks the mathematical and statistical knowledge to effectively evaluate the reliability of STRmix.

1. It is common ground that Ms Taupin does not have a degree in statistics or biostatistics and that she has not lectured in statistics or biostatistics at a tertiary level.  She has not worked in a laboratory which uses STRmix and has therefore never used STRmix herself.  She has not been able to attend STRmix training courses because they are not open to people or organisations that have not purchased the product.  However, in May and September 2014, Ms Taupin completed a two-part webcast course entitled ‘Probabilistic Genotyping and Software Programmes’, hosted by the National Institute of Standards and Technology in the USA, which considered fully continuous, probabilistic systems like STRmix.

1. The defence submits that Ms Taupin is eminently qualified to give evidence about:

(a)       both the old and new systems of DNA profiling/analysis;

(b)      how they are fundamentally different; and

(c)       where the use of the new system is controversial and unreliable or at least potentially so.

1. The defence proposes to have Ms Taupin give opinion evidence about the reliability of STRmix in response to the evidence of Dr Duncan Taylor, one of the developers of STRmix.

1. Dr Taylor is the principal forensic scientist at Forensic Science South Australia.  He has qualifications and experience in the fields of biology and biological statistics.  He has a Diploma of Biostatistics from Biostatistics Collaboration of Australia, lectures in DNA statistics in the faculty of Biological Sciences at Flinders University and is the current chair of the Australasian DNA Statistics Advisory Group, of which he has been a member since 2009.  He is one of the authors, along with Jo-Anne Bright and John Buckleton, of the paper titled ‘The interpretation of single source and mixed DNA profiles’[3] that describes the STRmix methodology and sets out the models that underpin it.

   [3]Duncan Taylor, Jo-Anne Bright and John Buckleton, ‘The Interpretation of single source and mixed DNA profiles’ (2013) 7 Forensic Science International: Genetics 516.

1. Dr Taylor has prepared an additional statement[4] which gives more specific details of the mathematical, statistical and biological modelling used in the STRmix program.  The explanation of the workings of STRmix runs over approximately 14 pages and includes literally dozens of mathematical equations or formulae describing the modelling used by STRmix.  To the untrained eye, the additional statement looks like a thesis in pure mathematics.  Dr Taylor states that he has set out the mathematics used in STRmix in order to allow a person ‘familiar with such models’ to assess how STRmix is performing.  It is plainly necessary for a person to be ‘familiar with such models’ to make that assessment.

   [4]Dated 17 April 2015.

1. Ms Taupin was shown Dr Taylor’s additional statement and asked whether she had the expertise to ‘unpick’ the contents of that paper and provide a critique of it, or whether that would have to be done by a person with a higher level of mathematical knowledge.  Having looked at the statement, Ms Taupin first said she could do most of it because she thought that most of it was an expansion of Dr Taylor’s first paper (referred to above[5]).  Then Ms Taupin said she would feel more comfortable if Dr Taylor’s additional statement ‘was in the peer review literature’.  Ultimately, however, she said ‘I wouldn’t feel comfortable doing it, no’.

   [5]Duncan Taylor, Jo-Anne Bright and John Buckleton, ‘The Interpretation of single source and mixed DNA profiles’ (2013) 7 Forensic Science International: Genetics 516.

1. I take this answer to mean that Ms Taupin does not consider that she has the expertise to analyse and critique the modelling used by STRmix.  This is unsurprising, given Ms Taupin’s qualifications and experience.  Ms Taupin has expertise in DNA profiling and the use in forensic laboratories of certain statistical methodologies used to produce likelihood ratios.  She does not have the qualifications or experience to understand and analyse the complex mathematical, statistical and biological modelling underpinning STRmix.

1. In my view, then,  Ms Taupin does not have the expertise to comment on the way in which STRmix deals with low-template DNA, because that is explained (or said to be explained) by the modelling described in Dr Taylor’s papers.  Insofar as the controversial features of STRmix, such as the way in which it accounts for stochastic effects in a profile, are explained in or by the modelling, Ms Taupin does not have the expertise to give an opinion about those features.

1. It follows that Ms Taupin cannot proffer an expert opinion about the need for STRmix to determine the ‘actual parameters’ of drop-out, or indeed, opine that there is a question about its reliability because she cannot see how drop-out is calculated or find any statement about the values that are being used for determining drop-out.  Likewise, Ms Taupin cannot give an opinion about the need for a ‘risk assessment’ based on peak heights, or about the dangers of relying on a genotype pair with only a ‘mediocre’ probability of existence.  These are all matters, so far as I am aware, for which there is an answer (good or bad) in the modelling.  What is required is a critique of the modelling, and that Ms Taupin cannot do.

1. As to the suitability of STRmix for analysis where there are certain mixture proportions, insofar as this concerns the reliability of the validation studies conducted by VPFSS, Ms Taupin can comment on the studies.  However, the answer to the question of whether, as a matter of principle, certain mixture proportions are unsuited to being analysed using STRmix is again one which depends on a fuller understanding of the STRmix modelling than Ms Taupin can claim to have.

1. The Crown submits that Ms Taupin has no expertise to comment on semi- or fully continuous probabilistic methodologies at all, and that her expertise ends with her knowledge of profiling and the application of binary statistical methodologies.  The defence counters that Ms Taupin gave unchallenged evidence that she had training in probabilistic genotyping and the Crown cannot now call that into question.

1. I take Ms Taupin’s evidence that she has training in probabilistic genotyping to be based on the two part web-based course that she undertook in 2014.  I do not consider such training capable of producing expertise of the kind I have identified as necessary to give opinion evidence about the shortcomings or limitations of STRmix.  Training of this kind does not qualify the trainee as an expert capable of understanding and analysing the mathematical modelling upon which STRmix is based and which (so it is contended) accounts for the very features that Ms Taupin calls into question.  Ms Taupin does not claim to have the relevant kind of expertise.

1. Nor, in my opinion, does the fact that in 2011 Ms Taupin presented at a DNA workshop on the problems associated with binary methods and how the probabilistic genotyping method could be introduced to overcome these problems give her the relevant type of expertise.

1. Having carefully reviewed Ms Taupin’s statement and her evidence, I accept the Crown’s submission that Ms Taupin’s ability to comment on the reliability of fully continuous probabilistic methodologies like STRmix or TrueAllele is limited.  She no doubt knows a lot more about probabilistic genotyping than the Court, and certainly much more than any ordinary member of the public, but, for the reasons I have given, her opinions about the reliability of STRmix generally and its application to the profiles in the present case demand a high level of mathematical and statistical expertise, which she does not have.

1. There are many parts of the DNA analysis carried out by VPFSS on the items recovered from the crime scene that Ms Taupin can give an opinion about.  She can give opinions about the handling of the items in the laboratory and the possibility of contamination, the creation of the profiles (including the extraction and amplification of the DNA), the quality of the profiles generally, the number of contributors to the profiles and the reliability of earlier SPURS statistical calculations performed on some of these items.

1. However, I do not consider that Ms Taupin has the specialised mathematical or statistical knowledge to enable her to give the opinions about the treatment of low-template DNA profiles by STRmix to which the Crown objects.

1. There remains a question as to whether Ms Taupin can give evidence about the literature with which she is familiar concerning the development and use of probabilistic methodologies in Australia and elsewhere.  The Crown submits that Ms Taupin reviewing the literature is no different from counsel or any other person reviewing literature, and that for Ms Taupin to review the literature would give her an aura of expertise or authority that she does not in fact have.

1. Ms Taupin has made a career of keeping up with developments in the field of DNA analysis.  I consider that Ms Taupin is well qualified to give evidence of a general kind about developments in DNA analysis and what is being done in laboratories in different parts of the world, particularly in the jurisdictions in which she has worked.  So far as probabilistic methodologies are concerned, Ms Taupin is able to give evidence that neither the United States ‘Scientific Working Group on DNA Analysis Methods’ nor the International Society for Forensic Genetics DNA Commission has published guidelines for the use of a fully continuous, probabilistic methodologies.  She can also give evidence of whether there has been an evaluation of the probabilistic statistical methodologies that have been introduced in other jurisdictions, although she is not qualified to comment on that evaluation.

1. As to the matters about which the defence says Ms Taupin can give evidence (broadly described in paragraph 13 above), such evidence would be limited to a ‘high level’ description of how the old and new systems of DNA profiling and analysis are different.

1. These are the parameters of the opinion evidence that Ms Taupin will be permitted to give during the trial in this proceeding.