Dr David Sly v Nura Holdings Pty Ltd
[2022] APO 43
•28 June 2022
IP AUSTRALIA
AUSTRALIAN PATENT OFFICE
Dr David Sly v Nura Holdings Pty Ltd [2022] APO 43
Patent Application: 2016316840
Title:Personalization of auditory stimulus
Patent Applicant: Nura Holdings Pty Ltd
Requestor: Dr David Sly
Delegate: Dr Leslie F. McCaffery
Decision Date: 28 June 2022
Hearing Date: 1 March 2022 by video conference
Catchwords: PATENTS – section 36 – inventorship – the application for a direction under section 36 is unsuccessful – costs awarded.
Representation: Counsel for the applicant: Ben Fitzpatrick
Patent attorney for the applicant: Pizzeys Patent and Trade Mark Attorneys Pty Ltd
Counsel for the requestor: Claire Cunliffe
Patent attorney for the requestor: Walker IP
IP AUSTRALIA
AUSTRALIAN PATENT OFFICE
Patent Application: 2016316840
Title:Personalization of auditory stimulus
Patent Applicant: Nura Holdings Pty Ltd
Date of Decision: 28 June 2022
DECISION
The application for a direction under section 36 is unsuccessful. The evidence does not establish that Dr Sly made a material contribution to the inventive concept.
Costs are awarded against the requestor, Dr David Sly.
REASONS FOR DECISION
Australian application 2016316840 was filed as PCT application PCT/US2016/049092 by Nura Holdings Pty Ltd (Nura, also referred to as the applicant) on 26 August 2016. The application claims earlier priority from a number of earlier applications: Australian provisional applications 2015903530 (31 August 2015), 2016900105, 2016900106, 2016900107 (all 14 January 2016) and US application 15/154694 (13 May 2016). The application named Luke John Campbell (Dr Campbell) and Kyle Damon Slater (Mr Slater) as inventors. The application entered the national phase on 23 February 2018. Following an adverse examination report, the application was accepted on 14 September 2020.
On 30 December 2020, David Sly (Dr Sly, also referred to as the requestor) made an application pursuant to the provisions of section 36, for a declaration that he is an eligible person along with Nura. The ground given in the application was that:
“I made substantive inventive contributions to the invention disclosed in the description and Claims 1 to 34 of Australian Patent Application No.: 2016316840.”
Evidence in support was completed on 4 May 2021 and comprises a declaration by David James Sly (Sly 1) dated 4 May 2021 and Exhibits DJS-1 to DJS-42.
Evidence in answer was completed on 4 August 2021 and comprises a declaration by Luke John Campbell dated 31 July 2021 and Annexures LJC-1 to LJC-20.
Evidence in reply was completed on 4 October 2021 and comprises a declaration by David James Sly (Sly 2) and Exhibits DJS-1 to DJS-10.
The matter was set for hearing on 1 March 2022. Written submissions were received from Dr Sly on 21 February 2022, and from Nura on 25 February 2022.
Onus of proof
The onus of proof lies with Dr Sly to establish on balance of probabilities that he is an eligible person.
Relevant Law
The case put forward by Dr Sly relates to his alleged contributions to the invention disclosed in the application, rather than on any contractual or fiduciary relationship between the parties at the time of the inventorship that would give rise to the proprietary rights in the invention.[1] I will therefore limit my consideration to the principles relevant to the case put forward by Dr Sly.
[1] University of Western Australia v Gray [2009] FCAFC 116 (UWA v Gray).
The Commissioner’s power to make directions in relation to entitlement is set out in section 36 of the Patents Act (the Act), which states:
(1) If:
(a) a patent application has been made and, in the case of a complete application, the patent has not been granted; and
(b) an application for a declaration by the Commissioner is made by one or more persons (the section 36 applicants) in accordance with the regulations; and
(c) the Commissioner is satisfied on the balance of probabilities, in relation to an invention disclosed in the specification filed in relation to the application for the patent:
(i) that the nominated person is not an eligible person, but that the section 36 applicants are eligible persons; or
(ii) that the nominated person is an eligible person, but that the section 36 applicants are also eligible persons;
the Commissioner may declare in writing that the persons who the Commissioner is satisfied are eligible persons are eligible persons in relation to the invention as so disclosed.
10. The dictionary in Schedule 1 of the Act defines an eligible person as follows:
“eligible person”, in relation to an invention, means a person to whom a patent for the invention may be granted under section 15.
11. Section 15(1) further provides that a patent may only be granted to a person who:
(a) is the inventor; or
(b) would, on the grant of a patent for the invention, be entitled to have the patent assigned to the person; or
(c) derives title to the invention from the inventor or a person mentioned in paragraph (b); or
(d) is the legal representative of a deceased person mentioned in paragraph (a), (b) or (c).
12. In UWA v Gray,[2] the Full Court of the Federal Court endorsed the following approach to determining entitlement:
- Identify the “inventive concept” of each relevant invention.
- Determine inventorship including the person responsible for the inventive concept and the time of conception as distinct from its verification and reduction into practice.
- Determine if contractual or fiduciary relationships at the time of the inventorship give rise to proprietary rights in the invention.
[2] Ibid at [253].
13. A number of relevant principles relating to entitlement were recently summarised by the Full Court of the Federal Court in Vehicle Monitoring Systems Pty Ltd v SARB Management Group Pty Ltd,[3] and by the Federal Court in Vector Corrosion Technologies Limited v E-Chem Technologies Ltd.[4]
[3] [2021] FCAFC 224 (VMS v SARB).
[4] [2022] FCA 188 (VCT v E-Chem).
14. Some of the relevant principles that flow from the authorities in relation to section 15 are:
·The primary person to whom a patent may be granted is the inventor. All other persons derive title to the patent via the inventor. Identification of the “inventor” is therefore central to the provision because it is by reference to the ‘inventor’ that patent ownership is established.[5] However, the term ‘inventor’ is not defined in the Act.[6]
[5] VMS v SARB at [48].
[6] VMS v SARB at [49].
·There is a distinction in the authorities between a discovery or an idea (an inventive concept) and the reduction of that inventive concept to practice.[7] The inventive concept marks a boundary between invention and verification.[8]
[7] VCT v E-Chem at [17(1)].
[8] VCT v E-Chem at [17(2)].
·The time at which the invention was developed and the person by which it was developed is to be ascertained by reference to the inventive concept of the invention so described. The time of invention, and the identity of the inventor will not be affected by the subsequent process of reduction to practice, some elements of which may have found their way into the claims in the application.[9]
[9] VCT v E-Chem at [17(3)].
·Determining inventorship takes a two-step enquiry. The starting point is to identify the inventive concept(s) in the application. The subsequent step is to identify the person responsible for the inventive concept – in the sense of the person who materially contributes to the inventive concept.[10]
·The inventive concept is determined from the specification as a whole (including the claims) rather than the form of the claims or the definition of the invention in the claims.[11] In relation to identifying the inventive concept, “what one is normally looking for is ‘the heart’ of the invention. There may be more than one ‘heart’ but each claim is not to be considered as a separate ‘heart’ on its own.”[12]
·Entitlement is not determined by reference to the inventiveness of the invention claimed in the patent. It is determined by reference to the history of the inventive concept in the minds of the inventors. To that end:
“[r]ights in an invention are determined by objectively assessing contributions to the invention, rather than an assessment of the inventiveness of respective contributions. If the final concept of the invention would not have come about without a particular person’s involvement, then that person has entitlement to the invention. One must have regard to the invention as a whole, as well as the component parts and the relationship between the participants.”[13]
·However, it does not follow that anyone who contributes to an invention is considered to be an inventor. For example, a claim to entitlement will not be established where the party’s claimed contribution to the inventive concept is merely “positing desiderata or making a vague proposal”. However, while vague suggestions may not constitute a material contribution to the inventive concept, an invention may reside in the conception of an idea without the need for reduction to practice.[14]
[10] VCT v E-Chem at [17(8).
[11] VMS v SARB at [65].
[12] VMS v SARB at [63].
[13] VMS v SARB at [68]-[69].
[14] VMS v SARB at [70]-[72].
Some background terminology
15. There are two broad means of assessing hearing – subjective tests and objective tests.
16. Subjective tests typically involve a patient providing a manual input such as by pressing a button in response to a noise, sound or word. Objective tests do not require a manual response from the patient. Rather, the hearing is assessed through objective measurements.[15]
[15] Applicant’s submissions at [52].
17. There are two main categories of objective test.
18. The first of these are electrical tests which use electrodes to measure electrical signals, usually from nerves at various locations in the hearing system (auditory nerve, brainstem, midbrain, brain). The auditory brainstem response (ABR) is an auditory evoked potential (AEP) extracted from ongoing electrical activity in the auditory system and measured via electrodes placed on the scalp. ABR is considered an exogenous response because it is dependent upon external factors. ABRs were first recorded in the 1960s with surface electrodes in humans. Since then, ABRs have been used extensively for new-born hearing screening, auditory threshold estimation, determining hearing loss type and degree and in the development of cochlear implants.[16]
[16] Applicant’s submissions at [54]-[55].
19. The second category of objective test is mechanical testing, which detect mechanical movement caused by sound, or mechanical properties, of elements of the hearing system such as the eardrum. These are typically detected using microphones and acoustic principles. The measurement of otoacoustic emissions (OAE) is an example of a mechanical test.[17]
[17] Applicant’s submissions at [53].
20. OAEs are a low-level sound that is emitted by the cochlea in response to an auditory stimulus. These provide information related to the function of the outer hair cells of the ear and can be used to measure how well the cochlea is working.[18] There are two broad types of OAE: spontaneous OAEs, which occur without external stimulation, and evoked OAEs, which require an evoking stimulus.[19]
[18] Applicant’s submissions at [57].
[19] Applicant’s submissions at [58].
21. There are two types of headphones referred to in the opposition: diagnostic headphones which are specialist headphones that are used to screen and diagnose hearing; and consumer headphones which are used for general purposes, such as listening to music, and includes hearing aids. The present invention relates to a consumer headphone, but much of the background relates to diagnostic headphones.
Summary of relevant events
22. Before considering the evidence from the parties, I note that the requestor submitted that Dr Campbell’s evidence was not supported with any corroborative evidence, and was vague inasmuch as it was largely based on recollections, and therefore should not be accepted.[20] They also submitted that the failure to lead evidence from Mr Slater inferred that his evidence would not have assisted Nura.[21] On the other hand the applicant submitted that there were two distinct projects, each with different aims and participants. The first of these was the hearings diagnostics project which was aimed at developing diagnostic headphones, while the second related to consumer headphones. The applicant submitted that much of the correspondence relied on by the requestor relates to the hearing diagnostics project.[22] They cautioned that Dr Sly’s evidence at times veered into advocacy on behalf of his case.[23] They also noted that Dr Sly had referred to a Wikipedia entry to assert that the research resulting in the company’s proprietary technology was carried out under his direction.[24] This reference had apparently been added by an anonymous user 10 days before Dr Sly filed his request.[25] The applicant considered it troubling that Dr Sly had not responded to deny involvement in this amendment or to explain his reliance in his evidence on this purportedly false entry.[26]
[20] Requestor’s submission at [122] to [125].
[21] Requestor’s submissions at [134].
[22] Applicant’s submissions at [70]-[71].
[23] Applicant’s submissions at [22].
[24] DJS-4 (Sly 1).
[25] Campbell at [63].
[26] Applicant’s submission at [17] to [18].
23. I acknowledge the concerns of the parties on these points but see no need to unilaterally reduce the weight of either party’s evidence based on these submissions. However, I have considered the substance of evidence relevant to the issue of inventorship together with contemporaneous evidence to determine the weight that evidence can be given.
The hearing diagnostics project
24. Dr Sly did not provide a curriculum vitae, but the evidence indicates that he was employed in the Department of Otolaryngology at the University of Melbourne from at least 2009.[27] A research grant application dated 25 March 2013 indicated that Dr Sly was a Senior Research Fellow at that time.[28] Dr Sly’s early research involved the development of a hearing diagnostics platform, which was described in an invention disclosure as follows:[29]
“This invention involves the use of a portable/handheld computing device such as an iPhone or iPod touch as a (sic) audiological testing tool in conjunction with a minimal amount of readily available consumer hardware such as earphones and microphones. Such a device could be used in common audiological tests in the assessment of hearing including audiometry, otoacoustic emissions and auditory brainstem responses. At present portable devices for hearing screening exist but are very expensive and limited in function.”
[27] DJS-5 (Sly 1).
[28] DJS-8 (Sly 1).
[29] Sly 1 at [19].
25. Dr Sly stated that in 2010 he commenced further research in this area, leading a team of four students in developing an iOS-based hearing testing application.[30] Between 2011 and 2015, Dr Sly led research into headphones and earbuds for use in hearing diagnostic tests.[31] During this period, Dr Sly sought, or obtained, funding through iFund,[32] Royal Victorian Eye and Ear Hospital (RVEEH),[33] Defence Health Foundation (DHF),[34] and the NHMRC.[35] Dr Campbell was named as a researcher or co-investigator on all of these research projects.
[30] Sly 1 at [20].
[31] Requestor’s submissions at [66], Sly 1 at [9].
[32] DJS-7 (Sly 1).
[33] DJS-8 (Sly 1).
[34] DJS-9 (Sly 1).
[35] DJS-13 (Sly 1).
26. The draft iFund grant application provides an overview of the intended outcomes of the research:
The product is a low cost, mobile, professional-grade hearing test that utilises Apple iOS based hardware. The proposed device would consist of an iOS app on the Apple iPhone or iPad connected remotely by Bluetooth technology to hardware placed on the patient. The device will perform the major tests used in diagnostic audiology, which are audiometry, auditory brainstem response and otoacoustic emissions.
In practice, all tests involve playing sounds through the software and hardware and recording patient responses. The patient responses involve either clicking a response button (audiometry), automatically acquired brainstem responses (auditory brainstem auditory responses), or sounds generated by the inner ear detected by a small microphone in the ear (otoacoustic emissions).
Simplification of the computing and software platform by using iOS, is alone a significant market opportunity, however due to the power of the iOS platform coupled with its wireless connectivity built in, we see this as an opportunity to also simplify the specialist hardware required and to keep this independent from the computing hardware by using wireless communications between the specialist hardware and the iOS hardware. We aim to simplify the hardware used by using largely cheaper, readily available components. Fewer components and cables reduces the cost of producing the device and increases reliability by reducing the number of potential points of failure of equipment.
The hardware would likely consist of:
• A single set of wireless canal phones (in-ear) connected to an attached separate amplifier/bluetooth module. These would look similar in form to current mobile phone ear pieces or hearing aids.
• A microphone embedded into the canal phone enclosure for detecting otoacoustic emissions. We anticipate the microphone we use may be those currently used as in-line microphones on headphones supplied with the iPhone.
• Two or three EEG recording electrodes placed on the scalp surface to record neural responses.
• Further sound proofing if required can be achieved through circumaural (over-the-ear) ear-muffs placed over the ears and over the canalphones. The EEG recording electrodes could be incorporated as part of the ear muff headband.
• The amplifier embedded in the canal phone piece will amplify all responses and transmit these to the iOS device over bluetooth or WiFi.
27. I note here that the proposed canal phones were intended to look similar to mobile phone earpieces or hearing aids, but were otherwise intended to be a custom-made, specialised diagnostic tool comprising two different objective tests (OAE and ABR) and a subjective test (audiometry). These multimodal headphones were intended to connect to a mobile device via Bluetooth. An artist’s impression of these headphones was provided in several of the grants (Fig 1).[36] As shown in this figure, the headphone was intended to comprise hardware for different measurements, including electrodes for ABR, and earphone microphone and speaker for OAE.
Fig. 1[36] See for example DJS-9 (Sly 1) at page 5.
28. Dr Sly’s research continued under several other grants. The RVEEH project had the aim of developing a low-cost, mobile, professional-grade hearing test equipment. The application stated that major portions of the required hardware had been developed, including a small custom bioamplifier for recording ABRs (Fig 2), with the aim being to combine the separate circuits shown in the figure into a single box of about 10x5x2cm.[37] The DHF project set out similar aims.[38]
Fig. 2
[37] DJS-8 (Sly 1), see application form, page 1 of 8.
[38] DJS-9 (Sly 1), see Scientific Description of Project under Aims.
29. The final report for the RVEEH grant[39] and the 6-monthly report for the DHF grant,[40] list similar achievements, including that software and hardware for audiometry had been completed and the device boxed, calibrated and successfully tested against existing clinical audiometry equipment. This device is shown in figure 3. I understand this device to use a subjective audiometric measurement, where the patient responds to a tone using a button (in this case the Bluetooth push button device shown on the left-hand side).
Fig. 3[39] DJS-8 (Sly 1) under the heading “RVEEH SMALL GRANT SCHEME FINAL REPORT”.
[40] DJS-9 (Sly 1) under the heading “DEFENCE HEALTH FOUNDATION - SIX MONTH PROGRESS REPORT”.
30. A further achievement was the development of a “novel” headphone/earmuff design for sound-proof audiometric testing that comprised insert earphone drivers located in the ear cup of the earmuffs (Figure 4). This was said to provide acoustic attenuation equivalent to a double walled sound booth. An iPad-based audiometry device incorporating this arrangement was deployed for use in the Democratic Republic of Congo in May 2014.[41]
Fig. 4
[41] DJS-9 (Sly 1) under the heading “ANNUAL PROGRESS REPORT”.
31. As noted above, the group had intended to develop the remote audiometer and the multimodal headphones under these grants. While the remote audiometry hardware and trials were achieved, the evidence suggests that a multimodal headphone had yet to be assembled as of at least 25 May 2015.[42] This is consistent with the evidence of Dr Campbell, who stated that, to the best of his knowledge, the technology has not progressed to the point where it has been validated.[43]
[42] DJS-29 (Sly 1) under point 2.
[43] Campbell at [30].
32. Around October 2014, Dr Sly and an intern had developed some CAD designs for a wireless OAE probe, which appears be referred to in an email to Dr Campbell as a “super probe”.[44] Dr Sly stated that this was a wireless OAE probe that would connect to iOS, Bluetooth clicker, and audiometry housings.[45] The email and attachments do not specifically refer to OAEs or label the components in the conceptual diagram in any way that clearly identifies it as including an OAE sensor, but the wiring may be indicative of speakers and a microphone in the earpiece. The ear loop also appears to comprise an external sensor or transducer on the ear loop, but I have no evidence on that point. Dr Sly subsequently developed a draft business plan for a company, NeuroX,[46] whose focus is said to be “wearable sensors and neurodiagnostics encompassing device design through to platform technologies Mobile OS, Health economics data analytics)” and describes an “ear super probe” that appears to be the same ear probe concept.[47] The document refers to having as many sensors and actuators as possible, and audiometry and electrophysiology are mentioned. The document states that the probe serves as a “standard” headphone and hearing aid, as well as being used to provide “super hearing” by shifting hearing frequencies that are not normally heard into hearable range, voice control and communication to smart devices, hearing protection, assessment and improvement as well as neurological assessment and general health measures (blood pressure, heart rate, sweat). The device is said to be for diagnostic purposes, but also for therapeutic and personal use for auditory rehabilitation, training and biofeedback of the user’s own neural signals as sounds.
[44] DJS-14 (Sly 1).
[45] Sly 1 at [29].
[46] Sly 2 at [36].
[47] DJS-5 (Sly 2).
33. Dr Campbell was named on the iFund, DHF and NHMRC grant applications but did not join the hearings diagnostic project until 2014. At the time he was completing a PhD at Melbourne University with Dr Sly as one of his supervisors. His PhD topic involved hearing tests from cochlear implants. Dr Campbell considered this a related topic, since both projects involved rapid automatic assessment of a person’s hearing, but the cochlear project did this by measuring electrical signals from inside the fluid within the inner ear (cochlea) in patients with cochlear implants.[48] Mr Slater also joined the hearings diagnostic project around November 2014.[49] Mr Slater was also a PhD student at Melbourne University, and was designing silicon computer chips for enhancing cochlear implants and bionic eyes.[50] Mr Slater did not provide evidence in the present matter, so I have no corroborative evidence from him on this point or, indeed, any other.
[48] Campbell at [12].
[49] Campbell at [25], Sly 1 at [30].
[50] Campbell at [75].
34. In January 2015, Dr Sly’s team was asked to present at the Avalon Air Show by the Defence Science Institute,[51] which Mr Slater prepared and presented.[52] Around the same time Dr Sly presented at the ORLIAC conference in Kuala Lumpur, Malaysia, where he provided a “public overview” of his ideas for putting hearing diagnostics inside headphones.[53] These presentations related to hearing diagnostics and not consumer headphones.[54]
[51] Sly 1 at [32].
[52] Sly 1 at [33].
[53] Sly 2 at [65].
[54] DJS-18 (Sly 1).
35. Following the Avalon Air Show, Mr Slater received an email from Stuart Anderson from Sydney Capital Partners (SCP) requesting a meeting. SCP were interested in acquisitions in the audiology sector.[55] Dr Sly, Mr Slater and Dr Campbell met with Mr Anderson on 19 March 2015. Dr Sly stated that he raised the ideas of discussion of convergence of audiology and consumer technologies during the meeting, which Mr Anderson wanted to follow up on.[56] He subsequently stated in his evidence in reply that Stuart Anderson “wanted to discuss more about our ‘sensing technology’ in relation to hearing diagnostics.”[57] Dr Campbell stated that he did not recall Dr Sly making any reference to consumer technologies at that meeting and that the notes from the meeting[58] do not include any reference to consumer technologies.[59] No evidence was adduced as to any further discussions that took place.
[55] DJS-17 (Sly 1).
[56] Sly 1 at [36].
[57] Sly 2 at [45].
[58] LJC-18.
[59] Campbell at [71].
36. In view of the evidence before me, I consider the main aim of the project up to March 2014 had clearly been to develop a diagnostic platform. Dr Campbell described this as follows:
“The hearing diagnostic project aimed to incorporate well-known subjective tests of hearing (e.g. push a button if you can hear this) and well-known objective tests of hearing (i.e. ways to automatically test hearing that do not require the user to take conscious action such as pushing a button to give feedback) and administering the testing through Apps and phones or iPads or the like. The aim and purpose of the hearing diagnostic project was to make the testing and diagnosis of hearing more portable and convenient for medical purposes.”[60]
[60] Campbell at [13].
37. Dr Sly emphasised that the project was not just about an app, but a comprehensive electrophysical objective testing system that required hardware.[61] I do not consider the evidence on this point differs significantly between the parties. In short, the focus of the group to this point in time had been on the development of a portable diagnostic platform using objective and subjective tests, including both ABR and OAE, coupled to a mobile device such as a tablet. A prototype device comprising a Bluetooth button for user input for audiometric testing had been developed, as had a pair of sound-proofed earmuffs that comprised earphone inserts for such testing. Some progress had been made towards the development of an ear probe for OAE measurement and a biosensor for ABR measurement, but a headphone incorporating these different diagnostic sensors and circuitry had yet to be assembled as of at least 25 May 2015.
The personalised audio “brainstorming session”
[61] Sly 2 at [25].
38. A key shift in direction occurred in March 2015. Dr Campbell stated that:
“On or around 30 March 2015, in a brainstorming session between myself and Mr Slater (which Dr Sly was not present at) Mr Slater and I conceived the idea of utilising existing techniques for testing hearing and using the information about the user’s hearing to modify the sound of music in real time in a consumer headphone. Our concept arose from discussing how it might be possible to take objective hearing tests out of labs and hospitals and make them something useful for everyone. And that’s when it hit us. We could use these measurements to improve listening to music. Mr Slater grew up with a family of musicians and was incredibly passionate about music and people’s experience of music. I had been building knowledge about how hearing and the auditory system works and developing lab-based hearing testing equipment. We both immediately realised this was a really good idea, that we had never touched on before. As soon as we had it, we excitedly discussed for hours how to deal with it all technically, including measuring hearing, specifically discussing ABR and OAE. We also discussed using subjective tests and other sorts of objective tests and the general approaches to mapping the results and applying personalization (i.e. using the hearing measurements to work out how to improve the music). We also did a quick check (10min on google) to see if it looked like anyone else was talking about or doing anything similar, and it appeared they weren’t. We decided at the end of the session to immediately set about forming a company to commercialise that idea.”[62]
[62] Campbell at [33].
39. Notably, Dr Campbell stated that he and Mr Slater discussed the concept of making diagnostic hearing testing useful more widely, and specifically using the information obtained to modify the sound of music in real time. Their concept was to provide a loudspeaker such as a headphone that used subjective assessment where the user is asked to give conscious feedback and/or objective information to automatically adapt the auditory stimulus (such as music) to be tailored to a user’s hearing profile.[63] As the speaker plays an audio signal to the user, the sensor would record the user’s response to the signal. Based on the measured response and the knowledge of how people perceive sound, the signal could be modified to compensate for any differences between the user’s hearing and ideal hearing.[64] He went on to state that:
“…Mr Slater and myself discussed whether ABR or OAE or other types of tests would be the best tests for putting our concept into practice. I recall saying to Mr Slater at that session that my preference was to use OAE as the testing times could be faster whilst I recall Mr Slater saying that he preferred to use ABR as he was more familiar with that type of testing and its superior accuracy. However, we both agreed that it didn’t really matter whether ABR or OAE (or other tests) were used to modify the sound of music in a consumer headphone and that we should proceed pursuing objective tests of hearing in the most general sense and subjective tests of hearing and practically determine the best practical, commercial approach later.” [65]
[63] Campbell at [36].
[64] Campbell at [37].
[65] Campbell at [38].
40. Dr Campbell indicates here that the discussions were not limited to any specific testing method. He went on to further state:
“The general approach of comparing the results measured in one user to a model of a normal user applies to any objective test of hearing with differences between OAE or ABR or other tests (such as electrocochleogram or auditory steady state responses or cortical evoked responses) and the difference between which test you use only manifest themselves in reduction to practice. For example, if using OAE, microphones in the ear canal are required. If using Electrocochleography, electrodes in the ear canal are required. Therefore, the electronics to support each approach are different. Also, the types of sounds you use in the hearing test and the duration of each test and signal processing used are different. However, both types of tests give you a picture of the hearing ability of the user. Accordingly, the principle of using that measure of hearing ability to derive a way to modify a user’s music is essentially the same regardless of the specific technique used to measure the hearing in the first place.”[66]
[66] Campbell at [39].
41. Dr Sly did not dispute the timing of the “brainstorming” session, nor the concept of personalised audio being generated by Dr Campbell and Mr Slater. He did not state what was specifically communicated to him by Dr Campbell and Mr Slater at the time concerning the nature of the personalised audio concept but asserted that he conceived “the solution to realise how to provide feedback into the personalised audio, that being via otoacoustic emissions.”[67]
[67] Sly 1 at [6].
42. Dr Campbell stated that following the brainstorming session, he and Mr Slater decided to immediately start registering email addresses and business names.[68] Dr Sly was included in the selection of the business name.[69] In an email of 30 March 2015, Mr Slater stated that he liked the name loop or neuroloop because it “relates to the closed-loop formed between sound and neural interface”. The business name registration for nuraloop was filed by Mr Slater on 1 April 2015.[70]
The Melbourne Accelerator Program and Medtech’s Got Talent applications
[68] Campbell at [42].
[69] DJS-3 (Sly 2).
[70] DJS-4 (Sly 2).
43. During April and May 2015, Dr Campbell, Mr Slater and Dr Sly pursued applications to the Melbourne Accelerator Program (MAP) and Medtech’s Got Talent (MTGT). Much of the evidence does not contribute greatly to the determination of inventorship so I will deal with it fairly briefly. The MTGT application, which was submitted on 8 April 2015, is probably of greatest relevance as it refers to real-time audio enhancement in the personal audio industry, and states:
“Nuraloop technology is able to probe both the cochlear and brains (sic) neural response to acoustic stimuli and use innovative DSP [digital signal processing] techniques to deduce the type and nature of hearing loss. This information can be used to inform treatment options in moderate to severe cases, or provide real-time enhancement for those with moderate loss.” [71]
[71] DJS-21 (Sly 1)
44. The application refers to the Nuraloop technology being able to probe both the cochlear and brain’s neural response to acoustic stimuli. This appears consistent with the hearings diagnostic group’s research up to that point, and in particular the work towards providing a diagnostic platform using multiple sensing techniques (electrocochleography (EcochG), ABR and OAE). The further reference to innovative DSP providing the ability to deduce hearing loss from both the cochlear and neural responses to enhance audio appears to relate to the present invention in developing a hearing profile that can be used to personalise audio signals.
45. A draft pitch for MTGT circulated by Mr Slater on 19 April 2015 described medical grade hearing diagnostics and headphone technology and audiological treatments. Dr Sly noted that this did not refer to personalised audio.[72] The pitch was delivered to Luke Martin from Tricycle Developments on 21 April 2015. Dr Sly stated that he presented the Nuraloop pitch to the MAP on 29 April 2015, and noted that the presentation suggested a feedback loop from the brain to the headphones but there was no indication at this stage of a feedback loop from the ear to the headphones involving ECochGs or OAEs.[73] Dr Campbell recalled Mr Slater presenting at the MAP session and that the lack of details on the slide was “deliberate so as not to create prior art before submitting a provisional patent”.[74] These points do not appear to have been disputed by Dr Sly, but I do not consider this evidence to be of great relevance to the issue of inventorship in any case.
[72] Sly 1 at [47].
[73] Sly 1 at [51].
[74] Campbell at [77].
46. On 14 April 2015, Mr Slater and Dr Campbell filmed footage for the MAP presentation.[75] On 16 April 2015, Dr Campbell and Dr Sly met with University of Melbourne Commercial Department to discuss Audio Diagnostic Technologies.[76] On the same day Mr Slater presented at the MTGT Rapid Fire Round. Nuraloop was selected as a finalist and partnered with Tricycle Developments for mentorship.[77] During May 2015 Nuraloop progressed to later stages of the MAP funding application,[78] and was announced winner of the MTGT.[79]
Email Correspondence between the team members
[75] Sly 1 at [43].
[76] Sly 1 at [44].
[77] Sly 1 at [45].
[78] Sly 1 at [58].
[79] Sly 1 at [59].
47. Sly stated that on 17th April 2015:
“I proposed my idea of using one singular device [headphones] to assess auditory status and use this to adjust acoustic sound for the purpose of personalised audio for improved intelligibility or enjoyment. The inclusion of outer hair cells (OHCs) implied OAEs as OAEs assesses OHC function.”[80] [emphasis from original document]
[80] Sly 1 at [46].
48. At hearing the respondent relied on this evidence to assert that Dr Sly conceived the solution of how to provide feedback into the personalised audio via OAEs.[81] The applicant disputed this assertion, submitting that the email does not specifically relate to OAEs, but more broadly to a range of measures: inner hair cells, outer hair cells and auditory nerve fibres.[82] Dr Sly’s email refers to a publication describing where ABR sits in the scope of Evoked Potential (EP) testing and notes that EP tests are not routinely used by audiologists except in the case of newborn screening. The email goes on to say that:
“the cochlear and auditory nerve is where diagnostics should be focussing and anything we can do in hardware and signal processing to improve the amount of detailed info about the health of the peripheral auditory nervous system and to simplify the hardware and minimise the invasiveness/cumbersome nature of current testing will be gold.”
“I think we need to look at preparing (and searching the patent databases - I know which are best - not just Google Patents) one or two patents ASAP.
1. The above info about improved diagnostics of auditory nervous system using multimodal/simultaneous testing in one device to comprehensively asses [sic] the physiological state of all IHCs, OHCs, excitatory and inhibitory ANFs, giving a map of the health of an individual’s entire cochlea (at least the direct sensorineural elements)2. Taking above info about auditory nervous system and using this to adjust acoustic stimuli for purpose of either i) improved rehabilitative hearing stimuli (e.g. hearing aids) ii) personalised audio for improved intelligibility or enjoyment.”[83][81] Requestor’s submissions at [89].
[82] Applicant’s submissions at [141].
[83] DJS-22 (Sly 1).
49. On balance, I do not find the requestor’s submission that this email shows Dr Sly conceived the concept of personalised audio using OAEs persuasive. As noted by the applicant, the email does not specifically refer to OAEs, but to a broader range of diagnostic tests to give an indication of the health of the entire cochlea. Admittedly the email refers to the use of the diagnostic data for delivering personalised audio, but this idea appears to be the same one put forward in the MTGT application discussed above at [43].
50. Dr Sly went on to state that on 23 April 2015:
“I sent an email to Campbell and Slater reiterating that the Auditory Nerve (AN), and not measuring ABR, is the best approach to detect hearing in a consumer headphone (earbud) that uses this to equalize incoming sound played through the headphone. This is the first mention of using electrocochleography (ECochG) and feeding ECochG measurements back in a loop. Therefore, this is the first time when the central idea for the invention of providing the personalised headphones appeared.” [84] [emphasis from original document]
[84] Sly 1 at [49].
51. Dr Sly appears to be suggesting here that the concept of personalised audio developed by Mr Slater and Dr Campbell did not extend to the use of headphones or earbuds that incorporated a feedback loop, and that he was also responsible for this aspect of the invention as well. Dr Campbell disputed this suggestion, stating that the email referred to by Dr Sly related to patent searches that he conducted once the inventive concept had been communicated to him.[85] The email in question refers to a patent document located by Dr Sly in his background research on ABR and evoked potentials, as follows:
“What is not in her patent is any specific mention of use of ABR for consumers who don’t have hearing aids. Note though at the end of the patent are sweeping claims about how the technology could be used in other fields not specifically mentioned. She also mentions the cABR could be used to improve microphones, amplifiers and speakers in the feature article linked to below.
We should get on to a patent in the consumer area ASAP. I also don’t necessarily believe that the cABR [complex auditory brainstem response] is all about the brainstem, there is a fair bit of auditory nerve activity per se in there. A real differentiator for us in a patent could be recording ECochG in a consumer earbud and feeding this info back.
I’m pretty convinced that the AN is the place to focus on. Once the incoming EQ is adjusted to the AN and reset to near normal levels, the brain can reset itself.”[86]
[85] Campbell at [75].
[86] DJS-23 (Sly 1).
52. Dr Sly clearly suggests here that the group should focus on the auditory nerve (AN) and suggests that EcochG could be an area of differentiation with the prior art disclosure. However, contrary to Dr Sly’s statement, I do not consider the email suggests ABR should not be measured, but rather that ABR provides an indication of AN activity.
53. Dr Sly stated that on 27 April 2015, he advanced the idea of using AEPs via the peripheral auditory system, such as cABR or ECochG.[87] Dr Sly went on to state the reply from Mr Slater on 28 April 2015 showed that neither he nor Dr Campbell had intended, nor formulated in their minds an idea of filing a patent application disclosing personalization of auditory stimulus and had not formulated the idea of using OAEs or evoked neural potentials for this.[88] The email from Dr Sly states:[89]
“Microsoft et al. have some new patens (sic) in to consumer-level assisted hearing space. I think this is the space that Sydney Capital were interested in.
Kyle I reckon if we can get a provisional patent for putting neural feedback inside headphones then Sydney Capital (and others) will be all over us. We need the patent soon as this will spread by word of mouth very quickly and a big player will file it before us.”
[87] Sly 1 at [50], DJS-24 (Sly 1).
[88] Sly 1 at [50].
[89] DJS-24 (Sly 1).
54. I note that the email does not specifically refer to the use of any specific measurement technique, though the reference to putting neural feedback inside the earphones appears to be consistent with Dr Sly’s earlier correspondence concerning the auditory nerve being the area to focus on. Mr Slater’s response is also notable:
“Hi David, I'd be very surprised if we could patent the neural feedback idea. Surely someone has thought of that.. their ability to actually protect it will lie in the specific claims they make. We would need to patent a DSP technology / sensing tech in that space I think.
I'm all for patents and getting one - but I'm keen to focus on defining our technology and our business. Background is good - but we have so many immediate hurdles right now - like my MAP presentation tomorrow, so I think we should focus on that. Know what I mean?”
55. This email exchange perhaps highlights that the distinction between the diagnostic hardware, which had been the core interest of the Hearing Diagnostic group (multimodal diagnostic headphones linked to a mobile device), and the concept of personalised headphones, which is reliant on the way in which test data is analysed. I also note Mr Slater’s reference to the information provided by Dr Sly being good “background”. This arguably suggests that the information about various devices provided by Dr Sly was considered by Mr Slater to be related to background technology rather than core to the invention.
56. On 2 May 2015 Dr Sly sent a patent landscape report to Mr Slater and Dr Campbell.[90] Dr Sly stated that up until this stage the focus of Dr Campbell and Mr Slater had been on the use of ABR, though Dr Campbell noted that it was Dr Sly who was responsible for the selection of the search parameters. The report broadly describes the invention as relating to auditory evoked potentials. The specific search terms used in the landscape report appear consistent with a broad search of background art (electrophysiology, bio-signal, neurofeedback, biofeedback, auditory brainstem response, non-contact sensors, auditory, hearing aid and electrocochleography). Dr Sly stated that the focus of the group shifted following the patent landscape report he prepared and a subsequent email in which he highlighted a patent by Samsung in the ABR area.[91] He went on to say that:[92]
“Campbell’s response to my email indicates that he believed this was significant. This is clearly seen in the following passage: “hmmm simple ABR is out of the question”, obviously indicating that we were looking for the inventive step together. It was me that introduced Campbell to cABR… and ASSR [audio steady-state response] being part of the hearing diagnostics (prior grants).”
[90] DJS-27 (Sly 1).
[91] DJS-6 (Sly 2).
[92] Sly 2 at [44].
57. The full transcript of Dr Campbell’s responding email is:[93]
“Hmm.. that puts using simple ABR out. They do specific (sic) "hearing aids" in the claim so headphones maybe still ok. They also specific (sic) peak based detection so maybe using RMS techniques instead still ok.”
[93] DJS-6 (Sly 2).
58. This email does not, in my opinion, support the submission that the focus of the group shifted as a result of Dr Sly’s patent landscape report. Dr Campbell certainly considered that simple ABR was out of the question but noted that there may still be scope for work in the area for headphones and the use of RMS in the signal analysis. Again, this arguably highlights the difference in the focus between the parties in relation to the devices as opposed to the use of data to provide personalised audio. Moreover, the subsequent evidence does not support the assertion that ABR was not further considered by Nuraloop – indeed the use of ABR was included in the patent applications that were subsequently filed.
59. At the heart of Dr Sly’s request is the assertion that he contributed to the inventive concept by identifying OAEs as the “ideal test and solution” to providing personalised audio. To this end he stated:[94]
“On 8 May 2015, I sent an email to Campbell proposing to use OAEs in a feedback loop and having a microphone in consumer earbuds. Therefore, 8 May 2015 is the date when I contributed to the first conception of the main claims (Claims 1, 4, 11, 12, 15, 20, 21 and 22 of the Patent Application) in the Patent Application relating to OAEs. Prior to this date (for example in the “Patent Search” document) we were looking at ABR and ECochG.”
[94] Sly 1 at [56].
60. The entire transcript of Dr Sly’s email of 8 May 2015 was:[95]
“OAEs in a feedback loop could be a lot easier. Also having a mic in consumer earbuds allows volume limiting to be a bit smarter than it is at present.”
[95] DJS-28 (Sly 1).
61. Dr Sly did not provide any evidence as to any discussions leading up to this particular email, even though the suggestion that OAEs in a feedback loop could be easier suggests he was contrasting OAE with another technique, and presumably ABR. It appears that he had circulated two documents relating to DP-OAE [distortion product OAE] in a previous email, so alternatively this could have been a reference to the content of these documents, but these were not in evidence or otherwise discussed. There was no evidence of the context of this email or of any response by Dr Campbell or Mr Slater, but Dr Campbell stated this was because there was no new information to respond to.[96]
[96] Campbell at [81].
62. On 24 May 2015, Mr Slater asked Dr Sly to complete a headphone market analysis. At the hearing the requestor highlighted that the completed report noted the existence of personalised audio and observed that Nuraloop’s headphones would record OAEs.[97] On 25 May 2015 Mr Slater asked Dr Sly and Dr Campbell to provide possible questions for a meeting with the MAP.[98] The requestor submitted at the hearing that it was clear from Dr Sly’s response that the headphones were the same headphones that his team had been working on for the previous couple of years. In response to the question “Have you completed development of your headphones?” Dr Sly suggested the following:
“We have spent the last year developing the miniaturised bioamplifiers and ear sensors (mic and speakers) and collecting measurements in people. We seek further investment to move from Technology Readiness Level (5) (bench validation) through to TRL 7 (assembled headphone prototype)”.
[97] Requestor’s submissions at [104], and DJS-30 (Sly 1).
[98] DJS-29 (Sly 1).
63. This response is consistent with the progress reports from the earlier research grants to the extent that the headphones were still in development as discussed at paragraph [37] above. Bioamplifiers and ear sensors had been developed to a stage where measurements had been achieved. Further investment was sought to move through bench validation to an assembled headphone prototype.
64. During May 2015, the group began preparing a provisional patent application. This is further discussed in detail below. On 24 July 2015 Mr Slater circulated a roadmap setting out a high-level picture of the path forward for Nuraloop.[99] The roadmap proposed separation of the business into two entities covering consumer and diagnostics products. The consumer product plan comprised three streams: front end design, in-line amplifier (which included OAE/ABR prediction of profile, hearing prediction and core tech integration), and “iECG crowdsourcing” (note: this aspect of the roadmap was not discussed in evidence). The Nuraloop Diagnostics (ND) plan included development of an OAE prototype, preclinical trial and product design. Dr Sly’s response appears to distinguish between the technology related to the consumer headphones, and his research on diagnostics:
“I am about to have a substantial amount of time over the coming months to pursue ND/diagnostics activities and I want to get on with it. I want to pursue the technologies listed in your roadmap for ND and those many that are not (the iPad screening device, headphone diagnostics, and several others). This includes working on the technologies themselves, seeking academic and industry grants, filing patents and evaluating the possibility of working with Tectonica on this and other projects related to hearing technologies. This is even more important to me now given that neither MAP or MTGT are immediate vehicles for the diagnostics and I don’t want to stall on this as it is the major career avenue I’ve focussed on for the past few years. Can we accommodate this into our company structure and/or plans for the future?
Also, I think the OAE probe is a great device and something I’ve been pushing due to its improvement on what is out there and for its simplicity and achievability. If this slide is disseminated as part of the pitch deck a potential investor may in turn check the feasibility of the plan with external clinicians. OAE in itself is not ‘diagnostic’ as it only tells us info about the sensory effectors (hair cells) not the auditory nervous system beyond that. It’s also hugely affected by any middle ear problems including from colds and flus (eg my OAEs are nearly absent due to retracted eardrums). I’m not criticising, but its something to be very clear about or it could look quite amateur to clinicians if we pitch it as a one-shot simple diagnostic device.”[100]
[99] DJS-35 (Sly 1).
[100] DJS-36 (Sly 1).
65. Dr Sly stated that on 5 August 2015, a shareholder’s meeting was held at which Mr Slater announced that there was agreement to separate the company into the diagnostic and consumer entities shown in the roadmap.[101] He apparently opposed this move and the meeting dissolved. He had no further personal contact with Mr Slater after that time and his involvement in the commercialisation of the consumer headphones was terminated.[102]
The record of invention and provisional patent application
[101] Sly 1 at [69].
[102] Campbell at [57].
66. Dr Sly, Mr Slater and Dr Campbell had met with Ross McFarlane of Phillips Ormonde Fitzpatrick Patent Attorneys early in May 2015 to discuss patenting their invention.[103] A record of invention that would be used to prepare a provisional patent application was subsequently prepared by Dr Campbell and circulated with Dr Sly and Mr Slater on 27 May 2015. The Inventors section provides details for Dr Campbell and Mr Slater, but notably there is a question mark against Dr Sly’s name (“? David Sly”) as inventor and no further details are provided for him.
[103] Campbell at [52].
67. The title of the invention is given as “Automatic personalization of audio signal”. The background section states the headphones use objective or subjective audiometric information to automatically equalize sound to compensate for any hearing damage. A number of prior art documents are then identified. The first is said to be the closest, but this is distinguished on the basis that it reduces the volume to protect the user’s hearing rather than increasing the sound to adjust for the hearing loss and does not describe how to extract the information from the objective measurements. The essential features of the invention are given as follows:
“1) A system for adjusting sound to compensate for an individual’s hearing.
2) A system for detecting the user’s hearing – either manually enter (via an app), or automatically detected via Auditory Evoked Responses (dry skin electrodes or capacitive sensors) and/or Otoacoustic Emissions.”
68. The explanation of the invention sets out the means by which personalization of audio is achieved. The headphone or earbuds have a stored copy of the estimated transfer functions for the user’s ears. An analogue or digital audio signal is input via a wired connection or via a wireless (Bluetooth) module. A DSP makes frequency specific adjustments for amplitude and phase to compensate for a user’s hearing loss. Block diagrams are provided outlining the audio pathways in each ear and the components in headphones and ear buds.
69. The disclosure goes on to describe the way the transfer function is estimated. This includes objective measurements by OAE and AEP. OAEs are said to be best suited for an earbud version rather than full headphones, since OAEs must be measured in a sealed ear canal. Traditional AEPs are said to be impractical since they are measured using wet electrodes after preparing the skin by fine abrasion and the invention will therefore use dry electrodes and/or capacitive sensors. The way in which the transfer function can be generated by each method is then detailed.
70. Subjective measurements use psychophysical audiometric data obtained from an audiologist or via a computer program or app, or from the headphones by having a user tap a button in response to a sound. Pure tone audiograms are said to be the obvious psychophysical test, but an alternative would be to test at the same loudness but different frequencies.
71. Several different types of OAE measurement are said to be useful: stimulus frequency, transient evoked, distortion product, or pulsed OAE. Thresholds are determined at multiple frequencies, or relative amplitudes of OAEs are determined at multiple frequencies to one or more suprathreshold sound levels. Comparisons of amplitude and phase can be compared to response ranges from normal hearing listeners to develop the transfer function for each ear.
72. Any type of AEP is said to be useful, and the document specifically identifies ABR, mid latency response, cABR, EcochG and cochlear microphonic. The transfer function for each ear is determined using frequency specific stimuli and determining either frequency specific thresholds or using one or more suprathreshold sound levels and determining relative amplitudes and/or latencies of responses. Comparison of amplitude and latency or phase can be made with response ranges from normal hearing to develop the transfer function. The exact method to detect a response is different for each AEP since the time course of the characteristic waveform for each AEP is different. Two AEPs are said to merit specific discussion: Auditory steady state response (ASSR) and cABR. ASSR is convenient because the detection of a response is made statistical via published methods. The cABR analyses have been adapted to estimate an ear’s transfer function in response to continuous sound such as music. The document then details the different ways in which high and low frequency transfer functions are determined.
73. The document goes on to provide other variations, including noise cancellation incorporated into the DSP (requiring external microphones), a method for detecting when the headphones are on the head (electrode impedance), or earphones in the ear (using a probe tone and in-ear-canal microphone), and an internal facing microphone to check calibration and effective noise of the environment.
74. Dr Sly responded to the draft document as follows:[104]
“Good going. I spent a bit of time on it, mostly thinking, but few corrections in the end. I suspect the block diagrams will need more work for the provisional submission.
I think the most novel (compared to prior art) and achievable aspect is likely to lie in the use of OAEs in insert earphones.
Related to that tech: I’ve mentioned before that having a mic in the earpiece would also allow ear bud manufacturers to also offer consumers an indication of the true volume in dB SPL they are listening at and a reasonable estimate of noise dose while music is playing. Volume limiters could then be much smarter, if not automatic. We could also get great public data from this in terms of dosimetry vs OAE responses.”
[104] DJS-32 (Sly 1).
75. On 29 May 2015, Dr Sly provided the diagrams that form the basis of Figures 1 and 3 of the current application.[105] The first shows a set of headphones with three sensors arranged on the earphones and headband for use in ABR. The second shows an earpiece with two speakers and a microphone, which I understand to be the general arrangement of components in a DP-OAE probe. Labels and text identifying additional components appear to have been subsequently added to these diagrams for the patent applications.
[105] DJS-33 and DJS-34 (Sly 1).
76. The record of invention was sent to Ross McFarlane on 1 July 2015 for preparation of the Provisional Patent application. Mr Slater subsequently sent an email to Mr McFarlane to clarify that he and Dr Campbell were the sole inventors. Dr Campbell stated that after reviewing the rules on what constitutes inventorship as a group, he, Mr Slater and Dr Sly decided that Dr Sly did not qualify as an inventor and Dr Sly was copied into Mr Slater’s email. He added that Dr Sly “was a little disheartened but fundamentally agreed” that the inventive concept belonged to Mr Slater and Dr Campbell, and in the end the decision was unanimous.[106] Dr Sly disputed this, stating that discussions in relation to filing the patent were paused due to the need for clarity on the IP issues with the University of Melbourne.[107]
[106] Campbell at [54].
[107] Sly 2 at [39].
77. The provisional patent application was filed on 31 August 2015. The specification states that:
“One aspect of the invention provides a method of personalising an auditory stimulus produced by one or more loudspeakers configured to be held in place close to or in a user’s ear, including the steps of:
acquiring objective and/or psychophysical audiometric data characterising (i) the audible threshold of the user across a frequency range or (ii) equal or relative loudness levels of the user across frequency;
estimating a frequency-dependant audio transfer function of the user’s ear from the objective and/or psychophysical audiometric data; and
varying the amplitude of specific frequency components of the auditory stimulus in accordance with the estimated transfer function so as to automatically compensate for hearing loss of the user.”
78. The provisional patent goes on to further describe the means of obtaining the audiometric data:
“The audiometric data can be generated by a variety of means, including subjective observation of the user of the level of intensity of the auditory stimulus, such as the audible threshold, that the user is able to observe at a particular frequency or the like. Alternatively, the psychophysical audiometric data may be obtained by detection of electrophysiological and physiological signals, such as auditory evoked potentials or otoacoustic emissions, detected from the user in response to the application of the auditory stimulus to the user.”
79. The provisional patent subsequently details the use of OAEs and various types of AEP, as well as the way in which transfer functions are generated and used to provide personalised audio. These are consistent with the description given in the record of invention discussed above. I note that four more priority documents were filed. None of these were entered into evidence and I have not given them any further consideration.
The complete specification: identifying the inventive concept
80. The assessment of the entitlement requires consideration of what the specification, as a whole, indicates is the inventive concept. The principles of construction are well established. As Middleton J stated in Eli Lilly and Co Ltd v Apotex Pty Ltd:[108]
“It is well settled that the Court should, from the outset, approach the task of patent construction with a generous measure of common sense. The Court must place itself in the position of a person skilled in the relevant art, being the subject matter of the patent. From this perspective, the patent is to be read as a whole, in the context of the specification and in light of the prevailing common general knowledge and state of the relevant art at the priority date.”
[108] [2013] FCA 214; 100 IPR 451 at 139.
81. The specification as filed comprised 52 claims. I do not intend to analyse the original claims in detail here but note that they include methods of personalizing an audio signal to a user by objective methods (for example Claim 42), including AEP (for example Claim 13) or OAE (for example Claim 9). User generated input (subjective methods) may also be used (for example Claim 10). The claims also include methods of recognising a user using a stored hearing profile for that user (for example Claim 24), and headphones configured to perform the claimed methods (for example Claim 29).
82. Following an examination report dated 10 August 2020, the applicant amended the claims, and the application was subsequently accepted. The claims as accepted comprise 34 claims and are limited to the use of OAE measurements. The requestor argued this was to overcome the novelty and inventive step objections raised in the examination report. The requestor also referred to the applicant’s response to the examiner which stated that the prior art did not disclose a sensing member configured to make an objective measurement of an OAE and compensate for a difference between a hearing profile of a user and a desired hearing profile based on that measurement. The prior art was said to construct the hearing transfer function based on user feedback (a subjective measurement).[109] The requestor argued that this confirms that the use of an objective measurement, and particularly OAEs, is at the heart of the invention. He submitted that the only question in dispute was whether Dr Sly contributed to the idea of using OAEs to compensate for a difference between the user’s hearing profile and a desired hearing profile by modifying an audio signal based on the otoacoustic emission and sending the modified audio signal.[110] The applicant considered the approach taken by the requestor misconceived the inventive concept, improperly relying on the claim language rather than the specification as a whole, which they argued encompasses any means of assessing hearing.[111]
[109] Requestor’s submissions at [46] to [50].
[110] Requestor’s submissions at [14].
[111] Applicant’s submissions at [62].
83. On balance I do not consider the requestor’s submissions in relation to the amendment of the claims during examination are determinative of the inventive concept being related to the use of OAEs. Admittedly, the applicant limited the claims to measurements using OAEs following examination, but I do not consider it necessarily follows that the inventive concept must be read as being limited to this combination of features. There may be any number of commercial or strategic reasons for an applicant to pursue a particular embodiment of an invention. In support of this observation, Dr Campbell acknowledged that subjective methods were dropped in later applications since they appeared to be covered by prior art, but the limitation of the present claims to use of OAEs was consistent with their IP strategy of “rolling follow-on patents”.[112] For example, they have obtained patent protection for headphones using electrocochleography tests in a related US patent.[113] I am also mindful that despite the claims being now limited to the use of OAEs, the time at which the invention was developed and the person by which it is developed is to be ascertained by reference to the inventive concept of the invention so described (in the specification as a whole), and is not affected by subsequent reduction to practice, some of which may have found their way into the claims.[114] As noted previously, the inventive concept is determined from the specification as a whole (including the claims) rather than the form of the claims or the definition of the invention in the claims.[115]
[112] Campbell at [51].
[113] Campbell at [51].
[114] VCT v E-Chem at [17(3)].
[115] VMS v SARB supra.
84. Turning to the specification, the technical field of the invention is described as follows:
“[t]he present invention relates generally to the provision of an auditory stimulus by one or more loudspeakers, such as consumer headphones or earphones, and in particular to the personalization of the auditory stimulus produced by such loudspeakers in order to adjust for the unique characteristics of a user’s hearing.”
85. The background states that headphones are generally designed with the expectation that all users hear in the same way. Standard earphones are unable to take into account differences in conductive and sensorineural hearing in users. Hearing loss is experienced by many people and even people with normal hearing have varying sensitivity to different sound.[116] To select headphones that are bested suited to an individual, they must try a range of headphones and pick those that best fit their hearing based on a subjective assessment of the sound reproduction.[117] Some headphones allow for audio equalization via controls on the headphone itself or via wired or wireless connection to a smartphone app, but these are based on a manual adjustment by the user rather than audiometric information.[118]
[116] Specification at [003].
[117] Specification at [004].
[118] Specification at [006].
86. The problem to be solved is then outlined in the following way:[119]
“It would be desirable to provide a method of personalizing an audio stimulus, produced by one or more loudspeakers configured to be held in place close to or in a user’s ear, that uses subjective and/or objective audiometric information to automatically adapt said auditory stimulus to be well suited to a user’s hearing profile. It would also be desirable to provide a method of personalizing an auditory stimulus produced by one or more loudspeakers that ameliorates or overcomes one or more disadvantages of known sound reproduction techniques.”
[119] Specification at [006].
87. The invention relates to loudspeakers that are held close to, or in, a user’s ear. I understand this to describe headphone arrangements, including cup-type headphones that go over the ears, and earbud-type headphones that are inserted in the ears. The method is said to use subjective and/or objective audiometric information to automatically adapt the auditory stimulus. I confirmed at the hearing that this “automatic” adaptation of the auditory stimulus is not necessarily a real-time process (that is a measurement is taken and the sound profile immediately adjusted). For example, subjective audiometric measurements may be performed by an audiologist and can be provided to the system from an external source.[120]
[120] Specification at [0068].
88. The summary of the invention states that:
“Presented here is an apparatus and method to increase a user’s enjoyment of sound by personalizing an audio signal so that the user perceives the audio signal as if the user had ideal hearing and/or desired hearing. In one embodiment, headphones on a user’s head include a sensor and a speaker. While the speaker plays an audio signal to the user, the sensor records the user’s response to the audio signal. The sensor can be a microphone, a brainwave sensor, an EEG sensor etc. The user’s response can be the audio response inside the user’s ear, the brainwave response associated with the user, the electrical skin response associated with the user etc. Based on the measured response, and based on the knowledge of how people perceive sound, the audio signal is modified to compensate for the difference between the user’s hearing and ideal hearing and/or desired hearing, therefore increasing the user’s enjoyment of sound.”
89. I note here that the headphones incorporate a sensor that may be “a microphone, a brainwave sensor, an EEG sensor etc.” The inclusion of a microphone indicates that OAEs are included, but the inclusion of other sensors indicates that the invention does not only to relate to the measurement of OAEs. This is also consistent with the detailed description of the invention under the heading “Headphones” which commences with a reference to Figures 1 and 2. These figures relate to a set of headphones incorporating dry electrode or capacitive sensors that are positioned to contact the user’s head and measure electrical impulses. Audiometric data may be provided by an external source, and user controls may be provided on the headset to generate subjective user data in response to the user’s perception of auditory stimuli.[121]
[121] Specification at [039] to [043].
90. Figure 3 is an earbud arrangement which is located within the ear canal of one of a user’s ears. The earbud includes two speakers, an internal microphone and an optional external microphone.[122] The arrangement of the internal microphone and speakers allows for a mechanical measurement of DP-OAEs. Other types of OAE may only require one speaker per ear canal.[123] The headphones depicted in Figures 1 and 3 each comprises an electronics module which includes a signal processing unit. The signal processing unit comprises various converters and processors that enable the data from various sources to be analysed and used to adapt the auditory signal to the individual’s hearing profile. Figure 4D illustrates the digital signal processing algorithm for measuring the hearing transfer function[124] and/or the hearing profile[125] associated with a user.[126] Figure 11 is said to depict an embodiment where the OAE probe also functions as a set of headphones for consumer audio use.
[122] Specification at [044].
[123] Specification at [045].
[124] Specification at [0029].
[125] Specification at [0030].
[126] Specification at [0049].
91. A copy of the hearing transfer function for each of the user’s ears is stored together with an estimate of the accuracy and completeness of the hearing transfer function and a user preference for a degree of correction to be applied to the hearing transfer function.[127] When in use, the processor uses the stored hearing transfer function, estimate and user preference to make frequency-specific adjustments for amplitude and phase to automatically compensate for a user’s hearing transfer function.[128]
[127] Specification at [0051].
[128] Specification at [0053].
92. The example referred to in Example 4D and the subsequent discussion of hearing transfer function uses a probe which measures DP-OAE.[129] However the specification states that the hearing transfer function may be generated in a number of ways: subjective measurement, by OAE, AEP, or other objective tests such as the middle ear reflex.[130] The specification goes on to describe various measurements using subjective means,[131] various OAEs[132] and various AEPs.[133] The objective measurement may be used to identify the user and modify the sound to their hearing profile.[134]
[129] Specification at [0047].
[130] Specification at [0067].
[131] Specification at [0068] to [0070].
[132] Specification at [0071] to [[0079].
[133] Specification at [0080] to [00105].
[134] Specification at [00106] to [00113].
93. On balance, I consider that the inventive concept described in the specification, as a whole, is the automated provision of personalised audio in consumer headphones using feedback from a user’s hearing responses. The responses may be obtained by subjective and/or objective means to generate a hearing profile that can be used to adjust the audio being received by the user. The emphasis in the application is on the use of test data to generate a hearing transfer profile that can deliver personalised audio to the user. There is no basis in the application as filed for me to conclude that the inventive concept is specific to the use of OAEs, even though the claims now comprise this feature.
Did Dr Sly make a material contribution to the inventive concept?
94. Having identified the inventive concept lies in the automated provision of personalised audio using information obtained from objective and/or subjective testing methods, the question is whether Dr Sly made a material contribution to this concept.
95. Dr Sly’s contention was that he contributed the inventive concept that OAE was the “solution”, or at least the best method, of providing personalised audio – this being central to the present claims. In this regard, Dr Sly appears to work backwards from the claimed subject matter, stating that:[135]
“…the complete conception of the solution described in the claims of the Patent Application required several critical ideas, including:
(a) The idea of creating consumer-grade headphones that could conduct objective hearing tests, including OAEs;
(b) The idea of providing personalised audio by taking the results of the tests from (a) above and using these to equalize/optimize audio output in the same headphone; and,
(c) The idea that OAEs were the ideal test and solution to providing personalised feedback to headphone audio.”
[135] Sly 1 at [7].
96. Dr Sly considered that each of ideas (a) and (b) above are contingent on the last idea (c). He stated that idea (a) related to work that he conducted between 2011 and 2015 at the University of Melbourne.[136] Ideas (b) and (c) required knowledge of (a) and were conceived around the time that Nuraloop was formed.[137] Dr Sly also stated that the concept of personalised audio conceived by Dr Campbell and Mr Slater was already known in the public domain, including transfer functions,[138] and the “novel concept is the use of an objective measurement of this response, and in particular the use of OAEs for this objective measurement,” referencing documents from the patent examination report on the present application.[139] He stated that:
“The conception of the commercial solution of the independent claims of the Patent Application was complete when I conceived idea (c)… that is, using the measurement of OAEs as objective measurements. The remainder of the disclosure, including the embodiments, of the Patent Application could have been completed by anyone reasonably skilled in the art at the priority date in question. The steps required to implement ideas (a) to (c)… into headphones, earphones and the sensors and speakers and circuitry required, would all have been obvious to those skilled in the art, and were certainly obvious to both Campbell and I in light of our prior work in developing hearing diagnostic headphones. In addition, transfer functions were common knowledge at the priority date in question, especially to those with expertise in hearing loss and hearing augmentation (hearing aids) and the implementation described in the Patent Application is very basic and self-explanatory as are the various types of OAEs which I have referred to in my previous work.”[140]
[136] Sly 1 at [9].
[137] Sly 1 at [10].
[138] Sly 2 at [41].
[139] Sly 2 at [8].
[140] Sly 1 at [11].
97. Dr Sly has broken the question of “conception” of the claimed invention (or as he calls it, “the solution”) into three separate and discrete “ideas” with an assessment of the inventiveness of each separate step in order to arrive at the “inventive” contribution. I do not consider this type of ex post facto analysis of the final invention to be of much assistance here since, as set out in VMS v SARB, entitlement is not determined by reference to the inventiveness of the invention claimed in the patent but with reference to the history of the inventive concept in the minds of the inventors.[141]
[141] VMS v SARB at [68]-[69].
98. At hearing the requestor submitted that the chronology of events is consistent with the conclusion that the Nuraloop technology was a development of the technology that Dr Sly had worked on since 2009 and used the headphones (including the OAE probe) that he had developed for use as a diagnostic tool.[142] Dr Sly stated that idea (a) related to work that he carried out between 2011 and 2015 and that the earliest description of consumer headphones incorporating OAE monitoring was presented in his iFund application on 13 November 2011.[143] This is a significant point of divergence in the evidence. Dr Campbell stated that it was never the aim of the Hearing Diagnostic Project that the diagnostic equipment could function as a normal headphone for listening to music, or that the information obtained from measuring a user’s hearing could be fed back and used to improve the sound of the music in real time by modifying the audio signal.[144] Based on my consideration of the evidence, including the various grant applications, I am led to prefer Dr Campbell’s evidence on this point.
[142] Requestor’s submissions at [119].
[143] Sly 1 at [22].
[144] Campbell at [15].
99. As an initial observation, Dr Sly’s contention relates to the use of OAEs, with the submissions at hearing focussing on OAE ear probes. The present specification makes use of an objective measurement, optionally in combination with a subjective measurement, and I understand that the devices described incorporate one type of objective measurement. In contrast the evidence from before March 2015 suggests the goal of the hearings diagnostic group had been to develop multimodal diagnostic headphones integrating audiometry, ABR and OAE. There is no evidence to suggest that there had been an intention to develop a standalone OAE (or ABR) diagnostic system. Dr Sly later indicated (in July 2015) that he was keen to pursue an OAE-alone device “due to its improvement on what is out there and for its simplicity and achievability” but considered that audiologists were unlikely to “be impressed” with an OAE-alone device since other devices were already available that combined OAE with another testing method. He noted OAE measurements are also affected by middle ear problems and therefore rarely used solely as a diagnostic,[145] and considered an OAE-only device could still be used for screening purposes (where the test is used to indicate a need for more comprehensive testing), as an additional tool in an audiologist’s toolkit, or combined with other new test methods. These limitations are consistent with the goal of the hearings diagnostic group prior to March 2015 of developing a multimodal diagnostic headphone where OAE is used in combination with another test method.
[145] DJS-36 (Sly 1).
Moreover, the only headphones that had been developed to a prototype stage were an audiometric-based system. The biosensor and OAE probes had been developed to a point where they were able to obtain data from subjects, with the ABR system being at a more advanced stage, but the integrated headphones had not been fully developed in time for the Defence trials in 2015.[146] There is also no clear evidence that the early-stage ABR and OAE systems, or the conceptual multimodal diagnostic headphones, were intended for, or even capable of, use as a consumer headphone. In this regard I note the differences in the components shown in Figure 3 (that is, an OAE probe for a single ear) and the embodiment shown in Figure 11 which is said to be an embodiment where two OAE probes function together as a set of headphones for consumer use.[147] In short, while the present invention is related to aspects of the work done in the hearing diagnostics project, the evidence does not support the assertion that this included the concept of consumer headphones incorporating an objective test method.
[146] Sly 2 at [62].
[147] Specification at [00113].
That is not to say that the inventive concept turns on this specific point. If the identification of OAE as a method of providing the data for use in the delivery of personalised audio somehow materially contributed to the inventive concept identified above, and Dr Sly provided this contribution, then he could be considered an inventor. For example, there might difficulties in implementing the use of OAE for delivering personalised audio. Alternatively, had the brainstorming session led to the proposal of a specific objective measurement technique such as ABR to the exclusion of any other, but the subsequent proposal of OAE measurement resulted in the method being recognised as being generally applicable to the full range of objective hearing tests, then this could be considered as contributing to the inventive concept. However, I do not consider that the evidence clearly establishes this to be the case.
It is not in dispute that Dr Campbell and Mr Slater conceived the idea of personalising audio in consumer headphones. This was said to have arisen from the idea of taking hearing diagnostics out of the lab and making it useful more broadly. Admittedly there is no documentary evidence of the discussions, or indeed of any subsequent development of the early ideas. But to the same extent, Dr Sly has not stated explicitly what was relayed to him by Dr Campbell and Mr Slater. I must rely on the contemporaneous evidence in this regard.
Mr Slater and Dr Campbell were both members of the hearing diagnostic project and were involved in the work towards incorporating OAE and ABR into a single device at the time of the brainstorming session. There is no evidence to suggest that Dr Campbell and Mr Slater were working in a vacuum and would not have been aware of OAE (or indeed any of the other testing methods envisaged in the hearings diagnostic work) when considering how to make hearing tests more broadly applicable. In this regard, Dr Campbell stated that any objective technique can be used as a source of the data for generating the hearing profile, and he and Mr Slater considered both OAE and ABR during their brainstorming session. This appears consistent with the MTGT application that was drafted shortly after the brainstorming session and refers to the delivery of “real-time” personalised audio using tests to obtain information from both the cochlea and auditory nerve. I consider this to be consistent with the evidence of Dr Campbell to the extent that the brainstorming session had established that the method was generally applicable across different objective hearing tests.
Dr Sly stated that he had the necessary knowledge and expertise to introduce Dr Campbell to techniques such as ASSR and cABR, but there is otherwise no evidence that Dr Campbell worked solely under the direction of Dr Sly. To the contrary, the evidence shows that even in the early stages of the project in 2009, Dr Campbell had sufficient knowledge of ABR and OAE to discuss in some detail the types of equipment, including hardware, that would be needed in a diagnostic app. Dr Campbell had also been named as a researcher (or co-investigator) on projects since 2009 and had oversight of hardware development and programming of nerve-based diagnostic equipment, including EcochG.[148] Dr Sly is certainly an experienced electrophysiologist with a high level of expertise and experience in techniques including OAE, ASSR and electrocochleography. However, the evidence shows that Dr Campbell was also involved in research into the use of such techniques in diagnostics, including the use of ABR and OAEs, and would have brought this background knowledge to the brainstorming session. To the extent that this may have drawn on intellectual property attributed to Dr Sly as set out in the various grant applications and disclosed at the ORLIAC conference,[149] that is a separate issue to the question of inventorship, and I make no further comment in that regard.
[148] DJS-13 (Sly 1), see Grant proposal at page 24.
[149] Sly 2 at [65].
I acknowledge that Dr Sly was involved in subsequent discussions within the group, providing background information and suggestions about different hearing tests. However, these were largely in relation to eking out a commercial position by identifying the closest competitors from a patent search and commenting on areas that the group could focus on, could be a differentiator or that could be the “most novel and achievable” or “easier” aspects. I do not see these as being core to the inventive concept I have identified from the complete specification, but rather are more consistent with the actions of reducing the invention to practice. Indeed, the references to OAEs being “ideal”, “easier” or even “commercial” implicitly acknowledge that the use of OAE measurements is not the only method by which personalised audio could be achieved.
Moreover, there is no reasoning in Dr Sly’s emails in relation to OAEs that would clearly direct a reader to consider that OAEs are essential to the delivery of personalised audio (or as Dr Sly put it, the solution to providing personalised audio), rather than one of a number of ways that the invention can be reduced to practice. Indeed, this followed emails expressed in more definite terms suggesting that the group should focus on the auditory nerve and identifying other areas of differentiation from the prior art. In comparison, the fairly vague suggestion that OAEs could be easier appears analogous with the principle that a claim to entitlement will not be established where the party’s claimed contribution to the inventive concept is merely “positing desiderata or making a vague proposal”.[150]
[150] VMS v SARB at [70]-[72].
In conclusion, I do not consider that the evidence provided establishes that Dr Sly made a material contribution to the inventive concept of the use of the information obtained from objective and/or subjective hearing tests to personalise the audio received by a user. The contribution provided by Dr Sly relates to the reduction of the inventive concept to practice, and not to the inventive concept itself.
Costs
Costs generally follow the event. The request has been unsuccessful. I therefore award costs against the requestor, Dr David Sly.
Dr Leslie F. McCaffery
Delegate of the Commissioner of Patents
0
0
0