Paige.AI, Inc.
[2023] APO 44
•4 September 2023
IP AUSTRALIA
AUSTRALIAN PATENT OFFICE
Paige.AI, Inc. [2023] APO 44
Patent Application: 2020276112
Title:Systems and methods for processing images to classify the processed images for digital pathology
Patent Applicant: Paige.AI, Inc.
Delegate: Andrew Burgess
Decision Date: 4 September 2023
Hearing Date: Written submissions filed on 2 May 2023
Catchwords: PATENTS – Examiner objection – quality assurance of pathology samples with a machine learning model - claims lack manner of manufacture – claims lack inventive step – opportunity to amend
Representation: Patent attorney for the applicant: Davies Collison Cave
IP AUSTRALIA
AUSTRALIAN PATENT OFFICE
Patent Application: 2020276112
Title:Systems and methods for processing images to classify the processed images for digital pathology
Patent Applicant: Paige.AI, Inc.
Date of Decision: 4 September 2023
DECISION
I consider that the invention defined by claims 1-12 do not comprise a patentable manner of manufacture, and claims 1, 5, 7, and 11 do not involve an inventive step.
The Applicant is provided an opportunity to respond to attempt to gain acceptance. Under Regulation 13.4(3) the final date to gain acceptance is extended to six (6) months from the date of this decision.
REASONS FOR DECISION
Background
Patent application 2020276112 (the application) was filed by Paige.AI, Inc (the Applicant) on 15 May 2020 under the provisions of the Patent Cooperation Treaty. The application claims priority from US62/848703 which was filed on 16 May 2019, and entered national phase in Australia on 13 December 2021.
On 31 January 2022, the Applicant requested expedited examination under the provisions of the Global Patent Prosecution Highway (GPPH), and simultaneously proposed amendments in anticipation of examination (the first amendment) to amend the claims to substantially correspond with claims granted in US 10891550 B2. A first examination report was issued on 11 March 2022, objecting on the grounds of lack of manner of manufacture, clarity, novelty, and inventive step. The Applicant filed amendments (the second amendment) and responding comments on 27 July 2022. A second examination report issued on 19 August 2022, maintaining objections of lack of manner of manufacture and inventive step. The Applicant filed further amendments (the third amendment) and responding comments on 23 December 2022. A third examination report was issued on 31 January 2023, maintaining objections of lack of manner of manufacture and inventive step and introducing objections to clarity and lack of unity. The Applicant filed further amendments (the fourth amendment) and responding comments on 15 February 2023. A fourth examination report was issued on 3 March 2023, maintaining objections for lack of manner of manufacture and inventive step. On 9 March 2023, the Applicant requested to be heard in relation to the outstanding objections from the fourth examination report.
Pursuant to Regulation 13.4(1)(b), the final date for gaining acceptance was 11 March 2023. Pursuant to Regulation 13.4(1)(g), when the Commissioner gives the applicant an opportunity to be heard in relation to a report under Section 45 and issues a written decision, the period for gaining acceptance of the application is extended until three months from the date the decision is issued. Under Regulation 13.4(3), I may substitute the three-month period for a longer period if I am satisfied that it is appropriate.
The Applicant filed their written submissions on 2 May 2023.
Applicable Law and the Standard of Proof
The examination of the present application is governed by the Patents Act 1990 (the Act) as amended by the Intellectual Property Laws Amendment (Raising the Bar) Act 2012 (the Raising the Bar Act). The standard of proof that applies to the examination of the present application is therefore the balance of probabilities.
I must accept the patent request and complete specification under Section 49 if I am satisfied, on the balance of probabilities, that it complies with the requirements set forth in that section, which includes, inter alia, the requirements of Section 18(1)(a) and (b) and Section 40(2) to 40(4). If I am not so satisfied, I may refuse the application. However, I will only refuse the application if I am also satisfied that providing the Applicant with an opportunity to amend will serve no useful purpose; for example, if I consider that any potential negative findings are not rectifiable by an allowable amendment.
The invention as described
The specification presently under consideration comprises description pages 1-2 as filed in the third amendment, description pages 3-36 and claim pages 37-42 as filed in the fourth amendment, and drawing pages 1/10 to 10/10 of the PCT pamphlet.
The application relates to the field of digital pathology. The background section explains that, in order to effectively use them, it is important for digital pathology images to be accurately categorised with respect to specimen tissue type and other parameters, which information is often stored in a laboratory information system (LIS). However, the LIS may have inaccuracies or be incomplete, or may not be available to researchers due to containing sensitive information[1]. The application in general describes a number of machine learning based tools to address or alleviate issues associated with categorising pathology images without requiring LIS data/access.
[1] Specification at [003].
More specifically, the presently claimed invention relates to a method/system for performing quality control related to digital pathology images. The specification very briefly describes the preparation of pathology samples as cutting the sample into sections, applying appropriate stains, and preparing the slide[2]. The specification mentions a range of stains may be used, most commonly Hematoxylin and Eosin (H&E), and additionally/alternatively other techniques such as immunohistochemistry[3]. The details of these techniques are not discussed.
[2] Specification at [037]
[3] Specification at [041]
The specification in general is quite repetitious (for example, [017]-[021] repeats [011]-[015] almost verbatim, and the sentence providing example sources for pathology images is repeated fourteen times throughout the specification), with very similar methods and processes being described multiple times for various use-cases (i.e., tool for determining specimen property at [077]-[084] and Fig. 3, tool for determining specimen type at [085]-[087] and Fig. 4, tool for specimen classification and quality control at [088]-[091] and Fig. 5, and tool for determining prior effect at [092]-[095] and Fig. 6). For the quality control method/system, the relevant paragraphs of the specification appear to be [048], [069], [088]-[091], and Fig. 5. The rest of the specification is directed to either different tools (e.g. Fig. 6 and the prior tissue treatment identification tool) or broader, more abstract descriptions of the invention.
The preferred embodiment of the quality assurance process is exemplified in Figure 5, reproduced below.
Generating the model is described[4] as essentially receiving (or generating) a dataset of digital pathology samples with examples of quality defects and using the dataset to train a machine learning model to classify images by outputting label(s) indicating the presence of a quality defect and/or an overall quality score for the image. The specification states the training may be performed by a machine learning model (i.e., a machine learning model is used to train a machine learning model), however this step does not appear essential as it is not described in detail.
[4] Specification at [089]-[090]
Regarding the training set of images[5], the specification repeatedly notes that training may come from real sources or “synthetic sources”[6], which I understand to refer to either augmented images (real images which have been modified with noise/flipping/distortions[7]) or images fully synthesised from models[8]. Notably, the specification does not provide details of the process for producing synthetic images beyond the mere reference to graphics rendering engines and 3D models. I also note that the detailed description refers to a real or synthetic dataset, while the consistory paragraphs refer to training data comprising real images and synthetic images[9].
[5] Specification at [089]
[6] E.g., specification at [010], [063], [070], [073], [089]
[7] Specification at [070]
[8] Specification at [063]
[9] Specification at [010]
When the model is applied to a target pathology image[10], it determines if a quality issue(s) is present (e.g., poorly cut specimen, scanning artifacts, etc.) and an overall quality score, and may recommend action (e.g., rescan of image, recut of the specimen, etc.) to mitigate the quality issue(s).
[10] Specification at [088], [091]
As a concise summary, I consider the aim of the QC tool is to determine/detect a quality issue, and to recommend an action to mitigate the issue.
Examiner’s Objections
As noted above, the fourth report maintained objections relating to lack of manner of manufacture and lack of inventive step. These remaining objections are quite lengthy and in places makes reference to comments from earlier reports and comments from the Applicant.
Regarding manner of manufacture, the remaining objection relevantly states:
“However, an assessment of manner of manufacture for computer-implemented inventions requires an assessment of the substance of the invention, as discussed in previous examination reports, in particular the question of whether the computer is merely the intermediary, configured to carry out the method, but adding nothing to the substance of the idea. In the current application, the classification of images represents the substance of the idea, while the use of image processing and machine learning represents the technical form of the invention. There is no identified technical problem which has been overcome by the use of machine learning to identify particular defects in the pathology images, or the particular defects which are detected, or in the recommendations in response to those identified defects. The claimed invention simply uses routine image classification techniques, and routine techniques do not contribute substance.” (emphasis added)
And:
“However, such a broadly-claimed quality score is very non-specific, and is considered to include within its scope a measure of risk that a defective image is not rectified. The recited quality control issues of poorly cut specimen sections, scanning artifacts, damaged slides, and markings on slides are all subjective quality control measures, and the claim recites that action is taken if the quality score is below a predetermined threshold. The specification states that ‘the quality score determiner module 133 may identify quality control (QC) issues (e.g., imperfections) for the training images at a global or local level that may greatly affect the usability of a digital pathology image’ (para [052]). Thus this predetermined threshold represents the point at which there is an unacceptable risk of determining ‘a slide to be insufficient to make a diagnosis’ (para [027]).” (emphasis added)
And:
“I agree that image processing itself is a technical process; however the claimed invention simply makes use of standard image processing techniques, and therefore this technical environment represents the form of the invention rather than its substance. The substance, as discussed in previous examination reports, concerns the classification of pathology images according to identified quality control issues, rather than any technical aspects of image recognition, via machine learning or otherwise.” (emphasis added)
Regarding inventive step, the fourth report relevantly states:
“The invention defined in claims 1-12 does not involve an inventive step in light of D1, in further light of common general knowledge, as discussed in the previous examination report for this case.
Independent claims 1 and 7 have been amended to specify that the determined quality control issues include at least one of poorly cut specimen sections, scanning artifacts, damaged slides, and markings on slides, and that the recommendations explicitly address those identified issues (as recited in previous claim 6). However, as discussed in the previous examination report regarding previous claim 6, these are specific and obvious quality control issues with obvious recommendations arising directly from the identification, which cannot be considered to be inventive.” (emphasis added)
And:
“However I consider that it is obvious to a person skilled in the art that a machine learning model would not be trained with just any images, but would instead be trained with images which include the specific issues which the machine learning model is intended to identify. It is clear from the specification that the contemplated issues such as poorly cut specimen sections, scanning artefacts, damaged slides and markings on slides are all well know (sic) in the art and will each affect pathology images in a distinct manner. The person skilled in the art would thus immediately perceive the necessity of training any machine learning system to identify each distinct issue. Furthermore, the Manual section 2.5.3.6 also states that ‘it cannot be said that the invention lies in identifying the real nature of the problem if the nature of the problem is obvious’. In your case, I believe that the nature of the problem is obvious - to identify specific control issues, it is obvious that the image recognition model must be trained on images containing those specific issues.” (emphasis added)
And:
“However, the claimed invention merely recommends:
·performing a specimen cut in response to detecting poorly cut specimen sections
·performing a scan using the scanning parameter in response to detecting scanning artifacts
·performing a slide reconstruction in response to detecting damaged slides, and
·performing a slide marking in response to detecting markings on slides
These are all obvious recommendations which directly arise from the identified issues, without invention.” (emphasis added)
The invention as claimed
The application includes 12 claims, with a full copy included as an annex to this decision. The independent claims are (with indenting/feature numbering added for clarity/reference):
Claim 1: A method of performing quality control with respect to pathology specimen preparation by analyzing an image corresponding to a specimen, the method comprising:
1.1 receiving a target image corresponding to a target specimen, the target specimen comprising a tissue sample of a patient;
1.2 applying a machine learning system to the target image to determine at least one characteristic of the target specimen and/or at least one characteristic of the target image, the at least one characteristic being indicative of a quality control issue, including at least one of
1.2.1 poorly cut specimen sections,
1.2.2 scanning artifacts,
1.2.3 damaged slides, and
1.2.4 markings on slides,
1.3 and the machine learning system having been generated by processing a plurality of training images to predict at least one characteristic,
1.4 the training images comprising real images of human tissue and at least some synthetic images, and wherein at least some of the training images include examples of quality control issues;
1.5 outputting the at least one characteristic of the target specimen and/or the at least one characteristic of the target image;
1.6 identifying a quality score for the target image, the quality score being determined according to the machine learning system;
1.7 determining whether the quality score for the target image is less than a predetermined value;
1.8 in response to the quality score for the target image being less than the predetermined value, outputting a recommendation for increasing the quality score for the target image, wherein the recommendation comprises any one or any combination of
1.8.1 a specimen cut,
1.8.2 a scanning parameter,
1.8.3 a slide reconstruction, and
1.8.4 a slide marking;
1.9 and, using the recommendation to receive a new target image, by performing any one or any combination of:
1.9.1 performing a specimen cut;
1.9.2 performing a scan using the scanning parameter;
1.9.3 performing a slide reconstruction; and
1.9.4 performing a slide marking.
And
Claim 7: A system for performing quality control with respect to pathology specimen preparation by analyzing an image corresponding to a specimen, the system comprising:
7.1 at least one memory storing instructions; and
7.2 at least one processor executing the instructions to perform operations comprising:
7.3 receiving a target image corresponding to a target specimen, the target specimen comprising a tissue sample of a patient;
7.4 applying a machine learning system to the target image to determine at least one characteristic of the target specimen and/or at least one characteristic of the target image, the at least one characteristic being indicative of a quality control issue, including at least one of
7.4.1 poorly cut specimen sections,
7.4.2 scanning artifacts,
7.4.3 damaged slides, and
7.4.4 markings on slides, and
7.5 the machine learning system having been generated by processing a plurality of training images to predict at least one characteristic,
7.6 the training images comprising real images of human tissue and at least some synthetic images, and wherein at least some of the training images include examples of quality control issues;
7.7 outputting the at least one characteristic of the target specimen and/or the at least one characteristic of the target image;
7.8 identifying a quality score for the target image, the quality score being determined according to the machine learning system;
7.9 determining whether the quality score for the target image is less than a predetermined value;
7.10 in response to the quality score for the target image being less than the predetermined value, outputting a recommendation for increasing a quality of the target image,
7.11 wherein the recommendation comprises any one or any combination of
7.11.1 a specimen cut,
7.11.2 a scanning parameter,
7.11.3 a slide reconstruction, and
7.11.4 a slide marking; and,
7.12 using the recommendation to receive a new target image, by performing any one or any combination of:
7.12.1 performing a specimen cut;
7.12.2 performing a scan using the scanning parameter;
7.12.3 performing a slide reconstruction; and
7.12.4 performing a slide marking.
For the sake of brevity, I will refer to integers 1.2.1-1.2.4 (and corresponding 7.4.1-7.4.4) as the quality defects[11], integers 1.8.1-1.8.4 (and corresponding 7.11.1-7.11.4) as the recommended actions, and 1.9.1-1.9.4 (and corresponding 7.12.1-7.12.4) as the remedial actions.
[11] Note I will reserve the term “quality control issue” for a general class of issues, and “quality control defect” for subset of issues defined by integers 1.2.1-1.2.4.
In terms of construction, I note the comments of Middleton J. in Eli Lilly and Company v Apotex Pty Ltd[12] at 139:
“It is well settled that the Court should, from the outset, approach the task of patent construction with a generous measure of common sense. The Court must place itself in the position of a person skilled in the relevant art, being the subject matter of the patent. From this perspective, the patent is to be read as a whole, in the context of the specification and in light of the prevailing common general knowledge and state of the relevant art at the priority date.”
[12] [2013] FCA 214
The Applicant has not presented detailed comments directed to the construction of the claims[13], which I take to mean there was no particular disagreement on this point. However, there are several points which I think need to be more closely inspected.
[13] Note the distinction between construction and characterisation.
The person skilled in the art
The person skilled in the art in this case is likely a team comprising a pathologist, a technician, and a machine learning specialist. While I do not have the benefit of expert testimony from any of these professions, I will bear this perspective in mind in my construction of the claims.
Construction
1.2 applying a machine learning system to the target image to determine at least one characteristic of the target specimen and/or at least one characteristic of the target image, the at least one characteristic being indicative of a quality control issue, including at least one of [the listed quality defects]
As a first point, by the plain meaning of the above the claim encompasses a machine learning system which determines only one characteristic. Additional characteristics may be determined, but only one is required by the claim.
The claim is worded so that the characteristic is indicative of a quality control issue, in the singular, but is then followed by the statement “including at least one of” the listed quality defects. At face value, this suggests that the singular quality issue may be one quality defect, or it may be a combination of multiple quality defects. An example of this may be determining that there are very few sharp lines/edges in the image (a characteristic of low sharpness), which may be indicative that the scan was out of focus (scanning parameter issue) and/or that the specimen was poorly cut.
Note that the scope is inclusive of where the at least one characteristic is indicative of a quality issue which itself is a single, specific quality defect, however, the phrase is not limited to that. Limiting the scope in such a way would require the last part of 1.2 to be read as “the at least one characteristic being indicative of one of [the quality defects]”. That is, I would only arrive at the more limited scope by discarding words from the claim. Not all claim drafting is flawless, but in general I should seek to give meaning to each word of the claim, which in this case results in the claim scope inclusively encompassing where each characteristic is indicative of singular or multiple quality issues, where quality issues comprise one or more quality defects.
As a brief note, the Macquarie Dictionary defines “indicative” as “pointing out” or “suggestive”. I take this to mean the characteristic determined by the machine learning system need not be specifically determinative of a quality defect, but the characteristic must be such as to suggest (in a practical and useful sense) the presence of a quality issue. To put that in slightly more concrete terms, a method otherwise falling within scope will not be excluded because of occasional false readings.
1.4 the training images comprising real images of human tissue and at least some synthetic images, and wherein at least some of the training images include examples of quality control issues;
“Synthetic images” is never expressly defined, however, as I noted above it is reasonably clear it simply means where the image is not a real digital pathology image. The specification briefly mentions augmented images[14] and model generated images[15]. Either would appear to be synthetic.
[14] Specification at [070]
[15] Specification at [063]
Notably, the claim requires that the training data includes “examples of quality control issues”, not specifically the one or more quality defects. However, based on my construction above that the quality issue comprises one or more of the quality defects, it follows the training data must include examples of one or more quality defects.
1.6 identifying a quality score for the target image, the quality score being determined according to the machine learning system;
The specific meaning of “quality score’ is not defined in the claims. “Score” is normally regarded as a numerical value (e.g., Macquarie Dictionary) but need not always be (e.g., a star rating may be regarded as a score). Looking to the specification, it is described as being determined on the basis of “overall quality of the pathology slide itself, or tissue morphology characteristics”[16]. In the context of the claim, I consider a person skilled in the art would recognise there are a range of measures that may serve as an “overall quality score”; for example number of quality issues detected, percentage of area affected by quality issues, or significance of quality issues (e.g., issues which make pathological diagnosis more difficult but still possible). I also note the quality score need not be directly related to the quality defects determined at integer 1.2 – the overall quality score may reflect the quality defects, other quality issues, or a combination of both. Furthermore, given the nature of machine learning models, the score may relate to the product of a complex multivariable algorithm which may not map directly to anything a human would intuitively recognise as being indicative of quality, but that the model has nevertheless determined correlates with quality.
[16] Specification at [064]
The specification notes that quality issues/score may be local (only a portion of the image) or global (relating to the image as a whole). Given the claim defines the quality score as “for the target image” (not a portion thereof, or a plurality of scores), I consider the claimed quality score relates to a “global” score.
1.8 in response to the quality score for the target image being less than the predetermined value, outputting a recommendation for increasing the quality score for the target image, wherein the recommendation comprises any one or any combination of [the recommended actions]
There are a couple of notable points about this integer. Firstly, the recommendation is triggered by the quality score being below the predetermined value. As I have noted above, there is no requirement for the quality score to be directly based upon the quality defects; however it would be unreasonable to expect the quality score to not be impacted by the quality defects. A highly severe or plurality of quality defects can be presumed to reduce the quality score. Relatedly, the way the present claim is drafted it is clear a higher quality score is better – so quality score is not merely a count of defects.
Secondly, there is again some ambiguity between singular and plurals. As written, a singular recommendation is made, which comprises one or more of the recommended actions. The singular recommendation might, for example, indicate a need to check the scan parameters and rescan the slide, or a need to reconstruct slide and rescan it.
Thirdly, it is noted that the recommendation is “for increasing the quality score”. If the quality score relates to overall quality, it follows any recommended action which addresses any present quality issue may improve the quality score, even if it does not address all of the quality issues. Where the recommendation includes multiple recommended actions, so long as one of the recommended actions relates to a present quality issue, undertaking the action may be reasonably expected to increase the quality score.
The quality defects, recommended actions, and remedial actions.
The quality defects are listed as 1.2.1 poorly cut specimens, 1.2.2 scanning artifacts, 1.2.3 damaged slides, and 1.2.4 markings on slides.
The specification does not provide a clear definition or examples of what constitutes a poorly cut specimen, so I have interpreted it broadly. It is inclusive any issue which may be attributable to the cutting stage of preparation. Within this art, such issues might be where the section was cut too thick/thin, unevenly, with a defective cutting tool, or cut with an incorrect orientation.
The specification does not provide a clear definition or examples of what constitutes a scanning artifact, so I have interpreted it broadly as inclusive of any defect associated with the capturing of the digital image. These might include blurred patches, missing pixels, lighting reflections, and potentially many other defects.
The specification does not provide a clear definition or examples of what constitutes markings on slides. I note that the associated remedial action is to perform a slide marking rather than remove the marking, implying a defect may include a lack of specific markers on the slide. However, in the absence of further guidance from the specification, I consider marking should be construed broadly as inclusive of unintentional marks (e.g., finger marks, scuffs), or incorrect/missing intentional markings (e.g., slide labels, or staining).
I also note that in each category of quality defect, there may be binary determinations (the slide is either cracked or it is not) or it may be a spectrum (a marking on the slide may be an almost imperceptible speck or obscure the whole slide). The claim as presented determines a defect is present, not its severity.
As an observation, these defects, when taken as a whole, would appear to encompass almost any quality error that might reasonably impact on the useability of the sample.
The recommended actions and remedial actions have an obvious correspondence with the quality defects, where each recommendation may address one of the defects detected. However, it should be noted that the claim does not define that the recommended action is selected on the basis of the defect detected, but rather on the basis of improving the quality score. While it is reasonable that the recommended action most likely to increase the quality score is the one associated with the quality defect detected, it should also be noted that the recommendation may be a combination of multiple recommended actions.
Regarding “scanning parameter”, the claim defines recommending a scanning parameter and that the remedial action is to perform using the scanning parameter. This suggests that the machine learning model does not merely detect which parameter is causing the scanning artifacts, but also determines what the correct parameter setting should be.
I also note that some of the recommended/remedial actions are not independent: where a specimen is to be re-cut, it is unavoidable that there will need to be a slide reconstruction, new markings, and a new scan.
Manner of Manufacture
Subsection 18(1) of the Patents Act states that:
“Subject to subsection (2), an invention is a patentable invention for the purposes of a standard patent if the invention, so far as claimed in any claim:
(a)is a manner of manufacture within the meaning of section 6 of the Statute of Monopolies”
Case law and legal principles
The concept of manner of manufacture has developed over time and is not readily reduced to a simple formula. The classic definition of manner of manufacture is set out in National Research Development Corporation v Commissioner of Patents[17] at [14]:
“The inquiry which the definition demands is an inquiry into the scope of the permissible subject matter of letters patent and grants of privilege protected by the section ... The right question is: ‘Is this a proper subject of letters patent according to the principles which have been developed for the application of s 6 of the Statute of Monopolies?’”
[17] [1959] HCA 67; 102 CLR 252 (NRDC)
The High Court has consistently made it plain that NRDC, and all other cases, were not laying down a precise formulation that can be applied unthinkingly. This is stated in Apotex Pty Ltd v Sanofi-Aventis Australia Pty Ltd[18] at [83]:
“Nothing said in the Court's reasons for decision in that case can be taken as an exact verbal formula which alone captures the breadth of the ideas to which effect must be given.”
[18] [2013] HCA 50; 253 CLR 284 (Apotex)
And in D'Arcy v Myriad Genetics Inc[19] at [23]:
“This Court in NRDC did not prescribe a well-defined pathway for the development of the concept of ‘manner of manufacture’ in its application to unimagined technologies with unimagined characteristics and implications. Rather, it authorised a case-by-case methodology.”
[19] [2015] HCA 35; 258 CLR 334 (Myriad)
That case-by-case approach must have regard to the substance of the claimed invention, not simply the form of the claim[20]. The point is made most succinctly by Gageler and Nettle JJ in the Myriad case at [144]:
“Whatever words have been used, the matter must be looked at as one of substance and effect must be given to the true nature of the claim.”
[20] Myriad at [6] and [88].
In Commissioner of Patents v RPL Central Pty Ltd[21] the Full Court of the Federal Court said the same thing in the context of a computer-implemented invention at [96]-[98]:
“A claimed invention must be examined to ascertain whether it is in substance a scheme or plan or whether it can broadly be described as an improvement in computer technology. The basis for the analysis starts with the fact that a business method, or mere scheme, is not, per se, patentable. The fact that it is a scheme or business method does not exclude it from properly being the subject of letters patent, but it must be more than that. There must be more than an abstract idea; it must involve the creation of an artificial state of affairs where the computer is integral to the invention, rather than a mere tool in which the invention is performed. Where the claimed invention is to a computerised business method, the invention must lie in that computerisation. It is not a patentable invention simply to ‘put’ a business method ‘into’ a computer to implement the business method using the computer for its well- known and understood functions.
Is the mere implementation of an abstract idea in a well-known machine sufficient to render patentable subject matter? Is the artificial effect that arises, because information is stored in RAM and there is communication over the Internet or Wi-Fi, sufficient? Does any physical effect give rise to a manner of manufacture? Are the mere presence of an artificial effect and economic utility, without more, sufficient to determine manner of manufacture?
It is not a question of stating precise guidelines but of deciding, in each case, whether the claimed invention, as a matter of substance not form, is properly the subject of a patent” (emphasis added).
[21] [2015] FCAFC 177; 115 IPR 461 (RPL Central)
In Research Affiliates LLC v Commissioner of Patents[22] the Full Court of the Federal Court stated at [94]:
[22] [2014] FCAFC 150; (2014) 109 IPR 364 (Research Affiliates)
“When the authorities in Australia prior to and including Grant are considered, a consistent approach emerges as to the relevance of:
·a distinction between a claim to a business scheme and claims to methods which in practice result in a new machine or process or an old machine giving a new and improved result – that is, a distinction between mere intellectual information and a method that affects the operation of an apparatus in a physical form (Grant at [18]);
·the fact that the claimed steps are foreign to the normal use of computers, such as the production of an improved curve image (IBM 2 at FCR 225-226; ALR 395; IPR 424);
·the particular mode or manner of achieving an end result which is an artificially created state of affairs, such as the storage of data as to Chinese characters and retrieval of graphic representations to enable word processing (CCOM at FCR 295; ALR 450; IPR 514);
·whether part of the invention is an inventive method which includes the application and operation in a physical device (Grant at [30]);
·the distinction drawn in Catuity, as explained in Grant (at [24]), between ‘a technological innovation which is patentable and a business innovation which is not’. In Catuity, Heerey J. did not accept that a physically observable effect was necessarily required (at [128]) but the Full Court in Grant expressed the opinion that a physical effect in the sense of a concrete effect or phenomenon, or manifestation or transformation is required (at [32]).
·the fact that a physical effect is required does not make it sufficient to confer patentability;
·the fact that a method may be called a business method does not prevent it being properly the subject of letters patent (Grant at [26] citing Catuity at [125]-[126]);
·the fact that for claimed computer programs, the courts look to the application of the program to produce a practical and useful result, so that more than ‘intellectual information’ is involved (Grant at [29]). A method that is in the nature of directions for use does not constitute an invention or a manner of manufacture in the absence of some previously unrecognised property of an aspect of the method (Grant at [29]).”
In considering the substance of the invention the Full Court went on to say at [118]:
“The claimed method in this case clearly involves what may well be an inventive idea, but it is an abstract idea. The specification makes it apparent that any inventive step arises in the creation of the index as information and as a scheme. There is no suggestion in the specification or the claims that any part of the inventive step lies in the computer implementation. Rather, it is apparent that the scheme is merely implemented in a computer and a standard computer at that. It is not part of the claimed method that there is an improvement in what might broadly be called ‘computer technology’.” (emphasis added)
The Full Court in Encompass Pty Ltd v InfoTrack Pty Ltd[23] did not find it necessary to revisit the correctness of RPL Central or Research Affiliates, stating at [91] that:
“In each case, the Full Court was seeking to describe the conceptual distinction between a manner of manufacture and an unpatentable abstraction. In each case, the Full Court was explaining that a claimed method that is unpatentable does not change its legal character merely because the method is implemented by the instrumentality of a computer.”
[23] [2019] FCAFC 161; 372 ALR 646 (Encompass)
The Full Court in Commissioner of Patents v Aristocrat Technologies Australia Pty Ltd[24] observed in relation to a claim defining an electronic gaming machine (EGM) with a particular feature game at [56]-[57]:
“What this purpose-specific but extremely common computer does is play the feature game. Consequently, the substance of the invention disclosed by Claim 1 is that feature game implemented on the computer which is an EGM. It is therefore a computer-implemented invention.
As we have already observed, integers 1.10-1.12 embody an abstract idea which may be characterised both as a set of rules defining a family of games and as a business scheme for increasing player interest in an EGM. As such its implementation in the computer which is an EGM cannot constitute patentable subject matter unless it represents an advance in computer technology.”
[24] [2021] FCAFC 202; 163 IPR 231 (Aristocrat ’21)
On appeal to the High Court in Aristocrat Technologies Australia Pty Ltd v Commissioner of Patents[25] the High Court was evenly split, and via section 23(2)(a) of the Judiciary Act 1903 affirmed the Aristocrat ’21 decision. However, both sets of reasons appear to have indicated that an inquiry as to whether there is an advance in computer technology is not a useful test for patentability[26]. The High Court affirmed the importance of properly characterising the claimed invention[27]. Additionally, the reasons of the High Court confirm that the Full Court decisions in Research Affiliates, RPL Central, Encompass and Rokt were correctly decided[28]. Therefore, I understand that the law as applied in these Full Court decisions still stands and remains relevant.
[25] [2022] HCA 29, (Aristocrat ’22)
[26] Ibid., see the reasons by Kiefel CJ, Gageler J. and Keane J. at [77] and the reasons by Gordon J, Edelman J. and Steward J. at [122].
[27] Ibid. at [73], [101]-[105].
[28] Ibid. at [121]-[122].
The Applicant directs my attention to certain passages of the Patent Manual of Practice and Procedure (the PMPP), specifically PMPP 2.9.2.1 and 2.9.2.7. Of particular relevance to the current matter is the reference to Bio-Rad Laboratories, Inc[29], an office decision relating to an application concerning a system for optimising a quality control strategy. Bio-Rad concludes at [57]:
“However when this identified QC rule is put into effect during testing, the resulting reduction in the number of tests to be carried out and the reduction in the number of reference samples that are used are, in my view, ‘artificial effects’. Furthermore the contribution of the claimed invention is clearly technical in nature as it has application in the technology of medical diagnostic devices and their testing. In my view this invention is not dissimilar to IBM in that it involves the application of certain mathematical formulae in the creation of a new process that finds application in a field of technology. It is also clearly not a mere business scheme or business innovation. In my view these are sufficient to satisfy the requirements of an ‘artificially created state of affairs’ under the principles enunciated in NRDC.”
[29] [2018] APO 24 (Bio-Rad)
In their submissions, the Applicant has also made repeated reference to Advanced New Technologies Co., Ltd[30] as a recent patent office decision providing “a structured test.”[31] While I take no issue with ANT, I am mindful that I should not elevate the reasoning of a patent office decision to the status of a legal test. It is also unnecessary to do so, as ANT principally follows and applies the teachings of RPL Central and Research Affiliates. ANT then adopts the non-exhaustive summary of considerations drawn from the case law set out by the delegate in Aristocrat Technologies Australia Pty Ltd[32] which stated at [35]:
[30] [2021] APO 29 (ANT)
[31] Applicant submissions page 5
[32] [2016] APO 49 (Aristocrat ’16)
“I conclude that it is relevant to consider a range of matters. Without seeking to be exhaustive, these include:
·there must be more than an abstract idea, mere scheme or mere intellectual information;
·is the contribution of the claimed invention technical in nature;
·does the invention solve a technical problem within the computer or outside the computer;
·does the invention result in improvement in the functioning of the computer, irrespective of the data being processed;
·does the application of the method produce a practical and useful result;
·can it be broadly described as an improvement in computer technology;
·does the method merely require generic computer implementation;
·is the computer merely an intermediary or tool for performing the method while adding nothing of substance to the idea;
·is there ingenuity in the way in which the computer is utilised;
·does the invention involve steps that are foreign to the normal use of computers; and
·does the invention lie in the generation, presentation or arrangement of intellectual information.”
While these are a summary of considerations extracted from case law in 2016, subsequent decisions of the courts have confirmed their relevance and the Applicant appears to largely agree with these as a suitable summary of relevant considerations[33]. As a further observation, it appears to me that the Court decisions on manner of manufacture for computer-implemented inventions do not always contain the full list of the above relevant considerations, which would appear to suggest that, depending on the circumstances of the case, not all of the relevant matters included in the list need to be considered in order to reach a conclusion. It could also be said that some of the above considerations are somewhat overlapping in their scope.
Considerations
[33] Applicant submissions at pages 5 and 7-13.
The Applicant has helpfully summarised their arguments in relation to four of the considerations listed in Aristocrat ’16 in a table at page 13 of their submissions which compares the invention in ANT with the present invention. While I do not assume it to be a concession on the other considerations, it appears the Applicant is advancing that the most relevant consideration in the present case is in respect to the question of solving a technical problem. Consequently, I will start with this consideration.
Does the invention solve a technical problem and is the contribution of the claimed invention technical?
The Applicant identifies at least three problems in the art[34]:
a.Significant time required for preparation of pathology slides,
b.Improving effectiveness and reducing the time to undertake quality control/assurance, and
c.The isolated nature of analysis that pathologist engage in.
[34] Applicant submissions at page 8
The first of these may relate to a technical problem, but there is nothing in the present application which improves the process of preparing the slides themselves – the present invention applies exclusively to the digital images of the already prepared slides. The Applicant described this problem in the context of [037] of the specification, which recites that the current digital pathology process involves a technician preparing a sample, the sample later being inspected by a pathologist who may only at that stage identify that further cuts, tests, or stains are necessary, and then instructing the technician to perform the further tasks. But inefficiencies in a given pattern of work are emblematic of an administrative problem. Other than administrative complexity/cost constraints there would appear to be no technical barrier preventing a pathologist from being present at the time of preparing samples to make any further instructions in real time (or even having a pathologist prepare the slides).
The second problem is described in the context of [044] of the specification, which describes the problems of existing systematic quality control processes as being resource and time intensive and/or involving duplicative effort. Again, this is clearly an administrative problem. There is no technical requirement for this work. It is, instead, a matter of laboratory practice of how pathologists conduct quality assurance.
The third problem, regarding the current nature of the way pathologist organise their work, is similarly administrative in nature. The specification and submissions do not identify any technical barrier or limitation which forces pathologists to work in such a way. It is also not entirely clear how the current invention relates to this problem.
Elsewhere, the Applicant appears to elaborate on the second problem by rephrasing it as a need to reduce the amount of non-useful data sent to the pathologist[35]. They then describe the benefits of this in terms of improved time and efficiency for the pathologist. I note the Applicant treats this as the solution to the second problem, however whether it is the problem which is technical or if it is the solution which is technical does not alter my analysis in this instance.
[35] Applicant submissions at page 11.
The Applicant appears to be conflating data processing (or possibly data itself) with “technical”. The presence, manipulation, or reduction of data is not in and of itself inherently technical. The Applicant argued that the removal of non-useful data improved the pathologist’s efficiency, irrespective of how the non-useful data is removed. The Applicant seemed to think this was an argument suggestive of a technical problem. However, at the presented level of generality it appears far more administrative in nature. At a similarly high level, a business scheme which more efficiently routes documents to only the appropriate action officer is still a business scheme, regardless of whether the document itself contains technical data.
Elsewhere, the Applicant identifies variations/alternative problems as:
“How the invention automatically obtains slides before intervention by the pathologist is required while maintaining sufficient quality control; and,
How the invention integrates online data stores to form a real time monitoring, forecasting of health patterns and ensuring sufficient QA/QC.”[36] (emphasis added)[36] Applicant submissions at page 8
The Applicant regards these as being at least part of the substance of the invention. However, the use of “How” makes these appear more like problems to be solved rather than the invention. This distinction does not alter my analysis.
As I noted above, the specification does not teach how to automate obtaining slides or slide preparation itself. There are no details of how to automatically cut a sample, how to automatically apply stains to a sample, or how to automate the setting of the sample on the slide. These processes are all simply done by the technician (or, if they are automated, it must be done by existing systems not related to the Applicant’s contributions). The Applicant’s contribution to the “automated slide preparation pipeline” referred to in the description at [039] appears to be in respect to the pathology data. The automated segmenting and staining machines referred to are not described by the specification, and therefore appear to be tacitly accepted as prior art. The Applicant’s contribution lies in automatically determining certain data from the already prepared pathology slide, for input into/correction of the LIS and for external aggregation purposes, as well as for sending to the pathologist. Notably, this curating and distribution of data (which would typically be available in the LIS) is essentially an administrative arrangement to allow access to certain data without giving access to the LIS. I also note there is no teaching about how to integrate online data sources; merely teaching that this would be desirable.
Regardless, these problems (solutions/alleged substance) do not appear to relate to the claimed invention, which relates to the quality control method/system. None of the current claims relate to automatically obtaining slides, nor to integrating online data sources.
While not discussed by the Applicant in depth, there are a couple of additional points which merit closer inspection. The first of these lies in the selecting of an appropriate recommended action based on the quality defects detected. Where this refers to a simple, one to one mapping where the recommendation follows naturally and directly from the defect detected (e.g., determine that the slide is damaged, therefore recommend slide reconstruction) the true nature of the invention would be the determination of the quality defect – after that, the user does not need to be told what to do, so the recommendation itself is superfluous. In contrast, where the situation is more complex, such as determining that a plurality of defects and defect types exist but that the sample would be adequately salvaged by taking a single remedial action, the nature of the invention would lie in the determining of an appropriate or optimal remedial action to recommend. The second point is a more specific iteration of the first; where the quality defect detected is a scanning artifact(s), the recommendation may be simple (e.g., “check scanning parameters and repeat scan”) or complex (e.g., “set magnification to X, scan speed to Y, and lighting to Z, and repeat scan”). In the simple case, the nature of the invention lies in determining the defect and directly outputting the matching recommendation, in the complex case the nature of the invention lies in determining the correct, or optimal, remedy from a potentially vast set of options. This problem may be even further exacerbated when considering a quality issue which is the product of a plurality of quality defects.
However, the presently claimed invention appears to be directed to (or at least encompass) the simple case of a recommendation arising directly from a detected defect. In fact, the claimed invention is broad enough to encompass the situation where the recommended action is always the same: “recommend performing one or more of specimen cut, alter scan parameter, reconstruct slide, and/or mark slide”. The claimed invention does not appear to be directed to the more complex (and arguably technical) problem of selecting an optimal action. Since none of the claims appear directed to this problem, I will refrain from further analysis of this point.
From my reading of the specification, as far as the claimed quality control method/system is concerned, the problem in the art is this: how to automate the detection of quality issues in a digital pathology sample? This is not an inherently technical question but may be in some cases. For example, it may be the case that the proper assessment of the specimen cut is a largely qualitative question traditionally requiring the expertise of a trained pathologist. This would represent a technical barrier preventing traditional or simple automation efforts. However, the application does not clearly define any such barrier crossed.
The Applicant’s solution is to apply machine learning models to determine if there are certain quality defects, determine an overall quality score, and if needed output recommendations. That is, the solution is automating or computerising the default quality assurance process. Quality assurance, at this level of generality, is an administrative scheme, which suggests that, if there is any patentable subject matter, it must lie in the computerisation.
Artificial state of affairs
The applicant submits that the application of the claimed invention results in an artificial state of affairs being a reduction in the non-useful data presented to the pathologist.[37] They make particular note of the discussion in Bio-Rad, where the artificial state of affairs was said to be a reduction in the number of tests/reference samples to be carried out in a quality control scheme.
[37] Applicant submissions at page 14.
As an observation, it is not clear to me that the invention (at least as claimed) reduces the amount of non-useful information sent to the pathologist. While I do not have the advantage of expert testimony to the facts, I would presume that under pre-existing sample preparation methods the technician checks the digital pathology image for such obvious defects as a cracked slide (one of the expressly listed quality defects), or obvious lighting/camera focus errors (which would fall under “scanning parameters”). Consequently, the artificial effect produced by the operation of the present invention is the recommendation itself.
I also consider the present case is distinguished from Bio-Rad. In that case, the character of the invention was the real-world impact of reducing the quantity of samples required, but in the present case the operation of the invention does not alter the need for remedial actions, merely whether the recommendation is made by the pathologist (or technician) or the computer. This may be regarded as an artificial state of affairs, in that it would not occur absent the present invention, however, this by itself is not determinative. The lessons of Research Affiliates, RPL Central, Encompass, and other cases I discussed above is that, more than mere artificiality, it is the character of the invention which is important.
Does the method merely require generic computer implementation?
The Applicant concedes that the present invention requires only generic computer implementation, although they deny the combination of integers is generic[38]. They also note that this consideration is not on its own determinative.
[38] Applicant submissions page 11-12
The Applicant is correct that this consideration on its own is not determinative, but, as noted above, where the invention appears to lie in the computerisation of an administrative method, and the computerisation may be done with no more than generic computer implementation, this consideration weighs against the Applicant.
Is there ingenuity in the way the computer is utilised?
The Applicant did not make specific comments about this consideration under their “Patentable Subject Matter” heading, as this consideration did not come into play for ANT. However, this point involves some similar considerations as inventive ingenuity, so I will make reference to some of the Applicant’s comments made under their “Novelty and Inventive Step” heading.
The Applicant identifies key contributions to inventiveness[39] as being (rephrased slightly to be consistent with the terminology I have used above):
a.Training images involving both real and synthetic images.
b.Outputting a recommendation(s) to improve the quality score.
c.That the recommendation is one of the specific recommendations claimed.
d.Performing the remedial action(s)
e.Receiving the new target image.
[39] Applicant submissions page 3
Points b, d, and e relate to steps of the administrative method, and point c is most relevant to discussions of inventive step rather than ingenuity in how the computer is utilised. Point a describes the nature of the data which may be used by the computer, rather than the way the computer is utilised. In addition, I understand that the use of synthetic images (especially images augmented with noise) is typical in machine learning models[40].
[40] See for example >
As noted above it is possible that the assessment of a sample cut may involve technical difficulties, as it may traditionally involve a qualitative assessment by a skilled pathologist. The Applicant’s contribution in this regard is training the machine learning model on a training data set which includes examples of the quality defects. However, this would appear to be the definition of training data. A machine learning model cannot distinguish a defect without being trained with examples of the defect.
Regarding the recommendation being one of the recommended actions, as I have construed these features they encompass where the model does not distinguish between actions and instead recommends “any combination of” the recommended actions. For reasons similar to my comments at paragraph 69, where the invention is directed to a simple one-to-one recommendation the nature of the invention is simply determining the defect.
I consider there to be no ingenuity in the way the computer is utilised. It is simply applying general machine learning principles to the specific field of digital pathology.
Does the application of the method produce a practical and useful result?
The Applicant submits that the operation of the invention produces at least three practical and useful results[41], namely a) saving the pathologist time by reducing the non-useful information presented, b) performing quality control/quality assurance process which assists in providing more accurate results, and c) integrating/augmenting the analysis process with data from outside any specific pathologist’s lab.
[41] Applicant submissions page 12
While point c) does not seem to relate to the currently claimed invention, the Applicant notes that Encompass teaches that the practical and useful result does not need to be immediately realised, so long as the result is at least ultimately derived from the invention.
Regarding a) and b), I agree with the Applicant that the application of the invention will offer time saving advantages by detecting quality defects without waiting for the pathologist to consider the sample. This is, clearly, practical and useful. I am less convinced it necessarily leads to more accurate results, as it appears to me to merely automate the existing quality assurance processes. A more accurate result would require that the invention be better able to distinguish between acceptable quality and unacceptable quality images than a pathologist. But the accuracy of the machine learning model would appear to be limited to the accuracy of the training dataset, which in the present case comprises labelled images provided by human pathologists, so it is not clear that the invention would be meaningfully more accurate.
The time saving offered by the invention represents a practical and useful result. However, this is not on its own determinative. Many unpatentable business and administrative schemes can also deliver time savings or efficiencies. The nature of the useful result achieved is indicative (but not determinative) that the substance of the invention lies in the automation of the quality assurance processes, i.e., the automation of the administrative scheme.
Does the invention lie in the generation, presentation or arrangement of intellectual information?
The Applicant’s comment on this point is brief, and not entirely clear. They identify the invention as involving “automatic QC/QA, machine learning of pathology information and automatic presentation of pathology information. The individual steps clearly do not generate/present merely intellectual information”[42] (emphasis added). As best as I can ascertain, the Applicant’s intention here was that the information presented is not merely intellectual information, but rather technical data, i.e., pathology data useful for diagnostics.
[42] Applicant submissions page 12
It is worth noting that the quality score does not relate to any physical or observable characteristic of the target pathology sample. It is not a specific measure or observation of the sample, but rather a generalised indicator of the quality of the sample generated by the machine learning model. The examiner characterised this as a measure of risk that the defect would inhibit diagnosis, which I agree would fall within the claimed quality score, but the claim as currently defined does not require an assessment of risk. Consider where the diagnosis is able to be accurately rendered based on only a small portion of the image. The fact the rest of the image is out of focus does not materially impact the risk factor but may be regarded as impacting on quality of the image. Consequently, the quality score is not used to make technical decisions about the target sample, but rather used as an intellectual tool for quality assurance.
I also note that the method/system involves a model determining a characteristic, which is a generalised representation of a quality issue, which itself is a generalised class of quality defects, which themselves encompass a wide class of qualitative issues. The characteristic itself is not technical information but intellectual information derived from the image by the machine learning model, several steps removed from the physical data of the target sample itself.
From my reading of the specification, the machine learning system described is used to analyse a target image, determine a characteristic indicative of quality defects and a quality score, and output one or more recommendations as needed. The determining of quality issues amounts to simply generating and presenting intellectual information about the target image, which is then presented to a user.
The substance of the invention.
In light of the above consideration, I consider the substance of the invention lies in implementing quality assurance of pathology samples with a machine learning model. It involves essentially generating abstract, intellectual information about a target image (that there are quality issues and assigning a quality score) and utilising that information to replicate the existing administrative quality assurance scheme (that the issues be addressed with remedial actions). At the level at which it is described and claimed, the substance is merely using the computer to perform an administrative quality assurance process.
While the majority of my analysis above is directed to the invention as defined by claims 1 and 7, to my mind there is nothing in any of the dependent claims which would materially alter the substance of the invention.
Claims 2-4 and 8-10 add features of further categorising the sample by specimen type (generating and presenting a label), and each of claims 5 and 11 outputs the quality score (presenting information). All of these features are, at most, automating the collection of data which would previously have been captured with the LIS. No technical problem is described in the specification which needed to be overcome, and all solutions are implemented by simply applying standard machine learning principles.
Claims 6 and 12 add determining whether the specimen is pre- or post-treatment. While I can imagine that this could involve significant technical challenges to implement, the specification does not identify any or provide any details of how such issues may be overcome, so at the level of generality claimed, this appears to fall under the same category of merely generating and presenting intellectual information about the sample.
The claimed invention does not comprise patentable subject matter.
Inventive Step
Section 18(1)(b) of the Patents Act provides:
“Subject to subsection (2), an invention is a patentable invention for the purposes of a standard patent if the invention, so far as claimed in any claim:
…(b)when compared with the prior art base as it existed before the priority date of that claim:
(i) is novel; and
(ii) involves an inventive step;”The requirements of inventive step are set out at Subsection 7(2):
“For the purposes of this Act, an invention is to be taken to involve an inventive step when compared with the prior art base unless the invention would have been obvious to a person skilled in the relevant art in the light of the common general knowledge as it existed (whether in or out of the patent area) before the priority date of the relevant claim, whether that knowledge is considered separately or together with the information mentioned in subsection (3).”
It is generally established that a useful approach to inventive step is the reformulated “Cripps question”, which is discussed in Aktiebolaget Hassle v Alphapharm Pty Ltd[43] at [53]:
“That way of approaching the matter has an affinity with the reformulation of the ‘Cripps question’ by Graham J. in Olin Mathieson Chemical Corporation v Biorex Laboratories Ltd[44]. This Court had been referred to Olin in the argument in Wellcome Foundation[45]. Graham J. had posed the question [59]:
‘Would the notional research group at the relevant date, in all the circumstances, which include a knowledge of all the relevant prior art and of the facts of the nature and success of chlorpromazine, directly be led as a matter of course to try the –CF3 substitution in the '2' position in place of the –C1 atom in chlorpromazine or in any other body which, apart from the –CF3 substitution, has the other characteristics of the formula of claim 1, in the expectation that it might well produce a useful alternative to or better drug than chlorpromazine or a body useful for any other purpose?’ (Emphasis added)
That approach should be accepted.”
[43] [2002] HCA 59; 212 CLR 411 (Alphapharm)
[44] [1970] RPC 157 (Olin)
[45] [1981] HCA 12; 148 CLR 262 (Welcome Foundation)
The Applicant further directs my attention to PMPP 2.5.3.5 (Obvious Combinations of Features of Common General Knowledge) and 2.5.3.6 (Invention in Identifying the ‘Real Nature’ of the Problem). These pages provide practical guidance to examiners primarily distilled from Minnesota Mining and Manufacturing Co v Beiersdorf (Australia) Limited[46], Welcome Foundation[47], and Winner & Another v Ammar Holdings Pty Ltd[48].
[46] [1980] HCA 9; 144 CLR 253 (3M)
[47] Ibid., at page 281
[48] 24 IPR 137 (Winner)
In 3M Aickin J. stated:
“The proper question is not whether it would have been obvious to the hypothetical addressee who was presented with an ex post facto selection of prior specifications that elements from them could be combined to produce a new product or process. It is rather whether it would have been obvious to a non-inventive skilled worker in the field to select from a possibly very large range of publications the particular combination subsequently chosen by the opponent in the glare of hindsight and also whether it would have been obvious to that worker to select the particular combination of integers from those selected publications. In the case of a combination patent the invention will lie in the selection of integers, a process which will necessarily involve rejection of other possible integers. The prior existence of publications revealing those integers, as separate items, and other possible integers does not of itself make an alleged invention obvious. It is the selection of the integers out of, perhaps many possibilities, which must be shown to be obvious.”[49] (emphasis added)
[49] 3M at 116
I note that the above acknowledges that invention may lie in the selection of known integers, it does not stand for the argument that any new combination of old integers is inherently inventive. As an example, in May v Higgins[50] it was stated by Griffith C.J. that:
“A combination is not an invention unless the combination is substantially a new thing.”[51]
[50] [1916] HCA 8; 21 CLR 119 (May)
[51] Ibid. at page 121
And in the same decision Isaacs J. provided that:
“Then comes the question whether the presence of this feature makes the whole thing a combination. It appears to me that it is a mere improvement of one previously existing integer. It is not a new integer giving a better result, nor the substitution of a totally different integer, the presence of which is such as to make the whole machine an essentially different machine, a new unit. It is, I think, at best an improvement upon a prior integer not altering the essential character of the machine.”[52] (emphasis added)
[52] Ibid. at page 122-123.
This was reiterated more recently in Winner v Ammar Holdings Pty Ltd[53] by Cooper J. at 25:
“Notwithstanding that the inventive step may lie in the choice and management of integers in a combination patent, where one starts with a known article or thing and merely substitutes or adds a known device or means to facilitate the better use of the thing, there is a risk of want of inventive step "unless the combination is substantially a new thing" (May v. Higgins (1916) 21 CLR 119 at 121).” (additional citations omitted)
[53] [1993] FCA 134; 41 FCR 205 (Winner ’93)
In respect of invention in recognising the real nature of the invention, Aickin J. in Welcome Foundation stated:
“Evidence of what he did by way of experiment may be another matter. It might show that the experiments devised for the purpose were part of an inventive step. Alternatively it might show that the experiments were of a routine character which the uninventive worker in the field would try as a matter of course. The latter could be relevant though not decisive in every case. It may be that the perception of the true nature of the problem was the inventive step which, once taken, revealed that straightforward experiments will provide the solution. It will always be necessary to distinguish between experiments leading to an invention and subsequent experiments for checking and testing the product or process the subject of the invention. The latter would not be material to obviousness but might be material to the question of utility.”[54] (emphasis added)
[54] Welcome Foundation at 281
This was contrasted in Winner:
“There may of course be cases where: ‘…the perception of the true nature of the problem was the inventive step which, once taken revealed that straightforward experiments will provide the solution.’ See [citing Welcome Foundation as quoted above] per Aickin J, with whom all the other members of the court agreed.
However in the present case there is no suggestion in the specification itself that discovery of the problem involved an inventive step.”[55] (emphasis added)
[55] Winner at page 140-141
Winner was appealed, and relevant to the above portion Cooper J. noted that:
“I do not consider that it is part of the ratio of the decision in Acme Bedstead Co, as was submitted by counsel for the appellant, that the defect must be one which is pointed out to the hypothetical competent workman before the test of obviousness is to be applied. In my view it is sufficient if the workman or the user of the article would as a matter of observation and use himself or herself recognise the defect which it is alleged the invention overcomes to call in question the issue of obviousness.”[56] (emphasis added)
[56] Winner v Ammar Holdings Pty Ltd [1993] FCA 134; 41 FCR 205 (Winner ’93) at page 295
Regarding the reference to Acme Bedstead Co, the ratio Cooper J. refers to appears to be:
“He has combined together well known mechanical integers and has used each of them for its natural and well known purpose … The plaintiff has simply applied well known things to an article to they had not formerly been applied… If a competent workman, seeing either a given specification or an article in actual use, could, upon a defect being pointed out, devise, without the exercise of any inventive ingenuity, a means of overcoming the defect, there would not be invention in the result which he so achieved.”[57]
[57] (1937) 58 CLR 689 at 700-701
When taken together, these set out that the invention may lie in perceiving the true nature of the problem, after which the steps of reducing to practice (such as selecting certain integers) may be routine. However, if this is the case it should be made clear by the specification, and further it must be a “true nature” which would not have been apparent to the person skilled in the art (i.e., if the true nature of the problem is obvious, perceiving the true nature of the problem cannot be inventive).
The disclosure of WO 2018/156133 A1
The remaining examiner objection of inventive step is that the claimed invention lacks an inventive step over the teaching of WO 2018/156133 A1 (D1).
D1 is directed to a system for assisting a pathologist to identify the presence of tumour cells in lymph node tissue of a digital pathology image[58]. The general operation of the assistant tool is to create a “heatmap” highlighting areas the machine learning model determines to have high risk of tumour[59]. However, more relevant to the present invention, D1 discloses a precursor “image quality assessment” component[60].
[58] D1 abstract. Page 6 line 1-9 further contemplate alternative tissue types, but only lymph node tissue are described in depth.
[59] D1 page 6 lines 10-23
[60] D1 page 8 line 23 to page 9 line 19.
The image quality assessment component of D1 comprises a pattern recognition neural network which is used to categorise a target image as “Gradeable” or “Not Gradeable”, where Gradeable indicates that the patch is of adequate quality for further analysis. The neural network is developed by processing training images comprising a set of H&E stained biopsy slides which have been labelled by pathologists as gradeable or not gradeable. Examples given for why a slide may be not gradeable are the presence of ink or dirt in the slide, or the image being out of focus[61].
[61] D1 page 9 line 15-17
In use, the quality assessment component analyses a “patch” (being a portion of an overall slide image) and returns a rating of Gradeable or Not Gradeable, and, if an assessment of Not Gradeable is returned, colour codes the patch as grey for the heatmap, and under some cases may instruct a user to rescan the slide or send for further evaluation. D1 also contemplates adding the quality assessment neural network to the “ensemble of trained pattern recognizers of Figure 5.”[62] D1 also mentions that the quality assessment may involve determining a score which is indicates whether the image is sufficiently high quality for further processing[63].
[62] D1 page 9 line 14-15
[63] D1 page 22 line 5-11
D1 refers to the presence of ink or dirt (i.e., markings) on the slide and out of focus images (i.e., scanning artifact) as issues which impact the gradeability of the sample. That is, gradeability is a characteristic, where low (or no) gradeability is indicative of quality issues including scanning artifacts or slide markings.
The Applicant submits that the examiner has conceded the following features are novel (i.e., not disclosed by D1)[64]:
a.Training images involving both real and synthetic images;
b.Outputting a recommendation for increasing the quality score for the target image;
c.The recommendation comprises any one/combination of specimen cut, a scanning parameter, a slide reconstruction and a slide marking;
d.Using the recommendation to receive a new target image; and,
e.Receiving the new target image by performing one/combination of a specimen cut, a scan using the scanning parameter, a slide reconstruction or a slide marking.
[64] Applicant submissions page 3
To dispense with the obvious first: D1 does disclose b) outputting a recommendation for increasing the quality score (“rescan the entire slide”[65]). Further, where D1 discloses such a recommendation, it is implied (or at least trivially obvious) that a user will follow it to produce a new target image (i.e., item d from the above).
[65] D1 page 9 line 2
I have a few observations regarding features c and e (corresponding to integers 1.8 and 1.9 respectively). Integers 1.8 and 1.9 are inextricably linked – where a specific recommendation is made, it is implicit (or at least trivially obvious) that an operator should follow it. In D1, the quality assessment tool assesses the patches to determine a characteristic, gradeability, which is indicative of quality issues, including focus and slide markings. The tool may output that the image is not gradeable and recommend rescanning the image. Where the person skilled in the art is informed that the image is not gradeable, they will comprehend that the reason it is not gradeable will be because of one or more of the quality issues that the tool has been trained to detect: in D1 this includes focus and slide markings. Since such issues are unlikely to spontaneously fix themselves, a technician reading D1 would not read the recommendation to rescan purely literally as to simply rescan without alteration. Instead, a skilled technician would understand the recommendation to be to fix the problem and then rescan. With this in mind, I consider D1 teaches that the quality assessment tool determines a characteristic, gradeability, which is indicative of one or more of focus errors or slide markings, and may recommend rectifying the issue, which the technician would understand to be to check the focus and/or the slide for markings. Once fixed, the technician would rescan the slide. Consequently, I consider D1 to teach to integers 1.8 and 1.9.
Regarding the training data, as I noted above D1 describes training data comprising a set of slides labelled by pathologists (i.e., real images). D1 further discloses producing synthetic or augmented training data for use in training the heatmap tool, including adding rotations and jitter[66]. This is described as “boosting the effect size of the training data” and “preventing memorization (sic) of exact patches and increases the diversity of the patches.”
[66] D1 page 17 line 25 to page 18 line 12
D1 does not describe using the synthetic images to train specifically the image quality assessment component (i.e., item a from the above, corresponding with integer 1.4).
Overall, I consider D1 teaches to all of the features of claim 1 except integer 1.4 (training data for the quality assessment tool comprising real and synthetic images). Similarly, D1 teaches to all of the features of claim 7 except integer 7.6.
Applicant submissions as to inventive step
To paraphrase the examiners objections, the examiner contended that the use of synthetic data in training sets was well known in the art[67], and that the recommended/remedial actions are “purely routine, derived routinely from the identified quality control issue”[68].
[67] Examination Report 2 objection 7
[68] Examination Report 3 objection 11
The Applicant does not appear to contest that these features are common general knowledge on their own, rather they submit that the invention lies in the combination of these features. The Applicant does not describe any barriers or issues which would have prevented the combination being made, but instead appears to rely solely on an alleged lack of reasoning as to why “such a wide variety of features of multiple parts of a broader pathology image analysis would be obvious.”[69]
[69] Applicant submissions on page 3.
Regarding the feature of synthetic images, as noted above the Applicant tacitly accepts that this feature, in isolation at least, is common general knowledge. As noted above, D1 itself discloses synthetic data used in the training of the heatmap tool. It appears to be a standard tool in machine learning applications, particularly where “real” datasets are limited[70]. Consequently, I consider a machine learning specialist would be motivated to incorporate synthetic data into the training dataset when implementing the teachings of D1.
[70] D1 page 17 line 25-27 refers to the need to “boost the effect size of the training data.”
I also note that the specification does not describe any advantage associated with this feature. In fact, in the preferred embodiment exemplified by Figure 5, the dataset comprises real or synthetic images, indicating that in even the preferred embodiment it is not critical as to whether the data is real, or synthetic, or combination thereof.
In light of the above, it would appear to me that the real/synthetic images are technically equivalent for the purposes of training the machine learning system. The substitution does not alter the function of the device, nor does any specific advantage arise that would alter the character of the invention to make the alleged combination a substantially new device.
Regarding the inventive step lying in the recognition of the true nature of the invention, the Applicant has submitted that:
“Thus, the invention, at least in part, lies in the recognition that it is not sufficient to train a machine learning model with any images, but rather the images must have specific quality control issues to thereby allow specific control issues to be identified. This in turn allows specific recommendations to be made to allow new images to be generated addressing the quality issues, which is not taught or suggested by the prior art.”[71]
[71] Applicant submissions page 3
Plainly, this is not persuasive. While it may hypothetically be possible to do otherwise, it is common general knowledge that to train a machine learning model to detect or classify anything, the training dataset should include examples of the thing to be detected. There can be no inventive ingenuity in recognising this “problem”.
Further, I have considered the idea that the system provides a tailored recommendation based on the specific quality defects that have been detected. As I noted in the construction section above, this is a subtle but important distinction from the invention as currently claimed, which merely makes one or more recommendations to improve the quality score. On my construction, the claimed invention need only be capable of determining a single defect type and consequently only ever provide the one recommendation when quality is insufficient (or in the case of D1, two defect types which are addressed by a single recommendation).
Consequently, as the alleged differences between D1 and the claimed invention are either disclosed by D1, obvious steps, or technical equivalents, I consider that the invention defined by claims 1 and 7 do not involve an inventive step over the teachings of D1.
The dependent claims
The dependent claims were not discussed in depth either in the Examination Reports or the Applicant’s submissions, and consequently I also will keep my comments brief.
Regarding claims 2 and 8, the claims define where the method/system further make a prediction of the specimen type “based on the at least one characteristic of the target specimen.” That is, the claims define that one single characteristic is indicative of both a quality defect (e.g., out of focus) and the specimen type (e.g., skin cells). I accept this is not obvious, as it would be quite surprising if there exists a single characteristic that is reliably indicative of both of these classes of information.
Regarding claims 3 and 9, these define essentially the same features as claims 2 and 3, and further define a confidence interval for the prediction, where, if the confidence is low, the system outputs “not identifiable”. I note in passing that the claimed use of confidence intervals/not predictable rating are, on their own, generic to machine learning systems; however, these claims are not obvious for at least the same reason as claims 2 and 8.
Regarding claims 4 and 10, these claims define essentially the same features as claims 3 and 9 but further output the confidence value. As an observation, “output” is not read as restrictive as “displaying to a user”, as “output” may refer to where the method/system provides the output to some other processing system or merely to stored memory. I also note that outputting (or displaying) a confidence value is generic to machine learning systems. Nevertheless, these claims are inventive for at least the same reasons as claims 3 and 9.
Regarding claims 5 and 11, these claims add the feature of outputting the quality score. I note that D1 discloses where the image quality assessor determines a patch is not gradeable, it is displayed as greyed out[72]. Displaying the same information as a numerical output does not comprise an inventive step.
[72] D1 page 9 line 3
Regarding claims 6 and 12, these claims add the features of determining whether the target image is pre- or post-treatment, and further determining a predicted degree to which the target specimen has been treated. D1 appears to be silent as to this feature, so I am not satisfied it has been demonstrated that this feature lacks an inventive step.
In summary, I consider claims 1, 5, 7, and 11 do not involve an inventive step over D1. I am not satisfied that claims 2-4, 6, 8-10, and 12 have been shown to lack an inventive step.
Additional Observations
In the process of writing this decision, I have become aware of certain issues which have not been addressed in the examination reports but may have significant impact on the progress of the application. Specifically, there are certain features which have been described in very general terms, potentially giving rise to subsection 40(2)(a) objections.
Of particular concern are the “scanning artifact” and “specimen cut” quality defects and associated recommended/remedial actions, the “tissue type” determination, and the “treatment degree” determination.
The law regarding sufficiency
Subsection 40(2)(a) of the act states that:
“(2) A complete specification must:
(a) Disclose the invention in a manner which is clear enough and complete enough for the invention to be performed by a person skilled in the relevant art.”
This requirement was introduced by the Raising the Bar Act, and is often referred to as sufficiency or enablement. For current purposes, I consider that the requirements are best summarised in Novartis AG v Johnson & Johnson Medical Limited[73] at 74:
“The heart of the test is: ‘Can the skilled person readily perform the invention over the whole area claimed without undue burden and without needing inventive skill?’”
Scanning artifact/parameter
[73] [2010] EWCA Civ 1039 (Novartis)
As currently drafted, this feature reads as “applying a machine learning system to determine at least one characteristic… indicative of … scanning artifacts” and “outputting a recommendation for increasing the quality score, the recommendation comprising … a scanning parameter.” As I have noted above under construction, this would appear to include (although not be limited to) determining the presence of a scanning artifact, being any defect that may be attributed to the image capture process, and then recommending a specific parameter value (e.g., scan rate, magnification, focal length) which will address the determined scanning artifact.
The specification provides limited elaboration beyond reiterating the words of the claims. The most specific teaching given comes from [089] which I quoted above, where it informs the reader that the machine learning model may be trained using a dataset that may include local or global label(s) to indicate presence of a quality control issue, which “may include but are not limited to: poorly cut specimen, scanning artifacts …”. I take this as a general instruction that the training data comprises images provided with labels identifying not merely that there is a scanning artifact, but what the scanning artifact is (e.g., out of focus).
In order to arrive at a specific parameter value recommendation, the system would need to determine not merely that there is a scanning artifact which may be attributed to a particular scanning parameter (out of perhaps many scanning parameters), but also to determine the correct parameter value (which, where the parameter is a continuum like focal distance, could be a potentially infinite number of possible parameter values). An example of this would be where the model determines that the image is out of focus, and to then identify what the correct focal depth setting should be. While I can accept determining an image is out of focus may be a fairly simple matter for those skilled in the art of machine learning or image analysis, taking the out of focus image and determining the correct setting is another matter. I do not suggest this is impossible, and I could speculate as to possible paths a person skilled in the art may contemplate, but it would appear any such attempt would involve considerable experimentation without any guarantee of success. Debatably, a machine learning specialist may consider that trial and error/preparing various training data sets is both all that is necessary and routine for this field of work. However, when the claim expands the scope to encompass any scanning artifact and any scanning parameter, the scope and breadth of further experimentation increases exponentially.
Specimen cut
The claimed invention defines “applying a machine learning system to determine at least one characteristic … indicative of… poorly cut specimen sections” and “outputting a recommendation for increasing the quality score, the recommendation comprising … a specimen cut”. As I have noted above, poorly cut specimen is construed broadly, encompassing thickness/evenness of the section, orientation, or any other issue which may be attributable to the cutting stage of preparation.
Very similarly to my above comments, the specification does not elaborate on what a poorly cut specimen comprises, or how it may be determined. The most specific instructions are again from [089], referring generally to providing labelled training data with “QC labels may include but are not limited to: poorly cut specimen sections…”
While I do not have the advantage of expert testimony, “poorly cut specimen” would appear to involve a very qualitative assessment of the sample by a trained pathologist. From the specification, the pathologist may understand that, in preparing the training data they should provide labels which identify if the sample is poorly cut and what is poor about the cut, even though the specification only mentions the first of these. I note that different tissue types may have different requirements for quality of cut, and, as such, a machine learning specialist may understand that the training data must be prepared for each different tissue type.
At this point we arrive at a paradox: If the quality of cut depends on a machine learning model trained on a specific tissue type, it appears the quality assessment must follow after the tissue-type determination. However, since the tissue type determination would appear to require a sufficiently high-quality cut, the tissue type would appear to follow the quality assessment. This may not be insurmountable (e.g., if the tests can be conducted in parallel or iteratively), but the specification offers no guidance on how to resolve the issue.
Regardless, the claimed invention includes where the quality assessment is not dependent on any assessment of tissue type. Phrased differently, the claimed invention goes to a generalised cut quality assessment. At face value, this requires some elaboration as to how to train the system beyond merely “provided labelled training data”. I note in passing that if it is the case that the solution is as simple as providing labelled training data to any suitable machine learning system, and that, from such a meagre instruction, the person skilled in the art could work the full scope of the invention, I struggle to see how there could be any inventive step in the idea.
Treatment degree
Claims 6 and 12 define that the method/system further determines whether the target specimen is post-treatment or pre-treatment based on whether the target image displays “treatment effects”, and further to output a predicted treatment degree. Treatment has not been defined specifically, so it would appear to encompass any treatment that may reasonably be investigated in a pathology slide. Treatment effects are reasonably any visible effects attributed to the treatment, including for example tumour reduction and cell damage. Degree of treatment may relate to either how much treatment has been undertaken, or the degree of success of the treatment taken.
The specification describes the exemplary embodiment of this process at [093]-[095]. At the risk of being repetitive, the detailed disclosure essentially amounts to an instruction to provide a data set of training images that includes images that have treatment effects, either of one tissue type or multiple, and using it to train a machine learning model to classify a target image. The specification states that the model training may be a supervised classification method, unsupervised density estimation, or anomaly detection method. However, the specification provides no further details beyond merely listing examples of suitable models (neural networks, random forest, logistic regression, and nearest neighbour[74]).
[74] Specification at [094]
Again, while I do not have the advantage of expert testimony, it would appear to me that the very technical, complex analysis of treatment effects that necessitates the role of skilled pathologists strongly indicates that the solution is not as simple as “feed the model data and hope for the best”. The vast array of combinations of treatment types, tissue types, and potential treatment effects would tend to indicate there is need for extensive further trial and error experimentation in order to implement the invention across the whole scope.
Treatment degree is also worth mentioning here; in some treatments, an assessment of the degree of treatment would, at face value, seem to rely on a comparison of pre- and post- treatment samples (e.g. size or quantity of tumour(s)). The present invention appears to make the assessment on the basis of a single image. The specification is silent to this issue.
Tissue type determination
Finally, the claimed invention in respect of claim 2-4 and 8-10 refers to where the tissue type is determined based on the same characteristic of the image that is also indicative of a quality issue. This appears to be a transcription error as a result of the amendment process, where, in the original claims, the characteristic was non-specific, with different dependent claims being directed to different characteristics. The specification makes no reference to how a single characteristic may be indicative of both attributes.
Since this appears to be in error, I will refrain from further analysis of this point.
SUMMARY
For the reasons stated above, I consider that the invention defined by claims 1-12 do not comprise a patentable manner of manufacture, and claims 1, 5, 7, and 11 do not involve an inventive step.
While it is not clear to me if there is any way to amend the specification to overcome the above, I nevertheless allow the Applicant the opportunity to try. Since I have identified potential Section 40(2)(a) issues which have not been previously explored and which will need to be considered by the examiner in light of the Applicant’s response, I consider that it is appropriate to invoke Regulation 13.4(3) to allow the Applicant six (6) months from the date of this decision to gain acceptance. If the Applicant does not respond, or acceptance is not achieved, the application will lapse at the end of this six months.
Andrew Burgess
Delegate of the CommissionerANNEX
Claims
1.A method of performing quality control with respect to pathology specimen preparation by analyzing an image corresponding to a specimen, the method comprising:
receiving a target image corresponding to a target specimen, the target specimen comprising a tissue sample of a patient;
applying a machine learning system to the target image to determine at least one characteristic of the target specimen and/or at least one characteristic of the target image, the at least one characteristic being indicative of a quality control issue, including at least one of poorly cut specimen sections, scanning artifacts, damaged slides, and markings on slides, and the machine learning system having been generated by processing a plurality of training images to predict at least one characteristic, the training images comprising real images of human tissue and at least some synthetic images, and wherein at least some of the training images include examples of quality control issues;
outputting the at least one characteristic of the target specimen and/or the at least one characteristic of the target image;
identifying a quality score for the target image, the quality score being determined according to the machine learning system;
determining whether the quality score for the target image is less than a predetermined value;
in response to the quality score for the target image being less than the predetermined value, outputting a recommendation for increasing the quality score for the target image, wherein the recommendation comprises any one or any combination of a specimen cut, a scanning parameter, a slide reconstruction, and a slide marking; and,
using the recommendation to receive a new target image, by performing any one or any combination of:
performing a specimen cut;
performing a scan using the scanning parameter;
performing a slide reconstruction; and
performing a slide marking.
2.The method of claim 1, further comprising:
determining a prediction of a specimen type of the target specimen based on the at least one characteristic of the target specimen; and
outputting the prediction of the specimen type of the target specimen.
3.The method of claim 1 or claim 2, further comprising:
determining a prediction of a specimen type of the target specimen based on the at least one characteristic of the target specimen; and
in response to determining that a confidence value of the prediction does not exceed a predetermined threshold, outputting an alert indicating that the specimen type of the target specimen is not identifiable.
4.The method of any one of the claims 1 to 3, further comprising:
determining a confidence value of a prediction of a specimen type of the target specimen based on the at least one characteristic of the target specimen; and
outputting the confidence value.
5.The method of any one of the claims 1 to 4, further comprising:
outputting the quality score.
6.The method of any one of the claims 1 to 5, further comprising:
determining, using the target image and the machine learning system, whether the target specimen is post-treatment or pre-treatment based on the target image having or not having treatment effects;
upon determining that the target specimen is post-treatment, determining a predicted degree to which the target specimen has been treated based on the target image; and
outputting the predicted degree to which the target specimen has been treated.
7.A system for performing quality control with respect to pathology specimen preparation by analyzing an image corresponding to a specimen, the system comprising:
at least one memory storing instructions; and
at least one processor executing the instructions to perform operations comprising:
receiving a target image corresponding to a target specimen, the target specimen comprising a tissue sample of a patient;
applying a machine learning system to the target image to determine at least one characteristic of the target specimen and/or at least one characteristic of the target image, the at least one characteristic being indicative of a quality control issue, including at least one of poorly cut specimen sections, scanning artifacts, damaged slides, and markings on slides, and the machine learning system having been generated by processing a plurality of training images to predict at least one characteristic, the training images comprising real images of human tissue and at least some synthetic images, and wherein at least some of the training images include examples of quality control issues;
outputting the at least one characteristic of the target specimen and/or the at least one characteristic of the target image;
identifying a quality score for the target image, the quality score being determined according to the machine learning system;
determining whether the quality score for the target image is less than a predetermined value;
in response to the quality score for the target image being less than the predetermined value, outputting a recommendation for increasing a quality of the target image, wherein the recommendation comprises any one or any combination of a specimen cut, a scanning parameter, a slide reconstruction, and a slide marking; and,
using the recommendation to receive a new target image, by performing any one or any combination of:
performing a specimen cut;
performing a scan using the scanning parameter;
performing a slide reconstruction; and
performing a slide marking.
8.The system of claim 7, the operations further comprising:
determining a prediction of a specimen type of the target specimen based on the at least one characteristic of the target specimen; and outputting the prediction of the specimen type of the target specimen.
9.The system of claim 7 or claim 8, the operations further comprising:
determining a prediction of a specimen type of the target specimen based on the at least one characteristic of the target specimen; and
in response to determining that a confidence value of the prediction does not exceed a predetermined threshold, outputting an alert indicating that the specimen type of the target specimen is not identifiable.
10.The system of any one of the claims 7 to 9, the operations further comprising:
determining a confidence value of a prediction of a specimen type of the target specimen based on the at least one characteristic of the target specimen; and
outputting the confidence value.
11.The system of any one of the claims 7 to 10, the operations further comprising:
outputting the quality score.
12.The system of any one of the claims 7 to 11, the operations further comprising:
determining, using the target image and the machine learning system, whether the target specimen is post-treatment or pre-treatment based on the target image having or not having treatment effects;
upon determining that the target specimen is post-treatment, determining a predicted degree to which the target specimen has been treated based on the target image; and
outputting the predicted degree to which the target specimen has been treated.
0
16
0