Apple Inc.
[2018] APO 72
•26 October 2018
IP AUSTRALIA
AUSTRALIAN PATENT OFFICE
Apple Inc. [2018] APO 72
Patent Application: 2016203040
Title:Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback
Patent Applicant: Apple Inc.
Hearing Officer: Greg Powell
Decision Date: 26 October 2018
Hearing Date: Written Submissions filed 2 February 2018
Catchwords: PATENTS – examiner’s objections – graphical user interfaces with haptic control – novelty and inventive step –– application accepted
Representation: FPA Patent Attorneys Pty Ltd
IP AUSTRALIA
AUSTRALIAN PATENT OFFICE
Apple Inc. [2018] APO 72
Patent Application: 2016203040
Title:Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback
Patent Applicant: Apple Inc.
Delegate: Greg Powell
Decision Date: 26 October 2018
DECISION
The claims as proposed to be amended are novel and involve an inventive step. I direct that the application be accepted.REASONS FOR DECISION
Background
Apple Inc. (“the applicant”) filed patent application 2016203040 (“the application”) on 8 March 2016 as an international application under the Patent Cooperation Treaty (“PCT”). The international application is designated PCT/US2016/021400. The earliest priority date is 8 March 2015.
The application was subjected to two examination reports. The first examination report issued 17 October 2016 raised new documents found in an original search, and also relied on documents raised during international examination. The applicant filed a statement of proposed amendments under Section 104 to address the objections raised in that report on 28 September 2017.
A second examination report was issued 16 October 2017, the day before the final date for acceptance, and maintained the objections of the first report. After the second report the applicant requested a hearing on 17 October 2017, the final date for acceptance.
The applicant filed written submissions for the hearing and filed amendments on 2 February 2018. I have considered these proposed amendments and am of the view they are clearly allowable as they comply with requirements of s102 (1). This decision is therefore based on the claims as proposed to be amended on 2 February 2018.
While the final date for acceptance of the application was 17 October 2017, patent sub-regulation 13.4(1) (g) is available to extend the time for gaining acceptance to at least 3 months from the date of the present decision.
Applicable law
The present application was filed after 15 April 2013. As such, it is governed by the Patents Act 1990 (“the Act”) as amended by the Intellectual Property Laws Amendment (Raising the Bar) Act 2012 (“the Raising the Bar Act”). Amendments to sections 7, 40 and 49 of the Act apply to the present case as a consequence of Schedule 1, items 55(1)(d) and 55(4)(a), and Schedule 6, item 133(7)(d) of the Raising the Bar Act.
Thus the standard of proof that applies in the present case is the balance of probabilities. I must accept the application if satisfied on the balance of probabilities that the application complies with the Act. If I am not so satisfied, then I can refuse the application.
The specification
In general terms the application relates to an electronic device with a touch sensitive surface and a display for a graphical user interface (“GUI”), whereby it is able to detect a change in intensity of the force of a contact on the surface and dynamically change the display of user objects in the GUI based on this intensity. More particularly, the disclosure relates to dynamically cycling forward through a series of user objects when a certain threshold of contact intensity is exceeded, and cycling backwards through said series when the intensity has dropped below another contact threshold.
The preferred embodiment of what is claimed with the proposed amendments is directed to the ability for a device, upon sensing a level of contact force, to play a sequence of images in the background. As the force level falls, the sequence is then played in reverse. This appears to be of use for a ‘lock screen’ on a mobile phone.
The claims
The amendments proposed on 2 February 2018 have reduced the number of claims from 28 to 18, and introduced additional limitations to the only independent (Claim 1) to further define the invention. Amended Claim 1 reads as follows:
- A method, comprising:
at an electronic device with a touch-sensitive surface, a display, and one or more sensors to detect intensities of contacts with the touch-sensitive surface:
displaying a first user interface on the display, wherein the first user interface includes:
one or more foreground objects, and
a background with a first appearance, wherein the background is common to the one or more foreground objects and includes a representative image in a sequence of images;
while displaying the first user interface on the display, detecting a first input by a first contact on the touch-sensitive surface while a first focus selector is at a location in the first user interface that corresponds to the background of the first user interface;
in response to detecting the first input by the first contact, in accordance with a determination that the first contact has a characteristic intensity above a first intensity threshold, dynamically changing the appearance of the background of the first user interface without changing the appearance of the one or more foreground objects in the first user interface, wherein the dynamic change in the appearance of the background of the first user interface is based at least in part on the characteristic intensity of the first contact and wherein dynamically changing the appearance of the background of the first user interface includes displaying in sequence at least some of the sequence of images based at least in part on the characteristic intensity of the first contact; and
while detecting the first input by the first contact, after determining that the first contact has a characteristic intensity above the first intensity threshold:
detecting a decrease in the characteristic intensity of the first contact; and,
in response to detecting the decrease in the characteristic intensity of the first contact, reversing the changing of the appearance of the background of the first user interface.
The amendments have introduced the feature of displaying a sequence of images based on the contact intensity on the touch sensitive surface, and reversing the sequence when the intensity is decreased to a lower threshold. The features of this particular embodiment are illustrated in Figures 8BF–8BI, and discussed in Paragraphs [00288]–[00301] of the description as originally filed.
The Grounds of Objection
The second examination report raised objections under both the grounds of novelty and inventive step. I will now consider each of the grounds of objection raised.
Novelty
It is well established that the general test for lack of novelty is the reverse infringement test. The classic formulation of this test is that given by Aickin J in Meyers Taylor Pty Ltd v Vicarr Industries Ltd, [1977] HCA 19 at [20]; [1977] HCA 19; 137 CLR 228 at 235:
“The basic test for anticipation or want of novelty is the same as that for infringement and generally one can properly ask oneself whether the alleged anticipation would, if the patent were valid, constitute an infringement”.
For lack of novelty to be established, the prior art must contain “clear and unmistakable directions” to produce the invention as claimed (Pfizer Overseas Pharmaceuticals v Eli Lilley and Company [2005] FCAFC 224; 68 IPR 1 at [314]). However, if the prior publication contains a direction which is capable of being carried out in a manner which would infringe the claimed invention, but would at least as likely be carried out in such a way that would not do so, the invention as claimed will not be anticipated (General Tire & Rubber Co v Firestone Tyre & Rubber Co Ltd [1972] RPC 457 at 485-486; 1A IPR 121 at 138, Novozymes A/S v Danisco A/S [2013] FCAFC 6; 99 IPR 417 at [177]).
The second examination report maintained an objection of lack of novelty in light of US 2006/0132457 A1 (D4), which was published on 22 June 2006. No other prior art was identified under this ground. D4 is directed to the use of a stylus with a pressure sensitive interface of an electronic device, such as a tablet or PDA.
In Objection 7 of the second examination report, the examiner stated:
“Claim 1 is not novel and lacks an inventive step in light of the prior art document D4.
I have considered the amendments, the arguments raised and the prior art. With the amendment, it is considered that what has changed is the appearance of the foreground object is changed is not part of the common background on figures 13 and 14, as it was discussed in objection 3 of the previous report. Therefore this novelty objection, as reasoned in the previous report, has been overcome. However:
I consider that given its broadest interpretation ‘foreground objects’ would include characters of text displayed. Therefore considering fig 11 instead of fig 13 and 14 (that were referred to in objection 3 of the previous report), the prior art document D4 still discloses the features of the claim.”
With respect to the as now proposed to be amended claims, the applicant submitted that D4 failed to disclose all of the features of claim 1 given that (inter alia):
- “Figure 11 of D4 fails to disclose a background including a representative image in a sequence of images. First, there appears to be no image in the background of Figure 11. Figure 11 merely shows text in the foreground, which is either highlighted or not highlighted. Alternatively, if the examiner is interpreting the highlighting as the background, this is not a representative image in a sequence of images, as that term is to be understood in the context of the specification as a whole.”
- “Figure 11 of D4 fails to disclose any process involving detecting input in relation to the background of the first user interface as claimed. Instead, D4 relates to selection of text, which examiner concedes is a foreground object.” (underlining in original)
- “Figure 11 of D4 relates to selecting a word, a sentence or a paragraph. There is no sequence of images. At most, highlighting is determined on the fly depending on the locations of the ends of the word, sentence or paragraph respectively. Selecting different portions of a foreground object in the form of text, dependent on pressure and the content of the text, does not disclose displaying images in a sequence of images based on intensity of a contact and, as indicated by the preceding point is inconsistent with such contact being in relation to the background.”
- “Figure 11 of D4 fails to disclose or teach that the appearances of the foreground objects are not changed in response to the changing of the background. Rather Figure 11 illustrates that after selection of the relevant words and paragraphs the foreground text will change in appearance from being black in colour to being white in colour in response to the background changing in appearance (e.g. from white to black). Therefore, Applicant submits that the objects in Figure 11 considered to be the foreground objects by the Examiner change in appearance.” (underlining in original)
- “Because there is no sequence of images disclosed in D4, there is necessarily no disclosure of reversing a sequence of images. Additionally or alternatively, if there is a sequence of images disclosed in D4, then there is no disclosure of an operation of reversing the sequence of images responsive to a decrease in the contact intensity.”
I agree with the applicant’s submission that D4 fails to disclose at least the feature of “displaying in sequence at least some of the sequence of images based at least in part on the characteristic intensity of the first contact”. D4 is concerned with the controls of the graphical user interface such as scrollbars, buttons, resize handles or dropdown menus. The stylus of D4 is used as an input via its contact in the user interface, and the pressure exerted by the stylus is a characteristic intensity of the contact. This contact intensity is then used to determine the magnitude of the effect exerted by the control. However, there is no disclosure of a sequence of images being controlled by the contact intensity, and I do not consider that rendering objects on the foreground of the screen is equivalent to a sequence of images. For these reasons it also follows that there is no disclosure of subsequently reversing the sequence of images, when the contact intensity is decreased after the first contact. Therefore I consider that Claim 1 (as proposed to be amended) is novel in light of D4.
As Claims 2–18 as proposed to be amended are appended to Claim 1, it follows that D4 similarly fails to disclose all the features defined by them, mutatis mutandis.
Inventive Step
The statutory basis for inventive step is set out at s7(2) and s7(3) of the Act, and is reproduced below:
“(2)For the purposes of this Act, an invention is to be taken to involve an inventive step when compared with the prior art base unless the invention would have been obvious to a person skilled in the relevant art in the light of the common general knowledge as it existed (whether in or out of the patent area) before the priority date of the relevant claim, whether that knowledge is considered separately or together with the information mentioned in subsection (3).
(3)The information for the purposes of subsection (2) is:
(a)any single piece of prior art information; or
(b)a combination of any 2 or more pieces of prior art information that the skilled person mentioned in subsection (2) could, before the priority date of the relevant claim, be reasonably expected to have combined.”
The test for whether an invention is obvious is whether it would have been a matter of routine to proceed to the claimed invention. In Wellcome Foundation Ltd v V.R. Laboratories (Aust.) Pty Ltd [1981] HCA 12 at [45], 148 CLR 262 at 286 Aickin J stated:
“The test is whether the hypothetical addressee faced with the same problem would have taken as a matter of routine whatever steps might have led from the prior art to the invention, whether they be the steps of the inventor or not.”
Citations
The second examination report maintained a number of objections raised in the first report on the grounds of inventive step in view of number of documents. Objection 6 of the second report maintained, firstly, that claim 1 lacked an inventive step in light of two video citations available on the internet:
D1“Water Ripple Effects on the Home and Lock Screen | AuqaBoard [sic] Cydia Tweak Review”, Published on 24 September 2012 at URL < and
D2“How to Enable Ripple Effect on Lock Screen of Galaxy S2”, Published on 12 February 2013 at URL <>
Objection 6 also maintained that claim 1 lacked inventive step when the teachings of two documents, US 2009/0140985 (D5) and US 2013/0227450 (D6), were combined.
D5 is considered to disclose variable pressure sensing. D5 teaches using the sensor results to modify the magnitude of an effect exerted by the control of a graphical user interface.
I also believe that D4 would be relevant for the consideration of inventive step.
I note that the disclosure of D5 is similar to that of D4, in so far as it applies to Claim 1, as it is concerned with the controls of the graphical user interface such as scrollbars, buttons, resize handles or dropdown menus.
Is D3 representative of the common general knowledge?
For completeness I note that a third document, US 2015/062052 (D3), was included in the examiner’s report to establish common general knowledge with respect to a known technique of sensing a force.
D3 is considered to disclose detecting variable contact intensity and using that as a control element to progress through a series of display states. In Objection 6 of the second examination report, the examiner stated with respect to D3:
“I consider what a person skilled in the art having a ripple screen display on a device without a force sensor would do when faced with a device that now senses the force applied to the display (as, for example a device discussed in D3 would have). [Note: D3 is marked as an A document, as it describes a technology, Force Touch, that was available in widely known and sold product since September 9, 2014 which therefore is considered common general knowledge.]. Based on how the real objects work, the person skilled in the art would be readily led to modify to ripple effect according to the force applied, directly leading to the claimed invention.”
D3 discloses progressing through a series of display states in a user interface based on the level of contact intensity, inclusive of progressing in reverse as the intensity decreases. Figures 5B1‑5B14 of the drawings disclose progressing through a sequence of images, each associated with threshold of intensity. Be that as it may, I am not satisfied this is representative of the common general knowledge in the art.
I note that it was said at page 482 in General Tire & Rubber Company v Firestone Tyre and Rubber Company Ltd [1972] RPC 457 that:
“... it is clear that individual patent specifications and their contents do not normally form part of the relevant common general knowledge ...”
I also note that it was stated in Minnesota Mining & Manufacturing Co v Beiersdorf (Australia) Ltd (1980) 144 CLR 253 at page 294:
“There may be some fields of endeavour in which those who work therein study and make themselves familiar with all patent specifications as they become available for inspection in one or in many countries so that what was contained therein becomes common general knowledge in that particular trade or field of manufacture in the country in question.”
Examples of this outcome are provided by Vidal Dyes Syndicate Ltd v Levinstein Ltd. (1912) 29 RPC 245 at pages 279-280 and British Celanese Ltd v Courtaulds Ltd. (1933) 50 RPC 259 at page 280. However, this situation has not been established here. The situation is not analogous to those of Minnesota, Vidal Dyes or British Celanese. The situation might have been different if a number of patent specifications had referred to a piece of knowledge (particularly if the applications had been by different applicants), but this was not the case.
It follows that I will not use D3 in the same manner as the examiner – as evidence of common general knowledge. However, that does not preclude me from considering D3 in an obvious combination with other citations.
The person skilled in the art and the problem
Neither the applicant nor the examiner expressed an opinion as to who the person skilled in the art would be. However, the examiner postulated in examination, and the applicant agreed in their written submissions, that the problem faced by the person skilled in the art was:
“what a person skilled in the art having a ripple screen display on a device without a force sensor would do when faced with a device that now senses the force applied to the display”
In light of this, I consider that the person skilled in the art would be a software engineer experienced in designing graphical user interfaces for a touch sensitive device.
Consideration
As a first point, I will consider whether it is obvious to modify the teaching of any one of D1, D2 or D6 to arrive at the claimed invention. I will then consider whether it is obvious to modify the teaching of either one of D4 or D5 to arrive at the claimed invention. Then I will consider whether combining any one of D1, D2 or D6 with any one of D4 or D5 would result in the person skilled in the art arriving at the claimed invention. Lastly, although not considered by the examiner, I will consider whether combining any one of D1, D2 or D6 with D3 would result in the person skilled in the art arriving at the claimed invention.
Obviousness in light of D1, D2 or D6
D1, D2 and D6 are all similar in that their disclosures are directed to a “ripple effect” on the display of an electronic device in response to a first input (a touch).
I do not consider the “ripple effect” as disclosed in D1, D2 and D6 to be a sequence of images. Insofar as the “ripple effect” is disclosed in D1, D2 and D6, this would appear to be a dynamic overlay on the graphical user interface applied via a renderer. While it is certainly a display effect produced in response to a first input, I do not consider that this is a sequence of images occurring in the background of the interface whilst the foreground object remains unchanged, as required by the claim. Furthermore, even if it is taken to be a sequence of images, there is no disclosure or teaching of subsequently reversing the “ripple effect” sequence when the contact intensity of the first input is decreased. However, I do consider that it is a technical equivalent to altering a background and that the principle could be applied to a sequence of images in the background of a user interface as a matter of routine by person skilled in the art.
As such, were a person skilled in the art to modify the “ripple effect” disclosed in either of D1, D2 or D6, to take advantage of the ability to distinguish the contact intensity of the user input via touch, I would expect the person skilled in the art would modify the effect to produce a different result when a different level of contact intensity was used. It would follow that the different result would be a different magnitude of the “ripple effect”, be it in size or of speed. That is, a harder touch would lead to larger waves, a larger expansion of waves, or faster waves. However, be that as it may, there is no teaching or suggestion to begin the ripple effect based on a first contact intensity, and to reverse it when the contact intensity is decreased below a threshold. Moreover, it is not clear to me that this is a standard mechanism of control and, thus, I am of the opinion that it would not be obvious to the person skilled in the art to implement it. I do not see why a person skilled in the art would be motivated to make this alteration via non-inventive means. Therefore, I consider that Claim 1 is inventive in light of either of D1, D2 or D6.
Obviousness in light of D4 or D5
D4 and D5 both disclose dynamically adjusting the magnitude of a control element effect (such as scrollbars, buttons, resize handles or dropdown menus) in a graphical user interface based on the intensity of the touch. However, I do not consider that the utilisation of scrollbars, buttons, resize handles or dropdown menus is the equivalent of a sequence of images in the background of the user interface. Similarly, neither of these documents discloses anything equivalent to “reversing” the control element effect when the intensity is subsequently decreased past a threshold.
If a person skilled in the art were to use the teaching of D4 or D5 with respect to adjusting the magnitude of a control based on the contact intensity, and applied this control element effect to a sequence of images, it follows that the rate of changing of the sequence would be proportional to the contact intensity. That is to say, a harder touch would play the sequence faster than a lighter touch. There is no teaching or suggestion to begin the sequence based on a first contact intensity, and to reverse it when the contact intensity is decreased below a threshold. I do not consider that this is a standard mechanism of control and, thus, it would not be obvious to the person skilled in the art. I do not see why a person skilled in the art would be motivated to make this alteration via non-inventive means. Therefore I consider that Claim 1 is inventive in light of either of D4, D5.
Obviousness in light of any one of D1, D2 or D6 combined with either D4 or D5
Given the above reasoning, if a person skilled in the art took the teaching of either of D1, D2 or D6 and combined it with the teaching of either of D4 or D5 (and I have not made a finding whether such a combination would be reasonable or not), at least one feature would be missing from the combination. Simply put, there would be no teaching in the combination to begin the sequence of images based on a first contact intensity, and to reverse it when the contact intensity is decreased. Furthermore, I do not see why a person skilled in the art would be motivated to make such an alteration via non-inventive means. Therefore, I consider that Claim 1 is inventive when one of D1, D2 or D6 is combined with either of D4 or D5.
Obviousness in light of any one of D1, D2 or D6 combined with D3
For similar reasons, if a person skilled in the art took the teaching of either of D1, D2 or D6 and combined it with the teaching of D3 (again, I have made no finding on the legitimacy of such a combination), they would arrive at the similar result to that indicated above. While the combination could be said to show that, as the contact intensity decreased, the magnitude of the effect decreased accordingly, there would still be no teaching, or motivation, to play the “ripple effect” in reverse when the contact intensity decreases.
Therefore, I can see no reason as to why a person skilled in the art, in considering the prior art and being faced with the aforementioned problem, would be motivated to display in sequence at least some of the sequence of images based at least in part on the characteristic intensity of the first contact, and subsequently reverse the sequence when that intensity dropped below a threshold. Therefore, I consider that this feature makes an inventive contribution to the claim over the prior art.
As the proposed amended Claims 2-18 are appended to Claim 1 it follows that these claims also make an inventive contribution over the prior art, mutatis mutandis.
CONCLUSION
I consider that the claims as proposed to be amended are novel and inventive. I am satisfied that no other ground of objection applies. Therefore, I will direct that the application be accepted.
Greg Powell
Delegate of the Commissioner of Patents
4
0