Apple Inc. - [2022] APO 15

IP AUSTRALIA
AUSTRALIAN PATENT OFFICE
Apple Inc. [2022] APO 15
Patent Application: 2017208225

Title:Device, method, and graphical user interface for adjusting the appearance of a control

Patent Applicant: Apple Inc.

Delegate:Dr W.E. Guinea

Decision Date: 24 February 2022

Hearing Date: Written submissions filed on 31 January 2020 and 16 March 2020

Catchwords: PATENTS – examiner objection – manner of manufacture – graphic user interfaces – blur function and colour emphasis applied to a control background – claimed invention not for a manner of manufacture – patentable subject matter may exist in the specification – application remitted to examination

Representation: Patent attorney for the applicant: Carl Harrap, Elyse Schinella from FPA Patent Attorneys Pty Ltd

IP AUSTRALIA
AUSTRALIAN PATENT OFFICE

Patent Application: 2017208225

Title:Device, method, and graphical user interface for adjusting the appearance of a control

Patent Applicant: Apple Inc.
Date of Decision: 24 February 2022

DECISION

Claims 1-19 are not directed towards patentable subject matter. The specification contains content which, following amendment, could provide for a manner of manufacture. The application is remitted to examination. Pursuant to Regulation 13.4(3) the Applicant has twelve months from the date of this decision to gain acceptance of this application.

REASONS FOR DECISION

Background

Patent application 2017208225 (the “application”) was filed on 25 July 2017 and claims an earliest priority date of 15 March 2013. The application is a divisional application of 2014238102.
Apple Inc. (the “Applicant”) requested examination of the application on 25 July 2017. Two adverse examination reports were issued. The first examination report issued on 19 September 2018. This examination report contained a manner of manufacture objection, and a clarity objection under section 40. The Examiner elected to reserve opinion on novelty and inventive step.
The Applicant responded to the report and filed proposed amendments on 23 July 2019 (“First Examination Response”).
A second adverse examination report issued on 10 September 2019. This report maintained the manner of manufacture objection. In view of the proposed amendments the Examiner withdrew the clarity objection. The Examiner continued to reserve opinion on novelty and inventive step.
On 19 September 2019 the Applicant wrote to the Commissioner requesting to be heard in relation to the outstanding objection. The Commissioner replied to the Applicant on 03 December 2019 indicating that the hearing would be conducted by way of written submissions. The Applicant filed a submission (the “Applicant’s submissions”) on 31 January 2020, on the due date.
A letter seeking further comment regarding Claims 7-8, 16-17 was sent to the Applicant on 02 March 2020. The Applicant responded on 16 March 2020 and elected to amend these claims (“Second Amendments”). Therefore, in this decision, I shall consider the claims as proposed to be amended via this Second Amendment and note that the amendments appear to be allowable. In this same letter the Commissioner gave the Applicant the opportunity to submit comments on what they believed the common general knowledge regarding blur functions and colour filters (that is, colour emphasis) was. I shall make further comments upon the Applicant’s submissions (“Applicant’s letter response”) in regard to these below.
The Specification – Description and Drawings
The description follows a largely traditional layout. It starts by describing that the Technical Field is related to electronic devices with displays that display soft keyboards and other virtual controls. This is followed by a Background, Summary, Brief Description of the Drawings, and a Description of the Embodiments. This latter section is further broken into two sub-sections: Exemplary Devices, and User Interfaces and Associated Processes.
The Background of the specification is briefly described in paragraphs [0002]-[0003]. These paragraphs disclose that the proposed invention is related to virtual controls on electronic device displays. These paragraphs are reproduced below verbatim.
[0002] Electronic computing devices often display virtual controls on their displays. For example, keyboards, menus, dialog boxes, alerts, and other controls may appear and disappear while using applications on a portable electronic device (e.g., a smart phone, tablet, or notebook computer).
[0003] The initial display of a virtual control may be distracting to a user, causing the user to momentarily lose their place in an application. In addition, virtual controls that are displayed on top of other elements in an application user interface obscure or hide those elements, which also makes it harder for a user maintain context in an application. This is particularly true for portable devices with small displays, where the virtual control (e.g., a keyboard) may occupy a significant portion of the display.
The summary section of the specification discloses that the invention is directed towards new methods and interfaces for virtual controls which are less distracting for a user.
[0004] Accordingly, there is a need for electronic devices with less distracting virtual controls that help to maintain context in an application. This requires new methods and interfaces for adjusting the appearance of virtual controls. Such devices, methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface.
This is further expanded upon in the Description of the Embodiments, paragraphs [0026]-[0028]:
[0026] A given translucent control is displayed over content in a particular region of the display. The device determines values for various content-display properties in the particular region (e.g., values for the hue, saturation, brightness, and/or contrast of the content underneath the control). In turn, the device determines a set of values for various control appearance parameters (e.g., values for blur radius, saturation adjustment, opacity of a white overlay, opacity of a black overlay, opacity of user interface elements in the control, and/or color of text in the control) based on the determined values for the content-display properties. The appearance of the translucent control is then based on the content underneath the control and the determined values of the control-appearance parameters. The values for the content display properties, the values for the control-appearance parameters, and thus the appearance of the control are automatically, dynamically updated as the content underneath the control changes.
[0027] These adjustments produce a translucent control whose appearance is less distracting than an opaque control or a translucent control that simply blurs the underlying content in a fixed manner. At the same time, these adjustments produce a translucent control that is easy to see and interact with. Overall, these adjustments provide translucent controls that reduce the cognitive burden on a user and produce a more efficient human-machine interface.
[0028] Note that the adjustments described herein to translucent controls can also be applied to other types of translucent user interface elements (e.g., docks, trays, and status bars).
Figures 1A-1B, 2 and 3 disclose block diagrams containing both hardware and software features of exemplary electronic devices which utilise touch-sensitive displays. The features of these exemplary devices are described in paragraphs [0034]-[00122], and some specific exemplary devices are provided in paragraph [0034], and which is reproduced below. None of the paragraphs indicate that a technical effect is present via an unusual interaction with any of these hardware features. Rather these paragraphs outline features which are typical of portable electronic devices of the types described in paragraph [0034].
[0034] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communications device, such as a mobile telephone, that also contains other functions, such as PDA and/or music player functions. Exemplary embodiments of portable multifunction devices include, without limitation, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. Other portable electronic devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and/or touch pads), are, optionally, used. It should also be understood that, in some embodiments, the device is not a portable communications device, but is a desktop computer with a touch-sensitive surface (e.g., a touch screen display and/or a touch pad).
Figures 5A-5K provide illustrations of the user interface according to a variety of preferred embodiments. The figures are explained in paragraphs [00130]-[00131] and [00137]-[00154]. The most relevant of these figures are reproduced and described in more detail below.
The claimed invention is best understood by reference to Figures 5E, 5F, and 5I. Figure 5I shows a default “user interface image” located within the touchscreen display (item 112). This particular user interface image is modified via translation, zooming, and panning for each of Figures 5A-5H, 5J-5K. The user interface image in Figure 5I has not had a blur function applied to it nor any colour emphasis. This user interface image could be the operating system background, an image viewed in an application, or similar. As noted in paragraph [0026] (reproduced above), this is not limited to a picture and may be any content, and the properties of the control are dynamically updated as the content underneath the control changes.
Figure 5F displays a virtual control, in the form of a keyboard, overlaying the user interface image. The virtual keyboard extends from the bottom of the display up to the horizontal line half-way up the display, and the features of the keyboard (letters, key outlines, symbols) are black in colour. The user interface image has been unaltered above the virtual keyboard – that is, no blurring has been applied to the image and no colour emphasis has been added. The user interface image underneath the keyboard has had a blur function applied to it and is referred to as the “control background”. The blur is clearly observed when comparing the image of the tree branch beside the virtual keyboard’s Q and A keys to that of the unaltered image above the keyboard. An enlarged portion of this section of Figure 5F (annotations added) is also provided to illustrate the concept more clearly. Using this enlarged portion, it is observed that the details of branches and needles on the trees may be discerned on the user interface portion of the image (upper). The details of the branches and needles cannot be properly discerned within the control background (lower), which has been blurred. No detail is provided regarding colour emphasis of the control background for this particular figure. Paragraphs [00137] and [00141] describe that the appearance of the virtual keyboard is based on the content under that virtual control.
Figure 5E (above) has similar properties to Figure 5F. In this figure the colour of the virtual keyboard is configured such that it is white. The user interface image has a blurring function applied to it and the colour emphasis is such that the user interface image has been darkened in order to form the control background.
Figures 5A, 5C-5D also demonstrate the blurring and colour emphasis effects, similar to that shown in Figure 5F. The key differences between these figures reside in the diameter of the blur radius, in the colour of the onscreen keyboard, and differences in the contrast and brightness of the control background image. Figure 5B shows the portion of the underlying user interface image which is manipulated to form the control background image.
Figure 5H shows an embodiment whereby a second control region (in the form of box 520 containing a symbol of a ringer) is displayed partially in front of a first control region (the onscreen keyboard). The first control region has had blur and colour emphasis added to it as described above. In the second control region the blur function and colour emphasis are applied only to the second user interface background. Elements of the first control region that are nominally positioned between the background image and the second control region are not used to determine the background of the second control. The effect of this is that there is no visible stacking of blurred user interfaces which happen to overlap in some way. That is, in the example the onscreen keyboard control elements that are “behind” the ringer are not visible and therefore are not available to be blurred. Similarly, the colour emphasis is only applied to the portion of the user interface image under the second control region. This is described in paragraphs [00151]-[00152] of the application.
Figure 5J shows onscreen text, indicative of a personal digital assistant (PDA), in front of the user interface image. The specification at paragraph [00153] indicates that, in transitioning from Figure 5I to 5J, the user interface image, located behind the PDA text, would be modified by applying a blurring animation designed to indicate that the image is moving out of focus and receding into the background. The specification is silent on how this effect is to be achieved, and only describes the desired effect in general terms.
Figure 5K displays an embodiment whereby control 522, comprising virtual sliding button 526, is displayed above the user interface image. Part of the user interface image forming the background image of control 522 is not modified (see e.g., item 524). Other areas of the user interface image forming the background of the control are blurred and have colour emphasis applied to them. This is described in paragraph [00154].
Figures 6-8 provide simple example illustrations of image properties which may be modified to form the control background image, for example hue, saturation, brightness, and contrast. It is noted that only relative values can be determined from these figures, and that only one example is given. The figures are best described as illustrative only. The description of these figures at paragraphs [00132]-[00136] is also general in nature and indicate that the skilled addressee would be expected to utilise their common general knowledge to implement the concept.
As the above embodiments illustrate, the invention is related to the production of virtual controls and their visual interaction with the underlying background image or other content. The aim of the invention appears to be in providing combinations of blur and colour emphasis to the background image of the virtual control(s) and modifying the control elements such that the user interface underlying the virtual control(s) may simultaneously be discerned while the virtual control is more easily identified and interacted with. It is desired that there be a reduction in terms of cognitive burden of the user as a result of the blurring and colour emphasis.
The Skilled Addressee
Based on the above description of the specification it becomes apparent that the problem to be solved is related to the visual appearance of the user interface. I characterise the problem to be solved as follows: How should a user interface be designed such that the display of a virtual control(s) reduces distraction and maintains context within an application, whilst simultaneously allowing the control to be identified and interacted with?
From the above embodiments it is apparent that the skilled addressee must be one who is familiar with the programming of user interfaces. Further, the majority of the specification is directed towards the application of specific image manipulation effects to portions of the user interface (the control elements and the control background). The skilled addressee therefore must also be familiar with programming image manipulation techniques so that the desired functionality may be obtained. Since most user interfaces are visual in nature, I consider that someone who programs user interfaces would also be able to implement such functions.
There are few details on the settings required to implement the image manipulation effects, such as the type or quantity of blur, or how the colours are emphasised. In order to put the invention into practice the skilled addressee needs to also possess a background such that they can implement the desired image manipulation effects in the described manner. At the level of detail presented and based on the knowledge that a user interface programmer will have experience with designing and modifying images in order to present their user interface, I am satisfied that the skilled addressee would be a user interface programmer.
Common general knowledge required to implement the invention
As described above in paragraph 6, the Commissioner sought comment from the Applicant in relation to what they believed the common general knowledge was regarding blur functions and colour filters. Page 5 of the Applicant’s letter response submits the following: (emphasis added)
“In this case, a consideration of the functions available in Adobe Photoshop or any other image editing application is therefore an incorrect starting point. Adobe Photoshop or any other imaging editing application are directed to providing users with a software application that includes image editing options and tools. They provide a vast array of image editing functions and to focus specifically on a combination of blur and colour filters is using hindsight reasoning.
Further, and arguably more fundamentally, it is not understood to be common general knowledge for a device to provide a user interface by making a request to an image editing application. The device may control the user interface of the image editing application, but does not involve use of the imaging editing functions of that application. Paragraph [00123] of the specification as filed describes that various application icons are shown and the user may launch one or more of them by a selection action. As noted in paragraph [00124], the icons are merely exemplary. If Adobe Photoshop were installed on the relevant device, it may have an associated selectable icon.
As such, Applicant respectfully submits that the functions provided by image editing applications are not relevant to the present inquiry. These functions are provided in a completely different field.”
While I do not necessarily disagree with the Applicant’s submissions, I note that in order to correctly construe the claims and identify the invention it is necessary to understand who the notional skilled addressee is and what common general knowledge is available to them. Failure to do so means it is not possible to identify what the inventor has allegedly added to human knowledge and whether or not they have added something which, in substance, is technical in nature. Similar reasoning was presented in D’Arcy v Myriad Genetics Inc. [2015] HCA 35 (“Myriad”) at [253].
The Applicant’s specification and submissions do not review the displayed appearance of user interfaces in the absence of their invention, and do not describe the prior art in the specific context of the background image for virtual controls.
In the absence of any review by the Applicant in the specification, I have both considered and accept what the Applicant has proposed in the Applicant’s submissions (p10 third dot point) in relation to what the prior art indicates is common general knowledge. Namely that:
“…there was some use of transparency in application windows or indication windows. The transparency is to allow content beneath the application window or indication window to be viewed.”

I also conclude that any other image or solid colour may be used in the background image of virtual controls and virtual indicators, and that this knowledge would be available to the skilled addressee. Such information is treated as common general knowledge within the Background sections of D1 and D2, which were presented as examples indicative of the common general knowledge by the Examiner and is also implied in paragraph [0003] of the present application via “… virtual controls that are displayed on top of other elements in an application user interface obscure or hide those elements …”.
I will now return to the request for clarification on blur functions and colour filters. Blur functions, as I understand them, are image manipulation effects which enable specific visual features to be de‑emphasised (blur function applied) while others are emphasised (no application of blur function). The term “blur” is given its plain meaning.[1] Consequently, a blur function is a function which makes an item appear more confused in outline or makes it appear indistinct so that the blurred features are de-emphasised. The present application uses a blur function to modify an image, in this case an image of the user interface which would reside under a virtual control. Based on the level of detail supplied in the specification regarding blur functions, either the blur functions must form part of the common general knowledge available to the skilled addressee, or such functionality cannot satisfy the requirements of section 40. Based on the identified skilled addressee I am satisfied, on balance, that the programming or incorporation of libraries containing blur functions would be considered part of the general knowledge available to the skilled addressee, and that methods of their implementation and use would also be well known to the skilled addressee.
[1] verb: to obscure by making confused in form or outline; make indistinct
My understanding of colour filters is that they are image manipulation effects which modify the properties of at least a portion of an image in a uniform and consistent way. For example, the property modified may be the value of one or more of: RGB (red, green, blue), HSL (hue, saturation, lightness), BCG (brightness, contrast, and gamma), opacity, and other similar settings. This is commensurate with the image emphasis function described in the specification, and to which I shall refer to simply as “colour emphasis” hereon. On balance, I consider that the use of colour emphasis would also form part of the knowledge base and be well known to the skilled addressee. They would be able to program the functions or incorporate libraries containing functions to do so and would be able implement the functions using their common general knowledge. Again, failure to do so would result in a failure to satisfy the requirements of section 40.
The Specification – Claims
The specification as proposed to be amended in the Second Amendments, dated 16 March 2020, contains a total of 19 claims. Claims 1 and 10 are independent claims, and I reproduce Claim 1 below. The full set of claims, as proposed to be amended, is attached in Annex 1. . Claim 1, as proposed to be amended, is: (enumeration added to each feature)
(1.0) A computer program comprising instructions which, when executed by an electronic device with a display, cause the electronic device to:
(1.1) display a user interface on the display;
(1.2) while displaying the user interface on the display, receive a request to display a control region overlaid on at least a first portion of the user interface;
(1.3) in response to the request, display a control region overlaid on the user interface, wherein displaying the control region includes:
(1.4) displaying a background of the control region, wherein the background of the control region has an appearance that is based on a blurred version of the first portion of user interface with visual properties of the blurred version of the first portion of the user interface changed to emphasize the colors in the first portion of the user interface on which the control region is overlaid; and
(1.5) displaying one or more control elements on the background of the control region;
(1.6) while displaying the control region, detect an input that corresponds to activation of a respective control element of the one or more control elements; and,
(1.7) in response to detecting the input that corresponds to activation of the respective control element, perform an operation that corresponds to the respective control element.
Independent Claim 10 presents a method comprising essentially the same features as Claim 1. Similarly dependent claims 11-18 contain essentially the same features as Claims 2-9. Consequently, my considerations of Claims 1-9 are applicable to Claims 10-18. Claim 19 comprises any suitable electronic device which is configured to run the method of Claims 10-18. My considerations of Claims 1-9 therefore also address this claim.
Claim construction
On p5 section 5 of the Applicant’s Submissions, it is submitted that there are no difficulties in construing the independent claims, and that the terms themselves are well understood within the art. However, the specific combination of the terms in the claimed features present limitations which might not be immediately obvious to the skilled addressee.
Feature 1.0 relates to a computer program which is executed on an electronic device with a display. These terms are given their plain meaning. Feature 1.1 requires that a user interface is presented upon the (electronic) display. The term ‘user interface’ indicates that the user is to somehow interact and modify some function of the electronic device via an interface. For the purposes of this claim it is apparent that the physical display device is distinct from the user interface. The user interface, at least in the context of the claims, is some sort of image rendered in software which is presented upon the physical display of the electronic device. The physical inputs required for the user to interact with the electronic device, for example physical buttons or touchscreens, are not considered to be part of the user interface. Rather they are considered part of the electronic device.
In feature 1.2, while the user interface is displayed, the electronic device receives a request to display a (virtual) control region and in feature 1.3 the electronic device responds to this request and displays “a control region”. I have construed this second control region in feature 1.3 to be “the control region” of feature 1.2. The request is for the control region to be overlaid on at least a first portion of the user interface. This defines that the control must be capable of being positioned in front of the user interface and, as per the plain meaning of overlaid[2], indicates that the control forms one or more layers of some sort. The use of controls in layers would be well known and understood by the skilled addressee. It is apparent that the “first portion” is simply defined by the area of the user interface that the control region overlies. Consequently, the size of the control region is such that it cannot be larger than the entire user interface.
[2] verb: to lay or place (one thing) over or upon another
A distinction is also made between the user interface and the control region in feature 1.2 and again in feature 1.3. This is unusual in that controls (and therefore control regions) are typically the part of the user interface by which the user interacts with the electronic device. Consequently, while the control region would generally be understood as forming part of the user interface, it is quite clear that the claim is distinguishing the control region concerned as a distinct subcomponent of the user interface that is overlaid on the remainder of the user interface.
In this light it also worth considering claim 7 which is reproduced below:
The computer program of claim 1, wherein the background of the control region does not display elements that are in one or more other layers that are between the control region and the user interface.
Here it is apparent that claim 7 is introducing a limitation whereby other “elements” are also distinguished from the user interface. These other “elements” are also overlaid on the user interface in some way, but still lie behind the control region as recited in claim 1. Based on claim 7, it is apparent that a distinction is made between both the other “elements” and the control region that overlies the user interface in some way. What specifically these other “elements” are, and how they differ from the user interface is not claimed. They, could, for example, be other virtual controls, virtual indicators, or components thereof, but are not necessarily so limited. Feature 1.4 results in the appearance of the control region’s background being defined such that it is based on a blurred version of the first portion of the underlying user interface image. As the background is only based on a first portion of the user interface it is possible that other controls or elements, which are located between the current control and the user interface, are also included in the image manipulation of Claim 1. This also allows differentiation from Claim 7.
The appearance of the control background is modified so that the first portion is blurred. The skilled addressee would likely know several image manipulation functions which are capable of achieving this result. After blurring, the background image is further changed to emphasize the colours of the user interface image. The term “emphasize” is also given its plain meaning[3]. This partial feature appears to be at odds with the general concept of an overlayed virtual control and what the specification presents in Figures 5A-5J. These controls are typically designed such that they stand out from the background so that the user is aware of them, and that they are easier to interact with. The present claims stipulate the control background is emphasised in such a way that it emphasises the colours in the first portion of the user interface. This emphasis might result in reducing distraction of the user, albeit at the expense of increased cognitive burden.
[3] verb: to give emphasis to; lay stress upon; stress.
Feature 1.5 displays one or more control elements upon the control background. No details are claimed which indicate what the control elements are, how these control elements are displayed, or the functionality which the control elements trigger. As an example only, and with reference to Figure 5F, the control element could represent a single key on the virtual keyboard, such as the “A” key. Such a control element would therefore entail the key outline of a particular colour (black in Figure 5F), a control element background (which, in this case, is transparent or identical to the control background) and a key label displaying “A” (also black in Figure 5F). Alternately, the control element might simply be the outline of the key, or only the text label, and several of the elements would be needed to form a control. It could also comprise something of a solid shape or colour displayed on the background of the control region.
There are no limitations within the claims regarding the colours to be used for the control element features, no size requirements, and there is no other relationship between the control elements and control background other than the control elements appearing in front of the control background. This could lead to scenarios where the control might not be easier to use. For example, if the control elements and control background were implemented in a similar light colour, the blurring was a small radius (and thereby less noticeable), and the colour emphasis applied only a small increase in the brightness then the control elements might be harder to distinguish than if traditional control elements using transparency or solid colours were used. I note that paragraphs such as [00137] and [00141] contain potential display features for the control elements, however there is no reason to defer to these paragraphs in the absence of claimed features.
The remaining features, 1.6 and 1.7, detect an input from the electronic device related to the virtual control and carry out the operation to which the control element refers. Apart from what is discussed above, there are no interactions between the features. Each feature is effectively distinct.
The Applicant has used several other terms in their various submissions and documents, such as translucent and virtual controls. The skilled addressee would use the plain meaning of these terms and, since nothing turns on these terms, no further comment is required.
Examination Reports 1 and 2
In Examination Report No 1, the Examiner’s objection included the following reasoning in the Manner of Manufacture objection:
“It was well known at the priority date to display the user-interface control such as an on screen keyboard on the display of a computing device. It was well known for item son [sic] the display of computing device to be overlaid on one another. It was well known for user interface controls to have translucent portions that show a blurred version of the content displayed underneath.
Thus when considering the claimed features in combination, the specification as a whole, and the state of the art, it is apparent that the applicant’s contribution, the ingenuity of the inventors and therefore the substance of the claimed invention lies not in a new or improved user-interface controls but merely in the choice of its aesthetic appearance.”
and
“The claimed invention as a matter of substance is considered a presentation of information characterised solely by the content. The contribution to the claimed invention is considered purely aesthetic, and does not appear to overcome any technical difficulty. While the claimed invention can be implemented on any computing device with a display therefore, the device adds nothing to the substance.
The description states that the new interface results in the benefits of a reduced the cognitive burden on a use and produces a more efficient human-machine interface. However, the description does not provide further detail to elaborate on these assertions.”
Feature 1.4 is the only feature which could give rise to the Examiner’s opinion that the substance is related to the choice of a user interface’s aesthetic appearance. The other features are clearly related to operations of the user interface, or the display of the user interface.
A similar line of reasoning and conclusion regarding the substance was given in Examination Report No 2.
An objection related to clarity was also taken in Examination Report No 1. The objection was withdrawn in Examination Report No 2. Opinion on novelty and inventive step was reserved in both Examination Reports.
Consequently, the remainder of this decision will focus on whether or not this invention is for a manner of manufacture. If I find that it is for a manner of manufacture, then it is necessary for novelty and for inventive step to be evaluated and the application would be remitted for further examination. If I find that the invention is not for a manner of manufacture, then it may be that evaluation of novelty and of inventive step is a moot point if the entire subject matter of the specification is clearly directed towards non-patentable subject matter and refusal would thereby be appropriate. Conversely it may be possible to overcome a finding of there being no manner of manufacture in the claimed invention by amendment, and in that case, I would seek to remit the application for further examination. The latter is the appropriate course of action as will be understood below.
Applicable Law and Manner of Manufacture
The Intellectual Property Laws Amendment (Raising the Bar) Act 2012 came into effect on 15 April 2013 and predates the filing date of this application. Thus, the standard of proof that applies in the present case is the balance of probabilities.
As identified in paragraphs 2 and 4 above, both Examination Reports contain a manner of manufacture objection. Specifically, they start with the following sentence: “Claims 1-19 do not define a manner of manufacture within the meaning of Section 18(1)(a) of the Patents Act 1990.”
Section 18 of the Patents Act 1990 provides that:
(1) Subject to subsection (2), an invention is a patentable invention for the purposes of a standard patent if the invention, so far as claimed in any claim:
(a) is a manner of manufacture within the meaning of section 6 of the Statute of Monopolies;
...
In National Research Development Corporation v Commissioner of Patents [1959] HCA 67 (“NRDC”) the High Court considered whether a method for eradicating weeds from crop areas possessed a manner of manufacture. In NRDC [14] at 269, the High Court stated that
“… The right question is: ‘Is this a proper subject of letters patent according to the principles which have been developed for the application of s. 6 of the Statute of Monopolies?’”
NRDC [22] at 275 also explained that (citations removed)
“… The point is that a process, to fall within the limits of patentability which the context of the Statute of Monopolies has supplied, must be one that offers some advantage which is material, in the sense that the process belongs to a useful art as distinct from a fine art … - that its value to the country is in the field of economic endeavour. … ”
Myriad at [23] explained that the NRDC ruling was not a strict formula to be applied to ascertain whether an invention was patentable, rather a case-by-case analysis was required:
“This Court in NRDC did not prescribe a well-defined pathway for the development of the concept of ‘manner of manufacture’ in its application to unimagined technologies with unimagined characteristics and implications. Rather, it authorised a case-by-case methodology. Consistently with that approach, and without resort to the ‘generally inconvenient’ proviso in s 6 of the Statute of Monopolies, there may be cases in which the court will decide that the implications of patentability of a new class of invention are such that the invention as claimed should not be treated as patentable by judicial decision.”
The claimed invention is directed to a computer program, a method of executing said computer program, and electronic devices configured to execute said method. In this respect Research Affiliates LLC v Commissioner of Patents [2014] FCAFC 150 (“Research Affiliates”) and Commissioner of Patents v RPL Central Pty. Ltd. [2015] FCAFC 177 (“RPL”) provide further guidance in how section 18(1)(a) should be applied to computer-related inventions.
RPL at [96]-[98] identifies that: (emphasis added)
“A claimed invention must be examined to ascertain whether it is in substance a scheme or plan or whether it can broadly be described as an improvement in computer technology. The basis for the analysis starts with the fact that a business method, or mere scheme, is not, per se, patentable. The fact that it is a scheme or business method does not exclude it from properly being the subject of letters patent, but it must be more than that. There must be more than an abstract idea; it must involve the creation of an artificial state of affairs where the computer is integral to the invention, rather than a mere tool in which the invention is performed. Where the claimed invention is to a computerised business method, the invention must lie in that computerisation. It is not a patentable invention simply to 'put' a business method 'into' a computer to implement the business method using the computer for its well- known and understood functions.
Is the mere implementation of an abstract idea in a well-known machine sufficient to render patentable subject matter? Is the artificial effect that arises, because information is stored in RAM and there is communication over the Internet or wifi, sufficient? Does any physical effect give rise to a manner of manufacture? Are the mere presence of an artificial effect and economic utility, without more, sufficient to determine manner of manufacture?
It is not a question of stating precise guidelines but of deciding, in each case, whether the claimed invention, as a matter of substance not form, is properly the subject of a patent.”

Research Affiliates at [103] identifies, for the purposes of assessing manner of manufacture, that:
“… there is a distinction, between mere implementation of an abstract idea in a computer and implementation of an abstract idea in a computer that creates an improvement in the computer”.
RPL at [112] states that: (emphasis added)
“Recognising that the claims are to a method and system comprising a combination of integers, it is necessary to understand where the inventiveness or ingenuity is said to lie. Turning to the integers of the invention as set out at [36] and [38] and summarised at [37] and [39] above, it is apparent that, other than the integers providing that the computer processes the criteria to generate corresponding questions and presents those questions to the user, the method does not include any steps that are outside the normal use of a computer.”
The considerations used to assess the substance of the invention from RPL and Research Affiliates, and others, were summarised by the Delegate in Aristocrat Technologies Australia Pty Limited [2016] APO 49 (Aristocrat ‘16) at [35]:
“I conclude that it is relevant to consider a range of matters. Without seeking to be exhaustive, these include:
·there must be more than an abstract idea, mere scheme or mere intellectual information;
·is the contribution of the claimed invention technical in nature;
·does the invention solve a technical problem within the computer or outside the computer;
·does the invention result in improvement in the functioning of the computer, irrespective of the data being processed;
·does the application of the method produce a practical and useful result;
·can it be broadly described as an improvement in computer technology;
·does the method merely require generic computer implementation;
·is the computer merely an intermediary or tool for performing the method while adding nothing of substance to the idea;
·is there ingenuity in the way in which the computer is utilised;
·does the invention involve steps that are foreign to the normal use of computers; and
·does the invention lie in the generation, presentation or arrangement of intellectual information.”
Also of immediate relevance are decisions which relate to the use of colour or image-based effects, and I will briefly list these below.
The Moore Business Forms, Inc. [1979] AOJP 2521 (“Moore”) decision found that the printing of alternating lines of background colours resulted in a technical effect. The claims, in part, recited that a “… business form … having printed thereon in half-tone a plurality of continuous parallel bars of two different colours, the bars extending transversely across the form in a direction perpendicular to the row of feed holes in spaced relation to form unprinted bars between said printed bars, alternate printed bars being of alternate colours whereby a line of characters printed on one printed or unprinted bar will be clearly distinguishable from lines of characters printed on other bars on the surface of the form.” (emphasis added). The details of which colours were preferred, and the line-printing density appeared in the dependent claims.
In the decision the delegate identified that the printing resulted in a user being able to read text which had been printed at 8 lines per inch, as opposed to 6 lines per inch, without “the use of a straight-edge or some other guide to assist in line separation during a reading exercise”. The coloured bars effectively acted as that straight edge. When the invention was put into practice the result was that text could be printed at a higher density, using fewer paper resources and without recourse to additional reading aids. The delegate concluded that this was a technical effect, and consequently the use of particularly arranged coloured lines may be patentable subject matter. I also note that the desired technical effect appears in the claim (see emphasis above).
In ITS Rubber Limited’s Application (1979) RPC 318 (“ITS Rubber”) a squash ball, characterised by a blue colour, was found to possess a manner of new manufacture. It was described that the use of blue provided “superior visual perception … to both the black conventional ball and the green ball”. The use of the specific blue colour against the white background of the court provided an overall objective improvement which was considered patentable.
The invention in Cooper’s Application (1902) 19 RPC 53 was directed towards leaving a blank, longitudinal space during the printing of a newspaper. No printed material would occupy the blank space. The newspaper could then be folded through this blank space, and the reader would “thus avoid the trouble involved in reading over a folded part of the paper.” It was found that this blank space was an improvement in the newspaper such that it was physically more convenient to use. Consequently, it possessed a manner of new manufacture.
From the above it is apparent that the use of background colours and an arrangement of information in and of itself does not immediately result in an invention being unpatentable. Similarly, the visual layout may also result in a technical improvement which possesses a manner of manufacture if some material or technical advantage is also present. It is thus necessary to identify what, if any, material advantage the use of such colours and image manipulation effects possesses.
The Applicant’s Submissions
The Applicant submitted in the First Examination Response that:
“In reducing the invention to ‘a presentation of information characterised solely by the content’, however, the Examiner has focused on an isolated part of the invention only without considering the context in which that part operates or – accordingly – the substance of the invention as a whole.
When the specification and claims are properly considered, Applicant submits that characterising the invention as ‘a presentation of information characterised solely by the content’ simply is not reasonable. Doing so ignores both the claims (which are clearly directed to more than this) and the detailed disclosure and figures supporting those claims.”
This was elaborated upon in Section 6.2 of the Applicant Submissions. The Applicant submits that the problem the claims aim to address, and the result of the entire claim is: (Applicant’s Submissions, p6 final para - p7 para 1, italicised roman numerals added)
“Still further, what is being displayed is a virtual control which is requesting input from the user. Another problem is therefore displaying the virtual control in a way that facilitates operation of the controls as required by the user. It will be appreciated that this may involve a trade-off with the problem of maintaining context or a place in an application. That trade-off may be particularly keen for electronic devices that are associated with small display screens where the display of the virtual control on top of another element in an application user interface occupies a significant portion of the display.
The solution provided by the claimed invention results in a new user interface that operates in a way that achieves a new balance in the trade-off mentioned above. In at least some implementations and use cases the balance may provide for (i) a control region that is less distracting to the user and assists the user in maintaining context in an application, while still permitting the user to observe and interact with the virtual control elements. (ii) In contrast to conventional opaque and translucent controls, the control region of the present invention may provide a less distracting control region that assists the user to maintain context in an application. (iii) Further, the interoperation of the one or more control elements and the background of the control region, in the context of the invention, may assist the user in identifying the location of the control elements and interacting with these control elements on the display while also assisting the user to maintain context in an application. This is particularly beneficial for portable electronic devices that are often associated with small display screens (see, for example, paragraphs [0023]-[0027] of the specification as filed).”
The Applicant has submitted that: (AS p7 para 3)
“Applicant submits that the contribution of the claimed invention, provided by at least the specific implementation steps identified above, provide a technical solution to the technical problem faced by conventional electronic devices. For example, the state of the art (see e.g. D1 and D2 discussed below) specifically recognises that a problem in user interface design is how to reduce distraction of users. It is submitted that the evidence shows that the relevant skilled person considers this to be a technical problem, requiring a technical solution, such as utilisation of transparency in windows, timers and other mechanisms for generating and operating user interfaces.”
Page 10, Section 6.3 of the Applicant Submissions states:
“Examiner asserts that it was well known to have user interface controls to have translucent portions. However, the evidence shows windows and notifications as having translucent portions. No specific consideration has been given in D1 or D2 to soft keyboards or similar virtual controls that have an activation function. The mere knowledge of these virtual controls in different contexts does not result in the combination being so well known so as to not be able to be a relevant contribution for the purposes of the inquiry into patentable subject matter.
Applicant submits that the skilled person would understand the contribution to include the development of a user interface that comprises a control region including one or more control elements displayed on a background of the control region. The contribution thus provides a user interface for the specific situation in which an activation of a respective control element and performance of an operation corresponding to activation of this control element may be required.”
Consideration
- Is the contribution of the claimed invention technical in nature?
The Applicant has emphasised that it is not merely feature 1.4 that is to be considered, but rather the application of feature 1.4 to the remainder of the user interface features. It may also be inferred that the Examiner believed that the contribution to the art lay in the application of feature 1.4 to a user interface via the text appearing in the Examination Report No 1:
“Thus when considering the claimed features in combination, the specification as a whole, and the state of the art, it is apparent that the applicant’s contribution, the ingenuity of the inventors and therefore the substance of the claimed invention lies not in a new or improved user-interface controls but merely in the choice of its aesthetic appearance.”
Consequently, both the Applicant and the Examiner agree that it is the application of feature 1.4 to the other features of the claim that both represent the ingenuity of the inventors and is where any inventiveness should lie. It is the interaction of this specific feature with the other integers of the claim that determine whether or not the substance of the invention is technical or not. I see no reason to believe otherwise.
The Examiner has asserted that the application of feature 1.4 to the remainder of the claim’s features results only in an aesthetic effect being applied to a user interface, while the Applicant submits that the application of the feature solves a technical problem related to user interface design. I can accept that if the user interface is in fact easier to use as a result of the claim and results in a material advantage then this can be characterised as being technical in nature. It is clearly relevant to understand whether the claimed invention inevitably leads to a material advantage or, as the Examiner suggests, is nothing more than aesthetic in nature and purpose.
In items (i) and (ii), identified in paragraph 68, the Applicant has submitted that claim 1 “may provide a less distracting control region that assists the user to maintain context in an application.” (emphasis added)
Noting that the use of transparency and translucency in user interface images is known, that there are no algorithms or pseudocode, that the claims are directed to all types of blur functions and colour emphasis, and that there are no blur or colour emphasis ranges, then I fail to see how simply the utilisation of a blur function and colour emphasis in general would necessarily reduce potential distraction or improve a user’s ability to maintain context within an application. Each of these settings and the combination would have to be filled by the skilled addressee’s common general knowledge.
Depending upon the settings used by the skilled addressee for blur radius (for example), on the particular options used within the colour emphasis, on the colour and background of the control elements, and the particular user interface image used, the user’s context in an application might be difficult to establish. Certainly, the specific settings, or at least setting ranges, would be highly relevant in order to determine whether or not the use of a blur function and colour emphasis would improve or impair a user’s ability to maintain context. For the above reasons, and based on the balance of probabilities, I find that there is no evidence to demonstrate that such an effect will necessarily occur. This is in contrast, for example, to the situation in Moore and ITS Rubber. In both those cases it was apparent that there was an objective effect that provided for patentability; in Moore this was the ability to read denser text without an aid, while in ITS Rubber the blue colour provided for better visibility as compared to other colours.
Similarly, there is no evidence to suggest that a reduction in cognitive burden would necessarily result from putting the claimed invention into practice. Rather, a subset of the settings may be able to accomplish this feat, while other combinations oppose it. As an example, if the colours of the control elements are selected such that they are similar to that of the control background, then the cognitive burden is likely to be increased while users try to identify what control they are interacting with. This result of the invention, at least in the manner in which it has been claimed, is not considered to be a technical improvement to the user interface. In other words, the contribution of the invention as currently claimed in claim 1 appears to include effects that can be merely aesthetic or “subjective”– particularly since the skilled addressee can implement the blurring and colour emphasis in almost any way they choose to achieve any effect they choose.
The Applicant also submitted that there is an interaction between the control region background and the control element. As identified above, claim 1 does not define that the properties of the control element are dependent upon the properties of the control region background or vice versa. In fact, none of the claims define more than the mere display and activation of these control elements. However, substantial reference to modifying the appearance of the control elements is present in the specification.
The specification at paragraphs [00137], [00141]-[00143], and [00158]-[00159] discusses interactions between a control element (the keyboard) and the control region background only in general terms. Some embodiments describing how the skilled addressee could implement the controls and control region background are described in paragraphs [00144]-[00148]. However, I note that no details are supplied as to which settings should be used and when, other than the generic concepts disclosed in Figures 6-8 and paragraph [0158]-[00163]. These paragraphs merely describe that different “content-display values” (which are visual properties such as hue, saturation, brightness, and contrast according to [0158]) are determined for, presumably, the user interface and, dependent upon the result, applied to the control (see [0161]). No specific settings are described. The missing details would need to be filled by the skilled addressee’s common general knowledge and would be implemented in any manner in which they desired.
- Can it be broadly described as an improvement in computer technology?
Feature 1.4 applies two separate image manipulation effects to generate a background image upon which the control elements of feature 1.5 are displayed. The specification does not describe how to implement either of these image manipulation functions. The programming and implementation of the functions is left entirely to the skilled addressee to do as they see fit. The result is that there can be no improvement in how the functions themselves are implemented. Therefore, there is no apparent improvement in computer technology in relation to the programming of the blur function or in the programming of the colour emphasis.
Similarly, the specification does not provide any particular settings for the blur functions, colour emphasis, particular colour ratios, nor any pseudo-code, algorithms, or the like which might indicate a technical relationship between the blur functions, colour emphasis and the user interface image. The method of implementing and combining these features is such that it must form part of the common general knowledge of the skilled addressee. As discussed in the previous section, while the claimed invention might result in an overall improvement to a user interface which could be considered an improvement in the computer technology, it is not apparent that everything that falls within the scope of the claim necessarily will result in such an improvement.
- Is there more than an abstract idea, mere scheme or mere intellectual information?
It is clear from the specification as a whole that the application is not directed towards a business or financial scheme. Nor is it explicitly directed to making a user interface more “aesthetically pleasing”. In this respect I note that there are no references in the specification to artistry, aesthetics, or other terms which are meant to be pleasing to the eye. The specification is directed to the concept that image manipulation effects could improve a graphical user interface by reducing the cognitive burden of a user. Specific image manipulation effects, in the form of image blur and colour emphasis, have been identified and applied in combination in an attempt to accomplish this. Similarly, this proposed invention is not merely related to a specific piece of intellectual information or, in particular as is more relevant to this application, merely a specific image or other visual content.
In principle, there is potential for a material advantage to be realised from using blur functions and colour emphasis to objectively reduce cognitive burden or increase the ability of a user to maintain context in their application. Moore achieved such an objective effect through claiming specific effects (a “… plurality of continuous parallel bars of two different colours”) and the desired technical effect of implementing those effects (“whereby a line of characters printed on one printed or unprinted bar will be clearly distinguishable from lines of characters printed on other bars on the surface of the form”). In ITS Rubber, the use of the blue colour objectively improved the visibility of the ball when compared against the white background. However, in the present application, and as identified above, no details are present on how much blur or how much colour emphasis should be utilised, nor any relationship between them. The specification is largely, and the claims are certainly, presented in such a way that the result of the invention appears to be equally directed towards an artistic or aesthetic effect rather than necessarily resulting in a method to reduce cognitive burden and increase user context via a user interface. There appears to be no objective improvement.
Summary of considerations for the independent Claims

As described above, the invention may be implemented on any electronic device known in the art which has a display and some method of interacting with the elements of the user interface. There are no unusual interactions with any claimed feature and the electronic device, and therefore the electronic device aspects do not contribute to the substance of the invention.
The blur function and colour emphasis as such are not improved individually or in combination. They are merely applied to an image in whatever manner the skilled addressee sees fit. Further, the mere utilisation of a blur function and colour emphasis, in combination with the user interface, does not necessarily result in a technical effect.
There are no unusual interactions between the features. Each feature of the user interface performs its well-known and understood function. The only relationship between the features appears to be the positioning of one feature in front of another.
While it is clear that this is not a mere financial or business scheme, any effect, technical or otherwise, is purely due to what the skilled addressee elects to implement. The presence of results, which, it would seem, do not achieve the effects desired indicates that the claim may only extend aesthetic or “subjective” effects.
Based on all of the above considerations, I find that the substance of the claimed invention is related to an artistic or aesthetic effect applied to a component within a user interface. The result is potentially subjective and does not necessarily lead to an objective benefit or effect that would provide for patentability. Specifically, I find that the substance of the invention is a virtual control within a user interface whose background image has been blurred and emphasised in a manner in which the skilled addressee sees fit.
Consequently, I do not consider that the claimed invention is patentable.
Independent Claim 10 is substantially the same as Claim 1 for the purposes of manner of manufacture analysis. The same reasoning applies mutatis mutandis and the invention defined in Claim 10 therefore is not patentable.
Dependent Claims
Dependent Claims 2-4 and 11-13 add further general details to the application of the image manipulation effects. The additional details do not modify the interaction between the control element and the control region background. There remain no settings or details other than the general idea of where the blurred background is positioned, increasing the colour saturation of the blurred image by some unknown quantity, or adding a coloured layer to the control region background. These are therefore directed towards non-patentable subject matter for the same reasons as outlined for Claim 1 above.
Dependent Claims 5 and 14 add that the appearance of the background of the control region is modified when the first portion of the user interface is moved relative to the control region. These claims encompass cases where (i) the control region is translated/rotated on the screen and/or (ii) where the underlying first portion of the user interface is translated/rotated on the screen. In effect this is no different to repeating the process of claim 1 at different points in time and/or in a different position. The net result is still dependent upon specific settings implemented by the skilled addressee. In some cases it may have a technical effect, in others it will not. These claims are therefore directed towards non-patentable subject matter for the same reasons as outlined for Claim 1 above.
Claims 6 and 15, which are dependent upon claims 5 and 14 respectively, update the image of the control region at a slower rate than the user interface content when the user interface is translated relative to the control region. On first appearance the effect of putting this into practice might appear to result in a user interface which requires fewer computer resources to implement. That is, it might allow those resources to be used for other computing functions – as is emphasised in paragraph [00179] of the specification. However, as per the analysis above, the implementation of the claim is entirely at the discretion of the skilled addressee. There is no requirement that computer resources be conserved, merely that the appearance be updated at a lower rate. The calculation steps might very well continue anyway. That is, the implementation may be such that there is not necessarily any reduction in computer resources as compared to when there is no change in control appearance values.
For example, such an effect, rather than appearing to make the user interface more responsive, could also indicate the exact opposite. That is, a noticeable difference between the update rate of the blurred background image and the control image during a translation of the user interface image might give a user the impression that the device is not able to handle the functionality. This could inadvertently increase the cognitive burden exhibited by the user. Claims 6 and 15 are therefore not patentable.
As discussed above, claims 7 and 16 were amended in response to the Commissioner’s letter. Claim 7 now reads (claim 16 is amended mutatis mutandis):
“The computer program of claim 1, wherein the background of the control region does not display elements that are in one or more other layers that are between the control region and the user interface.”
This proposed amendment, as described by the Applicant, is intended to match the specification at paragraph [00151] and Figure 5H (displayed above). The effect of claim 7 is that where several controls are stacked, the control elements of control regions that lie below the uppermost control are hidden. When this is put in practice, specifically when two or more virtual controls overlap, there appears to be the possibility of a material advantage. When considered compared to standard virtual controls with opaque backgrounds, the user is able to maintain context in the application, while still being alerted to the presence of the front-most virtual control and potentially seeing any non-overlapping regions of ‘lower’ virtual controls. When compared to transparent or translucent controls, the portions of any controls which are located between the front-most control and the user interface are not displayed. This may enhance the user’s ability to maintain context within the application and be aware of which control is active or front-most.
However, it is quite clear that claim 7 still leaves it open for the skilled addressee to choose, however they please, the “blurring” and manner of emphasising colours. Thus, while the proposed to be amended Claims 7 and 16 appear to add an effect that may lead to a material advantage, these claims, as they stand, are not currently for a manner of manufacture because of the issues discussed with regard to claim 1.
Claims 8 and 17 were also amended as a result of the Commissioner’s letter. Claims 8 now reads (claim 17 is amended mutatis mutandis):
The computer program of claim 1, wherein displaying the control region further includes displaying content of the background of the control region to have an appearance of shifting backward on the display.
Page 4 of the Applicant’s Letter Response states that:
“The invention recited by dependent claim 8 (and dependent claim 17) may assist at least some users to observe and interact with the virtual control elements. For example, the apparent shifting backwards of the photograph may assist to draw focus to the control element that is displayed. At the same time, the user may maintain context in an application, due to the photograph still being displayed, with modifications as per the features of claim 1 that work together with the blurring animation to provide the combined effect on user focus and context. Again, this may be particularly beneficial for portable electronic devices that are often associated with small display screens.”
In combination with the blurring and colour emphasis, I cannot identify a technical interaction due to the appearance of moving the content of the background of the control region backward in the display. The description at paragraph [00153] briefly discloses that this is a defocussing effect – which is similar to a blur function. The method of implementation is likely to be highly relevant to whether or not this occurs. As per Claim 1, examples which both do and do not result in an improved user interface can be trivially constructed. Consequently, I consider that the claims do not define patentable subject matter for the same reasons as Claim 1.
Claims 9 and 18 similarly add further image manipulation effects to that of claims 8 and 17 respectively. Again, there are no details on how much to add, and the effect is subjective for the user. They are not patentable.
Conclusions
I have found that Claims 1-19 are not directed towards patentable subject matter. However, as identified above, there are instances where a specific combination of effects may lead to a technical effect that would provide for patentability. On this basis it is appropriate to grant the Applicant the opportunity to amend. In the circumstances, it is also seems desirable that novelty and inventive step be evaluated. Consequently, the application will be remitted for further examination. The Applicant should consider amending their application and respond to the findings in this decision during further examination of this application. Pursuant to Regulation 13.4(3) the Applicant has twelve months from the date of this decision to gain acceptance of this application.
Based on the above, it would seem that there are embodiments that fall withing the scope of the claims that may not achieve what appears to be the promised benefit of the specification. This immediately raises the prospect that utility should also be considered during further examination. However, I stress that I am making no finding at the present time in relation to whether or not the claims meet the requirements of utility of invention.
Dr W.E. Guinea
Delegate of the Commissioner of Patents
Annex 1: Claims as proposed to be amended
1. A computer program comprising instructions which, when executed by an electronic device with a display, cause the electronic device to:
display a user interface on the display;
while displaying the user interface on the display, receive a request to display a control region overlaid on at least a first portion of the user interface;
in response to the request, display a control region overlaid on the user interface, wherein displaying the control region includes:
displaying a background of the control region, wherein the background of the control region has an appearance that is based on a blurred version of the first portion of user interface with visual properties of the blurred version of the first portion of the user interface changed to emphasize the colors in the first portion of the user interface on which the control region is overlaid; and
displaying one or more control elements on the background of the control region;
while displaying the control region, detect an input that corresponds to activation of a respective control element of the one or more control elements; and,
in response to detecting the input that corresponds to activation of the respective control element, perform an operation that corresponds to the respective control element.
2. The computer program of claim 1, wherein the control region overlays the first portion of the user interface but not a second portion of the user interface distinct from the first portion of the user interface, and the second portion of the user interface is not blurred.
3. The computer program of claim 1, wherein the change in the visual properties of the blurred version of the first portion of the user interface includes increasing a saturation of the blurred version of the first portion of the user interface.
4. The computer program of claim 1, wherein the change in the visual properties of the blurred version of the first portion of the user interface includes introducing a colored layer that is based on the colors of the background content.
5. The computer program of claim 1, including instructions which, when executed by the electronic device with the display, cause the electronic device to change the appearance of the background of the control region as the first portion of the user interface moves relative to the control region.
6. The computer program of claim 5, wherein control-appearance values of the control region are updated at a slower rate than content moving behind the control is updated.
7. The computer program of claim 1, wherein the background of the control region does not display elements that are in one or more other layers that are between the control region and the user interface.
8. The computer program of claim 1, wherein displaying the control region further includes displaying content of the background of the control region to have an appearance of shifting backward on the display
9. The computer program of claim 8, wherein content of the background of the control region is extended towards an edge of the display by adding pixels at the edge of the content, wherein the added pixels are based on the content.
10. A method comprising:
at an electronic device with a display:
displaying a user interface on the display;
while displaying the user interface on the display, receiving a request to display a control region overlaid on at least a first portion of the user interface;
in response to the request, displaying a control region overlaid on the user interface, wherein displaying the control region includes:
displaying a background of the control region, wherein the background of the control region has an appearance that is based on a blurred version of the first portion of user interface with visual properties of the blurred version of the first portion of the user interface changed to emphasize the colors in the first portion of the user interface on which the control region is overlaid; and
displaying one or more control elements on the background of the control region;
while displaying the control region, detecting an input that corresponds to activation of a respective control element of the one or more control elements; and,
in response to detecting the input that corresponds to activation of the respective control element, performing an operation that corresponds to the respective control element.
11. The method of claim 10, wherein the control region overlays the first portion of the user interface but not a second portion of the user interface distinct from the first portion of the user interface, and the second portion of the user interface is not blurred.
12. The method of claim 10, wherein the change in the visual properties of the blurred version of the first portion of the user interface includes increasing a saturation of the blurred version of the first portion of the user interface.
13. The method of claim 10, wherein the change in the visual properties of the blurred version of the first portion of the user interface includes introducing a colored layer that is based on the colors of the background content.
14. The method of claim 10, including changing the appearance of the background of the control region as the first portion of the user interface moves relative to the control region.
15. The method of claim 14, wherein control-appearance values of the control region are updated at a slower rate than content moving behind the control is updated.
16. The method of claim 10, wherein the background of the control region does not display elements that are in one or more other layers that are between the control region and the user interface.
17. The method of claim 10, wherein displaying the control region further includes displaying content of the background of the control region to have an appearance of shifting backward on the display.
18. The method of claim 17, wherein content of the background of the control region is extended towards an edge of the display by adding pixels at the edge of the content, wherein the added pixels are based on the content.
19. An electronic device, comprising:
a display;
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods of claims 10-18.