Apple Inc.
[2024] APO 23
•12 June 2024
IP AUSTRALIA
AUSTRALIAN PATENT OFFICE
Apple Inc. [2024] APO 23
Patent Application: 2021200174
Title:Device, method, and graphical user interface for organizing and presenting a collection of media items
Patent Applicant: Apple Inc.
Delegate: Dr V. Z. Kolev
Decision Date: 12 June 2024
Hearing Date: Written submissions filed on 2 May 2023
Catchwords: PATENTS – sections 45 and 49 – hearing with respect to examiner’s objection – manner of manufacture – inventive step – electronic computing device with touch-sensitive surface (e.g., touch screen, touch pad) – multimedia item (e.g., image, video) management software running on the device – organising multimedia items in events and groups within hierarchy of granularity levels – different granularity levels corresponding to different timescales – presenting the organised multimedia items to users – user input via gestures (e.g., tap, pinch, depinch) on the touch-sensitive surface – logistical process – known devices – known user interface techniques – the claimed invention is not a manner of manufacture – no patentable subject matter described in the specification – inventive step not considered – application refused
Representation: Patent attorney for the applicant: FPA Patent Attorneys Pty Ltd
IP AUSTRALIA
AUSTRALIAN PATENT OFFICE
Patent Application: 2021200174
Title: Device, method, and graphical user interface for organizing and
presenting a collection of media itemsPatent Applicant: Apple Inc.
Date of Decision: 12 June 2024
DECISION
The invention claimed in each one of the claims, as proposed to be amended, is not a manner of manufacture. Furthermore, based on the matter described in the body of the specification, I am of the opinion that no allowable amendment could result in claiming a patentable invention. In light of this, I will not decide with respect to inventive step as this will be inconsequential.
I refuse the application.
REASONS FOR DECISION
Throughout this decision, unless explicitly stated otherwise, any reference to the Act, or to a specific section or subsection, refers to the Patents Act 1990, and any reference to the Regulations, or to a specific regulation or subregulation, refers to the Patents Regulations 1991. In addition, any reference to the Commissioner refers to the Commissioner of Patents as per the Act.
Background
The matter relates to patent application 2021200174 (the Application) in the name of Apple Inc. (the Applicant). The Application was filed on 13 January 2021 as a divisional application of 2019203932 (the Parent), which in turn is divisional of 2017203287 (the Grandparent), which is a divisional of 2014238116 (the Great Grandparent), which was filed under the Patent Cooperation Treaty as PCT/US2014/018908 and published as WO 2014/149488. The earliest claimed priority date is 15 March 2013.
It is worth noting that the Great Grandparent was accepted and subsequently granted following one examination report; the Grandparent lapsed for failure to gain acceptance within the prescribed period following three examination reports; and the Parent was accepted and subsequently granted following one examination report. None of these examination reports included an objection that the claimed invention is not a manner of manufacture.
The request for examination with respect to the Application was filed on 13 January 2021, and the Application was subject to four examination reports as detailed below.
Examination report No. 1 was issued on 12 January 2022. The report contained the sole objection that the claimed invention does not involve an inventive step. A response to the report was filed on 3 March 2022, together with proposed amendments to the description and claims.
Examination report No. 2 was issued on 30 March 2022. In the report, the objection with respect to inventive step was maintained. No other objections were raised. A response to the report was filed on 11 July 2022, together with proposed amendments to the claims.
Examination report No. 3 was issued on 8 August 2022. In the report, the objection with respect to inventive step was maintained. In addition, a new objection was raised, namely that the claimed invention is not a manner of manufacture. A response to the report was filed on 15 November 2022, together with proposed amendments to the description and claims.
Examination report No. 4 (the Last Report) was issued on 13 December 2022. In the report, both objections with respect to inventive step and manner of manufacture were maintained.
On 12 January 2023 (the final date for gaining acceptance), the Applicant requested a hearing, which was conducted by way of written submissions.
Submissions
Written submissions in relation to the hearing (the Applicant’s Submissions or AS) were filed on 2 May 2023.
Applicable law
On 15 April 2013, the Intellectual Property Laws Amendment (Raising the Bar) Act 2012 commenced which resulted in significant amendments to the Act and Regulations. The Application was filed on 13 January 2021; hence the amended provisions of the Act and Regulations apply to the examination of the Application and to the instant hearing decision. This means that I must accept the Application if I am satisfied, on the balance of probabilities, that the Application complies with the Act. If I am not so satisfied, I may refuse the Application. In such a situation, the extended period for gaining acceptance provided under reg 13.4(1)(g) will make no difference if the Application is refused. However, I consider that it is only appropriate to refuse the Application if I am satisfied that providing the Applicant with an opportunity to overcome any negative findings would serve no useful purpose; in other words, I will only refuse the Application if I consider that any potential negative findings are inevitably fatal to the Application.
The law on manner of manufacture
The relevant parts of s 18 stipulate:
“(1) Subject to subsection (2), an invention is a patentable invention for the purposes of a standard patent if the invention, so far as claimed in any claim:
(a) is a manner of manufacture within the meaning of section 6 of the Statute of Monopolies; and
…
(2) Human beings, and the biological processes for their generation, are not patentable inventions.”
A number of Court decisions have discussed and determined whether the particular invention in each case “is a manner of manufacture within the meaning of section 6 of the Statute of Monopolies”. These Court decisions constitute a significant body of case law that the Commissioner must follow. For the purpose of this decision, I do not consider it necessary to provide a comprehensive review of the case law and I will only point to Court decisions which I consider pertinent to the instant considerations.
National Research Development Corporation v Commissioner of Patents [1959] HCA 67; (1959) 102 CLR 252 (NRDC) is the seminal authority on the issue, establishing, inter alia, the general approach for deciding the question of manner of manufacture at [14] (at p 269):
“The right question is: ‘Is this a proper subject of letters patent according to the principles which have been developed for the application of s. 6 of the Statute of Monopolies?’”
D’Arcy v Myriad Genetics Inc [2015] HCA 35 (Myriad) is a much more recent High Court decision on manner of manufacture; however, I note that both NRDC and Myriad are not related to inventions implemented on computing devices, hence they do not discuss some of the more specific issues associated with such inventions. In contrast, the decisions of the Full Court of the Federal Court in Research Affiliates LLC v Commissioner of Patents [2014] FCAFC 150 (Research Affiliates) and Commissioner of Patents v RPL Central Pty Ltd [2015] FCAFC 177 (RPL Central) were made with respect to business methods implemented on computing devices. In particular, in RPL Central the Full Court, having considered in some detail the decisions in Research Affiliates and Myriad, developed an approach to determining the issue of manner of manufacture for inventions implemented using computers. This approach of RPL Central was later approved in Encompass Corporation Pty Ltd v InfoTrack Pty Ltd [2019] FCAFC 161 (Encompass), which was a decision by an enlarged bench (consisting of five judges) of the Full Court. The approach was also relied on in subsequent Full Court decisions, including Commissioner of Patents v Rokt Pte Ltd [2020] FCAFC 86 (Rokt) and Repipe Pty Ltd v Commissioner of Patents [2021] FCAFC 223 (Repipe). The cases in Encompass, Rokt, and Repipe also involved inventions implemented using computer technology.
Aristocrat Technologies Australia Pty Ltd v Commissioner of Patents [2022] HCA 29 is the most recent High Court decision on manner of manufacture. Because, in that decision, the six judges were evenly divided as to the outcome, there was no majority and no usual binding precedent. Importantly however, the outcomes of the above mentioned decisions of the Full Court were not criticised and remain undisturbed. Also, as a result of the operation of s 23(2)(a) of the Judiciary Act 1903, the decision of the Full Court of the Federal Court in Commissioner of Patents v Aristocrat Technologies Australia Pty Ltd [2021] FCAFC 202 that the invention concerned was not for a manner of manufacture was affirmed.
For the present purpose of deciding on manner of manufacture in relation to the Application, I consider it helpful to provide a brief overview of some of the main guiding principles that can be extracted from the above Court decisions. Below, I will present those principles, in italic, as separate paragraphs supported by relevant quotations from the decisions. Where the breadth of some quotations makes them relevant to more than one principle, for brevity, I will not repeat those quotations in different paragraphs (underlining added in all quotations).
A determination on manner of manufacture must be done on a case by case basis, having regard to the substance of the invention not merely the form of the claims; there are no strict rules or a precise formula to be applied mechanistically:
“The purpose of s. 6, it must be remembered, was to allow the use of the prerogative to encourage national development in a field which already, in 1623, was seen to be excitingly unpredictable. To attempt to place upon the idea the fetters of an exact verbal formula could never have been sound. It would be unsound to the point of folly to attempt to do so now, when science has made such advances that the concrete applications of the notion which were familiar in 1623 can be seen to provide only the more obvious, not to say the more primitive, illustrations of the broad sweep of the concept.” (NRDC at [15]; at p 271)
“This Court in NRDCdid not prescribe a well-defined pathway for the development of the concept of ‘manner of manufacture’ in its application to unimagined technologies with unimagined characteristics and implications. Rather, it authorised a case-by-case methodology.” (Myriad at [23])
“Whatever words have been used, the matter must be looked at as one of substance and effect must be given to the true nature of the claim.” (Myriad at [144], reasons by Gageler and Nettle JJ)
“The approach to be taken to deciding whether a claimed method or product is properly the subject of letters patent must be flexible and must allow for new technologies presently unknown … There is no formula to be mechanically applied …
… in examining whether a claimed invention is properly the subject of letters patent, it is necessary to look not only at the integers of that claimed invention but also at the substance of that invention.” (Research Affiliates at [116]-[117])
“The determination … is made not by some mechanistic application of the criterion of artificiality or physical effect, but by an understanding of the claimed invention itself. The invention is to be understood as a matter of substance and not merely as a matter of form.” (Research Affiliates at [106])
The characterisation as “an artificially created state of affairs of economic significance” is useful, but not ultimately determinative for manner of manufacture; to be patentable, a process must belong to the useful arts:
“The point is that a process, to fall within the limits of patentability which the context of the Statute of Monopolies has supplied, must be one that offers some advantage which is material, in the sense that the process belongs to a useful art as distinct from a fine art … that its value to the country is in the field of economic endeavour.” (NRDC at [22]; at p 275)
“… the view which we think is correct in the present case is that the method the subject of the relevant claims has as its end result an artificial effect falling squarely within the true concept of what must be produced by a process if it is to be held patentable … The effect produced by the appellant’s method exhibits the two essential qualities upon which ‘product’ and ‘vendible’ seem designed to insist. It is a ‘product’ because it consists in an artificially created state of affairs, discernible by observing over a period the growth of weeds and crops respectively on sown land on which the method has been put into practice. And the significance of the product is economic; for it provides a remarkable advantage, indeed to the lay mind a sensational advantage, for one of the most elemental activities by which man has served his material needs, the cultivation of the soil for the production of its fruits … It achieves a separate result, and the result possesses its own economic utility consisting in an important improvement in the conditions in which the crop is to grow, whereby it is afforded a better opportunity to flourish and yield a good harvest.” (NRDC at [25]; at p 277)
“If a process is to be patentable, it must offer some advantage which is material, in the sense that the process belongs to a useful art. The characterisation of patentability by reference only to the description in NRDC of a product which consists of an artificially created state of affairs of economic significance was part of the High Court’s reasoning but did not represent a sufficient or exhaustive statement of the circumstances in which a claimed invention is patentable.” (Research Affiliates at [101])
“In this context, their Honours emphasised [in Myriad] (at [20]) that, while satisfaction of an ‘artificially created state of affairs of economic significance’ as stated in NRDC may ‘suffice for a large number of cases in which there are no countervailing considerations’, this terminology is not to be treated as a formula exhaustive of the concept of manner of manufacture, or a formula which alone captures the breadth of the ideas to which effect must be given. Similarly, Gageler and Nettle JJ noted (at [125]) that the holding in NRDCdoes not mean that an ‘artificial state of affairs’ and ‘economic utility’ are the only relevant considerations in this context. However, the majority and Gageler and Nettle JJ acknowledged the usefulness of such characterisation in appropriate circumstances.” (RPL Central at [116])
In deciding on manner of manufacture, it is important to identify where the inventor’s ingenuity lies:
“It is a question of understanding what has been the work of, the output of, and the result of, human ingenuity, and to apply the principles that have been developed and explained so well in NRDC.” (Research Affiliates at [116])
“Recognising that the claims are to a method and system comprising a combination of integers, it is necessary to understand where the inventiveness or ingenuity is said to lie.” (RPL Central at [112])
Technological innovations are patentable, whereas business innovations are not; a business method implemented on computing devices could be patentable if it involves ingenuity in the computer implementation:
“Relevantly, the Full Court in Research Affiliates said (at [94]) that the distinction to be drawn was between the employment of an abstract idea or law of nature and the idea or law itself and that there is a distinction between a technological innovation which is patentable and a business innovation which is not. Their Honours repeated an observation from Grant (at [29]) ‘[a] product of a method is something in which a new and useful effect may be observed. For claimed computer programs, the courts looked to the application of the program to produce a practical and useful result, so that more than “intellectual information” was involved.’ A technological innovation is patentable; a business innovation is not, although a business method may be the subject of letters patent.” (RPL Central at [100])
“A claimed invention must be examined to ascertain whether it is in substance a scheme or plan or whether it can broadly be described as an improvement in computer technology. The basis for the analysis starts with the fact that a business method, or mere scheme, is not, per se, patentable. The fact that it is a scheme or business method does not exclude it from properly being the subject of letters patent, but it must be more than that. There must be more than an abstract idea; it must involve the creation of an artificial state of affairs where the computer is integral to the invention, rather than a mere tool in which the invention is performed. Where the claimed invention is to a computerised business method, the invention must lie in that computerisation. It is not a patentable invention simply to ‘put’ a business method ‘into’ a computer to implement the business method using the computer for its well-known and understood functions.” (RPL Central at [96])
“RPL Central does not claim any invention or ingenuity in any program or operation of a computer, or implementation by a computer to operate the method. Accordingly, the ingenuity of the inventors must be in the steps of the method itself. The method does utilise the speed and processing power and ability of a computer but there is no suggestion that this is other than a standard operation of generic computers with generic software to implement a business method …
The problem may be one of confronting the ‘maze’ of available information concerning the RPL of different Units of Competency in different institutions, but the solution to that problem, to be patentable, must involve more than the utilisation of the well-known search and processing functions of a computer, for example an invention in the way in which the computer is utilised.” (RPL Central at [110]-[111])
“There is no suggestion in the specification or the claims that any part of the inventive step lies in the computer implementation. Rather, it is apparent that the scheme is merely implemented in a computer and a standard computer at that. It is no part of the claimed method that there is an improvement in what might broadly be called ‘computer technology’.” (Research Affiliates at [118])
“In Research Affiliates, looking at the claimed method as a matter of substance, the Full Court concluded that, in that case, the computer was merely the means by which the analyst accessed data to generate an index, that being the work of the analyst rather than a technical generation by the computer. There was no suggestion of the utilisation of an unusual technical effect. The inventors made changes to the computer program to cause the program to gather and process data and perform data manipulations and calculations, including macros to manipulate and refine data. Such use of algorithms was not ‘foreign’ to the normal use of computers.” (RPL Central at [102])
“To reiterate some of the matters discussed in Research Affiliates:
·It is necessary to ascertain whether the contribution to the claimed invention is technical in nature. In Aerotel Ltd v Telco Holdings Ltd; Macrossan’s Application [2007] 1 All ER 225, the subject matter was an interactive system whereby questions were asked, the answers incorporated in a draft and, depending on some particular answers, further questions were asked. It was held that, apart from the fact of running a computer program, there was nothing technical about the contribution and the method was for the business of advising upon and creating appropriate company documents.
·One consideration is whether the invention solves a ‘technical’ problem within the computer or outside the computer, or whether it results in an improvement in the functioning of the computer, irrespective of the data being processed.
·Does the claimed method merely require generic computer implementation?
·Is the computer merely the intermediary, configured to carry out the method using a computer readable medium containing program code for performing the method, but adding nothing to the substance of the idea? In Alice Corporation, the method was for exchanging financial obligations in which the computer was used to create records, track multiple transactions and issue simultaneous instructions. The majority in the Supreme Court of the United States concluded that the use of the computer added nothing to the substance of the abstract idea of reducing settlement risk in exchanging financial obligations.” (RPL Central at [99])
Whether a method can or cannot be practically implemented without a computer is not determinative for manner of manufacture:
“Simply putting a business method or scheme into a computer is not patentable unless there is an invention in the way in which the computer carries out the scheme or method. Is the fact that the scheme cannot practically be implemented without a computer, that is, that the computer is integral to the working of the scheme, sufficient to make it patentable? The answer is not straightforward because this is not a case where the computer simply processes the information entered by the user, for example by using an algorithm, or retrieves information from the Internet in response to a user’s question.” (RPL Central at [107])
“Is the mere implementation of an abstract idea in a well-known machine sufficient to render patentable subject matter? Is the artificial effect that arises, because information is stored in RAM and there is communication over the Internet or wifi, sufficient? Does any physical effect give rise to a manner of manufacture? Are the mere presence of an artificial effect and economic utility, without more, sufficient to determine manner of manufacture?
It is not a question of stating precise guidelines but of deciding, in each case, whether the claimed invention, as a matter of substance not form, is properly the subject of a patent.” (RPL Central at [97]-[98])
“It is stated in the specification, and was accepted by the primary Judge, that the method could not be carried out without the use of a computer. This alone cannot render the claimed invention patentable if it involves simply the speed of processing and the creation of information for which computers are routinely used. In those circumstances, the claimed invention is still to the business method itself. A computer-implemented business method can be patentable where the invention lies in the way in which the method is carried out in the computer. This necessitates some ingenuity in the way in which the computer is utilised (Research Affiliates).” (RPL Central at [104])
The case law on manner of manufacture was extensively discussed in prior decisions on the matter by the Australian Patent Office, including in Aristocrat Technologies Australia Pty Limited [2016] APO 49 (Aristocrat’16), where the Delegate provided a general indication, distilled from the case law, as to the matters relevant to a determination on manner of manufacture for inventions implemented on a computer:
“35. I conclude that it is relevant to consider a range of matters. Without seeking to be exhaustive, these include:
· there must be more than an abstract idea, mere scheme or mere intellectual information;
· is the contribution of the claimed invention technical in nature;
· does the invention solve a technical problem within the computer or outside the computer;
· does the invention result in improvement in the functioning of the computer, irrespective of the data being processed;
· does the application of the method produce a practical and useful result;
· can it be broadly described as an improvement in computer technology;
· does the method merely require generic computer implementation;
· is the computer merely an intermediary or tool for performing the method while adding nothing of substance to the idea;
· is there ingenuity in the way in which the computer is utilised;
· does the invention involve steps that are foreign to the normal use of computers; and
· does the invention lie in the generation, presentation or arrangement of intellectual information.” (underlining added)
It is clear from the Delegate’s wording that the above list is not a “check-list” to determine whether a particular claimed invention is a manner of manufacture. Instead, the list includes, in a non-exhaustive manner, relevant considerations or “touch stones” helpful in the decision process. This was also noted in Todd Martin [2017] APO 33:
“47. With this list of points, the delegate in the Aristocrat[’16] case was not intending to indicate a list of conditions, for computer-implemented cases, to be met to define a manner of manufacture. That is evident from the delegate’s statement that the list was not intended to be exhaustive. Moreover, it would appear improper to find there was a manner of manufacture simply on the basis that one or more points could be answered in favour. In the present case for example, it may be that at least the fifth dot-point, regarding whether the application of the method produces a practical and useful result, is satisfied. On the other hand, that consideration on its own would be insufficient in the present case. Conversely it would also appear improper to find there was no manner of manufacture simply on the basis that one or more points could not be answered favourably. Rather than these points being seen as conditions to be met, they should be seen as relevant matters to consider, as the delegate stated at [35]. The substance and contribution of the claimed invention in each case should be considered on its merits overall and various points under the law would appear to have varying degrees of relevance depending on the case.”
As a further observation, it appears that the Court decisions on manner of manufacture for computer-implemented inventions do not always explicitly refer to all of the above relevant considerations, which would appear to suggest that, depending on the circumstances of the case, not all of the relevant matters included in the list need to be considered in order to reach a conclusion. It could also be said that some of the above considerations are somewhat overlapping in their scope.
The Specification
The proposed amendments
While I cannot see any immediate reasons why the amendments would not be allowable, I do not need to decide this conclusively as this will not affect the outcome of my decision. Suffice to say that the amendments were not objected to during examination, and for the benefit of the Applicant, I will consider the Specification as most recently proposed to be amended on 15 November 2022.
The invention as described
The body of the Specification includes 69 pages of description and 36 pages of drawings. With respect to the technical field of the invention, it is stated that:
“[0001a] This relates generally to electronic devices with touch-sensitive surfaces, including but not limited to electronic devices with touch-sensitive surfaces that organize a collection of media items.” (underlining added)
The media items, in the context of the Specification, are exemplified by photos and videos. The section “Background” acknowledges that:
“[0002] The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has increased significantly in recent years. Exemplary touch-sensitive surfaces include touch pads and touch screen displays. Such surfaces are widely used to manipulate user interface objects on a display.
[0003] Exemplary manipulations include adjusting the position and/or size of one or more images or viewing images that are associated with an album of images. Exemplary user interface objects include digital images, video, text, icons, control elements such as buttons and other graphics. A user will, in some circumstances, need to perform such manipulations on user interface objects in a file management program (e.g., Finder from Apple Inc. of Cupertino, California), an image management application (e.g., Aperture or iPhoto from Apple Inc. of Cupertino, California) …” (underlining added)
However, the description notes:
“[0004] But methods for performing these manipulations are cumbersome and inefficient. For example, using a sequence of mouse based inputs to select one or more user interface objects and perform one or more actions on the selected user interface objects is tedious and creates a significant cognitive burden on a user. In addition, these methods take longer than necessary, thereby wasting energy. This latter consideration is particularly important in battery-operated devices.” (underlining added)
The next section, “Summary”, begins by outlining the objectives of the present invention:
“[0005] Accordingly, there is a need for electronic devices with faster, more efficient methods and interfaces for organizing and presenting a collection of media items. Such methods and interfaces optionally complement or replace conventional methods for organizing and presenting a collection of media items. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated devices, such methods and interfaces conserve power and increase the time between battery charges.” (underlining added)
In light of paragraphs [0001a], [0002], and [0003] quoted above, which appear to be directed explicitly towards the use of touch-sensitive surfaces, I find it somewhat curious, that the only example given for “cumbersome and inefficient” performing of manipulations is related to “mouse based inputs”, which do not normally involve the use of such surfaces. In any event, it is notable that, despite touch based inputs being “widely used”, the entire Specification does not provide any example of such input that is inefficient, and with respect to which, the present invention “produce[s] a more efficient human-machine interface”.
The electronic device can be a desktop computer, a notebook computer, a tablet computer, or a handheld device (at [0006]). “In some embodiments”, the device has a touch pad or a touch screen as well as “a graphical user interface (GUI), one or more processors, memory and one or more modules, programs or sets of instructions stored in the memory for performing multiple functions” (at [0006]). The remainder of the section includes summarised explanations spread over several paragraphs, some of which resembling consistory statements broadly corresponding to some the claims.
A brief description of the drawings, consisting of Figure 1A to Figure 9, is followed by a section titled “Description of Embodiments”. In this relatively short section, it is explained that “[m]any electronic devices have graphical user interfaces that organize a plurality of images. For example, the plurality of images is organized according to the capture dates of the respective images” (at [0025], underlining added). I have my doubts that a software application that organises a plurality of images would necessarily be considered a user interface, and I will provide more discussion on this topic later in the decision.
The present invention includes “obtaining a collection of media items ordered in a chronological order in accordance with a time indicator associated with each of the media items” and “determin[ing] one or more patterns in the collection of media items based at least in part on a comparison between properties (e.g., metadata including time, date, location, altitude, speed, network connections, weather, biometrics, facial recognition, etc.) of sequentially adjacent media items in the collection of media items and divid[ing] the collection of media items into multiple events that includes a plurality of multi-item events” (at [0025], underlining added). Somewhat more related to the presently claimed invention appears to be the explanation that:
“[0026] In some embodiments, the device obtains a collection of media items organized into a hierarchy of granularity levels that includes one or more detailed-granularity levels within one or more overview-granularity levels. The device receives a receiving [sic] a first request to display a first portion of the collection organized in accordance with a first detailed-granularity level and displays a first plurality of representations of the media items in response to the first request. The device also receives a second request to display a second portion of the collection organized in accordance with a respective overview-granularity level that corresponds to multiple detailed-granularity levels and displays a second plurality of representations of the media items in response to the second request.” (underlining added)
The next section is titled “Exemplary devices”. In the beginning of the section, it is made clear that “[i]n the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments” (at [0028], underlining added).
A substantial part of the section explains in some detail Figure 1A (reproduced below), which is “a block diagram illustrating a portable multifunction device with a touch-sensitive display in accordance with some embodiments” (at [0014]). It may be worth noting that I have made some adjustments to all reproduced drawings in order to improve their readability. In my opinion, Figure 1A is somewhat self-explanatory as it represents a generalised diagram of known portable electronic devices such as, e.g., smart phones, having a variety of software modules, for which the description notes that “[t]hese modules (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules are, optionally, combined or otherwise re-arranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 102 optionally stores additional modules and data structures not described above” (at [0087], underlining and italic added). This, in combination with the above quoted statement from paragraph [0028], makes it unnecessary for me to discuss the explanations with respect to Figure 1A provided at paragraphs [0036]-[0089]. I note that, spread throughout these explanations, there are definitions for some terms as used in the Specification, including “intensity” (at [0037]), “tactile output” (at [0038]), “graphics” (at [0063]), and “instant messaging” (at [0074]). However, these definitions are not of significant importance for my decision.
Figure 1B (reproduced below) is “a block diagram illustrating exemplary components for event handling in accordance with some embodiments” (at [0090]). According to the description:
“[0091] Event sorter 170 receives event information and determines the application 136-1 and application view 191 of application 136-1 to which to deliver the event information. Event sorter 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates the current application view(s) displayed on touch sensitive display 112 when the application is active or executing. In some embodiments, device/global internal state 157 is used by event sorter 170 to determine which application(s) is (are) currently active, and application internal state 192 is used by event sorter 170 to determine application views 191 to which to deliver event information.
…
[0093] Event monitor 171 receives event information from peripherals interface 118. Event information includes information about a sub-event (e.g., a user touch on touch-sensitive display 112, as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I/O subsystem 106 or a sensor, such as proximity sensor 166, accelerometer(s) 168, and/or microphone 113 (through audio circuitry 110). Information that peripherals interface 118 receives from I/O subsystem 106 includes information from touch sensitive-display 112 or a touch-sensitive surface.
…
[0097] Another aspect of the user interface associated with an application is a set of views, sometimes herein called application views or user interface windows, in which information is displayed and touch-based gestures occur. …” (underlining added)
In my opinion, the broad non-specific manner, in which the explanations concerning Figure 1B are worded, strongly suggests that what is described represents the usual way, in which known portable electronic devices could handle user input. Alternative design possibilities are often introduced by expressions like “[i]n some embodiments”, “[i]n other embodiments” (e.g., in [0094]), etc. As a mere example, it is stated that “[i]n some embodiments, operating system 126 includes event sorter 170. Alternatively, application 136-1 includes event sorter 170. In yet other embodiments, event sorter 170 is a stand-alone module, or a part of another module stored in memory 102, such as contact/motion module 130” (at [00101]).
Figure 2 “illustrates [schematically] a portable multifunction device 100 having a touch screen 112 in accordance with some embodiments. The touch screen optionally displays one or more graphics within user interface (UI)”, wherein “a user is enabled to select one or more of the graphics by making a gesture on the graphics, for example, with one or more fingers … or one or more styluses” (at [00115]). Figure 3 is “a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface in accordance with some embodiments”, wherein the device “need not be portable” (at [00118]). I am of the view that Figure 2 and Figure 3 as well as the associated explanations in the description in paragraphs [00115]-[00120] do not introduce any matter that goes beyond the general structure and functioning of known electronic devices (portable or not) such as smart phones, tablets, laptops, desktop computers, or similar devices with computing functionality. The non-specific nature of the description is reaffirmed by the fact that:
“[00119] Each of the above identified elements in Figure 3 are, optionally, stored in one or more of the previously mentioned memory devices. Each of the above identified modules corresponds to a set of instructions for performing a function described above. The above identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules are, optionally, combined or otherwise re-arranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.” (underlining and italic added)
In the interest of brevity, I have not reproduced Figure 2 and Figure 3, and I do not consider it necessary to provide any further discussions with respect to these drawings.
Figure 4A (reproduced below) “illustrates an exemplary user interface [400] for a menu of applications on portable multifunction device 100 in accordance with some embodiments” (at [00121]). Importantly, “[i]n some embodiments, user interface 400 includes the following elements, or a subset or superset thereof: …” (at [00121], underlining and italic added), which indicates that the illustrated interface elements are mere examples of what may generally appear on a user interface of this type. In contrast to the user interface of Figure 4A, which operates using a touch screen (see item 112), Figure 4B (also reproduced below) “illustrates an exemplary user interface on a device … with a touch-sensitive surface 451 … that is separate from the display 450 (e.g., touch screen display 112)” (at [00123]). A possible example for such a device that comes to mind is a laptop having a touch pad separate from the (touch) screen. Items 460 and 462 represent contacts with the touch-sensitive surface which correspond to locations 468 and 470, respectively, on the screen. As is normally the case with such devices, “[i]n this way, user inputs (e.g., contacts 460 and 462, and movements thereof) detected by the device on the touch-sensitive surface (e.g., 451 in Figure 4B) are used by the device to manipulate the user interface on the display (e.g., 450 in Figure 4B) of the multifunction device when the touch-sensitive surface is separate from the display” (at [00124]). It is also explained (at [00125]) that while the examples are primarily related to user input using fingers, the description should not be considered as so limited and finger inputs may, for example, be replaced with a device such as a mouse or a stylus.
In the context of user interfaces, the term “focus selector” is defined and further explained by the Specification as follows:
“[00126] As used herein, the term ‘focus selector’ refers to an input element that indicates a current part of a user interface with which a user is interacting. In some implementations that include a cursor or other location marker, the cursor acts as a ‘focus selector,’ so that when an input (e.g., a press input) is detected on a touch-sensitive surface … while the cursor is over a particular user interface element (e.g., a button, window, slider or other user interface element), the particular user interface element is adjusted in accordance with the detected input. In some implementations that include a touch-screen display … that enables direct interaction with user interface elements on the touch-screen display, a detected contact on the touch-screen acts as a ‘focus selector,’ so that when an input (e.g., a press input by the contact) is detected on the touch-screen display at a location of a particular user interface element (e.g., a button, window, slider or other user interface element), the particular user interface element is adjusted in accordance with the detected input. …” (underlining added)
It appears that the section “Exemplary devices”, as a whole, presents no more than a generalised discussion of known electronic computing devices, to which the present invention pertains. The next section is titled “User interfaces and associated processes”. This is the last section of the description, and it directs attention “towards embodiments of user interfaces (‘UI’) and associated processes that may be implemented on an electronic device with a display and a touch-sensitive surface … or portable multifunction device” (at [00127]).
Figures 5A-S “illustrate exemplary user interfaces for organizing and presenting a collection of media items in accordance with some embodiments” (at [00128]). I consider that these drawings are important for understanding the working of the invention, as they schematically illustrate what is displayed on the touch screen of a portable multifunction device and how this changes as a result of the user interaction with the device. For this reason, I have reproduced and discussed all of Figures 5A-S. My discussion is based on the description at [00129]-[00149].
Figure 5A shows the events view (see item 503), which includes a plurality of representations 510a-d of media items (e.g., photos or videos), organised in selectable multi-item events 502a-d. Each event has associated event title 506a-d and event location(s) 508a-d. The media items that have their representations displayed are part of a collection of media items. It is explained that “[i]n some embodiments, the device automatically divides the collection of media items into multi-item events by determining one or more patterns in the collection of media items and identifying transitions points in the collection of the media items, where a transition point corresponds to a change in a pattern of the collection of media items. In some embodiments, the transition point is based on a change in metadata (e.g., time, date, location, altitude, speed, network connections, weather, biometrics, facial recognition, etc.) between sequentially adjacent media items” (at [00129], underlining added). In addition, “[i]n some embodiments, the device is configured to automatically populate an event title and event location for a respective event in accordance with metadata associated with media items within the respective event. … In some embodiments, the event locations are truncated (e.g., event locations 508d), whereby less than all of the event locations associated with the event are displayed” (at [00130]). It is further explained that Figure 5A “illustrates displaying the one or more representations of media items within the events at a detailed-granularity level. In some embodiments, a respective representation of a media item includes a thumbnail associated with a photo or a still or a gif image associated with a video” (at [00130], underlining added). A months affordance 512 is displayed to the left from the view title (i.e., “events” 503), and can be used to switch to months view (see Figure 5H) when the user performs a tap gesture on the affordance. Figure 5A also shows a schematical representation of a swipe or drag gesture performed on the touch screen by a user. The gesture consists of the movement of contact 514 from position 514a to position 514b.
Figure 5B illustrates the changes to what is displayed as a result of the swipe or drag gesture. It could be seen that, because of the scrolling up of the events in the events view, event 502a is no longer visible on the screen and a new event 502e has appeared. Figure 5B also shows a tap gesture 516. It is worth noting that “[i]n some embodiments, a tap gesture on a representation of a media item causes the device to display an enlarged representation of the media item, while a tap gesture at a location corresponding to a header of an event (e.g., the white space above the representations 510b) causes a representation of the event (or one or more sub-events) to be displayed” (at [00132]).
Figure 5C shows such a representation of an event (or one or more sub-events), in which only the representations of the media items belonging to the Hawaii Vacation event 502b are displayed. The representations are divided (in some embodiments automatically, based, e.g., on the metadata) into three sub-events 518a-c within the event 502b. The different sub-events represent media items captured at different locations 507a-c (acting as titles for the sub-events), and on different dates 509a-c. It can also be seen that the event title “Hawaii Vacation” 502b is replacing the view title “events” 503 seen on Figure 5B. Importantly, the months affordance 512 that can be seen on Figure 5B is now replaced with an events affordance 504. While not explicitly mentioned, it could be reasonably implied that the events affordance can be used to return to the events view, in the same way as the months affordance of Figure 5B can be used to switch to months view. Figure 5C also shows a drag gesture consisting of moving the contact 520 from location 520a to location 520b.
Figure 5D illustrates the moving of media item representation 522 from sub-event 507c to sub-event 507b as a result of the drag gesture shown on Figure 5C. In addition, Figure 5D shows a press and hold gesture with contact 524 over sub-event location or title 507b. The description explains that “[i]n some embodiments, in a response to a tap gesture (e.g., a contact detected for less than a predefined time threshold such as 0.05, 0.1, 0.2, 0.5 or 1.0 seconds or some other reasonable amount of time), the device displays an expanded representation of the sub-event, and in response to detecting a press and hold gesture (e.g., a contact detected for more than a predefined time threshold such as 0.05, 0.1, 0.2, 0.5 or 1.0 seconds or some other reasonable amount of time), the device displays a user interface for editing the title of the sub-event” (at [00133], underlining and italic added).
Figure 5E shows the edit sub-event details pop-up window 525 appearing as a result of the press and hold gesture on Figure 5D. It is explained that “[i]n some embodiments, the device is configured to automatically populate a sub-event title and sub-event date(s) for a respective sub-event in accordance with metadata associated with the items within the event or sub-event. … In some embodiments, a user is enabled to edit the sub-event title field 526 [as illustrated in Figure 5E] and the sub-event date(s) field 528 with a text [item 530] or speech input” (at [00134]).
Figure 5F shows the new sub-event title 507b appearing as a result of the text input 530 of Figure 5E. In addition, Figure 5F illustrates a pinch gesture corresponding to the movement of contacts 532 and 534 towards each other.
Figure 5G shows switching back to events view (see Figure 5A) as a result of the pinch gesture of Figure 5F. Figure 5G also shows a pinch gesture corresponding to movement of contacts 536 and 538 towards each other. Importantly:
“[00136] … In some embodiments, the device is enabled to switch between views (e.g., event view 503, month view 513 (Figure 5H), seasons view 551 (Figure 5I), and year view 561 (Figure 5J)) by detecting a pinch or depinch gesture (e.g., a pinch gesture corresponding to movement of contacts 532 and 534 in Figure 5F or a depinch gesture corresponding to movement of contacts 582 and 584 in Figure 5O) or, optionally, by detecting a horizontal swipe gesture with three or more contacts. In some embodiments, the device is similarly enabled to switch between views by detecting a tap gesture over on an affordance corresponding to a respective view (e.g., events affordance 504 in Figure 5F, months affordance 512 in Figure 5G, seasons affordance 550 in Figure 5H, and years affordance 560 in Figure 5I). …” (underlining added)
Hence, the pinch and depinch gestures are just one of the possibilities to switch between different views using the touch screen.
Figure 5H illustrates months view (see item 513), appearing as a result of the pinch gesture of Figure 5G. The months view includes a plurality of selectable multi-item months 540a-d, having respective months titles 542a-d, location(s) indicators 544a-d, and pluralities of representations of media items 545a-d corresponding to the media items acquired within the respective month. As explained above, switching to seasons view (see Figure 5I) and to events view (see Figure 5G) could be done by using the respective affordances 550 and 504 located on both sides of the view title 513. Figure 5H also shows a pinch gesture corresponding to the movement of contacts 546 and 548 towards each other.
Figure 5I illustrates seasons view (see item 551), appearing as a result of the pinch gesture of Figure 5H. The seasons view includes a plurality of selectable multi-item seasons 552a-d, having respective season titles 553a-d, location(s) indicators 554a-d, and pluralities of representations of media items 555a-d corresponding to the media items acquired within the respective season. In an analogous fashion, switching to years view (Figure 5J) and to months view (Figure 5H) could be done by using the respective affordances 560 and 512 located on both sides of the view title 551. In addition, Figure 5I again shows a pinch gesture corresponding to the movement of contacts 556 and 558 towards each other.
Figure 5J illustrates years view (see item 561), appearing as a result of the pinch gesture of Figure 5I. The years view includes a plurality of years 562a-c, having respective location(s) indicators 564a-b, and pluralities of representations of media items 566a-c corresponding to the media items acquired within the respective year. The representations of media items are displayed at an overview-granularity level (e.g., they appear relatively smaller and more numerous on Figure 5J), in contrast with the other views discussed above, which display the representations of media items at a detailed-granularity level. Switching to seasons view (Figure 5I) could be done by using the affordance 550, however an affordance for switching to a view associated with a longer than a year period of time is not provided, which may suggest that the years view is the top level of the hierarchy. Figure 5J also shows a tap gesture having a contact 568 in a location associated with one of the representations of media items.
Figure 5K illustrates an identical state of the touch screen following the tap gesture of Figure 5J, hence, it could be inferred that this tap gesture has no effect.
Figure 5L demonstrates that while a tap gesture may have no effect, the same is not true for a press and hold gesture. Such a gesture having a contact 570 at location 570a, corresponding a specific representation of a media item, results in ceasing to display that specific representation and displaying an enlarged representation (or a preview) 572 of the media item. As illustrated, “[i]n some embodiments, in the year view, while performing a press and hold gesture, the device selects a representation of a media item that is slightly offset from the contact (e.g., a focus selector corresponding to the contact is slightly offset from the contact), so that the user can see a location of the representation of the media item in the array of representations of media items that is currently being selected/enlarged (e.g., so that the location is not obscured by the user’s finger)” (at [00141]).
Figure 5M illustrates that when a contact 570 is detected at a further location 570b, corresponding to a further specific representation of a media item, this results in ceasing to display the further specific representation and displaying an enlarged representation (or a preview) 574 of the further media item in the same way as this was done with respect to the previous media item as illustrated on Figure 5L.
Figure 5N illustrates a photo view 578, where the media item 576 is displayed in full-size in response to detecting a liftoff of contact 570 from position 570a (see Figure 5L). In addition, a multi-item event 502n (with its title 506n and location 508n) associated with the respective media item 576 is displayed together with an indicator 580 showing the sequential number of media item 576 within the multi-item event 502n. Figure 5N also shows “a years affordance 560 which enables the device to display the years view 561 (described in more detail above with reference to Figures 5J-5L) in response to a tap gesture on the affordance 560 and an events affordance 504 which enables the device to display the events view 503 (described in more detail above with reference to Figures 5A-5B and 5G) in response to a tap gesture on the affordance 504” (at [00143]).
Figure 5O illustrates a depinch gesture corresponding to movement of contacts 582 and 584 away from each other in the part of the touch screen representing year 2012, when the device is in years view as, e.g., shown on Figure 5K.
Figure 5P displays a seasons view 551, in response to the depinch gesture of Figure 5O. Figure 5P (the reference to Figure 5N at [00145] appears to be in error) also shows a depinch gesture corresponding to movement of contacts 586 and 588 away from each other.
Figure 5Q displays months view 513, in response to the depinch gesture of Figure 5P (the reference to Figure 5N at [00146] appears again to be in error). Figure 5Q also illustrates a tap gesture with contact 590 over an affordance 591c associated with the selectable multi-item month group 540c, corresponding to January 2013. It is further explained that “[i]n some embodiments, only the affordance associated with a grouping (e.g., affordances 591a, 591b, 591c, 591d) is selectable. In some other embodiments, the entire description box or area associated with a grouping is selectable. For example, in some embodiments, the entire area within description box 589 corresponding to month 540c is selectable” (at [00146]).
Figure 5R displays a view corresponding to January 2013 (see item 540c), in response to the tap gesture of Figure 5Q. Two events, 502c and 502f, occurring in the month of January 2013 are also displayed, including the respective representations of media items 510c and 510f. In addition, Figure 5R shows a tap gesture with contact 594 over one of the representations 596.
Figure 5S illustrates, in response to the tap gesture of Figure 5R, a photo view 578 displaying in full-size the media item 598 corresponding to the representation 596 of Figure 5R. This is done in a way very similar to that of Figure 5N.
The part of the description dealing with Figures 5A-S concludes with the explanation:
“[00149] Thus, in some embodiments, an event that spans two months or two years is displayed in event groups for both months and/or both years (e.g., the New Year’s Eve Party event is grouped with both the December events and the January events). In some embodiments, even if an event is grouped with multiple different events groups, the images from the event are not repeated in the year view (e.g., the view shown in Figure 5K). For example, the last row of representations in 2012 in Figure 5K would include representations that correspond to images from December 31 at the New Year’s Eve Party, while the first row of representations of 2013 would include representations that correspond to images from January 1 at the New Year’s Eve Party.”
The above discussion of the user interface is followed by the explanations with respect to Figures 6A-E, which “are flow diagrams illustrating a method 600 of dividing a collection of media items into a plurality of multi-item events in accordance with some embodiments”, wherein “[t]he method 600 is performed at an electronic device (e.g., device 300, Figure 3, or portable multifunction device 100, Figure 1A) with a display and a touch-sensitive surface. In some embodiments, the display is a touch screen display and the touch-sensitive surface is on the display. In some embodiments, the display is separate from the touch-sensitive surface” (at [00150]). Importantly however, “[s]ome operations in method 600 are, optionally, combined and/or the order of some operations is, optionally, changed” (at [00150]).
I have reproduced Figures 6A-E below and, for additional clarity, I have added black frames and grey frames around key processes in the method 600. Figures 6A-C illustrate the process of obtaining a collection of media items and dividing the collection into multi-item events. Understandably, item 606, representing the process of dividing the collection into events, extends over substantial part of Figure 6A and the entirety of Figures 6B-C. Item 622 relates to the ways to determine the change in pattern of data corresponding to media items, as part of the process of item 606. This is an important process, and item 622 occupies (as part of item 606) a substantial portion of Figure 6B and, in essence, the entirety of Figure 6C. Figure 6D illustrates the process of grouping consecutive (in time) multi-item events into pattern-based event groups and date-based event groups. Figure 6E illustrates the process of displaying (in response to a corresponding request) a portion of the collection of media items organised in events as well as moving a media item (by moving its representation on the screen) from one event to another, as discussed with respect to Figures 5A-S.
The description states that:
“[00151] As described below, the method 600 provides an intuitive way to divide a collection of media items into a plurality of multi-item events [such as those discussed with respect to Figures 5A-S above]. The method reduces the cognitive burden on a user when dividing a collection of media items into a plurality of multi-item events, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to divide a collection of media items into a plurality of multi-item events faster and more efficiently conserves power and increases the time between battery charges.
[00152] The device obtains (602) a collection of media items (e.g., a ‘library’ of images and/or videos), where the media items are ordered in a chronological order in accordance with a time indicator associated with each of the media items.
[00153] The device determines (604) one or more patterns in the collection of media items based at least in part on a comparison between properties of sequentially adjacent media items in the collection of media items (e.g., in the chronological order). Figure 5A, for example, shows the device displaying a plurality of respective events (e.g., 502a, 502b, 502c, 502d) related to corresponding groups of media items in a collection of sequentially adjacent media items. The device divides the collection of media items (or a portion of the collection) into groups (or respective events 502) by identifying one or more patterns based on a comparison between properties of sequentially adjacent media items.
[00154] … dividing the collection of media items into multiple events [is done] automatically without user intervention …” (underlining added)
I agree that the described method 600 appears intuitive, and furthermore, the drawings include a considerable amount of wording to facilitate its understanding. The detailed explanations of method 600, provided at [00152]-[00178], do not offer any specifics about the computer implementation. The explanations in question do not go beyond the discussion of the purely logical steps of the method as illustrated on Figures 6A-E, merely further explaining these steps, and providing some examples to clarify the broad language used in the drawings. I do not consider it necessary to discuss these explanations at length, instead I will concentrate on what appears to be the main points worth mentioning:
·the events are non-overlapping since their corresponding event time ranges are non-overlapping (i.e., one event ends before the other begins) or, in other words, the sequence of media items is divided at event transition points such that no media item is shared between two events (at [00155]);
·a date-transition boundary could be an end of day boundary such as midnight, an end of week boundary such as midnight on Saturday, an end of month boundary such as midnight on the last day of the month, and an end of year boundary such as midnight on 31 December (at [00156]);
·date-transition boundaries are not an absolute indicator of event boundaries as events could, e.g., begin in the evening and end in the next morning (at [00156]);
·the difference between individuals (see item 638, Figure 6C) could be ascertained using face recognition, metadata tags, or network connection identifiers (at [00158]);
·the device could group (see item 650, Figure 6D) a set of consecutive events into date-based event groups (e.g., grouping the events by month, season, and/or year as shown on Figures 5A-S) (at [00170]);
·if an event includes media items with date/time indicators in a range associated with one date-based event group and media items with date/time indicators in a range associated with another date-based event group, the event could be included in both date-based event groups (at [00171]);
·a user could rename (i.e., change the title of) events and/or pattern-based event groups (see, e.g., Figures 5D-E), but not date-based event groups (at [00171]);
·when displaying a portion of the collection of media items in response to a corresponding request (item 656, Figure 6E), the portion (usually corresponding to a specific time period, e.g., December 2012 to February 2013 on Figure 5A) is specified by the user, which could be done by using, e.g., pinch/depinch or vertical scrolling gestures as explained with reference to Figures 5A-S (at [00172]);
·the event identifier (or title) could be user selected, device generated, or default generated (at [00173]);
·the displayed locations could be “the most unique or interesting locations in the respective event, which includes locations that are infrequently visited by the user (e.g., locations for which there are few photos in the photo library at the location) and/or locations for which there is a large block of temporally contiguous photos in the respective event” (at [00174]).
The description continues by discussing Figures 7A-D, which represent “flow diagrams illustrating a method 700 of organizing a collection of media items into a hierarchy of granularity levels in accordance with some embodiments” (at [00179]). Similarly to what was stated with respect to the benefits of method 600 (see [00151], quoted above), it is stated that:
“[00180] As described below, the method 700 provides an intuitive way to organize a collection of media items into a hierarchy of granularity levels. The method reduces the cognitive burden on a user when organizing a collection of media items into a hierarchy of granularity levels, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, enabling a user to organize a collection of media items into a hierarchy of granularity levels faster and more efficiently conserves power and increases the time between battery charges.” (underlining added)
I have reproduced Figures 7A-D below. The detailed explanations of method 700 at [00181]-[00211] are of similar nature to the explanations of method 600, and do not contain any specifics about the computer implementation. To clarify the wording in Figures 7A-D, I consider that the following is worth mentioning:
·the hierarchy of granularity levels could include “an events view 503 displayed at a detailed-granularity level (shown in Figure 5A), a months view 513 displayed at a detailed-granularity level (shown in Figure 5H), a seasons view 551 displayed at a detailed-granularity level (shown in Figure 5I), and a years view 561 displayed at an overview-granularity level (shown in Figure 5J)” (at [00181], underlining and italic added);
·the first timescale could be a month, and “Figure 5H, for example, shows the device displaying a portion of the collection of media items in a months view 513 with a plurality of month groups 540 corresponding to a month timescale” (at [00183], underlining added); by analogy, it would appear that Figure 5I shows a plurality of season groups 552 corresponding to a season timescale;
·the media item representations each having (or being displayed at) a detailed-granularity size, and being spaced apart from each other by a detailed-granularity distance, appears to be a characteristic of the detailed-granularity levels (at [00185]-[00187]);
·the minimum object-selection size could be 44x44 points/pixels, which makes the media item representations, having detailed-granularity size (which is larger than, or equal to, the minimum object-selection size), selectable by the user (at [00186]);
·the object-selection margin could be the spacing that enables the user to easily distinguish and select between different media item representations; the detailed-granularity distance is greater than the object-selection margin (at [00188]);
·regarding the first/second time range, “Figure 5I, for example, shows a first plurality of representations of the media items 555a that corresponds to media items related to a first time range designated by season group 552a (e.g., Summer 2012). Figure 5I, for example, further shows a second plurality of representations of the media items 555b that corresponds to media items related to a second time range designated by season group 552b (e.g., Fall 2012)” (at [00189], underlining added);
·the third timescale could be a year, and “Figure 5J, for example, shows the device displaying a portion of the collection of media items in a years view 661 with a plurality of year groups 562 corresponding to year timescales” (at [00192], underlining added); importantly, the reference to “the second timescale” (see item 724, Figure 7B) becomes clear later, i.e., at items 740 and 742 on Figure 7C;
·the overview-granularity size could be “below the minimum object-selection size. Figure 5J, for example, shows the device detecting a tap gesture with contact 568 over a respective representation of a media item in the plurality of representations of media items 566a. Figure 5K, for example, shows the device continuing to display the plurality of representations of media items 566a in response to detecting the tap gesture in Figure 5J. In Figures 5J-5K, the respective representations in the plurality of representations of media items 566a are displayed at the overview-granularity size, and the respective representations in the plurality of representations of media items 566a are not selectable (e.g., each respective representation is less than 44x44 points/pixels). As shown in Figures 5J-5K, the respective representation is unaffected by the tap gesture with contact 568 over the respective representation” (at [00195], underlining and italic added). However, it would appear to me that the expression “not selectable” (with respect to a representation displayed at an overview-granularity size below the minimum object-selection size) is not used to merely indicate that the representation is too small to be practically selected by the user. For example, on Figures 5L-M the representations of media items 566a are clearly selectable, despite the fact that their overview-granularity size appears to be the same as on Figures 5J-K (see also items 758-764, Figure 7D);
·the overview-granularity spacing could be “zero and the representations in the overview-granularity [could be] adjacent to each other (e.g., as shown in Figure 5J)” (at [00197]);
·the device could “omit thumbnails of photos [i.e., media item representations] when displaying a representation of an event (e.g., in events view 503) or event group (e.g., in months view 513 or seasons 551) but then displays thumbnails of the omitted photos when displaying an overview of photos taken over the course of a year (e.g., in years view 661) that includes the event” (at [00198]);
·the second distance can be as small as the line or border 567 on Figure 5J (at [00199]);
·regarding the second detailed-granularity level corresponding to a second timescale larger than the first timescale, “Figure 5I, for example, shows the device displaying a seasons view 541 [it seems that this should be 551] at the second detailed-granularity level corresponding to a multi-month or seasons timescale. In contrast, Figure 5H, for example, shows the device displaying a months view 513 at the first detailed-granularity level corresponding to single month timescale” (at [00201]);
·the second respective representation could be adjacent to the first respective representation (at [00211]).
There appears to be some overlap between methods 600 and 700. For example, following the discussion of method 700, the description explains:
“[00212] It should be understood that the particular order in which the operations in Figures 7A-7D have been described is merely exemplary and is not intended to indicate that the described order is the only order in which the operations could be performed. One of ordinary skill in the art would recognize various ways to reorder the operations described herein. Additionally, it should be noted that details of other processes described herein with respect to other methods described herein (e.g., method 600) are also applicable in an analogous manner to method 700 described above with respect to Figures 7A-7D. For example, the contacts, granularity levels, media items, representations of media items, events, event groups, and other user interface elements described above with reference to method 700 optionally have one or more of the characteristics of the contacts, granularity levels, media items, representations of media items, events, event groups, and other user interface elements described herein with reference to other methods described herein (e.g., method 600). For brevity, these details are not repeated here.”
Nearly identical, mutatis mutandis, explanation is provided at [00178] following the discussion of method 600.
In addition, the detailed explanations provided in the description with respect to both Figures 6A-E and (even to a larger extent) Figures 7A-D often refer to some of Figures 5A-S to illustrate the working of methods 600 and 700. This leads to the conclusion that these methods are not completely different from the procedures already explained earlier with reference to Figures 5A-S, but instead largely represent what appears to be different ways to describe (with occasional extra details in the logic) that same procedures. Indeed, my understanding is that the procedures described by reference to Figures 5A-S involve a collection of media items divided into a plurality of multi-item events (method 600) and organised into a hierarchy of granularity levels (method 700).
Following the discussion of methods 600 and 700, the description returns to the devices, on which the methods could be implemented. This is done by reference to Figure 8 and Figure 9. Figure 8 (reproduced below) “shows a functional block diagram of an electronic device 800 configured in accordance with the principles of the various described embodiments” (at [00213]).
In my opinion, Figure 8 is a very good example of the level of generality, at which the implementation is described. The corresponding explanations are also telling:
“[00213] … The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described in Figure 8 are, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.” (underlining and italic added)
The additional explanations with respect to Figure 8 at [00214]-[00227] do not provide any further details that are materially different to what was already described.
Figure 9 “shows a functional block diagram of an electronic device 900 configured in accordance with the principles of the various described embodiments” (at [00228]). Again:
“[00228] … The functional blocks of the device are, optionally, implemented by hardware, software, or a combination of hardware and software to carry out the principles of the various described embodiments. It is understood by persons of skill in the art that the functional blocks described in Figure 9 are, optionally, combined or separated into sub-blocks to implement the principles of the various described embodiments. Therefore, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.” (underlining and italic added)
Figure 9 is very similar to Figure 8 and, in the interest of brevity, I have not reproduced Figure 9. By way of an overview:
“[00229] As shown in Figure 9, an electronic device 900 includes a display unit 902 configured to display a graphic user interface, a touch-sensitive surface unit 904 configured to receive contacts, and a processing unit 906 coupled to the display unit 902 and the touch-sensitive surface unit 904. In some embodiments, the processing unit 906 includes an obtaining unit 908, a receiving unit 910, a displaying enabling unit 912, a detecting unit 914, and a determining unit 916.”
As it was the case with the explanations with respect to Figure 8, the explanations with respect to Figure 9 at [00230]-[00241] do not provide any further details that are materially different to what was already described; hence I do not see the need to comment on these.
At the end of the description, it is clarified that:
“[00244] The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best use the invention and various described embodiments with various modifications as are suited to the particular use contemplated.”
The claimed invention
The Specification ends with 15 claims. Claim 1 is the only independent claim, and it defines a method performed “at an electronic device with a display and a touch-sensitive surface”. Dependent claims 2-13 also define a method, whereas claim 14 defines an electronic device which comprises, inter alia, “one or more programs … stored in the memory [of the device] and configured to be executed by the one or more processors [of the device], the one or more programs including instructions for performing any of the methods of claims 1-13”, and claim 15 defines a computer program comprising instructions “to perform any of the methods of claims 1-13”. Claim 1 is reproduced below (some formatting added to assist readability):
“1. A method comprising:
at an electronic device with a display and a touch-sensitive surface:
accessing a collection of media items,
wherein the media items are organized into a hierarchy of granularity levels,
wherein each granularity level in the hierarchy of granularity levels corresponds to a respective timescale;
displaying a first plurality of representations of the media items of the collection
organized in accordance with a first granularity level in the hierarchy of granularity levels,
wherein the first granularity level corresponds to a first timescale;
detecting a first pinch gesture on the touch-sensitive surface; and
in response to detecting the first pinch gesture on the touch-sensitive surface,
replacing display of the first plurality of representations of the media items with display of a second plurality of representations of the media items of the collection
organized in accordance with a second granularity level in the hierarchy of granularity levels,
wherein the second granularity level corresponds to a second timescale that is longer than the first timescale,
wherein the second plurality of representations of the media items includes more media items than the first plurality of representations of the media items;
wherein the first plurality of representations that correspond to the first granularity level are grouped into a first plurality of groups,
wherein a first group and a second group in the first plurality of groups each corresponds to the first timescale; and
wherein the second plurality of representations that correspond to the second granularity level are grouped into a second plurality of groups,
wherein a third group and a fourth group in the second plurality of groups each corresponds to the second timescale.”
Prima facie, it does not appear that the claim interpretation presents any difficulties that would be impossible to resolve by referring to the body of the Specification. Given that clarity was not raised as an objection during examination, and taking into account the outcome of my decision, I do not consider it warranted for me to further investigate any potential clarity issues if they indeed exist.
As a mere visualisation of the invention defined in claim 1, one could consider the procedure depicted in Figures 5H-I. The first plurality of representations of the media items is displayed as shown on Figure 5H, whereas the second plurality of representations is displayed as shown on Figure 5I. The first granularity level is represented by the months view of Figure 5H, the first timescale being a month, the first plurality of groups being represented by the plurality of months, the first and second groups being represented by two of the months. The second granularity level is represented by the seasons view of Figure 5I, the second timescale being a three-month season, the second plurality of groups being represented by the plurality of seasons, the third and fourth groups being represented by two of the seasons.
Alternatively, one could consider the seasons view as depicted on Figure 5I as the illustration of the first plurality of representations that correspond to the first granularity level, grouped into a first plurality of groups. In this case, the years view as depicted on Figure 5J would be an illustration of the second plurality of representations that correspond to the second granularity level, grouped into a second plurality of groups, wherein the third and fourth groups are represented by two of the years. This alternative visualisation has the benefit of unambiguously showing that the second plurality of representations of the media items includes more media items than the first plurality of representations of the media items, and it also appears to be preferred by the Applicant as noted later in the decision.
Manner of manufacture – the objection and the submissions
The Examiner’s objection on manner of manufacture
[00136] … In some embodiments, the device is enabled to switch between views (e.g., event view 503, month view 513 (Figure 5H), seasons view 551 (Figure 5I), and year view 561 (Figure 5J)) by detecting a pinch or depinch gesture (e.g., a pinch gesture corresponding to movement of contacts 532 and 534 in Figure 5F or a depinch gesture corresponding to movement of contacts 582 and 584 in Figure 5O) or, optionally, by detecting a horizontal swipe gesture with three or more contacts. In some embodiments, the device is similarly enabled to switch between views by detecting a tap gesture over on an affordance corresponding to a respective view (e.g., events affordance 504 in Figure 5F, months affordance 512 in Figure 5G, seasons affordance 550 in Figure 5H, and years affordance 560 in Figure 5I). …” (underlining and italic added)
133. In other words, using a pinch gesture to initiate the display transition as claimed is merely a simple selection of one well-known user input technique from a group, including (in addition to the selected) other, apparently also well-known, user input techniques (i.e., a horizontal swipe gesture with three or more contacts and a tap gesture on an affordance), with no particular advantage of that selection over the other options. The Specification strongly suggests that this is no more than a mere selection from several (known) alternatives, the selection being largely irrelevant to the solution proposed by the present invention.
134. Finally, the Specification does not appear to unambiguously suggest, and certainly does not allow me to conclude, that the particular gestures or other user interface techniques, employed by the proposed media item management software, are any different, or work any differently as parts of the user interface, in comparison with those used in the prior art.
135. In the present circumstances, unlike the Delegate in Aristocrat’16, I cannot infer any advance in the user interface, because, in light of the Specification and the available evidence, I am not aware of any prior art user interface problems that appear to be solved by the user interface of the presently proposed software solution. I conclude that the proposed user interface does not provide any technical solutions and does not contribute any technicality to the effect of the working of the invention.
Other solutions and effects of the working of the invention
136. I have already discussed the issue of devices with small form factors and concluded that the present invention does not provide anything specifically addressing the shortcomings of such devices.
137. As for “battery-operated electronic devices, [for which] such methods and interfaces may conserve power and increase the time between battery charges”, it is worth noting that this is a potential consequence of replacing any, inefficient for a particular task, software application with more efficient one for the same task, and not necessarily a technical effect produced as a result of the working of the invention. In addition, conserving resources does not necessarily result in patentability in all cases. For example, one cannot patent the scheme of daylight savings (even if it were novel and inventive) or any other similar administrative arrangement for conserving electricity or other resources.
138. In addressing the logistical problems discussed earlier, the media item management software of the present invention has the logistical functionality of organising a collection of media items so that a user of the software could easily navigate the collection using the user interface of the software. This functionality is achieved by creating an appropriate set of logical rules for, inter alia, organising the media items into a hierarchy of granularity levels, wherein each granularity level in the hierarchy of granularity levels corresponds to a respective timescale, and grouping the representations of media item into different groups. I consider that creating and following a set of rules to achieve a particular logistical outcome is not a process that belongs to the useful arts within the meaning of NRDC. Hence, the mere fact that the proposed software has logistical functionality (which may well be useful to the user of the software) does not provide a technical effect of the type that warrants finding of manner of manufacture. The presently proposed software may result in easier and more efficient performing of some specific tasks, related to the navigation of the collection of media items, but this is due to its logistical functionality and (as discussed above) not due to its functionality with respect to the user interface.
139. The Specification asserts that “[s]uch methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface” (at [0005]). I am unsure that reducing the cognitive burden on a user is necessarily a technical effect. Furthermore, in the present case, the cognitive burden was presumably caused by the limited logistical functionality of the prior art media item management software, and not by any technical or user interface related issues.
140. In that respect, it is perhaps worth referring to Repipe, and considering claim 1 of patent 2017100943 as reproduced in paragraph [3] of that decision:
“3 Claim 1 of the 943 Patent as granted is as follows:
1. A method of providing information for risk management to a user of a personal computing device performing a job in the field, said method comprising:
identifying the user;
selecting a job to be performed by the identified user;
downloading information related to the selected job to the portable device, wherein the downloaded information comprises information determined according to the user’s role in relation to the selected job, and the maximum authority that role has in relation to the selected job;
displaying the downloaded information for selection of one or more indicia in the information so as to complete a document required for the identified user’s role in relation to the selected job;
receiving input to the portable device, wherein the input comprises a selection of at least one of the indicia;
uploading the input, whereby a record of the input is able to be used in relation to the selected job.”
141. It could potentially be argued that the above invention also reduces the cognitive burden on the user, and provides an improved or more efficient user interface, because of the simplified user input to the personal computing device, whereby the user does not have to provide a large number of inputs to search for the job and/or the information related to the job. Despite this, the Court decided that the claimed invention was not a manner of manufacture.
142. Finally, I do not consider that software merely providing easier and faster navigation of a collection of media items has any effect on “what might broadly be called ‘computer technology’” (Research Affiliates at [118] as quoted earlier). It does not represent any improvement, or alteration, to the general functionality of the electronic computing devices as such. Instead, it is simply specialised software, which provides particular functionality that does not go beyond what is normally provided by any new software. In other words, it does not provide any enabling functionality to the electronic computing devices.
143. This, for example, can perhaps be contrasted with the invention in Re International Business Machines Corporation v Patrick Anselm Smith, Commissioner of Patents [1991] FCA 625 (IBM). In that invention, the specific differences, in the way, in which computers perform integer vs floating point mathematical operations, were utilised to produce an effect, which was much more general in its applicability and provided enabling functionality in the field of “computer graphics”, which I consider part of the “computer technology”:
“3. In the present case, the complete specification related to what was described as a ‘method and apparatus for producing a curve image’, regarding which it stated: ‘The present invention relates to computer graphics and more specifically to a method and apparatus for generating curves on computer graphics displays.’ The recital of the prior art commenced by pointing out that the ‘evolution of computer technology has resulted in the creation of a sophisticated technical area devoted to the representation of graphical information generated by computers’, an area termed computer graphics. The method generally used for the generation of smooth curves in computer graphics is then said to have the disadvantage that a curve ‘will in general have perceptible discontinuities in slopes of curvature at joints between successive curve segments unless specific curve types or forms are used’. Difficulties ‘in generating the coordinate values’ are also mentioned, and compounded inaccuracies resulting from accepted techniques. The nature of the solution described in the specification has been summarised by the Delegate, as follows:
‘The invention involves the use of forward differencing or incrementing techniques to produce sets of integral numbers for the control points in each dimension. The numbers are produced by incremental scaling of initial parameters using integer arithmetic, which increases the speed and accuracy of the computations. Integer arithmetic multiplication operations are performed by binary shifting operations in paths connecting registers in computer curve generator circuits, and integer arithmetic adding operations are performed by adders placed serially with the registers such that the computations may be performed in a pipelined fashion. A final computational stage includes a division operation only requiring binary shifting and adding rather than a floating point operation.’
…
16. … What is new is the application of the selected mathematical methods to computers, and in particular, to the production of the desired curve by computer. This is said to involve steps which are foreign to the normal use of computers and, for that reason, to be inventive. The production of an improved curve image is a commercially useful effect in computer graphics.” (underlining added)
144. In summary, I consider that the effect of working of the invention is limited to providing the users with an easy and convenient way for navigating collections of media items stored, e.g., in well-known electronic computing devices. This is achieved through the logistical functionality of the media item management software of the present invention, running on such devices and utilising a normal touch based user interface. Importantly, management of media items is just one very specific non-technical task from the huge variety of tasks, for which the electronic computing devices are generally used. The inevitable conclusion is that the working of the invention does not produce a technical effect.
The substance of the invention defined in claim 1
145. Relevantly, the Applicant submits that:
“… the contribution or substance of the claimed invention includes, at the very least, the following combination of features:
·displaying a first plurality of representations of the media items of the collection organized in accordance with a first granularity level in the hierarchy of granularity levels, wherein the first granularity level corresponds to a first timescale;
·the first plurality of representations that are displayed correspond to the first granularity level are grouped into a first plurality of groups, wherein a first group and a second group in the first plurality of groups each corresponds to the first timescale;
·in response to detecting a first pinch gesture on the touch-sensitive surface;
·replacing display of the first plurality of representations of the media items with display of a second plurality of representations of the media items of the collection organized in accordance with a second granularity level in the hierarchy of granularity levels; and
·the second plurality of representations that correspond to the second granularity level are grouped into a second plurality of groups, wherein a third group and a fourth group in the second plurality of groups each corresponds to the second timescale.” (AS, page 7, original bold, underlining added)
146. While I may not disagree with the above in its entirety, it would appear that the Applicant actually recites the majority of features of claim 1, notable omissions being that the second timescale is longer than the first timescale, and that the second plurality of representations includes more media items than the first plurality. I am unsure of the reasons for omitting these features but, based on the qualifier “at the very least”, it may be possible that the Applicant considers that the listed features, in themselves, represent sufficient contribution to the state of the art, and no more features are necessary. Alternatively, the missing features appear the be somewhat unrelated to the rest of the features, which may also be the reason for omitting them. The Applicant continues:
“Even if pinching gestures do form part of the state of the art as asserted by the examiner (which is not conceded), the applicant respectfully submits that the mere knowledge of these pinching gestures do not result in the combination of features claimed in the present invention being so well known so as to not be able to be a relevant contribution for the purposes of the inquiry into patentable subject matter.
… Accordingly, the applicant submits that the skilled person would understand the contribution of the presently claimed invention to include the above combination of features. This combination includes, inter alia, a new use of known pinching gestures, per se, in a way that it has never been used or thought to be used previously. …
Further, as currently understood, the applicant submits that the state of the art does not include, at the very least, the electronic device grouping representations of media items according to different time scales as the user navigates through the hierarchy of timescales by performing a pinch gesture. With respect, the examiner’s assertion that the substance of the claimed invention lies ‘in the scheme of arranging the information in a more convenient manner by recategorising and regrouping media items based on hierarchy level, before being displayed under the hierarchy level’ disregards the claimed interrelationship between these functions and the specific mechanism of detecting the touch gesture that is performed by the claimed electronic device to enable these functions to be performed.” (AS, page 7, original italic, underlining added)
147. While I acknowledge that the claimed invention defines a number of features, some of which having inevitable working interrelationships simply because they represent logically ordered steps of a method, it is important to emphasise that:
“Recognising that the claims are to a method and system comprising a combination of integers, it is necessary to understand where the inventiveness or ingenuity is said to lie.” (RPL Central at [112] as quoted earlier, underlining added)
148. Despite the Applicant’s assertion to the contrary, based on my previous analysis, I do not consider that there is any “inventiveness or ingenuity” in the use of the pinch gesture as defined in the claim. The fact that pinch gestures have never been used, or thought to be used, previously in the same way (i.e., as an input triggering the claimed response) is a direct trivial consequence of the fact that, apparently, no software with the same functionality existed previously. The same is true for the vast majority of new software applications where known input gestures are used to produce a response that was never produced before.
149. The Applicant also talks about “the specific mechanism of detecting the touch gesture that is performed by the claimed electronic device”. While a “touch gesture” is not clearly defined in the claims (as opposed to pinch, de-pinch, tap, contact, liftoff, and movement of the contact), this is of no consequence as I am unable to identify in the Specification any such “specific mechanism of detecting” in relation to any gesture. As I noted earlier, the programming is left entirely to the skilled addressee wishing to perform the invention, hence it is reasonable to conclude that detecting any user gestures is not limited to a specific mechanism; instead, it is to be implemented in the normal way that the skilled addressee will be familiar with.
150. I conclude that the substance of the claimed invention involves a set of logical rules, utilised to produce software in unspecified manner employing known user interaction techniques, including the well-known and well-understood transitional feature of a pinch gesture, the software to run on well-known electronic computing devices having touch-sensitive surfaces, the software having functionality for:
·organising a collection of media items into a hierarchy of at least two granularity levels, each corresponding to a different timescale;
·displaying a plurality of representations of media items corresponding to one of the granularity levels, the representations grouped into a plurality of groups, each corresponding to the timescale of the granularity level; and
·in response to receiving an appropriate user input via a touch-sensitive surface, displaying another plurality of representations of media items corresponding to another of the granularity levels, the representations grouped into a plurality of groups each corresponding to the timescale of the granularity level.
Noting that the latter timescale is longer than the former, and the latter plurality of representations of media items is more numerous than the former.
151. I note the possible argument that the actual organising of the collection of media items is not part of claim 1, because this claim explicitly defines only “accessing a collection of media items, wherein the media items are organized into …”. However, I consider that the particular way, in which the media items are organised, is a clear limitation of claim 1, hence, on purposive construction, the organising of media items in that particular way, i.e., following a corresponding particular set of rules with respect to the hierarchy of granularity levels, etc., is implicit in the claim.
Does any of the claims define an invention that is a manner of manufacture?
152. Having identified the substance of the invention as above, it is clear that, as a matter of substance, the invention defined in claim 1 cannot be characterised as a patentable technological innovation. Instead, I consider that it is properly characterised as a logistical scheme for organising and presenting on an electronic computing device, a collection of digital media items, in a particular manner following a particular set of rules. The scheme is associated with well-known electronic computing devices, and is implemented on these devices in an unspecified way. The resulting software interacts with the user (i.e., presents information, receives input, and responds to the input) in a way, which is not foreign to the usual user interaction performed by software running on these devices. I conclude that the invention defined in claim 1 is not a manner of manufacture.
153. On reviewing the rest of the claims, I consider that they, at most, define further aspects of the logistical scheme represented as additional logical rules in the set. In the section “5 Patentable subject matter and inventive step - dependent claims”, the Applicant provides submissions with respect to claims 6, 10, and 13. Claim 6 is reproduced below:
“6. The method of claim 1, including:
detecting a tap input at a location on the touch-sensitive surface that corresponds to a respective representation of a respective media item; and
in response to detecting the tap input:
in accordance with a determination that the respective representation was displayed in the first plurality of representations in accordance with the first granularity level,
replacing display of the plurality of representations with display of the respective media item; and,
in accordance with a determination that the respective representation was displayed in the second plurality of representations in accordance with the second granularity level,
continuing to display the second plurality of representations.”
154. In other words, upon detecting a tap gesture with respect to a representation of a media item, depending on the granularity level, displaying, or not, the respective media item. With respect to claim 6, the Applicant submits that “the contribution provided by the claimed invention is technical and is analogous to the invention considered patentable in [Aristocrat’16]” (AS, page 18), and further notes that:
“Performing a two-step determination in accordance with the electronic device detecting a tap at a representation of a media item may provide an improved human machine interface by eliminating the need for extra, separate, inputs or steps to organise and navigate the collection of media items, which may, in turn, result in an improvement in the functioning of the electronic device by reducing power usage and increasing battery life of the electronic device. As discussed in further detail below, the applicant submits that it was not normal (at the priority date of 15 March 2013) to configure an electronic device in this particular way.” (AS, page 19, underlining added)
155. In my view, “[p]erforming a two-step determination” as defined in claim 6 is, in fact, performing an action (in response to a corresponding input) only if a particular condition is satisfied. This is simply a rule in the set of logical rules. I have already discussed the nature of the invention and the Delegate’s reasoning in Aristocrat’16, and I am unable to find any similarities between the invention in that case and the present invention defined in claim 6. Regarding the Applicant’s arguments concerning “improved human machine interface”, “eliminating the need for extra, separate, inputs or steps”, and “reducing power usage and increasing battery life”, these were also addressed in my previous discussions. The additional features, defined in claim 6, do not have the potential to alter the nature or the outcome of these discussions.
156. Claim 10 (together with claim 8, to which it is appended) is reproduced below:
“8. The method of claim 1, including:
while displaying the second plurality of representations of the media items,
detecting a contact at a location on the touch-sensitive surface that corresponds to a first representation of a first media item in the second plurality of representations of the media items; and,
in response to detecting the contact:
displaying an enlarged representation of the first media item.”
“10. The method of claim 8, including:
while displaying the enlarged representation of the first media item,
detecting liftoff of the contact from the touch-sensitive surface; and
in response to detecting liftoff of the contact from the touch-sensitive surface,
replacing display of the second plurality of representations with display of the respective media item.”
157. The Applicant’s submissions with respect to claim 10 are based mostly on the existence of “a definite link and working interrelationship between sensing of the contact [on the touch-sensitive surface] and the replacement of the display of the second plurality of representations with the display of the respective media item, which cannot be described as ‘a mere scheme’”, and on providing “a technical means [e.g., via the duration of the contact] for the user to control the manner in which the respective media item is displayed at the electronic device”, in addition to “providing context for the media item to the user prior to the liftoff of the contact being detected and the representation of the media item [it seems that this should read the media item itself] replacing the display of the second plurality of representations” (AS, page 20).
158. In my view, what is defined is that an enlarged representation of the media item will be displayed while the user is in contact with (i.e., touching) the corresponding location on the touch-sensitive surface, and the media item itself will be displayed when the user terminates the contact. This is simply another logical rule. Neither the Specification, nor the available evidence, suggests that there is anything unusual in this way of operating a user interface through a touch-sensitive surface.
159. Claim 13 is also reproduced below:
“13. The method of any of claims 1-12, further comprising:
before displaying the first plurality of representations of the media items of the collection organized in accordance with the first granularity level in the hierarchy of granularity levels, wherein the first granularity level corresponds to the first timescale,
displaying a third plurality of representations of the media items of the collection organized in accordance with a third granularity level in the hierarchy of granularity levels,
wherein the third granularity level corresponds to one or more events,
wherein each event includes a plurality of the media items selected in accordance with one or more patterns determined in metadata associated with the collection of media items;
while displaying the third plurality of representations of the media items of the collection organized in accordance with the third granularity level in the hierarchy of granularity levels,
detecting a third pinch gesture on the touch-sensitive surface; and
in response to detecting the third pinch gesture on the touch-sensitive surface,
replacing the display of the third plurality of representations of the media items with the first plurality of representations of the media items of the collection organized in accordance with the first granularity level in the hierarchy of granularity levels,
wherein the first granularity level corresponds to the first timescale.”
160. The granularity level corresponding to one or more events is exemplified by the events view in Figures 5A, B, and G. The Applicant’s submissions with respect to claim 13 mostly emphasise the “transitioning from event-based organisation to a timescale-based organisation of media representations” and providing “a particular working interrelationship between a gesture and the transitioning of event-based grouping to timescale based grouping that cannot be described as ‘a mere scheme’” (AS, page 22).
161. It is important to note that whether a granularity level corresponds to a timescale, or to one or more events, is also governed by the set of logical rules of the logistical scheme. The transition between “event-based organisation” and “timescale-based organisation” in response to a specified input or gesture is likewise part of these rules. As to the “particular working interrelationship …”, this is not different in nature to the corresponding working interrelationship of claim 1.
162. In conclusion, I consider that each one of the dependent claims is still directed, as a matter of substance, to the above mentioned logistical scheme for organising and presenting a collection of digital media items in a particular manner following a particular set of rules. As I also mentioned, the scheme is associated with well-known electronic computing devices, and is implemented on these devices in an unspecified way. The resulting software interacts with the user (i.e., presents information, receives input, and responds to the input) in a way, which is not foreign to the usual user interaction performed by software running on these devices.
163. Any differences between the dependent claims and claim 1 are merely reflective of further details of the scheme and the corresponding set of rules implemented by using well-known and well-understood computing technology, including user interface techniques. I am not satisfied that any of the additional features defined in these claims involves aspects that could change the nature of the invention as a matter of substance. Therefore, I consider that none of the claims in the Specification defines an invention that is a manner of manufacture.
Can an allowable amendment overcome the negative finding?
164. Earlier in this decision, I have discussed the body of the Specification in sufficient detail (perhaps even excessive detail), and I am unable to identify any material that could form the basis for a claim that defines a patentable invention. The disclosure is limited to the high-level logical details of the logistical functionality of the proposed software solution.
165. In the circumstances where I can see no clear way to overcome my negative finding with respect to manner of manufacture, and in view of the public interest in certainty as to the status of the Application, I consider that providing the Applicant with a further opportunity to amend and continuing the examination would serve no useful purpose.
Inventive step
166. The Last Report also contains an objection that the claimed invention does not involve an inventive step and the Applicant has provided detailed submissions on the issue. I consider that, in view of my finding with respect to manner of manufacture, any consideration of inventive step would be futile as it will not affect the final outcome of my decision.
167. Therefore, I will not decide on inventive step.
Conclusion
168. I have found that no claim of the Application, as proposed to be amended, defines an invention that is a manner of manufacture. Furthermore, I have formed the view that no allowable amendment could result in claiming a patentable invention.
169. It follows that the Application should be refused regardless of whether the claimed invention is obvious or not.
Dr V. Z. Kolev
Delegate of the Commissioner of Patents
0