Commissioner’s Decision #1484
Décision du commissaire #1484
TOPICS: O00 Obviousness
SUJETS: O00 Évidence
Application No: 2,539,645
Demande no: 2 539 645
IN THE CANADIAN PATENT OFFICE
DECISION OF THE COMMISSIONER OF PATENTS
Patent application number 2,539,645, having been rejected under subsection 30(3) of the Patent Rules, SOR/96-423, has consequently been reviewed in accordance with paragraph 30(6)(c) of the Patent Rules. The recommendation of the Board and the decision of the Commissioner are to refuse the application unless the necessary amendments are made.
Agent for the Applicant
BORDEN LADNER GERVAIS LLP
World Exchange Plaza
1100 – 100 Queen Street
OTTAWA Ontario
K1P 1J7
INTRODUCTION
[1] This recommendation concerns the review of rejected Canadian patent application number 2,539,645 (“the instant application”), which is entitled “TIME OF FLIGHT TEAT LOCATION SYSTEM” and is owned by GEA FARM TECHNOLOGIES GMBH. (“the Applicant”). A review of the rejected application has been conducted by the Patent Appeal Board (“the Board”) pursuant to paragraph 30(6)(c) of the Patent Rules. As explained in more detail below, our recommendation is that the Commissioner of Patents refuse the application unless the necessary amendments are made.
BACKGROUND
The Application
[2] Patent application no. 2,539,645 was filed in Canada on March 15, 2006 and was laid open to the public on September 15, 2007.
[3] The instant application relates to an automated milking system that is equipped with time-of-flight cameras (which enable a three-dimensional acquisition of objects in space) for acquiring the locations of the teats of an animal and the milking cups of a milking apparatus, and means for enabling an application of the milking cups on the teats.
Prosecution History
[4] On April 3, 2014, a Final Action (“FA”) was written pursuant to subsection 30(4) of the Patent Rules. The FA stated that the instant application is defective on the ground that the claims 1-3 on file at the time of the FA (“claims on file”) would have been obvious and are therefore non-compliant with section 28.3 of the Patent Act.
[5] In a September 29, 2014 response to the FA (“R-FA”), the Applicant submitted arguments in favour of the non-obviousness of the claims on file.
[6] As the Examiner considered the application not to comply with the Patent Act, pursuant to paragraph 30(6)(c) of the Patent Rules, the application was forwarded to the Board for review on April 7, 2015, along with an explanation outlined in a Summary of Reasons (“SOR”) for maintaining the rejection.
[7] In a letter dated July 7, 2015, the Board forwarded to the Applicant a copy of the SOR and requested that the Applicant confirm its continued interest in having the application reviewed or that the application be withdrawn.
[8] In a response dated October 2, 2015, the Applicant indicated its continued interest in having the application reviewed and requested an oral hearing.
[9] The present panel (“the Panel”) was formed to review the instant application under paragraph 30(6)(c) of the Patent Rules.
[10] In a preliminary review letter (“PR letter”) dated January 21, 2019, the Panel set out its preliminary analysis of the obviousness issue with respect to the claims on file.
[11] In a letter dated February 11, 2019, the Applicant indicated that it wished to attend an oral hearing, which was scheduled for February 25, 2019.
[12] In a response to the PR letter (“R-PR”) dated February 15, 2019, the Applicant submitted a set of proposed claims 1-8 and made arguments in favour of the proposed claims. The Applicant further indicated that it no longer wished to attend an oral hearing.
ISSUE
[13] The issue to be addressed by the present review is whether the subject matter of claims 1-3 (“the claims on file”) is obvious and therefore non-compliant with section 28.3 of the Patent Act.
[14] If the claims on file are considered to be defective, under subsection 30(6.3) of the Patent Rules we may turn to the proposed claims and consider whether they constitute amendments necessary for compliance with the Patent Act.
LEGAL PRINCIPLES AND OFFICE PRACTICE
Claim Construction
[15] In accordance with Free World Trust v Électro Santé Inc, 2000 SCC 66, essential elements are identified through a purposive construction of the claims done by considering the whole of the disclosure, including the specification and drawings (see also Whirlpool Corp v Camco Inc, 2000 SCC 67 at paras 49(f) and (g) and 52). In accordance with the Manual of Patent Office Practice, §13.05 (revised June 2015), the first step of purposive claim construction is to identify the person skilled in the art and their relevant common general knowledge (CGK). The next step is to identify the problem addressed by the inventors and the solution put forth in the application. Essential elements can then be identified as those required to achieve the disclosed solution as claimed.
Obviousness
[16] Section 28.3 of the Patent Act sets out the conditions under which the subject matter of a claim may be found to be obvious:
28.3 The subject-matter defined by a claim in an application for a patent in Canada must be subject-matter that would not have been obvious on the claim date to a person skilled in the art or science to which it pertains, having regard to
(a) information disclosed more than one year before the filing date by the applicant, or by a person who obtained knowledge, directly or indirectly, from the applicant in such a manner that the information became available to the public in Canada or elsewhere; and
(b) information disclosed before the claim date by a person not mentioned in paragraph (a) in such a manner that the information became available to the public in Canada or elsewhere.
[17] In Apotex Inc v Sanofi Synthelabo Canada Inc, 2008 SCC 61 [Sanofi] at paragraph 67, the Supreme Court of Canada proposed a four-step approach to performing the obviousness assessment:
(1) (a) Identify the notional “person skilled in the art”;
(b) Identify the relevant common general knowledge of that person;
(2) Identify the inventive concept of the claim in question or if that cannot readily be done, construe it;
(3) Identify what, if any, differences exist between the matter cited as forming part of the “state of the art” and the inventive concept of the claim or the claim as construed;
(4) Viewed without any knowledge of the alleged invention as claimed, do those differences constitute steps which would have been obvious to the person skilled in the art or do they require any degree of invention?
ANALYSIS
Claim Construction
[18] The claim set on file contains three independent claims, which read:
1. In an automated milking system having milking apparatus for extracting milk from a plurality of teats on a live animal, use of a light source for simultaneously illuminating said plurality of teats and of a two-dimensional array camera having a plurality of pixels that are capable of individually providing time of flight data for capturing and delivering data for determining the location of said teats in three dimensions.
2. An automated milking system for extracting milk from a plurality of teats on a live animal, comprising:
movable apparatus adapted to receive instructions characterizing the location of at least one of said teats, to move to said location, to engage said at least one of said teats for extracting milk therefrom;
a light source for simultaneously illuminating an area encompassing said plurality of teats;
a two-dimensional array camera having a plurality of pixels that are capable of individually providing time of flight data; and
electronic means for capturing the output of said array and for delivering data characterizing said location.
3. A method for automatically milking at least one teat among a plurality of teats of a live animal comprising the steps of:
providing movable apparatus adapted to receive instructions characterizing the location of said at least one teat, to move to said location and to engage said teat for extracting milk therefrom;
using a light source to simultaneously illuminate substantially the whole of an area encompassing said plurality of teats;
using a two-dimensional array camera having a plurality of pixels that are capable of individually providing time of flight data to capture an image of said simultaneously illuminated area on a two-dimensional pixel array and for each pixel of said array, determining the distance travelled by reflections of said light from said teats to said each pixel to provide a range value in relation to said pixel;
deriving a three-dimensional representation of said plurality of teats from a plurality of said range values;
processing said three-dimensional representation to characterize said location for said at least one teat;
using said information to provide said instructions; and
moving said apparatus to engage said at least one teat.
The person skilled in the art
[19] In the PR letter, the Panel clarified its understanding of the person skilled in the art:
In the Final Action (“FA”) at page 2, the Examiner accepted the Applicant’s characterization of the person skilled in the art:
(1)(a) The person skilled in the art
The applicant has identified, in their response dated 9 January 2013, in paragraph 3, of page 3, that the person skilled in the art was a mechanical engineer, with robotics experience, who has worked in the field of automated milking for 3-5 years. The examiner agrees.
We preliminarily adopt this characterization of the skilled person for the purposes of our analysis.
[20] The above characterization was not disputed by the Applicant in the R-PR. We apply it in our analysis below.
The relevant common general knowledge
[21] In the PR letter, the Panel clarified its understanding of the relevant CGK:
In the FA at page 2, the CGK was set out as follows:
(1)(b) The common general knowledge
The person skilled in the art would be expected to possess the common general knowledge relating to automated milking, which, together with knowledge of robotics, would include knowledge of 3D camera systems to locate the teats for automatic attachment of the teat cups. See D1 and D3.
The Applicant did not dispute the above in the response to the Final Action (“R-FA”), and we preliminarily adopt this characterization of the relevant CGK for the purposes of our analysis.
[22] This characterization was not disputed by the Applicant in the R-PR. We apply it in our obviousness analysis.
Essential/Non-Essential Elements
[23] In the PR letter, the Panel stated the following with respect to the essential and non-essential features of the claims:
The FA did not identify any elements of the claims as being non-essential, nor did it identify any issues regarding the interpretation of claim language. There was no disagreement between the Examiner and Applicant in this regard and so we proceed on the same basis in our analysis.
[24] This construction was not disputed by the Applicant in the R-PR. We apply it in our analysis below.
Obviousness
(1)(a) Identify the notional “person skilled in the art”
[25] The person skilled in the art has been set out above under Claim Construction at paragraph [19].
(1)(b) Identify the relevant common general knowledge of that person
[26] The relevant CGK has also been identified above under Claim Construction at paragraph [21].
(2) Identify the inventive concept of the claim in question or if that cannot readily be done, construe it
[27] In the PR letter, we stated with respect to the inventive concept of claims 1-3 that:
In the FA at page 2 a summary of the present invention was presented:
The present application discloses an automatic milking system and method featuring a teat location system comprising a light projection source and a camera having a two dimensional array of pixels, each of which provides time of flight data for determining the location of the teats in three dimensions.
The inventive concept was then identified as:
an improved accuracy in the 3D image by using a two-dimensional array camera having a plurality of pixels that are capable of individually providing time of flight data.
The Applicant did not dispute the above in the R-FA, and we adopt it for the purpose of our analysis. This inventive concept is in our view representative of all three claims.
[28] The Applicant made no submission with respect to the above in the R-PR. We therefore apply the inventive concept as stated above in our analysis below.
(3) Identify what if any differences exist between the matter cited as forming part of the “state of the art” and the inventive concept of the claim or the claim as construed
[29] In the PR letter, with respect to the differences between the inventive concept and the state of the art, we stated:
In the Final Action, the following prior art documents were applied:
D1: WO 02/00011 Cöp January 3, 2002
D2: R. Schwarte, “Dynamic 3D Vision”, Proceedings EDMO 2001/Vienna, pp 241-248 (2001)
D3: US 4,867,103 Montalescot September 19, 1989
D4: US 6,323,942 Bamji November 27, 2001
The claim date of the claims of the instant application is the Canadian filing date, March 15, 2006, and all of the prior art references D1-D4 have earlier publication dates.
D1 (Cöp) discloses an automated milking system comprising a milking apparatus for extracting milk from a plurality of teats on a live animal, use of a light source for illuminating the plurality of teats, and two cameras for capturing images used to deduce a three-dimensional stereoscopic image of the udder using an image processing program.
D2 (Schwarte) discloses a time of flight data optical system utilising pixel array cameras, for use in applications including level detection, product monitoring and machine vision. The cameras are disclosed as providing an alternative to laser scanners and stereo vision, which are described as requiring too time-consuming signal processing in fast changing respectively dynamic scenes.
D3 (Montalescot) discloses an automated milking system comprising a milking apparatus for extracting milk from a plurality of teats on a live animal, use of a light source for simultaneously illuminating the plurality of teats, an optical scanning laser and a CCD camera, wherein lines produced by the laser scanner are imaged by the camera, triangulation is used to calculate and the coordinates of the teat, knowing the positions of the scanner and camera, and repeatedly imaging different portions of the surface of an object allows the building of a 3D profile of the object.
D4 (Bamji) discloses a time of flight data optical system using pixel array cameras, the system described as an improvement over typical systems such as scanning laser tomography systems (providing a less costly, less obtrusive system) or scanning laser range finding systems (providing a less bulky, less complex and less costly system) such as those using CCD cameras. D4 further discloses that many applications requiring accurate distance and velocity tracking would benefit from such an improved system that could directly capture 3D imagery, for example, an assembly line welding robot that must determine the precise distance and speed of the object to be welded.
The difference between D1 and D3, considered separately, and the inventive concept of claims 1-3 is that the inventive concept uses a time of flight data optical system having pixel array cameras to determine the location of the teats in three dimensions, whereas D1 uses a stereoscopic vision system, and D3 uses a scanning laser system, to accomplish this function.
[30] The Applicant made no submission with respect to the above in the R-PR. We proceed on the basis of the difference identified in the PR letter, i.e., that the inventive concept uses a time of flight data optical system having pixel array cameras to determine the location of the teats in three dimensions, whereas D1 uses a stereoscopic vision system, and D3 uses a scanning laser system, to accomplish this function.
(4) Viewed without any knowledge of the alleged invention as claimed, do those differences constitute steps which would have been obvious to the person skilled in the art or do they require any degree of invention?
[31] In the R-FA, the Applicant argued that the claims on file would not have been obvious and supported its arguments by referring to three documents:
• Statutory Declaration of Leonard Metcalfe, co-inventor of the invention of the instant application, filed with the Applicant’s response dated January 9, 2013, which declaration in turn refers to two pieces of art as factors in the obviousness analysis (addressed under separate headings, below):
• US Patent application 11/045,898 to Kriesel, with a publication date of June 23, 2005 [“Kriesel”]; and
• M. Westberg, The Time of Flight Based Teat Location (May 5, 2009), a thesis published three years after the claim date of the instant application [“Westberg”].
[32] With reference to these three documents, the Applicant stated that it was not obvious on the claim date to use time of flight cameras in teat detection in automated milking, relying on the following factors:
(a) time of flight technology was immature and still under development on the claim date;
(b) time of flight technology was not considered suitable for the close ranges involved in assessing livestock in a chute or station on the claim date; and
(c) even three years after the claim date, the use of time of flight technology in automated milking systems was considered to be a matter of investigation and research.
[33] In the PR letter, we addressed each of these factors, as follows:
(a) time of flight technology was immature and still under development on the claim date
The Metcalfe declaration states that D2 demonstrates the immature nature of time of flight technology, speaking of the “future promise” of time of flight sensing, but stating that it was “under development.”
However, we note that D2 also teaches, in the Introduction:
After a long way of trial and error finally a door has been opened to dynamic 3D-vision by a new opto-electronic principle and device, the Photonic Mixer Device (PMD).
…
The Photo Gate(PG)-PMD was the first implementation of the PMD-principle, enabling for the first time the realisation of a 3D camera based on PMD.
D2 then discloses different implementations of PMD-pixels for 3D cameras, and states, in the Conclusion:
As demonstrated the PMD-principle leads like a central thread through a number of several PMD-implementations. This new technology provides a wide range of innumerable applications from everyday to industrial and scientific life.
Further, we note that D2 was published in 2001 and, based on the date alone, we do not see how it could be considered to evidence the immature nature of time of flight technology on the claim date of the instant application, March 15, 2006.
Still further, we note that D4, which describes and claims a time of flight data optical system using pixel array cameras, is a granted US patent. As such, it is presumed to have met all the criteria for a valid patent, including the requirement to provide an enabling disclosure of the claimed invention, at the time the patent was granted, on November 27, 2001 (more than four years before the claim date of the instant application). The Applicant has thus far offered no evidence to rebut this presumption.
Finally, if time of flight technology had been as immature on March 15, 2006 as the Applicant argues, then one might expect to see a description in the instant application replete with technical details concerning time of flight cameras, in order to fill any gaps in the knowledge of the skilled person required to enable that person to make and use the claimed invention. But when we turn to the instant application, we note that the Detailed Description of the Preferred Embodiment contains only nine paragraphs, and provides only a broad overview of the invention, with no technical detail on the time of flight cameras. If time of flight technology were as immature as the Applicant suggests, the specification would appear to be insufficient, contravening subsection 27(3) of the Patent Act.
However, in the view we take of the application, the specification appears to be sufficient. Technical details of the time of flight cameras are not required in the description, because such details were within the knowledge of the skilled person on the claim date.
(b) time of flight technology was not considered suitable for the close ranges involved in assessing livestock in a chute or station on the claim date
The Metcalfe declaration states:
33. Prior to March 2006, time of flight cameras were generally considered to be appropriate for measuring relatively long distances (in the order of 100m to many km). This was the case because over shorter distances, the spatial resolution and accuracy was limited by the electronics and the high frequencies required to assess time scales in the order of the speed of light. Time of flight cameras were also known to be subject to temperature-dependent variations in the electronics which affect the accuracy of their resolution considering the picoseconds ranges that needed to be assessed. This also implied that they were best suited to measuring large distances.
And:
37. As noted above, in March 2006, time of flight sensors suffered from the perception that time of flight measurements were only suitable for use over long distances and not as accurate and reliable as triangulation for close range measurements. As a result, they were not thought of as an option for automated milking apparatus that would need to deal with multiple moving teats on a moving animal at a very close range.
However, against the declaration of Metcalfe we have the disclosures of D2 and D4, which, as we have stated above, would have been known to the skilled person. The applications disclosed by D2 (product monitoring, machine vision) and D4 (assembly line robots) would appear to involve target distances similar to those relating to the claimed subject matter of the instant application, and there is nothing in either D2 or D4 to suggest the unsuitability of time of flight systems for such applications. On the contrary, these references expressly teach the use of time of flight systems for such applications.
Further, in contrast to Metcalfe’s contention that time of flight measurements were only considered suitable for use over long distances, D4 discloses that “[i]n practical applications, sensor array 230 preferably has sufficient resolution to differentiate target distances on the order of 1 cm” [column 7, lines 47-49].
Still further, even if time of flight cameras were considered to be “best suited” to measuring large distances, this would not indicate that they were unsuited to measuring smaller distances. And even if they were considered less accurate and reliable than triangulation for close range measurements, this would not demonstrate that the skilled person would have eliminated them as an option in an automated milking apparatus. We have provided our reasons for concluding that the skilled person would have been motivated to use the time of flight cameras of D2 (or D4) as an alternative to those used in the analogous application of teat detection as taught by D1 (or D3). An obvious alternative does not become less so merely because in one respect or another it is inferior to another alternative.
The Applicant also relies on the Kriesel patent, which states that time of flight systems were not among “the range camera technologies best suited for the present invention” in the field of livestock assessment because they “may lack axial resolution due to the high frequency processing required to measure variations in light propagation times” [column 24, lines 13-22].
However, merely because time of flight systems were not considered “best suited” for the particular applications taught by Kriesel (involving volumetric and dimensional measurements of livestock) does not mean that they were unsuited to an automated milking apparatus.
Further, even if Kriesel were understood as teaching the unsuitability of time of flight cameras in applications such as claimed in the instant application, this would have to be considered in view of the totality of the evidence. As noted above, D2 and D4 teach the use of time of flight cameras in applications (machine vision, assembly line robots) that would appear to be more analogous to automated milking apparatuses than to the applications taught by Kriesel.
Finally, we note that Kriesel itself characterizes D4, which discloses a time of flight system, as among the inventions that “have contributed to the current state-of-the-art for the measurement of animals” [column 2, lines 30-31, and column 3, lines 12-23]. Such a statement would not make sense if Kriesel considered time of flight systems to be unsuited for such applications.
(c) even three years after the claim date, the use of time of flight technology in automated milking systems was considered to be a matter of investigation and research
The Metcalfe declaration at ¶41 refers to the Westberg thesis as showing that “[e]ven three years after the filing of the present application in 2006, the use of time of flight cameras was still considered to be a matter of investigation and research, and to hold promise for eventual future applications.”
However, Westberg does not appear to demonstrate what the person skilled in the art would have known about time of flight cameras in 2009. To this point, we note that Westberg fails to mention D2 and D4, which teach the use of time of flight cameras in analogous applications, Kriesel, published in 2005, which discusses time of flight cameras in livestock applications, and the instant patent application, laid open to the public on September 15, 2007, which discloses time of flight cameras used in teat detection in automated milking.
[34] The Applicant made no submission with respect to the above in the R-PR. We adopt the rationale in our analysis.
[35] In the PR letter, we stated that in our preliminary view, the difference identified at step (3)—i.e., that the inventive concept uses a time of flight data optical system having pixel array cameras to determine the location of the teats in three dimensions, whereas D1 uses a stereoscopic vision system, and D3 uses a scanning laser system, to accomplish this function—comprised a step that would have been obvious, for the following reasons:
In the R-FA the Applicant argued that the FA did not provide any reasoning as to why a person of ordinary skill would be motivated to modify the primary references (D1 and D3), by substituting cameras based on time of flight as taught by D2 and D4 for the cameras taught by D1 and D3, respectively, for the specific purpose of teat detection.
However, as noted above, D2 identifies time of flight cameras as an alternative to stereo vision, which is described as requiring too time-consuming signal processing in fast changing respectively dynamic scenes. We note that stereo vision is precisely the system taught by D1. While D2 does not refer specifically to teat detection systems, it discloses applications including level detection, product monitoring and machine vision. As the instant application states that such cameras are known to those skilled in the art of machine vision, the skilled person in the field of the instant application—a mechanical engineer with robotics experience who has worked in the field of automated milking for 3-5 years—would be aware of this reference. The skilled person would have understood from D2 that time of flight cameras would be suitable for many robotic applications using 3D imaging systems. Thus, it is our preliminary view that the skilled person would have been motivated to use the time of flight cameras of D2 as an alternative to those used in the analogous application of teat detection as taught by D1.
Alternatively, D4 describes its system as an improvement over typical systems such as scanning laser tomography systems (providing a less costly, less obtrusive system) or scanning laser range finding systems (providing a less bulky, less complex and less costly system) such as those using CCD cameras. Again, we note that a scanning laser system using CCD cameras is precisely the system taught by D3. While D4 does not refer specifically to teat detection systems, it discloses applications requiring accurate distance and velocity tracking, for example, an assembly line welding robot. As the instant application states that such cameras are known to those skilled in the art of machine vision, the skilled person in the field of the instant application—a mechanical engineer with robotics experience who has worked in the field of automated milking for 3-5 years—would be aware of this reference. (We further note that the Kriesel reference relied upon in the Metcalfe declaration states that D4 contributed to the current [2005] state-of-the-art for the measurement of animals.) The skilled person would have understood from D4 that time of flight cameras would be suitable for many robotic applications using 3D imaging systems. Thus, it is our preliminary view that the skilled person would have been motivated to use the time of flight cameras of D4 as an improvement over the cameras used in the analogous application of teat detection as taught by D3.
For the above reasons, we are of the preliminary view that the subject matter of claims 1-3 would have been obvious to the skilled person on the claim date, and are therefore non-compliant with section 28.3 of the Patent Act.
[36] The Applicant made no submission with respect to the above analysis. Accordingly, we conclude that claims 1-3 on file would have been obvious and are therefore non-compliant with section 28.3 of the Patent Act.
Proposed Claims
[37] In the R-PR, the Applicant proposed claims 1-8 and argued that they were not obvious in view of the cited references. Proposed claims 1-8 read as follows:
1. An automated milking system having milking apparatus having milking cups for extracting milk from a plurality of teats on a live animal, comprising:
a movable apparatus adapted to receive instructions characterizing a location of at least one teat of said plurality of teats, to move to said location, to engage said at least one teat of said plurality of teats for extracting milk therefrom;
a light source for simultaneously illuminating an area encompassing said plurality of teats;
a two-dimensional array camera having a two-dimensional array having a plurality of pixels that are capable of individually providing time of flight data; and
electronic means for capturing an output of said two-dimensional array and for delivering data characterizing said location, wherein the two-dimensional array camera is configured to image the milking cups and the plurality of teats in one field of view.
2. Automated milking system according to claim 1, wherein the two-dimensional array camera is configured to image milking cup openings of the milking cups and the plurality of teats in the same image, so that a location of the plurality of teats becomes a relative measurement in relation to the camera.
3. Automated milking system according to claim 1 or 2, wherein the two-dimensional array camera is configured to image hind legs of the animal in the one field of view, so that a movement of the live animal is tracked.
4. Automated milking system according to claim 3, wherein a location of the plurality of teats and a location of the hind legs at various distances and depths are determined.
5. A method for automatically milking at least one teat of a plurality of teats of a live animal by means of a milking system having a milking apparatus having milking cups, comprising the steps of:
providing movable apparatus adapted to receive instructions characterizing a location of said at least one teat, to move to said location and to engage said at least one teat for extracting milk therefrom;
using a light source to simultaneously illuminate an area encompassing said plurality of teats;
using a two-dimensional array camera having a plurality of pixels that are capable of individually providing time of flight data to capture an image of said at least one teat;
processing said image to provide three dimensional information characterizing said location;
using said information to provide said instructions; and
moving said apparatus to engage said at least one teat, wherein the two-dimensional array camera is configured to image the milking cups and the plurality of teats in one field of view.
6. Method according to claim 5, wherein the two-dimensional array camera is configured to image milking cup openings of the milking cups and the plurality of teats in the same image, so that a location of the plurality of teats becomes a relative measurement in relation to the camera.
7. Method according to claim 5 or 6 wherein the two-dimensional array camera is configured to image hind legs of the animal in the one field of view, so that a movement of the live animal is tracked.
8. Method according to claim 7, wherein the location of the plurality of teats and a location of the hind legs at various distances and depths are determined.
[38] In the R-PR, the Applicant stated that the proposed claims are supported by the application as originally filed and the Panel has confirmed that this is the case.
[39] The Applicant also provided arguments in favour of the non-obviousness of the proposed claims, stating at pages 5-6:
In accordance with the independent claims, a time-of-flight (TOF) camera is provided to acquire both the milking cups and the teats simultaneously in the same field of view. In this respect, the independent patent claims specify: …the two dimensional array camera is configured to image the milking cups and the teats in one field of view.
From this, a person of average skill in the art would readily conclude that the TOF camera according to the invention simultaneously acquires the teats and the milking cups. In other words: the TOF camera acquires the teats and the milking cups at the same time and in the same field of view.
…
In this respect, reference is also made to the disclosure of the original description (page 6, lines 22-26), where the following is stated:
As the camera can image the milking cup openings and the teats in the same image, the location of the teats becomes a relative measurement in relation to the camera, further simplifying the attachment process, as no absolute position data necessarily need be exchanged between the sensor and the robot controller.
From this, the person skilled in the art gathers – unambiguously and without doubt – that absolute position data of the milking cups and the teats need not necessarily be forwarded to the control of the milking robot because the TOF camera also allows a difference measurement or a measurement of the spatial distance difference between the respective teat and the corresponding milking cup to be made. Here, this is referred to as relative measurement in relation to the camera. Therefore, the milking robot no longer needs to drive to an absolute position of the teat in space, but rather it can be configured to reduce a distance difference between the respective teat and the associated milking cup until the milking cup has reached the teat.
Further, from this, the person skilled in the art gathers without doubt that the TOF camera acquires both the absolute spatial position of the respective teat and the absolute spatial position of the associated milking cup in relation to a reference point (in this case the TOF camera itself) and establishes a distance difference in the three spatial dimensions between these. An acquisition of a distance difference or a relative measurement by means of the TOF camera is only possible by forming the difference between the absolute spatial positions, but these need not necessarily be forwarded to the milking robot.
[40] The R-PR continues, at pages 8-9:
Therefore, a person skilled in the art gathers without doubt from the subject matter of the present invention that the TOF camera according to the invention is configured and provided for establishing the distances to the teats and to the milking cups at the same time and in one field of view. Such an establishment or identification of the distance between the camera and the milking cups can likewise not be gathered from the cited documents.
The distinguishing features have the technical effect that identification at the same time, or a simultaneous identification, of the three-dimensional position of both the teats and the milking cups is made possible. This allows quick and particularly accurate application of the individual milking cups to the respective teats. Moreover, exact determination of the position of the milking cups and alignment of the milking cups in respect of the respective teats is possible here, both during the application and during the milking.
In the known embodiments of prior art milking robots, the position of the milking cups was established in each case by means of mechanical sensors, wherein relatively small deviations in the position and alignment of the milking cup in this case cannot be established as exactly as is possible by means of the TOF camera according to the invention. In particular, the TOF camera according to the invention also renders it possible to identify an unwanted alignment of a milking cup in the milking robot as well as during the application process, and to forward appropriate information to the milking robot.
Objective problem
Therefore, the invention is based on the objective problem of enabling an application of the milking cups on the teats which is as quick and as accurate as possible.
The invention solves this problem by virtue of the fact that the three-dimensional position data of all teats and of all milking cups are acquirable simultaneously by means of the TOF camera according to the invention.
Solution to the problem
The cited documents cannot be combined to arrive at the claimed invention without involving an inventive step. In this respect, reference is made to the explanations above. None of the documents disclosing a TOF camera simultaneously provide a person skilled in the art with an indication that this measurement process is suitable for the application in a milking robot. When seeking for a possible solution to the objective technical problem, a person skilled in the art may find documents D2 and D9 [D4] but he would not consider them because these documents originate from a different technical field and provide no indication in respect of milking robots or milking machines.
Further, neither D2 nor D9 [D4] faces a problem where the simultaneous identification of a plurality of objects (teats and milking cups) is to be made possible, wherein these objects are to be connected to one another as quickly and as accurately as possible.
Analysis of the proposed claims using the Sanofi framework
(1): Person skilled in the art and common general knowledge of that person
[41] The identification of the person skilled in the art and the common general knowledge possessed by this person is the same as set out in step (1) with respect to the claims on file.
(2): Inventive concept of the claims
[42] The inventive concept of independent claims 1 and 5 of the proposed claim set is to improve automated milking systems by enabling an application of milking cups on the teats of an animal that is quicker and more accurate than was possible with previously-existing systems, the improvement comprising:
(i) using a two-dimensional array camera having a plurality of pixels that are capable of individually providing time of flight data; and
(ii) the two-dimensional array camera being configured to image the milking cups and the plurality of teats in one field of view.
(3): Differences between “state of the art” and inventive concept of the claims
[43] The differences between D1 and D3, considered separately, and the inventive concept of proposed claims 1 and 5 are:
(a) the inventive concept uses a time of flight data optical system having pixel array cameras to determine the location of the teats in three dimensions, whereas D1 uses a stereoscopic vision system, and D3 uses a scanning laser system, to accomplish this function; and
(b) the inventive concept includes the two-dimensional array camera being configured to image the milking cups and the plurality of teats in one field of view.
(4): Do the differences constitute steps that would have been obvious?
[44] Difference (a) is the same as that identified with respect to claims 1-3 on file. Accordingly, for the reasons provided above at para [35], this difference constitutes a step that would have been obvious to the person skilled in the art on the relevant date.
[45] However, the Panel is of the view that difference (b) constitutes a step that would not have been obvious to the skilled person on the claim date. We agree with the Applicant that neither D1 nor D3 indicate that both the milking cups and the teats lie in the field of view of one of the cameras. We further agree with the Applicant that neither D2 nor D4 faces a problem where the simultaneous identification of a plurality of objects (teats and milking cups) is to be made possible, wherein these objects are to be connected as quickly and as accurately as possible. While D2 and D4 could have been adapted to achieve the solution to the problem by implementing a configuration as defined by the proposed claims, there is no direction in either D2 or D4 to do so. Accordingly, we conclude that proposed claims 1 and 5 would not have been obvious on the claim date to a person skilled in the art. It follows that proposed dependent claims 2-4 and 6-8, which are narrower in scope than independent claims 1 and 5, respectively, would also have been unobvious.
[46] In summary, we conclude that the proposed claims 1-8 would not have been obvious and are therefore compliant with section 28.3 of the Patent Act.
CONCLUSIONS AND RECOMMENDATION OF THE BOARD
[47] The Panel concludes that the subject-matter of claims 1-3 on file would have been obvious to the person skilled in the art on the claim date, contrary to section 28.3 of the Patent Act. We also conclude that proposed claims 1-8 as submitted in the letter of February 15, 2019 overcome this defect and do not introduce any new defects. We therefore recommend that the Applicant be notified, in accordance with subsection 30(6.3) of the Patent Rules, that the deletion of claims 1-3 on file and the insertion of claims 1-8 as proposed in the letter of February 15, 2019 are “necessary” for compliance with the Patent Act.
Paul Fitzner Ed MacLaurin Mark Janczarski
Member Member Member
DECISION OF THE COMMISSIONER
[48] I concur with the conclusions and recommendation of the Patent Appeal Board. In accordance with subsection 30(6.3) of the Patent Rules, I hereby notify the Applicant that the following amendments, and only these amendments, must be made in accordance with paragraph 31(b) of the Patent Rules within three (3) months of the date of this decision, failing which I intend to refuse the application:
i) delete claims 1-3 on file; and
ii) insert claims 1-8 as proposed in the response to the preliminary review letter dated February 15, 2019.
Johanne Bélisle
Commissioner of Patents
Dated at Gatineau, Quebec,
this 26th day of April, 2019