Beware of Bogus Patent Analytics: Forward-Citation Analysis Leads to False Conclusions about Significance of Client’s Patent

Patents: more than just data sources.
Patents: more than just data sources.

Patent application filing and issuance data can be a useful tool to extract valuable competitive business information that is “hiding in plain sight.”  For example, in industries where patents are viewed as pertinent for creating and protecting long term value, patent filing data can present a strong signal about where your competitors are investing their time and money in innovation that may result in their future product or technology offerings.  In another example, such data analysis, also known as “patent analytics” or “patent landscaping,” can provide useful information about potential new markets for your company’s technology.  In this regard, for example, a chemical manufacturer can review how others are utilizing their products by reviewing patent filings.  For patent owners, analytics can reveal whether infringement may be occurring or whether it might have a higher value using forward citation analysis, which is a review of how many times a patent is cited in the later record of other patents.

Various flavors of patent analytics are offered by any number of companies today.  As a corporate IP attorney, I purchased such products, which certainly do not come cheap.  In the last couple of years, I have also evaluated a number of these products to see whether they can provide value to my IP Strategy clients.  No product has satisfied my– admittedly high–standards for providing useful information.  My main beef with all “canned” patent analytics products is that each looks at patent documents as data sources first and foremost, with the result that each product effectively ignores the fact that embedded with legal significance.  Moreover, in most cases, the people who create the algorithms forming the basis upon which the data is filtered for analysis do not take into consideration the underlying legal context of the data.

A great example of the erroneous or “bogus” conclusions that can result by not understanding the legal context of patent filing data is a forward citation analysis I conducted for a client recently.  This client, who owns a patent that predates other patents of the similar subject matter by several years, asked me to perform a detailed analysis of what patents cited his patent.  My client’s assumption was that those patents that cited his patent were likely close in subject matter to that of his patent and could possibly indicate infringement or potential purchasers or licensees of his rights.  Conceptually, this makes sense, right?–citation frequency logically should indicate the “influence” of my client’s patent.  Not in this case.

First, the good news:  my client’s patent was cited in 135 later-issued patents.  This should ostensibly signal either or both of:  a) that the Patent Office considered this patent to constitute a “pioneering” invention in the subject matter; or b) that later filing patentees identified my client’s patent as relevant at some time.  Typical patent analytics products based upon the supposed significance of forward citations would present these 135 patent citations as conclusive to the data collection and analysis process.  And, certainly, the 135 citations for a single patent would appear to be signal that my client owned something of value.  The bad news is that the forward citation analysis was widely off the mark with respect to the significance of my client’s patent in the marketplace.  While my client’s patent is undoubtedly important and, arguably, can be considered “pioneering,” bare adherence to the forward citation data alone would lead to erroneous conclusions about what companies might be infringing or that might be potential licensees of the patent.

Specifically, the forward citation analysis showed that about 35% (i.e., 49) of the forward citations were owned by a single major credit card processor.  A second major credit card processor owned 11 additional forward citations.  Still further, a now-defunct Dot.com start-up owned another 11 patents.  So, 71 of the 135 total citations were owned by 3 companies. 

As someone who has managed large patent portfolios over the years, this forward citation data indicated to me not that my clients’ patent is necessarily “pioneering,” but that the law firms handling the prosecution for the credit card processors and Dot.com companies submitted my client’s patent in what we call an “omibus” information disclosure statement.  That is, the attorneys handling the respective portfolios automatically submitted my client’s patent to the US Patent Office without consideration of whether it was remotely relevant to the application at issue.  In other words, my client’s patent generated the majority of its forward citations as a result of administrative action, not because it was particularly relevant to the patent application into which the patent was being cited as of record.

This is not to say that my client’s patent is not an important patent in its technological area.  To the contrary, not trusting the forward citation analysis, I also conducted a robust boolean search of what other patents had been filed in related subject matters.  This search, which took considerable time to conduct and analyze, revealed that that my client’s patent predates other relevant patent filings as demonstrated by actual substantive review of pertinent filings.  However, few of the patent filings that came up in the substantive searches were duplicated in the forward citation analysis.   A deep dive into the respective data sets (that is, both disclosure and claim review) reinforced the conclusion that the forward citation analysis of my client’s patent resulted in largely irrelevant citations because the patents in which my client’s patent was cited were not relevant to the pertinent subject matter.  In short, most of the citations did not generate leads as to potential infringers or licensees of my client’s patent.

Notably, the forward citation analysis took only about 25% of the time that the robust boolean collection and analysis did, thus indicating to me why people view this to be a desirable way to conduct patent landscaping analysis.  Nonetheless, if my client obtained only this analysis, he would have generated erroneous conclusions and would have likely based business decisions based on this false data.  Once started done this wrong path, it is possible that my client would not be able to recover the lost time and money resulting from this erroneous data.

In summary, those seeking to generate insights from patent filing data must understand that patents filings comprise more than data points.  These documents include both legal significance and administrative context that must be understood and processed prior to making decisions based on a collection of disparate patent filings.  Directionally correct data is possible to obtain from patent documents, but prior to making business and legal decisions on such information, one needs to make sure that the data is based upon fact-based collection methodologies.

7 thoughts on “Beware of Bogus Patent Analytics: Forward-Citation Analysis Leads to False Conclusions about Significance of Client’s Patent

  1. A major problem with forward citation analysis is that is assumes several factors which may be true for some technology areas but not for others. The first assumption is that the inventors filing after a foundational patent are aware of the foundational patent. This may not be true as occasionally there are inventors far ‘ahead of their time’ so to speak. So far ahead that no one knows who they are. I imagine some of it comes down to marketing as well.

    The second assumption is that the examiner is fully aware of his/her technology class. Based on some of the office actions I receive, this is not always true.

  2. We have also observed that applicant-supplied prior art can often be of low relevance. In an empirical study (forthcoming) we have confirmed the hypothesis that US patent citations are less ‘relevant’, on average, than citations from other offices (that do not report applicant citations). Other patent offices follow more stringent rules and practices. Citations from other patent offices are therefore more meaningful, on average. Currently the average relative relevance of US citations compared to other offices is approximately 25%. Interesting literature is Bregonje, 2005 (World Patent Information) and Burke, 2007 (Research Policy).

    Patent citation analysis on the level of individual patents should always be interpreted with great care. Furthermore, it is useful to take into account the ‘quality’ of these citations and to use worldwide data.

  3. I believe that this is an issue with all fully automated analyses , they can give rise to misleading results. However, I have been involved in projects where this automated forward citation information has been of great value, evidenced by a subsequent very successful out-licensing program. Interpretation of results is all important and as with statistics you shouldn’t automatically believe what the analytics appear to show

  4. I agree with Jackie to a certain extent. It is important to understand how this kind of analysis is used. Directly basing business or strategy decisions on this can create a problem. But, if there is a large set of data and one is struggling to understand where to start, this could be really useful. One of my clients had a large patent portfolio which they wanted to rationalize/optimize and also evaluate out-licensing opportunity. Manually looking at each patent would have consumed lot of time and would have been expensive. Citation analysis helped our client create different clusters and prioritize them so that the client can pick clusters in order of importance and devote time and effort for detailed analysis to reach business or strategy decisions.

    Citation analysis should always be used as a tool to prioritize your time and effort towards evaluating your portfolio. The business or strategy decisions should be taken after detailed analysis.

  5. The shortcomings of the methodology adopted for performing a forward citation analysis results in a illogical inference. Citation analysis should be typically done for a single patent and not for a portfolio as a whole. Relying on automated quantitative analysis will never help when it comes to citation analysis. There are lot of factors that have to be considered. Some of them include no. of non-self citations, same patent family members citing multiple times, whether the citing patents are directed towards incremental improvements or complimentary technologies, whether the citing patents are directed towards the same application or a different area and so on…A lot of time we fall for automated tools in order to save on time & money. The inference derived from a citation analysis should be looked into from a learning perspective and mere indicators when it comes strategy decisions. Devising the strategy purely based on the analysis will not help. I have tried to capture some of my views here >> http://www.slideshare.net/Rakesh_Analyst/quantitative-vs-qualitative-patent-evaluation?from=ss_embed

  6. The shortcomings of the methodology adopted for performing a forward citation analysis results in a illogical inference. Citation analysis should be typically done for a single patent and not for a portfolio as a whole. Relying on automated quantitative analysis will never help when it comes to citation analysis. There are lot of factors that have to be considered. Some of them include no. of non-self citations, same patent family members citing multiple times, whether the citing patents are directed towards incremental improvements or complimentary technologies, whether the citing patents are directed towards the same application or a different area and so on…A lot of time we fall for automated tools in order to save on time & money. The inference derived from a citation analysis should be looked into from a learning perspective and mere indicators when it comes strategy decisions. Devising the strategy purely based on the analysis will not help.

Comments are closed.