Artificial Intelligence v. Human Curation in Legal Research Platforms

Watch The Webinar

A Review of Feit Consulting’s recent study entitled “Patent Litigation Research: The Importance of Precision”

Docket Navigator recently engaged Feit Consulting to conduct an unbiased study comparing two patent litigation research tools – Docket Navigator – and a competing tool that relies heavily on AI to process legal data. The results revealed strengths and weaknesses in several areas such as user-friendliness, comprehensiveness, and accuracy. With any platform, the ability to produce accurate results depends in large part on the expertise of the user. However, Feit’s study revealed that regardless of a user’s research prowess with these tools, the secret to finding comprehensive and accurate results hinges on the architecture of the database itself; how the data points are being collected, and how they are connected to other related data points.

The following review highlights the inner-workings of Docket Navigator’s platform to answer one main question: Why is Docket Navigator able to answer real-word legal research questions, while other platforms are not?

Claim Construction

While it would be imprudent to state that artificial intelligence has no place organizing legal data, claim constructions are one area in which AI has a long way to go. The reason that Docket Navigator can return every construed claim term, along with the patent number, court’s construction, case name/number, etc., is simple – it was designed to do so.

At Docket Navigator, human beings read through every Markman order, locate the terms being construed, and enter those terms into a specially designed interface.

At Docket Navigator, human beings read through every Markman order, locate the terms being construed, and enter those terms into a specially designed interface. Those humans also type in the court’s construction and the patent number containing the disputed terms. All of these data points are then connected to the case, judge, court and document in which the terms were construed.

So, rather than expecting researchers to rely on text-searching documents for construed terms, which returns more noise than useful data, Docket Navigator users can click the Claim Constructions category on the main search page, enter a specific claim term and receive a list of every construction of that term going back to the year 2000. This is why the answer to the question, “How many times has the claim term ‘individual’ been construed, and what were those constructions?” is so easy to find with Docket Navigator.

Download White Paper

Rulings on Specific Motions, Based on Specific Legal Issues 

Artificial intelligence has a much better chance of finding specific rulings, even rulings that rely on specific legal concepts as a basis for the decision. The problem though is the degree of granularity that AI can provide. For example, it is easy for a machine to distinguish if a court order is ruling on a motion for summary judgment. Where it falters though is differentiating the various types of MSJs being ruled on. Is it an MSJ for infringement, or for non-infringement? A human being can make that distinction easily, which is how Docket Navigator is able to track those and hundreds of other motion types comprehensively.

As with Markman orders, at Docket Navigator a human being reads every MSJ ruling that occurs in patent cases, enters the specific type of MSJ, the result of the ruling, the legal concepts used as a basis for the ruling, the judge that signed it, etc. Because each of these data fields are entered separately, but linked together, users can click on the Documents search, select any type of MSJ ruling (Docket Navigator tracks over a dozen types) and find all such rulings in a few seconds by a specific court, judge or even in cases involving a specific firm or attorney. Entering the data at this fine level of granularity is what makes answering the question “How many U.S. district court decisions have addressed motions for summary judgment asserting patent invalidity based on lack of patentable subject matter under 35 USC § 101?” a snap with Docket Navigator.

Postures of Motions

Another failure of artificial intelligence when it comes to processing legal documents is its inability to discern the “posture” of a motion, meaning the way a motion is teed up to the court. Is the motion contested? Stipulated? Sua sponte? Renewed? The posture of a motion has a great impact on the court’s willingness to grant or deny it.

The posture of a motion has a great impact on the court’s willingness to grant or deny it.

For example, contested motions to stay pending an inter partes review are granted only 43% of the time. However, stipulated motions of the same type are granted a whopping 94% of the time. When advising clients on a judge’s track record for a specific type of motion, it only makes sense to compare apples to apples. If your motion is contested, it would not make sense to rely on historical data based on stipulated motions. The ability to exclude or include specific postures in search results is vital when building a strategy.

In the Docket Navigator platform, court rulings are diagrammed with several different components, but the four main components are (i) the posture (how the motion was presented), (ii) the type of motion (the procedural tool used to request relief), (iii) the result (how the court ruled on the motion), and (iv) legal issues (the legal concepts addressed in the court’s ruling). Because those exist as four different data points, users can filter by just one of them, or use any combination of the four. This model is called a “Document Filter Group,” and this is how it appears in the user interface, using the contested motion to stay pending IPR as an example:

By designing the data in this way, questions such as “How many rulings on original, contested motions to stay pending IPR have there been?” take only a few seconds to answer, comprehensively, in the Docket Navigator platform.

Accusations 

Perhaps the most difficult question for machines to answer when it comes to a patent case is “Who owns the patent being asserted?”  Many platforms assume that it’s always the plaintiff, but of course that is not the case with declaratory judgment cases.  And what happens when there are claims and counterclaims with both plaintiff and defendant asserting patents against each other?  This is why Docket Navigator created Patent Accusations. 

Perhaps the most difficult question for machines to answer when it comes to a patent case is “Who owns the patent being asserted?”

In a Patent Case, an Accusation is a request for relief, the resolution of which could determine if a patent has been infringed or the patent’s validity or enforceability.   

Docket Navigator models Patent Accusations by using three main components, (i) the party that owns the patent at issue, (ii) the patent number, and (iii) the party challenging the patent at issue.  The combination of those three components form one Patent Accusation. 

For example, a case with one plaintiff asserting one patent against one defendant would involve one Accusation. A case with one plaintiff asserting 5 patents against 10 defendants would result in 50 Accusations. 

Here is an example of some common Accusation scenarios: 

Artboard 1DefendantDefendantDefendantPatentPatentPatentPlaintiff1 Case4 Litigants9 Accusations=DefendantDefendantDefendantPatentPlaintiff1 Case4 Litigants3 Accusations=DefendantPatentPlaintiff1 Case2 Litigants1 Accusations=

Entering parties and patents based on the model above is what allows Docket Navigator to answer the question “How many patents has Apple Inc. asserted in U.S. district court cases? 

Conclusion

Although artificial intelligence certainly has its place in the research and practice of law and should be used whenever possible, it’s important to differentiate where AI shines and where it fails.  In the majority of the research questions surveyed in Feit’s study, it is readily apparent that the human component in finding, understandingmodeling and organizing data is a critical part of producing accurate, comprehensive search results.  

Although artificial intelligence certainly has its place in the research and practice of law and should be used whenever possible, it’s important to differentiate where AI shines and where it fails.