Canadian legal & tech people are very keen on Ross, a software system built on top of IBM's Watson that's designed for law. One of the co-founders of the Ross team* describes it as "[b]asically, what we built is a [sic] the best legal researcher available". I wish this was true but I'm very skeptical.
Watson is a question-answering computer system that was designed to win at Jeopardy (it beat the previous champion, Ken Jennings) but has since been expanded to tackle more general question-answering. IBM promises that it's a "new era of computing" but the area it's operating in, "natural language processing", is a hot space with companies like Facebook pouring resources into their own solutions. It's an area that has been of interest to researchers for more than half a century yet applications that use it are relatively rare. Has IBM made enormous breakthroughs that will soon change the way we interact with computers?
To take this blog post back to the first sentence: can Watson be applied to legal reasoning in a way that puts today's articling students to shame (as the Ross team promises)? I'd be delighted to be wrong but I'd be surprised if that's true having worked as a legal researcher (articling student at McCarthy Tetrault) and programmed legal-related software systems (e.g. FederalMonitor.ca). There are several reasons why I'm skeptical of this claim (and the near-term promise - even IBM uses forward-looking statements about Watson):
1. Legal research is a very hard language processing task. Legal decisions are not just unstructured, they are filled with vague statements and fuzzy rules. Teasing the applicable rules out of the jurisprudence is not like figuring out facts for Jeopardy. Unlike in Jeopardy, the right answer in law is often "I don't know".
2. Judges love to pretend that their decision is consistent with jurisprudence (because they have to) but they also like making decisions that seem fair in the circumstances. This is a recipe for confusing and conflicting rules. There are sometimes one or more strands of thought about a given issue that are only resolved years later by a higher court (if at all).
3. Canadian legal decisions are controlled by three publishers and even IBM might have a hard time gaining access to them (and a team of U of T students definitely will). Watson-like systems only work when they have access to enormous volumes of documents. (The Ross team says the use "the entire body of law" without further explanation.)
4. IBM's revenue has been stagnant/falling for years. IBM's CEO recently said that they need to "reinvent ourself like we've done in prior generations". They're desperate to show investors that they have a plan to increase revenues. It makes sense for IBM to hype up long-shot research projects like Watson.
5. Will any lawyer want to advise their client based on what Watson/Ross says? I can see it as a good starting place for research but the idea of replacing lawyers with Watson? We're just not there yet and I doubt the Law Society would be very impressed with the lawyer who thinks we are (a different problem but important for Watson/Ross).
6. There's not a lot of information on the Ross website. Big claims mean big proof and I'd expect a lot more details from the team that claims to have built a pure-software "super intelligent attorney".
I'm a big proponent of adding technology to the practice of law but I think the claims about Watson are premature. It's far too early to say if Watson will replace legal researchers/junior associates and I don't think it's realistic that this will happen even in the next few years. Furthermore, if there was such a technology it would be first applied to far more lucrative problems that don't have the thorny vagueness of our legal system.
*I love what the Ross team is working on and it's an important problem. I use "Ross" in this blog post mostly as a stand-in for applying Watson to law. I haven't met the team and based this article on what I've read/heard from other lawyers. I hope the Ross team will write a bit more about what they're up to and how their solution addresses the problems I've highlighted in this post.