Navigating Generative AI in Legal Practice: Benefits, Pitfalls, and Best Practices

A: This depends on how you filter the results. AI generating hallucinations is a daily occurrence for most lawyers—but they catch it. When they don’t, most are then caught on review by co-counsel or a cite-checker. There are also instances when they’re caught by an opposing counsel and managed privately. But on occasion, they do slip through the cracks and survive undetected. However, this is likely not a tremendous number of cases.

AI models are more likely to hallucinate issues or arguments that are complicated or harder to make—which makes them critical for a case. Therefore, they’re likely to be reviewed by the other party or the judge. Mundane, uncontested points of procedure might be less likely to lead to hallucinations and wouldn’t command the same attention or scrutiny because they’re not as important to the outcome.

A: A mix of all this. The main error modes right now are:

(i) stupid mistakes born of trusting AI (and/or legal AI) tools too much; or
(ii) pure recklessness from “bad actors” (e.g., vexatious litigants).
This is less of an issue for big firms, because they’re likely to have people checking the cites and motions before filing. However, there are instances when there is a lot of time or financial pressure, which can lead to diluted responsibility (i.e. many people working on the same brief), and that can allow cases to slip through the cracks.

A: Misrepresented holdings might be the main issue. First, they are harder to identify. Second, they might overlap with the normal and expected “stretching” of a case by a lawyer.

A: My main concern is that we are breaking the chain of authority between the legal norm and its application. A court misrepresenting a precedent by error is still legitimate (though wrong). Appeal is there to remedy the error, but there are responsible (and accountable) humans all the way from the legislator to the norm-applier. AI just makes things out of whole cloth, and that’s harder to deal with.

A: That study was great, but it became rapidly outdated—tools have become better at mitigating hallucinations, but sometimes at the price of curtailing answers or staying on safe grounds. On the other hand, as I have catalogued, all LegalAI vendors have moved from “hallucination-free” claims to “you are in charge of verifying”.

A: Loads. Mostly that automating cite-checking is hard, and not necessarily in terms of cognition / understanding by the AI (though this is also a limit for some citations). But simply locating a source in the first place is very hard, and this is a task that’s supplemented by a lot of intuition from human lawyers. Also that there are a lot of citation errors or mistakes (or undue “stretch” of authorities) in the past as well. We were far from perfect, but at least in the past you could see what people were aiming at—there was a target citation, not a phantom.

A: False confidence is indeed a big issue. We have seen one example in the database of someone checking AI for the 99th time and finding no issue, only to trip the one time they did not check. More generally, the underlying issue (of automating drafting and searching) means that one does not really master the file and the arguments, and can be surprised by the gaps therein.

“Verifying thus becomes the critical last opportunity to familiarise oneself with what one is intent on arguing.”

A: Always know who wrote what, and what AI was used for. Pass anything through a cite-checker (like PelAIkan) to automate the boring parts (e.g., finding the source or the pinpoint).

Distinguish between the levels of risk: If your case hinges on one particular authority, be sure to double/triple-check it (as you would have done pre-AI, anyway).

A: I always insist that people actually learn how AI actually works. There are loads of tutorials or videos online, and it’s literally only a few minutes of your time, but so few people do it. I am not an absolutist of “knowing how things work”, but as long as a technology is immature (as AI is), understanding the technical side does most of the work in preparing you to use it.

A: That will become a permanent risk given the current paradigm of how this technology works. And, although it can be very annoying when you are the one spending time looking for a fake citation, it’s relatively fine—there are worse things in life. It can also shed light on “bad actors” and provide a reason to sanction them. Perhaps this will incentivise people to build better legal datasets that help with cite-checking.

A: I was teaching property law when the first ChatGPT model came out. I ran a query on [classic 1805 legal case] Pierson v. Post to get a brief. The brief was terrible, and I told my students that, clearly, there was nothing to worry about with AI.

But then, about a month later, the summary had gotten much better. The AI models have gotten exponentially better, so I began to incorporate them into my clinical teaching with LexisNexis Protégé in the fall of last year. But it was really last spring when I started to rely more heavily on these products, and I told my students, “this is a tool that if you learn how to use it, it can be very useful for you. It has the guardrails of Lexis (we also looked at Westlaw and Bloomberg tools). It started becoming very useful when these large legal service providers began releasing tools that they stood behind.”

We incorporated it sporadically into the clinic to see how AI would approach an issue, or to improve the quality of work students had already done.

A: Last fall, I was teaching at NYU with a colleague, and we agreed that the AI models are so good at this point, and some students are using it so much, that we should incorporate it into the syllabus. 

The aim is to teach law students to be sophisticated users of this technology, which is very powerful and can really speed up your process, help you issue-spot, catch mistakes, and other tasks. BUT, the big concern is that we don’t want to use it in a way that would interfere with students developing their own legal judgment. 

The big question is: How does a beginner use it versus how does an expert use it, and these are two very different questions.

One general theme in the clinical context is to have students do their own thinking, and then have them bring in AI as part of the evaluation of their work product. Use your professors and fellow students, but then use AI to critique, evaluate, and improve your work product. 

From a teaching perspective, I think this makes a lot of sense. From an expert user perspective, I wouldn’t necessarily use that strategy; I might use a different strategy. 

For instance, if I’m really good at something—say I’m a really good real estate lawyer—I might let AI do a first draft of a contract based on precedents and term sheets that I would use as applicable to the deal. But I’d have to check the quality of the output because it’s a very different interaction between an expert who knows what a correct contract looks like and a student who really has no idea. 

If students perceive that ChatGPT is the expert, they’ll defer to its output, but that’s not good for their learning. 

Don’t assume the work product is accurate. Don’t give up on your legal judgment. 

If AI had 100% accuracy, we’d assume all the time that it’s accurate. If it was 20% accurate, we wouldn’t rely on it much. The issue is that it’s maybe 70-75% accurate, which means you’re committing malpractice if you don’t use your independent legal judgment to confirm what it’s telling you. We know this from the horrifically obvious cases we know from the law journals about people who’ve generated briefs with hallucinated cases. 

However, anything that’s less than 100% accurate—even though it looks clean and coherent—you’re being paid and expected as a lawyer to confirm that it’s consistent with your judgment. In my practice, I might query something like a very complicated regulatory question. The answer might look great, but when I go and double-check every source, I realize it’s not correct. It’s not hallucinated, but it’s just not reading it right. So, if you’re going to rely on this output without confirming that you agree that the analysis is correct, it’s a problem. 

As soon as you’re in any slightly complicated area of the law, there’s a very good chance that the AI output isn’t right. You will benefit from using the AI tool because it’s like having a second opinion about what it says, but to just defer to using the AI tool is committing malpractice.

A: In a sophisticated transactional practice, there’s going to be at least one other party who’s looking at it—there’s a buyer and a seller, there’s a buyer and a lender, etc. Sometimes there’s a third party like a title company. So, hopefully there’s going to be some process for catching errors generated on one or both sides, even though you don’t have the 3rd party that’s a judge like you would in a litigation context. 

I don’t know in the greater scheme of things, whether they’re more likely to get caught in litigation or not, but they’re more likely to be publicly announced in litigation. The judge will issue an order from the bench and say it was professional malfeasance.

In a transaction between a buyer and seller, it would be embarrassing but might not go beyond the parties—though hopefully the person who made the mistake will learn their lesson. It just might not come out if it isn’t discovered. 

These types of mistakes can happen when you’re not using AI, too. For example, using a boilerplate contract could have mistakes by setting forth the terms of the last deal without reflecting the terms of the new deal. Or, you might put a decimal in the wrong place, so instead of a $5 million deal, it becomes a $500,000 deal—things like that. AI can catch some of these errors. So, there is value in the AI; it cuts both directions. 

One of the risks is that in transactional practice, a mistake—or failing to pay enough attention—can result in a deal closing quickly, and then the mistake doesn’t turn up until there’s a dispute. For example, a mistake could be in the renewal clause of a lease or a provision that governs a dispute in a contract. The deal is done before the mistake is discovered.

A: Before you begin to trust it to do important or client-facing tasks, or handle confidential information, you want to have a good use of the tools. In other words, you don’t start by using AI first in your client work. 

There are so many ways AI can be helpful in your personal life—planning travel, answering questions about your retirement accounts—all sorts of things where you can play around and get a sense of its strengths and weaknesses. 

You want to be a sophisticated consumer of AI before you incorporate it into your legal practice. This is your first and foremost priority.

Second, you have professional obligations to uphold. You need to be certain that whatever tool you’re using offers the level of confidentiality that your bar requires and that’s required of you as a licensed attorney, and that your clients are comfortable with. 

This can be difficult because it can be hard to parse the privacy statements of the tools. Often, they’re subject to change, also. Just yesterday, Google sent me a notice that probably a billion other people in the world received saying our privacy requirements have changed. 

The other nuance to this is that some of these tools are wrappers around other products like ChatGPT or Claude. What people don’t realize is that the wrapper, like Harvey or Legora, or whatever it is, might have its privacy statement about how it uses your data, but you need to understand how the model it relies on—and most rely on multiple models—is also allowed to be using your data. 

ChatGPT might be doing insider trading on your client data; you have an obligation to make sure the data is being kept confidential. Otherwise, at a minimum, you could lose attorney-client privilege, and that would be devastating in litigation. So, that’s a major professional obligation. 

Every major provider is going to tell you they offer privacy protection. And while I’m not suggesting you assume the worst of them, you do need to be a sophisticated consumer as a lawyer. You need to know before using these products that they are protecting your client’s information in a way that’s consistent with your professional obligations. 

Also, some people get “used to” a particular AI tool—like, I played around a lot with ChatGPT because I used Harvey. But ChatGPT was giving me answers I didn’t like because they were inaccurate. They weren’t updated to recent events. So I evaluated the different AI products to determine what the strengths and weaknesses are of each. 

Perplexity, for example, is better at capturing recent events than ChatGPT. If I’m asking a question about financial markets or something timely, I’ll be more likely to begin with Perplexity because I think it will provide more accurate information. 

So, understand that each product has strengths and weaknesses and figure out what your needs are… that’s how you choose your product. For different tasks, I use different tools. 

This is not like magic. Different tools have different strengths, and you need to understand what’s good for you. 

The final point on this is that you need to figure out whether to use a general AI tool, or a specific legal AI tool. Lexis Protege is probably expensive for a solo practitioner, but there might be an affordable AI tool that works for your specific practice area. For instance, SpellBook is focused on transactional work. There’s also a company working on an AI tool for immigration lawyers. 

You have to ask yourself: Is there a bespoke legal AI tool that’s affordable and focused on your needs, that’s going to make more sense for you than purchasing one of the big ones, which is probably more expensive and maybe has more use cases than you actually need?