Last week, I posted on the issue of whether law schools should be teaching students how to use tools like ChatGPT. After I posted this, James Lau, well known legal tech author, and former Chief Legal Officer, pointed out to me that Open AI, GPT-4 Technical Report, 14 March 2023, states, “In particular, our usage policies prohibit the use of our models and products in the contexts of high risk government decision making (e.g., law enforcement, criminal justice, migration, and asylum), or for offering legal or health advice.” (page 6)
.
I was generally aware of this prohibition (or disclaimer, depending on your point of view) but failed to mention it in my post. The problem is that I used the term ChatGPT like we often use the term Kleenex. Kleenex is the name of a brand of tissue paper, but there are other brands of tissue paper as well. But we often use the word Kleenex when we really mean tissue paper.
I was guilty of using the term ChatGPT similarly. ChatGPT is the public open generative AI tool developed by OpenAI. It searches data from all public areas to provide its answers. ChatGPT is a sort of “brand” of generative AI. But it is not the only one. And other generative AI tools work differently and, importantly, use different and more limited data. Casetext’s product, CoCounsel, for example, is an AI tool tailored for use by lawyers. It thus promises to be more accurate than ChatGPT, in part because the data it uses is more limited to legal sources.
Not only are there other stand alone generative AI products being developed, but developers are applying OpenAI’s product to more limited data sets to improve its accuracy and reduce its errors (hallucinations). This allows the AI product to be more focused on the needs of particular users.
I fear many lawyers and legal professionals hear generative AI and think that ChatGPT is the only product
All well and good, except I fear many lawyers and legal professionals hear generative AI and think that ChatGPT is the only product. The one they should use for any and all purposes. This week at LegalWeek 2023, the results of a survey done by LexisNexis were released. (The full Report will come soon). These results suggest that there may be uninformed lawyers who believe that the hyped ChatGPT is something they should be using. And they may unknowingly rush to use it.
The Survey was based on the responses of over 4,100 lawyers, law students, and consumers (1,176 were lawyers). According to the Survey, 86% of the lawyers said they were aware of generative AI such as ChatGPT. 51% had either already used it or were planning to. 84% believe it will increase lawyers’ efficiency. It’s hard to believe that many lawyers have taken the time to understand how generative AI works and the differences among the various products.
Since ChatGPT is the generative tool that has received the most publicity, many of the respondents may be doing what I did. Using and thinking of ChatGPT as shorthand for generative AI without realizing that there are now and will be various generative AI programs. In a rush to use what everyone seems think is a great tool, the temptation may be for lawyers to jump to the public ChartGPT. But that public model is not exactly the best for what lawyers want to achieve. The error rate and hallucinations for ChatGPT are still high and, as the developers said, can’t be relied on where accuracy is essential.
Another problem with using ChatGBT without understanding it: lawyers need to know when and how to use it and other generative AI tools. As the OpenAI developers noted, ChatGPT should not be relied on for “legal advice.” As lawyers, we can better see the line between helpful background information and pure legal advice. But that line, even to us, is blurry. And I’m not sure whether our understanding as lawyers of what is or isn’t legal advice may differ from those who are not lawyers. Like the ChatGBT developers. And as Lau astutely pointed out to me, lawyers might need to consider whether their malpractice insurance extends to the use of ChatGPT, particularly given the position of OpenAI on legal use.
And, of course, there are confidentiality issues with using a public generative tool like ChatGPT. ChatGBT looks at all available information to formulate responses. If a lawyer supplies confidential information in asking ChatGPT a question, that material is no longer confidential. It can be seen and used by others who also use ChatGPT. By providing client information to ChatGPT, a lawyer may have breached confidentiality obligations. And may waive the privilege.
At the very least, lawyers contemplating using any generative AI tool need a baseline understanding of the tool.
At the very least, lawyers contemplating using any generative AI tool need a baseline understanding of the tool. They need to know what it does, and what data it’s looking at. At an opening panel this week at LegalWeek, a Panel of experts in a session entitled, Reshaping the Legal Profession: Thriving in the Age of Generative AI & ChatGPT, identified several things a lawyer should know in using generative AI:
Can the results be trusted?
Can the system explain how the result was reached?
What are the impacts of interactions between the human with the tool?
Who are the major players in the space (and how different)?
What has the experience with the tool been?
Bottom line, the need for lawyers to understand the risks and benefits of all technology has never been greater. This need is especialy true for generative AI. And the role of law schools in facilitating that recognition has also never been higher. Law schools need to ensure that students know the need to assess benefits and risks. To understand what the program is looking at, its bias, and its accuracy. They need to be sure lawyers know how to assess the validity of what the generative AI tools are telling them. And learn how to effectively and accurately use the tools.
So if I misled anyone about what ChatGPT can do, I am sorry. But I stand by my original thesis. Law schools have a responsibility to teach students about the need to understand risks and benefits of technology. And help them know what to do with tech tools like generative AI.