In this episode of The Geek in Review, hosts Marlene Gebauer and Greg Lambert are joined by Carolyn Austin, Director of Practice Innovation at K&L Gates; Brendan McDonnell, Partner at K&L Gates; and Abdi Shayesteh, CEO of AltaClaro. The conversation delves into innovative legal education strategies and the integration of generative AI in legal practice, highlighting how experiential learning platforms are shaping the future of legal training.

Abdi introduces AltaClaro’s experiential learning platform, designed to make lawyers practice-ready through simulation-based training. Utilizing a “Learn, Do, Review” methodology, the platform presents associates with real-world assignments, provides feedback from seasoned practitioners, and facilitates live review sessions. This approach aims to build confidence, enhance critical thinking skills, and improve efficiency among legal professionals by immersing them in practical scenarios that mirror actual legal work.

Brendan discusses K&L Gates’ longstanding commitment to innovation and education within the legal industry. He reflects on the firm’s history of embracing technology—such as developing one of the first e-discovery platforms—and its proactive stance on integrating generative AI into legal workflows. Recognizing the transformative potential of AI, K&L Gates formed an AI Solutions Group to prepare attorneys and allied professionals for the technological advancements reshaping legal practice.

Carolyn explains how K&L Gates collaborated with AltaClaro to develop specialized training programs focused on generative AI, including prompt engineering and supervisory courses. These programs address the challenges partners and managers face when integrating AI tools into their workflows, emphasizing ethical considerations, risk assessment, and effective supervision. The partnership ensures that all firm members, from associates to senior partners, are equipped with the necessary skills to use AI responsibly and effectively.

The guests explore the broader implications of AI adoption in the legal field, discussing challenges such as change management, ethical obligations, and the need for data-driven practices. They highlight the importance of fostering a culture of continuous learning and collaboration to stay ahead in a rapidly evolving industry. Looking toward the future, they express optimism about AI’s potential to enhance legal services, improve access to justice, and drive innovation, emphasizing the critical role of education and adaptability in this transformation.

Listen on mobile platforms:  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Apple Podcasts⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ |  ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Spotify⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠YouTube⁠⁠⁠⁠⁠⁠⁠⁠⁠

Twitter: ⁠⁠⁠⁠⁠@gebauerm⁠⁠⁠⁠⁠, or ⁠⁠⁠⁠⁠@glambert

⁠⁠⁠⁠⁠Email: geekinreviewpodcast@gmail.com

Music: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Jerry David DeCicca⁠⁠⁠⁠⁠⁠⁠⁠

 

Transcript

Marlene Gebauer: Welcome to The Geek in Review, the podcast focused on innovative and creative ideas in the legal industry. I’m Marlene Gebauer

Greg Lambert: and I’m Greg Lambert.

Marlene Gebauer: So on today’s episode, we are joined by Carolyn Austin, Director of Practice Innovation at K&L Gates as well as Brendan McDonnell partner at K&L Gates. And joining Carolyn and Brendan is Abdi Shayesteh CEO at AltaClaro. Welcome all to the Geek in Review.

Abdi Shayesteh: Hello.

Carolyn Austin: Thank

Abdi Shayesteh: Thank you.

Greg Lambert: So Abdi, do you mind, before we dive deep into this conversation, can you give us a general overview of what AltaClaro does and what kind of unique value it brings when we talk about legal education?

Abdi Shayesteh: Sure. By the way, thank you, Marlene. Thank you, Greg, for having us. Really excited to have this conversation. Yeah, Altaclaro is an experiential learning platform, a simulation-based training platform for lawyers. All of our courses have been designed by practitioners, for practitioners, really with one goal in mind, and that’s to get legal professionals practice ready. And what we mean by that is that when anyone goes through any one of our courses, you’re going to learn by doing. You’re going to work on simulated assignments taken from real-world documents.

You’re work on them, you’re going to receive feedback on that from our vetted practitioners as well as some real-time feedback. But it’s all designed to give you that confidence, empower the associate or the legal professional to own their career trajectory, to have dress rehearsal, to practice and make mistakes, and as well as to make things efficient for the firm when it comes to training. You know, less time spent on training because the foundational teachings have been put in place through us and now the

firms can build on top of that, less time correcting mistakes, more efficient mentoring conversations. It all was something I wished I had. As I mentioned, Greg, you know, I started early in my career at King & Spalding. And early on, you realize you become a better practitioner through practice. More assignments, more feedback you get, the better you become. And the spectrum of assignments you get as well, the better you become. And even in those mentoring sessions, those senior lawyers lean in.

when you come in with good questions, when you know the terms, at least, and have some understanding of something. And so that was the impetus behind it, sitting in my office at 1185 Avenue of the Americas, look, 11 o’clock at night as a third year wish I had this thing. so.

Marlene Gebauer: You

Greg Lambert: I was gonna ask you what bad thing happened to us at King & Spalding that made you decide to really need to focus in on the training.

Brendan Gutierrez McDonnell: Thank

Marlene Gebauer: Hahaha.

Abdi Shayesteh: Well.

You know, I was very grateful that I had someone down the hall, a couple of people. You may remember Andy Metcalf, you know, a partner, really generous with their time to give me feedback, to spend some time with me. But I realized not everybody had that access and that sometimes, you know, because it’s just a matter of deal flow or economic environment you’re in or sometimes unconscious bias, same people get the same deals over and over again. And so people could be left out.

And so I got lucky to have that access, but I realized that not everybody had that, and that’s what was also the drive behind creating AltaClaro, to create that space.

Marlene Gebauer: So Abdi, you’ve just mentioned how AltaClaro’s experimental learning model has been a game changer for many lawyers. Can you share some specific examples of how the learn, do, review method has helped lawyers in real world scenarios, especially in relation to your prompt engineering and supervisory classes?

Abdi Shayesteh: Yeah, sure thing. And maybe just take a step back to explain what LearnDo review is.

Marlene Gebauer: Yes, please do.

Carolyn Austin: Thank you.

Abdi Shayesteh: Yeah, absolutely. So all of our courses and we have 60 of them that cut across a variety of practice areas, know, corporate capital markets, real estate lending, litigation, as well as professional development to gen AI training and even writing courses. But all of those courses have the same three key pillars, learn, do, review. the outset of a learner’s journey, they’re going to watch an hour’s worth of pre-recorded bite-sized videos. These are seven to

Carolyn Austin: Thanks for watching.

Abdi Shayesteh: to 10 minute segments. In between the segments, there’s set of assessments and small exercises to apply quickly what they’ve learned and get feedback. They do this self-paced, 10 minutes a day, 20 minutes every other day. But when they’re done with that, the next thing that happens

is the most important part. That’s the doing part. That’s when inside the portal, they’re now going to get a simulation assignment taken from real world documents, real world matters. There’s a fact pattern based from the real world. Here’s your client. They’re about to engage in this transaction. You’re the associate on the matter. Here are the issues. Here are the tasks. Even the documents you need to synthesize. So for deal lawyers, it might be an email from a client, a term sheet, a due diligence memo.

even a precedent form that’s imperfect from a past deal that’s in the office box. They got to go in, find some good provisions, come back and mark up their document. That’s that assignment self-paced. It shouldn’t take him more than two hours, but it’s going to be a challenge. It’s not going to be straightforward, but it’s good building critical thinking skill muscles. They work on that. They submit it in the portal and they get ready for the third part, the review, which is a pre-scheduled live review session.

And that’s when they come in to meet with one of our vetted practitioners to go over the assignment, ask questions. Instructor’s going to ask them questions, really to get to the whys and hows of what they need to do for that assignment. Now we run these in small groups of 10. So even when a firm has 100 participants, we’ll break them up into 10 cohorts of 10. And that’s because the instructors will review this. They’re going to see what people missed. They’re going to see what they got right and give them feedback.

in that live session based on the trends they saw. So associates would get feedback. The associates also get a model answer for them to reflect to see. So this learn, do, review really gives them that assignment they would normally see. They would get feedback on it. Then they get to talk about it in the sessions. It continues to build as you move. It’s not a one and done. There’s a variety of assignments that we build a curriculum for.

But associates have told us what this has done for them and other learners who have gone through our programs. It’s not going to make you a rock star. It will do three things. One, to be able to raise your hand with confidence to do that assignment. 98 % of those get delegated to you, but when you raise your hand to want to do something, that’s when you shine. So that’s what we focus on. Two, to have more context, especially to be able to ask the right questions. A lot of those assignments come in at 4 o’clock.

afternoon, next thing you know it’s 11 o’clock at night, you’re I should have asked these questions. So this really helps you formulate those questions so you spend less time. And that’s the third thing, more efficiency, spending less time spinning your wheels. Instead of 10 hours staring at the screen, what am I supposed to do with this? And where do I start? And now you know where to start. You’ve seen this before. You’re going to spend maybe four hours on it. So that’s the goal on the associate side. But it really empowers the firms to have better internal training sessions.

and mentoring sessions to build on top of it. They don’t have to start from ground zero to explain something. Folks come in prepared and becomes more engaging and more dynamic in those sessions. A lot of efficiencies created in the firm. Now, in the context of the prompt engineering course, same thing. know, one of the challenges we hear from firms across the AmLaw 200 is how do we get the attorneys into the tools to actually use them?

in their workflow and actually start to apply it in their use cases. our course is an immersive framework where you have to get in the tool in order to go through the course. And you’re going to, as you go through those exercises, pull up your tool, have small fact patterns that are like, know, something you would see as a lawyer or a legal professional, like draft and review a provision of confidentiality using the…

GPT or any other tool that your firm might be wanting to use. So they got to get in the tool, got to input, see what the output is, and now it says, well, what’s wrong with this? How can we make this better? Now let’s iterate. Let’s go back and keep prompting. So in the journey, you’re going to get small opportunities to exercise the types of things you would normally have to do. And then at the end, you’re going to get an assignment. And this is really where our partnership with K&L Gates has been invaluable because of the feedback we heard.

but how to tailor these assignments for different roles. So we have assignments for litigators, we have an assignment for transactional lawyers, and we have an assignment for allied legal professionals. And it’s gonna be a self-paced assignment, but they’re gonna have to use that tool to finish it. Dive into it, synthesize, use the tool, make mistakes, turn it in, and then they’re gonna get into that live session and go over it, and everybody’s had a different answer.

which is awesome because now you’re to talk about, I got this, I got that, and now it gets really interesting. So that in the context of prompt engineering and then again, because of KNO Gates, the feedback we heard is, we ought to have one for partners and supervisors, which brought us to the course that we just released. And that was, again, getting close to the use cases of how the tools are being used and where training needs to apply. we’re…

We’re going through situations and scenarios with these partners and managers, but how to design a workflow that will incorporate both human and gen AI tools and analysis. How to deal with errors that come up and to deal with the associate or the legal professional. How to even put into a workflow a gen AI tool you know, we know is imperfect, but how do you build that workflow to mitigate the imperfection? It’s all designed around getting the partners engaged to supervise.

and also design the workflow and deal with situations that come. And then at the end of that, they’re going to have to do an assignment, small assignment, but they’re going to have to create a best practices checklist on deploying a framework for supervising and managing GenAI tools for their legal staff, something they use right away for their practice. hopefully that was a good sort of example of how we get people in the water.

Greg Lambert: Yep, so Brendan, I don’t know if you could tell there’s a little bit of passion, I think, in what Avdi’s doing. But, you know, and I understand there at K&L Gates that, you know, there’s a big focus on the firm for innovation and education. Can you talk a little bit more about kind of that, where your…

Brendan Gutierrez McDonnell: Beep.

Greg Lambert: where your focus is on innovation and education across the board, and also how did that then lead you to working with OBDI?

Brendan Gutierrez McDonnell: Okay, great. I’ll start with, you the history has a long tradition of innovation and we are the Gates is part of the Gates, the Bill Gates that we all know. His dad’s law firm basically is our part of our firm now and that’s who we are. And going back in time when we were some significant litigation and you had tons of emails to go through, whether it was antitrust litigation or whatever.

we needed a program to actually go through and try to figure out what, you know, which emails were germane to like a discovery request. So we came up with one of the first e-discovery platforms called a 10X. So just by the nature of that client, it made us, you know, be creative, made us realize innovation was a huge part of who we had to be to service them. So that’s where it started. We eventually, you know, we…

We won a large patent case in 2016 with CMU, 2015 maybe. And in that, we talked to CMU about what could we do for them? And we agreed to sponsor, to fund a fellowship. It’s like more of a professorship in AI and ethics. And that seat still exists today. And there are a number of programs that are run out of that. So we’ve always believed that innovation and looking forward is extremely important for our clients and for our lawyers and allied professionals.

As time went by and was clear, we had a lot of changes over the last 20, 30 years in the legal profession, but nothing like Gen. AI. And as we saw that coming, and we saw it about 18 months ago, I went to the firm, Carolyn went to the firm, and somebody else, Chris Valente, who’s not here, went to the firm, together basically said, listen, something’s got to be done here. We’ve got to come up with a task force and really start to be prepared because we think this is a game changer.

much different than we’ve seen in the past. So that was sort of the genesis of how we began. And that became the AI Solutions Group. And our job was to figure out quickly, you know, what’s coming and what do we need to do about it? And it wasn’t long before we realized that this technology was significant and was a game changer. We also saw a lot out there about hallucinations and things that caused a lot of concern.

Lawyers by their nature are extremely critical. They’re thoughtful. They’re a little hesitant. And that’s good. That’s probably by what is required to be a great lawyer. And so we knew right off that they weren’t going to embrace a new technology that’s creating hallucinations and citing the wrong court cases. But we saw something much more than that. We saw a lot of development in the technologies and a lot of good solutions out there.

and some really incredible solutions. And so we realized that this is going to be the wave of the future. But the only way to get our people there, the only way to change a law firm, which is slow to change, as you know, I call it the race of the turtles. You have to be faster than the other turtle. But with that, we knew that education had to be, education and training had to be the cornerstone of the change. And, you know, we began to immerse ourselves in how do we do this?

And it brought us to AltaClara because we had a long time partnership. I want to say it goes back 10 years. OptiHealth, how does it go back? How long? Maybe not that long. Five years? Okay. I doubled it. It feels so long and great. That’s right. Yeah.

Abdi Shayesteh: 2019. First client though. K&L Gates was the first Amlo 100 client.

Greg Lambert: I’m

Brendan Gutierrez McDonnell: Yeah, so we brought in Abdi to do a training program for our corporate lawyers. you know, wasn’t the, the, the litigation wasn’t necessarily the impetus there. It was a real realization that sometimes you don’t do things as well as others and education and training as far as like, you know, what I call real formal program. It’s, it’s, there’s nothing like having pros. I mean, the, the learned, you know, do review and the way that Abdi has kind of come up with in his team has come up with this formula.

It’s just so well designed for lawyers and they do it so much better than we do it. And there’s things where we’re going to train internally, of course. mean, lawyers are good at training on some things. But when you really want to get something in depth, you really want to have a really good teacher and program that it’s super important to make sure that they walk away learning something. You really have to go into the pros, makes a lot of sense. And then we double down on that as soon as we saw that

that JNI was gonna be big, we went to OBDI. And I’m on the legal ethics committee for the Oregon State Bar. So I was very keen on the ethical rules and the duty to supervise. And as we quickly jumped into, we got into prompt engineering, that was the first part. And that was something where, we were just, really, everyone putting their toe in the water. And we had such a great experience with OBDI and this team, and they were so creative and so.

They received feedback so well that, you know, it was just a great partnership. But, you know, with the legal ethics rules coming out, we imagine that the supervisory role was huge because there’s a duty of competence. There’s a duty of confidentiality. There’s a duty to supervise. There’s a duty to charge a fair fee. All those things, all those concepts need to be brought to bear when using AI. And for our partners who are hesitant, we go back to that.

They’re careful. Having a program is super important to them. Having something where they can rely on, you know, an outsider that’s helping them through it and give them that confidence because these tools are only going to be used by people with confidence. It’s like it’s I hate to say it’s like a child. You watch a child when they first learn something, they’re slow to it and they kind of hesitate. But then they get in there, they go all in. And we kind of trying to create that same environment at K &L. So that’s why we did. And I’ll you know, you can I’ll let Carolyn probably get out of that, but I’ll let you.

I’ll let you run the interview, Greg. That’s your job.

Marlene Gebauer: I do have a question for Carolyn. So with AI becoming more prevalent in legal practice, what do you see as the challenge for partners and managers when it comes to integrating generative AI in their workflows? And how does this, the new, I think this is a new Gen I supervisory course that you have, how does that address those challenges?

Carolyn Austin: That’s a great question and thank you for having me Marlene. I’m going to answer that in two ways. I’m going to talk about the challenges and I’m also going to talk about the opportunities, so both sides of the coin, to build on what Brendan has said by way of introduction. But I think one of the biggest challenges that we saw and we tried to step into the shoes of our people and think about what they needed to know and how they might be feeling, which is really important. And one of the things that we thought was, wow, if I’m

managing, if I’m senior partner and I’m managing a team of associates and allied professionals, and I’m looking at introducing generative AI into my practice because I understand that there will be opportunities here and I want to be able to improve my client service, I’m going to need to understand how I do that securely and I’m going to need to understand how I do that with the ability to…

make it as easy as I can for my team to be able to get on board as well. So when we talked about this with Abdi, and by the way, we had actually already run a number of the AltaClaro prompt engineering sessions with our lawyers by then. And so with that, as well as the corporate law training that Abdi referred to, we had a decent understanding of the way that AltaClaro approached things and the learn, do, review.

And we thought that would be a great fit when we looked at this particular training need for partners. And it was around those objectives that Abdi just outlined. It’s about building confidence. It’s about understanding how you can ask good questions sooner. And it’s how you can build efficiency. And so all of those things we thought were just a great fit for this. So we tried to build in working with Abdi and the team.

a really good range of those challenges that we thought partners would come up against when they’re looking at how to do this. And so they included everything from, do I even make sure that I have the right training in place for my team so that they are ready for these changes? How do I make sure that I understand what the obligations are on me as a practitioner? And Brendan just went through some. And sometimes of course it goes even broader than that.

There are state restrictions, there are court restrictions, there are even sometimes individual judge restrictions. So you’ve got to get across a wide gamut of that. So that’s one challenge. You also have the challenge of being able to understand enough about the technology without being overwhelmed. We saw a lot of training in the early days on generative AI, Marlene, and it was putting lawyers through, you this is how a neural network works.

They did not need to know it. I mean, they might have been curious and interested, and there are certainly some partners who live for that stuff, but it’s not everybody’s cup of tea. What they really needed to know was enough to understand how they could assess the risks and the opportunities. So they needed to know the difference between a public system and a closed system and what the relevant risk profiles were for those. They needed to be aware of when they were supervising work that was given to them.

Greg Lambert: Ha ha.

Carolyn Austin: by an associate that they needed to review and check before it went out the door to a client. They needed to know what kind of a system it was built on so that they could calibrate how they were going to supervise on that. They needed to understand enough to be able to do their job, but they didn’t need to go in, they didn’t need a PhD in neural nets. So that was the first thing. And I think it’s always…

With any change management initiative, right, with any adoption project, you really need to be thinking about not only the logistical challenges, but also the human challenges. What motivates us as human beings to be able to get on board with a new initiative? And how do we make sure that we make that pathway as easy as we can for the leaders of the firm?

who ultimately are going to be so critical to driving that adoption. And that’s not just in our firm, that’s every firm.

Marlene Gebauer: Yeah, true.

Greg Lambert: I’m just wondering, Abdi, have you focused on non-legal practice specific training with other modules that you have? And so was this something that you were able to look at and adapt some of the previous training onto the…

and plug in GEN.AI into this in a way that you’ve had experience, and it could be management, change management, any of that. Were you able to kind of leverage previous experience and modules to take on this idea?

Abdi Shayesteh: Yeah, no, that’s great, because originally we started teaching substantive courses in practice areas. And then I would say this was the first time where we really deployed a course that went beyond just attorneys and into allied legal professionals. And I would say thanks to KNO Gates, because they have a very inclusive approach to their training that at all levels,

and they were going through our prompt engineering in different roles from junior associates to seniors to partners and everything. And through that experience, they were giving us feedback in real time where we needed to make adjustments. And we realized quickly because the original version, the assignment was just for someone who was an attorney or was about to be an attorney. And we said, well, I know. We ought to create one that’s for allied legal professionals that doesn’t require legal knowledge, but it would be an assignment that they would normally

do that lands on

doing research to figure out best practices for remote office for the firm and coming up with a policy, something like that. And I have to really give credit to Patricia Libby, our executive editor, who’s our head of curriculum, who has a team, assembles a team of educational designers and subject matter experts and work closely with Carol and Brendan and team to get all these things right by testing, iterating, and getting feedback. So we definitely use the same principles of getting people

into the water. It just was the different water for different folks and getting that personalized feedback to them at scale. But the adjustments were about who’s going through it.

Greg Lambert: So, Brendan, we talked about the desire that you have for innovation. I’m really kind of curious. We’ve been talking as an industry a lot about the effects that generative AI will have on the practice of law and kind of shaping how firms go forward. And it sounds like you and Carolyn and others are kind of helping to mold.

where you want at K&L Gates to go. But when it comes to partners, partners can be a little bit of a different animal, especially when it comes to training. So how has this program been received by the partners there at K&L Gates?

Brendan Gutierrez McDonnell: I think it’s a mixed bag. think it’s some are so gung-ho that they can’t wait and they’re they want to try something new. That’s like it’s like I go back to the kindergartens. It’s like three or four of them that just jump on the new toy. Right. And that’s there’s that we have them and we have those that stand back. Right. And they wait. And the good news is, you know, we’re you we have what we call we come creative, we call a power user group. They’re called pugs. And I basically give them a pug, which is a little

Carolyn Austin: Thank you.

Greg Lambert: Ha ha ha.

You

Marlene Gebauer: Ha ha ha.

Brendan Gutierrez McDonnell: It’s actually a pug with a rainbow, like, I don’t know what it is, sweater on that I send out if they become a pug. And it’s become quite the, you know, the valued medallion or whatever that you can get if you are successful. But the idea is that we want these people that have really jumped in there, that really want to learn from partners to associates, whoever they may be, so they can spread the word and also make it easier for people. you know, we have the mixed baggage, you could guess.

I do think that we’re getting by the initial issue was some said don’t touch it, right? Don’t touch the stove. It’s hot. You’ll burn your hand, right? And eventually you you learn that you can boil water on the stove, right? Or you can do a lot of things right for the stove. So I think we’re past the initial stage. So I think people are even our most senior partners realize that we do a lot of demos. We show them what can be done and they you know they look they.

A lot of them expected it to make a bunch of mistakes and we’ve shown them some of the programs we’ve licensed that and they don’t. And they have to vet the work, of course, that’s part of their duties, part of the courses that we teach them, know, that’s part of the curriculum. But in reality, it’s a very advanced system. And I think they see it’s changing them. And I think that they’re happy that we’re leading the charge and we’re organized. And we’re not leading the charge in such a way that we’re way out there. We’re definitely, we have a good gate.

We are definitely in the forefront, but we’re not just, you know, we’re not being ridiculous. We know we can’t bring our partners along unless we’ve thoughtful and we’ve got a great program. But education is at the front of that. Carolyn, you want to add to that?

Carolyn Austin: Sure. I wanted to talk about that and I wanted to talk a little bit about the kinds of strategies that we’ve looked at to try to foster an environment that really supports innovation, particularly dwell on some change management issues around that. If that’s okay, I’ll step into that area. So I think the things that I see are part of the secret sauce that we’ve managed to put together so far.

And certainly we are on a journey and we have a long way to go and a lot to learn. But I think the learnings that we’ve had so far would include, first of all, it’s great to start in our firm because we do have that culture of innovation that we’re pretty proud of. And Brendan talked about a few of the reasons why we are where we are. But it also means things like

Empowering people to bring new ideas forward is something we started with our idea group, which is our innovation committee, I think about five years ago, globally. there’s already that. We’ve spent some time on inculcating a growth mindset in our lawyers and our allied professionals. we’ve run this supporting courses around the importance of being open, the importance of being curious.

the fact that we will celebrate you if you bring a new idea forward and the opportunities on to do that. So we have some good fertile ground in which to plan generative AI. The AI Solutions Group has been an amazing group. In all my career, I’ve never had the privilege of working with a greater team.

And one of the things that’s made that so good is the diversity, the intentional diversity of the team, because it is not limited to partners, although it includes very senior partners like Brendan. It also includes representatives from across our firm and across our practices. But it is very much around including people from the allied professional side of the business who all have a stake in this and who all will be users of generative AI.

And I think the attitude that that 100 % on 110 % brought into hard work around generative AI. So that helps a lot. We kind of in it together. And then there’s the wider network. So the AI solutions group is one sort of centre of that, but we have a nominated AI on every practice. Some of those liaisons.

the working groups around themselves that are focused on the practice group that I’m privileged to attend. So we have a communications network that goes around the firm so we can find out what’s happening, what are we hearing from clients, what do we need to address as well as being able to use that network to say, something new is happening and filter that back out. I think that the third pillar of that is really education. So from the get-go,

we kind of thought we need an intentional approach to education around generative AI. And so we’ve had a multi-tiered approach that includes a foundational layer. So we want to lift everybody’s AI literacy in the firm, regardless of where you are, regardless of your role, so that people get to understand what the technology is and isn’t and can engage in conversations about that.

driven by the hype that we all hear outside of the phone. And at that foundational level, we’ve also got particular cohorts. So you’ve got the summer associates, they really need training in generative AI and they’re really hungry for it and ready for that. They know it’s going to be important. Our first years as well as another cohort where we identify and we invest in them, right? So it made sense to look at then above that.

Obviously policy, compliance, and ethical standards, really important one to bed in across the firm. And then we have our expert training across that. And know, one of the things we found with the prompt engineering training that ABDI does is people get a lot more out of it if they’ve already had a play with their generative AI tools. As they then kind of get it, they understand more than theoretically why it’s important to get good at prompting. So that kind of comes into that specialist layer.

And the supervisory layer just fits into that multi-tiered strategy really nicely to be able to get to those educational requirements that we identified for that part of the layer. so there’s a lot more to it, but that’s broadly it. And you can tell that I’ve got just as much passion when I start talking about it. I’m going to stop. But I mean, those things, think, were really helpful because then when you’re introducing a new tool, people have a context for it.

Greg Lambert: Yeah.

Marlene Gebauer: Exactly.

Carolyn Austin: And people have somebody they know. know, our pugs that Brendan mentioned, that’s a whole network of folks out there who are really our generative AI ninjas. know, they’re there to be able to say, let me show you how to do that. I’ve done the following, you know, with those things in my practice has been really helpful. And that peer-to-peer learning is great. We’re running clinics at the moment, which is also peer-to-peer where people could come together and go, I’m having trouble knowing how I can use generative AI more effectively in this particular circumstance.

and they can solve that together. So it is multi-layered and with all of that, none of which happens overnight. And by the way, we’re very grateful to our leadership for supporting all of that. It just helps to move to the next step. Yeah, Karen, when you mentioned something that…

Greg Lambert: Yeah, Carolyn, you mentioned something that in.

when you were talking about it was better for the people going through the training if they had had some kind of personal experience using the generative AI tool before they show up in the initial training. And I know there was some reactions last year, especially after the New York lawyer,

Carolyn Austin: you

So,

Greg Lambert: citing the fake cases that really I think there were some firms especially on the security side and ethics side that kind of Quash some of that enthusiasm a little bit How how did K &L gates, you know? Encourage people to use it but also kind of make sure you’re hitting that those ethical Standards as well so that when they do show up

Marlene Gebauer: Double down.

Greg Lambert: You know, they’ve got that experience that they’re not, one, they’re not just starting from scratch, and two, they’re not just petrified of using it and feeling like they’re gonna make a mistake.

Carolyn Austin: Thanks

Yeah, it’s a great question. And I want to clarify because I may not have been clear. The foundational training and in welcome to the product. This is how you know, here’s the toolbar. This is where you need to click to be able to do that. That’s important. And people have in our firm, it’s mandatory to do that training before you can get in and start to play. The second part of our training that is mandatory and we call it GNI bootcamp.

is to be able to do the generative AI policy training. So we take people through what is our policy around use, safe use, correct use, incorrect use of generative AI. They must do that before they can get access to our generative AI tools. So we are not letting them in with, we’re not setting them up to fail, we’re setting them up to succeed with that level. But my comment about the prompt engineering was,

I can put people through a module on prompt engineering, but if they’ve never had the experience of being inside a generative AI tool and doing prompting and they don’t really get why it’s important. If you’ve already been in there though, having done your foundational eGenAI bootcamp and you’re in there and you’re starting to prompt for the first time, you soon as I say, I can ask this question six times. And what comes out of it is going to be really, really,

dependent on the way that I ask the question. I need to learn some tools to ask those questions better. Aha! Now you’re ready to do the prompt engineering course and get hands on with that. Does that make sense?

Greg Lambert: Yeah, yeah. Thanks.

Brendan Gutierrez McDonnell: But let’s reel this way that there are a number of people in the firm that are going to go out and get their chat CPT like me, pay their $20 a month and not really put firm stuff through there. They might put a legal question in there, but they’re just playing and we want those people. We need those people. We don’t want them to put firm stuff in there, right? We’ve got to have them trained. That goes back to what Carolyn said. But we have gotten the advantage of the fact that there are people that tinker.

And those tinkerers are probably our pugs to some degree, right? They really, you know, learn to do that. But it’s, you know, it also probably helps a little. I may have a couple gray hairs. Somebody told me there’s a few gray hairs up there. And yeah, exactly. And I think, you know, when I, you know, coming from me, it kind of comes differently. So I’ve had some senior partners come up to me and said, if this Gen. AI stuff is taking over what I do every day, I’m going to quit. I said, well, then pack your bags.

Cause you’re gone. And I joke with them. And then I go, no, but I want to sit down with you. said, you know, I’m in jests really. I, you know, of course I like these people, but I didn’t say, let me, let me run you through this. Let’s see it. And then they go, well, that’s really useful. And a lot of them just have misconceived ideas about what it does and cannot do. And I think that’s really what’s important about it. And I think that to the extent that I’m sort of a senior partner that helps to speak to those that are and say, listen, guys, you have to.

You have to get in there and at least understand that’s where that supervisory course comes in. It’s designed for people who are supervising who may not be jumping into that they may not be the tinker. They’re not going to be the pubs, but they have enough knowledge about it that they can supervise it and make sure that we’re vetting our work. You know, we emphasize our brand. Our brand is huge. a mistake happens out there, that’s really big to our brand. But it’s also just showing them that you can supervise this.

The newer people are going to want to use it. They’re excited to use it. So let’s learn how to use, let’s learn how to use the stove correctly and let’s see what we can bake.

Marlene Gebauer: So Carolyn, you teed up the next question perfectly talking about the multifaceted training that you have at your firm. And Abdi, I’m going to turn a question over to you now in terms of training for associates. You know, I’m sure firms have all different approaches to this, and I’m very curious to hear what you are seeing generally. I mean, are you seeing the same

of training that we’re hearing about here or are you hearing different things? What’s been the response?

Abdi Shayesteh: Yeah, no, thank you. Great question because we got our start. We were the first to market a prompt engineering course for legal professionals. 2023 summer, Oric came to us and asked to create this course for their summer associates. They said they wanted something that was tech neutral, that really trained on critical thinking.

And they said they also wanted something that was hands-on, got their hands dirty in the tool to start practicing, which is what we went to market with. And from there, we signed up other firms very quickly, K&L Gates and others. And because of that sort of getting in early, we saw the spectrum.

of different approaches and different level of where folks were inside firms. So, and I want to talk about that in a minute, but I want to talk about the associate experience and why this has been helpful for them. We’ve been talking about sort of…

context, context, context, giving them context, whether it’s partner or junior associate. And that’s really what this has started from, was let’s get these junior associates into the tool so they can practice using fake client scenarios and they could see what could go wrong. So I know there’s the hot stove analogy. I want to use the beer analogy when you sat down with your parents at 18 and they gave you the beer.

Greg Lambert: I like this analogy already.

Marlene Gebauer: Parents never did that.

Brendan Gutierrez McDonnell: It’s more fun.

Abdi Shayesteh: Yeah, you can see what could go wrong very quickly. And that’s what this became, was that the first the associates were now like, I actually see this in the context of assignments. see that you need a human involved. You need a supervisor involved. You can’t just rely on the tool. Otherwise, it could go in a variety of wrong places. So it really kind of put things into context, how to use it safely.

where could help you, it doesn’t eliminate what you do or the need for a senior lawyer. So I think that was the brilliance of the firms who went in to approach this and give this to the hands of their junior associates and summers and to train them while they’re doing it. But you’re right. Getting in early allowed us to see the spectrum of what I call the tire kickers and the serious firms.

like K-note gates. yeah, those tire kickers are still trying to figure it out. And some of them, good intentions, but they’re kind of stuck in bureaucracy. They have the naysayers, the detractors, versus the very early adopters. And then the serious ones, though, I see the same theme, this immersive approach. see this, the reason to do it.

You know, if you’re doing it for summers, it’s because in addition to giving them exposure for safety and ethics, but it’s also a recruiting tool that, you know, this is a firm that’s innovative and will train you and will invest in you. It’s also a signal to their clients that we’re going to drive on innovation. We’re going to drive efficiency. And we’re taking this seriously. And so you see this seriousness, you know, tone at the top where you have in these firms

senior leaders like Brendan and Carolyn involved in the training with the junior associates and it’s very immersive across a variety of roles. They also embrace that there’s no magic pill to training that this can’t be achieved by just putting people in front of the videos and checking the box. That there’s got to be some workout here and the firm is really around that. And they also see that it’s tailored.

that you have to have a tailored approach as what we’ve done here with the different roles and even different levels. And that’s kind of put us in a place where we’ve even organized the cohorts based on different roles and experience levels because things can be different for them. And so yes, those who did dabble and tinker, they get a different experience because now they have a better way to be even better and more effective. And those who are at ground level also get a different experience to start and build on top.

get their confidence. So all of these things came from our experience with serious law firms who’ve taken it in this multidisciplinary approach, this immersive approach, had a plan, whether it was experimentation or deployment and building, and we’ve benefited from that as well. So anyway,

Greg Lambert: So, Abdi, if you could wave a magic wand, is there anything that law schools could be doing better to get the students while they have them better prepared?

Abdi Shayesteh: Yeah, if they can move faster and make decisions when making a decision, that’s their problem. Well, you said magic won. You said magic. And to give fair, we have worked with a few law firms where their deans have been entrepreneurial or the professor has pushed for this. But yeah, it’s limited compared to where you see it inside the firms today.

Marlene Gebauer: Ha ha ha ha.

Greg Lambert: Well, can’t do miracles, I guess. That’s true, that’s true, I did, I stepped into that.

Marlene Gebauer: Magic. It’s magic. It’s magic.

Brendan Gutierrez McDonnell: today.

Marlene Gebauer: So I have a follow up for this.

And it probably plays in on law schools. also, ABDI, also talks, it plays in on these scenarios, but sort of with a slightly different bent. So recently we’ve had Dr. Megan Ma on from Codex, who has basically created a negotiation tool that, using a persona that was trained to help students get through the negotiation process for an &A matter.

And recently I read and I posted about, I think it was the University of Missouri Law School, one of the professors there, taking a lot of initiative and moving quickly. She also trained a model to help students learn negotiation in a litigation process. So I’m wondering if there is room for this type of immersive tool

in your education tools moving forward or any firm’s tools moving forward.

Abdi Shayesteh: absolutely. And that’s the direction we’re going, where you do go through simulations and you get feedback. I don’t want to take the cat out of the bag, but very soon, that ability, you guys, I to say, just wait, but absolutely.

Marlene Gebauer: I didn’t even do this on purpose. He didn’t pay me for it, folks.

Abdi Shayesteh: I appreciate that. No, no, because this is the interesting thing. Why the legal industry, we’re kind of laggards, right? All this stuff is taking place in other verticals. Pilots get this pilot simulator. Doctors get to play on, they have simulations they can even do. So we too need to have this real time simulation where you’re negotiating with personas, leveraging AI, leveraging gen AI. And we have built that in where now associates are gonna get individual feedback on their assignments. But even going out further where,

Imagine if you had a library of simulations that reacted based on how you did and the level of difficulty changed each time. It’s kind of like choose your own adventure. You can start to throw in more wrenches in their workflow, see how they deal with that to build their critical thinking skills. So I think what they have done is exactly what we need to be doing, is to transform the way we’re going about education in our industry. The more practice, the better you become. So let’s create that environment where you can practice safely and go at it.

in the shallow end before going out into the deep end.

Marlene Gebauer: I’m almost thinking you can customize it for firms. Like if there’s a particular partner whose style you want to emulate, you want people to emulate, you could do something like that, customize for firms.

Carolyn Austin: Can I just add, yeah, we’re going to talk. Can I add something there to that, which is that, you know, the introduction of not only generative AI, but of innovation in law schools is so critical. And, you know, it’s so exciting to see those undergraduate subjects coming up. It’s good to see them also at graduate level.

Abdi Shayesteh: I’m gonna call you in a couple weeks.

Marlene Gebauer: Yeah

Abdi Shayesteh: Yeah.

Marlene Gebauer: Yeah.

Carolyn Austin: What I wanted to add is we’re part of a legal ecosystem, folks. And if we get better law students through the door against law firms, then our job is easier. And we then build on that and grow their skill set when they’re with us. We have, if you consider all of the parts of the legal ecosystem, which includes courts, judges, regulatory authorities, includes government, right?

All of those parts dealing with the introduction of generative AI and grappling with what that means. And I think we do need to be outward facing to some extent as we’re going through this journey ourselves and think about how we can, I guess, advance the legal ecosystem generally for the benefit of the community that we serve. And to that end, Brendan and I have been really looking at the potential for using artificial intelligence for access to justice as well.

And we have been able to be one of the foundational sponsors actually of an event that’s coming up in the next couple of weeks held at Stanford Law School through their legal design labs to bring together various people from different parts of that legal ecosystem who are involved in serving the community beyond the dollar.

bringing together those people who want to give of their time and their expertise to be able to think how can we make sure that artificial intelligence is going to be something that is not going to leave people behind. So there’s, I get very excited about the prospect of doing that.

Marlene Gebauer: Well that I think teases up nicely for the last question for everybody. So we ask all of our guests a crystal ball question. And Brendan, we’ll start with you. So what changes or challenges do you see happening in the legal industry over the next couple years that we need to face?

Brendan Gutierrez McDonnell: Well, I think this can dovetail into the questions that were just being answered about universities, law schools, and getting ready. If you think about it, understanding the solutions, I’m sitting through pilots and demos like every week. And I feel like it’s never going to stop because it may never stop for a while, which is there’s so much coming out. There’s so many different tools that are going to be available to us that we actually have to, being a lawyer is not just about getting to the right answer and understanding

how to apply legal principles. It’s also picking the right tool and staffing it with the right people. in the past, was always lawyer, associate, everything. There was just a formula. And now, you think about it, you may have some non-legal team members who actually tee up some results from an AI solution that they gave you and your job is to verify it. the whole job may be different. And I think that that’s

a big part of the crystal ball, is, now how do I get us ready as a firm, that change management to understand, this is one of the reasons we came up with the term allied professional in our firm is that, they’re gonna be part of the team, they’re gonna be billing, they’re gonna be no different than a lawyer, because having the technical person who’s there is gonna figure out how are gonna take these 300,000 documents and we’re gonna actually cull through them.

find the right information that we have and make sure that we get all the value, not only the information that might get a transaction to them, but also information that might be valuable to the client for perhaps extracting at a later date, right? Because it’s all of that that’s coming to bear on how we provide that value to the client. So I think we’re going to face competition from outside consulting firms and insurance firms and accounting firms. It’s already happening throughout the world.

because they now can slide into our area with these tools. And in fact, we can slide into their area. So you’re seeing a lot happen in these different areas. And I think my job or what I’d like to do or the crystal ball is make sure we’re ready as a law firm, make sure that our management has the information that they need, make sure our people have the training that we need, make sure we have the right personnel and make sure we listen to our clients. mean, listening to the clients and hearing…

what their needs are and what they’re looking for. And then actually trying to make sure that we service those needs in the right way. And that’s the goal.

Marlene Gebauer: Greg’s on mute. Wait, we’re at a Teams meeting. Hang on.

Brendan Gutierrez McDonnell: for filming.

Greg Lambert: There, sorry. Wait, Hey, it’s not a true meeting until somebody says that. So Carolyn, how about you? What kind of changes or challenges do you see?

Marlene Gebauer: Exactly.

Carolyn Austin: I am just so glad to be working in this space right now because I feel excited every day about what’s happening and what’s going to come. So I’m very optimistic. I think there are three things I just wanted to touch on. And crystal ball gazing is fairly foolish and fraught with peril, but three things. One is Brandon mentioned clients and I feel that

And when we’re talking about being in a legal ecosystem, of course, the clients are a huge part of our legal ecosystem. When we talk with clients, we have to meet them where they are on this journey that we’re all going through. And some of them are way in advance of us in terms of their uptake of generative AI. And some of them are yet to start. And so I can see immense opportunities for us and for

for law firms to be able to share our experiences, to listen to what our clients’ experiences are, potentially some of the things they may be fearful about or hesitant about, and try to go on the journey together and be very collaborative about that. I think generally collaboration is gonna become more and more important. So that’s the first thing. The other thing I wanted to mention really quickly is data. You know, we need to become

more of a data-driven profession. At the moment, we are still trying to work out, I think, to a large extent, how to gather data, how to engage in metrics, particularly around technology, certainly generative AI. And we measure that. There are some great AI benchmarking initiatives happening around the traps, which are really good to see and to watch. But I think internally, we need to get good at capturing data.

And then we’re good at capturing data, we need to also get good at managing the data and applying some quality assurance to that data so that we as a profession haven’t been great at that, generally speaking. And I think that we’re going to see more of that. And thirdly, and finally, I think that when we look back on this, I think we’re going to be very interested with the kinds of use cases that we saw in 2024 and 2025 as low hanging fruit.

And I think with the benefit of hindsight, we might think, you know, they thought it was all going to be about assisting with routine tasks that lawyers were otherwise doing. Maybe that’s not going to be where we see value ultimately. I’m really curious about the higher level thinking kind of use cases, the generative AI. How can it make me as a human a better thinker? How can it teach me? How can it engage with them?

improve my thinking, my critical thinking skills, how can it give me different perspectives than my own and provide me with even greater options than might have occurred to me as an individual. I think that might be really interesting to explore further once we get over the yes, sure, we can use it for due diligence, but what about if we can use it for some things that are a little bit more interesting? Sorry, to the due diligence people who find that.

very fascinating. But there you go. That’s my three things.

Greg Lambert: Well, luckily, I gave Abdi a magic wand earlier. So unfortunately, it looks like he’s had some technical difficulties and has dropped off. So we’ll just assume that his magic wand is working for the crystal ball section as well. So Carolyn Austin and Brendan McDonald from KNO Gates really want to thank

Carolyn Austin: Thanks.

Greg Lambert: both of you coming on and for Abdi for coming on as well and talking with us today. Thank you.

Carolyn Austin: for having us. It’s been a pleasure.

Brendan Gutierrez McDonnell: Thank for having us. Absolutely. Great conversation. Thank you.

Marlene Gebauer: And of course, thanks to all of you, our listeners, for taking the time to listen to the Geek in Review podcast. If you enjoy the show, share it with a colleague. We’d love to hear from you, so reach out to us on LinkedIn.

Greg Lambert: So I will make sure that I put links out for Abdi and Alteclero and I’ll make sure that those get on the show notes.

Marlene Gebauer: And Carolyn and Brendan, where can listeners find out more about your collaboration and training program with AltaClaro?

Carolyn Austin: Thank you.

Well, apart from through Abdi, I’m always happy to take a message. So if anyone wants to email me, I’ll have you to talk about it.

Brendan Gutierrez McDonnell: Yeah, go ahead, Carolyn. You try that one.

Marlene Gebauer: you

Great. And as always, the music you hear is from Jerry David DeSica. Thank you so much, Jerry.

Greg Lambert: All right, thanks Jerry. Thanks Marlene. Talk to you later.

Marlene Gebauer: Okay, bye.

Carolyn Austin: Bye.