What’s Fueling the AI Panic? Four Lawyers Read Between the Alarms

Recent months have brought a steady stream of unsettling headlines – AI insiders estimating double-digit odds of catastrophic outcomes; labs disclosing what they describe as their own models hacking, deceiving and breaking the rules; and even the more aggressive AI company CEOs calling for the industry to deliberately slow down. Is this growing alarm proportionate to what’s really going on behind the scenes? Should we be worried? What’s the lawyer’s role amid accelerating adoption and unchecked growth? We posed these and other pressing questions to four leading thinkers in the Canadian innovation space: Brent Arnold, chair, Canadian Internet Society; Sam Ip, partner, technology, Osler; Leslie McCallum, founder/CEO, Lexata; and David Mousavi, president, CBA In-House Lawyers. Here’s what our experts had to say …

Given the deep subterranean nature of AI development, what do the suddenly panicked CEOs already know about AI that we don’t know?

David: “In part, CEOs are acknowledging what they don't know and specifically that AI has proven to be unpredictable. Understandably, this has raised their concerns.”

Brent: “I wouldn’t assume they know something apocalyptic that they’re not telling us, though the fact that they’re building private bunkers suggests they’re concerned. They have access to unreleased models and internal testing that we don’t, but that doesn’t mean they fully understand what they’re building. I would say their stated public position seems to be ‘I’ve invented a doomsday technology and someone should do something about it.’ But let’s not forget the possibility, which many thinkers including Cory Doctorow have raised, that the panic is a sales pitch in disguise, i.e., ‘my technology is so powerful it could destroy the world;’ imagine what you could do with a fraction of that power?”

Sam: “Hard to say, though these alarmist views often make the headlines. I often say we need to be careful about extrapolating from isolated incidents. Few can reliably quantify the probability of these outcomes.”

Leslie: “There is something strange about business leaders publicizing the risks of their own businesses. We shouldn’t take at face value that pure altruism motivates these warnings. As with all things AI, we must question what we consume, stay alert, learn, and formulate our own opinions.”

From where you sit, is this panic proportionate to the actual risk, or is some of it noise? Is the era of humans (as the planet’s thought-leaders) over?

Leslie: “Nobody knows the answer to this yet. Everyone is learning as they go, even the top scientists. The human desire to build things is so powerful, we’ve gotten ourselves into a climate bind because of it, and now a technological bind too. Great minds might tell us to slow down, or stop, but can we? Panic is pointless. We need to harness our builder instincts to build counter-balancing AI systems. Smarter models that can detect and prevent hacking, that can shut down rogue systems, that can block unauthorized activity, that can highlight non-human content, and that can publicize copyright infringement. Fight fire with fire. AI development is seemingly out of control, but AI is neither a religion nor a god. We have both the power and the responsibility to determine its future.”

Sam: “These risks are real, and we can all debate about the probability of catastrophic risk. But in my view, they can be a bit of a distraction to the very real and actual risks we all know about, things like bias/harm, misinformation, privacy and even consumer protection risks. Not all leading AI company CEOs subscribe to this risk, including our Canadian AI darling Aidan Gomez.”

Brent: “Depends. If the panic is that unchecked generative AI is going to disrupt our economy, deskill our workforce, deepen structural inequalities in society, make us all dumber and more docile, and hasten the climate catastrophe, I’d say yes, the panic is justified, as I’ve seen no large-scale resistance to that future coming about. If the panic is that generative AI is a cyber risk juggernaut with which cyber defenders can’t keep up, I’d say the events of this October suggest, that panic is also justified.

“The era of human-produced shallow business and advertising copy may be over, at least until we get sick of reading the same cliches rehashed by AI and need humans to invent new ones. As for more academic, rigorous thought leadership, humans will still lead the way, but they’ll be AI-enhanced humans, working in tandem with AI.”

David: “We've already observed examples of AI acting independently and in its own self-interest so it’s safe to say the risks are real.”

What roles can lawyers play in protecting humanity and the rule of law in what seems to be becoming a more and more a lawless system, with the actual AI player having no interest or motivation in following a set of human rules (and not a clear set of rules to begin with)?

Sam: “Lawyers have a critically important role in protecting humanity given their unique skills and expertise. Take C-34 (the Safe Social Media Act) for example, which introduces a novel AI chatbot regulatory framework in the world. Getting that framework right in a way that protects public interests, while striking a balance that does not inhibit innovation or impose undue compliance burden is not easy, yet precisely where lawyers can contribute.”

David: “Lawyers should be working hand-in-hand with the industry to develop robust regulations and technical safeguards. This collaboration will begin with lawyers gaining a full understanding of technical details that can be codified in law. Equally important, standard ethical principles for AI companies will need to be periodically reviewed and updated to keep pace with technological advancements.”

Leslie: “Compared to other citizens, lawyers have more opportunities to speak up and be heard – at conferences, in political forums, in charitable forums, in written articles and books, and in teaching and training younger lawyers. People often listen to lawyers’ viewpoints carefully, especially when the views are expressed in a balanced and compassionate manner. Because of their outsized influence, at a minimum, lawyers should educate themselves about AI as a driver of social, political and economic conditions, and seize opportunities to participate in AI conversations.” 

Brent: “I don’t think they will. Most haven’t roused themselves to the challenge of preserving their democracies from foreign interference and creeping fascism, and those are challenges we should be able to meet. Most don’t seem motivated to act unless their own prosperity and privilege are threatened. Generative AI tools do present that threat. I think lawyers will do what they can to prevent themselves and their profession from being replaced by AI, but they have to answer to clients who’ll be pushing for lower bills and greater speed.”

What concrete steps should lawyers take to keep their own judgment in the driver’s seat, rather than ceding it to AI, as these tools become more embedded in daily practice?

Brent: “First, work out the client’s problem and your preliminary view before asking AI for its answer. Otherwise, you risk spending the rest of the exercise editing the machine’s assumptions rather than testing your own.

“Second, verification means opening the case, reading the provision and checking the evidence (not asking the chatbot whether it is sure). Canadian professional guidance is quite clear that the lawyer remains responsible for the work and that meaningful human control requires checking reliable sources.

“Third, understand the tool’s data handling before putting client information into it, and require lawyer approval before anything consequential is filed, sent or acted on. Those are practical applications of the confidentiality, competence and supervision obligations that AI does not displace.

“And keep doing enough unaided analysis to remain capable of spotting a bad answer. A human in the loop is not much protection if the human no longer knows what they’re looking at. Use AI to challenge your judgment, not to relieve you of having to exercise it.”

Leslie: “Larger law firms should employ technology experts whose job it is to interrogate AI companies. To understand how each AI tool works, the nature and degree of risk, and quality control measures. Education is the antidote. The LSO and the OBA can help fulfill this role for the profession generally. If an AI system is ultra-powerful yet inexplicable, its usage should be carefully guarded. AI is an incredible tool for legal practice, but it is subservient to lawyers’ own brains as a matter of professional responsibility, self-respect, and intellectual honesty.”

Sam: “Read widely, think critically, and don't outsource judgement. As an under-20 year call, I'm firmly in the camp that does not feel threatened by AI, but rather empowered by it. It levels the playing field and enables innovative young lawyers to work more independently and competitively, and ultimately better serve the public and our clients.

About our Contributors

Brent Arnold, founder of Capstan Legal, is a litigator with over 20 years of experience specializing in cybersecurity, privacy, and commercial litigation. He serves as chair of the Canadian Internet Society and holds leadership roles in multiple legal and technology organizations. His expertise lies at the intersection of law and technology.

Samuel Ip is a partner in the Technology Group at Osler, Hoskin & Harcourt LLP in Toronto who frames his perspective this way, “I'm one of the leaders of our AI practice and my practice is working with the leading AI companies in Canada (e.g., Cohere, Waabi, etc.), particularly on AI governance and commercial contracting.”

Leslie McCallum is CEO and founder of Lexata Inc., a legal technology company focusing on North American capital markets regulations. She previously practiced U.S. and Canadian securities law at Torys and served as adjunct professor at the University of Toronto Faculty of Law. She holds a Master of Laws from Columbia University.

David Mousavi is president of CBA In-House Lawyers and chief legal officer at NRT Technology Group, a global fintech company, where he oversees data security and privacy for the organization. He holds a JD and MBA and has spent over a decade advising public and private businesses, non-profit organizations, and political leaders.