AI, Pursuit of Justice & Questions Lawyers Should Ask

A timely article highlighting the key questions that lawyers must consider in a context where they are increasingly held accountable for understanding the impact of the AI systems that they employ or any system on which they advise. The article is drafted by Julia Brickell (Columbia University), Jeanna Matthew ( Clarkson University), and Denia Psarrou (University of Athens & Shelley Podolny, Columbia University) - under the Working Group on Judicial and Legal Capacity Building on AI.

Amongst others, lawyers may start their evaluation of an AI system by asking:

• Under what conditions did the developers test the systems?

• How might the environment of the intended use differ?

• Is there research on bias and the potential for disparate impact in systems of this type?

• Are there key demographic groups for which the system was not tested?

• Might this system in this instance with this data have a disparate impact?

• How is effectiveness to be tested in the context of the current use?

• What samples need to be drawn?

• What are the competencies needed to evaluate the effectiveness of the AI system and to deploy it effectively?

• What evidence is there that the system has been tested on use cases similar to the proposed use?

The article was originally published in April 2022 for Bloomberg Law.