Reading Assignments

To allow for this course to be flexible in response to the pace and dynamics of our in-class discussions, reading assignments will be scheduled on a rolling basis. To allow you enough time to read the assigned material, I plan to post reading assignments at least one week in advance of class. Once I have posted a reading assignment for a particular day, I will not change the assignment by adding more reading for that day.

Do not forget to submit your reading response the evening before class. Details about your weekly reading responses can be found on the assignments and grading page.

Lesson 01 - Aug. 22

Our first class will address some foundational questions.

  • Why does this class exist and why are you taking it?
  • How are algorithms, artificial intelligence, and machine learning already affecting legal decision-making? What are the benefits? What concerns does this development raise?

Assignment:

Part One: In prior versions of this course, I’ve devoted time on the first day for students to introduce themselves and share what topics they’re interested in learning about in the class. I’d still like to take the time for you to introduce yourselves. But I have an additional assignment I’d like you to do before class. Instead of sharing aloud with the class what you’d like to get out of the class, I’d like you to ask a large language model (such as ChatGPT) to write your introduction to this class based upon your background and interests. On the first day, each student will share with the rest of the class the personal introduction that the LLM has crafted for them — along with any commentary you may have about the LLM‘s statements. In turn, I will share with the class ChatGPT’s vision of a class entitled “Law, Justice, and Algorithms” and share how our class differs from the AI model’s prediction.

Part Two: Use some of the language from the introduction that the LLM has created for you as a prompt for Dall-E (or another text-to-image model like StableDiffusion or MidJourney) to create an image.

Please email me both the LLM text and image as part of your reading response. You are welcome to create multiple versions and send me your preferred text and image.

Readings:

DOWNLOAD ALL READINGS FOR LESSON 01

AI in the Criminal Justice System
Epic.org
Read all.

How the Police Use Facial Recognition, and Where It Falls Short
Jennifer Valentino-DeVries, New York Times (Jan. 22 2020)
Read all.

Chicago’s “Race-Neutral” Traffic Cameras Ticket Black and Latino Drivers the Most
Emily Hopkins & Melissa Sanchez, ProPublica (Jan. 11, 2022)
Read all.

The Coming Collision Between Autonomous Vehicles and the Liability System
Gary Marchant and Rachel Lindor, 52 Santa Clara L. Rev. 1321 (2012)
Read pages 1321-1330.

The Promise and Perils of Algorithmic Lenders’ Use of Big Data
Matthew Adam Bruckner, 93 Chi.-Kent L. Rev. 3 (2018)
Read pages 31-38.

DeepFakes: A Looming Challenge for Privacy, Democracy, and National Security
Citron & Chesney (Scholarly Common at BU, 2019)
Read pages 1768-86.

Sanctions Issued in Case Where Lawyers Cited ChatGPT-Hallucinated Precedents
Eugene Volokh, Volokh Conspiracy (June 22, 2023)
Read all.

Lesson 02 - Aug. 29

An introduction to machine learning

In this class, we will discuss how the field of machine learning has developed, what it looks like now, and how it may look in the future. With a firmer understanding of what machine learning is, we can address the question of whether we need distinct legal or regulatory frameworks for governing algorithmic decision-making systems or whether the “law of the algorithm” is an unnecessarily specific instance of more general principles.

Readings:

DOWNLOAD ALL READINGS FOR LESSON 02

Machine Learning: A Primer: an introduction for both technical and non-technical readers
Lizzie Turner, Medium: Artificial Intelligence (May 26, 2018)
Read all.

An Introduction to Statistical Learning with Applications in R
Gareth James, Daniela Witten, Trevor Hastie, & Robert Tibshirani (2021)
Read Introduction pages 1-9 (stop at “Who Should Read This Book?), 15-42.

Anatomy of an A.I. System
Kate Crawford & Vladan Joler (2018)
Read all.

Do Artifacts Have Politics?
Langdon Winner, Deadlus Volume 109(1) (1980)
Read all.

Cyberspace and the Law of the Horse
Frank Easterbrook, University of Chicago Legal Forum 207 (1996)
Read all.

Lesson 03 - Sep. 5

What concepts and tools do computational resources offer for realizing legal values and policies? What cautions and objections should lawyers and communities sharpen in the face of increasing use of computational and algorithmic tools in public and private settings?

Readings:

DOWNLOAD ALL READINGS FOR LESSON 03

Prediction Policy Problems
Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Ziad Obermeyer, American Economic Review (2015) 
Read pages 491-95.

A Guide to Solving Social Problems with Machine Learning
Jon Kleinberg, Jens Ludwig, and Sendhil Mullainathan, Harvard Business Review (Dec. 8, 2016)
Read all.

Of prediction and policy
The Economist (Aug. 20, 2016)
Read all.

Biased Algorithms Are Easier to Fix Than Biased People
Sendhil Mullainathan, N.Y. Times (Dec. 6, 2019)
Read all.

Want Less-Biased Decisions? Use Algorithms.
Alex P. Miller, Harvard Business Review (2018)
Read all.

Lesson 04 - Sep. 12

Fairness and discrimination

Recent developments in A.I. and machine learning raise questions about how fairness, equality, and nondiscrimination should be understood, defined, assessed, and advanced. As you make your way through this week’s readings, keep the following questions in mind:

  • What are the contrasting conceptions of fairness at work in these different sources?
  • How should we reconcile competing concerns of accuracy and equity?
  • How should an understanding of historic and systemic inequality influence the approach to incorporating machine learning into legal decision-making?
  • Do risk scores pose the same or different problems depending on the decision-making context (e.g., access to credit, eligibility for pretrial release without bail, parole eligibility, policing, child welfare, and so on)?

Readings:

DOWNLOAD ALL READINGS FOR LESSON 04

Two Conceptions of Procedural Fairness
Cass R. Sunstein, Social Research, Vol. 73, No. 2 (Summer 2006).
Read all.

Discrimination in the Age of Algorithms
Jon Kleinberg, Jens Ludwi, Sendhil Mullainathan and Cass R. Sunstein, Journal of Legal Analysis (2018).
Read pages 113-146.

Fairness and Abstraction in Sociotechnical Systems
Andrew W. Selbst, dana boyd, Sorelle Friedler, Suresh Venkatusabramanian, and Janet Vertsei, ACM Conference on Fairness, Accountability, and Transparency (2019)
Read all.

Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse
Anna Lauren Hoffmann, Information, Communications, and Society (2019)
Read all.

Lesson 05 - Sep. 19

Case study on risk assessments

What was your attitude toward risk assessments before doing these readings? What changed and why?

If you had to align yourself with one of the authors or between multiple authors, who would they be? How would your perspective differ from theirs?

What do you think of the “perfect is the enemy of good” argument from the “Open Letter”? Does your answer depend on a conception of risk assessments as either a positive incremental change or a distraction from other interventions?

What do you make of Mayson’s argument to use risk assessments to predict needs and intervene in positive ways? Mayson’s article leaves out what to do for pretrial incarceration in the absence of risk assessments. How do you expect that the open letter authors would respond? Would you buy their response? How would you approach the challenge that Mayson leaves unanswered?

DOWNLOAD ALL READINGS FOR LESSON 05

Machine Bias
Julia Angwin, et al., ProPublica (May 22, 2016).
Read the whole thing.

False Positives, False Negatives, and False Analyses: A Rejoinder to ‘Machine Bias: There’s Software Used Across the Country to Predict Future Criminals. And It’s Biased Against Blacks,’
Anthony W. Flores et al., 80:2 Federal Probation (Sept. 2016).
Read the whole thing.

More than 100 Civil Rights, Digital Justice, and Community-Based Organizations Raise Concerns About Pretrial Risk Assessment
The Leadership Conference on Civil and Human Rights (2018).
Read the whole thing.

Updated Position on Pretrial Risk Assessments Tools
The Pretrial Justice Institute, (2020).
Read the whole thing.

Open Letter to the Pretrial Justice Institute
James Austin, Sarah L Desmarais & John Monahan, (2020).
Read the whole thing.

The Accuracy, Equity, and Jurisprudence of Criminal Risk Assessment
Sharad Goel et al. (2018).
Read pages 1-4, 7-12.

Bias In, Bias Out
Sandra G. Mayson, 128 Yale L.J. 2218 (2019).
Read the introduction, pages 2221-27.

Algorithmic Risk Assessments and The Double-edged Sword of Youth
Megan T Stevenson & Christopher Slobogin, (2018).
Read the introduction, pages 1-3.

Lesson 06 - Sep. 26

Transparency, interpretability, and explainability

In this session we will discuss different approaches to achieve explainability, both from a legal and technical perspective. We will learn about the difference between interpretable algorithms and non-interpretable algorithms, and the difference between transparency ex-ante and transparency ex-post including different auditing methods. We will also discuss the tradeoff between transparency and accuracy, and how balance between the two can be achieved.

Questions to consider:

  • Does a requirement of transparency or explanation in the use of algorithms in decision-making promote fairness?
  • How would it work and what would be limitations?
  • How should different legal contexts require different transparency practices?

DOWNLOAD ALL READINGS FOR LESSON 06

Transparency and Accountability in ML-Enabled Systems
Christian Kästner, Medium, 2022
Read the whole thing.

Interpretability and explainability
Christian Kästner, Medium, 2021
Read the whole thing.

The Mythos of Model Interpretability
Zachary C. Lipton, 2016 ICML Workshop on Human Interpretability in Machine Learning
Read the whole thing.

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead
Cynthia Rudin, Nature (2019)
Read the whole thing.

The Intuitive Appeal of Explanable Machines
Andrew D. Selbts and Solon Barocas, 87 Fordham L. Rev., 1085 (2018).
Read 1087-1099.

The Hidden Costs of Automated Thinking
Jonathan Zittrain, New Yorker (2019).
Read the whole thing.

Lesson 07 - Oct. 3

Prediction and probability

Many legal concepts and practices are rooted in the language and logic of prediction and probability. Before issuing a preliminary injunction, a judge must predict whether the plaintiffs will win their case on the merits. Police must have probable cause for many arrests, searches, and seizures to be constitutionally permissible. Child welfare agencies triage investigations of suspected neglect based on predictions of which claims will be substantiated, while public housing authorities manage waitlists for housing based on predictions of who will use public housing for the shortest length of time before living independently. Across legal systems nationwide, algorithmic predictions are replacing or informing predictions traditionally made by humans. Today, algorithms can deny a person government food benefits, send a social worker to investigate a home, or ban a person from flying on commercial airlines.

As you go through this week’s readings, ask yourself about the compatibility of traditional legal concepts and emerging algorithmic systems. How much is a legal idea like “probable cause” governed by our understanding of probability? As we develop or encounter systems that consider probability much more rigorously than judges or police traditionally would, how much should statistical thinking govern our decision-making? Are there some legal concepts that can be reduced to numerical probability and some that should not be understood in purely probabilistic terms? Why? When is it fair to make a legal judgment that depends upon a prediction about someone based on that person’s similarity to a broader group? Is it ever possible to make a prediction about someone that doesn’t rely upon their similarity to a broader group?

Readings:

DOWNLOAD ALL READINGS FOR LESSON 07

Law and the Crystal Ball
Barbara Underwood, 88 Yale L.J. 1408 (1979).
Read the first part, pages 1408-1420.

Naked Statistical Evidence of Liability: Is Subjective Probability Enough?
Gary L. Wells, J. Personality & Social Psych. 62:3 (1992).
Read the whole thing.

On Individual Risk
Philip Dawid, arXiv (2017).
Read the whole thing.

The Prediction of Violent Behavior: Toward a Second Generation of Theory and Policy
John Monahan, 141 American Journal of Psychiatry 10 (1984).
Read the whole thing.

Situating methods in the magic of Big Data and AI
M. C. Elish & danah boyd, 85 Communication Monographs 57 (2018).
Read the introduction pages 57-58 and pages 67-72, starting with “Faith in Prediction.”

Lesson 08 - Oct. 17

Generative A.I. & Law

In this class, we’ll explore how A.I. — particularly generative A.I. — may or may not disrupt our contemporary legal systems. As you work through this week’s readings, ask yourself how our legal systems should adapt to a world of generative A.I.. What foundations should remain secure and what foundations are threatened? What are the benefits that may accrue and what are the harms? Who stands to lose and who stands to gain?

Readings:

DOWNLOAD ALL READINGS FOR LESSON 08

How GPT/ChatGPT Work - An Understandable Introduction to the Technology
Harry Surden, YouTube (2023).
https://www.youtube.com/watch?v=IMAhwv5dn8E

Legal Tech, Civil Procedure, and the Future of Adversarialism
David Freeman Engstrom & Jonah B. Gelbach, 169 U. Penn. L. Rev. 1001 (2021).
Read Introduction and Part III, pgs. 1001-1008, 1086-1099.

Generative Interpretation
Yonathan Arbel & David A. Hoffman, 99 NYU L. REV. ___ (2024).
Read Introduction and Part III, pgs. 1-8, 43-57.

Envisioning Legal Mitigations for Intentional and Unintentional Harms Associated with Large Language Models (Extended Abstract)
Inyoung Cheong, Aylin Caliskan, & Tadayoshi Kohno.
Read the whole thing.

Protecting Visual Artists from Generative AI: An Interdisciplinary Perspective (Extended Abstract)
Eunseo Dana Choi.
Read the whole thing.

Guiding large language models to write legal treatises (Working Draft)
Colin Doyle.
Read the whole thing.

Bonus reading, an article to help you think through your topics for the final paper in this class:
Martha Minow, Archetypal Legal Scholarship: A Field Guide, 63 J. L. Ed. 65 (2013).
Read however much you’d like to help you focus your final paper.

Lesson 09 - Oct. 24

Generative A.I. & the Legal Profession

What are the possible benefits and drawbacks of large language models like GPT-4 for legal education and the legal profession? How can law schools best integrate these technologies into legal education? How can lawyers best integrate these technologies into legal work?

How might the widespread adoption of large language models in the legal profession affect the job market for lawyers and other legal professionals? Will these technologies complement or replace traditional legal roles?

To what extent should large language models be regulated within the legal profession? Can large language models be held accountable for providing incorrect or misleading legal advice? How can we ensure that AI technologies maintain the ethical standards of the legal profession?

Readings:

DOWNLOAD ALL READINGS FOR LESSON 09

How to Use Large Language Models for Empirical Legal Research (Working Draft)
Jonathan H. Choi
Read the whole thing.

Chain Of Reference prompting helps LLM to think like a lawyer (Extended Abstract)
Aditya Kuppa, Nikon Rasumov-Rahe, Marc Voses
Read the whole thing.

AI Assistance in Legal Analysis: An Empirical Study (Working Draft)
Jonathan H. Choi & Daniel Schwarcz
Read pages 1-35.

Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence (Working Draft)
Shakked Noy & Whitney Zhang
Read the whole thing.

Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence (Working Draft)
John J. Nay et al.
Read the whole thing.

Lesson 10 - Oct. 31

Regulating A.I.

This class will exam legal regimes that could be used to govern or regulate the use of algorithms. As A.I. and machine learning proliferates, what regulations are required? Are our institutions up to the task or is technology’s disruptive power overblown? Where might governmental oversight succeed and where might it fail? What would underegulation and overregulation look like in this space? How could we measure it? What existing rights should the government protect? What new rights ought to be protected in an age of automation?

Readings:

DOWNLOAD ALL READINGS FOR LESSON 10

How to regulate algorithmic decision‐making: A framework of regulatory requirements for different applications
Tobis D. Krafft et al., 16 Regulation & Governance 119 (2022).
Read the whole thing.

A comprehensive and distributed approach to AI regulation
Alex Engler, Brookings Institute (2023).
Read the whole thing.

The EU AI Act: A Primer
Tessa Baker, Center for Security and Emerginy Technology (2023).
Read the whole thing.

In U.S., Regulating A.I. Is in Its ‘Early Days’
Cecilia Kang, N.Y. Times (2023).
Read the whole thing.

The AI rules that US policymakers are considering, explained
Dylan Matthews, Vox (2023).
Read the whole thing.

AI Regulation in the U.S.: What’s Coming, and What Companies Need to Do in 2023
Dan Felz, Kim Petetti, & Alysa Austin, Alston & Bird (2022).
Read the whole thing.

A Hiring Law Blazes a Path for A.I. Regulation
Steve Lohr, N.Y. Times (2023).
Read the whole thing.

Lesson 11 - Nov. 7

Algorithmic Imagination

In our final discussion-based class we’ll be examining a set of articles and projects that reconceive the role that algorithms can play in law and society. The readings for this week include art installations, parody websites, thought experiments, and proposals for practical applications.

As you work your way through the readings this week, take note of what critiques and perspectives resonate with you — even if you disagree with the broader argument of an article. In our class discussion, we’ll continue the work of imagining algorithmic alternatives to the status quo.

Readings:

DOWNLOAD ALL READINGS FOR LESSON 11

On Missing Data Sets
Mimi Onuoha, Github Repository (2018).
Read the whole thing.
https://github.com/MimiOnuoha/missing-datasets

White Collar Risk Zones
Sam Lavigne, et al., The New Inquiry, (2017).
Visit the website
And read the white paper.

Against Predictive Optimization: On the Legitimacy of Decision-Making Algorithms that Optimize Predictive Accuracy
Angelina Wang, et al. (2023).
Read pgs. 1-22.

Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models
Shawn Shan, et al., arXiv (2023).
Read the whole thing. Skim the technical parts and focus on the legal component: “8 Poison Attacks for Copyright Protection.”

JusticeBot: A Methodology for Building Augmented Intelligence Tools for Laypeople to Increase Access to Justice
Hannes Westermann & Karim Benyekhelf, arXiv (2023).
Read the whole thing.

Lesson 12 - Nov. 14

Student Presentations (No reading assignment)

Lesson 13 - Nov. 21

Student Presentations (No reading assignment)

Last updated on Nov 1, 2023