EAI Papers — Derek Leben

Ethics & AI Paper Topics

Paper 1

Write a 6-8 page paper defending a detailed policy about one of the topics below. Please focus on ONE ethical issue (autonomy, explainability, discrimination), using ONE normative theory to justify your policy. You may consider alternative theories BRIEFLY in objections (but objections do not need to contain theories).

1. Data Ownership

Your firm has trained a LLM on data from platforms such as Reddit, Facebook, and the Poetry Foundation. The firm has reached an agreement to make some payment to the platforms which has satisfied them. All the firm's lawyers agree that it is legally permissible to use the copyrighted poems to train the model, so long as those poems do not appear in the output of the model.

However, many users on these platforms are complaining that they did not consent to this, and are not gaining compensation from the profit your company is generating from the LLM. What should your company's response be, and generally, what should the company's policy be about data used from "public" spaces to train AI models? What are the implications of this policy to using data from artists, musicians, and other professionals?

2. Explainability:

Your firm is offering small personal loans to applicants via an app on their phone. The app uses data from the phone, along with hundreds of other data points, to make a decision about the terms of the loan (e.g., amount, interest rate, etc.). All applicants have provided a blanket consent for usage of all their data, and the firm's lawyers agree that it is in compliance with laws like the GDPR and ECOA to simply tell customers that the decision was "made by a very complex algorithm."

Many customers have complained that they don't understand the reasons for rejection, or why they didn't get better terms. Currently, the firm is evaluating applicants with a model based on a neural network that makes it very difficult to interpret the reasons why particular decisions were reached. What should your company's response be, and generally, what should a company's policy be about what sort of explanations are owed to customers? What are the implications of this policy to explanations in other areas (e.g., marketing and hiring)?

3. Discrimination

In the same scenario as above, your firm has applied an xAI method to the lending model, and discovered that the most important features are: (1) the frequency of phone calls that a person makes during regular business hours, (2) how long they charge their phone at night, and (3) the regularity of their geolocation data. The data scientists who designed the model are just as surprised as anyone else that these features are most significant; the machine learning procedure simply "discovered" that these were the most statistically relevant to loan repayment. The firm's lawyers have assured you that there's nothing illegal about using this sort of data, since these are not traditionally "protected" features, and all applicants have consented to its usage.

However, many stakeholders are concerned that this is still a form of discrimination, in the sense that they appear to be arbitrary or irrelevant to the decision being made. What should your company's response be, and generally, what should a company's policy be about which features in a model are arbitrary or irrelevant?

If you would like to write about another topic, you must discuss it with me to get permission first.

The paper is argumentative, which means you will present a clear thesis (which should be obvious from p.1) in the form of a corporate policy, and then you will justify that policy in a way that is designed to convince the skeptical reader. Imagine that you are trying to convince people who take the opposite position. You should identify a range of positions/policies which disagree with your own, and make it clear why we should adopt your recommendation. If you are arguing for a policy which no reasonable person could disagree with, it's not an interesting or substantive ethical debate.

I expect you to make use of the required and optional readings from the weekly topic that you are discussing, as well as relevant secondary sources from academic sources. Sources like Wikipedia and YouTube may be a good starting point, but they are not rigorous and reliable source material for a term paper. For ethics materials, the Stanford Encyclopedia of Philosophy is an excellent peer-reviewed resource written by experts in the field, which you can cite properly and use as a launching point for bibliographic materials.

Feel free to schedule zoom meetings with me to talk about outlines and drafts. I do not read drafts that you email me (unless there is a specific question associated with it), I will read through it with you and discuss it.

The paper content must be a minimum of 6 double-spaced pages, and a maximum of 8 pages. This does not include a title page, header, and references. You may use any formatting or citation style (e.g., MLA, APA, footnotes or endnotes, etc.), as long as it is consistent throughout. If you are using LaTex, please only submit the output file (in PDF). Whatever document software you are using, I strongly recommend submitting the file to Canvas as a PDF to preserve all formatting.

The papers will be run through a "TurnItIn" function which provides a "similarity score" to other published materials and student papers. I will look through papers with a similarity score higher than 10% to check for plagiarism. Please consult the academic integrity guidelines and the syllabus for what counts as plagiarism; this is taken very seriously.

Late papers will be assigned a penalty of 3 points per day after the deadline.

Finally: do not procrastinate! It is traditional and common for students (and all humans) to procrastinate until the last minute on these sorts of tasks. However, the quality of the paper is, up to a very high grade, usually proportional to the amount of time you put into it. Almost without fail, the more time you put in, the better it will be. This means early outlines, drafts, mock debates with family and friends, visits to office hours, etc.

Paper 2

Write a 6-8 page paper defending a detailed policy about one of the topics below. Please focus on ONE ethical issue (fairness, benefit, responsibility), using ONE normative theory to justify your policy. You may consider alternative theories BRIEFLY in objections (but objections do not need to contain theories).

1. Fairness:

You work at a healthcare firm called Optum, developing models for assigning risk to patients, as well as making recommendations for resource allocation under budget constraints. These models have recently been accused of having different recall rates for Black patients compared with White patients (see here. and here.). However, the overall accuracy and predictive rates for both groups are equal, and the model does not use race as a feature in the data. Does the firm have an obligation to mitigate these models to ensure equal rates of recall across groups? Should these models be held to disparate impact standards?

2. Safety (and Fairness)

In the same scenario as above, it turns out that in order to ensure equality of recall, some decrease in accuracy is needed. It may even be the case, as Mittelstadt et al argue., that the decreases in accuracy are entirely for the disadvantaged group. Assuming this is the case, is greater fairness justified in this case at the expense of less accuracy? What if the accuracy is still greater than human performance in this domain? What level of performance should be considered "minimally safe" for models in this domain, and how should this be evaluated?

3. Liability

In response to the controversy above, Optum has issued a statement which says:

"this model is just one of many data elements intended to be used to select patients for clinical engagement programs, including, most importantly, the doctor's expertise and knowledge of his or her patient's individual need."

This statement implies that the company is not responsible for any unfairness or inaccuracies of the model, but it is entirely the responsibility of the users. Is this a permissible corporate policy on responsibility/liability, where the firm takes no responsibility for the performance of the model? If not, then what level of responsibility/liability should the firm adopt with respect to the performance of models which are designed to be used by professionals in high-risk domains?

If you would like to write about another topic, you must discuss it with me to get permission first.

Late papers will be assigned a penalty of 3 points per day after the deadline.