OpenAI is funding academic research focused on developing algorithms capable of predicting human moral judgments.
In a recent filing with the IRS, OpenAI Inc., the for-profit arm of OpenAI, revealed that it had granted funding to researchers at Duke University for a project titled "Research AI Morality." An OpenAI spokesperson confirmed that this grant is part of a larger, three-year initiative, totaling $1 million, aimed at helping Duke professors explore the development of "moral AI."
Details about the "morality" research funded by OpenAI remain scarce, with the only known information being that the grant will conclude in 2025. The study's principal investigator, Walter Sinnott-Armstrong, a professor of practical ethics at Duke University, told TechCrunch via email that he "will not be able to talk" about the project.
Sinnott-Armstrong, along with the project's co-investigator, Jana Borg, has authored multiple studies and a book exploring AI's potential as a “moral GPS” to assist humans in making better decisions. As part of larger research teams, they have developed a “morally-aligned” algorithm to determine recipients for kidney donations and examined scenarios in which people would prefer AI to make moral decisions.
The press release states that the aim of the OpenAI-funded research is to develop algorithms capable of "predicting human moral judgments" in situations where there are conflicts "among morally relevant features in medicine, law, and business."
However, it remains uncertain whether a complex concept like morality is within the grasp of current technology.
In 2021, the nonprofit Allen Institute for AI developed a tool called Ask Delphi, designed to provide ethically sound recommendations. While it correctly judged basic moral dilemmas—such as recognizing that cheating on an exam was wrong—the system was easily manipulated. By slightly rephrasing or rewording questions, users could get Delphi to approve of nearly anything, including extreme actions like smothering infants.
The challenge lies in how modern AI systems function. Machine learning models are statistical tools, trained on vast amounts of data from the web, learning patterns to make predictions—like how the phrase “to whom” often precedes “it may concern.” However, AI does not comprehend ethical concepts or the reasoning and emotions involved in moral decision-making. This limitation results in AI reflecting the values dominant in Western, educated, and industrialized nations, as these perspectives dominate the web and AI’s training data.
As a result, many people's values may not align with AI's answers, especially if they are not contributing to the online content that shapes the AI's training. Beyond this, AI often internalizes biases, such as when Delphi ranked being straight as more "morally acceptable" than being gay.
The difficulty for OpenAI and the researchers it supports is compounded by the inherently subjective nature of morality. Philosophers have debated ethical theories for millennia, and there is still no universally accepted framework. For instance, AI systems like Claude tend to favor Kantianism (emphasizing absolute moral rules), while ChatGPT leans towards utilitarianism (prioritizing the greatest good for the most people). Which approach is superior? That depends on who you ask.
Developing an algorithm capable of predicting human moral judgments will require addressing all these complexities—setting an extraordinarily high bar, assuming such an algorithm is even achievable.
(Source: TechCrunch)
BD-Pratidin English/Mazdud
