Reverse-Engineering the Moral Mind​​

Can approaches from engineering help us understand how moral reasoning works? Joshua Tenenbaum, Max Kleiman-Weiner and Sydney Levine seek to give humans cutting-edge tools from computer science to understand how we make decisions of right and wrong.

This project defines moral intelligence as the unique human ability for rapid and robust learning as well as generalizing moral knowledge across infinite situations and people. Throughout the ages, moral philosophers have proposed distinct accounts of moral judgment. Their theories—e.g., deontology, consequentialism, and contractualism—often seem at odds with each other.

This project hypothesizes that by combining aspects of these three theories in a unified computational model, they can act as the foundational principles for a reverse-engineering account of human moral thinking that is both predictive and interpretable. To test this hypothesis, the team’s main activities combine building formal models of moral reasoning that combine recent advances in artificial intelligence with computational cognitive modeling. In large-scale online experiments, they will conduct empirical tests on the model’s fine-grained predictions. These models point towards ways people and AI can communicate with each other and work together in morally charged scenarios.

Outputs will include workshops and publications aimed at bi-directional impact between the scientific study of human morality and industrial applications. Key outcomes of the research will be twofold:

1. Increasing the philosophical sophistication of AI researchers and the prevalence of a computational approach among moral psychologists and philosophers.
2. Expanding our ethical capacity by enabling machines to interact with us as partners in thought by strengthening the role of human judgment in industrial deployments of AI technology.

Together, this work will ensure that the power and flexibility of human morality remains robust even in the machine age.