Resources
Responsible data management
J. Stoyanovich, S. Abiteboul, B. Howe, H. V. Jagadish, and S. Schelter
Communications of the ACM, Vol 65, Issue 6 (2022) read online
Resume format, Linkedin urls and other unexpected influences on AI personality prediction in hiring: Results of an audit
A. K. Rhea, K. Markey, L. D’Arinzo, H. Schellmann, M. Sloane, P. Squires, and J. Stoyanovich
Proceedings of the Fifth AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2022, forthcoming
An external stability audit framework to test the validity of personality prediction in AI hiring
A. K. Rhea, K. Markey, L. D’Arinzo, H. Schellmann, M. Sloane, P. Squires, F. Arif Khan, and J. Stoyanovich
Data Mining and Knowledge Discovery, Special Issue on Bias and Fairness in AI, 2022, forthcoming
Fairness in Ranking, Part I: Score-based Ranking
Meike Zehlike, Ke Yang & Julia Stoyanovich
ACM Computer Surveys (2022) read online
Fairness in Ranking, II: Learning-to-Rank and Recommender Systems
Meike Zehlike, Ke Yang & Julia Stoyanovich
ACM Computer Surveys (2022) read online
It’s Just Not That Simple: An Empirical Study of the Accuracy-Explainability Trade-off in Machine Learning for Public Policy
Andrew Bell, Ian Solano-Kamaiko, Oded Nov & Julia Stoyanovich
FAccT ’22: 2022 ACM Conference on Fairness, Accountability, and Transparency (2022) read online
German AI Start-Ups and “AI Ethics”: Using A Social Practice Lens for Assessing and Implementing Socio-Technical Innovation
Mona Sloane & Janina Zakrzewski
FAccT ’22: 2022 ACM Conference on Fairness, Accountability, and Transparency (2022) read online
Teaching Responsible Data Science
Julia Stoyanovich
DataEd ’22: 1st International Workshop on Data Systems Education (2022) read online
Towards substantive conceptions of algorithmic fairness: Normative guidance from equal opportunity doctrines
Falaah Arif Khan, Eleni Manis, Julia Stoyanovich
CoRR, vol. abs/2207.02912, (2022) read online
Spending Privacy Budget Fairly and Wisely
Lucas Rosenblatt, Joshua Allen, Julia Stoyanovich
CoRR, vol. abs/2204.12903, (2022) read online
Think About the Stakeholders First! Towards an Algorithmic Transparency Playbook for Regulatory Compliance
Andrew Bell, Oded Nov, Julia Stoyanovich
CoRR, vol. abs/2207.01482, (2022) read online
Covid-19 Brings Data Equity Challenges to the Fore
H.V. Jagadish, Julia Stoyanovich & Bill Howe
ACM Digital Government Research and Practice (2021) read online
Disaggregated Interventions to Reduce Inequality
Lucius Bynum, Joshua R. Loftus & Julia Stoyanovich
EAAMO’21, ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (2021) read online
Lightweight Inspection of Data Preprocessing In Native Machine Learning Pipelines
Stefan Grafberger, Julia Stoyanovich & Sebastian Schelter
CIDR’21, 11th Conference on Innovative Data Systems Research (2021) read online
Causal Intersectionality and Fair Ranking
Ke Yang, Joshua R. Loftus & Julia Stoyanovich
FORC’21, 2nd Symposium on Foundations of Responsible Computing (2021) read online
Comparing Apples and Oranges: Fairness and Diversity in Ranking
Julia Stoyanovich
ICDT’21, 24th International Conference on Database Theory (2021) read online watch the talk
The Imperative of Interpretable Machines
Julia Stoyanovich, Jay Van Bavel & Tessa V. West
Nature Machine Intelligence (2020) read online
Responsible Data Management
Julia Stoyanovich, Bill Howe & H.V. Jagadish
Proceedings of VLDB Endowment (2020) read online
Taming Technical Bias in Machine Learning Pipelines
Sebastian Schelter & Julia Stoyanovich
IEEE Data Engineering Bulletin (2020) read online
FairPrep: Promoting Data to a First-Class Citizen in Studies on Fairness-Enhancing Interventions
Sebastian Schelter, Yuxuan He, Jatin Khilnani & Julia Stoyanovich
EDBT’20, 23rd International Conference on Extending Database Technology (2020) read online
Balanced Ranking with Diversity Constraints
Ke Yang, Vasilis Gkatzelis & Julia Stoyanovich
Proceedings of IJCAI (2019) read online
Designing Fair Ranking Schemes
Abolfazl Asudeh, H. V. Jagadish, Julia Stoyanovich & Gautam Das
Proceedings of ACM SIGMOD (2019) read online
Mithra Ranking: A System for Responsible Ranking Design
Yifan Guan, Abolfazl Asudeh, Pranav Mayuram, H. V. Jagadish, Julia Stoyanovich, Gerome Miklau & Gautam Das
Proceedings of ACM SIGMOD (2019) read online
Transparency, Fairness, Data Protection, Neutrality: Data Management in the Face of New Regulation
Serge Abiteboul & Julia Stoyanovich
ACM Journal of Data and Information Quality (2019) read online
Nutritional Labels for Data and Models
Julia Stoyanovich & Bill Howe
IEEE Data Engineering Bulletin 42(3): 13-23 (2019) read online
Towards Responsible Data-Driven Decisions Making in Score-Based Systems
Abolfazl Asudeh, H. V. Jagadish & Julia Stoyanovich
IEEE Data Engineering Bulletin 42(3): 76-87 (2019) read online
The Responsibility Challenge for Data
H. V. Jagadish, Francesco Bonchi, Tina Eliassi-Rad, Lise Getoor, Krishna P. Gummadi & Julia Stoyanovich
Proceedings of ACM SIGMOD (2019) read online
TransFAT: Translating Fairness, Accountability, and Transparency into Data Science Practice
Julia Stoyanovich
International Workshop on Processing Information Ethically (2019) read online
AI Procurement Primer
Mona Sloane, Rumman Chowdhury, John C. Havens, Tomo Lazovich & Luis C. Rincon Alba (2021)
Public Engagement Showreel Int 1894
Julia Stoyanovich, Steven Kuyan, Meghan McDermott, Maria Grillo & Mona Sloane (2020)
We Are AI
We Are AI is a 5-week learning circle course that provides an introduction to the basics of AI and the social and ethical dimensions of the use of AI in modern life.
Responsible Data Science Courses at the NYU Center for Data Science
Responsible Data Science is a technical course for graduate and undergraduate students at NYU that tackles the issues of ethics, legal compliance, data quality, algorithmic fairness and diversity, transparency of data and algorithms, privacy, and data protection.
AI Ethics: Global Perspectives
This course is designed to raise awareness of the societal impacts of technology and to give individuals and institutions the tools to pursue responsible AI use.
Data Responsibly Series
Vol. 1: Mirror, Mirror
Falaah Arif Khan and Julia Stoyanovich
Vol. 1: Miroir, mon beau Miroir (French Edition)
Falaah Arif Khan and Julia Stoyanovich
Translated by Serge Abiteboul, Pascal Guitton, Victor Vianu, and Eve Francois Trement
Vol. 1: Espejito, Espejito (Spanish Edition)
Falaah Arif Khan and Julia Stoyanovich
Translated by Charles Schroeder and Ana Elisa Mendez
Vol. 2: Fairness and Friends
Falaah Arif Khan, Eleni Manis and Julia Stoyanovich
We Are AI Series
Vol. 1: What is AI?
Somos IA #1: “¿Qué es la IA?” (Spanish Edition)
Julia Stoyanovich and Falaah Arif Khan
Vol. 2: Learning from Data
Somos IA #2: Aprender de Los Datos (Spanish Edition)
Julia Stoyanovich and Falaah Arif Khan
Vol. 3: Who Lives, Who Dies, Who Decides?
Somos IA #3: ¿Quién vive, quién muere, quién decide? (Spanish Edition)
Julia Stoyanovich, Mona Sloane and Falaah Arif Khan
Vol. 4: All About that Bias
Somos IA #4: Todo Sobre Ese Sesgo (Spanish Edition)
Julia Stoyanovich and Falaah Arif Khan
Tools
Ranking Facts
Ranking Facts is a nutritional label for rankings. The tool allows you to upload data, design a ranking methodology, and understand the fairness, diversity, and stability of the result. Use it through a web-based UI, or incorporate it into your python code.
Fair Prep
FairPrep is a design and evaluation framework for fairness-enhancing interventions that treats data as a first-class citizen in complex data science pipelines.
Data Synthesizer
The Data Synthesizer is a tool that generates privacy-preserving synthetic data, using the framework of differential privacy. Synthetic data can facilitate collaborations between data scientists and owners of sensitive data, and it can support algorithmic transparency and disclosure. Use it through a web-based UI, or incorporate it into your python code.
MLinspect
MLInspect is a data distribution debugger. Use it to tame technical bias in your complex data science pipelines that use common libraries like pandas and scikit-learn.