Below, you will find an extensive compilation of our academic
publications and software artifacts, providing a window into our
scholarly contributions. Each entry includes an external link to the
full text of the paper for a comprehensive understanding.
In addition, you can explore our ongoing technology policy and
education projects, complete with detailed descriptions and direct
links to the project repositories. We hope that this collection of
resources will serve as a valuable tool for researchers, students, and
the broader community to engage with our work.
Data-centric AI and responsible data management
Incorporating ethics and legal
compliance into data-driven algorithmic systems has been attracting
significant attention from the computing research community, most
notably under the umbrella of fair and interpretable machine
learning. While important, much of this work has been limited in scope
to the last mile of data analysis and has disregarded both the
system’s design, development, and use life cycle (What are we
automating and why? Is the system working as intended? Are there any
unforeseen consequences post-deployment?) and the data life cycle
(Where did the data come from? How long is it valid and
appropriate?). Our work on data-centric responsible AI and on
responsible data management is based on the observation that the
decisions we make during data collection and preparation profoundly
impact the robustness, fairness, and interpretability of the systems
we build.
2026
Seasoning Data Modeling Education with GARLIC: A Participatory
Co-Design Framework
Viktoriia Makovska, Ihor Michurin, Mariia Tokhtamysh, George Fletcher, and Julia Stoyanovich
Proceedings of the DataEd Workshop, co-located with EDBT/ICDT 2026
@article{garlic,author={Makovska, Viktoriia and Michurin, Ihor and Tokhtamysh, Mariia and Fletcher, George and Stoyanovich, Julia},title={Seasoning Data Modeling Education with {GARLIC:} {A} Participatory
Co-Design Framework},journal={Proceedings of the DataEd Workshop, co-located with EDBT/ICDT},volume={abs/2602.18274},year={2026},keywords={RAIforUkraine,data,explainability,transparency,education},}
Memory Undone: Between Knowing and Not Knowing in Data Systems
Viktoriia Makovska, George Fletcher, Julia Stoyanovich, and Tetiana Zakharchenko
@article{memory,author={Makovska, Viktoriia and Fletcher, George and Stoyanovich, Julia and Zakharchenko, Tetiana},title={Memory Undone: Between Knowing and Not Knowing in Data Systems},journal={CoRR},volume={abs/2602.21180},year={2026},keywords={RAIforUkraine,data,explainability,transparency},}
Explanation Multiplicity in SHAP: Characterization and Assessment
Hyunseung Hwang, Seungeun Lee, Lucas Rosenblatt, Steven Euijong Whang, and Julia Stoyanovich
@article{hwang2026explanationmultiplicity,title={Explanation Multiplicity in {SHAP}: Characterization and Assessment},author={Hwang, Hyunseung and Lee, Seungeun and Rosenblatt, Lucas and Whang, Steven Euijong and Stoyanovich, Julia},year={2026},eprint={2601.12654},archiveprefix={arXiv},journal={CoRR},primaryclass={cs.LG},keywords={data,explainability,transparency},}
2025
VirnyFlow: A Design Space for Responsible Model Development
Denys Herasymuk, Nazar Protsiv, and Julia Stoyanovich
@article{herasymuk2025virnyflow,title={{VirnyFlow}: A Design Space for Responsible Model Development},author={Herasymuk, Denys and Protsiv, Nazar and Stoyanovich, Julia},year={2025},eprint={2506.01584},archiveprefix={arXiv},journal={CoRR},primaryclass={cs.LG},keywords={working,RAIforUkraine,data}}
Still More Shades of Null: An Evaluation Suite for Responsible Missing Value Imputation
Falaah Arif Khan, Denys Herasymuk, Nazar Protsiv, and Julia Stoyanovich
@article{shades,title={Still More Shades of {Null}: An Evaluation Suite for Responsible Missing Value Imputation},author={{Arif Khan}, Falaah and Herasymuk, Denys and Protsiv, Nazar and Stoyanovich, Julia},year={2025},journal={Proc. {VLDB} Endow.},volume={18},number={9},doi={14778/3746405.3746416},keywords={journal,RAIforUkraine,data}}
ShaRP: Explaining Rankings and Preferences with Shapley Values
Venetia Pliatsika, João Fonseca, Kateryna Akhynko, Ivan Shevchenko, and Julia Stoyanovich
@article{sharp,author={Pliatsika, Venetia and Fonseca, Jo{\~{a}}o and Akhynko, Kateryna and Shevchenko, Ivan and Stoyanovich, Julia},title={{ShaRP}: Explaining Rankings and Preferences with Shapley Values},journal={Proc. {VLDB} Endow.},volume={18},number={11},year={2025},doi={10.14778/3749646.3749682},keywords={journal,data,ranking,explainability,transparency,RAIforUkraine}}
SHAP-based Explanations are Sensitive to Feature Representation
Hyunseung Hwang, Andrew Bell, Joao Fonseca, Venetia Pliatsika, Julia Stoyanovich, and Steven Euijong Whang
In Conference on Fairness, Accountability, and Transparency, ACM FAccT 2025
@inproceedings{sharp_attack,title={{SHAP}-based Explanations are Sensitive to Feature Representation},author={Hwang, Hyunseung and Bell, Andrew and Fonseca, Joao and Pliatsika, Venetia and Stoyanovich, Julia and Whang, Steven Euijong},year={2025},booktitle={Conference on Fairness, Accountability, and Transparency, {ACM FAccT}},keywords={conference,data,explainability,transparency}}
CREDAL: Close Reading of Data Models
George Fletcher, Olha Nahurna, Matvii Prytula, and Julia Stoyanovich
In Proceedings of the Workshop on Human-In-the-Loop Data Analytics (HILDA) at ACM SIGMOD 2025
@inproceedings{credal2025,title={{CREDAL}: Close Reading of Data Models},author={Fletcher, George and Nahurna, Olha and Prytula, Matvii and Stoyanovich, Julia},year={2025},booktitle={Proceedings of the Workshop on Human-In-the-Loop Data Analytics (HILDA) at ACM SIGMOD},url={https://dl.acm.org/doi/10.1145/3736733.3736737},keywords={data,RAIforUkraine}}
ONION: A Multi-Layered Framework for Participatory ER Design
Viktoriia Makovska, George Fletcher, and Julia Stoyanovich
In Proceedings of the Workshop on Human-In-the-Loop Data Analytics (HILDA) at ACM SIGMOD 2025
@inproceedings{onion20205,title={{ONION}: A Multi-Layered Framework for Participatory ER Design},author={Makovska, Viktoriia and Fletcher, George and Stoyanovich, Julia},year={2025},booktitle={Proceedings of the Workshop on Human-In-the-Loop Data Analytics (HILDA) at ACM SIGMOD},url={https://dl.acm.org/doi/10.1145/3736733.3736736},keywords={data,education,RAIforUkraine}}
Do You Really Need Public Data? Surrogate Public Data for Differential Privacy on Tabular Data
Shlomi Hod, Lucas Rosenblatt, and Julia Stoyanovich
The Thirty-Ninth Annual Conference on Neural Information Processing Systems (NeurIPS), Datasets & Benchmarks Track 2025
@article{hod2025reallyneedpublicdata,title={Do You Really Need Public Data? Surrogate Public Data for Differential Privacy on Tabular Data},author={Hod, Shlomi and Rosenblatt, Lucas and Stoyanovich, Julia},year={2025},journal={The Thirty-Ninth Annual Conference on Neural Information Processing Systems (NeurIPS), Datasets \& Benchmarks Track},keywords={privacy, data, genAI},}
2024
Rodeo: Making Refinements for Diverse Top-k Queries
Felix S. Campbell, Julia Stoyanovich, and Yuval Moskovitch
@article{DBLP:journals/pvldb/CampbellSM24,author={Campbell, Felix S. and Stoyanovich, Julia and Moskovitch, Yuval},title={Rodeo: Making Refinements for Diverse Top-k Queries},journal={Proc. {VLDB} Endow.},volume={17},number={12},pages={4341--4344},year={2024},keywords={conference,fairness,data,demo,ranking}}
Responsible Model Selection with Virny and VirnyView
Denys Herasymuk, Falaah Arif Khan, and Julia Stoyanovich
In Companion of the International Conference on Management of Data,
SIGMOD/PODS, Santiago, Chile 2024
@inproceedings{DBLP:conf/sigmod/HerasymukKS24,author={Herasymuk, Denys and Khan, Falaah Arif and Stoyanovich, Julia},editor={Barcel{\'{o}}, Pablo and Pi, Nayat S{\'{a}}nchez and Meliou, Alexandra and Sudarshan, S.},title={Responsible Model Selection with Virny and VirnyView},booktitle={Companion of the International Conference on Management of Data,
{SIGMOD/PODS}, Santiago, Chile},pages={488--491},publisher={{ACM}},year={2024},url={https://doi.org/10.1145/3626246.3654738},doi={10.1145/3626246.3654738},keywords={conference,demo,RAIforUkraine,data},author+an={1=self;2=self;3=self}}
Query Refinement for Diverse Top-k Selection
Felix S. Campbell, Alon Silberstein, Julia Stoyanovich, and Yuval Moskovitch
@article{DBLP:journals/pacmmod/CampbellSSM24,author={Campbell, Felix S. and Silberstein, Alon and Stoyanovich, Julia and Moskovitch, Yuval},title={Query Refinement for Diverse Top-k Selection},journal={Proc. {ACM} Manag. Data},volume={2},number={3},pages={166},year={2024},doi={10.1145/3654969},timestamp={Fri, 19 Jul 2024 23:17:58 +0200},biburl={https://dblp.org/rec/journals/pacmmod/CampbellSSM24.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,fairness,data},}
2023
Query Refinement for Diversity Constraint Satisfaction
Jinyang Li, Yuval Moskovitch, Julia Stoyanovich, and H. V. Jagadish
@article{DBLP:journals/pvldb/LiMSJ23,author={Li, Jinyang and Moskovitch, Yuval and Stoyanovich, Julia and Jagadish, H. V.},title={Query Refinement for Diversity Constraint Satisfaction},journal={Proc. {VLDB} Endow.},volume={17},number={2},pages={106--118},year={2023},timestamp={Mon, 27 Nov 2023 13:13:34 +0100},biburl={https://dblp.org/rec/journals/pvldb/LiMSJ23.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,fairness,data,ranking},}
ERICA: Query Refinement for Diversity Constraint Satisfaction
Jinyang Li, Alon Silberstein, Yuval Moskovitch, Julia Stoyanovich, and H. V. Jagadish
@article{DBLP:journals/pvldb/LiSMSJ23,author={Li, Jinyang and Silberstein, Alon and Moskovitch, Yuval and Stoyanovich, Julia and Jagadish, H. V.},title={{ERICA:} Query Refinement for Diversity Constraint Satisfaction},journal={Proc. {VLDB} Endow.},volume={16},number={12},pages={4070--4073},year={2023},doi={10.14778/3611540.3611623},timestamp={Mon, 23 Oct 2023 16:16:16 +0200},biburl={https://dblp.org/rec/journals/pvldb/LiSMSJ23.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,fairness,data},}
Subset Modelling: A Domain Partitioning Strategy for Data-efficient
Machine-Learning
Vı́tor Ribeiro, Eduardo H. M. Pena, Raphael Freitas Saldanha, Reza Akbarinia, Patrick Valduriez, Falaah Arif Khan, Julia Stoyanovich, and Fábio Porto
In Proceedings of the 38th Brazilian Symposium on Databases, SBBD 2023,
Belo Horizonte, MG, Brazil, September 25-29, 2023 2023
@inproceedings{DBLP:conf/sbbd/RibeiroPSAVKS023,author={Ribeiro, V{\'{\i}}tor and Pena, Eduardo H. M. and de Freitas Saldanha, Raphael and Akbarinia, Reza and Valduriez, Patrick and {Arif Khan}, Falaah and Stoyanovich, Julia and Porto, F{\'{a}}bio},title={Subset Modelling: {A} Domain Partitioning Strategy for Data-efficient
Machine-Learning},booktitle={Proceedings of the 38th Brazilian Symposium on Databases, {SBBD} 2023,
Belo Horizonte, MG, Brazil, September 25-29, 2023},pages={318--323},publisher={{SBC}},year={2023},timestamp={Wed, 25 Oct 2023 16:14:33 +0200},biburl={https://dblp.org/rec/conf/sbbd/RibeiroPSAVKS023.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,data},}
Automated Data Cleaning Can Hurt Fairness in Machine Learning-based Decision Making
Shubha Guha, Falaah Arif Khan, Julia Stoyanovich, and Sebastian Schelter
In Proceedings of the 39th International Conference on Data Engineering, ICDE 2023
@inproceedings{guha_23,author={Guha, Shubha and Khan, Falaah Arif and Stoyanovich, Julia and Schelter, Sebastian},year={2023},title={Automated Data Cleaning Can Hurt Fairness in Machine Learning-based Decision Making},keywords={conference,data,fair},booktitle={Proceedings of the 39th International Conference on Data Engineering, {ICDE}},}
@article{jdiq23,author={Jagadish, H.V. and Stoyanovich, Julia and Howe, Bill},title={The Many Facets of Data Equity},journal={ACM Journal of Data and Information Quality},year={2023},keywords={journal,data,fair},volume={14},issue={4},pages={1--21},doi={https://doi.org/10.1145/3533425},}
Personal Data for Personal Use: Vision or Reality?
Xin Luna Dong, Bo Li, Julia Stoyanovich, Anthony Kum Hoe Tung, Gerhard Weikum, Alon Y. Halevy, and Wang-Chiew Tan
In Companion of the 2023 International Conference on Management of Data,
SIGMOD/PODS 2023, Seattle, WA, USA, June 18-23, 2023 2023
@inproceedings{DBLP:conf/sigmod/DongLSTWHT23,author={Dong, Xin Luna and Li, Bo and Stoyanovich, Julia and Tung, Anthony Kum Hoe and Weikum, Gerhard and Halevy, Alon Y. and Tan, Wang{-}Chiew},title={Personal Data for Personal Use: Vision or Reality?},booktitle={Companion of the 2023 International Conference on Management of Data,
{SIGMOD/PODS} 2023, Seattle, WA, USA, June 18-23, 2023},pages={263--264},publisher={{ACM}},year={2023},doi={10.1145/3555041.3589378},keywords={panel,data,privacy},addendum={panelist, by invitation},}
2022
Spending Privacy Budget Fairly and Wisely
Lucas Rosenblatt, Joshua Allen, and Julia Stoyanovich
Theory and Practice of Differential Privacy (@ICML) 2022
@article{rosenblatt2022spending,title={Spending Privacy Budget Fairly and Wisely},author={Rosenblatt, Lucas and Allen, Joshua and Stoyanovich, Julia},journal={Theory and Practice of Differential Privacy (@ICML)},year={2022},keywords={differential privacy, fairness, databases},}
Responsible Data Management
Julia Stoyanovich, Serge Abiteboul, Bill Howe, H. V. Jagadish, and Sebastian Schelter
@article{DBLP:journals/cacm/StoyanovichAHJS22,author={Stoyanovich, Julia and Abiteboul, Serge and Howe, Bill and Jagadish, H. V. and Schelter, Sebastian},title={Responsible Data Management},journal={Communications of the {ACM}},volume={65},number={6},pages={64--74},year={2022},doi={10.1145/3488717},timestamp={Wed, 25 May 2022 10:40:29 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={data},author+an={1=self},}
Developing data capability with non-profit organisations using participatory methods
Anthony McCosker, Xiaofang Yao, Kath Albury, Alexia Maddox, Jane Farmer, and Julia Stoyanovich
@article{doi:10.1177/20539517221099882,author={McCosker, Anthony and Yao, Xiaofang and Albury, Kath and Maddox, Alexia and Farmer, Jane and Stoyanovich, Julia},title={Developing data capability with non-profit organisations using participatory methods},journal={Big Data \& Society},volume={9},number={1},pages={20539517221099882},year={2022},doi={10.1177/20539517221099882},eprint={https://doi.org/10.1177/20539517221099882},keywords={journal,data,policy,governance},}
@article{DBLP:journals/pvldb/StoyanovichHJ20,title={Responsible Data Management},author={Stoyanovich, Julia and Howe, Bill and Jagadish, HV},journal={Proceedings of the VLDB Endowment},volume={13},number={12},year={2020},keywords={fairness,data,journal},doi={10.14778/3415478.3415570},}
FairPrep: Promoting Data to a First-Class Citizen in Studies on Fairness-Enhancing
Interventions
Sebastian Schelter, Yuxuan He, Jatin Khilnani, and Julia Stoyanovich
In Proceedings of the 23nd International Conference on Extending Database
Technology, EDBT 2020
@inproceedings{DBLP:conf/edbt/SchelterHKS20,author={Schelter, Sebastian and He, Yuxuan and Khilnani, Jatin and Stoyanovich, Julia},title={{F}air{P}rep: Promoting Data to a First-Class Citizen in Studies on Fairness-Enhancing
Interventions},booktitle={Proceedings of the 23nd International Conference on Extending Database
Technology, {EDBT}},pages={395--398},year={2020},doi={10.5441/002/edbt.2020.41},timestamp={Wed, 25 Mar 2020 15:46:06 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,data,fair},}
Fairness-Aware Instrumentation of Preprocessing Pipelines for Machine Learning
Ke Yang, Biao Huang, Julia Stoyanovich, and Sebastian Schelter
In Proceedings of the Workshop on Human-In-the-Loop Data Analytics, HILDA at SIGMOD 2020
@inproceedings{DBLP:hilda20,author={Yang, Ke and Huang, Biao and Stoyanovich, Julia and Schelter, Sebastian},title={Fairness-Aware Instrumentation of Preprocessing~Pipelines for Machine Learning},booktitle={Proceedings of the Workshop on Human-In-the-Loop Data Analytics, {HILDA} at {SIGMOD}},year={2020},doi={https://doi.org/10.1145/3398730.3399194},keywords={workshop,data,fairness},}
Taming Technical Bias in Machine Learning Pipelines
@article{debull21,author={Schelter, Sebastian and Stoyanovich, Julia},title={Taming Technical Bias in Machine Learning Pipelines},journal={{IEEE} Data Eng. Bull.},year={2020},volume={43},number={4},keywords={journal,data,fairness},author+an={2=self},}
2019
Transparency, Fairness, Data Protection, Neutrality: Data Management
Challenges in the Face of New Regulation
@article{DBLP:journals/jdiq/AbiteboulS19,author={Abiteboul, Serge and Stoyanovich, Julia},title={Transparency, Fairness, Data Protection, Neutrality: Data Management
Challenges in the Face of New Regulation},journal={ACM Journal of Data and Information Quality},volume={11},number={3},pages={15:1--15:9},year={2019},doi={10.1145/3310231},timestamp={Tue, 20 Aug 2019 08:40:18 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,data,fairness,policy,governance},}
MithraLabel: Flexible Dataset Nutritional Labels for Responsible Data
Science
Chenkai Sun, Abolfazl Asudeh, H. V. Jagadish, Bill Howe, and Julia Stoyanovich
In Proceedings of the 28th International Conference on Information
and Knowledge Management, CIKM 2019
@inproceedings{DBLP:conf/cikm/SunAJHS19,author={Sun, Chenkai and Asudeh, Abolfazl and Jagadish, H. V. and Howe, Bill and Stoyanovich, Julia},title={MithraLabel: Flexible Dataset Nutritional Labels for Responsible Data
Science},booktitle={Proceedings of the 28th International Conference on Information
and Knowledge Management, {CIKM}},pages={2893--2896},publisher={{ACM}},year={2019},doi={10.1145/3357384.3357853},timestamp={Mon, 04 Nov 2019 11:09:32 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,data,explainability},}
The Responsibility Challenge for Data
H. V. Jagadish, Francesco Bonchi, Tina Eliassi-Rad, Lise Getoor, Krishna P. Gummadi, and Julia Stoyanovich
In Proceedings of the 2019 International Conference on the Management of
Data, SIGMOD 2019
@inproceedings{DBLP:conf/sigmod/JagadishBEGGS19,author={Jagadish, H. V. and Bonchi, Francesco and Eliassi{-}Rad, Tina and Getoor, Lise and Gummadi, Krishna P. and Stoyanovich, Julia},title={The Responsibility Challenge for Data},booktitle={Proceedings of the 2019 International Conference on the Management of
Data, {SIGMOD}},pages={412--414},publisher={{ACM}},year={2019},doi={10.1145/3299869.3314327},timestamp={Sat, 22 Jun 2019 17:10:04 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={panel,data},author+an={6=self},addendum={plenary panel co-organizer and presenter},}
TransFAT: Translating Fairness, Accountability, and Transparency into
Data Science Practice
Julia Stoyanovich
In Proceedings of the 1st International Workshop on Processing Information
Ethically, PIE at CAiSE 2019
@inproceedings{DBLP:conf/caise/Stoyanovich19,author={Stoyanovich, Julia},title={TransFAT: Translating Fairness, Accountability, and Transparency into
Data Science Practice},booktitle={Proceedings of the 1st International Workshop on Processing Information
Ethically, {PIE} at {CAiSE}},series={{CEUR} Workshop Proceedings},volume={2417},publisher={CEUR-WS.org},year={2019},timestamp={Wed, 12 Feb 2020 16:44:33 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={invited,data},author+an={1=self}}
Explainability
There is a variety of terms associated with this topic: transparency,
interpretability, explainability, intelligibility. But let’s not get
too tangled up in terminology. The main point is that we need to
allow people to understand the data, the operation, and the decisions
or predictions of an AI system, and to also understand why these
decisions or predictions are made. This understanding is critical
because it allows people to exercise agency and take control over
their interactions with AI systems. And so, no matter what
terminology we use, the overarching idea behind transparency & friends
is to expose the “knobs of responsibility” to people, as a means to
support the responsible design, development, use, and oversight of AI
systems.
2026
Seasoning Data Modeling Education with GARLIC: A Participatory
Co-Design Framework
Viktoriia Makovska, Ihor Michurin, Mariia Tokhtamysh, George Fletcher, and Julia Stoyanovich
Proceedings of the DataEd Workshop, co-located with EDBT/ICDT 2026
@article{garlic,author={Makovska, Viktoriia and Michurin, Ihor and Tokhtamysh, Mariia and Fletcher, George and Stoyanovich, Julia},title={Seasoning Data Modeling Education with {GARLIC:} {A} Participatory
Co-Design Framework},journal={Proceedings of the DataEd Workshop, co-located with EDBT/ICDT},volume={abs/2602.18274},year={2026},keywords={RAIforUkraine,data,explainability,transparency,education},}
Memory Undone: Between Knowing and Not Knowing in Data Systems
Viktoriia Makovska, George Fletcher, Julia Stoyanovich, and Tetiana Zakharchenko
@article{memory,author={Makovska, Viktoriia and Fletcher, George and Stoyanovich, Julia and Zakharchenko, Tetiana},title={Memory Undone: Between Knowing and Not Knowing in Data Systems},journal={CoRR},volume={abs/2602.21180},year={2026},keywords={RAIforUkraine,data,explainability,transparency},}
Explanation Multiplicity in SHAP: Characterization and Assessment
Hyunseung Hwang, Seungeun Lee, Lucas Rosenblatt, Steven Euijong Whang, and Julia Stoyanovich
@article{hwang2026explanationmultiplicity,title={Explanation Multiplicity in {SHAP}: Characterization and Assessment},author={Hwang, Hyunseung and Lee, Seungeun and Rosenblatt, Lucas and Whang, Steven Euijong and Stoyanovich, Julia},year={2026},eprint={2601.12654},archiveprefix={arXiv},journal={CoRR},primaryclass={cs.LG},keywords={data,explainability,transparency},}
ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations
@article{explainerPFN,title={ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations},author={Fonseca, Joao and Stoyanovich, Julia},year={2026},eprint={2601.23068},archiveprefix={arXiv},primaryclass={cs.LG},author+an={2=self},keywords={working,explainability,transparency},}
2025
ShaRP: Explaining Rankings and Preferences with Shapley Values
Venetia Pliatsika, João Fonseca, Kateryna Akhynko, Ivan Shevchenko, and Julia Stoyanovich
@article{sharp,author={Pliatsika, Venetia and Fonseca, Jo{\~{a}}o and Akhynko, Kateryna and Shevchenko, Ivan and Stoyanovich, Julia},title={{ShaRP}: Explaining Rankings and Preferences with Shapley Values},journal={Proc. {VLDB} Endow.},volume={18},number={11},year={2025},doi={10.14778/3749646.3749682},keywords={journal,data,ranking,explainability,transparency,RAIforUkraine}}
SHAP-based Explanations are Sensitive to Feature Representation
Hyunseung Hwang, Andrew Bell, Joao Fonseca, Venetia Pliatsika, Julia Stoyanovich, and Steven Euijong Whang
In Conference on Fairness, Accountability, and Transparency, ACM FAccT 2025
@inproceedings{sharp_attack,title={{SHAP}-based Explanations are Sensitive to Feature Representation},author={Hwang, Hyunseung and Bell, Andrew and Fonseca, Joao and Pliatsika, Venetia and Stoyanovich, Julia and Whang, Steven Euijong},year={2025},booktitle={Conference on Fairness, Accountability, and Transparency, {ACM FAccT}},keywords={conference,data,explainability,transparency}}
2024
Making Transparency Influencers: A Case Study of an Educational
Approach to Improve Responsible AI Practices in News and Media
Andrew Bell, and Julia Stoyanovich
In Extended Abstracts of the CHI Conference on Human Factors in Computing
Systems, CHI EA 2024, Honolulu, HI, USA, May 11-16, 2024 2024
@inproceedings{DBLP:conf/chi/BellS24,author={Bell, Andrew and Stoyanovich, Julia},editor={Mueller, Florian 'Floyd' and Kyburz, Penny and Williamson, Julie R. and Sas, Corina},title={Making Transparency Influencers: {A} Case Study of an Educational
Approach to Improve Responsible {AI} Practices in News and Media},booktitle={Extended Abstracts of the {CHI} Conference on Human Factors in Computing
Systems, {CHI} {EA} 2024, Honolulu, HI, USA, May 11-16, 2024},pages={523:1--523:8},publisher={{ACM}},year={2024},doi={10.1145/3613905.3637113},timestamp={Fri, 17 May 2024 21:42:37 +0200},biburl={https://dblp.org/rec/conf/chi/BellS24.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={explainability, transparency, policy,governance},}
2023
Introducing contextual transparency for automated decision systems
Mona Sloane, Ian Solano-Kamaiko, Jun Yuan, Aritra Dasgupta, and Julia Stoyanovich
@article{NMI_2023,author={Sloane, Mona and Solano-Kamaiko, Ian and Yuan, Jun and Dasgupta, Aritra and Stoyanovich, Julia},title={Introducing contextual transparency for automated decision systems},journal={Nature Machine Intelligence},volume={5},year={2023},pages={187-–195},doi={https://doi.org/10.1038/s42256-023-00623-7},keywords={journal,explainability,policy,hiring},}
Think About the Stakeholders First! Towards an Algorithmic Transparency
Playbook for Regulatory Compliance
@article{bell_nov_stoyanovich_2023,author={Bell, Andrew and Nov, Oded and Stoyanovich, Julia},title={Think About the Stakeholders First! {T}owards an Algorithmic Transparency
Playbook for Regulatory Compliance},volume={5},journal={Data \& Policy},publisher={Cambridge University Press},year={2023},keywords={journal,policy,explainability,education,playbook,governance},}
2022
Rankers, Rankees, & Rankings: Peeking into the Pandora’s Box
from a Socio-Technical Perspective
@article{DBLP:journals/corr/abs-2211-02932,author={Yuan, Jun and Stoyanovich, Julia and Dasgupta, Aritra},title={Rankers, Rankees, {\&} Rankings: Peeking into the Pandora's Box
from a Socio-Technical Perspective},journal={CoRR},volume={abs/2211.02932},year={2022},url={https://doi.org/10.48550/arXiv.2211.02932},doi={10.48550/arXiv.2211.02932},keywords={working,ranking,explainability},eprinttype={arXiv},eprint={2211.02932},timestamp={Wed, 09 Nov 2022 17:33:26 +0100},biburl={https://dblp.org/rec/journals/corr/abs-2211-02932.bib},bibsource={dblp computer science bibliography, https://dblp.org},}
It’s Just Not That Simple: An Empirical Study of the Accuracy-Explainability
Trade-off in Machine Learning for Public Policy
Andrew Bell, Ian Solano-Kamaiko, Oded Nov, and Julia Stoyanovich
In Proceedings of the 5th Annual ACM Conference on Fairness, Accountability, and
Transparency, FAccT 2022
@inproceedings{DBLP:conf/fat/BellSNS22,author={Bell, Andrew and Solano{-}Kamaiko, Ian and Nov, Oded and Stoyanovich, Julia},title={It's Just Not That Simple: An Empirical Study of the Accuracy-Explainability
Trade-off in Machine Learning for Public Policy},booktitle={Proceedings of the 5th Annual {ACM} Conference on Fairness, Accountability, and
Transparency, {FAccT}},pages={248--266},publisher={{ACM}},year={2022},doi={10.1145/3531146.3533090},timestamp={Wed, 22 Jun 2022 10:20:30 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={explainability,transparency,policy,governance,conference},}
2020
The Imperative of Interpretable Machines
Julia Stoyanovich, Jay J. Van Bavel, and Tessa V. West
@article{NMI_2020,author={Stoyanovich, Julia and {Van Bavel}, Jay J. and West, Tessa V.},title={The Imperative of Interpretable Machines},journal={Nature Machine Intelligence},volume={2},year={2020},pages={197--199},doi={https://doi.org/10.1038/s42256-020-0171-8},keywords={journal,explainability},}
@article{DBLP:journals/debu/StoyanovichH19,author={Stoyanovich, Julia and Howe, Bill},title={Nutritional Labels for Data and Models},journal={{IEEE} Data Eng. Bull.},volume={42},number={3},pages={13--23},year={2019},timestamp={Tue, 10 Mar 2020 16:23:50 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,explainability},}
MithraLabel: Flexible Dataset Nutritional Labels for Responsible Data
Science
Chenkai Sun, Abolfazl Asudeh, H. V. Jagadish, Bill Howe, and Julia Stoyanovich
In Proceedings of the 28th International Conference on Information
and Knowledge Management, CIKM 2019
@inproceedings{DBLP:conf/cikm/SunAJHS19,author={Sun, Chenkai and Asudeh, Abolfazl and Jagadish, H. V. and Howe, Bill and Stoyanovich, Julia},title={MithraLabel: Flexible Dataset Nutritional Labels for Responsible Data
Science},booktitle={Proceedings of the 28th International Conference on Information
and Knowledge Management, {CIKM}},pages={2893--2896},publisher={{ACM}},year={2019},doi={10.1145/3357384.3357853},timestamp={Mon, 04 Nov 2019 11:09:32 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,data,explainability},}
Fairness
Algorithmic fairness is a central topic in responsible AI. Fairness is a complex concept, and we
treat it through a socio-legal-technical lens, in ways that are domain- and context-specific. We always start
with a statement of the normative criteria (what are we trying to accomplish and why?), and propose technical solutions only as appropriate, in ways that align with the
normative criteria.
Highlights of our work include fairness in ranking, connections between equality of opportunity from political philosophy
and algoroithmic fairness, and investigating the trade-offs between fairness and other normative dimensions of responsible AI.
2025
From Benchmarking to Understanding FairML
Mykola Pechenizkiy, Hilde Weerts, Cassio Campos, Yuya Sasaki, and Julia Stoyanovich
@inproceedings{ecai25,title={From Benchmarking to Understanding {FairML}},author={Pechenizkiy, Mykola and Weerts, Hilde and de Campos, Cassio and Sasaki, Yuya and Stoyanovich, Julia},year={2025},booktitle={Proceedings of ECAI},keywords={fairness},doi={10.3233/FAIA250783},}
How Much Effort Is Enough? Fairness in Algorithmic Recourse Through the Lens of Substantive Equality of Opportunity
Andrew Bell, Joao Fonseca, Carlo Abrate, Francesco Bonchi, and Julia Stoyanovich
In Equity and Access in Algorithms, Mechanisms, and Optimization, EAAMO 2025
@inproceedings{fairrecourse,title={How Much Effort Is Enough? Fairness in Algorithmic Recourse Through the Lens of Substantive Equality of Opportunity},author={Bell, Andrew and Fonseca, Joao and Abrate, Carlo and Bonchi, Francesco and Stoyanovich, Julia},year={2025},booktitle={Equity and Access in Algorithms, Mechanisms, and Optimization, {EAAMO}},publisher={{ACM}},doi={10.1145/3757887.3763014},keywords={conference,fairness}}
2024
Rodeo: Making Refinements for Diverse Top-k Queries
Felix S. Campbell, Julia Stoyanovich, and Yuval Moskovitch
@article{DBLP:journals/pvldb/CampbellSM24,author={Campbell, Felix S. and Stoyanovich, Julia and Moskovitch, Yuval},title={Rodeo: Making Refinements for Diverse Top-k Queries},journal={Proc. {VLDB} Endow.},volume={17},number={12},pages={4341--4344},year={2024},keywords={conference,fairness,data,demo,ranking}}
A New Paradigm for Counterfactual Reasoning in Fairness and Recourse
Lucius E. J. Bynum, Joshua R. Loftus, and Julia Stoyanovich
In Proceedings of the Thirty-Third International Joint Conference on
Artificial Intelligence, IJCAI 2024, Jeju, South Korea 2024
@inproceedings{DBLP:conf/ijcai/BynumLS24,author={Bynum, Lucius E. J. and Loftus, Joshua R. and Stoyanovich, Julia},title={A New Paradigm for Counterfactual Reasoning in Fairness and Recourse},booktitle={Proceedings of the Thirty-Third International Joint Conference on
Artificial Intelligence, {IJCAI} 2024, Jeju, South Korea},pages={7092--7100},publisher={ijcai.org},year={2024},timestamp={Fri, 18 Oct 2024 20:55:14 +0200},biburl={https://dblp.org/rec/conf/ijcai/BynumLS24.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,fairness,causal,counterfactuals}}
Query Refinement for Diverse Top-k Selection
Felix S. Campbell, Alon Silberstein, Julia Stoyanovich, and Yuval Moskovitch
@article{DBLP:journals/pacmmod/CampbellSSM24,author={Campbell, Felix S. and Silberstein, Alon and Stoyanovich, Julia and Moskovitch, Yuval},title={Query Refinement for Diverse Top-k Selection},journal={Proc. {ACM} Manag. Data},volume={2},number={3},pages={166},year={2024},doi={10.1145/3654969},timestamp={Fri, 19 Jul 2024 23:17:58 +0200},biburl={https://dblp.org/rec/journals/pacmmod/CampbellSSM24.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,fairness,data},}
A Simple and Practical Method for Reducing the Disparate Impact of
Differential Privacy
Lucas Rosenblatt, Julia Stoyanovich, and Christopher Musco
In Thirty-Eighth AAAI Conference on Artificial Intelligence 2024
@inproceedings{DBLP:conf/aaai/RosenblattSM24,author={Rosenblatt, Lucas and Stoyanovich, Julia and Musco, Christopher},editor={Wooldridge, Michael J. and Dy, Jennifer G. and Natarajan, Sriraam},title={A Simple and Practical Method for Reducing the Disparate Impact of
Differential Privacy},booktitle={Thirty-Eighth {AAAI} Conference on Artificial Intelligence},pages={21554--21562},publisher={{AAAI} Press},year={2024},url={https://doi.org/10.1609/aaai.v38i19.30153},doi={10.1609/AAAI.V38I19.30153},keywords={fairness,privacy,conference},author+an={1=self;2=self},}
The Game Of Recourse: Simulating Algorithmic Recourse over Time to
Improve Its Reliability and Fairness
Andrew Bell, João Fonseca, and Julia Stoyanovich
In Companion of the 2024 International Conference on Management of Data,
SIGMOD/PODS 2024, Santiago AA, Chile, June 9-15, 2024 2024
@inproceedings{DBLP:conf/sigmod/BellFS24,author={Bell, Andrew and Fonseca, Jo{\~{a}}o and Stoyanovich, Julia},editor={Barcel{\'{o}}, Pablo and Pi, Nayat S{\'{a}}nchez and Meliou, Alexandra and Sudarshan, S.},title={The Game Of Recourse: Simulating Algorithmic Recourse over Time to
Improve Its Reliability and Fairness},booktitle={Companion of the 2024 International Conference on Management of Data,
{SIGMOD/PODS} 2024, Santiago AA, Chile, June 9-15, 2024},pages={464--467},publisher={{ACM}},year={2024},url={https://doi.org/10.1145/3626246.3654742},doi={10.1145/3626246.3654742},keywords={fairness,conference,demo},author+an={1=self;2=self;3=self},}
2023
The Possibility of Fairness: Revisiting the Impossibility Theorem
in Practice
Andrew Bell, Lucius Bynum, Nazarii Drushchak, Tetiana Zakharchenko, Lucas Rosenblatt, and Julia Stoyanovich
In Proceedings of the ACM Conference on Fairness, Accountability,
and Transparency, FAccT, Chicago, IL, USA 2023
@inproceedings{DBLP:conf/fat/BellBDZRS23,author={Bell, Andrew and Bynum, Lucius and Drushchak, Nazarii and Zakharchenko, Tetiana and Rosenblatt, Lucas and Stoyanovich, Julia},title={The Possibility of Fairness: Revisiting the Impossibility Theorem
in Practice},booktitle={Proceedings of the {ACM} Conference on Fairness, Accountability,
and Transparency, FAccT, Chicago, IL, USA},pages={400--422},publisher={{ACM}},year={2023},doi={10.1145/3593013.3594007},keywords={conference,fairness,RAIforUkraine},author+an={1=self;2=self;3=self;4=self;5=self;6=self}}
Setting the Right Expectations: Algorithmic Recourse Over Time
João Fonseca, Andrew Bell, Carlo Abrate, Francesco Bonchi, and Julia Stoyanovich
In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms,
Mechanisms, and Optimization, EAAMO 2023, Boston, MA, USA, 30 October
2023 - 1 November 2023 2023
@inproceedings{DBLP:conf/eaamo/FonsecaBABS23,author={Fonseca, Jo{\~{a}}o and Bell, Andrew and Abrate, Carlo and Bonchi, Francesco and Stoyanovich, Julia},title={Setting the Right Expectations: Algorithmic Recourse Over Time},booktitle={Proceedings of the 3rd {ACM} Conference on Equity and Access in Algorithms,
Mechanisms, and Optimization, {EAAMO} 2023, Boston, MA, USA, 30 October
2023 - 1 November 2023},pages={29:1--29:11},publisher={{ACM}},year={2023},url={https://doi.org/10.1145/3617694.3623251},doi={10.1145/3617694.3623251},timestamp={Thu, 09 Nov 2023 21:12:58 +0100},biburl={https://dblp.org/rec/conf/eaamo/FonsecaBABS23.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference, fairness}}
Query Refinement for Diversity Constraint Satisfaction
Jinyang Li, Yuval Moskovitch, Julia Stoyanovich, and H. V. Jagadish
@article{DBLP:journals/pvldb/LiMSJ23,author={Li, Jinyang and Moskovitch, Yuval and Stoyanovich, Julia and Jagadish, H. V.},title={Query Refinement for Diversity Constraint Satisfaction},journal={Proc. {VLDB} Endow.},volume={17},number={2},pages={106--118},year={2023},timestamp={Mon, 27 Nov 2023 13:13:34 +0100},biburl={https://dblp.org/rec/journals/pvldb/LiMSJ23.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,fairness,data,ranking},}
ERICA: Query Refinement for Diversity Constraint Satisfaction
Jinyang Li, Alon Silberstein, Yuval Moskovitch, Julia Stoyanovich, and H. V. Jagadish
@article{DBLP:journals/pvldb/LiSMSJ23,author={Li, Jinyang and Silberstein, Alon and Moskovitch, Yuval and Stoyanovich, Julia and Jagadish, H. V.},title={{ERICA:} Query Refinement for Diversity Constraint Satisfaction},journal={Proc. {VLDB} Endow.},volume={16},number={12},pages={4070--4073},year={2023},doi={10.14778/3611540.3611623},timestamp={Mon, 23 Oct 2023 16:16:16 +0200},biburl={https://dblp.org/rec/journals/pvldb/LiSMSJ23.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,fairness,data},}
Counterfactual Fairness Is Basically Demographic Parity
Lucas Rosenblatt, and R Teal Witter
Proceedings of the AAAI Conference on Artificial Intelligence 2023
@article{rosenblatt2022counterfactual,title={Counterfactual Fairness Is Basically Demographic Parity},author={Rosenblatt, Lucas and Witter, R Teal},journal={Proceedings of the AAAI Conference on Artificial Intelligence},year={2023},keywords={fairness, counterfactuals, causal},}
Counterfactuals for the Future
Lucius E. J. Bynum, Joshua R. Loftus, and Julia Stoyanovich
@article{DBLP:journals/corr/abs-2212-03974,author={Bynum, Lucius E. J. and Loftus, Joshua R. and Stoyanovich, Julia},title={Counterfactuals for the Future},booktitle={Thirty-Seventh {AAAI} Conference on Artificial Intelligence, {AAAI}
2023, Thirty-Fifth Conference on Innovative Applications of Artificial
Intelligence, {IAAI} 2023, Thirteenth Symposium on Educational Advances
in Artificial Intelligence, {EAAI} 2023, Washington, DC, USA},pages={14144--14152},publisher={{AAAI} Press},year={2023},doi={10.1609/AAAI.V37I12.26655},keywords={conference,fairness,fair,causal,counterfactuals}}
Fairness in Ranking: From Values to Technical Choices and Back
Julia Stoyanovich, Meike Zehlike, and Ke Yang
In Companion of the 2023 International Conference on Management of Data,
SIGMOD/PODS 2023, Seattle, WA, USA, June 18-23, 2023 2023
@inproceedings{DBLP:conf/sigmod/StoyanovichZY23,author={Stoyanovich, Julia and Zehlike, Meike and Yang, Ke},editor={Das, Sudipto and Pandis, Ippokratis and Candan, K. Sel{\c{c}}uk and Amer{-}Yahia, Sihem},title={Fairness in Ranking: From Values to Technical Choices and Back},booktitle={Companion of the 2023 International Conference on Management of Data,
{SIGMOD/PODS} 2023, Seattle, WA, USA, June 18-23, 2023},pages={7--12},publisher={{ACM}},year={2023},doi={10.1145/3555041.3589405},keywords={panel,fairness,ranking},addendum={peer-reviewed tutorial},}
On Fairness and Stability: Is Estimator Variance a Friend or a Foe?
Falaah Arif Khan, Denys Herasymuk, and Julia Stoyanovich
@article{DBLP:journals/corr/abs-2302-04525,author={Khan, Falaah Arif and Herasymuk, Denys and Stoyanovich, Julia},title={On Fairness and Stability: Is Estimator Variance a Friend or a Foe?},journal={CoRR},volume={abs/2302.04525},year={2023},doi={10.48550/arXiv.2302.04525},eprinttype={arXiv},eprint={2302.04525},keywords={working,fairness,stability,RAIforUkraine},}
2022
Spending Privacy Budget Fairly and Wisely
Lucas Rosenblatt, Joshua Allen, and Julia Stoyanovich
Theory and Practice of Differential Privacy (@ICML) 2022
@article{rosenblatt2022spending,title={Spending Privacy Budget Fairly and Wisely},author={Rosenblatt, Lucas and Allen, Joshua and Stoyanovich, Julia},journal={Theory and Practice of Differential Privacy (@ICML)},year={2022},keywords={differential privacy, fairness, databases},}
Critical Perspectives: A Benchmark Revealing Pitfalls in PerspectiveAPI
Lucas Rosenblatt, Lorena Piedras, and Julia Wilkins
In Proceedings of the Second Workshop on NLP for Positive Impact (NLP4PI) 2022
@inproceedings{rosenblatt2022critical,title={Critical Perspectives: A Benchmark Revealing Pitfalls in PerspectiveAPI},author={Rosenblatt, Lucas and Piedras, Lorena and Wilkins, Julia},booktitle={Proceedings of the Second Workshop on NLP for Positive Impact (NLP4PI)},pages={15--24},year={2022},keywords={fairness, nlp, benchmark},}
Towards Substantive Conceptions of Algorithmic Fairness: Normative
Guidance from Equal Opportunity Doctrines
Falaah Arif Khan, Eleni Manis, and Julia Stoyanovich
In Equity and Access in Algorithms, Mechanisms, and Optimization, EAAMO
2022, Arlington, VA, USA, October 6-9, 2022 2022
@inproceedings{DBLP:conf/eaamo/ArifKhanMS22,author={{Arif Khan}, Falaah and Manis, Eleni and Stoyanovich, Julia},title={Towards Substantive Conceptions of Algorithmic Fairness: Normative
Guidance from Equal Opportunity Doctrines},booktitle={Equity and Access in Algorithms, Mechanisms, and Optimization, {EAAMO}
2022, Arlington, VA, USA, October 6-9, 2022},pages={18:1--18:10},publisher={{ACM}},year={2022},url={https://doi.org/10.1145/3551624.3555303},doi={10.1145/3551624.3555303},keywords={conference, fairness}}
2021
The Many Facets of Data Equity
H. V. Jagadish, Julia Stoyanovich, and Bill Howe
In Proceedings of the Workshops of the EDBT/ICDT 2021 Joint Conference,
Nicosia, Cyprus, March 23, 2021 2021
@inproceedings{DBLP:conf/edbt/JagadishSH21,author={Jagadish, H. V. and Stoyanovich, Julia and Howe, Bill},editor={Costa, Constantinos and Pitoura, Evaggelia},title={The Many Facets of Data Equity},booktitle={Proceedings of the Workshops of the {EDBT/ICDT} 2021 Joint Conference,
Nicosia, Cyprus, March 23, 2021},series={{CEUR} Workshop Proceedings},volume={2841},publisher={CEUR-WS.org},year={2021},keywords={workshop,data,fairness},}
COVID-19 Brings Data Equity Challenges to the Fore
@article{DBLP:journals/dgov/JagadishSH21,author={Jagadish, H. V. and Stoyanovich, Julia and Howe, Bill},title={{COVID-19} Brings Data Equity Challenges to the Fore},journal={Digit. Gov. Res. Pract.},volume={2},number={2},pages={24:1--24:7},year={2021},doi={10.1145/3440889},keywords={journal,data,fairness},}
Comparing Apples and Oranges: Fairness and Diversity in Ranking (Invited
Talk)
Julia Stoyanovich
In 24th International Conference on Database Theory, ICDT 2021, March
23-26, 2021, Nicosia, Cyprus 2021
@inproceedings{DBLP:conf/icdt/Stoyanovich21,author={Stoyanovich, Julia},editor={Yi, Ke and Wei, Zhewei},title={Comparing Apples and Oranges: Fairness and Diversity in Ranking (Invited
Talk)},booktitle={24th International Conference on Database Theory, {ICDT} 2021, March
23-26, 2021, Nicosia, Cyprus},series={LIPIcs},volume={186},pages={2:1--2:1},publisher={Schloss Dagstuhl - Leibniz-Zentrum f{\"{u}}r Informatik},year={2021},doi={10.4230/LIPIcs.ICDT.2021.2},bibsource={dblp computer science bibliography, https://dblp.org},keywords={invited,fairness,ranking},author+an={1=self},}
@article{DBLP:journals/pvldb/StoyanovichHJ20,title={Responsible Data Management},author={Stoyanovich, Julia and Howe, Bill and Jagadish, HV},journal={Proceedings of the VLDB Endowment},volume={13},number={12},year={2020},keywords={fairness,data,journal},doi={10.14778/3415478.3415570},}
Fairness-Aware Instrumentation of Preprocessing Pipelines for Machine Learning
Ke Yang, Biao Huang, Julia Stoyanovich, and Sebastian Schelter
In Proceedings of the Workshop on Human-In-the-Loop Data Analytics, HILDA at SIGMOD 2020
@inproceedings{DBLP:hilda20,author={Yang, Ke and Huang, Biao and Stoyanovich, Julia and Schelter, Sebastian},title={Fairness-Aware Instrumentation of Preprocessing~Pipelines for Machine Learning},booktitle={Proceedings of the Workshop on Human-In-the-Loop Data Analytics, {HILDA} at {SIGMOD}},year={2020},doi={https://doi.org/10.1145/3398730.3399194},keywords={workshop,data,fairness},}
Taming Technical Bias in Machine Learning Pipelines
@article{debull21,author={Schelter, Sebastian and Stoyanovich, Julia},title={Taming Technical Bias in Machine Learning Pipelines},journal={{IEEE} Data Eng. Bull.},year={2020},volume={43},number={4},keywords={journal,data,fairness},author+an={2=self},}
2019
Transparency, Fairness, Data Protection, Neutrality: Data Management
Challenges in the Face of New Regulation
@article{DBLP:journals/jdiq/AbiteboulS19,author={Abiteboul, Serge and Stoyanovich, Julia},title={Transparency, Fairness, Data Protection, Neutrality: Data Management
Challenges in the Face of New Regulation},journal={ACM Journal of Data and Information Quality},volume={11},number={3},pages={15:1--15:9},year={2019},doi={10.1145/3310231},timestamp={Tue, 20 Aug 2019 08:40:18 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,data,fairness,policy,governance},}
MithraRanking: A System for Responsible Ranking Design
Yifan Guan, Abolfazl Asudeh, Pranav Mayuram, H. V. Jagadish, Julia Stoyanovich, Gerome Miklau, and Gautam Das
In Proceedings of the 2019 International Conference on the Management of
Data, SIGMOD 2019
@inproceedings{DBLP:conf/sigmod/GuanAMJSM019,author={Guan, Yifan and Asudeh, Abolfazl and Mayuram, Pranav and Jagadish, H. V. and Stoyanovich, Julia and Miklau, Gerome and Das, Gautam},title={{M}ithra{R}anking: {A} System for Responsible Ranking Design},booktitle={Proceedings of the 2019 International Conference on the Management of
Data, {SIGMOD}},pages={1913--1916},publisher={{ACM}},year={2019},doi={10.1145/3299869.3320244},timestamp={Sat, 22 Jun 2019 17:10:04 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,fairness,ranking},addendum={demonstration}}
2017
Measuring Fairness in Ranked Outputs
Ke Yang, and Julia Stoyanovich
In Proceedings of the 29th International Conference on Scientific and
Statistical Database Management, SSDBM 2017
@inproceedings{DBLP:conf/ssdbm/YangS17,author={Yang, Ke and Stoyanovich, Julia},title={Measuring Fairness in Ranked Outputs},booktitle={Proceedings of the 29th International Conference on Scientific and
Statistical Database Management, {SSDBM}},pages={22:1--22:6},publisher={{ACM}},year={2017},doi={10.1145/3085504.3085526},timestamp={Tue, 06 Nov 2018 16:57:31 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={fairness, ranking, data management},author+an={1=self;2=self}}
Policy
We engage in technology policy work in the US and internationally.
These practical engagements are based
on our research, highlighted below.
2025
Estimating the impact of the Russian invasion on the displacement of graduating high school students in Ukraine
Tetiana Zakharchenko, Andrew Bell, Nazarii Drushchak, Oleksandra Konopatska, Falaah Arif Khan, and Julia Stoyanovich
Nature Humanities and Social Sciences Communications 2025
@article{ukraine_edu25,title={Estimating the impact of the {Russian} invasion on the displacement of graduating high school students in {Ukraine}},author={Zakharchenko, Tetiana and Bell, Andrew and Drushchak, Nazarii and Konopatska, Oleksandra and Khan, Falaah Arif and Stoyanovich, Julia},year={2025},journal={Nature Humanities and Social Sciences Communications},keywords={journal,education,policy,RAIforUkraine},}
Promoting transparency in AI for biomedical and behavioral research
Tina Hernandez-Boussard, Aaron Y. Lee, Julia Stoyanovich, and Laura Biven
@article{natmed_25,author={Hernandez-Boussard, Tina and Lee, Aaron Y. and Stoyanovich, Julia and Biven, Laura},title={Promoting transparency in {AI} for biomedical and behavioral research},journal={Nature Medicine},year={2025},doi={10.1038/s41591-025-03680-0},keywords={journal,policy,transparency},url={https://doi.org/10.1038/s41591-025-03680-0},}
Making Transparency Advocates: An Educational Approach Towards Better Algorithmic Transparency in Practice
Andrew Bell, and Julia Stoyanovich
In Proceedings of the AAAI Conference on Artificial Intelligence 2025
@inproceedings{bellstoyanovich_eaai25,title={Making Transparency Advocates: An Educational Approach Towards Better Algorithmic Transparency in Practice},author={Bell, Andrew and Stoyanovich, Julia},year={2025},volume={39},pages={28964-28972},booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},keywords={conference,education,policy,transparency,playbook},doi={10.1609/aaai.v39i28.35165},url={https://doi.org/10.1609/aaai.v39i28.35165}}
2024
Making Transparency Influencers: A Case Study of an Educational
Approach to Improve Responsible AI Practices in News and Media
Andrew Bell, and Julia Stoyanovich
In Extended Abstracts of the CHI Conference on Human Factors in Computing
Systems, CHI EA 2024, Honolulu, HI, USA, May 11-16, 2024 2024
@inproceedings{DBLP:conf/chi/BellS24,author={Bell, Andrew and Stoyanovich, Julia},editor={Mueller, Florian 'Floyd' and Kyburz, Penny and Williamson, Julie R. and Sas, Corina},title={Making Transparency Influencers: {A} Case Study of an Educational
Approach to Improve Responsible {AI} Practices in News and Media},booktitle={Extended Abstracts of the {CHI} Conference on Human Factors in Computing
Systems, {CHI} {EA} 2024, Honolulu, HI, USA, May 11-16, 2024},pages={523:1--523:8},publisher={{ACM}},year={2024},doi={10.1145/3613905.3637113},timestamp={Fri, 17 May 2024 21:42:37 +0200},biburl={https://dblp.org/rec/conf/chi/BellS24.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={explainability, transparency, policy,governance},}
2023
Introducing contextual transparency for automated decision systems
Mona Sloane, Ian Solano-Kamaiko, Jun Yuan, Aritra Dasgupta, and Julia Stoyanovich
@article{NMI_2023,author={Sloane, Mona and Solano-Kamaiko, Ian and Yuan, Jun and Dasgupta, Aritra and Stoyanovich, Julia},title={Introducing contextual transparency for automated decision systems},journal={Nature Machine Intelligence},volume={5},year={2023},pages={187-–195},doi={https://doi.org/10.1038/s42256-023-00623-7},keywords={journal,explainability,policy,hiring},}
The Algorithmic Transparency Playbook: A Stakeholder-first Approach
to Creating Transparency for Your Organization’s Algorithms
Andrew Bell, Oded Nov, and Julia Stoyanovich
In Extended Abstracts of the 2023 CHI Conference on Human Factors in
Computing Systems, CHI EA 2023, Hamburg, Germany, April 23-28,
2023 2023
@inproceedings{DBLP:conf/chi/BellNS23,author={Bell, Andrew and Nov, Oded and Stoyanovich, Julia},title={The Algorithmic Transparency Playbook: {A} Stakeholder-first Approach
to Creating Transparency for Your Organization's Algorithms},booktitle={Extended Abstracts of the 2023 {CHI} Conference on Human Factors in
Computing Systems, {CHI} {EA} 2023, Hamburg, Germany, April 23-28,
2023},pages={554:1--554:4},publisher={{ACM}},year={2023},site={https://r-ai.co/transparency-playbook},doi={10.1145/3544549.3574169},keywords={panel,policy,explanability,education,playbook,governance},addendum={peer-reviewed course},author+an={1=self;3=self}}
Think About the Stakeholders First! Towards an Algorithmic Transparency
Playbook for Regulatory Compliance
@article{bell_nov_stoyanovich_2023,author={Bell, Andrew and Nov, Oded and Stoyanovich, Julia},title={Think About the Stakeholders First! {T}owards an Algorithmic Transparency
Playbook for Regulatory Compliance},volume={5},journal={Data \& Policy},publisher={Cambridge University Press},year={2023},keywords={journal,policy,explainability,education,playbook,governance},}
2022
It’s Just Not That Simple: An Empirical Study of the Accuracy-Explainability
Trade-off in Machine Learning for Public Policy
Andrew Bell, Ian Solano-Kamaiko, Oded Nov, and Julia Stoyanovich
In Proceedings of the 5th Annual ACM Conference on Fairness, Accountability, and
Transparency, FAccT 2022
@inproceedings{DBLP:conf/fat/BellSNS22,author={Bell, Andrew and Solano{-}Kamaiko, Ian and Nov, Oded and Stoyanovich, Julia},title={It's Just Not That Simple: An Empirical Study of the Accuracy-Explainability
Trade-off in Machine Learning for Public Policy},booktitle={Proceedings of the 5th Annual {ACM} Conference on Fairness, Accountability, and
Transparency, {FAccT}},pages={248--266},publisher={{ACM}},year={2022},doi={10.1145/3531146.3533090},timestamp={Wed, 22 Jun 2022 10:20:30 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={explainability,transparency,policy,governance,conference},}
Developing data capability with non-profit organisations using participatory methods
Anthony McCosker, Xiaofang Yao, Kath Albury, Alexia Maddox, Jane Farmer, and Julia Stoyanovich
@article{doi:10.1177/20539517221099882,author={McCosker, Anthony and Yao, Xiaofang and Albury, Kath and Maddox, Alexia and Farmer, Jane and Stoyanovich, Julia},title={Developing data capability with non-profit organisations using participatory methods},journal={Big Data \& Society},volume={9},number={1},pages={20539517221099882},year={2022},doi={10.1177/20539517221099882},eprint={https://doi.org/10.1177/20539517221099882},keywords={journal,data,policy,governance},}
2019
Transparency, Fairness, Data Protection, Neutrality: Data Management
Challenges in the Face of New Regulation
@article{DBLP:journals/jdiq/AbiteboulS19,author={Abiteboul, Serge and Stoyanovich, Julia},title={Transparency, Fairness, Data Protection, Neutrality: Data Management
Challenges in the Face of New Regulation},journal={ACM Journal of Data and Information Quality},volume={11},number={3},pages={15:1--15:9},year={2019},doi={10.1145/3310231},timestamp={Tue, 20 Aug 2019 08:40:18 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,data,fairness,policy,governance},}
2018
Follow the Data! Algorithmic Transparency Starts with Data Transparency
@article{Follow,title={Follow the Data! {A}lgorithmic Transparency Starts with Data Transparency},author={Stoyanovich, Julia and Howe, Bill},journal={The Ethical Machine},year={2018},month=nov,day={27},publisher={Shorenstein Center on Media, Politics, and Public Policy, Harvard Kennedy School},keywords={public,data,explainability,policy},author+an={1=self}}
2017
Safety and Privacy
Our work in privacy includes peer-reviewed papers, as well as tools and benchmarks.
Take a look at the Data Synthesizer tool
that we've been using to teach differentially private data synthesis.
And check out our latest DP synethetic data benchmarking package,
SynRD, posing the question:
"Can a DP synthesizer produce private (tabular) data that preserves scientific findings?"
In other words, do DP synthesizers satisfy epistemic parity?
We also started working on safety assessment and interventions, take a look at our SafeNudge project.
2026
SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems
Rima Hazra, Bikram Ghuku, Ilona Marchenko, Yaroslava Tokarieva, Sayan Layek, Somnath Banerjee, Julia Stoyanovich, and Mykola Pechenizkiy
@article{safetutors,title={SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems},author={Hazra, Rima and Ghuku, Bikram and Marchenko, Ilona and Tokarieva, Yaroslava and Layek, Sayan and Banerjee, Somnath and Stoyanovich, Julia and Pechenizkiy, Mykola},year={2026},eprint={2603.17373},archiveprefix={arXiv},primaryclass={cs.CL},keywords={RAIforUkraine,education,safety},}
2025
Fragments to Facts: Partial-Information Fragment Inference from LLMs
Lucas Rosenblatt, HAN Bin, Robert Wolfe, and Bill Howe
@inproceedings{rosenblattfragments,title={Fragments to Facts: Partial-Information Fragment Inference from LLMs},author={Rosenblatt, Lucas and Bin, HAN and Wolfe, Robert and Howe, Bill},booktitle={In Proceedings of ICML},year={2025},keywords={privacy},}
Differential Privacy Under Class Imbalance: Methods and Empirical Insights
Lucas Rosenblatt, Yuliia Lut, Ethan Turok, Marco Avella Medina, and Rachel Cummings
@inproceedings{rosenblattdifferential,title={Differential Privacy Under Class Imbalance: Methods and Empirical Insights},author={Rosenblatt, Lucas and Lut, Yuliia and Turok, Ethan and Medina, Marco Avella and Cummings, Rachel},booktitle={In Proceedings of ICML},year={2025},keywords={privacy}}
SafeNudge: Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs
Joao Fonseca, Andrew Bell, and Julia Stoyanovich
In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP) Nov 2025
@inproceedings{fonseca-etal-2025-safenudge,title={{SafeNudge}: Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs},author={Fonseca, Joao and Bell, Andrew and Stoyanovich, Julia},booktitle={Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP)},month=nov,year={2025},publisher={ACM},doi={10.18653/v1/2025.emnlp-main.1010},pages={19955--19969},isbn={979-8-89176-332-6},keywords={conference,privacy,safety}}
Do You Really Need Public Data? Surrogate Public Data for Differential Privacy on Tabular Data
Shlomi Hod, Lucas Rosenblatt, and Julia Stoyanovich
The Thirty-Ninth Annual Conference on Neural Information Processing Systems (NeurIPS), Datasets & Benchmarks Track Nov 2025
@article{hod2025reallyneedpublicdata,title={Do You Really Need Public Data? Surrogate Public Data for Differential Privacy on Tabular Data},author={Hod, Shlomi and Rosenblatt, Lucas and Stoyanovich, Julia},year={2025},journal={The Thirty-Ninth Annual Conference on Neural Information Processing Systems (NeurIPS), Datasets \& Benchmarks Track},keywords={privacy, data, genAI},}
How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacy
Natalia Ponomareva, Zheng Xu, H Brendan McMahan, Peter Kairouz, Lucas Rosenblatt, Vincent Cohen-Addad, Cristóbal Guzmán, Ryan McKenna, Galen Andrew, Alex Bie, and others
@article{ponomareva2025dp,title={How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacy},author={Ponomareva, Natalia and Xu, Zheng and McMahan, H Brendan and Kairouz, Peter and Rosenblatt, Lucas and Cohen-Addad, Vincent and Guzm{\'a}n, Crist{\'o}bal and McKenna, Ryan and Andrew, Galen and Bie, Alex and others},journal={CoRR},year={2025},keywords={working,privacy}}
SimKey: A Semantically Aware Key Module for Watermarking Language Models
Shingo Kodama, Haya Diwan, Lucas Rosenblatt, R Teal Witter, and Niv Cohen
@article{kodama2025simkey,title={SimKey: A Semantically Aware Key Module for Watermarking Language Models},author={Kodama, Shingo and Diwan, Haya and Rosenblatt, Lucas and Witter, R Teal and Cohen, Niv},journal={CoRR},year={2025},keywords={working,safety}}
2024
Epistemic Parity: Reproducibility as an Evaluation Metric for Differential
Privacy
Lucas Rosenblatt, Bernease Herman, Anastasia Holovenko, Wonkwon Lee, Joshua R. Loftus, Elizabeth McKinnie, Taras Rumezhak, Andrii Stadnik, Bill Howe, and Julia Stoyanovich
@article{DBLP:journals/sigmod/RosenblattHHLLMRSHS24,author={Rosenblatt, Lucas and Herman, Bernease and Holovenko, Anastasia and Lee, Wonkwon and Loftus, Joshua R. and McKinnie, Elizabeth and Rumezhak, Taras and Stadnik, Andrii and Howe, Bill and Stoyanovich, Julia},title={Epistemic Parity: Reproducibility as an Evaluation Metric for Differential
Privacy},journal={{SIGMOD} Rec.},volume={53},number={1},pages={65--74},year={2024},url={https://doi.org/10.1145/3665252.3665267},doi={10.1145/3665252.3665267},keywords={journal,privacy,RAIforUkraine},author+an={1=self;3=self;4=self;6=self;7=self;8=self;10=self}}
A Simple and Practical Method for Reducing the Disparate Impact of
Differential Privacy
Lucas Rosenblatt, Julia Stoyanovich, and Christopher Musco
In Thirty-Eighth AAAI Conference on Artificial Intelligence Nov 2024
@inproceedings{DBLP:conf/aaai/RosenblattSM24,author={Rosenblatt, Lucas and Stoyanovich, Julia and Musco, Christopher},editor={Wooldridge, Michael J. and Dy, Jennifer G. and Natarajan, Sriraam},title={A Simple and Practical Method for Reducing the Disparate Impact of
Differential Privacy},booktitle={Thirty-Eighth {AAAI} Conference on Artificial Intelligence},pages={21554--21562},publisher={{AAAI} Press},year={2024},url={https://doi.org/10.1609/aaai.v38i19.30153},doi={10.1609/AAAI.V38I19.30153},keywords={fairness,privacy,conference},author+an={1=self;2=self},}
2023
Personal Data for Personal Use: Vision or Reality?
Xin Luna Dong, Bo Li, Julia Stoyanovich, Anthony Kum Hoe Tung, Gerhard Weikum, Alon Y. Halevy, and Wang-Chiew Tan
In Companion of the 2023 International Conference on Management of Data,
SIGMOD/PODS 2023, Seattle, WA, USA, June 18-23, 2023 Nov 2023
@inproceedings{DBLP:conf/sigmod/DongLSTWHT23,author={Dong, Xin Luna and Li, Bo and Stoyanovich, Julia and Tung, Anthony Kum Hoe and Weikum, Gerhard and Halevy, Alon Y. and Tan, Wang{-}Chiew},title={Personal Data for Personal Use: Vision or Reality?},booktitle={Companion of the 2023 International Conference on Management of Data,
{SIGMOD/PODS} 2023, Seattle, WA, USA, June 18-23, 2023},pages={263--264},publisher={{ACM}},year={2023},doi={10.1145/3555041.3589378},keywords={panel,data,privacy},addendum={panelist, by invitation},}
Epistemic Parity: Reproducibility as an Evaluation Metric for Differential
Privacy
Lucas Rosenblatt, Bernease Herman, Anastasia Holovenko, Wonkwon Lee, Joshua R. Loftus, Elizabeth Mckinnie, Taras Rumezhak, Andrii Stadnik, Bill Howe, and Julia Stoyanovich
@article{DBLP:journals/pvldb/RosenblattHHLLM23,author={Rosenblatt, Lucas and Herman, Bernease and Holovenko, Anastasia and Lee, Wonkwon and Loftus, Joshua R. and Mckinnie, Elizabeth and Rumezhak, Taras and Stadnik, Andrii and Howe, Bill and Stoyanovich, Julia},title={Epistemic Parity: Reproducibility as an Evaluation Metric for Differential
Privacy},journal={Proc. {VLDB} Endow.},volume={16},number={11},pages={3178--3191},year={2023},doi={10.14778/3611479.3611517},timestamp={Mon, 23 Oct 2023 16:16:16 +0200},biburl={https://dblp.org/rec/journals/pvldb/RosenblattHHLLM23.bib},keywords={journal,privacy,RAIforUkraine},author+an={1=self;3=self;4=self;6=self;7=self;8=self;10=self}}
2022
Spending Privacy Budget Fairly and Wisely
Lucas Rosenblatt, Joshua Allen, and Julia Stoyanovich
Theory and Practice of Differential Privacy (@ICML) Nov 2022
@article{rosenblatt2022spending,title={Spending Privacy Budget Fairly and Wisely},author={Rosenblatt, Lucas and Allen, Joshua and Stoyanovich, Julia},journal={Theory and Practice of Differential Privacy (@ICML)},year={2022},keywords={differential privacy, fairness, databases},}
2020
Differentially private synthetic data: Applied evaluations and enhancements
Lucas Rosenblatt, Xiaoyan Liu, Samira Pouyanfar, Eduardo Leon, Anuj Desai, and Joshua Allen
@article{rosenblatt2020differentially,title={Differentially private synthetic data: Applied evaluations and enhancements},author={Rosenblatt, Lucas and Liu, Xiaoyan and Pouyanfar, Samira and de Leon, Eduardo and Desai, Anuj and Allen, Joshua},journal={arXiv preprint arXiv:2011.05537},year={2020},keywords={differential privacy, benchmark},}
@inproceedings{DBLP:conf/vldb/RodriguezSPSH19,author={Rodriguez, Luke and Salimi, Babak and Ping, Haoyue and Stoyanovich, Julia and Howe, Bill},editor={Oliveira, Jonice and de Farias, Claudio M. and Pacitti, Esther and Fortino, Giancarlo},title={MobilityMirror: Bias-Adjusted Transportation Datasets},booktitle={Big Social Data and Urban Computing - First Workshop, BiDU@VLDB 2018,
Rio de Janeiro, Brazil, August 31, 2018, Revised Selected Papers},series={Communications in Computer and Information Science},volume={926},pages={18--39},publisher={Springer},year={2018},url={https://doi.org/10.1007/978-3-030-11238-7_2},doi={10.1007/978-3-030-11238-7_2},keywords={privacy},timestamp={Tue, 29 Jan 2019 14:00:34 +0100},biburl={https://dblp.org/rec/conf/vldb/RodriguezSPSH18.bib},bibsource={dblp computer science bibliography, https://dblp.org}}
@inproceedings{DBLP:conf/ssdbm/PingSH17,author={Ping, Haoyue and Stoyanovich, Julia and Howe, Bill},title={{DataSynthesizer}: Privacy-Preserving Synthetic Datasets},booktitle={Proceedings of the 29th International Conference on Scientific and
Statistical Database Management, {SSDBM}},pages={42:1--42:5},publisher={{ACM}},year={2017},doi={10.1145/3085504.3091117},timestamp={Tue, 06 Nov 2018 16:57:31 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,privacy},author+an={1=self;3=self}}
Ranking
One kind of algorithm
that is at once especially obscure, powerful, and common is the
ranking algorithm. Algorithms rank individuals to determine credit
worthiness, desirability for college admissions and employment, and
compatibility as dating partners. They encode ideas of what counts
as the best schools, neighborhoods, and technologies. Despite their
importance, we actually can know very little about why one person
was ranked higher than another in a dating app, or why one school
has a better rank than that one. This is true even if we have access
to the ranking algorithm, for example, if we have complete knowledge
about the factors used by the ranker and their relative weights, as
is the case for US News ranking of colleges. We have been working
on several aspects of responsible ranking design and use, including
fairness, transparency and interpretability, and stability.
2025
ShaRP: Explaining Rankings and Preferences with Shapley Values
Venetia Pliatsika, João Fonseca, Kateryna Akhynko, Ivan Shevchenko, and Julia Stoyanovich
@article{sharp,author={Pliatsika, Venetia and Fonseca, Jo{\~{a}}o and Akhynko, Kateryna and Shevchenko, Ivan and Stoyanovich, Julia},title={{ShaRP}: Explaining Rankings and Preferences with Shapley Values},journal={Proc. {VLDB} Endow.},volume={18},number={11},year={2025},doi={10.14778/3749646.3749682},keywords={journal,data,ranking,explainability,transparency,RAIforUkraine}}
2024
Rodeo: Making Refinements for Diverse Top-k Queries
Felix S. Campbell, Julia Stoyanovich, and Yuval Moskovitch
@article{DBLP:journals/pvldb/CampbellSM24,author={Campbell, Felix S. and Stoyanovich, Julia and Moskovitch, Yuval},title={Rodeo: Making Refinements for Diverse Top-k Queries},journal={Proc. {VLDB} Endow.},volume={17},number={12},pages={4341--4344},year={2024},keywords={conference,fairness,data,demo,ranking}}
2023
Query Refinement for Diversity Constraint Satisfaction
Jinyang Li, Yuval Moskovitch, Julia Stoyanovich, and H. V. Jagadish
@article{DBLP:journals/pvldb/LiMSJ23,author={Li, Jinyang and Moskovitch, Yuval and Stoyanovich, Julia and Jagadish, H. V.},title={Query Refinement for Diversity Constraint Satisfaction},journal={Proc. {VLDB} Endow.},volume={17},number={2},pages={106--118},year={2023},timestamp={Mon, 27 Nov 2023 13:13:34 +0100},biburl={https://dblp.org/rec/journals/pvldb/LiMSJ23.bib},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,fairness,data,ranking},}
Fairness in Ranking: From Values to Technical Choices and Back
Julia Stoyanovich, Meike Zehlike, and Ke Yang
In Companion of the 2023 International Conference on Management of Data,
SIGMOD/PODS 2023, Seattle, WA, USA, June 18-23, 2023 2023
@inproceedings{DBLP:conf/sigmod/StoyanovichZY23,author={Stoyanovich, Julia and Zehlike, Meike and Yang, Ke},editor={Das, Sudipto and Pandis, Ippokratis and Candan, K. Sel{\c{c}}uk and Amer{-}Yahia, Sihem},title={Fairness in Ranking: From Values to Technical Choices and Back},booktitle={Companion of the 2023 International Conference on Management of Data,
{SIGMOD/PODS} 2023, Seattle, WA, USA, June 18-23, 2023},pages={7--12},publisher={{ACM}},year={2023},doi={10.1145/3555041.3589405},keywords={panel,fairness,ranking},addendum={peer-reviewed tutorial},}
@article{10.1145/3533379,author={Zehlike, Meike and Yang, Ke and Stoyanovich, Julia},title={Fairness in Ranking, Part I: Score-Based Ranking},journal={{ACM} Computing Surveys},volume={55},number={6},pages={118:1--118:36},year={2023},doi={10.1145/3533379},keywords={journal,fair,ranking},author+an={2=self;3=self},}
Fairness in Ranking, Part II: Learning-to-Rank and Recommender Systems
@article{10.1145/3533380,author={Zehlike, Meike and Yang, Ke and Stoyanovich, Julia},title={Fairness in Ranking, Part II: Learning-to-Rank and Recommender Systems},journal={{ACM} Computing Surveys},volume={55},number={6},pages={117:1--117:41},year={2023},doi={10.1145/3533380},keywords={journal,fair,ranking},}
2022
Rankers, Rankees, & Rankings: Peeking into the Pandora’s Box
from a Socio-Technical Perspective
@article{DBLP:journals/corr/abs-2211-02932,author={Yuan, Jun and Stoyanovich, Julia and Dasgupta, Aritra},title={Rankers, Rankees, {\&} Rankings: Peeking into the Pandora's Box
from a Socio-Technical Perspective},journal={CoRR},volume={abs/2211.02932},year={2022},url={https://doi.org/10.48550/arXiv.2211.02932},doi={10.48550/arXiv.2211.02932},keywords={working,ranking,explainability},eprinttype={arXiv},eprint={2211.02932},timestamp={Wed, 09 Nov 2022 17:33:26 +0100},biburl={https://dblp.org/rec/journals/corr/abs-2211-02932.bib},bibsource={dblp computer science bibliography, https://dblp.org},}
2021
Causal Intersectionality and Fair Ranking
Ke Yang, Joshua Loftus, and Julia Stoyanovich
In Symposium on the Foundations of Responsible Computing FORC 2021
@inproceedings{forc21,author={Yang, Ke and Loftus, Joshua and Stoyanovich, Julia},title={Causal Intersectionality and Fair Ranking},booktitle={Symposium on the Foundations of Responsible Computing {FORC}},year={2021},doi={10.4230/LIPIcs.FORC.2021.7},keywords={conference,fair,ranking},author+an={1=self;3=self}}
Comparing Apples and Oranges: Fairness and Diversity in Ranking (Invited
Talk)
Julia Stoyanovich
In 24th International Conference on Database Theory, ICDT 2021, March
23-26, 2021, Nicosia, Cyprus 2021
@inproceedings{DBLP:conf/icdt/Stoyanovich21,author={Stoyanovich, Julia},editor={Yi, Ke and Wei, Zhewei},title={Comparing Apples and Oranges: Fairness and Diversity in Ranking (Invited
Talk)},booktitle={24th International Conference on Database Theory, {ICDT} 2021, March
23-26, 2021, Nicosia, Cyprus},series={LIPIcs},volume={186},pages={2:1--2:1},publisher={Schloss Dagstuhl - Leibniz-Zentrum f{\"{u}}r Informatik},year={2021},doi={10.4230/LIPIcs.ICDT.2021.2},bibsource={dblp computer science bibliography, https://dblp.org},keywords={invited,fairness,ranking},author+an={1=self},}
2019
Towards Responsible Data-driven Decision Making in Score-Based Systems
Abolfazl Asudeh, H. V. Jagadish, and Julia Stoyanovich
@article{DBLP:journals/debu/AsudehJS19,author={Asudeh, Abolfazl and Jagadish, H. V. and Stoyanovich, Julia},title={Towards Responsible Data-driven Decision Making in Score-Based Systems},journal={{IEEE} Data Eng. Bull.},volume={42},number={3},pages={76--87},year={2019},timestamp={Tue, 10 Mar 2020 16:23:50 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,fair,ranking},author+an={3=self}}
Balanced Ranking with Diversity Constraints
Ke Yang, Vasilis Gkatzelis, and Julia Stoyanovich
In Proceedings of the 28th International Joint Conference on
Artificial Intelligence, IJCAI 2019
@inproceedings{DBLP:conf/ijcai/YangGS19,author={Yang, Ke and Gkatzelis, Vasilis and Stoyanovich, Julia},title={Balanced Ranking with Diversity Constraints},booktitle={Proceedings of the 28th International Joint Conference on
Artificial Intelligence, {IJCAI}},pages={6035--6042},publisher={ijcai.org},year={2019},doi={10.24963/ijcai.2019/836},timestamp={Tue, 20 Aug 2019 16:18:18 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,fair,ranking},author+an={1=self;3=self}}
Designing Fair Ranking Schemes
Abolfazl Asudeh, H. V. Jagadish, Julia Stoyanovich, and Gautam Das
In Proceedings of the 2019 International Conference on the Management of
Data, SIGMOD 2019
@inproceedings{DBLP:conf/sigmod/AsudehJS019,author={Asudeh, Abolfazl and Jagadish, H. V. and Stoyanovich, Julia and Das, Gautam},title={Designing Fair Ranking Schemes},booktitle={Proceedings of the 2019 International Conference on the Management of
Data, {SIGMOD}},pages={1259--1276},publisher={{ACM}},year={2019},doi={10.1145/3299869.3300079},timestamp={Sat, 22 Jun 2019 17:10:04 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,fair,ranking},author+an={3=self}}
MithraRanking: A System for Responsible Ranking Design
Yifan Guan, Abolfazl Asudeh, Pranav Mayuram, H. V. Jagadish, Julia Stoyanovich, Gerome Miklau, and Gautam Das
In Proceedings of the 2019 International Conference on the Management of
Data, SIGMOD 2019
@inproceedings{DBLP:conf/sigmod/GuanAMJSM019,author={Guan, Yifan and Asudeh, Abolfazl and Mayuram, Pranav and Jagadish, H. V. and Stoyanovich, Julia and Miklau, Gerome and Das, Gautam},title={{M}ithra{R}anking: {A} System for Responsible Ranking Design},booktitle={Proceedings of the 2019 International Conference on the Management of
Data, {SIGMOD}},pages={1913--1916},publisher={{ACM}},year={2019},doi={10.1145/3299869.3320244},timestamp={Sat, 22 Jun 2019 17:10:04 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,fairness,ranking},addendum={demonstration}}
2018
On Obtaining Stable Rankings
Abolfazl Asudeh, H. V. Jagadish, Gerome Miklau, and Julia Stoyanovich
@article{DBLP:journals/pvldb/AsudehJMS18,author={Asudeh, Abolfazl and Jagadish, H. V. and Miklau, Gerome and Stoyanovich, Julia},title={On Obtaining Stable Rankings},journal={PVLDB},volume={12},number={3},pages={237--250},year={2018},doi={10.14778/3291264.3291269},timestamp={Sat, 25 Apr 2020 13:58:47 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={journal,stable,ranking},author+an={4=self}}
Online Set Selection with Fairness and Diversity Constraints
Julia Stoyanovich, Ke Yang, and H. V. Jagadish
In Proceedings of the 21th International Conference on Extending Database
Technology, EDBT 2018
@inproceedings{DBLP:conf/edbt/StoyanovichYJ18,author={Stoyanovich, Julia and Yang, Ke and Jagadish, H. V.},title={Online Set Selection with Fairness and Diversity Constraints},booktitle={Proceedings of the 21th International Conference on Extending Database
Technology, {EDBT}},pages={241--252},publisher={OpenProceedings.org},year={2018},doi={10.5441/002/edbt.2018.22},timestamp={Fri, 16 Mar 2018 23:12:22 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,fair,ranking},author+an={1=self;2=self}}
A Nutritional Label for Rankings
Ke Yang, Julia Stoyanovich, Abolfazl Asudeh, Bill Howe, H. V. Jagadish, and Gerome Miklau
In Proceedings of the 2018 International Conference on the Management of
Data, SIGMOD 2018
@inproceedings{DBLP:conf/sigmod/YangSAHJM18,author={Yang, Ke and Stoyanovich, Julia and Asudeh, Abolfazl and Howe, Bill and Jagadish, H. V. and Miklau, Gerome},title={A Nutritional Label for Rankings},booktitle={Proceedings of the 2018 International Conference on the Management of
Data, {SIGMOD}},publisher={{ACM}},year={2018},doi={10.1145/3183713.3193568},timestamp={Wed, 21 Nov 2018 12:44:08 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={conference,ranking,explanability},}
Refining the Concept of a Nutritional Label for Data and Models
Julia Stoyanovich, and Bill Howe
Freedom to Tinker, Center for Information Technology Policy, Princeton University May 2018
@article{citp_2,author={Stoyanovich, Julia and Howe, Bill},title={Refining the Concept of a Nutritional Label for Data and Models},journal={Freedom to Tinker, Center for Information Technology Policy, Princeton University},year={2018},month=may,day={3},keywords={public,ranking,explainability},}
2017
Measuring Fairness in Ranked Outputs
Ke Yang, and Julia Stoyanovich
In Proceedings of the 29th International Conference on Scientific and
Statistical Database Management, SSDBM May 2017
@inproceedings{DBLP:conf/ssdbm/YangS17,author={Yang, Ke and Stoyanovich, Julia},title={Measuring Fairness in Ranked Outputs},booktitle={Proceedings of the 29th International Conference on Scientific and
Statistical Database Management, {SSDBM}},pages={22:1--22:6},publisher={{ACM}},year={2017},doi={10.1145/3085504.3085526},timestamp={Tue, 06 Nov 2018 16:57:31 +0100},bibsource={dblp computer science bibliography, https://dblp.org},keywords={fairness, ranking, data management},author+an={1=self;2=self}}
2016
Revealing Algorithmic Rankers
Julia Stoyanovich, and Ellen P. Goodman
Freedom to Tinker, Center for Information Technology Policy, Princeton University Aug 2016
@article{citp_1,author={Stoyanovich, Julia and Goodman, Ellen P.},title={Revealing Algorithmic Rankers},journal={Freedom to Tinker, Center for Information Technology Policy, Princeton University},year={2016},month=aug,day={5},keywords={public,ranking,explainability},author+an={1=self}}
2015
Analyzing Crowd Rankings
Julia Stoyanovich, Marie Jacob, and Xuemei Gong
In Proceedings of the 18th International Workshop on Web and Databases,
Melbourne, VIC, Australia, May 31, 2015 Aug 2015
@inproceedings{DBLP:conf/webdb/StoyanovichJG16,author={Stoyanovich, Julia and Jacob, Marie and Gong, Xuemei},editor={Stoyanovich, Julia and Suchanek, Fabian M.},title={Analyzing Crowd Rankings},booktitle={Proceedings of the 18th International Workshop on Web and Databases,
Melbourne, VIC, Australia, May 31, 2015},pages={41--47},keywords={ranking},publisher={{ACM}},year={2015},url={https://doi.org/10.1145/2767109.2767110},doi={10.1145/2767109.2767110},timestamp={Tue, 06 Nov 2018 16:58:14 +0100},biburl={https://dblp.org/rec/conf/webdb/StoyanovichJG15.bib},bibsource={dblp computer science bibliography, https://dblp.org}}
Responsible AI Education
We cannot understand the impact – and especially the risks – of AI
systems without active and thoughtful participation of everyone in
society, either directly or through their trusted representatives. To
think otherwise is to go against our democratic values. To enable
broad participation, we have been developing responsible AI curricula
and methodologies for different stakeholders: university students,
working practitioners, and the public at large. In this section, you
will find our publication on responsible AI education. Take a look at
the education area of the site to access our
courses and other open-source materials we have developed.
2026
Seasoning Data Modeling Education with GARLIC: A Participatory
Co-Design Framework
Viktoriia Makovska, Ihor Michurin, Mariia Tokhtamysh, George Fletcher, and Julia Stoyanovich
Proceedings of the DataEd Workshop, co-located with EDBT/ICDT 2026
@article{garlic,author={Makovska, Viktoriia and Michurin, Ihor and Tokhtamysh, Mariia and Fletcher, George and Stoyanovich, Julia},title={Seasoning Data Modeling Education with {GARLIC:} {A} Participatory
Co-Design Framework},journal={Proceedings of the DataEd Workshop, co-located with EDBT/ICDT},volume={abs/2602.18274},year={2026},keywords={RAIforUkraine,data,explainability,transparency,education},}
SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems
Rima Hazra, Bikram Ghuku, Ilona Marchenko, Yaroslava Tokarieva, Sayan Layek, Somnath Banerjee, Julia Stoyanovich, and Mykola Pechenizkiy
@article{safetutors,title={SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems},author={Hazra, Rima and Ghuku, Bikram and Marchenko, Ilona and Tokarieva, Yaroslava and Layek, Sayan and Banerjee, Somnath and Stoyanovich, Julia and Pechenizkiy, Mykola},year={2026},eprint={2603.17373},archiveprefix={arXiv},primaryclass={cs.CL},keywords={RAIforUkraine,education,safety},}
2025
Estimating the impact of the Russian invasion on the displacement of graduating high school students in Ukraine
Tetiana Zakharchenko, Andrew Bell, Nazarii Drushchak, Oleksandra Konopatska, Falaah Arif Khan, and Julia Stoyanovich
Nature Humanities and Social Sciences Communications 2025
@article{ukraine_edu25,title={Estimating the impact of the {Russian} invasion on the displacement of graduating high school students in {Ukraine}},author={Zakharchenko, Tetiana and Bell, Andrew and Drushchak, Nazarii and Konopatska, Oleksandra and Khan, Falaah Arif and Stoyanovich, Julia},year={2025},journal={Nature Humanities and Social Sciences Communications},keywords={journal,education,policy,RAIforUkraine},}
ONION: A Multi-Layered Framework for Participatory ER Design
Viktoriia Makovska, George Fletcher, and Julia Stoyanovich
In Proceedings of the Workshop on Human-In-the-Loop Data Analytics (HILDA) at ACM SIGMOD 2025
@inproceedings{onion20205,title={{ONION}: A Multi-Layered Framework for Participatory ER Design},author={Makovska, Viktoriia and Fletcher, George and Stoyanovich, Julia},year={2025},booktitle={Proceedings of the Workshop on Human-In-the-Loop Data Analytics (HILDA) at ACM SIGMOD},url={https://dl.acm.org/doi/10.1145/3736733.3736736},keywords={data,education,RAIforUkraine}}
Using Case Studies to Teach Responsible AI to Industry Practitioners
Julia Stoyanovich, Rodrigo Kreis Paula, Armanda Lewis, and Chloe Zheng
In Proceedings of the AAAI Conference on Artificial Intelligence 2025
@inproceedings{stoyanovich_meta_eaai25,title={Using Case Studies to Teach Responsible AI to Industry Practitioners},author={Stoyanovich, Julia and de Paula, Rodrigo Kreis and Lewis, Armanda and Zheng, Chloe},year={2025},booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},keywords={education},pages={29062-29069},url={https://doi.org/10.1609/aaai.v39i28.35177},doi={https://doi.org/10.1609/aaai.v39i28.35177}}
Making Transparency Advocates: An Educational Approach Towards Better Algorithmic Transparency in Practice
Andrew Bell, and Julia Stoyanovich
In Proceedings of the AAAI Conference on Artificial Intelligence 2025
@inproceedings{bellstoyanovich_eaai25,title={Making Transparency Advocates: An Educational Approach Towards Better Algorithmic Transparency in Practice},author={Bell, Andrew and Stoyanovich, Julia},year={2025},volume={39},pages={28964-28972},booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},keywords={conference,education,policy,transparency,playbook},doi={10.1609/aaai.v39i28.35165},url={https://doi.org/10.1609/aaai.v39i28.35165}}
We Are AI: Taking Control of Technology
Julia Stoyanovich, Armanda Lewis, Eric Corbett, Lucius Bynum, Lucas Rosenblatt, and Falaah Arif Khan
In Proceedings of the AAAI Conference on Artificial Intelligence 2025
@inproceedings{stoyanovich_weareai_eaai25,title={We Are {AI}: Taking Control of Technology},author={Stoyanovich, Julia and Lewis, Armanda and Corbett, Eric and Bynum, Lucius and Rosenblatt, Lucas and {Arif Khan}, Falaah},year={2025},booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},keywords={conference,education,weareai},pages={29070-29077},url={https://doi.org/10.1609/aaai.v39i28.35178},doi={https://doi.org/10.1609/aaai.v39i28.35178}}
@article{weareaicomic_vol1_uk,author={Stoyanovich, Julia and Khan, Falaah Arif},title={What is AI? (Ukrainian Edition)},journal={We are AI Comic Series},volume={1},year={2024},keywords={public,education,comics,ukrainian},}
@article{weareaicomic_vol2_uk,author={Stoyanovich, Julia and Khan, Falaah Arif},title={Learning from data (Ukrainian Edition)},journal={We are AI Comic Series},volume={2},year={2024},keywords={public,education,comics,ukrainian},}
Who lives? Who dies? Who decides? (Ukrainian Edition)
@article{weareaicomic_vol3_uk,author={Stoyanovich, Julia and Khan, Falaah Arif},title={Who lives? Who dies? Who decides? (Ukrainian Edition)},journal={We are AI Comic Series},volume={3},year={2024},keywords={public,education,comics,ukrainian},}
@article{weareaicomic_vol4_uk,author={Stoyanovich, Julia and Khan, Falaah Arif},title={All about that bias (Ukrainian Edition)},journal={We are AI Comic Series},volume={4},year={2024},keywords={public,education,comics,ukrainian},}
@article{weareaicomic_vol5_uk,author={Stoyanovich, Julia and Khan, Falaah Arif},title={We are AI (Ukrainian Edition)},journal={We are AI Comic Series},volume={5},year={2024},keywords={public,education,comics,ukrainian},}
2023
Responsible AI literacy: A stakeholder-first approach
@article{DominguezStoyanovich23,author={Dominguez, Daniel and Stoyanovich, Julia},title={Responsible AI literacy: A stakeholder-first approach},journal={Big Data and Society},year={2023},keywords={journal,education,rds},doi={https://doi.org/10.1177/20539517231219958},}
All Aboard! Making AI Education Accessible
Falaah Arif Khan, Lucius Bynum, Amy Hurst, Lucas Rosenblatt, Meghana Shanbhogue, Mona Sloane, and Julia Stoyanovich
Center for Responsible AI, New York University 2023
@article{AllAboard,author={{Arif Khan}, Falaah and Bynum, Lucius and Hurst, Amy and Rosenblatt, Lucas and Shanbhogue, Meghana and Sloane, Mona and Stoyanovich, Julia},title={{All Aboard! Making AI Education Accessible}},journal={Center for Responsible AI, New York University},keywords={panel,education,weareai},year={2023},}
The Algorithmic Transparency Playbook: A Stakeholder-first Approach
to Creating Transparency for Your Organization’s Algorithms
Andrew Bell, Oded Nov, and Julia Stoyanovich
In Extended Abstracts of the 2023 CHI Conference on Human Factors in
Computing Systems, CHI EA 2023, Hamburg, Germany, April 23-28,
2023 2023
@inproceedings{DBLP:conf/chi/BellNS23,author={Bell, Andrew and Nov, Oded and Stoyanovich, Julia},title={The Algorithmic Transparency Playbook: {A} Stakeholder-first Approach
to Creating Transparency for Your Organization's Algorithms},booktitle={Extended Abstracts of the 2023 {CHI} Conference on Human Factors in
Computing Systems, {CHI} {EA} 2023, Hamburg, Germany, April 23-28,
2023},pages={554:1--554:4},publisher={{ACM}},year={2023},site={https://r-ai.co/transparency-playbook},doi={10.1145/3544549.3574169},keywords={panel,policy,explanability,education,playbook,governance},addendum={peer-reviewed course},author+an={1=self;3=self}}
Think About the Stakeholders First! Towards an Algorithmic Transparency
Playbook for Regulatory Compliance
@article{bell_nov_stoyanovich_2023,author={Bell, Andrew and Nov, Oded and Stoyanovich, Julia},title={Think About the Stakeholders First! {T}owards an Algorithmic Transparency
Playbook for Regulatory Compliance},volume={5},journal={Data \& Policy},publisher={Cambridge University Press},year={2023},keywords={journal,policy,explainability,education,playbook,governance},}
2022
An Interactive Introduction to Causal Inference
Lucius E.J. Bynum, Falaah Arif Khan, Oleksandra Konopatska, Joshua R. Loftus, and Julia Stoyanovich
VISxAI: Workshop on Visualization for AI Explainability 2022
@article{bynum2022interactive,author={Bynum, Lucius E.J. and Khan, Falaah Arif and Konopatska, Oleksandra and Loftus, Joshua R. and Stoyanovich, Julia},title={An Interactive Introduction to Causal Inference},journal={VISxAI: Workshop on Visualization for AI Explainability},year={2022},site={https://r-ai.co/ci-playground},publisher={IEEE},keywords={workshop,education,playground,RAIforUkraine},}
Teaching Responsible Data Science
Julia Stoyanovich
In Proceedings of the 1st ACM SIGMOD International Workshop on Data
Systems Education: Bridging Education Practice with Education Research,
DataEd@SIGMOD 2022, 17 June 2022, Philadelphia, PA, USA 2022
@inproceedings{DBLP:conf/dataed/Stoyanovich22,author={Stoyanovich, Julia},editor={Aivaloglou, Efthimia and Fletcher, George and Miedema, Daphne},title={Teaching Responsible Data Science},booktitle={Proceedings of the 1st {ACM} {SIGMOD} International Workshop on Data
Systems Education: Bridging Education Practice with Education Research,
DataEd@SIGMOD 2022, 17 June 2022, Philadelphia, PA, {USA}},pages={4--9},publisher={{ACM}},year={2022},doi={10.1145/3531072.3535318},timestamp={Wed, 29 Jun 2022 12:08:29 +0200},bibsource={dblp computer science bibliography, https://dblp.org},keywords={invited,education},}
Teaching Responsible Data Science
Armanda Lewis, and Julia Stoyanovich
International Journal of Artificial Intelligence in Education (IJAIED) 2022
@article{LewisStoyanovich21,author={Lewis, Armanda and Stoyanovich, Julia},title={Teaching Responsible Data Science},journal={International Journal of Artificial Intelligence in Education (IJAIED)},volume={32},number={3},pages={783--807},year={2022},keywords={journal,education,rds},addendum={Special Issue: The FATE of AI in Education: Fairness, Accountability, Transparency, and Ethics},doi={https://doi.org/10.1007/s40593-021-00241-7},site={https://dataresponsibly.github.io/rds/},}
@article{weareaicomic_vol1,author={Stoyanovich, Julia and Khan, Falaah Arif},title={What is AI?},journal={We are AI Comic Series},volume={1},year={2021},keywords={public,education,comics,english},}
@article{weareaicomic_vol1_sp,author={Stoyanovich, Julia and Khan, Falaah Arif},title={What is AI? (Spanish Edition)},journal={We are AI Comic Series},volume={1},year={2021},keywords={public,education,comics,spanish},}
@article{weareaicomic_vol1_gr,author={Stoyanovich, Julia and Khan, Falaah Arif},title={What is AI? (Greek Edition)},journal={We are AI Comic Series},volume={1},year={2021},keywords={public,education,comics,other},}
@article{weareaicomic_vol2,author={Stoyanovich, Julia and Khan, Falaah Arif},title={Learning from data},journal={We are AI Comic Series},volume={2},year={2021},keywords={public,education,comics,english},}
@article{weareaicomic_vol2_sp,author={Stoyanovich, Julia and Khan, Falaah Arif},title={Learning from data (Spanish Edition)},journal={We are AI Comic Series},volume={2},year={2021},keywords={public,education,comics,spanish},}
Who lives, who dies, who decides?
Julia Stoyanovich, Mona Sloane, and Falaah Arif Khan
@article{weareaicomic_vol3,author={Stoyanovich, Julia and Sloane, Mona and Khan, Falaah Arif},title={Who lives, who dies, who decides?},journal={We are AI Comic Series},volume={3},year={2021},keywords={public,education,comics,english},}
Who lives, who dies, who decides? (Spanish Edition)
Julia Stoyanovich, Mona Sloane, and Falaah Arif Khan
@article{weareaicomic_vol3_sp,author={Stoyanovich, Julia and Sloane, Mona and Khan, Falaah Arif},title={Who lives, who dies, who decides? (Spanish Edition)},journal={We are AI Comic Series},volume={3},year={2021},keywords={public,education,comics,spanish},}
@article{weareaicomic_vol4,author={Stoyanovich, Julia and Khan, Falaah Arif},title={All about that bias},journal={We are AI Comic Series},volume={4},year={2021},keywords={public,education,comics,english}}
@article{weareaicomic_vol4_sp,author={Stoyanovich, Julia and Khan, Falaah Arif},title={All about that bias (Spanish Edition)},journal={We are AI Comic Series},volume={4},year={2021},keywords={public,education,comics,spanish}}
@article{weareaicomic_vol5,author={Stoyanovich, Julia and Khan, Falaah Arif},title={We are AI},journal={We are AI Comic Series},volume={5},year={2021},keywords={public,education,comics,english},}
@article{weareaicomic_vol5_sp,author={Stoyanovich, Julia and Khan, Falaah Arif},title={We are AI},journal={We are AI Comic Series},volume={5},year={2021},keywords={public,education,comics,spanish},}
Fairness and Friends
Falaah Arif Khan, Eleni Manis, and Julia Stoyanovich
@article{rdscomic_Vol2,author={Khan, Falaah Arif and Manis, Eleni and Stoyanovich, Julia},title={Fairness and Friends},journal={Data, Responsibly Comic Series},volume={2},year={2021},keywords={public,edu,comics,english},}
We are AI: Taking Control of Technology
Julia Stoyanovich, and Eric Corbett
Center for Responsible AI, New York University 2021
@article{rdscomic_vol1,author={Khan, Falaah Arif and Stoyanovich, Julia},title={Mirror, Mirror},journal={Data, Responsibly Comic Series},volume={1},year={2020},keywords={public,edu,comics,english},}
A Simple and Practical Method for Reducing the Disparate Impact of Differential PrivacyIn Thirty-Eighth AAAI Conference on Artificial Intelligence 2024
The Algorithmic Transparency Playbook: A Stakeholder-first Approach to Creating Transparency for Your Organization’s AlgorithmsIn Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA 2023, Hamburg, Germany, April 23-28, 2023 2023
Who lives? Who dies? Who decides? (Ukrainian Edition)We are AI Comic Series 2024
An Interactive Introduction to Causal InferenceVISxAI: Workshop on Visualization for AI Explainability 2022
Teaching Responsible Data ScienceInternational Journal of Artificial Intelligence in Education (IJAIED) 2022
We are AI: Taking Control of TechnologyCenter for Responsible AI, New York University 2021
