Community-Informed Policies and Best-Practices for the National Artificial Intelligence Research Resource (NAIRR)
Workshop Goals and Broad Overview
On July 29-31, 2024, an in-person workshop will be convened at New York University, bringing together leading experts in the cyberinfrastructure and AI policy / governance / responsible AI communities. The goals of this workshop are to to obtain community feedback regarding policies and best practices, and inform the development of priorities and policies for the National AI Research Resource (NAIRR). By convening a diverse group of stakeholders to offer their perspectives, the workshop will provide suggestions for improving the trustworthiness of the resources that are components of the NAIRR, and for supporting the equitable use of these resources by a diverse group of researchers and students.
This workshop is convened by Julia Stoyanovich, and is co-hosted by the NYU Tandon Center for Responsible AI and the NYU Center for Data Science. This workshop is supported by NSF Award No. 2432040 and by Omidyar Network.
Resources
For background, please refer to the following resources
- NAIRR website
- NAIRR Pilot website
- NAIRR Task Force Report
- NIST AI Risk Management Framework (AI RMF 1.0)
- Blueprint for the AI Bill of Rights
- Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence
- Executive Office of the President, Office of Management and Budget (OMB) Memorandum on Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence
Event Details
When: Monday, July 29, 2024, 4:00 pm EDT through Wednesday, July 31, 2024, 2:00 pm EDT
Where: NYU Center for Data Sciece, 60 5th Avenue, 7th Floor, New York, NY 10011
Who: See participant list
Agenda
July 29, 2024
- 4:00 - 4:30 PM: Arrival, registration
- 4:30 - 6:30 PM: Opening reception
July 30, 2024
- 9:00 am - 9:30 am: Arrival, registration, breakfast
- 9:30 am- 9:45 am: Welcome and opening remarks
- 9:45 am - 10:30 am: Introductions
- 10:30 am - 11:00 am: NAIRR Task Force panel
- 11:00 am - 11:15 am: Open discussion
- 11:15 am - 11:30 am: Update on the NAIRR
- 11:30 am - 11:45 am: Overview of afternoon activities, open discussion
- 11:45 am - 12:00 pm: Break
- 12:00 pm - 2:30 pm: Breakout sessions, with lunch
- NAIRR operations: Room 204
- Assessing progress towards advancing trustworthy AI: Room 206
- Community engagement: Room 446
- Engaging with industry partners: Room 650
- 2:30 pm - 3:00 pm: Coffee break
- 3:00 pm - 4:00 pm: Plenary readout and discussion
- 4:00 pm - 4:15 pm: Summary remarks
- 6:00 pm - 8:00 pm: Group dinner
July 31, 2024
- 9:00 am - 9:30 am: Arrival, breakfast
- 9:30 am - 9:45 am: Welcome back, recap, and overview of the day’s activities
- 9:45 am - 10:15 am: NAIRR Pilot partner and contributor panel
- 10:15 am - 10:30 am: Open discussion
- 10:30 am - 12:30 pm: Breakout sessions
- The NAIRR as a data equity infrastructure: Room 204
- Responsible development and use of generative AI: Room 206
- Education and training: Room 446
- Transparency as an enabler of trustworthy AI: Room 650
- 12:30 pm - 1:30 pm: Plenary readout and discussion, with lunch
- 13:30 pm - 1:45 pm: Open discussion
- 1:45 pm - 2:00 pm: Workshop closing, adjourn
Workshop Lead
 | Julia Stoyanovich - Associate Professor and Director of the Center for Responsible AI, New York University (NYU) Julia Stoyanovich is an Institute Associate Professor in the Department of Computer Science and Engineering at the Tandon School of Engineering, and an Associate Professor of Data Science at the Center for Data Science at New York University. She also directs the NYU Tandon Center for Responsible AI. |
Participants
 | Alan R. Blatecky — Visiting Fellow, Statistics and Psychometrics, RTI International Alan R. Blatecky is a visiting fellow with RTI, focusing on integrating and deploying advanced technologies to transform research and education. Prior to joining RTI, he served as the Director Office of Cyberinfrastructure National Science Foundation and the Deputy Director of Renaissance Computing Institute. |
 | Alejandro Suarez — Program Director, National Science Foundation (NSF) Alejandro Suarez is a Program Director of the Office of Advanced Cyberinfrastructure (CISE/OAC) at the National Science Foundation (NSF). |
 | Alondra Nelson — Harold F. Linder Professor, Institute for Advanced Study. Alondra Nelson is the Harold F. Linder Professor at the Institute for Advanced Study. She served as deputy assistant to President Joe Biden and acting director of the White House Office of Science and Technology Policy (OSTP). |
 | Anita Nikolich — Director of Research and Technology Innovation and Research Scientist, University of Illinois Urbana-Champaign Anita Nikolich is the Director of Research and Technology Innovation and Research Scientist at the University of Illinois Urbana-Champaign. She also served as Program Director for Cybersecurity in the Division of Advanced Cyberinfrastructure at the National Science Foundation (NSF). |
 | Bronson Messer — Distinguished Scientist and Director of Science, Oak Ridge Leadership Computing Facility (OLCF) at ORNL / Professor, University of Tennessee Bronson Messer is a Distinguished Scientist and Director of Science at the Oak Ridge Leadership Computing Facility (OLCF) at Oak Ridge National Laboratory. He is also Joint Faculty Professor in the Department of Physics & Astronomy at the University of Tennessee. |
 | Carol Song — Chief Scientist, Rosen Center for Advanced Computing (RCAC), Purdue University Carol Song is Chief Scientist at the Rosen Center for Advanced Computing (RCAC) at Purdue University. Carol is the Principal Investigator of Anvil, a new large-capacity national computational system funded by NSF in 2020. |
 | Chaitanya K. Baru — Senior Advisor, National Science Foundation (NSF) Chaitanya Baru is Senior Advisor in NSF's new Technology, Innovation, and Partnerships (TIP) Directorate. He joined NSF in October 2022 after a 25-year career at the San Diego Supercomputer Center, University of California, San Diego. |
 | Christine Cutillo — Health Data Scientist for AI Ethics, Office of Data Science Strategy (ODSS), National Institutes of Health (NIH) Christine Cutillo is a Health Data Scientist for AI Ethics in the Office of Data Science Strategy (ODSS), within the Integrated Infrastructure and Emerging Technologies (IIET) unit. She is responsible for multidisciplinary NIH-wide AI Ethics efforts – focusing on integrating ethics through the AI lifecycle in biomedical research applications. |
 | Danaë Metaxa — Assistant Professor in the Computer and Information Science department, University of Pennsylvania Danaë Metaxa is an Assistant Professor at the University of Pennsylvania in the Computer and Information Science department, with a secondary appointment in the Annenberg School for Communication. Along with Andrew Head, they are the co-founder of the Penn HCI group. |
 | Daniela Braga — Founder and CEO, Defined.ai Daniela Braga is founder and CEO of Defined.ai, one of the fastest-growing scale-ups in the AI space. Braga has been recognized as the Entrepreneur of the Year 2019 by the João Vasconcelos Award, and is an Entrepreneur Of The Year® 2020 Pacific Northwest finalist. |
 | David White — Senior Vice President, Technology Practice, RTI International David White is the senior Vice President of Technology Practice at RTI. He joined RTI in 2022. Prior, he served as the vice president and chief information officer for Battelle. |
 | Dawn Thurman — Associate Professor, Morgan State University Dawn Thurman is an Associate Professor at Morgan State University and a dedicated social work professional. |
 | Fred Streitz — Deputy Associate Director, Computing Directorate, Lawrence Livermore National Lab (LLNL) Fred Streitz is Chief Computational Scientist at Lawrence Livermore National Lab (LLNL) and Deputy Associate Director for Strategic Partnerships in the Computing Directorate. |
 | Govind Shivkumar — Director of Responsible Technology, Omidyar Network Govind Shivkumar is the Director of the Responsible Technology team at Omidyar Network. His work focuses on topics related to policy, advocacy, and investments that support digital forms of identification including privacy, user value and control, and security. |
 | Helen Nissenbaum — Andrew H. and Ann R. Tisch Professor, Cornell Tech Helen Nissenbaum is the Andrew H. and Ann R. Tisch Professor at Cornell Tech and in the Information Science Department at Cornell University. She is also Director of the Digital Life Initiative, a research lab dedicated to exploring societal perspectives surrounding digital technologies. |
 | HV Jagadish — Director of the Michigan Institute for Data and AI in Society (MIDAS), University of Michigan HV Jagadish is the Director of the Michigan Institute for Data and AI in Society and Bernard A Galler Collegiate Professor of Electrical Engineering and Computer Science at University of Michigan. |
 | Ilan Strauss - Program Director of the AI Disclosures Project at the Social Science Research Council Ilan Strauss is Program Director of the AI Disclosures Project at the Social Science Research Council (SSRC). br> Institutional Website |
 | Ilkay Altintaş — Research Scientist, University of California San Diego İlkay Altıntaş is a Research Scientist at the University of California San Diego, Chief Data Science Officer of the San Diego Supercomputer Center as well as a Founding Fellow of the Halıcıoğlu Data Science Institute. She is the Founding Director of the Workflows for Data Science (WorDS) Center of Excellence and the WIFIRE Lab. |
 | Jeanna Matthews — Professor of Computer Science, Clarkson University / DuckDuckGo Jeanna Matthews is a professor of computer science at Clarkson University. She is a a Chair of the IEEE-USA AI Policy Committee and a founding co-chair of the ACM Technology Policy Subcommittee on Artificial Intelligence and Algorithmic Accountability. She is a member of the NSF Computer and Information Science and Engineering Advisory Committee. |
 | Jennifer Wortman Vaughan — Senior Principal Researcher, Microsoft Research Jennifer Wortman Vaughan is a Senior Principal Researcher at Microsoft Research in New York City. She works on responsible AI—specifically transparency, interpretability, and fairness of AI systems—as part of Microsoft's FATE group and Co-Chair of Microsoft's Aether working group on transparency. |
 | Joshua A. Tucker — Professor, Director of the Jordan Center for Advanced Study of Russia, New York University (NYU) Joshua A. Tucker is a Professor of Politics, an affiliated Professor of Russian and Slavic Studies, and an affiliated Professor of Data Science at New York University. He’s the Director of NYU’s Jordan Center for Advanced Study of Russia, a co-director of the NYU Center for Social Media and Politics, and a co-editor of the politics and policy blog The Monkey Cage at The Washington Post. |
 | Joshua M. Greenberg — Program Director, Alfred P. Sloan Foundation Joshua Greenberg is a Program Director at the Alfred P. Sloan Foundation, where he is responsible for overseeing the Technology and New York City programs. From 2007 to 2010, he served as the New York Public Library's first Director of Digital Strategy and Scholarship. |
 | Julia Lane — Professor, NYU Wagner Graduate School of Public Service Julia is a tenured full Professor at the NYU Wagner Graduate School of Public Service. She recently served on the Advisory Committee on Data for Evidence Building and the National AI Research Resources Task Force. |
 | Katie Shilton — Professor, Program Co-Director, College of Information Studies, University of Maryland Katie Shilton is a Professor and Program Co-Director of the Bachelor’s Program in Social Data Science at the University of Maryland. She also leads the Ethics & Values in Design (EViD) Lab and Principal Investigator of the PERVADE project, a multi-campus collaboration focused on big data research ethics. |
 | Kristian J. Hammond — Bill and Cathy Osborn Professor of Computer Science and Director of CASMI, Northwestern University Kristian J. Hammond is a Bill and Cathy Osborn Professor of Computer Science at Northwestern University. He is also the Director of the Master of Science in Artificial Intelligence Program, as well as Director of the Center for Advancing Safety of Machine Intelligence (CASMI). |
 | Laura Courchesne — Head of Strategy and Operations, Frontier Model Forum Laura Courchesne is the Head of Strategy and Operations at Frontier Model Forum. She’s also the Co-Founder and served as Co-Director of the Centre for AI, Data and Conflict. |
 | Leo Peyronnin — Associate, Programs, Omidyar Network Leo Peyronnin serves as an Associate on the Reimagining Capitalism team at Omidyar Network. In this role, he supports a portfolio focused on Corporations, Capital Markets, and the Common Good. |
 | Manish Parashar — Director of the Scientific Computing and Imaging Institute and Presidential Professor in the Kahlert School of Computing, University of Utah Manish Parashar is Director of the Scientific Computing and Imaging Institute and Presidential Professor in the University of Utah’s Kahlert School of Computing. He recently completed an IPA term as Office Director of NSF's Office of Advanced Cyberinfrastructure, where he oversaw investments in national cyberinfrastructure. |
 | Michael Garris — Senior Principal AI and Autonomy Policy Analyst at MITRE Michael Garris is a Senior Principal Artificial Intelligence and Autonomy Policy Analyst at MITRE. |
 | Michael Holland — Vice Chancellor for Science Policy and Research Strategies, University of Pittsburgh Michael Holland is the Vice Chancellor for Science Policy and Research Strategies at the University of Pittsburgh. Previously, he served as Executive Director of New York University’s Center for Urban Science and Progress. |
 | Michael E. Papka — Deputy Associate Laboratory Director/Director, Argonne Leadership Computing Facility (ALCF) / Professor, University of Illinois Chicago Michael E. Papka is a senior scientist at Argonne National Laboratory, where he is also deputy associate laboratory director for Computing, Environment and Life Sciences (CELS) and division director of the Argonne Leadership Computing Facility (ALCF). In addition to his duties at Argonne, Michael is a professor of computer science at University of Illinois Chicago. |
 | Paola Buitrago - Director of Artificial Intelligence and Big Data, Pittsburgh Supercomputing Center, Carnegie Melon University Paola Buitrago leads the Artificial Intelligence and Big Data group at the Pittsburgh Supercomputing Center, which is a joint effort of Carnegie Mellon University and the University of Pittsburgh. Her group is focused on advancing and supporting the convergence of High Performance Computing (HPC), Artificial Intelligence (AI) and Big Data. |
 | Pete Beckman — Senior Scientist, Argonne National Laboratory Pete Beckman is a Senior Scientist at the Argonne National Laboratory. He was the founding co-director for the Northwestern University / Argonne Institute for Science and Engineering in 2012 and held that position for more than 12 years. |
 | Rayid Ghani — Distinguished Career Professor, Carnegie Mellon University Rayid Ghani is a Professor in the Machine Learning Department (in the School of Computer Science) and the Heinz College of Information Systems and Public Policy at Carnegie Mellon University and leads the Data Science and Public Policy Group as well as the Data Science for Social Good Program. He’s also the co-lead of the Responsible AI Initiative at CMU. |
 | Rebecca Boyles — Deputy Director, Renaissance Computing Institute (RENCI), University of North Carolina at Chapel Hill Rebecca Boyles is the Deputy Director of the Renaissance Computing Institute at UNC Chapel Hill. Prior to joining RENCI, she was the Founding Director of the Center for Data Modernization Solutions at RTI International. Her mission is to transform and upgrade the data infrastructure of large public health and scientific agencies, enabling them to leverage data for better decision-making, research, and policy. |
 | Rumman Chowdhury - CEO of Humane Intelligence and the United States Science Envoy for Artificial Intelligence Rumman Chowdhury is a data and social scientist, CEO of the tech nonprofit Humane Intelligence, and the United States Science Envoy for Artificial Intelligence. She is also a Responsible AI Fellow at Harvard's Berkman Klein Center. |
 | Russel Wald — Deputy Director, Stanford’s Institute for Human-Centered Artificial Intelligence (HAI) Russell Wald is the Deputy Director of Stanford’s Institute for Human-Centered Artificial Intelligence (HAI). From 2020 - 2022 he served as HAI's first Director of Policy and later Managing Director for Policy and Society. |
 | Sergiu Sanielevici — Director, Support for Scientific Applications, Pittsburgh Supercomputing Center (PSC), Carnegie Mellon University Sergiu Sanielevici is the Director of Support for Scientific Applications at the Pittsburgh Supercomputing Center (PSC). He currently serves as the Principal Investigator of the Bridges-2 project and as co-Principal Investigator of the Neocortex project at Carnegie Mellon University. |
 | Solon Barocas — Principal Researcher, Microsoft Research / Adjunct Assistant Professor, Cornell University Solon Barocas is the Principal Researcher in the New York City lab of Microsoft Research, where he’s a member of the Fairness, Accountability, Transparency, and Ethics in AI (FATE) research group. He’s also an Adjunct Assistant Professor in the Department of Information Science at Cornell University, where he co-lead the initiative on Artificial Intelligence, Policy, and Practice (AIPP). |
 | Srinivasan Parthasarathy — Professor, Department of Computer Science and Engineering, Ohio State University Srinivasan Parthasarathy is a Professor and University Distinguished Scholar of Computer Science and Engineering at Ohio State. His research interests span databases, machine learning, high-performance computing and responsible data science. |
 | Stefaan Verhulst — Co-Founder, Chief Research and Development Officer, Director, The Governance Lab's Data Program Stefaan G. Verhulst is the Co-Founder and Chief Research and Development Officer as well as Director of GovLab's Data Program. He is also the Editor-in-Chief of Data & Policy and the research director of the MacArthur Research Network on Opening Governance. |
 | Susan Aaronson - Research Professor; Director, Digital Trade and Data Governance Hub, George Washington University Susan Ariel Aaronson is Research Professor of International Affairs at George Washington University (GWU). Aaronson is also co-principal investigator with the NSF-NIST Institute for Trustworthy AI in Law & Society (TRAILS), where she leads research on data and AI governance. |
 | Suzette Kent — Kent Advisory Services / former Federal Chief Information Officer of the United States Suzette Kent is a global business transformation executive at Kent Advisory Services. Previously, she served as Federal Chief Information Officer of the United States from 2018 until 2020. |
 | Tim O'Reilly - Founder, CEO and Chairman, O'Reilly Media Tim O'Reilly is the founder, CEO, and Chairman of O’Reilly Media, and the Principal Investigator and Co-Director of the AI Disclosures Project at the Social Science Research Council. Known for coining terms such as “Open Source” and “Web 2.0,” Tim has pioneered technology publishing, conferences, digital media, and the concepts we use to understand technology trends. |
 | Tom Goldstein - Volpi-Cupal Endowed Professor of Computer Science, University of Maryland Tom Goldstein is a Volpi-Cupal Endowed Professor of Computer Science with an appointment in the University of Maryland Institute for Advanced Computer Studies, where he is also the director of the University of Maryland Center for Machine Learning. He is also co-principal investigator with the NSF-NIST Institute for Trustworthy AI in Law & Society (TRAILS), where he leads the methods and metrics research thrust. |
 | Travis Hoppe — Assistant Director of AI Research and Development, White House Office of Science and Technology Policy (OSTP) Travis Hoppe is the Assistant Director of AI Research and Development at the White House Office of Science and Technology Policy (OSTP). |
 | Varun Chandola — Program Director, National Science Foundation (NSF) Varun Chandola is a Program Director of the Office of Advanced Cyberinfrastructure (CISE/OAC) at the National Science Foundation (NSF). Previously, he worked as an Associate Professor at University at Buffalo. |
 | William D. Gropp — Director of the National Center for Supercomputing Applications, University of Illinois Urbana-Champaign William Gropp is the director of the National Center for Supercomputing Applications (NCSA), holds a Grainger Distinguished Chair in Engineering, and is a professor in the Siebel School of Computing and Data at the University of Illinois at Urbana–Champaign. |
 | William L. Miller — Senior Advisor, National Science Foundation (NSF) William L. Miller is Senior Advisor for Cyberinfrastructure within the Office of Advanced Cyberinfrastructure (CISE/OAC) at the National Science Foundation (NSF). |
 | Yacine Jernite — Machine Learning and Society Lead, Hugging Face Yacine Jernite leads the ML and Society team at Hugging Face, where he works on ML systems governance at the intersection of regulatory and technical tools. |
Local Organizers
Andrew Bell, NYU Tandon School of Engineering Institutional Website
Caterina Fuligni, NYU Tandon School of Engineering
Chastity Hidalgo, NYU Center for Data Science
Lucas Rosenblatt, NYU Tandon School of Engineering Institutional Website
Lucius Bynum, NYU Center for Data Science Institutional Website
Sarah Lawson, NYU Tandon School of Engineering
Venetia Pliatskia, NYU Tandon School of Engineering Institutional Website
Breakout Sessions
Session 1: NAIRR operations
Moderators: Anita Nikolich and Pete Beckman
Summary: The NAIRR is envisioned as a national cyberinfrastructure to democratize and accelerate AI R&D. Its success is predicated on an effective operational strategy, which must align with the four measurable goals of the NAIRR, and, specifically, with advancing trustworthy AI - through its own operations and also as an enabler of trustworthy AI research.
Session 2: Assessing progress towards trustworthy AI
Moderators: Ilkay Altintas and Stefaan Verhulst
Summary: The NAIRR Task Force report presents a theory of change and proposes key performance indicators (KPIs) to assess progress towards the four measurable goals. Yet, progress towards trustworthy AI is particularly challenging to assess, due to the complex socio-technical dynamics, and to the impacts that a cyberinfrastructure has that are outside the system’s direct control. Developing assessment methodologies and defining performance indicators requires a robust dialogue between the responsible AI and cyberinfrastructure communities.
Session 3: Community engagement
Moderators: Dawn Thurman and Danae Metaxa
Summary: The NAIRR aims to democratize access to the AI innovation ecosystem. The NAIRR Task Force report identifies US-based researchers, educators, students, and small and medium-sized businesses as its primary constituency. Additionally, in alignment with the goals of protecting privacy, civil rights and civil liberties, and of ensuring that AI development benefits society at large, there is a need to give a voice to the individuals and groups whose data fuels the ecosystem.
Session 4: Engaging industry partners
Moderators: Julia Stoyanovich and Solon Barocas
Summary: The White House Executive Order directs the NSF to promote innovation by developing and strengthening “public-private partnerships for advancing innovation, commercialization, and risk-mitigation methods for AI” and by helping promote “safe, responsible, fair, privacy-protecting and trustworthy AI systems.” Thus, engaging industry partners is both a priority and a necessity for the NAIRR.
Session 5: Transparency as an enabler of trustworthy AI
Moderators: Jenn Wortman Vaughan and Julia Stoyanovich
Summary: Transparency is concerned with providing the information that stakeholders need to make informed, responsible, and ethical decisions for data and model development, sharing, and reuse. Transparency is a big-tent concept that may be interpreted differently depending on the stakeholders, the context of use, and the applicable legal and ethical considerations. In the context of the NAIRR, an actionable interpretation of transparency centers on the role of data, model, and lifecycle documentation as an enabler of trustworthy AI.
Session 6: The NAIRR as a data equity infrastructure
Moderators: H V Jagadish and Katie Shilton
The NAIRR aims to democratize access to datasets and data products (most prominently, models) to accelerate AI R&D while facilitating their responsible sharing and use. Data equity is concerned with the identification and, when possible, mitigation of biases in data and models. Importantly, because data and models are created by complex multi-step processes, and are often re-purposed, bias detection and mitigation requires lifecycle-wide support.
Session 7: Education and training
Moderators: Kristian Hammond and Sergiu Sanielevici
Summary: Education and training are essential for meeting all four measurable goals of the NAIRR. As a cyberinfrastructure, the NAIRR will offer training capabilities to ensure that its users have the skills to use the resources. However, to advance the goals of trustworthy AI, it is crucial to train researchers and students on the principles and techniques of responsible data engineering, algorithmic fairness, transparency, and privacy and data protection. This type of training should also be offered to current data scientists, particularly those who work in small and medium-sized businesses, which typically cannot afford the necessary upskilling of their staff.
Session 8: Responsible development and use of generative AI
Moderators: Jeanna Matthews and Yacine Jernite
Summary: Generative AI is seeing ever more research activity and ever broader commercial adoption. To put generative AI into safe use as part of the NAIRR, it is essential to articulate robust policies and best practices for assessing its performance, in terms of veracity, safety, fairness, reproducibility, and legal compliance. Further, it is crucial to develop technical support for the evaluation of systems of which generative AI models form part, rather than of these models in isolation, to assess their impacts in the context of design, development and use.
Questions?
If you have any questions, please contact us by emailing stoyanovich (at) nyu (dot) edu.