Community-Informed Policies and Best-Practices for the National Artificial Intelligence Research Resource (NAIRR)


Workshop Goals and Broad Overview

On July 29-31, 2024, an in-person workshop will be convened at New York University, bringing together leading experts in the cyberinfrastructure and AI policy / governance / responsible AI communities. The goals of this workshop are to to obtain community feedback regarding policies and best practices, and inform the development of priorities and policies for the National AI Research Resource (NAIRR). By convening a diverse group of stakeholders to offer their perspectives, the workshop will provide suggestions for improving the trustworthiness of the resources that are components of the NAIRR, and for supporting the equitable use of these resources by a diverse group of researchers and students.

This workshop is convened by Prof. Julia Stoyanovich, and is co-hosted by the NYU Tandon Center for Responsible AI and the NYU Center for Data Science. This workshop is supported by NSF Award No. 2432040 and by Omidyar Network.

Resources

For background, please refer to the following resources

Event Details

When: Monday, July 29, 2024, 4:00 pm EDT through Wednesday, July 31, 2024, 2:00 pm EDT

Where: NYU Center for Data Sciece, 60 5th Avenue, 7th Floor, New York, NY 10011

Who: See participant list

Agenda

July 29, 2024
  • 4:00 - 4:30 PM: Arrival, registration
  • 4:30 - 6:30 PM: Opening reception
July 30, 2024
  • 9:00 am - 9:30 am: Arrival, registration, breakfast
  • 9:30 am- 9:45 am: Welcome and opening remarks
    • Julia Stoyanovich
  • 9:45 am - 10:30 am: Introductions
  • 10:30 am - 11:00 am: NAIRR Task Force panel
  • 11:00 am - 11:15 am: Open discussion
  • 11:15 am - 11:30 am: Update on the NAIRR
    • Robert Beverly and Alejandro Suarez
  • 11:30 am - 11:45 am: Overview of afternoon activities, open discussion
  • 11:45 am - 12:00 pm: Break
  • 12:00 pm - 2:30 pm: Breakout sessions, with lunch
  • 2:30 pm - 3:00 pm: Coffee break
  • 3:00 pm - 4:00 pm: Plenary readout and discussion
  • 4:00 pm - 4:15 pm: Summary remarks
  • 6:00 pm - 8:00 pm: Group dinner
July 31, 2024
Note: The schedule and sessions are subject to change. Please check back for the latest updates.



Workshop Lead

Julia Stoyanovich - Associate Professor and Director of the Center for Responsible AI, New York University (NYU)

Julia Stoyanovich is an Institute Associate Professor in the Department of Computer Science and Engineering at the Tandon School of Engineering, and an Associate Professor of Data Science at the Center for Data Science at New York University. She also directs the NYU Tandon Center for Responsible AI.
Institutional Website

Participants

Alan R. Blatecky — Visiting Fellow, Statistics and Psychometrics, RTI International

Alan R. Blatecky is a visiting fellow with RTI, focusing on integrating and deploying advanced technologies to transform research and education. Prior to joining RTI, he served as the Director Office of Cyberinfrastructure National Science Foundation and the Deputy Director of Renaissance Computing Institute.
Institutional Website

Alejandro Suarez — Program Director, National Science Foundation (NSF)

Alejandro Suarez is a Program Director of the Office of Advanced Cyberinfrastructure (CISE/OAC) at the National Science Foundation (NSF).
Institutional Website

Alondra Nelson — Harold F. Linder Professor, Institute for Advanced Study

Alondra Nelson is the Harold F. Linder Professor at the Institute for Advanced Study. She served as deputy assistant to President Joe Biden and acting director of the White House Office of Science and Technology Policy (OSTP).
Personal Website

Anita Nikolich — Director of Research and Technology Innovation and Research Scientist, University of Illinois Urbana-Champaign

Anita Nikolich is the Director of Research and Technology Innovation and Research Scientist at the University of Illinois Urbana-Champaign. She also served as Program Director for Cybersecurity in the Division of Advanced Cyberinfrastructure at the National Science Foundation (NSF).
Institutional Website

Bronson Messer — Distinguished Scientist and Director of Science, Oak Ridge Leadership Computing Facility (OLCF) at ORNL / Professor, University of Tennessee

Bronson Messer is a Distinguished Scientist and Director of Science at the Oak Ridge Leadership Computing Facility (OLCF) at Oak Ridge National Laboratory. He is also Joint Faculty Professor in the Department of Physics & Astronomy at the University of Tennessee.
Institutional Website

Carol Song — Chief Scientist, Rosen Center for Advanced Computing (RCAC), Purdue University

Carol Song is Chief Scientist at the Rosen Center for Advanced Computing (RCAC) at Purdue University. Carol is the Principal Investigator of Anvil, a new large-capacity national computational system funded by NSF in 2020.
Institutional Website

Chaitanya K. Baru — Senior Advisor, National Science Foundation (NSF)

Chaitanya Baru is Senior Advisor in NSF's new Technology, Innovation, and Partnerships (TIP) Directorate. He joined NSF in October 2022 after a 25-year career at the San Diego Supercomputer Center, University of California, San Diego.
Institutional Website

Christine Cutillo — Data Science Lead, National Center for Advancing Translational Sciences (NCATS), National Institutes of Health

Christine Cutillo is Data Science Lead at the National Center for Advancing Translational Sciences (NCATS) within the Office of Data Science Strategy at the National Institutes of Health (NIH).
Institutional Website

Dan Stanzione — Associate Vice President For Research and Executive Director of the Texas Advanced Computing Center (TACC), University of Texas

Dan Stanzione is Associate Vice President for Research at The University of Texas at Austin and Executive Director of the Texas Advanced Computing Center (TACC). He is the Principal Investigator for several projects including a National Science Foundation (NSF) grant to acquire and deploy Frontera, which will be the fastest supercomputer at a U.S. university.
Institutional Website

Danaë Metaxa — Assistant Professor in the Computer and Information Science department, University of Pennsylvania

Danaë Metaxa is an Assistant Professor at the University of Pennsylvania in the Computer and Information Science department, with a secondary appointment in the Annenberg School for Communication. Along with Andrew Head, they are the co-founder of the Penn HCI group.
Personal Website

Daniela Braga — Founder and CEO, Defined.ai

Daniela Braga is founder and CEO of Defined.ai, one of the fastest-growing scale-ups in the AI space. Braga has been recognized as the Entrepreneur of the Year 2019 by the João Vasconcelos Award, and is an Entrepreneur Of The Year® 2020 Pacific Northwest finalist.
Institutional Website

David White — Senior Vice President, Technology Practice, RTI International

David White is the senior Vice President of Technology Practice at RTI. He joined RTI in 2022. Prior, he served as the vice president and chief information officer for Battelle.
Institutional Website

Dawn Thurman — Associate Professor, Morgan State University

Dawn Thurman is an Associate Professor at Morgan State University and a dedicated social work professional.
Institutional Website

Divesh Srivastava — Head of Database Research, AT&T Labs-Research

Divesh Srivastava is the head of the Database Research Department at AT&T Labs-Research. He is on the board of trustees of the VLDB Endowment, the associate Editor-in-Chief of the IEEE Transactions on Knowledge and Data Engineering, and an associate editor of the ACM Transactions on Database Systems.

Personal Website

Fred Streitz — Deputy Associate Director, Computing Directorate, Lawrence Livermore National Lab (LLNL)

Fred Streitz is Chief Computational Scientist at Lawrence Livermore National Lab (LLNL) and Deputy Associate Director for Strategic Partnerships in the Computing Directorate.
Institutional Website

Govind Shivkumar — Director of Responsible Technology, Omidyar Network

Govind Shivkumar is the Director of the Responsible Technology team at Omidyar Network. His work focuses on topics related to policy, advocacy, and investments that support digital forms of identification including privacy, user value and control, and security.
Institutional Website

Helen Nissenbaum — Andrew H. and Ann R. Tisch Professor, Cornell Tech

Helen Nissenbaum is the Andrew H. and Ann R. Tisch Professor at Cornell Tech and in the Information Science Department at Cornell University. She is also Director of the Digital Life Initiative, a research lab dedicated to exploring societal perspectives surrounding digital technologies.
Personal Website

HV Jagadish — Director of the Michigan Institute for Data Science, University of Michigan

HV Jagadish is the Director of the Michigan Institute for Data Science and Bernard A Galler Collegiate Professor of Electrical Engineering and Computer Science at University of Michigan.
Institutional Website

Ilkay Altintaş — Research Scientist, University of California

İlkay Altıntaş is a Research Scientist at the University of California San Diego, Chief Data Science Officer of the San Diego Supercomputer Center as well as a Founding Fellow of the Halıcıoğlu Data Science Institute. She is the Founding Director of the Workflows for Data Science (WorDS) Center of Excellence and the WIFIRE Lab.
Institutional Website

Jeanna Matthews — Professor of Computer Science, Clarkson University / DuckDuckGo

Jeanna Matthews is a full professor of computer science at Clarkson University and an affiliate at Data and Society. She is an ACM Distinguished Speaker, a Fulbright Specialist, founding co-chair of the ACM Technology Policy Subcommittee on Artificial Intelligence and Algorithm Accountability and a member of the ACM Technology Policy Committee.
Institutional Website

Jennifer Wortman Vaughan — Senior Principal Researcher, Microsoft Research

Jennifer Wortman Vaughan is a Senior Principal Researcher at Microsoft Research in New York City. She works on responsible AI—specifically transparency, interpretability, and fairness of AI systems—as part of Microsoft's FATE group and Co-Chair of Microsoft's Aether working group on transparency.
Personal Website

Joshua A. Tucker — Professor, Director of the Jordan Center for Advanced Study of Russia, New York University (NYU)

Joshua A. Tucker is a Professor of Politics, an affiliated Professor of Russian and Slavic Studies, and an affiliated Professor of Data Science at New York University. He’s the Director of NYU’s Jordan Center for Advanced Study of Russia, a co-director of the NYU Center for Social Media and Politics, and a co-editor of the politics and policy blog The Monkey Cage at The Washington Post.
Institutional Website

Joshua M. Greenberg — Program Director, Alfred P. Sloan Foundation

Joshua Greenberg is a Program Director at the Alfred P. Sloan Foundation, where he is responsible for overseeing the Technology and New York City programs. From 2007 to 2010, he served as the New York Public Library's first Director of Digital Strategy and Scholarship.
Institutional Website

Julia Lane — Professor, NYU Wagner Graduate School of Public Service

Julia is a tenured full Professor at the NYU Wagner Graduate School of Public Service. She recently served on the Advisory Committee on Data for Evidence Building and the National AI Research Resources Task Force.
Personal Website

Katie Shilton — Professor, Program Co-Director, College of Information Studies, University of Maryland

Katie Shilton is a Professor and Program Co-Director of the Bachelor’s Program in Social Data Science at the University of Maryland. She also leads the Ethics & Values in Design (EViD) Lab and Principal Investigator of the PERVADE project, a multi-campus collaboration focused on big data research ethics.
Institutional Website

Kristian J. Hammond — Bill and Cathy Osborn Professor of Computer Science, Northwestern University / Director, CASMI

Kristian J. Hammond is a Bill and Cathy Osborn Professor of Computer Science at Northwestern University. He is also the Director of the Master of Science in Artificial Intelligence Program, as well as Director of the Center for Advancing Safety of Machine Intelligence (CASMI).
Institutional Website

Laura Courchesne — Head of Strategy and Operations, Frontier Model Forum

Laura Courchesne is the Head of Strategy and Operations at Frontier Model Forum. She’s also the Co-Founder and served as Co-Director of the Centre for AI, Data and Conflict.

Leo Peyronnin — Associate, Programs, Omidyar Network

Leo Peyronnin serves as an Associate on the Reimagining Capitalism team at Omidyar Network. In this role, he supports a portfolio focused on Corporations, Capital Markets, and the Common Good.
Institutional Website

Manish Parashar — Director of the Scientific Computing and Imaging Institute and Presidential Professor in the Kahlert School of Computing, University of Utah

Manish Parashar is Director of the Scientific Computing and Imaging Institute and Presidential Professor in the University of Utah’s Kahlert School of Computing. He recently completed an IPA term as Office Director of NSF's Office of Advanced Cyberinfrastructure, where he oversaw investments in national cyberinfrastructure.
Personal Website

Michael E. Papka — Deputy Associate Laboratory Director/Director, Argonne Leadership Computing Facility (ALCF) / Professor, University of Illinois Chicago

Michael E. Papka is a senior scientist at Argonne National Laboratory, where he is also deputy associate laboratory director for Computing, Environment and Life Sciences (CELS) and division director of the Argonne Leadership Computing Facility (ALCF). In addition to his duties at Argonne, Michael is a professor of computer science at University of Illinois Chicago.

Institutional Website

Michael Holland — Vice Chancellor for Science Policy and Research Strategies, University of Pittsburgh

Michael Holland is the Vice Chancellor for Science Policy and Research Strategies at the University of Pittsburgh. Previously, he served as Executive Director of New York University’s Center for Urban Science and Progress.
Institutional Website

Pete Beckman — Senior Scientist, Argonne National Laboratory

Pete Beckman is a Senior Scientist at the Argonne National Laboratory. He was the founding co-director for the Northwestern University / Argonne Institute for Science and Engineering in 2012 and held that position for more than 12 years.
Institutional Website

Rayid Ghani — Distinguished Career Professor, Carnegie Mellon University

Rayid Ghani is a Professor in the Machine Learning Department (in the School of Computer Science) and the Heinz College of Information Systems and Public Policy at Carnegie Mellon University and leads the Data Science and Public Policy Group as well as the Data Science for Social Good Program. He’s also the co-lead of the Responsible AI Initiative at CMU.

Personal Website

Robert Beverly — Program Director, National Science Foundation (NSF)

Robert Beverly is currently on detail at the National Science Foundation as a Program Director in the CISE Office of Advanced Cyberinfrastructure. He leads the Center for Measurement and Analysis of Network Data (CMAND) lab. His group's research focuses on large-scale network traffic analysis and cybersecurity.
Institutional Website

Rebecca Boyles — Founding Director, Center for Data Modernization Solutions, RTI International

Rebecca Boyles is the Founding Director of the Center for Data Modernization Solutions at RTI International. Her mission is to transform and upgrade the data infrastructure of large public health and scientific agencies, enabling them to leverage data for better decision-making, research, and policy.
Institutional Website

Russel Wald — Deputy Director of Policy, Stanford’s Institute for Human-Centered Artificial Intelligence (HAI)

Russell Wald is the Director of Policy for Stanford’s Institute for Human-Centered Artificial Intelligence (HAI). From 2020 - 2022 he served as HAI's first Director of Policy and later Managing Director for Policy and Society.
Institutional Website

Sergiu Sanielevici — Director, Support for Scientific Applications, Pittsburgh Supercomputing Center (PSC) / co-PI, Carnegie Mellon University

Sergiu Sanielevici is the Director of Support for Scientific Applications at the Pittsburgh Supercomputing Center (PSC). He currently serves as the Principal Investigator of the Bridges-2 project and as co- Principal Investigator of the Neocortex project at Carnegie Mellon University.
Institutional Website

Solon Barocas — Principal Researcher, Microsoft Research / Adjunct Assistant Professor, Cornell University

Solon Barocas is the Principal Researcher in the New York City lab of Microsoft Research, where he’s a member of the Fairness, Accountability, Transparency, and Ethics in AI (FATE) research group. He’s also an Adjunct Assistant Professor in the Department of Information Science at Cornell University, where he co-lead the initiative on Artificial Intelligence, Policy, and Practice (AIPP).
Personal Website

Srinivasan Parthasarathy — Professor, Department of Computer Science and Engineering, Ohio State University

Srinivasan Parthasarathy is a Professor in the Department of Computer Science and Engineering at Ohio State University. In 2024, he earned The Ohio State University Distinguished Scholar Award.
Institutional Website

Stefaan Verhulst — Co-Founder, Chief Research and Development Officer, Director, The Governance Lab's Data Program

Stefaan G. Verhulst is the Co-Founder and Chief Research and Development Officer as well as Director of GovLab's Data Program. He is also the Editor-in-Chief of Data & Policy and the research director of the MacArthur Research Network on Opening Governance.
Institutional Website

Suzette Kent — Kent Advisory Services / former Federal Chief Information Officer of the United States

Suzette Kent is a global business transformation executive at Kent Advisory Services. Previously, she served as Federal Chief Information Officer of the United States from 2018 until 2020.

Travis Hoppe — Assistant Director of AI Research and Development, White House Office of Science and Technology Policy (OSTP)

Travis Hoppe is the Assistant Director of AI Research and Development at the White House Office of Science and Technology Policy (OSTP).

Varun Chandola — Program Director, National Science Foundation (NSF)

Varun Chandola is a Program Director of the Office of Advanced Cyberinfrastructure (CISE/OAC) at the National Science Foundation (NSF). Previously, he worked as an Associate Professor at University at Buffalo.
Institutional Website

William D. Gropp — Director of the National Center for Supercomputing Applications, University of Illinois Urbana-Champaign

William Douglas Gropp is the director of the National Center for Supercomputing Applications (NCSA) and Thomas M. Siebel Chair in the Department of Computer Science at the University of Illinois at Urbana–Champaign. He is also the founding Director of the Parallel Computing Institute.
Institutional Website

William L. Miller — Senior Advisor, National Science Foundation (NSF)

William L. Miller is Senior Advisor for Cyberinfrastructure within the Office of Advanced Cyberinfrastructure (CISE/OAC) at the National Science Foundation (NSF).
Institutional Website

Yacine Jernite — Machine Learning and Society Lead, Hugging Face

Yacine Jernite leads the ML and Society team at Hugging Face, where he works on ML systems governance at the intersection of regulatory and technical tools.
Personal Website


Local Organizers

Andrew Bell, NYU Tandon School of Engineering Institutional Website

Caterina Fuligni, NYU Tandon School of Engineering

Chastity Hidalgo, NYU Center for Data Science

Lucas Rosenblatt, NYU Tandon School of Engineering Institutional Website

Lucius Bynum, NYU Center for Data Science Institutional Website

Sarah Lawson, NYU Tandon School of Engineering

Venetia Pliatskia, NYU Tandon School of Engineering Institutional Website

Breakout Sessions

Session 1: NAIRR operations

Summary: The NAIRR is envisioned as a national cyberinfrastructure to democratize and accelerate AI R&D. Its success is predicated on an effective operational strategy, which must align with the four measurable goals of the NAIRR, and, specifically, with advancing trustworthy AI - through its own operations and also as an enabler of trustworthy AI research.

Session 2: Assessing progress towards trustworthy AI

Summary: The NAIRR Task Force report presents a theory of change and proposes key performance indicators (KPIs) to assess progress towards the four measurable goals. Yet, progress towards trustworthy AI is particularly challenging to assess, due to the complex socio-technical dynamics, and to the impacts that a cyberinfrastructure has that are outside the system’s direct control. Developing assessment methodologies and defining performance indicators requires a robust dialogue between the responsible AI and cyberinfrastructure communities.

Session 3: Community engagement

Summary: The NAIRR aims to democratize access to the AI innovation ecosystem. The NAIRR Task Force report identifies US-based researchers, educators, students, and small and medium-sized businesses as its primary constituency. Additionally, in alignment with the goals of protecting privacy, civil rights and civil liberties, and of ensuring that AI development benefits society at large, there is a need to give a voice to the individuals and groups whose data fuels the ecosystem.

Session 4: Engaging industry partners

Summary: The White House Executive Order directs the NSF to promote innovation by developing and strengthening “public-private partnerships for advancing innovation, commercialization, and risk-mitigation methods for AI” and by helping promote “safe, responsible, fair, privacy-protecting and trustworthy AI systems.” Thus, engaging industry partners is both a priority and a necessity for the NAIRR.

Session 5: Transparency as an enabler of trustworthy AI

Summary: Transparency is concerned with providing the information that stakeholders need to make informed, responsible, and ethical decisions for data and model development, sharing, and reuse. Transparency is a big-tent concept that may be interpreted differently depending on the stakeholders, the context of use, and the applicable legal and ethical considerations. In the context of the NAIRR, an actionable interpretation of transparency centers on the role of data, model, and lifecycle documentation as an enabler of trustworthy AI.

Session 6: The NAIRR as a data equity infrastructure

The NAIRR aims to democratize access to datasets and data products (most prominently, models) to accelerate AI R&D while facilitating their responsible sharing and use. Data equity is concerned with the identification and, when possible, mitigation of biases in data and models. Importantly, because data and models are created by complex multi-step processes, and are often re-purposed, bias detection and mitigation requires lifecycle-wide support.

Session 7: Education and training

Summary: Education and training are essential for meeting all four measurable goals of the NAIRR. As a cyberinfrastructure, the NAIRR will offers training capabilities to ensure that its users have the skills to use the resources. However, to advance the goals of trustworthy AI, it is crucial to train researchers and students on the principles and techniques of responsible data engineering, algorithmic fairness, transparency, and privacy and data protection. This type of training should also be offered to current data scientists, particularly those who work in small and medium-sized businesses, which typically cannot afford the necessary upskilling of their staff.

Session 8: Responsible development and use of generative AI

Summary: Generative AI is seeing ever more research activity and ever broader commercial adoption. To put generative AI into safe use as part of the NAIRR, it is essential to articulate robust policies and best practices for assessing its performance, in terms of veracity, safety, fairness, reproducibility, and legal compliance. Further, it is crucial to develop technical support for the evaluation of systems of which generative AI models form part, rather than of these models in isolation, to assess their impacts in the context of design, development and use.

Questions?

If you have any questions, please contact us by emailing stoyanovich (at) nyu (dot) edu.