LawFlash

Governor Newsom Issues Executive Order on Use, Procurement, and Development of Generative AI

September 13, 2023

California Governor Gavin Newsom signed a new executive order (EO) on September 6 regarding the use, procurement, and development of generative artificial intelligence (GenAI) by California state agencies and employees. The EO announces the initiation of various studies, reports, and guidelines in the coming year, which will address opportunities and potential risks related to the use of GenAI.

While the EO is currently only applicable to the state government’s use of GenAI, California companies should expect the guidelines and studies produced as a result of the EO to be used as a basis for future rules and regulations impacting the use of AI in the private sector.

KEY TAKEAWAYS

  • Within 60 days of the publishing of the EO, the California Government Operations Agency, Department of Technology, Office of Data and Innovation, and Governor’s Office of Business and Economic Development will prepare a report examining (1) the most important and potentially beneficial uses of GenAI by the state and (2) the potential risks of such uses, with a focus on “high-risk use cases.”
  • Within 60 days of the publishing of the EO, all state agencies and departments must submit an inventory of their current high-risk uses of GenAI. The EO does not define “high-risk” use cases but highlights those areas “where GenAI is used to make a consequential decision affecting access to essential goods and services.” The EO also references risks stemming from bad actors and insufficiently guarded governmental systems, unintended or emergent effects, and potential risks toward democratic and legal processes, public health and safety, and the economy.
  • General guidelines for public sector employee training, procurement, and use of GenAI tools, modeled after the White House’s AI Bill of Rights and the National Institute of Standards and Technology’s AI Risk Management Framework, will be released by January 2024. Trainings for state government workers should be made available by July 2024.
  • A joint risk analysis report, to be completed by March 2024, will study the impact of GenAI tools on California’s critical energy infrastructure, including where it could lead to mass casualty events and environmental emergencies. Responsible agencies will provide public recommendations for regular testing and monitoring based on this report.
  • Guidelines to analyze the impact of state agencies’ use of GenAI tools on vulnerable communities, including whether they achieve equitable outcomes in their deployment and implementation, will be created by July 2024. These guidelines will inform whether state agencies should use a particular tool.
    • These guidelines will be used to update the state’s contract terms for the project approval and procurement of new GenAI tools by January 2025.
  • California will create infrastructure to allow state agencies to carry out pilot projects and allow for “sandbox” testing of new GenAI applications.
  • California will engage with the legislature and stakeholders, including civil society, academia, industry experts, as well as historically vulnerable and marginalized communities in the development of these guidelines, reports, trainings, and pilot projects.
  • California will pursue a partnership with UC Berkeley and Stanford to evaluate the impacts of GenAI on California and recommend efforts for the state to advance its leadership in the area, with the aim of hosting a joint summit in 2024 to discuss findings.

PRACTICAL CONSIDERATIONS

Contacts

If you have any questions or would like more information on the issues discussed in this LawFlash, please contact any of the following:

Authors
Alicia J. Farquhar (Silicon Valley)
Zachary W. Shine (San Francisco)