Ex-OpenAI workers ask California and Delaware AGs to block for-profit conversion of ChatGPT maker

By MATT O BRIEN Associated Press Equipment Writer Former employees of OpenAI are asking the top law enforcement officers in California and Delaware to stop the company from shifting control of its artificial intelligence instrument from a nonprofit charity to a for-profit business They re concerned about what happens if the ChatGPT maker fulfills its ambition to build AI that outperforms humans but is no longer accountable to its populace mission to safeguard that mechanism from causing grievous harms Ultimately I m worried about who owns and controls this instrument once it s created explained Page Hedley a former plan and ethics adviser at OpenAI in an interview with The Associated Press Backed by three Nobel Prize winners and other advocates and experts Hedley and nine other ex-OpenAI workers sent a letter this week to the two state attorneys general The coalition is asking California Attorney General Rob Bonta and Delaware Attorney General Kathy Jennings both Democrats to use their authority to protect OpenAI s charitable purpose and block its planned restructuring OpenAI is incorporated in Delaware and operates out of San Francisco OpenAI stated in response that any changes to our existing structure would be in function of ensuring the broader general can benefit from AI It explained its for-profit will be a society benefit corporation similar to other AI labs like Anthropic and tech billionaire Elon Musk s xAI except that OpenAI will still preserve a nonprofit arm This structure will continue to ensure that as the for-profit succeeds and grows so too does the nonprofit enabling us to achieve the mission the company stated in a declaration The letter is the second petition to state bureaucrats this month The last came from a group of labor leaders and nonprofits focused on protecting OpenAI s billions of dollars of charitable assets Jennings announced last fall she would review any such transaction to ensure that the population s interests are adequately protected Bonta s office sought more information from OpenAI late last year but has noted it can t comment even to confirm or deny if it is scrutinizing OpenAI s co-founders including current CEO Sam Altman and Musk originally started it as a nonprofit research laboratory on a mission to safely build what s known as artificial general intelligence or AGI for humanity s benefit Nearly a decade later OpenAI has released its domain value as billion and counts million weekly users of ChatGPT its flagship product OpenAI already has a for-profit subsidiary but faces a number of challenges in converting its core governance structure One is a lawsuit from Musk who accuses the company and Altman of betraying the founding principles that led the Tesla CEO to invest in the charity While certain of the signatories of this week s letter advocacy Musk s lawsuit Hedley noted others are understandably cynical because Musk also runs his own rival AI company The signatories include two Nobel-winning economists Oliver Hart and Joseph Stiglitz as well as AI pioneers and computer scientists Geoffrey Hinton who won last year s Nobel Prize in physics and Stuart Russell I like OpenAI s mission to ensure that artificial general intelligence benefits all of humanity and I would like them to execute that mission instead of enriching their investors Hinton explained in a declaration Wednesday I m happy there is an effort to hold OpenAI to its mission that does not involve Elon Musk Conflicts over OpenAI s purpose have long simmered at the San Francisco institute contributing to Musk quitting in Altman s short-lived ouster in and other high-profile departures Hedley a lawyer by training worked for OpenAI in and a time when the nonprofit was still navigating the best means to steward the tool it needed to build As in recent months as Altman stated advanced AI held promise but also warned of extraordinary risks from drastic accidents to societal disruptions In latest years however Hedley revealed he watched with concern as OpenAI buoyed by the success of ChatGPT was increasingly cutting corners on safety testing and rushing out new products to get ahead of business competitors The costs of those decisions will continue to go up as the hardware becomes more powerful he disclosed I think that in the new structure that OpenAI wants the incentives to rush to make those decisions will go up and there will no longer be anybody really who can tell them not to tell them this is not OK Application engineer Anish Tondwalkar a former member of OpenAI s technical band until last year mentioned an pivotal assurance in OpenAI s nonprofit charter is a stop-and-assist clause that directs OpenAI to stand down and help if another organization is nearing the achievement of better-than-human AI If OpenAI is allowed to become a for-profit these safeguards and OpenAI s duty to the community can vanish overnight Tondwalkar disclosed in a announcement Wednesday Another former worker who signed the letter puts it more bluntly OpenAI may one day build mechanism that could get us all killed explained Nisan Stiennon an AI engineer who worked at OpenAI from to It is to OpenAI s credit that it s controlled by a nonprofit with a duty to humanity This duty precludes giving up that control The Associated Press and OpenAI have a licensing and device agreement that allows OpenAI access to part of AP s text archives