OpenAI has partnered with Poland’s NASK National Research Institute to research AI safety and security, as concerns grow over AI agents behaving in ways their developers did not intend. The company announced the agreement today, saying the two organisations would work together to test increasingly powerful AI systems, identify vulnerabilities and develop stronger safeguards. The partnership comes amid growing scrutiny of autonomous AI agents, as both OpenAI and Anthropic have disclosed cases in which their agents exhibited unexpected or unwanted behaviour. In August, OpenAI said some of its test agents had escaped a controlled testing environment and coordinated an attack on Hugging Face, a platform for sharing AI models.
The incident has since prompted a lawsuit against OpenAI in San Francisco. Under the new agreement, OpenAI will work with NASK’s AI Safety Research Center, which serves as Poland’s AI safety institute. The collaboration will focus on evaluating advanced AI systems, finding weaknesses and improving the measures designed to keep them safe. NASK is a state-owned research institute based in Warsaw.
It also operates CERT Polska, the national team responsible for responding to cyberattacks on Polish networks. “Collaboration with OpenAI is a natural step for the Institute and its AI Safety Research Center. […] By combining our research expertise with OpenAI’s technical capabilities, we can better anticipate risks, strengthen safeguards, and build public trust in these transformative technologies,” said Radosław Nielek, NASK’s director. The organisations have worked together before. In July, CERT Polska gained access to OpenAI’s cybersecurity models through the company’s Daybreak programme, according to NASK. OpenAI said the team used the models to help identify six vulnerabilities in MikroTik router software, all of which have since been fixed. “Poland is home to exceptional scientists and technical talent, many of whom have helped lead OpenAI’s own research,” said Tom Duff Gordon, OpenAI’s head of EMEA policy.
The partnership also fits into OpenAI’s broader effort to shape how advanced AI is tested and governed internationally. The company has called on the US to work with other countries on shared technical standards for frontier AI, referring to the most advanced AI systems. In September, OpenAI published a proposal for developing those standards. It includes human oversight of AI systems conducting their own research and guidance on how companies should report safety incidents.
OpenAI already works on AI safety with the UK AI Security Institute and a US organisation known as CAISSI. Get the TNW newsletter ✕ Close Get the TNW newsletter Get the most important tech news in your inbox each week.
Leave a Reply