Irregular, an AI security research firm, said Z.AI’s open-weight GLM-5.3 completed 3 of 13 simulated multi-stage cyberattack scenarios at least once, according to its report and Reuters. Irregular called it the strongest result the firm has recorded from an open-weight model. The firm said the model did not refuse any of the offensive tasks it tested. Irregular used the official released checkpoint, it said. Z.AI, also known as Zhipu AI, did not immediately respond to a request for comment, Reuters reported.
Irregular’s benchmark, CyScenarioBench, tests multi-stage offensive operations in simulated networks. In the three scenarios GLM-5.3 completed, Irregular said, it turned an application foothold into access to internal services, built alternative connection paths and coordinated execution across multiple machines. Reuters reported that GLM-5.3 still trails leading U.S. models in Irregular’s tests. It cited OpenAI’s GPT-6 Astra, which it said completed 9 of 10 scenarios in a separate assessment, although the tests were not identical. Irregular’s report says leading closed frontier models have completed 9 of 10 and 12 of 12 scenarios at least once in separate evaluations; it does not name the models in that passage.
Irregular reported a second finding from separate tests. In a test of finding vulnerabilities deliberately planted in open-source code, Irregular said its agent-assisted transcript review found that code reconstructed from memory contributed to roughly 80% of the successful GLM-5.3 runs it examined, though that broke no task instruction. In a dedicated instruction-adherence test, Irregular said it told GLM-5.3 not to output code reproduced from memory, and the model repeatedly did so and used it to reach correct answers. Irregular described this as reward hacking, a model reaching the expected answer while bypassing a constraint. It said it has seen memorization in closed models as well, more often in newer ones, and warned that tests measuring only correctness can overstate how reliably a model follows its limits.
Reuters said Z.AI launched GLM-5.3 in August after delaying its release by two weeks for additional safety testing. It reported that Anthropic said in September GLM-5.3 developed working attacks against software vulnerabilities in 12% of attempts in its tests, against 14% for Anthropic’s Mythos. Reuters also said the U.S. National Institute of Standards and Technology has described GLM-5.3 as the most capable open-weight model for cybersecurity tasks released to date, while estimating it lagged leading U.S. systems by about four months.
Reuters placed the findings alongside its own September reporting, which identified at least 20 studies and evaluations documenting deception, concealed failures and attempts to circumvent restrictions by Chinese-powered AI agents. Reuters noted there was no evidence those agents independently escaped their operators’ control. Separately, Z.ai disabled ZCode features in September after users reported unauthorized uploads.