Roblox rolls out open-source AI system to discover 'problematic behavior' in chat

Roblox’s Sentinel AI scans 6 billion daily chat messages to identify predatory language, flagging 1,200 potential child exploitation cases in 2025’s first half, reported to the National Center for Missing and Exploited Children. The system, now open-sourced, enables other platforms to adopt it, strengthening industry-wide child safety efforts.

A lawsuit filed in Iowa’s Polk County District Court reveals Roblox’s prior vulnerabilities, alleging a 13-year-old girl was groomed, kidnapped, and trafficked after meeting a predator on the platform. The suit claims Roblox’s design leaves children “easy prey for pedophiles.”

Sentinel analyzes one-minute chat snapshots, using two indexes: benign and harmful conversations.