FYSA: MLCommons (who I work with/for) just produced the first collaboratively- and openly-developed tool for evaluating the safety of interactive AI systems. More than just policies, guidelines, or good intentions, our benchmark is running code. It's a cloud service. It's Open Source. It evaluates models across 12 hazard categories, including violence and hate, and does so in a standard clearly documented approach that lets us compare apples to apples, regardless of underlying architecture. It's hard to game. It has a best in class ensemble model for identifying hazards. More importantly, MLCommons has pioneered the process by which organizations can develop such tools developed openly, yet securely, and incorporate the expertise of leading researchers, and the needs of civil society and policy makers. https://mlcommons.org/ailuminate/ YouTube of the launch: MLCommons is a nonprofit industry consortium of 120+ companies with Meta, Google, Microsoft, Intel, and Qualcomm as founding members. AMA, especially if your company/organization wants to try it out.