Comment by Japan AI Safety Institute

For businesses involved in the development and provision of AI systems to practice AI safety evaluation based on the “Guide to Evaluation Perspectives on AI Safety” and the “Guide to Red Teaming Methods for AI Safety”, it is necessary to incorporate the contents described in the guide into specific evaluation items and datasets before conducting evaluation tests on AI systems. The need for specialized knowledge and a process of trial and error to achieve this was an issue for promoting the spread of AI safety evaluation activities. For this reason, we developed an AI safety assessment environment as a reference implementation of an evaluation tool and dataset that embodies the guide-based AI safety assessment, and released it to the public as open source software (OSS), including the source code and dataset of the tool. In addition to being available as a free evaluation tool and dataset, the AI safety assessment environment is also intended to be used to support various businesses in developing customized evaluation tools and datasets according to their own environments and conditions by referring to source code and datasets.