We will share the code after the paper's acceptance. UnsafeBench dataset is released on HuggingFace: Dataset