What AI says about HH-RLHF

Anthropic provides human preference data and red teaming datasets for training helpful and harmless AI assistants through reinforcement learning from human feedback. The datasets include conversations with human preference labels and adversarial interactions designed to test and improve AI safety.