Incredibly powerful AI systems are suddenly here, and our level of concern over them hasn’t caught up. There are government projects racing to embed AI systems into our healthcare systems and public education. THE CHALLENGE OF ALIGNMENTThis general term for the goal of building safe AI systems is called the problem of alignment. WHY ALIGNMENT IS EVEN HARDER THAN AI COMPANIES THINKThe technical challenges of alignment aren’t the only potentially fatal flaw in the broader project of building safe AI. Even if we could teach AI systems human values, this leaves only two general options, and both are bad.