We don't use -Wcast-align ourselves, but the headers trigger warnings
for those users who do, which is not nice.