Revert "Work around weird behavior in GCC 4.8 where it thinks this destructor's declaration and definition hase mismatching noexcept specs even though neither one has a noexcept spec at all."

This reverts commit c088dd95.

Reverted in favor of dwrensha's more complete fix.