Solution: add an option to enable it, and default to false. Enable it in
the CI so we don't lose coverage, as it works on Travis.

Fixes #3725