Solution: if not building with tweetnacl, do not include its sources
in Makefile.am. CMake already includes them optionally.