In MacOSx x64 it was getting a memory leak even without initializing points:
http://code.opencv.org/issues/4250

The matrix templateZM needs to be initialized because otherwise
uninitialized values leak into the correlation in:

    const double correlation = templateZM.dot(imageWarped)

In the worst case this will lead the correlation to be NaN ruining the
whole routine. The subtraction does not initialize templateZM due to the
mask.

Unfortunately, the uninitialized values (by altering the correlation)
have the side effect of dragging out the computation a little longer
giving a slightly better error bound. This means that fixing this bug
breaks perf_ecc where

    SANITY_CHECK(warpMat, 1e-3);

is just a little too tight and happens to work due to the uninitialized
values. Since this is a performance not a accuracy test I think it is OK
to just relax the error bound a little bit (the tight error bound being
after all the result of a bug).

Small error - object fs does not exist in this code example

temporarily use EPNP in SolvePnP instead of UPNP or DLS algorithms, since the latter two are not quite stable