Image registration is a viable task in the field of computer vision with many applications. Researchers propose various local modules insensitive to illumination changes across cross-spectral image pairs to handle the registration challenges under different spectrum conditions. In this paper, we develop an optimized feature-based approach to register natural cross-spectral image pairs. It works on the phase information to quickly identify and describe reliable keypoints that are insensitive to illumination. It then employs a sequence of outlier removal processes to accurately find the matching feature points and the direct linear transformation to estimate the geometric transformation to align the image pair. We benchmark the proposed method and six state-of-the-art feature-based methods on the dataset provided by Ecole Polytechnique Fédérale De Lausanne (EPFL), which includes 477 pairs of RGB-NIR images. The comprehensive analysis demonstrates that the proposed method achieves up to 13.90% accuracy improvement over the second best registration method.