Content-based image retrieval methods based on the Euclidean metric expect the feature space to be isotropic. They suffer from unequal differential relevance of features in computing the similarity between images in the input feature space. We propose a learning method that attempts to overcome this limitation by capturing local differential relevance of features based on user feedback. This feedback, in the form of accept or reject examples generated in response to a query image, is used to locally estimate the strength of features along each dimension while taking into consideration the correlation between features. This results in local neighborhoods that are constricted along feature dimensions that are most relevant, while enlongated along less relevant ones. In addition to exploring and exploiting local principal information, the system seeks a global space for efficient independent feature analysis by combining such local information. We provide experimental results that demonstrate the efficacy of our technique using real-world data.