Analyzing and studying Dimensionality Reduction Techniques for High-Dimensional Data

Abstract

In the fast-moving world, data is accumulating at an unprecedented speed from vivid sectors across the sphere such as micro-array gene expression data, medical data, ECG and MEG data research, satellite images, IoT devices, etc. is considered as high dimensional data. This data has a lot of features and thus directly affects the output of machine learning algorithms at an exponential rate. Thus, dimensionality reduction (DR) helps to solve the problem of the curse of dimensionality by extracting the relevant features without forfeiting the useful data. The purpose of this research is to compare and analyze different dimensionality reduction techniques namely Principal Component Analysis (PCA), Independent Component Analysis (ICA), Singular Value Decomposition (SVD), Truncated-SVD and Non-negative Matrix Factorization (NMF) on Imagenet dataset (unsupervised dataset) for five different values of components - 40, 45, 50, 55 and 60 each. These algorithms are examined on the basis of execution time, accuracy of dimensionality reduction techniques and load analysis, that is, Mean Squared Error (MSE). The algorithm with the least execution time and number of components giving the most information is concluded as a suitable algorithm for dimensionally reducing high-dimensional data.