A REVIEW ON INSTANCE AND FEATURE SELECTION IN BIG DATA ENVIRONMENT

Abstract

Instance and feature selection has become an effective approach due to the enormous data which is continuously being produced in the field of research. It is difficult to process such large datasets by many systems. Though the traditional techniques are useful for large datasets, the numbers when in hundreds, thousands or millions face scaling problems. The proposed work focuses on, scalable instance and feature selection in big data environment. Locality-sensitive hashing instance selection F (LSH-IS-F) is a two pass method used to find similar instances along with Pearson correlation coefficient for feature selection. Hash function family is used which is a general method of reducing the size of a set; this is achieved by reindexing the elements into buckets. This process find similar instance and features in same bucket, hence instance/features can be reduced. The work aims at improving the performance of locality sensitive hashing by storing extra statistics of the instances and features that is assigned to each class in the bucket and also to improve accuracy of instance and feature selection algorithm by prototype generation.