Detecting Anomalies with Spectral Analysis
In Data Mining Projects and working with very large datasets in particular just like when analyzing people’s reactions on several forms of advertisement in a website, it is very common that anomalies(occurrence or object that is strange, unusual, or unique) will exist and the most common method of identifying/removing anomalies is Spectral Analysis.
Spectral Analysis is commonly known in Statistics as frequency domains. Further analysis of data reveals patterns that can be described by fourier transforms wherein most of the data are broken down into smaller infinite sinusoidal wave. But like any other conditions that exist in nature, data needs to undergo cleansing by filtering algorithm. Filtering at this stage means removing those datasets that are not very much correlated with other datasets. In electronics, signal filtering is commonly known to pass the signal being analyzed into a circuit that discards certain bands of frequencies. This technique is also applied in Data Mining so that the resulting information are the most relevant to what is being studied. The seemingly senseless information has now revealed a new pattern previously hidden and can be the basis for further analysis using common data mining algorithm.
