Automation of Descriptive Data Mining : The Use of External Data in the Evaluation Phase - Nekvapil, Viktor (2019)
Data mining has reached a mature state. There is a plethora of algorithms available, used more and more in everyday business. As far as the descriptive data mining is concerned, its assets are not clear but it often represents a necessary first step in the predictive task. Therefore, there is a distinct requirement to automate the descriptive task and thus reduce costs which are not easily justifiable considering the perspective of business people. This thesis strives to contribute to this issue.The thesis focuses on an automation of descriptive data mining task, specifically, the evaluation phase. Automation of data mining requires encoding a significant amount of domain knowledge. The thesis tries to use cheaper sources of domain knowledge than the domain experts source, that is, processed external data (External Knowledge), which can be either internal data of a company or publicly available resources, such as open data.The main aim of the thesis is to propose a new way of automation of the evaluation phase of descriptive data mining task based on external data. The thesis provides a comprehensive overview of data mining, domain knowledge and automation. The newly proposed External Knowledge Framework includes two approaches to the utilization of External Knowledge. The first approach, called Explanation System, offers the user an additional knowledge that could help him in the evaluation of the results. The second approach, referred to as SEI-formulas, further automates the evaluation phase of the descriptive data mining task and prepares automatic conclusions based on the consequences or contradictions of the defined SEI-formula. SEI-formula is a pre-defined relationship of the attribute from the resulting pattern and the attribute from External Knowledge.Both Explanation system and the SEI-formulas are implemented making use of custom-made Python engine; SQLite database is used as a storage of External Knowledge.