CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories

Fernando Martínez-Plumed; Lidia Contreras-Ochando; Cesar Ferri; Jose Hernandez-Orallo; Meelis Kull; Nicolas J A H Lachiche; María José Ramírez-Quintana; Peter A Flach

doi:10.1109/TKDE.2019.2962680

CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories

Fernando Martínez-Plumed, Lidia Contreras-Ochando, Cesar Ferri, Jose Hernandez-Orallo, Meelis Kull, Nicolas J A H Lachiche, María José Ramírez-Quintana, Peter A Flach

Research output: Contribution to journal › Article (Academic Journal) › peer-review

196 Citations (Scopus)

5016 Downloads (Pure)

Abstract

CRISP-DM (CRoss-Industry Standard Process for Data Mining) has its origins in the second half of the nineties and is thus about two decades old. According to many surveys and user polls it is still the de facto standard for developing data mining and knowledge discovery projects. However, undoubtedly the field has moved on considerably in twenty years, with data science now the leading term being favoured over data mining. In this paper we investigate whether, and in what contexts, CRISP-DM is still fit for purpose for data science projects. We argue that if the project is goal-directed and process-driven the process model view still largely holds. On the other hand, when data science projects become more exploratory the paths that the project can take become more varied, and a more flexible model is called for. We suggest what the outlines of such a trajectory-based model might look like and how it can be used to categorise data science projects (goal-directed, exploratory or data management). We examine seven real-life exemplars where exploratory activities play an important role and compare them against 51 use cases extracted from the NIST Big
Data Public Working Group. We anticipate this categorisation can help project planning in terms of time and cost characteristics.

Original language	English
Number of pages	14
Journal	IEEE Transactions on Knowledge and Data Engineering
Early online date	27 Dec 2019
DOIs	https://doi.org/10.1109/TKDE.2019.2962680
Publication status	E-pub ahead of print - 27 Dec 2019

Keywords

Data Science Trajectories
Data Mining
Knowledge Discovery Process
Data-driven Methodologies

Access to Document

10.1109/TKDE.2019.2962680

Full-text PDF (author’s accepted manuscript)
This is the author accepted manuscript (AAM). The final published version (version of record) is available online via IEEE at https://ieeexplore.ieee.org/document/8943998. Please refer to any applicable terms of use of the publisher.
Accepted author manuscript, 913 KB

Handle.net

Persistent link

Cite this

@article{0678e9c074c24448b9246071d5d80b4f,

title = "CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories",

abstract = "CRISP-DM (CRoss-Industry Standard Process for Data Mining) has its origins in the second half of the nineties and is thus about two decades old. According to many surveys and user polls it is still the de facto standard for developing data mining and knowledge discovery projects. However, undoubtedly the field has moved on considerably in twenty years, with data science now the leading term being favoured over data mining. In this paper we investigate whether, and in what contexts, CRISP-DM is still fit for purpose for data science projects. We argue that if the project is goal-directed and process-driven the process model view still largely holds. On the other hand, when data science projects become more exploratory the paths that the project can take become more varied, and a more flexible model is called for. We suggest what the outlines of such a trajectory-based model might look like and how it can be used to categorise data science projects (goal-directed, exploratory or data management). We examine seven real-life exemplars where exploratory activities play an important role and compare them against 51 use cases extracted from the NIST BigData Public Working Group. We anticipate this categorisation can help project planning in terms of time and cost characteristics.",

keywords = "Data Science Trajectories, Data Mining, Knowledge Discovery Process, Data-driven Methodologies",

author = "Fernando Mart{\'i}nez-Plumed and Lidia Contreras-Ochando and Cesar Ferri and Jose Hernandez-Orallo and Meelis Kull and Lachiche, {Nicolas J A H} and Ram{\'i}rez-Quintana, {Mar{\'i}a Jos{\'e}} and Flach, {Peter A}",

year = "2019",

month = dec,

day = "27",

doi = "10.1109/TKDE.2019.2962680",

language = "English",

journal = "IEEE Transactions on Knowledge and Data Engineering",

issn = "1041-4347",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

}

TY - JOUR

T1 - CRISP-DM Twenty Years Later

T2 - From Data Mining Processes to Data Science Trajectories

AU - Martínez-Plumed, Fernando

AU - Contreras-Ochando, Lidia

AU - Ferri, Cesar

AU - Hernandez-Orallo, Jose

AU - Kull, Meelis

AU - Lachiche, Nicolas J A H

AU - Ramírez-Quintana, María José

AU - Flach, Peter A

PY - 2019/12/27

Y1 - 2019/12/27

N2 - CRISP-DM (CRoss-Industry Standard Process for Data Mining) has its origins in the second half of the nineties and is thus about two decades old. According to many surveys and user polls it is still the de facto standard for developing data mining and knowledge discovery projects. However, undoubtedly the field has moved on considerably in twenty years, with data science now the leading term being favoured over data mining. In this paper we investigate whether, and in what contexts, CRISP-DM is still fit for purpose for data science projects. We argue that if the project is goal-directed and process-driven the process model view still largely holds. On the other hand, when data science projects become more exploratory the paths that the project can take become more varied, and a more flexible model is called for. We suggest what the outlines of such a trajectory-based model might look like and how it can be used to categorise data science projects (goal-directed, exploratory or data management). We examine seven real-life exemplars where exploratory activities play an important role and compare them against 51 use cases extracted from the NIST BigData Public Working Group. We anticipate this categorisation can help project planning in terms of time and cost characteristics.

AB - CRISP-DM (CRoss-Industry Standard Process for Data Mining) has its origins in the second half of the nineties and is thus about two decades old. According to many surveys and user polls it is still the de facto standard for developing data mining and knowledge discovery projects. However, undoubtedly the field has moved on considerably in twenty years, with data science now the leading term being favoured over data mining. In this paper we investigate whether, and in what contexts, CRISP-DM is still fit for purpose for data science projects. We argue that if the project is goal-directed and process-driven the process model view still largely holds. On the other hand, when data science projects become more exploratory the paths that the project can take become more varied, and a more flexible model is called for. We suggest what the outlines of such a trajectory-based model might look like and how it can be used to categorise data science projects (goal-directed, exploratory or data management). We examine seven real-life exemplars where exploratory activities play an important role and compare them against 51 use cases extracted from the NIST BigData Public Working Group. We anticipate this categorisation can help project planning in terms of time and cost characteristics.

KW - Data Science Trajectories

KW - Data Mining

KW - Knowledge Discovery Process

KW - Data-driven Methodologies

U2 - 10.1109/TKDE.2019.2962680

DO - 10.1109/TKDE.2019.2962680

M3 - Article (Academic Journal)

SN - 1041-4347

JO - IEEE Transactions on Knowledge and Data Engineering

JF - IEEE Transactions on Knowledge and Data Engineering

ER -

CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories

Abstract

Keywords

Access to Document

Handle.net

Fingerprint

Cite this