A framework for distributed cleaning of data streams

Saul Gill; Brian Lee

doi:10.1016/j.procs.2015.05.156

A framework for distributed cleaning of data streams

Saul Gill, Brian Lee

Research and External Services: Midlands

Limerick Institute of Technology,

Research output: Contribution to journal › Conference article › peer-review

12 Citations (Scopus)

Abstract

Vast and ever increasing quantities of data are produced by sensors in the Internet of Things (IoT). The quality of this data can be very variable due to problems with sensors, incorrect calibration etc. Data quality can be greatly enhanced by cleaning the data before it reaches its end user. This paper reports on the construction of a distributed cleaning system (DCS) to clean data streams in real-time for an environmental case-study. A combination of declarative and statistical model based cleaning methods are applied and initial results are reported.

Original language	English
Pages (from-to)	1186-1191
Number of pages	6
Journal	Procedia Computer Science
Volume	52
Issue number	1
DOIs	https://doi.org/10.1016/j.procs.2015.05.156
Publication status	Published - 2015
Event	The International Conference on Ambient Systems, Networks and Technologies, ANT-2015, the International Conference on Sustainable Energy Information Technology, SEIT-2015 - London, United Kingdom Duration: 2 Jun 2015 → 5 Jun 2015

Keywords

Data stream cleaning
Declarative cleaning
Distributed cleaning
Internet of things
Model based cleaning
Regression
Sensor data modelling

Access to Document

10.1016/j.procs.2015.05.156

Cite this

@article{71b7e84d785842bfb7d891de0be0ec6f,

title = "A framework for distributed cleaning of data streams",

abstract = "Vast and ever increasing quantities of data are produced by sensors in the Internet of Things (IoT). The quality of this data can be very variable due to problems with sensors, incorrect calibration etc. Data quality can be greatly enhanced by cleaning the data before it reaches its end user. This paper reports on the construction of a distributed cleaning system (DCS) to clean data streams in real-time for an environmental case-study. A combination of declarative and statistical model based cleaning methods are applied and initial results are reported.",

keywords = "Data stream cleaning, Declarative cleaning, Distributed cleaning, Internet of things, Model based cleaning, Regression, Sensor data modelling",

author = "Saul Gill and Brian Lee",

note = "Publisher Copyright: {\textcopyright} 2015 The Authors. Published by Elsevier B.V.; The International Conference on Ambient Systems, Networks and Technologies, ANT-2015, the International Conference on Sustainable Energy Information Technology, SEIT-2015 ; Conference date: 02-06-2015 Through 05-06-2015",

year = "2015",

doi = "10.1016/j.procs.2015.05.156",

language = "English",

volume = "52",

pages = "1186--1191",

journal = "Procedia Computer Science",

issn = "1877-0509",

publisher = "Elsevier B.V.",

number = "1",

}

TY - JOUR

T1 - A framework for distributed cleaning of data streams

AU - Gill, Saul

AU - Lee, Brian

PY - 2015

Y1 - 2015

N2 - Vast and ever increasing quantities of data are produced by sensors in the Internet of Things (IoT). The quality of this data can be very variable due to problems with sensors, incorrect calibration etc. Data quality can be greatly enhanced by cleaning the data before it reaches its end user. This paper reports on the construction of a distributed cleaning system (DCS) to clean data streams in real-time for an environmental case-study. A combination of declarative and statistical model based cleaning methods are applied and initial results are reported.

AB - Vast and ever increasing quantities of data are produced by sensors in the Internet of Things (IoT). The quality of this data can be very variable due to problems with sensors, incorrect calibration etc. Data quality can be greatly enhanced by cleaning the data before it reaches its end user. This paper reports on the construction of a distributed cleaning system (DCS) to clean data streams in real-time for an environmental case-study. A combination of declarative and statistical model based cleaning methods are applied and initial results are reported.

KW - Data stream cleaning

KW - Declarative cleaning

KW - Distributed cleaning

KW - Internet of things

KW - Model based cleaning

KW - Regression

KW - Sensor data modelling

UR - http://www.scopus.com/inward/record.url?scp=84939199005&partnerID=8YFLogxK

U2 - 10.1016/j.procs.2015.05.156

DO - 10.1016/j.procs.2015.05.156

M3 - Conference article

AN - SCOPUS:84939199005

SN - 1877-0509

VL - 52

SP - 1186

EP - 1191

JO - Procedia Computer Science

JF - Procedia Computer Science

IS - 1

T2 - The International Conference on Ambient Systems, Networks and Technologies, ANT-2015, the International Conference on Sustainable Energy Information Technology, SEIT-2015

Y2 - 2 June 2015 through 5 June 2015

ER -

A framework for distributed cleaning of data streams

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this