THE USE OF STATISTICAL TECHNIQUES TO JUSTIFY THE RECOVERY OF CORRUPTED POLLUTION DATA
Oct 06 2014
Author: Andrew Newman on behalf of CEM
ABSTRACT:
Data from continuous pollution monitoring can be corrupted due to many reasons, for example, by instrumentation failure, calibration faults or operator error. Although distorted, this data often exhibits an underlying relationship to the correct measurements. Operators intuitively feel that such data can be recovered by minor manipulation, and thus included in their compliance reports. However this can be hard to justify, and regulatory authorities have understandably preferred to regard data as either “good” or “bad”.
Borrowing terminology from self validating (SEVA) instruments, data may usefully beregarded as “blurred” where the underlying measurements can still be recovered. The justification of such data manipulation has been aided by three developments:
i) Much wider adoption of statistical techniques in industry (e.g. six sigma) has
improved the skills base;
ii) There is greater availability of software tools; and,
iii) An increase in logger memory capacity means more system and analyser
health diagnostics are now recorded.
As a contribution to the debate about “best practice” two detailed industrial case studies are presented to illustrate how several months of blurred data have been recovered, and how statistical techniques and diagnostic logs have been used to justify the data manipulation to the regulatory authorities. Case study one concerns calibration error. Case study two concerns instrument malfunction. To allow this paper to be used as a training aid, all techniques used are explained with examples.
Digital Edition
IET 33.5 Sept/Oct 2023
October 2023
In This Edition Water/Wastewater - Method development of a segmented flow system for the analysis of Total Cyanide in drinking water and natural waters using an on-line UV digestion and amper...
View all digital editions
Events
Meteorological Technology World Expo 2023
Oct 03 2023 Geneva, Switzerland
Oct 04 2023 Rotterdam, Netherlands
Oct 09 2023 Cape Town, South Africa
Oct 10 2023 Melbourne, Australia
Oct 10 2023 Galveston, TX, USA