
|
|
publications > poster > dealing with data realities - automation of evaluation of data quality and estimation of missing data for the everglades depth estimation network (eden)
Dealing with Data Realities - Automation of Evaluation of Data Quality and Estimation of Missing Data for the Everglades Depth Estimation Network (EDEN)Poster presented July 2009 at the 3rd National Conference on Ecosystem Restoration (NCER)Paul A. Conrads1, Matthew D. Petkewich1, Ruby Daamen2, and Edwin A. Roehl, Jr.2 1USGS South Carolina Water Science Center, Columbia, S.C. Background
Approach
EDEN Data Gap Estimation ProgramTo increase the accuracy of the daily water-surface elevation model, linear regression equations to estimate missing data for each gaging station in EDEN were developed (Conrads and Petkewich, 2009). To minimize the occurrences of no estimation of data due to missing data for an input station, a minimum of three linear regression equations were developed for each station using different input stations. For each site, an order was established for the regression equation to be used to fill a data gap. The 726 equations were incorporated into a database application that automatically estimates missing record (EDEN GAP). The performance statistics computed for each equation provides documentation of the "goodness-of-fit" of the equations (table 1). In addition, although the majority of the equations provide satisfactory estimations of water levels, the performance statistic provides a prioritization for identifying stations where improved equations are needed to provide more satisfactory water-level estimates. EDEN Data Evaluation Program
The development of the EDEN Data Evaluation Program (EDEN DEP) is the first phase of a project to address these data validation issues by developing an intelligent software application to automate the validation and correction of the data. As shown in Figure 3, the software, hereafter referred to the Inferential Sensor, will reside between the National Water Information System (NWIS) server and EDEN web applications. As part of Phase I of the development of the Inferential Sensor software, the EDEN DEP was developed to pre-process raw data to prepare it for automated analysis. A series of tunable filters were developed from the EDEN database and are used to evaluate the data for each site based on historical behaviors. Filters include thresholds based on historical values and user-specifications and time derivatives of rates of changes over specified periods Other elements of EDEN DEP are:
SummaryData-quality evaluation and estimation of missing data can be a time-consuming process, especially for a network as large as EDEN with 253 gaging stations. To increase the accuracy of the daily water-surface elevation model, two applications were developed to address data-quality issues from the network. One program, EDEN GAP, estimates water levels to fill data gaps. The other program, EDEN DEP, uses a series of tunable filters to validate the data. The two programs effectively and efficiently address data-quality issues by automating many of the processes for data estimation and data validation and will improve the consistency and utility of the EDEN data. Conrads, P.A., and Petkewich, M.D., 2009, Estimation of missing water-level data for the Everglades Depth Estimation Network (EDEN): U.S. Geological Survey Open-File Report 2009-1120, 53 p. Pearlstine, L., Higer, A., Palaseanu, M., Fujisaki, I., and Mazzotti, F., 2007, Spatially continuous interpolation of water stage and water depths using the Everglades Depth Estimation Network (EDEN): Gainesville, Fl, Institute of Food and Agricultural, University of Florida, CIR 1521, 18 p., 2 apps. Telis, Pamela A., 2006, The Everglades Depth Estimation Network (EDEN) for Support of Ecological and Biological Assessments: U.S. Geological Survey Fact Sheet 2006-3087, 4 p. |
U.S. Department of the Interior, U.S. Geological Survey
This page is: http://sofia.usgs.gov/publications/posters/ncer09_auto_eden/index.html
Comments and suggestions? Contact: Heather Henkel - Webmaster
Last updated: 16 March, 2011 @ 04:09 PM(TJE)