USGS - science for a changing world

South Florida Information Access (SOFIA)

projects > greater everglades hydrology monitoring network: data mining and modeling to separate human and natural hydrologic dynamics > work plan

Project Work Plan

Department of Interior USGS GE PES

Fiscal Year 2008 Study Work Plan

Study Title: Hydrology Monitoring Network: Data Mining and Modeling to Separate Human and Natural Hydrologic Dynamics
Study Start Date: 10/01/2004   Study End Date: 9/30/2008
Web Sites:
Location (Subregions, Counties, Park or Refuge): Total System
Funding Source: GE PES
Other Complementary Funding Source(s): none
Funding History: FY05 was the first year of funding for this project
Principal Investigator(s): Paul Conrads
Study Personnel: Paul Conrads, Ed Roehl, Ruby Daamen, Matt Petkewich, Mark Lowery
Supporting Organizations: USGS-South Carolina Water Science Center
Associated / Linked Studies: South Florida Surface Water Hydrologic Network for Support of MAP Projects (Telis, Higer, Pearlstine, Jones PIs); Water Quality Monitoring and Modeling for the A.R.M. Loxahatchee National Wildlife Refuge (Harwell, Waldon, Surratt, and Brandt PIs); Estimation of Critical Parameters in Conjunction with Monitoring of the Florida Snail Kite Population (Wiley Kitchens, PI).

Overview & Objective(s): New technologies in environmental monitoring have made it cost effective to acquire tremendous amounts of hydrologic and water-quality data. Although these data are a valuable resource for understanding environmental systems, often these data are under utilized and/or under interpreted. The monitoring network(s) supported by the Comprehensive Everglades Restoration Plan (CERP) records tremendous amounts of data each day and the data base incorporates millions of data points describing the environmental response of the system to changing conditions. To enhance the evaluation of the CERP data base, there is an immediate need to apply new methodologies to systematically analyze data sets to address critical issues such as water depths at ungaged locations, water-depths and water-quality responses to controlled flow releases, and optimization of existing hydrologic data-collection networks. There also is a need to maximize data resources by integrating disparate hydrologic and ecologic data bases.

The objectives of the study for FY08 include: (1) development of "soft sensors" for the water-level gages of the Everglades Depth Estimation Network (EDEN); (2) document the technical development of to the Snail Kite hydrology decision support system (DSS) in a USGS Open-file Report; (3) analyze the existing coastal water-quality monitoring networks to determine priority gaging stations; (4) begin exploratory work linking landscape and hydrology changes in WCA 3A; and (5) continue the development a data viewer and hydrologic process model(s) for the A.R.M. Loxahatchee National Wildlife Refuge;

Specific Relevance to Major Unanswered Questions and Information Needs Identified: (Page numbers below refer to DOI Science Plan.)

An important part of the USGS mission is to provide scientific information for the effective water-resources management of the Nation. To assess the quantity and quality of the Nation's surface-water, many agencies and universities collects hydrologic and water-quality data from rivers, lakes, and estuaries. The techniques used for this study in FY05, FY06, and FY07 have demonstrated how valuable information can be extracted from existing databases to assist local, state and Federal agencies. The application of data-mining techniques, including artificial neural network (ANN) models, to the CERP supported databases demonstrates how empirical models of complex hydrologic systems can be developed, disparate databases and models can be integrated to support multidisciplinary research, and study results can be easily disseminated to meet the needs of a broad range of end users.

An important part of the USGS mission is to provide scientific information to manage the water resources of the Nation, including the other Agencies of the Department of Interior (DOI). The objectives for this study addresses science needs to support DOI managers in fulfilling their stewardship responsibility as identified in The Science Plan in Support of Ecosystem Restoration, Preservation, and Protection in South Florida (U.S. Department of Interior, 2004). This is consistent with primary USGS activities that include providing knowledge and expertise to assist various levels of government in understanding and solving critical water-resources problems.

The study objective to develop soft sensors (virtual sensor) is part of the overall objective of the EDEN project to support the South Florida Hydrology Monitoring Network and the Monitoring and Assessment Plan (MAP). The MAP was developed as the primary tool to assess the system-wide performance of the CERP by the REstoration, COordination and VERification (RECOVER) program (p. 17, DOI Science Plan). The MAP describes and outlines the monitoring and supporting enhancement of scientific information and technology needed to measure the responses of the South Florida ecosystem to CERP projects. The use of soft sensor with benefit the overall quality of the real-time data by providing a automated approach for evaluating sensor drift and providing accurate water-level estimates for periods of missing data.

The study objective of continuing the development hydrologic and water-quality process model(s) for the Arthur R. Marshall Loxahatchee NWR, including Internal Canal Structures and STAs (stormwater treatment area) meets a stated need in the Science Plan for the "synthesis and integration of data about historic hydrologic and ecological conditions on the refuge" and "research to understand the ecological effects of hydrology and water quality on refuge resources..."(p. 37 and 40, DOI Science Plan). The study objective will benefit the DOI and other Federal and State Agencies in South Florida by providing data analysis needed by water-resource managers to make decisions concerning the quantity and quality of inflows to the Refuge.

The documentation of the technical development of the Snail Kite Hydrology DSS supports the Water Conservation Area 3 Decompartmentalization and Sheetflow Enhancement Project (DECOMP) by addressing the science needed for "...additional research to understand the effects of different hydrologic regimes and ecological processes on restoring and maintaining ecosystem function..." (p. 64, DOI Science Plan) and supports ecological studies of impacts of hydrologic change on Everglade snail kite habitat. The study also supports the Combined Structural and Operational Plan project (CSOP) by addressing the needed science for "...refinement of hydrologic targets and operating protocols (p. 63, DOI Science Plan)."

Status: There were four objectives for the second year (FY07) of the Data Mining Study:

  1. Continued development of water-depth prediction models for ungaged locations in the Everglades Depth Estimation Network (EDEN),
  2. Develop a data viewer and hydrologic process model(s) for the A.R.M. Loxahatchee National Wildlife Refuge,
  3. Provide technical support and enhancements to the Snail Kite hydrology decision support system (DSS),
  4. Analyze the existing coastal water-level monitoring networks to determine priority gaging stations.

The first objective has been completed. Prediction models were developed for 25 recently added marsh gages in EDEN. The models were used to hindcast the water-level records at these sites to be concurrent with the EDEN database from January 1, 2000 to the present. The hindcasted water levels were used to augment the available data for surfacing the water levels of the Everglades.

The second objective has been partially met. Flow and water-level data for WCA-1A were analyzed. A spreadsheet application developed that allows user to analyze the dynamic interaction between flow, water level, and rainfall signals.

The third objective has been partially met. The USGS-SCWSC continues to provide ecologist at the Florida Cooperative Unit with technical support of the Snail Kite Hydrology decision support system (DSS). Substantial enhancements to the DSS were under taken in FY07 to better meet the needs of the plant ecologists. At each continuous monitoring location, vegetation samples are collected twice a year. For each sampling site (over 6,000 sites), ecologists need to know the water-depth hydrograph. The DSS enhancements include the generation of these hydrographs in addition to reading and writing to external databases. Other enhancements include additional statistics (hydro-ecological indices), updating the application with retrained ANN models, generation of elevation transects, and writing of user's manual for the DSS. The majority of the enhancements have been completed. The ANN models are currently (September 2007) being retrained. The enhancements should be completed in the fall of 2007.

The fourth objective has been completed for the EDEN coastal water-level gages. The approach will be expanded in the FY08 to include all the USGS coastal water-level and water-quality gages maintained by the Ft. Lauderdale and Ft. Myers Offices.

Recent Products: Products from Year Three of the study included:

  1. Development of prediction models and computation of record extensions (hindcasts) at 25 gaging stations. Hindcasts will be available on the Sofia website upon approval of Open-File Report.
  2. USGS Open-file Report, Hydrologic Record Extension of Water-Level Data in the Everglades Depth Estimation Network (EDEN) Using Artificial Neural Networks Models, 2000-2006, in review.

Planned Products (FY08): Anticipated major products include (1) development of soft sensors for EDEN and prototype data automation system for evaluating real-time data; (2) Snail Kite Hydrology Decision Support System application; (3) documentation of the technical development of the Snail Kite DSS; (4) summary of coastal water-level collection data network to identify priority gages; and, (5) development of a data viewer and hydrologic process models for the A.R.M. Loxahatchee NWR.


Title of Task 1: Hydrology Monitoring Network: Data Mining and Modeling to Separate Human and Natural Hydrologic Dynamics
Task Funding: USGS Priority Ecosystems Science
Task Leaders: Paul Conrads
Phone: (803) 750-6140
FAX: (803) 750-6181
Task Status: Active (fourth year)
Task priority: high
Time Frame for Task 1: 2006-2008
Task Personnel: Paul Conrads, Matt Petkewich, Ed Roehl , and Ruby Daamen

Task Summary and Objectives: The Everglades Depth Estimation Network (EDEN) was established to support the South Florida Hydrology Monitoring Network module of the Comprehensive Everglades Restoration Plan (CERP) and the Monitoring and Assessment Plan (MAP) and Restoration Coordination and Verification Team (RECOVER). The goals of EDEN are to help guide large-scale field operations, integrate hydrologic and biologic responses, and to support the MAP assessments by scientists and principal investigators across disciplines. One objective of EDEN is to relate water-level data at real-time stage gages to ungaged areas using ground elevation data, so that water depths throughout the greater Everglades can be estimated (Telis, 2005: ).

The generation of EDEN water-level surfaces is dependent on high quality real-time data. Data is automatically checked for outliers by minimum and maximum thresholds for each station. More difficult to detect are smaller errors in the data such as gradual drift from a malfunctioning pressure transducers. Drift in a water-level sensor is difficult to immediately identify with visual inspection of time-series plots and may only be identified during an inspection of the gage. Correcting smaller errors in the data often is very time consuming and water-level data may not be finalized for over 12-months. Many USGS Water-Science Centers are transitioning to a system of continuously finalizing data with the goal of finalizing records within 3- to 4-months. To accomplish the transition, automated processes need to be developed to improve the efficiency of finalizing records.

A technology used for industrial application is the "soft sensor." Rather than installing a redundant sensor to measure a process, such as an additional water-level gage, a soft sensor (virtual sensor) is developed that makes very accurate estimates of the process measured by the hard sensor. The soft sensor typically is empirical or mechanistic model. The advantage of a soft sensor is that it provides a redundant signal to the sensor in the field but without the environmental threats (floods or hurricanes, for example). In the event that a gage does malfunction, the soft sensor provides an accurate estimate for the period of missing data. The soft sensor also can be used in the quality assurance and quality control of the data. The virtual signal can be compared to the real-time data and if the difference between the two signals exceeds a certain tolerance, corrective action can be taken. The process can be automated so that the real-time data is continuously being compared to the soft sensor signal and digital reports of the status of the real-time data can be sent periodically to the necessary personnel.

In addition to developing the soft sensors for the EDEN network, the enhancements to the Snail Kite Hydrology Decision Support System will be completed in the fall of 2007. The draft manuscript describing the technical development the DSS will be completed by June of 2008.

In FY2008 we will begin exploratory work linking landscape and hydrology changes in WCA 3A. The vegetation and landscape of the Everglades is an expression to the hydrologic history of area. The objective of this study is to develop methodologies and approaches for linking hydrologic dynamics with changes in vegetation and landscape. Much of the study will be exploring various ways of integrating and analyzing the available data. Clustering is a way of grouping data by similar response. The challenge to this study is that the areas of interest (hydrology, vegetation, and landscape) changes on various time scales. The data representing these processes also are collected on various time scales - from hourly to annual sampling intervals.

The network analysis of the EDEN coastal water-level gages will be expanded in FY08 to the all coastal water-level monitoring networks maintained by the USGS. After completing the water-level analysis, the water-quality gaging network will be evaluated.

Work to be undertaken during the proposal year and a description of the methods and procedures:

The initial approach for developing soft sensors for the EDEN network will be to develop a prototype for one site in the network. An ANN model of the water level for the site will be developed. A automated process will be developed that will pull the necessary input data from the real-time data base, run the ANN model, statistically compare the soft sensor and real-time data, and generate status reports. After completion of the automated process, all the EDEN marsh sites (approximately 150 sites) will be modeled and incorporated into the automated process.

The enhancements to the Snail Kite Hydrology DSS will be completed and tested. The development of the DSS application will be documented in a USGS Open-File Report.

To analyze the linkage between changes in hydrology and vegetation; historical hydrologic and vegetation datasets will obtained. Understanding the historic changes in the patterning of the ridge and slough and tree island topology necessitates the understanding of long-term change in hydrology. Currently there are various monitoring networks throughout the Everglades. As one moves back in time to the 1950s, the temporal and spatial extent of water-level gages diminishes greatly. An approach for enhancing the information content of historical databases of monitoring networks is to estimate (or "hindcast") historical time series using a combination of short-term, for example, 1 to 2 years, and long-term, for example, greater than 5 years, time-series datasets. By developing accurate water-level estimates, contemporary and historical databases can be integrated and used to enhance scientific investigations that seek to link hydrologic and ecological change.

Specific Task Product(s):

  1. ANN model completed for test site (October 2007)
  2. Develop automated process for retrieving data, executing ANN model, evaluating real-time and soft-sensor data, and reporting of (February 2008)
  3. Develop soft sensor ANN models for remaining EDEN marsh sites (June 2008)
  4. Incorporate additional sites into the automated process (September 2008)
  5. Complete enhancements to the Snail Kite Hydrology DSS (December 2007)
  6. Complete draft of report on the Snail Kite Hydrology DSS (June 2008)
  7. Methodology for linking landscape and hydrology change (March 2008)
  8. Summary of analysis of coastal water-level and water-quality gage network (January 2008)

Title of Task 2: A Synthesis of Hydrology and Water-Quality Data of A.R.M. Loxahatchee NWR
Task Funding:
USGS Priority Ecosystems Science
Task Leaders: Paul Conrads
Phone: (803) 750-6140
FAX: (803) 750-6181
Task Status: Active (second year)
Task priority: High
Time Frame for Task 2: 2006-2008
Task Personnel: Paul Conrads, Ed Roehl, Matt Petkewich, and Ruby Daamen

Task Summary and Objectives: The Arthur R. Marshal Loxahatchee National Wildlife Refuge is the last of the soft-water ecological systems in the Everglades. Historically, the ecosystem was driven by precipitation inputs to the system that were low in conductance and nutrients. With controlled releases into the canal that surround the Refuge, the transport of water with higher conductance and nutrient concentration could potentially alter critical ecosystem functions. With potential alteration of flow patterns to accommodate the restoration of the Everglades, the Refuge could be affected not only by changes in the timing and frequency of hydroperiods but by the quality of the water that inundate the Refuge.

There is a long history of collecting hydrologic and water quality data in the Refuge. Data characterizing the hydrology of the system - inflows, outflows, precipitation and water levels have been collected since the 1950's. Data characterizing the water quality of the system, including conductance and phosphorus, has been collected since the late 1970's. To enhance the understanding of the hydrology and water quality of the Refuge, there is an immediate need to apply new methodologies to systematically synthesize and analyze the data set to answer critical questions such as relative impacts of controlled releases, precipitation, groundwater interaction, and meteorological forcing on water level, conductance, and phosphorous. There also is a need to integrate longer-term hydrologic data with shorter-term hydrologic data collected for biological and ecological resource studies.

Work to be undertaken during the proposal year and a description of the methods and procedures:

ANN model hydrologic and water-quality process model development will continue. In FY2007, a preliminary "model" (tool) was developed to analyze system dynamics by adjusting time delays of inflow, outflows, and precipitation time series. The tool allows an analyst to change time delays of major inputs and outputs to the system and determine how the correlation to water levels change. The tool is a critical first step in developing an accurate empirical model of the system by determining the optimal combination of inputs, outputs, time delays of the data prior to training ANN models. The following steps (also described in the FY2007 Scope of Work) will be undertaken during the continued development of the process water-level and water-quality model of the Refuge.

Development of a Temporal and Spatial Data Viewer

A three-dimensional hydrologic and water-quality data viewer will be developed to visualize the historical data over time and by location. The general approach involved overlaying a rectangular grid onto Refuge; aggregating data collection sites into the grid cells; and aggregating measurements from sites within each cell into time steps. The viewer will provide an integrated, interactive environment for exploring and analyzing the data.

Correlation Analysis and Sensitivity Estimation

Continue correlation analysis quantifies the relationships between many variables and provides deeper understanding of the data. The computer systematically correlates factors that influence parameters of interest, such as water level, conductance, and phosphorous to combinations of controlled and uncontrolled variables, such as inflows, outflows and rainfall. Correlation methods based on statistics and machine learning are applied in combination. Comparing them to known patterns of behavior validates promising results found by the computer. Correlation analysis identifies:

  1. Relative impact - For example, "What variables impact the increased conductance and phosphorous? And to what degree?"

  2. Relationships between controlled (inflows and outflows) and uncontrolled variables (meteorology forcing).

  3. Quantifiable answers to complex questions - For example, "What are the critical temporal and spatial relationships between the controlled releases and the water level, conductance, and phosphorous response in the interior of the Refuge? Which has more effect on these responses - large releases over a short period of time or weekly flow volumes? What are the relative impacts of the inflows/outflow locations on these responses?"

Predictive Modeling

Using machine learning, predictive models are developed directly from the data and correlations (Step 1 above). To maximize accuracy, the model is constructed from sub-models, which independently correlate periodic and chaotic components. Their outputs are combined to obtain an overall prediction that manifests all of the different forcing functions that are represented by input variables, which affect the output variables. The models of the Refuge will predict water level, conductance, and phosphorous at multiple locations from inputs such as inflow, outflow, rainfall, wind direction and speed. The models will provide powerful analysis tools for understanding the dynamics of the system. In particular, 3-dimensional response surfaces showing the interaction of two explanatory variables (such as canal inflow, outflow, canal water level, and rainfall) on a response variable (interior water level, conductance, and phosphorous) will be generated.

Develop the DSS with Water-Level, Conductance, and Phosphorous Optimizer and Write User's Manual

The models will be integrated in a user-friendly Excel/Visual Basic program called a decision support system (DSS). The DSS requires no typing to operate or to obtain output. It will contain a historical database (including hind-casted values) to allow for running long-term simulations. Run-time monitoring and DSS output will be in the form of supporting graphics and Excel worksheets. A constrained optimization routine will be integrated into the DSS that will determine the necessary inflows/outflows to meet specified water level, conductance, and phosphorus levels. A user's manual will describe installation, operation, and features of the Simulator.

Specific Task Product(s):

  1. Develop spatial and temporal data viewer (January 2008)
  2. Preliminary hydrologic process model (February 2008)
  3. Preliminary water-quality process model (March 2008)
  4. Develop prototype DSS (September 2008)

| Disclaimer | Privacy Statement | Accessibility |

U.S. Department of the Interior, U.S. Geological Survey
This page is:
Comments and suggestions? Contact: Heather Henkel - Webmaster
Last updated: 04 September, 2013 @ 02:09 PM(KP)