Saturday, February 20, 2010

reading in sigma for each data stream at each timestep

At ESA and AGU this year I presented some work I've been doing with Andrew, Dave H and Dan R.

In that work, with a lot of advice from Bill via email, I made some changes to my branch of the repository:

These changes require that an extra file FILENAME.sigma be specified which mirrors FILENAME.dat and contains the sigma values for each data point.

Run modelF with given parameters at location loc, compare output with measured data
using data types given by dataTypeIndices[0..numDataTypes-1]
Return a measure of the difference between measured and predicted data
(Higher = worse)

/*Added read in ***sigmas
* removed ***sigma from double difference(double *sigma, ***sigmas, OutputInfo *outputInfo,

21: static double ***sigmas; /* (dm) data uncertainty read in once at start of program
70: /*Added read in ***sigmas
71: * removed ***sigma from double difference(double *sigma, ***sigmas, OutputInfo *outputInfo,
100: thisSigma = sigmas[loc][i][dataNum];
103: logLike += log(thisSigma);//(dm) now the individual sigmas for each data point is added to the logLike
522: /* Read measured data (from fileName.dat) and valid fractions (from fileName.valid) into arrays (used to also read sigmas)
597: // (dm) added code to read sigmas for each time step and each data type
598: sigmas = (double ***)malloc(numLocs * sizeof(double **));
600: sigmas[loc] = make2DArray(steps[loc], numDataTypes); // make 2-d array just big enough for known # of time steps in this location
619: readDataLine(in3, oneLine, totNumDataTypes); // read sigmas file
620: // assign sigmas elements appropriately, based on which data types we're using:
622: sigmas[loc][index][i] = oneLine[dataTypeIndices[i]];

Wednesday, February 10, 2010

Zobitz modifications

Here is a quick rundown of what I have done:
- Added MODIS satellite fAPAR data as a co-constraint with NEE data at Niwot. I am currently investigating if this improves the parameter estimation and what effect it has on the results. Code mofidications included a change in the variables to estimate, as well as additional columns in .data and .valid files.
- Added a new tracker variable (meanDLight), which computes the running average of the fractional light multiplier (Dlight) over 8 days, which I then compare against measured fAPAR data.
- Implemented a "weighted likelihood" cost function, where each assimilated data stream is multiplied by a fraction from 0 to 1. I did this because I wanted to determine how much the parameter estimation improves by including (or not including) a particular data stream.
- Concurrent with this approach, I did a tweak on the code to show the likelihood for each particular data stream by running sipnet forward over input data.

--> Any of these changes worth merging to the main code?

The benevolent branch czar

This should beg the question (this may already be answered, so I apologize if I am out of the loop) - when we make changes to SIPNET code to suit a particular research question, by what process should we decide to merge those changes into the code repository? Should we all comment on the blog, and if we all say "yes - that would be helpful for us all", then merge appropriately? While I know we are all doing good things with SIPNET, running all these parallel threads might not match well when merged together in the repository (it has happened before), potentially losing the "SI" in SIPNET.

Can we come to a consensus on a "core" SIPNET model? I am not well-versed enough in programming to know what would the appropriate code-development scheme would be to all of our multiple research threads. I don't think having branches from the repository with our own code tweaks would be useful - it seems that we would have multiple copies of very similar code. Maybe a sort of "plug - in" scheme, where each thread has a few base files that are plugged into the model, and maybe supersedes conflicting code. That way it could be easier to distinguish between all our different changes.

Branch projects

To my knowledge the branch projects are

1) Multiple data streams with explicit consideration of data error (Dave plus Howland group)
2) Capability of assimilating remote sensing data (Zobitz plus Tris and Ankur)
3) Multiple formulations of the cost function (Dave plus Howland group)

We are also using the model spatially explicit option (put together by Bill) for the ACME project over the Rocky Mountains - there is no particular change to the code for this project - it simply requires an adjusted file arrangement

Are there any other branches that could be added to this?
Are there any new features to the code that you think should be added to the core code?


Quick summary of published work From Aug 2009

I made this simplistic but hopefully helpful slide for ESA in August 2009. It summarizes where we are in terms of what has been published with SIPNET at Niwot Ridge. I did this to make the point that there are some applications for SIPNET that cannot be realized without moving beyond simply assimilating eddy covariance data.

I'm currently writing a paper drawn from the analyses I presented at ESA 2009 and AGU 2009. It's based on work done at Howland Forest and involves assimilating day/night tower based NEE fluxes (filter >0.75), Soil efflux (weekly), Plant Woody Biomass (annual), Litterfall (annual) and LAI – (occasionally). More updates on this in the coming weeks.

MSc projects using SIPNET at KCL

Some of my MSc students have been trying to applying the same principles to other sites with varying degrees of success

2007/08 We had a go at Duke Forest. Files and such are in the correct format but there was little exploration of the parameterization at Duke.

2008/09 David Yong (MSc. KCL EMMM) explored Tharandt Forest, Germany, Loobos Forest, Netherlands, Norunda Forest, Sweden.
David's thesis ended up focusing on Tharandt Forest and he made a good start at getting the parameterizations working. Dave had a go at doing some climate sensitivity at the site. We could make some more progress on Tharandt if someone had time to take this forward.

Figure above shows observed half-daily NEE from 1998 to 2003 compared to the output of SIPNET optimized on CO2 fluxes from the Tharandt.
In a related project Laura Green (MSc. KCL EMMM) looked at changes in Albedo in some of the same forests.



SIPNET Data and Parameter Requirements

SIPNET requires several driver files to calculate NEE and other ecosystem fluxes. Each of the files are distinguished by a different file extension but must share the same base name. For example to run the model at Niwot Ridge we require the following files: niwots.clim and niwots.param - the niwots.clim file contains the climate record for the period of time we want to run the model for, niwots.param contains the values for the more than 54 parameters.

An explanation of the files can be found here:

SIPNET Requirements

The structure of SIPNET

SIPNET was designed to estimate fluxes that could also be measured using an Eddy Flux
tower. The major pools of Carbon and Water in the SIPNET model are shown below in
boxes; the arrows represent fluxes into, out of or between pools.

The SIPNET model has vegetation carbon pools in wood, leaves and roots and soil carbon pools with three different substrate pools depending on quality of decomposition in soil organic matter. Photosynthesis is dependent on PPFD, soil water and atmospheric humidity, whereas respiration is dependent on both substrate quality and temperature. Evapotranspiration is partitioned according to stomatal control over transpiration. Evaporation is driven by climate and soil transport properties. The structure of the SIPNET model is shown in the figure below:

The SIPNET project

The SIPNET project
Collaborators: Russell K. Monson, Laura Scott-Denton, Diego A. Riveros-Iregui (University of Colorado), David Schimel, (National Ecological Observatory Network), Bill Sacks & Ankur Desai, (University of Wisconsin) and John Zobitz (Augburg College Minnesota)

The SIPNET model is a simple ecosystem model which has been used to interpret carbon and water exchange between vegetated ecosystems and the atmosphere. The SIPNET model has been developed from the PnET family of models (Aber & Federer 1992) to include a data-assimilation capability. Initial work by Rob Braswell and Bill Sacks has allowed this model to be implimented at Harvard Forest (Braswell et al 2005) and Niwot Ridge (Sacks et al. 2006, 2007). The SIPNET group is now expanding to other forests and ecosystems, using different data streams (Moore et al 2008, Zobitz et al 2008) to parameterize the model and also using the model at larger spatial scales (ACME project) and at longer temporal scales (Niwot Ridge climate change projections).

We do not currently have support to maintain regular code updates for the SIPNET model. However the code is held at the University of Wisconsin and maintained collectively by the collaborators. We are exploring ways to make the code more widely available.

If you have used the SIPNET model for your field site please let us know and contribute to this blog.

Aber JD, & Federer CA (1992). A generalized, lumped-parameter model of photosynthesis, evapotranspiration and net primary production in temperate and boreal forest ecosystems. Oecologia 92(4): 463-474.

Braswell BH, Sacks WJ, Linder E, Schimel DS (2005) Estimating diurnal to annual ecosystem parameters by synthesis of a carbon flux model with eddy covariance net ecosystem exchange observations. Global Change Biol. 11: 335-355.

Moore DJP, Hu J, Sacks WJ, Schimel, DS & Monson, RK (2008) Estimating transpiration and the sensitivity of carbon uptake to water availability in a subalpine forest using a simple ecosystem process model informed by measured net CO2 and H2O fluxes. Agricultural and Forest Meteorology 148, 10(3), 1467-1477

Sacks WJ, Schimel DS, Monson RK, Braswell BH (2006) Model-data synthesis of diurnal and seasonal CO2 fluxes at Niwot Ridge, Colorado. Global Change Biology 12(2): 240-259.

Sacks WJ, Schimel DS, Monson RK (2007) Coupling between carbon cycling and climate in a high-elevation, subalpine forest: a model-data fusion analysis. Oecologia 151(1): 54-68.

Zobitz JMZ, Moore DJP, Sacks WJ, Monson,RK, Bowling, DR & Schimel DS (2007) Integration of Process-based Soil Respiration Models with Whole-Ecosystem CO2 Measurements. Ecosystems DOI: 10.1007/s10021-007-9120-1