Welcome to the Data Assimilation Research Testbed¶
The Data Assimilation Research Testbed (DART) is an open-source, freely available community facility for ensemble data assimilation (DA). 1 DART is developed and maintained by the Data Assimilation Research Section (DAReS) at the National Center for Atmospheric Research (NCAR).
Ensemble Data Assimilation¶
Ensemble DA is a technique for combining observations with numerical models to estimate the state of a physical system.
It enables modelers, observational scientists, and geophysicists to:
Generate initial conditions for forecasts.
Create a retrospective estimate of the state of a system, a practice known as producing a reanalysis.
Assess the relative value of specific observations on forecast skill, a practice known as conducting an observing system experiment (OSE).
Estimate the value of hypothetical observations in order to inform the design of an observing system, a practice known as conducting an observing system simulation experiment (OSSE).
Determine a model’s systematic bias in estimating the state of a system, a practice known as diagnosing model error.
The DART software environment makes it easy to explore a variety of data assimilation methods and observations with different numerical models. It provides powerful, flexible DA tools that are easy to use and customize to support efficient and reliable DA applications. While DART is primarily oriented for DA research, it has also been used in operational settings.
DART includes:
A comprehensive tutorial introducing the concepts of ensemble DA.
Extensive documentation of its source code.
Interfaces to a variety of models and observation sets that can be used to introduce new users or graduate students to ensemble DA.
DART is also designed to facilitate the combination of assimilation algorithms, models, and real or synthetic observations to allow increased understanding of all three. It provides a framework for developing, testing, and distributing advances in ensemble DA to a broad community of users by removing the implementation-specific peculiarities of one-off DA systems.
These tools are intended for use by the full range of geosciencies community: beginners and experts; students and teachers; national centers and university research labs.
Organization of the documentation¶
Because of DART’s extensive scope, this documentation is detailed and carefully organized, enabling you to easily find the information you need. If you have any questions or suggestions for improvements, please contact DAReS staff by emailing dart@ucar.edu.
The documentation is partitioned into three parts:
a user guide that explains how to install DART and perform data assimilation
source code documentation that provides a detailed description of the programs and modules in the repository
a comprehensive description of data assimilation theory
Manhattan Release¶
The Manhattan release is new and currently supports only a subset of the models. DAReS staff will port over any requested model. Email dart@ucar.edu if yours is not on the list.
For more information on this release, see Manhattan.
Quick-start¶
fork the NCAR/DART repo
clone your (new) fork to your machine - this will set up a remote named ‘origin’
create a remote to point back to the NCAR/DART repo … convention dictates that this remote should be called ‘upstream’
check out the appropriate branch
Download one of the tar files (listed below) of ‘large’ files so you can test your DART installation.
If you want to issue a PR, create a feature branch and push that to your fork and issue the PR.
There are several large files that are needed to run some of the tests and examples but are not included in order to keep the repository as small as possible. If you are interested in running bgrid_solo, cam-fv, or testing the NCEP/prep_bufr observation converter, you will need these files. These files are available at:
Release |
Size |
Filename |
---|---|---|
“Manhattan” |
189M |
|
“wrf-chem.r13172” |
141M |
|
“Lanai” |
158M |
|
“Kodiak” |
158M |
|
“Jamaica” |
32M |
|
“Hawaii” |
32M |
Download the appropriate tar file and untar it into your DART repository. Ignore
any warnings about tar: Ignoring unknown extended header keyword
.
Go into the build_templates
directory and copy over the closest
mkmf.template
._compiler.system_ file into mkmf.template
.
Edit it to set the NETCDF directory location if not in /usr/local
or comment
it out and set $NETCDF in your environment. This NetCDF library must have been
compiled with the same compiler that you use to compile DART and must include
the F90 interfaces.
Go into models/lorenz_63/work
and run quickbuild.csh.
$ cd models/lorenz_63/work
$ ./quickbuild.csh
If it compiles, run this series of commands to do a very basic test:
$ ./perfect_model_obs
$ ./filter
If that runs and you have Matlab installed on your system add
DART/diagnostics/matlab
to your matlab search path and run the
plot_total_err
diagnostic script while in the models/lorenz_63/work
directory. If the output plots and looks reasonable (error level stays around 2
and doesn’t grow unbounded) you have successfully installed DART and completed
your first assimilation with it.
If you are planning to run one of the larger models and want to use the Lorenz
63 model as a test, run ./quickbuild.csh -mpi
. It will build filter and any
other MPI-capable executables with MPI.
Important
The mpif90
command you use must have been built with the same version of
the compiler as you are using.
If any of these steps fail or you don’t know how to do them, go to the DART project web page listed above for very detailed instructions that should get you over any bumps in the process.
Citing DART¶
Cite DART using the following text:
The Data Assimilation Research Testbed (Version X.Y.Z) [Software]. (2019). Boulder, Colorado: UCAR/NCAR/CISL/DAReS. http://doi.org/10.5065/D6WQ0202
Update the DART version and year as appropriate.
References¶
- 1
Anderson, J. L., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn and A. Arellano, 2009 The Data Assimilation Research Testbed: A Community Facility. Bulletin of the American Meteorological Society, 90, 1283-1296, doi:10.1175/2009BAMS2618.1
- Manhattan
- Lanai
- DART Lanai release documentation
- Dart overview
- Getting started
- Installation
- Downloading the distribution
- Customizing the build scripts – overview
- Building the Lorenz_63 DART project
- Running Lorenz_63
- Experiment overview
- The tutorial
- Matlab® diagnostics
- Bias, filter divergence and covariance inflation (with the l63 model)
- Synthetic observations
- Notes for current users
- Non-backwards compatible changes
- New features
- New models
- Changed models
- New observation types/forward operators
- New observation types/sources
- New diagnostics and documentation
- New utilities
- Kodiak
- DART Kodiak release documentation
- DART overview
- Getting started
- Installation
- Unpacking the distribution
- Customizing the build scripts – overview
- Building the Lorenz_63 DART project
- Running Lorenz_63
- Experiment overview
- The tutorial
- Matlab® diagnostics
- Bias, filter divergence and covariance inflation (with the l63 model)
- Synthetic observations
- Notes for current users
- Non-backwards compatible changes
- New features
- New models
- Changed models
- New observation types/sources
- New diagnostics and documentation
- New utilities
- Known problems
- Jamaica
- DART Jamaica release documentation
- Overview of DART
- What’s new
- What’s required
- What’s nice to have
- Where’s the (gui) interface?
- Installation
- Building and customizing the ‘mkmf.template’ file
- Building the Lorenz_63 DART project
- Running Lorenz_63
- Experiment overview
- 1. Integrate the L63 model for a ‘long’ time
- The tutorial
- Matlab® diagnostics
- Bias, filter divergence and covariance inflation (with the l63 model)
- Synthetic observations
- Iceland
- DART Iceland release documentation
- Overview of DART
- Requirements: an F90 compiler
- Unpacking the distribution
- Customizing the build scripts – overview
- Building the Lorenz_63 DART project
- Running Lorenz_63
- Experiment overview
- Matlab® diagnostics
- Bias, filter divergence and covariance inflation (with the l63 model)
- Synthetic observations
- Hawaii
- DART Hawaii release documentation
- Overview of DART
- Requirements: an F90 compiler
- Unpacking the distribution
- Customizing the build scripts – overview
- Building the Lorenz_63 DART project
- Running Lorenz_63
- Experiment overview
- Matlab® diagnostics
- Bias, filter divergence and covariance inflation (with the l63 model)
- Synthetic observations
- Guam
- DART Guam release documentation
- Overview of DART
- Requirements: an F90 compiler
- Unpacking the distribution
- Customizing the build scripts – overview
- Building the Lorenz_63 DART project
- Running Lorenz_63
- Experiment overview
- Matlab diagnostics
- Bias, filter divergence and covariance inflation (with the l63 model)
- Synthetic observations
- Fiji
- DART Fiji release documentation
- Overview of DART
- Requirements: an F90 compiler
- Unpacking the distribution
- Customizing the build scripts – overview
- Building the Lorenz_63 DART project
- Running Lorenz_63
- Experiment overview
- Matlab® diagnostics
- Bias, filter divergence and covariance inflation (with the l63 model)
- Synthetic observations
- DART Manhattan Release Notes
- RMA notes
- DART Manhattan Differences from Lanai Release Notes
- Forward Operator
- PROGRM OR MODULE name_of_thing
- PROGRM OR MODULE name_of_thing
- Vertical Conversion of Observations
- Bitwise Considerations
- Netcdf Inflation Files
- State Stucture
- Filter async modes
- Distributed State
- DART Lanai Differences from Kodiak Release Notes
- DART “Jamaica release” difference document
- DART “pre_j release” Documentation
- Overview of DART
- DART POST-Iceland revisions
- DART “POST Iceland release” summary of changes.
- DART “Iceland release” Documentation
- DART “post-Iceland release” Documentation
- Overview of DART
- DART Iceland revisions
- DART “pre_hawaii release” Documentation
- Overview of DART
- DART “pre_guam release” Documentation
- Overview of DART
- MODULE location_mod (channel)
- MODULE location_mod
- MODULE (1D) location_mod
- MODULE location_mod (threed_cartesian)
- MODULE location_mod (threed_sphere)
- program
obs_seq_verify
- PROGRAM
wakeup_filter
- PROGRAM
compare_states
- PROGRAM
gen_sampling_err_table
- PROGRAM
perturb_single_instance
- system simulation programs
- PROGRAM
compute_error
- PROGRAM preprocess
- PROGRAM
obs_impact_tool
- program
create_fixed_network_seq
- program
obs_loop
- program
perfect_model_obs
- program
obs_selection
- program
obs_sequence_tool
- PROGRAM
integrate_model
- PROGRAM
obs_diag
(for 1D observations) - PROGRAM
obs_diag
(for observations that use the threed_cartesian location module) - PROGRAM
obs_diag
(for observations that use the threed_sphere location module) - PROGRAM
fill_inflation_restart
- program
obs_seq_coverage
- PROGRAM
advance_time
- program
model_mod_check
- PROGRAM
closest_member_tool
- PROGRAM
restart_file_tool
- PROGRAM
filter
- program
obs_keep_a_few
- program
create_obs_sequence
- PROGRAM
obs_seq_to_netcdf
- program
obs_common_subset
- MODULE ensemble_manager_mod
- MODULE random_seq_mod
- MODULE mpi_utilities_mod
- MODULE time_manager_mod
- MODULE utilities_mod
- MODULE types_mod
- MODULE schedule_mod
- MODULE
obs_kind_mod
- MODULE DEFAULT_obs_kind_mod
- MODULE obs_sequence_mod
- MODULE smoother_mod
- MODULE assim_model_mod
- MODULE assim_tools_mod
- MODULE cov_cutoff_mod
- MODULE obs_model_mod
- MODULE reg_factor
- MODULE adaptive_inflate_mod
- MODULE quality_control_mod
- MODULE filter_mod
- MODULE
obs_def_gps_mod
- MODULE obs_def_dew_point_mod
- MODULE obs_def_ocean_mod
- MODULE
obs_def_1d_state_mod
- MODULE
obs_def_radar_mod
- MODULE DEFAULT_obs_def_mod
- MODULE obs_def_mod
- MODULE
obs_def_rttov_mod
- SSEC Data Center
- GTSPP Observations
- GPS Observations
- GSI2DART
- WOD Observations
- Total Precipitable Water Observations
- ROMS observations to DART observation sequences
- PROGRAM
COSMOS_to_obs
- PROGRAM
COSMOS_development
- PROGRAM
littler_tf_dart
- PROGRAM
rad_3dvar_to_dart
- 3DVAR/4DVAR Observation Converters
- PROGRAM
tc_to_obs
- Tropical Cyclone ATCF File to DART Converter
- PROGRAM
level4_to_obs
- PROGRAM
cice_to_obs
- PROGRAM
dwl_to_obs
- PROGRAM
MIDAS_to_obs
- PROGRAM
sst_to_obs, oi_sst_to_obs
- PROGRAM
MOD15A2_to_obs
- DART observations and MODIS products.
- PROGRAM
prepbufr
- PROGRAM create_real_obs
- SSUSI F16 EDR-DSK format to observation sequence converters
- Oklahoma Mesonet MDF Data
- PROGRAM
snow_to_obs
- PROGRAM
text_to_obs
- Radar Observations
- MADIS Data Ingest System
- QuikSCAT SeaWinds Data
- AIRS Observations
- Aviso+/CMEMS Observations
- null_model
- PROGRAM
dart_to_ncommas
- PROGRAM
ncommas_to_dart
- MODULE dart_pop_mod (POP)
- PROGRAM
model_to_dart
for MPAS OCN - PROGRAM
mpas_dart_obs_preprocess
- CESM+DART setup
- MODULE model_mod
- module atmos_radon_mod
- module atmos_sulfur_hex_mod
- module atmos_tracer_driver_mod
- module atmos_carbon_aerosol_mod
- module atmos_tracer_utilities_mod
- module vert_advection_mod
- module time_manager_mod
- module field_manager_mod
- module horiz_interp_mod
- Module fms_mod
- module constants_mod
- module platform_mod
- Module Utilities
- module tracer_manager_mod
- module mpp_domains_mod
- module mpp_io_mod
- module mpp_mod
- module fft_mod
- Module sat_vapor_pres_mod
- module topography_mod
- module gaussian_topog_mod
- module diag_manager_mod
- diag_table_tk
- module bgrid_polar_filter_mod
- module bgrid_halo_mod
- module bgrid_horiz_mod
- module bgrid_cold_start_mod
- module bgrid_prog_var_mod
- module bgrid_diagnostics_mod
- module bgrid_integrals
- module bgrid_change_grid_mod
- module bgrid_masks_mod
- module bgrid_vert_mod
- module atmosphere_mod
- module bgrid_core_mod
- module bgrid_core_driver_mod
- Module hs_forcing_mod
- program atmos_model
- PROGRAM
replace_wrf_fields
- PROGRAM
wrf_dart_obs_preprocess
- PROGRAM
netcdf_to_gitm_blocks
- PROGRAM
gitm_blocks_to_netcdf
- MODULE model_mod
- PROGRAM
cam_to_dart
- CAM
- PROGRAM
dart_to_cam
- PROGRAM
trans_pv_sv
- PROGRAM
create_ocean_obs
- PROGRAM
trans_sv_pv
- Copyright
- Changelog
- The changes are now listed with the most recent at the top
- Nov 20 2019 : FESOM,NOAH-MP model support, better testing Tag: v9.8.0
- Aug 03 2018 : performance fix for distributed mean Revision: 12758
- May 21 2018 : enhanced inflation option, scripting Revision: 12591
- Dec 01 2017 : ROMS scripting, debugging aids Revision: 12166
- Nov 21 2017 : 1D obs_diag fix, 1D power forward operator Revision: 12138
- Aug 2 2017 : single filenames, random distributions, bug fixes. Revision: 11864
- July 7th 2017 : cam-fv, mpas_atm scripts, single file i/o. Revision: 11807
- Jun 2nd 2017 : tutorial, DART_LAB, and various updates. Revision: 11696
- April 27th 2017 : diagnostic file changes. Revision: 11545
- Feb 15th 2017 : rma_single_file merge changes. Revision: 11136
- Ancient history
- 404 Error