Changelog¶
This file documents the most user-visible changes to the DART code. It is not intended to document every change, but instead is intended to inform people what features are now available or have been removed. Detailed changes are always available through the version control framework.
DART now uses Git for version control but has preserved the revision history from when subversion (and CVS before that)
was used. The previous revision numbers can be related to git hashes by searching the output of git log
0[1011] machine:dartGIT % git log > full_git_log.txt
A reminder that since many files were moved or renamed, the best way to get the complete log is to use
git log --follow
for information on individual files.
The changes are now listed with the most recent at the top¶
Oct 29 2020 : radiance support, MPAS, obs converters Tag: v9.9.0 |
Use RTTOV (Radiative Transfer for TOVS) routines to support radiance assimilation. - Introduction to DART support for RTTOV - WRF, MPAS, and CAM-FV model interfaces now support radiance assimilation. - Added GOES 16-19 ABI converter |
NOTE: The |
MPAS regional configurations now supported. |
Converted CHANGELOG to a markdown document, put newest content at top. |
Converted many HTML documents to markdown - renamed
|
declare hexadecimal constants according to the Fortran standard. |
GSI2DART converter updated - Thanks to Craig Schwartz & Jamie Bresch. |
The WRF-DART tutorial has been rewritten as
|
Hydro-DART (AKA wrf-hydro/DART) has been updated to be Manhattan-compliant. - also support masked bucket - added perturbed forcing capability |
The support for POP and CESM2 has been implemented and documented. |
|
|
WRF-Chem/DART documentation and datasets have been updated for Manhattan. WRF-Chem contact information. |
Fixed bug in |
Support absolute humidity observations - Thanks to Michael Ying. |
|
new observation converters including (but not limited to): - absolute humidity - streamflow observations from the Mexican water agency - streamflow observations from the USGS - total water storage observations from GRACE - radiance observations from GOES |
the following forward operator modules are either new or modified: - (M)
|
|
GITM is now fully Manhattan compliant. |
fix bug in madis rawin converter |
avoid computing posterior inflation if using the ‘relaxation to prior spread’ inflation option – Thanks to Craig Schwartz. |
add additional reporting options to the |
Nov 20 2019 : FESOM,NOAH-MP model support, better testing Tag: v9.8.0¶
first release entirely from GIT
fixed bug in
fill_inflation_restart
tool which used the prior inflation mean and sd for both prior and posterior inflation files. now correctly uses the posterior mean/sd if requested.fixed a typo in the location test script that prevented it from running
additional functionality in the quad interpolation code, now supports grids which start at 90 (north) and end at -90 (south).
if possible, send shorter MPI messages. improves performance on some platforms and MPI implementations.
add explicit call to
initalize_utilities()
where it was missing in a couple of the WRF utility routines.added an example of how to use a namelist to the
text_to_obs.f90
observation converter program.Removing the clamping messages in
clamp_variable()
of clamped valueschanged argument names using reserved keywords.
state_vector_io_mod:read_state() 'time' to 'model_time'
random_seq_mod:random_gamma() 'shape' to 'rshape', 'scale' to 'rscale'
random_seq_mod:random_inverse_gamma() 'shape' to 'rshape', 'scale' to 'rscale'
obs_def_mod:init_obs_def() 'kind' to 'obkind', 'time' to 'obtime'
obs_utilities_mod: 'start' to 'varstart', 'count' to 'varcount'
The FESOM model is now Manhattan-ready. Thanks to Ali Aydogdu
The noah model is now Manhattan-ready and may be used with NOAH-MP.
bugfixed references to the
documentation
directory that was renamedguide
to comply with GitHub Pages.improved
test_dart.csh
functionality.
Apr 30 2019 : cam-fv refactor, posteriors optional, QC 8 Revision: 13138 |
The CAM Finite Volume (cam-fv) |
cam-fv setup and scripting support added for CESM 2.1, including advanced archiving and compression |
fix for WRF’s wind direction vectors when using the Polar Stereographic map projection. Thanks to Kevin Manning for the fix. |
Add filter namelist option to avoid calling the posterior forward
operators and to not create those copies in the |
Use less memory if writing ensemble member values into the
|
added a DART QC of 8 for failed vertical conversions |
updated Matlab scripts support QC=8 and no posterior in obs sequence files. |
sampling error correction table now has all ensemble sizes between 3 and 200 |
|
|
|
support for NAG compiler |
fixed Intel compiler bug in |
|
Default routines now create a time variable as |
|
|
expanded functionality in |
Removed ‘slow’ sorting routines from |
replacing some repeated native netCDF library calls with routines from the
|
Updated dewpoint equation to avoid dividing by zero given a very unlikely scenario (r12832) |
More efficient implementation of adaptive inflation |
Yongfei Zhang and Cecilia Bitz added improvements to the CICE model and observation converters and forward operators. These changes also use the locations of the ‘new’ glade filesystem. They used CESM tag: cesm2_0_alpha06n |
Worked with Yongfei Zhang to remove prototype codes and more completely document observation converters and data sources for cice assimilation. |
removed |
increased the maximum number of input files to |
Updated the |
Updated |
Better error and informational messages in various routines. |
Aug 03 2018 : performance fix for distributed mean Revision: 12758¶
Important performance fix if model does vertical conversion for localization.
Results were not wrong but performance was poor if distribute_mean = .true.
was selected in the &assim_tools_nml
namelist.
Now distributing the mean runs in close to the non-distributed time and uses
much less memory for large models. This only impacts models which do a vertical
conversion of either the observation or state vertical coordinate for
localization AND which set &assim_tools_nml :: distribute_mean = .true.
to
use less memory.
When using a distributed mean convert_all_obs_verticals_first = .true.
should be set. If your observations will impact most of the model state, then
convert_all_state_verticals_first = .true.
can also be set.
Jun 18 2018 : CAM/CESM 2.0, DART QC 8, closest_member_tool Revision: 12682 |
Support for cam-fv assimilations in the CESM 2.0 release. See
documentation in |
|
Updates to fix minor problems with the new WRF scripts |
Added the |
Revival of the |
Revival of the |
Added more functions to the |
May 21 2018 : enhanced inflation option, scripting Revision: 12591¶
Enhanced inflation algorithm added. See the
filter_mod.html
for new documentation on this option.Updated WRF scripts for the Manhattan release.
obs_diag
reports statistics on DART QC 8, observation failed vertical conversion. Matlab scripts also updated to support QC 8.New parallel conversion scripts for GPS Radio Occultation observations and NCEP prepbufr conversions.
Further updates to documentation files to change KIND to QTY or Quantity.
Documented required changes when moving from the Lanai/Classic release to Manhattan in
documentation/html/Manhattan_diffs_from_Lanai.html
Expanded the routines in the
netcdf_utilities_mod.f90
Add an ensemble handle parameter to the 6 ensemble manager routines where it was missing.
The
advance_time
program can read/generate CESM format time strings (YYYY-MM-DD-SSSSS).Fixed a bug in the netcdf read routines that under certain circumstances could report an array was using the unlimited dimension incorrectly.
Removed the option to try to bitwise reproduce Lanai results; due to the number of changes this is no longer possible.
Minor bug fixes to the (seldom used) perturb routines in the WRF and mpas_atm
model_mod.f90
files. (used to add gaussian noise to a single model state to generate an ensemble; this is never the recommended method of starting a new experiment but the code remains for testing purposes.)Several remaining model-specific
model_mod_check
programs were removed in favor of a single common program source file.Keep
filter_mod.dopplerfold.f90
in sync withfilter_mod.f90
, andassim_tools_mod.pf.f90
in sync withassim_tools_mod.f90
.Removed makefiles for the obsolete
trans_time
program.
Mar 01 2018 : ROMS, MMC, PMO, mpas_atm debug, etc Revision: 12419 |
Fix a debug message in the mpas_atm model which might have caused a buffer overflow crash when formatting a message for a larger ensemble size. |
Update the ROMS shell scripts to support PBS, SLURM, as well as LSF. Update the ROMS model_mod html documentation. |
Update the default cam-fv |
Fixed variable type for |
Miscellaneous minor Bug fixes: - Print format wider for fractional levels
in |
Add surface pressure to the default state list in the wrf
|
|
several |
the DART QTYs are now available via the state structure in the wrf and
clm |
support the NAG compiler better. (contact dart@ucar.edu for more help if you want to use this compiler. some hand work is still needed.) |
streamlined the debug output from the |
minor formatting change to the dart log file output for the list of observation types being assimilated, evaluated, and using precomputed forward operators. |
fixed an uninitialized variable in the BGRID model code in a routine that isn’t normally used. |
Updated the |
Fixed an incorrect test when printing out a log message describing if the inflation would be variance-adaptive or not. |
Change the location of the POP MDT reference file to be relative to the current run directory and not an absolute file location on cheyenne. |
Make the ROMS, CM1, and POP model_mod log namelist information to the namelist log file and not the main DART log file. |
Updated several html documentation files, including the
|
Updated the instructions for the GSI to DART obs converter to suggest some needed compiler flags in certain cases. |
Updated the location module test programs. |
Dec 01 2017 : ROMS scripting, debugging aids Revision: 12166¶
Added an option to the ROMS model scripting to advance the model ensemble members in parallel using a job array.
Updated the DART_LAB Matlab GUIs to log a history of the settings and results.
- Added a debug option to the filter namelist,
write_obs_every_cycle
, to output the fullobs_seq.final
during each cycle of filter.(Very slow - use only when debugging a filter crash.) Allow the test grid in
model_mod_check
to cross the prime meridian for testing longitude interpolation in grids that cross the 360/0 line.
## Nov 22 2017 :: minor updates for DA challenge files Revision: 12144 |
added |
added new obs types to the workshop version of the |
Nov 21 2017 : 1D obs_diag fix, 1D power forward operator Revision: 12138¶
fixed a bad URL reference in tutorial section 18
fixed a crash with the 1D version of the observation diagnostics program when including identity observations.
all models with a
workshop_setup.csh
now build the same set of programs. (some/most did not build obs_diag - which is used in the tutorial)added a 1D obs-to-a-power forward operator.
updates to the matlab plotting routines for NetCDF observation formats
World Ocean Database (WOD) converter supports partial year conversions and 2013 file formats.
Oct 17 2017 : mpas_atm bug fix, various other updates. Revision: 12002 |
Fixed a bug in the mpas_atm |
Fixed a bug in the cam-fv |
Added two new modules: |
Removed |
Updated and improved the observation converter for ionospheric observations from the COSMIC GPS satellite. |
Updated the cam-fv scripts for cesm2_0_beta05. |
Updated the Matlab diagnostics documentation. ‘help DART’ or ‘doc DART’ will give an overview of the available Matlab diagnostics shipped with the dart distribution. |
Added the observation type |
|
Updated the channel and column location modules to be compatible with the current required interfaces. |
Updated the |
Fixed a misleading status message in the |
Added some additional debugging options to the mpi utilities module. if you have problems that appear to be MPI related, contact us for more help in enabling them. |
Improved some error messages in |
Aug 2 2017 : single filenames, random distributions, bug fixes. Revision: 11864¶
added code to support listing input and output filenames directly in the namelist instead of having to go through an indirect text file. most useful for programs that take a single input and output file, but works for all cases.
bug fix in
location_io_mod.f90
that affectedobs_seq_to_netcdf
(error in adding vertical location types to output file).fix to
convert_gpsro_bufr.f90
converter (GPS obs from BUFR files) that failed if r8 defined to be r4.added draws from gamma, inverse gamma, and exponential distributions to the random sequence module.
various updates to the cam scripts to work more smoothly with the most recent CIME changes and DART Manhattan updates.
added
QTY_CWP_PATH
andQTY_CWP_PATH_ZERO
to the default quantities list for theobs_def_cwp_mod.f90
forward operator.improved some error messages in the diagnostic matlab scripts
## July 18 2017 :: bug fixes, documentation updates. Revision: 11830 |
fixed bug in |
fixed a bug that would not allow filter to start from a single ensemble
member if |
updates to HTML documentation especially for types/quantities (replacing kinds) |
updates to |
July 7th 2017 : cam-fv, mpas_atm scripts, single file i/o. Revision: 11807¶
mpas_atm: scripts completely revised for the Manhattan release. Many thanks to Soyoung Ha and Ryan Torn for the contributed code.
cam-fv: scripts and
model_mod.f90
updated for cesm2_0_beta05.
Single File I/O:
Now we are able to run
single_file_in
andsingle_file_out
with MPI.single_file_io_mod.f90
has been removed and its functionality has been moved todirect_netcdf_mod.f90
.single_file_io_mod.f90
has been removed from all of thepath_names_*
files in the repository. (Remove it from any privatepath_names_*
files.)
June 27rd 2017 : CICE 5, model_mod_check, tutorial. Revision: 11770 |
Updated support for CICE5. |
Updated support for |
Ensured consistency between low-order namelists and the updated DART tutorial. Updated documentation of many namelists. More to come. |
|
|
Jun 2nd 2017 : tutorial, DART_LAB, and various updates. Revision: 11696¶
bring the DART tutorial pdf slides up to date with the current release.
include new GUIs with adaptive inflation options in DART_LAB:
oned_model_inf.m
run_lorenz_96_inf.m
added the lorenz_96_2scale model - additional kinds of
QTY_SMALL_SCALE_STATE
andQTY_LARGE_SCALE_STATE
added as required.add useful attributes to the variables in the diagnostic files
updates and minor bug fixes to the matlab diagnostic scripts
updates to the default input.nmls for models
updates to the cam-fv shell scripts to work with the CESM2.0 framework
updates to the cam-fv
model_mod
for support ofcam-chem
variables Added more QUANTITIES/KINDS for chemistry species. Removed support for ‘stand-alone’ cam and cam-se (cam-se will be a separate ‘model’).major bug fix in the simple_advection
model_mod
: Fixed an error with the layout of the state vector.obs_def_radar_mod
: Fixed a serious bug in the fall velocity forward operator. If the fall speed field is not in the state the test for a bad istatus from the interpolate() call was looking at the wrong variable and returning ok even if interpolate() had set bad values.bug fix in the wrf model_mod for fields which have a vertical stagger
fix to the makefiles for the GSI2DART observation converter
added additional netcdf and location utility routines
various fixes to documentation and test code
renamed
QTY_RAW_STATE_VARIABLE
toQTY_STATE_VARIABLE
(RAW is redundant)direct_netcdf_mod
: Renamedlimit_mem
tobuffer_state_io
.buffer_state_io
is now a logical that states if a variable that tells DART it it should read and write variables all at once or variable-by-variable.
May 5th 2017 : major changes to model_mod interfaces. Revision: 11615 |
A long-awaited overhaul of the model_mod interfaces. All models which are in our subversion repository and are supported in the Manhattan release have been updated to match the new interfaces. If you have model_mods with extensive changes, our recommendation is to diff your changes with the version you checked out and insert those changes into the new version. The changes for this update are unfortunately extensive. |
The detailed list of changes: |
|
|
|
|
|
|
|
A new module |
A new module |
|
We have removed |
Models now have more control over when vertical conversion happens - on demand as needed, or all up front before assimilation. |
Models that were doing vertical conversion in |
Models which were not should set: convert_all_state_verticals_first = .false. convert_all_obs_verticals_first = .true. |
The |
Models doing vertical localization should add a call to
|
Most |
Some of the internal changes include pulling common code from the locations
modules into a |
|
Many utility programs use the |
The |
Sampling Error Correction now reads the values it needs from a single netcdf
file found in |
April 27th 2017 : diagnostic file changes. Revision: 11545¶
Two additional Diagnostic Files (forecast and analysis) in Filter which can be set with the namelist option (stages_to_write)
input writes out mean and sd if requested.
For low order models, mean and sd are only inserted into restart files with a single time step.
forecast
contains the forecast and potentially the mean and sd for the, this is mostly important for lower order models which cycle
preassim before assimilation
No Inflation: same as forecast
Prior Inf: the inflated ensemble and damped prior inf
Post Inf: same as forecast
Prior and Post Inf: the inflated ensemble and damped prior inf
postassim after assimilation (before posterior infation)
No Inflation: same as analysis
Prior Inf: same as analysis
Post Inf: assimilated ensemble and damped posterior inflation
Prior and Post Inf: assimilated ensemble and damped posterior inflation
analysis after assimilation and before potentially update posterior inflation ensemble and updated prior inf
No Inflation: assimilated ensemble
Prior Inf: assimilated ensemble and updated prior inf
Post Inf: post inflated ensemble and updated posterior inflation
Prior and Post Inf: post inflated ensemble and updated prior inf and posterior inflation
output
a single time step of the output ensemble and potentially updated prior inf and posterior inflation
Feb 15th 2017 : filter updates. Revision: 11160 |
The postassim diagnostics file was being incorrectly written after posterior inflation was applied. It is now written immediately after the assimilation update, and then posterior inflation, if enabled, is applied. |
Sampling Error Correction now reads data from a single netcdf file for any
ensemble size. To add other sizes, a program can generate any ensemble size
and append it to this file. The default file is currently in
|
|
Filter and PMO no longer need the “has_cycling” flag. |
#### Changes to the filter_nml are : |
|
#### Changes to the perfect_model_obs_nml are : |
|
Feb 15th 2017 : rma_single_file merge changes. Revision: 11136¶
Filter and PMO can now run with multiple cycles for low order models. The output for this is only supported with single file output (members, inflation, mean, sd are all in the same file).
Added matlab support for diagnostics format in lower order models.
Changes to the filter_nml are :¶
output_restart
RENAMED tooutput_members
restart_in_file_name
RENAMED toinput_state_file_list
restart_out_file_name
RENAMED tooutput_state_file_list
single_restart_file_in
RENAMED tosingle_file_in
single_restart_file_out
RENAMED tosingle_file_out
input_state_files
ADDED - not currently workingoutput_state_files
ADDED - not currently workinghas_cycling
ADDED for low order models
Changes to the perfect_model_obs_nml are :¶
start_from_restart
RENAMEDread_input_state_from_file
output_restart
RENAMEDwrite_output_state_to_file
restart_in_file_name
RENAMEDinput_state_files
restart_out_file_name
RENAMEDoutput_state_files
single_file_in
ADDED for low order modelssingle_file_out
ADDED for low order modelshas_cycling
ADDED for low order models
Jan 13th 2017 : rma_fixed_filenames merge changes. Revision: 10902 |
Specific namelist changes include: |
1. Earlier versions of the RMA branch code supported both direct NetCDF reads/writes and the original binary/ascii DART format restart files. As of the next update DART format files are no longer supported. All I/O is NetCDF only. If your model does not use NetCDF you will still need a model_to_dart and dart_to_model converter; otherwise all DART programs read the model’s NetCDF files directly. The namelist options related to selecting direct netcdf I/O have been removed. |
1. Diagnostic and state space data (such as inflation, mean and sd information) that were previously stored in {Prior,Posterior}_Diag.nc are now broken up into multiple files and have fixed filenames. This decreases the IO time for diagnostic output and reduces the number of namelist options. |
1. There is no longer support for observation space inflation (i.e. inf_flavor = 1). Contact us at dart@ucar.edu if you have an interest in using this option. |
#### Changes to the filter_nml are : |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#### Changes to the perfect_model_obs_nml are : |
|
|
#### Changes to the state_space_diag_nml are : |
|
|
|
#### Changes to the state_vector_io_nml are : |
|
#### Changes to the ensemble_manager_nml are : |
|
#### Changes to the integrate_model_nml are : |
|
#### Scripting with CESM |
See |
(To help find things: input_priorinf_mean output_priorinf_mean )
|
! stage_name is {input,preassim,postassim,output} ! base_name is
|
This shows where inflation file names are defined. > grep -I
|
subroutine set_member_file_metadata(file_info, ens_size, my_copy_start) call set_file_metadata(file_info, icopy, stage_name, base_name, desc, offset) |
subroutine set_stage_file_metadata(file_info, copy_number, stage, base_name, desc, offset) write(string1,‘(A,’‘.nc’‘)’) trim(stage_name)//’_’//trim(base_name) |
subroutine set_explicit_file_metadata(file_info, cnum, fnames, desc) file_info%stage_metadata%filenames(cnum,idom) = trim(fnames(idom)) file_info%stage_metadata%file_description(cnum,idom) = trim(string1) |
function construct_file_names(file_info, ens_size, copy, domain)
write(construct_file_names, ‘(A,’‘ |
Also see harnesses/filename_harness/files: ENS_MEAN_COPY PriorDiag_mean.nc |
#### ADDITIONAL NOTES : |
1. currently the closest_member_tool is broken but plans on being fixed soon. 1. restart_file_tool and most model_to_dart/dart_to_model programs have been deprecated, since DART formated restarts are no longer supported. 1. some programs such as model_mod_check have not been fully tested and need to be exercised with the new naming conventions. |
Ancient history¶
To see previous history, use:
$ git log --follow
$ git diff --name-status XXXX YYYY
where XXXX
and YYYY
are commits or branches.