ERDDAP New Features and Updates for Administrators

Explore the latest features and enhancements in ERDDAP (Environmental Research Division's Data Access Program) for administrators. Learn about version updates, email logging, dataset reloading versus updating, improved metadata handling, and archiving datasets using BagIt bags for NCEI submission.

• ERDDAP
• Updates
• Administrators
• Dataset Management
• Metadata

avar_704 Follow

Uploaded on Sep 26, 2024 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

Presentation Transcript

1. New or Little Known ERDDAP Features (for Administrators) Bob Simons DOC / NOAA / NMFS / SWFSC / ERD Monterey, CA bob.simons@noaa.gov

2. ERDDAP v1.74 released 2016-10-07 Didn't publicly announce it. Long story. But v1.74 is available and good. v1.76 is coming very soon.

3. email.log Please set up the email system in ERDDAP (in setup.xml). The subscription system requires them. Daily Report email Significant errors that you need to deal with, e.g. , bad files in datasets email.log - If you don't, please periodically read the emails ERDDAP would have sent which are in bigParentDirectory/logs/emailYYYY-MM-DD.txt

4. Flag vs. HardFlag (v1.74) Flag URLs - listed in Daily Report email Great, e.g., in a script, for keeping datasets up-to-date. Flag file directories - bigParentDirectory/flag/ bigParentDirectory/hardFlag/ (new in v1.74) HardFlag rarely needed. Perhaps when first setting up a dataset. There are no hardFlag URLs, just the directory. Deletes all stored information for the dataset, then reloads the dataset. Useful, when you make significant changes to datasets.xml for a dataset, e.g., changing <matchAxisNDigits> to force ERDDAP to reread all data files in a dataset. Thanks to John Kerfoot and all the Argo groups.

5. Dataset: Reload vs. Update Reload a dataset - ERDDAP generates a new version of the dataset via a thorough method (e.g., looking for new/changed files, or redownloading .das, .dds, axis values fromDAP). Major reload (a process in ERDDAP) - ERDDAP scans datasets.xml every <loadDatasetsMinMinutes> (in setup.xml) for datasets where <reloadEveryNMinutes> has passed. Reloads them. Minor reload (a process in ERDDAP) - Reload a dataset in response to a flag: internal (trouble?) or external (from admin). Update a dataset - Occurs at the start of a request from a user if <updateEveryNMillis> has passed. Very fast (has anything changed?), but not thorough. Thanks to Dominic Fuller-Rowell and NGDC.

6. Lots of Changes/Improvements to GenerateDatasetsXml New EDDType=ncdump option Improved guessing of cdm_data_type Improved guessing of subset_variables Improved conversion of other metadata to CF/ACDD Improved error corrections (e.g., misspellings) ...

7. ArchiveADataset Makes BagIt Bags Useful for submitting data to NCEI for archiving. Testing now.

8. Time Zones (v1.74) New attribute for time/timestamp variables: <att name="time_zone">US/Pacific</att> ERDDAP will convert the times to Zulu/GMT timezone. Thanks to Margaret O'Brien, LTER, and EML.

9. Better Diagnostics on status.html (v1.76) A time series: each line has info as of the last major loadDatasets. Thanks to our troublesome RAID.

10. Full Support Internally for 64-bit Longs and Unicode chars and Strings (v1.76) Previously haphazard, because so few file formats support them Now, internally, fully supported. So ready for future file formats (e.g., json). Thanks to Damian Smyth, Rob Fuller, and Adam Leadbetter.

11. Truncated .nc Files Now Caught (v1.76) .nc files = header + data netcdf-java default: if truncated, not an error, return FillValue! ucar.nc2.iosp.netcdf3.N3header.disallowFileTruncation = true; Thanks to our troublesome RAID.

12. Data Provider Form (v1.64) https://coastwatch.pfeg.noaa.gov/erddap/download/setupDatasetsXml.html#DataProviderForm Thanks to PARR and NMFS.

13. Read about ERDDAP and try it out http://coastwatch.pfeg.noaa.gov/erddap/ Download and install ERDDAP http://coastwatch.pfeg.noaa.gov/erddap/download/setup.html Thank you! Questions? Comments? Suggestions? bob.simons@noaa.gov