ERDDAP - Facilitating Data Access and Download into Your Favorite Client Software
ERDDAP acts as a middleman data server that simplifies the process of finding, accessing, and downloading data from various sources into preferred client software by addressing common challenges such as diverse data formats, search systems, and metadata availability. Its capabilities include improving metadata, standardizing time data, and providing a unified search and data request interface.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
ERDDAP is a middleman data server designed to help you get data easily from any data source into your favorite client software. ERDDAP Your Favorite Client Software Bob Simons DOC / NOAA / NMFS / SWFSC / ERD Monterey, CA bob.simons@noaa.gov
Finding data and downloading it into your favorite client software is too hard! 1. Most data servers lack a search system. It's hard to find datasets of interest. 2. Different data servers use different request protocols. (XML, SOAP+XML, OPeNDAP, WCS, WFS, SOS, HTML form, shopping cart, ...!) 3. Different data servers return data in different formats. (XML, SOAP+XML, DAP binary, ASCII text, HDF4, HDF 5, NetCDF, ...!) and it usually isn't the file format that you want. (html table , ESRI. asc, .kml, .mat, .csv, .tsv, .json, .xhtml) 4. Different datasets have time formatted in different ways. ("Jan 2, 1985", "02-JAN-1985", "1/2/85", "2/1/85", "1985-01-02", Year + DayOfyear, "seconds since 1900-01-01", "days since 1-1-1", ...!) 5. Most datasets have insufficient metadata. That makes it hard to understand the data.
2 3 1 4
Acting as a middleman allows ERDDAP to Improve each dataset's metadata. Generate ISO 19115 metadata. Standardize the format of time data. Provide a unified way for users to search for datasets. Offer a standard way to request data from any dataset. Let users specify the response file format. Solve all of the problems I identified earlier. Make life easier for data providers and for users.
ERDDAP's web applications are built on ERDDAP's web services. Web Applications - a web page with a form URL URL RESTful Web Services - a single request URL specifies an entire request
Gridded Data A RESTful URL specifies an entire request: dataset, response file type, subset: http://coastwatch.pfeg.noaa.gov/erddap/griddap/jplMURSST41.html ?analysed_sst[(2016-07-01T09:00:00Z)][(-89.99):(89.99)][(-179.00):(180)] Special file types: .html (Data Access Form), .graph (graph form), .fgdc, .iso19115, .das, .dds Data file types: .asc, .csv, .esriAscii, .json, .htmlTable, .mat, .nc, .odvTxt, .tsv, ... Image file types: .geotif, .kml, .pdf, .png, .transparentPng
Tabular Data A RESTful URL specifies an entire request: dataset, response file type, subset: http://coastwatch.pfeg.noaa.gov/erddap/tabledap/pmelTaoDySst.html ?longitude,latitude,T_25,time&time=2011-08-10T12:00:00Z Special file types: .html (Data Access Form), .graph (graph form), .fgdc, .iso19115, .das, .dds, .subset Data file types: .asc, .csv, .esriCsv, .htmlTable, .geoJson, .json, .mat, .nc, .ncCF, .ncCFMA, .odvTxt, .tsv, .xhtml, ... Image file types: .geotif, .kml, .pdf, .png, .transparentPng
ERDDAP users don't need a new plug-in or library. Plug-ins and libraries (client-side solutions) can be great. However, they have disadvantages: Users may be reluctant to install them, may not have permission to install them, may have difficulty installing them. Plug-ins may break when the application is updated. If there is new version of the plug-in (e.g., with a bug fix), it is impossible to get all users to update. You probably don't even know who they are. ERDDAP is a server-side solution. It doesn't have these problems. Users can keep using their favorite client software. One administrator updates ERDDAP and the changes apply to all users, immediately.
Public Access to Research Results (PARR) Requirements Government funded data shall be publicly and freely: Discoverable via a catalog (data.noaa.gov -> data.gov) Understandable via metadata Accessible via a web service (e.g., DAP) (not just downloadable files or a shopping cart) Archived Within one year (by last year ASAP!) ERDDAP can help with all of these requirements! https://www.whitehouse.gov/blog/2013/02/22/expanding-public-access-results-federally-funded- research https://www.whitehouse.gov/sites/default/files/microsites/ostp/ostp_public_access_memo_2013.pdf
For data providers: How does ERDDAP relate to other data servers? If you already have a data server or web pages, keep them, but consider installing ERDDAP as an alternative. If you don't have a data server, consider using ERDDAP. ERDDAP is reusable, free, and open-source software. Spending a day to install ERDDAP saves you months or years of effort compared to making your own system (which won't be as good). What a deal! ERDDAP lets you offer additional features to users: Catalog search services (full text and faceted search) OPeNDAP and WMS web services Users can download subsets of datasets in many common file formats (generated on- the-fly). Don't make users waste their time converting files from one format to another! Users don't need plugins or libraries. Users can generate customizable maps and graphs (generated on-the-fly). ERDDAP handles gridded data and tabular/in-situ data.
Read about ERDDAP and try it out http://coastwatch.pfeg.noaa.gov/erddap/ Download and install ERDDAP http://coastwatch.pfeg.noaa.gov/erddap/download/setup.html Thank you! bob.simons@noaa.gov
