MATLAB Answers

Downloading HYCOM data using Matlab and OPeNDAP

55 views (last 30 days)
Amitava Guha
Amitava Guha on 5 Jun 2020
Answered: Amitava Guha on 4 Jan 2021 at 7:06
I am trying to download HYCOM (www.hycom.org) data using the OPeNDAP access method and MATLAB. The data url can be generated by selecting desired variables from this link. I need to download the data using scripts to automate the download of many years of data.
My MATLAB code looks like below, which you should be able to run as well without any modification:
% This script download the Global HYCOM reanalysis data for a given time step
for i=0:2863
timestep = 1; % This value need to be adjusted for duration of the record needed
j=i+timestep;
url = ['http://tds.hycom.org/thredds/dodsC/GLBv0.08/expt_53.X/data/1994?depth[0:1:39],lat[0:1:3250],lon[0:1:4499],time['...
num2str(i) ':1:' num2str(j) ...
'],tau[0:1:2864],water_u[0:1:0][0:1:0][0:1:0][0:1:0],water_u_bottom[0:1:0][0:1:0][0:1:0],water_v[0:1:0][0:1:0][0:1:0][0:1:0],water_v_bottom[0:1:0][0:1:0][0:1:0],water_temp[0:1:0][0:1:0][0:1:0][0:1:0],water_temp_bottom[0:1:0][0:1:0][0:1:0],salinity[0:1:0][0:1:0][0:1:0][0:1:0],salinity_bottom[0:1:0][0:1:0][0:1:0],surf_el[0:1:0][0:1:0][0:1:0]'];
info = ncinfo(url);
% Successfully read these variables
depth = ncread(url, 'depth');
lat = ncread(url, 'lat');
lon = ncread(url, 'lon');
time = ncread(url, 'time');
% Trouble reading below variables
water_u = ncread(url, 'water_u');
water_v = ncread(url, 'water_v');
water_u_bottom = ncread(url, 'water_u_bottom');
water_v_bottom = ncread(url, 'water_v_bottom');
surf_el = ncread(url, 'surf_el');
water_temp = ncread(url, 'water_temp');
end
The scripts reads the depth, lat, lon, time fine, but gives error in reading the water_u variable on wards. The error message is:
Error using netcdflib
The NetCDF library encountered an error during execution of 'getVarShort' function - 'Index exceeds dimension bound
(NC_EINVALCOORDS)'.
Error in netcdf.getVar (line 136)
data = netcdflib(funcstr,ncid,varid,varargin{:});
Error in internal.matlab.imagesci.nc/read (line 605)
data = netcdf.getVar(gid, varid);
Error in ncread (line 58)
vardata = ncObj.read(varName, varargin{:});
I am not sure what is wrong with my code, and couldn't find any answer searching online so far. Appreciate your help.

  1 Comment

Belinda Finlay
Belinda Finlay on 31 Jul 2020
Did you solve this as I am having the same issue (although I got the surf_el values)?

Sign in to comment.

Accepted Answer

Amitava Guha
Amitava Guha on 4 Jan 2021 at 7:06
I found out that downloading data from HYCOM is easier using Powershell script. Below is my code for a single grid point and for one year. I have been successful in running 5 such scripts at the same time, downloading 5 years of data for 1 grid point in 4-6 hour timeframe.
# This script is used to download current hindcast data from www.hycom.org
# Result directory
# --------------------------------------------------------
$resultDirectory = $PSScriptRoot + "\Data"
If (!(test-path $resultDirectory))
{
md $resultDirectory
}
# Input
# -------------------------------------------------------
# Select coordinate for region of interest
# Location: Latitude 6° 20' S , Longitude: 11° 15' E
#East
$east = 11.30
#West
$west = 11.25
#South
$south = -6.33
#North
$north = -6.31
$date_start = '01-Jan-1994 12:00:00'
$date_end = '31-Dec-1994 23:00:00'
# --------------------------------------------------------
# Converting dates to datetime
$startDate = [datetime]::ParseExact($date_start,'dd-MMM-yyyy HH:mm:ss',$null)
$endDate = [datetime]::ParseExact($date_end,'dd-MMM-yyyy HH:mm:ss',$null)
Write-Host "Downloading data from " $startDate.ToString('yyyy-MM-ddTHH:mm:ssZ') " to " $endDate.ToString('yyyy-MM-ddTHH:mm:ssZ')
for ($time = $startDate; $time -le $endDate; $time=$time.AddHours(3)){
$error_flag = 1
while ($error_flag -eq 1){
# Example url. Do not delete. Used for date time format reference purposes
#$url = "http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/2015?var=water_u&var=water_v&north=8.6&west=-57.3&east=-57.25&south=8.5&time=2015-01-01T18:00:00Z&accept=netcdf4"
# Download url
$url = "http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/" + $time.ToString('yyyy') + "?var=water_u&var=water_v&north=" + $north.ToString() + "&west=" + $west.ToString() + "&east=" + $east.ToString() + "&south=" + $south.ToString() + "&time=" + $time.ToString('yyyy-MM-ddTHH:mm:ssZ') + "&accept=netcdf4"
# Output file name
$fileName = $time.ToString('yyyyMMdd_HH') + ".nc"
$output = $PSScriptRoot + "\Data\" + $fileName
Try{
# Creating a web client which has the download file functionality
#WebProxy = New-Object System.Net.WebProxy("hoeprx01.na.xom.com:8080",$true)
$WebClient = New-Object System.Net.WebClient
#$WebClient.Proxy=$WebProxy
$WebClient.DownloadFile($url,$output)
Write-Host "Successfully downloaded file:" $fileName
$error_flag = 0
}
Catch {
Write-Host $_.Exception.Message`n
$error_flag = 1
Write-Host "Retrying downloading file:" $fileName " ...."
}
}
}

  0 Comments

Sign in to comment.

More Answers (1)

Zhiguo Mei
Zhiguo Mei on 26 Nov 2020
Maybe you can try this "water_u = ncread(url, 'water_u',[1,1,1,1],[200,200,40,1]);"
Reduce the size of the variable you want read at once,and it always works for me

  0 Comments

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!