Downloading HYCOM data using Matlab and OPeNDAP

I am trying to download HYCOM (www.hycom.org) data using the OPeNDAP access method and MATLAB. The data url can be generated by selecting desired variables from this link. I need to download the data using scripts to automate the download of many years of data.

My MATLAB code looks like below, which you should be able to run as well without any modification:

% This script download the Global HYCOM reanalysis data for a given time step
for i=0:2863
timestep = 1; % This value need to be adjusted for duration of the record needed
j=i+timestep;
url = ['http://tds.hycom.org/thredds/dodsC/GLBv0.08/expt_53.X/data/1994?depth[0:1:39],lat[0:1:3250],lon[0:1:4499],time['...
    num2str(i) ':1:' num2str(j) ...
    '],tau[0:1:2864],water_u[0:1:0][0:1:0][0:1:0][0:1:0],water_u_bottom[0:1:0][0:1:0][0:1:0],water_v[0:1:0][0:1:0][0:1:0][0:1:0],water_v_bottom[0:1:0][0:1:0][0:1:0],water_temp[0:1:0][0:1:0][0:1:0][0:1:0],water_temp_bottom[0:1:0][0:1:0][0:1:0],salinity[0:1:0][0:1:0][0:1:0][0:1:0],salinity_bottom[0:1:0][0:1:0][0:1:0],surf_el[0:1:0][0:1:0][0:1:0]'];
 
info = ncinfo(url);
    
% Successfully read these variables 
depth = ncread(url, 'depth');  
lat = ncread(url, 'lat');
lon = ncread(url, 'lon');
time = ncread(url, 'time');
    
% Trouble reading below variables
water_u = ncread(url, 'water_u');
water_v = ncread(url, 'water_v');
water_u_bottom = ncread(url, 'water_u_bottom');
water_v_bottom = ncread(url, 'water_v_bottom');
surf_el = ncread(url, 'surf_el');
water_temp = ncread(url, 'water_temp');

end

The scripts reads the depth, lat, lon, time fine, but gives error in reading the water_u variable on wards. The error message is:

Error using netcdflib
The NetCDF library encountered an error during execution of 'getVarShort' function - 'Index exceeds dimension bound
(NC_EINVALCOORDS)'.
Error in netcdf.getVar (line 136)
data = netcdflib(funcstr,ncid,varid,varargin{:});
Error in internal.matlab.imagesci.nc/read (line 605)
data = netcdf.getVar(gid, varid);
Error in ncread (line 58)
vardata = ncObj.read(varName, varargin{:});

I am not sure what is wrong with my code, and couldn't find any answer searching online so far. Appreciate your help.

[Belinda Finlay]

Did you solve this as I am having the same issue (although I got the surf_el values)?

Answer 1

[Amitava Guha]

I found out that downloading data from HYCOM is easier using Powershell script. Below is my code for a single grid point and for one year. I have been successful in running 5 such scripts at the same time, downloading 5 years of data for 1 grid point in 4-6 hour timeframe.

# This script is used to download current hindcast data from www.hycom.org 

# Result directory
# --------------------------------------------------------
$resultDirectory = $PSScriptRoot + "\Data"
If (!(test-path $resultDirectory))
{
    md $resultDirectory
}

# Input
# -------------------------------------------------------
# Select coordinate for region of interest
# Location: Latitude 6° 20' S , Longitude: 11° 15' E

#East
$east = 11.30

#West
$west = 11.25

#South
$south = -6.33

#North
$north = -6.31

$date_start = '01-Jan-1994 12:00:00'
$date_end = '31-Dec-1994 23:00:00' 

# --------------------------------------------------------
# Converting dates to datetime
$startDate = [datetime]::ParseExact($date_start,'dd-MMM-yyyy HH:mm:ss',$null)
$endDate = [datetime]::ParseExact($date_end,'dd-MMM-yyyy HH:mm:ss',$null)

Write-Host "Downloading data from " $startDate.ToString('yyyy-MM-ddTHH:mm:ssZ') " to "  $endDate.ToString('yyyy-MM-ddTHH:mm:ssZ')


for ($time = $startDate; $time -le $endDate; $time=$time.AddHours(3)){

$error_flag = 1
while ($error_flag -eq 1){
	
	# Example url. Do not delete. Used for date time format reference purposes
	#$url = "http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/2015?var=water_u&var=water_v&north=8.6&west=-57.3&east=-57.25&south=8.5&time=2015-01-01T18:00:00Z&accept=netcdf4" 

	# Download url
	$url = "http://ncss.hycom.org/thredds/ncss/GLBv0.08/expt_53.X/data/" + $time.ToString('yyyy') + "?var=water_u&var=water_v&north=" + $north.ToString() + "&west=" + $west.ToString() + "&east=" + $east.ToString() + "&south=" + $south.ToString() + "&time=" + $time.ToString('yyyy-MM-ddTHH:mm:ssZ') + "&accept=netcdf4"

	# Output file name
	$fileName = $time.ToString('yyyyMMdd_HH') + ".nc"
	$output = $PSScriptRoot + "\Data\" + $fileName


	Try{
		# Creating a web client which has the download file functionality
		#WebProxy = New-Object System.Net.WebProxy("hoeprx01.na.xom.com:8080",$true)
		$WebClient = New-Object System.Net.WebClient
		#$WebClient.Proxy=$WebProxy
		$WebClient.DownloadFile($url,$output)
		Write-Host "Successfully downloaded file:"  $fileName
		
		$error_flag = 0
	}
	Catch {
		Write-Host $_.Exception.Message`n
		$error_flag = 1

		Write-Host  "Retrying downloading file:" $fileName " ...."
	}

}

}

Answer 2

[Zhiguo Mei]

Maybe you can try this "water_u = ncread(url, 'water_u',[1,1,1,1],[200,200,40,1]);"

Reduce the size of the variable you want read at once,and it always works for me