Webread Fails to Get All Data

6 views (last 30 days)
I am using webread to extract tabular data from a website; however, only part of the data is returned. The problem occurs because the website delays in loading the table, returning the mesage "loading additional data" at the bottom of the table until it fills. The webread function thinks that the website has finished loading and terminates the bytestream.
Is there a workaround to this problem, or do I have to develop my own webread function that will wait for all the data to load?
  2 Comments
Tarunbir Gambhir
Tarunbir Gambhir on 27 Oct 2020
It would be really helpful, if you could provide the URL to the website with that table you are referring.
Adam
Adam on 27 Oct 2020
Edited: Adam on 27 Oct 2020
For example:
If you load the site on your browser and scroll down quickly before the table fully loads you will read "loading more data..."

Sign in to comment.

Accepted Answer

Tarunbir Gambhir
Tarunbir Gambhir on 2 Dec 2020
webread() is typically used to access a RESTful API but it is also able to download the HTML source of a website. However, content that is dynamically loaded and not present in the HTML source will not be downloaded. A possible workaround is to use the website's API (if one is available) to collect the data and import that into MATLAB.
This is a current limitation of the webread() function and I have brought this issue to the concerned people and it might be considered in any future release.

More Answers (0)

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!