Recently needed to get flow and level data for a watercourse hydrological station as well as regional precipitation data for relevant location(s) upstream within the watershed of the station.
The objective was to combine two decades of watercourse flow and level with regional watershed precipitation data into a single set of analysis and reporting.
Environment Canada has all of this data and provides it in a variety of different ways.
Flow and level data
For many users the Water Level and Flow web portal wateroffice.ec.gc.ca site is sufficient. However I needed multiple years’ data so I looked around for alternatives.
I found the Hydat National Water Data Archive which is provided as a SQLite database was a good source. It contains all Canadian watercourse flow and level data up until about a month or so prior to current date and was an quick and easy download and convenient way to get required data.
Real-time / more recent flow data is available using a nice search interface in the same wateroffice.ec.gc.ca portal.
In addition, this same real-time / more recent data can also be obtained by going directly to an FTP site by clicking the ‘Datamart’ link on wateroffice.ec.gc.ca.
More details about the FTP site content are available here.
Both of these methods result in downloading csv files.
The website portal allows you to search by station name or reference id. The FTP site files are named with station reference id so you need to know the reference id to get the correct csv file.
This hydrological flow and level data is also available through the which is described MSC GeoMet API. The API was a bit too complex to use to get data and do analysis so I passed on it but it looks very powerful and well documented.
Precipitation data
Precipitation data is available for stations across Canada and this website https://climate.weather.gc.ca/historical_data/search_historic_data_e.html allows search by name, reference id or geographical location.
However, since this only allows download by single month of data, I needed to find another method to more quickly get multiple years and months of data.
When you use the website search and follow the links to the download page you will also see a link to get more historical data. This link brings you to a Google Drive folder. This folder documents how to use their ‘wget’ method to download files. However, I was using Windows and didn’t want to mess around with Cygwin or Windows linux to be able to use wget.
It turned out to be relatively simple to replicate the wget process using Python Requests, then loop through the csv files to process them. Python io.StringIO was also used to stream request content into a ‘fake csv’ file in each loop, which were then aggregated into a Python List that was converted into a Pandas dataframe so Pandas could be used to process data.
The Python code is in my Github ‘Environment-Canada-Precip-and-Flow-Data’ Repository.