Tutorial: Collect Open Data from online sources

1. Introduction

2. Download Africa Groundwater Atlas Country Hydrogeology Maps

First we're going to download data on geology and hydrogeology for Malawi from the Africa Groundwater Atlas.

1. Go to the following website: https://www.bgs.ac.uk/africagroundwateratlas/downloadGIS.html

Here you can download ESRI Shapefiles with the hydrogeology (aquifer type and productivity) and geology (with particular relevance to hydrogeology) of 38 countries in Africa.

2. Click on Malawi.

3. Fill in the Download Form and click Submit Form.

4. After submitting the form you'll see a page where you can download the data directly.

You'll also receive an e-mail with the link.

5. Download and extract the zip file to your hard disk, e.g. Z:\.

Note that the files are zipped at the folder level. When you extract the file the folder Malawi will appear in the folder where you extract the file (e.g. Z:\Malawi). A good open source zip programme is 7-Zip.

In the next section we're going to open the data in QGIS.

3. Visualise Africa Groundwater Atlas Country Hydrogeology Maps

4. Search for layers on GeoNode

Malawi has a Spatial Data Infrastructure (SDI): the Malawi Spatial Data Platform (MASDAP), which is built on GeoNode.

In this section we're going to add data from MASDAP to our QGIS project.

1. Browse to the Malawi Spatial Data Platform

2. In the Search field type groundwater and press the <Enter> button.

Now you'll see this result.

3. Click on the layer name and inspect the metadata and attributes of the layer.

With the Download button you could download this data. In the next section however, we're going to connect to this GeoNode from QGIS and load the layer into our project.

5. Load a WFS layer from GeoNode

Instead of downloading the groundwater_monitoring_wells_Coordinates layer we're going to make a connection between QGIS and the GeoNode of MASDAP to load the layer as WFS in our QGIS project.

1. Go back to the QGIS project.

2. Click the Open Data Source Manager button in the toolbar.

3. Choose the GeoNode tab.

4. Click the New button to create a new service connection.

5. In the Create a New GeoNode Connection dialogue type MASDAP as Name and http://www.masdap.mw as URL.

6. Click Test Connection.

If the test is successful you'll see this popup:

If the connection fails, check your internet connection and the URL.

7. Click OK to close the popup.

8. Click OK in the dialogue to close it and the new connection is added.

9. Click the Connect button.

You'll now see the layers on the GeoNode listed.

10. Type groundwater at Filter.

It shows the groundwater_monitoring_wells_Coordinates layer as WMS and WFS Web Service.

WMS (Web Map Services) are rendered pictures of the data. That's useful for quick visualisation of the data. WFS (Web Feature Services) are the vector features that you can further use in analysis in GIS. Here we're going to use the WFS layer.

11. Click the WFS layer and click the Add button. Close the dialogue.

The project should now look like the figure below.

In the next section we're going to export the GeoNode WFS layer to a local GIS vector file.

6. Export WFS layer to vector format

The GeoNode layer fromt the previous section is still a WFS layer.

To use it locally it's recommended to export it to a GIS vector format. In this section we'll convert it to an ESRI Shapefile and change the projection from the Geographical Coordinate System (GCS, EPSG: 4326) to UTM Zone 36S / WGS-84 (EPSG: 32736).

1. Click right on the geonode:groundwater_monitoring_wells_Coordinates layer in the Layers panel and choose Export | Save Features As...

2. In the Save Vector Layer as... dialogue choose ESRI Shapefile as the output Format. Use the button to browse to the project folder and save the output layer as groundwater_monitoring_wells.shp.

3. Click the Select CRS button .

4. In the Coordinate Reference System Selector type 32736 at Filter and select the WGS 84 / UTM zone 36 S projection.

5. Click OK to return to the Save Vector Layer as... dialogue, which should look like the figure below.

6. Click OK to perform the layer export.

After processing the layer is added to the map canvas.

7. Remove the geonode:groundwater_monitoring_wells_Coordinates layer from the Layers panel.

The project should now look like the figure below.

We'll add more data in the next section.

7. Import a CSV file

In this section we're going to import a CSV file with Ramsar sites in Malawi.

Ramsar sites can be downloaded from the Ramsar Sites Information Service. It contains 2,403 Sites covering 254,307,159 ha.

1. Browse to the Ramsar Sites Information Service.

2. Under Explore by filters choose Africa.

3. After the map with Ramsar sites in Africa has loaded choose Malawi from the list under Explore by filters.

4. After the map of Malawi is loaded choose the Exports tab.

5. Choose the CSV file under Format, type your e-mail address and click Generate.

You'll receive immediately an e-mail from the Ramsar Sites Information Service with the link to download the CSV file.

6. Download the CSV file to the folder of your GIS project.

7. Open the CSV file in a text editor such as Notepad and check the contents.

You can see that the first line contains the column headers. The columns are separated by commas. Strings are in double quotes. Because it has columns for latitude and longitude we can load the CSV file in a GIS as a points layer with the other columns as attributes.

8. Go back to QGIS

9. Click the Open Data Source Manager button .

10. Go to the Delimited Text tab by clicking

11. In the Delimited Text dialogue use to browse to the CSV file.

• Replace the Layer name with Ramsar sites Malawi.
• Check that under Geometry Definition the Longitude column is selected for the X field and the Latitude column is selected for the Y field. Choose EPSG: 4326 for the Geometry CRS. This information is needed to plot the file on the map.
  

12. Check the preview under Sample Data. If everything looks okay click Add and Close the dialogue.

Now the Ramsar sites Malawi layer is added to the Layers panel. This layer, however, is not yet a GIS vector file, so we need to export it and reproject the data to the UTM Zone 36S / WGS 84 projection like we did for the GeoNode data.

13. Click right on the Ramsar sites Malawi layer in the Layers panel and choose Export | Save Features As...

14. In the Save Vector Layer as... dialogue choose ESRI Shapefile as the output Format. Use the button to browse to the project folder and save the output layer as ramsar_sites_malawi.shp.

15. Click the Select CRS button .

16. In the Coordinate Reference System Selector type 32736 at Filter and select the WGS 84 / UTM zone 36 S projection.

17. Click OK to return to the Save Vector Layer as... dialogue, which should look like the figure below.

18. Click OK to perform the layer export.

After processing the layer is added to the map canvas.

19. Remove the Ramsar sites Malawi layer from the Layers panel (make sure you remove the CSV file! When you hover your mouse over the layer name it will display the file name and the projection in a tool tip).

Let's style the  Ramsar sites, because they're a bit hidden.

20. Click on the ramsar_sites_malawi layer in the Layers panel and open the Layer Styling panel by clicking .

21. Click on Simple marker and change the Symbol layer type to SVG marker. Under SVG Groups choose symbol and look for the red marker

22. Change the width and height to 12 mm.

The project should now look like the figure below.

In the next section we're going to add more data.

8. Add transboundary aquifers from the SADC Groundwater Information Portal

The final layer for our groundwater project in Malawi is the transboundary aquifers dataset from the SADC-GIP.

1. Browse to https://sadc-gip.org/ and search for aquifers using the Search field.

2. Select the 2020 - Transboundary Aquifers of the World (unpublished) layer.

3. Check the metadata and attributes.

4. Go to back the QGIS project.

We're going to create a new GeoNode connection with SADC-GIP. You can do that in the same way as you've learned before in section 5. Here we'll present another method.

5. Go to the Browser panel.

At the bottom of the Browser panel you can see the GeoNode connections. If you expand the GeoNode group you can already find the MASDAP connection that we made in section 5.

Here you can also add a new connection.

6. Click right on GeoNode and choose New Connection...

7. In the Create a New GeoNode Connection dialogue type SADC-GIP as Name and https://sadc-gip.org as URL.

8. Click Test Connection.

If the test is successful you'll see this popup:

If the connection fails, check your internet connection and the URL.

9. Click OK to close the popup.

10. Click OK in the dialogue to close it and the new connection is added.

11. Expand the SADC-GIP connection and the WFS group:

12. Drag the 2020 - Transboundary Aquifers of the World (unpublished) layer to the map canvas.

Like we did before we need to export the layer to a local GIS file before we can use it further in GIS.

13. Use the same steps as in section 6 and save the layer as a shapefile with the name aquifers.shp and reprojected to the UTM Zone 36S / WGS-84 projection.

14. Remove the 2020 - Transboundary Aquifers of the World (unpublished) layer from the Layers panel.

Now your project should look like the figure below.

Now we have all the data ready it is good practice to save all the layers in a GeoPackag, including the project, styling and the correct projection. We'll do that in the next section.

9. Save data and styling to a GeoPackage

All the shapefiles that we've created in this project are not very easy to share with other colleagues working in the project. The best practice is to keep the data in an SDI, but if internet connections or access to SDI's are issue and you need to use the project offline it is important to wrap all data, styling and project information in one GeoPackage that you can share.

1. From the main menu choose Processing | Toolbox to open the Processing Toolbox panel.

2. In the Processing Toolbox go to Database | Package layers.

3. In the Package layers dialogue click to select the layers to add to the GeoPackage.

4. Click Select All and OK.

5. Keep the box Save layer styles into GeoPackage selected. In that way you can load individual layers to another project and keep the layer styling. Save the Destination GeoPackage as Malawi_GIS_data.gpkg.

6. Click Run. Click Close after processing.

10. Save an entire QGIS project in a GeoPackage

1. Click to open a blank project.

2. Go to the Browser panel, expand the Malawi_GIS_data.gpkg GeoPackage and drag the layers to the map canvas.

3. Zoom to Malawi (some layers are much larger) by clicking right on the Geology layer in the Layers panel and choose Zoom to Layer.

4. Check if the on the fly projection is EPSG: 32736 (UTM Zone 36S/WGS-84). You can see this on the bottom right of the QGIS window. If there's another EPSG code (e.g. 4326), then click on the EPSG code and choose the correct one.

We can now save the QGIS project in the same GeoPackage so when you share the GeoPackage file others can open the project with QGIS.

5. In the main menu choose Project | Save To | GeoPackage...

6. In the Save project to GeoPackage dialogue browse the the Malawi_GIS_data.gpkg GeoPackage that we've created before. Type Malawi_Groundwater as the Project name and click OK.

7. Go to the file explorer and copy the Malawi_GIS_data.gpkg to another folder e.g. Z:\Malawi2\ so we can test if the data and project are really portable.

8. In the main menu go to Project | Open From | GeoPackage...

9. In the Load project from GeoPackage dialogue use the button to browse to the new folder with the copy of the Malawi_GIS_data.gpkg file and choose the Malawi_Groundwater project. Click OK to open.

Now you'll see the project as we configured it and anyone who uses the GeoPackage file will see the same.

In the next tutorial we're going to further process this data in QGIS.