Tutorial: Create a field data collection app for groundwater studies

1. Introduction

In the previous topic we have learned how to use data that is available on the internet.

Often, however, you have to collect your own data, because it's not available or too coarse in the online sources. Nowadays you would use an app for that.

After this tutorial you'll be able to:

• use QGIS to setup a surveying project
• configure map themes with online and offline layers
  
• design the field form
• synchronise the QGIS project with the Mergin cloud service
• synchronise the Input app on the mobile phone with the Mergin cloud service
• use the Input app for surveying
• synchronise the data collected in the field with the Mergin cloud service and the QGIS project

2. Project preparation

We will start with preparing the project in QGIS.

In the project we have to distinguish:

• Background layers that provide context in the field

  We can use both offline and online layers and they can be raster or vector.
  

• Survey layers

Our project will have the following background layers:

• Google satellite online
• Google satellite offline
  
• OpenStreetMap online
• OpenStreetMap offline
• Vector layer with boundary of the study area
  

2.1. Adding the study area boundary

First we're going to define the study area boundary and add that polygon to the project, so we also know where the border is when we're navigating with the app in the field.

1. Start QGIS with a blank project

2. Change the projection of the project to UTM Zone 36S/WGS-84 by clicking on the EPSG code in the lower right corner .

3. In the dialogue filter on the EPSG code 32736, select the projection and click OK.

13. Click the button to open the GeoCoding plugin.

21. In the Create layer from extent dialogue click and choose Use Canvas Extent.

24. Select the Bangula_study_area layer from the Layers panel and click to open the Layer Styling panel.

2.2. Add online layers and make available offline

If you have internet connectivity in the field, you can use a WM(T)S or online XYZ tiles as your background map. In the previous section we've already added the OSM Standard layer. In this section we'll also add Google Satellite to our survey project. We'll also make the online layers offline available just in case we don't have internet connectivity in the field.

Let's first add Google Satellite to the project.

1. In the main menu go to Web | QuickMapServices | Google | Google Satellite.

Now we can clearly see the pivots.

The next step is to make the OSM Standard and Google Satellite layers offline available for the study area. For that purpose we need to save the map canvas to MBTiles at a specified zoom level. First we're going to find the appropriate zoom level.

2. In the main menu choose View | Panels | Tile Scale.

The Tile Scale panel is now added.

3. Click on Google Satellite in the Layers panel and move the slider in the Tile Scale panel to the desired zoom level.

Note that the higher the level, the higher the resolution and the larger the file. Here we'll use zoom level 17.

4. Close the Tile Scale panel.

5. Click right on Bangula_study_area layer in the Layers panel and choose Zoom to layer.

6. In the Layers panel uncheck the boxes of all layers except Google Satellite.

7. Go to the Processing Toolbox.

8. Choose Raster tools | Generate XYZ tiles (MBTiles)

9. In the Generate XYZ tiles (MBTiles) dialogue choose for Extent Use Layer Extent...

10. Choose Bangula_study_area and click OK.

11. Set the minimum and maximum zoom level to 17.

If you give a range here, it will create MBTiles for different zoom levels. This will make the file bigger, but navigation smoother when you zoom in and out a lot. Most often in the field you only want the detailed level. Therefore here we choose just zoom level 17.

12. Save the layer to the dedicated folder with project files as Satellite.mbtiles. Leave other settings at default.

Note that the mbtiles can't be saved directly to a GeoPackage. We'll later drag it to the GeoPackage.

13. Click Run. Click Close after processing.

14. In the Browser panel drag the Satellite.mbtiles raster to the Bangula_groundwater_survey GeoPackage.

15. Drag the Satellite layer from the Bangula_groundwater_survey GeoPackage to the map canvas.

16. Repeat the steps for OSM Standard.

17. Save the project.

Now the online and offline layers are ready.

In the next section we're going to add the survey layer.

2.3. Adding a survey layer

Vector layers can be added as survey layer for the Input app.

In this section we'll create a survey layer for collecting groundwater data.

The following steps will be covered:

• Create a point vector layer with the necessary attributes
• Styling of the points
  
• Make an attributes form using widgets
• Settings for the Input preview panel

We continue with the same project from the previous section.

Create a point vector layer with the necessary attributes

1. In the Main Menu go to Layer | Create Layer | New GeoPackage Layer....

2. Click the Browse button to go to your project folder and choose the GeoPackage from the previous section, in our case bangula_study_area.gpkg.

3. For Table name type Survey. For Geometry type choose Point. Make sure the projection is set to EPSG: 32736.

4. Now we're going to add the attributes. Add the attributes like in the screenshot below. Add them by using the Add to Fields List button.

5. Click OK when done.

6. A popup will ask you if you want to overwrite or add a new layer. Choose Add New Layer so it will be added to the GeoPackage.

Styling of the survey points

The Input app will use the same styling as defined in QGIS. Let's create a point symbology.

7. Click to open the Layer Styling panel.

8. Use the Single symbol renderer and click to add a symbol layer.

9. For the first symbol layer make the Fill color transparent and the Stroke color black. Increase the Size to 4 mm and Stroke width to 1 mm For the second symbol layer choose a cross as marker symbol and choose Stroke color black. Increase the Size to 8 mm and Stroke width to 1 mm.

Make an attribute form using widgets

The next step is to design the attribute form using widgets.

10. Click right on the Survey layer and choose Properties...

11. In the Properties window go to the Attributes Form tab.

Here we can design the form. We'll use the default Autogenerate option. Under fields we see all attributes in the Survey layer.

12. Click on fid. On the right side of the window we can configure the widget. The value of fid is assigned automatically by QGIS to GeoPackage layers and does not need to be edited by user. Set the Widget type to Hidden to make it invisible in the form.

13. Click on Observation ID.  Type for Alias Observation number. This is how it will appear on the form. Choose the Text Edit widget and check the Not null box  and Enforce not null constraint under Constraints. This means that it is a mandatory field. Keep the other fields as default.

14. Click on Date. Type Date and time as Alias. Widget Type is automatically set to Date/Time. Change Widget Display to Custom and make it dd-MM-yyyy HH:mm:ss. Check the preview to see how it is formatted. Uncheck the box before Calendar popup. Under Defaults type as default value $now to have the current date/time automatically filled in.

15. Click on Observer. At Alias type Observer name. Choose the Text Edit widget and check the Not null box and Enforce not null constraint under Constraints. If you have a default observer you can type his/her name in single quotes under Defaults at Default value. Keep the other fields as default.

16. Click Picture. For Widget Type choose Attachment. Check the box for Relative paths. Under Integrated document viewer choose Image. At Constraints check the box before Not null.

17. The fields X coordinate and Y coordinate we can keep as default with the Text edit widget and no constraint.

18. Click Z. Set the Alias to Elevation. Keep the Text edit widget and no constraint.

19. For Type give the Alias Feature type. Set the Widget type to Value Map. Fill in the Value Map as the figure below. The user will get these items in a dropdown list and can choose from what is written in the Description column. The corresponding Value will be written to the attribute table. Under Constraints check Not Null and Enforce not null constraint.

20. For Depth type as Alias Depth to bottom (m). Keep Widget type as Text edit. We only want this field to be filled in for wells or boreholes. Therefore we add an Expression under Constraints:
  "Type"  = 'Wells' OR "Type" = 'Boreholes'.
Check the box to Enforce expression constraint.

21. For GW level type the Alias Water Level (m). Keep Widget type as Text edit. We only want this field to be filled in for wells or boreholes. Therefore we add an Expression under Constraints:
  "Type"  = 'Wells' OR "Type" = 'Boreholes'
Check the box to Enforce expression constraint.

22. For Name keep the default Text edit widget and no constraint.

23. For Address keep the widget as Text edit but check the Multiline box to allow for multiple lines.

24. For Remarks keep the widget as Text edit but check the Multiline box to allow for multiple lines.

25. Click OK to close the window and apply the changes.

26. Open the attribute table of the Survey layer and switch to Form View using to check the widgets.

Settings for the Input preview panel

Similar to Google maps, when a feature is identified on the map a preview panel will appear. To enable a layer being identified, the option should be enabled in QGIS desktop, under Project properties.

27. For setting the preview panel, you need to click right on the Survey layer in the Layers panel and choose Properties. Then select the Display tab.

28. Change Display Name to Observation ID. This will show the Observation ID when you tap a surveyed point.

29. Add the following lines under HTML Map Tip:

# image
file:///[%@project_folder%]/[% "Picture" %]

This will be interpreted by the Input app to show the picture when the surveyed point is tapped.

30. Click OK to close the dialogue.

Our Survey layer is ready now.

In the next section we'll do some final settings of the project.

2.4. Adding Map Themes

To facilitate the changing of background maps, we can configure map themes in our QGIS project.

With map themes you can show only visible layers that are stored in a theme.

All our themes need the survey layer (Survey) on top, followed by the Bangula_study_area layer.

We'll make map themes to show them on:

• Google Satellite online
  
• Google Satellite offline
  
• OSM online
• OSM offline

3. Click the Manage Map Themes button and choose Add theme...

3. Synchronizing your project with Mergin

Input comes with built-in functionality to seamlessly synchronise your projects and data through Mergin.

In this section we'll synchronize the project from the previouse sections with the Mergin cloud service.

1. In your file explorer go to the project folder and remove every file except:

Remember that we created mbtiles and other intermediate layers. Everything should be now in the GeoPackage, so you can safely remove these files. Note that a GeoPackage is one file, but here we have 3 files. The 2 other files are only there when you have you QGIS project open. Once you close it they'll be gone. Mergin will not synchronise these temporary files.

2. Go to your internet browser and sign up to Mergin (https://public.cloudmergin.com/).

3. In QGIS install the Mergin plugin.

4. Restart QGIS and load the project.

5. In the Browser panel you'll now find a Mergin folder.

6. Click right on Mergin and choose Configure.

7. A new window will appear:

• For Username: type your Mergin username
• For Password: type your Mergin password
• Click Test Connection and it should show OK.

8. Click OK to return to the Browser panel.

Now you can see under Mergin:

• My projects. If you expand you can see your own projects on Mergin.
• Shared with me. Here you find the projects that others have shared with you.
• Explore. Here you can explore public projects on Mergin.
  

Now let's add our project.

9. Click right on Mergin and choose Create new project.

11. Click OK.

Now your project will be synchronized with Mergin.

When the synchronisation is completed successfully you'll see this popup.

12. Click Close.

Now you'll find the project under My projects.

means that it's also a local project.

means that the project is available in the cloud but not locally.

The project is now also visible in the web browser:

If you click on the project you'll see it only contains the GeoPackage. Note that for synchronisation to you mobile phone in the next step you need the space that is mentioned, here 30.66 MB.

Now we're ready to synchronize the project with the Input app on the mobile phone. We'll eplain that in the next section.

4. Using Input for field surveys

1. Install the Input app from Google Play Store or Apple App Store on your mobile phone or tablet.

Once you open the application for the first time, it will take you to Home screen. This screen displays all the projects stored locally (under /sdcard/INPUT folder).

​

2. To access your projects on the Mergin cloud, select My projects. You will need your log in details to sign in to Mergin:

​

Once logged in, you can download your project to work on it locally by pressing download button to the right of the project.

3. Download the survey project from the previous sections by tapping .

You'll see the progress of the synchronisation.

When the synchronisation is finished you'll see:

The project will be available for loading in Input after you download it locally.

4. Select Home and you should be able to see your downloaded project:

5. Tap the project to open it in Input.

You'll now see one of the map themes.

6. Tap the More button .

10.  To add a feature, tap the Record button  from the lower panel. When you tap on the record button, it will list all the layers, you can edit. In our case that's only the Survey layer.

Our survey layer is a point layer, it will show a cross-hair at your GPS location.

11. Move the map to the location that you are observing and tap Add Point . The point below the cross-hair will be used as the recorded location. You can move back to your location if needed, by tapping GPS .​ If needed you can switch to other Map themes to better orient yourself.

12. Fill in the form. To fill in the current date/time automatically, tap the calendar button . The fields that are depending on choices are displayed in red.

13. Tap Save to save the form data to the attribute table of the survey layer.

You might still want to correct  something in the form while you're in the field.

14. Tap the point you've just created. A preview will show up as defined in QGIS.

15. Tap the edit button and edit the form. You can also tap Edit geometry to change the location of the point. Tap Save to save the updates. If you want to delete the item, you can press the recycle bin .

16. Tap Projects from the lower panel.

17. Tap My projects

Now you see next to your project.

18. Tap   to synchronize your edits with Mergin.

5. Synchronize field survey to QGIS

After the field data collection we can synchronize the project to QGIS through the Mergin plugin.

1. Start QGIS

2. Click right on the project under My projects and choose Synchronize.

When synchronisation is done successfully you'll see this popup.

3. Click Close.

4. Now click right again on the project and choose Open QGIS project.

Now you'll see the added point in your project.

5. Click right on the Survey layer and choose Open Attribute Table. Make sure you're in the form view.

Now you see the form with the point that we entered in the field, including the picture.

6. More resources

More resources on using QGIS, Input and Mergin can be found here.