Tutorial: Field surveys using QGIS, Mergin and Input

Site: OpenCourseWare for GIS
Course: Field surveys with QGIS, Mergin and Input
Book: Tutorial: Field surveys using QGIS, Mergin and Input
Printed by: Guest user
Date: Monday, 2 December 2024, 11:13 AM

1. Introduction

Input is a simple survey app allowing users to capture data in field. Forms and data preparation can be done in QGIS software and synchronised with the Input app using the Mergin plugin and repository.

Input is not aimed to be a full GIS/mapping application. It is designed with simplicity, ease of use and seamless data synchronisation in mind.

A typical workflow for using Input consists of the following steps:

  • Preparing project: user loads background and survey layers, set up the forms, apply styling to the layers, set up map themes and define the layers to be used for survey purpose.

  • Data/project transfer: once the project set up, users need to transfer data to the device. This can be achieved through Mergin.

  • Working with Input: map navigation, data editing, including filling the forms, viewing existing data and uploading the changes back to Mergin.

In this tutorial the workflow will be demonstrated for field data collection for crop mapping using remote sensing.

2. Project preparation

We will start with preparing the project in QGIS.

In the project we have to distinguish:

  • Background layers that provide context in the field

    We can use both offline and online layers and they can be raster or vector.

  • Survey layers

Our project will have the following background layers:

  • Sentinel 2 true color (RGB = bands 2, 3, 4)
  • Sentinel 2 false colour (RGB = bands 8, 4, 3)
  • OpenStreetMap online
  • OpenStreetMap offline
  • Vector layer with boundary of the study area
We'll create one survey layer for crop mapping.

The Sentinel 2 subset for this exercise can be downloaded from here.

2.1. Adding a raster background layer

In this section we'll add a Sentinel 2 True Colour image and repeat the steps for a Sentinel 2 False Colour image.

1. Open QGIS Desktop.
2. Go to the Browser panel.
3. Look for the Sentinel 2 subset that you've downloaded for this tutorial and drag it to the Map Canvas


The provided Sentinel 2 subset contains:

  • Band 2 (Blue)
  • Band 3 (Green)
  • Band 4 (Red)
  • Band 8 (Near Infrared)
In the file they are stored as band 1 - 4. To create a True Colour composite we need to use R = 3, G = 2 and B = 1.

Style the layer

4. Go to the Layers panel.
5. Click on Open Layer Styling Panel button to open the Layer Styling panel.
6. In the Multiband color renderer choose Band 3 for the Red band, Band 2 for the Green band and Band 1 for the Blue band.

Layer styling panel settings for true colour
For this image the default settings are okay. This dialogue however, allows to stretch the colour. You can change the Min/Max settings and calculate it for the whole raster or the map canvas to increase the contrast. In this case we'll keep the defaults.

Create tiles

For more efficient use of the resources at your mobile phone you can best convert this GeoTiff to XYZ tiles in an .mbtiles file.

7. Go to the Processing Toolbox (if you don't see the Processing Toolbox panel, you can activiate it: in the Main Menu choose Processing | Toolbox) and double click Raster tools | Generate XYZ tiles (MBTiles).

8. In the dialogue that opens click Browse button and choose Use Layer Extent. Then choose the sentinel image.
9. Type for Minimum and Maximum Zoom 15. This gives the best compromise for this case. In other cases you can experiment with it. If you make the zoom level too high, the file will be very large. If you make it too small, you'll loose detail. If you give a range, the file will be bigger, but the performance for visualisation will be better.
10. Keep the rest at default and save the file in your project folder as: Sentinel_True_Colour.mbtiles.
11. Click Run.

12. Click Close when the calculation is finished.
13. Go to the Browser panel and drag the Sentinel_True_Colour.mbtiles raster layer to the Map Canvas.
14. Zoom in and compare the result with the original. In this case the result is okay. If not, you repeat the steps for other zoom levels to choose the best one.
15. Repeat the steps to create a Sentinel_False_Colour.mbtiles raster with bands R=4, G=3 and B=2.
16. Remove the S2_20190330_stack2348_subset layer from the Layers panel.
17. Save the project.

The procedure presented in this section is applicable to all offline raster layers that you want to use in your survey project.
Add raster to your survey project, the final result


2.2. Adding an online background layer

If you have internet connectivity in the field, you can use a WM(T)S or online XYZ tiles as your background map. In this section we'll add an online layer from OpenStreetMap to our survey project. We'll also make the online layer offline available just in case we don't have internet connectivity in the field.

We'll continue with the project from the previous section.

1. From the Browser panel, expand the XYZ tiles and double-click on OpenStreetMap

Adding OpenStreetMap as XYZ Tiles in QGIS

The OpenStreetMap layer now shows up in your Layers panel and is shown in the Map Canvas. 

QGIS offers other ways to add online maps: if you open the Data Source Manager with this button Open Data Source Manager buttonyou can choose to add WM(T)S layers.  A useful plugin for using online maps is the QuickMapServices plugin. In this tutorial we only use the OpenStreetMap layer.

Make online layers offline available

In the case that we don't have internet connectivity in the field but still want to use the online map, we need to export it to a .mbtiles file like we did in the previous section.

2. First change the on-the-fly reprojection to EPSG:3857. You can do this by clicking right on one of the layers and selecting Set CRS | Set Project CRS from layer. This is necessary, because the XYZ Tiles are using Web Mercator CRS (EPSG:3857).

Set Project SRS from layer

3. Go to Processing Toolbox | Raster tools | Generate XYZ tiles (MBTiles).

4. Choose for the Layer extent one of the Sentinel layers from the previous section. Then the tiles will only be generated for that area.

5. Choose again 15 for the Zoom levels, keep the rest at default and save the layer as OSM_Standard.mbtiles.

6. Click Run and Close when the calculation is finished.


7. Drag the OSM_Standard.mbtiles raster from the Browser panel to the Map Canvas and compare with the online OSM Standard layer.

8. To avoid confusion, rename the online layer to OSM Online and the offline layer to OSM Offline. You can do that by clicking right on the layer name and chosing Rename...

9. Save the project.

Now we have offline and online rasters in our survey project. In the next section we'll add a background vector layer.



2.3. Adding a vector background layer

If you are using a vector layer as a background layer:

  • Ensure to use a simplified geometry version of your data. This will help with smooth map navigation.

  • All vector layers not intended to be used as survey layers, have to be set as read-only. 

In this section we'll create a GeoPackage with a layer containing a polygon with the boundary of our study area. You could also create a Shapefile in a similar way.

We'll continue with the project from the previous section.

1. Make sure the True Color Sentinel image is visible and zoom to the extent.

2. Change the on-the-fly reprojection to EPSG:32632 that we're using in this area. Click on the EPSG code on the lower right of the screen and choose the right projection.

3. In the Main Menu go to Layer | Create Layer | New GeoPackage Layer....

Create new geopackage layer

4. Click the Browse button Browse button to go to your project folder and type a new name for a GeoPackage database, e.g. survey.gpkg.

5. For Table name type Boundary. For Geometry type choose Multipolygon. Make sure the projection is set to EPSG:32632.

6. If you need attributes, you can add it here. A GeoPackage automatically adds a feature id to the attribute table, so you can also leave this open if you don't need attributes. In the next section we'll create the attributes for the survey layer.

7. Click OK to return to the Map Canvas.

Now an empty polygon vector layer with the name Boundary has been added to the Layers panel.

8. To start digitizing make sure the Boundary layer is on top and selected. Click the toggle editing button Toggle editing button in the toolbar.

9. Click the Add polygon feature button Add polygon feature buttonand draw a polygon in the Map Canvas to indicate the border of your study area.

10. Click right to close the polygon. A popup screen will ask to fill in the attributes. We keep the default and click OK.

11. Click the toggle editing button  to stop editing and save the edits.

Now we need to style the polygon in such way that we can only see the border.

12. Click the Open Layer Styling Panel buttonbutton to open the Layer Styling panel.

13. Click Simple fill and click the drop down arrow at Fill. Choose Transparent fill.

Set transparent fill

14. Choose for Stroke color red and a Stroke width of 1 mm.

Boundary polygon styling result

Because this isn't a survey layer, we need to make this vector layer read-only.

15. In the Main Menu select Project | Properties...

16. Select the Data Sources tab.

17. Check the box for Read-only for the Boundary layer.

Set vector to read-only

18. Click OK.

Now all our raster and vector background layers are ready for this survey project.
In the next section we'll add the survey layer.

2.4. Adding a survey layer

Vector layers can be added as survey layer for the Input app.

In this section we'll create a survey layer for collecting ground truth for crop cover classification with remote sensing.

The following steps will be covered:

  • Create a point vector layer with the necessary attributes
  • Styling of the points
  • Make an attributes form using widgets
  • Settings for the Input preview panel
We continue with the same project from the previous section.

Create a point vector layer with the necessary attributes

1. In the Main Menu go to Layer | Create Layer | New GeoPackage Layer....

Create new geopackage layer

2. Click the Browse button Browse button to go to your project folder and choose the GeoPackage from the previous section, e.g. survey.gpkg.

3. For Table name type Groundtruth. For Geometry type choose Point. Make sure the projection is set to EPSG:32632.

4. Now we're going to add the attributes. A unique feature ID is automatically generated in a GeoPackage, so we don't have to add the observation ID attribute. Add the attributes like in the screenshot below. Add them by using the Add to list button.

Create Geopackage survey layer

5. Click OK when done.

6. A popup will ask you if you want to overwrite or add a new layer. Choose Add New Layer so it will be added to the GeoPackage.

Add new layer

Styling of the survey points

The Input app will use the same styling as defined in QGIS. Let's create a point symbology.
7. Click Open Layer Styling Panel buttonto open the Layer Styling panel.
8. Use the Single symbol renderer and click Add symbol layerto add a symbol layer.
9. For the first symbol layer make the Fill color transparent and the Stroke color black. Increase the Size to 4 mm and Stroke width to 1 mm For the second symbol layer choose a cross as marker symbol and choose Stroke color black. Increase the Size to 8 mm and Stroke width to 1 mm.
survey point marker

Make an attribute form using widgets

The next step is to design the attribute form using widgets.
10. Click right on the Groundtruth layer and choose Properties...
11. In the Properties window go to the Attributes Form tab.

Here we can design the form. We'll use the default Autogenerate option. Under fields we see all attributes in the Groundtruth layer.

12. Click on fid. On the right side of the window we can configure the widget. The value of fid is assigned automatically by QGIS and does not need to be edited by user. Set the Widget type to hidden to make it invisible in the form.
No widget for fid

13. Click on Observation number.  Choose the Text Edit widget and check the Not null box under Constraints. This means that it is a mandatory field. Keep the other fields as default.
Observation number widget
14. Click on Observer. At Alias type Observer name. Choose the Text Edit widget and check the Not null box under Constraints. Keep the other fields as default.
15. Click on Date and Time. We don't need an alias. Widget Type is automatically set to Date/Time. Change Widget Display to Custom and make it dd-MM-yyyy HH:mm:ss. Don't check the box before Calendar popup. Under Defaults type as default value $now to have the current date/time automatically filled in.

Date time widget

16. Click Picture. For Widget Type choose Attachment. Check the box for Relative paths. Under Integrated document viewer choose Image. At Constraints check the box before Not null.
17. Click Land Cover. For Alias type Land Cover Class. For Widget Type choose Value Map. With value maps we can create drop down menus in the Input app. Fill in the table as follows:


Value Description
1 Crop land
2 Tree cover
3 Shrubland
4 Grassland
5 Wetland
6 Artificial
7 Bare soil
8 Fallow land
9 Seasonal water
10 Permanent water

18. Set Constraints to Not null. And put the default value on 1, because for the purpose of crop mapping this will be chosen mostly.
19. Click Crop type.
20. For Widget Type choose Value Map. Use the following table:


Value Description
1 Wheat
2 Maize
3 Rice
4 Barley
5 Soybeens
6 Pulses
7 Cotton
8 Potatoes
9 Alfalfa
10 Sorghum

21. At Constraints write the following Expression"Land Cover"  = 1. This means that this can only be filled in when the Land Cover equals 1. By checking the box Enforce expression constraint you can prevent users to override the restriction.
22. For the field Water Source the Widget Type is Value Map. Fill in the dialogue as the screenshot below.
Water source widget
23. For Irrigation method fill in the dialogue as the screenshot below:
Irrigation method widget

24. For Coverage use the following screenshot:
coverage widget
25. For Size use the screenshot below:
Field size widget
26. For the Notes use this:
Notes widget

27. Click OK.
28. Open the attribute table of the Groundtruth layer and switch to Form View using Form View button to check the widgets.

Settings for the Input preview panel

Similar to Google maps, when a feature is identified on the map a preview panel will appear. To enable a layer being identified, the option should be enabled in QGIS desktop, under Project properties.

29. For setting the preview panel, you need to click right on the Groundtruth layer and choose Properties. Then select the Display tab.

  • title - this will use "Display Name" which can be a field name or an expression.

  • content - there are three options for what to show in content area: 1. field values, 2. image, 3. HTML. This will use "HTML Map Tip". QGIS always interprets the content of map tip as being HTML, however Input extends the syntax to allow two more modes. If the map tip is not specified, Input will try to use the first three fields and show their attribute values.

Field values

Sample map tip content that will show "description" and "time" field values:

# fields
fid
date and time

If the map tip content has # fields marker on the first line, the following lines will be understood as field names that should be listed in the preview. At most three fields will be shown. Expressions are not allowed.

Image

Sample map tip content that will cause an image to be show specified by file path in field "image_1" (containing path relative to the project directory):

# image
file:///[%@project_folder%]/[% "Picture" %]

If the map tip has # image marker on the first line, the following line is understood as the URL for the image. It can be a regular file on the file system, but it could be even a remote image from network. Expressions embedded in the image URL will be evaluated (enclosed in [% 1+1 %]).

HTML

Sample map tip content that will show render as HTML page:

<p><strong>Notes:</strong>[% "notes" %]</p>

If the map tip does not contain any special marker, it is assumed that the map tip is HTML content. Only a limited subset of HTML is supported - see https://doc.qt.io/qt-5/richtext-html-subset.html


2.5. Creating map themes

To facilitate the changing of background maps, we can configure map themes in our QGIS project.

With map themes you can show only visible layers that are stored in a theme.

All our themes need the survey layer (Groundtruth) on top, followed by the Boundary layer.

We'll make map themes to show them on:

  • Sentinel True Colour
  • Sentinel False Colour
  • OSM online
  • OSM offline
Let's start with the first one:
1. In the Layers panel organise the layers in such a way that Groundtruth is on top, followed by Boundary and then Sentinel_True_Colour.
2. Make sure that these layers are visible by checking the boxes and the others are not visible by unchecking the boxes.
Preparing the Sentinel True Color map theme
3. Click the Manage Map Themes button Manage map themes button and choose Add theme...
Add theme
4. Save the theme as Sentinel True Colour and click OK .
Save as Sentinel True Colour theme
5. Repeat the steps for Sentinel_False_Colour. Make sure the layers are checked like this:
Save as Sentinel False Colour theme
6. Do the same for OSM offline and OSM online.
7. If you now go to the Manage Map Themes button Manage map themes button you can see all themes and switch easily between the themes. Try it.
Map themes
8. Save the project.

These themes can later be chosen in the Input app.


3. Synchronizing your project with Mergin

Input comes with built-in functionality to seamlessly synchronise your projects and data through Mergin.

In this section we'll synchronize the project from the previouse sections with the Mergin cloud service.

1. Go to your internet browser and sign up to Mergin (https://public.cloudmergin.com/).

2. In QGIS install the Mergin plugin.

3. In the Browser panel you'll now find a Mergin folder.

Note 1: that there is no menu entry or toolbar icons for Mergin plugin. The only method to interact with the service is through the QGIS Browser panel.

Note 2: You may need to restart your QGIS to see Mergin in your QGIS browser panel after the installation.

To be able to view Mergin projects, we need to sign in:

4. In the browser panel, right-click on Mergin.
5. Select Configure.

Configure Mergin Plugin

6. A new window will appear:
  • For URL: type https://public.cloudmergin.com
  • For Username: type your Mergin username
  • For Password: type your Mergin password
  • Click Test Connection and it should show OK.
7. Click OK.


To view the list of your projects, click on the arrow to the right of Mergin in your QGIS browser panel.

Using Mergin plugin

The following functionalities are available from Mergin plugin:

Create a new project

8. You can start a new project by right-clicking on Mergin from the browser panel and selecting Create new project:

  • Type a name for Project name
  • Selecting Public will make your project available to all Mergin users
  • Project Directory will be the folder where your project and associated layers reside (the project that we made in the previous sections.
Create Mergin Project Dialogue

9. Click OK.

10. A popup window will appear that shows that the project was successfully created. Click Close.

Mergin project created successfully
11. You can find the project now under Mergin | My projects.

Mergin My Projects

12. Check the result on the Mergin website (https://public.cloudmergin.com/). Your project and its files should be listed there now.

Your project in Mergin

Now your project is synchronised with the Mergin cloud service and can be synchronised with the Input app on a smartphone or tablet, which will be explained in the section 4.


The following subsections are for reference for working with the Mergin plugin.


Download a project

To download a project:

  • Right-click on the project under Mergin, from the browser panel
  • Select Download
  • A new window will appear to set the folder path
  • Once the project is downloaded, you can select to Open the downloaded project.

Synchronise the changes

With Mergin service you can synchronise your local changes back to the server. Simply, right-click on the project from the browser panel and select Synchronize

You can also use Synchronize function to download changes made to your local projects from Mergin.

Remove a local/downloaded project

If you no longer want to have the project and its associated files available locally, you can delete them by right-clicking on the project from browser panel and select Remove locally.

Ensure to use this function to remove the projects. Deleting the files manually might cause synchronisation problems.

Remove project from server

This will remove your project from Mergin server. You need to first remove the project locally, to be able to delete it from the server.


You can also drag-and-drop your project folder (including layers, SVGs, etc) to Mergin project webpage.


4. Capturing data and editing forms using Input

Accessing projects

1. Install the Input app from Google Play Store.

Once you open the application for the first time, it will take you to Home screen. This screen displays all the projects stored locally (under /sdcard/INPUT folder).

Input Home

2. To access your projects on the Mergin cloud, select My projects. You will need your log in details to sign in to Mergin:

Login Input

Once logged in, you can download your project to work on it locally by pressing download button to the right of the project.

3. Download the survey project from the previous sections by tapping the cloud with the downward arrow.

Input My Projects

The project will be available for loading in Input after you download it locally.

4. Select Home and you should be able to see your downloaded project:

Input homescreen

5. Tap the project under to open it in Input.

You'll now see one of the map themes.

Input project

Change background layers

We can change the background layers to be displayed using the Map themes that we created in QGIS.
6. Tap the More button Input more button.
7. Choose Map themes.
Input_more_menu

8. Choose Sentinel True Colour
Input map_themes
Input will now show the Sentinel True Colour image as a background map with the study area boundary on top.

9. Tap at the border of the Boundary layer polygon. The attributes will be shown.

Adding/editing features

Now we're going to add a feature to our survey layer (Groundtruth).

10.  To add a feature, tap the Record button Input record button from the lower panel. When you tap on the record button, it will list all the layers, you can edit. In our case that's only the Groundtruth layer.

Input survey layer

In Input, you have two modes to capture feature:

  • GPS location or streaming (if you are capturing lines/areas)

  • Free hand: by adding points or nodes to lines or areas

Capturing points

Our survey layer is a point layer, it will show a cross-hair at your GPS location.

11. Move the map to the parcel that you are observing and tap Add Point Input Add point button. The point below the cross-hair will be used as the recorded location. You can move back to your location if needed, by tapping GPS Input GPS button.​ If needed you can switch to other Map themes to better orient yourself.

Input Add point to parcel

12. Fill in the form. To fill in the current date/time automatically, tap the calendar button Input Calendar button. The fields that are depending on choices are displayed in red.
Input form

13. Tap Save Input save point button to save the form data to the attribute table of the survey layer.

Edit captured points

You might still want to correct  something in the form while you're in the field.
14. Tap the point you've just created. A preview will show up as defined in QGIS.

Input point preview panel
15. Tap the edit button Input edit button and edit the form. You can also tap Edit geometry Input Edit geometry button to change the location of the point. Tap Save Input save point button to save the updates. If you want to delete the item, you can press the recycle bin Input recycle bin.

Synchronisation of data/ projects


You can upload your changes through the Mergin service, once you have network connectivity.
16. Tap Projects Input projects buttonfrom the lower panel.
17. Tap My projects Input My Projects button
Now you see Input Sync buttonnext to your project.
Input automatically flag schanged projects with a synchronise icon.
18. Tap Input Sync button  to synchronize your edits with Mergin.
Input Synch my project

Similarly, if a project and its data has been updated through the Mergin service, you will be notified to synchronise the changes to your local copy.

Download the data into QGIS

Now we want to further process the data in QGIS.
With the Mergin plugin we can syncronize the project in QGIS.
19. Right-click on the project under Mergin, from the browser panel
20. Select Download
21. A new window will appear to set the folder path. Select the project folder of the original project.
22. Once the project is downloaded, select Open the downloaded project.

23. Check the results using the Form view of the attribute table.
QGIS Form view of the attribute table

Now you've completed this tutorial.
Below some instructions for capturing lines or areas.

Capturing lines or areas

If your survey layer is a line or area, the panel at the bottom changes to a figure similar to the one below:

You can:

  • Use Add point Input Add point button to define the shape of your line or area manually by navigating through the map

  • If you want to stream the GPS to shape your line or area, press and hold GPS button Input GPS button on the bottom left corner.

Once finished, press Done. If your feature is an area, it will join the last point to the first point to close the shape.