Construction of a Point Cloud Data Set, True Orthomosaic, and Digital Surface Model Using Pix4D Software
Introduction: Basic Overview of Software
What is Pix4D?
Pix4D is a drone mapping and photogrammetry company that specializes in taking data from a sensor package and merging it together so that the user may perform data analytics on it. The company has multiple software packages however the most common and the one used for this lab is Pix4DMapper.
What is the overlap needed for Pix4D to process imagery?
The amount of overlap needed for Pix4D to process the imagery depends on what is being imaged. This is documented in the Pix4D manual under the ‘Ideal Image Acquisition Plan.’ There are several different cases listed including: a general forest and dense vegetation, flat vegetation with agriculture fields, building reconstruction, special cases, and many more. Below table 1 shows the cases mentioned above, where they should be applied and, what their overlap and sidelap should be.
 |
| Table 1: Necessary Overlap Cases |
What if the user is flying over sand/snow, or uniform fields?
Using the special cases case from table 1, feature-deficient areas should be flown as high as practical with 85% overlap, and 70% sidelap should be used.
What is Rapid Check?
Rapid Check is a means of quickly checking the quality of the data while in the field by using one of the ‘Rapid/LowRes’ processing templates to do an abbreviated initial processing step to get an indication of the quality of the data. This step generates a quality report for the user to check the number of calibrated/enabled images, and analyze the quality of the data.
Can Pix4D process multiple flights? What does the pilot need to maintain if so?
Yes, Pix4D can process multiple flights provided the flights overlap enough that Pix4D can capture enough keypoints to merge the two and, the flights are conducted under the same conditions such as, sun angle, weather conditions, and altitude.
Can Pix4D process oblique images? What type of data do you need if so?
Yes, Pix4D can process oblique images and create 3D models, 3D ‘Maps’, or orthomosaics; however, to create these, multiple flights should be made with multiple altitudes and camera angles. The Pix4D manual recommends flying around the object with the camera at 45° taking images every 5-10° around the object, then ascending to a higher altitude not over twice the original altitude, and lastly repeating the flight but with the camera angle of 30°. It also recommends having GCPs if the user wishes to combine the data with nadir data.
What is the difference between a global and linear rolling Shutter?
A global shutter captures all parts of an image at the same time. A rolling shutter is exposed in a progressive motion which means that certain parts of the image at the top will be exposed first and will be older in time than other parts of the image further towards the bottom. This can lead to some interesting distortions as shown in figure 1.
 |
| Figure 1: Global Shutter vs. Rolling Shutter (Image sourced from Wikipedia) |
Are GCPs necessary for Pix4D? When are they highly recommended?
Ground Control Points (GCPs) are not absolutely necessary however they are highly recommended when imaging a 3D object and the user wishes to combine the oblique data with nadir data or, when mapping ‘corridors’ such as railways, roads and, rivers.
What is the quality report?
The quality report is an extremely useful report that is automatically generated and details in depth, various aspects of the quality of the data collected.
The quality report gives the user a quality check, a preview of the DSM and orthomosaic, the initial image positions, computed image/GCPs/manual tie points, absolute camera position and orientation uncertainties, 2d keypoint matches, geolocation details, system information, coordinate systems and, processing options. To learn more about the quality report, click here.
What outputs/products can Pix4D generate?
Pix4D is capable of generating a host of outputs including, point clouds, DSMs, DTMs, orthomosaics, index maps, contour lines, 3D meshes, 3D digitized objects, KML Google tiles, HTML Google tiles, LOD meshes, video animations, Bingo txt and, shapefiles. Most commonly for our datasets, we most often generate Orthomosaics and DSMs as the output.
How does Pix4D organize the output files/folders?
Normally, when a Pix4D is run, it will create its own folder and file structure for the data it produces along with a link to the project in Pix4DMapper.
In the first folder are four subfolders and a log file of the processing record. The subfolders were labeled 1_initial, 2_densification, 3_dsm_ortho and, temp. Within the 1_initial folder, there were three additional folders labeled params, project_data and, report. Within the 2_densification subfolder were three sub-subfolders labeled 3d_mesh, point_cloud and, Project_data. Within the 3_dsm_ortho subfolder were four sub-subfolders labeled 1_dsm, 2_mosaic, extras and, project_data. Finally, within the temp subfolder was a single sub-subfolder labeled rays. Figure 2 is a graphical representation of the folder structure spelled out above.
 |
| Figure 2: Pix4D Folder Structure |
Methods: A Run Through of The Software
Step 1:
Before using Pix4DMapper, the first step in the process is to clean up the data to remove bad images, and GCP data. Bad image can include images that will not show a graphical preview when the graphical preview is selected in File Explorer. Images that should be removed include those taken while the platform is on the ground, launching, landing, climbing to the mission altitude, or blurry ones taken while the UAS platform is maneuvering. Bad GCP data should also be removed where ever possible. If the user is using the Propeller Aeropoints GCP products, the user can simply enter the GCP data into the web application and the software will identify which data are usable and which aren’t.Step 2:
Once the data has been cleaned up, open Pix4DMapper and click on New Project. Next, name the project using a naming convention as discussed in the previous two previous posts, change the directory location to a folder of choice and, click Next (See Figure 3).
 |
| Figure 3: Naming and Choosing a Directory |
Step 3:
Add the images to be processed by clicking on either the Add Images or the Add Directories tab at the top of the page. When clicking on Add Images, the user has to select each photo that he or she wishes to add, whereas the Add Directories button allows the user to select the folder where the unprocessed images are located and they will be automatically added. Once the images are added as shown in figure 4, Click Next. Figure 4 shows the image selection page with the two ways of adding images circled.
 |
| Figure 4: Adding Images |
Step 4:
Under the Image Geolocation area, check that the coordinate system matches the in field user created metadata file as shown in figure 5. If not, click Edit, then click Advanced and correct the coordinate system.
 |
| Figure 5: Checking Image Geolocation Coordinate System |
Step 5:
Under the Selected Camera Model area, make sure the camera is set correctly to the camera that is being used. Click Edit and then Edit again in the Edit Camera Model page and make sure the Camera Model Parameters are set correctly. In this case, the camera Shutter Model was set incorrectly to Global Shutter or Fast Readout and had to be changed to Linear Rolling Shutter. Figure 6 shows the process of how to edit the camera model information and figure 7 shows changing the Shutter Model.
 |
| Figure 6: Checking the Camera Model |
 |
| Figure 7: Changing the Shutter Model |
Step 6:
Under the Selected Coordinate System area, make sure the Datum and Coordinate System are set to match the user created key metadata file containing the important metadata, including how the dataset was collected and coordinate systems were used. If these are not correct, select Known Coordinate System [m], check the box labeled Advanced Coordinate Options and, select the correct ones by selecting From List. *Note: The user can change the vertical coordinate system if necessary during this step as well. Figure 8 shows how to change the output coordinate system.
 |
| Figure 8: How to Change the Output Coordinate System |
For the purposes of this assignment, the Output Coordinate System will be left to Auto Detected. Once these settings are adjusted, click Next.
Step 7:
Select a template from the Processing Options Template page depending on the type of data being processed and click Finish. For the purposes of this assignment, 3D Maps was selected (See Figure 9).
 |
| Figure 9: Selecting a Processing Template |
Step 8:
In the bottom left corner of the screen, deselect 2. Point Cloud and Mesh and 3. DSM, Orthomosaic and Index so that once the initial processing is complete, the user can check the quality report and make sure the data processing is on track before spending several hours to several days processing data only to find that an error occurred and the data is unusable. Figure 10 below shows steps 2 and 3 deselected.
 |
| Figure 10: Deselecting Steps 2 and 3, Point Cloud and Mesh and DSM, Orthomosaic and Index |
Step 9:
Click on Processing Options
located in the bottom right side of the window and of the sidebar options, open 2. Point Cloud and Mesh as shown in figure 11 below. |
| Figure 11: Opening Point Cloud and Mesh Tab |
Within the 2. Point Cloud and Mesh tab under the Point Cloud page in the Point Cloud Densification area, adjust the Point Density and Minimum Number of Matches as needed. If the data has a lot of varied texture, such as dense vegetation or forests, the user may choose to increase the Point Density and set the Minimum Number of Matches to greater than 3. *Note: The downside of increasing these parameters is that the processing time will be significantly longer.
Under Point Cloud Classification area, the user may choose to select Classify Point Cloud in order to apply filtering techniques to the data later on. Figure 12 shows where to adjust the Point Density, set the Minimum Number of Matches and, Classify [the] Point Cloud.
 |
| Figure 12: Adjusting Point Density, Minimum Number of Matches, and Classifying Point Cloud. |
For the purposes of this assignment, the 3D Textured Mesh page and the Point Cloud page will be left at default.
Step 10:
Open the 3. DSM, Orthomosaic and Index sidebar and within the DSM and Orthomosaic page under the Raster DSM area, check both the GeoTIFF and Merge Tiles and change the method to Triangulation (See Figure 13). In the Orthomosaic area check the Google Maps Tiles and KML box so that the user may upload the data to google earth and view it that way if he or she wishes (See Figure 13.
 |
| Figure 13: Adjusting Raster DSM and Orthomosaic Parameters |
Optionally, the user can adjust the parameters in the Additional Outputs and Index Calculator pages. Under the Additional Outputs page, it is recommended to generate a shapefile if for example, the end client is a geologist or surveyer. The user may do so by checking the SHP box and adjusting the settings within the Contour Lines area. Figure 14 shows the Additional Outputs page with SHP selected.
 |
| Figure 14: Checking SHP within Additional Outputs Page |
*Note: The last page under the 3. DSM, Orthomosaic and Index sidebar is labeled Index Calculator. This page is only used to process multispectral data and will not be used in this assignment.
Step 11:
Click the OK button at the bottom of the Processing Options window, double check that 2. Point Cloud and Mesh and 3. DSM, Orthomosaic and Index highlighted in red are unchecked, and click Start.
Step 12:
Once the 1. Initial Processing is complete, examine the quality report to make sure that the data is good enough to continue. If so, uncheck Initial Processing, check both 2. Point Cloud and Mesh and 3. DSM,Orthomosaic and Index and, click Start. To learn more about using the quality report to assess the data quality, click here.
Step 13:
Once Pix4DMapper is done processing the data, the outputs for orthomosaics dsms and other data will be located in 3_dsm_ortho folder, see figure 1, and can be manipulated using software such as ArcGIS Pro, ArcMap etc..
Results: Discussion of Data and Results
Once the data was processed, the final quality report was opened and certain aspects were viewed. These included the summary, quality check and, previews of the DSM and orthomosaic (See Figure 15).
The summary section per the figure contains the project name, when the project was processed, the camera model used, the average ground sampling distance (GSD), area covered, and the amount of time needed for the initial processing.
Ground sampling distance is the measurement, on the ground, of the distance between two adjacent pixels. Essentially, it is a measure of the spatial resolution of the data. A larger GSD correlates to less spatial resolution. For example, if a RedEdge sensor is flown at 25 meters, the spatial resolution will be 1.7 centimeters per pixel, whereas when flown at 85 meters, the spatial resolution will be 5.8 centimeters per pixel, resulting in less visible details.
The quality check section is a table containing the median number of keypoints per image, how many images were calibrated and used, the parameters necessary to optimize the camera settings, the average number of matches between calibrated images and, GCP information.
What are keypoints, how many were generated and, how many are considered good enough to have enough visual content to be processed?
A keypoint is a point unique to a specific area on the ground or object. Pix4D locates these keypoints and matches them across images. These keypoints that link images are called matched keypoints. The more overlap between images there is, the more matched keypoints there are, the better the point cloud will be. The median number of keypoints generated was 75553 points per image and the mean number of matches per image was 18702. Generally if the number of keypoints and matched keypoints is over 10,000, then the images have enough context to be processed.
Looking at the Report generated from step 1, how many images were used, and how many were rejected?
According to the quality report, 66 of 67 images were calibrated and all 67 were used (See Figure 16).
Referring to the final quality report generated, how long did each processing step take? (create a table).
Table 2 below shows the time each step took to process.
After viewing the quality report, the Ray Cloud icon on the side bar was clicked on and a video flythrough was created and uploaded to YouTube. The video was created using the camera icon located in the Create area. Figure 17 shows a video being created using the camera icon.
*Note: to learn how to create a video animation within Pix4D click here.
Once the video was created, Pix4D was closed and ArcGIS Pro was used to create two maps using the DSM and Orthomosaic outputs from Pix4D. These maps are displayed below in figures 18 and 19.
Figure 19: Map of Pix4D Output Orthomosaic of Home Residence
Looking at the DSM and orthomosaic maps above, the residence is located on a mound with a high spot in front of the residence. Behind the residence, there is a wooded area and in front is a grass lawn with one tree. Along the road, there are trees planted on either side. Looking at the orthomosaic, the quality is very good. There is practically no stitching errors within the cropped output file, however there are some issues. Both the orthomosaic and the DSM are not located correctly with respect to the topographic basemap. This is due to the fact that there were no GCPs used when processing the data. Within the DSM, wooded areas and areas near the edge located near houses were distorted. This probably occurred when generating the DSM; the confusing differences in values on the ground in certain areas, and branches of a tree in other areas caused Pix4D to connect these in incorrect locations, or to just skip over the area as noise and only connect the fringes where data was more consistent.
Conclusions: Final Critique
Summarize why Pix4D is important for processing UAS data.
Without having a tool such as Pix4DMapper, offering many options in an easy to use package, the processing of high quality data would be very difficult to replicate in another software package. The many compatible output files that it generates allows the produced data to be easily usable in other software packages.
Discuss the benefits and drawbacks (think time and computing needs).
The benefit of using Pix4DMapper is generating very high quality outputs from data.
The downsides of running Pix4DMapper are that it is very computationally expensive in time and requires relatively high end computer hardware to run.
 |
| Figure 15: Summary, Quality Check and, Preview |
The summary section per the figure contains the project name, when the project was processed, the camera model used, the average ground sampling distance (GSD), area covered, and the amount of time needed for the initial processing.
Ground sampling distance is the measurement, on the ground, of the distance between two adjacent pixels. Essentially, it is a measure of the spatial resolution of the data. A larger GSD correlates to less spatial resolution. For example, if a RedEdge sensor is flown at 25 meters, the spatial resolution will be 1.7 centimeters per pixel, whereas when flown at 85 meters, the spatial resolution will be 5.8 centimeters per pixel, resulting in less visible details.
The quality check section is a table containing the median number of keypoints per image, how many images were calibrated and used, the parameters necessary to optimize the camera settings, the average number of matches between calibrated images and, GCP information.
What are keypoints, how many were generated and, how many are considered good enough to have enough visual content to be processed?
A keypoint is a point unique to a specific area on the ground or object. Pix4D locates these keypoints and matches them across images. These keypoints that link images are called matched keypoints. The more overlap between images there is, the more matched keypoints there are, the better the point cloud will be. The median number of keypoints generated was 75553 points per image and the mean number of matches per image was 18702. Generally if the number of keypoints and matched keypoints is over 10,000, then the images have enough context to be processed.
Looking at the Report generated from step 1, how many images were used, and how many were rejected?
According to the quality report, 66 of 67 images were calibrated and all 67 were used (See Figure 16).
 |
| Figure 16: Calibrated and Used Images |
Referring to the final quality report generated, how long did each processing step take? (create a table).
Table 2 below shows the time each step took to process.
 |
| Table 2: Pix4D Processing Times |
After viewing the quality report, the Ray Cloud icon on the side bar was clicked on and a video flythrough was created and uploaded to YouTube. The video was created using the camera icon located in the Create area. Figure 17 shows a video being created using the camera icon.
 |
| Figure 17: Video Animation Creation |
*Note: to learn how to create a video animation within Pix4D click here.
Once the video was created, Pix4D was closed and ArcGIS Pro was used to create two maps using the DSM and Orthomosaic outputs from Pix4D. These maps are displayed below in figures 18 and 19.
 |
| Figure 18: Map of Pix4D Output DSM of Home Residence |
Figure 19: Map of Pix4D Output Orthomosaic of Home Residence
Looking at the DSM and orthomosaic maps above, the residence is located on a mound with a high spot in front of the residence. Behind the residence, there is a wooded area and in front is a grass lawn with one tree. Along the road, there are trees planted on either side. Looking at the orthomosaic, the quality is very good. There is practically no stitching errors within the cropped output file, however there are some issues. Both the orthomosaic and the DSM are not located correctly with respect to the topographic basemap. This is due to the fact that there were no GCPs used when processing the data. Within the DSM, wooded areas and areas near the edge located near houses were distorted. This probably occurred when generating the DSM; the confusing differences in values on the ground in certain areas, and branches of a tree in other areas caused Pix4D to connect these in incorrect locations, or to just skip over the area as noise and only connect the fringes where data was more consistent.
Conclusions: Final Critique
Summarize why Pix4D is important for processing UAS data.
Without having a tool such as Pix4DMapper, offering many options in an easy to use package, the processing of high quality data would be very difficult to replicate in another software package. The many compatible output files that it generates allows the produced data to be easily usable in other software packages.
Discuss the benefits and drawbacks (think time and computing needs).
The benefit of using Pix4DMapper is generating very high quality outputs from data.
The downsides of running Pix4DMapper are that it is very computationally expensive in time and requires relatively high end computer hardware to run.
