AUTOMATIC SEGMENTATION OF TREE CROWNS IN PINE FORESTS USING MASK R-CNN ON RGB IMAGERY FROM UAVS

А. D. Nikitina

 

Center for Forest Ecology and Productivity of the RAS

Profsoyuznaya st. 84/32 bldg. 14, Moscow, 117997, Russia

 

E-mail: nikitina.al.dm@gmail.com

Received: 18.05.2024

Revised: 05.06.2024

Accepted: 22.06.2024

The article presents the results of applying an improved method for automatic segmentation of RGB imagery obtained using consumer-grade UAVs, based on the Mask R-CNN neural network architecture. Blocks for the preparation and post-processing of raster and vector files have been developed for working with geospatial data. The model was trained on 7000 crowns identified in pine forest of automorphic habitats in the mixed coniferous-broadleaf forest subzone. Training was carried out using cross-validation. Additional data of 1337 crowns were used for verification. During the sequential filtering by area, confidence level, and duplicate segments, the quality of the final segmentation results improved for all age groups of pine forests. The final average precision is 0.87, recall – 0.81, F1-score – 0.83. The results demonstrate the high efficiency of the filtering algorithm in reducing segment redundancy and increasing data reliability. The Mask R-CNN automatic segmentation method is an effective tool for analyzing the characteristics of pine canopies using RGB imagery from UAV surveys. It is capable of replicating the results of visual interpretation with high accuracy. This method particularly advantageous for scaling studies to large areas where manual delineation becomes labor-intensive.

Keywords: Mask R-CNN, automatic segmentation, detection trees, pine forests, RGB imagery, UAVs, ecological monitoring, remote sensing

 

REFERENCES

Agisoft Metashape, available at: http://www.agisoft.com (2024, 01 June).

Aubry-Kientz M., Dutrieux R., Ferraz A., Saatchi S., Hamraz H., Williams J., A comparative assessment of the performance of individual tree crowns delineation algorithms from ALS data in tropical forests, Remote Sensing, 2019, Vol. 11, No 9, pp. 1086 (1–21).

Ball J. G., Hickman S. H., Jackson T. D., Koay X. J., Hirst J., Jay W., Coomes D. A., Accurate delineation of individual tree crowns in tropical forests from aerial RGB imagery using Mask R-CNN, Remote Sensing in Ecology and Conservation, 2023, Vol. 9, No 5, pp. 641–655.

Beloiu M., Heinzmann L., Rehush N., Gessler A., Griess V. C., Individual Tree-Crown Detection and Species Identification in Heterogeneous Forests Using Aerial RGB Imagery and Deep Learning, Remote Sensing, 2023, Vol. 15, p. 1463.

Chen X., Shen X., Cao L., Tree Species Classification in Subtropical Natural Forests Using High-Resolution UAV RGB and SuperView-1 Multispectral Imageries Based on Deep Learning Network Approaches: A Case Study within the Baima Snow Mountain National Nature Reserve, China, Remote Sensing, 2023, Vol. 15, p. 2697.

Diez Y., Kentsch S., Fukuda M., Caceres M. L. L., Moritake K., Cabezas M., Deep Learning in Forestry Using UAV-Acquired RGB Data: A Practical Review, Remote Sens., 2021, Vol. 13, p. 2837.

Espíndola R. P., Ebecken N. F. F., Advances in remote sensing for sustainable forest management: monitoring and protecting natural resources, Revista Caribeña de Ciencias Sociales, 2023, Vol. 12, No 4, pp. 1605–1617.

Hao Z., Lin L., Post C.J., Mikhailova E.A., Li M., Chen Y. et al., Automated tree-crown and height detection in a young forest plantation using mask region-based convolutional neural network (Mask R-CNN), ISPRS Journal of Photogrammetry and Remote Sensing, 2021, Vol. 178, pp. 112–123.

He K., Gkioxari G., Dollár P., Girshick R., Mask R-CNN, Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2961–2969.

https://cepl.rssi.ru/bio/forest/index.htm (2024, 1 June).

Ivanova N.V., Shashkov M.P., Shanin V.N., Study of pine forest stand structure in the priosko-terrasny state nature biosphere reserve (Russia) based on aerial photography by quadrocopter, Nature Conservation Research, 2021, Vol. 6, No 4, pp. 1–14.

Medvedev A. A., Tel’nova N. O., Kudikov A. V., Alekseenko N. A., Analiz i kartografirovanie strukturnyh parametrov redkostojnyh severotajozhnyh lesov na osnove fotogrammetricheskih oblakov tochek (Use of photogrammetric point clouds for the analysis and mapping of structural variables in sparse northern boreal forests), Sovremennye problemy distancionnogo zondirovanija Zemli iz kosmosa, 2020, Vol. 17, No 1, pp. 150–163.

Nevalainen O., Honkavaara E., Tuominen S., Viljanen N., Hakala T., Yu X., Hyyppä J., Saari H., Pölönen I., Imai N. N., Tommaselli A. M. G., Individual tree detection and classification with UAV-based photogrammetric point clouds and hyperspectral imaging, Remote Sensing, 2017, Vol. 9, No 3, p. 185.

Nezami S., Khoramshahi E., Nevalainen O., Pölönen I., Honkavaara E., Tree species classification of drone hyperspectral and RGB imagery with deep learning convolutional neural networks, Remote Sensing, 2020, Vol. 12, No. 7, p. 1070.

Ocer N. E., Kaplan G., Erdem F., Matci D.K., Avdan U., Tree extraction from multi-scale UAV images using Mask R-CNN with FPN, Remote Sensing, 2020, Vol. 11, p. 847–856.

Puliti S., Ene L. T., Gobakken T., Næsset E., Use of partial-coverage UAV data in sampling for large scale forest inventories, Remote Sensing of Environment, 2017, Vol. 194, pp. 115–126.

Tuominen S., Näsi R., Honkavaara E., Balazs A., Hakala T., Viljanen N., Reinikainen J., Tree species recognition in species rich area using UAV-borne hyperspectral imagery and stereo-photogrammetric point cloud, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, Vol. XLII-3/W3, pp. 185–194.

Zhou J., Chen X., Li S., Dong R., Wang X., Zhang C., Zhang L., Multispecies individual tree crown extraction and classification based on BlendMask and high-resolution UAV images, Journal of Applied Remote Sensing, 2023, Vol. 17, No 1, p. 016503.