Multi-perspective monitoring of wildlife and human activities from camera traps and drones with deep learning models

Multi-perspective monitoring of wildlife and human activities from camera traps and drones with deep learning models addresses the need for understanding spatial distributions of wildlife and human activities to evaluate interactions and inform conservation planning. The gist: This study reveals human–wildlife conflicts within the conserved landscape by integrating multi-perspective monitoring with automated object detection, using YOLOv11s for camera trap imagery and an enhanced Faster R-CNN model for drone thermal imagery.

Context

Wildlife and human activities are key components of landscape systems, and understanding their spatial distribution is essential for evaluating human–wildlife interactions and informing effective conservation planning. Human activities frequently change or destroy wildlife habitats, with illegal hunting being a detrimental impact. Wildlife interacts with humans in varying frequencies, leading to positive or negative interactions referred to as human-wildlife conflict. Monitoring these activities is crucial for understanding these interactions and playing a role in effective wildlife conservation and sustainable ecological landscape management.

Methods

The study was conducted in Chitwan National Park (CNP), Nepal, and adjacent regions between February and July 2022. Images were collected by visible and near-infrared camera traps, processed to create training and testing datasets for deep learning models to automatic identify wildlife and human activities. Drone-collected thermal imagery was used for detecting targets to provide a multiple monitoring perspective. The dataset included images of domestic goats, tigers, rhinos, rhesus macaques, elephants, spotted deer, sambar deer, and humans. To create the dataset from camera trapping images containing both wildlife and human activities (Table 1), the Roboflow platform was utilized to annotate images accurately for YOLO model efficiency. The drone imagery was captured using a DJI Mavic 2 Enterprise equipped with both thermal and optical visual cameras, capturing infrared and visible RGB images simultaneously.

Deep Learning Models

The study tested several deep learning models for object detection in camera trap imagery. YOLOv11s achieved the highest performance with a precision of 96.2%, recall of 92.3%, mAP50 of 96.7%, and mAP50-90 of 81.3, making it the most effective for detecting objects in camera trap imagery. YOLO models were utilized, including YOLOv5n, YOLOv5s, and YOLOv5m, along with YOLOv11n, YOLOv11s, and YOLOv11m. The training process utilized an input image size of 640 and 32 as the batch size for the models. For drone-based thermal imagery analysis, an enhanced Faster R-CNN model was used to detect deer, achieving an Average precision score of 91.6% for all deer objects when integrated with FPN and ResNet18.

Spatial Analysis

Spatial pattern analysis was performed to identify animal and resident activity hotspots and delineation potential human–wildlife conflict zones. Kernel Density Estimation (KDE) was employed to identify hotspots where wildlife and human activities intersect, using results from camera trapping detection for wildlife (tigers, rhinos) and detections of people and domestic goats for human activities. The Kernel Density spatial analysis tool in ArcGIS Pro was used to calculate density by applying a Kernel function to create a continuous surface. Clustering results based on correlation-based hierarchical clustering categorized the 80 camera traps into three distinct groups: human, wildlife, and conflict zones, where both humans and wildlife are frequently detected by the same camera traps.

Conclusions

This study reveals human–wildlife conflicts within the conserved landscape. Integrating multi-perspective monitoring with automated object detection enhances wildlife surveillance and landscape management. The integration of camera traps and drone-based monitoring provides a promising framework for future three-dimensional and flexible wildlife monitoring systems, offering actionable data to inform local authorities in the development of targeted management strategies. Future research will aim to advance this framework by incorporating automated individual identification and analyzing spatial interactions using state-of-the-art machine learning and geographic information system (GIS) techniques within protected landscapes.

The gist

This study reveals human–wildlife conflicts within the conserved landscape by integrating multi-perspective monitoring with automated object detection, using YOLOv11s for camera trap imagery and an enhanced Faster R-CNN model for drone thermal imagery.

How it works

1. Camera trapping data were collected throughout the study area to support the training and detection of deep learning models, capturing images with motion-sensitive camera traps that produce motion-activated 32MP photos and 4K 30 fps videos. These systems are strategically positioned at sites exhibiting signs of wildlife and human activities, with GPS coordinates recorded for each trap location.

2. To reflect human activities, images including human and domestic goats were collected in the dataset, alongside images of wildlife such as spotted deer, sambar deer, rhesus macaques, rhinos, tigers, and elephants.

Here are specific improvements to AI systems based on the provided research, along with what those improved systems could achieve:

1. [Improved System]: A Multi-Modal Surveillance Platform integrating Camera Trap (RGB/NIR) and Drone Thermal Imagery via a Federated Deep Learning Architecture.

2. [Capability]: This system can perform real-time, cross-perspective detection of wildlife and human activities across large, complex landscapes (e.g., national parks). Specifically:

3. [Specific Improvement 1 - Detection Accuracy]: By integrating YOLOv11s (for camera trap imagery) with an enhanced Faster R-CNN model (for thermal drone imagery), the system achieves superior target detection accuracy compared to single-sensor systems, particularly under challenging conditions like low light, heavy vegetation occlusion, and partial body visibility.

4. [Specific Improvement 2 - Data Fusion]: The system can perform effective data fusion by cross-validating detections between ground-level camera traps and aerial thermal imagery (e.g., confirming a detected animal presence from a drone flight path against ground evidence), thereby reducing false positives and increasing the reliability of abundance estimates.

5. [Specific Improvement 3 - Automated Conflict Zoning]: The system can automatically generate dynamic spatial maps identifying high-probability human-wildlife conflict zones by performing Kernel Density Estimation (KDE) on aggregated detection results from both data sources, specifically targeting overlaps between human activity classes (people, domestic goats) and key wildlife species (tigers, rhinos).

6. [Specific Improvement 4 - Predictive Modeling]: The system can utilize the spatial clustering results derived from hierarchical clustering to categorize camera trap locations into distinct ecological groups (Human-dominated, Wildlife-dominated, Conflict zones), allowing conservation managers to proactively predict where future conflicts are most likely to occur based on current spatial patterns.

7. [Specific Improvement 5 - Species/Activity Identification]: The system can be trained to automatically identify specific wildlife species and human activities from raw imagery using state-of-the-art deep learning models (like YOLOv11), moving beyond simple object detection to detailed ecological classification for precise monitoring.

8. [Improved System]: A Real-Time, Adaptive Wildlife Monitoring Network utilizing Multi-Scale Object Detection Models on Unmanned Aerial Vehicles (UAVs).

9. [Capability]: This system can enable flexible, adaptive aerial surveillance by leveraging the YOLO architecture (specifically YOLOv5/YOLOv11 variants) optimized for real-time performance on drone imagery. Specifically:

10. [Specific Improvement 6 - Real-Time Aerial Tracking]: The system can perform high-speed, real-time detection and tracking of specific wildlife species (e.g., deer, rhinos) using thermal imagery from drones, allowing for rapid response to unusual movements or incursions into sensitive areas.

11. [Specific Improvement 7 - Enhanced Thermal Differentiation]: By utilizing thermal infrared sensors on drones, the system can effectively detect targets obscured by vegetation or operating under low-light conditions where standard optical systems fail, providing a critical 'eyes in the dark' capability for monitoring nocturnal wildlife or hidden human movements.

Wildlife and human activities are key components of landscape systems. Understanding their spatial distribution is essential for evaluating human wildlife interactions and informing effective conservation planning. Multiperspective monitoring of wildlife and human activities by combining camera traps and drone imagery. Capturing the spatial patterns of their distributions, which allows the identification of the overlap of their activity zones and the assessment of the degree of human wildlife conflict. The study was conducted in Chitwan National Park (CNP), Nepal, and adjacent regions. Images collected by visible and nearinfrared camera traps and thermal infrared drones from February to July 2022 were processed to create training and testing datasets, which were used to build deep learning models to automatic identify wildlife and human activities. Drone collected thermal imagery was used for detecting targets to provide a multiple monitoring perspective. Spatial pattern analysis was performed to identify animal and resident activity hotspots and delineation potential human wildlife conflict zones. Among the deep learning models tested, YOLOv11s achieved the highest performance with a precision of 96.2%, recall of 92.3%, mAP50 of 96.7%, and mAP50 of 81.3%, making it the most effective for detecting objects in camera trap imagery. Drone based thermal imagery, analyzed with an enhanced Faster RCNN model, added a complementary aerial viewpoint for camera trap detections. Spatial pattern analysis identified clear hotspots for both wildlife and human activities and their overlapping patterns within certain areas in the CNP and buffer zones indicating potential conflict. This study reveals human wildlife conflicts within the conserved landscape. Integrating multiperspective monitoring with automated object detection enhances wildlife surveillance and landscape management.