1．Overview
Reservoir dams are an essential component of the basin flood control engineering system, a crucial part of the national water network, and a vital component of ensuring national water security. Currently, China has approximately 98,000 dams of various types, among which 665 are used for power generation. Many of these are high dams with large reservoirs, including a group of dams with heights exceeding 200 meters, ranking among the tallest in the world. High dams and large reservoirs serve multiple functions such as power generation, flood control, transportation, and water supply. They are the most important comprehensive infrastructure for ensuring safety and promoting development.
In the event of a dam failure and collapse, especially for high dam reservoirs, uncontrolled discharge of the reservoir water will lead to a chain reaction of downstream dam groups, and the resulting secondary disasters cannot be ignored. In China, hydropower stations are mainly distributed in the southwest region, which is also prone to the formation of dammed lakes induced by geological disasters such as earthquakes, landslides, and mudslides. Additionally, some dams have been in operation for a long time and are gradually revealing issues of dam disease and aging problems due to historical conditions and limitations.
Reservoir Flooding
Currently, in China, the inspection of most dams is still conducted through traditional manual methods. This approach primarily relies on using ropes and setting up scaffolding to provide a work platform. Inspection personnel manually approach the inspection sites and conduct visual inspections with the naked eye. The manual inspection method primarily faces the following issues:
- Inspection personnel are prone to errors, omissions, and incorrect assessments due to factors such as technical limitations, individual experience, and levels of responsibility.
- Conventional inspection methods have limitations for high dam reservoirs, presenting significant operational difficulties and inspection blind spots.
- The cost of manual inspections is high, efficiency is low, and there are safety risks.
- Data acquisition is incomplete, imaging data quality is poor, and positioning and measurement accuracy are low.
- The digitization and integration levels are inadequate, resulting in less intuitive outcomes, making it challenging to review inspection results comprehensively.
Manual inspection
With the improvement of the national economy, the acceleration of infrastructure construction, and the continuous enhancement of technological innovation capabilities, the technical methods for inspections and monitoring in various industries have greatly improved. One notable advancement is the widespread and in-depth application of drone inspection technology, particularly in fields such as electric power, forestry, and agriculture.
However, in the field of dam inspections, the application of drones is currently in a spontaneous, discrete, and endogenous state. It lacks systematic planning and has not been effectively integrated into and supported by management systems. In terms of the industry development level of drone technology, the use of drones for dam inspections is still in its early stages and lags more maturely applied industries. Therefore, there is a significant need and urgency to establish a digital, intelligent, and efficient drone-based dam inspection system.
Based on these circumstances, Riebo Technology, leveraging its core technologies and extensive research and development experience in drones, photogrammetry, and LiDAR system, has developed a comprehensive inspection system based on digital twin technology for the entire inspection process. Additionally, we have published a unique drone-based inspection solution.
Drone-based inspection
2．Inspection Solution
In response to the challenges and pain points of traditional manual inspections, Riebo Technology's unmanned aerial inspection solution has the following advantages:
(1) Drones can easily reach various parts of the dam, combining digital twin simulation and full-coverage inspection to address the issues of omissions and blind spots in manual inspections.
(2) High-resolution inspection images, combined with AI recognition and manual review, result in more accurate and comprehensive identification of damage points.
(3) The civilian drone market is mature, with low learning, usage, and maintenance costs. It is easy to operate, deploy flexibly, and is highly efficient. A single operator with a single drone can complete the inspection task.
(4) High-precision self-positioning technology is integrated into the drone with a professional inspection camera, enabling the acquisition of high-definition images and spatial positions of damage points, with measurement accuracy reaching millimeter level.
(5) Digital twin technology combined with the inspection management system visually presents and analyzes damage data in three dimensions, providing a more accurate and comprehensive basis for scientific and efficient management and operation.
The specific process of the solution is as follows:
Step 1: Create a 3D model to achieve a digital twin of the inspection object.
Step 2: Plan the inspection route based on the 3D model.
Step 3: Collect high-resolution image data by the drone along the route.
Step 4: Process the image data for position and orientation metadata.
Step 5: Identify damages using AI and manual interaction.
Step 6: Browse, manage, count, analyze, and generate inspection reports for damage points in a 3D visualization through the Inspection Management System.
Inspection solution
2.1 Digital Twin
In the initial dam inspection, a 3D model is created to achieve a digital twin of the dam, accurately reproducing the true condition of the dam with a precision reaching the centimeter level.
3D model of the dam (Overall View)
3D model of the dam (Partial View)
The 3D model serves two main purposes:
(1) 3D models can serve as a digital base map for the visualization and management of damage data. For all identified issues, the model is used to display and manage them, allowing maintenance personnel to intuitively assess the true condition of the dam.
(2) 3D reality models can serve as the basis for inspection route planning, enabling autonomous drone flights and significantly reducing manual intervention.
Considering the structure and characteristics of dams, Riebo Technology has selected its M-series products and designed an inspection function in its self-developed route planning software, Riebo Route Assistant (RRA), specifically for 3D modeling of dams. While ensuring the success rate of aerial data acquisition and model quality, it significantly reduces the number of flight routes and images, thereby greatly improving modeling efficiency.
Through practical testing, in typical scenarios, data collection takes only 17 minutes, and the reconstruction of the 3D model takes just 94 minutes.
2.2 Route Planning
Drone inspections differ from regular drone aerial survey operations. According to the relevant inspection standards, the strictest requirement is to detect concrete cracks with a width of as small as 0.1mm. As a result, during inspections, drones need to closely approach the inspection objects to capture images. To ensure safe drone flight in complex environments and to guarantee complete coverage of the inspection objects, flight route planning is necessary to allow the drone to autonomously perform the inspection.
The inspection function in Riebo Route Assistant (RRA) can automatically generate simulated flight routes based on the 3D model. It takes an average of 5 minutes to complete route planning. Combined with the quick modeling technology mentioned earlier, it allows for the initial inspection of typical scenes to be completed within one day, including modeling and inspection.
Route Planning in RRA
2.3 Data Collection
The collection of inspection image data is carried out by drones equipped with cameras, and field operations only require one person, one vehicle, and a set of data collection equipment. With algorithm optimization and the use of long-focus, medium format professional inspection cameras, the operational efficiency is very high. Testing has shown that for a dam with dimensions of 190 meters in length and 170 meters in height, field inspection operations take only about 4 hours.
Currently, the drone platform used in the solution is the DJI M300/350 RTK, which is one of the most stable multi-rotor drones available on the market. The camera used is Riebo's self-developed SF10 inspection camera, with a maximum spatial resolution of 0.1mm, making it capable of performing inspections to the highest standards.
Field operation of Dam inspection
2.4 Data processing
Data processing is performed using the SkyScanner software, which is compatible with the full range of Riebo cameras. This software allows the export of inspection data by sorties, enabling efficient management of massive inspection data. Additionally, the software can embed geographical location information into images, achieving high-precision georeferencing. It can also use the drone and camera's rotation information to calculate the directions of image's FOV, providing data support for precise positioning of damage points in subsequent analysis.
2.5 AI recognition
With advancements in image recognition algorithms based on convolutional neural networks (CNN) and deep learning, the application of artificial intelligence for disease point recognition using high-resolution inspection images has become easier and more mature. The specific workflow is as follows:
Step 1: Annotate the collected image data and deliver it for training to a convolutional neural network (CNN) model. The trained neural network model can rapidly perform crack recognition and detection on images. The model's recognition accuracy can be further improved by refining the model and collecting more training data.
Step 2: For images where defects have been recognized, a series of processing steps are applied, including enhancement, noise reduction, image segmentation, and edge detection. This process generates crack segmentation results and pixel-level measurements. By using optimization segmentation algorithms based on deep learning, high pixel-level overlap is achieved for cracks, even in the presence of small cracks, complex backgrounds, and interference like artificial markings, ensuring correct segmentation results. Additionally, other dam defects such as seepage, corrosion, and water stain crystallization are also identified.
AI recognition training
2.6 Inspection management
Currently, the majority of inspection operations result in inspection reports. However, whether it's related to specific defects or the overall condition of a bridge, the presentation is not very intuitive. Furthermore, the inspection process lacks effective digital management, the re-usability of inspection results is low, and it's challenging to supervise and inspect the inspection process. Some existing inspection software also suffers from low levels of integration and a lack of data interoperability.
In response to these challenges, Riebo Technology has developed the Riebo Inspect Master System. This system digitizes and streamlines the entire inspection process, offering an all-in-one solution that integrates personnel management, data management, defect identification, defect localization, 3D visualization of defects, historical defect comparison, data statistical analysis, and customized inspection report generation.
Smart Inspection Management System
3．Inspection Management System
The entire system consists of two subsystems: the collection system and the management system. The collection system includes the drone platform, quick modeling camera, inspection camera, route planning software, and data processing software. The management system is the Tongtu Smart Inspection Management System.
3.1 Collection system
The collection system primarily utilizes a drone platform with a gimbal camera, in conjunction with relevant software, to capture high-precision location information, high-resolution image data, and defect data of various components of the inspection object. The system includes:
(1) Drone Platform:
The drone platform used is the DJI M300/350 RTK multi-rotor drone. This drone is equipped with an RTK module, supporting custom network RTK, which enables high-precision and accurate positioning. Its features of stable flight, six-axis environmental perception, and obstacle avoidance capabilities make it highly suitable for complex bridge inspection environments.
M300 RTK
(2) Modeling Camera:
The modeling camera used is from Riebo's M series products. The M series cameras are single-lens metric cameras developed by Riebo, primarily used in scenarios like oblique photogrammetry, close-range photogrammetry, facade modeling, and orthophoto collection. The series includes several high-end products such as M4Pro, M6/M6pro, M10Pro, M15Pro, with full-frame and medium-format sensors, and focal lengths ranging from 18mm to 75mm. They all incorporate Riebo's patented MLE lens and MLS shutter technology.
The MLE lens employs aspherical lens elements and, while controlling the lens's volume, incorporates extra-low dispersion (ED) lens elements to correct and eliminate astigmatism in imaging, effectively controlling chromatic aberration. This results in lens distortion being controlled to within one-thousandth. Compared to traditional photogrammetric lenses, the number of tie points is improved by 10% to 20%, and the root mean square (RMS) of the reprojection error is reduced by around 20%.
The MLS shutter is a mechanical distortion-free shutter specially developed by Riebo for metric cameras. Unlike traditional photogrammetric cameras with mechanical rolling shutters that can cause motion distortion, have a shorter lifespan, and are susceptible to freezing in low-temperature environments, the MLS shutter offers advantages such as lightweight, compact size, larger aperture, and no motion distortion.
M10Pro camera
- Inspection Camera
SF10 is a camera specially designed by Riebo for inspection data collection. It utilizes a medium-format CMOS sensor with over 100 megapixels and a long focal length of up to 120mm, making it capable of easily handling inspections to the highest standards.
Various parameters of SF10 inspection camera are adjustable, allowing it to adapt to the operational requirements in various lighting conditions. It can capture high-quality inspection images even in darker conditions, such as under bridges with reduced illumination.
In consideration of the characteristics of inspection operations, Riebo has also optimized image algorithms and adopted more appropriate metering modes, ensuring that there won't be issues of underexposure even in areas with strong contrast in lighting conditions under the bridge edges.
The use of a long focal length allows for an increased relative distance between the drone and the inspection object while maintaining spatial resolution. This adaptation is valuable in various complex situations, ensuring flight safety.
SF10 long focal length inspection camera
- Route planning software
Riebo Route Assistant (RRA) is a route planning software developed by Riebo technology. In addition to supporting the mesh, orbit, terrain following, and vertical route planning functions for the full range of Riebo cameras, it also features the capability to import DEM elevation files autonomously and automatically recognize elevation types. Furthermore, it offers one-click import and automatic alignment of ground features for 3D models.
The inspection function in RRA supports the automatic generation of inspection flight routes based on the 3D model. The route parameters are adjustable, ensuring comprehensive coverage of the inspection objects without omissions while maintaining flight safety. The algorithm-optimized route design can also significantly reduce redundant data, saving time in subsequent processing tasks.
RRA Route planning software
(5) Data Processing Software:
Data preprocessing is carried out using SkyScanner software. This software supports automatic grouping of data by missions and offers data management capabilities. It allows for precise georeferencing of location information to the camera CMOS center using calibration parameters, writing this information into the images. Furthermore, it supports the accurate directions of the image's FOV by combining camera distortion parameters and the camera's rotation information during photography. This data support aids in the precise positioning and measurement of damage points in subsequent analysis.
The aerial survey and modeling software market is currently quite mature, with multiple software options available that can be selected based on the specific needs of customers.
SkyScanner
4．Development Trends in the Inspection Industry
In China, dam safety has consistently been a matter of high importance to various levels of government. The country allocates special funds annually to reinforce and promptly address potential risks in reservoirs and dams, ensuring the safety of dams. Since the introduction of economic reforms and opening up, there hasn't been a single instance of dam failure in China.
In February of this year, the National Energy Administration initiated a two-year nationwide special operation for the improvement of dam safety at hydroelectric power stations. The objective is to further enhance the supervision and management of dam safety at hydroelectric power stations, conduct thorough inspections and rectifications of dam safety issues, and effectively raise the overall level of dam safety.
While China has maintained a good safety record for reservoirs and dams, we must be aware that dam safety is facing new challenges. On one hand, the recent global warming has led to frequent extreme weather events, with local heavy rainfall and floods posing threats to dam safety. On the other hand, as the construction and development of hydropower bases progress, cascade reservoirs within river basins are gradually increasing in scale, which adds pressure to ensure dam safety.
With the continuous development of technologies like big data, mobile internet, artificial intelligence, and drones, the nation places a high emphasis on the application and integrated innovation of new technologies in the water resources industry. This has raised higher demands for the maintenance and operation management of water facilities. Under the guidance of national strategies, ministerial and provincial-level departments have put forward development objectives and guidance on the use of technologies such as artificial intelligence, automatic inspection, and drones in the water resources industry. Local governments have also issued documents to promote research and application in these areas. The alignment of national policy direction and the practical advancement of local governments will strongly drive the application of drone technology in dam inspection and monitoring, enhancing dam inspection efficiency and safety, advancing the modernization of water facility maintenance, and generating significant economic benefits.
