China is a major bridge-building nation, often hailed as the "homeland of bridges" in the world. It boasts an impressive inventory of over one million bridges and possesses cutting-edge bridge construction technologies. However, concurrently, China faces the pressing issue of a substantial number of deteriorating bridges, periodic bridge collapses, and an increasingly severe bridge safety situation.
Research conducted by the Highway Research Institute of the Ministry of Transport reveals that more than 10% of China's highway bridges are categorized as hazardous, and on average, they require significant repairs or complete reconstruction after 30 years of use.
Consequently, ensuring the scientific, efficient maintenance and emergency response of bridges has become a top priority in the current stage of transportation initiatives.
Bridge collapse
Currently, most of the bridge inspection work still relies on traditional manual methods. It primarily depends on equipment such as bridge inspection vehicles and aerial work platforms to provide a working platform, assisting operators in approaching the areas to be inspected for visual examination. The manual inspection methods primarily suffer from the following issues:
(1) Inspection personnel may cause misdiagnosis, missed detection, and incorrect detection due to technical reasons, personal experience, and a lack of responsibility.
(2) Conventional inspection methods have limitations in areas such as high bridge piers, bridge towers, or bridge components that are inaccessible to bridge inspection vehicles. The inspection of these areas is very challenging, and there may be some blind spots in the detection.
(3) Renting inspection vehicles is costly, inefficient, and involves safety risks.
(4) Data acquisition is not comprehensive, image data quality is poor, and positioning and measurement accuracy are low.
(5) The level of digitization and integration is low, and the results are not intuitive, making it difficult to review inspection results.
Manual bridge inspection
With the improvement of the national economic level, the acceleration of infrastructure construction, and the continuous enhancement of technological innovation capabilities, the technical methods for inspection and monitoring in various industries have greatly improved.
However, in the field of bridge inspection, the use of drones is still in a spontaneous, decentralized, and internal state, lacking systematic top-level planning and effective integration into the management system. In terms of the industry development level of drone application technology, the use of drones in bridge inspection is still in its infancy, with a considerable gap compared to more mature industries. Therefore, the construction of a digital, intelligent, and efficient drone bridge inspection system is of significant necessity and urgency.
Based on these circumstances, Riebo Technology has combined the core technology and extensive full-process R&D experience in drones, visible light photogrammetry, and LiDAR to develop a full-process inspection system based on digital twin technology. Riebo launches unique drone inspection solution.
Drone Bridge Inspection
1. Riebo Bridge Inspection Solution
Regarding the challenges and pain points associated with traditional manual inspections, Riebo's drone inspection solution offers the following advantages:
(1) Drones can easily reach various parts of the bridge, combined with digital twin simulation and full-coverage inspections, addressing the issues of oversight and blind spots in manual inspections.
(2) With high-resolution inspection images and the use of AI identification along with manual verification, the identification of defects becomes more accurate and comprehensive.
(3) The civilian drone market is mature, with low learning, usage, and maintenance costs. Drones are user-friendly, flexible to deploy, and highly efficient, allowing single operators to complete inspection tasks.
(4) Equipped with high-precision self-positioning technology, professional inspection cameras on drones can capture high-definition images and spatial locations of defects, achieving millimeter-level defect measurement accuracy.
(5) The combination of digital twin technology and inspection management systems provides t3D visualization and analysis of defect data, offering a more accurate and comprehensive basis for scientific and efficient management and maintenance.
The specific process of the solution is as follows:
Step 1: Creating a 3D model to establish a digital twin of the inspection target.
Step 2: Planning the inspection mission based on the 3D model.
Step 3: Utilizing drones to collect high-resolution image data following the planned mission.
Step 4: Data processing by Riebo's self-developed Skyscanner data processing software.
Step 5: Employing AI for defect identification.
Step 6: Using the Ribeo Smart Inspect Master System to visualize, manage, analyze, and generate inspection reports for defect points in three dimensions.
Inspection Solution
Step 1: Digital Twin
During the first bridge inspection, the digital twin of the bridge is realized by establishing a 3D model to restore the real situation of the bridge, and the reconstruction accuracy can reach centimeter level.
3D realistic modeling of bridges (overall)
3D realistic model of the bridge (partial)
There are two main uses of the 3D modeling:
(1) The 3D modeling serves as a digital base map for 3D visualization and management of defect data. It allows maintenance and operation managers to visually assess the actual condition of the bridge by displaying and managing all identified defects using the model.
(2) The 3D modeling serves as the basis for planning inspection routes, enabling fully autonomous drone flights.
In response to the structure and characteristics of bridges, Riebo Technology, combining its own technology and experience, selected its M-series products. It also designed an inspection module within its self-developed mission planning software called Riebo Route Assistant (RRA), specifically for 3D modeling of bridges. This design not only ensures a high success rate in acquiring aerial image and model quality but also significantly reduces the number of flight routes and images, thereby greatly improving modeling efficiency.
Through practical testing, it has been demonstrated that for a bridge measuring 500 meters in length, data collection takes only 17 minutes, and the generation of a three-dimensional model requires only 94 minutes.
Step 2: Route Planning
Drone inspection differs from traditional drone aerial surveys. According to relevant inspection standards, the most stringent requirement is to detect concrete cracks as narrow as 0.1mm in width. Therefore, during inspections, drones need to get close to the inspection targets for photography. To ensure the safe operation of drones in complex environments and to ensure complete coverage of the objects being inspected, flight mission planning is necessary for drones to autonomously perform inspection tasks along predefined routes.
In the Riebo Route Assistant (RRA) route planning software, there is an inspection module that can automatically generate simulated flight route based on the 3D model. On average, it takes only 5 minutes to complete route planning for one bridge. When combined with the quick modeling technology mentioned earlier, it is possible to achieve modeling and inspection of typical scenarios within a single day for initial inspections.
Automatic generation of inspection routes
Step3: Data Capture
The collection of inspection image data is carried out by a camera mounted on the drone. Field operations require only one person, one vehicle, and one set of collection equipment to complete. Moreover, with optimized algorithms and the use of professional-grade inspection cameras with long focal lengths and medium formats, the operational efficiency is very high. Through testing, it has been demonstrated that for a 500 meters multi-span bridge, field inspection work can be completed in approximately 90 minutes.
inspection operation
Step4: Data Processing
Data processing is carried out using the SkyScanner software that comes with the complete range of Riebo cameras. This software allows the export of inspection data by sorties or in batches, enabling efficient management of large volumes of inspection data. Additionally, the software can embed geographical location information into the photos, achieving high-precision coordinate correction. It also calculates the image field of view based on the attitude information of the drone and camera, providing data support for accurate positioning of defect points in subsequent analysis.
Step5: AI Identification
With the advancement of image recognition algorithms based on convolutional neural networks (CNN) and deep learning, the application of artificial intelligence for defect detection using high-resolution inspection images has become more accessible and mature. The specific workflow is as follows:
Step One: Marking the collected image data and delivering it for training to a convolutional neural network model. The trained neural network model can quickly detect cracks in images. Ongoing improvements to the model and the collection of more training data collectively enhance the model's detection accuracy.
Step Two: For images where defects have been identified, a series of processes including enhancement, noise reduction, image segmentation, and edge detection are applied. This results in segmented images of cracks and pixel-level measurement results. By using deep learning-based optimized segmentation algorithms, high pixel-level overlap rates for cracks are achieved. The algorithm can provide accurate segmentation results even in the presence of fine cracks, complex backgrounds, and interference like artificial markings.
Simultaneously, identification is also carried out for other defects in the bridge, such as cracks in steel structure welds, corrosion, loosened bolts, and detachment, among other conditions.
Step6: Riebo Inspect Master
Currently, the majority of inspection operations result in inspection reports, and whether it pertains to specific defects or the overall condition of the bridge, the presentation is not intuitive. Additionally, the inspection process lacks effective digital management, resulting in low reusability of inspection results and difficulties in supervising and reviewing the inspection process. Some existing inspection software also suffers from low integration levels and non-universal data outcomes.
To address these issues, Riebo Technology has developed Riebo Inspect Master, which digitizes and centrally manages the entire inspection process. It integrates functions such as personnel management, data management, defect identification, defect localization, three-dimensional visual management of defects, historical defect comparisons, data statistical analysis, and customized inspection report generation, all in one comprehensive platform.
Rie bo Inspect Master
2. Inspection System
The entire system consists of two subsystems: the Acquisition System and the Management System. Specifically, the Acquisition System comprises the drone platform, modeling camera, inspection camera, route planning software, and data pre-processing software. The Management System is Riebo Inspect Master.
Collection System
The Acquisition System primarily utilizes a drone platform equipped with a gimbal camera, in combination with corresponding software, to accomplish the high-precision acquisition of positional information, high-resolution image data, and defect data for various bridge components. The system includes:
(1) Drone Platform
The drone platform used is the DJI M300/350 RTK multi-rotor drone. This drone comes with an integrated RTK module, supporting custom network RTK, which enables high-precision and accurate positioning. Its features of stable flight, six-axis environmental perception, and obstacle avoidance capabilities make it highly suitable for complex bridge inspection environments.
Modeling Camera
The modeling camera used is from Riebo's M-series product line. The M-series cameras are single-lens metric cameras developed in-house by Riebo and are primarily used in applications such as oblique photogrammetry, close-range photogrammetry, facade modeling, and ortho-image acquisition. The series includes a range of high-end products, including M4Pro, M6/M6pro, M10Pro, M15Pro, featuring both full-frame and medium-format sensors, focal lengths spanning from 18mm to 75mm, and all equipped with Riebo's patented MLE lens and MLS shutter.
The MLE lenses utilize aspherical lens elements while controlling the lens's volume. They also incorporate ultra-low dispersion (ED) lens elements to correct and eliminate spherical aberration-type imaging, effectively managing chromatic aberration, and keeping lens distortion within 0.03%. Compared to traditional photogrammetric lenses, this results in an overall increase in the number of matching points by approximately 10% to 20%. Additionally, the root mean square (RMS) reprojection error can be reduced by around 20%.
The MLS shutter is a distortion-free mechanical shutter specially developed by Riebo for aerial survey cameras. Unlike traditional photogrammetric cameras with rolling shutters, which can cause motion distortion, have a shorter shutter life, and are prone to freezing in low-temperature environments, the MLS shutter offers advantages such as being lightweight, compact, longer shutter life and being free from motion distortion.
(3) Inspection Camera
All camera parameters are adjustable, making it adaptable to various environmental lighting conditions. Even in relatively dark areas underneath bridges, it can capture high-quality inspection images.
To suit the characteristics of inspection tasks, Riebo has also optimized image algorithms and adopted more rational metering modes. This ensures that even in regions with strong contrasts in lighting conditions at the edges under the bridge, there won't be issues with underexposure.
The long focal length allows for maintaining spatial resolution while increasing the relative distance between the drone and the inspection object. That is crucial for ensuring flight safety in various complex situations.
Inspection Image
Bridge edge photometric comparison
(4) Route Planning Software
Riebo Route Assistant (RRA) is a route planning software developed in-house by Riebo. In addition to supporting four major categories of route planning functions for the full range of Riebo devices (grid, orbit, terrain follow, and vertical). Furthermore, it offers the capability to import three-dimensional models with a single click .
RRA's inspection module supports the automatic generation of inspection flight route based on three-dimensional models, with adjustable parameters for each path. This ensures complete and omission-free coverage of the inspection object while maintaining flight safety. The optimized route design achieved through algorithmic enhancements also significantly reduces redundant data, saving time in subsequent processing tasks.
RRA route planning software
According to the model, the aqueduct inspection route was generated
(5) Data Pre-processing Software
Data pre-preprocessing is performed using the Sky Scanner software. This software supports data management with automatic sortie grouping and the precise correction of location information to the center of the camera CMOS sensor, which is written into the photos using calibration parameters. It also incorporates camera distortion parameters and attitude information at the time of capture to accurately calculate the photo's field of view. This provides data support for precise positioning and measurement of defect points in subsequent analysis.
The market for modeling software is currently mature, offering multiple software options that can be chosen based on the specific needs of customers.
Skyscanner
3. Inspection Industry Development Trends
With the continuous development of technologies such as big data, mobile internet, artificial intelligence, and drones, the nation has placed high importance on the application and integration of new technologies in the transportation industry. This has led to increased demands and higher standards for highway maintenance and operational management. Guided by national strategies, ministries and provincial-level departments have outlined development goals and guidance for the application of technologies such as artificial intelligence, autonomous driving, and drones in transportation. Local governments have also issued documents to promote research and applications in these areas.
The alignment of national policies and the practical efforts of local governments will strongly propel the application of drone technology in highway inspection and monitoring. This will enhance the efficiency of highway maintenance, ensure safety, improve service levels, advance modernization in highway maintenance, and stimulate economic benefits.
