Novel deep neural networks for inspection of solar panel anomalies using multimodal images/videos
The Project is funded by Sarkis and Nune Sepetjian.
From January 2025 the project is funded and administered by the Higher Education and Science Committee of Armenia.
Principal Investigator: Prof. Sos Agaian
University: City University of New York
Research team: Hayk Gasparyan, Sargis Hovhannisyan, Hrach Ayunts, , Sargis Grigoryan, Armine Bayramyan, Diana Sargsyan, Narek Sardaryan
Contributing Researchers: Anna Hovakimyan, David Shaduts, Tatevik Davtyan, Anna Ohanyan, Hasmik Mkhoyan, Ejmin Vartoumian, Karen Tatalyan, Anush Khachatryan, David Shadunts, Arthur Osipyan
Duration: 2022-2026
Project Importance
According to the latest reports, renewable energy sources such as photovoltaic (PV or solar panels) and wind energy systems will generate 88% of worldwide electricity demand by 2050. The installation of PV generation plants has been rapidly increasing every year, with PV capacity reaching 77 GW in the US, and over 500 GW worldwide.
The Armenian government has adopted several laws on developing domestic, especially renewable, energy resources and implementing energy efficiency measures. And with their global availability, reliability, easy installation, and pollution-free energy generation, renewable PV energy sources are rising in popularity in Armenia and abroad. However, PV modules usually suffer from temperature, rain, wind, and other environmental, damage and mechanical damage during transportation and installation, which could shorten their lifespan. In addition, the damage to PV modules could affect the entire PV system, leading to economic efficiency and energy loss problems as growing plant sizes render manual inspection impractical. So, fast, reliable, automatic, regularly nondestructive inspection, and monitoring and maintaining PV modules is essential for an efficient operation with minimal energy loss and maximal lifespan.
Remote sensing of PV has been addressing these pain points through different techniques such as electroluminescence (EL) images, infrared (IR) images, and color image techniques. These methods provide a fast capture of PV images but require specialists to inspect the obtained images visually, and have a long processing time due to a large amount of data. As a result, intensive research is required to speed up damage inspection and localization in PV images. Multimodal images based on machine learning methods face three key challenges: large variations in the visual appearance of objects, complex background noise, and small geographic objects in high-resolution images. Moreover, collecting labeled data is expensive, often requires experts, and scales poorly with the number of tasks. And even though traditional approaches work well for few-shot learning, they are likely to ignore the spatial information encoded within feature maps, making the model very sensitive to background clutter on image examples. Therefore, it is essential to research few-shot learning for multimodal drone sensing image analysis and interpretation.
Expected Results
The project will significantly impact the development of AI technology with application in Armenia.
Extend knowledge on creating solar panels' multimodal-multi camera analysis and management systems.
Generate new knowledge and information about special damages caused to solar panels by Armenia’s climate, dust, and wind.
Better understand the complexities of renewable PV energy sources. Gain large-scale knowledge about deep neural networks for detection, localization, and classification of solar panel damages using multimodal images/videos.
Deliver baseline data for subsequent targeted studies using more specialized sampling and study designs such as mark/recapture and patch occupancy.
Learn how to use known databases, generate so-called Armenian baseline/relevant data and information, and identify the gaps to run such models for Armenia.
Increase practical experience in creating novel machine learning modules, including transformers and a few-shot learning system.
Transform the developed technology and database accurately and inexpensively, and catalyze many fields of ecology, renewable PV energy, global warming, and carbon dioxide emission into "big data" sciences.
Publish several research papers in international top machine learning and computer vision journals.
