A Performance Evaluation Framework for Continuous Methane Monitoring: Effects of Event Association and Metric Choice

Методика оценки эффективности непрерывного мониторинга метана: влияние сопоставления событий и выбора метрик
Ziliang Mao, Penggang Zheng, Bo Fu, Shan Zhou
2026-09-02

90% detection limitcontinuous methane monitoringevent associationleak detection and quantificationquantification error metrics
Abstract Continuous monitoring (CM) methane leak detection and quantification (LDAQ) systems provide high-time-resolution information on intermittent and persistent emissions, but the reported performance of these systems depends on how their outputs are evaluated. Reported detection events must first be associated with actual emission events before detection classification, detection limit, localization accuracy, and quantification error can be interpreted. Here, we develop and apply a performance evaluation framework for CM methane LDAQ systems using a three-month single-blind controlled-release dataset. The framework defines rules for temporal association, spatial association, concurrent emissions, and excess reported detections, and introduces complementary event-level metrics for quantification agreement, reporting frequency, temporal precision, temporal coverage, and start-time identification. We compared the resulting performance interpretation with two existing protocols when applicable. For the same actual emission and reported detection events, different event association rules change true positive (TP), false positive, and false negative classifications, leading to different 90% detection limit values. Quantification interpretation also depends on metric choice: mean relative error can suggest overestimation even when most individual TP events are underestimated, whereas the bounded Q metric provides a complementary fold-error summary. Event-level metrics further show that TP events differ in temporal coverage and start-time identification. These results demonstrate that CM performance should be interpreted together with the evaluation rules and metric definitions.
1
A performance evaluation framework for continuous methane leak detection and quantification systems was developed using a three-month single-blind controlled-release dataset.
2
Different event-association rules alter true-positive, false-positive, and false-negative classifications and consequently produce different 90% detection-limit values.
3
Event-level metrics reveal differences among true-positive events in temporal coverage and start-time identification, demonstrating that performance results must be interpreted alongside evaluation rules and metric definitions.
4
Quantification conclusions depend on the selected metric: mean relative error may indicate overestimation even when most true-positive events are underestimated, while the bounded Q metric summarizes fold-error agreement.
5
The framework specifies temporal and spatial event association, treatment of concurrent emissions, and handling of excess reported detections before performance classification.

continuous methane leak detection and quantification (LDAQ) systems

the effects of event-association rules and metric choice on performance evaluation, including detection classification, detection limits, quantification error, temporal coverage, and start-time identification

Publication Details
Publication Date
2026-09-02
Journal
Publisher
ISSN
Cited by
0
Access Type
Author Information
Authors
Ziliang Mao
Penggang Zheng
Bo Fu
Shan Zhou
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%