Evaluation of Seismic Artificial Intelligence with Uncertainty

Оценка искусственного интеллекта в сейсмологии с учетом неопределенности
Dave Higdon, Nidhi Parikh, Samuel Myren, Rosalyn Rael, Garrison S. Flynn, Emily Casleton
2025-07-29

PhaseNetlearning efficiencyperformance uncertaintyseismic artificial intelligenceseismic phase picking
Abstract Artificial intelligence has transformed the seismic community with deep learning models (DLMs) that are trained to complete specific tasks within workflows. However, there is still a lack of robust evaluation frameworks for evaluating and comparing DLMs. We address this gap by designing an evaluation framework that jointly incorporates two crucial aspects: performance uncertainty and learning efficiency. To target these aspects, we meticulously construct the training, validation, and test splits using a clustering method tailored to seismic data and enact an expansive training design to segregate performance uncertainty arising from stochastic training processes and random data sampling. The framework’s ability to guard against misleading declarations of model superiority is demonstrated through the evaluation of PhaseNet (Zhu and Beroza, 2018), a popular seismic phase picking DLM, under three training approaches. Our framework helps practitioners choose the best model for their problem and set performance expectations by explicitly analyzing model performance with uncertainty at varying budgets of training data.
1
An expansive training design separates performance uncertainty caused by stochastic optimization from uncertainty caused by random data sampling.
2
Clustering-based construction of training, validation, and test splits is used to better account for seismic-data characteristics.
3
Evaluation of PhaseNet under three training approaches demonstrates that the framework can prevent misleading claims of model superiority.
4
The framework supports model selection and performance expectation setting across different training-data budgets by explicitly quantifying uncertainty.
5
The paper introduces a seismic-specific evaluation framework jointly addressing model performance uncertainty and learning efficiency.

seismic deep learning models (DLMs), exemplified by PhaseNet

performance uncertainty and learning efficiency in training and evaluating seismic DLMs

Publication Details
Publication Date
2025-07-29
Journal
Publisher
ISSN
Cited by
8
Access Type
Author Information
Authors
Dave Higdon
Nidhi Parikh
Samuel Myren
Rosalyn Rael
Garrison S. Flynn
Emily Casleton
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%