Research Project D03

This project's goal is a statistical framework for the comparative evaluation of the SFB's computational models through uncertainty-aware validation benchmarks. Here, the main challenge arises from the possibly large uncertainties present in the experimental data and the simulation results. A so-called validation metric that compares system response quantities of an experiment with those from a computational model has to incorporate parameter and conceptual uncertainties rigorously.

Bayesian validation framework

To assess the overall accuracy of the computational models participating in the benchmarks under uncertainty of both the simulation results and the experimental data, we have developed a Bayesian calibration and validation framework that incorporates a probabilistic modelling technique quantifying remaining post-calibration uncertainty. Figure 1 shows an overview of this framework. Within this framework, the parametric uncertainty of a computational model, based on the modeler's expert knowledge, is propagated to obtain a so-called prior predictive distribution. Then, in the calibration step, a posterior knowledge is achieved by updating the prior belief based on Bayesian notions, incorporating all sources of errors in the experiment and the model. During this process, global sensitivity indices can be calculated that rank the influence of the uncertain model parameters.

In the validation of a single model, the hypothesis is whether this model can satisfactorily represent the real system of interest. The calibrated model's results are compared to a new unseen set of experimental data, corresponding to an evaluation of the validation metric. Moreover, several models might exist with different approaches and assumptions to analyse the occurring processes. For this case, a comparative validation is performed, where the hypothesis is which model within the pool of available models can make the best prediction regarding the observed values in the experiments. To provide a quantitative validation metric and an objective model ranking, we use Bayesian model evidence employing the so-called Bayes factor introduced in Bayesian hypothesis testing.

Propagating the parametric uncertainty through the given computationally demanding models in the framework is not feasible via the Monte Carlo or even Markov chain Monte Carlo approaches for the competing physical models. Therefore, one main focus of the project has been to substitute the original computational models with easy-to-evaluate surrogates, based on recent developments in the theory of polynomial chaos expansion (PCE). We used a data-driven arbitrary PCE (aPCE), which can operate with probability measures implicitly and incompletely defined via statistical moments. We have extended aPCE to a Bayesian sparse aPCE (BsaPCE) representation through the Bayesian sparse learning using a fast marginal likelihood maximisation algorithm. Further, we have introduced a sampling strategy to sequentially refine the surrogate model to avoid clusters in specific regions of the polynomial representation.

Project Area B

In a model comparison study resulting from a successful collaboration between SFB members within Project Area B during the first SFB 1313 summer school in September 2019, we have compared two models describing flow in fractured porous media against experiments. The first investigated model, denoted by B01, originated from Project B01-Keip and employed a phase-field representation of fractures and a finite-element discretisation. The second model, indicated by B03, from B03-Rohde used a discrete-fracture-matrix method and a finite-volume discretisation. We have analysed these two models against microfluidic experiments performed by B05-Steeb/Nowak.

Figure 2 illustrates the Bayes factor's distribution as a key measure obtained from applying our validation framework to the benchmark case described above. It also shows the related significance levels with vertical solid lines. We conclude that both models are equivalent in the sense that there is no substantial evidence to prefer one over the other based on their predictive capability for the selected scenario because the probability density distribution of the Bayes factor is close to one.

Apart from applying the Bayesian validation framework to individual projects, other substantial benchmarking efforts have been performed in the context of Project Area B. In preliminary work, verification benchmark cases for single-phase flow in two-dimensional fractured porous media have been defined, and the performance of several discretisation methods has been evaluated. By distributing the CFP and organising a corresponding mini-symposium at the SIAM GS 2019 in March 2019, a group of 26 participants representing 17 different discrete-fracture-matrix methods formed, which led to a publication.

Project Area C

We have recently performed a Bayesian assessment of computational models describing geochemical processes in subsurface reservoirs. These models predict the change in material properties of porous media due to microbial activities. This study assessed a full complexity microbially induced calcite precipitation model (M_FC) along with two simplifications.  The first simplified model is denoted as initial biofilm model (M_IB), considering that the biofilm is already established at the beginning. The second simplified model assumes that all the urea injected into the system precipitates as calcite, and it is denoted as simple chemistry model (M_SC). We have used our framework components to perform a Bayesian justifiability analysis to judge the competing models' performance against the available experimental data (MD) and assess similarities between models.

Figure 3 illustrates the model ranking against the experimental data and their predictions using so-called model weights for predicting the calcium concentration. The first columns of the so-called model confusion matrices show the competing models' ability (M_FC, M_IB and M_SC) to replicate the experimental data (MD). Other entries indicate the probability that the model M_k (rows) is the data-generating process of the predictions made by the model M_l (columns). The off-diagonal entries indicate similarities between the models, where the initial biofilm model M_IB has been identified to be similar to the full complexity model M_FC. The main-diagonal entries represent the models’ ability to identify their own predictions. Our analysis indicated that the simple chemistry model M_SC and the full complexity model M_FC identify themselves best. We also observed that the experimental data set size is too small to justify the initial biofilm model M_IB in terms of the calcium concentration.

In the context of thermochemical heat storage, a computational model for an indirectly operated CaO/Ca(OH)₂ reactor was calibrated and validated using our framework. By employing Bayesian inference, the reaction rate's decrease with progressing conversion of reactive material could be identified to be essential for the desired match with experimental results. A correspondingly calibrated model revealed that more heat is lost over the reactor surface than transported in the heat transfer channel, which caused a considerable speed-up of the discharge reaction. We validated the calibrated model with the second set of experimental results. The computational model was replaced by a surrogate based on PCE and principal component analysis.

We were also involved in developing an interactive demonstrator for geothermal energy sites using the on-the-fly CUDA-based evaluation of aPCE surrogates. The interactive demonstrator was presented in SFB 1313's public science exhibition "Pretty Porous - Alles Porös", shown at the Planetarium Stuttgart from 18 June to 31 August 2020. Visitors could interactively explore the underground heat transport at a geothermal energy site modifying its properties via a touch screen.

Project Area A

We have initiated an uncertainty-aware validation benchmark to address the question of how to appropriately conceptualise the interface conditions and related modelling parameters for coupling free flow and porous-medium flow. The considered computational models consist of the coupled Stokes-Darcy model with the classical set of interface conditions, the pore-network model developed in A02-Helmig/Weigand, and the generalised interface conditions for the REV-scale model from A03-Rybak. We defined data extraction points for velocity and pressure for both the calibration and the validation phase concerning system response quantities. As the data-generating reference solution process, we considered a pore-scale resolved model.

With the assessment of the pore-network model, we illustrate briefly one particular component in the following. Figure 4 illustrates preliminary results, where the pore-network model captures the reference process very accurately in the porous medium and as well in the free-flow domain. We evaluated the two REV-scale models in a similar manner and quantified the influence of the uncertain modelling parameters in terms of Bayes factors.

We will continue collaborating with A03-Rybak and A02-Helmig/Weigand concerning the comparison and validation of models for coupling free flow with the porous-medium flow. In particular, we plan to conduct an open benchmarking process based on the existing single-phase cases and define new two-phase cases with the help of the experimental data from A06-Lamanna/Poser and our external partners at Utrecht University.