5 posts tagged with "PINN"

这是一篇使用 PINN 用于通用生物反应器建模的论文。

我们先来看看这篇论文的摘要：

摘要​

英文原文：

Many previous studies have explored hybrid semiparametric models merging Artificial Neural Networks (ANNs) with mechanistic models for bioprocess applications. More recently, Physics-Informed Neural Networks (PINNs) have emerged as promising alternatives. Both approaches seek to incorporate prior knowledge in ANN models, thereby decreasing data dependency whilst improving model transparency and generalization capacity. In the case of hybrid semiparametric modelling, the mechanistic equations are hard coded directly into the model structure in interaction with the ANN. In the case of PINNs, the same mechanistic equations must be “learned” by the ANN structure during the training. This study evaluates a dual-ANN PINN structure for generic bioreactor problems that decouples state and reaction kinetics parameterization. Furthermore, the dual-ANN PINN is benchmarked against the general hybrid semiparametric bioreactor model under comparable prior knowledge scenarios across 2 case studies. Our findings show that the dual-ANN PINN can level the prediction accuracy of hybrid semiparametric models for simple problems. However, its performance degrades significantly when applied to extended temporal extrapolation or to complex problems involving high-dimensional process states subject to time-varying control inputs. The latter is primarily due to the more complex multi-objective training of the dual-ANN PINN structure and to physics-based extrapolation errors beyond the training domain.

翻译：

许多先前的研究探索了将人工神经网络（ANN）与机制模型融合的混合半参数模型，用于生物过程应用。近年来，物理信息神经网络（PINN）作为有前景的替代方案应运而生。这两种方法都试图将先验知识融入 ANN 模型，从而降低对数据的依赖性，同时提高模型的透明度和泛化能力。在混合半参数建模中，机理方程被直接硬编码到模型结构中并与 ANN 交互作用；而在 PINN 中，相同机理方程需由 ANN 结构在训练过程中“学习”获得。本研究针对通用生物反应器问题评估了一种双 ANN-PINN 结构，该结构实现了状态参数与反应动力学参数的解耦。此外，通过两个案例研究，在可比先验知识情境下，将双 ANN PINN 模型与通用混合半参数化生物反应器模型进行基准对比。研究发现：对于简单问题，双 ANN-PINN 可达到与混合半参数模型相当的预测精度；但在扩展时间外推或涉及高维过程状态与时变控制输入的复杂问题中，其性能显著下降。后者主要源于双 ANN-PINN 结构更复杂的多目标训练过程，以及训练域外基于物理的外推误差。

通过摘要可以看出，文章利用 PINN 来求解一类同样由微分方程描述的生物反应器系统。这确实展示了 PINN 在传统物理模型之外的又一类应用场景——它同样能够有效处理其他类型的微分方程。至于该方法的具体实现方式，感兴趣的读者可以继续阅读下文以进一步了解。

这是一篇关于处理变系数边界问题的 PINN 论文。

摘要​

摘要原文：

This paper proposes a novel and general framework of the discontinuity-removing physicsinformed neural networks (DR-PINNs) for addressing elliptic interface problems. In the DR-PINNs, the solution is split into a smooth component and a non-smooth component, each represented by a separate network surrogate that can be trained either independently or together. The decoupling strategy involves training the two components sequentially. The first network handles the non-smooth part and pre-learns partial or full jumps to assist the second network in learning the complementary PDE conditions. Three decoupling strategies of handling interface problems are built by removing some jumps and incorporating cusp-capturing techniques. On the other hand, the decoupled approaches rely heavily on the cusp-enforced level-set function and are less efficient due to the need for two separate training stages. To overcome these limitations, a novel DR-PINN coupled approach is proposed in this work, where both components learn complementary conditions simultaneously in an integrated single network, eliminating the need for cusp-enforced level-set functions. Furthermore, the stability and accuracy of training are enhanced by an innovative architecture of the lightweight feedforward neural network (FNN) and a powerful geodesic acceleration Levenberg-Marquardt (gd-LM) optimizer. Several numerical experiments illustrate the effectiveness and great potential of the proposed method, with accuracy outperforming most deep neural network approaches and achieving the state-of-the-art results.

摘要翻译：

本文提出了一种新型且通用的断续性消除物理信息神经网络（DR-PINNs）框架，用于解决椭圆型界面问题。在 DR-PINNs 中，解被分解为光滑部分和非光滑部分，每个部分由独立的神经网络代理表示，这些代理可以单独训练或共同训练。解耦策略涉及依次训练这两个部分。第一个网络处理非光滑部分，并预先学习部分或全部跳跃，以协助第二个网络学习互补的偏微分方程（PDE）条件。通过去除部分跳跃并结合尖点捕获技术，构建了三种处理界面问题的解耦策略。另一方面，解耦方法高度依赖于尖点强制水平集函数，且由于需要两个独立的训练阶段而效率较低。为克服这些局限性，本文提出了一种新型 DR-PINN 耦合方法，其中两个组件在集成单一网络中同时学习互补条件，消除了对尖点强制水平集函数的需求。此外，通过轻量级前馈神经网络（FNN）的创新架构和强大的几何加速 Levenberg-Marquardt（gd-LM）优化器，训练的稳定性和准确性得到提升。多个数值实验验证了所提方法的有效性和巨大潜力，其精度显著优于现有方法。

这是一篇开山之作提出了一个深度学习框架 DeepONet 用于求解偏微分方程的求解器，这篇论文介绍了原理。

摘要​

摘要原文：

While it is widely known that neural networks are universal approximators of continuous functions, a less known and perhaps more powerful result is that a neural network with a single hidden layer can approximate accurately any nonlinear continuous operator [5]. This universal approximation theorem is suggestive of the potential application of neural networks in learning nonlinear operators from data. However, the theorem guarantees only a small approximation error for a sufficient large network, and does not consider the important optimization and generalization errors. To realize this theorem in practice, we propose deep operator networks (DeepONets) to learn operators accurately and efficiently from a relatively small dataset. A DeepONet consists of two sub-networks, one for encoding the input function at a fixed number of sensors $x_i = 1, ... , m$ (branch net), and another for encoding the locations for the output functions (trunk net). We perform systematic simulations for identifying two types of operators, i.e., dynamic systems and partial differential equations, and demonstrate that DeepONet significantly reduces the generalization error compared to the fully-connected networks. We also derive theoretically the dependence of the approximation error in terms of the number of sensors (where the input function is defined) as well as the input function type, and we verify the theorem with computational results. More importantly, we observe high-order error convergence in our computational tests, namely polynomial rates (from half order to fourth order) and even exponential convergence with respect to the training dataset size.

摘要翻译：

尽管神经网络是连续函数的通用逼近器这一事实广为人知，但一个较少为人所知且可能更强大的结果是：具有单个隐藏层的神经网络能够精确逼近任何非线性连续算子。这一通用逼近定理暗示了神经网络在从数据中学习非线性算子方面的潜在应用。然而，该定理仅保证在网络规模足够大时存在较小的逼近误差，并未考虑重要的优化误差和泛化误差。为了在实践中实现这一定理，我们提出**深度算子网络（DeepONets）**以从相对较小的数据集准确高效地学习算子。一个 DeepONet 由两个子网络组成：一个用于在固定数量的传感器上编码输入函数 $x_i = 1, ..., m$（分支网络），另一个用于编码输出函数的位置（主干网络）。我们通过系统性模拟识别两种类型的算子，即动态系统和偏微分方程，并证明 DeepONet 相较于全连接网络显著降低了泛化误差。我们还从理论上推导了近似误差与传感器数量（即输入函数定义的传感器数量）以及输入函数类型之间的依赖关系，并通过计算结果验证了该定理。更重要的是，我们在计算测试中观察到高阶误差收敛，即多项式收敛率（从半阶到四阶）甚至与训练数据集大小相关的指数收敛。

这是一篇使用谱元方法与 PINN（物理信息神经网络）相结合的论文。

摘要​

摘要原文：

Spectral methods are an important part of scientific computing’s arsenal for solving partial differential equations (PDEs). However, their applicability and effectiveness depend crucially on the choice of basis functions used to expand the solution of a PDE. The last decade has seen the emergence of deep learning as a strong contender in providing efficient representations of complex functions. In the current work, we present an approach for combining deep neural networks with spectral methods to solve PDEs. In particular, we use a deep learning technique known as the Deep Operator Network (DeepONet) to identify candidate functions on which to expand the solution of PDEs. We have devised an approach that uses the candidate functions provided by the DeepONet as a starting point to construct a set of functions that have the following properties: (1) they constitute a basis, (2) they are orthonormal, and (3) they are hierarchical, i.e., akin to Fourier series or orthogonal polynomials. We have exploited the favorable properties of our custom-made basis functions to both study their approximation capability and use them to expand the solution of linear and nonlinear time-dependent PDEs. The proposed approach advances the state of the art and versatility of spectral methods and, more generally, promotes the synergy between traditional scientific computing and machine learning.

摘要翻译：

谱方法是科学计算用于求解偏微分方程 (PDE) 的重要工具。然而，它们的适用性和有效性在很大程度上取决于用于扩展偏微分方程解的基函数的选择。近十年来，深度学习异军突起，成为提供复杂函数高效表示的有力竞争者。在当前的工作中，我们提出了一种将深度神经网络与光谱方法相结合来求解 PDE 的方法。特别是，我们使用一种被称为深度算子网络（DeepONet）的深度学习技术来识别候选函数，并在此基础上扩展 PDE 的求解。我们设计了一种方法，以 DeepONet 提供的候选函数为起点，构建一组具有以下特性的函数：(1) 它们构成一个基础；(2) 它们是正交的；(3) 它们是分层的，即类似于傅里叶级数或正交多项式。我们利用定制基函数的有利特性，研究了它们的近似能力，并利用它们扩展了线性和非线性时变 PDE 的解。所提出的方法推进了频谱方法的技术水平和多功能性，更广泛地说，促进了传统科学计算与机器学习之间的协同作用。

其实只要细读了他的摘要就可以发现这篇论文的做法比较独特，他并不是把机器学习与谱方法相融合，而是采取了一种分阶段的模式，第一步是使用 DeepONet 去识别候选函数，再利用常规的谱方法来完成剩下的计算工作，他并不是一整块的模型而是耦合度低的分阶段模型。这种思想其实结合误差达到 $10^{-4}$ 到 $10^{-8}$ 就能够理解，不过也算是给机器学习融合谱方法提供了一种思路，接下来我们详细的阅读这篇文章的结果以及方法。

这是一篇关于使用数据驱动方法实现的 Physics-Informed Deep Learning（PINN）经典论文。

论文的来源​

    首先，本人通过搜索很多 PINN 的论文，发现许多论文都在引用这篇论文，在好奇心的驱使下就在 google 学术上搜索了这篇论文，我们可以看到出现了两个版本，从标题名上看大致相同，作者也没变化。据开组会时，覃老师介绍说可能是因为前面这个版本是相当于没有正式发表还处于一个草稿阶段，后面那篇是经过整理并发表到了比较好的期刊中，我们可以从引用量(比较粗的红线)以及 easyScholar (比较细的红线)打上的标签还有作者希望我们引用这项工作的论文排名(作者更希望我们引用 2019 年正式分布的那篇)中看到区别，但不妨碍这几篇论文的优秀性，总的来说 M Raissi 等人的工作是非常出色的。