神经网络解常微分方程

神经网络解常微分方程

对于常微分方程

$$
\left{
\begin{array}{c}
\frac{dy}{dx}=x^3-\frac{y}{x}\\
y(1)=\frac{2}{5}
\end{array}
\right.
$$
其精确解为 $y(x)=\frac{x^4}{5}+\frac{1}{5x}$ .

下面用3层 1xNx1 神经网络解该问题。

设 $f(x)=y_t=\frac{dy}{dx} = y_0+xN(x,p)$ , 其中 $N(x,p)$ 是三层神经网络的输出, $p$ 是网络中的参数,维度为 $3n$ 。在区间 $[1,2]$ 中取 m 个点做训练数据。求解最优化问题
$$
min\sum_{i=1}^m{ \frac{dy_t(x,p)}{dx}|_{x=x_i}-f(x_i,y_t(x_i,p)) }^2
$$
将得到的 $p$ 带入到 $y_t(x,p)$ 中就能得出训练出来的该微分方程的近似解。

通过计算可以得出上式平方内的项为:
$$
J(x_i) = \sum_{j=1}^n\frac{2v_j}{1+e^{\theta_j-\omega_jx_i}}+\sum_{j=1}^n\frac{xv_j\omega_je^{\theta_j-\omega_jx_i}}{(1+e^{\theta_j-\omega_jx_i})^2}-x_i^3+\frac{0.4}{x_i}
$$
给出表示上述函数及其导数的代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
function f = fnde(p)
%% Loss function of neural network method to solve differential equation
% a 1-n-1 3 layer neural network
% p is 3n dimensional parameter column vector
m = 10; % points selected in interval [1,2]
xs = [1:1/(m-1):2]';
f = 0;
for iter = 1:length(xs)
ftmp = finner(xs(iter), p);
f = f + ftmp^2;
end
end
function fx = finner(x, p)
% x 1 dimensional number
% p is 3n dimensional parameter column vector
% f is 1 dim number
n = length(p)/3;
v = p(1:n);
theta = p(n+1: 2*n);
omega = p(2*n+1 : end);
z = x*omega - theta;
fx = 2*v'*sigmoid(z) + x*(v.*omega)'*(exp(-z).*(sigmoid(z).^2)) - x^3 + 2/5/x;
end
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
%% gnde: Gradient of loss function of neural network method to solve differential equation
function grad = gnde(p)
m = 10; % points selected in interval [1,2]
xs = [1:1/(m-1):2]';
grad = zeros(length(p), 1);
for iter = 1:length(xs)
gradtmp = ginner(xs(iter), p);
grad = grad + gradtmp;
end
end
function [gx] = ginner(x, p)
n = length(p)/3;
v = p(1:n);
theta = p(n+1: 2*n);
omega = p(2*n+1 : end);
fx = finner(x, p); % 1 dim
z = x*omega - theta;
d_v = 2*sigmoid(z) + x*omega.*exp(-z).*(sigmoid(z).^2);
d_theta = (x*v.*omega - 2*v).*exp(-z).*(sigmoid(z).^2) - ...
2*x*v.*omega.*exp(-2*z).*(sigmoid(z).^3);
d_omega = (3*x*v - x^2*v.*omega).*exp(-z).*(sigmoid(z).^2) + ...
2*x^2*v.*omega.*exp(-2*z).*(sigmoid(z).^3);
gx = [d_v; d_theta; d_omega];
gx = 2*fx*gx;
end
function fx = finner(x, p)
% x 1 dimensional number
% p is 3n dimensional parameter column vector
% f is 1 dim number
n = length(p)/3;
v = p(1:n);
theta = p(n+1: 2*n);
omega = p(2*n+1 : end);
z = x*omega - theta;
fx = 2*v'*sigmoid(z) + x*(v.*omega)'*(exp(-z).*(sigmoid(z).^2)) - x^3 + 2/5/x;
end

利用拟牛方法计算参数 $p$ 的值并画图和真实解析解进行比较:

fnde