Forecasting weather using neural ODEs

• Machine learning to model dynamics of weather.
• Modeling the changes between time steps using Julia and SciML.

Neural ODEs for time series

Instead of a neural network way,

\begin{equation}y_{i}=f\left(t_{i} ;\theta\right)\end{equation}

we use a differential equation to represent the change.

\begin{equation}\frac{\partial y}{\partial t}=f(y ;\theta)\end{equation}

The goal is to learn the dynamics of change. This formulation can be seen as “neural network inside ODE”. Forward pass of NN == Initial Value Problem

\begin{equation} y(t_{0}) \end{equation}

is the input features and we replace hand crafted equations with a NN. So, forward pass gives us an entire trajectory in contrast to RNNs which give a single prediction in time at once.

So, it uses a trained neural network instead of equations.

\begin{equation}\frac{\partial y}{\partial t}=f^{*}(y(t))\end{equation}

• How to train network inside ODE

  Take gradient of θ w.r.t. loss we can train using standard gradient based methods. In Julia, we can use DiffEqFlux.jl for it.

References:

https://sebastiancallh.github.io/post/neural-ode-weather-forecast/ https://sebastiancallh.github.io/post/neural-ode-continuous-layer/