Graph Conv

– What is inductive bias ? – Inductive bias of a learning algorithm is the set of assumptions that the learner uses to predict outputs given inputs that it has not encountered. – Neural Networks tend to discard the features that have very less significance in task at hand. – Similary, GNNs have bias: By structuring the data in a way that prioritizes certain patterns, we can improve model performance, even if the data is the same. – Graphs are more abstract, and variables like node degree, proximity, and neighborhood structure provide far more information about the data

– Convolutions are basically pooling, kernels and strides. As like in euclidean space, where we have notion of left right, and so on ; we can’t use such concepts in graphs due to their irregular structure.

– So, to perform convolution; we need to start from what convolution means so that we can mimic the process At a high level, convolutions aggregate information from surrounding or adjacent entities. Convolutions in Deep Learning take this aggregated information to build feature maps (the grid values), which are used to make predictions with a neural network. We want to do this on graphs.

– Now we know what are we trying to do. So, we need a method to do this now. – Spatial and Spectral methods: One that requires Eigen and another that doesn’take

– we will have covariance matrix of order ’d’ by ’d’ and mean vector of order 1 by ’d’ for the kernel. – patch operator weighting functions (fancy way of saying kernel) used in different convolutional operations on the manifold. ( the heatmap figure of 3d models) – This kernel is learned only in MoNet, versus hard-coded in GCNNs and ACNNs. The first thing one might notice is how much “smoother” the weights (red curves) are in MoNet, versus the more distinct (GCNN) and more integrated (ACNN) weights of the other approaches. – This flexibility is part of the reason why both allow for the sharing of parameters cross-dataset. This opens up the capacity for ideas like Transfer Learning to be applied to the geometric domain.

– Spectral convolutions are mathematically as well as theoretically similar to their vanilla convolution counterparts whereas spatial convolutions are conceptually as well as markedly similar to vanilla convolutions.