Efficient Nets:
-
Why all this ? – Scaling CNN’s only in one direction (eg depth only) will result in rapidly deteriorating gains relative to the computational increase needed.
– ResNet 1000 isn’t much more accurate than ResNet152 for example, as after 100 -150 layer’s gains rapidly drop off.
– Scaling depth, width and resolution all benifits quickly saturate so not at all possibility of sota.
-
We now know the problem so let’s do this instead: – In order to scale up efficiently, all dimensions of depth, width and resolution have to be scaled together, and there is an optimal balance for each dimension relative to the others. VOILA!!!
– after an extensive grid search derived the theoretically optimal formula of “compound scaling” using the following co-efficients:
Depth = 1.20
Width = 1.10
Resolution = 1.15
Only google can do this kind of computational search !! :(
-
By showing the existing heatmaps in different widths, depths and resolution, they show that their model captures more real image like features than some nonsense these NNs learn on the way to correct prediction. At last!! We can at least have our brothers in image recognition who detects images more naturally looking on the inside.
-
Now not just large GPUs can do it, my phone can load a model and use it properly. Thanks to reduced number of parameters. Thanks to Gooooogogoogogogogogle’s computational for having this grid search and the paper.
- Not just google, I need to experiment with how much to give to depth, width and resolution For example, when given 2^N computational resources, we could increase network depth by α^N, width by β^N, and image size by γ^N, where α, β, and γ are constant coefficients determined by a small grid search on the original small model.