Brain-inspired Weighted Normalization for CNN Image Classification

AO_SCPLOWBSTRACTC_SCPLOWWe studied a local normalization paradigm, namely weighted normalization, that better reflects the current understanding of the brain. Specifically, the normalization weight is trainable, and has a more realistic surround pool selection. Weighted normalization outperformed other normalizations in image classification tasks on Cifar10, Imagenet and a customized textured MNIST dataset. The superior performance is more prominent when the CNN is shallow. The good performance of weighted normalization may be related to its statistical effect of gaussianizing the responses.