Exponential Discretization of Weights of Neural Network Connections in Pre-Trained Neural Networks

Abstract: To reduce random access memory (RAM) requirements and to increase speed of recognition algorithms we consider a weight discretization problem for trained neural networks. We show that an exponential discretization is preferable to a linear discretization since it allows one to achieve the same accuracy when the number of bits is 1 or 2 less. The quality of the neural network VGG-16 is already satisfactory (top5 accuracy 69%) in the case of 3 bit exponential discretization. The ResNet50 neural network shows top5 accuracy 84% at 4 bits. Other neural networks perform fairly well at 5 bits (top5 accuracies of Xception, Inception-v3, and MobileNet-v2 top5 were 87%, 90%, and 77%, respectively). At less number of bits, the accuracy decreases rapidly. © 2019, Allerton Press, Inc.

Авторы
Malsagov M.Y.1 , Khayrov E.M. 1 , Pushkareva M.M.1 , Karandashev I.M. 1, 2
Издательство
Allerton Press Incorporation
Номер выпуска
4
Язык
Английский
Страницы
262-270
Статус
Опубликовано
Том
28
Год
2019
Организации
  • 1 Scientific Research Institute for System Analysis, Russian Academy of Sciences, Moscow, 117218, Russian Federation
  • 2 Peoples Friendship University of Russia (RUDN University Moscow), Moscow, 117198, Russian Federation
Ключевые слова
equidistant discretization; exponential discretization; neural network; neural network compression; number of bits; reduction of bit depth of weights; weight quantization
Цитировать
Поделиться

Другие записи

Ateya A.A., Khayyat M., Muthanna A., Koucheryavy A.
11th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops, ICUMT 2019, Dublin, Ireland, October 28-30, 2019. IEEE. Том 2019-October. 2019.