Exponential Discretization of Weights of Neural Network Connections in Pre-Trained Neural Networks

Abstract: To reduce random access memory (RAM) requirements and to increase speed of recognition algorithms we consider a weight discretization problem for trained neural networks. We show that an exponential discretization is preferable to a linear discretization since it allows one to achieve the same accuracy when the number of bits is 1 or 2 less. The quality of the neural network VGG-16 is already satisfactory (top5 accuracy 69%) in the case of 3 bit exponential discretization. The ResNet50 neural network shows top5 accuracy 84% at 4 bits. Other neural networks perform fairly well at 5 bits (top5 accuracies of Xception, Inception-v3, and MobileNet-v2 top5 were 87%, 90%, and 77%, respectively). At less number of bits, the accuracy decreases rapidly. © 2019, Allerton Press, Inc.

Авторы
Malsagov M.Y.1 , Khayrov E.M. 1 , Pushkareva M.M.1 , Karandashev I.M. 1, 2
Издательство
Allerton Press Incorporation
Номер выпуска
4
Язык
Английский
Страницы
262-270
Статус
Опубликовано
Том
28
Год
2019
Организации
  • 1 Scientific Research Institute for System Analysis, Russian Academy of Sciences, Moscow, 117218, Russian Federation
  • 2 Peoples Friendship University of Russia (RUDN University Moscow), Moscow, 117198, Russian Federation
Ключевые слова
equidistant discretization; exponential discretization; neural network; neural network compression; number of bits; reduction of bit depth of weights; weight quantization
Дата создания
02.11.2020
Дата изменения
02.11.2020
Постоянная ссылка
https://repository.rudn.ru/ru/records/article/record/65730/
Поделиться

Другие записи

Ateya A.A., Khayyat M., Muthanna A., Koucheryavy A.
International Congress on Ultra Modern Telecommunications and Control Systems and Workshops. Том 2019-October. 2019.