The last three decades witnessed the emergence of multiple online services providing machine translation. At the turn of the century the quality of translations was very low. They often could not be used even for familiarization with the general content of the source text. Numerous mistakes in syntax, tenses, articles, government and word meanings made most translations unreadable. The quality of these services was steadily improving, but the pace of this progress left much to be desired. Four years ago machine translation reached an entirely new level. In November 2016, Google Translate began to switch to neural translation, based on the analysis of a huge number of examples. The new engine was activated for nine languages: English, Spanish, Chinese, Korean, German, Portuguese, Turkish, French and Japanese. In March 2017, five more languages were added, including Russian. The popularity of Google Translate among Russian people increased significantly, since the service became suitable for introductory or urgent translation and for rendering relatively simple texts. Nevertheless, the quality of neural translation is inferior to the level demonstrated by specialists. This paper examines the errors and inaccuracies caused by the insufficient volume of general and terminological vocabulary in the service's database and incorrect translations of certain words and phrases. We decided to consider the translation from Russian into English, since Google Translate performs it at a higher level. This allows us to provide more accurate assessment of its current potential. The majority of the selected terms and concepts are related to philology and pedagogy. It can be stated that the use of neural translation led to significant improvement in the quality of the services provided by Google Translate. Nevertheless, this service still makes mistakes when rendering the following categories of vocabulary: compound terms and concepts; lexical units and phrases that do not have clear equivalents in English; two words with the same English equivalent; abbreviations and nonce words. Besides, Google Translate does not always choose the correct analogue for quite unambiguous and widely used Russian concepts. Therefore, machine translation still needs proofreading, albeit not as much as earlier, before the introduction of neural methods.