A number of new fundamental problems expanding Vasiliev’s and Tarkhov’s methodology worked out for neural network models constructed on the basis of differential equations and other data has been stated and solved in this paper. The possibility of extending the parameter range in the same neural network model without loss of accuracy was studied. The influence of the new approach to choosing test points and using the heterogeneous complementary data on the solution accuracy was analyzed. The additional conditions in equation form derived from the asymptotic decomposition were used apart from the point data. The classical and non-classical definitions of the problem were compared by entering a parameter into the additional data. A new sampling scheme of test point choice at different stages of minimization (the procedure of test point regeneration) under various initial conditions was investigated. A way of combining two approaches (classical and neural-network) based on the Adams PECE method was considered.