A joint project of the Graduate School, Peabody College, and the Jean & Alexander Heard Library

Title page for ETD etd-03232018-130812


Type of Document Master's Thesis
Author Wang, Xiaotian
Author's Email Address wxt1994@gmail.com
URN etd-03232018-130812
Title Investigation of Training Order Effects on Artificial Neural Networks for Image Recognition
Degree Master of Science
Department Computer Science
Advisory Committee
Advisor Name Title
Maithilee Kunda Committee Chair
Richard Alan Peters Committee Member
Keywords
  • training efficiency
  • mini-batch
  • stochastic gradient descent
  • training order
Date of Defense 2018-03-21
Availability unrestricted
Abstract
The stochastic gradient descent algorithm for training neural networks is widely used in many machine learning, especially deep learning tasks. The stochastic gradient descent algorithm operates by choosing a small fraction of the training data, called a mini-batch, at each iteration to compute an approximation of the gradient of the objective function to be optimized. In practice, researchers tend to use small batch sizes, and the training data fed into the neural network is usually of various categories and is in random order. Researchers have shown the advantages of smaller sizes of mini-batches quantitatively, yet in the past, there were very few formal investigations into the question of how the order of training data would affect the training efficiency and generalizability of the neural network. To gain more insight into this problem, we have investigated effects of training order and the composition of a mini-batch by conducting a series of controlled experiments. In our experiments, we retrained an existing neural network model for object recognition with images from the ImageNet dataset and from a newly-collected dataset called the Toy-Box dataset. We investigated using optimization techniques like genetic algorithms and simulated annealing to optimize the order of training data. Also, we compared training efficiency for different compositions of mini-batches.
Files
  Filename       Size       Approximate Download Time (Hours:Minutes:Seconds) 
 
 28.8 Modem   56K Modem   ISDN (64 Kb)   ISDN (128 Kb)   Higher-speed Access 
  Xiaotian_Wang.pdf 3.53 Mb 00:16:20 00:08:24 00:07:21 00:03:40 00:00:18

Browse All Available ETDs by ( Author | Department )

If you have more questions or technical problems, please Contact LITS.