Some new results on the architecture, training process, and estimation error bounds for learning machines

Persistent Link:
http://hdl.handle.net/10150/280015
Title:
Some new results on the architecture, training process, and estimation error bounds for learning machines
Author:
Zegers, Pablo
Issue Date:
2002
Publisher:
The University of Arizona.
Rights:
Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.
Abstract:
The importance of the problem of designing learning machines rests on the promise of one day delivering systems able to learn complex behavior and assisting us in a myriad of situations. Despite its long standing in the scientific arena, progress towards producing useful machines is hampered by many unanswered questions. This dissertation makes some important contributions towards this overall goal. In particular it focuses on providing a practical solution that allows to build systems that can learn and modulate dynamic behavior, on presenting an incremental learning scheme that permits to check if a learning machine has attained generalization capability just from studying its adaptation behavior, and on studying a bound that limits the learning capacity of any machine. The first contribution develops a Dynamic Neural Network (DNN), a hybrid architecture that employs a Recurrent Neural Network (RNN) in cascade with a Non-Recurrent Neural Network (NRNN). The RNN is in charge of generating a simple limit cycle while the NRNN is devoted to reshaping the limit cycle into the desired spatio-temporal behavior. The main advantage of this architecture is the simplicity of training which results from the simplification of the overall training task due to its decomposition into independent spatial and temporal learning subtasks, which in turn permits to reduce the overall training complexity to that of training a feedforward neural network alone. The second contribution of this dissertation presents an incremental learning procedure that permits to determine whether a learning system has generalized or not. The procedure employs some concepts from statistical learning theory to prove that when a system generalizes the probability that it will encounter unexpected situations decreases exponentially to zero. The third contribution uses the well known fact that the problem underlying the design of a learning machine corresponds to an estimation problem and is thus bounded by the Fisher information quantity. Given how important it is to know more about this bound, a series of properties of the Fisher information are presented.
Type:
text; Dissertation-Reproduction (electronic)
Keywords:
Engineering, Electronics and Electrical.; Artificial Intelligence.
Degree Name:
Ph.D.
Degree Level:
doctoral
Degree Program:
Graduate College; Electrical and Computer Engineering
Degree Grantor:
University of Arizona
Advisor:
Sundareshan, Malur K.

Full metadata record

DC FieldValue Language
dc.language.isoen_USen_US
dc.titleSome new results on the architecture, training process, and estimation error bounds for learning machinesen_US
dc.creatorZegers, Pabloen_US
dc.contributor.authorZegers, Pabloen_US
dc.date.issued2002en_US
dc.publisherThe University of Arizona.en_US
dc.rightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.en_US
dc.description.abstractThe importance of the problem of designing learning machines rests on the promise of one day delivering systems able to learn complex behavior and assisting us in a myriad of situations. Despite its long standing in the scientific arena, progress towards producing useful machines is hampered by many unanswered questions. This dissertation makes some important contributions towards this overall goal. In particular it focuses on providing a practical solution that allows to build systems that can learn and modulate dynamic behavior, on presenting an incremental learning scheme that permits to check if a learning machine has attained generalization capability just from studying its adaptation behavior, and on studying a bound that limits the learning capacity of any machine. The first contribution develops a Dynamic Neural Network (DNN), a hybrid architecture that employs a Recurrent Neural Network (RNN) in cascade with a Non-Recurrent Neural Network (NRNN). The RNN is in charge of generating a simple limit cycle while the NRNN is devoted to reshaping the limit cycle into the desired spatio-temporal behavior. The main advantage of this architecture is the simplicity of training which results from the simplification of the overall training task due to its decomposition into independent spatial and temporal learning subtasks, which in turn permits to reduce the overall training complexity to that of training a feedforward neural network alone. The second contribution of this dissertation presents an incremental learning procedure that permits to determine whether a learning system has generalized or not. The procedure employs some concepts from statistical learning theory to prove that when a system generalizes the probability that it will encounter unexpected situations decreases exponentially to zero. The third contribution uses the well known fact that the problem underlying the design of a learning machine corresponds to an estimation problem and is thus bounded by the Fisher information quantity. Given how important it is to know more about this bound, a series of properties of the Fisher information are presented.en_US
dc.typetexten_US
dc.typeDissertation-Reproduction (electronic)en_US
dc.subjectEngineering, Electronics and Electrical.en_US
dc.subjectArtificial Intelligence.en_US
thesis.degree.namePh.D.en_US
thesis.degree.leveldoctoralen_US
thesis.degree.disciplineGraduate Collegeen_US
thesis.degree.disciplineElectrical and Computer Engineeringen_US
thesis.degree.grantorUniversity of Arizonaen_US
dc.contributor.advisorSundareshan, Malur K.en_US
dc.identifier.proquest3053880en_US
dc.identifier.bibrecord.b42812537en_US
All Items in UA Campus Repository are protected by copyright, with all rights reserved, unless otherwise indicated.