Some Results Regarding the Estimation of Densities and Random Variate Generation Using Neural Networks

Magdon-Ismail, Malik and Atiya, Amir (2000) Some Results Regarding the Estimation of Densities and Random Variate Generation Using Neural Networks. Technical Report. California Institute of Technology. [CaltechCSTR:2000.005]

Full text available as:

	Postscript - Requires a viewer, such as GhostView
	Other (Adobe PDF (1.57MB))

Abstract

In this paper we consider two important topics: density estimation and random variate generation. We will present a framework that is easily implemented using the familiar multilayer neural network. First, we develop two new methods for density estimation, a stochastic method and a related deterministic method. Both methods are based on approximating the distribution function, the density being obtained by differentiation. In the second part of the paper, we develop new random number generation methods. Our methods do not suffer from some of the restrictions of existing methods in that they can be used to generate numbers from an observed inverse relationship between the density estimation process and the random number generation process. We present two variants of this method -- a stochastic and a deterministic version. We propose a second method that is based on formulating the task as a control problem, where a "controller network" is trained to shape a given density into the desired density. We justify the use of all the methods that we propose by providing theoretical convergence results. In particular, we prove that the L8 convergence to the true density to both the density estimation and random variate generation techniques occurs as a rate O((log log N/N)^((1-e)/2) where N is the number of data points and e can be made arbitrarily small for sufficiently smooth target densities. This bound is very close to the optimally achievable convergence rate under similar smoothness conditions. Also, for comparison, the L2 (RMS) convergence rate of a positive kernel density estimator is O(N^(-2/5)) when the optimal kernel width is used. We present numerical simulations to illustrate the performance of the proposed density estimation and random variate generation methods. In addition, we present an extended introduction and bibliography that serves as an overview and reference for the practitioner.

EPrint Type:	Monograph (Technical Report)
Subjects:	All Records
ID Code:	168
Deposited By:	Caltech Library System
Deposited On:	25 April 2001
Record Number:	CaltechCSTR:2000.005
Official Persistent URL:	http://resolver.caltech.edu/CaltechCSTR:2000.005
Usage Policy:	You are granted permission for individual, educational, research and non-commercial reproduction, distribution, display and performance of this work in any format.

Archive Staff Only: edit this record