Stochastic neural computation. I. Computational elements
Стохастические нейронные вычисления. I. Вычислительные элементы
2001-01-01
SCID: 54.1/c8tf2vxs
Discuss with AI
VLSI artificial neural networksstate machine-based nonlinearitiesstochastic computational elementsstochastic neural computationstochastic-to-deterministic conversion
Figures from the paper
Abstract (AI)
This paper examines a number of stochastic computational elements employed in artificial neural networks, several of which are introduced for the first time, together with an analysis of their operation. We briefly include multiplication, squaring, addition, subtraction, and division circuits in both unipolar and bipolar formats, the principles of which are well-known, at least for unipolar signals. We have introduced several modifications to improve the speed of the division operation. The primary contribution of this paper, however, is in introducing several state machine-based computational elements for performing sigmoid nonlinearity mappings, linear gain, and exponentiation functions. We also describe an efficient method for the generation of, and conversion between, stochastic and deterministic binary signals. The validity of the present approach is demonstrated in a companion paper through a sample application, the recognition of noisy optical characters using soft competitive learning. Network generalization capabilities of the stochastic network maintain a squared error within 10 percent of that of a floating-point implementation for a wide range of noise levels. While the accuracy of stochastic computation may not compare favorably with more conventional binary radix-based computation, the low circuit area, power, and speed characteristics may, in certain situations, make them attractive for VLSI implementation of artificial neural networks.
Key Findings
1
An efficient method is described for generating stochastic signals and converting between stochastic and deterministic binary representations.
2
Several division-operation modifications are introduced to improve computational speed.
3
State-machine-based elements are introduced for sigmoid nonlinearities, linear gain, and exponentiation functions.
4
Stochastic computation can offer low circuit area and power for VLSI neural networks, despite lower accuracy than conventional binary-radix computation; a companion application reports squared error within 10% of floating-point performance across broad noise levels.
5
The paper analyzes stochastic neural-network computational elements, including multiplication, squaring, addition, subtraction, and division in unipolar and bipolar formats.
Research Object
Stochastic computational elements for artificial neural networks
Research Subject
Operation and implementation of arithmetic and nonlinear-function computational elements, including multiplication, division, sigmoid mappings, linear gain, exponentiation, and stochastic–deterministic signal conversion
Publication Details
Publication Date
2001-01-01
Journal
Publisher
ISSN
Cited by
553
Access Type
Author Information
Download PDF
Subscribe to digest