Stochastic neural computation. I. Computational elements

Стохастические нейронные вычисления. I. Вычислительные элементы
Bryce Brown, HOWARD C. CARD
2001-01-01

VLSI artificial neural networksstate machine-based nonlinearitiesstochastic computational elementsstochastic neural computationstochastic-to-deterministic conversion
This paper examines a number of stochastic computational elements employed in artificial neural networks, several of which are introduced for the first time, together with an analysis of their operation. We briefly include multiplication, squaring, addition, subtraction, and division circuits in both unipolar and bipolar formats, the principles of which are well-known, at least for unipolar signals. We have introduced several modifications to improve the speed of the division operation. The primary contribution of this paper, however, is in introducing several state machine-based computational elements for performing sigmoid nonlinearity mappings, linear gain, and exponentiation functions. We also describe an efficient method for the generation of, and conversion between, stochastic and deterministic binary signals. The validity of the present approach is demonstrated in a companion paper through a sample application, the recognition of noisy optical characters using soft competitive learning. Network generalization capabilities of the stochastic network maintain a squared error within 10 percent of that of a floating-point implementation for a wide range of noise levels. While the accuracy of stochastic computation may not compare favorably with more conventional binary radix-based computation, the low circuit area, power, and speed characteristics may, in certain situations, make them attractive for VLSI implementation of artificial neural networks.
1
An efficient method is described for generating stochastic signals and converting between stochastic and deterministic binary representations.
2
Several division-operation modifications are introduced to improve computational speed.
3
State-machine-based elements are introduced for sigmoid nonlinearities, linear gain, and exponentiation functions.
4
Stochastic computation can offer low circuit area and power for VLSI neural networks, despite lower accuracy than conventional binary-radix computation; a companion application reports squared error within 10% of floating-point performance across broad noise levels.
5
The paper analyzes stochastic neural-network computational elements, including multiplication, squaring, addition, subtraction, and division in unipolar and bipolar formats.

Stochastic computational elements for artificial neural networks

Operation and implementation of arithmetic and nonlinear-function computational elements, including multiplication, division, sigmoid mappings, linear gain, exponentiation, and stochastic–deterministic signal conversion

Publication Details
Publication Date
2001-01-01
Journal
Publisher
ISSN
Cited by
553
Access Type
Author Information
Authors
Bryce Brown
HOWARD C. CARD
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%