Neural information transfer in a noisy environment

Abstract

For an array of N summing comparators, each with the same internal noise, how should the set of thresholds, (theta) i, be arranged to maximize the information at the output, given the input signal, x, has an arbitrary probability density, P(x)? This problem is easy to solve when there is no internal noise. In this case, the transmitted information is equal to the entropy of the output signal, y. For N comparators there are N+1 possible output states and hence y can take on N+1 values. The transmitted information is maximized when all output states have the same probability of occupation, that is, 1/(N+1). In this paper we address some preliminary considerations relating to the maximization of the transmitted information I = H(y) - H(y|x) when there is finite internal noise.