Rosenbaum 2010 Pooling

7/25/2019 Rosenbaum 2010 Pooling

1/14


2/14

Frontiers in Computational Neuroscience www.frontiersin.org April 2010 | Volume 4 | Article 9 | 2

Rosenbaum et al. Pooling and correlated neural activity

Vogels and Abbott, 2005). This is in contrast to recurrent networkswhich can display stable asynchronous states (Hertz, 2010; Renartet al., 2010) similar to those observed in vivo (Ecker et al., 2010).

MATERIALS AND METHODS

CORRELATIONS BETWEEN STOCHASTIC PROCESSES

The cross-covariance of a pair of stationary stochastic processes,

x(t) and y(t), is Cxy(t) = cov(x(s), y(s+t)). The auto-covariancefunction,Cxx

(t), is the cross-covariance between a process and itself.The cross- and auto-covariance functions measure second orderdependencies at time lag tbetween two processes, or a process anditself. We quantify the total magnitude of interactions over all timeusing the asymptotic statistics,

xy xy x xx xyxy

x y

C t dt = = =

( ) , , .2

(1)

While the asymptotic correlation,xy,

, measures correlations betweenx(t) andy(t) over large timescales, the auto- and cross-covariancefunctions determine the timescale of these dependencies.

CORRELATIONS BETWEEN SUMS OF RANDOM VARIABLES

Given two collections of correlated random variables { }xi inx=1 and

{ } ,yj jny=1 define the pooled variables, X xi i= and Y yi i= . Since

covariance is bilinear ( ( , ) ( , ))cov cov i i j j ij i j x y x y= the varianceand covariance of the pooled variables are

X x x x xjj i

n

x

i

n

i

n

XY x y xi j i j

x

i

xx

i j i

2

1

2

11

= + ==

== , and yy

j

ny

i

nx

j

==

11

,

and similarly for Y2 .

Using these expressions along with some algebraic manipula-tion, the correlation coefficient, XY XY X Y= / ,between the

pooled variables can be written as

XY

x xx

x

x

x y

x xx y yy

y

y

x y

xy

wn

vw w

n

v=

+

+ 1 1

wwy yy

(2)

= +

xy

xx yy x yn nO

1,

(3)

where

w v n

n n

x x x

x y

x

x

x

i

n

x y

x y

x y

j

n

i

n

i

x

i i

yx

= =

=

=

==

, ,

,

1

1

2

1

11

x x

x x

x x

jj

n

i

n

xy

x y x y

x y x

n n

n n

i j

xx

i j

=( )

=

=

=1

1

1

11

1

,

ii j

yx

i j i j

x

y

j

n

i

n

xx

x x x x

x y x x

jj

n

i

n

n n

==

=

=

= ( )

11

11

1

1

1

,

xx

and similarly for wy, v

y, y y, and yy. In deriving Eq. (3) we

assumed that all pairwise statistics are uniformly bounded awayfrom zero in the asymptotic limit.

Each overlined term above is a population average. Notably, xyrepresents the average correlation betweenx

iandy

jpairs, weighted

by the product of their standard deviations, and similarly for xxandyy.Correlation between weighted sums can be obtained by substi-

tutingx w xi x ii andy w yj y jj for weightswxi and wyjand makingthe appropriate changes to the terms in the equation above (e.g., x x x x y i j x yi i i i j i j w w w , ( )sign ). Overlap between the twopopulations can be modeled by taking x yi j = 1for some pairs.

Assuming that variances are homogeneous within each popula-tion, that is x xi = and y yj = for i= 1,,nxandj= 1,...,ny,simplifies these expressions. In particular, vx x x x= =

2 , x y x y= ,and

XY

xy

xx

x

xx yy

y

yyn n

=

+ ( )

+ ( )

11

11

.

(4)

Assuming further that the populations are symmetric,

x=

y= , n

x= n

y= n, and xx yy= , the expression above sim-

plifies to

XY

b

w w

b

w

nn

=+ ( )

= +

1

1

1O ,

(5)

where b xy= is the average pairwise correlation between the twopopulations and w xx yy= = is the average pairwise correlationwithin each population. Eq. (5) was derived in Bedenbaugh andGerstein (1997)in an examination of correlations between mul-tiunit recordings. In Chen et al. (2006), a version of Eq. (5) with

w

= b

is derived in the context of correlations between two VSDsignals. The asymptotic,

xy0, limit when w= bis discussed

in Renart et al. (2010).Note that the results above hold for correlations computed over

arbitrary time windows. We concentrate on infinite windows, anddiscuss extensions in the Appendix.

NEURON MODEL

In the second part of the presentation we consider two excitatoryand two inhibitory input populations projecting to two postsyn-aptic cells. Thejthexcitatory input to cell kis labeled e

j,k(t) (k = 1

or 2). Similarly, ij,k

(t) denotes thejthinhibitory input to cell k. Eachcell receives n

eexcitatory and n

iinhibitory inputs with individual

rateseandirespectively.Each of the excitatory and inhibitory inputs to cell k, are station-

ary spike trains modeled by point processes, ej k i j kit t t, ,( ) ( )=

and ij k i j kit t s, ,( ) ( )= where { },tj k

i and{ },sj ki are input spike times.

We assume that the spike trains are stationary in a multivariate sense(Stratonovich, 1963). The pooled excitatory and inhibitory inputsto neuron kare E t tk j

n

j kek( ) ( ),,= = 1e and I t tk j

n

j kk( ) ( ).,= = 1

i iTo generate correlated inputs to cells, we used the multiple inter-

action process (MIP) method (Kuhn et al., 2003), then jittered eachspike time independently by a random value drawn from an expo-nential distribution with mean 5ms. The resulting processes are


3/14



Poisson with cross-covariance functions proportional to a doubleexponential, C

xy(t) e|t| /5. Note that since each input is Poisson,

e e2 = and i i

2 = .While the dynamics of the afferent population were not mod-

eled explicitly, the response of the two downstream neurons wasobtained using a conductance-based IF model. The membranepotentials of the neurons were described by

CdV

dtg V V g t V V g t V Vm

kL k L E k E I k I k k

= ( ) ( )( ) ( )( ),

(6)

with excitatory and inhibitory conductances determined byg t tE kk ( ) ( )( )= E e and g t tk kI iI( ) ( )( )= where denotesconvolution. We used synaptic responses of the form e e e e( ) ( )

/t t tt= E 2 and i i i( ) ( )/t t tt= I 2 e where (t) is the

Heaviside function. The areaof a single excitatory or inhibitorypostsynaptic conductance (EPSC or IPSC) is therefore equal tothe synaptic weight, EorI, with units nSms. This analysis caneasily be extended to situations where each input, e

j,kor i

j,k, has

a distinct synaptic weight.

When examining spiking activity, we assume that when Vkcrosses a threshold voltage, Vth, an output spike is produced andV

k is reset to V

L. When examining sub-threshold dynamics, we

considered the free membrane potential without threshold.As a measure of balance between excitation and inhibition we

used (Troyer and Miller, 1997; Salinas and Sejnowski, 2000)

=

V V n

V V nL

L i i

E e e

I

E

I.

When = 1, the net excitation and inhibition are balanced andthe mean free membrane potential equals V

L. In simulations, we

set VL= 60 mV, V

E= 0 mV, V

I= 90 mV,

e= 10 ms,

i= 20 ms,

Cm= 114 pF, andgL= 4.086 nS, giving a membrane time constant,

m= C

m/g

L= 27.9 ms. In all simulations except those in Figure 7,

the cells are balanced (= 1).The conductance-based IF neuron behaves as a nonlinear filter

in the sense that membrane potentials cannot be written as a lin-ear transformation of the inputs. However, following Kuhn et al.(2004)and Coombes et al. (2007), we derive a linear approxima-tion to the conductance based model. Let U= V

kV

Lso that Eq.

(6) becomes

CdU

dtg g t g t U g t V V g t V Vm L L L= ( ) ( ) ( )E I E E I I( ) ( ) ( ) ( ) .

Define the effective membrane time constant,eff

=Cm

/(E[g

L +gE(t)+gL(t)]) = Cm/(gL +neveE+niviISubstituting this aver-age value in the previous equation yields the linear approximationto the conductance based model,

dU

dtU J tk= +

1

eff( ),

(7)

where J t g t V V g t V V Ck L L mk k( ) ( ( )( ) ( )( ))/= E E I I is the totalinput current to cell k. Solving and reverting to the original vari-ables gives the linear approximation V

k(t) = (J

kK)(t) +V

L, where

K t t t( ) ( ) /= e eff is the kernel of the linear filter induced by Eq. (7).

RESULTS

The pooling of signals from groups of neurons can impact bothrecordings of population activity and the structure of inputs topostsynaptic cells. We start by discussing correlations in pooledrecordings using a simple linear model. A similar model is thenused to examine the impact of pooling on the statistics of inputsto cells. For simplicity we assume that all spike trains are station-

ary. However, non-stationary results can be obtained using similarmethods as outlined in the Section Discussion. Though all param-eters are defined in the Meterials and Methods, Tables 1 and 2inthe Appendix contain brief descriptions of parameters for quickreference. Also, Tables 3 and 4summarize the values of parametersused for simulations throughout the article.

CORRELATIONS BETWEEN POOLED RECORDINGS

Pooling can impact correlations between recordings of populationactivity obtained from voltage sensitive dyes (VSDs), multi-unitrecordings and other techniques. Such signals might each repre-sent the summed activity of hundreds or thousands of neurons.Let two recorded signals,X

1(t) andX

2(t), represent the weighted

activity of cells in two populations (see diagram in Figure 1A). Ifwe assume homogeneity in the input variances and equal size ofthe recorded populations, Eq. (4) gives the correlation betweenthe recorded signals

X X

n n

1 2

12

11 11 22 22

12

11 221 11

1

=+ ( )

+ ( )

= +OO1

n

.

(8)

Here nrepresents the number of neurons recorded, kk ,k = 1,2represents the average correlation between cells contributing tosignal X

k(t), and 12 represents the average correlation between

cells contributing to different signals. The averages are weightedso that cells that contribute more strongly to the recording, suchas those closer to the recording site, contribute more to the averagecorrelations (see Materials and Methods). Cells common to bothrecorded populations can be modeled by setting the correspondingcorrelation coefficients to unity. A form of Eq. (8) with 11 22= was derived by Bedenbaugh and Gerstein (1997).

When the two recording sites are nearby, so that 12 11 22 ,even small correlations between individual cells are amplified bypooling so that the correlations between the recorded signals canbe close to 1. This effect was observed in experiments and explainedin similar terms in Stark et al. (2008).

A significant stimulus-dependent change in correlations

between individual cells might be reflected only weakly in thecorrelation between the pooled signals. This can occur, forinstance, in recordings of large populations when 12 ,11,and 22are increased by the same factor when a stimulus is presented.Similarly, an increase in correlations between cells can actually leadto a decreasein correlations between recorded signals when 11and22increase by a larger factor than 12.

To illustrate these effects, we construct a simple model of stimulusdependent correlations motivated by the experiments in Chen et al.(2006), in which VSDs were used to record the population responsein visual area V1 during an attention task. In their experiments,


4/14



the imaged area is divided into 64 pixels, each 0.25 mm .25 mmin size. The signal recorded from each pixel represents the pooledactivity of n1.25 104neurons.

We model correlations between the signals, X1(t) and X

2(t),

recorded from two pixels in the presence or absence of a stimulus(see Figure 2B), using a simplified model of stimulus dependentrates and correlations. The firing rate of a cell located at distance dfrom the center of the retinotopic image of a stimulus is

r dB

B d

B

( )( )( cos( ))

= +

+

1 1

2

stimulus present

stimulus absent.

(9)

Here, B[0,1] represents baseline activity and 1 controlsthe rate at which activity decays with d. Both dand rwere scaledso that their maximum value is 1 (see Figure 2A).

We assume that the correlation between the responses of twoneurons is proportional to the geometric mean of their firingrates (de la Rocha et al., 2007; Shea-Brown et al., 2008), and that

correlations decay exponentially with cell distance (Smith andKohn, 2008; see however Poort and Roelfsema, 2009; Ecker et al.,2010). We therefore model the correlation between two cells as jk j k

DS r d r d j k= ( ) ( ) ,e where d

j and d

k are the distances from

each cell to the center of the retinotopic image of the stimu-lus, D

j,k is the distance between cells j and k, is the rate at

which correlations decay with distance, and S 1 is a constantof proportionality.

If pixels are small compared to the scales at which correlations areassumed to decay, then the average correlation between cells withinthe same pixel are 11 1= Sr d( )and 22 2= Sr d( ).The average correla-tion between cells in different pixels is 12 1 2 1 2=

S r d r d

D( ) ( ) .,eIn this case, whether a stimulus is present or not, the correlation

between the pooled signals is of the form X XD

n1 2

1 2 1= +e , ( / ).O Thus, even significant stimulus dependent changes in correlationswould be invisible in the recorded signals. This overall trend is consist-ent with the results in Chen et al. (2006)(compare Figure 2Cto theirFigure 2f). In such settings, it is difficult to conclude whether pairwisecorrelations are stimulus dependent or not from the pooled data.

FIGURE 1 | Models of pooled recordings and the effects of pooling on

correlations. (A)Pooling in experimental recordings. Cells from different

populations are correlated with average correlation coefficient 12and cells fromthe same population have average correlation coefficient 11or22.We examinecorrelations between the pooled signals, X

1and X

2. (B)Pooled inputs to cell pairs.

Individual excitatory (e) and inhibitory (i) inputs are correlated with coefficient ee

,

ii, and ei, respectively. The total input to a cell is the summed activity of itsexcitatory (E

K) and inhibitory (I

K) presynaptic population. The membrane potentials,

V1and V

2, are obtained by filtering these inputs. (C)A simulation of the setup in

(B)with background noise. Correlations between excitatory and between

inhibitory cells were uniform (ee

= ii= 0.05), but excitatoryinhibitory correlations

were absent (ei= 0). The raster plot shows the activity in a subset of the input

population. The correlation coefficient between the sub-threshold activity of the

postsynpatic cells was vv

= 0.768 0.001 s.e. Each cell receives 250 correlatedand 250 uncorrelated excitatory Poisson inputs as well as 84 correlated and 84

uncorrelated inhibitory Poisson inputs (ne= 250 and n

i= 84, q

e= q

i= 1,

e= 5 Hz,

i= 7.5 Hz,I= 4E, and E2.3 nSms see Materials and Methods andFigure 3Afor notation and precise model description). (D)Same as (B), but with

ei= 0.05,

e=

i=5 Hz,I= 6Eto maintain balance. In this case

VV= 0.0085 0.0024 s.e..

The simulations were run 8000 times at 10 s each.


5/14



However, in Supplementary Figure 3 of Chen et al. (2006) thepresence of a stimulus apparently results in a slight decrease incorrelations between more distant pixels. In Figure 2Dthis effectwas reproduced using the alternative model described above, withthe additional assumption that baseline activity, B, decreases in thepresence of a stimulus (Mitchell et al., 2009). The effect can also be

reproduced by assuming that spatial correlation decay, , increaseswhen a stimulus is present.As this example shows, care needs to be taken when inferring

underlying correlation structures from pooled activity. The sta-tistical structure of the recordings can depend on pairwise cor-relations between individual cells in a subtle way, and differentunderlying correlation structures may be difficult to distinguishfrom the pooled signals. However, downstream neurons may alsobe insensitive to the precise structure of pairwise correlations, asthey are driven by the pooled input from many afferents.

CORRELATIONS BETWEEN THE POOLED INPUTS TO CELLS

We next examine the effects of pooling by relating the correla-

tions between the activity of downstream cells to the pairwise cor-relations between cells in the input populations (see Figure 1B).The idea that pooling amplifies correlations carries over from theprevious section. However, the presence of inhibition and non-instantaneous synaptic responses introduces new issues.

A homogeneous population with overlapping and independent inputs

For simplicity, we first consider a homogeneous population model(see Figure 3A). Each cell receives n

einputs from a homogeneous

pool of inputs with pairwise correlation coefficients ee

and anadditional q

en

einputs from an outside pool of independent inputs.

The two cells sharepen

eof the inputs drawn from the correlated

pool. Processes in the independent pool are uncorrelated with allother processes. All excitatory inputs have variance e

2 .The correlation between the pooled excitatory inputs is given

by (see Appendix)

E E

eee

e

ee

eee

ee e

1 2

1

11

=

+ ( )

+ +( )

p

n

nq

.

(10)

A form of this equation, withpe= 0 and q

e= 0, is derived inChen

et al. (2006). In the absence of correlations between processes in theinput pools,

ee= 0, the correlation between the pooled signals is

just the proportion of shared inputs,E E e1 2 =p .When ee>0 and neis large, pooled excitatory inputs are highly correlated, even whenpairwise correlations in the presynaptic pool,

ee, are small, and the

neurons do not share inputs (pe= 0). Even when most inputs to the

downstream cells are independent (qe>1), correlations between

the pooled signals will be nearly 1 for sufficiently large input pools

(see Figure 4A).Under analogous homogeneity assumptions for the inhibitory

pools, the correlation, I I1 2 ,between the pooled inhibitory inputsis given by an equation identical to Eq. (10), and the correlationbetween the pooled excitatory and inhibitory inputs is given by

E I I E

ei

eee

ee e iii

ii i

1 2 1 2

11

11

= =

+ +( )

+ +( )n

qn

q

.

(11)

Interestingly, since | | ,E I1 2 1 pairwise excitatoryinhibitorycorrelations obey the bound | | ( / ). ei ee ii e i +O 1 n n

Combining this inequality with Eq. (10) and the analogous equa-tion for I I1 2 , it follows that | | E I E E I I e i1 2 1 2 1 2 1 +O( / )n n forhomogeneous populations. These are a result of the non-negativedefiniteness of covariance matrices.

Heterogeneity and the effects of spatially dependent correlations

We next discuss how heterogeneity can dampen the amplificationof correlations due to pooling. In the absence of any homogeneityassumptions on the excitatory input population (see the popula-tion model in the Materials and Methods), Eq. (3) gives the pooledexcitatory signals, E E e e e e e e e e1 2 1 2 1 1 2 2 1 21= +/ ( / ).O n n The terme e1 2 is a weighted average of the correlation coefficients betweenthetwo excitatory populations, and e e1 1and e e2 2 are weighted averages

of the correlations withineach excitatory input population.To illuminate this result, we assume symmetry between the

populations: Let n nke e

= and e ekj= for k = 1,2 andj= 1,2,,ne,

and assume e e e e1 1 2 2= .The average within and between correla-tions, are ee

we e e e= =1 1 2 2 and ee e e= 1 2respectively (see Figure 3B).

Under these assumptions, Eq. (5) can be applied to obtain (See alsoBedenbaugh and Gerstein, 1997)

E E

eeb

eew

eeew

eeb

eew

e1 2 1

1

1=

+ ( )= +

n

nO

(12)

FIGURE 2 | The effect of pooling on recordings of stimulus dependent

correlations. (A)The response amplitude of a model neuron as a function of its

distance from the retinotopic image of a stimulus [Eq. (9)] with B = 0.05 and

= 10. (B)A diagram of our model. Signals X1(t) and X

2(t) are recorded from two

pixels (red and blue squares). The activity in response to a stimulus is shown as

a gradient centered at some pixel (the center of the retinotopic image of the

stimulus). (C)The prediction of the correlation between two pixels obtained

using the stimulus-dependent model considered in the text with stimulus

present (red) and absent (green). We assumed that one pixel is located at the

stimulus center (d1= 0). Parameters are as in (A)with = 1, S = 0.1, and

n= 1.25104. A stimulus dependent change in correlations is undetectable. (D)Same as in (C), except that baseline activity, B, was scaled by 0.5 in the

presence of a stimulus. Compare to Figure 2f in Chen et al., 2006.


6/14



which is plotted in Figure 4A (green line) and Figure 4B. Forlarge n

e, the correlation between the pooled signals is the ratio of

between and within correlations.This observation has implications for a situation ubiquitous in

the cortex. A neuron is likely to receive afferents from cells that arephysically close. The activity of nearby cells may be more stronglycorrelated than the activity of more distant cells (Chen et al., 2006;

Smith and Kohn, 2008). We therefore expect that pairwise correla-tions withineach input pool are on average larger than correlationsbetweentwo input pools, that is, ee

wee> . This reduces the correla-

tion between the inputs, regardless of the input population size.An increase in correlations in the presynaptic pool can also decor-

relate the pooled signals. If correlations withineach input pool increaseby a greater amount than correlationsbetweenthe two pools, then the

variance in the input to each cell will increased by a larger amount thanthe covariance between the inputs. As a consequence the correlationsbetween the pooled inputs will be reduced. Modulations in correla-tion have been observed as a consequence of attention in V4 (Cohenand Maunsell, 2009; Mitchell et al., 2009; but apparently not in V1,Roelfsema et al., 2004). Such changes may be, in part, a consequenceof small changes in within correlations between neurons in V1.

Equation 12 implies that correlations between large populationscannot be significantly larger than the correlations within eachpopulation. Since | | ,E E1 2 1 it follows that | | | | ( / ). ee

beew

e +O 1 nThe correlation, I I1 2 , between the pooled inhibitory inputs

is given by an identical equation to Eq. (12) and the correlationbetween the pooled excitatory and inhibitory inputs is given by

E I I E

eib

eew

eeew

iiw

iiiw

1 2 1 2

11

11

= =

+ ( )

+ ( )

n n

= +

eib

eew

iiw

e i

O1

n n.

Correlations between the free membrane potentials

We now look at the correlation between the free membrane poten-tials of two downstream neurons. Thefreemembrane potentials areobtained by assuming an absence of threshold or spiking activity.For simplicity we assume symmetry in the statistics of the inputs tothe postsynaptic cells: E Ek = , I Ik = , E I E I1 2 2 1= , E I E I1 1 2 2= , E E E E1 1 2 2= and I I I I1 1 2 2= .The analysis is similar in the asymmet-ric case.

In the Section Materials and Methods, we derive a linearapproximation of the free membrane potentials,

V t J K t V k k L( ) ( ) ,= ( ) +

where J t g t V V g t V V Ck L L mk k( ) ( ( )( ) ( )( ))/= + E E I I are the totalinput currents and K t t t( ) ( ) /= e eff for k = 1,2. Under this approxi-mation, the correlation, V V1 2 , between the membrane potentials is

FIGURE 3 | Two population models considered in the text. (A)

Homogeneous population with overlap and independent inputs:

A homogeneous pool of correlated inputs (large black circle) with correlation

coefficient between any pair of processes equal to ee

. Each cell draws ne

inputs (larger red and blue circles) from this homogeneous input pool. Of

these necorrelated inputs, p

en

eare shared between the two neurons (purple

dots). In addition, each cell receives qen

eindependent inputs (smaller red and

blue circles), for a total of ne+q

en

einputs. All inputs have variance e

2.(B)A

population model with distinct within and between correlations: Each cellreceives n

einputs. The average correlation between two inputs to the

same cell is eew ,and between inputs to different cells is ee

b .

FIGURE 4 | The effect of pooling on correlations between summed input

spike trains. (A)The correlation coefficient between the pooled excitatory spike

trains (EE

) is shown as a function of the size of the correlated excitatory input

pool (ne) for various parameter settings. The solid blue line was obtained by

setting ee

= 0.05 for the population model inFigure 3A in the absence of shared

or independent inputs (pe= q

e= 0). The dashed line illustrates the decorrelating

effects of the addition of neindependent inputs (q

e= 1, q

e= 0,

ee= 0.05). The

dotted blue line shows that shared inputs increase correlations, but have a

diminishing effect on EE

with increasing input population size (pe= 0.2, q

e= 0,

ee

= 0.05). The solid pink line shows the effect of reducing the pairwise input

correlations (ee

= 0.005, pe= q

e= 0). The dashed tan line was obtained with

uncorrelated inputs so that correlations reflected shared inputs alone (pe= 0.2,

ee

= qe= 0). The green line was obtained with disparity in the within and

between correlations (eb = 0 05. and e

w = 0 1. ) using the model inFigure 3B.(B)The correlations coefficient,

EE, between the pooled inputs as a function of

the within and between correlations (eband e

w) for n

e= 50. Note that the pooled

correlation is relatively constant along lines through the origin. Thus, changing eb

and ewby the same proportion does not affect the pooled correlation.


7/14



equal to the correlation, in= J J1 2 , between the total input currentsand can be written as a weighted average of the pooled excitatoryand inhibitory spike train correlations (see Appendix),

V VW W W W

W W W W 1 22

2 =

+ + in

E2

E E I2

I I E I E I

E2

I2

E I E I

1 2 1 2 1 2

1 1 (13)

where E E E I I I1 2 1 2 1 2, , and E I1 1 are derived above, andW

E= E|V

EV

L|

Eand W

I=I|V

IV

L|

Iare weights for the exci-

tatory and inhibitory contributions to the correlation. In Figure 5,we compare this approximation with simulations.

The correlation between the membrane potentials haspositivecontributions from the correlation between the excitatory inputs( ),E E1 2 and between the inhibitory inputs ( ).I I1 2 Contributionscoming from excitatoryinhibitory correlations (E I1 2 and E I2 1) arenegative, and can thus decorrelate the activity of downstream cells.This cancellation of correlations is observed in Figures 1D and 5,and can lead to asynchrony in recurrent networks (Hertz, 2010;Renart et al., 2010).

IMPLICATIONS FOR SYNCHRONIZATION IN FEEDFORWARD CHAINSFeedforward chains, like that depicted in Figure 6A, have beenstudied extensively (Diesmann et al., 1999; van Rossum et al., 2002;Litvak et al., 2003; Reyes, 2003; Tetzlaff et al., 2003; Cteau andReyes, 2006; Doiron et al., 2006; Kumar et al., 2008). In such net-works, cells in a layer necessarily share some of their inputs, leadingto correlations in their spiking activity (Shadlen and Newsome,1998). Frequently, spiking in deeper layers is highly synchronous(Reyes, 2003; Tetzlaff et al., 2003). However, in the presence of back-ground noise, correlations can remain negligible (van Rossum et al.,2002; Vogels and Abbott, 2005).

Feedforward chains amplify correlations as follows: When inputsto the network are independent, small correlations are introducedin the second layer by overlapping inputs. The inputs to each sub-sequent layer are pooled from the previous layer. The amplifica-tion of correlations by pooling is the primary mechanism for thedevelopment of synchrony (Compare solid and dotted blue linesin Figure 4A). Overlapping inputs serve primarily to seed syn-

chrony in early layers. The internal dynamics of the neurons andbackground noise can decorrelate the output of a layer, and competewith the correlation amplification due to pooling.

We develop this explanation by considering a feedforward net-work with each layer containing N

eexcitatory and N

i inhibitory

cells. Each cell in layer k +1 receives neexcitatory and n

iinhibitory

inputs selected randomly from layer k. For simplicity we assumethat all excitatory and inhibitory cells are dynamically identicaland E|V

EV

L| =I|V

EV

L|. Spike trains driving the first layer are

statistically homogeneous with pairwise correlations 0.

FIGURE 5 | The effects of pooling on correlations between postsynaptic

membrane potentials. Results of the linear approximation (solid, dotted, and

dashed lines) match simulations (points). For the solid blue line,

ee=

ii= 0.05, and

ei= p

e= p

i= q

e= q

i= 0. The total number of excitatory

and inhibitory inputs to each cell was n= ne+q

en

e, and n

i+q

in

irespectively.

Here ni= n

e/3, with other parameters given in the Section Materials and

Methods. The dotted blue line was obtained by including independent

inputs, qe= q

i= 1. The pink line was obtained by decreasing input correlat ions

to ee

= ii= 0.005. The solid green line was obtained by including excitatory

inhibitory correlations, ei= 0.05, so that total input correlations canceled. The

dashed tan line was obtained by setting ee

= ii=

ei= q

e= q

i= 0 and

pe= p

i= 0.2 so that correlations are due to input overlap alone. In all cases,

E=590

nnSms, andI= 4E. Standard errors are smaller than twice the radii of

the points.

FIGURE 6 | The development of synchrony in a feedforward chain can be

understood using a model dynamical system (Tetzlaff et al. , 2003).

(A)Schematic diagram of the network. Each layer consists of Neexcitatory

and Niinhibitory cells. Each cell in layer kreceives precisely n

eexcitatory and

niinhibitory, randomly selected inputs from layer k 1. (B)Stages of

processing in the feedforward network. Inputs from layer k 1 are pooledwith overlap, and drive the cells in layer k. (C)The correlation transfer map

described by the pooling function, P() (blue dotted line), is composed withthe decorrelating transfer function, S() = 2(red dotted line), to obtain themapping, T = SP(solid blue line). Cobwebs show the development of

correlations in the discrete dynamical system defined by k kT+ =1out out( )with

0

= 0. Nearly perfect correlat ions develop by the fifth layer. The identity is

shown as a dashed line. (D)Closer to balance (1), the correlating effects ofpooling are weakened, and the model develops a stable fixed point close to

= 0. However, cells may no longer decorrelate their inputs in the balancedregime, and fluctuations in the input statistics due to random connectivity

can destabilize the fixed point and lead to synchrony. The shaded region in the

inset represents the region two standard deviations away from the mean

(blue line) when randomness in the overlap is taken into account (see

Appendix). The standard deviations were calculated using Monte Carlo

simulations. In C and D, Ne= 12000 and n

e= 600. In C, N

i= 8000 and

ni= 400. In D, N

i= 10500 and n

i= 525 to obtain approximate balance

(= 600/525). Filled black circles represent stable fixed points and open blackcircles represent unstable fixed points.


8/14



To explain the development of correlations, we consider a sim-plified model of correlation propagation (See also Renart et al.,2010for a recurrent version). In the model, any two cells in a layershare the expected proportionp

e= n

e/N

eof their excitatory inputs

andpi= n

i/N

iof their inhibitory inputs (the expected proportions

are taken with respect to random connectivity). We also assumethat inputs are statistically identical across a layer.

For a pair of cells in layer k 1, let kin

and kout

represent thecorrelation coefficient between the total input currents and out-put spike trains respectively. The outputs from layer kare pooled(with overlap) to obtain the inputs to layer k +1. Using the resultsdeveloped above, in

10= P( )and k kP+ =1

in out( ),for k 1, where (seeAppendix and Tetzlaff et al., 2003for a similar derivation)

Pn

p p

n

( )

( ) ( )

( ) ( )( ).

=

+ +( )

+ +

11

1

11

1 1

2

2

ie i

i

(14)

Here measures the balance between excitation and inhibi-

tion (see Materials and Methods). From our assumptions, = ne/ni.With imbalance ( 1) and a large number of cells in a layer, pool-ing amplifies small correlations, P() >, as discussed earlier.

To complete the description of correlation transfer from layerto layer, we relate the correlations between inputs to a pair of cells,k

in ,to correlations in their output spike trains, kout .We assume that

there is a transfer function, S, so that k kSout in= ( )at each layer k.

We additionally assume that S(0) = 0 and S(1) = 1, that is uncor-related (perfectly correlated) inputs result in uncorrelated (perfectlycorrelated) outputs. We also assume that the cells are decorrelat-ing, || >|S()| >0 for 0,1 (Shea-Brown et al., 2008). This isan idealized model of correlation transfer, as output correlationsdepend on cell dynamics and higher order statistics of the inputs

(Moreno-Bote and Parga, 2006; de la Rocha et al., 2007; Barreiroet al., 2009; Ostojic et al. , 2009).Correlations between the spiking activity of cells in layers k +1

are related to correlations in layer kby the layer-to-layer trans-fer function, T = S P. The development of correlations acrosslayers is modeled by the dynamical system, k kT+ =1

out out( ), with 1 0

out = S( ).When the network is not balanced ( 1), pooling amplifies

correlations at each layer and the activity between cells in deeperlayers can become highly correlated (see Figure 6C). The output ofthe first layer is uncorrelated if the individual inputs are independ-ent (

0= 0). In this case all of the correlations between the total

inputs to the second layer come from shared inputs,

2 0in e e i i

e i

= = ++

Pn p n p

n n( ) .

These correlations are then reduced by the second layer of cells, 2 2 0 0

out in= = >S T( ) ( ) , and subsequently amplified by pooling andinput sharing before being received by layer 3, 3

in2out= P( ).This

process continues in subsequent layers. If the correlating effectsof pooling and input sharing dominate the decorrelating effectsof internal cell dynamics, correlations will increase from layer tolayer (see Figure 6C).

When 0= 0, overlapping inputs increase the input correlation

to layer 2, but have a negligible effect on the mapping once correla-tions have developed since the effects of pooling dominate [see Eq.(14) and the dashed blue line in Figure 4Awhich shows that theeffects of input overlaps are small when n

eis large, >0 and 1].

Therefore, shared inputs seed correlated activity at the first layer,and pooling drives the development of larger correlations. When

0= 0, we cannot expect large correlations before layer 3, but when0>0 large correlations can develop by layer 2.To verify this conclusion, we constructed a two-layer feed-

forward network with no overlap between inputs (Pe= P

i= 0).

In Figure 7A, the inputs to layer 1 were independent (0= 0),

and the firing of cells in layer 2 was uncorrelated. In Figure 7B,we introduced small correlations (

0= 0.05) between inputs to

layer 1. These correlations were amplified by pooling so thatstrong synchrony is observed between cells in layer 2. We com-pared these results with a standard feedforward network withoverlap in cell inputs (Figure 7C, where P

e= P

i= 0.05). Inputs

to layer 1 were independent (0= 0), and hence outputs from

layer 1 uncorrelated. Dependencies between inputs to layer 2

were weak and due to overlap alone, 2in

= =P( ) . .0 0 05 Cells inlayer 3 received pooled inputs from layer 2, and their outputwas highly correlated.

These results predict that correlations between spike trainsdevelop in deeper layers, but they do not not directly addressthe timescale of the correlated behavior. In simulations, spikingbecomes tightly synchronized in deeper layers (see for instanceLitvak et al., 2003; Reyes, 2003;and Figure 7). This can be under-stood using results in Marlek et al. (1997)and Diesmann et al.(1999)where it is shown that the response of cells to volleys ofspikes is tighter than the volley itself. The firing of individual cells inthe network becomes bursty in deeper layers and large correlationsare manifested in tightly synchronized spiking events. Alternatively,

one can predict the emergence of synchrony by observing thatpooling increases correlations over finite time windows (see nextsection and Appendix) and therefore the analysis developed abovecan be adapted to correlations over small windows.

Balanced feedforward networks

In the simplified feedforward model above, when excitation balancesinhibition, that is 1, correlations between the pooled inputs toa layer are due to overlap alone, k

in oute i= +P p pk( ) ( )/1 2for all k.

The correlating effects of this map are weak, and this would seemto imply that cells in balanced feedforward chains remain asyn-chronous. Indeed, our model of correlation propagation displaysa stable fixed point at low values of when 1 (see Figure 6D).However, in practice, synchrony is difficult to avoid without carefulfine-tuning (Tetzlaff et al., 2003), and almost always develops infeedforward chains (Litvak et al., 2003). We provide some reasonsfor this discrepancy.

Our focus so far has been on correlations over infinitely largetime windows (see Materials and Methods where we define

xy).

Even when the membrane potentials are nearly uncorrelated overlarge time windows, differences between the excitatory and inhib-itory synaptic time constants can cause larger correlations oversmaller time windows (Renart et al., 2010). This can, in turn, lead to


9/14



significant correlations between the output spike trains. We discussthis effect further in the Appendix and give an example in Figure 8.In this example, the correlations between the membrane poten-tials over long windows are nearly zero due to cancellation (seeFigure 8Awhere

VV= 0.0174 0.0024 s.e. with threshold present),

but positive over shorter timescales. The cross-covariance func-tion between the output spike trains is primarily positive, yieldingsignificant spike train correlations (

spikes= 0.1570 0.0033 s.e.).

Therefore, the assumption that pairs of cells decorrelate their inputsmay not be valid in the balanced case.

Another source of discrepancies between the idealized modeland simulations of feedforward networks are inhomogeneities,which become important when balance is exact. Note that Eq. (14)is an approximation obtained by ignoring fluctuations in connec-tivity from layer to layer. In a random network, inhomogeneitieswill be introduced by variability in input population overlaps. Tofully describe the development of correlations in a feedforwardnetwork, it is necessary to include such fluctuations in a model of

correlation propagation. The asynchronous fixed point that appearsin the balanced case has a small basin of attraction and fluctuationsinduced by input inhomogeneities could destroy its stability (seeFigure 6D). Other sources of heterogeneity can further destabilizethe asynchronous state (see Appendix).

It has been shown that asynchronous states can be stabilizedthrough the decorrelating effects of background noise (van Rossumet al., 2002; Vogels and Abbott, 2005). To emulate these effects, athird transfer function, N, can be added to our model. The cor-relation transfer map then becomes T() = S NP(). Sufficientlystrong background noise can increase decorrelation from input tooutput of a layer, and stabilize the asynchronous fixed point.

DISCUSSION

We have illustrated how pooling and shared inputs can impact cor-relations between the inputs and free membrane voltages of post-synaptic cells in a feedforward setting. The increase in correlationdue to pooling was discussed in a simpler setting in (Bedenbaugh

FIGURE 7 | Development of synchrony in feedforward networks. (A)A

feedforward network with no overlap and independent, Poisson input. For

excitatory cells, we set E 1.55nSms, andI 4.67nSms. For inhibitory cells,E 3.61nSms, andI 10.82nSms. (B)Same as A, except inputs to layer 1 arecorrelated with coefficient

0= 0.05. The network is highly synchronized in the

second layer, even though inputs do not overlap. (C)Same as A, except for the

presence of overlapping inputs (pe= p

i= 0.05). Correlations due to overlap in the

input to layer 2 result in average correlations of 0.05 between input currents. Layer

3 cells in C synchronize (Compare with layer 2 in B). In all three figures, each cell in

the first layer was driven by excitatory Poisson inputs with rate 0= 100 Hz.

FIGURE 8 | Cross-covariance functions between membrane potentials

and output spike trains. (A)The cross-covariance function between

membrane potentials, scaled so that its maximum is 1. The linear

approximation in Eq. (16) (blue, shaded) agrees with simulations of the full

conductance-based model (black dashed line). Differences between

simulations with and without threshold are too small to be observable (8000

simulations 10s each; simulations with and without threshold are shown).

Parameters are as inFigure 1D. The cells are balanced with

ee

= ii=

ei= 0.05 so that the correlation between the membrane potentials

over long time windows is essentially zero (vv

= 0.0085 0.0024 s.e.unthresholded, and

vv= 0.0174 0.0024 s.e. thresholded). However,

correlations over shorter time windows are positive as indicated by the central

peak in the cross-covariance function. (B)The cross-covariance between the

output spike trains is mostly positive. The correlation between the output spike

trains was spikes

= 0.1570 0.0033 s.e. (500 simulations of 100s each withsame parameters as in A.


10/14



and Gerstein, 1997; Super and Roelfsema, 2005; Chen et al., 2006;Stark et al., 2008), and similar ideas were also developed for thevariance alone in (Salinas and Sejnowski, 2000; Moreno-Bote et al.,2008). The saturation of the signal-to-noise ratio with increasingpopulation size observed in (Zohary et al., 1994) has a similar ori-gin. Our aim was to present a unified discussion of these results,with several generalizations.

Other mechanisms, such as recurrent connectivity between cellsreceiving the inputs, can modulate correlated activity (Schneideret al., 2006; Ostojic et al., 2009). Importantly, the cancellation ofcorrelations may be a dynamic phenomenon in recurrent net-works, as observed in (Hertz, 2010; Renart et al., 2010). On theother hand, neurons may become entrained to network oscilla-tions, resulting in more synchronous firing (Womelsdorf et al.,2007). A full understanding of the statistics of population activityin neuronal networks will require an understanding of how thesemechanisms interact to shape the spatiotemporal properties of theneural response.

The results we presented relied on the assumption of linearity atthe different levels of input integration. These assumptions can be

expected to hold at least approximately. For instance, there is evi-dence that membrane conductances are tuned to produce a linearresponse in the subthreshold regime (Morel and Levy, 2009). Theassumptions we make are likely to break down at the level of singledendrites where nonlinear effects may be much stronger (Johnstonand Narayanan, 2008). The effects of correlated inputs to a singledendritic branch deserve further theoretical study (Gasparini andMagee, 2006; Li and Ascoli, 2006).

We demonstrated that the structure of correlations in a popula-tion may be difficult to infer from pooled activity. For instance, achange in pairwise correlations between individual cells in two pop-ulations causes a much smaller change in the correlation betweenthe pooled signals. With a large number of inputs, the change in

correlations between the pooled signals might not be detectableeven when the change in the pairwise correlations is significant.

While we discussed the growth of second order correlations only,higher order correlations also saturate with increasing populationsize. For example, in a 3-variable generalization of the homogene-ous model from Figure 3A, it can be shown that E E E e1 2 3 = 1 1O( / )n where n

eis the size of each population and E E E1 2 3is the triple correla-

tion coefficient (Stratonovich, 1963) between the pooled signals E1,

E2, and E

3. The reason that higher order correlations also saturate fol-

lows from the generalization of the following observation at secondorder: Pooling amplifies correlations because the variance and covari-ance grow asymptotically with the same rate in n

e. In particular E

2 and E E1 2both behave asymptotically like n ne

2ee e e

2 +O( ),and their

ratio, E E E E E1 2 1 2= / ,approaches unity (Bedenbaugh and Gerstein,1997; Salinas and Sejnowski, 2000; Moreno-Bote et al., 2008).

We concentrated on correlations over infinitely long time win-dows (see Materials and Methods where we define

xy). However,

pooling amplifies correlations over finite time windows in exactlythe same way as correlations over large time windows. Due tothe filtering properties of the cells, the timescale of correlationsbetween downstream membrane potentials may not reflect that ofthe inputs. We discuss this further in the Appendix where the auto-and cross-covariance functions between the membrane potentialsare derived.

To simplify the presentation, we have so far assumed stationary.However, since Eq. (2) applies to the Pearson correlation between anypooled data, all of the results on pooling can easily be extended to thenon-stationary case. In the non-stationary setting, the cross-covariancefunction has the form R

xy(s, t) = cov (x(s),y(s+t)), but there is no nat-

ural generalization of the asymptotic statistics defined in Eq. (1).Correlated neural activity has been observed in a variety of neu-

ral populations (Gawne and Richmond, 1993; Zohary et al., 1994;Vaadia et al., 1995), and has been implicated in the propagation andprocessing of information (Oram et al., 1998; Maynard et al., 1999;Romo et al., 2003; Tiesinga et al., 2004; Womelsdorf et al., 2007;Stark et al., 2008), and attention (Steinmetz et al., 2000; Mitchellet al., 2009). However, correlations can also introduce redundancyand decrease the efficiency with which networks of neurons repre-sent information (Zohary et al., 1994; Gutnisky and Dragoi, 2008;Goard and Dan, 2009). Since the joint response of cells and recordedsignals can reflect the activity of large neuronal populations, it willbe important to understand the effects of pooling to understandthe neural code (Chen et al., 2006).

APPENDIXDERIVATION OF EQ. (10)

Equation (10) can be derived from Eq. (2). However, we find thatit is more easily derived directly. We will calculate the variance, E

2E2

1 2= ,and covariance E E1 2 between the pooled signals.

The covariance is given by the sum of all pairwise covariancesbetween the populations, E E e E ,e E e e e e1 2 1 1 2 2 1 2= 1 2 . Each cellreceives n q ne e e+ inputs so that there are (ne+qene)

2 terms thatappear in this sum. However, the q

en

eindependent inputs from

each pool are uncorrelated with all other inputs and therefore dontcontribute to the sum. Of the remaining ne

2 pairs, nep

eare shared

and therefore have correlation e e1 2 = 1. These shared processestherefore collectively contribute n pe e e

2 to E E1 2 . The remaining

n n pe2

e e processes are correlated with coefficient eeand collec-tively contribute ( ) .n n pe e e ee e2 2 The pooled covariance is thus

E E e2

e e ee e

Correlated

e e e2

Shared

1 2= +( ) .n p n n p2

The variance is given by the sum of all pairwise covariances withina population, E

2e E e E e e e e1 1 1 2 1

= , .1 2 1 2 As above, there are ne+qeneneurons in the population, so that the sum has (n

e+q

en

e)2terms.

Of these, ne+q

en

eare diagonal terms (e

1= e

2), each contributing

e2 , for a total contribution of ( )n q ne e e e+

2 to E12 .The processes

from the independent pool do not contribute any additional terms.This leaves n

e(n

e1) correlated pairs which each contribute e ee

2

for a collective contribution ofn n

e e e ee( ) ,12

giving E E e e e ee

Correlated

e e e e

Diagon

1 21 2 2= = + +n n n n q( ) ( )

aal

.

Now, E E E E E1 2 1 2= / 1can be simplified to give Eq. (10). Equationsfor I I1 2and E I I E1 2 1 2= can be derived identically.

FINITE-TIME CORRELATIONS AND CROSS-COVARIANCES

Throughout the text, we concentrated on correlations over large timewindows. However, the effects of pooling described by Eq. (2) apply tothe correlation,

xy(t), between spike counts over any time window of


11/14



sizet, defined byxy x y x yt N t N t N t N t ( ) cov( ( ), ( ))/ var( ( ))var( ( ))= whereN t x s sx

t( ) ( )= 0 d is the spike count over [0,t] for the spiketrainx(t). The equation also applies to the instantaneous correla-tion at time t, defined by R t C t C t C t xy xy xx yy( ) ( )/ ( ) ( ).= Thus pool-ing increases correlations over all timescales equally.

However, the cell filters the pooled inputs to obtain the mem-brane potentials and, as a result, the correlations between mem-

brane potentials is spread out in time (Tetzlaff et al., 2008). Toquantify this effect, we derive an approximation to the auto- andcross-covariance functions between the membrane potentials.

The pooled input spike trains are obtained from from a weightedsum of the individual excitatory and inhibitory spike trains (seeMaterials and Methods). As a result cross-covariance functionsbetween the pooled spike trains are just sums of the individual cross-covariance functions, C t C t XY x X y Y xy( ) ( ),= forX, Y= E1, E2, I1, I2andx,y = e, i accordingly. Thus only the magnitude of the cross-covariance functions is affected by pooling. The change in magni-tude is quadratic in n

eor n

i. This is consistent with the observation

that pooling amplifies correlations equally over all timescales.The conductances are obtained by convolving the total inputs

with the synaptic filter kernels,

g t t g t t kk kk k iE e I

E and I( ) ( )( ), ( ) ( )( ); , .= = = 1 2

The cross-covariance between the conductances can therefore bewritten as a convolution of the cross-covariance function betweenthe input signals and the deterministic cross-covariance betweenthe synaptic kernels (Tetzlaff et al., 2008). In particular,

C t C t g g XY x yX Y ( ) ( ( ))( )= (15)

for X, Y= E1, E

2, I

1, I

2 and x, y = e, i accordingly, where

( )( ) ( ) ( ) x y x yt s t s s = +

d is the deterministic cross-covariance between the synaptic filters,

xand

y. Note that total

correlations remain unchanged by convolution of the input spiketrains with the synaptic filters, since the integral of a convolutionwill be equal to the product of the integrals ( Tetzlaff et al., 2008).

The total input currents,JK(t) = ( ( )g tEk (VLVI))/Cm, obtained

from the linearization of the conductance-based model describedin the Section Materials and Methods are simply linear combina-tions of the individual conductances. The cross-covariance functionbetween the input currents is therefore a linear combination ofthose between the conductances,

C t V V C t V V C t

V V V

J J L g g L g g

L

h k h k h k( ) | | ( ) | | ( )

| ||

= +

E I

E I

E E I I

2 2

2 V C tL g gh k

| ( ).I E

Combining this result with Eq. (1), yields the correlation, in= J J1 2 , between the total input currents given in Eq. (13).Using the solution of the linearized equations described in the

Section Materials and Methods, we obtain a linear approximationto the cross-covariance functions,

C t C K K t V V J J h k h k( ) ( ( ))( ) 1

2eff

(16)

for h,k = 1,2 where ( )( ) | |/K K t e t = eff eff2

is the cross-covariancebetween the linear kernel, K, and itself. The convolution with(KK)(t) scales the area of both the auto- and cross-covariance

functions by a factor of eff2 ,and therefore leaves the ratio of the

areas, V V1 2 unchanged. Thus, the linear approximation predictsthat V V1 2 in .

When the total inputs are strong, eff

is small and we can simplifyEq. (16) by approximating (KK)(t) with a delta function withmass eff

2 so that C t C t V V J J 1 2 1 2 2( ) ( )/ eff and similarly for C tV V1 1 ( ).This approximation is valid when the synaptic time constants are

significantly larger than eff, which is likely to hold in high conduct-ance states. We compare this approximation to cross-covariancefunctions obtained from simulations in Figure 8.

In all examples considered, the cross-covariance functions haveexponentially decaying tails. We define the correlation time con-stant, xy

txyt C t= lim /ln( ( )),as a measure of the decay rate of the

exponential tail. If txy

, thenx(s) andy(s +t) can be regardedas approximately uncorrelated and xy t

t

xyt s t C s s ( | | / ) ( )d (Stratonovich, 1963).

The time constant of a convolution between two exponen-tially decaying functions is just the maximum time constant ofthe two functions. Thus, from the results above, the correlationtime constant between the membrane potentials is the maximum

of the correlation time constants between the inputs, the syn-aptic time constants, and the effective membrane time constant V V1 2 1 2 1 2 1 2 2 1= max{ , , , , , , }E E I I E I E I e i eff whereE E1 2 ,I I1 2 ,E I1 2 ,andE I2 1are the time constants of the input spike trains and eand iare synaptic time constants. Thus the cross-covariances functionsbetween the membrane potentials are generally broader than thecross-covariance functions between the spike train inputs.

DERIVATION OF EQ. (14)

Consider a feedforward network where each layer consists of Ne

excitatory cells andNiinhibitory cells; each cell in layer kreceives n

e

excitatory and niinhibitory inputs from layer (k 1), and these con-

nections are chosen randomly and independently across neurons in

layer k. Then the degree of overlap in the excitatory and inhibitoryinputs to a pair of cells in layer kis a random variable. Followingthe derivation in Derivation of Eq. (10) in Appendix,

E E

E E

E

e e e e ee e

e e e e ee e1 2

1 2= = +

+ 22 2 2

2 2 2

s n s

n n n

( )

( ),

where sedenotes the number of common excitatory inputs between

the two cells. To understand the origin of se, suppose the n

eexcita-

tory inputs to cell 1 have been selected. Then the selection of the ne

excitatory inputs to cell 2 involves choosing, without replacement,from two pools: the first, of size n

e, projects to cell 1, and the second,

of size (Nen

e), does not. Therefore, s

eis follows a hyper-geometric

distribution with parameters (Ne,ne,ne), and has meann N n pe2

e e e/ .= In addition, this random variable is independently selected amongsteach pair in layer k. Using the mean value of s

e, we obtain Eq. (10).

For simplicity, we assume that E|VEV

L| =I|V

IV

L|, so that

= ni/n

e. If we assume that the statistics in the (k1)stlayer are uniform

across all cells and cell types (i.e., = = = = ee ii eioutk 1and e= i)

then by substituting Eq. (10) and the equivalent forms for II,

EIin to

Eq. (13), we may write the input correlations to the kthlayer as

inE E E I I I E I

E

1 2 1 2= +

| | | | | || |

|

V V V V V V V V

V V

L L L L E I

L

2 2 21 2

|| | | | || |.

2 2 2 2 2 E I I E I E I+ V V V V V V L L L


12/14



Substituting the values of the covariances and variances, anddividing the numerator and denominator by (E|V

EV

L|

e)2,

we get

ine e

2e i i

2i e i

e e2

e i i2

= + + +

+ + + s n s s n s n n

n n n n n

( ) ( )

( ) (

2

nn n ni e i ).

2

Rearranging terms and dividing numerator and denominatorby ni2 ,along with the substitution = n

e/n

i, we have

in

ie i

ie i

= + +( )

+ +

( ) ( )

( ) ( )( )

11

1

11

1

22

22

ns s

nn n

(17)

This equation takes into account the variations in overlap dueto finite size effects since s

eand s

iare random variables. Eq. (14)

in the text represents the expected value P() = inwhich can

be obtained by replacing the variables se and s

i in Eq. (17) with

their respective means, se= n

ep

eand s

i= n

ip

i. The expectation

above is taken over realizations of the random connectivity of thefeedforward network.

To calculate the standard deviation for the inset in Figure 6D,we ran Monte Carlo simulations, drawing s

eand s

ifrom a hyper-

geometric distribution and calculating the resulting transfer,S( ) in in

2= using Eq. (17). Note, however, that Eq. (17) and the insetin Figure 6D, do not account for all of the effects of randomnesswhich may destabilize the balanced network. In deriving Eq. (17),we assumed that the statistics in the second layer were uniform.However, variations in the degree of overlap in one layer will causeinhomogeneities in the variances and rates at the next layer. Ina feedforward setting, these inhomogeneities are compounded ateach layer to destabilize the asynchronous fixed point.

DEFINITIONS AND VALUES OF VARIABLES USED IN THE TEXT

Table 1 | Definitions of variables pertaining to recordings.

X1(t), X

2(t) Signals from two populations.

X X1 2 Correlation between the signals.jk Average pairwise correlation between a cell in populationj and a cell in population k.

r(d) Firing rate of a cell at distance d from the center of a stimulus.

Table 2 | Definitions of variables pertaining to downstream cells.

Subscripts e and E (i and I) denote excitation (inhibition).

ne, n

i Number of correlated inputs to a cell.

e,

i Input rates.

E,I Synaptic weights.

qe, q

i Neurons received n

eq

e(n

iq

i) independent excitatory

(inhibitory) inputs.

ee

, ii,

ei Correlations between pairs of afferents.

xy xy w , Correlations within or between two non-overlapping

populations (x,y = e,i).

pe, p

i Proportion of shared input to the post-synaptic pair.

Ne, N

i Number of cells per layer in feed-forward network model.

Ej, I

j Pooled input spike trains to cellj.

E,

I Standard deviation of pooled excitatory

or inhibitory spike trains.

E E E I I I, ,j k j k j k Covariance between pooled spike trains. E E E I I I, ,j k j k j k Correlations between pooled spike trains. Measure of balance between excitation and inhibition in the inputs.

k kin out, Correlations between inputs to or outputs from cell

pairs in a feedforward network.

CXY

(t) Cross-covariance function between processes Xand Y.

P() Correlations between the pooled inputs to cells in the feedforward model.

S() Correlation between output spike trains in terms of input current correlations between

cell pairs in the feedforward model.

Table 3 | Parameter values for simulations of two downstream cells. For

fields with var, various values of the indicated parameters were used and

are described in the captions. For all simulations, VL= 60 mV, VE= 0 mV,V

I= 90 mV, C

m= 114 pF, g

L= 4.086 nS,

e= 10 ms,

i= 20 ms.

ee

, ii

ei n

e, n

i p

e, p

i q

e, q

i

e,

i(Hz) , (nSms)

Figure 1C 0.05 0 250, 84 0 1 5, 7.5 2.3, 9.2

Figure 1D 0.05 0.05 250, 84 0 1 5 2.3, 13.8

Figure 5 var var var, ne/3 var var 5, 7.5 590/n

e, 4E

Figure 8 0.05 0.05 250, 84 0 1 5 2.3, 13.8

Table 4 | Parameter values for simulations of feedforward networks. The parameter0is the input rate to the first layer, (E,I)

eindicates synaptic weights

for excitatory cells, and (E,I)i

for inhibitory cells. For all simulations, VL

= 60 mV, VE

= 0 mV, VI

= 90 mV, Cm

= 114 pF, gL

= 4.086 nS, e

= 10 ms, i

= 20 ms.

For Figure 6,theoretical values were obtained under the assumption thate=

iand E|V

EV

L| =I|V

IV

L|.

ne, n

i N

e,N

i p

e, p

i

0(Hz)

0 (, )

e, (, )

i(nSms)

Figure 6C 600, 400 12000, 8000 0.05 NA 0 NA

Figure 6D 600, 525 12000, 10500 0.05 NA 0 NA

Figure 7A 225, 75 NA 0 100 0 (1.55,5.67),(3.61,10.82)

Figure 7B 225, 75 NA 0 100 0.05 (1.55,5.67),(3.61,10.82)

Figure 7C 225, 75 4500, 1500 0.05 100 0 (1.55,5.67),(3.61,10.82)


13/14



rons: implications for connectivity,computation, and information cod-

ing.J. Neurosci.18, 387096.Shea-Brown, E., Josic, K., de La ocha, J.,

and Doiron, B. (2008). Correlationand synchrony transfer in integrate-and-fire neurons: basic propertiesand consequences for coding. Phys.Rev. Lett. 100, 108102.

Smith, M. A., and Kohn, A. (2008). Spatialand temporal scales of neuronal cor-relation in primary visual cortex. J.Neurosci.28, 1259112603.

Stark, E., Globerson, A., Asher, I., andAbeles, M. (2008). Correlationsbetween groups of premotor neuronscarry information about prehension.J. Neurosci.28, 1061810630.

Steinmetz, P., Roy, A., Fitzgerald, P., Hsiao,S., Johnson, K., and Niebur, E. (2000).Attention modulates synchronizedneuronal firing in primate somatosen-sory cortex.Nature 404, 187190.

Stratonovich, R. (1963). Topics in theTheory of Random Noise: GeneralTheory of Random Processes. NonlinearTransformations of Signals and Noise.New York, NY: Gordon and Breach.

Super, H., and Roelfsema, P. (2005).Chronic Multiunit Recordings inBehaving Animals: Advantages andLimitations. Development, dynamicsand pathology of neuronal networks:

from molecules to functional circuits:proceedings of the23rd InternationalSummer School of Brain Research,held at the Royal Netherlands Academyof Arts and Sciences, Amsterdam, from2529 August 2003, 263.

Tetzlaff, T., Buschermhle, M., Geisel, T.,and Diesmann, M. (2003). The spreadof rate and correlation in stationarycortical networks. Neurocomputing52, 949954.

Tetzlaff, T., Rotter, S., Stark, E., Abeles, M.,Aertsen, A., and Diesmann, M., (2008).Dependence of neuronal correlationson filter characteristics and marginalspike train statistics.Neural Comput.

20, 21332184.Tiesinga, P. H., Fellous, J.-M., Salinas, E.,

Jos, J. V., and Sejnowski, T. J. (2004).Inhibitory synchrony as a mechanismfor attentional gain modulation. J.Physiol. (Paris)98, 296314.

Troyer, T., and Miller, K. (1997).Physiological gain leads to high ISIvariability in a simple model of acortical regular spiking cell. Neural.Comput. 9, 971983.

integrate-and-fire neurons with slowsynapses. Phys. Rev. Lett. 96, 28101.

Moreno-Bote, R., Renart, A., and Parga,N. (2008). Theory of input spike auto-and cross-correlations and their effecton the response of spiking neurons.Neural Comput.20, 16511705.

Nunez, P., and Srinivasan, R. (2006).Electric Fields of the Brain: TheNeurophysics of EEG. New York, NY:Oxford University Press.

Okun, M., and Lampl, I. (2008).Instantaneous correlation of excita-tion and inhibition during ongoingand sensory-evoked activities. Nat.Neurosci.11, 535537.

Oram, M. W., Fldik, P., Perrett, D. I., andSengpiel, F. (1998). The ideal homuncu-lus: decoding neural population signals.Trends Neurosci. 21, 259265.

Ostoj ic, S., Brunel, N. , and Hakim, V.(2009). How connectivity, backgroundactivity, and synaptic properties shapethe cross-correlation between spiketrains.J. Neurosci.29, 1023410253.

Poort, J., and Roelfsema, P. (2009). Noisecorrelations have little influence on thecoding of selective attention in area v1.Cereb. Cortex 19, 543.

Renart, A., de la Rocha, J., Bartho, P.,Hollender, L., Parga, N., Reyes, A., andHarris, K. (2010). The asynchronousstate in cortical circuits. Science 327,

587590.Reyes, A. (2003). Synchrony-dependentpropagation of firing rate in iterativelyconstructed networks in vitro. Nat.Neurosci. 6, 593599.

Roelfsema, P., Lamme, V., and Spekreijse,H. (2004). Synchrony and covariationof firing rates in the primary visualcortex during contour grouping.Nat.Neurosci. 7, 982991.

Romo, R., Hernndez, A., Zainos, A., andSalinas, E. (2003). Correlated neuronaldischarges that increase coding effi-ciency during perceptual discrimina-tion.Neuron 38, 649657.

Salinas, E., and Sejnowski, T. J. (2000).

Impact of correlated synaptic inputon output firing rate and variabilityin simple neuronal models.J. Neurosci.20, 6193209.

Schneider, A., Lewis, T., and Rinzel, J.(2006). Effects of correlated inputand electrical coupling on syn-chrony in fast-spiking cell networks.Neurocomputing 69, 11251129.

Shadlen, M. N., and Newsome, W. T. (1998).The variable discharge of cortical neu-

ACKNOWLEDGMENTS

We thank Jaime de la Rocha, Brent Doiron and Eric Shea-Brown forhelpful discussions. We also thank the reviewers and the handling

editor for numerous useful suggestions. This work was supportedby NSF Grants DMS-0604429 and DMS-0817649 and a TexasARP/ATP award.

REFERENCESBarreiro, A., Shea-Brown, E., and Thilo,

E. (2009). Timescales of spike-traincorrelation for neural oscillators withcommon drive. Phys. Rev. E81, Arxivpreprint arXiv:0907.3924.

Bedenbaugh, P., and Gerstein, G. (1997).Multiunit normalized cross correla-tion differs from the average single-unit normalized correlation.Neural.Comput. 9, 12651275.

Cteau, H., and Reyes, A. (2006). Relationbetween single neuron and populationspiking statistics and effects on networkactivity. Phys. Rev. Lett. 96, 58101.

Chen, Y., Geisler, W. S., and Seidemann,E. (2006). Optimal decoding of cor-related neural population responsesin the primate visual cortex. Nat.Neurosci. 9, 14121420.

Cohen, M., and Maunsell, J. (2009).Attention improves performance pri-marily by reducing interneuronal corre-lations.Nat. Neurosci.12, 15941600.

Coombes, S., Timofeeva, Y., Svensson, C.,Lord, G., Josic, K., Cox, S., and Colbert,C. (2007). Branching dendrites with res-onant membrane: A sum-over-tripsapproach. Biol. Cybern. 97, 137149.

de la Rocha, J., Doiron, B., Shea-Brown,E., Josic , K., and Reyes, A. (2007).Correlation between neural spiketrains increases with firing rate.Nature

448, 802806.Diesmann, M., Gewaltig, M., and Aertsen,A. (1999). Stable propagation of syn-chronous spiking in cortical neuralnetworks.Nature 402, 529533.

Doiron, B., Rinzel, J., and Reyes, A. (2006).Stochastic synchronization in finitesize spiking networks. Phys. Rev. E 74,30903.

Ecker, A., Berens, P., Keliris, G., Bethge, M.,Logothetis, N., and Tolias, A. (2010).Decorrelated neuronal firing in corticalmicrocircuits. Science 327, 584587.

Gasparini, S., and Magee, J. (2006). State-dependent dendritic computation inhippocampal CA1 pyramidal neurons.

J. Neurosci.26, 2088.Gawne, T., and Richmond, B. (1993). How

independent are the messages carriedby adjacent inferior temporal corticalneurons?J. Neurosci.13, 2758.

Goard, M., and Dan, Y. (2009). Basal fore-brain activation enhances cortical cod-ing of natural scenes. Nat. Neurosci.12, 14441449.

Gutnisky, D., and Dragoi, V. (2008).Adaptive coding of visual information

in neural populations. Nature 452,220224.

Gutnisky, D. A., and Jos ic, K. (2010).Generation of spatio-temporallycorrelated spike-trains and local-field potentials using a multivariateautoregressive process. J. Neurophys.doi:10.1152/jn.00518.2009.

Hertz, J. (2010). Cross-correlations inhigh-conductance states of a modelcortical network.Neural. Comput. 22,427447.

Johnston, D., and Narayanan, R. (2008).Active dendrites: colorful wings of themysterious butterflies.Trends Neurosci.31, 309316.

Kuhn, A., Aertsen, A., and Rotter, S. (2003).Higher-order statistics of input ensem-bles and the response of simple modelneurons.Neural Comput.15, 67101.

Kuhn, A., Aertsen, A., and Rotter, S.(2004). Neuronal integration of syn-aptic input in the fluctuation-drivenregime.J. Neurosci.24, 2345.

Kumar, A., Rotter, S., and Aertsen, A.(2008). Conditions for propagatingsynchronous spiking and asynchro-nous firing rates in a cortical networkmodel.J. Neurosci. 28, 5268.

Li, X., and Ascoli, G. A. (2006).Computational simulation of theinputoutput relationship in hip-pocampal pyramidal cells.J. Comput.

Neurosci. 21, 191209.Litvak, V., Sompolinsky, H., Segev, I., andAbeles, M. (2003). On the transmis-sion of rate code in long feedforwardnetworks with excitatory-inhibitorybalance.J. Neurosci. 23, 3006.

Marlek, P., Koch, C., and Maunsell, J.(1997). On the relationship betweensynaptic input and spike output jitterin individual neurons.Proc. Natl. Acad.Sci. U.S.A 94, 735.

Maynard, E. M., Hatsopoulos, N. G.,Ojakangas, C. L., Acuna, B. D., Sanes, J.N., Normann, R. A., and Donoghue, J. P.(1999). Neuronal interactions improvecortical population coding of movement

direction.J. Neurosci.19, 80838093.Mitchell, J. F., Sundberg, K. A., and Reynolds,

J. H. (2009). Spatial attention decorre-lates intrinsic activity fluctuations inmacaque area v4.Neuron63, 879888.

Morel, D., and Levy, W. (2009). The costof linearization.J. Comput. Neurosci.27, 259275.

Moreno-Bote, R., and Parga, N. (2006).Auto-and crosscorrelogramsfor the spike response of leaky


14/14

Frontiers in Computational Neuroscience www frontiersin org April 2010 | Volume 4 | Article 9 | 14


Citation: Rosenbaum RJ, Trousdale J and

JosicK (2010) Pooling and correlated neuralactivity. Front. Comput. Neurosci. 4:9. doi:

10.3389/fncom.2010.00009

Copyright 2010 Rosenbaum, Trousdale

and Josic. This is an open-access articlesubject to an exclusive license agree-

ment between the authors and the

Frontiers Research Foundation, which

permi ts unrestri cted use, dist ribution,and reproduction in any medium, pro-

vided the original authors and source

are credited.

Vaadia, E., Haalman, I., Abeles, M.,Bergman, H., Prut, Y., Slovin, H., andAertsen, A. (1995). Dynamics of neu-ronal interactions in monkey cortex inrelation to behavioural events.Nature373, 515518.

van Rossum, M., Turrigiano, G., andNelson, S. (2002). Fast propagationof firing rates through layered net-

works of noisy neurons. J. Neurosci.22, 19561966.

Vogels, T. P., and Abbott, L. F., (2005).Signal propagation and logic

gating in networks of integrate-and-fire neurons. J. Neurosci . 25,1078610795.

Womelsdorf, T., Schoffelen, J.-M., Oostenveld, R., Singer, W.,Desimone, R., Engel, A. K., Fries,P., and Jun. (2007). Modulation ofneuronal interactions through neu-ronal synchronization. Science 316,

16091612.Zohary, E., Shadlen, M., and Newsome,

W. (1994). Correlated neuronal dis-charge rate and its implications for

psychophysical performance.Nature370, 140143.

Conflict of Interest Statement: Theauthors declare that the research was con-ducted in the absence of any commercial orfinancial relationships that could be con-strued as a potential conflict of interest.

Received: 26 November 2009; paper pend-ing published: 21 December 2009; accepted:

24 March 2010; published online: 19 April

2010.

Rosenbaum 2010 Pooling

Documents

Transcript of Rosenbaum 2010 Pooling