RCT tutorial 1

RC tutorialDavid Verstraeten

RCT tutorial 2

Reservoir Computing

Random and fixedTrained

RCT tutorial 3

Steps in training and testing

• Create random weight matrices – Input to reservoir– Reservoir to reservoir– (Output to reservoir)

• Scale weight matrices globally to the desired value• Feed all input samples to the reservoir• Collect reservoir states• Train readout weight using linear regression or

ridge regression

RCT tutorial 4

Scaling reservoir matrix:Spectral radius

€

x[k +1] = Ax[k]+ Bu[k +1]

y[k +1] =Cx[k +1]+Du[k +1]

€

x[k +1] = f Wresresx[k]+Wres

inu[k +1]( )

y[k +1] =Woutoutx[k +1]+Wout

in u[k +1]

LTI system

RC system

€

ρ =max eig Wresres

( )( )Spectral radius

RCT tutorial 5

Training

M N P

Inputs Reservoir Outputs

T1

T2

TS

1

2

S

…

1

2

S

…

1

2

S

…

A B

RCT tutorial 6

Linear/ridge regression

A BWX =

€

wopt = (ATA)−1ATB

€

wopt = (ATA + λ I)−1ATB

Linear regression

Ridge regression€

wopt = argminwAw −b

2

€

wopt = argminwAw −b

2+ λ w

2

RCT tutorial 7

Pattern generation tasks: training

Fixed connections

Trained connections

RCT tutorial 8

Pattern generation: generation/testing

WarmupFreerunFixed connections

Trained connections

RCT tutorial 9

Leaky integrator neurons

RCT tutorial 10

Global optimization parameters

• Input scaling• Reservoir scaling (spectral radius)• Leak rate• Feedback scaling • Regularization parameter (through cross-

validation)• Not:

– connection fraction (only for spiking neurons)– reservoir size

RCT tutorial 11

RC toolbox

RCT tutorial 12

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

RCT tutorial 13

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

RCT tutorial 14

Rationale

• Reservoir Computing toolbox:• Box of tools for

– Quick-and-dirty experiments– Large scale parameter sweeps and optimizations

• Modular setup: function hooks for customization– Reservoir types– Learning rules– Scoring and evaluation methods

RCT tutorial 15

Getting started

• Download from http://snn.elis.ugent.be/rct

• Unpack, go to directory $RCT_ROOT and type:

>> install• For ‘Hello world’ experiment, type

>> rc_simulate

RCT tutorial 16

Usage

• Default settings– Are ok as starting point for many experiments– Contained in $RCT_ROOT/default_settings/default_*.m

– Look at these scripts! They give an overview of RCToolbox functionality

RCT tutorial 17

Custom settings• Two ways of overriding defaults:

– Command line>> dataset_generation_function=… @my_dataset_function();>> rc_simulate;

– Custom configuration file

>> custom_configuration=‘my_conf’>> rc_simulate;

– Example: analog_speech.m• Other examples: $RCTROOT/examples/configurations

RCT tutorial 18

Flow diagram for basic experiment

RCT tutorial 19

Structure of rc_simulate

RCT tutorial 20

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

RCT tutorial 21

Standard topology for classification/regression

InputInput BiasBias

ReservoirReservoir

ReadoutReadout

Mandatory

Optional

RCT tutorial 22

Connectivity matrix between layers

From/to Input Bias Reservoir Readout

Input 0 0 1 0/1

Bias 0 0 0/1 0/1

Reservoir 0 0 1 1

Readout 0 0 0 0

InputInput BiasBias

ReservoirReservoir

ReadoutReadout

RCT tutorial 23

Standard topology for classification/regression

Mandatory

Optional BiasBias

ReservoirReservoir

ReadoutReadout

RCT tutorial 24

Connectivity matrix between layers

From/to Bias Reservoir Readout

Bias 0 0/1 0/1

Reservoir 0 1 1

Readout 0 1 0/1

BiasBias

ReservoirReservoir

ReadoutReadout

RCT tutorial 25

Topology datastructure in the toolbox

• topology.– layer(1,n)– conn(n,n)– with n = number of layers

• Default settings: – $RCTROOT/default_settings/default_topology.m– Layer 1 : input– Layer 2 : bias– Layer 3 : reservoir– Layer 4 : readout

RCT tutorial 26

• topology.layer(1:n).– size: integer– is_trained_offline: boolean– is_trained_online: boolean– node_type: string– training_function:function pointer– nonlin_functions: array of function pointers

Topology datastructure in the toolbox

RCT tutorial 27

• topology.layer(1:n).– init_simulation: function pointer– finish_simulation: function pointer

– dt: float– regul_param: float– scoring: function pointer

Topology datastructure in the toolbox

RCT tutorial 28

• topology.conn(1:n,1:n).– is_active: boolean– creation_pipeline: array of function pointers

– scaling_factor: float– conn_frac: float [0,1]– discrete_set: float array

Topology datastructure in the toolbox

RCT tutorial 29

Topology creation pipeline

• Array of function pointers which pass weight matrices

• Generation functions– @gen_rand, @gen_1D, @gen_load

• Assignment functions– @assign_rand, @assign_discrete

• Scaling functions– @scale_specrad, @scale_constant

RCT tutorial 30

Topology creation pipeline

• Example (default for reservoir to reservoir connection):– topology.conn(3,3).creation_pipeline= {@gen_rand,

@assign_randn, @scale_specrad}

• Can be customized! Function hooks

RCT tutorial 31

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

RCT tutorial 32

Dataset generation

• Function hook• Default

– dataset_generation_function=@dataset_narma_10(input_dt, input_maxtime, n_samples)

• Signature– function [inputs outputs] = my_dataset_function(…)– inputs = cell(1,nr_samples)

• inputs{:} = matrix (M,:);• M = input dimensionality

– outputs = cell(1,nr_samples)• outputs{:} = matrix (P,:) or [] for signal generation tasks.• P = output dimensionality

• Example: 10th order NARMA

RCT tutorial 33

Data struct

• data.layer(1:n).– r : required/teacher forced states– s : simulated states

RCT tutorial 34

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

RCT tutorial 35

Training and testing

• cross-validate – Training and testing using cross-validation

• cross_validate_grid- Training and testing using cross-validation with

optimization of regularization parameter

RCT tutorial 36

Cross-validate• Example: three-fold cross-validation with

three samples.

11

22

33

TrainingTraining

TestingTesting

ResultsResults11

22

33

error fold 1

error fold 1

Fold 1

RCT tutorial 37

Cross-validate• Example: three-fold cross-validation with

three samples.

11

22

33

TrainingTraining

TestingTesting

ResultsResults11

22

33

error fold 1

error fold 1

Fold 2

error fold 2

error fold 2

RCT tutorial 38

Cross-validate• Example: three-fold cross-validation with

three samples.

11

22

33

TrainingTraining

TestingTesting

ResultsResults

11

22

33

error fold 1

error fold 1

Fold 3

error fold 2

error fold 2

error fold 3

error fold 3

RCT tutorial 39

Cross-validation

• Function hook:– train_params.cross_val_set_function– Default: @random_cross_val– Other possibilities:

• cross_val_only_training: no test set• no_cross_val: simple train and test set• random_freerun_cross_val : cross-validation for freerun tasks

• Freerun cross-validation

11

22

TrainingTraining

TestingTesting

11

22

33 33

3

3 3 Fold 1

…

RCT tutorial 40

Cross-validate with gridsearch• For optimization of regularization parameter

TrainingTraining

TestingTesting

ResultsResults

1133

error fold 1

error fold 1

Fold 1

ValidationValidation

22

11 22 33

Val. results

Val. results

error error

error error

error nerror n

…

Subfold 1

22

11

Subfold 2

n opt error fold 2

error fold 2

error fold 3

error fold 3

Fold 2Fold 3

RCT tutorial 41

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

RCT tutorial 42

Parameter sweeps• parameter_ranges = struct( 'name', {}, 'range', {});

• Example– 1D sweep:parameter_ranges = struct(

'name', {‘topology.layer(3).size’},

'range’, {10:10:100});

– 2D sweep (all combinations!):parameter_ranges = struct(

'name', {‘topology.layer(3).size’, ‘topology.layer(1).scale_factor’},

'range', {10:10:100, .1:.1:1});

RCT tutorial 43

Parameter sweeps

• Results are saved in output_directory• Set save_results=true!

• Plot results of sweep – plot_error(output_directory)

• Get variables from simulation– get_saved_data(output_directory, variables)

RCT tutorial 44

Custom scripts

• Toolbox is: box of tools you can use to build your own experiments

• Good starting points:– $RCTROOT/examples/tutorial/tut1.m– $RCTROOT/examples/framework/rc_simulate_job.m

RCT tutorial 45

Questions