David Verstraeten

45
RCT tutorial 1 RC tutorial David Verstraeten

description

RC tutorial. David Verstraeten. Reservoir Computing. Random and fixed. Trained. Steps in training and testing. Create random weight matrices Input to reservoir Reservoir to reservoir (Output to reservoir) Scale weight matrices globally to the desired value - PowerPoint PPT Presentation

Transcript of David Verstraeten

Page 1: David Verstraeten

RCT tutorial 1

RC tutorialDavid Verstraeten

Page 2: David Verstraeten

RCT tutorial 2

Reservoir Computing

Random and fixedTrained

Page 3: David Verstraeten

RCT tutorial 3

Steps in training and testing

• Create random weight matrices – Input to reservoir– Reservoir to reservoir– (Output to reservoir)

• Scale weight matrices globally to the desired value• Feed all input samples to the reservoir• Collect reservoir states• Train readout weight using linear regression or

ridge regression

Page 4: David Verstraeten

RCT tutorial 4

Scaling reservoir matrix:Spectral radius

x[k +1] = Ax[k]+ Bu[k +1]

y[k +1] =Cx[k +1]+Du[k +1]

x[k +1] = f Wresresx[k]+Wres

inu[k +1]( )

y[k +1] =Woutoutx[k +1]+Wout

in u[k +1]

LTI system

RC system

ρ =max eig Wresres

( )( )Spectral radius

Page 5: David Verstraeten

RCT tutorial 5

Training

M N P

Inputs Reservoir Outputs

T1

T2

TS

1

2

S

1

2

S

1

2

S

A B

Page 6: David Verstraeten

RCT tutorial 6

Linear/ridge regression

A BWX =

wopt = (ATA)−1ATB

wopt = (ATA + λ I)−1ATB

Linear regression

Ridge regression€

wopt = argminwAw −b

2

wopt = argminwAw −b

2+ λ w

2

Page 7: David Verstraeten

RCT tutorial 7

Pattern generation tasks: training

Fixed connections

Trained connections

Page 8: David Verstraeten

RCT tutorial 8

Pattern generation: generation/testing

WarmupFreerunFixed connections

Trained connections

Page 9: David Verstraeten

RCT tutorial 9

Leaky integrator neurons

Page 10: David Verstraeten

RCT tutorial 10

Global optimization parameters

• Input scaling• Reservoir scaling (spectral radius)• Leak rate• Feedback scaling • Regularization parameter (through cross-

validation)• Not:

– connection fraction (only for spiking neurons)– reservoir size

Page 11: David Verstraeten

RCT tutorial 11

RC toolbox

Page 12: David Verstraeten

RCT tutorial 12

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

Page 13: David Verstraeten

RCT tutorial 13

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

Page 14: David Verstraeten

RCT tutorial 14

Rationale

• Reservoir Computing toolbox:• Box of tools for

– Quick-and-dirty experiments– Large scale parameter sweeps and optimizations

• Modular setup: function hooks for customization– Reservoir types– Learning rules– Scoring and evaluation methods

Page 15: David Verstraeten

RCT tutorial 15

Getting started

• Download from http://snn.elis.ugent.be/rct

• Unpack, go to directory $RCT_ROOT and type:

>> install• For ‘Hello world’ experiment, type

>> rc_simulate

Page 16: David Verstraeten

RCT tutorial 16

Usage

• Default settings– Are ok as starting point for many experiments– Contained in $RCT_ROOT/default_settings/default_*.m

– Look at these scripts! They give an overview of RCToolbox functionality

Page 17: David Verstraeten

RCT tutorial 17

Custom settings• Two ways of overriding defaults:

– Command line>> dataset_generation_function=… @my_dataset_function();>> rc_simulate;

– Custom configuration file

>> custom_configuration=‘my_conf’>> rc_simulate;

– Example: analog_speech.m• Other examples: $RCTROOT/examples/configurations

Page 18: David Verstraeten

RCT tutorial 18

Flow diagram for basic experiment

Page 19: David Verstraeten

RCT tutorial 19

Structure of rc_simulate

Page 20: David Verstraeten

RCT tutorial 20

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

Page 21: David Verstraeten

RCT tutorial 21

Standard topology for classification/regression

InputInput BiasBias

ReservoirReservoir

ReadoutReadout

Mandatory

Optional

Page 22: David Verstraeten

RCT tutorial 22

Connectivity matrix between layers

From/to Input Bias Reservoir Readout

Input 0 0 1 0/1

Bias 0 0 0/1 0/1

Reservoir 0 0 1 1

Readout 0 0 0 0

InputInput BiasBias

ReservoirReservoir

ReadoutReadout

Page 23: David Verstraeten

RCT tutorial 23

Standard topology for classification/regression

Mandatory

Optional BiasBias

ReservoirReservoir

ReadoutReadout

Page 24: David Verstraeten

RCT tutorial 24

Connectivity matrix between layers

From/to Bias Reservoir Readout

Bias 0 0/1 0/1

Reservoir 0 1 1

Readout 0 1 0/1

BiasBias

ReservoirReservoir

ReadoutReadout

Page 25: David Verstraeten

RCT tutorial 25

Topology datastructure in the toolbox

• topology.– layer(1,n)– conn(n,n)– with n = number of layers

• Default settings: – $RCTROOT/default_settings/default_topology.m– Layer 1 : input– Layer 2 : bias– Layer 3 : reservoir– Layer 4 : readout

Page 26: David Verstraeten

RCT tutorial 26

• topology.layer(1:n).– size: integer– is_trained_offline: boolean– is_trained_online: boolean– node_type: string– training_function:function pointer– nonlin_functions: array of function pointers

Topology datastructure in the toolbox

Page 27: David Verstraeten

RCT tutorial 27

• topology.layer(1:n).– init_simulation: function pointer– finish_simulation: function pointer

– dt: float– regul_param: float– scoring: function pointer

Topology datastructure in the toolbox

Page 28: David Verstraeten

RCT tutorial 28

• topology.conn(1:n,1:n).– is_active: boolean– creation_pipeline: array of function pointers

– scaling_factor: float– conn_frac: float [0,1]– discrete_set: float array

Topology datastructure in the toolbox

Page 29: David Verstraeten

RCT tutorial 29

Topology creation pipeline

• Array of function pointers which pass weight matrices

• Generation functions– @gen_rand, @gen_1D, @gen_load

• Assignment functions– @assign_rand, @assign_discrete

• Scaling functions– @scale_specrad, @scale_constant

Page 30: David Verstraeten

RCT tutorial 30

Topology creation pipeline

• Example (default for reservoir to reservoir connection):– topology.conn(3,3).creation_pipeline= {@gen_rand,

@assign_randn, @scale_specrad}

• Can be customized! Function hooks

Page 31: David Verstraeten

RCT tutorial 31

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

Page 32: David Verstraeten

RCT tutorial 32

Dataset generation

• Function hook• Default

– dataset_generation_function=@dataset_narma_10(input_dt, input_maxtime, n_samples)

• Signature– function [inputs outputs] = my_dataset_function(…)– inputs = cell(1,nr_samples)

• inputs{:} = matrix (M,:);• M = input dimensionality

– outputs = cell(1,nr_samples)• outputs{:} = matrix (P,:) or [] for signal generation tasks.• P = output dimensionality

• Example: 10th order NARMA

Page 33: David Verstraeten

RCT tutorial 33

Data struct

• data.layer(1:n).– r : required/teacher forced states– s : simulated states

Page 34: David Verstraeten

RCT tutorial 34

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

Page 35: David Verstraeten

RCT tutorial 35

Training and testing

• cross-validate – Training and testing using cross-validation

• cross_validate_grid- Training and testing using cross-validation with

optimization of regularization parameter

Page 36: David Verstraeten

RCT tutorial 36

Cross-validate• Example: three-fold cross-validation with

three samples.

11

22

33

TrainingTraining

TestingTesting

ResultsResults11

22

33

error fold 1

error fold 1

Fold 1

Page 37: David Verstraeten

RCT tutorial 37

Cross-validate• Example: three-fold cross-validation with

three samples.

11

22

33

TrainingTraining

TestingTesting

ResultsResults11

22

33

error fold 1

error fold 1

Fold 2

error fold 2

error fold 2

Page 38: David Verstraeten

RCT tutorial 38

Cross-validate• Example: three-fold cross-validation with

three samples.

11

22

33

TrainingTraining

TestingTesting

ResultsResults

11

22

33

error fold 1

error fold 1

Fold 3

error fold 2

error fold 2

error fold 3

error fold 3

Page 39: David Verstraeten

RCT tutorial 39

Cross-validation

• Function hook:– train_params.cross_val_set_function– Default: @random_cross_val– Other possibilities:

• cross_val_only_training: no test set• no_cross_val: simple train and test set• random_freerun_cross_val : cross-validation for freerun tasks

• Freerun cross-validation

11

22

TrainingTraining

TestingTesting

11

22

33 33

3

3 3 Fold 1

Page 40: David Verstraeten

RCT tutorial 40

Cross-validate with gridsearch• For optimization of regularization parameter

TrainingTraining

TestingTesting

ResultsResults

1133

error fold 1

error fold 1

Fold 1

ValidationValidation

22

11 22 33

Val. results

Val. results

error error

error error

error nerror n

Subfold 1

22

11

Subfold 2

n opt error fold 2

error fold 2

error fold 3

error fold 3

Fold 2Fold 3

Page 41: David Verstraeten

RCT tutorial 41

Outline

• Rationale, usage and structure

• Topology

• Datasets

• Cross-validation

• Parameter sweeps and custom scripts

Page 42: David Verstraeten

RCT tutorial 42

Parameter sweeps• parameter_ranges = struct( 'name', {}, 'range', {});

• Example– 1D sweep:parameter_ranges = struct(

'name', {‘topology.layer(3).size’},

'range’, {10:10:100});

– 2D sweep (all combinations!):parameter_ranges = struct(

'name', {‘topology.layer(3).size’, ‘topology.layer(1).scale_factor’},

'range', {10:10:100, .1:.1:1});

Page 43: David Verstraeten

RCT tutorial 43

Parameter sweeps

• Results are saved in output_directory• Set save_results=true!

• Plot results of sweep – plot_error(output_directory)

• Get variables from simulation– get_saved_data(output_directory, variables)

Page 44: David Verstraeten

RCT tutorial 44

Custom scripts

• Toolbox is: box of tools you can use to build your own experiments

• Good starting points:– $RCTROOT/examples/tutorial/tut1.m– $RCTROOT/examples/framework/rc_simulate_job.m

Page 45: David Verstraeten

RCT tutorial 45

Questions