1.2. Monte Carlo Filtering

1.2.1. Description

Monte Carlo filtering is a method for analyzing the sensitivity of a model’s characteristics to its parameters. The approach involves generating draws of parameters from their distribution (e.g. prior, posterior, support) and examining the model’s behavior for each draw.

By comparing the empirical marginal distributions of the draws that lead to desired behavior versus those that do not, key parameters that drive the model’s behavior can be identified.

Monte Carlo filtering is useful for locating the most important parameters of a model that contribute to its behavior.

A key reference in Global sensitivity analysis is [Ratto, 2008].

1.2.2. Quick start examples

A very simple example

 1     x=randn(2^13,8);
 2
 3     y=x(:,1)+x(:,2).^2+x(:,1).*x(:,2);
 4
 5     check_behavior=y<0;
 6
 7     obj=mcf(check_behavior(:).',x.')
 8     %%
 9     cdf_plot(obj)
10     %%
11     scatter(obj)
12     %%
13     correlation_patterns_plot(obj,[],'behave')

In this small example, we have a function y=y(x1,x2) and we interested in finding the combinations of parameters/covariates x1, x2, x3, x4, x5, x6, x7 and x8 for which y<0. In this controlled example, clearly, we know a priori that the only covariates that matter for the result are x1 and x2. But in general we may not know that in advance.

We first sample values for all the covariates and for each combination (x1(i), x2(i), x3(i), x4(i), x5(i), x6(i), x7(i), x8(i)), for i=1,2,…,2^13, we compute y(i) and set a flag telling us whether y(i) is negative for combination i.

Then we proceed to investigate whether there are some patterns among the different covariates both in situations where y<0 and in situations where y>=0.

A simple DSGE example

 1     %% A DSGE model
 2
 3     simpleModel={
 4         'model:someModel'
 5         '%% model without parameters and shocks'
 6         '@endogenous X1, X2, X3'
 7         '@exogenous E1, E2, E3'
 8         '@parameters a1, a2, a3, vol_tp_1_2, vol_tp_2_1'
 9         '@parameters(vol,2) s1, s2, s3'
10         '@model'
11         'X1=a1*X1{-1}+s1*E1;'
12         'X2=a2*X2{-1}+s2*E2;'
13         'X3=a3*X3{-1}+s3*E3;'
14         };
15
16     %% The parameters to vary and their ranges
17
18     plist={'a1', 'a2', 'a3', ...
19         'vol_tp_1_2', 'vol_tp_2_1',...
20         's1_vol_1', 's2_vol_1', 's3_vol_1',...
21         's1_vol_2', 's2_vol_2', 's3_vol_2'
22         };
23
24     bounds=[0.5,0.9;0.5,0.9;0.5,0.9;0,1;0,1;0,0.5;0,0.5;0,0.5;0.5,2;0.5,2;0.5,2];
25
26     %% build the model and set the behavior to investigate in a function handle
27
28     m=dsge_model(simpleModel);
29
30     % To do : write a simple function handle check that the model solves or that some correlations match
31     % Behavior = @...
32
33     %% run the behavior through the mcf
34
35     obj=mcf(Behavior,nsim_or_draws,bounds(:,1),bounds(:,2),plist,'latin_hypercubes');
36
37
38     %% smirnov test for equality of distributions
39     close all
40     % hfig=utils.plot.multiple(@(x)cdf_plot(obj,x),parameter_names,...
41     %     'Smirnov test of equality of distributions',3,3)
42     %
43     cdf_plot(obj)
44
45     pause
46
47     %% correlation patterns in the behavior sample
48     close all
49
50     correlation_patterns_plot(obj,[],'')
51
52     pause
53
54     %% correlation patterns in the behavior sample
55     close all
56
57     correlation_patterns_plot(obj,[],'behave')
58
59     pause
60
61     %% correlation patterns in the behavior sample
62     close all
63
64     correlation_patterns_plot(obj,[],'non-behave')
65
66     pause
67
68     %% scatter plot of the significant correlations
69     close all
70
71     scatter(obj,[],'',1/100)
72
73     pause
74     %% scatter plot of the significant correlations
75     close all
76
77     scatter(obj,[],'non-behave')

This second example transposes the workflow above to a DSGE model with regime-switching shock volatilities. The model has three AR(1) endogenous variables X1, X2, X3, three shocks E1, E2, E3, and two volatility regimes (vol) governed by the transition probabilities vol_tp_1_2 and vol_tp_2_1. The standard deviations s1, s2, s3 are declared switching over the vol chain, so each one carries two values in the parameter table (s1_vol_1 / s1_vol_2, etc.).

The parameter list plist collects every parameter the analyst wants to vary: the AR(1) persistences a1-a3, the two transition probabilities, and both regime values of each volatility. bounds gives the lower and upper sampling range for each.

The placeholder Behavior would be a function handle of the form @(x) someTest(x) that returns true when the sampled parameter vector x produces the model behaviour we want to single out – for instance, that the model solves, that a particular impulse response falls in a target range, or that a moment condition is met. The following section (Support facility for filtration) shows mcfprop as a convenient builder for that handle.

With a Behavior handle in place, mcf runs the same battery as in the previous example: cdf_plot reports the marginal distribution of each parameter conditional on behaviour vs. non-behaviour; correlation_patterns_plot highlights pairwise patterns within the behave / non-behave / full samples; scatter shows the joint distribution of the most informative pairs.

1.2.3. Details on the mcf class

Contents:

1.2.4. Support facility for filtration : the mcfprop class

Contents:

An example

 1     %% construct the support object
 2
 3     m=dsge_model(simpleModel);
 4
 5     self = mcfprop(m, plist);
 6
 7     %% Add a few properties
 8     self = self.add('solve'); % the model solves
 9     self = self.add({'corr{X2,X2,3:8,2}',@(x)x>0.01}); % correlation or order 3 in regime 2
10     self = self.add({'corr{X3,X3,0,2}',@(x)abs(x-1)<1e-9}); % correlation or order 0 in regime 2
11     self = self.add({'irf{X1,E1}',@(x)x>0.01}); % irf in period 1
12     self = self.add({'irf{X1,E1,3}',@(x)x>0.01}); % irf in period 3
13     self = self.add({'irf{X2,E2,3:5,2}',@(x)all(x>1e-3)}); % irf in period 3 to 5 in regime 2
14     self = self.add({'Tz{X1,E1}', @(x) x > 0}); % regime 1
15     self = self.add({'Tz{X1,@sig,2}', @(x) abs(x)<1e-8}); % regime 1
16
17     %% Create a function handle for the behavior
18
19     n=size(bounds,1);
20
21     propfun = create_handle(self);
22
23     %% Evaluate one parameter vector
24
25     x = bounds(:,1)+ rand(n,1).*(bounds(:,2)-bounds(:,1));
26
27     eval_ = propfun(x);

mcfprop is the builder side of mcf: it lets the user assemble a behaviour-checking function from named model properties without writing it by hand. Each call to self.add(...) registers one property to evaluate. The supported property names mirror RISE’s post-solve accessors:

  • 'solve' – the model solved successfully at the candidate parameter vector.

  • 'corr{Xi,Xj,k:l,r}' – the autocovariance / cross-covariance of order k (or range k:l) between variables Xi and Xj, in regime r.

  • 'irf{Xi,Ej,k:l,r}' – the impulse response of Xi to shock Ej at horizon k (or range), in regime r.

  • 'Tz{Xi,Ej,r}' / 'Tz{Xi,@sig,r}' – direct entries of the policy function in regime r (impact of shock Ej, or of the perturbation parameter @sig).

The optional second element of the cell is a predicate @(x) bool that the value of the property must satisfy for the parameter vector to count as behaving. Without a predicate the property’s value is simply recorded.

create_handle(self) collapses the entire menu into a single callable propfun(x) whose return value is the joint truth of all predicates. That handle is exactly what the Behavior slot of mcf expects – closing the loop from a higher-level statement of “model properties of interest” to a low-level boolean signal driving the sensitivity analysis.

Hint

With the handle/behavior in hand we can go ahead and pass it to the mcf class as above