2. Time Series and Data Management

2.1. Dates

In the RISE toolbox, dates play a crucial role in representing time series data. Dates can be either explicit, where each observation is associated with a specific date, or undated, where dates are simply represented as a sequence of integers.

To handle dates efficiently, RISE provides an abstract class called rise_dates.dates, from which various derived classes inherit. These derived classes include:

  • rise_dates.DailyDate (shortcut: rd)

  • rise_dates.WeeklyDate (shortcut: rw)

  • rise_dates.MonthlyDate (shortcut: rm)

  • rise_dates.QuarterlyDate (shortcut: rq)

  • rise_dates.HalfYearlyDate (shortcut: rh)

  • rise_dates.YearlyDate (shortcuts: ry or ra)

Each derived class represents a specific frequency of dates, such as daily, weekly, monthly, etc.

2.1.1. Base Class Methods

The base class rise_dates.dates provides several methods that can be used with any derived class. These methods allow you to manipulate and analyze date objects effectively. Here are the commonly used methods:

  • char(): Converts dates to character representations.

  • convert(): Converts dates to another frequency.

  • datestr(): Returns a string representation of dates.

  • eq(): Checks if dates are equal.

  • gt(): Checks if dates are greater than a given date.

  • ismember(): Checks if dates are members of a given set of dates.

  • lt(): Checks if dates are less than a given date.

  • min(): Returns the minimum date.

  • ne(): Checks if dates are not equal.

  • unique(): Returns unique dates in a sequence.

  • colon(): Creates a sequence of dates.

  • double(): Converts dates to double format.

  • ge(): Checks if dates are greater than or equal to a given date.

  • intersect(): Returns the intersection of two sets of dates.

  • le(): Checks if dates are less than or equal to a given date.

  • max(): Returns the maximum date.

  • minus(): Computes the difference between two dates.

  • plus(): Adds a given number of periods to a date.

  • ymd(): Extracts the year, month, and day components of dates.

These methods provide powerful functionality for manipulating, comparing, and performing operations on date objects. For more detailed information on each method, refer to the documentation.

2.1.2. Help Text for Derived Classes

To make it easier for users to understand and use the derived classes, we have provided detailed help text for each of them (rd, rw, rm, rq, rh, ry, and ra). The help text contains information about the specific properties and methods available for each class.

Daily dates : rd

RD DailyDate constructor.

Usage:

obj = rd(year, month, dayOfTheMonth) creates a

rd object for the specified year, month, and day.

obj = rd(year, dayOfTheYear) creates a rd

object for the specified year and day of the year.

obj = rd(dateString) creates a rd object from

a string in the format ‘mm/dd/yyyy’ or ‘yyyyDddd’ or ‘yyyy-mm-dd’.

obj = rd(dateNumber) creates a rd object from

a serial date number.

Inputs:

  • year: Year of the date.

  • month: Month of the date.

  • dayOfTheMonth: Day of the month.

  • dayOfTheYear: Day of the year.

  • dateString: A string in the format ‘mm/dd/yyyy’ or ‘yyyyDddd’.

  • dateNumber: A serial date number representing the date.

Example:

dateObj = rd(2023, 7, 15);

dateObj = rd(2023, 165);

dateObj = rd(‘07/15/2023’);

dateObj = rd(‘2023D165’);

dateObj = rd(737999);

See also rd, rw, rm, rq, rh, ra, ry

Weekly dates : rw

RW constructor.

Usage:

obj = rw(year, fullWeeks)

obj = rw(dateString)

obj = rw(datenumber)

Inputs:

  • year: Year of the date.

  • fullWeeks: Number of full weeks.

  • dateString: A string in the format ‘mm/dd/yyyy’ or ‘yyyyWp’ or ‘yyyy-mm-dd’.

Example:

dateObj = rw(2023, 10);

dateObj = rw(‘07/01/2023’);

dateObj = rw(‘1990W1’);

dateObj = rw(737999);

See also rd, rm, rq, rh, ra, ry

Monthly dates : rm

RM constructor.

Usage:

obj = rm(year, month)

obj = rm(dateString)

obj = rm(datenumber)

Inputs:

  • year: Year of the date.

  • month: Month of the date (1 to 12).

  • dateString: A string in the format ‘mm/dd/yyyy’ or ‘yyyy-mm-dd’.

  • datenumber: regular date number

Example:

dateObj = rm(2023, 7);

dateObj = rm(‘07/01/2023’);

dateObj = rm(737999);

See also rd, rw, rq, rh, ra, ry

Quarterly dates : rq

RQ constructor.

Usage:

  • obj = rq(year, quarter)

  • obj = rq(dateString)

  • obj = rq(datenumber)

Inputs:

  • year: Year of the date.

  • quarter: Quarter of the date (1, 2, 3, or 4).

  • dateString: A string in the format ‘mm/dd/yyyy’ or ‘yyyyQp’ where p=1,2,3,4 or ‘yyyy-mm-dd’

Example:

dateObj = rq(2023, 2);

dateObj = rq(‘2023Q2’);

dateObj = rq(‘04/01/2023’);

dateObj = rq(737999);

See also rd, rw, rm, rh, ra, ry

Half-yearly dates : rh

RH constructor.

Usage:

obj = rh(year, semi)

obj = rh(dateString)

obj = rh(datenumber)

Inputs:

  • year: The year of the date.

  • semi: The semi-annual period (1 or 2).

  • dateString: A string in the format ‘mm/dd/yyyy’ or ‘yyyyHp’ where p=1,2 or ‘yyyy-mm-dd’.

  • datenumber: regular date number

Example:

dateObj = rh(2023, 1);

dateObj = rh(‘2023H1’);

dateObj = rh(‘01/01/2023’);

dateObj = rh(737999);

See also rd, rw, rm, rq, ra, ry

Yearly dates : ry

RY constructor for YearlyDate.

Usage:

obj = ry(year)

obj = ry(dateString)

dateObj = ry(datenumber);

Inputs:

  • year: Year of the date in double (e.g., 2023)

  • dateString: A string in the format ‘mm/dd/yyyy’ or ‘yyyy’ or ‘yyyy-mm-dd’

  • datenumber: A regular matlab datenumber. Note that the function will treat the input as a date number if it has more than 4 digits. A warning will be issued as well

Example:

dateObj = ry(2023);

dateObj = ry(‘2023’);

dateObj = ry(‘01/01/2023’);

dateObj = ry(737999);

See also rd, rw, rm, rq, rh, ra.

Annual dates : ra

RA constructor for YearlyDate.

Usage:

obj = ra(year)

obj = ra(dateString)

dateObj = ra(datenumber);

Inputs:

  • year: Year of the date in double (e.g., 2023)

  • dateString: A string in the format ‘mm/dd/yyyy’ or ‘yyyy’ or ‘yyyy-mm-dd’

  • datenumber: A regular matlab datenumber. Note that the function will treat the input as a date number if it has more than 4 digits. A warning will be issued as well

Example:

dateObj = ra(2023);

dateObj = ra(‘2023’);

dateObj = ra(‘01/01/2023’);

dateObj = ra(737999);

See also rd, rw, rm, rq, rh, ry

Once you are familiar with the concepts and usage of date objects in the RISE toolbox, you can proceed to explore the handling of time series data, which builds upon the foundation of dates.

2.1.3. Technical documentation for rise_dates.dates objects

Contents:

2.2. Time Series

This section provides information about time series and their usage in the context of the RISE toolbox. It covers topics such as:

  • Introduction to time series analysis

  • Working with time series data in RISE

  • Time series transformations and manipulations

  • Statistical analysis techniques for time series

Time series data in essence is just a matrix, so in theory, one can just use matrices to represent the data. However, all practitioners will share their frustration in trying to remember the index location of the GDP and the annoyance in trying to find out what data(:,5) written by a co-worker represents. The RISE toolbox addresses this problem by providing a time series object that uses natural indices, improving readability and reducing errors.

To use the time series object, you can initialize it with the following code:

db = ts('1990q1', randn(10,4), {'gdp', 'r', 'wage', 'capital'})

The first argument specifies the starting date of the time series and its frequency. In this example, the time series starts from the first quarter of 1990. The second argument is the data matrix, where each row represents a time period and each column represents a variable. Here, the time series consists of 4 variables spanning 10 quarters. Lastly, the third argument provides the natural names of the variables for intuitive indexing.

If you want to provide descriptions or comments for each variable, you can construct the time series object with optional descriptions:

db = ts('1990q1', randn(10,2), {'gdp', 'pi'}, {'real gdp', 'inflation rate'})

You can display the time series, including its descriptions/comments, by calling:

disp(db)

Alternatively, you can set variable names and descriptions using the set function:

set(db, 'varnames', {'gdp', 'pi'}, 'description', {'real gdp', 'inflation rate'})

2.2.1. Indexing and assignment

The indexing surface of the time series object mirrors the built-in table / timetable conventions, so that users familiar with those classes feel at home, and the distinction between “sub-time-series” and “raw values” is explicit.

Expression

Result

db.PROP

a property of the ts (data, dates, frequency, NumberOfObservations, …).

db.VARNAME

if VARNAME matches a variable name (and is not a property/method), returns the single-variable ts.

db(I[,J[,K]])

a sub-time-series (returns a ts). I, J, K are dates/date strings/logical/cellstr/positions/:/ function-handle filters on pages.

db{I[,J[,K]]}

the raw numeric block at that selection (a double array of size nobs x nvars x npages). Use this when the next step is plain MATLAB arithmetic; otherwise stay in the ts world with ().

db{-k} / db(-k)

(dated ts only, k a scalar integer) shortcut for lag(db, k); returns a ts. db{+k} / db(+k) is lead(db, k). db{0} is db. On undated ts a scalar numeric is the implicit date value instead.

db.VARNAME = rhs

replace one variable’s column from a ts or a numeric column.

db(I,...) = rhs

replace a sub-time-series (rhs ts or numeric).

db{I,...} = rhs

raw numeric block write.

Scalar-numeric assignment (db{-1} = X, db(-1) = X) is not supported. The time-shift shortcut is read-only; build the shifted ts first and assign with a date or logical selector if you need to write into it.

Examples:

db = ts('1990q1', randn(10,4), {'gdp', 'r', 'wage', 'capital'});

% Access values
db.wage                          % single-variable ts
db('wage')                       % same, sub-ts form
db{'wage'}                       % raw 10x1 double array

% Slice rows and columns
db('1991q2', 'wage')             % single observation, as a ts
db{'1991q2', 'wage'}             % the same value as a 1x1 double
db('1990q2:1991q2', {'gdp','wage'})   % sub-ts (5 rows, 2 vars)
db{:, {'gdp','wage'}}            % full sample, two vars, as a 10x2 double

% Whole-year selection works across frequencies
db(ry(1991))                     % the 4 quarters of 1991, as a ts

% Underlying data, when you need plain MATLAB
values(db)                       % equivalent to db.data

Time shifts (lags and leads) are expressed either with the explicit methods lag(db, k), lead(db, k), shift(db, k) or, on a dated ts, with the brace/paren shortcut. Both preserve the date axis - the leading or trailing k observations are NaN-padded so that the length of the output equals the length of the input. The two forms are equivalent on a dated ts:

growth = 100*(db.gdp / lag(db.gdp, 1) - 1);   % q/q growth, % terms
growth = 100*(db.gdp / db.gdp{-1}      - 1);  % same thing, shorter

On an undated ts the shortcut is unavailable (a scalar numeric in the braces is treated as the date value itself); use lag/lead/ shift explicitly there.

The time series object seamlessly integrates with other parts of the RISE toolbox, enabling you to perform VAR and DSGE analyses effortlessly. The automation provided by the time series object simplifies various data management tasks. The following are some commonly used functions and their functionalities:

2.2.2. Data Cleaning

  • reset_data: Resets the data in the time series object.

  • reset_start_date: Resets the start date of the time series object.

  • interpolate: Fills in missing data by interpolating.

  • aggregate: Aggregates data to a coarser frequency (e.g., monthly data to quarterly).

  • dummy: Creates a dummy variable.

  • step_dummy: Creates a step dummy variable.

  • cat: Joins multiple time series objects.

2.2.3. Time Series Transformations

  • hpfilter: Applies the Hodrick-Prescott (HP) filter to the time series data.

  • ma_filter: Applies a moving-average filter to the time series data.

  • pdecomp: Performs a parametric trend-seasonal-residual decomposition.

  • npdecomp: Performs a non-parametric trend-seasonal-residual decomposition.

  • transform: applies a standard time-series transformation (level, growth, difference, log change, year-over-year variants) following the Haver mnemonics.

2.2.4. Time shifts

These methods preserve the date axis - the leading or trailing k rows are NaN-padded so that length(out) == length(db):

  • lag(db, k): lag-by-k. out.data(t,:,:) = db.data(t-k,:,:).

  • lead(db, k): lead-by-k. out.data(t,:,:) = db.data(t+k,:,:).

  • shift(db, k): generic shift. Positive k is a lead, negative is a lag.

On a dated ts these are also available as the shortcut db{-k} / db(-k) (lag) and db{+k} / db(+k) (lead) - see the Indexing and assignment section above.

2.2.5. Statistical Analysis

  • regress: Performs OLS regression on the time series data.

  • chowlin: Temporal disaggregation using the Chow-Lin method (interpolates a low-frequency series to high frequency using one or more related indicators).

  • jbtest: Performs the Jarque-Bera (JB) test.

2.2.6. Plotting tools for time series

  • bar: Creates a bar plot.

  • fanchart: Generates a fan chart.

  • plot: Plots the time series data.

  • plot_real_time: Plots the time series data in real time conditioning.

  • plotyy: Creates a plot with two separate axes.

2.2.7. Other Automations

  • transform: Provides a variety of transformations for the time series data.

For a comprehensive understanding of all the automation and functionalities available in the time series object, refer to the ts-technicals documentation.

2.2.8. Technical documentation for ts objects

Contents: