The website uses cookies. By using this site, you agree to our use of cookies as described in the Privacy Policy.
I Agree
Shijing Cai
104 articles
My Web Markups - Shijing Cai
  • It is beyond the scope of this article to describe in detail the algorithms developed for filtering, smoothing and prediction. The main goal of this article series is to apply Hidden Markov Models to Regime Detection. Hence the task at hand becomes determining what the current "market regime state" the world is in
  • apply Hidden Markov Models to Regime Detection
  • state
  • Forecasting subsequent
  • Filtering, Smoothing and Prediction
  • state-space modelling
  • conditional multivariate Gaussian distribution
  • observation transition functio
  • choice
  • observation
  • continuous
  • observations
  • asset returns
  • time-invariant
  • state and observation transition functions
  • probability of seeing the observations
  • robability of simply seeing the hidden states
  • joint probability of seeing the full set of hidden states and observations
  • joint density function
  • market regimes
  • remain in that state for some time
  • stay in a particular state
  • observation transition probabilities
  • observati
  • specified state
  • observations
  • detection
  • regime
  • set of discrete states
  • time-invariant transition matrix
  • a joint density function for the observations.
  • observations are the asset returns
  • as hidden "regimes"
  • Discrete-State Markov Chain (DSMC).
  • consist of a small, discrete number of regimes (or "states")
  • time series
  • price derivatives contracts
  • continuous-time machinery of stochastic calculus
  • not considered
  • continuous-time Markov processes
  • ticks
  • quantitative trading
  • OpenAI
  • or "gym"
  • Reinforcement Learning models
  • Partially Observable Markov Decision Processes (POMDP).
  • partially observable
  • controlled
  • optimise the action-selection policy for an agent under a Markov Decision Process model
  • Q-Learning[11]
  • Markov Decision Process (MDP)
  • fully observable, but controlled
  • third "pillar" of machine learning
  • Reinforcement Learning (RL)
  • "controlled" by an agent(s)
  • no need
  • observation states
  • latent states do possess the Markov Property
  • influence the "observations".
  • not directly observable
  • probability transitions
  • there are underlying latent states
  • Hidden Markov Model
  • y partially observable
  • fully autonomous
  • all or part of the information
  • autonomy of the system
  • Markov Models
  • four broad classes
  • algorithmic trading performance varies with and without regime detection.
  • to various assets to detect regimes.
  • Kalman Filter
  • problem of regime detection
  • mathematical theory
  • indirect noisy observations
  • asset returns
  • underlying regime state
  • hidden
  • "noisy" indirect observations
  • "hidden" generative processes
  • Hidden Markov Model
  • statistical time series technique
  • position sizing
  • risk management
  • adjust strategy deployment
  • identify when a new regime
  • regimes
  • categorise
  • detect
  • "market regimes"
  • Hidden Markov Models - An Introduction
91 annotations
  • Machine learning and factor investing
  • to always start by running all code chunks from Chapter 1.
  • please make sure that the environment includes all relevant variables.
  • function name conflicts (especially with the select() function), we use the syntax package::function() to make sure the function call is from the right source
  • A list of the packages we use
  • packages
  • coding requirements
  • R for Data Science
  • REPRODUCIBILITY
  • adaptation of the tensorflow and keras libraries to R.
  • to benefit from Python tools
  • statistics-orientated algorithms
  • more powerful concept than correlation
  • black boxes
  • stocks, accounting quantities (e.g., book value) will not be defined
  • implement more exotic models, like causal graphs (Chapter 14), Bayesian additive trees (Chapter 9), and hybrid autoencoders (Chapter 7).
  • seek to pivot towards allocation methods
  • translated into investment decisions.
  • evaluate sentiment
  • some approaches
  • not be covered
  • predict sales, earning reports, and, ultimately, future returns.
  • leverage textual data from social media, satellite imagery, or credit card logs
  • departs from traditional analyses which rely on price and volume data only, like classical portfolio theory à la Markowitz (1952), or high frequency trading.
  • characteristics of these firm
  • explained
  • differences in the returns of firms
  • asset allocation, quantitative trading and wealth management
  • Factor investing
  • their applications in factor investing.
  • machine learning (ML) tools
  • encourage reproducibility.
  • hands-on R code samples
  • academic references
  • equity asset allocation
  • mainstream machine learning algorithms
36 annotations
  • data providers
  • Errors in loops or variable indexing
  • Persistent series
  • To maximize out-of-sample efficiency
  • model
  • probably much more important
  • inputs
  • dependent variables and the explanatory features
  • choices
  • assumptions
  • economic or econometric framing is key.
  • analyst formulates views on the model
  • no-free lunch theorem
  • key points that we have learned
  • heuristic guesses are often hard to beat.
  • portfolio weighting schemes
  • each part should not be dealt with independently
  • intertwined
  • minimize squared errors
  • the
  • the errors
  • errors
  • errors
  • vanilla quadratic programming is the best way to go
  • modelling families for fff
  • are at least as important.
  • the choice and engineering of inputs,
  • we believe
  • tempting to believe
  • the most crucial part is the choice of fff
  • better out-of-sample.
  • researchers and practitioners
  • monthly returns,
  • dependent variable
  • market capitalization, accounting ratios, risk measures, momentum proxies
  • classical predictors reported in the literature
  • features
  • gather data and to process it
  • the first step
  • shares similarity with panel approaches.
  • the model is common to all assets
  • (fff is not indexed by nnn),
  • computed at time ttt,
  • expected return for time t+1t+1t+1
  • forecast returns in the cross-section.
  • supervised learning
  • focuses predominantly on the prediction part.
  • adapt to changing market conditions.
  • documented asset pricing anomalies
  • detect some hidden patterns
  • future returns depend on firm characteristics.
  • the only as
  • We agree with this stance, and the only
  • assumption
  • model that makes sense economically.
  • since the 2010 decade
  • asset pricing
  • Nonlinear relationships
  • hedge funds
  • academics in financial economics
  • computer scientists and information system experts
  • this drawback does not hold for high-frequency strategies.
  • chronological depth is probably the weak point
  • a few hundred attributes.
  • several thousand stocks (US listed at least)
  • a few hundred monthly observations
  • order of magnitude (in 2019)
  • now sufficiently large to be plugged into ML algorithms
  • size of XX\mathbf{X}
  • high-frequency data and derivative quotes
  • in factor investing
  • principles and ideas
72 annotations