The website uses cookies. By using this site, you agree to our use of cookies as described in the
Privacy Policy
.
I Agree
My Markups
Toggle navigation
Login
Upgrade
Shijing Cai
104 articles
My Web Markups - Shijing Cai
access to the article?
send me your CV please
present your work before our research team
The submission deadline is 9am UK time on Monday, 18th Jan.
邮件 - scai@student.ethz.ch
4 annotations
mail.ethz.ch
399
The law of total probability, can also be stated for conditional probabilities.
Law of total probability - Wikipedia
1 annotation
en.wikipedia.org
454
The test is available for the next 7 days.
邮件 - scai@student.ethz.ch
1 annotation
mail.ethz.ch
368
Swap historical volatility or Cash historical volatility
Swaption implied volatility
What IR Historical Data Can You Chart?
2 annotations
www2.superderivatives.com
37
How to make 3D line plot in R (waterfall plot) - Stack Overflow
stackoverflow.com
548
American Option Definition
www.investopedia.com
599
value of call options also drops in the time leading up to the ex-dividend date
price of the stock drops on the ex-dividend date
Call options become cheaper
to capture the dividend payment owed to the underlying shares.
Holders of deep-in-the-money American-style calls may choose to exercise those options early
Understanding How Dividends Affect Option Prices
5 annotations
www.investopedia.com
569
RStudio - RStudio
rstudio.com
586
Averaging Across Asset Allocation Models in: Jahrbücher für Nationalökonomie und Statistik Volume 235 Issue 1 (2015)
www.degruyter.com
572
ShijingCAI_thesis - Online LaTeX Editor Overleaf
www.overleaf.com
583
Newsletters - The Wall Street Journal
www.wsj.com
575
Drawdown Definition and Example
www.investopedia.com
602
邮件 - scai@student.ethz.ch
mail.ethz.ch
521
10+ Best Answers for Why Do You Want to Work Here?
10+ Best Answers for Why Do You Want to Work Here?
1 annotation
novoresume.com
628
It is beyond the scope of this article to describe in detail the algorithms developed for filtering, smoothing and prediction. The main goal of this article series is to apply Hidden Markov Models to Regime Detection. Hence the task at hand becomes determining what the current "market regime state" the world is in
apply Hidden Markov Models to Regime Detection
state
Forecasting subsequent
Filtering, Smoothing and Prediction
state-space modelling
conditional multivariate Gaussian distribution
observation transition functio
choice
observation
continuous
observations
asset returns
time-invariant
state and observation transition functions
probability of seeing the observations
robability of simply seeing the hidden states
joint probability of seeing the full set of hidden states and observations
joint density function
market regimes
remain in that state for some time
stay in a particular state
observation transition probabilities
observati
specified state
observations
detection
regime
set of discrete states
time-invariant transition matrix
a joint density function for the observations.
observations are the asset returns
as hidden "regimes"
Discrete-State Markov Chain (DSMC).
consist of a small, discrete number of regimes (or "states")
time series
price derivatives contracts
continuous-time machinery of stochastic calculus
not considered
continuous-time Markov processes
ticks
quantitative trading
OpenAI
or "gym"
Reinforcement Learning models
Partially Observable Markov Decision Processes (POMDP).
partially observable
controlled
optimise the action-selection policy for an agent under a Markov Decision Process model
Q-Learning[11]
Markov Decision Process (MDP)
fully observable, but controlled
third "pillar" of machine learning
Reinforcement Learning (RL)
"controlled" by an agent(s)
no need
observation states
latent states do possess the Markov Property
influence the "observations".
not directly observable
probability transitions
there are underlying latent states
Hidden Markov Model
y partially observable
fully autonomous
all or part of the information
autonomy of the system
Markov Models
four broad classes
algorithmic trading performance varies with and without regime detection.
to various assets to detect regimes.
Kalman Filter
problem of regime detection
mathematical theory
indirect noisy observations
asset returns
underlying regime state
hidden
"noisy" indirect observations
"hidden" generative processes
Hidden Markov Model
statistical time series technique
position sizing
risk management
adjust strategy deployment
identify when a new regime
regimes
categorise
detect
"market regimes"
Hidden Markov Models - An Introduction
Hidden Markov Models - An Introduction | QuantStart
91 annotations
www.quantstart.com
645
One clever way to find the elbow i
how do we choose the quantity of clusters k?
Clustering: “Two’s company, three’s a crowd” | Quantdare
2 annotations
quantdare.com
600
我们发现您的商品与订单的描述不符，需要您进一步确认（详见留言板）您确认商品无问题后，可正常提交包裹 若需退货，请您先确认信息，待商品入库之后
我的服务中心-用户中心-Superbuy.com-海外华人留学生代购淘宝平台
1 annotation
www.superbuy.com
615
2020 Quantitative Finance Off-cycle Internship (London) - Morgan Stanley Campus
morganstanley.tal.net
420
Quantica_Interview.ipynb - Colaboratory
colab.research.google.com
594
Load and Explore Time Series Data in Python
How to Load and Explore Time Series Data in Python
1 annotation
machinelearningmastery.com
662
e
SPY 326.54 -3.44 -1.04% : SPDR S&P 500 - Yahoo Finance
1 annotation
finance.yahoo.com
640
log likelihood of the observation
python - How to increase HMM score on training samples with `hmmlearn` - Stack Overflow
1 annotation
stackoverflow.com
599
dividing line isn’t always clear.
some hedge funds use similar strategies
serving clients of the bank and executing trades on their behalf
sales & trading is
security prices
hedge funds
exploiting small pricing inefficiencies (market-making)
prop trading firms
Traders
trading algorithms
mathematical models
intuition and judgment
model/software/automated approach
manage risk
buy and sell securities
market-making.
market-making
directional
buy and sell securities using the firm’s own money to make a profit
proprietary trading
Proprietary Trading: Full Career and Recruiting Guide
20 annotations
www.mergersandinquisitions.com
676
客服服务-superbuy
www.superbuy.com
594
convert the FF data to decimal and create a new column called R_excess to hold our returns above the risk-free rate.
Introduction to Fama French · R Views
1 annotation
rviews.rstudio.com
464
If TRUE, will be stored as a factor, which preserves the original ordering of the columns.
Gather columns into key-value pairs — gather • tidyr
1 annotation
tidyr.tidyverse.org
437
where we put some short examples
that this is a function that we want to be available to the user
describe what the function returns.
we need to provide a description.
describes the parameters (or arguments) for the function
paragraph giving a more detailed description of the function
blank line.
first line is the title of the function
Documenting functions
5 Documenting functions | R package workshop
9 annotations
combine-australia.github.io
425
choosing the initial number of states
switching behaviour between Regime #2 and Regime #3 in the calmer period of 2004-2007.
somewhat trickier to interpre
utilise three states.
fitting the HMM with two regime states to S&P500 returns
nor is there any "ground truth" on which to "train" the HMM.
number of states is not known
an unsupervised learning
Full Code
improve profitability
backtesting and live trading engine
hree-state HMM
high posterior probability to Regime #2
e EM algorithm
two-state Hidden Markov Model
bearish
bullish returns
low variance
positive mean
"bullish" or "bearish" market regime.
a third intermediate state
daily returns data in equities markets
not clear how many regime states exist
. Are there two, three, four or more "true" hidden market regimes?
f unsupervised learning.
two-state underlying regimes.
identify the probability of being in a particular regime state.
two separate regimes - "bullish" and "bearish"
adjust trade signal suggestions
when US equities markets are in various regime states.
Hidden Markov Models for Regime Detection using R | QuantStart
30 annotations
www.quantstart.com
607
all the objects defined in your current workspace,
all of the objects and functions you have either defined in the current session, or have loaded from a previous session.
working environment)
workspace
YaRrr! The Pirate’s Guide to R
4 annotations
bookdown.org
341
Backtesting Minimum Variance portfolios
Backtesting Minimum Variance portfolios | R-bloggers
1 annotation
www.r-bloggers.com
391
or re-load it from an external file.
Thankfully, if your R code is complete and well-documented, you should easily be able to either re-create a lost object
YaRrr! The Pirate’s Guide to R
2 annotations
bookdown.org
366
UZH - Master of Science UZH ETH in Quantitative Finance - Administrative Steps
www.msfinance.uzh.ch
385
chain together these period returns
the assumption is that you rebalance the portfolio at the close of the previous period to achieve these weights
summing across assets will give you the portfolio return for that period
for that period; then summing across assets will give you the portfolio return for that period
With rebalancing
a specific period
multiply each asset's return
with its initial weight
multiply the total return of each asset
initial weight
Without rebalancing
get the portfolio return up to some point in tim
but 46% and 54%.
after period 1, the weights of the assets are not 50% each
without rebalancing,
is in the LHS
Portfolio forward return - Quantitative Finance Stack Exchange
16 annotations
quant.stackexchange.com
420
Source script to separate environment in R, not the global environment - Stack Overflow
stackoverflow.com
318
depmix(list(LogReturns~1,ATR~1)
Inovance - A Tutorial in R on Using A Hidden Markov Model (HMM)
1 annotation
inovancetech.com
679
85 This answer is not useful Show activity on this post. No, scaling is not necessary for random forests.
random forest - Do I need to normalize (or scale) data for randomForest (R package)? - Stack Overflow
1 annotation
stackoverflow.com
621
RStudio automatically detects the presence of version control for projects by scanning for a .git or .svn directory.
indexed for code navigation
automatically added to .Rbuildignore, .gitignore, etc. if required.
stored
project-specific temporary files
hidden directory
a project file (with an .Rproj extension) within the project directory
shortcut for opening the projec
various project options
Using Projects – RStudio Support
9 annotations
support.rstudio.com
326
your first package development workflow
oop - Loading object into global environment in R Package using .onLoad() - Stack Overflow
1 annotation
stackoverflow.com
317
a directory of files with a specific structur
Chapter 4 Package structure and state | R Packages
1 annotation
r-pkgs.org
492
Plotting two variables as lines using ggplot2 on the same graph Ask Question
r - Plotting two variables as lines using ggplot2 on the same graph - Stack Overflow
1 annotation
stackoverflow.com
402
Is train/test-Split in unsupervised learning necessary/useful? Ask Question
Is train/test-Split in unsupervised learning necessary/useful? - Stack Overflow
1 annotation
stackoverflow.com
583
probability of each regime
high volatility
medium volatility
low volatility
14-period average true range (ATR) indicator
daily log returns
three regimes
uses two technical indicators
Hidden Markov Models
Identifying Market Conditions Using Machine Learning | Investing.com
9 annotations
www.investing.com
642
will run code multiple times
comparing the speed of several functions
running small sections of code to assess performance
profiling and timing tools
Mastering Software Development in R
4 annotations
rdpeng.github.io
679
accumulated the returns of the different market regimes
characteristics
Individual days are grouped
o
optimal number of clusters
assign 6 clusters (regimes).
to the
principal components
k-means clustering
PCA to reduce dimensions
over the price series
a variety of characteristics
Price Index evolution
“unknown” regime definitions
extreme up
+1
-1
Extreme downtrend
labels between -1 and 1
multiple classes
separations
magnitude
direction
choosing an appropriate magnitude
one direction or another.
magnitude of its movements
separates the series
algorithm
movements.
magnitude and sequence
centred averages
include the price evolution.
daily returns.
3 classes
6 “unknown” classes
down
up
“known” classes
Price
Classification of Market Regimes | Quantdare
39 annotations
quantdare.com
613
provide a wrapper function
set user-friendly defaults, introduce helpful interactive behaviour
Chapter 3 System setup | R Packages
2 annotations
r-pkgs.org
481
rows to skip
CSV file's name
downloads, transforms and visualizes this data.
script
Download and Plot Factor Returns from the Fama-French Research Data Library | DataScience+
4 annotations
datascienceplus.com
445
Backtesting Asset Allocation portfolios
Backtesting Asset Allocation portfolios | R-bloggers
1 annotation
www.r-bloggers.com
447
14 Top Chicago Trading Firms 2020 | Built In Chicago
www.builtinchicago.org
647
5 working days to complete this assessment.
邮件 - scai@student.ethz.ch
1 annotation
mail.ethz.ch
641
Hints and tips | Barclays Early Careers and Graduates
joinus.barclays
646
Untitled3.ipynb - Colaboratory
colab.research.google.com
623
image - R: save figures in the zoomed window with command? - Stack Overflow
stackoverflow.com
399
take
use r console when console is running - Google Search
1 annotation
www.google.com
350
1. The Very Basics - Hands-On Programming with R [Book]
www.oreilly.com
345
chart.CumReturns function | R Documentation
www.rdocumentation.org
355
eights specified
thought of as "end-of-period" weights.
a time series of weights
Irregular rebalancing
Return.portfolio function | R Documentation
4 annotations
www.rdocumentation.org
323
Machine learning and factor investing
to always start by running all code chunks from Chapter 1.
please make sure that the environment includes all relevant variables.
function name conflicts (especially with the select() function), we use the syntax package::function() to make sure the function call is from the right source
A list of the packages we use
packages
coding requirements
R for Data Science
REPRODUCIBILITY
adaptation of the tensorflow and keras libraries to R.
to benefit from Python tools
statistics-orientated algorithms
more powerful concept than correlation
black boxes
stocks, accounting quantities (e.g., book value) will not be defined
implement more exotic models, like causal graphs (Chapter 14), Bayesian additive trees (Chapter 9), and hybrid autoencoders (Chapter 7).
seek to pivot towards allocation methods
translated into investment decisions.
evaluate sentiment
some approaches
not be covered
predict sales, earning reports, and, ultimately, future returns.
leverage textual data from social media, satellite imagery, or credit card logs
departs from traditional analyses which rely on price and volume data only, like classical portfolio theory à la Markowitz (1952), or high frequency trading.
characteristics of these firm
explained
differences in the returns of firms
asset allocation, quantitative trading and wealth management
Factor investing
their applications in factor investing.
machine learning (ML) tools
encourage reproducibility.
hands-on R code samples
academic references
equity asset allocation
mainstream machine learning algorithms
Machine Learning for Factor Investing
36 annotations
www.mlfactor.com
381
original return is above that of the median return
equal to 1 (true)
create additional labels that are categorical.
classification
predict a scalar real number
categories
original labels (future returns) are numerical and will be used for regression exercises
for any given feature and time point, the distribution is uniform
The predictors have been uniformized
keep a much shorter list of predictors
keep the name of the predictors in memory
a better proxy of financial gain compared to price returns only
incorporate potential dividend payments over the considered periods.
1-month, 3-month, 6-month and 12-month future/forward returns of the stocks. The returns are total returns,
four immediate labels in the dataset
number of assets changes with time
monthly frequency
last four columns are the labels
features
next 93 columns
stock identifier and the date
first two columns
99 columns and 268336 rows
The sample is not perfectly rectangular
93 characteristics describe the firms
November 1998 and ends in March 2019.
1,207 stocks listed in the US
based on a single financial dataset
cardinal function which evaluates the number of elements
return is always computed over one period
from price data
discrete returns
Chapter 1 Notations and data | Machine Learning for Factor Investing
32 annotations
www.mlfactor.com
442
return forward 3 months
return forward 1 month (LABEL)
Mom_11M_Usd | p price momentum 12 - 1 months in USD | Mom_5M_Usd | pr price momentum 6 - 1 months in USD | Mom_Sharp_11M_Usd | pri price momentum 12 - 1 months in USD divided by volatility | Mom_Sharp_5M_Usd | pric price momentum 6 - 1 months in USD divided by volatility |
List of all variables (features and labels) in the dataset
Chapter 17 Data description | Machine Learning for Factor Investing
4 annotations
www.mlfactor.com
442
data providers
Errors in loops or variable indexing
Persistent series
To maximize out-of-sample efficiency
model
probably much more important
inputs
dependent variables and the explanatory features
choices
assumptions
economic or econometric framing is key.
analyst formulates views on the model
no-free lunch theorem
key points that we have learned
heuristic guesses are often hard to beat.
portfolio weighting schemes
each part should not be dealt with independently
intertwined
minimize squared errors
the
the errors
errors
errors
vanilla quadratic programming is the best way to go
modelling families for fff
are at least as important.
the choice and engineering of inputs,
we believe
tempting to believe
the most crucial part is the choice of fff
better out-of-sample.
researchers and practitioners
monthly returns,
dependent variable
market capitalization, accounting ratios, risk measures, momentum proxies
classical predictors reported in the literature
features
gather data and to process it
the first step
shares similarity with panel approaches.
the model is common to all assets
(fff is not indexed by nnn),
computed at time ttt,
expected return for time t+1t+1t+1
forecast returns in the cross-section.
supervised learning
focuses predominantly on the prediction part.
adapt to changing market conditions.
documented asset pricing anomalies
detect some hidden patterns
future returns depend on firm characteristics.
the only as
We agree with this stance, and the only
assumption
model that makes sense economically.
since the 2010 decade
asset pricing
Nonlinear relationships
hedge funds
academics in financial economics
computer scientists and information system experts
this drawback does not hold for high-frequency strategies.
chronological depth is probably the weak point
a few hundred attributes.
several thousand stocks (US listed at least)
a few hundred monthly observations
order of magnitude (in 2019)
now sufficiently large to be plugged into ML algorithms
size of XX\mathbf{X}
high-frequency data and derivative quotes
in factor investing
principles and ideas
Chapter 2 Introduction | Machine Learning for Factor Investing
72 annotations
www.mlfactor.com
468
minimize squared errors
objective
mainstream
vanilla quadratic programming
the
Finally
errors
important
choice and engineering of inputs
dependent variable
better out-of-sample.
monthly returns
market capitalization, accounting ratios, risk measures, momentum proxies
classical predictors
gather data and to process
first step
shares similarity with panel approaches.
the model is common to all assets (fff is not indexed by nnn)
prediction
supervised learning
cross-section
changing market conditions.
asset pricing anomalies
ML
ML
. This is why M
detect some hidden patterns
time-
varying
unknown
features and performance
relationship
firm characteristics
future returns
only assumption
model that makes sense economically
mainstream
Nonlinear relationships
hedge funds
financial economics
computer scientists and information system experts
weak
chronological depth
a few hundred attributes
several thousand stocks (US listed at least)
a few hundred monthly observations
order of magnitude (in 2019)
magnitude
size of X
magnitude
quotes
derivative
data and de
high-frequency
economic
computational
data
factor investing
machine learning
Chapter 2 Introduction | Machine Learning for Factor Investing
59 annotations
www.mlfactor.com
444
factor investing
rationale
understand the drivers of asset prices.
impact of factor indices
quantify and qualify
factor exposure or transaction costs)
pragmatic topics
researchers
key scholarly findings (e.g., asset pricing anomalies)
Practitioners
gained tract
ained tractio
topic of factor investing
the rise of equity traded funds (ETFs)
style allocation
Beyond academic articles
in the first family of journals.
shorter, easier to read
long and often technical.
peer-reviewed financial journals
empirical facts
basic factor models
Asset pricing anomalies
Chapter 3 Factor investing and asset pricing anomalies | Machine Learning for Factor Investing
23 annotations
www.mlfactor.com
498
how they help international students to get a job in CH.
their own experience entering the Swiss job market as non EU / EFTA citizens
The representatives of a-link
detail
3 annotations
ethcareer.ch
459
statistics-orientated algorithms
Machine Learning for Factor Investing
Machine Learning for Factor Investing
2 annotations
www.mlfactor.com
405
backtest in four parts
backtest
of a ML-based strate
a full detailed example
covariance matrix of the assets
constrained minimum variance
equal risk contributions
Equally weighted portfolios are known to be hard to beat
then a simple weighting scheme
signal
assets
rankings
an efficient way to test the relevance of the signal
quantiles of underlying characteristics
most favorable predictions
select the assets
signal
likely to
underperform
signal
exclude
for long-only portfolios
forecasting
signal can be used
two steps in the portfolio construction
integrated in an investment decision
signal
information on the future profitability of assets
provide a signal
Chapter 12 Portfolio backtesting
Chapter 12 Portfolio backtesting | Machine Learning for Factor Investing
30 annotations
www.mlfactor.com
389
which will contain your R code, data files, notes, and other material relevant to your project
A project is simply a working directory designated with a .RProj file.
YaRrr! The Pirate’s Guide to R
2 annotations
bookdown.org
359
OLZ
OLZ AG Dataspace
1 annotation
dataspace.olz.ch
323
ETZ, Gloriastrasse 35, Zürich to ASVZ ETHZ Hönggerberg - Google Maps
www.google.com
393
RepRisk: Übersicht | LinkedIn
www.linkedin.com
284
fusselentferner - Google Shopping
www.google.com
239
Status: offline Sebastian Stoeckl
Hanlin Yang | LinkedIn
1 annotation
www.linkedin.com
329
7.3 Code style
used inside multiple package functions
define small utilities
h
family of related functions
a single .R file will contain multiple function definitions
meaningful and convey which functions
file name
“How should I name this?” each time you create a new file.
formally checking an in-development package
fundamental workflows for test-driving
recognizing the stricter requirements for functions in a package
maintaining a consistent style,
organising your functions into files
Chapter 7 R code | R Packages
14 annotations
r-pkgs.org
384
The description should start with a capital letter and end with a full stop
provide a succinct summary of the type of the parameter
describes the function’s inputs or parameters
Add roxygen comments to your .R files.
use roxygen2 which turns specially formatted comments into .Rd files.
Chapter 10 Object documentation | R Packages
5 annotations
r-pkgs.org
441
prepare your system
small toy package.
mastered the basics
Writing R extensions
this book does.
the most common and useful components
learn more about those details.
from the low-level details
automate common development task
through the devtools package,
what you
want your package to do
automated
For example
Organising code in a package
develop packages
tests
documentation
package
data
code
Chapter 1 Introduction | R Packages
21 annotations
r-pkgs.org
564
as an exported function.
call functions in other packages
Initiate a new .R file
add a second function to foofactors
declare your general intent
add another function to foofactors
use a function from another package
Run this test interactively,
formalize and expand this with some unit tests.
use RStudio,
in its source file
write a specially formatted comment right above
have a special R documentation file
help
metadata about your package
incremental development
checking this often.
moving parts of the foofactors package
file
commit the new
much faster iteration
simulates the process of building, installing, and attaching the foofactors package
does not exist in the global workspace.
test drive
devtools offers a more robust approach
clare their dependenc
NOT contain any of the other top-level code
only the definition
creates and/or opens a script
group related functions together
make a new .R file for each function
name the file after the function.
Where shall we define
initialize
foofactors package (and Project)
It should not be nested inside another RStudio Project, R package, or Git repo.
existing or new projec
Chapter 2 The whole game | R Packages
37 annotations
r-pkgs.org
464
each source package is also an RStudio Project.
that a source package lives in a directory on your computer
A project might be
5.2 RStudio Proj
evaluate a potential package name from many angles:
Chapter 5 Fundamental development workflows | R Packages
5 annotations
r-pkgs.org
463
in the special roxygen comment above each function
putting
t
export
By default, functions in a package are only for internal use.
make a package
ut what the helper functions should be and how they should work.
Here is the next version of the data cleaning script,
into one or more separate files
move reusable data and logic out of the analysis script
using functions from tidyverse packages
use the tidyverse
use of some add-on packages,
in proper objects and functions, respectively.
isolating as much data and logic
refactor this code
logic into dozens or hundreds of data ingest scripts.
inlining the same dat
package,
make these standard data maneuvers available to themselves as functions
an be more efficient and consisten
ar pre-processing of many similar data files over time
a fictional data analysis script
a data analysis script
package’s R code and how it differs from R code in a script.
aid practically nothing about the R code inside the package
Chapter 6 The package within | R Packages
26 annotations
r-pkgs.org
504
enhance pricing and risk calculations
IT, other Strat functions and business lines
.
productionize models, manage and maintain code libraries and rapidly develop innovative trader tools.
- Quant Developers Strategists
design, implement, back-test, deploy and measure intelligent automated trading components and systematic trading strategies.
financial and software engineers
Electronic Trading Strategists
desks across business lines
Responsibilities
Strategists
arket knowledge and product and technical trainin
curriculum
Firm's data resources and analytical tools
one-on-one sessions
on-the-job training
make strategic decisions, develop quantitative edge
particular business lines or desks
strategists (‘strats’) team
working with desk strategists
Morgan Stanley Quantitative Finance Off-Cycle Internship Program
Off-cycle Internship institutional-securities quantitative-finance-program emea | Morgan Stanley
21 annotations
www.morganstanley.com
439
Relevant
Shijing's Resume BoA - Online LaTeX Editor Overleaf
1 annotation
www.overleaf.com
440
Regime Shift Models – A Fascinating Use Case of Time Series Modeling
segregating the time series into different “states
state-space models
triggered by fundamental changes in macroeconomic
variables, policies or regulations.
Regime shift models
state-space models
two or more states
characterized by their own probability distributions
crossing a threshold triggers a regime shift
prices moving below the 200-day moving average
a ‘bearish regime’
take macroeconomic variables
input and predict the next period risk.
the regime to be a ‘hidden state’
Markov
a homogenous first-order Markov chain.
three-state variance Markov switching model
Regime Shift Models | Regime Shift Models in Financial Market
18 annotations
www.analyticsvidhya.com
609
Unsupervised Learning to Market Behavior Forecasting
first type of potential features
derivatives of price data.
second type
volume derivatives.
only price data (or asset returns)
Hidden Markov Model.
these fe
atures i
feature engineering and modeling
the hidden states.
hidden states are our market behavior.
interpret the hidden state after the modeling
Observed
data
market features
how each state describes the current state
state #0
downside market condition
current
will go on
current state #1
uncertainty for the tendency
current state #2
upside market condition
will go on
feature engineering
Unsupervised Learning to Market Behavior Forecasting | by Sergey Malchevskiy | Towards Data Science
27 annotations
towardsdatascience.com
656
K Means Clustering and Creating a Simple Trading Rule for Smoother Returns
K Means Clustering and Creating a Simple Trading Rule for Smoother Returns | Quant News
1 annotation
www.quantnews.com
622
t Phases
Machine Learning-Based Classification of Marke
constant market conditions.
f market regimes.
a grouping according to risk aspects
identify market phases.
Market regimes
flexible criteria
“stressed VaR” periods
“natural” clusters
d with the clustering algorithms kmeans and DBSCAN.
ike kmeans or DBSCAN
a large “normal cluster”
fell into a smaller “crisis” cluster.
bubble detector”.
he classification algorithms
f 3,000 points of time,
1,000 points.
Machine Learning-Based Classification of Market Phases – RiskDataScience
18 annotations
riskdatascience.net
677
case of HMM
daily standard deviation, daily volatility
daily returns as the observable variable
Fahim's complete project, data files and HMM files
Market Regime Detection with Hidden Markov Model
Market Regime Detection with Hidden Markov Model
5 annotations
blog.quantinsti.com
659
optimal number of trees in a random forest
machine learning - Does the optimal number of trees in a random forest depend on the number of predictors? - Cross Validated
1 annotation
stats.stackexchange.com
617
predicting the
Apple stock price
Neural Networks in Finance
Neural Networks to Predict the Market
Neural Networks to Predict the Market | by Vivek Palaniappan | Towards Data Science
4 annotations
towardsdatascience.com
659
PCA: A Practical Guide to Principal Component Analysis in R & Python
dimension reduction
Extract the important factors
obtaining important variables
from a large set of variables
extracts low dimensional set of features
with 3 or higher dimensional data.
PCA: Practical Guide to Principal Component Analysis in R & Python
7 annotations
www.analyticsvidhya.com
664
calculate the rolling/moving variance of a time series in python?
numpy - How can I simply calculate the rolling/moving variance of a time series in python? - Stack Overflow
1 annotation
stackoverflow.com
667
to obtain optimal number you can try training random forest at a grid of ntree parameter (simple, but more CPU-consuming)
number of trees between 64 - 128 trees
training random forest at a grid of ntree parameter
number of trees is too smal
number of predictors
) to grow a tree. If the number of observations
number of trees
space
ndom su
bagging
machine learning - Does the optimal number of trees in a random forest depend on the number of predictors? - Cross Validated
10 annotations
stats.stackexchange.com
647
function
observation transitio
for the observations
for the states
transition function
full set of hidden states and observations
joint probability
they do change they are expected to persist for some time.
regimes themselves are not expected to change too quickl
precisely the behaviour that is desired from such a model when trying to apply it to market regimes
remain in that state for some time
suddenly jump to a new state a
stay in a particular stat
observation transition probabilities
specified state
"noisy"
nference on "hidden" generative processes
Hidden Markov Model
Regime Detection
Hidden Markov Models
determining what the current "market regime state"
the asset returns available to date
filtering
ns up to time t
Hidden Markov Models - An Introduction | QuantStart
24 annotations
www.quantstart.com
682
aggregating all the volatility and daily return values
overfitting
granularity
more subtleties that come with setting up a Hidden Markov Model
expectation maximization algorithm iteration threshold to 7
observation variables
train a Hidden Markov Model t
instead of artificially defining arbitrary volatility
volatility clustering,
Algorithmic Factor Investing with Market Regime Classification | by Matthew Wang | Medium
9 annotations
medium.com
591
Which are the methods to validate an unsupervised machine learning algorithm?
Which are the methods to validate an unsupervised machine learning algorithm?
1 annotation
www.researchgate.net
641
case of multiple sequences
inferred optimal hidden states
score
various initializations
select the highest scored model.
observations
train
iterative Expectation-Maximization (EM)
Forward-Backward
Viterbi
dynamic programming
model parameters.
observed
e model likelihood.
observed
parameters
optimal sequence of hidden states.
observed data
model parameter
Tutorial — hmmlearn 0.2.4 documentation
19 annotations
hmmlearn.readthedocs.io
613
run fit with various initializations
lengths of the sequences
matrix of concatenated sequences of observations
Tutorial — hmmlearn 0.2.4 documentation
3 annotations
hmmlearn.readthedocs.io
620
我的仓库
www.superbuy.com
600
Open-High-Low-Close
more sophisticated clustering algorithms,
robust
due to errors or bad ticks
"outlier" data
due to flash crashes
outlier data points
noisy datase
"clusters" are not truly separate distributions of data
data is highly noisy
forced to generate k clusters
hard to extract the predictive signal
signal-to-noise ratio of financial pricing data is low
can then be used to ascertain if certain market regimes exist
clusters
open, high, low, close
K-Means Clustering.
partition observational data into separate subgroups or clusters
regimes in asset price series
unsupervised clustering
K-Means Clustering of Daily OHLC Bar Data | QuantStart
20 annotations
www.quantstart.com
609
different expected return-risk patterns.
different regim
handles simultaneously the heterogeneity across stock markets and over time
dynamic patterns
similar
identify clusters of time series
model-based clustering of regime switching models
Clustering financial time series: New insights from an extended hidden Markov model - ScienceDirect
7 annotations
www.sciencedirect.com
655
revisions of strategic asset allocation decisions
Python and Machine Learning for Asset Management - Machine learning techniques for regime analysis | Coursera
1 annotation
www.coursera.org
617
log probability under the model.
API Reference — hmmlearn 0.2.4 documentation
1 annotation
hmmlearn.readthedocs.io
669
Snippets: Importing libraries - Colaboratory
colab.research.google.com
657
Market regime detection using machine
using
Market Regime detection using PCA and KMeans
2 annotations
www.quantopian.com
649
three states
Quantica_Interview.ipynb - Colaboratory
1 annotation
colab.research.google.com
655
The mission of WorldQuant University is to
WorldQuant University | Tuition-Free Financial Engineering MSc
1 annotation
wqu.org
665
the score of the algorithm output
The model is fitted
e number of iteration
A full covariance matrix
Two states
number of states
object requires spe
The GaussianHMM
hidden state/regime
masked by tha
adjusted closing price
reates a subplo
e prices DataFrame
l training period
adjusted closing prices
calculates the percentage returns
serialise the model for use in the regime detection risk manager
highly volatile periods
regime detection largely captures "trending" periods
Training the Hidden Markov Model
Regime Detection
t really achieve much
simple moving averages (SMA)
S&P500
discovering latent "market regimes".
Market Regime Detection using Hidden Markov Models in QSTrader | QuantStart
25 annotations
www.quantstart.com
620
Share
Share