Skip to content

Latest commit

 

History

History
 
 

05_strategy_evaluation

README.md

Chapter 05: Strategy Evaluation & Portfolio Management

This chapter covers:

  • How to build and test a portfolio based on alpha factors using zipline
  • How to measure portfolio risk and return
  • How to avoid the pitfalls of backtesting
  • How to evaluate portfolio performance using pyfolio
  • How to manage portfolio weights using mean-variance optimization and alternatives
  • How to use machine learning to optimize asset allocation in a portfolio context

How to build and test a portfolio with zipline

In Chapter 4, we introduced zipline to simulate the computation of alpha factors from trailing cross-sectional market, fundamental, and alternative data. Now we will exploit the alpha factors to derive and act on buy and sell signals.

We will postpone optimizing the portfolio weights until later in this chapter, and for now, just assign positions of equal value to each holding.

The code for this section is in the subdirectory trading_zipline subdirectory; the notebook alpha_factor_zipline_with_trades simulates the trading decisions that build a portfolio based on the simple MeanReversion alpha factor from the last chapter using zipline.

How to measure performance with pyfolio

ML is about optimizing objective functions. In algorithmic trading, the objectives are the return and the risk of the overall investment portfolio, typically relative to a benchmark (which may be cash, the risk-free interest rate, or an asset price index like the S&P 500).

The Sharpe Ratio

The ex-ante Sharpe Ratio (SR) compares the portfolio's expected excess portfolio to the volatility of this excess return, measured by its standard deviation. It measures the compensation as the average excess return per unit of risk taken. It can be estimated from data.

Financial returns often violate the iid assumptions. Andrew Lo has derived the necessary adjustments to the distribution and the time aggregation for returns that are stationary but autocorrelated. This is important because the time-series properties of investment strategies (for example, mean reversion, momentum, and other forms of serial correlation) can have a non-trivial impact on the SR estimator itself, especially when annualizing the SR from higher-frequency data.

The Fundamental Law of Active Management

A high Information Ratio (IR) implies attractive out-performance relative to the additional risk taken. The Fundamental Law of Active Management breaks the IR down into the information coefficient (IC) as a measure of forecasting skill, and the ability to apply this skill through independent bets. It summarizes the importance to play both often (high breadth) and to play well (high IC).

The IC measures the correlation between an alpha factor and the forward returns resulting from its signals and captures the accuracy of a manager's forecasting skills. The breadth of the strategy is measured by the independent number of bets an investor makes in a given time period, and the product of both values is proportional to the IR, also known as appraisal risk (Treynor and Black).

The fundamental law is important because it highlights the key drivers of outperformance: both accurate predictions and the ability to make independent forecasts and act on these forecasts matter. In practice, estimating the breadth of a strategy is difficult given the cross-sectional and time-series correlation among forecasts.

In- and out-of-sample performance with pyfolio

Pyfolio facilitates the analysis of portfolio performance and risk in-sample and out-of-sample using many standard metrics. It produces tear sheets covering the analysis of returns, positions, and transactions, as well as event risk during periods of market stress using several built-in scenarios, and also includes Bayesian out-of-sample performance analysis.

Code Examples

The directory risk_metrics_pyfolio contains the notebook pyfolio_demo that illustrates how to extract the pyfolio input from the backtest conducted in the previous folder. It then proceeds to calcuate several performance metrics and tear sheets using pyfolio

How to avoid the pitfalls of backtesting

Data Challenges

Backtesting simulates an algorithmic strategy using historical data with the goal of identifying patterns that generalize to new market conditions. In addition to the generic challenges of predicting an uncertain future in changing markets, numerous factors make mistaking positive in-sample performance for the discovery of true patterns very likely.

These factors include aspects of the data, the implementation of the strategy simulation, and flaws with the statistical tests and their interpretation. The risks of false discoveries multiply with the use of more computing power, bigger datasets, and more complex algorithms that facilitate the identification of apparent patterns in the noise.

Data-snooping and backtest overfitting

The most prominent challenge to backtest validity, including to published results, relates to the discovery of spurious patterns due to multiple testing during the strategy-selection process. Selecting a strategy after testing different candidates on the same data will likely bias the choice because a positive outcome is more likely to be due to the stochastic nature of the performance measure itself. In other words, the strategy is overly tailored, or overfit, to the data at hand and produces deceptively positive results.

Marcos Lopez de Prado has published extensively on the risks of backtesting, and how to detect or avoid it. This includes an online simulator of backtest-overfitting.

The deflated Sharpe Ratio

De Lopez Prado and Bailey (2014) derive a deflated SR to compute the probability that the SR is statistically significant while controlling for the inflationary effect of multiple testing, non-normal returns, and shorter sample lengths.

The pyton script deflated_sharpe_ratio in the directory multiple_testing contains the Python implementation with references for the derivation of the related formulas.

References

How to Manage Portfolio Risk & Return

Mean-variance optimization

MPT solves for the optimal portfolio weights to minimize volatility for a given expected return, or maximize returns for a given level of volatility. The key requisite input are expected asset returns, standard deviations, and the covariance matrix.

Code Examples

We can calculate an efficient frontier using scipy.optimize.minimize and the historical estimates for asset returns, standard deviations, and the covariance matrix.

The directory efficient_frontier contains the notebook mean_variance_optimization to compute the efficient frontier in python.

Alternatives to mean-variance optimization

The Black-Litterman approach

The Kelly Rule

Code Example

The directory kelly Kelly Rule contains the notebooks kelly_rule to compute the Kelly rule portfolio.

Hierarchical Risk Parity