This is the code for this video on Youtube by Siraj Raval. The author of this code is edwardhdlu . It's implementation of Q-learning applied to (short-term) stock trading. The model uses n-day windows of closing prices to determine if the best action to take at a given time is to buy, sell or sit.
As a result of the short-term state representation, the model is not very good at making decisions over long-term trends, but is quite good at predicting peaks and troughs.
Some examples of results on test sets:
!^GSPC 2015 S&P 500, 2015. Profit of $431.04.
Alibaba Group Holding Ltd, 2015. Loss of $351.59.
Apple, Inc, 2016. Profit of $162.73.
Google, Inc, August 2017. Profit of $19.37.
To train the model, download a training and test csv files from Yahoo! Finance into data/
mkdir models
python train.py ^GSPC 10 1000
Then when training finishes (minimum 200 episodes for results):
python evaluate.py ^GSPC_2011 model_ep1000
- Had to modify sigmoid slightly to prevent overflow occuring in Math.exp
- Added a boolean for first iteration which forces a buy so there is something in agent.inventory (May look into finding the best statistical time to buy in a future imp based on the current price of entry...for now this has gotten things working when evaluating)
Deep Q-Learning with Keras and Gym - Q-learning overview and Agent skeleton code