Trading with Python: Interactive Brokers (GCP) — How to setup environment
Algorithmic trading relies on a computer program, the trading strategy. The application contains an algorithm and steps to automate some or all elements of a trading strategy. Extensive trading, business, and mathematical knowledge are necessary to build the algorithms. It is a sequence of steps or rules to achieve a goal and can take many forms. Machine learning (ML) algorithms pursue the objective of supervised and unsupervised (deep) learning other algorithms, namely rules, to achieve a target based on data, such as minimising a prediction error.
At the most basic level, the algorithm can be a set of conditions periodically checked and with another set of actions (open or closed position) when a state is valid. The algorithm needs two inputs. In the case of Foreign Exchange (FOREX/FX) trading, prices and signals data streams. The signal can be received from external paid API or generated by an algorithm from the ingested stream of data (prices). The complexity of the algorithmic part can be enormous, considering many factors and combining statistical methods with machine learning, deep learning (recurrent neural networks), and common-sense rules.
Charles Dow published (around 1900) the assumption[1] that financial markets quickly discount for all news, and thereby prices reflect the available information correctly. His further hypothesis that prices do move trends with specific characteristics lead to the development of technical analysis[2]. Technical analysis solely relies on market price history and examines price series for trends, patterns, anomalies, etc. to support or suggest investment decisions. As this is a simple quantitative task, computers have been increasingly used in the field of algorithmic analysis and decision making over the last thirty years. Today, automated trading accounts for 70 percent[3] of the U.S. stock market activity, and when the markets are extremely volatile, it can go up to 90% of the trades.
Nowadays, learn how to programmatically access price data is an essential skill for all trading related roles.
1)Account
InteractiveBrokers: https://www.interactivebrokers.co.uk/en/home.php
Setup works with both Live/Paper account. Paper account is very limited, however Python FX example works without any issue.
2)Gateway
New GCP Compute Engine (VM) instance:
Size of the virtual machine depends on workload produced by algorithms. I would recommend to start with n1-standard-2 (2 vCPUs, 7.5 GB memory) with 20GB HDD with Ubuntu 18.04 minimal image. There is always option to scale out and add more cores and memory.
(note: Standalone gateway version is not reliable for production solution.)
Initial requirements:
$ sudo bash (you can install it with another user and use sudo)
$ apt update
$ apt upgrade -y
$ apt install vim wget curl net-tools unzip xvfb x11vnc
$ apt install openjdk-8-jdk
(note: x11vnc is a popular Linux remote screen utility with xvfb is an x11 screen simulator)
VNC and x11 is requirement. The gateway won’t start itself and the UI provides more easier (java old-school) access to configuration settings without restart. VNC client (realvnc) is available here https://www.realvnc.com/en/connect/download/viewer/macos/.
Edit your ~/.bashrc
export PATH=$PATH:/usr/lib/jvm/java-8-openjdk-amd64/bin
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
Load new variables:
$ source ~/.bashrc
Run following command to create virtual screen and run VNC server:
$ Xvfb :1 -ac -screen 0 1024x768x24 &
$ export DISPLAY=:1
$ x11vnc -ncache 10 -ncache_cr -display :1 -forever -shared -logappend /var/log/x11vnc.log -bg -noipv6
(VNC starts on port 5900, to verify run following command)
$ netstat -tulpn
In case you want to secure your instance:
$ sudo apt-get install ufw
$ sudo ufw enable
$ sudo ufw allow 5900
$ sudo ufw allow 4000:4003/tcp
After the environment preparation, lets install the Gateway (as root):
$ cd ~
$ wget https://download2.interactivebrokers.com/installers/ibgateway/latest-standalone/ibgateway-latest-standalone-linux-x64.sh
$ chmod x ibgateway-latest-standalone-linux-x64.sh
$ sh ibgateway-latest-standalone-linux-x64.sh -c
(note: prompt to start the gateway — better no, to start later on port 4001, if you want to start it, it will be restarted later and probably run on port 4002. Check with $ net stat -tulpn)
Now install IBController: ! make sure you have the latest stable version !
(https://github.com/ib-controller/ib-controller/releases)
$ cd ~
$ wget https://github.com/ib-controller/ib-controller/releases/download/<version>/IBController-<version>.zip
$ unzip ./IBController-<version>.zip -d ./ibcontroller.<paper/live>
chmod x ./ibcontroller.<paper/live>/*.sh ./ibcontroller.<paper/live>/*/*.sh
The final step is to adjust configuration files:
/root/Jts/jts.ini (this file is also edited by Gateway, it will change)
! important is the tradingMode in Logon= l for Live or p for Paper !
[IBGateway]
WriteDebug=false
TrustedIPs=
RemoteHostOrderRouting=ndc1.ibllc.com
MainWindow.Height=550
RemotePortOrderRouting=4000
LocalServerPort=4000
ApiOnly=true
MainWindow.Width=700
[Logon]
useRemoteSettings=false
UserNameToDirectory=
TimeZone=Europe/London
Individual=1
tradingMode=l
colorPalletName=dark
Steps=15
Locale=en
UseSSL=false
SupportsSSL=ndc1.ibllc.com:4000,true,20200623,false;cdc1.ibllc.com:4000,true,20200624,false
s3store=true
ibkrBranding=pro
[Communication]
Internal=false
LocalPort=0
Peer=cdc1.ibllc.com:4000
Region=us
Next file is:
/root/ibcontroller.<live/paper>/IBControllerGatewayStart.sh
TWS_MAJOR_VRSN=<leave original version>
IBC_INI=/root/ibcontroller.<live/paper>/IBController.ini
IBC_PATH=/root/ibcontroller.<live/paper>
TWS_PATH=/root/Jts
LOG_PATH=/root/ibcontroller.<live/paper>/Logs
Last file to change is:
/root/ibcontroller.paper/IBController.ini
IbLoginId=<your IB username>
IbPassword=<your IB password>
Now start the IB Gateway:
$ DISPLAY=:1 ~/ibcontroller.paper/IBControllerGatewayStart.sh
All is up and running. Try the VNC with yout VM external IP address:
Screen will look similar to this screenshot:
! only Interactive Brokers API Server will be in green !
You may need to check/adjust few parameters:
Socket port or Master API client ID are the most important parameters for the connection.
Do not miss this change:
Untick “Allow connections from localhost only” and click Create to add your remote IP address (laptop and workstation with Python, where you wont to test the connection)
Now all is ready to test connection from Python.
3) Python application
Requirements: pip install ibapi
(note: tested with 3.7/3.8)
The official Interactive Brokers API is only offered through their Github site and not the Python Package Index (PyPI) because its distributed under a different license. You can however build a wheel from the provided source code and then install the wheel. These are the steps:
Download “API Latest” from http://interactivebrokers.github.io/
Unzip or install (if its a .msi file) the download.
Go totws-api/source/pythonclient/
Build a wheel with:python3 setup.py bdist_wheel
Install the wheel with:python3 -m pip install --user --upgrade dist/ibapi-9.73.7-py3-none-any.whl
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
import threading
import time
class IBapi(EWrapper, EClient):
def __init__(self):
EClient.__init__(self, self)
def historicalData(self, reqId, bar):
print(f'Time: {bar.date} Close: {bar.close}')
def run_loop():
app.run()
app = IBapi()
app.connect('<VM external IP address>', 4001, 123) "123 is clientID"
# Start the socket in a thread
api_thread = threading.Thread(target=run_loop, daemon=True)
api_thread.start()
time.sleep(1) # Sleep interval to allow time for connection to server
# Create contract object
eurusd_contract = Contract()
eurusd_contract.symbol = 'GBP'
eurusd_contract.secType = 'CASH' #CMDTY
eurusd_contract.exchange = 'IDEALPRO' #SMART
eurusd_contract.currency = 'USD'
# Request historical candles
app.reqHistoricalData(1, eurusd_contract, '', '2 D', '1 hour', 'BID', 0, 2, False, [])
time.sleep(5) # sleep to allow enough time for data to be returned
app.disconnect()
Output of 2 days GBP/USD data (1 hour aggregation):
Time: 1592860500 Close: 1.24704
Time: 1592863200 Close: 1.24886
Time: 1592866800 Close: 1.25002
…
Detail description of API is available here: http://interactivebrokers.github.io/tws-api/
Reference:
[1]DOW, Charles, SELDEN, George. Scientific Stock Speculation. 1. US: Wentworth Press, 2016. ISBN 978–1374152151.
[2]Hamilton 1922, Rhea 1932, Schaefer 1960, Russel 1961.
[3]CNN. CNN [online]. 6th Feb 2019. Internet: https://money.cnn.com/2018/02/06/investing/wall-street-computers-program-trading/index.html