ZeroMQ to MetaTrader Connectivity
We present a solution employing ZeroMQ (an Open Source, Asynchronous Messaging Library and Concurrency Framework) for building an extensible, high performance bridge between external (non-MQL) programming languages and MetaTrader.
Reasons for development
- Lack of comprehensive, publicly available literature about this topic on the web.
- Traders considering alternatives to programming robots in MetaTrader have traditionally relied on Winsock/WinAPI based solutions. These often require revision with both Microsoft™ and MetaQuotes™ updates, and are hence cumbersome to maintain.
- Alternatives to ZeroMQ include named pipes, and approaches where filesystem-dependent functionality forms the bridge between MetaTrader and external languages.
Architecture
In this section we lay the foundation for a trading system that will:
- Use MetaTrader for acquiring market data, and perform trade execution and management
- Support multiple, independent, non-MQL trading strategies interfacing with MetaTrader in parallel.
Infographic: ZeroMQ-Enabled Trading Infrastructure (with MetaTrader)
Why ZeroMQ?
- Enables programmers to connect any code to any other code, in a number of ways.
- Eliminates a MetaTrader user’s dependency on just MetaTrader-supported technology (features, indicators, language constructs, libraries, etc.)
- Traders can develop indicators and strategies in C/C#/C++, Python, R and Java (to name a few), and deploy to market via MetaTrader.
- Leverage machine learning toolkits in Python and R for complex data analysis and strategy development, while interfacing with MetaTrader for trade execution and management.
- ZeroMQ can be used as a high-performance transport layer in sophisticated, trading systems otherwise difficult to implement in MQL.
- Different strategy components can be built in different languages if required, and seamlessly talk to each other over TCP, in-process, inter-process or multicast protocols.
- Multiple communication patterns and disconnected operation.
ZeroMQ: Supported Programming Languages
Though we focus on MQL interfaced with Python & R in this post, the basic process described here can be implemented easily in other ZeroMQ-supported languages.
A comprehensive list of ZeroMQ language bindings is available here.
Planning Flow Control
Please note that this section is not intended to be a detailed tutorial on ZeroMQ.
However, it is still important to understand a few things about ZeroMQ that make it particularly suited to the task of connecting external programming languages such as Python and R to MetaTrader.
It supports TCP, inter-process, in-process, PGM and EPGM enabled multicast networking.
We use the TCP transport type for the implementation in this manual.
ZeroMQ enables servers and clients to connect “to each other” on demand, particularly useful for designing trading infrastructure.
In addition to support for asynchronous communication and disconnected operation, ZeroMQ supports several communication patterns that permit higher-level data transfer, freeing programmers to focus more on the transfer logic rather than low-level mechanisms.
These patterns include:
- Request (REQ) / Reply (REP)
- Publish (PUB) / Subscribe (SUB) and,
- Push (PUSH) / Pull (PULL).
For the implementation in this blog post, we will employ ZeroMQ’s PUB/SUB and PUSH/PULL communication patterns.
MetaTrader will be our “Server”, and trading strategies will be “Clients”.
Please note that this (MT=Server, Strategy=Client) is not a MUST – you will need to decide on whatever flow control suits your particular needs best.
The accompanying source code presents a working solution for this particular design pattern, but users are of course free to modify it to suit their own requirements.
There are a number of ways you could achieve the end goal; carefully planning flow control will lead to efficient functionality.
Publish (PUB) / Subscribe (SUB) Pattern
The Server (MetaTrader EA) will employ a TCP socket of type PUB, to publish tick data for a predetermined list of symbols, as soon as it arrives.
The Client (Trading Strategy, e.g. in Python) will employ a TCP socket of type SUB, to subscribe to any number of symbols being published by the MetaTrader Server.
Push (PUSH) / Pull (PULL) Pattern
The Client (Trading Strategy) will employ a PUSH socket to send trading commands to the Server and a PULL socket to receive responses.
The Server (MetaTrader EA) will also employ a PUSH socket to send command responses to Clients (Trading Strategies).
The PUSH/PULL pattern enables servers and clients to exchange data with each other on-demand, but in one direction without expecting a response. This could of course be swapped out for another REQ/REP pattern, depending on your application’s flow control requirements.
In summary, for the implementation presented in this manual:
- The Server will employ three sockets, a PUSH, PULL and PUB.
- Each Client will employ three sockets, a PUSH, PULL and SUB.
MetaTrader Expert Advisor – Components
As displayed in the infographic above, the MetaTrader EA will serve as our ZeroMQ-enabled Server, with three main modules:
Message router
This allows the EA to receive commands and send acknowledgements back to connecting Clients (trading strategies) through the PUSH socket.
The Router passes all messages on to the Parser.
Message parser
Messages received by this module are decomposed into actions for the next module (Interpreter & Executor).
Interpreter & executor
This module literally “interprets” decomposed messages and performs requested actions accordingly.
If the Client is requesting a BUY or SELL trade be opened on e.g. the EUR/USD, it sends the trade to market and a notification of success/failure/ticket-info to the Client via the PULL socket.
Implementation Requirements
Click here to visit the project’s GitHub repository
Important disclaimer:
The MQL EA has live-trading functionality in an ACTIVE state, meaning it is equipped to send and manipulate orders to/from the market.
Therefore, please ensure you check everything thoroughly prior to using it on a real money account. All source code is provided AS-IS and users are responsible for any and all outcomes as a result of using it.