Rust Crypto Ticker using Interactive Brokers' TWS API directly

Rust

Hi all, I’d like to share some adventures I’ve recently had with interacting with Interactive Brokers’ TWS API directly using Rust, without an SDK.

You’ll need TWS (or IB Gateway) running on your machine. Note that their API host is TWS - you don’t talk directly to their servers. You’ll also need a real account, but not a paid data subscription.

The code can be found here in completed form: main.rs gist

And without further ado…

Go here and download the latest TWS: https://www.interactivebrokers.com.au/en/trading/tws.php
You must login with a real live or paper account. If you use the ‘try the demo’ mode the API does not respond.
You will not require a paid data subscription for the symbols we’re watching here.
Go File > Global Configuration > Api > Settings > check ‘Enable ActiveX and Socket Clients’
!!!! ENSURE “Read-Only API” IS CHECKED !!!!
Also take note of ‘Socket port’ on that screen (7496 is what I see).
Select Apply and Ok
Run nc -z localhost 7496 to test that TWS API is listening.

Ok lets start talking to this with Rust.

First lets make a new rust project:

Install rust if you need to!
cargo new my_tws_api_talker
cd my_tws_api_talker
cargo run

You should see a Hello message.

Lets make it connect. Open up src/main.rs:

use std::net::TcpStream;

fn main() {
    let mut stream = TcpStream::connect("127.0.0.1:7497").expect("connect");
    stream.set_nodelay(true).expect("nodelay"); // Because we're wannabe HFT traders.
    println!("Able to connect")
}

Perform cargo run, it should say it was able to connect. Now lets send the greeting. Most messages to TWS need a big-endian 4-byte length prefix. So lets implement that as a prerequisite:

fn add_length_prefix(bytes: &[u8]) -> Vec<u8> {
    let len = bytes.len() as u32;
    let mut data: Vec<u8> = Vec::new();
    data.push(((len >> 24) & 0xff) as u8);
    data.push(((len >> 16) & 0xff) as u8);
    data.push(((len >> 8) & 0xff) as u8);
    data.push((len & 0xff) as u8);
    data.extend(bytes);
    data
}

And one small helper. Not sure why this isn’t part of the stdlib, unless I’m missing something:

fn concat(a: &[u8], b: &[u8]) -> Vec<u8> {
    let mut both = a.to_owned();
    both.extend(b);
    both
}

So now we can use these helpers to send the greeting:

use std::io::{Read, Write}; // Add this up top.

const MIN_SERVER_VER_BOND_ISSUERID: u32 = 176; // From server_versions.py.
const DESIRED_VERSION: u32 = MIN_SERVER_VER_BOND_ISSUERID;

fn send_greeting(stream: &mut TcpStream) {
    let prefix = "API\0";
    let version = format!("v{}..{}", DESIRED_VERSION, DESIRED_VERSION);
    let version_msg = add_length_prefix(version.as_bytes());
    let both = concat(prefix.as_bytes(), &version_msg);
    stream.write_all(&both).expect("Greeting");
}

Now to actually call that greeting, add this to the bottom of the main function:

send_greeting(&mut stream);
println!("Greeting sent");

Then cargo run and it should say greeting sent. Great!

Next we need to read a message from TWS. These messages all look like this:

4 bytes big endian message length
Message

A message is a list of fields. Each field is a string, with a zero after it. Numbers are strings, and bools are “0” (false) or “1” (true). Eg a simple 3-field message would be “Field A\0Field B\0Field C\0”.

First we need to read a length:

fn read_length(reader: &mut TcpStream) -> u32 {
    let mut len_buf: [u8; 4] = [0; 4];
    reader.read_exact(&mut len_buf).expect("Read length");
    let length: u32 = ((len_buf[0] as u32) << 24)
        | ((len_buf[1] as u32) << 16)
        | ((len_buf[2] as u32) << 8)
        | len_buf[3] as u32;
    length
}

Next we need something to split a read message into its fields:

fn split_message(message: &[u8]) -> Vec<String> {
    // Split into an array of buffers:
    let mut components = Vec::<Vec<u8>>::new();
    let mut current_component = Vec::<u8>::new();
    for byte in message {
        if *byte == 0 {
            if !current_component.is_empty() {
                components.push(current_component.clone());
                current_component.clear();
            }
        } else {
            current_component.push(*byte);
        }
    }
    if !current_component.is_empty() {
        components.push(current_component);
    }

    // Convert the buffers into strings:
    components
        .into_iter()
        .map(|v| String::from_utf8_lossy(&v).to_string())
        .collect()
}

Next lets bring all that together. Read the length, then read the message, then split it:

fn read_message(reader: &mut TcpStream) -> Vec<String> {
    let length = read_length(reader);
    let mut buffer: Vec<u8> = vec![0; length as usize];
    reader.read_exact(&mut buffer).expect("Read message");
    split_message(&buffer)
}

To use this, add the following to the bottom of main.rs:

let greeting_response = read_message(&mut stream);
println!("Greeting response: {:?}", greeting_response);

cargo run and you should see TWS respond with something like ["176", "20230826 22:07:41 AEST"] These fields are the api version then datetime. The version should match DESIRED_VERSION from our code. You should verify that it does, in production code.

Right after the greeting, we have to send the ‘Start API’ message. If you download IB’s python api and look in message.py, you can see all the request type codes. Lets write an enum with only a few request types we’ll use:

enum OutgoingRequestType {
    ReqMktData = 1,
    StartApi   = 71,
}

Now we’ll need a function to compose a message:

// Join and delimit the fields, prefixing the length.
fn make_message(fields: &[String]) -> Vec<u8> {
    let mut delimited_fields = Vec::<u8>::new();
    for field in fields {
        delimited_fields.extend(field.as_bytes());
        delimited_fields.push(0); // Even goes after the last field.
    }
    add_length_prefix(&delimited_fields)
}

Then we need our function to compose and send the ‘start api’ message:

fn send_start_api(stream: &mut TcpStream, client_id: u32) {
    let fields: Vec<String> = vec![
        (OutgoingRequestType::StartApi as u32).to_string(),
        "2".to_string(), // Version of the Start API message.
        client_id.to_string(),
        "".to_string(), // Optional capabilities.
    ];
    let message = make_message(&fields);
    stream.write_all(&message).expect("Start api");
}

And let’s call it. Add this to the end of main():

send_start_api(&mut stream, 123456); // Client ID 123456.

If you cargo run now it should work.

At this stage, the API will want to send us a bunch of messages. So we’d better loop for those. Firstly lets identify some of the incoming message types:

enum IncomingRequestType {
    TickPrice = 1,
    ErrMsg    = 4,
} // From message.py.

Using the above message types, lets add a message handler:

fn message_received(fields: &[String]) {
    if fields.is_empty() { return }
    let Ok(t) = fields[0].parse::<u32>() else { return };
    if t == IncomingRequestType::ErrMsg as u32 {
        let _request_id = fields.get(2);
        let _code = fields.get(3);
        let Some(text) = fields.get(4) else { return };
        println!("Message: {}", text);
    } else if t == IncomingRequestType::TickPrice as u32 {
        let Some(tick_type) = fields.get(3) else { return };
        if tick_type != "4" { return } // We only want 4 = 'Last' from ticktype.py.
        let Some(price) = fields.get(4) else { return };
        println!("Price: {}", price);
    } else {
        println!("Received: {:?}", fields); 
    }
}

Now, lets listen for messages. Add this at the bottom of main(), but keep in mind we’ll replace it with a threaded version soon:

loop {
    let message = read_message(&mut stream);
    message_received(&message);
}

Run it with cargo run and you should see a bunch of messages printed. You’ll need to do ctrl-c to quit.

But you won’t see any prices come in, because we haven’t requested any. This led me to some thinking around how to receive and send messages. I personally like the idea of making a reading thread, a writing thread, and a control (main) thread, to spread the load. So firstly, in main(), remove the loop above, and add this in its place, to ‘split the socket’ into a reader and writer:

// !! Ensure you remove the previous loop above !!
let mut writer = stream.try_clone().expect("Clone");
let mut reader = stream;

Next we want a mechanism for the main thread to enqueue messages for the writer thread to send:

// use std::sync::mpsc::channel;
let (writer_tx, writer_rx) = channel::<Vec<String>>();

Next lets spawn our reader thread for incoming messages:

// use std::thread;
let reader_handle = thread::Builder::new().name("Reader".into()).spawn(move || {
    loop {
        let message = read_message(&mut reader);
        message_received(&message);
    }
}).expect("Spawn");

Next lets spawn the writer thread that listens to the channel, then writes the messages to the socket:

let writer_handle = thread::Builder::new().name("Writer".into()).spawn(move || {
    loop {
        let fields = writer_rx.recv().expect("Writer queue receive");
        let message = make_message(&fields);
        println!("Sending {:?} aka {:?} aka {:?}", 
            fields, String::from_utf8_lossy(&message), message);
        writer.write_all(&message).expect("Write all");
    }
}).expect("Spawn");

Now we need to get the main thread to wait for the new threads:

writer_handle.join().unwrap();
reader_handle.join().unwrap();

You can run it now, and it should receive messages and run until you perform ctrl-c.

It still isn’t showing price data however, as we need to ask TWS for that. We’ll do that now by sending a message from the main thread to the writer thread. Add the following code just above the previously-added calls to ‘join()’:

// Post-handshake delay or TWS bugs out.
thread::sleep(std::time::Duration::from_millis(500));

let request_market_data: Vec<String> = vec![
    (OutgoingRequestType::ReqMktData as u32).to_string(),
    "11".to_string(),     // Version
    "999".to_string(),    // Request id aka Ticker id
    "".to_string(),       // Contract id
    "BTC".to_string(),    // Symbol
    "CRYPTO".to_string(), // Security type
    "".to_string(),       // LastTradeDateOrContractMonth
    "".to_string(),       // strike
    "".to_string(),       // right
    "".to_string(),       // multiplier
    "PAXOS".to_string(),  // exchange
    "".to_string(),       // primaryExchange
    "USD".to_string(),    // currency
    "".to_string(),       // localSymbol
    "".to_string(),       // tradingClass
    "0".to_string(),      // Delta neutral contract? 0=false, 1=true
    "".to_string(),       // genericTickList
    "0".to_string(),      // is a snapshot?
    "0".to_string(),      // regulatorySnapshot - costs 1c ea.
    "".to_string(),       // mktDataOptions - unsupported.
];
writer_tx.send(request_market_data).expect("Writer queue send");

// join calls go here...

Then run it, and you’ll get (among other messages) the real-time price of Bitcoin in USD! You can take it from here and build the trading bot of your dreams.

This code carries no warranty, use it at your own risk, and is MIT licensed!

For reference for the other messages, you’ll have to read through IB’s Python SDK. Thanks for reading, God bless, and have a nice week :)

Thanks for reading! And if you want to get in touch, I'd love to hear from you: chris.hulbert at gmail.

Chris Hulbert

(Comp Sci, Hons - UTS)

Software Developer (Freelancer / Contractor) in Australia.

I have worked at places such as Google, Cochlear, Assembly Payments, News Corp, Fox Sports, NineMSN, FetchTV, Coles, Woolworths, Trust Bank, and Westpac, among others. If you're looking for help developing an iOS app, drop me a line!

Get in touch:
[email protected]
github.com/chrishulbert
linkedin

Subscribe via RSS