Ding Zhou

use interactive brokers API to get historical forex data

Ding — Mon, 31 Jan 2011 01:16:56 +0000

I’ve been trying to use ibPy + TWS + django to get forex trading data from Interactive Brokers’ API. I was blocked by various configuration errors due to lack of a good documentation on the internet. Just wanted to share this with interested algorithmic traders to save you some time.

Let’s suppose you wanted to get hourly trading candles for the past 10 day. You first need to make a contract that specifies the currency pair. Suppose you use ‘USD’ as your base currency then your python code using ibPY would look more or less like below (the configuration should be consistent in other languages).

def make_contract_forex(currency_symbol):
  '''
  contract for a currency pair
  '''
  contract = Contract()
  contract.m_symbol = currency_symbol
  # STK or S - stocks,
  # FUT or F- futures
  # FOP or P - options on futures
  # OPT or O - options
  # IND or I - indexes
  # CASH or C -cash (ideal FX)
  contract.m_secType = 'CASH'
  contract.m_exchange = 'IDEALPRO'
  contract.m_primaryExch = 'IDEALPRO'
  contract.m_currency = 'USD'
  contract.m_localSymbol = '%s.USD' % currency_symbol
  return contract

Then this is how you can request candles from your local TWS client:

contract = make_contract_forex('GBP')
self.connection.reqHistoricalData(
  reqId,
  contract,
  nowStr,
  durStr,
  barSize,
  'MIDPOINT',
  useRTH,
  form)

, where the toShow needs to be ‘MIDPOINT’ instead of anything else. The reqId, nowStr, durStr, barSize, useRTH, form can be configured according to IB’s API guide.

Hope this helps…

Share on Facebook

topic detection & tracking

Ding — Wed, 14 Apr 2010 21:31:38 +0000

over a decade went by, i have seen tons of papers on numerous data sets, but i can no longer find papers as solid as this one:

Topic Detection and Tracking Pilot Study Final Report

surprisingly, from the limited number of citations, few people have seen the value in this report

Share on Facebook

set up remote desktop in ubuntu

Ding — Sat, 26 Dec 2009 21:07:15 +0000

1. first install the package via apt-get

sudo apt-get install vnc4server

2. add two lines in ~/.vnc/xstartup

 unset SESSION_MANAGER
 sh /etc/X11/xinit/xinitrc

3. set up the password required to go to the desktop

vncpasswd

4. launch the remote desktop server:

vncserver :1

This will start a new desktop in the background ready for a remote user to connect to. Note the :1 above tells the server to bind port 5901 for vncserver.

5. You can also make sure you start a vncserver upon restarting. In /etc/rc.local add a line (can assign a port for each user you want remote desktop access):

6. now if you want to connect from a windows machine, download the free VNC client here:

7. Sometimes, you want to set up ssh for security. What you do is to set up a port forwarding so you’d only have to maintain a ssh tunnel to the server. Try this when you ssh:

 ssh -fNC -L9999:localhost:5909 myvncserver.com

Share on Facebook

invest in the investors; not the MF

Ding — Wed, 16 Dec 2009 07:17:00 +0000

The world is becoming more open and transparent, as evident by more private content and info being shared on the Web or through social networks every day. People know more about each other and understand more about the world’s information at an amazing speed. However, there is one place in the U.S. that has been a forbidden city full of mysteries; there is one thing in the world that is a carefully guarded secret — do we really know how those Wall Street firms invest *our* money?

In this interesting country, people care a lot about how the government spend their mandatory tax money so they get together every four years and burn hundreds of millions of dollars to listen to a few candidate proposals. This way, they feel secured as they understand how 1/3 of their gross income would be spent. However, when it comes to dealing with their life-long savings, most of them surprisingly know very little about how the retirement fund or mutual funds or hedge fund works. For most of them, when deciding on their investment, all they take as input is the name of the financial firm and the sector of mutual fund (MF). Very few of them look at who the fund manager is. And I’ve never hear of anyone switching to a new MF only because the previous star manager left. However, it’s really the fund manager who drives the fund and they jump around all the time — why do you still buy the season ticket now that Michael Jordan has left Bulls? Few people understand this or with limited options choose to neglect this. They buy and hold through ups and downs, letting go most of profits made *their money* into the Wall street’s pocket. The root cause of this is the market inefficiency set forth by people’s limited options and lack of transparency from Wall street. I thought about this a few years ago — there has to be a better and more transparent way to invest my money! And here is one:

http://www.techcrunch.com/2009/12/15/kaching-funding/

It’s the people that matters. Why should we pay so much to the firms that hire them? A small amount of insurance premium to hedge against risks is fine but we have obviously been paying to fly their jets as well…

Share on Facebook

create your calculator with awk

Ding — Thu, 05 Nov 2009 08:11:18 +0000

in your bash, do the following

calc(){ awk "BEGIN{ print $* }" ; }

and then do the following:

calc 2*1 - 12

and

calc '2*(1 - 12)'

you can put this in your ~/.bash_profile too…

Share on Facebook

Good stocks? bad stocks?

Ding — Sat, 10 Oct 2009 21:35:36 +0000

So one of the many challenges an algorithmic trader faces is to come up with a good watch list. Why? — A trading strategy is almost always conditioned on a watch list. And the total performance is the product of the quality of the watchlist and that of the trading strategy. In the past, I’ve read a lot of good articles proposing various watchlist generation ideas. Despite the rosy pictures they paint, I believe in data. So I wrote a small program to provide another data point.

This is what I did. I write a wrapper around one trading strategy of mine. Randomly sample n stocks from a grand watch list. Pair each of the sample with a few index ETF and run my trading over x years of history data until a few years ago. Then based on the performances of each sample stock, I divide them into two buckets, and run the same trading strategy from that point onward until now, on two new watch lists respectively. Here are my observations:

1. Overall, over a long period of time, there is no positive correlation between a stock’s past performance with a stock’s future performance, conditioned on the same trading strategy

2. For large cap, there is in fact a negative correlation between past and future performance. In many cases, the future gain between the top and bottom half of the stocks by past performance, can differ by 2x or more.

3. For small cap, there is a weak positive correlation between past and future performance. The future performances differ even more dramatically between *good* and *bad* stocks than the past performances.

Of course, i can’t tell you on what time period i’ve drawn the above conclusion. But in one word, I observe that, in the market, there are trends and there’re waves.

Takeaways? There’re a few. Some specific to traders are: 1) past successful strategies will very likely fail in the future, at least on the same set of stocks (so develop new strategies or at least update your watchlists, if you don’t want to be another prey of the wall street); 2) do not get too excited with your backtest results, history does not repeat, again, at least on the same set of stocks.

An even more profound lesson here applies to not only the stock market. Note how very unlikely a successful company (a large cap is a built on top of its past success I assume) can remain successful over a long time? It’s much easier for a great company to remain good when it is small. But is much much harder for it to repeat its success, at least according to the statistics.

Share on Facebook

To plug in Dell S2409W for old laptops (e.g. IBM X60s)

Ding — Mon, 21 Sep 2009 08:51:50 +0000

My IBM X60s is so old that it has to upgrade its bios in order for the right resolution (1920 x 1080) to show up. I’m on an XP OS. Here are the step-by-steps that’ll hopefully save you sometime:

1. right click My Computer on Desktop and choose System Properties. Go to the Device Manager Tab and Choose Device Manager. There you’ll find out what your Display Adapter’s Chipset is. I have a X60s and my Video Adapter is Mobile Intel(R) 945 Express Chipset Family.

2. Go to intel.com or google to search for an upgrade of your chipset. Here’s the download link for Intel 945GM Chipset . Now install the new chipset driver. You can also use the Intel driver tool to detect whether your video driver needs upgrading.

3. Go to Dell to download the config file for the monitor. I have a S2409W so my config file is downloaded from the Dell’s driver support page. Extract files from the downloaded zip archive and remember the location.

4. Go to the Device Manager and choose the Monitor from the list. Right click and choose Update Driver. Then choose to manually upgrade driver using files as extracted in the previous step.

5. Now upgrade your Laptop’s bios so it knows that there’re a monitor model called S2409W. Usually your laptop vendor’s web page has upgrade file. I have a IBM X60s and I downloaded the file from here.

6. Unzip the downloaded bios upgrade file and double click to run the WINUPTP.EXE.

7. now you can restart your machine for the first time.

8. Right click on desktop and click on Properties. Go to setting and you should be able to use the new resolution of (1920 x 1080)

Share on Facebook

Lee Kaifu leaving Google China

Ding — Fri, 04 Sep 2009 19:15:50 +0000

Wikipedia has updated Kai-Fu’s page almost right after the official announcement by Google China.

While many are celebrating Kai-Fu’s achievement today, let’s take a look at his previous jobs:

Corporate Vice President, Natural Interactive Services Division (NISD), Microsoft Corp. 2000 – July, 2005 [3]
Founder, Microsoft Research Asia, China, 1998-2000
President, Cosmo Software
President, Multimedia Software Business Unit, Silicon Graphics Inc. (SGI)
Vice President & General Manager, Web Products, Silicon Graphics Inc. (SGI)
Vice President, Interactive Media Group, Apple Computer, 1990-1996
Assistant Professor, Carnegie Mellon University, 1988-1990

It’s rather a long professional history of 20 years before he decides to do something of his own. On the other hand, 20 years, being spent to reach where he is now, is indeed an amazing journey. I’m holding my breath for KaiFu’s ventures in China. I thought we’ve passed the golden age of Internet? KaiFu could prove me wrong on China.

Share on Facebook

so you think you can trade with algorithms?

Ding — Wed, 26 Aug 2009 05:33:30 +0000

interesting read — performance really depends on your selection of initial watchlist. on SP100, in my implementation, the performance has been 10%-20% year over year for the past 10 years. the portfolio shrinks to 40% in worst times…

“Algorithms for Portfolio Management based on the Newton Method”

Share on Facebook

Emacs: Change function color

Ding — Thu, 20 Aug 2009 23:27:06 +0000

An emacs tip to share:

M-x customize-face
// twice to view all kinds for things to cutomize
// font-lock-* are the things you can attach color
Customize face> font-lock-function-name-face RET
// Then you can edit the color options in a nice form

Enjoy coding on servers…

Share on Facebook