Getting Data: Real-Time Quotes

As promised I'll discuss the options for real-time data... it seems that getting real-time data has become increasingly easier and some brokerages actually have APIs that help you get real-time data or they allow you to download small interval historical prices (minutes, seconds or even ticks).

In this post I'll discuss how to get free "real-time" data and how you can use that data to build your own historical prices database. Yahoo offers delayed quotes (up to 15 minutes), but it also has "real-time" data and we can take advantage of that with a little programming.

First I'd like to introduce you to the Yahoo "API", which is nothing more than a collection of tags. You can see all the available tags here: http://www.gummy-stuff.org/Yahoo-data.htm

Getting real-time quotes is much like getting historical data from Yahoo: you will build a URL with the tags you're interested in and you'll use that URL to get a CSV file containing a quote. Here is how to construct a URL:

http://finance.yahoo.com/d/quotes.csv?s= + STOCK_SYMBOL(S) + &f= + TAG(S)

Let's try it with one symbol and one tag: the symbol will be "GOOG" (Google) and the tag wil be "g" (day's low):

http://finance.yahoo.com/d/quotes.csv?s=GOOG&f=g

If we want to get multiple tags we can just string them together (will return a CSV file with the high and the low of the day):

http://finance.yahoo.com/d/quotes.csv?s=GOOG&f=hg

Let's fetch a CSV file containing Google's high and low values for the day on the first line and Microsoft's high and low values for the day on the next line:

http://finance.yahoo.com/d/quotes.csv?s=GOOG+MSFT&f=hg

Here is an example that downloads all the data for the specified tags and prints them to the screen alongside their description:

// A dictionary with tags where the key is the tag 
// and the value is the description of the tag.
private Dictionary _tags;

private void DownloadData(String symbol)
{
    string url = String.Format(
        "http://finance.yahoo.com/d/quotes.csv?s={0}&f=", symbol);

    //Get page showing the table with the chosen indices
    HttpWebRequest request = null;
    DFDataSet ds = new DFDataSet();
    Random rand = new Random(DateTime.Now.Millisecond);
    try
    {
        while (_running)
        {
            foreach (String key in _tags.Keys)
            {
                lock (_sync)
                {
                    request = (HttpWebRequest)WebRequest.CreateDefault(
                        new Uri(url + key));
                    request.Timeout = 30000;
                    
                    using (var response = (HttpWebResponse)request.GetResponse())
                    using (StreamReader input = new StreamReader(
                        response.GetResponseStream()))
                    {
                        Console.WriteLine(String.Format("{0} {1} = {2}", 
                            symbol, _tags[key], input.ReadLine());
                    }
                }
            }
            Console.WriteLine(Thread.CurrentThread.Name + " running.");
            Thread.Sleep(60*1000); // 60 seconds
        }
    }
    catch (Exception exc)
    {
        Console.WriteLine(exc.Message);
    }
}

At this point you should have everything you need to build your own historical database:
1. Download the CSV file with all the tags you're interested in.
2. Open a connection to your database.
3. Bulk insert the CSV file into your database.