Chris' Babbling Blog

As part of a grand plan to be able to map a bunch of addresses onto a graphical map of sydney or australia, firstly you need all the australian postcodes and their longitude and latitude. Now in this article i describe how to screen scrape this list from google maps (please forgive me Mr Google!).

Firstly you'll need the list of all postcodes from Australia Post. You can get it here - it is a zip of a CSV file. It contains a whole bunch of non-physical addresses such as PO boxes - if you're inclined, open it in Excel and remove these. What you want to end up with is a file called 'pc-full.csv' which we'll use later.

For the next step you'll need Curl installed. You can get it here. If you're behind a firewall, there's some hints here on how to get Curl/Wget to play nicely with firewalls (even though the instructions are for wget, it still applies to curl). Once you think you've got it sorted, try this from the command prompt:

curl "http://maps.google.com.au/maps?f=q&hl=en&geocode=&q=2768+australia&output=js"

You should get a printout of a whole bunch of javascript with 'center:{lat:xxx,lng:yyy}' in it somewhere. Sweet - you've got curl working.

Next up we want to write a script to do all the work. I've used ruby, because it's awesome, but you can use whatever you want. The following code opens up the 'pc-full.csv' file, reads the list of distinct postcodes, runs curl against each one, parses the longitude and latitude from each, and outputs a nice CSV file with 3 columns: Postcode, Latitude, Longitude.
Now i'll have to apologise for the sloppy code, but it does work (updated to fix wordpress' curly quotes):

def IsGood(fname)
  # Does a file contain the longitude eg did it connect to gmaps correctly?
  # Ironically, in C# this code would be a one liner: File.ReadAllText(fname).Contains("center:{lat:");
  r = "Missing"
  if File.exist?(fname)
    f = File.new(fname)
    lines = f.read
    f.close
    if lines.include?("center:{lat:")
      r = 'Good'
    else
      r = 'Bad'
    end
  end
  r
end

# get the list of unique postcodes from the CSV file downloaded from australia post
@postcodes = []
File.new('pc-full.csv').readlines.each {|l|
  x = l.gsub(/[\",]/,"").to_i
 @postcodes << x if x>0 && !@postcodes.include?(x)
}
puts "Total #{@postcodes.length} unique postcodes"

# Scrape them all from google maps
@postcodes.each {|postcode|
  fname = "data_#{postcode}.txt"
  if !File.exist?(fname)
   system "curl -o #{fname} \"http://maps.google.com.au/" +
     "maps?f=q&hl=en&geocode=&q=#{"%04d" % postcode}+australia&output=js\""
   puts "Any good? #{IsGood(fname)}"
  end
}

# Go through the resultant files, parsing the longitude and latitude
@results = ["Postcode,Lat,Lng"]
@postcodes.each {|postcode|
  fname = "data_#{postcode}.txt"

  status = IsGood(fname)
  puts "#{fname} : #{status}"

  # Grab the long & lat
  if status=='Good'
    f = File.new(fname)
    lines = f.read
    f.close
    m = /center:\{lat:([\-.0-9]*),lng:([\-.0-9]*)\}/.match(lines)
    @results << "#{postcode},#{m[1]},#{m[2]}"
  end
}

# Write it out to a CSV file
File.open("PostcodeLatLng.csv","w") {|f|
  @results.each {|line|
    f.write line
    f.write "\n"
  }
}

Note: Put curl.exe in the same folder as your ruby file above, if it's not in the path.
And your output from running all that should be a 'PostcodeLatLng.csv' file with the contents something like this:

Postcode,Lat,Lng
2000,-33.869027000000003,151.21024499999999
2001,-37.808776999999999,144.94928899999999
2002,-25.335448,135.74507600000001
2004,-33.891787999999998,151.17625100000001

Bob's your uncle! You should now have the longitude and latitude of all australian postcodes. In a later article, i'll show how i use this to make a map of australian post offices overlaying a map of australia, for instance.

The other day i had to figure out how to convert HSV to RGB using C#, and after browsing the net i could only find C sample code to do it with, so i had to convert to C#. The code may be useful to someone out there (please let me know if it is):

/// <summary>
/// Convert HSV to RGB
/// h is from 0-360
/// s,v values are 0-1
/// r,g,b values are 0-255
/// Based upon http://ilab.usc.edu/wiki/index.php/HSV_And_H2SV_Color_Space#HSV_Transformation_C_.2F_C.2B.2B_Code_2
/// </summary>
void HsvToRgb(double h, double S, double V, out int r, out int g, out int b)
{
  // ######################################################################
  // T. Nathan Mundhenk
  // mundhenk@usc.edu
  // C/C++ Macro HSV to RGB

  double H = h;
  while (H < 0) { H += 360; };
  while (H >= 360) { H -= 360; };
  double R, G, B;
  if (V <= 0)
    { R = G = B = 0; }
  else if (S <= 0)
  {
    R = G = B = V;
  }
  else
  {
    double hf = H / 60.0;
    int i = (int)Math.Floor(hf);
    double f = hf - i;
    double pv = V * (1 - S);
    double qv = V * (1 - S * f);
    double tv = V * (1 - S * (1 - f));
    switch (i)
    {

      // Red is the dominant color

      case 0:
        R = V;
        G = tv;
        B = pv;
        break;

      // Green is the dominant color

      case 1:
        R = qv;
        G = V;
        B = pv;
        break;
      case 2:
        R = pv;
        G = V;
        B = tv;
        break;

      // Blue is the dominant color

      case 3:
        R = pv;
        G = qv;
        B = V;
        break;
      case 4:
        R = tv;
        G = pv;
        B = V;
        break;

      // Red is the dominant color

      case 5:
        R = V;
        G = pv;
        B = qv;
        break;

      // Just in case we overshoot on our math by a little, we put these here. Since its a switch it won't slow us down at all to put these here.

      case 6:
        R = V;
        G = tv;
        B = pv;
        break;
      case -1:
        R = V;
        G = pv;
        B = qv;
        break;

      // The color is not defined, we should throw an error.

      default:
        //LFATAL("i Value error in Pixel conversion, Value is %d", i);
        R = G = B = V; // Just pretend its black/white
        break;
    }
  }
  r = Clamp((int)(R * 255.0));
  g = Clamp((int)(G * 255.0));
  b = Clamp((int)(B * 255.0));
}

/// <summary>
/// Clamp a value to 0-255
/// </summary>
int Clamp(int i)
{
  if (i < 0) return 0;
  if (i > 255) return 255;
  return i;
}

And here's how you'd use it:

int r,g,b;
HsvToRgb(110, 1, 1, out r, out g, out b);

The TcpClient class in C# is great for opening a TCP connection, I must say that it's one of the nicest TCP libraries i've used. You just have to watch out for the occasional bug and you'll be right.

One limitation is a frustration though: the inability to set a timeout when opening a connection to a remote server. The default timeout is 60 seconds, which is quite a while to have the user strumming their fingers waiting for things to happen.

My solution to this is to spawn a thread which opens the TCP connection, while the original thread waits up to a user-specified timeout for it to connect. If it hasn't connected by then, it kills the thread and gives up. Notice the use of the thread's Join function which allows the original thread to stop waiting if the connection is quicker than the timeout. Without further ado, the TcpClientWithTimeout.cs class:

using System;
using System.Net.Sockets;
using System.Threading;

/// 

/// TcpClientWithTimeout is used to open a TcpClient connection, with a user definable connection timeout in milliseconds (1000=1second)
/// Use it like this:
/// TcpClient connection = new TcpClientWithTimeout('127.0.0.1',80,1000).Connect();
/// 
public class TcpClientWithTimeout
{
  protected string _hostname;
  protected int _port;
  protected int _timeout_milliseconds;
  protected TcpClient connection;
  protected bool connected;
  protected Exception exception;

  public TcpClientWithTimeout(string hostname,int port,int timeout_milliseconds)
  {
    _hostname = hostname;
    _port = port;
    _timeout_milliseconds = timeout_milliseconds;
  }
	public TcpClient Connect()
	{
    // kick off the thread that tries to connect
    connected = false;
    exception = null;
    Thread thread = new Thread(new ThreadStart(BeginConnect));
    thread.Start();

    // wait for either the timeout or the thread to finish
    thread.Join(_timeout_milliseconds);

    if (connected == true)
    {
      // it succeeded, so return the connection
      thread.Abort();
      return connection;
    }
    if (exception != null)
    {
      // it crashed, so return the exception to the caller
      thread.Abort();
      throw exception;
    }
    else
    {
      // if it gets here, it timed out, so abort the thread and throw an exception
      thread.Abort();
      string message = string.Format("TcpClient connection to {0}:{1} timed out", _hostname, _port);
      throw new TimeoutException(message);
    }
	}
  protected void BeginConnect()
  {
    try
    {
      connection = new TcpClient(_hostname, _port);
      // record that it succeeded, for the main thread to return to the caller
      connected = true;
    }
    catch (Exception ex)
    {
      // record the exception for the main thread to re-throw back to the calling code
      exception = ex;
    }
  }
}

And here's a little example of how to use this to open a connection, send 10 bytes, and receive 10 bytes:

// connect with a 5 second timeout on the connection
TcpClient connection = new TcpClientWithTimeout("www.google.com", 80, 5000).Connect();
NetworkStream stream = connection.GetStream();

// Send 10 bytes
byte[] to_send = {0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0xa};
stream.Write(to_send, 0, to_send.Length); 

// Receive 10 bytes
byte[] readbuf = new byte[10]; // you must allocate space first
stream.ReadTimeout = 10000; // 10 second timeout on the read
stream.Read(readbuf, 0, 10); // read

// Disconnect nicely
stream.Close(); // workaround for a .net bug: http://support.microsoft.com/kb/821625
connection.Close();

Cheers all, please let me know if this is useful or if you have any constructive criticism.

Hi all, here's Oracle Explorer, a very simple C# (VS2005) open source Toad alternative for browsing through oracle databases.
It was created in reaction to the awful load times of oracle's java-based Sql Developer, and i can't be bothered with the procurement process for Toad. Maybe someone wants to start a project with this and take it further?
Anyway here's the obligatory screenshot:

And the download (including both the EXE and source code): (updated to fix 2 bugs)

Oracle Explorer source and exe

To install and run it, make sure you get the Instant Client from oracle's website, and copy the oci.dll, orannzsbb*.dll, oraocci*.dll, and oraociicus*.dll files into the same folder as the oracle explorer EXE file.

This project is a follow on from a previous blog entry here: Connecting to Oracle from C# / Winforms / Asp.net without tnsnames.ora

Constructive comments are very welcome! (But abuse will be ignored & removed - sheesh!)

This post is intended to give a step-by-step example of linking D with a C library. If you want to mimic my windows development environment, to start with, you'll need to download Tango and DigitalMars C. I used the below versions, but you may also want to check for the latest. I'm also toying with the idea of creating a Rack-style D http server, and coming up with a D web framework. Download dm850c.zip from here, and tango-0.99.6-bin-win32-dmd.1.029.zip from here.

Amusingly, i'd like to highlight that here we're combining Ragel, which creates the Mongrel HTTP parser C code, which was intended for a Ruby web server, finally linking it with D.

Download this article's code here
Make sure you read this first, too.

The (awesome) Mongrel HTTP parser

I grabbed parser.c and parser.h from Ebb, as i had problems with the Mongrel one. You can get it here.
Place the two files into a folder and run the following command to compile them with this command:
\dm\bin\dmc -c parser.c -I.
This will produce the parser.obj if you've installed dmc (Digitalmars' C) correctly.

Bindings between C and D

To make bindings to D, I basically placed the contents of the parser.h into an 'extern(C)' block and deleted all the 'const' references, like so:

extern(C)
{
  enum { MONGREL_CONTENT_LENGTH
       , MONGREL_CONTENT_TYPE
       , MONGREL_FRAGMENT
       , MONGREL_HTTP_VERSION
       , MONGREL_QUERY_STRING
       , MONGREL_REQUEST_PATH
       , MONGREL_REQUEST_METHOD
       , MONGREL_REQUEST_URI
       };

  typedef void (*field_cb)(void *data, char *field, size_t flen, char *value, size_t vlen);
  typedef void (*element_cb)(void *data, int type, char *at, size_t length);

  struct http_parser {
    int cs;
    int overflow_error;
    size_t body_start;
    size_t content_length;
    size_t nread;
    size_t mark;
    size_t field_start;
    size_t field_len;
    size_t query_start;

    void *data;

    field_cb http_field;
    element_cb on_element;
  };

  void http_parser_init(http_parser *parser);
  int http_parser_finish(http_parser *parser);
  size_t http_parser_execute(http_parser *parser, char *data, size_t len, size_t off);
  int http_parser_has_error(http_parser *parser);
  int http_parser_is_finished(http_parser *parser);
}

Since mongrel uses a couple of call-back functions, they must also be within 'extern(C)' blocks:

extern (C) void http_field_cb(void *data, char *field, size_t flen, char *value, size_t vlen)
{ ...code here... }

extern (C) void on_element(void *data, int type, char *at, size_t length)
{ ...code here... }

The D Server

I'm using Tango in the simplest possible way to listen to port 80, feed the input to Mongrel, and return an html page. Here's the entire serve.d:

import tango.io.Stdout;
import tango.stdc.stringz;
import tango.net.ServerSocket, tango.net.SocketConduit;

// Stuff for the mongrel parser
extern(C)
{

  enum { MONGREL_CONTENT_LENGTH
       , MONGREL_CONTENT_TYPE
       , MONGREL_FRAGMENT
       , MONGREL_HTTP_VERSION
       , MONGREL_QUERY_STRING
       , MONGREL_REQUEST_PATH
       , MONGREL_REQUEST_METHOD
       , MONGREL_REQUEST_URI
       };

  typedef void (*field_cb)(void *data, char *field, size_t flen, char *value, size_t vlen);
  typedef void (*element_cb)(void *data, int type, char *at, size_t length);

  struct http_parser {
    int cs;
    int overflow_error;
    size_t body_start;
    size_t content_length;
    size_t nread;
    size_t mark;
    size_t field_start;
    size_t field_len;
    size_t query_start;

    void *data;

    field_cb http_field;
    element_cb on_element;
  };

  void http_parser_init(http_parser *parser);
  int http_parser_finish(http_parser *parser);
  size_t http_parser_execute(http_parser *parser, char *data, size_t len, size_t off);
  int http_parser_has_error(http_parser *parser);
  int http_parser_is_finished(http_parser *parser);
}
// End of stuff for the mongrel parser

// My callback functions

extern (C) void http_field_cb(void *data, char *field, size_t flen, char *value, size_t vlen)
{
  char[]* display = cast(char[]*)data;
  *display ~= "<i>http_field_cb</i> <b>";
  *display ~= field[0..flen];
  *display ~= "</b> = ";
  *display ~= value[0..vlen];
  *display ~= "<br>";
}

extern (C) void on_element(void *data, int type, char *at, size_t length)
{
  char[]* display = cast(char[]*)data;

  char[] line;
  line ~= "<i>on_element</i> <b>";
  if (type==MONGREL_CONTENT_LENGTH) line ~= "MONGREL_CONTENT_LENGTH";
  if (type==MONGREL_CONTENT_TYPE) line ~= "MONGREL_CONTENT_TYPE";
  if (type==MONGREL_FRAGMENT) line ~= "MONGREL_FRAGMENT";
  if (type==MONGREL_HTTP_VERSION) line ~= "MONGREL_HTTP_VERSION";
  if (type==MONGREL_QUERY_STRING) line ~= "MONGREL_QUERY_STRING";
  if (type==MONGREL_REQUEST_PATH) line ~= "MONGREL_REQUEST_PATH";
  if (type==MONGREL_REQUEST_METHOD) line ~= "MONGREL_REQUEST_METHOD";
  if (type==MONGREL_REQUEST_URI) line ~= "MONGREL_REQUEST_URI";
  line ~= "</b> = " ~ at[0..length] ~ "<br>";
  *display ~= line;
}

void main()
{
  // Set up the server
  Stdout("Now open your browser to http://localhost/").newline();
  Stdout("Ctrl-Break or Ctrl-C to quit").newline();
  auto server = new ServerSocket (new InternetAddress(80));

  while(1)
  {
    // wait for requests
    SocketConduit request = server.accept;

    // wait for the 'http get ...'
    char[1024] response;
    uint len = request.read (response);
    Stdout(response[0..len]).newline; // you'll want to comment this out if you run apachebench against this...

    // parse it
    char[] display;
    display = "";
    http_parser parser;
    http_parser_init(&parser);
    parser.data = &display; // data that is sent to the callback function
    parser.http_field = &http_field_cb;
    parser.on_element = &on_element;
    http_parser_execute( &parser
                       , cast(char*)response
                       , len
                       , 0
                       );
    display ~= http_parser_has_error(&parser) ? "-mongrel error;" : "-no mongrel error;";
    display ~= http_parser_is_finished(&parser) ? "mongrel finished-" : "mongrel not finished-";;

    // send HTML
    request.output.write("HTTP/1.1 200 OK

<html>
<style>
  body {font-family:trebuchet ms;background:#524D4A;color:#fff;}
  .comment {font-size:80%;margin:0;color:#cccccc;}
  form {background:#6B7552; margin:20px 0 20px 0; padding:5px;}
</style>
<body>
  <h1>D + Mongrel's HTTP parser</h1>

  <form method='get'>
    <h2>GET form</h2>
    <input name='blah' value='test value'>
    <input type='submit'>
  </form>

  <form method='post'>
    <h2>POST form</h2>
    <input name='blah' value='test value'>
    <input type='submit'>
  </form>  

  <form method='post'>
    <h2>POST form with file</h2>
    <input type='file' name='blah'>
    <input type='submit'>
  </form>

  <form>
    <h2>Mongrel Parser Results:</h2>
    " ~ display ~ "
  </form>
  </body>
</html>");

    request.close();
  }
}

Compiling it all together

To compile it together, use the following command, which will produce serve.exe:
\dmd\bin\dmd serve.d parser.obj
Then open your web browser to http://localhost/ and you should see this:

Voila! That's all until next time.

Summary:
The other day, I needed to access an oracle database from a C# console application, and google found a whole heap of different ways to achieve this. Here's the best way i found that works, and you don't even need to install any oracle software. I've deliberately kept this entry short, to emphasise that this is the simple and straightforward way.
Also, you may want to check out another post for a simple application using this technique: Oracle Explorer - very simple open source Toad alternative

Oracle bits:
Full credit: This part uses details from http://www.codeproject.com/KB/database/C__Instant_Oracle.aspx

You'll need the following files, which can be obtained from within the 'basic lite' version of oracle's instant client:
oci.dll
orannzsbb11.dll
oraocci11.dll
oraociicus11.dll
In my case, i downloaded the 'instantclient-basiclite-win32-11.1.0.6.0.zip' file from their website, the link is here.
You'll need to place these DLL's in the same folder as your application's .EXE. If you're using asp.net, put it in the 'Bin' folder of your application. The important thing to note is that you don't need to install the instant client, or register these DLLs or anything, just grab them from the zip file and ignore the rest of its contents.

Visual Studio bits:
Note that i'm using .Net 2, with Visual Studio 2005. There may be differences in other versions, however i'm hoping for your sake that it is similar enough to figure out.
In the solution explorer, right click on 'References' and choose 'Add Reference…' as below:

Then scroll through the list until you find 'System.Data.OracleClient' and click OK.

Code:
Now for the code. You'll need this 'using' at the top of your file:

using System.Data.OracleClient;

Here's a simple function I wrote to create an oracle connection string, it may be useful to you:

public string OracleConnString(string host,string port,string servicename,string user,string pass)
{
  return String.Format(
    "Data Source=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST={0})" +
    "(PORT={1}))(CONNECT_DATA=(SERVICE_NAME={2})));User Id={3};Password={4};",
    host,
    port,
    servicename,
    user,
    pass);
}

And here's the gist of opening a connection, and reading all the lines. You'll notice the interface is pretty much the same as accessing SQL server.

string connectionstring = OracleConnString("aaa","1521","bbb","ccc","ddd");
string sql = "select * from some_table";

using (OracleConnection conn = new OracleConnection(connectionstring)) // connect to oracle
{
  conn.Open(); // open the oracle connection
  using (OracleCommand comm = new OracleCommand(sql, conn)) // create the oracle sql command
  {
    using (OracleDataReader rdr = comm.ExecuteReader()) // execute the oracle sql and start reading it
    {
      while (rdr.Read()) // loop through each row from oracle
      {
        Console.WriteLine( rdr[0] );             // You can do this
        Console.WriteLine( rdr.GetString(0); );  // or this
        Console.WriteLine( rdr["column_name"] ); // or this
      }
      rdr.Close(); // close the oracle reader
    }
  }
  conn.Close(); // close the oracle connection
}

That's all!

Copying from Oracle to Sql Server
Here's an example of copying a 2-column table from Oracle to Sql server. Lets pretend this table is called 'foo', and has two integer columns 'x' and 'y'.

private void CopyOracleToSql()
{
  string connectionstring = OracleConnString("host","1521","servicename","user","pass");
  using (OracleConnection conn = new OracleConnection(connectionstring)) // connect to oracle
  {
    conn.Open(); // open the oracle connection
    string sql = "select x,y from foo";
    using (OracleCommand comm = new OracleCommand(sql, conn)) // create the oracle sql command
    {
      using (OracleDataReader rdr = comm.ExecuteReader()) // execute the oracle sql and start reading it
      {
        using (SqlConnection sql_conn = new SqlConnection(sqlconnstring)) // connect to the sql 2005 database
        {
          sql_conn.Open(); // open the sql database

          // Read the data from oracle, and every 100 rows send it to the sql database
          StringBuilder sb = new StringBuilder(); // build the insert statements with this
          int row = 0;
          while (rdr.Read()) // loop through each row from oracle
          {
            sb.AppendFormat(
              "insert into foo(x,y) values({0},{1});",
              rdr[0], rdr[1]);
            row++;

            if (row >= 100) // send to the DB every 100 rows
            {
              using (SqlCommand sql_comm = new SqlCommand(sb.ToString(), sql_conn))
              {
                sql_comm.ExecuteNonQuery();
                row = 0;
                sb = new StringBuilder();
              }
            }
          }

          // get the remaining few rows and send them to the DB
          if (sb.Length > 0)
          {
            using (SqlCommand sql_comm = new SqlCommand(sb.ToString(), sql_conn))
            {
              sql_comm.ExecuteNonQuery();
            }
          }

          rdr.Close(); // close the oracle reader
          sql_conn.Close(); // close the sql connection
        }
      }
    }
    conn.Close(); // close the oracle connection
  }
}

Firstly, a note: The aim of this article is more of a step by step 'first steps with D' kinda thing, i'm not trying to make too much of a point about D's performance here. Calm down, redditors!

Environment

I'm working with Windows XP here, which adds a few complications you wouldn't see on other platforms.
Using the Digitalmars' C compiler (dm850c.zip) from here, unzipped to c:\dm.
Also using the Tango D package (tango-0.99.6-bin-win32-dmd.1.029.zip) from here, unzipped to c:\dmd.

Libev

Libev is a high performance event loop that i'm using to wait for incoming port 80 connections.
It's here: http://software.schmorp.de/pkg/libev.html
Docs are here: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod
Download the CVS version (it contains fixes so it'll compile) here: http://cvs.schmorp.de/libev.tar.gz?view=tar

Unzip it all into a folder, and create a 'myev.c' file in that folder, with the following contents:

#define EV_STANDALONE 1 // so i don't need a config.h file
#define EV_SELECT_IS_WINSOCKET 1 // windows
//#define EV_USE_POLL 1 // uncomment me for unix
//#define EV_USE_EPOLL 1 // uncomment me for gnu/linux
//#define EV_USE_KQUEUE 1 // uncomment me for bsd/osx
//#define EV_USE_PORT 1 // uncomment me for solaris
#define EV_STAT_ENABLE 0 // disable file watching
#include "ev.c"

To compile it, do the following from that folder (you'll get a couple of warnings which you can ignore):

\dm\bin\dmc myev.c -c

This will produce a myev.obj which you'll need later

Server

1: Create a new folder for the simple 'D' server. You'll need the ev.d with the libev bindings, which you can download or copy n paste from here.

2: Copy the 'myev.obj' that you compiled earlier from the libev folder into this folder.

3: You'll then create the 'server.d' which is the main point of this article:

import tango.io.Console;
import tango.net.ServerSocket, tango.net.SocketConduit;
import ev; 

extern (C)
{
   static void libev_cb (ev_loop_t *loop, ev_io *w, int revents)
   {
     callback();
   }

  // http://msdn.microsoft.com/en-us/library/bdts1c9x(VS.71).aspx
  int _open_osfhandle (long osfhandle, int flags);
}

ServerSocket listener;
void main()
{
  listener = new ServerSocket (new InternetAddress(80));

  int fd = _open_osfhandle(listener.fileHandle,0); // for win32: convert from socket to file descriptor

  // Start libev
  ev_loop_t* loop = ev_default_loop(0);
  ev_io io_watcher;
  ev_io_init(&io_watcher, &libev_cb, fd, READ);
  ev_io_start(loop, &io_watcher);
  ev_loop(loop, 0);
} 

void callback()
{
  // accept the connection
  SocketConduit request = listener.accept;

  // wait for the 'http get ...'
  char[1024] response;
  uint len = request.input.read (response);
  Cout (response[0..len]).newline; // you'll want to comment this out if you run apachebench against this...

  // send HTML
  request.output.write("HTTP/1.1 200 OK

<html>
<style>body {font-family:trebuchet ms;background:#424242;color:#fff;text-align:center;margin-top:10em;}</style>
<body><h1>Your 'D' web server is alive!</h1></body>
</html>");

  request.close();
}

To compile and run all this, do the following:

\dmd\bin\dmd server.d ev.d myev.obj
server

Next test it in your browser, by browsing to 'localhost':

Voila! And if i do an apachebench (ab -n 10000 -c 100), my requests-per-second is 2071 for an 2 years old P4-3ghz, and it only uses 2.4mb of memory, which is pretty impressive considering each Ruby on Rails Mongrel typically uses ~30megs last time i checked.
I guess the next question is: why doesn't someone make a super-scalable web framework using the speed/efficiency and ease of D?

If any of this doesn't work, make sure you post a comment so i can sort it out - cheers.

I've recently been working on converting my 'rosters' rails 1.x app to rails version 2, and there are plenty of pitfalls that i spent lots of time googling around trying to find the solutions.
So, for just as much need for my own future reference as anyone else's, here are all the specific traps *I* ran into, and their solutions, in the one place.
If you're interested in the source code to the whole app, you can see the rails 1.x version here:
http://rosters.rubyforge.org/
The new version is running here, and if enough people request it, i'll release the code.

Basics

Firstly, I created a new rails 2 skeleton app, using 'rails myappname' from the command line. Then i copied the contents of all my app and public folders over from the old app to the new one. Finally i went through the db and config directories, modifying the new files manually, rather than simply overwriting them with the old ones. This way you'll get to see how the configuration files (especially routes!) have changed, and won't be overwriting them with legacy code (it feels strange to refer to rails 1 as 'legacy'!)

What? Rails is embracing security now?

Forms now have a special 'key' that needs to be present for POSTs, to prevent cross-site-request-forgery (try saying that 10 times fast!). The gist of it is that you can't just have code like this in your views any more:

<form name="f" action="/account/login" method="post" onsubmit="wait();">
...
</form>

Now what you want is more like this:

<% form_tag("/account/login", :name=>'f', :onsubmit=>"wait();") { %>
...
<% } %>

The above code takes care of the new CSRF-proof keys and all that. As you can see, my example is a bit more complex than usual, with a form name and an onsubmit javascript callback, but hey, its real code!
For a simpler, bare-bones form, Rails 2 style, here you go:

<% form_tag :action=> "new" do %>
...
<% end %>


Where on earth did 'find_first' go?

Back in controller-land, i hit problems with all my find_first's that i like to use. This kind of thing simply wont work any more:

u = find_first(["login = ? AND password = ?", login, sha1(pass)])

You need something more like this nowadays:

u = find :first, :conditions=>["login = ? AND password = ?", login, sha1(pass)]

It appears that all the find_* functions (find_first,find_all,etc) have all been rolled into the one-function-to-rule-them-all 'find' function. I guess it is neater.

No more @session or @request

This one's simple. Simply replace these kind of things:

@session
@request

With this:

session
request

One big 'find-in-files' (Ctrl-Shift-F in Notepad++) will sort you out with this one.

Redirect

Another simple one, but its likely you'll have one of these in each of your updating actions:

redirect_to_url '/blah/foo'
...becomes...
redirect_to '/yada/yada'


RIP Pagination

Apparently pagination wasn't quite good enough to meet muster for Rails 2, so its been scrapped totally.
There's no simple replacement for this one, but hopefully you're only really using it in your admin scaffolds, so what i did was a simple pagination-dectomy in my controllers:

@user_pages, @users = paginate :users, :per_page => 30, :order=>'login'

Became:

@users = User.find :all, :order=>'login'

Its pretty simple, and you'll need to remove all references to pagination in your views. If you really *really* need pagination, you'll need something more complex, but there isn't anything out-of-the-box for you.
For more details: http://wiki.rubyonrails.org/rails/pages/HowtoPagination

End Results

Well, now the site's up and running, you're all welcome to have a look:
http://rosters.morphexchange.com/

DanceInforma is a web-zine devoted to dancers (hence the name).
My role consists of maintaining the back end database involved with maintaining the membership details of all members, ~3-4000 at last count.
http://danceinforma.com/

FreelanceSwitch contracted me to develop their Client Analyser, which is publicly accessible here:

http://clientanalyser.freelanceswitch.com/

This is another Rails application, like most of my public work.