Hi all, here’s the second on my series on neural networks / machine learning / AI from scratch. In the previous article (please read it first!), I explained
how a single neuron works. In this article, I’ll explain how you can determine the ‘gradients’ of that neuron, in other words how much effect the weight and bias has on the final ‘loss’, using some high-school calculus. This is an prerequisite for training, which I’ll cover later.
I recommend opening this spreadsheet in a separate tab, and viewing it as you read this post which explains the maths: Single neuron gradients.
In case the linked spreadsheet is lost to posterity, here it is in slightly less well-formatted form (note: for brevity’s sake, I’ve shortened references such as B2 to simply ‘B’ when referring to a column in the same row):
A | B | C | D | E | F | G | H | I | J | K | |
---|---|---|---|---|---|---|---|---|---|---|---|
1 | Input | Weight | Bias | Net | Output | Target | Error | Loss | |||
2 | Neuron maths: | 0.4 | 0.5 | 0.6 | 0.8 (B*C+D) | 0.664 (tanh(E)) | 0.7 | -0.035963 (F-G) | 0.0006467 (H^2 / 2) | ||
3 | Real local gradients: | 0.5 (C2) | 0.4 (B2) | 1 | 0.5591 (1-F2^2) | -0.036 (H2) | |||||
4 | Real global gradients: | -0.0101 (B3*E) | -0.0080 (C3*E) | -0.0201 (E) | -0.0201 (E3*F) | -0.036 (F3) | |||||
5 | Faux gradient | ||||||||||
6 | Faux gradient of ‘output’: | 0.66414 (F2+Tiny) | 0.7 | -0.035863 (F-G) | 0.0006431 (H^2 / 2) | -0.0359 ((I - I2)/Tiny) | |||||
7 | Faux gradient of ‘net’: | 0.8001 (E2+Tiny) | 0.66409 (tanh(E)) | 0.7 | -0.035907 (F-G) | 0.0006447 (H^2 / 2) | -0.0201 ((I - I2)/Tiny) | ||||
8 | Faux gradient of ‘bias’: | 0.4 | 0.5 | 0.6001 (D2+Tiny) | 0.8001 (B*C+D) | 0.66409 (tanh(E)) | 0.7 | -0.035907 (F-G) | 0.0006447 (H^2 / 2) | -0.0201 ((I - I2)/Tiny) | |
9 | Faux gradient of ‘weight’: | 0.4 | 0.5001 (C2+Tiny) | 0.6 | 0.80004 (B*C+D) | 0.66406 (tanh(E)) | 0.7 | -0.035941 (F-G) | 0.0006459 (H^2 / 2) | -0.0080 ((I - I2)/Tiny) | |
10 | Faux gradient of ‘input’: | 0.4001 (B2+Tiny) | 0.5 | 0.6 | 0.80005 (B*C+D) | 0.66406 (tanh(E)) | 0.7 | -0.035935 (F-G) | 0.0006457 (H^2 / 2) | -0.0100 ((I - I2)/Tiny) | |
Tiny | 0.0001 | Moved down here to help with readability |
Firstly: what is the gradient? It is also known as the slope, derivative, or velocity of an equation.
For a simple example, consider tides in a river mouth:
In this analogy, the height of the water is the position (like the values for the weights, bias, net, output, or loss), and the velocity of the water is the gradient (or derivative, or slope). Figuring out that gradient is what this article is all about.
For a more thorough explanation of gradients, check out Wikipedia.
The reason we want the gradients of a neuron’s weight(s) and bias, is that we can use them to figure out whether we need to nudge their values up or down a bit or leave them as-is, in order to get an output that’s closer to the target during training.
You can fake a gradient by comparing the result of an equation vs the result when adding a tiny amount to the input. These faux gradients are helpful for verifying our calculus later.
Here’s the general way to fake a gradient:
Faux gradient of f(x) = ( f(x + tiny) - f(x) ) / tiny
To make it more specific to our neuron:
Faux gradient of how weight affects output = (
tanh(input * (weight + tiny) + bias) -
tanh(input * weight + bias)
) / tiny
Or the full kahuna on the loss function:
Faux gradient of how bias affects loss = (
(tanh(input * weight + (bias + tiny)) - target)^2 / 2
-
(tanh(input * weight + bias) - target)^2 / 2
) / tiny
Please note that the loss function changed vs the previous article (it now has a / 2
) - this is to make the calculus simpler.
You can look at rows 6 through 10 in the spreadsheet to see how these faux gradients are calculated. In columns B to I, various
things have the tiny value added to them, to see how this affects the final ‘loss’. For instance, on row 6, you can see I’m adding
the tiny value to the output, then feeding that through to the loss function, and doing the (loss with tiny - loss without tiny) / tiny
to
calculate the faux gradient. The rest of these faux gradients are similar.
Lets use calculus to calculate the real gradients. Firstly we need to calculate the ‘local’ gradients. See row 3 in the spreadsheet as you follow along:
What is a local gradient? Since all our calculations are performed in stages (eg net > output > error > loss), a local gradient is how much impact changes in one stage have on the next stage.
A better maths teacher than I would be able to explain how we arrive at the following, but here are the formulas below:
(Note when I say ‘the gradient of Y with respect to X’ it means that X is the input/earlier stage, Y is the output/later stage, and it roughly means ‘if you nudge X, what impact will that have on Y?’.)
/ 2
in our loss helps) (see H3)Next we need to combine the gradients using the calculus ‘chain rule’, so that we can get the impacts of each variable on the loss.
These are calculated in reverse order (this is why it is called _back_propagation) because most of these rely on the next step’s gradient.
You may like to compare these with the respective faux gradients and see that they are (roughly) the same.
And there you have it, you have the gradients for a single neuron. Next I’ll explain how to calculate the gradients for a network of them!
Just for the hell of it, here’s an implementation in Rust:
struct Neuron {
input: f32,
weight: f32,
bias: f32,
target: f32,
}
impl Neuron {
fn net(&self) -> f32 {
self.input * self.weight + self.bias
}
fn output(&self) -> f32 {
self.net().tanh()
}
fn error(&self) -> f32 {
self.output() - self.target
}
fn loss(&self) -> f32 {
let e = self.error();
e * e / 2.
}
fn output_gradient(&self) -> f32 {
self.error()
}
fn net_gradient(&self) -> f32 {
let o = self.output();
let net_local_derivative = 1. - o * o;
net_local_derivative * self.output_gradient()
}
fn bias_gradient(&self) -> f32 {
self.net_gradient()
}
fn weight_gradient(&self) -> f32 {
self.input * self.net_gradient()
}
}
fn main() {
let neuron = Neuron {
input: 0.4,
weight: 0.5,
bias: 0.6,
target: 0.7,
};
println!("Weight gradient: {:.4}", neuron.weight_gradient());
println!("Bias gradient: {:.4}", neuron.bias_gradient());
}
Which outputs:
Weight gradient: -0.0080
Bias gradient: -0.0201
Which matches the spreadsheet nicely!
Thanks for reading, hope you found this helpful, at least a tiny bit, God bless!
Photo by Chinnu Indrakumar on Unsplash
Hi all, I’d like to do a series on neural networks (or machine learning, or AI), starting from the very basics, not using any frameworks. This is inspired by Andrej Karpathy’s intro video here so perhaps consider watching that (if you can find a few hours spare!). I seem to be writing about maths a lot lately, which gave me an idea: everyone understands spreadsheets (Excel / Pages / Google Sheets), so I’m going to use them to (hopefully!) make the maths clearer.
I want to make ‘what is a neuron’ concrete in some way, to give you a ‘scaffold’ to build your learning on, I believe that helps.
So: say you want to define a mathematical formula for ‘how much is a square block of land worth’. It has two inputs: width and length. It might look like this:
Land price($) = width(m) * length(m) * 200 + 100000
You could call this a function: value(s) in, value out. A machine learning neuron is just one of these: It takes some input(s), does some maths with them, and outputs a value. And a massive grid of these neurons all connected together can achieve surprisingly complex results.
Here’s how the maths behind a single neuron works. There’s not much to it:
Net input = Input 1 * Weight 1 + Input 2 * Weight 2 + Bias
Output = tanh(Net input)
Please click here to see the above in spreadsheet form. I tried embedding a nice JS spreadsheet but it didn’t work on mobile, thus the google sheets link. In case that doesn’t work, it looks like so:
A | B | C | D | E | F | G | H | I | |
---|---|---|---|---|---|---|---|---|---|
1 | Input 1 | Input 2 | Weight 1 | Weight 2 | Bias | Net | Output | Target | Loss |
2 | 0.9 | 0.8 | 0.7 | 0.6 | 0.5 | =A2 * C2 + B2 * D2 + E2 | =tanh(F2) | 0.8 | =(G2 - H2)^2 |
You may be wondering what ‘tanh’ is. It’s a hyperbolic tangent, which neatly squashes the net and spits out a value between -1 and 1. This is called the ‘activation function’ - there are other options (eg the logistic function) that can be used instead.
Initial values for the weights and bias are random numbers in the range -1..1. They are tweaked in the learning process, which I’ll explain in an upcoming article. The collection of weights and biases are also called the parameters.
Loss is used to calculate how ‘good’ a neural network is at calculating the desired target. It will be always positive, and the closer to zero the better. In this simple example, it is calculated as the square of the output-vs-target delta:
loss = (output - target)^2
Because I enjoy fooling around with Rust, here’s a little demo, perhaps this will solidify the concepts from a developer’s perspective:
struct Neuron {
input1: f32,
input2: f32,
weight1: f32,
weight2: f32,
bias: f32,
}
impl Neuron {
fn net(&self) -> f32 {
self.input1 * self.weight1 +
self.input2 * self.weight2 +
self.bias
}
fn output(&self) -> f32 {
self.net().tanh()
}
fn loss(&self, target: f32) -> f32 {
let delta = target - self.output();
delta * delta
}
}
fn main() {
let neuron = Neuron {
input1: 0.1,
input2: 0.2,
weight1: 0.3,
weight2: 0.4,
bias: 0.5,
};
println!("Net: {:.3}", neuron.net());
println!("Output: {:.3}", neuron.output());
println!("Loss: {:.3}", neuron.loss(0.5));
}
Thanks for reading, hope you found this helpful, at least a tiny bit, God bless!
Photo by Josh Riemer on Unsplash
Hi all, here’s how to implement a position-and-velocity Kalman filter from scratch, in C++ish pseudocode, from the perspective of a developer who isn’t a professional mathematician and doesn’t understand the kind of symbology used in many Kalman articles. I wrote another article about a simpler Kalman filter as a primer on the subject, if you read it first it will hopefully help make this article easier to understand.
Firstly we need to implement some matrix maths stuff as a prerequisite. It might help to skim-read Wikipedia on the topic first.
// A 2-row column vector.
struct Vector {
double a // aka Position or Price
double b // aka Velocity
}
// A 2x2 matrix.
struct Matrix {
double tl // Top Left, eg row 1 column 1
double tr // Top right, r1c2
double bl // Bottom left, r2c1
double br // Bottom right, r2c2
}
// vectorA + vectorB.
+(Vector lhs, Vector rhs) -> Vector {
return Vector {
a: lhs.a + rhs.a,
b: lhs.b + rhs.b
}
}
// vectorA - vectorB.
-(Vector lhs, Vector rhs) -> Vector {
return Vector(
a: lhs.a - rhs.a,
b: lhs.b - rhs.b
)
}
// matrixA + matrixB.
+(Matrix lhs, Matrix rhs) -> Matrix {
return Matrix(
tl: lhs.tl + rhs.tl, tr: lhs.tr + rhs.tr,
bl: lhs.bl + rhs.bl, br: lhs.br + rhs.br
)
}
// matrixA - matrixB.
-(Matrix l, Matrix r) -> Matrix {
return Matrix(
tl: l.tl - r.tl, tr: l.tr - r.tr,
bl: l.bl - r.bl, br: l.br - r.br
)
}
// matrixA * matrixB.
*(Matrix l, Matrix r) -> Matrix {
return Matrix(
tl: l.tl * r.tl + l.tr * r.bl,
tr: l.tl * r.tr + l.tr * r.br,
bl: l.bl * r.tl + l.br * r.bl,
br: l.bl * r.tr + l.br * r.br
)
}
// vectorOutput = matrixA * vectorB.
*(Matrix m, Vector v) -> Vector {
return Vector(
a: m.tl * v.a + m.tr * v.b,
b: m.bl * v.a + m.br * v.b
)
}
// Invert a matrix (the matrix version of 1 / m).
invert(Matrix m) -> Matrix {
return Matrix(
tl: m.br, tr: -m.tr,
bl: -m.bl, br: m.tl
)
}
// Transpose a matrix (mirror it diagonally).
transpose(Matrix m) -> Matrix {
return Matrix(
tl: m.tl, tr: m.bl,
bl: m.tr, br: m.br
)
}
Next, you’ll need the ‘external noise covariance matrix’. This is usually referred to as Q in Kalman literature:
const double Q_POSITION_VARIANCE = 0.1
const double Q_VELOCITY_VARIANCE = 0.1
const double Q_POSITION_VELOCITY_COVARIANCE = 0.1
const Matrix Q_EXTERNAL_NOISE = {
tl: Q_POSITION_VARIANCE, tr: Q_POSITION_VELOCITY_COVARIANCE,
bl: Q_POSITION_VELOCITY_COVARIANCE, br: Q_VELOCITY_VARIANCE,
}
Next, you’ll need the ‘measurement noise covariance matrix’, aka R in Kalman articles:
const double R_POSITION_VARIANCE = 0.1
const double R_VELOCITY_VARIANCE = 0.1
const double R_POSITION_VELOCITY_COVARIANCE = 0.1
const Matrix R_MEASUREMENT_NOISE = {
tl: R_POSITION_VARIANCE, tr: R_POSITION_VELOCITY_COVARIANCE,
bl: R_POSITION_VELOCITY_COVARIANCE, br: R_VELOCITY_VARIANCE,
}
How do you select good Q and R values? I’m not an expert, but here’s my two tips:
Next, we need our ‘state transition matrix’, aka F in Kalman articles. This is a constant matrix that, when multiplied by the current state vector (position + velocity), gives us our expected next state vector (old position + velocity, same velocity). Eg it looks like:
const Matrix F_STATE_TRANSITION = {
tl: 1, tr: 1,
bl: 0, br: 1,
}
Maybe the above needs more explaining. Say our old state is position 100m, velocity 10m/s. Multiplying this by F works out like so:
Normal matrix * vector formula:
a: mat.tl * vec.a + mat.tr * vec.b,
b: mat.bl * vec.a + mat.br * vec.b
AKA:
new position: f.tl * position + f.tr * velocity
new velocity: f.bl * position + f.br * velocity
AKA:
new position: 1 * position + 1 * velocity
new velocity: 0 * position + 1 * velocity
AKA:
new position: position + velocity
new velocity: velocity
AKA:
new position: 110m
new velocity: 10m/s
In other words, for a timestep of 1 second, F simply adds the velocity-per-second to the current position, and assumes no friction hence velocity stays constant.
Next, we need the ‘state error’, aka P in Kalman articles. I find it usually works fine to start filtering with this as 1. (Perhaps this should be scaled to match your typical values.)
Matrix pStateError = {
tl: 1, tr: 1,
bl: 1, br: 1,
}
Next, we need our ‘current state’, aka X in Kalman articles. If, like me, you’re faking the velocity, start the filter with this set using the first two measurements:
Vector xCurrentState = {
a: data[1],
b: data[1] - data[0], // Faux velocity.
}
Phew! Now for the interesting bit. For each new data point/measurement, we need to first predict what it’ll be, then blend that prediction with the measurement to produce an estimate. This estimate is the filter’s output. A ‘Kalman gain’ (known as K in Kalman articles) is calculated then is used to weight the prediction vs measurement when blending them to form the estimate:
// Make predictions.
Vector xPredicted = F_STATE_TRANSITION * xCurrentState
Matrix pPredicted = F_STATE_TRANSITION * pStateError * transpose(F_STATE_TRANSITION) + Q_EXTERNAL_NOISE_VARIANCE
// Update it with the measurement.
Vector zMeasurement = Vector {
a: data[current_index], // Position.
b: data[current_index] - data[previous_index] // Faux velocity.
}
Matrix kKalmanGain = pPredicted * invert(pPredicted + R_MEASUREMENT_NOISE_VARIANCE)
Vector xEstimate = xPredicted + kKalmanGain * (zMeasurement - xPredicted)
Matrix pEstimate = pPredicted - kKalmanGain * pPredicted
// Store for the next iteration.
pStateError = pEstimate
xCurrentState = xEstimate // This is the output!
Thanks for reading, I hope this helped, God bless, and have a nice week :)
You can see older posts in the right panel, under 'archive'.
Neural Networks explained with spreadsheets, 2: Gradients for a single neuron 20 Mar 2024
Neural Networks explained with spreadsheets, 1: A single neuron 10 Mar 2024
How to implement a position-and-velocity Kalman Filter 15 Dec 2023
How to implement a position-only Kalman Filter 14 Dec 2023
Rust Crypto Ticker using Interactive Brokers' TWS API directly 28 Aug 2023
Rust PNG writer from scratch 12 Jul 2022
Swift Security framework wrapper for RSA and Elliptic Curve encryption / decryption 21 Sep 2021
Simple, practical async await Swift examples 3 Jul 2021
Xcode pbxproj project generator in Swift 17 May 2021
UITableViewDiffableDataSource for adding and removing rows automatically to a table view in Swift 10 May 2021
Super simple iOS Combine example 23 Feb 2021
Introducing Chalkinator: Native desktop blogging app 7 Jun 2020
Flare: Open source 2-way folder sync to Backblaze B2 in Swift 28 May 2020
Making a baby monitor out of a couple of ESP32s, an I2S microphone, and a small speaker 16 Apr 2020
Chris' 2020 guide to hosting a HTTPS static site on AWS S3 + Cloudfront 15 Mar 2020
Simple Javascript debounce, no libraries needed 20 Feb 2020
Asynchronous NSOperations in Swift 5 3 Jan 2020
Deploying Golang Revel sites to AWS Elastic Beanstalk 9 Dec 2019
Golang and pure Swift Compression and Decompression 28 Jul 2019
Pure Swift simple Keychain wrapper 23 Jun 2019
Pure Swift 5 CommonCrypto AES Encryption 9 Jun 2019
Bluetooth example code for Swift/iOS 6 Jun 2019
Talking to a Bluetooth LE peripheral with Swift/iOS 18 May 2019
Obfuscating Keys using Swift 5 May 2019
State Machines in Swift using enums 10 Apr 2019
iOS timers without circular references with Pendulum 28 Mar 2019
Pragmatic Reactive Programming 11 Oct 2017
React Native first impressions 7 Apr 2017
Gondola 26 Feb 2017
Scalable Swift 22 Nov 2016
Swift 3 Migration 6 Nov 2016
Enum-Driven View Controllers 3 Jan 2016
Status bar colours: Everything there is to know 30 Dec 2015
Android server 20 Dec 2015
Generating heightmap terrain with Swift 8 Nov 2015
Swift Education Screencasts 27 Oct 2015
Swift Image Cache 24 Sep 2015
Don't be slack 13 Sep 2015
Swift KVO alternative 23 Jul 2015
Swift Keychain wrapper 21 Jun 2015
Swift NSURLSession wrapper 12 Jun 2015
iOS8 View Controller transitioning bug 17 Apr 2015
IB Designable 18 Mar 2015
iOS App Architecture 2 Mar 2015
Video Course Launch 14 Feb 2015
Video Course Pre-launch 8 Feb 2015
Blogging Platforms 13 Jan 2015
Mobile in 2014 - Year in Review 11 Jan 2015
Secret Keys talk 16 Nov 2014
Dimmi 11 Nov 2014
Project setup in Xcode6 22 Oct 2014
Uploading to an S3 bucket from iOS 15 Oct 2014
iOS8 App Testing Roundup 28 Sep 2014
Storing obfuscated secret keys in your iOS app 16 Sep 2014
Getting Core Location / CLLocationManager to work on iOS8 14 Sep 2014
Accessing the response body in failure blocks with AFNetworking 2 10 Sep 2014
How to allow your UITextFields to scroll out of the way of the keyboard 8 Sep 2014
How to subclass UIButton in iOS7 and make a UIButtonTypeSystem 4 Sep 2014
New season 1 Aug 2014
House finished 17 Jun 2014
WebP decoding on iOS 9 Feb 2014
Moving on again 22 Jan 2014
Lossy images for retina iPads - JPEG vs WebP 30 Nov 2013
Career options I wish I knew about when I was younger 20 Oct 2013
Positivity and your friends 7 Oct 2013
Tactility 26 Jul 2013
WWDC-induced narcolepsy 15 Jul 2013
Back on rails 31 May 2013
Full circle 6 May 2013
Programmatic UI on iOS 3 May 2013
Screencasts and positivity 8 Apr 2013
Year of positivity 14 Mar 2013
iOS Dev State of the Union 6 Feb 2013
Adventures with IAPs 3 Feb 2013
No longer a Googler 23 Dec 2012
Localising iPhone apps with Microsoft Translator 8 Dec 2012
Fight back (app biz update 13) 12 Nov 2012
Sent to the backburner (app biz update 12) 25 Oct 2012
Lisi Schappi 7 Oct 2012
Today's happy plateau (app biz update 11) 26 Aug 2012
First week's sales of Today (app biz update 10) 19 Aug 2012
Today launch! And a difficult decision made... (app biz update 9) 15 Aug 2012
Approved! (app biz update 8) 5 Aug 2012
Creating a graph in Objective-C on the iPhone 3 Aug 2012
Hurry up and wait (app biz update 7) 30 Jul 2012
Today app marketing site 27 Jul 2012
Today app submitted 25 Jul 2012
UIAlertView input wrapper 24 Jul 2012
Mentoring 23 Jul 2012
This is too hard! (app biz update 6) 20 Jul 2012
Perspectives (app biz update 5) 9 Jul 2012
4th starting-my-own-biz update 1 Jul 2012
ScrumFox landing page 28 Jun 2012
Server Scope landing page 27 Jun 2012
Telstra Calls and Data Usage 26 Jun 2012
Service History + Dropbox 26 Jun 2012
Impromptu Presenter 26 Jun 2012
Fertility Tracker 26 Jun 2012
Baby Allergy Tracker 26 Jun 2012
Starting my own business, update 3 22 Jun 2012
Starting my own business, update 2 17 Jun 2012
Starting my own business - First update 10 Jun 2012
I must be crazy 6 Jun 2012
Finding your location on an iPhone 7 May 2012
A generous career 4 May 2012
Skeleton Key Cocoaheads presentation 3 May 2012
CHBgDropboxSync - Dropbox auto-sync for your iOS apps 1 May 2012
That book about that Steve Jobs guy 30 Apr 2012
Another app marketing idea 23 Apr 2012
Sweet grouped tables on the iPhone 17 Apr 2012
Skeleton Key App 11 Apr 2012
Another app marketing idea... 5 Apr 2012
Quickly check for any missing retina graphics in your project 3 Apr 2012
Skeleton Key Password Manager with Dropbox 2 Apr 2012
RC Boat motor finally mounted 2 Apr 2012
Promoting apps presentation slides 1 Apr 2012
How i just wasted a month on my latest app, and how you don't need to 26 Mar 2012
The Finishing Line 20 Mar 2012
Using Launchd to run a script every 5 mins on a Mac 20 Feb 2012
Generating AES256 keys from a password/passphrase in ObjC 20 Feb 2012
Indie iPhone app marketing, part 2 19 Feb 2012
My App Manifesto: Syncing + Dropbox + YAML = Awesome 15 Feb 2012
Indie iPhone App Marketing part 1 7 Feb 2012
Perspectives 2 Feb 2012
Accountability and Free Will 1 Feb 2012
Badassery 31 Jan 2012
Sacrifice 30 Jan 2012
Lead Yourself First 29 Jan 2012
How to ping a server in Objective-C / iPhone 26 Jan 2012
iOS Automated Builds with Xcode4 16 Jan 2012
Xcode 4 - Command line builds of iPhone apps 15 Jan 2012
Guest post by Jason McDougall 13 Jan 2012
Scouts, Games and Motivation 10 Jan 2012
2011 Re-cap 8 Jan 2012
Ruby script to increment a build number 4 Jan 2012
Turning 30? All ideas, no execution? 18 Dec 2011
CHDropboxSync - simply sync your iOS app's documents to Dropbox 14 Dec 2011
Deep-enumerating a directory on the iphone, getting file attributes as you go 10 Dec 2011
Getting a date without the time component in objective-c 6 Dec 2011
Memory management in Objective-C 4 Dec 2011
Starting small 29 Nov 2011
Dictionary Types Helper 29 Nov 2011
Observer Pattern in Objective-C 16 Nov 2011
Why you should give presentations 13 Nov 2011
How to get a programming or design job in Sydney 9 Nov 2011
Custom nav bar / toolbar backgrounds in iOS5 8 Nov 2011
Stuck 27 Oct 2011
Dead easy singletons in Obj-C 19 Oct 2011
JSON vs OCON (Objective-C Object Notation) 18 Oct 2011
In defence of Objective-C 16 Oct 2011
Update the MessagePack objective-c library to support packing 12 Oct 2011
Icons 11 Oct 2011
How to host a site on Amazon AWS S3, step-by-step 7 Oct 2011
Drawing a textured pattern over the default UINavigationBar 6 Oct 2011
Markdown Presentations 1 Oct 2011
More MegaComet testing: Ruling out keepalives 15 Sep 2011
MegaComet test #4 - This time with more kernel 14 Sep 2011
Building People 10 Sep 2011
Half way there: Getting MegaComet to 523,000 concurrent HTTP connections 5 Sep 2011
Making a progress bar in your iPhone UINavigationBar 22 Aug 2011
Hacker News Reader 20 Aug 2011
How to programmatically resize elements for landscape vs portrait in your iphone interface 16 Aug 2011
MegaComet testing part 2 3 Aug 2011
Australian Baby Colours 28 Jul 2011
Boat prop shaft 25 Jul 2011
Megacomet with 1 million queued messages 24 Jul 2011
Installed the strut and rudder 18 Jul 2011
Painted the inside of the boat 17 Jul 2011
Fuzzy iphone graphics when using an UIImageView set to UIViewContentModeCenter 13 Jul 2011
My 3 Data and Calls Usage 11 Jul 2011
Reading a line from the console in node.js 10 Jul 2011
Trim whitespaces on all text fields in a view controller 9 Jul 2011
Final finish 9 Jul 2011
MessagePack parser for Objective-C / iPhone 30 Jun 2011
Lacquering the starboard side 25 Jun 2011
What do do with EXC_ARM_DA_ALIGN on an iPhone app 23 Jun 2011
Lacquering the hull 23 Jun 2011
Staining the boat 22 Jun 2011
NSMutableSet with weak references in objective-c 20 Jun 2011
Iphone gesture recogniser that works for baby games 20 Jun 2011
Image manipulation pixel by pixel in objective C for the iphone 19 Jun 2011
Baby Allergy Tracker 12 Jun 2011
Power sanding the deck 10 Jun 2011
Planing the edge of the deck 2 Jun 2011
Figured out the deck 2 Jun 2011
Boat bulkheads 2 Jun 2011
Simulating iOS memory warnings 31 May 2011
Putting a UIButton in a UIToolbar 29 May 2011
How to allow closing a UIActionSheet by tapping outside it 29 May 2011
Finding the currently visible view in a UITabBarController 24 May 2011
Random Chef 17 May 2011
Centered UIButton in a navigation bar on the iphone 16 May 2011
Little Orchard 13 May 2011
Boat update 13 May 2011
How to get the current time in all time zones for the iphone / obj-c 12 May 2011
Design portfolio 10 May 2011
Tricks with grand central dispatch, such as objective-c's equivalent to setTimeout 9 May 2011
How to make an iphone view controller detect left or right swipes 5 May 2011
Centered section headers on a UITableView 5 May 2011
Christmas in may 4 May 2011
Finished trimming the boat (its floatable now!) and got some parts 29 Apr 2011
How to make a multiline label with dynamic text on the iphone and get the correct height 27 Apr 2011
Forcing an image size on the image in a table view cell on an iphone 20 Apr 2011
Git on the Mac 19 Apr 2011
Build a url query string in obj-c from a dictionary of params like jquery does 12 Apr 2011
Rendering a radial gradient on the iphone / objective-c 11 Apr 2011
Skinning the port side of the boat 8 Apr 2011
Skinning the side of the boat 5 Apr 2011
Sending a UDP broadcast packet in C / Objective-C 5 Apr 2011
How to talk to a unix socket / named pipe with python 4 Apr 2011
Skinning the bottom of the boat 31 Mar 2011
Service discovery using node.js and ssdp / universal plug n play 30 Mar 2011
Extremely simple python threading 29 Mar 2011
New rescue boat 26 Mar 2011
HttpContext vs HttpContextBase vs HttpContextWrapper 5 Nov 2010
Simple C# Wiki engine 30 Sep 2010
Simple way to throttle parts of your Asp.Net web app 29 Sep 2010
How to implement DES and Triple DES from scratch 4 Aug 2010
How to use sessions with Struts 2 30 Jul 2010
How to use Cookies in Struts 2 with ServletRequest and ServletResponse 30 Jul 2010
Using Quartz Scheduler in a Java web app (servlet) 27 Jul 2010
Javascript date picker that Doesn't Suck!(tm) 27 Jul 2010
Using Oracle XE with Hibernate 20 Jul 2010
A simple implementation of AES in Ruby from scratch 29 Jun 2010
Asp.Net Forms authentication to your own database 28 May 2010
AS2805 (like ISO8583) financial message parser in C# 7 May 2010
Ruby hex dumper 4 May 2010
Using Spring to manage Hibernate sessions in Struts2 (and other web frameworks) 13 Jan 2010
Emails in C#: Delivery and Read receipts / Attachments 12 Jan 2010
Using Java libraries in a C# app with IKVM 16 Dec 2009
Learning Java tutorial 27 Nov 2009
Using generic database providers with C# 17 Nov 2009
Scheduled task executable batch babysitter 29 Oct 2009
Working with query strings in Javascript using Prototype 30 Sep 2009
Still fighting with String.Format? 9 Sep 2009
How I'd build the next Google 24 Aug 2009
Getting IIS and Tomcat to play nicely with isapi_redirect 24 Aug 2009
Using the new ODP.Net to access Oracle from C# with simple deployment 11 Aug 2009
C# Cryptography - Encrypting a bunch of bytes 14 Jul 2009
Sorting enormous files using a C# external merge sort 10 Jul 2009
Reconciling/comparing huge data sets with C# 9 Jul 2009
Some keyboard-friendly DHTML tricks 10 Jun 2009
How to figure out what/who is connected to your SQL server 18 Mar 2009
Adding a column to a massive Sql server table 16 Mar 2009
Multithreading using Delegates in C# 10 Mar 2009
Using C# locks and threads to rip through a to-do list 6 Feb 2009
Using threads and lock in C# 3 Feb 2009
Compressing using the 7Zip LZMA algorithm in C# beats GZipStream 14 Jan 2009
MS Sql Server 2005 locking 17 Dec 2008
Simple Comet demo for Ruby on Rails 19 Nov 2008
Geocoding part 2 - Plotting postcodes onto a map of Australia with C# 24 Oct 2008
Using evolutionary algorithms to make a walkthrough for the light-bot game with C# 20 Oct 2008
How to tell when memory leaks are about to kill your Asp.Net application 16 Oct 2008
C# version of isxdigit - is a character a hex digit? 15 Sep 2008
Geocoding part 1 - Getting the longitude and latitude of all australian postcodes from google maps 26 Aug 2008
Converting HSV to RGB colour using C# 14 Aug 2008
Opening a TCP connection in C# with a custom timeout 11 Aug 2008
Oracle Explorer - a very simple C# open source Toad alternative 31 Jul 2008
Linking DigitalMars' D with a C library (Mongrel's HTTP parser) 23 Jun 2008
Connecting to Oracle from C# / Winforms / Asp.net without tnsnames.ora 16 Jun 2008
A simple server: DigitalMars' D + Libev 6 Jun 2008
Travelling from Rails 1 to Rails 2 9 Apr 2008
Online Rostering System 9 Apr 2008
DanceInforma 9 Apr 2008
Using RSS or Atom to keep an eye on your company's heartbeat 10 Nov 2007
Easy Integrated Active Directory Security in ASP.Net 24 Oct 2007