Air-Writing Recognition
Write in the air with your finger and get digital text. A contribution to an existing open-source computer vision project, extended and presented at university.
Problem
Turning a fingertip traced through the air into recognised text joins two machine learning problems that do not naturally connect. Hand tracking produces a noisy time series of coordinates; handwriting recognition expects an image in the distribution it was trained on. The interesting work is the conversion between them.
Approach
Took an existing open-source project and worked on top of it, extending the pipeline and presenting the result academically. The architecture combines a pre-trained hand landmark model with a handwriting recogniser trained on a standard academic dataset augmented with synthetic data, since off-the-shelf recognition is trained on scanned documents rather than thin single-stroke characters drawn in the air. The middle step is the hard one: a fingertip trajectory has to be smoothed against frame-to-frame jitter, resampled for frame-rate independence, and rendered and normalised into the canvas the recogniser expects.
Outcome
A real-time pipeline from webcam to recognised text, with reported accuracy on both training and validation splits rather than a single headline number.
How it's built
Two models in sequence: a pre-trained landmark detector for hand tracking, a purpose-trained recogniser for characters. Everything runs locally, so the camera feed never leaves the machine.
Use a pre-trained model for tracking, train the recogniser
Hand landmark detection is solved and freely available. Recognising air-drawn strokes is not, because the input distribution is unlike any scanned handwriting dataset. Buy what is solved, build what is novel.
A dependency on someone else model for the first stage, which is the correct trade against months spent reproducing it.
Augment a real academic dataset with synthetic data
Real handwriting datasets are pen-on-tablet: steadier, with pen-up and pen-down signals air-writing does not have. Synthetic augmentation covers the gap the real data under-represents.
Synthetic data can encode its own assumptions, which the validation split is there to expose.
Adjacent systems
Cipher Pipeline Platform
Chain cipher steps into a pipeline, encrypt or decrypt text, and review every intermediate stage. A cryptography course project built as a real application.