Writing without a screen: surface-free pens

For decades, capturing handwriting digitally meant writing on something special, a tablet, a touchscreen, a grid of dots. The dream is the opposite: a pen that records what you write on any scrap of paper. The obstacle has always been drift.

An inertial measurement unit, the same kind of chip that knows which way your phone is facing, reports acceleration and rotation many times a second. In principle you can integrate those readings to recover position: where the pen went, and how fast. In practice, tiny errors in each reading pile up. Integrate them twice to get position and the path doesn't just wander, it runs away, often within a second or two.

The traditional fix is classical sensor fusion, careful filters that fold gyroscope and accelerometer data together and try to cancel the drift. It helps, but it tends to be brittle: tuned for one kind of motion, limited to tracing a single short stroke before the error swamps the signal. A page of real handwriting, with dozens of strokes and lifts, is far beyond it.

Letting the data learn the mapping

A 2022 study in Sensors by Mohamad Wehbi and colleagues at FAU Erlangen-Nürnberg, working with the pen-maker STABILO, takes a different route. Rather than hand-designing the physics, they let a convolutional neural network learn the mapping directly, from raw IMU readings to the pen tip's relative displacement, how far and in which direction the tip moved from one moment to the next.

Two design choices make it practical. First, predicting relative displacement instead of absolute position keeps errors local, so a mistake in one moment doesn't poison the rest of the line. Second, the network reconstructs multiple strokes without needing the writing to be segmented first, and without hand-tuned correction afterward. To train it, the team recorded people writing on a tablet, which supplies the ground-truth trajectory, then aligned that against the pen's own sensor stream despite the two devices sampling at different rates.

The reported result, a normalised error of about 0.18 against the tablet's ground-truth trajectory, is the kind of number that signals a method has crossed from "interesting" to "usable." More importantly, the system reconstructs writing done on paper, not just on the tablet it learned from.

This is the bet underneath Vahini's pen: that you should be able to write the way you always have, on whatever paper is in front of you, and still capture every stroke. Learned reconstruction is what makes "surface-free" more than a slogan.

Summary and interpretation are our own. We link the original work so you can read it in full; we don't reproduce it.