AI That Mimics Human Handwriting — See If You Can Tell the Difference

A team of researchers at Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) in Abu Dhabi has come up with an innovative AI tool called Handwriting Transformers (HWT), designed to replicate an individual's unique handwriting style. 

The team consisted of Assistant Professor Rao Muhammad Anwer and Associate Professor Salman Khan, both specializing in Computer Vision. Additionally, Fahad Shahbaz Khan, a Professor of Computer Vision and the Deputy Department Chair, along with Ankan Kumar Bhunia, were also part of the group.

How did they make it happen?

Unlike conventional methods that use generative adversarial networks (GANs), the researchers opted for vision transformers, a specific type of neural network originally crafted for computer vision tasks, allowing the AI to grasp long-range dependencies within the text. This can capture both the global structure and nuanced local style patterns more effectively.

Even though GANs can replicate a writer's broad stylistic elements, like the slant of letters and the width of strokes, they struggle when it comes to accurately reproducing the unique details of how people form specific characters and the connecting lines, known as ligatures, that bind characters together.

The researchers presented their findings by comparing HWT with other handwriting generators, such as GANwriting, in a study where participants preferred HWT (81% preference). HWT can easily generate realistic handwriting by comprehensively analyzing both the overall context and intricate details of individual characters.

“To mimic someone’s handwriting style, we want to look at the whole text, and only then will we start to understand how the writer ligated characters, how the writer connected letters, or spaced words,” Fahad Khan said in a university blog post. “All these tasks require a kind of global receptive field, which is not easy using convolutional neural networks. We identified this gap in existing methods and adopted this transformer-based method.”

Beyond its potential benefits in helping individuals with impaired handwriting, the technology also serves as a valuable tool for generating large datasets to enhance the capabilities of machine learning models when processing handwritten scripts.

Although this research is just based on creating handwritten text in English, the next aim is to expand it to other languages,like Arabic, where the primary problem is the unique way its letters are connected in handwritten scripts.