Aaron Hertzmann: the Artificial Intelligence Art Critic

Aaron Hertzmann, Rice CS PhD alumnus and principal research scientist for Adobe.When Aaron Hertzmann started programming his home computer as a kid, he didn’t consider it a future career move. Similarly, his love of drawing and art wasn’t likely to land him a dependable income.

The Rice University alumnus (B.A. ’96) said, “I took a lot of college classes that covered the classics, and engaged in a variety of courses. One time, in order to get out of writing a Humanities class book report, I became an art major. Throughout college, I was painting a lot and also programming, but I didn’t see them as connected. They were just two disciplines I found interesting.”

When well-meaning friends and mentors steered him toward computer graphics, he demurred. Hertzmann spent summers and winter breaks working for a cousin’s Silicon Valley startup and appreciated the experience the work provided. But he also realized he had little interest in a career that required him to spend most of his day writing code.

“Programming wasn’t my passion,” said Hertzmann, “but my options seemed limited at the time and I started thinking about grad school. A few months after graduating from Rice as a double major in Computer Science and Art and Art History, I drove from Houston to New Orleans for a conference called SIGGRAPH 96 and it opened my eyes.”

SIGGRAPH is the Association for Computing Machinery (ACM) special interest group on computer graphics and interactive techniques. Hertzmann said the New Orleans conference revealed the overlap between venue art and cutting edge technology.

“Academic researchers were exploring new technologies that would enable art [and artists to evolve in new directions and mediums]. I met people there and sat in on talks that shaped my own research focus. At SIGGRAPH 97, one of the talks was about images and artists’ brush strokes, and I remember thinking, ‘that is NOT how artists work at all.’

“I went back to grad school with that research idea: to work on code that better replicated how artists work. The following year, I gave my own SIGGRAPH talk about turning photos into images that appeared to be hand-painted, through the use of multiple layers and altering brush sizes and strokes by adjusting algorithms. That was my first research paper.”

Hertzmann had found his passion.

Since then, his career and research have focused on the computational modeling of visual phenomena, particularly related to how humans understand the world visually, through art, painting or drawing, and movement or dance motion in animation.

His projects have mapped a career trajectory through both industry and academia, including Pixar and the University of Washington, where he is still an affiliate professor. After spending a decade as a CS professor at the University of Toronto, Hertzmann moved back into industry. As a principal research scientist for Adobe, he spends the majority of his time immersed in a research lab, working on his own projects and mentoring other scientists.

“As with anyone, my favorite project is my most recent,” he said. “It also happens to be more easily understood by general audiences, and links my depth of knowledge on topics of specialized technology with lessons I learned as an undergrad or in different posts throughout my life.”

Hertzmann’s research – “Can Computers Create Art?”  – places him in the unusual role of an artificial intelligence (AI) art critic. His previous talks helped refine his delivery, demonstrating how art, images, and computations tell a story in an interesting way.

“But my previous research also involved writing code, trying equations and algorithms, explaining the research, then defending why it’s worth doing. My current challenges are in thinking critically and organizing ideas to be compelling, getting rid of distractions that aren’t relevant, and convincing my audience of my point,” he said.

Hence, the art critic perspective. Pan ahead to a different gallery to catch the critic preparing his next analysis: what makes neural network art interesting.

“How did and do the technical underpinnings of this new technology lead to this new art form?” he asked. “It’s a philosophical argument, one I explore in my next paper.

“Since I’ve been paying attention to what artists are doing all over the world, I’ve noticed how technology contributes to the ways they create new works, and it’s an ongoing process of improvement. The development of artwork is improved by new technologies. Then artists do new cool things with that technology, and then a few months later there is a new technology to try.”

He’s hit the sweet spot between research and funding and says his lab at Adobe is similar to an academic research lab.

Hertzmann said, “What I do is very much like the job of a professor. I get to work with Ph.D. students developing ideas and publishing them. I just don’t have to write grants.

“A good day at work is one when I have really productive meetings with colleagues, developing new ideas and applying them. Writing things down is also incredibly satisfying…solving problems, challenging my brain. That’s what is so great about the Adobe Research lab. That kind of work is valued.”