“It’s a fairly widespread issue that the government wants to get ahead of with this three-year project,” said Christye Sisson, the Ronald and Mabel Francis Professor and chair of the photographic sciences program in RIT’s School of Photographic Arts and Sciences. Sisson is RIT’s principal investigator for the project.

“The more mature digital imaging has become, the methods by which you can manipulate images become increasingly sophisticated,” she added. “The speed at which this is happening is outpacing the tools that people have for detection, so we’re trying to exhaust and analyze every possibility for how these images could potentially be manipulated and how that manipulation can be detected.”

Research teams include imaging experts from academia, industry and government agencies, including the National Institute of Standards and Technology (NIST). According to Sisson, RIT is creating high-provenance data and image manipulations as part of the project’s data team, led by PAR Government Systems Corp., a government contractor based in Rome, N.Y.

Sisson noted that applications of the research’s findings are widespread—from law enforcement and intelligence agencies to counter-terrorism and academic research integrity.

A growing problem

While some image manipulation is benign, performed for fun or artistic value, other “doctoring” is done for nefarious or adversarial purposes, including propaganda or misinformation campaigns, Sisson said.

In recent years, consumer imaging technology—primarily digital cameras and mobile phones—has become ubiquitous, enabling people the world over to take and share images and video instantaneously. Mirroring this rise in digital imagery, however, has been the ability for even relatively unskilled users to manipulate and distort visual media.

The manipulation of visual media is enabled by the wide-scale availability of sophisticated image- and video-editing applications that falsify images in ways that are extremely difficult to detect—either visually or with current image analysis and visual media forensics tools. Many of the forensic tools used today lack sophistication and address only some aspects of media authentication.

The MediFor project’s goal is to develop an “end-to-end platform” that will perform a complete and automated forensic analysis of imagery that does not exist today.

According to Sisson, RIT is creating the ground-truth data for the project by photographing a large variety of scenes and contexts with as many devices as possible to create a wide frame of reference for government officials. The RIT team uses these images to create manipulations based on real scenarios to test new algorithms of detection being developed by other partner universities on the MediFor project.

Along with these manipulations, Sisson said, the RIT team will create an “answer key” that will chronicle all actions made in a manipulation to see how well the algorithms perform in their evaluation.

RIT professors Ted Kinsman and Christopher Kanan are assisting Sisson on the project, along with several undergraduate students in the photo sciences, imaging science, motion picture science, and media arts and technology programs.

“RIT has this incredible and unparalled imaging continuum with our programs that you just don’t see anywhere else,” Sisson said. “We’re extremely well-suited for this project.”