AI is digitally pasting products into your favorite films and TV

New Scientist

18 November 2019

Ad blockers aren’t going to be useful for much longer. Major entertainment companies including Univision and 20th Century Fox are now using artificial intelligence to digitally insert advertisements and products into movies and TV shows after they have been filmed.

The firms are using technology developed by UK company Mirriad to insert flat posters on buildings, walls and buses in already-filmed scenes and even to add 3D objects. It has the potential to make advertising more targeted and ubiquitous than ever before, and also virtually impossible to avoid. The technology blurs the line between advertising and entertainment, and could take the feel of a production outside a director’s creative vision for it.

The technique has been used to insert ads into shows such as Modern Family, which is produced by 20th Century Fox. In the US, Mirriad has also worked with Univision and Sony Pictures Television so far. In Europe, Mirriad has partnerships with French broadcasters France Television and TF1, and with RTL in Germany. In China, Mirriad-inserted ads have been seen by more than 100 million viewers on video-streaming website Tencent Video.

Mirriad analyses films or TV episodes for space where ads or objects could be inserted, using an AI to identify information about each scene – whether it’s on a street, for example, or of a family. The AI creates an inventory of all the potential times and spaces for where ads could be inserted into a film or TV show.

“We can recognise all the scenes when people are in an elevator,” says Ann Wang at Tencent Video. It allows them to digitally insert advertising posters onto elevator walls. Other scenes include those shot in living rooms and office meeting rooms, which contain table space where bottles or cups of particular drinks brands can be inserted.

The goal is to train the algorithm to understand emotion, context and continuity across scenes, so that the process of ad insertion can be fully automated, says Mirriad CEO Stephan Beringer.

Mirriad and Tencent Video both have plans under way to tailor these in-video ads to individual viewers. Tencent has the technology to show different advertisements to different people, but hasn’t yet launched the feature commercially, says Wang. “Our goal is definitely to go in that direction,” she says.

Take a scenario in which two people are separately watching a television series. “You would be watching the exact same scene, but we would be seeing different things,” says Beringer – say, a silver or red car, or different variations of a brand of soft drink. Mirriad can produce several versions of each ad, and what the viewer sees would depend on how the entertainment company targets them.

“Consumers are not always aware that the commercial content is being blended in with the cultural content,” says Caroline Moraes at the University of Birmingham, UK. “There’s a blurring of lines.”

“If people are not aware, then you could argue that there is an element of deception there, or that people have not been fully informed of how they’re being communicated to,” she says.

Companies have long paid for product placement in entertainment. For example, Reese’s Pieces appear in the Steven Spielberg film E.T. and there are multiple brands featured in the film clip for Lady Gaga’s 2010 song “Telephone”. But in these examples, it is incorporated during filming.

But there are some specific concerns with ad-insertion technology. For example, actors may be seen to be endorsing items that appear in the scenes with their fictional characters, which they may have little say in because it happens after the fact.

To avoid conflict with pre-existing endorsement deals, Tencent uses technology that recognises well-known actors in its content, to identify scenes where ads shouldn’t be inserted.

Films may have a theatrical release, followed by distribution on streaming services and network television, as well as internationally. Each round could have different adverts, and whether the money goes towards the distributors or content creators is another consideration, says Beringer.

Creative integrity and continuity within a show or film are also factors. New ads inserted into older content retrospectively may not be in line with what a film or TV director wanted to do, for example.

There are strict rules around product placement, but these may need updating for this more sophisticated and potentially more targeted kind of digitally inserted advertising.

20th Century Fox and NBC Universal didn’t respond to New Scientist’s requests for comment.

See the article here.