Beware the pitfalls of allowing computers to do your advertising for you

Remember the kerfuffle about the company selling deeply offensive T-shirts on Amazon? They had horrible slogans such as "Keep calm and rape a lot" and "Keep calm and choke her". People were rightly upset that Amazon was selling these things and demanded that they be withdrawn. They were.

And then it became clear that this wasn’t just a story about an idiotic retailer. There was something else going on here, something we should all think about.

To start with, Amazon wasn’t selling these shirts,

it was just hosting/facilitating/whatever you want to call it a third-party store. Amazon had no idea these T-shirts were being sold. Second, the T-shirts didn’t actually exist. They would only have been printed if someone had ordered one. The real-looking photo on the site was just a mock-up. Third, no-one decided to write that "rape" slogan on a shirt and put it on sale. Instead, the store owner had built a software system – an algorithm – that took a huge combination of verbs and nouns, inserted them into the "Keep calm and do something" formula, mocked them up in a visual and put them on sale on Amazon.

We need to take responsibility for the content our algorithms generate. You’ll be running ads no-one has seen

Automatically. It doesn’t cost anything to create and offer these T-shirts, so why not create an infinite number of them? Thus, in the non-apology apology that got offered at the end of this, the retailer could honestly say he didn’t know about this design before it went on sale. No-one did. He had more than 500,000 items on sale on Amazon.

This obviously isn’t good enough. We need to take responsibility for the content our algorithms generate. We certainly need the brains to remove "rape" from our list of verbs. But we also need to attune ourselves to these realities. This is now how content gets made, how some words are written.

This is going to happen in advertising. You can easily imagine a scenario where someone would like to auto-generate a range of messages, aimed at different people, based on a simple formula. It’s probably already happening. That means you’ll be running ads that no-one has ever seen, let alone approved. Is that a conversation you’re ready to have with your clients?

Russell Davies is a creative director at Government Digital Services           
@undermanager

This article was first published on campaignlive.co.uk