How did Facebook algorithms not protect my customers?

This week, my husband’s business has been subject to a scam which Facebook has the power to prevent.

“Hey, have you set up a new profile?”

That was the message that started it. Confusion. Realisation. Anger. Fear.

Facebook can be central to the success of small businesses. Building a reputation, engaging with customers, and selling online — it’s been the primary source of most of our revenue this year.

My husband bakes brownies and sells them online. We’d previously run a weekly market stall, but when the UK was locked down in early 2020 he spotted the opportunity to begin selling online.

Using his platform on Facebook, we’ve been able to sell out his bakes in just a few minutes every week. It enabled our reach to grow and soon we were progressing from hand-deliveries in the local community to postal shipments to every corner of the country.

His reputation, trust and authentic personality on social media was critical to that — so when those very attributes were brought into question this week, it raised alarm and fear.

“Is this you? I’m confused?”

On Tuesday 3rd November 2020, we were first alerted to a new Facebook profile that was mimicking our business page.

It had been set up with the same name, same profile picture of our new logo, and had shared all our recent posts. And it was followed by hundreds of fake profiles.

It was super-convincing. The only real giveaway that it wasn’t genuine was the fact that it was set up as a profile, not a page.

Initially, we assumed it was a bot so reported it and moved on, assuming Facebook would remove it with no harm done.

“I’ve just received a message from you”

The following day, we were inundated by dozens of messages from confused customers asking about a message we had sent out.

Except we hadn’t.

This fake profile had started adding all our customers as friends, and then messaging them to say they’d won our recent competition.

All they had to do was click a link and fill out a form.

Some found this suspicious. The message didn’t have Aaron’s usual friendly tone, and didn’t link out to his own website. But others had fallen for it, and clicked through to claim their free box of goodies.

“Damn, I thought I was gonna get cake!!!”

How is this happening? We had both reported it the day before.

But then we got notifications from Facebook saying they’d reviewed the account and it was all fine. There was no option to reply or appeal. That was it.

At this point, Aaron was starting to get very worried. His reputation was built online, but it could easily be broken there too. He’s built his brand on trust and authenticity, but this was eroding both of them. Will people now think twice about engaging with his brand in case it’s the imposter? Impressions like that can be long-lasting.

Messages and comments from people who had fallen victim to the scam started to increase. Customers were telling us how excited they were, and how much they had been looking forward to receiving their prize. One said it was her birthday and was looking forward to birthday brownies.

As soon as I could, I threw together a statement explaining what was happening, what we’re doing, and how customers can protect themselves. I posted it on our website and shared it across social media.

We messaged everybody we could, asking them to report the account. They all did. And they all got the same reply as we did.

Facebook thinks it’s fine.

“It doesn’t go against our community standards”

As a digital marketing professional, I really don’t understand how Facebook’s platform can allow this to happen.

First, I would expect them to have algorithms to detect suspicious activity like this.

Remember that this profile was set up to mirror our existing page as closely as possible. How does Facebook — with all its data power, and increasingly-complex algorithms, and growing machine learning capabilities — not have safeguards in place to identify this pattern?

Let’s list what happened:

• A brand new Facebook profile was set up
• It was instantly followed by hundreds of bot-generated profiles
• It used the exact same name as an existing Facebook page
• It used the exact same profile image as that existing Facebook page
• It shared every recent post from that existing Facebook page
• It sent friend requests to all users who followed that existing Facebook page
• It send bit.ly links to all of its new friends

That, surely is a pattern that Facebook should be able to detect? I’d expect that to happen automatically — but if not, it should at least kick in after being reported.

Facebook can recognise my face in photos uploaded by my friends, but can’t work out that a mirrored profile is a bit sus? Facebook’s documentation on its facial recognition technology even cites security as a priority:

We may be able to detect if you appear in someone else’s profile picture and send you a notification. If you review the profile picture and think someone is impersonating you, you can report that profile.

Of course, this is a little different from that. There’s no face, just a logo, and there are other factors at play. But this statement declares that Facebook is actively looking for impersonations — so why was it unable to spot something as obvious as this? I would assume all the signals above should make this much easier to spot than similar faces in photos. It feels like a massive oversight and lack of care from Facebook.

And by declining to take action when reported, Facebook is also going against its own policies which state that a Facebook profile must be for a person, not a business:

If you want to represent your business, organization, brand or product on Facebook, you can use your personal account to create and manage a Page. Keep in mind that a personal profile is for non-commercial use and represents an individual person.

“Hi, I’ve just received a friend request so it’s still going on”

On Thursday — 2 days after this started, 2 days after we first reported it to Facebook, and a day after dozens of customers did the same — we received a message to inform us that this was still happening.

I’m really grateful that so many of our customers have been able to spot this and keep us updated. But also really concerned about those that were taken in. And worried about the impact on our reputation.

We’ve already lost revenue this week because of it. The time it’s taken for damage-control means we only have very little stock available for this week’s sales.

I feel out of options right now. The profile has been reported by loads of customers, and speaking to anyone at Facebook is almost impossible.

And the longer this goes on, the more damage is done to our brand, reputation, and our future growth ambitions.

Facebook’s power comes with certain responsibilities. To protect its users. And to protect businesses which rely on it. And our experience this week suggests it’s failing on both accounts.

If you enjoyed this, please consider buying me a coffee.