The Human Cost of Moderating Facebook

The Verge’s Casey Newton has written an excellent, yet stomach-churning and heart-breaking article about the people in charge of moderating all of the filth (e.g., pornography, violent videos, conspiracy theories) that’s posted on Facebook, and the terrible stress and side-effects of their job.

For this portion of her education, Chloe will have to moderate a Facebook post in front of her fellow trainees. When it’s her turn, she walks to the front of the room, where a monitor displays a video that has been posted to the world’s largest social network. None of the trainees have seen it before, Chloe included. She presses play.

The video depicts a man being murdered. Someone is stabbing him, dozens of times, while he screams and begs for his life. Chloe’s job is to tell the room whether this post should be removed. She knows that section 13 of the Facebook community standards prohibits videos that depict the murder of one or more people. When Chloe explains this to the class, she hears her voice shaking.

Returning to her seat, Chloe feels an overpowering urge to sob. Another trainee has gone up to review the next post, but Chloe cannot concentrate. She leaves the room, and begins to cry so hard that she has trouble breathing.

No one tries to comfort her. This is the job she was hired to do. And for the 1,000 people like Chloe moderating content for Facebook at the Phoenix site, and for 15,000 content reviewers around the world, today is just another day at the office.

From the human cost that comes with fighting all of the crap that gets posted on Facebook to the company’s ongoing privacy breaches (and lame responses to them), it’s increasingly difficult to see Facebook as anything resembling a positive for society.

But I don’t know what the solution is. Obviously, on a personal level, you can close your Facebook account, but that won’t solve anything on a larger scale. Bad press doesn’t seem to affect Facebook, either; despite gross privacy abuses, people still want a place to post memes, videos, and photos of their kids for free.

But therein lies the problem, because nothing is free, not really. You may not have to pay for a Facebook account, but there’s still a cost involved, whether it’s your private information being gobbled up and sent to countless advertisers or the psychological and spiritual toll paid by those being exposed to the dregs of humanity in order to ensure that Facebook remains “safe” and “clean” for everyone else.

I’m no fan of government overreach, etc., but let’s just say that I wouldn’t shed any tears were the government to lean more heavily on Mark Zuckerberg and Co. (That is, of course, hoping that the government’s actions are led by people who understand technology.)