Islamic State terrorist propaganda is going viral on Facebook

On August 10, 2019, a woman on Facebook posted a status warning her contacts not to accept friend requests from her sister Charlotte, whose account had been hacked She shared a screenshot of the account: an image of a blonde-haired boy dressed in desert camouflage and gripping a pistol with both his hands stared out from the screen. The image was captioned, in Arabic, “Luqmen Ben Tachafin. He does not tire. He does not bore”.

Over the coming months, the account went silent, and the incident became just another story of social media hacking. That changed in March 2020, when Facebook accounts whose bios read “Luqmen Ben Tachafin. I shake your throne and destroy your dreams. Never tired, never bored, until the Judgement Day” began sharing Isis content across the platform. In a span of 87 days, 90 of them would appear – all with the same signature images and bios. Luqmen, the “destroyer of dreams”.

The accounts, were part of a small army of Isis supporters on Facebook who called themselves the Fuouaris Upload network – after Fuorusiyya, the practice of equestrian fighting popular in 14th Century Islam. I first came across the Fuouaris Upload network by following a daily smattering of pro-Isis Twitter accounts flooding hashtags trending in Saudi Arabia with videos of beheadings, photographs of attacks, and links to cloud-based storage drives containing thousands of pieces of terrorist propaganda.

For the past ten months, following the killing of Abu Bakr al Baghdadi, Isis’s leader and chief ideologue, these Twitter accounts, exhibiting bot-like behaviour, have appeared day in and day out in an attempt to make Isis content trend on the platform. The Twitter accounts rarely survive more than a day before being taken down.

A little past midnight on April 7, one of those accounts shared links to a Facebook Watch Party hosted by an account going by Miro Williams. The Watch Party – a feature enabling Facebook users to watch videos in real time with friends – featured an Isis video titled “The Grit of War – The Bloodshed of Mosul”, and linked to multiple other Isis accounts spread out across the social network. One of those accounts was Luqmen Ben Tachafin.

That account was at the centre of a network using new tactics to share terrorist content on Facebook, where it was viewed by tens of thousands of people. Despite Facebook saying that “99 percent of terrorist content” is removed before it is flagged by other users, Fuouaris Upload uses unsophisticated but effective methods to elude moderation and pollute the social network with terrorist propaganda.

From March to May 2020, the network of accounts used Facebook’s platform to spread beheading videos and other Isis videos – sharing altogether 50 pieces of content, which raked in more than 34,000 views. While the numbers might seem small in absolute terms, the persistence of the network counts as a failure on Facebook’s part – especially as almost all the content seeded through these accounts managed to survive for over a month. And while 70 per cent of the accounts under the control of Fuouaris Upload had been removed at the time of publication, 29 accounts are still active on the platform today. Some are sharing Isis content that has been watched thousands of times and reposted by other Isis-supporting accounts across Facebook – highlighting a worrying breach in the social network’s anti-terrorist defences.

Between April and early July, Luqmen would commandeer 90 Facebook accounts, to fan out more Isis content on a daily basis. Luqmen labelled the captured accounts, using jihadist parlance, as ghanima – war spoils, immediately converted into central nodes to share Isis propaganda. The person behind the account did not hide the fact that their presence on the platform was single-mindedly aimed to “shower you with wave after wave” of content, confident that it would survive on the platform long enough to reach hundreds of potential recruits.

While a majority of the accounts were seeding content in Arabic, the monitoring and tracking period revealed there were parallel networks operating in Albanian, Turkish, Somali, Ethiopian, and Indonesian languages.

Luqmen’s style of writing was peculiar: the account’s pro-Isis screeds featured commas randomly placed within Arabic words. This quirk recurred across the network of Fuouaris Upload accounts on Facebook, with the lone distinction that some stuck to commas, while others used full stops. While rudimentary, the method denoted a sound understanding of Facebook’s moderation and its shortcomings: the random punctuation was an attempt at “cloaking”– confusing the automated systems that scan posts for suspicious keywords, such as “Islamic State,” or “kill”, and which would interpret these posts as a series of unfinished words. The system was unable to make the leap that the human mind would do when reading these posts.

That was just one of the tricks the person behind the network deployed to spread their pro-Isis messages. For videos, the Fuouaris Upload network relied on another system, based on obfuscating the branding of Isis content just enough to save them from takedowns. A 52-minute-long video by the Yemen chapter of Isis – a vitriol-laden screed against rival terrorist group al-Qaeda and other islamist organisations, released on April 29 – shared through the Fuouaris Upload Facebook network was masked, to get past both automated and manual detection.

The Yemen video began with a 36-second introduction that included footage from a Quranic recital and excerpts from a 2014 British documentary titled An Insight into Isis. This 30-second gap was used as a ploy to stave off detection: the hope was that moderators, or software used for moderation, would spare the video since its beginning did not have the trappings of Isis content. Around the 36-second mark, the documentary film intro cuts into the official Isis video with its branding intact. Similar tactics to mask the content were used for all of the Fuouaris Upload network’s content.

In some instances, the network accounts created completely new content. Oftentimes these were audio speeches turned into Isis video montages. The Fuouaris Upload network account Youcef Ibrahim was one account posting such content. On April 28, the account shared a two-and-a-half hour-long audio message titled “rules for those who do not excommunicate unbelievers”, a two-part Arabic audio series delivered by Shaykh Abu Malik al Tamimi, a Saudi cleric turned Isis fighter, who was killed in Homs in 2015. While the content is not the atypical violent videos commonly-attributed to Isis, it does violate platform guidelines by honouring the legacy of a terrorist ideologue.

Despite its duration, the video did relatively well, accruing 29 shares. The video opened with on-screen text against a backdrop of shoddy graphics and title cards with the words “Shaykh al Muhajid Abu Malik al Tamimi, may God bless him.” This was followed by a screenshot image of al Tamimi that stayed in place for the full length of the video.

Houd Nouh, another Fuouaris Upload account, shared similar Isis audio content, but instead of a lesson delivered by an ideologue it was an hour-long program from Isis’s al Bayan radio station, which was bombed off the airwaves back in 2016.

The video shared by Houd Nouh began as an orange screen that faded into a retro windmill graphic, revealing a stationary Isis al Bayan logo that stayed on screen for the length of the video. The video, which dodged takedowns by either manual or automated detection, is remarkable as it is clearly branded as Isis content and uses the radio station’s well-known logo. The video also contains on-screen text that reads “broadcast by the Islamic State” in both English and Arabic.

The Fuouaris Upload network prides itself on sharing the lessons it learnt from its conflict with Facebook’s content moderation. Maybe the most crucial of these lessons is the hijacking and hoarding of auxiliary accounts. Hacking accounts has a relative advantage for Isis-supporters, as the accounts have not been previously flagged for sharing platform-violating content.

To hijack Facebook and WhatsApp accounts, Fuouaris Upload relies on Textnow and 2ndLine – mobile applications that allow users to obtain virtual United States and Canada phone numbers. American phone numbers are sometimes recycled – that is, quickly reassigned to a different user when the previous user disconnects or the number is left idle for too long.

The Fuouaris Upload network’s method depends on creating several virtual phone numbers on Textnow and 2ndLine, in hope that some of them will be associated with an existing Facebook account as a password-recovery option – as the number’s previous owner might have forgotten to change their settings after changing phone number. Fuouaris Upload’s operation preyed on that forgetfulness.

Luqmen created two different tutorial videos outlining how to use Textnow and 2ndLine to hijack Facebook accounts. Using a newly acquired virtual US or Canadian phone number from Textnow and 2ndLine, Luqmen would use the virtual number to try to login on Facebook, and then report a forgotten password. The system would then send a code to the virtual number, allowing Luqmen to reset the password and steal the account. A similar video tutorial was also produced and shared on Facebook by another Fuouaris Upload Facebook account, called Ömer Can.

One Facebook page called “The Techniques of the Digital Caliphate”, which had garnered 835 followers about a month after its creation on April 5, functioned as a library for simple video tutorials on hacking and hijacking of Facebook accounts. The two videos uploaded by the page were titled “Techniques for garnering a phone number for Facebook and Telegram”, and “Getting an American number for WhatsApp (Numero)”, which explained how to hack an account using another virtual phone app called Numero. Both videos were viewed over 1,800 times in six weeks. Textnow, 2ndLine, and Numero were contacted for comment but had not replied at the time of publication.

Even hijacked accounts, however, have a shelf-life. The Fuouaris Upload network and its friends and followers across Facebook, are not immune to takedowns. Between April 7 through July 3, Fuouaris Upload lost 61 of the 90 accounts it controlled. However, 29 accounts continue to elude moderation, and spread Isis content across Facebook.

Presented with a list of 288 pro-terror accounts, including the whole Fuouaris Upload network, a Facebook spokesperson says the social network had already removed 250 accounts; it took down the remaining accounts on June 7 following WIRED’s enquiry. “We have no tolerance for terrorist propaganda on our platform and remove content and accounts that violate our policy as soon as we identify them,” the spokesperson says. The spokesperson did not answer direct questions about how it moderates video content or whether it is taking countermeasures to prevent the hacking of accounts through virtual phone number apps.

Luqmen ben Tachafin was among the accounts eventually taken down over the course of the network’s 87-day-long spell of terrorist preaching. On July 3, an account with identical name and profile image reappeared – and quickly started sharing new propaganda.

Source: Wired