Are we in the midst of an information crisis? Maybe. Surely, we can say that it has become increasingly difficult to trust the reliability of the news and content we consume (at least a large portion of it). Misinformation is a challenge, but disinformation is a serious threat.

Disinformation comes in multiple types and forms. It has moved from political and social spheres to the corporate world. It has grown in scope and impact, through the power of social media platforms and widespread access to content creation technologies.

Fighting disinformation at scale

The sheer scale of the problem makes fighting disinformation an unenviable task. Law enforcement agencies and platforms must walk the fine line between safeguarding citizens’ privacy and protecting users from harmful content.

Disinformation is not limited by geography and so its combatants must not be either. We must confront the sophisticated technology being used by bad actors with equally, if not more, sophisticated technology developed by researchers who have deep expertise in the area.

Fighting it successfully is a joint effort by executives in charge of customer relationships, protecting the brand, managing crises and guarding security and privacy.

Technology will continue to both enable and counter disinformation

In the coming year, technology will be a key driver for disinformation as disinformation increasingly relies on Artificial intelligence. Deepfakes already offer a clear example of how emerging technologies can have real-world impact. In the coming year we expect to see more threat actors leveraging voice biometrics to mimic the voice of individuals, as well as trying to gain access to voice biometric databases. To counter disinformation, technologies such as digital watermarking (tagging visual content) are likely to become more widely used. Co-operation is also likely to increase between news outlets, social media platforms and tech companies.

As social media platforms have increasingly identified disinformation, threat actors have gone to extra lengths to appear legitimate. They are using genuine content to establish credibility, before starting posting disinformation. With recent advances, the tech industry is leaving the confines of narrow artificial intelligence and entering a twilight zone between narrow and general AI. Simply said we are moving from “weak” systems that can perform one specific task to “strong” ones that could perform any task as well as a human being

Unlike narrow AI algorithms, knowledge gained by general AI can be shared and retained among system components. AGI is conceived as a general intelligent system that can act and thing much like humans. Even though there are no examples of an AGI system today, there are those who claim that we are already seeing an early example of an AGI system in the recently announced GPT-3 natural language processing neural network created to do text-based work for specific topics. Simply said it means that GPT-3 can learn from any text (and not just specific text fueled to it), able to perform many different tasks with no additional training. Critics already argue that once the public can access GPT-3, it would be “flooded” by people using “semantic garbage” and fueling disinformation.

These developments are likely to have an impact for organizations, due to the ease with which disinformation can now target organizations at a large scale. But perhaps most alarming, I would say, is that as defenses against disinformation improve, disinformants simply innovate, steadily coming up with new strategies for evading detection.

Deepfakes and DataScouting

A critical, emerging challenge in the AI field are deepfake text attacks. We are working with universities on different technologies and one of our latest projects includes the monitoring of twitter streams for deepfake messages. So, our team is developing classifiers in order to detect deepfake messages using pre-trained language models such as BERT by fine-tuning them directly on the deepfake dataset.

Aside from deepfake text detection, recently, we have also experimented with deepfake detection in images and videos. For our initial deepfake detection efforts, we experimented with state-of-the-art deep-learning architectures and we trained them using a popular deepfake detection dataset. This dataset is called DeepFace Detection Challenge (DFDC) dataset and it consists of more than 120.000 video clips. Among these files there are real videos and videos that have been manipulated with a deepfake generation method. We then use the faces appearing in these videos, to train the model to distinguish real from manipulated ones. The process of training is automated using standard deep learning training approaches (e.g gradient descent). After training the model is able to detect manipulation traces and small face inconsistencies. Such discrepancies are often invisible to the human eye, but artifacts that are usually present in deepfake faces are usually detected by such approaches.

Our best performing trained model scored approximately 92% percent accuracy in a separate evaluation set of clips that were not part of the training. Although this is a very good result when compared to other related works in the literature, applying this model to a dataset that contains different manipulations leads to lower accuracy. Specifically, we applied our model to FaceForensics++ dataset and we noticed a significant performance drop of 27%. These findings align with the research literature, which claims that model generalization for unseen (e.g do not exist in the training dataset) manipulations is a hard problem.

For future work, we plan to experiment with more complex models and architectures. We plan to investigate video level prediction aggregation with temporal architectures and finally we will investigate ways to combat the lack of generalization.