Now, when we really are living in interesting times in the sense this phrase has found meaning in modern culture, this has become a particular problem.
For the avoidance of doubt, I don't mean it's a problem to believe Billy said something, but there's a real problem with believing things because of who said them, and the tide of disinformation currently racking up a burgeoning death toll is one that has us asking important questions.
Many are asking "how do I bring my (insert relative/friend here) back?"
This is not new territory for me. I've spent most of the last tumpty-tum years looking hard at beliefs and dogma and how they arise, better to understand how we think about things. In that time, some have said my writing played some part of them shedding their previously-held doctrines, so what's the secret sauce? How do I get back the person X was before they got into this?
The short answer is this: you don't.
This isn't necessarily a bad thing, so let's be careful about what we mean here.
The first thing to note is that it wasn't anything I said or did. That seems to contradict the testimony already given, but it doesn't, because I didn't do any of the work. That's entirely done by the person shedding the dogma. I, and you, can only show the path. We can't make the decision to walk it. Still, there are bigger fish to fry here, so let's get to those.
Experiences like this are impactful; People who come out the other side of such experiences are not the people who went in. Not in the sense that no man crosses the same river twice, as the saying goes, but in the sense that trauma - and this really is trauma - changes you; changes the way you think and the way you see the world. In particular, it robs you of innocence, and that's always profound. It can make you grow, it can make you shrink, it can change you in all sorts of ways. These are formative experiences, in other words.
In fact, falling into these rabbit-holes is very much like addiction. IS addiction. And just like any kind of addiction, finding your way out requires first recognising there's a problem, and that's where the rubber really meets the road for our purposes. The experience of realising you're an addict is deep. In this particular setting, realising how easily they got inside your head can be quite frightening. It's not uncommon for people to feel violated and manipulated. Which is in many cases what happened, of course. We'll explore how they did this as we proceed, because it's a lot easier than you might think.
Let me say this up front, because it's important: in studying this, I've been down some rabbit-holes and, despite being fully aware of all the dangers I'm talking about here, have still managed to find myself being sucked into modes of thinking that are really quite destructive of thought. This isn't something that's easy to protect yourself against, for reasons propagandists and apologists are fully aware of, and know how to play to full effect.
All that said, there are some strategies that can be effective in bringing somebody back from such thinking, but it requires patience and understanding. It's easy to provide somebody with factually correct information to counter their beliefs; it's much harder to ensure impact.
One of the deepest issues is simply how protectionist we are of our beliefs. Our beliefs are the bricks in the walls that shield us in some respects, but they also shield us from viewing countering evidence. Fantasy author David Zindell once described beliefs as the 'eyelids of the mind', which is an apt metaphor; once we believe something to any real degree, we close our minds to the possibility that the belief is incorrect. We'll look at this in more detail shortly, but first I want to talk a bit about the nature of ideas and, in particular, some features of ideas not readily apparent, because understanding how ideas are constructed and grouped gives insight into what would be necessary to deconstruct them. The most important feature of ideas for our purposes here is something we're going to call 'mass'.
Mass is exactly what you think it is. Indeed, we're going to be talking about physical forces a lot here because, as we'll see, it's almost the perfect metaphor for ideas. In physics, mass is a quantitative expression of an object's inertia, or 'resistance to acceleration', where acceleration refers to any change in velocity (speeding up, slowing down, changing direction).
Ideas have this property. Certain ideas can have a large amount of mass and be very attractive. Individual ideas generally don't have large mass, but they can, and they get passed around, and become 'memes', in sensu Dawkins, who coined the term.
The ideas with the largest mass are the really big ones, the ones that speak to momentous or historic events (tyranny, liberty), and especially those that have some intrigue in them (conspiracy), because who doesn't like a tale of skullduggery?
Here's the really important feature of the mass of ideas, though; it's cumulative. Once we accept an idea as true, whether it is or not, it attains some mass. For a single idea in isolation, it's usually trivial to shift belief by the simple application of countering evidence of greater mass (our internalisations of evidence are ideas, after all). Most are very accepting of countering evidence early on, but as consilient ideas - memes - accumulate, becoming complexes of memes, or memeplexes, the mass accumulates and the central idea becomes harder and harder to knock down. You end up employing that trick so beloved of creationists, picking away at minute bits of the edifice while ignoring the meat; the edifice itself. You can show any one individual belief in the edifice of belief to be false, and the believer can even shed the belief with almost no impact on the edifice at all. Getting to the core idea, then, can easily seem a Sisyphean endeavour.
I've termed this 'cognitive inertia', and it's incredibly difficult to shift. It's very much akin to the collective scientific inertia exposed by Kuhn, in which it requires the deaths of the dogmatic old guard to allow the revolutionary new ideas to come to the fore. In some ways, once a certain mass of ideas is attained, it behaves more like it's operating under magnetism than gravity, with attractive and repulsive elements to its behaviour*. There's a wonderful post by former creationist Glenn Morton who suggested how this might work by analogy with Maxwell's Demon, a thought experiment in thermodynamics.
Morton's demon was a demon who sat at the gate of my sensory input apparatus and if and when he saw supportive evidence coming in, he opened the gate. But if he saw contradictory data coming in, he closed the gate. In this way, the demon allowed me to believe that I was right and to avoid any nasty contradictory data. Fortunately, I eventually realized that the demon was there and began to open the gate when he wasn't looking.
So how does all this begin?
Let's start with Youtube, because it's a huge player in this market.
A video pops up in your recommended viewing. How it arrived there is complicated under the hood, but it's all driven by fairly straightforward processes controlled by algorithms. An algorithm is a set of instructions to carry out a task. In the case of how your feed is generated, there's a complex set of variables, but their basic principles are simple enough; a popular video with some 'tags' that match content you've seen before, for example, might pop up. That one alone can throw up some strange results, especially if videos aren't well tagged. We might watch some physics videos that, among their tags, have the word 'mass'. Generally speaking, you'll get mostly physics videos but, if somebody's mucked their tags up, by typing 'wave' in their tags instead of 'nave', you get a video about church architecture popping up in your feed seemingly at random, despite never having looked at such a video before.
But the thumbnail looks interesting, and suddenly you're hearing about the great nave at Norwich cathedral, and learning about the roof bosses and Walter Lyhart†. When you return to your feed, you suddenly see smatterings of videos about ecclesiastical architecture, or other videos by the same presenter. And, of course, the most popular videos get the most airing based on these and other algorithms. It's worth flagging those two concepts now; presenter and popularity. Both of these become important as we delve into disinformation.
For the most part, the process is fairly benign. This is trivially because there's not a whole lot of excitement, intrigue and controversy in topics like the minutiae of ecclesiastical and architectural symbology (though you wouldn't believe the carpet-biting kerfuffle this stuff caused in the mid-to-late 1600s), but some topics by their nature - child safety, for example - generate extreme depth of feeling; they have the potential to radicalise, and that's where the cooling device suffers a coprolitic barrage. When you start to accumulate a history of watching videos tagged with such topics, it quickly becomes your entire window on the world. You have embodied the bell jar.
Your feed is filled with the most popular videos on the topics and, because of the ubiquitous influence of the cult of celebrity, we find that it's filled with the most popular commentators on the subject. You get used to hearing them. You begin to trust them. You become familiar with their speech patterns. Some of the things they say are so succinct, so compelling, they can't not be true. They become quotable. Their quotes become slogans.
The names in the comments become familiar as well, and you begin to interact with them, backslapping each other for particularly witty takedowns, sharing your favourite quotes, further sloganising them. You might catch a livestream, and then it becomes a different level of involvement, because you can interact with the celebrities directly, as well as the familiar names in the chat. If a lot of people are tagging you because they recognise you from your comments, it's a good bet the host will spot you and mention you by name. That gives you the impression of being in the inner circle. The channel host might even make you a chat moderator if they like the cut of your jib, and then you have an actual position, and your name is highlighted in blue and you have power over other users in the chat. You become tribal, and you end up entirely occupying the thought-space of your tribe. Before you know it, you live in an echo chamber, wherein most of your sensory input is filtered. All your experiences are lensed through this view and nothing, even your personal interactions with those closest to you, is not coloured by it. It really can be quite intoxicating, and therein lies the danger.
Immediately we see the beginnings of what we'd rightly recognise as cult behaviour, and it's a cult we indoctrinated ourselves into, by the simple expedient of self-propagandising by algorithm. You're not likely to fall down a rabbit-hole into extremism via watching videos about church symbology, but there are some things that engender extremely strong reactions.
The most potent subjects for this are those involving our children; all ideas involving children have large mass. Because of course they do. It's the most foundational principle of our chosen evolutionary strategy, a strategy largely shared among social species, but massively protracted in humans because of the requirement to give birth while the skull is sufficiently small and plastic, leading to extremely long post-birth development.
The reasons for the effectiveness of such tactics are complex, rooted partially in that protracted post-birth development - particularly an evolutionarily advantageous gullibility as children - but also because the emotions associated with them are located almost at the opposite end of the brain from our social judgement. When we appeal to fear, we're triggering the amygdala, much closer to the brain stem, home of our lizard brain, where our 'fight or flight' responses are, our deepest primal instincts, quick-firing and primed for crisis. This in contrast to the cerebral cortex, where our higher level judgement resides, judgement of the sort that can override the more primal behaviours.
One of the manifestations of these behaviours is causing a particular problem at the moment, though it isn't a new problem. I mention vaccines here only because they're both of particular moment now and also illustrative of something that drives us to think first of our children.
I won't relitigate the history of the anti-vaccine movement here, as it's been amply covered elsewhere, but it begins, of course, with children. It needn't remain there because, once the seed of fear is planted in fertile ground, it grows, and it attains mass, and its influence extends beyond its foundations and infects other areas of your thinking. One of the rare instance of an actually true truism is this; a lie is halfway around the world before the truth has its boots one. Certain ideas, because of their colossal mass, are almost impossible to stop once they get up to speed. When ideas of threats to the safety of our children are invoked, they accelerate to full velocity extremely rapidly. When Wakefield's fraudulent paper was published in The Lancet, he triggered a lie we're still trying to undo the damage of 23 years later during a global pandemic that, at the time of this writing, has claimed 4.4 million lives globally. Very many of the people currently refusing the vaccine can have their entire line of reasoning tracked back to that single fraudulent and shockingly-designed paper.
It's trivial to see how easy it is to self-propagandise, but there are some tools in the propagandist's arsenal that warrant special attention.
The first of these goes back to the notion of having a familiar host. What often makes somebody popular in general terms is the way they express ideas. That can be accidental but, in general, even then, there will be certain things about they way they express ideas that make them stick. This is propaganda central.
The first tool of the propagandist is the slogan. Slogans are extremely powerful, because they can encapsulate really enormous ideas in just a few memorable words. Speech writers and orators have known this for centuries, and work hard to punctuate their writings with rhetorical devices in exactly the same way I do and for the same reasons. Lest we forget, propaganda isn't a bad thing, it's simply material advocating for a position. All journalism is propaganda, even the unbiased stuff, because it's advocating for a position; the truth.
We use devices like alliteration, rhetorical triplets, rhymes, ebb and flow in metre, even occasionally throwing back to archaic language like "lest we forget" to imbue our thoughts with gravitas and to engender a sense of the lyrical or numinous in our readers. We use loan words and phrases to connect to the great orators of the past. We combine them in a short, sharp shock (sorry). And that's on the page.
When speaking publicly, we have other devices and, more importantly, entirely other strategies. If we can see a particular bit of our message losing the enthusiasm of the crowd, for example, we can divert into something completely tangential with known support to lift them back up and then segue back into it, making the thing we were tepid about feel like the thing we were enthusiastic about all along. We know how to work a crowd.
One of the things about working a crowd which may not be immediately obvious is how we respond to people who are good with crowds. In fact, a lot of that is simply about confidence. My professional background is as a performer, so I've developed a certain set of skills. I know, however, that many to most people really struggle with being the focus of attention in public in any measure, and the particular admiration we have for people who can speak confidently in front of strangers is just another smuggled bit of self-propaganda. You advocate internally for the speaker just on this basis. Any move toward sympathy with a speaker is self-propaganda.
This all begins to coalesce when we start to combine some of what's been discussed here. The popular voice whose most common expressions become familiar ways of thinking, the repetition of simple slogans, the sense of belonging and comradeship...
One of the more pernicious tools in the toolbox of the propagandist is something know as "agitprop", or "agitation propaganda". This takes many forms, the most direct having been made apparent in the several 'outings' of police personnel being directly involved in the escalation of the 2020 demonstrations. I was somewhat embedded in those demonstrations, having been virtually on the ground with many independent, syndicated and stringer journalists, in research for future material on how we think about authority, and I saw first hand how much of the violence was initiated by police without provocation. That doesn't matter, of course; when the mainstream media shows violence and fires and tell you it was protesters who did it, that's the story. The TV news rabbit-hole is, in most senses, even more problematic than any other news resource, not least because it's more insidious. It's a given that what you see is only what they want you to see, but they have the advantage of a degree of trust - self- propaganda again.
This notion of an agent provocateur, as it's known, an agent of the opposition within your own ranks only there to provoke the worst elements of your cohort to bring you within the influence of the law, is a well-worn tactic of law-enforcement. Actually placing an agent, though, isn't the worst. The worst damage is often done by words.
There are ways of weaponising language that aren't always obvious. A very clear example of this, once it's pointed out, is the commentary surrounding "taking the knee". This goes through several stages, so you have to look to unpick it, but it starts with a spurious connection between some of the founders of the US organisation "Black Lives Matter" and Marxism. In fact, the connection is so tenuous it hardly warrants mention. The sum total of it is that organisers had found some useful tips about organising and messaging in Bolshevik texts. Suddenly, taking the knee, which never had anything to do with this organisation or even the phrase "black lives matter" and in fact preceded both by decades, was associated with Marxism.
The most deeply ironic thing in that sad little episode is that this tactic, weaponising the language of your opposition by recasting it as the language of your enemy, is straight out of the Bolshevik playbook. In short, the weaponisers are using a tactic they're critiquing to critique it. Dafuq?
There's a particularly insidious for of this tactic known as 'dog-whistling', that's really come to the fore in recent times. It's always been there, but it's become almost the primary mode of political discourse in some circles‡. Dog whistles are really insidious, because they're perfectly reasonable sounding statements, but they carry hidden meaning "for those with ears to hear". The name, of course, comes from this notion that only those in on the secret can grasp the true meaning of what's being said.
An example of a dog whistle is 'all lives matter'. Seems OK on the surface, and you could easily see it being uttered by perfectly reasonable people. However, when its's erected in the very specific context it has been, it has a pernicious hidden meaning, because it's trying to control the narrative. I'm not going to go any deeper on here, but see The Supremacy of History (long read), where I explore this dog whistle in detail, along with others and more propaganda tactics in a very specific context.
So what we've covered here, while not exhaustive, is just a cursory overview of some of the things influencing our descent into areas of the arcane. Before I move on to mitigations, there's one more feature of ideas that both highlights a difficulty and a potential strategy, and it's simply this: ideas - and complexes of ideas - evolve.
This fact alone raises some interesting problems, many of which are probably reasonably obvious, but there's one feature of evolving systems that throws a bit of a curve to results; stochasticity.
A stochastic system is a system in which future states are determined by initial conditions plus the outcome of one or more random variables. 'Random' here means 'statistically independent' rather than 'uncaused'. The term 'stochastic' has found increasing use of late outside systems thinking, in the phrase 'stochastic terrorism', which is an incredibly important term for our purposes, because it's an amalgam of propaganda and violence. In essence, stochastic terrorism is where inflammatory or aggressive language is used to demonise a target, leading to the entirely predictable act of somebody carrying out violence on members of the target group. The connection is obvious, because the inflammatory language is the initial conditions and the 'lone wolf' or violent actor is the random (statistically independent) variable.
In biological evolution, there's a law, known as Dollo's law. Although its domain of applicability is very strictly evolutionary theory, it points us to something more broadly applicable to stochastic systems generally, especially stochastic systems composed of large numbers of variables. It states, in essence, that backwards evolution is not possible. The reason for this is that mutations are random, and for evolution to fully reverse would mean reversing every mutation in exact order and detail. The probability of such an event is so low that it's functionally impossible.
It is possible for populations to evolve forward into states that are more primitive††, such as blind cave fish losing their unused eyes because of economy and environment, but tracing exact ancestral pathways in reverse is somewhere in the region of a fully grown human walking through a wall. Possible in theory and inline with perfectly valid science, but with a probability so low that it isn't likely to happen at any time in the future history of the universe.
Our memeplexes evolve as well, and this gives us some pointers to some of the difficulties
pattern seeking
DARVO
biases
cast their aims in terms of yours - i.e., attaining freedom through acquiescence
* In fact, there's good reason to suppose gravity has both attractive and repulsive characteristics. There are certainly repulsive solutions to general relativity, and this is in fact on of the leading candidates for dark energy, the force that's causing the expansion of the universe to accelerate.
† Walter Lyhart was the architect who designed the nave at Norwich Cathedral. I didn't learn this from Youtube, but from my former English teacher who, at 4'9", was easily the largest man I ever knew, and who worked as a voluntary tour guide sat Norwich after retirement. He it was who had taught me about points of reference as aides memoire in teaching many years before. In this case, one of the roof bosses is a picture of a white hart lying down next to water - Walter Lyhart.
‡ It occurred to me some weeks ago that , here in the UK, we're pretty much being exclusively governed by dead cat** and dog whistle while avoiding the elephant in the room, a new veterinary form of government.
** A dead cat is a story thrown out to distract the public specifically to cover-up some political chicanery. The use of such tactics is referred to by many names, but the most common is 'burying the lead', which is a press term for hiding the really important news down behind an innocuous- or trivial-seeming headline.
†† In technical parlance, the term 'primitive' has a very specific definition, namely 'resembling ancestral form. I'm slightly misusing it here because, in fact, blind cave fish aren't really more primitive, because none of their ancestral forms had vestigial eyes and sockets.
