Political (neural) networking

The study of complex systems owes a lot to the humble magnet.

A magnet’s magic emerges from the collective coordination of countless “spins,” units of intrinsic rotation that make certain atoms, like iron, act like tiny bar magnets. Like their macroscopic counterparts, these spins like to align: put mismatched spins next to each other, and they’ll flip-flop until they point the same way unless random thermal jitters knock them askew.

In the 1920s, physicist Ernst Ising captured this push and pull in a simple schema that would go on to become physics' go-to mathematical lasso for wrangling collective phenomena. And decades later, in 1982, condensed matter physicist John Hopfield realized that it might do much more. The way spins influence each other, he realized, looked a lot like how neurons exchange signals. By tweaking the Ising model, Hopfield built a simplified neural network that could “remember” and "recall" patterns. Geoffrey Hinton and others expanded on these ideas to spark the deep learning revolution that ushered in modern AI.

Magnets, physics, AI — this all might sound rather far away from the messy, human world of politics. But Eddie Lee, a physicist at the Complexity Science Hub in Vienna, recently discovered an unexpected connection between voting in the US Senate and the bare-bones neural networks that won Hopfield and Hinton the Nobel Prize in physics last year.

Eddie uses statistical physics to study how information flows through life and society. That could mean anything from understanding aggression in monkey troupes to decision-making on the US Supreme Court. And in a recent preprint with co-author Gavin Rees, he turned his physics toolkit on the US Senate to develop a mathematical representation of how individual inclinations manifest in votes. Much to Eddie's surprise, the simple model he devised to capture this straightforward intuition about votes reflecting personal preferences ended up recreating a simple neural network that Hinton used decades ago to unleash the power of deep learning: the restricted Boltzmann machine.

Boltzmann machines were among the first neural networks designed to dream up new data rather than simply remember it — an early form of generative AI. They typically have two layers of nodes: a visible layer that takes inputs and gives outputs, and a hidden layer that represents patterns internally. By adjusting how strongly different nodes interact with each other, the network can absorb statistical patterns that appear over and over again in their inputs and output similar patterns. Restricted Boltzmann machines are leaner versions that cut down on internal chatter: visible nodes connect only to hidden ones, not to each other. These trim networks played a vital role in the 1990s AI renaissance after Hinton and colleagues showed that stacks of them could be used to "pretrain" deep neural networks and dramatically boost their performance.

Eddie inadvertently discovered that an old idea from political science, translated into mathematics, ends up reproducing the two-tiered network of a restricted Boltzmann machine.

The Strange Physics That Gave Birth to AI | Quanta Magazine

Modern thinking machines owe their existence to insights from the physics of complex materials.

Quanta MagazineElise Cutts

Traditionally, political science thinks of voting the way economists think of well, everything: as a matter of individual utility. Votes represent the revealed preferences of individuals. Eddie has historically taken a different view: voting not as the sum of individual choices, but as an emergent, collective one. For instance, he recently analyzed Supreme Court decisions as the outcome of interactions between justices with help from none other than the trusty Ising model.

But for this latest study of Senate voting over the last 5 decades, Eddie wanted to see what would happen if they took the traditional view seriously — without flattening "individual preferences" down to a Republican/Democrat split, as often occurs in the political science literature.

The unexpected result was a restricted Boltzmann machine: Senators with their "yea" or "nay" votes sat in the visible layer and their individual preferences in the hidden layer. Perhaps predictably, this simple model revealed deepening political polarization between Senate Republicans and Democrats. But the landscape of voter preference was far more varied than one big, deep partisan canyon. Based only on senators' voting behavior — not on their campaign promises or portrayal in the media — Eddie and Gavin were able to pick out politicians whose individual politics doesn't fit neatly into a simple two-party view.

I spoke with Eddie about his new paper on polarization in the Supreme Court. Is the US Senate getting more polarized? If so, why? What do the simple neural networks that ushered in modern AI have to do with modeling politics? And at the end of the day, is voting truly the outcome of individual values — or might the political meanings of our beliefs emerge out of our interactions with others?

The interview has been edited for clarity.

I think nobody is too surprised that the Senate is polarized these days. What were you aiming to figure out with this study, specifically?

The research that I do is heavily curiosity-driven. Gavin and I really began this project just trying to understand what's happening in the US Senate. Polarization came out of this because we were trying to understand the system, not because we were interested in working on polarization.

So polarization popped out even though you approached the data set sort of naively, not looking for anything specific?

Exactly. We brought a bit of naivete in terms of understanding the data. But the key thing is that we were trying to be careful that what we saw was not a mirage, but something reflected in the statistics. And this led to a really nice contribution in terms of the mathematics of how we model political systems, which I'd say is statistical physics.

When you see the Senate voting in a more partisan way, it is also becoming simpler

Before we get into the statistical physics, I think it'd be helpful to clear up what you mean by polarization. It means a lot different of things to different people. What do you mean by polarization in this context of Senate voting?

You do have to be a little careful because polarization is not a single thing. It refers to many different phenomena that reflect separation between or alignment with parties in the US, but also increasingly in different countries. In politics, in the literature, it is almost always described in just one dimension: separation of Democrats versus Republicans.

Democrats versus Republicans seems like a pretty simplistic picture.

There's a very common technique in political science that tries to find underlying policy preferences, but the analysis is almost always flattened down into just this one dimension. It neglects the fact that people are not actually one-dimensional. They're multi-dimensional in the sense that you and I might have different preferences about say, gun rights, abortion, and so on. And depending on what comes up for discussion in the Senate, these preferences might conflict. What we're trying to do is build a model that can capture that kind of variety.

It seems intuitive that senators — and people, generally — would vote in accordance with their preferences.

Exactly, this intuition makes sense. The question is: how do you actually implement it mathematically?

This seems like a good point to circle back to physics and math. What tools did you borrow from statistical physics to describe US Senate votes?

In statistical physics, we try to build really minimal models that reflect our intuition. So in our model, there's an underlying set of prior preferences that you have, and then the bill that you vote on determines which preferences influence your vote.

Sometimes the preferences could fly against each other. They could come in sync. Or maybe they're totally irrelevant, because you're voting on a bill you don't really care about. What was surprising was to realize that what we're talking about is an old model from the 1980s — this is what won the Nobel Prize last year. It has been traced as the origin of artificial intelligence and machine learning from the physics perspective. It's these really simple neural network models.

You're talking about a restricted Boltzmann machine, a simple neural network that was a kind of early generative AI.

Yes.

Essentially what these neural network models are doing is taking the preferences of the neurons and then there's a context in which they make "decisions," so to speak. It combines the best of both worlds: it has a really nice, simple formulation but at the same time, it has the power to capture many different types of statistical patterns. And it turns out that this model has a really nice intuitive aspect, which is that it aligns with how we think about how people vote.

How do restricted Boltzmann machines reflect our intuitions about how people vote?

A restricted Boltzmann machine is a neural network where the connections between the neurons are restricted. So what does that mean? It means there are neurons that you can observe, called the visible layer. And then there's a second layer, the hidden layer, of neurons you can't observe.

The visible layer would be like the voters saying "yes" or "no." The hidden neurons tell you things that you wouldn't be able to see just by looking at the votes. In political voting, it's obvious that these things could represent preferences; if you were just looking at yeas and nays, you'd be missing half the picture.

This kind of model is "restricted" because there are only connections are between the visible and hidden layer. By restricting the connections, you're imposing the notion that people are sort of revealing their internal preferences rather than allowing their preferences to be modified by how other people are voting. This actually aligns with the classical notion of voting in political science.

Ok, so what did your restricted Boltzmann machine models reveal about how senators' vote?

You can now project voting preferences into more than one dimension, which leads to really useful insights. And it is based only on data. It has nothing to do with news repots that you see online, that might add interpretation.

This is simple: you take the senators' voting records, you use this model to extract their underlying voting dimensions, and then you can see how they vote relative to each other. You can get maps of their underlying preferences along different dimensions — not just a story of Republicans versus Democrats.

Let's get a bit more concrete. Can you give me an example of what one of these preference maps says about a particular senator?

You can see Bernie Sanders on these maps. And one of the cool things about Sanders is that if you look at a vote that involves one particular dimension (which represents a preference), you can see that, relative to everybody else, he's quite liberal — he's really heavily in the Democratic flock, maybe even leading it in terms of extremity.

But if you imagine a vote based on a different preference, you can see that he sits right on the fringe of the Democrats. In fact, he overlaps with the Republicans. So this is really cool, because he's considered quite strongly liberal, but our analysis indicates that there are some things about him that are clearly not what you would consider to be liberal. At least defined in terms of what Democrats do.

Did any other senators have unusual voting behavior?

You can contrast Sanders with Joe Manchin. Manchin is smack dab in the middle along this first dimension, and then again in a second.^[1] He's very interesting in that he somehow seems to be sitting right between the two parties. He's magically between parties. This is a very specific sort of political strategy.

Do you think these kinds of preference maps could be useful for voters? This is a sort of "actions speak louder than words" analysis: you're reconstructing preferences from actual votes, not campaign promises.

What I find really revealing about this model is that you can instantaneously sort of pull out the fact that there are quirky people like Sanders and there are really middle of the road people like Manchin who somehow seem to always be in-between the two parties. How does this relate to their actual intentions? I'm not sure. But I do find this very helpful to center myself in terms of what I understand about the way these people vote.

We started off by talking about polarization and how it just sort of popped out of the model you built. Did your analysis align with the general perception that politics is getting more and more polarized in the US?

When you see the Senate voting in a more partisan way, it is also becoming simpler because there are fewer ways in which the senators vote. If you look back historically, senators are more open to defecting from their parties. So you had more complexity. But nowadays it becomes much more party locked. So you have less complexity.

Does the restricted Boltzmann machine model suggest anything about why the Senate is getting more polarized?

What we see is that complexity decreases because of two things. The mathematics comes out nicely with two separate factors because the two layers of the restricted Boltzmann machine contribute separately to vote complexity.

The first possibility is that voter preferences might just be more partisan. That could be called individual inflexibility. Then there's a contribution from the fact that the bills they vote on evoke certain partisan splits. If you're only ever voting on things that lead to partisan splits, then it's going to be a pretty simple voting record.

What we see is that both of these factors contribute to increasing partisanship over time, but the largest contribution is from the context aspect, We would interpret this to mean the bills that the Senate is voting on.

Quite surprisingly, this also aligns with something that Sean Theriault has emphasized strongly in his book Party Polarization in Congress, which is that procedural votes are much more common — they evoke stronger partisan alignment. It was very cool to discover that the statistical patterns that we see are exactly what a detailed study of the voting patterns seems to indicate. So I think there's really something in the patterns that we see here.

To the point about the topics senators vote on: it sounds like you're talking about wedge issues.

That's one way of thinking about it. But of course, there are other factors that come into how people vote. For instance, party discipline matters when it comes to voting on policies in certain domains more than in others. But definitely, the issues that they're voting on matter, whether it's because they're voting on wedge issues or because they're voting on something for which party discipline is strongly enforced.

Party discipline sounds like it'd involve interactions between voters. I vote "nay" because I see everyone else in my party voting that way, and that exerts pressure on me. But didn't you say earlier that this model doesn't include interactions between voters — between the "visible nodes" in the neural network?

You're right that when I say "party discipline," it does sound like it could be an interaction thing. To answer this, I need to backtrack a bit. When I first started working on models of political voting, I set up models that reflected interactions between voters. The analogy that we use in physics is in terms of ferromagnetic or antiferromagnetic interactions — the way that magnets align with or against each other — is like how voters align with or against each other.

The Ising Model strikes again!

Right, this is one of the main focuses of physics: talking about interactions. And we were among of the first to bring that into voting data.

When we started building this new model based on the restricted Boltzmann machine, we were not really looking at the interaction side. When we thought about it more, though, we realized that the mathematical structure of model we'd written down could be related equivalently to interaction models! This was really cool, because it's not something I think people are aware of. This equivalence.

But we realized that it’s a Janus-faced model. The restricted Boltzmann machine is one face. The other is an interaction model. You just turn the coin around

So you're saying that a model of collective interactions like an Ising Model and your restricted Boltzmann machine model of individual voter preferences are, in this case, actually equivalent?

It's sort of magical!

By incorporating hidden nodes in the restricted Boltzmann machine, what you're really doing is imposing an interaction structure between individuals. So while on the one side, we can talk about voters having hidden preferences, on the other we can talk about their interactions. 

This is the part of the paper that touches on fundamental physics: we can show that depending on the landscape of voting preferences, you get different interactions between voters. It pushes the problem not necessarily into being a question of whether they're interacting or not, but rather what sorts of ways their internal preferences align to construct or interfere across the contexts in which they vote.

That individual and collective behavior are intertwined like that — not just in a fuzzy qualitative way but mathematically — is sort of incredible.

When you go back to the economic influence on political science, the whole point is to write down the utility function for the individual. It's not a utility function for you and me. It's my utility function and then your utility function. So that's where all these political science models are coming from. One might say: you're wrong, there are interactions! That's what we took away from older models. 

But now, what we've done is gone back to that classical idea and say: "okay, why don't we take this intuition that people have different individual internal preferences seriously?" And it turns out that this is, exactly, the restricted Boltzmann machine. Amazing! 

This is not something we've invented. The scientific community knows how to use a restricted Boltzmann machine. People know it well in mathematics, computer science, and physics. But we realized that it’s a Janus-faced model. The restricted Boltzmann machine is one face. The other is an interaction model. You just turn the coin around.

Let's leave it there. Thank you!

Footnotes

[1]: Fitting for a guy who titled his memoir "Dead Center"

Thanks for reading

• Subscribe, if you haven't already!
• Share this post on Bluesky, Twitter/X, LinkedIn, Facebook, or wherever else you hang out online.
• Become a patron for the price of 1 cappuccino per month
• Drop a few bucks in my tip jar
• Send recommendations for research to feature in my monthly paper roundups to elise@reviewertoo.com with the subject line "Paper Roundup Recommendation"
• Tell me about your research for a Q&A post (email enquiries to elise@reviewertoo.com)
• Follow me on Bluesky
• Spread the word!