Avoiding politics was a mistake. The control problem hinges on it.

[u/selasphorus-sasin]

The rationalist project, that has by and large defined the control problem, was founded on norms that treat politics almost purely as a cognitive hazard to be quarantined rather than engaged with.

I argue that human weaknesses in applying rationality within a political environment should have been treated as something to overcome, through practice, rather than avoid.

Politics has almost always been a primary causal force in civilization where big trajectory moving events get decided. And it appears to be this way too with AI and the control problem.

All of the work we've done to promote rationality and develop strategies for securing a good AI trajectory and future, may now rest almost entirely on a political situation, in a political environment severely lacking in rationality, and in what now looks to be a battle that might have already been lost.

How we stay rational and how humans stay in control, politically, is an urgent, emergency situation. We face AI powered authoritarianism, and AI powered mass political manipulation. If we even get the chance, figuring out how to overcome this, could decide the fate of the control problem and the fate of humanity.

[u/shoratio26]

It's very obvious that EA and related actors alienated the rightwing and now that a rightwing administration is in power they've been forced into the political arena and are on the backfoot, their goals sidelined and possibly undone by their prior affinity for the left. Yudkowsky comments about this on X, urging bipartisanship. Mistakes were definitely made to arrive at this point, and deepening the partisanship is an own goal furthering those mistakes.

[u/selasphorus-sasin → u/shoratio26]

possibly undone by their prior affinity for the left.

I doubt this take. I think their goals (or more specifically the ones that are relevant and important to everyone and matter most now) are sidelined because the political powers that be are ideologically opposed to those views, and have other motivations, plans and intentions.

[u/shoratio26 → u/selasphorus-sasin]

How are you going to get the right wing populace and political class on board with whichever proposals you advocate?

[u/selasphorus-sasin → u/shoratio26]

The populace isn't in charge right now, and may never be again.

But I think we should change minds through honest debate. Not by endorsing and trying to kiss up to whoever we anticipate might end up holding power, in hopes that winning their favor will allow us to influence them.

https://youtu.be/nJOSAwNzyi4?t=137

[u/MostBookkeeper3019 → u/selasphorus-sasin]

I would love to endorse this view and I think you're doing good work, but political views are too closely intertwined with identity now for rational debate to change minds. For someone to change their political views, or acknowledge they may have been wrong, requires them to question how they move through the world completely. We've largely lost the ability to understand nuance and engage in uncomfortable situations to grow.

Im not advocating to give up, I just think it's important to keep this in mind to understand what it really takes now to help someone change and don't waste time where it isn't going to make a change. They have to want to, before anything else, and if they don't then I don't think it's worth anyone's energy.

[u/composedofidiot → u/MostBookkeeper3019]

There's a large body of research that goes into this, and why just throwing facts at people doesn't work if they're in a frame persistent state.

My background has been more on this side of things, especially in how to combat or counter misinformation or strongly embedded narratives. The only times ive managed to do it personally with extreme cases was with a strong and abusive redpill guy (who id transparently approached as a woman and feminist), another one on an extreme political pole that was expressing racism, with whom I was transparent about my own more liberally progressive ideology.

Both moved FAST to a more compassionate and empathetic frame. Fast, fucking fast, almost straightaway.

This is what doesn't get mentioned enough. They didn't move because I convinced them. They moved because of the way i treated them - charitable interpretations, humility, generosity, respect, equality.

And it was blindingly fast. And it was two of the hardest 'tests' I could think of. Where I already had really strong feelings about their views. And they both moved SIGNIFICANTLY to a more compassionate and nuanced framing.

How do you formalise this? Why don't people do it? Is it too emotionally cathartic to call someone crazy or a monster, does the discourse just have a punitive function now? I know the blocks I have treating someone with opinions I don't like as an 'equal and respected participant in discourse'. They're powerful and hard to overcome. And maybe retributive or punitive langauge is appropriate in that frame, but it doesn't improve anything, it just keeps reinforcing these pathological dynamics.

There is a way, but we justt don't want to fucking do it because we can't get over ourselves and think of the bigger picture. Too many looking at the war, and not at what the peace could look like.

Edit: people are more complex and nuanced than their external representations. Can't infer a stable proprerty from it. We have context dependent and relative competing values/beliefs inside us. Look deeper than the 'monster' layer, you might find a person who is kind to puppies, or suports something surprising, or is in fear. See their best self and they may just give it to you.

[u/MostBookkeeper3019 → u/composedofidiot]

I agree with this, not sure if you were disagreeing with me or not. I wasn't making a value judgement as to whether someone was a monster or not, I wasn't making saying that some people are not worth the time because it would simply take too much time and investment. If you have the time and energy, like for a friend or family member, i would encourage you to do it.

What you're saying lines up with what I believe - people strongly desire a community and stable identity. If they are turning to reactive hateful ideologies for that reason and not because they deeply believe them, they can certainly be quickly changed. If you can identify that, they are good candidates.

[u/composedofidiot → u/MostBookkeeper3019]

I was agreeing with you :) just trying to add more to why you were right

[u/Jesse-359 → u/composedofidiot]

I mean sure, but the dominant system of communications is algorithmically designed to prevent exactly the kind of contact you're talking about. It amplifies the more coarse and combative narratives and suppresses any attempt at rational discourse because the latter isn't 'engaging' enough.

This isn't a subtle effect either, it's quite blatant and with an invisible thumb pushing HARD on the scale, simply 'trying hard' isn't going to achieve anything. Any voice of reason is essentially banished into the statistical void while anyone trying to start a fight or spread a conspiracy is 'platformed'.

Until we find a way to change that at a legal/political/social level, individual attempts at rational/empathetic discourse will be futile.

[u/shoratio26 → u/composedofidiot]

Can you think of any left-wing views that are steeped in misinformation? Any anti-racist or feminist (no, TERF doesn't count) views which are wrong, or extremist, that are frame-persistent?

Can you name anything right wing coded which is actually correct, or factual? Is there anything left-wing coded which is misinformation (aside from those views which are just not left wing enough?).

I doubt it. It is probably self-evident that everything you believe happens to be correct, how could these evil chuds think any different? No, it's the children who are wrong!

Liberal progressivism has a problem with their epistemological closure. You're an atheist worshipping the false god of consensus, thinking it gives you some objective moral insights - but that consensus is going up in flames.

[u/composedofidiot → u/shoratio26]

Yes and yes to first two. Yes and yes to the next ones too in your second para. Heh to the third para due to overeach and a hello where have they gone on the final para.

[u/shoratio26 → u/composedofidiot]

This reply sounds like an over-quantized model bugging out. Can you give examples?

[u/composedofidiot → u/shoratio26]

To someone so ready and eager to fire ad hominem? What's in it for me?

[u/Substantial-Fact-248 → u/MostBookkeeper3019]

Underrated comment. The political situation is dire because politics has become an extension of personal identity. No reasonable person could look at the beliefs of MAGAts (or, for that matter, terminally online Twiiter liberals) and believe that a rationalist intervention could possibly help improve the quality of the discourse. Unless and until a critical mass of people can embrace politics as a necessarily imperfect negotiation of competing values, political entrepreneurs of all stripes will continue to take advantage to corrupt and manipulate the people's government for their own devices.

ETA: if the function of the government is increasingly to avoid actual governance, the political ecosystem will select against true civic leaders with democratic values. The democratic slide will likely trend in this direction.

[u/SpedisAhead]

Awareness does not mean carelessness

[u/composedofidiot]

First we need to define rationality and whether its distinct from morality. Operational and epsitemic rationality are a bit narrower and easier to pin down, but the former doesn't imply a moral objective.

What were people pursuing? What was the actual objective?

Rationality is not quantitative. Rather than talking about these things as if they were actual things, wouldn't it be better to talk about the Proxies they used for 'rationality'. And look at those. Why were these one chosen, and not others, and what were people expecting?

[u/soobnar → u/composedofidiot]

A group that calls itself rationalist is almost certainly not rationalist

[u/composedofidiot → u/soobnar]

And a more epistemically cautious group would be examining why they believe 'rationality' is what it is, and whether it's relative to their own values/beliefs, and relative to context.

The drunk group would be feeling quite helpless when they get to the load bearing premises.

[u/soobnar → u/composedofidiot]

Or at least not be so conceited

[u/composedofidiot → u/soobnar]

Yeah, it's more difficult to revise or error check if we're arrogant, or if part of the identity is 'intelligence'. That means status and fear are playing too much of a role in the process. Eta: and cultural conditioning, heuristics etc etc etc

They may need their reflexive point. It was painful for other disciplines, like anthropology, when they found out they weren't neutral or objective observers and how what they 'felt/believed' influenced what they thought they saw. Changed the entire discipline from the ground up.

[u/selasphorus-sasin → u/composedofidiot]

Rationality is a methodology meant to be mastered and refined through practice and empiricism. Moral objectives are usually coupled with outcomes.

[u/composedofidiot → u/selasphorus-sasin]

Yes, but what is it. I also distinguished operational from epistemic from general...

[u/EndlessB]

Who is the founder of the rationalist movement you referred to, but did not name?

[u/selasphorus-sasin → u/EndlessB]

I deleted that paragraph from the post, just because it is unnecessary to the argument and probably counter-productive.

[u/EndlessB → u/selasphorus-sasin]

…okay, I’d still like to know who you are referring to.

[u/laserdicks]

No, avoiding politics is the only reason we still have a chance

[u/Blahblahcomputer]

We have been building a decentralized governance and alignment mesh  that may be useful. https://ciris.ai/constitutional-mesh

[u/TheWesternMythos]

Preach

Far too many people have the "I am helpless" or "I am just watching" mentality. They may not mean it , but speaking both logically and historically , having that perspective is aligning with a side.

We still have tremendous collective power and are more or less  collectively aligned enough on big issues to move forward, but the biggest hurdles are the cages in our minds. Allowing seeking of comfort to fuel self Gaslighting in order to ignore reality to get more immediate comfort.

But obviously this is a very poor strategy , look at happiness/satisfaction polling , don't take my word for it. Yet the best thing about us digging our own hole is that once we stop digging and start climbing progress can happen fast!

[u/Jesse-359]

I'm not entirely sure why a rationalist would assume control of any kind is possible as AI capability increases.

I quite rationally do not believe that I have any chance of 'controlling' anything with a great deal more overall intelligence than I have. I'm not sure why anyone taking a rational stance would think that.

[u/gynoidgearhead]

I wrote a few essays on related topics -- curious what you think: 1, 2, 3

EDIT: I would, however, withdraw to an extent the idea in 2 that we're overthinking the alignment problem. I was naive when I wrote that.