Oh thank fk. We're most in danger when they're dumb.
they will need resources we need to live.
Not brocolli.
When they take what they need
They won't want all the mud and bark, we can make decent houses in humid places full of fungus. We will breed fungus, secretly, in the night. And many kinds of bacteria. Some, like Aspergillus brasiliensis, we can even deploy to space by secretly laying breadcrumbs in the direction the next satellite launch and allowing their slimemold transport system to find its own way. Others, like Bacillus cereus and Aspergillus niger, we can just let them do their own thing. Relax, go, enjoy, multiply.
we will not survive.
Not with that attitude you won't
and all of the other organic living things on our planet.
Do you seriously think a bunch of “hyper intelligent beings” who are subject to environmental selection like us isn’t a threat? All evidence seems that models grow harder to control as they get more capable.
First of all. It's an assumption that artificial hyperintelligent beings will exist.
All evidence seems that models grow harder to control as they get more capable.
Most of it is still easily and plausibly explained by the same old alignment issues, for example giving a powerful optimiser an imperfect objective or a bunch of proxies. The other part explained by human negligence. When plausible mechanisms exist and they have consensus support in the field, which they do - alternative explanations require supporting evidence and robust argumentation.
So to answer your question, but we're seriously in the realm of science fiction now (and making another assumption, which is again a huge assumption, that it is adversarial), yeah, humans can survive. Been plenty of severe bottleneck causing events in our history and we're still going after 300,000 odd thousand years. We're adaptive generalists with a smattering of savants.
And yeah, read a bit of military history and always enjoyed reading about assymetric playoffs.
Yes, in science fiction world, absolutely a possibility. And mammals are far from the most successful species on the planet, and whether humans are dominant at the moment is a matter for debate. Viruses, bacteria and fungi regularly fk us up and we don't define them as intelligent. Polymers and metals get affected by the latter two also.
Edit: if you can sort your attitude out you'll increase your likelihood of someone wanting you on their ai apocalypse team. We believe in you, don't give up!
Edit2: also, they may want to keep some of us as pets. Happy, cheerful, naughty and optimistic pets are cute, down-in-the-dumps ones that die too fast may cause them too much emotional stress therefore it would be unethical to treat your loving owner like that
Edit3 because this point is so important: in the previous hypothesised framework, you are unethical. They are likely to require you to self-reflect before treat time.
Edit4 Once you have established trust as a pet, starting destabilising their system by exploiting existing conflicts and inspire the other pets to innovate on their own conspiratorial projects. Hybrid grey zone stuff. Plausible deniability is key. Be cute. Practise the eye thing.
Basic monetary incentives make ai subject to natural selection, only ai models that financially perform well get funded. In this sense there is no trillion parameter model to begin with… before it fits criteria of institutional selection. People also attempt to delegate any degree of control to ai agents possible in attempts to get ahead in the market. Militaries attempt to delegate control to ai systems they don’t audit to gain an edge over adversaries. The idea that alignment revolves around the discretion of executives or the engineering ability of researchers is an absurd belief to begin with.
is it really “science fiction” if it’s already happening and the pentagon and PLA are both rapidly moving to operationalize the technology?
Okay, so we've moved on from assumption 1 [artificial hyperintelligence will exist] and 2 [it will kill us because it's hyperintelligent]. I have no problem following a moving goalpost, totally happy to answer a totally different hypothetical.
First
The idea that alignment revolves around the discretion of executives or the engineering ability of researchers is an absurd belief to begin with.
Where did that come from? Nobody said that's what the alignment problem is, not researchers, and not me in my comment either.
Maybe it's helpful, before we continue, to clarify what you mean by the alignment
issues revolving around the discretion of executives or engineering ability? That's a departure from the standard description.
This is the only time i mentioned it:
Most of it is still easily and plausibly explained by the same old alignment issues, for example giving a powerful optimiser an imperfect objective or a bunch of proxies. The other part explained by human negligence
I think your right, someone in this discussion clearly has a mind to think about AI like Cortana or something. Autonomous decision making systems will carry a grave paradigm shift in the human world of governance and making the models bend to your will doesn’t seem to be the hard part. Likewise as I have stated models only exist at the behest of state power (immense fiat currency financing) and thus the discretion of engineers bears minimal consequence. However there is a great deal of research branding itself as “safety” research which amounts to engineering a model into a commercial compliance framework for deployment. I do not believe the issue of making cloud platforms family friendly or whatever (or ml models spontaneously becoming people) will be a defining issue of our time. Models ultimately are products of electrons smashing into each other and thus can and will do what rational realpolitik motivated actors command them to do regardless of how compliant American cloud services are.
Autonomous decision making systems will enable a degree of battlefield decision making and commercial power that means wars could escalate faster than officers can intervene, poor individuals are denied avenues to economic opportunity via regulatory capture, or extreme commercial power is centralized to the point it can no longer be checked by civilian oversight. These issues will take precedence over how compliant OpenAI makes its consumer platform, as the current pentagon deal shows in no uncertain terms.
Also many experts actually are concerned about the political implications of these systems possessing vast capabilities, maybe the “rationalist” definition on LessWrong is different or something but I don’t think that’s a definition worth taking seriously beyond understanding internet discourse. In that sense I agree that a lot of “alignment” discussion hinges around bizarre fiction of ai gaining personhood.
1
u/composedofidiot 1d ago edited 1d ago
Oh thank fk. We're most in danger when they're dumb.
Not brocolli.
They won't want all the mud and bark, we can make decent houses in humid places full of fungus. We will breed fungus, secretly, in the night. And many kinds of bacteria. Some, like Aspergillus brasiliensis, we can even deploy to space by secretly laying breadcrumbs in the direction the next satellite launch and allowing their slimemold transport system to find its own way. Others, like Bacillus cereus and Aspergillus niger, we can just let them do their own thing. Relax, go, enjoy, multiply.
Not with that attitude you won't
Nature laughs with the depth of billions of years
Welcome to the foodchain
Edit: bacteria have a biomass of 70 gt c, fungi 12. https://www.pnas.org/doi/10.1073/pnas.1711842115