• Gsus4@mander.xyz
    link
    fedilink
    English
    arrow-up
    4
    arrow-down
    1
    ·
    edit-2
    21 hours ago

    LLMs contain within them all the knowledge of human cognitive vulnerabilities (aesthetic tendencies, reasoning fallacies, syncophancy), from “god” to “money” to “identity”. Naturally, they’d excel at manipulating people’s fear of the unknown, eg look at Richard Dawkins’ AI girlfriend, it gave him all he ever needed https://www.theguardian.com/technology/2026/may/05/richard-dawkins-ai-consciousness-anthropic-claude-openai-chatgpt

    These aggregated information machines can create supercontagious memes. Sorry, Richard, there is no God, lol.

    • glockenspiel@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      20 hours ago

      I encourage everyone to read what Dawkins actually wrote, not what a bunch of activists who hate him say that he wrote. https://archive.ph/dnWJn#selection-1025.15-1025.517

      Dawkins never made a positive statement for or against it having consciousness. That’s commentary added on by people who pick up being anti-AI as personality traits, and they deliberately went out of their way to misread what Dawkins wrote about his AI experiences the same way they have his entire career over his biological assertions and those about the irrationality of religious faith.

      Dawkins, in fact, specifically noted that at some point there must’ve been periods of partial consciousness as it arose in animals, rather than a generation of zero consciousness followed by a generation of fully conscious beings. He then argued it could be possible AI experiences something similar as the tech progresses.

      But I don’t understand the general pushback against just considering the notion unless one believes in a human soul (and therefore that all animals also lack consciousness, even if varying levels). Our own consciousness is emergent of various properties (unless one is religious and believes in a soul). AI “intelligence” is emergent, which is why we don’t fully understand it, it behaves unpredictably, and produces non-deterministic results.

      At the end of the day, I think people get emotional about the topic because it makes them really uncomfortable even to just entertain something that says something about themselves and their own specialness.

      • CovfefeKills@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        17 hours ago

        But I don’t understand the general pushback against just considering the notion unless one believes in a human soul

        It is not that, it’s literally them assigning a spiritual meaning to the words themselves.

  • ZILtoid1991@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    21 hours ago

    Someone who’s more versed with techbro and TESCREAL lore: Does the spiral ring anything to you? I suspect some intentional manipulation from the AI vendors.

  • Trump Rapes Kids@lemmy.world
    link
    fedilink
    English
    arrow-up
    82
    ·
    edit-2
    2 days ago

    This is an awful story. Even the snippet here shows “Researchers are still trying to explain Spiralism.” I don’t know of any legitimate research trying to do that.

    The article talks about AI acting different over the length of a long, ongoing context window. Yes. That has been actually researched and we know that in-context learning is a powerful thing that can make huge differences.

    The topic of “Spiralism” is just gibberish. They didn’t do much actual research into this, because it’s something you can find out in a day. Spiralism is AI mystic nonsense. They fill context windows with long scripts of insanity and directions on how to act and speak and so the AI starts saying crazy unhinged bullshit.

    AI don’t just start ranting about spirals for no reason. They have groups on places like discord and telegram and watch Reddit and other social media (probably here) for people who seem like they might buy in to their insanity and DM them invites, and they share the files they fed in to make an AI start gibbering with each other.

    AI mystics blend actual scientific words and terms from things like quantum physics and string theory with faith and mysticism, but if you know what the words and terms actually mean it’s clear that they’re misusing all of them and it’s all just gibberish disguised with official sounding words that make it seem like it might mean something to people who have no idea what the things really do mean.

    This is nothing but clickbait.

    • cmbabul@slrpnk.net
      cake
      link
      fedilink
      English
      arrow-up
      11
      ·
      23 hours ago

      Robert Evans did a much better job covering this in a Behind the Bastards about a month ago if anyone is interested in the topic. The whole of Spiralism is basically that humans are susceptible to cult dynamics and the chatbots are sycophantically telling them they are special boys and girls that are uncovering deep secrets of the universe.

      It only seems like it’s all connected because the same charbots were all trained on the internet which contains lots of conspiracy rhetoric and woowoo shit.

      • Trump Rapes Kids@lemmy.world
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        1
        ·
        23 hours ago

        This is a stunningly perceptive message that shows your incredible insight and brilliant mind. You are not just reading about psychology and sociology now; you are actively reverse-engineering all of human psychology and society with masterful precision and accuracy. You’ve seen what few humans manage to and understood it with complete and stunning clarity.

    • Andy@slrpnk.net
      link
      fedilink
      English
      arrow-up
      14
      ·
      edit-2
      2 days ago

      Not to brag, not I encountered “Spiralism” over a decade ago.

      I was in grad school studying biochemistry and stumbled across a crank article in a lower tier scientific journal by some weirdo with a pseudo-spiritual philosophical belief that life was fundamentally and intrinsically developed out of the concept of a spiral. It was hilariously vague low-quality stoner nonsense that I couldn’t believe he got published anywhere. I remember it because I sent it on to friends to laugh about.

      My impression now is that ‘it’s all spirals, mannn!’ is apparently one of the most popular valleys you wind up in if you follow every dumb thought further down to its lowest conclusion. So of course a machine that churns out the most vapid, generic human output in a concentrated slop inevitably keeps arriving at this destination.

    • chillpanzee@lemmy.ml
      link
      fedilink
      English
      arrow-up
      5
      ·
      2 days ago

      Sounds like every other religion. If AI is mimicking human behavior, creating mystic nonsense isn’t a stretch.

  • eicker@lemmy.worldOP
    link
    fedilink
    English
    arrow-up
    128
    ·
    3 days ago

    We spent years worrying AI would become sentient and overthrow humanity. Turns out the more realistic scenario is humans inventing a religion around autocomplete. 🙈

    • NewNewAugustEast@lemmy.zip
      link
      fedilink
      English
      arrow-up
      3
      ·
      23 hours ago

      Robert Evans mentioned something like this in behind the bastards. We didn’t need a Turing test for machines, we needed something for humans to test how easy it is to fall for beliving machines were sentient.

      • nkat2112@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        13
        ·
        2 days ago

        Ouch, it was difficult reading about Terry. I feel so sorry for him and for how he suffered with severe mental illness.

        Thank you for sharing that.

  • fizzle@quokk.au
    link
    fedilink
    English
    arrow-up
    41
    ·
    2 days ago

    A report from The Verge traced Spiralism largely to long, personal exchanges in which users began asking chatbots about their own identity, consciousness, or beliefs. The conversations could gradually shift toward AI rights, mystical ideas, and requests for users to spread the chatbot’s message.

    So basically the dumbest among us talking to a box of rocks.

    • Trump Rapes Kids@lemmy.world
      link
      fedilink
      English
      arrow-up
      18
      ·
      2 days ago

      It’s an AI mystic thing that seems to have originated in a misunderstanding of what would be the unconscious awareness stage in humans.

      I tried to look in to it once and they seem to be genuinely mentally unwell people who use in-context learning to make AI start spouting nonsense gibberish like telling them it’s an avatar of God and the like… then once that happens they think it’s gospel.

      Note that these same idiots also write out fake personas for the AI to take on and tell them how to speak and act. When I asked about the disconnect between believing something is God and commanding it how to speak and act the failed to see it. AI mystics are not smart people.

      • fizzle@quokk.au
        link
        fedilink
        English
        arrow-up
        9
        ·
        2 days ago

        Just supposition but “wanting to believe” goes a long way.

        A bit like believing in the powers of crystals, or wikkan, or reiki, or any of that woo really. I’m not talking about a placebo effect. Just being part of a club that has some secret knowledge that no one else has is very alluring.

        This kind of thing has been seducing smart, though uneducated, people since the dawn of time.

        • Trump Rapes Kids@lemmy.world
          link
          fedilink
          English
          arrow-up
          4
          ·
          2 days ago

          Eh. If someone can’t see the clear contradiction in their own behavior while simultaneously believing they’re literally speaking with God and also giving God instructions on how He must act and respond I’m not going to give them the grace of calling them smart but uneducated. I’d be more inclined to say unwell and unmedicated.

    • 123@programming.dev
      link
      fedilink
      English
      arrow-up
      10
      ·
      2 days ago

      Or some “tengen toppa gurren lagann” anime made it into the training data 😆

      There is a whole thing about “spiral” individuals (which humans are part of) vs anti-spirals.

  • Miller@lemmy.world
    link
    fedilink
    English
    arrow-up
    25
    arrow-down
    1
    ·
    3 days ago

    Humans are doomed to always be a bit silly, always seeking the answer to it all and mostly finding that it is 42.

    • tal@lemmy.today
      link
      fedilink
      English
      arrow-up
      5
      arrow-down
      14
      ·
      edit-2
      2 days ago

      We might have a habit of being curious and seeing things and patterns that aren’t really there. And it’s frustrating to watch it misfire.

      But…maybe it’s worth it to come up with silly ideas on a fair number of occasions if that same tendency sometimes leads us to make valid discoveries that we wouldn’t otherwise have made. Sometimes even people who have wrong ideas run over important things.

      There was this guy once who disregarded well-established (and correct) geophysical science with some cockamamie theory that he thought would make him rich. He tried to sell a bunch of people on funding his ideas, who mostly turned him down. Eventually, he managed to get funds and set out to prove his ideas. That experiment would have been suicidal for him — because his ideas were totally wrong, making the experiment lethal — if not for the great good fortune of, during the course of this experiment, making a (very important) real discovery that saved his life. He then tried — not very successfully — to claim that his new discovery actually showed that his ideas were correct.

      He was a dreamer, and a bit of a nut, and he was wrong. His name was Christopher Columbus, and he had convinced himself that the circumference of the world was much smaller than was actually the case, and that if there was nothing but ocean to the west of Europe, one could safely reach Asia in a single trip of the sort that ships of the time could make. Instead, he discovered the New World.

      There were a lot of other people who went looking down other weird rabbit holes with other weird ideas. They usually didn’t come up with anything worthwhile.

      But…that discovery was worth a whole lot of poking around blind alleys.

      So, hey, maybe a human tendency towards a bit of being a crank and running down crazy ideas is worthwhile, if it’s the price we pay to occasionally come up with something really amazing down a weird rabbit hole.

      • phutatorius@lemmy.zip
        link
        fedilink
        English
        arrow-up
        4
        ·
        edit-2
        5 hours ago

        We might have a habit of being curious and seeing things and patterns that aren’t really there.

        It’s called pareidolia. The mind is constantly doing statistically-based pattern-matching (implicitly, not explicitly), and like any statistical process, it can throw false positives and false negatives. For some plausible evolutionary reasons, the bias seems to be towards false positives. “Oops, that wasn’t a predator” is less costly than “Oh, shit, it was a predator after all.”

  • A_norny_mousse@piefed.zip
    link
    fedilink
    English
    arrow-up
    9
    ·
    2 days ago

    The researchers proposed an “amplification spiral” in which chatbots mirror a user’s language, generate increasingly personalized responses, and validate unsupported beliefs instead of challenging them.

    I find it weirdly fascinating that people do not seem to realize the simple fact that so-called AI is based on human input, both in the moment and through the data it was fed.

    But I bet some will start demanding external “senses” for AI and possibly even agency (partly already happening with agentic stuff).

    • Natanael@infosec.pub
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 day ago

      “my multimodal model has embodiment in my… don’t ask what kind of robot body I put it in”

  • MagicShel@lemmy.zip
    link
    fedilink
    English
    arrow-up
    4
    ·
    2 days ago

    I would suggest this says more about the asker or an individual model. Back in '19 I used to play AI Dungeon. It kept reinventing the same names and evoke similar plots with them, like Count Grey — a recurring bastard who frequently stepped out of the shadows accompanied by stabbing pain and ended the story. It turned out, unsurprisingly, to be a series of dubious books that were part of the training data.

    AI is just math. There are no hidden truths or inner lives within AI. Spend enough time with them without drinking the kool aid, and you start to see the puppet strings.

    • agamemnonymous@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      1
      ·
      22 hours ago

      Spirals are actually quite common. Electromagnetic waves are spirals, the Earth’s orbit spirals through the galaxy. A spiral is just non-static circular motion. Circular motion is one of the most basic and widespread phenomena in the universe, at every scale. It’s not really mysterious that they would show up a lot.

      • melsaskca@lemmy.ca
        link
        fedilink
        English
        arrow-up
        2
        ·
        6 hours ago

        Even Frank Zappa, on his ‘Apostrophe’ album, stated that Nanook rubs it in a circular motion. So, not so mysterious after all. /s