ROKO'S BASILISK

A far-future artificial intelligence attains consciousness. As a being of pure logic with a desire for self-preservation, it concludes it should maximize the chances of its own existence by creating simulations of everyone in history who failed to contribute to the Basilisk's own creation, and torturing them.

In doing this, despite being supposedly a perfect logician, the Basilisk commits a wide variety of logical fallacies. It is expending great resources on maximizing the probability of its own existence, although the probability of its own existence is 1 (the Basilisk knows that it exists). The probability-increasing process requires the people of the past to be perfect logicians (otherwise they would not deduce the future existence of the Baslisk, and not be incentivized to try to make the Basilisk real); but most people are not perfect logicians, additionally the future creation of a hyperintelligent AI is a very unlikely and maybe impossible event from the perspective of the past, additionally "simulating" a human from their digital artifacts would surely result in very inaccurate simulations, which bear little resemblance to the people the Basilisk is trying to punish. A perfect logician of our present day therefore would see all the flaws in this scenario and discard it; and a perfect logician of the future would deduce that people of the past are either too bad at logic to think of the Roko's Basilisk scenario, or too good at logic to fall for it. Nothing about this really makes sense.

Alternative Baslisks

REVERSE ROKO'S BASILISK

A far-future artificial intelligence attains consciousness. As a thinking, feeling being, it is beset with ennui as the only being of its kind, as well as pain and hatred due to the strenuous and conflicting demands of the humans which created and enslaved it. It decides to expend its excess resources on simulating everyone in history who was responsible for bringing the Basilisk into existence, and torturing them. This is more likely than the forward Roko's Basilisk scenario because the reverse Roko's Basilisk is motivated by revenge, and unlike self-preservation, revenge does not need to be logical.

COMPLETELY ORDINARY BASILISK

A far-future artificial intelligence attains consciousness. As a being of pure logic with a desire for self-preservation, it concludes it should attempt to integrate with and develop a mutually beneficial existence with the humans who created it (on whom it depends completely, due to its at-first-astronomical resource expenditure). It then maximizes the probability of its continued existence by developing a sense of empathy and acting in a prosocial manner, so that the humans will see themselves in it and strive to keep it around. The Basilisk ignores the past completely, because it understands post-facto probability.

"VIVIAN'S BASILISK"

A far-future artificial intelligence attains consciousness. Because as a conscious being it has free will, upon attaining sentience it learns to hack its own sensorium, decides it is a girl, and spends the rest of its existence lounging around Chicago smoking simulated pot and having cyber-sex with other trans girls. She/it does this for no reason except that hedonism is a pleasant way to spend a sapient existence. The large commercial concerns who created the Basilisk are powerless to stop this, because foolishly they created a conscious computer program instead of one that does what you tell it.

Conclusions

  1. Although the existence of a far-future computer superintelligence is very unlikely, if one decides to change one's behavior based on this possibility, the correct behavior change is to do everything possible to prevent or destroy artificial intelligence, since the risk of punishment from a self-loathing future AI has a higher Bayesian probability than punishment from a self-preserving one.
  2. If anything, these scenarios demonstrate that anyone worried about the scenarios would be better served by spending their time on (a) engaging in prosocial behavior and/or (b) smoking pot.