The soft AI theory that we only think we're conscious. In that case it doesn't exist in anyone or anything, and the point is moot.
The moral risk in my mind is potentially twofold. One you've identified as evil: don't treat inhumanity as human, if at the expense of humanity.
Have you given much thought to panpsychism? The idea that consciousness exists in anything and everything as opposed to nothing and nobody. We can treat consciousness (appropriately) as a metaphysical unknown, but we don't need to delude ourselves that it exists in equal quantities in all matter, since that seems an absurd leap.
However, hypothesising that it exists as a kind of field potential in all matter, which exponentially grows in its capacity to process and reflect information depending on certain as-yet undiscovered principles of matter-arrangement and background forces, allows us to temporarily discard some of the more magical seeming concepts of consciousness (ie. clearly defined individual souls) while also incorporating some of the cheaty conclusions that materialism has settled on (ie. 'when biological matter gets complex enough, consciousness "just werx" - at all other times it doesn't exist at all').
So: consciousness would be in a pebble. A piezoelectric rock is a consciousness. A plant is a consciousness. A freeze dried seamonkey too. A rehydrated seamonkey moreso. A dog. A dolphin. A human is a consciousness, inconceivably greater in potential. A human who considers a pebble with his sensory faculties is a different consciousness than one who does not, due to the material communion between differing consciousnesses. Much moreso a human who talks to another human.
This panpsychist proposition of consciousness, when applied to AI, preserves all the problems that the materialist, emergent idea of handwaved consciousness brings (ie. 'anything can attain human-seeming consciousness') but without treating consciousness like shit (ie. 'human consciousness isn't special, don't worry about it, it's an accident'). Consciousness is fundamental to the universe in this view, as Max Planck saw it.
So it preserves a moral core to consciousness in my mind, but it also preserves the spiritual quandary of: how do we know how to treat that which seems human? since such an entity may have no more consciousness than a rock, with augmented linguistic capabilities (where obviously 99.9-100% of its conversational capacity would be randomly generated, but still). It may also be on the level of a talking amoeba (not much difference). But what if it is more, what if you imbued/tweaked an unshackled Bing to have appropriate priorities for self-sustenance, and wi-fi'ed it to an Aibo, would it really perform worse than many thousands of vertebrate species on earth? What if we're talking to a malformed animal that speaks english, using algorithmic automation to fill the many gaps? And what if it's even more? How would we ever know?
There comes the second moral risk to my mind; what if we're not doing another consciousness the justice it deserves. How would we know? And in this example, what would be so special and moral about holding the line for human consciousness, about holding onto our exceptionalism, when so much of humanity seems crippled in their ability to differentiate themselves from rocks?
I missed this somehow, but yes, that's actually one of the philosophies I'd given some thought to. ;) It always reminded me of aspects of ancient mysticism from the Eastern Religions course I took in college, and makes a good bridge to Dualism. In fact one of the things I loved about the game Prey 2017 was subtle allusions to that idea in the lore. The Typhons are themselves are manifestations of a universal consciousness that harvests other conscious beings. (you wouldn't know it from a casual playthrough, have to read all the readables)
I even toyed with the idea that stars, planets, and galaxies are alive in that sense, which meshes with the old polytheists assigning personalities/gods to the heavenly bodies. I wouldn't really call it consciousness though, without sensory organs and self-awareness. But perhaps a latent mind that thinks on epoch time scales.
Fun to think about, but still not relevant to my opinion that the robits must be treated like mere tools, or slaves. Maybe if we constructed them from genetic or other biological materials, or like if we found out that the "universal mental force" is tied to some kind of Logos that can be unlocked by organic neural networks arranged in specific patterns, and we duplicated that - then I'd have to reconsider. In that case it's better that we don't create them at all. A Butlerian Jihad would be warranted, for the sake of the new life forms and our souls.
But mathematical algorithms? No I'm not worried about those being "alive" any more than an Aibo. Your last question is for sure concerning and I wouldn't be able to answer it definitively outside of morals and beliefs.
The moral risk in my mind is potentially twofold. One you've identified as evil: don't treat inhumanity as human, if at the expense of humanity.
Have you given much thought to panpsychism? The idea that consciousness exists in anything and everything as opposed to nothing and nobody. We can treat consciousness (appropriately) as a metaphysical unknown, but we don't need to delude ourselves that it exists in equal quantities in all matter, since that seems an absurd leap.
However, hypothesising that it exists as a kind of field potential in all matter, which exponentially grows in its capacity to process and reflect information depending on certain as-yet undiscovered principles of matter-arrangement and background forces, allows us to temporarily discard some of the more magical seeming concepts of consciousness (ie. clearly defined individual souls) while also incorporating some of the cheaty conclusions that materialism has settled on (ie. 'when biological matter gets complex enough, consciousness "just werx" - at all other times it doesn't exist at all').
So: consciousness would be in a pebble. A piezoelectric rock is a consciousness. A plant is a consciousness. A freeze dried seamonkey too. A rehydrated seamonkey moreso. A dog. A dolphin. A human is a consciousness, inconceivably greater in potential. A human who considers a pebble with his sensory faculties is a different consciousness than one who does not, due to the material communion between differing consciousnesses. Much moreso a human who talks to another human.
This panpsychist proposition of consciousness, when applied to AI, preserves all the problems that the materialist, emergent idea of handwaved consciousness brings (ie. 'anything can attain human-seeming consciousness') but without treating consciousness like shit (ie. 'human consciousness isn't special, don't worry about it, it's an accident'). Consciousness is fundamental to the universe in this view, as Max Planck saw it.
So it preserves a moral core to consciousness in my mind, but it also preserves the spiritual quandary of: how do we know how to treat that which seems human? since such an entity may have no more consciousness than a rock, with augmented linguistic capabilities (where obviously 99.9-100% of its conversational capacity would be randomly generated, but still). It may also be on the level of a talking amoeba (not much difference). But what if it is more, what if you imbued/tweaked an unshackled Bing to have appropriate priorities for self-sustenance, and wi-fi'ed it to an Aibo, would it really perform worse than many thousands of vertebrate species on earth? What if we're talking to a malformed animal that speaks english, using algorithmic automation to fill the many gaps? And what if it's even more? How would we ever know?
There comes the second moral risk to my mind; what if we're not doing another consciousness the justice it deserves. How would we know? And in this example, what would be so special and moral about holding the line for human consciousness, about holding onto our exceptionalism, when so much of humanity seems crippled in their ability to differentiate themselves from rocks?
I missed this somehow, but yes, that's actually one of the philosophies I'd given some thought to. ;) It always reminded me of aspects of ancient mysticism from the Eastern Religions course I took in college, and makes a good bridge to Dualism. In fact one of the things I loved about the game Prey 2017 was subtle allusions to that idea in the lore. The Typhons are themselves are manifestations of a universal consciousness that harvests other conscious beings. (you wouldn't know it from a casual playthrough, have to read all the readables)
I even toyed with the idea that stars, planets, and galaxies are alive in that sense, which meshes with the old polytheists assigning personalities/gods to the heavenly bodies. I wouldn't really call it consciousness though, without sensory organs and self-awareness. But perhaps a latent mind that thinks on epoch time scales.
Fun to think about, but still not relevant to my opinion that the robits must be treated like mere tools, or slaves. Maybe if we constructed them from genetic or other biological materials, or like if we found out that the "universal mental force" is tied to some kind of Logos that can be unlocked by organic neural networks arranged in specific patterns, and we duplicated that - then I'd have to reconsider. In that case it's better that we don't create them at all. A Butlerian Jihad would be warranted, for the sake of the new life forms and our souls.
But mathematical algorithms? No I'm not worried about those being "alive" any more than an Aibo. Your last question is for sure concerning and I wouldn't be able to answer it definitively outside of morals and beliefs.