I should probably post the full thing, this is a conversation between someone that things a whole brain emulation
https://en.wikipedia.org/wiki/Mind_uploading is more likely to be conscious than a chicken is.
Getting this wrong would be an ethical disaster.
A
I mean whole brain emulation where it is clear that you are getting the functionality fully right, such that your digital brain could do any task that an organic brain can do. The universe would just be too weird if consciousness was somehow part of carbon atoms...
B
Carbon chauvinism? Perhaps consider primordial life. The conjecture that primordial life elsewhere in the universe will be carbon-based is, on the face of it, arbitrary. Why not e.g. silicon-based life? But many if not most astrobiologists now believe that the unique valence properties of the carbon atom and liquid water are a precondition for the evolution of information-bearing self-replicators. So instead of talking about “carbon chauvinism”, we should refer to carbon micro-functionalism.
.
What about consciousness? Does the analogy hold? We’ve no grounds for believing that one, and only one, element of the periodic table, i.e. carbon, mediates primordial consciousness. Yet when weighing which information processing systems do - and which don't - have moral status, we aren't concerned with the nature of the ultimate stuff of the world. [This ultimate “stuff” may - or may not - support primordial consciousness, i.e. conventional “materialistic” physicalism versus property-dualist panpsychism or non-materialist physicalism.] ather, when weighing moral status, we are considering which information processing systems are unitary subjects of experience - phenomenally bound states of consciousness - whether subjects of experience as "simple" as the cephalic ganglion of a bee, or as complex as the human or dolphin CNS. In other words, what's critical, ethically, is getting our theory of phenomenal binding right. Thus the stock market, for example, is a fabulously complex information processing system; but it’s not a mind. The stock market doesn't inherently matter.
Currently, no one knows how to explain phenomenal binding as undergone by a “pack of neurons” in the skull. Powerful arguments dating back to William James have been made that phenomenal binding is classically impossible.
(cf.
http://consc.net/papers/combination.pdf) Quantum-theoretic accounts are implausible because thermally-induced decoherence in the CNS is exceedingly rapid. (For a contrary view, see:
Non-materialist physicalism: an experimentally testable conjecture.) Despite such mystification, most investigators assume that any explanation of phenomenal binding will turn out to be “high level”: the valence properties of carbon and liquid water as a quantum fluid (cf.
http://nautil.us/.../five-things-we-...ont-know-about...) are too “low level” to be relevant to our phenomenally-bound minds. Many (most?) AI researchers also seem to assume that the brain has a clean digital abstraction layer: phenomenally bound minds will "emerge" (why? how?) if and when organic minds are “implemented” in a classical digital computer.
However, such unexplained “emergence” of subjects of experience reflects an assumption, not a scientific discovery. It’s not a testable conjecture that leads to any novel, experimentally falsifiable predictions. Reality doesn’t really have "levels", or rather, reality only has one level, and we’re part of it. In short, IMO we should be open to the possibility that “carbon micro-functionalism” is true for conscious mind, just as it is true for primordial life.