Appearing suspicious may come at a cost

suspicion.jpeg
Suspicion by Gültekin Bilge

How do you trust others? What makes you become suspicious when working with other people? Though we rely on collecting information about others when making decisions, the ways we collect that information may change how others perceive us. People may be less likely to trust others when they inquire about their behavior in a way that appears suspicious, a new study shows.

Decisions of whether to trust other people can vary based on risk and uncertainty. In simulating how people trust one another using a game between trustees and investors, researchers studied social context, monetary cost, and the trustee’s trustworthiness to show that an investor appearing suspicious may lessen trust and appearing averse to betrayal, increase trust.

Using computational models of how individuals collecting information before making an investment decision, the researchers examined the intuitive patterns behind why participants made the decisions they did. They tested participants under conditions of whether collecting information cost money and whether the trustee would know the investor was collecting information. Participants gathered less information when it came at a cost and when they could find conclusive results from the information about others.

The researchers tested various model to determine which one fit the experimental results best. When the difference between outcomes that would favor investing was different enough from those that wouldn’t, participants stopped collecting information. To test how strong of a role suspicion played in these decisions, they compared the Cost of Appearing Suspicious (CAS) model against a Sample Cost model, an Uncertainty model, and a discrete Drift Duffusion Model (dDDM). For the CAS, Sample Cost, and Uncertainty models, the participants’ decisions are Bayesian in the sense they determined how likely they are to invest based on their previous beliefs about either the trustee or the investor.

The CAS model used a suspicion factor, that gathering information decreases the probability of investment. The Sample Cost model does not use this suspicion factor, and, under the Uncertainty model, participants would stop when the posterior belief distribution width dropped below a tolerance threshold. Under the dDDM, on the other hand, participants would collect information until their evidence for trustworthiness or untrustworthiness met a certain criteria. After determining how well models fit various conditions on the individual level, they found the CAS model predicted better than the others, and it even fit well to investment decisions after collecting information.

When individuals choose to invest or not, scientists have proposed various models for decision-making across different contexts. They come into play as people choose whether to work with or trust others. One study in the Proceedings of the National Academy of Sciences showed, when people work together, they care about whether their partner gathers information about how much they would profit if they defected. They instead engage in blind cooperation cooperate among trustworthy and untrustworthy people, but don’t collect information to make themselves appear suspicious. This is further shown that people are perceived as selfish when they gather information for their own profit in a study in the Journal of Behavioral and Experimental Economics.

The work has practical applications in information collection and use for psychiatric disorders. Biases in collecting information when information is uncertain or scarce has been associated with weighing negative information heavily in depression and compulsivity. How well we model the moral character of other individuals are central to disorders like borderline personality disorder and autism.

How the brain makes things simple

homunculus
The homunculus depicts how the body represents itself in the somatosensory cortex of the brain.

Physical tasks the may seem basic to humans can be much more complicated for machines. The sheer number of degrees of freedom and dimensions the brain must use to make something as versatile and complex as hand motions means that the brain must reduce these high-dimensional spaces to achieve these feats of motion. This is a method of taking complicated processes such as the movement of individual fingers and finding ways of classifying these movements so that they may be accessed without as much information as they would otherwise take. Computational neuroscientists Yuke Yan and James G. Goodman along with biologist Sliman J. Bensmania observed the musculoskeletal movements of human and monkey hand kinematics during sign language, typing, and grasping objects with a principal component analysis. This let them figure out how the motions were specific to certain conditions such as the object trapped or the letter signed using linear discriminate analysis to classify different conditions.

They removed components in descending order of variance by reconstructing the hand posture on each trial and classified them using linear discriminant analysis to determine how many experimental results could be classified correctly. This let them determine the functions of complicated hand gestures using a small number of variables that represent the various motions. Using combinations that account for the motions, this would let researchers understand how such intricate, complicated motions can be governed by a set of neurons or neurophysiological mechanisms. They tested how similar the results were between humans and monkeys and found monkeys had hand postures that weren’t as specific for different objects.

Finally the researchers classified hand motions using spike counts of neurons in the primary motor cortex to determine which ares of the brain corresponded with different motions. This gave them an overarching picture from neuroscience to behavior in how the brain makes sense of complicated, concerted movements.

Other methods of taking advantage of the probability let scientists create maps that can measure across variables in space. One may use a stochastic process, a family of random variables, on a space of probabilities to choose appropriate parts of the space that correspond with different outcomes. This can be applied from simple axon models of individual neurons to entire neural networks.

Very little is known about exactly how the brain interprets somatosensory activity as a particular tactile sensation. Some initial evidence towards understanding this comes from individuals with brain damage due to stroke. Researchers studying the somatosensory cortex of the brain, the part linked with tactile localization, have shown that hands and forearms tend to mislocalize sensations of touch towards their centers in healthy individuals and those who have had strokes in somatosensory regions. Psychologists Janellen Huttenlocher and Susan Duncan and statistician Larry Hedges proposed that spatial locations bias towrds the middle of categorical spaces and away from boundaries the more uncertain their information is. This method of dealing with uncertainty is another example of how the brain simplifies problems when it needs to.

Flies and humans use coding strategies when processing visual information to create more-refined images. Fly and cone photoreceptors respond to light by adapting their gain to a local mean intensity. This creates a contrast between intensity at the receptor and the local mean. The receptors then code the response to this contrast among all the values of input light. This way, the nervous system simplifies intensities to proportions that are more easily evaluated by the neuron’s response range. It also lets the fly or human identify same objects under different lighting.

A machine learning approach to Traditional Chinese Medicine

Screen Shot 2019-07-11 at 10.48.36 PM

Modern science can uncover ancient wisdom. While it may seem regressive or pseudoscientific to study concepts from Traditional Chinese Medicine (TCM), they reveal deeper meanings about who we are as humans when subject to scrutiny by the scientific method. The herb formulas, plant-derived nature produces of TCM are still used in disease prevention and treatment despite the dominance of modern science. When medical researchers performed machine learning classification methods on 646 herbs are according to organ systems, known as Meridians, they found the 20 molecule features were most important for predicting these Meridian. It included structure-based fingerprints and properties of absorption, distribution, metabolism, and excretion. As the first time molecular properties of herb compounds have been associated with Meridians, this provides molecular evidence of Meridian systems. 

The Meridian system dictates how he life-energy qi flows through the body. Qi includes actuation of the body, warming, defense again excess, containment of body fluids, and transformation between qi and food, drink, and breath. Each Meridian corresponds to a yin yang quality, an extremity (hand or foot), one of the five elements (metal, fire, earth, wood, or water), an organ (such as heart or kidney), and a time of day. The yin yang qualities describe how complementary, opposite forces of the universe interact, such as Greater Yin or Lesser Yang. Given these roots in traditional, non-scientific thought, scholars have debated the scientific justification behind why and how TCM works. In their paper “Predicting Meridian in Chinese Traditional Medicine Using Machine Learning 2 Approaches,” the researchers assumed Meridian can be found through scientific methods to begin with. The five elements are qi are metaphysical, not modern physiological or medical phenomena. The researchers emphasized the need to examine the herb medicine actions as they relate to disease etiology to create a formal understanding of TCM.

Qi and yin yang as they relate to human health date back to texts of discussion and debates from the Warring States period (475–221 BC) of ancient China. Philosopher Zhaunghzi noted qi was the basis of the body’s physical being with the six qi (wind, cold, summer heat, fire, dryness, and damp) in harmony with one another as they affect the seasons. These theories would be used in medicine to describe relations and analogies between the body, the state, and the cosmos, or the universe.

The neuroscientific basis of consciousness

furiousdreams
Part of “Furious Dreams” by Marc Garrison

Scientists and philosophers alike have examined and pondered about consciousness, one of the most central problems in our experience of the world. Both approaches may seek different methods, with science being empirical and philosophy, speculative, but they’re both relevant to any discussion on such a complicated phenomena. Understanding how neural correlates of conscious experience correspond to various parts in the brain lets scientists take “bottom-up” approaches of beginning with empirical phenomena and determining what cognitive and psychological behaviors result. On the other hand, beginning with the philosophy of mental states, including beliefs, intents, desires, emotions, knowledge, and figuring out how those can be attributed to the mind moves in the opposite direction. These mental states are whatever is the nature of the mental phenomena the mind occupies. It’s the way the brain makes sense of the world. A combination of empirical neural data from both computational and psychological models alongside a philosophical analysis would let us bridge the gap between the brain and consciousness. Either way, researchers end up with a neurobiologically accurate view of the brain in how consciousness emerges. There are many challenges to neuroscience explaining consciousness and its findings have philosophical significance to many ideas in epistemology, ethics, and metaphysics.

The difficulty of the subjective experience presents a challenge to consciousness. Any person’s experience is an external phenomena to anyone else. Philosopher Edmund Husserl’s notions of phenomenological and natural attitude show this relationship between the first- and third-person experiences. When someone perceives a car, their consciousness is directed at the car and that person is not necessarily aware of the details of the experience, such as what the steel car feels like. This is Husserl’s natural attitude. When that person focuses on the experience of perceiving the car as its own experience, it is a phenomenological attitude. A neuroscientist would generally concern themselves with the external phenomena they research and endorse the natural attitude. But the concepts of a scientific model (such as water being composed of two hydrogen atoms and one oxygen) are conscious phenomena a neuroscientist uses to understand the model. The neuroscientist can use the phenomenological attitude towards those models as experiences, and this shows we can’t escape our own point of view. Even a scientific model is a representation in one’s mind.

Consciousness as a neuroscientific phenomena requires a study of this relationship between an experience and the scientific model of it. The gap between the two can depend upon whether one endorses a realism view of science or an antirealism one. Under realism, mature scientific theories can be true and describe the world such that the gap is non-existent or narrow. Scientific realism means that all natural phenomena can be modeled using the structures and relations among them. We may model qualitative feelings, or qualia, our subjective experiences, using these structures and relations. It still leaves certain experiences difficult, such as how one may see the color red as a structural relationship given that introspection about redness doesn’t reveal its nature. Anti-realism, on the other hand, dictates we can only say whether a neuroscientific theory of consciousness is compatible with observations, not whether it captures nature. There is a large epistemic gap between any concrete phenomenon and the corresponding scientific models. The experiences are not different in this respect.

Before the 20th century difference between philosophy and science, philosophers generally studied consciousness through both philosophical and scientific means. French philosopher René Descartes performed research in mathematics, neuroscience, and philosophy. Psychologist-philosopher William James would create philosophical theories in light of empirical psychology research. With the rise of logical positivism, the idea that only statements that can be empirically verified are meaningful, in the early 20th century, philosophers focused on the semantic content of arguments instead of empirical scientific results. During this time, the three traditional perspectives of the mind-brain question, physicalism, mentalism, and dualism, emerged. Physicalism is the thesis that everything can be reduced to physical phenomena, mentalism, to mental phenomena, and dualism, that everything is either mental or physical. he rise of physicalism in the late 19th century meant to take consciousness as unscientific in some interpretations. Philosopher Galen Strawson’s realistic physicalism means the physical nature of the nervous system can manifest consciousness through mental activity. This can be illustrated with an example of philosopher Frank Jackson’s knowledge argument.

The argument uses the fictional story of a famous neuroscientist Mary who learns everything about the world through a computer but remains confined to a room that is entirely black and white. If physicalism were true, one might argue Mary knows everything about the world, but, because she has not experienced color, she does not know what it is. Upon seeing color, this argument would dictate that she would experience a subjective experience of her consciousness that she had never encountered before and, therefore, learns something new – showing she did not know everything about the world. One may conclude, from this line of reasoning, physicalism doesn’t entail everything. Possible responses to this may include the ability hypothesis – that Mary learns how to see color, but doesn’t learn what color is, therefore, what she learns doesn’t contradict that she knew everything about what the world was.

The historical trends meant changes, not only for consciousness, but for science as a whole. Scientific terms began taking new meanings. With the rise of thermodynamics around the turn of the 20th century, heat went from meaning boiling water to a specific dimension of temperature variation. Consciousness became an empirical phenomena of brain activity variation.

Consciousness presents researchers with the problem of how to explain when a mental state is conscious rather than not as well as what the content of a conscious state is given the subject experience of everyone’s consciousness. From a philosophical perspective, we rely instead on behavior and introspective testimony on the nature of consciousness in searching for common features from which we deduce knowledge about consciousness. From the neuroscientific angle, we study the central nervous system and the neural properties in the cerebral cortex. Psychologists Jussi Jylkka and Henry Railo have argued scientific models of consciousness need to explain consciousness’ constituents, contents, causes, and causal power. Though consciousness has many aspects to it, we focus on perception in this essay. Consciousness can further be differentiated into phenomenal consciousness, the properties of experience that correspond to what it’s like to have those experiences, and access consciousness, consciousness based on which states one can access.

The global neuronal workspace explains when a mental state is in consciousness such that it is accessible to systems related to memory, attention, and perception. Accessing content means it can use its content in performing computations and processing. Consciousness resides in these accessed states by the cortical structure of the brain that is involved with perceptual, mnemonic, attentional, evaluational and motoric systems. Whatever neurons are involved in someone’s current state constitute the workspace neurons. They activate such that the neural activation causes activity between workspace systems. It’s tempting to say the cortical workspace network correlates with the phenomenon of consciousness itself especially given imaging results can show which areas of the brain activate when a subject is conscious, but this correlation doesn’t tell us whether brain activity is of phenomenal or access consciousness.

Another approach, recurrent processing theory, ties perceptual to processing consciousness without a workspace and, instead, a focus on activity connecting sensory areas of the brain. It uses first-order neural representation, that we may perceptually represent the content of a mental state to mean perceiving that content, to explain consciousness. Interconnected sensory systems may use feedforward and feedback connections, such as those in the first cortical visual area, V1, which carry information to higher-level processing areas. The forward sweep of processing uses feedback connections linking visual areas. Global workspace theory and recurrent processing theory differ in the stages of visual processing they depend upon. The four stages of visual processing are superficial feedforward processing (process visual signals locally in the visual system), deep feedforward processing (the signals can influence action), superficial recurrent processing (information travels to previously visited visual areas), and widespread recurrent processing (across broad areas such as the global workspace access). Recurrent processing at the third stage, superficial recurrent processing, is necessary and sufficient for consciousness according to recurrent processing theory, and at the fourth stage, widespread recurrent processing, for the global workspace theory.

Philosopher Victor Lamme has argued that superficial recurrent processing is sufficient for consciousness because features of widespread recurrent processing are also found in superficial recurrent processing. Recurrent processing is found in both stages, and the global neuronal workspace theory allows superficial recurrent processing to correlate with widespread processing. Lamme believes that, in response to visual stimuli, there is first a fast forward sweep of processing, proceeding through the cortex. This first stage is nonconscious. Only a second stage of recurrent processing, when earlier parts of the visual cortex are activated once more by feedback from later parts, is taken to be conscious (with, again, empirical support).

Another approach to consciousness holds that one can be in a conscious state if and only if one represents oneself in that state. If one were in a conscious visual state of seeing am moving, that person must represent themselves in that visual state. The higher-order state represents the first-order state of the world and results from the consciousness of the first-order state. One must be aware of a conscious state to be in it. This lets neuroscientists correlate empirical work on higher-order representations of states with prefrontal cortex activity. For some higher-order theories, one can be in a conscious state by representing oneself even if there is no visual system activity.

Neuroscientists can use empirical tests of higher-order theory against other accounts, but neurologist Melanie Boly has argued that individuals with the prefrontal cortex removed can still have perceptual consciousness. This may prefrontal cortical activity isn’t necessary for consciousness, but one may argue the experiments didn’t remove all of the prefrontal cortex or that the prefrontal cortex is necessary, but in a more complicated system than previously suggested. Psychologist Hakwan Lau and philosopher Richard Brown have used experimental results to suggest consciousness cannot exist without the corresponding sensory processing as predicted by some higher-order accounts.

Finally, the Information Integration Theory of Consciousness (IIT) uses integrated information to explain whether one is in a state of consciousness. Integrated information is the effective information parts of a system carry in light of the causal profile of the system. If the information a system carries is greater than the sum of the information of each of the individual parts, then the information of that system is integrated information. IIT holds that this integrated information implies that a neural system is consciousness, and, the more integrated information there is, the more conscious the system is. Neuroscientist Giulio Tononi has argued the cerebellum has a low amount of consciousness compared to the cortex as it has far fewer connections even though it has more neurons. IIT suffers from treating many things as conscious even when they don’t seem to be. Tononi has proposed that a loop connecting the thalamus and cortex forms a dynamic core of functional neural clusters, varying over time. This core is assumed to integrate and differentiate information in such a way that consciousness results.

Neuroscientist J. H. van Hateren has presented a computational theory of consciousness in which the neurobiology of the brain allows it to compute a fitness estimate by a specific inversion mechanism that also causes the feeling of consciousness. His conjecture that consciousness is a transient and distinct cause the individual produces when he or she prepares to communicate—externally or internally. Citing the thalamocortical feedback loop, the internal variables involved in this process estimate the individual’s evolutionary fitness.

Despite how flashy it sounds to say researchers can completely understand consciousness, the challenges that neuroscientists and philosophers face mean things are far from completely figured out. There remains a lot to be discovered and examined from both scientific and philosophical angles.

A metamodernist narrative of genetic engineering

mm

Screen Shot 2019-07-06 at 9.18.18 PMWith the ethical concerns raised by issues of gene editing of human embryos, academic ethics research has set the foundation for and discussed the bioethical threats mankind faces. Alongside artificial intelligence (AI) and similar issues such as data science privacy and the power of social media, the steps into baby manufacturing are illustrated through a mix of modernist and postmodernist ideologies and require a revised notion of a biological-digital autonomy that can account for the changing self. The CRISPR-Cas9 gene editing technology have already shocked and disgusted scholars in science and philosophy around the world. Questions of how much of who we are we should be able to change and what we should do with the rapid power of artificial intelligence on the horizon have taken center stage. With the newfound metamodernism appraoch to science, reality, and existence, we step into gene editing the same way we jump off the deep end of a lake and hold our breath until we rise to the surface.

The oft-repeated truism “science is moving so fast that ethics just can’t keep up” couldn’t be farther from the truth. Ignoring the baseless assumption that science and ethics were racing against one another, the scientistic idea that philosophers and ethicists in similar fields have not addressed the power and potential of science would be to disregard the decades of ethics research on genetic engineering. The claim also seems to treat science as an uncontrollable force that must be braced against because we can’t do anything to stop it. It’s false that mankind has complete control over nature, but the notion inaccurately portrays mankind as weak and vulnerable to the world when we can take a metamodernist approach that rests somewhere in between. Researchers in ethics have been paying close attention. They’ve been studying everything closely.

Screen Shot 2019-07-06 at 9.18.12 PMSociety and individuals have been shifting from postmodernism into metamodernism. We create the self as something between a postmodernist and modernist notion of reality through gene editing. As opposed to postmodernist traditions that nothing is real and modernist ones that reality is there beyond media, language, and symbols, right now we’re sure reality is somewhere in the middle in our notions of metamodernism. We are both a modernist believer in the power of science and technology and a postmodernist skeptical of the reality we find. Genetic engineers have begun using pluirpotent stem cells, ones that have the same properties of embryonic stem cells but come from manipulating ordinary adult cells rather than destroying embryos, that are more effective in providing dozens or hundreds of offspring for individual parent cells. As more stem cell research goes into how male sex cells can result from female cells and vice versa, this could even allow single parents or same-sex couples to produce biological children. Researchers have even predicted scenarios in which children result from the DNA of more than two biological parents, known as “multiplex parenting.”

Recent success in both cloning and CRISPR technologies have let scientists understand better the embryology and developmental physiology of human embryos as a result of the pluripotent stem cell advancements and in vitro fertilization (IVF). We must warn of the issues that may arise as stem cell reproduction methods gear towards manufacturing embryos for desirable traits. Couples who choose to keep their unwanted embryos frozen or donate them to further research or to other couples need to be aware of how those embryos are being used to assess their role of responsibility in stem cell research.

This stem cell method comes with the advantage as the daughter cells result similar to the adult ones, and researchers have posed solutions for the issues of eugenic control that would result. We can critique these ideas for their shortcomings in characterizing the eugenics movement. These movements do not thoroughly emphasize the social forces governing how individuals would be manufactured. To address the issues raised by gene editing, we need a deeper, more multidimensional view of the moral problems raised by eugenic control that accounts for the changing self and reality in a metamodernist world. We can engage in these subjects through personal narratives and humanized ideas of who we are that embrace ethics and threats of existentialism.

Researchers have, however, brought up solutions that derive from dangerous principles of eugenics and extreme notions of individual autonomy. In the transition to metamodernism, they prevent mankind from pushing back against looming threat of a full-fledged surveillance state, and, instead disregard the idea that a particular line of research can be inherently morally wrong. These transhumanist thinkers such as philosopher Nick Bostrom (who has also warned about the threat of Superintelligence) who proposes a solution to use stem cell sex cells for performing eugenic selection for intelligence, partially as a method for combating superintelligent AI. If humans can replicate natural selection on a group of embryos over the course of several generations, they can produce the most intelligent humans possible. This eugenics approach isn’t uncommon, either. Ethics professor Nicholas Agar wants prospective parents to choose how they can improve their children in a “liberal eugenics” fashion. This sort of scientific perfection fails to capture how these humans would supposedly relate to the rest of society given that they’ve attacked the fundamental ideals of community and sharedness that humans share. Bostrom does suggest there may be religious or moral grounds to prohibit this method of creating genetically enhanced children, but claims having children at a disadvantage to those around them would cause everyone to eventually pick up the technique. This reasoning rests on a dangerous egalitarian notion of human success and morality driven by competition in a way that forces those who disagree to accept the technique. It doesn’t rest upon morally reasoned principles or the virtues of humanism and research. Much the same way individuals would rapidly evolve under Bostrom’s scheme, the entirety of society should follow suit regardless of choice.

Bostrom’s idea also doesn’t recognize the reality of how natural selection and evolution work. Much like natural selection, this method wouldn’t automatically and instantly choose the most optimal DNA. Instead, it would choose a heritable trait without regard to DNA the same way nature influences which traits are optimal for survival and reproduction. Bostrom’s method relies on knowing which parts of the DNA are responsible for the trait which can be edited directly without the need for cycling through generations. Besides, the genetic basis for these traits have been shown to be limited as the traits themselves are a complicated amalgamation of environmental interactions, genetic pathways, what epigenetic factors activate throughout an individuals’ lifetime, how nature would “select” for certain traits, and the resulting phenotypes. It wouldn’t dictate how a superintelligent human may emerge. The typical issues of the artificial selection process being prone to error and having an inherent natural selection to it raise concerns as well. All of this lies on the inhumane assumption that we may find human perfection through genes regardless of how an objectified individual may control their own fate and what right they have to do so. Indeed, the vacuous claim that “science is moving so fast that ethics just can’t keep up” only measures unethical, unjustified notions of how fast science is moving to begin with.

The ethics of some reproductive technologies become blurrier in light of the newly complex understanding of heredity’s cross-currents. A maternal surrogate, for example, will likely exchange stem cells with the fetus she carries, opening the door to claims that baby and surrogate are related. If the surrogate later carries her own baby, or that of a different woman, are the children related? Parenthood becomes even stranger with so-called mitochondrial-replacement therapy. If a woman with a mitochondrial disorder wants a biological child, it is now possible to inject the nucleus of one of her eggs into a healthy woman’s egg (after removing its nucleus), and then perform in vitro fertilization. The result is a “three-parent baby,” the first of which was born in 2016. Zimmer doesn’t presume to make ethical judgments about procedures such as this, but warns that “informed consent” in such cases can be unexpectedly difficult to determine.

The more honored individuals in bioethics such as Stanford law professor Henry Greely have voiced similar arguments. Greely has argued insurance companies and government agencies can help fund the effective DNA sequencing methods in fighting genetic disease. In his book The End of Sex and the Future of Human Reproduction, he predicts how we may perceive and judge the potential of stem cell technologies. He notes he wouldn’t ban embryos created from a single parent, but would still require pre-implamentation genetic diagnosis to select for optimal offspring. But, above all, Greely emphasizes that these should be closely scrutinized by a standing commission that can recognize what principles all people would believe in. What sort of principles all individuals would agree upon, such as the four common principles of medical ethics: autonomy, beneficence, non-maleficence, and justice, are up for debate. The principles of parenthood should be upheld even for extreme scenarios of the future that may select for the most desirable traits such that biological parenthood becomes meaningless. We must protect every notion of humanity that comes from our current methods of reproduction, both biologically and artificially, to address these issues.

With the growing threat of AI, namely that computers may become more and more Screen Shot 2019-07-06 at 9.19.35 PMhuman-like, our autonomy should reflect how the self has been changing through these innovations. The self can be changed artificially much the same way robots and computers are programmed, but they’re not completely fragmented that humans share nothing with one another. These stem cell methods can give humanity a more unified individualistic self that, when appropriately regulated, allow for even modified individuals to exercise appropriate rights and responsibilities. Greely’s ideas still disgust by suggesting market-driven factors to influence human reproductive choices. The principles that doctors and scientists must stand upon are far too likely to become cold, calculated treatments for dehumanized problems. Professor of public health Annelien L. Bredenoord and professor of bioethics Insoo Hyun argued in “Ethics of stem cell‐derived gametes made in a dish: fertility for everyone?” multiplex parenting will shake the very notions of responsibility and autonomy and moreso than other reproductive techniques. They will disgust individuals, citing ideas of the reaction by professor of philosophy Martha Nussbaum’s book Hiding from Humanity: Disgust, Shame, and the Law. Phyisican Leon Kass noted there is wisdom in this repugnance in his paper “The Wisdom of Repugnance: Why We Should Ban the Cloning of Humans.” No amount of sociological or psychological research into the well-being of multiplex children can prevent this natural sense of disgust that we feel at this idea – and with good reason. We must hold onto this disgust and other aesthetic, physiological responses and assess them to the extent which they provide us with moral clarity. From there, a metamodernistic view of gene editing can take place. Writer Carl Zimmer noted in She Has Her Mother’s Laugh that he doesn’t make ethical judgments about multiplex parenting, but “informed consent” in such cases can difficult to determine.

Reading, learning, and writing about these issues is the first step. For anyone to learn more about science and technology in this age would help spread humanism to fight ignorance. These issues need to enter the sphere of public debate and discussion in contrast to how they’re currently only governed by scientists, ethicists, and philosophers. We need laws in place to prevent these catastrophic consequences long before they occur. In our metamodernist society, we need not reject science and technology entirely. We may remain skeptical of the notions of progress and reality, but only to a point where we can begin a new direction for scientific research. Given the existential crises of genetic engineering and artificial intelligence, we may imagine a moral society through personal development and psychological growth in wrestling with and understanding these struggles. We need a humanized notion of reproduction to address psychological needs of individuals in a society that has the power of gene editing. We can create a grand narrative that mankind has an overarching worldview to connect all humans to one another, but hold it lightly enough to recognize the limits of what we know and should do. We can understand that we may create an idea of “reality” and methods of understanding the world around us that respect scientific research while still questioning the authority of problematic research techniques. By creating a “reality,” we can embrace the truths that we can create a moral society that determines who we are despite the changing self brought upon by genetic engineering and the digital age. We can determine what is honest, authentic, and true without being cynical, showing contempt for the beliefs and sensibilities of others, and turning to eugenics approaches for solutions. Metamodernism, as it begins to infect all areas of life, means our scientific research should seek elegant, morally refined methods that we embrace for knowledge.

Only then will we come closer to our selves in a metamodernist future. The intellectual thought to counter the doomsday dystopia scenarios of the future can involve selecting for desirable traits in offspring through trustworthy, verified methods that acknowledge the rights and responsibilities of the individual. We must remain skeptical of harmful progress, but remain open to gene editing technologies insofar as they may help mankind without raising ethical concerns.

Recognition and representation: two faces of the same coin

twosides

While humans and computers both have the capacity to recognize faces, pattern recognition problems in computer vision seek to represent data in an appropriate way for the problem to solve. Machine learning methods approach pattern recognition as a statistical problem of searching for patterns in data. Through classifying data into different categories, reducing the data’s dimensions, approximating parts of the data, and other techniques that exploiting geometry, algebra, and other quantitative features of the data, computers can recognize faces, handwriting, images, and other visual stimuli in ways similar to humans.

Neural networks and deep learning methods have had practical applications in fingerprint analysis, disease etiology, and voice recognition, to name a few examples. Artificial intelligence continues to find success in various areas of research. But before computers can behave like humans, they need to represent the world in some way. The way computers interpret input data lets them represent the world.

These representation methods include principal component analysis (PCA), in which eigenfaces estimate variance among data. These eigenfaces, mathematical representations of key features used in human face recognition, are calculated using the the covariance matrix of the probability distribution over the high-dimensional vector space of face images. In other words, they let computers discern basic patterns among images of faces. PCA reveals eigenfaces that correspond to the least-squares solution so that the data variance is maintained while getting rid of existing correlations that don’t contribute to it. Generating eigenfaces involves extracting relevant facial information, through methods like searching for statistical variation between images, and representing them efficienctly, such as through using symmetry or other geometric features. While eigenfaces are automatic and easy to code, especially in how they can make complicated faces simple, they can become very sensitive to external features such as lighting and struggle to provide useful information about the faces themselves.

Other machine learning techniques such as classification find categories to map input data by discriminating between different features and representing those features with their distance from one another depending on their similarity. Fisherfaces, named after statistician Ronald Fisher, result from the basis vectors of a subspace representaiton of face images when performing linear discriminant analysis (LDA). Fisherfaces are less sensitive to lighting issues than eigenfaces and require data built upon continuous independent variables, such as skin tone or shape.

While eigenfaces depend upon PCA to account for data variance, fisherfaces use LDA searches for differences between classes of data. A team lead by computer science professor Peter Belhaumer at Columbia University found lower error rates among fisherfaces than among eigenfaces. In their paper, “Eigenfaces vs. Fisherfaces: recognition using class specific linear projection,” they accounted for variations in lighting and facial expressions. Regardless, researchers in computer vision can use both methods to minimize error when solving problems in recognition and representation.

Both eigenfaces and fisherfaces also struggle in capturing changes in expression and emotion among faces. For researchers in computer vision to approach facial expression analysis means understanding the nuance and complexity of the face. Muscular movements originate in the nerves by the VIIth cranial nerve from the brainstem between pons and medulla. The motor root of the nerve gives somatic muscle fibers to the face that create facial expressions. With enough data and efficient routines, pattern recognition can identify emotions from expressions and determine which parts of the brain may be involved in creating them. Neuroscientists have shown face muscles in lower parts of the face are more represented in the motor cortex which are especially involved in speech. From these patterns from hundreds and thousands of faces, computers may soon be be able to discern function from the form of a face itself.

The seven universal facial expressions of emotion (happy, surprise, sadness, fear, anger, contempt, and disgust) have seven ways to regulate themselves (expression, deamplified, neutralized, qualified, masked, amplified, and simulated). Psychologist Paul Ekman determined these emotions and expressions were universal among humans across different countries and levels of industrialization or development in his manuscripts “A New Pan-Cultural Facial Expression of Emotion” and ” The repertoire of nonverbal behavior: Categories, origins, usage, and coding.” He found that humans produced these expressions in response to similar conditions even regardless of how we may judge a face as expressing a certain emotion. The extent to which the emotions are universal, however, remains up to debate. Ekman supported his work by surveying humans across different civilizations, including tribes in New Guinea, but he argued this non-cognitive component is only a part of emotion. This automatic appraisal detects stimuli almost instantly identify elicitors which then activate the seven universal expressions that further cause the physiological elements of the emotional response. This would include any bodily change such as the feeling of one’s heart dropping, skeletal muscles tightening, facial muscles loosening,  changing voice pitch, and other features of the nervous system.

As recognition and representation depend upon one another, neuroscience and artificial intelligence continue to support one another despite their different paths. As expression and emotion intertwine into one another, we create a more nuanced picture of perception that speaks to who we are as humans. While eigenfaces and fisherfaces support similar goals as well, their different methods lets computer vision researchers attend to the variety of challenges deep learning has to offer.

Philosophy faculty empower students to tackle ethical issues in data science

How would Aristotle approach Facebook’s methods?

The future of data science is a mystery surrounded by concerns about autonomy, responsibility, and other ethical issues related to technology. The recent Facebook-Cambridge Analytica data scandal raised concerns of personal privacy, ethical principles for social media businesses, as well as misinformation.

In his single-credit PHIL 293 course “Ethics for Data Sciences”, assistant professor of philosophy Taylor Davis challenges students to re-think what they believe about morality in the wake of these controversies in science and technology.

In light of advancing technologies in artificial intelligence and machine learning, Davis said it is becoming more clear that ethical training is necessary for data science.

“Broadly what is happening, especially we see Mark Zuckerberg testify in front of Congress, is ethics pervades all of our lives as we become more reliant on it in many ways,” Davis said.

Partnered with the Integrative Data Science Initiative, faculty are planing to make the course three credits with an online version, said Matthew Kroll, post-doctoral researcher in philosophy, and even advance ethics through other initiatives.

“One of the questions I start the class with is ‘What does it mean to be ethical in the age of the big data?'” Kroll said. The students discuss current events like the Facebook Cambridge Analytica data scandal and individuals and organizations should have done.

Kroll has also previously taught the course by giving students a “tool kit” of philosophical methods of reasoning to address these issues as well as concerns of rights and autonomy with respect to artificial intelligence and autonomous cars.

“If you’ve never taken an ethics course, you at least get to dip your toes in the water of what ethics is,” Kroll said.

By teaching students how to form philosophical arguments and put issues in historical perspectives, Kroll hopes to teach students how to make better ethical decisions.

“If these students feel like in their professional lives they reach a moment or a threshold that, if there’s an ethical issue in their workplace like ‘Are we gonna sell data to a particular government interest?’ to at least say ‘I don’t think this is right’ or ‘I think this is an unethical use of user data,'” Kroll said. “If one, two, or three students do that, then I’ll feel like the class is a success.”

Through an applied ethics course, Davis emphasized philosophy runs in the background as he instructs. In contrast to the standard philosophy-based methods of beginning with general, theoretical principles, he delivers current events as case studies.

Davis created the course by substituted data science with engineering from an engineering ethics course.

With a case-study based bottom-up approach, students begin with real world problems then figure out how to reason with those actions in mind. From these methods of isolating actions from agents and forming arguments, they learn how ethical principles such as not causing unnecessary harm to others and respecting personal rights come into play.

Through these methods, students find truth and clarity on tricky issues that they may apply to other problems such as autonomy of self-driving cars and gene editing technologies, Kroll said.

“Machine learning algorithms perform calculations in opaque methods,” Davis said. “They create a complicated model that researchers need to understand.” Davis said this would let researchers determine how to resolve social inequality issues related to women and minorities or forming predictions on individual habits. He wants students to consider how similar people are to robots that may perform human-like actions and how that raises issues about personal agency.

“The idea is to give people that kind of training to see and identify ethical issues,” Davis said.

Students will continue to grapple with what it means to be human in the “age of big data”, Kroll said. “The ethics in data science instruction serves to empower students in their future careers.”

Digital Divide Index measures economic disparities across Indiana

The lack of high-speed internet access in Indiana’s rural areas means limited access to educational and employment opportunities. Robert Gallardo, a community regional economics specialist at Purdue University, wants to fix that.

Sixteen percent of Hoosiers live without broadband internet access, down 2% since 2017, according to the advocacy website, BroadbandNow. The Southeastern Indiana Regional Planing Commission (SIRPC) 2019 “taste of Broadband” report revealed more than 90 percent of residents in Franklin and Switzerland counties do not use 25 Mbps and Ohio County has the highest property info households without computing devices at 22.3 percent.

This is partly due to the work of Gallardo, including how he and a team of researchers created the Digital Divide Index (DDI) to quantify these disparities. Ranging from 0 to 100, with 100 indicating the largest digital divide, the DDI quartiles showed the highest disparities among west, south, and southwest counties, according to the Digital Divide in Indiana report published by Gallardo and other researchers. Meanwhile, the smallest divides were located in the northwest and central counties.

Composed of scores measuring the infrastructure and adoption of broadband and the socioeconomic characteristics that effect technology adoption, the DDI can differnetiate 25/3 access and internet sped the former category and educational level, poverty rate, and elderly or disabled population in the latter.

The report found greater decreases in working age population of counties with the highest DDI when compared with the state and national averages over the course of 2010 to 2015. These counties also experienced smaller increases in the number of jobs and estbalishments.

As the 25th most connected state, according to BroadbandNow, there are 237 internet providers in Indiana. Some 757,000 people in Indiana do not have access to a 25 Mpbs wired connection. Bridging the digital divide means using the DDI statistic to understand disparities in the state, Gallardo says.

That is a problem holding the state back, Gallardo says. Providing broadband internet access to all ares of Indiana, he says, woods provide individual with equal access to a global economy. It also would give communities the ability to communicate and work with more commercial and residential developers throughout the entire state. It would improve the quality of life for the residents affected and, as technology advances, give everyone equal access to opportunities.

Understanding how internet access affects the nation as a whole means defining the nature of internet speeds. I n2015, the Federal Communications Commission rose the minimum of what qualifies as broadband service. The agency voted to raise the download speed threshold from 4 megabits per second (Mbps) to 25 Mbps and upload speed from 1-3 Mbps. The nation recognizes this “25/3” speed as broadband speed.

Gallardo said over the past few years, researchers have gained access to granular data – data that can be divided and analyzed for the most basic feathers. He has described how the broadband infrastructure in Indiana continues to improve. He has stressed the importance of putting data in context and coming up with ways to cross-check and validate what the data means.

Gallardo said he hopes the work can create more consistent and widespread resources as the SIRPC report found 45% of individuals did not have access to internet for five or more days due to unpaid bills, broken devices, or similar reasons due to lack of resources.

“We’re trying to translate that data where it can be used for meaningful discussions,” Gallardo said. “This report is crunch gin all these numbers just so that you and I in the region can say, ‘OK, what can we do about it?'”

In 2014, 83 percent of housing units in Indiana had access to the 25/3 broadband standard. That percentage is now at 90.4 percent, but Gallardo warned there are statistical shortcomings in the ways these data are collected such that they have not been appropriately validated.

“The problem with that dataset is that it’s user-provided,” Gallardo said, “not checked by a federal agency.”

Susan Craig, executive director of the Southeastern Indiana Regional Planning Commission, said she’s been working with Gallardo since last Fall in serving nine rural counties in Southeastern Indiana.

“There were a lot of conversations on the local level before it was finalized to go with the studies,” Craig said about Gallardo’s research of the socioeconomic disparities across Indiana.

Gallardo also said the data may represent what the speeds re advertised as rather than what they actually are.

Still, the project involves legal and financial competence through public-private partnerships. These large-scale processes depend upon these complex factors interfacing with one another. Solving Indiana’s broadband issues means creating a discussion between policymakers and the public as well, Gallardo says.

Scott Rudd, director of broadband opportunities fro Indiana, works with Gallardo on attracting broadband providers to various counties. With help from the Indiana Lt. Gov Suzanne Crouch, Rudd held the 2018 Broadband Summit, “Connecting Hoosier Communities” last October with one of the goals to improve the dialogue between different groups of people such as stakeholders across the state.

Gallardo said,” There is no one-size fits-all model when deploying or upgrading broadband infrastructure.”

Data-based analysis reveals data science job prospects

With Purdue University’s Data Science Initiative emphasizing on students having the best skills necessary for jobs, here’s a data-driven look at what the job market will look like.

The job description data came from studies by Corey Seliger, Director of Integration and Data Platforms, ITaP and the web scraping service PromptCloud. The data sets that scraped job posting websites were obtained through the Purdue University Research Repository and the machine learning community Kaggle.

Though Purdue does not offer as many courses from their Data Science Initiative when compared to other schools, this is less of a statement about Purdue’s Data Science Initiative and more of what universities classify as “data science” courses. The interdisciplinary nature of Purdue’s Data Science efforts meant that many courses through various other departments were not counted as data science courses. This analysis did not include, for example, courses from biology or neuroscience departments that may be considered intensive with data analysis. 

The Statistics Living-Learning Community’s publication and presentation record since their inception shows the influence the Data Science Initiative has had in getting students’ work published. 

The majority of them are in big cities like Chicago and San Francisco. These cities show where the future data science jobs are likely to be. They can be classified with corresponding metrics of predictability.

More data science jobs require a Ph.D. when compared to other education qualifications of Bachelor’s degrees or Master’s degrees. 16,618 require a Ph.D, 10,695 required a Bachelor’s, and 8,759 required a Master’s. Over half of the jobs listed, 82,726 contained no educational requirements.

The associated wordcloud provides another glimpse into the words these job postings most commonly use. Though these words may be used frequently throughout the job postings, their importance to data science as a whole shows by their usage.

The wordcloud shows relative frequencies of each keyword from the job descriptions. Key phrases like “practical experience,” “complex data,” and “strategic tactical” show the types of skills data science jobs require. That jobs requiring these skills means students seeking data science careers should understand them for their career goals. 

The data was taken from over 90,000 job postings of data science jobs. The job postings contained various information including salary, background, experience, and personal skills relevant to the job. They emphasized specific skills they look for in applicants and the types of experience required. 

Latent Dirichlet allocation (LDA) is a generative statistical model to use unobserved groups that explain why parts of the data are similar to make observations. The LDA analysis of data science job prospects can be found here. With LDA, we observe how keywords or phrases cluster among themselves in groups of principal components. LDA has uses in Natural Language processing and topic modeling for large amounts of text. Modeling texts on various ways their keywords cluster can be used to predict books to recommend to someone based on their previous reading. Publications can model topics to extract key features from articles or cluster articles that are similar close together. 

LDA involves estimating topic assignments for key topics in the text. For each document, an amount of topics is chosen and words are generated according to the Dirichlet distribution. Then the model works backwards to figure out how to find a set of topics and corresponding distribution that could generate the document’s content. 

It determines how document topics vary among themselves, the words associated with each topic, and the phi value. The phi value is how likely a word belongs to a certain topic. The script, written in the python programming language, takes about 20 seconds to extract and analyze data while outputting results nearly 100,000 job descriptions. The speed of the pipeline came through programming techniques to take advantage of the the similarity between the input files. A multi-threaded procedure was used to run these tasks in parallel with one another and a search approach that saves memory by directly converting the file text to manageable data reduced the pipeline’s runtime from several hours to less than a minute.

Through various sampling methods, LDA can learn how to improve its method of determining the document’s content. We can train the LDA on various known sets of data to learn how to extract features and form predictions. 

The process included parsing each file for keywords. This meant removing unnecessary words (such as “and” and “the”) as well as combining similar words into one another (such as “analysis” and “analyze”). 

The LDA models can be further optimized for use in sampling such that predictions may be formed on future job prospects in data science. Tools to extract data science job information would prove beneficial for improving the LDA model. Education curricula can reflect these trends in a fluctuating job market. Future work can include finding more varied sets of data such as including machine learning and artificial intelligence keywords in addition to data sciences ones. 

Using a support vector machine for classification and data-fitting methods training and testing, the models can be used as the basis for forming predictions on the data science job market. The entirety of these techniques can be found here: https://github.com/HussainAther/journalism/blob/master/src/python/postings/postingAnalysis.py.

Deep convolutional neural networks as models of the visual system

We can discuss convolutional neural networks (CNNs) as they relate to biological vision in a way that captures how close these neural networks simulate biological reality and how they depend on artificial techniques. Understanding the relationships between CNNs and the visual system will bring us closer to determining how deep learning relates to the brain. CNNs have shown exceptional results in machine learning techniques involving visual systems such as encoding and decoding of the human brain.

CNNs are a class of artificial neural networks. They are comprised of units called neurons, which take in a weighted sum of inputs and output an activity level. The activity level is always a nonlinear function of the input, frequently just a rectified linear unit (ReLU) where the activity is equal to the input for all positive input and 0 for all non-positive input. hey depend upon a convolution, as the name would suggest, that reduces the number of weights of a given layer as a method reducing the free parameters. This lets them use methods like backpropagation for weighing various elements at each layer. This is the primary method neural networks learn, but researchers have debated how close they are to biological neurons.  The backpropagation algorithm is the cornerstone of deep learning. Despite its importance, few variations of the algorithm have been attempted. CNNs can respond to the issues of multi-layer perceptrons (MLPs) with the strong spatially local correlation present in natural images. With three-dimensional volumes of neurons, local connectivity between neurons of adjacent layers, and sharing weights across entire visual fields, CNNs can generalize themselves better in vision problems.

In a feedforward neural network, like CNNs, units are organized into layers and the units at a given layer only get input from units in the layer below (i.e. no inputs from other units at the same layer, later layers, or—in most cases—layers more than one before the current layer). However unlike standard vanilla feedforward networks, units in a CNN have a spatial arrangement. At each layer, units are organized into 2-D grids called feature maps. Each of these feature maps is the result of a convolution (hence the name) performed on the layer below. The parameters of the neural network have a relationship with the error the network produces, and when the parameters change, the error does, too. Researchers change the parameters to minimize error using optimization algorithms. This means that the same convolutional filter (set of weights) is applied at each location in the layer below. Therefore a unit at a particular location on the 2-D grid can only receive input from units at a similar location at the layer below. Furthermore, the weights attached to the inputs are the same for each unit in a feature map and different across feature maps.

After the convolution and nonlinearity, a few other computations are usually done. One possible computation, though no longer popular in modern high-performing CNNs, is cross-feature normalization. Here the activity of a unit at a particular spatial location in a feature map is divided by the activity of units at the same location in other feature maps. A more common operation is pooling. Here, the maximum activity in a small spatial area of each each 2-D feature map grid is used to represent that area. This shrinks the size of the feature maps. This set of operations (convolution+nonlin[—->normalization]—> pooling) is collectively referred to as a “layer.” The architecture of a network is defined by the number of layers and choices about various parameters associated with them (e.g. the size of the convolutional filters, etc). After obtaining features using convolution, researchers use them for classification. In theory, one could use all the extracted features with a classifier such as a softmax function, but this can be computationally challenging.

Most modern CNNs have at least five of these layers, the final of which feeds into a fully-connected layer. Fully-connected layers are like standard feedforward networks in that they do not have a spatial layout or restricted connectivity. Frequently 2-3 fully connected layers are used in a row and the final layer of the network performs a classification. If the network is performing a 10-way object classification, for example, the final layer will have 10 units and a softmax function will be applied to their activity levels to produce a probability associated with each category.

These networks are largely trained with supervised learning and backpropagation. Here, pairs of images and their associated category label are given to the network. Image pixel values feed into the first layer of the network and the final layer of the network produces a predicted category. If this predicted label doesn’t match the provided one, gradients are calculated that determine how the weight ,i.e. the values in the convolutional filters, should change in order to make the classification correct. Most of these networks are trained on the ImageNet database which contains over 1 million images from 1000 object categories produces models that can have very high levels of accuracy on held-out test images. Variants of CNNs now reach 4.94% error rates or lower, better than human-level performance. Many training “tricks” are usually needed to get this to work well such as smart learning rate choice and weight regularization mostly via dropout, where a random half of the weights are turned off at each training stage.

Historically, unsupervised pre-training was used to initialize the weights, which were then refined with supervised learning. However, this no longer appears necessary for good performance. For an in-depth neuroscientist-friendly introduction to CNNs, check out Deep Neural Networks: A New Framework for Modeling Biological Vision and Brain Information Processing.

First, artificial neural networks as whole were inspired—as their name suggests—by the emerging biology of neurons being developed in the mid-20th century. Artificial neurons were designed to mimic the basic characteristics of how neurons take in and transform information.

Second, the main features and computations done by convolutional networks were directly inspired by some of the early findings about the visual system. In 1962 Hubel and Wiesel discovered that neurons in primary visual cortex respond to specific, simple features in the visual environment (particularly, oriented edges). Furthermore, they noticed two different kinds of cells. There were simple cells—which responded most strongly to their preferred orientation only at a very particular spatial location—and complex cells—which had more spatial invariance in their response. They concluded that complex cells achieved this invariance by pooling over inputs from multiple simple cells, each with a different preferred location. These two features (selectivity to particular features and increasing spatial invariance through feedforward connections) formed the basis for artificial visual systems like CNNs.

This discovery can be directly traced to the development of CNNs through a model known as the Neocognitron. This model, developed in 1980 by computer scientist Kunihiko Fukushima, synthesized the current knowledge about biological visual in an attempt to build a functioning artificial visual system. The neocognitron is comprised of “S-cells” and “C-cells” and learns to recognize simple images via unsupervised learning. Layers of “S-cells” and “C-cells” are arranged alternately in the hierarchical network. Yann LeCun, the AI researcher who initially developed CNNs, explicitly states that they have their roots in the neocognitron.

Throughout the history of computer vision, much work focused on hand-designing the features that were to be detected in an image, based on beliefs about what would be most informative. After filtering based on these handcrafted features, learning would only be done at the final stage, to map the features to the object class. CNNs trained end-to-end via supervised learning thus offered a way to auto-generate the features, in a way that was most suitable for the task.

The first major example of this came in 1989. When LeCun et al. trained a small CNN to do hand-written digit recognition using backpropagation. Further advances and proof of CNN abilities came in 1999 with the introduction of the MNIST dataset. Despite this success, these methods faded from the research community as the training was considered difficult and non-neural network approaches (such as support vector machines) became the rage.

The next major event came in 2012, when a deep CNN trained fully via supervised methods won the annual ImageNet competition. At this time a good error rate for 1000-way object classification was ~25%, but AlexNet achieved 16% error, a huge improvement. Previous winners of this challenge relied on older techniques such as shallow networks and SVMs. The advance with CNNs was aided by the use of some novel techniques such as the use of the ReLu (instead of sigmoid or hyperbolic tangent nonlinearities), splitting the network over 2 GPUs, and dropout regularization. This did not emerge out of nothing however, as a resurgence in neural networks can be seen as early as 2006. Most of these networks, however, used unsupervised pre-training. This 2012 advance was definitely a huge moment for the modern deep learning explosion.

As explained in Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review, Much of the hype about CNNs in neuroscience today stems from a few studies published in ~2014. These studies explicitly compared neural activity recorded from humans and macaques to artificial activity in CNNs when the different systems were shown the same images.

The first is Yamins et al. (2014). This study explored many different CNN architectures to determine what leads to a good ability to predict responses of monkey IT cells. For a given network, a subset of the data was used to train linear regression models that mapped activity in the artificial network to individual IT cell activity. The predictive power on held-out data was used to assess the models. A second method, representational similarity analysis, was also used. This method does not involve direct prediction of neural activity, but rather asks if two systems are representing information the same way. This is done by building a matrix for each system, wherein the values represent how similar the response is for two different inputs. If these matrices look the same for different systems, then they are representing information similarly.

By both measures, CNNs optimized for object recognition outperformed other models. Furthermore, the 3rd layer of the network better predicted V4 cell activity while the 4th (and final) layer better predicted IT. Indicating a correspondence between model layers and brain areas.

Another finding was that networks that performed better on object recognition also performed better on capturing IT activity, without a need to be directly optimized on IT data. This trend has largely held true for larger and better networks, up to some limits.

Another paper, Khaligh-Razavi and Kriegeskorte (2014), also uses representational similarity analysis to compare 37 different models to human and monkey IT. They too found that models better at object recognition better matched IT representations. Furthermore, the deep CNN trained via supervised learning (“AlexNet”) was the best performing and the best match, with later layers in the network performing better than earlier ones.

The neocognitron model was inspired by the findings of Hubel and Wiesel and went on to inspire modern CNNs, but it also spawned a branch of research in visual neuroscience recognized perhaps most visibly in the labs of Tomaso Poggio, Thomas Serre, Maximilian Riesenhuber, and Jim DiCarlo, among others. Models based on stacks of convolutions and max-pooling were used to explain various properties of the visual system. These models tended to use different nonlinearities than current CNNs and unsupervised training of features as was popular in machine learning at the time as well, and they didn’t reach the scale of modern CNNs.

The path taken by visual neuroscientists and computer vision researchers has variously merged and diverged, as they pursued separate but related goals. But in total, CNNs can readily be viewed as a continuation of the modeling trajectory set upon by visual neuroscientists. The contributions from the field of deep learning relate to the computational power and training methods (and data) that allowed these models to finally become functional. CNNs have three main traits that support their use as models of biological vision. They can perform visual tasks at near-human levels, they do this with an architecture that replicates basic features known about the visual system, and they produce activity that is directly relatable to the activity of different areas in the visual system

To start, by their very nature and architecture, they have two important components of the visual hierarchy. First, receptive field sizes of individual units grow as we progress through the layers of the network just as they do as we progress from V1 to IT. Second, neurons respond to increasingly complex image features as we progress through the layers just as tuning goes from simple lines in V1 to object parts in IT. This increase in feature complexity can be seen directly through visualization techniques available in CNNs.

Looking more deeply into the third reason, many studies subsequent to the original 2014 work have further established the relationship between activity in CNNs and the visual systems. These all show the same general finding. The activity of artificial networks can be related to the activity of the visual system when both are shown the same images. Furthermore, later layers in the network correspond to later areas in the ventral visual stream or later time points in the response when using methods such as magnetoencephalography.

Many different methods and datasets have been used to make these points, as can be seen in the following studies: Seibert et al. (2016), Cadena et al. (2017), Cichy et al. (2016), Wen et al. (2018), Eickenberg et al. (2017), Güçlü and van Gerven (2015), and Seeliger et al. (2017). Correlation between the representations at different CNN layers and brain areas from Cichy et al.

The focus of these studies is generally on the initial neural response to briefly-presented natural images of various object categories. Thus, these CNNs are capturing what’s been referred to as “core object recognition,” or “the ability to rapidly discriminate a given visual object from all other objects even in the face of identity-preserving transformations (position, size, viewpoint, and visual context).” In general, standard feedforward CNNs best capture the early components of the visual response, suggesting they are replicating the initial feedforward sweep of information from retina to higher cortical areas.

The fact that the succession of neural representations created by the visual system can be replicated by CNNs suggests that they are doing the same “untangling” process. That is, both systems take representations of different object categories that are inseparable at the image/retinal level and create representations that allow for linear separability.

In addition to comparing activities, we can also delve deeper into the first reason i.e., the performance of the network. Detailed comparisons of the behavior of these networks to humans and animals can further serve to verify their use as a model and identify areas where progress is still needed. The findings from this kind of work have shown that these networks can capture patterns of human classification behavior better than previous models in multiple domains and even predict or manipulate it, but fall short in certain specifics such as how performance falls off with noise, or when variations in images are small.

Such behavioral effects have been studied in Rajalingham et al. (2018), Kheradpishesh et al. (2015), Elsayed et al. (2018), Jozwik et al. (2017), Kubilius et al. (2016), Dodge and Karam (2017), Berardino et al. (2017), and Geirhos et al. (2017).

Whether all this meets the specification of a good model of the brain is probably best addressed by looking at what people in vision have said they wanted out of a model of the visual system.

“Progress in understanding the brain’s solution to object recognition requires the construction of artificial recognition systems that ultimately aim to emulate our own visual abilities, often with biological inspiration (e.g., [2–6]). Such computational approaches are critically important because they can provide experimentally testable hypotheses, and because instantiation of a working recognition system represents a particularly effective measure of success in understanding object recognition.” – Pinto et al. (2007).

From this perspective it’s clear that CNNs do not represent a moving of the target in vision science, but more a reaching of it. Several studies have directly compared the ability of CNNs and previous models of the visual system (such as HMAX) to capture neural activity. CNNs come out on top. Such studies include Yamins et al. (2014), Cichy et al. (2017), and Cadieu et al. (2014). A reasonable definition of a mechanistic model is one in which internal parts of the model can be mapped to internal parts of the system of interest. Descriptive models, on the other hand, are only matched in their overall input-output relationship. So a descriptive model of the visual system may be one that takes in an image and outputs an object label that aligns with human labels, but does so in a way that has no obvious relation to the brain. As described above, however, layers of a CNN can be mapped to areas of the brain. Therefore, CNNs are mechanistic models of the representational transformation carried out by the ventral system as it does object recognition.

For a CNN to, as a whole, be a mechanistic model does not require that we accept that all sub-components are mechanistic. They can be compared to rate-based neurons in traditional circuit models of the brain. Rate-based neural models are simply a function that maps input strength to output firing rate. As such, these are descriptive models of neurons. There are no internal components of the model that relate to the neural processes that lead to firing rate. Detailed bio-physical models such as Hodgkin-Huxley neurons would be mechanistic. Yet we can still use rate-based neurons to build mechanistic models of circuit. All mechanistic models rely on descriptive models as their base unit. Otherwise we’d all need to get down to quantum mechanics to build a model.

Whether the components of the CNN (i.e. the layers—comprised of convolutions, nonlinearities, possibly normalization, and pooling) are mechanistic or descriptive models of brain areas is difficult to determine. While these layers are comprised of artificial neurons which could plausibly be mapped to (groups of) real neurons, the implementations of many of the computations are not biological. For example, normalization (in the networks that use it) is implemented with a highly-parameterized divisive equation. We believe that these computations can be implemented with realistic neural mechanisms (see the above-cited example network), but those are not what are at present used in these models. For neuroscientists used to dealing with things on the cellular level, models like CNNs may feel abstracted beyond the point of usefulness. Cognitive scientists who have worked with abstract multi-area modeling for some time though may find them more familiar.

But even without exact biological details, we can still map components of the CNN to components of the visual system. First, inputs to a CNN are usually 3-D (RGB) pixel values that have been normalized or whitened in some way, roughly corresponding to computations performed by the retina and lateral geniculate nucleus. The convolutions create feature maps that have a spatial layout, like the retinotopy found in visual areas, which means that each artificial neuron has a spatially-restricted receptive field. The convolutional filter associated with each feature map determines the feature tuning of the neurons in that feature map. Individual artificial neurons are not meant to be mapped directly to individual real neurons; it may be more reasonable to think of individual units as cortical columns.

Which layers of the CNN correspond to which brain areas? The early work using models that only contained a small number of layers provided support for a one layer to one brain area mapping. For example, in Yamins et al. (2014), the final convolutional layer best predicts IT activity and the second to last best predicts V4. The exact relationship, however, will depend on the model used (with deeper models allowing for more layers per brain area).

The fully connected layers at the end of a CNN have a more complicated interpretation. Their close relationship to the final decision made by the classifier and the fact that they no longer have a retinotopy makes them prefrontal cortex-like. But they also may perform well when predicting IT activity.

Of course these are features that the most standard CNNs used as models today don’t have by default. But many of them have already been worked into newer CNN models, such as skip connections, feedback connections, saccades, spikes, lateral connections, and a fovea.

So clearly CNNs are not direct replicas of primate vision. It should also be clear that this fact is not disqualifying. No model will be or should be a complete replica of the system of interest. The goal is to capture the features necessary to explain what we want to know about vision. Different researchers will want to know different things about the visual system, and so the absence of a particular feature will matter more or less to one person than to someone else. Which features are required in order to, say, predict the response of IT neurons averaged over the first ~100ms of image presentation? This is an empirical question. We cannot say a priori that any biological feature is necessary or that the model is a bad one for not having it.

We can say that a model without details such as spiking, E-I types, and other implementation specifics is more abstract than one that has those. But there’s nothing wrong with abstraction. It just means we’re willing to separate problems out into a hierarchy and work on them independently. One day we should be able to piece together the different levels of explanation and have a model that replicates the brain on the large and fine scale.

This first issue is that convolutional weights are positive and negative. This means that feedforward connections are excitatory and inhibitory (whereas in the brain connections between brain areas are largely excitatory) and that individual artificial neurons can have excitatory and inhibitory influences. This is not terribly problematic if we simply consider that the weights indicate net effects, which may in reality be executed via feedforward excitatory connections to inhibitory cells.

Next, weights are shared. This means that a neuron at one location in a feature map uses the exact same weights on its inputs as a different neuron in the same feature map. While it is the case that something like orientation tuning is represented across the retinotopic map in V1, we don’t believe that a neuron that prefers vertical lines in the one area of visual space has the exact same input weights as a vertical-preferring neuron at another location. There is no “spooky action at a distance” that ensures all weights are coordinated and shared. Thus, the current use of weight sharing to help train these networks should be able to be replaced by a more biologically plausible way of creating spatially-invariant tuning.

Third, what’s up with max-pooling? The max-pooling operation is, in neuroscience terms, akin to a neuron’s firing rate being equal to the firing rate of its highest firing input. Because neurons pool from many neurons, it’s hard to devise a neuron that could straightforwardly do this. But the pooling operation was inspired by the discovery of complex cells and originally started as an averaging operation, something trivially achievable by neurons. Max-pooling, however, has been found to be more successful in terms of object recognition performance and fitting biological data and is now widely used.

The further development of CNNs by machine learning researchers have taken them even farther away from the visual system (as the goal for ML people is performance alone). Some of the best performing CNNs now have many features that would seem strange from a biological perspective. Furthermore, the extreme depths of these newer models (~50 layers) has made their activity less relatable to the visual system.

There is also of course the issue of how these networks are trained via backpropagation. One of the main reasons I’m a computational neuroscientist is because (without the constraints of experimental setups) we can do whatever we want. So, yes! We can make standard CNNs have more biology-inspired features. Let’s see what’s been done so far.

Many architectural elements have been added to different variants of CNNs, which make them more similar to the ventral stream. Furthermore, work has been done to increase the plausibility of the learning procedure.

Some more specific work to replicate biological details includes Spoerer et al. (2017) which, inspired by biology, shows how adding lateral and feedback connections can make models better at recognizing occluded and noisy objects. Costa et al. (2017) implemented long-short-term-memory networks using biologically-inspired components. LSTMs are frequently used when adding recurrence to artificial neural networks, so determining how their functions could be implemented biologically is very useful.

Backpropagation involves calculating how a weight anywhere in the network should change in order to decrease the error that the classifier made. It means that a synapse at layer one would have some information about what went wrong all the way at the top layer. Real neurons, however, tend to rely on local learning rules such as Hebbian plasticity, where the change in weight is determined mainly by the activity of the pre- and post-synaptic neuron, not any far away influences. Backpropagation does not seem biologically realistic.

This doesn’t need to impact our interpretation of the fully-trained CNN as a model of the visual system. Parameters in computational models are frequently fit using techniques that aren’t intended to bear any resemblance to how the brain learns, for example, Bayesian inference to get functional connectivity. Yet that doesn’t render the resulting circuit model uninterpretable. In an extreme view, then, we can consider backpropagation as merely a parameter-fitting tool like any other. And indeed, Yamins et al. (2014) did use a different parameter fitting technique – not backpropagation.

However taking this view does mean that certain aspects of the model are not up for interpretation. For example, we wouldn’t expect the learning curve (that is, how the error changes as the model learns) to relate to the errors that humans or animals make when learning.

While the current implementation of backpropagation is not biologically-plausible, it could be interpreted as an abstract version of something the brain is actually doing. Various efforts to make backpropagation biologically plausible by implementing it with local computations and realistic cells types are underway (for example, this and this). This would open up the learning process to better biological interpretation. Whether the use of more biologically plausible learning procedures leads to neural activity that better matches the data is an as-yet-unanswered empirical question.

On the other hand, unsupervised learning seems a likely mechanism for the brain as it doesn’t require explicit feedback about labels, but rather uses the natural statistics of the environment to develop representations. Thus far, unsupervised learning has not been able to achieve the high object categorization performance reached by supervised learning. But advances in unsupervised learning and methods to make it biologically plausible may ultimately lead to better models of the visual system. Nothing can be learned from CNNs in isolation. All insights and developments will need to be verified and furthered through an interaction with experimental data. That said, there are three ways in which CNNs can contribute to our understanding of the visual system.

The first is to verify our intuitions. To paraphrase Feynman “we don’t understand what we can’t build.” For all the data collected and theories developed about the visual system, why couldn’t neuroscientists make a functioning visual system? That should be alarming in that it signifies we were missing something crucial. Now we can say our intuitions about the visual system were largely right, we were just missing the computing power and training data.

The second is to allow for an idealized experimental testing ground. This is a common use of mechanistic models in science. We use existing data to establish a model as a reasonable facsimile of what we’re interested in. Then we poke, prod, lesion, and lob off parts of it to see what really matters to the functioning. This serves as hypothesis generation for future experiments and/or a way to explain previous data that wasn’t used to build the model.

The third way is through mathematical analysis. As is always the case with computational modeling, putting our beliefs about how the visual system works into concrete mathematical terms opens them up to new types of investigation. While doing analysis on a model usually requires simplifying it even further, it can still offer helpful insights about the general trends and limits of a model’s behavior. In this particular case, there is extra fire power here because some ML researchers are also interested in mathematically dissecting these models. So their insights can become ours in the right circumstance (for example). First, we verified our intuitions by showing they can actually build a functioning visual system. Furthermore, this approach has helped us to define the (in Marr’s terms) computational and algorithmic levels of the visual system. The ability to capture so much neural and behavioral data by training on object recognition suggests that is a core computational role of the ventral stream. And a series of convolutions and pooling is part of the algorithm needed to do it.

The success of these networks has also, I believe, helped allow for a shift in what we consider the units of study in visual neuroscience. Much of visual neuroscience has been dominated by an approach that centers individual cells and their tuning preferences. Abstract models that capture neural data without a strict one neuron-to-one neuron correspondence put the focus on population coding. It’s possible that trying to make sense of individual tuning functions would someday yield the same results, but the population-level approach seems more efficient.

Furthermore, viewing the visual system as just that—an entire system—rather than isolated areas, reframes how we should expect to understand those areas. Much work has gone into studying V4, e.g., by trying to describe in words or simple math what causes cells in that area to respond. When V4 is viewed as a middle ground on a path to object recognition, it seems less likely that it should be neatly describable on its own. From this review, “A verbal functional interpretation of a unit, e.g., as an eye or a face detector, may help our intuitive understanding and capture something important. However, such verbal interpretations may overstate the degree of category and localization, and understate the statistical and distributed nature of these representations.” Indeed, analysis of trained networks has indicated that strong, interpretable tuning of individual units is not correlated with good performance, suggesting the historic focus on that has been misguided.

Some more concrete progress is coming from exploring different architectures. By seeing which details are required for capturing which elements of the neural and behavioral response, we can make a direct connection between structure and function. In this study, lateral connections added to the network did more to help explain the time course of the dorsal stream’s response than the ventral stream’s. Other studies have suggested that feedback connections will be important for capturing ventral stream dynamics. It’s also been shown that certain components of the neural response can be captured in a model with random weights, suggesting the hierarchical architecture alone can explain them. While other components requiring training on natural and valid image categories.

Furthermore, observing that certain well-performing CNNs are not capable of accurately predicting neural activity is important because it indicates that not all models that do vision will be good models of the brain. This lends credence to the idea that the architectures that we have seen predict neural activity well (with a correspondence between brain areas and layers) do so because they are indeed capturing something about the transformations the brain does.

Because CNNs offer an “image computable” way of generating realistic neural responses, they are also useful for relating lesser understood signals to visual processing, as has been done here and here for contextualizing oscillations. Using CNNs as a model of the visual system, we can demonstrate that the feature similarity gain model (which describes the neural impacts of attention) can explain attention’s beneficial performance effects. Finally, some studies have documented neural or behavioral elements not captured by CNNs. These help identify areas that need further experimental and computational exploration.