Anonymity. Does anyone have it?

This post is by North Carolina A&T grad student Siobahn Day

SiobahnD-0144-cropGreetings! My name is Siobahn Day. I’m currently a PhD student in the Computer Science at North Carolina A&T State University. I work as a graduate researcher in the Center for Advanced Studies in Identity Science. I have developed the concept of Adversarial Authorship as a means of preserving author anonymity. I’m currently developing and evaluating an Interactive Evolutionary Computation for Adversarial Authorship which allows users to conceal their writing style.

This research is particularly important to me because as technology has advanced over the years, our laws have not (US). Due to the rapid growth of the internet and social networks it’s very hard for one to have anonymity. As a result, many Anonymous Social Network (ASNs) have arose. Some believe that privacy is dead and I’d like to see what could be done to change that outlook. I’m excited to share with you some of my current research and snippets of a publication that will appear in the The 25th International Conference on Computer Communication and Networks (ICCCN 2016) proceedings later this year. BEACON has given me new and innovative ways at looking at my problem in order to find an effective solution.

Over the last few years, we have seen an increase in the number of Anonymous Social Networks (ASNs). What many internet users may not know is that their writing style can be tracked across the internet and even through an ASN. The good news is that by using a technique referred to as Adversarial Stylometry one can effectively imitate the writing style of another or even obfuscate their own writing style in an effort to conceal their true writing style – for a short term. The bad news is that recent research has shown that Adversarial Stylometry is not effective in concealing ones writing style over the long term. We introduce a number of underlying concepts that will allow users to conceal their writing style over the long term. One such concept we refer to as Adversarial Authorship.

In Adversarial Authorship, authors are provided an AuthorWeb which allows them to see graphically how their writing style compares with others in the AuthorWeb. The AuthorWeb presented uses Entropy-based Evolutionary Clustering (EBEC) in an effort to cluster writing styles. Our results show that EBEC outperforms a number of other machine learning techniques for author recognition. Users of an AuthorWeb can then write to user-specified clusters in an effort to conceal their writing style.

If this research interests you in any way, feel free to read my previous publication: Towards the Development of a Cyber Analysis & Advisement Tool (CAAT) for Mitigating De-Anonymization Attacks . You can also visit my research team at The Center for Advanced Studies in Identity Science. I look forward to continuing to share with BEACON much more of my research as it evolves.

Posted in Uncategorized | Leave a comment

Evolving Deep Neural Networks

This post is by UT Austin grad student Jason Liang

Deep learning has revolutionized the field of machine learning in many ways. From achieving state-of-the-art results in many benchmarks and competitions to effectively exploiting the computational power of the cloud, deep learning has received widespread attention not just in academia but also in industry. Deep learning has helped researchers and scientists obtain state-of-the-art results in speech recognition, object detection, time-series prediction, reinforcement learning, sequential decision-making, video/image processing, and many other supervised and unsupervised learning tasks. One of the leaders in this field is Sentient Technologies, an AI startup based in San Francisco that specializes in financial trading, e-commerce, and healthcare applications using deep learning, evolutionary computation, and other machine learning and data science approaches. I am currently working as an intern at Sentient, developing ways to make deep learning not only easier to implement, but also more applicable to more general problem domains. This internship allows transferring my dissertation research to industry, and also gives me access to computational resources that makes such work possible.

Deep learning, despite its newfound popularity among the machine learning and artificial intelligence community, is actually an extension of decades old neural network research; the major difference is that the size of both the datasets and available computing power have increased exponentially. One of the problems with deep learning is that the architecture design has a large impact on its performance and some problems require specialized architectures. For example, the Googlenet architecture (shown below), which won the 2014 Imagenet competition for image classification, contains specialized submodules which themselves are deep networks. Also, as the networks become more complex, the number of parameters and configurations that needs to be optimized increases as well. At Sentient, my advisor Risto Miikkulainen and I are developing evolutionary algorithms to automatically discover and train the best deep neural networks for a particular problem. Our vision is to eventually create a general framework that is applicable to any problem and uses machines to automate AI and machine learning research.

Googlenet architecture

One of the downsides of deep learning is that training a neural network is very computationally intensive. Most networks of moderate complexity and above take hours, if not days to train in machines with powerful GPUs. This compute cost is even worst for evolution of deep networks, since now there is a whole population of networks that must be trained and evaluated during every generation. Due to the immense computational requirements, evolutionary deep learning has been considered to be impractical until now. Fortunately, Sentient has developed a massively scalable evolutionary algorithm that runs on millions of CPUs all over the world  to evolve stock trading agents. We are currently extending it to utilize GPUs as well, to perform parallel training of each deep neural network simultaneously. This framework will eventually be scalable to hundreds of thousands of GPUs. Since GPUs are expensive and relatively rare, we are also looking at ways of utilizing also CPUs for training deep neural networks. If the training of a single network model can be parallelized across many CPU machines, then it is truly possible scale up evolution of neural nets to millions of machines.

As computing power becomes faster and cheaper, I believe that there is going to a lot newfound interest in applying evolutionary algorithms to deep networks. This approach should be particularly useful in automatic discovery of new architectures for new problem domains, such as understanding cluttered images, video, and natural language, as well as reinforcement learning and sequential decision making. This process will depend on extreme computational resources, thereby making it productive to combine the resources of academia and industry.

Posted in BEACON Researchers at Work | Tagged , , , , | Comments Off on Evolving Deep Neural Networks

Mass Extinctions, Evolution, and…. Robots?

Check out this great video produced by the UT Alumni Association talking about research by BEACONites Joel Lehman and Risto Miikkulainen at UT Austin.

Lehman and Miikkulainen published an awesome paper in PLOS ONE looking at evolution after a mass extinction.  I, for one, welcome our new robot overlords.

Here’s their abstract,

Extinction events impact the trajectory of biological evolution significantly. They are often viewed as upheavals to the evolutionary process. In contrast, this paper supports the hypothesis that although they are unpredictably destructive, extinction events may in the long term accelerate evolution by increasing evolvability. In particular, if extinction events extinguish indiscriminately many ways of life, indirectly they may select for the ability to expand rapidly through vacated niches. Lineages with such an ability are more likely to persist through multiple extinctions. Lending computational support for this hypothesis, this paper shows how increased evolvability will result from simulated extinction events in two computational models of evolved behavior. The conclusion is that although they are destructive in the short term, extinction events may make evolution more prolific in the long term.

Posted in BEACON in the News, BEACON Researchers at Work, BEACONites, Member Announcements | Tagged , , , | Comments Off on Mass Extinctions, Evolution, and…. Robots?

A microbe-dependent world: studying the legume-rhizobia symbiosis for a more sustainable future

This post is by MSU grad student Shawna Rowe

Living in a world full of fascinating visual elements and intriguing macro-organisms often results in people forgetting the most abundant group of earth’s inhabitants— microbes. Microbes are not only the most abundant and diverse group of living organisms but are also, in my personal opinion, the most fascinating. Whether it be the Demodex brevis that colonize human faces or the rhizobia that live in our soils or the Thermus aquaticus that live in the depths of Yellowstone, microbes are inescapable and responsible for endless biological processes.

One group of bacteria, rhizobia, are soil-dwelling and underappreciated powerhouses of agricultural productivity. These bacteria form a specialized relationship with leguminous plants (soybean, bean, lentils, peanuts, etc.) in which they supply nitrogen, a globally limiting resource, in exchange for carbon. When undisturbed, this interaction naturally increases soil nitrogen content. Agricultural soils are frequently nitrogen limited which causes farmers to deposit approximately 80 million tons of nitrogen fertilizers on agricultural fields each year! This practice has resulted in increased crop yields at the expense of the environment. Toxic algal blooms pollute water sources, microbial communities have been destroyed, fossil fuels are burned to produce the fertilizers, and gaseous nitrogen compounds are released into the atmosphere as consequences of modern fertilizer production and use. Fortunately, the relationship between legumes and rhizobia offers an opportunity to offset the excessive use of fertilizers and begin shifting away from these environmentally detrimental practices.

Medicago truncatula, a model legume on which I conduct research growing in two different types of growth containers. The fully encased one (test tubes) provides sterile conditions for assays that require a more controlled environment.

In this relationship the host legume provides the infrastructure in the form of specialized organs known as nodules. Inside these nodules live the hardworking rhizobia. The plant nodules serve as a protected space for the microbes to reproduce and expand as they complete the energy expensive task of converting N2 to NH3. Years of evolutionary pressure has resulted in a very tightly controlled balance of resource trade. However, as with most relationships there exists opportunities for trouble— in context of this mutually beneficial relationship the rhizobial partners have the opportunity to take more resources from the host plant while supplying comparably less nitrogen. This act has been termed “cheating.” Cheaters are problematic since they threaten to destabilize the long-established and important relationship; a reality that would further strengthen our dependence on nitrogen fertilizers in the agriculture sector. In Dr. Maren Friesen’s lab, I aim to elucidate molecular mechanisms of this resource trade between legumes and rhizobia. My work focuses on understanding how host plants are able to differentially recognize and respond to rhizobial partners of varying effectiveness. Developing an understanding of these response and control mechanisms is critical to understand how microbes are able to exploit their hosts and how external pressures are driving the emergence of cheaters.

Shawna working in a biosafety cabinet in the Friesen lab space

As a native of southwest Missouri, ranked 6th in soybean production in the U.S., I spent most of my life surrounded by agricultural fields. Traveling to school frequently involved getting stuck behind a tractor when planting season arrived. Future Farmers of America was the largest student organization and roughly half of the student population had milked a cow before the age of 10. Although charming and hardworking, small agricultural towns are often times inherently (but unintentionally) anti-science. STEM education was severely lacking and evolution was a dirty word capable of eliciting dramatic arguments and endless frustration. Because of this, I loathed the idea of working in agriculture. Upon graduating high school, I entered college as a Biochemistry major with no clear idea of what “biochemistry” was nor what I could do with it. I was fortunate enough to land a job in a plant biochemistry research lab. There, they focused on understanding basic mechanisms of plant immune responses to pathogenic bacteria. That job set up the stage for my future research interests. I discovered the complex world of molecular signaling events and microbial associations. I learned about the co-evolution of organisms that commonly associate and how these associations drive the development and establishment of complex features of host-microbe interactions. I fell in love with the unseen world.

Years later, these experiences still serve as the foundation for the questions I ask and the topics I find intriguing. In the Friesen lab, I hope to better understand how hormones, specialized proteins, and various other plant derived molecules serve as regulatory components for the unique relationship leguminous plants have with the microbial world. Further developing our understanding of the regulatory mechanisms will both shed light on the co-evolution of legumes and rhizobia as well as the factors that threaten to destabilize this biologically important relationship.

Posted in BEACON Researchers at Work | Tagged , , , , | Comments Off on A microbe-dependent world: studying the legume-rhizobia symbiosis for a more sustainable future

Male battles split species apart

Picture of me: Behind me are some of the hundreds of fish tanks in the basement of Giltner containing all the baby sticklebacks we generated for this experiment.

This post is by MSU postdoc Jason Keagy

How do species form?

Stated more precisely, how does one species become two? This turns out to be an immensely difficult question to answer, because 1) species are not always distinct entities (species definitions are argued about ad naseum [1]) and 2) the formation of species (speciation) is a process that often takes a long time to complete.

One way in which species could form is if selection is divergent and a population responds to that selection [2] – for example, Anolis lizards that have adapted such that each species has limbs that are optimal for living in different types of vegetation [3], or insects that have specialized on feeding on different plants [4]. One way to represent the relationship between phenotypes (traits such as limbs, coloration, or digestive enzymes) and fitness is with a “fitness landscape” [5], so called because in three-dimensional representations (e.g., two traits as the x and y axes and fitness as the z axis), it can resemble a landscape of peaks and valleys. However, we don’t have a lot of great examples of these because it is often difficult to measure fitness and fitness often depends on multiple independent phenotypic traits in complicated ways.

The power of sticklebacks

In some freshwater lakes in British Columbia, you can find two different types of stickleback, called “benthics” and “limnetics” that are reproductively isolated, and therefore, typically referred to as species. These benthic and limnetic sticklebacks are descended from marine sticklebacks who bred in glacially fed streams. After the glaciers melted ~12,000 years ago, the weight of the ice being removed caused the land to rebound, and the uplifted streams became isolated lakes. Because of this relatively short timescale, these fish have become a model system for studying adaptation and speciation.

What is the difference between benthic and limnetic sticklebacks? Limnetics live in open water, eat plankton, and are more visually oriented, whereas benthics eat invertebrates off of plants or the lake bottom, live in complex spatially structured vegetated habitats, and are more dependent on smell. Limnetic and benthic sticklebacks also differ in body size, shape, and mating traits. In other words, they are really different! Critical for maintaining these differences is strong reproductive isolation and so the Boughman lab has long been interested in understanding what influences this isolation.

The role of male competition

Typically, the focus in speciation research has been on natural selection (even in sticklebacks). Much less studied and controversial is whether sexual selection can drive speciation. Especially unstudied is intrasexual (often male-male) competition’s role. That seems like a pretty big oversight to me. Flip on any nature show and you’re sure to see at least one scene of males bashing each other to pieces. It turns out Jenny Boughman, Liliana Lettieri, and I were already working on a project which was perfect for studying how male competition might impact speciation.

Fig. 1. Males compete intensely over territories on which they build nests. Pictured here are three males in a tank at KBS. The male in the foreground is directly over his nest. It’s pretty well concealed!

Male sticklebacks compete for territories on which they build their nests (Fig. 1). They’ll even destroy each others’ nests and steal pieces such as the choicest algae. Eventually, these males will try to attract females via courtship behavior to convince them to lay eggs in their nests. Male competition is extremely important to determining male fitness: if males can’t successfully obtain and keep a territory, and build and keep a nest, they are unable to reproduce (we rarely see sneak spawning). Male competition could have important impacts on speciation because males of each species build nests very close to each other in nature and are therefore direct competitors for space and resources.

Our research

Our main research questions included: How do male phenotypes relate to male competitive fitness? Do the resulting fitness landscapes have multiple peaks? Would these peaks promote speciation? We created hundreds of hybrid males in the laboratory through artificial crosses. This greatly expanded the combinations of phenotypes from that seen in the wild. Then we put these males in large outdoor tanks at Kellogg Biological Station that had sand and algae and food caught from nearby ponds. We measured lots of physical traits on the males and spent hundreds of hours recording their male competition behavior (with the help of an awesome army of undergrads).

Fig 2. Be really careful about what you are taking with you into water bodies. Your actions can have serious evolutionary and ecological consequences!

Our research revealed some surprises [6]. First, there were indeed two fitness peaks corresponding to pure benthic and pure limnetic multivariate phenotypes. But there was another region of high fitness (a bridge connecting the peaks) that implies certain intermediate hybrids were also good competitors. Interestingly, these hybrids had phenotypes like fish now seen in Enos Lake, where after anthropogenic disturbance (someone released crayfish into the water, Fig. 2) formerly distinct benthic and limnetic species are now a hybrid swarm (a depressing example of evolution in action). Previously the hybridization had been attributed to the crayfish’s introduction resulting in generalist rather than specialist sticklebacks having higher survival, a change in natural selection [7]. These generalists would have been produced by hybridization, which before happened at inconsequential numbers, but this trickle would have become larger as hybrids were now surviving to adulthood. However, our results show that sexual selection through male competition may also have been a contributing factor that sped up the species collapse. The hybrid males with phenotypes corresponding to the bridge within our fitness landscape would have likely been very successful at getting nests, increasing the likelihood of further hybridization. Our data strongly suggest male competition could be very important in the speciation process and impact speciation in complex ways.

[1] As one example of this disagreement, see Wu, C-I. 2001. The genic view of the process of speciation. Journal of Evolutionary Biology. 14: 851-865 and the ten responses.
[2] For a book dedicated to this topic, see Nosil, P. 2012. Ecological speciation. Oxford: Oxford University Press.
[3] A nice HHMI video description of this research is here:
[4] There are many nice examples of this including 1) pea aphids that have diverged to specialize on red clover and alfalfa, 2) fruit flies feeding on different species of cactus, 3) the races of apple maggot fly that feed on either hawthorn or apples, and 4) stick insects adapted to wildly different plants in California.
[5] There is some disagreement over the what specifically “fitness landscape” refers to and what the proper term is for what I refer to as a “fitness landscape” here (especially among philosophers of science). You can read about it in the first section of this book: Svensson, E., Calsbeek, R. (eds) 2012. The Adaptive Landscape in Evolutionary Biology. Oxford: Oxford University Press.
[6] Keagy, J., Lettieri, L., Boughman, J.W. 2016. Male competition fitness landscapes predict both forward and reverse speciation. Ecology letters. 19: 71-80.
[7] Behm, J.E., Ives, A.R., Boughman, J.W. 2010. Breakdown in postmating isolation and the collapse of a species pair through hybridization. American Naturalist. 175: 11–26.

Posted in BEACON Researchers at Work, Notes from the Field | Tagged , , , , , , | Comments Off on Male battles split species apart