BrainVR: One Step Closer to Visualizing the Brain in Virtual Reality


The three pound organ sitting behind your eyes is nothing short of extraordinary.  Symphonic activity among billions of neurons gives rise to thoughts, feelings, and ultimately the perspective and personality that makes you, you. Your brain makes a remarkable journey over the course of a lifetime — from an infant unable to speak to a capable adult able to philosophize and implement great dreams.

Our brains have evolved a special affinity for storytelling. Through narrative we share experiences. Elegant linguistic and visual combinations teach lessons from the past and facilitate fantastically imagined futures. Stories connect us with friends and loved ones.  In much the same way that technological revolutions like virtual and augmented reality are transfiguring how we tell stories, innovations in an entirely different field – neuroscience – are beginning to reveal how stories make their way into that most amazing source of self: the human brain.

Eyewire Neurons by Alex Norton, eyewire, neurons, brainvr

Thanks to cutting edge artificial intelligence, nanoscale resolution imaging, seriously powerful computers, high resolution portable displays and unprecedented international collaboration, we teamed up with MIT Media Lab’s Fluid Interfaces Group to create a never-before-possible interactive immersive neuroscience visualization that debuted at the 2016 Tribeca Film Festival’s Games and Media Summit.

amy sterling at tribeca, Amy Sterling, Amy Robinson Sterling, eyewire, Tribeca Film Festival, amyleesterling, brainVRIn Virtual Reality, visitors teleport into a foreign yet familiar world, exploring a new pathway of narrative: the electrophysical connections among neurons. This is BrainVR.

Venture into a freshly discovered circuit of cells responsible in part for the brain’s ability to perceive motion – the very ebb and flow of story itself – using HTC Vive. Participants are invited to play with virtual objects and are challenged to trigger specific neural circuits which send movement information to the brain. Explore a stunningly beautiful, cutting-edge ideascape while you enjoy wonder and awe for the intricate complexity and nearly immeasurable wonder of the brain.

In 2014, we created Neurons in Space, a virtual reality experience that debuted at the TED Conference. This first VR ever shown at TED was a huge hit and catalyzed an ongoing side project at Eyewire: visualizing the brain in ever more immersive experiences. BrainVR is the latest iteration in an ongoing quest to immerse participants in the brain:

 

The combined efforts of hundreds of thousands of gamers have resulted in nanoscale resolution 3D models of neurons and their associated circuitry. These models serve as the basis for BrainVR. Learn more at Eyewire.org.

I had a fantastic time creating and sharing this experience at Tribeca. Tremendously excited to continue taking it forward. Ping me if you’re interested in collaborating! Huge shoutout to the team: Alex Norton, Daniel Citron, Scott Greenwald, Max Rose, and Julian Samal who created the sound.

amy, amy sterling, amy robinson, amy robinson sterling, Tribeca Film Festival, Eyewire, brainVR, Brain, neuroscience, presentation, Amy Eyewire, MIT Media Lab, Fluid Interfaces,

Continue Reading

BRAIN Initiative: Thoughts on Interim Report to the NIH

neurons reconstructed in eyewire, connectome

We stand on the verge of a great journey into the unknown—the interior terrain of thinking, feeling, perceiving, learning, deciding, and acting to achieve our goals—that lives within the human brain. These capacities are the essence of our minds and the aspects of being human that matter most to us. Remarkably, these powerful yet exquisitely nuanced capacities emerge from electrical and chemical interactions among roughly 100 billion nerve cells and glial cells that compose our brains. All human brains share basic anatomical circuits and synaptic interactions, but the precise pattern of connections and interactions are highly variable from person to person—and therein lies the source of the remarkable variation we see in human behavior, from the breathtaking dance of a ballerina, to the elegant craftsmanship of a master carpenter, to the shrewd judgment of an expert trader. Our brains make us who we are, enabling us to perceive beauty, teach our children, remember loved ones, react against injustice, learn from history, and imagine a different future.

Preamble, The Goals of the BRAIN Initiative

Today is my one year anniversary at MIT. I’ve grown as a person and learned more than I ever imagined. To celebrate, I read a 50+ page NIH report, which was surprisingly awe inspiring. Here are my thoughts.

On Sept 16, 2013, an advisory committee of prominent neuroscientists presented a report to the NIH Director outlining the goals and objectives of the BRAIN Initiative for FY 2014.

Before we proceed, Iet’s back up. My greatest passion in life is understanding consciousness; understanding how the brain yields a thinking, feeling human being. That’s why I’m at Seung Lab working on EyeWire, attempting to catalyze exponential progress advancing neuroscience from the unknown into the understood. How exciting, how chill-inducing to read that the greatest minds on minds of our generation are teaming up and calling for unprecedented action in this direction, progress backed with investment and goals for disruptive, interdisciplinary collaboration that “reconceives what it means to be an experimental neuroscientist.” (13)

The report outlines several themes and high-priority research areas for 2014:

  1. Generate a Census of Cell Types
  2. Create Structural Maps of the Brain
  3. Develop New Large-Scale Network Recording Capabilities
  4. Develop A Suite of Tools for Circuit Manipulation
  5. Link Neuronal Activity to Behavior
  6. Integrate Theory, Modeling, Statistics, and Computation with Experimentation
  7. Delineate Mechanisms Underlying Human Imaging Technologies
  8. Create Mechanisms to Enable Collection of Human Data
  9. Disseminate Knowledge and Training

I am particularly excited about, well, all of these but for the interest of time I’ll hone it down. First, let’s reflect on the fact that we don’t even know how many types of cells there are in our own heads..and how quickly that’s changing. A catalog of cell types would provide a framework for existing research and a foundation for future experiments. Like everything in this report, it’s extremely exciting.

Second. Create a structural map of the brain. The report calls for a movement “towards [mapping] a full connectome.” Oh yes. There are many layers of functional circuits in the brain, all of which are important to our broader understanding. Special emphasis is given to integrating scales in both time and space and creating platforms that “enable understanding of the relation between neural structure and function.” The report calls for “faster, less expensive, and scalable approaches” that will reveal how how neural dynamics relate to complex behavior. Crowd-sourcing is specifically mentioned in this section, which needless to say amps me up.

neuron branches, eyewire,

I’ll skip now to numbers 5 and 6, which read as a call for revolutionary cross-boundary collaboration among researchers to create integrative tools for creating enriched, multidimensional datasets that, for example, might integrate molecular, functional and connectomic information. The report indicates a preferences for involvement from fields outside neuroscience such as computer science, statistics, engineering, physics and theory (and even calls for new theoretical tools and techniques — a “brain based theory of higher functions is notably lacking.” hello, opportunity).

This is not the time to play is safe.

I love this. Neuroscience is due a bit of disruption; we need to bring in minds from different fields — fields that the authors of this report may not have even considered, such as design. I think crucial components will be identifying and communicating neuroscience’s biggest accomplishments, current state of knowledge and present/future hurdles to communities of talented individuals and organizations, many of which may be outside academia, who will develop and test innovative solutions to them. We may use tools like Kaggle or host Hackathons, challenging a burgeoning global developer community to create software that will help solve neuroscience’s biggest difficulties. I’ve come to realize that while techniques do cost a considerable amount of time, it seems that software and big data analysis are what’s really needed to increase technolgical and analytical throughput by 100-1000x, as the report calls for. One infamous case of analytic delay is that of Earl Miller’s Lab at MIT. The team spent 2 weeks collecting functional activity data and 2 years analyzing it. Damn. We need tools, better AI and brave collaborations on all neruoscientific fronts. This is one area I’m particularly interested in catalyzing.

Another interesting component of this research is an open call to create a neuroscience data reservoir where researchers can store and one day maybe even crowd-source analysis of their research findings, specifically image data. Seung Lab (my lab) has an interest in this, so stay tuned.

synapse in eyewire

A final point of interest is #9, the dissemination of knowledge and training. In the context of citizen science, which will likely play an increasingly important role in neuroscientific progress, we need to share best practices and methods through which labs can involve the general public in the scientific method. We are actively working on such toolkits and events. Ping me if you are interested in collaborating on this or any other area.

Finally, going back to the big picture (my favorite), I’d like to point out that the report specifically calls for research “composed with a specific eye toward eventual impact for humans” and that “encourages changes in the culture of neuroscience.”

The challenge is to map the circuits of the brain, measure the fluctuating patterns of electrical and chemical activity flowing within those circuits, and understand how their interplay creates our unique cognitive and behavioral capabilities.

Our ultimate goal is to understand our own brains.. to understand the circuits and patterns of neural activity that give rise to mental experience and behavior.

It’s a wonderful time to be alive. I’m honored to have the opportunity to play a leading role in the future of neuroscience through EyeWire and specifically Seung Lab at MIT.

for science eyewire black on yellow, for science, eyewire, brain, design

Continue Reading

MIT Neuroscience Harlem Shake

Today, besides officially naming the first neuron mapped by gamers ‘IFLS’ for “I Fucking Love Science,” finalizing a talk for TEDActive and organizing the backend public beta for a rad new MIT Media Lab 3D matrix-style animation project, our lab made a Harlem Shake video. I’m not sure what this is..or why it’s a thing..but it’s clearly awesome and gave Sebastian Seung a chance to show off his unbelievably amazing dance skills. More moves in the blooper reel below.

Tomorrow I’m off to LA..maybe we’ll make a TEDActive Shake..

Continue Reading

Live Broadcast from the Shoulders of Giants or How (Funny) Videos Augment Science Communication

In the past week, over 10,000 people have joined EyeWire, a citizen science neuron mapping game from Seung Lab at MIT.

I’m about 8 weeks new in this wonderful lab and have started mixing things up a bit. What do typical researchers do on a daily basis? Who knows! They’re spectacular and secretive creatures. Seung Lab is not so secretive. Dev ninjas have even started doing spontaneous video sessions (who knew you could catch a ninja on video?!) and our whole team hosted a G+ hangout during a Christmas party, which might have included copious amounts of alcohol, dancing and wouldn’t have been complete without popping champagne. Our cameras are locked and loaded — charged and mic’d up. Don’t you love it when scientists have personality (yea that’s a GIF of Sebastian dancing created by EyeWirer Dylan Holtz), a sense of humor and are willing to share it with the world? I sure as hell do. I think it makes science more attractive and accessible, which is, in the words of brewery Sam Adams, always a good decision.

As you can see, we take science — but not ourselves —  seriously here.

i dont always pretend to be a scientist but when I do it's on EyeWire.org

That was created by EyeWirer Adam Brabant.

Seriously, though, Why EyeWire? You could read this post….or watch the video below.

We interviewed Harvard’s Joshua Sanes (a legend among neuroscientists.. though admittedly I did not know this chap until I learned he discovered a new cell, which made me wonder: if the EyeWire community discovers new cells, do they get to name them?  If I have anything to say about it, new cells will not be called “Junctional Adhesion Molecule B” Cells. No, no. I’d opt for the “epic” neuron so that people come to know us as the Discoverers of Epic).

A little EyeWire Tutorial:

A fun video we made when I Fucking Love Science + Reddit crashed EyeWire on Tuesday:

More videos are coming, including action-packed theatrical trailers, ultramicrotome drama, and dancing (lots of dancing) from Sebastian. Oh yea, and educational productions. After all, we do want to create a smarter world. I leave you with this: Sebastian goes Gangnam style. Created by Seung Lab’s great Spaniard postdoc Ignacio aka Nacho.

“Discover what is to be taken seriously and laugh at the rest,” Herman Hesse.

Want more examples of rad science videos making a splash? Check out

ASAP Science | Dr. Carin Bondar’s Wild Sex (Biology Series) | TED-Ed

Continue Reading

A bit of what I’ve been up to at MIT

I recently moved to Cambridge, MA to take the best job of all time helping Sebastian Seung’s Computational Neuroscience Lab at MIT build a game to map the human brain. Yea. It’s called EyeWire and you should check it out.

Best job in the world for several reasons. For someone obsessed with thinking about thinking, this life is positively dreamy.  I think I live one solid series of awesome moments. I love the people in my lab. I got to move to Cambridge and live one mile from Harvard and one mile from MIT. I love walking to work. I love work! It doesn’t feel like a job. I love going to hackathons. I love hanging out with geniuses. I love MIT Media Lab. I love working with neuroscientists. I love learning new things. I love being around curious people ready to share their passion for creating game-changing technologies. I love going to intellectual events at MIT and Harvard. I love connecting with so many TEDxers on the east coast! I love snow (though we haven’t had much yet). I love great food and even greater company. I love talking about molecules and python and infographics and chilling with scientists every day. Bascially, I love life. I love life very much.

I’m aslo helping a group at the Media Lab (which I’m not really supposed to talk about), developing a new app for TEDx music (also not supposed to talk about..but no one reads this blog 😉 and building an anonymized open-source database of health and lifestyle data with WIkiLife. Other things too..but it’s late and I want to read Nietzsche.

Below is a post I just wrote for the EyeWire blog. I blog at MIT now. Rad. Life is amazing. I hope you, dear reader, are following you passion and pursuing diligently the ideas that strike you most curious.  Reality will exceed your wildest expectations if you let it.

Cheers, much love.

Amy

It may come as a surprise that although we know much about how the eye works, neuroscience researchers do not fully understand how visual signals translate into perception.

We’ve landed on Mars, can grow organs, and even skydive from space, yet when it comes to a thorough understanding of the territory so close to home that it is home, much is missing. Neuroscientists don’t even know precisely how many different types of cells are in the brain. Here at Sebastian Seung‘s Computational Neuroscience Lab at MIT, we’re taking a different approach: crowd-sourcing. In order to solve the mind’s great mysteries, we need you.

Why don’t we know how the mind works? One reason is that your mind is massive. Researchers estimate that there are 100 billion neurons in your brain with about one million miles of connectivity. A million miles is equivalent to driving around Earth 40 times. You can infer that in order for such great length of neurons to fit into your three micron scale image by FSUpound brain these structures must be very tiny. A large neuron is about 100 microns in diameter while the contact area of a synapse is about 400 nm in length.

In order to see neurons and the tiny structures called dendrites through which they function, researchers utilize a new imaging technique. “Fix whole brain tissue, slice off layers just a few microns thick, image each slice with an electron microscope, and trace the path of each neuron,” explains David Zhou, Masters Student at Carnegie Mellon, on Quora. These gamechanging techniques generate terabytes of data for even a cubic milimeter of brain tissue. Now that we can see the brain at the synaptic scale, we have to analyze the images. How?

neuron cell reconstruction Seung Lab

The image above shows the process of layering image slices to render 3D reconstructions. Like most neuroscience labs, the Seung Lab uses a combination of AI algorithms and tracing (3D reconstruction) performed by humans. Why not just use algorithms? Images can be challenging to identify, particularly for a computer. Pure algorithms make many mistakes, such as slicing a single cell into thousands of pieces and merging multiple cells into one monstrously massive neuron. See below image for an example of AI missing a chunk of a neuron.

correcting a computer's mistake, Seung Lab

We hope to one day train computers to map neurons on their own; however, that day will be far in the future and we need to accelerate neuroscience discovery now. To achieve this, we need something more intelligent than even the most powerful supercomputer— you.

It takes an MIT-trained neuroscientist anywhere from 15 to 80 hours to reconstruct a single neuron. At that rate, it would take about 570,000,000 years to map the connectivity of an enture human brain, known as aconnectome. This is why we need your help.

Rather than mapping and entire brain, we’re starting with a retina. Our goal is to map the connections of a specific type of cell: J-Cells. These neurons are responsible for perception of upward motion. We plan to publish the outcome in a scientific journal and list EyeWire users as co-authors.

By playing the 3D game Eyewire, you become part of the Seung Lab at MIT by helping to map the connections of a neural network.

Scientific American writes that “no specialized knowledge of neuroscience is required [to play EyeWire]; citizen scientists need only be curious, intelligent and observant. Your input will help scientists understand how the retina functions. It will also be used by engineers to improve the underlying computational technology, eventually making it powerful enough to detect “miswirings” of the brain that are hypothesized to underlie disorders like autism and schizophrenia.”

We hope that you will help us trace the wires of perception through EyeWire. Play EyeWire and let us know what you think on Facebook.

Continue Reading

Idea flow

Sitting outside. A storm approaches.  Strong air. Thinking about life, ideas, posture, air quality, metabolic networks, connectomics, neuroscience, biomimicry, dynamic architectures, infographics many things.  How do I choose what to blog about?  Which idea arrays generate the most spikes of late.. how am I most passionate?  What ideas are most valuable to you, for you to think, to stimulate your own mind?  Hm exploring other minds, understanding and fueling curiosity, exploring what exists.. What ideas spark me as splendid and why.  That’s what I will blog about.

Neuroscientists are combining viral addictive game mechanics with EM resolution imaging analytics of 3D brain tissue. Game changer.

Background: you are the connections among your neurons.  Humans have never mapped this “connectome.”  Not even of a mouse.  Some scientists did c. elegans, a 1 mm worm with 4,000 total connections.  It took 12 years.  You have 100 billion neurons, some with tens of thousands of connections.

Researchers use a blend of AI and manual mapping to trace the 3D shape of neurons (colorful picture above).  It takes 1,000 lab hours to map 1 cubic milimeter of tissue.  How big is a full brain?  One million times larger.  We’re working to solve the bottleneck by crowd-sourcing the analytics. Sebastian Seung’s lab at MIT conceptualized a game called WiredDifferently (because we are, and we must map it to see and understand ourselves) and built a live beta that allows users to help map retinal connectivity at the synaptic level by filling in a 3D coloring book of sorts (we don’t even know how we see!).  I will write more about this on healthsterling.com.

Is this splended? You decide.  I experience joy by thinking about neural network complexity and the sheer magnitude of challenge. We don’t even know how many different types of cells there are in our own heads!  Opportunity.  Time to accelerate the rate of exponential progress.  Develop new questions, technologies, understandings..

Other splendidities..

Language.  How does the way I think reflect a configuration of neurons?  I wonder.

Quora continues to fuel surprise discoveries.  Exploring how others explore fascinates me.  I love navigating thoughts with questions.  There is an entire network of ideas growng now.  Splended! You can be a part of it.  See how my ideas evolve. And check out these stats about your body.  Visualize..

What might you find curious…Last week was in San Fran working on the MIT project, before that was in Dubai and Doha for TEDxSummit (blog that) and prior to that TEDMED.  Hyper development in the past couple months.  Exceptional connectivity in rare environments catalyzes rapid idea prototyping.  Theories are evolving. More on that before the end of the year. For now, launched Healthy City pilot with DailyFeats; the TEDx Global Music Project was granted forward progress by TED, other projects hm this post is about what, making you think? You want to hear thoughts that come out of experiences.  Alright.

The Exploratorium: epic interactive museum.  These are whoa.

Uncoupled “simple” pendulum waveforms.

Magnetic sand (black and magnetic because it contains iron)

Ferrofluids, strongly magnetizable fluids.


In conclusion, think something new, think differently frequently.  Here’s a drop of surprise that I happened upon on Quora to get you on your way: what is the most badass ancient city?

Continue Reading