Big Data: A Shift to Multidimensional Consciousness
by Catherine Baird, PhD
Over the last decade, your society has literally grown up. Even the most environmentally incorrect is now busily forging some sort of alliances with those who more deeply protect your many fragile habitats…You need to use technology to create a proper habitat and not to destroy it. ~Sheldan Nidle, March 10, 2015.
#GSCLIONShare Big Data Conference an interesting question was posed: I’ll paraphrase it as “This Big Data stuff is very interesting, but also very technical. As a BBA major, how do I get involved?
One of the neatest things about Big Data is that it’s so interdisciplinary. It calls for such broad understandings—not just in computer technology or math and statistics (STEM is the common acronym now) or even business, marketing and finance—but all human thought inclusive of philosophy, language, arts and the humanities… holistically, globally.
Did you know that one of the most sought-after disciplines for Big Data recruiters is philosophy?
“ Jeremy Howard, chief scientist at an Internet startup that runs data prediction competitions has a degree in philosophy. He believes that the key job requirements in data science is really curiosity, flexibility, and the willingness to learn, capabilities that can be found in a wide variety of studies and job backgrounds (Hall, 2012).”
This shift comes from the very nature of Big Data. It’s vast. As we say Big Data is all about the V’s — velocity, volume, variety… (I still argue the 4th V = validity remains an emerging capability!) … So people trained in conceptualizing the big picture and in visualizing and analyzing multiple, seemingly unrelated correlations are in high demand.
Finding those grains of sand on the beach requires we open ourselves to being aware of non-obvious connections, allow for anomalies, and jump into everything from the placement and etymology of words and numbers to location and timing of context. Discovery is definitely not linear. At minimum it’s 3 dimensional. Many Big Data problems (questions) seek 4D or even 5D resolution. The only rule I’ve found is, expect the unexpected!
(A thought for another time… if technology is just humans optimizing their experience with life– doing things faster, and perhaps more accurately than they’d do it by hand–and studies show organizations today are using ~ 10% of the data they have available, hence they’re adopting Big Data to help them use more than that 10% …Whoa! Wait! Doesn’t that sound familiar? How long have we been hearing that human beings are using only ~ 10% of their brain?)
But before I get distracted. This tie back to the humanities and non-technical, scientific training was fascinating for me, especially as I’m one of the anomalies in the field (a Ph.D. in History, B.Sc. in Zoology who somehow took the road less travelled to become a Cloud Computing evangelist and Big Data Architect?)
Recently, I converted my 1997 doctoral thesis into Revolution from Within, a narrative nonfiction covering the story of “The 160” (an elite group of Russian intellectuals exiled from the country by Lenin in 1922), and their interactions with a most unusual sponsor, the YMCA, culminating in the return of their ideas to Russia almost 70 years later with the fall of Communism. The book was released for publication in 2015. For more information, see www.catherinebairdbooks.com.
Thanks to that eclectic evolution, I’ve long been aware that computer programming is really nothing more than another language, a philosophical-logical system devised by humans. All computing is just logic (human’s made-up language). It’s truly all in our minds.
So if Big Data is to be the catalyst that brings people trained in ALL disciplines together once more, I can only welcome and embrace it. For far too long we’ve limited science and the technologies to a very limited, rational, sensationalist (meaning what we can perceive with our senses) perspective. And in the quest for immediate gratification, instant results, I must say that we’ve ignored some very important considerations to our peril. Questions other than “Can we?” or “How fast?” or “How much will it cost?”
Rather than focusing on what can we do now to help the people and the world around us today—safely, securely and responsibly—many Big Data initiatives founder on trying to boil the ocean (cheaply of course) or conquer the world.
It was so inspirational to hear the great story from Children’s Healthcare of Atlanta about their “Frankendoop” project. Knowing Big Data could help solve pressing questions about better ways to treat children yet hands tied by non-existent budgets for new hardware, the team cobbled together a Hadoop cluster using PCs and laptops that were headed for the scrap heap.
Literally on a shoestring budget, they interconnected the end-of-life computers, installed Hadoop, and began collecting vital ICU monitoring data, then analyzing it to answer top debates being held between the doctors and nurses in the ward. One outcome was magnificent! Improved pain management leading to faster recovery and self-sufficiency in premature babies. You can read more about this heart-warming story here.
Just like the whole designed implementation of Big Data Ecosystems is holistic, heterogeneous, and unifying—supporting almost unlimited compute and storage paradigms on one cluster—so too are the talent and resource demands of this emerging field requiring people skilled in the full spectrum of disciplines and creative inquiry.
If Big Data is revolutionizing the way human beings work with digital information, it is also disrupting the way human beings work together: breaking down the walls and insisting “no more silos”.