This question was posted on the http://www.edge.org
“What game-changing scientific ideas and developments do you expect to live to see?”
The evolution of dark data. Dark Data is the human data info sphere which is analogous to what physics currently labels as Dark Matter in the Universe. The currently unknown aspect of the universe which occupies the majority of it’s volume. Like Dark Matter, Dark Data is expanding. In the age of the Internet the expansion of Dark Data is exponentially beyond our current ability to organize and interpret it very effectively AND efficiently.
I believe that in my lifetime we will create improved ways to bring light to the Dark Data. Methods will be employed to organize, visualize and interpret the newly lighted dark data. At first these methods will be laborious (like mapping the human genome was), but over time the methods will require less and less human intervention as at first human minds develop processes & systems to interpret the data….and then the machines enhance and develop additionally more effective and efficient ways to bring the dark data into the light.
The closest analogue to this right now is the BI/DW industry, currently requiring huge amounts of human and physical capital to convert dark data into information. The cost is dropping as more efficient ETL tools are created to Engineer, Transform and Load the data into a Light-data format for human perusal. Over time the process of ETL will become more automated as the collective intelligence regarding dimensional data schema and the underlying ETL process is catalogued for reuse.
Imagine the effect of truly granular information placed in the context of time and other levels of granularity in a format that is easily visualized in multiple ways at the discretion of the information consumer. Imagine that this information is also contextually aware, meaning if your walking down a street and feel like eating pizza that you merely have to say the words “I’m hungry for Pizza” and the technology available to you at that instant is able to correctly interpret that and place in your view any Pizza eatery within 10 minutes of your current location, along with the menu’s, prices and estimated time of arrival….even allowing you to place the order and pay for the meal before you arrive. Upon arrival your custom slice of pizza is waiting for you timed to be ready at the time of the ETA your personal system communicated to the vendor.
Becoming a locationally-aware node within the larger dimensional schema of the web will involve the loss of some privacy with the payback being uber convenience.
Small business owners will be able to know every detail about there business at the appropriate contextual moment of need. More likely than not, the system(s) interacting with the available internal data will already have offered suggested actions requiring human intervention.
To me…this type of data ubiquity combined with high utility will occur in my lifetime and be truly transformational and since I like pizza quite a bit (I’m a pizza snob)…useful.