Big Data

Big Data

Big Data[83] refers to Huge Volume of Data that cannot be stored and processed using a traditional approach within a given time frame. It is a term for data sets that are so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, data curation, search, sharing, storage, transfer, visualization, querying, updating and information privacy. The term often refers simply to the use of predictive analytics or certain other advanced methods to extract value from data, and seldom to a particular size of data set. Accuracy in big data may lead to more confident decision making, and better decisions can result in greater operational efficiency, cost reduction and reduced risk.

It is the NeXT big thing in computing, and generates value from large data sets that cannot be analyzed with traditional computing techniques. The quantity of computer data generated on Planet Earth is growing exponentially for many reasons. Retailer are building large databases of recorded customer activity. Organizations working in Logistics, Financials, Healthcare & many other sectors are also capturing more data and social media is creating vast quantities of digital material. As vision recognition improves, if it is additionally starting to become possible for computers to extract meaningful information from still images and video. As more smart objects go Online, Big Data is being generated by an expanding Internet of Things. Several areas of scientific advancement are starting to generate quantities of data that were until recently almost unimaginable.

Big Data is characterized by Volume, Velocity and Variety. Volume poses the greatest challenge and opportunity as Big Data could help many organizations to understand people better and to allocate resources more effectively. Velocity raises issues with the rate at which date was flowing into organizations now exceeding their capacity of their IT system. In addition, users increasingly demand data in real time, and delivering this could prove a challenge. Finally, the Variety of data processed is becoming increasingly diverse. Before, data sets dealt with documents, financial transactions, stock records and personnel files. Today, photographs, audio, video, 3D models, compact simulations and location data are being piled into many of corporate data silo. Many data sources are also unstructured and hence not easy to categorize with traditional computing techniques. A certain amount of data could mean Big Data for one company but not for another. It depends on the capabilities of the company. Traditional systems are not capable of handling Big Data and challenges spring up at multiple levels, including Capture, Curation, Storage, Analysis, Search, Sharing, Transfer and Visualization. The Big Data becomes a challenge to the system not merely because of it’s size, but also because of the speed at which data is coming in, and also because it could contain items of various formats.

Where does Big Data come from? A number of users, applications, systems and sensors are producing large files. These files are produced at a very high speed and contain a variety of data items, and thus create challenges for traditional systems and hence the term: Big Data. Traditional systems are not able to handle Big Data, since they cannot handle the enormous size, and speed of data coming in. Because of the limitations of traditional systems the term Big Data has entered the scenes as a problem and that has been accepted by the Industry, for which Big Data Tools are being developed. Companies gain edge by collecting, analyzing, and understanding information. Governments are forecasting events and taking proactive actions.  

Data is constantly being produced. For ex, by a Social Media, Public Transport, and GPS. To process Big Data there is no need for big computers. Data is processed with the Cloud, an endless Network of normal servers and powerful algorithms. This way they can analyze powerful pieces of data in minutes. For ex, Video-streaming service Netflix analyzes the data of the viewers, like popular shows and patterns. In this way, they produce successful series with a combination of actors, directors and storylines. Big Data is being used to develop a car that can drive completely accident free all by itself! In the future, Big Data of DNA can be used to determine the perfect treatment to cure genetic diseases like cancer, etc. Companies today are flooded with data. 90% of the world’s data was created in the last 2 years. 85% of business leaders believe Big Data will revolutionize business. Data is the most precious raw material in the 21st century. A growth of 800% is expected in the NeXT 5 years. In 2009, the total data was estimated to be 1 ZB (Zettabytes). In 2020, it is estimated to be 35 ZB. 35 times growth in little over 10 years shows the how exponential this Revolution is!!! Harnessing data and extracting value from data is a top challenge and priority for Business & IT. It is not just about Volume, but also variety. New sources, new devices generate mounds of data, every second of every day. Text Messages, Youtube videos, Facebook posts, Twitter tweets, Blogs podcasts, Emails, Photos, Music… Since 90% of data is unstructured, it is difficult for businesses to get a value out of it. Most organizations make decisions based on their structured data only, that is just 10% of the picture. They lack a 360° view of what is happening in their supply chain, with the partners, in social media channels, and the world around them. Big Data, which poses a challenge, is really a fantastic opportunity for business development. A whole new industry is being created just by developing applications to manage abundant data. As explained before, this data is very easy to find but difficult to compile and manage. Customers have a huge need for applications that can transform this data in valuable information for them. Find the GENIUS in you loyal reader, use your head and build those applications that the customers need. The opportunity is at hand, use your imagination, put your creativity to test, and shape your own destiny. Today, more than ever, you can control your destiny!!!