It cannot be denied the data has been increasingly shaping the world we live in - from helping us pick the next TV series we may like to influencing the course of elections. To quote The Economist - "Data are to this century what oil was to the last one: a driver of growth and … Continue reading Why Data Science ?
HDFS
When your data outgrows the capacity of a single machine it becomes necessary to split your data across separate machines. This gave rise to the distributed filesystem and since these are network-based all the complications of network-programming kick in, such as filesystem tolerating node failure (when one of the systems supporting the distributed filesystem becomes … Continue reading HDFS
Journey to Hadoop
"Hadoop is now a kernel, for, pretty much an operating system for processing big data" - Doug Cutting, Hadoop Co-founder. But how did it come to be what is today ? To understand this journey to Hadoop, let's go back to when we initially hit the limits of computation. The period - the late 70's. AC/DC … Continue reading Journey to Hadoop
Birth of the Data Scientist
Have you been hearing the words "machine learning" and "data science" quite often lately? If you are in the IT industry then you must have joined a conference call where developers quite boldy state "And we are using machine learning to automate/predict... ". According to Google trends the interest in the field has been on the … Continue reading Birth of the Data Scientist
New Beginning
November 2015, posed no new challenges at first. As a Software Development Engineer for Order Processing at Dell Technologies my week ahead was fairly predictable - features, defects and deployments. However a new Director of Engineering had been appointed and he was slowly making waves across the team with innovations, quality improvement and new POCs. … Continue reading New Beginning