Introduction to Big Data and Hadoop

Introduction to Big Data and Hadoop

Nowadays 'data' is an important thing in the real world as well as in Software Development, and it is mandatory for every enterprise like, Banking, Insurance, Healthcare, ECommerce Telecom, Social Network etc...

1. What is Big Data?

> In order to understand the 'Big Data', we first need to know what 'data' is. 

"The quantities, characters, or symbols on which operations are performed by a computer.

> So, 'Big Data' is also a data but with a huge size. 'Big Data' is a term used to   describe collection of data that is huge in size.
2) Examples Of 'Big Data'

i. The New York Stock Exchange generates about one terabyte of new trade data per day.

ii. 500+terabytes of new data gets ingested into the databases of social media site Facebook, every day. This data is mainly generated in terms of photo and video uploads, message exchanges, putting comments etc.

Note: Nowadays Big Data platform used by IT giants Yahoo, Facebook & Google etc...and it is required for every Industry like Banking, Financial, Insurance,      ECommerece, Telecom and Government etc...
3) What are the Categories Of 'Big Data'?

Big data' could be found in three forms:

i. Structured
ii. Unstructured
iii. Semi-structured

i. Structured
> Any data that can be stored, accessed and processed in the form of fixed format is termed as a 'structured' data.

Examples Of Structured Data

An 'Employee' table in a database is an example of Structured Data
> Any data with unknown form or the structure is classified as unstructured Data. ex: data source containing a combination of simple text files, images, videos etc.

Examples Of Un-structured Data

Output returned by 'Google Search'
iii. Semi-structured
> Semi-structured data can contain both the forms of data. We can see semi-
  structured data as a strcutured in form but it is actually not defined with e.g. 
  a table definition in relational DBMS. 

Example of semi-structured data is a data represented in XML file.
4) What are the Characteristics Of 'Big Data'?

i. Volume – The name 'Big Data' itself is related to a size which is enormous.

ii. Variety - Data in various forms, Text Files, Tables, Images, Videos etc...

iii. Velocity – The term 'velocity' refers to the speed of generation of data. How  fast the data is generated and processed to meet the demands, determines real  potential in the data.

iv. Variability – This refers to the inconsistency which can be shown by the data at times.
5) What are the Benefits of Big Data Processing?

> Ability to process 'Big Data' brings in multiple benefits, such as-

i. Businesses can utilize outside intelligence while taking decisions

ii. Improved customer service

iii. Early identification of risk to the product/services, if any

iv. Better operational efficiency
6) What is Hadoop?

> Apache Hadoop is a framework used to develop data processing applications which are executed in a distributed computing environment. It is solution for Big Data Processing.

> We can’t compare Big Data and Hadoop because Big Data is a problem and Hadoop Provided solution to it. Hadoop developer and Hadoop administrator are fields of Hadoop.
7) What is the Difference between Big Data and Hadoop?

> Data is collected widely all over the world. This large amount of data is called Big data or Big Data and cannot be handled by regular storage devices.

> Hadoop software framework, which is an open source framework by the Apache Software Foundation, can be used to overcome this problem. The key difference between Big Data and Hadoop is that Big Data is a large quantity of complex data whereas Hadoop is a mechanism to store Big data effectively and efficiently.
Big Data Guide
Software Testing Videos (Manual Testing, Selenium, UFT/QTP, Live Project, Java, SQL, Python, VBScript, and FAQ)

Subscribe to G C Reddy Software Testing Video Channel (Get Regular Updates)

1) Selenium Step by Step Videos

2) Selenium Latest Videos

3) Selenium Quick Videos

4) Manual Testing Videos

5) UFT/QTP Videos

6) Java Videos

7) SQL Videos

8) Python Videos

9) Career Guidance

10) Introductions

12) Selenium Live Project

13) VBScript for UFT/QTP

14) Software Testing Interview Questions

15) Software Testing Practical

16) Selenium Detailed Videos

17) Health and Fitness Videos

18) TestNG Testing Framework for Selenium

19) Selenium Tutorials for Beginners…

20) Selenium Real Time Project...

21) Robotic Process Automation...

22) Selenium WebDriver Tutorials

23) Selenium 2018 Training Videos


Post a Comment