Article Categories
- All Categories
-
Data Structure
-
Networking
-
RDBMS
-
Operating System
-
Java
-
MS Excel
-
iOS
-
HTML
-
CSS
-
Android
-
Python
-
C Programming
-
C++
-
C#
-
MongoDB
-
MySQL
-
Javascript
-
PHP
-
Economics & Finance
Articles by Nitin
3 articles
What is Bucketing in Hive?
Bucketing is a method in Hive which is used for organizing the data. It is a concept of separating data into ranges known as buckets. Bucketing in hives comes helpful when the use of partitioning becomes hard. A user can determine the range of a specific bucket by the hash value. Partitioned tables can be bucketed to separate the data further to perform queries more efficiently. Every bucket is stored as a file within the table or the partition’s directories on HDFS. The records having a similar value within a column are always stored in the same bucket. Bucketing can ...
Read MoreRDD Shared Variables In Spark
The full name of the RDD is a distributed database. Spark performance is based on this ambiguous set, enabling it to consistently cope with major data processing conditions, including MapReduce, streaming, SQL, machine learning, graphs, etc. Spark supports many programming languages, including Scala, Python, and R. RDD also supports the maintenance of material in these languages. How to create RDD Spark supports RDDS architecture in many areas, including local file systems, HDFS file systems, memory, and HBase. For the local file system, we can create RDD through the following way − val distFile = sc.textFile("file:///user/root/rddData.txt") By default, Spark takes ...
Read MoreSqoop Integration with Hadoop Ecosystem
Data was previously stored in relational data management systems when Hadoop and big data concepts were not available. After introducing Big Data concepts, it was essential to store the data more concisely and efficiently. However all data stored in the related data management system needs to be transferred to the Hadoop archive. With Sqoop, we can transfer this amount of personal data. Sqoop transfers data from a related database management system to a Hadoop server. Thus, it facilitates the transfer of large volumes of data from one source to another. Here are the basic features of Sqoop − Sqoop ...
Read More