Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Pre-processing data refers to cleaning of data, removing invalid data, noise, replacing data with relevant values and so on. This doesn’t always mean text data; it could also be images or video processing as well.Data pre-processing basically refers to the task of gathering all the data (which is collected from various resources or a single resource) into a common format or into uniform datasets (depending on the type of data). Since real-world data is never ideal, there is a possibility that the data would have missing cells, errors, outliers, discrepancies in columns, and much more.Sometimes, images may not be correctly ... Read More
The idea behind using gradient descent is to minimize the loss when in various machine learning algorithms. Mathematically speaking, the local minimum of a function is obtained.To implement this, a set of parameters are defined, and they need to be minimized. Once the parameters are assigned coefficients, the error or loss is calculated. Next, the weights are updated to ensure that the error is minimized. Instead of parameters, weak learners can be users, such as decision trees.Once the loss is calculated, gradient descent is performed, and tree is added to the algorithm step wise, so that loss is minimal.Some examples ... Read More
When regression models are being built, multicollinearity is checked for. This is because we need to understand the correlation present between all different combinations of continuous variables. If multicollinearity exists between the variables, we have to make sure that it is removed from the data.The data in real world is usually non-linear. We need to find mechanisms to fit such non-linear data to the model. We will be using Anscombe’s dataset to visualize this data.The ‘implot’ function is used with non-linear data −Exampleimport pandas as pd import seaborn as sb from matplotlib import pyplot as plt my_df = sb.load_dataset('anscombe') sb.lmplot(x ... Read More
SciPy can be used to determine the permutation and combination with respect to two values.A function named ‘perm’ present in the class ‘special’ in ‘SciPy’ is used.Syntax of ‘perm’ functionscipy.special.perm(N, k)Performing permutation on a set of values has been shown belowExample Live Demofrom scipy.special import perm my_permute = perm(6, 2, exact = True) print("The permutation of 6 and 2 is ") print(my_permute)OutputThe permutation of 6 and 2 is 30ExplanationThe required libraries are imported.Parameters are passed to the ‘perm’ function that computes the value.The value is assigned to a variable.This variable is displayed on the console.Computing combination of two values in SciPyA ... Read More
When regression models are being built, multicollinearity is checked for. This is because we need to understand the correlation present between all different combinations of continuous variables. If multicollinearity exists between the variables, we have to make sure that it is removed from the data.This is where functions ‘regpot’ and ‘implot’ come into play. They help visualize a linear relationship between variables in linear regression.The ‘regplot’ function accepts values for variables ‘x’ and ‘y’ in a variety of formats, and this includes numpy arrays, pandas series objects, references to variables or values from a pandas dataframe.On the other hand, the ... Read More
Seaborn is a library that helps in visualizing data. It comes with customized themes and a high-level interface.When regression models are being built, multicollinearity is checked for. This is because we need to understand the correlation present between all different combinations of continuous variables. If multicollinearity exists between the variables, we have to make sure that it is removed from the data. This is where functions ‘regpot’ and ‘implot’ come into play. They help visualize a linear relationship between variables in linear regression.The ‘regplot’ function accepts values for variables ‘x’ and ‘y’ in a variety of formats, and this includes ... Read More
The barplot function establishes the relationship between a categorical variable and a continuous variable. Data is represented in the form of rectangular bars where the length of the bar indicates the proportion of data in that specific category.Point plots are similar to bar plots but instead of representing the fill bar, the estimated value of the data point is represented by a point at a specific height on the other axis.Categorical data can be visualized using categorical scatter plots or two separate plots with the help of pointplot or a higher level function known as factorplot. The factorplot function draws ... Read More
The barplot function establishes the relationship between a categorical variable and a continuous variable. Data is represented in the form of rectangular bars where the length of the bar indicates the proportion of data in that specific category.Point plots are similar to bar plots but instead of representing the fill bar, the estimated value of the data point is represented by a point at a specific height on the other axis.Categorical data can be visualized using categorical scatter plots or two separate plots with the help of pointplot or a higher level function known as factorplot.The factorplot function draws a ... Read More
Seaborn is a library that helps in visualizing data. It comes with customized themes and a high level interface.General scatter plots, histograms, etc can’t be used when the variables that need to be worked with are categorical in nature. This is when categorical scatterplots need to be used.Plots such as ‘stripplot’, ‘swarmplot’ are used to work with categorical variables. The ‘stripplot’ function is used when atleast one of the variable is categorical. The data is represented in a sorted manner along one of the axes.Syntax of stripplot functionseaborn.stripplot(x, y, data, …)Let us see how ‘stripplot’ function can be used to ... Read More
Series is a one-dimensional, labelled data structure present in the Pandas library. The axis label is collectively known as index.Series structure can store any type of data such as integer, float, string, python objects, and so on. It can be created using an array, a dictionary or a constant value.Let us see how an empty series can be created in Python −Example Live Demoimport pandas as pd my_series = pd.Series() print("This is an empty series data structure") print(my_series)OutputThis is an empty series data structure Series([], dtype: float64)ExplanationIn the above code, ‘pandas’ library is imported and given an alias name as ‘pd’.Next, ... Read More