Found 507 Articles for Pandas

Assume, you have a dataframe and the shift index by two periods in positive and negative direction is, shift the index by three periods in positive direction                      Id Age 2020-01-01 00:00:00 NaN NaN 2020-01-01 12:00:00 NaN NaN 2020-01-02 00:00:00 1.0 10.0 2020-01-02 12:00:00 2.0 12.0 2020-01-03 00:00:00 3.0 14.0 shift the index by three periods in negative direction                      Id Age 2020-01-01 00:00:00 3.0 14.0 2020-01-01 12:00:00 4.0 11.0 2020-01-02 00:00:00 5.0 13.0 2020-01-02 12:00:00 NaN NaN 2020-01-03 00:00:00 NaN NaNSolutionTo ... Read More

Assume, you have a dataframe and the result for removing first duplicate rows are,     Id Age 0    1 12 3    4 13 4    5 14 5    6 12 6    2 13 7    7 16 8    3 14 9    9 15 10  10 14SolutionTo solve this, we will follow the steps given below −Define a dataframeApply drop_duplicates function inside Id and Age column then assign keep initial value as ‘last’.df.drop_duplicates(subset=['Id', 'Age'], keep='last')Store the result inside same dataframe and print itExampleLet’s see the below implementation to get a better understanding −import pandas ... Read More

Assume, you have a dataframe and the result for calculating covariance from grouped data and corresponding column as, Grouped data covariance is:                   mark1       mark2 subjects maths    mark1    25.0    12.500000          mark2    12.5    108.333333 science  mark1    28.0    50.000000          mark2    50.0    233.333333 Grouped data covariance between two columns: subjects maths    12.5 science  50.0 dtype: float64SolutionTo solve this, we will follow the steps given below −Define a dataframeApply groupby function inside dataframe subjects ... Read More

We can reshape a dataframe using melt(), stack(), unstack() and pivot() function.Solution 1Define a dataframe.Apply melt() function to convert wide dataframe column as rows. It is defined below, df.melt()ExampleLet’s see the below code to get a better understanding −import pandas as pd df = pd.DataFrame({'Id':[1, 2, 3], 'Age':[13, 14, 13], 'Mark':[80, 90, 85]}) print("Dataframe is:", df) print(df.melt())OutputDataframe is:  Id Age Mark 0 1 13   80 1 2 14   90 2 3 13   85 variable value 0    Id    1 1    Id    2 2    Id    3 3   Age    13 4   ... Read More

Assume you have a dataframe with time series data and the result for truncated data is, before truncate:  Id time_series 0 1 2020-01-05 1 2 2020-01-12 2 3 2020-01-19 3 4 2020-01-26 4 5 2020-02-02 5 6 2020-02-09 6 7 2020-02-16 7 8 2020-02-23 8 9 2020-03-01 9 10 2020-03-08 after truncate:  Id time_series 1 2 2020-01-12SolutionTo solve this, we will follow the steps given below −Define a dataframe.Create date_range function inside start=’01/01/2020’, periods = 10 and assign freq = ‘W’. It will generate ten dates from given start date to next weekly start dates and store it as df[‘time_series’].df['time_series'] ... Read More

Assume, you have series and the result for autocorrelation with lag 2 is, Series is: 0    2.0 1    10.0 2    3.0 3    4.0 4    9.0 5    10.0 6    2.0 7    NaN 8    3.0 dtype: float64 series correlation:    -0.4711538461538461 series correlation with lags:    -0.2933396642805515SolutionTo solve this, we will follow the steps given below −Define a seriesFind the series autocorrelation using the below method, series.autocorr()Calculate the autocorrelation with lag=2 as follows, series.autocorr(lag=2)ExampleLet’s see the below code to get a better understanding, import pandas as pd import numpy as np series = ... Read More

Assume you have a dataframe and the result for exporting into pickle file and read the contents from file as, Export to pickle file: Read contents from pickle file:   Fruits    City 0 Apple    Shimla 1 Orange   Sydney 2 Mango    Lucknow 3 Kiwi    WellingtonSolutionTo solve this, we will follow the steps given below −Define a dataframe.Export the dataframe to pickle format and name it as ‘pandas.pickle’, df.to_pickle('pandas.pickle')Read the contents from ‘pandas.pickle’ file and store it as result, result = pd.read_pickle('pandas.pickle')ExampleLet’s see the below implementation to get better understanding, import pandas as pd df = pd.DataFrame({'Fruits': ... Read More

Assume you have the following sample json data stored in a file as pandas_sample.json{    "employee": {       "name": "emp1",       "salary": 50000,       "age": 31    } }The result for after converting to csv as, , employee age, 31 name, emp1 salary, 50000SolutionTo solve this, we will follow the steps given below −Create pandas_sample.json file and store the JSON data.Read json data from the file and store it as data.data = pd.read_json('pandas_sample.json')Convert the data into dataframedf = pd.DataFrame(data)Apple df.to_csv function to convert the data as csv file format, df.to_csv('pandas_json.csv')ExampleLet’s see the below implementation ... Read More

Assume, you have time series and the result for maximum month-end frequency, DataFrame is:  Id time_series 0 1 2020-01-05 1 2 2020-01-12 2 3 2020-01-19 3 4 2020-01-26 4 5 2020-02-02 5 6 2020-02-09 6 7 2020-02-16 7 8 2020-02-23 8 9 2020-03-01 9 10 2020-03-08 Maximum month end frequency:               Id time_series time_series 2020-01-31    4 2020-01-26 2020-02-29    8 2020-02-23 2020-03-31    10 2020-03-08SolutionTo solve this, we will follow the steps given below −Define a dataframe with one column, d = {'Id': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]} ... Read More

Assume, we have already saved pandas.csv file and export the file to Html formatSolutionTo solve this, we will follow the steps given below −Read the csv file using the read_csv method as follows −df = pd.read_csv('pandas.csv')Create new file pandas.html in write mode using file object, f = open('pandas.html', 'w')Declare result variable to convert dataframe to html file format, result = df.to_html()Using the file object, write all the data from the result. Finally close the file object, f.write(result) f.close()ExampleLet’s see the below implementation to get a better understanding −import pandas as pd df = pd.read_csv('pandas.csv') print(df) f = open('pandas.html', 'w') result ... Read More