GroupBy is a pretty simple concept. We can create a grouping of categories and apply a function to the categories. It’s a simple concept, but it’s an extremely valuable technique that’s widely used in data science. In real data science projects, you’ll be dealing with large amounts of data and trying things over and over, so for efficiency, we use Groupby concept. Groupby concept is really important because of its ability to summarize, aggregate, and group data efficiently.
Summarize
Summarization includes counting, describing all the data present in data frame. We can summarize the data present in the data frame using describe() method. This method is used to get min, max, sum, count values from the data frame along with data types of that particular column.
- describe(): This method elaborates the type of data and its attributes.
Syntax:
dataframe_name.describe()
- unique(): This method is used to get all unique values from the given column.
Syntax:
dataframe[‘column_name].unique()
- nunique(): This method is similar to unique but it will return the count the unique values.
Syntax:
dataframe_name[‘column_name].nunique()
- info(): This command is used to get the data types and columns information
Syntax:
dataframe.info()
- columns: This command is used to display all the column names present in data frame
Syntax:
dataframe.columns
Example:
We are going to analyze the student marks data in this example.
Python3
# importing pandas as pd for using data frameimport pandas as pd# creating dataframe with student detailsdataframe = pd.DataFrame({'id': [7058, 4511, 7014, 7033], 'name': ['sravan', 'manoj', 'aditya', 'bhanu'], 'Maths_marks': [99, 97, 88, 90], 'Chemistry_marks': [89, 99, 99, 90], 'telugu_marks': [99, 97, 88, 80], 'hindi_marks': [99, 97, 56, 67], 'social_marks': [79, 97, 78, 90], })# display dataframedataframe |
Output:
Python3
# describing the data frameprint(dataframe.describe())print("-----------------------------")# finding unique valuesprint(dataframe['Maths_marks'].unique())print("-----------------------------")# counting unique valuesprint(dataframe['Maths_marks'].nunique())print("-----------------------------")# display the columns in the data frameprint(dataframe.columns)print("-----------------------------")# information about dataframeprint(dataframe.info()) |
Output:
Aggregation
Aggregation is used to get the mean, average, variance and standard deviation of all column in a dataframe or particular column in a data frame.
- sum(): It returns the sum of the data frame
Syntax:
dataframe[‘column].sum()
- mean(): It returns the mean of the particular column in a data frame
Syntax:
dataframe[‘column].mean()
- std(): It returns the standard deviation of that column.
Syntax:
dataframe[‘column].std()
- var(): It returns the variance of that column
dataframe[‘column’].var()
- min(): It returns the minimum value in column
Syntax:
dataframe[‘column’].min()
- max(): It returns maximum value in column
Syntax:
dataframe[‘column’].max()
Example:
In the below program we will aggregate data.
Python3
# importing pandas as pd for using data frameimport pandas as pd# creating dataframe with student detailsdataframe = pd.DataFrame({'id': [7058, 4511, 7014, 7033], 'name': ['sravan', 'manoj', 'aditya', 'bhanu'], 'Maths_marks': [99, 97, 88, 90], 'Chemistry_marks': [89, 99, 99, 90], 'telugu_marks': [99, 97, 88, 80], 'hindi_marks': [99, 97, 56, 67], 'social_marks': [79, 97, 78, 90], })# display dataframedataframe |
Output:
Python3
# getting all minimum values from# all columns in a dataframeprint(dataframe.min())print("-----------------------------------------")# minimum value from a particular# column in a data frameprint(dataframe['Maths_marks'].min())print("-----------------------------------------")# computing maximum valuesprint(dataframe.max())print("-----------------------------------------")# computing sumprint(dataframe.sum())print("-----------------------------------------")# finding countprint(dataframe.count())print("-----------------------------------------")# computing standard deviationprint(dataframe.std())print("-----------------------------------------")# computing varianceprint(dataframe.var()) |
Output:
Grouping
It is used to group one or more columns in a dataframe by using the groupby() method. Groupby mainly refers to a process involving one or more of the following steps they are:
- Splitting: It is a process in which we split data into group by applying some conditions on datasets.
- Applying: It is a process in which we apply a function to each group independently
- Combining: It is a process in which we combine different datasets after applying groupby and results in a data structure
Example 1:
Python3
# importing pandas as pd for using data frameimport pandas as pd# creating dataframe with student detailsdataframe = pd.DataFrame({'id': [7058, 4511, 7014, 7033], 'name': ['sravan', 'manoj', 'aditya', 'bhanu'], 'Maths_marks': [99, 97, 88, 90], 'Chemistry_marks': [89, 99, 99, 90], 'telugu_marks': [99, 97, 88, 80], 'hindi_marks': [99, 97, 56, 67], 'social_marks': [79, 97, 78, 90], })# group by nameprint(dataframe.groupby('name').first())print("---------------------------------")# group by name with social_marks sumprint(dataframe.groupby('name')['social_marks'].sum())print("---------------------------------")# group by name with maths_marks countprint(dataframe.groupby('name')['Maths_marks'].count())print("---------------------------------")# group by name with maths_marksprint(dataframe.groupby('name')['Maths_marks']) |
Output:
Example 2:
Python3
# importing pandas as pd for using data frameimport pandas as pd# creating dataframe with student detailsdataframe = pd.DataFrame({'id': [7058, 4511, 7014, 7033], 'name': ['sravan', 'manoj', 'aditya', 'bhanu'], 'Maths_marks': [99, 97, 88, 90], 'Chemistry_marks': [89, 99, 99, 90], 'telugu_marks': [99, 97, 88, 80], 'hindi_marks': [99, 97, 56, 67], 'social_marks': [79, 97, 78, 90], })# group by nameprint(dataframe.groupby('name').first())print("------------------------")# group by name with social_marks sumprint(dataframe.groupby('name')['social_marks'].sum())print("------------------------")# group by name with maths_marks countprint(dataframe.groupby('name')['Maths_marks'].count()) |
Output:
