Before manipulating the dataframe with pandas we have to understand what is data manipulation. The data in the real world is very unpleasant & unordered so by performing certain operations we can make data understandable based on one’s requirements, this process of converting unordered data into meaningful information can be done by data manipulation.
Here, we will learn how to manipulate dataframes with pandas. Pandas is an open-source library that is used from data manipulation to data analysis & is very powerful, flexible & easy to use tool which can be imported using import pandas as pd. Pandas deal essentially with data in 1-D and 2-D arrays; Although, pandas handles these two differently. In pandas, 1-D arrays are stated as a series & a dataframe is simply a 2-D array. The dataset used here is country_code.csv.
Below are various operations used to manipulate the dataframe:
- First, import the library which is used in data manipulation i.e. pandas then assign and read the dataframe:
Python3
# import moduleimport pandas as pd# assign datasetdf = pd.read_csv("country_code.csv")# displayprint("Type-", type(df))df |
Output:
- We can read the dataframe by using head() function also which is having an argument (n) i.e. number of rows to be displayed.
Python3
df.head(10) |
Output:
- Counting the rows and columns in DataFrame using shape(). It returns the no. of rows and columns enclosed in a tuple.
Python3
df.shape |
Output:
- Summary of Statistics of DataFrame using describe() method.
Python3
df.describe() |
Output:
- Dropping the missing values in DataFrame, it can be done using the dropna() method, it removes all the NaN values in the dataframe.
Python3
df.dropna() |
Output:
Another example is:
Python3
df.dropna(axis=1) |
This will drop all the columns with any missing values.
Output:
- Merging DataFrames using merge(), arguments passed are the dataframes to be merged along with the column name.
Python3
df1 = pd.read_csv("country_code.csv")merged_col = pd.merge(df, df1, on='Name')merged_col |
Output:
- An additional argument ‘on’ is the name of the common column, here ‘Name’ is the common column given to the merge() function. df is the first dataframe & df1 is the second dataframe that is to be merged.
- Renaming the columns of dataframe using rename(), arguments passed are the columns to be renamed & inplace.
Python3
country_code = df.rename(columns={'Name': 'CountryName', 'Code': 'CountryCode'}, inplace=False)country_code |
Output:
The code ‘inplace = False’ means the result would be stored in a new DataFrame instead of the original one.
- Creating a dataframe manually:
Python3
student = pd.DataFrame({'Name': ['Rohan', 'Rahul', 'Gaurav', 'Ananya', 'Vinay', 'Rohan', 'Vivek', 'Vinay'], 'Score': [76, 69, 70, 88, 79, 64, 62, 57]})# Reading Dataframestudent |
Output:
- Sorting the DataFrame using sort_values() method.
Python3
student.sort_values(by=['Score'], ascending=True) |
Output:
- Sorting the DataFrame using multiple columns:
Python3
student.sort_values(by=['Name', 'Score'], ascending=[True, False]) |
Output:
- Creating another column in DataFrame, Here we will create column name percentage which will calculate the percentage of student score by using aggregate function sum().
Python3
student['Percentage'] = (student['Score'] / student['Score'].sum()) * 100student |
Output:
- Selecting DataFrame rows using logical operators:
Python3
# Selecting rows where score is# greater than 70print(student[student.Score>70])# Selecting rows where score is greater than 60# OR less than 70print(student[(student.Score>60) | (student.Score<70)]) |
Output:
- Indexing & Slicing :
Here .loc is label base & .iloc is integer position based methods used for slicing & indexing of data.
Python3
# Printing five rows with name column only# i.e. printing first 5 student names.print(student.loc[0:4, 'Name'])# Printing all the rows with score column# only i.e. printing score of all the# studentsprint(student.loc[:, 'Score'])# Printing only first rows having name,# score columns i.e. print first student# name & their score.print(student.iloc[0, 0:2])# Printing first 3 rows having name,score &# percentage columns i.e. printing first three# student name,score & percentage.print(student.iloc[0:3, 0:3])# Printing all rows having name & score# columns i.e. printing all student# name & their score.print(student.iloc[:, 0:2]) |
Output:
.loc:
.iloc:
- Apply Functions, this function is used to apply a function along an axis of dataframe whether it can be row (axis=0) or column (axis=1).
Python3
# explicit functiondef double(a): return 2*astudent['Score'] = student['Score'].apply(double)# Reading Dataframestudent |
Output:
