Data wrangling involves processing the data in various formats like – merging, grouping, concatenating etc. for the aim of analysing or getting them able to be used with another set of data. Python has built-in features to use these wrangling methods to varied data sets to realize the analytical goal. during this blog we’ll check out few examples describing these methods.

Merging Data

The Pandas library in python provides one function, merge, because the entry point for all standard database join operations between DataFrame objects −

pd.merge(left, right, how='inner', on=None, left_on=None, right_on=None,
left_index=False, right_index=False, sort=True)

Let us now create two different DataFrames and perform the merging operations thereon.

# import the pandas library
import pandas as pd
left = pd.DataFrame({
 'id':[1,2,3,4,5],
 'Name': ['Alex', 'Amy', 'Allen', 'Alice', 'Ayoung'],
 'subject_id':['sub1','sub2','sub4','sub6','sub5']})
right = pd.DataFrame(
 {'id':[1,2,3,4,5],
 'Name': ['Billy', 'Brian', 'Bran', 'Bryce', 'Betty'],
 'subject_id':['sub2','sub4','sub3','sub6','sub5']})
print left
print right

Its output is as follows −

Name id subject_id

0 Alex 1 sub1
1 Amy 2 sub2
2 Allen 3 sub4
3 Alice 4 sub6
4 Ayoung 5 sub5

 Name id subject_id

0 Billy 1 sub2
1 Brian 2 sub4
2 Bran 3 sub3
3 Bryce 4 sub6
4 Betty 5 sub5

Grouping Data

Grouping data sets may be a frequent need in data analysis where we’d like the end in terms of varied groups present within the data set. Panadas has in-built methods which may roll the data into various groups.

In the below example we group the data by year then get the result for a selected year.

# import the pandas library
import pandas as pd
ipl_data = {'Team': ['Riders', 'Riders', 'Devils', 'Devils', 'Kings',
 'kings', 'Kings', 'Kings', 'Riders', 'Royals', 'Royals', 'Riders'],
 'Rank': [1, 2, 2, 3, 3,4 ,1 ,1,2 , 4,1,2],
 'Year': [2014,2015,2014,2015,2014,2015,2016,2017,2016,2014,2015,2017],
 'Points':[876,789,863,673,741,812,756,788,694,701,804,690]}
df = pd.DataFrame(ipl_data)
grouped = df.groupby('Year')
print grouped.get_group(2014)
Its output is as follows −
 Points Rank Team Year
0 876 1 Riders 2014
2 863 2 Devils 2014
4 741 3 Kings 2014
9 701 4 Royals 2014

Concatenating Data

Pandas provides various facilities for easily combining together Series, DataFrame, and Panel objects. within the below example the concat function performs concatenation operations along an axis. allow us to create different objects and do concatenation.

import pandas as pd

one = pd.DataFrame({

 'Name': ['Alex', 'Amy', 'Allen', 'Alice', 'Ayoung'],

 'subject_id':['sub1','sub2','sub4','sub6','sub5'],

 'Marks_scored':[98,90,87,69,78]},

 index=[1,2,3,4,5])

two = pd.DataFrame({

 'Name': ['Billy', 'Brian', 'Bran', 'Bryce', 'Betty'],

 'subject_id':['sub2','sub4','sub3','sub6','sub5'],

 'Marks_scored':[89,80,79,97,88]},

 index=[1,2,3,4,5])

print pd.concat([one,two])

Its output is as follows −

 Marks_scored Name subject_id

1 98 Alex sub1

2 90 Amy sub2

3 87 Allen sub4

4 69 Alice sub6

5 78 Ayoung sub5

1 89 Billy sub2

2 80 Brian sub4

3 79 Bran sub3

4 97 Bryce sub6

5 88 Betty sub5

So, this brings us to the end of blog. This Tecklearn ‘Performing Data Wrangling in Python’ blog helps you with commonly asked questions if you are looking out for a job in Python Programming. If you wish to learn Python and build a career in Data Science domain, then check out our interactive, Python with Data Science Training, that comes with 24*7 support to guide you throughout your learning period. Please find the link for course details:

https://www.tecklearn.com/course/python-with-data-science/

Python with Data Science Training

About the Course

Python with Data Science training lets you master the concepts of the widely used and powerful programming language, Python. This Python Course will also help you master important Python programming concepts such as data operations, file operations, object-oriented programming and various Python libraries such as Pandas, NumPy, Matplotlib which are essential for Data Science. You will work on real-world projects in the domain of Python and apply it for various domains of Big Data, Data Science and Machine Learning.

Why Should you take Python with Data Science Training?

Python is the preferred language for new technologies such as Data Science and Machine Learning.
Average salary of Python Certified Developer is $123,656 per annum – Indeed.com
Python is by far the most popular language for data science. Python held 65.6% of the data science market.

What you will Learn in this Course?

Introduction to Python

Define Python
Understand the need for Programming
Know why to choose Python over other languages
Setup Python environment
Understand Various Python concepts – Variables, Data Types Operators, Conditional Statements and Loops
Illustrate String formatting
Understand Command Line Parameters and Flow control

Python Environment Setup and Essentials

Python installation
Windows, Mac & Linux distribution for Anaconda Python
Deploying Python IDE
Basic Python commands, data types, variables, keywords and more

Python language Basic Constructs

Looping in Python
Data Structures: List, Tuple, Dictionary, Set
First Python program
Write a Python Function (with and without parameters)
Create a member function and a variable
Tuple
Dictionary
Set and Frozen Set
Lambda function

OOP (Object Oriented Programming) in Python

Object-Oriented Concepts

Working with Modules, Handling Exceptions and File Handling

Standard Libraries
Modules Used in Python (OS, Sys, Date and Time etc.)
The Import statements
Module search path
Package installation ways
Errors and Exception Handling
Handling multiple exceptions

Introduction to NumPy

Introduction to arrays and matrices
Indexing of array, datatypes, broadcasting of array math
Standard deviation, Conditional probability
Correlation and covariance
NumPy Exercise Solution

Introduction to Pandas

Pandas for data analysis and machine learning
Pandas for data analysis and machine learning Continued
Time series analysis
Linear regression
Logistic Regression
ROC Curve
Neural Network Implementation
K Means Clustering Method

Data Visualisation

Matplotlib library
Grids, axes, plots
Markers, colours, fonts and styling
Types of plots – bar graphs, pie charts, histograms
Contour plots

Data Manipulation

Perform function manipulations on Data objects
Perform Concatenation, Merging and Joining on DataFrames
Iterate through DataFrames
Explore Datasets and extract insights from it

Scikit-Learn for Natural Language Processing

What is natural language processing, working with NLP on text data
Scikit-Learn for Natural Language Processing
The Scikit-Learn machine learning algorithms
Sentimental Analysis – Twitter

Introduction to Python for Hadoop

Deploying Python coding for MapReduce jobs on Hadoop framework.
Python for Apache Spark coding
Deploying Spark code with Python
Machine learning library of Spark MLlib
Deploying Spark MLlib for Classification, Clustering and Regression

Got a question for us? Please mention it in the comments section and we will get back to you.

611