We use Regression analysis to create models which describe the effect of variation in predictor variables on the response variable. Sometimes, if we have a categorical variable with values like Yes/No or Male/Female etc. The simple regression analysis gives multiple results for each value of the categorical variable. In such scenario, we can study the effect of the categorical variable by using it along with the predictor variable and comparing the regression lines for each level of the categorical variable. Such an analysis is termed as Analysis of Covariance also called as ANCOVA.

Example

Consider the R built in data set mtcars. In it we observer that the field “am” represents the type of transmission (auto or manual). It is a categorical variable with values 0 and 1. The miles per gallon value(mpg) of a car can also depend on it besides the value of horse power(“hp”).

We study the effect of the value of “am” on the regression between “mpg” and “hp”. It is done by using the aov() function followed by the anova() function to compare the multiple regressions.

Input Data

Create a data frame containing the fields “mpg”, “hp” and “am” from the data set mtcars. Here we take “mpg” as the response variable, “hp” as the predictor variable and “am” as the categorical variable.

  

input <- mtcars[,c("am","mpg","hp")]
print(head(input))

When we execute the above code, it produces the following result −

                   am   mpg   hp

Mazda RX4          1    21.0  110

Mazda RX4 Wag      1    21.0  110

Datsun 710         1    22.8   93

Hornet 4 Drive     0    21.4  110

Hornet Sportabout  0    18.7  175

Valiant            0    18.1  105

ANCOVA Analysis

We create a regression model taking “hp” as the predictor variable and “mpg” as the response variable taking into account the interaction between “am” and “hp”.

Model with interaction between categorical variable and predictor variable

  

# Get the dataset.

input <- mtcars
# Create the regression model.
result <- aov(mpg~hp*am,data = input)
print(summary(result))

When we execute the above code, it produces the following result −

Df Sum Sq Mean Sq F value Pr(>F)

hp 1 678.4 678.4 77.391 1.50e-09 ***

am 1 202.2 202.2 23.072 4.75e-05 ***

hp:am 1 0.0 0.0 0.001 0.981

Residuals 28 245.4 8.8

—

Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

This result shows that both horse power and transmission type has significant effect on miles per gallon as the p value in both cases is less than 0.05. But the interaction between these two variables is not significant as the p-value is more than 0.05.

Model without interaction between categorical variable and predictor variable

  

# Get the dataset.
input <- mtcars
# Create the regression model.
result <- aov(mpg~hp+am,data = input)
print(summary(result))

When we execute the above code, it produces the following result −

            Df  Sum Sq  Mean Sq   F value   Pr(>F)   

hp           1  678.4   678.4   80.15 7.63e-10 ***

am           1  202.2   202.2   23.89 3.46e-05 ***

Residuals   29  245.4     8.5                    

---

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

This result shows that both horse power and transmission type has significant effect on miles per gallon as the p value in both cases is less than 0.05.

Comparing Two Models

Now we can compare the two models to conclude if the interaction of the variables is truly in-significant. For this we use the anova() function.

  

# Get the dataset.
input <- mtcars
# Create the regression models.
result1 <- aov(mpg~hp*am,data = input)
result2 <- aov(mpg~hp+am,data = input)
# Compare the two models.
print(anova(result1,result2))

When we execute the above code, it produces the following result −

Model 1: mpg ~ hp * am
Model 2: mpg ~ hp + am
  Res.Df    RSS Df  Sum of Sq     F Pr(>F)
1     28 245.43                          
2     29 245.44 -1 -0.0052515 6e-04 0.9806

As the p-value is greater than 0.05 we conclude that the interaction between horse power and transmission type is not significant. So, the mileage per gallon will depend in a similar manner on the horse power of the car in both auto and manual transmission mode.

So, this brings us to the end of blog. This Tecklearn ‘Analysis of Covariance in R Language’ blog helps you with commonly asked questions if you are looking out for a job in Data Science. If you wish to learn R Language and build a career in Data Science domain, then check out our interactive, Data Science using R Language Training, that comes with 24*7 support to guide you throughout your learning period. Please find the link for course details:

https://www.tecklearn.com/course/data-science-training-using-r-language/

Data Science using R Language Training

About the Course

Tecklearn’s Data Science using R Language Training develops knowledge and skills to visualize, transform, and model data in R language. It helps you to master the Data Science with R concepts such as data visualization, data manipulation, machine learning algorithms, charts, hypothesis testing, etc. through industry use cases, and real-time examples. Data Science course certification training lets you master data analysis, R statistical computing, connecting R with Hadoop framework, Machine Learning algorithms, time-series analysis, K-Means Clustering, Naïve Bayes, business analytics and more. This course will help you gain hands-on experience in deploying Recommender using R, Evaluation, Data Transformation etc.

Why Should you take Data Science Using R Training?

The Average salary of a Data Scientist in R is $123k per annum – Glassdoor.com
A recent market study shows that the Data Analytics Market is expected to grow at a CAGR of 30.08% from 2020 to 2023, which would equate to $77.6 billion.
IBM, Amazon, Apple, Google, Facebook, Microsoft, Oracle & other MNCs worldwide are using data science for their Data analysis.

What you will Learn in this Course?

Introduction to Data Science

Need for Data Science
What is Data Science
Life Cycle of Data Science
Applications of Data Science
Introduction to Big Data
Introduction to Machine Learning
Introduction to Deep Learning
Introduction to R&R-Studio
Project Based Data Science

Introduction to R

Introduction to R
Data Exploration
Operators in R
Inbuilt Functions in R
Flow Control Statements & User Defined Functions
Data Structures in R

Data Manipulation

Need for Data Manipulation
Introduction to dplyr package
Select (), filter(), mutate(), sample_n(), sample_frac() & count() functions
Getting summarized results with the summarise() function,
Combining different functions with the pipe operator
Implementing sql like operations with sqldf()

Visualization of Data

Loading different types of datasets in R
Arranging the data
Plotting the graphs

Introduction to Statistics

Types of Data
Probability
Correlation and Co-variance
Hypothesis Testing
Standardization and Normalization

Introduction to Machine Learning

What is Machine Learning?
Machine Learning Use-Cases
Machine Learning Process Flow
Machine Learning Categories
Supervised Learning algorithm: Linear Regression and Logistic Regression

Logistic Regression

Intro to Logistic Regression
Simple Logistic Regression in R
Multiple Logistic Regression in R
Confusion Matrix
ROC Curve

Classification Techniques

What are classification and its use cases?
What is Decision Tree?
Algorithm for Decision Tree Induction
Creating a Perfect Decision Tree
Confusion Matrix
What is Random Forest?
What is Naive Bayes?
Support Vector Machine: Classification

Decision Tree

Decision Tree in R
Information Gain
Gini Index
Pruning

Recommender Engines

What is Association Rules & its use cases?
What is Recommendation Engine & it’s working?
Types of Recommendations
User-Based Recommendation
Item-Based Recommendation
Difference: User-Based and Item-Based Recommendation
Recommendation use cases

Time Series Analysis

What is Time Series data?
Time Series variables
Different components of Time Series data
Visualize the data to identify Time Series Components
Implement ARIMA model for forecasting
Exponential smoothing models
Identifying different time series scenario based on which different Exponential Smoothing model can be applied

Got a question for us? Please mention it in the comments section and we will get back to you.

720