All data needs to be clean before you can explore and create models. Common sense, right. Cleaning data can be tedious but I created a function that will help.
The function do the following:
- Clean Data from NA’s and Blanks
- Separate the clean data – Integer dataframe, Double dataframe, Factor dataframe, Numeric dataframe, and Factor and Numeric dataframe.
- View the new dataframes
- Create a view of the summary and describe from the clean data.
- Create histograms of the data frames.
- Save all the objects
This will happen in seconds.
Package
First, load Hmisc
package. I always save the original file.
The code below is the engine that cleans the data file.
cleandata <- dataname[complete.cases(dataname),]
The function
The function is below. You need to copy the code and save it in an R file. Run the code and the function cleanme
will appear.
cleanme <- function(dataname){ #SAVE THE ORIGINAL FILE oldfile <- write.csv(dataname, file = "oldfile.csv", row.names = FALSE, na = "") #CLEAN THE FILE. SAVE THE CLEAN. IMPORT THE CLEAN FILE. CHANGE THE TO A DATAFRAME. cleandata <- dataname[complete.cases(dataname),] cleanfile <- write.csv(cleandata, file = "cleanfile.csv", row.names = FALSE, na = "") cleanfileread <- read.csv(file = "cleanfile.csv") cleanfiledata <- as.data.frame(cleanfileread) #SUBSETTING THE DATA TO TYPES logicmeint <- cleandata[,sapply(cleandata,is.integer)] logicmedouble <- cleandata[,sapply(cleandata,is.double)] logicmefactor <- cleandata[,sapply(cleandata,is.factor)] logicmenum <- cleandata[,sapply(cleandata,is.numeric)] mainlogicmefactors <- cleandata[,sapply(cleandata,is.factor) | sapply(cleandata,is.numeric)] #VIEW ALL FILES View(cleandata) View(logicmeint) View(logicmedouble) View(logicmefactor) View(logicmenum) View(mainlogicmefactors) #describeFast(mainlogicmefactors) #ANALYTICS OF THE MAIN DATAFRAME cleansum <- summary(cleanfiledata) print(cleansum) cleandec <- describe(cleanfiledata) print(cleandec) #ANALYTICS OF THE FACTOR DATAFRAME factorsum <- summary(logicmefactor) print(factorsum) factordec <- describe(logicmefactor) print(factordec) #ANALYTICS OF THE NUMBER DATAFRAME numbersum <- summary(logicmenum) print(numbersum) numberdec <- describe(logicmefactor) print(numberdec) mainlogicmefactorsdec <- describe(mainlogicmefactors) print(mainlogicmefactorsdec) mainlogicmefactorssum <- describe(mainlogicmefactors) print(mainlogicmefactorssum) #savemenow <- saveRDS("cleanmework.rds") #readnow <- readRDS(savemenow) #HISTOGRAM PLOTS OF ALL TYPES hist(cleandata) hist(logicmeint) hist(logicmedouble) hist(logicmefactor) hist(logicmenum) #plot(mainlogicmefactors) #save(cleanfiledata, logicmeint, mainlogicmefactors, logicmedouble, logicmefactor, logicmenum, numberdec, numbersum, factordec, factorsum, cleandec, oldfile, cleandata, cleanfile, cleanfileread, #file = "cleanmework.RData") }
Type in and run:
cleanme(dataname)
When all the data frames appear, type to load the workspace as objects.
load("cleanmework.RData")
Enjoy