How John Deere uses R

HT: Revolution Analytics
 
Very good discussion about real applied econometrics and analytics including the use of ARIMA models, decision trees, and genetic algorithms. He also has a very smart approach in his attitude toward big data and data science. I also appreciated his views on the pitfalls of 'black box' analytics software and outsourcing analytics. I do think there is a role for analytical consulting, but it requires careful management with a close client-partner relationship with NDA, contract, etc. to be effective.

Intuition Behind Eigenvector Centrality

The following R code visualizes the network and provides intuitive connections between degree and eigenvector centrality as well as the power iteration method of estimating  the dominant eigenvector from a matrix.

# *------------------------------------------------------------------
# | PROGRAM NAME: EV Centrality v3
# | DATE: 11/9/12
# | CREATED BY: MATT BOGARD
# | PROJECT FILE: P:\R  Code References\SNA            
# *----------------------------------------------------------------
# | PURPOSE:  An update to my companion code to Justification and Application of
# |  Eigenvector Centrality by Leo Spizzirri        
# |  https://www.math.washington.edu/~morrow/336_11/papers/leo.pdf
# *------------------------------------------------------------------
 
# specify the adjacency matrix
A <- matrix(c(0,1,0,0,0,0,
                        1,0,1,0,0,0,
                        0,1,0,1,1,1,
                        0,0,1,0,1,0,
                        0,0,1,1,0,1,
                        0,0,1,0,1,0 ),6,6, byrow= TRUE)
 
# plot the network
library(igraph)
G<-graph.adjacency(A, mode=c("undirected")) # convert adjacency matrix to an igraph object
plot(G, layout = layout.fruchterman.reingold)  # initial plot
cent<-data.frame(bet=betweenness(G),eig=evcent(G)$vector) # calculate betweeness & eigenvector centrality 
 
# create vertex names and scale by centrality
plot(G, layout = layout.fruchterman.reingold, vertex.size = 20*evcent(G)$vector, vertex.label = as.factor(rownames(cent)), main = 'Network Visualization in R')
 
#-----------------------------------------------------------------
# compute eigenvalues and eigenvectors directly via eigen function
#------------------------------------------------------------------            
EV <- eigen(A) 
max(EV$values)  # find the maximum eigenvalue
 
# get the eigenvector associated with the largest eigenvalue
centrality <- data.frame(EV$vectors[,1]) 
names(centrality) <- "Centrality"
print(centrality)
 
#-------------------------------------------------
# user defined calculations for degree centrality
#--------------------------------------------------
 
# compute degree centrality
x <- c(1,1,1,1,1,1)
 
A%*%x # sum of all 1st degree connections
# gives sum of # of 1st degree connections of neighbors
A%*%A%*%x
# gives sum of # of 2nd degree connections of neighbors
A%*%A%*%A%*%x
 
#-----------------------------------------------------
#  intuition behind eigenvector centraltiy
#----------------------------------------------------
 
# function for iterative summation      
MM <- function(k,M){
                  B_k <- NULL
       B_k <- M
 
           for (i in 1:k){
    B_k <- B_k%*%M
     }
     return(B_k)
}  
# sum of # of 1st degree connections of neighbors for each vertex
MM(1,A)%*%x
# sum of # of 2nd degree connections of neighbors for each vertex
MM(2,A)%*%x
# sum of # of 3rd degree connections of neighbors for each vertex
MM(3,A)%*%x
 
# if we normalize this with each iteration as k -> infinity 
# the resulting vector approaches the EV of A 
 
# k = 1
MM(1,A)%*%x/(norm(MM(1,A)%*%x,type="F"))
# k = 10
MM(10,A)%*%x/(norm(MM(10,A)%*%x,type="F"))
# k = 100
MM(100,A)%*%x/(norm(MM(100,A)%*%x,type="F"))
 
# this is essentially the power iteration algorithm for computing EV centrality

Created by Pretty R at inside-R.org

BISC Presentation: An Introduction to Social Network Analysis with Applications

RESEARCH SYMPOSIUM

“Strengthening Collaborations through Interactive Posters”

Friday, November 9, 2012

1:00 – 3:00 pm

Snell 2102 and 2113

Abstract

An introduction to Social Network Analysis tools with applications in viral marketing, social media analytics, epidemiology, homeland security, bioinformatics, student persistence, and technology diffusion.

Suggested Citation

Matt Bogard. "Social Network Analysis: An introduction with applications from literature" WKU Bioinformatics and Information Science Center.. Jan. 2012.
Available at: http://works.bepress.com/matt_bogard/23

Nonnegative Matrix Factorization and Recommendor Systems

Albert Au Yeung provides a very nice tutorial on non-negative matrix factorization and an implementation in python. This is based very loosely on his approach. Suppose we have the following matrix of users and ratings on movies:

If we use the information above to form a matrix R it can be decomposed into two matrices  W and H such that R~ WH'

where R is an n x p matrix of users and ratings

            W = n x r user feature matrix

            H = r x p  movie feature matrix

Similar to principle components analysis, the columns in W can be interpreted to represent latent user features while the columns in H’ can be interpreted as latent movie features. This factorization allows us to classify or cluster user types and movie types based on these latent factors.

For example, using the nmf function in R, we can decompose the matrix R above and obtain the following column vectors from H. 

We can see that the first column vector ‘loads’ heavily on ‘military’ movies while the second feature more heavily ‘loads’ onto the ‘western’ themed movies. These vectors form a ‘feature space’ for movie types.  Each movie can be visualized in this space as being a member of a cluster associated with its respective latent feature.

 

If a new user gives a high recommendation to a movie belonging to one of the clusters created by the matrix factorization, other movies belonging to the same cluster can be recommended.

References:

Matrix Factorization Techniques for Recommender Systems

Yehuda Koren, Yahoo Research Robert Bell and Chris Volinsky, AT&T Labs—Research

IEEE Computer Society 2009

Matrix Factorisation: A Simple Tutorial and Implementation in Python

Albert Au Yeung http://www.albertauyeung.com/mf.php

R Code:

#   ------------------------------------------------------------------
#  | PROGRAM NAME: R nmf example
#  | DATE: 10/20/12
#  | CREATED BY:  MATT BOGARD
#  | PROJECT FILE: /Users/wkuuser/Desktop/Briefcase/R Programs             
#  |----------------------------------------------------------------
#  | PURPOSE: very basic example of a recommendor system based on               
#  | non-negative matrix factorization
#  |
#  | 
#  |------------------------------------------------------------------
 
 
library(NMF)
 
# X ~ WH' 
# X is an n x p matrix
# W = n x r  user feature matrix
# H = r x p  movie feature matrix
 
# get ratings for 5 users on 4 movies
x1 <- c(5,4,1,1)
x2 <- c(4,5,1,1)
x3 <- c(1,1,5,5)
x4 <- c(1,1,4,5)
x5 <- c(1,1,5,4)
 
R <- as.matrix(rbind(x1,x2,x3,x4,x5)) # n = 5 rows p = 4 columns 
 
set.seed(12345)
 
res <- nmf(R, 4,"lee") # lee & seung method
 
V.hat <- fitted(res) 
print(V.hat) # estimated target matrix
 
w <- basis(res) #  W  user feature matrix matrix
dim(w) # n x r (n= 5  r = 4)
print(w) 
 
h <- coef(res) # H  movie feature matrix
dim(h) #  r x p (r = 4 p = 4)
print(h) 
 
# recommendor system via clustering based on vectors in H
movies <- data.frame(t(h))
features <- cbind(movies$X1,movies$X2)
plot(features)
title("Movie Feature Plot")

Created by Pretty R at inside-R.org

Get a Data Science Attitude

Do the following terms mean anything to you?

load balance toggle join index normalize key 

If you are a statistician and aspiring data scientist they should. If not this is one area where you should expand your knowledge base. In her article 'Being a data scientist is as much about IT as it is analysis' Carla Gentry explains why. 

"With knowledge of the client's IT setup from a data management/quality perspective, you'll be equipped to handle most situations you run into when dealing with data, even if the Architect and Programmer are out sick. Your professional knowledge is going to be a big help in getting the assignment or job complete."

http://hubdesignsmagazine.com/2012/10/16/being-a-data-scientist-by-carla-gentry/ 

If all you want is a cook to order data set from IT so you can run a regression, that's not the attitude or the skillset that employers have in mind when they seek out data scientists. There's plenty of jobs in academia for traditional statisticians, but that's not what Hal Varian was talking about when he said that the sexy job in the next 10 years will be statisticians. 

This reminds me of an article I read not long ago about building data science teams:

http://radar.oreilly.com/2011/09/building-data-science-teams.html

"Most of the data was available online, but due to its size, the data was in special formats and spread out over many different systems. To make that data useful for my research, I created a system that took over every computer in the department from 1 AM to 8 AM. During that time, it acquired, cleaned, and processed that data. Once done, my final dataset could easily fit in a single computer's RAM. And that's the whole point. The heavy lifting was required before I could start my research. Good data scientists understand, in a deep way, that the heavy lifting of cleanup and preparation isn't something that gets in the way of solving the problem: it is the problem."

BMC Proceedings: A comparison of random forests, boosting and support vector machines for genomic selection

A very cool combination of machine learning/quantitative genetics/bioinformatics

"Genomic selection (GS) involves estimating breeding values using molecular markers spanning the entire genome. Accurate prediction of genomic breeding values (GEBVs) presents a central challenge to contemporary plant and animal breeders. The existence of a wide array of marker-based approaches for predicting breeding values makes it essential to evaluate and compare their relative predictive performances to identify approaches able to accurately predict breeding values. We evaluated the predictive accuracy of random forests (RF), stochastic gradient boosting (boosting) and support vector machines (SVMs) for predicting genomic breeding values using dense SNP markers and explored the utility of RF for ranking the predictive importance of markers for pre-screening markers or discovering chromosomal locations of QTLs."


http://www.biomedcentral.com/1753-6561/5/S3/S11

Non-Negative Matrix Factorization

From Matrix Factorization Techniques for Recommendor Systems:

"Modern consumers are inundated with choices. Electronic retailers and content providers offer a huge selection of products, with unprecedented opportunities to meet a variety of special needs and tastes. Matching consumers with the most appropriate products is key to enhancing user satisfaction and loyalty. Therefore, more retailers have become interested in recommender systems, which analyze patterns of user interest in products to provide personalized recommendations that suit a user’s taste. Because good personalized recommendations can add another dimension to the user experience, e-commerce leaders like Amazon.com and Netflix have made recommender systems a salient part of their websites."

"matrix factorization models are superior to classic nearest-neighbor techniques for producing product recommendations, allowing the incorporation of additional information such as implicit feedback, temporal effects, and confidence levels"

See also: