In this past June's issue of R journal, the 'neuralnet' package was introduced. I had recently been familiar with utilizing neural networks via the 'nnet' package (see my post on Data Mining in A Nutshell) but I find the neuralnet package more useful because it will allow you to actually plot the network nodes and connections. (it may be possible to do this with nnet, but I'm not aware of how).The neuralnet package was written primarily for multilayer perceptron architectures, which may be a limitation if you are interested in other architectures.

The data set used was a default data set found in the package 'datasets' and consisted of 248 observations and 8 variables:

"education" "age" "parity" "induced" "case" "spontaneous" "stratum" "pooled.stratum"

The following code runs the network (with 2 hidden layers) classifying 'case' (a binary variable) as a function of several independent varaibles. The neural network is estimated, and the results are stored in the data frame 'nn.'

nn <- neuralnet(

case~age+parity+induced+spontaneous,

data=infert, hidden=2, err.fct="ce",

linear.output=FALSE)

The weight estimates can be obtained with the following command:

nn$result.matrix

And, the network can be plotted or visualized with the simple command:

plot(nn)

As can be seen below, the output weights correspond directly with the visualized network. While this tool may be more difficult to interpret with more complex network architectures, I find this simplified version very useful for demonstration/instructional purposes.

Example: The weight for the path from input 'age' to the first hidden layer is -3.0689 (age.to.1layhid1) which can easily be found in the network diagram. After all inputs feed into hidden layer 1, the weight associated with the path from hidden layer 1(1layhid.1.to.case) to the output layer (which along with information from the other layers of the network will give us the classification of 'case') is -1001.15.

**Estimated Network Weights**

error | 123.811 |

reached.threshold | 0.009853 |

steps | 16822 |

Intercept.to.1layhid1 | 0.535237 |

age.to.1layhid1 | -3.0689 |

parity.to.1layhid1 | 2.262792 |

induced.to.1layhid1 | 30.50824 |

spontaneous.to.1layhid1 | 0.000961 |

Intercept.to.1layhid2 | -5.52491 |

age.to.1layhid2 | 0.11538 |

parity.to.1layhid2 | -1.98372 |

induced.to.1layhid2 | 2.550932 |

spontaneous.to.1layhid2 | 3.829613 |

Intercept.to.case | -1.7714 |

1layhid.1.to.case | -1001.15 |

1layhid.2.to.case | 4.570324 |

**Neural Network Visualization (**click to enlarge)

There are many other options avaiable with the neural net package, I encourage you to read the article referenced below for more details.

**Reference:**neuralnet: Training of Neural Networks by Frauke G¸nther and Stefan Fritsch The R Journal Vol. 2/1, June 2010

R-Code:

# ------------------------------------------------------------------ # |PROGRAM NAME: NEURALNET_PKG_R # |DATE: 12/3/10 # |CREATED BY: MATT BOGARD # |PROJECT FILE: P:\R Code References\Data Mining_R # |---------------------------------------------------------------- # | PURPOSE: DEMO OF THE 'neuralnet' PACKAGE AND OUTPUT INTERPRETATION # | # | ADAPTED FROM: neuralnet: Training of Neural Networks # | by Frauke Günther and Stefan Fritsch The R Journal Vol. 2/1, June 2010 # | ISSN 2073-4859 (LOCATED: P:\TOOLS AND REFERENCES (Copy)\R References\Neural Networks # | # | # |------------------------------------------------------------------ # |DATA USED: 'infert' FROM THE 'datasets' LIBRARY # |------------------------------------------------------------------ # |CONTENTS: # | # | PART 1: get the data # | PART 2: train the network # | PART 3: # | PART 4: # | PART 5: # |------------------------------------------------------------------ # |COMMENTS: # | # |----------------------------------------------------------------- # |UPDATES: # | # | # |------------------------------------------------------------------ # *------------------------------------------------------------------* # | get the data # *------------------------------------------------------------------* library(datasets) names(infert) # *------------------------------------------------------------------* # | train the network # *------------------------------------------------------------------* library(neuralnet) nn <- neuralnet( case~age+parity+induced+spontaneous, data=infert, hidden=2, err.fct="ce", linear.output=FALSE) # *------------------------------------------------------------------* # | output training results # *------------------------------------------------------------------* # basic nn # reults options names(nn) # result matrix nn$result.matrix # The given data is saved in nn$covariate and # nn$response as well as in nn$data for the whole data # set inclusive non-used variables. The output of the # neural network, i.e. the fitted values o(x), is provided # by nn$net.result: out <- cbind(nn$covariate,nn$net.result[[1]]) dimnames(out) <- list(NULL, c("age", "parity","induced","spontaneous","nn-output")) head(out) # generalized weights # The generalized weight expresses the effect of each # ovariate xi and thus has an analogous interpretation # as the ith regression parameter in regression models. # However, the generalized weight depends on all # other covariates. Its distribution indicates whether # the effect of the covariate is linear since a small variance # suggests a linear effect # The columns refer to the four covariates age (j = # 1), parity (j = 2), induced (j = 3), and spontaneous (j=4) head(nn$generalized.weights[[1]]) # visualization plot(nn)

I am a newbie. Would you mind writing all commands needed to run the example you gave? How to load the package and stuff? Many thanks

ReplyDeleteI will try to find the program and post the code and add some instructions. In the mean time I would refer you to http://journal.r-project.org/archive/2010-1/RJournal_2010-1_Guenther+Fritsch.pdf

ReplyDeletewhich is the link to the article I cited. It provides much more detail and instructions for loading the package etc.

I've now included the code I used, but I have to say, each time I run it I get different results. I'm not sure if there is an option for specifying a seed or some way to get consistent results. there is also a lot more I need to know about interpreting the output beyond what I've already posted. Probably time to give the journal article and documentation a second read.

ReplyDeleteNote- to read the code you may have to use the pretty R scroll bar, or you should be able to copy and paste into a text doc or editor.

ReplyDeleteThanks Matt, this is very useful. Do you know how we can specify the maximum number of epochs or the target error rate?

ReplyDeleteHello Gayathri..

DeleteI am starting to work with neuralnet.

Did you discovered if it is possible specify the maximum number of epochs or the target error rate??

Thank you..

@Gyathri- I'm not sure about this. I first learned about Neural nets via the Rattle Package in R, and the journal article I cited above. I would like to know more if I get time to dig into it. If you use twitter, you might send out a tweet with #rstats hash tag and ask about this. I might if I get a chance in the future. You could try the R help listserv, but I've never used it as I've heard so much about it not being user friendly or welcoming to anyone that is not an expert in R.

ReplyDeleteI write:

ReplyDelete>library(datasets)

>names(infert)

>nn <- nnet(case~age+parity+induced+spontaneous, data=infert, hidden=2, err.fct="ce", linear.output=FALSE)

But error occurs:

Erro em c(dim(x)[2L], size, dim(y)[2L]) : 'size' is missing

How to solve? Thanks

Use Size option, you will get the weights

Delete> nn<-nnet(case~age+parity+induced+spontaneous,data=infert, size=10, hidden=2,linear.output=FALSE)

# weights: 61

initial value 88.005015

final value 83.000000

converged

I have a question on how to predict future ts-values after a svm-regression:

ReplyDeleteIf I execute:

library(e1071)

library(dyn)

y=c(0,2,1,3,2,4,3,5,4,6,5,7,6,8,7,9,8,10,9,11)

y<-ts(y)

y.svm <- dyn$svm(y ~ lag(y))

yp <- predict(y.svm)

ts.plot(y, yp, col = 1:2)

I only get values for yp[1] .. yp[19]

What do I have to do to get future values yp[20] .. yp[26] for example?

@ Anonymous & Anonymous- I'll have to look at these issues- I'm not sure I'll get the answer. I'm still just learning these techniques and the code too! But my purpose of this blog is to facilitate more learning. I can barely keep up with new posts, ( I have a lot more I want to write) not to mention addressing these issues.

ReplyDeleteIn the mean time, typically what I do when I have an issue like this, is I'll make a blog post and ask the question via twitter (with a link to the post like this one) using the #rstats hash tag. Stack overlfow is another web site that is useful to post questions about R-code, as well as the R-user group. If you have a linkedin profile, I notice that R-users are often using the linkedin R user interest group for these kind of discussions.

If feel bad for not providing you with a direct answer, but hopefully one of these resources will be helpful.

Anonymous is not using the correct package ('neuralnet', rather than 'nnet' - I am guessing nnet doesn't have the dataset... After loading nnet I got the same error, but the post's neuralnet example ran fine.

ReplyDeleteThanks!

ReplyDeleteHow to test model using neural network in R..? Please help me..

ReplyDeleteYou're right that people, notably Brian Ripley, can get snarky about poorly written questions, but I found a lot of people willing to help me when I was first learning R and most beginner questions already have answers. Try a google search like "R how to install package" etc. Also take a look at the posting guide before posting:

ReplyDeletehttp://www.r-project.org/posting-guide.html

i am getting below error, when i run above code

ReplyDelete> nn <- neuralnet(case~age+parity+induced+spontaneous, data=infert, hidden=2, err.fct="ce", linear.output=FALSE)

Warning message:

algorithm did not converge in 1 of 1 repetition(s) within the stepmax

Dear matt,

ReplyDeleteI am trying to use Nnet for predicting continous data, wil it work? my dependent is a gamma distributed varible, so i used ur code but with little tweek for linout=true. but i am not getting results. please see the error below and advice me what other changes can be made

> nn <- nnet(Low.formula~Type.of.property+Type.of.occupation+Job+Number.of.rooms, data=France1, size=1, skip=FALSE, maxit=1000,hidden=1, err.fct="ce", linout=TRUE)

# weights: 12

initial value 19575694.371960

iter 10 value 1513776.253055

iter 20 value 1205329.256042

iter 30 value 1046068.800635

iter 40 value 969627.109067

iter 50 value 739564.881771

iter 60 value 727653.907905

iter 70 value 661323.506699

final value 661205.493839

converged

> nn$result.matrix

NULL

> plot(nn)

Error in xy.coords(x, y, xlabel, ylabel, log) :

'x' is a list, but does not have components 'x' and 'y'

> fix(nn)

Warning message:

In edit.default(get(subx, envir = parent), title = subx, ...) :

deparse may be incomplete

> weights(nn)

NULL

thank you

I have the same problems:

ReplyDelete> nn$result.matrix

NULL

or

algorithm did not converge in 1 of 1 repetition(s) within the stepmax

-------------------------------------

Does this mean that algorithm didn't find any relationships between dependent variable and covariates?

Thanks!

Can we use Neural net R packages for predicting Continuous data like prices,

ReplyDeleteIf yes what changes we need to make in the code u have given above.

From the perspective of R, how do we use the result of the neural network exercise? Do we come up with an equation? Does it work like a regression equation where, after the procedure, we can just plug in numbers to get a predicted value?

ReplyDeleteI am testing out your codes on a sample data that I have. Can you please share me some method of obtaining forecasts using the network thus obtained.

ReplyDeleteAnil, I really want to work on this when I get time. I've done it with SAS Enterprise Miner but have not demonstrated it in R. I will as soon as I can! Thanks.

ReplyDeleteDo you know how to interpret the error in the output? Thanks.

ReplyDeleteAre we sure that this package works properly? I am trying to figure it out with the simplest of data, and even though it gives the very known error: Warning message:

ReplyDeletealgorithm did not converge in 1 of 1 repetition(s) within the stepmax

or a huge Error in the network prediction that makes outputs have no sense. It doesn't even working with the simple data table when you have a first column a number and the second column is numbers' 2X and as output you want the sum of these two columns...