Multiple identical CvAnn_MLPs yield different training results (Feature #1411)


Added by moeter84 - over 5 years ago. Updated over 1 year ago.


Status:Open Start date:
Priority:Low Due date:
Assignee:Maria Dimashova % Done:

0%

Category:ml
Target version:Next Hackathon
Difficulty: Pull request:

Description

Hi,
when training two seperate ANN's with identical training sets and parameters, they yield completely different results. Here is what I do:

//layout is a random working layout...
m_network->create(layout,CvANN_MLP::SIGMOID_SYM,1.1,1.1);
testwork->create(layout,CvANN_MLP::SIGMOID_SYM,1.1,1.1);

//snip... fill training_hists,training_outputs,weights

//setup tr_pars...
CvANN_MLP_TrainParams tr_pars;
tr_pars.train_method=CvANN_MLP_TrainParams::RPROP;
tr_pars.term_crit.epsilon=0.001;
tr_pars.term_crit.max_iter=1000;
tr_pars.term_crit.type=CV_TERMCRIT_ITER+CV_TERMCRIT_EPS;

//run training...
qDebug()<<"train iterations:"<<
m_network->train(training_hists,training_outputs,weights,cv::Mat(),tr_pars);
qDebug()<<"test train iterations:"<<
testwork->train(training_hists,training_outputs,weights,cv::Mat(),tr_pars);

//run a simplistic test
m_network->predict(training_hists.row(0),test);
qDebug()<<test.at<float>(0)<<test.at<float>(1)<<test.at<float>(2);
testwork->predict(training_hists.row(0),test);
qDebug()<<test.at<float>(0)<<test.at<float>(1)<<test.at<float>(2);

Now, here is what I get:

train iterations: 83
test train iterations: 52

1.12094 -1.1355 -1.15789
0.997264 -1.02149 -1.15789

These results are reproducable and at each new run, they are the same. If I trigger a new training, again with the same data set and parameters, I get again a different result, even though I use new CvANN_MLP objects and destroy the old ones.

Interestingly, all later runs of train() have a lower iteration count than the first one.


History

Updated by moeter84 - over 5 years ago

After further investigation, I guess this is okayish behaviour, as ANN weights are initialized with a Nguyen-Widrow window. Still some problems remain:

-The RNG is NOT random at all by default. It's the same for every run!
-Initialization of weights should be made user accessible.
-Why are all later runs terminating quicker than the first one?

I don't mind this "bug" any more, as I switched to FANN for Neural Networks.

Updated by Alexander Smorkalov about 3 years ago

  • Target version set to Next Hackathon

Updated by Maksim Shabunin over 1 year ago

Issue has been transferred to GitHub: https://github.com/Itseez/opencv/issues/4305

Also available in: Atom PDF