h1. *Ideas Page for OpenCV Google Summer of Code 2016 (GSoC 2016)*

*OpenCV Augmented Reality from ArUco Markers last year. (click image for video showing all 2015 projects)*

" !AR_Aruco.png! ":https://www.youtube.com/watch?v=OUbUFn71S4s

----

* *Go back to:*

** "OpenCV Developer Page":http://code.opencv.org/projects/opencv/wiki

** "OpenCV User Page":http://opencv.org/

h2. General Information:

* "GSoC 2016 site":https://developers.google.com/open-source/gsoc/

* Mailing list for OpenCV GSOC 2016: [email protected]

** Group site https://groups.google.com/forum/#!forum/opencv-gsoc-2016

* IRC Channel:

<pre>

#opencv on freenode

</pre>

* *Timelines*

** "Timeline for GSoC 2016":https://developers.google.com/open-source/gsoc/timeline

** "Major GSoC 2016 Milestones":https://summerofcode.withgoogle.com/

** *Next important dates:*

*** -Feb 19 19:00 UTC: Mentoring organization application deadline.- *Submitted!*

*** -Feb 29 19:00 UTC: List of accepted Organizations.- *"We are in!!":https://summerofcode.withgoogle.com/organizations/?sp-search=OpenCV*

*** Mar 14 16:00 UTC: Student Application period begins!

*** Mar 25 19:00 UTC: Student Application period ends!

*** ? Deadline for OpenCV to request # of slots.

*** ? Google publishes how many slots we get.

*** ? Try to eliminate duplication of students.

*** ? IRC for OpenCV to resolve all duplications.

*** Apr 22 19:00 UTC: Accepted students announced.

*** May 23 Let Coding begin!

*** Jun 20 19:00 UTC: Mid term evaluation begins. *Pull requests must be in*

*** Jun 27 19:00 UTC: Mid term evaluation ends. *Pull requests must be in*

*** Aug 23 19:00 UTC: Coding stop; Pull requests must be in; Mentors: final evaluation begins...

*** Aug 29 19:00 UTC: Final evaluation deadline.

*** Late October: Mentor Summit!

*Times:*

*  UTC to PDT (California uses PST in the winter (from Nov 1st) and PDT in the summer (from March 8)).

** "utc time converter":http://www.timebie.com/std/gmt.php

*Resources:*

* "OpenCV Developers Site":http://code.opencv.org/projects/opencv/wiki

* "OpenCV User's Site":http://opencv.org

* [[How_to_contribute|How to do a pull request/How to Contribute Code]]

* Source Code can be found at "GitHub":https://github.com/Itseez/opencv. 

** To get the source code, point your git client to @git://github.com/Itseez/opencv.git@. Here are [[Working_with_OpenCV_git_repository|instructions]].

<pre>

git clone git://github.com/Itseez/opencv.git

</pre>

** To get user contributions, GSoC code, go to: https://github.com/Itseez/opencv_contrib.git

<pre>

git clone git://github.com/Itseez/opencv_contrib.git

</pre>

** The rest of the history plus matches between git commits and SVN revisions are stored at a separate "OpenCV Attic" repository: git://code.opencv.org/opencv_attic.git.

** Also OpenCV Extra was put to a separate repository: git://code.opencv.org/opencv_extra.git.

* [[OpenCV Development Process]]

** [[Meeting_notes|Developer meeting notes]]

h2. How you will be evaluated if you are an accepted student

* Student projects to be paid only if:

** *Midterm:*

*** You must generate a pull request

**** That builds

**** Has at least stubbed out functionality

**** With appropriate Doxygen documentation

**** Has at least stubbed out unit test

**** Has a stubbed out example of use that builds

** *End of summer:*

*** A full pull request

**** Full Doxygen documentation

**** A good unit test

**** Example of use code

*** Create a (short!) Movie (preferably on Youtube, but any movie) that demonstrates your code

**** We use this to create an overall summary. Past years:

***** "The 2015 Movie":https://www.youtube.com/watch?v=OUbUFn71S4s

***** "The 2014 Movie":https://www.youtube.com/watch?v=3f76HCHJJRA

***** "The 2013 Movie":https://www.youtube.com/watch?v=_TTtN4frMEA

h2. For students interested in applying

# You *must* already know how to program fluently in C++

** Some projects may require Python or Matlab skills

# Ask to join the "OpenCV GSoC Forum List":https://groups.google.com/forum/#!forum/opencv-gsoc-2016

** Discuss projects below or other ideas with us there (Feb 29 - March 13)

# On March 14th - 25th, Go to the "GSoC site":https://developers.google.com/open-source/gsoc/ and sign up to be a student with OpenCV

# Post the project from below or your own agreed on project on the GSoC to [email protected]

** Include Name, google email, age

** Include how you think you are qualified to accomplish this project (skills, courses, relevant background)

** Include Country of origin, school you are enrolled in, Professor you work with (if any)

** Include a projected timeline and milestones for the project

# Once (and if!) OpenCV gets accepted as GSoC 2016 org (Feb 29th), and we are told how many slots we will get *and* you've signed up for a project with us (March 14-25th)

** We will weight the students and projects against mentors and their interests and choose which students/project to pursue.

** Accepted students will be posted on the GSoC site on April 22nd (and we will notify the accepted students ourselves).

h2. For computer vision professionals interested in mentoring

# Go to the "GSoC site":https://developers.google.com/open-source/gsoc/ and sign up.

# Request to be a mentor for OpenCV there.

# We accept or reject you

# Students submit projects and you 

** find a project you want to mentor (several students might try for the same project)

** contact us about it through [email protected] so we can be sure there is not someone already handling that project

** contact the students to assess which one if any are capable of coding that up in a summer.

# We get a slot allocation from Google, the administrators then _"spend"_ the slots in order of priority

# Students must finally actually accept to do that project (some sign up for multiple organizations and then choose)

*If* you are accepted as a mentor *and* you find a suitable student *and* we give you a slot *and* the student signs up for it, +*then*+ you are a mentor. 

** It sounds harder than it is.

<pre>

\/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/  \/

</pre>

h1. 2016 Project Ideas:

*Students may propose their own projects (give us a clear summary and why you can do this project). However, below are some of our priorities for this year* Contact us and/or discuss ideas at https://groups.google.com/forum/?fromgroups#!forum/opencv-gsoc-2016

_These are not in order of priority_

*1. Google Cardboard*

* Do visual odometry, SLAM etc for Google Cardboard

*2. Deep learning*

* It would be cool if OpenCV could load and run deep networks trained with popular DNN packages like Caffe, Theano or Torch.

* GPU/OpenCL optimization and/or other optimizations (that can partly be contributed back to the upstream projects) are also encouraged.

*3. Projection mapping (Projected Augmented Reality)*

* Calibrate a camera to a projector

* Put out structured light to learn the 3D map of what's in front of the camera

* Project the camera image, suitably rotated and shifted onto the scene

* Unity or some other program to start transforming the scene for projection mapping

** See [[http://www.creativebloq.com/video/projection-mapping-912849]] to get ideas of what you can do with this

*4. Update OpenCV file storage so that it's readable by Python in OpenCV (move to YAML 1.2 Edition 3)*

* Mentor Prasanna

* OpenCV C++ writes out its own YAML tags which makes it incompatible with it's own Python and with external python readers

* Speedup fs reading/writing, add JSON support, add base64 (+RLE) support for large amounts of numerical data

* References

** YAML [[http://www.yaml.org]]

** Problem statement: [[http://stackoverflow.com/questions/11141336/filestorage-for-opencv-python-api]]

*5. openh264 support*

* We are currently use ffmpeg, which is great library, but: 1) its API changes often and 2) it does not have its own h264 encoder and instead uses GPL-ed x264.

* There is BSD-licensed openh264 library that includes H264 encoder and decoder. It may be interesting to use this library directly (although ffmpeg can also use it) as light-weight method to get full support for H264.

*6. Better interactive camera calibration*

* Modern algorithms would allow us to recompute camera parameters after almost each frame, compute reprojection error, compute confidence intervals for the parameters.

* the partially occluded calibration patterns could be handled somehow to, especially if we want to estimate intrinsic parameters only.

* the application may show us the areas where the corner borders have been shown so that user can see how good are the data

*7. Color Calibration and white balance*

* With McBeth chart and with gray world algorithm (improve exisiting color constancy algorithms)

* Possibly collect deep learning way of doing this (needs large training set)

*8. Tracking improvements*

* TLD algorithm is implemented in OpenCV, but it needs improvements and optimizations.

* ASM/AAM can also be implemented, as well as the particle filter.

*9. Improving ICF/ACF + <some>Boost training and detection*

* In opencv_contrib there is initial version of the training application and detector based on integral channel features and boosting. Further improvements of this part would be really useful.

*10. lua+OpenCV*

* the other language that is now used more often for ML and CV (thanks to Torch) is Lua; there are old bindings for a tiny part of OpenCV; could be useful to extend our autogenerated bindings to Lua.

*11. human interaction*

* gesture recognition

* human skeleton based on 2d or 3d

*12. Quad Rotor Contrib*

* Apply much of the above into a quad rotor drone automatic tracking, mapping navigation etc module

*13. Efficient dense optical flow algorithm*

* OpenCV will benefit greatly from having some real alternative for the Farneback's dense optical flow algorithm. A few other algorithms that we've implemented (e.g. SimpleFlow, TVL1) are very, very slow. It would be nice to have an algorithm that is at least as fast as Farneback and gives a better quality at the same time.

*14. Speeding up various OpenCV parts (it can be multiple projects).*

  OpenCV is great library, but some of its algorithms may be made more efficient. Here is a list of things that can be optimized:

   * text detection module

   * shape module

   * bioinspired module

   * hdr algorithms from ximgproc and xphoto.

   * non-local-means denoising algorithm from photo module

   * kaze & akaze features

   * you name it - proposals are welcome

New 2016 ideas:

* Architecture

** Exceptions in OpenCV need to be easily disabled ? (need to check if that's the case or not)

** Minimal OpenCV: give the functions you want, it gives you a minimal version of the library.

** reduce the size of OpenCV on Android

** create a more C++ type (to be able to deal with constness and separate the pointer from the type)

** create an image type (that would also have color information, maybe calibration too) and implement a proper resize algorithm for that type

* Tracking

** David Tan’s tracking

2d: [[https://www.youtube.com/watch?v=W689ffCMDF4]]

3d: [[https://www.youtube.com/watch?v=TqyFEZzhPbE]]

* misc

** convexHull is implemented with Sklansky's second, 1982 algorithm, which is incorrect in some case, and suboptimal complexity-wise. cf. [[http://cgm.cs.mcgill.ca/~athens/cs601/]].

<code>cv::contourArea(cv::convexHull(contour))</code> for a convex contour is different from <code>cv::contourArea(contour)</code>, somtimes by nearly 30%.

** Lightfield: [[https://www.youtube.com/watch?v=WU-D4PA5plo]]

** a resize algorithm for images taking color into account. [[http://www.4p8.com/eric.brasseur/gamma.html]]?

<pre>

/\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\ /\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\  /\ 

</pre>

h1. Accepted Projects for OpenCV GSoC 2016:

OpenCV GSoC 2016 students, projects, mentors are below. *WHEN THEY ARE KNOWN*

*GSoC Progress. Need Pull requests (don't have to be complete) to pass by July 3rd*

*Mailing Lists (only for use of Mentors and Students, please to not request to join these lists otherwise):*

* Mentors

** opencv-gsoc2016-mentors

* Students

** opencv-gsoc2016-students

* [[How_to_contribute|How to do a pull request]]

*Pre-Midterm status:*

Project notes will be compiled "here":https://sites.google.com/a/opencv.org/opencv/gsoc-2016-project-notes .

h2. Mentors

*Actual signed up mentors*

* Alexander Mordvintsev	zzznah(+)gmail.com

* Sid Bao	ybao(+)magicleap.com

* Vincent Rabaud	vincent.rabaud(+)gmail.com

* Vadim Pisarevsky	vadim.pisarevsky(+)gmail.com

* Adrian Kaehler	therealadrian(+)gmail.com

* Terry Boult	tboult(+)vast.uccs.edu

* spmallick	spmallick(+)taaz.com

* Serge Belongie	sjb344(+)cornell.edu

* Stefano	s.fabri10(+)gmail.com

* Prasanna	pras.bits(+)gmail.com

* Pablo Alcantarilla	pablofdezalc(+)gmail.com

* Bence Magyar	mw.mzperx(+)gmail.com

* Manuele	manuele.tamburrano(+)gmail.com

* Grace Vesom	grace.vesom(+)gmail.com

* Open Source Computer Vision Library (OpenCV)	garybradski(+)gmail.com

* Douglas Lee	dougabug(+)gmail.com

* Claudia Rapuano	c.rapuano(+)gmail.com

* Antonella Cascitelli	antonellacascitelli(+)gmail.com

* Anatoly Baksheev	anatoly.baksheev(+)itseez.com

* Alexander	alexander.shishkov(+)itseez.com

* Alexander Smorkalov	alexander.smorkalov(+)itseez.com

* Alexander Bovyrin	alexander.bovyrin(+)itseez.com

In the below, get rid of the ''-delete-'' to make the emails work.

 Anatoly Baksheev

 Researcher, Vision Algorithms on GPU

 Argus/Itseez

 [email protected]  

  Alexander Bovyrin

  PhD, Senior Researcher

  Argus/Itseez founder

  NNU Lecturer

  [email protected]

 Gary Bradski

 Founder, Industrial Perception Inc.

 Consulting Prof. Stanford U.

 OpenCV Founder, Technical Content Owner, GSoC Admin

 Co-author of Learning OpenCV Book

 http://www.willowgarage.com/pages/people/gary-bradski-senior-researcher

 [email protected]

 Antonella Cascitelli

 Grad student, University of Rome

 [email protected]

 Eric Christiansen

 PhD, UCSD

 [email protected]

 Stefano Fabri

 CRR Team leader, University of Rome

 [email protected]

 Victor Eruhimov

 OpenCV founding team/Senior Researcher

 Argus/Itseez founder

 NNU Lecturer

 [email protected]

 Adrian Kaehler

 Principle Engineer, Applied Minds

 Co-author of Learning OpenCV Book.

 [email protected]

 Peter Karasev

 PhD Student, MINERVA Research Group

 Georgia Tech

 [email protected]

 Vadim Pisarevsky

 OpenCV founding team/Czar

 [email protected]

 Manuele Tamburrano

 Grad Student, University of Rome

 [email protected]

 Vincent Rabaud

 Principle Engineer, Aldebaran Robotics

 [email protected]

  Claudia Rapuano

  Grad student, University of Rome

  [email protected]

 Grace Vesom

 Senior Software Engineer, Magic Leap

 -delete-grace.vesom@-delete_gmail.com

 Bence Magyar

 Pal Robotics

 [email protected]

 Pablo Alcantarilla

 Toshiba Research Europe Ltd.

 [email protected]

h3.  Back up Mentors

 Mark Asbach

 Fraunhofer IAIS

 Schloss Birlinghoven

 Sankt Augustin, Germany

 http://mmprec.iais.fraunhofer.de/asbach.html

 [email protected]

 Nicolas Saunier, Ph.D.

 Assistant Professor

 Civil, Geological and Mining Department (CGM)

 École Polytechnique de Montréal

 http://nicolas.saunier.confins.net

 [email protected]

 Alexander Mordvintsev

 Software Engineer

 http://znah.net

 [email protected]

 Andrey Morozov

 Software Engineer

 Argus/Itseez

 [email protected] 

h2. OpenCV Org. Application to GSoC 2016

[[OpenCV GSoC Application]]

h2. Previous Years 

* 2015 "GSoC 2015 Ideas Page":http://code.opencv.org/projects/opencv/wiki/GSoC_2015

** "The 2015 Movie":https://www.youtube.com/watch?v=OUbUFn71S4s

* 2014 "GSoC 2014 Ideas Page":http://code.opencv.org/projects/opencv/wiki/GSoC_2014

** "The 2014 Movie":https://www.youtube.com/watch?v=3f76HCHJJRA

* 2013 "GSoC 2013 Ideas Page":http://code.opencv.org/projects/gsoc2013/wiki

** "The 2013 Movie":https://www.youtube.com/watch?v=_TTtN4frMEA

* 2012 Ideas site http://code.opencv.org/projects/gsoc2012/wiki/Wiki/

* 2011 Ideas site http://opencv.willowgarage.com/wiki/GSOC_OpenCV2011

** 2011 Application GSoC page is http://opencv.willowgarage.com/wiki/GSoC2011OpenCVApplication

* 2010 Ideas site http://opencv.willowgarage.com/wiki/GSOC_OpenCV2010

** 2010 Application page http://opencv.willowgarage.com/wiki/ApplicationGSOC2010

----