History

GSoC OpenCV Application¶

Org ID?¶

OpenCV

Org name?¶

Open Source Computer Vision Library (OpenCV)

Description?¶

The Open Source Computer Vision Library (OpenCV) is a comprehensive computer vision library and machine learning (over 2500 functions) written in C++ and C with additional Python and Java interfaces. It officially supports Linux, Mac OS, Windows, Android and iOS. OpenCV has specific optimizations for SSE instructions, CUDA and especially Tegra.

OpenCV is now supported by a non-profit organization: OpenCV.org. It has over 10M downloads. OpenCV has uses from gesture recognition, Android and iPhone vision apps on up to medical, robotics, mine safety and Google Streetview.

Tags?¶

OpenCV, Computer Vision, Machine Learning, Image Processing, Computational Photography, Fun

URL?¶

http://opencv.org/

License?¶

New and Simplified BSD

Backup Admin?¶

Vadim.Pisarevsky

Ideas Page?¶

http://code.opencv.org/projects/gsoc2013/wiki

IRC channel?¶

#opencv on freenode

Vetran: Past summer of code, If so, challenges?¶

veteran history

OpenCV has been involved in GSoC every year since 2010. The past idea pages are listed at the bottom of the current ideas list:
http://code.opencv.org/projects/opencv/wiki/GSoC_2015

We do surprisingly well to meeting our top objectives each year. Contributions are listed in our change log:
http://code.opencv.org/projects/opencv/wiki/ChangeLog Note that some major milestones such as Android and iOS port derived from GSoC.

I co-founded a company with one of my past GSoC students (Ethan Rublee) and sold that company to Google. He's now a part of Google! Project Replicant (robotics).

Google has tended to grant us many slots, 15 last year, all passed, see below.

Our biggest challenge is students who look very good on paper but not in practice. They misrepresent their coding or project skills in particular. Those are most at risk of dropping or failing out. This year we are going to have some Google hangout coding interviews. Simple ones since even simple interviews can filter much of this out.

VETERAN HISTORY:
OpenCV (C++) has been involved in GSoC for 5 years now. It has been an amazing help to us. It generated our ports to Android, IOS and initialized our Python and Java interfaces. It has vastly improved our sample code base and added some very key algorithms such as image denoising, image stitching and camera stabilization. A very popular addition (I use it every time I code) is a QT backend for viewing images. This came from GSOC 2010, Yannick Verdie.

GSOC 14 HIGHLIGHTS:
This was our most productive year ever. For a student to pass, they must complete an accepted Pull request to our repository. The code is now available in OpenCV. From the Change Log:

Our new release OpenCV 3.0 and its contribution repository feature a lot of new functionality created by our GSoC students last year, namely:

Text detection and recognition by Lluis Gomez and Stefano Fabri
HDR by Fedor Morozov
Smart segmentation and edge-aware filters by Vitaly Lyudvichenko, Yuri Gitman, Alexander Shishkov and Alexander Mordvintsev
Car detection using Waldboost, ACF by Vlad Shakhuro and Nikita Manovich
TLD tracker and several common-use optimization algorithms by Alex Leontiev
RGBD module by Vincent Rabaud and associated students
Line Segment Detector by Daniel Angelov along with custom calibration
Many useful Computational Photography algorithms by Siddharth Kherada
Long-term tracking + saliency-based improvements (tracking module) by Antonella Cascitelli and Francesco Puja
Another good pose estimation algorithm and the tutorial on pose estimation by Edgar Riba and Alexander Shishkov
Line descriptors and matchers by Biagio Montesano and Manuele Tamburrano

2014 PASS/FAIL RATES
We had 15 students this year, all 15 passed (100% complete, a first!)

GSOC 13 HIGHLIGHTS:

Abidrahmank Did a great job on building out the OpenCV-Python interface. This is rock solid now.
Oli Wilkie did great Android development example code
Hilton Bristow did a great foundation and start to a fully compliant Matlab interface for OpenCV
Di YangPb segmentation works and fairly fast
Daniel Angelov Line Segment Detection. Maybe the most useful contribution this year.
Juan Perez Rua, Shape context descriptor
Ozan Tonkal 3D visualizer. We use this a lot
Luise Bigorda Text detection in scenes. Great.
Antonella Cascitelli, long term tracking optical flow API.

2013 PASS/FAIL RATES
We had 14 people this year (plus one more we paid ourselves) and failed 3.

GSOC 12 HIGHLIGHTS:
An offical iOS port was added to OpenCV. Eduard Feicho and Charu Hans improved the port and created detailed tutorials on how to add OpenCV to an iOS app
http://docs.opencv.org/doc/tutorials/ios/table_of_content_ios/table_of_content_ios.html
They made a cool tutorial for it:
gsoc2012:source:/ios/trunk/doc/CVPR2012_OpenCV4IOS_Tutorial.pdf

Python was a big success for '12. The interface was improved and the student, Alexander Mordvintsev, wrote several key code examples
https://github.com/Itseez/opencv/tree/master/samples/python2

OpenCV has a lot of machine learning algorithms. Many people want to use these "out of the box", so in the course of working on new feature types for our cascade classifier, Attila Novak, pre-trained it on two new classes. The algorithm is extensively used for frontal face, he added profile faces. And, silverware detectors (for a robot project):
https://github.com/Itseez/opencv/tree/master/data/lbpcascades

People have been asking for better computational photography support. Victor Passichenko implemented a non-local means denoising algorithm
http://docs.opencv.org/trunk/modules/photo/doc/denoising.html

For movie effects and VR, we've been wanting a dense optical flow algorithm for a long time. Yuri Zemlyansky implemented "simple flow" (it's not so simple, but it is dense -- assigns motion vectors to every point in a reasonable way).
https://github.com/Itseez/opencv/tree/master/samples/cpp/simpleflow_demo.cpp

OpenCV has been dying for SLAM and SfM (basically 3D reconstruction code) for years. It's a hard project to get right, it's not in yet, but Srimalj made good progress and we'll probably combine this with Google's own libmv this year.

Finally, no one student, but we kicked off a tutorial section 2 years ago and many students have contributed since. We'd now like each student to end their work with a tutorial example of how to use what they coded ... which ensures that it actually does get used. Tutorials have grown quite a bit and past students have continued to add tutorials:
http://docs.opencv.org/doc/tutorials/tutorials.html

2012 PASS/FAIL RATES
We were allocated 12 students, 10 passed.

The failures were in: Active Appearance Models, and Fast Linear Program Solver. Our library is computer vision and machine learning. Many things border on active research, many algorithms have complex mathematical underpinnings. Some students look capable of performing these tasks "on paper" and with code samples but simply aren't.

Mentors try to help but sometimes the student just stops responding. Sometimes we suspect they have multiple jobs and may just be overwhelmed or have other priorities. We often have the best results from people who are pushing code at us well before the start and/or whose thesis area covers what we want implemented. We'll try to vet more closely based on these criteria, but we always seem to get about this rate of fall off.

Why participate?¶

OpenCV has several full time contractors working on robotic applications and another paid group working on CUDA. However, the scope of the library is much larger and GSoC interns were invaluable last year in addressing other critical areas:

getting a solid port of OpenCV to Android,
getting a solid port of OpenCV to iOS,
developing a QT based graphical interface, the old GUI was decidedly limited and stale
creating an automatic interface to PASCAL VOC http://pascallin.ecs.soton.ac.uk/challenges/VOC/ (the main computer vision object recognition challenge)
coding up a long time winning classifier in PASCAL VOC: Latent SVM
features and support for feature tracking
A set of tutorials
A set of python examples

This year, among many other ideas, we'd like to use GSoC to support vision applications for

Google Cardboard! Pose tracking and visual odometry
Projector based augmented reality. This can be done for whole buildings, see here
- https://www.youtube.com/watch?v=XSR0Xady02o
Deep learning. We are already installing seemless interface to Caffe and have plans (GSoC willing) to connect to Torch/LUA

All the above things were out of scope of the paid work and had substantial impact on OpenCV, especially making it work with Mobile. In addition, 3 of the interns are now regular contributors. 2 more are occasional contributors.

Thus, why we are applying: We want to bring in a new generation of contributors and also to fill out areas where to don't have active development. We hope to gain pretty much a repeat of last year: Filling out new areas, perhaps co-authoring new papers and getting new regular contributors.

Mentor selection criterion?¶

1) All mentors but one are known developers of OpenCV. We also allow Professors to mentor since they have great experience in student-mentor interactions.

(2) All mentors have extensive computer vision AND coding background.

(3) All mentors are people who have had interns or students to manage, either in companies, academic settings or research groups. If we have a particularly high performing and social student, we would let them mentor also because they have recently seen things from the other side.

Disappearing students?¶

We will try to prevent this by having mentors keep regular contact via skype meetings 1/week, twitter and email, but if it happens:
We will post the orphaned project to our user group and offer a some of the mentor compensation (~$200) to someone who completes the project.
Something we've now learned, that we will "interview" potential students with a google hangout simple coding test. Most of our problems have been from students who represented to us that they could program but could not/had never done a project at all. We will now try to weed that out.

Disappearing Mentors?¶

(1) We will have back up mentors on hand. We have a pool of mentors already who are busy and so don't want to promise to be mentors but who are willing to serve as a backup.
-- Last year we had several such mentors who were not needed.

(2) The admins are the back up of last resort. Both of us have extensive mentoring experience, both have already mentored up to 3 GSoC students at a time.

Steps for interaction?¶

These techniques were used in 2010 - 2014 and worked well, so we'll use them again:

Before the project, we will encourage them to post a detailed project plan to the user group (also socially committing them to complete).
We will set up a 2x daily twitter feed for all the summer's development projects.

We ask they put project results up on a youtube channel that will be posted to the large user group. This provides positive reinforcement and feedback.

We hope to set up projects with 3 goals:

(1) Quick and turn around (within a few weeks they should have a baseline accomplishment), this will be followed by

(2) the main, reasonable summer goal, but there will also be

(3) a more comprehensive "stretch goal" that we will encourage them to complete, getting feedback and feature requests from the community.

For the more advanced projects, we try to write a paper with the students (in the past we have an ICCV, ACCV and CVPR submissions)
-- The best students may become interns at mentor's work places (which has happened 5 times now).

Encourage students to keep contributing/stick to their projects.¶

Right after the project, the student can, almost always, still add documentation, tutorials, test code.

As above, for advanced projects, we like to see if we can publish joint papers together, a win-win for students and for us and a strong incentive for students to stay involved. Our ICCV paper "ORB: An efficient alternative to SIFT or SURF", is one such example -- highest cited paper from that whole conference.

In 2013, we also hired one student contributor for an independent internship by OpenCV itself. Since some money is coming in, we hope to do more of this. Many get hired into the places the mentors work.

These are often smaller side projects that can keep the student involved.

For the medium to longer term:
OpenCV will be fully refactored into small, independent modules by summer coding season. This means it will be easier to add new modules.
One place we've seen this already is in the continued growth of the tutorial section where its easy for student's to do something bit sized.

OpenCV is now on GIT which, again, makes it easy to do minor (or extensive) submissions. But small tweaks are easier for students to keep contributing to. We now have many pull requests and many of these from former students, often when they show up in some new workplace.

Starting in 2013 and continuing to 2014 and this year, we also require students to create a youtube video of their accomplishments. That is a big incentive and Resume item for them.

Ideas page?¶

We have a page at http://code.opencv.org/projects/gsoc2013/wiki

Summary:

Contact info:

Please provide:
- Your name
- A phone number
- An email address where we can reach you for daily communication.
  - If you have a website that discusses your research, work and/or coding, let us know.

Requirements:

We are looking for people who have strong programming backgrounds, there is no time to "learn on the job".
- Most of the positions will require advanced ability and experience in C++ and/or Python.
  - Exception might be for help working on the website itself
- Most of the projects require knowledge of computer vision techniques. If so, your application will not be accepted if you don't have such experience. These will be stated on the projects ideas page.

Again, you must meet the requirements -- it is not enough to "want" to work on computer vision, we have limited time and so we need people who already have the necessary background. If you lack this background, spend the next year getting the background and apply then, we intend to be back for GSoC 2013.

Students will be expected to ''meet'' with their mentors by email, skype or google chat once a week and to twitter at least 2x/day on their current coding activities

Sample Code:

With your application: Please send in some sample code that you are proud of and are prepared to answer questions on.
If you have other useful skills such as experience with code optimization, or knowledge of SSE, MMX, CUDA or especially development experience on Android or iPhone let us know.

Courses Taken:

List the courses that you have taken in:

Math
Computer programming, especially:
Programming languages
Data structures
Web programming
Data base
Computer vision
Engineering
h2. Work Experience

List any work experience that you've had in
- software development and/or in
- computer vision.

Open Source Experience:

If you have already contributed to other open source projects, please tell us what it was and when you did this.

Which project you are interested in and and why you want to do it:

Please tell us which of the project ideas you are interested and why you want to work on that one.
If you have your own idea, please describe it clearly and provide a timeline of progress towards that goal.

References:

Please list 2 or 3 academic or programming work references.

New organization with someone to vouch for us?¶

No, we're an established org, 2010 and 2011. Still, talk to Jean-Yves Bouguet "Jean-Yves Bouguet" <jyb atthat google>, in the streetview group for a view. Or talk to Sebastian Thrun "Sebastian Thrun" <thrun atthat google>, head of GoogleX who knows me (Gary Bradski) well.

Vouch for younger orgs?¶

zxing

Back

Login	Password

Wiki