OpenCV Meeting notes for 2009 year¶

Contents

• OpenCV Meeting notes for 2009 year
  • 2009-01-06
  • 2009-01-13
  • 2009-01-20
  • 2009-01-27
  • 2009-02-03
  • 2009-02-10
  • 2009-02-18
  • 2009-02-24
  • 2009-02-24
  • 2009-03-03
  • 2009-03-10
  • 2009-03-17
  • 2009-03-31
  • 2009-04-07

2009-01-06¶

Agenda

• Update
• VTK?
• Finding planes with Radu's code?

Minutes

• Russian holidays
  • Vadim working on using Lapack+BLAS in OpenCV
    • Using ATLAS & MKL
• See about getting some of Radu Ruso's mesh routines in OpenCV
• Want to move C-descriptor into OpenCV
  • What's needed to make a paper out of it
• 3D reconstruction
  • Good camera
  • Textured light
  • Rotating platform and rotate object captured frame by frame
  • Robot can scan object
• Structure from motion
  • What is JD working on with 3D reconstruction
• "OpenCV" entry in manufacturing robots

Action Items

Gary

• Talk to JD about code
• Record minutes of tomorrows discussion with the Vision workbench team
• Talk to Calendar about producing a Star-C-descriptor paper

Vadim

• Compare OpenCV star detector with ROS star detector

2009-01-13¶

• Updates
• Interface issues (what next)
• Pattern Recognition
• 3D functionality

Minutes

Update Gary:

• Set up stereo capture node and will collect 100's of outlets and doorknobs
• Collecting 3D object database of Ikea objects
• Writing an opencv tutorial for Stanford class, will post that

Update Vadim:

• Continued work on BLAS and LAPACK interface
  • Eigen vectors and values,
  • SVD,
  • cvsolve,
  • cvinvert and
  • qr decomposition
  • Accuracy is great and performance is great
    • 500x500 matrix is 2.5x faster
    • Small matrixes, 10x10, the new implementation is slower
      • Need to add branches in this case
  • Netlib.org Jack Dungar's implementation, puts out clapack with same interface
    • Eigen2 lib, small matrices are faster than lapack, but algorithms aren't as well developed as lapack
• multiwin replaced using wImage (code from Google)
• This week is after Russian new year vacation.
  • Anatoli working on Octrees for Spin images
• CVS becomes obsolete this week

Update Victor

• Was sick with mono for 3 weeks
• Outlets
  • Filter keypoints using random forest
    • Train image passes around keypoints
    • 2 class filter for inside/outside outlets, works 90%
    • 70% recognition on hard datasets with textured walls and different lighting condition
    • Low false positive rate
      • Need to turn down false positive rate to gain low false negative (missed detection) rate and then apply stronger models to the detections
    • Needs to wait to final camera setup
  • Need datasets for US outlets for keypoint detection
    • Adapt hole searching code for US outlets
• Doorknobs
  • Dark round objects using series of thresholds and connected components
  • RCG collected doorknobs together with chessboards.
    • Perspective corrected
    • Trained adaboost
    • Same for test set
      • Results weren't that good -- 40% detection with 15% false alarm
      • Gary suspects making many different templates to match -- one each 10 degrees.
• Random forest improvements
  • Extreme random trees code implemented
    • Bugs fixed while in hospital
    • Compare with same datasets measure test errors.
    • Errors with noise variables
    • Variable importance -- correction noise variables
    • If improvements, possible paper

Videri camera is supported in OpenCV

Nearest Neighbor

• cvFeatureTree, already in OpenCV
• David Lowes FLANN -- test against cvFeatureTree()

Action Items

Gary

• Get doorknob and outlet datasets to Victor
• Get outlets from Milestone dataset to Victor
• Get cannon outlet sites to Argus or Willow sites
• Get Willow doorknobs to Victor

Vadim

• Send SVN instructions for Window to Gary

Victor

• Test random trees against random forest

2009-01-20¶

Agenda

Update:

• Milestone2
  • Outlets up close
  • => Victor to Doorknobs?
    • 3D box with chessboard
• Planning for Summer release, what should be in it

Minutes

Gary
Close range outlet detection to subpixel

Vadim

1. LAPACK is now included into OpenCV tree.
   cvSVD, cvEigenVV, cvDet, cvSolve and cvInvert have been rewritten using LAPACK;
   cvInvert and cvSolve have got an additional method CV_CHOLESKY, cvSolve also got additional method CV_QR.
2. Anatoly has implemented a draft version of octree for computing spin images. The existing implementation of octree did not contain the key algorithm for retrieving all the points in the cloud that belong to the given sphere and it was difficult to integrate it, so the new class has been written. Anatoly compared performance of this new code with OBB tree from VTK, which has been used by the existing spin image code and found that the performance of the two is very close now:
   • Hope to speed up spin feature by factor of 10
   • Future to speed up Radu's histogram of normals

Next release

• C++ interface with templates
  • 30-50% done
  • Templates will be optional
  • More compatible with STL
• Much better python interface
  • Mark submitted a patch for python wrapper and use with numpy
• 3D features/related algorithm
  • Spin images -- goal is 10x speedup.
  • Plane fitting from Radu
• Alpha and/or 4 channel support
• 2D features
  • C-descriptor (Calandor descriptor)
  • Maybe Boundary fragments
• Stitching/VSLAM
  • Move in the V-SLAM stitching into OpenCV (?, more probably next ...)
  • Place localization/recognition as an app or example code for C-Descriptors
• Optical flow
  • Dense
  • Rigid
  • Affine
  • Intern from Michael Black
• Machine learning library
  • Extremely Randomized Trees
  • Nistor Tree
  • Convolutional NN
  • Additions to RF
  • Extend cascade to other features beyond Haar

PRIORITIES

1. C++ interface
2. Python
3. 3D Features (for Obj rec)
4. 2D Features (for Obj rec)

(no order)

• MLL
• Alpha / multi-channel
• Optical flow improvements
• Stitching

Victor
Extremely Randomized Trees (ET) algorithm shows comparable performance with published in the original Geurts paper on 4 datasets while on 2 datasets (spambase and vehicle) it demonstrates larger errors (see the table below). We are still investigating this issue, the current main hypothesis is that there is a bug that plays a significant role only on these two datasets. Variable importance charts built for Random Forest (RF) and ET models are similar.

Name	RF MLL	ET MLL	ET Geurts
waveform	17.1	16.06	16.61
vehicle	21.88	47.29	26
spambase	5.41	22.68	4.17
isolet	6.61	10.52	7.61
twonorm7400	4.523944	3.745071	3.53
ringnorm7400	6.164789	3.138028	3.27

There is a bug on Vehicle and Spambase. UPDATE: This bug is fixed.

The application for automatic tracking of planar objects has been tested on windows and linux with usb camera and video.

A method for learning an RF model of an outlet invariant to affine transformations has been implemented. An RF is learned on patches of frontal view outlets that have been distorted with random affine transforms (following Lepetit's paper). Each outlet candidate gets a weight (in [0,1]) from this model and low weighted samples are filtered out. Attached is a precision-recall curve of this algorithm obtained by varying the weights threshold. It shows the performance of the model (recall = 1 corresponds to no filtering). The most of false positives are gathered on few images with rich texture.

• Have outlets detectors in 2D
  • Single image
  • At least 50x50 ROI
  • Based on finding dark round objects (European) and dark round (American)
  • 2 stage RF (European):
    • First stage dark small round holes (where metal prongs goes in)
      • But many false positives, filtered by RF
      • Look for pairs of holes
    • Second stage does the neighborhood of the found holes look like an outlet
  • Need more American data and can train on that
• Keep going -- use this as one method of attack.

• 3D box relative to chessboard. The RCG (Alexi (tracking, doorknob) Maria (RF, ET))
  • Today he's training adaboost on doorknobs
  • Error control for planar object position
    • Calibration

Action Items

Victor

• Send images of current outlet detector
• Train on Willow images to be sent

Gary

• Send images of outlets to Victor (today)
• Try to get several hundred
• Get doorknob imagery to Victor
• Take 3D model of doorknob
• CC Vadim with Ara
• CC to Jean-Bouguet about 4 channel
• Talk with Michael about what needs to be done to get C-descriptors in OpenCV
• Michael Black's intern
• Talk with Kurt about moving VSLAM stitching

Vadim

• Spin images
  • Preliminary study is quite fast, could be optmized more
  • Most of the time in spin image is not in octree, but in the spin image
• Send me OpenCV goals spreadsheet
• Put up initial next release goals on OpenCV wiki

2009-01-27¶

Agenda

• Updates

Minutes

For OpenCV Release

• Company Retreat
• Outlets
  • Bottlenecks
    • Find outlets to sub-mm when robot is "in position"
      • Must detect that nothing is plugged into the outlet
    • Visual servoing after robot touches the wall with the plug
    • Finding the plug orientation
      • Finding a dropped plug(?)
  • Other things which are "good" but not critical for milestone 2
    • Detecting Willow Garage outlets from a distance with and without stereo and/or laser data
    • Detecting outlets in general from a distance with and without stereo or laser data
• Method for first bottleneck (find plugs to sub-mm accuracy)
1. Using a calibrated camera with rectified images:
   1. Segment the outlet "ovals", find 4 centroids
   2. Compute homography to canonical view and remap
   3. Match templates to find holes and more exact center of outlets
   4. Re-project more exact center found above into the image to find floating point centers
   5. Use these reprojected floating point centers to again compute a homography to the canonical view and remap
   6. Use templates to find the exact location of outlet holes. This should now be sub-mm precision
   7. Using the homography and measured outlet, compute the X,Y,Z location of the outlet holes
• Arrange for common repository

Vadim

• Lapack debugged -- now using clapack
  • Simple small matrix operations still use native OpenCV code
  • Try fast lapack algorithms like ATLAS
  • Make use of MKL if available
• New memory allocator -- fast for small matrixies A*B+C
  • ~5x faster than malloc and free
• Spin images
  • 20x speed up on spin images
  • Compare against Point Feature Histogram PFH
  • Ready in 2-3 more days
  • Can be further optimized by factor of 2x ~ 1 week
  • 20 seconds on 200K points scene of a conference room using 40cm radius
• Stereo
  • Textured areas have problems, either use textured light or something like Kolmogorov
  • Stereo SSE-2 optimization uses only gcc 4.2 or higher
  • Chessboard detector is breaking on high angle views
    • High angles are not so useful for calibration, but good for plane finding
      • Moderate priority
  • Optimizing Kolmogorov
    • Sparse graph cuts -- might not work, but idea is to soft fix points that are unlikely to change
    • Run as background project for now
• Next week
  • C-descriptor
  • C++ image classes
  • cvBoostPatch() from Steve Gould and from Ian Goodfellow

C descriptor

In .../ros/ros-pkg/visino/calonder_descriptor (it should be named C-descriptor for "Classifier Descriptor")
I spoke with Michael Calonder about getting his descriptor (which we use in visual odometry and place recognition) into OpenCV. He's going to write up a short description of how it works. It is already fairly optimized using 16 bit vectors and SSE -- it can do a 512x512 image in about 5ms (depending on complexity of course).

They've been using the "FAST" detector which has a GPL license. We might consider doing our own FAST detector. But, it can use any intrest point detector, it's mainly a really good descriptor. At the end of training he used Matlab to compress the vectors using PCA -- we'd have to replace that step with clapack.

Place Recognition/Nistor/Kmeans

In ../ros/ros-pkg/vision/place_recognition there are several things of interest:

1. Improved kmeans code (src/kmeans.cpp)
2. Nistor vocabulary tree which we can add to MLL (src/vocabulary_tree.cpp)
3. Place recognition (src/place_recognizer.cpp). This could be sample code or a really useful app that uses (is an example of use of) the C-descriptor above, along with the vocabulary tree and kmeans.

Action Items

Gary

• Determine time for outlet meetings with Patrick
• Data with plugs already in outlet
• Get data to Victor of plug images
• Put Victor in touch with Michael Calonder
• Put Vadim in touch with Michael about C-descriptor
• Find out from Kurt status or priority of stereo speedups
• Talk to Scott about getting Vadim SSH access to machines in Willow
• Sandbox in ROS for things like outlet detection experimentation.

Vadim

• Stereo OpenCV vs Kurt algorithm comparison for speed on a known good machine.

2009-02-03¶

Agenda

Outlets will be moved to a meeting directly after the OpenCV meeting, so we will no longer discuss outlet specific issues here but instead focus on OpenCV issues.

• Updates
  • Vadim
  • Victor
  • Gary
• Progress towards Summer release
  • OpenCV200906
  • Memory limitations in machine learning -- design matrix limits
  • Error handling???
• PnP problem approaches

Minutes

Vadim

1. Anatoly and me: finished optimization of OctTree & Spinimage code. OpenMP and SSE2 gave ~2x speedup each. In result, the running time of one of our benchmarks dropped from (ROS was 80 secs), new c code is 880ms and now downto ~230ms with SSE and OpenMP. That is, together with ~20x speedup we achieved with new C code, the total speedup is over 75x. The code has been prepared for integration to OpenCV.
2. The 2 optimizations of the boosting code, proposed by Stephen Gould and Ian Goodfellow, have been integrated to MLL. No feedback from the guys yet.
3. Studied the C-descriptor source code the relevant papers by Michael Calonder and Vincent Lepetit. The algorithm is pretty clear now and the preparation to integration into OpenCV has been started.
4. ~30% of the new matrix class integration has been done. This appeared to be significantly bigger task than I expected.
5. Received the adaptive skin detector code from Farhad Dadgostar; reviewed the code and recieved the corrected variant; The code will be integrated this week.

Victor

• Outlet detection: calibration, template matching and hole detection have been combined in a single application. The method has been adjusted for another camera (narrow lens resulting in higher resolution outlets). A method for finding a tuple of 4 outlets has been improved for the case of large viewing angle. Recalculation to 3D coordinates in cm scale has been implemented and tested. If an outlet is found correctly the error of hole detection is less than 0.5mm for all cases but one.
• Extreme trees (Maria): the code has been refactored, all ET functionality has been encapsulated into a separate class inherited from CvRTrees. The performance of RF and ET has been compared. For very deep trees (depth 20) the performance is comparable, while for more shallow trees (less than 10) ET is faster at least 2x.
  • Has natural parallelism, look at it.
  • Parallize all boosted and random prediction trees.
  • Possible paper in statistical convergence to correct answer
• 3D object tracking (Alexey): the method for tracking 3D will be based on 5 points set by user on two different frames. Stereo pair calibration has been implemented.
  • Right now we are trying to catch a bug in calculating 3D from disparity. Alexi out so this will wait.

Other: Alexey is on vacation Feb2-8, back in the office Feb 9.

Gary

Boosting

• Will try to see if we're still artificially limited by memory size

OpenCV errors

• New implmentation moves completely to exceptions
• Should work with debugger now

OpenCV website

• For demos etc

Action Items

NEW

Gary

• Email Steve Gould about boosting memory limits with Victor and Vadim
• Email Ian Goodfellow about opencv error catching with Vadim.

Vadim

• Check ML design matrix

Victor

FROM LAST TIME

Gary

• Determine time for outlet meetings with Patrick
  • 10am (9pm NN) on Tuesday
• Data with plugs already in outlet
• Get data to Victor of plug images
• Put Victor in touch with Michael Calonder
• Put Vadim in touch with Michael about C-descriptor
• (?) Find out from Kurt status or priority of stereo speedups
• (?) Talk to Scott about getting Vadim SSH access to machines in Willow
  • I talked, but no action yet.
• Sandbox in ROS for things like outlet detection experimentation.
  • Just use sourceforge and ROS can pull it in if you build say the outlet node.

Vadim

• Stereo OpenCV vs Kurt algorithm comparison for speed on a known good machine.

2009-02-10¶

Agenda

Quick updates:

Gary

• Helped get Mech Turk interface into Willow Garage (thanks to Alexander Sorokin and Scott Hassan's 2 daze of hacking last week). View some results (click on the numbers under "completed hits").
  • We can use this for massive data sets and even say sponsor people to create better face recognizers etc.

Vadim

• Matrix interface
• 80% of cxcore is ported to the new C++ interface
  • Automatic garbage collection (reference counter based)
  • Can use with STL vector of images and matrixes
  • Have template interface matrix(i,j) does the right thing
    • Thread safety coming, can use atomic operations, efficient on Intel and AMD archs
• IPP will now be installed/built at configure instead of dynamic
  • Using cmake, if IPP is detected the build flag will be on by default, user can explicitly turn it off (same way as ffmpg libs etc)
• Decision tree memory optimization, needs about 2 weeks (Maria knows the code a bit, but Vadim expert)
  • Needed to move cascade recognizer to general decision trees, beyond Haar features
  • Think about collaborating with someone to train good working models for the release
• Spin images are done but waiting to complete the C++ integration before integration (probably by Friday)
• Focus results, ... lower priority

Victor

• Extreme Trees
  • Categorical support is added
  • Don't need sorting of variables, so lower memory requirements
  • Can handle 100K samples of 1000 features, Extreme trees worked OK
  • Code is in SVN
  • Recognition performance is comparable to RF but faster
    • Inherently more parallel
    • Try on outlets?
• 3D object tracking (Alexander was on vacation)
  • Estimate pose of box in space from motion and knowlege of box
  • Function for disparity to 3D has bug
  • 2D version ... finished ... but special case of 3D, integrate

OpenCV release tracking

• Go over the development plans

Minutes

Tracking the summer release:

1. General Structure:
   • Decisions on library structure (Google's request, separate packages as is ...)
     • Provide way to build OpenCV statically, then size varies with binary size
2. New Features
   • C++ interface
     • ... see above
   • Better python
     • Vadim, Alex ...
     • C++ will take another month but can improve python in parallel
   • 2D features C-descriptor, HoG Boundary Fragments ...
     • C-descriptor ... resolved issues with Michael, Vadim understands code more deeply (next week)
     • Boundary fragments ... working on focus.
     • Get contributions of 2D features externally
     • MSER ... (Gary has code)
     • Star is already in and in for ROS build
   • Alpha channel, alternatives to contours RLE
     • Introduce functions with advance ways of blending
     • Mask channels
     • Optical flow
   • 3D features (Plane fit, PFH, FPFH from Radu), 3D Delaunay, 3D convex hull, mesh generation
     • SPIN almost done
     • Radu is organizing his code into a library ... maybe we can just lift this
   • VSLAM
     • Wait on what Kurt and James get together
   • Image stitching
     • will ask Jean-Yves for pointers
   • Optical flow
     • Move code derived form former work in OpenCV
   • MLL
     • Extreme trees are coming
     • Reduce memory footprint
     • Need to work on input and output of data in CSV (or other simple deliminators), XML and YML
     • Regression tests ... maybe general cross validation
   • Misc: Focus detector, adaptive skin tracking
     • As discussed above
3. Optimizations
   • .
4. Documenation
   • .
5. High GUI
   • .

Action Items

NEW

Meeting next week will be on Wednesday

Gary

• Send Vadim MSER code
• Ask Jean-Yves for priority list of functions to work with mask and alpha channels
• Ask Jean-Yves about image stitching support/libraries/functionality
• Ask Radu about his library progress.
• Send email reminder for Wednesday meeting next week.

Vadim

• Adaptive skin tracker app/test
• Finish cxcore rewrite, make sure all the tests pass, update the documentation.
• Start cv module revision.
• Integrate spin images.

Victor

FROM LAST TIME

Gary

• Email Steve Gould about boosting memory limits with Victor and Vadim
• Email Ian Goodfellow about opencv error catching with Vadim.

Vadim

• Check ML design matrix

Victor

2009-02-18¶

Agenda

Accomplishment Tracking

• .

Chessboard issues

• Detection failures
• 2 functions? One chessboards for calibration, another chessboards for pose?

3D capture, discuss implementation of

• http://www.robots.ox.ac.uk/~awf/turntable/index.html

cxcore re-write, cv-rewrite

Spin, skin

Release tracking.

Minutes

Tracking the summer release:

1. General Structure:
   • Decisions on library structure (Google's request, separate packages as is ...)
     • Provide way to build OpenCV statically, then size varies with binary size
2. New Features
   • C++ interface
     • ... see below
   • Better python
     • Vadim, Alex ...
     • C++ will take another month but can improve python in parallel
   • 2D features C-descriptor, HoG Boundary Fragments ...
     • C-descriptor ... resolved issues with Michael, Vadim understands code more deeply (next week)
     • Boundary fragments ... working on focus.
     • Get contributions of 2D features externally
     • MSER ... (Gary has code)
     • Star is already in and in for ROS build
   • Alpha channel, alternatives to contours RLE
     • Introduce functions with advance ways of blending
     • Mask channels
     • Optical flow
   • 3D features (Plane fit, PFH, FPFH from Radu), 3D Delaunay, 3D convex hull, mesh generation
     • SPIN almost done
     • Radu is organizing his code into a library ... maybe we can just lift this
   • VSLAM
     • Wait on what Kurt and James get together
   • Image stitching
     • will ask Jean-Yves for pointers
   • Optical flow
     • Move code derived form former work in OpenCV
   • MLL
     • Extreme trees are coming
     • Reduce memory footprint
     • Need to work on input and output of data in CSV (or other simple deliminators), XML and YML
     • Regression tests ... maybe general cross validation
   • Misc: Focus detector, adaptive skin tracking
     • As discussed above
3. Optimizations
   • .
4. Documentation
   • .
5. High GUI
   • .

New Interface

• cxcore revision is complete (except for XML/YAML input/output functions). All the algorithmic tests for cxcore and cv pass, all the samples run well.
  • As a small illustration of the new API capabilities, here is a some code that I used to debug SVD:

 1    // matrix A comes in
 2    SVD svd(A);
 3    int n = A.rows;
 4
 5    double a = norm(svd.u*svd.u.t() - Mat::eye(n,n), CV_C);
 6    double b = norm(svd.vt*svd.vt.t() - Mat::eye(n,n), CV_C);
 7    int nz = countNonZero(svd.w >= 0);
 8    double c = norm(svd.u*Mat::diag(svd.w)*svd.vt - Mat::eye(n,n), CV_C);
 9    CV_Assert( a < 1e-5 && b < 1e-5 && c < 1e-5 && nz == n );

• Can template this
  • Can create image with up to 64 channels
  • Some functions limit to 4 channels
  • Splitting channels is easy
    • Work with IPP, for cache, want to extract channel

General Issues:

Chessboard

• cvFindCornersSubpix fails with large search window, diverges.
  • Ordering of chessboard corners and splitting
  • Will lookat
• Initial location of corners can be wrong
  • Extreme angles can create problems for morphological operations
    • vertical and horizontal operations can make distance between corners smaller than within squares

3D

• http://www.robots.ox.ac.uk/~awf/turntable/index.html
  Skin detection should work using highGUI camera

Spin images, trying on Ikea

3D tracking (Victor, Alexi) (3D box)

• 3d positions of markers using Chessboard
  • Currently unstable depending on where the user clicks
  • Somewhere in disparity to 3D transformation
  • Generating tests on synthetic data by tomorrow.

Memory use with MLL

• Shrank memory use several times
• Debugging

Action Items

NEW

ALL

• Send in list of what's been accomplished so far in OpenCV since Willow started supporting development.

Gary

• Look into sending prosilica and other cameras to Argus
• Send full data link with mask images for Spin image work
• Get Zisserman 3D paper to Vadim (on ACM)

Vadim

• Look at better calibration and pose object
  • Equal or better in accuracy and stability
• Check what you think 3D scanning would take

Victor

FROM LAST TIME

Meeting next week will be on Wednesday

Gary

• Send Vadim MSER code
• [but he didn't answer yet] Ask Jean-Yves for priority list of functions to work with mask and alpha channels
• [same] Ask Jean-Yves about image stitching support/libraries/functionality
• Ask Radu about his library progress. Radu:
  

  We took a few steps today to unify the point cloud -related libraries in ROS under one umbrella. The result is that
  there is now a _point_cloud_mapping_ package which contains the following things:
  - cloud_geometry: areas, distances, intersections, 3D normals/features estimation, norms, projections, statistics,
  transformations, etc;
  - cloud_io: a library for Input/Output PCD (Point Cloud Data) processing;
  - cloud_kdtree: a library for nD spatial decomposition and nearest neighbors searches for PointCloud messages using k-D
  trees;
  - cloud_octree: a library for 3D spatial decomposition of PointCloud messages using octree data structures;
  - sample_consensus: a library for SAmple Consensus (SAC) like methods (RANSAC, MSAC, MLESAC, etc,) and geometric shape
  models;
  There are also 2 basic nodes included with the package:
  - cloud_downsampler_node: a node which uniformly downsamples a 3D point cloud
  - normal_estimation_node: a node which estimates local surface properties at each point such as surface normals,
  curvature changes, moment invariants, etc.
  

• Send email reminder for Wednesday meeting next week.

Vadim

• Adaptive skin tracker app/test
• Finish cxcore rewrite, make sure all the tests pass, update the documentation.
• Start cv module revision.
• Integrate spin images.

Victor

2009-02-24¶

Agenda

• Accomplishments
• Interface update
• Chessboard improvements
• 3D Box relative to chessboard
• 3D Object Capture paper/analysis
• Spin images
• Summer release tracking

Minutes

Tracking the summer release:

1. General Structure:
   • Decisions on library structure (Google's request, separate packages as is ...)
     • Provide way to build OpenCV statically, then size varies with binary size
2. New Features
   • C++ interface
     • cxcore
     • cv -- start with filtering functions, resize, optical flow, object detection (2 weeks)
   • Better python
     • Vadim, Alex ...
     • C++ will take another month but can improve python in parallel
   • 2D features C-descriptor, HoG Boundary Fragments ...
     • C-descriptor ... resolved issues with Michael, Vadim understands code more deeply (next week)
       • No re-implementation yet. Code based on randomized tree, but specialized version.
     • Boundary fragments ... working on focus.
     • Get contributions of 2D features externally
     • MSER ... (Gary has code)
     • HoG
     • Star is already in and in for ROS build
   • Alpha channel, alternatives to contours RLE
     • Introduce functions with advance ways of blending
     • Mask channels
     • Optical flow
   • 3D features (Plane fit, PFH, FPFH from Radu), 3D Delaunay, 3D convex hull, mesh generation
     • SPIN almost done
     • Radu is organizing his code into a library ... maybe we can just lift this
   • VSLAM
     • Wait on what Kurt and James get together
   • Image stitching
     • will ask Jean-Yves for pointers
   • Optical flow
     • Move code derived form former work in OpenCV
   • MLL
     • Extreme trees are coming
     • Reduce memory footprint
     • Need to work on input and output of data in CSV (or other simple deliminators), XML and YML
     • Regression tests ... maybe general cross validation
     • Generalize cascade
   • Misc: Focus detector, adaptive skin tracking
     • As discussed above
3. Optimizations
   • .
4. Documentation
   • .
5. High GUI
   • .

OTHER

New Interface

• Whole new interface for cxcore is finsished
  • Moving to cv
  • Examples: Update samples. .c code will be old interface, .cpp will be new interface
  • Output is simplified by automatically correct type
  • Start effort to doxygen (lower background priority)

Chessboard code that

• Vadim has released a patch that seems to fix all known problems. Congrats!
• High angle (60 degrees) boards still cannot be done
• Detect lines, automatically detect board size, perhaps remap image to frontal view.
  • Sub-pixel finding is now put inside the algorithm rather than after.

3D box

• 3D object tracking: the code is in debugging phase. The calibration of a stereo pair is still unstable, producing large reprojection errors too often.
• Alexi not much time to work on in
• Make relative to chessboard

3D Object Capture

• Bundle adjustment missing from OpenCV
• Mesh reconstruction, not in OpenCV
• Might take a month if we can't get the above code
  • http://www.3dsom.com/index.html seems to use the same technique.

Spin images

• Not incorporated yet, code is ready, just time to integrate
  • Test code is ready
  • 1 or 2 days

Extremely random trees

• Put a callback in the splitting function

MSER

• Alex has MSER code

Action Items

NEW

Gary

• Ask Kurt about where Bundle adjustment is ... if it is
• Send Victor Florian's paper

Vadim

• Doxygen documentation as background, lower priority task.

Victor

FROM LAST TIME

ALL

• Send in list of what's been accomplished so far in OpenCV since Willow started supporting development.

Gary

• Look into sending prosilica and other cameras to Argus
• Send full data link with mask images for Spin image work
• Get Zisserman 3D paper to Vadim (on ACM)

Alexi

• Finish up 3D box

Vadim

• Look at better calibration and pose object
  • Equal or better in accuracy and stability
• Check what you think 3D scanning would take

Victor

2009-02-24¶

Agenda

• Accomplishments
• Interface update
• Chessboard improvements
• 3D Box relative to chessboard
• 3D Object Capture paper/analysis
• Spin images
• Summer release tracking

Minutes

Tracking the summer release:

1. General Structure:
   • Decisions on library structure (Google's request, separate packages as is ...)
     • Provide way to build OpenCV statically, then size varies with binary size
2. New Features
   • C++ interface
     • cxcore
     • cv -- start with filtering functions, resize, optical flow, object detection (2 weeks)
   • Better python
     • Vadim, Alex ...
     • C++ will take another month but can improve python in parallel
   • 2D features C-descriptor, HoG Boundary Fragments ...
     • C-descriptor ... resolved issues with Michael, Vadim understands code more deeply (next week)
       • No re-implementation yet. Code based on randomized tree, but specialized version.
     • Boundary fragments ... working on focus.
     • Get contributions of 2D features externally
     • MSER ... (Gary has code)
     • HoG
     • Star is already in and in for ROS build
   • Alpha channel, alternatives to contours RLE
     • Introduce functions with advance ways of blending
     • Mask channels
     • Optical flow
   • 3D features (Plane fit, PFH, FPFH from Radu), 3D Delaunay, 3D convex hull, mesh generation
     • SPIN almost done
     • Radu is organizing his code into a library ... maybe we can just lift this
   • VSLAM
     • Wait on what Kurt and James get together
   • Image stitching
     • will ask Jean-Yves for pointers
   • Optical flow
     • Move code derived form former work in OpenCV
   • MLL
     • Extreme trees are coming
     • Reduce memory footprint
     • Need to work on input and output of data in CSV (or other simple deliminators), XML and YML
     • Regression tests ... maybe general cross validation
     • Generalize cascade
   • Misc: Focus detector, adaptive skin tracking
     • As discussed above
3. Optimizations
   • .
4. Documentation
   • .
5. High GUI
   • .

OTHER

New Interface

• Whole new interface for cxcore is finsished
  • Moving to cv
  • Examples: Update samples. .c code will be old interface, .cpp will be new interface
  • Output is simplified by automatically correct type
  • Start effort to doxygen (lower background priority)

Chessboard code that

• Vadim has released a patch that seems to fix all known problems. Congrats!
• High angle (60 degrees) boards still cannot be done
• Detect lines, automatically detect board size, perhaps remap image to frontal view.
  • Sub-pixel finding is now put inside the algorithm rather than after.

3D box

• 3D object tracking: the code is in debugging phase. The calibration of a stereo pair is still unstable, producing large reprojection errors too often.
• Alexi not much time to work on in
• Make relative to chessboard

3D Object Capture

• Bundle adjustment missing from OpenCV
• Mesh reconstruction, not in OpenCV
• Might take a month if we can't get the above code
  • http://www.3dsom.com/index.html seems to use the same technique.

Spin images

• Not incorporated yet, code is ready, just time to integrate
  • Test code is ready
  • 1 or 2 days

Extremely random trees

• Put a callback in the splitting function

MSER

• Alex has MSER code

Action Items

NEW

Gary

• Ask Kurt about where Bundle adjustment is ... if it is
• Send Victor Florian's paper

Vadim

• Doxygen documentation as background, lower priority task.

Victor

FROM LAST TIME

ALL

• Send in list of what's been accomplished so far in OpenCV since Willow started supporting development.

Gary

• Look into sending prosilica and other cameras to Argus
• Send full data link with mask images for Spin image work
• Get Zisserman 3D paper to Vadim (on ACM)

Alexi

• Finish up 3D box

Vadim

• Look at better calibration and pose object
  • Equal or better in accuracy and stability
• Check what you think 3D scanning would take

Victor

2009-03-03¶

Agenda

• SURF and OpenCV Announce
• Marius Muja is at Willow for 6 months
  • FLANN incorporation
• New interface update
• 3D Box -- it slips
• 3D Model
• MSER

Minutes

Tracking the summer release:

1. General Structure:
   • Decisions on library structure (Google's request, separate packages as is ...)
     • Provide way to build OpenCV statically, then size varies with binary size
2. New Features
   • C++ interface
     • cxcore
     • cv -- start with filtering functions, resize, optical flow, object detection (2 weeks)
   • Better python
     • Vadim, Alex ...
     • C++ will take another month but can improve python in parallel
   • 2D features C-descriptor, HoG Boundary Fragments ...
     • C-descriptor ... resolved issues with Michael, Vadim understands code more deeply (next week)
       • No re-implementation yet. Code based on randomized tree, but specialized version.
     • Boundary fragments ... working on focus.
     • Get contributions of 2D features externally
     • MSER ... (Gary has code)
     • HoG
     • Star is already in and in for ROS build
   • Alpha channel, alternatives to contours RLE
     • New alpha transparency (per pixel) added
     • Introduce functions with advance ways of blending
     • Mask channels
     • Optical flow
   • 3D features (Plane fit, PFH, FPFH from Radu), 3D Delaunay, 3D convex hull, mesh generation
     • SPIN almost done
     • Radu is organizing his code into a library ... maybe we can just lift this
   • 3D capture http://www.3dsom.com/index.html
   • VSLAM
     • Wait on what Kurt and James get together
   • Image stitching
     • will ask Jean-Yves for pointers
   • Optical flow
     • Move code derived form former work in OpenCV
   • MLL
     • Extreme trees are coming
     • Reduce memory footprint
     • Need to work on input and output of data in CSV (or other simple deliminators), XML and YML
     • Regression tests ... maybe general cross validation
     • Generalize cascade
   • Misc: Focus detector, adaptive skin tracking
     • As discussed above
3. Optimizations
   • .
4. Documentation
   • .
5. High GUI
   • .

OTHER

OpenCV Announce ...

• SURF is in #1 in change log
• Post change log to wiki
• OpenCV Weekly ...

FLANN

• Some timing results at bottom
• Code is in good shape
  • 3D party directoy can be put there
  • For mac, can be put in system path
  • Linux can select
• In CV module, can create a wrapper for CvMat interface to FLANN
  • Make test, so can use included version of FLANN
  • First looks to internal unless user specifies external
    • Put flag to download external
    • We keep reasonably up to date with external

Interface update

• cxcore in SVN
  • Mac and Windows,
  • Linux: gcc caused changes due to template handling differences
• convolution in cv
  • Then geometrical transforms
  • Optical flow
  • Add new alpha blending function with separate transparency for each pixel
  • Further functions can be lower priority

3D box

• Box slips up (pretty sure)
• Image is rectified
  • If camera is close, distortion can be large
  • Tomorrow will ship new video with box being tracked
• Could be chessboard slippage ?

3D Object Capture

• Bundle adjustment missing from OpenCV
• Mesh reconstruction, not in OpenCV
• Might take a month if we can't get the above code
  • http://www.3dsom.com/index.html seems to use the same technique.
• Initialization of extrinsics (pose) is hardest part of the code
• Different from general SFM or stereo
• smart inversion of J^t*J
  • pass to SVD (might be 10x slower but easier)

MSER

• Gary sent
  • Victor looking at right now,
    • Alex doing his own version faster at lower priority ~ couple of weeks

SURF

• detection and descriptor united
  • Should be split
  • "Find_obj.cpp" is demo code in samples/c

Vadim:

1. cxcore has been built with GCC 4.0 and GCC 4.2 on MacOSX. Quite a few changes have been made because GCC handles some template stuff differently and usually catches more bugs. Another portion of changes were made to make all the tests pass. This tested version of cxcore has been committed to SVN repository.
2. The work on revising cv has been started, new interface for filtering functions (linear separable and non-separable, Laplacian, median, morphogical operations) is ~90% ready (the code was written and now debugged).
3. fixed 1 minor bug in boosting (reported by Ian Goodfellow) and 2 bugs in camera calibration (reported by Sergey).
4. Anatoly started the work on benchmarking various fast n-dim search algorithms for quality & speed. For now we have 3 algorithms in OpenCV: kdtree, spilltree, LSH (locally-sensitive hash), and there is also FLANN. We could not run LSH properly yet, but here are some first results on synthetic uniformly-distributed data in n-dim cubes:
   • tAvgDist is the average (over the test set) distance to the nearest point, as returned by various alorithms (DumbBruteForce is the naive algorithm), the closer to the naive algorithm, the better.
   • M1 is average over the test set of (min_dist/average_dist), the smaller the better.
   • M2 is the number of times the returned point was closer than MAX, 0)
   • and M3 is average of (average_dist - min_dist)/stddev(distance). More than 3 should good enough from statistical point of view, but the larger - the better.
• These preliminary results show that kd-tree is the fastest method and generally not that bad, FLANN is as accurate as naive algorithm, yet significantly faster (but significantly slower than kd-tree, we wonder, whether it can be tuned to be less accurate but faster)
• spill tree is only good for low-dimensional space, otherwise brute force is much faster.
• LSH is yet to be tested.

Action Items

NEW

Gary

• Make change log wiki page, and send to Vadim
• Make OpenCV Montly Newsletter and send to Vadim
• Ask J-Y again about mask image uses/image stitching
• Find JD's code and send to Vadim
• Send email to Radu to point to mesh code/techniques
• Calendor to Andrew Ng

Vadim

• Put up change log wiki page and monthly page after Gary set's up

Victor

• Do new video of box

FROM LAST TIME

Gary

• Ask Kurt about where Bundle adjustment is ... if it is
  • Umm, JD Chen (no longer with Willow) supposedly did this, but for stereo code, sent him an email
• Send Victor Florian's paper
  • http://www.robots.ox.ac.uk/~vgg/publications/papers/schroff08.pdf
  • http://www.robots.ox.ac.uk/~vgg/publications/papers/schroff06.pdf

Vadim

• Doxygen documentation as background, lower priority task.

Victor

2009-03-10¶

Agenda

Summer of Code

New interface status

FLANN Benchmarks

Status of Extremely Random Trees -- is it in

MSER?

3D Box

3D capture

• Bundle adjustment missing from OpenCV
• Mesh reconstruction, not in OpenCV
• Might take a month if we can't get the above code
  • http://www.3dsom.com/index.html seems to use the same technique.

Minutes

Tracking the summer release:

1. General Structure:
   • Decisions on library structure (Google's request, separate packages as is ...)
     • Provide way to build OpenCV statically, then size varies with binary size
2. New Features
   • C++ interface
     • cxcore
     • cv -- start with filtering functions, resize, optical flow, object detection (2 weeks)
   • Better python
     • Vadim, Alex ...
     • C++ will take another month but can improve python in parallel
   • 2D features C-descriptor, HoG Boundary Fragments ...
     • C-descriptor ... resolved issues with Michael, Vadim understands code more deeply (next week)
       • No re-implementation yet. Code based on randomized tree, but specialized version.
     • Boundary fragments ... working on focus.
     • Get contributions of 2D features externally
     • MSER ... (Gary has code)
     • HoG
     • Star is already in and in for ROS build
   • Alpha channel, alternatives to contours RLE
     • New alpha transparency (per pixel) added
     • Introduce functions with advance ways of blending
     • Mask channels
     • Optical flow
   • 3D features (Plane fit, PFH, FPFH from Radu), 3D Delaunay, 3D convex hull, mesh generation
     • SPIN almost done
     • Radu is organizing his code into a library ... maybe we can just lift this
   • 3D capture http://www.3dsom.com/index.html
   • VSLAM
     • Wait on what Kurt and James get together
   • Image stitching
     • will ask Jean-Yves for pointers
   • Optical flow
     • Move code derived form former work in OpenCV
   • MLL
     • Extreme trees are coming
     • Reduce memory footprint
     • Need to work on input and output of data in CSV (or other simple deliminators), XML and YML
     • Regression tests ... maybe general cross validation
   • Misc: Focus detector, adaptive skin tracking
     • As discussed above
3. Optimizations
   • .
4. Documentation
   • .
5. High GUI
   • .

OTHER

Summer of code

• We already have not just a cascade, but a cascade-tree
• Gradient boosting trees(?) -- project?
• Optimize general boosting, not haar
• Have Mechanical Turk
• Image stitching
• HighGUI

New interface

• Filtering functions in cv all pass test
  • SSE optimizations added for most important filtering functions
• Image warping next
• Usergroup glitch (old macro) posted to SVM

FLANN

• Investigation is continuing by Anatoly
  • With new FLANN #2, the performance gap is smaller
    • Accuracy is the same
    • kd-tree in OpenCV is very fast, but not so accurate
    • FLANN, accuracy is high, but speed is slower.
      • We made it faster, with same accuracy, but don't know how to make it less accurate for more speed.

Extremely Random Trees

• Is in. Needs sample code

MSER

• Got it, stripped out other parts and compiled it.
• Alexander working on it.

3D Box

• There was a bug, found it, corrected it.
• Sent email

REPORTS

Vadim

1. Debugged the new filtering functions (called by cvSmooth, cvFilter2D, cvErode, cvDilate, cvSobel). Now all the tests pass. Also, added SSE2 optimization to some important functions, like morphology and separable filters (like Sobel and Gaussian), so they should be much faster even if IPP is not available.
2. Started revision of image warping functions.
3. Anatoly continued investigation of various n-dim search algorithms. In the latest version of benchmark and with FLANN #2 the FLANN part runs faster, but still there is gap in speed-accuracy tradeoff. Also, the LSH part still does not give adequate results to be included into the benchmark. These 2 items are being investigated. Anatoly also ported the benchmark to Linux and sent it to Marius.

Victor

• See outlet detection for main report
• 3D tracking: several bugs have been corrected, the new video is at ftp://argus-cv.dnsalias.org/3DTrackDemo/logitec_demo.avi, login willow, password awgarage.

Action Items

NEW

Gary

• Put up GSofC projects
• Job application page
• Put up application
• Ask Marius to interact with Anatoly on FLANN

Vadim

• Put up GSofC projects

Victor

• Put up GSofC projects

FROM LAST TIME

Gary

• Make change log wiki page, and send to Vadim
• Make OpenCV Montly Newsletter and send to Vadim
• Ask J-Y again about mask image uses/image stitching
• Find JD's code and send to Vadim
• Send email to Radu to point to mesh code/techniques
• Calendor to Andrew Ng

Vadim

• Put up change log wiki page and monthly page after Gary set's up

Victor

• Do new video of box

2009-03-17¶

Agenda

[No meeting next week, I won't be in]

Google Summer of Code

HOG and SPIN

3D rotation + Translation of stereo images

Updates

• New interface

Monthly announce, Change Logs

FLANN Benchmarks

3D Box

3D capture

Minutes

OpenCV Summer Release

1. General Structure:
   • Decisions on library structure (Google's request, separate packages as is ...)
     • Provide way to build OpenCV statically, then size varies with binary size
2. New Features
   • C++ interface
     • cxcore
     • cv -- start with filtering functions, resize, optical flow, object detection (2 weeks)
   • Better python
     • Vadim, Alex ...
     • C++ will take another month but can improve python in parallel
   • 2D features C-descriptor, HoG Boundary Fragments ...
     • C-descriptor ... resolved issues with Michael, Vadim understands code more deeply (next week)
       • No re-implementation yet. Code based on randomized tree, but specialized version.
     • Boundary fragments ... working on focus.
     • Get contributions of 2D features externally
     • MSER ... (Gary has code)
     • HoG
     • Star is already in and in for ROS build
   • Alpha channel, alternatives to contours RLE
     • New alpha transparency (per pixel) added
     • Introduce functions with advance ways of blending
     • Mask channels
     • Optical flow
   • 3D features (Plane fit, PFH, FPFH from Radu), 3D Delaunay, 3D convex hull, mesh generation
     • SPIN almost done
     • Radu is organizing his code into a library ... maybe we can just lift this
   • 3D capture http://www.3dsom.com/index.html
   • VSLAM
     • Wait on what Kurt and James get together
   • Image stitching
     • will ask Jean-Yves for pointers
   • Optical flow
     • Move code derived form former work in OpenCV
   • MLL
     • Extreme trees are coming
     • Reduce memory footprint
     • Need to work on input and output of data in CSV (or other simple deliminators), XML and YML
     • Regression tests ... maybe general cross validation
   • Misc: Focus detector, adaptive skin tracking
     • As discussed above
3. Optimizations
   • .
4. Documentation
   • .
5. High GUI
   • .

OTHER

GSoC

• Already received an application, but Google decides who's in on the 18th
• I've been working on some stereo data collection for people detection
  • Also for object detection too

HOG

• 2D feature

SPIN

• Code and
• Demo reads vrml
• Constructs oct tree C++
• Has compute normal points separate functions (takes oct tree)
• Then computes spin image

3D ROTATION TO NEW IMAGE

• 3d perspective transform
• perspective projection
• for each point, find generalized interpolation
  • Computer weighted sum of K nearest points
  • To search in projected view, use quad tree or regular grid

Vadim update

1. Made another few corrections in filtering functions to make the API a bit more convenient. Updated filtering-depdendent functions (that used not cvSobel, cvFilter2D etc., but the internal API): SURF, Snakes, CornerMinEigenVal etc. Now the tests for these functions pass too. Here are some performance numbers (on Core2Duo #6GHz laptop, assuming that performance of C code has not been changed much comparing to the old version of OpenCV, this is how much faster the summer version will be comparing to #0 and #1pre).
2. Started the work on updating image warping functions. cvResize & cvWarpAffine have been rewritten and optimized using SSE2 intrinsics. The code is now being debugged.

FLANN:

Anatoly finished benchmarking FLANN and OpenCV's KD-tree.

• Below is the table (M1 and M3 mean the same thing as in previous report:
  • M1 - ratio of the distance to the found neighbor to the average distance to points in DB. The smaller - the better.
  • M3 - normalized to standard deviation difference between average distance and best distance. >= 2-2.5 is already very good. The larger - the better.

• Conclusion
  • Current conclusion is that OpenCV's kd-tree is comparable to FLANN in both speed and accuracy (by varying parameters we can get either better accuracy or better performance).
  • Anatoly now started working on HoG and the framework for model matching using spin-images.

3D Box

• Sent video of tracking another box
• Put in local SVN

Application area

• Under trunk put OpenCV Apps

3D Reconstruction

• Nothing

Action Items

NEW

Gary

• Talk to Radu about 3D rotation of points and create an image
• Gary to send Anatoly stereo data for trying HOG and SPIN
• Send gaussian curvature paper
• Kurt to send paper on fast plane fit
• Histogram of Gradient link

Vadim

• Anatoly -- prioritize HOG, SPIN
  • Fast plane fit

Victor

• Set up the first OpenCV appliation: 3D box
  • Video of it working
  • Raw video that spits out marked object

FROM LAST TIME

Gary

• Put up GSofC projects
• Job application page
• Put up application
• Ask Marius to interact with Anatoly on FLANN

Vadim

• Put up GSofC projects

Victor

• Put up GSofC projects

2009-03-31¶

Agenda

Interface progress

• Python machine learning

Updates for

• Accomplishments
• Monthly announce, Change Logs
• Contributors credit page?

Spin and HOG progress

• How was HOG done?

Trying 3D capture code

Point cloud library

Application areas

New HighGUI

Improving the machine learning interface(?)

• There is also the issue of a new classifier design for robotics that needs the following capabilities:
  • Needs to do one shot learning (but can refine later)
  • Needs to be able to combine separately learned objects in a sub-linear way
  • Needs the ability to express some structure in the data, particularly spatial or geometric structure.

Minutes

OpenCV Summer Release

1. General Structure:
   • Decisions on library structure (Google's request, separate packages as is ...)
     • Provide way to build OpenCV statically, then size varies with binary size
2. New Features
   • C++ interface
     • cxcore
     • cv -- start with filtering functions, resize, optical flow, object detection (2 weeks)
   • Better python
     • Vadim, Alex ...
   • 2D features C-descriptor, HoG Boundary Fragments ...
     • C-descriptor ... resolved issues with Michael, Vadim understands code more deeply (next week)
       • No re-implementation yet. Code based on randomized tree, but specialized version.
     • Boundary fragments ... working on focus.
     • Get contributions of 2D features externally
     • MSER ... (Gary has code)
     • HoG
     • Star is already in and in for ROS build
   • Alpha channel, alternatives to contours RLE
     • New alpha transparency (per pixel) added
     • Introduce functions with advance ways of blending
     • Mask channels
     • Optical flow
   • 3D features (Plane fit, PFH, FPFH from Radu), 3D Delaunay, 3D convex hull, mesh generation
     • SPIN almost done
     • Radu is organizing his code into a library ... maybe we can just lift this
   • 3D capture http://www.3dsom.com/index.html
   • VSLAM
     • Wait on what Kurt and James get together
   • Image stitching
     • will ask Jean-Yves for pointers
   • Optical flow
     • Move code derived form former work in OpenCV
   • MLL
     • Extreme trees are coming
     • Reduce memory footprint
     • Need to work on input and output of data in CSV (or other simple deliminators), XML and YML
     • Regression tests ... maybe general cross validation
   • Misc: Focus detector, adaptive skin tracking
     • As discussed above
3. Optimizations
   • .
4. Documentation
   • .
5. High GUI
   • .

OTHER

Vadim:

1. Revised, converted to the new API and tested:
   • all the image warping functions (Resize, WarpAffine, WarpPerspective, Remap & Undistort), with SSE2 optimization the performance improves up to 50-100%.
   • integral images
   • fixed-level threshold and adaptive threshold + SSE2 optimization
   • functions for background statistics accumulation
   • For the previously added median blur function added SSE2 optimization for 3x3 and 5x5 kernels, up to x10 speedup.
2. Fixed haartraining app. problem (reported by Caroline Pantofaru) by reverting couple of files to the versions from OpenCV #0.
3. Modified spin images to use the new C++ matrix class. The updated code is in SVN.
4. Integrated MSER code by Liu Liu. For now the C interface is retained.
5. Anatoly implemented function for extracting HoG features based on the contributed by Alberto Orlandi code. At new version is also ~50% faster. Some further optimization is possible. The code is in SVN now.
6. Started working with the dataset of IKEA objects. HoG features did not provide any reasonable output (e.g. comparing to SURF), because the object outlines are very simple and HoG seems to require a classifier on top of it. Now trying to run spin images.

• Next for API is optical flow
  • Pyramid and adding
  • Dense optical flow)
  • High Gui
• MLL
  • Extremely random trees Importance Variables
  • Input interface: to csv and loaded with standard function in SVN tomorrow

• ROS stable this week.
  • latest_tested_snapshot is the SVN flag for ROS

Stereo data

• On mechanical turk
• Radu, has developed a plane finder interface

Nearest neighbor

• .

Spin features

• Tried on synthetic data and vmrl files
• Missing elements vs Andrew Johnson features
  • Using geometric constraints if occluded object
• API 3D point cloud from oct tree
  • Pass set of points into spin function
  • List of 3D locations to compute spin images at
  • Computes normals
  • Pass pts and normals to a function that computes spin images
    • Support is the radius of included points
• Memory and time hogs -- hard to optimize

HOG

• Anatoli tried HOG in a "Haar style" manner
• HOG features compared in matching set of points

Lighting

• Want to start getting in specularity and lighting direction detectors
• Train by by rendering

Stanford feature

• Sub patches from mean

3D capture code

• Start Anatoly on 3D capture

Action Items

NEW

Gary

Vadim

• Check kdd trees on small vectors

Victor

FROM LAST TIME

Gary

• Talk to Radu about 3D rotation of points and create an image
• Gary to send Anatoly stereo data for trying HOG and SPIN
  • Look in
  • http://vault.willowgarage.com/wgdata1/vol1/IKea_Related/Class
  • For file Objects_kjlKJL.tar.gz (0.5G)
• Send gaussian curvature paper
• Kurt to send paper on fast plane fit
• Histogram of Gradient link

Vadim

• Anatoly -- prioritize HOG, SPIN
  • Fast plane fit

Victor

• Set up the first OpenCV appliation: 3D box
  • Video of it working
  • Raw video that spits out marked object

2009-04-07¶

Agenda

• We need to update:
  • Accomplishments
  • Monthly announce
  • Change Logs Can we get new functions listed here?
  • Contributors credit page?
  • Update what people are working on.
• Vadim
  • Stereo block match at two scales. Progress?
  • kd_tree tests for small vectors?
  • New interface progress
  • 3D Capture -- software in? Testing?
  • Python interface based on numpy matrices?
  • Optical flow code?
• Victor
  • Code for 3D box tracking
  • Chamfer matching
    • 2D
    • 3D?
  • Machine learning reading csv
    • Other MLL improvements

Minutes

Victor

• 3D box code (put into OpenCV application section)
• Chamfer matching
  • Used in outlet matching -- works well for orange matching
    • Many false positives in high texture
    • Works worse on white on white because edges are unreliable
    • Useful for European outlets
    • Edge template better, but depends on edge fidelity
• MLL
  • a major commit to opencv svn.
    • The new memory-efficient structure for tree-based classifiers has been implemented. The new memory limitations are still under investigation (because of the recent changes in cxcore).
    • About 2X saving for old version of OpenCV, looking at new version
    • 10,000 samples, 1000 variables.
    • Doing new tests (Maria)

Application section

• Same svn link, but put opencv_extra
• For example https://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/trunk/opencv_extra/3d/createsamples3d/

Vadim

• Stereo starting tommorrow
• kd-tree on small vectors -- will send table

Activity results:

1. Converted to the new API and optimized (by factor of 2) pyramid functions.
2. Integrated Fuzzy MeanShift Tracker by Farhad Dadgostar (the same person who contributed adaptive skin detector).
3. Integrated Farnerback dense optical flow.
4. Anatoly started investigation of 3DSom software for 3D reconstruction. A brief report has been sent.

Plans for this week:

1. Purchase some lighting setup for better background isolation. We have found some good possible setups in the internet already.
2. Finish optical flow intergration (other dense optical flow algorithm and better sparse optical flow).
3. Implement adaptive-window variant of stereo correspondence algorithm
• Issues: we still have not received the license update from 3DSom.

Python

• Can create numpy array and call OpenCV function
• C-type tool in more use
• Supports overload

Optical Flow

• Forward back algorithm
• Have another dense algorithm almost ready, will integrate
• Improve sparse optical flow with color and image gradients
• About 1 week more work (delayed for stereo)

Contours and Object detection

• Need C++ interface
• Contours
  • 2 week working time (data structure issues)
• Object detection
  • Probably 1 week

HighGUI and MLL

• Can be converted in interface
• HighGUI fast
• MLL ~ 1 month

External code, improvement in

The details of the implementation can be found here:

http://www.springerlink.com/content/p32p311820r63k68/

and one of the related videos also is here:

http://www.youtube.com/watch?v=gVPCrQBKKqE&feature=channel_page

OpenCV Summer Release

1. General Structure:
   • Decisions on library structure (Google's request, separate packages as is ...)
     • Provide way to build OpenCV statically, then size varies with binary size
2. New Features
   • C++ interface
     • cxcore
     • cv -- start with filtering functions, resize, optical flow, object detection (2 weeks)
   • Better python
     • Vadim, Alex ...
   • 2D features C-descriptor, HoG Boundary Fragments ...
     • C-descriptor ... resolved issues with Michael, Vadim understands code more deeply (next week)
       • No re-implementation yet. Code based on randomized tree, but specialized version.
     • Boundary fragments ... working on focus. *