Using SVM with HOG object detector in OpenCV

Hi everyone! For this post I will give you guys a quick and easy tip on how to use a trained SVM classifier on the HOG object detector from OpenCV. But first, one big shout-out to Dalal and Triggs for their great work on the HOG (Histogram of Oriented Gradients) descriptor! If you still don’t know about it, it is worth to check it out.

But back to the subject: why am I writing about this, since the OpenCV already have the implementations of both SVM and HoG which are quite easy to use? Well, they may be easy to use, but they don’t work very well together. The HoG object detector may be called with an SVM classifier, but not in the format that the SVM classifier from OpenCV works. That really means that if you train a SVM using HoG features, it is not possible to use it on the cv::HOGDescriptor::detect() function.

Fortunately, this is easy to solve: we just need to convert the trained SVM classifier to the Primal Form. This can be done by first creating the class PrimalSVM, which is an inheritance from the the class SVM:

class PrimalSVM: public cv::SVM {
    public:
    void getSupportVector(std::vector<float>& support_vector) const;
};

And then, to the magical part:

void PrimalSVM::getSupportVector(std::vector<float>& support_vector) const {
   int sv_count = get_support_vector_count();
   const CvSVMDecisionFunc* df = decision_func;
   const double* alphas = df[0].alpha;

   double rho = df[0].rho;
   int var_count = get_var_count();
   support_vector.resize(var_count, 0);

   for (unsigned int r = 0; r < (unsigned)sv_count; r++) {
       float myalpha = alphas[r];
       const float* v = get_support_vector(r);
       for (int j = 0; j < var_count; j++,v++)
           support_vector[j] += (-myalpha) * (*v);
       }
       support_vector.push_back(rho);
}

Now you can use the PrimalSVM to train a classifier just like you would do with cv::SVM, and then call getSupportVector that will give you the support vectors in the format that cv::HOGDescriptor::setSVMDetector expects. And here you go! Now you can easily create an object detector entirely on OpenCV, and using only a few lines of codes :D! You may be surprised with the results that you can achieve when training with only a handful of images. Actually, I may get into more details on the process of creating an object detector in the future…

And last but not least, another shout-out goes to DXM from Stack Overflow, which was, as far as I know, the first one to propose this solution.

 

PS: For the ones with more attention to details, you will notice that the signals of rho and the alphas are not the same. This may be due to some characteristics of the (older) libSVM, which was the base of the SVM OpenCV code. I don’t quite understands this particular SVM implementation details, but I don’t lose sleep over it :P.

Writing Python wrappers for C++ (OpenCV) code, part I.

As I mentioned a couple of posts ago, we love to use C++ to make our methods to run fast. And, as many Vision engineers, we use OpenCV. However, there are a couple of things that can be tricky in C++, such as web services. Instead, for an upcoming API that we are developing (!), we decided to go with Python for the web part while not losing the performance gain of C++. Hence, there is a need to build a Python wrapper for the C++ code.

There are a couple of library options to that, but for our needs the Boost.Python is by far the best fit. The major aspect of writing these wrappers is to convert the data from/to Python to/from C++. The first conversion that it will be required is between matrices types: cv::Mat from OpenCV to/from numpy nd arrays.

Fortunately, there is a very useful lib that implements this converter called numpy-opencv-converter. However, there are a lot of these conversions that need to be implemented manually as it is impossible to predict the combination of data types one can write in his/her code.

We will begin with a simple example that uses the boost library to convert the Python parameters (tuples) to a C++ function (cv::Rect).

The definition of the method is this:

cv::Mat FaceDetection::getPose(const cv::Mat &image, const cv::Rect &rect);

Notice that, using the converter lib mentioned above, we will not have problems for the return value and the image parameter as they are cv::Mat. However, the Python code has no idea what is a cv::Rect, therefore we need a helper function to call this method.

Usually, you can export a method using boost.python as simple as this:

py::class_("FaceDetection")
       .def("getPose", &FaceDetection::getPose)

But again, if we do this without a converter (will be discussed in Part II), there will be a problem when calling this method. Now, we can create a helper function to allow the conversion to cv::Rect when calling this method. In the Python version of OpenCV, a rectangle is defined by tuples, so the helper function becomes:

cv::Mat FaceDetection_getPose(FaceDetection& self, const cv::Mat &image, py::tuple rect) {
 py::tuple tl = py::extract(rect[0])();
 py::tuple br = py::extract(rect[1])();

 int tl_x = py::extract(tl[0])();
 int tl_y = py::extract(tl[1])();
 int br_x = py::extract(br[0])();
 int br_y = py::extract(br[1])();

 cv::Rect cv_rect(cv::Point(tl_x, tl_y), cv::Point(br_x, br_y));

 return self.getPose(image, cv_rect);
}

Now, if I call this function with a tuple the system knows what to do. The only thing is to bind this helper function to the method of our class:

py::class_("FaceDetection")
       .def("getPose", &FaceDetection_getPose)

And voilà. I can call the method FaceDetection.getPose(…) from the Python (once the module is imported, of course) without any problem. This is nice and all, but you may be wondering if you have to do this kind of functions every time your data is not natively support by the boost.python. The answer is no, and it is fairly simple to create some converters for your datatype. We’ll show that in a future post, Part II.