An Example Processor

No comments

In previous posts I did quick introductions to the Zorroa Platform, and to the Curator, the platform's web based front end. Today I'll do a quick overview of a Zorroa Processor.

Processors are the building blocks of the import pipeline. They are basically plugins that can be written in Java or Python, and they can be extremely simple.

Processors can be used to find files to add to the database in a drive or cloud service, to group processed assets together into groups for further processing, or to add data to the asset in some way.

Let's look at one example of that last kind, which we call a document processor. The one thing I'd like to convey in this example is how easy it is to deploy a new algorithms onto a large collection using the Zorroa Platform.

Making a face_recognition Processor

We are going to take Adam Geitgey's excellent face_recognition API and make a processor that can find and count faces. This API makes it super easy to detect and recognize faces. In this example we'll just use the face detector. To find faces, all we need to do is load the input image, and call the face_locations() method:

image = face_recognition.load_image_file("your_file.jpg")
face_locations = face_recognition.face_locations(image)

To use this in a document processor, we will inherit from the DocumentProcessor class and implement the execute() method. An instance of the current asset being processed will be passed to execute(), which can add arbitrary data to the asset. It will look something like this:

def execute(self, frame):
  p_path = get_image_path(asset, self.ofs)
  face_locations = face_recognition.face_locations(p_path)
  frame.doc.set_attr("faceRec.number", len(face_locations))

In the execute() method, we first use a convenience function to get the path to a representation of the asset. This can be one of the thumbnails or the original image. Or it can be a frame of video if the pipeline is processing video. Then we call the face_locations() method of face_recognition with this file name. Finally, we add an attribute called "faceRec.number" that contains the number of faces we found.

We are done! Of course a real-life processor would probably do some more interesting stuff with face_recognition: save the locations of the faces found, or save the feature vectors that are used to recognize faces from each other into the asset. But this is basically all it takes to implement a new processor in the Zorroa Platform.

Once installed on a server, face_recognition will be available to use in pipelines as a processor without arguments. Let's see how the results of processing a bunch of images look in Curator. In the image below I made a facet selector for the attribute we created, "faceRec.number", and I'm selecting through the values.

It's a good way to detect images of crowds!