The Voice-Controlled, Face Recognizing, Drone Journey – Part 5

Total Shares

Introduction to the Drone Journey

Part 5

This post is the sixth post in documenting the steps I went through on my journey to build an autonomous, voice-controlled, face recognizing drone. There are 5 other posts building up to this one which you can find at the end of this post.

Focus of this post

In this post I am going to pick-up where we left off and look at :

  • How we can make use of the Microsoft Cognitive Services Face API to recognize a specific face.  We will explore one of the approaches to doing that which reflects the journey I went through in this blog and the next blog will look at another approach.

Face API – similar

In looking at how to do the face recognition I have to admit that I ran into a few issues initially with the “identify” function which is part of the Face API.  I tried in vain to make it work but having go stuck at a certain point I switched to the “similar” function.  I eventually worked out “identify” and will cover that next but I wanted to share how I did it with identify too.

As we will be developing a solution here create a file FaceSimilar.js which you will build your code into. To ensure we have the relevant node packages start out including the correct required packages as shown below and also connect into the correct cognitive service using the api key you have for the Face API we created previously.

var oxford = require('project-oxford');
var apikey ="INSERT YOUR API KEY HERE"
var client = new oxford.Client(apikey);

Once you have that we can shift focus to the similar function of the Face API. You can think of similar as being a function that looks at a face and takes a guess at the likelihood that it is a specific stored person or group of people.  This might be useful when you are not looking for a specific person but, for example, looking for someone from a known team or a watch list with one simple check.

When using “similar” you need to go through a numbers of steps to set things up first.  These are:

  1. Create a faceList which is nothing more than a named collection of faces you are looking for.
  2. Use the function addFace  to associate the faces you want to look for to the facelist.

Because of the steps I switched to defining functions I could call in node.js. It will become clearer why shortly. I am not going to really explain functions in detail here as I am not trying to write a node.js guide.

Creating a facelist

To create a facelist we need to use the “faceList.create” function on our client object we defined earlier.  the faceList.create function essentially takes 2 parameters.

The first one is the ID for the list you want to create and the second one is the name. Both are string objects. in its simplest form it looks as shown below with “facegroup” being the ID and “Mark Torr” being the name.

  client.face.faceList.create("facegroup",{
    name : "Mark Torr"
  });

Obviously you could call this several times to create different lists with different faces in it which was how I was thinking. One facelist per person with a number of faces of that person stored in it. I then packaged that up into a function so I could call it better and I added some error handling (as if the list already exists you get errors).  that left me with this code.

// Function to create a new facelist to add faces to
function createNewFaceList(ListIdName,ListName){ 
  client.face.faceList.create(ListIdName,{
    name : ListName
  })
  .catch(function(e) {
    console.log(e); // "Something went wrong !"
   }).then(function (){
   console.log("Created Your FaceList");  
  });
}

You can now call this with a simple function call such as createNewFaceList(“facegroup”,”Mark Torr“);  from within your code.  At this stage all you have is a container to put faces into. You likely only need to run this the once which is why having it as a function is good as this helps us avoid the asynchronous traps of node.js later using comments rather than having to work on more complex approaches.

Using addFace to get the faces you want to find into your facelist

Now that there is a container facelist for you to store faces into you can use the faceList.addFace function to get images into your facelist. The call is again pretty simple

   client.face.faceList.addFace("facegroup",{
    path : "Mark.jpg",
    name : "Mark Torr"
   });

As before you could call this several times to include many different images of the person (in this case me). This would hopefully increase the chances of me being identified.  As we will need to call this, normally, many times to build multiple lists I again put this into a function and ensured that we have error handling in place.

// This function is inserting faces into a created facelist
// parameters are the facelist group name, the location of 
// the image and the name of the person
function includeASearchFace(groupname,imgpath,personname){
   client.face.faceList.addFace(groupname,{
    path : imgpath,
    name : personname
   })
   .catch(function(e) {
      console.log(e); // "Something went wrong!"
   }).then(function (){
      console.log("Face Added");
   });  
}

With that in place we can make a simple function call now to add a face once we have a face list. It would look a little like this  includeASearchFace(“facegroup”,”Mark.jpg”,”Mark Torr”); and you can see how this would be simple to call to populate many different lists and add many different faces.

When do we start to look for faces?

So far all we have done is setup the container(s) and add the face(s) into those containers. To look for the faces we have to use a combination of face.similar function (which is new) and the face.detect function which we already covered and which you have in your FaceTest.js file.

In short it works like this.  First you take the face image you want to examine. You then run face.detect on that which generates a faceId. Using that faceId you then make a call to face.similar sharing the name of the list you want to look for a similar face.

I again created a function for this. You can see it below.

It takes a faceId and face list name as its parameters as that is what face.similar needs to function.  It then calls the face.similar function. If there is an error it handles it gracefully.

If there is no problem it then looks at a returned value, which is the confidence that the backend cognitive service has that the face I provided (coming from the face.detect function), confidence and if it is above 50% (0.5) then I say that it is a good match and thus is most likely a person in that group (which in this case is only Mark Torr) and if it is under 50% (0.5) then I am going to say the match is not good enough so no match was found.

Essentially what I have done is say that if the service thinks there is a 50% match you have found Mark Torr

// This function is detecting if a specific detected FaceID is in 
// a given EXISTING facelist group
function findKnownFaces(DetectId,searchFaceListName){
  client.face.similar(DetectId,{
    candidateFaceListId : searchFaceListName
  }).catch(function(e) {
     console.log(e); // "oh, no!"
  }).then(function (response) {
   if response[0] != null) {
        if (response[0].confidence > 0.5) {
          console.log("Good Match - Found you");
          Console.log(response);
       } 
   } else {
     console.log("Poor or No Match");  
    console.log(response);  
  }
  });
}

So of course the last step is to go ahead and build a wrapper function that I can call passing it an image and a facelistname I wanted to look for. My thinking here originally was that I could use a Web Form or Spoken Text to determine the facelist to compare to. As a result I ended up with the following function call to trigger face.detect which in turn triggered face.similar (as then we have a faceId to pass)

 function runAll(imageFileName,searchFaceListName){
  client.face.detect({
      path: imageFileName,
      analyzesAge: true,
      analyzesGender: true,
      returnFaceId : true
   }).then(function (response) {  
      DetectId = response[0].faceId;
      findKnownFaces(DetectId,searchFaceListName); 
   });    
}

We can then go ahead and plug all this together with a few calls at the end of your file:

var searchFaceListName = "facegroup";
createNewFaceList(searchFaceListName,"Mark Torr");
//includeASearchFace(searchFaceListName,"Mark.jpg","Mark");
//runAll("Mark.jpg",searchFaceListName);

As you will see the last 2 lines have the // characters in front of them. This essentially comments them out. The Face API is specific in the order actions need to be taken. Node.JS, on the other hand, is asynchronous. This means if you have all 4 lines uncommented Node.JS starts all 4 running. This causes issues sometimes as things get called in the wrong order. My solution, while not elegant, was to comment things in the file and then simply to run what I needed, come back to the file and comment other things to get the run order I needed. I know there is a better way 🙂

So on the first run I would have the last 2 lined commented out. If I save and execute then I get this output.

Face Similar 1

On the second run I would comment out lines 1 and 3 . If I save and execute then I get this output.

Face Similar 2

On the last run I would have the 2nd and 3rd lines commented out forcing the execution of the runAll function (passing it a local image file) which in turn is running face.detect and face.similar.

Face Similar 3

On the third run you will get back the persistedFaceId which relates to the discovered face in your facelist that has been most closely matched and you also get the confidence. As we used the same image to search as we put into our facelist the confidence is 1 or 100%.

We can now send in a image of someone else by changing the call to runAll as shown below:

var searchFaceListName = "facegroup";
runAll("Joe.jpg",searchFaceListName);

Which gives us the following output.

Face Similar 4Essentially the second image which we passed in was of a different person totally and as such the output was that there was no-one similar.

But … Mark – using comments is a hack

I agree.  Using comments is not the ideal way to do this. There are many other ways if you want to nest call backs and work using more advanced approaches. The truth is the order we need to control is most often during setup so it is not too much of an overhead. In this case using comments is a simple way to enforce the order we need to make sure the containers are setup in the right order before we use them.

In your final project you would mostly only make the call to runAll which in turn would use only face.detect and face.similar. Given I had this approach working I did build it into my DroneWebServer.js file so the drone would land when seeing me but I knew this was not the best approach so I basked in the small victory and moved on.

Where are we and next steps

We have now covered one approach to using the Face API, the find.similar approach.  You can already embed this into your DroneWebServer.js by moving the functions over and then making the call at the right point in time.

Then, instead of writing Good Match, in the findKnownFaces function you could call the land() function to cause the drone to land as you do currently in your express application with a click. This is in fact what I did first.

Ultimately, while I could do cool stuff with face.similar, I knew that was not the right approach. I needed to work out how to make face.identify work and the next blog focuses on that approach, which is very powerful, but also a little more complex.

I hope you are enjoying the journey. Much of what we are covering works without the drone as well, so could be applied to MANY use cases easily, but by now I hope you see the drone itself makes the project much more fun 🙂

Previous Posts In This Series

 

Leave a Reply

Your email address will not be published. Required fields are marked *