Building an autonomous, voice-controlled face recognizing drone

Total Shares

The inspiration

Our world, and the technology landscape that is helping to fuel it, are changing at a staggering pace. With nearly 20 years in the IT industry the only constant I have had to handle is change.

Many years ago I resolved to ensure I was learning on all levels and, while possible, that includes keeping my hands near the tools of the transformation. The technology!

The quote above really sums up why twice a year I try to find a side project that lets me dive deep into selective technology I have not used much previously. I do that so that I can “touch” it. That in turn enables me understand it, and how it might be useful, better.

I value such periods. The time spent on such projects helps me when it comes to my day to day job of helping steer the business and helping companies transform through the use of new technologies.

With that in mind, in the run up to Christmas, I was looking around for my next project.  It was then I stumbled across a blog post from October 2016 by Lukas Biewald. The blog post was entitled “How to build an autonomous, voice-controlled, face recognizing drone for $200“. I knew immediately I had found my next project.

Drone Intro

I selected it as my project because it would expose me more to the node.js programming language than I had ever been exposed before. It would get me more familiarity with the cognitive services APIs of Microsoft, for which I am responsible amongst many other things, and it would let me bring that all together in a fun way. Also – it would end up as a cool showcase as to what you can do with newer technologies which I thought would open up my thinking about possibilities.

My last project was one using and Arduino together with the Azure IoT Hub, Azure Streaming Analytics  and PowerBI. I was sure this would allow me to build upon, and refresh, the knowledge I had gained from that effort and refreshing knowledge is never a bad thing.

The Plan

So the first step was to get a Parrot AR Drone 2.0 as shown below. It would be the core equipment for this effort.

Parrot AR Drone
Parrot AR Drone

Turns out it was a bit more expensive in the inflated price world of Europe. It cost €240 but still reasonable. Having got the drone there was no turning back.

I have to be honest. When I read the blog of Lukas I was thinking that it would be a matter of simply stepping through and replicating what he had.  Unfortunately it turned out not to be like that so the problem solving me had to show up.

My idea to spend a week over the new year to do something cool turned into a real deep dive and a LOT of fun :).

The Reality

The journey to develop something similar to what Lukas has was an interesting one. It was a maze. I had a set of clues to move through the maze, thanks to the post from Lukas, but no step by step guide.

MazeThere were lots of things I had to deal with to navigate the maze. Thankfully I knew the destination I was headed. Knowing my goal kept me going at times of being challenged in many areas.

I had many issues to overcome including:

  • Installing Node.js on a new machine
  • Having to start from scratch with some aspects of node.js
  • Needing a UNIX/Linux Shell on windows to be able to get my drone onto my WPA2 protected network
  • Needing to install additional software such as ffmpeg, Visual Studio amongst others (and make it work)
  • Programmatically trying to do some things with Cognitive Services that Lukas had done via the online interactive site which I was determined to do via code only.
  • and more …

There is no doubt that the basics are well covered in the blog post by Lukas. There is also no doubt that his post helped guide me, and inspire me, on my journey.

Having said that there were a large number of things I needed to do which did not get the coverage and left me stuck at times searching for answers. I am a believer you need to get stuck to learn. As such I did not see that as a bad thing ultimately.

Messaging Lukas

At one stage, in my frustration , I reached out to Lukas over twitter and we swapped some messages.  It was very nice of Lukas to answer me and I hope he does not mind me building on his efforts via this and subsequent blogs. Ultimately it turned out a couple of things I was doing programmatically he had done otherwise. This meant I was on my own a little if I wanted to do it via code which was my aim. I do not get easily beaten by technology though 🙂

Microsoft Engineering

So I battled on. Being a Microsoft employee I was able to touch base with the engineering team  and got a little support from them. They even tweaked some GitHub examples based on our dialogue.  Thanks to Chris Thrasher,  Shen Zhao and Song Li I was able to get enough info to overcome some obstacles. I will share where they helped later in this series when we get to it.

The End Result

The end result is I now have everything Lukas had working in his demo fully setup (minus the nice WAV visualizations he shows in his video from the browser). I have also extended it a little to capture some of the streaming data about the drone flight into the Azure IoT Hub and visualize things via PowerBI.

My cool project is therefore coming to an end with a wealth of learning and I am now going back over it to write it up.

As Lukas mentioned, building this sort of demo results in the need for at least one or two drone repairs along the way. I feel like I have also become a drone repair engineer :). An unexpected benefit of this project.

What’s Next

Over a series of blogs I am going to go step-by-step through the entire process I followed to build out this demo. Writing this up is not my primary focus so it may take me a month or so to do it. I hope you will follow along and try it out yourself.

Using my planned step-by-step set of instructions anyone with a Parrot Drone and a Windows 10 based machine (running the anniversary edition) will be able to replicate the demo I have built hopefully with a lot less time needed. I will share EVERY line of code I created and every download or install I made.

Before we step into the guide I will admit I am not a node.js expert. I have been learning as I went along. The asynchronous nature of node.js caused me some pain in using the face apis which need specific sequences of calls to work correctly.

In my project I simply hacked around those using comments and executing in the sequence I needed. I know there are better ways, from online reading, but without going on a deep node.js course, or re-writing code (which I do not have time to do),  I did not get there.

So over the next month or so I will push out a series of blog posts on this topic so you too can build this end to end fun project.

I look forward to your feedback. Once again thanks to Lukas. Without his blog post I would not have thought of going on the journey I have. Without knowing it was possible from the start I may have given up along the way 🙂

So grab your drone and lets make it smart!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.