The Voice-Controlled, Face Recognizing Drone Journey – Last Installment

Total Shares

Introduction to the Drone Journey

Final Installment

This post is the eleventh, and final, post in documenting the steps I went through on my journey to build an autonomous, voice-controlled, face recognizing drone. There are 10 other posts building up to this one which you can find at the end of this post.

Focus of this post

In this post I will share a video of the complete end-to-end demo and share details of the architecture which sits behind it. I will also share information on what I bought/used to bring this all together and relist all the different software, services and node packages in a single place.

Pulling It All Together

A lot of what we have been doing with this project is humanizing the way we communicate with machines/computers/things. That means talking and observing to drive intelligent interaction rather than using a mouse, keyboard or touch screen.

Our Autonomous Voice-Controlled, Face Recognizing, Drone is a smart drone which showcases, albeit crudely, how interaction with services filled with intelligence is going to evolve. It highlights the importance of cognitive services to the success of organizations in the future.

So with that said take a look at the entire end to end demo in the video below.

Reviewing The Architecture

Below you can see the high level architecture for building out this drone project (if you click on it you can see it full size and with a better fidelity).

 

The core is, of course, the computer in the middle which plays the role of the server (I could have put this into Azure as well if I wanted to. There are great hosting options for that).

On that computer we have node executing and we have built an express web application which acts as a web server listening on port 3000. That web application is serving up the web client we are looking at in the browser (in the most part on the same computer – localhost – although that need not be the case).

We are then interacting mostly with that web client. We are using the microphone and the mouse local to that web client to get input we need. The speaker we are using is actually on the host computer which means that if you were not on localhost you may not hear sound. You can adjust how the sound is handled so it is presented via the browser too but I did not get that far.

To the left we see the Drone itself. It has a camera plus a whole bunch of sensors. We are interacting with the drone via the ar-drone node library. We are both sending commands (takeoff, land, take pictures) as well as receiving data (battery percentage, altitude, pictures). All that back and forth is handled by the host computer.

To the right we can see Microsoft Azure. We are using a variety of services including Cognitive Services, IoT Hub, Stream Analytics and PowerBI for various tasks.  Telemetry data is fed into the IoT Hub from the host computer (which is acting as a gateway through which the drone sends its data) and that is in turn grabbed and interpreted by stream analytics which then makes it available to PowerBI.

From the Web Client we can then open the PowerBI interface and see the data arriving.

The components and the cost

Building this project was not that expensive. We actually spent nothing on the software side of things!

Indeed, with a little luck, you can do it for just the cost of the drone itself running at €240 or less if you find a second hand one on eBay.

What I will say is that you will very quickly get frustrated with the short battery life, that you get from the shipped battery, and you will fly your drone into walls and furniture causing damage you need to deal with. Below are some pictures of some of the damage I did to a motor and the blades.

Broken Motor
Broken Motor – See the Glue?

Smashed Propellers Largest Cause Of Drift

In my case I bought a few more things along the way in terms of replacement parts and accessories. In the interests of showing you the full costs the information is below.

  • AR Drone Parrot 2.0 Elite Edition with GPS Flight Recorder – €240
  • Long Life Battery for Drone €32,49
  • Two sets of replacement propellers for the Drone. (Busted propellers are the biggest cause of drone drifting) – €13,98
  • Tools to mount the propellers (not really needed) – €12,99
  • Replacement motor – €34,89

So there you have it. The TOTAL COST of this project was  €334,35 plus all of my time.

The cloud services, software and packages I used

To build this out I used a collection of different software, services and node packages. Below is a complete listing. Everything I used is free unless you go beyond certain usage thresholds which I did not in building this demo. There really is no reason for organizations to not be experimenting now.

Software
Node Packages
Cloud Services

Conclusion

This project has proven to be a very interesting one to do taking me on a wild technological journey. I have emerged from the project having 100% met my goals to better understand the technology.

I also now have lots of ideas as to how this technology could be applied to modernize the services organizations are delivering today so that they are ready for tomorrow. I will for sure look to write a blog focused on what I am calling the “Humanizing of the computer interface” and why this is important for the future of all organizations.

It was impressive to me that I could do this entire project using the trial/low volume services of Microsoft Azure meaning I did not need to spend anything. This is not science fiction. It is there today and you can already start to build up the skills and try build out new services especially when trying costs nothing!

Lastly – I want to again come back to two things. Thanks to Lukas for inspiring me with his post to go on this journey and thanks to Chris Thrasher, Song Li and Xin Dong for helping me at certain times.

I hope you have enjoyed this series of posts. It has been great fun to pull it together although a pain to document 😉

Previous Posts In This Series

Although I work for Microsoft these thoughts are 100% my own & may not reflect the views of my employer.

Leave a Reply

Your email address will not be published. Required fields are marked *