Depth and AI with Brandon Gilles and Brian Weinstein

Welcome Brandon Gilles and Brian Weinstein of Luxonis Holding Corporation

• They make the DepthAI system, which uses the Intel Movidius Myriad X
• It was initially created for life safety problems, specifically targeted at distracted drivers.
• There is research that a car horn on a bike can help to lower fatal interactions.
• Brandon had multiple people in his life hurt or killed by cars while biking.
• Though a systemic change is needed, a tech solution is more likely in the short term.
• They built a hardware protoype and spent a couple hours of coding to prove out the concept.
• "Depth perception + CV is a cheat code", but it needs to be useful in the physical world
• No way to embed this much power
• Current depth cameras on the market
  • Intel D435
  • Intel D455
• Two camera solution senses the disparity in the images
• AI (OpenCV) helps tell what is there (Car, bus, etc)
• Intel compute sticks
• Combining depth plus AI
• There are other Myriad X SOMs
• Jay Carlson episode where he was talking about SOMs and MIPI
• 2 lane vs 4 lane MIPI
• Base boards open source on github
• Luxonis did a Kickstarter with OpenCV and raised $1.4M
• Working with vendors
• Niche that didn't exist: embedded, low cost, fast boot, performant
• OpenCV is a compiled library
• PyTorch
• DNN Module
• DARPA grand challenge
• OpenCV is a huge community
• The DepthAI has bindings for ROS/Python
• What is the Myriad X chip background?
• Intel bought the Movidius team in 2016 (Vox)
• 28 processors on the die, with a "Network on chip architecture"
• What gets access to the data from the camera first?
• It is flexible, so the software configures it
• Asymmetric fell out of favor
• Geometries of the boards for the Kickstarter
• 0.4mm pitch, 400 pin BGA
• OAK-1 has the color camera, OAK-D has the color camera plus stereo vision
• December delivery, which they are on time for
• Possible to get the boards right now at higher prices
• They are working with Arducam to try and source different camera modules.
• Lower cost overall will come from lower cost cameras
• Camera module industry is very opaque
• This is why there is a custom RPi camera
• IMX 378
• Apple and Android are doing visual depth preception on phones, including Apple's new LiDAR on the iPhone
• Luxonis did a competition around spatial AI and got over 300 entries
• Many of the entries involved "giving another sense" to the visually impared.
• Following the bald referee
• Privacy centric
• What is edge computing?
• "Embedded is barely a thing yet for AI"
• OpenMV by Kwabena Agyeman works on a ST part
• They are working on building out the team.
• If you're a PCB layout engineer proficient in Altium, send them an email. (remember: The Amp Hour always recommends sharing a portfolio of your work)
• SOM Info, including the pinout of the connector on the SOM, is on the github
• For more information, check out Luxonis.com

Many thanks to our Patrons! You can join at Patreon.com/TheAmpHour if you’d like to join the crowd. A special thanks to our corporate sponsor Binho, who now distribute the Sensepeek PCBite.