How to build Intelligent control systems using new tools from Microsoft and simulations by Mathworks

Hi, my name’s Keen Browne. I’m Head of Product for Project Bonsai. Project Bonsai is a new service from Microsoft that enables engineers, subject matter experts and others who are interested, to teach an AI to solve complex, dynamic control problems. At its heart, this new technology uses something called machine teaching. Machine teaching is a new way to express what it is you want an AI to learn and how you wanna learn it. One of the core components in this new technology is a reliance on simulation. With simulation, the AI can learn from lots and lots of data before we go and put it onto a real system. Steve is with me today from Mathworks. As part of the Bonsai service, we spent time working with Mathworks to integrate Simulink. I’d like to introduce Steve and have him tell us about Simulink and how to use that to build a simulation for a physical system. Hi, my name is Steve Miller and I’m the Product Manager for Simscape. I work at the Mathworks office in Munich, Germany and I help engineers model mechanical, electrical, fluid and other physical systems using Simscape. I’m here to help Keen create a simulation model so he can create an intelligent control algorithm. Today we’re going to learn about Project MOAB and how to build a control system using the Bonsai service that can interact with MOAB in the context of the Simulink simulation. We have a goal of helping engineers develop AI systems using their own domain expertise and knowledge. Having a model of the physical system, in this case MOAB, will make generating the training data for the AI algorithm much easier. Instead of working with just one system on a desk, you can use many virtual copies of the MOAB system. The MOAB is a mechanism that has a plate that can be tilted and the tilt angle of the plate is controlled by three mechanical linkages beneath it. With Simscape, it’s really easy. All you have to do is import the CAD model. All of the kinematic relationships, the parameters that affect the dynamics, are all integrated automatically into the model that you see here. The intelligent control system that Keen is going to develop goes in here. It’s going to figure out what the tilt angles are for the plaque. If you look in the animation, you can see we don’t have a very good control system yet. Now those tilt plate angles are not what the mechanism needs. We need the motor angles for those linkages. This inverse kinematics control algorithm will control those tilt angles to the motor angles. Those motor angles are what’s going to drive the mechanical model of MOAB. In this subsystem, you can see the Simscape model. You can see the ball, the plate, the motors and the base. If we look in the animation, you can see the linkages that are tilting the plate. It looks just like the real system. So if I needed to modify it, I would know exactly what to do. We also had to model the contact between the ball and the plate. We have to characterize this behavior very accurately. If we generate our training data with an inaccurate model, the resulting AI algorithm won’t work in the real world. To get an accurate model is why we’re using Simscape. In the real system, a camera’s going to record the position of the ball. In the simulation model, we can measure the position exactly. These measurements are what we’re going to use to generate the training data. So to use Bonsai with technology like Simulink, there are three overall steps you go through. First is to integrate the simulation with the service. Second is to train an AI to control the device that’s modeled in the simulation. Third of course, is the ability to export the brain and either use it with the device or the simulation wherever it’s needed. Today I have a Simulink model. In this model you can see that I have a block that is a controller for controlling the device inside the simulation. I can connect that simulation to the Bonsai service. Then, with the Bonsai service, I can use techniques like goals, concepts, in order to teach the kind of control system that I need to solve my problem. The first step is to run the Simulink model. You can see this here. Then, once I’m satisfied that my model’s working properly in its integration with the Bonsai service, I have the ability to zip it up, upload it, and then use that to scale training for an accelerate AI development. You can see here I have some code. Some, what we call inkling code that describes how you wanna teach the AI to control the service. When I have this code, I can click the Train button. After that, we generate an AI and train in coordination with Simulink to create a control system that has exposed the dynamics of the ball moving on the plate. We encourage you to try out the Bonsai service itself. You can find it as an Azure service, at We also have a lot of publicly available documentation. We encourage you to visit, search for Bonsai, and look at the information we have available there. Thanks for joining us today and seeing as we used Bonsai and Simulink to create an intelligent control system. (soft music)

One thought on “How to build Intelligent control systems using new tools from Microsoft and simulations by Mathworks

  1. Depending on the sensor inputs that can be developed in simulation, this kind of kinematic system could be used to relieve personnel stress in meatpacking lines, by aligning the geometry of carcass for each cutting operation.

Leave a Reply

Your email address will not be published. Required fields are marked *