MIT 6.S191: LiDAR for Autonomous Driving

Transcription for the video titled "MIT 6.S191: LiDAR for Autonomous Driving".


Note: This transcription is split and grouped by topics and subtopics. You can navigate through the Table of Contents on the left. It's interactive. All paragraphs are timed to the original video. Click on the time (e.g., 01:53) to jump to the specific portion of the video.


Intro (00:00)

Thank you very much Alex. So I'm very happy to be here today and I hope we'll be able to cover everything that we wanted. There is about 100 pages here in 45 minutes. But we'll try. Okay, so we're very in short, in very, in very short, you know, this is developing lighters for autonomous vehicles. We're very active in automotive, trying to pursue autonomous driving level two, level three, we have several products that we offer to the market. But actually here here today we're going to focus on two main topics one of them is about what our car makers are trying to achieve in developing autonomous vehicle and how we're helping them not only with the lidar but also with the perceptions of them that's something that Amir will cover you might have heard of InnoVis through our program with BMW. We have our LiDAR here, InnoVis 1, that's our first generation that is going to be part of BMW series production for level 3 for highway driving. It's going to be used on several models, seven model series, series the ix very fortunate to be part of this program and of course many other things as i said uh today i'll cover i'll cover some topics that are coming from the lidar space and talk about possibly some standardization that is required in that space and later Amir will translate some of those requirements also taking from the perception software.

Advancements In Autonomous Driving

L4 L5 Autonomous Drive (01:27)

We have a white paper that we shared on our website. Some of the material that I'm going to cover here very quickly because we don't have time, is explained very in depth in that documents. You can find it on our website and i'm sure you'll find it interesting today most of the cars on all of the cars that are trying to do some automation of driving are at level two meaning that the car controls either the pedals or the wheel, but still require the attention of the passenger. You just probably heard about a car that a person that was accused in killing a person due to an accident in an automated driving, basically because the car makers are still not taking liability.

Available Autonomous Drive Technologies (02:40)

The quantum leap between level 2 and level 3 comes from the car maker actually taking full responsibility on anything that happens when driving and does not require the person to be attentive it obviously requires them to have much higher confidence and that comes from additional sensors and capabilities in order to reach a full autonomous driving, you need to have a good visibility, good prediction on what's everything is going on on the road. And there is an eye diagram, a certain way that you need to cover the space with different types of sensors. And LiDAR is one of them. And what you're seeing here is just one, I would say, one way to try to do it with existing solutions in the market. Someone who took a specific sensors and try to map how putting them one next to the other gives you the full visibility that is required by the system.

Stop Create System Louisiana (03:41)

There are other ways to do it. This is just one example. I want to talk about, explain first what is a LiDAR and maybe specifically how we approach this problem. A LiDAR using a laser beam that we move around by using a two-axis manner scanning mechanism that allows us to scan the scene. That light is emitted towards an object and the light that bounces back to the system, a fraction of the light is collected by the system. And there is a sphere that comes from 200 meters away and you have a certain flux of photons that are collected by the system. The aperture of the system is, you can say it's like an antenna. That's the antenna of the system. It defines how many photons are eventually collected into the system. Those photons are collimated on a detector. And of course, the sensitivity of the system defines how well we are able to react to each photon. You want to have the lowest noise figure in order to be able to detect each and every photon. Of course, photons could come from either the object or photons that actually came from the sun. The sun is like the nemesis of flydars. And that's our job to try to differentiate between them and there are ways to do it those photons that are converted to electrons through an avalanche reaction of the silicon are collected by the signal processing mechanism of course there is also self noise of the of detector itself. And it's also part of what we need to do is to see the difference between them. Eventually, light that comes back from the system is detected by the system, by the unit, and by measuring the time in which it took for the light to bounce back, we know how far things are. Now, eventually, a LiDAR is like a communication system as you might know you can define it by a signal to noise ratio the signal to noise ratio for lidars is defined by the emission because that's the transmission you you're using the aperture of the system which is the antenna the photon detection efficiency which defines the gain of the system, and of course the noise. The noise that either comes within from the system or the noise that comes from the sun. Now, we use this equation to see how we can improve our system from one generation to the other. Between INNOVIS 1 and INNOVIS 2iz two our second generation which we recently showed we improved this equation by a factor of more than 30 times okay this comes from being able to control different knobs of the system using 905 means that we are kept by the amount of laser that we are allowed to use but we there are many other measures that we do in order to improve the system significantly and today inovis 2 is is a few orders of magnitude really better than any other lighter company any system that is available i'm showing you an ex a demo of inobis 2 and this is actually also in a very early stage but already showing a very nice I would say point cloud just to get you understanding every point that you see here is a reflection of light coming back from the scene but after shooting a pulse of light too and we do that, very fast in order to cover the entire field of view.

Visualization Of Autonomous Driving (07:15)

And we can see very far away, it's very nice field of view and range and resolution. And that is how we solve the problem, which is defined by requirements we get from automakers. Now, autonomous driving could be described by different applications, shuttles, trucks, passenger vehicles. Those provide different requirements. For example, passenger vehicles on highway require a certain requirement for driving very fast, but a shuttle that drives in the city with much more complex scenarios and traffic light and complex intersections require a different set of requirements. Today, I'm going to cover this area, the highway driving. The highway driving is what we see as the MVP of autonomous driving because it actually simplifies and reduces the variance of different use cases that could happen on the road because highways are more alike and it actually narrows the number of use cases that you need to catch it shortens the validation process. Ourar can support obviously all of those applications but we see that level two and level three are the the opportunities that probably would go the fastest in the market now when a car maker is trying to develop a a level three it starts from trying to understand how fast it wants to drive because the faster the car wants to drive, it needs to be able to predict things further ahead. It needs higher resolution. It needs higher frame rate.

Interface With Autonomous Driving System (09:11)

And those are the interpretation of the different requirements from the features of the car into the LIDAR space, which this is covered in the white paper, and I'm inviting you to read it. Of course, on top of it, there is the design of the vehicle, whether you want to mount it on the roof, in the grill, the window tilt, the cleaning system. There are many things that are from practical elements require some modification for the design of the LIDAR, which we obviously need to take into account. For those of you that are less familiar with lidars, then obviously a lidar is needed to provide redundancy for cameras due to low light condition or mist, you can see here an example of, a very simple example of just some water that is aggregated on the camera.

Sensor Requirements Part 1 (10:08)

And of course, every drop of water can create the same problem and that's not, it's not okay to be in this situation. This is why you need to have redundancy. Another case is direct sun. Of course, some might say that if you have sufficient processing power and collected millions of objects, a camera might be enough. But obviously, if you're unable to see because of limitation of the physical layer of the sensor, it's not enough. You need to have a secondary sensor that is not sensitive. So today we're talking about level three. Level three requirements is defined by the ability to see the front of the vehicle mostly. And a good system is not only there to provide safety to the person, because if all the car needs to do is to make you leave after the car travel, it can decide to brake every five minutes. And for everything that might happen on the road, which will slow down, you will be exhausted and possibly want to throw up after such a drive, which means that the system in order to be good, it has to be smooth in its driving. And to have a smooth driving, you need to be able to predict things very well. And in order to do a smooth acceleration, brakes, maneuvers, and that's really what defines the requirements of the sensor. I will not go very deep in these diagrams. These are part of the things that you can find on the white paper, talking about the requirements of the field of view from cutting scenario analysis, and whether you place it on the grill or you place it on the roof, how you manage a highway with different curvature, slope, and then you have the vertical field of view that is very much affected by the height of the vehicle and the need to support the ability to see the ground very well and under drivable again I don't want to go very deep here but if you're interested to learn about those principles and how they are translated to the requirements of the liar again you can find this on a white paper and actually there is also a lecture which i gave just a month ago about this white paper it's it's possibly another hour where you need to hear me uh but uh you know i don't do i don't want you to do that twice at least um before we go to the perception area i think this is possibly something that i do want to area i think this is possibly something that i do want to dwell on eventually in order to have a good driving a smooth driving it's about predicting it's about being able to classify an object as a person knowing that a certain object is a person allows the car to have better prediction of its trajectory on the way it can actually move in space if it's a car there is a certain degree of freedom of how it can move and the same for a bicycle and a person and its velocity the higher the resolution of the sensor is it allows you to have a better capability in doing that at a longer range, because of course, the longer the person is, you get less pixels on it, less points. So the vertical resolution in this example and the horizontal resolution is key in order allow the sensor to have good capabilities in identifying objects. This talks about the frame rate, also related to breaking distance, and I don't want to spend time here. It's again, another example of why a certain frame rate is better than the other, or why this is enough. I'll let you read it in the white paper. This example is something that I do want to spend some time on, sorry. Yeah, here. Okay, so this is a use case we hear a lot from car makers. You are driving behind a vehicle and there are two vehicles next to you, so you can't move in case you're seeing a problem. And at some point, this vehicle here identifies an object that he wants to avoid crashing into and and this use case it tries to provide indication of how fast or how well the car would be able to do emergency braking assuming that you're driving at 80 miles an hour now imagine that you have an object that is very far away from you, and you want to be able to make a decision that this object is non-overdrivable, meaning that it's high enough to cause damage to the car. And basically, this is about 14 centimeters because of the suspension of the vehicle, which cannot absorb a collision into an object that is too high.

Vertical Resolution Example (15:24)

suspension of the vehicle, which cannot absorb a collision into an object that is too high. So the vertical resolution is very important because it's not enough to make a decision on an object because of a single pixel. If you're seeing a single pixel at far away, you don't know if it's a reflection from the road, a reflection from a bot.ket eye, or just anything else, you need to have at least two pixels. So you have a good clarity that you're looking at something that is tall and therefore the vertical resolution is very important. Once you're able to collect enough frames to identify that at good capability, there is the latency of the vehicle itself in terms of how slow it can eventually stop. Now, this analysis here is trying to show you the sensitivity of parameters of the LiDAR. Even if the LiDAR had twice the range capabilities, the ability to see an object at twice the range would not help me because if I only get one pixel, it will not help me to make a decision earlier. If I have higher frame rates, even once I'll see the object and start to aggregate frames to make a decision that this is something I worry about, it will only save me about six meters of decision. The time in which I collect, start to seeing the object and collecting enough frames to make a decision, it's a very short period, which I will try to say, if I will have doubled the vertical resolution, I will be able to identify this obstacle 100 meters more away. So just to show you the importance of the different parameters of the lidar is not very obvious, but are critical to the safety of the different parameters of the LIDAR is not very obvious, but are critical to the safety of the vehicle. I will let Amir take it from here. Thank you. Thanks, Omer. I wish you missed me. It took more time than I told you. Maybe while Amir is getting his screen set up, I have a quick question, Omer. How do you and the company view the evolution of solid-state LIDARs? Is this something that's also in the business radar and you want to also develop that kind of technology or you believe the mechanical LIDARs are... We have... Our LIDAR is a solid state yeah it is a solid state but we are also working on a product i didn't talk about it we're also working on a product for a 360 but as such that is you know about 10 times the resolution of today available solutions i mean the best in class 360 solutions are 128 lines. We are working on a LIDAR with 1,280 lines at a fraction of the price. We decided to step into that direction because there are still applications that leverage on a wider field of view for non-automotive or non-front looking LIDARs. And that's something that we announced just just a month ago and we will show later in the year very exciting thank you yeah okay so so thanks once again um so now now we'll speak uh about how we take this um oem requirements and specification and actually build a system to support this reception system. So first, before we dive in, I would like to speak about, I think, the most obvious but yet most important characteristics of Point Cloud, of LiDAR, and that LiDAR is actually measured in 3D, the scene around it.

Measuring Collision Relevancy in 3D (18:54)

It means that each point represents a physical object in the scene. So it's really easy to measure distance between two points. It's easy to measure the height of object. It's easier to fit planes to the ground, for example. And I want to take you through a really simple algorithm and we'll see together how far can we go just with LiDAR and really simple algorithms support much of the requirements Omer mentioned before. So the simple algorithm, the essence of this simple algorithm is detecting or classifying the collision relevancy of each point in the point cloud. So in this visualization, you can see pink points and green points. The pink points are collision relevant points, which you don't want to drive through them. And the green points are actually drivable points, in this case, the road. So the most simple algorithm you can come up with, it just takes each pair of points in the point line. And if they are close enough, so just measure the height difference between these two points. And if it's greater than 40 centimeters, like Omer says, you can just classify these two points as collision relevant. So here, this turnover track is easily detected as collision relevant. And the car won't drive through this track. So while I'm talking about deep learning network, it's really easy to, it's really hard to generalize deep learning networks to new examples, to new scenarios. So you can have a really good deep learning networks to new examples, to new scenarios. So you can have a really good deep learning network that detects cars, trucks, pedestrians, whatever. But then you get this Santa on the road. And it's not trivial to generalize and to understand this Santa as actually an object that you want to avoid and not just a background. And with Point Cloud, with this really really simple algorithm uh this task become really easy uh another example just is fire trucks uh maybe and ambulances and other um other cars which are not represented um sufficiently in the train set and And you probably heard about accidents that might be to similar reasons. And another, but related case is completely different scenery. I mean, most of the data tends to be from places like North America or Europe, but then you can end up in India, a city full of rickshaws, and you just want to make sure you never collide them. So again, with LiDAR and this really simple algorithm I described before, this problem still exists obviously, but it's suppressed and it's under control once you can actually measure this in. So now let's look a little bit more complete example. Maybe some of you recognize this video from one of the Tesla crashes. So you can see the white Tesla actually crash into this turn of the trucks. So there are many reasons for this crash. Some say it's lighting uh other maybe uh because uh um but during training uh the the network never seen a turnover truck so it might be a problem um but but as as honor says and and as i i mentioned before with lidar uh this this whole accident would be would be avoided so you can see here, this is how it would look in LiDAR. So the truck is easily detected from really, really far. And the car would never actually crash this vehicle. And this algorithm is the same algorithm as I described before.

Detecting Multi Class Objects (23:01)

So really, really simple algorithm that actually makes all the safety critical scenarios much more solid and under control. So maybe some of you guys saying that detecting this huge truck is easy. So here's a different example from our action IDOT. This is the end of this one. Look at the distance. Two, I don't know if you guys can see uh a a tire uh this distance this is a tire um so this same really simple algorithm just looking at two points uh one above the other can easily uh detect this tire as a as a collision relevant so what the car actually sees uh the car just take the collision relevant point and maybe project them on a X, Y plane, like on the ground plane or something, and use this information to navigate through the many other obstacles in the city. And this is just the close up, just so you can see this really, really a tire and actually a pallet next to the tire that cannot be seen from distance. But as Oma mentioned before, it's not enough. I mean, get a good understanding of the static object in the scene, small obstacles, big obstacles, it's really important and safety critical but it's not enough because eventually we want to understand where these objects are going maybe they're moving in in our velocity so we don't need to do anything just be aware or maybe they're going to enter the ego lane so we need to break so so still detecting an object as an object is really important so let's take this example uh just just pedestrian detection this is actually a pedestrian captured by by the lidar so i and i think everybody everybody would agree this is this is an easy example right um it expects it from every average network to say uh this is actually pedestrian and classified as a pedestrian but what about this this example? I mean, here it's not obvious anymore, right? I mean, maybe you can see legs, a little bit of head, torso, but it's not obvious. But still, I think a good trained network or system can still say this is a pedestrian, giving it's around here, maybe a little more context. So here again, it's to to be detected and classified this position but what about these two points these two points really distant points um so now now our vehicle is moving really fast and we want to be super aware of any anything even even if it's high distance so what what what can you do what do we expect do we expect from deep learning network or to look at the appearance of the object? It's really, I think everybody agrees how to say this is a pedestrian. But with LIDAR, luckily, it's not critical. I mean, we can still classify or cluster these two points as an object because we know they are close by. And if we two points as an object because we know they are close by. And if we classified it as an object and now we have a bounded box, we can keep track and estimate all the attributes like velocity, shape, position obviously, all that needed for the car to predict its motion and act accordingly. So taking this simple, really simple clustering algorithm and putting it on real scenario, like a normal highway drive.

Scene semantic array (26:34)

So it would look roughly like this. You can see many, many clusters of actual cars and objects around with zero scene semantic. But since we don't have the scene semantic, you would also see other objects which are not relevant necessarily for drawing, because they're not moving, classified or not classified, detected as objects. But if our tracking system is good enough, we would say, OK, this is an object. classified or not classified detected as objects. But if our tracking system is good enough, we would say, okay, this is an object. I don't know what is it, but it's stationary. So just don't drive into it, but don't worry, it will never cross your lane. So you can go really far with these two simple algorithms, just PowerPoint collision relevant and clustering. You can really go far with perception task force and driving. But now the question, is it enough? I mean, is it enough to really create a system which is robust and useful for the app-level stacker? So here's an example where this cluster mechanism is not perfect. What you see here, you see in blue is actually deep learning network, the first time showing deep learning results. So this is, the blue is deep learning network detects this whole object as a truck. But unfortunately, the cluster mechanism actually split it into two different objects and reported if we use just the cluster mechanism, we would report it into two different objects and reported if we use just the clustering mechanism, we would report it as two different objects. So you can imagine this ambiguity or instability of the clustering mechanism actually make it a little bit harder for the upper layers of the stack to get a good understanding of what it's in. And if you're not classified as a truck, so the motion model is not clear stack to get a good understanding of what it's in and if you're not classified as a truck so the motion model is not is not clear um and and again the upper layer of the autonomous deeper stuff as truck and autonomous stack can't be sure which uh how how this uh object will behave objects will behave. So semantic is still important and still critical for this full system. So now let's see how we can do deep learning on point cloud. So first thing we need to decide, we need to decide how should we represent the data. So now point cloud as it sounds is just a set of points. So the first thing we need to understand while now point cloud as it sounds it's just a set of points uh so the first thing we need to uh need to understand while processing point card is that it's unstructured it means if we took all the points of of on this car and order it differently in the memory it will still be the same card still be the exact same scene uh so uh there are actually uh deep learning architecture which take advantages take advantage of this, like a partner at a particular class. But for sure, this is not standard and we need to make sure we understand this before processing the data. Another characteristic which is important and we need to consider is the sparsity of point cloud. If you're looking at point cloud at the cartesian coordinate system and this visualization from the top so you would see that most of the points are concentrated in in the beginning of the scene because we sample the world in spherical coordinate system so this this again challenge some of the edge, computational efficiency is a key element and sometimes actually defines the solution. So it's really important to make sure your algorithms are efficient. And as a presentation now, which is structured, okay, like images is front view.

Visualizing and exploiting the 3D properties of LiDAR point clouds (30:44)

So you can see the camera image just normal camera image above and below a point cloud which are projected on the on the ladder itself so it looks like uh it looks like an image uh the only difference is that each point here has some has different attributes it has the reflectivity as you can see here but it also has um it also has the the xyz position relative to the sensor um so now now the data is structured and we can apply uh many legacy networks that yes you are aware of or been made aware of during the schools and leverage from from a lot of legacy but but now it's a little bit harder to exploit the 3G measurement characteristic of Point Cloud. For instance, even though we know this car and this car are roughly the same size and same shape roughly, while looking at it from a front view, it's a little bit hard to use this advantage. Now we get a lot of scale per distance and this is a kind of increasing data set that we need we need to use in order to have a good generalization but it's a useful representation another representation which is also common while processing point cloud is voxelization. So if we take all the volume we want to process and predict road users or estimate road users location and we split it into multiple smaller volumes like you can see here this is the visualization I'll try to give here. And in HVoxel, just put a representation of the points in it or its surrounding. Then we can get, again, a structured representation of the point column. And in HVoxel, we can start with really simple representation like a Pupency, whether it has points in it or not, or we can go for much more complex representations like statistics, even small networks that actually estimate the best representation of this voxelization representation. So this is the voxelized map looking from the top. Okay, so it's an image. And each voxel is represented by the reflectivity of the center point. And I put here just some anchors so you can associate these two pictures so by the way this is really coarse representation mostly it's much better so you can see how the network might see but now we lost maybe a key information that we have if we look for at the point cloud from the front view. Now it's a little bit harder to understand which object includes which object. For instance, this break in the guardrail, it's a little bit harder to understand that this break is actually due to this car and not just due to a break in the guardrail. So in order to do this again we need to build a network with greater receptive field and a little bit more deeper network to get a deeper understanding of the scene and sometimes I said before we want to avoid want to still be as efficient as possible. But once again, likely with point cloud, it is still easy to get all this occlusion information. So what you can see here, this is an example of an occlusion map. So all the white is non-occluded point cloud. So if you take this clustering mechanism and just color all the free space, you would get this occlusion map. So you can add this occlusion map as an extra layer for the network so it has this information and events, and you don't need to create really big, fat networks in order to understand these occlusions. So now after we know how to represent data with deep learning, we pick our favorite architecture. The question is, what are the key elements? and we picked our favorite architecture.

Capped thorop (35:08)

The question is what are the key elements we wanna achieve with the full system? So if we take the cutting use case Omar mentioned, we wanna make sure this motorcycle is not in our lane. We wanna make sure we never crash into this motorcycle. So first we need to detect it with the LiDAR and luckily our LiDAR is good enough for detection of this motorcycle so first we need to detect it with the lidar and luckily our ladder is good enough for detection of this motorcycle it has a large enough field of view and then you want to put a bounding box around it right so you can tell to the car where it's located where it's going you can you can track it but now it's really critical that this bounding box is is really tight on this object right because if we just miss

Safety In Fusion Of Streams

Two streams fused (36:12)

by a few centimeters in this case to the right uh we might think that this this motorcycle is actually not in our learning there's no problem because you can just keep going and it might crush crush the motorcycle uh so we want to get really good accuracy uh with bounding boxes detection um so if we with bounding voxel, with output detection. So if we take this voxel to the presentation of the field of view, now we have a problem because on one hand we want to get really dense and really fine grid in order to be much more accurate and to reduce the ambiguity between the center of the cell and the actual object but as I said before it is computationally expensive so we want to still find a way to work with reduced representations but it's shield this this this accuracy so a possible solution is a fusion between the deep learning approach and the classical approach.

Clear wins (36:41)

Leverage the best from each approach to create a solid object list for the alphabet. So this deep learning stream gives you the semantics. It can say roughly where the object starts, where it ends. Is it classified object? Is it the car, is it the pedestrian, motorcycle? And the clustering stream actually gives you the accuracy that you need in order to drive the car safely.

Safety Critical Systems (37:17)

So this is an example of how it looks. So again, in blue, this is the deep learning, and in white, this is the clustering. So you can see the deep learning. No, this is a car, and actually put a bounding box around the car, but it's not accurate enough. It's a few centimeters off, and these few centimeters are important for the safety critical objects, objects which are close by. And the clustering actually really good, it fits really good the object itself. So once we did this, we actually gained one more, one more thing, which is again important in safety critical systems. And this part, the clustering, the clustering path and the fusion is fully interpretable path. And it's really helped to get to root cause of problems and look at the system as a white box. So you can understand exactly what it does. And in some cases, this is important. It's really useful that you have a safety path which is fully interpretable so this is how it all kind of uh adds up so this is uh the deep learning output you can see the bounded box you can see them a little bit shaky not all the objects are fully detected all the time this is the clustering output so you can see it's solid, but you have many false positives and the object length is not predicted and you don't know which object is it obviously. And this is the fused output. So you can kind of get the best from everything. You have classes, you have binders which are pretty solid and it's really helpful for again for the upper left at this time so i know i don't move fast because i didn't have much time but this is it thank you very much thank you so much i'm trying to answer some questions in the meantime.

Clustering/property for fmcw (39:20)

If you want, I can answer some of them. I think that maybe one thing that is important for me to add, because there was a question. Innoviz is not developing an autonomous vehicle, meaning we're not developing the driving decision, we are developing the lidar and the perception software which allows car makers to have a more seamless integration. Assume that the processing unit of the car maker has a certain input from the camera, an object detection classification interface, and they are just getting another one from another sensor. And you can imagine that they don't really care if it's a LiDAR or not. All they care is that that secondary interface, which tells them where things are, is in redundant to the other and gives them higher confidence in certain conditions so we're not developing so we are not doing driving decision but we are aiding our customers um do you want to ask specific questions you want me to go over questions that came up and you know maybe choose one of those oh yeah either either one is perfectly fine. Or if anyone else has other questions, feel free to just jump in and ask. Sure. Someone asked me about weather condition. Although it's less related to perception, maybe anyway, quickly on that, rain is actually not very affecting lidars because a drop of water is almost meaningless in terms of how much light is reflected back when you're you know meeting a drop of water in mid-air and even if very with very dense rain it's it's only reducing possibly a few percentages of range fog is like an extrapolation of rain imagine a very large volume of particles that each one of them by itself reflects light back so it creates attenuation of the light going through it doesn't blind it completely but it does reduce it quite significantly depends on the density of course there was a question here let me just check um so when we someone asked about uh false positive etc or actually there is another question i prefer someone asked me um what's uh how what what makes our lidar possibly uh a better fit to this application compared to others. So beyond of course, the obvious of cost and size, which I think are important for automotive, if you would follow the white paper, you would see that there are really, there is a trade-off between different parameters. It's very important not to fall in love with only one, because just, again, we talked about range and as example just seeing like doing a lidar that sees one kilometer with a single laser pointer is obviously you can say you have a lidar that sees one kilometer and you can probably uh spark it and and raise a lot of money but eventually it will not help autonomous vehicles so there is there are many parameters and i think what what innobis is doing uh well is the private system that has a very nice working point and tradable meaning that we can actually we we can trade between parameters but the working point the overall snr that we have in our system is significantly higher which allows us to meet all of the parameters that we show in that document, including resolution, frame rate. It's not only resolution, it's also frame rate, it's also field of view, and of course, range. So it's not, and of course, there's the pricing.

Classifiers (43:18)

So that's, I think the white paper explains it probably better than me. There is a question here on classifiers amir maybe this is for you is it possible in theory to rig the loss function of the classifier to be more or maybe that was a joke actually sorry i could there are two questions one is the first positive and the other one is training i can can take two of them. So let's start with the training. I think we have two major concerns in training. One which is related directly to the training is sampling, the most beneficial samples for annotation. I think like, especially in autonomous driving, especially on highway, most of the scenes, especially in North America and Europe, most of the scene is just identical. So we won't get much from just sampling random, random frames for the training. And we actually built a system of active learning. And I've described it in previous talks, so you can look it up at night.

Additional questions in the chat. (44:34)

So this is really like a key element at Innoviz. Like make sure- There is a question here. Oh, sorry. Make sure you say- Do we have another one? I'm sorry. Sorry, yeah, I think there might be a little bit of lag. But yeah, maybe we have time for one more question if there's one more. There's someone asking a question here about the different types of LIDARs, FMCW and time of flight. And it's actually there are there are different camps in the lidar space you have the the wavelength camp 905 1550 that's kind of a big kind of discussion and then you have uh the laser modulation whether it's time of flight and fmcw and i think other than that you have the scanning mechanism, like whether it's mechanical, solid state or, I don't know, optical phased array. So those are primarily the three main kind of branches in the market, starting with the question of FMCW and in time of flight. So the only benefits proclaimed by the FMCW is the ability to do direct measurement of velocity, meaning you modulate the laser in a certain way that allows you to measure both range and velocity by measuring Doppler, very similar on how you do it with radars. The only thing that the disadvantages comes with the need to use 1550 and again, very expensive. And there is a very strong coupling between the range, frame rate and field of view. So the trade, the working point there is quite limited. So FMCW systems can reach around 200K samples a second. InnoVis 1 is about seven mega samples per second. And InnoVis 2, it's even significantly higher. And it means that when you need to trade between resolution, number of points, frame rate and field of view fmcw mostly is using a very very narrow periscope kind of lidars because of that limitation and eventually measuring the the velocity of the vehicle in fmcw is is only possible in the longitude uh vector because you're measuring velocity in the longitude vector, because you're measuring velocity in the vector of the light direction.

Velocity Perception And Exposure

Velocity perception. (46:36)

You cannot measure velocity in the lateral, which is as important. So the need to calculate velocity is there anyway. With time of flight, you can calculate velocity very nice. If you have very high resolution and high frame rate it's not less well and eventually when it comes to the trade-off between parameters definitely resolution range field of view frame rate comes on top of the requirement for velocity and seeing probably tens of rfis and rQs in the market, I haven't seen yet anyone asking for velocity really.

Innoviz exposure (47:41)

So the value there is I think very limited and comes with very high cost. Excellent. Yeah, so thank you both so much and maybe one more quick question. I know car makers are probably your primary customer but i was wondering do you also sell your sensor to others beyond the car makers for example academia and universities doing autonomous driving research you know someone yeah sure we do uh and we're happy to work with teams that are trying to innovate.

Concept Of 3D Word

3D Word (48:17)

And of course, we can talk about it after this session. Of course, of course. Yes, of course. I mean, we work with construction companies, smart cities, surveillance. I mean, look, today, in every corner of the world, you have a 2D camera somewhere. We live in a 3D world. Anything you might ever want to automate, you would prefer to use a 3D world. Okay, anything you might ever want to automate, you would prefer to use a 3D sensor. It gives you much faster capability, ability to exercise an application. I'm sure LiDARs would be in the same position in several years from today. Excellent, thank you so much. And thank you both for your presentation today.

Great! You’ve successfully signed up.

Welcome back! You've successfully signed in.

You've successfully subscribed to Wisdom In a Nutshell.

Success! Check your email for magic link to sign-in.

Success! Your billing info has been updated.

Your billing was not updated.