Ray Perception Sensor Component Tutorial
A Penguin with a RayPerceptionSensorComponent3D. It is using 5 spherecasts to detect the orange fish in the water.
RayPerceptionSensorComponent3D is the new way to use raycast observations in Unity ML-Agents as of version 0.12. Good news; it's very simple to use.
If you are converting an existing project to use the new component, you'll need to do a couple things first:
Remove the RayPerception3D component from your agent
Remove all code relating to RayPerception in your CollectObservations() function
Figure out how many vector values RayPerception was contributing to your Vector Observation Space Size and subtract that number in the inspector for Behavior Parameters
RayPerceptionSensorComponent automatically adds observations to your agent, so you no longer need to add any RayPerception code to CollectObservations(). This makes things a lot easier!
Here’s how to configure a RayPerceptionSensorComponent3D:
Add a RayperceptionSensorComponent3D component to your agent
Increase the size of the Detectable Tags list to account for the tags you are interested in
Add any tags that you want your agent to see.
Configure the rest of your parameters as desired.
A sample RayPerceptionSensorComponent3D from the Penguin tutorial.
Same image as above.
Rays Per Direction specifies how many raycasts will be performed on either side of the center line. By default, there will be at least one raycast that points straight forward. The penguin has a value of 2, which means there will be 5 total raycasts.
Max Ray Degrees specifies the angle at which to spread out the raycasts. A value of 60 degrees will result in rays spread over 30 degrees to the left and right of the center line. The penguin’s rays are 15 degrees apart because there are 4 spaces between the 5 raycasts and 60 / 4 = 15.
Sphere Cast Radius specifies how large of a sphere to cast along the ray. Rather than using a thin line for detection, anything within this radius of the ray will count as a hit. The penguin has a sphere cast radius of 0.5 meters and you can see the spheres in red when a fish is hit.
Ray Length specifies how far to project each ray for sight. The default of 20 means it will not see anything beyond 20 meters.
Observation Stacks specifies how many previous observations to stack together. The default of 1 means only look at the current observation. The penguin only uses 1.
Debug Gizmos allow you to specify the hit and miss color as well as whether to Use World Positions. You can see in the penguin image above that the red rays hit fish. The white rays didn’t hit anything.
You can also modify the Vertical Offsets so that your agent can see at an upward or downward angle. The penguin isn’t using this feature.
Start Vertical Offset means the ray will start at some offset along the Y-axis (i.e. above or below the center of the agent).
End Vertical Offset means the ray will end at some offset along the Y-axis (i.e. above or below the horizon).
If you haven’t already seen the ML-Agents Penguin Tutorial, make sure to check that out!