GoKawiilArchitecture
The black-box ML planner uses a modified version of the DriveIRL architecture1,2 (Fig. 1b). For the trajectory generator G, we use a heuristic generator that produces 143 jerk-optimal trajectories to anchor waypoints along the route. For the scene encoder H, we use the hierarchical vector transformer (HiVT)51 pretrained for multi-agent motion prediction. Apart from the scene embedding h, this produces an additional 3 trajectories for the AV, for a total of k = 146 candidate trajectories. In the scene-trajectory encoder E, trajectories are encoded using a recurrent neural network and then fed jointly with the scene embedding into a transformer layer, which produces the scene-trajectory embeddings z i . The reward model R is a multilayer perceptron (MLP). In CW-Net (Fig. 1c), the classifier C and the new reward model \({R}^{{\prime} }\) are MLPs.
We avoided testing other methods apart from CW-Net because, at the time of writing, we are unaware of other works that are capable of modelling interpretable-by-design IRL systems. Moreover, we broadly seek a general comparison of concept-based explanations (that is, CW-Net) against no explanation (that is, our control) to help generalizability of the results across the myriad concept-based explainability techniques in the literature.
CW-Net training
We used two datasets:
Dataset 1: 500,000 scenarios and 8 concept labels (Supplementary Table 1)
Dataset 2: 3,000,000 scenarios and 10 concept labels (Supplementary Table 2)
For a full list of the concept labels and their meanings, see section ‘Concept details’. Each scenario was associated with 146 trajectories, thus giving between 73–438 million training data points for the concept classifier, each with multiple concept labels. Our algorithm assumes CW-Net training has access to the original dataset used to train the black-box ML planner, along with annotated human-understandable concept labels for each of these data points. The annotations can be multi-label, meaning that one datum can be associated with as many concepts as desired or useful. The experiments with CLOSE and ASV concepts used models trained on dataset 1. The experiments with BIKE and PEDESTRIAN concepts used models trained on dataset 2.
During training, the parameters of the trajectory generator G, the scene encoder H and the scene-trajectory encoder E are frozen, and only the concept classifier C and the new reward model R′ are trainable. Two separate losses are optimized jointly.
First, a concept classification loss \({{\mathcal{L}}}_{\mathrm{concept}}\) is used to train C to predict the correct concept label(s). In our setting, this loss combines cross-entropy with binary cross-entropy for different concepts, depending on the semantics of the corresponding scenario types. For example, in dataset 1, we use cross-entropy to model the steering concepts of the car (LEFT, RIGHT and STRAIGHT), and the speed concepts (STOP and SLOW), while also using binary cross-entropy to predict the presence of other concepts such as ASV, INTERSECTION and CLOSE. These losses are then averaged into one:
... continue reading