LMDrive Dataset consists of 64K instruction-sensor-control data clips collected in the CARLA simulator, where each clip includes one navigation instruction, several notice instructions, a sequence of multi-modal multi-view sensor data, and control signals. The duration of the clip spans from 2 to 20 seconds.

Dataset details

• data/: dataset folder, the entire dataset contains about 2T of data.

• data/Town01: sub dataset folder, which only consists of the data folder for the Town01

• data/Town02: sub dataset folder, which only consists of the data folder for the Town02

• ...

• dataset_index.txt: the data list for pretraining the vision encoder

• navigation_instruction_list.txt: the data list for instruction finetuning

• notice_instruction_list.json: the data list for instruction finetuning (optional if the notice instruction data is not engaged in the training)