142 lines
6.4 KiB
ReStructuredText
142 lines
6.4 KiB
ReStructuredText
#######################
|
|
Waymo Example
|
|
#######################
|
|
|
|
In this example, we will show you how to convert a small batch of `Waymo <https://waymo.com/intl/en_us/open/>`_ scenarios into the internal Scenario Description.
|
|
After that, the scenarios will be loaded to simulator for closed-loop simulation.
|
|
First of all, please install `MetaDrive <https://github.com/metadriverse/metadrive>`_ and `ScenarioNet <https://github.com/metadriverse/scenarionet>`_ following these steps :ref:`installation`.
|
|
|
|
1. Setup Waymo toolkit
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
For any dataset, this step is necessary after installing ScenarioNet,
|
|
as we need to use the official toolkits of the data provider to parse the original scenario description and convert to our internal scenario description.
|
|
For Waymo data, please install the toolkit via::
|
|
|
|
pip install waymo-open-dataset-tf-2-11-0==1.5.0
|
|
|
|
# Or install with scenarionet
|
|
pip install -e .[waymo]
|
|
|
|
.. note::
|
|
This package is only supported on Linux platform.
|
|
|
|
For other datasets like nuPlan and nuScenes, you need to setup `nuplan-devkit <https://github.com/motional/nuplan-devkit>`_ and `nuscenes-devkit <https://github.com/nutonomy/nuscenes-devkit>`_ respectively.
|
|
Guidance on how to setup these datasets and connect them with ScenarioNet can be found at :ref:`datasets`.
|
|
|
|
2. Prepare Data
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Access the Waymo motion data at `Google Cloud <https://console.cloud.google.com/storage/browser/waymo_open_dataset_motion_v_1_2_0>`_.
|
|
Download one tfrecord scenario file from ``waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario/training_20s``.
|
|
In this tutorial, we only use the first file ``training_20s.tfrecord-00000-of-01000``.
|
|
Just click the download button |:arrow_down:| on the right side to download it.
|
|
And place the downloaded tfrecord file to a folder. Let's call it ``exp_waymo`` and the structure is like this::
|
|
|
|
exp_waymo
|
|
├──training_20s.tfrecord-00000-of-01000
|
|
|
|
.. note::
|
|
For building database from all scenarios, install ``gsutil`` and use this command:
|
|
``gsutil -m cp -r "gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario/training_20s" .``
|
|
Likewise, place all downloaded tfrecord files to the same folder.
|
|
|
|
|
|
3. Build Database
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Run the following command to extract scenarios in ``exp_waymo`` to ``exp_converted``::
|
|
|
|
python -m scenarionet.convert_waymo -d /path/to/exp_converted/ --raw_data_path /path/to/exp_waymo --num_files=1
|
|
|
|
.. note::
|
|
When running ``python -m``, make sure the directory you are at doesn't contain a folder called ``scenarionet``.
|
|
Otherwise, the running may fail. For more details about the command, use ``python -m scenarionet.convert_waymo -h``
|
|
|
|
Now all extracted scenarios will be placed in ``exp_converted`` directory.
|
|
If we list the directory with ``ll`` command, the structure will be like::
|
|
|
|
exp_converted
|
|
├──exp_converted_0
|
|
├──exp_converted_1
|
|
├──exp_converted_2
|
|
├──exp_converted_3
|
|
├──exp_converted_4
|
|
├──exp_converted_5
|
|
├──exp_converted_6
|
|
├──exp_converted_7
|
|
├──dataset_mapping.pkl
|
|
├──dataset_summary.pkl
|
|
|
|
This is because we use 8 workers to extract the scenarios, and thus the converted scenarios will be stored in 8 subfolders.
|
|
If we go check ``exp_converted_0``, we will see the structure is like::
|
|
|
|
├──sd_waymo_v1.2_2085c5cffcd4727b.pkl
|
|
├──sd_waymo_v1.2_27997d88023ff2a2.pkl
|
|
├──sd_waymo_v1.2_3ece8d267ce5847c.pkl
|
|
├──sd_waymo_v1.2_53e9adfdac0eb822.pkl
|
|
├──sd_waymo_v1.2_8e40ffb80dd2f541.pkl
|
|
├──sd_waymo_v1.2_df72c5dc77a73ed6.pkl
|
|
├──sd_waymo_v1.2_f1f6068fabe77dc8.pkl
|
|
├──dataset_mapping.pkl
|
|
├──dataset_summary.pkl
|
|
|
|
Therefore, the subfolder produced by each worker is actually where the converted scenarios are placed.
|
|
To aggregate the scenarios produced by all workers, the ``exp_converted/dataset_mapping.pkl`` stores the mapping
|
|
from `scenario_id` to the path of the target scenario file relative to ``exp_converted``.
|
|
As a result, we can get all scenarios produced by 8 workers by loading the database `exp_converted`.
|
|
|
|
4. Database Operations
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Several basic operations are available and allow us to split, merge, move, and check the databases.
|
|
First of all, let's check how many scenarios are included in this database built from ``training_20s.tfrecord-00000-of-01000``::
|
|
|
|
python -m scenarionet.num -d /path/to/exp_converted/
|
|
|
|
It will show that there are totally 61 scenarios.
|
|
For machine learning applications, we usually want to split training/test sets.
|
|
To this end, we can use the following command to build the training set::
|
|
|
|
python -m scenarionet.split --from /path/to/exp_converted/ --to /path/to/exp_train --num_scenarios 40
|
|
|
|
Again, use the following commands to build the test set::
|
|
|
|
python -m scenarionet.split --from /path/toexp_converted/ --to /path/to/exp_test --num_scenarios 21 --start_index 40
|
|
|
|
We add the ``start_index`` argument to select the last 21 scenarios as the test set.
|
|
To ensure that no overlap exists, we can run this command::
|
|
|
|
python -m scenarionet.check_overlap --d_1 /path/to/exp_train/ --d_2 /path/to/exp_test/
|
|
|
|
It will report `No overlapping in two database!`.
|
|
Now, let's suppose that the ``/exp_train/`` and ``/exp_test/`` are two databases built
|
|
from different source and we want to merge them into a larger one.
|
|
This can be achieved by::
|
|
|
|
python -m scenarionet.merge --from /path/to/exp_train/ /path/to/exp_test -d /path/to/exp_merged
|
|
|
|
Let's check if the merged database is the same as the original one::
|
|
|
|
python -m scenarionet.check_overlap --d_1 /path/to/exp_merged/ --d_2 /path/to/exp_converted
|
|
|
|
It will show there are 61 overlapped scenarios.
|
|
Congratulations! Now you are already familiar with some common operations.
|
|
More operations and details is available at :ref:`operations`.
|
|
|
|
5. Simulation
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The database can be loaded to MetaDrive simulator for scenario replay or closed-loop simulation.
|
|
First of all, let's replay scenarios in the ``exp_converted`` database::
|
|
|
|
python -m scenarionet.sim -d /path/to/exp_converted
|
|
|
|
|
|
By adding ``--render 3D`` flag, we can use 3D renderer::
|
|
|
|
python -m scenarionet.sim -d /path/to/exp_converted --render 3D
|
|
|
|
.. note::
|
|
``--render advanced`` enables the advanced deferred rendering pipeline,
|
|
but an advanced GPU better than RTX 2060 is required. |