waymo example

documentation/example.rst

@@ -16,7 +16,7 @@ For Waymo data, please install the toolkit via::
     pip install waymo-open-dataset-tf-2-11-0==1.5.0
 
     # Or install with scenarionet
-    pip install -e .[scenarionet]
+    pip install -e .[waymo]
 
 .. note::
     This package is only supported on Linux platform.
@@ -53,7 +53,7 @@ Run the following command to extract scenarios in ``exp_waymo`` to ``exp_convert
     When running ``python -m``, make sure the directory you are at doesn't contain a folder called ``scenarionet``.
     Otherwise, the running may fail. For more details about the command, use ``python -m scenarionet.convert_waymo -h``
 
-Now all exracted scenarios will be placed in ``exp_converted`` directory.
+Now all extracted scenarios will be placed in ``exp_converted`` directory.
 If we list the directory with ``ll`` command, the structure will be like::
 
     exp_converted

documentation/index.rst

@@ -23,11 +23,6 @@ Please feel free to contact us if you have any suggestions or ideas!
 
    install.rst
    example.rst
-
-.. toctree::
-   :maxdepth: 2
-   :caption: Database Operations
-
    operations.rst
 
 .. modify the toctree in datasets.rst together

documentation/operations.rst

@@ -1,4 +1,6 @@
-.. _operations:
+###########
+Operations
+###########
 
 How to run
 ~~~~~~~~~~

documentation/waymo.rst

@@ -2,5 +2,87 @@
 Waymo
 #############################
 
+| Website: https://waymo.com/open/
+| Download: https://waymo.com/open/download/
+| Paper: https://arxiv.org/abs/2104.10133
+
+The dataset includes:
+
+- 103,354, 20s 10Hz segments (over 20 million frames), mined for interesting interactions
+- 574 hours of data
+- Sensor data
+    - 4 short-range lidars
+    - 1 mid-range lidar
+- Object data
+    - 10.8M objects with tracking IDs
+    - Labels for 3 object classes - Vehicles, Pedestrians, Cyclists
+    - 3D bounding boxes for each object
+    - Mined for interesting behaviors and scenarios for behavior prediction research, such as unprotected turns, merges, lane changes, and intersections
+    - 3D bounding boxes are generated by a model trained on the Perception Dataset and detailed in our paper: Offboard 3D Object Detection from Point Cloud Sequences
+      Map data
+- 3D map data for each segment
+- Locations include: San Francisco, Phoenix, Mountain View, Los Angeles, Detroit, and Seattle
+- Added entrances to driveways (the map already Includes lane centers, lane boundaries, road boundaries, crosswalks, speed bumps and stop signs)
+- Adjusted some road edge boundary height estimates
+
+
+1. Install requirements
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+First of all, we have to install the waymo toolkit and tensorflow::
+
+    pip install waymo-open-dataset-tf-2-11-0
+    pip install tensorflow==2.11.0
+
+    # Or install with scenarionet
+    pip install -e .[waymo]
+
+.. note::
+    This package is only supported on Linux platform.
+
+2. Download Raw Data
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Waymo motion dataset is at `Google Cloud <https://console.cloud.google.com/storage/browser/waymo_open_dataset_motion_v_1_2_0>`_.
+For downloading all datasets, ``gsutil`` is required.
+The installation tutorial is at https://cloud.google.com/storage/docs/gsutil_install.
+
+After this, you can access all data and download them to current directory ``./`` by::
+
+    gsutil -m cp -r "gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario" .
+
+Or one just can download a part of the dataset using command like::
+
+    gsutil -m cp -r "gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario/training_20s" .
+
+The downloaded data should be stored in a directory like this::
+
+    waymo
+    ├── training_20s/
+    |   ├── training_20s.tfrecord-00000-of-01000
+    |   ├── training_20s.tfrecord-00001-of-01000
+    |   └── ...
+    ├── validation/
+    |   ├── validation.tfrecord-00000-of-00150
+    |   ├── validation.tfrecord-00001-of-00150
+    |   └── ...
+    └── testing/
+        ├── testing.tfrecord-00000-of-00150
+        ├── testing.tfrecord-00001-of-00150
+        └── ...
+
+
+3. Build Database
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Run the following command to extract scenarios in any directory containing ``tfrecord``.
+Here we take converting raw data in ``training_20s`` as an example::
+
+    python -m scenarionet.convert_waymo -d /path/to/your/database --raw_data_path ./waymo/training_20s --num_files=1000
+
+Now all converted scenarios will be placed at ``/path/to/your/database`` and are ready to be used in your work.
+
 Known Issues
 ==================
+
+N/A