QC Tool Overview - 2D-3D Linking
Last updated
Last updated
When the tool loads, the annotation list on the right sidebar indicates the status of correction for every listed annotation. By default, the status always indicates as incomplete in the form of a 'question mark'.
Upon selecting an annotation, the annotation detail panel slides open. There are selectable crosses and ticks for every corresponding detail. Based on your actions, you may encounter Below is the list of statuses you may encounter based on the annotation
a) Incomplete
When all details are left unmarked.
b) Correct
When all details are marked as correct.
c) Incorrect
This can appear in two cases:
When all details are marked as incorrect.
When one or a few details are marked as incorrect and the rest, as correct.
d) Partially Correct
When one or a few details are marked as correct and the rest, left unmarked.
e) Partially Incorrect
This can appear in two cases:
When one or few details are marked as incorrect and the rest, unmarked.
When one or few details are marked as incorrect, some as correct, and some left unmarked.
Here's the video showing all the status
Upon selecting an annotation, the annotation detail panel slides open. There are crosses and ticks for every corresponding detail that can be selected.
On clicking 'Mark unmarked as correct below' all the details that are incomplete, get marked as correct. The status of the annotation on the Annotation list will then reflect appropriately.
On clicking 'Mark unmarked as correct - across frames' over the Annotation List, all incomplete details for annotations across frames get marked as corrected.
Tips for Efficiency :
Use the View by Group feature to easily check the class and attributes in a frame and across frames for an annotation
Mark what is incorrect in every frame and then click the 'Mark unmarked as correct - across frames' button over the Annotation list right at the end before submitting. This will mark everything else that is incomplete as correct with just one click.
The description is a tracker-level description
If you see any error in the existing annotation, you can add the description by selecting the annotation
Issue - Issue is any type of quality error observed in the task. It could be of 3 types -
Missed annotation - If an annotation is missing.
Annotation level - If something is wrong with an existing annotation like geometry, class, attribute, etc.
Other - If any error doesn't belong to the above categories e.g. a guideline change, or a misc case.
Description - Each issue will have its own description which contains the details about the issue.
For missed annotation and annotation level issues it is optional.
For other issues adding a description is mandatory.
For Missed and Other issues you'll see a Mark Issue button on the top left,
If the primary view is LiDAR
Clicking on the Mark issue button will enable drawing a beacon which you can use to mark the location in the LiDAR view where you see the error.
The beacon can be dropped only on a point, if you observe that dropping a beacon is not working, then it’s because it’s not on a point.
If the primary view is Camera
Clicking on the Mark issue button will enable drawing a doodle which you can use to mark the location in the frame/image where you see the error
You will have to change the camera image in the primary view to mark errors in other cameras in the same frame
2. Once you have added an issue using a doodle or beacon, it'll be selected as missed annotation by default. You can change it according to the type of issue you observed.
For Missed annotation Issue -
Adding a class is mandatory
The counter - No. of missed objects is used to depict the number of missed objects in that frame.
This will be used to calculate the recall metric for the QC task.
Description - It is optional to add the description for missed annotation issues.
For Other issue -
Description - It will be mandatory to add a description.
Other issues won't be used for QC metric calculation, its purpose is to get subjective feedback for any misc case.
If you see any error in the existing annotation, you can add the description by selecting the annotation.
You’ll be able to add separate descriptions for each sensor where the object is present
The description is a tracker-level description.
3. Issues in the right-side panel will be active only for the primary sensor.
In the below image LiDAR view is the primary view hence the camera issues are disabled.
When you switch to the camera view, the issue present in that particular camera sensor will become active/clickable and the rest of them will become inactive.
