Development of Deep Learning-Based Logistics Sorting System
Object Detection and Pose Estimation
based on Deep Learning
Present Name :
12010611
相昊阳
2024-3-26
Project Summary
•
Our project investigates automated parcel sorting with a robotic arm in a simulated environment. With the rapid
development of the logistics industry, traditional manual sorting can no longer cope with heavy sorting loads.
Therefore, vision-based intelligent logistics sorting systems have significant research value. Detecting objects’
positions, categories, and attitude information accurately and quickly has become a key issue in the implementation
of intelligent sorting systems.
•
We'll delve into literature on object detection, pose estimation, and robotics control for background. And the detailed
content of the references will be listed in the following slides.
•
We plan to utilize well-established datasets such as YCB, Linemod, COCO, and the Cornell Grasping Dataset, known
for their diversity in object types and complexity in scenes, providing a robust foundation for training our models. If
necessary, we might collect new data from simulated environments tailored to our project's specific needs, then we
will label the object's position and orientation data ourselves.
Presenter Name & Date of Presentation
Project Proposal Presentation
2
Project Summary
•
We propose to use the YOLO model for object detection due to its efficiency and accuracy. For pose estimation,
we will explore deep learning-based pose estimation methods, such as DenseFusion. While there are existing
implementations available, we plan to fine-tune these models with our collected data to enhance their
performance.
•
To evaluate our results, we decided to adopt both score assessment and ratio assessment methods. Score
assessment refers to using a specific score range to represent the quality of experimental results, where a high
score indicates good results, and conversely, a low score suggests the results are worse than expected. Ratio
assessment means quantifying experimental results through objective ratios, such as accuracy, precision, and
recall, etc.
Presenter Name & Date of Presentation
Project Proposal Presentation
3
What is the problem that you will be investigating?
•
Develop a logistics sorting system based on deep learning for parcel classification and visual sorting. The
system should be able to recognize the dimensions of parcels and sort them in a disorderly arrangement.
•
This research problem arises from the rapid development of e-commerce, which has led to a significant
increase in parcel logistics volume. Traditional manual classification and sorting methods are no longer
sufficient to meet the demands of efficient operations in modern logistics centers. Therefore, the development
of robotic systems capable of automatically completing sorting tasks is particularly important. Solving this
problem not only improves the efficiency and accuracy of logistics processing but also has the potential to
significantly reduce logistics costs and enhance customer satisfaction.
Why is it interesting?
Presenter Name & Date of Presentation
Project Proposal Presentation
4
What reading will you examine?
1. Han, Song, Xiaoping Liu, Xing Han, Gang Wang, and Shaobo Wu. 2020. "Visual
Sorting of Express Parcels Based on Multi-Task Deep Learning" Sensors 20, no. 23:
6785.
https://-doi.org/10.3390/s20236785
The article "Visual Sorting of Express Parcels Based on Multi-Task Deep
Learning" focuses on enhancing the efficiency and accuracy of sorting
express parcels in complex scenarios using a robot sorting method
powered by multi-task deep learning.
In addition, we will read more
paper
s on object detection and pose
estimation including:
2. Xing, H.; Xiao-Ping, L. Robotic sorting method in complex scene based on deep
neural network. J. BeijingUniv. Posts Telecommun. 2019, 42, 22–28.
3. Andy, Z.; Shuran, S.; Kuan-Ting, Y.; Elliott, D.; Alberto, R. Robotic pick-and-place
of novel objects in clutter with multi-affordance grasping and cross-domain image
matching. In Proceedings of the 2018 IEEE International Conference on Robotics and
Automation (ICRA), Brisbane, Australia, 21–26 May 2018.
To provide context and background
Presenter Name & Date of Presentation
Project Proposal Presentation
5
What data will you use?
•
Dataset from web
•
We will use well-established datasets such as YCB, Linemod and COCO to train our model.
•
New data
•
Get object photos and pose information from RGB-D camera.
•
Do the image processing using OpenCV.
•
Label the object's position and orientation data ourselves.
If you are collecting new data, how will you do it?
Presenter Name & Date of Presentation
Project Proposal Presentation
6
What method or algorithm are you proposing?
•
Computer Vision
•
Data Acquisition
: imagery and depth information from
RGB-D cameras
•
Image Preprocessing
: reading, cropping, resizing, enhancement using
OpenCV
•
Deep Learning
Utilizing the
TensorFlow
deep learning framework
•
Object Detection
: pre-trained
YOLO
model
•
Finetune the YOLO model with annotated data generated from the environment
•
Pose Estimation
:
pose estimation
method
•
Explore available pose estimation (e.g.
DenseFusion
)
Computer Vision and Deep learning
Presenter Name & Date of Presentation
Project Proposal Presentation
7
What method or algorithm are you proposing?
•
Algorithm Optimization
•
Adjust
network structure
,
loss functions
, and
training strategies
to enhance accuracy
and real-time performance.
•
System Integration
•
Integrate
object detection
,
pose estimation
, and
robot arm gripping
into a unified
system.
Potential Future Improvements
Presenter Name & Date of Presentation
Project Proposal Presentation
8
How will you evaluate your results?
Presenter Name & Date of Presentation
Project Proposal Presentation
9
To evaluate our result, we will apply some objective
evaluation criteria, depending on the specific
circumstances.
1)
Accuracy: the proportion of objects that are
correctly grasped and placed among all objects.
2) Precision: the proportion of samples correctly
predicted as positive out of all samples predicted as
positive by the model.
3) Recall, F1 Score …
Two kinds of evaluation methods
Thank you for listening
Investigating automated parcel sorting with a robotic arm in a simulated environment using object detection and pose estimation based on deep learning. The project aims to enhance sorting efficiency in response to the increasing volume of parcels in the logistics industry with the application of vision-based intelligent systems. Utilizing datasets like YCB, Linemod, COCO, and Cornell Grasping Dataset, the project plans to implement models such as YOLO for object detection and DenseFusion for pose estimation, fine-tuning them for improved performance.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Object Detection and Pose Estimation based on Deep Learning Present Name : 12010611 2024-3-26 AncoraSIR.com
Project Summary Our project investigates automated parcel sorting with a robotic arm in a simulated environment. With the rapid development of the logistics industry, traditional manual sorting can no longer cope with heavy sorting loads. Therefore, vision-based intelligent logistics sorting systems have significant research value. Detecting objects positions, categories, and attitude information accurately and quickly has become a key issue in the implementation of intelligent sorting systems. We'll delve into literature on object detection, pose estimation, and robotics control for background. And the detailed content of the references will be listed in the following slides. We plan to utilize well-established datasets such as YCB, Linemod, COCO, and the Cornell Grasping Dataset, known for their diversity in object types and complexity in scenes, providing a robust foundation for training our models. If necessary, we might collect new data from simulated environments tailored to our project's specific needs, then we will label the object's position and orientation data ourselves. AncoraSIR.com Presenter Name & Date of Presentation Project Proposal Presentation 2
Project Summary We propose to use the YOLO model for object detection due to its efficiency and accuracy. For pose estimation, we will explore deep learning-based pose estimation methods, such as DenseFusion. While there are existing implementations available, we plan to fine-tune these models with our collected data to enhance their performance. To evaluate our results, we decided to adopt both score assessment and ratio assessment methods. Score assessment refers to using a specific score range to represent the quality of experimental results, where a high score indicates good results, and conversely, a low score suggests the results are worse than expected. Ratio assessment means quantifying experimental results through objective ratios, such as accuracy, precision, and recall, etc. AncoraSIR.com Presenter Name & Date of Presentation Project Proposal Presentation 3
What is the problem that you will be investigating? Why is it interesting? Develop a logistics sorting system based on deep learning for parcel classification and visual sorting. The system should be able to recognize the dimensions of parcels and sort them in a disorderly arrangement. This research problem arises from the rapid development of e-commerce, which has led to a significant increase in parcel logistics volume. Traditional manual classification and sorting methods are no longer sufficient to meet the demands of efficient operations in modern logistics centers. Therefore, the development of robotic systems capable of automatically completing sorting tasks is particularly important. Solving this problem not only improves the efficiency and accuracy of logistics processing but also has the potential to significantly reduce logistics costs and enhance customer satisfaction. AncoraSIR.com Presenter Name & Date of Presentation Project Proposal Presentation 4
What reading will you examine? To provide context and background 1. Han, Song, Xiaoping Liu, Xing Han, Gang Wang, and Shaobo Wu. 2020. "Visual Sorting of Express Parcels Based on Multi-Task Deep Learning" Sensors 20, no. 23: 6785. https://-doi.org/10.3390/s20236785 The article "Visual Sorting of Express Parcels Based on Multi-Task Deep Learning" focuses on enhancing the efficiency and accuracy of sorting express parcels in complex scenarios using a robot sorting method powered by multi-task deep learning. In addition, we will read more papers on object detection and pose estimation including: 2. Xing, H.; Xiao-Ping, L. Robotic sorting method in complex scene based on deep neural network. J. BeijingUniv. Posts Telecommun. 2019, 42, 22 28. 3. Andy, Z.; Shuran, S.; Kuan-Ting, Y.; Elliott, D.; Alberto, R. Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21 26 May 2018. AncoraSIR.com Presenter Name & Date of Presentation Project Proposal Presentation 5
What data will you use? If you are collecting new data, how will you do it? Dataset from web We will use well-established datasets such as YCB, Linemod and COCO to train our model. New data Get object photos and pose information from RGB-D camera. Do the image processing using OpenCV. Label the object's position and orientation data ourselves. AncoraSIR.com Presenter Name & Date of Presentation Project Proposal Presentation 6
What method or algorithm are you proposing? Computer Vision and Deep learning Computer Vision Data Acquisition: imagery and depth information from RGB-D cameras Image Preprocessing: reading, cropping, resizing, enhancement using OpenCV Deep Learning Utilizing the TensorFlow deep learning framework Object Detection: pre-trained YOLO model Finetune the YOLO model with annotated data generated from the environment Pose Estimation: pose estimation method Explore available pose estimation (e.g. DenseFusion) AncoraSIR.com Presenter Name & Date of Presentation Project Proposal Presentation 7
What method or algorithm are you proposing? Potential Future Improvements Algorithm Optimization Adjust network structure, loss functions, and training strategies to enhance accuracy and real-time performance. System Integration Integrate object detection, pose estimation, and robot arm gripping into a unified system. AncoraSIR.com Presenter Name & Date of Presentation Project Proposal Presentation 8
How will you evaluate your results? Two kinds of evaluation methods Score To evaluate our result, we will apply some objective evaluation criteria, depending on the specific circumstances. 60> Unable to complete the task at all, serious errors occurred during the gripping process. The box can be gripped to the specified area, but there are issues with uneven placement or unstable gripping. The box can be gripped to the correct position and roughly placed neatly. The box can be accurately gripped and placed. 60-70 1) Accuracy: the proportion of objects that are correctly grasped and placed among all objects. 70-80 2) Precision: the proportion of samples correctly predicted as positive out of all samples predicted as positive by the model. 80-90 3) Recall, F1 Score 90-100 The box can be gripped and placed quickly and accurately. AncoraSIR.com Presenter Name & Date of Presentation Project Proposal Presentation 9
Thank you for listening AncoraSIR.com
