Research projects
Research papers available in my ResearchGate profile:
Playlist of research videos available in my Youtube profile:
Overview of main researach areas
-
Mobile robotics (3 and 6 DoF robot self-localization and mapping)
-
Planning / optimization of the disassembly of components / products
-
2D and 3D perception
-
Projected augmented reality
Augmented reality assembly
1 - Projection of the parts location avoids the construction of fixtures (to hold and align the parts that the robot will need to pick) while also allowing for offline programming of the robot, avoiding error prone tasks such as object recognition, and grasp / motion planning.
2 - The projection system allows to visually guide / teach the operator the assembly operations while also improving the cooperation with a robotic systems:
2.1 - The next task that the operator needs to perform is shown in the projected text and video while the expected work area is highlighted in green.
2.2 - The pick and place locations of the parts are highlighted for both the operator and the robot (with different colors) to guide the operator and also show where the robot will be performing his tasks.
3D perception for packing
Packing system deployed at Plastaze (Simoldes) factory using 3D perception for accurate 6 DoF pose estimation for allowing a Yaskawa HC10 robotic arm to perform pick and pack operations alongside a conveyor.
ROS package for 6 DoF object pose estimation:
Horse webpage:
Scalable 4.0 review meeting
Review meeting video displaying the technology demonstrators for the Simoldes and PSA use cases.
ScalABLE 4.0 webpage:
3D perception for packing
Multi-product production line with an ABB robot simulating injection molding machines and two transportable collaborative manipulators (Yaskawa HC10) for packing objects from the conveyor to the respective containers.
ROS package for 6 DoF pose estimation:
ScalABLE 4.0 webpage:
3D perception for packing
Multi-product production line with an ABB robot simulating injection molding machines and two transportable collaborative manipulators (Yaskawa HC10) for packing objects from the conveyor to the respective containers.
ROS package for 6 DoF pose estimation:
ScalABLE 4.0 webpage:
3D perception for packing
System for pick and pack operations with accurate 6 DoF pose estimation of objects using CAD alignment.
ROS package for 6 DoF pose estimation:
ScalABLE 4.0 webpage:
Bin picking
Generic bin picking system capable of handling objects preparing kits of parts to be delivered to operators for assembling aircrafts.
ROS package for 6 DoF pose estimation:
Fasten webpage:
Bin picking
Preliminary results of a generic bin picking system capable of handling objects with smooth surfaces that are suitable to be grasped with a vacuum gripper.
ROS package for 6 DoF pose estimation:
Fasten webpage:
Point cloud registration
Prototype for correction of spatial offsets in 6 DoF robot trajectories by relying on point cloud registration of a reference point cloud (captured during the trajectory teaching performed by the operator) with an updated point cloud retrieved before starting the automated coating process.
ROS package for 6 DoF point cloud registration:
Horse webpage:
HORSE FLEXCoating project
The goal of the HORSE Project was to transform an existing traditional robotic cell, where the robot is protected by a cage without interacting with the operator, into a collaborative robotic cell, where the robot and the operator can paint the same component simultaneously.
Horse webpage:
Projection mapping for silos marking
Augmented reality system that was designed to assist an operator by projecting marking, cutting or welding information on top of metal components, such as silos, which can be used to store food or other raw materials. It is equipped with a 3D perception system that detects the relative 6 DoF pose between the projection system and the silo, avoiding the usage of alignment fixtures and also improving the marking accuracy when compared with manual measurements done by the operator.
ProduTech SIF project webpage:
Projection mapping for shipbuilding marking
Projected augmented reality system that was designed as a tool to assist the operator in performing marking / cutting / welding tasks on metal components of interest. This contributes to minimizing the occurrence of errors, avoiding the need to perform manual measurements and markings, increasing the efficiency in the execution of these operations. It is equipped with a 3D perception system, which allows to avoid the usage of fixtures. It is also compact enough to be attached to a tripod, making it transportable on the factory floor.
Manufactur 4.0 project webpage:
Augmented reality welding
Projection mapping system using laser projectors for helping human operators place and tack weld parts into a structural beam.
By projecting the placement position of the parts the operator can work faster and without the need of measurement equipment.
CoopWeld webpage:
Robotic welding
The CoopWeld project aimed to develop a collaborative robotic cell to perform the assembly and welding of components in steel structures. The solution developed combined offline programming software, advanced perception and augmented reality, to obtain a flexible manufacturing cell, especially for production in SME. In this approach the robot is responsible for more repetitive and time consuming tasks (welding) freeing the operator for tasks of higher added value (assembly).
CoopWeld webpage:
Laser projection mapping for welding marking
Tests at I4K-Tekniker of the projection mapping system for welding operator assistance using laser projectors.
GitHub repositories:
CLARiSSA project webpage:
Laser projection mapping for welding marking
Closer view of the tests at I4K-Tekniker of the projection mapping system for welding operator assistance using laser projectors.
GitHub repositories:
-
laser_projector_bridge_csharp
Laser projection mapping for welding marking
Demonstration at Automatica 2016 of the projection mapping system for welding operator assistance using laser projectors.
GitHub repositories:
DLP projection mapping for welding marking
Tests at INESC of the projection mapping system for welding operator assistance using DLP projectors.
ROS package repository:
DLP projection mapping for welding marking
Tests at INESC of the projection mapping system and part recognition system for welding operator assistance using DLP projectors.
ROS package repository:
Cooperative assembly and welding
The CLARiSSA project proposes the development of a dual-arm cooperative robot to perform the welding of small parts in beams.
CLARiSSA project webpage:
SolidWorks disassembly planning
SolidWorks add-in that is capable of planning optimal disassembly sequences in useful time while requiring minimal input from the operator (only the CAD model and its associated natural language instruction / repair manual are needed). It is able to automatically compute the disassembly precedences and decoupling directions using only the CAD spatial relationships and an interference detection algorithm.
Assisted assembly of a starter motor
Assisted assembly of the Mitsubishi M000T20873 starter motor using an interactive projection mapping system capable of detecting the operator fingers for navigating within the text / video assembly instructions.
The last stage of assembly includes an inspection system capable of estimating the pose of the assembled object and projecting the 3d model into the environment.
Assembly projection mapping teaching GitHub repository:
Immersive natural interaction for assisted assembly operations
Demonstration of an immersive natural interaction system for the assisted assembly of a starter engine.
Assembly projection mapping teaching GitHub repository:
Object pose estimation for assisted assembly operations
Inspection system capable of estimating the pose of an assembled object and projecting the 3d model into the environment.
Assembly projection mapping teaching GitHub repository:
Projection mapping for assisted assembly operations
Projection mapping of the Mitsubishi M000T20873 starter motor for assisted assembly operations.
Assembly projection mapping teaching GitHub repository:
Sensor placement optimization
Gazebo visualization of the estimation of the best views for active perception of a starter motor.
Assembly projection mapping teaching GitHub repository:
Sensor placement optimization
Rviz visualization of the estimation of the best views for active perception of a starter motor.
Assembly projection mapping teaching GitHub repository:
Sensor placement optimization
Estimation of the best sensor deployment for active perception of a set of target objects.
Assembly projection mapping teaching GitHub repository:
CARLoS project
Overview of the CARLoS project (CooperAtive Robot for Large spaces manufacturing), which include the development of a mobile robot for stud welding inside shipyards along with augmented reality systems for helping complex manual operations.
CARLoS project webpage:
CARLoS welding
Welding tests performed with the Guardian platform in a ship interior environment at shipyards José Valiña.
CARLoS project webpage:
CARLoS welding
Welding tests performed with the Guardian platform in a ship interior environment at shipyards José Valiña.
CARLoS project webpage:
CARLoS localization, navigation and welding
Test of the dynamic_robot_localization, oea_controller, oea_planner, carlos_mission_control and Aimen welding ROS packages from the CARLoS FP7 research project performed on the Guardian platform in a ship interior environment at shipyards José Valiña.
The robot is autonomously navigating and welding based on predefined mission goals and environment description.
ROS package repositories:
CARLoS localization and navigation
Test of the dynamic_robot_localization, oea_controller, oea_planner, carlos_mission_control and Aimen welding ROS packages from the CARLoS FP7 research project performed on the Guardian platform in a ship interior environment at shipyards José Valiña.
The robot is autonomously navigating and welding based on predefined mission goals and environment description.
ROS package repositories:
CARLoS localization and navigation
Test 1 of the dynamic_robot_localization, oea_controller and oea_planner ROS packages from the CARLoS FP7 research project performed on the Guardian platform in a ship interior environment at Deltamatic.
The robot is autonomously navigating to a series of predefined way points in which it is expected to weld (welding arm not present in this video).
ROS package repositories:
CARLoS localization and navigation
Test 2 of the dynamic_robot_localization, oea_controller and oea_planner ROS packages from the CARLoS FP7 research project performed on the Guardian platform in a ship interior environment at Deltamatic.
The robot is autonomously navigating to a series of predefined way points in which it is expected to weld (welding arm not present in this video).
ROS package repositories:
CARLoS localization on Pioneer platform
3 DoF test of the dynamic_robot_localization ROS package with the Pioneer LIDAR dataset at medium speeds (26 cm/s) with the SLAM1 path.
ROS package repository:
Dataset:
CARLoS localization on Pioneer platform
3 DoF test of the dynamic_robot_localization ROS package with the with the Jarvis LIDAR dataset at medium speeds (50 cm/s).
ROS package repository:
Dataset:
DRL 3 DoF selective mapping
3 DoF mapping test of the dynamic_robot_localization ROS package with the Guardian robot at medium speeds (30 cm/s).
ROS packages repositories:
Video legend:
-
Left side: Mapping
-
Middle: Selective mapping with an initial world model
-
Right: Localization with an static world model
-
Green dots: Laser measurements considered as inliers (correctly registered points)
-
Red dots: Laser measurements considered as outliers (unknown registered points: not present in reference map)
DRL 6 DoF localization
6 DoF test of the dynamic_robot_localization ROS package with the Guardian robot using a Structure IO sensor.
Video legend:
-
Green dots: Measurements considered as inliers (correctly registered points)
-
Red dots: Measurements considered as outliers (unknown registered points: not present in reference map)
ROS packages repositories:
DRL 6 DoF localization
6 DoF test of the dynamic_robot_localization ROS package with the ETHZ-ASL IROS 2011 Kinect dataset. Test done with the Kinect in the high complexity environment moving at low speeds with the free fly motion path.
ROS package repository:
Dataset:
DRL 6 DoF mapping
6 DoF mapping test of the dynamic_robot_localization ROS package with the a rotating LIDAR.
Video legend:
-
Green dots: Measurements considered as inliers (correctly registered points)
-
Red dots: Measurements considered as outliers (unknown registered points: not present in reference map)
ROS packages repositories:
DRL 6 DoF mapping
6 DoF mapping test of the dynamic_robot_localization ROS package with the ETHZ-ASL IROS 2011 Kinect dataset.
Test done with the Kinect in the high complexity environment moving at low speeds with the free fly motion path.
Video legend:
-
Green dots: Measurements considered as inliers (correctly registered points)
-
Red dots: Measurements considered as outliers (unknown registered points: not present in reference map)
ROS packages repositories: