top of page

Learn Robotics and AI members

Public·1172 Roboticists



🟢 ROS 2 based Robot Perception Project for beginners: Robot Arm Teleoperation Through Computer Vision Hand-Tracking 🤖💡


(ROS is not enough: A Problem-Focused Approach to Becoming a Robotics Software Engineer - https://lnkd.in/eNWsHY8a)



🤖🔧 Franka Emika Panda robot arm


🕞 Duration: 10 weeks


📷 Hand Tracking & Gesture Recognition: Leveraging Google MediaPipe and OpenCV to achieve seamless hand tracking and gesture recognition. The vision pipeline can adeptly track and decipher gestures from multiple hands simultaneously. 👋✨



🏃‍♂️ Advanced Functionality: With PickNik Robotics's moveit_servo package, collision avoidance, singularity checking, joint position and velocity limits, and motion smoothing were implemented, ensuring precise and safe operation. 🛡️🔄



🎛 Custom ROS 2 Package Development: A tailored ROS 2 package for the Franka arm, involving intricate adjustments and fusion of URDF, SRDF, and other configuration files was developed. 🛠️🤖



🤖 Robust ROS 2 Implementation: The project boasts multiple ROS2 nodes and packages coded in both C++ and Python, ensuring versatility and efficiency in operation. 🐍🔨



💻 The system Graham developed is composed of three main nodes: handcv, cv_franka_bridge, and franka_teleop. The handcv node captures the 3D position of the user’s hands and provides gesture recognition. The cv_franka_bridge processes the information provided from handcv and sends commands to the franka_teleop node. The franka_teleop node runs an implementation of the moveit_servo package, enabling smooth real-time control of the Franka robot. 



💪🚀 Project page: https://lnkd.in/ercAt5qz 


🐱 GitHub: https://lnkd.in/eDb6gRyy



Right now, gestures from both hands are captured, but only gestures from the right hand are used to control the robot.



Here’s a list of the gestures the system recognizes and what they do:


👍 Thumbs Up (Start/Stop Tracking): This gesture is used to start/stop tracking the position of your right hand. It also allows you to adjust your hand in the camera frame without moving the robot. While the camera sees you giving a thumbs up, the robot won’t move, but once you release your hand, the robot will start tracking your hand. [thumbs_up]


👎 Thumbs Down (Shutdown): This gesture is used to stop tracking your hand and to end the program. You will not be able to give the thumbs up gesture anymore and will have to restart the program to start tracking your hand again. [thumbs_down]


👊 Close Fist (Close Gripper): This gesture will close the gripper of the robot. [closed_fist]


🤲 Open Palm (Open Gripper): This gesture will open the gripper of the robot. [open_palm]



Credits: Graham Clifford completed this project as part of M.S. in Robotics at Northwestern University 🎓💼

Ridwanullah Jimoh
Mar 27, 2024

I would like to try this.

About

Welcome to the group! You can connect with other members, ge...

bottom of page