Robot LUNA: Vision based Restaurant Robot

Robot LUNA is a Restaurant waiter robot capable of Vision-based navigation, with a dual camera setup- HD wide-angle camera and a Kinect-2 depth camera in a grid

**Robot LUNA** is a state-of-the-art restaurant waiter robot, designed with a focus on stability and precision. It is equipped with a dual camera setup, including an HD wide-angle camera and a Kinect-2 depth camera, enabling vision-based navigation in a 3D restaurant environment grid. The robot's enhanced stability ensures the safe delivery of food and drinks without spillage, even during braking, thanks to a dedicated stabilization tray circuit LUNA will ensure safety. Internally, LUNA has three Raspberry Pis for separate parallel processing tasks and is connected to custom PCB based on Atmega2560 to get stable and accurate sensor and encoder readings, and to control motor drivers accurately. Externally it communicates with a Restaurant's local server computer with ROS 1 Noetic through the local wifi network ensuring precise navigation path and food order locations based on the restaurant tables. This combination of advanced technology and innovative design makes LUNA a reliable and efficient addition to any restaurant staff.

> Tools & technologies used: 

- ROS 1 Noetic Ninjemys, 

- TensorFlow, 

- OpenCV, 

- Open3D,  

- C++ with Atmega2560 custom PCB for Motor and Stability Control, 

- Python with Raspberry Pi 4b ; (3 SBCs),

- Kinect v2 depth camera and a 150 Degree Wide Angle HD camera.

> Team Members:

- [Hasitha Gallella](https://github.com/HasithaGallella)

- [Sandun Herath](https://github.com/sandun21)

- [Yasiru Basnayaka](https://github.com/YasiruDEX)

- [Lasith Haputhanthri](https://github.com/lasithhaputhanthri)

- [S Thamirawaran](https://github.com/Thamirawaran)

Demonstration VideoLink: https://

Special thanks to Dr. Ranga Rodrigo, Prof. Rohan Munasinghe, and Prof. J.A.K.S. Jayasinghe for their guidance and support throughout our journey.

Also we would like to extend our special gratitude to [Department of Electronic and Telecommunication Engineering, University of Moratuwa](https://ent.uom.lk/) for the invaluable support to the project. 

With Robot LUNA at Sri Lanka Robotics Challenge 2024 (SLRC 24);

LUNA Sub-System Explanation

1. Central Restaurant Computer and Local WiFi Network: The central restaurant computer serves as the main control hub, interfacing with Robot LUNA via a local WiFi network. This setup enables seamless communication for path planning and order management, ensuring that LUNA can navigate the restaurant environment effectively. For more details, refer to Chapter 2.1. 

2. Setting Up the Workspace: Setting up the workspace involves configuring the environment to support LUNA’s operations, including network configurations and initial setup procedures. This step is critical for ensuring that all components can communicate and function as intended. For more details, refer to Chapter 2.2. 

3. ROS 2 Control System Setup: The Robot Operating System (ROS) 2 provides the framework for LUNA’s control system, enabling advanced functionalities such as real-time sensor data processing and path planning. This subsystem ensures that LUNA can make intelligent decisions based on its environment. For more details, refer to Chapter 2.3. 

4. Serial Communication with PCB: The custom PCB, based on the Atmega2560, communicates with LUNA’s computer unit via serial communication. This subsystem is responsible for relaying sensor and encoder readings, which are essential for accurate motor control and navigation. For more details, refer to Chapter 2.4. 

5. Wide Angle Camera: The wide-angle camera is a key component of LUNA’s dual-camera setup, providing a broad field of view for navigation and obstacle avoidance. This camera works in conjunction with other sensors to create a comprehensive understanding of the environment. For more details, refer to Chapter 2.5. 

6. Kinect v2 Depth Camera: The Kinect v2 depth camera captures 3D point cloud data, which is crucial for precise navigation and obstacle detection. This data allows LUNA to understand the spatial relationships within its environment, enhancing its ability to deliver orders accurately. For more details, refer to Chapter 2.6. 

7. Motor Controller Unit: The motor controller unit, integrated into the Robot LUNA PCB, manages the main motor drivers, enabling precise control over LUNA’s movements. This unit ensures smooth and accurate navigation throughout the restaurant. For more details, refer to Chapter 2.7. 

8. Stabilization Tray Controller Unit: This unit is responsible for maintaining the stability of the delivery tray, utilizing data from the accelerometer and gyroscope sensors. The Kalman filter processes this data to make real-time adjustments, preventing spillage during movement. For more details, refer to Chapter 2.8. 

9. Power Unit: The power unit supplies the necessary power to all of LUNA’s components, ensuring consistent and reliable operation. This subsystem is designed to provide stable power even under varying loads, supporting the robot’s demanding functions. For more details, refer to Chapter 2.9.