Adaptive Robust Learning Control for a 6-DOF Robotic Arm with Real-Time Object Detection

The original open-source arm depended on expensive CNC-machined components and imported harmonic reducers, pushing the estimated cost above RMB 10,000. I replaced parts of that stack with lower-cost reducers and PETG structures. The cost fell, but backlash, compliance, and manufacturing uncertainty became first-order control problems.

Instead of treating these constraints only as defects, I reframed them as a research question: could a learning-based controller recover precision from an affordable mechanical system?

I integrated a custom six-degree-of-freedom arm using harmonic reducers, stepper motors, motor-control PCBs, CAN communication, a custom gripper, and a Gemini Pro depth camera. I designed the gripper in Autodesk Fusion 360 and produced both CNC-machined and PETG 3D-printed variants.

The implementation included sourcing components, assembling and soldering PCBs, installing firmware, calibrating motors and cameras, resolving mechanical tolerances, and integrating the complete ROS-based system.

I mounted the depth camera on the end effector, prepared a custom dataset, and trained YOLOv10 to recognize and locate nuts and screws in real time. The model reached 97% classification accuracy for both classes and a mAP@0.5 of 0.98.

Combining depth and object detection allowed the perception pipeline to produce spatial targets for manipulation rather than isolated image classifications.

I proposed an Adaptive Robust Learning controller that combines a Disturbance Observer with Iterative Learning Control. The observer estimates unmodeled disturbances while the learning term improves repeated trajectory execution.

The controller achieved an RMSE of 1.004, a 55.39% improvement over traditional PID control under the same hardware uncertainty.

Several boards failed during assembly, and I damaged CAN chips twice through incorrect connections. Other failures came from incomplete bills of materials, firmware installation, camera drivers, virtual machines, calibration, and even the orientation-dependent behavior of a dual-purpose USB-C port.

The project taught me that open-source files do not remove engineering difficulty. Every tolerance, electrical connection, dependency, and calibration decision changes the final system. Persistent debugging transformed practical constraints into a publishable control problem.

My long-term goal is to extend the platform with language models, speech recognition, richer visual reasoning, responsive kinematics, precise control, and compact high-performance hardware.