实验环境

（1）硬件环境

（2）软件环境

实验步骤

1. 启动虚拟机，在Windows电脑中，使用Vmware运行Ubuntu系统。
2. 找到路径 /Home/dofbot_ws/src/dofbot_color_follow/颜色追踪color_follow.ipynb，右击选择用Jupyter Notebook启动，输入密码 shujia。
3. 按照代码单元顺序执行程序，初始化机械臂位置并启动交互界面。
4. 点击【learning_color】按钮，画面中心出现一个方框，在方框内放置待追踪物体，实时观察HSV的高低阈值，当方框下面出现【OK !!!】字样时，说明识别成功。
5. 点击【learning_follow】按钮，机械臂将对学习到的颜色进行实时跟踪。
6. 点击【follow_cancel】按钮可取消跟踪（不退出程序）；点击【Exit】按钮可退出程序。
7. 也可直接点击【color_follow】按钮，通过【choose_color】选择预设的红、绿、蓝、黄四种颜色进行追踪。

实验结果

取色追踪结果显示

实验原理

主要代码

代码路径：/Home/dofbot_ws/src/dofbot_color_follow/颜色追踪color_follow.ipynb

1. 导入头文件与初始化：

# 导入头文件
import cv2 as cv 
import threading 
import random
from time import sleep
import ipywidgets as widgets
from IPython.display import display
from color_follow import color_follow

# 初始化机械臂位置
import Arm_Lib
Arm = Arm_Lib.Arm_Device()
joints_0 = [90, 135, 20, 25, 90, 30]
Arm.Arm_serial_servo_write6_array(joints_0, 1000)

# 创建实例,初始化参数
follow = color_follow()
# 初始化模式
model = 'General'
# 初始化HSV_learning值
HSV_learning = ()
# 初始化HSV值
color_hsv = {"red" : ((0, 43, 46), (10, 255, 255)), 
             "green" : ((35, 43, 46), (77, 255, 255)),
             "blue" : ((100, 43, 46), (124, 255, 255)), 
             "yellow": ((26, 43, 46), (34, 255, 255))}
# 设置随机颜色
color = [[random.randint(0, 255) for _ in range(3)] for _ in range(255)]

2. 创建交互控件：

button_layout = widgets.Layout(width='200px', height='100px',align_self='center')
# 输出控件
output = widgets.Output()
# 颜色追踪
color_follow = widgets.Button(description='color_follow', button_style='success', layout=button_layout)
# 选择颜色
choose_color = widgets.ToggleButtons(options=['red', 'green', 'blue', 'yellow'],  
                                     button_style='success',
                                     tooltips=['Description of slow', 'Description of regular', 'Description of fast'])
# 取消追踪
follow_cancel = widgets.Button(description='follow_cancel', button_style='danger', layout=button_layout)
# 学习颜色
learning_color = widgets.Button(description='learning_color', button_style='primary', layout=button_layout)
# 学习颜色追踪
learning_follow = widgets.Button(description='learning_follow', button_style='success', layout=button_layout)
# 退出
exit_button = widgets.Button(description='Exit', button_style='danger', layout=button_layout)
# 图像控件
imgbox = widgets.Image(format='jpg', height=480, width=640, layout=widgets.Layout(align_self='auto'))
# 垂直布局
img_box = widgets.VBox([imgbox, choose_color], layout=widgets.Layout(align_self='auto'))
# 垂直布局
Slider_box = widgets.VBox([color_follow, learning_color,
                          learning_follow,follow_cancel,exit_button],
                          layout=widgets.Layout(align_self='auto'))
# 水平布局
controls_box = widgets.HBox([img_box, Slider_box],layout=widgets.Layout(align_self='auto'))

3. 模式切换函数：

def color_follow_Callback(value): 
    global model
    model = 'color_follow'

def learning_color_Callback(value): 
    global model
    model = 'learning_color'

def learning_follow_Callback(value): 
    global model
    model = 'learning_follow'

def follow_cancel_Callback(value): 
    global model
    model = 'General'

def exit_button_Callback(value): 
    global model
    model = 'Exit'

# 绑定按钮事件
color_follow.on_click(color_follow_Callback)
learning_color.on_click(learning_color_Callback)
learning_follow.on_click(learning_follow_Callback)
follow_cancel.on_click(follow_cancel_Callback) 
exit_button.on_click(exit_button_Callback)

4. 主程序：

def camera():
    global HSV_learning,model
    # 打开摄像头
    capture = cv.VideoCapture(0) 
    capture.set(3, 640)
    capture.set(4, 480)
    capture.set(5, 30)  # 设置帧率
    
    # 当摄像头正常打开的情况下循环执行
    while capture.isOpened(): 
        try:
            # 读取相机的每一帧
            _, img = capture.read()
            # 统一图像大小
            img = cv.resize(img, (640, 480)) 
            
            if model == 'color_follow':
                img = follow.follow_function(img,color_hsv[choose_color.value])
                # 添加文字
                cv.putText(img, choose_color.value, (int(img.shape[0] / 2), 50), 
                          cv.FONT_HERSHEY_SIMPLEX, 2, color[random.randint(0, 254)], 2)
            
            if model == 'learning_color':
                img,HSV_learning = follow.get_hsv(img)
            
            if model == 'learning_follow':
                img = follow.learning_follow(img, HSV_learning)
                # 添加文字
                cv.putText(img,'LeColor', (200, 50), 
                          cv.FONT_HERSHEY_SIMPLEX, 1, color[random.randint(0, 254)], 1)
            
            if model == 'Exit':
                cv.destroyAllWindows() 
                capture.release()
                break
            
            # 更新图像显示
            imgbox.value = cv.imencode('.jpg', img)[1].tobytes() 
            
        except KeyboardInterrupt:
            capture.release()

# 启动程序
display(controls_box,output)
threading.Thread(target=camera, ).start()