前面我们已经部署了单节点的OpenStack环境，这次我们尝试部署一个多节点的环境(包括1个controller节点，1个compute节点，1个storage节点)，之前的单节点因为已经安装了kolla-ansible和docker registry，我们把它当做部署节点。 # 环境准备 准备三台服务器，分别作为controller节点，compute节点，storage节点。其中controller节点3张网卡，compute、storage节点2张网卡。操作系统为centos7.4 1）关闭Selinux vi /etc/sysconfig/selinux SELINUX=disabled 2）关闭firewalld systemctl stop firewalld systemctl disable firewalld 3）设置主机名，hosts文件 vi /etc/hosts 172.16.31.130 control01 172.16.31.131 controller 172.16.31.132 compute 172.16.31.133 storage 4）storage节点加盘 为storage节点划分磁盘，作为cinder的lvm后端，storage节点有13tb的硬盘采用parted分区，分区过程略去  创建pv、vg [root@storage ~]# pvcreate /dev/sdb Physical volume "/dev/sdb" successfully created. [root@storage ~]# vgcreate cinder-volumes /dev/sdb Volume group "cinder-volumes" successfully created 修改配置 vi /etc/lvm/lvm.conf  重启lvm服务 systemctl restart lvm2-lvmetad.service 5）确认节点配置 controller 3 张网卡 compute 2 张网卡 storage 2 张网卡(其实一张即可，因为不需要tunnel网络) storage 磁盘已经分区  # 安装docker 1）下载相关docker rpm包 这里我已经到docker把docker的rpm下载下来了，我们上传的相关的目录下，当然还是需要配centos和epel的源才能正常安装的  2）安装Docker 1.12.6（三台服务器都要安装） yum install docker-engine-* -y 在各个节点安装 yum install python-docker-py -y 3）设置Docker mkdir /etc/systemd/system/docker.service.d tee /etc/systemd/system/docker.service.d/kolla.conf << 'EOF' [Service] MountFlags=shared EOF 4）重启相关服务 systemctl daemon-reload systemctl enable docker systemctl restart docker 5）编辑/usr/lib/systemd/system/docker.service文件 ExecStart=/usr/bin/dockerd --insecure-registry 172.16.31.130:4000 6）重启Docker服务 systemctl daemon-reload systemctl restart docker # 配置免密登录 部署节点和其他节点免密登陆 [root@control01 ~]# ssh-keygen [root@control01 ~]# ssh-copy-id controller [root@control01 ~]# ssh-copy-id compute [root@control01 ~]# ssh-copy-id storage # 部署OpenStack 1）停止部署节点容器 因为部署节点之前已经部署了All in One的OpenStack，为了降低资源占用，我们可以先停止本节的OpenStack相关容器。 [root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/tools/ [root@control01 tools]# . kolla-ansible stop 2）修改multinode文件 [root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/ [root@control01 kolla-ansible-4.0.3.dev36]# cp ansible/inventory/* /home/ [root@control01 ~]# cd /home [root@control01 home]# ls all-in-one multinode [root@control01 home]# cp multinode mymultinode [root@control01 home]# vi mymultinode [control] controller [network] controller [compute] compute [monitoring] controller [storage] storage ...... 里面的内容很好理解，表示一个控制节点 controller，网络节点也安装到控制节点，一个计算节点 compute ，一个存储节点 storage，后面的部分不用修改 3）修改global和password配置文件 [root@control01 ~]# cd /etc/kolla/ [root@control01 kolla]# vi globals.yml ————————————分割线——————————————— kolla_internal_vip_address: "172.16.31.131" 外部访问地址，如果是非HA环境，是控制节点IP docker_registry: "172.16.31.130:4000" 镜像库，用部署节点的镜像库 docker_namespace: "99cloud" network_interface: "eno1" 网络接口，默认就这一个接口，这里我们以eth0作为API网络 api_interface: "{{ network_interface }}" 用的是eno2 tunnel_interface: "eno2" eno2作为vxlan的承载网络 neutron_external_interface: "eno3" eno3作为外部网络 enable_cinder: "yes" 启用cinder enable_cinder_backend_lvm: "yes" cinder后端用lvm cinder_volume_group: "cinder-volumes" enable_haproxy: "no" 不启用haproxy ————————————分割线——————————————— [root@control01 kolla]# vi passwords.yml keystone_admin_password: admin admin用户登录密码 4）部署前检查 [root@control01 ~]# cd /root/kolla-ansible-4.0.3.dev36/tools/ [root@control01 ~]# . kolla-ansible prechecks -i /home/mymultinode 没有报错即可正常部署 5）部署 [root@control01 tools]# . kolla-ansible deploy -i /home/mymultinode 开始部署，等待大约20分钟，部署完成，一般不会有什么问题，如果有问题，则需要根据报错及容器日志查找原因  登录dashboard http://172.16.31.131  登录系统，查看各个服务正常，表明部署成功。 可以通过docker ps命令查看各个节点运行的容器 controller节点： [root@controller ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1ecab1b4246e 172.16.31.130:4000/99cloud/centos-source-horizon:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours horizon b264ea5467ff 172.16.31.130:4000/99cloud/centos-source-heat-engine:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours heat_engine 08ab0aa8603c 172.16.31.130:4000/99cloud/centos-source-heat-api-cfn:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours heat_api_cfn 6f6aba4804a2 172.16.31.130:4000/99cloud/centos-source-heat-api:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours heat_api 67729f946b23 172.16.31.130:4000/99cloud/centos-source-neutron-metadata-agent:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_metadata_agent f8f8d9f766e9 172.16.31.130:4000/99cloud/centos-source-neutron-l3-agent:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_l3_agent fd3a8ca63fdf 172.16.31.130:4000/99cloud/centos-source-neutron-dhcp-agent:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_dhcp_agent c9f9f72090a4 172.16.31.130:4000/99cloud/centos-source-neutron-openvswitch-agent:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_openvswitch_agent 9f19bca0759c 172.16.31.130:4000/99cloud/centos-source-neutron-server:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_server e1286083b3f7 172.16.31.130:4000/99cloud/centos-source-openvswitch-vswitchd:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours openvswitch_vswitchd dd4986364575 172.16.31.130:4000/99cloud/centos-source-openvswitch-db-server:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours openvswitch_db a6692acea93f 172.16.31.130:4000/99cloud/centos-source-nova-novncproxy:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_novncproxy cd704f0bc2c7 172.16.31.130:4000/99cloud/centos-source-nova-consoleauth:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_consoleauth 105978eda1a9 172.16.31.130:4000/99cloud/centos-source-nova-conductor:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_conductor 9d9e7571d7dd 172.16.31.130:4000/99cloud/centos-source-nova-scheduler:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_scheduler 7436ca003817 172.16.31.130:4000/99cloud/centos-source-nova-api:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_api ae8d830091d3 172.16.31.130:4000/99cloud/centos-source-nova-placement-api:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours placement_api f053087cfde5 172.16.31.130:4000/99cloud/centos-source-cinder-scheduler:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cinder_scheduler 8c849182ee63 172.16.31.130:4000/99cloud/centos-source-cinder-api:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cinder_api 129996bd76ee 172.16.31.130:4000/99cloud/centos-source-glance-registry:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours glance_registry fcc184936be5 172.16.31.130:4000/99cloud/centos-source-glance-api:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours glance_api 06a35ba7eb7f 172.16.31.130:4000/99cloud/centos-source-keystone:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours keystone 6821886162bc 172.16.31.130:4000/99cloud/centos-source-rabbitmq:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours rabbitmq 04b556da8365 172.16.31.130:4000/99cloud/centos-source-mariadb:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours mariadb 859a1d8d0137 172.16.31.130:4000/99cloud/centos-source-cron:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cron 4e7230af5339 172.16.31.130:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours kolla_toolbox 50be20c91704 172.16.31.130:4000/99cloud/centos-source-fluentd:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours fluentd 72b2538e1beb 172.16.31.130:4000/99cloud/centos-source-memcached:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours memcached compute节点： [root@compute ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0f954ef9095e 172.16.31.130:4000/99cloud/centos-source-neutron-openvswitch-agent:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours neutron_openvswitch_agent 8f62c376e2b5 172.16.31.130:4000/99cloud/centos-source-openvswitch-vswitchd:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours openvswitch_vswitchd 005c3dd582a9 172.16.31.130:4000/99cloud/centos-source-openvswitch-db-server:4.0.2.1 "kolla_start" 24 hours ago Up 24 hours openvswitch_db d6a9b676c0ad 172.16.31.130:4000/99cloud/centos-source-nova-compute:4.0.2.1 "kolla_start" 36 hours ago Up 36 hours nova_compute fbc66c3f40ba 172.16.31.130:4000/99cloud/centos-source-nova-libvirt:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours nova_libvirt 6f942f376c83 172.16.31.130:4000/99cloud/centos-source-nova-ssh:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours nova_ssh 4872cfb2b025 172.16.31.130:4000/99cloud/centos-source-iscsid:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours iscsid c55ec16b0752 172.16.31.130:4000/99cloud/centos-source-cron:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cron 426d77add095 172.16.31.130:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours kolla_toolbox 5f959b2372f0 172.16.31.130:4000/99cloud/centos-sourc stroage节点： [root@cinder ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 7eca045eeff1 172.16.31.130:4000/99cloud/centos-source-cinder-backup:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cinder_backup bcdf196510f2 172.16.31.130:4000/99cloud/centos-source-cinder-volume:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cinder_volume d59fc8ec7716 172.16.31.130:4000/99cloud/centos-source-tgtd:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours tgtd 7a8d9dab2755 172.16.31.130:4000/99cloud/centos-source-iscsid:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours iscsid ea471af172de 172.16.31.130:4000/99cloud/centos-source-cron:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours cron 4b41ebf3ea62 172.16.31.130:4000/99cloud/centos-source-kolla-toolbox:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours kolla_toolbox 91c086f63421 172.16.31.130:4000/99cloud/centos-source-fluentd:4.0.2.1 "kolla_start" 37 hours ago Up 37 hours fluentd