Ceph-0.87对接OpenStack-kilo版本的cinder存储_综合

Ceph-0.87对接OpenStack-kilo版本的cinder存储

环境

集群	主机	ip	版本
OpenStack	controller	66.66.66.71	openstack-kilo
OpenStack	compute	66.66.66.72	openstack-kilo
Ceph	ceph01	66.66.66.235	ceph-0.87

首先检测一下ceph集群的状态

[root@ceph01 ceph]# ceph- s-bash: ceph-: command not found[root@ceph01 ceph]# ceph -scluster 3ec3f0f5-74d7-41ce-a497-1425705cd717health HEALTH_OKmonmap e1: 1 mons at {ceph01=66.66.66.235:6789/0}, election epoch 1, quorum 0 ceph01osdmap e42: 3 osds: 3 up, 3 inpgmap v273: 320 pgs, 3 pools, 16 bytes data, 3 objects27094 MB used, 243 GB / 269 GB avail320 active+clean

Ceph01节点操作

创建存储池

[root@ceph01 ~]# ceph osd pool create ceph-cinder 128pool 'ceph-cinder' created

创建认证用户

[root@ceph01 ~]# ceph auth get-or-create client.ceph-cinder mon 'allow r' osd 'allow class-read ohildren, allow rwx pool=ceph-cinder'[client.ceph-cinder]key = AQCIVY5beGJDNxAAphG+hdDC1vG4yVC5Ew7Y+w==

生成keyring，并发送到compute节点

[root@ceph01 ~]# ceph auth get-or-create client.ceph-cinder |ssh compute /etc/ceph/ceph.client.ceph-cinder.keyring

发送ceph配置文件到compute节点

[root@ceph01 ~]# scp /etc/ceph/ceph.conf compute:/etc/ceph/ceph.conf

compute节点操作

生成uuid，并写入到秘钥文件中

[root@compute ~]# uuidgen207a92a6-acaf-47c2-9556-e560a79ba472[root@compute ~]# cat > secret.xml <<EOF<secret ephemeral='no' private='no'><uuid>207a92a6-acaf-47c2-9556-e560a79ba472</uuid><usage type='ceph'><name>client.ceph-cinder secret </name></usage></secret>EOF

定义秘钥

[root@compute ~]# virsh secret-define --file secret.xmlSecret 207a92a6-acaf-47c2-9556-e560a79ba472 created

设置秘钥加密值

[root@compute ~]# virsh secret-set-value --secret 207a92a6-acaf-47c2-9556-e560a79ba472 --base64 AQCIVY5beGJDNxAAphG+hdDC1vG4yVC5Ew7Y+w==

编辑cinder配置文件

[root@compute ~]# vi /etc/cinder/cinder.conf[DEFAULT]enabled_backends = ceph[ceph]volume_driver = cinder.volume.drivers.rbd.RBDDriverrbd_pool = ceph-cinderrbd_ceph_conf = /etc/ceph/ceph.confrbd_flatten_volume_from_snapshot = falserbd_max_clone_depth = 5rbd_store_chunk_size = 4rados_connect_timeout = -1glance_api_version = 2rbd_user = ceph-cinderrbd_secret_uuid = 207a92a6-acaf-47c2-9556-e560a79ba472# 这里的uuid是我们刚刚生成的uuid并注释掉[lvm]块下的所有配置

重启cinder服务

[root@compute ~]# systemctl restart openstack-cinder-volume

Controller节点操作

查看服务是否启动

 [root@compute ~]# ssh controller[root@controller ~]# source admin-openrc[root@controller ~]# cinder service-list+------------------+-----------------+------+---------+-------+----------------------------+-----------------+|      Binary      |       Host      | Zone |  Status | State |         Updated_at         | Disabled Reason |+------------------+-----------------+------+---------+-------+----------------------------+-----------------+| cinder-scheduler |    controller   | nova | enabled |   up  | 2018-09-04T10:37:36.000000 |        -        ||  cinder-volume   |   compute@ceph  | nova | enabled |   up  | 2018-09-04T10:37:37.000000 |        -        ||  cinder-volume   |  controller@lvm | nova | enabled |  down | 2018-09-03T09:04:57.000000 |        -        |+------------------+-----------------+------+---------+-------+----------------------------+-----------------+

查看到lvm存储是down状态，ceph存储是up状态

创建cinder卷

[root@controller ~]# cinder create --display-name vol-1 1+---------------------+--------------------------------------+|       Property      |                Value                 |+---------------------+--------------------------------------+|     attachments     |                  []                  ||  availability_zone  |                 nova                 ||       bootable      |                false                 ||      created_at     |      2018-09-04T10:43:50.386006      || display_description |                 None                 ||     display_name    |                vol-1                 ||      encrypted      |                False                 ||          id         | 418344cf-3955-479b-8d5e-94633abae1f8 ||       metadata      |                  {}                  ||     multiattach     |                false                 ||         size        |                  1                   ||     snapshot_id     |                 None                 ||     source_volid    |                 None                 ||        status       |               creating               ||     volume_type     |                 None                 |+---------------------+--------------------------------------+

查看卷是否创建成功

[root@controller ~]# cinder list+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+|                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+| 418344cf-3955-479b-8d5e-94633abae1f8 | available |    vol-1     |  1   |      -      |  false   |             |+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

这时候看到vol-1卷的状态是available

查看卷是否存储到ceph的存储池中

在ceph01上操作

[root@ceph01 ceph]# rbd ls ceph-cindervolume-418344cf-3955-479b-8d5e-94633abae1f8

能看到已经存储到ceph的存储池中，并且id也是相对应的

错误

创建卷失败了

查看/var/log/cinder/volume.log日志

oslo_messaging.rpc.dispatcher PermissionError: error creating image

cinder.volume.manager PermissionError: error creating image

发现是权限问题

后来查到是没有生成uuid，所以我就执行了导入操作到virsh中，成功创建卷