Ceph分布式存储安装

前言

参照官方文档中的快速安装，结合国内环境，将相关安装源修改为国内镜像源提高安装速度。

http://docs.ceph.org.cn/start/

基础系统环境

添加阿里云YUM源

mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
sed -i -e '/mirrors.cloud.aliyuncs.com/d' -e '/mirrors.aliyuncs.com/d' /etc/yum.repos.d/CentOS-Base.repo

更新系统

yum update -y```

### 关闭防火墙、SELinux

systemctl stop firewalld && systemctl disable firewalld
setenforce 0
sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config

### 添加hosts主机名解析（集群采用四台虚拟机） 
四台机器均添加

echo "192.168.5.191 admin-node
192.168.5.192 node1
192.168.5.193 node2
192.168.5.194 node3" >> /etc/hosts```

安装NTP服务，免因时钟漂移导致故障

yum install ntp ntpdate ntp-doc -y ```

### 免密钥登录

admin-node #到node1 node2 node3之间

ssh-keygen #生成密钥文件```
分别添加至三个node

hostname=(admin-node node1 node2 node3)
for i in ${hostname[*]};do echo yes|ssh-copy-id -i id_rsa.pub root@$i ;done

添加中科大源,否则 ceph-deploy安装ceph是会缺少依整包而报错

yum install -y yum-utils && yum-config-manager --add-repo http://mirrors.ustc.edu.cn/epel/7/x86_64/ &&  yum install --nogpgcheck -y epel-release &&  rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7 &&  rm -rf /etc/yum.repos.d/mirrors.ustc.edu.cn_epel*

添加 Ceph的国内源，安装ceph-deploy部署工具

cat >> /etc/yum.repos.d/ceph.repo << EOF 

[Ceph]
name=Ceph packages for $basearch
baseurl=http://mirrors.163.com/ceph/rpm-nautilus/el7/x86_64/
enabled=1
gpgcheck=0
type=rpm-md
gpgkey=http://mirrors.163.com/ceph/keys/release.asc
priority=1

[Ceph-noarch]
name=Ceph noarch packages
baseurl=http://mirrors.163.com/ceph/rpm-nautilus/el7/noarch/
enabled=1 
gpgcheck=0
type=rpm-md
gpgkey=http://mirrors.163.com/ceph/keys/release.asc
priority=1

[ceph-source]
name=Ceph source packages
baseurl=http://mirrors.163.com/ceph/rpm-nautilus/el7/SRPMS/
enabled=1
gpgcheck=0
type=rpm-md
gpgkey=http://mirrors.163.com/ceph/keys/release.asc
priority=1
EOF

yum -y update &&  yum -y install ceph-deploy```

### 使用ceph-deploy在所有节点安装软件包

#### 命令清除配置：

ceph-deploy purgedata {ceph-node} [{ceph-node}]
ceph-deploy forgetkeys

#### 连 Ceph 安装包一起清除
ceph-deploy purge {ceph-node} [{ceph-node}]

在管理节点(admin-node)

  `mkdir my-cluster && cd my-cluster`

 ##### ceph-deploy  new admin-node node1 node2 node3 创建集群。

sed -i '2a osd pool default size = 2' ceph.conf # 修改默认副本数从 3 改成 2
sed -i '3a public network = 10.0.5.0/24' ceph.conf

#### 修改ceph-deploy安装过程中的源

export CEPH_DEPLOY_REPO_URL=http://mirrors.163.com/ceph/rpm-nautilus/el7/
export CEPH_DEPLOY_GPG_URL=http://mirrors.163.com/ceph/keys/release.asc

ceph-deploy install admin-node node1 node2 node3

#### 生成密钥环 

ceph-deploy mon create-initial```

用 ceph-deploy 把配置文件和 admin 密钥拷贝到管理节点和 Ceph 节点

ceph-deploy admin admin-node node1 node2 node3```

#### 检查集群健康状态

ceph health ```

添加OSD进程

 ceph-deploy osd create node1 --data /dev/sdb
 ceph-deploy osd create node1 --data /dev/sdb
 ceph-deploy osd create node3 --data /dev/sdb