当前位置 主页 > 服务器问题 > Linux/apache问题 >

    centos7搭建hadoop2.10高可用(HA)(5)

    栏目:Linux/apache问题 时间:2020-02-07 11:41

    hdfs haadmin常用命令:

    至此手动容灾高可用配置完成,但是这种方式不智能,不能够自动感知容灾,所以下面介绍自动容灾配置

    5.自动容灾配置

    需要引入zookeeperquarum 和 zk 容灾控制器(ZKFC)两个组件

      搭建zookeeper集群,选择s141,s142,s143三台机器,下载 zookeeper:http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.5.6

      1) 解压zookeeper:

    tar -xzvf apache-zookeeper-3.5.6-bin.tar.gz -C /opt/soft/zookeeper-3.5.6

      2) 配置环境变量,在/etc/profile中添加zk环境变量,并重新编译/etc/profile文件

    复制代码 代码如下:source /etc/profile

      3) 配置zk配置文件,三台机器配置文件统一

    # The number of milliseconds of each tick
    tickTime=2000
    # The number of ticks that the initial 
    # synchronization phase can take
    initLimit=10
    # The number of ticks that can pass between 
    # sending a request and getting an acknowledgement
    syncLimit=5
    # the directory where the snapshot is stored.
    # do not use /tmp for storage, /tmp here is just 
    # example sakes.
    dataDir=/home/hdfs/zookeeper
    # the port at which the clients will connect
    clientPort=2181
    # the maximum number of client connections.
    # increase this if you need to handle more clients
    #maxClientCnxns=60
    #
    # Be sure to read the maintenance section of the 
    # administrator guide before turning on autopurge.
    #
    # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
    #
    # The number of snapshots to retain in dataDir
    #autopurge.snapRetainCount=3
    # Purge task interval in hours
    # Set to "0" to disable auto purge feature
    #autopurge.purgeInterval=1
    server.1=s141:2888:3888
    server.2=s142:2888:3888
    server.3=s143:2888:3888

      4)分别

        在s141的/home/hdfs/zookeeper(在zoo.cfg配置文件中配置的dataDir路径)目录下创建myid文件,值为1(对应zoo.cfg配置文件中的server.1)

        在s142的/home/hdfs/zookeeper(在zoo.cfg配置文件中配置的dataDir路径)目录下创建myid文件,值为2(对应zoo.cfg配置文件中的server.2)

        在s143的/home/hdfs/zookeeper(在zoo.cfg配置文件中配置的dataDir路径)目录下创建myid文件,值为3(对应zoo.cfg配置文件中的server.3)

      5) 分别在每台机器上启动zk

    zkServer.sh start

      启动成功会出现zk进程:

    配置hdfs相关配置:

      1)停止hdfs所有进程

    stop-all.sh

      2)配置hdfs-site.xml,启用自动容灾.

    [hdfs-site.xml]
    <property>
     <name>dfs.ha.automatic-failover.enabled</name>
     <value>true</value>
    </property>

      3) 配置core-site.xml,指定zk的连接地址.

    <property>
     <name>ha.zookeeper.quorum</name>
     <value>s141:2181,s142:2181,s143:2181</value>
    </property>

      4) 分发以上两个文件到所有节点。