Hadoop-02 环境搭建

Hadoop 环境搭建

1. 安装jdk

2. 安装ssh 设置免密登录

   ssh-keygen -t rsa
   cp ~/.ssh/id_rsa_.pub ~/.ssh/authorized_keys

3. Hadoop 环境安装

   /etc/hadoop/hadoop-env.sh 修改java_home
   Hadoop可运行于一般的商用服务器上，具有高容错、高可靠性、高扩展性等特点特别适合写一次，读多次的场景。

   • 3.1 core-site.xml

   <configuration>
       <property>
           <name>fs.defaultFS</name>
           <value>hdfs://172.16.100.151:8020</value>
       </property>
       <property>
           <name>hadoop.tmp.dir</name>
           <value>/home/hadoop/app/tmp</value>
       </property>
   </configuration>

   • 3.2 hdfs-site.xml

   <configuration>
   <!--副本个数-->
       <property>
           <name>dfs.replication</name>
           <value>1</value>
       </property>
   </configuration>

解决安全模式问题 “hadoop Cannot create directory Name node is in safe mode.”

离开安全模式方法：bin/hadoop dfsadmin -safemode leave

yarn 环境搭建：

1. etc/hadoop/mapred-site.xml

   <configuration>
        <property>
            <name>mapreduce.framework.name</name>
            <value>yarn</value>
        </property>
    </configuration>

2. etc/hadoop/yarn-site.xml

   <configuration>
        <property>
            <name>yarn.nodemanager.aux-services</name>
            <value>mapreduce_shuffle</value>
        </property>
    </configuration>

服务启动：

sbin/start-yarn.sh

服务停止：

sbin/stop-yarn.sh

ResourceManager - http://localhost:8088/