當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

CENTOS6.5安装CDH5.12.1(一) https://mp.weixin.qq.com/s/AP_m0QqKgzEUfjf0PQCX-w

發布時間：2024/3/24 编程问答 41 豆豆

生活随笔收集整理的這篇文章主要介紹了 CENTOS6.5安装CDH5.12.1(一) https://mp.weixin.qq.com/s/AP_m0QqKgzEUfjf0PQCX-w 小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

CENTOS6.5安裝CDH5.12.1(一)

原創：?Fayson?Hadoop實操?2017-09-13

溫馨提示：要看高清無碼套圖，請使用手機打開并單擊圖片放大查看。

1.概述

本文檔主要描述離線環境下，CentOS6.5操作系統部署CDH5.12.1企業版的過程。本安裝文檔主要分為4個步驟：

1.前期準備（包括hostname、關閉防火墻、關閉SELinux、時鐘同步等）

2.安裝Cloudera Manager Server

3.安裝CDH集群

4.集群完整性檢查（包括HDFS文件系統、MapReduce、Hive等服務是否可以正常運行）

這篇文檔將著重介紹Cloudera管理器與CDH的安裝，并基于以下假設：

1.操作系統版本：CentOS6.5

2.MySQL版本：5.1.73

3.CM版本：CM 5.12.1

4.CDH版本：CDH 5.12.1

5.采用root對集群進行部署

6.您已經下載CDH和CM的安裝包（CDH使用parcels包方式安裝）

2.前期準備

2.1hostname及hosts配置

集群中各個節點之間能互相通信使用靜態IP地址。IP地址和主機名通過/etc/hosts配置，主機名/etc/hostname進行配置（推薦hostname使用FQDN完全域名的方式配置）。

以服務器（172.31.6.148）為例：

hostname配置

修改/etc/sysconfig/network文件：

[root@ip-172-31-6-148~]# vim /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=ip-172-31-6-148.fayson.com

注意：修改hostname后需要重啟服務器。

檢查hostname是否修改成功：

[root@ip-172-31-6-148~]# hostname
ip-172-31-6-148.fayson.com
[root@ip-172-31-6-148?~]#?

hosts配置

/etc/hosts文件如下:

[root@ip-172-31-6-148~]# more /etc/hosts
127.0.0.1???????????????localhost.localdomain localhost
::1?????????????localhost6.localdomain6 localhost6

172.31.6.148??ip-172-31-6-148.fayson.com
172.31.5.190??ip-172-31-5-190.fayson.com
172.31.10.118?ip-172-31-10-118.fayson.com
172.31.9.33???ip-172-31-9-33.fayson.com
[root@ip-172-31-6-148?~]#?

以上兩步操作，在集群中其它節點做相應配置。

2.2禁用SELinux

在所有節點執行sudo setenforce 0?命令

[root@ip-172-31-6-148~]# sudo setenforce?0

修改集群所有節點的/etc/selinux/config文件，內容如下：

[root@ip-172-31-6-148~]# vim /etc/selinux/config?
# This file controls the state ofSELinux on the system.
# SELINUX= can take one of these three values:
#???? enforcing - SELinux security policyis enforced.
#???? permissive - SELinux printswarnings instead of enforcing.
#???? disabled - No SELinux policy isloaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these twovalues:
#???? targeted - Targeted processes areprotected,
#???? mls - Multi Level Securityprotection.
SELINUXTYPE=targeted

2.3關閉防火墻

在集群所有節點執行如下操作，并永久關閉防火墻

[root@ip-172-31-6-148~]# service iptables stop
iptables:?Setting chains to policy ACCEPT: filter??????????[??OK??]
iptables:?Flushing firewall rules:?????????????????????????[??OK??]
iptables:?Unloading modules:???????????????????? ??????????[??OK??]
[root@ip-172-31-6-148?~]# chkconfig iptables off
[root@ip-172-31-6-148?~]#?

2.4配置操作系統本地yum源

掛載操作系統iso文件

[root@ip-172-31-6-148~]# mkdir /mnt/iso
[root@ip-172-31-6-148?~]# mount -o loop CentOS-6.5-x86_64-minimal.iso/mnt/iso/
[root@ip-172-31-6-148?~]# df -h
Filesystem???????????????????????????Size? UsedAvail Use% Mounted on
/dev/xvde?????????????????????????????99G?3.4G?? 91G?? 4% /
tmpfs????????????????????????????????7.4G?????0??7.4G?? 0%/dev/shm
/root/CentOS-6.5-x86_64-minimal.iso??398M? 398M????0?100% /mnt/iso
[root@ip-172-31-6-148?~]#?

配置操作系統repo

[root@ip-172-31-6-148~]# vim /etc/yum.repos.d/os.repo
[localrepo]
name=CentOS6.5
baseurl=file:///mnt/iso
gpgcheck=false
enabled=true

查看yum源列表是否有新建的localrepo

[root@ip-172-31-6-148~]# yum repolist
Loaded?plugins: fastestmirror, presto
Loading?mirror speeds from cached hostfile
?*?base: mirror.0x.sg
?*?extras: mirror.0x.sg
?*?updates: mirror.0x.sg
localrepo?????????????????????????????????????????????????????????????????????????????????????|?3.6?kB???? 00:00...?
localrepo/primary_db??????????????????????????????????????????????????????????????????????????|?492?kB????00:00 ...?
repo?id???????????????????????????????????????? reponame????????????????????????? ?????????????????????????????status
base????????????????????????????????????????????CentOS-6 - Base????????????????????????????????????????????????6,706
cloudera-manager????????????????????????????????Cloudera Manager, Version 5.12.0????????????????????? ??????????????7
cmrepo??????????????????????????????????????????cm_repo????????????????????????????????????????????????????????????7
extras??????????????????????????????????????????CentOS-6 - Extras?????????????????????????????????????????????????45
localrepo???????????????????????????????????????CentOS6.5????????????????????????????????????????????????????????248
updates?????????????????????????????????????????CentOS-6 - Updates???????????????????????????????????????????????589
repolist:?7,602
[root@ip-172-31-6-148?~]#?

2.5安裝http服務

使用yum安裝http服務

在集群其中一節點上安裝http服務，執行如下命令：

[root@ip-172-31-6-148~]# yum -y install httpd

如上圖所示則表明安裝成功。

將httpd服務加入系統自啟動服務并設置開機啟動

[root@ip-172-31-6-148~]# chkconfig --add httpd
[root@ip-172-31-6-148?~]# chkconfig httpd on
[root@ip-172-31-6-148?~]# chkconfig --list |grep?httpd
httpd???????????0:off?? 1:off?? 2:on???3:on??? 4:on??? 5:on???6:off
[root@ip-172-31-6-148?~]#?

啟動http服務

[root@ip-172-31-6-148~]# service httpd start
Starting?httpd:????????????????????????????????????????????[??OK??]

2.6重做操作系統yum源

在2.5安裝了httpd服務后，重新制作操作系統的yum源，使用http的方式，操作如下：

在/var/www/html目錄下創建centos6.5目錄

[root@ip-172-31-6-148html]# mkdir centos6.5
[root@ip-172-31-6-148?html]# ll
total?4
drwxr-xr-x?2?rootroot?4096?Sep??4?16:04 centos6.5
[root@ip-172-31-6-148?html]#?

將/mnt/iso目錄下的所有文件拷貝至/var/www/html/centos6.5目錄下

[root@ip-172-31-6-148html]# scp -r /mnt/iso/*/var/www/html/centos6.5/

修改/etc/yum.repo.d/os.repo配置

[root@ip-172-31-6-148~]# vim /etc/yum.repos.d/os.repo?
[localrepo]
name=CentOS6.5
baseurl=http://ip-172-31-6-148.fayson.com/centos6.5
gpgcheck=false
enabled=true

檢查yum源是否配置正常

將/etc/yum.repo.d/os.repo配置文件同步到集群所有節點的/etc/yum.repo.d/目錄下

2.7集群時鐘同步

在集群的所有服務器上安裝ntp服務，用于集群時鐘同步，此處選擇ip-172-31-6-148.fayson.com服務器作為本地NTP服務器，集群其它節點與其保持同步，配置如下：

在集群所有節點安裝ntp服務，命令如下

[root@ip-172-31-6-148~]# yum -y install ntp

如上圖顯示，則說明安裝成功。

將ntpd加入系統自啟動服務并設置開機啟動

在所有節點執行如下操作：

[root@ip-172-31-6-148~]# chkconfig --add ntpd
[root@ip-172-31-6-148?~]# chkconfig ntpd on

在ip-172-31-6-148.fayson.com上配置與自己同步

[root@ip-172-31-6-148~]# vim /etc/ntp.conf?
...
# Use public servers from thepool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
server??127.127.1.0?????#local clock
fudge???127.127.1.0 stratum?10

集群其它節點配置如下

[root@ip-172-31-5-190~]# vim /etc/ntp.conf?
...
# Use public servers from thepool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
server?172.31.6.148 iburst

重啟集群所有節點的ntpd服務

[root@ip-172-31-6-148~]# service ntpd restart
Shutting?down ntpd:????????????????????????????????????????[??OK??]
Starting?ntpd:?????????????????????????????????????????????[??OK??]
[root@ip-172-31-6-148?~]#?

驗證時鐘同步，在所有節點執行ntpq -p命令

[root@ip-172-31-6-148~]# ntpq -p
?????remote???????????refid????? stt when poll reach?? delay?? offset?jitter
==============================================================================
*LOCAL(0)????????.LOCL.?????????10?l???15?? 64??? 7????0.000???0.000?? 0.001
[root@ip-172-31-6-148?~]# ssh -i fayson.pem.txtip-172-31-5-190.fayson.com?"ntpq-p"
?????remote? ?????????refid????? st t when pollreach?? delay?? offset?jitter
==============================================================================
*ip-172-31-6-148?LOCAL(0)????????11?u???68? 128?377????0.274??? 1.005?? 0.243
[root@ip-172-31-6-148?~]# ssh -i fayson.pem.txtip-172-31-10-118.fayson.com?"ntpq-p"???????????????
?????remote???????????refid????? stt when poll reach?? delay?? offset?jitter
==============================================================================
*ip-172-31-6-148?LOCAL(0)??? ????11?u??75?128? 377????0.285??? 1.214??0.213
[root@ip-172-31-6-148?~]# ssh -i fayson.pem.txt ip-172-31-9-33.fayson.com"ntpq -p"????????????????
?????remote???????????refid????? stt when poll reach?? delay?? offset?jitter
==============================================================================
*ip-172-31-6-148?LOCAL(0)????????11?u????1?? 64?367????0.294?? -9.249?? 1.511
[root@ip-172-31-6-148?~]#?

注：標記部分，有“*”顯示則表示同步成功。

2.8安裝MySQL

安裝MySQL

[root@ip-172-31-6-148~]# yum -y install mysql mysql-server

如上截圖則表示MySQL安裝成功。

將mysqld加入系統自啟動服務并設置開機啟動

[root@ip-172-31-6-148~]# chkconfig --add mysqld
[root@ip-172-31-6-148?~]# chkconfig mysqld on

啟動并配置Mysql

[root@ip-172-31-6-148~]# service mysqld start
...
??????????????????????????????????????????????????????????[?OK??]
Starting?mysqld:???????????????????????????????????????????[??OK??]

初始化MySQL

[root@ip-172-31-6-148~]# mysql_secure_installation?

NOTE:?RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
??????SERVERS?IN PRODUCTIONUSE!? PLEASE READ EACH STEP CAREFULLY!

In?order to log into MySQL to secure it, we'll need the current
password for the root user.? If you've just installed MySQL, and
you?haven't set the root password yet, the passwordwill be blank,
so you should just press enter here.

Enter current password for root (enter for none):?
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.

Set root password? [Y/n]?y
New password:?
Re-enter new password:?
Password updated successfully!
Reloading privilege tables..
?... Success!

By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them.? This is intended only for testing,and to make the installation
go a bit smoother.? You should removethem before moving into a
production environment.

Remove anonymous users? [Y/n]?y
?... Success!

Normally, root should only be allowed to connect from 'localhost'.? This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n]?n
?... skipping.

By default, MySQL comes with a database named 'test' that anyone can
access.? This is also intended only fortesting, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n]?y
?- Dropping test database...
?... Success!
?- Removing privileges on test database...
?... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n]?y
?... Success!

Cleaning up...

All done!? If you've completed all of the above steps, your MySQL
installation?should now be secure.

Thanks?for?usingMySQL!

[root@ip-172-31-6-148?~]#?

注意：標紅部分為輸入項，出現“Thanks for using MySQL!”則表示初始化成功。

創建CM及CDH服務的數據庫

建庫語句：

create?database metastore default character set utf8;
CREATE?USER?'hive'@'%'?IDENTIFIED BY?'password';
GRANT?ALL PRIVILEGES ON metastore. * TO?'hive'@'%';
FLUSH?PRIVILEGES;

create?database cm default character set utf8;
CREATE?USER?'cm'@'%'?IDENTIFIED BY?'password';
GRANT?ALL PRIVILEGES ON cm. * TO?'cm'@'%';
FLUSH?PRIVILEGES;

create?database am default character set utf8;
CREATE?USER?'am'@'%'?IDENTIFIED BY?'password';
GRANT?ALL PRIVILEGES ON am. * TO?'am'@'%';
FLUSH?PRIVILEGES;

create?database rm default character set utf8;
CREATE?USER?'rm'@'%'?IDENTIFIED BY?'password';
GRANT?ALL PRIVILEGES ON rm. * TO?'rm'@'%';
FLUSH?PRIVILEGES;

create?database hue default character set utf8;
CREATE?USER?'hue'@'%'?IDENTIFIED BY?'password';
GRANT?ALL PRIVILEGES ON hue. * TO?'hue'@'%';
FLUSH?PRIVILEGES;

create?database oozie default character set utf8;
CREATE?USER?'oozie'@'%'?IDENTIFIED BY?'password';
GRANT?ALL PRIVILEGES ON oozie. * TO?'oozie'@'%';
FLUSH?PRIVILEGES;

命令行操作如下：

[root@ip-172-31-6-148~]# mysql -uroot -p
Enter?password:?
...
mysql> create?database metastore default character set utf8;
Query?OK,?1?row affected (0.00?sec)
mysql> CREATE?USER?'hive'@'%'?IDENTIFIEDBY?'password';
Query?OK,?0?rows affected (0.00?sec)
mysql> GRANT?ALL?PRIVILEGES ON metastore. * TO?'hive'@'%';
Query?OK,?0?rows affected (0.00?sec)
mysql> FLUSH?PRIVILEGES;
Query?OK,?0?rows affected (0.00?sec)
mysql>?
mysql> create?databasecm default character set utf8;
Query?OK,?1?row affected (0.00?sec)
mysql> CREATE?USER?'cm'@'%'?IDENTIFIEDBY?'password';
Query?OK,?0?rows affected (0.00?sec)
mysql> GRANT?ALL?PRIVILEGES ON cm. * TO?'cm'@'%';
Query?OK,?0?rows affected (0.00?sec)
mysql> FLUSH?PRIVILEGES;
Query?OK,?0?rows affected (0.00?sec)
mysql> create?databaseam default character set utf8;
Query?OK,?1?row affected (0.00?sec)
mysql> CREATE?USER?'am'@'%'?IDENTIFIEDBY?'password';
Query?OK,?0?rows affected (0.00?sec)
mysql> GRANT?ALL?PRIVILEGES ON am. * TO?'am'@'%';
Query?OK,?0?rows affected (0.00?sec)
mysql> FLUSH?PRIVILEGES;
Query?OK,?0?rows affected (0.00?sec)
mysql>?
mysql> create?databaserm default character set utf8;
Query?OK,?1?row affected (0.00?sec)
mysql> CREATE?USER?'rm'@'%'?IDENTIFIEDBY?'password';
Query?OK,?0?rows affected (0.00?sec)
mysql> GRANT?ALL?PRIVILEGES ON rm. * TO?'rm'@'%';
Query?OK,?0?rows affected (0.00?sec)
mysql> FLUSH?PRIVILEGES;
Query?OK,?0?rows affected (0.00?sec)
mysql>?
mysql> create?databasehue default character set utf8;
Query?OK,?1?row affected (0.00?sec)
mysql> CREATE?USER?'hue'@'%'?IDENTIFIEDBY?'password';
Query?OK,?0?rows affected (0.00?sec)
mysql> GRANT?ALL?PRIVILEGES ON hue. * TO?'hue'@'%';
Query?OK,?0?rows affected (0.00?sec)
mysql> FLUSH?PRIVILEGES;
Query?OK,?0?rows affected (0.00?sec)
mysql>?
mysql> create?databaseoozie default character set utf8;
Query?OK,?1?row affected (0.00?sec)
mysql> CREATE?USER?'oozie'@'%'?IDENTIFIEDBY?'password';
Query?OK,?0?rows affected (0.00?sec)
mysql> GRANT?ALL?PRIVILEGES ON oozie. * TO?'oozie'@'%';
Query?OK,?0?rows affected (0.00?sec)
mysql> FLUSH?PRIVILEGES;
Query?OK,?0?rows affected (0.00?sec)

mysql>

安裝jdbc驅動

[root@ip-172-31-6-148~]# mkdir -p /usr/share/java
[root@ip-172-31-6-148?~]# mv mysql-connector-java-5.1.34.jar/usr/share/java/
[root@ip-172-31-6-148?~]# cd /usr/share/java/
[root@ip-172-31-6-148?java]# ln -s mysql-connector-java-5.1.34.jar mysql-connector-java.jar?????????
[root@ip-172-31-6-148?java]# ll
total?940
-rwxr-xr-x.?1?root root?960372?Aug?30?15:16 mysql-connector-java-5.1.34.jar
lrwxrwxrwx??1root root?????31?Sep?4?16:32 mysql-connector-java.jar -> mysql-connector-java-5.1.34.jar
[root@ip-172-31-6-148?java]#?

3.Cloudera Manager安裝

3.1配置CM本地repo源

CM5.12.1下載地址：

http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/cloudera-manager-agent-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/cloudera-manager-daemons-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/cloudera-manager-server-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/cloudera-manager-server-db-2-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/enterprise-debuginfo-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/jdk-6u31-linux-amd64.rpm
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.12.1/RPMS/x86_64/oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm

將以上7個文件下載至服務器的/var/www/html/cm5.12.1目錄下

[root@ip-172-31-6-148~]# mkdir -p /var/www/html/cm5.12.1
[root@ip-172-31-6-148?~]# cd /var/www/html/cm5.12.1/
[root@ip-172-31-6-148?cm5.12.1]# ll
total?942088
-rw-r--r--?1?rootroot???9676820?Aug?30?14:45cloudera-manager-agent-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm
-rw-r--r--?1?rootroot?709818804?Aug?30?14:45 cloudera-manager-daemons-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm
-rw-r--r--?1?rootroot??????8692?Aug?30?14:45cloudera-manager-server-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm
-rw-r--r--?1?rootroot?????10600?Aug?30?14:45cloudera-manager-server-db-2-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm
-rw-r--r--?1?rootroot??31918952?Aug?30?14:45enterprise-debuginfo-5.12.1-1.cm5121.p0.6.el6.x86_64.rpm
-rw-r--r--?1?rootroot??71204325?Aug?30?14:45jdk-6u31-linux-amd64.rpm
-rw-r--r--?1?rootroot?142039186?Aug?30?14:45 oracle-j2sdk1.7-1.7.0+update67-1.x86_64.rpm
[root@ip-172-31-6-148?cm5.12.1]#?

在/var/www/html/cm5.12.1目錄下執行命令

[root@ip-172-31-6-148?cm5.12.1]# pwd
/var/www/html/cm5.12.1
[root@ip-172-31-6-148?cm5.12.1]#createrepo .
Spawning?worker?0?with?7?pkgs
Workers?Finished
Gathering?worker results

Saving?Primary metadata
Saving?file lists metadata
Saving?other metadata
Generating?sqlite DBs
Sqlite?DBs complete
[root@ip-172-31-6-148?cm5.12.1]#?

確認http是否能正常訪問

在/etc/yum.repo.d/目錄下增加cm.repo文件，內容如下

[root@ip-172-31-6-148~]# vim /etc/yum.repos.d/cm.repo
[cmrepo]
name=CM5.12.1
baseurl=http://ip-172-31-6-148.fayson.com/cm5.12.1
gpgcheck=false
enabled=true

驗證CM源是否配置成功

[root@ip-172-31-6-148~]# yum repolist
...
cmrepo?????????????????????????????????????????????CM5.12.1????????????????????????????????????????????????????????7
localrepo??????????????????????????????????????????CentOS6.5?????????????????????????????????????????????????????248

驗證安裝JDK

[root@ip-172-31-6-148?~]#yum -y install oracle-j2sdk1.7-1.7.0+update67-1

3.2配置http訪問CDH Parcel包

CDH5.12.1下載地址：

http://archive.cloudera.com/cdh5/parcels/5.12.1/CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel
http://archive.cloudera.com/cdh5/parcels/5.12.1/CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel.sha1
http://archive.cloudera.com/cdh5/parcels/5.12.1/manifest.json

將以上3個文件下載至/var/www/html/cdh5.12.1目錄下

[root@ip-172-31-6-148~]# mkdir -p /var/www/html/cdh5.12.1
[root@ip-172-31-6-148?~]# cd /var/www/html/cdh5.12.1/
[root@ip-172-31-6-148?cdh5.12.1]# ll
total?1582508
-rw-r--r--?1?rootroot?1620405492?Aug?30?19:09 CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel
-rw-r--r--?1?rootroot?????????41?Aug?30?19:09 CDH-5.12.1-1.cdh5.12.1.p0.3-el6.parcel.sha1
-rw-r--r--?1?rootroot??????72312?Aug?30?19:11 manifest.json
[root@ip-172-31-6-148?cdh5.12.1]#?

確認http是否能正常訪問

3.3安裝Cloudera Manager Server

通過yum安裝ClouderaManager Server

[root@ip-172-31-6-148~]# yum -y installcloudera-manager-server

如上圖所示則表明安裝成功。

初始化CM數據庫

[root@ip-172-31-6-148~]#/usr/share/cmf/schema/scm_prepare_database.sh mysql cm cm password
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
Verifying?that we can write to /etc/cloudera-scm-server
Creating?SCM configuration file?in?/etc/cloudera-scm-server
Executing:??/usr/java/jdk1.7.0_67-cloudera/bin/java-cp/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/schema/../lib/*com.cloudera.enterprise.dbutil.DbCommandExecutor/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
[??????????????????????????main] DbCommandExecutor????????????? INFO? Successfully connected to database.
All?done,?your SCM database is configured correctly!
[root@ip-172-31-6-148?~]#?

啟動ClouderaManager Server

[root@ip-172-31-6-148~]# service cloudera-scm-server start
Starting?cloudera-scm-server:??????????????????????????????[??OK??]
[root@ip-172-31-6-148?~]#

檢查端口是否監聽

[root@ip-172-31-6-148~]# netstat -lnpt |?grep?7180
tcp????????0????? 0?0.0.0.0:7180???????????????0.0.0.0:*??????????????????LISTEN????? 4056/java???????????
[root@ip-172-31-6-148?~]#?

通過http:// 52.221.194.202:7180/cmf/login訪問CM控制臺

4.CDH安裝

4.1CDH集群安裝向導

1.admin/admin登錄到CM

2.同意license協議，點擊繼續

3.選擇60試用，點擊繼續

4.點擊“繼續”

5.輸入主機ip或者名稱，點擊搜索找到主機后點擊繼續

6.點擊“繼續”

7.使用parcel選項，點擊“更多選項”,點擊“-”刪除其它所有地址，輸入

http:// ip-172-31-6-148.fayson.com/cdh5.12.1/點擊“保存更改”

8.選擇自定義存儲庫，輸入cm的http地址

9.點擊“繼續”，進入下一步安裝jdk

10.點擊“繼續”，進入下一步，默認多用戶模式

11.點擊“繼續”，進入下一步配置ssh賬號密碼

12.點擊“繼續”，進入下一步，安裝Cloudera Manager相關到各個節點

13.點擊“繼續”，進入下一步安裝cdh到各個節點

14.點擊“繼續”，進入下一步主機檢查，確保所有檢查項均通過

點擊完成進入服務安裝向導。

4.2集群設置安裝向導

1.選擇需要安裝的服務

此處使用自定義服務，如下圖

2.點擊“繼續”，進入集群角色分配

HDFS角色分配：

Hive角色分配：

Cloudera Manager Service角色分配：

Spark角色分配：（Spark on Yarn所以沒有spark的master和worker角色）

Yarn角色分配：

Zookeeper角色分配：（至少3個Server）

3.角色分配完成點擊“繼續”，進入下一步，測試數據庫連接

4.測試成功，點擊“繼續”，進入目錄設置，此處使用默認默認目錄，根據實際情況進行目錄修改

5.點擊“繼續”，等待服務啟動成功

6.點擊“繼續”，顯示集群安裝成功

7.安裝成功后進入home管理界面

醉酒鞭名馬，少年多浮夸！嶺南浣溪沙，嘔吐酒肆下！摯友不肯放，數據玩的花！

溫馨提示：要看高清無碼套圖，請使用手機打開并單擊圖片放大查看。

歡迎關注Hadoop實操，第一時間，分享更多Hadoop干貨，喜歡請關注分享。

原創文章，歡迎轉載，轉載請注明：轉載自微信公眾號Hadoop實操

閱讀?1236 7 精選留言

作者已設置關注后才可以留言

?1 Duke 測試數據庫連接的時候一個可以連接通過，其他幾個死活通不過，麻煩可以看看嗎？User cannot run DDL statements on the specified database. Attempt to create and drop a table failed
?1 梁豐額，我安裝完畢，用您的安裝方式安裝的java是1.6的啊，怎么升級呢 ? 作者我的安裝方式是1.7的，注意cm的rpm包一共是7個，有一個就是jdk1.7，在安裝cm的其他包比如agent或者server之前，需要你自己先安裝那個1.7的包。
? 祁技超為啥還是jdk6 不是推薦java8嗎？ ?1 作者是jdk1.7，不是1.6，CDH很早就支持1.8了，安裝的時候可以選擇的，當然也可以后期升級為1.8，CDH現在最新的Spark2.2就只支持jdk1.8了。本公眾號最近會推出基于JDK1.8安裝CDH，以及如何將CDH從1.7升級為1.8。歡迎關注。
? robin 請教大師一個問題，在文章中"2.4配置操作系統本地yum源" 為什么要掛載操作系統iso文件，這一步未能理解。 ? 作者你沒有iso文件，就沒有本地的yum倉庫，就無法安裝比如ntp，mysql等這些，另外在安裝cloudera agent的時候也需要安裝一些OS的依賴也會失敗。
? 涂艷偉你好，我給我的虛擬機擴大了磁盤容量，但是CM頁面上還顯示原來的大小，改怎么弄
? 涂艷偉你好，我安裝的時候老是有一個節點停留在正在刷新元數據包，試了很多次都是這樣 ? 作者那個節點yum源有問題，自己檢查一下。
? Baddy 大佬，這種安裝方式可以用于生產環境么？？ ? 作者可以。
? Baddy 2.4到2.6我看不懂那是干啥作用是啥，求解
? Baddy CDH那個版本比較穩定啊，還有那個CDH的HA安裝我就是沒找到 ? 作者 ha是后面再做的，歷史文章里有，你點‘必點’去找吧
? 魁子請問大神有沒有cm安裝cdh時候選擇HA安裝（Namenode和resourcemanager）文檔？ ? 作者 ha是后面再做的，歷史文章里有，你點‘必點’去找吧
? 了想問如果是在這里裝的虛擬機，想分成多臺，一般多少內存可以支持 ? 作者你總共16G內存，就別分了，就一臺吧，玩玩就行了。配置其實很低了。我們一般最少要準備4核16G的機器4臺才開始玩。
? 了想問樓主，機器應該是什么配置的，我現在有2臺電腦，都是16G內存，一個12線程，一個4線程，怎么搭配虛擬機環境，配幾臺虛擬機合適 ? 作者四線程的安裝cm，12線程裝nn和dn，否則至少弄四臺吧。
? 朝霞之巔比較尷尬的是在4.1第14步后，遭遇了停電，現在重新進去cm控制臺，發現沒法進入4.2安裝步驟了 ? 作者 1.到了4.1說明你cloudera-scm-server安裝成功了的，停電了，服務器起來，還是能進clouderas-scm-server的，可以添加“群里”來繼續完成。 2.或者你可以卸載了重做，
? Healthy國健遇到報錯了，求救大神～error:package:cloudera-manager-agent-5.12.1-1.cm5121.p0.6.el6.x86_64(cloudera-manager) Requires:libpython2.6.so.1.0()(64bit) ? 作者看看OS的yum源有沒有問題呢。
? 驪 4.2之前的步驟都執行正常，4.2后啟動服務，顯示啟動失敗，該怎么解決。日志報錯java文件無權限 ? 作者有沒有異常日志呢？這個說的范圍太泛不太好定位
? 梁豐好的謝謝
? 梁豐我自己配了應該也不會有問題吧 ? 作者沒問題的。
? 梁豐機器之間不配置免密登陸嗎 ? 作者不用配置的。
? 問心生產環境要用kudu，哪個版本最好？ ? 作者 CDH5.12.1，Kudu1.4
? 祁技超你試過大于121的嗎？
? 祁技超我現在都配java8了……有個問題請教一下大神有必要嚴格按照文檔建議嗎？貌似不支持最新java8 ? 作者 JDK8就用CDH官網給的1.8u121，1.8u111，1.8u102，1.8u91，1.8u74，1.8u31。

轉載于:https://www.cnblogs.com/gaoyuechen/p/10305703.html

總結

以上是生活随笔為你收集整理的CENTOS6.5安装CDH5.12.1(一) https://mp.weixin.qq.com/s/AP_m0QqKgzEUfjf0PQCX-w的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。

上一篇： aria2的使用
下一篇：中国科学院大学计算机录取比,2020-2