Centos7.5 Ambari2.7.4部署
目錄
?
?
1.簡介
2.環境準備
3.SSH 免密碼登錄
4.服務環境
5.本地源搭建
6.安裝
7.啟動
8.Web頁面部署服務
?
1.簡介
本文介紹了Ambari2.7.4在CentOS7.5中使用本地鏡像方式進行加速部署。
Ambari
Apache Ambari是一種基于Web的工具，支持Apache Hadoop集群的供應、管理和監控。Ambari已支持大多數Hadoop組件，包括HDFS、MapReduce、Hive、Pig、 Hbase、Zookeeper、Sqoop和Hcatalog等。
?
詳細官方說明內容參見
https://docs.cloudera.com/HDPDocuments/Ambari-2.7.4.0/bk_ambari-installation/content/ch_Installing_Ambari.html
?
2.環境準備
節點IP及映射
192.168.21.129 master
192.168.21.130 node1
192.168.21.131 node2
?
4臺均安裝好jdk
yum install -y java-1.8.0-openjdkvi /etc/profileexport JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el7_7.x86_64/jreexport PATH=$PATH:$JAVA_HOME/binexport CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jarsource /etc/profilefor a in {1..2} ; do scp /etc/profile node$a:/etc/profile ; donefor a in {1..2}; do ssh root@node$a source /etc/profile; done?
系統初始化參數
cat >> /etc/sysctl.conf << EOF fs.file-max=1000000 net.ipv4.tcp_max_tw_buckets = 6000 net.ipv4.tcp_sack = 1 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_rmem = 4096 87380 4194304 net.ipv4.tcp_wmem = 4096 16384 4194304 net.ipv4.tcp_max_syn_backlog = 16384 net.core.netdev_max_backlog = 32768 net.core.somaxconn = 32768 net.core.wmem_default = 8388608 net.core.rmem_default = 8388608 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_timestamps = 1 net.ipv4.tcp_fin_timeout = 20 net.ipv4.tcp_synack_retries = 2 net.ipv4.tcp_syn_retries = 2 net.ipv4.tcp_syncookies = 1 #net.ipv4.tcp_tw_len = 1 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_mem = 94500000 915000000 927000000 net.ipv4.tcp_max_orphans = 3276800 net.ipv4.ip_local_port_range = 1024 65000 net.nf_conntrack_max = 6553500 net.netfilter.nf_conntrack_max = 6553500 net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60 net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120 net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120 net.netfilter.nf_conntrack_tcp_timeout_established = 3600 net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 kernel.numa_balancing = 0 kernel.shmmax = 68719476736 kernel.printk = 5 kernel.sysrq = 1 vm.overcommit_memory = 0 vm.swappiness = 0 EOF文件描述符和用戶最大進程數
### 調整文件描述符 cat >> /etc/security/limits.conf <<EOF * soft nproc 65535 * hard nproc 65535 * soft nofile 65535 * hard nofile 65535 EOF ##用戶進程限制 cat >> /etc/security/limits.d/20-nproc.conf <<EOF * soft nproc 8192 root soft nproc unlimited EOF網絡配置
修改 hostname
master執行
hostnamectl set-hostname master依次修改所有節點?node[1-2]上分別執行
hostnamectl set-hostname node1?
vi /etc/hosts增加如下內容
192.168.21.129 master
192.168.21.130 node1
192.168.21.131 node2
?
for a in {1..2} ; do scp /etc/hosts node$a:/etc/hosts ; done?
?
3.SSH 免密碼登錄
1.在集群master的?/etc/ssh/sshd_config??文件去掉以下選項的注釋
vi /etc/ssh/sshd_config
RSAAuthentication yes #開啟私鑰驗證 PubkeyAuthentication yes #開啟公鑰驗證
2.將集群master 修改后的?/etc/ssh/sshd_config??通過?scp?命令復制發送到集群的每一個節點
for a in {1..2} ; do scp /etc/ssh/sshd_config node$a:/etc/ssh/sshd_config ; done
3.生成公鑰、私鑰
1.在集群的每一個節點節點輸入命令?ssh-keygen -t rsa -P ''，生成 key，一律回車
ssh-keygen -t rsa -P ''
4.在集群的master 節點輸入命令
將集群每一個節點的公鑰id_rsa.pub放入到自己的認證文件中authorized_keys;
ssh root@master cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys
for a in {1..2}; do ssh root@node$a cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys; done
5.在集群的master 節點輸入命令
將自己的認證文件?authorized_keys??通過?scp?命令復制發送到每一個節點上去:?/root/.ssh/authorized_keys`
for a in {1..2}; do scp /root/.ssh/authorized_keys root@node$a:/root/.ssh/authorized_keys ; done
6.在集群的每一個節點節點輸入命令
接重啟ssh服務
sudo systemctl restart sshd.service
7.驗證 ssh 無密登錄
開一個其他窗口測試下能否免密登陸
例如：在node3
ssh root@node2
exit?退出
?
4.服務環境
關閉SELINUX
vi /etc/selinux/config
將?SELINUX=enforcing?改為?SELINUX=disabled
設置后需要重啟才能生效
PS 我是修改node1?的?/etc/selinux/config?后，把配置文件復制到其他節點
for a in {1..2}; do scp /etc/selinux/config root@node$a:/etc/selinux/config ; done?
關閉防火墻（各臺均執行）
systemctl stop firewalld.servicesystemctl disable firewalld?
安裝配置 MySql
yum install -y wgetwget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpmrpm -ivh mysql57-community-release-el7-10.noarch.rpmyum -y install mysql-community-serversystemctl enable mysqldsystemctl start mysqld.servicesystemctl status mysqld.servicegrep "password" /var/log/mysqld.logmysql -uroot -p set global validate_password_policy=0;set global validate_password_length=1;set global validate_password_special_char_count=0;set global validate_password_mixed_case_count=0;set global validate_password_number_count=0;select @@validate_password_number_count,@@validate_password_mixed_case_count,@@validate_password_number_count,@@validate_password_length;ALTER USER 'root'@'localhost' IDENTIFIED BY 'password';grant all privileges on *.* to 'root'@'%' identified by 'password' with grant option;flush privileges;exit yum -y remove mysql57-community-release-el7-10.noarch?
下載mysql驅動，放到三臺的
/opt/ambari/mysql-connector-java-5.1.48.jar
?
?
5.本地源搭建
下載repo
https://docs.cloudera.com/HDPDocuments/Ambari-2.7.4.0/bk_ambari-installation/content/download_the_ambari_repo_lnx7.html
yum install -y wgetwget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0/ambari.repo -O /etc/yum.repos.d/ambari.repowget -nv http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/hdp.repo -O /etc/yum.repos.d/hdp.repo?
安裝httpd本地源
yum -y install httpdsystemctl restart httpdsystemctl enable httpd此時瀏覽器訪問http://ip/即可看到站點首頁
網站目錄為/var/www/html/，目前為空
下載
http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0/ambari-2.7.4.0-centos7.tar.gz
http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/HDP-3.1.4.0-centos7-rpm.tar.gz
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.4.0/HDP-GPL-3.1.4.0-centos7-gpl.tar.gz
mkdir /var/www/html/ambarimkdir /var/www/html/hdpmkdir /var/www/html/hdp/HDP-UTILS-1.1.0.22tar -zxvf ambari-2.7.4.0-centos7.tar.gz -C /var/www/html/ambari/tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz -C /var/www/html/hdp/tar -zxvf HDP-GPL-3.1.4.0-centos7-gpl.tar.gz -C /var/www/html/hdp/tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp/HDP-UTILS-1.1.0.22/?
修改
vi /etc/yum.repos.d/ambari.repo#VERSION_NUMBER=2.7.4.0-118
[ambari-2.7.4.0]
#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
name=ambari Version - ambari-2.7.4.0
baseurl=http://192.168.21.129/ambari/ambari/centos7/2.7.4.0-118
gpgcheck=1
gpgkey=http://192.168.21.129/ambari/ambari/centos7/2.7.4.0-118/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
?
?
?
6.安裝
https://docs.cloudera.com/HDPDocuments/Ambari-2.7.4.0/bk_ambari-installation/content/set_up_the_ambari_server.html
yum install ambari-server
ambari-server setup
?
不推薦直接用內嵌的postgresql，因為其他服務還要用mysql
?
[root@localhost download]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'enabled' SELinux mode is 'permissive' WARNING: SELinux is set to 'permissive' mode and temporarily disabled. OK to continue [y/n] (y)? y Customize user account for ambari-server daemon [y/n] (n)? y Enter user account for ambari-server daemon (root): Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 2 WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/jre Validating JDK on Ambari Server...done. Check JDK version for Ambari Server... JDK version found: 8 Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server. Checking GPL software agreement... GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? y Configuring database... ============================================================================== Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL / MariaDB [4] - PostgreSQL [5] - Microsoft SQL Server (Tech Preview) [6] - SQL Anywhere [7] - BDB ============================================================================== Enter choice (1): 3 Hostname (localhost): Port (3306): Database name (ambari): Username (ambari): Enter Database Password (bigdata): Configuring ambari database... Enter full path to custom jdbc driver: /opt/ambari/mysql-connector-java-5.1.48.jar Copying /opt/ambari/mysql-connector-java-5.1.48.jar to /usr/share/java Configuring remote database connection properties... WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql Proceed with configuring remote database connection properties [y/n] (y)? y Extracting system views... ..... Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it... Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully.?
?
初始化數據庫
mysql -uroot -p create database ambari;use ambarisource /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sqlCREATE USER 'ambari'@'localhost' IDENTIFIED BY 'bigdata';CREATE USER 'ambari'@'%' IDENTIFIED BY 'bigdata';GRANT ALL PRIVILEGES ON ambari.* TO 'ambari'@'localhost';GRANT ALL PRIVILEGES ON ambari.* TO 'ambari'@'%';FLUSH PRIVILEGES;?
?
7.啟動
僅從master機器上啟動即可：
ambari-server start出現Server started listening on 8080
表示成功
?
如果主機啟動監聽8080，但超時失敗，需要netstat -tunlp找到8080端口的進程號，kill -9 pid停掉已啟動的占用端口進程方可。
?
8.Web頁面部署服務
進入web頁面繼續部署
訪問
http://192.168.81.147:8080/
用戶名密碼都是admin
?
點擊 “LAUNCH INSTALL WIZARD”，開始創建一個集群
起一個名字 下一步
?
Repositories里移除掉其他系統，只保留redhat7，修改為本地源，下一步
?
http://192.168.21.129/hdp/HDP/centos7/3.1.4.0-315
http://192.168.21.129/hdp/HDP-GPL/centos7/3.1.4.0-315
http://192.168.21.129/hdp/HDP-UTILS-1.1.0.22/HDP-UTILS/centos7/1.1.0.22
?
Target Hosts里填寫其他IP，如
master
node1
node2
SSH Private Key這個文本框里，把cat /root/.ssh/id_rsa 的內容拷貝過來
?
下一步等待安裝
?
?
安裝第一次有可能失敗，
從機報錯
Host registration aborted. Ambari Agent host cannot reach Ambari Server 'localhost:8080'. Please check the network connectivity between the Ambari Agent host and the Ambari Server
看上面日志，scp拷貝/usr/lib/ambari-server/lib/ambari_server/setupAgent.py這個文件，然后執行
vi /usr/lib/ambari-server/lib/ambari_server/setupAgent.py
編輯下這個文件
(expected_hostname, passPhrase, hostname, user_run_as, projectVersion, server_port) = retcode["parsed_args"]
hostname="192.168.21.129"
server_port=8080
retcode = checkServerReachability(hostname, server_port)
添加標紅一行，指定hostname
?
再retry一次就好了
?
報錯
Ambari agent machine hostname (localhost.localdomain) does not match expected ambari server hostname (node1). Aborting registration. Please check hostname, hostname -f and /etc/hosts file to confirm your hostname is setup correctly
一臺主機名不對（漏掉了），改名即可
?
#三臺從機上會出現/var/lib/ambari-agent/目錄
?
下一步選擇需要安裝的服務，此步驟選擇一些自己需要的服務，
?
下一步下一步輸入密碼再下一步
適當的分配下服務器上的服務，下一步
各種下一步……Install, Start and Test界面等待安裝
?
?
Hive Client安裝報錯
Failed to download file from http://master:8080/resources/mysql-connector-java.jar due to HTTP error: HTTP Error 404: Not Found
解決
cd /var/lib/ambari-server/resources/
ln /opt/ambari/mysql-connector-java-5.1.48.jar mysql-connector-java.jar
?
安裝完畢如下
?
下一步進入系統頁面
?
維護
重啟系統后ambari-server和ambari-agent會自動啟動
查看狀態
ambari-server status
ambari-agent status
但是內部其他服務可能會啟動失敗，一片紅
可以沿著服務依賴關系啟動
Zookeeper——HDFS——……
或者直接StartAll啟動所有服務
?
?
總結
以上是生活随笔為你收集整理的Centos7.5 Ambari2.7.4部署的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： smartadmin mysql_sma
- 下一篇： （一）Neo4j在Centos7虚拟机上