當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

HBase环境搭建与使用

發布時間：2024/9/30 编程问答 27 豆豆

生活随笔收集整理的這篇文章主要介紹了 HBase环境搭建与使用小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

環境

Hadoop集群
Zookeeper單機
Hbase單機

開發環境可以關注 “后端碼匠” 回復電腦環境獲取，也可自行下載，也很快的.

環境變量

可自行度娘

#JAVA_HOME export JAVA_HOME=/usr/java/jdk1.8.0_221 export PATH=$PATH:$JAVA_HOME/bin export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar#HADOOP_HOME export HADOOP_HOME=/opt/module/hadoop-3.1.1 export PATH=$PATH:$HADOOP_HOME/bin export PATH=$PATH:$HADOOP_HOME/sbin#ZOOKEEPER_HOME export ZOOKEEPER_HOME=/opt/module/zookeeper-3.5.6 export PATH=.:$HADOOP_HOME/bin:$ZOOKEEPER_HOME/bin:$JAVA_HOME/bin:$PATH

配置

Hadoop配置在另外一條推文已經寫過了

Hadoop集群搭建

Zookeeper配置

沒有使用Hbase自帶的zk , 本想用docker中的zk來的, 但是有點小問題(解決了會用的)

本次使用的是官網下載的 3.5.6版本 http://archive.apache.org/dist/zookeeper/

# The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/opt/module/zk/data # 自己創建的 dataLogDir=/opt/module/zk/dataLog # 自己創建的 # the port at which the clients will connect clientPort=2181

本次所有的開發環境運行的用戶為 codingce, 大家可自己創建類似賬戶并分配權限，此過程在我寫的 Hadoop集群搭建里面有如何創建用戶.

文章地址

切換完用戶后

啟動 ZooKeeper

進入到bin目錄下

$ ./zkServer.sh start

查看 zk 狀態

./zkServer.sh status

停止 zk

./zkServer.sh stop

重啟 zk

./zkServer.sh restart

測試連接 zk

./zkCli.sh -server 127.0.0.1:2181

期間遇到的問題

./zkCli.sh -server 127.0.0.1:2181

使用 ls / 來掃描zookeeper中的數據

使用 rmr /hbase 刪除zookeeper中的hbase數據

重新啟動hbase即可

Hbase配置

config 目錄下

hbase-env.sh

export JAVA_HOME=/usr/java/jdk1.8.0_221/ export HBASE_MANAGES_ZK=false

hbase-site.xml

<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration><property><name>hbase.rootdir</name><value>hdfs://hadoop102:8020/hbase</value></property><property><name>hbase.cluster.distributed</name><value>true</value></property><property><name>hbase.unsafe.stream.capability.enforce</name><value>false</value></property><property><name>hbase.master.port</name><value>16000</value></property><property><name>hbase.zookeeper.quorum</name><value>127.0.0.1:2181</value></property><property><name>hbase.zookeeper.property.dataDir</name><value>/opt/module/zk/data</value></property> </configuration>

軟連接 hadoop 配置文件到 hbase

[codingce@linuxmxz conf]$ ln -s /opt/module/hadoop-3.1.1/etc/hadoop/core-site.xml /opt/module/hbase-2.2.7/conf/core-site.xml [codingce@linuxmxz conf]$ ln -s /opt/module/hadoop-3.1.1/etc/hadoop/hdfs-site.xml /opt/module/hbase-2.2.7/conf/hdfs-site.xml

啟動
bin/start-hbase.sh

停止
bin/stop-hbase.sh

環境啟動無誤后先試下 Shell操作然后進行代碼操作

HBase Shell 基本操作

Java 操作

maven項目

pom

<dependencies>     <dependency><groupId>org.apache.hbase</groupId><artifactId>hbase-client</artifactId><version>1.4.2</version></dependency><dependency><groupId>org.apache.hbase</groupId><artifactId>hbase-server</artifactId><version>1.4.2</version></dependency>               <dependency><groupId>org.apache.hadoop</groupId><artifactId>hadoop-client</artifactId><version>3.1.1</version></dependency><dependency><groupId>junit</groupId><artifactId>junit</artifactId><version>4.13.1</version></dependency><dependency><groupId>org.slf4j</groupId><artifactId>slf4j-log4j12</artifactId><version>1.7.30</version></dependency></dependencies>

業務代碼

測試代碼

業務封裝

package cn.com.codingce.hbase.mydemo;import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.Cell; import org.apache.hadoop.hbase.CellUtil; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.HColumnDescriptor; import org.apache.hadoop.hbase.HTableDescriptor; import org.apache.hadoop.hbase.MasterNotRunningException; import org.apache.hadoop.hbase.NamespaceDescriptor; import org.apache.hadoop.hbase.NamespaceNotFoundException; import org.apache.hadoop.hbase.TableName; import org.apache.hadoop.hbase.ZooKeeperConnectionException; import org.apache.hadoop.hbase.client.Admin; import org.apache.hadoop.hbase.client.Connection; import org.apache.hadoop.hbase.client.ConnectionFactory; import org.apache.hadoop.hbase.client.Delete; import org.apache.hadoop.hbase.client.Get; import org.apache.hadoop.hbase.client.Put; import org.apache.hadoop.hbase.client.Result; import org.apache.hadoop.hbase.client.ResultScanner; import org.apache.hadoop.hbase.client.Scan; import org.apache.hadoop.hbase.client.Table; import org.apache.hadoop.hbase.util.Bytes;import java.io.IOException; import java.util.ArrayList; import java.util.List;/*** 1.4.2 版本寫法** @author williamma*/ public class TestApi_2 {public static void main(String[] args) throws IOException {// dropTable("codingce:student"); // createTable("user", "info");getAllRows("codingce:student");}//獲取 Configuration 對象public static Configuration conf;static {//使用 HBaseConfiguration 的單例方法實例化conf = HBaseConfiguration.create();conf.set("hbase.zookeeper.quorum", "139.9.34.48");conf.set("hbase.zookeeper.property.clientPort", "2181");}/*** 獲取連接** @return* @throws IOException*/public static Connection getConnection() throws IOException {Connection connection = ConnectionFactory.createConnection(conf);System.out.println("創建連接。。。" + connection);return connection;}/*** 判斷表是否存在** @param tableName* @return* @throws MasterNotRunningException* @throws ZooKeeperConnectionException* @throws IOException*/public static boolean isTableExist(String tableName) throws MasterNotRunningException,ZooKeeperConnectionException, IOException {//在 HBase 中管理、訪問表需要先創建 HBaseAdmin 對象//Connection connection = ConnectionFactory.createConnection(conf);// HBaseAdmin admin = (HBaseAdmin) connection.getAdmin();Connection connection = getConnection();Admin admin = connection.getAdmin();boolean result = admin.tableExists(TableName.valueOf(tableName));System.out.println("表是否存在：" + result);return result;}/*** 創建表** @param tableName* @param columnFamily* @throws MasterNotRunningException* @throws ZooKeeperConnectionException* @throws IOException*/public static void createTable(String tableName, String... columnFamily) throwsMasterNotRunningException, ZooKeeperConnectionException, IOException {Connection connection = getConnection();Admin admin = connection.getAdmin();//操作數據庫判斷數據庫中有沒有某張表//判斷命名空間try {admin.getNamespaceDescriptor("codingce");} catch (NamespaceNotFoundException e) {//創建命名空間NamespaceDescriptor codingce = NamespaceDescriptor.create("codingce").build();admin.createNamespace(codingce);}//判斷表是否存在if (isTableExist("codingce:" + tableName)) {System.out.println("表 " + "codingce:" + tableName + "已存在");//System.exit(0);} else {//創建表//創建表屬性對象,表名需要轉字節HTableDescriptor descriptor = new HTableDescriptor(TableName.valueOf("codingce:" + tableName));//創建多個列族for (String cf : columnFamily) {descriptor.addFamily(new HColumnDescriptor(cf));}//根據對表的配置，創建表admin.createTable(descriptor);System.out.println("表 " + "codingce:" + tableName + "創建成功!");}}/*** 刪除表* <p>* codingce:student* 命名空間 : 表名** @param tableName* @throws MasterNotRunningException* @throws ZooKeeperConnectionException* @throws IOException*/public static void dropTable(String tableName) throws MasterNotRunningException, ZooKeeperConnectionException, IOException {Connection connection = getConnection();Admin admin = connection.getAdmin();if (isTableExist(tableName)) {admin.disableTable(TableName.valueOf(tableName));admin.deleteTable(TableName.valueOf(tableName));System.out.println("表" + tableName + "刪除成功!");} else {System.out.println("表" + tableName + "不存在!");}}/*** 向表中插入數據** @param tableName* @param rowKey* @param columnFamily* @param column* @param value* @throws IOException*/public static void addRowData(String tableName,String rowKey,String columnFamily,String column,String value) throws IOException {Connection connection = getConnection();//查詢數據//獲取指定表對象Table table = connection.getTable(TableName.valueOf(tableName));//注意字符編碼問題//Get get = new Get(rowkey.getBytes());Get get = new Get(Bytes.toBytes(rowKey));//查詢結果Result result = table.get(get);boolean empty = result.isEmpty();System.out.println("是否存在：" + empty);if (empty) {//新增數據Put put = new Put(Bytes.toBytes(rowKey));put.addColumn(Bytes.toBytes(columnFamily), Bytes.toBytes(column), Bytes.toBytes(value));table.put(put);System.out.println("插入數據成功。。。");} else {System.out.println("已存在此數據。。。");}}/*** 刪除多行數據** @param tableName* @param rows* @throws IOException*/public static void deleteMultiRow(String tableName, String... rows) throws IOException {Connection connection = getConnection();Table table = connection.getTable(TableName.valueOf(tableName));List<Delete> deleteList = new ArrayList<Delete>();for (String row : rows) {Delete delete = new Delete(Bytes.toBytes(row));deleteList.add(delete);}table.delete(deleteList);table.close();}/*** 獲取所有數據** @param tableName* @throws IOException*/public static void getAllRows(String tableName) throws IOException {Connection connection = getConnection();Table table = connection.getTable(TableName.valueOf(tableName));//得到用于掃描 region 的對象Scan scan = new Scan();//使用 HTable 得到 resultcanner 實現類的對象ResultScanner resultScanner = table.getScanner(scan);for (Result result : resultScanner) {Cell[] cells = result.rawCells();for (Cell cell : cells) {//上下兩個循環待改System.out.println(" value " + Bytes.toString(CellUtil.cloneValue(cell)));System.out.println(" rowkey " + Bytes.toString(CellUtil.cloneRow(cell)));System.out.println(" family " + Bytes.toString(CellUtil.cloneFamily(cell)));System.out.println(" column " + Bytes.toString(CellUtil.cloneQualifier(cell)));}}}/*** 獲取某一行數據** @param tableName* @param rowKey* @throws IOException*/public static void getRow(String tableName, String rowKey) throws IOException {Connection connection = getConnection();Table table = connection.getTable(TableName.valueOf(tableName));Get get = new Get(Bytes.toBytes(rowKey));//get.setMaxVersions();顯示所有版本//get.setTimeStamp();顯示指定時間戳的版本Result result = table.get(get);for (Cell cell : result.rawCells()) {System.out.println(" value " + Bytes.toString(CellUtil.cloneValue(cell)));System.out.println(" rowkey " + Bytes.toString(CellUtil.cloneRow(cell)));System.out.println(" family " + Bytes.toString(CellUtil.cloneFamily(cell)));System.out.println(" column " + Bytes.toString(CellUtil.cloneQualifier(cell)));}} }

項目地址

目前還在github(太慢了最近在遷移項目到gitee)

Github

Gitee

今天剛踩的坑

如有問題歡迎討論，我在后端碼匠等你

總結

以上是生活随笔為你收集整理的HBase环境搭建与使用的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。

环境
HBase

上一篇： HBase Shell 基本操作
下一篇： Apache Software Foun