作者: 大圓那些事 | 文章可以轉(zhuǎn)載，請(qǐng)以超鏈接形式標(biāo)明文章原始出處和作者信息

網(wǎng)址: http://www.cnblogs.com/panfeng412/archive/2013/04/29/hive-view-usage-and-analysis.html

本文是對(duì)Hive中邏輯視圖的介紹，通過一個(gè)簡單的視圖例子，說明其使用方法及執(zhí)行過程。

Hive 0.6版本及以上支持視圖（View，詳見Hive的RELEASE_NOTES.txt），Hive View具有以下特點(diǎn)：

1）View是邏輯視圖，暫不支持物化視圖（后續(xù)將在1.0.3版本以后支持）；

2）View是只讀的，不支持LOAD/INSERT/ALTER。需要改變View定義，可以是用Alter View；

3）View內(nèi)可能包含ORDER BY/LIMIT語句，假如一個(gè)針對(duì)View的查詢也包含這些語句， 則View中的語句優(yōu)先級(jí)高；

4）支持迭代View。

CDH4中自帶的Hive版本為0.10.0，支持的View是邏輯視圖，因此本質(zhì)上來說View只是為了使用上的方便，從執(zhí)行效率上來說沒有區(qū)別，甚至可能因?yàn)橐嘁淮螌?duì)MetaStore元數(shù)據(jù)的操作效率略有下降（這里只是一種理論上的推測，實(shí)際可能看不出太大區(qū)別）。

下面是簡單的驗(yàn)證過程（感興趣的可以看下，以下過程如有問題，可以一起交流）：

1）創(chuàng)建一個(gè)測試表：

hive> create table test (id int, name string); OK Time taken: 0.19 seconds hive> desc test; OK id int name string Time taken: 0.16 seconds

2）創(chuàng)建一個(gè)View之前，使用explain命令查看創(chuàng)建View的命令是如何被Hive解釋執(zhí)行的：

hive> explain create view test_view (id, name_length) as select id, length(name) from test; OK ABSTRACT SYNTAX TREE:(TOK_CREATEVIEW (TOK_TABNAME test_view) (TOK_TABCOLNAME (TOK_TABCOL id TOK_NULL) (TOK_TABCOL name_length TOK_NULL)) (TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME test))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL id)) (TOK_SELEXPR (TOK_FUNCTION length (TOK_TABLE_OR_COL name))))))) STAGE DEPENDENCIES:Stage-0 is a root stage STAGE PLANS:Stage: Stage-0Create View Operator:Create Viewif not exists: falseor replace: falsecolumns: id int, name_length intexpanded text: SELECT `id` AS `id`, `_c1` AS `name_length` FROM (select `test`.`id`, length(`test`.`name`) from `default`.`test`) `test_view`name: test_vieworiginal text: select id, length(name) from test Time taken: 0.088 seconds

可見，創(chuàng)建View的過程解釋后并沒有實(shí)際執(zhí)行Map Reduce的Stage，只包含一個(gè)Create View Operator的Stage，這個(gè)階段只是對(duì)MySQL MetaStore進(jìn)行元數(shù)據(jù)操作，記錄View的相關(guān)元數(shù)據(jù)而已。

3）接下來，實(shí)際創(chuàng)建這個(gè)View：

hive> create view test_view (id, name_length) as select id, length(name) from test; OK Time taken: 0.1 seconds

4）執(zhí)行這個(gè)View之前，先explain查看實(shí)際被翻譯后的執(zhí)行過程：

hive> explain select name_length from test_view; OK ABSTRACT SYNTAX TREE:(TOK_QUERY (TOK_FROM (TOK_TABREF (TOK_TABNAME test_view))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL name_length))))) STAGE DEPENDENCIES:Stage-1 is a root stageStage-0 is a root stage STAGE PLANS:Stage: Stage-1Map ReduceAlias -> Map Operator Tree:test_view:test_view:testTableScanalias: testSelect Operatorexpressions:expr: length(name)type: intoutputColumnNames: _col1Select Operatorexpressions:expr: _col1type: intoutputColumnNames: _col1Select Operatorexpressions:expr: _col1type: intoutputColumnNames: _col0File Output Operatorcompressed: falseGlobalTableId: 0table:input format: org.apache.hadoop.mapred.TextInputFormatoutput format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormatStage: Stage-0Fetch Operatorlimit: -1 Time taken: 0.107 seconds

可以看出，對(duì)View進(jìn)行的查找過程，實(shí)際還是對(duì)原始test表進(jìn)行的查詢操作（分為Stage-0和Stage-1兩個(gè)階段）。

5）最后，實(shí)際對(duì)這個(gè)View執(zhí)行一次查詢，顯示Stage-1階段對(duì)原始表test進(jìn)行了MapReduce過程：

hive> select name_length from test_view; Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201303092253_0057, Tracking URL = http://jobtracker.host:50030/jobdetails.jsp?jobid=job_201303092253_0057 Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_201303092253_0057 Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0 2013-03-13 22:43:39,044 Stage-1 map = 0%, reduce = 0% 2013-03-13 22:43:42,074 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.73 sec 2013-03-13 22:43:43,086 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.73 sec 2013-03-13 22:43:44,098 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.73 sec 2013-03-13 22:43:45,113 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 0.73 sec MapReduce Total cumulative CPU time: 730 msec Ended Job = job_201303092253_0057 MapReduce Jobs Launched: Job 0: Map: 1 Cumulative CPU: 0.73 sec HDFS Read: 250 HDFS Write: 0 SUCCESS Total MapReduce CPU Time Spent: 730 msec OK Time taken: 15.793 seconds

?

轉(zhuǎn)載于:https://www.cnblogs.com/panfeng412/archive/2013/04/29/hive-view-usage-and-analysis.html

總結(jié)

以上是生活随笔為你收集整理的Hive中视图机制的初步使用及分析的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。