日韩性视频-久久久蜜桃-www中文字幕-在线中文字幕av-亚洲欧美一区二区三区四区-撸久久-香蕉视频一区-久久无码精品丰满人妻-国产高潮av-激情福利社-日韩av网址大全-国产精品久久999-日本五十路在线-性欧美在线-久久99精品波多结衣一区-男女午夜免费视频-黑人极品ⅴideos精品欧美棵-人人妻人人澡人人爽精品欧美一区-日韩一区在线看-欧美a级在线免费观看

        歡迎訪問 生活随笔!

        生活随笔

        當(dāng)前位置: 首頁 > 编程资源 > 编程问答 >内容正文

        编程问答

        java客户端运行hadoop_JAVA客户端连接部署在docker上的hdfs

        發(fā)布時間:2024/4/11 编程问答 26 豆豆
        生活随笔 收集整理的這篇文章主要介紹了 java客户端运行hadoop_JAVA客户端连接部署在docker上的hdfs 小編覺得挺不錯的,現(xiàn)在分享給大家,幫大家做個參考.

        docker上已經(jīng)成功部署了hdfs應(yīng)用,jps命令也顯示了namenode和datanode啟動正常。瀏覽器顯示 宿主機ip:50070 頁面

        我在eclipse使用java連接hdfs,代碼如下:

        FileSystem hdfs;

        Configuration conf = new Configuration();

        try {

        hdfs = FileSystem.get(new URI("hdfs://10.8.2.11:9999"),conf);

        Path des =new Path("E:/AB4/hdfs/worksapce");

        String srcPath = "/abcloud";

        Path src = new Path(srcPath);

        hdfs.copyFromLocalFile(false,true,des, src);

        運行結(jié)果:

        org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /abcloud/test.txt could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.

        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1559)

        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3245)

        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:663)

        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:482)

        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)

        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:975)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)

        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2036)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)

        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2034)

        at org.apache.hadoop.ipc.Client.call(Client.java:1411)

        at org.apache.hadoop.ipc.Client.call(Client.java:1364)

        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

        at java.lang.reflect.Method.invoke(Unknown Source)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)

        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)

        at com.sun.proxy.$Proxy7.addBlock(Unknown Source)

        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:368)

        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1449)

        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1270)

        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:526)

        有沒有大神曾在docker上成功部署hdfs,并在可以使用java api操作的,分享下經(jīng)驗

        補充:我使用eclipse-hadoop插件連接后,文件系統(tǒng)里有那個文件,但是沒有內(nèi)容

        總結(jié)

        以上是生活随笔為你收集整理的java客户端运行hadoop_JAVA客户端连接部署在docker上的hdfs的全部內(nèi)容,希望文章能夠幫你解決所遇到的問題。

        如果覺得生活随笔網(wǎng)站內(nèi)容還不錯,歡迎將生活随笔推薦給好友。