?那最简单的IO说吧 在本地文件系统和HDFS文件系统中的IO操作
?下面这个例子是合并本地文件系统的目录下的文件 到HDFS文件系统
?public static void main(String[] args) throws IOException, URISyntaxException {
??? ??? ?//Hadoop resource configuration class
??? ??? ?Configuration conf=new Configuration();
??? ???? //get FileSystem's instance
??? ??? ?//call factory method FileSystem.get(Configuration conf)
??? ??? ?FileSystem hdfs=FileSystem.get(new URI("hdfs://127.0.0.1:9000"),conf);
??? ??? ?//local FileSystem's instance
??? ??? ?FileSystem local =FileSystem.getLocal(conf);
??? ??? ?//Path Object encode file and direcotory
??? ??? ?Path? inputDir=new Path("/home/hadoop/Documents"); //(args[0]);
??? ??? ?Path? hdfsFile=new Path("/mytest_example2.txt");
??? ??? ?//use the FileSystem's listStatus()method() to get? a list of files in a directory
??? ??? ?FileStatus[]? inputFiles=local.listStatus(inputDir);
??? ??? ?
??? ??? ?//create FSDataOutputStream
??? ??? ?FSDataOutputStream out=hdfs.create(hdfsFile);
??? ??? ?//loop local files and merge a new file?
??? ??? ?for(int i=0;i<inputFiles.length;i++)
??? ??? ?{
??? ??? ??? ?//user the Path to request an FSDataInputStream object for reading in the file
??? ??? ??? ?System.err.println(inputFiles[i].getPath().getName()+"-----------------");
??? ??? ??? ?FSDataInputStream? in =local.open(inputFiles[i].getPath());
??? ??? ??? ?byte [] buffer=new byte[256];
??? ??? ??? ?int bytesRead=0;
??? ??? ??? ?while ((bytesRead=in.read(buffer))>0)
??? ??? ??? ?{
????????????????? out.write(buffer, 0, bytesRead);?????? ??? ??? ??? ???
??? ??? ??? ?}
??? ??? ??? ?in.close();
??? ??? ?}
??? ??? ?out.close();????
??? }
以上的代码是可行的没有什么错误。但是如果是一下这样，就会报错前提呢在你的项目的classpath下也就是bin目录下没有hadoop的HDFS配置文件（core-site.xml和hdfs-site.xml）
FileSystem hdfs=FileSystem.get(/*new URI("hdfs://127.0.0.1:9000"),*/conf);
Path? hdfsFile=new Path("hdfs://127.0.0.1:9000/mytest_example2.txt");
?
?
错误提示：
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: hdfs://127.0.0.1:9000/, expected: file:///
??? at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:646)
??? at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:82)
??? at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:513)
??? at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:499)
??? at org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:594)
??? at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:448)
??? at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:435)
??? at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:909)
??? at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:890)
??? at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:787)
??? at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:776)
??? at com.liweichao.PutMerge.main(PutMerge.java:40)
??
?
?? 解决途径一个是在初始化Filesystem的时候
?1 ： Configuration conf=new Configuration();
??? ???? //get FileSystem's instance
??? ??? ?//call factory method FileSystem.get(Configuration conf)
???????? //就会自动加载到hadoop的资源配置文件
??? ??? ?FileSystem hdfs=FileSystem.get(new URI("hdfs://127.0.0.1:9000"),conf);
???????? 在有Path对象定位的具体目录
???????? Path? hdfsFile=new Path("/mytest_example2.txt");
2：在你的项目的classpath下也就是bin目录下没有hadoop的HDFS配置文件（core-site.xml和hdfs-site.xml）