当前位置: 代码迷 >> Android >> android学习札记-32_文件断点上传器,解决多用户并发,以及自定义协议,注意协议中的漏洞
  详细解决方案

android学习札记-32_文件断点上传器,解决多用户并发,以及自定义协议,注意协议中的漏洞

热度:89   发布时间:2016-05-01 10:59:20.0
android学习笔记---32_文件断点上传器,解决多用户并发,以及自定义协议,注意协议中的漏洞
32_文件断点上传器
---------------------------
1.当文件很大的时候就无法通过http协议进行上传了,因为get,post的安全原因,很多服


务器会
  禁止这些协议,而且get也不支持大文件上传,那么这个时候就需要使用Tcp/IP


(Socket)协议实现断点上传,实现多用户并发访问
-------------------------------------------------
2.//
在Android中使用WebService之类的网络服务时出现:
request time failed: java.net.SocketException: Address family not supported 


by protocol
的错误提示。
有可能的原因一:电脑上同时开启了多个上网的网卡,导致模拟器访问网络时出现问题



解决办法:只留一个可上网的网卡,暂时禁用其他网卡。
有可能的原因二:模拟器出问题,例如存储容量过低。
解决办法:新建AVD或者修改原有AVD的SIZE之类的。
----------------------------------------


----
3.Socket客户端                                          socket服务端
-------------------------
用到了自定义的协议:
客户端连接到服务器端,客户端发信息给服务器端:
这段消息:content-length:文件长度;filename=文件名字;sourceid=在服务器端唯一
标识这个应用.
a.当第一次上传的时候sourceid为null,当服务器接收到协议内容的时候,那么服务器会
  判断sourceid是否为null,当sourceid为null,的时候那么服务器端会生成一个  


sourceid来标识这个应用,然后把协议返回给客户端,然后再传一个参数,postion=
  要求客户端从文件的什么地方开始上传数据,当客户端接收到服务器的信息后,
  就会用sourceid来关联上这个文件,然后从文件的postion位置开始上传数据给服务器
  端,
b.当用户在上传的时候应用如果突然停止,那么当再次上传的时候,客户端会首先上传
  协议信息给服务器端,这时候服务器端会接收到sourceid,然后根据sourceid找到上次
  客户端上传的文件,然后要求客户端从上次上传的地方,开始上传数据.这样就实现了
  断点上传.
c.会先读取协议的第一行信息,然后通过回车换行,'\n',判断协议的第一行
  然后再读取协议的第二行信息.
  假如,有用户知道这个问题的话,那么,可能会连续的进行发送协议信息,而没有进行
  回车换行,那么这个时候,程序会一直读取协议的第一行信息,放到内存中,如果这一行
  数据很多而且没有回车换行的话,那么内存就被填满了.就出现了问题
--------------------
d.解决这个漏洞的方法:可以在读取第一行信息的时候,进行判断,比如当判断
  协议第一行信息的大小大于1m,的时候,那么可以断定肯定不是自己要的协议信息了
  也就是说,肯定有人恶意攻击服务器,那么就立即终止对它的服务.
-------------------------------
e.导出可运行的jar包的方法:在项目上右击,然后选择export,然后选择java下的
  runable jar文件,如果以前运行过这个类的话,那么会在下拉框中显示这个类名
  选择该类名后,然后点击finsh.就可以导出了,然后复制到其他的盘里面,直接双击
  打开就可以了.
------------------
1.下面是断点上传的所有代码:
  a.新建java项目:
    socket
  b./socket/src/cn/itcast/net/client/SocketClient.java
   package com.credream.net.client;


import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PushbackInputStream;
import java.io.RandomAccessFile;
import java.net.Socket;


import com.credream.utils.StreamTool;


public class SocketClient {
/**
* @param args
*/
public static void main(String[] args) {
try {
Socket socket = new Socket("127.0.0.1", 7878);
            OutputStream outStream = socket.getOutputStream();            
            String filename = "QQWubiSetup.exe";
            File file = new File(filename);
            String head = "Content-Length="+ file.length() + ";filename="+ 


filename + ";sourceid=\r\n";
            outStream.write(head.getBytes());
            
            PushbackInputStream inStream = new PushbackInputStream


(socket.getInputStream());
String response = StreamTool.readLine(inStream);
            System.out.println(response);
            String[] items = response.split(";");
String position = items[1].substring(items


[1].indexOf("=")+1);

RandomAccessFile fileOutStream = new 


RandomAccessFile(file, "r");
fileOutStream.seek(Integer.valueOf(position));
byte[] buffer = new byte[1024];
int len = -1;
while( (len = fileOutStream.read(buffer)) != -1){
outStream.write(buffer, 0, len);
}
fileOutStream.close();
outStream.close();
            inStream.close();
            socket.close();
        } catch (Exception e) {                    
            e.printStackTrace();
        }


}
/**
* 读取流
* @param inStream
* @return 字节数组
* @throws Exception
*/
public static byte[] readStream(InputStream inStream) throws 


Exception{
ByteArrayOutputStream outSteam = new 


ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int len = -1;
while( (len=inStream.read(buffer)) != -1){
outSteam.write(buffer, 0, len);
}
outSteam.close();
inStream.close();
return outSteam.toByteArray();
}
}
------------------------------------------------------
c./socket/src/cn/itcast/net/server/FileServer.java
  package com.credream.net.server;


import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.io.PushbackInputStream;
import java.io.RandomAccessFile;
import java.net.ServerSocket;
import java.net.Socket;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;
import java.util.Properties;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;


import com.credream.utils.StreamTool;


public class FileServer {

private ExecutorService executorService;//线程池,实现网络多用户并发
private int port;//监听端口
private boolean quit = false;//退出
private ServerSocket server;
private Map<Long, FileLog> datas = new HashMap<Long, FileLog>();//


存放断点数据
 
public FileServer(int port){
this.port = port;
//创建线程池,池中具有(cpu个数*50)条线程
executorService = Executors.newFixedThreadPool


(Runtime.getRuntime().availableProcessors() * 50);
}
/**
 * 退出
 */
public void quit(){
this.quit = true;
try {
server.close();
} catch (IOException e) {
}
}
/**
 * 启动服务
 * @throws Exception
 */
public void start() throws Exception{
server = new ServerSocket(port);
while(!quit){
        try {
          Socket socket = server.accept();
          //为支持多用户并发访问,采用线程池管理每一个用户的连接请



          executorService.execute(new SocketTask(socket));
        } catch (Exception e) {
          //  e.printStackTrace();
        }
    }
}
 
private final class SocketTask implements Runnable{
private Socket socket = null;
public SocketTask(Socket socket) {
this.socket = socket;
}

public void run() {
try {
System.out.println("accepted connection "+ 


socket.getInetAddress()+ ":"+ socket.getPort());
PushbackInputStream inStream = new 


PushbackInputStream(socket.getInputStream());
//得到客户端发来的第一行协议数据:Content-


Length=143253434;filename=xxx.3gp;sourceid=
//如果用户初次上传文件,sourceid的值为空。
String head = StreamTool.readLine(inStream);
System.out.println(head);
if(head!=null){
//下面从协议数据中提取各项参数值
String[] items = head.split(";");
String filelength = items


[0].substring(items[0].indexOf("=")+1);
String filename = items


[1].substring(items[1].indexOf("=")+1);
String sourceid = items


[2].substring(items[2].indexOf("=")+1);
long id = System.currentTimeMillis


();//生产资源id,如果需要唯一性,可以采用UUID
FileLog log = null;
if(sourceid!=null && !"".equals


(sourceid)){
id = Long.valueOf(sourceid);
log = find(id);//查找上传的


文件是否存在上传记录
}
File file = null;
int position = 0;
if(log==null){//如果不存在上传记录,


为文件添加跟踪记录
String path = new 


SimpleDateFormat("yyyy/MM/dd/HH/mm").format(new Date());
File dir = new File("file/"+ 


path);
if(!dir.exists()) 


dir.mkdirs();
file = new File(dir, 


filename);
if(file.exists()){//如果上传


的文件发生重名,然后进行改名
filename = 


filename.substring(0, filename.indexOf(".")-1)+ dir.listFiles().length+ 


filename.substring(filename.indexOf("."));
file = new File(dir, 


filename);
}
save(id, file);
}else{// 如果存在上传记录,读取已经上


传的数据长度
file = new File(log.getPath


());//从上传记录中得到文件的路径
if(file.exists()){
File logFile = new 


File(file.getParentFile(), file.getName()+".log");
if(logFile.exists


()){
Properties 


properties = new Properties();



properties.load(new FileInputStream(logFile));
position = 


Integer.valueOf(properties.getProperty("length"));//读取已经上传的数据长度
}
}
}

OutputStream outStream = 


socket.getOutputStream();
String response = "sourceid="+ id+ 


";position="+ position+ "\r\n";
//服务器收到客户端的请求信息后,给客


户端返回响应信息:sourceid=1274773833264;position=0
//sourceid由服务器端生成,唯一标识上


传的文件,position指示客户端从文件的什么位置开始上传
outStream.write(response.getBytes


());

RandomAccessFile fileOutStream = new 


RandomAccessFile(file, "rwd");
if(position==0) 


fileOutStream.setLength(Integer.valueOf(filelength));//设置文件长度
fileOutStream.seek(position);//指定


从文件的特定位置开始写入数据
byte[] buffer = new byte[1024];
int len = -1;
int length = position;
while( (len=inStream.read(buffer)) 


!= -1){//从输入流中读取数据写入到文件中
fileOutStream.write(buffer, 


0, len);
length += len;
Properties properties = new 


Properties();
properties.put("length", 


String.valueOf(length));
FileOutputStream logFile = 


new FileOutputStream(new File(file.getParentFile(), file.getName()+".log"));
properties.store(logFile, 


null);//实时记录已经接收的文件长度
logFile.close();
}
if(length==fileOutStream.length()) 


delete(id);
fileOutStream.close();



inStream.close();
outStream.close();
file = null;

}
} catch (Exception e) {
e.printStackTrace();
}finally{
           try {
               if(socket!=null && !socket.isClosed()) socket.close


();
           } catch (IOException e) {}
       }
}
}
 
public FileLog find(Long sourceid){
return datas.get(sourceid);
}
//保存上传记录
public void save(Long id, File saveFile){
//日后可以改成通过数据库存放
datas.put(id, new FileLog(id, saveFile.getAbsolutePath()));
}
//当文件上传完毕,删除记录
public void delete(long sourceid){
if(datas.containsKey(sourceid)) datas.remove(sourceid);
}
 
private class FileLog{
private Long id;
private String path;
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getPath() {
return path;
}
public void setPath(String path) {
this.path = path;
}
public FileLog(Long id, String path) {
this.id = id;
this.path = path;
}
}


}
---------------------
d./socket/src/cn/itcast/net/server/ServerWindow.java
  package com.credream.net.server;


import java.awt.BorderLayout;
import java.awt.Frame;
import java.awt.Label;
import java.awt.event.WindowEvent;
import java.awt.event.WindowListener;


public class ServerWindow extends Frame{
private FileServer s = new FileServer(7878);
private Label label;

public ServerWindow(String title){
super(title);
label = new Label();
add(label, BorderLayout.PAGE_START);
label.setText("服务器已经启动");
this.addWindowListener(new WindowListener() {
public void windowOpened(WindowEvent e) {
new Thread(new Runnable() {
public void run() {
try {
s.start();
} catch (Exception e) {
//e.printStackTrace


();
}
}
}).start();
}

public void windowIconified(WindowEvent e) {
}

public void windowDeiconified(WindowEvent e) {
}

public void windowDeactivated(WindowEvent e) {
}

public void windowClosing(WindowEvent e) {
s.quit();
System.exit(0);
}

public void windowClosed(WindowEvent e) {
}

public void windowActivated(WindowEvent e) {
}
});
}
/**
* @param args
*/
public static void main(String[] args) {
ServerWindow window = new ServerWindow("文件上传服务端"); 
window.setSize(300, 300); 
window.setVisible(true);

}


}
-------------------------------------------
e./socket/src/cn/itcast/utils/StreamTool.java
  package com.credream.utils;


import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.PushbackInputStream;


public class StreamTool {


public static void save(File file, byte[] data) throws Exception {
FileOutputStream outStream = new FileOutputStream(file);
outStream.write(data);
outStream.close();
}
 
public static String readLine(PushbackInputStream in) throws 


IOException {
char buf[] = new char[128];
int room = buf.length;
int offset = 0;
int c;
loop: while (true) {
switch (c = in.read()) {
case -1:
case '\n':
break loop;
case '\r':
int c2 = in.read();
if ((c2 != '\n') && (c2 != 


-1)) in.unread(c2);
break loop;
default:
if (--room < 0) {
char[] lineBuffer = 


buf;
buf = new char


[offset + 128];
   room = buf.length - 


offset - 1;
   System.arraycopy


(lineBuffer, 0, buf, 0, offset);
  
}
buf[offset++] = (char) c;
break;
}
}
if ((c == -1) && (offset == 0)) return null;
return String.copyValueOf(buf, 0, offset);
}
 
/**
* 读取流
* @param inStream
* @return 字节数组
* @throws Exception
*/
public static byte[] readStream(InputStream inStream) throws 


Exception{
ByteArrayOutputStream outSteam = new 


ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int len = -1;
while( (len=inStream.read(buffer)) != -1){
outSteam.write(buffer, 0, len);
}
outSteam.close();
inStream.close();
return outSteam.toByteArray();
}
}
---------------------------------------------------
2.新建上传的android客户端:
  项目:videoUpload
  /videoUpload/src/cn/itcast/service/DBOpenHelper.java
  package com.credream.service;


import android.content.Context;
import android.database.sqlite.SQLiteDatabase;
import android.database.sqlite.SQLiteOpenHelper;


public class DBOpenHelper extends SQLiteOpenHelper {


public DBOpenHelper(Context context) {
super(context, "itcast.db", null, 1);
}


@Override
public void onCreate(SQLiteDatabase db) {
db.execSQL("CREATE TABLE IF NOT EXISTS uploadlog (_id 


integer primary key autoincrement, path varchar(20), sourceid varchar


(20))");
}


@Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int 


newVersion) {
// TODO Auto-generated method stub


}


}
-----------------------------
/videoUpload/src/cn/itcast/service/UploadLogService.java
package com.credream.service;


import java.io.File;


import android.content.Context;
import android.database.Cursor;
import android.database.sqlite.SQLiteDatabase;


public class UploadLogService {
private DBOpenHelper dbOpenHelper;

public UploadLogService(Context context){
dbOpenHelper = new DBOpenHelper(context);
}

public String getBindId(File file){
SQLiteDatabase db = dbOpenHelper.getReadableDatabase();
Cursor cursor = db.rawQuery("select sourceid from uploadlog 


where path=?", new String[]{file.getAbsolutePath()});
if(cursor.moveToFirst()){
return cursor.getString(0);
}
return null;
}

public void save(String sourceid, File file){
SQLiteDatabase db = dbOpenHelper.getWritableDatabase();
db.execSQL("insert into uploadlog(path,sourceid) values


(?,?)", 
new Object[]{file.getAbsolutePath(), 


sourceid});
}

public void delete(File file){
SQLiteDatabase db = dbOpenHelper.getWritableDatabase();
db.execSQL("delete from uploadlog where path=?", new Object


[]{file.getAbsolutePath()});
}
 }
---------------------------
/videoUpload/src/cn/itcast/upload/MainActivity.java
package com.credream.upload;




import java.io.File;


import java.io.OutputStream;


import java.io.PushbackInputStream;


import java.io.RandomAccessFile;


import java.net.Socket;




import com.credream.service.UploadLogService;


import com.credream.utils.StreamTool;




import android.app.Activity;


import android.os.Bundle;


import android.os.Environment;


import android.os.Handler;


import android.os.Message;


import android.view.View;


import android.widget.Button;


import android.widget.EditText;


import android.widget.ProgressBar;


import android.widget.TextView;


import android.widget.Toast;




public class MainActivity extends Activity {
  
  private EditText filenameText;
  
  private TextView resultView;
   
 private ProgressBar uploadbar;
  
  private UploadLogService service;


    private Handler handler = new Handler(){


@Override

public void handleMessage(Message msg) {



uploadbar.setProgress(msg.getData().getInt("length"));



float num = (float)uploadbar.getProgress() / (float)uploadbar.getMax();



int result = (int)(num * 100);

resultView.setText(result + "%");


if(uploadbar.getProgress() == uploadbar.getMax()){



Toast.makeText(MainActivity.this, R.string.success, 1).show();



}
}
    };
    
  
  @Override
    
public void onCreate(Bundle savedInstanceState) {
  
      super.onCreate(savedInstanceState);
  
      setContentView(R.layout.main);
     
   
        service =  new UploadLogService(this);
 
       filenameText = (EditText)findViewById(R.id.filename);
        


resultView = (TextView)findViewById(R.id.result);
  
      uploadbar = (ProgressBar)findViewById(R.id.uploadbar);
   
     Button button = (Button)findViewById(R.id.button);
        


button.setOnClickListener(new View.OnClickListener() {



public void onClick(View v) {

String filename = filenameText.getText().toString();



if(Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED)){



File file = new File(Environment.getExternalStorageDirectory(), filename);



if(file.exists()){



uploadbar.setMax((int)file.length());



uploadFile(file);

}else{



Toast.makeText(MainActivity.this, R.string.notexsit, 1).show();




}
}else{

Toast.makeText(MainActivity.this, R.string.sdcarderror, 1).show();



}
}

});
    }




private void uploadFile(final File file) {


new Thread(new Runnable() {

public void run() {

try {

String sourceid = service.getBindId(file);



Socket socket = new Socket("192.168.0.110", 6118);
           


OutputStream outStream = socket.getOutputStream(); 
           


String head = "Content-Length="+ file.length() + ";
filename="+ file.getName() + ";
sourceid="+(sourceid!=null ? sourceid : "")+"\r\n";
           


outStream.write(head.getBytes());
           
    


PushbackInputStream inStream = new PushbackInputStrea(socket.getInputStream


());

String response = StreamTool.readLine(inStream);
           


String[] items = response.split(";");



String responseSourceid = items[0].substring(items[0].indexOf("=")+1);



String position = items[1].substring(items[1].indexOf("=")+1);



if(sourceid==null){
//如果是第一次上传文件,在数据库中不存在该文件所绑定的资源id



service.save(responseSourceid, file);



}

RandomAccessFile fileOutStream = new RandomAccessFile(file, "r");



fileOutStream.seek(Integer.valueOf(position));



byte[] buffer = new byte[1024];



int len = -1;

int length = Integer.valueOf(position);



while( (len = fileOutStream.read(buffer)) != -1){



outStream.write(buffer, 0, len);



length += len;//累加已经上传的数据长度



Message msg = new Message();



msg.getData().putInt("length", length);



handler.sendMessage(msg);
}



if(length == file.length()) service.delete(file);



fileOutStream.close();



outStream.close();
          
 inStream.close();
        
   socket.close();
    
   } catch (Exception e) {                    
       


Toast.makeText(MainActivity.this, R.string.error, 1).show();
    


   }
}
}).start();


}
}
-------------------------------------------------
/videoUpload/src/cn/itcast/utils/StreamTool.java
package com.credream.utils;


import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.PushbackInputStream;


public class StreamTool {
 
public static void save(File file, byte[] data) throws Exception {
FileOutputStream outStream = new FileOutputStream(file);
outStream.write(data);
outStream.close();
}
 
public static String readLine(PushbackInputStream in) throws 


IOException {
char buf[] = new char[128];
int room = buf.length;
int offset = 0;
int c;
loop: while (true) {
switch (c = in.read()) {
case -1:
case '\n':
break loop;
case '\r':
int c2 = in.read();
if ((c2 != '\n') && (c2 != 


-1)) in.unread(c2);
break loop;
default:
if (--room < 0) {
char[] lineBuffer = 


buf;
buf = new char


[offset + 128];
   room = buf.length - 


offset - 1;
   System.arraycopy


(lineBuffer, 0, buf, 0, offset);
  
}
buf[offset++] = (char) c;
break;
}
}
if ((c == -1) && (offset == 0)) return null;
return String.copyValueOf(buf, 0, offset);
}
 
/**
* 读取流
* @param inStream
* @return 字节数组
* @throws Exception
*/
public static byte[] readStream(InputStream inStream) throws 


Exception{
ByteArrayOutputStream outSteam = new 


ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int len = -1;
while( (len=inStream.read(buffer)) != -1){
outSteam.write(buffer, 0, len);
}
outSteam.close();
inStream.close();
return outSteam.toByteArray();
}
}
--------------------
/videoUpload/res/layout/main.xml
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    >
<TextView  
    android:layout_width="fill_parent" 
    android:layout_height="wrap_content" 
    android:text="@string/filename"
    />
    
    <EditText  
    android:layout_width="fill_parent" 
    android:layout_height="wrap_content" 
    android:text="shopping.rar"
    android:id="@+id/filename"
    />
    
   <Button  
   android:layout_width="wrap_content" 
   android:layout_height="wrap_content" 
   android:text="@string/button"
   android:id="@+id/button"
   />
   
<ProgressBar 
   android:layout_width="fill_parent" 
   android:layout_height="20px"
   style="?android:attr/progressBarStyleHorizontal"
   android:id="@+id/uploadbar"
   /> 
<TextView  
   android:layout_width="fill_parent" 
   android:layout_height="wrap_content" 
   android:gravity="center"
   android:id="@+id/result"
   />    
</LinearLayout>
-----------------------------------
/videoUpload/res/values/strings.xml
<?xml version="1.0" encoding="utf-8"?>
<resources>
    <string name="hello">Hello World, MainActivity!</string>
    <string name="app_name">视频上传</string>
    <string name="filename">视频文件</string>
    <string name="button">上传</string>
    <string name="success">上传成功</string>
    <string name="notexsit">文件不存在</string>
    <string name="error">上传失败</string>
    <string name="sdcarderror">SDCard不存在或者写保护</string>
</resources>
--------------------------------
/videoUpload/AndroidManifest.xml
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
      package="com.credream.upload"
      android:versionCode="1"
      android:versionName="1.0">
    <application android:icon="@drawable/icon" 


android:label="@string/app_name">
        <activity android:name=".MainActivity"
                  android:label="@string/app_name">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>


    </application>
    <uses-sdk android:minSdkVersion="8" />
    <!-- 在SDCard中创建与删除文件权限 -->
<uses-permission 


android:name="android.permission.MOUNT_UNMOUNT_FILESYSTEMS"/>
<!-- 往SDCard写入数据权限 -->
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
 <!-- 访问internet权限 -->
<uses-permission android:name="android.permission.INTERNET"/>
</manifest> 
-----------------------------
1.测试的时候,先运行导出的jar文件,然后在运行android项目进行上传测试
2.等上传一半的时候,强行关闭,然后重新开启应用,继续上传,可以看到
  这时候已经实现断点保存上传了
3.多个用户同时使用的时候,还可以实现多用户并发.
-------------------------


  









  相关解决方案