相关问题以及参考文章:
- 1. 为什么说Volley适合数据量小，通信频繁的网络操作
- 2. 为什么volley不适合post大量数据，以及为什么不适合上传下载大量文件？
- 3. Android Http缓存数据处理
- 4. Volley 源码解析
尝试从下面几个方面进行分析:
1. 关于线程管理;
2. 关于网络请求;
3. 关于数据缓存;
一、关于线程的管理:
1.1 Volley.newRequestQueue;
public class Volley {
public static RequestQueue newRequestQueue(Context context) {
return newRequestQueue(context, (BaseHttpStack) null);
}
public static RequestQueue newRequestQueue(Context context, BaseHttpStack stack) {
BasicNetwork network;
if (stack == null) {
/**
* 此时只考虑了sdk > 14的情况
*/
network = new BasicNetwork(new HurlStack());
} else {
network = new BasicNetwork(stack);
}
return newRequestQueue(context, network);
}
}
1.2 Volley.newRequestQueue:
public class Volley {
private static RequestQueue newRequestQueue(Context context, Network network) {
String path = "" + Environment.getExternalStorageDirectory();
File cacheDir = new File(path, DEFAULT_CACHE_DIR);
/**
* 然后开始创建线程;
*/
RequestQueue queue = new RequestQueue(new DiskBasedCache(cacheDir), network);
queue.start();
return queue;
}
}
1.3 RequestQueue:
public class RequestQueue {
private static final int DEFAULT_NETWORK_THREAD_POOL_SIZE = 4;
public RequestQueue(Cache cache, Network network) {
this(cache, network, DEFAULT_NETWORK_THREAD_POOL_SIZE);
}
public RequestQueue(Cache cache, Network network, int threadPoolSize) {
this(cache, network, threadPoolSize, new ExecutorDelivery(new Handler(Looper.getMainLooper())));
}
public RequestQueue(Cache cache, Network network, int threadPoolSize, ResponseDelivery delivery) {
mCache = cache;
mNetwork = network;
/**
* 每次调用Volley.newRequestQueue时都会创建四个线程;
*/
mDispatchers = new NetworkDispatcher[threadPoolSize];
mDelivery = delivery;
}
}
public class NetworkDispatcher extends Thread;
1.4 RequestQueue.start:
public class CacheDispatcher extends Thread;
public class RequestQueue {
public void start() {
/**
* 停止当前正在运行的线程;
*/
stop();
/**
* 再次创建一个CacheDispatcher线程, 到目前为止已经创建了5个线程, 也就是说在
* 初始化RequestQueue时, 会创建5个线程, 然后通过start让其一直处于运行状态;
*/
mCacheDispatcher = new CacheDispatcher(mCacheQueue, mNetworkQueue, mCache, mDelivery);
mCacheDispatcher.start();
for (int i = 0; i < mDispatchers.length; i++) {
NetworkDispatcher networkDispatcher = new NetworkDispatcher(mNetworkQueue, mNetwork, mCache, mDelivery);
mDispatchers[i] = networkDispatcher;
networkDispatcher.start();
}
}
public void stop() {
if (mCacheDispatcher != null) {
mCacheDispatcher.quit();
}
for (final NetworkDispatcher mDispatcher : mDispatchers) {
if (mDispatcher != null) {
mDispatcher.quit();
}
}
}
}
1.5 CacheDispatcher.run:
public class CacheDispatcher extends Thread {
private final BlockingQueue<Request<?>> mCacheQueue;
@Override
public void run() {
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
/**
* 在运行缓存线程之前, 先进行磁盘缓存数据读取到内存中;模块<3.4>
*/
mCache.initialize();
while (true) {
/**
* 其实到这里可以推翻模块<1.4>里面的部分结论, Volley初始化时, 确实会创建5个线程,
* 但是这5个线程并不会一直处于运行状态, 当我们没有显示调用RequesQueue时, 这五个
* 线程经过mCacheQueue.take()这个经典的生产者-消费者模式会被挂起, 直到有Request
* 被添加至RequestQueue中, 所以如果处于空闲状态, 这5个线程并不会占用资源;
*/
final Request<?> request = mCacheQueue.take();
request.addMarker("cache-queue-take");
if (request.isCanceled()) {
request.finish("cache-discard-canceled");
continue;
}
// Attempt to retrieve this item from cache.
Cache.Entry entry = mCache.get(request.getCacheKey());
/**
* 1. entry == null, 说明针对当前Request, 不存在缓存, 然后进入if内部;
*/
if (entry == null) {
request.addMarker("cache-miss");
if (!mWaitingRequestManager.maybeAddToWaitingRequests(request)) {
/**
* 1. 结合模块<1.4>可知, CacheDispatcher和NetworkDispatcher持有同一个
* mNetworkQueue;
* 2. 然后执行continue再次重复这次操作;
* 3. 由于CacheDispatcher与NetworkDispatcher持有同一个mNetworkQueue, 这里肯定是通过
* mNetworkQueue去控制NetworkDispatcher里面的相关逻辑
*/
mNetworkQueue.put(request);
}
continue;
}
}
}
}
1.6 NetworkDispatcher.run:
public class NetworkDispatcher extends Thread {
@Override
public void run() {
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
while (true) {
long startTimeMs = SystemClock.elapsedRealtime();
Request<?> request;
/**
* 这里的mQueue就是模块<1.5>提及的mNetworkQueue, NetworkDispatcher线程被挂起/激活;
*/
request = mQueue.take();
...
}
}
}
- 1、模块一可知, 初始化Volley时, 会创建5个线程, 然后分别执行其run方法, 但是通过BlockingQueue的生产者-消费者模型, 控制线程的挂起与激活, 如果没有Request时, 5个线程会处于挂起状态, 直到有Request时, 才会被激活;
- 2、CacheDispatcher与NetworkDispatcher共用同一个NetworkQueue, NetworkDispatcher的激活需要CacheDispatcher通过调用NetworkQueue.put去触发;
- 3、CacheDispatcher的激活谁去触发呢?模块二关于请求会涉及到CacheDispatcher的激活条件;
二、关于网络请求:
2.1 RequestQueue.add:
public class RequestQueue {
public <T> Request<T> add(Request<T> request) {
request.setRequestQueue(this);
synchronized (mCurrentRequests) {
mCurrentRequests.add(request);
}
request.setSequence(getSequenceNumber());
request.addMarker("add-to-queue");
/**
* 网络请求响应结果默认是否应当被缓存起来, 默认true, 表示应当被缓存, 既然response会被缓存,
* 那么当请求到来时, 肯定优先将Request交给CacheDispatcher去处理;
*/
if (!request.shouldCache()) {
mNetworkQueue.add(request);
return request;
}
/**
* 1. 结合模块<1.4> ~ <1.5>可知CacheDispatcher内部的mCacheQueue指向此处的mCacheQueue;
* 2. 在模块<1.5>中, 如果mCacheQueue未空, CacheDispatcher会被挂起, 直到外界显示唤醒, 这里通过
* mCacheQueue.add的方式唤醒CacheDispatcher线程;
*/
mCacheQueue.add(request);
return request;
}
}
public abstract class Request<T> implements Comparable<Request<T>> {
/**
* Whether or not responses to this request should be cached.
*/
private boolean mShouldCache = true;
/**
* Returns true if responses to this request should be cached.
*/
public final boolean shouldCache() {
return mShouldCache;
}
}
2.2 CacheDispatcher.run:
public class CacheDispatcher extends Thread {
@Override
public void run() {
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
mCache.initialize();
while (true) {
final Request<?> request = mCacheQueue.take();
request.addMarker("cache-queue-take");
if (request.isCanceled()) {
request.finish("cache-discard-canceled");
continue;
}
//1--->
/**
* 1. 如果没有缓存, 即entry = null, 则进入if内部通过mNetworkQueue.put
* 方式唤醒NetworkDispatcher, 模块<2.3>;
* 2. 如果存在缓存, 进入模块<//2--->>;
*/
Cache.Entry entry = mCache.get(request.getCacheKey());
if (entry == null) {
request.addMarker("cache-miss");
if (!mWaitingRequestManager.maybeAddToWaitingRequests(request)) {
mNetworkQueue.put(request);
}
continue;
}
//2--->
/**
* 1. 如果缓存不为空, 在这里判断缓存是否过期, 如果缓存过期, 则将Request交给NetworkDispatcher进行处理;
* 2. 缓存是否过期的依据?<TODO>
*/
if (entry.isExpired()) {
request.addMarker("cache-hit-expired");
request.setCacheEntry(entry);
if (!mWaitingRequestManager.maybeAddToWaitingRequests(request)) {
mNetworkQueue.put(request);
}
continue;
}
/**
* 1. 如果缓存存在, 且服务器端缓存数据没有过期, 则从内存缓存中读取数据到Response中;
* 2. 然后再次判断缓存数据是否需要进行刷新, 如果不需要刷新, 则直接在<//3--->>内部
* 进行数据分发, 反之进入<//4--->>将Request交给NetworkDispatcher;
*/
request.addMarker("cache-hit");
Response<?> response = request.parseNetworkResponse(
new NetworkResponse(entry.data, entry.responseHeaders));
request.addMarker("cache-hit-parsed");
//3--->
if (!entry.refreshNeeded()) {
mDelivery.postResponse(request, response);
} else {
//4--->
request.addMarker("cache-hit-refresh-needed");
request.setCacheEntry(entry);
response.intermediate = true;
if (!mWaitingRequestManager.maybeAddToWaitingRequests(request)) {
mDelivery.postResponse(request, response, new Runnable() {
@Override
public void run() {
mNetworkQueue.put(request);
}
});
} else {
mDelivery.postResponse(request, response);
}
}
}
}
}
2.3 NetworkDispatcher.run:
public class NetworkDispatcher extends Thread {
@Override
public void run() {
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
while (true) {
long startTimeMs = SystemClock.elapsedRealtime();
Request<?> request;
//1--->
/**
* 线程在模块<2.2>中被唤醒, 执行模块<//2--->>
*/
request = mQueue.take();
request.addMarker("network-queue-take");
if (request.isCanceled()) {
request.finish("network-discard-cancelled");
request.notifyListenerResponseNotUsable();
continue;
}
//2--->
/**
* 进入到模块<2.4>中执行网络请求, 在进行Volley初始化时, 只考虑SDK>14的情况,
* 此处mNetwork实际指向BasicNetwork, 其内部BaseHttpStack实际指向HurlStack;
*/
NetworkResponse networkResponse = mNetwork.performRequest(request);
request.addMarker("network-http-complete");
if (networkResponse.notModified && request.hasHadResponseDelivered()) {
request.finish("not-modified");
request.notifyListenerResponseNotUsable();
continue;
}
/**
* 1. 将网络数据进行解析并封装进Response中;模块<3.5>, 然后通过<//3--->>完成数据内存, 磁盘缓存;
* 2. parseNetworkResponse内部主要完成对响应头的解析与缓存, 然后等待下一次请求相同
* Request时, 可以通过响应头判断是否需要进行网络请求;
*/
Response<?> response = request.parseNetworkResponse(networkResponse);
request.addMarker("network-parse-complete");
/**
* 执行到这里判断request获取的结果是否需要进行缓存, shouldCache默认为true,
* 并且如果response有结果, 则对获取的结果进行缓存;模块<3.1>
*/
//3--->
if (request.shouldCache() && response.cacheEntry != null) {
mCache.put(request.getCacheKey(), response.cacheEntry);
request.addMarker("network-cache-written");
}
request.markDelivered();
mDelivery.postResponse(request, response);
request.notifyListenerResponseReceived(response);
}
}
}
2.4 BasicNetwork.performRequest:
public class BasicNetwork implements Network {
@Override
public NetworkResponse performRequest(Request<?> request) throws VolleyError {
long requestStart = SystemClock.elapsedRealtime();
while (true) {
HttpResponse httpResponse = null;
byte[] responseContents = null;
List<Header> responseHeaders = Collections.emptyList();
// Gather headers.
Map<String, String> additionalRequestHeaders = getCacheHeaders(request.getCacheEntry());
/**
* 通过mBaseHttpStack最终将请求交给其内部的HttpURLConnection进行处理;
*/
httpResponse = mBaseHttpStack.executeRequest(request, additionalRequestHeaders);
int statusCode = httpResponse.getStatusCode();
responseHeaders = httpResponse.getHeaders();
/**
* 1. 根据不同的状态码做出不同的处理, https://www.jianshu.com/writer#/notebooks/11927320/notes/27327761
* 对响应码做了简单的记录;
* 2. 如果状态码为304, 则表示该Request已经请求过至少一次, 且服务器端数据没有发生变化,
* 此时直接从缓存中取数据;
* 3. 如果状态码不是304, 则跳转到模块<//2--->>读取流数据;
*/
//1--->
if (statusCode == HttpURLConnection.HTTP_NOT_MODIFIED) {
Entry entry = request.getCacheEntry();
if (entry == null) {
return new NetworkResponse(HttpURLConnection.HTTP_NOT_MODIFIED, null, true, SystemClock.elapsedRealtime() - requestStart, responseHeaders);
}
List<Header> combinedHeaders = combineHeaders(responseHeaders, entry);
return new NetworkResponse(HttpURLConnection.HTTP_NOT_MODIFIED, entry.data, true, SystemClock.elapsedRealtime() - requestStart, combinedHeaders);
}
//2--->
/**
* 1. 将流数据写入到reponseContents中, 并封装进NetworkResponse中;
* 2. 在模块<2.5>进行流数据的写入操作;
*/
InputStream inputStream = httpResponse.getContent();
if (inputStream != null) {
responseContents = inputStreamToBytes(inputStream, httpResponse.getContentLength());
} else {
responseContents = new byte[0];
}
// if the request is slow, log it.
long requestLifetime = SystemClock.elapsedRealtime() - requestStart;
logSlowRequests(requestLifetime, request, responseContents, statusCode);
if (statusCode < 200 || statusCode > 299) {
throw new IOException();
}
return new NetworkResponse(statusCode, responseContents, false, SystemClock.elapsedRealtime() - requestStart, responseHeaders);
}
}
}
2.5 BasicNetwork.inputStreamToBytes:
public class BasicNetwork implements Network {
private byte[] inputStreamToBytes(InputStream in, int contentLength) {
PoolingByteArrayOutputStream bytes = new PoolingByteArrayOutputStream(mPool, contentLength);
byte[] buffer = null;
try {
if (in == null) {
throw new ServerError();
}
buffer = mPool.getBuf(1024);
int count;
while ((count = in.read(buffer)) != -1) {
bytes.write(buffer, 0, count);
}
return bytes.toByteArray();
} finally {
if (in != null) {
in.close();
}
mPool.returnBuf(buffer);
bytes.close();
}
}
}
三、数据缓存:
3.1 DiskBasedCache.put:
public class DiskBasedCache implements Cache {
@Override
public synchronized void put(String key, Entry entry) {
/**
* 1. 在对数据进行缓存之前, 需要先判断缓存数据是否已满, 如果满了, 需要先对缓存数据进行处理,
* 这里的处理是采用LRU方式, 对最近最少使用的数据进行删除;
* 2. 对下文的继续分析可知, 数据缓存时, 以requestKey作为key, 然后删除时, 通过RequestKey
* 删除对应的CacheHeader;
* 3. 删除模块<3.2>
*/
pruneIfNeeded(entry.data.length);
/**
* 1. 对数据进行缓存, 此时缓存包括两部分:
* (1) 通过File完成磁盘缓存;
* (2) 通过mEntries完成内存缓存;
* 2. 其实还有一个是网络缓存, 即把获取的数据的响应头缓存起来, 再次请求相同Request时
* 带上响应头数据, 并判断返回结果, 如果结果为304, 则说明服务器端的缓存数据没有发生变化,
* 此时直接从内存中读取数据;
*/
File file = getFileForKey(key);
BufferedOutputStream fos = new BufferedOutputStream(createOutputStream(file));
CacheHeader e = new CacheHeader(key, entry);
/**
* 缓存到底缓存了哪些内容? 模块<3.3>
*/
boolean success = e.writeHeader(fos);
fos.write(entry.data);
fos.close();
putEntry(key, e);
return;
}
}
3.2 DiskBasedCache.pruneIfNeeded:
public class DiskBasedCache implements Cache {
private void pruneIfNeeded(int neededSpace) {
/**
* 如果缓存容量没有达到上限, 则不做处理;
*/
if ((mTotalSize + neededSpace) < mMaxCacheSizeInBytes) {
return;
}
int prunedFiles = 0;
/**
* mEntries为LinkedHashMap, 其内部有一个链表, put或get操作如果操作成功,
* 则将对应的Node置于链表尾部, 删除时从表头开始循环获取删除;
*/
Iterator<Map.Entry<String, CacheHeader>> iterator = mEntries.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry<String, CacheHeader> entry = iterator.next();
CacheHeader e = entry.getValue();
/**
* 对缓存数据进行删除;模块<//1--->>
*/
boolean deleted = getFileForKey(e.key).delete();
if (deleted) {
mTotalSize -= e.size;
}
iterator.remove();
prunedFiles++;
if ((mTotalSize + neededSpace) < mMaxCacheSizeInBytes * HYSTERESIS_FACTOR) {
break;
}
}
}
public File getFileForKey(String key) {
return new File(mRootDirectory, getFilenameForKey(key));
}
//1--->
private String getFilenameForKey(String key) {
int firstHalfLength = key.length() / 2;
String localFilename = String.valueOf(key.substring(0, firstHalfLength).hashCode());
localFilename += String.valueOf(key.substring(firstHalfLength).hashCode());
return localFilename;
}
}
3.3 CacheHeader.put:
static class CacheHeader {
// http响应首部中用于缓存新鲜度验证的ETag
final String etag;
// http响应首部中的响应产生时间
final long serverDate;
// 缓存的过期时间
final long ttl;
// 缓存的新鲜时间
final long softTtl;
// 响应的Headers
final List<Header> allResponseHeaders;
CacheHeader(String key, Entry entry) {
this(key, entry.etag, entry.serverDate, entry.lastModified, entry.ttl, entry.softTtl,
getAllResponseHeaders(entry));
size = entry.data.length;
}
private CacheHeader(String key, String etag, long serverDate, long lastModified, long ttl,
long softTtl, List<Header> allResponseHeaders) {
this.key = key;
this.etag = ("".equals(etag)) ? null : etag;
this.serverDate = serverDate;
this.lastModified = lastModified;
this.ttl = ttl;
this.softTtl = softTtl;
this.allResponseHeaders = allResponseHeaders;
}
}
3.4 DiskBaseCache.initialize:
public class DiskBasedCache implements Cache {
@Override
public synchronized void initialize() {
...
File[] files = mRootDirectory.listFiles();
if (files == null) {
return;
}
/**
* 读取指定目录下缓存文件, 然后通过putEntry将数据进行内存缓存, 在读取缓存时其实没有进行
* totalSize与MaxSize比较, 这是因为在进行缓存时, 已经做了限制最大容量不能超过MaxSize,
* 所以这里的totalSize < MaxSize一定成立;
*/
for (File file : files) {
long entrySize = file.length();
CountingInputStream cis = new CountingInputStream(
new BufferedInputStream(createInputStream(file)), entrySize);
try {
CacheHeader entry = CacheHeader.readHeader(cis);
entry.size = entrySize;
putEntry(entry.key, entry);
} finally {
cis.close();
}
}
}
private void putEntry(String key, CacheHeader entry) {
if (!mEntries.containsKey(key)) {
mTotalSize += entry.size;
} else {
CacheHeader oldEntry = mEntries.get(key);
mTotalSize += (entry.size - oldEntry.size);
}
mEntries.put(key, entry);
}
}
3.5 Request.parseNetworkResponse:
public class StringRequest extends Request<String> {
@Override
protected Response<String> parseNetworkResponse(NetworkResponse response) {
String parsed;
try {
parsed = new String(response.data, HttpHeaderParser.parseCharset(response.headers));
} catch (UnsupportedEncodingException e) {
parsed = new String(response.data);
}
return Response.success(parsed, HttpHeaderParser.parseCacheHeaders(response));
}
}
public class HttpHeaderParser {
public static Cache.Entry parseCacheHeaders(NetworkResponse response) {
long now = System.currentTimeMillis();
Map<String, String> headers = response.headers;
/**
* 1. 后续代码都是为了给这几个变量赋值;
* 2. 关于下面几个变量, 在<https://www.jianshu.com/p/064a51cdd79b>中进行了总结;
*/
long serverDate = 0;
long lastModified = 0;
long serverExpires = 0;
long softExpire = 0;
long finalExpire = 0;
long maxAge = 0;
long staleWhileRevalidate = 0;
boolean hasCacheControl = false;
boolean mustRevalidate = false;
String serverEtag = null;
String headerValue;
headerValue = headers.get("Date");
headerValue = headers.get("Cache-Control");
if (headerValue != null) {
hasCacheControl = true;
String[] tokens = headerValue.split(",");
for (int i = 0; i < tokens.length; i++) {
String token = tokens[i].trim();
if (token.equals("no-cache") || token.equals("no-store")) {
return null;
} else if (token.startsWith("max-age=")) {
maxAge = Long.parseLong(token.substring(8));
} else if (token.startsWith("stale-while-revalidate=")) {
staleWhileRevalidate = Long.parseLong(token.substring(23));
} else if (token.equals("must-revalidate") || token.equals("proxy-revalidate")) {
mustRevalidate = true;
}
}
}
headerValue = headers.get("Expires");
...
headerValue = headers.get("Last-Modified");
...
serverEtag = headers.get("ETag");
...
return entry;
}
}
针对响应头, 抓了一下简书首页的包:
https://www.jianshu.com/
网友评论