Android 大量图片加载导致缓存频繁请求问题梳理(Glide为例)

没有了遇见
511 阅读7分钟

问题梳理

前提:

公司项目是一个直播类项目,等级,礼物,头像,发布的照片视频等场景产生了巨量的图片(大量图片缓存).

问题:

大量图片缓存后,图片缓存框架缓存超过设置的阀值,根据LruCache算法开始清理图片,导致一些图片的Url频繁从网络获取,后台炸了,为什么头像,礼物图片接口频繁调用,所以问题就来了,开发牛马开始发力了.

思考:

根据问题逆向,图片接口频繁调用,是因为图片本地缓存被清理,导致需要从网络获取图片,那么只要是从本地加载图片就可以避免接口频繁调用问题的存在.那么有什么方法处理大量图片缓存不被清理呢???

  • 增大图片磁盘缓存区大小(简单粗暴)
  • 图片分类型存储,特定类型使用不同的磁盘存储控件(需要重建缓存逻辑)
  • 图片增加永久存储的图片类型(会导致缓存图片一直存在)

以下以Glide为例部分代码实现逻辑

注意:

Glide默认磁盘大小是250M 当超过这个阀值的时候,会根据LruCache算法 清理文件

image.png

1.自定义Glide磁盘缓存空间

Glide说明

image.png


@GlideModule
public class YourAppGlideModule extends AppGlideModule {
  @Override
  public void applyOptions(Context context, GlideBuilder builder) {
    int diskCacheSizeBytes = 1024  1024  100;  100 MB
    builder.setDiskCache(
        new InternalCacheDiskCacheFactory(context, cacheFolderName, diskCacheSizeBytes));
  }
}

2.图片分类类型存储(项目使用中,待验证)

核心思路:

Glide的默认磁盘缓存大小是250M,既然大量图片撑爆了这个阀值,那么我们就多创建几个缓存区就好了.但是Glide不支持多个缓存区的设置,所以我们就要拦截Glide的网络请求将我们特定的图片存储在特定的区域.然后获取的时候从特定区域获取就行了

步骤:

  • 指定类型图片携带类型请求

    • Url拼接类型(在用)
    • RequestOption signature (获取的时候出现问题)
    • header 同上

  • 拦截Glide网络请求

  • 分区域存储特定类型的图片

  • 分区域读取特定的图片

2.1 指定类型图片携带类型请求


  /**
   * 处理按类型缓存图片
   * @param context Context
   * @param url String
   * @param imageView ImageView
   * @param requestOptions RequestOptions?
   * @param type String 图片类型 GlideConstant
   */
private  fun loadImageByType(   context: Context,  url: Any, imageView: ImageView, requestOptions: RequestOptions?,
                         type: String){
      var imgUrl=url
      //存在特殊类型做拼接
      if (!TextUtils.isEmpty(type)&&url is String){
          imgUrl= UriUtil.appendParameter(url,GlideConstant.glide_save_type_key,type)
      }

      requestOptions?.let {
          GlideApp.with(context).load(imgUrl).apply(requestOptions).into(imageView)
      }.apply {
          val request =  RequestOptions()
          request.error(R.mipmap.iv_glide_error)
          request.placeholder(R.mipmap.iv_default9)
          request.priority(Priority.HIGH)

          GlideApp.with(context).load(imgUrl).apply(request)  .centerCrop().into(imageView)
      }
  }



// 拼接参数
fun appendParameter(url: String, paramKey: String, paramValue: String): String {
    try {

        if (TextUtils.isEmpty(url) || TextUtils.isEmpty(paramKey)||url.contains(paramKey)) return url
        // 对参数值进行编码
        val encodedValue = URLEncoder.encode(paramValue, StandardCharsets.UTF_8.toString())
        // 检查 URL 中是否已经有参数
        val separator = if (url.contains("?")) "&" else "?"
        return "$url$separator$paramKey=$encodedValue"
    } catch (e: UnsupportedEncodingException) {

        return ""
    }
}

2.2 拦截Glide网络请求

2.2.1 自定义 AppGlideModule 拦截网络请求

/**
 * @Author: wkq
 * @Time: 2025/4/10 15:16
 * @Desc:
 */
@GlideModule
public class VoiceGlideApp extends AppGlideModule{

    @Override
    public void applyOptions(@NonNull Context context, @NonNull GlideBuilder builder) {
        builder.setDiskCache(new VoiceDiskCacheFactory(new VoiceDiskCacheFactory.CacheDirectoryGetter() {
            @NonNull
            @Override
            public File getCacheDirectory() {
                return new File(context.getCacheDir(), GlideConstant.INSTANCE.getDir());
            }
        }));
    }
    public boolean isManifestParsingEnabled() {
        return false;
    }

    @Override
    public void registerComponents(@NonNull Context context, @NonNull Glide glide, @NonNull Registry registry) {
        OkHttpClient.Builder builder = new OkHttpClient.Builder();
        builder.addInterceptor(new ProgressInterceptor());
        OkHttpClient okHttpClient = builder.build();
        registry.replace(GlideUrl.class, InputStream.class, new OkHttpGlideUrlLoader.Factory(okHttpClient));

    }
}
2.2.2自定义 ModelLoader
/**
 * @Author: wkq
 * @Time: 2025/4/10 17:44
 * @Desc:
 */
public class OkHttpGlideUrlLoader implements ModelLoader<GlideUrl, InputStream> {
    private final Call.Factory client;

    @SuppressWarnings("WeakerAccess")
    public OkHttpGlideUrlLoader(@NonNull Call.Factory client) {
        this.client = client;
    }

    @Override
    public boolean handles(@NonNull GlideUrl url) {
        return true;
    }

    @Override
    public LoadData<InputStream> buildLoadData(@NonNull GlideUrl model, int width, int height,
                                               @NonNull Options options) {
        return new LoadData<>(model, new OkHttpFetcher(client, model));
    }

    @SuppressWarnings("WeakerAccess")
    public static class Factory implements ModelLoaderFactory<GlideUrl, InputStream> {
        private static volatile Call.Factory internalClient;
        private final Call.Factory client;

        private static Call.Factory getInternalClient() {
            if (internalClient == null) {
                synchronized (OkHttpGlideUrlLoader.Factory.class) {
                    if (internalClient == null) {
                        internalClient = new OkHttpClient();
                    }
                }
            }
            return internalClient;
        }

        public Factory() {
            this(getInternalClient());
        }

        public Factory(@NonNull Call.Factory client) {
            this.client = client;
        }

        @NonNull
        @Override
        public ModelLoader<GlideUrl, InputStream> build(MultiModelLoaderFactory multiFactory) {
            return new OkHttpGlideUrlLoader(client);
        }

        @Override
        public void teardown() {}
    }
}
2.2.3自定义 DataFetcher(核心代码)
/**
 * @Author: wkq
 * @Time: 2025/4/10 17:46
 * @Desc:
 */
public class OkHttpFetcher implements DataFetcher<InputStream>, okhttp3.Callback {
    private final Call.Factory client;
    private final GlideUrl url;
    private InputStream stream;
    private ResponseBody responseBody;
    private DataFetcher.DataCallback<? super InputStream> callback;
    private volatile Call call;



    @SuppressWarnings("WeakerAccess")
    public OkHttpFetcher(Call.Factory client, GlideUrl url) {
        this.client = client;
        this.url = url;
    }

    @Override
    public void loadData(@NonNull Priority priority,
                         @NonNull final DataCallback<? super InputStream> callback) {
        Request.Builder requestBuilder = new Request.Builder().url(url.toStringUrl());
        for (Map.Entry<String, String> headerEntry : url.getHeaders().entrySet()) {
            String key = headerEntry.getKey();
            requestBuilder.addHeader(key, headerEntry.getValue());
        }
        Request request = requestBuilder.build();
        this.callback = callback;

        call = client.newCall(request);
        call.enqueue(this);
    }

    @Override
    public void onFailure(@NonNull Call call, @NonNull IOException e) {
        callback.onLoadFailed(e);
    }

    @Override
    public void onResponse(@NonNull Call call, @NonNull Response response) throws IOException {
        responseBody = response.body();
        if (response.isSuccessful()) {
            long contentLength = Preconditions.checkNotNull(responseBody).contentLength();
            stream = ContentLengthInputStream.obtain(responseBody.byteStream(), contentLength);
            String type=UriUtil.INSTANCE.extractSourceKey(GlideConstant.INSTANCE.getGlide_save_type_key(),url.toStringUrl());

            if (url!=null&&!TextUtils.isEmpty(url.toStringUrl())&&! TextUtils.isEmpty(type)){
                String mdfUrl= SecretUtil.getMD5Result(url.toStringUrl());
                DiskCacheManager.getInstance(VoliceApplication(),type).put(mdfUrl,stream);
            }
            callback.onDataReady(stream);
        } else {
            callback.onLoadFailed(new HttpException(response.message(), response.code()));
        }
    }

    @Override
    public void cleanup() {
        try {
            if (stream != null) {
                stream.close();
            }
        } catch (IOException e) {
            // Ignored
        }
        if (responseBody != null) {
            responseBody.close();
        }
        callback = null;
    }

    @Override
    public void cancel() {
        Call local = call;
        if (local != null) {
            local.cancel();
        }
    }

    @NonNull
    @Override
    public Class<InputStream> getDataClass() {
        return InputStream.class;
    }

    @NonNull
    @Override
    public DataSource getDataSource() {
        return DataSource.REMOTE;
    }
}

2.3 分区域存储

在OkHttpFetcher 的请求响应中分区域存储图片

    @Override
    public void onResponse(@NonNull Call call, @NonNull Response response) throws IOException {
        responseBody = response.body();
        if (response.isSuccessful()) {
            long contentLength = Preconditions.checkNotNull(responseBody).contentLength();
            stream = ContentLengthInputStream.obtain(responseBody.byteStream(), contentLength);
            //获取类型 根据指定的类型存储图片
            String type=UriUtil.INSTANCE.extractSourceKey(GlideConstant.INSTANCE.getGlide_save_type_key(),url.toStringUrl());

            if (url!=null&&!TextUtils.isEmpty(url.toStringUrl())&&! TextUtils.isEmpty(type)){
            //将url转为md5 方便存读取
                String mdfUrl= SecretUtil.getMD5Result(url.toStringUrl());
               //自定义 DiskLruCache 存储文件 
               DiskCacheManager.getInstance(VoliceApplication(),type).put(mdfUrl,stream);
            }
            callback.onDataReady(stream);
        } else {
            callback.onLoadFailed(new HttpException(response.message(), response.code()));
        }
    }

2.4 分区域获取图片

2.4.1 自定义DiskCache.Factory 处理磁盘缓存

image.png

/**
 *
 *@Author: wkq
 *
 *@Time: 2025/4/10 13:43
 *
 *@Desc:
 */
class VoiceDiskCacheFactory(var cacheDirectoryGetter: CacheDirectoryGetter) :
    DiskCache.Factory {

    interface CacheDirectoryGetter {
        val cacheDirectory: File

    }

    override fun build(): DiskCache? {
        val cacheDir: File =
            cacheDirectoryGetter.cacheDirectory

        cacheDir.mkdirs()


        return if ((!cacheDir.exists() || !cacheDir.isDirectory)) {
            null
        } else VoiceDiskLruCacheWrapper.create(
            cacheDir,
            500 * 1024 * 1024
        )

    }
}
2.4.2 处理自定义缓存的读取

/**
 * @Author: wkq
 * @Time: 2025/4/11 14:46
 * @Desc:
 */
public class VoiceDiskLruCacheWrapper implements DiskCache {

    private static final String TAG = "VoiceDiskLruCacheWrapper";

    private static final int APP_VERSION = 1;
    private static final int VALUE_COUNT = 1;
    private static VoiceDiskLruCacheWrapper wrapper;

    private final SafeKeyGenerator safeKeyGenerator;
    private final File directory;
    private final long maxSize;
    private final DiskCacheWriteLocker writeLocker = new DiskCacheWriteLocker();
    private DiskLruCache diskLruCache;


    @Deprecated
    public static synchronized DiskCache get(File directory, long maxSize) {

        if (wrapper == null) {
            wrapper = new VoiceDiskLruCacheWrapper(directory, maxSize);
        }
        return wrapper;
    }


    public static VoiceDiskLruCacheWrapper create(File directory, long maxSize) {
        return new VoiceDiskLruCacheWrapper(directory, maxSize);
    }


    protected VoiceDiskLruCacheWrapper(File directory, long maxSize) {
        this.directory = directory;
        this.maxSize = maxSize;
        this.safeKeyGenerator = new SafeKeyGenerator();
    }

    private synchronized DiskLruCache getDiskCache() throws IOException {
        if (diskLruCache == null) {
            diskLruCache = DiskLruCache.open(directory, APP_VERSION, VALUE_COUNT, maxSize);
        }
        return diskLruCache;
    }

    //正则表达式获取数据中的url(Glide做了处理不能直接获取,只能从字符串中截取)
    public String extractSourceKey(String key, String input) {
        if (input == null || input.isEmpty()) {
            return null;
        }
        // 定义正则表达式
        String regex = key + "=([^,]+)";
        Pattern pattern = Pattern.compile(regex);
        Matcher matcher = pattern.matcher(input);

        if (matcher.find()) {
            // 提取匹配到的组
            return matcher.group(1);
        }
        return null;
    }


    @Override
    public File get(Key key) {
        //获取文件
        File result = null;
        String safeKey = safeKeyGenerator.getSafeKey(key);
        try {
            final DiskLruCache.Value value = getDiskCache().get(safeKey);
            if (value != null) {
                result = value.getFile(0);
            }
            if (result==null||result.length()==0){
                
                String url = extractSourceKey("sourceKey", key.toString());
                    // 获取url上边的key
                String type = UriUtil.INSTANCE.extractSourceKey(GlideConstant.INSTANCE.getGlide_save_type_key(), key.toString());
                if (!TextUtils.isEmpty(type)) {
                    String mdfUrl = SecretUtil.getMD5Result(url);
                    // 从自定义的disklrucache 工具类中获取 图片
                    //注意 文件末尾会自动拼接.0或者.1 获取的时候要手动拼接上
                    String data = DiskCacheManager.getInstance(VoliceApplication(),type).getFilePath(mdfUrl);
                    if (!TextUtils.isEmpty(data)) {
                        return new File(data);
                    }
                }
            }


        } catch (IOException e) {

        }



        return result;
    }

    @Override
    public void put(Key key, DiskCache.Writer writer) {

        String safeKey = safeKeyGenerator.getSafeKey(key);
        writeLocker.acquire(safeKey);
        try {
            try {

                DiskLruCache diskCache = getDiskCache();
                DiskLruCache.Value current = diskCache.get(safeKey);
                if (current != null) {
                    return;
                }

                DiskLruCache.Editor editor = diskCache.edit(safeKey);
                if (editor == null) {
                    throw new IllegalStateException("Had two simultaneous puts for: " + safeKey);
                }
                try {
                    File file = editor.getFile(0);
                    if (writer.write(file)) {
                        editor.commit();
                    }
                } finally {
                    editor.abortUnlessCommitted();
                }
            } catch (IOException e) {

            }
        } finally {
            writeLocker.release(safeKey);
        }
    }

    @Override
    public void delete(Key key) {
        String safeKey = safeKeyGenerator.getSafeKey(key);
        try {
            getDiskCache().remove(safeKey);
        } catch (IOException e) {

        }
    }

    @Override
    public synchronized void clear() {
        try {
            getDiskCache().delete();
        } catch (IOException e) {

        } finally {

            resetDiskCache();
        }
    }

    private synchronized void resetDiskCache() {
        diskLruCache = null;
    }
}

注意:

Disklrucache 会在尾部自动拼接.0或者.1

image.png

3.修改Glide缓存逻辑

Glide 缓存文件有 journal 文件其中维护了 缓存文件的状态

  • DIRTY 行用于跟踪条目正在被创建或更新。 每次成功的 DIRTY 操作后都应执行 CLEAN 或 REMOVE 操作。没有匹配 CLEAN 或 REMOVE 操作的 DIRTY 行表示可能需要删除 临时文件。
  • CLEAN 行用于跟踪已成功发布且可以读取的缓存条目。发布行后跟其每个值的长度。
  • READ 行用于跟踪 LRU 的访问。
  • REMOVE 行会跟踪已删除的条目。
private static final String CLEAN = "CLEAN"; 
private static final String DIRTY = "DIRTY";
private static final String REMOVE = "REMOVE";  移除
private static final String READ = "READ";  读取

image.png

思路:

Glide 的缓存是根据 journal 文件每行的状态动态删除文件的逻辑 所以想要处理磁盘缓存的数据,需要动态处理DiskLruCache文件中的状态,自定义Glide的文件 增加一个不可删除的状态就可以了

总结

Glide 缓存磁盘缓存处理,需要自定义Glide的缓存配置,或增加缓存大小,或增加多个缓存路径,或自定义缓存逻辑.

avatar
文章
阅读
粉丝