ARTICLE AD BOX
I am developing a video editor app for Android and trying to extract frames from videos for timeline preview (like CapCut, VN, InShot apps).
I have already tried multiple approaches but all of them are too slow for real-time or smooth timeline scrubbing:
What I have tried:
FFmpeg (JavaCV / native FFmpeg build)
MediaMetadataRetriever
MediaExtractor + MediaCodec
OpenCV frame extraction
GPU/OpenGL based rendering experiments
Problem:
All approaches are slow when extracting frames in sequence, especially for:
timeline scrubbing
generating thumbnails for video editor timeline
fast seeking with smooth preview
Even MediaCodec + MediaExtractor becomes slow when decoding multiple frames continuously.
What I want to achieve:
I want frame extraction performance similar to apps like:
CapCut
VN Editor
InShot
These apps provide:
instant timeline scrubbing
fast frame preview
smooth thumbnail generation without lag
My questions:
How do these apps achieve such fast frame extraction?
Are they using FFmpeg, MediaCodec, or custom native solutions?
Do they use keyframe-only decoding or proxy videos?
What is the best architecture for fast video frame extraction on Android?
Is GPU (OpenGL / SurfaceTexture) required for this performance level?
Am I missing any optimization technique for MediaCodec or FFmpeg?
What I suspect:
Maybe they are using:
hardware accelerated decoding
keyframe seeking only
frame caching system
proxy/low-resolution video processing
GPU rendering instead of Bitmap conversion
But I am not sure how to implement this properly.
