MediaCleaner Pro processes your image library in a single streaming pass. Every image file discovered in the source directory is routed through all 10 detection stages in sequence — no multi-pass scans, no temporary databases. The pipeline is designed to stay memory-efficient regardless of collection size: images are read, hashed, staged, and moved as a continuous stream rather than loaded into memory all at once.Documentation Index
Fetch the complete documentation index at: https://mintlify.com/xcoder-es/media-cleaner-pro/llms.txt
Use this file to discover all available pages before exploring further.
Architecture
The pipeline is split into five phases that hand work off to each other through async channels:Directory Walk
MediaCleaner Pro recursively scans
SOURCE_DIR using WalkDir, collecting every file whose extension matches a supported image format: jpg, jpeg, png, bmp, webp, gif, tiff, or tif. The total file count is captured upfront so progress percentages stay accurate throughout the run.Concurrent Reading
Up to 16 files are read in parallel via a Tokio async I/O semaphore. Each read spawns a non-blocking task that acquires a permit, reads the raw bytes from disk, and forwards them to the hash worker channel. The semaphore cap prevents the pipeline from saturating the filesystem with hundreds of simultaneous open file handles.
Hash Computation
A dedicated hash worker receives raw file bytes and dispatches CPU-bound work onto Rayon’s thread pool via
block_in_place. For each image, three operations run synchronously:- SHA-256 — a 256-bit cryptographic hash used for exact duplicate detection
- dHash (difference hash) — a 64-bit perceptual fingerprint computed by downsampling the image to 9×8 grayscale and comparing adjacent pixel brightness
ImageMetadata struct (path, filename, dimensions, format, sha256, dhash) is forwarded to the stage pipeline channel.Stage Pipeline
Each
ImageMetadata value passes through all 10 stages in order. The stages run as pure functions — no shared mutable state between images except for the exact-hash set (a HashSet<String> behind a Mutex) and the perceptual duplicate detector (a bucket map behind a second Mutex). Every stage returns a StageResult recording whether the image passed, the destination subdirectory if it was claimed, and an optional confidence score.File Routing
After all 10 stages have run, the pipeline determines the final destination for each image. The first stage that did not pass wins and sets the output subdirectory. If all rejection stages passed but the AI Classification stage assigned a category, that category’s subdirectory is used. Images that pass every stage with no category remain in the source directory untouched.
The 10 Stages at a Glance
Each stage is implemented as a method onStageProcessor in mc-core/src/services.rs. Stages 1–8 can reject an image and stop routing; Stages 9 and 10 always pass (they classify and rank rather than reject).
| # | Stage | Detection Method | Output Destination |
|---|---|---|---|
| 1 | Exact Duplicate Removal | SHA-256 hash comparison against all previously seen hashes | duplicates/exact/ |
| 2 | Perceptual Duplicate Removal | dHash + Hamming distance ≤ HAMMING_THRESHOLD (default 4) | duplicates/perceptual/ |
| 3 | Tiny Image Detection | Width < MIN_WIDTH or height < MIN_HEIGHT (default 100 × 100) | rejected/tiny/ |
| 4 | Icon Detection | Square aspect ratio (0.9–1.1) and both dimensions ≤ 256 px | categories/icons/ |
| 5 | Thumbnail Detection | Filename contains thumb/thumbnail/small/mini/preview/tn_, or width ≤ 320 or height ≤ 240 | categories/thumbnails/ |
| 6 | Screenshot Detection | Matches one of 16 common screen resolutions (both orientations), or filename contains screenshot/screencapture | categories/screenshots/ |
| 7 | Wallpaper Detection | Ultrawide aspect ratio 1.8–3.5, or 4K+ resolution with aspect ≥ 1.5 | categories/wallpapers/ |
| 8 | Document Detection | Aspect ratio within ±0.15 of standard paper sizes (A4, Letter, etc.) and width ≥ 1200 px | categories/documents/ |
| 9 | AI Classification | Heuristic classifier — portrait, landscape, macro, panorama, social, low_quality | categories/{type}/ |
| 10 | Quality Ranking | Composite score (0–100) from resolution, file size, and aspect ratio | quality/{excellent|good|average|below_average|poor}/ |
The pipeline processes images in a streaming fashion — each image is read, hashed, staged, and handed off before the next batch is read. Memory usage stays bounded even for collections with hundreds of thousands of images.