Prompt
I'm unhappy with our current media library because people keep uploading duplicates. How do I fix that?
Latest observation
Nov 3, 2026
OpenAI APIWeb search: off
To fix duplicate uploads in a media library, you usually want two layers of protection: one at upload time and one for existing content.
1. Prevent duplicates on upload
Best practice is to compare files by a content hash rather than filename.
- Generate a hash when a file is uploaded:
- e.g. SHA-256, MD5, or another checksum
- Store that hash in your database
- Before saving a new upload, check whether the same hash already exists
- If yes, reject the upload, link it to the existing asset, or warn the user
Why this works:
- Same filename can mean different files
- Different filenames can mean the same file
- Hashes identify the actual content
2. Normalize metadata and naming
Duplicates often slip in because the system treats minor differences as new files.
You can also:
- Strip metadata differences if that matters for your use case
- Standardize filenames on upload
- Track original filename separately from stored filename
- Create rules for near-duplicates if needed, like resized versions or re-exports
3. Deduplicate existing files
For a library already full of duplicates:
- Scan all files
- Compute hashes for each
- Group identical hashes
- Decide which copy to keep
- Update references to point to the canonical file
- Delete or archive the extras
If exact duplicates are not the only issue, you can also use:
- perceptual hashing for images
- audio fingerprinting for audio
- video frame hashes or fingerprints for video
4. Add UI feedback
Help users avoid re-uploading:
- Show “This file already exists”
- Offer a “use existing file” option
- Show similar files before upload completes
5. Enforce uniqueness at the database level
Don’t rely only on application logic.
- Add a unique constraint on the file hash column
- This prevents race conditions where two users upload the same file at the same time
6. Decide what “duplicate” means for your library
This matters a lot. You may want different rules for:
- exact binary duplicates
- same image with different dimensions
- same document with different metadata
- same content in different formats
If you want, I can help you design:
- a simple deduplication strategy for exact duplicates, or
- a more advanced system for detecting near-duplicates too.