Incredibly, I found room on the internet for yet another duplicate file remover. What sets this one apart is that it will make guesses about which files you want to keep based on the filename. The app tries to select the shortest, most descriptive name as the one to keep. As an example: a file with a name containing "copy of", or ".1.txt" would be considered less descriptive than one without; and a file named "lkePic.jpg" is considered less descriptive than "Lake Pictures.jpg". This way the time you spend picking the right file to keep is minimized.
The app also makes use of a speed improvement I haven't seen anywhere else: It makes an extra pass of the file list to create a 'fasthash'. The fasthash uses small samples of the file (16 kilobytes), taken from the beginning, end, and three places in the middle; then does a duplicate check based on the hash of the samples. This is very quick for large files, and it eliminates the vast majority of potential duplicates, as most files will have different samples. Most other duplicate finders omit this step, but it really speeds things up.
As of beta 6 the app is fairly robust; avoiding most of the traps in this excellent reference.
Note: There's no fuzzy searching, this app only finds exact duplicates.
Version 4.6.2 or better of the .net framework is recommended. Get it from Microsoft.
No installation; just unpack and run. A settings file and ancillary files may be created in the program folder.
- Pick a directory where you have duplicate files and hit scan.
- After the program identifies the duplicate files, you'll be given a list to choose what to keep and what to delete. The app makes suggestions, but you can make changes by right clicking and selecting delete or keep; Or by double-clicking the 'Action' column to toggle it.
- Hit the 'Delete Marked Files' to execute your decisions. You can make a log to see what was done. The app will reset after this.
Each duplicate will have an action automatically assigned, and you can manually set the action by right clicking or using the keyboard. Available actions are: Keep [spacebar], Delete [delete key], Link [F4 key]. The context menu option 'Mark as auto' will reset the action to whatever was automatically assigned. The 'Invert Action' option will toggle the action between 'Keep' and whatever the default action is (set this in settings, by default it's 'Delete').
In version 0.8 and above, an 'Advanced criteria' option is added that lets you include or exclude multiple folders and have filtering options. These criteria can be named and permanently saved. To create a criteria, select 'Advanced Criteria..." from the 'Look In" drop-down; the edit criteria form will the show with further options.
When using Advanced Criteria the '...' button is used to edit the criteria. To remove a saved criteria, select it, edit it, and hit the 'Delete this Criteria' button in the lower left.
Notes about symbolic links
In recent versions the application can create symbolic links instead of deleting files. Symbolic links are like shortcuts, they point to the original file; this lets you keep the file's alternate name and location, without wasting the space of having the file's data duplicated.
A few caveats: +The app usually needs to be run as an administrator to create symbolic links. +At least one file must be kept to create links (obviously). +The links created by the app will always target the first file you are keeping.
Links are created in steps, to prevent inadvertent deletions if link creation fails; for each file we want to link we:
1: Rename the file we are removeing/replacing to a dummy name.
2: Try to create our link, using the original name from the file in step 1.
3: If creating the link suceeded, we delete the orginal (renamed) file.
After scanning, the following keys can be used on the file list:
- [Enter] Open the file.
- [Delete]: Mark the file for deletion
- [Space]: Mark the file to keep.
- [Ctrl+A]: Select all
- [Ctrl+C]: Copy filenames to clipboard
- [Ctrl+Alt+C]: Copy all info to the clipboard.
- [Ctrl+X]: Exchange the selected items actions. (Swaps between Keep and the default removal action)
Note: The directory textBox, in additon to specifying plain directories, can have wildcard '*' or any-character '?' filters. eg:
Command Line Usage:
If an instance of the program is not already running, starts the program and immediately hides it.
Scan the specified directory.
Version 1.0; 2019-08-21
- Added: Blake2B hash function; in most cases it's faster than even SHA1 and md5 while providing a vastly larger address space. This is the new default hash, no more worrying about those damn SHAttered pdf's :)
- Added: Extra warning in case the user has selected all extant copies of a file for deletion.
- Added: You can now drag a folder to the app to start scanning it. (Files too, it'll scan the parent folder.)
- Changed: Better progress bar(s). Also we now use a wait cursor during the 'execute actions' work.
- Changed: Some new requirements: we now target version 4.6.2 of the .net framework to take advantage of long-path aware API's. We also moved to an x64 executable to take advantage of BLAKE2B's speed. The older 'any CPU' version supporting .net 4.0 will remain here: https://cresstone.com/apps/DupeKill/DupeKill_v0.8.zip#oldVersion
- Fixed: Some improvements in data entry & tab navigation on the 'advanced criteria' form.
Beta 8; 2017-07-09
- Added: Major feature: advanced criteria scans can be configured via the 'address bar' drop-down.
- Added: "Include subfolders" is now a remembered setting.
- Added: New option in settings to remember recent scan paths.
- Added: New option in settings to sort scan results by size or # of dupes. (On-demand sorts may be next, but are not easy due to the potential size of scans)
- Added: "Copy all info" command file list context menu (keyboard Ctrl+Alt+C). The following information about the selected items is copied to the clipboard (tab separated): full path, file size, creation date, modification date, file signature, action
- Fixed: list no longer flickers on action change.
Beta 7; 2016-12-29
- Added: Symbolic link code finished and activated.
- Added: Ctrl+mousewheel zooming.
- Added: Some keyboard shortcuts for buttons, check the tooltips.
- Changed: Settings for window position now stored in a sane way.
- Fixed: Turned off single-instance limit.
- Changed: Start with windows removed (another setting inherited from the template that didn't make a lot of sense, since we're mostly running on-demand.)
Beta 6; 2016-02-09
- Fixed: Now ignoring reparse points & symbolic links.
- Added: Settings dialog updated (links in about textbox and start menu shortcut option)
Beta 5; 2015-11-07
- Fixed: Item coloring regression.
- Added: File creation/modification date columns
- Added: column sizes remembered
Beta 4; 2015-11-05
- Added: Choices for hash algo added to settings; also added hash benchmark option to see what's fastest.
- Some optimizations
- UI tweaks
Beta 3; 2015-11-03
- Changed: Now using FindFirstFile/FindNextFile via pInvoke instead of Directory.EnumerateFiles/FileInfo; Because on network drives FileInfo is slow
- Added: Abort button to cancel active scan
- Added: Space savings from perspective deletions now shown.
- Added: Shell integration now available in settings.
- Added: Size is now shown under the id; Still not entirely happy with how this looks.
- Fixed: Can now delete read-only files.
Beta 1; 2015-10-28
- Initial Release. From concept to working in a single Sunday. Damn it feels good to be a gangster.
This software includes code or resources from the following sources:
This software is distributed as-is, without any representations or warranties of any kind.
The author of this software imposes no additional license terms or limits upon its use or redistribution.
Send to email@example.com