https://1password.community/discussion/comment/697252/#Comment_697252 I have several logins that somehow have been duplicated in 1Password 8. This may have happened in different times when I've attempted to migrate my 1Password 7 data to my 1Password.com account. In any case, I have a load of duplicates in 1Password 8 and it would be very helpful if there were an efficient tool or feature to help me merge/dedupe these entries (and ideally test them to make sure that I'm preserving the correct credentials). Any thoughts/ideas for how to handle this situation? 1Password Version: 1Password for Mac 8.5.0 80500017, on BETA channel Extension Version: 2.1.4 OS Version: macOS 12.0.1 (21A559) https://1password.community/discussion/comment/662923/#Comment_662923

Hello everyone! 👋 With the release of the latest beta for Mac, Windows, and Linux you can now see a list of your duplicate items in the app. Choose “Items with duplicates” in your Watchtower dashboard to see or delete all of your duplicates. Make sure that you're running the latest beta to see the feature. As always, please let us know if you find the tool useful and if you have any suggestions for the team. 🙂 -Dave

In an ideal world, nobody will duplicate data at import time, but actually it will happen. I will not discuss how it can happen or can be avoided, only that it will happen - proof is this thread and others. Duplicate items are not always user error. Switching to a decent password manager often means merging from multiple sources, then consolidating: import from previous password manager plus Chrome's plus Firefox's plus Safari's internal password managers. One does import once directly after account creation, then never again. You at Agilebits do it never again, so you don't see really any need for postprocessing aid. However, for a new user it's crucial, it's a barrier. With duplicate items I mean login items that contain the same userid+password combination. What I propose you can do about this: - give an overview how many items match above description, in a similar way Watchtower does this. Also create filters, so the user can see a list of only matching entries. - also give an overview how many items match with url+userid, but different password. These might represent current and obsolete entries of the same login. The category could be something like "unable to merge", "merge conflict" or "password mismatch". Also make a filter for these. - explain if duplicates are distributed over multiple vaults. If 2 vaults contain the same, because the user imported the same 2 times, the user should see this and the easy fix for the user is to just delete one of these vaults - make it clear over how many different vaults how many duplicated entries are distributed - add the opportunity to "merge" duplicate entries in one vault, in spirit similar to functions to merge different address entries in address books in other apps. Merge only items within the same vault. - this merging would be available from the context menu you get if you select multiple entries. Merging candidates are only taken from the entries the user selected. - This merging functionality could find login entries with same userid+password combination, then choose the oldest entry as "merged" entry. Then merge all URLs from the other entries to this entry, the same with any custom fields and sections with different values (don't add custom fields with same title+value multiple times). And merge the password history of course. Also skip merging for history entries with identical password+timestamp combinations. After merging, add the tag "merged " to the merged entry (so the user can find all processed entries and can check if all is ok). Then delete the items their content was merged into the merged item. It's an investment in getting new users. I bet the turnover rate from ex-Lastpass users would be higher if you had such a functionality. When I was new, I merged entries from Password Safe and Chrome. I almost gave up, because it was so tedious - after all, Chrome autofill wasn't that bad, and Password Safe for just a local vault also wasn't that bad for all non-browser login items. A month after the trial expired I gave it a second shot and decided to do the tedious work.

I think duplicates issue is the biggest tragedy of 1password8. If user has 1password on two devices, there will be prompt on both and it will migrate vault twice. Can imagine high segment of 1password users who have multiple devices and gotten into this problem. So what happens when you write email to support+tradein@1Password.com as suggested above ? Do they give you special spell, hidden key combination to remove duplicates?

Duplicates issue is the disaster of 1Password8 upgrade. If user has 1Password on two devices, there will be prompt on both and it will migrate vault twice. That's a huge number of users! And now it's been months and still no remove duplicates function. Why the team can not take it seriously? What is more important than thousands of users experiencing very annoying issue.

@"Chris Messina" thanks for all your posts and bringing this to light. I'm in the exact same boat as you. I used the standalone version of 1password for many many years. And finally decided... I get enough value out of this that I don't mind paying a small monthly fee for the one software application that I use every single day, all day long. When we made the move to the new cloud subscription service everything seemed okay. But now as I'm going to use a login identity I'm seeing at least two versions of the exact same credentials. I recognize this may take some significant development to build a duplicate item cleanup tool, but I think it would really helpful to streamline our online identities.

Tool for detecting and removing duplicates? [Available in 8.10.18]

179 Replies

1P_Dave
Moderator
2 years ago
Since the 1Password 8 beta now has a tool to see and remove duplicate items I'm closing this thread. If you run into any trouble with the duplicates tool, or have any suggestions on how to make it better, then please open a new thread.

Thank you again for all of the feedback! 🙂

-Dave
1P_Dave
Moderator
2 years ago
@Taytay

Thanks for sharing your work! I hope that you find the new built-in “Items with duplicates” feature in Watchtower useful. 🙂

Obligatory disclaimer: 1Password hasn't reviewed the tool posted by @Taytay and folks wanting to try it do so at their own risk.

-Dave
ctphoto
Occasional Contributor
2 years ago
Taytay, This is good but there are many people who will have no idea how to use this. You should make a stand alone app and then sell it to 1Password.
Former Member
2 years ago
That's hilarious. I JUST wrote a CLI tool to do this after it drove me crazy enough. I should have checked here first!
For posterity's sake, here it is: https://github.com/Taytay/1password_utils
spinoza
Dedicated Contributor
2 years ago
Thank you! Delighted to see this.

I only found four duplicates, even though I have many near duplicates (same password, differ in trivial metadata.) Could you expand this feature to find items with the same password but different metadata?

Also, how about a way to bulk change http: to https: in Watchtower/unsecured websites.
1P_Tommy
Moderator
2 years ago
Indeed!
mikesilva
Occasional Contributor
2 years ago
Awesome sauce!
1P_Dave
Moderator
2 years ago

Hello everyone! 👋

With the release of the latest beta for Mac, Windows, and Linux you can now see a list of your duplicate items in the app. Choose “Items with duplicates” in your Watchtower dashboard to see or delete all of your duplicates.

Make sure that you're running the latest beta to see the feature. As always, please let us know if you find the tool useful and if you have any suggestions for the team. 🙂

-Dave
spinoza
Dedicated Contributor
2 years ago
How about a duplicate detection/merging test in Labs? (By duplicate, I mean near-duplicate.)
emusic917
New Contributor
3 years ago
Can you please stop with the whole "we're tallying votes," which is absolute nonsense. You didn't care five years ago about this issue and you care even less now. The software is useless and the support staff is condescending.
Embarrassed that I actually defended your team for years only to find out your heads are officially implanted in your choose your word of interest.