Two months ago, Troy Hunt, the security professional behind Have I been pwned?, released an incredibly comprehensive password list in the hope that it would allow web developers to steer their users away from passwords that have been compromised in past breaches.
While the list released by HIBP is hashed, the plaintext passwords are out there and one should assume that password crackers have access to them. So if you use a password on that list, you can be fairly confident that it's very easy to guess or crack your password.
I wanted to check my active passwords against that list to check whether or not any of them are compromised and should be changed immediately. This meant that I needed to download the list and do these lookups locally since it's not a good idea to send your current passwords to this third-party service.
I put my tool up on Launchpad / PyPI and you are more than welcome to give it a go. Install Postgres and Psycopg2 and then follow the README instructions to setup your database.
That pwned list of a password is a fantastic resource. Thanks for posting a pointer to it.
But Egad! - using postgres to index and search it?? You must have the patience of a saint.
Given a false positive isn't a death sentence, a bloom filter is a better choice. Setting the parameters to give a false positive range of 1e-9 (roughly 50/50 chance of getting 1 false positive if I checked a password with it every second for my entire life), the resulting filter occupies 2.6G - about 1/2 the size of the compressed original. Creating the filter takes about 3 hours on my laptop (please forgive the butt ugly inline python):
Querying it: