Tapestry Training -- From The Source

Let me help you get your team up to speed in Tapestry ... fast. Visit howardlewisship.com for details on training, mentoring and support!

Monday, February 06, 2006

Why store passwords in the database?

I'm always amazed whenever I hear about some security fiasco where a whole bunch of user information, including passwords, is obtained from some unprotected database.

I mean, I may not be a security expert, but even I know that you don't need to store the actual password in a database to allow users to login in securely.

My standard approach is to store the MD5 checksum of the password, rather than the password itself. In fact, I go further; I build a string combining the user's login id and their password and generate and MD5 from that. This has the advantage that different users with the same password will have a different value stored in the database, making it that much more difficult to gain wider access, even if some other security flaw gives the black hats read access. It doesn't even give up the size of the password, since md5 checksum are always the same length, regardless of the number of bytes in the stream used to build the checksum.

This is still a naive approach; you can do more, including adding additional "salt" to the alogiritm (such as a random number, stored only on the server, that is also factored into the checksum). However, at the core of this approach is the desire to never store a password as plain text. Encrypt it, hash it, obscure it. All you need to do is prove that the user knows the password and you don't need to store the password in plain text to do that.

As I remember, this is the way that Unix has traditionally stored password data, so I continue to wonder whenever I see a design that stores passwords out in the open, waiting to be harvested by the black hats.


Anonymous said...

I agree...except I use SHA-1, since MD5 is not so secure anymore.

Avah said...

If it's an intranet application, the best way is to even not have a storage for passwords; Just use the native, in-office identification method via some JAAS or whatever you are comfortable with.

Let the OS do the identification, if the users are already registered in its directory service, be it Active Directory or JES Directory.

Jamie Orchard-Hays said...

No kidding, Howard. I am constantly amazed at how often I sign up for something and I am sent my username and password as plaintext in an email! TextDrive--a company you'd think would know better--did this when I signed up recently. Inexcusable!

Anonymous said...

When I suggested this very thing two companies ago, the engineers expressed that it would only cause trouble and make their work more cumbersome as they usually forgot what the test accounts were.

The SHA-1 hashing was accepted nonetheless but not without the occasional groans about having to remember to call that stupid hashing method. Eventually everybody got used to it eventually and started groaning about some other difficult task.

Anjan said...

hi howard/all,

what is your strategy for providing links "Lost your username / password" ? Do you usually ask for a hint and then send the users their passwords(which you can't since you don't store them) OR do you ask users to register a new password ?


ciukes said...

> what is your strategy for providing
> links "Lost your username /
> password" ?
I ask for hint and email new generated password if this don't break security. Otherwise I allow set new password.

What about https? How often do you have to logon/fill forms on unsecured http page?
I really don't like my data flow through my provider's proxies etc. But it seems many of site admins don't care about that.

Anonymous said...

as you cannot send them their old password (because it's "encrypted"), you have to send a new one.
i let them enter the email they registered with, and then send a mail with a confirmation link, to make sure they really want their pw to be changed (and not someone else..). if there is an extra login name, don't send it in the mail with the new (random) password..

i hate this "hint" approach: either it's too easy to guess for anyone who knows you , or it's to hard to remember for yourself. .)

Rickard said...

I'm always amazed that people want to store user information at all in a database. When you get to the point where you have two apps in an organization (which typically happens reasonably soon) then that user database is a f-g nightmare. Better to use an LDAP directory from the start. Or even better, develop your app as a portlet and avoid the user authentication completely.

Anonymous said...

Your're forgetting that a LDAP directory is nothing than just another kind of database...

Rickard said...

And I guess it's the kind of ignorance that Anonymous displays that is the reason for why so few store user information in an LDAP directory. Apart from being a blatant half-truth (a filesystem is a database too, but it's not really a database, is it?), it fails to recognize how LDAP directories are used in real life.

All of our customers who are not using a hosted solution have their own LDAP directory (either eDirectory or Active Directory) where they store their user information (and the hosted customers use directories since we provide it behind the scenes). They store it there in part because that's what UNIX and Windows networks integrate with, and in part because by doing so you don't get the double administration that having user information spread out in umpteen application databases would imply.

Another big benefit you get by using an LDAP directory is that they are actually quite well structured for what they are used. Not only can directories store user data, they can also store hierarchical organizational information, what groups users are members of, and groups can even be created dynamically based on user attributes. Things like that, stuff that matters.

That passwords are not stored as-is is so ridiculously rudimentary that it shouldn't even be a topic.

So by reinventing the wheel over and over again by using custom user databases you are on the one hand declaring your own ignorance, and on the other you are saying "fuck you" to your customer who will, once they get that second application, deal with the headache of keeping two user databases updated.

I see this problem with user databases ALLLLL the time, and the desperation of the customers and production people who have to deal with it is quite depressing.

If it wasn't so sad this whole tragedy would be funny.

Alejandro said...

I'm the one who had make User & Role pojos in TRAILS.
I stored plain text password just to make it abalilable faster. I was planning to use MD5 after everthing else works. After all it is still under development :)
I promise you TRAILS will use MD5 to store passwords.

Rickard said...

alejandro, I sincerely hope that your comment was not directed to me, because it saddens me to see that apparently you didn't read a single word of what I wrote.

Do Not Use Your Own User Database. Period.

Anonymous said...

alejando.. what do you mean by making it available faster. setting the passwords hashed doesnt really take any time?

rickard, how would you implement the LDAP solution to a, say, webmail application where the users need to access the webapp from random locations? If you can implement better solution than hashed passwords (salted and all) please tell me.

What comes to retrieving lost passwords, you just need to use tsl/ssl when you ask the verifying question, and after it give a forced password change form. since the password isn't know to the server, you either need to generate a new or trust that credentials were correct with the security question and have a valid login. and what point would it be to generate a quite hard password for the user for him to just forget it or change it immediatly. so let him do it after giving correct answer to the security question

NEVER ... NEVER EVER!.. send the password over unsecured connection.

Rickard said...

Anonymous, I'm not sure I understand the question. If you're doing a webmail app, then that is only a frontend for some IMAP store typically, in which case that IMAP store is responsible for the authentication of the user. The application would only be a user interface for that underlying service.

I was referring to the case when YOU are in charge of the actual application, and have a choice with regard to where to store the password. In which case the answer should be: don't.

But preferably I would even take it one step further: don't implement user authentication at all! Instead write the app as a portlet and rely on the portal implementation for authentication. In our portal product we support many different kinds of authentications, like certificates and SMS logins and things like that. It would be way way too costly I think if each app tried to keep up with the different kinds of authentication schemes that are available out there. By delegating that to the portal you just avoided a major problem. That is, given that you have access to a portal platform that you like and which has such support for different kinds of authentication.

Music Snob said...

I agree completely with Rickard here.

Never ever, ever build a system that stores passwords. In fact never build one that stores user account data altogether. This is what directory services are for; it’s what they are good at.

By doing it yourself you are only wasting your time.

1. You are essentially just writing your own directory service which will pale in comparison.
2. It is highly unlikely that you will ever build an encryption and authentication system to the level of sophistication and performance of 99% of the directory services out there.
3. You are making a maintenance nightmare.
4. It will take you twice as long to rebuild your own directory service as doing the research into how to use one.
5. You will take even longer to develop all the extra features such as auditing, security levels, replication & synchronization etc. etc.
6. You will make mistakes which will create holes.

And the great thing is with things like ADAM you can create standalone directory services which are application specific so there is no excuse. Plus you are getting one of the most sophisticated and secure (I know it’s MS but if it’s good enough for government…) directory systems there is.

Also, DS are not the same as relational databases. They are optimised for read intensive activities which RDBMS’ aren’t (so your performance will go through the roof in comparison) and they are also hierarchical and not relational which is more realistic to user data and allows you to ‘share’ the same data schemas across multiple applications without having to rebuild new databases etc. for each application.

Word of warning: over use of the DS can be as bad as not using it. It is not a replacement for an RDBMS so don’t start using it for shopping baskets and storing state etc.

Basically just think of DS's as an out-of-the-box user authentication and profile store which remove all the need to do code yourself (which goes by the rule "?The Easiest Code to Debug is That Which is Not Written").

Witek Baryluk said...

don't use MD5, use salted SHA-1. use /dev/urandom for salt.
And even more: use 8192 iterations of SHA256, with additionall counter and salt in each step. It is still fast to login if you know good password, but makes bruteforce attack much harder. There are ways even better like bcrypt or scrypt (amazing one). One can also use SRP protocol, which is even safer, but is harder to implement and needs some tricks on client side (it uses big integer modular arithmetic), but it works and is VERY seucre even over unencrypted channels.

Howard said...

I think its your comfort level. I don't think anyone should be comfortable with passwords in plaintext, or with a simple two-way cipher (i.e., something like rot13). But if you are worried about real infiltration, not casual, yes there are a lot of options that are more secure.