How should I escape or cleanse user-provided passwords before I hash them and store them in my database?

When PHP developers consider hashing users' passwords for security purposes, they often tend to think of those passwords like they would any other user-provided data. This subject comes up often in PHP questions related to password storage; the developer often wants to cleanse the password using functions such as escape_string()(in various iterations), htmlspecialchars(), addslashes() and others before hashing it and storing it in the database.

upvote
  flag
u can user base64 encode – MSS
upvote
  flag
No @MSS, you shouldn't because base64 is encoding, not encrypting or hashing. Passwords should always be hashed. – Jay Blanchard
upvote
  flag
I mean before hash ;) – MSS
upvote
  flag
You shouldn’t and do not need to do that before hashing. It’ll cause you to have to write unnecessary additional code @MSS – Jay Blanchard

2 Answers 11

up vote 76 down vote accepted

You should never escape, trim or use any other cleansing mechanism on passwords you'll be hashing with PHP's password_hash() for a number of reasons, the single largest of which is because doing additional cleansing to the password requires unnecessary additional code.

You will argue (and you see it in every post where user data is accepted for use in your systems) that we should cleanse all user input and you would be right for every other piece of information we're accepting from our users. Passwords are different. Hashed passwords cannot offer any SQL injection threat because the string is turned into hash prior to storing in the database.

The act of hashing a password is the act of making the password safe to store in your database. The hash function doesn't give special meaning to any bytes, so no cleansing of its input is required for security reasons

If you follow the mantras of allowing users to use the passwords / phrases they desire and you don't limit passwords, allowing any length, any number of spaces and any special characters hashing will make the password/passphrase safe no matter what is contained within the password. As of right now the most common hash (the default), PASSWORD_BCRYPT, turns the password into a 60 character wide string containing a random salt along with the hashed password information and a cost (the algorithmic cost of creating the hash):

PASSWORD_BCRYPT is used to create new password hashes using the CRYPT_BLOWFISH algorithm. This will always result in a hash using the "$2y$" crypt format, which is always 60 characters wide.

The space requirements for storing the hash are subject to change as different hashing methods are added to the function, so it is always better to go larger on the column type for the stored hash, such as VARCHAR(255) or TEXT.

You could use a complete SQL query as your password and it would be hashed, making it unexecutable by the SQL engine e.g.,

SELECT * FROM `users`;

Could be hashed to $2y$10$1tOKcWUWBW5gBka04tGMO.BH7gs/qjAHZsC5wyG0zmI2C.KgaqU5G

Let's see how different sanitizing methods affect the password -

The password is I'm a "dessert topping" & a <floor wax>! (There are 5 spaces at the end of the password which are not displayed here.)

When we apply the following methods of trimming we get some wildy different results:

var_dump(trim($_POST['upassword']));
var_dump(htmlentities($_POST['upassword']));
var_dump(htmlspecialchars($_POST['upassword']));
var_dump(addslashes($_POST['upassword']));
var_dump(strip_tags($_POST['upassword']));

Results:

string(40) "I'm a "dessert topping" & a <floor wax>!" // spaces at the end are missing
string(65) "I'm a &quot;dessert topping&quot; &amp; a &lt;floor wax&gt;!     " // double quotes, ampersand and braces have been changed
string(65) "I'm a &quot;dessert topping&quot; &amp; a &lt;floor wax&gt;!     " // same here
string(48) "I\'m a \"dessert topping\" & a <floor wax>!     " // escape characters have been added
string(34) "I'm a "dessert topping" & a !     " // looks like we have something missing

What happens when we send these to password_hash()? They all get hashed, just as the query did above. The problem comes in when you try to verify the password. If we employ one or more of these methods we must re-employ them prior to comparing them with password_verify(). The following would fail:

password_verify($_POST['upassword'], $hashed_password); // where $hashed_password comes from a database query

You would have to run the posted password through the cleansing method you chose before using the result of that in password verification. It is an unnecessary set of steps and will make the hash no better.


Using a PHP version less than 5.5? You can use the password_hash() compatibility pack.

You really shouldn't use MD5 password hashes.

upvote
  flag
If you hash the password with the trailing spaces, and buddy does not include them on his next attempt to authenticate, does he still get access? – Dan Bracuk
10 upvote
  flag
No. If he created his password with trailing spaces, which is allowed, he must use them on login @DanBracuk – Jay Blanchard
upvote
  flag
So your recommendation has a major disadvantage with respect to usability. – Dan Bracuk
10 upvote
  flag
How so @DanBracuk? If we allow the user to setup the password s/he desires, including leading/trailing spaces? – Jay Blanchard
15 upvote
  flag
That's why most things require you to enter your chosen password twice. If user added the spaces on accident they will figure it out before getting any further. If user did it on purpose than it's a non-issue. – I wrestled a bear once.
1 upvote
  flag
I didn't really get the whole point of this question-answer thing. The rule of thumb is to sanitize what the business requires to and to escape everything going in the db. Period. You have to escape the hash, nobody know how things will change in the future. Regarding the password cleansing, that's an hot topic. Google for example trims the password, I personally don't like it. But its help center surely handle more angry users than I do. – Margaret Bloom
3 upvote
  flag
@MargaretBloom, a rule of thumb is just a heuristic. We sometimes still need to think things through, like for passwords. You say "nobody knows how things will change in the future", but it seems if anything is going to change it's the way we escape data before we put it into the database, in which cases users would find themselves locked out when their passwords no longer match what we've stored. What is the danger in not escaping password hashes vs. the danger of escaping them? – DavidS
upvote
  flag
And maybe things are different in the PHP world, but in the Java world the rule of thumb isn't to escape everything going in the db, it's to use prepared statements. So much for rules of thumb. – DavidS
2 upvote
  flag
Exactly: you will of course "escape the hash" in the limited sense of correctly passing it to a parameterized SQL query, where some code in your SQL connector may or may not do anything with it that corresponds to "escaping", you don't know and don't care. You just won't have to write any specific code to achieve that, because it's completely routine for all your SQL queries unless you've previously made some poor life decisions. – Steve Jessop
2 upvote
  flag
The PHP world is trying to get to the prepared statements rule of thumb, but older database API's are wide-spread in the wild and have promoted bad habits @DavidS – Jay Blanchard
upvote
  flag
What about filter_input() I personnaly filter the password input just to strip ASCII < 32. – Louis Loudog Trottier
upvote
  flag
Are you filtering when hashing and when verifying @LouisLoudogTrottier? Even if you are you shouldn't because the password is made safe by the act of hashing it. – Jay Blanchard
upvote
  flag
@JayBlanchard I filter the raw user input from a html form like so $password=filter_input(INPUT_POST,"password",FILTER_UNSAFE_R‌​AW,FILTER_FLAG_STRIP‌​_LOW) that "Strip characters with ASCII value less than 32". Then, it's processed by PHP (hash, verify) and passed to mysqli_statement. So ya the only filter i ever apply only strip ASCII les than 32 ei: NULL, [TAB], [New line], etc.. see asciitable.com – Louis Loudog Trottier
upvote
  flag
in order to prevent this => "Please note that password_hash will truncate the password at the first NULL-byte." => 3rd comment on php.net php.net/manual/en/function.password-hash.php – Louis Loudog Trottier
1 upvote
  flag
Good point @LouisLoudogTrottier, but how often would a NULL byte character be entered into a passphrase? It would seem rather low odds for someone specify an octal in their passphrase. – Jay Blanchard
upvote
  flag
i don't know either, maybe character encoding on a foreign language, miss configured machine, defect hardware or maybe just like the good old ALT-255 on windows. Or some od string like %00. Truly i don't know. I'm just paranoid (or OCD) that way. On the other hand, if anyone does used those low ascii, why not just strip them away to prevent the infinitly small chance that a NULL goes through. i appreciate the reply. – Louis Loudog Trottier
upvote
  flag
I am going to perform some testing and, if warranted, I will update the answer with this information @LouisLoudogTrottier – Jay Blanchard
upvote
  flag
I've made a fiddle a ran some test on your string and some other random string made with my fr-ca keyboard (±@£¢¤£¦¬¬¤³¦²½³½¾§¶[]}{~l­¯µ) and successfully password_verify() and password_hash() from a filter_input and it works. I don't know how or with what tools we can send null bytes so i didn't post my results. Seems like all other function failed comparing 'RAW' exvept filter_input(). I've dump the code on a sandbox => sandbox.onlinephpfunctions.com/code/… and can run it on phpfiddle. Geetings. – Louis Loudog Trottier
upvote
  flag
Problem occurs, and we have funny behavior (from a funny input) if we force $_POST by adding $_POST['upassword']="\01234567 "; bewteen line 2 and 3. Most of them return 'Good', trim fails (beacuse of the trailing spaces) and filter_input 'become' NULL. Maybe my way of testing is wrong or innacurate but seems like my test conclude that it is even worst to use filter input. Now we still need a way to strip NULL bytes from going to password_hash(). – Louis Loudog Trottier

Before hashing the password, you should normalise it as described in section 4 of RFC 7613. In particular:

  1. Additional Mapping Rule: Any instances of non-ASCII space MUST be mapped to ASCII space (U+0020); a non-ASCII space is any Unicode code point having a Unicode general category of "Zs" (with the exception of U+0020).

and:

  1. Normalization Rule: Unicode Normalization Form C (NFC) MUST be applied to all characters.

This attempts to ensure that if the user types the same password but using a different input method, the password should still be accepted.

upvote
  flag
What would be an example of "the same password but using a different input method"? – DavidS
3 upvote
  flag
@DavidS, A super shiny North american Mac Book (that Joe used just before leaving) and a poorly internationalized Taiwanese internet café computer (that Joe is trying to use to download is flight back boarding card). – Margaret Bloom
2 upvote
  flag
Sounds jingoistic. :-) Thanks though. – DavidS
3 upvote
  flag
Hmm. If you do this, then you should also validate passwords to reject any that contain as-yet-unassigned characters. It would be terrible if a user uses NEWFANGLED SPACE, which your app doesn't recognize and therefore hashes as-is, and then you upgrade your Unicode Character Database and suddenly NEWFANGLED SPACE gets mapped to SPACE before hashing, such that (s)he can no longer enter a password that your app will hash to the old hash. – ruakh
upvote
  flag
Curious - how would a non-Ascii space get entered into the password if the application is UTF-8 all the way through? – Jay Blanchard
upvote
  flag
@JayBlanchard Unicode has 17 different space characters. – Paul Crovella
upvote
  flag
Right @PaulCrovella, but if an applications and its database are setup to use UTF-8 all the way through why would you need any additional mapping? – Jay Blanchard
4 upvote
  flag
@JayBlanchard Because when you press a space bar on one machine and when you press it on another machine you might get two different Unicode code points, and they'll have two different UTF-8 encodings, without the user being aware of anything. It could be argued that this is a problem you wish to ignore, but RFC 7613 was borne out of such real-life issues, it's not a make-work recommendation. – Kuba Ober
1 upvote
  flag
@ruakh Once you decide on handling passwords in a certain way, they must remain handled that way, or else things will break for existing use cases. If you intend to change the preprocessing method in the future, you should store it along the preprocessed and hashed representation of the password. That way, once you receive the input, you select the preprocessing/hashing method based on what you're comparing to. – Kuba Ober
1 upvote
  flag
@JayBlanchard For the same reason you want to apply NFC - it's not always clear to a user that, for example, their phone's on-screen keyboard might be sending different codepoints that just happen to look the same as what they send via their laptop's keyboard. For a more visible example (and reason to apply NFC normalization) one might send ά as U+03AC while the other sends it as U+03B1 followed by U+0301. – Paul Crovella
upvote
  flag
Thanks, those are good points to consider @PaulCrovella. I want to do some testing based on this. – Jay Blanchard
1 upvote
  flag
@KubaOber: NFC is guaranteed to be stable for future versions of Unicode, as is the "general category" of each character, provided the input consists only of already-assigned characters. So legoscia's approach is fine, even if you continue to add support for new characters over time, as long as you reject as-yet-unassigned characters. – ruakh
upvote
  flag
By "reject" do you mean they should be stripped out @ruakh? – Jay Blanchard
upvote
  flag
@JayBlanchard: Certainly not! That would cause the same problem. By "reject" I mean that you should show an error message. – ruakh
upvote
  flag
So, if I am understanding correctly @ruakh, the methodology you would employ is to analyse the proposed password for non-assigned characters and if it contains any of those reject the password and deliver a message to the user stating they cannot use certain characters in their password (I would tell them the non-assigned characters found in their attempt.)? I will try to find resources on the unassigned characters and try to determine what kind of UX impact that might have. – Jay Blanchard

Not the answer you're looking for? Browse other questions tagged or ask your own question.