JavaScript-based ESAPI: An In-Depth Overview

Read this to make your opinion:

https://www.owasp.org/images/0/0b/ESAPI4JS-Marcus.Niemietz.pdf

Fast and correct htmlEncoding for JavaScript

Feel free to adjust two last matches in the regexp to any of your needs. I encode all > 0x07 and from allowed ascii, I encode only big 5. Using regexp is still faster (specially on MSIE) than any other impl. var _encodeHtmlRegExpImpl = (function() { // performance is 78ms on MSIE 7 (the slowest one) // on 80KB html markup from: http://www.w3.org/TR/html4/ var re = new RegExp( // surrogate pair (sp) "([\uD800-\uDBFF][\uDC00-\uDFFF])" + // html UNUSED including standalone surogates (un) "|([\u0000-\u0008\u000B\u000C\u000E-\u001F\u007F-\u009F\uD800-\uDFFF])" + // out of ascii (oa) "|([^\u0000-\u007F])" + // big 5 + add others (b5) "|([\u0022\u0026\u0027\u003C\u003E])", "g" ), toCodePoint = function(high, low) { return ((high - 0xD800) << 10) + (low - 0xDC00) + 0x010000; }, enc = function(m, sp, un, oa, b5) { // extracted out from main function and ifs changed to ternary // thanx to Andrea Giammarchi return "&#" + (oa || b5 ? m.charCodeAt(0) : (un ? "xFFFD" : toCodePoint(m.charCodeAt(0), m.charCodeAt(1)))) + ";"; }; return function(s) { return s.replace(re, enc); } } ());

org/owasp/esapi/codecs/HTMLEntityCodec.java (is it really correct ?)

org/owasp/esapi/codecs/HTMLEntityCodec.java

After reply from jwilliams, I have almost apologized for arogancy...
but...I gues jwilliams is the same person, that coded this:
org/owasp/esapi/codecs/HTMLEntityCodec.java
@author Jeff Williams

I have spent some more time to tune my implementation,
and to figure out what the OWASps HTMLEntityCodec does,
since the jwilliams comment did not match what I have saw in his code:

I have tested against ESAPI-2.0-rc6
the primitive code

return ESAPI.encoder().encodeForHTML(in);


This is what gets encoded (all green are "encoded somehow")


and please see HTML source code
for what is produced in markup !

So basically what you have posted as comment
is SOMETHING ELSE as YOUR code does:
Your post:

1) Encoding characters > 255 isn't useful, barring games with the character set.
2) There is no security problem with rendering named entities, although ESAPI uses hex entities to help performance.
3) Nobody is immune to charset switching
4) It's dangerous to remove characters entirely, you should replace with u+FFFD

What I think:
1) but you ARE encoding > 255, and incorrectly.
You ARE using ALSO, NAMED entities, not hex ! (and for huge ranges),
you print out standalone surrogates as hex which is error and violation of SGML def of HTML,
and correct surrage pairs are encoded as two hex codes instead of one hex int
2) esapi uses NAMED as well for wide range of chars !
3) no comment yet waiting for explanation
4) You are NOT using u+FFFD but whitespace " "

So please if I'm wrong "again", correct me but I do not want to waste ANY more time with
fixing external api,
I will stick to my fixed code,
and I warn the others to make their own versions as well (or use better code than OWASPs RI).

Please update DOCS to make things clear, if this is intended encoding and
OWASP considers this as safe, I will skip the libs just by reading docs, and
save some time on communication and testings.

Final Note ?

I hope this can open eyes a bit:


public static void main(String[] args) {
EscapeUtils2 eu=new EscapeUtils2();
String s1="<>abc123+-";
String s2=new String(new int[]{0xdc00,65823,65839,65855},0,4);
out.println(eu.escapeHtmlFull(s1+s2));
out.println(ESAPI.encoder().encodeForHTML(s1+s2));
}

a.in.the.k:
&#60;&#62;abc123+-&#xFFFD;&#65823;&#65839;&#65855;

with ESAPI gou get:
&lt;&gt;abc123&#x2b;-&#xdc00;&#xd800;&#xdd1f;&#xd800;&#xdd2f;&#xd800;&#xdd3f;

extra encoded + sign
&#xdc00 entity outputed (should not apear in HTML !!! by specification)
and 3 more &#xd800 entities (should not apear in HTML !!! by specification),
which happens because SMP should be encode(codePoint) not encode(char)+encode(char).


+ with my code you get all strange data situations logged:
27.9.2010 13:35:43 EscapeUtils2 log
WARNING: UNUSED DESCSET: codePoint=56320 at index:10
27.9.2010 13:35:43 EscapeUtils2 log
WARNING: SMP: codePoint=65823 at index:11
27.9.2010 13:35:43 EscapeUtils2 log
WARNING: SMP: codePoint=65839 at index:13
27.9.2010 13:35:43 EscapeUtils2 log
WARNING: SMP: codePoint=65855 at index:15



I do not expect that my data will contain SMPs or unpaired surrogates. But at least if they do, I produce correct markup.

org.owasp.esapi.encodeForURL (shocked again)

My quick look at OWASP ESAPI-2.0-rc6 again

java.lang.String encodeForURL(java.lang.String input) throws EncodingException
Encode for use in a URL. This method performs URL encoding on the entire string.

For the docs the "URL encoding" is defined by referencing wikipedia !
http://en.wikipedia.org/wiki/Percent-encoding
Don't we have RFCs for this ?

Being curious what it really does ? Look in the code:

return java.net.URLEncoder.encode(input,
ESAPI.securityConfiguration().getCharacterEncoding());


What ? Read JavaDoc !

Translates a string into application/x-www-form-urlencoded format.

This is not ment for building URLs but for encoding form data !
There is another JavaDoc:

The URLEncoder and URLDecoder classes can also be used,
but only for HTML form encoding,
which is not the same as the encoding scheme defined in RFC2396.

And another one:

The recommended way to manage the encoding and decoding of URLs is to use URI

Reading URI docs you will learn about all deviations Java has from RFC2396.
javase/6/docs/api/java/net/URI.html

The OWASP JavaScript version of "the same" is even "better"
(I bet a beer, not producing the same results as Java code):


encodeForURL: function(sInput) {
return !sInput ? null : escape(sInput);
}


Reading MDC docs:

escape and unescape Functions
The escape and unescape functions do not work properly for non-ASCII characters
and have been deprecated. In JavaScript 1.5

and later, use encodeURI, decodeURI, encodeURIComponent, and decodeURIComponent.

Bad naming or ignorance ?

There is just small chance that authors realy ment to code
"HTML form encoding" and not to solve URI building and encoding,
and that the method has just a bad name. I would suggest Encoder.encodeForHtmlForm
instead of misleading encodeForURL with even more confisung wiki link !


In the case OWASP really ment to solve
encoding for URI or http scheme URLs, there should be totaly
another code behind !!!!


If you really plan to encode URI components there is API needed to
encode path, path-segment, query, fragment with separate rules defined by
RFC (and I vote for the "new rfc3986" instead of buggy java implementation of old "RFC2396").

If you code or find rfc3986 compliant java uri implementation,
let me know,
until then I will not
replace my code for UNRELIABLE OWASP REFERENCE IMPLEMENTATION.


Strong suggestion again: search for "Jena IRI".

Can this be any worse ?

Can this be any worse ?


if (!Array.prototype.containsKey) {
Array.prototype.containsKey = function(srch) {
for ( var key in this ) {
if ( key.toLowerCase() == srch.toLowerCase() ) {
return true;
}
}
return false;
};
}
................
var getNamedEntity = function(input) {
var entity = '';
while (input.hasNext()) {
var c = input.peek();
if (c.match(/[A-Za-z]/)) {
entity += c;
input.next();
if (entityToCharacterMap.containsKey('&' + entity)) {
if (input.peek(';')) input.next();
break;
}
} else if (c == ';') {
input.next();
} else {
break;
}
}

return String.fromCharCode(entityToCharacterMap.getCaseInsensitive('&' + entity));
};
............
var entityToCharacterMap = [];
entityToCharacterMap["""] = "34"; /* 34 : quotation mark */
entityToCharacterMap["&"] = "38"; /* 38 : ampersand */
entityToCharacterMap["<"] = "60"; /* 60 : less-than sign */
entityToCharacterMap[">"] = "62"; /* 62 : greater-than sign */



Let's make contest ;-)
How many "bad practices" (cannot find other polite word)
can you "spot" in this code ?