org.owasp.esapi.encodeForURL (shocked again)

My quick look at OWASP ESAPI-2.0-rc6 again

java.lang.String encodeForURL(java.lang.String input) throws EncodingException
Encode for use in a URL. This method performs URL encoding on the entire string.

For the docs the "URL encoding" is defined by referencing wikipedia !
http://en.wikipedia.org/wiki/Percent-encoding
Don't we have RFCs for this ?

Being curious what it really does ? Look in the code:

return java.net.URLEncoder.encode(input,
ESAPI.securityConfiguration().getCharacterEncoding());


What ? Read JavaDoc !

Translates a string into application/x-www-form-urlencoded format.

This is not ment for building URLs but for encoding form data !
There is another JavaDoc:

The URLEncoder and URLDecoder classes can also be used,
but only for HTML form encoding,
which is not the same as the encoding scheme defined in RFC2396.

And another one:

The recommended way to manage the encoding and decoding of URLs is to use URI

Reading URI docs you will learn about all deviations Java has from RFC2396.
javase/6/docs/api/java/net/URI.html

The OWASP JavaScript version of "the same" is even "better"
(I bet a beer, not producing the same results as Java code):


encodeForURL: function(sInput) {
return !sInput ? null : escape(sInput);
}


Reading MDC docs:

escape and unescape Functions
The escape and unescape functions do not work properly for non-ASCII characters
and have been deprecated. In JavaScript 1.5

and later, use encodeURI, decodeURI, encodeURIComponent, and decodeURIComponent.

Bad naming or ignorance ?

There is just small chance that authors realy ment to code
"HTML form encoding" and not to solve URI building and encoding,
and that the method has just a bad name. I would suggest Encoder.encodeForHtmlForm
instead of misleading encodeForURL with even more confisung wiki link !


In the case OWASP really ment to solve
encoding for URI or http scheme URLs, there should be totaly
another code behind !!!!


If you really plan to encode URI components there is API needed to
encode path, path-segment, query, fragment with separate rules defined by
RFC (and I vote for the "new rfc3986" instead of buggy java implementation of old "RFC2396").

If you code or find rfc3986 compliant java uri implementation,
let me know,
until then I will not
replace my code for UNRELIABLE OWASP REFERENCE IMPLEMENTATION.


Strong suggestion again: search for "Jena IRI".

request.getRequestDispatcher vs servletContext.getRequestDispatcher

ServletRequest.getRequestDispatcher(String path)

The pathname specified may be relative,
although it cannot extend outside the current servlet context.
If the path begins with a "/" it is interpreted as relative to the current context root.
...
If it is relative, it must be relative against the current servlet

ServletContext.getRequestDispatcher(String path)

The pathname must begin with a "/" and
is interpreted as relative to the current context root.
Use getContext to obtain a RequestDispatcher
for resources in foreign contexts.

Basic difference is that

request.getRequestDispatcher allows for relative paths.
Relative to what ?

relative path as coincidence

If the servlet (let's call him B) is accessible using only one URI (/ctx/B)
that seems to be easy answer.
Event if mapped with with url-pattern /path/B,
most containers are "trailing slash ignorant" and your servlet will be accessible with
both /ctx/path/B and /ctx/path/B/ variants.
With the same code in servlet:
request.getRequestDispatcher("../c.jspx").include(request, response);
you will get two different results of course,
/ctx/c.jsp and /ctx/path/c.jspx
one of them will end up with null pointer of course.


Usually using realtive path is coincidence and not design decision.

relative path as design

But wait "relative path" can be used as intentional design:

Imagine that the same servlet is accessible using

/a/a/a
/a/b
/f/g/h/ mappings

and based on the requested uri you want to include different view (c.jspx).
If the view exists in the relative location use it, otherwise use predefined global view.


// get view on current level
d=request.getRequestDispatcher("./c.jspx");
if(d==null)
{
d=request.getRequestDispatcher("/c.jspx");
}
d.include();


Reative Path - Double dispatch

what if b is already dispathed.
If the call goes througn A to B and the to C.
What "it must be relative against the current servlet" means ?
(test your self on several containers, reading specs is not enough for this case ;-))

My suggestion

PREFERE:
getServletContext().getRequestDispatcher() and /contextRelativePath

OVER:
request.getRequestDispatcher and relativePath

Otherwise you code may
not be movable up and down in web hierarchy withou moving views,
not work when mapped to different level URIs" without duplicating views,
and probably will suffer from container's trailing slash ignoracy.

Use realtivePaths only "as design not as coincidence",
and always document relative path dependencies in your servlet documentation.

HTTP Trace method and Query

Should the first line of trace echo the query string or not ?
Or is it up to container (web server ?)

WAS, TOMCAT, WLS all return trace without the query string
so if you browse

http://localhost/path?query

server responds with:

TRACE http://localhost/path HTTP/1.1

But I have seen containers returning

TRACE http://localhost/path?query HTTP/1.1


Any ideas why someone does not want to echo query ?

.href or setAttribute("href") in MSIE "sometimes" overrides the link text

Call to .href or setAttribute("href") in MSIE "sometimes" overrides the link text

If the text link contains ..@.. (hard to say exactly).
However MSIE DOES NOT override
the text if A tag contains other elements.

jQuery.attr does not solve the problem (or I use wrong API ?)

Proposed detection:

var supports = new (function() {
var a = document.createElement("a");
t = a.innerHTML = "a@a";
a.href = "http://msie";
this.aTextHrefOverride = (a.innerHTML != t);
});


Proposed solution:

var setAttributeHref = !supports.aTextHrefOverride ? function(that, value) { that.setAttribute("href", value); } : function(that, value) {
var t1 = getTextContent(that), t2;
that.setAttribute("href", value);
if ((t2 = getTextContent(that)) != t1) {
// asrt(has only text child no element childs);
setTextContent(that, t1);
}
}

getTextContent, setTextContent are XB helpers over .text and .textContent
properties.


TODo: lets check the dom-deviations

issues.apache.org

I cannot resist, sorry:
My issues and comments on issues.apache.org/bugzilla

Expression Language, Conditional Operator , WebLogic Tomcat incompatibility problem

Web logic server requires extra whitespaces in this situation:

<td>${fn:escapeXml(r.queryString==null?'null':r.queryString)}</td>


Apache Tomcat/6.0.28 - works fine
WebLogic Server 10.3.3.0 fails with


weblogic.servlet.jsp.CompilationException: Failed to compile JSP /RequestUrl.jspx
RequestUrl.jspx:44:50: Syntax error in expression. Encountered ":r". Expected one of ...


WLS requires extra space after :

<td>${fn:escapeXml(r.queryString==null?'null': r.queryString)}</td>

//TODO: check grammar in specs .... please
//JSP.2.3.8 Conditional Operator - A ? B : C
//and find out who is right or wrong.

Raw reflection vs java.beans.Expression

I have been just curious if java.beans.Expression and java.beans.Statement
are just convenience APIs over raw reflection APIs
or if those classes really bring up some "performance" or other boost.

First testcase:

java beans API:

String s1=(String)(new Expression(b1,"method1",args)).getValue()


raw reflection:

String s2=(String)b1.getClass().getMethod("method1",new Class[]{Date.class}).invoke(b1,args);


"Cached reflected method":

Method m=b1.getClass().getMethod("method1",new Class[]{Date.class});
for(..)
String s3=(String)m.invoke(b1,args);


Results:

360 [100000,loops,ms]
238 [100000,loops,ms]
105 [100000,loops,ms]

java.beans api seems to be the slowest.
Even if Expression is designed with statefull API,
which provokes "the intend to reuse same Expression object"
and change only arguments for example and keep target and method the same,
the Statement.invoke() code
seems to do all reflection work again and again
without any optimization and many extra ifs.

Just Another useless testcase for
standardized and useless convenience API ?

bugs.eclipse.org

Here is the
list of bugs reported by a.in.the.k to bugs.eclipse.org

"RowSet Wrapper List Strategy" and reality

"RowSet Wrapper List Strategy" and reality....

From Core J2EE Patterns: Best Practices and Design Strategies.

While the Transfer Object Collection strategy is one way to implement your finder methods,it might prove expensive if the query returns a large set of results and you end up creating a large collection of transfer objects, which are used sparingly by the client. Clients typically search and use the first few result items and discard the rest. When an application is executing a query that returns a large set of results, the RowSet Wrapper List strategy might be more efficient though it requires additional work and adds more complexity.

Code presented in the book is focused on RowSet wrapper and the code has 61+73 lines just as sample, and is
suitable only for RowSets. Get the book, read the samples, it is nice reading. However:

Using generics and AbstractList we can rewrite the original to 20lines long
"General Wrapper List" implementation, capable to transform any List<O> into List<E> by supplied convert method.


import java.util.AbstractList;
import java.util.List;

public abstract class WrappedList<O, E> extends AbstractList<E> {
public abstract E convert(O original);
private List<? extends O> original;
public WrappedList(List<? extends O> original) {
this.original = original;
// TODO: defensive copy ?, performance will suffer,
// and here we code for performance
// this.original=new ArrayList<O>(original);
}
public E get(int index) {
// TODO: caching ?
return this.convert(original.get(index));
}
public int size() {
return original.size();
}
}


This code has the same "quality" as the one presented in the book, and creates:
unmodifiable list (unexpected that view modifies list) which is
indirectly mutable (for sake of performance we do not make defensive copy of supplied original list),
that produce not equals (!=) instances with subsequent get(index) for the same index.

Using this to produce RowSet Wrapperfrom book, requires subclasing (anonymous inner class)
and this extra code (from book):

21 public Object get(int index) {
22 try {
23 rowSet.absolute(index);
24 } catch (SQLException anException) {
25 // handle exception
26 }
27 // create a new transfer object and return
28 return
29 TORowMapper.createCustomerTO(this);
30 }

moved to overrided convert method.

This way we have implemented optimized collection suitable to be used inside
c:forEach or jsf:table tags supporting start and end attribues.

Imagine situation where you have 1000 records from model and need to view only 10 of them.


<c:forEach items="${list}" var="l" begin="0" end="9">....


If list is constructed with the usuall "Value Object Collection" (copy and transform all)
it caouses 1000 calls to convert instances
of Value Object of type O into Value Objects of type E,
and creates 1000 of new instances of E.
In out case only 10 calls will be made and 10 new objects will be created.

Surprise:

<c:forEach items="${list}" var="l" begin="10" end="19">....


I would expect 10 again. But wrong it is 20 !.

<c:forEach items="${list}" var="l" begin="990" end="999">....

Will be 1000.

Thanx for clever code inside javax.servlet.jsp.jstl.core.LoopTagSupport.

Another idea of "too early full copy" arrays can be found here:
org/apache/taglibs/standard/tag/common/core/ForEachSupport.

Conclusions:

Even if you try to design with performance in mind,
even if you try to design by patterns and strategies,
choice of other libraries can eliminate
or even negate your efford..

Maybe, we will fix this soon, stay tuned.

Beware, my lovely specs

SRV.2.3.2 Initialization
After the servlet object is instantiated, the container must initialize the servlet before
it can handle requests from clients. Initialization is provided so that a servlet can
read persistent configuration data, initialize costly resources (such as JDBC API based
connections), and perform other one-time activities.

SRV.2.3.3.1 Multithreading Issues
A servlet container may send concurrent requests through the service method of
the servlet. To handle the requests, the Servlet Developer must make adequate provisions
for concurrent processing with multiple threads in the service method.
Although it is not recommended, an alternative for the Developer is to implement
the SingleThreadModel interface

EE.4.2.3Transactions and Threads
In web components not implementing SingleThreadModel,
transactional resource objects
should not be stored in class instance fields,
and should be acquired and released within the same invocation of the service method.

I do not understand what "initialize JDBC connection means" or the first suggestion is just excelent antisample ;-))

Quote of the day

Enhancing productivity without compromising architectural principles. Without adequate
infrastructure it is tempting to cut corners by adopting quick, hacky solutions that will cause
ongoing problems. Appropriate infrastructure should encourage and facilitate the application of
sound design principles.

Credits for this one go to Rod Johnson and his work:

This sentence can be used in many of my post from now ;-) Thanx for inspiration.

${}, fmt:message and fn:escapeXml

As expected ${exp} nor fmt:message
DO NOT PERFORM xml escaping.

99% of mine usage of both is to produce texts (text elements, or attribute values).
Only 1% of mine situations contain markup inside "exp" expression.

As EL values come from various and 99% enescaped sources (params,resources, db...)
I have to use c:out or fn:escapeXml over and over which enlarges the source code,
and creates unnecesary mess.

This is very sad, and I would like to propose new $$ operator for EL
which I would like to use as default,
(shall I implement this my self ? How ? When ? Where ?)
or I have missed some trivial trick ?

The hope for declarative tag based markup, disapears soon with my paranoid escaping of output:

<fmt:setBundle var="localizationContext" basename="tags" />
<c:set var="bundle" value="${localizationContext.resourceBundle}" />
.....

<fmt:message bundle="${localizationContext}" key="fileTable.lastModified" var="strLastModified"/>
<c:out value="${strLastModified}"/>

the last two lines hurt my eyes,
and polute pageScope with useless variable,
so I tend to rewrite it soon into:

${fn:escapeXml(bundle['fileTable.lastModified'])}

Apart from losing the declarative beauty, I have no clue about runtime consequences ;-) I have to read, thing and reverse engineer maybe a bit.

Please is anyone willing to educate me ?
Thanx.

Update:

c:out CAN HAVE BODY,
just stupid design,
since it MUST have value attribute.

Only when value is null, the body is processed.

<c:out value="${null}">
<fmt:message bundle="${localizationContext}" key="parametrized.markup">
<fmt:param value="${bundle['namespaced.markup']}"/>
<fmt:param value="${fn:escapeXml(bundle['namespaced.markup'])}"/>
<!-- probably wrong, double escaping -->
</fmt:message>
</c:out>

This can save us from exporting strLastModified.

But, how do you specify null in EL ?
After reading specs again I have found
NullLiteral ::= 'null'

Look at MS ideas here:

http://weblogs.asp.net/scottgu/archive/2010/04/06/new-lt-gt-syntax-for-html-encoding-output-in-asp-net-4-and-asp-net-mvc-2.aspx

Can this be any worse ?

Can this be any worse ?


if (!Array.prototype.containsKey) {
Array.prototype.containsKey = function(srch) {
for ( var key in this ) {
if ( key.toLowerCase() == srch.toLowerCase() ) {
return true;
}
}
return false;
};
}
................
var getNamedEntity = function(input) {
var entity = '';
while (input.hasNext()) {
var c = input.peek();
if (c.match(/[A-Za-z]/)) {
entity += c;
input.next();
if (entityToCharacterMap.containsKey('&' + entity)) {
if (input.peek(';')) input.next();
break;
}
} else if (c == ';') {
input.next();
} else {
break;
}
}

return String.fromCharCode(entityToCharacterMap.getCaseInsensitive('&' + entity));
};
............
var entityToCharacterMap = [];
entityToCharacterMap["""] = "34"; /* 34 : quotation mark */
entityToCharacterMap["&"] = "38"; /* 38 : ampersand */
entityToCharacterMap["<"] = "60"; /* 60 : less-than sign */
entityToCharacterMap[">"] = "62"; /* 62 : greater-than sign */



Let's make contest ;-)
How many "bad practices" (cannot find other polite word)
can you "spot" in this code ?

"Interesting project" - owasp-esapi-js

Just to keep in touch with some security topics, I have downloaded latest code of OWASP ESAPi for Java
and JavaScript today.
I'm bloging about the JavaScript part today.

This is the quote from original site (if you do not know OWASp and/or ESAPI:
http://code.google.com/p/owasp-esapi-js/

The purpose of the ESAPI is to provide a simple interface that provides all the security functions a developer is likely to need in a clear, consistent, and easy to use way. The ESAPI architecture is very simple, just a collection of classes that encapsulate the key security operations most applications need.

Just the first look is shocking:

the esapi.js
does not use closure to hide it's internal functions,
pulutes global space,
modifies Array and String prototypes,
$,
uses bad uncompressable techniques,
unefficient constructs,
and ...
is "unsecure" and "destructive".


Unsecure can be shown here:

if (!Array.prototype.each) {
Array.prototype.each = function(fIterator) {
if (typeof fIterator != 'function') {
throw 'Illegal Argument for Array.each';
}

for (var i = 0; i < this.length; i ++) {
fIterator(this[i]);
}
};
}


What is the purpose of the if here ? If already defined, use the defined function
override otherwise. Defined by who ?

Browser ? As far as I hnow, none JavaScript version supports Array.prototype.each
(there si forEach in JS 1.6)
Other library ? How trusted ?
Or injected poisoned version of XSSed script ?

This method and others constructed in the same style are then used in subsequent
"security APIs".

Destructive means:

var $type = function( oVar, oType ) {
if ( !oVar instanceof oType ) {
throw new SyntaxError();
}
};

In global scope of course.

Those two are ultimately candidates for criticism, specially in "security related library".

The rest of the code which is not big (3000 lines including spaces and comments),
shows quite inconsistent coding style,
and lack of professional Java Script knowledge.

So much for now, I will have deeper look later,
but I'm not impressed.

"Shame of OWASP label"

Eclipse and JSP Validation rtexprvalue

Sometimes the ment for RAD slows you down.
Specially if not implemented by specs:

Tag:
<jsp:directive.attribute name="title"/>

Usage:
<l:holy-grail-no-quirks-mode title="${param.title}">

Result:
"title" does not support runtime expressions holy-grail-no-quirks-mode.jspx

Read the docs (jsp-2_0-fr-spec.pdf, Table JSP.8-3 Details of attribute directive attributes):

rtexprvalue (optional) Whether the attribute’s value may be dynamically
calculated at runtime by a scriptlet expression.Unlike the
corresponding TLD element, this attribute defaults to true.

Fix: you have to type extra chars if you want this warning to disapear:
<jsp:directive.attribute name="title" rtexprvalue="true"/>

Already registered ;-(

https://bugs.eclipse.org/bugs/show_bug.cgi?id=302060

Pure JSP templating

Here is small proposal for templating using just basic
JSP techniques (no Struts Tiles, JSF nor Facelets needed).
It uses JSP Tag Files to implement the "template (tag)".

Layout is inspired by my favorite

http://matthewjamestaylor.com/blog/holy-grail-no-quirks-mode.htm

And war file with all 3 files *tag, sample and css) can be downloaded from my web site:

http://www.aitk.info/download/Layouts.1.0.war


Any comments and imrovements are welcomed.