Re: Manually computing sha1 digest of reference containing base64 encoded string and comparing it to digest value of same reference generated by SignedXML.ComputeSignature - Does not match

Hi,
welcome to the "wonderful" world of XML security ;).

You problem is that XML security specs are full of ambiguities that
often lead to incompatible interpretation and incompatible
implementations. One of the major pitas is that XML signature requires
XML explicit canonical form, which will be signed in resulting
document. XML Base64 binary data serialization doesn't require line
breaks and doesn't prohibit them, so from serialization point of view,
whether or not you insert line breaks in BASE 64 serialization - it
doesn't matter for XML data. However, signature is calculated on
serialized XML and is dependent on whether or not you insert line
breaks in signed elements. XML canonical serialization spec is
ambiguous in this regard. From one side it requires to preserve all
white spaces, significant or not, from other side it requires removing
of insignificant whitespaces from XPATH selection. Signature operates
with result of XPATH, so by taking XPATH removing of insignificant
whitespaces you may interpret it as if you need to remove them (what
is done in Microsoft implementation of XML digital signature). But
from other side, XML signature spec says that signature must be
applied to section of serialized xml document, which may be
interpreted as if you should preserve insignificant whitespaces before
signing (what is done in Apache XML security). WSE implementation uses
workaround and checks signature on two versions of signed elements:
one without insignificant whitespaces (canonical serialization of
XPATH selection) and another with all insignificant whitespaces
preserved (unmodified section of decrypted content). When WSE signs
XML it always remove insignificant whitespaces, but preserves all
significant whitespaces, which is OK with any implementation, because
deserialization may never add whitespaces back.
Now, when you serialize base64 encoded binary data, all whitespaces in
base 64 serialization are significant! So, whitespace inside base 64
encoded data should not affect validity of signature, because these
whitespaces must be always preserved. But this is also easy to
misunderstand because base 64 bit XML serialization doesn't specify
addition of whitespaces...
To avoid signature problem with encrypted data, use Sign-Then-Encrypt
order of operations. In this case, hash of encrypted data is
calculated on source data that is independent from base 64
serializations of cipher data. But in this case, encrypted data must
be serialized with explicit canonical form before you calculate hash
of it for signature and you must encrypt exactly the same data that
you calculate hash of (serialized canonical XML). Different
implementations do different things with this part as well. Some
implementations preserves insignificant whitespaces on source data for
encrypted block (Apache XML Security), others remove this
insignificant whitespaces, because this is always resulting from XPATH
selection (Microsoft XML signature). Microsoft implementation of
explicit canonical XML always removes insignificant whitespaces, even
if you do that on whole document! And many consider it a bug, but
others arguing that it's not. This usually never provides any problem
when you send out signed and encrypted XML (no implementation adds new
insignificant whitespaces), but if you are receiving XML from
somebody, you have no choice, but do it as it is done with WSE - try
validate two version of XML from decrypted data - one is unmodified
XML from what you receive, and another is canonically serialized XML.
if any of these signatures matches - you get your validation. This
solution sucks, and any seasoned cryptographer will immediately say
that this is a new protocol that doesn't have any assessment of
security... and that the results of established security protocols
could not be applied here because of significant modification of the
protocol... but from practical point of view - it may be secure... (if
you ignore the fact that there are actually ways of modifying xml
serialization that will not be detected by signature validation).

In other words - if you are going to do XML security with .Net - use
WSE... or be prepared to do a lot of crazy wiggling of code to
workaround implementation details...

-Valery

.

Quantcast