Ahhh ... it has been a while, hasnt it ?

My life is just torn between working with bits of 2, beats of 4 and nucleotides of 4. But while challenging, it has been really fun. As spoken to a friend today, my passions in life seeks out to expand the comfort boundaries of gray matter, which we called the mind and to constantly challenge and stimulate the brain to learn and absorb new things that one would never think of learning if one boxed themself in a virtual space, which techies like me would call "typecast".

One example that I highlighted to my friend today, which I respectfully pointed out to them that he falls under, is when he said: "But we tech people are not good at talking to people and engaging them in meaningful conversations ..."

Typecast alert !

I ended up talking with him (not to him) for a good 20 minutes and told him we just had a meaningful conversation and that he could hold one really well. I told him that he himself set up this virtual boundary to box himself in. No one did and that he could easily remove this barrier and elevate himself to do and more importantly, to learn new things and behaviors. Instead of having new curiousities about old things, have new questions, passions and interests towards new things.

Anyways, I wont be talking about my new-found passions here but I will be briefly touching on a topic that many people knew I have passions for (and I still do) - and that is the innards and the plumbings of software technologies.

I came across types of this type of questions a lot in emails, forum questions and usergroup events:

 I have this WSDL file that looks something like this:

<?xml version='1.0' encoding='UTF-8'?>

<definitions name="someCustomer" targetNamespace="urn:someCustomer" xmlns:typens="urn:someCustomer" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns="http://schemas.xmlsoap.org/wsdl/">
      <message name="add_someCustomer">
            <part name="resId" type="xsd:string"/>
            <part name="cPortable" type="xsd:string"/>
      </message>
      <message name="add_someCustomerResponse">
            <part name="add_someCustomerReturn" type="xsd:string"/>
      </message>
      <portType name="someCustomerPortType">
            <operation name="add_someCustomer">
                  <input message="typens:add_someCustomer"/>
                  <output message="typens:add_someCustomerResponse"/>
            </operation>
      </portType>
      <binding name="someCustomerBinding" type="typens:someCustomerPortType">
            <soap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/>
            <operation name="add_someCustomer">
                  <soap:operation soapAction="urn:someCustomerAction"/>
                  <input>
                        <soap:body namespace="urn:someCustomer" use="encoded" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>
                  </input>
                  <output>
                        <soap:body namespace="urn:someCustomer" use="encoded" encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"/>
                  </output>
            </operation>
      </binding>
      <service name="someCustomerService">
            <port name="someCustomerPort" binding="typens:someCustomerBinding">
                  <soap:address location="http://foo/bar/someCustomer.php"/>
            </port>
      </service>
</definitions>

However, I need to change the add_someCustomerReturn  type from xsd:string to a complex type.

I’ve made several tests variants around trying to add a complex type, like the following:

      <message name="add_someCustomerResponse">
            <xsd:complexType name="respType" >
                  <xsd:sequence>
                        <xsd:element name="someStatus" type="xsd:boolean" />
                        <xsd:element name="someResult" type="xsd:boolean" />
                  </xsd:sequence>
            </xsd:complexType>
            <part name="add_someCustomerReturn" type="typens:respType"/>
      </message>

However I always end up having an error like:

Custom tool error: Unable to import WebService/Schema. Unable to import binding 'customerBinding' from namespace 'urn:customer'. Unable to import operation 'add_customer'. The datatype 'urn:customer:respType' is missing.

One thing to note is the above "web service" is using: soap:binding style="rpc". While I am not advocating one over another (document/literal), I personally think that if you stripped the religious and philisophical debates, one can certainly build a RPC-style web service using doc/literal encoding.

The above exceptions funs afoul of what many techies called: Section 5 Encoding

For the above to be resolved, you need to define a complexType reference by wsdl:part “add_someCustomerReturn” in the schema.
To do this, you MUST define wsdl:types and add the schema to the WSDL that defines the complex and change the type=”xsd:string” (of the wsdl:part) to the identifying complexType in the schema (encoded in wsdl:types)

While this is an old article written by Tim, the same principles apply. Do check it out of you need to stimulate your brain: The Argument against SOAP Encoding

Finally, OASIS announced last week that it is calling for participation for Web Services Federation. The formation of the WS-Federation Technical Committee is announced here.

WS-Fed is an important addition to the WS-* protocol suite that enables users to sign-in seamlessly to systems outside of their own organization without requiring (more) new usernames and passwords using Single-Sign-On (SSO) between separate organizations with an established trust relationship.

WS-Fed builds upon and composes with other WS-* protocols:

• WS-Fed extends WS-Trust
• WS-Fed composes with WS-Security and WS-SecureConversation to ensure data integrity and privacy
• WS-Fed composes with WS-MetadataExchange and WS-Policy to enable simple provisioning and trust relationship configuration

Does WS-Fed compete with Liberty SAML?

• Both SAML and WS-Fed enable browser-based identity federation (Passive-Mode)
• However, WS-Fed enables a superset of scenarios, including:
  • Seamless federation with Web Services and/or Rick-Client applications
  • Separation of identities, token types, protocols and encodings
  • Multi-purpose Security Token Service (STS) that can return tokens stating different assertions based upon the scenario
    

WS-Fed adds identity federation capabilities to the existing WS-* suite of protocols resulting in:

• A single protocol stack that supports the majority of your needs and scenarios
• Simplified development, deployment, management and control

The formation of the Technical Committee to drive the standardization of the WS-Fed is an important step in evolution of the industry-wide effort to create a single, comprehensive communication protocol suite that enables many current and new scenarios most effectively.

Apache Axis2/C version 0.93 is released. This should please the REST camp and highlights the momentum of how REST can be implemented in Web Services. If you are in KL next week, I will be briefly touching base on the REST-vs-SOAP style of implementing Web Services.

Key Features

1. AXIOM, an XML object model optimized for SOAP 1.1/1.2 Messages.
   This has complete XML infoset support.
2. Support for one-way messaging (In-Only) and request response
   messaging (In-Out)
3. Description hierarchy (configuration, service groups, services,
   operations and messages)
4. Directory based deployment model
5. Archive based deployment model
6. Context hierarchy (corresponding contexts to map to each level of
   description hierarchy)
7. Raw XML message receiver
8. Module architecture, mechanism to extend the SOAP processing model
9. Module version support
10. Transports supports: HTTP\
    1. Both simple axis server and Apache2 httpd module for server side
    2. Client transport with ability to enable SSL support
11. Service client and operation client APIs
12. REST support (HTTP POST case)
13. WS-Addressing, both the submission (2004/08) and final (2005/08) versions
14. MTOM/XOP support
15. Code generation tool for stub and skeleton generation for a given
    WSDL (based on Java tool)
    1. Axis Data Binding (ADB) support
16. Security module with UsernameToken support
17. REST support (HTTP GET case) - New
18. Dynamic invocation support (based on XML schema and WSDL
    implementations) - New

Major Changes Since Last Release

1. REST support for HTTP GET case
2. XML Schema implementation
3. Woden/C implementation that supports both WSDL 1.1 and WSDL 2.0
4. Dynamic client invocation (given a WSDL, consume services dynamically)
5. Numerous improvements to API and API documentation
6. Many bug fixes, especially, many paths of execution previously untouched were tested along with Sandesha2/C implementation

Download the above release here.

During my webcast on "Why we need Reliablility in SOAP: Web Services", there were a couple of hiccups which hindered a better listening experience.

1. I cannot see the animation on the slides I am presenting, even though I am assured by the producer that the floor is seeing it. Therefore, I am "guessing" what the audience is actually seeing in my click-animation and gauging my content from there. It was neither easy nor pleasant.
2. There was a disconnect incident in my demos that also marred the listener's experience. I had to re-login again. Not Good.

Isnt it ironic? My network connection showed lack of reliability when I am talking about Reliability as a topic. . Now the least I can do is to answer a couple of questions that popped up after the session:

Q: Is RM available for all the bindings in Windows Communication Foundation (WCF, previously - Indigo) ? 
A: Yes, it is available for MOST of the standard bindings in WCF. In some bindings such as the netTcpBinding I showed, it is On-by-Default. In bindings such as wsDualHttpBinding where you need correlation of different channels and such, it is Always-On. It doesnt make sense to stick <reliableSession /> in a netMsmqBinding, for example.

Q: Is this the same WS-RM spec that is authored by IBM, Microsoft and TIBCO ? 
A: Yes. In my slide, I mentioned - I.B.M and TIBCO. I.B.M is actually the acronym I used for IBM, BEA and Microsoft.

Q: Can I get the demo you showed? 
A: No, I am sorry. In any case, my demos will not work with the lastest WinFX B2 bits today. I will need time to port them over. I recommend you go bug Shy when you see him and ask him for his WS-RM demo which consists of a WPF stack in there and a "awesomely" cool Rubik's Cube demo and is 100x better than mine.

All in all, it is quite a different experience than doing an on-stage presentation, especially when you spent an hour talking to yourself and you cannot see the audience faces and cannot manipulate your content and presentation based on their moods.

But then again, no one can see that I am wearing my Mickey-Mouse boxers while I am presenting, so I guess that is a good trade-off.

Here I am - proud to announce that I will be doing a MSDN Redmond-hosted Webcast right from the other side of the hemisphere in Singapore.

I will be speaking on concepts of Reliability in Soap:Web Services, why its needed, as well as the context of it in Windows Communication Foundation (WCF, previously - Indigo).

More importantly, a 40GB Creative (another homegrown Singaporean product) ZEN MP3 player is at stake here waiting to be won. So, do sign up quickly for a chance to win this. Rules here.

If you are one of those insomniacs in Asia-Pacific, do try to tune-in. I hope this blazes a trail for the other community leaders in Asia-Pacific to follow suit and show that we are right on par there with the best in technology.

Click here for more details on this webcast.

Apache Axis2/C version 0.91 has been released via here:
http://ws.apache.org/axis2/c/download.cgi

Key Features

1. AXIOM, an XML object model optimized for SOAP 1.1/1.2 messages with complete XML Infoset support
2. Support for One-Way Messaging (In-Only) and Request Response Messaging (In-Out)
3. Module architecture, mechanism to extend the SOAP processing model
4. Context hierarchy
5. Directory based deployment model
6. Raw XML providers
7. WS-Addressing, both the submission (2004/08) and final (2005/08) versions
8. Transports: HTTP * Both simple axis server and Apache2 httpd module and * SSL client transport - New
9. Service Groups - New
10. Service client and operation client APIs - New
11. REST support (POST case) - New
12. Module version support - New
13. MTOM support - New

Other notes

1. Interoperability tested with Axis2/Java for XML in/out client and services
2. Addressing 1.0 interoperability

Major changes since last release

1. Full Addressing 1.0 support
2. Improved fault handling model
3. SSL client transport
4. MTOM implementation
5. Implementation of easy to use service client and operation client APIs for client side programming
6. REST support (POST case)
7. Module version support
8. Service groups
9. Numerous bug fixes since last release

Un-Implemented Architecture Features (TBD in 1.0)

1. Sessions scoping for application, SOAP, transport and request levels
2. Different character encoding support
3. Dynamic invocation
4. Archive based deployment Model

Un-Implemented Architecture Features (TBD post 1.0)

1. WSDL code generation tool for stub and skeletons (based on Java tool)
2. Security module
3. REST (REpresentational State Transfer) support (GET case)
4. Web Services policy support
5. Axis2 Web application (Web App)

Fellow Microsoft Regional Director and well-known distributed systems expert Matevz Gacnik has a great blog to add on top of mine called: Windows Workflow Foundation: Exposing Workflows as Services. Trust me, Matevz is a lot more than Request-Response or what his blog suggested .

In it, he explains some of the ways you can do so and the pitfalls to watch out for. Of course, you can get around the "workflow runtime can only get loaded once per appdomain" issues by having it static to the service implementation class OR if you want only one instance of each - Windows Communication Foundation (WCF, previously - Indigo) also gives you a singleton-like instancing mode as well - InstanceContextMode:=InstanceContextMode.Single.

(I would be interested to find out the naming convention to call it Single in the latest CTP instead of Singleton)

My tip highlighted here is not really about exposing workflows as services. It is more about how you can hook a workflow into a already-hosted WCF service as part of its configured behavior, if need be. For example, you may want to have a non-intrusive workflow for you to raise an event that calls into your defined HandleExternalMethod (called EventSink before) and then you may just terminate that activity. The reason is because the workflow thread doesnt return unless you called a WaitHandle.Set, which you can call when a workflow is completed or terminated.

Having said all that - Remember that most of the current implementations of Web Services today work on a Request-Response model and many more are betting on that it will remain like that for a long time. Isnt this one of the reasons we have long arguments of POX/REST ? Workflows, on the other hand, are made to handle long running work and therefore, you need to design and handle both properly as its design principles and most-used implementations do conflict.

Now, if you forsake the Request-Response model and think about the wonderful partnership and the options abound once you hit <OperationContract(IsOneWay:=True)> on top of the MSMQ transport, Windows Workflow Foundation (WF) + Windows Communication Foundation (WCF, previously - Indigo) does look very delicious and promising indeed.

Now if XML-RPC aint enough (I had blogged about this here and here), now we can add REST-RPC into the mix. The main difference would be the use of HTTP to provide application semantics via its verbs. This would mean that there would hardly be any XML or Request payload of any kind.

What a way to start off a new category on my blog called "Interoperability".
I recently came across an interesting post in the forums that goes something like this:

[QUOTE]

  [Signature xmlns="http://www.w3.org/2000/09/xmldsig#"]
    [SignedInfo]
      [CanonicalizationMethod
Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315" /]
............
  [/Signature]

But I need the signature to be in a namespace that should be identified by
the dsig prefix:

[dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#"]
[dsig:SignedInfo]
.....
[/dsig:Signature]

I really didnt think anything of this. At first glance, I thought the problem lies not in the code BUT the processor / validator that was used to read this.

The dsig or any prefix, for that matter, doesn't indicate whether they 2 use different namespaces. Check
the [default] namespaces and compare.

Strictly speaking -

• [ds:Signature xmlns:ds="urn:softwaremaker.net-foo.bar" /]
• [dsig:Signature xmlns:dsig="urn:softwaremaker.net-foo.bar" /]
• [Signature xmlns="urn:softwaremaker.net-foo.bar /]

are isomorphically the same. If the end processor / validator reads it and treats differently, I believe that it should be a design flaw at the other end as it is really poor design to rely on namespace prefix.

If you look at the XML-Digital Signature Specifications, Section 1.3 states that:

This namespace is also used as the prefix for algorithm identifiers used by this specification. While applications MUST support XML and XML namespaces, the use of internal entities [XML] or our "dsig" XML namespace prefix and defaulting/scoping conventions are OPTIONAL; we use these facilities to provide compact and readable examples.

Therefore, it is NOT necessarily to have a prefix to it as long as it points to the same namespace.

However, I spoke too fast. Further explanations by the other party has made me put my thinking cap on. He provided 2 reasons being:

1. Compatibility with our existing signer.
2. We are planning to extend the signature to XML Advanced Electronic Signatures (XAdES) format. In that case the prefix is mandatory.

I am surprised [which kinda shows how much I know, or dont know ???]. I spent some minutes digging into the XML Advanced Electronic Signatures (XAdES) specifications and true enough, it declares:

XMLDSIG | <ds:Signature ID?>- - - - - - - - -+- - - - -+ <ds:SignedInfo> | | <ds:CanonicalizationMethod/> | | <ds:SignatureMethod/> | | (<ds:Reference URI? > | | (<ds:Transforms>)? | | <ds:DigestMethod> | | <ds:DigestValue> | | </ds:Reference>)+ | | </ds:SignedInfo> | | <ds:SignatureValue> | | (<ds:KeyInfo>)?- - - - - - - - - + | | <ds:Object> | | <QualifyingProperties> | | <SignedProperties> | | <SignedSignatureProperties> | (SigningTime) | (SigningCertificate) | (SignaturePolicyIdentifier) | (SignatureProductionPlace)? | (SignerRole)? | </SignedSignatureProperties> | | <SignedDataObjectProperties> | (DataObjectFormat)* | (CommitmentTypeIndication)* | (AllDataObjectsTimeStamp)* | (IndividualDataObjectsTimeStamp)* | </SignedDataObjectProperties> | | </SignedProperties> | | <UnsignedProperties> | | <UnsignedSignatureProperties> | (CounterSignature)* | </UnsignedSignatureProperties> | | </UnsignedProperties> | | </QualifyingProperties> | | </ds:Object> | | </ds:Signature>- - - - - - - - - - - - - - - + | XAdES

                       
Readers must take into account that the XAdES forms build up on the[XMLDSIG] by adding new XML elements containing qualifying information within the shown [XMLDSIG]ds:Object element, according to the rules defined in the present document. This ds:Object element will act as a bag for the whole set of qualifying properties defined in the present document, conveniently grouped.

So, there are 2 questions to answer here:

1. Is there a way to handle the Digital Signature prefix in the SignedXML Class in .NET Framework 1.1
2. If so - How ? If not - How ?

I decided to spend some time on this and after much disassembling some of the System.Security.Crytography.XML binaries, I found out to my dismay that the answer to Question [1] is NO. This is because the constants and the URIs of the XML Digital Signature functions in the System.Security.Crytography.XML space are found in the XMLSignature class and that class is declared as an internal class .

Therefore, the answer to Question [2] would be to build our own customized Digital Signature stack. This may actually sound harder than it is. Truth is:- With Reflector and work done behind the MONO-Project and published on Koders.com, I hacked a workaround in a few hours time. That actually means that I didnt really do much testing on it and so I disclaim myself from any liabilities, including, but not limited to, mistakes, injuries, deaths, etc caused if you choose to use it.

You would use this assembly just like you would with System.Security.Cryptography.Xml. The namespace would be Softwaremaker.NET.Security.Cryptography.Xml.PfDsigInterop.

Do take note that I has ONLY implemented the XML Digital Signature in this assembly.

To verify the signed XML, we would just have to use back the System.Security.Cryptography.Xml found in the .NET Framework. At least, the .NET stack got the design of the namespaces and the prefixes right.

You can download my [prefixed-XMLDSIG] custom assembly here. Do let me know if you have any comments or feedback. Enjoy !!!

[AUTHOR UPDATE]:

The XAdES specifications did not EXPLICITLY state that the prefix is needed so I don't see how the conclusions are drawn that prefixes are fixed. Maybe I am missing something.

It looked to me like all the spec was saying was that the *examples* used those prefixes.

It strikes me as surprising that any specification worth its salt would specify a *fixed prefix*. It would have been too restrictive and not something that many vendors would agree and abide.

I have advised the other party to check with the other parties/vendors for this. In the meantime, I will pull this assembly offline until I get better clarifications.

Now-Now Scott, I think we both agree on my thoughts (below) since Day 1, but I wouldnt go as far as you. .

My Evil trophy would (have) go(ne) to "Add Web Reference". Why the past tense in that ? It is rectified.

I recently had a chance to sit down with a client to discuss about software systems as well as to find out what we can do for them in their new system proposal.

I mentioned the 80/20 rule and explained to him what is was.

Then, we talked about (Web) Services and he was aghast that we could still be using (Web) Services even within his own LAN - and not connected to the WWW.

"...but I thought you need to be using a browser and connected to the internet to be using (Web) Services ? ..."

Sigh. I have heard that one too many times. One of the recent misnomers, no doubt.

Services, Services, Services. Code and Location is irrelevant. I had posted something like this a while back. It is good that the industry is taking some steps to correct this. See Point [1] of this referenced post.

With regards to my post here, I thought I expand on one of many enhancements that WS-Security Specifications 1.1 brings.

"MutualCertificate11Security" assertion is one of the few security turnkey assertions in Web Services Enhancements (WSE) 3.0 and what basically it is is that the client and server are authenticated using X.509 certificates (X509SecurityToken). Message-level security is implemented using X509SecurityToken security tokens. This turnkey security assertion requires WS-Security 1.1

Once that is configured and implemented properly, it is rather interesting to see what transcends on the wire. Here is a brief snippet:

[xenc:EncryptedKey Id="SecurityToken-6ec8..." xmlns:xenc="http://www.w3.org/2001/04/xmlenc#"]
[xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p"]
[ds:DigestMethod xmlns:ds="http://www.w3.org/2000/09/xmldsig#" Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" /]
[/xenc:EncryptionMethod]
[KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#"]
[wsse:SecurityTokenReference]
[wsse:KeyIdentifier ValueType="...oasis-wss-soap-message-security-1.1#ThumbprintSHA1" EncodingType="...oasis-200401-wss-soap-message-security-1.0#Base64Binary"]qRTA40Xfk6w1Os3mgpgy8UgwR/Y=[/wsse:KeyIdentifier]
[/wsse:SecurityTokenReference]
[/KeyInfo]
[xenc:CipherData]
[xenc:CipherValue]hBfCfVmg...[/xenc:CipherValue]
[/xenc:CipherData]
[xenc:ReferenceList]
...
[/xenc:ReferenceList]
[/xenc:EncryptedKey]

[Signature Id="Sig-b679..." xmlns="http://www.w3.org/2000/09/xmldsig#"]
  [SignedInfo]
  [ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" /]
  [SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#hmac-sha1" /]
  [Reference URI="#Id-5cdc..."]
  ...
  [/Reference]
  [/SignedInfo]
  [SignatureValue]O/PdsVMS4PTIBtrx8eyFNzbTnjc=[/SignatureValue]
  [KeyInfo]
  [wsse:SecurityTokenReference]
  [wsse:Reference URI="#SecurityToken-6ec8..." ValueType="...oasis-wss-soap-message-security-1.1#EncryptedKey" /]
  [/wsse:SecurityTokenReference]
  [/KeyInfo]
[/Signature]

[Signature xmlns="http://www.w3.org/2000/09/xmldsig#"]
  [SignedInfo]
  [ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" /]
  [SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" /]
  [Reference URI="#Sig-b679..."]
  ...
  [/Reference]
  [/SignedInfo]
  [SignatureValue]PDm4wS+3hzmXugHL1wcTWZXHcaGKkODVHU48XvVNC6catxiOr25
xq9AGN8u8CgYo1JlnoEf2tuCUl86krKiUBSnMR/towfAs2doGg6a+vtjIl9F54c/VZtTPgwn
QdZtJ28E8+ep5MIS2i+9Tamnui6qpX16IS3J1FcMjVBHQpMs=
[/SignatureValue]
  [KeyInfo]
  [wsse:SecurityTokenReference]
  [wsse:Reference URI="#SecurityToken-76ae..." ValueType="...wss-x509-token-profile-1.0#X509v3" /]
  [/wsse:SecurityTokenReference]
  [/KeyInfo]
[/Signature]
...

One thing that you will noticed is that there are 2 Digital Signatures generated.

The first one has a ReferenceID, which hints that it will be subject to encryption/signatures later on, and it is signed by a EncryptedKey type (which I talked about in my earlier post). Because it is encrypted by a symmetric key "#SecurityToken-6ec8", the [SignatureValue] is rather short and this signature basically signs the soap:Body with an URI of "#Id-5cdc..." The [EncryptedKey] value can be decrypted and derived by the server's private key

The second signature basically signs the first signature (#Sig-b679...) and it signs it with the Client's Private Key that only the corresponding Public Key Pair can decrypt. The Public Key, together with the client's cert is sent over the wire via a [wsse:BinarySecurityToken] (#SecurityToken-76ae...). Because an asymmetric key is utilized here, the [SignatureValue] is relatively longer than the first signature.

As we can see from here, the first signature signs the soap:Body and the second signature signs the first signature. These are generally known as "Supporting Tokens". These additional tokens may be specified to augment the claims provided by the token associated with the “message signature” provided by the Security Binding. Supporting tokens may be specified at a different scope than the binding assertion which provides support for securing the exchange.

There are four properties related to supporting token requirements which may be referenced by a Security Binding: [Supporting Tokens], [Signed Supporting Tokens], [Endorsing Supporting Tokens] and [Signed Endorsing Supporting Tokens]. Four assertions are then defined to populate those properties: SupportingTokens, SignedSupportingTokens, EndorsingSupportingTokens, and SignedEndorsingSupportingTokens.

What I have shown above is known as the [EndorsingSupportingTokens].

The [SignedEndorsingSupportingTokens] is a combination of [SignedSupportingToken] and [EndorsingSupportingToken] and I will talk about that in a future post.

I have been back fiddling at the Web Services Enhancements (WSE) 3.0 technology space for the past few weeks and I am now grokking into the plumbings since there are quite a few forums, blogs, posts out there focusing on the higher-level abstract programming model.

I am very pleased that WSE 3.0 has implemented WS-Security Specifications 1.1 at its very core, so much so that it is the de-facto security standard to be used in most of the Turnkey Security Assertions that comes with the product.

One of the things that I have always been asking for through my contacts and channels in the OASIS WS-Security Technical Committee is the ability to sign my document snippets with a symmetric key instead. Since XML-Encryption already utilized symmetric key encryption via the [xenc:EncryptedKey] [1] for performance and throughput reasons, there is no reason why XML-Digital Signature cannot do the same. While the improvements in throughput may be slight due to the fact that in digital signatures, messages are already hashed before encryption, it is still a viable option that should be made available.

In WS-Security 1.0, this is what is commonly seen in the Digital Signature Parts:

[SignatureValue]SJfnqZeDHboWDI2n2gWHqTJO5hXvZOFQw8UtDdajktzR40H+W6D
prs5CW/l9A5TF3xcFfyryA3hz7c+0vdlZSnaA+cBn2qPxt7/YmwaAx5Ave
awuach6YPYI123I4I3f58eSMUgPsx6/uuFQFcJltEMw1nWLE6Wb6CPg5OdtXLs=[/SignatureValue]
  [KeyInfo]
  [wsse:SecurityTokenReference]
  [wsse:Reference URI="#SecurityToken-7b5d..."
ValueType="...oasis-200401-wss-x509-token-profile-1.0#X509v3" /]
  [/wsse:SecurityTokenReference]
  [/KeyInfo]
  [/Signature]

Now in WS-Security 1.1 via WSE 3.0, I can do this:

[xenc:CipherData] 
[xenc:CipherValue]n6PnkIWb+QsIeOPehLdtQQKYZn202uGqhN+ShCWyBaCf20rmVcta
Bw2MhB1fv9pE0hOLpAxMMT5ffk4/hnwZ/ef2XcZediF6ySfpse14TI2TGy
cp9XErpeYlZNn1wSchHlOEz2gVYfViZoEOIwn8qR7EofLN3U3Mc5Zp2qG2coI=[/xenc:CipherValue]
  [/xenc:CipherData]
  [xenc:ReferenceList]
  [xenc:DataReference URI="#Enc-0914..." /]
  [xenc:DataReference URI="#Enc-3aab..." /]
  [/xenc:ReferenceList]
  [/xenc:EncryptedKey]
  [xenc:EncryptedData Id="Enc-0914..." Type="http://www.w3.org/2001/04/xmlenc#Element" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#"]
  [xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes256-cbc" /]
  [xenc:CipherData]  [xenc:CipherValue]RLASn...[/xenc:CipherValue]
  [/xenc:CipherData]
  [/xenc:EncryptedData]

[Signature xmlns="http://www.w3.org/2000/09/xmldsig#"]
  [SignedInfo]
  ...
  [SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#hmac-sha1" /]
  [Reference URI="#SecurityToken-d217..."]
  [Transforms]
  [Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" /]
  [/Transforms]
  [DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" /]
  [DigestValue]A0K7OVwZN3vP4rIXfbTZYy+f+ck=[/DigestValue]
  [/Reference]
  [Reference URI="#Timestamp-6f12..."]
  ...
  [/Reference]
  [/SignedInfo]
  [SignatureValue]d8KmXpfspmKiEOZ1eBVY7mk05Wo=[/SignatureValue]
  [KeyInfo]
  [wsse:SecurityTokenReference]
  [wsse:Reference URI="#SecurityToken-32e4..."
ValueType="...oasis-wss-soap-message-security-1.1#EncryptedKey" /]
  [/wsse:SecurityTokenReference]
  [/KeyInfo]
[/Signature]

Note the fonts in RED. There is a new URI: http://docs.oasisopen.org/wss/oasiswss-soap-messagesecurity-1.1#EncryptedKey which ties to a ValueType: A token type of an [xenc:EncryptedKey]

... and more importantly ...

the [SignatureValue] contents is slightly shorter now because it is actually encrypted with a symmetric key now. I know I may be picking here BUT hey, even a single byte in reduction means a lot in throughput performance in terms of wire transfer of documents. .

There are other improvements WS-Security 1.1 brings over its predecessor 1.0, which I will blog more about as I go along.

[1] Just a note to defuse any confusion if it crops up: Asymmetric (Public-Private) key technologies are still very much used in WS-Security and other document-related security specifications today. I dont think it will be dropped anytime soon unless Quantum Cryptography takes off mainstream in a big way or the subtle effects of this reverberates adversely through the security space. Having said that, Symmetric (Master/Session) key technologies have a huge place in the security specifications world too, just by the fact that it is a 1000x faster than much-secure asymmetric ones. Therefore, both technologies co-exist and work very well together in the document security space. To put it simply, a symmetric key (KeyA) is randomly-generated (key-length can be specified by the application - the longer, the better) and then used to encrypt/decrypt messages (MsgA). Once that is done - KeyA is then subsequently encrypted by an asymmetric key (KeyB) and then transmitted through the message. Only the holder of the Private-Key pair of KeyB will be able to decrypt KeyA which can then decrypt/encrypt MsgA.

I was approached to write about Windows Communication Foundation (WCF, previously - Indigo) a while back for SearchVB.com, a member of TechTarget.com.

Between a few new transitional CTP drops and major Microsoft events, I managed to hack out a simple piece here targetted at entry WCF-VB developers. There is another intermediate one (that deals with security) which will appear slightly later on TechTarget.com. Stay tuned to this space for further updates.

Do take note that this piece is currently one of the very few out there that is written against one of the latest WCF versions (The Sept-CTP WinFX drop).

Enjoy and send those comments over.

I was told that there appears to be a couple of non-working samples in the Windows Communication Foundation (WCF, previously - Indigo) Sept-CTP drop. I found out (the hard way) that one of them was the BasicSecurityProfile sample.

There is a workaround to it. There are basically 2 bugs in that sample. It may help solve the other bugs in the samples. This will only affect the security samples that uses the WS-Security Specifications in the [basicHttpBinding] bindings.

BUG 1. The affected sample will only work with X.509 Digital Certificates that has the Subject Key Identifier (SKI) installed. Unfortunately, the cert samples, which are used, are being issued by makecert.exe which doesnt generate X.509 certs with the SKI.

1. You can create test certificates from Verisign. Those test certs will come with SKI
2. You can set up a Certificate Authority (CA) on Windows 2003 Server. This is not installed by default and you need to add that component into your server setup. This will issue you a cert with SKI.

On a separate note, X.509 Digital Certificates that come with SKI offer the best approach in interoperability, so it is best recommended that you work with certs that comes installed with it.

BUG 2. Once you fix the workaround to BUG 1, and you run the BasicSecurityProfile sample and the client barfs this exception at you:

1. On the client proxy, change the replyAction = "*"

Once you have these 2 workarounds done up, the BasicSecurityProfile sample should work.

Both these bugs will be fixed in the subsequent WCF drop. I hope this at least helps someone.

In Web Services Enhancements (WSE) 2.0, one could exercise some control over what one xml element/fragments wants to encrypt within a soap:Body. Therefore, if I wanted to encrypt the account string in my credit card type, I could do something like this:

[At your Service Side]

Public Class SecuredString
  'Set the Oasis Id that our security reference will point to
  <XmlAttributeAttribute("Id", _
Namespace:=".../2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd")> _
  Public ID As String
  <XmlText()> _
  Public Data As String
End Class

[At your Calling side]

Dim c As SoapContext = a.RequestSoapContext

b.CreditCardType = "VISA"
z.Id = "uri:demoId.softwaremaker.net" 'or some guid
z.Value = "123-456-789"
b.CreditCardAccount = z

c.Security.Elements.Add(New EncryptedData(tok, "#uri:demoId.softwaremaker.net"))

Note: To reduce headache-inducing typo bugs, you may want to use some WSE Constants such as
WSUtility.Prefix
WSUtility.AttributeNames.Id
WSUtility.NamespaceURI

The end result of this is a soap:Body on the wire looks like this:

While I dont know how much of performance benefits this has over one that has the entire SecureCreditCard encrypted (since it is an symmetric-key encryption at its core), I think in terms of latency and throughput, it does offer some benefits especially with a sizable payload (>20-30 kb, for instance ?)

Windows Communication Foundation (WCF, previously - Indigo) does not currently have that feature build in at the moment (Sept05-CTP or known as the PDC-bits). In other words, in WCF today, you encrypt the entire contents of the soap:Body, lock-stock-barrel. I would still love that WSE feature in there: To be able to exercise finer grain control over what I want to or not to encrypt within a soap:Body.

Would really like to find out if I am the only odd one out there. Any users using that existing WSE feature out there that would love to see the same in WCF or do you have other better ideas ? Leave a comment or email me via the contact link on the side. Thank you.

BTW: Whether you encrypt certain elements of the contents or encrypt the entire contents of the soap:Body, both are WS-Security Specifications compliant .
 

I had posted this thought quite a while back on an old blog. Seems like the recent exchange in W3C, more or less, have re-visted my thoughts on this issue.

Jim Webber sums it well here.

Another speaking assignment that landed at my desk is for me to talk about "Web Services Security: How to track along the Security Standards Jungle" on the 21 October 2005

Somehow, I tend to attact the dry topics . This event is owned by OASIS and XMLOne Usergroup. More information can be found here.

If you are around the vicinity, do feel free to drop by and catch up.

I received my Certified Web Services Professional (CWSP) Certificate today in an official business ceremony. I have known this for some time but this is the first time it was made known to the public in an Infocomm Development Authority of Singapore (IDA) Weave Event. I am one of only 8 people who are in the pioneering batch to receive this certificate. The other 7 people come from technology vendors such as SUN Microsystems, IBM, Software-AG and members of the tertiary educations of higher learning academia.

More on the CWSP:

The CWSP is announced during XMLAsia 2003 and the Framework is developed jointly by IDA, the National Infocomm Competency Centre (NICC), Singapore Information Technology Federation (SiTF) and XMLOne User Group (UG). It aims to develop Infocomm professionals and re-skill them with Web Services competencies by meeting the career and skills-progression needs of Web Services talent at all levels, ranging from Developers, Professionals, Architects to Consultants.

I have been following the exchanges between Savas, Jim Webber, and Michi Henning here and here. There are more links but I will leave it up to the interested reader to use a RESTful approach to refer to those resources from the above 2 links.

To be honest, this has been piped over the newsgroups, forums, conferences, etc for some time now and it is really nothing interesting to debate about, really.

All this noise has led to a lot of FUD in the field and I get a constant barrage of questions in any of the technology conferences I speak in or attend. I work in the fields out there and therefore I tend to approach technology unlike that of academics, trainers or vendors. I do what clients want in the most efficient way (read:cost_and_resource-effective) possible. I have met a few people from academics as well as from the product vendors who entered the field thinking that just because they can point out which exact page number explains the ds:SignedInfo in _WS-Security Specs_, they can convince and conquer the field and have every single customer out there upgrade their existing technological infrastructure every 6 years.

Well, the Mainfraimes, the CICS and the COBOL wonks are still out there. Still making a lot of money for its vendors and service providers. People are still driving Ladas and these legacy will be there for some time, probably a long long time.

We get a lot of younER developers who are very confused with all barrage of technologies out there and sometimes people on the field (which means customers as well) get the short end of the stick when these developers use the wrong technology in the different parts of the technical solution. So, yes, some of the stuff you read in the Daily WTF is not ficticious.

Sometimes, when I come across comments like these here from Savas's blog here:

> Microsoft is betting on SOAP and made it a key part of its distributed computing platform, not DCOM.

Betting on SOAP? Hmmm... .NET remoting does not use SOAP. It uses a binary protocol for performance reasons. So, I'm not sure that Microsoft are "betting on SOAP". They certainly are not for their .NET remoting protocol. And DCOM failed because it could not be made to scale, due to its misguided garbage collection idea. And because DCOM, amazing as that may sound, was even more complex than CORBA.

Somehow, I either feel that I still dont get the picture or that irrationality is clouding good judgement (still).

Of course, .NET Remoting doesnt use SOAP. In fact, it used to and is deprecated for good reasons. It is a distributed object technology which implies implicit method invocation. SOAP is not a distributed object technology. It is all about services, all about standard schemas, being explicit in design and yes, it also means dispatching these XML documents on a "hopeless transport" or the Lada of the network protocol today. You cannot compare them just like you cannot compare the performance of objects and services.

Is this the best we can do ? Of course NOT. Should we all dump our existing heap of scrap metal in our garage and get the shiniest and fastest aluminium today ? Of course. Are we going to empty our bank account, forfeit and compensate on our current loan arrangments to do it ? NO, NOPE, NADA ... This is a just a fact that we have to deal with.

Having said that, to me, both sets of distributed technologies will have its place to stay, regardless of what the vendors say to sell more and the trainers sell to teach more. Each have its place and their merits tend to show up best if used and deployed wisely in the different layers, tiers and boundaries of a decent, usable and viable solution. When I say Solution, I mean an entire composition of different new and old systems that services a business program, initiative and ultimately a goal. Isnt that what we are building systems for in the first place or have I totally lost my mind and lost track of who my paymaster is ? Dont get me wrong, progress is not possible without making full proof and implementations of the latest rocket science or theories. But Progress can be measured in many ways. To most people, progress is measured by how they can make legacy or existing technologies and architetures last and endure given the rapidly evolving set of standards, protocols and environments and how fast they can go home and spend time with their families as age increases and TTL declines.

COM+, DCOM, .NET Remoting is something we use very frequently on the field, and for good reasons too. I am known to be a (W3C) SOAP Wonk BUT I will not give them up easily within the innards of my system and I will use SOAP for the reasons it was designed for. In fact, to me, one of the most important features in SOAP is the @role (@actor) and the @mustUnderstand. Or else, I would just stick with just plain old XML.

Is Microsoft betting on SOAP ? You bet, and so is the entire industry. It is a well known fact that while it is simplistic in design (in fact, this is one of the wierd specification that becomes simpler as the newer iteration evolve. Let hope it remains this way), getting across-the-board agreement is the costly element. In fact, it took 5 years from the original design meeting and prototype to become an “official” specification and it (SOAP 1.1) is still not an official W3C Submission. The cost to each participating organization easily crosses several million USD. The rough estimates to putting the final WS-* specs to bed (if there is such a thing) would easily be more than 20 million USD.

Just like life, in the field of software engineering, compromise is something we need to work with. I once had a straight-through exhaust pipe the circumference of a toilet bowl fitted underneath my car as well as a wide-open air filter underneath the car bonnet. After 3 months, I couldnt deal with the generated noise as well as the (less-dense) hot air the air cone was taking in from all angles so I dumped both of them. While it may take me slightly longer (by a few seconds, perhaps) to reach the market to buy groceries, the brat in the younger me learnt to deal with it.

While, I have my own ideas on the Request/Reply MEP, RPC, Document-Literal Messaging, etc and I like to share my research and thoughts with some of the brightest minds in the industry over a hot cup of Java, it is not something I lose sleep or sweat about. It just serves to keep my sanity in check when I still have to deal with OLE and VB3 issues today and it does make good conversation with some of the most intelligent geeks out there.

Sometimes, I feel the point of technology is lost when people argue over stuff like that. To these people, I recommend a good classic read: Technopoly: The surrender of Culture to Technology by Neil Postman

I have recently flashed some performance graphs between Java 1.5 and the .NET counterparts version 1.1 and 2.0 Beta2. This is where I got the statistics from.

You will see the outright disparity in performance gains when .NET 2.0 Beta2 is put through the same tests with its poorer 1.1 brother and its distant cousin Java 1.5.

...And we are still talking about .NET 2.0 BETA 2 ... Let us see what happens when we get to November 7 2005

My own little test utility tool - Manual SOAP Post has been seeing a fair number of downloads and I have received some good compliments on it as well as good feedback.

Actually, it is really nothing to shout about. It is one of those tool things that people have been using under the hood and therefore taken for granted. Since I have been doing a fair amount of work in terms of wire-interoperability, I figured that an abstraction (or proxy)-neutral tool would be good from a common point. So, I am being explicit in crafting out a message (bypassing the various abstractions that SOAPHttpClientProtocol, WebServicesClientProtocol, ServiceModel.ProxyBase brings), it would be good to send the same (W3C) SOAP message to different endpoints running different implementations and platforms and see how interoperable is that message from a single console. It is also a great tool for testing / troubleshooting a production environment.

This new version just consist of a couple of minor enhancements (due to public feedback):

1. The Windows Form application can be re-sized now. . I had missed that out in the earlier version.
2. Explicit inclusion of other HTTP Headers can be added into the transmission of the SOAP Message.
3. SSL / HTTPS Testing should work fine now. Because this is a test tool, I have set it to trust all X509 incoming certificates. So, use it ONLY for testing within a trusted application domain.
4. Timeout can be set in the configuration file now.

As I am doing a fair amount of work with Indigo, I discovered that version 1.0 of Manual SOAP Post cannot dispatch a properly-formed and valid message to an Indigo endpoint that had an implementation of the basicProfileBinding with FormatMode=ContractFormatMode.XMLSerializer. After spending an insane amount of time spent digging into the HTTPHeaders in the wee hours of the morning, I found that the Indigo is very anal with the charset. Specifically, the header of Content-Type: text/xml is not enough. It was good enough for an ASMX and a Web Services Enhancements (WSE) 2.0 endpoint. But Indigo wanted more. So, I had to include the charset=utf-8 into the media-type before it would open its doors to talk to you.

This is now fixed in this 1.1 version of the Manual SOAP Post. You can explicity add extra headers into the transmission in the configuration file which renders it slightly more extensible.

If you are interested, get it here and Enjoy and keep those feedback and criticisms coming.

[Author Note:] Please uninstall previous versions of Manual SOAP Post before installing the new version. Thank you.

Fellow Microsoft Regional Director and a software legend, Rocky Lhotka posted an interesting entry on Message-based WebMethod "overloading" here.

In it, he recommends (and I quote) that service methods use a request/response or just request method signature: 

   response = f(request)

or

   f(request)

 "request" and "response" are both messages, defined by a type or schema (with a little "s", not necessarily XSD).

The actual procedure, f, is then implemented as a Message Router, routing each call to an appropriate handler method depending on the specific type of the request message.

I couldnt agree more. While easier to comprehend, the practice of passing parameters to a Web Method call often sends the wrong messages across as (W3C) SOAP as just another RPC. Ultimately, if you have no idea and dont even want to grok the innards of the XMLSerializer, you would really think you are passing method parameters across or worse, think that you are ferrying objects across the wire.

Therefore, I firmly believe that it is for the good of all if you explicity specify that you are expecting a XMLDocument / XMLElement / XMLNode and you will dispatch the same on your endpoint. What you do at your implementation (whether it is serializing from a type to deserializing from one) is placed squarely at the mercy of the tools in your hand or the knowledge in your head.

With the current state of tools (or lack there-of) today, I sit on the same camp as Aaron Skonnard and Christian Weyer (plus others, I believe) as I believe firmly in the contract-first approach. Good devs should dig what is going on as close to the wire as possible, at least once. Then they can use all the wizards and helpers they want. This will help them understand better what is going on if they should need to troubleshoot later (leaky abstractions) or find other ways to improve performance.

This is just something that my team and me here have gathered over the years esp when I have got many developers out there working on the same solution offshore in different countries.

While I agree that not many people out there enjoy or want to grok the angle brackets and that the lack of tools out there is hugely to be blamed, tools make a productive developer but not necessarily a proficient one.

Until today, I still come across developers that still think Web Services are still about transferring an object across the wire. Having terms like "Web References" and "proxies", even deemed to be more abstract and dev-friendly, does potray the wrong ideas across.

I have always recommended younger developers who are interested in learning about Web Services / ASMX and SOAP to try out the (just-now-defunct) Microsoft SOAP Toolkit first before moving on. I find that to be a great interesting way to learn about XML SOAP Services as abstractions are kept to a minimum. Another great SOAP Toolkit in the face of Microsoft's non-supporting stance of its own is Simon Fell's PocketSOAP.

Another fellow Regional Director, Bill Wagner (who authored an very impressive book "Effective C#") posted his solution to Rocky's post here. I have used the same sort of approach that Bill had documented before and it is good and does serve its purpose. However (and please correct me if I am wrong), it bounds the message contracts and datatypes tightly to the WSDL. If I am going to add a third Request / Response pair of classes, it will render my initial WSDL invalid (unless of course, if I am willing to add an additional endpoint)

I worked on a project before which specify that (newer) datatypes are to be added in phases and therefore, we had to decouple the XSD from the WSDL (which is in accordance with one of the best practice of WSDL Deployments -- modular and the separation of concerns). Oh, by the way, this is practiced in Indigo.

I don't know if there is a way to decouple the XSDs from the WSDL in VS2003 today. Even if there is, I am guessing it is a difficult hack at best. So, what I did was to create the XSDs for each datatype as they come along and do a XSD import in my WSDL. At my message exchange contract, I used an open content type with xsd:any. Thereafter, I author my own WSDL with the help of Christian's and Thinktecture's WSContractFirst. Since the message and the datatype XSDs are all imported, the wsdl actually has a small footprint now. With the wsdl /server switch, xsd:any becomes an XMLElement type. For abstraction within my service, I changed it to an object in my implementation detail.

Note: .NET 1.1's WSDL.exe /server switch, in my mind, is still fairly limited and comes with a couple of annoying things I didn't like BUT I will expand this in detail later.

[System.Xml.Serialization.XmlTypeAttribute([Namespace]:="urn:softwaremaker-n
et.swmstoreexchangemessages"]> _
Public Class Response
  Public Result As Result
End Class

[System.Xml.Serialization.XmlTypeAttribute([Namespace]:="urn:softwaremaker-n
et.swmstoreexchangemessages")] _
Public Class Result
  [System.Xml.Serialization.XmlElement(ElementName:="Book",
Type:=GetType(BookType),
Namespace:="urn:softwaremaker-net.swmstoreextendedtypes"), _
  System.Xml.Serialization.XmlElement(ElementName:="CD",
Type:=GetType(CDType),
Namespace:="urn:softwaremaker-net.swmstoreextendedtypes"), _
  System.Xml.Serialization.XmlElement(ElementName:="Anything",
Type:=GetType(AnyType),
Namespace:="urn:softwaremaker-net.swmstoreextendedtypes")] _
  Public Any As Object
End Class

  Public Function
ProcessRequest([System.Xml.Serialization.XmlElementAttribute([Namespace]:="u
rn:softwaremaker-net.swmstoreexchangemessages")] ByVal Request As Object) As [System.Xml.Serialization.XmlElementAttribute("Response",
[Namespace]:="urn:softwaremaker-net.swmstoreexchangemessages")] Response
   '...
   End Function

Once the consumer points to my wsdl (I turned off documentation of asmx by default), all the XSDs are imported by VS2003 as well (and that is a good thing). The xsd:any is still an XMLElement over at their end and we leave it as it is since we cannot control what they do there. The consumer can choose to deal with the raw XML if they want to OR do an xsd.exe imported.xsd /c and let the XMLSerializer do its (limited) magic .

In this sense, no matter how many more datatypes I add or remove, my wire-contract remains the same. I just let the endpoints serialize / deserialize the open-content xml (xsd:any). In my project I had mentioned earlier, I have one asmx endpoint servicing multiple consumers each sending different messages that conforms to different XSDs (For example, GroupA sends BookType and GroupB sends the CDType as well as a GroupC next time that sends a type that is unknown to me today). The thing I need to take care of is to send the appropriate datatype xsd to the appropriate groups so they can serialize / deserialize into the appropriate types. As you can see from the code snippet above, at my asmx, I will just add any new datatypes that come along.

While, this may seem like too much compared to Bill's solution above, it was necessary to decouple the datatypes from my wsdl in that project and the XSD Editor (codenamed: daVinci ?) in VS.NET2003 was the best tool I had in hand at that time. Developers are too comfortable with objects. This is obviously natural with the decade-long evolution of Object-Orientation and the power of abstractions the OO tools and languages bring today.

However, other factors like time, extensibility, standardization make abstractions expensive...and among others, these are all factors that make up the principles of the move towards Service-Orientation. Now, if we have to make and designed services to be explicit, this means that devs need to know they are explicity invoking a service across the wire, how far can the tools go to hide all the plumbings and yet not hide the fact that the service is on the other side and not just a mere shadow copy of a remoting object that seems to reside on the same application domain.

Will Service-Orientation fail to hit the mainstream because of this ? I dont know. I guess only time will tell.

Ian Yates interviewed me for what turns me on for MSDN Magazine SEA Edition. Read about it here.
© MSDN Magazine SEA Edition

(W3C) SOAP is very like to XML-RPC and not the other way around.

It works by marshaling procedure calls over HTTP as XML documents. However, one major difference is that SOAP allows the documents to be treated literally as --- well, Documents.

As you will be able to see from the angle-brackets snippets below -- you will notice some major differences between SOAP and XML-RPC. One major difference you will notice right away is that XML-RPC doesnt have any of the XML Namespaces, doesnt support XML Schemas and has a very definite set of types and primitives. One of the major improvements of SOAP 1.2 over its preceding version is the removal of Section 5 Encoding, which, just like XML-RPC, is to improve the room for interoperablity by allowing people and parties to agree by having lesser opportunities for people to disagree (^tm). I wont go much into details here. Eric Kidd has built up a fantastic repository about XML-RPC here.

As I have mentioned before, I had done some work before revolving around XML-RPC. In fact, I still see major operations out there still communicating via XML-RPC. While I love SOAP, XML-RPC still does get the job done if you wanted simplicity and faster interoperability. I am however, willing to bet that, as distributed computing really pushes the realm of distribution and enterprise requirements of Security, Transactions and Reliability becomes more deeply rooted in the message, SOAP will become the defacto standard in message transmission. In the same tone of distribution, I do still remember that while doing XML-RPC, while some people and companies will argue that it is a loose-coupled concept, it sure doesnt apply to the parties involved in the transactions and invocations. We spent unproductive hours on the phone and meeting tables arguing what should be sent over and many more over what it is supposed to look like on the wire. If weapons were allowed then, I am sure we would have reached an agreement faster. Of course, the weapons I meant here were things like the Colt .45 Pistol and the S&W .460 Magnum. Because, tools were more primitive then, we actually had to write every single element out before it goes onto the wire. Serialization was unheard of then.

I have built 3 simple Service Operations which is then broken into SOAP and XML-RPC Formats. The 3 methods and their I/Os are simple but they should be able to illustrate the different signatures and return types.

Function HelloWorld() AsString...

Function SumAndDiff(ByVal x AsInteger, ByVal y AsInteger) As SumAndDiffStruc...

Function RetSoftwareDevelopers() As SoftwareDeveloper()...

These functions are then decorated with [XmlRpcMethod()] and [SoapMethod()] to distinguish themselves to their operation and binding approaches.

Let us see how each of them is represented differently on the wire. I will show the XML-RPC format first then followed by SOAP:

<methodCall>

  <methodName>HelloWorld</methodName>

  <params />

</methodCall>

<methodResponse>

  <params>

    <param>

      <value>

        <string>Hello World</string>

      </value>

    </param>

  </params>

</methodResponse>

<methodCall>

  <methodName>SumAndDiff</methodName>

  <params>

    <param>

      <value>

        <i4>15</i4>

      </value>

    </param>

    <param>

      <value>

        <i4>10</i4>

      </value>

    </param>

  </params>

</methodCall>

<methodResponse>

  <params>

    <param>

      <value>

        <struct>

          <member>

            <name>sum</name>

            <value>

              <i4>25</i4>

            </value>

          </member>

          <member>

            <name>diff</name>

            <value>

              <i4>5</i4>

            </value>

          </member>

        </struct>

      </value>

    </param>

  </params>

</methodResponse>

<methodCall>

  <methodName>RetSoftwareDevelopers</methodName>

  <params />

</methodCall>

<methodResponse>

  <params>

    <param>

      <value>

        <array>

          <data>

            <value>

              <struct>

               
<member>

                 
<name>Name</name>

                 
<value>

                   
<string>William Tay</string>

                 
</value>

               
</member>

               
<member>

                 
<name>Age</name>

                 
<value>

                   
<i4>30</i4>

                 
</value>

               
</member>

               
<member>

                 
<name>Profile</name>

                 
<value>

                   
<struct>

                     
<member>

                       
<name>Experience</name>

                       
<value>

                         
<i4>10</i4>

                       
</value>

                     
</member>

                     
<member>

                       
<name>Plaform</name>

                       
<value>

                         
<string>Microsoft .NET Technologies</string>

                       
</value>

                     
</member>

                   
</struct>

                 
</value>

               
</member>

              </struct>

            </value>

            <value>

              <struct>

               
<member>

                 
<name>Name</name>

                 
<value>

                   
<string>ABC DEF GHI</string>

                 
</value>

               
</member>

               
<member>

                 
<name>Age</name>

                 
<value>

                   
<i4>50</i4>

                 
</value>

               
</member>

               
<member>

                 
<name>Profile</name>

                 
<value>

                   
<struct>

                     
<member>

                       
<name>Experience</name>

                       
<value>

                         
<i4>10</i4>

                       
</value>

                     
</member>

                     
<member>

                       
<name>Plaform</name>

                       
<value>

                         
<string>Microsoft .NET Technologies</string>

                       
</value>

                     
</member>

                   
</struct>

                 
</value>

               
</member>

              </struct>

            </value>

          </data>

        </array>

      </value>

    </param>

  </params>

</methodResponse>

Below are the respective SOAP Representations:

<soap:Envelope xmlns:soap="..." xmlns:xsi="..." xmlns:xsd="...">

  <soap:Body>

    <HelloWorld xmlns="..." /> 


  </soap:Body>

</soap:Envelope>

<soap:Envelope xmlns:soap="..." xmlns:xsi="..." xmlns:xsd="...">

  <soap:Body>

    <HelloWorldResponse xmlns="...">

      <HelloWorldResult>Hello World</HelloWorldResult> 


    </HelloWorldResponse>

  </soap:Body>

</soap:Envelope>

<soap:Envelope xmlns:soap="..." xmlns:xsi="..." xmlns:xsd="...">

  <soap:Body>

    <SumAndDiff xmlns="...">

      <x>15</x> 


      <y>10</y> 


    </SumAndDiff>

  </soap:Body>

</soap:Envelope>

<soap:Envelope xmlns:soap="..." xmlns:xsi="..." xmlns:xsd="...">

  <soap:Body>

    <SumAndDiffResponse xmlns="...">

      <SumAndDiffResult>

        <sum>25</sum> 


        <diff>5</diff> 


      </SumAndDiffResult>

    </SumAndDiffResponse>

  </soap:Body>

</soap:Envelope>

<soap:Envelope xmlns:soap="..." xmlns:xsi="..." xmlns:xsd="...">

  <soap:Body>

    <RetSoftwareDevelopers xmlns="..." /> 


  </soap:Body>

</soap:Envelope>

<soap:Envelope xmlns:soap="..." xmlns:xsi="..." xmlns:xsd="...">

  <soap:Body>

    <RetSoftwareDevelopersResponse xmlns="...">

      <RetSoftwareDevelopersResult>

        <SoftwareDeveloper>

          <Name>William Tay</Name> 


          <Age>30</Age> 


          <Profile>

            <Experience>10</Experience> 


            <Plaform>Microsoft
.NET Technologies</Plaform> 


          </Profile>

        </SoftwareDeveloper>

        <SoftwareDeveloper>

          <Name>ABC DEF GHI</Name> 


          <Age>50</Age> 


          <Profile>

            <Experience>10</Experience> 


            <Plaform>Microsoft
.NET Technologies</Plaform> 


          </Profile>

        </SoftwareDeveloper>

      </RetSoftwareDevelopersResult>

    </RetSoftwareDevelopersResponse>

  </soap:Body>

</soap:Envelope>

As you would have noticed by now, XML-RPC has a fixed set of types and abstractions such as methodCall, methodName, params, value, struct, etc and of course, its response member-name-value pair. Like its name suggests, the param elements do represent each a parameter and the encapsulating structure does look like an object-instance representation. SOAP, on the other hand, makes heavy use of XML Namespaces and Schemas to validate how and what the message should look like.

We are, however, moving away from this RPC behaviour of transmission. This is a good thing as SOAP, while requiring much more engineering complexity, does offer much more extensibility in terms of message exchange patterns and extensible headers. Even the RPC-Encoding RPC-ish style messages that are generated by the SOAP toolkits of a couple of years back is making way for Document-Literal as the default style today. You take the messages "AS-IS" and how you mapped them into your platform native language and what you do with them is left to the implementation details of each platform. The XML schemas act as a way for you to tell what is legal or illegal as IO to your service.

Please note that I am using Charles Cook's fantastic XML-RPC .NET for my XML-RPC implementation here. Another way for you NOT to re-invent the wheel is to use Simon Fell's PocketXMLRPC. I am a huge fan of Simon Fell's SOAP Tools and I am sure this is a winner as well.

In my recent tour of the local university campuses giving talks and presentations, I noticed that most of the students find my talks hard to digest and understand. This is not surprising since I have always been known to be a server guy and not much of a tools person. Actually I used to be a client-tools person BUT have since spent the last year or so dabbling in distributed computing protocols.

This is not just associated with students, BUT some of the younger folks that I had a chance to engage as well. This goes really far to show how old I am. It is very hard to extol the benefits of XML/SOAP and Services if the younger folks dont understand and appreciate the history of COM, MTS. Pretty soon, when Indigo matures from the wild, technologies like COM+ and .NET Remoting will become legacy as well.

Everyone who grew up in the .NET era never knew about DLL Hell in deployment. Those things I need to do at 12 midnight when everyone is asleep and I am given the safe-clear command to do a regsvr32 for a 10KB Win32 COM DLL and then reboot the servers after...and I have not even mentioned about those nightmarish versioning problems when someone else's DLL is installed over mine....

Well, it is good for those folks just like it was good for me not to have gone through Yourdon's Structured Programming era. Now thinking about it made me realize that maybe I would have appreciated object-abstractions better if I had.

Anyways, since I was very focused on distributed computing protocos and XML SOAP Services in my talks and since the audience were mainly younger students who had no professional experience, I tend not to dabble too deep into BEST PRACTICES in Deployments, Versioning and Interoperability scenarios. I am sure most of them would rather poke a needle into their eye than to watch me write angle brackets, do some xsd typing, and then import namespaces before binding them all together in a WSDL file. Therefore, I usually give a more general broader overview of the industry, how come we have what we have today and what are we getting tomorrow. The history part usually sits well with them and so most of the questions that came up usually relates to that.

Since most of their curriculum and labworks never went back to technologies that were more than 2 years old, I do find it useful sometimes to show them how COM, MTS and DCOM was done last time. That usually catches their attention although I think it was more of an amusement to them more than anything else.

"Now, see this black thingy, we call them vinyl in our days and we put this needle-like shaped thing on it so that music can be piped out from this trumpet-like thingy...and we have to put it carefully on the edges so that it starts from the first tune...and if we want to repeat the same tune again, we have to sit up, go back to the machine, lift the needle up, squint our eyes on the vinyl so we can pick out the correct sequence of the groove on the vinyl that the song starts on and put it carkefully and slowly on that groove and VOILA, we have replay....What ? Oh, we have to do that everytime we want to replay that song...Whats that ? Shuffle, Random --- What's that ?"

So, running DCOMCNFG.EXE and OLEView is not something I would do too often just to get a chuckle from the student audience

However, I promise some of them I would blog about XML-RPC in relation to (W3C) SOAP and that is what I intend to do. There are still more than a few implementations out there in the real-world that are happily humming along with XML-RPC. I had done a couple of them before using MS SOAP Toolkit to XML-RPC with an apache server. I dont see them clamouring after SOAP so soon as those functionality are pretty straight-forward Request-Response endpoint invocations. In fact, the banner hanging in their server room reads: "NO SOAP, NO Problems" which I thought was quite cool even though I am a big fan of SOAP.

Anyways, how can a Web Service history lesson be complete without XML-RPC. Dave Winer, who came up with XML-RPC amongst other things is still one of the key guys today, in my mind, who has shaped the landscape of distributed computing today. There is no doubt that SOAP has its roots in XML-RPC. XML-RPC is sort of an early generation of Web Services and it went a long way to define one of the principles of Service-Orientation which is to create as little shared abstractions as possible. In fact, there are so few primitives and types in XML-RPC that there is so little to agree and disagree on which, in turn, sets a positive tone in integration and interoperability. Incidentally, that was one of the reasons why SOAP 1.1 Section 5 Encoding was taken out in SOAP 1.2.

I would try to see what I can come up with to write some XML-RPC scripts on .NET, just for the fun of it. I dont see any scenarios that I could use it unless one needs to write a .NET client to connect to a UNIX XML-RPC Server. Extensibility could be an issue as I could not route those messages, at least, not easily. On the other hand, because XML-RPC is slightly simpler (XML-RPC = SOAP++), it may pose less of an interop hazard.

In some of my recent XML SOAP Services demos and presentations, I found myself needing to manually post a SOAP message to a listening (W3C) SOAP Services on http and also on the tcp transport protocol.

While doing that, I can also change the SOAP Message on the fly and post it instead of having to change the way VS.NET XMLSerializer serializes the types and then recompiling and running it again. Besides saving time, it does add an element of oommph as well as there is no break in momentum and more importantly, audiences really do see and can appreciate what is going onto the wire real-time.

Being able to change the messages on the fly as well also lets you see very quickly and clearly what the receiving endpoints can or cannot do. In this sense, with just a few tweaks of the angle brackets, I can generate SOAP Faults at will and it gives the audience great examples for learning how and when SOAP Faults are generated. For example, changing the element names OR re-ordering the elements of a SOAP Message and see what and how the receiving endpoint reacts. I will leave this as an exercise to the reader.

I have written a small tool to help me do that. Recently, I have just added an ability to the application to allow it to post SOAP messages over the TCP Transport channel to a listening Web Services Enhancements (WSE)-enabled SOAP Service.

Granted, most people will still need to manually pre-compose a SOAP Message before-hand. Do this as part of your pre-presentation work and save the messages for posting to the receiving endpoints later. Of course, if you are an angle bracket freak like me or Tim or Radovan, then you should all feel comfortable typing them into the textbox.

Seriously though, this tool allows you to save your SOAP message scripts as part of your preparation work. You can either post these as files (via the OpenFile Dialog button) or copy-n-paste those scripts into the textbox.

I believe that this can serve some basic purpose scenarios. Do note that besides doing presentation work with this, I think this is a great tool for developers who need to further test their SOAP Services to make sure it is able to generate the exact SOAP Responses (SOAP Faults and all) with every single differing SOAP Request.

For example, users testing their WSE-Enabled SOAP Services can change the wsu:Timestamp element to test the service timeouts or play around with the wsse:Security elements of _WS-Security Specs_.

[Pre-requisites]:

• You will need to install the Web Services Enhancements (WSE) 2.0 Runtime for the TCP Transport function to work
• .NET Framework 1.1
• MSXML 4 SP2

[Notes]:

• This will only work for the Request-Response MEP for WSE-Enabled SOAP Services

[Upcoming Plans]:

• Incorporate the ability to listen for incoming SOAP Messages as well. So it will work for truly asynchronous, duplex type SOAP Services as well
• ...

I have made it available for download here. I plan to make the source codes available once I incorporate most of the features the market wants.

Please feel free to leave your comments and feedback in this post so I know how to better improve it.

Tim and Dare had some great exchanges here where they discussed on the importance and significance of staring at angle brackets and service reach.

Tim hit on a sweet spot which I can definitely identify with when he wrote this:

"Is it an issue in companies doing lots of legacy integration work?"

I guess we all come from and work in different IT surrounding environments. I work in a fairly large Systems Integrator where loads of our work revolves around integration and maintainence of legacy systems in vertical Line-Of-Businesses. Since it is also a free business environment, we also have to integrate with work done by other SIs, systems and technology vendors as well. This is a fairly tough proposition and if we are not involved in sitting around tables discussing how to integrate data formats and semantics, we are usually thrown a stack of documents describing the data schemas of other companies and people doing previous work with other technology systems.

Now, XML-Schema is usually (Thank God for that) the agreed-upon format for describing these data formats and that is all we have. Nothing more. Most of the time, we are just given documents with table formats and we have to markup those table formats in XSD ourselves.

So, the XSD-Editor in Visual Studio is definitely a God-sent (Geek-speak, of course) to generate the schema. I was left to my own barehands and notepad to create the WSDL.

No, other WSDL toolkits were not an option, cost-wise. Thinktecture's WSCF definitely gave us a huge productivity boost in terms of generation of those WSDL files. By that time, our brains are so well-tuned to angle brackets that we could just hand-author the resultant WSDL to produce the exact result we want. Examples include having multiple service elements and soap:location address attributes for the same SOAP Binding, importing of re-usable data schemas and the separation of abstact and concrete defintions in the WSDL File. By doing all that, we built extensibility, versioing and interoperability as well...and we went in there with our eyes wide open. Troubleshooting is relatively a breeze now, one look at the SOAP Request and the WSDL; we know where and how the SOAP Fault happens.

Is that a problem with primitive tooling that can be solved down the road ? Maybe and I truly hope so. In this instance, I hope that tool utility developer companies can strive for that goal. I am, however, sceptical that a holy grail can be achieved so soon.

I still think, with all things status quo, that developers need to know explicitywhat goes onto the wire. This has got nothing to do with what is the "native" or real contract or the the canonical forms used to describe the messages. The problem is that when someone gives us these contracts or schemas described in canonical forms, it is the only thing we have that we best understood what is to be sent on the wire so that the other end of the pipe can intepret, understand and process these messages.

...Therefore, I have no choice to not just stare at angle brackets BUT understand what it is trying to convey as well.

p/s: One of my favourite article that exhorts the virtues of the Contract-First Approach

Can anyone be surprised with Cisco's decision to enter the XML Messaging market ? With the rapid propagation of XML into the mainstream market esp in the areas of XML Messaging (aka XML SOAP Services), Middleware, Brokering, Intermediaries, it is about time customers get more choices and exposure in this arena.

With Cisco in this segment of the market, I do expect more alternatives and choices in XML Boxes to be had by customers and them paying lesser as well.

I still remember very clearly it was only less than 2 years back when I brought up the idea of introducing an XML Firewall into a client's environment by then Flamenco Networks (which is now acquired by Digital Evolution) and being pooed-pooed right in my face on how a text-based representation can be used for messaging and how the contents of this representation can be inspected at wire-speed at the hardware level.

Well, Moore's Law has made it so that the price and speed of high-powered specialized processing such as XML-Encryption and XML-Digital Signature besides just plain inspecting and routing of SOAP packets can be made available to the grasp of the general masses. This will also bring forward the concept of Application-Oriented Networking into the market. I dunno BUT are we seeing another layer of abstraction into the mix now ?

Too bad this client is not around anymore for me to say "I told you so..."

I am very interested to see how this whole affair pans out with the middleware app-server companies such as Big Blue

SDA-Asia has recently published one of my articles online. In this article, I talked about how Web Services Enhancements (WSE) can be used to solve Real-World business problems with some proper thought and design processes.

This is not as technical an article comparred to the ones I have written before, however, I feel it gives a good overview and insight to what the advanced XML services are and how to make use of some of them to solve some of the business problems of today...and needless to say, WSE 2.0 is THE tool to do that today in .NET

I have spent a fair amount of time writing up a REAL technical article (on WSE, of course) recently which I hope will get published soon enough. Will update all once it goes live. Enjoy.

Bill Gates talks about Microsoft's commitment to interop

Finally, this is done. Read the cover story here.

Even though I usually dont advocate the use of SOAP to transfer large BLOBS of binary bits (I tend to think that there are slightly better ways of transferring files and/or attachments), I do believe that there will be more and more calls for uses for it and that is why I think this is an important specification which is way better than W3C's SwA or MSFT Corp's/IETF's  DIME (Can you say Security ?). The latter, incidentally, is being implemented by Web Services Enhancements (WSE) 1 and 2.

Of course, since DIME is being superseded by MTOM, I am really looking forward (hopefully) to the implementations of it in WSE 3.0

I recently extended a chat program written in Web Services Enhancements (WSE) 2.0 utililizing the TCP transport protocol. I wanted it to run over the internet now. This was prompted by some members of our Singapore Professional DotNet Usergroup who wanted to implement a mini Pub-Sub (aka WS-Eventing) project using the tcp protocol.

They had indicated that they had some trouble binding the FQDN specified for the URI in the EndpointReference to a local network interface. If this cannot be resolved, the binding will fail.

At the most basic level, the EndpointReference.Address is both the name of the target and its location. For example, [http://localhost/SomeService] can be used as a name and a transport address. WSE 2.0 comes with an extended model to allow a single named service to be hosted on multiple transport addresses. This can also be useful if we want to apply a single corporate policy on them.

This is where EndpointReference.Via comes in. It allows you to host a well-known service that can only be accessible via different transports such as the soap.tcp protocol of WSE 2.0 and others.

For example, my well-known service is hosted at soap.tcp://softwaremaker.net:2088 (or soap.tcp://219.74.47.214:2088, if you dont have a DNS tagged to it). This is the public FQDN address of the service. For the broadband cable and ADSL users of dynamic IPs, this address may be the address of your home network router.

Now, your WSE 2.0 application has a SOAP Receiver object that allows your application to listen for SOAP requests coming in on WSE2-custom defined transports. Once you define your own address on your own listening machine such as soap.tcp://192.168.2.100:2088/Softwaremaker, you would now have 2 URIs:

Public Facing: soap.tcp://softwaremaker.net:2088
Private Internal: soap.tcp://192.168.2.100:2088/Softwaremaker

The Public Facing well-known URI becomes the EndpointReference.Address while the transport address mechanism becomes the EndpointReference.Via

In other words, soap.tcp://192.168.2.100:2088/Softwaremaker will listen for any messages that is sent to soap.tcp://softwaremaker.net:2088 (the address element in the WS-Addressing headers). Put it in another way, the custom soap.tcp transport will only accept messages sent to soap.tcp://softwaremaker.net:2088 if they arrive via the internal address:192.168.2.100.

The Via address is not in the message itself, only the well-known URI is. Based on the network address the message is being received on, the custom soap.tcp transport mechanism will know how use both well-known and private URIs to dispatch the message to the registered and configured-properly SOAP Receiver.

    EndpointReference EPR = new EndpointReference
    (new Address(new Uri("soap.tcp://softwaremaker.net:2088")),
    new Via(new Uri("soap.tcp://192.168.2.100:2088/Softwaremaker")));

    SoapReceivers.Add(EPR, yourOwnSoapReceiver);

The last step you need to take is to be able to forward all requests going to the well-known URI (soap.tcp://softwaremaker.net:2088) to your local machine with the internal IP 192.168.2.100:2088

This is not something you can do in WSE 2.0 or in your machine. This has to be done at your broadband or cable router. You just have to forward requests coming in to your Public port:2088 to the Private port:2088 of machine 192.168.2.100 (Note that these ports doesnt have to be the same number at all)

Of course, the huge assumption is that you have control over the port routing and forwarding of your broadband or cable router (which may be rather tricky if you are within a corporate or organizational boundary) BUT technically, you will be able to carry out a p-2-p SOAP message chat or even do a Pub/Sub model approach with your own implementation of the “Notification” and “Solicit-Response” message-exchange patterns over the internet using the custom soap.tcp protocol of WSE 2.0

[Author Note] What do ya know ? I just found that that Hervey has got an even better post on the EndpointReference.Via of WSE 2.0 here.

William Stacy has a new blog in MSN Spaces and he blogged about a topic that has spurned from recent activities in the WSE newsgroups.

It is about the security and the use of username tokens in the real world, specifically when it relates to passwords being stored as hash in the user database.

William proposes a method here to solve the issue of how to authenticate username tokens embedded in _WS-Security Specs_ security headers.

In my own opinion, his case represents a very major common representation of how passwords are stored in the database. They are mostly and SHOULD be hased. To add another realm of security, SALT is used. However, I dont really agree that his method may prove to more secure. In fact, I think it gives a somewhat false sense of security (and William, please correct me if I am wrong anywhere).

1)   By using your verifier as a matching mechanism to gain authentication, then this verifier (SALT Hash and all) effectively becomes a password. If Mallet steals the hash from the user database, nothing prevents him/her from writing a client that sends the hash and thus he would authenticate successfully. This is exactly what the hash scheme is trying to avoid in the first place.

If used this way, the hash is no safer than storing the passwords in clear text in the database...which ultimately means that the weakest link is still the security of the Username Database which must be protected at all costs.

In other words, this must be made known clearly so that it doesnt preach a false sense of security that people think that this scheme NOT only protects their cleartext password BUT also their SOAP calls as well.

This post here explains real well on my thoughts as well.

2)   SALT should be a completely random value and should be algorithmically introduced to further obfuscate the one-way hash. In your approach, it seems that the SALT is tied very much to your username (am I wrong ?). This doesnt help much because I dont need to guess your publicly-available username, there are only a handful of hashing algorithms out there and you are NOT passing any SALT into your PasswordDeriveBytes constructor. In fact, the only saving grace is the iteration count which basically sets the number of iterations for the operation. Is that enough ? I believe it may serve well for some smaller-scale simpler systems BUT I dont know how it sits well with some of the security policies of bigger enterprises.

IMHO, I think the current username tokens as defined by WS-Security doesnt fit very well into most EXISTING user databases which store passwords as hashes which may or may not be SALTED. I dont think there is a clean way to do this in WSE without using or representing the hashes as passwords. However, I am still thinking about this issue and will blog more about it later if (not when) I reach enlightenment.

Your approach, however, does allow passwords to remain as they are in the EXISTING user databases. The only obstacle you require is a change in the schema of that table, which requires management approval and buy-in and usually triggers off many other related procedures, and that in my experience, is sometimes harder than finding the password from a hash.

[Author Note]: This topic has got nothing to do with the security of username tokens as I have blogged about earlier here. This is all about the authentication of username tokens with the stored credentials in the user databases.

After installing Web Services Enhancements (WSE) 2.0 SP2, the most natural thing for me to do is to run my previous WSE 2.0 SP1 projects and see if it works with the new assembly.

Most of it still do run <phew>. A couple of them (specifically the SOAP Routing ones) did NOT run as intended. A further drilldown into the code revealed that the ExtendedSecurity doesnt serialize properly if you are planning to send another set of Security Headers to the next SOAP node...In fact, it doesnt get serialized at all.

...Obviously, this next node cannot be the ultimate soap receiver as target service cannot process 2 security headers as specified in the _WS-Security Specs_. You can refer to my previous discussions with HerveyWhere and here.

So, if your next node is another SOAP Router and you need a different set of Security Headers for the next node to process, this is how to do it with the ExtendedSecurity function in SP2:

You need to add a Security Utility Timestamp into the Security Header in order for the ExtendedSecurity function to serialize a new set of Security Headers properly.

Dim t As Utility.Timestamp = AnotherSecurityHeaderBlock.Timestamp

Thereafter, when you call the ExtendedSecurity function and add this new Security Header block, the serialization will take place.

In SP1, it is NOT necessarily for you to add this timestamp.

The WSE team has confirmed this as a workaround to this minor bug.

Mike Champion has a great post here on the overheads of XML.

IMHO, I think a binary representation of XML brings on a whole different set of issues, namely removing the abstraction it is supposed to represent.

On a slightly different perspective and from a designing front, I have been advocating against the use of XML in all layers of a enterprise application esp. when tightly bound object technology is much more desired. In my presentations on SO(A), I have always preach on best using service-messaging as communication b/w applications, NOT between tiers of an application.

However, many businesses are using XML Services as a communication mechanism JUST SO they can be seen as implementing an SO(A)... and of course, for all the wrong reasons.

Hence, many of them complain when performance suffers.

What do they expect when they are making verbose, chatty calls to their own Data Access Layer via SOAP ?

Another good XML Learning Resource TopXML has built the XML News Reblogger. As quoted here on their Website:

“Reblogger is a free aggregator service provided by TopXML. We collect XML blog items four times a day from around the web, so that you can view all the most interesting and most useful news right here in our reblogger. Enjoy

View the blogs and feeds we fetch... ”

With reference to my earlier post here, I had explained how we can encrypt the Usernametoken element <wsse:UsernameToken> if we choose to use the PasswordOption.SendPlainText enumeration, for real-world reasons such as

• Windows Authentication
• Passwords are stored as Hash in the UserDB

Other elements in the <wsse:Security> Header element can be encrypted too, although great care and design must be taken as it may reduce the extensibility of SOAP Headers through routing intermediaries. One of them is the <Signature> element.

As I had explained in another post here, digital signatures can and may be verified for authentication and trust by any SOAP node. If the <Signature> element is encrypted, we may be preventing any SOAP intermediary from authenticating and verifying the digital signature. Unlike digital signatures, <xenc:EncryptedData> elements are encrypted for a specific receiver in mind and therefore, only that one party SHOULD be able to decipher it with a corresponding Private key or shared secret. SOAP intermediaries, trusted or not, SHOULD NOT be able to decrypt or view the the encrypted content(s) and therefore cannot authenticate and verify the signature.

However, if one should decide that their dispatching mechanism is based on a non-intermediary route or if the <Signature> element may not be meant for the ultimate SOAP receiver and therefore can be removed by the SOAP intermediary, this can be done easily as well with WSE 2.0.

The key is to create a MessageSignature on its own and assign it an ID. Here is the code snippet on how to implement it:

And the wonderful result that comes out from the oven: (geez...I need a life )

<wsse:Security soap:mustUnderstand="1">
<wsse:BinarySecurityToken ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3" EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" wsu:Id="SecurityToken-e8f64eea-1d63-4db2-943c-9bfb5dfccbfc">MIIBxDC...du2fPMER8ajJfl</wsse:BinarySecurityToken>
<wsse:UsernameToken wsu:Id="SecurityToken-3370d9ae-deb9-4a01-9b9c-c8dd072568fa">
<wsse:Username>ABC</wsse:Username>
<wsse:Nonce>Amx9amrLXR02etJLsNwdwA==</wsse:Nonce>
<wsu:Created>2004-12-11T07:38:33Z</wsu:Created>
</wsse:UsernameToken>
<xenc:ReferenceList xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:DataReference URI="#EncryptedContent-0e6936bf-67a5-48a5-ba8a-d9ba6141e75f" />
</xenc:ReferenceList>
<Signature Id="2c091bb3-bcdc-4da1-97c5-dcd60dac7312" xmlns="http://www.w3.org/2000/09/xmldsig#">
<xenc:EncryptedData Id="EncryptedContent-0e6936bf-67a5-48a5-ba8a-d9ba6141e75f" Type="http://www.w3.org/2001/04/xmlenc#Content" xmlns:xenc="http://www.w3.org/2001/04/xmlenc#">
<xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes128-cbc" />
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
<wsse:SecurityTokenReference>
<wsse:Reference URI="#SecurityToken-3370d9ae-deb9-4a01-9b9c-c8dd072568fa" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#UsernameToken" />
</wsse:SecurityTokenReference>
</KeyInfo>
<xenc:CipherData>  <xenc:CipherValue>rFfJS87yAdFFkW1dVbot...tJ+9U8+CU5qsI=</xenc:CipherValue>
</xenc:CipherData>
</xenc:EncryptedData>
</Signature>
</wsse:Security>

Voila! The Red bold font will show that the Signature Element is now encrypted.

To slightly expand on my post and Hervey's a little bit further, there was mention on the use of Enveloped Signatures in the SOAP Headers. Enveloped Signature (as defined by XML-Digital Signature) is a signature over the XML content that contains the signature as an element. The content provides the root XML document element. Obviously, enveloped signatures must take care not to include their own value in the calculation of the SignatureValue. In other words, Enveloped Signature would sign the contents of the SOAP headers WITHOUT the signature. This is the only way a Security header can be signed without creating a circular reference dependency.

To do the above, you are enforcing the prevention of intermediaries from modifying the SOAP Headers.

However, as taken from the _WS-Security Specs_

WS-Security doesnt *believe* in the Enveloped Signatures because it stands on the belief that SOAP Headers are mutable. Since SOAP Headers can change and the likelihood is there that a SOAP intermediary can change the headers, an Enveloped Signature would not be as extensible and work as well.

I am a strong believer of that. If a normal signature is used instead of an enveloped one, then an intermediary can safely add more tokens and more signatures to a Security header targeted at another node on the message path...and that is why there is WS-Security. Security cannot just be based on End-to-End scenarios or else SSL / HTTPS will suffice.

I also further believe that an intermediary should be able to extend Security headers which are meant for other target nodes. Since a SOAP node can only process a single Security Header (because of re-ordering constraints of SOAP Headers), this option may not be as far-fetched or ridiculous as it may sound.

Of course, anyone can still choose to implement Enveloped Signatures over their SOAP Headers if they are just implementing End-to-End scenarios and enforcing non-tampering measures over any desired or un-desired intermediaries. However, extensibility may not be an option here should intermediaries be required to offload certain processing functionality off the ultimate receiver or even add more tokens and signatures along the way.

I have been noticing an increasing number of emails and newgroup threads asking for the security of Usernametokens as specified by _WS-Security Specs_ on OASIS. Most people would like to use it because it is the only alternative they have and there are no other options for using X.509 PKI Digital Certificates. Here is my personal take on it.

I think some of the security concerns are slightly misplaced here. Firstly, I dont think WS-I or OASIS would include Usernametokens inside the WS-Security Specifications if they doubt its security. As I would like to say, --- Implementation is key.

A Username token does NOT use any simpler or less-standard security algorithm than any other tokens. In fact, it uses the same hashing algorithm, symmetric algorithm such as the 128-key Cipher Block Chain (cbc) Advanced Encryption Standard, etc as any other token. Many people, also, do not realize that the same symmetric algorithm is used to encrypt the SOAP message body when an asymmetric X509SecurityToken is used as well. The asymmetric key algorithm is only used to encrypt the secret key that is doing the actual symmetric encryption processing. This is done for the purpose of reducing cipher bloat and increasing processing speed. The paranoia in me, however, would go for a higher-bit key implementation, which is possible.

Remember that your secret can be stolen and kept for years and tried to be broken with much higher-end and cheaper deciphering machines in the future. OK, OK, that is my paraniod self talking.

I believe when statements are made against the security of Usernametokens, they are made against the passwords of the Usernametokens. Therefore, the statement: "Usernametokens, on their own they are only as secure as the passwords"

Usernametokens are as secure as your passwords. That means that if you have a good security policy on how your company treats passwords, ie...

1. Minimum password length
2. Implementation of alphanumeric and other different characters and symbols in password
3. Password change frequency (in months instead of years )
4. Elimination of Weak Passwords such as using names and such
5. ...

you should NOT be so fearful of using a Usernametoken in your Web Service implementation.

On the other hand, if you don't treat or administer your passwords with good password policies, then you cannot expect Usernametokens to give your message as secure a protection as you would like.

I would also recommend using the PasswordOption.SendNone, if possible. The hash of the password and other elements are used to produce the cipher. NO password is sent over using this enumerated option. Of course, the only caveat is except through a dictionary attack, which of course, can be made so much more difficult (or almost impossible) by having a good password policy administration system.

If you have to send your Usernametoken over in PasswordOption.SendPlainText for whatever reasons (using Windows, LDAP Authentication or you may have hashed versions of your passwords stored in your UserDB), you SHOULD encrypt the UsernameToken with a X.509 digital certificate. Read my post here for my own implementation of it.

Another thing to take note is one that relates to the real world and why I believe Usernametokens have its place here. It is easiest to implement and common in any business environments. Therefore, it can be plugged into any existing IT systems with relatively lesser effort. Also, X509 digital certs are usually used to authenticate machines and / or companies, it would be more expensive and unrealistic to expect every user in a 100+ user organization to have a digital cert and a private / public key pair. Therefore, I strongly believe that Usernametokens are more appt to authenticate the users themselves in the real world and will continue to be one of the most popular way to authenticate users in the near biometric-less future. However, if you are using authentication between machines, you should opt for X509 digital certs instead.

• For security reasons, it is strongly recommended to encrypt Username tokens, especially when they contain password information. The SecurityTokenServiceClient class now automatically encrypts any UsernameToken security tokens included in outgoing SOAP requests. Similarly, the SecurityTokenService class automatically encrypts any UsernameToken security tokens included in outgoing SOAP responses.

For all those wanting to do WS-ReliableMessaging today with WSE 2.0, you can check out what Andrés has done here.

Word from the man Hervey himself. Find it here.

WSE2.0 SP2 Pre-release Full product and the Runtime

Contents from the ReadMe:

Core product changes:

• A new compatibility section is used to select the wire format on the sending
  side. The mode attribute tells WSE runtime to generate a message which will be
  compatibable to a particular release of WSE. By default, the mode is WSE2RTM. It can be WSE2RTM, WSE2SP1 WSE2SP2 and so on. On the receiving end, a particular version of WSE runtime will be able to accept all types of wire format in all its previous releases. In a request-response message exchange, a server will generate a response message which is compatible with the request message. The server will, by default, still use the compatibility section to generate its response if it cannot determine the compatibility mode based on the request message.
  
• A new implementation of the Kerberos token based on SSPI interfaces is included in this release. The new token is named KerberosToken2. Please see the reference documentation for more details.
  
• The TokenIssuer element in the KerberosToken security token assertion is
  no longer supported on the receiving end.
  
• The SecurityTokenManager no longer throws exception in LoadTokenFromXml() when token.IsCurrent returns false. WSE security input filter will continue checking that and throw exception if token.IsCurrent returns false.
  
• The inclusion of unencrypted Username tokens in a message may represent a security vulnerability. The SecurityTokenServiceClient class will now automatically encrypt any Username tokens included in a request. Similarly, the SecurityTokenService class will automatically encrypt any username tokens included in a response.

  The following methods have been added to the token issuing framework:

  protected virtual void SecurityTokenServiceClient.EnforceRequestUsernameTokenEncryption().
  Called from EnforceRequestPolicy(), this method enforces the requirement that any Username tokens in an RST message must be encrypted. The issuerToken is used as the encrypting token. This method will throw an exception if it cannot encrypt the UsernameTokens in an RST message.
  Override this method to suppress this behavior.

  protected virtual void SecurityTokenService.VerifyRequestUsernameTokenEncryption().
  Called from VerifyRequestPolicy(), this method verifies that tokens in an RST message are encrypted. This method will throw an exception if it encounters an unencrypted UsernameToken in an RST message.
  Override this method to suppress this test.

  protected virtual void SecurityTokenService.EnforceResponseUsernameTokenEncryption().
  Called from EnforceResponsePolicy(), this method enforces the requirement that any Username tokens in an RSTR message must be encrypted. The ResponseEncryptingToken is used as the encrypting token. This method will throw an exception if it cannot encrypt the UsernameTokens in an RSTR message.
  Override this method to suppress this behavior.

  protected virtual void SecurityTokenServiceClient.VerifyResponseUsernameTokenEncryption().
  Called from VerifyResponsePolicy(), this method verifies that tokens in an RSTR message are encrypted. This method will throw an exception if it encounters an unencrypted UsernameToken in an RSTR message.
  Override this method to suppress this test.
  

• A new method, protected virtual void SecurityTokenServiceClient.ClearRequestSoapContext(), has been added. This method was added to fix a bug in which successive requests for a SecurityContextToken, made through a single instance of SecurityContextTokenClient, would fail. The problem was caused by a failure to clear security elements and tokens from the soap context after a request was made. In the event that a sub-class of SecurityTokenServiceClient requires that security elements or tokens in the soap context be preserved from one request to another, the new behavior may be suppressed by overriding the ClearRequestSoapContext method.
  
• SoapService will now send back an empty SoapEnvelope back if the soap method it is invoking returns null for a request/response scenario.
  
• EncryptedData.Decrypt will only support decryption to one and only one xml element. It throws a security fault otherwise.
  
• In the SoapHttpOutputChannel.Send method, if the response has an unsupported content type, such as text/html, the response stream will be read and stored in the exception text.
  
• Two new properties, SimpleDisplayName and FriendlyDisplayName, have been added to the X509Certificate class.
  
• The default Label used in DerivedKeyToken has changed from "WS-SecureConversation" to "WS-SecureConversationWS-SecureConversation".
  
• If an incoming message contains multiple security tokens with envelope signature inside those tokens, the server was returning a security fault. This is now fixed.
  
• If WSE SOAP messaging stack is used over HTTP/HTTPS transport, a simple WSE805 exception was thrown when the response content type was not supported. Now if the response stream is readable, WSE runtime will read from response stream and throw a WSE805 exception with a more detailed error information read from the response stream.
  
• WSE configuration section will now allow whitespace or comments as child nodes for the following configuration elements: diagnostics/trace, diagnostics/policyTrace, diagnostics/detailedErrors, referralCache/cache, security/x509, security/limits.
  
• When a security context token was deserialized, WSE runtime will retrieve a token from its cache based on its globally unique Identifier. The token retrieved from the cache sometimes have a different Id than the token received from the incoming message. If that happens, WSE runtime would previously fail to verify a signature or decrypt the message. It is now fixed as it will assign the Id of the token from the incoming message to the token retrieved from the cache.
  
• The built-in SecurityContextToken service would previously cache a newly issued security context token instance before those properties defined in the IIssuedToken interface are set. Now it is fixed so that those properties are set before the newly issued token is cached.
  
• WSE runtime would previouly always generate a relative token reference in calculating message signature. When the token is not sent with the message, the receiver will return a security fault. It is now fixed so that an absolute token reference will be generated in the case when the security token is not in the message.
  
• A server fault was thrown previously when a server tries to verify a signature or decrypt a message based on a username token which uses plaintext password option or no password option and does not have nonce and/or created element. The exception is now changed to be a client fault.
  
• SoapService instances configured to automatically issue SecurityContextToken (SCT) security tokens no longer re-use the same instance of the SCT issuer. With this service pack, a new instance is created per request. This instance is accessible via the SoapService.AutoIssueSCTService property. If the old behavior
  is required, there are two work-arounds:
  
  
  
• override the SoapService.AutoIssueSCTService property to return a singleton instance of a SecurityContextTokenService, or
  
  
• create a wrapper SecurityContextTokenService that delegates to a singleton instance and register it through configuration.

Visual Studio tool integration changes:

• The default value for http Routing handler type is updated.
  
• WSE setting tool can support VS C++ project.
  
• WSE setting tool would always prompt users for confirmation when the Cancel button was clicked. Now it is fixed so that the tool will prompt user only when some changes are made by users.
  
• The Security Settings Wizard can support creating Policy files for remote service.
  

WseWsdl2 tool changes:

• The WseWsdl2 tool now properly generates proxy classes when an Web service uses a Guid type.
  
• If an input schema contains a type definition which is derived from another complex type, the proxy class would generate a class definition for that derived type with incorrect namespace. This is now fixed.

I came across this blog last week, detailing a problem he is facing with WSE 2.0

Now I that realize who he is (Hello Fai ), I peep into his project to see how I could help. Fai, incidentally, is the Academic DE for MSFT Malaysia and I met him in MS Tech.ED 2004 Asia in Msia.

Since I cannot comment on this blog (since theSpoke doesnt allow it and I dont intend to keep another account), I will comment on his problem here.

I reproduce Fai's blog snippets here.

I've been mucking around with WSE 2.0 Messaging, now moving on to SoapSender and SoapReceiver. I noticed a potential problem when I have a class that extends the SoapReceiver class, and within the Receive() method, it also sends out a SoapEnvelope via the SoapSender.Send() method. The problem only arises when I use a real transport protocol, i.e., TCP, but not when I'm using the in-proc transport. I've tried all sort of ways, including transfering the SoapSender code away from the Receive() method to a method in another class. But it still doesn't work.

env.Context.Addressing.Destination = e.Context.Addressing.ReplyTo.Address.Value;

What you missed out was actually pass in the address property of the ReplyTo class. This address property (which is inherited from an EPR) will get you the address of the destination.

So, your final code snippet should simply look like this (without the pipelines class):

SoapSender responseSender =new SoapSender(e.Context.Addressing.ReplyTo.Address.Value);
SoapEnvelope env =new SoapEnvelope();
env.Context.Addressing.Action = e.Context.Addressing.Action;
env.SetBodyObject(resp);
responseSender.Send(env);

I hope this helps. Try it and let me know here if it works.