This is targetted at our Singapore Government Public Sector customers and obviously this blog post is targetted at those customers who are reading this post. .

I will be speaking at the above event, which will be held at our office premises. As to the topic I am speaking about, this area is something which I have been focusing of late and it really is all about architecture and the alignment of it through businesss and information technology.

Aligning Business and IT for Greater Corporate Agility
For long, IT and Business has communicated. While they may be speaking in the same language but are they speaking in the same "dialect”?  In this session, William Tay will share his field experience as well as explore Microsoft's vision for service-oriented architecture:

• What are the best implementation approaches so that instead of being led by the business, IT strategies are driven by business needs
• How Microsoft supports this vision by way of an integrated platform, etc.

If you are coming for my session at the Enterprise Architecture Summit 2007 (18th Oct 2007), which coincidentally, is also targetted at the Public Sector customers (regionally, of course), the above event on the 22nd Oct 2007 will be a good sequel to my Enterprise Architecture Summit pitch where I will be discussing in greater details on the value of the Microsoft platform to implement those business capabilities.

I have been involved in a couple of pretty good competitive bids in some public tenders recently and am thoroughly enjoying it. I have come head-to-head with the usual suspects of heavy hitters in the areas of collaboration, SOA / ESB and portal plays such as IBM, Oracle and the likes.

On another related note: Since I am in the compete space recently, I have been keeping watch on what is coming out of the standards body and I think that WS-BPEL (which was BPEL4WS ...) should have their specifications ready by next month (March 2007). Having said that, one of the points that will make Microsoft a even stronger challenger in the enterprise space is what I believe will come out of the pipelines next month is that the next version of Windows Workflow Foundation (WF) (CTP) will support BPEL 1.1.

Moving forward, I believe that the v.Next of Orcas will support BPEL 2.0 as well as BPEL 1.1 and that will be very important in the BizTalk roadmap as well.

I recently had a chance to sit down with a client to discuss about software systems as well as to find out what we can do for them in their new system proposal.

I mentioned the 80/20 rule and explained to him what is was.

Then, we talked about (Web) Services and he was aghast that we could still be using (Web) Services even within his own LAN - and not connected to the WWW.

"...but I thought you need to be using a browser and connected to the internet to be using (Web) Services ? ..."

Sigh. I have heard that one too many times. One of the recent misnomers, no doubt.

Services, Services, Services. Code and Location is irrelevant. I had posted something like this a while back. It is good that the industry is taking some steps to correct this. See Point [1] of this referenced post.

On the back of this, from Paul Fallon's spacehere, Max Feingold has a real good summary on Whats New with WS-AT, WS-BA and WS-C.

Should make for some good reading on my looooooooong flight to LAX via the Pacific Route.

According to Don's blog here: Indigo has support for REST since Day 1. In the same breadth, he disclosed that it will explicitly support HTTP PUT/POST/DELETE/GET to please the REST purists out there.

Like Steve mentioned here, I, too, am not a fan of technology stacks that complete with their own ideologies. To me, technology is there just to get a job done efficiently and effectively. My post here attracted quite a fair bit of hits and same-echo-sentiment emails. It even made to ZiffDavis blog list here.

As Steve has echoed as well, I am also even more convinced now that Indigo is going to be the platform to beat when it comes to distributed communications. No doubt about it.

I am really glad that the Indigo team, whom I think comprises some of the brightest minds in the software field, is all united to work towards a common goal of creating a religion and ideology-agnostic product. If this is not innovation, I dont know what is.

Show me the money, Beta1

I have been invited by the Ark Group to speak in one of their major technology management conferences in Singapore on the 27th July 2005.

This conference is themed around Planning and Implementing Service-Oriented Architecture : Developing SOA to Reduce the Cost of Integration, Leverage Legacy Systems and Improve Business Agility.

My 45-min presentation is titled:

Design and Security Notes from the Field

• Distributed Computing
  • Thoughts
  • When to use What
  • More Thoughts
• Distributed Security
  • Thoughts
  • When to use What
  • More Thoughts

As usual to my presentation style, I tend to keep my topics interesting and a refreshing experience. The conferences in this field has always been dominated by the academics, researchers and technology vendors who sometimes lose touch of whats happening on the ground. I will go against this bias and inject some field experience in there. It should be an interesting conference for attendees and delegates. I think it will even be much more interesting when I sit down with them for networking lunch.

I hope to see you there.

I have been following the exchanges between Savas, Jim Webber, and Michi Henning here and here. There are more links but I will leave it up to the interested reader to use a RESTful approach to refer to those resources from the above 2 links.

To be honest, this has been piped over the newsgroups, forums, conferences, etc for some time now and it is really nothing interesting to debate about, really.

All this noise has led to a lot of FUD in the field and I get a constant barrage of questions in any of the technology conferences I speak in or attend. I work in the fields out there and therefore I tend to approach technology unlike that of academics, trainers or vendors. I do what clients want in the most efficient way (read:cost_and_resource-effective) possible. I have met a few people from academics as well as from the product vendors who entered the field thinking that just because they can point out which exact page number explains the ds:SignedInfo in _WS-Security Specs_, they can convince and conquer the field and have every single customer out there upgrade their existing technological infrastructure every 6 years.

Well, the Mainfraimes, the CICS and the COBOL wonks are still out there. Still making a lot of money for its vendors and service providers. People are still driving Ladas and these legacy will be there for some time, probably a long long time.

We get a lot of younER developers who are very confused with all barrage of technologies out there and sometimes people on the field (which means customers as well) get the short end of the stick when these developers use the wrong technology in the different parts of the technical solution. So, yes, some of the stuff you read in the Daily WTF is not ficticious.

Sometimes, when I come across comments like these here from Savas's blog here:

> Microsoft is betting on SOAP and made it a key part of its distributed computing platform, not DCOM.

Betting on SOAP? Hmmm... .NET remoting does not use SOAP. It uses a binary protocol for performance reasons. So, I'm not sure that Microsoft are "betting on SOAP". They certainly are not for their .NET remoting protocol. And DCOM failed because it could not be made to scale, due to its misguided garbage collection idea. And because DCOM, amazing as that may sound, was even more complex than CORBA.

Somehow, I either feel that I still dont get the picture or that irrationality is clouding good judgement (still).

Of course, .NET Remoting doesnt use SOAP. In fact, it used to and is deprecated for good reasons. It is a distributed object technology which implies implicit method invocation. SOAP is not a distributed object technology. It is all about services, all about standard schemas, being explicit in design and yes, it also means dispatching these XML documents on a "hopeless transport" or the Lada of the network protocol today. You cannot compare them just like you cannot compare the performance of objects and services.

Is this the best we can do ? Of course NOT. Should we all dump our existing heap of scrap metal in our garage and get the shiniest and fastest aluminium today ? Of course. Are we going to empty our bank account, forfeit and compensate on our current loan arrangments to do it ? NO, NOPE, NADA ... This is a just a fact that we have to deal with.

Having said that, to me, both sets of distributed technologies will have its place to stay, regardless of what the vendors say to sell more and the trainers sell to teach more. Each have its place and their merits tend to show up best if used and deployed wisely in the different layers, tiers and boundaries of a decent, usable and viable solution. When I say Solution, I mean an entire composition of different new and old systems that services a business program, initiative and ultimately a goal. Isnt that what we are building systems for in the first place or have I totally lost my mind and lost track of who my paymaster is ? Dont get me wrong, progress is not possible without making full proof and implementations of the latest rocket science or theories. But Progress can be measured in many ways. To most people, progress is measured by how they can make legacy or existing technologies and architetures last and endure given the rapidly evolving set of standards, protocols and environments and how fast they can go home and spend time with their families as age increases and TTL declines.

COM+, DCOM, .NET Remoting is something we use very frequently on the field, and for good reasons too. I am known to be a (W3C) SOAP Wonk BUT I will not give them up easily within the innards of my system and I will use SOAP for the reasons it was designed for. In fact, to me, one of the most important features in SOAP is the @role (@actor) and the @mustUnderstand. Or else, I would just stick with just plain old XML.

Is Microsoft betting on SOAP ? You bet, and so is the entire industry. It is a well known fact that while it is simplistic in design (in fact, this is one of the wierd specification that becomes simpler as the newer iteration evolve. Let hope it remains this way), getting across-the-board agreement is the costly element. In fact, it took 5 years from the original design meeting and prototype to become an “official” specification and it (SOAP 1.1) is still not an official W3C Submission. The cost to each participating organization easily crosses several million USD. The rough estimates to putting the final WS-* specs to bed (if there is such a thing) would easily be more than 20 million USD.

Just like life, in the field of software engineering, compromise is something we need to work with. I once had a straight-through exhaust pipe the circumference of a toilet bowl fitted underneath my car as well as a wide-open air filter underneath the car bonnet. After 3 months, I couldnt deal with the generated noise as well as the (less-dense) hot air the air cone was taking in from all angles so I dumped both of them. While it may take me slightly longer (by a few seconds, perhaps) to reach the market to buy groceries, the brat in the younger me learnt to deal with it.

While, I have my own ideas on the Request/Reply MEP, RPC, Document-Literal Messaging, etc and I like to share my research and thoughts with some of the brightest minds in the industry over a hot cup of Java, it is not something I lose sleep or sweat about. It just serves to keep my sanity in check when I still have to deal with OLE and VB3 issues today and it does make good conversation with some of the most intelligent geeks out there.

Sometimes, I feel the point of technology is lost when people argue over stuff like that. To these people, I recommend a good classic read: Technopoly: The surrender of Culture to Technology by Neil Postman

Fellow Microsoft Regional Director and a software legend, Rocky Lhotka posted an interesting entry on Message-based WebMethod "overloading" here.

In it, he recommends (and I quote) that service methods use a request/response or just request method signature: 

   response = f(request)

or

   f(request)

 "request" and "response" are both messages, defined by a type or schema (with a little "s", not necessarily XSD).

The actual procedure, f, is then implemented as a Message Router, routing each call to an appropriate handler method depending on the specific type of the request message.

I couldnt agree more. While easier to comprehend, the practice of passing parameters to a Web Method call often sends the wrong messages across as (W3C) SOAP as just another RPC. Ultimately, if you have no idea and dont even want to grok the innards of the XMLSerializer, you would really think you are passing method parameters across or worse, think that you are ferrying objects across the wire.

Therefore, I firmly believe that it is for the good of all if you explicity specify that you are expecting a XMLDocument / XMLElement / XMLNode and you will dispatch the same on your endpoint. What you do at your implementation (whether it is serializing from a type to deserializing from one) is placed squarely at the mercy of the tools in your hand or the knowledge in your head.

With the current state of tools (or lack there-of) today, I sit on the same camp as Aaron Skonnard and Christian Weyer (plus others, I believe) as I believe firmly in the contract-first approach. Good devs should dig what is going on as close to the wire as possible, at least once. Then they can use all the wizards and helpers they want. This will help them understand better what is going on if they should need to troubleshoot later (leaky abstractions) or find other ways to improve performance.

This is just something that my team and me here have gathered over the years esp when I have got many developers out there working on the same solution offshore in different countries.

While I agree that not many people out there enjoy or want to grok the angle brackets and that the lack of tools out there is hugely to be blamed, tools make a productive developer but not necessarily a proficient one.

Until today, I still come across developers that still think Web Services are still about transferring an object across the wire. Having terms like "Web References" and "proxies", even deemed to be more abstract and dev-friendly, does potray the wrong ideas across.

I have always recommended younger developers who are interested in learning about Web Services / ASMX and SOAP to try out the (just-now-defunct) Microsoft SOAP Toolkit first before moving on. I find that to be a great interesting way to learn about XML SOAP Services as abstractions are kept to a minimum. Another great SOAP Toolkit in the face of Microsoft's non-supporting stance of its own is Simon Fell's PocketSOAP.

Another fellow Regional Director, Bill Wagner (who authored an very impressive book "Effective C#") posted his solution to Rocky's post here. I have used the same sort of approach that Bill had documented before and it is good and does serve its purpose. However (and please correct me if I am wrong), it bounds the message contracts and datatypes tightly to the WSDL. If I am going to add a third Request / Response pair of classes, it will render my initial WSDL invalid (unless of course, if I am willing to add an additional endpoint)

I worked on a project before which specify that (newer) datatypes are to be added in phases and therefore, we had to decouple the XSD from the WSDL (which is in accordance with one of the best practice of WSDL Deployments -- modular and the separation of concerns). Oh, by the way, this is practiced in Indigo.

I don't know if there is a way to decouple the XSDs from the WSDL in VS2003 today. Even if there is, I am guessing it is a difficult hack at best. So, what I did was to create the XSDs for each datatype as they come along and do a XSD import in my WSDL. At my message exchange contract, I used an open content type with xsd:any. Thereafter, I author my own WSDL with the help of Christian's and Thinktecture's WSContractFirst. Since the message and the datatype XSDs are all imported, the wsdl actually has a small footprint now. With the wsdl /server switch, xsd:any becomes an XMLElement type. For abstraction within my service, I changed it to an object in my implementation detail.

Note: .NET 1.1's WSDL.exe /server switch, in my mind, is still fairly limited and comes with a couple of annoying things I didn't like BUT I will expand this in detail later.

[System.Xml.Serialization.XmlTypeAttribute([Namespace]:="urn:softwaremaker-n
et.swmstoreexchangemessages"]> _
Public Class Response
  Public Result As Result
End Class

[System.Xml.Serialization.XmlTypeAttribute([Namespace]:="urn:softwaremaker-n
et.swmstoreexchangemessages")] _
Public Class Result
  [System.Xml.Serialization.XmlElement(ElementName:="Book",
Type:=GetType(BookType),
Namespace:="urn:softwaremaker-net.swmstoreextendedtypes"), _
  System.Xml.Serialization.XmlElement(ElementName:="CD",
Type:=GetType(CDType),
Namespace:="urn:softwaremaker-net.swmstoreextendedtypes"), _
  System.Xml.Serialization.XmlElement(ElementName:="Anything",
Type:=GetType(AnyType),
Namespace:="urn:softwaremaker-net.swmstoreextendedtypes")] _
  Public Any As Object
End Class

  Public Function
ProcessRequest([System.Xml.Serialization.XmlElementAttribute([Namespace]:="u
rn:softwaremaker-net.swmstoreexchangemessages")] ByVal Request As Object) As [System.Xml.Serialization.XmlElementAttribute("Response",
[Namespace]:="urn:softwaremaker-net.swmstoreexchangemessages")] Response
   '...
   End Function

Once the consumer points to my wsdl (I turned off documentation of asmx by default), all the XSDs are imported by VS2003 as well (and that is a good thing). The xsd:any is still an XMLElement over at their end and we leave it as it is since we cannot control what they do there. The consumer can choose to deal with the raw XML if they want to OR do an xsd.exe imported.xsd /c and let the XMLSerializer do its (limited) magic .

In this sense, no matter how many more datatypes I add or remove, my wire-contract remains the same. I just let the endpoints serialize / deserialize the open-content xml (xsd:any). In my project I had mentioned earlier, I have one asmx endpoint servicing multiple consumers each sending different messages that conforms to different XSDs (For example, GroupA sends BookType and GroupB sends the CDType as well as a GroupC next time that sends a type that is unknown to me today). The thing I need to take care of is to send the appropriate datatype xsd to the appropriate groups so they can serialize / deserialize into the appropriate types. As you can see from the code snippet above, at my asmx, I will just add any new datatypes that come along.

While, this may seem like too much compared to Bill's solution above, it was necessary to decouple the datatypes from my wsdl in that project and the XSD Editor (codenamed: daVinci ?) in VS.NET2003 was the best tool I had in hand at that time. Developers are too comfortable with objects. This is obviously natural with the decade-long evolution of Object-Orientation and the power of abstractions the OO tools and languages bring today.

However, other factors like time, extensibility, standardization make abstractions expensive...and among others, these are all factors that make up the principles of the move towards Service-Orientation. Now, if we have to make and designed services to be explicit, this means that devs need to know they are explicity invoking a service across the wire, how far can the tools go to hide all the plumbings and yet not hide the fact that the service is on the other side and not just a mere shadow copy of a remoting object that seems to reside on the same application domain.

Will Service-Orientation fail to hit the mainstream because of this ? I dont know. I guess only time will tell.

Michael Liebow, VP of Web Services at IBM Global Services has a great entry on his weblog where he asked if "Customers really want an SOA". This is really a great post and I couldnt agree more.

He stated that Service Oriented Modeling and Architecture (SOMA) provides an approach to building an SOA that aligns to the business goals. It helps customers tie business processes to underlying applications to help the business realize benefits more rapidly. However, the key differentiator for SOMA is where the discussion starts. It's with the business problem.

He makes a few quoteworthy notes worth repeating:

1. "...businesses need better visibility into their business processes. Breaking the business down into component view -- from a discrete process or the business processes supporting the entire enterprise -- is critical to achieving business improvement and growth..."
2. "...Business process modeling will map out a companies' business processes and help determine which business processes provide strategic differentiation over competitors, what processes are core and what business processes may not be considered strategic..."
3. "...Once the business process change or enhancement aimed at growth has been identified, the technology conversation can now begin..."
4. "...Customers need to approach building an SOA based on the needs of the business. A detailed identification and prioritization of services that a business needs to develop or expose to support improved business processes must be developed..."
5. "...Evolving an SOA across the enterprise frees up IT resources and helps to ensure that investments in technology are focused on core capabilities aimed at growing the business..."
6. "...An SOA is a roadmap. It's a means to an end. What they are demanding is flexibility..."

Like what I have always been saying in my presentation and consulting rounds, and many-a-times conflicting to what a lot of people believe, when what they all want is just to implement XML and SOAP Services OR to be seen jumping on the SO(A) bandwagon so to look and seen to be in-tune with the times. Sometimes, people think they are implementing a SO(A) when all they are doing is just implement a SOAP interface to their data-element-layer. In other words, this isn't any different from implementing an n-tier architecture.

--- The goal of every business should be to build an agile business system framework to better adapt itself to its ever-changing business environments; And like what Mike mentioned in point [6] above, an SO(A) is the current best approach to build that agility.

Mike Champion has a great post here on the overheads of XML.

IMHO, I think a binary representation of XML brings on a whole different set of issues, namely removing the abstraction it is supposed to represent.

On a slightly different perspective and from a designing front, I have been advocating against the use of XML in all layers of a enterprise application esp. when tightly bound object technology is much more desired. In my presentations on SO(A), I have always preach on best using service-messaging as communication b/w applications, NOT between tiers of an application.

However, many businesses are using XML Services as a communication mechanism JUST SO they can be seen as implementing an SO(A)... and of course, for all the wrong reasons.

Hence, many of them complain when performance suffers.

What do they expect when they are making verbose, chatty calls to their own Data Access Layer via SOAP ?

I have noticed Rich Salz's absence for a rather long time and he comes back to give WSDL 2.0 a kick in the head.

Looks like I have to reserve my comments until I read through those specs.

Frankly though, I am rather disappointed with the handling of the later XML Services Specs by W3C and this is not just the beef I have to pick with W3C.

W3C's definition of SOA goes like this:

“A set of components which can be invoked and whose interface descriptions can be published and discovered”

Hell, why not they just say its XML Web Services ? This is just a plain crappy, poorly thought of definition.

And it contributes to all the market mis-definitions of _SO(A)_ where the terms SO(A) and XML Web Services, although tightly-linked, are NOT synonymous.

While a lot of people have been caught up with the recent advancements of Web Services Standards of WS-I, OASIS, W3C (the advanced communication standard protocols, the framework, etc), that is just the tip of the Web Services iceberg.

Once (not if) XML Web Services becomes the de-facto standards for cross domain communication for enterprise mission-critical connected distributed systems (EMCCDS) in the future, the focus will be on the IT Infrastructure. Why ? Well, someone or something has to keep it up and make sure it stays running ;) Messages of Service-Orientation are supposed to be self-contained and independent so the BUS must be there to make sure the dispatch doesnt fail.

Infrastructure-level Web services are XML services that implement part of the distributed computing infrastructure. They support other Web services communicate with each other. In particular, these robust services (and not necessarily WEB Services in any way) are the ones that makes or breaks the communication framework. They provide such functionality as:

• Provisioning
• Performance Managment (Message Payload, Response Time, etc)
• Operational Management
• Usage Metering
• Billing and Payments
• Routing (XML-Aware HW Appliances such as Routers and Firewalls, SW Routers)
• Orchestration (Workflows, Business Processes, etc)
• Advertisement and Discovery (WS-Discovery is key)
• Caching (Implementing different techniques and patterns for Caching, etc)
• Queuing (For added Reliability)
• State Management and Persistence (Transactional, 2-Phase Committ, Co-ordination, etc)

To understand this a bit further, now with more of a federated approach to services, there will be more applications and servers to enable the use of Identity Management, Authorization, Directory and other shared services. What follows next are supporting services and application hookups. Supporting services are especially important in crowded IT environments. “Retrofitting” has to be considered to improve extensibility (afterall, much money has been sunk in to create standards-based interfaces to reusable code). This puts further weight on the current infrastructure. Connectivity is very vital here as monolithic solutions are being cleaved apart to form intelligent shared services, then cleaved together again via connections to form a logical virtual composite system.

Very importantly, all these are very much tied to Service Level Agreements (SLAs) and Quality of Service (QoS) which will form the basis of the business returns and implications of Web Services in Service-Orientation.

Anne Thomas Manes, one the leading thought thinkers on Web Services, has an article that talks about Infrastructure Level Web Services here.

I am presently head-down and knee-deep working on writing a white paper that explains how XML Services and Service-Orientation will eventually empower the IT Infrastructure and some of the design considerations as we move forward to greater adoption of messaging-based solutions in the EMCCDS.

My previous blogs on Service-Orientation