Blog Home  Sign In RSS 2.0 Atom 1.0 CDF  

  def Softwaremaker() :
         return "William Tay", "<Challenging Conventions />"

  knownType_Serialize, about = Softwaremaker()
 

 Monday, February 25, 2008

I remember back in 2005/2006 when I was still touring the APAC circuits such as Sydney (Australia) and Kuala Lumpur (Malaysia) doing training and consulting gigs for customers, partners about Windows Workflow Foundation (WF) and Windows Communication Foundation (WCF, previously - Indigo) and some of the initial Windows Workflow questions came up regarding the use of Parallel Activities. It came as a surprise to many people that parallel activities are not independently asynchronous.

I explained that a WF instance gets only one instance from the runtime. There are reasons for this single-threaded execution model so each activity have to work with this single thread efficiently. There are ways to spin off differents thread when real parallelism activities are reqquired but because documentation was scare at that time, I had some trouble articulating how to do so.

I just read "Multithreaded Parallelism in Windows Workflow Foundation" on MSDN and while it is a definite deep technical article, if you can grok it, you will understand how "MultiThreaded Parallelism" can be done in WF using both the (rather hard-to-use) "Call External Method Activity (CEMA)" and the "Handle External Event Activity (HEMA)". Not only that, the authors (whom actually implemented such a system for their own use) also shared how to pair those 2 activities up using correlation and how to create wrappers aoround them so that it can be reused and therefore "not require talented software developer use of call-external-method and handle-external-event activities along with the CLR thread-pool"

A gem of a read.

Monday, February 25, 2008 8:44:37 AM (Malay Peninsula Standard Time, UTC+08:00)  #    Disclaimer  Comments [0]
  • Blog reactions

  •  Saturday, October 13, 2007

    I know I havent been posting deep technical stuff that I used to do. Contrary to what people think my current role entails, keeping abreast of the technology landscape is what I am supposed to do and what I enjoy and so when colleagues joked with me when was the last time I booted up Visual Studio, for example, I enjoyed seeing their shocked faces when I told them: "oh - just last night. why you asked ?" .

    I dont dwell deep like I used to but I still code up decent projects which I implement within my own developmental testing environment (yes, I have one running latest versions of Active Directory, SharePoint, Exchange and all the other goodies), driving the houshold crazy when I think of new and different ways to document expenses, publish a Book or CD library, home automation projects using all sorts of different technologies (yes, that includes Ruby-on-Rails) or in my new Windows Mobile 6 Device. Of course, I admit I dont post topics deep like I used to. It is not so much the content but more so, the limiting factor of time.

    Recently, I was involved in some internal technical discussions with regards to the issue of scale comparisions between Windows Communication Foundation (WCF, previously - Indigo) and ASMX. Below are some discussions:

    If you have a web service that is going to be IO bound, you would definitely want it to be scalable and almost every resource in the world tells you to implement ASP.NET asynchronous pattern (BeginSomething/EndSomething, etc) on it so to go easy on the thread pool. ASP.NET uses an IAsyncHttpHandler to handle the request, which means the worker threads are not blocked while the IO-bound operation executes somewhere else. Sounds good so far.

    If you make a WCF version of it with webHttpBinding (which actually means you can invoked it AJAX-style) following the same async pattern for the methods, you may find that each invocation of the WCF service eats up two threads – one for its ASP.NET HttpModule.ProcessRequest and the other for the actual IO. Ouch! You may think that this means your WCF implementation may end up eating all threads reserved for ASP.NET, which would indeed scale down the server

    Is this true OR are we missing the complete picture ?

    While the scenarios explained above are reasonable observations, it doesnt paint the complete picture. WCF does perform better scalability than ASMX.

    • Threading:
      For ASMX, when a request comes in, it would be queued up immediately for async ASMX. So the thread is released for that request and a new thread will pick up the work item later.

    For WCF, when a request comes in, we queue it up in WCF and let an IO thread handle the request. At the same time, the request thread-pool thread is held to wait for the request to complete.

    Yes, WCF uses more threads than async ASMX. But there is a reason for this. Using asynchronous ASMX is dangerous and not really a good practice (and I have hinted at this many times in the many Web Service/ASMX presentations I have done over the past few years). While it does well at what it is supposed to do, it does trick the developer into a "false sense of security". Essentially, if you dont know how the ASP.NET blackbox works, you may find yourself thrown against the car wall when you take a hidden, unsuspecting corner at high speeds. It does not provide enough throttling for client loads. Basically the server takes all items and queue them up for later processing. The server does not have a good throttling mechanism to control the number of work items. To everyone else, it seems that the server is quite friendly to all clients. However, if the number of clients is unbounded, this is really bad. First of all, the server working set would grow unlimited due to unlimited requests queued up. Secondly, many client requests would become obsolete when it’s picked up by the server from the queue. The latter accounts for a a good set of problematic scenarios I have come across in my past consulting gigs with regards to high-load and high-transactional ASMX asynchronous implementations before I joined the borg.

    Think of it as a side of the brain (that tells you that you are about to be full) not functioning properly when you sit down at a buffet table. You eat and eat and eat without knowning when to stop and then your ingestion/digestion system starts kicking in, you actually hit the wall. Hard. Literally.

    • Server Throughput
      When you measure scalability, the most important measurement is the server throughput. That is, how many requests the server can handle per time unit? For async ASMX, it would be pretty fast at the initial phase. However, like the ingestion/digestion analogy I was referring to above - Once the server is in a steady phase (as when CPU is fully loaded), the throughput will go down because the server capacity has reached. You can compare the data between async ASMX and sync ASMX over the long run to see what I mean.

    Also you would see higher memory usage of the async approach.

    • ASP.NET Throttling
      That said, ASP.NET does have a throttling mechanism that is used for sync ASMX, which is the threadpool thread limit. The number of threads used to handle requests are bounded (http://support.microsoft.com/kb/821268). WCF uses this fact to throttle incoming requests. You can always change the configuration settings to increase number of threads to be used to allow more work items to be queued up.

    The max number of threads follows the following formula:
    MaxWorkerThreads x #CPU – MinFreeThreads
    This is 12 by default on a single-proc machine.

    • Two-level Throttling for WCF
      WCF leverages the ASP.NET threadpool throttling to throttle client requests. At the same time, WCF has its own item queue throttling. The former is throttled by the setting mentioned in the immediate above point, while the latter is controlled by WCF throttling settings (maxConcurrentCalls etc). ASP.NET can automatically adjust threads based on CPU loads so that you would always get full load of the server.

    In this way, you may experience client failures because the requests are rejected at ASP.NET layer beforehand. So you can increase the ASP.NET throttling to get better experience. But eventually you would still be bounded by the physical server capacity, no matter whether you use async ASMX, sync ASMX, or WCF as mentioned above.

    There is improvement work done in .NET 3.0 SP1 and of course, .NET 3.5 (beta 2 here), with the use of prioritized item queues. Do expect even-better WCF performance even in some of the common scenarios. Fine tuning minWorkerThreads will even give us even better results.

    Thanks to Wenlong for helping out with the guidance and explanation. The complete scenario and the design principles for it will be published in greater detail in a MSDN whitepaper later. Do watch out for it.

    Friday, October 12, 2007 10:40:28 PM (Malay Peninsula Standard Time, UTC+08:00)  #    Disclaimer  Comments [0]
  • Blog reactions

  •  Saturday, August 18, 2007

    MSFT Corp has always talked about its ecosystem with regards to its developers and partners as one of its key value propositions which it offers through its platform to its customers.

    Sometimes it is humbling to have customere call on you because its other systems can only work with yours and no one else. I have totally lost track of our ISV partners who are building great vertical solutions on top of our horizontal application stack, such as Microsoft Office SharePoint Server (MOSS) and others and that list is growing as I write this.

    This is another take on it, albeit from another interesting perspective.

    Friday, August 17, 2007 10:19:05 PM (Malay Peninsula Standard Time, UTC+08:00)  #    Disclaimer  Comments [0]
  • Blog reactions

  •  Wednesday, March 07, 2007
    Wednesday, March 07, 2007 2:12:58 PM (Malay Peninsula Standard Time, UTC+08:00)  #    Disclaimer  Comments [0]
  • Blog reactions

  •  Friday, August 11, 2006

    Many people have asked how we can retrieve and process a token sent by a subject or a Security Token Service. Just to showcase the non-intrusiveness of Microsoft Cardspace, in that we can easily incorporate a Cardspace "Sign-in with Cardspace" into your website today such as this or this, I have shown snippets of how we can do so via php here.

    As an another example of non-intrusiveness, I have also got a simple working sample that you can use to integrate into your ASP.NET 2.0 page today.

    Of course, this is a hack job and there are no try catch and the xml parsing code is hastily done. To summarize, I wont win the elegance award here  ... but you can get it to work on .NET 2.0 (or maybe even 1.1) ...

    Of course, this is a hack job and there are no try catch and the xml parsing code is hastily done. To summarize, I wont win the elegance award here  ... but you can get it to work on .NET 2.0 (or maybe even 1.1) ...

        [BEGIN]
        'This will get you your encrypted token
        Dim str As Stream
        str = Request.InputStream
        Dim sr As New StreamReader(str)
        Dim s As String = sr.ReadToEnd
        s = Server.HtmlDecode(Server.UrlDecode(s))

        'the stream will be ASCII encoded
        Dim u8 As System.Text.UTF8Encoding = New System.Text.UTF8Encoding

        Dim fs As New FileStream("C:\temp\CardspaceXMLToken.xml", FileMode.Create)
        'Remove the "xmlToken="
        Dim i As Integer = s.IndexOf("<")
        s = s.Substring(i)
        fs.Write(u8.GetBytes(s), 0, u8.GetBytes(s).Length)
        fs.Flush()
        fs.Close()
        '--------------------------------
        'The will decrypt that request token and return a Signed Security SAML Token
        Dim encryptedDoc As New XmlDocument()
        encryptedDoc.Load("C:\temp\CardspaceXMLToken.xml")

        'Check for <EncryptedData> element
        If encryptedDoc.GetElementsByTagName _
        ("EncryptedData", "
    http://www.w3.org/2001/04/xmlenc#").Count = 0 Then Return

        Dim encryptedD As XmlElement = CType(encryptedDoc.GetElementsByTagName _
        ("EncryptedData", "
    http://www.w3.org/2001/04/xmlenc#")(0), XmlElement)

        'Check for @Algorithm attribute and make sure its http://www.w3.org/2001/04/xmlenc#aes256-cbc
        If encryptedD.ChildNodes(0).Attributes("Algorithm") Is Nothing Or _
        encryptedD.ChildNodes(0).Attributes("Algorithm").Value <> _
        "
    http://www.w3.org/2001/04/xmlenc#aes256-cbc" Then Return

        Dim a As New RijndaelManaged

        'Check for <KeyInfo> element
        If encryptedD.ChildNodes(1).Name <> "KeyInfo" Then Return

        'Check for <EncryptionMethod> element
        Dim encryptedM As XmlElement = CType(encryptedDoc.GetElementsByTagName _
        ("EncryptionMethod", "
    http://www.w3.org/2001/04/xmlenc#")(1), XmlElement)

        'Check for @Algorithm attribute and make sure its http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p
        If encryptedM.Attributes("Algorithm") Is Nothing Or _
        encryptedM.Attributes("Algorithm").Value <> _
        "
    http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p" Then Return

        'Check for <CipherData\CipherValue>
        If encryptedD.GetElementsByTagName("CipherValue", _
        "
    http://www.w3.org/2001/04/xmlenc#").Count = 0 Then Return

        'Retrieving the Encrypted Symmetric Key and the Security Token
        Dim e_SymmKey As String = encryptedDoc.GetElementsByTagName _
        ("CipherValue", "
    http://www.w3.org/2001/04/xmlenc#")(0).InnerText
        Dim e_SecurityToken As String = encryptedDoc.GetElementsByTagName _
        ("CipherValue", "
    http://www.w3.org/2001/04/xmlenc#")(1).InnerText

        Dim rsaCsp As New RSACryptoServiceProvider
        Dim privatecert As New X509Certificates.X509Certificate2
        privatecert.Import("Softwaremaker.NET.SSL.pfx", "Softwaremaker_NET_SSL_pfx_password",     X509Certificates.X509KeyStorageFlags.Exportable)
        rsaCsp.FromXmlString(privatecert.PrivateKey.ToXmlString(True))

        'Now we have the Symmetric Key
        Dim DecryptedStrAsByt() As Byte = _
        rsaCsp.Decrypt(System.Convert.FromBase64String(e_SymmKey), True)

        'Moving on to use the Symmetric Key to decrypt Token
        Dim rijndaelM As New RijndaelManaged
        rijndaelM.Mode = CipherMode.CBC

        Dim swmDecryptor As ICryptoTransform
        swmDecryptor = rijndaelM.CreateDecryptor(DecryptedStrAsByt, Nothing)

        'Define memory stream which will be used to hold encrypted data.
        Dim cipherTextBytes As Byte()
        cipherTextBytes = System.Convert.FromBase64String(e_SecurityToken)

        Dim memoryStream As MemoryStream
        memoryStream = New MemoryStream(cipherTextBytes)

        ' Define memory stream which will be used to hold encrypted data.
        Dim cryptoStream As CryptoStream
        cryptoStream = New CryptoStream(memoryStream, swmDecryptor, CryptoStreamMode.Read)

        ' Since at this point we don't know what the size of decrypted data
        ' will be, allocate the buffer long enough to hold ciphertext;
        ' plaintext is never longer than ciphertext.
        Dim plainTextBytes As Byte()
        ReDim plainTextBytes(cipherTextBytes.Length)

        ' Start decrypting.
        Dim decryptedByteCount As Integer
        decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length)

        ' Close both streams.
        memoryStream.Close()
        cryptoStream.Close()

        ' Convert decrypted data into a string.
        ' Let us assume that the original plaintext string was UTF8-encoded.
        Dim plainTextToken As String
        plainTextToken = System.Text.Encoding.UTF8.GetString(plainTextBytes, 16, _
                                            decryptedByteCount - 16)

        return plainTextToken
        [END]
        '--------------------------------

    Of course, you would have to verify the signed token yourself BUT this decryption process is definitely a step in the right direction

    If you have benefitted from these sample snippets, maybe you can pay-it-forward by implementing and then sharing the signature-verification codes with me here? I am too lazy ... .

     

    Friday, August 11, 2006 6:18:49 AM (Malay Peninsula Standard Time, UTC+08:00)  #    Disclaimer  Comments [0]
  • Blog reactions

  •  Wednesday, March 15, 2006

    Ar - Arguments...It does make us move forward at times.

    Richard Grimes has managed to kick up a storm again with his article here, again. While, I would not go very far in saying a kernel Operating System should be written in managed-code. God knows I will not use one if it is and you shouldnt to. As far as I can tell, .NET was not created for writing operating systems?  It sits on top of the operating system and thats that

    It is, however, very important to note, the investments MSFT Corp have on managed code. Instead of giving you the usual bullets and docs. How about this ?

    Lines of Managed Code

    And lets not forget that Microsoft CRM is the first Enterprise Business Solutions that is written on managed code from Microsoft. Read: Dogfood

    And YES - Windows Communication Foundation (WCF, previously - Indigo) is written in C# as well.

    Case Closed. <EOM>

     

    Tuesday, March 14, 2006 11:44:31 PM (Malay Peninsula Standard Time, UTC+08:00)  #    Disclaimer  Comments [0]
  • Blog reactions

  •  Friday, January 20, 2006