How do I handle message failure in MSMQ bindings for WCF
I have create a WCF service and am utilising netMsmqBinding binding.
This is a simple service that passes a Dto to my service method and does not expect a response. The message is placed in an MSMQ, and once picked up inserted into a database.
What is the best method to make sure no data is being lost.
I have tried the 2 following methods:
Throw an exception
This places the message in a dead letter queue for manual perusal. I can process this when my strvice starts
set the receiveRetryCount="3" on the binding
After 3 tries - which happen instantanously, this seems to leave the message in queue, but fault my service. Restarting my service repeats this process.
Ideally I would like to do the follow:
Try process the message
- If this fails, wait 5 minutes for that message and try again.
- If that process fails 3 times, move the message to a dead letter queue.
- Restarting the service will push all messages from the dead letter queue back into the queue so that it can be processed.
Can I achieve this? If so how? Can you point me to any good articles on how best to utilize WCF and MSMQ for my given sceneria.
Any help would be much appreciated. Thanks!
Some additional information
I am using MSMQ 3.0 on Windows XP and Windows Server 2003. Unfortunately I can't use the built in poison message support targeted at MSMQ 4.0 and Vista/2008.
There's a sample in the SDK that might be useful in your case. Basically, what it does is attach an IErrorHandler implementation to your service that will catch the error when WCF declares the message to be "poison" (i.e. when all configured retries have been exhausted). What the sample does is move the message to another queue and then restart the ServiceHost associated with the message (since it will have faulted when the poison message was found).
It's not a very pretty sample, but it can be useful. There are a couple of limitations, though:
1- If you have multiple endpoints associated with your service (i.e. exposed through several queues), there's no way to know which queue the poison message arrived in. If you only have a single queue, this won't be a problem. I haven't seen any official workaround for this, but I've experimented with one possible alternative which I've documented here: http://winterdom.com/weblog/2008/05/27/NetMSMQAndPoisonMessages.aspx
2- Once the problem message is moved to another queue, it becomes your responsibility, so it's up to you to move it back to the processing queue once the timeout is done (or attach a new service to that queue to handle it).
To be honest, in either case, you're looking at some "manual" work here that WCF just doesn't cover on it's own.
I've been recently working on a different project where I have a requirement to explicitly control how often retries happen, and my current solution was to create a set of retry queues and manually move messages between the retry queues and the main processing queue based on a set of timers and some heuristics, just using the raw System.Messaging stuff to handle the MSMQ queues. It seems to work pretty nicely, though there are a couple of gotchas if you go this way.
I think with MSMQ (avaiable only on Vista) you might be able to to do like this:
<bindings> <netMsmqBinding> <binding name="PosionMessageHandling" receiveRetryCount="3" retryCycleDelay="00:05:00" maxRetryCycles="3" receiveErrorHandling="Move" /> </netMsmqBinding> </bindings>
WCF will immediately retry for ReceiveRetryCount times after the first call failure. After the batch has failed the message is moved to the retry queue. After a delay of RetryCycleDelay minute, the message moved from the retry queue to the endpoint queue and the batch is retried. This will be repeated MaxRetryCycle time. If all that fails the message is handled according to receiveErrorHandling which can be move (to poison queue), reject, drop or fault
By the way a good text about WCF and MSMQ is the chapther 9 of Progammig WCF book from Juval Lowy