BizTalk Community series: Introducing Oliver Hauth

Recently I had a conversation through email with Oliver Hauth, the person who as a BizTalk community member I like to bring to the foreground in this blog post. He has been awarded Microsoft Valuable Professional little over a year ago. I met Oliver in Berlin during the TechEd Europe 2010 and we'll probably meet again during the upcoming TechEd Europe 2012 in Amsterdam. He is a passionate IT professional, BizTalk forum member and fun guy to hang out with. Here is his story.

Oliver is 32 years old and lives in Germany. He is married and with his wife has a two year old daughter. They are expecting their second child in September. He is the team coordinator and lead architect, responsible for the area of Business Process Integration and Workflow Solutions at MAN Diesel & Turbo SE. His job covers mostly BizTalk Server, SharePoint and Nintex Workflow, SQL Server Reporting Services, SQL Server Integration Services and Forefront Identity Management.

Oliver was introduced to BizTalk Server at MAN Diesel & Turbo SE a few years ago. He started as a lightweight developer, trying to understand the system. From that moment he started focusing more to be a landscape and solution architect for distributed applications and cloud solutions in general. One of his key skills is finding the BizTalk based solution for the more complex scenarios (dynamic routing, property / content based routing , multi-part messages, filter criteria).

Oliver’s view of BizTalk is and I quote:

“The biggest advantage of BizTalk server is the great tooling and monitoring toolkit. It delivers the stable basis we need for our daily work in the enterprise as well as the flexibility to connect to a wide range of systems. With the newer versions and the enhancements of WCF adapters and Azure bindings it integrates much better in the new .NET world and is shows that it’s built to integrate with current cloud solutions. Furthermore it’s fun to develop. When starting with BizTalk you do not really understand how the tool works, but once you’re into it you can impress a lot of people with how quickly and flexible you can build integration processes.”

Oliver spends most of his time with his family; trying to spend at least the evening with his daughter and wife. Besides that he has just started building their new house, which will consume a lot of time during the next 12 months. The remaining (few) hours he normally uses to have a look at new technologies, listen to music (from what he saw at Facebook we have a very similar taste) or play some video games. From time to time Oliver reads a non-technical book (mostly when he’s on holiday).

Oliver started doing some Wing-Tsung with two colleagues. Furthermore he likes watching NFL / Football games if  he can find a German TV channel showing these games (not very popular here). He also plans to reactivate his skiing activity in the future – if he can find the time to do so.

Oliver’s final quote:

“Keep reading this blog – Steef-Jan is doing an amazing job. Also keep in mind: BizTalk’s not dead. Have a look at Azure and AppFabric but do not give up BizTalk too early. It’s still one of the strongest tools on the market.
Especially to your German readers: If you use BizTalk please start participating in the community, we need you!”

I like to thank Oliver for his contributions to the BizTalk community and time to have a chat with me.I will probably meet him again in June during TechEd Europe.

Help my BizTalk databases are growing!

The BizTalk Event in Italy was a blast and I had a great time. Today a day after I arrived home I got intrigued by a scenario on the BizTalk forums. The scenario described a situation that BizTalk databases are growing larger even though the BizTalk database jobs are running correctly (see also database script on MSDN Code Gallery). The execution of the BizTalk Server SQL Agent jobs are crucial for managing the BizTalk Server databases and for maintaining optimal performance. Experienced BizTalk professionals know that all the BizTalk SQL Server Agent jobs except the MessageBox_Message_Cleanup_BizTalkMsgBoxDb job should be enabled and running successfully. Besides this job the other jobs should not be disabled!
By default, the following BizTalk jobs aren’t configured and enabled upon installation.

• Backup BizTalk Server (BizTalkMgmtDb)
• DTA Purge and Archive (BizTalkDTADb)
• MessageBox_Message_Cleanup_BizTalkMsgBoxDb

So as a BizTalk pro you need to configure these according to your requirements for backups, and purging. To configure the first two jobs, see this post from fellow MVP Sandro Pereira where you will find enough detail and background information. The last job is started by the MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb job. Therefore, it is recommended that you disable this job.

Now having the jobs running properly is one thing yet there can be more causes to uncontrolled growth of the databases like:

• Excessive suspended message or service instances
• Disk failures
• High levels of tracking
• BizTalk Server throttling
• Poor SQL Server performance
• Network latency issues

Below you will find more detail on probable causes of uncontrolled growth of the BizTalk databases:

Excessive suspended message or service instances

Service instances can be suspended (resumable) or suspended (not resumable). These service instances may be Messaging, Orchestration, or Port. A proliferation of suspended messages and service instances may cause the BizTalk Server Messagebox database to increase in size beyond what would be expected during normal operations.
BizTalk Server accommodates termination and removal of these instances by using the Group Hub page in the BizTalk Server Administration Console or through the use of the Terminate.vbs script. Through group hub page you can remove them all or selectively (see this post by fellow MVP Ben Cline).

Zombies or orphaned message can be another cause for increase of suspended messages of service instances (this script from Tord G. Nordahl can help you find information on these kind of messages). An orphaned or zombie message is a message that does not have an associated service instance, typically because the service instance has terminated before the message was received. An orphaned or zombie service is a service that does not have any associated messages. Again the terminate script can resolve this issue.

Disk failures

Important for SQL Server is the disk configuration. A wrong configuration could easily lead to disk failures. If one disk fails the disk contention on the other can increase (see also this post). Failure can be related to I/O errors (which cannot be identified through the event log). This can be prevented by performing a disk I/O simulation using SQLIOSim. To prevent disk failures is one thing, but how do identify them. You can look into SQL Server error logs, or you can use SCOM to monitor for failures.

High levels of tracking

By default, tracking is enabled on the default host. BizTalk requires the Allow Host Tracking option be checked on a single HOST. When tracking is enabled, the Tracking Data Decode Service (TDDS) moves the tracking event data from the BizTalk Server MessageBox database to the BizTalk Server tracking database. If no BizTalk Server hosts are configured with the option to Allow Host Tracking or if the tracking host is stopped, then TDDS will not run and the TrackingData_x_x tables in the BizTalk Server MessageBox database will grow unchecked. Therefore, a dedicated BizTalk Server host should be configured with the option to Allow Host Tracking (see also Configuring a Dedicated Tracking Host).

BizTalk Server throttling

There are lot of scenarios because of which Host throttling can occur. It can be because of Low Process Memory, high database size,high thread count and some more. Now the BizTalk Server host throttling mechanism continually monitors for a throttling condition, calculates the severity of the throttling condition, and applies host throttling progressively depending on the calculated severity. The throttling mechanism moderates the workload of the host instance to ensure that the workload does not exceed the capacity of the host instance or any downstream host instances.
Throttling protects BizTalk from completely shutting down. High database size can cause throttling and is an undesired effect. Growth of databases can be caused by fact that more messages are coming into BizTalk than going out (see post by Tord G. Nordahl). To mitigate this problem, see section Throttling condition triggers, actions, and mitigation strategies of How BizTalk Server Implements Host Throttling.

Poor SQL Server Performance

BizTalk Server makes a large number of short, very quick transactions to SQL Server within one minute. If the SQL Server cannot sustain this activity, you may experience BizTalk Server performance issues. Monitor the Avg. Disk sec/Read, Avg. Disk sec/Transfer, and Avg. Disk sec/Write performance monitor counters in the PhysicalDisk performance object. The optimal value is less than 10 ms (milliseconds). A value of 20 ms or larger is considered poor performance. You can use Windows Performance Monitor to monitor these counters. With the BizTalk Server 2010 Performance Optimization Guide that contains prescriptive guidance for optimizing BizTalk Server performance you can try to resolve poor performance.

Network latency issues

SQL Server is one of those few applications that is very sensitive to disk and network latency. If your SQL Server is waiting around for disk or network they will start to complain. To mitigate the impact of network latency on the database throughput, you should consider using gigabit network cards and increase of network bandwidth.
Some of the information found here is from MSDN, TechNet Wiki (i.e. Database Survival Guide) and blogs from fellow MVP’s and BizTalk community members. The health of the BizTalk Server databases is critical for having a stable, robust and sustainable BizTalk infrastructure. There are also some tools that can help in identifying the problem for having uncontrolled growth of the BizTalk databases.

The MsgBoxViewer tool is useful for troubleshooting, because it provides an HTML report that has detailed information about table sizes and the row count. The report can also help determine whether BizTalk Server is throttling. Additionally, the tool provides a snapshot of the BizTalk Server databases and the BizTalk Server configuration. When BizTalk Server is running slower than usual, run the MsgBoxViewer tool, and then review the generated HTML report for any problems. The Summary section lists warnings in yellow and potential problems in red (see section Troubleshooting of How to maintain and troubleshoot BizTalk Server databases).

You can also check you BizTalk deployment by using the BizTalk Server Best Practices Analyzer V1.2. This tool examines a BizTalk Server 2006, BizTalk Server 2006 R2, or BizTalk Server 2010 deployment and generates a list of issues pertaining to best practices standards for BizTalk Server deployments. There are other tools available that can help you test you SQL Server environment and able to tune it, see SQLCrunch.

There are a lot of factors that can cause BizTalk databases to grow and result in a poor health of your overall BizTalk environment. With the information in this post I hope you have a number of tools to either prevent or solve this issue.

Cheers!

- Steef-Jan

OVERNET BizTalk Innovation Event | May 24, 2012 – Milan, Italy

Next week there will be BizTalk event in Milan. Overnet Education in collaboration with Microsoft Italy and User Group Italiano Connected System will organize, for the first time an interesting event related to BizTalk Server and integration services with the presence of international speakers. Myself, Sandro, Tord and Saravana will speak at this event.

Event is organized by Nino Crudele in collaboration with me and few others. Nino is the founder of the User Group Italiano Connected System and will be our host. I would like to invite you all to join us next May 24, 2012 in Milan, Italy.

The agenda for the event:

13:45 – 14:00: Registration;

14:00 – 14:15: Welcome;

14:15 – 15:00: BizTalk Host thresholds and automatic throttling
by Tord Glad Nordahl;

15:00 – 15:45: BizTalk360
by Saravana Kumar (MVP BizTalk Server)

15:45 – 16:00: Break;

16:00 – 16:45: Adapter Pack Integration Capabilities (BizTalk)
by Steef-Jan Wiggers (MVP BizTalk Server)

16:45 – 17:30: Introduction to the Azure Service Bus EAI/EDI features
by Sandro Pereira (MVP BizTalk Server)

17:30 – 18:00: BizTalk Innovation Event Wrap up
by Nino Crudele (MVP BizTalk Server)

18:00: Closure

Click here to ensure your registration or to get more information’s about this event.

BizTalk Community series: Introducing Sriram Chidambaram Narayanan

It has been a few weeks since my last post. Why, well I have been busy promoting my BizTalk Server 2010 Cookbook, which was published last month. I recommend you to buy a copy (eBook and/or printed version) if you haven’t done so already. A few weeks ago, just before the book was published, I had a conversation through email with Sriram, the person who as a BizTalk community member I like to bring to the foreground now. He has recently been awarded Microsoft Community Contributor. Well done and here is his story.

Sriram Chidambaram Narayanan is 27 years old professional from Chennai located in the southern part of India (State: TamilNadu). He works as a Senior Application Architect for Computer Sciences Corporation(CSC) in Bangalore. Sriram was at Unum for 14 months in the United States and returned back to India a couple of months ago.

Sriram has a total of 6 years of BizTalk experience.He started his career with developing and gradually evolved to a role where he is designing and architecting integration solutions. During his career he started with experimenting and creating connections with BizTalk Server to different applications. More recently his interest shifted towards learning the concepts and technology behind the ESB toolkit and Windows Azure AppFabric Service Bus (now renamed to Service Bus Connect). These are currently the two areas where he is focusing on.

Sriram loves the BizTalk product because I quoted:

• its a fresh approach for designing Business flows and connections through built in shapes and configurations instead of the traditional 20000 lines of code
• provides a clear view on everything that flows through the system via various tracking options and monitoring tools
• it has and is maturing as a product with various enhancements and accelerators since the time I started working on the 2004 version
• it also has many interesting concepts, designs and logics that one has to think on opposed to just configure type products
• it never stopped challenging me and I never stopped dissolving them to feel the exhilaration.

His enthusiasm for the product shows through his contributions on the BizTalk forums (profile) and other channels. Whenever Sriram gets time outside of his regular work he will spend time on the forums. He enjoys doing this as it introduces him to different issues and solutions, which otherwise would take lots of time for him to personally explore. He also maintains a blog, where he shares his experience and knowledge or some times writes a post on issue resolutions that might help others. He also contributes to Codit's(http://www.codit.eu) technical blogs. To the readers of his personal blog he would like to say:

“Show love, keep learning, be strong and don’t compromise on your goals.Keep a rockin’ attitude!”

Beside blogging Sriram has recently written his first article for the TechNet Wiki.So we have another BizTalk community member as contributor.

In India, cricket is a religion and Sriram is a great fan of Sachin Tendulkar. He support his home team, Chennai Super Kings and enjoys playing tennis once and a while. Besides sports Sriram enjoys listening to music especially to A. R. Rahman. Furthermore he regularly visits his family every weekend and now and then he spends some time with his friends.

Finally Sriram has a word for me and this blog:

“Steef is a great inspiration to all new BizTalk and Integration professionals and his contributions in the forums and TechNet articles are tremendous. I would thank and appreciate Steef for all his contributions and for keeping the ball rolling.Nandri!(Thank you)”

Thanks Sriram for your time and your contributions to forums, your blog and now also the TechNet Wiki.