Doctor Deploys Blog

Almost 2 years ago I asked for feedback on your experience deploying .NET applications and using the .NET Framework deployment packages (Do you deploy a managed App).

We took a lot of the feedback from my blog and various other places and implemented features in .NET 4.0. Based on the feedback, we focused on size, performance, and robustness which I also recently posted about (The .NET Framework 4 Installer Improvements).

I’m really interested in getting feedback from everyone on how we did and get ideas for what we can do to make the experience better in the next release. We definitely take every bit of feedback seriously and value it highly and I want to say I always appreciate the time people spend to respond. Feel free to answer as many of these questions as you are compelled to.

Questions:

1. Have you decided to update your application to .NET 4? Was deployment a factor in that decision?

2. Did .NET Framework 4 solve all of your deployment problems? Did you like it?

3. Where did we miss with .NET Framework 4 deployment? What didn’t you like?

4. What do you think about the .NET Framework 4 Client Profile option?

5. Did you find all the documentation and examples you needed to be able to use .NET 4?

6. What do you think about the new size of the .NET package?

7. Do you know about the redist and web installers? If so, which do you use?

8. Do you use a bootrapper/chainer that preinstalls .NET? If not, do you block? If so,

- Which one (VSI, InstallShield, Wise, ClickOnce, custom)?

- Do you run .NET in silent mode or UI mode?

9. Do you have any specific problems you can tell me about that you have had in deploying the .NET Framework?

Some more question that will help me understand your scenario:

10. How large is your application? (Size, install base)

11. How is your product deployed (Web download, CD, DVD, USB)?

Optional:

12. What company are you with?

13. What is your application?

14. Can I follow up with you? If so, email me your contact info along with your answers

As always. I very much appreciate getting everyone’s feedback. Hope to hear from many of you.

Original post by Peter Marcu

Much of this is a repost of my Beta 2 post with updated content and data.

Thanks to everyone that gave feedback both on my blog and through other forums about .NET Framework installs. .NET 4 just released so I thought I’d take this opportunity to talk a little about the stuff we have done to improve the installation. My team and I have been focused over the past year on incorporating feedback and striving to make the installer better for the .NET Framework 4. There has been a particular focus on making it better for client applications to install it with their apps.

The key focus areas for the .NET 4 installer have been Size, Robustness, and Performance. I’ll speak to some of the major things we did and give a brief description below.

Size

You can see from the chart below that we have reduced the size of the .Net redist significantly. It went from client applications needing to carry 237MB to 41MB for the client profile. This is a game changer for a lot of client applications.

Size Comparison of the redist size since 3.5 SP1

Better Compression Across Packages

We implemented the use of a better compression technology into our packages which reduced the size of our packages by around 15%.

Separate packages for AMD64 and IA64

We found that there was little to no need to ever install the same package on both amd64 and ia64. Because of this, we decided to produce amd64 packages that excluded ia64 binaries as well as ia64 packages that didn’t contain amd64 binaries.

Client Profile

We determined the subset of framework functionality that was used by 95+% of client applications and produced a first class package for this scenario. The result of this is that, unless you are taking advantage of features such as ASP.NET, you can now take a dependency on a smaller framework. More details of what is in the client profile can be found here.

Remove Duplicate MSIL

We identified many assemblies that were functionally identical but differed by the architecture they were built under. These assemblies were all managed CPU neutral assemblies which meant that it didn’t matter whether they were built for x86 or amd64. Their strong names and functionality are the same. We solved this by only carrying one of them.

Robustness

The numbers are already starting to come in from the RTM release and we are very happy with the success rates of the install. At this point we are seeing greater than a 99% success rate which is much better than previous .NET redist releases right out of the gate. Here are some of the larger things we did to increase robustness.

Remove Prerequisites

In a chain of installs, the chain is only as strong as its weakest link. In addition, small weaknesses in each part of the chain compound to lead to higher failure rates for the whole chain. By removing numerous prerequisites and combining the whole client install into a single MSI, we were able to get rid of the compounding effect of failures as well as focus our efforts on making the single MSI as solid as possible.

Simplify the MSI

Custom actions are very common places for installs to fail. The more you have, the more complex the installer gets and the number of points of failure goes up. Removing the need for customactions in many cases and in the cases where we needed them, simplifying them has increased our success rates.

Remove slipstreamed feature MSP’s

In Beta1, we slipstreamed features into the installer’s msi using patches. This proved to be a point of complexity and the root cause of many unsolvable bugs. Due to that, we simplified our install to be completely contained in a single msi per platform.

Fix and Retry

Through thorough investigation of our past installers, looking through KB articles, feedback from customers, and through our past Beta’s, we found numerous install failure conditions that were fixable after the error. We implemented the KB articles and other workarounds so, in failure cases, we can fix the users machine and try again. We’ve seen quite an increase in our success rates due to this work. My hope is that this will also make the windows installer ecosystem cleaner and that msi’s installed after .NET 4 will have a better chance to succeed because our installer put the machine in a better state.

Triple fallback on Download failures

Through analysis of our download failures in the past, we determined that using a single implementation for downloads left you only as successful as that technology allows. We found that between Winhttp, URLMon, and BITS, their failures were in different scenarios and where one would fail, the others would succeed. In order to take advantage of this, our chainer falls back and retries on different download stacks to do everything we can to get a successful download.

Separated out Server configuration from Client Profile

The Client Profile installer should be more robust for applications now because some of the most common failing custom actions in .NET 3.5 were in configuring things like ASP.NET and WCF which are mainly server scenarios and not used by client applications. By moving these to the full install, we are seeing higher success rates for the client install.

Performance

The key metric here is overall install time for the redist. In 3.5 SP1 the average install time was 12-15 minutes. With v4, the average times are closer to 3-5 minutes for the client profile and 5-7 minutes for Full.

Smart Cabbing

Smart cabbing is a technique used to allow you to install the same file to multiple locations but only carry the file once in the msi’s cabs. This technique has been used for years but during our perf investigations, we found that, depending on how many duplicate files there were and where they were in the cab, performance degraded significantly. We made some bug fixes in the tools we use to smart cab (WiX) to reduce the impact of duplicated items while still gaining the benefits of smart cabbing.

Remove Prerequisites

This one is fairly self explanatory. We need to install less packages so we are faster. This is mostly the result of changing the .NET Framework itself to not have certain dependencies or carry subsets of the dependencies within the framework. In a few cases, this was possible because the base functionality was either built into all the supported OS’s or had enough ubiquity in the ecosystem to not warrant us carrying it.

Remove Slipstreamed Msp’s

We found that when applying large slipstreams to a product, there was a significant perf hit towards the end of the install when Windows Installer is caching the packages for source resiliency. By adding all the features into the MSI, we got rid of this performance hit.

Parallel Ngen and removal of synchronous 64bit assemblies

The CLR implemented the ability to ngen on multiple cores in parallel. We made changes in our installer to take advantage of this so now on a multicore machine, ngen times should be significantly reduced. Also, on 64bit machines, most .NET applications run as 32bit. This means that paying the price of creating 64bit native images is not something most apps need to do.

Client Profile

By producing a subset of the .NET install that contains the features most client applications need, most client applications can take advantage of shorter install times by installing less.

Parallel Download and Install

If you are using the web bootstrapper which we made available for the first time in Beta2, you can use the web bootstrapper to install .NET Framework 4. This has the advantage of downloading and installing the payload in parallel. For example, as it is installing the Client Profile, it will be downloading the rest of the framework. In cases where you have enough bandwidth to download the rest before the Client Profile install finishes, you essentially save the time it took to download the rest.

Original post by Petermarcu

Thanks to everyone that gave feedback both on my blog and through other forums about .NET Framework installs. Beta2 just got released on MSDN so I thought I’d take this opportunity to talk a little but about the stuff we have done to improve the installation. My team and I have been focused over the past year on incorporating feedback and striving to make the installer better for the .NET Framework 4.0 . There has been a particular focus on making it better for client applications to install it with their apps.

The key focus areas for the .NET 4.0 installer have been Size, Robustness, and Performance. I’ll speak to some of the major things we did and give a brief description.

Size

Comparison Chart for Size improvements since 3.5 SP1

Better Compression Across Packages

We implemented the use of a better compression technology into our packages which reduced the size of our packages by around 15%.

Separate packages for AMD64 and IA64

We found that there was little to no need to ever install the same package on both amd64 and ia64. Because of this, we decided to produce amd64 packages that excluded ia64 binaries as well as ia64 packages that didn’t contain amd64 binaries.

Client Profile

We determined the subset of framework functionality that was used by 95+% of client applications and produced a first class package for this scenario. The result of this is that, unless you are taking advantage of features such as ASP.NET, you can now take a dependency on a smaller framework. More details of what is in the client profile can be found here <Will provide link once it is posted>.

Remove Duplicate MSIL

We identified many assemblies that were functionally identical but differed by the architecture they were built under. These assemblies were all managed CPU neutral assemblies which meant that it didn’t matter whether they were built for x86 or amd64. Their strong names and functionality are the same. We solved this by only carrying one of them.

Robustness

Remove Prerequisites

In a chain of installs, the chain is only as strong as its weakest link. In addition, small weaknesses in each part of the chain compound to lead to higher failure rates for the whole chain. By removing numerous prerequisites and combining the whole client install into a single MSI, we were able to get rid of the compounding effect of failures as well as focus our efforts on making the single MSI as solid as possible.

Simplify the MSI

Custom actions are very common places for installs to fail. The more you have, the more complex the installer gets and the number of points of failure goes up. Removing the need for customactions in many cases and in the cases where we needed them, simplifying them has increased our success rates.

Remove slipstreamed feature MSP’s

In Beta1, we slipstreamed features into the installer’s msi using patches. This proved to be a point of complexity and the root cause of many unsolvable bugs. Due to that, we simplified our install to be completely contained in a single msi per platform.

Fix and Retry

Through thorough investigation of our past installers, looking through KB articles, feedback from customers, and through our past Beta’s, we found numerous install failure conditions that were fixable after the error. We implemented the KB articles and other workarounds so, in failure cases, we can fix the users machine and try again. We’ve seen quite and increase in our success rates due to this feature. My hope is that this will also make the windows installer ecosystem cleaner and that msi’s installed after .NET 4.0 will have a better chance to succeed because our installer put the machine in a better state.

Triple fallback on Download failures

Through analysis of our download failures in the past, we determined that using a single implementation for downloads left you only as successful as that technology allows. We found that between Winhttp, URLMon, and BITS, their failures were in different scenarios and where one would fail, the others would succeed. In order to take advantage of this, our chainer falls back and retries on different download stacks to do everything we can to get a successful download.

Separate out server configuration from Client Profile

The Client Profile installer should be more robust for applications now because some of the most common failing custom actions in .NET 3.5 were in configuring things like ASP.NET and WCF which are mainly server scenarios and not used by client applications. By moving these to the full install, we are seeing higher success rates for the client install.

New Chainer

We implemented a new chainer in Dev10 that is being used by 4.0. It is smaller, faster, and more flexible than its predecessor. The flexibility is a major part of what allowed us to make investments such as the download fallbacks, Fix and retry, and better compression. I’m excited that this chainer is the foundation of what is being built into the WiX community under the name of Burn. This should be available for others to use in the WiX 3.5 release.

Performance

Smart Cabbing

Smart cabbing is a technique used to allow you to install the same file to multiple locations but only carry the file once in the msi’s cabs. This technique has been used for years but during our perf investigations, we found that, depending on how many duplicate files there were and where they were in the cab, performance degraded significantly. We made some bug fixes in the tools we use to smart cab (WiX) to reduce the impact of duplicated items while still gaining the benefits of smart cabbing.

Remove Prerequisites

This one is fairly self explanatory. We need to install less packages so we are faster. This is mostly the result of changing the .NET Framework itself to not have certain dependencies or carry subsets of the dependencies within the framework. In a few cases, this was possible because the base functionality was either built into all the supported OS’s or had enough ubiquity in the ecosystem to not warrant us carrying it.

Remove Slipstreamed Msp’s

We found that when applying large slipstreams to a product, there was a significant perf hit towards the end of the install when Windows Installer is caching the packages for source resiliency. By adding all the features into the MSI, we got rid of this performance hit.

Parallel Ngen and removal of synchronous 64bit assemblies

The CLR implemented the ability to ngen on multiple cores in parallel. We made changes in our installer to take advantage of this so now on a multicore machine, ngen times should be significantly reduced. Also, on 64bit machines, most .NET applications run as 32bit. This means that paying the price of creating 64bit native images is not something most apps need to do.

Client Profile

By producing a subset of the .NET install that contains the features most client applications need, most client applications can take advantage of shorter install times by installing less.

Parallel Download and Install

If you are using the web bootstrapper which we made available for the first time in Beta2, you can use the web bootstrapper to install .NET Framework 4.0. This has the advantage of downloading and installing the payload in parallel. For example, as it is installing the Client Profile, it will be downloading the rest of the framework. In cases where you have enough bandwidth to download the rest before the Client Profile install finishes, you essentially save the time it took to download the rest.

Original post by Petermarcu

I am officially launching RobMensching.com LLC today, Labor Day 2009. This new business will begin by offering software installation consulting services for the Windows Installer and WiX toolset. You can read more details about the business on my consulting page.

Starting a business on my own has been a dream of mine for a number of years now. Early this year, I felt that the my life was becoming organized enough that I could direct some attention to a business. So throughout the month of August, I quietly laid the groundwork. I acquired a business credit card, filed the new business name with the Washington Secretary of State and ordered some new hardware.

There won’t be many other changes around here. I’m still working for Microsoft and will continue to lead the WiX toolset. RobMensching.com LLC simply provides me a new professional and independent identity to work with those people and organizations that require it.

If you have questions or comments, feel free to contact me. Otherwise, keep coding. You know I am.

Original post by Rob Mensching

I skipped the progress report for the WiX toolset last week and had a couple people ask about the count. So I guess a couple people are finding this data useful. Anyway, the short answer is that the bug count is down from 26 two weeks ago but steady from last week at 22.

Despite treading water, there are two big positive trends I see on the horizon. First, one of the hard open issues I’ve been worried about for a long while is seeing traction. In this case, I’m talking about handling fallback languages with .wxl files in MSBuild. Hopefully next week the solution will be functional.

Second, Jason has turned his attention on the Votive backlog and we’re starting to see those bugs drop. We even had a surprise visit by Justin tonight and he’s started looking into the VS2005 integration issues. If all continues, Votive will be in very good shape in a month or three.

Sadly, this was likely Bob’s last meeting with us in Redmond before he heads off to Boston. While his presence here will definitely be missed, I am excited to see how "WiX East" develops. I’m also hoping with Bob remote we can get a bit more data about how to distribute culture.

In the meantime, keep coding… you know we are.

Original post by Rob Mensching

A year ago, I had a problem posting to my blog here due to an overflowing log file. At that time, I investigated other blogging engines and came up generally dissatisfied with the available options. Ultimately, I came away thinking it’d be fun to write my own blog engine.

Well, before moving to the Live Mesh team I decided I should work on some project to learn all the new features of .NET v3.5 such as LINQ and extension methods. The WiX toolset wasn’t a good option since it purposely targets .NET Framework 2.0. So I decided to create something new.

At that time the ASP.NET MVC project seemed to be shaping up. Given my experiments with shoving Model View Presenter into ASP.NET I was very interested to see if ASP.NET MVC was any better. Then my ISP announced my servers would be moving to Windows 2008 & IIS7.

That cinched it. I was going to learn .NET v3.5 and ASP.NET MVC by writing a new blog engine.

I now have an early version ready to upload once I get the server configuration all worked out. It took four months of on and off development with a solid push over the last couple weeks around Christmas to get ready. It was a great learning experience.

I do have a short list of other features to develop later and there are probably more than a couple bugs to fix once it’s live. So I wanted to warn everyone that things might be a little bouncy here over the next couple weeks until I get all the kinks worked out.

 

Original post by Rob Mensching

It is the last day of 2008. Jenny mentioned this morning that she thought we had been through a lot of change. I immediately agreed. I said I felt we were at the end of the beginning. Let’s recap.

On the personal side, about half way through the year Jenny and I finally decided that we would sell “my house” in preparation to find “our house”. We spent all of the summer updating the house to sell in a very tough market. We painted everything inside and out, updated the kitchen and all of the bathrooms, planted plants and flowers then watered and watered. Finally, by August the house was on the market

44 days later, after three open houses and too many showings to count we had an offer. Looking back Jenny and I were incredibly fortunate. Not fortunate just because the average time on market was 90 - 120 days and the market took an incredible beating shortly after we closed. We were fortunate because the day we had a solid offer, Jenny and I found our dream house. The sellers had a contingent offer on another house but were about to give up since their house was not closing.

In a very stressful two weeks Jenny and I negotiated the final deal on the house we were selling and made a contingent offer on the dream house. There were several days where it felt like everything would just come unglued. But by the beginning of November we were moved in and still pinching ourselves that we now lived in such a beautiful house. I expect Jenny and I will be here for quite a long time firmly ending the semi-nomadic life of early adulthood.

On the professional side, this year has been the most action packed 12 months of my life. Lots of code checked in and shipped out. Let’s just go through the list.

At the beginning of the year we declared WiX v2 Production/Stable. That was a huge accomplishment only overshadowed by the announcement that WiX v3 would be shipping in Visual Studio. Technically speaking it was last year that I announced that the WiX toolset would ship as part of Visual Studio but it was this year that I feel we really made significant progress toward actually shipping. We made necessary the adjustments to the WiX v3 roadmap to stay on track and over the last 6 months the WiX v3 bug count has plummeted. Every Thursday of every week this year I have enjoyed the hours spent with all the people that volunteer on the WiX toolset. It’s a great project with great people.

All the other professional excitement happened at the end of the year. First, I announced that I was joining the Live Mesh team during all of the turmoil of preparing a house for sale. However, I couldn’t talk about the project I was leaving since it didn’t officially launch until November: Microsoft Store. Trevin Chow, the lead Program Manager, had a good blog entry about the launch of Microsoft Store. A little later he wrote an in depth entry about the piece of the project that I worked on most: the Microsoft Store Download Manager. Those two blog entries provide a pretty good overview of what I was working on for most of 2008. That team was a lot of fun and I really liked the way the Download Manager turned out. I stop by and talk to friends on Microsoft Store team every once in a while since the WiX Working Group meets on the 4th floor of their building.

Of course, the end of the year finds me on the Live Mesh team. The weeks before the PDC were some of the most intense days of programming since I left college. Delivering a technology that forms the linchpin of several executive key notes at one of the largest gathering of developers for the Microsoft platform is a bit awe inspiring if not incredibly tense. In the days since, we’ve been gathering feedback about how the technology previews are being used and creating a plan for the next year to come. But this blog entry isn’t about the future so I’ll stop there. <smile/>

As I noted in the beginning it feels like I’m at the end of the beginning. I’ve enjoyed getting here and now I’m looking forward to finding my way into the future.

Have a Happy New Year!

 

Original post by Rob Mensching

Last week Bob pushed the latest WiX v3 release to SourceForge. The original intention was to declare that drop as the final Beta build for WiX v3. However, I changed my mind for two reasons.

First, I know a lot people don’t pick up weekly releases, only SourceForge drops. Sad but true. Since it had been a while since the last SourceForge drop and I wanted to make the drop then watch to see if anything new came up. Nothing has.

Second, we were very close to zero bugs on the core toolset and had a couple extension bugs that could be considered blockers. With an extra two weeks I was pretty certain that we could knock out a few more of those. Bob and I have made good progress through those this week (holidays can be good for the volunteer hours).

That means it is very likely that next week’s release will be pushed as the “official WiX v3 Beta release”.

You might wonder what it means for a project already called “beta” to have an official Beta release. In this case the Beta release marks the turning point where all of the major features for this release are finished and the bug flow such is under control and on a solid downward trend.

The goal is to provide a build and encourage all WiX v3 users to upgrade. That way we focus our efforts on finding all of the remaining bugs and figure out what bugs get fixed in WiX v3 and what gets moved to WiX v4. If you’ve found a bug in the WiX toolset get it filed soon.

Bob Arson is actually the architect of this plan for finishing WiX v3 in 2009 so look for more details on Bob’s blog over the next nine months.

 

Original post by Rob Mensching