Disclaimer: Please note that solution presented here is not supported by Microsoft.
UPDATE 23.02.3013: I got information from Microsoft .NET team that .NET framework SDK tools can be installed on Windows Server 2008 so all of this hacking might no be needed at all. Refer to topic titled ".NET Framework 4.5 SDK Tools on Windows Vista and Windows Server 2008" @ msdn.microsoft.com/.../hh852363#rn
At work we are upgrading our product to build on .Net 4.5 and there’s many preparation steps to it. Technically it is easy, just install the update package, however for us with product running as a service in different customer hosting environments and us needing to support several versions of our product in live environments, it is not so easy. All hosting servers must have .net 4.5 installed before any .net 4.5 requiring release can be made, we need to also upgrade all of our development and test and stage environments to make sure the deployment package is built correctly. This is nothing that happens quickly.
Today I will focus on the development environment, where we have long since switched to VS 2012, however are suffering with the migration of TFS 2010 to TFS 2012. I say migration, because our Team Project is using Scrum for Team System v3 template and as many know, the template is long dead. We could upgrade TFS easily, however we need to get rid off the dead process template and move on to something that is being consistently maintained such as Microsoft Scrum v2 process template. This will be a topic for another post, since this is currently in progress and I will share my findings once its done.
The problem at hand
So, stuck with TFS 2010 we still planned and progressed with .NET 4.5 upgrade. Our TFS is running on Windows Server 2008 Standard Edition 32bit. There was no prior reason to upgrade. So we did all the project file changes and everything worked nicely locally, however once you checked in the changes – the build fails. Of course we had already taken care of the obvious such as installing .NET 4.5 on TFS servers. This was not enough.
After some digging I found that .NET 4.5 SDK is needed or Visual Studio 2012. Makes sense, however it was not obvious to check that .NET 4.5 SDK or VS 2012 are at all installable on Windows Server 2008. They are NOT. Surprise!
Keep in mind that we have a working TFS with Build machine and the build machine has .NET 4.0 SDK and Visual Studio Team Explorer and its running on Windows Server 2008 32bit. This machine successfully builds our .net 4.0 based application, runs unit tests, prepares deployment package etc.
So, now we need to somehow figure our a way to get .NET 4.5 application to build. The good thing in this case is that .NET 4.5 is an in-place upgrade of .NET 4.0 (in some cases the in-place upgrade can be a disadvantage, however here it helped), so compilation actually would work, however the projects were referring to new Web Application MSBuild files and the .net 4.5 SDK is also missing so the “SdkToolsPath” is empty.
The imported project "C:\Program Files\MSBuild\Microsoft\VisualStudio\v11.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
Copy the “C:\Program Files\MSBuild\Microsoft\VisualStudio\v11.0” from your development machine where you have VS 2012 to the TFS Build machine.
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets (2742): Task could not find "sgen.exe" using the SdkToolsPath "" or the registry key "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v8.0A\WinSDK-NetFx40Tools-x86". Make sure the SdkToolsPath is set and the tool exists in the correct processor specific location under the SdkToolsPath and that the Microsoft Windows SDK is installed
Step 1. Copy everything from “C:\Program Files\Microsoft SDKs\Windows\v8.0A” to the TFS build machine matching location.
Step 2. Export the registry key at “HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\Windows\” from your development machine. BEFORE RUNNING make sure the paths in the .reg file are correct. e.g. my development machine is 64bit, but the TFS build server is 32bit. Also you should only need to keep the Windows key level settings and v8.0A, you don’t need the registry keys for v7.0A or others.
Run the registry file in TFS build server and this will add the missing keys for .NET 4.5 SDK.
After this everything worked again as before!
Last week on Friday the 13th of April (for superstitious it might be a bad day, but for us it was good ;) ), we (VS ALM Rangers team working on this guidance) released Branching and Merging Guidance Beta for Team Foundation Server 11. It has been long in the working and it is finally out! You can read Willy’s announcement here. So what's new in the vNext of Branching and Merging Guidance (Beta)? The previous version of the guidance was split in several chunks such as main document, scenarios document, Q&A document, Hands-on-labs, etc. In this version of the guidance we have merged main document and scenarios document together to make the guidance easier to follow. We will continue to merge some of the other documents into the main document. The main document also has new content such as Managing shared resources in TFS Baseless merging guidance Local Workspaces in Team Foundation Server 11 Merge Improvements in Team Foundation Server 11 Additional to basic, standard and advances branch plan we have Branch by Feature branch plan Code promotion branch plan New Hands-on-Labs We have aimed to improve the current branch plans to better explain why and where certain branching strategies are valuable to be used. There is more to come in RC version. We are eagerly waiting for feedback and input from community to improve the content further and make it more valuable to everyone. So please, download the guidance, have a bedside read and don’t hesitate to provide feedback on the new version on this blog post using comments, direct contact to me, on the codeplex project discussion board or submit a suggestion on UserVoice, using the Rangers Project category tag. . Some more links on VS ALM Rangers: Rangers projects and solutions Visual Studio ALM Rangers Index Thanks!
Last Friday the new release of Team Foundation Server Power Tools became available and it is called the August 2011 release. In this release there’s lots of new features and fixes and I would like to make a small stop (already there’s plenty of details out there, so I will not repeat it) on two of them. First, with this release, you can do full-text searching over you work items. There is similar functionality in a codeplex project called Search Work Items, however this is a functionality that was requested on the Visual Studio User voice forum and it is something that has been implemented for the vNext of Visual Studio and TFS. TFS Power Tools make it possible for the product group to bring the functionality to customers earlier. The second is that you can now rollback changeset’s from Source Control Explorer UI – either by right clicking on a specific changeset or on a folder/branch. If you click on a folder or branch, then you will be able to rollback one changeset or a range. The functionality to rollback changeset’s was implemented in TFS 2010, however previously it was only available through command line (http://msdn.microsoft.com/en-us/library/dd380776.aspx). NB! If you have never tried to rollback anything in TFS, then when you rollback – the changes will be done on your local disk and you will need to check-in the rollback changes as a changeset to the TFS. The model is similar to merging, but e.g. allows you to rollback a rollback ;) Anyhow, more information incl. pictures can be found following links. Brian Harry introduces the latest TFS Power Tools Ed Blankenship - small intro to some features (Note! builddefinition and workitemsearch.mht) And you can download the new TFS Power Tools from the following link. Download latest TFS Power Tools
This is a topic I’ve wanted to better understand for a while and never had time to investigate. Also it has been in my backlog to write as a post for a weeks now, and I finally made to writing it. Some of the questions many developers ask daily are – Why is my Visual Studio taking so much memory? Why did I get Out Of Memory exception? Task manager shows I have enough free memory. A developer is normally concerned with these questions when they are running on 32bit OS. It is even likelier problem when you are using lots of plugins/additions to your Visual Studio or solutions with many projects or big projects. The goal of this post is to explain the topic to some degree (I am not a OS/memory expert) and offer a tool and guidance to analyzing why your Visual Studio is eating as much memory as it is. In the end I will offer some recommendations that could help you and that have helped me. Also note that 64bit OS changes the game a lot and I’ve not heard memory/Out Of Memory problems from a 64bit development environment. So what's the problem? Well, the real problem is that Visual Studio (devenv.exe) loads a huge amount binaries (VS components, extensions, user dll’s built by the IDE, …) into its virtual address space. Back when Visual Studio was created such components were smaller and there weren't so many (and although we had less memory and needed less memory, the 32bit OS provided us with a 2GB virtual address space), so the solution of loading everything into the same process virtual address space made sense. Over time however the amount of components, IDE binaries, etc has increased and the size of those components has increased, and developers running on 32bit environment are experiencing more and more out of memory or performance degradation issues. With the current model, just too much is loaded into one process address space. The following picture is a partial from a tool called VMMap (intro follows) and the purple is the color of image type items in the devenv address space ordered by the size. This is only a fraction of the list, however you may notice this is already consuming > 150MB address space. The tool and how to analyze? A month ago I was experiencing a growing frequency of Out Of Memory exceptions and I really wanted to understand why. What was causing it and why suddenly VS ate more than 1GB of memory. The tool is from SysInternals (they have a bunch of clever tools) and is called VMMap. It is a clever tool that gives you a picture of what is taking up the memory in a running process. Description from VMMap help: VMMap is a process virtual and physical memory analysis utility. It shows a breakdown of a process's committed virtual memory types as well as the amount of physical memory (working set) assigned by the operating system to those types. Besides graphical representations of memory usage, VMMap also shows summary information and a detailed process memory map. Powerful filtering, refresh and snapshot comparison capabilities allow you to identify the sources of process memory usage and the memory cost of application features. The tool gives you a quick overview and detail grid of each item in the address space. The following is an overview of my biggest memory usage with no solution loaded but only VS and addins such as Resharper, VSCommands, Productivity Tools, TFS Power Tools (so basically a new VS opened). The key things I read from this are (and they are marked with red boxes): Committed – the total usage of virtual memory by process by type (described with color scheme). Private Bytes – committed private virtual memory (backed by paging file and charged against system commit limit) Working Set – committed virtual memory that is in physical memory and owned by process What is the type of biggest consumer of memory (virtual, private, working set)? - Image type or “.exe”/”.dll” How much free address space the process has. Additional to this, you can see the growth or decrease in the process memory usage by using the devenv tool a bit and then refreshing the data in VMMap. VMMap will keep statistical information and offer a graph the stats. Please note that the image above doesn’t match the initial overview listing. Its taken from a devenv run where I had been working and refreshed the VMMap regularily. One approach for analyzing your Visual Studio The following would be a good way to compare a Visual Studio where all extensions have been loaded to a Visual Studio where no extensions have been loaded. 1. Run Visual Studio in SafeMode (no extensions loaded) and see what is the memory consumption then. 2. Run Visual Studio in normal mode and see what is the memory consumption then. Recommendations to improve Out Of Memory Exceptions on 32bit OS and VS 1. Disable and do not use so many extensions. (im just putting this out there, im sure most add-ins that have been installed are actually valuable and required for you) 2. Sometimes it is not viable to just disable extensions (e.g. they offer critical productivity). Tools such as Resharper or CodeRush that tend to be more memory extensive, also have ways to optimize where the tool is being used, so make sure you configure the extensions to work optimally for you solutions/projects. 3. Create several solution files and distribute the projects between those solution files to support the team to work with the source code that they normally want to change together. Don’t create and work with one huge solution file which has all the 10…N projects loaded into it and hope that Visual Studio will work with it. You will optimize the performance by limiting the amount of projects/files loaded at once into Visual Studio. One more option to optimize the work here is to unload the projects that you do not want to change, restart and the next time you open solution Visual Studio should remember those and not load the projects. 4. The fourth option is to increase the amount of virtual memory a process running in user mode can have. By default 32bit Windows OS limits it to 2GB and the other 2GB is reserved for System. However, you can modify this and have 3GB for user and 1GB for system. Please note that if you have lots of services/process running under system, those might starve now! However in my development machine and in many of my colleagues machines we have applied this change and it has had a positive effect of being able to work with solution files that use to throw Out Of Memory exception before. And we have not notices any system processes starving. In reality on a development machine, there shouldn’t really be so many system services that would require more than 1GB address space. To increase user mode virtual memory address space to 3GB In Windows Vista/Windows 7 run the following command in a CMD that is running in admin mode: bcdedit /set IncreaseUserVa 3072 3072 is the maximum value. You can set it to less that 3072 as well, e.g. 2560 (2,5GB). You will need to restart the computer for the change to take effect. You can revert the change using the following command: bcdedit /deletevalue IncreaseUserVa You will need to restart the computer for the change to take effect. In Windows XP, you can do this with the /3GB switch. More information on following MSDN article. 5. Migrate to 64bit OS (again im just putting this out there). This is not solving the problem, that VS is using lots of memory. That can be better affected by the previous points (solution, project file modifications, addins, extensions etc) I'm finding two main values for myself from this tool. Firstly, I can see what is consuming all the virtual address space in the process and based on that possibly I can resolve by not using some add-ins. At least I know what is causing the issue – pure usage, or too many add-ins or something else. The question is – why am I running out of memory or why am I using so much memory. Secondly (and not so related to Visual Studio), I can image using VMMap tool to analyze my own product services and applications while they are running and analyzing what is consuming all the memory. This is another tool in the arsenal. There are others profilers that offer also information about memory usage and consumption in application, but most of them don’t show you e.g. how much binaries take compared to heap. In the end … In the end I will leave you with a picture from my development environment after I applied the change to increase user virtual address space to 3GB. What do you see from this? Do you read and notice the effect of the change? I would also like to thank Nathan Halstead from Microsoft Visual Studio Platform team for answering my questions and providing me valuable information that helped me write this post.
This is a tool I first heard of at the MVP Summit and was really excited about, but back then it wasn’t yet released or public. It has beeb available for a while now, however unfortunately I found this our lately.
I am very excited about this because it allows me to see how long the UI thread is working on other things and easily send that feedback back to Microsoft to analyze for future versions. In v2010 there’s still plenty of situations where I have roll my thumbs, while the UI is not responding to my commands – not always due to Visual Studio itself, sometimes due to the add-ins I use. Most commonly experienced and known is the build activity and pre-build/post-build event activities. Don’t get me wrong – v2010 is better than v2008 on this, however not perfect.
Anyway – what is this PerfWatson? Lots of people might know the original Watson from Microsoft as the tool that collects error information, symptoms and sends them to Microsoft for analysis. PerfWatson is a similar tool that monitors Visual Studio for UI latency (how long the UI thread has been frozen/occupied with non UI activities) and if it has been busy for more than 2 seconds, it will collect a minidump for Visual Studio and send the information to Microsoft for analysis so the future Visual Studio versions would have far better UI responsiveness and flow.
You can download them from following links:
Visual Studio PerfWatson (the platform)
Visual Studio PerfWatson Monitor (shows UI latency)
The PerfWatson is the monitoring platform for Visual Studio (the one that monitors and collects the minidump/info and offers to send it to Microsoft) and the PerfWatson Monitor offers a nice UI to see the current latency of UI thread. Its more of a geeky tool :)
Some sample screens from my daily encounters with the tool. Notice the number is showing latency in milliseconds. As soon as it goes over 2 sec, things go red.
The normal graphic – or Green behavior (good):
Some of red graphs I’ve had. The memory numbers are resharpers – not PerfWatson’s:
And one with Resharper v6 doing it’s magic:
My next post will offer some guidance on how to analyze Visual Studio memory consumption and some tips to improve performance on 32bit system. I still run half of my developments on 32bit machine and it can be a pain!
Microsoft has many channels (such as connect, MSDN forums, newsgroups, etc.) to gather feedback from users, customers, partners, MVP’s. This month a new channel called uservoice has been added to the list. Through uservoice new ideas can be proposed and those ideas can be commented/voted by other people. If some ideas have enough votes (they are important), then it is possible that Microsoft will listen to the people and the vNext of Visual Studio and/or Team Foundation Server will contain that feature or change. This uservoice channel is for Visual Studio, Team Foundation Server and the whole Application Lifecycle Management (ALM). So make sure you give all your ideas. Each user has 10 votes and thus can vote for 10 ideas. Make sure you vote for the ones that are most relevant and important for you, to make sure it will be in the vNext You can vote here: http://visualstudio.uservoice.com/