Quantcast
Channel: The Microsoft MVP Award Program Blog
Viewing all 788 articles
Browse latest View live

The New MVP Award Web Site has Launched!

0
0

The new MVP Award Web site went live today, offering a more in-depth look at this dynamic global community. Here are a few new features you’ll find at the site’s new URL: http://mvp.microsoft.com:

· MVP Spotlights—a monthly series that offers a picture of the way individual MVPs make a difference in the community.

· Regional highlights and events—the site is published in 11 languages and showcases what’s happening in MVP communities around the world.

· Live social media feeds—these often reflect local languages and events.

And don’t forget to sneak a peek at the new Find an MVP page, featuring a mosaic of MVP profiles that updates regularly. Recognize anyone?

clip_image002


MVP Friday Five: June 1, 2012

0
0

1. SDK 1.5 is Out

By Dynamics MVP Jivtesh  - @jivtesh

This article covers the v1.5 release of SDK.

2.  Learn About Windows Azure and Chat with Experts, June 7th.

By Windows Azure MVP Michael Wood – @mikewo

This article details an upcoming event which discusses what’s up and coming with Windows Azure.

3. The Great Ecosystem of the Open Data Protocol (OData)

By Data Platform Development MVP Chris Woodruff – @cwoodruff

This article gives a description of the Odata ecosystem that has risen over the last few years around this protocol and technology.

4. Give me some Features… (Site creation troubles, missing features, SharePoint localization, Routing Workflows and other bad jokes)

By Sharepoint MVP Adis Jugo – @adisjugo

This article helps with troubleshooting problems stemming from the transfer of site definition from one server to another.

5. Getting Started with CSS Media Queries

By Silverlight MVP Dan Wahlin – @danwahlin

This article discusses how CSS media types such as "print", backgrounds, navigation bars, and other items can be hidden as a user prints which makes the printed page look much more readable.

Using sys.dm_os_ring_buffers To Diagnose Memory Issues in SQL Server

0
0

Editor's Note: The following MVP Monday post is by SQL Server MVP Grant Fritchey

Because of the ways that SQL Server interacts with the Windows operating system, memory and memory management between the two can be pretty confusing. If you just look at the allocated memory within the operating system on a standard SQL Server configuration you’ll probably see all, or most, of the memory is used up by SQL Server. Within SQL Server if you look at resource counters like total server memory, these too don’t provide much information to help you determine whether or not you are experiencing pressure due to inadequate memory on your system. You need to be able to quickly and easily determine if you are experiencing memory issues. Taking advantage of the dynamic management object (DMO) sys.dm_os_ring_buffers can get you that quick and accurate determination if you have memory problems and if they’re coming from the Windows operating system or within SQL Server.

Memory Allocation

Before I show you how to query the information within sys.dm_os_ring_buffers, let’s be sure we’ve got agreement on how memory is managed between SQL Server and Windows. Thinking of this strictly from the context of SQL Server, memory, and memory problems can be broken down as being internal, that is to say a part of SQL Server and it’s memory management, or external, a part of the Windows operating system and it’s memory management. But if you reverse the view and look at things from the Windows operating system you’ll be looking at physical memory, the memory managed by Windows, or a Virtual Address Space (VAS) which is memory that has been allocated to SQL Server, but it may or may not be actual physical memory. Within the VAS, SQL Server gets to allocate memory in whatever manner it requires without having to refer back to the Windows operating system. However, the operating system is still managing memory, moving it into physical memory or to the disk through the page file.

When you first set up SQL Server, you may have set a maximum limit for memory. As SQL Server works, it will request memory from the operating system up to this maximum. It will seldom return this memory to the operating system and that’s why you so frequently see memory maxed out on the OS when SQL Server is running. It may seem like a problem, but it isn’t. However, it does add to the load placed on memory management within the operating system since less and less memory is available for things that the operating system might need. This is especially problematic if you’re running processes other than SQL Server on that system.

Because of this split in memory, you have two places where you can “run out” of memory due to different processes. You can need more memory for processes within the operating system, in which case, you’re looking at a physical, or external, memory issue. You can also require more memory within the VAS for SQL Server in which case you’re looking at a virtual, or internal, memory issue.

Is There Memory Pressure?

But the question still comes down to, how do you determine if you have memory issues and where are those issues coming from. Let’s take a look at sys.dm_os_ring_buffers to understand what it can do for us to answer this question.

This DMO is not documented in the Books Online (BOL). Because of this, it is subject to change without notice from Microsoft.

The ring buffers are simply a record of alerts fired within the system that are recorded for internal diagnostics. There are a number of different alerts that can be viewed through the DMO. Running this query against the system:

SELECT *

FROM sys.dm_os_ring_buffers AS dorb;

Results in this set of results (which will look different on your machine):

clip_image002

There are only four columns, but the final column is XML and that’s where the good information can be found. There are a number of different ring buffers but we’re only interested in the type called RING_BUFFER_RESOURCE_MONITOR. This records changes to memory allocations, which is a great way to see when you’re running out of memory because a message that your memory is low is a pretty good indicator that you might be seeing memory pressure.

If we look just at this sample XML for a RING_BUFFER_RESOURCE_MONITOR message it looks like this:

<Record id = "1" type ="RING_BUFFER_RESOURCE_MONITOR" time ="129031647">

<ResourceMonitor>

<Notification>RESOURCE_MEMPHYSICAL_LOW</Notification>

<IndicatorsProcess>2</IndicatorsProcess>

<IndicatorsSystem>0</IndicatorsSystem>

<NodeId>0</NodeId>

<Effect type="APPLY_LOWPM" state="EFFECT_OFF" reversed="0">0</Effect>

<Effect type="APPLY_HIGHPM" state="EFFECT_IGNORE" reversed="0">128991020</Effect>

<Effect type="REVERT_HIGHPM" state="EFFECT_OFF" reversed="0">0</Effect>

</ResourceMonitor>

<MemoryNode id="0">

<TargetMemory>153576</TargetMemory>

<ReservedMemory>9022400</ReservedMemory>

<CommittedMemory>201728</CommittedMemory>

<SharedMemory>0</SharedMemory>

<AWEMemory>0</AWEMemory>

<PagesMemory>152320</PagesMemory>

</MemoryNode>

<MemoryRecord>

<MemoryUtilization>100</MemoryUtilization>

<TotalPhysicalMemory>4193848</TotalPhysicalMemory>

<AvailablePhysicalMemory>2175836</AvailablePhysicalMemory>

<TotalPageFile>8385844</TotalPageFile>

<AvailablePageFile>6064244</AvailablePageFile>

<TotalVirtualAddressSpace>8589934464</TotalVirtualAddressSpace>

<AvailableVirtualAddressSpace>8580567180</AvailableVirtualAddressSpace>

<AvailableExtendedVirtualAddressSpace>0</AvailableExtendedVirtualAddressSpace>

</MemoryRecord>

</Record>

The information presented in the MemoryNode and the MemoryRecord are useful for attempting to figure out what went wrong, but the key points are up in the ResourceMonitor element with the Notification, IndicatorsProcess, and IndicatorsSystem values. First, the notification tells us that this was RESOURCE_MEMPHYSICAL_LOW message that was captured in the ring buffers. That means that physical memory was low. The next two indicators let us know what was low. If the IndicatorsProcess returns 0 and the IndicatorsSystem returns a value then the problem was system wide. But, in our case the IndicatorsProcess has a value and IndicatorsSystem is returning 0. This means that this alert was for a single process that ran suffered from low memory, not the entire system. The values break down as follows:

Value

Meaning

1

High Physical Memory

2

Low Physical Memory

4

Low Virtual Memory

When you receive these, you now know that you’re looking at some degree of memory pressure within your system. Determining if it’s a process-only or a system-wide issue is helpful. Even if it’s process-only, you’re still looking at low memory on the system for the processes involved. That may mean that you’re low on memory, but it could also just mean that your processes need some rewriting.

Note: You get the indication that a process had memory issues, but it doesn’t tell you which process. You can capture these events using Extended Events and then marry them up with queries, but you can’t get directly from these messages to a particular system process.

How to Query It

Knowing that you have memory pressure from these alerts captured in the ring buffers is good, but you’re going to want to be able to capture this information programmatically. You’re not going to want to have to scroll through tons of data looking for the particular events that have occurred. Instead, let’s use XQuery to pull that information right out of the DMO:

WITH RingBuffer

AS (SELECT CAST(dorb.record AS XML) AS xRecord,

dorb.timestamp

FROM sys.dm_os_ring_buffers AS dorb

WHERE dorb.ring_buffer_type = 'RING_BUFFER_RESOURCE_MONITOR'

)

SELECT xr.value('(ResourceMonitor/Notification)[1]', 'varchar(75)') AS RmNotification,

xr.value('(ResourceMonitor/IndicatorsProcess)[1]','tinyint') AS IndicatorsProcess,

xr.value('(ResourceMonitor/IndicatorsSystem)[1]','tinyint') AS IndicatorsSystem,

DATEADD(ms, -1 * dosi.ms_ticks - rb.timestamp, GETDATE()) AS RmDateTime

FROM RingBuffer AS rb

CROSS APPLY rb.xRecord.nodes('Record') record (xr)

CROSS JOIN sys.dm_os_sys_info AS dosi

ORDER BY RmDateTime DESC;

This query shows the basic information available from the DMO and shows off one additional piece of information embedded in the results, the actual date and time derived from the timestamp value. If all you’re interested in is the fact of the memory issue, you can stop here. But if you’re interested in a little diagnostic work on top of this, you’re going to want to pull the rest of the memory information that’s available like this:

WITH RingBuffer

AS (SELECT CAST(dorb.record AS XML) AS xRecord,

dorb.timestamp

FROM sys.dm_os_ring_buffers AS dorb

WHERE dorb.ring_buffer_type = 'RING_BUFFER_RESOURCE_MONITOR'

)

SELECT xr.value('(ResourceMonitor/Notification)[1]', 'varchar(75)') AS RmNotification,

xr.value('(ResourceMonitor/IndicatorsProcess)[1]','tinyint') AS IndicatorsProcess,

xr.value('(ResourceMonitor/IndicatorsSystem)[1]','tinyint') AS IndicatorsSystem,

DATEADD(ms, -1 * dosi.ms_ticks - rb.timestamp, GETDATE()) AS RmDateTime,

xr.value('(MemoryNode/TargetMemory)[1]','bigint') AS TargetMemory,

xr.value('(MemoryNode/ReserveMemory)[1]','bigint') AS ReserveMemory,

xr.value('(MemoryNode/CommittedMemory)[1]','bigint') AS CommitedMemory,

xr.value('(MemoryNode/SharedMemory)[1]','bigint') AS SharedMemory,

xr.value('(MemoryNode/PagesMemory)[1]','bigint') AS PagesMemory,

xr.value('(MemoryRecord/MemoryUtilization)[1]','bigint') AS MemoryUtilization,

xr.value('(MemoryRecord/TotalPhysicalMemory)[1]','bigint') AS TotalPhysicalMemory,

xr.value('(MemoryRecord/AvailablePhysicalMemory)[1]','bigint') AS AvailablePhysicalMemory,

xr.value('(MemoryRecord/TotalPageFile)[1]','bigint') AS TotalPageFile,

xr.value('(MemoryRecord/AvailablePageFile)[1]','bigint') AS AvailablePageFile,

xr.value('(MemoryRecord/TotalVirtualAddressSpace)[1]','bigint') AS TotalVirtualAddressSpace,

xr.value('(MemoryRecord/AvailableVirtualAddressSpace)[1]','bigint') AS AvailableVirtualAddressSpace,

xr.value('(MemoryRecord/AvailableExtendedVirtualAddressSpace)[1]','bigint') AS AvailableExtendedVirtualAddressSpace

FROM RingBuffer AS rb

CROSS APPLY rb.xRecord.nodes('Record') record (xr)

CROSS JOIN sys.dm_os_sys_info AS dosi

ORDER BY RmDateTime DESC;

Now the query includes all the available information from the XML. There is one exception. I left out AWE memory since most people are running 64 bit systems now and this value will always be zero. Now you information that will enable you to see where the memory issues are occurring.

Summary

As I said in the text, this won’t tell you where your memory issues are coming from, but using the sys.dm_os_ring_buffers DMO to look for the RING_BUFFER_RESOURCE_MONITOR event types will absolutely let you know that you are experiencing memory pressure. You can query this DMO directly as part of a monitoring solution, or you can capture the events using the resource_monitor_ring_buffer_recorded event. Either way, you’ll have to deal with the XML output. Hopefully these queries will help you track your memory issues in SQL Server.

Additional Reading

For more information on how memory management works within SQL Server

For more detail on the RING_BUFFER_RESOURCE_MONITOR

Tips on improving performance by turning off ring buffer data collection

 

Author Bio
Grant Fritchey

Grant Fritchey works for Red Gate Software as a Product Evangelist. Grant has 20 years experience in the industry as a DBA and developer. Grant is an active participant in the SQL Server Central and AskSQLServerCentral discussion forums. He writes articles for SQL Server Central and Simple-Talk. Grant is the author of the books "SQL Server 2012 Performance Tuning Distilled" and "SQL Server Execution Plans." He is co-author on the books "SQL Server Team-Based Development,"  "SQL Server MVP Deep Dives 2," and "Beginning SQL Server 2012 Administration."  Grant blogs at http://scarydba.com

MVP Mondays

The MVP Monday Series is created by Melissa Travers. In this series we work to provide readers with a guest post from an MVP every Monday. Melissa is a Community Program Manager for Dynamics, Excel, Office 365, Platforms and SharePoint in the United States. She has been working with MVPs since her early days as Microsoft Exchange Support Engineer when MVPs would answer all the questions in the old newsgroups before she could get to them

TechEd 2012–MVP Brent Stineman Shares His Excitement

0
0

The following post was written by Windows Azure MVP Brent Stineman

I’m going to TechEd North America 2012. It sounds simple, and for many it may even seem mundane. But I don’t do many conferences, and outside of the Microsoft MVP summit, I haven’t been to a vendor specific conference since a Borland event back in the late 90’s. So it’s safe to say, I’m excited.

I was recently asked what I was looking forward to the most at TechEd. As an MVP, I’ve already been given sneak peeks at many of the new features that will be announced (at least as it relates to Microsoft’s cloud initiatives). So I’d say what I’m looking forward to most is spending time with everyone there; the speakers, the attendees, the staff. Talking with people, the sharing of ideas, these to me are the real values of these events. Call it “networking” if you will. But to me it’s actually a rare chance to do in person what I do virtually most of the year, talking with people, sharing experiences, and learning.

This is where I count myself a bit fortunate. Unlike most of the conferences I’ve attended lately, I’m not speaking or presenting. Luckily, I have no concerns about preparing session materials, putting myself into “presenter mode”, or stressing out about being on time and living up to the audiences expectations. Instead, I’ve been selected to help staff some of the “expert” areas. I couldn’t imagine a better outcome for my first trip to TechEd. I get to spend at least 16 hours of the event specifically talking with and helping people with the technology I’ve focused the last 3+ years of my career on.

Now I don’t want to discount all the great sessions that will be at the conference. I always pick up new info no matter how many times I see presentations from the likes of Mark Russinovich, Clemans Vasters, Nathan Totten, Nick Harris, Brad Calder… the list goes on. But it’s the stories and questions from the vast array of fellow professionals, all working to solve their own challenges, that I never tire of. I’ll slow down when I overhear something in a hallway or inadvertently eavesdrop on conversations on an airplane. If they’re struggling with an issue, I stop and offer to help. Sometimes I’m fortunate and can give them that one piece of info they need to solve the problem, but as often as not, I learn from them.

As a consultant, my experiences are limited largely by the clients that engage my services. I rarely have the time to work on side projects. But at TechEd I’m hoping to have a never ending buffet of stories, ideas, and challenges I can learn from.

So if you’re going this year, please stop by the Windows Azure booth or the expert center, look for a guy named Brent wearing a green shirt, and say “hi”, and share your stories with me. I’ll be easy to spot. I’m the guy that’s grinning like a kid in a candy store.

PS – oh, and swag. Swag is always good.

About the author

clip_image001

Brent’s nearly 20 year career has spanned platforms from the mainframe to mobile devices. Brent started working with the Windows Azure platform early in its CTP phase, and is now focused on helping colleagues and clients explore the promise of cloud computing. Brent is also one of the inaugural Microsoft MVP’s for the Windows Azure Platform.

Talking Cloud Tuesday with Windows Azure MVP Ming Chung Chu

0
0

ming

1. How long have you been an MVP?

I have been an MVP for 9 years (2004-2012)

2. How did you first start in community?

My community experience started with the Programmer-Club managing “Training and Certification” and “.NET group” forums at 2001. In 2003, I joined the Microsoft online community newsgroups to answer questions about .NET, ASP.NET, ADO.NET, Web Service, SQL Server, Windows Server etc. I was awarded as an MVP for the first time in SQL Server in October 2003. Currently, I am a Windows Azure MVP.

3. Which technical community or communities are you most active in (where can people find you)? Provide links if necessary

MSDN and TechNet Forum, and my blog: http://www.dotblogs.com.tw/regionbbs

4. What’s the best technical tip you have today for implementing a cloud deployment?

A Cloud solution is different to an on-premise solutions with high availability, state management, storage and deployment strategy. Good design of solution architectures and plans is very important for Cloud solution deployment.

5. When considering using the Cloud, what do you tell people if they aren’t sure about moving to the Cloud?

The Cloud environment is highly useful, extensive and low cost. Enterprise resources could be provided by the Cloud which is suitable for newly founded companies or organizations that do not need an IT team.

6. Do you have a blog/website link to Cloud related Tips or deployment stories you would like to share?

Yes.  http://www.dotblogs.com.tw/regionbbs

7. Speaking from your experience, what words of advice do you have for new MVPs?

Enjoying sharing knowledge will bring happiness and richness.

TechEd 2012– Silverlight MVP Michael Crump Shares Excitement

0
0

The following post was written by Silverlight MVP Michael Crump

This Sunday I will depart to Orlando, FL to attend TechEd 2012 which is Microsoft’s premier technology conference for IT Professionals and Developers. This is my first TechEd that I have had the chance of attending and I will be representing Telerik, where I have a focus in our XAML control suites. While I have been preparing for the trip, I decided to sit down and write out a short list of what I’m most looking forward to this year and share it with the MVP Award Blog. So, in no particular order, here goes:

Networking: I’ve said it before and I’ll say it again, networking is one of the single most important things you can do a tech conference such as TechEd 2012. It is a unique opportunity that allows you to have access to several thousands of IT/Developer professionals just like you. You should seize every opportunity that you get to mix and mingle with liked-minded individuals. You never know what new business cards or new knowledge that you will take home.

Deeper Understanding of Microsoft Products: If you are a Developer (like me), then you will get a new and unique look at the new platform for building Metro style applications for Windows 8. But it doesn’t stop there; other sessions include: Windows Phone 7, WebMatrix, Azure, SQL, ASP.NET and much more. Not only will you take this information back to the office, but it puts your organization at the forefront of cutting edge technology. They will also have tons of tracks focused around IT Professionals, so no one is left out.

TechExpo: The TechExpo consist of two sections:

o Partner Solutions Exhibition – This allows you to review all the new stuff from Microsoft Partners (companies making products using Microsoft technologies) and to meet and greet with members of the team.

o Technical Learning Center – This allows you to see Microsoft products demoed by the Microsoft team. Again, this is another great opportunity to ask questions and meet the team behind the products you use every day.

Other Familiar Brands: I’m looking forward to seeing the Channel 9 broadcast live and seeing the Coding4Fun BoxingBots! I also plan on hanging out in the bloggers hub and getting others excited about blogging. Also, being an MVP means that I’ll also be hanging out in the MVP Lounge to answer any questions that you might have. The Connect Zone will also be a great place to stop by to learn about INETA, which can help you find a user group in your area.

Archive of Sessions: Let’s face it – it is hard to go to every session with all the different things going on. Sometimes, it is just easier to learn at your desk with your favorite pair of headphones on. Thankfully, the session videos and slides of most sessions are available to members of myTechEd after the event. This is an absolutely great resource in case you forgot something or had to pick between two different sessions at the same time.

Conclusion

As you can tell, there are a lot of reasons that I’m looking forward to attending TechEd this year. I haven’t even mentioned all of the pre-conference seminars, workshops, hands-on-labs and certification information available. I’d also encourage you to sign up at myTechEd, where you can make sure that you don’t miss anything and it will help you connect with other attendees. Also, if you are in the mood to talk technology then I’m open for any discussion, so feel free to drop by the Telerik booth or send me a tweet, @mbcrmp. Thanks for reading and I hope to see you at TechEd 2012.

TechEd 2012–Exchange MVP J. Peter Bruzzese Talks Sessions and Deloreans

0
0

Editor's Note: The following post is by Exchange MVP J. Peter Bruzzese

What I am looking forward to most at Tech Ed?

Every year the event that builds up in me the most anticipation is TechEd. And this year I’m more excited than ever before. There are so many product releases between this year and next that I cannot wait to see the incredible sessions coming up. TechEd is a combination of great sessions, great speakers, a great expo and one big party. Let me tell you what I’m most excited about this year.

The Sessions and Speakers

I always attend the Keynote and it is usually a lot of fun. In 2007 they had Christopher Lloyd come out with the DeLorean from Back to the Future. Other shows have been equally interesting (although that was my favorite so far). So I’m looking forward to seeing what this year’s “theme” will be. This year the keynote speakers are Nadella (President, Server & Tools Business), Zander (Corporate Vice President, Visual Studio) and Leblond (Corporate Vice President, Windows Web Services). An interesting mixture. We’ll see what they come up with.

I plan on hitting every session I can on Windows 8, Server 8 and System Center. With Windows and Server 8 these are new solutions with new features that I absolutely must know to do my job as a consultant, a journalist and a speaker. There are two Windows 8 sessions I’m hoping to attend, one called “Windows 8: Windows To Go Overview” and another entitled “Windows 8: Overview for the Enterprise – Enabling Flexible Workstyles”. System Center has been this mystery to me since it arrived on the scene and although I’ve worked with some aspects of it like System Center Operations Manager and System Center Data Protection Manager I don’t really have my head around the entire suite of tools. So I’m especially going to focus on anything relating to System Center, with Configuration Manager getting much of my time.

Beyond that I’m looking forward to seeing sessions from Microsoft speakers and fellow MVPs like Greg Shields (who has 3 sessions with one entitled “Lessons from the Field: 22 VDI and RDS Mistakes You’ll Want to Avoid), Don Jones (who has 3 sessions with one entitled “Windows PowerShell Crash Course), Mark Minasi (who has 2 sessions with one entitled “Clouds and Your Organization: A (Former) Professional Economist’s View for IT Pros”),Paula Januszkiewicz (who has 3 sessions with one entitled “Crouching Admin, Hidden Hacker: Techniques for Hiding and Detecting Traces”), and Rhonda Layfield with the session “Everything You Need to Know about Microsoft Deployment Toolkit 2012 in the Real World”.

In addition, I’m looking forward to giving my session on Thursday entitled “Getting Microsoft Exchange and SharePoint to Play Together” which is a basic discussion on how to use Exchange to send email to lists and libraries as well as receive alerts back from SharePoint. We’ll look at both paths, that of using Exchange and SharePoint as separate servers and that of using them with Small Business Server 2011.

There are so many sessions and so many great speakers that it’s impossible to list them all here. At the Orange County Convention Center (where TechEd 2012 is going to be held this year) it’s a challenge to get from one session to another if they are on opposite ends of the venue. I’ll be wearing good shoes with lots of cushiony support and I recommend you do the same. I’ll be running to get in as many sessions as possible.

Now as an Exchange MVP you would think that I’m also pretty excited about anything that revolves around Exchange and to a degree I am. There are some great sessions revolving around both Exchange and Lync, like Microsoft Exchange Server 2010 SP2 Tips and Tricks with Scott Schnoll. Bharat Suneja is giving a session called “Archiving in the Cloud with Exchange Online Archiving (EOA)”. There will be plenty of great sessions like “Lync and the Enterprise Network” by Bryan Nyce and “Understanding Microsoft Forefront Online Protection for Exchange” with Robert Gillies. I’ll tell you it is exciting just going through the list!!! However, with the Microsoft Exchange Conference (http://www.mecisback.com/) just around the corner in September and the promise of lots of E15 content, I can tell you that I’m going to try and get in as many sessions on other topics this time around and gear up for a smorgasbord of Exchange at MEC.

The Expo, the Parties, the Networking… the Theme Parks?

TechEd is more than educational. It’s one big party for me from beginning to end. I’m running into friends I haven’t seen in a while (possibly since last TechEd) and grabbing a drink, a lunch, a dinner, whatever. The Expo is an incredible opportunity for me to see the latest products from the top companies in the world as well as start-ups that are hitting the expo floor in the hopes that this may be their “big break” with a product nobody has seen yet. I use that cornucopia of products to formulate ideas for my Enterprise Windows column on InfoWorld. And it gives me a great opportunity to network with folks who are doing new and exciting things with Windows.

And Microsoft makes sure they keep the party flowing all the way to the end, which this year is at Universal’s Islands of Adventure theme park. They’ve had it there before when they have TechEd in the Orlando area and it is always a lot of fun running around the park after the visitors have left and it is just TechEd-ites.

That’s it in a nutshell folks. The best sessions, speakers, expo and parties once could want all crammed into a week down in sunny Orlando Florida. I’ll exhausted by the end but I’ll have a great deal to focus on and write about in the months ahead.

Maybe I’ll see you there!

About the Author:

image

J. Peter Bruzzese has an extensive record in consulting with companies like Goldman Sachs, Solomon Smith Barney, CommVault Systems, and Microsoft, to name a few. J. Peter is the co-founder of ClipTraining, provider of the ClipTraining LMS and ClipTraining Library, which is designed to assist users in learning more about Windows and Office. (www.cliptraining.com)

MVP–Friday Five June 8, 2012

0
0

1. Force Metro App Update Check in Windows 8 Release Preview Windows Store

By Windows Expert – Consumer MVP Richard Hay -@WinObs

This article covers how the Windows Store periodically checks for updates to any Metro Apps you have installed on your system but there maybe times that you would rather get the update a little quicker.

2. The 600 Million Windows 7 Mark

By Windows Expert – Consumer MVP Mike Halsey - @halseymike

This article details the popularity of Windows 7

3. Display tips in a status bar instead of a tooltip in C#

By Visual Basic MVP Rod Stephens

This article covers the methods for providing hints in a status bar instead of a tooltip in C#

4. Microsoft Silverlight 5 and Windows Azure Enterprise Integration

By Windows Azure MVP Neil Mackenzie - @mknz

This article includes reading materials on Windows Azure and Silverlight 5

5. Canadians at TechEd 2012! Join us for a drink, Eh?

By Windows Expert IT Pro MVP Mitch Garvis - @MGarvis

Calling all Canadians going to TechEd 2012


SkyDrive Privacy Settings at a Glance

0
0

Editor's Note: The following MVP Monday post is by Windows Expert - Consumer MVP Arun Kumar

SkyDrive Privacy Settings at a Glance

Microsoft SkyDrive is not just a cloud storage and collaboration service. You can use SkyDrive integrated with Office Web Applications to create and edit documents, presentations and spreadsheets. You do not need a local copy of Microsoft Office to create, view or edit the above file types.

Microsoft SkyDrive Privacy Policy

Before proceeding to study the privacy settings in SkyDrive, here is a summary of the privacy policy at Microsoft when it comes to the cloud.

With some cloud service providers, there are ambiguities as to who owns your files when you upload it to the storage space they offer. This doubt is there because they say they get the rights automatically and that they can modify and distribute the content you use via their cloud services.

Coming to SkyDrive, Microsoft does not claim ownership of the content (files) you upload to the cloud. You are the owner and you get to decide what to do with the files you upload.

If you choose to share content in the public area (with everyone) you acknowledge that anyone can use the content. If you chose to decide that you want to share the content with only few known people, you get to share only with those people. You need to know that the people with whom you share your data (files) may reuse the content. Microsoft also provides for sharing content with none – that is, you get to share the content only with you and none else.

Microsoft stores your personal data like login information. It says it needs the data for providing better service, which in this case, is offering a secure service. In its own words, “You understand that Microsoft may need, and you hereby grant Microsoft the right, to use, modify, adapt, reproduce, distribute, and display content posted on the service solely to the extent necessary to provide the service.” This means Microsoft may sometimes access your files and if it does, it will notify you about the same. Microsoft will access your data only on certain occasions and only to the extent required to provide you with the service. In other words, when you click open a file, Microsoft applications will access it to display the file contents to you and to the people with whom you share the content.

Privacy Settings in Microsoft SkyDrive

When you create a new account with Microsoft SkyDrive, you get three folders already there – Documents, Shared Documents and Photos. By default, the privacy setting for Shared Documents is “Share with everyone”. The privacy setting for “Document” is “Just Me”. This means the content in Shared Documents folder can be viewed (not edited) by everyone and the contents in Documents folder can be viewed (and edited) only by you – the owner of the account.

You can view the share settings of any folder by clicking on the folder name. When the folder loads, you can view the share settings towards the right corner of the screen.

clip_image001

Fig: View Privacy Settings Under Sharing

Inheritance of Privacy Settings in SkyDrive

When you create a file or sub-folder in any folder on the SkyDrive, the file or folder created automatically inherits the share settings of the parent folder. For example, if you create a file in “Shared Documents”, the file’s privacy settings will be “Share with everyone”. Similarly, when you create a folder inside “Documents”, the privacy settings will be “Just Me”.

For each file and folder, you can get two types of links:

1. Link to View only

2. Link to View and Edit

In addition to the privacy setting of the files and folders you wish to share, you can use these links and share via email or social media.

Changing the Privacy Settings in SkyDrive

For any file or folder that you create in the root directory of SkyDrive, the default privacy setting is “Just Me”. To change the privacy setting, click the checkbox preceding the file or folder name. Go to the right corner and under “Sharing”, click “Share”. You will get a dialog box that allows you to:

1. Email links to people (with an option to let them edit the file)

2. Share using social media sites (with an option to let them edit the file)

3. Create a link to let others view OR edit OR share the file with “Everyone” (view only).

clip_image002

Fig: Creating a Link to Let People View and Edit

To send an email, use the first option “Send Email” and enter the recipients’ address in the space provided. Optionally, you can select “Allow Editing” to let the recipients edit the file.

To share using a link to view only, click “Get a Link”. Towards the right side of the box, click “Create” under “View Only”. A text box appears with a link that you can copy to clipboard for sharing. Click the Create under “View and Edit” to get a link that allows people to view and edit the file. The last option is “Make Public”. Click the button to “Share with Everyone”. Note that the “Make Public” allows only viewing and not editing. If you want people to edit your file, you need to share the link you created under “View and Edit”.

The above explains SkyDrive Privacy Policy and Privacy Settings. If you still have any doubts or wish to add anything, please feel free to contact me.

Autho’s Bio

Arun Kumar

Arun Kumar loves writing on Microsoft products and small business. He is Microsoft MVP for Windows Expert–Consumer. He runs a SEO consultancy based in Hyderabad. You can contact him at www.powercutmedia.in

 

About MVP Mondays

The MVP Monday Series is created by Melissa Travers. In this series we work to provide readers with a guest post from an MVP every Monday. Melissa is a Community Program Manager for Dynamics, Excel, Office 365, Platforms and SharePoint in the United States. She has been working with MVPs since her early days as Microsoft Exchange Support Engineer when MVPs would answer all the questions in the old newsgroups before she could get to them.

melissa 

Talking Cloud Tuesday with Visual C# MVP Marcel Meijer

0
0

 

How long have you been an MVP?

Second year

Which technical community or communities are you most active in?

SDN (www.sdn.nl): where I organize events and magazines, enabling others to write and present.

Idem http://www.code-camp.nl

http://blogs.msmvps.com/marcelmeijer/

http://www.marcelmeijer.net

http://www.dotnetmag.nl

What’s the best technical tip you have today for implementing a cloud deployment?

Start small and get a feeling with what cloud means

When considering using the Cloud, what do you tell people if they aren’t sure about moving to the Cloud?

Just do it!  I usually talk to ISV’s and IT is not there core business. Windows Azure can help them to skip parts of the IT part.

Others I ask how flexible their current IT is and how frustrating that can be some times.

Do you have a blog/website link to Cloud related Tips or deployment stories you would like to share?

http://blogs.msmvps.com/marcelmeijer/

http://www.marcelmeijer.net

Speaking from your experience, what words of advice do you have for new MVP’s?

First of all, just do it. Second, talk about a subject you know and present it with passion. And if you don’t know, you don’t know. It is like presenting for a customer ;-)

TechEd North America–First Impressions

0
0

Editor’s note:  The following post was written by Windows Azure MVP Brent Stineman

Just got out of the second day keynote for TechEd North America in sunny and almost unbearably muggy (at least for this Minnesotan) Orlando, Fl. Its my first trip to TechEd and honestly the experience has been awesome. I’ve only attended a couple sessions so far, mainly because I’ve been either networking with folks I know, a few new folks, or best yet helping staff the Windows Azure booths for Microsoft.

The Keynotes

There are only two keynotes for this event, day 1 and day 2. The Monday keynote was all about cloud. Public and private. I’d seen all the public cloud stuff last week online during the Meet Azure live event. But the private cloud gave me much to chew on.

First thing I need to do is give credit to the System Center and Server 2012 teams. About a year ago, I posted a blog post about missed opportunities. Essentially, I didn’t believe that Microsoft’s “private cloud” was really that. It was just another example of cloud washing. Well I was WRONG. Microsoft’s private cloud has the features that are IMHO critical: resource pooling, redundancy/failover, automated resource management, and most importantly self service. While it doesn’t have the seemly deployment model that is present in Windows Azure (yet), it does give us the ability to easily provision and deploy virtual machines. Add to it some of the “infrastructure glue” things that aren’t really my expertise and you have a solution that’s really competitive with VMWare. It will take some time to change minds enough to get people to leave existing investments, but Microsoft has definitely gotten serious about competing with them.

Day two was all about Windows 8. I learned a couple UI tricks which addressed some issues I had with the platform (the mail app still needs SMTP/POP support) as well as got a better idea of what the WOA (Windows on ARM) experience will be like. Yes, you WILL be able to manage and trust ARM devices from your enterprise, yes you will be able to have internal/private app stores, yes you will still have a desktop mode. Not any real clarification though on building apps for ARM beyond what’s already been published regarding WinRT. But that will hopefully come in time. Still, the future it bright and I think consumers are going to really like Windows8. Unfortunately, beyond providing support for BYOD (bring your own device), I can’t see anything compelling enough to force the enterprise to migrate, especially if they already migrated to Windows 7.

Staffing the Booth

I spent about 4.5hrs yesterday staffing the “Migrating Applications” Windows Azure booth. The last time I manned a convention booth like this was at E3 when they launched the XBox. It was much easier than that experience. Partially because TechEd is about 90% less of a zoo, and partially because I know the product. Or at least I knew it well enough yesterday to help answer folks questions about migration of applications.

I met some great folks, many of which I shared some experiences with. There were a couple folks from the University of Iowa (my brother is an alum there and I grew up less than an hour from the campus). They worked in the civil engineering area and had some great questions about data and Homeland Security requirements. They were the only real stumper I had during the shift and were still great to talk to and very understanding that I couldn’t directly answer the question. But they appreciated the compliance knowledge I did have.

What really surprised me was how much interest there was in moving apps. The message the Windows Azure is ready for prime time is really coming through. One person even looked at me and said “Windows Azure is everywhere, I can’t ignore it”.

I’m back at the booth this afternoon for my final 4.5hr shift. Between the keynote and writing this blog post, I won’t really make any sessions today but It’s a small price to pay for the great opportunity to network with the folks at Microsoft and potential cloud adopters. I’ve never been asked for and handed out so many business cards.

Sessions

I’ve only squeeze in two sessions for far. One was Scott Gu on Windows Azure (pretty much what we saw last week), and the other was a System Center MVP from Belgium on creating a private cloud in 75 minutes. There’s no way I could do it that fast, but I could follow along enough to see how complete the picture was for Microsoft’s Private cloud.

Both sessions were great, but I’m looking forward to more. Tomorrow and Thursday will definitely be stacked with great info so I’m going to reexamine my planned sessions. There’s not much new for Windows Azure for me here, but there is definitely a lot to learn about Microsoft’s Private Cloud and the potential for Windows 8.

Until next time!

TechEd Week-in Review from Windows Expert–Consumer MVP Hal Hostetler

0
0

Editor’s note:  The following post was written by Windows Expert – Consumer MVP Hal Hostetler.

Let me start out by saying this is an experience of a lifetime! If you’ve never been to Teched, its Disneyland, the Superbowl, and the World Series all wrapped up into a 4 day extravaganza! There are sessions on all things Microsoft, vendors from all over the world and the coolest folks you’d ever want to meet! I’m here helping out at the Technical Communities booth representing the MVP community. We’re running a photo shoot where folks can stop by and have their picture taken above a cutout of their favorite community star, MVP, Community Contributor, Answers Geek, etc. We’re also answering questions attendees have about the Answers, TechNet, and MSDN forums, the MVP program and what it entails, and how folks can both benefit and help others through their involvement.

where we are teched   teched fun3   technical communities teched 

Monday started out with a problem to solve. We had an issue with a networked printer that wouldn’t print. Alas, the printer didn’t realize that MVPs were present, it was quickly beaten into submission and pictures would print again. I should add that the solution to the printing issue was found in both the Answers and TechNet forums! I’m impressed by the number of people here! I’ve only been to one other TechEd NA and this is at least as big. If you’ve never been to one, the most important Item in your packing list should be comfortable shoes; you will be walking a LOT! Attire is your choice; I’ve seen everything from suits to T-shirts with cargo shorts. In spite of the size, there is a friendly, almost family atmosphere; lots of smiles and happy folks in every direction.

Tuesday saw the Internet become intermittent; another quick fix, the Cat5 connector feeding Internet into our local router came partway out. The crowds in the Expo area are a little smaller today, but surge to life again at the end of each session. Anyone seen the movie “Real Steel”? TechEd has its own version of boxing robots, have a look! If you’re here, you can be Rocky Balboa during lunchtime! Of course, you can always come by the Technical Communities booth and have your picture taken. If you’d like to see what we’re doing, check out the Facebook page.

Wednesday crowds came back to life and it’s been busy in the booth. The hardware misbehaved a bit, but that sort of thing is normal for something set up on a huge display floor with oodles of traffic. I’ve spoken to a number of folks and I’m hearing wonderful news about the sessions. I have to agree with them, I’ve looked in at a few of the sessions and have learned a bunch already. There is something for everyone here, Developers, IT Pros, User Groups, Forum Junkies, EVERYONE! The sessions I looked at were for Windows 8 and Office 365…Excellent stuff! I can’t wait to get my hands on Windows 8! I’ve downloaded and worked with both the Consumer Preview and the Release Preview; both ROCK and I LOVE Office 365! Having built and cared for three Exchange Servers, keeping them updated, keeping the client pieces updated, Office 365 is a gift from heaven! No more updates, patches, arguments with AV/malware applications, it just works and it’s always current! Any Windows Phone enthusiasts out there? This is hog heaven for you folks! Every manufacturer is here along with developer tools, user sessions, the whole enchilada. Now, if my cellular carrier would just get the Windows Phone I want…

We’ve made it to Thursday, the last day of the event. It looks like we finally have the bugs worked out of the photo/print system as it has performed flawlessly all morning. This is a mixed emotions day for me; part of me is exhausted and really wants to go home, the rest of me is sad to see it end because of the fun, the learning, and, most of all, the community! Folks from all over the world have stopped by the Technical Communities booth; they enter with smiles and leave with bigger smiles. They ask questions and get answers, their problems get solved, and they leave happy with pictures to prove it. All in all, one of the most memorable and informative events I’ve ever attended. We’ll be packing things up in an hour or so, getting everything ready for shipment to TechEd Europe in two weeks. I’m overjoyed to report that I’ll be there! If you’re attending TEE, be sure to stop by the booth, have your picture taken, and let us show you how the Answers, TechNet, and MSDN communities can make your life a whole lot easier.

MVP–Friday Five June 15, 2012

0
0

 

1. Ridiculously Fast and Easy Migration to the Cloud with Windows Azure

By Windows Azure MVP David Pallman

This article covers tips on how to easily migrate to the Cloud

2. Considering migrating from existing technology to SharePoint 2010?

By Sharepoint Server MVP Dave Coleman - @davecoleman146

This article details factors to consider when migrating to SharePoint 2010

3. Windows Server 2012 Hyper-V Live Migration Changes

By Virtual Machine MVP Aidan Finn - @joe_elway

This article covers thoughts from Jeff Woolsey’s presentation at TechEd 2012

4. Visualizing Transaction Isolations For SQL Server

By SQL Server MVP Michael Swart - @MJSwart

A fun post about visualization for SQL Server

5. Windows Phone Application Life Cycle and Fast Application Switching

By Connected System Developer MVP Dhananjay Kumar - @debug_mode

This post covers Windows Phone terminology

Hierarchies: Convert Adjacency List to Nested Sets

0
0

Editor's Note: The following MVP Monday post is by SQL Server MVP Jeff Moden and is part of our special series on SQL Server 2012.

Introduction

If you search the Internet for articles on hierarchical data in SQL Server, you’ll find that there are many types of hierarchies and hierarchical structures to store them in database. For most of us, a “Hierarchy” is usually something that looks like an Organizational Chart and is usually stored in one of three different basic hierarchical structures. Even at that level of “simplicity”, hierarchies and how to use them in T-SQL are still huge subjects that I could never hope to do justice to in just one relatively short article.

This article covers a lot of different and sometimes fairly advanced techniques in a very short time and I have to assume certain prior knowledge on your part. For example, I have to assume you know a bit about recursive CTEs (rCTEs), what a numbers or Tally table is and how it can be used to replace certain loops, and even a fair bit about the 3 hierarchical structures included in this article.

This article isn’t about how to use each of the three hierarchical structures. It’s really about how to convert an Adjacency List to Nested Sets.

With that thought in mind, we’re going to blast through a quick reminder of what three common basic types of hierarchies are and then go right into a very high speed method for converting an Adjacency List to the hierarchical heaven of Nested Sets.

The Adjacency List

This is probably the most common type of hierarchical structure there is in databases. Part of the reason for its popularity is that it’s easy for humans to relate to and maintain. This is also where we start with data for this article.

Organizational Chart

clip_image002
One picture is worth a thousand words. Here’s the organizational chart of the data that we’ll be working with.

Each box (known as a “Node”) contains several different pieces of information.

  • Line 1 in each box contains the name of an employee.
  • Line 2 contains just what it says, the EmployeeID which is also the “ChildID”.
  • Line 3 contains the ManagerID which is also the “ParentID” for the given EmployeeID.
  • Line 4 contains 3 items. From left to right, they are the “LeftBower, a Sales amount, and the “RightBower”.

The “LeftBower” and “RightBower” are the “coordinates”, “addresses”, or “anchors” for the Nested Set hierarchy structure. I just couldn’t bring myself to calling them “lft” and “rgt” like many people do. The term “Bower”, in this case, is a nautical term. On large ships, the two large anchors on the bow are called “Bowers” and seem to be quite appropriate here.

The Example Table

Just to keep everyone safe during experiments, we’ll create the example table in TempDB. This table will contain the data from the organizational chart above stored as an Adjacency List..

What makes this an “Adjacency List” is the fact that the EmployeeID (ChildID) and the ManagerID (ParentID) for any given employee are both in the same row or “adjacent” to each other.

Here’s the code to build the example table in TempDB. It doesn’t contain all of the possible constraints or other checks you might want to put on such a table because I wanted to keep it simple for this article. It also doesn’t contain anything to prevent a “cycle” (interpret that as “never ending cycle”). It does contain some fairly useful indexes, though.

/*************************************************************************************

Create an "Adjacency List" Hierarchical Model

*************************************************************************************/

--===== Since we're going to drop and create tables, do this in a nice safe place that

-- everyone has.

USE tempdb

;

--===== Conditionally drop Temp tables to make reruns easy

IF OBJECT_ID('dbo.Employee','U') IS NOT NULL

DROP TABLE dbo.Employee

;

--===== Create the test table with a clustered PK

CREATE TABLE dbo.Employee

(

EmployeeID INT NOT NULL,

ManagerID INT NULL,

EmployeeName VARCHAR(10) NOT NULL,

Sales INT NOT NULL,

CONSTRAINT PK_Employee PRIMARY KEY CLUSTERED (EmployeeID),

CONSTRAINT FK_Employee_Employee FOREIGN KEY (ManagerID)

REFERENCES dbo.Employee (EmployeeID)

)

;

--===== An Index to help speed things up

CREATE INDEX IX_Employee_Composite01

ON dbo.Employee (ManagerID, EmployeeID, EmployeeName)

;

The Example Data

Here’s the code to populate the above Adjacency List table using the same data that we have represented in the organizational chart from the beginning of this section.

--===== Populate the test table with test data. Each row forms a parent/child
-- relationship which is why it's called an "Adjacency" Model.
INSERT INTO dbo.Employee
(EmployeeID, ManagerID, EmployeeName, Sales)
SELECT 1,NULL,'Jim' ,200000 UNION ALL
SELECT 2, 1,'Lynne' , 90000 UNION ALL
SELECT 3, 1,'Bob' ,100000 UNION ALL
SELECT 6, 17,'Eric' , 75000 UNION ALL
SELECT 8, 3,'Bill' , 80000 UNION ALL
SELECT 7, 3,'Vivian' , 60000 UNION ALL
SELECT 12, 8,'Megan' , 50000 UNION ALL
SELECT 13, 8,'Kim' , 55000 UNION ALL
SELECT 17, 2,'Butch' , 70000 UNION ALL
SELECT 18, 39,'Lisa' , 40000 UNION ALL
SELECT 20, 3,'Natalie', 40000 UNION ALL
SELECT 21, 39,'Homer' , 30000 UNION ALL
SELECT 39, 1,'Ken' , 90000 UNION ALL
SELECT 40, 1,'Marge' ,120000
;

The greatest advantage to Adjacency Lists is that they’re both easy and intuitive for humans to understand and maintain. For example, to move “Bob” and his entire sub-tree of 5 other people to all fall under Marge, all that needs to be done is to change Bob’s ManagerID from 1 to 40 and we’re done. To replace Bob, all that we need to do is change the Name and EmployeeID in Bob’s node to the Name and EmployeeID of the new person and change the ManagerId of the direct-reports to the new person. Adding new employees in new positions is usually just as easy or easier.

The greatest disadvantage to using Adjacency Lists is that traversing or “expanding the hierarchy” (as it is also known) is comparatively very slow because it requires some form of looping or recursion to do the traversals. That’s seems fine for the normal single employee sub-tree lookups that a GUI might do but can become quite painful and slow when aggregates and batch runs are involved. Even without aggregations, when many connections are each expanding their own hierarchies, the server can become quite busy in very short order because of all the looping and I/O required.

To summarize, the Adjacency List is great for maintenance but mostly horrible to use both from a code standpoint and a server impact standpoint. In contrast, other hierarchical structures are a lot faster and easier to use but are much more difficult for humans to understand for maintenance. That’s onme of the reasons why converting Adjacency Lists to Nested Sets in as high a performance manner as possible is a common request.

Hierarchical Path

Also widely known as a “Materialized Path”, the Hierarchical Path type of structure is less common than the Adjacency List but is gaining some favor because the HierarchyID data type (available as of SQL Server 2008) can be used to support this type of hierarchical structure. Coverage of how to use the HierarchyID is beyond the scope of this article.

There are actually 2 different types of Hierarchical Path structures. One stores the actual information, such as the EmployeeID, and the other stores positions for each relative to the root (which the HierarchyID data-type supports). We’ll use the type that stores the actual information which again means that we won’t be covering the HierarchyID data-type.

To be brief, the Hierarchical Path isn’t much more than the concatenation of all of the ancestors for a given node. Using a marked up version of the previous organizational chart, here’s what the Hierarchical Path would be for each node. Remember, this path example consists of EmployeeIDs (in Blue in the graphic below).

clip_image004

The next code block below is the code to convert the adjacency list to a Hierarchical Path using a recursive CTE that processes one level of the hierarchy for each iteration. In preparation for what’s coming up, I store the Hierarchical Path as the concatenation of the EmployeeIDs converted to 4 bytes (like an Integer) as a binary “string”. Notice that we not only build the Hierarchical Path, but we calculate the level, we number the nodes in the correct sorted order, and we also create some “place holders” for when we do the conversion to Nested Sets. As you can see by the timings listed in the header, this code is quite fast (although an equivalent While Loop will actually beat it).

For simplicity in typing, I’ve simply called the Hierarchical Path the “SortPath” in this example because sorting on the path will, indeed, list the nodes in the correct expected order.

/*************************************************************************************
Create the Hierarchical Model in a new table to include a lot of the original data as
well as some "place holders" for Bowers and node counts.
(Apologies for the squished code).
(about 00:00:52 on a million rows)
(about 00:00:03 on a 100,000 rows)
*************************************************************************************/
--===== Do this in a nice safe place that everyone has.
USE TempDB
;
--===== Conditionally drop Temp tables to make reruns easy
IF OBJECT_ID('dbo.HierarchyNew','U') IS NOT NULL
DROP TABLE dbo.HierarchyNew
;
--===== Build the new table on-the-fly including some place holders
WITH
cteBuildPath AS
(--==== This is the "anchor" part of the recursive CTE
SELECT anchor.EmployeeID, anchor.ManagerID, HLevel = 1, anchor.Sales,
SortPath = CAST(CAST(anchor.EmployeeID AS BINARY(4)) AS VARBINARY(4000))
FROM dbo.Employee AS anchor
WHERE ManagerID IS NULL --This could be an EmployeeID for a given employee!!!
UNION ALL
--===== This is the "recursive" part of the CTE that adds 1 for each level
-- and concatenates each level of EmployeeID's to the SortPath column.
SELECT recur.EmployeeID, recur.ManagerID, HLevel = cte.HLevel + 1, recur.Sales,
SortPath = CAST(cte.SortPath + CAST(Recur.EmployeeID AS BINARY(4)) AS VARBINARY(4000))
FROM dbo.Employee AS recur
INNER JOIN cteBuildPath AS cte
ON cte.EmployeeID = recur.ManagerID
)--==== This final INSERT/SELECT creates the "narrow surrogate" column (NodeNumber)
-- that we'll be able to sort on as well as populating the new table.
SELECT EmployeeID = sorted.EmployeeID, sorted.ManagerID, sorted.HLevel, sorted.Sales,
LeftBower = CAST(0 AS INT), RightBower = CAST(0 AS INT), --Place holders
NodeNumber = ROW_NUMBER() OVER (ORDER BY sorted.SortPath),
NodeCount = CAST(0 AS INT), sorted.SortPath --Place holder
INTO dbo.HierarchyNew
FROM cteBuildPath AS sorted
;

The big advantage of the Hierarchical Path structure is that you can sort by the path and have it come out in an order that you would reasonably expect of a hierarchy without much work at all. Another advantage is that you can avoid the use loops and recursion to do traversals using a bit of clever code but it’s still slower than a Nested Set.

The big disadvantage is that Hierarchical Path structures are not so easy or intuitive for humans to maintain.

Nested Sets

If you don’t already know, many consider Nested Sets to be the holy grail of hierarchical structures because they’re so very fast during usage.

Quick Review, How Nested Sets Work

Take a look at the following.

clip_image006

If you look at “Bob”, we can see that we can find everyone in Bob’s down line simply by finding everyone who has left and right bowers that are between Bob’s. You can just imagine how fast all of this can be especially when the Bowers are properly indexed. Because it’s such straight forward code, you can also see how easily you can do aggregates and the like and all in a 100% super high speed, set based fashion.

Here’s the code to do a simple “lookup” of Bob’s down line.

--===== Declare and preset a variable to find a given employee

DECLARE @EmployeeIdToFind INT;

SELECT @EmployeeIdToFind = 3 --Bob

;

--===== Find the "Downline" for the given employee

-- INCLUDING that employee.

-- Almost same code with addition of "="

SELECT downline.*

FROM dbo.HierarchyNew tofind

INNER JOIN dbo.HierarchyNew downline

ON downline.LeftBower >= tofind.LeftBower

AND downline.RightBower <= tofind.RightBower

WHERE tofind.EmployeeID = @EmployeeIdToFind

ORDER BY downline.LeftBower

;

In a similar fashion, it’s also very easy to calculate “up lines”. Here’s the code to do a simple “lookup” of Megan’s up line. Notice that the only real code changes are the direction the comparison operators are pointed in. Everything else is the same as the down line code.

--===== Declare and preset a variable to find a given employee
DECLARE @EmployeeIdToFind INT;
SELECT @EmployeeIdToFind = 8 --Megan
;
--===== Find the "Upline" for the given employee
-- INCLUDING that employee.
-- Almost same code with addition of "="
SELECT upline.*
FROM dbo.Employee tofind
INNER JOIN dbo.Employee downline
ON upline.LeftBower <= tofind.LeftBower
AND upline.RightBower >= tofind.RightBower
WHERE tofind.EmployeeID = @EmployeeIdToFind
ORDER BY upline.LeftBower
;

Here’s the graphic for the code above. Notice that we can find someone’s up line simply by finding everyone who has a Left Bower that’s less than Megan’s Left Bower and who also has a Right Bower that’s greater than Megan’s Right Bower.

clip_image008

Since the Bowers can be queried in a set based fashion on indexed columns, there’s no much that can beat Nested Sets for performance during usage and that’s their single big advantage.

They can be, however, horribly slow to create from an Adjacency List especially when using the traditional “push stack” method of creation which basically looks like the following. Note that each Red arrow is a “push” or a “pop” which involves at least one select and at least one insert or delete in a highly RBAR1 fashion.

clip_image010

To avoid such agony, many people will use some extraordinary code to do sub-tree moves and other changes. The code to replace a single employee in a single position is actually easier than it is in an Adjacency list. The code to move a sub-tree requires some pretty fancy code to double the value of some Bowers and cut others in half except for the first node where you must also either add or subtract 1 to the top node of the sub-tree being moved (whew!). Although computers certainly have no problem with all of that, if anything ever goes haywire, humans are going to have one heck of a time figuring out how to straighten the mess out.

Again, it’s much easier for humans to realize, understand, and maintain Adjacency Lists but they really want the speed of usage that Nested Sets provide. The key to it all is to have some method of quickly converting an Adjacency List to Nested Sets.

Converting the Adjacency List to Nested Sets

There are a lot of different methods out there for converting an Adjacency List to Nested Sets… and most of them involve a “push stack” or other form of RBAR1 which makes most of the them much too slow to be practical to use on a regular basis.

Itzik Ben-Gan came up with a brilliant method for doing this type of conversion and you can find it in his book “Inside Microsoft SQL Server 2008: T-SQL Programming” (Microsoft Press). To summarize his method, he doubles the number of rows and then numbers them in a particular fashion.

I wanted to take a different approach. I thought there might be some relatively easy way to number the Bowers of Nested Sets and was fortunate enough to not even have to think about what those formulas might be. I ran into a great article by Adam Machanic.

http://sqlblog.com/blogs/adam_machanic/archive/2006/07/12/swinging-from-tree-to-tree-using-ctes-part-1-adjacency-to-nested-sets.aspx

The formulas he came up with were absolutely brilliant in their simplicity. There was just one catch. One of the formulas needs the number of nodes in the down line for each node for it to work correctly. Adam’s code did that mathematically fine but it also took more than 16 minutes on a pretty good laptop just to convert 20,000 rows because of an accidental Cartesian product in the code. Itzik’s code would convert a million node hierarchy in just a couple of minutes. I was darned sure that Adam’s formulas would do the trick but we had the “Catch 22” thing going here. We needed a hierarchical structure to quickly calculate the down line count of nodes so that we could make a hierarchical structure to count the down line count of nodes (the Nested Sets).

Quickly Counting Down line Nodes

It turns out that we actually have a major part of the solution in the code that we have so far. Let’s take a look at it again.

/*************************************************************************************
Create the Hierarchical Model in a new table to include a lot of the original data as
well as some "place holders" for Bowers and node counts.
(Apologies for the squished code).
(about 00:00:52 on a million rows)
(about 00:00:03 on a 100,000 rows)
*************************************************************************************/
--===== Do this in a nice safe place that everyone has.
USE TempDB
;
--===== Conditionally drop Temp tables to make reruns easy
IF OBJECT_ID('dbo.HierarchyNew','U') IS NOT NULL
DROP TABLE dbo.HierarchyNew
;
--===== Build the new table on-the-fly including some place holders
WITH
cteBuildPath AS
(--==== This is the "anchor" part of the recursive CTE
SELECT anchor.EmployeeID, anchor.ManagerID, HLevel = 1, anchor.Sales,
SortPath = CAST(CAST(anchor.EmployeeID AS BINARY(4)) AS VARBINARY(4000))
FROM dbo.Employee AS anchor
WHERE ManagerID IS NULL --NOTICE that could be EmployeeID for a given employee!!!
UNION ALL
--===== This is the "recursive" part of the CTE that adds 1 for each level
-- and concatenates each level of EmployeeID's to the SortPath column.
SELECT recur.EmployeeID, recur.ManagerID, HLevel = cte.HLevel + 1, recur.Sales,
SortPath = CAST(cte.SortPath + CAST(Recur.EmployeeID AS BINARY(4)) AS VARBINARY(4000))
FROM dbo.Employee AS recur
INNER JOIN cteBuildPath AS cte
ON cte.EmployeeID = recur.ManagerID
)--==== This final INSERT/SELECT creates the "narrow surrogate" column (NodeNumber)
-- that we'll be able to sort on as well as populating the new table.
SELECT EmployeeID = sorted.EmployeeID, sorted.ManagerID, sorted.HLevel, sorted.Sales,
LeftBower = CAST(0 AS INT), RightBower = CAST(0 AS INT), --Place holders
NodeNumber = ROW_NUMBER() OVER (ORDER BY sorted.SortPath),
NodeCount = CAST(0 AS INT), sorted.SortPath --Place holder
INTO dbo.HierarchyNew
FROM cteBuildPath AS sorted
;

Yeah… I know. It took me a while to see it, as well. The key is in the SortPath column itself. The number of people in the down line of each node is easily determined by the SortPath because that EmployeeID will appear that many times in the SortPath column. Take a look at the final table we’re going to end up with to see what I mean.

clip_image012

Look at the Red squares first. That’s EmployeeID #1 (Jim) and he’s at the very top of the 14 node hierarchy. His node count is “14” and, if you look at the “SortPath”, he appears in the same relative position (happens to match the Level, BTW) of all the SortPaths 14 times including himself.

Look at Bob. Bob is EmployeeID #3 and I’ve marked his information with Blue squares. If you’ll recall, Bob had a total of 5 people in his sub-tree and, if you include Bob, Bob has a total node count of 6. Looking at the SortPath, Bob appears in the SortPath (don’t forget this is a binary sort path) a total of 6 times which is the same as his node count.

Splitting Out the Down line Node Count

How are we going to split out that information? Enter my friend and the proverbial Swiss Army Knife of T-SQL, the Tally Table.

The Tally Table (and certain non-recursive CTEs that mimic it) can be used to very effectively replace certain While Loops for things like splitting sort paths. For more information on how to build a Tally Table and how it works, please see the following article.

http://www.sqlservercentral.com/articles/T-SQL/62867/

Once we have a Tally Table built, we can use it to simultaneously split and count the number of times each EmployeeID appears in the SortPath.

If you look at the SortPath in the previous figure, the SortPath is made up of 4 bytes per Level from left to right. The first level starts at byte #1 and goes for 4 bytes. The second level starts at byte #5 and goes for 4 bytes. Each Level progresses in a similar manner.

We can have the Tally Table “count” through the Levels. The “starting position” of each Level in the SortPath can be calculated as (t.N*4)-3 where “t.N” is the number from the Tally Table and is equal to the value of each Level number. Of course, the length for each Level will always be “4”.

Once we have the split of the EmployeeIDs done, all that’s left is to count the number of times each appears. The following code will do that bit of Tally Table magic for us.

SELECT EmployeeID = SUBSTRING(h.SortPath,(t.N*4)-3,4),
NodeCount = COUNT(*) --Includes current node
FROM dbo.HierarchyNew h,
dbo.Tally t
WHERE t.N BETWEEN 1 AND DATALENGTH(SortPath)/4
GROUP BY SUBSTRING(h.SortPath,(t.N*4)-3,4)
;

Adam’s Formulas

All that’s left now is to apply Adam’s formulas and we’ll have our Nested Sets. Using the “node count” code from above, here’s the code to convert the binary coded Hierarchical Path into Nested Sets.

/*************************************************************************************
Split the sort path every 4 bytes to get the EmployeeID (results in dupes, of course)
and do a count on the resulting EmployeeID to get a count of "downline" nodes
including the EmployeeID. Then, convert that information into the Left and Right
Bowers of Nested Sets using Adam's formulas.
(about 00:00:41 on a million rows)
(about 00:00:03 on a 100,000 rows)
*************************************************************************************/
--===== We need this variable to keep from having to make an additional pass to
-- calculate the Right Bowers.
DECLARE @LeftBower INT;
--===== Build the NodeCount and the Bowers
WITH
cteCountNodes AS
(--==== This splits the SortPath column every 4 bytes and counts the number of
-- matching nodes.
SELECT EmployeeID = SUBSTRING(h.SortPath,(t.N*4)-3,4),
NodeCount = COUNT(*) --Includes current node
FROM dbo.HierarchyNew h,
dbo.Tally t
WHERE t.N BETWEEN 1 AND DATALENGTH(SortPath)/4
GROUP BY SUBSTRING(h.SortPath,(t.N*4)-3,4)
)--==== This calculates the Bowers based on the formulas previously discussed.
-- And, no... this is NOT a "Quirky Update" although it is a proprietary
-- update supported in SQL Server.
UPDATE dbo.HierarchyNew
SET NodeCount = cn.NodeCount,
@LeftBower = LeftBower = (hn.NodeNumber * 2) - hn.HLevel,
RightBower = ((cn.NodeCount-1) * 2) + @LeftBower + 1
FROM dbo.HierarchyNew hn
INNER JOIN cteCountNodes cn
ON hn.EmployeeID = cn.EmployeeID
;

Like I said, Adam’s formulas are absolutely brilliant in their simplicity.

The Left Bower is a simple calculation which makes the realization that each node (Numbered as “NodeNumber” in order by the SortOrder column) has two Bowers and you need to subtract the number of incomplete nodes in the upline (including the current node) for the node. That’s easily done simply by subtracting the Level of the current node.

The Right Bower calculation is a bit similar. It starts off with the downline node count not including the current node (that’s where the -1 comes in), mulitplies that by 2 because each node has 2 Bowers, adds all of that to the Left Bower and adds 1 because it’s a new Bower not previously accounted for.

All we needed to get the speed out of all of this was to use the Tally Table to get the downline node counts already available in the SortPath in a high performance manner.

Wrapping It All Up

We covered an awful lot of information in a very short period of time. Let’s quickly review what we got for our efforts.

What we end up with is the best of all 3 worlds. Here’s the final table, again.

clip_image012[1]

We end up with our original Adjacency List (EmployeeID and ManagerID columns) which is easy for us mere humans to understand and maintain.

We end up with our Hierarchical Path information (HLevel, NodeNumber, NodeCount, and SortPath columns) which allowed us to do many things including sort the final output

Last but not least, we also ended up with the LeftBower and RightBower columns of the Nested Sets hierarchical structure. Now we have ease of maintenance and super high performance capabilities for hierarches and we can do it all on a million rows in about a minute and a half and in about 6 seconds on 100,000 rows. That’s fast enough to make it worth simply recalculating everything with some remarkably simple code instead of trying to maintain Nested Sets.

To make life a little easier, here’s all the code we used to do the conversion in one place.

/*************************************************************************************
Create the Hierarchical Model in a new table to include a lot of the original data as
well as some "place holders" for Bowers and node counts.
(Apologies for the squished code).
(about 00:00:52 on a million rows)
(about 00:00:03 on a 100,000 rows)
*************************************************************************************/
--===== Do this in a nice safe place that everyone has.
USE TempDB
;
--===== Conditionally drop Temp tables to make reruns easy
IF OBJECT_ID('dbo.HierarchyNew','U') IS NOT NULL
DROP TABLE dbo.HierarchyNew
;
--===== Build the new table on-the-fly including some place holders
WITH
cteBuildPath AS
(--==== This is the "anchor" part of the recursive CTE
SELECT anchor.EmployeeID, anchor.ManagerID, HLevel = 1, anchor.Sales,
SortPath = CAST(CAST(anchor.EmployeeID AS BINARY(4)) AS VARBINARY(4000))
FROM dbo.Employee AS anchor
WHERE ManagerID IS NULL --NOTICE that could be EmployeeID for a given employee!!!
UNION ALL
--===== This is the "recursive" part of the CTE that adds 1 for each level
-- and concatenates each level of EmployeeID's to the SortPath column.
SELECT recur.EmployeeID, recur.ManagerID, HLevel = cte.HLevel + 1, recur.Sales,
SortPath = CAST(cte.SortPath + CAST(Recur.EmployeeID AS BINARY(4)) AS VARBINARY(4000))
FROM dbo.Employee AS recur
INNER JOIN cteBuildPath AS cte
ON cte.EmployeeID = recur.ManagerID
)--==== This final INSERT/SELECT creates the "narrow surrogate" column (NodeNumber)
-- that we'll be able to sort on as well as populating the new table.
SELECT EmployeeID = sorted.EmployeeID, sorted.ManagerID, sorted.HLevel, sorted.Sales,
LeftBower = CAST(0 AS INT), RightBower = CAST(0 AS INT), --Place holders
NodeNumber = ROW_NUMBER() OVER (ORDER BY sorted.SortPath),
NodeCount = CAST(0 AS INT), sorted.SortPath --Place holder
INTO dbo.HierarchyNew
FROM cteBuildPath AS sorted
;

/*************************************************************************************
Split the sort path every 4 bytes to get the EmployeeID (results in dupes, of course)
and do a count on the resulting EmployeeID to get a count of "downline" nodes
including the EmployeeID. Then, convert that information into the Left and Right
Bowers of Nested Sets using Adam's formulas.
(about 00:00:41 on a million rows)
(about 00:00:03 on a 100,000 rows)
*************************************************************************************/
--===== We need this variable to keep from having to make an additional pass to
-- calculate the Right Bowers.
DECLARE @LeftBower INT;
--===== Build the NodeCount and the Bowers
WITH
cteCountNodes AS
(--==== This splits the SortPath column every 4 bytes and counts the number of
-- matching nodes.
SELECT EmployeeID = SUBSTRING(h.SortPath,(t.N*4)-3,4),
NodeCount = COUNT(*) --Includes current node
FROM dbo.HierarchyNew h,
dbo.Tally t
WHERE t.N BETWEEN 1 AND DATALENGTH(SortPath)/4
GROUP BY SUBSTRING(h.SortPath,(t.N*4)-3,4)
)--==== This calculates the Bowers based on the formulas previously discussed.
-- And, no... this is NOT a "Quirky Update" although it is a proprietary
-- update supported in SQL Server.
UPDATE dbo.HierarchyNew
SET NodeCount = cn.NodeCount,
@LeftBower = LeftBower = (hn.NodeNumber * 2) - hn.HLevel,
RightBower = ((cn.NodeCount-1) * 2) + @LeftBower + 1
FROM dbo.HierarchyNew hn
INNER JOIN cteCountNodes cn
ON hn.EmployeeID = cn.EmployeeID
;

At this point, the Left Bower provides the same sortability features as the NodeNumber and SortPath columns. You could delete them but why bother?

A Real Teaser

Here’s a real teaser for you… if you really think about how we built the NodeCount column to build the Bowers, I’m sure that you can see another article’s worth of information on how to make it so we might not even need Nested Sets for speed, anymore. We could preaggregate everything we need to know about any hierarchy and store it all in our own version of what a hierarchical datamart could look like. Like I said… future article. ;-)

Thanks for listening folks.

--Jeff Moden

RBAR1 is pronounced “ree-bar” like the steel rods stuck in cement forever and is a “Modenism” for “Row By Agonizing Row”. It has come to mean some “slow code” generally attributed to various forms of procedural programming when compared to other methods.

About Jeff Moden

SQL Server MVP Jeff Moden is mostly self taught and is a long time member of and one of the leading posters on SQLServerCentral.com. His articles have made “advanced” concepts such as the use and understanding of the Numbers or Tally Table more like child’s play. He coined the acronym of “RBAR” which is pronounced “ree-bar” and is a “Modenism” for Row-By-Agonizing-Row. It has come be known world round as an easy way to label performance challenged code. He presented at PASS 2010, a couple of SQLSaturdays, and at several local and remote PASS Chapters and User Groups. His passion for the art of SQL and teaching is apparent in his articles and his lectures. Jeff also won the Red Gate sponsored Worldwide Exceptional DBA Award for 2011 and continues to serve the SQL Server Community on a daily basis.

 

About MVP Mondays

The MVP Monday Series is created by Melissa Travers. In this series we work to provide readers with a guest post from an MVP every Monday. Melissa is a Community Program Manager for Dynamics, Excel, Office 365, Platforms and SharePoint in the United States. She has been working with MVPs since her early days as Microsoft Exchange Support Engineer when MVPs would answer all the questions in the old newsgroups before she could get to them.

melissa

Talking Cloud with Internet Explorer MVP Hans Le Roy

0
0

1. How long have you been an MVP?

Since 2003

2. How did you first start in community?

I was asked in Compuserve FRPROG (French Programmers) community, and did some basic VB and HTML courses online in the online community. I guess it was in 1996.

3. Which technical community or communities are you most active in?

Microsoft answers – mainly Internet Explorer forums (in Dutch, French and Spanish). And I manage the IESUPPORT, WINSUPPORT and MSOFORUM at Compuserve.

4. What’s the best technical tip you have today for implementing a cloud deployment?

Make sure you have a reliable synchronization solution, and that the people know what the cloud technology is all about.

5. When considering using the Cloud, what do you tell people if they aren’t sure about moving to the Cloud?

There are far more specialized people busy with the security in the cloud than there can possibly be in your network / at your home.

6. Do you have a blog/website link to Cloud related Tips or deployment stories you would like to share?

I post quite some tips about cloud technology (Skydrive, Office web apps, Mesh and others) for end users on my blog
http://hlrnet.com/technoblog/?s=skydrive
http://hlrnet.com/technoblog/?s=mesh

7. Speaking from your experience, what words of advice do you have for new MVP’s?

Be patient

Ask for details and if you don’t or can’t

Be afraid of boilerplates

Don’t post just a link


MVPs Win Awards for Windows Phone APPs

0
0

The Microsoft Development Platform Evangelist team and Nokia conducted a windows phone APP contest and 8 Taiwan MVPs are winners.  Check out what they came up with!  

Champion - Demo Fan - ASP/ASP.NET

clip_image002

Taiwan Weather

http://www.windowsphone.com/zh-TW/apps/32cb7a0e-d82a-451c-8c7c-00a3ceb6fe8c

 

Champion - Jeffray Huang - ASP.NET/IIS

clip_image004

Easy File Hub 2

http://www.windowsphone.com/zh-tw/apps/68432e49-0d74-44a2-a321-00e07beccbdf

 

Runner Up - Bill Chung - Visual Basic

clip_image006

Ding Ding MatroMarquee

http://www.windowsphone.com/zh-TW/apps/ca7d7ec7-2e71-4725-95f9-73d1473de306

 

Best APP sharing award - Ming En Hsieh - Visual C#

clip_image008http://www.windowsphone.com/zh-TW/apps/902c9eb8-153a-4f71-a17a-d2ef6f3beed3

 

Best APP sharing award - Alex Lee - Visual C#

clip_image010http://www.windowsphone.com/zh-TW/apps/fdfa5a69-2d4b-4064-bc57-2b293dc9d776

 

Best APP sharing award - Hung Chin Tai - Visual C#

clip_image012http://www.windowsphone.com/zh-TW/apps/06527bcf-9efe-4237-a3a7-039af3f1f4a4

 

Best APP contributor award - Hung Hsiang Chen - SQL Server

clip_image014http://www.windowsphone.com/zh-TW/apps/bd69a213-52da-4acf-a2ac-ca4cd77a04ce

 

Best APP contributor award - Xiao Yuan Chuang - Expression Blend

clip_image016

http://www.windowsphone.com/zh-TW/apps/a27c6d82-671a-4516-b658-afb8a36d098b

Friday Five for June 22, 2012

0
0

 

1. jQuery and Clicking an ASP.NET Linkbutton

By Visual C# MVP Michael Sellers

This post is going to cover one common scenario that will impact users that might be using DotNetNuke common styles or working to create their own custom button styles.

2. Evolution of the Windows Phone

By Device Application Development MVP Alexander Viken - @AlexanderViken

This article covers the evolution of the Windows Phone

3. When Twitter Is Down Where Do You Go?

By Windows Expert Consumer MVP Richard Hay - @WinObs

This article shares the frustrations of Twitter outages

4. The Dew Review – DevExpress DXv2 WPF 2012.1

By Visual C# MVP Alvin Ashcraft - @alvinashcraft

Discusses beta release of the WPF controls coming in the exciting new 2012.1 release of DevExpress DXv2

5. Set ASP.NET Label text with Javascript

By  Visual C# MVP Ming Man Chan

TechEd Europe Expectations from Windows Expert–Consumer MVP Hal Hostetler

0
0

Editor’s note:  The following post was written by Windows Expert –Consumer Hal Hostetler

It’s Thursday evening, we leave for Amsterdam Saturday morning, I’m thoroughly stoked, and really looking forward to it! Here are a few of the things that have my attention:

First of all, Amsterdam, itself! I’ve only been to Europe once and that was to attend TechEd Europe in Berlin, Germany in 2010. The trip to Berlin was a real eye opener and I expect nothing less from this journey. My wife and I like to watch travel shows and Amsterdam has been featured on most of them at one time or another. I’ve recorded some of these and moved them to my netbook’s hard drive so we’ll have a list of things to see, places to go, and food to eat when we get there.

Secondly, I’m looking forward to meeting the people attending the event. I had no idea what to expect in 2010 and I discovered that the folks attending TechEd there were very much like those who attend the North American event, friendly, knowledgeable, helpful, and holding a strong sense of community. Indeed, I encountered several folks in Germany that I’d met earlier in the year at TechEd NA in New Orleans. I made many new friends in Germany, most of whom still keep in touch; I anticipate that a number of them will be attending the event in Amsterdam, so I’m looking forward to many glorious reunions.

I’m also looking forward to the event, itself. I learned a great deal at the North American event last week, but there just wasn’t time to attend all the sessions I wanted to see, so I now have a second chance. Since technology marches on, there have been some new products released/updated since last week (Windows Phone 8, for example) that I’ll have the opportunity to learn about. All in all, this is shaping up to be a win-win-win situation. How will it actually go? Stay tuned, folks, I’ll be back with the details a little over a week hence!

Hierarchies: Convert Adjacency List to Nested Sets

0
0

Editor's Note: The following MVP Monday post is by SQL Server MVP Jeff Moden and is part of our special series on SQL Server 2012.

Hierarchies: Convert Adjacency List to Nested Sets

Introduction

If you search the Internet for articles on hierarchical data in SQL Server, you’ll find that there are many types of hierarchies and hierarchical structures to store them in database. For most of us, a “Hierarchy” is usually something that looks like an Organizational Chart and is usually stored in one of three different basic hierarchical structures. Even at that level of “simplicity”, hierarchies and how to use them in T-SQL are still huge subjects that I could never hope to do justice to in just one relatively short article.

This article covers a lot of different and sometimes fairly advanced techniques in a very short time and I have to assume certain prior knowledge on your part. For example, I have to assume you know a bit about recursive CTEs (rCTEs), what a numbers or Tally table is and how it can be used to replace certain loops, and even a fair bit about the 3 hierarchical structures included in this article.

This article isn’t about how to use each of the three hierarchical structures. It’s really about how to convert an Adjacency List to Nested Sets.

With that thought in mind, we’re going to blast through a quick reminder of what three common basic types of hierarchies are and then go right into a very high speed method for converting an Adjacency List to the hierarchical heaven of Nested Sets.

The Adjacency List

This is probably the most common type of hierarchical structure there is in databases. Part of the reason for its popularity is that it’s easy for humans to relate to and maintain. This is also where we start with data for this article.

Organizational Chart


One picture is worth a thousand words. Here’s the organizational chart of the data that we’ll be working with.

image

Each box (known as a “Node”) contains several different pieces of information.

  • Line 1 in each box contains the name of an employee.
  • Line 2 contains just what it says, the EmployeeID which is also the “ChildID”.
  • Line 3 contains the ManagerID which is also the “ParentID” for the given EmployeeID.
  • Line 4 contains 3 items. From left to right, they are the “LeftBower, a Sales amount, and the “RightBower”.

The “LeftBower” and “RightBower” are the “coordinates”, “addresses”, or “anchors” for the Nested Set hierarchy structure. I just couldn’t bring myself to calling them “lft” and “rgt” like many people do. The term “Bower”, in this case, is a nautical term. On large ships, the two large anchors on the bow are called “Bowers” and seem to be quite appropriate here.

The Example Table

Just to keep everyone safe during experiments, we’ll create the example table in TempDB. This table will contain the data from the organizational chart above stored as an Adjacency List..

What makes this an “Adjacency List” is the fact that the EmployeeID (ChildID) and the ManagerID (ParentID) for any given employee are both in the same row or “adjacent” to each other.

Here’s the code to build the example table in TempDB. It doesn’t contain all of the possible constraints or other checks you might want to put on such a table because I wanted to keep it simple for this article. It also doesn’t contain anything to prevent a “cycle” (interpret that as “never ending cycle”). It does contain some fairly useful indexes, though.

image
The Example Data

Here’s the code to populate the above Adjacency List table using the same data that we have represented in the organizational chart from the beginning of this section.

image

The greatest advantage to Adjacency Lists is that they’re both easy and intuitive for humans to understand and maintain. For example, to move “Bob” and his entire sub-tree of 5 other people to all fall under Marge, all that needs to be done is to change Bob’s ManagerID from 1 to 40 and we’re done. To replace Bob, all that we need to do is change the Name and EmployeeID in Bob’s node to the Name and EmployeeID of the new person and change the ManagerId of the direct-reports to the new person. Adding new employees in new positions is usually just as easy or easier.

The greatest disadvantage to using Adjacency Lists is that traversing or “expanding the hierarchy” (as it is also known) is comparatively very slow because it requires some form of looping or recursion to do the traversals. That’s seems fine for the normal single employee sub-tree lookups that a GUI might do but can become quite painful and slow when aggregates and batch runs are involved. Even without aggregations, when many connections are each expanding their own hierarchies, the server can become quite busy in very short order because of all the looping and I/O required.

To summarize, the Adjacency List is great for maintenance but mostly horrible to use both from a code standpoint and a server impact standpoint. In contrast, other hierarchical structures are a lot faster and easier to use but are much more difficult for humans to understand for maintenance. That’s onme of the reasons why converting Adjacency Lists to Nested Sets in as high a performance manner as possible is a common request.

Hierarchical Path

Also widely known as a “Materialized Path”, the Hierarchical Path type of structure is less common than the Adjacency List but is gaining some favor because the HierarchyID data type (available as of SQL Server 2008) can be used to support this type of hierarchical structure. Coverage of how to use the HierarchyID is beyond the scope of this article.

There are actually 2 different types of Hierarchical Path structures. One stores the actual information, such as the EmployeeID, and the other stores positions for each relative to the root (which the HierarchyID data-type supports). We’ll use the type that stores the actual information which again means that we won’t be covering the HierarchyID data-type.

To be brief, the Hierarchical Path isn’t much more than the concatenation of all of the ancestors for a given node. Using a marked up version of the previous organizational chart, here’s what the Hierarchical Path would be for each node. Remember, this path example consists of EmployeeIDs (in Blue in the graphic below).

image

The next code block below is the code to convert the adjacency list to a Hierarchical Path using a recursive CTE that processes one level of the hierarchy for each iteration. In preparation for what’s coming up, I store the Hierarchical Path as the concatenation of the EmployeeIDs converted to 4 bytes (like an Integer) as a binary “string”. Notice that we not only build the Hierarchical Path, but we calculate the level, we number the nodes in the correct sorted order, and we also create some “place holders” for when we do the conversion to Nested Sets. As you can see by the timings listed in the header, this code is quite fast (although an equivalent While Loop will actually beat it).

For simplicity in typing, I’ve simply called the Hierarchical Path the “SortPath” in this example because sorting on the path will, indeed, list the nodes in the correct expected order.

image

The big advantage of the Hierarchical Path structure is that you can sort by the path and have it come out in an order that you would reasonably expect of a hierarchy without much work at all. Another advantage is that you can avoid the use loops and recursion to do traversals using a bit of clever code but it’s still slower than a Nested Set.

The big disadvantage is that Hierarchical Path structures are not so easy or intuitive for humans to maintain.

Nested Sets

If you don’t already know, many consider Nested Sets to be the holy grail of hierarchical structures because they’re so very fast during usage.

Quick Review, How Nested Sets Work

Take a look at the following.

blog12

If you look at “Bob”, we can see that we can find everyone in Bob’s down line simply by finding everyone who has left and right bowers that are between Bob’s. You can just imagine how fast all of this can be especially when the Bowers are properly indexed. Because it’s such straight forward code, you can also see how easily you can do aggregates and the like and all in a 100% super high speed, set based fashion.

Here’s the code to do a simple “lookup” of Bob’s down line.

blog11

In a similar fashion, it’s also very easy to calculate “up lines”. Here’s the code to do a simple “lookup” of Megan’s up line. Notice that the only real code changes are the direction the comparison operators are pointed in. Everything else is the same as the down line code.

blog10

Here’s the graphic for the code above. Notice that we can find someone’s up line simply by finding everyone who has a Left Bower that’s less than Megan’s Left Bower and who also has a Right Bower that’s greater than Megan’s Right Bower.

blog9

Since the Bowers can be queried in a set based fashion on indexed columns, there’s no much that can beat Nested Sets for performance during usage and that’s their single big advantage.

They can be, however, horribly slow to create from an Adjacency List especially when using the traditional “push stack” method of creation which basically looks like the following. Note that each Red arrow is a “push” or a “pop” which involves at least one select and at least one insert or delete in a highly RBAR1 fashion.

blog8

To avoid such agony, many people will use some extraordinary code to do sub-tree moves and other changes. The code to replace a single employee in a single position is actually easier than it is in an Adjacency list. The code to move a sub-tree requires some pretty fancy code to double the value of some Bowers and cut others in half except for the first node where you must also either add or subtract 1 to the top node of the sub-tree being moved (whew!). Although computers certainly have no problem with all of that, if anything ever goes haywire, humans are going to have one heck of a time figuring out how to straighten the mess out.

Again, it’s much easier for humans to realize, understand, and maintain Adjacency Lists but they really want the speed of usage that Nested Sets provide. The key to it all is to have some method of quickly converting an Adjacency List to Nested Sets.

Converting the Adjacency List to Nested Sets

There are a lot of different methods out there for converting an Adjacency List to Nested Sets… and most of them involve a “push stack” or other form of RBAR1 which makes most of the them much too slow to be practical to use on a regular basis.

Itzik Ben-Gan came up with a brilliant method for doing this type of conversion and you can find it in his book “Inside Microsoft SQL Server 2008: T-SQL Programming” (Microsoft Press). To summarize his method, he doubles the number of rows and then numbers them in a particular fashion.

I wanted to take a different approach. I thought there might be some relatively easy way to number the Bowers of Nested Sets and was fortunate enough to not even have to think about what those formulas might be. I ran into a great article by Adam Machanic.

http://sqlblog.com/blogs/adam_machanic/archive/2006/07/12/swinging-from-tree-to-tree-using-ctes-part-1-adjacency-to-nested-sets.aspx

The formulas he came up with were absolutely brilliant in their simplicity. There was just one catch. One of the formulas needs the number of nodes in the down line for each node for it to work correctly. Adam’s code did that mathematically fine but it also took more than 16 minutes on a pretty good laptop just to convert 20,000 rows because of an accidental Cartesian product in the code. Itzik’s code would convert a million node hierarchy in just a couple of minutes. I was darned sure that Adam’s formulas would do the trick but we had the “Catch 22” thing going here. We needed a hierarchical structure to quickly calculate the down line count of nodes so that we could make a hierarchical structure to count the down line count of nodes (the Nested Sets).

Quickly Counting Down line Nodes
It turns out that we actually have a major part of the solution in the code that we have so far. Let’s take a look at it again.

blog7

Yeah… I know. It took me a while to see it, as well. The key is in the SortPath column itself. The number of people in the down line of each node is easily determined by the SortPath because that EmployeeID will appear that many times in the SortPath column. Take a look at the final table we’re going to end up with to see what I mean.

blog6

Look at the Red squares first. That’s EmployeeID #1 (Jim) and he’s at the very top of the 14 node hierarchy. His node count is “14” and, if you look at the “SortPath”, he appears in the same relative position (happens to match the Level, BTW) of all the SortPaths 14 times including himself.

Look at Bob. Bob is EmployeeID #3 and I’ve marked his information with Blue squares. If you’ll recall, Bob had a total of 5 people in his sub-tree and, if you include Bob, Bob has a total node count of 6. Looking at the SortPath, Bob appears in the SortPath (don’t forget this is a binary sort path) a total of 6 times which is the same as his node count.

Splitting Out the Down line Node Count

How are we going to split out that information? Enter my friend and the proverbial Swiss Army Knife of T-SQL, the Tally Table.

The Tally Table (and certain non-recursive CTEs that mimic it) can be used to very effectively replace certain While Loops for things like splitting sort paths. For more information on how to build a Tally Table and how it works, please see the following article.

http://www.sqlservercentral.com/articles/T-SQL/62867/

Once we have a Tally Table built, we can use it to simultaneously split and count the number of times each EmployeeID appears in the SortPath.

If you look at the SortPath in the previous figure, the SortPath is made up of 4 bytes per Level from left to right. The first level starts at byte #1 and goes for 4 bytes. The second level starts at byte #5 and goes for 4 bytes. Each Level progresses in a similar manner.

We can have the Tally Table “count” through the Levels. The “starting position” of each Level in the SortPath can be calculated as (t.N*4)-3 where “t.N” is the number from the Tally Table and is equal to the value of each Level number. Of course, the length for each Level will always be “4”.

Once we have the split of the EmployeeIDs done, all that’s left is to count the number of times each appears. The following code will do that bit of Tally Table magic for us.

blog5
Adam’s Formulas

All that’s left now is to apply Adam’s formulas and we’ll have our Nested Sets. Using the “node count” code from above, here’s the code to convert the binary coded Hierarchical Path into Nested Sets.

blog4

Like I said, Adam’s formulas are absolutely brilliant in their simplicity.

The Left Bower is a simple calculation which makes the realization that each node (Numbered as “NodeNumber” in order by the SortOrder column) has two Bowers and you need to subtract the number of incomplete nodes in the upline (including the current node) for the node. That’s easily done simply by subtracting the Level of the current node.

The Right Bower calculation is a bit similar. It starts off with the downline node count not including the current node (that’s where the -1 comes in), mulitplies that by 2 because each node has 2 Bowers, adds all of that to the Left Bower and adds 1 because it’s a new Bower not previously accounted for.

All we needed to get the speed out of all of this was to use the Tally Table to get the downline node counts already available in the SortPath in a high performance manner.

Wrapping It All Up

We covered an awful lot of information in a very short period of time. Let’s quickly review what we got for our efforts.

What we end up with is the best of all 3 worlds. Here’s the final table, again.

blog3

We end up with our original Adjacency List (EmployeeID and ManagerID columns) which is easy for us mere humans to understand and maintain.

We end up with our Hierarchical Path information (HLevel, NodeNumber, NodeCount, and SortPath columns) which allowed us to do many things including sort the final output

Last but not least, we also ended up with the LeftBower and RightBower columns of the Nested Sets hierarchical structure. Now we have ease of maintenance and super high performance capabilities for hierarches and we can do it all on a million rows in about a minute and a half and in about 6 seconds on 100,000 rows. That’s fast enough to make it worth simply recalculating everything with some remarkably simple code instead of trying to maintain Nested Sets.

To make life a little easier, here’s all the code we used to do the conversion in one place.

blog2

blog1

At this point, the Left Bower provides the same sortability features as the NodeNumber and SortPath columns. You could delete them but why bother?

A Real Teaser

Here’s a real teaser for you… if you really think about how we built the NodeCount column to build the Bowers, I’m sure that you can see another article’s worth of information on how to make it so we might not even need Nested Sets for speed, anymore. We could preaggregate everything we need to know about any hierarchy and store it all in our own version of what a hierarchical datamart could look like. Like I said… future article. ;-)

Thanks for listening folks.

--Jeff Moden

____________________________________________________________________________________________________________________________

RBAR1 is pronounced “ree-bar” like the steel rods stuck in cement forever and is a “Modenism” for “Row By Agonizing Row”. It has come to mean some “slow code” generally attributed to various forms of procedural programming when compared to other methods.

____________________________________________________________________________________________________________________________

About Jeff Moden

SQL Server MVP Jeff Moden is mostly self taught and is a long time member of and one of the leading posters on SQLServerCentral.com. His articles have made “advanced” concepts such as the use and understanding of the Numbers or Tally Table more like child’s play. He coined the acronym of “RBAR” which is pronounced “ree-bar” and is a “Modenism” for Row-By-Agonizing-Row. It has come be known world round as an easy way to label performance challenged code. He presented at PASS 2010, a couple of SQLSaturdays, and at several local and remote PASS Chapters and User Groups. His passion for the art of SQL and teaching is apparent in his articles and his lectures. Jeff also won the Red Gate sponsored Worldwide Exceptional DBA Award for 2011 and continues to serve the SQL Server Community on a daily basis.

About MVP Mondays

The MVP Monday Series is created by Melissa Travers. In this series we work to provide readers with a guest post from an MVP every Monday. Melissa is a Community Program Manager for Dynamics, Excel, Office 365, Platforms and SharePoint in the United States. She has been working with MVPs since her early days as Microsoft Exchange Support Engineer when MVPs would answer all the questions in the old newsgroups before she could get to them.

melissa

Impressions of TechEd Europe 2012

0
0

Editor’s note:  The following blog post was written by Window’s Expert – Consumer MVP Hal Hostetler

Hello and welcome from Amsterdam and TechEd Europe 2012! Orlando was so much fun that I just had to head over here to do it some more. So far, things are both the same and very different at the same time. The conference, itself, is very similar to Orlando – the booths and vendors are mostly the same; the hall we’re in is similar, though smaller. The attendees I meet and talk to are the same bright, cheerful, knowledgeable folks I encountered in Orlando. The main difference is Amsterdam, itself…what a beautiful city! My wife and I had Monday free, so we went down to the City Center (the Dam) and played tourist. We looked at the sights and tried to keep from being trampled by the throngs of fellow tourists. The weather here is highly unpredictable, going from cloudy, cold, and rainy to clear, warm, and sunny within just a few hours, VERY unlike summertime in Tucson! The lesson here is to pack both summer and winter clothing and be sure to bring an umbrella.

The conference started on Tuesday. I found very little setup when I got here, so I spent the first few hours setting up the photo booth. Setup was a bit difficult as we had no Internet service all morning and into the afternoon. When it finally returned, I was able to complete the setup and get the show on the road. Our location is at the front of the Expo hall, rather than the very back, where we were in Orlando. In fact, we’re right next to the battling robots, who made the journey across the sea in fine fashion. These mechanical maulers are as big a hit here as they were in Orlando and having them right next door means the crowds they draw are in prime position for our photo shoot. The Microsoft Channel 9 mini studio is also next door, so we’ve had a number of dignitaries stop by and chat after they finish their TV interviews. As in Orlando, the Delegate Reception was held Tuesday evening – lots of happy folks eating, enjoying the Dutch beer, and sharing experiences here at the RAI Convention Center.

Wednesday is busy! The photo booth is up and running flawlessly and we had quite a number of folks stop by to have their pictures taken. We’re only running one computer as there isn’t enough room in the booth to have all the props setup at once, plus we have to move them out of the way at lunchtime to make room for the red robot’s Kinect controller. I suppose I should try my hand at driving one of them; see if I can knock the other robot’s block off and release all those pent-up hostilities…trouble is, I’m having so much fun I’d be hard pressed to find hostility to release. I’ve had a chance to chat with a number of folks that attended Orlando; so far, the jury is still out on which event folks like the best. On the one hand, you have the USA with a bigger venue, more people, more vendors, pretty much more everything, while Amsterdam is smaller, more intimate, not quite so much walking, and not nearly so hot. Unlike TENA, the Delegate party is tonight, not Friday after the event concludes. We’re headed to the Amsterdam Arena for food, drink, and big TV screens to watch the UEFA semi-final match between Spain and Portugal. Soccer is THE spectator sport here in Europe; it will be interesting to watch how a big crowd of fans react. I’ll be back shortly with thoughts on the gathering and a wrap-up of the final two days.

Viewing all 788 articles
Browse latest View live




Latest Images