Display HwndHost Inside ToolTip/ContextMenu

This post is a further elaboration to my reply to this MSDN WPF forum thread.

Before digging into the details of how to solve this issue, let's start with some background information on ToolTip/ContextMenu and HwndHost.

In WPF, ToolTip/ContextMenu is hosted inside a Popup, Popup is essentially a hwnd with WS_POPUP window style. By default, the Popup created by ToolTip and ContextMenu services will have AllowsTransparency property set to true to enable layered windows on it. WPF uses application managed layered windows which is incompatible with the win32/GDI rendering model, which means that application managed layered window doesn't support child hwnd with WS_CHILD window style. So you want to display hwnd based content (through HwndHost) inside the a ToolTip/ContextMenu, you need to disable layered windows.

The caveat here is that the implementation of ToolTip and ContextMenu has hardcoded the AllowsTransparency property value when creating Popup, and it seems that this will not change at least in the next version of WPF. I think it might be common scenario to display content coming from other pre-WPF presentation technologies such as win32 controls, MFC/ActiveX controls or Windows Forms controls etc simply because you already have a control at hand,and it has already implemented all the functionalities you want. There are some known issues with hwnd interop in particular the airspace issue but as long as you don't overlap the hwnd content with WPF content, things should work properly at least visually.

The solution I posted in that WPF forum thread requires subclassing from ToolTip, and override the OnTemplateChanged protected method to wire up the code to set the Popup.AllowsTransparency at the right time before the internal implementation of ToolTip creates it for you which will be too late to set that property(setting Popup.AllowsTransparency to false after the ToolTip/ContextMenu is displayed doesn't take effect immediately, you need to hide the popup and then display it again to get this work). The ideal solution is that I could wire up those logic through the well-known WPF pattern - attached behavior pattern. So I could simply write something like the following, and things start to work for both ToolTip and ContextMenu without separate subclassing:

<StackPanel>
  <Button Content="ContextMenu Demo">
    <Button.ContextMenu>
      <ContextMenu cc:PopupBehavior.AllowsTransparency="False">
        <WebBrowser Width="300" Height="200" Source="http://www.live.com" />
      </ContextMenu>
    </Button.ContextMenu>
  </Button>
  <Button Content="ToolTip Demo">
    <Button.ToolTip>
      <ToolTip cc:PopupBehavior.AllowsTransparency="False">
        <WebBrowser Width="300" Height="200" Source="http://www.live.com" />
      </ToolTip>
    </Button.ToolTip>
  </Button>
</StackPanel>

Here is the complete implementation of PopupBehavior:

using System;
using System.Security;
using System.ComponentModel;

using System.Windows;
using System.Windows.Media;
using System.Windows.Threading;
using System.Windows.Controls;
using System.Windows.Controls.Primitives;

namespace Sheva.Windows.Controls
{
    /// <summary>
    ///Extends Popup control with added behaviors.
   /// </summary>
   public class PopupBehavior
   {
        public static readonly DependencyProperty AllowsTransparencyProperty =
            DependencyProperty.RegisterAttached(
            "AllowsTransparency",
            typeof(Boolean),
            typeof(PopupBehavior),
            new FrameworkPropertyMetadata(
                true,
                new PropertyChangedCallback(OnAllowsTransparencyChanged),
                new CoerceValueCallback(CoerceAllowsTransparency)));


        public static Boolean GetAllowsTransparency(Control element)
        {
            CheckElementType(element);
            return (Boolean)element.GetValue(AllowsTransparencyProperty);
        }

        public static void SetAllowsTransparency(Control element, Boolean value)
        {
            CheckElementType(element);
            element.SetValue(AllowsTransparencyProperty, value);
        }

        private static Object CoerceAllowsTransparency(DependencyObject element, object baseValue)
        {
            //WPF will force the Popup into WS_CHILD window when running under partial trust, layered windows
            //is only supported for top level windows, so it doesn't make any sense to set the AllowsTransparency to true
            //when running under partial trust.
           return IsRunningUnderFullTrust() ? baseValue : false;
        }

        private static Boolean IsRunningUnderFullTrust()
        {
            Boolean isRunningUnderFullTrust = true;
            try
           {
                NamedPermissionSet permissionSet = newNamedPermissionSet("FullTrust");
                permissionSet.Demand();
            }
            catch(SecurityException)
            {
                isRunningUnderFullTrust = false;
            }

            return isRunningUnderFullTrust;
        }

        private static void OnAllowsTransparencyChanged(DependencyObject sender, DependencyPropertyChangedEventArgs e)
        {
            Control element = (Control)sender;
            CheckElementType(element);

            if(element.IsLoaded)
            {
                //Find the Popup logical element root.
               Popup popup = GetPopupFromVisualChild(element);
                if(popup != null) popup.SetValue(Popup.AllowsTransparencyProperty, e.NewValue);
            }
            else
           {
                var templateDescriptor = DependencyPropertyDescriptor.FromProperty(Control.TemplateProperty, element.GetType());

                EventHandler handler = null;
                handler = (obj, args) =>
               {
                   //Not clear why the BeginInvoke call is needed here, but this could effectively
                   //workaround cyclic reference exception when evaluating ToolTip/ContextMenu's Style property.
                  element.Dispatcher.BeginInvoke(DispatcherPriority.Send, newAction(delegate
                  {
                       SetAllowsTransparencyInternal(element, e.NewValue);
                   }));

                   //Clear event handler to avoid resource leak.
                  templateDescriptor.RemoveValueChanged(element, handler);
               };

                templateDescriptor.AddValueChanged(element, handler);
            }
        }

        private static void CheckElementType(Control element)
        {
            if(!(element is ToolTip || element is ContextMenu))
            {
                throw new NotSupportedException("AllowsTransparency attached property can only be applied to ToolTip or ContextMenu");
            }
        }

        private static void SetAllowsTransparencyInternal(Control element, Object value)
        {
            ToolTip tooTip = element as ToolTip;
            ContextMenu contextMenu = element as ContextMenu;

            // Set the IsOpen property to true to let the ToolTip/ContextMenu create Popup instance early, since
            // we are only interesting in Popup.AllowsTransparency property rather than
            // opening the ToolTip/ContextMenu, set its Visibility to Collapsed.
           element.Visibility = Visibility.Collapsed;
            if(tooTip != null)
            {
                tooTip.IsOpen = true;
            }
            else if(contextMenu != null)
            {
                contextMenu.IsOpen = true;
            }

            //Find the Popup logical element root.
           Popup popup = GetPopupFromVisualChild(element);
            if(popup != null) popup.SetValue(Popup.AllowsTransparencyProperty, value);

            //Set properties back to what it is initially.
           if(tooTip != null)
            {
                tooTip.ClearValue(ToolTip.IsOpenProperty);
            }
            else if(contextMenu != null)
            {
                contextMenu.ClearValue(ToolTip.IsOpenProperty);
            }
            element.ClearValue(FrameworkElement.VisibilityProperty);
        }

        private static Popup GetPopupFromVisualChild(Visual child)
        {
            Visual parent = child;
            FrameworkElement visualRoot = null;

            //Traverse the visual tree up to find the PopupRoot instance.
           while(parent != null)
            {
                visualRoot = parent as FrameworkElement;
                parent = VisualTreeHelper.GetParent(parent) as Visual;
            }

            Popup popup = null;

            // Examine the PopupRoot's logical parent to get the Popup instance.
            // This might break in the future since it relies on the internal implementation of Popup's element tree.
           if(visualRoot != null)
            {
                popup = visualRoot.Parent as Popup;
            }

            return popup;
        }
    }
}

I have fully commented the code so it might be a little bit more straightforward to read. All in All, the implementation is a little bit like a hack, and might be broken in the future version of WPF as I commented above:)
Here is the complete VS project: HwndHostInPopupDemo.zip