Showing posts with label NServiceBus. Show all posts
Showing posts with label NServiceBus. Show all posts

Tuesday, February 12, 2013

NSB Custom Fault Handling in 15 Minutes

We had a need to divert some of our exceptions over to a level 2 help desk.  The help desk would then either fix the problem on behalf of the customer or initiate contact with the customer to fix the problem.  We implemented a custom fault handler and it really only took 15 minutes.

You must be aware that when you take control of the faults, it is up to you to handle all scenarios and in fact SLRs will not work.  In our case, that doesn't really matter since the majority of our exceptions will be handled by a person with this new process.

First we started out by adding some custom config to wire up the handler.  We wanted to maintain the existing process for exceptions that the help desk would not handle, so we borrowed the code from the forwarding handler.

 
public static Configure ForwardToHelpDeskInCaseOfFault(this Configure config)
{
      ....//left out for brevity
 config.Configurer.ConfigureComponent<HelpDeskFaultHandler>(DependencyLifecycle.InstancePerCall)
                .ConfigureProperty(fm => fm.ErrorQueue, ErrorQueue);

      return config;
}

Now we can just simply add that to our endpoint config.
.ForwardToHelpDeskInCaseOfFault();

Lastly, all we need to do is implement the interface and decide base on the exception where it should end up. With this, we are DONE!
 public class HelpDeskFaultHandler : IManageMessageFailures
    {
        private Address localAddress;
        private static readonly ILog Logger = LogManager.GetLogger("WebGateway.HelpDeskFaultHandler");

        public Address ErrorQueue { get; set; }

        public void Init(Address address)
        {
            this.localAddress = address;
        }

        public void ProcessingAlwaysFailsForMessage(TransportMessage message, Exception e)
        {
            this.SendToHelpDesk(message, e, "ProcessingFailed");
        }

        public void SerializationFailedForMessage(TransportMessage message, Exception e)
        {
            this.SendToErrorQueue(message, e, "SerializationFailed");
        }

        private void SendToErrorQueue(TransportMessage message, Exception e, String reason)
        {
            ...//left out for brevity
        }

        private void SendToHelpDesk(TransportMessage message, Exception e, String reason)
        {
            if ( e.GetType().IsAssignableFrom(typeof(MyCustomException)))
            {
                try
                {
                    HelpDeskException hd = new HelpDeskException(message, e.Message);
                    hd.Save();
                }
                catch (Exception ex)
                {
                    Logger.Warn("Failed to send to help desk, sending to error queue", ex);
                    this.SendToErrorQueue(message, e, "SendToHelpDeskFailed");
                }
            }
            else
            {
                this.SendToErrorQueue(message, e, reason);
            }
        }

        private void SetExceptionHeaders(TransportMessage message, Exception e, String reason)
        {
            ...//left out for brevity
        }
    }

Monday, December 17, 2012

NSB PowerShell Support in v3.3

Looks like this is the answer to my request for granular control over infrastructure installation. This works for me as PowerShell is our common denominator for our developers and admins. After getting through the new MSI installer, you will see a new short cut to a prompt.

I'm runing Windows 7 Enterprise and you will get an error OOTB

To get rid of this error you have to install PowerShell v3.  I won't show examples of all the commands available, but we do now have full control over what is installed.  Here is a listing of what is available:

Get-Message
Get-NServiceBusVersion
Install-Dtc
Install-License
Install-Msmq
Install-PerformanceCounters
Install-RavenDB

I think all of these are pretty self explanatory.  The great news is this takes care of allowing you to install only what you need instead of everything in the kitchen sink.

Thursday, April 12, 2012

NSB 3 Upgrade Continued

I finally got back around to my demo solution.  We run the Distributor as a stand-alone and would like to continue to so therefore I thought it would be good to validate that setup on 3.0.  The configuration and setup is much improved since you don't need any special dlls anymore.  The one thing you will run into is that when running in this mode, it is assumed that you will run the Distributor on another machine.  This is fine, but you'll need a license and if you don't have one, you'll need to get a temporary one.

Monday, March 12, 2012

NServiceBus 3.0 Upgrades

For those upgrading to the 3.0 RTM, please check out this video.  I've been upgrading my demo solution to check out the impact and so far all I've really had to do is name my endpoints and remove some config per the video.  Thus far I've gotten my version of Full Duplex working along with basic Pub/Sub.  Next I'll be hitting the Distributor since we use that pretty heavily.  I'll be sure to post back any gotchas that I find.

Thursday, January 12, 2012

JSON Serializer in NSB 3.0

I was answering a question over on SO and I discovered that there is a JSON/BSON serializer built-in to 3.0. It seems like it would be possible with a a bit more digging that you could build upon my hack to expose NSB as a REST endpoint and skip the WCF serialization and jump right down into NSB(assuming WCF is configured to use the JSON format).

Tuesday, December 27, 2011

NServiceBus Modeling Tools Review

After downloading and installing the package, the first thing that we have to do is create a new NSB application. So where is it? I expected it to be in a custom NServiceBus folder under Visual C#, but it wasn't. Then I tried Windows thinking that since we deploy as a Windows Service, it would be there. Nope. Modeling Projects? Nope. So then I did a search and found it at the Visual C# root. I eventually found it, but I'm hoping this moves somewhere more obvious. I'm also curious about the term "application". In the Advanced Distributed System Design course we talked at length on how an application is something that does not require network access, something like Microsoft Word. I'm thinking this should be changed to "System" instead.
It takes you right into the designer canvas which is nice. In the toolbox we have a pretty small set of items, a couple of message types, matching connectors, and two endpoints. I'm going to try to set up a simple send-only client that in turn publishes to a couple of subscribers.
Somehow my references are busted, so I opened the project to see the path and it's referencing some MSBuild variable, so I just updated all the refs. I'm not sure what the bug is here, but I'll blow right by it.
Now that we've generated the system, let's go and see what it did. So far the Client looks just fine:
All the message classes look fine and I'm glad to see that the commands are separated from the events into different assemblies. I'm also glad to see that the Publisher is defined correctly:
I didn't think it would do that based on the canvas. I would like to see a larger group of endpoints that are defined based on there role. This would include Publisher, Subscriber, Distributor, Worker, and Server. This would make it very obvious to the beginner what they are building. The Subscribers are also defined correctly:
It would also be nice if the endpoint configuration matched what was on the canvas, in this case that would be "AsA_Subscriber". Most people that I've helped to learn NSB get a little caught up on what the endpoints can do. If we added "AsA_Subscriber" we could also do some validation on the subscription itself(depending on the profile). All in all, this is a huge step forward. Congratulations to the team for significantly lowering the barrier to entry to getting started. I'd like to see some of the refinements I mentioned above and also would like some Distributor support(I'm sure that is on the list).

Tuesday, December 20, 2011

Monday, October 31, 2011

NServiceBus Modeling Tools Video

I just gave this a look and the tools really lower the barrier to entry to "getting on the bus". I'm going to take some time this week and give it a full evaluation and I'll post that back here. For now, enjoy the video.

NServiceBus Modeling Tools for Visual Studio from NServiceBus Ltd. on Vimeo.

Thursday, July 21, 2011

The Great Worker Pile Up Problem: Solved!


In our configuration of the Distributor pattern, we have the Distributor clustered across multiple VMs. We also run Workers on separate nodes in the same cluster. We are doing this so that we can deploy the same image to all nodes. With this configuration any Distributor/Worker can move from one node to the other in case of a failure. With all this sliding of nodes around we ran into one main issue.

Everything worked great up until we had the big pile up. What would happen is we would intentionally drop all the nodes in succession until we got down to one node. Usually this node would have the Distributor already there and Worker1 would slide over. No problems. When Worker2 slid over to that node all of a sudden one of the Workers would take itself out of business, all the work would move to the other Worker and then we'd have a couple messages disappear. Disappear temporarily that is. Eventually they would come back and be processed.

We initially thought this was due to MSDTC since we saw messages that weren't get ack'd hanging out. After chasing this awhile we then figured it must be some networking issue. Like just about any clustered VM we have 2 NICs configured on the VM, one for the SAN and one for everything else. Come to find out when the last Worker slid over, it was picking up the NIC for the SAN and not the general network. Therefore the messages weren't getting ack'd, but eventually they'd find their way back.

We tried setting the priority of the NICs and so on to no avail. After speaking with MS, we ended up pinning the IPs of the local machines in the registry to the correct network interface. We also had to do this on the clustered services as well. Now when the nodes slide about the local IPs don't change and we have a script that the services depend on that updates the clustered IPs if necessary(like when you change data centers).

I'd like to that our team for being persistent and finally tracking this one down. Now we can deploy the same image to all nodes and have them move about freely. Everything works as designed with full load balancing.

Friday, May 6, 2011

Introducing NServiceBus Visual Studio Templates

I really got tired of creating the same projects over and over so I decided to create some templates. I packaged them up and put them on the Visual Studio Gallery: http://visualstudiogallery.msdn.microsoft.com/9546d382-7ffa-4fb8-8c0f-b7825d5fd085

Right now it only includes a few project templates and a few items. If you are using the project templates there is a custom wizard that will prompt you for a path to the NSB binaries. This will auto adjust the HintPath in the project file to point to that directory.

I'm planning on adding more endpoints and some complete solutions in the near future. Let me know what you are looking for via the gallery so I can keep track of it there.

Wednesday, April 6, 2011

NServiceBus Customization Part 2: IWantToRunAtStartup

When NSB starts up it will look for all implementors of the interface IWantToRunAtStartup and call their Run() methods.  When NSB spins down it will also call the Stop() method of the same interface.  This gives us a good place to run expensive one time initialization code or to tweak out the bus prior to getting going.  A common usage of this interface is for the manual subscription to specific message types:
public IBus Bus { get; set; }

        #region IWantToRunAtStartup Members

        public void Run()
        {
            this.Bus.Subscribe<IProductUpdatedEvent>();
            this.Bus.Subscribe<IProductCreatedEvent>();
        }

        public void Stop()
        {
            this.Bus.Unsubscribe<IProductUpdatedEvent>();
            this.Bus.Unsubscribe<IProductCreatedEvent>();
        }

        #endregion
One thing that I would caution against doing in the Run() method is not exiting the method. I've seen cases where someone will go into an infinite loop and never exit Run(). Typically this is done to perform a task on a given interval. There are much better ways to do this, mostly commonly you can put a Timer into the container and wire into its events. If you never leave the Run() method, the Bus doesn't ever really startup and you will get some odd results.

Sunday, March 27, 2011

NServiceBus Customization Part 1: IWantCustomInitialization

In this first part we're going to explore how to take control of NServiceBus during the initialization phase.  By implementing the IWantCustomInitialization interface, you take full control away from NSB Roles.  What is a Role?  A Role tells an Endpoint how to participate on the bus.  Typical Roles are the Server, Client, and Publisher Roles.  These are defined by a class in your project, typically named EndpointConfig.  A Role will define certain aspects of behavior, like whether or not it can receive messages(AsA_Client).

Most of the time you don't end up ever overriding the default behavior of the Roles.  When you do, you'll implement the IWantCustomInitialization interface and have full control over NServiceBus.  You will use the static NServiceBus.Configure class to get things going.  Let's take a look at all the options(as of 2.5, 3.0 follows):

  • .With()/.WithWeb() - these first methods give you the ability to control assembly scanning.  When NSB fires up, it will scan the bin directory by default.  This gives you control over what gets scanned.  You can provide a list of Assemblies, a list of Types, or a path.  You'll want to use the WithWeb() version if you are using NSB with ASP.NET.  
    • NServiceBus.AllAssemblies -  This class allows you to provide a list of assemblies not to load.  An example of this would be "AllAssemblies.Except( "Assembly1" ).And("Assembly2")".  
  • .Synchronization() - tells NSB that message processing with occur in a synchronization domain.  This is useful for rich client applications where you need to marshal from a background thread back to the UI.
  • .DefaultBuilder() or .MyContainerOfChoiceBuilder() - this next set of methods controls the container.  You can take the default container(AutoFac as of 2.5) or specify your own.  You'll need to include the builder implementation for your container as well as specify an instance of your container.  
  • .UnicastBus() - use unicast messaging
    • DoNotAutoSubscribe() - do not auto subscribe to a Publisher's messages
    • ForwardReceivedMessagesTo( String ) - forward all messages to another endpoint.  This is helpful if you want to do some auditing.
    • LoadMessageHandlers() - tells the bus to scan and load message handlers.  There are a couple interesting variants on this method:
      • LoadMessageHandlers<TFirst> - tells the bus to load the handlers but load the assembly where TFirst resides before the rest
      • LoadMessageHandlers<T>( First<T> ) - load the handlers and specifies that the handlers in the given order should go first.  To use the First class to specify ordering, it has 2 methods, "Then" and "AndThen".  A typical usages would be "First<IHandler1>.Then<IHandler2>().AndThen<IHandler3>()"
      • PropagateReturnAddressOnSend( Boolean ) - receivers of the messages sent by this endpoint will see the address of the incoming messages
  • .XmlSerializer() or .BinarySerializer() - specify the serialization technique you'd like to use.
  • .CustomConfigurationSource( IConfigurationSource source ) - if you would rather not use the standard app.config file or its sections for Administrative configuration, you are welcome to use your own. Simply implement the IConfigurationSource interface and you are on your way.
  • .MsmqTransport() - tells NSB to use MSMQ as the underlying communication mechanism.  This may be swapped out in favor of other transports.  Along with this setting, you can configure a few things underneath it:
    • .DoNotCreateQueues() - don't try to create queues if they don't exist
    • .IsolationLevel( IsolationLevel ) - the isolation level of the transaction that MSMQ uses.  You may consider tweaking this for performance reasons or otherwise.
    • .IsTransactional( Boolean ) - for web scenarios, you may not want to use a transaction when pushing a message to the queue
    • .PurgeOnStartup( Boolean ) - determine whether or not to purge queues when NSB starts up.
    • .TransactionTimeout( Timespan ) - the time to wait for a transaction
  • .MsmqSubscriptionStorage() or DBSubscriptionStorage() - tells NSB where to persist subscriptions.  If you are using Profiles, this is set in the Profile of choice(Lite, Integration, or Production).  If using MSMQ, the queue to use is in the config file.  If  using NHibernate, you can default to the config file for NH properties or supply an IDictionary to the overload.
    • .DBSubscriptionStorageWithSQLLiteAndAutomaticSchemaGeneration() - pretty sure we all have this one down.
  • .Sagas() - tells NSB to look for Sagas in the assemblies provided to NSB in the beginning of the process.
  • .NHibernateSagaPersister() - this will use NH for Saga persistence.  Unless supplied to the overload, NSB will use the config file for the NH properties.  You can give this method an IDictionary of properties as this point.  
    • NHibernateSagaPersisterWithSQLLiteAndAutomaticSchemaGeneration() - pretty self explanatory I think.
  • .RijdaelEncryptionService() - if you are encrypting messages, this will load the keys from the config.
  • .Log4Net() - there are a few overloads here for you to tweak Log4Net.  You have access to the Appender, AppenderSkeleton, and you and specify your own configuration.
  • .RunCustomAction( Action ) - this is a place where you can run anything you want really.  Typically I've seen this used to tweak the container very early in the boot strapping process.
New 3.0 Options
     Some of the options have moved around to new classes, but the semantics remain the same.  Below are some of the new options available to address new features.
  • .MessageForwardingInCaseOfFault() - tells the bus to forward messages when there is a fault
  • .FtpTransport() - FTP transport options
  • .ImpersonateSender(Boolean) - whether we should impersonate the sender or not
  • .InMemoryFaultManagement() - sets up the in memory fault manager.  Faults are lost when the process shuts down.
  • .NHibernateFaultManager() - sets up the NH fault manager.  Faults are saved to a database with the exception that caused them.
  • .NHibernateUnitOfWork() - sets the UOW manager to use the internal NSB NH manager.  You can supply your own if you are also using NH in your code.
As we've found out, you have full control over how NSB is configured right from the start.  Next time we'll a close look at the Roles we talked about earlier in the article.

Monday, March 21, 2011

ReturnToSourceQueue PowerShell Script: Updated

UPDATE: I've added transaction support to the script.  Previous to this you would have lost a message if something had gone wrong.  PS does not support TransactionScope, so we had to use MessageQueueTransaction.

The ReturnToSourceQueue.exe tool that comes with NSB presented a challenge to us when we tried to deploy it on all our app servers.  We needed to schedule an on-demand job to run the tool by our Operations group.  Since the agent we use goes on each machine and is not cluster aware, this presented an issue.  Our resolution was to replicate the same functionality in PS with a bit of a twist.  We added the ability to hit a remote queue.  The script was done by one of our developers Brandon Moriarty and I've posted it to GitHub for everyone to enjoy!  Big thanks to Brandon for whipping this one up.  Don't forget I have a few other PS scripts in there to do some quick and dirty things with MSMQ.

Friday, March 18, 2011

NServiceBus Customization, Let me Count the Ways!

So as promised I'm going to start a new series on customizing the behaviour of NSB. I figured that we'd first try to count up all the ways to do it and then tackle each one by one. I'm going to categorize them by when they occur during the NSB lifetime, Startup, Processing, and Shutdown.  I'm not going to include everything(it would take forever), but I'm going to include the most popular places where you'd customize.

Startup

  • IWantCustomInitialization - take over control of how the bus is initialized, skipping Roles
  • Roles - AsA_* interfaces. Control how the bus is initialized.
  • IWantToRunAtStartup - run a custom action once at startup
  • IConfigureLoggingForProfile<T> - configure logging for a specific Profile
  • Profiles - IProfile. Determine settings for a given target environment. OOTB includes "Lite", "Integration", and "Production"
  • ISpecifyMessageOrderHandling - creates a "pipeline" of messages and gives them a specific order
  • The Container - you can pretty much change anything in the container, including passing in your own that is loaded with your stuff.

Processing

  • IMessageMutator - manipulate inbound and outbound messages
  • IMessageModule - inject some logic before and after message processing
  • IManageUnitsOfWork - UOW pattern handling
  • IManageMessageFailures - custom failed message handling
  • Processing Events - StartedMessageProcessing, TransportMessageReceived, FinishedMessageProcessing, FailedMessageProcessing.  You can use these to do something when message events are happening(say perf counters)


Shutdown

  • IWantToRunAtStartup - there is a Stop method here that gets called once during shutdown

ReturnToSourceQueue PowerShell Script

The ReturnToSourceQueue.exe tool that comes with NSB presented a challenge to us when we tried to deploy it on all our app servers.  We needed to schedule an on-demand job to run the tool by our Operations group.  Since the agent we use goes on each machine and is not cluster aware, this presented an issue.  Our resolution was to replicate the same functionality in PS with a bit of a twist.  We added the ability to hit a remote queue.  The script was done by one of our developers Brandon Moriarty and I've posted it to GitHub for everyone to enjoy!  Big thanks to Brandon for whipping this one up.  Don't forget I have a few other PS scripts in there to do some quick and dirty things with MSMQ.

Monday, March 7, 2011

Oracle AQS Transport Now on NServiceBus-Contrib

Source has been committed to the v2.5 branch on GitHub. I'll start work on the master(a.k.a. 3.0 version) shortly.

Monday, February 28, 2011

Oracle AQS Transport Complete & the Next Article Series

This went together pretty quickly. I have a local version tested and working from both directions. I can push messages in from the database side via a stored procedure and also from the .NET world. All that is left is to fork NServiceBus-Contrib and get the code into that solution structure and figure out how to build it. It's using Ruby/rake which I've never used before, but the format seems simple enough.

After that work is done, it's back to getting the 3.0 branch to function. I'll need to work on the serialization issues with the headers, hopefully that is not a big deal. I'll be sure to include all the SQL scripts I used to setup and test the transport.

During our implementation of NSB, we came across many times where we had to customize NSB to do various things. I'm thinking the next series will be on all the ways you can customize and extend NSB.

Tuesday, February 22, 2011

Working on Oracle AQS Transport

Internally we have a need to hook into certain changes within some data that is housed in Oracle. Our solution will be to use Oracle CDC(Change Data Capture) to grab the info we need right from the redo logs and push that data into a table. We're going use the table to provide those who would like to do ETL a simple way to grab data that is not on the live tables and only contains the changes. From this table we'll push the rows into Oracle AQ(Advanced Queuing). From here we have the opportunity to drive that data right onto NSB much like the Service Broker is being used in this article.

Thanks to the contributors over at NServiceBus/NServiceBus-Contrib I have a great head start. I started out going down the NSB 3.0 route since the barrier to entry has been lowered greatly when attempting to create a custom Transport. Inadvertently I realized that I'm going to need to support NSB 2.5 seeing as how most of my shop is still on 2.0 and moving to 2.5, not only that 3.0 is still considered alpha or pre-alpha even. I also ran into an interesting serialization error in 3.0 anyway, once which I'll have to revisit later. The message headers on TransportMessage in 3.0 is now a Dictionary(used to be a List) which the serializer does not like one bit.

Luckily I was able to re-use the code from the 3.0 implementation. Going back to the 2.5 methodology has been a bit painful, and I'm glad that the custom Transport story is changing drastically. I have everything all set up and I'm ready to test. I'm even running Oracle 10g Express locally so this thing should be easy to try out for everyone. I'm interested to see if I can clean up some of the serialization work from the Service Broker implementation. Oracle AQ stores everything in a special table that backs the queue. You can tell it to make the column that stores the data to be an XMLType column or RAW, which I'm hoping plays more nicely with the serializers.

Once I get something more solid my plan is to put the code out on my personal repository in github and then work on getting it integrated into NServiceBus-Contrib. Let me know if anyone has already built this or is trying to, I'm always on the lookout for any problems or suggestions that have come up.

Monday, February 7, 2011

NServiceBus 3.0: Message Mutators

In NSB 2 it was a bit more tricky to change messages as they were sent to and from endpoints.  In NSB 3 we now have a couple of simple interfaces to plug in our custom logic to change messages.  The typical scenario this is used for is to encrypt all or part of a message.  The encryption message mutator is built right in and can be used at any time.  In this article we'll show how to create a custom message mutator to simply do some math on Double fields in a message.  Let's first take a quick look at the interfaces involved.
Each interface gives us access to the message so that we can mutate on the inbound and/or outbound message.  All we have to do as a consumer is implement the desired interface and load it into the NSB container.  For my mutator, I will implement IMessageMutator and only implement the outgoing message for the time being.
public class MultiplierMutator : IMessageMutator
    {
        #region IMutateOutgoingMessages Members

        public NServiceBus.IMessage MutateOutgoing(NServiceBus.IMessage message)
        {
            var doubles = message.GetType().GetProperties().Where(p => typeof(double).IsAssignableFrom(p.PropertyType));

            foreach (var dbl in doubles)
            {
                double dblValue = (double)dbl.GetValue(message, null);
                dbl.SetValue(message, dblValue * 5, null);
            }

            return message;
        }

        #endregion

        #region IMutateIncomingMessages Members

        public NServiceBus.IMessage MutateIncoming(NServiceBus.IMessage message)
        {
            return message;
        }

        #endregion
    }
In this class we first find all Double fields. Once we have that list we simply multiply each field by an arbitrary number, in this case 5. The last thing we need to do is let the container know about the mutator. We do this in our implementation of IWantCustomInitialization.
NServiceBus.Configure.Instance.Configurer.ConfigureComponent<MultiplierMutator>(NServiceBus.ObjectBuilder.DependencyLifecycle.InstancePerCall);
Now we can fire up our endpoints and watch what happens. I stubbed this into the Encryption sample that comes with the NSB download. In the Client project I added a value to the Result field on the message.
while (Console.ReadLine() != null)
            {
                Bus.Send<MessageWithSecretData>(m =>
                    {
                        m.Secret = "betcha can't guess my secret";
                        m.Result = 5;
                    });
            }
Now for the output from the Server project:
So that's it! Now we can dream up any kind of change to our messages. Maybe you want to compress or hash some of the messages, it's all up to you.





Monday, January 24, 2011

Clustered MSDTC Mutual Authentication Gotcha Between 2003/2008 Servers

This is more of a mental note that anything else, but we ran across an issue with configuring our subscriptions database on a cluster.  What we had set up was a 2008 cluster with our Publisher on it and a 2003 cluster with our subscriptions DB on it.  MSTC was giving us a very distinct error:

[SQL: SELECT this_.SubscriberEndpoint as y0_ FROM [Subscription] this_ WHERE this_.MessageType in (?, ?, ?)]; Communication with the underlying transaction manager has failed.; The MSDTC transaction manager was unable to push the transaction to the destination transaction manager due to communication problems. Possible causes are: a firewall is present and it doesn't have an exception for the MSDTC process, the two machines cannot find each other by their NetBIOS names, or the support for network transactions is not enabled for one of the two transaction managers. (Exception from HRESULT: 0x8004D02A)

The solution for this was that 2003 servers in cluster doesn't support Mutual Authentication.  We ended up just using the Incoming Caller authentication between the clusters and everything started working again.  Hope this helps someone else with the same problem.