Check out my new blog at https://shibumiware.blogspot.com

Tuesday, July 18, 2017

History, Future Plans, and Common Controls

Progress Bar
See the first post on Common Controls here

A Little History and Future Plans

Every developer has this experience more than once: you build a component, go open source, or even buy to get it almost right.  You tweak it to meet the requirement at hand, and then you move onto the next problem.  All the while, this nagging in the back of your mind is telling you that “you could have done better, made it more extensible, more aesthetically pleasing.”  That’s over engineering if you try to make all of those things happen when other tasks remain on the critical path and you have met the requirement, especially with a paying customer--without their permission, that's downright unethical.  Still, and I think this is so for all of the good developers I have come across, there is a natural desire to perfect your craft, make it beautiful, and make it stand the test of time.

I wrote my first piece of commercial software when I was in high school and many apps before that,  used by friends primarily.  I believe I was about 11 I wrote my first useful piece of software. These apps were MS DOS apps written in Pascal or QuickBasic. The inner core of the application was a big loop that painted the ASCII characters that were used back then to create frames, menus, inputs, reports, and whatever else you did on the screen. The loop received user input, forked off to perform various tasks like reading or writing data to a floppy drive, and then came back and start painting the character-based UI to update it based on what occurred in the subroutine.  

 For those of you who weren't there to do this kind of development, you may wonder what all the weird characters in the DOS character page (see image from Wikipedia below) used for.  Those characters, my friends, were the building blocks of MS DOS applications that went beyond just standard letters.  You were quick to learn these codes and the algorithms for drawing and updating boxes, centering text in the boxes, making the box appear highlighted, etc.  A little like CSS these days--just a little, in the sense there were a ton of tricks you could master to make the app look good.

https://en.wikipedia.org/wiki/Box-drawing_character
box characters

At that point in computer history, there was no such thing as a GUI available to a little kid writing code on his TRS 80 in his basement. But that inner loop that looked a whole lot like a message loop in Windows, so the transition to Windows development was not a big deal. Plus, I switched to a much less verbose and lower level language--C.  Back in those days, there were few “controls” or much of anything to help create a functioning Windows application. You built your menus and dialogs in resource files and manipulated these objects directly. You had a heap to tend to with care, or you would leak like a stuck pig. There were the standard libraries available and the Windows headers to include to give the linker the data it needed and the macros you needed to make Windows programming possible. But it wasn’t that hard. It was labor intensive.

EnteEnter MFC with C++. This combination was freedom from the slavery of the message pump (well, sort of) and handed a shortcut or a wizard to perform all of those menial jobs you had to deal with before. Everything was good.

Enter OLE, COM, and DCOM. Back to everything being hairy and incomprehensible at first, but after a while that too became comfortable. The problem was the platform, runtime, and languages kept changing so the idea of making the last thing you did even better seem a little bit ridiculous because who knew what was coming out of Redmond next.

Well, that was VB 6.0, which we will skip, so I don’t have vicious flashbacks.

EnteEnter .NET.  I was at the PDC event in Florida when they announced it.  I remember thinking “Oh great; we are starting all over again.” And we were, to some degree.  I took the huge stack of DVDs (or were they CDs?) back to my hotel room and wrote my first "Hello World" in C#.  It felt like C.  I had been doing VB.NET.  I was frigging ecstatic!  The base class libraries, even in 1.0, were so wonderful in comparison to Visual Basic.  I almost called home to tell my manager that I was in love.  Thankfully, I didn't.  That would have been embarrassing for the both of us.

.NET has survived and thrived and will remain for many more years, if not decades, so now the idea of building a library of reusable, better engineered, better looking components seem reasonable. C and C++, and even MFC remain and are updated, but as a platform, managed code has punctuated Microsoft’s strategy from its inception. I remember thinking “this is for real” when SQL Server implemented a managed host inside the SQL engine. That’s no small feat and means a level of commitment. Now, with Azure, Office Online, and just about everything else coming out of Microsoft these days, it is either language agnostic (finally, Microsoft has decided that OPT—Other People’s Technology—is useful), .NET (C#, VB.NET, F#, and 20+ others implemented by others) or JavaScript.

I have been writing software for just about 25 years—and I still have the desire to perfect my craft, learn more about the subject, and keep fresh.  I spend several hours a week on Channel 9 (MSDN), YouTube, or my Kindle reading, listening or watching things about my craft.  Yet, it seems, a challenge for anyone who has been doing software for this long that there will be a particular technology that you end up specializing in. For me, there are two—back-end services, which is wonderful because it involves a great deal of diverse technologies, and Windows development.  I still love Windows development.  It's where I started.  I got to meet guys like Don Box and Charles Petzold when I worked at Microsoft. I have exchanged emails with various luminaries throughout the years, including Steve Jobs (yes, that Steve Jobs--he didn't do Windows but he was from that world too), Mark Russinovich, Dave Probert, and others.  A close friend of mine built HTTP.sys and helped invent various pieces of Windows you all use, plus holds a bunch of patents at Microsoft.  I came from Microsoft, practically born of its culture (original culture), and will always love Windows.  A small window in my nerdiness in this subject is that I am book two (not chapter two, BOOK two) of Windows Internals 6th edition, with the 7th Edition queued up on my Kindle.

Windows was cool and damn the haters, it still is.

But things are always changing in my industry so over the past couple of years, I have gathered what I know about Windows development in the managed world and started the ShibumiWare Stack  Aside from adding fresh content when I learn something new on the job, fixing bugs, and maintaining it, I am moving on to other topics as my primary focus—with services and other “plumbing” always being my other favorite. I see in my future much more web development, mobile development, machine learning, cryptography, system’s integration, and others. Microsoft Dynamics is going to be a big part of what I do. I will always love to do Windows development when called upon to do so, but on my 25th anniversary as a developer, I am going off to focus on other Microsoft technologies.

So What's Next with the Common Controls Series?

I plan on completing the series on Common Controls, which I figure will end up around eight to ten posts.  There is a lot of I have done and many tricks I will share along the way. 

Something to Keep in Mind

There is exactly zero code in the stack that I had to purchase.  I spent maybe 6 years doing Windows development using DevExpress and loved it most of the time.  I missed it when I wanted to do hobby projects that looked great but no way was I going to pay thousands of dollars.  I realized through that experience that with a little (or a lot) of work a dedicated developer can make standard Windows development (versus WPF) look very nice indeed.  The time I spent doing "raw" Windows development helps because I know deep down in that really cool looking component, there is just Win32 and Windows development techniques.   So it doesn't overwhelm me easily.

What to Expect in the Next Common Controls Post

I think I have written a login dialog about 20 times in my career so I took all I learned and ever needed and built a credentials management framework with a common dialog that is a shape-shifter of sorts.  Its object model is programmable to allow for themes, different kinds of user names (email address, domain credentials, and others), password complexity options (and UI components that visually tell you if you are meeting those requirements), options for remembering credentials with full encryption, profile management, web content to explain the login process, and other goodies.   It has extension points for providing your own crypto implementation via interfaces.  Various UI components can be made visible or hidden depending on your needs.  It is about as complete as I can make it. 

That post is forthcoming, but I have a lot on my plate right now so I cannot tell you when.

Thanks for staying tuned!


Colby-Tait

Monday, July 10, 2017

Microsoft Dynamics 365 & A Developer's Adventure Spelunking the SDK

Version

Microsoft Dynamics 365

Don't get blinded by the skirmishes between Microsoft and SalesForce.com over the hearts and dollars of salespeople across the world: Microsoft Dynamics 365 offers a tightly integrated, full-featured CRM and ERP platform that goes beyond just customer relationship management.  Visit https://explore.dynamic forces.com/ and take a tour of some of the fantastic capabilities the platform provides. I gave up writing like a marketer in about 1997, so I won't’ try to restart here.

Microsoft Dynamics 365 Solution Areas  
Microsoft Dynamics 365 Solution Areas

Fortunately, there are a great many technical topics that applicable to my daily work. Recently I was given access to an instance of Dynamics 365 with Project Services Automation and Field Management. It bears repeating “Project Services Automation (PSA),” which is an acronymic collision with “Professional Services Automation.” Having two PSA’s to contend with is regrettable because I think Gartner, Forrester, and IDC spent a prodigious amount of time and money explaining the original PSA to the world, and now we have a second. Project Services Automation fulfills a different business requirement than simply CRM and differently than Microsoft Project  Online, which is a traditional enterprise project management system.  Project Server Online has its place, but Project Services Automation is "end-to-end", as you can see from the quote below:

Project Service Automation in Microsoft Dynamics 365 (online) provides an end-to-end solution that helps sales and delivery teams engage customers and deliver billable projects on time and within budget. Microsoft Dynamics 365 for Project Service Automation Automation helps you: • Estimate, quote, and contract work • Plan and assign resources • Enable team collaboration • Capture time, expense, and progress data for real-time insights and accurate invoicing

- Manage Project Based Sales Project Service Automation

General Resources

Here are links for those interested in the solution generally and for those that would like to dive deep into the offering:

Developer Resources

One of the most exciting aspects of being involved with the company I work for is the changing technological landscape and variety of technologies used to accomplish a project’s goals.  Working with the Dynamics Platform differs from what I have been doing for the last decade or so. I have focused on enterprise project management systems, notably Microsoft Project, Project Server, and a touch of Primavera, with a focus on earned value management, API design, synchronization technologies, performance, domain-specific algorithms, cryptography, just to name a few. Dynamics is a big product, its cloud-based, and it has an extensive set of integration points to fulfill a broad range of structural challenges in building and integrating large-scale enterprise resource management systems (ERP). ERP rarely stands alone. Often, line-of-business off-the-shelf products augment and integrate with the ERP system. Even more often, internal solutions are built to satisfy a specific challenge a customer feels the market can not meet it. Putting all of these data flows, compute units, and human interfaces together is an incredible challenge.

Here are a few links to developer resources. If you are a Microsoft Partner, you may have access to a certain number of Dynamics instances, depending on your partnership level.  If you don't have this resource, you can follow the instructions here to start a 30 day trail.

The Beginning Is a Great Place to Start

After a little warm up, a lot of reading, and getting my hands on a Dynamics 365 instance, I was ready to jump right in. I had success with Project Server’s PSI (Project Server Interface) years ago by starting at the bottom and working my way towards more complicated integration scenarios. So successful, in fact, I ended up working for Microsoft, forProject Technology, and now DeltaBahn largely because of my experience making my way through the PSI.

I have an unnatural patience in dealing with poorly, incomplete, or flatly wrong documentation. I also won’t let go of a problem until I have resolved it, learned it, digested it, and usually written my own version of whatever extensibility “product (API)" a product group provides—or at the very least, encapsulated it to increase usability, decrease complexity, and improve performance wherever possible. If API design is exposing a system’s internal machinery and capability to the outside world in a reasonable, consistent, useful, extendable, and understandable fashion, then writing an API on top another API shares the same goals except done by people who have consumed the original API and found it lacking.

In other words, the API’s API is the result of field experience. Furthermore, encapsulation protects the consumer to future changes in the base API and is decoupled from the product group’s release cycle so, as encapsulation API designers, we are free to extend, bend, and often (legally) reverse engineer how the first API works to provide better documentation, samples,  and depth of knowledge.  In my experience dealing with APIs, particularly with Microsoft, there lies within most APIs a grand vision the developers wanted to share with the world but one of the first things to get cut to deliver on time and budget is extensibility.

Connecting to Dynamics 365 as an Office 365 User

I can't do much with my shiny new instance of Office 365 With PSA & Field Management until I authenticate. In early days of working with the PSI I had relied primarily on Active Directory integration, but I did write a Forms Authentication extension that enables the Forms scenario easily. Recently I have done quite a bit with Google's Blogger API, which is employed Open Authentication (OAuth) 2.0, and I had done certificate-based custom transport and user-level authentication on a massive WCF project at forProject Technology, Inc., so I felt comfortable about what I had read about Dynamics' authentication.

I went to the QuickStart and the "Simplified connection quick start using Microsoft Dynamics 365" samples and started in with gusto, only to be disappointed to find that neither sample worked as advertised.  The QuickStart demonstrator has plenty of ells and whistles, whereas the simplified version does not.  Problem number 1 when writing samples.  Have each sample do one thing and one thing only as you start, treat each sample as a teaching opportunity, ensure the developer has a firm grasp before moving on and never include extraneous code to support "other scenarios".    This applies to each method of the example.  In the "Simplified" version of the sample, the first call does what appears reasonable:

ServerConnection.Configuration config = serverConnect.GetServerConfiguration();

GetConfigurations(I show listing and images throughout this portion of the post because I provide the link above to the sample and it is available in the SDK--I do this for conciseness and to be able to illustrate points quicker.)

The point here is that I am first shown how to read previous connection profiles from an XML file and later to write this server configuration back to the file.   Even worse, the call to ReadConfiguration has global side-effects because it modifies a class property Configurations with the configuration information in the XML file and its return value is simply the count of the Configurations property.

Global side-effects are bad, plus any C# developer worth their weight in, say, barbershop floor detritus can figure out how to read and write an XML file. Plus, there are much better ways to do THAT than demonstrated in the sample. I would know from personal experience that many if not all samples were created by somebody other than the developers responsible for the code—one of my first tasks at Microsoft was to write sample code for the Solutions Development Kit back in the mid ‘90’s. Anway, end of rant.

In the addConfig section, the code attempts to choose the discovery URL and successfully chooses a URL but it is the wrong URL and results in an error.  I spend some time in the forums, particularly on StackOverflow and find the sample code is out of sync with the latest packages available from NuGet so I back everything out and use the binaries that ship with SDK.  From the URL built up by the sample, I get a "file not found" error back from the server so I go LOOK UP the correct URL in Dynamics-->Project Services Automation-->Settings--Customizations-->Developer Resources and here is the punch line:  the URL is nothing fancy like the sample code is building up, it is simply  https://disco.crm.dynamics.com/XRMServices/2011/Discovery.svc.

I look at this 1498 lines of code and think "forget it, it cannot possibly be that difficult to connect to Dynamics through my Office 365 account."  There are to many if\then\switch constructs depending if you are on prem, using a Live account (???), using federated AD, or Office 365.

Then What Happened?!

Login ScreenI look at a tool that ships with the SDK called the Plugin Registration Tool.  To the left is the login screen.  That's more like it.  

II mentioned earlier there are may ways to connect to Dynamics. I like OAuth as it is usable in a web app, a Windows app, or on a mobile device. I start focusing in on the parts of the various samples that deal with OAuth. I strip away the saving of configurations, the server connection constructor that is SUPPOSED to take an Organization name to work but does not (another hot topic on the forums), and do a little decompiling using dotPeek to see how it works under all the layers of “helpful” sample code.

I get the feeling the authors are embarrassed to ship something simple—as if doing just what is needed doesn’t show off how smart they are. I think the ability to simplify or distil something potentially complex down to its simplest form shows both intelligence and dedication to their audience.

I worked for about twenty minutes pulling out the pieces needed to connect, authenticate, and retrieve all of the groups I belonged to in our Dynamics implementation, including my newly created developer instance.  Here is a screen shot of the resulting widget I create that shows the groups, and upon selection, the property grid is populated with basic information about the group:

The code to accomplish this is so unbelievably simple I will provde the complete listing here.  First, I created a class called OAuthDynamicsOnlineConnector:


public class OAuthDynamicsOnlineConnector
{
    private readonly string _DiscoveryUrl;
 
    public OAuthDynamicsOnlineConnector(string discoveryUrl)
    {
        if (string.IsNullOrEmpty(discoveryUrl))
        {
            throw new ArgumentException(FoundationResources.ERROR_ARG_NULL_OR_EMPTYnameof(discoveryUrl));
        }
 
        _DiscoveryUrl = discoveryUrl;
 
        Organizations = new List<OrganizationDetail>();
    }
 
    public List<OrganizationDetailOrganizations { getset; }
 
    public void Connect(string userName, string password)
    {
        if (string.IsNullOrEmpty(userName))
        {
            throw new ArgumentException(FoundationResources.ERROR_ARG_NULL_OR_EMPTYnameof(userName));
        }
        if (string.IsNullOrEmpty(password))
        {
            throw new ArgumentException(FoundationResources.ERROR_ARG_NULL_OR_EMPTYnameof(password));
        }
 
        // Create Microsoft.Xrm.Sdk.Client.AuthenticationCredentials from the userName and password
        AuthenticationCredentials authenticationCredentials = CreateAuthenticationCredentials(userName, password);
 
        DiscoveryServiceProxy discoveryProxy = null;
 
        try
        {
            // Use the Microsoft.Xrm.Sdk.Client.ServiceConfigurationFactory to create the 
            // Microsoft.Xrm.Sdk.Client.IServiceManagement<IDiscoveryService> object from our discovery url
            IServiceManagement<IDiscoveryService> discoveryService = 
                    ServiceConfigurationFactory.CreateManagement<IDiscoveryService>(new Uri(_DiscoveryUrl));
 
            // Authenticate, which will populate the SecurityTokenResource object with a valid SecurityTokenResponse 
            // or it will throw a SecurityAccessDenied exception
            authenticationCredentials = discoveryService.Authenticate(authenticationCredentials);
 
            // Create the discovery proxy, which is combining the discoveryService object with the security metadata
            // to give you an authenticated, callable proxy for issuing calls to Dynamics
            discoveryProxy = new DiscoveryServiceProxy(discoveryService, authenticationCredentials.SecurityTokenResponse);
 
            // From here on down it is self-explanatory, resulting in a collection of OrganizationDetail objects
            RetrieveOrganizationsRequest orgRequest = new RetrieveOrganizationsRequest();
            RetrieveOrganizationsResponse response = (RetrieveOrganizationsResponsediscoveryProxy.Execute(orgRequest);
 
            OrganizationDetailCollection organizations = response.Details;
 
            if (organizations == null)
            {
                return;
            }
 
            foreach (OrganizationDetail organization in organizations)
            {
                Organizations.Add(organization);
            }
        }
        catch (SecurityAccessDeniedException accessDeniedException)
        {
            Debug.WriteLine(accessDeniedException.Message);
            throw;
        }
        catch (Exception exception)
        {
            Debug.WriteLine(exception.GetType().FullName);
            Debug.WriteLine(exception.Message);
            throw;
        }
        finally
        {
            discoveryProxy?.Dispose();
        }
    }
 
    private static AuthenticationCredentials CreateAuthenticationCredentials(string userName, string password)
    {
        AuthenticationCredentials authenticationCredentials = new AuthenticationCredentials();
        ClientCredentials clientCredentials = new ClientCredentials();
 
        clientCredentials.UserName.UserName = userName;
        clientCredentials.UserName.Password = password;
 
        authenticationCredentials.ClientCredentials = clientCredentials;
 
        return authenticationCredentials;
    }
} 

The code behind the form is equally simple:

public partial class DynamicsWorkBenchForm : Form
{
    private readonly string _DiscoveryUrl;
    private readonly string _Password;
    private readonly string _UserName;
 
    public DynamicsWorkBenchForm()
    {
        InitializeComponent();
    }
 
    public DynamicsWorkBenchForm(string userName, string password, string discoveryUrl)
    {
        if (string.IsNullOrEmpty(userName))
        {
            throw new ArgumentException(FoundationResources.ERROR_ARG_NULL_OR_EMPTYnameof(userName));
        }
        if (string.IsNullOrEmpty(password))
        {
            throw new ArgumentException(FoundationResources.ERROR_ARG_NULL_OR_EMPTYnameof(password));
        }
 
        _UserName = userName;
        _Password = password;
        _DiscoveryUrl = discoveryUrl;
        InitializeComponent();
    }
 
    private void DynamicsWorkBenchForm_Shown(object sender, EventArgs e)
    {
        OAuthDynamicsOnlineConnector dynamicsOnlineConnector = new OAuthDynamicsOnlineConnector(_DiscoveryUrl);
 
        ((Action)(() => dynamicsOnlineConnector.Connect(_UserName_Password))).TryCatchMethod(this);
 
        SortableBindingList<OrganizationDetail> organizationDetails = 
            ObjectExtensions.ToSortableBindingList(dynamicsOnlineConnector.Organizations);
 
        organizationsListBox.DisplayMember = @"FriendlyName";
        organizationsListBox.DataSource = organizationDetails;
    }
 
    public void OrganizationsListBox_SelectedIndexChanged(object sender, EventArgs e)
    {
        propertyGrid.SelectedObject = organizationsListBox.SelectedItem;
    }
}

Wrap-Up

I will continue writing about Dynamics (setup, cryptography, extension methods, and anything else that strikes my fancy too) over time. I am going to be working quite a bit in this area, and I find the documentation lacking and as a colleague or two of mine said “underwhelming.”  Please do leave a comment if you would like to see a particular topic covered and I will see what I can do to help.

I was going to explain in some detail what a SecurityResponseToken is, how it works, and where the OAuth specifications provides detailed information about it—it is central to how the mechanism works plus… it involves cryptography, specifically 192 bit Triple DES!


Colby-Tait

Tuesday, July 04, 2017

Happy 4th of July 2017

Colby & Parker

Quality Nerd Time Together

My son and I collaborated on this video--the first time for both of us combining music, video slices, and images into a single video.

We used Audacity, GIMP, LAME, and Microsoft Encoder.

Enjoy!

And Here is our Masterpiece...


Everyone have a safe and happy holiday!


Colby-Tait

Saturday, July 01, 2017

Testing 1-2.3

Microphone

Trying Various Microphones

The head set was reported to be too faint. It doesn't have gain control so I pulled out my Telex M-560 from the device archive. It has all sorts of bells and whistles; it was the first microphone sanctioned for voice recognition without a headset or sound card.

Welcome to Star Trek

It certainly doesn't look like any microphone I have seen since I bought it over a decade ago: Telex M-560

I think this will be perfect for video's BUT you can still hear background noise from a server that is sitting about 3 feet to my left.  I need to move it so I can turn the gain up a bit more.  Until then, give the volume a tick up or two when you viewing videos from the channel.

Test Video

Who knows, I may have to borrow Parker's fancy microphone but I think the only significant problem is the server fans.  I never notice the sound any more until it was pointed out.  Thanks for the feedback.  Start using the comments so you can see if I am addressing any problems!  I get too much email as it is, but I am not complaining.


Colby-Tait

Disclaimer

Content on this site is provided "AS IS" with no warranties and confers no rights. Additionally, all content on this site is my own personal opinion and does not represent my employer's view in any way.