Simple Interprocess Communication in .Net Core using Protobuf

In the past, I have used WCF to handle inter-process communication (IPC) between various separate components of my client applications. Since .Net Core doesn’t yet support WCF server side code, I had to look into alternatives. The two main approaches to this that I have found are TCPServer and NamedPipeServerStream. Others have covered the TCP approach, so I wanted to see what could be done with the NamedPipeServerStream.

I started reading the MSDN documentation on the basics of IPC with named pipes and found that it worked with .Net Core 2.0 with no changes. This is the true benefit of .Net Core. An older article about IPC is still completely relevant even though the code is now running on a Mac instead of a Windows Machine. One thing I didn’t like too much about that article was the StreamString class and I wanted to see what I could do with plain old C# objects.

I decided to start try out Protobuf. I had heard about it in the past and figured this would be a good foray into learning more about it. Since I was developing a client and a server, I decided I would start with the API and put that into a shared class project. So I created a Common project, added a reference to protobuf, and defined a Person class in there:

[ProtoContract]
public class Person
{
    [ProtoMember(1)]
    public string FirstName { get; set; }

    [ProtoMember(2)]
    public string LastName { get; set; }
}

Decorating the class with the protobuf attributes was all I had to do. Now that it is defined in the common class, I could write a server to serve up the data and a client to consume the data, each referencing the Common library. Next up, I created the server. Following the linked example above, I defined the server console application as:

static void Main(string[] args)
{
    Console.WriteLine("Starting Server");

    var pipe = new NamedPipeServerStream("MyTest.Pipe", PipeDirection.InOut);
    Console.WriteLine("Waiting for connection....");
    pipe.WaitForConnection();

    Console.WriteLine("Connected");

    Serializer.Serialize(pipe, new Person() { FirstName="Janey", LastName = "McJaneFace" });
    pipe.Disconnect();
}

I am simply defining the NamedPipeServerStream to listen on a pipe named “MyTest.Pipe”. For now, the code immediately returns an object to the connection, that can be read from the client side. This is achieved using protobuf’s Serializer.Serialize method. To define the client, I need to use a NamedPipeClientStream to connect to the same pipe.

static void Main(string[] args)
{
    Console.WriteLine("Client");
    var pipe = new NamedPipeClientStream(".", "MyTest.Pipe", PipeDirection.InOut, PipeOptions.None);
    Console.WriteLine("Connecting");
    pipe.Connect();
    var person = Serializer.Deserialize<Person>(pipe);
    Console.WriteLine($"Person: {person.FirstName} {person.LastName}");
    Console.WriteLine("Done");
}

Once I connect, I then use protobuf’s Serializer.Deserialize method to read from the stream and deserialize the person object. That’s it. I am passing data from one process to another in .Net core. If you are using .Net Core 1.x, you will need to explicitly add a reference to the System.IO.Pipes nuget package. And for both 1.x and 2.0 .net core, you need to add a nuget reference to protobuf.

Even though this is a basic example, it does demonstrate the functionality and could be easily extended to handle much more complex scenarios.

A fully working solution for this can be found as a sample GitHub project. There appear to be other .Net Core/Standard projects(1, 2) attempting to better facilitate IPC and it will be interesting to see how they mature with the ecosystem. My hope is that some flavor of WCF server makes its way over to .Net core, to make porting code that much easier.

Minified Javascript not Deploying With .Net Core Projects Created in Visual Studio 2017

I was working on a very simple site that I created using the new .Net Core project templates in Visual Studio 2017. Everything worked great on my machine, but, when I deployed to Azure, none of my custom javascript or CSS were working properly. What gives?

After doing some digging, I found that the deployed site was trying to use the site.min.js and the site.min.css, but those files weren’t deployed to Azure. After googling a bit, I found that it was probably an issue with my bundling and when I opened the bundleconfig.json, Visual Studio tried to be helpful:

Extensions are available...

Of course, I ignored the extension warning and comment at first, but the extension that is missing solves the exact problem I was having. The link in the comments has an article on how to enable and configure bundling in ASP.NET Core.

So, while the Visual Studio team could work on making this a better experience, I have to remember to read the warnings and comments that are left in the generated code. They are there for a reason.

Zero to CI in Thirty Minutes Or Less (Or its free!)

One of the biggest complaints I hear from teams about CI is that it is too much work. While getting it perfect can be a lot of work, getting started couldn’t be easier.

I am going to demonstrate continuously building a C# project using Jenkins as our CI host.

To get started we’ll need a machine to be our build agent. I am going to create a VM in Azure to be my build agent. Since I am building a C# project, I am going to choose the template that already has Visual Studio 2017 installed on it. But this could be any machine. It could be an extra machine you have sitting under your desk or a virtual machine in your own datacenter.

Azure template image

Once the machine is created, you can connect to it and install Jenkins. Start by downloading and running the windows installer.

Once installed, a browser window will open that you can use to administer Jenkins. It may open before Jenkins has a chance to start, so you may need to refresh the page. Follow the instructions on the page to unlock Jenkins for first time use.

You will be prompted to install plugins. Plugins are the lifeblood of the Jenkins ecosystem and there are plugins to do pretty much everything. You can start by installing the suggested plugins.

Customize Jenkins

This will install a handful of plugins to get us started. Once complete, you will be asked to setup an admin user. Go through the steps of setting up the user and then you can start using Jenkins. At this point, Jenkins is ready to go.

Jenkins is Ready

We are going to create a job to build a simple C# library I am hosting on GitHub to demonstrate Jenkins builds. Now we can create a new job and give it a name. The easiest project to configure in Jenkins is a freestyle project. This allows you to do any type of build you want by combining source control and scripts to accomplish your task (along with features from whatever plugins you have installed).

Jenkins freestyle project

Next, we will configure the project to pull from GitHub and give it a little batch script to build the project and run the tests. In the Source Code Management section, we will select Git and enter our repository URL.

Jenkins Source Control Configuration

Then in the Build section, we will setup our script to build the project. Since this is a .Net core project and I have the 2017 core tools installed, I can simply specify a batch command with the following script:

dotnet restore
dotnet build
cd UnitTests
dotnet test

Save the job. Then click the Build Now button on the left hand side. This will start a job which will appear in the Build History portion of the page. You can click on the build number to get more information about the build. The most useful information in here is the console. If you click on the Console Output you can see the full console output of your build. Since this is your first build on the machine, you will see information about populating your local cache for the first time and then you should see the project build output, and finally you should see the tests run and see them all pass.

At this point, we have a build server that builds our project on demand, but not continuously. To set that up, we can go back to the project page and select the Configure option. We’ll use the Poll SCM option to configure the job to poll for changes from GitHub every 15 minutes. In the Schedule box, enter the following value

H/15 * * * *

The format for this schedule mainly follows the cron syntax. Clicking on the ? next to the schedule box will give you plenty of examples and information on how you might want to configure your job. Save the job and you are good to go.

You now have a build server that will build within 15 minutes of a change to the repository. Congratulations, you have a CI server. As you can see, getting started with CI is not hard and there is really no excuse for not having some sort of automations around your builds.



FAQ

Q:But John, why did you ignore webhooks?

A: While setting up webhooks is very straightforward, securing a Jenkins installation to be accessible via the internet is another thing altogether. I decided that using polling was a better approach than teaching people to setup an insecure Jenkins installation and having them get hacked. I’ll probably have a few more posts where I cover setting up webhooks for your Jenkins jobs.

Using a CSharp Syntax Rewriter

One interesting feature of the roslyn engine is the CSharpSyntaxRewriter. This is a base class which you can extend that implements the visitor pattern. You simply override the Visit method that is appropriate for your use case and you can rewrite any portion of a syntax tree.

Consider the following code:

public class Foo
{
    public string _bar = "baz";
}

Now, let’s say we want to change "baz" to something else. To do this, we simply implement a new CSharpSyntaxRewriter that overrides the VisitLiteralExpression so that our code is only executed on LiteralExpression syntax nodes. We then check to see if it is a StringLiteralExpression and if it is, then we create a new node.

class LiteralRewriter : CSharpSyntaxRewriter
{
    public override SyntaxNode VisitLiteralExpression(LiteralExpressionSyntax node)
    {
        if (!node.IsKind(SyntaxKind.StringLiteralExpression))
        {
            return base.VisitLiteralExpression(node);
        }

        var retVal = SyntaxFactory.LiteralExpression(SyntaxKind.StringLiteralExpression, 
                                                      SyntaxFactory.Literal("NotBaz"));
        return retVal;
    }
}

The rewriter can be passed any SyntaxNode and it will run on that node and its descendants. So, to use this, I can get a SyntaxTree from a SemanticModel that I get from a CSharpCompilation. Here is a full working sample:

var tree = CSharpSyntaxTree.ParseText(@"
  public class Foo
  {
      public string _bar = ""baz"";
  }");
var Mscorlib = MetadataReference.CreateFromFile(typeof(object).Assembly.Location);
var compilation = CSharpCompilation.Create("MyCompilation",
    syntaxTrees: new[] { tree }, references: new[] { Mscorlib });
var model = compilation.GetSemanticModel(tree);
var root = model.SyntaxTree.GetRoot();

var rw = new LiteralRewriter();
var newRoot = rw.Visit(root);

Console.WriteLine(newRoot.GetText());
Console.ReadLine();

As you can see, a rewriter is a quick and easy way to manipulate a syntax node that you have access to. It’s a great tool to use when you have some simple changes you need to make to a syntax tree.

If you have come up with a good use for syntax rewriters, leave a comment below.

Windows Installer Error Codes

My team was recently working on our installer when we ran into a return code from one of our pre-requisites (specifically the Visual C++ 2015 Runtime). From the logs, we found the error code was 0x80070666 and the return code was 0x666 (a newer version of the runtime was installed). We could easily handle this scenario, but we were looking to find an extensive list of return codes for the C++ redistributables and could not easily find them. Eventually we ran across the list of MsiExec.exe and InstMsi.exe Error Messages and figured out why our searches were yielding no results. The error codes in the log files are all in hex and the error codes on the website are all in decimal. So, to help others with this in the future, here is a list of all of the error codes from that link with their hex equivalent.

Error code Value Description Hex Error Code
ERROR_SUCCESS 0 The action completed successfully. 0x0 0x80070000
ERROR_INVALID_DATA 13 The data is invalid. 0xD 0x8007000D
ERROR_INVALID_PARAMETER 87 One of the parameters was invalid. 0x57 0x80070057
ERROR_CALL_NOT_IMPLEMENTED 120 This value is returned when a custom action attempts to call a function that cannot be called from custom actions. The function returns the value ERROR_CALL_NOT_IMPLEMENTED. Available beginning with Windows Installer version 3.0. 0x78 0x80070078
ERROR_APPHELP_BLOCK 1259 If Windows Installer determines a product may be incompatible with the current operating system, it displays a dialog box informing the user and asking whether to try to install anyway. This error code is returned if the user chooses not to try the installation. 0x4EB 0x800704EB
ERROR_INSTALL_SERVICE_FAILURE 1601 The Windows Installer service could not be accessed. Contact your support personnel to verify that the Windows Installer service is properly registered. 0x641 0x80070641
ERROR_INSTALL_USEREXIT 1602 The user cancels installation. 0x642 0x80070642
ERROR_INSTALL_FAILURE 1603 A fatal error occurred during installation. 0x643 0x80070643
ERROR_INSTALL_SUSPEND 1604 Installation suspended, incomplete. 0x644 0x80070644
ERROR_UNKNOWN_PRODUCT 1605 This action is only valid for products that are currently installed. 0x645 0x80070645
ERROR_UNKNOWN_FEATURE 1606 The feature identifier is not registered. 0x646 0x80070646
ERROR_UNKNOWN_COMPONENT 1607 The component identifier is not registered. 0x647 0x80070647
ERROR_UNKNOWN_PROPERTY 1608 This is an unknown property. 0x648 0x80070648
ERROR_INVALID_HANDLE_STATE 1609 The handle is in an invalid state. 0x649 0x80070649
ERROR_BAD_CONFIGURATION 1610 The configuration data for this product is corrupt. Contact your support personnel. 0x64A 0x8007064A
ERROR_INDEX_ABSENT 1611 The component qualifier not present. 0x64B 0x8007064B
ERROR_INSTALL_SOURCE_ABSENT 1612 The installation source for this product is not available. Verify that the source exists and that you can access it. 0x64C 0x8007064C
ERROR_INSTALL_PACKAGE_VERSION 1613 This installation package cannot be installed by the Windows Installer service. You must install a Windows service pack that contains a newer version of the Windows Installer service. 0x64D 0x8007064D
ERROR_PRODUCT_UNINSTALLED 1614 The product is uninstalled. 0x64E 0x8007064E
ERROR_BAD_QUERY_SYNTAX 1615 The SQL query syntax is invalid or unsupported. 0x64F 0x8007064F
ERROR_INVALID_FIELD 1616 The record field does not exist. 0x650 0x80070650
ERROR_INSTALL_ALREADY_RUNNING 1618 Another installation is already in progress. Complete that installation before proceeding with this install. 0x652 0x80070652
ERROR_INSTALL_PACKAGE_OPEN_FAILED 1619 This installation package could not be opened. Verify that the package exists and is accessible, or contact the application vendor to verify that this is a valid Windows Installer package. 0x653 0x80070653
ERROR_INSTALL_PACKAGE_INVALID 1620 This installation package could not be opened. Contact the application vendor to verify that this is a valid Windows Installer package. 0x654 0x80070654
ERROR_INSTALL_UI_FAILURE 1621 There was an error starting the Windows Installer service user interface. Contact your support personnel. 0x655 0x80070655
ERROR_INSTALL_LOG_FAILURE 1622 There was an error opening installation log file. Verify that the specified log file location exists and is writable. 0x656 0x80070656
ERROR_INSTALL_LANGUAGE_UNSUPPORTED 1623 This language of this installation package is not supported by your system. 0x657 0x80070657
ERROR_INSTALL_TRANSFORM_FAILURE 1624 There was an error applying transforms. Verify that the specified transform paths are valid. 0x658 0x80070658
ERROR_INSTALL_PACKAGE_REJECTED 1625 This installation is forbidden by system policy. Contact your system administrator. 0x659 0x80070659
ERROR_FUNCTION_NOT_CALLED 1626 The function could not be executed. 0x65A 0x8007065A
ERROR_FUNCTION_FAILED 1627 The function failed during execution. 0x65B 0x8007065B
ERROR_INVALID_TABLE 1628 An invalid or unknown table was specified. 0x65C 0x8007065C
ERROR_DATATYPE_MISMATCH 1629 The data supplied is the wrong type. 0x65D 0x8007065D
ERROR_UNSUPPORTED_TYPE 1630 Data of this type is not supported. 0x65E 0x8007065E
ERROR_CREATE_FAILED 1631 The Windows Installer service failed to start. Contact your support personnel. 0x65F 0x8007065F
ERROR_INSTALL_TEMP_UNWRITABLE 1632 The Temp folder is either full or inaccessible. Verify that the Temp folder exists and that you can write to it. 0x660 0x80070660
ERROR_INSTALL_PLATFORM_UNSUPPORTED 1633 This installation package is not supported on this platform. Contact your application vendor. 0x661 0x80070661
ERROR_INSTALL_NOTUSED 1634 Component is not used on this machine. 0x662 0x80070662
ERROR_PATCH_PACKAGE_OPEN_FAILED 1635 This patch package could not be opened. Verify that the patch package exists and is accessible, or contact the application vendor to verify that this is a valid Windows Installer patch package. 0x663 0x80070663
ERROR_PATCH_PACKAGE_INVALID 1636 This patch package could not be opened. Contact the application vendor to verify that this is a valid Windows Installer patch package. 0x664 0x80070664
ERROR_PATCH_PACKAGE_UNSUPPORTED 1637 This patch package cannot be processed by the Windows Installer service. You must install a Windows service pack that contains a newer version of the Windows Installer service. 0x665 0x80070665
ERROR_PRODUCT_VERSION 1638 Another version of this product is already installed. Installation of this version cannot continue. To configure or remove the existing version of this product, use Add/Remove Programs in Control Panel. 0x666 0x80070666
ERROR_INVALID_COMMAND_LINE 1639 Invalid command line argument. Consult the Windows Installer SDK for detailed command-line help. 0x667 0x80070667
ERROR_INSTALL_REMOTE_DISALLOWED 1640 The current user is not permitted to perform installations from a client session of a server running the Terminal Server role service. 0x668 0x80070668
ERROR_SUCCESS_REBOOT_INITIATED 1641 The installer has initiated a restart. This message is indicative of a success. 0x669 0x80070669
ERROR_PATCH_TARGET_NOT_FOUND 1642 The installer cannot install the upgrade patch because the program being upgraded may be missing or the upgrade patch updates a different version of the program. Verify that the program to be upgraded exists on your computer and that you have the correct upgrade patch. 0x66A 0x8007066A
ERROR_PATCH_PACKAGE_REJECTED 1643 The patch package is not permitted by system policy. 0x66B 0x8007066B
ERROR_INSTALL_TRANSFORM_REJECTED 1644 One or more customizations are not permitted by system policy. 0x66C 0x8007066C
ERROR_INSTALL_REMOTE_PROHIBITED 1645 Windows Installer does not permit installation from a Remote Desktop Connection. 0x66D 0x8007066D
ERROR_PATCH_REMOVAL_UNSUPPORTED 1646 The patch package is not a removable patch package. Available beginning with Windows Installer version 3.0. 0x66E 0x8007066E
ERROR_UNKNOWN_PATCH 1647 The patch is not applied to this product. Available beginning with Windows Installer version 3.0. 0x66F 0x8007066F
ERROR_PATCH_NO_SEQUENCE 1648 No valid sequence could be found for the set of patches. Available beginning with Windows Installer version 3.0. 0x670 0x80070670
ERROR_PATCH_REMOVAL_DISALLOWED 1649 Patch removal was disallowed by policy. Available beginning with Windows Installer version 3.0. 0x671 0x80070671
ERROR_INVALID_PATCH_XML 1650 The XML patch data is invalid. Available beginning with Windows Installer version 3.0. 0x672 0x80070672
ERROR_PATCH_MANAGED _ADVERTISED_PRODUCT 1651 Administrative user failed to apply patch for a per-user managed or a per-machine application that is in advertise state. Available beginning with Windows Installer version 3.0. 0x673 0x80070673
ERROR_INSTALL_SERVICE_SAFEBOOT 1652 Windows Installer is not accessible when the computer is in Safe Mode. Exit Safe Mode and try again or try using System Restore to return your computer to a previous state. Available beginning with Windows Installer version 4.0. 0x674 0x80070674
ERROR_ROLLBACK_DISABLED 1653 Could not perform a multiple-package transaction because rollback has been disabled. Multiple-Package Installationscannot run if rollback is disabled. Available beginning with Windows Installer version 4.5. 0x675 0x80070675
ERROR_INSTALL_REJECTED 1654 The app that you are trying to run is not supported on this version of Windows. A Windows Installer package, patch, or transform that has not been signed by Microsoft cannot be installed on an ARM computer. 0x676 0x80070676
ERROR_SUCCESS_REBOOT_REQUIRED 3010 A restart is required to complete the install. This message is indicative of a success. This does not include installs where the ForceReboot action is run. 0xBC2 0x80070BC2

Any installer that uses standard MSI technology will likely use these same error codes(i.e the Microsoft C++ redistributables (vcredist_x86.exe, vcredist_x64.exe)).

View Archive (70 posts)