Making .Net Regular Expressions Easier To Create and Maintain

Recently I had the idea of a log parsing and analysis system, that would allow users (via a web interface) to define their own regular expressions to parse event logs in different ways. The main purpose of this is for trend analysis. Logstash is an open source project that does something similar, but it relies on configuration files for specifying regular expressions (and doesn’t have other unrelated features I am envisioning).

One issue I have with regular expressions is they can be very hard to create and maintain as they increase in complexity. This difficulty works against you if you are trying to create a system where different people with different qualifications should be able to define how certain logs are parsed.

While looking at Logstash, I was led to a project that it uses called Grok, which essentially allows you to define aliases for regular expressions and even chain aliases together for more complicated regular expressions. For example, if you needed a regular expression that included checking for an IP address, you can write %{IP} instead of (?<![0-9])(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?![0-9]). This makes regular expressions much easier to create and read at later on.

The problem with Grok is that it is written in C as a stand alone application. This makes it hard to use in .Net applications without calling out to the shell, and even then it is cumbersome to use with dynamic aliases due to having to reconfigure config files on the fly.

For those reasons, I created an open source library I called RapidRegex. The first part of this library is the freedom to generate regular expression aliases by creating an instance of the RegexAlias class. This class is used to give you full flexibility on how you store and edit regular expression aliases in any way your application sees fit, whether it’s in a web form or in a flat file. It does however come with a class that helps form RegexAlias structures from basic configuration files.

As a simple example, let’s look at the regular expression outlined earlier for IP addresses. With RapidRegex, you can create an alias for it and later convert aliased regular expressions into .net regular expressions. An example of this can be seen with the following code:

            var alias = new RegexAlias
            {
                Name = "IPAddress",
                RegexPattern = @"\b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b"
            };

            var resolver = new RegexAliasResolver(new[] { alias });

            const string pattern = "connection from %{IPAddress}";
            var regexPattern = resolver.ResolveToRegex(pattern);
            // Resolved pattern becomes "connection from \b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b"

What makes this even more powerful is the fact that you can chain aliases together. For instance, an IPv4 address can be defined as 4 sets of valid IP address bytes. So we can accomplish the same above with:

            var alias = new RegexAlias
            {
                Name = "IPDigit",
                RegexPattern = @"(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)"
            };

            var alias2 = new RegexAlias
            {
                Name = "IPAddress",
                RegexPattern = @"%{IPDigit}\.%{IPDigit}\.%{IPDigit}\.%{IPDigit}"
            };

            var resolver = new RegexAliasResolver(new[] { alias, alias2 });

            const string pattern = "connection from %{IPAddress}";
            var regexPattern = resolver.ResolveToRegex(pattern);

            // Resolve the pattern into becomes "connection from \b(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b"

The project can be found on Github and is licensed under the LGPL license.

Advertisements

Creating a Single Threaded Multi-User TCP Server In .Net

In my random desire to build a simple IRC server I decided the first step was to build a TCP server that can read data and send data simultaneously to multiple users. When someone first starts to write a networked server they quickly realize it’s not as easy as they had hoped due to all the network operations being done via blocking method calls. Before .Net 4.5, handling multiple socket connections meant having to put the socket handling calls into alternate threads, either by AsyncCallback calls or manually loading the method calls into the thread pool. These then require thread syncing when data needs to get back to the main thread so that the server’s business logic can run on the socket events.

With the introduction of the Async/Await keywords in .Net 4.5, it has become much easier to run your socket operations asynchronously while never leaving the main application thread. If you are not familiar with the Async/Await keywords it would probably be a good thing to read Stephen Cleary’s tutorial.

So here is how I went about making a single threaded TCP server that can handle multiple clients. This may not be the best way, but it did seem to be the most successful attempt I had for this.

Network Clients

The first thing I needed was a class to handle already connected clients. I needed the class to be able to take an already established connection and have it be able to receive network messages (and pass them to the business logic for processing) as well as sending messages back to the client. It also needs to determine when a connection has been dropped (either gracefully or not). Most importantly, these actions should not block the general server flow. As long as the server’s business logic processor is not actually processing a network message, it should not be waiting on a networked client.

Since both receiving messages from the client and a client being disconnected are important aspects that the underlying server should be aware of, I defined the following delegates:

    public delegate void MessageReceivedDelegate(NetworkClient client, string message);

    public delegate void ClientDisconnectedDelegate(NetworkClient client);

I then created the skeleton of the NetworkClient like so:

    public class NetworkClient
    {
        private readonly TcpClient _socket;
        private NetworkStream _networkStream;
        private readonly int _id;

        public bool IsActive { get; set; }
        public int Id { get { return _id; } }
        public TcpClient Socket { get { return _socket; } }

        public event MessageReceivedDelegate MessageReceived;
        public event ClientDisconnectedDelegate ClientDisconnected;

        public NetworkClient(TcpClient socket, int id)
        {
            _socket = socket;
            _id = id;
        }

        private void MarkAsDisconnected()
        {
            IsActive = false;
            if (ClientDisconnected != null)
                ClientDisconnected(this);
        }
     }

This handles creating the client (not the connection, but the class itself) and a helper method for handling when a client has been disconnected. Now we need to listen for incoming data on for this TCP client. Since reading TCP input is blocking, we need to perform this asynchronously with the following code:

        public async Task ReceiveInput()
        {
            IsActive = true;
            _networkStream = _socket.GetStream();

            using (var reader = new StreamReader(_networkStream))
            {
                while (IsActive)
                {
                    try
                    {
                        var content = await reader.ReadLineAsync();

                        // If content is null, that means the connection has been gracefully disconnected
                        if (content == null)
                        {
                            MarkAsDisconnected();
                            return;
                        }

                        if (MessageReceived != null)
                            MessageReceived(this, content);
                    }

                    // If the tcp connection is ungracefully disconnected, it will throw an exception
                    catch (IOException)
                    {
                        MarkAsDisconnected();
                        return;
                    }
                }
            }
        }

This method essentially loops until the network client is not activate anymore and awaits for a full line of data to be returned by the client. If a null response is returned or an IOException occurs, then that means the connection has been disconnected and I need to mark the client as such. By making sure we are awaiting for incoming data, it assures we do not block the application flow while waiting for data to come down the pipe. This method returns a Task, so that we can check for unhandled exceptions in the main server.

Next we need to be able to asynchronously send data to the user. I do this with the following method:

        public async Task SendLine(string line)
        {
            if (!IsActive)
                return;

            try
            {
                // Don't use a using statement as we do not want the stream closed
                //    after the write is completed
                var writer = new StreamWriter(_networkStream);
                await writer.WriteLineAsync(line);
                writer.Flush();
            }
            catch (IOException)
            {
                // socket closed
                MarkAsDisconnected();
            }
        }

I do this asynchronously so we don’t risk blocking the entire server while waiting for the whole TCP process to finish.

Server Infrastructure

Now that we have a fully working class to handle networked clients, we need to create the server infrastructure. We need some way to:

  • Accept new clients on a specific IP address and port
  • Turn those clients into NetworkClient instances
  • Process commands coming from network clients
  • Handle client disconnections

This begins with the following class skeleton:

    public class Server
    {
        private readonly TcpListener _listener;
        private readonly List&lt;NetworkClient&gt; _networkClients;
        private readonly List&lt;KeyValuePair&lt;Task, NetworkClient&gt;&gt; _networkClientReceiveInputTasks; 
        private Task _clientListenTask;

        public bool IsRunning { get; private set; }

        public Exception ClientListenTaskException
        {
            get { return _clientListenTask.Exception; }
        }

        public Server(IPAddress ip, int port)
        {
            _listener = new TcpListener(ip, port); 
            _networkClients = new List&lt;NetworkClient&gt;();
            _networkClientReceiveInputTasks = new List&lt;KeyValuePair&lt;Task, NetworkClient&gt;&gt;();
        }
    }

The _networClientRecieveInputTasks would be used to check for exceptions while listening for input from a client. The client listen task will be used to reference the asynchronous task that listens for new client connections, and this would be use this to check for unhandled exceptions being thrown. Everything else is to get data ready for actually running the server.

We need to consider what to do when a command is received by the client. To get up and running quickly we are just going to relay the incoming data out to all other clients, via the following method:

        private async void ProcessClientCommand(NetworkClient client, string command)
        {
            Console.WriteLine("Client {0} wrote: {1}", client.Id, command);

            foreach (var netClient in _networkClients)
                if (netClient.IsActive)
                    await netClient.SendLine(command);
        }

Now we need to handle the NetworkClient.ClientDisconnected event. In this case all we want to do is close the network socket and remove the client from our list.

        private void ClientDisconnected(NetworkClient client)
        {
            client.IsActive = false;
            client.Socket.Close();

            if (_networkClients.Contains(client))
                _networkClients.Remove(client);

            Console.WriteLine("Client {0} disconnected", client.Id);
        }

The next thing we need is to figure out what we want to do when a client is connected. When a client connects we need to create a NetworkClient instance for them, assign them an identification number (for internal use only), hook into the NetworkClient’s events, and start listening for input from that client. This can be accomplished with the following method:

        private void ClientConnected(TcpClient client, int clientNumber)
        {
            var netClient = new NetworkClient(client, clientNumber);
            netClient.MessageReceived += ProcessClientCommand;
            netClient.ClientDisconnected += ClientDisconnected;

            // Save the Resulting task from ReceiveInput as a Task so
            //   we can check for any unhandled exceptions that may have occured
            _networkClientReceiveInputTasks.Add(new KeyValuePair&lt;Task, NetworkClient&gt;(netClient.ReceiveInput(),
                                                                                      netClient));

            _networkClients.Add(netClient);
            Console.WriteLine("Client {0} Connected", clientNumber);
        }

This will take a TcpClient, create a new NetworkClient for it, tie up the events and start receiving input.

We now have everything e need to execute information from a client, we just need to actually accept incoming client connections. This of course needs to be done asynchronously so we do not block the server flow while waiting for a new connection.

        private async Task ListenForClients()
        {
            var numClients = 0;
            while (IsRunning)
            {
                var tcpClient = await _listener.AcceptTcpClientAsync();
                ClientConnected(tcpClient, numClients);
                numClients++;
            }

            _listener.Stop();
        }

        public void Run()
        {
            _listener.Start();
            IsRunning = true;

            _clientListenTask = ListenForClients();
        }

That’s pretty much all the code that is needed. Now all you have to do is add the server calls to your main method:

         static void Main(string[] args)
         {
             var server = new Server(IPAddress.Any, 9001);
             server.Run();

             while (server.IsRunning)
             {
                 Thread.Sleep(100);
             }
         }

We need the while() loop due to the main functionality of the server running asynchronously. Otherwise the program would immediately exit. Now if you run the server you will be able to connect multiple telnet sessions to each other and pass messages back and forth.

Threading

The problem with our server so far is that it is not operating in a single thread. This means that once you have a lot of clients connecting, sending messages, etc.. you will come up with syncing issues (especially once you start adding real business logic to the mix).

Outside of WPF and Winforms applications async/await operate in the threadpool not in the main thread. This means you cannot 100% predict which await operations will work on which threads (you can read more about this at this MSDN blog).

If you want proof of this on your sample server, you can add the following code everywhere you have any other Console.WriteLine() call:

Console.WriteLine("Thread Id: {0}", Thread.CurrentThread.ManagedThreadId);

If you also add this to your Main() method and run the application you will notice multiple thread ids being displayed in the console.

Async and await commands utilize the SynchronizationContext which controls how (and on what threads) the different actions are run on. Based on that reference, I created the following SynchronizationContext implementation.

    public class SingleThreadSynchronizationContext  : SynchronizationContext
    {
        private readonly Queue&lt;Action&gt; _messagesToProcess = new Queue&lt;Action&gt;();
        private readonly object _syncHandle = new object();
        private bool _isRunning = true;

        public override void Send(SendOrPostCallback codeToRun, object state)
        {
            throw new NotImplementedException();
        }

        public override void Post(SendOrPostCallback codeToRun, object state)
        {
            lock (_syncHandle)
            {
                _messagesToProcess.Enqueue(() =&gt; codeToRun(state));
                SignalContinue();
            }
        }

        public void RunMessagePump()
        {
            while (CanContinue())
            {
                Action nextToRun = GrabItem();
                nextToRun();
            }
        }

        private Action GrabItem()
        {
            lock (_syncHandle)
            {
                while (CanContinue() && _messagesToProcess.Count == 0)
                {
                    Monitor.Wait(_syncHandle);
                }
                return _messagesToProcess.Dequeue();
            }
        }

        private bool CanContinue()
        {
            lock (_syncHandle)
            {
                return _isRunning;
            }
        }

        public void Cancel()
        {
            lock (_syncHandle)
            {
                _isRunning = false;
                SignalContinue();
            }
        }

        private void SignalContinue()
        {
            Monitor.Pulse(_syncHandle);
        }
    }

I then had to update my program’s Main method to utilize the context.

         static void Main(string[] args)
         {
             var ctx = new SingleThreadSynchronizationContext();
             SynchronizationContext.SetSynchronizationContext(ctx);

             Console.WriteLine("Main Thread: {0}", Thread.CurrentThread.ManagedThreadId);
             var server = new Server(IPAddress.Any, 9001);
             server.Run();

             ctx.RunMessagePump();
         }

Now if you run the application and send some commands to the server you will see everything running asynchronously on a single thread.

Conclusion

This may not be the best approach, and a single-threaded TCP server is probably not the most efficient in a production environment, but it does give me a good baseline to work with to expand out its capabilities.

Mimicking Html.BeginForm() to reduce html div duplication in Asp.Net MVC sites

Recently I have been trying to find ways to reduce how much HTML I have to duplicate in my views and how I have to remember what css classes to give each set of divs. The problem comes with that the HTML that my views require don’t fit in the main layout, due to the fact that content still comes after it and some elements are optional, and they don’t fit well in partial views due to how customized the HTML inside the area is.

An example of the HTML I was dealing with, here’s an example of one of my views

<div class="grid1 floatLeft"> 
    <div class="lineSeperater"> 
        <div class="pageInfoBox"> 
            @using (Html.BeginForm(MVC.JobSearch.Edit())) 
            {
                @Html.HiddenFor(x => x.Id)
                
                <div class="grid3 marginBottom_10 marginAuto floatLeft"> 
                    <h3 class="floatLeft">@(isNewJobSearch ? "Start Job Search" : "Edit Job Search")</h3> 
                </div> 
                
                <div class="grid3 marginBottom_10 marginAuto floatLeft">
                    <div class="floatLeft">
                        <p>
                            Displayed page summary
                        </p>    
                    </div>
                </div>
                
                <div class="grid3 marginBottom_10 marginAuto floatleft">
                    <div class="floatLeft infoSpan">
                        @Html.ValidationSummary()
                    </div>
                </div>
                
                <div class="grid3 marginBottom_10 floatLeft"> 
                    <div class="floatLeft"><p class="greyHighlight">Title:</p>
                        <div class="infoSpan">@Html.TextBoxFor(x => x.Name, new { @class = "info" })</div>
                    </div> 
                </div> 

                <div class="grid3 marginBottom_10 floatLeft"> 
                    <div class="floatLeft"><p class="greyHighlight">Description:</p>
                        <div class="infoSpan">@Html.TextAreaFor(x => x.Description, new { @class = "textAreaInfo" })</div>
                    </div> 
                </div> 

                <div class="grid3 marginBottom_20 floatLeft"> 
                    <div class="submitBTN "><input type="submit" value="Save" /></div>                    
                </div> 
            }

            <div class="clear"></div> 
        </div> 
    </div> 
</div> 

<!-- More HTML Here -->

I started thinking of how I can increase my code re-use to make this easier to develop and maintain. While looking over the view my eyes gravitated towards the Html.BeginForm() and I realized the most logical idea was to utilize using statements. So after looking at the implementation of Html.BeginForm() for guidance (thanks to dotPeek) I came up with the following class to implement the first few divs automatically.

    public class PageInfoBoxWriter : IDisposable
    {
        protected ViewContext _viewContext;
        protected bool _includesSeparator;

        public PageInfoBoxWriter(ViewContext context, bool includeSeparator)
        {
            if (context == null)
                throw new ArgumentNullException("context");

            _viewContext = context;
            _includesSeparator = includeSeparator;

            // Write the html
            _viewContext.Writer.Write("<div class=\"grid1 floatLeft\">");

            if (_includesSeparator) _viewContext.Writer.Write("<div class=\"lineSeperater\">");

            _viewContext.Writer.Write("<div class=\"pageInfoBox\">");

            return;
        }

        public void Dispose()
        {
            _viewContext.Writer.Write("<div class=\"clear\"></div></div></div>");
            if (_includesSeparator) _viewContext.Writer.Write("</div>");
        }
    }

I then created an html helper to use this class

    public static class LayoutHelpers
    {
        public static PageInfoBoxWriter PageInfoBox(this HtmlHelper html, bool includeSeparator)
        {
            return new PageInfoBoxWriter(html.ViewContext, includeSeparator);
        }
    }

This will write the beginning divs upon creation and the div end tags upon closing. After following suit with the other elements of my page and creating more disposable layout classes and Html helpers, I now have the following view:

@using (Html.PageInfoBox(false))
{
    using (Html.BeginForm(MVC.JobSearch.Edit())) 
    {
        @Html.HiddenFor(x => x.Id)

        using (Html.OuterRow())
        {
            <h3 class="floatLeft">@(isNewJobSearch ? "Start Job Search" : "Edit Job Search")</h3> 
        }

        using (Html.OuterRow())
        {
            <div class="floatLeft">
                <p>
                    Displayed page summary
                </p>    
            </div>
        }

        using (Html.OuterRow())
        {
            <div class="floatLeft infoSpan">
                @Html.ValidationSummary()
            </div>
        }

        using (Html.OuterRow())
        {
            Html.FormField("Title:", Html.TextBoxFor(x => x.Name, new { @class = "info" }));
        }

        using (Html.OuterRow())
        {
            Html.FormField("Description:", @Html.TextAreaFor(x => x.Description, new { @class = "textAreaInfo" }));
        }

        using (Html.OuterRow())
        {
            using(Html.FormButtonArea())
            {
                <input type="submit" value="Save" />
            }
        }
    }
}

<!-- other html here -->

Now I have a MUCH more maintainable view that even gives me intellisense support, so I don’t have to worry about remembering how css classes are capitalized, how they are named, what order the need to be in, etc…

DotNetOpenAuth, OAuth, and MVC For Dummies

I recently was trying to understand OAuth so that I could utilize the LinkedIn API in my Asp.Net MVC application. The LinkedIn API has a pretty good summary of the OAuth process. However, the more I looked at it (and other documentation on OAuth), the more confused I got at implementing OAuth myself. So I went around and looked for a C# library to help me out, and found DotNetOpenAuth.Net. Unfortunately, the DotNetOpenAuth website is horribly designed without any real tutorials. After searching around the internet I was able to piece some things together, and hopefully this will help someone figure this out quicker than I was able to.

Requesting User Authorization

The first step of authorizing with OAuth and DotNetOpenAuth is to redirect to the OAuth provider’s authorization page, so the user can grant your application access to perform queries/service calls on their behalf. DotNetOpenAuth needs several pieces of information to begin this process. The first is to create a ServiceProviderDescription object, that contains the provider’s URL for retrieving the request token, URL for retrieving the access token, URL for requesting user authentication, what OAuth protocol version to use, and details of the tamper protection used to encode the OAuth signature. An example of creating the provider description for connecting to LinkedIn is:

        private ServiceProviderDescription GetServiceDescription()
        {
            return new ServiceProviderDescription
            {
                AccessTokenEndpoint = new MessageReceivingEndpoint("https://api.linkedin.com/uas/oauth/accessToken", HttpDeliveryMethods.PostRequest),
                RequestTokenEndpoint = new MessageReceivingEndpoint("https://api.linkedin.com/uas/oauth/requestToken", HttpDeliveryMethods.PostRequest),
                UserAuthorizationEndpoint = new MessageReceivingEndpoint("https://www.linkedin.com/uas/oauth/authorize", HttpDeliveryMethods.PostRequest),
                TamperProtectionElements = new ITamperProtectionChannelBindingElement[] { new HmacSha1SigningBindingElement() },
                ProtocolVersion = ProtocolVersion.V10a
            };
        }

The next thing that DotNetOpenAuth requires is a token manager. The token manager is a class which DotNetOpenAuth utilizes to store and retrieve the consumer key, consumer secret, and a token secret for a given access key. Since how you will store the user access tokens and token secrets will vary project to project, DotNetOpenAuth assumes you will create your own token storage and retrieval mechanism by implementing the IConsumerTokenManager interface.

For testing, I looked online for an in memory token manager class, and found the following code:

    public class InMemoryTokenManager : IConsumerTokenManager, IOpenIdOAuthTokenManager
    {
        private Dictionary<string, string> tokensAndSecrets = new Dictionary<string, string>();

        public InMemoryTokenManager(string consumerKey, string consumerSecret)
        {
            if (String.IsNullOrEmpty(consumerKey))
            {
                throw new ArgumentNullException("consumerKey");
            }

            this.ConsumerKey = consumerKey;
            this.ConsumerSecret = consumerSecret;
        }

        public string ConsumerKey { get; private set; }

        public string ConsumerSecret { get; private set; }

        #region ITokenManager Members

        public string GetConsumerSecret(string consumerKey)
        {
            if (consumerKey == this.ConsumerKey)
            {
                return this.ConsumerSecret;
            }
            else
            {
                throw new ArgumentException("Unrecognized consumer key.", "consumerKey");
            }
        }

        public string GetTokenSecret(string token)
        {
            return this.tokensAndSecrets[token];
        }

        public void StoreNewRequestToken(UnauthorizedTokenRequest request, ITokenSecretContainingMessage response)
        {
            this.tokensAndSecrets[response.Token] = response.TokenSecret;
        }

        public void ExpireRequestTokenAndStoreNewAccessToken(string consumerKey, string requestToken, string accessToken, string accessTokenSecret)
        {
            this.tokensAndSecrets.Remove(requestToken);
            this.tokensAndSecrets[accessToken] = accessTokenSecret;
        }

        /// <summary>
        /// Classifies a token as a request token or an access token.
        /// </summary>
        /// <param name="token">The token to classify.</param>
        /// <returns>Request or Access token, or invalid if the token is not recognized.</returns>
        public TokenType GetTokenType(string token)
        {
            throw new NotImplementedException();
        }

        #endregion

        #region IOpenIdOAuthTokenManager Members

        public void StoreOpenIdAuthorizedRequestToken(string consumerKey, AuthorizationApprovedResponse authorization)
        {
            this.tokensAndSecrets[authorization.RequestToken] = string.Empty;
        }

        #endregion
    }

Now that we have a Token Manager class to use, and a service description we can begin the authorization process. This can be accomplished with the following code:

        public ActionResult StartOAuth()
        {
            var serviceProvider = GetServiceDescription();
            var consumer = new WebConsumer(serviceProvider, _tokenManager);

            // Url to redirect to
            var authUrl = new Uri(Request.Url.Scheme + "://" + Request.Url.Authority + "/Home/OAuthCallBack");

            // request access
            consumer.Channel.Send(consumer.PrepareRequestUserAuthorization(authUrl, null, null));

            // This will not get hit!
            return null;
        }

This sets up the DotNetOpenAuth consumer object to use our in memory token manager, and our previously defined service description object. We then form the URL we want the service provider to redirect to after the user grants your application access. Finally we tell the consumer to send the user authorization request. The Send() method will end the execution of the Asp.Net page, and thus no code after the Send() call will be called. The user will then see the authorization page on the service provider’s website, which will allow them to allow or deny access for your application.

Receiving the OAuth CallBack

Once the user logs into the service provider and gives your application authorization, the service provider will redirect to the callback URL specified in the previous code. The service provider includes the oauth token and secret, which needs to be processed by DotNetOpenAuth. The following code can be done to process the auth token and store it and the secret in the token manager:

        public ActionResult OAuthCallback()
        {
            // Process result from the service provider
            var serviceProvider = GetServiceDescription();
            var consumer = new WebConsumer(serviceProvider, _tokenManager);
            var accessTokenResponse = consumer.ProcessUserAuthorization();

            // If we didn't have an access token response, this wasn't called by the service provider
            if (accessTokenResponse == null)
                return RedirectToAction("Index");

            // Extract the access token
            string accessToken = accessTokenResponse.AccessToken;

            ViewBag.Token = accessToken;
            ViewBag.Secret = _tokenManager.GetTokenSecret(accessToken);
            return View();
        }

Perform A Request Using OAuth Credentials

Now that we have the user’s authorization details we can perform API queries. In order to query the API though, we need to sign our requests with a combination of the user’s access token and our consumer key. Since we retrieved the user’s access token in the previous code, you need to figure out a way to store that somewhere, either in the user’s record in the database, in a cookie, or any other way you can quickly get at it again without requiring the user to constantly re-auth.

In order to use that access token to call a service provider API function, you can form a prepared HttpWebRequest by calling the PrepareAuthorizedRequest() method on the WebConsumer class. The following is an example how to use an access token to query the LinkedIn API.

        public ActionResult Test2()
        {
            // Process result from linked in
            var LiServiceProvider = GetServiceDescription();
            var linkedIn = new WebConsumer(LiServiceProvider, _tokenManager);
            var accessToken = GetAccessTokenForUser();

            // Retrieve the user's profile information
            var endpoint = new MessageReceivingEndpoint("http://api.linkedin.com/v1/people/~", HttpDeliveryMethods.GetRequest);
            var request = linkedIn.PrepareAuthorizedRequest(endpoint, accessToken);
            var response = request.GetResponse();
            ViewBag.Result = (new StreamReader(response.GetResponseStream())).ReadToEnd();

            return View();
        }

And now, if the user has authenticated going to /Home/Test2 will correctly access LinkedIn on behalf of the user!

Update: For those who are looking for a tool to help test API calls prior to having to write them down formally in code, please see my Testing Oauth APIs Without Coding article!

From C# To Java: Events

I have been developing in C# for the last 3-4 years, and before that I was primarily a php coder. During one semester of my freshman year of college I did a little Java, but not much, and I have not done any since. I have decided to work on an N-Tier application in Java to give me a good project under my belt and show that I can use Java (and not just say to people that I can learn Java).

The problem with going from C# to Java is that the language is very similar. This is a problem because while I can figure out the language itself pretty easily, most “getting started” guideand tutorials are too beginner level to really be helpful. However, jumping straight into something such as a Spring framework tutorial or many other more high level tutorials can be pretty overwhelming, as they all assume a general knowledge of Java concepts that are not immediately obvious by just looking at a C# to Java language guide. Therefore, I decided to write a series containing conceptual differences or stumbles I find on my journey through implementing a Java application.

Events in C#

The first conceptual hurdle I came across was dealing with Events. In C#, events are pretty easy to deal with, as an event is can be described as a collection of methods that all conform to a single delegate’s method signature. An example of this, based on a good tutorial on Code Project is:

using System;
namespace wildert
{
    public class Metronome
    {
        public event TickHandler Tick;
        public EventArgs e = null;
        public delegate void TickHandler(Metronome m, EventArgs e);
        public void Start()
        {
            while (true)
            {
                System.Threading.Thread.Sleep(3000);
                if (Tick != null)
                {
                    Tick(this, e);
                }
            }
        }
    }
	
    class Test
    {
        static void Main()
        {
            Metronome m = new Metronome();
            m.Tick += HeardIt;
            m.Start();
        }
		
		private void HeardIt(Metronome m, EventArgs e)
		{
			System.Console.WriteLine("HEARD IT");
		}
    }
}

This is essentially adding the HeardIt method to the Metronome.Tick event, so whenever the event is fired, it calls theTest.HeardIt() method (along with any other method attached to the event). This is pretty straightforward in my (biased) opinion.

Events In Java

I started reading some pages on the net about how events in Java are handled. The first few articles I came across were kind of confusing and all over the place. I felt like I almost understood how it’s done but I was missing one piece of crucial information that binds it all together for me. After reading a few more articles, I finally had my “Ah ha!” moment, and it all clicked.

The confusion was due to that fact that unlike in C#, there is no event keyword in Java. In fact, and this is the piece I was missing, there essentially is no such thing as an event in Java. In Java you are essentially faking events and event handling by using a standard set of naming conventions and fully utilizing object-oriented programming.

Using events in java is really just a matter of defining an interface that contains a method to handle your event (this interface is known as the listener interface). You then implement this interface in the class that you want to handle the event (this is the listener class). In the class that you wish to “fire off” the event from you maintain a collection of class instances that implements the listener interface, and provide a method so listener classes can pass an instance of themselves in to add and remove them from the collection. Finally, the act of firing off events is merely going through the collection of listener classes, and on each listener call the listener interface’s method. It’s essentially a pure-OOP solution masquerading as an event handling system.

To see this in action, let’s create the equivalent of the C# event example in Java.

package com.kalldrexx.app

// Listener interface 
public interface MetronomeEvent {
	void Tick(Date tickDate);
}

// Listener implementation
public class MainApp implements MetronomeEvent {

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        EventFiringSource source = new EventFiringSource();
		source.addMetronomeEventListener(this); // Adds itself as a listener for the event
		source.Start();
    }
	
	public void Tick(Date tickDate)
	{
		// Output the tick date here
	}
}

// Event source
public class EventFiringSource {

	// Our collection of classes that are subscribed as listeners of our
    protected Vector _listeners;
	
	// Method for listener classes to register themselves
	public void addMetronomeEventListener(MetronomeEvent listener)
	{
		if (_listeners == null)
			_listeners = new Vector();
			
		_listeners.addElement(listener);
	}
	
	// "fires" the event
	protected void fireMetronomeEvent()
	{
		if (_listeners != null && _listeners.isEmpty())
		{
			Enumeration e = _listeners.elements();
			while (e.hasMoreElements())
			{
				MetronomeEvent e = (MetronomeEvent)e.nextElement();
				e.Tick(new Date());
			}
		}
	}
	
	public void Start()
	{
		fireMetronomeEvent();
	}
}

When the app starts (and enters the MainApp’s main() method), it creates the event source and tells it that the MainApp class should be registered as a listener for the Metronome event. When the event source class starts, it will “fire” off the event by just looking at all classes registered that implement the MetronomeEvent interface, and call the Tick() method for that implemented class.

No special magic, just pure object-oriented programming!