Request Header Parser Microservice

Before we get started you can view the project requirements here, you can get the code on github.com to follow along here and you can find a live version of this project here.

As with my other projects, we will be following the steps outlined in this previous post to set up our GitHub repository, clone it to our local machine, set up our dependencies, and finally deploy it to the web. I recommend you check it out If you are still getting to grips with the whole server side javascript environment and workflow.


For this JSON API server we will handle two different GET requests.

  1. Requests to the home '/' path which spits out a simple description of what the app does
  2. Requests to our /whoami path which serves up our specified JSON response

The home path request is a very simple, static text response as seen in the code below:

app.get('/', function(req, res) { 
  res.send('<p>Visit <a href="whoami">/whoami</a> to get the IP address, language and operating system for your browser.</p>');
});

The second request is a little bit more interesting, we begin by initializing our result object which gets sent back to the browser as the response:

app.get('/whoami', function(req, res) { 
  var result = {};

We initialize this here so that the object is empty each time the GET request is made.

While the IP address is the easiest item to grab please note that for this to work on a local machine the code looks a little bit different. For now we will just concentrate on the code that will work once we deploy it to the web.

Here is what the code looks like to grab the IP from the header and parse it out into just the part that we want.

result.ipaddress = req.headers['x-forwarded-for'].split(',')[0];

First we grab the x-forwarded-for property from the request headers. This returns a comma separated string that contains several IP addresses. It will be formatted similar to below:

89.123.123.123,::ffff:10.20.30.123,::ffff:127.0.0.1,10.10.10.22,::ffff:182.27.0.1

The first item is the one we want here, so as you can see in the next part of our code, we use the .split() method with a comma as the separator to split this string up into an array of each of these IP addresses. Then as we want the first item in this array, we access it by its index, [0].

Next we get the language header property in an almost identical manner:

result.language = req.headers['accept-language'].split(',')[0];

Again, this at first spits out a comma separated string with the first item the property we actually want.

The last property we will add in our JSON response is the operating system, this one is a little more complex to parse out but nothing too hard:

var ua = req.headers['user-agent'];
ua = ua.substring(ua.indexOf('(') + 1, ua.indexOf(')')); 
result.software = ua;

The first line here grabs the user-agent property from the request header, this returns a string which will appear something similar to this:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36

As you can see, this is a little tricker to parse out, the text that we want, appears within the first set of parenthesis. To get this, we use the .indexOf() method to find the first instance of ‘(‘ and the first instance of ‘)’, then we pass these values into the .substring() method to get our desired substring. As per the .substring() documentation – the first parameter is:

… an integer between 0 and the length of the string, specifying the offset into the string of the first character to include in the returned substring.

Basically, since the new substring will start inclusive of the index we pass to start at, we need to add 1 to this index so it doesn’t include the opening ‘(‘.

Finally, we send our result object back to the browser as you can see in the code below. This also closes our app.get() code block:

  res.send(result);
});

If you have any questions or feedback on my solution please don’t hesitate to contact me.

A beginner’s guide to getting a Node.js project up and running

While this is of course just my humble opinion I think this is a great flow to get your Node.js project up and running. Taking the leap from client side Javascript to server side Node.js can be daunting. When you are trying to learn, tools like Express application generator do too much behind the scenes and leave you overwhelmed. I believe it’s best to start with just the items and features you need and expand from there. This guide will walk you through setting up a project on GitHub, cloning it onto your local machine, setting up the npm package, pushing it back up to GitHub, and finally deploying it to glitch.com.

For this tutorial we will assume you have git, node and npm installed locally on your machine.

Just a further note, this might look long and wordy but only because I have tried to be as detailed as possible about every step of the way, this whole process can be completed in less than 5 minutes!


1. Create your GitHub repository

First off, go to this link (https://github.com/new) and create a new repository for your project, by all means go into as much detail as you like but really all you need to do is enter a name and check the box for ‘Initialize this repository with a README’ and click ‘Create repository’. Last thing to do before you leave GitHub for now is to copy the ‘Clone with HTTPS’ link to your clipboard.

2. Clone the project to your local machine

The next step is to open your terminal and navigate to your projects folder, then run the following command:

git clone https://github.com/<USER-NAME>/<PROJECT-NAME>.git

You should now have a new project folder matching the name of your GitHub repository. You should now enter this folder in your terminal.

3. Initialize your npm package

Now inside your actual project folder which we cloned from GitHub, you should run the following command:

npm init

This will bring up the npm package.json wizard which will bring you through the steps of naming your package, the author, git link etc. Once this is complete, you should have a package.json file in your project folder.

4. Edit package.json

Next we open up your new package.json file and add the following:

"private": true,
"dependencies": {
  "express": "4.x"
}

The first line ensures that your project is not accidentally added back to the public npm package registry. The second lists Express.js as a module we are going to use for our Node.js app.

5. Install package dependencies

Pretty straight forward, now we just run the following code which will install our dependencies. As per our package.json file the only module we specified was Express.js.

npm install

6. Create main server file

We won’t go into the specifics of writing a Node.js app here, but copied below is a simple server that will respond with ‘hello world’ when a GET request is made to port 3000. Save this in a file called ‘server.js’.

const express = require('express');
const app = express();

app.get('/', function (req, res) {
 res.send('Hello, World!');
});

app.listen(3000, function () {
 console.log('Example app listening on port 3000!');
});

If you open up your terminal again and run …

node server.js

… you should see a message be logged to your console – ‘Example app listening on port 3000!’. To test this, point your browser to http://localhost:3000/ where you should see our ‘Hello, World!’ message.

We have now completed our simple, full stack Javascript app!

7. Push changes back up to GitHub

Next we are going to commit our changes back up to GitHub. Open your terminal back up and run the following command:

git status

git can sometimes be a little daunting, it is always a good idea to run this command to give you an idea of where you are or what files might be out of sync. Once you run this command you should get a message telling you what files have been modified. To get these files added, committed and pushed back up to GitHub we will run the following commands, one at a time:

git add -A
git commit -m "your message"
git push origin master

Now you can go back to GitHub and open the repository which you created for this project. You should have all the same files and code as on your local machine.

8. Deploy your code live on the web

Having this app running on your local machine is fine and well but if you want to show it to the world or add it to a portfolio we’ll need to get it running on the web.

For this we will use glitch.com, a great tool with many features that go beyond the scope of this blog post. For now you can think of it as a free hosting platform but really it is much more. I would highly encourage you to read up on it! Glitch is made by the team who brought Trello to the world and also co-founded Stack-overflow.

First-things-first, sign up for Glitch (https://glitch.com/) using your GitHub account, it should be a pretty simple case of authorizing access. Once you’re in, click on ‘Start a New Project’ in the top left hand corner. You should be brought to a new page that is a blank project. In the top left hand corner you should see a name randomly assigned to your project, you can change this later, but for now we will concentrate on importing our project from GitHub. To do this, click on your project name, and select ‘Advanced Options’ at the bottom of the dropdown. You should now see several options. The GitHub options to ‘import’ and ‘export’ will probably be grayed out, above them you will see an option to ‘Grant access to import and export a repo’ – click this and go ahead and follow the prompts to grant Glitch access. Once that is complete you can click the ‘Import from GitHub’ button and you will see a new pop up asking for the link to your project repository. Its a little ‘janky‘ in that it actually asks for just the part of the URL starting with your username, so something like – ‘diarmuid-murphy/timestamp-microservice’. The import process should only take a second and you should soon see your project files loaded into Glitch.

One of the first things you might notice is that you are seeing a ‘Logs: Error’ message in the lefthand sidebar. We just have one more step here to fix this error and get this working. Open up your ‘package.json’ file which should be listed in the sidebar, find the “scripts” property in the object. This property should itself contain an object, in my case this just contains a “test” property. Inside this object add this line – “start”: “node server.js”, your scripts property should now look something like this:

"scripts": {
  "test": "echo \"Error: no test specified\" && exit 1",
  "start": "node server.js"
}

This line is required by Glitch to tell it how to start your server. You should now see some lines in your console mentioning that the app is ‘Installing’ and then finally that ‘Example app listening on port 3000!’. Now click the ‘Show Live’ link in the header, this should open a new page with its own unique URL displaying the ‘Hello, World!’ message.

Congratulations, your Node.js app is now live and kicking on the web and can be shared with the world!

If you have any questions or feedback on this tutorial please don’t hesitate to contact me.

Timestamp Microservice

Before we get started you can view the project requirements hereYou can also get this code on github.com to follow along here.


This is a pretty simple straight forward project where we will accept a UNIX timestamp or a ‘natural language’ date in our url and if valid will return an object containing both the UNIX timestamp and ‘natural language’ values.

For example http://localhost:8080/December%2015,%202015 will return:

{
  "unix": 1450166400,
  "natural": "December 15, 2015"
}

Or http://localhost:8080/1504506722 will return:

{
  "unix": 1504506722,
  "natural": "September 3, 2017"
}

One thing to note here is that a UNIX timestamp is location specific, namely GMT, so as I am in California as I write this, http://localhost:8080/0 will actually give me the date ‘December 31, 1969’ rather than what you might expect, ‘January 1, 1970’.

For this project we are going to be using Express.js in our Node.js environment. The first thing we will need to do is get our project folder ready, what I like to do is create a new repository on github.com, give it a name, select ‘Initialize this repository with a README’ and done, hit ‘Create repository’. Once that is finished copy the ‘Clone with HTTPS’ link to your clipboard and open up your terminal. Now navigate to your projects folder or wherever you want the project to live locally and run the following:

git clone https://github.com/YOUR-USERNAME/timestamp-microservice.git

You should now have a folder for your new project with just a few git files inside.

The next thing I like to do is run:

npm init

This will run the npm package wizard where you can name your project, name the author etc. Once you have done this, open up your new package.json file and add:

"private": true,
"dependencies": {
  "express": "4.x"
}

This will ensure that your project is not accidentally added to the main npm repository and it will also state that you are going to be using Express.js in this project. Save your file and close it before returning to your terminal and running:

npm install

Now that we have got our package set up and our dependencies installed we should commit our package back to github.com, assuming you’ve got git all set up at this point that might be as simple as this:

git add -A
git commit -m "this can be any message you like"
git push origin master

Now the next thing I would do is create my actual server file, server.js, this is where all the magic of this project will happen.

The Code

We start our project off as you might expect, requiring the express module and setting the variable ‘app’ to an instance of it:

var express = require('express');
var app = express();

The only other variable I declare out here is an array of the months, this will be helpful when we create Date objects as the month part of a Date object is zero indexed (0-11). We can then check the indexOf('December') which will be 11, perfect once we are parsing our ‘natural language’ dates and making UNIX timestamps:

var months = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December'];

Now we jump into our actual server responses, starting with a simple response on the home URL that show the user some example API calls:

app.get('/', function(req, res) { 
  res.send('<p>Example usage:</p><code>http://localhost:8080/December%2015,%202015</code><br><code>http://localhost:8080/1450137600</code>');
});

We are only going to have two other server responses, one if the query is numbers and the other is the query starts with letters. First we will deal with the numbers / UNIX timestamp. Express.js is very helpful in that it allows us to use RegEx to route our queries. Here is my our GET request router for numbers:

app.get('/:date([0-9]*)', function(req, res) {...

The :date part is an Express.js placeholder which will give us access to that property at req.params.date . The RegEx then says take any number, 0-9, followed by anything. This RegEx could certainly be better but it works for us for this simple example.

Next we create out ‘result’ object which is what will be returned by our API, we initialize it with ‘null’ values so that it is ‘reset’ each time and the correct values if they exist are added before the object is return as a response.

var result = { "unix": null, "natural": null };

Next we pull these numbers out and set them equal to a variable called ‘timestamp’. We have to use parseInt because these param values come as a string:

var timestamp = parseInt(req.params.date);

Now we create a new Date object using this timestamp:

var date = new Date(timestamp * 1000);

Now can set our ‘unix’ value in our result object to the value of our ‘timestamp’ variable.

result.unix = timestamp;

Then we build our ‘natural’ value in our result object by taking the Month, Day and Year from our newly created Date object. Also notice how we use the Month value as the index to pull our actual Month name from our ‘months’ array:

result.natural = (months[date.getMonth()]) + ' ' + date.getDate() + ', ' + date.getFullYear();

Then finally we send this response back to the browser (this also closes our app.get() code block):

  res.send(result);
});

Now we move onto our final server response route which deals with a ‘natural language’ date, I use quotes because its actually pretty rigid about how it will take the date but it looks like its natural language. The date must be in the format: September 3, 2017. Again we use Express.js RegEx routing to pick up GET requests that start with letters:

app.get('/:natString([a-zA-Z]*)', function(req, res) {...

Here we use the placeholder :natString which again we can access as req.params.natString and our RegEx that says anything with a lower or uppercase letter followed by anything. Again this RegEx is pretty rudimentary but for our use it gets the job done. If we were to expand or further develop this API this would be one of the first places to start.

The next thing we do, as before, is create our result object containing null values:

var result = { "unix": null, "natural": null };

After this we take our ‘natural language’ date string and split it up into an array, at this point the input ‘September 15, 2017’ would return ['September', '15,' '2017']. Notice the comma after 15, we will need to deal with this later.

var dateArr = req.params.natString.split(' ');

Now we validate if the first item in our array, which should be a month, can be found in our array of months that we created at the start. If it is not, we treat this API request as invalid and return the result object with the null values:

if (months.indexOf(dateArr[0]) > -1) {...

If it is indeed in our array of months we treat this as a valid request and start parsing out our date values as required, here we come across one of JavaScripts wonderful quirks, days as you might expect take values from 1-31, years again as expected take a value such as 1995 which indeed corresponds to the year 1995 but months do not follow this convention and are 0 indexed, so December is represented by 11, January by 0 and so on. This is where our array of months that we created at the start will help us, we can check the index of the first value that the user provided which now lives in our dateArr variable and this will be perfect for building our new Date object:

var month = months.indexOf(dateArr[0]); 
var day = dateArr[1].replace(',', ''); 
var year = dateArr[2];

The only other thing to note here is that our day value still contains a comma. We trim this off simply using .replace().

Now that we have our date values parsed we build our new Date object:

var date = new Date(year, month, day);

And with this we can pull our UNIX timestamp and date values.

result.unix = date.getTime() / 1000;
result.natural = (months[date.getMonth()]) + ' ' + date.getDate() + ', ' + date.getFullYear();

Really we could have just joined our dateArr back together to give our ‘natural language’ date but by passing it into the Date object and pulling it back from there it feels more fool proof to me, once we do the reverse on a date, UNIX to ‘natural language’ or visa versa this makes sure they are correct rather than just assuming.

The last thing we do in this if () code block is pass the result object back as a response from the server:

  res.send(result);
}

Outside this if () code block the next thing we see is the default server response that will return the result object with the null values if our ‘natural language’ date is considered invalid:

  res.send(result); res.send(result);
});

As you can see this closes our final app.get() code block, the final piece of code is our app.listen() code block which tell the app to listen to port 8080:

app.listen(8080, function() { 
  console.log('App listening on port 8080.');
});

And that’s it, all done! You can check out this code in full on github.com here.

Please feel free to contact me if you have any questions or have any feedback on my code.