Top 10 Mistakes Node.js Developers Make _part 2(end)

3 Executing a callback multiple times

How many times have you saved a file and reloaded your Node web app only to see it crash really fast? The most likely scenario is that you executed the callback twice, meaning you forgot to return after the first time.
Let's create an example to replicate this situation. We will create a simple proxy server with some basic validation. To use it install the request dependency, run the example and open (for instance) http://localhost:1337/?url=http://www.google.com/. The source code for our example is the following:

1. var request = require('request');
2.   var http = require('http');
3.   var url = require('url');
4.   var PORT = process.env.PORT || 1337;
5. 
   
6.   var expression = /[-a-zA-Z0-9@:%_\+.~#?&//=]{2,256}\.[a-z]{2,4}\b(\/[-a-zA-Z0-9@:%_\+.~#?&//=]*)?/gi;
7.   var isUrl = new RegExp(expression);
8. 
   
9.   var respond = function(err, params) {
10.     var res = params.res;
11.     var body = params.body;
12.     var proxyUrl = params.proxyUrl;
13. 
    
14.     res.setHeader('Content-type', 'text/html; charset=utf-8');
15. 
    
16.     if (err) {
17.       console.error(err);
18.       res.end('An error occured. Please make sure the domain exists.');
19.     } else {
20.       res.end(body);
21.     }
22.   };
23. 
    
24.   http.createServer(function(req, res) {
25.     var queryParams = url.parse(req.url, true).query;
26.     var proxyUrl = queryParams.url;
27. 
    
28.     if (!proxyUrl || (!isUrl.test(proxyUrl))) {
29.       res.writeHead(200, { 'Content-Type': 'text/html' });
30.       res.write("Please provide a correct URL param. For ex: ");
31.       res.end("<a href='http://localhost:1337/?url=http://www.google.com/'>http://localhost:1337/?url=http://www.google.com/</a>");
32.     } else {
33.       // ------------------------
34.       // Proxying happens here
35.       // TO BE CONTINUED
36.       // ------------------------
37.     }
38.   }).listen(PORT);

The source code above contains almost everything except the proxying itself, because I want you to take a closer look at it:

1. request(proxyUrl, function(err, r, body) {
2. if (err) {
3.     respond(err, {
4.     res: res,
5.     proxyUrl: proxyUrl
6.     });
7. }
8. 
   
9. respond(null, {
10.     res: res,
11.     body: body,
12.     proxyUrl: proxyUrl
13. });
14. });

In the callback we have handled the error condition, but forgot to stop the execution flow after calling the respond function. That means that if we enter a domain that doesn't host a website, the respond function will be called twice and we will get the following message in the terminal:

1.   Error: Can't set headers after they are sent.
2.       at ServerResponse.OutgoingMessage.setHeader (http.js:691:11)
3.       at respond (/Users/alexandruvladutu/www/airpair-2/3-multi-callback/proxy-server.js:18:7)
4. 
   
5. This can be avoided either by using the `return` statement or by wrapping the 'success' callback in the `else` statement:

1.   request(.., function(..params) {
2.     if (err) {
3.       return respond(err, ..);
4.     }
5. 
   
6.     respond(..);
7.   });
8. 
   
9.   // OR:
10. 
    
11.   request(.., function(..params) {
12.     if (err) {
13.       respond(err, ..);
14.     } else {
15.       respond(..);
16.     }
17.   });

4 The Christmas tree of callbacks (Callback Hell)

Every time somebody wants to bash Node they come up with the 'callback hell' argument. Some of them see callback nesting as unavoidable, but that is simply untrue. There are a number of solutions out there to keep your code nice and tidy, such as:

• Using control flow modules (such as async);
• Promises; and
• Generators.

We are going to create a sample application and then refactor it to use the async module. The app will act as a naive frontend resource analyzer which does the following:

• Checks how many scripts / stylesheets / images are in the HTML code;
• Outputs the their total number to the terminal;
• Checks the content-length of each resource; then
• Puts the total length of the resources to the terminal.

Besides the async module, we will be using the following npm modules:

• request for getting the page data (body, headers, etc).
• cheerio as jQuery on the backend (DOM element selector).
• once to make sure our callback is executed once.

1.  var URL = process.env.URL;
2.   var assert = require('assert');
3.   var url = require('url');
4.   var request = require('request');
5.   var cheerio = require('cheerio');
6.   var once = require('once');
7.   var isUrl = new RegExp(/[-a-zA-Z0-9@:%_\+.~#?&//=]{2,256}\.[a-z]{2,4}\b(\/[-a-zA-Z0-9@:%_\+.~#?&//=]*)?/gi);
8. 
   
9.   assert(isUrl.test(URL), 'must provide a correct URL env variable');
10. 
    
11.   request({ url: URL, gzip: true }, function(err, res, body) {
12.     if (err) { throw err; }
13. 
    
14.     if (res.statusCode !== 200) {
15.       return console.error('Bad server response', res.statusCode);
16.     }
17. 
    
18.     var $ = cheerio.load(body);
19.     var resources = [];
20. 
    
21.     $('script').each(function(index, el) {
22.       var src = $(this).attr('src');
23.       if (src) { resources.push(src); }
24.     });
25. 
    
26.     // .....
27.     // similar code for stylesheets and images
28.     // checkout the github repo for the full version
29. 
    
30.     var counter = resources.length;
31.     var next = once(function(err, result) {
32.       if (err) { throw err; }
33. 
    
34.       var size = (result.size / 1024 / 1024).toFixed(2);
35. 
    
36.       console.log('There are ~ %s resources with a size of %s Mb.', result.length, size);
37.     });
38. 
    
39.     var totalSize = 0;
40. 
    
41.     resources.forEach(function(relative) {
42.       var resourceUrl = url.resolve(URL, relative);
43. 
    
44.       request({ url: resourceUrl, gzip: true }, function(err, res, body) {
45.         if (err) { return next(err); }
46. 
    
47.         if (res.statusCode !== 200) {
48.           return next(new Error(resourceUrl + ' responded with a bad code ' + res.statusCode));
49.         }
50. 
    
51.         if (res.headers['content-length']) {
52.           totalSize += parseInt(res.headers['content-length'], 10);
53.         } else {
54.           totalSize += Buffer.byteLength(body, 'utf8');
55.         }
56. 
    
57.         if (!--counter) {
58.           next(null, {
59.             length: resources.length,
60.             size: totalSize
61.           });
62.         }
63.       });
64.     });
65.   });

This doesn't look that horrible, but you can go even deeper with nested callbacks. From our previous example you can recognize the Christmas tree at the bottom, where you see indentation like this:

1.         if (!--counter) {
2.           next(null, {
3.             length: resources.length,
4.             size: totalSize
5.           });
6.         }
7.       });
8.     });
9.   });

To run the app type the following into the command line:

1.   $ URL=https://bbc.co.uk/ node before.js
2.   # Sample output:
3.   # There are ~ 24 resources with a size of 0.09 Mb.

After a bit of refactoring using async our code might look like the following:

1.  var async = require('async');
2. 
   
3.   var rootHtml = '';
4.   var resources = [];
5.   var totalSize = 0;
6. 
   
7.   var handleBadResponse = function(err, url, statusCode, cb) {
8.     if (!err && (statusCode !== 200)) {
9.       err = new Error(URL + ' responded with a bad code ' + res.statusCode);
10.     }
11. 
    
12.     if (err) {
13.       cb(err);
14.       return true;
15.     }
16. 
    
17.     return false;
18.   };
19. 
    
20.   async.series([
21.     function getRootHtml(cb) {
22.       request({ url: URL, gzip: true }, function(err, res, body) {
23.         if (handleBadResponse(err, URL, res.statusCode, cb)) { return; }
24. 
    
25.         rootHtml = body;
26. 
    
27.         cb();
28.       });
29.     },
30.     function aggregateResources(cb) {
31.       var $ = cheerio.load(rootHtml);
32. 
    
33.       $('script').each(function(index, el) {
34.         var src = $(this).attr('src');
35.         if (src) { resources.push(src); }
36.       });
37. 
    
38.       // similar code for stylesheets && images; check the full source for more
39. 
    
40.       setImmediate(cb);
41.     },
42.     function calculateSize(cb) {
43.       async.each(resources, function(relativeUrl, next) {
44.         var resourceUrl = url.resolve(URL, relativeUrl);
45. 
    
46.         request({ url: resourceUrl, gzip: true }, function(err, res, body) {
47.           if (handleBadResponse(err, resourceUrl, res.statusCode, cb)) { return; }
48. 
    
49.           if (res.headers['content-length']) {
50.             totalSize += parseInt(res.headers['content-length'], 10);
51.           } else {
52.             totalSize += Buffer.byteLength(body, 'utf8');
53.           }
54. 
    
55.           next();
56.         });
57.       }, cb);
58.     }
59.   ], function(err) {
60.     if (err) { throw err; }
61. 
    
62.     var size = (totalSize / 1024 / 1024).toFixed(2);
63.     console.log('There are ~ %s resources with a size of %s Mb.', resources.length, size);
64.   });

5 Creating big monolithic applications

Developers new to Node come with mindsets from different languages and they tend to do things differently. For example including everything into a single file, not breaking things into their own modules and publishing to NPM, etc.

Take our previous example for instance. We have pushed everything into a single file, making it hard to test and read the code. But no worries, with a bit of refactoring we can make it much nicer and more modular. This will also help with 'callback hell' in case you were wondering.

If we extract the URL validator, the response handler, the request functionality and the resource processor into their own files our main one will look like so:

1.  // ...
2.   var handleBadResponse = require('./lib/bad-response-handler');
3.   var isValidUrl = require('./lib/url-validator');
4.   var extractResources = require('./lib/resource-extractor');
5.   var request = require('./lib/requester');
6. 
   
7.   // ...
8.   async.series([
9.     function getRootHtml(cb) {
10.       request(URL, function(err, data) {
11.         if (err) { return cb(err); }
12. 
    
13.         rootHtml = data.body;
14. 
    
15.         cb(null, 123);
16.       });
17.     },
18.     function aggregateResources(cb) {
19.       resources = extractResources(rootHtml);
20. 
    
21.       setImmediate(cb);
22.     },
23.     function calculateSize(cb) {
24.       async.each(resources, function(relativeUrl, next) {
25.         var resourceUrl = url.resolve(URL, relativeUrl);
26. 
    
27.         request(resourceUrl, function(err, data) {
28.           if (err) { return next(err); }
29. 
    
30.           if (data.res.headers['content-length']) {
31.             totalSize += parseInt(data.res.headers['content-length'], 10);
32.           } else {
33.             totalSize += Buffer.byteLength(data.body, 'utf8');
34.           }
35. 
    
36.           next();
37.         });
38.       }, cb);
39.     }
40.   ], function(err) {
41.     if (err) { throw err; }
42. 
    
43.     var size = (totalSize / 1024 / 1024).toFixed(2);
44.     console.log('\nThere are ~ %s resources with a size of %s Mb.', resources.length, size);
45.   });

The request functionality might look like this:

1. var handleBadResponse = require('./bad-response-handler');
2.   var request = require('request');
3. 
   
4.   module.exports = function getSiteData(url, callback) {
5.     request({
6.       url: url,
7.       gzip: true,
8.       // lying a bit
9.       headers: {
10.         'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.111 Safari/537.36'
11.       }
12.     }, function(err, res, body) {
13.       if (handleBadResponse(err, url, res && res.statusCode, callback)) { return; }
14. 
    
15.       callback(null, {
16.         body: body,
17.         res: res
18.       });
19.     });
20.   };

Note: you can check the full example in the github repo.

Now things are simpler, way easier to read and we can start writing tests for our app. We can go on with the refactoring and extract the response length functionality into its own module as well.

The good thing about Node is that it encourages you to write tiny modules and publish them to NPM. You will find modules for all kinds of things such as generating a random number between an interval. You should strive for modularity in your Node applications and keeping things as simple as possible.

An interesting article on how to write modules is the one from substack.

6 Poor logging

Many Node tutorials show you a small example that contains console.log here and there, so some developers are left with the impression that that's how they should implement logging in their application.
You should use something better than console.log when coding Node apps, and here's why:

• No need to use util.inspect for large, complex objects;
• Built-in serializers for things like errors, request and response objects;
• Support multiple sources for controlling where the logs go;
• Automatic inclusion of hostname, process id, application name;
• Supports multiple levels of logging (debug, info, error, fatal etc);
• Advanced functionality such as log file rotation, etc.

You can get all of those for free when using a production-ready logging module such as bunyan. On top of that you also get a handy CLI tool for development if you install the module globally.

Let's take a look at one of their examples on how to use it:

1.  var http = require('http');
2.   var bunyan = require('bunyan');
3. 
   
4.   var log = bunyan.createLogger({
5.     name: 'myserver',
6.     serializers: {
7.       req: bunyan.stdSerializers.req,
8.       res: bunyan.stdSerializers.res
9.     }
10.   });
11. 
    
12.   var server = http.createServer(function (req, res) {
13.     log.info({ req: req }, 'start request');  // <-- this is the guy we're testing
14.     res.writeHead(200, { 'Content-Type': 'text/plain' });
15.     res.end('Hello World\n');
16.     log.info({ res: res }, 'done response');  // <-- this is the guy we're testing
17.   });
18. 
    
19.   server.listen(1337, '127.0.0.1', function() {
20.     log.info('server listening');
21. 
    
22.     var options = {
23.       port: 1337,
24.       hostname: '127.0.0.1',
25.       path: '/path?q=1#anchor',
26.       headers: {
27.         'X-Hi': 'Mom'
28.       }
29.     };
30. 
    
31.     var req = http.request(options, function(res) {
32.       res.resume();
33.       res.on('end', function() {
34.         process.exit();
35.       })
36.     });
37. 
    
38.     req.write('hi from the client');
39.     req.end();
40.   });

If you run the example in the terminal you will see something like the following:

1.   $ node server.js
2.   {"name":"myserver","hostname":"MBP.local","pid":14304,"level":30,"msg":"server listening","time":"2014-11-16T11:30:13.263Z","v":0}
3.   {"name":"myserver","hostname":"MBP.local","pid":14304,"level":30,"req":{"method":"GET","url":"/path?q=1#anchor","headers":{"x-hi":"Mom","host":"127.0.0.1:1337","connection":"keep-alive"},"remoteAddress":"127.0.0.1","remotePort":61580},"msg":"start request","time":"2014-11-16T11:30:13.271Z","v":0}
4.   {"name":"myserver","hostname":"MBP.local","pid":14304,"level":30,"res":{"statusCode":200,"header":"HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nDate: Sun, 16 Nov 2014 11:30:13 GMT\r\nConnection: keep-alive\r\nTransfer-Encoding: chunked\r\n\r\n"},"msg":"done response","time":"2014-11-16T11:30:13.273Z","v":0}

But in development it's better to use the CLI tool like in the screenshot:

As you can see, bunyan gives you a lot of useful information about the current process, which is vital into production. Another handy feature is that you can pipe the logs into a stream (or multiple streams).

7 No tests

We should never consider our applications 'done' if we didn't write any tests for them. There's really no excuse for that, considering how many existing tools we have for that:

• Testing frameworks: mocha, jasmine, tape and many other
• Assertion modules: chai, should.js
• Modules for mocks, spies, stubs or fake timers such as sinon
• Code coverage tools: istanbul, blanket

The convention for NPM modules is that you specify a test command in your package.json, for example:

1.   {
2.     "name": "express",
3.     ...
4.     "scripts": {
5.       "test": "mocha --require test/support/env --reporter spec --bail --check-leaks test/ test/acceptance/",
6.       ...
7.    }

Then the tests can be run with npm test, no matter of the testing framework used.

Another thing you should consider for your projects is to enforce having all your tests pass before committing. Fortunately it is as simple as doing npm i pre-commit --save-dev.

You can also decide to enforce a certain code coverage level and deny commits that don't adhere to that level. The pre-commit module simply runs npm test automatically for you as a pre-commit hook.

In case you are not sure how to get started with writing tests you can either find tutorials online or browse popular Node projects on Github, such as the following:

• express
• loopback
• ghost
• hapi
• haraka

8 Not using static analysis tools

Instead of spotting problems in production it's better to catch them right away in development by using static analysis tools.
Tools such as ESLint help solve a huge array of problems, such as:

• Possible errors, for example: disallow assignment in conditional expressions, disallow the use of debugger.
• Enforcing best practices, for example: disallow declaring the same variable more then once, disallow use of arguments.callee.
• Finding potential security issues, such as the use of eval() or unsafe regular expressions.
• Detecting possible performance problems.
• Enforcing a consistent style guide.

For a more complete set of rules checkout the ESLint rules documentation page. You should also read the configuration documents if you want to setup ESLint for your project.
In case you were wondering where you can find a sample configuration file for ESLint, the Esprima project has one.

There are other similar linting tools out there such as JSLint or JSHint.

In case you want to parse the AST (abstract source tree) and create a static analysis tool by yourself, consider Esprima or Acorn.

9 Zero monitoring or profiling

Not monitoring or profiling a Node applications leaves you in the dark. You are not aware of vital things such as event loop delay, CPU load, system load or memory usage.

There are proprietary services that care of these things for you, such as the ones from New Relic, StrongLoop or Concurix, AppDynamics.

You can also achieve that by yourself with open source modules such as look or by gluing different NPM modules. Whatever you choose make sure you are always aware of the status of your application at all times, unless you want to receive weird phone calls at night.

10 Debugging with console.log

When something goes bad it's easy to just insert console.log in some places and debug. After you figure out the problem you remove the console.log debugging leftovers and go on.

The problem is that the next developer (or even you) might come along and repeat the process. That's why module like debug exist. Instead of inserting and deleting console.log you can replace it with the debug function and just leave it there.

Once the next guy tries to figure out the problem they just start the application using the DEBUG environment variable.

This tiny module has its benefits:

• Unless you start the app using the DEBUG environment variable nothing is displayed to the console.
• You can selectively debug portions of your code (even with wildcards).
• The output is beautifully colored into your terminal.

Let's take a look at their official example:

1. // app.js
2.   var debug = require('debug')('http')
3.     , http = require('http')
4.     , name = 'My App';
5. 
   
6.   // fake app
7. 
   
8.   debug('booting %s', name);
9. 
   
10.   http.createServer(function(req, res){
11.     debug(req.method + ' ' + req.url);
12.     res.end('hello\n');
13.   }).listen(3000, function(){
14.     debug('listening');
15.   });
16. 
    
17.   // fake worker of some kind
18. 
    
19.   require('./worker');
20. 
    
21. <!--code lang=javascript linenums=true-->
22. 
    
23.   // worker.js
24.   var debug = require('debug')('worker');
25. 
    
26.   setInterval(function(){
27.     debug('doing some work');
28.   }, 1000);

If we run the example with node app.js nothing happens, but if we include the DEBUG flag voila:

Besides your applications, you can also use it for tiny modules published to NPM. Unlike a more complex logger it only does the debugging job and it does it well.
Written by Alexandru Vladutu

If you found this post interesting, follow and support us.
Suggest for you:

Complete Node JS Developer Course Building 5 Real World Apps

Node.js Tutorials: The Web Developer Bootcamp

Learn How To Deploy Node.Js App on Google Compute Engine

Learn and Understand NodeJS

Learn Nodejs by Building 12 Projects