How to scale application in NodeJS and concept of packaging

Last updated on Jan 18 2023
Prabhas Ramanathan

Table of Contents

Node.js – Scaling Application

Node.js runs in a single-thread mode, but it uses an event-driven paradigm to handle concurrency. It also facilitates creation of child processes to leverage parallel processing on multi-core CPU based systems.
Child processes always have three streams child.stdin, child.stdout, and child.stderr which may be shared with the stdio streams of the parent process.
Node provides child_process module which has the following three major ways to create a child process.
• exec − child_process.exec method runs a command in a shell/console and buffers the output.
• spawn − child_process.spawn launches a new process with a given command.
• fork − The child_process.fork method is a special case of the spawn() to create child processes.

The exec() method

child_process.exec method runs a command in a shell and buffers the output. It has the following signature −
child_process.exec(command[, options], callback)
Parameters
Here is the description of the parameters used −
• command (String) The command to run, with space-separated arguments
• options (Object) may comprise one or more of the following options −
o cwd (String) Current working directory of the child process
o env (Object) Environment key-value pairs
o encoding (String) (Default: ‘utf8’)
o shell (String) Shell to execute the command with (Default: ‘/bin/sh’ on UNIX, ‘cmd.exe’ on Windows, The shell should understand the -c switch on UNIX or /s /c on Windows. On Windows, command line parsing should be compatible with cmd.exe.)
o timeout (Number) (Default: 0)
o maxBuffer (Number) (Default: 200*1024)
o killSignal (String) (Default: ‘SIGTERM’)
o uid (Number) Sets the user identity of the process.
o gid (Number) Sets the group identity of the process.
• callback The function gets three arguments error, stdout, and stderr which are called with the output when the process terminates.
The exec() method returns a buffer with a max size and waits for the process to end and tries to return all the buffered data at once.

Example

Let us create two js files named support.js and master.js −

File: support.js
console.log("Child Process " + process.argv[2] + " executed." );
File: master.js
const fs = require('fs');
const child_process = require('child_process');

for(var i=0; i<3; i++) {
var workerProcess = child_process.exec('node support.js '+i,function 
(error, stdout, stderr) {

if (error) {
console.log(error.stack);
console.log('Error code: '+error.code);
console.log('Signal received: '+error.signal);
}
console.log('stdout: ' + stdout);
console.log('stderr: ' + stderr);
});

workerProcess.on('exit', function (code) {
console.log('Child process exited with exit code '+code);
});
}

 

Now run the master.js to see the result −
$ node master.js
Verify the Output. Server has started.
Child process exited with exit code 0
stdout: Child Process 1 executed.

stderr:
Child process exited with exit code 0
stdout: Child Process 0 executed.

stderr:
Child process exited with exit code 0
stdout: Child Process 2 executed.

The spawn() Method

child_process.spawn method launches a new process with a given command. It has the following signature −
child_process.spawn(command[, args][, options])
Parameters
Here is the description of the parameters used −
• command (String) The command to run
• args (Array) List of string arguments
• options (Object) may comprise one or more of the following options −
o cwd (String) Current working directory of the child process.
o env (Object) Environment key-value pairs.
o stdio (Array) String Child’s stdio configuration.
o customFds (Array) Deprecated File descriptors for the child to use for stdio.
o detached (Boolean) The child will be a process group leader.
o uid (Number) Sets the user identity of the process.
o gid (Number) Sets the group identity of the process.
The spawn() method returns streams (stdout &stderr) and it should be used when the process returns a volume amount of data. spawn() starts receiving the response as soon as the process starts executing.

Example

Create two js files named support.js and master.js −

File: support.js
console.log("Child Process " + process.argv[2] + " executed." );
File: master.js
const fs = require('fs');
const child_process = require('child_process');

for(var i = 0; i<3; i++) {
var workerProcess = child_process.spawn('node', ['support.js', i]);

workerProcess.stdout.on('data', function (data) {
console.log('stdout: ' + data);
});

workerProcess.stderr.on('data', function (data) {
console.log('stderr: ' + data);
});

workerProcess.on('close', function (code) {
console.log('child process exited with code ' + code);
});
}

 

Now run the master.js to see the result −
$ node master.js
Verify the Output. Server has started
stdout: Child Process 0 executed.

child process exited with code 0
stdout: Child Process 1 executed.

stdout: Child Process 2 executed.

child process exited with code 0
child process exited with code 0

The fork() Method

child_process.fork method is a special case of spawn() to create Node processes. It has the following signature −
child_process.fork(modulePath[, args][, options])
Parameters
Here is the description of the parameters used −
• modulePath (String) The module to run in the child.
• args (Array) List of string arguments
• options (Object) may comprise one or more of the following options −
o cwd (String) Current working directory of the child process.
o env (Object) Environment key-value pairs.
o execPath (String) Executable used to create the child process.
o execArgv (Array) List of string arguments passed to the executable (Default: process.execArgv).
o silent (Boolean) If true, stdin, stdout, and stderr of the child will be piped to the parent, otherwise they will be inherited from the parent, see the “pipe” and “inherit” options for spawn()’s stdio for more details (default is false).
o uid (Number) Sets the user identity of the process.
o gid (Number) Sets the group identity of the process.
The fork method returns an object with a built-in communication channel in addition to having all the methods in a normal ChildProcess instance.

Example

Create two js files named support.js and master.js −

File: support.js
console.log("Child Process " + process.argv[2] + " executed." );
File: master.js
const fs = require('fs');
const child_process = require('child_process');

for(var i=0; i<3; i++) {
var worker_process = child_process.fork("support.js", [i]);

worker_process.on('close', function (code) {
console.log('child process exited with code ' + code);
});
}

Now run the master.js to see the result −
$ node master.js
Verify the Output. Server has started.
Child Process 0 executed.
Child Process 1 executed.
Child Process 2 executed.
child process exited with code 0
child process exited with code 0
child process exited with code 0

 

Node.js – Packaging

JXcore, which is an open source project, introduces a unique feature for packaging and encryption of source files and other assets into JX packages.
Consider you have a large project consisting of many files. JXcore can pack them all into a single file to simplify the distribution. This chapter provides a quick overview of the whole process starting from installing JXcore.

JXcore Installation

Installing JXcore is quite simple. Here we have provided step-by-step instructions on how to install JXcore on your system. Follow the steps given below −

Step 1

Download the JXcore package from https://github.com/jxcore/jxcore, as per your operating system and machine architecture. We downloaded a package for Cenots running on 64-bit machine.
$ wget https://s3.amazonaws.com/nodejx/jx_rh64.zip

Step 2

Unpack the downloaded file jx_rh64.zipand copy the jx binary into /usr/bin or may be in any other directory based on your system setup.
$ unzip jx_rh64.zip
$ cp jx_rh64/jx /usr/bin

Step 3

Set your PATH variable appropriately to run jx from anywhere you like.
$ export PATH=$PATH:/usr/bin

Step 4

You can verify your installation by issuing a simple command as shown below. You should find it working and printing its version number as follows −
$ jx –version
v0.10.32

Packaging the Code

Consider you have a project with the following directories where you kept all your files including Node.js, main file, index.js, and all the modules installed locally.
drwxr-xr-x 2 root root 4096 Nov 13 12:42 images
-rwxr-xr-x 1 root root 30457 Mar 6 12:19 index.htm
-rwxr-xr-x 1 root root 30452 Mar 1 12:54 index.js
drwxr-xr-x 23 root root 4096 Jan 15 03:48 node_modules
drwxr-xr-x 2 root root 4096 Mar 21 06:10 scripts
drwxr-xr-x 2 root root 4096 Feb 15 11:56 style
To package the above project, you simply need to go inside this directory and issue the following jx command. Assuming index.js is the entry file for your Node.js project −
$ jx package index.js index
Here you could have used any other package name instead of index. We have used index because we wanted to keep our main file name as index.jx. However, the above command will pack everything and will create the following two files −
• index.jxp This is an intermediate file which contains the complete project detail needed to compile the project.
• index.jx This is the binary file having the complete package that is ready to be shipped to your client or to your production environment.

Launching JX File

Consider your original Node.js project was running as follows −
$ node index.js command_line_arguments
After compiling your package using JXcore, it can be started as follows −
$ jx index.jx command_line_arguments

So, this brings us to the end of blog. This Tecklearn ‘How to scale application in NodeJS and concept of packaging’ blog helps you with commonly asked questions if you are looking out for a job in NodeJS Programming. If you wish to learn NodeJS and build a career in NodeJS Programming domain, then check out our interactive, Node.js Training, that comes with 24*7 support to guide you throughout your learning period.

Node.js Training

About the Course

Tecklearn’s Node.js certification training course familiarizes you with the fundamental concepts of Node.js and provides hands-on experience in building applications efficiently using JavaScript. It helps you to learn how to develop scalable web applications using Express Framework and deploy them using Nginx. You will learn how to build applications backed by MongoDB and gain in-depth knowledge of REST APIs, implement testing, build applications using microservices architecture and write a real-time chat application using Socket IO. Accelerate your career as a Node.js developer by enrolling into this Node.js training.

Why Should you take Node.js Training?

• As per Indeed.com data, the average salary of Node.js developer is about $115000 USD per annum.
• IBM, LinkedIn, Microsoft, GoDaddy, Groupon, Netflix, PayPal, SAP have adopted Node.js – ITJungle.com
• There are numerous job opportunities for Node.js developers worldwide. The job market and popularity of Node.js is constantly growing over the past few years.

What you will Learn in this Course?

Introduction to Node.js

• What is Node.js?
• Why Node.js?
• Installing NodeJS
• Node in-built packages (buffer, fs, http, os, path, util, url)
• Node.js Modules
• Import your own Package
• Node Package Manager (NPM)
• Local and Global Packages

File System Module and Express.js

• File System Module
• Operations associated with File System Module
• JSON Data
• Http Server and Client
• Sending and receiving events with Event Emitters
• Express Framework
• Run a Web Server using Express Framework
• Routes
• Deploy application using PM2 and Nginx

Work with shrink-wrap to lock the node module versions

• What is shrink-wrap
• Working with npmvet
• Working with outdated command
• Install NPM Shrinkwrap

Learn asynchronous programming

• Asynchronous basics
• Call-back functions
• Working with Promises
• Advance promises
• Using Request module to make api calls
• Asynchronous Commands

Integration with MongoDB and Email Servers

• Introduction to NoSQL Databases and MongoDB
• Installation of MongoDB on Windows
• Installation of Database GUI Viewer
• Inserting Documents
• Querying, Updating and Deleting Documents
• Connect MongoDB and Node.js Application
• Exploring SendGrid
• Sending emails through Node.js application using SendGrid

REST APIs and GraphQL

• REST API
• REST API in Express
• Postman
• MongoDB Driver API
• Express Router
• Mongoose API
• GraphQL
• GraphQL Playground

Building Node.js Applications using ES6

• ES6 variables
• Functions with ES6
• Import and Export withES6
• Async/Await
• Introduction to Babel
• Rest API with ES6
• Browsing HTTP Requests with Fetch
• Processing Query String
• Creating API using ES6
• Building Dashboard API
• Creating dashboard UI with EJS
• ES6 Aside: Default Function Parameters
• Data Validation and Sanitization

User Authentication and Application Security

• Authentication
• Types of Authentication
• Session Vs Tokens
• JSON Web Tokens
• Bcrypt
• Node-local storage

Understand Buffers, Streams, and Events

• Using buffers for binary data
• Flowing vs. non-flowing streams
• Streaming I/O from files and other sources
• Processing streams asynchronously
• File System and Security

Build chat application using Socket.io

• Getting Started
• Adding Socket.io To Your App
• Exploring the Front-end
• Sending Live Data Back & Forth
• Creating the Front-end UI
• Showing Messages In App
• Working with Time
• Timestamps
• Show Message Time In Chat App
• Chat application Project

Microservices Application

• Why Microservices?
• What is Microservices?
• Why Docker?
• What is Docker?
• Terminologies in Docker
• Child Processes
• Types of child process

 

0 responses on "How to scale application in NodeJS and concept of packaging"

Leave a Message

Your email address will not be published. Required fields are marked *