The Javascript Onion¶

Is this Javascript?

function(a) {
    console.log(a);
}("hi")
var o = {}
var o["2"] = 2
var deepCopy = JSON.parse(JSON.stringify(o))

What about this?

const list = [1,2,3]
a = (l) => {
    const [, num2, ] = l
    console.log(num2)
}
const o = { "asdf": 2 }
const deepCopy = { ...o, b: 2 }

Both do seem to work when added as inline javascript to our local HTML file in the browser:

<html>
    <body>
        <script type="text/javascript">
            const list = [1,2,3]
            a = (l) => {
                const [, num2, ] = l
                console.log(num2)
            }
            const o = { "asdf": 2 }
            const deepCopy = { ...o, b: 2 }
            alert(JSON.stringify(deepCopy))
        </script>
    </body>
</html>

There’s no way that this could be valid Javascript:

[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]][([][(![]+[])[+[]]+(![]+[])[!+[]
+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]
]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[]
)[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]
+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!
+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]]((!![]+[])[+!+[]]+(!![]+[])[!+[]+!+[]+!+[]]+(!![
]+[])[+[]]+([][[]]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+!+[]]+(+[![]]+[][(![]+[])[+[]]+(![]+[])[!+
[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+!+[]]]+(!![]+[])[!+[]+!+[]+!+[]]+(+(!+[]+!+[]+!+[]
+[+!+[]]))[(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]
)[+!+[]+[+[]]]+([]+[])[([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[
]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]
]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(
![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+
(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+
!+[]]][([][[]]+[])[+!+[]]+(![]+[])[+!+[]]+((+[])[([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[
]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!
![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+
[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+
!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]
)[+!+[]+[+[]]]+(!![]+[])[+!+[]]]+[])[+!+[]+[+!+[]]]+(!![]+[])[!+[]+!+[]+!+[]]]](!+[]+!+[]+!+[]+[!+[]
+!+[]])+(![]+[])[+!+[]]+(![]+[])[!+[]+!+[]])()([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+
(!![]+[])[+[]]][([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+
!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[
]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])
[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[
][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]](
(!![]+[])[+!+[]]+(!![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+([][[]]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+
[])[+!+[]]+([]+[])[(![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[]
)[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(!![]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[
])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!
+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(![]+[])[!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(!
[]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]]()[+!+[]+[!+[]+!+[]]]+((![]+[])[+!+[]]+
(![]+[])[!+[]+!+[]]+(!![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+!+[]]+(!![]+[])[+[]]+[+!+[]]+[!+[]+!+[]+!+[
]+!+[]+!+[]+!+[]]+[!+[]+!+[]+!+[]+!+[]]+(!![]+[])[+[]]+[!+[]+!+[]+!+[]+!+[]+!+[]]+[+[]]+(!![]+[])[+[
]]+[!+[]+!+[]+!+[]+!+[]]+[!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]+(!![]+[])[+[]]+[+!+[]]+[!+[]+!+[]+!+[]
+!+[]+!+[]+!+[]]+[!+[]+!+[]+!+[]+!+[]]+(!![]+[])[+[]]+[+!+[]]+[!+[]+!+[]+!+[]+!+[]+!+[]]+[+[]]+([![]
]+[][[]])[+!+[]+[+[]]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+[!+[]+!+[]+!+[]+!+[]]+[+[]]+([![]]+[]
[[]])[+!+[]+[+[]]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+[!+[]+!+[]+!+[]+!+[]]+[+[]]+(!![]+[])[+[]
]+[+!+[]]+[!+[]+!+[]+!+[]+!+[]+!+[]]+[!+[]+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+[!+[]+!+[]+
!+[]+!+[]]+[!+[]+!+[]+!+[]+!+[]+!+[]+!+[]+!+[]]+(!![]+[])[+[]]+[!+[]+!+[]+!+[]+!+[]+!+[]]+[+!+[]])[(
![]+[])[!+[]+!+[]+!+[]]+(+(!+[]+!+[]+[+!+[]]+[+!+[]]))[(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[]
)[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([]+[])[([][(![]+[])[+[]]+(![]+[])[!+[]+!
+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+
(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[
+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[
])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[
]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]][([][[]]+[])[+!+[]]+(![]+[])[+!+[]]+((+[])[([][(![
]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])
[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[
!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[
]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[]
)[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]]+[])[+!+[]+[+!+[]]]+(!![
]+[])[!+[]+!+[]+!+[]]]](!+[]+!+[]+!+[]+[+!+[]])[+!+[]]+(![]+[])[!+[]+!+[]]+([![]]+[][[]])[+!+[]+[+[]
]]+(!![]+[])[+[]]]((!![]+[])[+[]])[([][(!![]+[])[!+[]+!+[]+!+[]]+([][[]]+[])[+!+[]]+(!![]+[])[+[]]+(
!![]+[])[+!+[]]+([![]]+[][[]])[+!+[]+[+[]]]+(!![]+[])[!+[]+!+[]+!+[]]+(![]+[])[!+[]+!+[]+!+[]]]()+[]
)[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+
[+[]]]+([![]]+[][[]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]](([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[]
)[+!+[]]+(!![]+[])[+[]]][([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!
+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[
]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([]
[(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]
]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])
[+!+[]]]((!![]+[])[+!+[]]+(!![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+([][[]]+[])[+[]]+(!![]+[])[+!+[]]
+([][[]]+[])[+!+[]]+(![]+[+[]])[([![]]+[][[]])[+!+[]+[+[]]]+(!![]+[])[+[]]+(![]+[])[+!+[]]+(![]+[])[
!+[]+!+[]]+([![]]+[][[]])[+!+[]+[+[]]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[
])[+[]]]+[])[!+[]+!+[]+!+[]]+(![]+[])[!+[]+!+[]+!+[]]]()[+!+[]+[+[]]]+![]+(![]+[+[]])[([![]]+[][[]])
[+!+[]+[+[]]]+(!![]+[])[+[]]+(![]+[])[+!+[]]+(![]+[])[!+[]+!+[]]+([![]]+[][[]])[+!+[]+[+[]]]+([][(![
]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(![]+[])[!+[]+!+[
]+!+[]]]()[+!+[]+[+[]]])()[([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])
[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[
+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+(
[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+
[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[
])[+!+[]]]((![]+[+[]])[([![]]+[][[]])[+!+[]+[+[]]]+(!![]+[])[+[]]+(![]+[])[+!+[]]+(![]+[])[!+[]+!+[]
]+([![]]+[][[]])[+!+[]+[+[]]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+
[])[!+[]+!+[]+!+[]]+(![]+[])[!+[]+!+[]+!+[]]]()[+!+[]+[+[]]])+[])[+!+[]])+([]+[])[(![]+[])[+[]]+(!![
]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+
[]]+(!![]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+
[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!
[]+[])[!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+
[+[]]]+(!![]+[])[+!+[]]]()[+!+[]+[!+[]+!+[]]])())

You may have heard of both the MEAN/MERN stacks, and both use Express. Let’s try adding Express - it is a Javascript library, after all! Try adding the Express.js starter code for a basic hello world server as inline javascript:

const express = require('express')
const app = express()
const port = 3000

app.get('/', (req, res) => {
res.send('Hello World!')
})

app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`)
})

When inspecting your browser debugger, you should immediately encounter a bug of the form Uncaught ReferenceError: require is not defined. Yikes. This Javscript isn’t designed for the browser.

Finally, let’s try making our tiny HTML file a React app. When navigating to the the Create React App tutorial, we are immediately recommended to “use an integrated toolchain for the best user and developer experience.” Thankfully, there’s a link to add React as a “plain” <script> tag, so let’s try that. Even in that section, there is a link to a complete HTML file shown here for your convenience:

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8" />
    <title>Hello World</title>
    <script src="https://unpkg.com/react@17/umd/react.development.js"></script>
    <script src="https://unpkg.com/react-dom@17/umd/react-dom.development.js"></script>

    <!-- Don't use this in production: -->
    <script src="https://unpkg.com/@babel/standalone/babel.min.js"></script>
</head>
<body>
    <div id="root"></div>
    <script type="text/babel">

    ReactDOM.render(
        <h1>Hello, world!</h1>,
        document.getElementById('root')
    );

    </script>
    <!--
    Note: this page is a great way to try React but it's not suitable for production.
    It slowly compiles JSX with Babel in the browser and uses a large development build of React.

    Read this section for a production-ready setup with JSX:
    https://reactjs.org/docs/add-react-to-a-website.html#add-jsx-to-a-project

    In a larger project, you can use an integrated toolchain that includes JSX instead:
    https://reactjs.org/docs/create-a-new-react-app.html

    You can also use React without JSX, in which case you can remove Babel:
    https://reactjs.org/docs/react-without-jsx.html
    -->
</body>
</html>

Even after we parse through all the warnings that this isn’t suited for production, we see that it “slowly compiles JSX with Babel.” There also looks to be HTML within the javascript section of the app. The <script> tag also doesn’t pretend that it’s javascript anymore: it is allegedly “text/babel.”

What on earth does this mean? Welcome to the Javascript Onion.

Vanilla Javascript¶

As the web grew in popularity, websites wanted to be able to run code that would execute in the browser. Netscape, an early web browser, designed a programming language called Javascript that could be included in webpages and then executed in the browser. This language caught on, and other browsers started to support Javascript. Now, Javascript is the single most popular programming language according to the latest Stack Overflow Developer Survey. As a result, Ecma International maintains a set of public standards around Javascript, which the browsers use to continuously update their own JS interepreters. These standards get frequuently updated and often add new features that the browsers eventually support. This ECMAScript-compliant JS should be able to be run via inline <script> tags out of the box in the latest browsers.

A far more comprehensive introduction to Javascript as a whole can be found on the MDN Web Docs.

Node.js¶

Javascript (and web technologies more broadly) have become so popular that developers wanted to write Javascript code across all aspects their software stack. Instead of just having Javascript run just in the browser, why not be able to run Javascript as a process? Node.js allows you to write scripts and servers in Javascript.

Once you install node, you can node to open up a Node interpreter. Here, you can run Javascript that doesn’t involve browser interactions. For example, alert('hi') no longer makes sense here. console.log('hi'), however, does still work. Since Express is a framework to help run a web server, and web servers run as processes, Express is meant to be run in Node. Let’s make an empty project directory. From there, make a src directory. We can thus try putting the Express starter code given above in a file src/server.js and then running node src/server.js in the terminal:

$ node src/server.js
internal/modules/cjs/loader.js:883
    throw err;
    ^

Error: Cannot find module 'express'

Interestingly, we no longer are experiencing the Uncaught ReferenceError, so we are getting closer. In fact, require is not supported in the ECMAScript specification, but is supported in Node (see the CommonJS Modules section). Our remaining issue is that this third-party Express framework is a “module” that Node doesn’t understand. Thankfully, Node.js comes with an excellent package manager that is mysteriously named “npm”. You can check out their homepage to see some possibilities regarding the meaning of the acronym. Regardless, to manage third-party libraries, we need to make this one little script an npm package, even if we don’t plan on publishing this package publicly.

$ npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.

See `npm help init` for definitive documentation on these fields
and exactly what they do.

Use `npm install <pkg>` afterwards to install a package and
save it as a dependency in the package.json file.
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.

See `npm help init` for definitive documentation on these fields
and exactly what they do.

Use `npm install <pkg>` afterwards to install a package and
save it as a dependency in the package.json file.

Press ^C at any time to quit.
package name: (guides)
version: (1.0.0)
description:
entry point: (server.js) src/server.js
test command:
git repository: (https://github.com/codethechange/guides)
keywords:
author:
license: (ISC)
About to write to ./package.json:

{
"name": "guides",
"version": "1.0.0",
"description": "=================================== Guides for Stanford Code the Change ===================================",
"main": "src/server.js",
"scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node src/server.js"
},
"repository": {
    "type": "git",
    "url": "git+https://github.com/codethechange/guides.git"
},
"author": "",
"license": "ISC",
"bugs": {
    "url": "https://github.com/codethechange/guides/issues"
},
"homepage": "https://github.com/codethechange/guides#readme"
}


Is this OK? (yes) yes

And voila! Inspecting our current directory reveals that a somewhat auto-generated package.json file was creaed for our convenience. The scripts field of this JSON file even automatically created some default commands for us:

$ npm run start

> guides@1.0.0 start
> node src/server.js

internal/modules/cjs/loader.js:883
    throw err;
    ^

Error: Cannot find module 'express'

As the handy npm output recommended for us, let’s install packages via npm install express.

Note

It is important to make sure that you verify the package name corresponds to the actual software you are interested in using. For example, the express package can be viewable from the npm package registry. Be sure to check out our guide on securing open source software for more information.

$ npm install express

> guides@1.0.0 start
> node src/server.js

internal/modules/cjs/loader.js:883
    throw err;
    ^

Error: Cannot find module 'express'

We should now be able to run this server.js file and be able to visit http://localhost:3000:

$ npm run start

> guides@1.0.0 start
> node src/server.js
Example app listening at http://localhost:3000

CommonJS Modules¶

Although our express server now works thanks to Node and NPM, how do does require work under the hood? cd into the newly created node_modules folder, which installs the npm packages specific to our current project as defined in package.json. We can then inspect the express folder which corresponds to the express package.

If you inspect node_modules/express/index.js, you will see that there is a special module.exports assignment to another require call:

/*!
* express
* Copyright(c) 2009-2013 TJ Holowaychuk
* Copyright(c) 2013 Roman Shtylman
* Copyright(c) 2014-2015 Douglas Christopher Wilson
* MIT Licensed
*/

'use strict';

module.exports = require('./lib/express');

This require() has a relative path included, which tells Node to look locally for the module file as opposed to installing a third-party NPM dependency. If we inspect node_modules/express/lib/express.js, we will see a much more intricate file with many module.exports assignments (but assigned to a exports convenience variable to not have to prefix with module each time).

Back in node_modules/express/index.js, let’s add an additional export field:

/*!
* express
* Copyright(c) 2009-2013 TJ Holowaychuk
* Copyright(c) 2013 Roman Shtylman
* Copyright(c) 2014-2015 Douglas Christopher Wilson
* MIT Licensed
*/

'use strict';

module.exports = require('./lib/express');
module.exports.hi = "hello there";

Did we just edit the Express package? Let’s see. Go back to src/server.js, add a simple console.log(express.hi) line, and then run node server.js. You should see “hello there” in the output in addition to your server!

In the same way that the express package used relative imports, you do not have to only import and export module information from within node_modules. To import your own source files in the project, merely add relative paths in the require() argument.

Warning

Editing an imported NPM package locally should not be done in practice – this exercise was for merely instructive purposes. A successive npm install could overwrite all your work.

Warning

NPM does support installing local packages as NPM packages. Some organizations, have faced security issues where someone registers their project’s local package name as a public package on NPM, causing a name collision. Here is a Medium post introducing this supply chain attack vector. GitHub published a blog post regarding how to install local packages via NPM safely.

ES6, Babel, and Transpilation¶

As we go back to the browser, you may have noticed an issue with continuously updating standards: browsers have to send updates to all their users so that all client-side browsers can execute the newest features of Javascript. Inevitably, there will be users with outdated browsers that are incapable of running modern versions of Javascript.

One especially big version change was ECMAScript 2015, or ES6. The let keyword and arrow function notation from the second code snippet are part of ES6. The spread operator in the deepCopy line is from ES9.

Thankfully, these language additions do not change the expressivity of the language and can be viewed as syntactic sugar. Babel is a JS framework that helps convert your modern Javascript code into Javascript that is compliant with the old ECMAScript standards so that it can be run out of the box. We will will see this in action on our modern server.js file that uses arrow functions.

Note

In fact, Babel can perform arbitrary transpilation, which is why React uses Babel to transform their JSX into pure, native javascript. We will want to explicitly tell Babel to transpile this modern javascript a pre-ECMA2015 world.

Next, we will install babel, babel’s cli, and the default presets, which include replacing arrow functions. Then, we will use Babel to transpile our server.js file:

$ npm install --save-dev @babel/core @babel/cli @babel/preset-env
$ ./node_modules/.bin/babel src --out-dir lib --presets=@babel/env
Successfully compiled 1 file with Babel (431ms).
$ cat lib/server.js
"use strict";

var express = require('express');

var app = express();
var port = 3000;
app.get('/', function (req, res) {
res.send('Hello World!');
});
app.listen(port, function () {
console.log("Example app listening at http://localhost:".concat(port));
});

Et Voila! Our arrow functions have been replaced with traditional functions. Be sure to continue reading here for more information on how to use Babel.

ES6 + CommonJS: Import & Export¶

When ES6 came about, Ecma International decided to put module imports and exports in to the JS standard. Crazily enough, the browsers support ES6 modules as well, although I have never tried using ES6 modules directly via inline JS. Node, by extension, also supports managing packages this way as well. One particularly cool aspect is that Node makes ES6 module syntax interoperable with other modules that use CommonJS syntax.

Try editing our server.js file to use ES6 import syntax:

import express from 'express'

console.log(express.hi)
var app = express();
var port = 3000;
app.get('/', function (req, res) {
    res.send('Hello World!');
});
app.listen(port, function () {
    console.log("Example app listening at http://localhost:" + port);
});

If we try running this, we get the following error message:

$ node server.js
(node:82672) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
./src/server.js:1
import express from 'express'
^^^^^^

Interestingly, Node doesn’t support interchanging CommonJS and ES6 syntax within the same module. Thankfully, this is an easy fix. Add the "type": "module" (“type” defaults to “commonjs” when the field isn’t specified), which tells Node.js to interpret this module in ES6. Even though express was written with CommonJS modules, we can still import Express with ES6:

$ node server.js
hello there
Example app listening at http://localhost:3000

Note

We ran ES6 code via a normal node command. What gives? The reality is that your new Node installation certainly supports ES6 out of the box without transpilation in “experimental mode”, and most browsers already support ES6 anyways. As time goes on, there will be less and less of a need to transpile to pre-2015 ECMAScript-compliant JS. Still, the underlying principles behind transpilation will likely persist – there will always be a gap between the newest versions of Javascript and the browser versions of billions of users all trying to run your javascrip on their local machines. You don’t need to transpile, however, if your JS is being solely on a Node.js process whose version you control.

Indeed, our tweaked Express package in node_modules is still unchanged (“hello there” is still being printed) and thus must still be written via CommonJS module syntax.

ES6 import/export statements also leverage the object destructuring syntax added in the standard. Here is another example. Add other_file.js to your src directory:

const hi  = 'hi from other file'

const say_bye = () => { console.log("good bye from other file") }

export {hi, say_bye}

Then, update our increasingly messy server.js to use this functionality from other_file:

import express from 'express'
import { hi, say_bye } from './other_file.js'

console.log(hi)
say_bye()
console.log(express.hi)
var app = express();
var port = 3000;
app.get('/', function (req, res) {
    res.send('Hello World!');
});
app.listen(port, function () {
    console.log("Example app listening at http://localhost:" + port);
});

Node.js, despite having experimental support with importing modules via ES6 syntax, doesn’t work too well with exports out of the box. Thankfully, Babel will convert our ES6 syntax into CommonJS syntax via transpilation. Remove the "type": "module" line from our package.json to tell node that we’re reverting into CommonJS syntax. Let’s now compile our ES6 JS into CommonJS syntax via the same Babel command we used before:

$ ./node_modules/.bin/babel src --out-dir lib --presets=@babel/env
$ node lib/server.js
hi from other file
good bye from other file
hello there
Example app listening at http://localhost:3000

Typescript¶

Imagine you have the following function to check if someone will be of legal drinking age in the US exactly 5 years from now given their current age:

function willBeOfAge(age) {
    if (age + 5 >= 21) {
        return true
    } else {
        return false
    }
}

What will happen if age is 10? What about “10”?

$ node
Welcome to Node.js v14.13.1.
Type ".help" for more information.
> 10 + 5 >= 21
false
> "10" + 5 >= 21
true
> "10" + 5
'105'

Javascript can support weird operations between different types, and this can make it difficult to catch bugs where the wrong types are being used. For example, imagine that, in the age example, the age value was derived from some string regex. A developer could have forgotten to cast the string into a number (there are no int types in JS). In fact, many bugs can be viewed as type bugs – incorrect function signatures, handling null values, and performing comparisons with the wrong implicit types all could be prevened with TypeScript.

Javascript does not support static types out of the box, so TypeScript was introduced to add static types to Javascript. A cool feature of TypeScript is that one can have incremental adoption of TypeScript – one can use different settings to enforce static types on only some Javascript files, or one could disable checks or null or undefined.

We’ll try to use TypeScript to prevent a developer from accidentally passing in a string to this willBeOfAge function. For our convenience, we’ll make a new file age_checker.ts and include the following function:

const willBeOfAge = (age) => {
    if (age + 5 >= 21) {
        return true
    }
    return false
}
console.log(willBeOfAge(10))
console.log(willBeOfAge("10"))

Here, we’re also leveraging arrow functions for a more modern JS feel. Run the file via node age_checker.ts to see how passing in the number as a string will not throw an exception but possibly give the wrong answer.

We can selectively add types to whatever variables we would like: we could add a type constraint to the argument age.

const willBeOfAge = (age : number) : boolean =>  {
if (age + 5 >= 21) {
    return true
}
return false
}
console.log(willBeOfAge(10))
console.log(willBeOfAge("10"))

Let’s now try to compile this with TypeScript and check for type correctness. First, install TypeScript globally on your machine:

$ npm install -g typescript

Next, try compiling your new .ts file:

$ tsc age_checker.ts
age_checker.ts:8:25 - error TS2345: Argument of type 'string' is not assignable to parameter of type 'number'.
8 console.log(willBeOfAge("10"))

Found 1 error.

Here, TypeScript catches incorrect argument types before we deploy our code. To be nice, Typescript will still compile the .ts into normal Javascript. If you inspect the newly created age_checker.js, you may notice that TypeScript performs a dual function – it converted our fancy arrow functions into older ES3 function definitions! Thus, TypeScript can already transpile most of what Babel also supports in addition to their static type checking.

To learn more about TypeScript, check out their documentation.