Tweet Miner App

Tweet Miner is a Simple Project to Scrape Tweets Dynamically from Specific Topics.

It comes with many features, such as authentication using JWT, request validation, unit and integration tests, continuous integration, docker support, API documentation, pagination, etc. For more details about the features, check the list below.

Features

• ES9: latest ECMAScript features
• NoSQL database: MongoDB object data modeling using Mongoose
• Authentication and authorization: using passport
• Validation: request data validation using Joi
• Logging: using winston and morgan
• Testing: unit and integration tests using Jest
• Error handling: centralized error handling mechanism
• API documentation: with swagger-jsdoc and swagger-ui-express
• Process management: advanced production process management using PM2
• Dependency management: with Yarn
• Environment variables: using dotenv and cross-env
• Security: set security HTTP headers using helmet
• Santizing: sanitize request data against xss and query injection
• CORS: Cross-Origin Resource-Sharing enabled using cors
• Compression: gzip compression with compression
• CI: continuous integration with Travis CI
• Docker support
• Code coverage: using coveralls
• Code quality: with Codacy
• Git hooks: with husky and lint-staged
• Linting: with ESLint and Prettier
• Editor config: consistent editor configuration using EditorConfig

Getting Started

Installation

Clone the repo:

git clone [email protected]:MRdevX/tweet-miner-api.git
cd tweet-miner-api

Install the dependencies:

yarn install

Set the environment variables:

cp .env.example .env

# open .env and modify the environment variables (if needed)

Commands

Running locally:

yarn dev

Running in production:

yarn start

Testing:

# run all tests
yarn test

# run all tests in watch mode
yarn test:watch

# run test coverage
yarn coverage

Docker:

# run docker container in development mode
yarn docker:dev

# run docker container in production mode
yarn docker:prod

# run all tests in a docker container
yarn docker:test

Linting:

# run ESLint
yarn lint

# fix ESLint errors
yarn lint:fix

# run prettier
yarn prettier

# fix prettier errors
yarn prettier:fix

Project Structure

src\
 |--config\         # Environment variables and configuration related things
 |--controllers\    # Route controllers (controller layer)
 |--docs\           # Swagger files
 |--middlewares\    # Custom express middlewares
 |--models\         # Mongoose models (data layer)
 |--routes\         # Routes
 |--services\       # Business logic (service layer)
 |--utils\          # Utility classes and functions
 |--validations\    # Request data validation schemas
 |--app.js          # Express app
 |--index.js        # App entry point

API Documentation

To view the list of available APIs and their specifications, run the server and go to http://localhost:3000/v1/docs in your browser or use the version deployed on Heroku https://tweet-miner-app.herokuapp.com/v1/docs/. This documentation page is automatically generated using the swagger definitions written as comments in the route files.

API Endpoints

List of available routes:

Auth routes:
POST /v1/auth/register - register
POST /v1/auth/login - login
POST /v1/auth/refresh-tokens - refresh auth tokens
POST /v1/auth/forgot-password - send reset password email
POST /v1/auth/reset-password - reset password

User routes:
POST /v1/users - create a user
GET /v1/users - get all users
GET /v1/users/:userId - get user
PATCH /v1/users/:userId - update user
DELETE /v1/users/:userId - delete user

Topic routes:
POST /v1/topics - create a topic
GET /v1/topics - get all topics
GET /v1/topics/:topicId - get topic
PATCH /v1/topics/:topicId - update topic
DELETE /v1/topics/:topicId - delete topic

Tweet routes:
POST /v1/tweets - create a tweet
GET /v1/tweets - get all tweets
GET /v1/tweets/:tweetId - get tweet
PATCH /v1/tweets/:tweetId - update tweet
DELETE /v1/tweets/:tweetId - delete tweet

Error Handling

The app has a centralized error handling mechanism.

Controllers should try to catch the errors and forward them to the error handling middleware (by calling next(error)). For convenience, you can also wrap the controller inside the catchAsync utility wrapper, which forwards the error.

Validation

Request data is validated using Joi. Check the documentation for more details on how to write Joi validation schemas.

The validation schemas are defined in the src/validations directory and are used in the routes by providing them as parameters to the validate middleware.

License

MIT