RESTFUL API: CrapR

1. Project Overview

This API is a RESTful web service built using Node.js, Express, Mongoose, and MongoDB. It is designed to serve as a proof of concept (PoC) for a fictitious marketplace application where users can exchange unwanted items, affectionately termed "crap," between users. The primary goal of the API is to demonstrate fundamental CRUD functionality, user authentication with Google OAuth 2.0, securing API routes using JWT, integrating Google Cloud Storage and the management of relationships between resources.

2. Key Features

Secure Authentication: Implements Google OAuth 2.0 and JWT for robust user authentication.
CRUD Operations on Crap Resource: Authenticated users (referred to as sellers) can create, read, update, and delete posts regarding items they want to discard, referred to as "crap." Other users (referred to as buyers) can view these posts and engage in an exchange process until the item is marked as taken.
Geo-Location Search: Enables users to find items based on proximity and search queries.
Data Protection: Ensures sensitive information is safeguarded and only accessible to authorized users.
Input Sanitization: User inputs are sanitized to prevent XSS attacks.
Responsive API Design: Provides clear and consistent JSON responses with comprehensive error handling.

Backend Framework: Node.js with Express.js
Database: MongoDB with Mongoose ODM
Authentication: Passport.js with Google OAuth 2.0 and JWT
API Design: RESTful principles for scalability and simplicity
Testing: Mocha and Jest
Deployment: Azure Cloud platform with Github Actions for CI/CD pipelines

Architecture and Design

MVC Structure: Adopted the Model-View-Controller pattern for organized and maintainable code.
Middleware Utilization: Employed middleware for authentication, input sanitization, and error handling.
Secure Data Models: Defined Mongoose schemas with validation and sanitation to protect against injection attacks.
Geospatial Queries: Leveraged MongoDB's geospatial indexing for efficient location-based searches.
State Management: Implemented a finite state machine to handle the various statuses of items throughout the buyer-seller interaction.

5. Challenges and Solutions

Google OAuth Integration

Challenge: Integrating Google OAuth for user authentication while maintaining security and user experience.
Solution: Utilized Passport.js strategies to seamlessly integrate OAuth 2.0, ensuring secure authentication flows and token management.

Google Cloud Storage Integration

Challenge: Integrating Google Cloud Storage for storing crap images.
Solution: Utilized Google Cloud Storage Docs to set up a Google Storage Bucket and connect it to the Api.

Technical Implementation

yarn add multer @google-cloud/storage

Used Multer to allow users to upload many images of their crap item

// src/middlewares/pickImages.js
 
const multer = require("multer");
const { ImageUploadError } = require("../utils/errors");
 
const storage = multer.memoryStorage();
 
const limits = {
  fileSize: 5 * 1024 * 1024,
};
 
const fileFilter = (req, file, cb) => {
  if (!file.mimetype.startsWith("image/")) {
    return cb(new ImageUploadError("File is not an image"), false);
  }
  cb(null, true);
};
 
const attachImages = multer({
  storage,
  limits,
  fileFilter,
}).array("images", 10);
 
module.exports = attachImages;

Then created images.service.js that interacts with Google Cloud Storage to store images.

const { Storage } = require("@google-cloud/storage");
const { ImageUploadError } = require("../utils/errors");
 
const storage = new Storage({
  keyFilename: process.env.GOOGLE_STORAGE_SECRET_PATH,
});
 
const bucket = storage.bucket(process.env.GOOGLE_STORAGE_BUCKET_NAME);
 
exports.uploadMany = async (files) => {
  try {
    if (!files || files.length === 0)
      throw new ImageUploadError("No images provided for upload");
 
    const uploadPromises = files.map((file) => {
      const filename = `${Date.now()}-${file.originalname}`;
      const blob = bucket.file(filename);
      const blobStream = blob.createWriteStream({
        resumable: false,
        contentType: file.mimetype,
      });
 
      return new Promise((resolve, reject) => {
        blobStream.on("error", (err) => {
          reject(
            new ImageUploadError(
              `Failed to upload ${file.originalname}: ${err.message}`
            )
          );
        });
 
        blobStream.on("finish", () => {
          const publicUrl = `https://storage.googleapis.com/${bucket.name}/${blob.name}`;
          resolve(publicUrl);
        });
 
        blobStream.end(file.buffer);
      });
    });
 
    const imageUrls = await Promise.all(uploadPromises);
    return imageUrls;
  } catch (error) {
    if (error instanceof ImageUploadError) {
      throw error;
    } else {
      throw new ImageUploadError(
        `Unexpected error during image upload: ${error.message}`
      );
    }
  }
};

Complex Workflow Handling

Challenge: Managing the intricate buyer-seller lifecycle with multiple status transitions and role-based actions.
Solution: Developed a robust state management system with clear rules and validations to handle transitions, ensuring data integrity and a smooth user experience.

Geo-Location Functionality

Challenge: Implementing efficient and accurate location-based search capabilities.
Solution: Employed MongoDB's geospatial features to store location data and execute proximity searches, optimizing query performance with proper indexing.

Code Sample

inside crap service module there crap.service.js

exports.listAllCrap = async ({ query, lat, long, distance, show_taken }) => {
  const filters = {};
 
  if (query)
    filters.$or = [
      { title: { $regex: query, $options: "i" } },
      { description: { $regex: query, $options: "i" } },
    ];
 
  if (show_taken === "true") {
    filters.status = { $ne: "FLUSHED" };
  } else {
    filters.status = "AVAILABLE";
  }
 
  if (lat && long && distance) {
    filters.location = {
      $near: {
        $geometry: {
          type: "Point",
          coordinates: [parseFloat(long), parseFloat(lat)],
        },
        $maxDistance: parseFloat(distance),
      },
    };
  }
 
  const crapList = await CrapModel.find(filters)
    .select("-location -buyer -suggestion -__v")
    .populate("owner", "name");
 
  return crapList;
};

6. Learning Outcomes

Advanced Authentication Techniques: Gained proficiency in implementing OAuth 2.0 and JWT for secure user authentication.
Middleware Expertise: Enhanced understanding of Express middleware to create reusable and efficient code components.
Database Optimization: Learned to optimize MongoDB queries and leverage indexing for performance improvements.
Security Best Practices: Strengthened knowledge of protecting applications against common web vulnerabilities like XSS and injection attacks.
State Machine Implementation: Acquired skills in designing and implementing state machines to manage complex workflows.

7. Visual Showcase

Architecture Diagram

diagram

Google Cloud Storage stores images while MongoDB stores metadata.

File Structure Following MVC Pattern

File structure

Since this is a RESTful API, there’s no traditional view (like frontend UI). Instead, the "view" is the JSON responses returned by the API.

API Workflow

Depicts the buyer-seller interaction flow and item status transitions.

Sample API Responses

{
  "data": {
    "_id": "641f45cc4de5a0f56bbc702e",
    "title": "Vintage Bicycle",
    "description": "A classic bike in good condition.",
    "images": [
      "https://storage.googleapis.com/craptracker-images/vintage_bike.png"
    ],
    "status": "AVAILABLE",
    "owner": {
      "_id": "66136751ed1aed6ce50e1d92",
      "name": "Alex"
    },
    "createdAt": "2024-04-09T20:39:56.459Z",
    "updatedAt": "2024-04-09T20:39:56.459Z"
  }
}

Example of a JSON response for retrieving an item.

8. Code and Live Demo

GitHub Repository: Click Here
Live API Demo: Click Here
- Note: The Server might be sleeping due to inactivity policy as its being hosted on a free tier plan.
- Note: Authentication is required to access API endpoints.

9. Future Enhancements

User Notifications: Implement real-time notifications for status updates using WebSockets or Socket.io or Firebase Cloud Messesing
Frontend Application: Develop a client-side application using React, NextJS or Flutter to provide an intuitive user interface.
Internationalization: Add multi-language support to cater to a diverse user base.
Enhanced Security: Incorporate two-factor authentication and monitor for suspicious activities.

Thank you for taking the time to learn about the project.