When developing APIs for services in a company with a large user base, scalability becomes a critical consideration. Scaling an API in production for such a company requires keeping certain fundamentals in mind. Today, I’ll discuss a few key points that you should also keep in mind when approaching this challenge. Here’s a breakdown of some key techniques and strategies you can use:

API Optimizations

  • Reduce Payload Size: Send only the necessary data in responses and avoid including unnecessary properties, as every byte contributes to network overhead, even if it’s just a single byte.

  • Response Compression: Enable compression techniques like Gzip or Brotli to reduce payload size and accelerate data transfer.

  • Pagination and Filtering: For large datasets, implement pagination and filtering mechanisms to limit the amount of data sent in a single request.

  • Stateless Applications: Design application servers to be stateless, processing only request-specific information without storing or persisting data locally. Persist state-related information, such as user authentication or preferences, externally using databases, in-memory caches (e.g., Redis).This approach ensures that application servers can be easily scaled up or down during deployment to handle varying request loads, improving flexibility, fault tolerance, and scalability.

  • Concurrency: Leverage concurrency to optimize performance when handling multiple tasks simultaneously. For instance, if an API request requires data from two or more services, you can make those service calls concurrently instead of sequentially. This reduces the overall response time by executing tasks in parallel, making better use of system resources. Consider using gRPC for inter-service communication, as it provides efficient, low-latency, and strongly-typed communication between services in a microservices architecture.

Database Optimizations

  • Indexing: Ensure that all frequently queried columns are properly indexed. This helps speed up SELECT queries by reducing the amount of data scanned. Avoid unnecessary indexing so that your write performance does not become bottleneck. Always explain the high RPS queries before putting in production server.

  • Query Optimization: Avoid complex JOINs and subqueries that slow down read and write operations. Also select the columns that are needed, avoid using * even if you have fewer or single column. Use EXPLAIN to analyze query performance and look for slow points.

  • Use Proper Data Types: Choose appropriate data types and sizes for database columns to minimize storage requirements and reduce overhead. For example, use TINYINT instead of INT for small numeric ranges, and VARCHAR with a defined maximum length instead of a generic TEXT column for variable-length strings. Avoid allocating unnecessary space by matching the data type to the actual range of values needed, which helps improve query performance and reduces memory and disk usage.

  • Avoid N+1 Queries: This issue occurs when a query fetches a list of items (1 query), and for each item, another query is executed (N queries). For example, fetching a list of users and then querying their associated orders individually leads to N+1 queries. To prevent this, use techniques like eager loading (e.g., JOIN statements or SELECT … IN queries) to retrieve related data in a single query. Avoiding N+1 queries reduces redundant database calls and significantly improves API response times.

  • Connection Pooling: Manage and optimize database connections with a connection pool to handle high concurrency, avoiding excessive connection overhead. Close connections that are no longer in use, and tune timeout settings to manage idle connections efficiently.

  • Working with Active Data: Keep only active and frequently accessed data in your main database tables to maintain optimal query performance. Archive unnecessary or historical data to separate tables or storage systems to reduce table size and improve query efficiency. This practice helps minimize index and table scan times, ensuring faster reads and writes.

  • Precompute Data: Precompute data for complex or frequently used calculations and store the results in the database. This reduces the need for on-the-fly computations during queries, improving response times and lowering database load. Use techniques like materialized views, denormalized tables, or scheduled background jobs to prepare precomputed data, especially for reports, summaries, or analytics.

  • Primary-Replica Setup: Use a primary-replica (master-slave) setup where the primary database handles writes, and replicas handle read traffic. Load balancers or application logic can direct read requests to replicas and write requests to the primary.

  • Read Replicas: Distribute read requests across multiple read replicas to scale horizontally and reduce read latency.

  • Partitioning: Partition tables based on certain fields (e.g., date, location) to split large tables into smaller, more manageable parts. This speeds up both reads and writes.

Caching Layers

  • Application-Level Caching: Implement caching at the application level for calculated or expensive operations. For instance, cache results of computationally intensive tasks or external API responses in memory or a caching system. Use Redis or Memcached to cache frequent reads. This offloads load from the database for common queries and reduces latency. Use Write-Through and Write-Behind caching techniques to keep cache and database in sync. Write-through writes data to both cache and database synchronously, while write-behind writes data to the cache immediately but updates the database asynchronously.

  • HTTP Caching: Leverage HTTP caching headers like ETag(Entity Tag), Last-Modified, and Cache-Control to minimize redundant data transmission. For example, when a client requests a resource, the server returns an ETag as a unique identifier (e.g., a hash of the file or response’s contents) for the resource’s current state. On subsequent requests, the client includes the ETag in the If-None-Match header. If the resource hasn’t changed, the server responds with a 304 Not Modified status, skipping the resource data transfer entirely. This approach is particularly effective for reducing bandwidth usage and improving response times for APIs, especially when dealing with large payloads or frequently accessed resources.

Optimizing Writes

  • Batching and Bulk Operations: Instead of writing one row at a time, batch inserts, updates, or deletes into bulk operations. Many RDBMS systems optimize for bulk operations, which reduces transaction overhead.

  • Asynchronous Writes: Offload non-critical writes, such as logging, analytics, or bulk update requests, to message queues like RabbitMQ or Kafka. Process these writes asynchronously using background job workers, ensuring they are written at a pace that aligns with the database’s capacity.

  • Debouncing and Throttling: For high-frequency writes (e.g., metrics or real-time events), use debouncing or throttling to limit writes per second or group them before committing.

Horizontal Scaling

  • Sharding: Divide the data into smaller, more manageable databases (shards) based on some criteria (e.g., user ID, geo location). Sharding reduces the size of each database, helping maintain performance at scale.

  • Microservices with Independent Databases: If you’re using a microservices architecture, each service can have its database. This ensures that high read/write loads for one service do not affect others. Distribute incoming API requests across multiple servers using a load balancer to avoid overloading a single server and ensure high availability. In modern systems, tools like Kubernetes and container orchestration platforms make managing such architectures easier. They enable seamless scaling, deployment, and high availability by distributing API requests across multiple service instances using load balancers.

API Rate Limiting and Throttling

  • API Gateway: Use an API gateway to centralize tasks like routing, authentication, rate limiting, caching, and monitoring. Tools like Kong, Traefik can effectively manage API traffic. The gateway can cache responses at the edge, allowing faster response times by serving data from the cache, reducing load on backend services.

  • Rate Limiting: Set limits on API requests to prevent abuse and reduce excessive database loads. Implement IP-based rate limiting or user-based quotas.

  • Backpressure Mechanism: For high-traffic APIs, implement backpressure strategies (like retry with exponential backoff) to prevent overloading your database.

Monitoring and Tuning

  • Database Monitoring: Use monitoring tools (e.g., Prometheus, Grafana, or APM tools like Datadog) to track query performance, slow queries, CPU, memory usage, and I/O operations.

  • Load Testing: Perform load and stress testing to identify bottlenecks and tune settings before the system goes live. Tools like Apache JMeter, Tsung and k6 can be helpful.

  • Tracing Tools: Use tools like OpenTelemetry (Otel) for API tracing to identify bottlenecks and performance issues in your APIs. Tracing helps you monitor and analyze the flow of requests across services, pinpointing delays or errors caused by downstream or upstream dependencies, such as network latency, service timeouts, or database query slowness. By integrating tracing into your system, you can gain valuable insights into your API’s behavior and resolve issues efficiently.

By following these steps, you can effectively manage large read and write operations in your REST API with an RDBMS, ensuring that it scales to meet demand while maintaining performance.

Engineering is all about tradeoffs, so before optimizing or scaling, it’s essential to assess your needs and the specific situation.