Building Scalable Laravel Applications

Building Scalable Laravel Applications: Queue Workers, Caching, and Database Optimization

Scaling Laravel applications requires careful attention to queues, caching strategies, and database optimization. Here's how I've built high-performance Laravel applications that handle thousands of requests per second.

🎯 Scaling Challenges

As applications grow, you face:

Slow response times from heavy operations
Database bottlenecks from N+1 queries
Memory issues from processing large datasets
Timeout errors from long-running tasks

The solution: Queues, caching, and database optimization.

🔄 Queue Workers

Why Queues?

Move time-consuming tasks to background processing:

Email sending
Image processing
Report generation
API calls to external services

Setup Queue System

```php // config/queue.php 'connections' => [ 'database' => [ 'driver' => 'database', 'table' => 'jobs', 'queue' => 'default', 'retry_after' => 90, ],

'redis' => [ 'driver' => 'redis', 'connection' => 'default', 'queue' => env('REDIS_QUEUE', 'default'), 'retry_after' => 90, 'block_for' => null, ], ], ```

Create Jobs

```php // app/Jobs/SendCampaignEmail.php class SendCampaignEmail implements ShouldQueue { use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

public $subscriber; public $campaign;

public function __construct($subscriber, $campaign) { $this->subscriber = $subscriber; $this->campaign = $campaign; }

public function handle() { Mail::to($this->subscriber->email) ->send(new CampaignEmail($this->campaign, $this->subscriber)); } }

// Dispatch job SendCampaignEmail::dispatch($subscriber, $campaign); ```

Run Queue Workers

```bash

Single worker

php artisan queue:work

Multiple workers

php artisan queue:work --tries=3 --timeout=60

Specific queue

php artisan queue:work --queue=emails,high-priority

With Supervisor (production)

```

Supervisor Configuration

```ini

/etc/supervisor/conf.d/laravel-worker.conf

[program:laravel-worker] process_name=%(program_name)s_%(process_num)02d command=php /var/www/app/artisan queue:work redis --sleep=3 --tries=3 --max-time=3600 autostart=true autorestart=true stopasgroup=true killasgroup=true user=www-data numprocs=4 redirect_stderr=true stdout_logfile=/var/www/app/storage/logs/worker.log stopwaitsecs=3600 ```

💾 Caching Strategies

1. Query Result Caching

Cache expensive database queries:

```php $posts = Cache::remember('featured_posts', 3600, function () { return Post::featured() ->with(['author', 'category', 'tags']) ->get(); }); ```

2. Model Caching

Cache entire models:

```php // In model protected static function booted() { static::saved(function ($post) { Cache::forget("post.{$post->id}"); }); }

// Usage $post = Cache::remember("post.{$id}", 3600, function () use ($id) { return Post::with('category')->findOrFail($id); }); ```

3. View Caching

Cache rendered views:

```php // In controller return Cache::remember("view.posts.index", 3600, function () { return view('posts.index', [ 'posts' => Post::paginate(15) ])->render(); }); ```

4. Redis Caching

Use Redis for faster caching:

```php // config/cache.php 'redis' => [ 'driver' => 'redis', 'connection' => 'cache', ],

// Usage Cache::store('redis')->put('key', 'value', 3600); ```

5. Cache Tags

Group related cache entries:

```php Cache::tags(['posts', 'featured'])->put('key', $value, 3600); Cache::tags(['posts'])->flush(); // Clear all post-related cache ```

🗄️ Database Optimization

1. Eager Loading

Always eager load relationships:

```php // Bad: N+1 queries $posts = Post::all(); foreach ($posts as $post) { echo $post->author->name; // Query per post! }

// Good: Single query $posts = Post::with(['author', 'category', 'tags'])->get(); ```

2. Lazy Eager Loading

Load relationships when needed:

```php $posts = Post::all(); $posts->load('author', 'category'); ```

3. Database Indexing

Add indexes for frequently queried columns:

```php // Migration Schema::table('posts', function (Blueprint $table) { $table->index('status'); $table->index('published_at'); $table->index(['status', 'published_at']); // Composite $table->index('slug'); // Unique lookups }); ```

4. Query Optimization

Use select() to limit columns:

```php // Only fetch needed columns $posts = Post::select('id', 'title', 'slug') ->where('status', 'published') ->get(); ```

5. Chunking Large Datasets

Process in batches:

```php Post::chunk(100, function ($posts) { foreach ($posts as $post) { // Process post } });

// Or cursor (memory efficient) foreach (Post::cursor() as $post) { // Process post } ```

6. Database Connection Pooling

Configure connection limits:

```php // config/database.php 'mysql' => [ 'driver' => 'mysql', 'host' => env('DB_HOST', '127.0.0.1'), 'port' => env('DB_PORT', '3306'), 'database' => env('DB_DATABASE', 'forge'), 'username' => env('DB_USERNAME', 'forge'), 'password' => env('DB_PASSWORD', ''), 'options' => [ PDO::ATTR_PERSISTENT => true, // Connection pooling ], ], ```

⚡ Performance Monitoring

1. Query Logging

Enable query logging in development:

```php DB::enableQueryLog(); // Your code $queries = DB::getQueryLog(); dd($queries); ```

2. Laravel Debugbar

Install Laravel Debugbar for performance insights:

```bash composer require barryvdh/laravel-debugbar --dev ```

3. APM Tools

Use tools like:

Laravel Telescope (built-in)
New Relic
Datadog

🚀 Advanced Techniques

1. Database Read/Write Splitting

```php // config/database.php 'mysql' => [ 'read' => [ 'host' => ['192.168.1.1', '192.168.1.2'], ], 'write' => [ 'host' => ['192.168.1.3'], ], ], ```

2. Query Caching Middleware

```php class CacheQueries { public function handle($request, Closure $next) { if ($request->has('cache')) { DB::enableQueryLog(); }

$response = $next($request);

if ($request->has('cache')) { Cache::put('queries', DB::getQueryLog(), 3600); }

return $response; } } ```

3. Background Job Batching

```php use Illuminate\Bus\Batch;

$batch = Bus::batch([ new SendCampaignEmail($subscriber1, $campaign), new SendCampaignEmail($subscriber2, $campaign), // ... more jobs ])->then(function (Batch $batch) { // All jobs completed })->catch(function (Batch $batch, Throwable $e) { // First batch job failure })->finally(function (Batch $batch) { // Batch finished })->dispatch(); ```

💡 Real-World Example

I optimized a Laravel application handling 10,000+ requests/day:

1. Moved email sending to queues → 90% faster response 2. Cached expensive queries → 80% reduction in DB load 3. Added database indexes → 95% faster queries 4. Eager loaded relationships → Eliminated N+1 queries 5. Used Redis caching → 10x faster cache retrieval

Result: Response time reduced from 2s to 200ms, database load reduced by 85%.

🎓 Key Takeaways

Use queues for time-consuming tasks
Cache aggressively but invalidate properly
Eager load relationships to avoid N+1
Add indexes for frequently queried columns
Monitor performance continuously
Optimize queries before scaling infrastructure

Conclusion

Scaling Laravel applications is about moving work off the request cycle (queues), reducing database load (caching), and optimizing queries. These techniques have helped me build applications that handle high traffic efficiently.