Multi-Game Scalability: The Path to 1000 Concurrent Users

To achieve our goal of 1000 concurrent players in the game engine, we must transition from a single-instance architecture to a distributed, horizontally scalable system.

1. Architectural Shift: From Synchronized to Distributed

The current engine relies on Java-level synchronization and embedded persistence. In a multi-node setup, this causes state fragmentation.

Key Changes:

• Centralized Persistence: Moving to a shared PostgreSQL or H2 Server instance.
• Optimistic Concurrency: Implementing versioned updates in the Game domain object to prevent race conditions during concurrent answer submissions.
• Event Orchestration: Introducing Redis as an event bus to synchronize SSE updates across all nodes.

2. Distributed SSE Event Flow

Players connected to Node A must see events triggered by players on Node B.

1. Trigger: An action occurs (e.g., participate() on Node A).
2. Publish: Node A publishes a PLAYER_JOINED event to the Redis channel game:<gameCode>.
3. Broadcast: All nodes (A, B, C...) subscribed to that channel receive the event.
4. Push: Each node pushes the event to its locally connected SSE clients.

3. Load Balancing & Reliability

• Stickiness: Use Nginx ip_hash to keep game sessions localized where possible.
• Re-hydration: Implement state recovery logic to allow the SMC state machine to resume from the database after a node restart.

4. Measuring Success

We will track our progress toward the 1000-user target using OTel-exported metrics in OpenObserve:
- active_games_total
- players_per_game_avg
- event_propagation_latency_ms

This design ensures that technical integrity remains high as we scale the product to its target audience.