Driver location ingestion

Every online driver's app sends its position every few seconds. Those updates are the highest-volume traffic in a ride-sharing system. The ingestion path has to take them in, keep the newest position for each driver, and drop drivers who stop reporting. It does not need a history for matching. Only the latest position matters.

The big idea

The pipeline has three stages. Gateways accept the updates over a long-lived connection. A stream carries the updates, partitioned by driver or by geographic cell. A geo index service consumes the stream and keeps the latest state in memory. The matching service reads from that index, not from the database.

The index is fast and disposable. If it is lost, it can be rebuilt from the stream and driver reconnects. The trip database remains the source of truth.

Walking through it

The write rate. Take a region with about 1 million drivers, each reporting every 4 seconds:

1,000,000 drivers ÷ 4 seconds = 250,000 updates per second

Writing each update as a row in a relational database means 250,000 row updates per second, on the path that matching depends on. The design avoids that by keeping the latest state in memory. The durable record of a trip is written separately.

Two sets of drivers. Only available drivers belong in the matchable index. A driver is available when online, not on a trip, and matched to the right product and vehicle. On-trip drivers still send their location, so ETAs stay accurate. But they live in a separate set, so matching never offers them a trip.

Stale drivers. A driver who has not reported for a set time, such as 30 seconds, is removed from the available set. A tunnel or a dead phone should not leave a ghost driver in the index. Matching also skips any candidate whose timestamp is too old.

History. Breadcrumbs for analytics can flow into the same stream and then into cold storage. Matching does not read them.

Common mistakes

• Writing every ping to a row-based database. The write rate will outrun it.
• Keeping on-trip drivers in the matchable set. They get offered trips they cannot take.
• Matching on stale positions. Check each candidate's timestamp.
• Treating the location index as billing truth. It is fast and rebuildable, not a record of trips.

Check yourself

1. A region has 1 million drivers, each reporting every 4 seconds. How many updates per second arrive?

   Answer: 250,000 (1,000,000 ÷ 4).

2. Why should on-trip drivers live in a different set from available drivers?

   Answer: matching reads only the available set, so an on-trip driver is never offered a new trip. The location of on-trip drivers still feeds ETAs.

Key takeaways

• Location updates are the highest-volume traffic, so they flow through a stream into an in-memory index.
• The index keeps only the latest position, and it can be rebuilt if lost.
• Keep only available drivers in the matchable set, and remove drivers who go quiet.
• The trip database, not the location index, is the source of truth.

Further learning

Next, see how one driver is handed to exactly one rider, in exclusive driver assignment.