Graduated Batch-Size Backpressure

ponyc runtime · PR #6285 · October 2026

Pony's current backpressure mechanism uses binary muting: when an actor sends to an overloaded receiver, the sender is immediately removed from scheduling. Under fan-in topologies, this cascades — muting propagates backwards through the actor graph, freezing the entire system for seconds at a time.

This report documents a graduated alternative: per-actor batch sizes that adjust dynamically under backpressure. Overloaded actors process more messages per run; senders to overloaded receivers process fewer. The mechanism eliminates the fan-in collapse and triples sustained-overload throughput.

1Results

Sustained overload

3.4×

22M msg/s vs 6.5M baseline

Fan-in collapse

Eliminated

40M msg/s vs 130 msg/s

Code change

+38 lines

3 files, 0 bytes added to actor struct

Complete benchmark results

All benchmarks run on a 16-core / 32 GB Linux box with --ponymaxthreads 16. Baseline is ponyc 0.75.0.

Benchmark Configuration Baseline Graduated
Sustained overload 10K senders, 1 receiver ~6.5M msg/s ~22M msg/s
Fan-in 100s → 2K analyzers, 10ms workload 39M → 119 msg/s 40M msg/s stable
Fan-in 100s → 2K analyzers, 5ms workload 39M ↔ 99 msg/s 41M msg/s stable
Fan-in 100s → 2K analyzers, 1ms workload 38M msg/s stable 42M msg/s stable
Fan-in (high load) 500s → 2K analyzers, 10ms workload 40M → 181 msg/s 44M msg/s stable
Fan-in (low fan-in) 100s → 200 analyzers, 10ms workload — 51M msg/s stable

The fan-in collapse — throughput dropping from 39M to <200 msg/s with a 20-second freeze — is completely eliminated across all configurations tested. The 1ms workload case, where the baseline was already stable, shows a ~10% throughput improvement from the receiver's batch growth.

2The mechanism

Two new fields on pony_actor_t: a batch_size (uint16_t, starts at PONY_SCHED_BATCH, default 100) and a sent_to_overloaded flag (bool) cleared each run. Both pack into existing alignment padding — sizeof(pony_actor_t) is unchanged. Two compile-time constants: PONY_SCHED_BATCH (default batch, 100) and PONY_SCHED_BATCH_STEP (adjustment step, 25).

full batch processed, queue not empty → mark OVERLOADED, batch += step
send to overloaded/muted receiver → batch −= step (mute if batch hits 0)
sender is itself overloaded → batch −= step, but floor at step (never muted)
run ends, didn't send to overloaded → batch += step (up to default, recovery)
actor unmuted → batch = step (ramp up gradually)
overload cleared (queue drained) → batch stays where it is (no reset)

Key properties

3Why the current system collapses

The fan-in benchmark: 100 senders → 2000 analyzers → 1 receiver.

100 Senders → 2000 Analyzers → 1 Receiver (10ms/msg)

Every 10 seconds, all 2000 analyzers send one message to the receiver. The receiver takes 10ms per message, so draining 2000 messages takes 20 seconds — longer than the 10-second tick interval. The receiver stays permanently overloaded.

With binary muting, every analyzer that sends to the overloaded receiver is immediately and fully muted. Muted analyzers stop processing messages from senders. Senders see their target analyzers as muted and are muted themselves. Within one tick, most of the system is frozen.

This is a limit cycle: the system oscillates between full throughput (39M msg/s) and near-total freeze (~130 msg/s). The feedback gain is greater than 1 — each muted actor causes further muting — producing an unstable oscillation.

The 5ms workload case is the boundary: drain time (10s) equals the tick interval, so the system alternates between stable and collapsed intervals. The 1ms case (drain time 2s) stays stable because the receiver always catches up.

4How graduated batch breaks the cascade

When 2000 analyzers each send one message to the overloaded receiver, each analyzer's batch drops by 25 (100 → 75). All 2000 analyzers are still scheduled at batch 75. They continue processing messages from senders. No muting cascades backward.

Meanwhile, the receiver's batch grows: 100 → 125 → 150 → … It processes more messages per scheduling run, increasing its effective service rate. The receiver's batch stabilizes when its throughput matches the arrival rate.

On the next tick, if the receiver is no longer overloaded, each analyzer recovers: 75 → 100 in one clean run. If the receiver is still overloaded, analyzers drop to 50, then 25 — but they never reach 0 because each analyzer sends to the receiver only once per tick (one step decrement per tick).

5Queuing theory analysis

The mechanism replaces a bang-bang controller (binary mute/unmute) with a proportional controller (graduated throttling). In queuing theory terms:

Model

Each actor is a single-server queue. Messages arrive at rate λ. The actor processes up to batch_size messages per scheduling run, giving effective service rate μ(b). An actor is overloaded when λ > μ(b) for the current interval.

Stability

The system converges because the receiver's batch growth is monotonic during overload and persistent after overload clears. This creates a ratchet: the batch stabilizes at whatever value lets the receiver keep up with arrivals. Since batch growth is unbounded, convergence is guaranteed for any finite per-message processing time.

Why unbounded batch growth is safe

From the Pollaczek-Khinchine formula, as utilization ρ approaches 1, mean queue length diverges. The only remedies are: decrease λ (throttle senders) or increase μ (process faster). The graduated mechanism does both. Capping the batch would cap μ, potentially leaving ρ ≥ 1 for large enough bursts — guaranteeing unbounded queue growth.

In practice, a very large batch means an actor monopolizes its scheduler thread longer. This is acceptable: the actor genuinely has that many messages waiting, other scheduler threads serve other actors, and the actor yields after its batch regardless.

Comparison with TCP congestion control

Property TCP (AIMD) Graduated batch Binary muting
Decrease Multiplicative (halve) Additive (−25) Total (→ 0)
Increase Additive (+1/RTT) Additive (+25/run) Binary (0 → 100)
Convergence Proven stable Stable (monotonic ratchet) Limit cycle
Bottleneck adaptation No (link capacity fixed) Yes (batch grows) No

The mechanism's most distinctive property is bottleneck adaptation. TCP has no analogue because network link capacity is physical. In the actor model, an actor's "capacity" is its batch size — a scheduling parameter, not a physical constraint. This extra degree of freedom makes the system more adaptive than network congestion control.

6Potential concerns

Scheduler fairness

An actor with a batch of 10,000 monopolizes its scheduler thread for 10,000 messages. With P scheduler threads and one large-batch actor, P−1 threads are unaffected. Large batches result from sustained overload, which is typically concentrated in a small number of receivers.

Step size sensitivity

The ratio B₀/Δ = 100/25 = 4 means an actor gets 4 "strikes" before muting. This is aggressive enough to respond quickly to congestion, but gradual enough to avoid cascading. Both values are compile-time constants (PONY_SCHED_BATCH and PONY_SCHED_BATCH_STEP).

Memory growth

Under sustained overload where arrival rate permanently exceeds service rate, queues grow regardless of mechanism. The graduated approach doesn't make this worse — the receiver processes its queue faster (larger batch), so memory growth is actually slower than baseline for the same arrival rate.

Oscillation

The system could oscillate if senders recover and resume full-rate sending, re-triggering overload. This is damped because the receiver's elevated batch persists after overload clears — it can handle a higher arrival rate before becoming overloaded again. Each cycle leaves the receiver with a higher batch, so the system converges.

7Implementation

Three files changed (PR #6285):

actor.h

#define PONY_SCHED_BATCH 100
#define PONY_SCHED_BATCH_STEP 25

typedef struct pony_actor_t
{
  ...
  uint8_t internal_flags;
  bool sent_to_overloaded;
  uint16_t batch_size;
  uint32_t live_asio_events;
  ...
}

actor.c — maybe_mark_should_mute()

if(triggers_muting(to) && ctx->current != to)
{
  ctx->current->sent_to_overloaded = true;
  if(has_sync_flag_any(ctx->current,
    ACTOR_SYNC_FLAG_OVERLOADED |
    ACTOR_SYNC_FLAG_UNDER_PRESSURE))
  {
    if(ctx->current->batch_size > PONY_SCHED_BATCH_STEP)
      ctx->current->batch_size -= PONY_SCHED_BATCH_STEP;
  }
  else if(ctx->current->batch_size <= PONY_SCHED_BATCH_STEP)
  {
    ctx->current->batch_size = 0;
    ponyint_sched_mute(ctx, ctx->current, to);
  }
  else
  {
    ctx->current->batch_size -= PONY_SCHED_BATCH_STEP;
  }
}

actor.c — batch_limit_reached()

actor->batch_size += PONY_SCHED_BATCH_STEP;

actor.c — recovery (end of ponyint_actor_run)

if(!actor->sent_to_overloaded
  && actor->batch_size < PONY_SCHED_BATCH)
{
  actor->batch_size += PONY_SCHED_BATCH_STEP;
  if(actor->batch_size > PONY_SCHED_BATCH)
    actor->batch_size = PONY_SCHED_BATCH;
}

scheduler.c — unmute recovery

actor->batch_size = PONY_SCHED_BATCH_STEP;