Apache Flink
jvm · stream-processor · Apache-2.0 · active · java
Homepage · Source · Its configuration in this repository
Has this configuration been reviewed upstream?
Not yet. This configuration has not been checked by the people who maintain this system, and until it has, treat its numbers as ours rather than theirs.
What it was asked to guarantee
at-least-once · checkpoint every 5s
The envelope it ran in
32 CPU · 96g memory
jm— control-plane · 1 CPU · 2gtm— data-plane · 32 CPU · 96g
Its arms
- 2.2.1 · RowBinary
rowbinary-nt· realistic · rowbinary_nt · defaultparallelism 32 · slots 32 · process_mib 21504 · network_memory_max 9223372036854775807b · runtime_image flink:2.2.1-java17 · jvm_opts · avro_mode specific · linger_ms 1000 · max_rows 12500 · buffered_rows 25000 · inflight 1 · max_batch_bytes 16777216 · poll_records 500 · partition_fetch_bytes 1048576 · fetch_bytes 52428800 · connections 10 · network_buffer_bytes 300000
Where it has been measured
c8gd-metal-24xl-ec2-docker · drain
| Arm | Place | Throughput per core | Throughput | Cores used | Measured |
|---|---|---|---|---|---|
| 2.2.1 · RowBinary | 3 of 6 | 547k range 4.3% | 5.74M range 2.8% | 10.41 range 3.8% | 2026-09-11 · 3 reps |
Configurations it refuses
buffered_rowsmust exceedmax_rows. Flink's AsyncSinkWriter requires maxBufferedRequests > maxBatchSize and refuses to construct otherwise, so the job fails at submission rather than running slowly. Raise buffered_rows with max_rows: it is also the bound on checkpoint state, since the sink checkpoints its buffered entries, and on retained payload memory across every subtask.
Declared deviations
- Runs as two containers: the 32 CPU / 96 GiB data plane, plus a JobManager control-plane container (1 CPU / 2 GiB) allocated on top of the envelope. — Flink cannot run a standalone job without a JobManager. Its measured consumption (0.07–0.21 cores) is published beside the arm's totals rather than silently pre-charged against the data plane's budget. · affects envelope
Tell us we got this wrong
If this system is configured badly here, that is a bug in this benchmark rather than a result about Apache Flink, and the pull request that fixes it is the most valuable one this repository can receive. How to send one. The whole configuration is at entrants/flink.
This system also reports CPU per row, Cores, data plane, Peak memory, data plane, Peak charged memory, Throttled, ClickHouse CPU per row, GC pause p99, ClickHouse CPU, ClickHouse CPU per written row, ClickHouse CPU wait, Bytes inserted, Rows per insert, Rows written server-side, GC pause max, GC pause p99.9, GC pause total, JVM heap committed peak, JVM heap configured, JVM heap live peak. Those figures are on each arm’s own disclosure in the results table.